* [RFC PATCH v4 17/26] http-backend: reword some documentation
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git; +Cc: Mark Lodato
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
From: Mark Lodato <lodatom@gmail.com>
Clarify some of the git-http-backend documentation, particularly:
* In the Description, state that smart/dumb HTTP fetch and smart HTTP
push are supported, state that authenticated clients allow push, and
remove the note that this is only suited for read-only updates.
* At the start of Examples, state explicitly what URL is mapping to what
location on disk.
Signed-off-by: Mark Lodato <lodatom@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
Documentation/git-http-backend.txt | 12 ++++++++----
1 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-http-backend.txt b/Documentation/git-http-backend.txt
index 99dbbfb..0b5e951 100644
--- a/Documentation/git-http-backend.txt
+++ b/Documentation/git-http-backend.txt
@@ -14,13 +14,15 @@ DESCRIPTION
-----------
A simple CGI program to serve the contents of a Git repository to Git
clients accessing the repository over http:// and https:// protocols.
+The program supports clients fetching using both the smart HTTP protcol
+and the backwards-compatible dumb HTTP protocol, as well as clients
+pushing using the smart HTTP protocol.
By default, only the `upload-pack` service is enabled, which serves
'git-fetch-pack' and 'git-ls-remote' clients, which are invoked from
-'git-fetch', 'git-pull', and 'git-clone'.
-
-This is ideally suited for read-only updates, i.e., pulling from
-git repositories.
+'git-fetch', 'git-pull', and 'git-clone'. If the client is authenticated,
+the `receive-pack` service is enabled, which serves 'git-send-pack'
+clients, which is invoked from 'git-push'.
SERVICES
--------
@@ -50,6 +52,8 @@ automatically by the web server.
EXAMPLES
--------
+All of the following examples map 'http://$hostname/git/foo/bar.git'
+to '/var/www/git/foo/bar.git'.
Apache 2.x::
Ensure mod_cgi, mod_alias, and mod_env are enabled, set
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 26/26] test smart http fetch and push
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
The top level directory "/git/" of the test Apache server is mapped
through our git-http-backend CGI, but uses the same underlying
repository space as the server's document root. This is the most
simple installation possible.
Server logs are checked to verify the client has accessed only the
smart URLs during the test. During fetch testing the headers are
also logged from libcurl to ensure we are making a reasonably sane
HTTP request, and getting back reasonably sane response headers
from the CGI.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
t/lib-httpd/apache.conf | 20 +++++++++
t/t5540-http-push.sh | 2 +-
t/t5541-http-push.sh | 103 +++++++++++++++++++++++++++++++++++++++++++++++
t/t5550-http-fetch.sh | 8 +++-
t/t5551-http-fetch.sh | 87 +++++++++++++++++++++++++++++++++++++++
5 files changed, 218 insertions(+), 2 deletions(-)
create mode 100755 t/t5541-http-push.sh
create mode 100755 t/t5551-http-fetch.sh
diff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf
index 21aa42f..2098ce0 100644
--- a/t/lib-httpd/apache.conf
+++ b/t/lib-httpd/apache.conf
@@ -8,6 +8,26 @@ ErrorLog error.log
<IfModule !mod_log_config.c>
LoadModule log_config_module modules/mod_log_config.so
</IfModule>
+<IfModule !mod_cgi.c>
+ LoadModule cgi_module modules/mod_cgi.so
+</IfModule>
+<IfModule !mod_alias.c>
+ LoadModule alias_module modules/mod_alias.so
+</IfModule>
+<IfModule !mod_env.c>
+ LoadModule env_module modules/mod_env.so
+</IfModule>
+
+<Location /git/>
+ SetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}
+</Location>
+ScriptAlias /git/ ${GIT_EXEC_PATH}/git-http-backend/
+<Directory ${GIT_EXEC_PATH}>
+ Options None
+</Directory>
+<Files ${GIT_EXEC_PATH}/git-http-backend>
+ Options ExecCGI
+</Files>
<IfDefine SSL>
LoadModule ssl_module modules/mod_ssl.so
diff --git a/t/t5540-http-push.sh b/t/t5540-http-push.sh
index 28a746e..5c0f4d7 100755
--- a/t/t5540-http-push.sh
+++ b/t/t5540-http-push.sh
@@ -3,7 +3,7 @@
# Copyright (c) 2008 Clemens Buchacher <drizzd@aon.at>
#
-test_description='test http-push
+test_description='test WebDAV http-push
This test runs various sanity checks on http-push.'
diff --git a/t/t5541-http-push.sh b/t/t5541-http-push.sh
new file mode 100755
index 0000000..690c466
--- /dev/null
+++ b/t/t5541-http-push.sh
@@ -0,0 +1,103 @@
+#!/bin/sh
+#
+# Copyright (c) 2008 Clemens Buchacher <drizzd@aon.at>
+#
+
+test_description='test smart pushing over http via http-backend'
+. ./test-lib.sh
+
+if test -n "$NO_CURL"; then
+ say 'skipping test, git built without http support'
+ test_done
+fi
+
+ROOT_PATH="$PWD"
+LIB_HTTPD_PORT=${LIB_HTTPD_PORT-'5550'}
+. "$TEST_DIRECTORY"/lib-httpd.sh
+start_httpd
+
+test_expect_success 'setup remote repository' '
+ cd "$ROOT_PATH" &&
+ mkdir test_repo &&
+ cd test_repo &&
+ git init &&
+ : >path1 &&
+ git add path1 &&
+ test_tick &&
+ git commit -m initial &&
+ cd - &&
+ git clone --bare test_repo test_repo.git &&
+ cd test_repo.git &&
+ git config http.receivepack true &&
+ ORIG_HEAD=$(git rev-parse --verify HEAD) &&
+ cd - &&
+ mv test_repo.git "$HTTPD_DOCUMENT_ROOT_PATH"
+'
+
+test_expect_success 'clone remote repository' '
+ cd "$ROOT_PATH" &&
+ git clone $HTTPD_URL/git/test_repo.git test_repo_clone
+'
+
+test_expect_success 'push to remote repository with packed refs' '
+ cd "$ROOT_PATH"/test_repo_clone &&
+ : >path2 &&
+ git add path2 &&
+ test_tick &&
+ git commit -m path2 &&
+ HEAD=$(git rev-parse --verify HEAD) &&
+ git push &&
+ (cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git &&
+ test $HEAD = $(git rev-parse --verify HEAD))
+'
+
+test_expect_success 'push already up-to-date' '
+ git push
+'
+
+test_expect_success 'push to remote repository with unpacked refs' '
+ (cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git &&
+ rm packed-refs &&
+ git update-ref refs/heads/master $ORIG_HEAD) &&
+ git push &&
+ (cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git &&
+ test $HEAD = $(git rev-parse --verify HEAD))
+'
+
+test_expect_success 'create and delete remote branch' '
+ cd "$ROOT_PATH"/test_repo_clone &&
+ git checkout -b dev &&
+ : >path3 &&
+ git add path3 &&
+ test_tick &&
+ git commit -m dev &&
+ git push origin dev &&
+ git push origin :dev &&
+ test_must_fail git show-ref --verify refs/remotes/origin/dev
+'
+
+cat >exp <<EOF
+GET /git/test_repo.git/info/refs?service=git-upload-pack HTTP/1.1 200
+POST /git/test_repo.git/git-upload-pack HTTP/1.1 200
+GET /git/test_repo.git/info/refs?service=git-receive-pack HTTP/1.1 200
+POST /git/test_repo.git/git-receive-pack HTTP/1.1 200
+GET /git/test_repo.git/info/refs?service=git-receive-pack HTTP/1.1 200
+GET /git/test_repo.git/info/refs?service=git-receive-pack HTTP/1.1 200
+POST /git/test_repo.git/git-receive-pack HTTP/1.1 200
+GET /git/test_repo.git/info/refs?service=git-receive-pack HTTP/1.1 200
+POST /git/test_repo.git/git-receive-pack HTTP/1.1 200
+GET /git/test_repo.git/info/refs?service=git-receive-pack HTTP/1.1 200
+POST /git/test_repo.git/git-receive-pack HTTP/1.1 200
+EOF
+test_expect_success 'used receive-pack service' '
+ sed -e "
+ s/^.* \"//
+ s/\"//
+ s/ [1-9][0-9]*\$//
+ s/^GET /GET /
+ " >act <"$HTTPD_ROOT_PATH"/access.log &&
+ test_cmp exp act
+'
+
+stop_httpd
+test_done
diff --git a/t/t5550-http-fetch.sh b/t/t5550-http-fetch.sh
index 0e69324..8d6443f 100755
--- a/t/t5550-http-fetch.sh
+++ b/t/t5550-http-fetch.sh
@@ -1,6 +1,6 @@
#!/bin/sh
-test_description='test fetching over http'
+test_description='test dumb fetching over http via static file'
. ./test-lib.sh
if test -n "$NO_CURL"; then
@@ -61,5 +61,11 @@ test_expect_success 'fetch packed objects' '
git clone $HTTPD_URL/repo_pack.git
'
+test_expect_success 'did not use upload-pack service' '
+ grep '/git-upload-pack' <"$HTTPD_ROOT_PATH"/access.log >act
+ : >exp
+ test_cmp exp act
+'
+
stop_httpd
test_done
diff --git a/t/t5551-http-fetch.sh b/t/t5551-http-fetch.sh
new file mode 100755
index 0000000..afddc2c
--- /dev/null
+++ b/t/t5551-http-fetch.sh
@@ -0,0 +1,87 @@
+#!/bin/sh
+
+test_description='test smart fetching over http via http-backend'
+. ./test-lib.sh
+
+if test -n "$NO_CURL"; then
+ say 'skipping test, git built without http support'
+ test_done
+fi
+
+LIB_HTTPD_PORT=${LIB_HTTPD_PORT-'5550'}
+. "$TEST_DIRECTORY"/lib-httpd.sh
+start_httpd
+
+test_expect_success 'setup repository' '
+ echo content >file &&
+ git add file &&
+ git commit -m one
+'
+
+test_expect_success 'create http-accessible bare repository' '
+ mkdir "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ (cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ git --bare init
+ ) &&
+ git remote add public "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+ git push public master:master
+'
+
+cat >exp <<EOF
+> GET /git/repo.git/info/refs?service=git-upload-pack HTTP/1.1
+Accept: */*
+Pragma: no-cache
+< HTTP/1.1 200 OK
+< Pragma: no-cache
+< Cache-Control: no-cache, max-age=0, must-revalidate
+< Content-Type: application/x-git-upload-pack-advertisement
+<
+> POST /git/repo.git/git-upload-pack HTTP/1.1
+Accept-Encoding: deflate, gzip
+Content-Type: application/x-git-upload-pack-request
+Accept: application/x-git-upload-pack-response
+Content-Length: xxxx
+< HTTP/1.1 200 OK
+< Pragma: no-cache
+< Cache-Control: no-cache, max-age=0, must-revalidate
+< Content-Type: application/x-git-upload-pack-result
+<
+EOF
+test_expect_success 'clone http repository' '
+ GIT_CURL_VERBOSE=1 git clone $HTTPD_URL/git/repo.git clone 2>err &&
+ test_cmp file clone/file &&
+ egrep "^([<>]|Pragma|Accept|Content-|Transfer-)" err |
+ egrep -v "^< (Server|Expires|Date|Content-Length:|Transfer-Encoding: chunked)" |
+ sed -e "
+ s/
//
+ s/^Content-Length: .*$/Content-Length: xxxx/
+ " >act &&
+ test_cmp exp act
+'
+
+test_expect_success 'fetch changes via http' '
+ echo content >>file &&
+ git commit -a -m two &&
+ git push public
+ (cd clone && git pull) &&
+ test_cmp file clone/file
+'
+
+cat >exp <<EOF
+GET /git/repo.git/info/refs?service=git-upload-pack HTTP/1.1 200
+POST /git/repo.git/git-upload-pack HTTP/1.1 200
+GET /git/repo.git/info/refs?service=git-upload-pack HTTP/1.1 200
+POST /git/repo.git/git-upload-pack HTTP/1.1 200
+EOF
+test_expect_success 'used upload-pack service' '
+ sed -e "
+ s/^.* \"//
+ s/\"//
+ s/ [1-9][0-9]*\$//
+ s/^GET /GET /
+ " >act <"$HTTPD_ROOT_PATH"/access.log &&
+ test_cmp exp act
+'
+
+stop_httpd
+test_done
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 22/26] Smart push over HTTP: client side
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git; +Cc: Daniel Barkalow
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
The git-remote-curl backend detects if the remote server supports
the git-receive-pack service, and if so, runs git-send-pack in a
pipe to dump the command and pack data as a single POST request.
The advertisements from the server that were obtained during the
discovery are passed into git-send-pack before the POST request
starts. This permits git-send-pack to operate largely unmodified.
For smaller packs (those under 1 MiB) a HTTP/1.0 POST with a
Content-Length is used, permitting interaction with any server.
The 1 MiB limit is arbitrary, but is sufficent to fit most deltas
created by human authors against text sources with the occasional
small binary file (e.g. few KiB icon image). The configuration
option http.postBuffer can be used to increase (or shink) this
buffer if the default is not sufficient.
For larger packs which cannot be spooled entirely into the helper's
memory space (due to http.postBuffer being too small), the POST
request requires HTTP/1.1 and sets "Transfer-Encoding: chunked".
This permits the client to upload an unknown amount of data in one
HTTP transaction without needing to pregenerate the entire pack
file locally.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Daniel Barkalow <barkalow@iabervon.org>
---
Documentation/config.txt | 8 ++
builtin-send-pack.c | 116 +++++++++++++++++++++-
http.c | 13 ++-
http.h | 2 +
remote-curl.c | 235 +++++++++++++++++++++++++++++++++++++++++++++-
send-pack.h | 3 +-
sideband.c | 11 ++-
transport.c | 1 +
8 files changed, 374 insertions(+), 15 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index cd17814..7130d07 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1089,6 +1089,14 @@ http.maxRequests::
How many HTTP requests to launch in parallel. Can be overridden
by the 'GIT_HTTP_MAX_REQUESTS' environment variable. Default is 5.
+http.postBuffer::
+ Maximum size in bytes of the buffer used by smart HTTP
+ transports when POSTing data to the remote system.
+ For requests larger than this buffer size, HTTP/1.1 and
+ Transfer-Encoding: chunked is used to avoid creating a
+ massive pack file locally. Default is 1 MiB, which is
+ sufficient for most requests.
+
http.lowSpeedLimit, http.lowSpeedTime::
If the HTTP transfer speed is less than 'http.lowSpeedLimit'
for longer than 'http.lowSpeedTime' seconds, the transfer is aborted.
diff --git a/builtin-send-pack.c b/builtin-send-pack.c
index 37e528e..a0fbad0 100644
--- a/builtin-send-pack.c
+++ b/builtin-send-pack.c
@@ -2,9 +2,11 @@
#include "commit.h"
#include "refs.h"
#include "pkt-line.h"
+#include "sideband.h"
#include "run-command.h"
#include "remote.h"
#include "send-pack.h"
+#include "quote.h"
static const char send_pack_usage[] =
"git send-pack [--all | --mirror] [--dry-run] [--force] [--receive-pack=<git-receive-pack>] [--verbose] [--thin] [<host>:]<directory> [<ref>...]\n"
@@ -59,7 +61,7 @@ static int pack_objects(int fd, struct ref *refs, struct extra_have_objects *ext
memset(&po, 0, sizeof(po));
po.argv = argv;
po.in = -1;
- po.out = fd;
+ po.out = args->stateless_rpc ? -1 : fd;
po.git_cmd = 1;
if (start_command(&po))
die_errno("git pack-objects failed");
@@ -83,6 +85,20 @@ static int pack_objects(int fd, struct ref *refs, struct extra_have_objects *ext
}
close(po.in);
+
+ if (args->stateless_rpc) {
+ char *buf = xmalloc(LARGE_PACKET_MAX);
+ while (1) {
+ ssize_t n = xread(po.out, buf, LARGE_PACKET_MAX);
+ if (n <= 0)
+ break;
+ send_sideband(fd, -1, buf, n, LARGE_PACKET_MAX);
+ }
+ free(buf);
+ close(po.out);
+ po.out = -1;
+ }
+
if (finish_command(&po))
return error("pack-objects died with strange error");
return 0;
@@ -303,6 +319,59 @@ static int refs_pushed(struct ref *ref)
return 0;
}
+static void print_helper_status(struct ref *ref)
+{
+ struct strbuf buf = STRBUF_INIT;
+
+ for (; ref; ref = ref->next) {
+ const char *msg = NULL;
+ const char *res;
+
+ switch(ref->status) {
+ case REF_STATUS_NONE:
+ res = "error";
+ msg = "no match";
+ break;
+
+ case REF_STATUS_OK:
+ res = "ok";
+ break;
+
+ case REF_STATUS_UPTODATE:
+ res = "ok";
+ msg = "up to date";
+ break;
+
+ case REF_STATUS_REJECT_NONFASTFORWARD:
+ res = "error";
+ msg = "non-fast forward";
+ break;
+
+ case REF_STATUS_REJECT_NODELETE:
+ case REF_STATUS_REMOTE_REJECT:
+ res = "error";
+ break;
+
+ case REF_STATUS_EXPECTING_REPORT:
+ default:
+ continue;
+ }
+
+ strbuf_reset(&buf);
+ strbuf_addf(&buf, "%s %s", res, ref->name);
+ if (ref->remote_status)
+ msg = ref->remote_status;
+ if (msg) {
+ strbuf_addch(&buf, ' ');
+ quote_two_c_style(&buf, "", msg, 0);
+ }
+ strbuf_addch(&buf, '\n');
+
+ safe_write(1, buf.buf, buf.len);
+ }
+ strbuf_release(&buf);
+}
+
int send_pack(struct send_pack_args *args,
int fd[], struct child_process *conn,
struct ref *remote_refs,
@@ -310,6 +379,7 @@ int send_pack(struct send_pack_args *args,
{
int in = fd[0];
int out = fd[1];
+ struct strbuf req_buf = STRBUF_INIT;
struct ref *ref;
int new_refs;
int ask_for_status_report = 0;
@@ -391,14 +461,14 @@ int send_pack(struct send_pack_args *args,
char *new_hex = sha1_to_hex(ref->new_sha1);
if (ask_for_status_report) {
- packet_write(out, "%s %s %s%c%s",
+ packet_buf_write(&req_buf, "%s %s %s%c%s",
old_hex, new_hex, ref->name, 0,
"report-status");
ask_for_status_report = 0;
expect_status_report = 1;
}
else
- packet_write(out, "%s %s %s",
+ packet_buf_write(&req_buf, "%s %s %s",
old_hex, new_hex, ref->name);
}
ref->status = expect_status_report ?
@@ -406,7 +476,17 @@ int send_pack(struct send_pack_args *args,
REF_STATUS_OK;
}
- packet_flush(out);
+ if (args->stateless_rpc) {
+ if (!args->dry_run) {
+ packet_buf_flush(&req_buf);
+ send_sideband(out, -1, req_buf.buf, req_buf.len, LARGE_PACKET_MAX);
+ }
+ } else {
+ safe_write(out, req_buf.buf, req_buf.len);
+ packet_flush(out);
+ }
+ strbuf_release(&req_buf);
+
if (new_refs && !args->dry_run) {
if (pack_objects(out, remote_refs, extra_have, args) < 0) {
for (ref = remote_refs; ref; ref = ref->next)
@@ -414,11 +494,15 @@ int send_pack(struct send_pack_args *args,
return -1;
}
}
+ if (args->stateless_rpc && !args->dry_run)
+ packet_flush(out);
if (expect_status_report)
ret = receive_status(in, remote_refs);
else
ret = 0;
+ if (args->stateless_rpc)
+ packet_flush(out);
if (ret < 0)
return ret;
@@ -478,6 +562,7 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)
struct extra_have_objects extra_have;
struct ref *remote_refs, *local_refs;
int ret;
+ int helper_status = 0;
int send_all = 0;
const char *receivepack = "git-receive-pack";
int flags;
@@ -523,6 +608,14 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)
args.use_thin_pack = 1;
continue;
}
+ if (!strcmp(arg, "--stateless-rpc")) {
+ args.stateless_rpc = 1;
+ continue;
+ }
+ if (!strcmp(arg, "--helper-status")) {
+ helper_status = 1;
+ continue;
+ }
usage(send_pack_usage);
}
if (!dest) {
@@ -551,7 +644,14 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)
}
}
- conn = git_connect(fd, dest, receivepack, args.verbose ? CONNECT_VERBOSE : 0);
+ if (args.stateless_rpc) {
+ conn = NULL;
+ fd[0] = 0;
+ fd[1] = 1;
+ } else {
+ conn = git_connect(fd, dest, receivepack,
+ args.verbose ? CONNECT_VERBOSE : 0);
+ }
memset(&extra_have, 0, sizeof(extra_have));
@@ -575,12 +675,16 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)
ret = send_pack(&args, fd, conn, remote_refs, &extra_have);
+ if (helper_status)
+ print_helper_status(remote_refs);
+
close(fd[1]);
close(fd[0]);
ret |= finish_connect(conn);
- print_push_status(dest, remote_refs);
+ if (!helper_status)
+ print_push_status(dest, remote_refs);
if (!args.dry_run && remote) {
struct ref *ref;
diff --git a/http.c b/http.c
index 23b2a19..ed6414a 100644
--- a/http.c
+++ b/http.c
@@ -1,9 +1,11 @@
#include "http.h"
#include "pack.h"
+#include "sideband.h"
int data_received;
int active_requests;
int http_is_verbose;
+size_t http_post_buffer = 16 * LARGE_PACKET_MAX;
#ifdef USE_CURL_MULTI
static int max_requests = -1;
@@ -97,8 +99,6 @@ size_t fwrite_null(const void *ptr, size_t eltsize, size_t nmemb, void *strbuf)
return eltsize * nmemb;
}
-static void finish_active_slot(struct active_request_slot *slot);
-
#ifdef USE_CURL_MULTI
static void process_curl_messages(void)
{
@@ -174,6 +174,13 @@ static int http_options(const char *var, const char *value, void *cb)
if (!strcmp("http.proxy", var))
return git_config_string(&curl_http_proxy, var, value);
+ if (!strcmp("http.postbuffer", var)) {
+ http_post_buffer = git_config_int(var, value);
+ if (http_post_buffer < LARGE_PACKET_MAX)
+ http_post_buffer = LARGE_PACKET_MAX;
+ return 0;
+ }
+
/* Fall back on the default ones */
return git_default_config(var, value, cb);
}
@@ -638,7 +645,7 @@ void release_active_slot(struct active_request_slot *slot)
#endif
}
-static void finish_active_slot(struct active_request_slot *slot)
+void finish_active_slot(struct active_request_slot *slot)
{
closedown_active_slot(slot);
curl_easy_getinfo(slot->curl, CURLINFO_HTTP_CODE, &slot->http_code);
diff --git a/http.h b/http.h
index 4c4e99c..f828e1d 100644
--- a/http.h
+++ b/http.h
@@ -79,6 +79,7 @@ extern curlioerr ioctl_buffer(CURL *handle, int cmd, void *clientp);
extern struct active_request_slot *get_active_slot(void);
extern int start_active_slot(struct active_request_slot *slot);
extern void run_active_slot(struct active_request_slot *slot);
+extern void finish_active_slot(struct active_request_slot *slot);
extern void finish_all_active_slots(void);
extern void release_active_slot(struct active_request_slot *slot);
@@ -94,6 +95,7 @@ extern void http_cleanup(void);
extern int data_received;
extern int active_requests;
extern int http_is_verbose;
+extern size_t http_post_buffer;
extern char curl_errorstr[CURL_ERROR_SIZE];
diff --git a/remote-curl.c b/remote-curl.c
index 3917d45..f1206cb 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -6,6 +6,7 @@
#include "exec_cmd.h"
#include "run-command.h"
#include "pkt-line.h"
+#include "sideband.h"
static struct remote *remote;
static const char *url;
@@ -16,7 +17,8 @@ struct options {
unsigned long depth;
unsigned progress : 1,
followtags : 1,
- dry_run : 1;
+ dry_run : 1,
+ thin : 1;
};
static struct options options;
@@ -274,6 +276,188 @@ static void output_refs(struct ref *refs)
free_refs(refs);
}
+struct rpc_state {
+ const char *service_name;
+ const char **argv;
+ char *service_url;
+ char *hdr_content_type;
+ char *hdr_accept;
+ char *buf;
+ size_t alloc;
+ size_t len;
+ size_t pos;
+ int in;
+ int out;
+ struct strbuf result;
+};
+
+static size_t rpc_out(void *ptr, size_t eltsize,
+ size_t nmemb, void *buffer_)
+{
+ size_t max = eltsize * nmemb;
+ struct rpc_state *rpc = buffer_;
+ size_t avail = rpc->len - rpc->pos;
+
+ if (!avail) {
+ avail = packet_read_line(rpc->out, rpc->buf, rpc->alloc);
+ if (!avail)
+ return 0;
+ rpc->pos = 0;
+ rpc->len = avail;
+ }
+
+ if (max < avail);
+ avail = max;
+ memcpy(ptr, rpc->buf + rpc->pos, avail);
+ rpc->pos += avail;
+ return avail;
+}
+
+static size_t rpc_in(const void *ptr, size_t eltsize,
+ size_t nmemb, void *buffer_)
+{
+ size_t size = eltsize * nmemb;
+ struct rpc_state *rpc = buffer_;
+ write_or_die(rpc->in, ptr, size);
+ return size;
+}
+
+static int post_rpc(struct rpc_state *rpc)
+{
+ struct active_request_slot *slot;
+ struct slot_results results;
+ struct curl_slist *headers = NULL;
+ int err = 0, large_request = 0;
+
+ /* Try to load the entire request, if we can fit it into the
+ * allocated buffer space we can use HTTP/1.0 and avoid the
+ * chunked encoding mess.
+ */
+ while (1) {
+ size_t left = rpc->alloc - rpc->len;
+ char *buf = rpc->buf + rpc->len;
+ int n;
+
+ if (left < LARGE_PACKET_MAX) {
+ large_request = 1;
+ break;
+ }
+
+ n = packet_read_line(rpc->out, buf, left);
+ if (!n)
+ break;
+ rpc->len += n;
+ }
+
+ slot = get_active_slot();
+ slot->results = &results;
+
+ curl_easy_setopt(slot->curl, CURLOPT_POST, 1);
+ curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0);
+ curl_easy_setopt(slot->curl, CURLOPT_URL, rpc->service_url);
+
+ headers = curl_slist_append(headers, rpc->hdr_content_type);
+ headers = curl_slist_append(headers, rpc->hdr_accept);
+
+ if (large_request) {
+ /* The request body is large and the size cannot be predicted.
+ * We must use chunked encoding to send it.
+ */
+ headers = curl_slist_append(headers, "Expect: 100-continue");
+ headers = curl_slist_append(headers, "Transfer-Encoding: chunked");
+ curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, rpc_out);
+ curl_easy_setopt(slot->curl, CURLOPT_INFILE, rpc);
+ if (options.verbosity > 1) {
+ fprintf(stderr, "POST %s (chunked)\n", rpc->service_name);
+ fflush(stderr);
+ }
+
+ } else {
+ /* We know the complete request size in advance, use the
+ * more normal Content-Length approach.
+ */
+ curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDS, rpc->buf);
+ curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDSIZE, rpc->len);
+ if (options.verbosity > 1) {
+ fprintf(stderr, "POST %s (%lu bytes)\n",
+ rpc->service_name, (unsigned long)rpc->len);
+ fflush(stderr);
+ }
+ }
+
+ curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, headers);
+ curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, rpc_in);
+ curl_easy_setopt(slot->curl, CURLOPT_FILE, rpc);
+
+ slot->curl_result = curl_easy_perform(slot->curl);
+ finish_active_slot(slot);
+
+ if (results.curl_result != CURLE_OK) {
+ err |= error("RPC failed; result=%d, HTTP code = %ld",
+ results.curl_result, results.http_code);
+ }
+
+ curl_slist_free_all(headers);
+ return err;
+}
+
+static int rpc_service(struct rpc_state *rpc, struct discovery *heads)
+{
+ const char *svc = rpc->service_name;
+ struct strbuf buf = STRBUF_INIT;
+ struct child_process client;
+ int err = 0;
+
+ init_walker();
+ memset(&client, 0, sizeof(client));
+ client.in = -1;
+ client.out = -1;
+ client.git_cmd = 1;
+ client.argv = rpc->argv;
+ if (start_command(&client))
+ exit(1);
+ if (heads)
+ write_or_die(client.in, heads->buf, heads->len);
+
+ rpc->alloc = http_post_buffer;
+ rpc->buf = xmalloc(rpc->alloc);
+ rpc->in = client.in;
+ rpc->out = client.out;
+ strbuf_init(&rpc->result, 0);
+
+ strbuf_addf(&buf, "%s/%s", url, svc);
+ rpc->service_url = strbuf_detach(&buf, NULL);
+
+ strbuf_addf(&buf, "Content-Type: application/x-%s-request", svc);
+ rpc->hdr_content_type = strbuf_detach(&buf, NULL);
+
+ strbuf_addf(&buf, "Accept: application/x-%s-response", svc);
+ rpc->hdr_accept = strbuf_detach(&buf, NULL);
+
+ while (!err) {
+ int n = packet_read_line(rpc->out, rpc->buf, rpc->alloc);
+ if (!n)
+ break;
+ rpc->pos = 0;
+ rpc->len = n;
+ err |= post_rpc(rpc);
+ }
+ strbuf_read(&rpc->result, client.out, 0);
+
+ close(client.in);
+ close(client.out);
+ client.in = -1;
+ client.out = -1;
+
+ err |= finish_command(&client);
+ free(rpc->service_url);
+ free(rpc->hdr_content_type);
+ free(rpc->hdr_accept);
+ free(rpc->buf);
+ strbuf_release(&buf);
+ return err;
+}
+
static int fetch_dumb(int nr_heads, struct ref **to_fetch)
{
char **targets = xmalloc(nr_heads * sizeof(char*));
@@ -371,6 +555,52 @@ static int push_dav(int nr_spec, char **specs)
return 0;
}
+static int push_git(struct discovery *heads, int nr_spec, char **specs)
+{
+ struct rpc_state rpc;
+ const char **argv;
+ int argc = 0, i, err;
+
+ argv = xmalloc((10 + nr_spec) * sizeof(char*));
+ argv[argc++] = "send-pack";
+ argv[argc++] = "--stateless-rpc";
+ argv[argc++] = "--helper-status";
+ if (options.thin)
+ argv[argc++] = "--thin";
+ if (options.dry_run)
+ argv[argc++] = "--dry-run";
+ if (options.verbosity > 1)
+ argv[argc++] = "--verbose";
+ argv[argc++] = url;
+ for (i = 0; i < nr_spec; i++)
+ argv[argc++] = specs[i];
+ argv[argc++] = NULL;
+
+ memset(&rpc, 0, sizeof(rpc));
+ rpc.service_name = "git-receive-pack",
+ rpc.argv = argv;
+
+ err = rpc_service(&rpc, heads);
+ if (rpc.result.len)
+ safe_write(1, rpc.result.buf, rpc.result.len);
+ strbuf_release(&rpc.result);
+ free(argv);
+ return err;
+}
+
+static int push(int nr_spec, char **specs)
+{
+ struct discovery *heads = discover_refs("git-receive-pack");
+ int ret;
+
+ if (heads->proto_git)
+ ret = push_git(heads, nr_spec, specs);
+ else
+ ret = push_dav(nr_spec, specs);
+ free_discovery(heads);
+ return ret;
+}
+
static void parse_push(struct strbuf *buf)
{
char **specs = NULL;
@@ -391,7 +621,7 @@ static void parse_push(struct strbuf *buf)
break;
} while (1);
- if (push_dav(nr_spec, specs))
+ if (push(nr_spec, specs))
exit(128); /* error already reported */
for (i = 0; i < nr_spec; i++)
free(specs[i]);
@@ -414,6 +644,7 @@ int main(int argc, const char **argv)
options.verbosity = 1;
options.progress = !!isatty(2);
+ options.thin = 1;
remote = remote_get(argv[1]);
diff --git a/send-pack.h b/send-pack.h
index 8b3cf02..28141ac 100644
--- a/send-pack.h
+++ b/send-pack.h
@@ -8,7 +8,8 @@ struct send_pack_args {
force_update:1,
use_thin_pack:1,
use_ofs_delta:1,
- dry_run:1;
+ dry_run:1,
+ stateless_rpc:1;
};
int send_pack(struct send_pack_args *args,
diff --git a/sideband.c b/sideband.c
index 899b1ff..d5ffa1c 100644
--- a/sideband.c
+++ b/sideband.c
@@ -135,9 +135,14 @@ ssize_t send_sideband(int fd, int band, const char *data, ssize_t sz, int packet
n = sz;
if (packet_max - 5 < n)
n = packet_max - 5;
- sprintf(hdr, "%04x", n + 5);
- hdr[4] = band;
- safe_write(fd, hdr, 5);
+ if (0 <= band) {
+ sprintf(hdr, "%04x", n + 5);
+ hdr[4] = band;
+ safe_write(fd, hdr, 5);
+ } else {
+ sprintf(hdr, "%04x", n + 4);
+ safe_write(fd, hdr, 4);
+ }
safe_write(fd, p, n);
p += n;
sz -= n;
diff --git a/transport.c b/transport.c
index 6d9652d..2ff1650 100644
--- a/transport.c
+++ b/transport.c
@@ -731,6 +731,7 @@ static int git_transport_push(struct transport *transport, struct ref *remote_re
NULL);
}
+ memset(&args, 0, sizeof(args));
args.send_mirror = !!(flags & TRANSPORT_PUSH_MIRROR);
args.force_update = !!(flags & TRANSPORT_PUSH_FORCE);
args.use_thin_pack = data->thin;
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 16/26] http-backend: add GIT_PROJECT_ROOT environment var
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git; +Cc: Mark Lodato
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
From: Mark Lodato <lodatom@gmail.com>
Add a new environment variable, GIT_PROJECT_ROOT, to override the
method of using PATH_TRANSLATED to find the git repository on disk.
This makes it much easier to configure the web server, especially when
the web server's DocumentRoot does not contain the git repositories,
which is the usual case.
Signed-off-by: Mark Lodato <lodatom@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
Documentation/git-http-backend.txt | 39 +++++++++++++++--------------------
http-backend.c | 25 ++++++++++++++++++++--
2 files changed, 39 insertions(+), 25 deletions(-)
diff --git a/Documentation/git-http-backend.txt b/Documentation/git-http-backend.txt
index 022a243..99dbbfb 100644
--- a/Documentation/git-http-backend.txt
+++ b/Documentation/git-http-backend.txt
@@ -41,29 +41,24 @@ http.receivepack::
URL TRANSLATION
---------------
-'git-http-backend' relies on the invoking web server to perform
-URL to path translation, and store the repository path into the
-PATH_TRANSLATED environment variable. Most web servers will do
-this translation automatically, resolving the suffix after the
-CGI name relative to the server's document root.
+To determine the location of the repository on disk, 'git-http-backend'
+concatenates the environment variables PATH_INFO, which is set
+automatically by the web server, and GIT_PROJECT_ROOT, which must be set
+manually in the web server configuration. If GIT_PROJECT_ROOT is not
+set, 'git-http-backend' reads PATH_TRANSLATED, which is also set
+automatically by the web server.
EXAMPLES
--------
Apache 2.x::
- To serve all Git repositories contained within the '/git/'
- subdirectory of the DocumentRoot, ensure mod_cgi and
- mod_alias are enabled, and create a ScriptAlias to the CGI:
+ Ensure mod_cgi, mod_alias, and mod_env are enabled, set
+ GIT_PROJECT_ROOT (or DocumentRoot) appropriately, and
+ create a ScriptAlias to the CGI:
+
----------------------------------------------------------------
-ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/git/
-
-<Directory /usr/libexec/git-core>
- Options None
-</Directory>
-<Files /usr/libexec/git-core/git-http-backend>
- Options ExecCGI
-</Files>
+SetEnv GIT_PROJECT_ROOT /var/www/git
+ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
----------------------------------------------------------------
+
To enable anonymous read access but authenticated write access,
@@ -78,16 +73,16 @@ require authorization with a LocationMatch directive:
</LocationMatch>
----------------------------------------------------------------
+
-To require authentication for both reads and writes, use a Directory
+To require authentication for both reads and writes, use a Location
directive around the repository, or one of its parent directories:
+
----------------------------------------------------------------
-<Directory /var/www/git/private>
+<Location /git/private>
AuthType Basic
AuthName "Private Git Access"
Require group committers
...
-</Directory>
+</Location>
----------------------------------------------------------------
Accelerated static Apache 2.x::
@@ -97,9 +92,9 @@ Accelerated static Apache 2.x::
file contents from the file system directly to the network:
+
----------------------------------------------------------------
-DocumentRoot /var/www
+SetEnv GIT_PROJECT_ROOT /var/www/git
-ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/git/
+ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
Alias /git_static/ /var/www/git/
RewriteEngine on
@@ -114,7 +109,7 @@ ENVIRONMENT
'git-http-backend' relies upon the CGI environment variables set
by the invoking web server, including:
-* PATH_TRANSLATED
+* PATH_INFO (if GIT_PROJECT_ROOT is set, otherwise PATH_TRANSLATED)
* REMOTE_USER
* REMOTE_ADDR
* CONTENT_TYPE
diff --git a/http-backend.c b/http-backend.c
index 67030b5..8e5c0a2 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -528,6 +528,26 @@ static NORETURN void die_webcgi(const char *err, va_list params)
exit(0);
}
+static char* getdir(void)
+{
+ struct strbuf buf = STRBUF_INIT;
+ char *pathinfo = getenv("PATH_INFO");
+ char *root = getenv("GIT_PROJECT_ROOT");
+ char *path = getenv("PATH_TRANSLATED");
+
+ if (root && *root) {
+ if (!pathinfo || !*pathinfo)
+ die("GIT_PROJECT_ROOT is set but PATH_INFO is not");
+ strbuf_addstr(&buf, root);
+ strbuf_addstr(&buf, pathinfo);
+ return strbuf_detach(&buf, NULL);
+ } else if (path && *path) {
+ return xstrdup(path);
+ } else
+ die("No GIT_PROJECT_ROOT or PATH_TRANSLATED from server");
+ return NULL;
+}
+
static struct service_cmd {
const char *method;
const char *pattern;
@@ -550,7 +570,7 @@ static struct service_cmd {
int main(int argc, char **argv)
{
char *method = getenv("REQUEST_METHOD");
- char *dir = getenv("PATH_TRANSLATED");
+ char *dir;
struct service_cmd *cmd = NULL;
char *cmd_arg = NULL;
int i;
@@ -562,8 +582,7 @@ int main(int argc, char **argv)
die("No REQUEST_METHOD from server");
if (!strcmp(method, "HEAD"))
method = "GET";
- if (!dir)
- die("No PATH_TRANSLATED from server");
+ dir = getdir();
for (i = 0; i < ARRAY_SIZE(services); i++) {
struct service_cmd *c = &services[i];
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 18/26] http-backend: use mod_alias instead of mod_rewrite
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git; +Cc: Mark Lodato
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
From: Mark Lodato <lodatom@gmail.com>
In the git-http-backend documentation, use mod_alias exlusively, instead
of using a combination of mod_alias and mod_rewrite. This makes the
example slightly shorted and a bit more clear.
Signed-off-by: Mark Lodato <lodatom@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
Documentation/git-http-backend.txt | 10 +++-------
1 files changed, 3 insertions(+), 7 deletions(-)
diff --git a/Documentation/git-http-backend.txt b/Documentation/git-http-backend.txt
index 0b5e951..e67519d 100644
--- a/Documentation/git-http-backend.txt
+++ b/Documentation/git-http-backend.txt
@@ -98,13 +98,9 @@ Accelerated static Apache 2.x::
----------------------------------------------------------------
SetEnv GIT_PROJECT_ROOT /var/www/git
-ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
-Alias /git_static/ /var/www/git/
-
-RewriteEngine on
-RewriteRule ^/git/(.*/objects/[0-9a-f]{2}/[0-9a-f]{38})$ /git_static/$1 [PT]
-RewriteRule ^/git/(.*/objects/pack/pack-[0-9a-f]{40}.pack)$ /git_static/$1 [PT]
-RewriteRule ^/git/(.*/objects/pack/pack-[0-9a-f]{40}.idx)$ /git_static/$1 [PT]
+AliasMatch ^/git/(.*/objects/[0-9a-f]{2}/[0-9a-f]{38})$ /var/www/git/$1
+AliasMatch ^/git/(.*/objects/pack/pack-[0-9a-f]{40}.(pack|idx))$ /var/www/git/$1
+ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
----------------------------------------------------------------
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 19/26] http-backend: add example for gitweb on same URL
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git; +Cc: Mark Lodato
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
From: Mark Lodato <lodatom@gmail.com>
In the git-http-backend documentation, add an example of how to set up
gitweb and git-http-backend on the same URL by using a series of
mod_alias commands.
Signed-off-by: Mark Lodato <lodatom@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
Documentation/git-http-backend.txt | 33 +++++++++++++++++++++++++++++++++
1 files changed, 33 insertions(+), 0 deletions(-)
diff --git a/Documentation/git-http-backend.txt b/Documentation/git-http-backend.txt
index e67519d..2989c9f 100644
--- a/Documentation/git-http-backend.txt
+++ b/Documentation/git-http-backend.txt
@@ -88,6 +88,23 @@ directive around the repository, or one of its parent directories:
...
</Location>
----------------------------------------------------------------
++
+To serve gitweb at the same url, use a ScriptAliasMatch to only
+those URLs that 'git-http-backend' can handle, and forward the
+rest to gitweb:
++
+----------------------------------------------------------------
+ScriptAliasMatch \
+ "(?x)^/git/(.*/(HEAD | \
+ info/refs | \
+ objects/(info/[^/]+ | \
+ [0-9a-f]{2}/[0-9a-f]{38} | \
+ pack/pack-[0-9a-f]{40}\.(pack|idx)) | \
+ git-(upload|receive)-pack))$" \
+ /usr/libexec/git-core/git-http-backend/$1
+
+ScriptAlias /git/ /var/www/cgi-bin/gitweb.cgi/
+----------------------------------------------------------------
Accelerated static Apache 2.x::
Similar to the above, but Apache can be used to return static
@@ -102,6 +119,22 @@ AliasMatch ^/git/(.*/objects/[0-9a-f]{2}/[0-9a-f]{38})$ /var/www/git/$1
AliasMatch ^/git/(.*/objects/pack/pack-[0-9a-f]{40}.(pack|idx))$ /var/www/git/$1
ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
----------------------------------------------------------------
++
+This can be combined with the gitweb configuration:
++
+----------------------------------------------------------------
+SetEnv GIT_PROJECT_ROOT /var/www/git
+
+AliasMatch ^/git/(.*/objects/[0-9a-f]{2}/[0-9a-f]{38})$ /var/www/git/$1
+AliasMatch ^/git/(.*/objects/pack/pack-[0-9a-f]{40}.(pack|idx))$ /var/www/git/$1
+ScriptAliasMatch \
+ "(?x)^/git/(.*/(HEAD | \
+ info/refs | \
+ objects/info/[^/]+ | \
+ git-(upload|receive)-pack))$" \
+ /usr/libexec/git-core/git-http-backend/$1
+ScriptAlias /git/ /var/www/cgi-bin/gitweb.cgi/
+----------------------------------------------------------------
ENVIRONMENT
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 21/26] Discover refs via smart HTTP server when available
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git; +Cc: Daniel Barkalow
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
Instead of loading the cached info/refs, try to use the smart HTTP
version when the server supports it. Since the smart variant is
actually the pkt-line stream from the start of either upload-pack
or receive-pack we need to parse these through get_remote_heads,
which requires a background thread to feed its pipe.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Daniel Barkalow <barkalow@iabervon.org>
---
remote-curl.c | 148 ++++++++++++++++++++++++++++++++++++++++++++++++++-------
1 files changed, 131 insertions(+), 17 deletions(-)
diff --git a/remote-curl.c b/remote-curl.c
index 5c9dd97..3917d45 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -5,6 +5,7 @@
#include "http.h"
#include "exec_cmd.h"
#include "run-command.h"
+#include "pkt-line.h"
static struct remote *remote;
static const char *url;
@@ -75,21 +76,46 @@ static int set_option(const char *name, const char *value)
}
}
-static struct ref *get_refs(void)
+struct discovery {
+ const char *service;
+ char *buf_alloc;
+ char *buf;
+ size_t len;
+ unsigned proto_git : 1;
+};
+static struct discovery *last_discovery;
+
+static void free_discovery(struct discovery *d)
+{
+ if (d) {
+ if (d == last_discovery)
+ last_discovery = NULL;
+ free(d->buf_alloc);
+ free(d);
+ }
+}
+
+static struct discovery* discover_refs(const char *service)
{
struct strbuf buffer = STRBUF_INIT;
- char *data, *start, *mid;
- char *ref_name;
+ struct discovery *last = last_discovery;
char *refs_url;
- int i = 0;
- int http_ret;
+ int http_ret, is_http = 0;
- struct ref *refs = NULL;
- struct ref *ref = NULL;
- struct ref *last_ref = NULL;
+ if (last && !strcmp(service, last->service))
+ return last;
+ free_discovery(last);
- refs_url = xmalloc(strlen(url) + 11);
- sprintf(refs_url, "%s/info/refs", url);
+ strbuf_addf(&buffer, "%s/info/refs", url);
+ if (!prefixcmp(url, "http://") || !prefixcmp(url, "https://")) {
+ is_http = 1;
+ if (!strchr(url, '?'))
+ strbuf_addch(&buffer, '?');
+ else
+ strbuf_addch(&buffer, '&');
+ strbuf_addf(&buffer, "service=%s", service);
+ }
+ refs_url = strbuf_detach(&buffer, NULL);
init_walker();
http_ret = http_get_strbuf(refs_url, &buffer, HTTP_NO_CACHE);
@@ -104,10 +130,86 @@ static struct ref *get_refs(void)
die("HTTP request failed");
}
- data = buffer.buf;
+ last= xcalloc(1, sizeof(*last_discovery));
+ last->service = service;
+ last->buf_alloc = strbuf_detach(&buffer, &last->len);
+ last->buf = last->buf_alloc;
+
+ if (is_http && 5 <= last->len && last->buf[4] == '#') {
+ /* smart HTTP response; validate that the service
+ * pkt-line matches our request.
+ */
+ struct strbuf exp = STRBUF_INIT;
+
+ if (packet_get_line(&buffer, &last->buf, &last->len) <= 0)
+ die("%s has invalid packet header", refs_url);
+ if (buffer.len && buffer.buf[buffer.len - 1] == '\n')
+ strbuf_setlen(&buffer, buffer.len - 1);
+
+ strbuf_addf(&exp, "# service=%s", service);
+ if (strbuf_cmp(&exp, &buffer))
+ die("invalid server response; got '%s'", buffer.buf);
+ strbuf_release(&exp);
+
+ /* The header can include additional metadata lines, up
+ * until a packet flush marker. Ignore these now, but
+ * in the future we might start to scan them.
+ */
+ strbuf_reset(&buffer);
+ while (packet_get_line(&buffer, &last->buf, &last->len) > 0)
+ strbuf_reset(&buffer);
+
+ last->proto_git = 1;
+ }
+
+ free(refs_url);
+ strbuf_release(&buffer);
+ last_discovery = last;
+ return last;
+}
+
+static int write_discovery(int fd, void *data)
+{
+ struct discovery *heads = data;
+ int err = 0;
+ if (write_in_full(fd, heads->buf, heads->len) != heads->len)
+ err = 1;
+ close(fd);
+ return err;
+}
+
+static struct ref *parse_git_refs(struct discovery *heads)
+{
+ struct ref *list = NULL;
+ struct async async;
+
+ memset(&async, 0, sizeof(async));
+ async.proc = write_discovery;
+ async.data = heads;
+
+ if (start_async(&async))
+ die("cannot start thread to parse advertised refs");
+ get_remote_heads(async.out, &list, 0, NULL, 0, NULL);
+ close(async.out);
+ if (finish_async(&async))
+ die("ref parsing thread failed");
+ return list;
+}
+
+static struct ref *parse_info_refs(struct discovery *heads)
+{
+ char *data, *start, *mid;
+ char *ref_name;
+ int i = 0;
+
+ struct ref *refs = NULL;
+ struct ref *ref = NULL;
+ struct ref *last_ref = NULL;
+
+ data = heads->buf;
start = NULL;
mid = data;
- while (i < buffer.len) {
+ while (i < heads->len) {
if (!start) {
start = &data[i];
}
@@ -131,8 +233,7 @@ static struct ref *get_refs(void)
i++;
}
- strbuf_release(&buffer);
-
+ init_walker();
ref = alloc_ref("HEAD");
if (!walker->fetch_ref(walker, ref) &&
!resolve_remote_symref(ref, refs)) {
@@ -142,11 +243,23 @@ static struct ref *get_refs(void)
free(ref);
}
- strbuf_release(&buffer);
- free(refs_url);
return refs;
}
+static struct ref *get_refs(int for_push)
+{
+ struct discovery *heads;
+
+ if (for_push)
+ heads = discover_refs("git-receive-pack");
+ else
+ heads = discover_refs("git-upload-pack");
+
+ if (heads->proto_git)
+ return parse_git_refs(heads);
+ return parse_info_refs(heads);
+}
+
static void output_refs(struct ref *refs)
{
struct ref *posn;
@@ -317,7 +430,8 @@ int main(int argc, const char **argv)
parse_fetch(&buf);
} else if (!strcmp(buf.buf, "list") || !prefixcmp(buf.buf, "list ")) {
- output_refs(get_refs());
+ int for_push = !!strstr(buf.buf + 4, "for-push");
+ output_refs(get_refs(for_push));
} else if (!prefixcmp(buf.buf, "push ")) {
parse_push(&buf);
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 11/26] Move WebDAV HTTP push under remote-curl
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git; +Cc: Tay Ray Chuan, Clemens Buchacher, Daniel Barkalow, Mike Hommey
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
The remote helper interface now supports the push capability,
which can be used to ask the implementation to push one or more
specs to the remote repository. For remote-curl we implement this
by calling the existing WebDAV based git-http-push executable.
Internally the helper interface uses the push_refs transport hook
so that the complexity of the refspec parsing and matching can be
reused between remote implementations. When possible however the
helper protocol uses source ref name rather than the source SHA-1,
thereby allowing the helper to access this name if it is useful.
>From Clemens Buchacher <drizzd@aon.at>:
update http tests according to remote-curl capabilities
o Pushing packed refs is now fixed.
o The transport helper fails if refs are already up-to-date. Add
a test for that.
o The transport helper will notice if refs are already
up-to-date. We therefore need to update server info in the
unpacked-refs test.
o The transport helper will purge deleted branches automatically.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Daniel Barkalow <barkalow@iabervon.org>
CC: Mike Hommey <mh@glandium.org>
---
Documentation/git-remote-helpers.txt | 33 ++++++++-
http-push.c | 29 +++++++-
remote-curl.c | 97 +++++++++++++++++++++---
t/t5540-http-push.sh | 14 ++-
transport-helper.c | 137 +++++++++++++++++++++++++++++++++-
transport.c | 31 --------
6 files changed, 287 insertions(+), 54 deletions(-)
diff --git a/Documentation/git-remote-helpers.txt b/Documentation/git-remote-helpers.txt
index 1133f04..8beb42d 100644
--- a/Documentation/git-remote-helpers.txt
+++ b/Documentation/git-remote-helpers.txt
@@ -34,6 +34,10 @@ Commands are given by the caller on the helper's standard input, one per line.
value of the ref. A space-separated list of attributes follows
the name; unrecognized attributes are ignored. After the
complete list, outputs a blank line.
++
+If 'push' is supported this may be called as 'list for-push'
+to obtain the current refs prior to sending one or more 'push'
+commands to the helper.
'option' <name> <value>::
Set the transport helper option <name> to <value>. Outputs a
@@ -59,6 +63,22 @@ suitably updated.
+
Supported if the helper has the "fetch" capability.
+'push' +<src>:<dst>::
+ Pushes the given <src> commit or branch locally to the
+ remote branch described by <dst>. A batch sequence of
+ one or more push commands is terminated with a blank line.
++
+Zero or more protocol options may be entered after the last 'push'
+command, before the batch's terminating blank line.
++
+When the push is complete, outputs one or more 'ok <dst>' or
+'error <dst> <why>?' lines to indicate success or failure of
+each pushed ref. The status report output is terminated by
+a blank line. The option field <why> may be quoted in a C
+style string if it contains an LF.
++
+Supported if the helper has the "push" capability.
+
If a fatal error occurs, the program writes the error message to
stderr and exits. The caller should expect that a suitable error
message has been printed if the child closes the connection without
@@ -76,10 +96,16 @@ CAPABILITIES
'option'::
This helper supports the option command.
+'push'::
+ This helper supports the 'push' command.
+
REF LIST ATTRIBUTES
-------------------
-None are defined yet, but the caller must accept any which are supplied.
+'for-push'::
+ The caller wants to use the ref list to prepare push
+ commands. A helper might chose to acquire the ref list by
+ opening a different type of connection to the destination.
OPTIONS
-------
@@ -106,6 +132,11 @@ OPTIONS
ask for the tag specifically. Some helpers may be able to
use this option to avoid a second network connection.
+'option dry-run' \{'true'|'false'\}:
+ If true, pretend the operation completed successfully,
+ but don't actually change any repository data. For most
+ helpers this only applies to the 'push', if supported.
+
Documentation
-------------
Documentation by Daniel Barkalow.
diff --git a/http-push.c b/http-push.c
index cc5d4b8..f10803a 100644
--- a/http-push.c
+++ b/http-push.c
@@ -78,6 +78,7 @@ static int push_verbosely;
static int push_all = MATCH_REFS_NONE;
static int force_all;
static int dry_run;
+static int helper_status;
static struct object_list *objects;
@@ -1813,6 +1814,10 @@ int main(int argc, char **argv)
dry_run = 1;
continue;
}
+ if (!strcmp(arg, "--helper-status")) {
+ helper_status = 1;
+ continue;
+ }
if (!strcmp(arg, "--verbose")) {
push_verbosely = 1;
http_is_verbose = 1;
@@ -1911,9 +1916,12 @@ int main(int argc, char **argv)
/* Remove a remote branch if -d or -D was specified */
if (delete_branch) {
- if (delete_remote_branch(refspec[0], force_delete) == -1)
+ if (delete_remote_branch(refspec[0], force_delete) == -1) {
fprintf(stderr, "Unable to delete remote branch %s\n",
refspec[0]);
+ if (helper_status)
+ printf("error %s cannot remove\n", refspec[0]);
+ }
goto cleanup;
}
@@ -1925,6 +1933,8 @@ int main(int argc, char **argv)
}
if (!remote_refs) {
fprintf(stderr, "No refs in common and none specified; doing nothing.\n");
+ if (helper_status)
+ printf("error null no match\n");
rc = 0;
goto cleanup;
}
@@ -1942,8 +1952,12 @@ int main(int argc, char **argv)
if (is_null_sha1(ref->peer_ref->new_sha1)) {
if (delete_remote_branch(ref->name, 1) == -1) {
error("Could not remove %s", ref->name);
+ if (helper_status)
+ printf("error %s cannot remove\n", ref->name);
rc = -4;
}
+ else if (helper_status)
+ printf("ok %s\n", ref->name);
new_refs++;
continue;
}
@@ -1951,6 +1965,8 @@ int main(int argc, char **argv)
if (!hashcmp(ref->old_sha1, ref->peer_ref->new_sha1)) {
if (push_verbosely || 1)
fprintf(stderr, "'%s': up-to-date\n", ref->name);
+ if (helper_status)
+ printf("ok %s up to date\n", ref->name);
continue;
}
@@ -1974,6 +1990,8 @@ int main(int argc, char **argv)
"need to pull first?",
ref->name,
ref->peer_ref->name);
+ if (helper_status)
+ printf("error %s non-fast forward\n", ref->name);
rc = -2;
continue;
}
@@ -1987,14 +2005,19 @@ int main(int argc, char **argv)
if (strcmp(ref->name, ref->peer_ref->name))
fprintf(stderr, " using '%s'", ref->peer_ref->name);
fprintf(stderr, "\n from %s\n to %s\n", old_hex, new_hex);
- if (dry_run)
+ if (dry_run) {
+ if (helper_status)
+ printf("ok %s\n", ref->name);
continue;
+ }
/* Lock remote branch ref */
ref_lock = lock_remote(ref->name, LOCK_TIME);
if (ref_lock == NULL) {
fprintf(stderr, "Unable to lock remote branch %s\n",
ref->name);
+ if (helper_status)
+ printf("error %s lock error\n", ref->name);
rc = 1;
continue;
}
@@ -2045,6 +2068,8 @@ int main(int argc, char **argv)
if (!rc)
fprintf(stderr, " done\n");
+ if (helper_status)
+ printf("%s %s\n", !rc ? "ok" : "error", ref->name);
unlock_remote(ref_lock);
check_locks();
}
diff --git a/remote-curl.c b/remote-curl.c
index 0951f11..5c9dd97 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -4,6 +4,7 @@
#include "walker.h"
#include "http.h"
#include "exec_cmd.h"
+#include "run-command.h"
static struct remote *remote;
static const char *url;
@@ -13,7 +14,8 @@ struct options {
int verbosity;
unsigned long depth;
unsigned progress : 1,
- followtags : 1;
+ followtags : 1,
+ dry_run : 1;
};
static struct options options;
@@ -59,6 +61,15 @@ static int set_option(const char *name, const char *value)
return -1;
return 1 /* TODO implement later */;
}
+ else if (!strcmp(name, "dry-run")) {
+ if (!strcmp(value, "true"))
+ options.dry_run = 1;
+ else if (!strcmp(value, "false"))
+ options.dry_run = 0;
+ else
+ return -1;
+ return 0;
+ }
else {
return 1 /* unsupported */;
}
@@ -136,6 +147,20 @@ static struct ref *get_refs(void)
return refs;
}
+static void output_refs(struct ref *refs)
+{
+ struct ref *posn;
+ for (posn = refs; posn; posn = posn->next) {
+ if (posn->symref)
+ printf("@%s %s\n", posn->symref, posn->name);
+ else
+ printf("%s %s\n", sha1_to_hex(posn->old_sha1), posn->name);
+ }
+ printf("\n");
+ fflush(stdout);
+ free_refs(refs);
+}
+
static int fetch_dumb(int nr_heads, struct ref **to_fetch)
{
char **targets = xmalloc(nr_heads * sizeof(char*));
@@ -211,6 +236,58 @@ static void parse_fetch(struct strbuf *buf)
strbuf_reset(buf);
}
+static int push_dav(int nr_spec, char **specs)
+{
+ const char **argv = xmalloc((10 + nr_spec) * sizeof(char*));
+ int argc = 0, i;
+
+ argv[argc++] = "http-push";
+ argv[argc++] = "--helper-status";
+ if (options.dry_run)
+ argv[argc++] = "--dry-run";
+ if (options.verbosity > 1)
+ argv[argc++] = "--verbose";
+ argv[argc++] = url;
+ for (i = 0; i < nr_spec; i++)
+ argv[argc++] = specs[i];
+ argv[argc++] = NULL;
+
+ if (run_command_v_opt(argv, RUN_GIT_CMD))
+ die("git-%s failed", argv[0]);
+ free(argv);
+ return 0;
+}
+
+static void parse_push(struct strbuf *buf)
+{
+ char **specs = NULL;
+ int alloc_spec = 0, nr_spec = 0, i;
+
+ do {
+ if (!prefixcmp(buf->buf, "push ")) {
+ ALLOC_GROW(specs, nr_spec + 1, alloc_spec);
+ specs[nr_spec++] = xstrdup(buf->buf + 5);
+ }
+ else
+ die("http transport does not support %s", buf->buf);
+
+ strbuf_reset(buf);
+ if (strbuf_getline(buf, stdin, '\n') == EOF)
+ return;
+ if (!*buf->buf)
+ break;
+ } while (1);
+
+ if (push_dav(nr_spec, specs))
+ exit(128); /* error already reported */
+ for (i = 0; i < nr_spec; i++)
+ free(specs[i]);
+ free(specs);
+
+ printf("\n");
+ fflush(stdout);
+}
+
int main(int argc, const char **argv)
{
struct strbuf buf = STRBUF_INIT;
@@ -239,17 +316,12 @@ int main(int argc, const char **argv)
if (!prefixcmp(buf.buf, "fetch ")) {
parse_fetch(&buf);
- } else if (!strcmp(buf.buf, "list")) {
- struct ref *refs = get_refs();
- struct ref *posn;
- for (posn = refs; posn; posn = posn->next) {
- if (posn->symref)
- printf("@%s %s\n", posn->symref, posn->name);
- else
- printf("%s %s\n", sha1_to_hex(posn->old_sha1), posn->name);
- }
- printf("\n");
- fflush(stdout);
+ } else if (!strcmp(buf.buf, "list") || !prefixcmp(buf.buf, "list ")) {
+ output_refs(get_refs());
+
+ } else if (!prefixcmp(buf.buf, "push ")) {
+ parse_push(&buf);
+
} else if (!prefixcmp(buf.buf, "option ")) {
char *name = buf.buf + strlen("option ");
char *value = strchr(name, ' ');
@@ -272,6 +344,7 @@ int main(int argc, const char **argv)
} else if (!strcmp(buf.buf, "capabilities")) {
printf("fetch\n");
printf("option\n");
+ printf("push\n");
printf("\n");
fflush(stdout);
} else {
diff --git a/t/t5540-http-push.sh b/t/t5540-http-push.sh
index f4a2cf6..09edd23 100755
--- a/t/t5540-http-push.sh
+++ b/t/t5540-http-push.sh
@@ -36,6 +36,7 @@ test_expect_success 'setup remote repository' '
cd test_repo.git &&
git --bare update-server-info &&
mv hooks/post-update.sample hooks/post-update &&
+ ORIG_HEAD=$(git rev-parse --verify HEAD) &&
cd - &&
mv test_repo.git "$HTTPD_DOCUMENT_ROOT_PATH"
'
@@ -45,7 +46,7 @@ test_expect_success 'clone remote repository' '
git clone $HTTPD_URL/test_repo.git test_repo_clone
'
-test_expect_failure 'push to remote repository with packed refs' '
+test_expect_success 'push to remote repository with packed refs' '
cd "$ROOT_PATH"/test_repo_clone &&
: >path2 &&
git add path2 &&
@@ -57,11 +58,15 @@ test_expect_failure 'push to remote repository with packed refs' '
test $HEAD = $(git rev-parse --verify HEAD))
'
-test_expect_success ' push to remote repository with unpacked refs' '
+test_expect_failure 'push already up-to-date' '
+ git push
+'
+
+test_expect_success 'push to remote repository with unpacked refs' '
(cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git &&
rm packed-refs &&
- git update-ref refs/heads/master \
- 0c973ae9bd51902a28466f3850b543fa66a6aaf4) &&
+ git update-ref refs/heads/master $ORIG_HEAD &&
+ git --bare update-server-info) &&
git push &&
(cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git &&
test $HEAD = $(git rev-parse --verify HEAD))
@@ -113,7 +118,6 @@ test_expect_success 'create and delete remote branch' '
git push origin dev &&
git fetch &&
git push origin :dev &&
- git branch -d -r origin/dev &&
git fetch &&
test_must_fail git show-ref --verify refs/remotes/origin/dev
'
diff --git a/transport-helper.c b/transport-helper.c
index 577abc6..16c6641 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -1,6 +1,6 @@
#include "cache.h"
#include "transport.h"
-
+#include "quote.h"
#include "run-command.h"
#include "commit.h"
#include "diff.h"
@@ -13,7 +13,8 @@ struct helper_data
struct child_process *helper;
FILE *out;
unsigned fetch : 1,
- option : 1;
+ option : 1,
+ push : 1;
};
static struct child_process *get_helper(struct transport *transport)
@@ -52,6 +53,8 @@ static struct child_process *get_helper(struct transport *transport)
data->fetch = 1;
if (!strcmp(buf.buf, "option"))
data->option = 1;
+ if (!strcmp(buf.buf, "push"))
+ data->push = 1;
}
return data->helper;
}
@@ -214,6 +217,130 @@ static int fetch(struct transport *transport,
return -1;
}
+static int push_refs(struct transport *transport,
+ struct ref *remote_refs, int flags)
+{
+ int force_all = flags & TRANSPORT_PUSH_FORCE;
+ int mirror = flags & TRANSPORT_PUSH_MIRROR;
+ struct helper_data *data = transport->data;
+ struct strbuf buf = STRBUF_INIT;
+ struct child_process *helper;
+ struct ref *ref;
+
+ if (!remote_refs)
+ return 0;
+
+ helper = get_helper(transport);
+ if (!data->push)
+ return 1;
+
+ for (ref = remote_refs; ref; ref = ref->next) {
+ if (ref->peer_ref)
+ hashcpy(ref->new_sha1, ref->peer_ref->new_sha1);
+ else if (!mirror)
+ continue;
+
+ ref->deletion = is_null_sha1(ref->new_sha1);
+ if (!ref->deletion &&
+ !hashcmp(ref->old_sha1, ref->new_sha1)) {
+ ref->status = REF_STATUS_UPTODATE;
+ continue;
+ }
+
+ if (force_all)
+ ref->force = 1;
+
+ strbuf_addstr(&buf, "push ");
+ if (!ref->deletion) {
+ if (ref->force)
+ strbuf_addch(&buf, '+');
+ if (ref->peer_ref)
+ strbuf_addstr(&buf, ref->peer_ref->name);
+ else
+ strbuf_addstr(&buf, sha1_to_hex(ref->new_sha1));
+ }
+ strbuf_addch(&buf, ':');
+ strbuf_addstr(&buf, ref->name);
+ strbuf_addch(&buf, '\n');
+ }
+
+ transport->verbose = flags & TRANSPORT_PUSH_VERBOSE ? 1 : 0;
+ standard_options(transport);
+
+ if (flags & TRANSPORT_PUSH_DRY_RUN) {
+ if (set_helper_option(transport, "dry-run", "true") != 0)
+ die("helper %s does not support dry-run", data->name);
+ }
+
+ strbuf_addch(&buf, '\n');
+ if (write_in_full(helper->in, buf.buf, buf.len) != buf.len)
+ exit(128);
+
+ ref = remote_refs;
+ while (1) {
+ char *refname, *msg;
+ int status;
+
+ strbuf_reset(&buf);
+ if (strbuf_getline(&buf, data->out, '\n') == EOF)
+ exit(128); /* child died, message supplied already */
+ if (!buf.len)
+ break;
+
+ if (!prefixcmp(buf.buf, "ok ")) {
+ status = REF_STATUS_OK;
+ refname = buf.buf + 3;
+ } else if (!prefixcmp(buf.buf, "error ")) {
+ status = REF_STATUS_REMOTE_REJECT;
+ refname = buf.buf + 6;
+ } else
+ die("expected ok/error, helper said '%s'\n", buf.buf);
+
+ msg = strchr(refname, ' ');
+ if (msg) {
+ struct strbuf msg_buf = STRBUF_INIT;
+ const char *end;
+
+ *msg++ = '\0';
+ if (!unquote_c_style(&msg_buf, msg, &end))
+ msg = strbuf_detach(&msg_buf, NULL);
+ else
+ msg = xstrdup(msg);
+ strbuf_release(&msg_buf);
+
+ if (!strcmp(msg, "no match")) {
+ status = REF_STATUS_NONE;
+ free(msg);
+ msg = NULL;
+ }
+ else if (!strcmp(msg, "up to date")) {
+ status = REF_STATUS_UPTODATE;
+ free(msg);
+ msg = NULL;
+ }
+ else if (!strcmp(msg, "non-fast forward")) {
+ status = REF_STATUS_REJECT_NONFASTFORWARD;
+ free(msg);
+ msg = NULL;
+ }
+ }
+
+ if (ref)
+ ref = find_ref_by_name(ref, refname);
+ if (!ref)
+ ref = find_ref_by_name(remote_refs, refname);
+ if (!ref) {
+ warning("helper reported unexpected status of %s", refname);
+ continue;
+ }
+
+ ref->status = status;
+ ref->remote_status = msg;
+ }
+ strbuf_release(&buf);
+ return 0;
+}
+
static struct ref *get_refs_list(struct transport *transport, int for_push)
{
struct helper_data *data = transport->data;
@@ -225,7 +352,10 @@ static struct ref *get_refs_list(struct transport *transport, int for_push)
helper = get_helper(transport);
- write_str_in_full(helper->in, "list\n");
+ if (data->push && for_push)
+ write_str_in_full(helper->in, "list for-push\n");
+ else
+ write_str_in_full(helper->in, "list\n");
while (1) {
char *eov, *eon;
@@ -266,6 +396,7 @@ int transport_helper_init(struct transport *transport, const char *name)
transport->set_option = set_helper_option;
transport->get_refs_list = get_refs_list;
transport->fetch = fetch;
+ transport->push_refs = push_refs;
transport->disconnect = disconnect_helper;
return 0;
}
diff --git a/transport.c b/transport.c
index 644a30a..6d9652d 100644
--- a/transport.c
+++ b/transport.c
@@ -349,35 +349,6 @@ static int rsync_transport_push(struct transport *transport,
return result;
}
-#ifndef NO_CURL
-static int curl_transport_push(struct transport *transport, int refspec_nr, const char **refspec, int flags)
-{
- const char **argv;
- int argc;
-
- if (flags & TRANSPORT_PUSH_MIRROR)
- return error("http transport does not support mirror mode");
-
- argv = xmalloc((refspec_nr + 12) * sizeof(char *));
- argv[0] = "http-push";
- argc = 1;
- if (flags & TRANSPORT_PUSH_ALL)
- argv[argc++] = "--all";
- if (flags & TRANSPORT_PUSH_FORCE)
- argv[argc++] = "--force";
- if (flags & TRANSPORT_PUSH_DRY_RUN)
- argv[argc++] = "--dry-run";
- if (flags & TRANSPORT_PUSH_VERBOSE)
- argv[argc++] = "--verbose";
- argv[argc++] = transport->url;
- while (refspec_nr--)
- argv[argc++] = *refspec++;
- argv[argc] = NULL;
- return !!run_command_v_opt(argv, RUN_GIT_CMD);
-}
-
-#endif
-
struct bundle_transport_data {
int fd;
struct bundle_header header;
@@ -826,8 +797,6 @@ struct transport *transport_get(struct remote *remote, const char *url)
transport_helper_init(ret, "curl");
#ifdef NO_CURL
error("git was compiled without libcurl support.");
-#else
- ret->push = curl_transport_push;
#endif
} else if (is_local(url) && is_file(url)) {
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 15/26] Smart fetch and push over HTTP: server side
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
Requests for $GIT_URL/git-receive-pack and $GIT_URL/git-upload-pack
are forwarded to the corresponding backend process by directly
executing it and leaving stdin and stdout connected to the invoking
web server. Prior to starting the backend process the HTTP response
headers are sent, thereby freeing the backend from needing to know
about the HTTP protocol.
Requests that are encoded with Content-Encoding: gzip are
automatically inflated before being streamed into the backend.
This is primarily useful for the git-upload-pack backend, which
receives highly repetitive text data from clients that easily
compresses to 50% of its original size.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
Documentation/git-http-backend.txt | 39 +++++-
http-backend.c | 324 +++++++++++++++++++++++++++++++++++-
2 files changed, 359 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-http-backend.txt b/Documentation/git-http-backend.txt
index 867675f..022a243 100644
--- a/Documentation/git-http-backend.txt
+++ b/Documentation/git-http-backend.txt
@@ -22,6 +22,23 @@ By default, only the `upload-pack` service is enabled, which serves
This is ideally suited for read-only updates, i.e., pulling from
git repositories.
+SERVICES
+--------
+These services can be enabled/disabled using the per-repository
+configuration file:
+
+http.uploadpack::
+ This serves 'git-fetch-pack' and 'git-ls-remote' clients.
+ It is enabled by default, but a repository can disable it
+ by setting this configuration item to `false`.
+
+http.receivepack::
+ This serves 'git-send-pack' clients, allowing push. It is
+ disabled by default for anonymous users, and enabled by
+ default for users authenticated by the web server. It can be
+ disabled by setting this item to `false`, or enabled for all
+ users, including anonymous users, by setting it to `true`.
+
URL TRANSLATION
---------------
'git-http-backend' relies on the invoking web server to perform
@@ -49,7 +66,19 @@ ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/git/
</Files>
----------------------------------------------------------------
+
-To require authentication for reads, use a Directory
+To enable anonymous read access but authenticated write access,
+require authorization with a LocationMatch directive:
++
+----------------------------------------------------------------
+<LocationMatch ".*/git-receive-pack$">
+ AuthType Basic
+ AuthName "Git Access"
+ Require group committers
+ ...
+</LocationMatch>
+----------------------------------------------------------------
++
+To require authentication for both reads and writes, use a Directory
directive around the repository, or one of its parent directories:
+
----------------------------------------------------------------
@@ -92,6 +121,14 @@ by the invoking web server, including:
* QUERY_STRING
* REQUEST_METHOD
+The backend process sets GIT_COMMITTER_NAME to '$REMOTE_USER' and
+GIT_COMMITTER_EMAIL to '$\{REMOTE_USER}@http.$\{REMOTE_ADDR\}',
+ensuring that any reflogs created by 'git-receive-pack' contain some
+identifying information of the remote user who performed the push.
+
+All CGI environment variables are available to each of the hooks
+invoked by the 'git-receive-pack'.
+
Author
------
Written by Shawn O. Pearce <spearce@spearce.org>.
diff --git a/http-backend.c b/http-backend.c
index 374f60d..67030b5 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -4,11 +4,109 @@
#include "object.h"
#include "tag.h"
#include "exec_cmd.h"
+#include "run-command.h"
+#include "string-list.h"
static const char content_type[] = "Content-Type";
static const char content_length[] = "Content-Length";
static const char last_modified[] = "Last-Modified";
+static struct string_list *query_params;
+
+struct rpc_service {
+ const char *name;
+ const char *config_name;
+ signed enabled : 2;
+};
+
+static struct rpc_service rpc_service[] = {
+ { "upload-pack", "uploadpack", 1 },
+ { "receive-pack", "receivepack", -1 },
+};
+
+static int decode_char(const char *q)
+{
+ int i;
+ unsigned char val = 0;
+ for (i = 0; i < 2; i++) {
+ unsigned char c = *q++;
+ val <<= 4;
+ if (c >= '0' && c <= '9')
+ val += c - '0';
+ else if (c >= 'a' && c <= 'f')
+ val += c - 'a' + 10;
+ else if (c >= 'A' && c <= 'F')
+ val += c - 'A' + 10;
+ else
+ return -1;
+ }
+ return val;
+}
+
+static char *decode_parameter(const char **query, int is_name)
+{
+ const char *q = *query;
+ struct strbuf out;
+
+ strbuf_init(&out, 16);
+ do {
+ unsigned char c = *q;
+
+ if (!c)
+ break;
+ if (c == '&' || (is_name && c == '=')) {
+ q++;
+ break;
+ }
+
+ if (c == '%') {
+ int val = decode_char(q + 1);
+ if (0 <= val) {
+ strbuf_addch(&out, val);
+ q += 3;
+ continue;
+ }
+ }
+
+ if (c == '+')
+ strbuf_addch(&out, ' ');
+ else
+ strbuf_addch(&out, c);
+ q++;
+ } while (1);
+ *query = q;
+ return strbuf_detach(&out, NULL);
+}
+
+static struct string_list *get_parameters(void)
+{
+ if (!query_params) {
+ const char *query = getenv("QUERY_STRING");
+
+ query_params = xcalloc(1, sizeof(*query_params));
+ while (query && *query) {
+ char *name = decode_parameter(&query, 1);
+ char *value = decode_parameter(&query, 0);
+ struct string_list_item *i;
+
+ i = string_list_lookup(name, query_params);
+ if (!i)
+ i = string_list_insert(name, query_params);
+ else
+ free(i->util);
+ i->util = value;
+ }
+ }
+ return query_params;
+}
+
+static const char *get_parameter(const char *name)
+{
+ struct string_list_item *i;
+ i = string_list_lookup(name, get_parameters());
+ return i ? i->util : NULL;
+}
+
static void format_write(int fd, const char *fmt, ...)
{
static char buffer[1024];
@@ -81,6 +179,21 @@ static NORETURN void not_found(const char *err, ...)
exit(0);
}
+static NORETURN void forbidden(const char *err, ...)
+{
+ va_list params;
+
+ http_status(403, "Forbidden");
+ hdr_nocache();
+ end_headers();
+
+ va_start(params, err);
+ if (err && *err)
+ vfprintf(stderr, err, params);
+ va_end(params);
+ exit(0);
+}
+
static void send_strbuf(const char *type, struct strbuf *buf)
{
hdr_int(content_length, buf->len);
@@ -147,6 +260,145 @@ static void get_idx_file(char *name)
send_file("application/x-git-packed-objects-toc", name);
}
+static int http_config(const char *var, const char *value, void *cb)
+{
+ struct rpc_service *svc = cb;
+
+ if (!prefixcmp(var, "http.") &&
+ !strcmp(var + 5, svc->config_name)) {
+ svc->enabled = git_config_bool(var, value);
+ return 0;
+ }
+
+ /* we are not interested in parsing any other configuration here */
+ return 0;
+}
+
+static struct rpc_service *select_service(const char *name)
+{
+ struct rpc_service *svc = NULL;
+ int i;
+
+ if (prefixcmp(name, "git-"))
+ forbidden("Unsupported service: '%s'", name);
+
+ for (i = 0; i < ARRAY_SIZE(rpc_service); i++) {
+ struct rpc_service *s = &rpc_service[i];
+ if (!strcmp(s->name, name + 4)) {
+ svc = s;
+ break;
+ }
+ }
+
+ if (!svc)
+ forbidden("Unsupported service: '%s'", name);
+
+ git_config(http_config, svc);
+ if (svc->enabled < 0) {
+ const char *user = getenv("REMOTE_USER");
+ svc->enabled = (user && *user) ? 1 : 0;
+ }
+ if (!svc->enabled)
+ forbidden("Service not enabled: '%s'", svc->name);
+ return svc;
+}
+
+static void inflate_request(const char *prog_name, int out)
+{
+ z_stream stream;
+ unsigned char in_buf[8192];
+ unsigned char out_buf[8192];
+ unsigned long cnt = 0;
+ int ret;
+
+ memset(&stream, 0, sizeof(stream));
+ ret = inflateInit2(&stream, (15 + 16));
+ if (ret != Z_OK)
+ die("cannot start zlib inflater, zlib err %d", ret);
+
+ while (1) {
+ ssize_t n = xread(0, in_buf, sizeof(in_buf));
+ if (n <= 0)
+ die("request ended in the middle of the gzip stream");
+
+ stream.next_in = in_buf;
+ stream.avail_in = n;
+
+ while (0 < stream.avail_in) {
+ int ret;
+
+ stream.next_out = out_buf;
+ stream.avail_out = sizeof(out_buf);
+
+ ret = inflate(&stream, Z_NO_FLUSH);
+ if (ret != Z_OK && ret != Z_STREAM_END)
+ die("zlib error inflating request, result %d", ret);
+
+ n = stream.total_out - cnt;
+ if (write_in_full(out, out_buf, n) != n)
+ die("%s aborted reading request", prog_name);
+ cnt += n;
+
+ if (ret == Z_STREAM_END)
+ goto done;
+ }
+ }
+
+done:
+ inflateEnd(&stream);
+ close(out);
+}
+
+static void run_service(const char **argv)
+{
+ const char *encoding = getenv("HTTP_CONTENT_ENCODING");
+ const char *user = getenv("REMOTE_USER");
+ const char *host = getenv("REMOTE_ADDR");
+ char *env[3];
+ struct strbuf buf = STRBUF_INIT;
+ int gzipped_request = 0;
+ struct child_process cld;
+
+ if (encoding && !strcmp(encoding, "gzip"))
+ gzipped_request = 1;
+ else if (encoding && !strcmp(encoding, "x-gzip"))
+ gzipped_request = 1;
+
+ if (!user || !*user)
+ user = "anonymous";
+ if (!host || !*host)
+ host = "(none)";
+
+ memset(&env, 0, sizeof(env));
+ strbuf_addf(&buf, "GIT_COMMITTER_NAME=%s", user);
+ env[0] = strbuf_detach(&buf, NULL);
+
+ strbuf_addf(&buf, "GIT_COMMITTER_EMAIL=%s@http.%s", user, host);
+ env[1] = strbuf_detach(&buf, NULL);
+ env[2] = NULL;
+
+ memset(&cld, 0, sizeof(cld));
+ cld.argv = argv;
+ cld.env = (const char *const *)env;
+ if (gzipped_request)
+ cld.in = -1;
+ cld.git_cmd = 1;
+ if (start_command(&cld))
+ exit(1);
+
+ close(1);
+ if (gzipped_request)
+ inflate_request(argv[0], cld.in);
+ else
+ close(0);
+
+ if (finish_command(&cld))
+ exit(1);
+ free(env[0]);
+ free(env[1]);
+ strbuf_release(&buf);
+}
+
static int show_text_ref(const char *name, const unsigned char *sha1,
int flag, void *cb_data)
{
@@ -167,11 +419,32 @@ static int show_text_ref(const char *name, const unsigned char *sha1,
static void get_info_refs(char *arg)
{
+ const char *service_name = get_parameter("service");
struct strbuf buf = STRBUF_INIT;
- for_each_ref(show_text_ref, &buf);
hdr_nocache();
- send_strbuf("text/plain", &buf);
+
+ if (service_name) {
+ const char *argv[] = {NULL /* service name */,
+ "--stateless-rpc", "--advertise-refs",
+ ".", NULL};
+ struct rpc_service *svc = select_service(service_name);
+
+ strbuf_addf(&buf, "application/x-git-%s-advertisement",
+ svc->name);
+ hdr_str(content_type, buf.buf);
+ end_headers();
+
+ packet_write(1, "# service=git-%s\n", svc->name);
+ packet_flush(1);
+
+ argv[0] = svc->name;
+ run_service(argv);
+
+ } else {
+ for_each_ref(show_text_ref, &buf);
+ send_strbuf("text/plain", &buf);
+ }
strbuf_release(&buf);
}
@@ -200,6 +473,48 @@ static void get_info_packs(char *arg)
strbuf_release(&buf);
}
+static void check_content_type(const char *accepted_type)
+{
+ const char *actual_type = getenv("CONTENT_TYPE");
+
+ if (!actual_type)
+ actual_type = "";
+
+ if (strcmp(actual_type, accepted_type)) {
+ http_status(415, "Unsupported Media Type");
+ hdr_nocache();
+ end_headers();
+ format_write(1,
+ "Expected POST with Content-Type '%s',"
+ " but received '%s' instead.\n",
+ accepted_type, actual_type);
+ exit(0);
+ }
+}
+
+static void service_rpc(char *service_name)
+{
+ const char *argv[] = {NULL, "--stateless-rpc", ".", NULL};
+ struct rpc_service *svc = select_service(service_name);
+ struct strbuf buf = STRBUF_INIT;
+
+ strbuf_reset(&buf);
+ strbuf_addf(&buf, "application/x-git-%s-request", svc->name);
+ check_content_type(buf.buf);
+
+ hdr_nocache();
+
+ strbuf_reset(&buf);
+ strbuf_addf(&buf, "application/x-git-%s-result", svc->name);
+ hdr_str(content_type, buf.buf);
+
+ end_headers();
+
+ argv[0] = svc->name;
+ run_service(argv);
+ strbuf_release(&buf);
+}
+
static NORETURN void die_webcgi(const char *err, va_list params)
{
char buffer[1000];
@@ -226,7 +541,10 @@ static struct service_cmd {
{"GET", "/objects/info/[^/]*$", get_text_file},
{"GET", "/objects/[0-9a-f]{2}/[0-9a-f]{38}$", get_loose_object},
{"GET", "/objects/pack/pack-[0-9a-f]{40}\\.pack$", get_pack_file},
- {"GET", "/objects/pack/pack-[0-9a-f]{40}\\.idx$", get_idx_file}
+ {"GET", "/objects/pack/pack-[0-9a-f]{40}\\.idx$", get_idx_file},
+
+ {"POST", "/git-upload-pack$", service_rpc},
+ {"POST", "/git-receive-pack$", service_rpc}
};
int main(int argc, char **argv)
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 12/26] remote-helpers: return successfully if everything up-to-date
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git; +Cc: Clemens Buchacher
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
From: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
t/t5540-http-push.sh | 2 +-
transport-helper.c | 2 ++
2 files changed, 3 insertions(+), 1 deletions(-)
diff --git a/t/t5540-http-push.sh b/t/t5540-http-push.sh
index 09edd23..2ece661 100755
--- a/t/t5540-http-push.sh
+++ b/t/t5540-http-push.sh
@@ -58,7 +58,7 @@ test_expect_success 'push to remote repository with packed refs' '
test $HEAD = $(git rev-parse --verify HEAD))
'
-test_expect_failure 'push already up-to-date' '
+test_expect_success 'push already up-to-date' '
git push
'
diff --git a/transport-helper.c b/transport-helper.c
index 16c6641..5078c71 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -263,6 +263,8 @@ static int push_refs(struct transport *transport,
strbuf_addstr(&buf, ref->name);
strbuf_addch(&buf, '\n');
}
+ if (buf.len == 0)
+ return 0;
transport->verbose = flags & TRANSPORT_PUSH_VERBOSE ? 1 : 0;
standard_options(transport);
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 14/26] Add stateless RPC options to upload-pack, receive-pack
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
When --stateless-rpc is passed as a command line parameter to
upload-pack or receive-pack the programs now assume they may
perform only a single read-write cycle with stdin and stdout.
This fits with the HTTP POST request processing model where a
program may read the request, write a response, and must exit.
When --advertise-refs is passed as a command line parameter only
the initial ref advertisement is output, and the program exits
immediately. This fits with the HTTP GET request model, where
no request content is received but a response must be produced.
HTTP headers and/or environment are not processed here, but
instead are assumed to be handled by the program invoking
either service backend.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
builtin-receive-pack.c | 26 ++++++++++++++++++++------
upload-pack.c | 40 ++++++++++++++++++++++++++++++++++++----
2 files changed, 56 insertions(+), 10 deletions(-)
diff --git a/builtin-receive-pack.c b/builtin-receive-pack.c
index b771fe9..70ff8c5 100644
--- a/builtin-receive-pack.c
+++ b/builtin-receive-pack.c
@@ -615,6 +615,8 @@ static void add_alternate_refs(void)
int cmd_receive_pack(int argc, const char **argv, const char *prefix)
{
+ int advertise_refs = 0;
+ int stateless_rpc = 0;
int i;
char *dir = NULL;
@@ -623,7 +625,15 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
const char *arg = *argv++;
if (*arg == '-') {
- /* Do flag handling here */
+ if (!strcmp(arg, "--advertise-refs")) {
+ advertise_refs = 1;
+ continue;
+ }
+ if (!strcmp(arg, "--stateless-rpc")) {
+ stateless_rpc = 1;
+ continue;
+ }
+
usage(receive_pack_usage);
}
if (dir)
@@ -652,12 +662,16 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
" report-status delete-refs ofs-delta " :
" report-status delete-refs ";
- add_alternate_refs();
- write_head_info();
- clear_extra_refs();
+ if (advertise_refs || !stateless_rpc) {
+ add_alternate_refs();
+ write_head_info();
+ clear_extra_refs();
- /* EOF */
- packet_flush(1);
+ /* EOF */
+ packet_flush(1);
+ }
+ if (advertise_refs)
+ return 0;
read_head_info();
if (commands) {
diff --git a/upload-pack.c b/upload-pack.c
index f1dc3a3..70badcf 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -39,6 +39,8 @@ static unsigned int timeout;
*/
static int use_sideband;
static int debug_fd;
+static int advertise_refs;
+static int stateless_rpc;
static void reset_timeout(void)
{
@@ -509,6 +511,8 @@ static int get_common_commits(void)
if (!len) {
if (have_obj.nr == 0 || multi_ack)
packet_write(1, "NAK\n");
+ if (stateless_rpc)
+ exit(0);
continue;
}
strip(line, len);
@@ -710,12 +714,32 @@ static int send_ref(const char *refname, const unsigned char *sha1, int flag, vo
return 0;
}
+static int mark_our_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)
+{
+ struct object *o = parse_object(sha1);
+ if (!o)
+ die("git upload-pack: cannot find object %s:", sha1_to_hex(sha1));
+ if (!(o->flags & OUR_REF)) {
+ o->flags |= OUR_REF;
+ nr_our_refs++;
+ }
+ return 0;
+}
+
static void upload_pack(void)
{
- reset_timeout();
- head_ref(send_ref, NULL);
- for_each_ref(send_ref, NULL);
- packet_flush(1);
+ if (advertise_refs || !stateless_rpc) {
+ reset_timeout();
+ head_ref(send_ref, NULL);
+ for_each_ref(send_ref, NULL);
+ packet_flush(1);
+ } else {
+ head_ref(mark_our_ref, NULL);
+ for_each_ref(mark_our_ref, NULL);
+ }
+ if (advertise_refs)
+ return;
+
receive_needs();
if (want_obj.nr) {
get_common_commits();
@@ -737,6 +761,14 @@ int main(int argc, char **argv)
if (arg[0] != '-')
break;
+ if (!strcmp(arg, "--advertise-refs")) {
+ advertise_refs = 1;
+ continue;
+ }
+ if (!strcmp(arg, "--stateless-rpc")) {
+ stateless_rpc = 1;
+ continue;
+ }
if (!strcmp(arg, "--strict")) {
strict = 1;
continue;
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 06/26] Add multi_ack_detailed capability to fetch-pack/upload-pack
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
When multi_ack_detailed is enabled the ACK continue messages returned
by the remote upload-pack are broken out to describe the different
states within the peer. This permits the client to better understand
the server's in-memory state.
The fetch-pack/upload-pack protocol now looks like:
NAK
---------------------------------
Always sent in response to "done" if there was no common base
selected from the "have" lines (or no have lines were sent).
* no multi_ack or multi_ack_detailed:
Sent when the client has sent a pkt-line flush ("0000") and
the server has not yet found a common base object.
* either multi_ack or multi_ack_detailed:
Always sent in response to a pkt-line flush.
ACK %s
-----------------------------------
* no multi_ack or multi_ack_detailed:
Sent in response to "have" when the object exists on the remote
side and is therefore an object in common between the peers.
The argument is the SHA-1 of the common object.
* either multi_ack or multi_ack_detailed:
Sent in response to "done" if there are common objects.
The argument is the last SHA-1 determined to be common.
ACK %s continue
-----------------------------------
* multi_ack only:
Sent in response to "have".
The remote side wants the client to consider this object as
common, and immediately stop transmitting additional "have"
lines for objects that are reachable from it. The reason
the client should stop is not given, but is one of the two
cases below available under multi_ack_detailed.
ACK %s common
-----------------------------------
* multi_ack_detailed only:
Sent in response to "have". Both sides have this object.
Like with "ACK %s continue" above the client should stop
sending have lines reachable for objects from the argument.
ACK %s ready
-----------------------------------
* multi_ack_detailed only:
Sent in response to "have".
The client should stop transmitting objects which are reachable
from the argument, and send "done" soon to get the objects.
If the remote side has the specified object, it should
first send an "ACK %s common" message prior to sending
"ACK %s ready".
Clients may still submit additional "have" lines if there are
more side branches for the client to explore that might be added
to the common set and reduce the number of objects to transfer.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
builtin-fetch-pack.c | 41 ++++++++++++++++++++++++++++++++---------
upload-pack.c | 31 ++++++++++++++++++-------------
2 files changed, 50 insertions(+), 22 deletions(-)
diff --git a/builtin-fetch-pack.c b/builtin-fetch-pack.c
index 7c09d46..615f549 100644
--- a/builtin-fetch-pack.c
+++ b/builtin-fetch-pack.c
@@ -157,7 +157,15 @@ static const unsigned char *get_rev(void)
return commit->object.sha1;
}
-static int get_ack(int fd, unsigned char *result_sha1)
+enum ack_type {
+ NAK = 0,
+ ACK,
+ ACK_continue,
+ ACK_common,
+ ACK_ready
+};
+
+static enum ack_type get_ack(int fd, unsigned char *result_sha1)
{
static char line[1000];
int len = packet_read_line(fd, line, sizeof(line));
@@ -167,12 +175,16 @@ static int get_ack(int fd, unsigned char *result_sha1)
if (line[len-1] == '\n')
line[--len] = 0;
if (!strcmp(line, "NAK"))
- return 0;
+ return NAK;
if (!prefixcmp(line, "ACK ")) {
if (!get_sha1_hex(line+4, result_sha1)) {
if (strstr(line+45, "continue"))
- return 2;
- return 1;
+ return ACK_continue;
+ if (strstr(line+45, "common"))
+ return ACK_common;
+ if (strstr(line+45, "ready"))
+ return ACK_ready;
+ return ACK;
}
}
die("git fetch_pack: expected ACK/NAK, got '%s'", line);
@@ -218,7 +230,8 @@ static int find_common(int fd[2], unsigned char *result_sha1,
remote_hex = sha1_to_hex(remote);
if (!fetching) {
struct strbuf c = STRBUF_INIT;
- if (multi_ack) strbuf_addstr(&c, " multi_ack");
+ if (multi_ack == 2) strbuf_addstr(&c, " multi_ack_detailed");
+ if (multi_ack == 1) strbuf_addstr(&c, " multi_ack");
if (use_sideband == 2) strbuf_addstr(&c, " side-band-64k");
if (use_sideband == 1) strbuf_addstr(&c, " side-band");
if (args.use_thin_pack) strbuf_addstr(&c, " thin-pack");
@@ -298,18 +311,23 @@ static int find_common(int fd[2], unsigned char *result_sha1,
if (args.verbose && ack)
fprintf(stderr, "got ack %d %s\n", ack,
sha1_to_hex(result_sha1));
- if (ack == 1) {
+ switch (ack) {
+ case ACK:
flushes = 0;
multi_ack = 0;
retval = 0;
goto done;
- } else if (ack == 2) {
+ case ACK_common:
+ case ACK_ready:
+ case ACK_continue: {
struct commit *commit =
lookup_commit(result_sha1);
mark_common(commit, 0, 1);
retval = 0;
in_vain = 0;
got_continue = 1;
+ break;
+ }
}
} while (ack);
flushes--;
@@ -336,7 +354,7 @@ done:
if (args.verbose)
fprintf(stderr, "got ack (%d) %s\n", ack,
sha1_to_hex(result_sha1));
- if (ack == 1)
+ if (ack == ACK)
return 0;
multi_ack = 1;
continue;
@@ -618,7 +636,12 @@ static struct ref *do_fetch_pack(int fd[2],
if (is_repository_shallow() && !server_supports("shallow"))
die("Server does not support shallow clients");
- if (server_supports("multi_ack")) {
+ if (server_supports("multi_ack_detailed")) {
+ if (args.verbose)
+ fprintf(stderr, "Server supports multi_ack_detailed\n");
+ multi_ack = 2;
+ }
+ else if (server_supports("multi_ack")) {
if (args.verbose)
fprintf(stderr, "Server supports multi_ack\n");
multi_ack = 1;
diff --git a/upload-pack.c b/upload-pack.c
index 38ddac2..f1dc3a3 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -498,7 +498,7 @@ static int get_common_commits(void)
{
static char line[1000];
unsigned char sha1[20];
- char hex[41], last_hex[41];
+ char last_hex[41];
save_commit_buffer = 0;
@@ -515,19 +515,22 @@ static int get_common_commits(void)
if (!prefixcmp(line, "have ")) {
switch (got_sha1(line+5, sha1)) {
case -1: /* they have what we do not */
- if (multi_ack && ok_to_give_up())
- packet_write(1, "ACK %s continue\n",
- sha1_to_hex(sha1));
+ if (multi_ack && ok_to_give_up()) {
+ const char *hex = sha1_to_hex(sha1);
+ if (multi_ack == 2)
+ packet_write(1, "ACK %s ready\n", hex);
+ else
+ packet_write(1, "ACK %s continue\n", hex);
+ }
break;
default:
- memcpy(hex, sha1_to_hex(sha1), 41);
- if (multi_ack) {
- const char *msg = "ACK %s continue\n";
- packet_write(1, msg, hex);
- memcpy(last_hex, hex, 41);
- }
+ memcpy(last_hex, sha1_to_hex(sha1), 41);
+ if (multi_ack == 2)
+ packet_write(1, "ACK %s common\n", last_hex);
+ else if (multi_ack)
+ packet_write(1, "ACK %s continue\n", last_hex);
else if (have_obj.nr == 1)
- packet_write(1, "ACK %s\n", hex);
+ packet_write(1, "ACK %s\n", last_hex);
break;
}
continue;
@@ -587,7 +590,9 @@ static void receive_needs(void)
get_sha1_hex(line+5, sha1_buf))
die("git upload-pack: protocol error, "
"expected to get sha, not '%s'", line);
- if (strstr(line+45, "multi_ack"))
+ if (strstr(line+45, "multi_ack_detailed"))
+ multi_ack = 2;
+ else if (strstr(line+45, "multi_ack"))
multi_ack = 1;
if (strstr(line+45, "thin-pack"))
use_thin_pack = 1;
@@ -681,7 +686,7 @@ static int send_ref(const char *refname, const unsigned char *sha1, int flag, vo
{
static const char *capabilities = "multi_ack thin-pack side-band"
" side-band-64k ofs-delta shallow no-progress"
- " include-tag";
+ " include-tag multi_ack_detailed";
struct object *o = parse_object(sha1);
if (!o)
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 03/26] pkt-line: Make packet_read_line easier to debug
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
When there is an error parsing the 4 byte length component we now
NUL terminate the string and display it as part of the die message,
this may hint as to what data was misunderstood by the application.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
pkt-line.c | 8 +++++---
1 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/pkt-line.c b/pkt-line.c
index bd603f8..893dd3c 100644
--- a/pkt-line.c
+++ b/pkt-line.c
@@ -124,12 +124,14 @@ static int packet_length(const char *linelen)
int packet_read_line(int fd, char *buffer, unsigned size)
{
int len;
- char linelen[4];
+ char linelen[5];
safe_read(fd, linelen, 4);
len = packet_length(linelen);
- if (len < 0)
- die("protocol error: bad line length character");
+ if (len < 0) {
+ linelen[4] = '\0';
+ die("protocol error: bad line length character: %s", linelen);
+ }
if (!len)
return 0;
len -= 4;
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 05/26] Move "get_ack()" back to fetch-pack
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git
In-Reply-To: <1256774448-7625-1-git-send-email-spearce@spearce.org>
In 41cb7488 Linus moved this function to connect.c for reuse inside
of the git-clone-pack command. That was 2005, but in 2006 Junio
retired git-clone-pack in commit efc7fa53. Since then the only
caller has been fetch-pack. Since this ACK/NAK exchange is only
used by the fetch-pack/upload-pack protocol we should keep move
it back to a private detail of fetch-pack.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
builtin-fetch-pack.c | 21 +++++++++++++++++++++
cache.h | 1 -
connect.c | 21 ---------------------
3 files changed, 21 insertions(+), 22 deletions(-)
diff --git a/builtin-fetch-pack.c b/builtin-fetch-pack.c
index 783c2b0..7c09d46 100644
--- a/builtin-fetch-pack.c
+++ b/builtin-fetch-pack.c
@@ -157,6 +157,27 @@ static const unsigned char *get_rev(void)
return commit->object.sha1;
}
+static int get_ack(int fd, unsigned char *result_sha1)
+{
+ static char line[1000];
+ int len = packet_read_line(fd, line, sizeof(line));
+
+ if (!len)
+ die("git fetch-pack: expected ACK/NAK, got EOF");
+ if (line[len-1] == '\n')
+ line[--len] = 0;
+ if (!strcmp(line, "NAK"))
+ return 0;
+ if (!prefixcmp(line, "ACK ")) {
+ if (!get_sha1_hex(line+4, result_sha1)) {
+ if (strstr(line+45, "continue"))
+ return 2;
+ return 1;
+ }
+ }
+ die("git fetch_pack: expected ACK/NAK, got '%s'", line);
+}
+
static int find_common(int fd[2], unsigned char *result_sha1,
struct ref *refs)
{
diff --git a/cache.h b/cache.h
index a5eeead..4e283be 100644
--- a/cache.h
+++ b/cache.h
@@ -856,7 +856,6 @@ extern struct ref *find_ref_by_name(const struct ref *list, const char *name);
extern struct child_process *git_connect(int fd[2], const char *url, const char *prog, int flags);
extern int finish_connect(struct child_process *conn);
extern int path_match(const char *path, int nr, char **match);
-extern int get_ack(int fd, unsigned char *result_sha1);
struct extra_have_objects {
int nr, alloc;
unsigned char (*array)[20];
diff --git a/connect.c b/connect.c
index 7945e38..839a103 100644
--- a/connect.c
+++ b/connect.c
@@ -107,27 +107,6 @@ int server_supports(const char *feature)
strstr(server_capabilities, feature) != NULL;
}
-int get_ack(int fd, unsigned char *result_sha1)
-{
- static char line[1000];
- int len = packet_read_line(fd, line, sizeof(line));
-
- if (!len)
- die("git fetch-pack: expected ACK/NAK, got EOF");
- if (line[len-1] == '\n')
- line[--len] = 0;
- if (!strcmp(line, "NAK"))
- return 0;
- if (!prefixcmp(line, "ACK ")) {
- if (!get_sha1_hex(line+4, result_sha1)) {
- if (strstr(line+45, "continue"))
- return 2;
- return 1;
- }
- }
- die("git fetch_pack: expected ACK/NAK, got '%s'", line);
-}
-
int path_match(const char *path, int nr, char **match)
{
int i;
--
1.6.5.2.181.gd6f41
^ permalink raw reply related
* [RFC PATCH v4 00/26] Return of smart HTTP
From: Shawn O. Pearce @ 2009-10-29 0:00 UTC (permalink / raw)
To: git
I think this is the final spin of the smart HTTP series. I've
collected patches from a few others (thanks folks!) and added
tests specific to the smart variant of the HTTP transport.
At this point, I think it is "next ready"... but would appreciate
any additional feedback if folks identify something we should
address before hitting next.
Clemens Buchacher (1):
remote-helpers: return successfully if everything up-to-date
Mark Lodato (5):
http-backend: add GIT_PROJECT_ROOT environment var
http-backend: reword some documentation
http-backend: use mod_alias instead of mod_rewrite
http-backend: add example for gitweb on same URL
http-backend: more explict LocationMatch
Shawn O. Pearce (18):
pkt-line: Add strbuf based functions
pkt-line: Make packet_read_line easier to debug
fetch-pack: Use a strbuf to compose the want list
Move "get_ack()" back to fetch-pack
Add multi_ack_detailed capability to fetch-pack/upload-pack
remote-curl: Refactor walker initialization
fetch: Allow transport -v -v -v to set verbosity to 3
remote-helpers: Fetch more than one ref in a batch
remote-helpers: Support custom transport options
Move WebDAV HTTP push under remote-curl
Git-aware CGI to provide dumb HTTP transport
Add stateless RPC options to upload-pack, receive-pack
Smart fetch and push over HTTP: server side
Discover refs via smart HTTP server when available
Smart push over HTTP: client side
Smart fetch over HTTP: client side
Smart HTTP fetch: gzip requests
test smart http fetch and push
Tay Ray Chuan (2):
http-push: fix check condition on http.c::finish_http_pack_request()
t5540-http-push: remove redundant fetches
.gitignore | 1 +
Documentation/config.txt | 8 +
Documentation/git-http-backend.txt | 170 ++++++++
Documentation/git-remote-helpers.txt | 85 ++++-
Makefile | 1 +
builtin-fetch-pack.c | 210 ++++++++--
builtin-fetch.c | 2 +-
builtin-receive-pack.c | 26 +-
builtin-send-pack.c | 116 +++++-
cache.h | 1 -
commit.c | 10 +-
commit.h | 2 +-
connect.c | 21 -
fetch-pack.h | 3 +-
http-backend.c | 627 ++++++++++++++++++++++++++++
http-push.c | 31 ++-
http.c | 13 +-
http.h | 2 +
pkt-line.c | 86 ++++-
pkt-line.h | 4 +
remote-curl.c | 759 ++++++++++++++++++++++++++++++++--
send-pack.h | 3 +-
sideband.c | 11 +-
t/lib-httpd/apache.conf | 20 +
t/t5540-http-push.sh | 18 +-
t/t5541-http-push.sh | 103 +++++
t/t5550-http-fetch.sh | 8 +-
t/t5551-http-fetch.sh | 87 ++++
transport-helper.c | 264 +++++++++++-
transport.c | 32 +--
transport.h | 2 +-
upload-pack.c | 71 +++-
32 files changed, 2574 insertions(+), 223 deletions(-)
create mode 100644 Documentation/git-http-backend.txt
create mode 100644 http-backend.c
create mode 100755 t/t5541-http-push.sh
create mode 100755 t/t5551-http-fetch.sh
^ permalink raw reply
* Re: [PATCH] mergetool--lib: add p4merge as a pre-configured mergetool option
From: Junio C Hamano @ 2009-10-28 23:37 UTC (permalink / raw)
To: Scott Chacon
Cc: Jay Soffian, git list, Junio C Hamano, Charles Bailey,
David Aguilar
In-Reply-To: <d411cc4a0910281439v3388c243v42b3700f73744623@mail.gmail.com>
Thanks. Is Jay happy with this version?
^ permalink raw reply
* Re: [Vcs-fast-import-devs] What's cooking in git.git (Oct 2009, #01; Wed, 07)
From: Ian Clatworthy @ 2009-10-28 23:19 UTC (permalink / raw)
To: Sverre Rabbelier
Cc: Shawn O. Pearce, vcs-fast-import-devs, git, Johannes Schindelin
In-Reply-To: <fabb9a1e0910281508m3e9bb8a6g7b39abc29fceae78@mail.gmail.com>
Sverre Rabbelier wrote:
> Shawn, what do you want to do with this, it seems the vcs devs are not
> very interested in this feature, should I implement it as described
> above?
Sverre,
I'll try to take a look today. Sorry for the lack of response so far -
other stuff has been swamping my time and this hasn't reached the top of
my TODO list unfortunately.
Ian C.
^ permalink raw reply
* Re: Add '--bisect' revision machinery argument
From: Linus Torvalds @ 2009-10-28 23:32 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Git Mailing List, Christian Couder
In-Reply-To: <7viqdzgls9.fsf@alter.siamese.dyndns.org>
On Wed, 28 Oct 2009, Junio C Hamano wrote:
>
> This shows a very nice direction to evolve, but your patch as-is breaks
> "rev-list --bisect", I think.
I think you're right. I tested git rev-parse, and the 'git log' machinery,
but I didn't think about the fact that we already had a meaning
for '--bisect' in rev-list.
> Also, the helper of "git bisect" can and probably should be taught to just
> ask this new behaviour from the revision machinery, instead of collecting
> good and bad refs itself using bisect.c::read_bisect_refs().
Yeah. And git-bisect.sh can be simplified too.
Linus
^ permalink raw reply
* Re: Add '--bisect' revision machinery argument
From: Junio C Hamano @ 2009-10-28 23:07 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Git Mailing List, Christian Couder
In-Reply-To: <alpine.LFD.2.01.0910271124110.31845@localhost.localdomain>
Linus Torvalds <torvalds@linux-foundation.org> writes:
> So this adds "--bisect" as a revision parsing argument, and as a result it
> just works with all the normal logging tools. So now I can just do
>
> gitk --bisect --simplify-by-decoration filename-here
This shows a very nice direction to evolve, but your patch as-is breaks
"rev-list --bisect", I think. Call to your setup_revisions() eats the
command line "--bisect" option but cmd_rev_list() wants to see it to go
into the "bisection" mode of traversal.
Also, the helper of "git bisect" can and probably should be taught to just
ask this new behaviour from the revision machinery, instead of collecting
good and bad refs itself using bisect.c::read_bisect_refs().
Here is a short-term fix that can be squashed in, in order to allow t6022
to pass again.
builtin-rev-list.c | 2 ++
revision.c | 1 +
revision.h | 1 +
3 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/builtin-rev-list.c b/builtin-rev-list.c
index 4ba1c12..32bf033 100644
--- a/builtin-rev-list.c
+++ b/builtin-rev-list.c
@@ -319,6 +319,8 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
memset(&info, 0, sizeof(info));
info.revs = &revs;
+ if (revs.bisect)
+ bisect_list = 1;
quiet = DIFF_OPT_TST(&revs.diffopt, QUIET);
for (i = 1 ; i < argc; i++) {
diff --git a/revision.c b/revision.c
index 80a0528..a36c0d9 100644
--- a/revision.c
+++ b/revision.c
@@ -1273,6 +1273,7 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, const ch
if (!strcmp(arg, "--bisect")) {
handle_refs(revs, flags, for_each_bad_bisect_ref);
handle_refs(revs, flags ^ UNINTERESTING, for_each_good_bisect_ref);
+ revs->bisect = 1;
continue;
}
if (!strcmp(arg, "--tags")) {
diff --git a/revision.h b/revision.h
index b6421a6..921656a 100644
--- a/revision.h
+++ b/revision.h
@@ -63,6 +63,7 @@ struct rev_info {
reverse:1,
reverse_output_stage:1,
cherry_pick:1,
+ bisect:1,
first_parent_only:1;
/* Diff flags */
^ permalink raw reply related
* [PATCH v3 2/2] filter-branch: nearest-ancestor rewriting outside subdir filter
From: Thomas Rast @ 2009-10-28 22:59 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Johannes Sixt
In-Reply-To: <6e01558f719f4bfcd12f3c6dc5657790e86c874d.1256770377.git.trast@student.ethz.ch>
Since a0e4639 (filter-branch: fix ref rewriting with
--subdirectory-filter, 2008-08-12) git-filter-branch has done
nearest-ancestor rewriting when using a --subdirectory-filter.
However, that rewriting strategy is also a useful building block in
other tasks. For example, if you want to split out a subset of files
from your history, you would typically call
git filter-branch -- <refs> -- <files>
But this fails for all refs that do not point directly to a commit
that affects <files>, because their referenced commit will not be
rewritten and the ref remains untouched.
The code was already there for the --subdirectory-filter case, so just
introduce an option that enables it independently.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
---
Same as v2.
Documentation/git-filter-branch.txt | 13 ++++++++++++-
git-filter-branch.sh | 9 ++++++++-
t/t7003-filter-branch.sh | 18 ++++++++++++++++++
3 files changed, 38 insertions(+), 2 deletions(-)
diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
index 2b40bab..394a77a 100644
--- a/Documentation/git-filter-branch.txt
+++ b/Documentation/git-filter-branch.txt
@@ -159,7 +159,18 @@ to other tags will be rewritten to point to the underlying commit.
--subdirectory-filter <directory>::
Only look at the history which touches the given subdirectory.
The result will contain that directory (and only that) as its
- project root.
+ project root. Implies --remap-to-ancestor.
+
+--remap-to-ancestor::
+ Rewrite refs to the nearest rewritten ancestor instead of
+ ignoring them.
++
+Normally, positive refs on the command line are only changed if the
+commit they point to was rewritten. However, you can limit the extent
+of this rewriting by using linkgit:rev-list[1] arguments, e.g., path
+limiters. Refs pointing to such excluded commits would then normally
+be ignored. With this option, they are instead rewritten to point at
+the nearest ancestor that was not excluded.
--prune-empty::
Some kind of filters will generate empty commits, that left the tree
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index da23b99..ad2bc6f 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -125,6 +125,7 @@ filter_subdir=
orig_namespace=refs/original/
force=
prune_empty=
+remap_to_ancestor=
while :
do
case "$1" in
@@ -137,6 +138,11 @@ do
force=t
continue
;;
+ --remap-to-ancestor)
+ shift
+ remap_to_ancestor=t
+ continue
+ ;;
--prune-empty)
shift
prune_empty=t
@@ -182,6 +188,7 @@ do
;;
--subdirectory-filter)
filter_subdir="$OPTARG"
+ remap_to_ancestor=t
;;
--original)
orig_namespace=$(expr "$OPTARG/" : '\(.*[^/]\)/*$')/
@@ -358,7 +365,7 @@ done <../revs
# revision walker. Fix it by mapping these heads to the unique nearest
# ancestor that survived the pruning.
-if test "$filter_subdir"
+if test "$remap_to_ancestor" = t
then
while read ref
do
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 329c851..9503875 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -288,4 +288,22 @@ test_expect_success 'Prune empty commits' '
test_cmp expect actual
'
+test_expect_success '--remap-to-ancestor with filename filters' '
+ git checkout master &&
+ git reset --hard A &&
+ test_commit add-foo foo 1 &&
+ git branch moved-foo &&
+ test_commit add-bar bar a &&
+ git branch invariant &&
+ orig_invariant=$(git rev-parse invariant) &&
+ git branch moved-bar &&
+ test_commit change-foo foo 2 &&
+ git filter-branch -f --remap-to-ancestor \
+ moved-foo moved-bar A..master \
+ -- -- foo &&
+ test $(git rev-parse moved-foo) = $(git rev-parse moved-bar) &&
+ test $(git rev-parse moved-foo) = $(git rev-parse master^) &&
+ test $orig_invariant = $(git rev-parse invariant)
+'
+
test_done
--
1.6.5.1.161.g3b9c0
^ permalink raw reply related
* [PATCH v3 1/2] filter-branch: stop special-casing $filter_subdir argument
From: Thomas Rast @ 2009-10-28 22:59 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Johannes Sixt
In-Reply-To: <4AE0187C.4040608@viscovery.net>
Handling $filter_subdir in the usual way requires a separate case at
every use, because the variable is empty when unused.
Furthermore, the case for --subdirectory-filter supplies its own --,
so the user cannot provide one himself, so the following was
impossible:
git filter-branch --subdirectory-filter subdir -- --all -- subdir/file
To keep the argument handling sane, we filter $@ to contain only the
non-revision arguments, and store all revisions in $ref_args. The
$ref_args are easy to handle since only the SHA1s are needed; the
actual branch names have already been stored in $tempdir/heads at this
point.
An extra separating -- is only required if the user did not provide
any non-revision arguments, as the latter disambiguate the
$filter_subdir following after them (or fail earlier because they are
ambiguous themselves).
Thanks to Johannes Sixt for suggesting this solution.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
---
Johannes Sixt wrote:
> When the shell expands $variable (outside quotes), it does not apply
> quotes anymore, but only word-splits using $IFS. In your code, the words
> would contain literal single-quotes, and paths with spaces would still be
> split into words.
If there's a good reason for these weird rules, I'm still missing
it...
But your suggestion works very nicely.
git-filter-branch.sh | 21 ++++++++++++++-------
1 files changed, 14 insertions(+), 7 deletions(-)
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index a480d6f..da23b99 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -257,15 +257,23 @@ git read-tree || die "Could not seed the index"
# map old->new commit ids for rewriting parents
mkdir ../map || die "Could not create map/ directory"
+dashdash=
+test -z "$(git rev-parse --no-revs "$@")" && dashdash=--
+ref_args=$(git rev-parse --revs-only "$@")
+
case "$filter_subdir" in
"")
- git rev-list --reverse --topo-order --default HEAD \
- --parents --simplify-merges "$@"
+ eval set -- "$(git rev-parse --sq --no-revs "$@")"
;;
*)
- git rev-list --reverse --topo-order --default HEAD \
- --parents --simplify-merges "$@" -- "$filter_subdir"
-esac > ../revs || die "Could not get the commits"
+ eval set -- "$(git rev-parse --sq --no-revs "$@")" \
+ $dashdash "$filter_subdir"
+ ;;
+esac
+
+git rev-list --reverse --topo-order --default HEAD \
+ --parents --simplify-merges $ref_args "$@" \
+ > ../revs || die "Could not get the commits"
commits=$(wc -l <../revs | tr -d " ")
test $commits -eq 0 && die "Found nothing to rewrite"
@@ -356,8 +364,7 @@ then
do
sha1=$(git rev-parse "$ref"^0)
test -f "$workdir"/../map/$sha1 && continue
- ancestor=$(git rev-list --simplify-merges -1 \
- $ref -- "$filter_subdir")
+ ancestor=$(git rev-list --simplify-merges -1 "$ref" "$@")
test "$ancestor" && echo $(map $ancestor) >> "$workdir"/../map/$sha1
done < "$tempdir"/heads
fi
--
1.6.5.1.161.g3b9c0
^ permalink raw reply related
* Re: [PATCH] Teach 'git merge' and 'git pull' the option --ff-only
From: Junio C Hamano @ 2009-10-28 22:45 UTC (permalink / raw)
To: Björn Gustavsson; +Cc: git
In-Reply-To: <4AE8C281.50104@gmail.com>
Björn Gustavsson <bgustavsson@gmail.com> writes:
> For convenience in scripts and aliases, add the option
> --ff-only to only allow fast-forwards.
>
> Acknowledgements: I did look at Yuval Kogman's earlier
> patch (107768 in gmane), mainly as shortcut to find my
> way in the code, but I did not copy anything directly.
>
> Signed-off-by: Björn Gustavsson <bgustavsson@gmail.com>
Thanks. I think you covered the points in the old discussion thread.
> diff --git a/Documentation/merge-options.txt b/Documentation/merge-options.txt
> index adadf8e..fbf8976 100644
> --- a/Documentation/merge-options.txt
> +++ b/Documentation/merge-options.txt
> @@ -60,6 +60,10 @@
> a fast-forward, only update the branch pointer. This is
> the default behavior of git-merge.
>
> +--ff-only::
> + Refuse to merge unless the merge can be resolved as a
> + fast-forward.
Do you or do you not allow "already up to date"? I think it makes sense
to allow it, but it is unclear from these two lines.
> @@ -874,6 +877,9 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
> option_commit = 0;
> }
>
> + if (!allow_fast_forward && fast_forward_only)
> + die("You cannot combine --no-ff with --ff-only.");
Are these the only nonsensical combinations? How should this interact
with other options, e.g. --squash or --message?
> @@ -969,8 +975,11 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
> }
>
> for (i = 0; i < use_strategies_nr; i++) {
> - if (use_strategies[i]->attr & NO_FAST_FORWARD)
> + if (use_strategies[i]->attr & NO_FAST_FORWARD) {
> allow_fast_forward = 0;
> + if (fast_forward_only)
> + die("You cannot combine --ff-only with the merge strategy '%s'.", use_strategies[i]->name);
> + }
I am not convinced this tests the right condition nor it is placed at the
right place in the codepath---even if a specified strategy happens to
allow fast-forward, wouldn't it be nonsense to say
$ git merge --ff-only -s resolve that-one
in the first place? Note that I am not saying "I am convinced this is
wrong."
> @@ -1040,7 +1049,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
> * only one common.
> */
> refresh_cache(REFRESH_QUIET);
> - if (allow_trivial) {
> + if (allow_trivial && !fast_forward_only) {
Good.
> @@ -1079,6 +1088,9 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
> }
> }
>
> + if (fast_forward_only)
> + die("Not possible to fast forward, aborting.");
Good.
^ permalink raw reply
* Re: Getting Ensimag students to work on Git for a few weeks
From: Jakub Narebski @ 2009-10-28 22:41 UTC (permalink / raw)
To: Clemens Buchacher; +Cc: Matthieu Moy, git
In-Reply-To: <20091027144405.GA12464@localhost>
Clemens Buchacher <drizzd@aon.at> writes:
> On Tue, Oct 27, 2009 at 11:12:52AM +0100, Matthieu Moy wrote:
>
> > The students work full-time for about 3 weeks (May 20th to June 16th),
> > and are grouped by teams of 2 to 4 students. Given my bandwidth, I
> > plan to propose only one group of 4 students this year, but we may
> > scale up later, who knows.
>
> That's not much time to get familiar with a complex project like git. So you
> will have to do something extremely simple, which probably means that it
> won't be anything exciting. If it were, someone else would have done it
> already.
Let's take a look at some projects from SoC2009Ideas and Wishlist:
* "smart" HTTP transport is being actively worked on
* narrow / sparse checkout is being worked on
* directory renames development stalled, I think last was in
http://thread.gmane.org/gmane.comp.version-control.git/99529
* git-svnserver has supposedly partial Python implementation
* restartable clone, which should be fairly easy to add to "dumb"
protocols, and quite challenging to add to "smart" protocols;
even without a code, having fresh ideas would be nice
* (optional) support for empty directories, needs index extension,
there were some patches, but the area might be muddy
It all depends on what you want to achieve in this short time. Would
it be to get to know OSS development workflow (submitting patches,
answering reviews, etc.), or would it be solving interesting real-life
problem, or perhaps solving some problem from beginning to the end
(the code being accepted).
--
Jakub Narebski
Poland
ShadeHawk on #git
^ permalink raw reply
* Re: [PATCH 0/3] fix "git diff --color-words -U0"
From: Markus Heidelberg @ 2009-10-28 22:21 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Johannes Schindelin
In-Reply-To: <7v7hufo07i.fsf@alter.siamese.dyndns.org>
Junio C Hamano, 28.10.2009:
> Is this a serious enough breakage that deserves to be fixed in the
> maintenance track (1.6.5.X)?
This problem exists since the introduction of this feature over three
years ago and apparently nobody complained so far. So I don't think it's
overly serious.
OTOH, depending on the change the produced diff can be totally wrong.
Found a good example: c5022f576aa583429c245054d8600564b788ff33
Compare "--color-words -U0" with "--color-words -U1".
^ permalink raw reply
* [PATCH] Teach 'git merge' and 'git pull' the option --ff-only
From: Björn Gustavsson @ 2009-10-28 22:15 UTC (permalink / raw)
To: git; +Cc: gitster
For convenience in scripts and aliases, add the option
--ff-only to only allow fast-forwards.
Acknowledgements: I did look at Yuval Kogman's earlier
patch (107768 in gmane), mainly as shortcut to find my
way in the code, but I did not copy anything directly.
Signed-off-by: Björn Gustavsson <bgustavsson@gmail.com>
---
When I started to use git earlier this year, I was suprised
that there was a --no-ff option but no --ff-only option.
I saw in the mailing list archive at gmane that there has
been two previous attempts to implement --ff-only. The first
patch was made to the git-merge.sh script (before
builtin-merge.c was created). As far as I understand it,
the author of the patch said he would send some corrections
of the patch, but he never did, and nothing more happened.
So here is my patch, the third attempt.
Documentation/merge-options.txt | 4 ++++
builtin-merge.c | 16 ++++++++++++++--
git-pull.sh | 7 +++++--
t/t7600-merge.sh | 29 +++++++++++++++++++++++++++++
4 files changed, 52 insertions(+), 4 deletions(-)
diff --git a/Documentation/merge-options.txt b/Documentation/merge-options.txt
index adadf8e..fbf8976 100644
--- a/Documentation/merge-options.txt
+++ b/Documentation/merge-options.txt
@@ -60,6 +60,10 @@
a fast-forward, only update the branch pointer. This is
the default behavior of git-merge.
+--ff-only::
+ Refuse to merge unless the merge can be resolved as a
+ fast-forward.
+
-s <strategy>::
--strategy=<strategy>::
Use the given merge strategy; can be supplied more than
diff --git a/builtin-merge.c b/builtin-merge.c
index b6b8428..298adfb 100644
--- a/builtin-merge.c
+++ b/builtin-merge.c
@@ -43,6 +43,7 @@ static const char * const builtin_merge_usage[] = {
static int show_diffstat = 1, option_log, squash;
static int option_commit = 1, allow_fast_forward = 1;
+static int fast_forward_only;
static int allow_trivial = 1, have_message;
static struct strbuf merge_msg;
static struct commit_list *remoteheads;
@@ -167,6 +168,8 @@ static struct option builtin_merge_options[] = {
"perform a commit if the merge succeeds (default)"),
OPT_BOOLEAN(0, "ff", &allow_fast_forward,
"allow fast forward (default)"),
+ OPT_BOOLEAN(0, "ff-only", &fast_forward_only,
+ "abort if fast forward is not possible"),
OPT_CALLBACK('s', "strategy", &use_strategies, "strategy",
"merge strategy to use", option_parse_strategy),
OPT_CALLBACK('m', "message", &merge_msg, "message",
@@ -874,6 +877,9 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
option_commit = 0;
}
+ if (!allow_fast_forward && fast_forward_only)
+ die("You cannot combine --no-ff with --ff-only.");
+
if (!argc)
usage_with_options(builtin_merge_usage,
builtin_merge_options);
@@ -969,8 +975,11 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
}
for (i = 0; i < use_strategies_nr; i++) {
- if (use_strategies[i]->attr & NO_FAST_FORWARD)
+ if (use_strategies[i]->attr & NO_FAST_FORWARD) {
allow_fast_forward = 0;
+ if (fast_forward_only)
+ die("You cannot combine --ff-only with the merge strategy '%s'.", use_strategies[i]->name);
+ }
if (use_strategies[i]->attr & NO_TRIVIAL)
allow_trivial = 0;
}
@@ -1040,7 +1049,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
* only one common.
*/
refresh_cache(REFRESH_QUIET);
- if (allow_trivial) {
+ if (allow_trivial && !fast_forward_only) {
/* See if it is really trivial. */
git_committer_info(IDENT_ERROR_ON_NO_NAME);
printf("Trying really trivial in-index merge...\n");
@@ -1079,6 +1088,9 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
}
}
+ if (fast_forward_only)
+ die("Not possible to fast forward, aborting.");
+
/* We are going to make a new commit. */
git_committer_info(IDENT_ERROR_ON_NO_NAME);
diff --git a/git-pull.sh b/git-pull.sh
index fc78592..37f3d93 100755
--- a/git-pull.sh
+++ b/git-pull.sh
@@ -16,7 +16,8 @@ cd_to_toplevel
test -z "$(git ls-files -u)" ||
die "You are in the middle of a conflicted merge."
-strategy_args= diffstat= no_commit= squash= no_ff= log_arg= verbosity=
+strategy_args= diffstat= no_commit= squash= no_ff= ff_only=
+log_arg= verbosity=
curr_branch=$(git symbolic-ref -q HEAD)
curr_branch_short=$(echo "$curr_branch" | sed "s|refs/heads/||")
rebase=$(git config --bool branch.$curr_branch_short.rebase)
@@ -45,6 +46,8 @@ do
no_ff=--ff ;;
--no-ff)
no_ff=--no-ff ;;
+ --ff-only)
+ ff_only=--ff-only ;;
-s=*|--s=*|--st=*|--str=*|--stra=*|--strat=*|--strate=*|\
--strateg=*|--strategy=*|\
-s|--s|--st|--str|--stra|--strat|--strate|--strateg|--strategy)
@@ -215,5 +218,5 @@ merge_name=$(git fmt-merge-msg $log_arg <"$GIT_DIR/FETCH_HEAD") || exit
test true = "$rebase" &&
exec git-rebase $diffstat $strategy_args --onto $merge_head \
${oldremoteref:-$merge_head}
-exec git-merge $diffstat $no_commit $squash $no_ff $log_arg $strategy_args \
+exec git-merge $diffstat $no_commit $squash $no_ff $ff_only $log_arg $strategy_args \
"$merge_name" HEAD $merge_head $verbosity
diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
index e5b210b..d696ea9 100755
--- a/t/t7600-merge.sh
+++ b/t/t7600-merge.sh
@@ -243,6 +243,16 @@ test_expect_success 'merge c0 with c1' '
test_debug 'gitk --all'
+test_expect_success 'merge c0 with c1 with --ff-only' '
+ git reset --hard c0 &&
+ git merge --ff-only c1 &&
+ git merge --ff-only HEAD c0 c1 &&
+ verify_merge file result.1 &&
+ verify_head "$c1"
+'
+
+test_debug 'gitk --all'
+
test_expect_success 'merge c1 with c2' '
git reset --hard c1 &&
test_tick &&
@@ -263,6 +273,14 @@ test_expect_success 'merge c1 with c2 and c3' '
test_debug 'gitk --all'
+test_expect_success 'failing merges with --ff-only' '
+ git reset --hard c1 &&
+ test_tick &&
+ test_must_fail git merge --ff-only c2 &&
+ test_must_fail git merge --ff-only c3 &&
+ test_must_fail git merge --ff-only c2 c3
+'
+
test_expect_success 'merge c0 with c1 (no-commit)' '
git reset --hard c0 &&
git merge --no-commit c1 &&
@@ -432,6 +450,17 @@ test_expect_success 'combining --squash and --no-ff is refused' '
test_must_fail git merge --no-ff --squash c1
'
+test_expect_success 'combining --ff-only and --no-ff is refused' '
+ test_must_fail git merge --ff-only --no-ff c1 &&
+ test_must_fail git merge --no-ff --ff-only c1
+'
+
+test_expect_success 'combining --ff-only with certain merge strategies is refused' '
+ git reset --hard c0 &&
+ test_must_fail git merge --ff-only --strategy=ours c1 &&
+ test_must_fail git merge --ff-only --strategy=subtree c1
+'
+
test_expect_success 'merge c0 with c1 (ff overrides no-ff)' '
git reset --hard c0 &&
git config branch.master.mergeoptions "--no-ff" &&
--
1.6.5.1.69.g36942
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox