* Re: [RFC/PATCH] gitweb: linkify author/committer names with search
From: Giuseppe Bilotta @ 2009-10-13 10:26 UTC (permalink / raw)
To: git; +Cc: Giuseppe Bilotta, Stephen Boyd, Jakub Narebski
In-Reply-To: <1255328340-28449-1-git-send-email-bebarino@gmail.com>
On Monday 12 October 2009 08:19, Stephen Boyd wrote:
> The problem is I can't get it to work with UTF-8 characters. I'm not sure
> if it's my system or not, so I'm just posting here to see if others
> experience the same problem and if there's interest.
Does it work if you use CGI::escape() on the author names when filling
the searchtext?
--
Giuseppe "Oblomov" Bilotta
^ permalink raw reply
* Changing branches in supermodule does not affect submodule?
From: Peter Krefting @ 2009-10-13 10:29 UTC (permalink / raw)
To: Git Mailing List
Hi!
If I have a repository with submodules that is in a clean state, and switch
branches in the super repository, the submodules are left in the state they
were in before I switched branches (with 1.6.4, at least). Is this the
expected behaviour?
--->8--8<---
#!/bin/bash
# Create supermodule.
mkdir super-$$
cd super-$$
git init
# Create submodule with commit.
mkdir sub
cd sub
git init
echo C > c.txt
echo D > d.txt
git add c.txt d.txt
git commit -m Created.
cd ..
# Create commit and branch in supermodule.
echo A > a.txt
echo B > b.txt
git add a.txt b.txt sub
git commit -m Created.
git tag branchpoint
git checkout -b branch1
# Create branch1 in submodule and commit.
cd sub
git checkout -b branch1
echo E > c.txt
echo F > d.txt
git add c.txt d.txt
git commit -m Branched.
cd ..
# Commit to a branch in the supermodule.
git add sub
git commit -m Sub-update.
# Status should now be clean.
git status
# Create a new branch in supermodule, from the original commit.
git checkout -b branch2 branchpoint
# I now expect the submodule to be up-to-date and the state clean.
git status
--->8--8<---
--
\\// Peter - http://www.softwolves.pp.se/
^ permalink raw reply
* Re: Git 1.6.5-rc git clone unhandled exception using http protocol
From: Johannes Sixt @ 2009-10-13 10:53 UTC (permalink / raw)
To: Michael Wookey, Junio C Hamano
Cc: eduard stefan, Tay Ray Chuan, Git Mailing List, msysgit
In-Reply-To: <d2e97e800910130310wa9731a6j9b9bdd25047ade85@mail.gmail.com>
Michael Wookey schrieb:
> Using the above repository, I see the same crash with msysGit at git
> revision 1.6.5. Using windbg as the post-mortem debugger, the
> following information is captured:
>
> (a14.e8c): Access violation - code c0000005 (!!! second chance !!!)
> eax=00000000 ebx=00000000 ecx=ffffffff edx=0046cc00 esi=0046f98f edi=00000000
> eip=00420354 esp=0022fd80 ebp=0022fda8 iopl=0 nv up ei pl zr na pe nc
> cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010246
>
> ...and the faulting instruction is:
>
> git_remote_curl+0x20354:
> 00420354 f2ae repne scas byte ptr es:[edi]
>
> so, a NULL dereference. The initial disassembly of the function is this:
>
> 0:000> u 0042033C
> git_remote_curl+0x2033c:
> 0042033c 55 push ebp
> 0042033d 89e5 mov ebp,esp
> 0042033f 57 push edi
> 00420340 56 push esi
> 00420341 53 push ebx
> 00420342 83ec1c sub esp,1Ch
> 00420345 8b5d08 mov ebx,dword ptr [ebp+8]
> 00420348 8b750c mov esi,dword ptr [ebp+0Ch]
> 0042034b 31c0 xor eax,eax
> 0042034d b9ffffffff mov ecx,0FFFFFFFFh
> 00420352 89df mov edi,ebx
> 00420354 f2ae repne scas byte ptr es:[edi]
> 00420356 f7d1 not ecx
> 00420358 8d51ff lea edx,[ecx-1]
> 0042035b b9ffffffff mov ecx,0FFFFFFFFh
> 00420360 89f7 mov edi,esi
> 00420362 f2ae repne scas byte ptr es:[edi]
> 00420364 f7d1 not ecx
> 00420366 49 dec ecx
> 00420367 7466 je git_remote_curl+0x203cf (004203cf)
> 00420369 85d2 test edx,edx
> 0042036b 0f84b1000000 je git_remote_curl+0x20422 (00420422)
> 00420371 89f7 mov edi,esi
> 00420373 89de mov esi,ebx
> ...
>
> So its the first parameter that is NULL. The second parameter is:
>
> 0:000> da poi(ebp+c)
> 0046f98f "libexec/git-core"
>
> I don't know how to build msysGit so that symbols are generated so
> I've attempted to reconstruct the source code; which ends up looking
> something like the following:
>
> int some_unknown_func(char *arg1, char *arg2)
> {
> len1 = strlen(arg1) - 1; // <- crash here
> len2 = strlen(arg2);
> len3 = len2 - 1;
>
> if (len2 != 1) {
> if (!len1)
> return 0;
> for (;;) {
> x = arg1[len1 - 1];
>
> if (x != '/' && x != '\\') {
> --len1;
> --len3;
> if (arg1[len1] != arg2[len3])
> return 0;
> } else {
> ...
> }
> }
> }
> }
>
> Perhaps those more familiar with git's sources might recognise code
> that looks similar to the above sequence.
Wow, this is great work, thank you very much! The function is
strip_path_suffix(). And here is a patch that fixes the crash.
--- >8 ---
From: Johannes Sixt <j6t@kdbg.org>
Subject: [PATCH] remote-curl: add missing initialization of argv0_path
All programs, in particular also the stand-alone programs (non-builtins)
must call git_extract_argv0_path(argv[0]) in order to help builds that
derive the installation prefix at runtime, such as the MinGW build.
Without this call, the program segfaults (or raises an assertion
failure).
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
remote-curl.c | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/remote-curl.c b/remote-curl.c
index ad6a163..d8d276a 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -82,6 +82,7 @@ int main(int argc, const char **argv)
const char *url;
struct walker *walker = NULL;
+ git_extract_argv0_path(argv[0]);
setup_git_directory();
if (argc < 2) {
fprintf(stderr, "Remote needed\n");
--
1.6.5.1024.g31034.dirty
^ permalink raw reply related
* Re: [RFC PATCH v2 10/16] Git-aware CGI to provide dumb HTTP transport
From: Johannes Sixt @ 2009-10-13 10:56 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git
In-Reply-To: <1255400715-10508-11-git-send-email-spearce@spearce.org>
Shawn O. Pearce schrieb:
> +int main(int argc, char **argv)
> +{
> + char *dir = getenv("PATH_TRANSLATED");
> + char *input_method = getenv("REQUEST_METHOD");
> + struct service_cmd *cmd = NULL;
> + char *cmd_arg = NULL;
> + int i;
> +
Please insert here:
git_extract_argv0_path(argv[0]);
> + set_die_routine(die_webcgi);
> ...
-- Hannes
^ permalink raw reply
* Re: [RFC PATCH v2 14/16] Smart push over HTTP: client side
From: Felipe Contreras @ 2009-10-13 11:02 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git, Daniel Barkalow
In-Reply-To: <1255400715-10508-15-git-send-email-spearce@spearce.org>
On Tue, Oct 13, 2009 at 5:25 AM, Shawn O. Pearce <spearce@spearce.org> wrote:
> The git-remote-curl backend detects if the remote server supports
> the git-receive-pack service, and if so, runs git-send-pack in a
> pipe to dump the command and pack data as a single POST request.
>
> The advertisements from the server that were obtained during the
> discovery are passed into git-send-pack before the POST request
> starts. This permits git-send-pack to operate largely unmodified.
>
> For smaller packs (those under 1 MiB) a tranditional POST with a
Traditional?
> Content-Length is used, permitting interaction with any HTTP/1.0
> compliant server. The 1 MiB limit is arbitrary, but is sufficent
> to fit most deltas created by human authors against text sources
> with the occasional small binary file (e.g. few KiB icon image).
>
> For larger packs which cannot be spooled entirely into the
> helper's memory space, the POST request requires HTTP/1.1 and
> Transfer-Encoding: chunked. This permits the client to upload an
> unknown amount of data in one HTTP transaction without needing to
> pregenerate the entire pack file locally.
>
> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
> CC: Daniel Barkalow <barkalow@iabervon.org>
> ---
> Documentation/config.txt | 8 ++
> builtin-send-pack.c | 116 ++++++++++++++++++++-
> remote-curl.c | 258 +++++++++++++++++++++++++++++++++++++++++++++-
> send-pack.h | 3 +-
> sideband.c | 11 ++-
> transport.c | 1 +
> 6 files changed, 384 insertions(+), 13 deletions(-)
>
> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index cd17814..7130d07 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -1089,6 +1089,14 @@ http.maxRequests::
> How many HTTP requests to launch in parallel. Can be overridden
> by the 'GIT_HTTP_MAX_REQUESTS' environment variable. Default is 5.
>
> +http.postBuffer::
> + Maximum size in bytes of the buffer used by smart HTTP
> + transports when POSTing data to the remote system.
> + For requests larger than this buffer size, HTTP/1.1 and
> + Transfer-Encoding: chunked is used to avoid creating a
> + massive pack file locally. Default is 1 MiB, which is
> + sufficient for most requests.
> +
> http.lowSpeedLimit, http.lowSpeedTime::
> If the HTTP transfer speed is less than 'http.lowSpeedLimit'
> for longer than 'http.lowSpeedTime' seconds, the transfer is aborted.
> diff --git a/builtin-send-pack.c b/builtin-send-pack.c
> index 37e528e..654c3d4 100644
> --- a/builtin-send-pack.c
> +++ b/builtin-send-pack.c
> @@ -2,9 +2,11 @@
> #include "commit.h"
> #include "refs.h"
> #include "pkt-line.h"
> +#include "sideband.h"
> #include "run-command.h"
> #include "remote.h"
> #include "send-pack.h"
> +#include "quote.h"
>
> static const char send_pack_usage[] =
> "git send-pack [--all | --mirror] [--dry-run] [--force] [--receive-pack=<git-receive-pack>] [--verbose] [--thin] [<host>:]<directory> [<ref>...]\n"
> @@ -59,7 +61,7 @@ static int pack_objects(int fd, struct ref *refs, struct extra_have_objects *ext
> memset(&po, 0, sizeof(po));
> po.argv = argv;
> po.in = -1;
> - po.out = fd;
> + po.out = args->one_shot_rpc ? -1 : fd;
> po.git_cmd = 1;
> if (start_command(&po))
> die_errno("git pack-objects failed");
> @@ -83,6 +85,20 @@ static int pack_objects(int fd, struct ref *refs, struct extra_have_objects *ext
> }
>
> close(po.in);
> +
> + if (args->one_shot_rpc) {
> + char *buf = xmalloc(LARGE_PACKET_MAX);
> + while (1) {
> + ssize_t n = xread(po.out, buf, LARGE_PACKET_MAX);
> + if (n <= 0)
> + break;
> + send_sideband(fd, -1, buf, n, LARGE_PACKET_MAX);
> + }
> + free(buf);
> + close(po.out);
> + po.out = -1;
> + }
> +
> if (finish_command(&po))
> return error("pack-objects died with strange error");
> return 0;
> @@ -303,6 +319,59 @@ static int refs_pushed(struct ref *ref)
> return 0;
> }
>
> +static void print_helper_status(struct ref *ref)
> +{
> + struct strbuf buf = STRBUF_INIT;
> +
> + for (; ref; ref = ref->next) {
> + const char *msg = NULL;
> + const char *res;
> +
> + switch(ref->status) {
> + case REF_STATUS_NONE:
> + res = "error";
> + msg = "no match";
> + break;
> +
> + case REF_STATUS_OK:
> + res = "ok";
> + break;
> +
> + case REF_STATUS_UPTODATE:
> + res = "ok";
> + msg = "up to date";
> + break;
> +
> + case REF_STATUS_REJECT_NONFASTFORWARD:
> + res = "error";
> + msg = "non-fast forward";
> + break;
> +
> + case REF_STATUS_REJECT_NODELETE:
> + case REF_STATUS_REMOTE_REJECT:
> + res = "error";
> + break;
> +
> + case REF_STATUS_EXPECTING_REPORT:
> + default:
> + continue;
> + }
> +
> + strbuf_reset(&buf);
> + strbuf_addf(&buf, "%s %s", res, ref->name);
> + if (ref->remote_status)
> + msg = ref->remote_status;
> + if (msg) {
> + strbuf_addch(&buf, ' ');
> + quote_two_c_style(&buf, "", msg, 0);
> + }
> + strbuf_addch(&buf, '\n');
> +
> + safe_write(1, buf.buf, buf.len);
> + }
> + strbuf_release(&buf);
> +}
> +
> int send_pack(struct send_pack_args *args,
> int fd[], struct child_process *conn,
> struct ref *remote_refs,
> @@ -310,6 +379,7 @@ int send_pack(struct send_pack_args *args,
> {
> int in = fd[0];
> int out = fd[1];
> + struct strbuf req_buf = STRBUF_INIT;
> struct ref *ref;
> int new_refs;
> int ask_for_status_report = 0;
> @@ -391,14 +461,14 @@ int send_pack(struct send_pack_args *args,
> char *new_hex = sha1_to_hex(ref->new_sha1);
>
> if (ask_for_status_report) {
> - packet_write(out, "%s %s %s%c%s",
> + packet_buf_write(&req_buf, "%s %s %s%c%s",
> old_hex, new_hex, ref->name, 0,
> "report-status");
> ask_for_status_report = 0;
> expect_status_report = 1;
> }
> else
> - packet_write(out, "%s %s %s",
> + packet_buf_write(&req_buf, "%s %s %s",
> old_hex, new_hex, ref->name);
> }
> ref->status = expect_status_report ?
> @@ -406,7 +476,17 @@ int send_pack(struct send_pack_args *args,
> REF_STATUS_OK;
> }
>
> - packet_flush(out);
> + if (args->one_shot_rpc) {
> + if (!args->dry_run) {
> + packet_buf_flush(&req_buf);
> + send_sideband(out, -1, req_buf.buf, req_buf.len, LARGE_PACKET_MAX);
> + }
> + } else {
> + safe_write(out, req_buf.buf, req_buf.len);
> + packet_flush(out);
> + }
> + strbuf_release(&req_buf);
> +
> if (new_refs && !args->dry_run) {
> if (pack_objects(out, remote_refs, extra_have, args) < 0) {
> for (ref = remote_refs; ref; ref = ref->next)
> @@ -414,11 +494,15 @@ int send_pack(struct send_pack_args *args,
> return -1;
> }
> }
> + if (args->one_shot_rpc && !args->dry_run)
> + packet_flush(out);
>
> if (expect_status_report)
> ret = receive_status(in, remote_refs);
> else
> ret = 0;
> + if (args->one_shot_rpc)
> + packet_flush(out);
>
> if (ret < 0)
> return ret;
> @@ -478,6 +562,7 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)
> struct extra_have_objects extra_have;
> struct ref *remote_refs, *local_refs;
> int ret;
> + int helper_status = 0;
> int send_all = 0;
> const char *receivepack = "git-receive-pack";
> int flags;
> @@ -523,6 +608,14 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)
> args.use_thin_pack = 1;
> continue;
> }
> + if (!strcmp(arg, "--one-shot-rpc")) {
> + args.one_shot_rpc = 1;
> + continue;
> + }
> + if (!strcmp(arg, "--helper-status")) {
> + helper_status = 1;
> + continue;
> + }
> usage(send_pack_usage);
> }
> if (!dest) {
> @@ -551,7 +644,14 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)
> }
> }
>
> - conn = git_connect(fd, dest, receivepack, args.verbose ? CONNECT_VERBOSE : 0);
> + if (args.one_shot_rpc) {
> + conn = NULL;
> + fd[0] = 0;
> + fd[1] = 1;
> + } else {
> + conn = git_connect(fd, dest, receivepack,
> + args.verbose ? CONNECT_VERBOSE : 0);
> + }
>
> memset(&extra_have, 0, sizeof(extra_have));
>
> @@ -575,12 +675,16 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)
>
> ret = send_pack(&args, fd, conn, remote_refs, &extra_have);
>
> + if (helper_status)
> + print_helper_status(remote_refs);
> +
> close(fd[1]);
> close(fd[0]);
>
> ret |= finish_connect(conn);
>
> - print_push_status(dest, remote_refs);
> + if (!helper_status)
> + print_push_status(dest, remote_refs);
>
> if (!args.dry_run && remote) {
> struct ref *ref;
> diff --git a/remote-curl.c b/remote-curl.c
> index 42fd06c..529df42 100644
> --- a/remote-curl.c
> +++ b/remote-curl.c
> @@ -5,17 +5,33 @@
> #include "walker.h"
> #include "http.h"
> #include "pkt-line.h"
> +#include "sideband.h"
> #include "run-command.h"
>
>
> +static size_t post_buffer_size = 16 * LARGE_PACKET_MAX;
> static struct remote *remote;
> static const char *url;
> static struct walker *walker;
>
> +static int http_options(const char *var, const char *value, void *cb)
> +{
> + if (!strcmp("http.postbuffer", var)) {
> + post_buffer_size = git_config_int(var, value);
> + if (post_buffer_size < LARGE_PACKET_MAX)
> + post_buffer_size = LARGE_PACKET_MAX;
> + return 0;
> + }
> +
> + return 0;
> +}
> +
> static void init_walker(void)
> {
> - if (!walker)
> + if (!walker) {
> walker = get_http_walker(url, remote);
> + git_config(http_options, NULL);
> + }
> }
>
> struct discovery {
> @@ -200,6 +216,193 @@ static void output_refs(struct ref *refs)
> free_refs(refs);
> }
>
> +struct rpc_state {
> + const char *service_name;
> + char *service_url;
> + char *hdr_content_type;
> + char *hdr_accept;
> + char *buf;
> + size_t alloc;
> + size_t len;
> + size_t pos;
> + int in;
> + int out;
> + unsigned verbose : 1;
> +};
> +
> +static size_t rpc_out(void *ptr, size_t eltsize,
> + size_t nmemb, void *buffer_)
> +{
> + size_t max = eltsize * nmemb;
> + struct rpc_state *state = buffer_;
> + size_t avail = state->len - state->pos;
> +
> + if (!avail) {
> + avail = packet_read_line(state->out, state->buf, state->alloc);
> + if (!avail)
> + return 0;
> + state->pos = 0;
> + state->len = avail;
> + }
> +
> + if (max < avail);
> + avail = max;
> + memcpy(ptr, state->buf + state->pos, avail);
> + state->pos += avail;
> + return avail;
> +}
> +
> +static size_t rpc_in(const void *ptr, size_t eltsize,
> + size_t nmemb, void *buffer_)
> +{
> + size_t size = eltsize * nmemb;
> + struct rpc_state *state = buffer_;
> + write_or_die(state->in, ptr, size);
> + return size;
> +}
> +
> +static int post_rpc(struct rpc_state *state)
> +{
> + struct active_request_slot *slot;
> + struct slot_results results;
> + struct curl_slist *headers = NULL;
> + int err = 0, large_request = 0;
> +
> + /* Try to load the entire request, if we can fit it into the
> + * allocated buffer space we can use HTTP/1.0 and avoid the
> + * chunked encoding mess.
> + */
> + while (1) {
> + size_t left = state->alloc - state->len;
> + char *buf = state->buf + state->len;
> + int n;
> +
> + if (left < LARGE_PACKET_MAX) {
> + large_request = 1;
> + break;
> + }
> +
> + n = packet_read_line(state->out, buf, left);
> + if (!n)
> + break;
> + state->len += n;
> + }
> +
> + slot = get_active_slot();
> + slot->results = &results;
> +
> + curl_easy_setopt(slot->curl, CURLOPT_POST, 1);
> + curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0);
> + curl_easy_setopt(slot->curl, CURLOPT_URL, state->service_url);
> +
> + headers = curl_slist_append(headers, state->hdr_content_type);
> + headers = curl_slist_append(headers, state->hdr_accept);
> +
> + if (large_request) {
> + /* The request body is large and the size cannot be predicted.
> + * We must use chunked encoding to send it.
> + */
> + headers = curl_slist_append(headers, "Expect: 100-continue");
> + headers = curl_slist_append(headers, "Transfer-Encoding: chunked");
> + curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, rpc_out);
> + curl_easy_setopt(slot->curl, CURLOPT_INFILE, state);
> + if (state->verbose) {
> + fprintf(stderr, "POST %s (chunked)\n", state->service_name);
> + fflush(stderr);
> + }
> +
> + } else {
> + /* We know the complete request size in advance, use the
> + * more normal Content-Length approach.
> + */
> + curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDS, state->buf);
> + curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDSIZE, state->len);
> + if (state->verbose) {
> + fprintf(stderr, "POST %s (%lu bytes)\n",
> + state->service_name, (unsigned long)state->len);
> + fflush(stderr);
> + }
> + }
> +
> + curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, headers);
> + curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, rpc_in);
> + curl_easy_setopt(slot->curl, CURLOPT_FILE, state);
> +
> + if (start_active_slot(slot)) {
> + run_active_slot(slot);
> + if (results.curl_result != CURLE_OK) {
> + err |= error("RPC failed; result=%d, HTTP code = %ld",
> + results.curl_result, results.http_code);
> + }
> + }
> + curl_slist_free_all(headers);
> + return err;
> +}
> +
> +static int one_shot_rpc_service(const char *service,
> + int verbose,
> + const char **client_argv,
> + struct discovery *heads,
> + struct strbuf *result)
> +{
> + struct child_process client;
> + struct rpc_state state;
> + struct strbuf buf = STRBUF_INIT;
> + int err = 0;
> +
> + init_walker();
> + memset(&client, 0, sizeof(client));
> + client.in = -1;
> + client.out = -1;
> + client.git_cmd = 1;
> + client.argv = client_argv;
> + if (start_command(&client))
> + die("%s failed to execute", client_argv[0]);
> + if (heads)
> + write_or_die(client.in, heads->buf, heads->len);
> +
> + memset(&state, 0, sizeof(state));
> + state.alloc = post_buffer_size;
> + state.buf = xmalloc(state.alloc);
> + state.in = client.in;
> + state.out = client.out;
> + state.service_name = service;
> + state.verbose = !!verbose;
> +
> + strbuf_addf(&buf, "%s/%s", url, service);
> + state.service_url = strbuf_detach(&buf, NULL);
> +
> + strbuf_addf(&buf, "Content-Type: application/x-%s-request", service);
> + state.hdr_content_type = strbuf_detach(&buf, NULL);
> +
> + strbuf_addf(&buf, "Accept: application/x-%s-response", service);
> + state.hdr_accept = strbuf_detach(&buf, NULL);
> +
> + while (!err) {
> + int n = packet_read_line(state.out, state.buf, state.alloc);
> + if (!n)
> + break;
> + state.pos = 0;
> + state.len = n;
> + err |= post_rpc(&state);
> + }
> + if (result)
> + strbuf_read(result, client.out, 0);
> +
> + close(client.in);
> + close(client.out);
> + client.in = -1;
> + client.out = -1;
> +
> + err |= finish_command(&client);
> + free(state.service_url);
> + free(state.hdr_content_type);
> + free(state.hdr_accept);
> + free(state.buf);
> + strbuf_release(&buf);
> + return err;
> +}
> +
> struct fetch_args {
> unsigned verbose : 1;
> };
> @@ -289,7 +492,8 @@ static void parse_fetch(struct strbuf *buf, int multiple)
>
> struct push_args {
> unsigned dry_run : 1,
> - verbose : 1;
> + verbose : 1,
> + thin : 1;
> };
>
> static int push_dav(struct push_args *args, int nr_spec, char **specs)
> @@ -314,6 +518,51 @@ static int push_dav(struct push_args *args, int nr_spec, char **specs)
> return 0;
> }
>
> +static int push_git(struct discovery *heads,
> + struct push_args *args, int nr_spec, char **specs)
> +{
> + struct strbuf res = STRBUF_INIT;
> + const char **argv;
> + int argc = 0, i, err;
> +
> + argv = xmalloc((10 + nr_spec) * sizeof(char*));
> + argv[argc++] = "send-pack";
> + argv[argc++] = "--one-shot-rpc";
> + argv[argc++] = "--helper-status";
> + if (args->thin)
> + argv[argc++] = "--thin";
> + if (args->dry_run)
> + argv[argc++] = "--dry-run";
> + if (args->verbose)
> + argv[argc++] = "--verbose";
> + argv[argc++] = url;
> + for (i = 0; i < nr_spec; i++)
> + argv[argc++] = specs[i];
> + argv[argc++] = NULL;
> +
> + err = one_shot_rpc_service("git-receive-pack",
> + args->verbose,
> + argv, heads, &res);
> + if (res.len)
> + safe_write(1, res.buf, res.len);
> + strbuf_release(&res);
> + free(argv);
> + return err;
> +}
> +
> +static int push(struct push_args *args, int nr_spec, char **specs)
> +{
> + struct discovery *heads = discover_refs("git-receive-pack");
> + int ret;
> +
> + if (heads->proto_git)
> + ret = push_git(heads, args, nr_spec, specs);
> + else
> + ret = push_dav(args, nr_spec, specs);
> + free_discovery(heads);
> + return ret;
> +}
> +
> static void parse_push(struct strbuf *buf)
> {
> char **specs = NULL;
> @@ -330,6 +579,8 @@ static void parse_push(struct strbuf *buf)
> args.dry_run = 1;
> else if (!strcmp(buf->buf, "option verbose"))
> args.verbose = 1;
> + else if (!strcmp(buf->buf, "option thin"))
> + args.thin = 1;
> else
> die("http transport does not support %s", buf->buf);
>
> @@ -340,7 +591,7 @@ static void parse_push(struct strbuf *buf)
> break;
> } while (1);
>
> - if (push_dav(&args, nr_spec, specs))
> + if (push(&args, nr_spec, specs))
> exit(128); /* error already reported */
> for (i = 0; i < nr_spec; i++)
> free(specs[i]);
> @@ -395,6 +646,7 @@ int main(int argc, const char **argv)
> printf("fetch-multiple\n");
> printf("push\n");
> printf("option dry-run\n");
> + printf("option thin\n");
> printf("option verbose\n");
> printf("\n");
> fflush(stdout);
> diff --git a/send-pack.h b/send-pack.h
> index 8b3cf02..a7f4abf 100644
> --- a/send-pack.h
> +++ b/send-pack.h
> @@ -8,7 +8,8 @@ struct send_pack_args {
> force_update:1,
> use_thin_pack:1,
> use_ofs_delta:1,
> - dry_run:1;
> + dry_run:1,
> + one_shot_rpc:1;
> };
>
> int send_pack(struct send_pack_args *args,
> diff --git a/sideband.c b/sideband.c
> index 899b1ff..d5ffa1c 100644
> --- a/sideband.c
> +++ b/sideband.c
> @@ -135,9 +135,14 @@ ssize_t send_sideband(int fd, int band, const char *data, ssize_t sz, int packet
> n = sz;
> if (packet_max - 5 < n)
> n = packet_max - 5;
> - sprintf(hdr, "%04x", n + 5);
> - hdr[4] = band;
> - safe_write(fd, hdr, 5);
> + if (0 <= band) {
> + sprintf(hdr, "%04x", n + 5);
> + hdr[4] = band;
> + safe_write(fd, hdr, 5);
> + } else {
> + sprintf(hdr, "%04x", n + 4);
> + safe_write(fd, hdr, 4);
> + }
> safe_write(fd, p, n);
> p += n;
> sz -= n;
> diff --git a/transport.c b/transport.c
> index 6d9652d..2ff1650 100644
> --- a/transport.c
> +++ b/transport.c
> @@ -731,6 +731,7 @@ static int git_transport_push(struct transport *transport, struct ref *remote_re
> NULL);
> }
>
> + memset(&args, 0, sizeof(args));
> args.send_mirror = !!(flags & TRANSPORT_PUSH_MIRROR);
> args.force_update = !!(flags & TRANSPORT_PUSH_FORCE);
> args.use_thin_pack = data->thin;
> --
> 1.6.5.52.g0ff2e
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
Felipe Contreras
^ permalink raw reply
* [PATCH] NO_PERL_MAKEMAKER should behave more like ExtUtils::MakeMaker
From: Tom G. Christensen @ 2009-10-13 11:14 UTC (permalink / raw)
To: git
This change makes NO_PERL_MAKEMAKER behave more as ExtUtils::MakeMaker
by installing the modules into the perl libdir and not $(prefix)/lib.
It will default to sitelib but will allow the user to choose by setting
the INSTALLDIRS variable which is also honored by ExtUtils::MakeMaker.
Signed-off-by: Tom G. Christensen <tgc@statsbiblioteket.dk>
---
perl/Makefile | 7 ++++++-
1 files changed, 6 insertions(+), 1 deletions(-)
diff --git a/perl/Makefile b/perl/Makefile
index 4ab21d6..264cfc1 100644
--- a/perl/Makefile
+++ b/perl/Makefile
@@ -20,7 +20,12 @@ clean:
$(RM) $(makfile).old
ifdef NO_PERL_MAKEMAKER
-instdir_SQ = $(subst ','\'',$(prefix)/lib)
+perllib = site
+ifdef INSTALLDIRS
+perllib = $(INSTALLDIRS)
+endif
+perlinstdir := $(shell sh -c "$(PERL_PATH_SQ) -V:install$(perllib)lib | cut -d\' -f2")
+instdir_SQ = $(subst ','\'',$(perlinstdir))
$(makfile): ../GIT-CFLAGS Makefile
echo all: private-Error.pm Git.pm > $@
echo ' mkdir -p blib/lib' >> $@
--
1.6.4.4
^ permalink raw reply related
* Re: Git 1.6.5-rc git clone unhandled exception using http protocol
From: Michael Wookey @ 2009-10-13 11:43 UTC (permalink / raw)
To: Johannes Sixt
Cc: Junio C Hamano, eduard stefan, Tay Ray Chuan, Git Mailing List,
msysgit
In-Reply-To: <4AD45C28.4080501@viscovery.net>
2009/10/13 Johannes Sixt <j.sixt@viscovery.net>:
> Michael Wookey schrieb:
>> Using the above repository, I see the same crash with msysGit at git
>> revision 1.6.5. Using windbg as the post-mortem debugger, the
>> following information is captured:
>> [ ... snip ... ]
> Wow, this is great work, thank you very much! The function is
> strip_path_suffix(). And here is a patch that fixes the crash.
>
> --- >8 ---
> From: Johannes Sixt <j6t@kdbg.org>
> Subject: [PATCH] remote-curl: add missing initialization of argv0_path
>
> All programs, in particular also the stand-alone programs (non-builtins)
> must call git_extract_argv0_path(argv[0]) in order to help builds that
> derive the installation prefix at runtime, such as the MinGW build.
> Without this call, the program segfaults (or raises an assertion
> failure).
>
> Signed-off-by: Johannes Sixt <j6t@kdbg.org>
> ---
> remote-curl.c | 1 +
> 1 files changed, 1 insertions(+), 0 deletions(-)
>
> diff --git a/remote-curl.c b/remote-curl.c
> index ad6a163..d8d276a 100644
> --- a/remote-curl.c
> +++ b/remote-curl.c
> @@ -82,6 +82,7 @@ int main(int argc, const char **argv)
> const char *url;
> struct walker *walker = NULL;
>
> + git_extract_argv0_path(argv[0]);
> setup_git_directory();
> if (argc < 2) {
> fprintf(stderr, "Remote needed\n");
> --
> 1.6.5.1024.g31034.dirty
No more crashes for me :)
Tested-by: Michael Wookey <michaelwookey@gmail.com>
^ permalink raw reply
* Re: gitweb atom feeds broken (on repo.or.cz only?)
From: Sebastian Pipping @ 2009-10-13 12:14 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <4AC0CBBE.7020603@hartwork.org>
Petr, any news on this one?
I just check and it still seems broken.
Sebastian
^ permalink raw reply
* Re: Changing branches in supermodule does not affect submodule?
From: Jens Lehmann @ 2009-10-13 13:11 UTC (permalink / raw)
To: Peter Krefting; +Cc: Git Mailing List
In-Reply-To: <alpine.DEB.2.00.0910131115160.14223@ds9.cixit.se>
Peter Krefting schrieb:
> If I have a repository with submodules that is in a clean state, and
> switch branches in the super repository, the submodules are left in the
> state they were in before I switched branches (with 1.6.4, at least). Is
> this the expected behaviour?
Yup, submodules have to be updated separatly.
When they are cloned from a remote with "git submodule add" followed by
a "git submodule init", just calling "git submodule update" every time
you want the submodule to be updated according to the state committed
in the superproject will do the trick (but keep in mind that all
referenced commits have to be accessible in the local clone of your
submodule, so you might have to do a fetch there once in a while).
Jens
^ permalink raw reply
* Re: quote in help code example
From: bill lam @ 2009-10-13 14:06 UTC (permalink / raw)
To: Miklos Vajna; +Cc: git
In-Reply-To: <20091013101916.GV23777@genesis.frugalware.org>
On Tue, 13 Oct 2009, Miklos Vajna wrote:
> On Tue, Oct 13, 2009 at 10:16:17AM +0800, bill lam <cbill.lam@gmail.com> wrote:
> > > Just a guess: do you have docbook-xsl >=1.73.0 and you did not set
> > > ASCIIDOC_NO_ROFF?
> > >
> > > Try rebuilding the documentation using 'make ASCIIDOC_NO_ROFF=YesPlease'.
> >
> > I'm not familiar with how to twist git makefile. By adding a line to ./Makefile
> >
> > # Platform specific tweaks
> > #
> >
> > # We choose to avoid "if .. else if .. else .. endif endif"
> > # because maintaining the nesting to match is a pain. If
> > # we had "elif" things would have been much nicer...
> >
> > ASCIIDOC_NO_ROFF = YesPlease # <--- this line added
> > ifeq ($(uname_S),Linux)
> >
> > However, the man page still display the same
>
> Don't edit the Makefile, just use the command 'make
> ASCIIDOC_NO_ROFF=YesPlease'. Also make sure to do a 'make clean' in the
> Documentation dir to get the manpages rebuilt.
I run these commands
make ASCIIDOC_NO_ROFF=YesPlease prefix=/usr all doc info
sudo make ASCIIDOC_NO_ROFF=YesPlease prefix=/usr install install-doc install-html install-info
1. did I need to set ASCIIDOC_NO_ROFF in both lines?
2. now the .ft pair fixed but it still displayed incorrect quote.
git filter-branch --tree-filter ´rm filename´ HEAD
it should be 'rm filename' not ´rm filename´
--
regards,
====================================================
GPG key 1024D/4434BAB3 2008-08-24
gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3
^ permalink raw reply
* Re: [PATCH] bisect reset: Allow resetting to any commit, not just a branch
From: Anders Kaseorg @ 2009-10-13 14:29 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johannes Sixt, git
In-Reply-To: <7vtyy3zo9m.fsf@alter.siamese.dyndns.org>
On Tue, 13 Oct 2009, Junio C Hamano wrote:
> I agree that "git bisect reset-and-detach-at-the-first-bad-one" would make
> a lot of sense.
Under my patch, that’s ‘git bisect reset bisect/bad’. Similar arguments
may be made as to whether that should or shouldn’t be a separate command
(although it’s less clear what to call it, in this case).
Anders
^ permalink raw reply
* Re: [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation
From: Shawn O. Pearce @ 2009-10-13 15:15 UTC (permalink / raw)
To: Imran M Yousuf; +Cc: git, egit-dev, Imran M Yousuf
In-Reply-To: <7bfdc29a0910121830y4dc9b3efpe17860e04457988d@mail.gmail.com>
Imran M Yousuf <imyousuf@gmail.com> wrote:
> Firstly, I am sorry but I am not intelligent enough to perceive, how
> do the user decide which instance of Config to use? I personally think
> that there is no API to achieve what you just mentioned :(; i.e. the
> user will have know CassandraConfig directly.
Yes. Well, almost.
The user will have to know that s/he wants a CassandraRepository or
a JdbcRepository in order to obtain the abstract Repository handle.
Each of these will need different configuration, possibly data which
is too complex to simply cram into a URL string, so I was expecting
the application would construct the concrete Repository class and
configure it with the proper arguments required for contact with
the underlying storage.
Since the Repository wants several things associated with it, each
concrete Repository class knows what concrete Config, ObjectDatabase
and RefDatabase it should create. Those concrete classes know how
to read a repository stored on that medium.
> Secondly, I instead was
> thinking of porting JGit for that matter to any system supporting
> streams (not any specific sub-class of them), such HBase/BigTable or
> HDFS anything.... Thirdly, I think we actually have several task in
> hand and I would state them as -
>
> 1. First introduce the I/O API such that it completely replaces java.io.File
> 2. Secondly segregate persistence of for config (or config like
> objects) and introduce a SPI for them for smarter storage.
Supporting streams on an arbitrary backend is difficult. DHTs like
BigTable/Cassandra aren't very good at providing streams, they tend
to have a limit on how big a row can be. They tend to have very
slow read latencies, but can return a small block of consecutive
rows in one reply.
I want to talk about the DHT backend more with Scott Chacon at the
GitTogether, but I have this feeling that just laying a pack file
into a stream in a DHT is going to perform very poorly.
Likewise JDBC has similar performance problems, you can only store
so much in a row before performance of the RDBMS drops off sharply.
You can get a handful of rows in a single reply pretty efficiently,
but each query takes longer than you'd like. Yes, there is often
a BLOB type that allows large file storage, but different RDBMS
support these differently and have different performance when it
comes to accessing the BLOB types. Some don't support random access,
some do. Even if they do support random access read, writing a large
2 GiB repository's pack file after repacking it would take ages.
Once you get outside of the pack file, *everything* else git stores
is either a loose object, or very tiny text files (aka refs, their
logs, config). The loose object case should be handled by the same
thing that handles the bulk of the object store, loose objects are
a trivial thing compared to packed objects.
The refs, the ref logs, and the config are all structured text.
If you lay a Git repository down into a database of some sort,
I think its reasonable to expect that the schema for these items
in that database permits query and update using relatively native
primitives in that database. E.g. if you put this in SQL I would
expect a schema like:
CREATE TABLE refs (
repository_id INT NOT NULL
,name VARCHAR(255) NOT NULL
,id CHAR(40)
,target VARCHAR(255)
,PRIMARY KEY (repository_id, name)
,CHECK (id IS NOT NULL OR target IS NOT NULL));
CREATE TABLE reflogs (
repository_id INT NOT NULL
,name VARCHAR(255) NOT NULL
,old_id CHAR(40) NOT NULL
,new_id CHAR(40) NOT NULL
,committer_name VARCHAR(255)
,committer_email VARCHAR(255)
,committer_date TIMESTAMP NOT NULL
,message VARCHAR(255)
,PRIMARY KEY (repository_id, name, committer_date));
CREATE TABLE config (
repository_id INT NOT NULL
,section VARCHAR(255) NOT NULL
,group VARCHAR(255)
,name VARCHAR(255) NOT NULL
,value VARCHAR(255)
,PRIMARY KEY(repository_id, section, group, name, value))
This makes it easier to manipulate settings, you can use direct
SQL UPDATE to modify the configuration, or SELECT to scan through
reflogs. Etc.
If we just threw everything as streams into the database this
would be a lot more difficult to work with through the database's
own native query and update interface. You'd lose alot of the
benefits of using a database, but still be paying the massive price
in performance.
> I am not thinking of storing "only" the bare content of a git
> repository, but I intent to be able to also store the versioned
> contents it self as well.
When I say "bare repository" I only mean a repository without a
working directory. It still holds the complete revision history.
If you wanted a git repository on Cassandra but wanted to actually
have a working directory checkout, you'd need the local filesystem
for the checkout and .git/index, but could otherwise keep the objects
and refs in Cassandra. Its nuts... but in theory one could do it.
--
Shawn.
^ permalink raw reply
* [PATCH v2] bisect reset: Allow resetting to any commit, not just a branch
From: Anders Kaseorg @ 2009-10-13 15:22 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vaazw6uyi.fsf@alter.siamese.dyndns.org>
‘git bisect reset’ accepts an optional argument specifying a branch to
check out after cleaning up the bisection state. This lets you
specify an arbitrary commit.
In particular, this provides a way to clean the bisection state
without moving HEAD: ‘git bisect reset HEAD’. This may be useful if
you are not interested in the state before you began a bisect,
especially if checking out the old commit would be expensive and
invalidate most of your compiled tree.
Clarify the ‘git bisect reset’ documentation to explain this optional
argument, which was previously mentioned only in the usage message.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
---
Documentation/git-bisect.txt | 23 +++++++++++++++++------
git-bisect.sh | 7 +++----
2 files changed, 20 insertions(+), 10 deletions(-)
diff --git a/Documentation/git-bisect.txt b/Documentation/git-bisect.txt
index 63e7a42..d2ffae0 100644
--- a/Documentation/git-bisect.txt
+++ b/Documentation/git-bisect.txt
@@ -20,7 +20,7 @@ on the subcommand:
git bisect bad [<rev>]
git bisect good [<rev>...]
git bisect skip [(<rev>|<range>)...]
- git bisect reset [<branch>]
+ git bisect reset [<commit>]
git bisect visualize
git bisect replay <logfile>
git bisect log
@@ -81,16 +81,27 @@ will have been left with the first bad kernel revision in "refs/bisect/bad".
Bisect reset
~~~~~~~~~~~~
-To return to the original head after a bisect session, issue the
-following command:
+After a bisect session, to clean up the bisection state and return to
+the original HEAD, issue the following command:
------------------------------------------------
$ git bisect reset
------------------------------------------------
-This resets the tree to the original branch instead of being on the
-bisection commit ("git bisect start" will also do that, as it resets
-the bisection state).
+By default, this will return your tree to the commit that was checked
+out before `git bisect start`. (A new `git bisect start` will also do
+that, as it cleans up the old bisection state.)
+
+With an optional argument, you can return to a different commit
+instead:
+
+------------------------------------------------
+$ git bisect reset <commit>
+------------------------------------------------
+
+For example, `git bisect reset HEAD` will leave you on the current
+bisection commit and avoid switching commits at all, while `git bisect
+reset bisect/bad` will check out the first bad revision.
Bisect visualize
~~~~~~~~~~~~~~~~
diff --git a/git-bisect.sh b/git-bisect.sh
index 6f6f039..0c56c26 100755
--- a/git-bisect.sh
+++ b/git-bisect.sh
@@ -13,8 +13,8 @@ git bisect skip [(<rev>|<range>)...]
mark <rev>... untestable revisions.
git bisect next
find next bisection to test and check it out.
-git bisect reset [<branch>]
- finish bisection search and go back to branch.
+git bisect reset [<commit>]
+ finish bisection search and go back to commit.
git bisect visualize
show bisect status in gitk.
git bisect replay <logfile>
@@ -311,8 +311,7 @@ bisect_reset() {
}
case "$#" in
0) branch=$(cat "$GIT_DIR/BISECT_START") ;;
- 1) git show-ref --verify --quiet -- "refs/heads/$1" ||
- die "$1 does not seem to be a valid branch"
+ 1) git rev-parse --verify "$1^{commit}" || exit
branch="$1" ;;
*)
usage ;;
--
1.6.5
^ permalink raw reply related
* Re: quote in help code example
From: Miklos Vajna @ 2009-10-13 15:30 UTC (permalink / raw)
To: bill lam; +Cc: git
In-Reply-To: <20091013140622.GA3927@debian.b2j>
[-- Attachment #1: Type: text/plain, Size: 759 bytes --]
On Tue, Oct 13, 2009 at 10:06:23PM +0800, bill lam <cbill.lam@gmail.com> wrote:
> I run these commands
>
> make ASCIIDOC_NO_ROFF=YesPlease prefix=/usr all doc info
> sudo make ASCIIDOC_NO_ROFF=YesPlease prefix=/usr install install-doc install-html install-info
>
> 1. did I need to set ASCIIDOC_NO_ROFF in both lines?
In general, it's always a good idea, though I don't think it's necessary
for the second time.
> 2. now the .ft pair fixed but it still displayed incorrect quote.
>
> git filter-branch --tree-filter ´rm filename´ HEAD
>
> it should be 'rm filename' not ´rm filename´
I can reproduce that here as well, that's how it is in the official
manpages as well (see the man branch), so that's not specific to your
system.
[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply
* Re: [JGit-io-RFC-PATCH v2 2/4] Add JGit IO SPI and default implementation
From: Imran M Yousuf @ 2009-10-13 15:59 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git, egit-dev, Imran M Yousuf
In-Reply-To: <20091013151522.GR9261@spearce.org>
Hi Shawn,
Firstly thanks for the reply, my comments are inline.
On Tue, Oct 13, 2009 at 10:15 PM, Shawn O. Pearce <spearce@spearce.org> wrote:
> Imran M Yousuf <imyousuf@gmail.com> wrote:
>> Firstly, I am sorry but I am not intelligent enough to perceive, how
>> do the user decide which instance of Config to use? I personally think
>> that there is no API to achieve what you just mentioned :(; i.e. the
>> user will have know CassandraConfig directly.
>
> Yes. Well, almost.
>
> The user will have to know that s/he wants a CassandraRepository or
> a JdbcRepository in order to obtain the abstract Repository handle.
> Each of these will need different configuration, possibly data which
> is too complex to simply cram into a URL string, so I was expecting
> the application would construct the concrete Repository class and
> configure it with the proper arguments required for contact with
> the underlying storage.
>
> Since the Repository wants several things associated with it, each
> concrete Repository class knows what concrete Config, ObjectDatabase
> and RefDatabase it should create. Those concrete classes know how
> to read a repository stored on that medium.
>
Hmm, when trying to come up with an API, where I essentially wanted a
to abstract all records, I noticed that everything uses java.io.File
and I never actually thought in this line.
Well, I was thinking of in terms of URI as I think Git and in turn
JGit (some how feels so) follows REST and from my understanding of git
storage (which very well could be incorrect) URI could be a perfect
match. Question of how a implementation requiring configuration be fed
to the SPI manager is simple before JGit is used the instance it self
is registered to the manager. So e.g. for JDBC the URI could very well
look like - jdbc://etc/X11/ for a repo path and the JDBC implementor
will already know the connection config specs etc. so there is no need
to cramp in all info in the URL.
>> Secondly, I instead was
>> thinking of porting JGit for that matter to any system supporting
>> streams (not any specific sub-class of them), such HBase/BigTable or
>> HDFS anything.... Thirdly, I think we actually have several task in
>> hand and I would state them as -
>>
>> 1. First introduce the I/O API such that it completely replaces java.io.File
>> 2. Secondly segregate persistence of for config (or config like
>> objects) and introduce a SPI for them for smarter storage.
>
> Supporting streams on an arbitrary backend is difficult. DHTs like
> BigTable/Cassandra aren't very good at providing streams, they tend
> to have a limit on how big a row can be. They tend to have very
> slow read latencies, but can return a small block of consecutive
> rows in one reply.
>
> I want to talk about the DHT backend more with Scott Chacon at the
> GitTogether, but I have this feeling that just laying a pack file
> into a stream in a DHT is going to perform very poorly.
>
> Likewise JDBC has similar performance problems, you can only store
> so much in a row before performance of the RDBMS drops off sharply.
> You can get a handful of rows in a single reply pretty efficiently,
> but each query takes longer than you'd like. Yes, there is often
> a BLOB type that allows large file storage, but different RDBMS
> support these differently and have different performance when it
> comes to accessing the BLOB types. Some don't support random access,
> some do. Even if they do support random access read, writing a large
> 2 GiB repository's pack file after repacking it would take ages.
>
> Once you get outside of the pack file, *everything* else git stores
> is either a loose object, or very tiny text files (aka refs, their
> logs, config). The loose object case should be handled by the same
> thing that handles the bulk of the object store, loose objects are
> a trivial thing compared to packed objects.
>
> The refs, the ref logs, and the config are all structured text.
> If you lay a Git repository down into a database of some sort,
> I think its reasonable to expect that the schema for these items
> in that database permits query and update using relatively native
> primitives in that database. E.g. if you put this in SQL I would
> expect a schema like:
>
> CREATE TABLE refs (
> repository_id INT NOT NULL
> ,name VARCHAR(255) NOT NULL
> ,id CHAR(40)
> ,target VARCHAR(255)
> ,PRIMARY KEY (repository_id, name)
> ,CHECK (id IS NOT NULL OR target IS NOT NULL));
>
> CREATE TABLE reflogs (
> repository_id INT NOT NULL
> ,name VARCHAR(255) NOT NULL
> ,old_id CHAR(40) NOT NULL
> ,new_id CHAR(40) NOT NULL
> ,committer_name VARCHAR(255)
> ,committer_email VARCHAR(255)
> ,committer_date TIMESTAMP NOT NULL
> ,message VARCHAR(255)
> ,PRIMARY KEY (repository_id, name, committer_date));
>
> CREATE TABLE config (
> repository_id INT NOT NULL
> ,section VARCHAR(255) NOT NULL
> ,group VARCHAR(255)
> ,name VARCHAR(255) NOT NULL
> ,value VARCHAR(255)
> ,PRIMARY KEY(repository_id, section, group, name, value))
>
> This makes it easier to manipulate settings, you can use direct
> SQL UPDATE to modify the configuration, or SELECT to scan through
> reflogs. Etc.
>
> If we just threw everything as streams into the database this
> would be a lot more difficult to work with through the database's
> own native query and update interface. You'd lose alot of the
> benefits of using a database, but still be paying the massive price
> in performance.
>
To be honest I understood your point the last time you mentioned it
:). I agree with performance part fully and I too have doubts, that is
why while you were mentioning HBase, I was HDFS :) and I was actually
thinking of in terms of FS. But after your elaboration, it just makes
more sense for the changes -
* Firstly we decouple from a particular FS and have our own I/O, for
packed and loose objects, so that we can easily retain the current
behavior.
* Then we first rework the key objects, e.g. Refs, Configs etc. to
segregate their persistence, that is introduce their persistence layer
which should friendly enough for native operations for platforms such
as RDBMS or HBase or BigTable. Using packs will depend of setup and
repositories, but certain implementations
* We implement this SPI to support different persistence platforms then :).
The thing is I first want to make JGit independent of java.io.File :),
thats was my motto to start with, but you showed me a path beyond that
:) and that is free from java.io.File and optimized persistence API
:). I want to have them both and let the implementor choose how to
implement it :).
>> I am not thinking of storing "only" the bare content of a git
>> repository, but I intent to be able to also store the versioned
>> contents it self as well.
>
> When I say "bare repository" I only mean a repository without a
> working directory. It still holds the complete revision history.
> If you wanted a git repository on Cassandra but wanted to actually
> have a working directory checkout, you'd need the local filesystem
> for the checkout and .git/index, but could otherwise keep the objects
> and refs in Cassandra. Its nuts... but in theory one could do it.
>
My requirement also involves needing to check it out on HDFS :), that
is why I was mentioning it, but it could be a different topic other
than that of JGit.
Eagerly waiting for a reply.
Thank you,
--
Imran M Yousuf
Entrepreneur & Software Engineer
Smart IT Engineering
Dhaka, Bangladesh
Email: imran@smartitengineering.com
Blog: http://imyousuf-tech.blogs.smartitengineering.com/
Mobile: +880-1711402557
^ permalink raw reply
* Re: backup git repo on every commit
From: Israel Garcia @ 2009-10-13 16:43 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git
In-Reply-To: <20091012143043.GJ9261@spearce.org>
Hi Shawn,
Sorry to ask again, but I'm a little confuse about how git work in my case.
I use gitosis on a server where I have all repos
(/usr/local/git/repositories/), so different people clone their repos
on their computers. What I want is to backup, on gitosis server, all
repos in /backups/git/repositories/ after every commit. So, my
questions are:
Do I have to run these two comands on tha gitosis server? Or on a
remote computer?
git --git-dir=/backup/project.git init
git remote add --mirror backup /backup/project.git
The post-commit are execute on gitosis server or on the remote pc?
I'm completely new using git so I'm a little confuse.. sorry for asking again.
thanks a lot.
regards
Israel.
On 10/12/09, Shawn O. Pearce <spearce@spearce.org> wrote:
> Israel Garcia <igalvarez@gmail.com> wrote:
>> That's OK, but I want to backup my git repo locally
>
> Just change the path of the backup remote (that final argument to
> git remote add) to point to the local directory.
>
> Though I guess you would also need to run git init there, e.g.:
>
> git --git-dir=/backup/project.git init
> git remote add --mirror backup /backup/project.git
>
> # and create the hook as below
>
> Of course, backup to another folder on the same disk isn't a backup
> at all, the disk can still fail and lose both repositories.
>
>> On 10/12/09, Shawn O. Pearce <spearce@spearce.org> wrote:
>> > Israel Garcia <igalvarez@gmail.com> wrote:
>> >> Which is the simplest way to backup a git repository after every
>> >> commit?
>> >
>> > Add a commit hook to push to another location, e.g.:
>> >
>> > git remote add --mirror backup you@another.host:path/to/backup.git
>> >
>> > cat >.git/hooks/post-commit
>> > #!/bin/sh
>> > git push backup
>> > ^D
>> >
>> > chmod a+x .git/hooks/post-commit
>
> --
> Shawn.
>
--
Regards;
Israel Garcia
^ permalink raw reply
* Re: Git: "No you can't handle my root!" (?)
From: Tony Finch @ 2009-10-13 17:46 UTC (permalink / raw)
To: Jeff King; +Cc: sylvain, Alex Riesen, Steven Noonan, git
In-Reply-To: <20091012183519.GA10686@coredump.intra.peff.net>
On Mon, 12 Oct 2009, Jeff King wrote:
>
> $ cd /
> $ git init
> $ cd /etc/whatever
> $ git add .
One reason that you don't want to do this (even if it does work) is that
careless use of git (e.g. by a user who is not the sysadmin playing with
git in their home directory) is going to find the root repository when you
expect it not to find any repository.
Also, I suggest this little wrapper script:
http://dotat.at/cgi/git?p=git-deploy.git;a=blob;f=git-root.sh
Tony.
--
f.anthony.n.finch <dot@dotat.at> http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.
^ permalink raw reply
* Re: backup git repo on every commit
From: Shawn O. Pearce @ 2009-10-13 17:49 UTC (permalink / raw)
To: Israel Garcia; +Cc: git
In-Reply-To: <194a2c240910130943j40c12902o760e463e7a8ce8fa@mail.gmail.com>
Israel Garcia <igalvarez@gmail.com> wrote:
> Sorry to ask again, but I'm a little confuse about how git work in my case.
> I use gitosis on a server where I have all repos
> (/usr/local/git/repositories/), so different people clone their repos
> on their computers. What I want is to backup, on gitosis server, all
> repos in /backups/git/repositories/ after every commit. So, my
> questions are:
>
> Do I have to run these two comands on tha gitosis server?
Yes.
> The post-commit are execute on gitosis server or on the remote pc?
Actually, you need the post-update hook. post-commit doesn't run
on the gitosis server.
I suggested post-commit because I thought you were talking about
backing up your local working directory each time you called
"git commit". But since you are actually backing up every
"git push" you need to use the hooks invoked by that instead.
--
Shawn.
^ permalink raw reply
* Re: [RFC PATCH v2 09/16] Move WebDAV HTTP push under remote-curl
From: Johannes Schindelin @ 2009-10-13 18:04 UTC (permalink / raw)
To: Mike Hommey; +Cc: Shawn O. Pearce, git, Daniel Barkalow, Tay Ray Chuan
In-Reply-To: <20091013044141.GA18961@glandium.org>
Hi,
On Tue, 13 Oct 2009, Mike Hommey wrote:
> On Mon, Oct 12, 2009 at 07:25:08PM -0700, Shawn O. Pearce wrote:
> > The remote helper interface now supports the push capability, which
> > can be used to ask the implementation to push one or more specs to the
> > remote repository. For remote-curl we implement this by calling the
> > existing WebDAV based git-http-push executable.
> >
> > Internally the helper interface uses the push_refs transport hook so
> > that the complexity of the refspec parsing and matching can be reused
> > between remote implementations. When possible however the helper
> > protocol uses source ref name rather than the source SHA-1, thereby
> > allowing the helper to access this name if it is useful.
>
> It's been a while I haven't followed changes in the remote code but this
> looks nice, though I haven't checked thoroughly. I guess the next step
> would be to kill http-push as an external program.
Ilari made signs on IRC that he got something working. After, ahem,
saying a few things about the design of the remote helpers.
Ciao,
Dscho
^ permalink raw reply
* Re: [RFC PATCH v2 07/16] remote-helpers: Fetch more than one ref in a batch
From: Shawn O. Pearce @ 2009-10-13 18:05 UTC (permalink / raw)
To: Daniel Barkalow; +Cc: git
In-Reply-To: <alpine.LNX.2.00.0910122326130.32515@iabervon.org>
Daniel Barkalow <barkalow@iabervon.org> wrote:
> On Mon, 12 Oct 2009, Shawn O. Pearce wrote:
> > Some network protocols (e.g. native git://) are able to fetch more
> > than one ref at a time and reduce the overall transfer cost by
> > combining the requests into a single exchange. Instead of feeding
> > each fetch request one at a time to the helper, feed all of them
> > at once so the helper can decide whether or not it should batch them.
> >
> > Because 'fetch' was already released in 1.6.5 we introduce the new
> > fetch-multiple capability/command to signal that the helper wants
> > to use batch oriented approach to fetching refs.
>
> In 1.6.5, there's no way to call a helper other than git-remote-curl, and
> no way to call git-remote-curl unless 1.6.5 was built with it. So I think
> the protocol is not set in stone quite yet. It's documentated for being an
> API, and it's supposed to be that, but it's not quite there in this
> version.
I originally had this change as just redefining "fetch" to be batch
oriented and requiring a blank line to terminate the batch group,
but reconsidered when 1.6.5 shipped and I realized this code was
in 1.6.5.
But, yea, you are right, with no way to invoke anything except
remote-curl there really isn't that much of a problem if we change
the protocol.
--
Shawn.
^ permalink raw reply
* Re: backup git repo on every commit
From: Israel Garcia @ 2009-10-13 18:08 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git
In-Reply-To: <20091013174913.GV9261@spearce.org>
On 10/13/09, Shawn O. Pearce <spearce@spearce.org> wrote:
> Israel Garcia <igalvarez@gmail.com> wrote:
>> Sorry to ask again, but I'm a little confuse about how git work in my
>> case.
>> I use gitosis on a server where I have all repos
>> (/usr/local/git/repositories/), so different people clone their repos
>> on their computers. What I want is to backup, on gitosis server, all
>> repos in /backups/git/repositories/ after every commit. So, my
>> questions are:
>>
>> Do I have to run these two comands on tha gitosis server?
>
> Yes.
>
>> The post-commit are execute on gitosis server or on the remote pc?
>
> Actually, you need the post-update hook. post-commit doesn't run
> on the gitosis server.
>
> I suggested post-commit because I thought you were talking about
> backing up your local working directory each time you called
> "git commit". But since you are actually backing up every
> "git push" you need to use the hooks invoked by that instead.
Hi Shawn,
Thanks for your answer, could you please put here how you think
post-update file should be setup? Remember in my case I want to backup
every repo to /backups/git folder for example.
Thanks in advance.
regrads
Israel.
>
> --
> Shawn.
>
--
Regards;
Israel Garcia
^ permalink raw reply
* Re: [RFC PATCH v2 01/16] pkt-line: Add strbuf based functions
From: Shawn O. Pearce @ 2009-10-13 18:10 UTC (permalink / raw)
To: Johannes Sixt; +Cc: git
In-Reply-To: <4AD42C52.80205@viscovery.net>
Johannes Sixt <j.sixt@viscovery.net> wrote:
> Shawn O. Pearce schrieb:
> > -int packet_read_line(int fd, char *buffer, unsigned size)
> > +static int packet_length(unsigned *ret_len, const char *linelen)
...
> > + *ret_len = len;
> > + return 0;
> > +}
>
> len can be signed: Valid lengths fit into a signed int. Then you can
> 'return len;' on success and 'return -1;' on failure and don't need return
> the result by reference. packet_read_line() ultimately converts it to int
> anyway:
Great catch, thanks. This is actually from a prior version of code
where I was exposing this function to callers... but even then the
method could have just returned int with the value because as you
point out, all valid lengths fit in int and must be >= 0.
--
Shawn.
^ permalink raw reply
* Efficient cloning from svn (with multiple branches/tags subdirs)
From: Bruno Harbulot @ 2009-10-13 18:13 UTC (permalink / raw)
To: git
Hello,
I'm trying to clone an existing subversion repository (Restlet:
http://restlet.tigris.org/source/browse/). I'm using Git 1.6.5. The
layout of the project is like this:
trunk/
branches/1.0
branches/1.1
tags/1.0/1.0b1
tags/1.0/1.0b2
...
tags/1.0/1.0.1
...
tags/1.1/1.1.0
tags/1.1/1.1.1
...
Therefore, I've tried to use this (with and without '-T trunk', but
that's a separate problem):
git init
git svn init --prefix=svn/ -t tags/1.0 -t tags/1.1 -t tags/1.2 -t
tags/2.0 -b branches/1.0 -b branches/1.1
http://restlet.tigris.org/svn/restlet
git svn fetch
This takes a while (I've had to interrupt this) and this creates a
number of branches such as:
remotes/svn/tags/1.0b1
remotes/svn/tags/1.0b2
remotes/svn/tags/1.0b3
remotes/svn/tags/1.0b3@1883
remotes/svn/tags/1.0b3@323
What surprises me is that it looks like it's looping over and over,
since sometimes it starts back from SVN revision 1 when it's trying to
import a new tag.
Tt starts like this:
>
> Checked through r101
> Checked through r201
> Checked through r301
> A www/index.html
> r1 = 2ec77afc2e491e2b7c825cb685101e3bcbe7a8f7 (refs/remotes/svn/tags/1.0b1@312)
> A source/impl/License.txt
> A source/impl/Copyright.txt
> A source/impl/org/restlet/UniformInterface.java
> A source/impl/org/restlet/RestletException.java
> ...
Then, when it reaches r312, it starts again at r1:
> r312 = 5b40558b5bb2b4b04f9520f89b699ff6b0f50cdb (refs/remotes/svn/tags/1.0b1@312)
> r313 = 7ebcbd9da535cfdc23aacb612271e625445a7516 (refs/remotes/svn/tags/1.0b1@1881)
> r1882 = aed1582d4868a1be8ae8fcc0f15546822099f339 (refs/remotes/svn/tags/1.0b1)
> Checked through r101
> Checked through r201
> Checked through r301
> A www/index.html
> r1 = 2ec77afc2e491e2b7c825cb685101e3bcbe7a8f7 (refs/remotes/svn/tags/1.0b2@321)
> A source/impl/License.txt
> A source/impl/Copyright.txt
> A source/impl/org/restlet/UniformInterface.java
> A source/impl/org/restlet/RestletException.java
> A source/impl/org/restlet/AbstractRestlet.java
> A source/impl/org/restlet/connector/Resolver.java
(And so on for each tag).
This seems particularly inefficient and unfriendly for the resource
provider (I stopped as soon as I noticed). Is there a better way to do this?
Best wishes,
Bruno.
^ permalink raw reply
* Re: backup git repo on every commit
From: Israel Garcia @ 2009-10-13 18:14 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git
In-Reply-To: <20091013174913.GV9261@spearce.org>
BTW, is there any git-dump or git-backup command to do some kind of
backup I'm looking for?
regards,
Israel.
On 10/13/09, Shawn O. Pearce <spearce@spearce.org> wrote:
> Israel Garcia <igalvarez@gmail.com> wrote:
>> Sorry to ask again, but I'm a little confuse about how git work in my
>> case.
>> I use gitosis on a server where I have all repos
>> (/usr/local/git/repositories/), so different people clone their repos
>> on their computers. What I want is to backup, on gitosis server, all
>> repos in /backups/git/repositories/ after every commit. So, my
>> questions are:
>>
>> Do I have to run these two comands on tha gitosis server?
>
> Yes.
>
>> The post-commit are execute on gitosis server or on the remote pc?
>
> Actually, you need the post-update hook. post-commit doesn't run
> on the gitosis server.
>
> I suggested post-commit because I thought you were talking about
> backing up your local working directory each time you called
> "git commit". But since you are actually backing up every
> "git push" you need to use the hooks invoked by that instead.
>
> --
> Shawn.
>
--
Regards;
Israel Garcia
^ permalink raw reply
* Re: backup git repo on every commit
From: Shawn O. Pearce @ 2009-10-13 18:18 UTC (permalink / raw)
To: Israel Garcia; +Cc: git
In-Reply-To: <194a2c240910131114q19b6c822t5806d20005341cb4@mail.gmail.com>
Israel Garcia <igalvarez@gmail.com> wrote:
> BTW, is there any git-dump or git-backup command to do some kind of
> backup I'm looking for?
No, you backup git by making a clone. E.g. `git clone --bare`.
Since this leaves you with a directory, you need to then perhaps
use some sort of file combiner tool like tar or zip to produce a
single file for backup to tape.
You can incrementally update that backup clone using git push to
write into it, or you can just blow it away and recreate it each
time you make a backup.
One could also use `git bundle` to create backup file that had
everything packaged in one neat file, but this can be slightly
harder to work with since a bundle is not a git repository.
--
Shawn.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox