* Re: Octopus merge
From: Claudia.Ignat @ 2009-12-09 15:10 UTC (permalink / raw)
To: Michael J Gruber, Thomas Rast; +Cc: git
In-Reply-To: <4B0EB257.8080002@drmicha.warpmail.net>
Michael and Thomas, thank you for your explanations. The fact that in
some cases octopus merge fails, but changes of the first merges subparts
of octopus merge are included in the worktree rests a dilemma.
Further, I have noticed that for the same conflicting changes the
conflict messages (referring to conflict types) received in the case of
an octopus merge are not the same as the conflict messages received for
the corresponding sequential merges.
I slightly changed the scenario I described in my previous message.
Workspaces ws2, ws3 and ws4 are clones of the workspace ws1. Suppose the
current workspace is ws3 and some changes are done in parallel in all
these 3 workspaces. Changes of ws2 and ws3 are conflicting: both modify
the same line of file file1.txt to different values and they rename the
same file file2.txt to fileWS2.txt and fileWS3.txt respectively. Changes
in ws4 are not conflicting with changes in ws2 and ws3.
While I am in the current workspace ws3 I perform the merge
git merge ws2
and content conflict in file1.txt as well as rename/rename conflict are
signaled.
If instead I perform the merge
git merge ws4 ws2
only the content conflict in file1.txt is raised, while the conflict
rename/rename of file2.txt is not mentioned.
The script for the second mentioned scenario is given below.
Do you think this should be the normal behavior of octopus merge?
Thanks.
Cheers,
Claudia
# !/bin/bash
TEST_NAME="TP1" # name of the working directory
rm -rf ${TEST_NAME} # cleaning working directory
mkdir -p ${TEST_NAME}
cd ${TEST_NAME}
# initialising initial git workspace
mkdir ws1
cd ws1
git init
# adding 2 files to ws1
# commit changes
echo -e -n "1\n2\n3\n4\n5\n" > file1.txt
echo -e -n "a\nb\nc\nd\ne\n" > file2.txt
git add file1.txt file2.txt
git commit -m "ws1 | add 12345 in file1.txt; add abcde in file2.txt"
cd ..
# cloning three times ws1 (as ws2, ws3 and ws4)
git clone ws1 ws2
git clone ws1 ws3
git clone ws1 ws4
# updating file1.txt in ws2 (insert X at line 3, then write and quit 'ed')
# rename file2.txt to fileWS2.txt
# commit changes
cd ws2
echo -e "3i\nX\n.\nw\nq\n" | ed file1.txt
git add file1.txt
git mv file2.txt fileWS2.txt
git commit -m "ws2 | insert 12X345 in file1.txt; rename file2.txt to
fileWS2.txt"
cd ..
# updating file1.txt in ws3 (insert Y at line 3, then write and quit 'ed')
# rename file2.txt to fileWS3.txt
# commit changes
cd ws3
echo -e "3i\nY\n.\nw\nq\n" | ed file1.txt
git add file1.txt
git mv file2.txt fileWS3.txt
git commit -m "ws3 | insert 12Y345 in file1.txt; rename file2.txt to
fileWS3.txt"
cd ..
# add file u.txt in ws4
cd ws4
echo -e -n "U1\n2\n3\n4\n5\n" > u.txt
git add u.txt
git commit -m "ws4 | add u.txt"
cd ..
# ws3 pull from ws2 ws4
cd ws3
git remote add bws2 ../ws2
git remote add bws4 ../ws4
git fetch bws2
git fetch bws4
git merge bws4/master bws2/master
cd ..
Michael J Gruber wrote:
> Claudia.Ignat@loria.fr venit, vidit, dixit 26.11.2009 16:56:
>> # !/bin/bash
>> TEST_NAME="TP1" # name of the working directory
>> rm -rf ${TEST_NAME} # cleaning working directory
>> mkdir -p ${TEST_NAME}
>> cd ${TEST_NAME}
>>
>> # initialising initial git workspace
>> mkdir ws1
>> cd ws1
>> git init
>>
>> # adding a file to ws1
>> # commit changes
>> echo -e -n "1\n2\n3\n4\n5\n" > file.txt
>> git add file.txt
>> git commit -m "ws1 | add 12345"
>> cd ..
>>
>> # cloning three times ws1 (as ws2, ws3 and ws4)
>> git clone ws1 ws2
>> git clone ws1 ws3
>> git clone ws1 ws4
>> git clone ws1 ws5
>>
>> # updating file.txt in ws2 (insert X at line 3, then write and quit 'ed')
>> # commit changes
>> cd ws2
>> echo -e "3i\nX\n.\nw\nq\n" | ed file.txt
>> git add file.txt
>> git commit -m "ws2 | insert 12X345"
>> cd ..
>>
>>
>> # updating file.txt in ws3 (insert Y at line 3, then write and quit 'ed')
>> # commit changes
>> cd ws3
>> echo -e "3i\nY\n.\nw\nq\n" | ed file.txt
>> git add file.txt
>> git commit -m "ws3 | insert 12Y345"
>> cd ..
>>
>> cd ws4
>> echo -e -n "U1\n2\n3\n4\n5\n" > u.txt
>> git add u.txt
>> git commit -m "ws4 | add u.txt"
>> cd ..
>>
>> cd ws5
>> echo -e -n "W1\n2\n3\n4\n5\n" > w.txt
>> git add w.txt
>> git commit -m "ws5 | add w.txt"
>> cd ..
>>
>> # ws3 pull from ws2 ws4 ws5
>> cd ws3
>> git remote add bws2 ../ws2
>> git remote add bws4 ../ws4
>> git remote add bws5 ../ws5
>> git fetch bws2
>> git fetch bws4
>> git fetch bws5
>> git merge bws4/master bws2/master bws5/master
>> cd ..
>>
>> # resolve conflict in ws3
>> cd ws3
>
> First of all, thanks for the clear description and test case!
>
> The octopus strategy cannot do merges which need manual resolution. Or
> so the doc says. After trying the merge with 4 2 5, Git tells you:
>
> Trying simple merge with 7ff9b5bd514cb600bac935ebd40eae366bba7d19
> Trying simple merge with 6872cd350154743d59cb4d313cbdb122ac43e537
> Simple merge did not work, trying automatic merge.
> Auto-merging file.txt
> ERROR: content conflict in file.txt
> fatal: merge program failed
> Automated merge did not work.
> Should not be doing an Octopus.
> Merge with strategy octopus failed.
>
> That is, it aborts the merge completely. If you "resolve" it and commit
> it's simply a commit that you make.
>
> If, instead, you merge 4 5 2, Git tells you:
>
> Trying simple merge with e4f78f6905bed39bcd96790a4f63e138a455a445
> Trying simple merge with 14c1f2a70767334df5d6d3120631752564094699
> Trying simple merge with 8540a039d3fc964d097d4f037357668441d1d4f5
> Simple merge did not work, trying automatic merge.
> Auto-merging file.txt
> ERROR: content conflict in file.txt
> fatal: merge program failed
> Automatic merge failed; fix conflicts and then commit the result.
>
> Admittedly this looks fatal also, but the last line tells you that the
> actual merge process is not aborted yet. If you resolve the conflict and
> commit without -m you even see the prepared commit message.
>
> So, octopus can deal with manual conflict resolution if the conflicts
> appear in the last step only. That is the difference between the two cases.
>
> Now, in the first case the aborted merge leaves some traces in the index
> as well as in the worktree. I'm not sure that is how it's supposed to be.
>
> Cheers,
> Michael
^ permalink raw reply
* Re: [REROLL PATCH 2/8] Support mandatory capabilities
From: Ilari Liusvaara @ 2009-12-09 15:12 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vzl5t59bp.fsf@alter.siamese.dyndns.org>
On Tue, Dec 08, 2009 at 03:34:34PM -0800, Junio C Hamano wrote:
> Ilari Liusvaara <ilari.liusvaara@elisanet.fi> writes:
> > + fflush(stderr);
> > + die("Unknown madatory capability %s. This remote "
> > + "helper probably needs newer version of Git.\n",
> > + capname);
>
> Why fflush() here? Is the reason for needing to flush stderr before
> letting die() to write into it very specific to this codepath, or shared
> among other callers of die()? I am wondering if we should add this
> fflush() to report() in usage.c instead.
No idea why its there (anymore). Die will flush stderr anyway via exit.
Removed.
-Ilari
^ permalink raw reply
* Re: [REROLL PATCH 3/8] Pass unknown protocols to external protocol handlers
From: Ilari Liusvaara @ 2009-12-09 15:15 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vskbl59ai.fsf@alter.siamese.dyndns.org>
On Tue, Dec 08, 2009 at 03:35:17PM -0800, Junio C Hamano wrote:
> Ilari Liusvaara <ilari.liusvaara@elisanet.fi> writes:
>
> > +const char *remove_ext_force(const char *url)
> > +{
> > + const char *url2 = url;
> > + const char *first_colon = NULL;
> > +
> > + if (!url)
> > + return NULL;
> > +
> > + while (*url2 && !first_colon)
> > + if (*url2 == ':')
> > + first_colon = url2;
> > + else
> > + url2++;
> > +
> > + if (first_colon && first_colon[1] == ':')
> > + return first_colon + 2;
> > + else
> > + return url;
> > +}
>
> Sorry, I am slow today, but is this any different from:
>
> if (url) {
> const char *colon = strchr(url, ':');
> if (colon && colon[1] == ':')
> return url2 + 2;
> }
> return url;
Undefined variable url2 (you mean colon?). Changed.
> > diff --git a/transport.c b/transport.c
> > index 3eea836..e42a82b 100644
> > --- a/transport.c
> > +++ b/transport.c
> > @@ -780,6 +780,58 @@ static int is_file(const char *url)
> > return S_ISREG(buf.st_mode);
> > }
> >
> > +static const char *strchrc(const char *str, int c)
> > +{
> > + while (*str)
> > + if (*str == c)
> > + return str;
> > + else
> > + str++;
> > + return NULL;
> > +}
>
> I cannot spot how this is different from strchr()...
Return type. Not that useful. Removed.
> > +static int is_url(const char *url)
> > +{
> > + const char *url2, *first_slash;
> > +
> > + if (!url)
> > + return 0;
> > + url2 = url;
> > + first_slash = strchrc(url, '/');
> > +
> > + /* Input with no slash at all or slash first can't be URL. */
> > + if (!first_slash || first_slash == url)
> > + return 0;
> > + /* Character before must be : and next must be /. */
> > + if (first_slash[-1] != ':' || first_slash[1] != '/')
> > + return 0;
> > + /* There must be something before the :// */
> > + if (first_slash == url + 1)
> > + return 0;
> > + /*
> > + * Check all characters up to first slash. Only alpha, num and
> > + * colon (":") are allowed. ":" must be followed by ":" or "/".
> > + */
I tightened this a bit, only checking (exclusive) character before first
"/" and requiring all-alphanum. The previous rules were apparently
leftover from times the "specify remote helper by url" patch didn't
exist.
Considering context at call site, it is equivalent, since any ':' must be
followed by '/' or is_url will never be called, and first ':/' must be
exactly at last_slash - 1 (by earlier checks).
> Hmm, so "a::::bc:://" is ok?
is_url never gets such thing. Due to "::" it bypasses URL validation. And
yes, it is ok.
> Is the reason this does not to check the string up to (first_slash-1) for
> a valid syntax for <scheme> (as in "<scheme>://") because this potentially
> has "<helper>::" in front of it?
It doesn't have '<helper>::' in front of it.
> I cannot tell if you want to allow "<helper1>::<helper2>::<scheme>://..."
> by reading this code.
Allowing or denying that is not up to this function. And one can't chain
helpers that way. It invokes <helper1>
> > +static int external_specification_len(const char *url)
> > +{
> > + return strchrc(url, ':') - url;
> > +}
>
> Does the caller guarantee url has at least one colon in it?
Anything passing is_url() does have ':' in it.
> > + } else if (!is_url(url)
> > + || !prefixcmp(url, "file://")
> > + || !prefixcmp(url, "git://")
> > + || !prefixcmp(url, "ssh://")
> > + || !prefixcmp(url, "git+ssh://")
> > + || !prefixcmp(url, "ssh+git://")) {
> > + /* These are builtin smart transports. */
>
> Hmm, what is !is_url(url) at the beginning for, if this lists "builtin"
> smart transports?
It catches the scp and path syntaxes as not URLs.
> > + } else {
> > + /* Unknown protocol in URL. Pass to external handler. */
> > + int len = external_specification_len(url);
> > + char *handler = xmalloc(len + 1);
> > + handler[len] = 0;
> > + strncpy(handler, url, len);
> > + transport_helper_init(ret, handler);
>
> Hmph, external_specification_len() may get passed a colon-less string
> after all, I think.
Nope, it can't. Else if above snarfs everything that doesn't pass
is_url, and string can't pass it without having ':' in it.
-Ilari
^ permalink raw reply
* Re: [REROLL PATCH 6/8] Support remote helpers implementing smart transports
From: Ilari Liusvaara @ 2009-12-09 15:16 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vein5594u.fsf@alter.siamese.dyndns.org>
On Tue, Dec 08, 2009 at 03:38:41PM -0800, Junio C Hamano wrote:
> Ilari Liusvaara <ilari.liusvaara@elisanet.fi> writes:
> >
> > +'connect' <service>::
> > + Connects to given service. Stdin and stdout of helper are
>
> A minor point, but in prose, unless it explains how to use "stdin" and
> "stdout" as a variable, keyword, etc. in code, I'd prefer to see these
> spelled out, like so:
>
> The standard input and output of helpers are connected to ...
Changed.
> > -static int recvline(struct helper_data *helper, struct strbuf *buffer)
> > +static int _recvline(FILE *helper, struct strbuf *buffer)
> > {
>
> recvline_fh() vs revline() might be better as most of the interaction in
> this layer are done on helper_data, which makes the name recvline() pair
> nicely with sendline that also takes helper_data; and the oddball one that
> is _not_ an implementation detail (i.e. you have calls outside recvline()
> implementation that call _recvline()) hints that it takes filehandle
> instead.
- _recvline -> recvline_fh
- _process_connect -> process_connect_service
-Ilari
^ permalink raw reply
* Re: [REROLL PATCH 7/8] Support remote archive from external protocol helpers
From: Ilari Liusvaara @ 2009-12-09 15:16 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7v7hsx5935.fsf@alter.siamese.dyndns.org>
On Tue, Dec 08, 2009 at 03:39:42PM -0800, Junio C Hamano wrote:
> Ilari Liusvaara <ilari.liusvaara@elisanet.fi> writes:
>
> > Helpers which support connect also should support remote archive
> > snapshot (or at least there's only one way to attempt it). So support
> > remote snapshotting for protocol helpers.
>
> Or "Because the transport layer has been restructured cleanly enough to
> allow passing general payload, there is no reason not to do this change to
> pass 'archive' output in addition to the 'git smart fetch/push protocol'
> payload, and this allows the archive command to take advantage of the
> helper based transports"???
This one is the correct interpretation.
Changed to:
-----------
Support remote archive from all smart transports
Previously, remote archive required internal (non remote-helper)
smart transport. Extend the remote archive to also support smart
transports implemented by remote helpers.
------------
-Ilari
^ permalink raw reply
* Re: [REROLL PATCH 5/8] Support taking over transports
From: Ilari Liusvaara @ 2009-12-09 15:17 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vljhd597h.fsf@alter.siamese.dyndns.org>
On Tue, Dec 08, 2009 at 03:37:06PM -0800, Junio C Hamano wrote:
> Ilari Liusvaara <ilari.liusvaara@elisanet.fi> writes:
>
> Will we ever have another 'xxxoptions' field in this structure that is not
> so git-ish? 'gitoptions' somehow doesn't feel right, unless you plan to
> later add scm specific options like 'hgoptions', 'bzroptions' in this
> field you need to distinguish "git" one from them.
>
> I know you needed to name the new field to store the transport option
> under a different name from the existing 'option' field, but for that
> purpose, 'transport_options' might be a more appropriate name.
Changed.
> > @@ -109,9 +111,19 @@ static struct child_process *get_helper(struct transport *transport)
> > die("Unable to run helper: git %s", helper->argv[0]);
> > data->helper = helper;
> >
> > + /*
> > + * Open the output as FILE* so strbuf_getline() can be used.
> > + * Do this with duped fd because fclose() will close the fd,
> > + * and stuff like taking over will require the fd to remain.
> > + *
> > + */
> > + duped = dup(helper->out);
> > + if (duped < 0)
> > + die_errno("Can't dup helper output fd");
> > + data->out = xfdopen(duped, "r");
> > +
>
> Neat hack (the kind I often love), but it makes something deep inside me
> cringe. This looks fragile and possibly wrong.
>
> Who guarantees that reading from the (FILE *)data->out by strbuf_getline()
> that eventually calls into fgetc() does not cause more data be read in the
> buffer assiciated with the (FILE *) than we will consume? The other FILE*
> you will make out of helper->out won't be able to read what data->out has
> already slurped in to its stdio buffer.
Causality impiles this can happen only if buffered version gets its buffer
filled after sending connect command. And looking at stdio operations that
can occur after sending the command:
- fprintfs on stderr (debug mode only).
- fgetc()s on unbuffered version.
-Ilari
^ permalink raw reply
* [REROLL PATCH v2 0/8] Remote helpers smart transport extensions
From: Ilari Liusvaara @ 2009-12-09 15:26 UTC (permalink / raw)
To: git
Changes from reroll v1:
- Wording fix in pass unknown protocols patch commit message
- Fix NO_CURL and hopefully make remote-http building bit clearer
- Remove extraneous fflush(stderr)
- Simplify remove_ext_force().
- Use strchr() instead of strchrc(). strchr() not const-correct, but...
- Simplify URL checking in is_url()
- Expand stdin and stdout in remote helper documentation
- _recvline -> recvline_fh
- _process_connect -> process_connect_service
- Reworded remote archive patch message
- gitoptions -> transport_options.
Ilari Liusvaara (8):
Add remote helper debug mode
Support mandatory capabilities
Pass unknown protocols to external protocol handlers
Refactor git transport options parsing
Support taking over transports
Support remote helpers implementing smart transports
Support remote archive from all smart transports
Remove special casing of http, https and ftp
.gitignore | 4 +
Documentation/git-remote-helpers.txt | 30 ++++-
Makefile | 27 +++-
builtin-archive.c | 17 ++-
transport-helper.c | 283 ++++++++++++++++++++++++++++++----
transport.c | 214 ++++++++++++++++++++------
transport.h | 22 +++
7 files changed, 504 insertions(+), 93 deletions(-)
^ permalink raw reply
* [REROLL PATCH v2 1/8] Add remote helper debug mode
From: Ilari Liusvaara @ 2009-12-09 15:26 UTC (permalink / raw)
To: git
In-Reply-To: <1260372394-16427-1-git-send-email-ilari.liusvaara@elisanet.fi>
Remote helpers deadlock easily, so support debug mode which shows the
interaction steps.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
---
transport-helper.c | 94 ++++++++++++++++++++++++++++++++++++++--------------
1 files changed, 69 insertions(+), 25 deletions(-)
diff --git a/transport-helper.c b/transport-helper.c
index 11f3d7e..a721dc2 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -8,6 +8,8 @@
#include "quote.h"
#include "remote.h"
+static int debug;
+
struct helper_data
{
const char *name;
@@ -22,6 +24,45 @@ struct helper_data
int refspec_nr;
};
+static void sendline(struct helper_data *helper, struct strbuf *buffer)
+{
+ if (debug)
+ fprintf(stderr, "Debug: Remote helper: -> %s", buffer->buf);
+ if (write_in_full(helper->helper->in, buffer->buf, buffer->len)
+ != buffer->len)
+ die_errno("Full write to remote helper failed");
+}
+
+static int recvline(struct helper_data *helper, struct strbuf *buffer)
+{
+ strbuf_reset(buffer);
+ if (debug)
+ fprintf(stderr, "Debug: Remote helper: Waiting...\n");
+ if (strbuf_getline(buffer, helper->out, '\n') == EOF) {
+ if (debug)
+ fprintf(stderr, "Debug: Remote helper quit.\n");
+ exit(128);
+ }
+
+ if (debug)
+ fprintf(stderr, "Debug: Remote helper: <- %s\n", buffer->buf);
+ return 0;
+}
+
+static void xchgline(struct helper_data *helper, struct strbuf *buffer)
+{
+ sendline(helper, buffer);
+ recvline(helper, buffer);
+}
+
+static void write_constant(int fd, const char *str)
+{
+ if (debug)
+ fprintf(stderr, "Debug: Remote helper: -> %s", str);
+ if (write_in_full(fd, str, strlen(str)) != strlen(str))
+ die_errno("Full write to remote helper failed");
+}
+
static struct child_process *get_helper(struct transport *transport)
{
struct helper_data *data = transport->data;
@@ -48,15 +89,16 @@ static struct child_process *get_helper(struct transport *transport)
die("Unable to run helper: git %s", helper->argv[0]);
data->helper = helper;
- write_str_in_full(helper->in, "capabilities\n");
+ write_constant(helper->in, "capabilities\n");
data->out = xfdopen(helper->out, "r");
while (1) {
- if (strbuf_getline(&buf, data->out, '\n') == EOF)
- exit(128); /* child died, message supplied already */
+ recvline(data, &buf);
if (!*buf.buf)
break;
+ if (debug)
+ fprintf(stderr, "Debug: Got cap %s\n", buf.buf);
if (!strcmp(buf.buf, "fetch"))
data->fetch = 1;
if (!strcmp(buf.buf, "option"))
@@ -82,14 +124,21 @@ static struct child_process *get_helper(struct transport *transport)
free(refspecs);
}
strbuf_release(&buf);
+ if (debug)
+ fprintf(stderr, "Debug: Capabilities complete.\n");
return data->helper;
}
static int disconnect_helper(struct transport *transport)
{
struct helper_data *data = transport->data;
+ struct strbuf buf = STRBUF_INIT;
+
if (data->helper) {
- write_str_in_full(data->helper->in, "\n");
+ if (debug)
+ fprintf(stderr, "Debug: Disconnecting.\n");
+ strbuf_addf(&buf, "\n");
+ sendline(data, &buf);
close(data->helper->in);
fclose(data->out);
finish_command(data->helper);
@@ -117,10 +166,11 @@ static int set_helper_option(struct transport *transport,
const char *name, const char *value)
{
struct helper_data *data = transport->data;
- struct child_process *helper = get_helper(transport);
struct strbuf buf = STRBUF_INIT;
int i, ret, is_bool = 0;
+ get_helper(transport);
+
if (!data->option)
return 1;
@@ -143,12 +193,7 @@ static int set_helper_option(struct transport *transport,
quote_c_style(value, &buf, NULL, 0);
strbuf_addch(&buf, '\n');
- if (write_in_full(helper->in, buf.buf, buf.len) != buf.len)
- die_errno("cannot send option to %s", data->name);
-
- strbuf_reset(&buf);
- if (strbuf_getline(&buf, data->out, '\n') == EOF)
- exit(128); /* child died, message supplied already */
+ xchgline(data, &buf);
if (!strcmp(buf.buf, "ok"))
ret = 0;
@@ -208,13 +253,10 @@ static int fetch_with_fetch(struct transport *transport,
}
strbuf_addch(&buf, '\n');
- if (write_in_full(data->helper->in, buf.buf, buf.len) != buf.len)
- die_errno("cannot send fetch to %s", data->name);
+ sendline(data, &buf);
while (1) {
- strbuf_reset(&buf);
- if (strbuf_getline(&buf, data->out, '\n') == EOF)
- exit(128); /* child died, message supplied already */
+ recvline(data, &buf);
if (!prefixcmp(buf.buf, "lock ")) {
const char *name = buf.buf + 5;
@@ -249,12 +291,13 @@ static int fetch_with_import(struct transport *transport,
int nr_heads, struct ref **to_fetch)
{
struct child_process fastimport;
- struct child_process *helper = get_helper(transport);
struct helper_data *data = transport->data;
int i;
struct ref *posn;
struct strbuf buf = STRBUF_INIT;
+ get_helper(transport);
+
if (get_importer(transport, &fastimport))
die("Couldn't run fast-import");
@@ -264,7 +307,7 @@ static int fetch_with_import(struct transport *transport,
continue;
strbuf_addf(&buf, "import %s\n", posn->name);
- write_in_full(helper->in, buf.buf, buf.len);
+ sendline(data, &buf);
strbuf_reset(&buf);
}
disconnect_helper(transport);
@@ -369,17 +412,14 @@ static int push_refs(struct transport *transport,
}
strbuf_addch(&buf, '\n');
- if (write_in_full(helper->in, buf.buf, buf.len) != buf.len)
- exit(128);
+ sendline(data, &buf);
ref = remote_refs;
while (1) {
char *refname, *msg;
int status;
- strbuf_reset(&buf);
- if (strbuf_getline(&buf, data->out, '\n') == EOF)
- exit(128); /* child died, message supplied already */
+ recvline(data, &buf);
if (!buf.len)
break;
@@ -471,8 +511,7 @@ static struct ref *get_refs_list(struct transport *transport, int for_push)
while (1) {
char *eov, *eon;
- if (strbuf_getline(&buf, data->out, '\n') == EOF)
- exit(128); /* child died, message supplied already */
+ recvline(data, &buf);
if (!*buf.buf)
break;
@@ -497,6 +536,8 @@ static struct ref *get_refs_list(struct transport *transport, int for_push)
}
tail = &((*tail)->next);
}
+ if (debug)
+ fprintf(stderr, "Debug: Read ref listing.\n");
strbuf_release(&buf);
for (posn = ret; posn; posn = posn->next)
@@ -510,6 +551,9 @@ int transport_helper_init(struct transport *transport, const char *name)
struct helper_data *data = xcalloc(sizeof(*data), 1);
data->name = name;
+ if (getenv("GIT_TRANSPORT_HELPER_DEBUG"))
+ debug = 1;
+
transport->data = data;
transport->set_option = set_helper_option;
transport->get_refs_list = get_refs_list;
--
1.6.6.rc1.300.gfbc27
^ permalink raw reply related
* [REROLL PATCH v2 3/8] Pass unknown protocols to external protocol handlers
From: Ilari Liusvaara @ 2009-12-09 15:26 UTC (permalink / raw)
To: git
In-Reply-To: <1260372394-16427-1-git-send-email-ilari.liusvaara@elisanet.fi>
Change URL handling to allow external protocol handlers to implement
new protocols without the '::' syntax if helper name does not conflict
with any built-in protocol.
foo:// now invokes git-remote-foo with foo:// as the URL.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
---
transport-helper.c | 12 +++++++-
transport.c | 76 +++++++++++++++++++++++++++++++++++++++++----------
2 files changed, 72 insertions(+), 16 deletions(-)
diff --git a/transport-helper.c b/transport-helper.c
index 4b17aaa..271af34 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -63,6 +63,16 @@ static void write_constant(int fd, const char *str)
die_errno("Full write to remote helper failed");
}
+const char *remove_ext_force(const char *url)
+{
+ if (url) {
+ const char *colon = strchr(url, ':');
+ if (colon && colon[1] == ':')
+ return colon + 2;
+ }
+ return url;
+}
+
static struct child_process *get_helper(struct transport *transport)
{
struct helper_data *data = transport->data;
@@ -83,7 +93,7 @@ static struct child_process *get_helper(struct transport *transport)
strbuf_addf(&buf, "remote-%s", data->name);
helper->argv[0] = strbuf_detach(&buf, NULL);
helper->argv[1] = transport->remote->name;
- helper->argv[2] = transport->url;
+ helper->argv[2] = remove_ext_force(transport->url);
helper->git_cmd = 1;
if (start_command(helper))
die("Unable to run helper: git %s", helper->argv[0]);
diff --git a/transport.c b/transport.c
index 3eea836..dea37d0 100644
--- a/transport.c
+++ b/transport.c
@@ -780,6 +780,44 @@ static int is_file(const char *url)
return S_ISREG(buf.st_mode);
}
+static int is_url(const char *url)
+{
+ const char *url2, *first_slash;
+
+ if (!url)
+ return 0;
+ url2 = url;
+ first_slash = strchr(url, '/');
+
+ /* Input with no slash at all or slash first can't be URL. */
+ if (!first_slash || first_slash == url)
+ return 0;
+ /* Character before must be : and next must be /. */
+ if (first_slash[-1] != ':' || first_slash[1] != '/')
+ return 0;
+ /* There must be something before the :// */
+ if (first_slash == url + 1)
+ return 0;
+ /*
+ * Check all characters up to first slash - 1. Only alphanum
+ * is allowed.
+ */
+ url2 = url;
+ while (url2 < first_slash - 1) {
+ if (!isalnum((unsigned char)*url2))
+ return 0;
+ url2++;
+ }
+
+ /* Valid enough. */
+ return 1;
+}
+
+static int external_specification_len(const char *url)
+{
+ return strchr(url, ':') - url;
+}
+
struct transport *transport_get(struct remote *remote, const char *url)
{
struct transport *ret = xcalloc(1, sizeof(*ret));
@@ -805,30 +843,23 @@ struct transport *transport_get(struct remote *remote, const char *url)
if (remote && remote->foreign_vcs) {
transport_helper_init(ret, remote->foreign_vcs);
- return ret;
- }
-
- if (!prefixcmp(url, "rsync:")) {
+ } else if (!prefixcmp(url, "rsync:")) {
ret->get_refs_list = get_refs_via_rsync;
ret->fetch = fetch_objs_via_rsync;
ret->push = rsync_transport_push;
-
- } else if (!prefixcmp(url, "http://")
- || !prefixcmp(url, "https://")
- || !prefixcmp(url, "ftp://")) {
- transport_helper_init(ret, "curl");
-#ifdef NO_CURL
- error("git was compiled without libcurl support.");
-#endif
-
} else if (is_local(url) && is_file(url)) {
struct bundle_transport_data *data = xcalloc(1, sizeof(*data));
ret->data = data;
ret->get_refs_list = get_refs_from_bundle;
ret->fetch = fetch_refs_from_bundle;
ret->disconnect = close_bundle;
-
- } else {
+ } else if (!is_url(url)
+ || !prefixcmp(url, "file://")
+ || !prefixcmp(url, "git://")
+ || !prefixcmp(url, "ssh://")
+ || !prefixcmp(url, "git+ssh://")
+ || !prefixcmp(url, "ssh+git://")) {
+ /* These are builtin smart transports. */
struct git_transport_data *data = xcalloc(1, sizeof(*data));
ret->data = data;
ret->set_option = set_git_option;
@@ -845,6 +876,21 @@ struct transport *transport_get(struct remote *remote, const char *url)
data->receivepack = "git-receive-pack";
if (remote->receivepack)
data->receivepack = remote->receivepack;
+ } else if (!prefixcmp(url, "http://")
+ || !prefixcmp(url, "https://")
+ || !prefixcmp(url, "ftp://")) {
+ /* These three are just plain special. */
+ transport_helper_init(ret, "curl");
+#ifdef NO_CURL
+ error("git was compiled without libcurl support.");
+#endif
+ } else {
+ /* Unknown protocol in URL. Pass to external handler. */
+ int len = external_specification_len(url);
+ char *handler = xmalloc(len + 1);
+ handler[len] = 0;
+ strncpy(handler, url, len);
+ transport_helper_init(ret, handler);
}
return ret;
--
1.6.6.rc1.300.gfbc27
^ permalink raw reply related
* [REROLL PATCH v2 5/8] Support taking over transports
From: Ilari Liusvaara @ 2009-12-09 15:26 UTC (permalink / raw)
To: git
In-Reply-To: <1260372394-16427-1-git-send-email-ilari.liusvaara@elisanet.fi>
Add support for taking over transports that turn out to be smart.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
---
transport-helper.c | 19 ++++++++++++++++++-
transport.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++-----
transport.h | 2 ++
3 files changed, 66 insertions(+), 6 deletions(-)
diff --git a/transport-helper.c b/transport-helper.c
index 271af34..97eed6c 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -22,6 +22,10 @@ struct helper_data
/* These go from remote name (as in "list") to private name */
struct refspec *refspecs;
int refspec_nr;
+ /* Transport options for fetch-pack/send-pack (should one of
+ * those be invoked).
+ */
+ struct git_transport_options transport_options;
};
static void sendline(struct helper_data *helper, struct strbuf *buffer)
@@ -81,6 +85,7 @@ static struct child_process *get_helper(struct transport *transport)
const char **refspecs = NULL;
int refspec_nr = 0;
int refspec_alloc = 0;
+ int duped;
if (data->helper)
return data->helper;
@@ -99,9 +104,19 @@ static struct child_process *get_helper(struct transport *transport)
die("Unable to run helper: git %s", helper->argv[0]);
data->helper = helper;
+ /*
+ * Open the output as FILE* so strbuf_getline() can be used.
+ * Do this with duped fd because fclose() will close the fd,
+ * and stuff like taking over will require the fd to remain.
+ *
+ */
+ duped = dup(helper->out);
+ if (duped < 0)
+ die_errno("Can't dup helper output fd");
+ data->out = xfdopen(duped, "r");
+
write_constant(helper->in, "capabilities\n");
- data->out = xfdopen(helper->out, "r");
while (1) {
const char *capname;
int mandatory = 0;
@@ -163,6 +178,7 @@ static int disconnect_helper(struct transport *transport)
strbuf_addf(&buf, "\n");
sendline(data, &buf);
close(data->helper->in);
+ close(data->helper->out);
fclose(data->out);
finish_command(data->helper);
free((char *)data->helper->argv[0]);
@@ -583,5 +599,6 @@ int transport_helper_init(struct transport *transport, const char *name)
transport->fetch = fetch;
transport->push_refs = push_refs;
transport->disconnect = release_helper;
+ transport->smart_options = &(data->transport_options);
return 0;
}
diff --git a/transport.c b/transport.c
index e6eb20e..ad25b98 100644
--- a/transport.c
+++ b/transport.c
@@ -398,6 +398,7 @@ struct git_transport_data {
struct git_transport_options options;
struct child_process *conn;
int fd[2];
+ unsigned got_remote_heads : 1;
struct extra_have_objects extra_have;
};
@@ -432,10 +433,15 @@ static int set_git_option(struct git_transport_options *opts,
static int connect_setup(struct transport *transport, int for_push, int verbose)
{
struct git_transport_data *data = transport->data;
+
+ if (data->conn)
+ return 0;
+
data->conn = git_connect(data->fd, transport->url,
for_push ? data->options.receivepack :
data->options.uploadpack,
verbose ? CONNECT_VERBOSE : 0);
+
return 0;
}
@@ -447,6 +453,7 @@ static struct ref *get_refs_via_connect(struct transport *transport, int for_pus
connect_setup(transport, for_push, 0);
get_remote_heads(data->fd[0], &refs, 0, NULL,
for_push ? REF_NORMAL : 0, &data->extra_have);
+ data->got_remote_heads = 1;
return refs;
}
@@ -477,9 +484,10 @@ static int fetch_refs_via_pack(struct transport *transport,
for (i = 0; i < nr_heads; i++)
origh[i] = heads[i] = xstrdup(to_fetch[i]->name);
- if (!data->conn) {
+ if (!data->got_remote_heads) {
connect_setup(transport, 0, 0);
get_remote_heads(data->fd[0], &refs_tmp, 0, NULL, 0, NULL);
+ data->got_remote_heads = 1;
}
refs = fetch_pack(&args, data->fd, data->conn,
@@ -490,6 +498,7 @@ static int fetch_refs_via_pack(struct transport *transport,
if (finish_connect(data->conn))
refs = NULL;
data->conn = NULL;
+ data->got_remote_heads = 0;
free_refs(refs_tmp);
@@ -718,12 +727,13 @@ static int git_transport_push(struct transport *transport, struct ref *remote_re
struct send_pack_args args;
int ret;
- if (!data->conn) {
+ if (!data->got_remote_heads) {
struct ref *tmp_refs;
connect_setup(transport, 1, 0);
get_remote_heads(data->fd[0], &tmp_refs, 0, NULL, REF_NORMAL,
NULL);
+ data->got_remote_heads = 1;
}
memset(&args, 0, sizeof(args));
@@ -741,6 +751,7 @@ static int git_transport_push(struct transport *transport, struct ref *remote_re
close(data->fd[0]);
ret |= finish_connect(data->conn);
data->conn = NULL;
+ data->got_remote_heads = 0;
return ret;
}
@@ -749,7 +760,8 @@ static int disconnect_git(struct transport *transport)
{
struct git_transport_data *data = transport->data;
if (data->conn) {
- packet_flush(data->fd[1]);
+ if (data->got_remote_heads)
+ packet_flush(data->fd[1]);
close(data->fd[0]);
close(data->fd[1]);
finish_connect(data->conn);
@@ -759,6 +771,32 @@ static int disconnect_git(struct transport *transport)
return 0;
}
+void transport_take_over(struct transport *transport,
+ struct child_process *child)
+{
+ struct git_transport_data *data;
+
+ if (!transport->smart_options)
+ die("Bug detected: Taking over transport requires non-NULL "
+ "smart_options field.");
+
+ data = xcalloc(1, sizeof(*data));
+ data->options = *transport->smart_options;
+ data->conn = child;
+ data->fd[0] = data->conn->out;
+ data->fd[1] = data->conn->in;
+ data->got_remote_heads = 0;
+ transport->data = data;
+
+ transport->set_option = NULL;
+ transport->get_refs_list = get_refs_via_connect;
+ transport->fetch = fetch_refs_via_pack;
+ transport->push = NULL;
+ transport->push_refs = git_transport_push;
+ transport->disconnect = disconnect_git;
+ transport->smart_options = &(data->options);
+}
+
static int is_local(const char *url)
{
const char *colon = strchr(url, ':');
@@ -867,6 +905,7 @@ struct transport *transport_get(struct remote *remote, const char *url)
ret->smart_options = &(data->options);
data->conn = NULL;
+ data->got_remote_heads = 0;
} else if (!prefixcmp(url, "http://")
|| !prefixcmp(url, "https://")
|| !prefixcmp(url, "ftp://")) {
@@ -927,9 +966,9 @@ int transport_push(struct transport *transport,
*nonfastforward = 0;
verify_remote_names(refspec_nr, refspec);
- if (transport->push)
+ if (transport->push) {
return transport->push(transport, refspec_nr, refspec, flags);
- if (transport->push_refs) {
+ } else if (transport->push_refs) {
struct ref *remote_refs =
transport->get_refs_list(transport, 1);
struct ref *local_refs = get_local_heads();
@@ -973,6 +1012,7 @@ const struct ref *transport_get_remote_refs(struct transport *transport)
{
if (!transport->remote_refs)
transport->remote_refs = transport->get_refs_list(transport, 0);
+
return transport->remote_refs;
}
@@ -1007,6 +1047,7 @@ int transport_fetch_refs(struct transport *transport, struct ref *refs)
}
rc = transport->fetch(transport, nr_heads, heads);
+
free(heads);
return rc;
}
diff --git a/transport.h b/transport.h
index e90c285..781db2e 100644
--- a/transport.h
+++ b/transport.h
@@ -130,6 +130,8 @@ int transport_fetch_refs(struct transport *transport, struct ref *refs);
void transport_unlock_pack(struct transport *transport);
int transport_disconnect(struct transport *transport);
char *transport_anonymize_url(const char *url);
+void transport_take_over(struct transport *transport,
+ struct child_process *child);
/* Transport methods defined outside transport.c */
int transport_helper_init(struct transport *transport, const char *name);
--
1.6.6.rc1.300.gfbc27
^ permalink raw reply related
* [REROLL PATCH v2 4/8] Refactor git transport options parsing
From: Ilari Liusvaara @ 2009-12-09 15:26 UTC (permalink / raw)
To: git
In-Reply-To: <1260372394-16427-1-git-send-email-ilari.liusvaara@elisanet.fi>
Refactor the transport options parsing so that protocols that aren't
directly smart transports (file://, git://, ssh:// & co) can record
the smart transport options for the case if it turns that transport
can actually be smart.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
---
transport.c | 78 +++++++++++++++++++++++++++++++++++-----------------------
transport.h | 15 +++++++++++
2 files changed, 62 insertions(+), 31 deletions(-)
diff --git a/transport.c b/transport.c
index dea37d0..e6eb20e 100644
--- a/transport.c
+++ b/transport.c
@@ -395,41 +395,35 @@ static int close_bundle(struct transport *transport)
}
struct git_transport_data {
- unsigned thin : 1;
- unsigned keep : 1;
- unsigned followtags : 1;
- int depth;
+ struct git_transport_options options;
struct child_process *conn;
int fd[2];
- const char *uploadpack;
- const char *receivepack;
struct extra_have_objects extra_have;
};
-static int set_git_option(struct transport *connection,
+static int set_git_option(struct git_transport_options *opts,
const char *name, const char *value)
{
- struct git_transport_data *data = connection->data;
if (!strcmp(name, TRANS_OPT_UPLOADPACK)) {
- data->uploadpack = value;
+ opts->uploadpack = value;
return 0;
} else if (!strcmp(name, TRANS_OPT_RECEIVEPACK)) {
- data->receivepack = value;
+ opts->receivepack = value;
return 0;
} else if (!strcmp(name, TRANS_OPT_THIN)) {
- data->thin = !!value;
+ opts->thin = !!value;
return 0;
} else if (!strcmp(name, TRANS_OPT_FOLLOWTAGS)) {
- data->followtags = !!value;
+ opts->followtags = !!value;
return 0;
} else if (!strcmp(name, TRANS_OPT_KEEP)) {
- data->keep = !!value;
+ opts->keep = !!value;
return 0;
} else if (!strcmp(name, TRANS_OPT_DEPTH)) {
if (!value)
- data->depth = 0;
+ opts->depth = 0;
else
- data->depth = atoi(value);
+ opts->depth = atoi(value);
return 0;
}
return 1;
@@ -439,7 +433,8 @@ static int connect_setup(struct transport *transport, int for_push, int verbose)
{
struct git_transport_data *data = transport->data;
data->conn = git_connect(data->fd, transport->url,
- for_push ? data->receivepack : data->uploadpack,
+ for_push ? data->options.receivepack :
+ data->options.uploadpack,
verbose ? CONNECT_VERBOSE : 0);
return 0;
}
@@ -469,15 +464,15 @@ static int fetch_refs_via_pack(struct transport *transport,
struct ref *refs_tmp = NULL;
memset(&args, 0, sizeof(args));
- args.uploadpack = data->uploadpack;
- args.keep_pack = data->keep;
+ args.uploadpack = data->options.uploadpack;
+ args.keep_pack = data->options.keep;
args.lock_pack = 1;
- args.use_thin_pack = data->thin;
- args.include_tag = data->followtags;
+ args.use_thin_pack = data->options.thin;
+ args.include_tag = data->options.followtags;
args.verbose = (transport->verbose > 0);
args.quiet = (transport->verbose < 0);
args.no_progress = args.quiet || (!transport->progress && !isatty(1));
- args.depth = data->depth;
+ args.depth = data->options.depth;
for (i = 0; i < nr_heads; i++)
origh[i] = heads[i] = xstrdup(to_fetch[i]->name);
@@ -734,7 +729,7 @@ static int git_transport_push(struct transport *transport, struct ref *remote_re
memset(&args, 0, sizeof(args));
args.send_mirror = !!(flags & TRANSPORT_PUSH_MIRROR);
args.force_update = !!(flags & TRANSPORT_PUSH_FORCE);
- args.use_thin_pack = data->thin;
+ args.use_thin_pack = data->options.thin;
args.verbose = !!(flags & TRANSPORT_PUSH_VERBOSE);
args.quiet = !!(flags & TRANSPORT_PUSH_QUIET);
args.dry_run = !!(flags & TRANSPORT_PUSH_DRY_RUN);
@@ -847,12 +842,14 @@ struct transport *transport_get(struct remote *remote, const char *url)
ret->get_refs_list = get_refs_via_rsync;
ret->fetch = fetch_objs_via_rsync;
ret->push = rsync_transport_push;
+ ret->smart_options = NULL;
} else if (is_local(url) && is_file(url)) {
struct bundle_transport_data *data = xcalloc(1, sizeof(*data));
ret->data = data;
ret->get_refs_list = get_refs_from_bundle;
ret->fetch = fetch_refs_from_bundle;
ret->disconnect = close_bundle;
+ ret->smart_options = NULL;
} else if (!is_url(url)
|| !prefixcmp(url, "file://")
|| !prefixcmp(url, "git://")
@@ -862,20 +859,14 @@ struct transport *transport_get(struct remote *remote, const char *url)
/* These are builtin smart transports. */
struct git_transport_data *data = xcalloc(1, sizeof(*data));
ret->data = data;
- ret->set_option = set_git_option;
+ ret->set_option = NULL;
ret->get_refs_list = get_refs_via_connect;
ret->fetch = fetch_refs_via_pack;
ret->push_refs = git_transport_push;
ret->disconnect = disconnect_git;
+ ret->smart_options = &(data->options);
- data->thin = 1;
data->conn = NULL;
- data->uploadpack = "git-upload-pack";
- if (remote->uploadpack)
- data->uploadpack = remote->uploadpack;
- data->receivepack = "git-receive-pack";
- if (remote->receivepack)
- data->receivepack = remote->receivepack;
} else if (!prefixcmp(url, "http://")
|| !prefixcmp(url, "https://")
|| !prefixcmp(url, "ftp://")) {
@@ -893,14 +884,39 @@ struct transport *transport_get(struct remote *remote, const char *url)
transport_helper_init(ret, handler);
}
+ if (ret->smart_options) {
+ ret->smart_options->thin = 1;
+ ret->smart_options->uploadpack = "git-upload-pack";
+ if (remote->uploadpack)
+ ret->smart_options->uploadpack = remote->uploadpack;
+ ret->smart_options->receivepack = "git-receive-pack";
+ if (remote->receivepack)
+ ret->smart_options->receivepack = remote->receivepack;
+ }
+
return ret;
}
int transport_set_option(struct transport *transport,
const char *name, const char *value)
{
+ int git_reports = 1, protocol_reports = 1;
+
+ if (transport->smart_options)
+ git_reports = set_git_option(transport->smart_options,
+ name, value);
+
if (transport->set_option)
- return transport->set_option(transport, name, value);
+ protocol_reports = transport->set_option(transport, name,
+ value);
+
+ /* If either report is 0, report 0 (success). */
+ if (!git_reports || !protocol_reports)
+ return 0;
+ /* If either reports -1 (invalid value), report -1. */
+ if ((git_reports == -1) || (protocol_reports == -1))
+ return -1;
+ /* Otherwise if both report unknown, report unknown. */
return 1;
}
diff --git a/transport.h b/transport.h
index 9e74406..e90c285 100644
--- a/transport.h
+++ b/transport.h
@@ -4,6 +4,15 @@
#include "cache.h"
#include "remote.h"
+struct git_transport_options {
+ unsigned thin : 1;
+ unsigned keep : 1;
+ unsigned followtags : 1;
+ int depth;
+ const char *uploadpack;
+ const char *receivepack;
+};
+
struct transport {
struct remote *remote;
const char *url;
@@ -65,6 +74,12 @@ struct transport {
signed verbose : 3;
/* Force progress even if the output is not a tty */
unsigned progress : 1;
+ /*
+ * If transport is at least potentially smart, this points to
+ * git_transport_options structure to use in case transport
+ * actually turns out to be smart.
+ */
+ struct git_transport_options *smart_options;
};
#define TRANSPORT_PUSH_ALL 1
--
1.6.6.rc1.300.gfbc27
^ permalink raw reply related
* [REROLL PATCH v2 7/8] Support remote archive from all smart transports
From: Ilari Liusvaara @ 2009-12-09 15:26 UTC (permalink / raw)
To: git
In-Reply-To: <1260372394-16427-1-git-send-email-ilari.liusvaara@elisanet.fi>
Previously, remote archive required internal (non remote-helper)
smart transport. Extend the remote archive to also support smart
transports implemented by remote helpers.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
---
builtin-archive.c | 17 ++++++++++-------
transport-helper.c | 19 +++++++++++++++++++
transport.c | 21 +++++++++++++++++++++
transport.h | 5 +++++
4 files changed, 55 insertions(+), 7 deletions(-)
diff --git a/builtin-archive.c b/builtin-archive.c
index 12351e9..d34b3fd 100644
--- a/builtin-archive.c
+++ b/builtin-archive.c
@@ -5,6 +5,7 @@
#include "cache.h"
#include "builtin.h"
#include "archive.h"
+#include "transport.h"
#include "parse-options.h"
#include "pkt-line.h"
#include "sideband.h"
@@ -25,12 +26,16 @@ static void create_output_file(const char *output_file)
static int run_remote_archiver(int argc, const char **argv,
const char *remote, const char *exec)
{
- char *url, buf[LARGE_PACKET_MAX];
+ char buf[LARGE_PACKET_MAX];
int fd[2], i, len, rv;
- struct child_process *conn;
+ struct transport *transport;
+ struct remote *_remote;
- url = xstrdup(remote);
- conn = git_connect(fd, url, exec, 0);
+ _remote = remote_get(remote);
+ if (!_remote->url[0])
+ die("git archive: Remote with no URL");
+ transport = transport_get(_remote, _remote->url[0]);
+ transport_connect(transport, "git-upload-archive", exec, fd);
for (i = 1; i < argc; i++)
packet_write(fd[1], "argument %s\n", argv[i]);
@@ -53,9 +58,7 @@ static int run_remote_archiver(int argc, const char **argv,
/* Now, start reading from fd[0] and spit it out to stdout */
rv = recv_sideband("archive", fd[0], 1);
- close(fd[0]);
- close(fd[1]);
- rv |= finish_connect(conn);
+ rv |= transport_disconnect(transport);
return !!rv;
}
diff --git a/transport-helper.c b/transport-helper.c
index 216af87..2ce4ef5 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -469,6 +469,24 @@ static int process_connect(struct transport *transport,
return process_connect_service(transport, name, exec);
}
+static int connect_helper(struct transport *transport, const char *name,
+ const char *exec, int fd[2])
+{
+ struct helper_data *data = transport->data;
+
+ /* Get_helper so connect is inited. */
+ get_helper(transport);
+ if (!data->connect)
+ die("Operation not supported by protocol.");
+
+ if (!process_connect_service(transport, name, exec))
+ die("Can't connect to subservice %s.", name);
+
+ fd[0] = data->helper->out;
+ fd[1] = data->helper->in;
+ return 0;
+}
+
static int fetch(struct transport *transport,
int nr_heads, struct ref **to_fetch)
{
@@ -713,6 +731,7 @@ int transport_helper_init(struct transport *transport, const char *name)
transport->fetch = fetch;
transport->push_refs = push_refs;
transport->disconnect = release_helper;
+ transport->connect = connect_helper;
transport->smart_options = &(data->transport_options);
return 0;
}
diff --git a/transport.c b/transport.c
index ad25b98..a7d67eb 100644
--- a/transport.c
+++ b/transport.c
@@ -756,6 +756,17 @@ static int git_transport_push(struct transport *transport, struct ref *remote_re
return ret;
}
+static int connect_git(struct transport *transport, const char *name,
+ const char *executable, int fd[2])
+{
+ struct git_transport_data *data = transport->data;
+ data->conn = git_connect(data->fd, transport->url,
+ executable, 0);
+ fd[0] = data->fd[0];
+ fd[1] = data->fd[1];
+ return 0;
+}
+
static int disconnect_git(struct transport *transport)
{
struct git_transport_data *data = transport->data;
@@ -901,6 +912,7 @@ struct transport *transport_get(struct remote *remote, const char *url)
ret->get_refs_list = get_refs_via_connect;
ret->fetch = fetch_refs_via_pack;
ret->push_refs = git_transport_push;
+ ret->connect = connect_git;
ret->disconnect = disconnect_git;
ret->smart_options = &(data->options);
@@ -1061,6 +1073,15 @@ void transport_unlock_pack(struct transport *transport)
}
}
+int transport_connect(struct transport *transport, const char *name,
+ const char *exec, int fd[2])
+{
+ if (transport->connect)
+ return transport->connect(transport, name, exec, fd);
+ else
+ die("Operation not supported by protocol");
+}
+
int transport_disconnect(struct transport *transport)
{
int ret = 0;
diff --git a/transport.h b/transport.h
index 781db2e..97ba251 100644
--- a/transport.h
+++ b/transport.h
@@ -64,6 +64,8 @@ struct transport {
**/
int (*push_refs)(struct transport *transport, struct ref *refs, int flags);
int (*push)(struct transport *connection, int refspec_nr, const char **refspec, int flags);
+ int (*connect)(struct transport *connection, const char *name,
+ const char *executable, int fd[2]);
/** get_refs_list(), fetch(), and push_refs() can keep
* resources (such as a connection) reserved for futher
@@ -133,6 +135,9 @@ char *transport_anonymize_url(const char *url);
void transport_take_over(struct transport *transport,
struct child_process *child);
+int transport_connect(struct transport *transport, const char *name,
+ const char *exec, int fd[2]);
+
/* Transport methods defined outside transport.c */
int transport_helper_init(struct transport *transport, const char *name);
--
1.6.6.rc1.300.gfbc27
^ permalink raw reply related
* [REROLL PATCH v2 2/8] Support mandatory capabilities
From: Ilari Liusvaara @ 2009-12-09 15:26 UTC (permalink / raw)
To: git
In-Reply-To: <1260372394-16427-1-git-send-email-ilari.liusvaara@elisanet.fi>
Add support for marking capability as mandatory for hosting git version
to understand. This is useful for helpers which require various types
of assistance from main git binary.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
---
Documentation/git-remote-helpers.txt | 5 ++++-
transport-helper.c | 25 +++++++++++++++++++------
2 files changed, 23 insertions(+), 7 deletions(-)
diff --git a/Documentation/git-remote-helpers.txt b/Documentation/git-remote-helpers.txt
index 5cfdc0c..20a05fe 100644
--- a/Documentation/git-remote-helpers.txt
+++ b/Documentation/git-remote-helpers.txt
@@ -25,7 +25,10 @@ Commands are given by the caller on the helper's standard input, one per line.
'capabilities'::
Lists the capabilities of the helper, one per line, ending
- with a blank line.
+ with a blank line. Each capability may be preceeded with '*'.
+ This marks them mandatory for git version using the remote
+ helper to understand (unknown mandatory capability is fatal
+ error).
'list'::
Lists the refs, one per line, in the format "<value> <name>
diff --git a/transport-helper.c b/transport-helper.c
index a721dc2..4b17aaa 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -93,25 +93,38 @@ static struct child_process *get_helper(struct transport *transport)
data->out = xfdopen(helper->out, "r");
while (1) {
+ const char *capname;
+ int mandatory = 0;
recvline(data, &buf);
if (!*buf.buf)
break;
+
+ if (*buf.buf == '*') {
+ capname = buf.buf + 1;
+ mandatory = 1;
+ } else
+ capname = buf.buf;
+
if (debug)
- fprintf(stderr, "Debug: Got cap %s\n", buf.buf);
- if (!strcmp(buf.buf, "fetch"))
+ fprintf(stderr, "Debug: Got cap %s\n", capname);
+ if (!strcmp(capname, "fetch"))
data->fetch = 1;
- if (!strcmp(buf.buf, "option"))
+ else if (!strcmp(capname, "option"))
data->option = 1;
- if (!strcmp(buf.buf, "push"))
+ else if (!strcmp(capname, "push"))
data->push = 1;
- if (!strcmp(buf.buf, "import"))
+ else if (!strcmp(capname, "import"))
data->import = 1;
- if (!data->refspecs && !prefixcmp(buf.buf, "refspec ")) {
+ else if (!data->refspecs && !prefixcmp(capname, "refspec ")) {
ALLOC_GROW(refspecs,
refspec_nr + 1,
refspec_alloc);
refspecs[refspec_nr++] = strdup(buf.buf + strlen("refspec "));
+ } else if (mandatory) {
+ die("Unknown madatory capability %s. This remote "
+ "helper probably needs newer version of Git.\n",
+ capname);
}
}
if (refspecs) {
--
1.6.6.rc1.300.gfbc27
^ permalink raw reply related
* [REROLL PATCH v2 8/8] Remove special casing of http, https and ftp
From: Ilari Liusvaara @ 2009-12-09 15:26 UTC (permalink / raw)
To: git
In-Reply-To: <1260372394-16427-1-git-send-email-ilari.liusvaara@elisanet.fi>
HTTP, HTTPS and FTP are no longer special to transport code. Also
add support for FTPS (curl supports it so it is easy).
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
---
.gitignore | 4 ++++
Makefile | 27 +++++++++++++++++++++++++--
transport.c | 8 --------
3 files changed, 29 insertions(+), 10 deletions(-)
diff --git a/.gitignore b/.gitignore
index ac02a58..aa7a8ac 100644
--- a/.gitignore
+++ b/.gitignore
@@ -107,6 +107,10 @@
/git-relink
/git-remote
/git-remote-curl
+/git-remote-http
+/git-remote-https
+/git-remote-ftp
+/git-remote-ftps
/git-repack
/git-replace
/git-repo-config
diff --git a/Makefile b/Makefile
index 2ad7e36..87fc7ff 100644
--- a/Makefile
+++ b/Makefile
@@ -424,6 +424,16 @@ BUILT_INS += git-stage$X
BUILT_INS += git-status$X
BUILT_INS += git-whatchanged$X
+ifdef NO_CURL
+REMOTE_CURL_PRIMARY =
+REMOTE_CURL_ALIASES =
+REMOTE_CURL_NAMES =
+else
+REMOTE_CURL_PRIMARY = git-remote-http$X
+REMOTE_CURL_ALIASES = git-remote-https$X git-remote-ftp$X git-remote-ftps$X
+REMOTE_CURL_NAMES = $(REMOTE_CURL_PRIMARY) $(REMOTE_CURL_ALIASES)
+endif
+
# what 'all' will build and 'install' will install in gitexecdir,
# excluding programs for built-in commands
ALL_PROGRAMS = $(PROGRAMS) $(SCRIPTS)
@@ -1097,7 +1107,7 @@ else
else
CURL_LIBCURL = -lcurl
endif
- PROGRAMS += git-remote-curl$X git-http-fetch$X
+ PROGRAMS += $(REMOTE_CURL_NAMES) git-http-fetch$X
curl_check := $(shell (echo 070908; curl-config --vernum) | sort -r | sed -ne 2p)
ifeq "$(curl_check)" "070908"
ifndef NO_EXPAT
@@ -1676,7 +1686,13 @@ git-http-push$X: revision.o http.o http-push.o $(GITLIBS)
$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
$(LIBS) $(CURL_LIBCURL) $(EXPAT_LIBEXPAT)
-git-remote-curl$X: remote-curl.o http.o http-walker.o $(GITLIBS)
+$(REMOTE_CURL_ALIASES): $(REMOTE_CURL_PRIMARY)
+ $(QUIET_LNCP)$(RM) $@ && \
+ ln $< $@ 2>/dev/null || \
+ ln -s $< $@ 2>/dev/null || \
+ cp $< $@
+
+$(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o $(GITLIBS)
$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
$(LIBS) $(CURL_LIBCURL) $(EXPAT_LIBEXPAT)
@@ -1852,6 +1868,7 @@ endif
ifneq (,$X)
$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_PROGRAMS) $(BUILT_INS) git$X)), test '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p' -ef '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p$X' || $(RM) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p';)
endif
+
bindir=$$(cd '$(DESTDIR_SQ)$(bindir_SQ)' && pwd) && \
execdir=$$(cd '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' && pwd) && \
{ test "$$bindir/" = "$$execdir/" || \
@@ -1865,6 +1882,12 @@ endif
ln -s "git$X" "$$execdir/$$p" 2>/dev/null || \
cp "$$execdir/git$X" "$$execdir/$$p" || exit; \
done; } && \
+ { for p in $(REMOTE_CURL_ALIASES); do \
+ $(RM) "$$execdir/$$p" && \
+ ln "$$execdir/git-remote-http$X" "$$execdir/$$p" 2>/dev/null || \
+ ln -s "git-remote-http$X" "$$execdir/$$p" 2>/dev/null || \
+ cp "$$execdir/git-remote-http$X" "$$execdir/$$p" || exit; \
+ done; } && \
./check_bindir "z$$bindir" "z$$execdir" "$$bindir/git-add$X"
install-doc:
diff --git a/transport.c b/transport.c
index a7d67eb..652bf0b 100644
--- a/transport.c
+++ b/transport.c
@@ -918,14 +918,6 @@ struct transport *transport_get(struct remote *remote, const char *url)
data->conn = NULL;
data->got_remote_heads = 0;
- } else if (!prefixcmp(url, "http://")
- || !prefixcmp(url, "https://")
- || !prefixcmp(url, "ftp://")) {
- /* These three are just plain special. */
- transport_helper_init(ret, "curl");
-#ifdef NO_CURL
- error("git was compiled without libcurl support.");
-#endif
} else {
/* Unknown protocol in URL. Pass to external handler. */
int len = external_specification_len(url);
--
1.6.6.rc1.300.gfbc27
^ permalink raw reply related
* [REROLL PATCH v2 6/8] Support remote helpers implementing smart transports
From: Ilari Liusvaara @ 2009-12-09 15:26 UTC (permalink / raw)
To: git
In-Reply-To: <1260372394-16427-1-git-send-email-ilari.liusvaara@elisanet.fi>
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
---
Documentation/git-remote-helpers.txt | 25 +++++++-
transport-helper.c | 126 ++++++++++++++++++++++++++++++++--
2 files changed, 144 insertions(+), 7 deletions(-)
diff --git a/Documentation/git-remote-helpers.txt b/Documentation/git-remote-helpers.txt
index 20a05fe..4685a89 100644
--- a/Documentation/git-remote-helpers.txt
+++ b/Documentation/git-remote-helpers.txt
@@ -93,6 +93,20 @@ Supported if the helper has the "push" capability.
+
Supported if the helper has the "import" capability.
+'connect' <service>::
+ Connects to given service. Standard input and standard output
+ of helper are connected to specified service (git prefix is
+ included in service name so e.g. fetching uses 'git-upload-pack'
+ as service) on remote side. Valid replies to this command are
+ empty line (connection established), 'fallback' (no smart
+ transport support, fall back to dumb transports) and just
+ exiting with error message printed (can't connect, don't
+ bother trying to fall back). After line feed terminating the
+ positive (empty) response, the output of service starts. After
+ the connection ends, the remote helper exits.
++
+Supported if the helper has the "connect" capability.
+
If a fatal error occurs, the program writes the error message to
stderr and exits. The caller should expect that a suitable error
message has been printed if the child closes the connection without
@@ -126,6 +140,9 @@ CAPABILITIES
all, it must cover all refs reported by the list command; if
it is not used, it is effectively "*:*"
+'connect'::
+ This helper supports the 'connect' command.
+
REF LIST ATTRIBUTES
-------------------
@@ -168,9 +185,15 @@ OPTIONS
but don't actually change any repository data. For most
helpers this only applies to the 'push', if supported.
+'option servpath <c-style-quoted-path>'::
+ Set service path (--upload-pack, --receive-pack etc.) for
+ next connect. Remote helper MAY support this option. Remote
+ helper MUST NOT rely on this option being set before
+ connect request occurs.
+
Documentation
-------------
-Documentation by Daniel Barkalow.
+Documentation by Daniel Barkalow and Ilari Liusvaara
GIT
---
diff --git a/transport-helper.c b/transport-helper.c
index 97eed6c..216af87 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -18,7 +18,9 @@ struct helper_data
unsigned fetch : 1,
import : 1,
option : 1,
- push : 1;
+ push : 1,
+ connect : 1,
+ no_disconnect_req : 1;
/* These go from remote name (as in "list") to private name */
struct refspec *refspecs;
int refspec_nr;
@@ -37,12 +39,12 @@ static void sendline(struct helper_data *helper, struct strbuf *buffer)
die_errno("Full write to remote helper failed");
}
-static int recvline(struct helper_data *helper, struct strbuf *buffer)
+static int recvline_fh(FILE *helper, struct strbuf *buffer)
{
strbuf_reset(buffer);
if (debug)
fprintf(stderr, "Debug: Remote helper: Waiting...\n");
- if (strbuf_getline(buffer, helper->out, '\n') == EOF) {
+ if (strbuf_getline(buffer, helper, '\n') == EOF) {
if (debug)
fprintf(stderr, "Debug: Remote helper quit.\n");
exit(128);
@@ -53,6 +55,11 @@ static int recvline(struct helper_data *helper, struct strbuf *buffer)
return 0;
}
+static int recvline(struct helper_data *helper, struct strbuf *buffer)
+{
+ return recvline_fh(helper->out, buffer);
+}
+
static void xchgline(struct helper_data *helper, struct strbuf *buffer)
{
sendline(helper, buffer);
@@ -77,6 +84,15 @@ const char *remove_ext_force(const char *url)
return url;
}
+static void do_take_over(struct transport *transport)
+{
+ struct helper_data *data;
+ data = (struct helper_data*)transport->data;
+ transport_take_over(transport, data->helper);
+ fclose(data->out);
+ free(data);
+}
+
static struct child_process *get_helper(struct transport *transport)
{
struct helper_data *data = transport->data;
@@ -103,12 +119,12 @@ static struct child_process *get_helper(struct transport *transport)
if (start_command(helper))
die("Unable to run helper: git %s", helper->argv[0]);
data->helper = helper;
+ data->no_disconnect_req = 0;
/*
* Open the output as FILE* so strbuf_getline() can be used.
* Do this with duped fd because fclose() will close the fd,
* and stuff like taking over will require the fd to remain.
- *
*/
duped = dup(helper->out);
if (duped < 0)
@@ -146,6 +162,8 @@ static struct child_process *get_helper(struct transport *transport)
refspec_nr + 1,
refspec_alloc);
refspecs[refspec_nr++] = strdup(buf.buf + strlen("refspec "));
+ } else if (!strcmp(capname, "connect")) {
+ data->connect = 1;
} else if (mandatory) {
die("Unknown madatory capability %s. This remote "
"helper probably needs newer version of Git.\n",
@@ -175,8 +193,10 @@ static int disconnect_helper(struct transport *transport)
if (data->helper) {
if (debug)
fprintf(stderr, "Debug: Disconnecting.\n");
- strbuf_addf(&buf, "\n");
- sendline(data, &buf);
+ if(!data->no_disconnect_req) {
+ strbuf_addf(&buf, "\n");
+ sendline(data, &buf);
+ }
close(data->helper->in);
close(data->helper->out);
fclose(data->out);
@@ -370,12 +390,96 @@ static int fetch_with_import(struct transport *transport,
return 0;
}
+static int process_connect_service(struct transport *transport,
+ const char *name, const char *exec)
+{
+ struct helper_data *data = transport->data;
+ struct strbuf cmdbuf = STRBUF_INIT;
+ struct child_process *helper;
+ int r, duped, ret = 0;
+ FILE *input;
+
+ helper = get_helper(transport);
+
+ /*
+ * Yes, dup the pipe another time, as we need unbuffered version
+ * of input pipe as FILE*. fclose() closes the underlying fd and
+ * stream buffering only can be changed before first I/O operation
+ * on it.
+ */
+ duped = dup(helper->out);
+ if (duped < 0)
+ die_errno("Can't dup helper output fd");
+ input = xfdopen(duped, "r");
+ setvbuf(input, NULL, _IONBF, 0);
+
+ /*
+ * Handle --upload-pack and friends. This is fire and forget...
+ * just warn if it fails.
+ */
+ if (strcmp(name, exec)) {
+ r = set_helper_option(transport, "servpath", exec);
+ if (r > 0)
+ fprintf(stderr, "Warning: Setting remote service path "
+ "not supported by protocol.\n");
+ else if (r < 0)
+ fprintf(stderr, "Warning: Invalid remote service "
+ "path.\n");
+ }
+
+ if (data->connect)
+ strbuf_addf(&cmdbuf, "connect %s\n", name);
+ else
+ goto exit;
+
+ sendline(data, &cmdbuf);
+ recvline_fh(input, &cmdbuf);
+ if (!strcmp(cmdbuf.buf, "")) {
+ data->no_disconnect_req = 1;
+ if (debug)
+ fprintf(stderr, "Debug: Smart transport connection "
+ "ready.\n");
+ ret = 1;
+ } else if (!strcmp(cmdbuf.buf, "fallback")) {
+ if (debug)
+ fprintf(stderr, "Debug: Falling back to dumb "
+ "transport.\n");
+ } else
+ die("Unknown response to connect: %s",
+ cmdbuf.buf);
+
+exit:
+ fclose(input);
+ return ret;
+}
+
+static int process_connect(struct transport *transport,
+ int for_push)
+{
+ struct helper_data *data = transport->data;
+ const char *name;
+ const char *exec;
+
+ name = for_push ? "git-receive-pack" : "git-upload-pack";
+ if (for_push)
+ exec = data->transport_options.receivepack;
+ else
+ exec = data->transport_options.uploadpack;
+
+ return process_connect_service(transport, name, exec);
+}
+
static int fetch(struct transport *transport,
int nr_heads, struct ref **to_fetch)
{
struct helper_data *data = transport->data;
int i, count;
+ if (process_connect(transport, 0)) {
+ do_take_over(transport);
+ return transport->fetch(transport, nr_heads, to_fetch);
+ }
+
count = 0;
for (i = 0; i < nr_heads; i++)
if (!(to_fetch[i]->status & REF_STATUS_UPTODATE))
@@ -403,6 +507,11 @@ static int push_refs(struct transport *transport,
struct child_process *helper;
struct ref *ref;
+ if (process_connect(transport, 1)) {
+ do_take_over(transport);
+ return transport->push_refs(transport, remote_refs, flags);
+ }
+
if (!remote_refs)
return 0;
@@ -543,6 +652,11 @@ static struct ref *get_refs_list(struct transport *transport, int for_push)
helper = get_helper(transport);
+ if (process_connect(transport, for_push)) {
+ do_take_over(transport);
+ return transport->get_refs_list(transport, for_push);
+ }
+
if (data->push && for_push)
write_str_in_full(helper->in, "list for-push\n");
else
--
1.6.6.rc1.300.gfbc27
^ permalink raw reply related
* cannot pread pack file
From: Steven A. Falco @ 2009-12-09 16:25 UTC (permalink / raw)
To: git
We have been using git for a year or two now, and suddenly, one
of our users got the error message:
fatal: cannot pread pack file: No such file or directory
fatal: index-pack failed
while doing a git-pull. We are using git version git-1.6.5.5.
I tried doing a "git fsck --full" on his repository and on the
one he is trying to pull from, and neither shows any error.
If I clone his repository, the clone also gets the error message
when trying to pull.
I can pull from him, but if he pulls from me, he gets the error
message.
This is worrying, because the fsck doesn't show an error, but
clearly something is wrong.
Is there anything further I can try to determine why his repo
is unable to pull?
Thanks,
Steve
--
A: Because it makes the logic of the discussion difficult to follow.
Q: Why shouldn't I top post?
A: No.
Q: Should I top post?
^ permalink raw reply
* Re: cannot pread pack file
From: Nicolas Pitre @ 2009-12-09 17:43 UTC (permalink / raw)
To: Steven A. Falco; +Cc: git
In-Reply-To: <4B1FCF81.2010807@harris.com>
On Wed, 9 Dec 2009, Steven A. Falco wrote:
> We have been using git for a year or two now, and suddenly, one
> of our users got the error message:
>
> fatal: cannot pread pack file: No such file or directory
> fatal: index-pack failed
>
> while doing a git-pull. We are using git version git-1.6.5.5.
>
> I tried doing a "git fsck --full" on his repository and on the
> one he is trying to pull from, and neither shows any error.
>
> If I clone his repository, the clone also gets the error message
> when trying to pull.
>
> I can pull from him, but if he pulls from me, he gets the error
> message.
>
> This is worrying, because the fsck doesn't show an error, but
> clearly something is wrong.
A few questions:
- What is the OS on which the failing Git is running?
- How long before the error occurs i.e. how many objects are received
(and the transfer size) before the error?
Nicolas
^ permalink raw reply
* Delete particular file contents from complete history
From: Patrick Grimard @ 2009-12-09 18:01 UTC (permalink / raw)
To: git
Is there some way to remove from the entire history a particular
file's content? So for example I did an initial commit which
contained a file I didn't want in the commit and since then I've
committed further changes to the project, even started a branch to
experiment with some additional features. I don't want to lose any of
my new commits or branch, just want to remove a particular file's
content from the entire history.
Thanks.
^ permalink raw reply
* git init respecting GIT_WORK_DIR
From: Tony Risinger @ 2009-12-09 18:37 UTC (permalink / raw)
To: git, Tony Risinger
should this be expected behavior?
-----DOC
core.bare
(snipped)
This setting is automatically guessed by git-clone(1) or
git-init(1) when the repository was created. By default a repository
that ends in "/.git" is assumed to be not bare (bare = false),
while all other repositories are assumed to be bare (bare = true).
-----PROBLEM
[cr@extOFme-d0 ~]$ export GIT_DIR=gitdir.git GIT_WORK_DIR=workdir.work
[cr@extOFme-d0 ~]$ git init
Initialized empty Git repository in /home/cr/gitdir.git/
[cr@extOFme-d0 ~]$ git config core.bare
true
[cr@extOFme-d0 ~]$ git config core.worktree
[cr@extOFme-d0 ~]$
-----SUMMARY
i think git should not be so restrictive in what it considers to be a
bare repository. when you use --work-tree:
[cr@extOFme-d0 ~]$ export GIT_DIR=gitdir.git GIT_WORK_DIR=workdir.work
[cr@extOFme-d0 ~]$ git --work-tree="$GIT_WORK_DIR" init
Initialized empty Git repository in /home/cr/gitdir.git/
[cr@extOFme-d0 ~]$ git config core.bare
false
[cr@extOFme-d0 ~]$ git config core.worktree
/home/cr/workdir.work
everything works as expected. "git init" should respect GIT_WORK_DIR
env variable and set core.bare and core.worktree appropriately.
^ permalink raw reply
* Re: cannot pread pack file
From: Steven A. Falco @ 2009-12-09 18:53 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: git
In-Reply-To: <alpine.LFD.2.00.0912091237330.31174@xanadu.home>
Nicolas Pitre wrote:
> On Wed, 9 Dec 2009, Steven A. Falco wrote:
>
>> We have been using git for a year or two now, and suddenly, one
>> of our users got the error message:
>>
>> fatal: cannot pread pack file: No such file or directory
>> fatal: index-pack failed
>>
>> while doing a git-pull. We are using git version git-1.6.5.5.
>>
>> I tried doing a "git fsck --full" on his repository and on the
>> one he is trying to pull from, and neither shows any error.
>>
>> If I clone his repository, the clone also gets the error message
>> when trying to pull.
>>
>> I can pull from him, but if he pulls from me, he gets the error
>> message.
>>
>> This is worrying, because the fsck doesn't show an error, but
>> clearly something is wrong.
>
> A few questions:
>
> - What is the OS on which the failing Git is running?
>
I am running on Centos 5.2. I believe I've fixed the problem.
I thought we were running 1.6.5.5, but apparently our libexec was
pointing to an older version. Once I corrected that, the error
went away.
Our repositories are hosted on an NFS file server, and the repo
that had the problem has some large binary fpga images in it.
Looking on the net, it seems that there was a problem with pread
returning part of a file. I see that the new code does the pread
in a loop to work around this. So I'm guessing that the old version
we were running didn't have the pread workaround and was therefore
failing. Anyway, it is behaving now.
Thanks very much for replying,
Steve
> - How long before the error occurs i.e. how many objects are received
> (and the transfer size) before the error?
>
>
> Nicolas
>
--
A: Because it makes the logic of the discussion difficult to follow.
Q: Why shouldn't I top post?
A: No.
Q: Should I top post?
^ permalink raw reply
* Re: [PATCH RFC] rebase: add --revisions flag
From: Junio C Hamano @ 2009-12-09 19:46 UTC (permalink / raw)
To: Björn Steinbrink
Cc: Michael S. Tsirkin, Andreas Schwab, Peter Krefting,
Git Mailing List
In-Reply-To: <20091209130653.GA30218@atjola.homenet>
Björn Steinbrink <B.Steinbrink@gmx.de> writes:
> "merge --squash" is one of the things I really dislike, because it turns
> off the "history" part of the merge. You can say "Merging in git is about
> histories, merging in svn is about changes only" to describe the major
> difference for the merge commands in the two systems... "But then
> there's --squash which turns git into svn".
I agree with this to some degree, but I do not offhand think of a better
alternative.
At the first sight, it looks as if what "merge --squash" does was
implemented as a new option "--squash" to the "merge" command merely
because the way _how_ it internally needs to compute the result was
already available in the implementation of "merge" command, and not
necessarily because _what_ it does was conceptually consistent with the
way "merge" works.
But at the conceptual level, "merge --squash" is a short-hand for this
command sequence:
git rebase -i HEAD that-branch
... make everything except the first one into "squash"
git checkout - ;# come back to the original branch
git merge that-branch ;# fast forward to it
So after all, it is "merge it after squashing them".
^ permalink raw reply
* Re: Git documentation consistency
From: Greg A. Woods @ 2009-12-09 19:56 UTC (permalink / raw)
To: The Git Mailing List
In-Reply-To: <e51f66da0912030822ye1541b4gb1b8a3e07eb72484@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2174 bytes --]
At Thu, 3 Dec 2009 18:22:27 +0200, Marko Kreen <markokr@gmail.com> wrote:
Subject: Re: Git documentation consistency
>
> On 12/3/09, Greg A. Woods <woods@planix.com> wrote:
> > At Wed, 02 Dec 2009 17:34:01 -0800, Junio C Hamano <gitster@pobox.com> wrote:
> > Subject: Re: Git documentation consistency
> > > I think you are showing ignorance here, as -? is *not* even close to
> > > standard, nor even widely used practice at all.
> >
> > I think I should know something about Unix command line and option
> > parsers, having used them for some 25 years or so now. In fact I've
> > used most every kind of unix that ever was, and I've worked on the
> > source to more than a few.
>
> '?' is what getopt(3) is supposed to return for unknown options.
Indeed, which is why it cannot ever, in general, be used as a valid
option with some command-specific meaning, and so why the one-line form
of the usage message should always be displayed when the user explicitly
gives a '-?' option (i.e. in the same way as if any unknown option is
given).
If the one-line usage message is too terse to explain more complex
command line syntax then '-h' (and --help) should display a short
multi-line summary of the command's usage.
I guess what I should have suggested in the first place is that all Git
sub-commands should respond with a one-line[*] usage message when they
encounter an unknown option, such as '-?', and that they should (only)
display a more detailed multi-line help summary when given '-h' or
'--help' options.
I was most surprised when I didn't get a one-line usage summary from
"git log -?", just getopt(3)'s error message.
[*] commands which have multiple distinct modes of operation with
separate and unique command-line syntax for each mode, should of course
display a one-line summary for each command mode, such as in this
example:
void
usage(void)
{
fprintf(stderr, "Usage: %s [-abcdef]\n", getprogname());
fprintf(stderr, " %s [-l]\n", getprogname());
exit(2);
}
--
Greg A. Woods
Planix, Inc.
<woods@planix.com> +1 416 218 0099 http://www.planix.com/
[-- Attachment #2: Type: application/pgp-signature, Size: 186 bytes --]
^ permalink raw reply
* Help: approach for rebasing to older commits after merging more recent commits
From: Jay Soffian @ 2009-12-09 20:04 UTC (permalink / raw)
To: git
I have an interesting problem I'm not sure how best to tackle.
A small development team is basing its product on an upstream git repo that is
itself an svn clone. Currently the process looks like this:
r1--r2--r3--r4--r5 upstream trunk (git svn clone)
\ \ \
A---B---C---D---E local trunk (git clone of upstream)
\ /
F---G developerN trunk (git clones of local)
So local trunk has both daily merges from the local developers, as well as
less periodic (typically weekly) merges from upstream trunk. The reason being
that it is necessary to remain on top of the upstream bleeding edge.
This works out okay, but there is a minor problem and a major problem.
The minor problem is that the local trunk is cluttered with the developerN
merges. That is easy to solve by having local developers rebase before pushing
to local trunk. That would look like:
r1--r2--r3--r4--r5 upstream trunk
\ \ \
A---B---C---D---E---F'---G' local trunk
The major problem is that local trunk is also cluttered with merges from
upstream. The is a problem because at some point in the future, upstream
is going to declare some rN as being officially blessed. And we're going to
want to rewind any rN changes past that point.
So the question is, what's the best way to do this? Say r2 is blessed by
upstream. The obvious thing to do (I think...) is:
(local-trunk)$ git rebase -i r2
removing C and E from the pick list.
But, occassionally the merges from upstream require much conflict resolution.
Would enabling rerere during merges help with the rebasing? I would want to
reuse as much conflict resolution as possible.
Is there a better approach altogether? Should we be doing something other
than merging to stay atop upstream?
Suggestions/comments greatly appreciated.
j.
^ permalink raw reply
* Re: [PATCH RFC] rebase: add --revisions flag
From: Junio C Hamano @ 2009-12-09 20:10 UTC (permalink / raw)
To: Matthieu Moy; +Cc: Michael S. Tsirkin, git
In-Reply-To: <vpqiqcgp95t.fsf@bauges.imag.fr>
Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:
> Perhaps "git rebase --cherry-pick A..B" would be a better name. No
> objection for --opposite either.
As somebody mentioned, "--cherry-picking", similar to "am --rebasing" that
is also an unadvertised internal implementation detail, is a good name.
In the very old days, the original "rebase" command was a command to
"set-up and run 'am -3' command in order to transplant the current branch
onto somewhere else."
That was obviously too long for a command name and description, and I
shorten it to "rebase", to name the command after what it is _for_ (not
"what it does", nor "how it does it"). Because the command was to "set up
and run am-3", it was natural at the conceptual level that the way to
continue after a failed/conflicted patch application was "am --resolved".
But since then, the concept of "rebasing" got established more firmly and
how "rebase" can be done has become much less relevant. The original name
"rebase" stopped being short for "set up and run am-3 for rebasing", but
about what it _does_ (i.e. "it rebases"). And "rebase --continue" has
become a natural way to drive "am --continue" at that point, to accomodate
for the change in the end-user conception. These days, "set up and run
am-3 for rebasing" is not even _how_ it does the "rebase", as "rebase -i"
does not even use am-3. So "rebase --continue" was a logical conclusion
of the command's evolution.
The lesson to be learned from this history is that "cherry-pick A..B" that
internally runs "rebase --cherry-picking" will need a similar "--continue"
support that delegates to "rebase". "running am", which was originally
the whole point of "rebase", later became an implementation detail, and we
needed to teach "--continue" to "rebase" at that point.
Because from day one "running rebase -i" will be an implementation detail
of "cherry-pick A..B", we need to teach "cherry-pick" to pass "--abort",
"--continue", etc. to underlying "rebase" for the same reason.
^ permalink raw reply
* Re: [BUG] Bad msysgit/egit interaction over dotfiles
From: Marius Storm-Olsen @ 2009-12-09 20:56 UTC (permalink / raw)
To: Yann Dirson; +Cc: kusmabite, Ferry Huberts, GIT ml
In-Reply-To: <20091208143700.GC5425@linagora.com>
Yann Dirson said the following on 08.12.2009 15:37:
> On Tue, Dec 08, 2009 at 03:23:55PM +0100, Erik Faye-Lund wrote:
>> You can follow the discussion here:
>> http://code.google.com/p/msysgit/issues/detail?id=288
>>
>> I believe the reason is something like "because someone suggested
>> it, and no one disagreed". Do you have a good argument why it
>> shouldn't be the default (other than "it's a change", because
>> changing it back now would also be a change)?
>
> Depending on the opinion of the Eclipse guys on this issue about
> "writing to hidden files only says 'could not write'", which
> arguably could be seen as a bug on their side, we can see changing
> this behaviour back to the default on the msysgit side as either a
> (possibly temporary) workaround for a known eclipse bug, or as
> getting again interoperable with egit.
Dot-files on unix are considered hidden. It's the only way files are
hidden there. Not so on Windows. Dot-files are just like any normal
file, and you need to mark a file hidden.
So, the logic of egit, that *actually* hidden files should not be
written to, but dot-files should, seems to me to be a bug in egit.
There should be no reason why egit shouldn't be able to write to any
file, pending permissions. I'd say file a bug report with egit.
--
.marius
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox