* Ŭŭ letter
From: Usievaład Čorny @ 2012-10-31 14:39 UTC (permalink / raw)
To: git
Hello!
When I type comment message in Git Gui (1.8.0 and previous), I can't
use Ŭŭ letter (U+016C, U+016D) — it just transforms into simple Uu.
Please fix it.
--
Z pavahaj, Usievaład Čorny
^ permalink raw reply
* Re: Git clone fails with "bad pack header", how to get remote log
From: kevin molcard @ 2012-10-31 14:34 UTC (permalink / raw)
To: Jeff King
Cc: Konstantin Khomoutov, git-users-/JYPxA39Uh5TLH3MbocFFw,
git-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20121031141955.GC24291-bBVMEuqLR+SYVEpFpFwlB0AkDMvbqDRI@public.gmane.org>
I forgot to mention that I am using scm manager:
https://bitbucket.org/sdorra/scm-manager/wiki/Home
So that maybe the " custom layer you are talking about.
Kevin
On 10/31/12 3:19 PM, Jeff King wrote:
> On Tue, Oct 30, 2012 at 03:57:36PM +0100, kevin molcard wrote:
>
>> I tried to install git 1.8 on the remote server and get exactly the
>> same problem :(.
>> [...]
>>>> Sometimes (very often when several git clone are sent at the same
>>>> time), I have the following error:
>>>> remote: internal server error
>>>> fatal: protocol error: bad pack header
> I'm very confused about who is printing "internal server error". The
> "remote:" indicates that it came to the git client via the sideband,
> which means it probably came from the stderr of a child process (e.g.,
> pack-objects). But git does not and has never generated the phrase
> "internal server error".
>
> So what program is producing that? Is there some kind of custom layer
> that might be run when upload-pack runs "git pack-objects ..."? Can you
> try running strace on the server?
>
> -Peff
>
--
^ permalink raw reply
* Re: Git clone fails with "bad pack header", how to get remote log
From: kevin molcard @ 2012-10-31 14:25 UTC (permalink / raw)
To: Jeff King
Cc: Konstantin Khomoutov, git-users-/JYPxA39Uh5TLH3MbocFFw,
git-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20121031141955.GC24291-bBVMEuqLR+SYVEpFpFwlB0AkDMvbqDRI@public.gmane.org>
Yes I can,
can you tell me how I have to do that?
thanks
Kevin
On 10/31/12 3:19 PM, Jeff King wrote:
> On Tue, Oct 30, 2012 at 03:57:36PM +0100, kevin molcard wrote:
>
>> I tried to install git 1.8 on the remote server and get exactly the
>> same problem :(.
>> [...]
>>>> Sometimes (very often when several git clone are sent at the same
>>>> time), I have the following error:
>>>> remote: internal server error
>>>> fatal: protocol error: bad pack header
> I'm very confused about who is printing "internal server error". The
> "remote:" indicates that it came to the git client via the sideband,
> which means it probably came from the stderr of a child process (e.g.,
> pack-objects). But git does not and has never generated the phrase
> "internal server error".
>
> So what program is producing that? Is there some kind of custom layer
> that might be run when upload-pack runs "git pack-objects ..."? Can you
> try running strace on the server?
>
> -Peff
>
--
^ permalink raw reply
* Fwd: Re: [git-users] Git clone fails with "bad pack header", how to get remote log
From: kevin molcard @ 2012-10-31 14:20 UTC (permalink / raw)
To: git
In-Reply-To: <20121031152837.4c0f8ce84fae14ba36429605@domain007.com>
Hi all again,
here is my second email :).
It contains the git versions in my system.
FYI, I updated git to 1.8.0 on my remote but still having the same issue.
Another thing that might be interesting is that it seems to happen only
when cloning from Windows build machine (i.e. I send 2 clone command on
from the mac and 2 from the Windows and it seems to always fails on the
Windows).
Thanks again
Kevin
-------- Original Message --------
Subject: Re: [git-users] Git clone fails with "bad pack header", how to
get remote log
Date: Wed, 31 Oct 2012 15:28:37 +0400
From: Konstantin Khomoutov <flatworm@users.sourceforge.net>
To: kevin molcard <kev2041@gmail.com>
CC: git-users@googlegroups.com
On Tue, 30 Oct 2012 08:01:12 +0100
kevin molcard <kev2041@gmail.com> wrote:
> thanks for the reply.
> The versions of git are:
> - on remote: 1.5.6.5
> - on windows build machine: 1.7.11.msysgit.1
> - on mac build machine: 1.7.3.4
>
> I will try to install latest git version on my remote server and get
> back to you.
Hi, Kevin.
I noticed my Cc'ed messages did not reach the main Git list for some
reason (even though I did see these my messages coming from the
list-management software), and the did not show up on Gmane [1].
I do not know what's the reason is, so please try forwarding my first
reply to your original message and the message I'm replying to now
(with versions of software involved) to that list yourself -- maybe
you'll be more lucky.
Sorry for the delay.
1. http://thread.gmane.org/gmane.comp.version-control.git/
^ permalink raw reply
* Re: [git-users] Git clone fails with "bad pack header", how to get remote log
From: Jeff King @ 2012-10-31 14:19 UTC (permalink / raw)
To: kevin molcard; +Cc: Konstantin Khomoutov, git-users, git
In-Reply-To: <508FEAE0.20204@gmail.com>
On Tue, Oct 30, 2012 at 03:57:36PM +0100, kevin molcard wrote:
> I tried to install git 1.8 on the remote server and get exactly the
> same problem :(.
> [...]
> >>Sometimes (very often when several git clone are sent at the same
> >>time), I have the following error:
> >> remote: internal server error
> >> fatal: protocol error: bad pack header
I'm very confused about who is printing "internal server error". The
"remote:" indicates that it came to the git client via the sideband,
which means it probably came from the stderr of a child process (e.g.,
pack-objects). But git does not and has never generated the phrase
"internal server error".
So what program is producing that? Is there some kind of custom layer
that might be run when upload-pack runs "git pack-objects ..."? Can you
try running strace on the server?
-Peff
^ permalink raw reply
* Fwd: Re: [git-users] Git clone fails with "bad pack header", how to get remote log
From: kevin molcard @ 2012-10-31 14:17 UTC (permalink / raw)
To: git
In-Reply-To: <20121029211854.b58c791d30a6c8d68665e574@domain007.com>
Hi all,
I am forwarding a reply I got from a message I sent to git user mailing
list because of a "bad pack header error" (more information below).
I will forward another email where I give all the git versions of my system.
Any clue on this would be much appreciated.
Thanks in advance
Kevin
-------- Original Message --------
Subject: Re: [git-users] Git clone fails with "bad pack header", how to
get remote log
Date: Mon, 29 Oct 2012 21:18:54 +0400
From: Konstantin Khomoutov <flatworm@users.sourceforge.net>
To: git-users@googlegroups.com
CC: Kevin Molcard <kev2041@gmail.com>, git@vger.kernel.org
On Mon, 29 Oct 2012 09:52:54 -0700 (PDT)
Kevin Molcard <kev2041@gmail.com> wrote:
> I have a problem with my build system.
>
> I have a remote server with a relatively large repository (around 12
> GB, each branch having a size of 3 GB).
>
> I have also 2 build servers (Mac, Windows) that are cloning the repo
> from the remote.
>
> Sometimes (very often when several git clone are sent at the same
> time), I have the following error:
>
> remote: internal server error
> fatal: protocol error: bad pack header
>
> I know that it happens when the remote is compressing objects (thanks
> to `--progress -v` flags) because the last line of the log before the
> erro is:
> remote: Compressing objects: 93% (17959/19284) [K
>
> * So I have 2 questions, does anybody what is the problem and what
> should I do?
> * Is there a way to get a more precise log from the remote to debug
> this problem?
This reminds me of a bug fixed in 1.7.12.1 [1]:
* When "git push" triggered the automatic gc on the receiving end, a
message from "git prune" that said it was removing cruft leaked to
the standard output, breaking the communication protocol.
In any case, bugs should be reported to the main Git list (which is
git at vger.kernel.org), not here.
I'm Cc'ing the main Git list so you'll get any responses from there, if
any.
Kevin, please answer to this message (keeping all the Ccs -- use "Reply
to group" or "Reply to all" in your MUA) and describe exactly what Git
versions on which platforms your have.
1. https://raw.github.com/git/git/master/Documentation/RelNotes/1.7.12.1.txt
^ permalink raw reply
* Re: [PATCH v6 2/3] completion: add new __gitcompadd helper
From: SZEDER Gábor @ 2012-10-31 14:15 UTC (permalink / raw)
To: Felipe Contreras; +Cc: git, Junio C Hamano, Jeff King, Matthieu Moy
In-Reply-To: <CAMP44s0dctpjobNNRTOFcX4ir+nzenTZMNWFbEvBa-QU93psbA@mail.gmail.com>
On Wed, Oct 31, 2012 at 12:05:34AM +0100, Felipe Contreras wrote:
> On Tue, Oct 30, 2012 at 11:58 PM, SZEDER Gábor <szeder@ira.uka.de> wrote:
> > On Mon, Oct 22, 2012 at 03:45:41AM +0200, Felipe Contreras wrote:
> >> The idea is to never touch the COMPREPLY variable directly.
> >>
> >> This allows other completion systems override __gitcompadd, and do
> >> something different instead.
> >>
> >> Also, this allows the simplification of the completion tests (separate
> >> patch).
> >
> > This doesn't apply anymore, does it? The mentioned simplification is
> > done in the other series.
>
> Yeah, but you mentioned you didn't like all the COMPREPLY=() changes
> and it might be time to get rid of them.
>
> So this series supersedes that one.
COMPREPLY=() has nothing to do with it. My point is that there is no
"separate patch" that performs the alleged simplification made
possible by this patch, therefore that sentence should have been
removed from the log message for the resubmission.
^ permalink raw reply
* Re: [OT] How to get the discussion details via notes
From: Jeff King @ 2012-10-31 14:10 UTC (permalink / raw)
To: Peter Baumann; +Cc: Thomas Rast, Jonathan Nieder, git
In-Reply-To: <20121031095327.GB18557@m62s10.vlinux.de>
On Wed, Oct 31, 2012 at 10:53:27AM +0100, Peter Baumann wrote:
> > covers the basics (current behavior and intent of the change) in its
> > first two paragraphs and anyone wanting more detail can use
> >
> > GIT_NOTES_REF=refs/remotes/charon/notes/full \
> > git show --show-notes <commit>
> >
> > to find more details.
>
> I seem to miss something here, but I don't get it how the notes ref
> becomes magically filled with the details of this discussion.
Thomas Rast (aka charon) keeps a mapping of commits to the email threads
that led to them. You can fetch it from:
git://repo.or.cz/git/trast.git
(try the notes/full and notes/terse refs).
-Peff
^ permalink raw reply
* Re: [PATCH] parse_dirstat_params(): use string_list to split comma-separated string
From: Jeff King @ 2012-10-31 14:06 UTC (permalink / raw)
To: Matt Kraai; +Cc: git
In-Reply-To: <loom.20121030T193428-242@post.gmane.org>
On Tue, Oct 30, 2012 at 06:43:51PM +0000, Matt Kraai wrote:
> Michael Haggerty <mhagger <at> alum.mit.edu> writes:
> ...
> > -static int parse_dirstat_params(struct diff_options *options, const char ...
> > +static int parse_dirstat_params(struct diff_options *options, const char ...
> > struct strbuf *errmsg)
> > {
> > - const char *p = params;
> > - int p_len, ret = 0;
> > + char *params_copy = xstrdup(params_string);
> > + struct string_list params = STRING_LIST_INIT_NODUP;
> > + int ret = 0;
> > + int i;
> >
> > - while (*p) {
> > - p_len = strchrnul(p, ',') - p;
> > - if (!memcmp(p, "changes", p_len)) {
> > + if (*params_copy)
>
> params_copy is set to the value returned by xstrdup, which cannot be NULL.
> This check can be removed and if params_string can be NULL, it should be
> checked before being passed to xstrdup.
If you are referring to the last line, isn't it checking whether the
string is empty, not NULL?
-Peff
^ permalink raw reply
* Re: git-p4 clone @all error
From: Thomas Berg @ 2012-10-31 14:01 UTC (permalink / raw)
To: Arthur; +Cc: git
In-Reply-To: <1351593879401-7570219.post@n2.nabble.com>
Hi,
Sorry, forgot to reply-to-all, here is my response again:
On Tue, Oct 30, 2012 at 11:44 AM, Arthur <a.foulon@amesys.fr> wrote:
> The problem :
>
> Importing revision 7727 (100%)Traceback (most recent call last):
> File "/usr/bin/git-p4", line 3183, in <module>
> main()
> File "/usr/bin/git-p4", line 3177, in main
> if not cmd.run(args):
> File "/usr/bin/git-p4", line 3048, in run
> if not P4Sync.run(self, depotPaths):
> File "/usr/bin/git-p4", line 2911, in run
> self.importChanges(changes)
> File "/usr/bin/git-p4", line 2618, in importChanges
> self.initialParent)
> File "/usr/bin/git-p4", line 2198, in commit
> epoch = details["time"]
> KeyError: 'time'
>
Are you permanently converting a project, or are you planning to
continue submitting to perforce with git-p4?
I have seen similar bugs myself when using the --detect-branches
option. The branch detection in git-p4 is flaky anyway: it is limited
what it can handle, and it used to require correct perforce branch
specs at least, so I would recommend not using it unless you know what
it is doing under the hood.
Instead I would just clone a single branch at a time (drop the
--detect-branches) and work on that.
I do this even in the rare cases when I need more than one perforce
branch in the same git repo - there are other ways to achieve the same
thing.
- Thomas
^ permalink raw reply
* Re: change symlink
From: Jeff King @ 2012-10-31 13:44 UTC (permalink / raw)
To: shawn wilson; +Cc: Andreas Schwab, git
In-Reply-To: <20121031123057.GE30879@sigill.intra.peff.net>
On Wed, Oct 31, 2012 at 08:30:57AM -0400, Jeff King wrote:
> Something like this seems to fix it for me, but I am not sure if that
> would affect other callers.
> [...]
> + return !is_dir || !S_ISGITLINK(istate->cache[pos]->ce_mode);
That's completely wrong, of course. It should be:
return is_dir && !S_ISGITLINK(...);
(we found an index entry, so if it isn't a directory, then we know that
it is not untracked, and should return 0).
With that, we at least pass the test suite.
-Peff
^ permalink raw reply
* Re: git-p4 clone @all error
From: Arthur @ 2012-10-31 13:18 UTC (permalink / raw)
To: git
In-Reply-To: <1351593879401-7570219.post@n2.nabble.com>
up
--
View this message in context: http://git.661346.n2.nabble.com/git-p4-clone-all-error-tp7570219p7570343.html
Sent from the git mailing list archive at Nabble.com.
^ permalink raw reply
* Re: [PATCH] git-push: update remote tags only with force
From: Chris Rorvick @ 2012-10-31 12:52 UTC (permalink / raw)
To: Drew Northup
Cc: git, Felipe Contreras, Jeff King, Michael Haggerty,
Angelo Borsotti, Philip Oakley, Johannes Sixt, Kacper Kornet
In-Reply-To: <CAM9Z-nk6YRtNXNHbp-ReXB78V0O01qng+zmWfzm7Yxz51x22Yw@mail.gmail.com>
On Wed, Oct 31, 2012 at 7:26 AM, Drew Northup <n1xim.email@gmail.com> wrote:
> On Wed, Oct 31, 2012 at 1:55 AM, Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
>
>> but I wonder if it might be
>> possible to split it to ease the review.
>
> Agreed.
I'll look at splitting it up, probably not tonight though. :-)
> Also, do please CC ALL interested parties from the pre-patch
> discussion thread as well as those who previously maintained that
> chunk of code.
>
> [Attempted to reconstruct CC list of discussion]
Thanks for fixing that.
Chris
^ permalink raw reply
* Re: [PATCH] git-push: update remote tags only with force
From: Chris Rorvick @ 2012-10-31 12:36 UTC (permalink / raw)
To: Felipe Contreras; +Cc: git
In-Reply-To: <CAMP44s2T9Rmfjd8r+2+eYh8JBPXEofm3cHuEkkY+R3cW6R6HxA@mail.gmail.com>
(oops, now my email was rejected)
On Wed, Oct 31, 2012 at 12:55 AM, Felipe Contreras
<felipe.contreras@gmail.com> wrote:
> Hi,
>
> (again because the mailing list rejected it) (Gmal switched interface
> and HTML is the default)
>
> On Wed, Oct 31, 2012 at 6:37 AM, Chris Rorvick <chris@rorvick.com> wrote:
>>
>> References are allowed to update from one commit-ish to another if the
>> former is a ancestor of the latter. This behavior is oriented to
>> branches which are expected to move with commits. Tag references are
>> expected to be static in a repository, though, thus an update to a
>> tag (lightweight and annotated) should be rejected unless the update is
>> forced.
>>
>> To enable this functionality, the following checks have been added to
>> set_ref_status_for_push() for updating refs (i.e, not new or deletion)
>> to restrict fast-forwarding in pushes:
>>
>> 1) The old and new references must be commits. If this fails,
>> it is not a valid update for a branch.
>>
>> 2) The reference name cannot start with "refs/tags/". This
>> catches lightweight tags which (usually) point to commits
>> and therefore would not be caught by (1).
>>
>> If either of these checks fails, then it is flagged (by default) with a
>> status indicating the update is being rejected due to the reference
>> already existing in the remote. This can be overridden by passing
>> --force to git push.
>>
>> The new status has the added benefit of being able to provide accurate
>> feedback as to why the ref update failed and what can be done.
>> Currently all ref update rejections are assumed to be for branches.
>
> Makes sense to me. I've believe I've been hit by this a couple of
> times when tags were updated, and a colleague did 'git push' and they
> went all back, or something like that. To handle that case properly
> probably more changes are needed, but this is a change in the right
> direction.
>
>> +test_expect_success 'push tag requires --force to update remote tag' '
>> + mk_test heads/master &&
>> + mk_child child1 &&
>> + mk_child child2 &&
>> + (
>> + cd child1 &&
>> + git tag lw_tag &&
>> + git tag -a -m "message 1" ann_tag &&
>> + git push ../child2 lw_tag &&
>> + git push ../child2 ann_tag &&
>> + >file1 &&
>> + git add file1 &&
>> + git commit -m "file1" &&
>> + git tag -f lw_tag &&
>> + git tag -f -a -m "message 2" ann_tag &&
>> + ! git push ../child2 lw_tag &&
>
> You probably should use test_must_fail.
Thanks, will fix.
> I don't see anything wrong with the patch, but I wonder if it might be
> possible to split it to ease the review.
I initially thought I'd split it into two: 1) to improve the feedback
and 2) to change the behavior. But (1) was shaping up to be similar
in size to the sum so I scrapped that idea. I will see what I can do.
Thanks,
Chris
^ permalink raw reply
* Re: change symlink
From: Jeff King @ 2012-10-31 12:30 UTC (permalink / raw)
To: shawn wilson; +Cc: Andreas Schwab, git
In-Reply-To: <20121031120505.GD30879@sigill.intra.peff.net>
On Wed, Oct 31, 2012 at 08:05:05AM -0400, Jeff King wrote:
> > however (what got me started wondering about this and a point i forgot
> > about) - t2/one/test doesn't show up under 'untracked files' in in
> > status that scenario. shouldn't it?
>
> Yes, I think that is a bug.
>
> My guess is that the presence of "test" in the index fools us from
> descending into the directory. And indeed, if you try "git status
> -uall", you will see the untracked file. So it is something with the
> directory traversal cutoff in the regular "-unormal" mode.
Something like this seems to fix it for me, but I am not sure if that
would affect other callers.
diff --git a/read-cache.c b/read-cache.c
index fda78bc..ae04a61 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1882,11 +1882,22 @@ int index_name_is_other(const struct index_state *istate, const char *name,
int namelen)
{
int pos;
- if (namelen && name[namelen - 1] == '/')
+ int is_dir = 0;
+
+ if (namelen && name[namelen - 1] == '/') {
namelen--;
+ is_dir = 1;
+ }
pos = index_name_pos(istate, name, namelen);
- if (0 <= pos)
- return 0; /* exact match */
+ if (0 <= pos) {
+ /* We got an exact match. However, if it is a directory,
+ * we still have to check that the entry in the index
+ * is a directory, too. If it isn't, then the old file went
+ * away, and now we may have untracked files inside the newly
+ * created directory.
+ */
+ return !is_dir || !S_ISGITLINK(istate->cache[pos]->ce_mode);
+ }
pos = -pos - 1;
if (pos < istate->cache_nr) {
struct cache_entry *ce = istate->cache[pos];
^ permalink raw reply related
* Re: [OT] How to get the discussion details via notes
From: Drew Northup @ 2012-10-31 12:29 UTC (permalink / raw)
To: Peter Baumann; +Cc: Jonathan Nieder, git
In-Reply-To: <20121031095327.GB18557@m62s10.vlinux.de>
On Wed, Oct 31, 2012 at 5:53 AM, Peter Baumann <waste.manager@gmx.de> wrote:
> Dropping the Cc list, as this is off topic
>
> On Tue, Oct 30, 2012 at 11:05:29PM -0700, Jonathan Nieder wrote:
>> Sverre Rabbelier wrote:
>>
>> > Thanks for the thorough explanation. Perhaps some of that could make
>> > it's way into the commit message?
>>
>> It's fine with me if it doesn't, since the original commit message
>> covers the basics (current behavior and intent of the change) in its
>> first two paragraphs and anyone wanting more detail can use
>>
>> GIT_NOTES_REF=refs/remotes/charon/notes/full \
>> git show --show-notes <commit>
>>
>> to find more details.
>
> I seem to miss something here, but I don't get it how the notes ref
> becomes magically filled with the details of this discussion.
>
> Care to explain?
If I have an email thread I'd like to store alongside a commit I'll
put that into a note, but I usually don't push that kind of thing out
to a remote repo.
Does that help?
--
-Drew Northup
--------------------------------------------------------------
"As opposed to vegetable or mineral error?"
-John Pescatore, SANS NewsBites Vol. 12 Num. 59
^ permalink raw reply
* Re: [PATCH] git-push: update remote tags only with force
From: Drew Northup @ 2012-10-31 12:26 UTC (permalink / raw)
To: Chris Rorvick
Cc: git, Felipe Contreras, Jeff King, Michael Haggerty,
Angelo Borsotti, Philip Oakley, Johannes Sixt, Kacper Kornet
In-Reply-To: <CAMP44s2T9Rmfjd8r+2+eYh8JBPXEofm3cHuEkkY+R3cW6R6HxA@mail.gmail.com>
On Wed, Oct 31, 2012 at 1:55 AM, Felipe Contreras
<felipe.contreras@gmail.com> wrote:
> but I wonder if it might be
> possible to split it to ease the review.
Agreed. Also, do please CC ALL interested parties from the pre-patch
discussion thread as well as those who previously maintained that
chunk of code.
[Attempted to reconstruct CC list of discussion]
--
-Drew Northup
--------------------------------------------------------------
"As opposed to vegetable or mineral error?"
-John Pescatore, SANS NewsBites Vol. 12 Num. 59
^ permalink raw reply
* Re: change symlink
From: Jeff King @ 2012-10-31 12:05 UTC (permalink / raw)
To: shawn wilson; +Cc: Andreas Schwab, git
In-Reply-To: <CAH_OBidmrJsmw1QQ2WONieA1EQmS_Y4WJ8Mu2Mh90tEPU0uWgw@mail.gmail.com>
On Tue, Oct 30, 2012 at 10:22:04PM +0000, shawn wilson wrote:
> On Tue, Oct 30, 2012 at 10:09 PM, Andreas Schwab <schwab@linux-m68k.org> wrote:
> > shawn wilson <ag4ve.us@gmail.com> writes:
> >
> >> and once it's added, status says:
> >> # renamed: t2 -> t2/one/test
> >>
> >> that's not exactly true, but...
> >
> > What's wrong with it? Both files have the same contents, which is the
> > link target for symlinks.
> >
>
> ok, you're right about that (another test where i changed where the
> symlink pointed):
> # deleted: test
> # new file: test/one/t3
>
>
> however (what got me started wondering about this and a point i forgot
> about) - t2/one/test doesn't show up under 'untracked files' in in
> status that scenario. shouldn't it?
Yes, I think that is a bug.
My guess is that the presence of "test" in the index fools us from
descending into the directory. And indeed, if you try "git status
-uall", you will see the untracked file. So it is something with the
directory traversal cutoff in the regular "-unormal" mode.
-Peff
^ permalink raw reply
* [PATCH 2/2] remote-curl: retry failed requests for auth even with gzip
From: Jeff King @ 2012-10-31 12:01 UTC (permalink / raw)
To: 乙酸鋰; +Cc: git
In-Reply-To: <20121031115346.GC30879@sigill.intra.peff.net>
Commit b81401c taught the post_rpc function to retry the
http request after prompting for credentials. However, it
did not handle two cases:
1. If we have a large request, we do not retry. That's OK,
since we would have sent a probe (with retry) already.
2. If we are gzipping the request, we do not retry. That
was considered OK, because the intended use was for
push (e.g., listing refs is OK, but actually pushing
objects is not), and we never gzip on push.
This patch teaches post_rpc to retry even a gzipped request.
This has two advantages:
1. It is possible to configure a "half-auth" state for
fetching, where the set of refs and their sha1s are
advertised, but one cannot actually fetch objects.
This is not a recommended configuration, as it leaks
some information about what is in the repository (e.g.,
an attacker can try brute-forcing possible content in
your repository and checking whether it matches your
branch sha1). However, it can be slightly more
convenient, since a no-op fetch will not require a
password at all.
2. It future-proofs us should we decide to ever gzip more
requests.
Signed-off-by: Jeff King <peff@peff.net>
---
I doubt we would ever want to gzip push requests. The bulk of the data
is objects, which are already compressed. In theory we could gzip the
non-pack parts, but that would first mean splitting them into a separate
request. The tiny bit of savings are almost certainly not worth the
complexity.
But the future-proofing might still be valuable if we ever modify the
protocol to have a new phase.
remote-curl.c | 11 ++++++++++-
t/lib-httpd/apache.conf | 7 +++++++
t/t5551-http-fetch.sh | 15 +++++++++++++++
3 files changed, 32 insertions(+), 1 deletion(-)
diff --git a/remote-curl.c b/remote-curl.c
index 10cd47d..fac2bef 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -474,6 +474,15 @@ retry:
fflush(stderr);
}
+ } else if (gzip_body) {
+ /*
+ * If we are looping to retry authentication, then the previous
+ * run will have set up the headers and gzip buffer already,
+ * and we just need to send it.
+ */
+ curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDS, gzip_body);
+ curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDSIZE, gzip_size);
+
} else if (use_gzip && 1024 < rpc->len) {
/* The client backend isn't giving us compressed data so
* we can try to deflate it ourselves, this may save on.
@@ -530,7 +539,7 @@ retry:
curl_easy_setopt(slot->curl, CURLOPT_FILE, rpc);
err = run_slot(slot);
- if (err == HTTP_REAUTH && !large_request && !use_gzip)
+ if (err == HTTP_REAUTH && !large_request)
goto retry;
if (err != HTTP_OK)
err = -1;
diff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf
index ec8618d..15a3c71 100644
--- a/t/lib-httpd/apache.conf
+++ b/t/lib-httpd/apache.conf
@@ -96,6 +96,13 @@ SSLEngine On
Require valid-user
</LocationMatch>
+<LocationMatch "^/auth-fetch/.*/git-upload-pack$">
+ AuthType Basic
+ AuthName "git-auth"
+ AuthUserFile passwd
+ Require valid-user
+</LocationMatch>
+
<IfDefine DAV>
LoadModule dav_module modules/mod_dav.so
LoadModule dav_fs_module modules/mod_dav_fs.so
diff --git a/t/t5551-http-fetch.sh b/t/t5551-http-fetch.sh
index 7380f2a..5f174da 100755
--- a/t/t5551-http-fetch.sh
+++ b/t/t5551-http-fetch.sh
@@ -129,6 +129,21 @@ test_expect_success 'clone from auth-only-for-push repository' '
test_cmp expect actual
'
+test_expect_success 'clone from auth-only-for-objects repository' '
+ echo two >expect &&
+ set_askpass user@host &&
+ git clone --bare "$HTTPD_URL/auth-fetch/smart/repo.git" half-auth &&
+ expect_askpass both user@host &&
+ git --git-dir=half-auth log -1 --format=%s >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'no-op half-auth fetch does not require a password' '
+ set_askpass wrong &&
+ git --git-dir=half-auth fetch &&
+ expect_askpass none
+'
+
test -n "$GIT_TEST_LONG" && test_set_prereq EXPENSIVE
test_expect_success EXPENSIVE 'create 50,000 tags in the repo' '
--
1.8.0.207.gdf2154c
^ permalink raw reply related
* [PATCH 1/2] remote-curl: hoist gzip buffer size to top of post_rpc
From: Jeff King @ 2012-10-31 11:55 UTC (permalink / raw)
To: 乙酸鋰; +Cc: git
In-Reply-To: <20121031115346.GC30879@sigill.intra.peff.net>
When we gzip the post data for a smart-http rpc request, we
compute the gzip body and its size inside the "use_gzip"
conditional. We keep track of the body after the conditional
ends, but not the size. Let's remember both, which will
enable us to retry failed gzip requests in a future patch.
Signed-off-by: Jeff King <peff@peff.net>
---
This is a tiny change conceptually, but the name change makes the diff
quite noisy. Thus I pulled it into a separate patch from 2/2.
remote-curl.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/remote-curl.c b/remote-curl.c
index aefafd3..10cd47d 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -413,6 +413,7 @@ static int post_rpc(struct rpc_state *rpc)
struct curl_slist *headers = NULL;
int use_gzip = rpc->gzip_request;
char *gzip_body = NULL;
+ size_t gzip_size;
int err, large_request = 0;
/* Try to load the entire request, if we can fit it into the
@@ -478,19 +479,18 @@ retry:
* we can try to deflate it ourselves, this may save on.
* the transfer time.
*/
- size_t size;
git_zstream stream;
int ret;
memset(&stream, 0, sizeof(stream));
git_deflate_init_gzip(&stream, Z_BEST_COMPRESSION);
- size = git_deflate_bound(&stream, rpc->len);
- gzip_body = xmalloc(size);
+ gzip_size = git_deflate_bound(&stream, rpc->len);
+ gzip_body = xmalloc(gzip_size);
stream.next_in = (unsigned char *)rpc->buf;
stream.avail_in = rpc->len;
stream.next_out = (unsigned char *)gzip_body;
- stream.avail_out = size;
+ stream.avail_out = gzip_size;
ret = git_deflate(&stream, Z_FINISH);
if (ret != Z_STREAM_END)
@@ -500,16 +500,16 @@ retry:
if (ret != Z_OK)
die("cannot deflate request; zlib end error %d", ret);
- size = stream.total_out;
+ gzip_size = stream.total_out;
headers = curl_slist_append(headers, "Content-Encoding: gzip");
curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDS, gzip_body);
- curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDSIZE, size);
+ curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDSIZE, gzip_size);
if (options.verbosity > 1) {
fprintf(stderr, "POST %s (gzip %lu to %lu bytes)\n",
rpc->service_name,
- (unsigned long)rpc->len, (unsigned long)size);
+ (unsigned long)rpc->len, (unsigned long)gzip_size);
fflush(stderr);
}
} else {
--
1.8.0.207.gdf2154c
^ permalink raw reply related
* Re: git smart-http do not authent to allow git ls-remote to be called anonymously
From: Jeff King @ 2012-10-31 11:53 UTC (permalink / raw)
To: 乙酸鋰; +Cc: git
In-Reply-To: <CAHtLG6SsMgwquaD_Rd0YvR9Ues-u1ktEdOXC2MAfEM9Naac=5g@mail.gmail.com>
[+cc git@vger; please keep discussion on the list]
On Sun, Oct 28, 2012 at 01:26:51PM +0800, 乙酸鋰 wrote:
> > POST /git/Cat1/SubCat1/xsp.git/git-upload-pack HTTP/1.1
> User-Agent: git/1.8.0
> Host: localhost
> Accept-Encoding: gzip
> Content-Type: application/x-git-upload-pack-request
> Accept: application/x-git-upload-pack-result
> Content-Length: 190
>
> * The requested URL returned error: 401
> * Closing connection #0
> Username for 'http://localhost': user
> Password for 'http://user@localhost':
> fatal: The remote end hung up unexpectedly
OK, I see what is going on. The code in b81401c to retry POST requests
does not handle gzipped contents, and upload-pack tends to gzip what it
sends.
Your apache configuration is not really something that we ever intended
to support, and I am a little dubious of the security tradeoff being
made. But it is actually pretty easy for us to support, and it
eliminates a special case from the code, so I am tempted to do so.
The following patch series (on top of the current master, as they
require some cleanup that did not make it into 1.8.0) seems to fix it
for me.
[1/2]: remote-curl: hoist gzip buffer size to top of post_rpc
[2/2]: remote-curl: retry failed requests for auth even with gzip
-Peff
^ permalink raw reply
* Re: relative objects/info/alternates doesn't work on remote SMB repo
From: Nguyen Thai Ngoc Duy @ 2012-10-31 11:41 UTC (permalink / raw)
To: Orgad Shaneh; +Cc: git, msysGit
In-Reply-To: <CAGHpTB+o4gHfgFgLqie_hbjzWjxL94xRQi11GwS9F-Qhik0qVA@mail.gmail.com>
On Tue, Oct 30, 2012 at 4:28 PM, Orgad Shaneh
> Any news? This still doesn't work with 1.8.0.
Nope, sorry. It's still in my todo list.
--
Duy
^ permalink raw reply
* Re: Links broken in ref docs.
From: Kevin @ 2012-10-31 11:17 UTC (permalink / raw)
To: Mike Norman; +Cc: Holger Hellmuth (IKS), Scott Chacon, Andrew Ardill, git
In-Reply-To: <CAJr+XPH4MTg_F_YuT=R1mLVwOUYPB0US3w2mZ1+DDvrQV7vVfQ@mail.gmail.com>
Looks fixed here too
On Wed, Oct 31, 2012 at 12:30 AM, Mike Norman <mknorman@gmail.com> wrote:
> I just checked and the issue seems to be fixed! Clicked around on a
> bunch of previously broken links and they work!
>
> On Tue, Oct 30, 2012 at 3:38 AM, Holger Hellmuth (IKS)
> <hellmuth@ira.uka.de> wrote:
>> Am 30.10.2012 09:07, schrieb Mike Norman:
>>
>>> Not seen any recently. I'm guessing the dev is in the path of
>>> hurricane Sandy? (Not sarcasm, btw.)
>>
>>
>> Do you still see failures? I checked out the website just now and it seemed
>> to work flawlessly (at least the links I tried, could not find any Sharing
>> or Updating section). New design since I last visited, more end-user polish.
>>
>>
^ permalink raw reply
* Re: Workflow for templates?
From: Josef Wolf @ 2012-10-31 10:44 UTC (permalink / raw)
To: git
In-Reply-To: <3190de06-2eaf-4a39-91aa-9cc34c20fc8e@zcs>
On Sat, Oct 27, 2012 at 08:45:45PM +0200, Enrico Weigelt wrote:
> I'd suggest a 3 level branch hierachy (IOW: the lower level
> is rebased ontop of the next higher level):
>
> * #0: upstream branch
> * #1: generic local maintenance branch
> * #2: per-instance cutomization branches
>
> Normal additions go to the lowest level #2. When you've got
> some generic commit, you propagate it to the next level
> (cherry-pick) and rebase layer #2 ontop of it.
> Now you can send your layer #1 to upstream for integration.
>
> When upstream updated his branch, you simply rebase #1
> ontop of it, do your checks etc, then proceed to rebasing #3.
>
> You could also introduce more intermediate layers (eg when you've
> got different groups of similar instance that share certain changes)
Thanks for the suggestion, Enrico!
I am somewhat unsure whether it would work this way. After all, there seems to
be an unbreakable rule with git: never rebase published branches.
Thus, once I have published my work to other people who also need to work on
the same localizations as I do, I have no longer the option of rebasing to get
rid of the localizations and put the generic template stuff for upstream.
I guess, my concern is because I have not yet fully understood the problems of
rebasing, and how to recover from them.
Maybe I should try to explain the problem in terms of repository
hierarchy. Let's assume, there is this hierarchy of repositories:
upstream: central repository, containing the generic template
foo-site: repository for site foo. Here we have localizations for a specific
administrative entity named foo (say, google).
This is where clones for production are made from, and production
boxes pull from here to be kept up-to-date.
foo-prodA: A clone of foo-site, put in production and pulling from a specific
branch on foo-site to receive released, blessed updates.
foo-prodB: Similar to foo-prodA, but on another box.
foo-devA: A clone of foo-site to make development, releases, and whatever for
foo.
foo-devB: One more clone of foo-site, Developer B is working here.
Then, we might have more administrative entities: bar-site, bar-prodA,
bar-prodB, bar-devA, bar-devB, for example. This might be Microsoft, for
example.
Further, foo-devA might be the same person as bar-devA.
So when foo-devA pulls from foo-devB, then foo-devB will create problems when
he rebases after that pull.
I think I have some kind of misunderstanding here, but I just can't figure
what it is.
Maybe I should try to explain the problem in yet other words:
What I am trying to achieve, is to extend the workflow from development to
deployment across multiple administrative entities. As a picture:
upstream (templates only).
^
|
v
development (configured, might contain experimental changes)
^
|
v
deployment (configured)
This workflow should not stop at administrative borders. Just replace foo by
google and bar by Microsoft to get an idea of what I am trying to achieve.
^ permalink raw reply
* Re: [PATCH v4 00/13] New remote-hg helper
From: Jeff King @ 2012-10-31 10:27 UTC (permalink / raw)
To: git
Cc: Michael J Gruber, Felipe Contreras, Johannes Schindelin,
Junio C Hamano, Sverre Rabbelier, Ilari Liusvaara,
Daniel Barkalow
In-Reply-To: <5090EFCA.7070606@drmicha.warpmail.net>
On Wed, Oct 31, 2012 at 10:30:50AM +0100, Michael J Gruber wrote:
> For the record, Johannes is not the only one being kept from looking at
> this series (further) by the tone of this discussion. Per hominem
> attacks are neither professional nor helpful. We prefer to discuss code
> here, just code. From my comments on an earlier version of your series
> you can see I've tried. The way other comment threads on this series
> unfolded made me choose to be a mere by-stander again.
Me too. I really like some of the directions the series is taking, and
as the maintainer, I'd like to pick it up. But there is a big question
mark for me still about how it relates to the work in msysgit,
especially:
- What advantages does this implementation have over the one in
msysgit (i.e., new features that the other one does not have)?
- What disadvantages? If this implementation goes into git.git,
the msysgit one is likely to wane in popularity. What will we be
losing by doing so? If the answer is not "nothing", how hard would
it be to port over the missing bits?
- The msysgit one got held up by fixes needed for fast-export. Why
aren't those a problem for this implementation? If we are using a
different strategy that avoids the issue, what are the limitations
(if any) of that strategy?
I have a feeling that some of those answers are buried deep within the
discussion, but I have had a hard time following all of the back and
forth due to the volume and tone of the discussion. Are we at a point
now where some of the participants can try to summarize the situation?
I am not saying that this implementation must be 100% better than the
msysgit one. I do not want perfect to to be the enemy of good and end up
with nothing. But at the same time, there really are two competing
implementations, one of which has received substantially more field use.
Even though the msysgit one is not in git.git, it seems like the path
for making it happen exists (even if it has not been followed yet).
Before merging an alternative implementation, I would want to know what
we are potentially throwing away from the msysgit side, and make sure
that we are not following a wrong path that msysgit has already tried
and found to be lacking.
-Peff
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox