* Re: cg-clone, tag objects and cg-push/git-push don't play nice
From: Junio C Hamano @ 2005-10-19 8:52 UTC (permalink / raw)
To: Martin Langhoff; +Cc: git
In-Reply-To: <46a038f90510190110g53c90c5t419ad6065292269e@mail.gmail.com>
Martin Langhoff <martin.langhoff@gmail.com> writes:
>> That is expected.
>
> Hmmm. I was under the impression that if I call git-push naming a
> particular head, it could restrict itself to the stuff needed for that
> head only. Just to clarify, I'm running
>
> git-push locke.catalyst.net.nz:/var/git/moodle-test.git master:mdl-topnz-prod
Yeah. But the problem is that the repository on the other end
claims that it has everything reachable from those incomplete
tags. Hearing that, git-push (the real name of it is send-pack)
decides not to send things that are already reachable from the
refs the other end claims to have. So if an incomplete tag
refers to a commit that contains a blob that the remote actually
does not have, and if that blob is part of the head you are
pushing, send-pack would not (and should not) send that blob to
the remote.
>> I do not understand why removing .git/refs/tags/* did not help,
>> and that is the biggest thing that disturbs me in this whole
>> problem report
>
> I can't understand that either, but I manually removed all the
> refs/tags, and the only heads I have are origin and master. git-push
> won't let me do it until the exact point where I have removed the
> object from the repo. And that's only possible on unpacked repos.
Hmph. It worries me even more.
This error message:
error: unpack should have generated
482d4b88aa482dfea7f7549470902049a050020a, but I can't find it!
comes from receive-pack that runs on the other repo (i.e. the
one with incomplete tags you just removed), so it means
send-pack decided it does not need to send that object --
meaning the other end claimed it already has it. What to send
and what need not to be sent is determined solely based on what
send-pack hears from receive-pack in the initial handshake,
which is in receive-pack.c::write_head_info(). It scans
everything under ".git/refs" directory (not just .git/refs/heads
or .git/refs/tags; if you had ".git/refs/FOOBAR", it will cause
the remote end to claim it has everything reachable from it --
the only exceptions are things that starts with a dot '.', which
is probably why cg-fetch places a temporary heads in
refs/*/.$name-fetching) and sends them -- it does not look at
the object directory and magically claim it has something that
is not recorded in its .git/refs/ directory.
^ permalink raw reply
* Re: git-daemon enabled on kernel.org
From: Anton Altaparmakov @ 2005-10-19 8:56 UTC (permalink / raw)
To: Erik Mouw; +Cc: H. Peter Anvin, Git Mailing List
In-Reply-To: <20051019083542.GA31526@harddisk-recovery.com>
On Wed, 19 Oct 2005, Erik Mouw wrote:
> On Tue, Oct 18, 2005 at 12:30:23PM -0700, H. Peter Anvin wrote:
> > After getting gitweb behind mod_cache, the load on kernel.org has gotten
> > down into the tolerable range, so I have enabled git-daemon in an
> > attempt to fix that :)
> >
> > The URL, obviously, is git://git.kernel.org/pub/scm/...
> >
> > (or, to specify a specific server, git1.kernel.org or git2.kernel.org.)
>
> How do I tell git to change the default repository to pull from?
Easy, depending on how old your repository either edit
.git/branches/origin or .git/remotes/origin. They contain the current
default, just change it to the new one. E.g. my .git/remotes/origin for
Linux-2.6 repository is:
URL: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
Pull: master:origin
Best regards,
Anton
--
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
^ permalink raw reply
* git-send-pack segfaulting on DebianPPC (was: Re: cg-clone, tag objects and cg-push/git-push don't play nice)
From: Martin Langhoff @ 2005-10-19 9:02 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
On 10/19/05, Junio C Hamano <junkio@cox.net> wrote:
> Although I do not follow Cogito development closely, I seem to
> recall that it fetched tags without making them complete at some
> point in the past; I hope it is now fixed but I am not sure.
Ok -- I can now shed some light on this. This was happening on a
DebianPPC machine, but I didn't have clear evidence of that being a
factor (specially when Linus is widely known to use a PPC, and I
hadn't seen problems in other ppc boxen around). Now I do.
For all my finger-pointing, cg-fetch was trying to fetch the refernces
properly. However, git-fetch-pack is segfaulting on this Debian PPC
(etch). This is true of the current git "master" and 0.99.8.c. I
don'thave strace on the box until tomorrow, so I can't tell you more
about it.
On Debian i386 and Ubuntu i386, the exact same versions work correctly.
I'll post more info as soon as I can get my hands on strace,
cheers,
martin
^ permalink raw reply
* Re: git-daemon enabled on kernel.org
From: Junio C Hamano @ 2005-10-19 9:02 UTC (permalink / raw)
To: Erik Mouw; +Cc: git
In-Reply-To: <20051019083542.GA31526@harddisk-recovery.com>
Erik Mouw <erik@harddisk-recovery.com> writes:
> How do I tell git to change the default repository to pull from?
Depends on how you are currently telling git what your "default
repository to pull from" is. If you are using git-core
(i.e. git-pull and git-fetch) for downloading, either
.git/remotes/origin or .git/branches/origin file are used in
this order; if you are using Cogito for downloading, it uses
.git/branches/origin IIRC.
^ permalink raw reply
* Re: cg-clone, tag objects and cg-push/git-push don't play nice
From: Martin Langhoff @ 2005-10-19 9:06 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vbr1ldh30.fsf@assigned-by-dhcp.cox.net>
On 10/19/05, Junio C Hamano <junkio@cox.net> wrote:
> Yeah. But the problem is that the repository on the other end
> claims that it has everything reachable from those incomplete
> tags.
It definitely does.
> Hearing that, git-push (the real name of it is send-pack)
> decides not to send things that are already reachable from the
> refs the other end claims to have. So if an incomplete tag
> refers to a commit that contains a blob that the remote actually
> does not have, and if that blob is part of the head you are
> pushing, send-pack would not (and should not) send that blob to
> the remote.
*shrug* it's a bit over my head, but I've figured out taht
git-fetch-pack is segfaulting. Perhaps that helps?
> Hmph. It worries me even more.
>
> This error message:
>
> error: unpack should have generated
> 482d4b88aa482dfea7f7549470902049a050020a, but I can't find it!
>
> comes from receive-pack that runs on the other repo (i.e. the
> one with incomplete tags you just removed), so it means
> send-pack decided it does not need to send that object --
> meaning the other end claimed it already has it. What to send
> and what need not to be sent is determined solely based on what
> send-pack hears from receive-pack in the initial handshake,
> which is in receive-pack.c::write_head_info(). It scans
> everything under ".git/refs" directory (not just .git/refs/heads
> or .git/refs/tags; if you had ".git/refs/FOOBAR", it will cause
> the remote end to claim it has everything reachable from it --
> the only exceptions are things that starts with a dot '.', which
> is probably why cg-fetch places a temporary heads in
> refs/*/.$name-fetching) and sends them -- it does not look at
> the object directory and magically claim it has something that
> is not recorded in its .git/refs/ directory.
*Very* strange. The remote repo is packed, but is working well for
everyone else. I think we should follow the segfault and see where it
leads...
cheers,
m
^ permalink raw reply
* Re: git-send-pack segfaulting on DebianPPC
From: Junio C Hamano @ 2005-10-19 9:21 UTC (permalink / raw)
To: Martin Langhoff; +Cc: git
In-Reply-To: <46a038f90510190202n60101c5cgf27bd714dce00513@mail.gmail.com>
Martin Langhoff <martin.langhoff@gmail.com> writes:
> ... However, git-fetch-pack is segfaulting on this Debian PPC
> (etch). This is true of the current git "master" and 0.99.8.c. I
> don'thave strace on the box until tomorrow, so I can't tell you more
> about it.
Thanks, that is good to know. As I said in another message, I
looked at cg-fetch's current tag completion code while working
on the ref^{tree} thing, and saw it used commit walker with '-a'
option (IOW, not producing an incomplete tag) to download
everything.
Even after fetch-pack segfaulted, since it does not update any
refs itself (it only writes them to stdout for the wrapper to
process), it should not have resulted in the repository that has
refs pointing at objects it does not have. In fact, it writes
its "SHA1 - name" list only after seeing unpack-objects exit
successfully, that should not have resulted in a corrupt
repository that records incomplete refs. I suspect there is
something else going on here.
In cg-fetch, fetch-pack is used only in one location, and it
does not have anything to do with the tag completion code.
Presumably that explains why removing tags did not have any
effect, although it does not explain why removing some objects
did -- maybe those object files you needed to remove were
corrupt?
^ permalink raw reply
* GIT 0.99.8f
From: Junio C Hamano @ 2005-10-19 10:04 UTC (permalink / raw)
To: git; +Cc: linux-kernel
GIT 0.99.8f is available as usual at:
RPMs and tarball: www.kernel.org:/pub/software/scm/git/
Debs and tarball: www.kernel.org:/pub/software/scm/git/debian/
Sorry, I said 0.99.8e was going to be the last 0.99.8
maintenance release, but it turns out that there was a flurry of
updates to git-daemon and rev-list (which matters to gitweb)
yesterday. So here it is.
Now, this _is_ going to be the last 0.99.8 maintenance release,
I promise ;-).
^ permalink raw reply
* Re: git-send-pack segfaulting on DebianPPC
From: Martin Langhoff @ 2005-10-19 10:07 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7v7jc9c15w.fsf@assigned-by-dhcp.cox.net>
On 10/19/05, Junio C Hamano <junkio@cox.net> wrote:
> Martin Langhoff <martin.langhoff@gmail.com> writes:
>
> > ... However, git-fetch-pack is segfaulting on this Debian PPC
> > (etch). This is true of the current git "master" and 0.99.8.c. I
> > don'thave strace on the box until tomorrow, so I can't tell you more
> > about it.
>
> Thanks, that is good to know. As I said in another message, I
> looked at cg-fetch's current tag completion code while working
> on the ref^{tree} thing, and saw it used commit walker with '-a'
> option (IOW, not producing an incomplete tag) to download
> everything.
Ok -- I'm at home now so I can't test it any further, but reading
cg-fetch, it would actually be running git-ssh-fetch which was failing
(but cogito hides STDERR). When I tried it by hand, I used
git-fetch-pack, which segfaulted on Debian PPC and worked on i386.
> Even after fetch-pack segfaulted, since it does not update any
> refs itself (it only writes them to stdout for the wrapper to
> process), it should not have resulted in the repository that has
> refs pointing at objects it does not have. In fact, it writes
> its "SHA1 - name" list only after seeing unpack-objects exit
> successfully, that should not have resulted in a corrupt
> repository that records incomplete refs. I suspect there is
> something else going on here.
Hmmm. You're right, it only spits out a list of things to retrieve...
> In cg-fetch, fetch-pack is used only in one location, and it
> does not have anything to do with the tag completion code.
> Presumably that explains why removing tags did not have any
> effect, although it does not explain why removing some objects
> did -- maybe those object files you needed to remove were
> corrupt?
Well, I'll strace the execution tomorrow and let you know. The tag
objects, however, are there. I can reliably git-cat-file tag <sha1>
them and they look pretty normal. The commit objects they refer to are
missing, though.
cheers,
martin
^ permalink raw reply
* Re: [RFC] Timeouts on HTTP requests
From: Petr Baudis @ 2005-10-19 11:09 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Nick Hengeveld, git
In-Reply-To: <7voe5mgi3d.fsf@assigned-by-dhcp.cox.net>
Dear diary, on Wed, Oct 19, 2005 at 08:02:14AM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> Nick Hengeveld <nickh@reactrix.com> writes:
>
> > Our QA department today checked what would happen if the network connection
> > went away completely in the middle of an HTTP transfer. It looks as though
> > the answer is that git-http-fetch sits there forever waiting for CURL to
> > return something.
>
> Ouch.
>
> > I'm thinking of taking advantage of CURL's capability of aborting a request
> > if the transfer rate drops below a threshold for a specified length of time
> > using a new pair of environment variables and/or config file settings:
> >
> > GIT_HTTP_LOW_SPEED_LIMIT/http.lowspeedlimit
> > GIT_HTTP_LOW_SPEED_TIME/http.lowspeedtime
> >
> > Does this make sense, and if so should there be defaults if nothing is
> > specified?
>
> I suspect these would be quite different between DSL and
> localnet, so I doubt if there is a reasonable default value to
> quick give-up.
>
> On the other hand, having _no_ activity for say 30 seconds would
> indicate a dead link on either modem or localnet.
I agree that we should definitely use timeout instead of some low speed
limit - can't curl do that?
But 30 seconds is too little (on bad links, I've seen TCP connections
stalled for much longer), I would use at least 120 seconds. Or perhaps
30 seconds, but retry three times or so.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply
* Fwd: cg-merge should use git-merge
From: Martin Langhoff @ 2005-10-19 11:21 UTC (permalink / raw)
To: Git Mailing List, Petr Baudis
In-Reply-To: <46a038f90510161644l35119401rdc05c081506ae715@mail.gmail.com>
Resending a post that never made it to the list due to a bug at my
end, and I think warrants some more discussion. Unfortunately, I'm
bug-fixing at the moment, so implementing this has taken a backseat.
---------- Forwarded message ----------
From: Martin Langhoff <martin.langhoff@gmail.com>
Date: Oct 17, 2005 12:44 PM
Subject: cg-merge should use git-merge
To: Petr Baudis <pasky@ucw.cz>, Git Mailing List <git@vger.kernel.org>
Petr,
After using git-merge a few times by hand to test-drive the new merge
drivers, I'm sold on the idea, and I 'd like to have cg-merge use
git-merge directly. I'll try and find the time one of these evenings
to outline something, but I guess it's pretty core to Cogito, so I'm
after some direction from you ;)
martin
^ permalink raw reply
* Re: Hard-linked trees with git?
From: Krzysztof Halasa @ 2005-10-19 11:23 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vpsq2pmyl.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano <junkio@cox.net> writes:
>> Or: is it possible to have some constant file timestamps, so that
>> changing the HEAD to something and returning to the old HEAD
>> (say, with hard resets) restores the old timestamps?
>
> That would screw up 'make'.
Not if the old timestamps matched the old files and if you make sure
you don't mix different branches with one object tree (i.e., you always
switch to correct HEAD before building).
--
Krzysztof Halasa
^ permalink raw reply
* Re: Pushing a single tag (ref + object)?
From: Horst von Brand @ 2005-10-19 12:39 UTC (permalink / raw)
To: Martin Langhoff; +Cc: Git Mailing List
In-Reply-To: <46a038f90510182305j1fa2c4bh6d2b36c2fdd058ce@mail.gmail.com>
Martin Langhoff <martin.langhoff@gmail.com> wrote:
> While we are using a repo which holds all our branches
> (dev/test/prod), locally we have a group of developers that checkout
> one repo-per-branch, working on it with a
> cg-clone/cg-update/cg-commit/cg-push workcycle. So far it's working
> great.
>
> Now, I am at a loss on how to push a _tag_ object+ref to the repo,
> without doing a git-push --all, which I naturally don't want to do. I
> managed to push the object itself, doing
>
> git-push repository tagrefname
>
> But that ddn't create the ref on the repo. So I had to do
>
> scp .git/refs/tags/refname repostory/refs/tags/
I've done:
git push repository refs/tags/refname
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
^ permalink raw reply
* [PATCH] git-clone: don't unpack objects
From: Timo Hirvonen @ 2005-10-19 12:43 UTC (permalink / raw)
To: Git Mailing List, Junio C Hamano
Pass --keep flag to git-clone-pack.
Signed-off-by: Timo Hirvonen <tihirvon@gmail.com>
---
git-clone.sh | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
applies-to: 984b42b74993da2be26cd2b50ac1d3e74b0c4cd5
63a266005116dd914c85b4c5f97d0e628f0b37ec
diff --git a/git-clone.sh b/git-clone.sh
index 18e692a..152ba15 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -184,8 +184,8 @@ yes,yes)
;;
*)
cd "$D" && case "$upload_pack" in
- '') git-clone-pack $quiet "$repo" ;;
- *) git-clone-pack $quiet "$upload_pack" "$repo" ;;
+ '') git-clone-pack $quiet --keep "$repo" ;;
+ *) git-clone-pack $quiet --keep "$upload_pack" "$repo" ;;
esac
;;
esac
---
0.99.8.GIT
^ permalink raw reply related
* Re: gitweb.cgi
From: Linus Torvalds @ 2005-10-19 13:59 UTC (permalink / raw)
To: H. Peter Anvin; +Cc: Kay Sievers, Git Mailing List
In-Reply-To: <43559DFE.7060503@zytor.com>
On Tue, 18 Oct 2005, H. Peter Anvin wrote:
>
> Yes, but I did that. It seems very strange when something hits the cache. A
> cgi script can apparently be run quite a few number of times before mod_cache
> sees it globally.
Well, I tried again this morning, and _boy_ is it better. The "projects"
thing came up immediately, no waiting.
Maybe it just took a while for the mod_cache thing to take effect as
Apache swarm members time out?
Or maybe it's just that the west coast is really quiet at 7AM.
Linus
^ permalink raw reply
* Re: Pushing a single tag (ref + object)?
From: Linus Torvalds @ 2005-10-19 14:13 UTC (permalink / raw)
To: Martin Langhoff, Junio C Hamano; +Cc: Git Mailing List
In-Reply-To: <46a038f90510182305j1fa2c4bh6d2b36c2fdd058ce@mail.gmail.com>
On Wed, 19 Oct 2005, Martin Langhoff wrote:
>
> Now, I am at a loss on how to push a _tag_ object+ref to the repo,
> without doing a git-push --all, which I naturally don't want to do. I
> managed to push the object itself, doing
>
> git-push repository tagrefname
This really should work.
Can you try with
git-send-pack repository tagname
first?
If that works for you (it really really should, since that's what I do all
the time), then the problem is in "get_remote_refs_for_push"..
Oh. Looking at "git-push.sh", I think it is just really really buggy.
It does all this magic to set "x", but then it never uses it, and uses
"$@" instead after all. Which it has shifted all away.
How does that thing work at all? Me, I've always used the raw
git-send-pack program, so I've never tested it, but obviously others are
using it.
Linus
^ permalink raw reply
* Re: git-send-pack segfaulting on DebianPPC (was: Re: cg-clone, tag objects and cg-push/git-push don't play nice)
From: Linus Torvalds @ 2005-10-19 14:31 UTC (permalink / raw)
To: Martin Langhoff; +Cc: Junio C Hamano, git
In-Reply-To: <46a038f90510190202n60101c5cgf27bd714dce00513@mail.gmail.com>
On Wed, 19 Oct 2005, Martin Langhoff wrote:
>
> On Debian i386 and Ubuntu i386, the exact same versions work correctly.
Interesting.
As you say, I'm obviously testing on ppc all the time, and if
git-fetch-pack has problems on ppc, I haven't seen them. And I fetch a
lot.
It might be just the repo you're using. I can't test, since I obviously
cannot ssh into the repo you pointed at (I tried to see if you had a
git-daemon running, but no such luck).
> I'll post more info as soon as I can get my hands on strace,
Even more interesting than strace might be for you to try to run
git-send-pack (even on an x86) under valgrind. valgrind really is a
wonderful tool.
One other potential issue: I'm not running debian. I've run YDL, but right
now the machine I use is FC4. There are likely _lots_ of library
differences etc there.
Linus
^ permalink raw reply
* Re: [RFC] Timeouts on HTTP requests
From: Nick Hengeveld @ 2005-10-19 15:34 UTC (permalink / raw)
To: Petr Baudis; +Cc: Junio C Hamano, git
In-Reply-To: <20051019110909.GO30889@pasky.or.cz>
On Wed, Oct 19, 2005 at 01:09:10PM +0200, Petr Baudis wrote:
> > On the other hand, having _no_ activity for say 30 seconds would
> > indicate a dead link on either modem or localnet.
>
> I agree that we should definitely use timeout instead of some low speed
> limit - can't curl do that?
Curl lets you set timeouts for the connect and the overall request, but
neither of these are related to activity. In our case, some of the files
we transfer can be quite large and need to transfer over a 56K modem line.
As long as data is moving, we're happy; but we need the transfer to abort
if the connection drops and data stops moving completely.
> But 30 seconds is too little (on bad links, I've seen TCP connections
> stalled for much longer), I would use at least 120 seconds. Or perhaps
> 30 seconds, but retry three times or so.
I've been testing with low speed limit of 1 and low speed time of 300 - if
data transfer stays below 1 byte/sec for 5 minutes, curl aborts the transfer.
I like Junio's suggestion regarding defaults though, don't compile them in
but let default config templates take care of it.
--
For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.
^ permalink raw reply
* Re: git-diff-tree rename detection for single file
From: Nicolas Pitre @ 2005-10-19 16:04 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Linus Torvalds, David Ho, git
In-Reply-To: <7virvuhylm.fsf@assigned-by-dhcp.cox.net>
On Tue, 18 Oct 2005, Junio C Hamano wrote:
> BTW, I really liked your example that piped multiple diff-trees
> together. That is a neat trick.
Yeah. Those should really be picked up to augment the documentation
section.
Nicolas
^ permalink raw reply
* Re: gitweb.cgi
From: H. Peter Anvin @ 2005-10-19 16:16 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Kay Sievers, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0510190655230.3369@g5.osdl.org>
Linus Torvalds wrote:
>
> Well, I tried again this morning, and _boy_ is it better. The "projects"
> thing came up immediately, no waiting.
>
> Maybe it just took a while for the mod_cache thing to take effect as
> Apache swarm members time out?
>
No, that's not it. It takes killing httpd and restarting it. Rather,
it looks like mod_cache in different swarm members doesn't always
communicate instantly, which is also underscored by the fact that cache
files don't appear into the filesystem until after a while.
-hpa
^ permalink raw reply
* Re: git-daemon enabled on kernel.org
From: H. Peter Anvin @ 2005-10-19 16:33 UTC (permalink / raw)
To: Nico -telmich- Schottelius; +Cc: Erik Mouw, Git Mailing List
In-Reply-To: <20051019084056.GL22986@schottelius.org>
Nico -telmich- Schottelius wrote:
> Erik Mouw [Wed, Oct 19, 2005 at 10:35:42AM +0200]:
>
>>On Tue, Oct 18, 2005 at 12:30:23PM -0700, H. Peter Anvin wrote:
>>
>>>After getting gitweb behind mod_cache, the load on kernel.org has gotten
>>>down into the tolerable range, so I have enabled git-daemon in an
>>>attempt to fix that :)
>>>
>>>The URL, obviously, is git://git.kernel.org/pub/scm/...
>>>
>>>(or, to specify a specific server, git1.kernel.org or git2.kernel.org.)
>>
>>How do I tell git to change the default repository to pull from?
>
>
> Do you mean cg-branch-add perhaps? Afaik there is no real 'default' repository, but
> the branches you specified. So adding a new branch will fix what you want.
>
With cogito, you want "cg-branch-chg origin".
-hpa
^ permalink raw reply
* Re: Pushing a single tag (ref + object)?
From: Junio C Hamano @ 2005-10-19 16:58 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0510190702360.3369@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> It does all this magic to set "x", but then it never uses it, and uses
> "$@" instead after all. Which it has shifted all away.
Your /bin/sh scripting is rotting ;-) That is not a variable.
If you saw the script actually used the value 'x' then you have
spotted a bug.
That 'x' is not a magic but an old idiom to protect 'set' from
getting confused by "$1" that happens to begin with a '-'.
Setting random things is done by "set x random things"
immediately followed by shifting that x away.
^ permalink raw reply
* [PATCH] git-daemon: timeout, eliminate double DWIM
From: H. Peter Anvin @ 2005-10-19 18:54 UTC (permalink / raw)
To: Git Mailing List
[-- Attachment #1: Type: text/plain, Size: 612 bytes --]
It turns out that not only did git-daemon do DWIM, but git-upload-pack
does as well. This is bad; security checks have to be performed *after*
canonicalization, not before.
Additionally, the current git-daemon can be trivially DoSed by spewing
SYNs at the target port.
This patch adds a --strict option to git-upload-pack to disable all
DWIM, a --timeout option to git-daemon and git-upload-pack, and an
--init-timeout option to git-daemon (which is typically set to a much
lower value, since the initial request should come immediately from the
client.)
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 4495 bytes --]
diff --git a/daemon.c b/daemon.c
--- a/daemon.c
+++ b/daemon.c
@@ -13,7 +13,9 @@
static int log_syslog;
static int verbose;
-static const char daemon_usage[] = "git-daemon [--verbose] [--syslog] [--inetd | --port=n] [--export-all] [directory...]";
+static const char daemon_usage[] =
+"git-daemon [--verbose] [--syslog] [--inetd | --port=n] [--export-all]\n"
+" [--timeout=n] [--init-timeout=n] [directory...]";
/* List of acceptable pathname prefixes */
static char **ok_paths = NULL;
@@ -21,6 +23,9 @@ static char **ok_paths = NULL;
/* If this is set, git-daemon-export-ok is not required */
static int export_all_trees = 0;
+/* Timeout, and initial timeout */
+static unsigned int timeout = 0;
+static unsigned int init_timeout = 0;
static void logreport(int priority, const char *err, va_list params)
{
@@ -170,6 +175,8 @@ static int upload(char *dir)
/* Enough for the longest path above including final null */
int buflen = strlen(dir)+10;
char *dirbuf = xmalloc(buflen);
+ /* Timeout as string */
+ char timeout_buf[64];
loginfo("Request for '%s'", dir);
@@ -190,8 +197,10 @@ static int upload(char *dir)
*/
signal(SIGTERM, SIG_IGN);
+ snprintf(timeout_buf, sizeof timeout_buf, "--timeout=%u", timeout);
+
/* git-upload-pack only ever reads stuff, so this is safe */
- execlp("git-upload-pack", "git-upload-pack", ".", NULL);
+ execlp("git-upload-pack", "git-upload-pack", "--strict", timeout_buf, ".", NULL);
return -1;
}
@@ -200,7 +209,9 @@ static int execute(void)
static char line[1000];
int len;
+ alarm(init_timeout ? init_timeout : timeout);
len = packet_read_line(0, line, sizeof(line));
+ alarm(0);
if (len && line[len-1] == '\n')
line[--len] = 0;
@@ -598,6 +609,12 @@ int main(int argc, char **argv)
export_all_trees = 1;
continue;
}
+ if (!strncmp(arg, "--timeout=")) {
+ timeout = atoi(arg+10);
+ }
+ if (!strncmp(arg, "--init-timeout=")) {
+ init_timeout = atoi(arg+15);
+ }
if (!strcmp(arg, "--")) {
ok_paths = &argv[i+1];
break;
diff --git a/upload-pack.c b/upload-pack.c
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -4,13 +4,19 @@
#include "tag.h"
#include "object.h"
-static const char upload_pack_usage[] = "git-upload-pack <dir>";
+static const char upload_pack_usage[] = "git-upload-pack [--strict] [--timeout=nn] <dir>";
-#define MAX_HAS (16)
-#define MAX_NEEDS (256)
+#define MAX_HAS 64
+#define MAX_NEEDS 4096
static int nr_has = 0, nr_needs = 0;
static unsigned char has_sha1[MAX_HAS][20];
static unsigned char needs_sha1[MAX_NEEDS][20];
+static unsigned int timeout = 0;
+
+static void reset_timeout(void)
+{
+ alarm(timeout);
+}
static int strip(char *line, int len)
{
@@ -100,6 +106,7 @@ static int get_common_commits(void)
for(;;) {
len = packet_read_line(0, line, sizeof(line));
+ reset_timeout();
if (!len) {
packet_write(1, "NAK\n");
@@ -122,6 +129,7 @@ static int get_common_commits(void)
for (;;) {
len = packet_read_line(0, line, sizeof(line));
+ reset_timeout();
if (!len)
continue;
len = strip(line, len);
@@ -145,6 +153,7 @@ static int receive_needs(void)
for (;;) {
unsigned char dummy[20], *sha1_buf;
len = packet_read_line(0, line, sizeof(line));
+ reset_timeout();
if (!len)
return needs;
@@ -179,6 +188,7 @@ static int send_ref(const char *refname,
static int upload_pack(void)
{
+ reset_timeout();
head_ref(send_ref);
for_each_ref(send_ref);
packet_flush(1);
@@ -193,18 +203,43 @@ static int upload_pack(void)
int main(int argc, char **argv)
{
const char *dir;
- if (argc != 2)
+ int i;
+ int strict = 0;
+
+ for (i = 1; i < argc; i++) {
+ char *arg = argv[i];
+
+ if (arg[0] != '-')
+ break;
+ if (!strcmp(arg, "--strict")) {
+ strict = 1;
+ continue;
+ }
+ if (!strncmp(arg, "--timeout=")) {
+ timeout = atoi(arg+10);
+ continue;
+ }
+ if (!strcmp(arg, "--")) {
+ i++;
+ break;
+ }
+ }
+
+ if (i != argc-1)
usage(upload_pack_usage);
- dir = argv[1];
+ dir = argv[i];
/* chdir to the directory. If that fails, try appending ".git" */
if (chdir(dir) < 0) {
- if (chdir(mkpath("%s.git", dir)) < 0)
+ if (strict || chdir(mkpath("%s.git", dir)) < 0)
die("git-upload-pack unable to chdir to %s", dir);
}
- chdir(".git");
+ if (!strict)
+ chdir(".git");
+
if (access("objects", X_OK) || access("refs", X_OK))
die("git-upload-pack: %s doesn't seem to be a git archive", dir);
+
putenv("GIT_DIR=.");
upload_pack();
return 0;
^ permalink raw reply
* [PATCH] Support for HTTP transfer timeouts based on transfer speed
From: Nick Hengeveld @ 2005-10-19 18:59 UTC (permalink / raw)
To: git
Add configuration settings to abort HTTP requests if the transfer rate
drops below a threshold for a specified length of time. Environment
variables override config file settings.
Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
---
It looks as though these curl options have been there since 1999 (before
version 6.5), so we shouldn't need to conditionally compile them
based on LIBCURL_VERSION_NUM.
http-fetch.c | 29 +++++++++++++++++++++++++++++
1 files changed, 29 insertions(+), 0 deletions(-)
e624aa205da9861a7d70bffbd22df471c92f1561
diff --git a/http-fetch.c b/http-fetch.c
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -100,6 +100,8 @@ static char *ssl_key = NULL;
static char *ssl_capath = NULL;
#endif
static char *ssl_cainfo = NULL;
+static long curl_low_speed_limit = -1;
+static long curl_low_speed_time = -1;
struct buffer
{
@@ -158,6 +160,17 @@ static int http_options(const char *var,
}
#endif
+ if (!strcmp("http.lowspeedlimit", var)) {
+ if (curl_low_speed_limit == -1)
+ curl_low_speed_limit = (long)git_config_int(var, value);
+ return 0;
+ }
+ if (!strcmp("http.lowspeedtime", var)) {
+ if (curl_low_speed_time == -1)
+ curl_low_speed_time = (long)git_config_int(var, value);
+ return 0;
+ }
+
/* Fall back on the default ones */
return git_default_config(var, value);
}
@@ -246,6 +259,13 @@ static CURL* get_curl_handle(void)
curl_easy_setopt(result, CURLOPT_CAINFO, ssl_cainfo);
curl_easy_setopt(result, CURLOPT_FAILONERROR, 1);
+ if (curl_low_speed_limit > 0 && curl_low_speed_time > 0) {
+ curl_easy_setopt(result, CURLOPT_LOW_SPEED_LIMIT,
+ curl_low_speed_limit);
+ curl_easy_setopt(result, CURLOPT_LOW_SPEED_TIME,
+ curl_low_speed_time);
+ }
+
return result;
}
@@ -1177,6 +1197,8 @@ int main(int argc, char **argv)
char *url;
int arg = 1;
struct active_request_slot *slot;
+ char *low_speed_limit;
+ char *low_speed_time;
while (arg < argc && argv[arg][0] == '-') {
if (argv[arg][1] == 't') {
@@ -1232,6 +1254,13 @@ int main(int argc, char **argv)
#endif
ssl_cainfo = getenv("GIT_SSL_CAINFO");
+ low_speed_limit = getenv("GIT_HTTP_LOW_SPEED_LIMIT");
+ if (low_speed_limit != NULL)
+ curl_low_speed_limit = strtol(low_speed_limit, NULL, 10);
+ low_speed_time = getenv("GIT_HTTP_LOW_SPEED_TIME");
+ if (low_speed_time != NULL)
+ curl_low_speed_time = strtol(low_speed_time, NULL, 10);
+
git_config(http_options);
if (curl_ssl_verify == -1)
^ permalink raw reply
* Errors from http-fetch
From: Daniel Barkalow @ 2005-10-19 19:46 UTC (permalink / raw)
To: git
I've now started getting errors from http-fetch; my guess for the output
is that it has problems if an object is not available loose, and is only
available in a pack file which is already being downloaded for a different
object, possibly only if the first request to be made for the loose
object is not the first one to fail.
On the other hand, it looks like just trying again works fine (or, at
least, makes progress), so there's no need to fall back to rsync or such.
But that also means that, in order to try to make a repeatable test, you
need to copy your state before you run it and get the error; I'll probably
try this evening to work it out.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* The git protocol and DoS
From: H. Peter Anvin @ 2005-10-19 20:00 UTC (permalink / raw)
To: Git Mailing List
I've been concerned for a while that the git protocol may be inherently
vulnerable to a "SYNful DoS" attack (spraying raw TCP SYN packets with
enough data to start substantial server activity.) Although SYN cookies
protect against this to some degree, it makes me wonder if something
should be added to the protocol itself.
One way to do this would be to start the transaction by having the
server transmit a cookie to the client, and to require the client to
send a SHA1 of the (cookie + request) together with the request. This
would be done with a fairly short timeout.
It would, however, require a protocol change; I would like to hear what
people think about this at this stac=ge.
-hpa
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox