* git repo corruption
From: Levend Sayar @ 2011-01-04 9:10 UTC (permalink / raw)
To: git
Hi, all.
We have a repo on a corporate server. The sysadmin changed access
rights of files on our repo by accedant.
Some directories have 2750 acces rights before, but he changed as
chmod -R 2770 *
Now when you make git status, every file that is tracked by git is said as
changed but not updated
So is there a way to get this back to normal ?
TIA
_lvnd_
(^_^)
^ permalink raw reply
* Re: clone --bare vs push
From: Neal Kreitzinger @ 2011-01-04 5:29 UTC (permalink / raw)
To: git
Cc: Levend Sayar, git, Neal Kreitzinger,
Ævar Arnfjörð Bjarmason
In-Reply-To: <20110104045839.GA8225@burratino>
On 1/3/2011 10:58 PM, Jonathan Nieder wrote:
> Neal Kreitzinger wrote:
>
>> you can also use the file:// url format to create a complete copy and not
>> use hardlinks. see git-clone manpage.
>>
>> e.g. git clone --bare file:///fullpathto/X y.git
>
> Thanks, that's useful; cc-ing Levend.
>
> [Side note: I wrote in the past:
>
> | In traditional newsgroups it seems to be most common to just reply to
> | the author by mail or follow-up to the group, so I wouldn't feel too
> | bad.
>
> but probably that was unclear of me. The convention on _this_ list[1] is
> to reply to all participants in a thread, so new participants do not
> need to subscribe if they don't want to. Thunderbird (e.g.) seems to
> be good at doing that for what it's worth.]
>
> [1] at least as I perceive it; please feel free to correct me if I
> have misunderstood
This is a "reply all" from thunderbird. This looks like it is doing the
cc correctly...
v/r,
Neal
^ permalink raw reply
* google code git support
From: Miles Bader @ 2011-01-04 5:08 UTC (permalink / raw)
To: git
New year's reminder: register your vote for native git support in google
code -- go to:
http://code.google.com/p/support/issues/detail?id=2454
and click the star in front of "Issue 2454: native git support"
-miles
--
Do not taunt Happy Fun Ball.
^ permalink raw reply
* [PATCH 2/2] gitweb: make logo optional
From: Jonathan Nieder @ 2011-01-04 5:05 UTC (permalink / raw)
To: git; +Cc: John 'Warthog9' Hawley, Jakub Narebski, Eric Wong
In-Reply-To: <20110104050206.GA8280@burratino>
Date: Fri, 3 Sep 2010 19:45:09 -0500
Some sites may not want to have a logo at all. In particular, git
instaweb can benefit from this.
While at it, use $cgi->img to simplify this code. (CGI.pm learned
most HTML4 tags by version 2.79, so this should be portable to perl
5.8, though I haven't tested.)
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Eric liked an earlier version of this patch that did not use
$cgi->img.
gitweb/gitweb.perl | 13 +++++++++----
1 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index c65af1a..064697c 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -3601,10 +3601,15 @@ EOF
insert_file($site_header);
}
- print "<div class=\"page_header\">\n" .
- $cgi->a({-href => esc_url($logo_url),
- -title => $logo_label},
- qq(<img src=").esc_url($logo).qq(" width="72" height="27" alt="git" class="logo"/>));
+ print "<div class=\"page_header\">\n";
+ if (defined $logo) {
+ print $cgi->a({-href => esc_url($logo_url),
+ -title => $logo_label},
+ $cgi->img({-src => esc_url($logo),
+ -width => 72, -height => 27,
+ -alt => "git",
+ -class => "logo"}));
+ }
print $cgi->a({-href => esc_url($home_link)}, $home_link_str) . " / ";
if (defined $project) {
print $cgi->a({-href => href(action=>"summary")}, esc_html($project));
--
1.7.4.rc0
^ permalink raw reply related
* [PATCH 1/2] gitweb: skip logo in atom feed when there is none
From: Jonathan Nieder @ 2011-01-04 5:02 UTC (permalink / raw)
To: git; +Cc: John 'Warthog9' Hawley, Jakub Narebski, Eric Wong
In-Reply-To: <20110104050206.GA8280@burratino>
Date: Fri, 3 Sep 2010 19:44:39 -0500
With v1.5.0-rc0~169 (gitweb: Fix Atom feed <logo>: it is $logo,
not $logo_url, 2006-12-04), the logo URI to be written to Atom
feeds was corrected but the case of no logo forgotten.
Acked-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
gitweb/gitweb.perl | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 4779618..c65af1a 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -7169,7 +7169,7 @@ XML
if (defined $favicon) {
print "<icon>" . esc_url($favicon) . "</icon>\n";
}
- if (defined $logo_url) {
+ if (defined $logo) {
# not twice as wide as tall: 72 x 27 pixels
print "<logo>" . esc_url($logo) . "</logo>\n";
}
--
1.7.4.rc0
^ permalink raw reply related
* [PATCH 0/2] gitweb: make logo optional
From: Jonathan Nieder @ 2011-01-04 5:02 UTC (permalink / raw)
To: git; +Cc: John 'Warthog9' Hawley, Jakub Narebski, Eric Wong
These patches were last seen in the instaweb 1.7.1 maintenance
thread[1] but I believe they are independently useful. The
idea is to allow disabling the logo in gitweb.
Jonathan Nieder (2):
gitweb: skip logo in atom feed when there is none
gitweb: make logo optional
gitweb/gitweb.perl | 15 ++++++++++-----
1 files changed, 10 insertions(+), 5 deletions(-)
[1] http://thread.gmane.org/gmane.comp.version-control.git/155224
^ permalink raw reply
* Re: clone --bare vs push
From: Jonathan Nieder @ 2011-01-04 4:58 UTC (permalink / raw)
To: Levend Sayar
Cc: git, Neal Kreitzinger, Ævar Arnfjörð Bjarmason
In-Reply-To: <ifu8d2$t61$1@dough.gmane.org>
Neal Kreitzinger wrote:
> you can also use the file:// url format to create a complete copy and not
> use hardlinks. see git-clone manpage.
>
> e.g. git clone --bare file:///fullpathto/X y.git
Thanks, that's useful; cc-ing Levend.
[Side note: I wrote in the past:
| In traditional newsgroups it seems to be most common to just reply to
| the author by mail or follow-up to the group, so I wouldn't feel too
| bad.
but probably that was unclear of me. The convention on _this_ list[1] is
to reply to all participants in a thread, so new participants do not
need to subscribe if they don't want to. Thunderbird (e.g.) seems to
be good at doing that for what it's worth.]
[1] at least as I perceive it; please feel free to correct me if I
have misunderstood
^ permalink raw reply
* Re: clone --bare vs push
From: Neal Kreitzinger @ 2011-01-04 4:40 UTC (permalink / raw)
To: git
In-Reply-To: <AANLkTi=RNDYrRbyEJXA_c30JEVr=SYUQ01cfA3FyWpLT@mail.gmail.com>
"Levend Sayar" <levendsayar@gmail.com> wrote in message
news:AANLkTi=RNDYrRbyEJXA_c30JEVr=SYUQ01cfA3FyWpLT@mail.gmail.com...
> Hi, all.
>
> We cloned a repo from github on our local server. Say X for this. At
> that repo, we did
>
> git clone --bare X y.git
>
> Now y.git is ready for other machines to clone.
>
> To update our upstream repo X, we do
>
> git pull
>
> and then
>
> git push --all
>
> to update y.git.
>
> Now questions:
>
> 1) When I compare X/.git directory and y.git directory there are many
> objects missing in y.git. What is the reason ?
>
> 2) git clone --bare is too fast. My .git directory is nearly 1GB. How
> can it be copied that much fast ?
>
> 3) Is this more safe then git pull, git push
>
> rm -rf y.git
> git pull
> git clone --bare X y.git
>
> Namely bare cloning each time when we update our main repo ?
>
> TIA
>
> _lvnd_
> (^_^)
you can also use the file:// url format to create a complete copy and not
use hardlinks. see git-clone manpage.
e.g. git clone --bare file:///fullpathto/X y.git
v/r,
Neal
^ permalink raw reply
* [PATCH] daemon: support <directory> arguments again
From: Jonathan Nieder @ 2011-01-04 4:04 UTC (permalink / raw)
To: Erik Faye-Lund; +Cc: git, gitster
In-Reply-To: <1288834524-2400-9-git-send-email-kusmabite@gmail.com>
Ever since v1.7.4-rc0~125^2~8 (daemon: use run-command api for async
serving, 2010-11-04), git daemon spawns child processes instead of
forking to serve requests. The child processes learn that they are
being run for this purpose from the presence of the --serve command
line flag.
When running with <ok_path> arguments, the --serve flag is treated
as one of the path arguments and the special child behavior does
not kick in. So the child becomes an ordinary git daemon process,
notices that all the addresses it needs are in use, and exits with
the message "fatal: unable to allocate any listen sockets on port
9418".
Fix it by putting --serve at the beginning of the command line,
where the flag cannot be mistaken for a path argument.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
On the client side:
$ git clone git://localhost/git/git.git
Cloning into git...
fatal: read error: Connection reset by peer
On the server side:
# $git_src/bin-wrappers/git daemon --verbose --base-path=/var/cache /var/cache/git
fatal: unable to allocate any listen sockets on port 9418
[3602] [3604] Disconnected (with error)
Bisects to v1.7.4-rc0~125^2~8. This patch seems to fix it. Thoughts?
daemon.c | 7 ++++---
1 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/daemon.c b/daemon.c
index 4059593..132b7a8 100644
--- a/daemon.c
+++ b/daemon.c
@@ -1181,9 +1181,10 @@ int main(int argc, char **argv)
/* prepare argv for serving-processes */
cld_argv = xmalloc(sizeof (char *) * (argc + 2));
- for (i = 0; i < argc; ++i)
- cld_argv[i] = argv[i];
- cld_argv[argc] = "--serve";
+ cld_argv[0] = argv[0]; /* git-daemon */
+ cld_argv[1] = "--serve";
+ for (i = 1; i < argc; ++i)
+ cld_argv[i+1] = argv[i];
cld_argv[argc+1] = NULL;
return serve(&listen_addr, listen_port, pass, gid);
--
1.7.4.rc0
^ permalink raw reply related
* Re: git-daemon serving repos with repo.git/git-daemon-export-ok
From: Jonathan Nieder @ 2011-01-04 3:09 UTC (permalink / raw)
To: John Griessen; +Cc: git
In-Reply-To: <4D227ED7.9010407@ecosensory.com>
John Griessen wrote:
> On 01/03/2011 05:55 PM, Jonathan Nieder wrote:
>>> drwxr-xr-x 7 gitosis gitosis 4096 Dec 30 12:32 extrudator.data.git
>>
>> So the gitdaemon user (used by git-daemon-run) wouldn't be able to
>> access them.
>
> It would be able, since I used
> sudo strace -o strace.out /usr/lib/git-core/git-daemon --verbose
> --strict-paths --user=git --group=gitosis /srv/gitosis/repositories
Also because the repositories are world readable. I don't know what
I was thinking; I should get some sleep.
Thanks again for reporting and good luck.
Jonathan
^ permalink raw reply
* Re: gitattributes don't work
From: Marcin Wiśnicki @ 2011-01-04 2:16 UTC (permalink / raw)
To: git
In-Reply-To: <AANLkTikbGQkREQAx0p6Q+DrXPP2zWDU7doOO6yg+NGeC@mail.gmail.com>
On Mon, 03 Jan 2011 11:34:00 +0700, Nguyen Thai Ngoc Duy wrote:
> 2011/1/3 Marcin Wiśnicki <mwisnicki@gmail.com>:
>>
>> According to gitattributes(5) it supports patterns from gitignore(5).
>
> While gitattributes(5) says that, actually gitattributes and gitignore
> [snip]
> Yeah, it looks like a bug to me. A better way to solve this once and for
> [snip]
> For the time being, anyone who changes gitignore should be reminded to
> consider whether it's applicable to gitattributes and vice versa.
I think that for the time being at least the manual page must change to
reflect reality.
^ permalink raw reply
* Re: git-daemon serving repos with repo.git/git-daemon-export-ok
From: Jonathan Nieder @ 2011-01-04 2:11 UTC (permalink / raw)
To: John Griessen; +Cc: git
In-Reply-To: <4D227ED7.9010407@ecosensory.com>
John Griessen wrote:
> sudo /usr/lib/git-core/git-daemon --verbose --strict-paths --user=git
> --group=gitosis /srv/gitosis/repositories &>git-daemon-err.log
>
> vking@mail:~$ cat git-daemon-err.log
>
> and git-daemon-err.log was empty.
Thanks for clarifying.
> Should I reinstall debian's git-daemon-run to do some tests?
Seems redundant next to the tests you've already done.
> On 01/03/2011 05:55 PM, Jonathan Nieder wrote:
>> an issue here. It looks like another git-daemon process is running at
>> the same time? (One can check with "netstat -t -a".)
>
> Nope, not as far as I can see.
I left out the "-p", sorry. I'll be more careful in the future.
> What suggests that?
bind(3, {sa_family=AF_INET, sin_port=htons(9418), sin_addr=inet_addr("0.0.0.0")}, 16) = -1 EADDRINUSE (Address already in use)
^ permalink raw reply
* Re: git-daemon serving repos with repo.git/git-daemon-export-ok
From: John Griessen @ 2011-01-04 1:58 UTC (permalink / raw)
To: git
In-Reply-To: <20110103235501.GA32262@burratino>
On 01/03/2011 05:55 PM, Jonathan Nieder wrote:
>> I can push to these repos with gitosis, and the permissions are:
>>
>> vking@mail:/srv/gitosis/repositories$ ll
>> total 16
>> drwxr-xr-x 7 gitosis gitosis 4096 Dec 30 12:32 extrudator.data.git
>
> So the gitdaemon user (used by git-daemon-run) wouldn't be able to
> access them.
It would be able, since I used
sudo strace -o strace.out /usr/lib/git-core/git-daemon --verbose --strict-paths --user=git --group=gitosis /srv/gitosis/repositories
git-daemon was asked to run as --user=git --group=gitosis, so it had read permissions and
there was no user git-daemon anymore -- removed it from /etc/passwd.
>
>> vking@mail:/etc/sv$ cat strace.out
>> execve("/usr/lib/git-core/git-daemon",
>> ["/usr/lib/git-core/git-daemon", "--verbose", "--strict-paths",
>> "--user=git", "--group=gitosis", "/srv/gitosis/repositories"], [/*
>> 10 vars */]) = 0
>
> In this case simple --verbose output would probably be more useful.
> (That's what cat-ing /var/log/git-daemon/current would have given
> if git-daemon-run were still installed.)
I tried this
sudo /usr/lib/git-core/git-daemon --verbose --strict-paths --user=git --group=gitosis /srv/gitosis/repositories &>git-daemon-err.log
vking@mail:~$ cat git-daemon-err.log
and git-daemon-err.log was empty. Not the same as the /etc/sv/run script. Should I reinstall
debian's git-daemon-run to do some tests?
>
> Thanks for the strace. I should have mentioned that strace -f can
> help by following child processes, though that doesn't seem to be
> an issue here. It looks like another git-daemon process is running at
> the same time? (One can check with "netstat -t -a".)
Nope, not as far as I can see. What suggests that?
vking@mail:/etc/sv$ netstat -t -a
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 *:discard *:* LISTEN
tcp 0 0 *:git *:* LISTEN
tcp 0 0 localhost:mysql *:* LISTEN
tcp 0 0 *:daytime *:* LISTEN
tcp 0 0 localhost:sunrpc *:* LISTEN
tcp 0 0 *:www *:* LISTEN
tcp 0 0 localhost:690 *:* LISTEN
tcp 0 0 cottagematic.com:domain *:* LISTEN
tcp 0 0 mail.cibolo.us:domain *:* LISTEN
tcp 0 0 localhost:domain *:* LISTEN
tcp 0 0 *:ssh *:* LISTEN
tcp 0 0 *:telnet *:* LISTEN
tcp 0 0 localhost:postgresql *:* LISTEN
tcp 0 0 *:smtp *:* LISTEN
tcp 0 0 *:time *:* LISTEN
tcp 0 0 *:ldap *:* LISTEN
tcp 0 0 mail.cibolo.us:ssh cpe-66-68-104-159:47248 ESTABLISHED
tcp 0 224 mail.cibolo.us:ssh cpe-66-68-104-159:47247 ESTABLISHED
tcp6 0 0 *:pop3 *:* LISTEN
tcp6 0 0 *:imap2 *:* LISTEN
tcp6 0 0 *:ssh *:* LISTEN
tcp6 0 0 *:imaps *:* LISTEN
tcp6 0 0 *:pop3s *:* LISTEN
tcp6 0 0 *:ldap *:* LISTEN
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:58203 ESTABLISHED
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:58226 ESTABLISHED
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:48747 ESTABLISHED
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:58175 ESTABLISHED
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:58174 ESTABLISHED
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:48951 ESTABLISHED
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:52013 ESTABLISHED
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:52012 ESTABLISHED
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:52015 ESTABLISHED
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:52014 ESTABLISHED
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:47076 ESTABLISHED
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:47079 ESTABLISHED
tcp6 0 0 mail.cibolo.us:imaps cpe-66-68-104-159:47078 ESTABLISHED
John Griessen
^ permalink raw reply
* [RFC PATCH v7 2.5/9] gitweb: Make die_error just die, and use send_error to create error pages
From: Jakub Narebski @ 2011-01-04 0:35 UTC (permalink / raw)
To: git; +Cc: J.H., John 'Warthog9' Hawley, Jonathan Nieder
In-Reply-To: <20101222235525.7998.99816.stgit@localhost.localdomain>
Unify error handling by treating errors from Perl (and thrown early in
process using 'die STRING'), and errors from gitweb in the same way.
This means that in both cases error page is generated after an error
is caught in run() subroutine.
die_error() subroutine is now split into three: gen_error() which
massages parameters (escaping HTML, turning HTTP status number into
full HTTP status code), die_error() which uses gen_error() and just
throws an error (and does not generate an error page), and
send_error() which catually generate error page based on provided
error / exception.
Sidenote: probably in the future instead of using simple hash for
throwing gitweb exception, gitweb would use some custom error class,
e.g. derivative of Exception::Class (like SVN::Web does it).
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
This is sent early to facilitate early comments. It passes test suite,
but it was not extensively tested.
Now die_error() functions mode like 'die'...
gitweb/gitweb.perl | 47 ++++++++++++++++++++++++++++++++++++-----------
1 files changed, 36 insertions(+), 11 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index c7a1892..5854f73 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -1153,9 +1154,13 @@ sub run {
if $pre_dispatch_hook;
eval { run_request() };
- if (defined $@ && !ref($@)) {
+ my $error = $@;
+ if ($error) {
# some Perl error, but not one thrown by die_error
- die_error(undef, undef, $@, -error_handler => 1);
+ $error = gen_error(undef, undef, $error)
+ unless ref($error);
+
+ send_error($error);
}
DONE_REQUEST:
@@ -3730,11 +3735,14 @@ sub git_footer_html {
# an unknown error occurred (e.g. the git binary died unexpectedly).
# 503: The server is currently unavailable (because it is overloaded,
# or down for maintenance). Generally, this is a temporary state.
-sub die_error {
+
+# gen_error() generates error object from parameters
+# die_error() uses gen_error() to generate error object and dies
+# send_error() generates an error page from provided error object
+sub gen_error {
my $status = shift || 500;
my $error = esc_html(shift) || "Internal Server Error";
my $extra = shift;
- my %opts = @_;
my %http_responses = (
400 => '400 Bad Request',
@@ -3743,23 +3751,40 @@ sub die_error {
500 => '500 Internal Server Error',
503 => '503 Service Unavailable',
);
- git_header_html($http_responses{$status}, undef, %opts);
+
+ my $err = {
+ 'status' => $status,
+ 'http_status' => $http_responses{$status},
+ 'error' => $error,
+ 'extra' => $extra,
+ };
+ return $err;
+}
+
+sub die_error {
+ my $error = gen_error(@_);
+ print STDERR Dumper($error);
+ die $error;
+}
+
+sub send_error {
+ my $error = shift;
+
+ git_header_html($error->{'http_status'}, undef);
+
print <<EOF;
<div class="page_body">
<br /><br />
-$status - $error
+$error->{'status'} - $error->{'error'}
<br />
EOF
- if (defined $extra) {
+ if (defined $error->{'extra'}) {
print "<hr />\n" .
- "$extra\n";
+ "$error->{'extra'}\n";
}
print "</div>\n";
git_footer_html();
-
- die {'status' => $status, 'error' => $error}
- unless ($opts{'-error_handler'});
}
## ----------------------------------------------------------------------
--
1.7.3
^ permalink raw reply related
* Re: [RFC PATCH v7 11/9] [PoC] gitweb/lib - tee, i.e. print and capture during cache entry generation
From: Jakub Narebski @ 2011-01-04 0:28 UTC (permalink / raw)
To: J.H.; +Cc: git, John 'Warthog9' Hawley
In-Reply-To: <4D225C6E.9000108@eaglescrag.net>
On Tue, 4 Jan 2011, J.H. wrote:
> On 01/03/2011 01:33 PM, Jakub Narebski wrote:
> > Instead of having gitweb use progress info indicator / throbber to
> > notify user that data is being generated by current process, gitweb
> > can now (provided that PerlIO::tee from PerlIO::Util is available)
> > send page to web browser while simultaneously saving it to cache
> > (print and capture, i.e. tee), thus having incremental generating of
> > page serve as a progress indicator.
>
> In general, and particularly for the large sites that caching is
> targeted at, teeing is a really bad idea. I've mentioned this several
> times before, and the progress indicator is a *MUCH* better idea. I'm
> not sure how many times I can say that, even if this was added it would
> have the potential to exacerbate disk thrashing and overall make things
> a lot more complex.
It might be true that tee-ing is bad for very large sites, as it
increases load a bit in those (I think) extremly rare cases where
clients concurrently access the very same error page. But it might
be a solution for those in between cases. I think that incrementally
generated page is better progress indicator than just "Generating..."
page.
Anyway this proof of concept patch is to show how such thing should
be implemented. I don't think that it makes things a lot more complex;
in this rewrite everything is quite well modularized, encapsulated, and
isolated.
But the main intent behind this patch was to avoid bad interaction between
'progress info' indicator (in the process that is generating page, see
below), and non-cached error pages.
>
> 1) Errors may still be generated in flight as the cache is being
> generated. It would be better to let the cache run with a progress
> indicator and should an error occur, display the error instead of giving
> any output that may have been generated (and thus likely a broken page).
On the contrary, with tee-ing (and zero size sanity check) you would be
able to see pages even if there are errors saving cache entry. Though
this wouldn't help very large sites which cannot function without caching,
it could be useful for smaller sites.
But see below.
> 2) Having multiple clients all waiting on the same page (in particular
> the index page) can lead to invalid output. In particular if you are
> teeing the output a reading client now must come in, read the current
> contents of the file (as written), then pick up on the the tee after
> that. It's actually possible for the reading client to miss data as it
> may be in flight to be written and the client is switching from reading
> the file to reading the tee. I don't see anything in your code to
> handle that kind of switch over.
Err... could you explain what do you mean by "client is switching from
reading the file to reading the tee"?
Hmmm... I thought that the code is clear. Generating data, whether it
is captured to be displayed later, or tee-ed i.e. printed and captured
to cache, is inside critical section, protected by exclusive lock. Only
after cache entry is written (in full), the lock is released, and clients
waiting for data can access it; they use shared (readers) lock for sync.
Note that in my rewrite (and I think also in _some_ cases in your version)
files are written atomically, by writing to temporary file then renaming
it to final destination.
> 3) This makes no allowance for the file to be generated completely in
> the background while serving stale data in the interim. Keep in mind
> that it can (as Fedora has experienced) take *HOURS* to generate the
> index page, teeing that output just means brokenness and isn't useful.
It does make allowance. cache_output from GitwebCache::CacheOutput uses
capturing and not tee-ing if we are in background process. When there
is stale data to serve, cache entry is (re)generated in background in
detached process.
Moreover by default cache_output has safety in that error pages generated
by such detached process are cached.
Note also that in my rewrite you can simply (by changing one single
configuration knob) configure gitweb to also cache error pages. This
might be best and safest solution for very large sites with very large
disk space, but not so good for smaller sites.
>
> It's much better to have a simple, lightweight waiting message get
> displayed while things happen. When they are done, output the completed
> page to all waiting clients.
The problem with 'lightweight waiting message', as it is implemented in
your code, and as I stole it ;-), is that it doesn't provide any indicator
how much work is already done, and how much work might there be left.
Well, at least for now.
With tee-ing client (well, at least the one that is generating data; other
would get "Generating...", or rather "Waiting..." page) can estimate how
long would he/she had to wait, and literally see progress, not just some
progress indicator.
P.S. In my rewrite clients would retry generating page if it was not
generated when they were waiting for it, till they try their own hand
at generating. This protects against process generating data being
killed; see also test suite for caching interface.
> - John 'Warthog9' Hawley
>
> P.S. I'm back to work full-time on Wednesday, which I'll be catching up
> on gitweb and trying to make forward progress on my gitweb code again.
I'll try to send much simplified (and easier to use in caching) error
handling using exceptions (die / eval used as throw / catch) today.
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: git-daemon serving repos with repo.git/git-daemon-export-ok
From: Jonathan Nieder @ 2011-01-03 23:55 UTC (permalink / raw)
To: John Griessen; +Cc: git
In-Reply-To: <4D225DFA.50805@ecosensory.com>
John Griessen wrote:
> git version 1.7.2.3
Good.
> vking@mail:/etc/sv$ sv stat git-daemon
> fail: git-daemon: unable to change to service directory: file does not exist
Ah, you switched to running git daemon directly.
> I can push to these repos with gitosis, and the permissions are:
>
> vking@mail:/srv/gitosis/repositories$ ll
> total 16
> drwxr-xr-x 7 gitosis gitosis 4096 Dec 30 12:32 extrudator.data.git
So the gitdaemon user (used by git-daemon-run) wouldn't be able to
access them.
> vking@mail:/etc/sv$ cat strace.out
> execve("/usr/lib/git-core/git-daemon",
> ["/usr/lib/git-core/git-daemon", "--verbose", "--strict-paths",
> "--user=git", "--group=gitosis", "/srv/gitosis/repositories"], [/*
> 10 vars */]) = 0
In this case simple --verbose output would probably be more useful.
(That's what cat-ing /var/log/git-daemon/current would have given
if git-daemon-run were still installed.)
Thanks for the strace. I should have mentioned that strace -f can
help by following child processes, though that doesn't seem to be
an issue here. It looks like another git-daemon process is running at
the same time? (One can check with "netstat -t -a".)
^ permalink raw reply
* Re: [RFC PATCH 3/3] filter-branch: support --submodule-filter
From: Jeffrey Phillips Freeman @ 2011-01-03 23:44 UTC (permalink / raw)
To: Thomas Rast; +Cc: git
In-Reply-To: <44e6104ba28c80a6befe0f39fa4e2d6eeec56aa9.1293809100.git.trast@student.ethz.ch>
So im kinda new with all this so bare with me guys. I wanted to figure
out how to apply these patches, now i know i can use git to do this with
its patch command and such. However i was curious does this exist as a
branch somewhere official or semi-official? I currently seem to be using
--split-submodule which is itself in a git repo i have (which i want to
also find its source repo so i can keep up to date with it). So applying
the patch itself might be somewhat troubling due to conflicts, therefore
id like to actually merge in a remote branch to keep things easy. So can
you guys point me to which repo would be best for me to keep up to date
with this would be?
On 12/31/2010 10:29 AM, Thomas Rast wrote:
> This new filter gets each submodule's current sha1 and path on stdin,
> separated by a tab. It can then emit a new submodule sha1 and/or
> path, and filter-branch will:
>
> * if the path differs, remove the submodule at the old path;
>
> * add/replace the new sha1 at the new path.
>
> Additionally, returning an empty new sha1 deletes the submodule.
>
> You can tie this together with the last two commits to filter the
> super-project after a subproject filtering as follows:
>
> ( cd sub1&& git filter-branch --dump-map map<filters|args> )
> ( cd sub2&& git filter-branch --dump-map map<filters|args> )
> cat sub1/map sub2/map> map
> git filter-branch --load-map map \
> --submodule-filter "map $(cut -f1)" \
> <args>
>
> Other use-cases should also be covered since we also pass through the
> path.
>
> Signed-off-by: Thomas Rast<trast@student.ethz.ch>
> ---
> git-filter-branch.sh | 24 ++++++++++++++++++++++++
> 1 files changed, 24 insertions(+), 0 deletions(-)
>
> diff --git a/git-filter-branch.sh b/git-filter-branch.sh
> index 9feeb26..4a321c4 100755
> --- a/git-filter-branch.sh
> +++ b/git-filter-branch.sh
> @@ -122,6 +122,7 @@ filter_msg=cat
> filter_commit=
> filter_tag_name=
> filter_subdir=
> +filter_submodule=
> orig_namespace=refs/original/
> force=
> prune_empty=
> @@ -193,6 +194,9 @@ do
> filter_subdir="$OPTARG"
> remap_to_ancestor=t
> ;;
> + --submodule-filter)
> + filter_submodule="$OPTARG"
> + ;;
> --original)
> orig_namespace=$(expr "$OPTARG/" : '\(.*[^/]\)/*$')/
> ;;
> @@ -379,6 +383,26 @@ while read commit parents; do
> eval "$filter_index"< /dev/null ||
> die "index filter failed: $filter_index"
>
> + if [ "$filter_submodule" ]; then
> + git ls-files -s |
> + grep '^160000' |
> + while read mode sha1 stage path; do
> + printf "$sha1\t$path\n" |
> + { eval "$filter_submodule" ||
> + die "submodule filter failed: $filter_submodule"; } |
> + read newsha1 newpath
> + if [ -z "$newsha1" ] || [ "$path" != "$newpath" ]; then
> + git update-index --remove "$path" ||
> + die "failed to remove submodule $path"
> + fi
> + if [ -n "$newsha1" ]&& [ "$sha1" != "$newsha1" ]; then
> + git update-index --add --replace --cacheinfo \
> + 160000 "$newsha1" "$newpath" ||
> + die "failed to add submodule $newpath"
> + fi
> + done
> + fi
> +
> parentstr=
> for parent in $parents; do
> for reparent in $(map "$parent"); do
^ permalink raw reply
* Re: git-daemon serving repos with repo.git/git-daemon-export-ok
From: John Griessen @ 2011-01-03 23:38 UTC (permalink / raw)
To: git
In-Reply-To: <20110103231153.GA10733@burratino>
On 01/03/2011 05:11 PM, Jonathan Nieder wrote:
> Hi John,
>
> Worrisome. Based on /usr/share/doc/git-daemon-run/README.Debian,
> I would be interested in:
>
> git --version
server:
vking@mail:/etc/sv$ git --version
git version 1.7.2.3
client:
john@toolbench:~/EEProjects/tek_7k_ext$ git version 1.7.2.3
git version 1.7.2.3
> dpkg -l git-daemon-run
dpkg -l git-daemon-run
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
||/ Name Version Description
+++-==============================-==============================-============================================================================
pc git-daemon-run 1:1.7.2.3-2.2 fast, scalable, distributed revision contro
> sv stat git-daemon
vking@mail:/etc/sv$ sv stat git-daemon
fail: git-daemon: unable to change to service directory: file does not exist
I uninstalled it so I could run a command line like:
sudo /usr/lib/git-core/git-daemon --verbose --strict-paths --user=git --group=gitosis /srv/gitosis/repositories
or
sudo strace -o strace.out /usr/lib/git-core/git-daemon --verbose --strict-paths --user=git --group=gitosis /srv/gitosis/repositories
I can push to these repos with gitosis, and the permissions are:
vking@mail:/srv/gitosis/repositories$ ll
total 16
drwxr-xr-x 7 gitosis gitosis 4096 Dec 30 12:32 extrudator.data.git
drwxr-xr-x 7 gitosis gitosis 4096 Nov 7 15:23 extrudator.git
drwxr-xr-x 8 gitosis gitosis 4096 Jan 3 14:42 gitosis-admin.git
drwxr-xr-x 7 gitosis gitosis 4096 Jan 2 23:46 tek_7k_ext.git
/etc/group on server
git:x:111:
gitosis:x:115:git
/etc/passwd on server
git:x:106:111:git version control,,,:/home/git:/bin/shh
gitlog:x:108:65534::/nonexistent:/bin/false
gitosis:x:110:115:git repository hosting,,,:/srv/gitosis:/bin/sh
> cat /var/log/git-daemon/current
john@toolbench:~/EEProjects/tek_7k_ext$ cat /var/log/git-daemon/current
cat: /var/log/git-daemon/current: No such file or directory
Thanks for looking at this.
John Griessen
===================================strace output================
vking@mail:/etc/sv$ cat strace.out
execve("/usr/lib/git-core/git-daemon", ["/usr/lib/git-core/git-daemon", "--verbose", "--strict-paths", "--user=git",
"--group=gitosis", "/srv/gitosis/repositories"], [/* 10 vars */]) = 0
brk(0) = 0x89c1000
uname({sys="Linux", node="mail.cibolo.us", ...}) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f19000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=22893, ...}) = 0
mmap2(NULL, 22893, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f13000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/usr/lib/libz.so.1", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\30\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=81012, ...}) = 0
mmap2(NULL, 83740, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x62c000
mmap2(0x640000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13) = 0x640000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/libpthread.so.0", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`I\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=117105, ...}) = 0
mmap2(NULL, 98784, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xbbf000
mprotect(0xbd3000, 4096, PROT_NONE) = 0
mmap2(0xbd4000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xbd4000
mmap2(0xbd6000, 4576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xbd6000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320m\1\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=1315080, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f12000
mmap2(NULL, 1325384, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x110000
mmap2(0x24e000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13d) = 0x24e000
mmap2(0x251000, 10568, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x251000
close(3) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f11000
set_thread_area({entry_number:-1 -> 6, base_addr:0xb7f116c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0,
limit_in_pages:1, seg_not_present:0, useable:1}) = 0
open("/dev/urandom", O_RDONLY) = 3
read(3, "\367\3Y", 3) = 3
close(3) = 0
mprotect(0x24e000, 8192, PROT_READ) = 0
mprotect(0xbd4000, 4096, PROT_READ) = 0
mprotect(0x855000, 4096, PROT_READ) = 0
munmap(0xb7f13000, 22893) = 0
set_tid_address(0xb7f11728) = 19676
set_robust_list(0xb7f11730, 0xc) = 0
futex(0xbf8e5370, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0xbf8e5370, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, bf8e5380) = -1 ENOSYS (Function not implemented)
rt_sigaction(SIGRTMIN, {0xbc33b0, [], SA_RESTORER|SA_SIGINFO, 0xbcce60}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0xbc3840, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0xbcce60}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
uname({sys="Linux", node="mail.cibolo.us", ...}) = 0
brk(0) = 0x89c1000
brk(0x89e2000) = 0x89e2000
socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
open("/etc/nsswitch.conf", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=465, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f18000
read(3, "# /etc/nsswitch.conf\n#\n# Example"..., 4096) = 465
read(3, "", 4096) = 0
close(3) = 0
munmap(0xb7f18000, 4096) = 0
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=22893, ...}) = 0
mmap2(NULL, 22893, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f13000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/libnss_compat.so.2", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0000\16\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=30496, ...}) = 0
mmap2(NULL, 29268, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x254000
mmap2(0x25a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0x25a000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/libnsl.so.1", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\2201\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=79676, ...}) = 0
mmap2(NULL, 92136, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb75000
mmap2(0xb88000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x12) = 0xb88000
mmap2(0xb8a000, 6120, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb8a000
close(3) = 0
mprotect(0xb88000, 4096, PROT_READ) = 0
mprotect(0x25a000, 4096, PROT_READ) = 0
munmap(0xb7f13000, 22893) = 0
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=22893, ...}) = 0
mmap2(NULL, 22893, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f13000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/libnss_nis.so.2", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0000\31\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=38504, ...}) = 0
mmap2(NULL, 37432, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x75f000
mmap2(0x767000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x8) = 0x767000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/libnss_files.so.2", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \32\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=42572, ...}) = 0
mmap2(NULL, 45772, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x284000
mmap2(0x28e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9) = 0x28e000
close(3) = 0
mprotect(0x28e000, 4096, PROT_READ) = 0
mprotect(0x767000, 4096, PROT_READ) = 0
munmap(0xb7f13000, 22893) = 0
open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
fcntl64(3, F_GETFD) = 0x1 (flags FD_CLOEXEC)
_llseek(3, 0, [0], SEEK_CUR) = 0
fstat64(3, {st_mode=S_IFREG|0644, st_size=1804, ...}) = 0
mmap2(NULL, 1804, PROT_READ, MAP_SHARED, 3, 0) = 0xb7f18000
_llseek(3, 1804, [1804], SEEK_SET) = 0
munmap(0xb7f18000, 1804) = 0
close(3) = 0
socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
open("/etc/group", O_RDONLY|O_CLOEXEC) = 3
_llseek(3, 0, [0], SEEK_CUR) = 0
fstat64(3, {st_mode=S_IFREG|0644, st_size=836, ...}) = 0
mmap2(NULL, 836, PROT_READ, MAP_SHARED, 3, 0) = 0xb7f18000
_llseek(3, 836, [836], SEEK_SET) = 0
munmap(0xb7f18000, 836) = 0
close(3) = 0
open("/dev/null", O_RDWR|O_LARGEFILE) = 3
close(3) = 0
socket(PF_NETLINK, SOCK_RAW, 0) = 3
bind(3, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0
getsockname(3, {sa_family=AF_NETLINK, pid=19676, groups=00000000}, [12]) = 0
time(NULL) = 1294097104
sendto(3, "\24\0\0\0\26\0\1\3\320Z\"M\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20
recvmsg(3, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000},
msg_iov(1)=[{"<\0\0\0\24\0\2\0\320Z\"M\334L\0\0\2\10\200\376\1\0\0\0\10\0\1\0\177\0\0\1"..., 4096}], msg_controllen=0,
msg_flags=0}, 0) = 240
recvmsg(3, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000},
msg_iov(1)=[{"@\0\0\0\24\0\2\0\320Z\"M\334L\0\0\n\200\200\376\1\0\0\0\24\0\1\0\0\0\0\0"..., 4096}], msg_controllen=0,
msg_flags=0}, 0) = 64
recvmsg(3, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000},
msg_iov(1)=[{"\24\0\0\0\3\0\2\0\320Z\"M\334L\0\0\0\0\0\0\1\0\0\0\24\0\1\0\0\0\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20
close(3) = 0
open("/etc/gai.conf", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=2940, ...}) = 0
fstat64(3, {st_mode=S_IFREG|0644, st_size=2940, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f18000
read(3, "# Configuration for getaddrinfo("..., 4096) = 2940
read(3, "", 4096) = 0
close(3) = 0
munmap(0xb7f18000, 4096) = 0
futex(0x251cd0, FUTEX_WAKE_PRIVATE, 2147483647) = 0
socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET6, sin6_port=htons(9418), inet_pton(AF_INET6, "::", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0},
28) = 0
getsockname(3, {sa_family=AF_INET6, sin6_port=htons(47484), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0,
sin6_scope_id=0}, [28]) = 0
connect(3, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(9418), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
getsockname(3, {sa_family=AF_INET6, sin6_port=htons(48606), inet_pton(AF_INET6, "::ffff:127.0.0.1", &sin6_addr), sin6_flowinfo=0,
sin6_scope_id=0}, [28]) = 0
close(3) = 0
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
bind(3, {sa_family=AF_INET, sin_port=htons(9418), sin_addr=inet_addr("0.0.0.0")}, 16) = -1 EADDRINUSE (Address already in use)
close(3) = 0
socket(PF_INET6, SOCK_STREAM, IPPROTO_TCP) = 3
setsockopt(3, SOL_IPV6, IPV6_V6ONLY, [1], 4) = 0
bind(3, {sa_family=AF_INET6, sin6_port=htons(9418), inet_pton(AF_INET6, "::", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = 0
listen(3, 5) = 0
fcntl64(3, F_GETFD) = 0
fcntl64(3, F_SETFD, FD_CLOEXEC) = 0
open("/proc/sys/kernel/ngroups_max", O_RDONLY) = 4
read(4, "65536\n", 31) = 6
close(4) = 0
open("/etc/group", O_RDONLY|O_CLOEXEC) = 4
_llseek(4, 0, [0], SEEK_CUR) = 0
fstat64(4, {st_mode=S_IFREG|0644, st_size=836, ...}) = 0
mmap2(NULL, 836, PROT_READ, MAP_SHARED, 4, 0) = 0xb7f18000
_llseek(4, 836, [836], SEEK_SET) = 0
fstat64(4, {st_mode=S_IFREG|0644, st_size=836, ...}) = 0
munmap(0xb7f18000, 836) = 0
close(4) = 0
setgroups32(1, [115]) = 0
setgid32(115) = 0
setuid32(106) = 0
rt_sigaction(SIGCHLD, {0x804ae30, [CHLD], SA_RESTORER|SA_RESTART, 0x13a5d8}, {SIG_DFL, [], 0}, 8) = 0
waitpid(-1, 0xbf8e525c, WNOHANG) = -1 ECHILD (No child processes)
poll([{fd=3, events=POLLIN}], 1, -1) = -1 EINTR (Interrupted system call)
--- SIGINT (Interrupt) @ 0 (0) ---
+++ killed by SIGINT +++
vking@mail:/etc/sv$
=========================================================
JG
^ permalink raw reply
* Re: [RFC PATCH v7 11/9] [PoC] gitweb/lib - tee, i.e. print and capture during cache entry generation
From: J.H. @ 2011-01-03 23:31 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git, John 'Warthog9' Hawley
In-Reply-To: <201101032233.16174.jnareb@gmail.com>
On 01/03/2011 01:33 PM, Jakub Narebski wrote:
> Instead of having gitweb use progress info indicator / throbber to
> notify user that data is being generated by current process, gitweb
> can now (provided that PerlIO::tee from PerlIO::Util is available)
> send page to web browser while simultaneously saving it to cache
> (print and capture, i.e. tee), thus having incremental generating of
> page serve as a progress indicator.
In general, and particularly for the large sites that caching is
targeted at, teeing is a really bad idea. I've mentioned this several
times before, and the progress indicator is a *MUCH* better idea. I'm
not sure how many times I can say that, even if this was added it would
have the potential to exacerbate disk thrashing and overall make things
a lot more complex.
1) Errors may still be generated in flight as the cache is being
generated. It would be better to let the cache run with a progress
indicator and should an error occur, display the error instead of giving
any output that may have been generated (and thus likely a broken page).
2) Having multiple clients all waiting on the same page (in particular
the index page) can lead to invalid output. In particular if you are
teeing the output a reading client now must come in, read the current
contents of the file (as written), then pick up on the the tee after
that. It's actually possible for the reading client to miss data as it
may be in flight to be written and the client is switching from reading
the file to reading the tee. I don't see anything in your code to
handle that kind of switch over.
3) This makes no allowance for the file to be generated completely in
the background while serving stale data in the interim. Keep in mind
that it can (as Fedora has experienced) take *HOURS* to generate the
index page, teeing that output just means brokenness and isn't useful.
It's much better to have a simple, lightweight waiting message get
displayed while things happen. When they are done, output the completed
page to all waiting clients.
- John 'Warthog9' Hawley
P.S. I'm back to work full-time on Wednesday, which I'll be catching up
on gitweb and trying to make forward progress on my gitweb code again.
^ permalink raw reply
* Re: git-daemon serving repos with repo.git/git-daemon-export-ok
From: Jonathan Nieder @ 2011-01-03 23:11 UTC (permalink / raw)
To: John Griessen; +Cc: git
In-Reply-To: <4D224475.1090805@ecosensory.com>
Hi John,
John Griessen wrote:
> john@toolbench:~/EEProjects/test$ git clone git://ecosensory.com/tek_7k_ext.git
> Cloning into tek_7k_ext...
> fatal: protocol error: bad line length character: fata
[...]
> Next I tried changing my command that runs git-daemon
> from
>
> exec 2>&1
> echo 'git-daemon starting.'
> exec chpst -ugitdaemon \
> "$(git --exec-path)"/git-daemon --verbose --base-path=/var/cache /var/cache/git
Worrisome. Based on /usr/share/doc/git-daemon-run/README.Debian,
I would be interested in:
git --version
dpkg -l git-daemon-run
sv stat git-daemon
cat /var/log/git-daemon/current
It looks like the daemon is sending a malformed reply starting with
"fatal: ", similar to the bug fixed by v1.7.0.3~7 (daemon:
parse_host_and_port SIGSEGV if port is specified, 2010-03-20).
It the error message does not provide any insight, I'd also suggest
using strace or ltrace to attach to the running git-daemon process to
see what's going on.
Thanks for reporting,
Jonathan
^ permalink raw reply
* Re: git-daemon serving repos with repo.git/git-daemon-export-ok
From: John Griessen @ 2011-01-03 22:47 UTC (permalink / raw)
To: git
In-Reply-To: <4D224475.1090805@ecosensory.com>
On 01/03/2011 03:49 PM, John Griessen wrote:
> I would like to serve repos with repo.git/git-daemon-export-ok from a debian linux server installation
on a virtual server (openVZ) -- I've read there is something bad about virtual servers and git-daemon?
John G
^ permalink raw reply
* [RFC PATCH v7 11/9] [PoC] gitweb/lib - tee, i.e. print and capture during cache entry generation
From: Jakub Narebski @ 2011-01-03 21:33 UTC (permalink / raw)
To: git; +Cc: J.H., John 'Warthog9' Hawley
In-Reply-To: <20101222234843.7998.87068.stgit@localhost.localdomain>
Instead of having gitweb use progress info indicator / throbber to
notify user that data is being generated by current process, gitweb
can now (provided that PerlIO::tee from PerlIO::Util is available)
send page to web browser while simultaneously saving it to cache
(print and capture, i.e. tee), thus having incremental generating of
page serve as a progress indicator.
To do this, the GitwebCache::Capture::ToFile module acquired ->tee()
subroutine, similar to ->capture(), but it prints while capturing
output. The ->tee() method (and its worker methods ->tee_start() and
->tee_end()) are available only if PerlIO::tee from PerlIO::Util
distribution is present. Tests checking if this feature works as
expected were added to t9510 test.
An alternative would be to provide two versions of
GitwebCache::Capture::ToFile. Note also that PerlIO::tee is not
strictly necessary, as Capture::Tiny shows, but in most generic case
(like done in Capture::Tiny) one needs separate process functioning as
multplexer.
Because tee-ing (printing while capturing) can function as a kind of
progress indicator only for process generating the data for cache
entry, and not for the processes waiting for data to be generated,
therefore 'generating_info' got splitinto 'get_progress_info' and
'set_progress_info'. You can set now in GitwebCache::FIleCacheWithLocking
those two separately. You are expected to unset 'set_progress_info'
when using tee-ing capturing engine. Some tests added to t9511 with
tee-like situation.
As a proof of concept gitweb now uses two slightly different versions
of "Generating..." page; if you worry about interaction between progress
indicator and non-cacheable error pages, you can set 'set_progress_info'
separately to undef.
The cache_output subroutine from GitwebCache::CacheOutput got updated
to use ->tee() subroutine if $capture supports it. If ->tee() is
used, then of course generated data doesn't need to and shouldn't be
printed; also cache_output unsets 'set_progress_info' locally. Note
that ->tee() is used only if we are not in background process; if we
are in background process, simple ->capture() is used. No new tests
for now.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
Note: the change to t/gitweb-lib.sh and some of changes to t9510 are
incidental fixes; original commits should be fixed instead.
This is proof of concept (PoC) patch, showing how one can use
"tee"-ing in capturing engine together with gitweb output caching.
Because we don't need and don't use 'generating_info' subroutine for
process that is writing data (one that acquired writers lock) we are
(or at least should be) now safe to have error pages not cached.
Currently the "tee"-ing support requires PerlIO::tee module from the
PerlIO::Util distribution, as it was easiest way to add such feature.
In the future we would have probably to do something similar what
'tee' in Capture::Tiny (or in other capture modules) does. I'm not
sure if PerlIO::Util is packaged as RPM package anywhere...; well, I have
googled that ALT Linux has it: http://sisyphus.ru/en/srpm/perl-PerlIO-Util
I have only ran tests, I haven't actually run gitweb with those
changes... :-P
gitweb/gitweb.perl | 32 +++++++----
gitweb/lib/GitwebCache/CacheOutput.pm | 18 ++++++-
gitweb/lib/GitwebCache/Capture/ToFile.pm | 67 ++++++++++++++++++++++-
gitweb/lib/GitwebCache/FileCacheWithLocking.pm | 52 +++++++++++++-----
t/gitweb-lib.sh | 2 +-
t/t9510/test_capture_interface.pl | 28 +++++++++-
t/t9511/test_cache_interface.pl | 29 ++++++++++
7 files changed, 194 insertions(+), 34 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 5ef668d..de283a0 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -325,19 +325,17 @@ our %cache_options = (
# serving possibly stale data.
'background_cache' => 1,
- # Subroutine which would be called when gitweb has to wait for data to
+ # Subroutines which would be called when gitweb has to wait for data to
# be generated (it can't serve stale data because there isn't any,
- # or if it exists it is older than 'max_lifetime'). The default
- # is to use git_generating_data_html(), which creates "Generating..."
- # page, which would then redirect or redraw/rewrite the page when
- # data is ready.
- # Set it to `undef' to disable this feature.
+ # or if it exists it is older than 'max_lifetime').
+ # Set them to `undef' to disable this feature.
#
- # Such subroutine (if invoked from GitwebCache::FileCacheWithLocking)
+ # Such subroutines (if invoked from GitwebCache::FileCacheWithLocking)
# is passed the following parameters: $cache instance, human-readable
# $key to current page, and $sync_coderef subroutine to invoke to wait
# (in a blocking way) for data.
- 'generating_info' => \&git_generating_data_html,
+ 'get_progress_info' => \&git_get_progress_info_html,
+ 'set_progress_info' => \&git_set_progress_info_html,
# How to handle runtime errors occurring during cache gets and cache
# sets. Options are:
@@ -3721,9 +3719,21 @@ sub get_page_title {
return $title;
}
+sub git_get_progress_info_html {
+ git_generating_data_html("Waiting", @_);
+}
+
+sub git_set_progress_info_html {
+ # minimum startup delay is 2 seconds, just in case, for error handling
+ local $generating_options{'startup_delay'} =
+ $generating_options{'startup_delay'} > 2 ? $generating_options{'startup_delay'} : 2;
+
+ git_generating_data_html("Generating", @_);
+}
+
# creates "Generating..." page when caching enabled and not in cache
sub git_generating_data_html {
- my ($cache, $key, $sync_coderef) = @_;
+ my ($msg, $cache, $key, $sync_coderef) = @_;
# when should gitweb show "Generating..." page
if ((defined $actions_info{$action}{'output_format'} &&
@@ -3749,7 +3759,7 @@ sub git_generating_data_html {
}
}
- my $title = "[Generating...] " . get_page_title();
+ my $title = "[$msg...] " . get_page_title();
# TODO: the following line of code duplicates the one
# in git_header_html, and it should probably be refactored.
my $mod_perl_version = $ENV{'MOD_PERL'} ? " $ENV{'MOD_PERL'}" : '';
@@ -3786,7 +3796,7 @@ sub git_generating_data_html {
EOF
local $| = 1; # autoflush
- print STDOUT 'Generating...';
+ print STDOUT "$msg...";
my $total_time = 0;
my $interval = $generating_options{'print_interval'} || 1;
diff --git a/gitweb/lib/GitwebCache/CacheOutput.pm b/gitweb/lib/GitwebCache/CacheOutput.pm
index 188d4ab..3bcd35b 100644
--- a/gitweb/lib/GitwebCache/CacheOutput.pm
+++ b/gitweb/lib/GitwebCache/CacheOutput.pm
@@ -42,6 +42,14 @@ sub cache_output {
my $pid = $$;
+ my $can_tee = $capture->can('tee');
+ # if $capture can tee, we don't need progress info for generating (on set).
+ # the below breaks encapsulation, but it is a bit simpler than
+ # $old = $cache->get_...; $cache->set_...(...); ...; $cache->set_...($old);
+ local $cache->{'set_progress_info'} = undef
+ if ($can_tee);
+
+ my $printed = 0;
my ($fh, $filename);
my ($capture_fh, $capture_filename);
eval { # this `eval` is to catch rethrown error, so we can print captured output
@@ -54,7 +62,12 @@ sub cache_output {
}
# this `eval` is to be able to cache error output (up till 'die')
- eval { $capture->capture($code, $capture_fh); };
+ if ($can_tee && $$ == $pid) {
+ $printed = 1;
+ eval { $capture->tee($code, $capture_fh); };
+ } else {
+ eval { $capture->capture($code, $capture_fh); };
+ }
# note that $cache can catch this error itself (like e.g. CHI);
# use "die"-ing error handler to rethrow this exception to outside
@@ -69,7 +82,8 @@ sub cache_output {
$filename ||= $capture_filename;
}
- if (defined $fh || defined $filename) {
+ if ((defined $fh || defined $filename) &&
+ !$printed) { # did we tee, i.e. already printed output?
# set binmode only if $fh is defined (is a filehandle)
# File::Copy::copy opens files given by filename in binary mode
binmode $fh, ':raw' if (defined $fh);
diff --git a/gitweb/lib/GitwebCache/Capture/ToFile.pm b/gitweb/lib/GitwebCache/Capture/ToFile.pm
index d2dbf0f..0290ec4 100644
--- a/gitweb/lib/GitwebCache/Capture/ToFile.pm
+++ b/gitweb/lib/GitwebCache/Capture/ToFile.pm
@@ -20,6 +20,10 @@ use warnings;
use PerlIO;
use Symbol qw(qualify_to_ref);
+BEGIN {
+ eval { use PerlIO::Util; };
+}
+
# Constructor
sub new {
my $class = shift;
@@ -30,22 +34,41 @@ sub new {
return $self;
}
-sub capture {
+sub capture_or_tee {
my $self = shift;
my $code = shift;
+ my ($start, $stop) = @{ shift() };
- $self->capture_start(@_); # pass rest of params
+ $self->$start(@_); # pass rest of params
eval { $code->(); 1; };
my $exit_code = $?; # save this for later
my $error = $@; # save this for later
- my $got_out = $self->capture_stop();
+ my $got_out = $self->$stop();
$? = $exit_code;
die $error if $error;
return $got_out;
}
+sub capture {
+ my ($self, $code, @args) = @_;
+
+ return
+ $self->capture_or_tee($code, ['capture_start', 'capture_stop'], @args);
+}
+
+BEGIN {
+ if ($INC{'PerlIO/Util.pm'}) {
+ *tee = sub {
+ my ($self, $code, @args) = @_;
+
+ return
+ $self->capture_or_tee($code, ['tee_start', 'tee_stop'], @args);
+ };
+ }
+}
+
# ----------------------------------------------------------------------
# Start capturing data (STDOUT)
@@ -92,6 +115,44 @@ sub capture_stop {
return exists $self->{'to'} ? $self->{'to'} : $self->{'data'};
}
+# ......................................................................
+
+BEGIN {
+ if ($INC{'PerlIO/Util.pm'}) {
+ *tee_start = sub {
+ my ($self, $to) = @_;
+
+ # save layers, to replay them on top of 'tee' layer (?)
+ my @layers = PerlIO::get_layers(\*STDOUT);
+
+ $self->{'to'} = $to;
+ *STDOUT->push_layer('tee' => $to);
+
+ _relayer(\*STDOUT, \@layers); # is it necessary?
+
+ # started tee-ing
+ $self->{'teeing'} = 1;
+ };
+ *tee_stop = sub {
+ my $self = shift;
+
+ # return if we didn't start tee-ing
+ return unless delete $self->{'teeing'};
+
+ my @top_layers;
+ while ((my $layer = *STDOUT->pop_layer()) ne 'tee') {
+ push @top_layers, $layer;
+ }
+ binmode(STDOUT, join(":", ":", @top_layers));
+ # or is it binmode(STDOUT, join(":", ":raw", @top_layers));
+
+ return exists $self->{'to'} ? $self->{'to'} : $self->{'data'};
+ };
+ }
+}
+
+# ----------------------------------------------------------------------
+
# taken from Capture::Tiny by David Golden, Apache License 2.0
# with debugging stripped out
sub _relayer {
diff --git a/gitweb/lib/GitwebCache/FileCacheWithLocking.pm b/gitweb/lib/GitwebCache/FileCacheWithLocking.pm
index 291526e..09ae7b2 100644
--- a/gitweb/lib/GitwebCache/FileCacheWithLocking.pm
+++ b/gitweb/lib/GitwebCache/FileCacheWithLocking.pm
@@ -76,12 +76,17 @@ our $EXPIRE_NOW = 0;
# * 'background_cache' (boolean)
# This enables/disables regenerating cache in background process.
# Defaults to true.
-# * 'generating_info'
+# * 'get_progress_info',
+# 'set_progress_info',
+# 'generating_info' (code reference)
# Subroutine (code) called when process has to wait for cache entry
# to be (re)generated (when there is no not-too-stale data to serve
# instead), for other process (or bacground process). It is passed
# $cache instance, $key, and $wait_code subroutine (code reference)
# to invoke (to call) to wait for cache entry to be ready.
+# 'get_progress_info' gets called on getting data from cache, i.e.
+# when waiting for data to be generated, 'set_progress_info' gets
+# called when waiting to generate data; 'generating_info' sets both.
# Unset by default (which means no activity indicator).
# * 'on_error' (similar to CHI 'on_get_error'/'on_set_error')
# How to handle runtime errors occurring during cache gets and cache
@@ -120,8 +125,14 @@ sub new {
$self->{'background_cache'} =
exists $opts{'background_cache'} ? $opts{'background_cache'} :
1;
- $self->{'generating_info'} = $opts{'generating_info'}
- if exists $opts{'generating_info'};
+ $self->{'get_progress_info'} =
+ exists $opts{'get_progress_info'} ? $opts{'get_progress_info'} :
+ exists $opts{'generating_info'} ? $opts{'generating_info'} :
+ undef;
+ $self->{'set_progress_info'} =
+ exists $opts{'set_progress_info'} ? $opts{'set_progress_info'} :
+ exists $opts{'generating_info'} ? $opts{'generating_info'} :
+ undef;
$self->{'on_error'} =
exists $opts{'on_error'} ? $opts{'on_error'} :
exists $opts{'on_get_error'} ? $opts{'on_get_error'} :
@@ -142,7 +153,7 @@ sub new {
# creates get_depth() and set_depth($depth) etc. methods
foreach my $i (qw(depth root namespace expires_in max_lifetime
- background_cache generating_info
+ background_cache get_progress_info set_progress_info
on_error)) {
my $field = $i;
no strict 'refs';
@@ -156,14 +167,25 @@ foreach my $i (qw(depth root namespace expires_in max_lifetime
};
}
-# $cache->generating_info($wait_code);
-# runs 'generating_info' subroutine, for activity indicator,
-# checking if it is defined first.
-sub generating_info {
+sub set_generating_info {
my $self = shift;
- if (defined $self->{'generating_info'}) {
- $self->{'generating_info'}->($self, @_);
+ $self->set_get_progress_info(@_);
+ $self->set_set_progress_info(@_);
+}
+
+# $cache->{get,set}_progress_info($key, $wait_code);
+# runs '{get,set}_progress_info' subroutine, for activity indicator,
+# checking if it is defined first.
+foreach my $name qw(get_progress_info set_progress_info) {
+ my $method = $name;
+ no strict 'refs';
+ *{"$method"} = sub {
+ my $self = shift;
+
+ if (defined $self->{$name}) {
+ $self->{$name}->($self, @_);
+ }
}
}
@@ -269,11 +291,11 @@ sub _tempfile_to_path {
# Wait for data to be available using (blocking) $code,
# then return filehandle and filename to read from for $key.
sub _wait_for_data {
- my ($self, $key, $sync_coderef) = @_;
+ my ($self, $key, $progress_info, $sync_coderef) = @_;
my @result;
# provide "generating page..." info, if exists
- $self->generating_info($key, $sync_coderef);
+ $self->$progress_info($key, $sync_coderef);
# generating info may exit, so we can not get there
# wait for data to be available
@@ -300,7 +322,7 @@ sub _set_maybe_background {
# or if main process would show progress indicator
$detach = @stale_result;
$pid = fork()
- if (@stale_result || $self->{'generating_info'});
+ if (@stale_result || $self->{'set_progress_info'});
}
if ($pid) {
@@ -503,7 +525,7 @@ sub compute_fh {
or $self->_handle_error("Couldn't reopen (for reading) lockfile '$lockfile': $!");
eval {
- @result = $self->_wait_for_data($key, sub {
+ @result = $self->_wait_for_data($key, 'set_progress_info', sub {
flock($lock_fh, LOCK_SH);
# or 'waitpid -1, 0;', or 'wait;', as we don't detach now in this situation
});
@@ -534,7 +556,7 @@ sub compute_fh {
# wait for regeneration if no stale data to serve,
# using shared / readers lock to sync (wait for data)
eval {
- @result = $self->_wait_for_data($key, sub {
+ @result = $self->_wait_for_data($key, 'get_progress_info', sub {
flock($lock_fh, LOCK_SH);
});
};
diff --git a/t/gitweb-lib.sh b/t/gitweb-lib.sh
index 8652c91..f0ef009 100755
--- a/t/gitweb-lib.sh
+++ b/t/gitweb-lib.sh
@@ -57,7 +57,7 @@ gitweb_enable_caching () {
cat >>gitweb_config.perl <<-\EOF &&
$caching_enabled = 1;
$cache_options{"expires_in"} = -1; # never expire cache for tests
- $cache_options{"cache_root"} = "$TRASH_DIRECTORY/cache"; # to clear the right thing
+ $cache_options{"cache_root"} = "cache"; # to clear the right thing
$cache_options{"background_cache"} = 0; # no background processes in test suite
$cache_options{"generating_info"} = undef; # tests do not use web browser
EOF
diff --git a/t/t9510/test_capture_interface.pl b/t/t9510/test_capture_interface.pl
index 6d90497..35e46ad 100755
--- a/t/t9510/test_capture_interface.pl
+++ b/t/t9510/test_capture_interface.pl
@@ -116,8 +116,8 @@ $captured = $outer_capture->capture(sub {
print "|post";
}, 'outer_actual');
-my $inner = read_file('inner_actual');
-my $outer = read_file('outer_actual');
+$inner = read_file('inner_actual');
+$outer = read_file('outer_actual');
is($inner, "INNER:pre|",
'nested capture with die: inner output captured up to die');
@@ -125,6 +125,30 @@ is($outer, "pre|@=die from inner\n|post",
'nested capture with die: outer caught rethrown exception from inner');
+# Testing tee feature, if available
+#
+SKIP: {
+ skip "PerlIO::Util module not found", 3
+ unless eval { require PerlIO::Util; 1 };
+
+ can_ok($capture, 'tee');
+
+ $captured = $outer_capture->capture(sub {
+ print "pre|";
+ my $captured = $capture->tee(sub {
+ print "INNER";
+ }, 'inner_actual');
+ print "|post";
+ }, 'outer_actual');
+
+ $inner = read_file('inner_actual');
+ $outer = read_file('outer_actual');
+
+ is($inner, "INNER", 'tee: captured');
+ is($outer, "pre|INNER|post", 'tee: printed');
+};
+
+
done_testing();
# Local Variables:
diff --git a/t/t9511/test_cache_interface.pl b/t/t9511/test_cache_interface.pl
index 1e8feb3..83f3894 100755
--- a/t/t9511/test_cache_interface.pl
+++ b/t/t9511/test_cache_interface.pl
@@ -154,6 +154,14 @@ sub get_value_slow_fh {
sleep $slow_time;
print {$fh} $value;
}
+sub tee_value_slow_fh {
+ my $fh = shift;
+
+ $call_count++;
+ sleep $slow_time;
+ print $value;
+ print {$fh} $value;
+}
sub get_value_die {
$call_count++;
die "get_value_die\n";
@@ -402,6 +410,26 @@ subtest 'generating progress info' => sub {
);
+ # with background generation, tee-like, no stale data
+ $cache->set_set_progress_info(undef);
+ $cache->set_background_cache(1);
+ $cache->remove($key); # no data and no stale value
+ $call_count = 0;
+
+ @output = parallel_run {
+ my $data = cache_compute_fh($cache, $key, \&tee_value_slow_fh);
+ print "$sep$call_count$sep";
+ print $data if defined $data;
+ };
+ my $getting_output = (grep /\Q${sep}0${sep}\E/, @output)[0];
+ my $setting_output = (grep /\Q${sep}1${sep}\E/, @output)[0];
+ is($getting_output, "$progress_info${sep}0$sep$value",
+ 'background, no stale, tee: waiting process prints progress, gets data');
+ is($setting_output, "$value${sep}1$sep$value",
+ 'background, no stale, tee: generating process prints data, sets data');
+ $cache->set_generating_info(\&test_generating_info); # restore
+
+
# with background generation, with stale value
cache_set_fh($cache, $key, $stale_value);
$cache->set_expires_in(0); # set value is now expired
@@ -427,6 +455,7 @@ subtest 'generating progress info' => sub {
};
$cache->set_expires_in(-1);
+
done_testing();
--
1.7.3
^ permalink raw reply related
* git-daemon serving repos with repo.git/git-daemon-export-ok
From: John Griessen @ 2011-01-03 21:49 UTC (permalink / raw)
To: git
In-Reply-To: <S1751603Ab1ACU6e/20110103205834Z+1762@vger.kernel.org>
I would like to serving repos with repo.git/git-daemon-export-ok from a debian linux server installation,
but I am getting symptoms of clone failing.
john@toolbench:~/EEProjects/test$ git clone git://ecosensory.com/tek_7k_ext.git
Cloning into tek_7k_ext...
fatal: protocol error: bad line length character: fata
I've read that that is due to the shell at the client end putting out extra characters...
but other testers on other machines get it the same way...
Next I tried changing my command that runs git-daemon
from
exec 2>&1
echo 'git-daemon starting.'
exec chpst -ugitdaemon \
"$(git --exec-path)"/git-daemon --verbose --base-path=/var/cache /var/cache/git
to
exec 2>&1
echo 'git-daemon starting.'
exec chpst -ugitdaemon \
"$(git --exec-path)"/git-daemon --verbose --base-path=/srv/gitosis /srv/gitosis/repositories
and results are:
john@toolbench:~/EEProjects/test$ git clone git://ecosensory.com/tek_7k_ext.git
Cloning into tek_7k_ext...
fatal: protocol error: bad line length character: fata
john@toolbench:~/EEProjects/test$ git clone git://ecosensory.com/repositories/tek_7k_ext.git
Cloning into tek_7k_ext...
fatal: protocol error: bad line length character: fata
What can I do to simplify this testing and pin point the problem?
Or is gitweb recommended?
thanks,
John Griessen
^ permalink raw reply
* [PATCH] Fix typos in the documentation
From: Ralf Wildenhues @ 2011-01-03 19:03 UTC (permalink / raw)
To: Drew Northup; +Cc: git
In-Reply-To: <1294077450.12519.15.camel@drew-northup.unet.maine.edu>
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
---
Hello Drew,
* Drew Northup wrote on Mon, Jan 03, 2011 at 06:57:30PM CET:
> On Sun, 2011-01-02 at 06:56 +0100, Ralf Wildenhues wrote:
> > --- a/Documentation/RelNotes/1.7.4.txt
> > +++ b/Documentation/RelNotes/1.7.4.txt
> > - * The option parsers of various commands that create new branch (or
> > + * The option parsers of various commands that create new branches (or
> > rename existing ones to a new name) were too loose and users were
> > - allowed to call a branch with a name that begins with a dash by
> > - creative abuse of their command line options, which only lead to
> > - burn themselves. The name of a branch cannot begin with a dash
> > - now.
> > + allowed to give a branch a name that begins with a dash by creative
> > + abuse of their command line options, which only led to burn themselves.
> > + The name of a branch cannot begin with a dash now.
> Ralf,
> Perhaps that should be:
>
> - * The option parsers of various commands that create new branch (or
> + * The option parsers of various commands that create new branches (or
> rename existing ones to a new name) were too loose and users were
> - allowed to call a branch with a name that begins with a dash by
> - creative abuse of their command line options, which only lead to
> - burn themselves. The name of a branch cannot begin with a dash
> - now.
> + allowed to give a branch a name that begins with a dash by creative
> + abuse of their command line options, which only led to burning
> + themselves. The name of a branch cannot begin with a dash now.
>
> (for consistency)?
As far as I can see, the only difference between my and your change is
"burn" vs. "burning" (ignoring trailing white space). I don't see what
your version is consistent to that mine isn't, but I think that yours
is the grammatically correct way. I'm not a native speaker though.
Anyway, here's the updated patch.
Thanks,
Ralf
Documentation/CodingGuidelines | 2 +-
Documentation/RelNotes/1.7.4.txt | 9 ++++-----
Documentation/config.txt | 4 ++--
Documentation/git-add.txt | 2 +-
Documentation/git-bundle.txt | 2 +-
Documentation/git-fmt-merge-msg.txt | 2 +-
Documentation/git-remote-ext.txt | 16 ++++++++--------
Documentation/git-remote-fd.txt | 8 ++++----
Documentation/git-svn.txt | 2 +-
Documentation/gitmodules.txt | 2 +-
Documentation/merge-config.txt | 2 +-
Documentation/technical/api-sigchain.txt | 2 +-
12 files changed, 26 insertions(+), 27 deletions(-)
diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines
index 1b1c45d..ba2006d 100644
--- a/Documentation/CodingGuidelines
+++ b/Documentation/CodingGuidelines
@@ -157,7 +157,7 @@ Writing Documentation:
--sort=<key>
--abbrev[=<n>]
- Possibility of multiple occurences is indicated by three dots:
+ Possibility of multiple occurrences is indicated by three dots:
<file>...
(One or more of <file>.)
diff --git a/Documentation/RelNotes/1.7.4.txt b/Documentation/RelNotes/1.7.4.txt
index b736d39..375fea1 100644
--- a/Documentation/RelNotes/1.7.4.txt
+++ b/Documentation/RelNotes/1.7.4.txt
@@ -8,12 +8,11 @@ Updates since v1.7.3
docbook-xsl >= 1.73. If you have older versions, you can set
ASCIIDOC7 and ASCIIDOC_ROFF, respectively.
- * The option parsers of various commands that create new branch (or
+ * The option parsers of various commands that create new branches (or
rename existing ones to a new name) were too loose and users were
- allowed to call a branch with a name that begins with a dash by
- creative abuse of their command line options, which only lead to
- burn themselves. The name of a branch cannot begin with a dash
- now.
+ allowed to give a branch a name that begins with a dash by creative
+ abuse of their command line options, which only led to burning
+ themselves. The name of a branch cannot begin with a dash now.
* System-wide fallback default attributes can be stored in
/etc/gitattributes; core.attributesfile configuration variable can
diff --git a/Documentation/config.txt b/Documentation/config.txt
index a8759cf..ff7c225 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -892,7 +892,7 @@ diff.wordRegex::
fetch.recurseSubmodules::
A boolean value which changes the behavior for fetch and pull, the
- default is to not recursively fetch populated sumodules unless
+ default is to not recursively fetch populated submodules unless
configured otherwise.
fetch.unpackLimit::
@@ -1811,7 +1811,7 @@ submodule.<name>.update::
submodule.<name>.fetchRecurseSubmodules::
This option can be used to enable/disable recursive fetching of this
- submodule. It can be overriden by using the --[no-]recurse-submodules
+ submodule. It can be overridden by using the --[no-]recurse-submodules
command line option to "git fetch" and "git pull".
This setting will override that from in the linkgit:gitmodules[5]
file.
diff --git a/Documentation/git-add.txt b/Documentation/git-add.txt
index 54aaaeb..a03448f 100644
--- a/Documentation/git-add.txt
+++ b/Documentation/git-add.txt
@@ -333,7 +333,7 @@ likely to introduce confusing changes to the index.
There are also more complex operations that can be performed. But beware
that because the patch is applied only to the index and not the working
tree, the working tree will appear to "undo" the change in the index.
-For example, introducing a a new line into the index that is in neither
+For example, introducing a new line into the index that is in neither
the HEAD nor the working tree will stage the new line for commit, but
the line will appear to be reverted in the working tree.
diff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt
index 6266a3a..299007b 100644
--- a/Documentation/git-bundle.txt
+++ b/Documentation/git-bundle.txt
@@ -59,7 +59,7 @@ unbundle <file>::
<git-rev-list-args>::
A list of arguments, acceptable to 'git rev-parse' and
- 'git rev-list' (and containg a named ref, see SPECIFYING REFERENCES
+ 'git rev-list' (and containing a named ref, see SPECIFYING REFERENCES
below), that specifies the specific objects and references
to transport. For example, `master{tilde}10..master` causes the
current master reference to be packaged along with all objects
diff --git a/Documentation/git-fmt-merge-msg.txt b/Documentation/git-fmt-merge-msg.txt
index 40dba8c..75adf7a 100644
--- a/Documentation/git-fmt-merge-msg.txt
+++ b/Documentation/git-fmt-merge-msg.txt
@@ -57,7 +57,7 @@ merge.log::
In addition to branch names, populate the log message with at
most the specified number of one-line descriptions from the
actual commits that are being merged. Defaults to false, and
- true is a synoym for 20.
+ true is a synonym for 20.
merge.summary::
Synonym to `merge.log`; this is deprecated and will be removed in
diff --git a/Documentation/git-remote-ext.txt b/Documentation/git-remote-ext.txt
index f4fbf67..2d65cfe 100644
--- a/Documentation/git-remote-ext.txt
+++ b/Documentation/git-remote-ext.txt
@@ -15,12 +15,12 @@ This remote helper uses the specified 'program' to connect
to a remote git server.
Data written to stdin of this specified 'program' is assumed
-to be sent to git:// server, git-upload-pack, git-receive-pack
+to be sent to a git:// server, git-upload-pack, git-receive-pack
or git-upload-archive (depending on situation), and data read
from stdout of this program is assumed to be received from
the same service.
-Command and arguments are separated by unescaped space.
+Command and arguments are separated by an unescaped space.
The following sequences have a special meaning:
@@ -39,19 +39,19 @@ The following sequences have a special meaning:
git-upload-pack, or git-upload-archive) of the service
git wants to invoke.
-'%G' (must be first characters in argument)::
+'%G' (must be the first characters in an argument)::
This argument will not be passed to 'program'. Instead, it
- will cause helper to start by sending git:// service request to
- remote side with service field set to approiate value and
- repository field set to rest of the argument. Default is not to send
- such request.
+ will cause the helper to start by sending git:// service requests to
+ the remote side with the service field set to an appropriate value and
+ the repository field set to rest of the argument. Default is not to send
+ such a request.
+
This is useful if remote side is git:// server accessed over
some tunnel.
'%V' (must be first characters in argument)::
This argument will not be passed to 'program'. Instead it sets
- the vhost field in git:// service request (to rest of the argument).
+ the vhost field in the git:// service request (to rest of the argument).
Default is not to send vhost in such request (if sent).
ENVIRONMENT VARIABLES:
diff --git a/Documentation/git-remote-fd.txt b/Documentation/git-remote-fd.txt
index abc4944..4aecd4d 100644
--- a/Documentation/git-remote-fd.txt
+++ b/Documentation/git-remote-fd.txt
@@ -11,20 +11,20 @@ SYNOPSIS
DESCRIPTION
-----------
-This helper uses specified file descriptors to connect to remote git server.
+This helper uses specified file descriptors to connect to a remote git server.
This is not meant for end users but for programs and scripts calling git
fetch, push or archive.
-If only <infd> is given, it is assumed to be bidirectional socket connected
+If only <infd> is given, it is assumed to be a bidirectional socket connected
to remote git server (git-upload-pack, git-receive-pack or
git-upload-achive). If both <infd> and <outfd> are given, they are assumed
-to be pipes connected to remote git server (<infd> being the inbound pipe
+to be pipes connected to a remote git server (<infd> being the inbound pipe
and <outfd> being the outbound pipe.
It is assumed that any handshaking procedures have already been completed
(such as sending service request for git://) before this helper is started.
-<anything> can be any string. It is ignored. It is meant for provoding
+<anything> can be any string. It is ignored. It is meant for providing
information to user in the URL in case that URL is displayed in some
context.
diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt
index 139d314..254d044 100644
--- a/Documentation/git-svn.txt
+++ b/Documentation/git-svn.txt
@@ -613,7 +613,7 @@ old references to SVN revision numbers in existing documentation, bug
reports and archives. If you plan to eventually migrate from SVN to git
and are certain about dropping SVN history, consider
linkgit:git-filter-branch[1] instead. filter-branch also allows
-reformating of metadata for ease-of-reading and rewriting authorship
+reformatting of metadata for ease-of-reading and rewriting authorship
info for non-"svn.authorsFile" users.
svn.useSvmProps::
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index 6c93202..6897794 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -49,7 +49,7 @@ submodule.<name>.fetchRecurseSubmodules::
submodule. If this option is also present in the submodules entry in
.git/config of the superproject, the setting there will override the
one found in .gitmodules.
- Both settings can be overriden on the command line by using the
+ Both settings can be overridden on the command line by using the
"--[no-]recurse-submodules" option to "git fetch" and "git pull"..
submodule.<name>.ignore::
diff --git a/Documentation/merge-config.txt b/Documentation/merge-config.txt
index 92772e7..1e5c22c 100644
--- a/Documentation/merge-config.txt
+++ b/Documentation/merge-config.txt
@@ -10,7 +10,7 @@ merge.log::
In addition to branch names, populate the log message with at
most the specified number of one-line descriptions from the
actual commits that are being merged. Defaults to false, and
- true is a synoym for 20.
+ true is a synonym for 20.
merge.renameLimit::
The number of files to consider when performing rename detection
diff --git a/Documentation/technical/api-sigchain.txt b/Documentation/technical/api-sigchain.txt
index 535cdff..9e1189e 100644
--- a/Documentation/technical/api-sigchain.txt
+++ b/Documentation/technical/api-sigchain.txt
@@ -32,7 +32,7 @@ and installation code should look something like:
}
------------------------------------------
-Handlers are given the typdef of sigchain_fun. This is the same type
+Handlers are given the typedef of sigchain_fun. This is the same type
that is given to signal() or sigaction(). It is perfectly reasonable to
push SIG_DFL or SIG_IGN onto the stack.
--
1.7.4.rc0
^ permalink raw reply related
* Re: clone --bare vs push
From: Ævar Arnfjörð Bjarmason @ 2011-01-03 18:41 UTC (permalink / raw)
To: Levend Sayar; +Cc: git
In-Reply-To: <AANLkTi=RNDYrRbyEJXA_c30JEVr=SYUQ01cfA3FyWpLT@mail.gmail.com>
On Mon, Jan 3, 2011 at 19:24, Levend Sayar <levendsayar@gmail.com> wrote:
> 1) When I compare X/.git directory and y.git directory there are many
> objects missing in y.git. What is the reason ?
Maybe things were repacked.
> 2) git clone --bare is too fast. My .git directory is nearly 1GB. How
> can it be copied that much fast ?
It will use hardlinks when using the same filesystem. See
--no-hardlinks in git-clone(1).
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox