* Re: Wanted - a file browser interface to git
From: Linus Torvalds @ 2005-10-19 1:07 UTC (permalink / raw)
To: John Ellson; +Cc: git
In-Reply-To: <dj45np$e88$1@sea.gmane.org>
On Tue, 18 Oct 2005, John Ellson wrote:
>
> An example is: "I know that file xxx contained algorithm yyy at some point in
> the past and now I'd like to browse back through the history of xxx to find
> the exact details."
You are aware of "git whatchanged -p xxx", right?
Yeah, it's not graphical, and I agree that it might be very cool to have a
graphical version of it. But I thought I'd mention it even so. A
surprising number of people seem to have never realized, and at least for
me personally, it's one of the most common things I do.
Linus
^ permalink raw reply
* [PATCH] revised^2: git-daemon extra paranoia, and path DWIM
From: H. Peter Anvin @ 2005-10-19 1:09 UTC (permalink / raw)
To: Git Mailing List, Linus Torvalds, Junio C Hamano
[-- Attachment #1: Type: text/plain, Size: 477 bytes --]
This patch adds some extra paranoia to the git-daemon filename test. In
particular, it now rejects pathnames containing //; it also adds a
redundant test for pathname absoluteness (belts and suspenders.)
A single / at the end of the path is still permitted, however, and the
.git and /.git append DWIM stuff is now handled in an integrated manner,
which means the resulting path will always be subjected to pathname checks.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 3056 bytes --]
diff --git a/daemon.c b/daemon.c
--- a/daemon.c
+++ b/daemon.c
@@ -80,17 +80,30 @@ static int path_ok(const char *dir)
{
const char *p = dir;
char **pp;
- int sl = 1, ndot = 0;
+ int sl, ndot;
+
+ /* The pathname here should be an absolute path. */
+ if ( *p++ != '/' )
+ return 0;
+
+ sl = 1; ndot = 0;
for (;;) {
if ( *p == '.' ) {
ndot++;
- } else if ( *p == '/' || *p == '\0' ) {
+ } else if ( *p == '\0' ) {
+ /* Reject "." and ".." at the end of the path */
if ( sl && ndot > 0 && ndot < 3 )
- return 0; /* . or .. in path */
+ return 0;
+
+ /* Otherwise OK */
+ break;
+ } else if ( *p == '/' ) {
+ /* Refuse "", "." or ".." */
+ if ( sl && ndot < 3 )
+ return 0;
sl = 1;
- if ( *p == '\0' )
- break; /* End of string and all is good */
+ ndot = 0;
} else {
sl = ndot = 0;
}
@@ -99,7 +112,7 @@ static int path_ok(const char *dir)
if ( ok_paths && *ok_paths ) {
int ok = 0;
- int dirlen = strlen(dir); /* read_packet_line can return embedded \0 */
+ int dirlen = strlen(dir);
for ( pp = ok_paths ; *pp ; pp++ ) {
int len = strlen(*pp);
@@ -118,22 +131,16 @@ static int path_ok(const char *dir)
return 1; /* Path acceptable */
}
-static int upload(char *dir, int dirlen)
+static int set_dir(const char *dir)
{
- loginfo("Request for '%s'", dir);
-
if (!path_ok(dir)) {
- logerror("Forbidden directory: %s\n", dir);
+ errno = EACCES;
return -1;
}
- if (chdir(dir) < 0) {
- logerror("Cannot chdir('%s'): %s", dir, strerror(errno));
+ if ( chdir(dir) )
return -1;
- }
-
- chdir(".git");
-
+
/*
* Security on the cheap.
*
@@ -141,10 +148,39 @@ static int upload(char *dir, int dirlen)
* a "git-daemon-export-ok" flag that says that the other side
* is ok with us doing this.
*/
- if ((!export_all_trees && access("git-daemon-export-ok", F_OK)) ||
- access("objects/", X_OK) ||
- access("HEAD", R_OK)) {
- logerror("Not a valid git-daemon-enabled repository: '%s'", dir);
+ if (!export_all_trees && access("git-daemon-export-ok", F_OK)) {
+ errno = EACCES;
+ return -1;
+ }
+
+ if (access("objects/", X_OK) || access("HEAD", R_OK)) {
+ errno = EINVAL;
+ return -1;
+ }
+
+ /* If all this passed, we're OK */
+ return 0;
+}
+
+static int upload(char *dir)
+{
+ /* Try paths in this order */
+ static const char *paths[] = { "%s", "%s/.git", "%s.git", "%s.git/.git", NULL };
+ const char **pp;
+ /* Enough for the longest path above including final null */
+ int buflen = strlen(dir)+10;
+ char *dirbuf = xmalloc(buflen);
+
+ loginfo("Request for '%s'", dir);
+
+ for ( pp = paths ; *pp ; pp++ ) {
+ snprintf(dirbuf, buflen, *pp, dir);
+ if ( !set_dir(dirbuf) )
+ break;
+ }
+
+ if ( !*pp ) {
+ logerror("Cannot set directory '%s': %s", dir, strerror(errno));
return -1;
}
@@ -170,7 +206,7 @@ static int execute(void)
line[--len] = 0;
if (!strncmp("git-upload-pack /", line, 17))
- return upload(line + 16, len - 16);
+ return upload(line+16);
logerror("Protocol error: '%s'", line);
return -1;
^ permalink raw reply
* Re: gitweb.cgi
From: H. Peter Anvin @ 2005-10-19 1:14 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Kay Sievers, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0510181753340.3369@g5.osdl.org>
Linus Torvalds wrote:
>
> On Tue, 18 Oct 2005, H. Peter Anvin wrote:
>
>>It turns out that the default CacheSize is only 256K. D'oh! Fixed.
>>
>>I also changed the CacheDefaultExpire to 600 seconds.
>
>
> Ok, that sounds like it should improve things. My quick tests didn't seem
> to show any difference, though. Do you need to re-load the apache module
> or something?
>
Yes, but I did that. It seems very strange when something hits the
cache. A cgi script can apparently be run quite a few number of times
before mod_cache sees it globally.
-hpa
^ permalink raw reply
* Re: [PATCH] git-daemon extra paranoia
From: Junio C Hamano @ 2005-10-19 1:18 UTC (permalink / raw)
To: git
In-Reply-To: <435596CB.6070401@zytor.com>
"H. Peter Anvin" <hpa@zytor.com> writes:
> Consider the whitelist/blacklist scenario I described in the previous
> email. You have:
>
> whitelist: /pub/scm
> blacklist: /pub/scm/foo/bar.git
>
> If you can bypass the blacklist by using the pathname /pub/scm/foo/bar,
> that's bad.
I like the simplicity of the check Linus suggested. Given
/pub/scm/fora/../foo/bar/, you would end up chdir() to
/pub/scm/foo/bar.git and getcwd() would hit the blacklist
entry. Which almost means that you do not even need path_ok()
;-).
^ permalink raw reply
* Re: Wanted - a file browser interface to git
From: John Ellson @ 2005-10-19 1:23 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0510181806250.3369@g5.osdl.org>
Linus Torvalds wrote:
> On Tue, 18 Oct 2005, John Ellson wrote:
>
>> An example is: "I know that file xxx contained algorithm yyy at some point in
>> the past and now I'd like to browse back through the history of xxx to find
>> the exact details."
>>
>
> You are aware of "git whatchanged -p xxx", right?
>
> Yeah, it's not graphical, and I agree that it might be very cool to have a
> graphical version of it. But I thought I'd mention it even so. A
> surprising number of people seem to have never realized, and at least for
> me personally, it's one of the most common things I do.
>
> Linus
>
Linus,
I wasn't aware of it, no. Looks very useful. Thanks.
I see that you can take the tree id from the diff-tree lines and
then produce the state of the file at that time with "cg-admin-cat -r
<id> xxx"
Is that how you would do it?
Are there any plans for cogito to support it?
John
^ permalink raw reply
* Re: gitweb.cgi
From: Kay Sievers @ 2005-10-19 1:23 UTC (permalink / raw)
To: Linus Torvalds; +Cc: H. Peter Anvin, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0510181753340.3369@g5.osdl.org>
On Tue, Oct 18, 2005 at 06:02:29PM -0700, Linus Torvalds wrote:
>
>
> On Tue, 18 Oct 2005, H. Peter Anvin wrote:
> >
> > It turns out that the default CacheSize is only 256K. D'oh! Fixed.
> >
> > I also changed the CacheDefaultExpire to 600 seconds.
>
> Ok, that sounds like it should improve things. My quick tests didn't seem
> to show any difference, though. Do you need to re-load the apache module
> or something?
>
> > The only thing the front page really should need is to know when the last
> > change to the tree was, which presumably means looking at each head of each
> > tree and follow the chain until there is a datable object.
>
> Yeah. I tried to follow gitweb.cgi, but I'm neither http- nor
> perl-literate, so I'm not sure I caught everything.
>
> But it does seem to basically end up doing a "git_read_commit()" for each
> project, and that in turn was doing the "git-rev-list --max-count=1" thing
> that I just sent out a suggested improvement for.
>
> It effectively removes two or more copies of
>
> stat64("/objects/xy/zzy", {...})
> fd = open("objects/xy/zzy", O_RDONLY|O_NOATIME)
> addr = mmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0)
> close(fd)
> munmap(addr, size)
>
> which really should be very cheap operations, but hey, if the disk head is
> somewhere else (and busy) and it's not cached, it can be quite expensive.
> Especially since we don't end up usign the result.
>
> I'm sure there's room for improvement inside gitweb itself too, but maybe
> the git-rev-list optimization will help.
There definitely is! But I tried a single "stat() all HEAD files" with a
simple script and it took more than 3 seconds for the 80 trees. Then I
gave up "optimizing" and was sure we want to have a single-file cached front
page instead. :)
Kay
^ permalink raw reply
* Re: gitweb.cgi
From: Kay Sievers @ 2005-10-19 1:33 UTC (permalink / raw)
To: H. Peter Anvin; +Cc: Git Mailing List
In-Reply-To: <43552FC2.3000000@zytor.com>
On Tue, Oct 18, 2005 at 10:24:18AM -0700, H. Peter Anvin wrote:
> Kay Sievers wrote:
> >
> >>Most of the hits we get are either the
> >>gitweb front page or the gitweb rss feeds, and it's eating I/O bandwidth
> >>like crazy.
> >
> >I tested some stuff on these boxes and 30 stat() calls alone take app. 2
> >seconds
> >on these boxes cause of I/O load ... :)
> >
>
> Welcome to my hell :)
Yeah, I get an idea now :)
> I set up mod_cache (which I didn't know about, silly me) and so far it
> seems to work and has produced a tremendous decrease in load and
> improvement in response time.
Great! Hope that will work.
> I do, have, however, a request. There
> are some gitweb pages which are more likely to change than others; in
> particular, some gitweb pages will *never* change (because they directly
> reflect immutable git data.)
Yes, makes sense.
> If gitweb could produce Last-Modified and Expires headers where
> appropriate, it should improve caching performance.
I've added the Expires: header to the commit and commitdiff pages with
one whole day ahead. Let's see if that will help...
Kay
^ permalink raw reply
* Re: git-diff-tree rename detection for single file
From: Junio C Hamano @ 2005-10-19 2:45 UTC (permalink / raw)
To: David Ho; +Cc: git, Linus Torvalds
In-Reply-To: <7vu0fepn0x.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano <junkio@cox.net> writes:
> David Ho <davidkwho@gmail.com> writes:
>
>> I have a small suggestion to make the diff of a renamed file a bit
>> more meaningful. I have a file that is renamed-edited and commited.
>> git-diff-tree -M -p <commit> shows one result and git-diff-tree -M -p
>> <commit> <filename> shows another. If they both show a rename
>> occurred then I think the single file git-diff-tree will be more
>> useful.
>
> Sorry, this was vetoed by Linus long time ago. The <filename>
> restricts the paths being passed to the diff machinery upfront,
> so once you say <filename>, the rename detection will see only
> that path and nothing else to compare and guess which other file
> that file in question is a copy of.
Having said that, I think we *could* introduce a new flag to
git-diff-* brothers, --late-pathspec, that makes them apply the
paths restriction on the output side instead. For obvious
reasons, using this flag would not make any sense unless you are
using one of -M, -C, or --pickaxe-all.
A related thing I have long longed for is a rename following
"git-diff-tree --stdin".
git-rev-list HEAD | git-diff-tree --stdin -M git-commit.sh
This command line, as everybody hopefully knows, is how "git
whatchanged" is implemented internally. If git-diff-tree were
taught to follow the rename history, when it hits the boundary
that git-commit-script was renamed to git-commit.sh, it could
start acting as if the pathspec given were git-commit-script
from that point. To see that rename it needs --late-pathspec;
the current pathspec filters the input so the above command line
would not even care what git-commit-script looked like when the
rename happend. It would just tell git-commit.sh appeared from
nowhere.
If implemented naively, this rename-following would have funny
interactions when it hits a merge commit, so it may probably be
harder than it sounds, but this would be a good way to do
annotate as well.
^ permalink raw reply
* Re: Wanted - a file browser interface to git
From: Linus Torvalds @ 2005-10-19 3:03 UTC (permalink / raw)
To: John Ellson; +Cc: git
In-Reply-To: <4355A00B.4000806@research.att.com>
On Tue, 18 Oct 2005, John Ellson wrote:
> Linus Torvalds wrote:
> >
> > You are aware of "git whatchanged -p xxx", right?
>
> I wasn't aware of it, no. Looks very useful. Thanks.
>
> I see that you can take the tree id from the diff-tree lines and
> then produce the state of the file at that time with "cg-admin-cat -r <id>
> xxx"
> Is that how you would do it?
Well, I'd do it with the git commands: once you see the diff, you should
know the SHA1's of the source and destination, and then you can just do
git-cat-file blob [sha1]
to get the before (or after) state.
The way I'd get the SHA1 is either (now with the extended diff format) in
the short form from the diff itself (the "index" line), or by just
separately doing a
git-diff-tree -r [sha-of-commit]
to see the "raw" diff format.
All the diff things can take a pathname limiter, the same way
"git-whatchanged" does, so if you are only interested in one file, just
name the file:
git-diff-tree -r [sha-of-commit] [filename]
And just to make clear how powerful this is: "filename" doesn't have to be
a single file. It can be a set of files and/or directories, so if you want
to track multiple things at the same time, just do multiple filenames.
What I do a lot is to check what has changed in some particular subsystem,
ie somebody says that something broke in SCSI, and then I do
git-whatchanged -p drivers/scsi/ include/scsi/
and it will show any changes to anything under either of those
directories.
"git-whatchanged" really is very powerful. The silly thing is that it
really boils down to just a single line script (well, with various
argument handling etc it's actually five lines, but the "core" is really
just a single pipeline of "git-rev-list | git-diff-tree --stdin".
> Are there any plans for cogito to support it?
Well, cogito could certainly just do a "cg-whatchanged", but it's really
the same thing. Since cogito depends on git anyway, cogito users could
just use the git-whatchanged functionality.
Or to make it more seamless, just do
alias cg-whatchanged=git-whatchanged
or something like that ;^)
Linus
^ permalink raw reply
* Re: git-diff-tree rename detection for single file
From: Linus Torvalds @ 2005-10-19 3:12 UTC (permalink / raw)
To: Junio C Hamano; +Cc: David Ho, git
In-Reply-To: <7virvujkcw.fsf@assigned-by-dhcp.cox.net>
On Tue, 18 Oct 2005, Junio C Hamano wrote:
>
> Having said that, I think we *could* introduce a new flag to
> git-diff-* brothers, --late-pathspec
Gaah. Why? It's really not possible to do it efficiently inside
git-diff-xyz, so whatever implementation would basically boil down to
something you can already do with some trivial scripting, basically
boiling down to:
git-diff-tree -r -M | grep pathnamelist | git-diff-helper
Now, several reasons why it's much better to do this kind of
"--late-pathspec" at a higher level (instead of inside the git-diff-xyz
family):
(a) git-diff-xyz is already some of the more complex core parts. It's not
likely a good idea to make them any more complex, unless there's some
very fundamental reason for it.
(b) without pathname limits, git-diff-tree is very slow. Well, it's
actually very fast compared to something braindead like CVS, but if
you want to track a single file over a thousand releases, it's MUCH
MUCH faster to do the pathname limit at the beginning. Otherwise
you'll spend all your time reading and comparing big trees with tens
of thousands of entries.
(c) with a higher-level thing, what you can do is have a TWO-phase thing:
use the fast pathname limiter in git-diff-tree to figure out when
that file changes in history, and then _only_ for those commits do
you go back and then do the much more expensive "git-diff-tree -r -M"
followed by the pathname-limiting post-processing.
See what I'm saying? You really can do the post-processing outside of
git-diff-tree, and you will in fact be much better off if you do so.
The performance impact of pruning the pathnames _before_ diffing them was
absolutely staggering. You couldn't reasonably do a "git-whatchanged -p"
on the kernel for a single file if you didn't do it the way we do it now.
Linus
^ permalink raw reply
* [PATCH] cg-history FILE [NTH_PARENT] - was: Re: Wanted - a file browser interface to git
From: John Ellson @ 2005-10-19 3:15 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0510181806250.3369@g5.osdl.org>
Linus Torvalds wrote:
>
> On Tue, 18 Oct 2005, John Ellson wrote:
>> An example is: "I know that file xxx contained algorithm yyy at some point in
>> the past and now I'd like to browse back through the history of xxx to find
>> the exact details."
>
> You are aware of "git whatchanged -p xxx", right?
>
> Yeah, it's not graphical, and I agree that it might be very cool to have a
> graphical version of it. But I thought I'd mention it even so. A
> surprising number of people seem to have never realized, and at least for
> me personally, it's one of the most common things I do.
>
> Linus
OK. Here is a not-very-smart cogito command to display the history of a file,
or the state of the nth parent of the file in its history.
Feedback and or complete rewrites are requested ;-)
John
produce the history of a file, or its state at its nth_parent
---
commit 8478ad1164e37e9cca039a3f9552d2a98f7bead6
tree 40b39d5f9af573a7815f073cada03c7903bfc6fa
parent 5d74e4859afc81a4658133d5a83809ac814dbf34
author John Ellson <ellson@ontap.ellson.com> Tue, 18 Oct 2005 23:13:44 -0400
committer John Ellson <ellson@ontap.ellson.com> Tue, 18 Oct 2005 23:13:44 -0400
cg-history | 36 ++++++++++++++++++++++++++++++++++++
1 files changed, 36 insertions(+), 0 deletions(-)
diff --git a/cg-history b/cg-history
new file mode 100755
index 0000000..ba86f71
--- /dev/null
+++ b/cg-history
@@ -0,0 +1,36 @@
+#!/usr/bin/env bash
+#
+# Display the change history of a file.
+# Copyright (c) John Ellson, 2005
+#
+# The change history of a file is displayed on stdout, or
+# if an integer is provided for NTH_PARENT, then the complete
+# state of the file at that step in its history is sent to stdout.
+#
+
+USAGE="cg-history FILE [NTH_PARENT]"
+
+. ${COGITO_LIB}cg-Xlib || exit 1
+
+[ "$ARGS" ] || usage
+
+if [ "${ARGS[1]}" = "" ]; then
+ git-whatchanged -p "${ARGS[0]}"
+else
+ i=0
+ git-whatchanged -p "${ARGS[0]}" |
+ while read -r cmd sha rest
+ do
+ case "$cmd" in
+ diff-tree)
+ i=`expr $i + 1`
+ if [ $i = ${ARGS[1]} ] ; then
+ cg-admin-cat -r "$sha" "${ARGS[0]}"
+ exit 0
+ fi
+ ;;
+ *)
+ ;;
+ esac
+ done
+fi
^ permalink raw reply related
* Re: [PATCH] cg-history FILE [NTH_PARENT]
From: Linus Torvalds @ 2005-10-19 4:01 UTC (permalink / raw)
To: John Ellson; +Cc: git
In-Reply-To: <4355BA43.5030509@research.att.com>
[ Ok, time for some serious power-git usage ]
On Tue, 18 Oct 2005, John Ellson wrote:
>
> produce the history of a file, or its state at its nth_parent
>
> + git-whatchanged -p "${ARGS[0]}" |
Actually, you're much better of _not_ using "-p" to generate a patch.
In fact, you don't even want the pretty format that "git-whatchanged"
does, you really want the raw output.
Try this instead:
git-rev-list HEAD | git-diff-tree --stdin -s -r "$ARG"
which will give _just_ a list of the commits that change the file "$ARG".
Here, the "--stdin" to git-diff-tree means that it should take its
revision input from stdin (ie obvously the list of commits generated by
git-rev-list), and the "-s" stands for "silent", ie git-diff-tree won't
actually output the diff itself.
And the "-r" means that it should check the trees "recursively", which is
needed since we want the diff-tree to traverse down the tree rather than
just look at the top-level ("-p" to generate patches enables recursive by
default since patches don't make sense on raw trees, but without the -p
you need to do it explicitly).
And since we didn't ask for the header, the only thing you get is the list
of commits that changed the file describled by the argument.
So now, you can just pick the n'th such commit, and do something like this
#
# Get the "${ARGS[1]}"th commit that changes file "${ARGS[0]}"
#
rev=$(git-rev-list HEAD |
git-diff-tree --stdin -s -r "${ARGS[0]}" |
head -n "${ARGS[1]}" |
tail -1)
#
# Pick up the file from that tree
#
filerev=$(git-ls-tree -r "$rev" "${ARGS[0]}" |
cut -f1 |
cut -d' ' -f3)
#
# And show it
#
git-cat-file blob $filerev
and you're done (untested, but you should get the idea).
Now, the interesting part about is that you can feed the output from
git-diff-tree _back_ to git-diff-tree, so you can do some really fancy
footwork like:
git-rev-list rev1..rev2 |
git-diff-tree --stdin -s -r "$ARG" |
git-diff-tree --stdin -M --pretty -p
and what this will do is:
- generate a list of all commits between rev1 and rev2
- filter out just the commits that change the file "$ARG", and pass those
on.
- for those commits, show the _whole_ diff, with rename detection and
with pretty-printed commit comment headers
In other words, you can basically look at all the full commits that
changed one file (or a set of files). Efficiently.
This is kind of like "git-whatchanged", but it shows the full context of
what changed. Of course, the second git-diff-tree can be used to limit the
context to something else, ie you could do a variation of the above,
something like
git-rev-list v2.6.12.. |
git-diff-tree --stdin -s -r drivers/usb/ |
git-diff-tree --stdin -M --pretty -p drivers/ include/
which will show any commit that changed the drivers/usb/ directory after
v2.6.12, but then limit the output of those commits to the drivers/ and
include/ subdirectories (so anything that was changed in that same commit
in a filesystem would _not_ be shown, for example, but if there were
changes to drivers/scsi/ at the same time, they _would_ show).
Or, if you want to go really wild, do something like this on the kernel
git tree:
git-rev-list v2.6.12.. |
git-diff-tree --stdin -s -r drivers/usb/ |
git-diff-tree --stdin -s -r drivers/scsi/ |
git-diff-tree --stdin -M --pretty -p drivers/scsi/ drivers/usb/ |
less -S
which says to print out only those commits that change something _both_ in
drivers/usb/ _and_ in drivers/scsi/ at the same time, and then show only
those parts of the changes.
Try it out. It really does work, and is extremely powerful. It's even
pretty efficient (make sure your tree is packed first, though ;). I can do
the above in about three seconds for the current kernel history on my
machine. That's 3 _seconds_ to go through what right now is 8005 commits:
git-rev-list v2.6.12.. | wc -l
and the reason is exactly that the filename-based parsing is very good at
efficiently pruning out all the tree information that isn't needed.
Very cool.
However, the "normal" situation is just the standard "git-whatchanged",
which is much easier to use than something more complex like the above.
The core git commands are really designed to be scriptable, but "real
life" seldom wants the complexity of quite that much flexibility.
Linus
^ permalink raw reply
* Re: git-diff-tree rename detection for single file
From: Junio C Hamano @ 2005-10-19 5:20 UTC (permalink / raw)
To: Linus Torvalds; +Cc: David Ho, git
In-Reply-To: <Pine.LNX.4.64.0510182004100.3369@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> See what I'm saying?
> ...
> The performance impact of pruning the pathnames _before_ diffing them was
> absolutely staggering. You couldn't reasonably do a "git-whatchanged -p"
Yes, I understood and agreed to that logic on May 27th. That's
why the message you are responding said we could add a new flag,
to give the choice to the user to accept full-tree scan.
The "diff-tree --stdin that follows rename history" example
might want to have some change in either the core side of diff,
or in the rev-list. It may help to have both. I still haven't
thought through the issues.
BTW, I really liked your example that piped multiple diff-trees
together. That is a neat trick.
^ permalink raw reply
* Re: [RFC] Timeouts on HTTP requests
From: Junio C Hamano @ 2005-10-19 6:02 UTC (permalink / raw)
To: Nick Hengeveld; +Cc: git
In-Reply-To: <20051018235104.GO5509@reactrix.com>
Nick Hengeveld <nickh@reactrix.com> writes:
> Our QA department today checked what would happen if the network connection
> went away completely in the middle of an HTTP transfer. It looks as though
> the answer is that git-http-fetch sits there forever waiting for CURL to
> return something.
Ouch.
> I'm thinking of taking advantage of CURL's capability of aborting a request
> if the transfer rate drops below a threshold for a specified length of time
> using a new pair of environment variables and/or config file settings:
>
> GIT_HTTP_LOW_SPEED_LIMIT/http.lowspeedlimit
> GIT_HTTP_LOW_SPEED_TIME/http.lowspeedtime
>
> Does this make sense, and if so should there be defaults if nothing is
> specified?
I suspect these would be quite different between DSL and
localnet, so I doubt if there is a reasonable default value to
quick give-up.
On the other hand, having _no_ activity for say 30 seconds would
indicate a dead link on either modem or localnet.
BTW, I've been thinking about giving defaults by shipping
templates/config (i.e. no compile-time defaults). One trick I
found cute is to have "clone.keeppack = 1" in the templates to
be applied for any newly built repository, especially now
kernel.org has git-daemon enabled.
^ permalink raw reply
* Pushing a single tag (ref + object)?
From: Martin Langhoff @ 2005-10-19 6:05 UTC (permalink / raw)
To: Git Mailing List
In-Reply-To: <46a038f90510131929m3dac4cc5y6071550e9e9c71ad@mail.gmail.com>
While we are using a repo which holds all our branches
(dev/test/prod), locally we have a group of developers that checkout
one repo-per-branch, working on it with a
cg-clone/cg-update/cg-commit/cg-push workcycle. So far it's working
great.
Now, I am at a loss on how to push a _tag_ object+ref to the repo,
without doing a git-push --all, which I naturally don't want to do. I
managed to push the object itself, doing
git-push repository tagrefname
But that ddn't create the ref on the repo. So I had to do
scp .git/refs/tags/refname repostory/refs/tags/
I'm feeling a tad lost here. Surely there's a way? Or should I be
crafting a patch against git-push-script? Problem is, git-push script
doesn't do any parsing of the params. Grmbl.
martin
^ permalink raw reply
* cg-clone, tag objects and cg-push/git-push don't play nice
From: Martin Langhoff @ 2005-10-19 6:38 UTC (permalink / raw)
To: Git Mailing List
I am seeing very strange issues with cloning one head with cg-clone
from a repo that holds many heads and tags. The session looks like
this:
cg-clone git+ssh://locke.catalyst.net.nz/var/git/moodle-test.git#mdl-topnz
testdir
cd testdir
echo "sillychange" >> version.php
cg-commit -m "testing" version.php
cg-push
updating 'refs/heads/mdl-topnz' using 'refs/heads/master'
from 06ca8b3c4826d60e8cf5850c6474e66f816ba5c7
to 482d4b88aa482dfea7f7549470902049a050020a
fatal: bad object 1b0efdd8f31e5b8c7d32c85d11492db122b62a0a
Packing 0 objects
Unpacking 0 objects
error: unpack should have generated
482d4b88aa482dfea7f7549470902049a050020a, but I can't find it!
I get the same error if I try git-push manually. Apparently, there's a
set of tag objects pointing nowhere.
git-fsck-objects --full --strict 482d4b88aa482dfea7f7549470902049a050020a
bad sha1 file: .git/objects/44/7472d455667e426a96acf116e27c2f1efe674e
3ae8dc25c642d8c59f3c44c5ba48faa6a0e7a2ee
2ddfec0dfd0cffd4892af9aaf48ee29c40c7ada3
missing commit 1b0efdd8f31e5b8c7d32c85d11492db122b62a0a
dangling commit 3e9472b3ef980e667d00d5374ccfa741cfb93fbc
broken link from tag 447472d455667e426a96acf116e27c2f1efe674e
to commit 1b0efdd8f31e5b8c7d32c85d11492db122b62a0a
dangling tag 447472d455667e426a96acf116e27c2f1efe674e
missing commit 5a0bdfb0f7af34d002c6ced40f96f977fd6471e2
dangling commit 715b3b0a7c94cb31760e87a320f0f612b962e3c3
missing commit 961bf469c6b309b8fc10d064368fd14480d231be
broken link from tag 984bed44df42422839705aa4bd6d8f00086b1307
to commit 961bf469c6b309b8fc10d064368fd14480d231be
dangling tag 984bed44df42422839705aa4bd6d8f00086b1307
broken link from tag ce3602d6a60c648402048b0970657a2961e1ed54
to commit dd8c6c172fbab9905dd306c17d83b8d21ea5bfda
dangling tag ce3602d6a60c648402048b0970657a2961e1ed54
missing commit dd8c6c172fbab9905dd306c17d83b8d21ea5bfda
broken link from tag ddb658d070ba6688541e5b91e963f629ddd63b6a
to commit 5a0bdfb0f7af34d002c6ced40f96f977fd6471e2
dangling tag ddb658d070ba6688541e5b91e963f629ddd63b6a
Now, if I go to the repository, it _has_ all the tags with their
matching commits. So the problem is in the cloned repo. It looks like:
+ cg-clone (cg-fetch actually) has brought in all the tag refs and
objects, regardless of whether they are relevant to this branch, but
did not fetch the commits, trees, or any other related bits and
pieces.
+ git-push is trying to walk all the refs it knows about when it does
the "what do I have that the repo doesn't" part, and it breaks on
those incomplete tag fetches.
Removing .git/refs/tags/* didn't help. If I actually rm all the
problematic tag objects from the object repository, I can then push
correctly. Lucky it wasn't packed.
I am going to write a quick'n'dirty script to fix our repos when this
happens, and I'll try my hand at a patch to cg-fetch. Can git-push be
taught to be smarter in these cases?
regards,
martin
^ permalink raw reply
* Re: cg-clone, tag objects and cg-push/git-push don't play nice
From: Junio C Hamano @ 2005-10-19 7:16 UTC (permalink / raw)
To: Martin Langhoff; +Cc: git
In-Reply-To: <46a038f90510182338k6d3d52fbyc2057e9b775d5b14@mail.gmail.com>
Martin Langhoff <martin.langhoff@gmail.com> writes:
> + git-push is trying to walk all the refs it knows about when it does
> the "what do I have that the repo doesn't" part, and it breaks on
> those incomplete tag fetches.
That is expected.
> Removing .git/refs/tags/* didn't help. If I actually rm all the
> problematic tag objects from the object repository, I can then push
> correctly. Lucky it wasn't packed.
>
> I am going to write a quick'n'dirty script to fix our repos when this
> happens, and I'll try my hand at a patch to cg-fetch. Can git-push be
> taught to be smarter in these cases?
Although I do not follow Cogito development closely, I seem to
recall that it fetched tags without making them complete at some
point in the past; I hope it is now fixed but I am not sure.
I cannot think offhand of a way how git-push could help to cope
with a repository broken that way. However, we could probably
have a tool that tangles from each ref, find incomplete ones and
remove them from .git/refs. After running that maybe pull from
a know good copy would fix the broken repository.
I do not understand why removing .git/refs/tags/* did not help,
and that is the biggest thing that disturbs me in this whole
problem report. We (meaning git-* transfer, not cg-* transfer
whose correctness I cannot vouch for) _should_ be relying only
on refs not object existence. Maybe there are some other files
under .git/refs/ directory that had copies of them?
^ permalink raw reply
* Problem getting older version
From: Nico -telmich- Schottelius @ 2005-10-19 8:00 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 1321 bytes --]
Hello!
The following situation:
- The last commit was a merge, mhich broke some files
- We want three files from the commit before
Now I was told to do the following:
1. get the sha1hash from the commit before (cg-log did that)
2. get the sha1hash from the file in that tree:
[9:27] srsyg01:walderlift% git-ls-tree 35ff687efc1b19b4db918e5af859894a9dc916e4 Code/lw1/Client/MainForm.xfm
100644 blob 605958b1435f6bdbd5cc502ae3a4c1a281d01f0a Code/lw1/Client/MainForm.xfm
3. cat the file with git-cat-file
[9:36] srsyg01:walderlift% git-cat-file blob 605958b1435f6bdbd5cc502ae3a4c1a281d01f0a | less
4. Now overwrite it
[9:38] srsyg01:walderlift% git-cat-file blob 605958b1435f6bdbd5cc502ae3a4c1a281d01f0a > Code/lw1/Client/MainForm.xfm
5. Goto 2 and repeat two times
Is this really the standard way to recover a file? As a developer / end user I would expect that:
cg-recover <filename> <commit id> and -f for overwriting the file if it exists
Did I overlook something or is it currently really this complicated? And it is
very error prone, if I have to overwrite the files using '>'-redirection (perhaps I mistype the
filename).
Nico
--
Latest project: cconfig (http://nico.schotteli.us/papers/linux/cconfig/)
Open Source nutures open minds and free, creative developers.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]
^ permalink raw reply
* Re: cg-clone, tag objects and cg-push/git-push don't play nice
From: Martin Langhoff @ 2005-10-19 8:10 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vzmp6dlii.fsf@assigned-by-dhcp.cox.net>
On 10/19/05, Junio C Hamano <junkio@cox.net> wrote:
> Martin Langhoff <martin.langhoff@gmail.com> writes:
>
> > + git-push is trying to walk all the refs it knows about when it does
> > the "what do I have that the repo doesn't" part, and it breaks on
> > those incomplete tag fetches.
>
> That is expected.
Hmmm. I was under the impression that if I call git-push naming a
particular head, it could restrict itself to the stuff needed for that
head only. Just to clarify, I'm running
git-push locke.catalyst.net.nz:/var/git/moodle-test.git master:mdl-topnz-prod
> Although I do not follow Cogito development closely, I seem to
> recall that it fetched tags without making them complete at some
> point in the past; I hope it is now fixed but I am not sure.
It isn't fixed, but I'm trying to address that one :-p
> I do not understand why removing .git/refs/tags/* did not help,
> and that is the biggest thing that disturbs me in this whole
> problem report
I can't understand that either, but I manually removed all the
refs/tags, and the only heads I have are origin and master. git-push
won't let me do it until the exact point where I have removed the
object from the repo. And that's only possible on unpacked repos.
> Maybe there are some other files
> under .git/refs/ directory that had copies of them?
No. I run a test again, to make sure. Removing .git/refs/tags is not
enough, and the only refs available are origin and master.
I've come up with this awful script to resolve it, while I try to fix cg-fetch:
#usr/bin/perl -w
use strict;
my @refs = `ls .git/refs/tags`;
foreach my $ref (@refs) {
chomp $ref;
print "testing for a commit linked from $ref\n";
my $commit = `git-rev-parse --verify "$ref"^{commit} 2>/dev/null`;
if ($?) {
# this one didn't even have a tag ref pointing to a commit!
#unlink ".git/refs/tags/$ref"
# or die "cannot remove .git/refs/tags/$ref";
#next;
}
# thest that we actually have the commit object...
chomp $commit;
my $file = `git-cat-file commit $commit 2>/dev/null`;
if ($?) {
# could not find the commit object, we better get rid of the
# tagref and tagobj
my $tagsha = `git-rev-parse --verify "$ref" 2>/dev/null`;
chomp $tagsha;
if ($tagsha) {
# doublecheck it is a tag
my $type = `git-cat-file -t $tagsha 2>/dev/null`;
chomp $type;
if ($type eq 'tag') {
my $fileobj = ".git/objects/" . substr($tagsha,0,2) . '/' .
substr($tagsha,2);
print " removing $fileobj for ref $ref tagsha $tagsha \n";
`rm -f $fileobj`;
unlink ".git/refs/tags/$ref";
}
}
}
}
cheers,
m
^ permalink raw reply
* Gitweb feature requests, Bug?
From: Nico -telmich- Schottelius @ 2005-10-19 8:12 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 726 bytes --]
Hello!
There are some things we see here, which are problematic:
- Clicking on history for a file does not return the last commit, but every
commit _before_ the last commit. Is this wanted? We were searching for that
specific commit and wondered why it is not in the history
- when selecting the blob of a file, we miss
o the sha1sum
o a link to the history
- Still there's the probelm having a '+' in the filename
Could you change gitweb so it does those things or if you've no time would you accept
a patch for it that does that?
Greetings,
Nico
--
Latest project: cconfig (http://nico.schotteli.us/papers/linux/cconfig/)
Open Source nutures open minds and free, creative developers.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]
^ permalink raw reply
* Problems with .gitignore
From: Nico -telmich- Schottelius @ 2005-10-19 8:16 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 2607 bytes --]
It looks like cg-status does not really use .gitignore:
[10:14] srsyg01:walderlift% cat .gitignore
*.~*
*.aux
Design/Programmieredoku/kapitel.lof
Design/Programmieredoku/kapitel.log
Design/Programmieredoku/kapitel.lot
Design/Programmieredoku/kapitel.out
Design/Programmieredoku/kapitel.pdf
Design/Programmieredoku/kapitel.toc
*.conf
*.dcp
*.dcu
*.ddp
*.dpu
*.kof
*.res
*.so
Code/Components/Utilities/lw1tools
Code/lw1/Client/lw1.log
Code/lw1/Client/lw1client
Code/lw1/lw1SyTestsystem
[10:15] srsyg01:walderlift% cg-status| grep -e '.so$' -e '.dcu$' -e '.dcp$'
? Code/Components/ColoredDBComponents/ColDB.dcp
? Code/Components/ColoredDBComponents/bplColDB.~so
? Code/Components/Planner/QPlanPkgK3.dcp
? Code/Components/Planner/bplQPlanPkgK3.so
? Code/Components/Utilities/LW1Tools.dcp
? Code/Components/Utilities/Utilities.dcp
? Code/Components/Utilities/bplLW1Tools.so
? Code/Components/Utilities/bplUtilities.so
? Code/Components/rpman21d/bplrppack.~so
? Code/Components/rpman21d/bplrppackv.~so
? Code/Components/rpman21d/rppack.dcp
? Code/Components/rpman21d/rppackv.dcp
? Code/Components/zeosdbo/packages/kylix3/bplZComponent.~so
? Code/Components/zeosdbo/packages/kylix3/bplZCore.~so
? Code/Components/zeosdbo/packages/kylix3/bplZDbc.~so
? Code/Components/zeosdbo/packages/kylix3/bplZParseSql.~so
? Code/Components/zeosdbo/packages/kylix3/bplZPlain.~so
Why does that happen?
Nico
--
Latest project: cconfig (http://nico.schotteli.us/papers/linux/cconfig/)
Open Source nutures open minds and free, creative developers.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]
^ permalink raw reply
* Re: Problem getting older version
From: Junio C Hamano @ 2005-10-19 8:26 UTC (permalink / raw)
To: Nico -telmich- Schottelius; +Cc: git
In-Reply-To: <20051019080046.GI22986@schottelius.org>
Nico -telmich- Schottelius <nico-linux-git@schottelius.org> writes:
> The following situation:
>
> - The last commit was a merge, mhich broke some files
> - We want three files from the commit before
With only git-core tool, with the tip of the master branch,
would be:
$ git pull somewhere ;# this caused the mismerge
$ git checkout HEAD^ foo.c bar.c baz.c
This assumes that HEAD is a merge and HEAD^ (= HEAD^1) is the tip
of your branch before that merge (HEAD^2 would be what you
pulled from "somewhere").
The latest "git checkout", when given extra paths parameters,
does not switch branches. Instead it pulls out named files from
the given version into your index, and checks them out.
> Now I was told to do the following:
These 5 steps look correct; after that, you probably would want
to run git-update-index on those three paths.
^ permalink raw reply
* Re: git-daemon enabled on kernel.org
From: Erik Mouw @ 2005-10-19 8:35 UTC (permalink / raw)
To: H. Peter Anvin; +Cc: Git Mailing List
In-Reply-To: <43554D4F.7040403@zytor.com>
On Tue, Oct 18, 2005 at 12:30:23PM -0700, H. Peter Anvin wrote:
> After getting gitweb behind mod_cache, the load on kernel.org has gotten
> down into the tolerable range, so I have enabled git-daemon in an
> attempt to fix that :)
>
> The URL, obviously, is git://git.kernel.org/pub/scm/...
>
> (or, to specify a specific server, git1.kernel.org or git2.kernel.org.)
How do I tell git to change the default repository to pull from?
Erik
--
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
^ permalink raw reply
* Re: git-daemon enabled on kernel.org
From: Nico -telmich- Schottelius @ 2005-10-19 8:40 UTC (permalink / raw)
To: Erik Mouw; +Cc: Git Mailing List
In-Reply-To: <20051019083542.GA31526@harddisk-recovery.com>
[-- Attachment #1: Type: text/plain, Size: 833 bytes --]
Erik Mouw [Wed, Oct 19, 2005 at 10:35:42AM +0200]:
> On Tue, Oct 18, 2005 at 12:30:23PM -0700, H. Peter Anvin wrote:
> > After getting gitweb behind mod_cache, the load on kernel.org has gotten
> > down into the tolerable range, so I have enabled git-daemon in an
> > attempt to fix that :)
> >
> > The URL, obviously, is git://git.kernel.org/pub/scm/...
> >
> > (or, to specify a specific server, git1.kernel.org or git2.kernel.org.)
>
> How do I tell git to change the default repository to pull from?
Do you mean cg-branch-add perhaps? Afaik there is no real 'default' repository, but
the branches you specified. So adding a new branch will fix what you want.
Nico
--
Latest project: cconfig (http://nico.schotteli.us/papers/linux/cconfig/)
Open Source nutures open minds and free, creative developers.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]
^ permalink raw reply
* Re: cg-clone, tag objects and cg-push/git-push don't play nice
From: Junio C Hamano @ 2005-10-19 8:52 UTC (permalink / raw)
To: Martin Langhoff; +Cc: git
In-Reply-To: <46a038f90510190110g53c90c5t419ad6065292269e@mail.gmail.com>
Martin Langhoff <martin.langhoff@gmail.com> writes:
>> That is expected.
>
> Hmmm. I was under the impression that if I call git-push naming a
> particular head, it could restrict itself to the stuff needed for that
> head only. Just to clarify, I'm running
>
> git-push locke.catalyst.net.nz:/var/git/moodle-test.git master:mdl-topnz-prod
Yeah. But the problem is that the repository on the other end
claims that it has everything reachable from those incomplete
tags. Hearing that, git-push (the real name of it is send-pack)
decides not to send things that are already reachable from the
refs the other end claims to have. So if an incomplete tag
refers to a commit that contains a blob that the remote actually
does not have, and if that blob is part of the head you are
pushing, send-pack would not (and should not) send that blob to
the remote.
>> I do not understand why removing .git/refs/tags/* did not help,
>> and that is the biggest thing that disturbs me in this whole
>> problem report
>
> I can't understand that either, but I manually removed all the
> refs/tags, and the only heads I have are origin and master. git-push
> won't let me do it until the exact point where I have removed the
> object from the repo. And that's only possible on unpacked repos.
Hmph. It worries me even more.
This error message:
error: unpack should have generated
482d4b88aa482dfea7f7549470902049a050020a, but I can't find it!
comes from receive-pack that runs on the other repo (i.e. the
one with incomplete tags you just removed), so it means
send-pack decided it does not need to send that object --
meaning the other end claimed it already has it. What to send
and what need not to be sent is determined solely based on what
send-pack hears from receive-pack in the initial handshake,
which is in receive-pack.c::write_head_info(). It scans
everything under ".git/refs" directory (not just .git/refs/heads
or .git/refs/tags; if you had ".git/refs/FOOBAR", it will cause
the remote end to claim it has everything reachable from it --
the only exceptions are things that starts with a dot '.', which
is probably why cg-fetch places a temporary heads in
refs/*/.$name-fetching) and sends them -- it does not look at
the object directory and magically claim it has something that
is not recorded in its .git/refs/ directory.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).