* Re: Some advanced index playing
From: Junio C Hamano @ 2006-12-03 19:36 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git, Alan Chandler
In-Reply-To: <Pine.LNX.4.64.0612031008360.3476@woody.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> I think that is actually a misfeature.
>
> This _should_ just work. It's the easy and logical way to do it, and it's
> the one that matches all the other behaviours of "git commit" these days.
>
> The reason for the safety valve is actually not really "safety" any more,
> it's purely "historical behaviour". Ie the sanity check is not there
> because you would be doing anything unsafe, but simply because the
> behaviour in this area _changed_, so the semantics are different from what
> they were originally.
> ...
> Or at _least_ there should be a flag to force it.
>
> Junio?
I agree that if this sequence:
$ edit foo
$ git update-index foo
$ edit foo
$ git commit foo
is what the user actually gives from the command line, after
this commit is made, the user would want to _lose_ the state of
foo at the update-index after this commit is made, 100% of the
time (not "most likely", nor "usually", but "always"). So I am
very in favor of removing that check.
I am a bit worried if the reason behind this safety valve might
have been something else, and we describe the reason as "we
would lose data with this sequence otherwise" only to illustrate
what's happening behind the scene in technical terms.
In other words, while I think no user would ever want to keep
the state of foo at update-index after the above exact sequence
as the end-user action, I am worried if a usage sequence that
involve a group of operations encapsulated in a larger command
(a synthetic command that touches index and working files
without making the user painfully aware of the index -- likes of
git-mv, git-rm, ...) might have been the true motivation of the
safety valve.
I need to be reminded by somebody who went back to the list
discussion around the time we introduced --only, and made sure
that the "you would lose the snapshot you staged in the index if
we allowed it" literally meant only that and nothing else; not
some other common sequence that had the above command sequence
inside, and keeping the state of 'foo' at update-index time made
sense for that usage pattern, although I do not think of
anything offhand...
^ permalink raw reply
* [PATCH] git-mv: search more precisely for source directory in index
From: Johannes Schindelin @ 2006-12-03 19:42 UTC (permalink / raw)
To: Sergey Vlasov; +Cc: Junio C Hamano, git
In-Reply-To: <20061203135725.GA7971@procyon.home>
A move of a directory should find the entries in the index by
searching for the name _including_ the slash. Otherwise, the
directory can be shadowed by a file when it matches the prefix
and is lexicographically smaller, e.g. "ab.c" shadows "ab/".
Noticed by Sergey Vlasov.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
---
Good catch. Thanks!
builtin-mv.c | 11 +++++++----
1 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/builtin-mv.c b/builtin-mv.c
index 54dd3bf..d14a4a7 100644
--- a/builtin-mv.c
+++ b/builtin-mv.c
@@ -146,21 +146,24 @@ int cmd_mv(int argc, const char **argv, const char *prefix)
&& lstat(dst, &st) == 0)
bad = "cannot move directory over file";
else if (src_is_dir) {
+ const char *src_w_slash = add_slash(src);
+ int len_w_slash = length + 1;
int first, last;
modes[i] = WORKING_DIRECTORY;
- first = cache_name_pos(src, length);
+ first = cache_name_pos(src_w_slash, len_w_slash);
if (first >= 0)
- die ("Huh? %s/ is in index?", src);
+ die ("Huh? %.*s is in index?",
+ len_w_slash, src_w_slash);
first = -1 - first;
for (last = first; last < active_nr; last++) {
const char *path = active_cache[last]->name;
- if (strncmp(path, src, length)
- || path[length] != '/')
+ if (strncmp(path, src_w_slash, len_w_slash))
break;
}
+ free((char *)src_w_slash);
if (last - first < 1)
bad = "source directory is empty";
--
1.4.4.1.g317bd
^ permalink raw reply related
* Set permissions of each new file before "cvs add"ing it.
From: Jim Meyering @ 2006-12-03 19:51 UTC (permalink / raw)
To: git
Without the following patch, git-cvsexportcommit would fail to propagate
permissions of files added in git to the CVS repository. I.e., when I
added an executable script in coreutils' git repo, then tried to propagate
that addition to the mirroring CVS repository, the script ended up added
not executable there.
Signed-off-by: Jim Meyering <jim@meyering.net>
---
git-cvsexportcommit.perl | 19 +++++++++++++++++++
1 files changed, 19 insertions(+), 0 deletions(-)
diff --git a/git-cvsexportcommit.perl b/git-cvsexportcommit.perl
index 7bac16e..f819eb2 100755
--- a/git-cvsexportcommit.perl
+++ b/git-cvsexportcommit.perl
@@ -268,6 +268,7 @@ if (($? >> 8) == 2) {
}
foreach my $f (@afiles) {
+ set_new_file_permissions($f);
if (grep { $_ eq $f } @bfiles) {
system('cvs', 'add','-kb',$f);
} else {
@@ -342,3 +343,21 @@ sub safe_pipe_capture {
}
return wantarray ? @output : join('',@output);
}
+
+# For any file we want to add to cvs, we must first set its permissions
+# properly, *before* the "cvs add ..." command. Otherwise, it is impossible
+# to change the permission of the file in the CVS repository using only cvs
+# commands. This should be fixed in cvs-1.12.14.
+sub set_new_file_permissions {
+ my ($file) = @_;
+ # Given input like this:
+ # ba45154d8e9f5f49f46c8c2c2d8a554db7c3465f ...
+ # :000000 100755 0000000... b595dc6... A tests/du/one-file-system
+ # extract the three octal permission digits:
+ my $cmd = 'git-whatchanged --max-count=1 --pretty=oneline -- $f'
+ . q! | sed -n '2s/^:00* [0-7][0-7][0-7]\([0-7][0-7][0-7]\) .*/\1/p'!;
+ my $perm = `$cmd`;
+
+ chmod oct($perm), $file
+ or die "failed to set permissions of \"$file\": $!\n";
+}
--
^ permalink raw reply related
* Re: [PATCH] git-mv: search more precisely for source directory in index
From: Sergey Vlasov @ 2006-12-03 20:04 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.63.0612032036030.28348@wbgn013.biozentrum.uni-wuerzburg.de>
[-- Attachment #1: Type: text/plain, Size: 1755 bytes --]
On Sun, Dec 03, 2006 at 08:42:47PM +0100, Johannes Schindelin wrote:
>
> A move of a directory should find the entries in the index by
> searching for the name _including_ the slash. Otherwise, the
> directory can be shadowed by a file when it matches the prefix
> and is lexicographically smaller, e.g. "ab.c" shadows "ab/".
Thanks - seems to work now, and the existing tests are not broken.
> Noticed by Sergey Vlasov.
>
> Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> ---
>
> Good catch. Thanks!
>
> builtin-mv.c | 11 +++++++----
> 1 files changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/builtin-mv.c b/builtin-mv.c
> index 54dd3bf..d14a4a7 100644
> --- a/builtin-mv.c
> +++ b/builtin-mv.c
> @@ -146,21 +146,24 @@ int cmd_mv(int argc, const char **argv, const char *prefix)
> && lstat(dst, &st) == 0)
> bad = "cannot move directory over file";
> else if (src_is_dir) {
> + const char *src_w_slash = add_slash(src);
> + int len_w_slash = length + 1;
> int first, last;
>
> modes[i] = WORKING_DIRECTORY;
>
> - first = cache_name_pos(src, length);
> + first = cache_name_pos(src_w_slash, len_w_slash);
> if (first >= 0)
> - die ("Huh? %s/ is in index?", src);
> + die ("Huh? %.*s is in index?",
> + len_w_slash, src_w_slash);
>
> first = -1 - first;
> for (last = first; last < active_nr; last++) {
> const char *path = active_cache[last]->name;
> - if (strncmp(path, src, length)
> - || path[length] != '/')
> + if (strncmp(path, src_w_slash, len_w_slash))
> break;
> }
> + free((char *)src_w_slash);
>
> if (last - first < 1)
> bad = "source directory is empty";
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: Some advanced index playing
From: Alan Chandler @ 2006-12-03 20:11 UTC (permalink / raw)
To: git; +Cc: Linus Torvalds, Junio C Hamano
In-Reply-To: <Pine.LNX.4.64.0612031008360.3476@woody.osdl.org>
On Sunday 03 December 2006 18:24, Linus Torvalds wrote:
> On Sun, 3 Dec 2006, Alan Chandler wrote:
> > With all the discussion about the index file in the last few days I would
> > have thought that this issue would have come up. But I don't think it
> > has.
> >
> > I have been editing a set of files to make a commit, and after editing
> > each one had done a git update-index.
> >
> > At this point I am just about to commit when I realise that one of the
> > files has changes in it that really ought to be a separate commit.
> >
> > So effectively, I want to do one of three things
> >
> > a) git-commit <that-file>
> >
> > Except I can't because there is a safety valve that prevents this and
> > there is no force option.
...
>
> git ls-tree HEAD -- that-file | git update-index --index-info
> git commit that-file
I don't quite understand this - maybe it should be
git ls-tree HEAD -- that-file | git update-index --index-info
git commit
git commit -a
either I want to ONLY commit that file at the working tree state (and index
before these commands), or I want to commit ALL except this file (so I can
later come and commit just that file)
so having reset the index to the state of HEAD for "that-file" then the commit
should make a commit with all the other changes in the index (but NOT
that-file) and then the git commit -a picks up "that-file"
--
Alan Chandler
^ permalink raw reply
* Re: Some advanced index playing
From: Jakub Narebski @ 2006-12-03 20:19 UTC (permalink / raw)
To: git
In-Reply-To: <200612032011.25922.alan@chandlerfamily.org.uk>
Alan Chandler wrote:
> On Sunday 03 December 2006 18:24, Linus Torvalds wrote:
> ...
>>
>> git ls-tree HEAD -- that-file | git update-index --index-info
>> git commit that-file
>
> I don't quite understand this - maybe it should be
>
> git ls-tree HEAD -- that-file | git update-index --index-info
> git commit
> git commit -a
>
> either I want to ONLY commit that file at the working tree state (and index
> before these commands), or I want to commit ALL except this file (so I can
> later come and commit just that file)
>
> so having reset the index to the state of HEAD for "that-file" then the commit
> should make a commit with all the other changes in the index (but NOT
> that-file) and then the git commit -a picks up "that-file"
$ git ls-tree HEAD -- that-file | git update-index --index-info
$ git commit that-file
$ git commit
resets index state of 'that-file' to the HEAD version, so the safety valve
is bypassed and "git commit that-file" commit _working directory_ version
only (because --only is default for "git commit <path>"). Then you commit
the rest of files.
While
$ git ls-tree HEAD -- that-file | git update-index --index-info
$ git commit
$ git commit -a
first commits rest of files, then all files to their working directory
version.
P.S. Could you wrap lines a little earlier (76 columns perhaps)? Thanks
in advance
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: Some advanced index playing
From: Junio C Hamano @ 2006-12-03 20:26 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git, Alan Chandler
In-Reply-To: <Pine.LNX.4.64.0612031024480.3476@woody.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> So "git reset" is generally your friend whenever something goes wrong. If
> you also want to reset your checked-out files (which you did NOT want to
> do in this case, of course), you would have added the "--hard" flag to git
> reset.
>
> And that (finally) concludes this particularly boring "git usage 101"
> session.
>
> Linus
One observation about git, made in a relatively distant past,
was "git is not a usable system yet; there is no 'git undo'". I
think it was on the kernel list (I think it was from Alan who
seems to have lost his last name from his From: line lately, but
I may be mistaken).
It left a deep psychological trauma in me, not because it was
stated in a brutal way (it wasn't) but because I fully agreed
with that statement from the end user point of view, but I did
not see a good solution to the problem (and I from the beginning
kept saying "I do not do Porcelains" and kept calling what is
shipped with core "Porcelain-ish").
"git reset" is one part of "undo". For example, undoing a
commit can be approximated with "reset HEAD^" or "reset --hard
HEAD^"; undoing a conflicted and unfinished merge can be
approximated with "reset HEAD" or "reset --hard HEAD".
But for one thing, these are only "approximations" (the working
tree files after these two forms of reset are different from the
state you had before running "commit" or "merge"). And for
another thing, "reset" is only one part of "undo". "reset"
would not help "undo"-ing a botched "git bisect good", for
example; you need "git bisect reset". Similarly, "git rebase"
in the middle can be undone with its own --abort option. But
the user has to know about them. Another twist is that once
completed, "rebase --abort" obviously would not mean "undo the
last rebase".
I think one cause of the "problem" (if not having a general
"undo" is a problem, and I think it is to some extent) is that
git Porcelain-ish commands try to stay stateless to allow mixing
and matching of different commands to leave the door open to the
end user to be flexible, but they go too far. Some of the
commands do leave its state (e.g. MERGE_HEAD is a sign of a
merge in progress, and git-commit notices it), and some of the
commands know about state markers from the other commands
(e.g. "git reset" removes MERGE_HEAD). However, I think we do
not do enough of inter-command coordination. Although I haven't
checked, I would be very surprised if we already prevented "git
bisect" from running, while a merge is in progress, for example.
While I do not think we need to supply "git undo" for a command
that already ran successfully to its completion (e.g. I think
the answer to "how would I undo a commit I just made" can be
left as "use one of the reset, or --amend, depending on what you
want"), it might be a worthwhile thing to aim for to give a
unified "git oops" command that recovers from a "unusual" state
your git working tree may be left in.
For UI-usability minded people, it might be a good thing to
enumerate the possible 'states' a working tree can be in, draw a
state transition diagram among them, with possible and forbidden
transitions. After that, we can annotate the allowed
transitions with "use this command to make this transtion
happen".
The following list may be a starter; it is not comprehensive,
and does not list the transitions, though.
- the base state
- bisect in progress
- conflicted merge in progress (or --no-commit given)
- immediately after "merge --squash" is run but not yet committed
- conflicted git am/git applymbox in progress.
- conflicted git rebase in progress (this actually has two
separate states depending on --merge was used or not).
- conflicted git revert/cherry-pick (I list this separately
from 'merge in progress' because this is an example of
command leaving too little state).
- conflicted 'git checkout -m' (although this is almost the
same as the base state, it has higher stages in the index).
^ permalink raw reply
* Re: Some advanced index playing
From: Alan Chandler @ 2006-12-03 20:29 UTC (permalink / raw)
To: git; +Cc: Jakub Narebski
In-Reply-To: <ekvbg3$jjd$1@sea.gmane.org>
On Sunday 03 December 2006 20:19, Jakub Narebski wrote:
> P.S. Could you wrap lines a little earlier (76 columns perhaps)?
> Thanks in advance
Sorry wrap was set at 78 (kmail). I've changed it to 72
--
Alan Chandler
^ permalink raw reply
* Re: Some advanced index playing
From: Alan Chandler @ 2006-12-03 20:40 UTC (permalink / raw)
To: git; +Cc: Linus Torvalds
In-Reply-To: <Pine.LNX.4.64.0612031024480.3476@woody.osdl.org>
On Sunday 03 December 2006 18:34, Linus Torvalds wrote:
...
> So in short, you should just have done "git reset", and you'd have
> reset your index back to the state of your last commit.
>
> So "git reset" is generally your friend whenever something goes
> wrong. If you also want to reset your checked-out files (which you
> did NOT want to do in this case, of course), you would have added the
> "--hard" flag to git reset.
Doh [slaps head with wet blanket]
I was so worried about NOT changing my working tree - I totally
overlooked the reset command. I had never quite understood
what --mixed really meant before. But re-reading the man page now, its
obvious.
--
Alan Chandler
^ permalink raw reply
* Re: Some advanced index playing
From: Linus Torvalds @ 2006-12-03 20:40 UTC (permalink / raw)
To: Alan Chandler; +Cc: git, Junio C Hamano
In-Reply-To: <200612032011.25922.alan@chandlerfamily.org.uk>
On Sun, 3 Dec 2006, Alan Chandler wrote:
> >
> > git ls-tree HEAD -- that-file | git update-index --index-info
> > git commit that-file
>
> I don't quite understand this - maybe it should be
>
> git ls-tree HEAD -- that-file | git update-index --index-info
> git commit
> git commit -a
Sure. It depends on which file you want to commit first.
If you want to commit "that-file" first, you do my sequence.
If you want to commit everything _but_ "that-file", you do the second
sequence (which basically removes the changes to "that-file" from the
index, then commits the index, and then with "git commit -a" commits the
remaining dirty state, which is obviously those changes to "that-file"
that you still had in the working tree).
> either I want to ONLY commit that file at the working tree state (and index
> before these commands), or I want to commit ALL except this file (so I can
> later come and commit just that file)
Right. If you do
git ls-tree HEAD -- that-file | git update-index --index-info
git commit that-file
you basically ONLY commit "that-file". You first reset it (in the index)
to the old state, but that's just so that "git commit that-file" will now
happily commit the current state (in the working tree) of "that-file".
So "git commit that-file" will basically _ignore_ your current index.
Because you told "git commit" (by naming "that-file") that you _only_
wanted to commit "that-file". So whatever state you had in your current
index doesn't matter at all - it will only look at the HEAD tree _and_
that single file that you specified.
^ permalink raw reply
* Re: [RFC] Submodules in GIT
From: Martin Waitz @ 2006-12-03 20:46 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Josef Weidendorfer, sf, git
In-Reply-To: <Pine.LNX.4.64.0612021242080.3476@woody.osdl.org>
[-- Attachment #1: Type: text/plain, Size: 404 bytes --]
hoi :)
On Sat, Dec 02, 2006 at 12:44:20PM -0800, Linus Torvalds wrote:
> And watch the memory usage.
hmm, really sad, it was such a nice concept until now...
You are right, I have to think more about scalability. O(N) anywhere is
really bad for submodules. They really should be able to bundle the kernel,
mozilla, qt and whatnot into one project and that will get huge.
--
Martin Waitz
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: jgit performance update
From: sf @ 2006-12-03 21:55 UTC (permalink / raw)
To: git
In-Reply-To: <20061203045953.GE26668@spearce.org>
Shawn Pearce wrote:
...
> One of the biggest annoyances has been the fact that although Java
> 1.4 offers a way to mmap a file into the process,
Be careful with mmap:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4724038
Regards
Stephan
^ permalink raw reply
* Re: jgit performance update
From: Shawn Pearce @ 2006-12-03 22:16 UTC (permalink / raw)
To: sf; +Cc: git
In-Reply-To: <457347E1.2020800@stephan-feder.de>
sf <sf-gmane@stephan-feder.de> wrote:
> Shawn Pearce wrote:
> ...
> > One of the biggest annoyances has been the fact that although Java
> > 1.4 offers a way to mmap a file into the process,
>
> Be careful with mmap:
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4724038
Thanks. That particular bug has been of some discussion on
#git before. I'm planning on making it a configuration flag in
.git/config for the user to decide how much pain they want: mmap
(with all its huge downsides) or read into byte[] (with all the
memory footprint that requires).
--
^ permalink raw reply
* Re: [RFC] Submodules in GIT
From: Sven Verdoolaege @ 2006-12-03 22:16 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Josef Weidendorfer, sf, git, Martin Waitz, sf
In-Reply-To: <Pine.LNX.4.64.0612011540010.3695@woody.osdl.org>
On Fri, Dec 01, 2006 at 04:12:10PM -0800, Linus Torvalds wrote:
> So within the supermodule, on a "git object" level, a submodule should
> just be named by the SHA1 that was it's HEAD when it was committed within
> the supermodule. So in the "tree object", you'd see something like the
> following when you go "git ls-tree HEAD" on the superproject:
>
> ...
> 100644 blob 08602f522183dc43787616f37cba9b8af4e3dade xdiff-interface.c
> 100644 blob 1346908bea31319aabeabdfd955e2ea9aab37456 xdiff-interface.h
> 040000 tree 959dd5d97e665998eb26c764d3a889ae7903d9c2 xdiff
> 050000 link 0215ffb08ce99e2bb59eca114a99499a4d06e704 xyzzy
>
> where that 050000 is the new magic type (I picked one out of my *ss: it's
> not a valid type for a file mode, so it's a godo choice, but it could be
> anythign that cannot conflict with a real file), which just specifies the
> "link" part. The SHA1 is the SHA1 of the commit, and the "xyzzy" is
> obviously just the name within the directory of the submodule.
>
> That's all that is actually required for a lot of git commands that
> already expect all objects to be available (ie "git checkout", "git diff"
> etc).
But is this object (and all the objects it points to) going to be
available (in the superproject) ?
The following seems to suggest that you think they shouldn't.
How is fsck-objects then going to check that such an object is
valid ? Is it going to call fsck-objects recursively on the
(available) submodules ?
On Fri, Dec 01, 2006 at 03:30:32PM -0800, Linus Torvalds wrote:
> The only thing that a submodule must NOT be allowed to do on its own is
> pruning (and it's distant cousin "git repack -d").
How are you going to enforce this if the submodule isn't supposed
to know that it is being used as a submodule ?
> You must always prune
> from the supermodule, because the submodule cannot really know on its own
> what references point into it.
How is one of the supermodules going to know what references from other
supermodules containing the submodule point into the submodule ?
^ permalink raw reply
* Re: [RFC] Submodules in GIT
From: Linus Torvalds @ 2006-12-03 22:32 UTC (permalink / raw)
To: skimo; +Cc: Josef Weidendorfer, sf, git, Martin Waitz, sf
In-Reply-To: <20061203221630.GA940MdfPADPa@greensroom.kotnet.org>
On Sun, 3 Dec 2006, Sven Verdoolaege wrote:
>
> On Fri, Dec 01, 2006 at 03:30:32PM -0800, Linus Torvalds wrote:
> > The only thing that a submodule must NOT be allowed to do on its own is
> > pruning (and it's distant cousin "git repack -d").
>
> How are you going to enforce this if the submodule isn't supposed
> to know that it is being used as a submodule ?
Note that there's actually two "submodules":
- there's the submodule "project" itself.
This one must be totally unaware of the supermodule, because this one
might be cloned and copied _independently_ of the supermodule.
- there's the PARTICULAR CHECKED-OUT COPY of the submodule that is
actually checked out in a supermodule.
This is just a specific _instance_ of the particular submodule.
So a particular instance of a submodule might be "aware" of the fact that
it's a submodule of a supermodule. For example, the "awareness" migth be
as simple as just a magic flag file inside it's .git/ directory. And that
awareness would be what simply disabled pruning or "repack -d" within that
particular instance.
But this magic flag doesn't affect the bigger-picture git repository. It's
a _private_ flag. So it doesn't affect the git part, any more than it
really affects the git repository that you may have a
[user]
name = Myname
email = myemail
in your .git/config file.
See? You can have private data in a git repository, but that doesn't mean
that it's visible as _repository_ data. But it can still affect how git
commands act (eg the "user" definitions above will affect the default user
information that "git commit" uses, of course, without actually affecting
the git archive in any other way)
> How is one of the supermodules going to know what references from other
> supermodules containing the submodule point into the submodule ?
Why would it care? They are other supermodules. It doesn't matter, the
same way it doesn't matter that _my_ "git" tree may not have all the same
references that _your_ "git" repo has. If I want to get the same
references, I'd need to fetch them from you, and at that point, I'd need
to get all the objects that are pointed to by those refs too. But only on
"git fetch" do you actually start caring.
^ permalink raw reply
* Re: jgit performance update
From: Shawn Pearce @ 2006-12-03 22:47 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0612030938140.3476@woody.osdl.org>
Linus Torvalds <torvalds@osdl.org> wrote:
> Very good. Are we any closer to actually having an eclipse plugin then?
We have some parts in place. But nothing that's user-ready.
> > Walking the 50,000 most recent commits from the Mozilla trunk[1]:
>
> Now, the _interesting_ case in many ways is not "--max-count", but the
> revision limiter. It _should_ be equally fast, but if you've done
> something wrong, it won't be.
We haven't implemented a rev-list equivalent yet. That's a major
feature which is missing. The --max-count test was simple to put
together. I really need to start thinking about replicating some
of the features of the revision functions in core Git.
> > One of the biggest annoyances has been the fact that although Java
> > 1.4 offers a way to mmap a file into the process, the overhead to
> > access that data seems to be far higher than just reading the file
> > content into a very large byte array, especially if we are going
> > to access that file content multiple times.
>
> That must suck for big packed repositories. What JVM and other environment
> are you using?
Mac OS 10.4.8 / Java 1.4.2. It appears as though Sun isn't going
to fix the mmap performance problems as they can't do it "securely".
So its likely to be an issue anywhere jgit gets used...
> Also, I have to say, one of the reasons I'm interested in your project is
> that I've never done any Java programming, because quite frankly, I've
> never had any reason what-so-ever to do so. But if there is some simple
> setup, and you have jgit exposed somewhere as a git archive, I'd love to
> take a look, if only to finally learn more about Java.
Of course its in Git. :-)
There's no webpage here, but you can clone it:
http://www.spearce.org/projects/scm/egit.git
The code you would recognize most is in org.spearce.jgit/.../lib.
E.g. BinaryDelta.java came from patch-delta.c in core Git.
Repository has some of the functionality of sha1_file.c, the pack
reading code is in PackFile.java.
--
^ permalink raw reply
* Re: [RFC] Submodules in GIT
From: Jakub Narebski @ 2006-12-03 22:49 UTC (permalink / raw)
To: git
In-Reply-To: <Pine.LNX.4.64.0612031421030.3476@woody.osdl.org>
Linus Torvalds wrote:
> On Sun, 3 Dec 2006, Sven Verdoolaege wrote:
>>
>> On Fri, Dec 01, 2006 at 03:30:32PM -0800, Linus Torvalds wrote:
>>> The only thing that a submodule must NOT be allowed to do on its own is
>>> pruning (and it's distant cousin "git repack -d").
>>
>> How are you going to enforce this if the submodule isn't supposed
>> to know that it is being used as a submodule ?
>
> Note that there's actually two "submodules":
>
> - there's the submodule "project" itself.
>
> This one must be totally unaware of the supermodule, because this one
> might be cloned and copied _independently_ of the supermodule.
>
> - there's the PARTICULAR CHECKED-OUT COPY of the submodule that is
> actually checked out in a supermodule.
>
> This is just a specific _instance_ of the particular submodule.
>
> So a particular instance of a submodule might be "aware" of the fact that
> it's a submodule of a supermodule. For example, the "awareness" migth be
> as simple as just a magic flag file inside it's .git/ directory. And that
> awareness would be what simply disabled pruning or "repack -d" within that
> particular instance.
If we use objects/info/alternates (or equivalent, e.g. objects/info/modules,
or modules file) in superproject to refer to submodule repository object
database (so superproject has access to all the objects including
submodule), I'd prefer to have in submodule objects/info/borrowers file,
which would point to superproject (and to other repositories which have
submodule as one of alternate object databases) for git-prune and friends
to check which parts are truly unreachable.
This would be generic solution to the problem with alternates, not only
specific to submodule support.
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: jgit performance update
From: Shawn Pearce @ 2006-12-03 22:59 UTC (permalink / raw)
To: Robin Rosenberg; +Cc: git
In-Reply-To: <200612031455.48032.robin.rosenberg.lists@dewire.com>
Robin Rosenberg <robin.rosenberg.lists@dewire.com> wrote:
> So, just go on to the next case. I added filtering on filenames (yes,
> CVS-induced brain damage, I should track the content. next version. filenames
> are so much handier to work with). That gives me 4.5s to retrieve a filtered
> history (from 10800 commits).Half of the time is spent in re-sorting tree
> entries. Is that really necessary?
Yea, I was looking at that code while doing the other performance
improvements and thought it might start to become a bottleneck. I
guess I was right.
What is happening here is jgit wants to store the items in the tree
in name ordering, but Git stores the items in the tree sorted such
that subtrees sort with a '/' on the end of their name. This is a
different ordering...
The reason I'm resorting them is so we can find an entry without
knowing what its type is first. Looks like that's going to have
to change somehow.
> Most of java's slowness comes from the programmers using it. (Lutz Prechelt.
> Technical opinion: comparing Java vs. C/C++ efficiency differences to
> interpersonal differences. ACM, Vol 42,#10, 1999)
Yes, that was clearly the case here with jgit! :-)
_This_ programmer made jgit slow. Learned from the mistake, and
made it faster.
> > One of the biggest annoyances has been the fact that although Java
> > 1.4 offers a way to mmap a file into the process, the overhead to
> > access that data seems to be far higher than just reading the file
> > content into a very large byte array, especially if we are going
> > to access that file content multiple times. So jgit performs worse
> > than core Git early on while it copies everything from the OS buffer
> > cache into the Java process, but then performs reasonably well once
> > the internal cache is hot. On the other hand using the mmap call
> > reduces early latency but hurts the access times so much that we're
> > talking closer to 3s average read times for the same log operation.
>
> Have you tried that with difference JVM's?
No, I'm on Mac OS X so I don't have a huge JVM selection (that I
know of). And I haven't tried jgit or egit on any other system yet.
--
^ permalink raw reply
* [RFC] gitweb: Add committags support (take 2)
From: Jakub Narebski @ 2006-12-03 23:01 UTC (permalink / raw)
To: git
This is my second attemt at adding properly committags support, similar
to what gitweb-xmms2 added in e89abf232a83e579b2bbc76398aefa048661dddf.
In this attempt I follow Junio idea of sequential committags support,
marking some fragments as not-for-parsing for later committags nor
esc_html, instead of building the list of literal replacements to be
applied simultaneously.
The plan of patch series is the following:
* [PATCH 1/3] gitweb: Add committags support
(which would add generic committags support, reusing existing
infrastructure of commit message line by line parsing and
simplification: leave git_print_log as is, replacing only
format_log_line_html by committags capable version)
* [PATCH 2/3] gitweb: Abuse committags support
(which would use committags for marking signoff lines, and for
commit message simplification: compacting empty lines, removing
initial and trailing empty lines)
* [PATCH 3/3] gitweb: Use committags in subject line and tag comment
(which would add committags support to subject line/title of commit,
which is first line of commit message, and to tag message/comment)
But there are still some problems to be solved before posting proper
even the first patch; I'll discuss them part by part.
1. Should we provide examples of committags, even if they _have_ to have
be per project, or at least per gitweb instalation configuration, like
bugtracker support (depending on the bugtracker used by
project/projects) and message id support (depending on mailing list
archive project/projects use)? So far I have come with the following
committag examples (they are for [PATCH 1/3] in the series, so they
don't include signoff committag, not empty_lines_simplification
committag).
Should we use anonymous subroutines, or should we define one subroutine
per committag (how to name them: format_<name>_tag, or <name>_committag,
or committag_<name>,... ?) and reference it? The code below uses
anonymous subroutines.
The subroutine should return either reference to scalar which means
"do not process", scalar which means changed available for further
processing, or void (undef) which means no change. In [PATCH 2/3] we
will enable returning also list of elements, each of which could be
reference to scalar or scalar (for example signoff would return three
element list: opening span element as ref, signoff text as scalar,
closing span element as ref).
our %committags = (
'sha1' => {
'pattern' => qr/[0-9a-fA-F]{40}/,
'sub' => sub {
my $hash_text = shift;
my $type = git_get_type($hash_text);
if ($type) {
return \$cgi->a({-href => href(action=>$type, hash=>$hash_text),
-class => "text"}, $hash_text);
}
return undef;
},
},
'mantis' => {
'pattern' => qr/(BUG|FEATURE)\(\d+\)/,
'options' => [ 'http://bugs.xmms2.xmms.se/view.php?id=' ],
'sub' => sub {
my $match = shift;
my $URL = shift;
my ($issue) = $match =~ /(\d+)/;
return
\$cgi->a({-href => "$URL$issue"},
$match);
},
},
'bugzilla' => {
'pattern' => qr/bug\s+\d+/,
'options' => [ 'http://bugzilla.kernel.org/show_bug.cgi?id=' ],
'sub' => sub {
my $match = shift;
my $URL = shift;
my ($bugno) = $match =~ /(\d+)/;
return
\$cgi->a({-href => "$URL=$bugno"},
$match);
},
},
'URL' => {
'pattern' => qr!(http|ftp)s?://[a-zA-Z0-9%./]+[a-zA-Z0-9/]!,
'sub' => sub {
my $url = shift;
return
\$cgi->a({-href => $url},
$url); # should be perhaps shortened
},
},
# this is committags v1 subroutine
'message_id' => { # use mailing list archive
'pattern' => qr!(message|msg)-id:?\s+<[^>]*>;!i,
'options' => [ 'http://news.gmane.org/find-root.php?message_id=' ],
'sub' => sub {
my $text = shift;
my $URL = shift;
$text =~ m/<([^>]*)>/;
my $msgid = $1;
my $link = $cgi->a({-href => "$URL<$msgid>"},
$msgid);
$text =~ s/$msgid/$link/;
return \$text;
},
},
);
From those committags, only 'sha1' (used now in the limited commitsha
form currently in gitweb) and 'URL' are installation independent.
For xmms2 the 'mantis' tag is needed; for Linux kernel it would be nice
to have both 'bugzilla' and 'message_id' (with some LKML archive) used.
By the way, if you have ideas for other committags, please do write.
For example should esc_html be the last committag?
2. We might not want to use all defined committags in %committags hash.
We might want only some of them for commit message [body], some of them
for subject line, and some of them for tag message. Additionally the
_sequence_ of committags is important. As we cannot rely on
"keys %committags" to be in any specific order, we have to provide
the ordering ourself. My idea is to use array with the hash keys
(the names of committags) to both define which committags are enabled,
and the sequence of committags itself.
In [PATCH 1/3] there is need for only one such array.
our @committags = ('sha1');
3. To not split message into many fragments we concatenate strings
if possible. This happens when committag subroutine returns string
(for example we might want to hyperlink only bug number in 'bugzilla'
committag), or return undef (for example 'sha1' committag if the
there does not exist object with given hash, for example in the commit
message of commit cherrypicked from no longer existing branch).
This is example where Perl subroutine prototypes makes sense.
# push_or_append ARRAY,LIST
# if ARRAY ends with scalar, and LIST begins with scalar, concatenate
sub push_or_append (\@@) {
my $list = shift;
if (ref $_[0] || ! @$list || ref $list->[-1]) {
push @$list, @_;
} else {
my $a = pop @$list;
my $b = shift @_;
push @$list, $a . $b, @_;
}
# imitate push
return scalar @$list;
}
4. Here is the main subroutine for committags support, currently without
the extra parts which would allow committags in the subject line: the
subject is link (<a> element) itself, and in (X)HTML link elements cannot
be nested, so the encompassing <a> element has to be closed and reopened).
I have thought about three solutions, neither without disadvantages.
* Mark committag as returning link in %committag hash, and wrap it in
the format_log_line_html subroutine. Disadvantages: prone to config
errors, doesn't work if link is only part of replacement.
* Use special a_link subroutine in the committag subs instead of
$cgi->a(). Disadvantages: need to pass attributes of encompassing
<a> element to committag subroutine, prone to committag subroutine
coding errors.
* Wrap on dereferencing in the final output stage, checking if referenced
string is wholly <a> element. Disadvantages: committags subroutine
must return <a> elements separately.
Well, yet another solution would be to forbid committags support in the
subject/title line. gitweb-xmms2 tries to implement this, but not fully,
and not without errors.
sub format_log_line_html {
my $line = shift;
my @committags = @_;
my @list = ( $line );
COMMITTAG:
foreach my $ctname (@committags) {
next COMMITTAG unless exists $committags{$ctname};
my @opts =
exists $committags{$ctname}{'options'} ?
@{$committags{$ctname}{'options'}} :
();
my @newlist = ();
PART:
foreach my $part (@list) {
next PART unless $part;
if (ref($part)) {
push @newlist, $part;
next PART;
}
my $oldpos = 0;
MATCH:
while ($part =~ m/($committags{$ctname}{'pattern'})/gc) {
my $match = $1;
my ($prepos, $postpos) = ($-[0], $+[0]);
my $repl = $committags{$ctname}{'sub'}->($match, @opts);
if (defined $repl) {
my $pre = substr($part, $oldpos, $prepos - $oldpos);
push_or_append @newlist, $pre if $pre;
push_or_append @newlist, $repl;
} else {
my $all = substr($part, $oldpos, $postpos - $oldpos);
push_or_append @newlist, $all if $all;
}
$oldpos = $postpos;
} # end while [regexp matches]
my $rest = substr($part, $oldpos);
push_or_append @newlist, $rest if $rest;
} # end foreach (@list)
@list = @newlist;
} # end foreach (@committags)
# print it
my $result = '';
for my $part (@list) {
if (ref($part)) {
$result .= $$part;
} else {
$result .= esc_html($part, -nbsp=>1);
}
}
return $result;
}
--
Jakub Narebski
^ permalink raw reply
* Re: jgit performance update
From: Shawn Pearce @ 2006-12-03 23:06 UTC (permalink / raw)
To: Robin Rosenberg; +Cc: Jakub Narebski, git
In-Reply-To: <200612031653.04019.robin.rosenberg.lists@dewire.com>
Robin Rosenberg <robin.rosenberg.lists@dewire.com> wrote:
> söndag 03 december 2006 15:19 skrev Jakub Narebski:
> > Robin Rosenberg wrote:
> > > CVS-induced brain damage, I should track the content. next version.
> > > filenames are so much handier to work with).
> >
> > Git uses <path> as _revision limiter_, not as output filter. Shouldn't
> > jgit do the same?
> It's egit, i.e. the eclipse plugin I'm referring to so it's a user interface
> thing and it uses the path name.
Jakub's point is that "git log -- a" lists only the revisions
which affect path 'a'. Once those revisions have been selected
then output begins. The 'a' gets reapplied as an output filter to
only show changes relevant to file 'a', but its very much a revision
filter during the revision listing process.
--
^ permalink raw reply
* Re: jgit performance update
From: Juergen Stuber @ 2006-12-03 22:42 UTC (permalink / raw)
To: git; +Cc: jnareb
In-Reply-To: <ekv34g$mck$1@sea.gmane.org>
Hi Jakub,
Jakub Narebski <jnareb@gmail.com> writes:
>
> GitWiki tells us about egit/jgit repository at
> http://www.spearce.org/projects/scm/egit.git
I tried to access that with git 1.4.4.1 from Debian but
% git clone http://www.spearce.org/projects/scm/egit.git
hangs, the first time after "walk e339766abc2b919e7bb396cae22ddef065821381",
the second time after "walk 9eec90ec5da239e063eaff6305d77294dc03396e"
which is the "walk" line just before it.
There's also the following error shortly after the start:
error: File bc01ab9e5fcd26918d7a334207183fa57ff1ce50 (http://www.spearce.org/projects/scm/egit.git/objects/75/1c8f2e504c40d1c41ebbd87d8f8968529e9c30) corrupt
Jürgen
--
Jürgen Stuber <juergen@jstuber.net>
http://www.jstuber.net/
gnupg key fingerprint = 2767 CA3C 5680 58BA 9A91 23D9 BED6 9A7A AF9E 68B4
^ permalink raw reply
* Re: jgit performance update
From: Robin Rosenberg @ 2006-12-03 23:39 UTC (permalink / raw)
To: Juergen Stuber; +Cc: git, jnareb
In-Reply-To: <874psceh4z.fsf@freitag.home.jstuber.net>
söndag 03 december 2006 23:42 skrev Juergen Stuber:
> Hi Jakub,
>
> Jakub Narebski <jnareb@gmail.com> writes:
> > GitWiki tells us about egit/jgit repository at
> > http://www.spearce.org/projects/scm/egit.git
>
> I tried to access that with git 1.4.4.1 from Debian but
>
> % git clone http://www.spearce.org/projects/scm/egit.git
>
> hangs, the first time after "walk
> e339766abc2b919e7bb396cae22ddef065821381", the second time after "walk
> 9eec90ec5da239e063eaff6305d77294dc03396e" which is the "walk" line just
> before it.
Works fine here. (git 1.4.4.gf05d).
>
> There's also the following error shortly after the start:
>
> error: File bc01ab9e5fcd26918d7a334207183fa57ff1ce50
> (http://www.spearce.org/projects/scm/egit.git/objects/75/1c8f2e504c40d1c41e
>bbd87d8f8968529e9c30) corrupt
Unfortunately, messages about corrupt objects are "normal" with clone over
http. I'm not sure it has to be that way though. Run git-fsck-objects to make
sure there are no errors. The hangs aren't normal.
-- robin
^ permalink raw reply
* Using sha1 of commit for id of gitweb Atom feed entry
From: Jakub Narebski @ 2006-12-03 23:48 UTC (permalink / raw)
To: git
To all using Atom version of gitweb feeds: perhaps instead of using
appropriate "commit" view gitweb URL as if of gitweb Atom feed entry,
use sha1 of commit? We could create a new (unofficial for now) scheme:
git:<sha1 of a commit>
or we can try to use tag URI scheme
* "How to make a good ID in Atom"
http://diveintomark.org/archives/2004/05/28/howto-atom-id
* "Tag URI"
http://www.taguri.org/
* RFC 4151 - The 'tag' URI Scheme
With tag URI we need:
1.) Identity by domain name or email address. We can use
(can we, Pasky?) use git.or.cz domain, or git@vger.kernel.org
email address
2.) Pick a date. We can use here epoch start date of 1970,
date of 1.0 git release of 2005-12-21, or date of creating git
mailing list of 2005, 2005-04 or 2005-04-10.
So for example the id for an entry which currently has id of
http://repo.or.cz/w/git.git?a=commit;h=278fcd7debf19c1efee18b32f8867becb18d1a22
would have either
git:278fcd7debf19c1efee18b32f8867becb18d1a22
or (one of the choices)
tag:git@vger.kernel.org,2005:278fcd7debf19c1efee18b32f8867becb18d1a22
What do you think about it?
P.S. Or we could register URN... ;-)
--
Jakub Narebski
^ permalink raw reply
* Re: jgit performance update
From: Jakub Narebski @ 2006-12-03 23:58 UTC (permalink / raw)
To: git
In-Reply-To: <200612040039.00315.robin.rosenberg.lists@dewire.com>
Robin Rosenberg wrote:
> söndag 03 december 2006 23:42 skrev Juergen Stuber:
>>
>> Jakub Narebski <jnareb@gmail.com> writes:
>>>
>>> GitWiki tells us about egit/jgit repository at
>>> http://www.spearce.org/projects/scm/egit.git
>>
>> I tried to access that with git 1.4.4.1 from Debian but
>>
>> % git clone http://www.spearce.org/projects/scm/egit.git
>>
>> hangs, the first time after "walk
>> e339766abc2b919e7bb396cae22ddef065821381", the second time after "walk
>> 9eec90ec5da239e063eaff6305d77294dc03396e" which is the "walk" line just
>> before it.
>
> Works fine here. (git 1.4.4.gf05d).
Works fine here. (git 1.4.4.1)
>> There's also the following error shortly after the start:
>>
>> error: File bc01ab9e5fcd26918d7a334207183fa57ff1ce50
>> (http://www.spearce.org/projects/scm/egit.git/objects/75/1c8f2e504c40d1c41e
>>bbd87d8f8968529e9c30) corrupt
>
> Unfortunately, messages about corrupt objects are "normal" with clone over
> http. I'm not sure it has to be that way though. Run git-fsck-objects to make
> sure there are no errors. The hangs aren't normal.
I got:
$ git clone http://www.spearce.org/projects/scm/egit.git
[...]
got 73ed47b2bb1fa5978f7368775979e5c85d354c5a
error: File 2332eacf114debb7a27d138811197f06eb262551
(http://www.spearce.org/projects/scm/egit.git/objects/75/1c8f2e504c40d1c41ebbd87d8f8968529e9c30) corrupt
Getting pack list for http://www.spearce.org/projects/scm/egit.git/
got afefbe09bacc08adb75fb46200a973001c6b02de
[...]
walk c1f287cb19b9910af19756cf29c08b1fda75da8c
Some loose object were found to be corrupt, but they might be just
a false '404 Not Found' error message sent with incorrect HTTP
status code. Suggest running git fsck-objects.
got eab86de8ac23e2e77878835007724146fdd83796
$ git fsck-objects --unreachable --full --strict ;# returns no errors
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: jgit performance update
From: Shawn Pearce @ 2006-12-04 0:46 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <ekvoan$lst$1@sea.gmane.org>
Jakub Narebski <jnareb@gmail.com> wrote:
> I got:
> $ git clone http://www.spearce.org/projects/scm/egit.git
> [...]
> got 73ed47b2bb1fa5978f7368775979e5c85d354c5a
> error: File 2332eacf114debb7a27d138811197f06eb262551
> (http://www.spearce.org/projects/scm/egit.git/objects/75/1c8f2e504c40d1c41ebbd87d8f8968529e9c30) corrupt
> Getting pack list for http://www.spearce.org/projects/scm/egit.git/
> got afefbe09bacc08adb75fb46200a973001c6b02de
> [...]
> walk c1f287cb19b9910af19756cf29c08b1fda75da8c
> Some loose object were found to be corrupt, but they might be just
> a false '404 Not Found' error message sent with incorrect HTTP
> status code. Suggest running git fsck-objects.
> got eab86de8ac23e2e77878835007724146fdd83796
> $ git fsck-objects --unreachable --full --strict ;# returns no errors
Right. My web server doesn't send back 404 messages when an object
isn't found (but is packed). Thus the error...
I've setup a project on repo.or.cz:
http://repo.or.cz/w/egit.git
--
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox