* Re: [PATCH 00/15] git-note: A mechanisim for providing free-form after-the-fact annotations on commits
From: Shawn O. Pearce @ 2007-05-29 3:22 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, Johan Herland, git
In-Reply-To: <alpine.LFD.0.98.0705282008400.26602@woody.linux-foundation.org>
Linus Torvalds <torvalds@linux-foundation.org> wrote:
>
> On Mon, 28 May 2007, Shawn O. Pearce wrote:
> >
> > What if we use a "slow" storage by "refs/notes/$objname/$notename",
>
> This _really_ won't scale. Even if the notes don't exist, just doing the
> lookup (which will fail for most commits) will be horribly slow, and will
> populate the dentry cache with negative entries.
Yes.
I think you missed what I was trying to say. We *definately* do not
want to probe the OS and ask "do you have $objname1? $objname2?"
for exactly the reason you just stated. (Though you being some
sort of kernel guru means you know a hellva lot more about that
dentry cache thing than I do!)
What I meant was something more like:
DIR *d = opendir("refs/notes");
if (d) {
... notes can exist in both packfiles and "loose" ...
only_packed_notes = 0;
while (readdir(d)) {
... the entry name here is the name of an object ...
... stuff into a hash table, we can come back later ...
... for the subdirectory contents if we need it ...
}
closedir(d);
... also examine "packed-refs", in case any are there ...
} else {
only_packed_notes = 1;
... we *only* have notes in packfiles, if we have any at all...
}
Then looking up a note is just a probe into our in-memory hash
(if only_packed_notes is false) and a probe into the packfile(s) to
find the notes for the object. Not very expensive if the packfiles
have the reverse obj->tag mappings indexed within them.
> To get good filesystem performance, you have to
> - have reasonable hit-rates (and looking it up for each commit is _not_
> going to do that)
> - not have millions of objects.
Which is why we:
a) allow these things to migrate into packed-refs, because
getting into there is a hellva lot cheaper than getting into
a packfile;
b) move them into a packfile when we repack loose objects,
because then we have really good access.
c) take them out of packed-refs once they are into a packfile,
and get them out of the loose refs/notes directory as early as
possible.
--
Shawn.
^ permalink raw reply
* Re: [PATCH 00/15] git-note: A mechanisim for providing free-form after-the-fact annotations on commits
From: Linus Torvalds @ 2007-05-29 3:12 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: Junio C Hamano, Johan Herland, git
In-Reply-To: <20070528213511.GB7044@spearce.org>
On Mon, 28 May 2007, Shawn O. Pearce wrote:
>
> What if we use a "slow" storage by "refs/notes/$objname/$notename",
This _really_ won't scale. Even if the notes don't exist, just doing the
lookup (which will fail for most commits) will be horribly slow, and will
populate the dentry cache with negative entries.
To get good filesystem performance, you have to
- have reasonable hit-rates (and looking it up for each commit is _not_
going to do that)
- not have millions of objects.
So you'd have to have a separate database. You could do it with a separate
index file (or mixing it up with the "index v4" and doing it with a single
index file that also contains normal objects), but the point is, it's
going to be a real separate database.
The current "decorations" approach is actually much more efficient with
packed refs (dense in the filesystem), and probably not very bad at all
for a smallish set (ie a few hundred to a few thousand) objects, but just
reading all the decorations ends up being an overhead at some point.
Linus
^ permalink raw reply
* Re: [PATCH] Don't ignore write failure from git-diff, git-log, etc.
From: Linus Torvalds @ 2007-05-29 3:01 UTC (permalink / raw)
To: Jim Meyering; +Cc: git
In-Reply-To: <871wh0ww80.fsf@rho.meyering.net>
On Mon, 28 May 2007, Jim Meyering wrote:
>
> Um... maybe you've forgotten that this patch fixes a hole in the
> "old code" (git.c). Many git tools ignore write (ENOSPC) failures.
Maybe you have not noticed, but my argument has ben about EPIPE.
> Compared to that aspect of the fix, I would have thought EPIPE-
> handling would be a minor detail. But now, the whole patch has
> become "crap"?
About half the patch was _removing_ EPIPE stuff - not at all about the
ENOSPC stuff you claim.
And the ENOSPC code could have added the same *correct* code that does the
right thing for EPIPE.
> 1) Continue to ignore EPIPE write failure: can obscure real errors.
> BTW, Linus, don't you agree? You never commented on this point.
THAT'S THE ONLY THING I'VE BEEN COMMENTING ON!
They aren't "obscure real errors". EPIPE is neither obscure _nor_ an
error.
The code-paths where you removed EPIPE handlign have two cases:
- SIGPIPE happens: you made no change
- SIGPIPE diesn't happen: you broke the code.
So remind me again, why the hell do you think your patch is so great and
so important, considering that it broke real code, and made things worse?
And why don't you just admit that EPIPE is special, isn't an error, and
shouldn't be complained about? If you get EPIPE on the write, it means
"the other end didn't care". It does NOT mean "I should now do a really
annoying message".
It's that simple. You seem to admit that SIGPIPE handling in bash should
have been fixed, and that it was annoying to complain about it there. Why
can't you just admit that it's annoyign and wrong to complain about the
same thing when it's EPIPE?
Linus
^ permalink raw reply
* Re: [PATCH 1/3] builtin-pack-objects: don't fail, if delta is not possible
From: Shawn O. Pearce @ 2007-05-29 2:53 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Martin Koegler, Junio C Hamano, git
In-Reply-To: <alpine.LFD.0.99.0705282243280.11491@xanadu.home>
Nicolas Pitre <nico@cam.org> wrote:
> On Mon, 28 May 2007, Martin Koegler wrote:
>
> > If builtin-pack-objects runs out of memory while finding
> > the best deltas, it bails out with an error.
> >
> > If the delta index creation fails (because there is not enough memory),
> > we can downgrade the error message to a warning and continue with the
> > next object.
>
> In the same vain, there is one realloc() that was turned into a
> xrealloc() in diff-delta.c. I think this was a mistake and should
> probably be a non fatal realloc again to let the caller go on.
And if those two calls fail to alloc their memory, they might want
to try calling the pack window gc thingy (release_pack_memory(need,
-1)) and then retry the alloc before they fail the delta generation.
Its possible that we are better off releasing the LRU pack window
and produce the delta, then to fail because we're hanging onto some
mmap we don't need...
And actaully release_pack_memory could get more aggressive now that
the index_data can be lazily loaded. We could actually unload LRU
indexes too.
--
Shawn.
^ permalink raw reply
* Re: [PATCH 1/3] builtin-pack-objects: don't fail, if delta is not possible
From: Nicolas Pitre @ 2007-05-29 2:45 UTC (permalink / raw)
To: Martin Koegler; +Cc: Junio C Hamano, git
In-Reply-To: <11803872591522-git-send-email-mkoegler@auto.tuwien.ac.at>
On Mon, 28 May 2007, Martin Koegler wrote:
> If builtin-pack-objects runs out of memory while finding
> the best deltas, it bails out with an error.
>
> If the delta index creation fails (because there is not enough memory),
> we can downgrade the error message to a warning and continue with the
> next object.
In the same vain, there is one realloc() that was turned into a
xrealloc() in diff-delta.c. I think this was a mistake and should
probably be a non fatal realloc again to let the caller go on.
Nicolas
^ permalink raw reply
* [PATCH] Makefile: Remove git-fsck and git-verify-pack from PROGRAMS
From: Nguyen Thai Ngoc Duy @ 2007-05-29 2:05 UTC (permalink / raw)
To: git
Those are builtins. Remove them from PROGRAMS variable
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
Makefile | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/Makefile b/Makefile
index 29243c6..fb11fa1 100644
--- a/Makefile
+++ b/Makefile
@@ -235,7 +235,7 @@ endif
# ... and all the rest that could be moved out of bindir to gitexecdir
PROGRAMS = \
- git-convert-objects$X git-fetch-pack$X git-fsck$X \
+ git-convert-objects$X git-fetch-pack$X \
git-hash-object$X git-index-pack$X git-local-fetch$X \
git-fast-import$X \
git-merge-base$X \
@@ -246,7 +246,7 @@ PROGRAMS = \
git-show-index$X git-ssh-fetch$X \
git-ssh-upload$X git-unpack-file$X \
git-update-server-info$X \
- git-upload-pack$X git-verify-pack$X \
+ git-upload-pack$X \
git-pack-redundant$X git-var$X \
git-merge-tree$X git-imap-send$X \
git-merge-recursive$X \
--
1.5.2
^ permalink raw reply related
* Re: [PATCH 3/3] builtin-pack-object: cache small deltas
From: Dana How @ 2007-05-29 0:33 UTC (permalink / raw)
To: Martin Koegler; +Cc: Junio C Hamano, git, danahow
In-Reply-To: <11803872602056-git-send-email-mkoegler@auto.tuwien.ac.at>
On 5/28/07, Martin Koegler <mkoegler@auto.tuwien.ac.at> wrote:
> Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
> ---
> Caching small deltas improves packing time even on small repostistories.
> Repacking git.git with a delta size limit of 1000 brings CPU time from
> 66 to 49 seconds down. A limit of 500 bytes is only two secondes slower.
>
> The implicit cache size limit is (#objects)*(delta size limit).
This patchset appears a lot more robust over more special cases.
Thanks for your extra work,
--
Dana L. How danahow@gmail.com +1 650 804 5991 cell
^ permalink raw reply
* Re: [PATCH 1/3] Lazily open pack index files on demand
From: linux @ 2007-05-29 0:09 UTC (permalink / raw)
To: git, torvalds
> First off, when comparing O(logn) to O(1), with "n" being less than a
> billion, they are pretty much exactly the same. Think of it this way:
> O(logn) == O(9) == O(1), if you know that n < 10**9.
Well, binary searches mean binary logarithms, so O(log n) = O(30).
Still, pretty low.
> Secondly, the cost of Newton isn't "almost O(1)". I don't know _what_ it
> is (the rule of thumb with Newton-Raphson should be that the number of
> significant correct digits in the answer doubles with each iteration: I
> think that probably means that it should approximate O(loglog(n)), but I
> haven't thought deeply about it.
Excellent intuition! The algorithm is most commonly known in the computer
science literature as "interpolation search" and it does indeed take O(log
log n) time for uniformly distributed data, which is a good assumption
for SHA-1 hashes.
Of course, for n = 10**9, log(n) is 30 and log log n is 5.
More to the point, for n = 10**6, log(n) is 20 and log(log(n)) is still 5.
Even losing a constant factor of 2, it still seems like it might offer a
factor-of-2 speedup for large repositories.
^ permalink raw reply
* [PATCH] Add fsck_verify_ref_to_tag_object() to verify that refname matches name stored in tag object
From: Johan Herland @ 2007-05-28 23:46 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Linus Torvalds
In-Reply-To: <7vps4kbrtb.fsf@assigned-by-dhcp.cox.net>
On Monday 28 May 2007, Junio C Hamano wrote:
> However it would be a good
> idea to add logic to fsck to warn upon inconsistencis (perhaps
> by mistake) between refname and tag's true name.
>
> The check would say something like:
>
> If an annotated (signed or unsigned) tag has a "tag"
> line to give it the official $name, and if it is pointed
> at by a ref, the refname must end with "/$name".
> Otherwise we warn.
>
> Trivially, the above rule says that having v2.6.22 tag under
> refs/tags/v2.6.20 is a mistake we would want to be warned upon.
This patch adds the check described by Junio.
It is assumed that the "tag" header is mandatory for all tag objects.
This might change in the future, at which point this patch should be
revised.
Signed-off-by: Johan Herland <johan@herland.net>
---
I hope this is what you had in mind :)
Have fun!
...Johan
builtin-fsck.c | 19 +++++++++++++++++++
1 files changed, 19 insertions(+), 0 deletions(-)
diff --git a/builtin-fsck.c b/builtin-fsck.c
index cbbcaf0..3594bd3 100644
--- a/builtin-fsck.c
+++ b/builtin-fsck.c
@@ -501,6 +501,23 @@ static int fsck_handle_reflog(const char *logname, const unsigned char *sha1, in
return 0;
}
+static void fsck_verify_ref_to_tag_object(const char *refname, struct object *obj)
+{
+ /* Verify that refname matches the name stored in obj's "tag" header */
+ struct tag *tagobj = (struct tag *) parse_object(obj->sha1);
+ size_t tagname_len = strlen(tagobj->tag);
+ size_t refname_len = strlen(refname);
+
+ if (tagname_len < refname_len &&
+ !memcmp(tagobj->tag, refname + (refname_len - tagname_len), tagname_len) &&
+ refname[(refname_len - tagname_len) - 1] == '/') {
+ /* OK: tag name is "$name", and refname ends with "/$name" */
+ return;
+ }
+ else
+ error("%s: Mismatch between tag ref and tag object's name %s", refname, tagobj->tag);
+}
+
static int fsck_handle_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)
{
struct object *obj;
@@ -515,6 +532,8 @@ static int fsck_handle_ref(const char *refname, const unsigned char *sha1, int f
/* We'll continue with the rest despite the error.. */
return 0;
}
+ if (obj->type == OBJ_TAG) /* ref to tag object */
+ fsck_verify_ref_to_tag_object(refname, obj);
default_refs++;
obj->used = 1;
mark_reachable(obj, REACHABLE);
--
1.5.2.87.g875de-dirty
^ permalink raw reply related
* Re: [PATCH] Don't ignore write failure from git-diff, git-log, etc.
From: Petr Baudis @ 2007-05-28 23:41 UTC (permalink / raw)
To: Jim Meyering; +Cc: Marco Roeland, git
In-Reply-To: <87veecvgsn.fsf@rho.meyering.net>
(I think that funnily enough, Linus is to a degree to the Git
community something like Al Viro and Chris Hellwig are to the Linux
kernel community. Don't get too derailed by his blunt^Whonest criticism,
which is however usually quite valid. ;-)
On Mon, May 28, 2007 at 10:23:20PM CEST, Jim Meyering wrote:
> Marco Roeland <marco.roeland@xs4all.nl> wrote:
> > On monday May 28th 2007 at 20:19 Jim Meyering wrote:
> >> Also, to be consistent, don't ignore EPIPE write failures.
> >
> > In practice I agree with someone else on this thread that EPIPE _is_
> > different. In a way the responsibility doesn't lie with the writer but
> > with the reader.
>
> Do you think it's ok for git-rev-list _not_ to diagnose an erroneous
> command like this (i.e., to exit(0)):
>
> git-rev-list HEAD | sync
>
> where "sync" could be any command that exits successfully
> without reading any input?
>
> Is it ok that it is currently *impossible* to diagnose that
> failure by looking at exit codes?
Actually, yes!
Because there's no "failure" per se. The command we piped the output
into just decided that he isn't actually interested in any (for whatever
reason; it might decide dynamically based on some parameters etc.). I
can't think of why it could be considered a failure for git-rev-list if
its customer doesn't happily eat all the output it generates. It's the
customer's job to report any real trouble that happenned and might be
cause of the premature end (or maybe the premature end was totally
valid).
Maybe it could expose some (IMHO contrived) error scenarios, but in
most cases I think it will end up just spitting out bogus error
messages. And what will people do? They won't bother to filter out this
particular one (which isn't even that easy if the strerror() is
localized, furthermore). They will just 2>/dev/null it. And cause the
*real* error messages go to the land of void as well. There's enough of
impossible-to-diagnose-error-conditions-because-stderr-goes-to-null
scripts in the land of UNIX already and this patch, while actually
well-meant to do the opposite, might well actually increase their number
because of this.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
-- Samuel Beckett
^ permalink raw reply
* Re: [PATCH 00/15] git-note: A mechanisim for providing free-form after-the-fact annotations on commits
From: Johannes Schindelin @ 2007-05-28 23:37 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: Junio C Hamano, Johan Herland, Linus Torvalds, git
In-Reply-To: <20070528213511.GB7044@spearce.org>
Hi,
On Mon, 28 May 2007, Shawn O. Pearce wrote:
> Junio C Hamano <junkio@cox.net> wrote:
> > Johan Herland <johan@herland.net> writes:
> > > Ok. But the reverse mapping will help with this, won't it?
> > > We'll look up the interesting commits and find their associated
> > > note objects directly.
> >
> > The issue Linus brought up worries me, too.
> >
> > The "efficient reverse mapping" is still handwaving at this
> > stage. What it needs to do is an equivalent to your
> > implementation with "refs/notes/<a dir per commit>/<note>". The
> > "efficient" one might do a flat file that says "notee note" per
> > line sorted by notee, or it might use BDB or sqlite, but the
> > amount of the data and complexity of the look-up is really the
> > same. A handful notes per each commit in the history (I think
> > Linus's "Acked-by after the fact" example a very sensible thing
> > to want from this subsystem).
>
> Please, don't use BDB or sqllite. I really don't trust either.
> I've lost data to both. I've *never* lost data to a Git packfile.
> ;-)
Also, there is the question of dependencies. We don't have to repeat SVN's
errors.
As for sqllite: the only reason I was _not_ horrified when I realized that
cvsserver needs sqllite was that it is not really important for the
common user.
> I'm actually thinking pack v4. OK, I know its just a virtual hand
> waving thing still, but there's really no reason Nico and I cannot
> get the damn thing finished before we both wind up buying the farm.
Maybe I am missing something important here, but I think we do not at all
need a generic database here. Just a key/value store.
As it happens, we already have such a beast. It is called object store
here.
Now, the only reason we cannot do something like "SHA-1 ^ 0xff[...]ff" is
the SHA-1 for the note for that commit, is that you could possibly have
more than one note for the commit.
Of course, without that restriction, we could reuse our object store logic
for notes.
Side note: the more I think about this notes thing, the more I get
disgusted by the deep changes we'd have to do just to accomodate for them.
Are they really worth it? Or would something like a pseudo-branch suffice,
being one strand of notes (commits), where the second commit parent is
actually the commit the note is for?
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH] gitweb: handle non UTF-8 text
From: Petr Baudis @ 2007-05-28 23:21 UTC (permalink / raw)
To: Martin Koegler; +Cc: Jakub Narebski, git
In-Reply-To: <1180385254576-git-send-email-mkoegler@auto.tuwien.ac.at>
On Mon, May 28, 2007 at 10:47:34PM CEST, Martin Koegler wrote:
> gitweb assumes, that everything is in UTF-8. If a text contains invalid
> UTF-8 character sequences, the text must be in a different encoding.
>
> This patch interprets such a text as latin1.
>
> Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
> ---
> For correct UTF-8, the patch does not change anything.
>
> If commit/blob/... is not in UTF-8, it displays the text
> with a very high probability correct.
>
> As git itself is not aware of any encoding, I know no better
> possibility to handle non UTF-8 text in gitweb.
I don't think this is a reasonable approach; I actually dispute the high
probability - in western Europe it's obvious to assume latin1, but does
majority of users using non-ascii characters come from there? Or rather
from central Europe (like me, Petr Baudiš? ;-))? Somewhere else?
If we do something like this, we should do it properly and look at
configured i18n.commitEncoding for the project. (But as config lookup
may be expensive, probably do it only when we need it.)
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
-- Samuel Beckett
^ permalink raw reply
* [PATCH] Fix stupid typo in lookup_tag()
From: Johan Herland @ 2007-05-28 23:21 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
Signed-off-by: Johan Herland <johan@herland.net>
---
tag.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/tag.c b/tag.c
index 330d287..bbacd59 100644
--- a/tag.c
+++ b/tag.c
@@ -26,7 +26,7 @@ struct tag *lookup_tag(const unsigned char *sha1)
if (!obj->type)
obj->type = OBJ_TAG;
if (obj->type != OBJ_TAG) {
- error("Object %s is a %s, not a tree",
+ error("Object %s is a %s, not a tag",
sha1_to_hex(sha1), typename(obj->type));
return NULL;
}
--
1.5.2.101.gee49f
^ permalink raw reply related
* What's new in git-gui.git
From: Shawn O. Pearce @ 2007-05-28 22:49 UTC (permalink / raw)
To: Junio C Hamano, git
I got around to working on git-gui this week. Yay me. Updates for
both the stable (0.7.x) series and the new development series
(0.8.x), and some roadmap items. See below...
--- Stable ---
The following changes since gitgui-0.7.1
Shawn O. Pearce (1):
git-gui: Gracefully handle bad TCL_PATH at compile time
are available in the git repository at:
repo.or.cz:/srv/git/git-gui.git maint
Shawn O. Pearce (3):
git-gui: Tighten internal pattern match for lib/ directory
Correct key bindings to Control-<foo>
git-gui: Guess our share/git-gui/lib path at runtime if possible
Makefile | 12 ++++++++++--
git-gui.sh | 17 +++++++++--------
2 files changed, 19 insertions(+), 10 deletions(-)
The final commit in this series has been tagged as gitgui-0.7.2
and should be packaged as part of the next maintenance release of
git 1.5.2. All are trivial fixes for minor issues raised since
0.7.1 was released.
--- Development ---
The following changes since gitgui-0.7.2
Shawn O. Pearce (1):
git-gui: Guess our share/git-gui/lib path at runtime if possible
are available in the git repository at:
repo.or.cz:/srv/git/git-gui.git master
Shawn O. Pearce (9):
git gui 0.8.0
git-gui: GUI support for running 'git remote prune <name>'
git-gui: Show the git-gui library path in 'About git-gui'
git-gui: Enable verbose Tcl loading earlier
git-gui: Provide fatal error if library is unavailable
git-gui: Disable tearoff menus on Windows, Mac OS X
git-gui: Allow users to rename branches through 'branch -m'
git-gui: Allow users to delete remote branches
git-gui: Expose the merge.diffstat configuration option
GIT-VERSION-GEN | 2 +-
git-gui.sh | 87 +++++++----
lib/branch_rename.tcl | 137 +++++++++++++++++
lib/option.tcl | 17 ++-
lib/remote.tcl | 11 ++
lib/remote_branch_delete.tcl | 348 ++++++++++++++++++++++++++++++++++++++++++
lib/transport.tcl | 16 ++-
7 files changed, 586 insertions(+), 32 deletions(-)
create mode 100644 lib/branch_rename.tcl
create mode 100644 lib/remote_branch_delete.tcl
This is actually the start of the 0.8.0 development cycle. It is
stable enough for git.git's master branch. I don't anticipate this
version of git-gui being packaged with any git earlier than 1.5.3.
Some major new features have been added:
- Rename an existing branch (git-brach -m).
- Delete an existing remote branch.
- Prune expired tracking branches.
- Automatically prune expired tracking branches during fetch.
I'm hoping to do a lot more before 0.8.0 goes final. Which is part
of the reason why I don't expect this to be packaged with anything
earlier than 1.5.3.
--- Roadmap ---
Looking ahead, I'd like to see some improvement in the following
areas. It would be nice if some Tcl/Tk-minded individuals were
willing to dive and help. ;-)
- Initial repository creation
- Initial repository cloning
- Select an existing repository from an OS native file dialog
These are mostly about better supporting the MinGW environment
(or a Cygwin environment) where some users are not used to
performing operations through a command line terminal window.
Sometimes being able to start an application through an icon
and have it prompt you for initial configuration is exactly what
the user wants.
At the same time we probably should support running against a
bare repository, at least for the cases of the "browser" and
"blame" subcommands.
- Blame viewer coloring should be less horrid
- Blame viewer should support digging into the prior version
- Jump to line in blame viewer (use vi's :nnn syntax?)
- Find substring in blame viewer (use vi's / syntax?)
These have been discussed before. The current blame viewer is
useful, but not fun to look at. It also doesn't let you dig
back into history further than the revision it was started on.
Now that the blame viewer is cleaned up and uses git-gui's
"class/field/method" system this is probably a lot easier to
get done (pre-0.7.0 that code was just a disaster).
- Fetch+merge from arbitrary URL and branch(es)
Aka "git pull", but in a GUI. Pulling from random URLs is not
uncommon in the Git user community and we should support it as
a first-class operation in git-gui.
The UI I just put in for deleting remote branches may offer a
good starting point for this, as it does an async ls-remote to
get the available branch heads.
- Preserve author during amend
- Set commit author line
These sort of go hand-in-hand. Currently there is no way to enter
an author; the authorship of a commit is always the current user.
But I know a few people using git-gui who really should be doing
`git commit --author=$someoneelse` every once in a while, as
they are committing files for $someoneelse.
I'm also getting bit by the amend bug myself when I apply
patches from other people and then go back and try to tweak
something minor.
--
Shawn.
^ permalink raw reply
* Re: [PATCH 00/15] git-note: A mechanisim for providing free-form after-the-fact annotations on commits
From: Shawn O. Pearce @ 2007-05-28 21:35 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johan Herland, Linus Torvalds, git
In-Reply-To: <7vwsysbrtg.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano <junkio@cox.net> wrote:
> Johan Herland <johan@herland.net> writes:
> > Ok. But the reverse mapping will help with this, won't it?
> > We'll look up the interesting commits and find their associated
> > note objects directly.
>
> The issue Linus brought up worries me, too.
>
> The "efficient reverse mapping" is still handwaving at this
> stage. What it needs to do is an equivalent to your
> implementation with "refs/notes/<a dir per commit>/<note>". The
> "efficient" one might do a flat file that says "notee note" per
> line sorted by notee, or it might use BDB or sqlite, but the
> amount of the data and complexity of the look-up is really the
> same. A handful notes per each commit in the history (I think
> Linus's "Acked-by after the fact" example a very sensible thing
> to want from this subsystem).
Please, don't use BDB or sqllite. I really don't trust either.
I've lost data to both. I've *never* lost data to a Git packfile.
;-)
I'm actually thinking pack v4. OK, I know its just a virtual hand
waving thing still, but there's really no reason Nico and I cannot
get the damn thing finished before we both wind up buying the farm.
What if we use a "slow" storage by "refs/notes/$objname/$notename",
and we also allow them to appear in the packed-refs file. But during
a repack we instead stick the annotations into the same packfile as
$objname, and we also include a list of $notename after $objname's
other data.
This way we have quick access to the $notename(s) of all notes of
$objname through the pack, and we can lazily go get the notes raw
data if we need them. This isn't too different from what we do
with parent fields. We initialize the commit_list when we parse
the commit but we don't parse the parents until we really need them.
Once packed we delete the note ref (if loose) and during a repack
of the packed-refs file we delete the note if $notename exists in
the packfile.
If someone wants notes we can check to see if refs/notes exists; if
it does then we enumerate all refs and catalog the notes we found
in memory. Note search then works off the in-memory list and off
the packfiles. If refs/notes doesn't exist (and we should delete it
when we prune away those ref files or prune them out of packed-refs)
then we can skip the ref enumeration and just go straight to the
packfile(s). Most notes will be in the packfiles. I think most
people repack often enough that the handful of unpacked notes before
the next repack won't be a major bottleneck. Especially since we
can get the target $objname directly from a readdir() call, or by
splitting the string in the packed-refs file.
>From an object enumeration standpoint during packfile generation
the notes for a given object are treated like the parent fields in
a commit; they come after the object itself, but unlike the parent
fields they are always output if the object itself was output.
(Hence an --objects-edge enumeration would include the notes only
if the commit itself had been included.)
Unfortunately that doesn't cover the case of a note being added
months later and needing to distribute it to clients that already
have the object the note is attached to.
I haven't been following this discussion very closely, but I'd also
like to suggest that if annotated tags are being used for notes
that the "tag <name>" field be left out of them. I don't see why
a note should be given a specific name that sits in a (roughly)
global namespace.
--
Shawn.
^ permalink raw reply
* Re: [PATCH] Don't ignore write failure from git-diff, git-log, etc.
From: Junio C Hamano @ 2007-05-28 21:27 UTC (permalink / raw)
To: Jim Meyering; +Cc: Linus Torvalds, git
In-Reply-To: <87sl9hw0o0.fsf@rho.meyering.net>
Jim Meyering <jim@meyering.net> writes:
> Of course error messages are annoying when your short-pipe-read is
> _deliberate_ (tho, most real uses of git tools will actually get no
> message to be annoyed about[*]), but what if there really *is* a mistake?
> Try this:
>
> # You want to force git to ignore the error.
> $ trap '' PIPE; git-rev-list HEAD | sync
> $
It is perfectly valid (although it is stupid) for a Porcelain
script to do this:
latest_by_jim=$(git log --pretty=oneline --author='Jim' | head -n 1)
case "$latest_by_jim" in
'') echo "No commit by Jim" ;;
*) # do something interesting on the commit
;;;
esac
In such a case, it is a bit too much for my taste to force the
script to redirect what comes out of fd 2 of the upstream of the
pipe, so that it can filter out only the "write error" message
but still show other kinds of error messages. You could do so
by elaborate shell magic, perhaps like this:
filter_pipe_error () {
exec 3>&1
(eval "$1" 2>&1 1>&3 | grep >&2 -v 'Broken pipe')
}
latest_by_jim=$(filter_pipe_error \
'git log --pretty=oneline --author='\''Jim'\'' | head -n 1'
)
but what's the point?
I think something like this instead might be more palatable.
diff --git a/write_or_die.c b/write_or_die.c
index 5c4bc85..fadfcaa 100644
--- a/write_or_die.c
+++ b/write_or_die.c
@@ -41,8 +41,8 @@ int write_in_full(int fd, const void *buf, size_t count)
void write_or_die(int fd, const void *buf, size_t count)
{
if (write_in_full(fd, buf, count) < 0) {
- if (errno == EPIPE)
- exit(0);
- die("write error (%s)", strerror(errno));
+ if (errno != EPIPE)
+ die("write error (%s)", strerror(errno));
+ exit(1);
}
}
^ permalink raw reply related
* [PATCH 3/3] builtin-pack-object: cache small deltas
From: Martin Koegler @ 2007-05-28 21:20 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Martin Koegler
In-Reply-To: <11803872591103-git-send-email-mkoegler@auto.tuwien.ac.at>
Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
---
Caching small deltas improves packing time even on small repostistories.
Repacking git.git with a delta size limit of 1000 brings CPU time from
66 to 49 seconds down. A limit of 500 bytes is only two secondes slower.
The implicit cache size limit is (#objects)*(delta size limit).
Documentation/config.txt | 4 ++++
builtin-pack-objects.c | 8 ++++++++
2 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 83cc4cd..0061f7f 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -572,6 +572,10 @@ pack.deltaCacheSize::
gitlink:git-pack-objects[1].
A value of 0 means no limit. Defaults to 0.
+pack.deltaCacheLimit::
+ The maxium size of a delta, that is cached in
+ gitlink:git-pack-objects[1]. Defaults to 1000.
+
pull.octopus::
The default merge strategy to use when pulling multiple branches
at once.
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 85e08dc..c316fea 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -79,6 +79,7 @@ static int pack_compression_seen;
static unsigned long delta_cache_size = 0;
static unsigned long max_delta_cache_size = 0;
+static unsigned long cache_max_small_delta_size = 1000;
/*
* The object names in objects array are hashed with this hashtable,
@@ -1403,6 +1404,9 @@ static int delta_cacheable (struct unpacked *trg, struct unpacked *src,
if (max_delta_cache_size && delta_cache_size + delta_size > max_delta_cache_size)
return 0;
+ if (delta_size < cache_max_small_delta_size)
+ return 1;
+
/* cache delta, if objects are large enough compared to delta size */
if ((src_size >> 20) + (trg_size >> 21) > (delta_size >> 10))
return 1;
@@ -1654,6 +1658,10 @@ static int git_pack_config(const char *k, const char *v)
max_delta_cache_size = git_config_int(k, v);
return 0;
}
+ if(!strcmp(k, "pack.deltacachelimit")) {
+ cache_max_small_delta_size = git_config_int(k, v);
+ return 0;
+ }
return git_default_config(k, v);
}
--
1.5.2.846.g9a144
^ permalink raw reply related
* [PATCH 2/3] git-pack-objects: cache small deltas between big objects
From: Martin Koegler @ 2007-05-28 21:20 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Martin Koegler
In-Reply-To: <11803872591522-git-send-email-mkoegler@auto.tuwien.ac.at>
Creating deltas between big blobs is a CPU and memory intensive task.
In the writing phase, all (not reused) deltas are redone.
This patch adds support for caching deltas from the deltifing phase, so
that that the writing phase is faster.
The caching is limited to small deltas to avoid increasing memory usage very much.
The implemented limit is (memory needed to create the delta)/1024.
Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
---
The patch is an improved version of the last patch. I added a limit
for the cache size. It's only configureable by git-config, as
builtin-back-objects (git-repack) has already enough options.
The patch applies on top of next.
Documentation/config.txt | 5 +++
builtin-pack-objects.c | 69 ++++++++++++++++++++++++++++++++++++----------
2 files changed, 59 insertions(+), 15 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 3d8f03d..83cc4cd 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -567,6 +567,11 @@ pack.compression::
slowest. If not set, defaults to core.compression. If that is
not set, defaults to -1.
+pack.deltaCacheSize::
+ The maxium memory in bytes used for caching deltas in
+ gitlink:git-pack-objects[1].
+ A value of 0 means no limit. Defaults to 0.
+
pull.octopus::
The default merge strategy to use when pulling multiple branches
at once.
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 17627b3..85e08dc 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -36,6 +36,7 @@ struct object_entry {
struct object_entry *delta_sibling; /* other deltified objects who
* uses the same base as me
*/
+ void *delta_data; /* cached delta (uncompressed) */
unsigned long delta_size; /* delta data size (uncompressed) */
enum object_type type;
enum object_type in_pack_type; /* could be delta */
@@ -76,6 +77,9 @@ static struct progress progress_state;
static int pack_compression_level = Z_DEFAULT_COMPRESSION;
static int pack_compression_seen;
+static unsigned long delta_cache_size = 0;
+static unsigned long max_delta_cache_size = 0;
+
/*
* The object names in objects array are hashed with this hashtable,
* to help looking up the entry by object name.
@@ -405,24 +409,31 @@ static unsigned long write_object(struct sha1file *f,
z_stream stream;
unsigned long maxsize;
void *out;
- buf = read_sha1_file(entry->sha1, &type, &size);
- if (!buf)
- die("unable to read %s", sha1_to_hex(entry->sha1));
- if (size != entry->size)
- die("object %s size inconsistency (%lu vs %lu)",
- sha1_to_hex(entry->sha1), size, entry->size);
- if (usable_delta) {
- buf = delta_against(buf, size, entry);
+ if (entry->delta_data && usable_delta) {
+ buf = entry->delta_data;
size = entry->delta_size;
obj_type = (allow_ofs_delta && entry->delta->offset) ?
OBJ_OFS_DELTA : OBJ_REF_DELTA;
} else {
- /*
- * recover real object type in case
- * check_object() wanted to re-use a delta,
- * but we couldn't since base was in previous split pack
- */
- obj_type = type;
+ buf = read_sha1_file(entry->sha1, &type, &size);
+ if (!buf)
+ die("unable to read %s", sha1_to_hex(entry->sha1));
+ if (size != entry->size)
+ die("object %s size inconsistency (%lu vs %lu)",
+ sha1_to_hex(entry->sha1), size, entry->size);
+ if (usable_delta) {
+ buf = delta_against(buf, size, entry);
+ size = entry->delta_size;
+ obj_type = (allow_ofs_delta && entry->delta->offset) ?
+ OBJ_OFS_DELTA : OBJ_REF_DELTA;
+ } else {
+ /*
+ * recover real object type in case
+ * check_object() wanted to re-use a delta,
+ * but we couldn't since base was in previous split pack
+ */
+ obj_type = type;
+ }
}
/* compress the data to store and put compressed length in datalen */
memset(&stream, 0, sizeof(stream));
@@ -1385,6 +1396,20 @@ struct unpacked {
struct delta_index *index;
};
+static int delta_cacheable (struct unpacked *trg, struct unpacked *src,
+ unsigned long src_size, unsigned long trg_size,
+ unsigned long delta_size)
+{
+ if (max_delta_cache_size && delta_cache_size + delta_size > max_delta_cache_size)
+ return 0;
+
+ /* cache delta, if objects are large enough compared to delta size */
+ if ((src_size >> 20) + (trg_size >> 21) > (delta_size >> 10))
+ return 1;
+
+ return 0;
+}
+
/*
* We search for deltas _backwards_ in a list sorted by type and
* by size, so that we see progressively smaller and smaller files.
@@ -1466,10 +1491,20 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,
if (!delta_buf)
return 0;
+ if (trg_entry->delta_data) {
+ delta_cache_size -= trg_entry->delta_size;
+ free (trg_entry->delta_data);
+ }
+ trg_entry->delta_data = 0;
trg_entry->delta = src_entry;
trg_entry->delta_size = delta_size;
trg_entry->depth = src_entry->depth + 1;
- free(delta_buf);
+
+ if (delta_cacheable (src, trg, src_size, trg_size, delta_size)) {
+ trg_entry->delta_data = realloc(delta_buf, delta_size);
+ delta_cache_size += trg_entry->delta_size;
+ } else
+ free(delta_buf);
return 1;
}
@@ -1615,6 +1650,10 @@ static int git_pack_config(const char *k, const char *v)
pack_compression_seen = 1;
return 0;
}
+ if(!strcmp(k, "pack.deltacachesize")) {
+ max_delta_cache_size = git_config_int(k, v);
+ return 0;
+ }
return git_default_config(k, v);
}
--
1.5.2.846.g9a144
^ permalink raw reply related
* [PATCH 1/3] builtin-pack-objects: don't fail, if delta is not possible
From: Martin Koegler @ 2007-05-28 21:20 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Martin Koegler
If builtin-pack-objects runs out of memory while finding
the best deltas, it bails out with an error.
If the delta index creation fails (because there is not enough memory),
we can downgrade the error message to a warning and continue with the
next object.
Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
---
The patches apply on top of next.
builtin-pack-objects.c | 8 ++++++--
1 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index e52332d..17627b3 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1454,8 +1454,12 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,
}
if (!src->index) {
src->index = create_delta_index(src->data, src_size);
- if (!src->index)
- die("out of memory");
+ if (!src->index) {
+ static int warned = 0;
+ if (!warned++)
+ warning("suboptimal pack - out of memory");
+ return 0;
+ }
}
delta_buf = create_delta(src->index, trg->data, trg_size, &delta_size, max_size);
--
1.5.2.846.g9a144
^ permalink raw reply related
* Re: [PATCH 00/15] git-note: A mechanisim for providing free-form after-the-fact annotations on commits
From: Shawn O. Pearce @ 2007-05-28 21:19 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johan Herland, Linus Torvalds, git
In-Reply-To: <7vps4kbrtb.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano <junkio@cox.net> wrote:
> Shawn graciously names
> his tags in git-gui project "gitgui-0.7.1", to avoid tag name
> collisions with "v1.5.0" style tag names in git.git project, but
> we could have stored his v0.7.1 tag as refs/tags/gitgui/v0.7.1
> to differenciate the tag namespace (this needs an update to the
> "tag following" code in git-fetch with a new configuration).
Actually I use that same naming convention everywhere else except
in egit. I've found it easier to have the "major project name"
prefixed in front of the version number as it sort of reminds me
of what I'm looking at... yes, I get confused easily. There are
far too many "v1.0.0", "v1.0.1" around... ;-)
--
Shawn.
^ permalink raw reply
* [PATCH] Let .git/config specify the url for submodules
From: Lars Hjemli @ 2007-05-28 20:51 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
This changes git-submodule in a few ways:
a. Section names in .gitmodules are now called 'path', not 'module'.
b. Each 'path' section is required to have a 'submodule' key, used to
give the submodule a name.
c. During 'git-submodule init', the submodule name and url is found in
.gitmodules and stored in .git/config under the key 'submodule.$name.url'.
The 'init' command does not clone the submodule.
d. A new git-submodule command, 'clone', will map submodule path to submodule
name in .gitmodules and use the name to find a url for the submodule in
.git/config. This url is then used to clone the submodule before checking
out the commit named in the index of the containing repository.
The main reason for these changes is to make it easier to specify alternate
urls for a submodule without touching .gitmodules, i.e. like this:
$ git submodule init
$ git config submodule.foobar.url /pub/git/foobar.git
$ git submodule clone
It also might turn out to be nice to have submodules identified by a 'logical
name' instead of by path when a single path is used for different submodules
in different versions of a 'superproject', and if/when the submodules gets
cloned somewhere below .git of the containing repository.
Signed-off-by: Lars Hjemli <hjemli@gmail.com>
---
This is on top of my previous 'submodule test-script' patch (which was on top
of 'next', sorry for not mentioning this in the mail).
Documentation/git-submodule.txt | 36 ++++++++-----
git-submodule.sh | 109 ++++++++++++++++++++++++++++-----------
t/t7400-submodule-basic.sh | 69 ++++++++++++++++--------
3 files changed, 148 insertions(+), 66 deletions(-)
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index cb0424f..c93e40f 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -8,7 +8,7 @@ git-submodule - Initialize, update or inspect submodules
SYNOPSIS
--------
-'git-submodule' [--quiet] [--cached] [status|init|update] [--] [<path>...]
+'git-submodule' [--quiet] [--cached] [status|init|clone|update] [--] [<path>...]
COMMANDS
@@ -16,22 +16,25 @@ COMMANDS
status::
Show the status of the submodules. This will print the SHA-1 of the
currently checked out commit for each submodule, along with the
- submodule path and the output of gitlink:git-describe[1] for the
- SHA-1. Each SHA-1 will be prefixed with `-` if the submodule is not
- initialized and `+` if the currently checked out submodule commit
+ submodule name, path and the output of gitlink:git-describe[1] for
+ the SHA-1. Each SHA-1 will be prefixed with `-` if the submodule is
+ not initialized and `+` if the currently checked out submodule commit
does not match the SHA-1 found in the index of the containing
repository. This command is the default command for git-submodule.
init::
- Initialize the submodules, i.e. clone the git repositories specified
- in the .gitmodules file and checkout the submodule commits specified
- in the index of the containing repository. This will make the
- submodules HEAD be detached.
+ Initialize the submodules, i.e. register the submodules and their
+ suggested urls in $GIT_DIR/config of the containing repository.
+
+clone::
+ Clone the initialized git repositories and checkout the submodule
+ commits specified in the index of the containing repository. This
+ will make the submodule HEADs be detached.
update::
Update the initialized submodules, i.e. checkout the submodule commits
specified in the index of the containing repository. This will make
- the submodules HEAD be detached.
+ the submodule HEADs be detached.
OPTIONS
@@ -48,18 +51,25 @@ OPTIONS
Path to submodule(s). When specified this will restrict the command
to only operate on the submodules found at the specified paths.
+
FILES
-----
-When cloning submodules, a .gitmodules file in the top-level directory
-of the containing repository is used to find the url of each submodule.
-This file should be formatted in the same way as $GIR_DIR/config. The key
-to each submodule url is "module.$path.url".
+git-submodule uses a file named .gitmodules in the top-level directory of the
+containing repository. This file should be formatted in the same way as
+$GIR_DIR/config.
+
+The .gitmodules file is used to find the name and suggested url for each
+submodule path, using the keys `path.$path.submodule` and 'path.$path.url'.
+The submodule name and url is stored in $GIT_DIR/config by
+`git-submodule init`, using the key `submodule.$name.url`, and this key is
+then used during `git-submodule clone` to lookup the preferred submodule url.
AUTHOR
------
Written by Lars Hjemli <hjemli@gmail.com>
+
GIT
---
Part of the gitlink:git[7] suite
diff --git a/git-submodule.sh b/git-submodule.sh
index 6ed5a6c..31de80e 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -4,11 +4,12 @@
#
# Copyright (c) 2007 Lars Hjemli
-USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
+USAGE='[--quiet] [--cached] [status|init|clone|update] [--] [<path>...]'
. git-sh-setup
require_work_tree
init=
+clone=
update=
status=
quiet=
@@ -26,7 +27,7 @@ say()
}
#
-# Run clone + checkout on missing submodules
+# Register submodules in .git/config
#
# $@ = requested paths (default to all)
#
@@ -35,9 +36,56 @@ modules_init()
git ls-files --stage -- "$@" | grep -e '^160000 ' |
while read mode sha1 stage path
do
+ # Get the submodule name for this path
+ name=$(GIT_CONFIG=.gitmodules git-config path."$path".submodule)
+ test -z "$name" && die "Unnamed submodule at path '$path'"
+
+ # Skip registered submodules
+ url=$(git-config submodule."$name".url)
+ test -z "$url" || continue
+
+ # find suggested url in .gitmodules
+ url=$(GIT_CONFIG=.gitmodules git-config path."$path".url)
+ test -z "$url" &&
+ die "No url found for submodule '$name' (path '$path') in .gitmodules"
+
+ # Save the url in local config, indicating that this
+ # submodule is now registered (and enabling the user to
+ # override the url before running 'git submodule clone'
+ git-config submodule."$name".url "$url" ||
+ die "Failed to register url '$url' for submodule '$name'"
+
+ say "Submodule '$name' registered for path '$path', using url '$url'"
+ done
+}
+
+#
+# Run clone + checkout on missing submodules
+#
+# $@ = requested paths (default to all)
+#
+modules_clone()
+{
+ git ls-files --stage -- "$@" | grep -e '^160000 ' |
+ while read mode sha1 stage path
+ do
+ # Get the submodule name for this path
+ name=$(GIT_CONFIG=.gitmodules git-config path."$path".submodule)
+ test -z "$name" && die "Unnamed submodule at path '$path'"
+
+ # Get the url for the module from local config
+ url=$(git-config submodule."$name".url)
+ if test -z "$url"
+ then
+ # If this path was specified on the command line,
+ # make sure the user gets a proper errormessage.
+ test "$#" != "$0" &&
+ die "Submodule '$name' (path '$path') not initialized"
+
+ continue
+ fi
+
# Skip submodule paths that already contain a .git directory.
- # This will also trigger if $path is a symlink to a git
- # repository
test -d "$path"/.git && continue
# If there already is a directory at the submodule path,
@@ -54,28 +102,17 @@ modules_init()
test -e "$path" &&
die "A file already exist at path '$path'"
- url=$(GIT_CONFIG=.gitmodules git-config module."$path".url)
- test -z "$url" &&
- die "No url found for submodule '$path' in .gitmodules"
-
- # MAYBE FIXME: this would be the place to check GIT_CONFIG
- # for a preferred url for this submodule, possibly like this:
- #
- # modname=$(GIT_CONFIG=.gitmodules git-config module."$path".name)
- # alturl=$(git-config module."$modname".url)
- #
- # This would let the versioned .gitmodules file use the submodule
- # path as key, while the unversioned GIT_CONFIG would use the
- # logical modulename (if present) as key. But this would need
- # another fallback mechanism if the module wasn't named.
+ # Use the submodule name to find the preferred url in local config
+ url=$(git-config submodule."$name".url)
+ test -z "$url" && die "No url found for submodule '$name' (path '$path')"
git-clone -n "$url" "$path" ||
- die "Clone of submodule '$path' failed"
+ die "Clone of submodule '$name' (path '$path') failed"
(unset GIT_DIR && cd "$path" && git-checkout -q "$sha1") ||
- die "Checkout of submodule '$path' failed"
+ die "Checkout of '$sha1' in submodule '$name' (path '$path') failed"
- say "Submodule '$path' initialized"
+ say "Submodule '$name' (path '$path') cloned and checked out"
done
}
@@ -94,27 +131,33 @@ modules_update()
# Only mention uninitialized submodules when its
# path have been specified
test "$#" != "0" &&
- say "Submodule '$path' not initialized"
+ say "Submodule in directory '$path' not initialized"
continue;
fi
+
+ # Get the module name for this path
+ name=$(GIT_CONFIG=.gitmodules git-config path."$path".submodule)
+ test -z "$name" && die "Unnamed submodule at path '$path'"
+
+ # Get the SHA-1 of the submodule HEAD
subsha1=$(unset GIT_DIR && cd "$path" &&
git-rev-parse --verify HEAD) ||
- die "Unable to find current revision of submodule '$path'"
+ die "Unable to find current revision of submodule '$module' (path '$path')"
if test "$subsha1" != "$sha1"
then
(unset GIT_DIR && cd "$path" && git-fetch &&
git-checkout -q "$sha1") ||
- die "Unable to checkout '$sha1' in submodule '$path'"
+ die "Unable to checkout '$sha1' for submodule '$name' (path '$path')"
- say "Submodule '$path': checked out '$sha1'"
+ say "Submodule '$name' (path '$path'): checked out '$sha1'"
fi
done
}
#
# List all registered submodules, prefixed with:
-# - submodule not initialized
+# - submodule not initialized or cloned
# + different revision checked out
#
# If --cached was specified the revision in the index will be printed
@@ -153,6 +196,9 @@ do
init)
init=1
;;
+ clone)
+ clone=1
+ ;;
update)
update=1
;;
@@ -178,14 +224,17 @@ do
shift
done
-case "$init,$update,$status,$cached" in
-1,,,)
+case "$init,$clone,$update,$status,$cached" in
+1,,,,)
modules_init "$@"
;;
-,1,,)
+,1,,,)
+ modules_clone "$@"
+ ;;
+,,1,,)
modules_update "$@"
;;
-,,*,*)
+,,,*,*)
modules_list "$@"
;;
*)
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index a023c87..234f940 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -5,8 +5,8 @@
test_description='Basic submodule support in porcelain
-This test tries to run the init, update and status commands of git-submodule
-with a known good setup (and two known bad)
+This test tries to verify basic sanity of the init, clone, update and
+status commands of git-submodule.
'
. ./test-lib.sh
@@ -24,32 +24,57 @@ cd ..
echo a >a && echo z >z
git-add a lib z && git-commit -q -m "super commit 1"
-# move submodule to another location, register repo url in .gitmodules
+# move submodule to another location
mv lib .subrepo
-GIT_CONFIG=.gitmodules git-config module.lib.url ./.subrepo
+
+# register a broken url in .gitmodules
+GIT_CONFIG=.gitmodules git-config path.lib.url ./broken
test_expect_success 'status is "missing"' \
'git-submodule status | grep "^-$rev1"'
-# make sure 'init' will not overwrite a regular file
-touch lib
-test_expect_failure 'init fails when path is used by a file' \
+# 'init' should fail, since the submodule is unnamed in .gitmodules
+test_expect_failure 'do not initialize unnamed submodules' \
'git-submodule init'
-# make sure 'init' will not overwrite a nonempty directory
+# give the submodule a logical name
+GIT_CONFIG=.gitmodules git-config path.lib.submodule thelib
+test_expect_success 'init should register the submodule in .git/config' '
+ git-submodule init &&
+ url=$(git-config submodule.thelib.url) &&
+ test "$url" = "./broken"
+'
+
+# specify correct url in local config
+git-config submodule.thelib.url ./.subrepo
+
+# make sure 'clone' will not overwrite a regular file
+touch lib
+test_expect_failure 'clone fails when path is used by a file' \
+ 'git-submodule clone'
+
+test_expect_success 'the annoying file is preserved' \
+ 'test -f lib'
+
+# make sure 'clone' will not overwrite a nonempty directory
rm lib
mkdir -p lib/foo
-test_expect_failure 'init fails when path is used by a nonempty directory' \
- 'git-submodule init'
+test_expect_failure 'clone fails when path is used by a nonempty directory' \
+ 'git-submodule clone'
-# turn lib into an empty directory, just like git-checkout would do
+test_expect_success 'the annoying directory is preserved' \
+ 'test -d lib/foo'
+
+# make lib be an empty directory, just like git-checkout would have done
rmdir lib/foo
-test_expect_success 'init works when path is an empty dir' \
- 'git-submodule init && test -d lib/.git && git-diff --exit-code'
-head=$(cd lib && git-rev-parse HEAD)
-test_expect_success 'submodule HEAD should match rev1' \
- 'test "$head" = "$rev1"'
+test_expect_success 'clone into empty directory should work' \
+ 'git-submodule clone'
+
+test_expect_success 'submodule HEAD should match rev1' '
+ head=$(cd lib && git-rev-parse HEAD) &&
+ test "$head" = "$rev1"
+'
test_expect_success 'status is "up-to-date" after init' \
'git-submodule status | grep "^ $rev1"'
@@ -66,15 +91,13 @@ test_expect_success 'status is "modified" after submodule commit' \
test_expect_success 'the --cached sha1 should be rev1' \
'git-submodule --cached status | grep "^\+$rev1"'
-test_expect_failure 'git-diff --exit-code reports local modifications' \
- 'git-diff --exit-code'
-
-test_expect_success 'update should checkout the correct commit' \
+test_expect_success 'after update, there should be no local modifications' \
'git-submodule update && git-diff --exit-code'
-head=$(cd lib && git-rev-parse HEAD)
-test_expect_success 'submodule HEAD should match rev1' \
- 'test "$head" = "$rev1"'
+test_expect_success 'submodule HEAD should match rev1' '
+ head=$(cd lib && git-rev-parse HEAD) &&
+ test "$head" = "$rev1"
+'
test_expect_success 'status is "up-to-date" after update' \
'git-submodule status | grep "^ $rev1"'
--
1.5.2.74.g6b2d
^ permalink raw reply related
* [PATCH] gitweb: handle non UTF-8 text
From: Martin Koegler @ 2007-05-28 20:47 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git, Martin Koegler
gitweb assumes, that everything is in UTF-8. If a text contains invalid
UTF-8 character sequences, the text must be in a different encoding.
This patch interprets such a text as latin1.
Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
---
For correct UTF-8, the patch does not change anything.
If commit/blob/... is not in UTF-8, it displays the text
with a very high probability correct.
As git itself is not aware of any encoding, I know no better
possibility to handle non UTF-8 text in gitweb.
gitweb/gitweb.perl | 27 +++++++++++++++++----------
1 files changed, 17 insertions(+), 10 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index c3921cb..dfd564d 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -621,12 +621,19 @@ sub esc_url {
return $str;
}
+sub my_decode_utf8 {
+ my $str = shift;
+ my $res;
+ eval { $res = decode_utf8 ($str, 1); };
+ return $res || decode('latin1', $str);
+}
+
# replace invalid utf8 character with SUBSTITUTION sequence
sub esc_html ($;%) {
my $str = shift;
my %opts = @_;
- $str = decode_utf8($str);
+ $str = my_decode_utf8($str);
$str = $cgi->escapeHTML($str);
if ($opts{'-nbsp'}) {
$str =~ s/ / /g;
@@ -640,7 +647,7 @@ sub esc_path {
my $str = shift;
my %opts = @_;
- $str = decode_utf8($str);
+ $str = my_decode_utf8($str);
$str = $cgi->escapeHTML($str);
if ($opts{'-nbsp'}) {
$str =~ s/ / /g;
@@ -925,7 +932,7 @@ sub format_subject_html {
if (length($short) < length($long)) {
return $cgi->a({-href => $href, -class => "list subject",
- -title => decode_utf8($long)},
+ -title => my_decode_utf8($long)},
esc_html($short) . $extra);
} else {
return $cgi->a({-href => $href, -class => "list subject"},
@@ -1239,7 +1246,7 @@ sub git_get_projects_list {
if (check_export_ok("$projectroot/$path")) {
my $pr = {
path => $path,
- owner => decode_utf8($owner),
+ owner => my_decode_utf8($owner),
};
push @list, $pr;
(my $forks_path = $path) =~ s/\.git$//;
@@ -1269,7 +1276,7 @@ sub git_get_project_owner {
$pr = unescape($pr);
$ow = unescape($ow);
if ($pr eq $project) {
- $owner = decode_utf8($ow);
+ $owner = my_decode_utf8($ow);
last;
}
}
@@ -1759,7 +1766,7 @@ sub get_file_owner {
}
my $owner = $gcos;
$owner =~ s/[,;].*$//;
- return decode_utf8($owner);
+ return my_decode_utf8($owner);
}
## ......................................................................
@@ -1842,7 +1849,7 @@ sub git_header_html {
my $title = "$site_name";
if (defined $project) {
- $title .= " - " . decode_utf8($project);
+ $title .= " - " . my_decode_utf8($project);
if (defined $action) {
$title .= "/$action";
if (defined $file_name) {
@@ -2116,7 +2123,7 @@ sub git_print_page_path {
print "<div class=\"page_path\">";
print $cgi->a({-href => href(action=>"tree", hash_base=>$hb),
- -title => 'tree root'}, decode_utf8("[$project]"));
+ -title => 'tree root'}, my_decode_utf8("[$project]"));
print " / ";
if (defined $name) {
my @dirname = split '/', $name;
@@ -2936,7 +2943,7 @@ sub git_project_list_body {
($pr->{'age'}, $pr->{'age_string'}) = @aa;
if (!defined $pr->{'descr'}) {
my $descr = git_get_project_description($pr->{'path'}) || "";
- $pr->{'descr_long'} = decode_utf8($descr);
+ $pr->{'descr_long'} = my_decode_utf8($descr);
$pr->{'descr'} = chop_str($descr, 25, 5);
}
if (!defined $pr->{'owner'}) {
@@ -3981,7 +3988,7 @@ sub git_snapshot {
my $git = git_cmd_str();
my $name = $project;
$name =~ s/\047/\047\\\047\047/g;
- my $filename = decode_utf8(basename($project));
+ my $filename = my_decode_utf8(basename($project));
my $cmd;
if ($suffix eq 'zip') {
$filename .= "-$hash.$suffix";
--
1.5.2.846.g9a144
^ permalink raw reply related
* Re: [PATCH 00/15] git-note: A mechanisim for providing free-form after-the-fact annotations on commits
From: Junio C Hamano @ 2007-05-28 20:45 UTC (permalink / raw)
To: Johan Herland; +Cc: Linus Torvalds, git
In-Reply-To: <200705281254.23297.johan@herland.net>
Johan Herland <johan@herland.net> writes:
> And while we're on the subject of changing the tag object, I'd like for
> the "tag" header (the one holding the tag name) to become optional.
> When doing my ref <-> name trick above, I conveniently forgot this little
> bugger. Basically the only reason for this one to exist is to include the
> name of the tag in the data passed to gpg for signing. This is of course
> necesssary in order to make renaming a signed tag impossible. (Allowing
> renaming would make it possible to replace it with a malicious tag with
> the original name.) Therefore the "tag" header must be mandatory for
> signed tags. But for all other tags (including notes) this header is
> pretty much useless.
This brings up an interesting point, by the way.
Currently when you say
$ git show v2.6.22
we "dwim_ref" v2.6.22. That is, we try to see if ".git/v2.6.22"
is there (it isn't), check ".git/refs/v2.6.22" (no, again), then
happily find that ".git/refs/tags/v2.6.22" is there and use it.
During this process we _never_ check if the tag object pointed
at by the ref v2.6.22 has tagname v2.6.22 at all.
I do not think it is a problem at all that we do not use "tag"
name during the look-up process. However it would be a good
idea to add logic to fsck to warn upon inconsistencis (perhaps
by mistake) between refname and tag's true name.
The check would say something like:
If an annotated (signed or unsigned) tag has a "tag"
line to give it the official $name, and if it is pointed
at by a ref, the refname must end with "/$name".
Otherwise we warn.
Trivially, the above rule says that having v2.6.22 tag under
refs/tags/v2.6.20 is a mistake we would want to be warned upon.
The "/$name" part is to support merging a separate project into
another project after the fact better. Shawn graciously names
his tags in git-gui project "gitgui-0.7.1", to avoid tag name
collisions with "v1.5.0" style tag names in git.git project, but
we could have stored his v0.7.1 tag as refs/tags/gitgui/v0.7.1
to differenciate the tag namespace (this needs an update to the
"tag following" code in git-fetch with a new configuration).
^ permalink raw reply
* Re: [PATCH 00/15] git-note: A mechanisim for providing free-form after-the-fact annotations on commits
From: Junio C Hamano @ 2007-05-28 20:45 UTC (permalink / raw)
To: Johan Herland; +Cc: Linus Torvalds, git
In-Reply-To: <200705281948.27329.johan@herland.net>
Johan Herland <johan@herland.net> writes:
> On Monday 28 May 2007, Linus Torvalds wrote:
>>
>> On Mon, 28 May 2007, Johan Herland wrote:
>> >
>> > I still don't see what makes note objects inherently more expensive than
>> > commit objects. Except for the refs, of course, but we're getting rid
>> > of those (at least replacing them with a more efficient reverse mapping).
>>
>> It's exactly the refs that I worry about.
>>
>> Anything that needs to read in all notes at startup is going to be _slow_.
>>
>> In contrast, commits we read when (and only when) we need them.
>
> Ok. But the reverse mapping will help with this, won't it?
> We'll look up the interesting commits and find their associated
> note objects directly.
The issue Linus brought up worries me, too.
The "efficient reverse mapping" is still handwaving at this
stage. What it needs to do is an equivalent to your
implementation with "refs/notes/<a dir per commit>/<note>". The
"efficient" one might do a flat file that says "notee note" per
line sorted by notee, or it might use BDB or sqlite, but the
amount of the data and complexity of the look-up is really the
same. A handful notes per each commit in the history (I think
Linus's "Acked-by after the fact" example a very sensible thing
to want from this subsystem).
I am not saying that it is impossible to make the set-up cost
for the "efficient lookup" almost zero, and to make it lazy and
on-demand. The concern above just adds one design constraints
to that "efficient reverse mapping" code yet to come.
^ permalink raw reply
* Re: [PATCH] Don't ignore write failure from git-diff, git-log, etc.
From: Junio C Hamano @ 2007-05-28 20:44 UTC (permalink / raw)
To: Jim Meyering; +Cc: Marco Roeland, git
In-Reply-To: <87646cx13d.fsf@rho.meyering.net>
Jim Meyering <jim@meyering.net> writes:
> Subject: [PATCH] Don't ignore write failure from git-diff, git-log, etc.
> ...
> You can demonstrate this with git's own --version output, too:
> (but git --help detects the failure without this patch)
>
> $ ./git --version > /dev/full
> fatal: write failure on standard output: No space left on device
> [Exit 128]
>
> Note that the fcntl test (for whether the fileno may be closed) is
> required in order to avoid EBADF upon closing an already-closed stdout,
> as would happen for each git command that already closes stdout; I think
> update-index was the one I noticed in the failure of t5400, before I
> added that test.
>
> Also, to be consistent, don't ignore EPIPE write failures.
>
> Signed-off-by: Jim Meyering <jim@meyering.net>
I do not think anybody has much objection about the change to
handle_internal_command() in git.c you made. Earlier we relied
on exit(3) to close still open filehandles (while ignoring
errors), and you made the close explicit in order to detect
errors.
But "to be consistent" above is not a very good justification.
In the presense of shells that give "annoying" behaviour (we
have to remember that not everybody have enough privilege,
expertise, or motivation to update /bin/sh or install their own
copy in $HOME/bin/sh), "EPIPE is different from other errors" is
a more practical point of view, I'd have to say. IOW, it is not
clear if it is a good thing "to be consistent" to begin with.
I would have appreciated if this were two separate patches. I
think the EPIPE one is an independent issue. We could even make
it a configuration option to ignore EPIPE ;-)
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox