* Cloning remote HTTP repository: Can only see 'master' branch
From: Michael Tyson @ 2013-01-29 5:54 UTC (permalink / raw)
To: git
Hello!
I've a readonly git repository that I'm hosting via HTTP (a bare git repository located within the appropriate directory on the server). I push to it via my own SSH account (local repository with a remote pointing to the ssh:// URL).
This has all worked fine so far - I push via ssh, and others can clone and pull via the HTTP URL.
I've recently added a branch - "beta" - which pushed just fine, but now cloning via the HTTP URL doesn't seem to show the new branch - just master:
$ git clone http://server.tld/path/to/repository.git repository
Cloning into 'repository'...
$ cd repository
$ git branch -a
* master
remotes/origin/HEAD -> origin/master
remotes/origin/master
Cloning via the readwrite ssh URL works fine though:
$ git clone ssh://user@server.tld:www/path/to/repository.git repository
Cloning into 'repository'...
stdin: is not a tty
remote: Counting objects: 225, done.
remote: Compressing objects: 100% (196/196), done.
remote: Total 225 (delta 109), reused 77 (delta 25)
Receiving objects: 100% (225/225), 9.55 MiB | 295 KiB/s, done.
Resolving deltas: 100% (109/109), done.
$ cd repository
$ git branch -a
* master
remotes/origin/HEAD -> origin/master
remotes/origin/beta
remotes/origin/master
I've tried 'git clone --mirror' just in case, but this just resulted in a bare repository with only the 'master' branch, still.
Anyone have any ideas about what I'm doing wrong?
Cheers,
Michael
^ permalink raw reply
* Re: [RFC] The design of new pathspec features
From: Duy Nguyen @ 2013-01-29 6:13 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vd2wotuxm.fsf@alter.siamese.dyndns.org>
On Tue, Jan 29, 2013 at 12:33 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Have you considered if it may be helpful to have a :(literal) magic
> (or any magic in general) that applies only to the first N
> characters of the pathspec pattern?
Not user-driven. But the prefix part is :(literal)-applied. :(glob) is
currently implemented this way, using nowildcard_len as you mentioned.
:(icase) is more complicated and does not follow yet.
> I personally think we do not need to support something like this:
>
> prefix=$(git rev-parse --show-prefix)
> n=${#prefix}
> pathspec=":(literal-$n)$prefix$1"
>
> but other aspiring Porcelain script writers may disagree and would
> want to have it. We can always solve it by giving them an easy and
> uniform way to get the glob-quoted version of prefix to solve this
> particular issue, i.e.
>
> prefixq=$(git rev-parse --show-prefix-glob-quoted)
> pathspec="$prefixq$1"
>
> but magic that applies only to a substring may have other uses.
Yeah, that simplifies things. Supporting applying magic over just
parts of the pathspec pattern sounds complex. Just a small
modification. That rev-parse needs to look at "$1" as well. If
:(literal) is already specified, glob quoting will backfire. The user
script can deal with that, but it's harder (e.g. parsing magic from
scripts and deal with magic combination) than letting rev-parse does
it.
I've done some form of this already, for supporting add--interactive.
git-add prefixes the pathspec but keeps all the magic in place, before
passing pathspec to add--interactive. But I missed the quoting point
you mentioned above. I probably need de-quoting the prefix as well.
Many optimizations stop short at the sign of any glob symbols,
including backslash. This could be a new task for wildmatch.
--
Duy
^ permalink raw reply
* Re: [RFC] The design of new pathspec features
From: Junio C Hamano @ 2013-01-29 6:31 UTC (permalink / raw)
To: Duy Nguyen; +Cc: git
In-Reply-To: <CACsJy8DsKFfGoPUt5b=3MuSmZ2z2Z5vJ20D_kqEowAUc8QTryA@mail.gmail.com>
Duy Nguyen <pclouds@gmail.com> writes:
>> prefixq=$(git rev-parse --show-prefix-glob-quoted)
>> pathspec="$prefixq$1"
>>
>> but magic that applies only to a substring may have other uses.
>
> Yeah, that simplifies things. Supporting applying magic over just
> parts of the pathspec pattern sounds complex. Just a small
> modification. That rev-parse needs to look at "$1" as well.
Makes sense. More like
prefix=$(git rev-parse --show-prefix)
test -z "$prefix" || cd $(git rev-parse --show-cdup)
...
pathspec=$(git rev-parse --prefix-pathspec "$prefix" "$1")
git ls-files "$pathspec"
which may take prefix="D*cumentati*n" (from the previous example),
and the user may have given ":(icase)hell*.txt" to "$1". Ideally
the scriptors shouldn't have to worry about how to add the prefix to
the user supplied pathspec.
^ permalink raw reply
* Re: [PATCH] l10n: de.po: translate 11 new messages
From: Ralf Thielow @ 2013-01-29 7:42 UTC (permalink / raw)
To: Philip Oakley; +Cc: Thomas Rast, jk, stimming, Git List
In-Reply-To: <AE5DE037F2E44705AC6716EEB19948B1@PhilipOakley>
2013/1/28 Philip Oakley <philipoakley@iee.org>:
> From: "Ralf Thielow" <ralf.thielow@gmail.com>
> Sent: Monday, January 28, 2013 5:55 PM
>
>> On Mon, Jan 28, 2013 at 11:33:09AM +0100, Thomas Rast wrote:
>>>
>>> Ralf Thielow <ralf.thielow@gmail.com> writes:
>>>
>>> > #: builtin/reset.c:275
>>> > -#, fuzzy, c-format
>>> > +#, c-format
>>> > msgid "Failed to resolve '%s' as a valid revision."
>>> > -msgstr "Konnte '%s' nicht als gültige Referenz auflösen."
>>> > +msgstr "Konnte '%s' nicht als gültige Revision auflösen."
>>>
>>> You don't have "revision" in the glossary[1] yet. Wouldn't it be
>>> appropriate to treat it as "commit", and translate as "Version" to
>>> avoid
>>> introducing yet another term?
>>>
>>> Or am I missing some subtle distinction between commit and revision?
>>>
>>
>> I don't think there's a distinction.
>
>
> It was a problem I had http://stackoverflow.com/a/11792712/717355
> answered as:
>
> See "SPECIFYING REVISIONS" of git rev-parse:
>
> A revision parameter <rev> typically, but not necessarily, names a commit
> object.
> It uses what is called an extended SHA1 syntax, [and includes] various ways
> to spell object names.
>
> It had me confused for a while.
>
Thanks. So it would be a mistake to translate "revision" and "commit"
with the same word. This would hide the special meaning of revision and
the user wouldn't see that there's a difference.
I'll amend the commit which unifys the translation of "revision" and "commit"
in the way that "revision" will be translated as "Revision" in the whole file.
I'll also update the glossary.
>> Since we've already translated
>> "revision" as "Revision" in a couple of other messages, I'll make a
>> new "s/Revision/Version" commit on top.
>>
>>> Since it's only a single nit, feel free to add my ack when you
>>> reroll:
>>>
>>> Acked-by: Thomas Rast <trast@inf.ethz.ch>
>>>
>>>
>>> [1] https://github.com/ralfth/git-po-de/wiki/Glossary
>>>
>>> --
>>> Thomas Rast
>>> trast@{inf,student}.ethz.ch
>>
>> --
>
> Philip Oakley
^ permalink raw reply
* Why Same Files show up with different Statuses in Git?
From: Matthew Johnson @ 2013-01-29 7:38 UTC (permalink / raw)
To: git
First, some background. I have one hard disk separated into two
partitions: one for the version of Windows 7 that shipped on this
rather new Thinkpad (Windows 7 Professional SP1), the other for Fedora
17 (which I installed and keep up-to-date). Of course, it is rather
easy to access the Windows partition from the F17 partition, which I
have been doing with no noticeable problems: the File Explorer
equivalent, Dolphin (the KDE equivalent) will automount the Windows
partition as a filesystem under /media, I only have to enter the
password for 'su'.
I don't think it makes a difference, but for completeness I provide
the options with which I find it mounted:
/dev/sda2 on /media/Windows7_OS type fuseblk
(rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096).
Anyway, the problem is this: the very same files, e.g.
/media/Windows7_OS/cygwin/home/Matthew
Johnson/MrEd/mr.ed/lesson1/images/home-icon.jpg show up as 'modified'
when I type "git status" (in the mr.ed/ dir) using the F17 git client,
but do not show up at all when I type "git status" under the Cygwin
client. This problem is not on all files, only a few, predominantly in
two directories.
Under Windows (cygwin) "git status" shows nothing to commit, only
untracked files, which is what I expect; only under F17 do I get
several modified files -- and these have old dates (e.g. Nov 27).
Yes, that is another important note: I am running Cygwin on the
Windows partition.
The versions of things under the F17 partition are:
Linux 3.7.3-101.fc17.x86_64 x86_64
System: Fedora release 17 (Beefy Miracle)
KDE: 4.9.5
git --version
git version 1.7.11.7
Under Cygwin, the git version is 1.7.9.
Finally, I did check the git configuration to make sure the same
user.name and user.email are set in both. And in both cases, I made
mr.ed the current directory before executing "git status". Shouldn't
that be enough? Is there anything else that must match in the git
config?
^ permalink raw reply
* Re: [PATCH 0/2] optimizing pack access on "read only" fetch repos
From: Shawn Pearce @ 2013-01-29 8:06 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jeff King, git
In-Reply-To: <7vlibfxhit.fsf@alter.siamese.dyndns.org>
On Sat, Jan 26, 2013 at 10:32 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jeff King <peff@peff.net> writes:
>
>> This is a repost from here:
>>
>> http://thread.gmane.org/gmane.comp.version-control.git/211176
>>
>> which got no response initially. Basically the issue is that read-only
>> repos (e.g., a CI server) whose workflow is something like:
>>
>> git fetch $some_branch &&
>> git checkout -f $some_branch &&
>> make test
>>
>> will never run git-gc, and will accumulate a bunch of small packs and
>> loose objects, leading to poor performance.
...
> I also wonder if we would be helped by another "repack" mode that
> coalesces small packs into a single one with minimum overhead, and
> run that often from "gc --auto", so that we do not end up having to
> have 50 packfiles.
Yes. This does help....
> When we have 2 or more small and young packs, we could:
>
> - iterate over idx files for these packs to enumerate the objects
> to be packed, replacing read_object_list_from_stdin() step;
>
> - always choose to copy the data we have in these existing packs,
> instead of doing a full prepare_pack(); and
>
> - use the order the objects appear in the original packs, bypassing
> compute_write_order().
Hmm, sounds familiar. Seems like its what we do in JGit for Android. :-)
^ permalink raw reply
* Re: [PATCH 1/2] upload-pack: avoid parsing objects during ref advertisement
From: Shawn Pearce @ 2013-01-29 8:10 UTC (permalink / raw)
To: Jeff King; +Cc: Junio C Hamano, git, git-dev
In-Reply-To: <20120106191740.GA12903@sigill.intra.peff.net>
On Fri, Jan 6, 2012 at 11:17 AM, Jeff King <peff@peff.net> wrote:
> When we advertise a ref, the first thing we do is parse the
> pointed-to object. This gives us two things:
...
> The downside is that we are no longer verifying objects that
> we advertise by fully parsing them (however, we do still
> know we actually have them, because sha1_object_info must
> find them to get the type). While we might fail to detect a
> corrupt object here, if the client actually fetches the
> object, we will parse (and verify) it then.
As you explain, its not necessary to verify during the advertisement
phase. Its fine to delay verification to when a client actually
"wants" the object.
> On a repository with 120K refs, the advertisement portion of
> upload-pack goes from ~3.4s to 3.2s (the failure to speed up
> more is largely due to the fact that most of these refs are
> tags, which need dereferenced to find the tag destination
> anyway).
Why aren't we using the peeled information from the packed-refs file?
JGit does this and it saves a lot of time on advertisements from a
well packed repository.
^ permalink raw reply
* Re: [PATCH 1/2] upload-pack: avoid parsing objects during ref advertisement
From: Jeff King @ 2013-01-29 8:14 UTC (permalink / raw)
To: Shawn Pearce; +Cc: Junio C Hamano, git, git-dev
In-Reply-To: <CAJo=hJubUF2mmkzXU77kQ5uj1cbBZrnLcw2u736wMqxhs39jdQ@mail.gmail.com>
On Tue, Jan 29, 2013 at 12:10:59AM -0800, Shawn O. Pearce wrote:
> > On a repository with 120K refs, the advertisement portion of
> > upload-pack goes from ~3.4s to 3.2s (the failure to speed up
> > more is largely due to the fact that most of these refs are
> > tags, which need dereferenced to find the tag destination
> > anyway).
>
> Why aren't we using the peeled information from the packed-refs file?
> JGit does this and it saves a lot of time on advertisements from a
> well packed repository.
The patch you are replying to is a year old. Since then, I did:
435c833 (upload-pack: use peel_ref for ref advertisements, 2012-10-04)
-Peff
^ permalink raw reply
* Re: Cloning remote HTTP repository: Can only see 'master' branch
From: Jeff King @ 2013-01-29 8:23 UTC (permalink / raw)
To: Michael Tyson; +Cc: git
In-Reply-To: <83605E83-1110-449B-969D-AB43158376EA@atastypixel.com>
On Tue, Jan 29, 2013 at 04:54:13PM +1100, Michael Tyson wrote:
> I've a readonly git repository that I'm hosting via HTTP (a bare git
> repository located within the appropriate directory on the server). I
> push to it via my own SSH account (local repository with a remote
> pointing to the ssh:// URL).
>
> This has all worked fine so far - I push via ssh, and others can clone
> and pull via the HTTP URL.
>
> I've recently added a branch - "beta" - which pushed just fine, but
> now cloning via the HTTP URL doesn't seem to show the new branch -
> just master:
If you are using the "dumb" http protocol (i.e., the web server knows
nothing about git, and just serves the repo files), you need to run "git
update-server-info" after each push in order to update the static file
that tells the git client about each ref. You can have git do it
automatically for you by setting receive.updateServerInfo in the server
repo's config.
If the server is yours to control, consider setting up the "smart" http
protocol, as it is much more efficient. Details are in "git help
http-backend".
-Peff
^ permalink raw reply
* Re: [PATCH 0/2] optimizing pack access on "read only" fetch repos
From: Jeff King @ 2013-01-29 8:29 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Shawn O. Pearce, git
In-Reply-To: <7vlibfxhit.fsf@alter.siamese.dyndns.org>
On Sat, Jan 26, 2013 at 10:32:42PM -0800, Junio C Hamano wrote:
> Both makes sense to me.
>
> I also wonder if we would be helped by another "repack" mode that
> coalesces small packs into a single one with minimum overhead, and
> run that often from "gc --auto", so that we do not end up having to
> have 50 packfiles.
>
> When we have 2 or more small and young packs, we could:
>
> - iterate over idx files for these packs to enumerate the objects
> to be packed, replacing read_object_list_from_stdin() step;
>
> - always choose to copy the data we have in these existing packs,
> instead of doing a full prepare_pack(); and
>
> - use the order the objects appear in the original packs, bypassing
> compute_write_order().
I'm not sure. If I understand you correctly, it would basically just be
concatenating packs without trying to do delta compression between the
objects which are ending up in the same pack. So it would save us from
having to do (up to) 50 binary searches to find an object in a pack, but
would not actually save us much space.
I would be interested to see the timing on how quick it is compared to a
real repack, as the I/O that happens during a repack is non-trivial
(although if you are leaving aside the big "main" pack, then it is
probably not bad).
But how do these somewhat mediocre concatenated packs get turned into
real packs? Pack-objects does not consider deltas between objects in the
same pack. And when would you decide to make a real pack? How do you
know you have 50 young and small packs, and not 50 mediocre coalesced
packs?
-Peff
^ permalink raw reply
* [PATCH] git p4: chdir resolves symlinks only for relative paths
From: Miklós Fazekas @ 2013-01-29 8:37 UTC (permalink / raw)
To: git; +Cc: Gary Gibbons
In-Reply-To: <CAAMmcSSvrsZqEVf68Nrqy_ZG6r5ESKhtx7JdQ7vzypkZ3gOFnA@mail.gmail.com>
[resending as plain text]
If a p4 client is configured to /p/foo which is a symlink
to /vol/bar/projects/foo, then resolving symlink, which
is done by git-p4's chdir will confuse p4: "Path
/vol/bar/projects/foo/... is not under client root /p/foo"
While AltRoots in p4 client specification can be used as a
workaround on p4 side, git-p4 should not resolve symlinks
in client paths.
chdir(dir) uses os.getcwd() after os.chdir(dir) to resolve
relative paths, but as a side effect it resolves symlinks
too. Now it checks if the dir is relative before resolving.
Signed-off-by: Miklós Fazekas <mfazekas@szemafor.com>
---
git-p4.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/git-p4.py b/git-p4.py
index 2da5649..5d74649 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -64,7 +64,10 @@ def chdir(dir):
# not using the shell, we have to set it ourselves. This path could
# be relative, so go there first, then figure out where we ended up.
os.chdir(dir)
- os.environ['PWD'] = os.getcwd()
+ if os.path.isabs(dir):
+ os.environ['PWD'] = dir
+ else:
+ os.environ['PWD'] = os.getcwd()
def die(msg):
if verbose:
--
1.7.10.2 (Apple Git-33)
^ permalink raw reply related
* [PATCH/RFC 0/6] commit caching
From: Jeff King @ 2013-01-29 9:14 UTC (permalink / raw)
To: git; +Cc: Duy Nguyen, Shawn O. Pearce
This is the cleaned-up version of the commit caching patches I mentioned
here:
http://article.gmane.org/gmane.comp.version-control.git/212329
The basic idea is to generate a cache file that sits alongside a
packfile and contains the timestamp, tree, and parents in a more compact
and easy-to-access format.
The timings from this one are roughly similar to what I posted earlier.
Unlike the earlier version, this one keeps the data for a single commit
together for better cache locality (though I don't think it made a big
difference in my tests, since my cold-cache timing test ends up touching
every commit anyway). The short of it is that for an extra 31M of disk
space (~4%), I get a warm-cache speedup for "git rev-list --all" of
~4.2s to ~0.66s.
The big thing it does not (yet) do is use offsets to reference sha1s, as
Shawn suggested. This would potentially drop the on-disk size from 84
bytes to 16 bytes per commit (or about 6M total for linux.git).
Coupled with using compression level 0 for trees (which do not compress
well at all, and yield only a 2% increase in size when left
uncompressed), my "git rev-list --objects --all" time drops from ~40s to
~25s. Perf reveals that we're spending most of the remaining time in
lookup_object. I've spent a fair bit of time trying to optimize that,
but with no luck; I think it's fairly close to optimal. The problem is
just that we call it a very large number of times, since it is the
mechanism by which we recognize that we have already processed each
sha1.
[1/6]: csum-file: make sha1write const-correct
[2/6]: strbuf: add string-chomping functions
[3/6]: introduce pack metadata cache files
[4/6]: introduce a commit metapack
[5/6]: add git-metapack command
[6/6]: commit: look up commit info in metapack
-Peff
^ permalink raw reply
* [PATCH 1/6] csum-file: make sha1write const-correct
From: Jeff King @ 2013-01-29 9:15 UTC (permalink / raw)
To: git; +Cc: Duy Nguyen, Shawn O. Pearce
In-Reply-To: <20130129091434.GA6975@sigill.intra.peff.net>
We do not modify the buffer we are asked to write; mark it
with const so that callers with const buffers do not get
unnecessary complaints from the compiler.
Signed-off-by: Jeff King <peff@peff.net>
---
csum-file.c | 6 +++---
csum-file.h | 2 +-
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/csum-file.c b/csum-file.c
index 53f5375..465971c 100644
--- a/csum-file.c
+++ b/csum-file.c
@@ -11,7 +11,7 @@
#include "progress.h"
#include "csum-file.h"
-static void flush(struct sha1file *f, void *buf, unsigned int count)
+static void flush(struct sha1file *f, const void *buf, unsigned int count)
{
if (0 <= f->check_fd && count) {
unsigned char check_buffer[8192];
@@ -86,13 +86,13 @@ int sha1write(struct sha1file *f, void *buf, unsigned int count)
return fd;
}
-int sha1write(struct sha1file *f, void *buf, unsigned int count)
+int sha1write(struct sha1file *f, const void *buf, unsigned int count)
{
while (count) {
unsigned offset = f->offset;
unsigned left = sizeof(f->buffer) - offset;
unsigned nr = count > left ? left : count;
- void *data;
+ const void *data;
if (f->do_crc)
f->crc32 = crc32(f->crc32, buf, nr);
diff --git a/csum-file.h b/csum-file.h
index 3b540bd..9dedb03 100644
--- a/csum-file.h
+++ b/csum-file.h
@@ -34,7 +34,7 @@ extern int sha1close(struct sha1file *, unsigned char *, unsigned int);
extern struct sha1file *sha1fd_check(const char *name);
extern struct sha1file *sha1fd_throughput(int fd, const char *name, struct progress *tp);
extern int sha1close(struct sha1file *, unsigned char *, unsigned int);
-extern int sha1write(struct sha1file *, void *, unsigned int);
+extern int sha1write(struct sha1file *, const void *, unsigned int);
extern void sha1flush(struct sha1file *f);
extern void crc32_begin(struct sha1file *);
extern uint32_t crc32_end(struct sha1file *);
--
1.8.0.2.16.g72e2fc9
^ permalink raw reply related
* [PATCH 2/6] strbuf: add string-chomping functions
From: Jeff King @ 2013-01-29 9:15 UTC (permalink / raw)
To: git; +Cc: Duy Nguyen, Shawn O. Pearce
In-Reply-To: <20130129091434.GA6975@sigill.intra.peff.net>
Sometimes it is handy to cut a trailing string off the end
of a strbuf (e.g., a file extension). These helper functions
make it a one-liner.
Signed-off-by: Jeff King <peff@peff.net>
---
strbuf.c | 11 +++++++++++
strbuf.h | 2 ++
2 files changed, 13 insertions(+)
diff --git a/strbuf.c b/strbuf.c
index 9a373be..8199ced 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -106,6 +106,17 @@ void strbuf_ltrim(struct strbuf *sb)
sb->buf[sb->len] = '\0';
}
+void strbuf_chompmem(struct strbuf *sb, const void *data, size_t len)
+{
+ if (sb->len >= len && !memcmp(data, sb->buf + sb->len - len, len))
+ strbuf_setlen(sb, sb->len - len);
+}
+
+void strbuf_chompstr(struct strbuf *sb, const char *str)
+{
+ strbuf_chompmem(sb, str, strlen(str));
+}
+
struct strbuf **strbuf_split_buf(const char *str, size_t slen,
int terminator, int max)
{
diff --git a/strbuf.h b/strbuf.h
index ecae4e2..3aeb815 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -42,6 +42,8 @@ extern void strbuf_ltrim(struct strbuf *);
extern void strbuf_trim(struct strbuf *);
extern void strbuf_rtrim(struct strbuf *);
extern void strbuf_ltrim(struct strbuf *);
+extern void strbuf_chompmem(struct strbuf *, const void *, size_t);
+extern void strbuf_chompstr(struct strbuf *, const char *);
extern int strbuf_cmp(const struct strbuf *, const struct strbuf *);
/*
--
1.8.0.2.16.g72e2fc9
^ permalink raw reply related
* [PATCH 3/6] introduce pack metadata cache files
From: Jeff King @ 2013-01-29 9:15 UTC (permalink / raw)
To: git; +Cc: Duy Nguyen, Shawn O. Pearce
In-Reply-To: <20130129091434.GA6975@sigill.intra.peff.net>
The on-disk packfile format is nicely compact, but it does
not always provide the fastest format for looking up
information. This patch introduces the concept of
"metapacks", optional metadata files which can live
alongside packs and represent their data in different ways.
This can allow space-time tradeoffs in accessing certain
object data.
Such space-time tradeoffs have traditionally gone into the
.idx file (e.g., the fact that we can quickly find an
object's offset in the packfile is due to the index). In
theory, cached data could also go into the .idx file.
However, keeping it in a separate file makes backwards
compatibility much simpler. Older versions of git can simply
ignore the extra files and use the existing methods for
accessing object data. This also makes metapacks optional,
so you can easily tune the space-time tradeoff on a per-repo
basis.
TODO: document on-disk format in Documentation/technical
TODO: document api
Signed-off-by: Jeff King <peff@peff.net>
---
Makefile | 2 +
metapack.c | 158 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
metapack.h | 42 ++++++++++++++++
3 files changed, 202 insertions(+)
create mode 100644 metapack.c
create mode 100644 metapack.h
diff --git a/Makefile b/Makefile
index 731b6a8..3e4ae1b 100644
--- a/Makefile
+++ b/Makefile
@@ -660,6 +660,7 @@ LIB_H += mergesort.h
LIB_H += merge-blobs.h
LIB_H += merge-recursive.h
LIB_H += mergesort.h
+LIB_H += metapack.h
LIB_H += notes-cache.h
LIB_H += notes-merge.h
LIB_H += notes.h
@@ -778,6 +779,7 @@ LIB_OBJS += mergesort.o
LIB_OBJS += merge-blobs.o
LIB_OBJS += merge-recursive.o
LIB_OBJS += mergesort.o
+LIB_OBJS += metapack.o
LIB_OBJS += name-hash.o
LIB_OBJS += notes.o
LIB_OBJS += notes-cache.o
diff --git a/metapack.c b/metapack.c
new file mode 100644
index 0000000..c859c95
--- /dev/null
+++ b/metapack.c
@@ -0,0 +1,158 @@
+#include "cache.h"
+#include "metapack.h"
+#include "csum-file.h"
+
+static struct sha1file *create_meta_tmp(void)
+{
+ char tmp[PATH_MAX];
+ int fd;
+
+ fd = odb_mkstemp(tmp, sizeof(tmp), "pack/tmp_meta_XXXXXX");
+ return sha1fd(fd, xstrdup(tmp));
+}
+
+static void write_meta_header(struct metapack_writer *mw, const char *id,
+ uint32_t version)
+{
+ version = htonl(version);
+
+ sha1write(mw->out, "META", 4);
+ sha1write(mw->out, "\0\0\0\1", 4);
+ sha1write(mw->out, mw->pack->sha1, 20);
+ sha1write(mw->out, id, 4);
+ sha1write(mw->out, &version, 4);
+}
+
+void metapack_writer_init(struct metapack_writer *mw,
+ const char *pack_idx,
+ const char *name,
+ int version)
+{
+ struct strbuf path = STRBUF_INIT;
+
+ memset(mw, 0, sizeof(*mw));
+
+ mw->pack = add_packed_git(pack_idx, strlen(pack_idx), 1);
+ if (!mw->pack || open_pack_index(mw->pack))
+ die("unable to open packfile '%s'", pack_idx);
+
+ strbuf_addstr(&path, pack_idx);
+ strbuf_chompstr(&path, ".idx");
+ strbuf_addch(&path, '.');
+ strbuf_addstr(&path, name);
+ mw->path = strbuf_detach(&path, NULL);
+
+ mw->out = create_meta_tmp();
+ write_meta_header(mw, name, version);
+}
+
+void metapack_writer_finish(struct metapack_writer *mw)
+{
+ const char *tmp = mw->out->name;
+
+ sha1close(mw->out, NULL, CSUM_FSYNC);
+ if (rename(tmp, mw->path))
+ die_errno("unable to rename temporary metapack file");
+
+ close_pack_index(mw->pack);
+ free(mw->pack);
+ free(mw->path);
+ free((char *)tmp);
+}
+
+void metapack_writer_add(struct metapack_writer *mw, const void *data, int len)
+{
+ sha1write(mw->out, data, len);
+}
+
+void metapack_writer_add_uint32(struct metapack_writer *mw, uint32_t v)
+{
+ v = htonl(v);
+ metapack_writer_add(mw, &v, 4);
+}
+
+void metapack_writer_foreach(struct metapack_writer *mw,
+ metapack_writer_each_fn cb,
+ void *data)
+{
+ const unsigned char *sha1;
+ uint32_t i = 0;
+
+ /*
+ * We'll feed these to the callback in sorted order, since that is the
+ * order that they are stored in the .idx file.
+ */
+ while ((sha1 = nth_packed_object_sha1(mw->pack, i++)))
+ cb(mw, sha1, data);
+}
+
+int metapack_init(struct metapack *m,
+ struct packed_git *pack,
+ const char *name,
+ uint32_t *version)
+{
+ struct strbuf path = STRBUF_INIT;
+ int fd;
+ struct stat st;
+
+ memset(m, 0, sizeof(*m));
+
+ strbuf_addstr(&path, pack->pack_name);
+ strbuf_chompstr(&path, ".pack");
+ strbuf_addch(&path, '.');
+ strbuf_addstr(&path, name);
+
+ fd = open(path.buf, O_RDONLY);
+ strbuf_release(&path);
+ if (fd < 0)
+ return -1;
+ if (fstat(fd, &st) < 0) {
+ close(fd);
+ return -1;
+ }
+
+ m->mapped_len = xsize_t(st.st_size);
+ m->mapped_buf = xmmap(NULL, m->mapped_len, PROT_READ, MAP_PRIVATE, fd, 0);
+ close(fd);
+
+ m->data = m->mapped_buf;
+ m->len = m->mapped_len;
+
+ if (m->len < 8 ||
+ memcmp(m->mapped_buf, "META", 4) ||
+ memcmp(m->mapped_buf + 4, "\0\0\0\1", 4)) {
+ warning("metapack '%s' for '%s' does not have a valid header",
+ name, pack->pack_name);
+ metapack_close(m);
+ return -1;
+ }
+ m->data += 8;
+ m->len -= 8;
+
+ if (m->len < 20 || hashcmp(m->data, pack->sha1)) {
+ warning("metapack '%s' for '%s' does not match pack sha1",
+ name, pack->pack_name);
+ metapack_close(m);
+ return -1;
+ }
+ m->data += 20;
+ m->len -= 20;
+
+ if (m->len < 8 || memcmp(m->data, name, 4)) {
+ warning("metapack '%s' for '%s' does not have expected header id",
+ name, pack->pack_name);
+ metapack_close(m);
+ return -1;
+ }
+ memcpy(version, m->data + 4, 4);
+ *version = ntohl(*version);
+ m->data += 8;
+ m->len -= 8;
+
+ return 0;
+}
+
+void metapack_close(struct metapack *m)
+{
+ munmap(m->mapped_buf, m->mapped_len);
+}
diff --git a/metapack.h b/metapack.h
new file mode 100644
index 0000000..6af17fe
--- /dev/null
+++ b/metapack.h
@@ -0,0 +1,42 @@
+#ifndef METAPACK_H
+#define METAPACK_H
+
+struct packed_git;
+struct sha1file;
+
+struct metapack_writer {
+ char *path;
+ struct packed_git *pack;
+ struct sha1file *out;
+};
+
+void metapack_writer_init(struct metapack_writer *mw,
+ const char *pack_idx,
+ const char *name,
+ int version);
+void metapack_writer_add(struct metapack_writer *mw, const void *data, int len);
+void metapack_writer_add_uint32(struct metapack_writer *mw, uint32_t v);
+void metapack_writer_finish(struct metapack_writer *mw);
+
+typedef void (*metapack_writer_each_fn)(struct metapack_writer *,
+ const unsigned char *sha1,
+ void *data);
+void metapack_writer_foreach(struct metapack_writer *mw,
+ metapack_writer_each_fn cb,
+ void *data);
+
+struct metapack {
+ void *mapped_buf;
+ size_t mapped_len;
+
+ void *data;
+ size_t len;
+};
+
+int metapack_init(struct metapack *m,
+ struct packed_git *pack,
+ const char *name,
+ uint32_t *version);
+void metapack_close(struct metapack *m);
+
+#endif
--
1.8.0.2.16.g72e2fc9
^ permalink raw reply related
* [PATCH 4/6] introduce a commit metapack
From: Jeff King @ 2013-01-29 9:16 UTC (permalink / raw)
To: git; +Cc: Duy Nguyen, Shawn O. Pearce
In-Reply-To: <20130129091434.GA6975@sigill.intra.peff.net>
When we are doing a commit traversal that does not need to
look at the commit messages themselves (e.g., rev-list,
merge-base, etc), we spend a lot of time accessing,
decompressing, and parsing the commit objects just to find
the parent and timestamp information. We can make a
space-time tradeoff by caching that information on disk in a
compact, uncompressed format.
TODO: document on-disk format in Documentation/technical
TODO: document API
Signed-off-by: Jeff King <peff@peff.net>
---
Makefile | 2 +
commit-metapack.c | 175 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
commit-metapack.h | 12 ++++
3 files changed, 189 insertions(+)
create mode 100644 commit-metapack.c
create mode 100644 commit-metapack.h
diff --git a/Makefile b/Makefile
index 3e4ae1b..6ca5320 100644
--- a/Makefile
+++ b/Makefile
@@ -619,6 +619,7 @@ LIB_H += column.h
LIB_H += cache.h
LIB_H += color.h
LIB_H += column.h
+LIB_H += commit-metapack.h
LIB_H += commit.h
LIB_H += compat/bswap.h
LIB_H += compat/cygwin.h
@@ -730,6 +731,7 @@ LIB_OBJS += combine-diff.o
LIB_OBJS += color.o
LIB_OBJS += column.o
LIB_OBJS += combine-diff.o
+LIB_OBJS += commit-metapack.o
LIB_OBJS += commit.o
LIB_OBJS += compat/obstack.o
LIB_OBJS += compat/terminal.o
diff --git a/commit-metapack.c b/commit-metapack.c
new file mode 100644
index 0000000..2c19f48
--- /dev/null
+++ b/commit-metapack.c
@@ -0,0 +1,175 @@
+#include "cache.h"
+#include "commit-metapack.h"
+#include "metapack.h"
+#include "commit.h"
+#include "sha1-lookup.h"
+
+struct commit_metapack {
+ struct metapack mp;
+ uint32_t nr;
+ unsigned char *index;
+ unsigned char *data;
+ struct commit_metapack *next;
+};
+static struct commit_metapack *commit_metapacks;
+
+static struct commit_metapack *alloc_commit_metapack(struct packed_git *pack)
+{
+ struct commit_metapack *it = xcalloc(1, sizeof(*it));
+ uint32_t version;
+
+ if (metapack_init(&it->mp, pack, "commits", &version) < 0) {
+ free(it);
+ return NULL;
+ }
+ if (version != 1) {
+ /*
+ * This file comes from a more recent git version. Don't bother
+ * warning the user, as we'll just fallback to reading the
+ * commits.
+ */
+ metapack_close(&it->mp);
+ free(it);
+ return NULL;
+ }
+
+ if (it->mp.len < 4) {
+ warning("commit metapack for '%s' is truncated", pack->pack_name);
+ metapack_close(&it->mp);
+ free(it);
+ return NULL;
+ }
+ memcpy(&it->nr, it->mp.data, 4);
+ it->nr = ntohl(it->nr);
+
+ /*
+ * We need 84 bytes for each entry: sha1(20), date(4), tree(20),
+ * parents(40).
+ */
+ if (it->mp.len < (84 * it->nr + 4)) {
+ warning("commit metapack for '%s' is truncated", pack->pack_name);
+ metapack_close(&it->mp);
+ free(it);
+ return NULL;
+ }
+
+ it->index = it->mp.data + 4;
+ it->data = it->index + 20 * it->nr;
+
+ return it;
+}
+
+static void prepare_commit_metapacks(void)
+{
+ static int initialized;
+ struct commit_metapack **tail = &commit_metapacks;
+ struct packed_git *p;
+
+ if (initialized)
+ return;
+
+ prepare_packed_git();
+ for (p = packed_git; p; p = p->next) {
+ struct commit_metapack *it = alloc_commit_metapack(p);
+
+ if (it) {
+ *tail = it;
+ tail = &it->next;
+ }
+ }
+
+ initialized = 1;
+}
+
+int commit_metapack(unsigned char *sha1,
+ uint32_t *timestamp,
+ unsigned char **tree,
+ unsigned char **parent1,
+ unsigned char **parent2)
+{
+ struct commit_metapack *p;
+
+ prepare_commit_metapacks();
+ for (p = commit_metapacks; p; p = p->next) {
+ unsigned char *data;
+ int pos = sha1_entry_pos(p->index, 20, 0, 0, p->nr, p->nr, sha1);
+ if (pos < 0)
+ continue;
+
+ /* timestamp(4) + tree(20) + parents(40) */
+ data = p->data + 64 * pos;
+ *timestamp = *(uint32_t *)data;
+ *timestamp = ntohl(*timestamp);
+ data += 4;
+ *tree = data;
+ data += 20;
+ *parent1 = data;
+ data += 20;
+ *parent2 = data;
+
+ return 0;
+ }
+
+ return -1;
+}
+
+static void get_commits(struct metapack_writer *mw,
+ const unsigned char *sha1,
+ void *data)
+{
+ struct commit_list ***tail = data;
+ enum object_type type = sha1_object_info(sha1, NULL);
+ struct commit *c;
+
+ if (type != OBJ_COMMIT)
+ return;
+
+ c = lookup_commit(sha1);
+ if (!c || parse_commit(c))
+ die("unable to read commit %s", sha1_to_hex(sha1));
+
+ /*
+ * Our fixed-size parent list cannot represent root commits, nor
+ * octopus merges. Just skip those commits, as we can fallback
+ * in those rare cases to reading the actual commit object.
+ */
+ if (!c->parents ||
+ (c->parents && c->parents->next && c->parents->next->next))
+ return;
+
+ *tail = &commit_list_insert(c, *tail)->next;
+}
+
+void commit_metapack_write(const char *idx)
+{
+ struct metapack_writer mw;
+ struct commit_list *commits = NULL, *p;
+ struct commit_list **tail = &commits;
+ uint32_t nr = 0;
+
+ metapack_writer_init(&mw, idx, "commits", 1);
+
+ /* Figure out how many eligible commits we've got in this pack. */
+ metapack_writer_foreach(&mw, get_commits, &tail);
+ for (p = commits; p; p = p->next)
+ nr++;
+ metapack_writer_add_uint32(&mw, nr);
+
+ /* Then write an index of commit sha1s */
+ for (p = commits; p; p = p->next)
+ metapack_writer_add(&mw, p->item->object.sha1, 20);
+
+ /* Followed by the actual date/tree/parents data */
+ for (p = commits; p; p = p->next) {
+ struct commit *c = p->item;
+ metapack_writer_add_uint32(&mw, c->date);
+ metapack_writer_add(&mw, c->tree->object.sha1, 20);
+ metapack_writer_add(&mw, c->parents->item->object.sha1, 20);
+ metapack_writer_add(&mw,
+ c->parents->next ?
+ c->parents->next->item->object.sha1 :
+ null_sha1, 20);
+ }
+
+ metapack_writer_finish(&mw);
+}
diff --git a/commit-metapack.h b/commit-metapack.h
new file mode 100644
index 0000000..4684573
--- /dev/null
+++ b/commit-metapack.h
@@ -0,0 +1,12 @@
+#ifndef METAPACK_COMMIT_H
+#define METAPACK_COMMIT_H
+
+int commit_metapack(unsigned char *sha1,
+ uint32_t *timestamp,
+ unsigned char **tree,
+ unsigned char **parent1,
+ unsigned char **parent2);
+
+void commit_metapack_write(const char *idx_file);
+
+#endif
--
1.8.0.2.16.g72e2fc9
^ permalink raw reply related
* [PATCH 5/6] add git-metapack command
From: Jeff King @ 2013-01-29 9:16 UTC (permalink / raw)
To: git; +Cc: Duy Nguyen, Shawn O. Pearce
In-Reply-To: <20130129091434.GA6975@sigill.intra.peff.net>
This is a plumbing command for generating metapack files.
Right now it understands only the "commits" metapack (and
there is not yet a reader). Eventually we may want to build
this metapack automatically when we generate a new pack.
Let's be conservative for now, though, and let the idea
prove itself in practice before turning it on for everyone.
The commits metapack generated by this command is 84 bytes
per commit; for linux-2.6.git, this is about 31M.
TODO: documentation
Signed-off-by: Jeff King <peff@peff.net>
---
.gitignore | 1 +
Makefile | 1 +
builtin.h | 1 +
builtin/metapack.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
git-repack.sh | 2 +-
git.c | 1 +
6 files changed, 78 insertions(+), 1 deletion(-)
create mode 100644 builtin/metapack.c
diff --git a/.gitignore b/.gitignore
index 6669bf0..3f67a36 100644
--- a/.gitignore
+++ b/.gitignore
@@ -93,6 +93,7 @@
/git-merge-subtree
/git-mergetool
/git-mergetool--lib
+/git-metapack
/git-mktag
/git-mktree
/git-name-rev
diff --git a/Makefile b/Makefile
index 6ca5320..3899699 100644
--- a/Makefile
+++ b/Makefile
@@ -905,6 +905,7 @@ BUILTIN_OBJS += builtin/merge-tree.o
BUILTIN_OBJS += builtin/merge-ours.o
BUILTIN_OBJS += builtin/merge-recursive.o
BUILTIN_OBJS += builtin/merge-tree.o
+BUILTIN_OBJS += builtin/metapack.o
BUILTIN_OBJS += builtin/mktag.o
BUILTIN_OBJS += builtin/mktree.o
BUILTIN_OBJS += builtin/mv.o
diff --git a/builtin.h b/builtin.h
index faef559..30108ab 100644
--- a/builtin.h
+++ b/builtin.h
@@ -97,6 +97,7 @@ extern int cmd_merge_tree(int argc, const char **argv, const char *prefix);
extern int cmd_merge_file(int argc, const char **argv, const char *prefix);
extern int cmd_merge_recursive(int argc, const char **argv, const char *prefix);
extern int cmd_merge_tree(int argc, const char **argv, const char *prefix);
+extern int cmd_metapack(int argc, const char **argv, const char *prefix);
extern int cmd_mktag(int argc, const char **argv, const char *prefix);
extern int cmd_mktree(int argc, const char **argv, const char *prefix);
extern int cmd_mv(int argc, const char **argv, const char *prefix);
diff --git a/builtin/metapack.c b/builtin/metapack.c
new file mode 100644
index 0000000..5fee6cf
--- /dev/null
+++ b/builtin/metapack.c
@@ -0,0 +1,73 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "commit-metapack.h"
+
+static const char *metapack_usage[] = {
+ N_("git metapack [options] <packindex...>"),
+ NULL
+};
+
+#define METAPACK_COMMITS (1<<0)
+
+static void metapack_one(const char *idx, int type)
+{
+ if (type & METAPACK_COMMITS)
+ commit_metapack_write(idx);
+}
+
+static void metapack_all(int type)
+{
+ struct strbuf path = STRBUF_INIT;
+ size_t dirlen;
+ DIR *dh;
+ struct dirent *de;
+
+ strbuf_addstr(&path, get_object_directory());
+ strbuf_addstr(&path, "/pack");
+ dirlen = path.len;
+
+ dh = opendir(path.buf);
+ if (!dh)
+ die_errno("unable to open pack directory '%s'", path.buf);
+ while ((de = readdir(dh))) {
+ if (!has_extension(de->d_name, ".idx"))
+ continue;
+
+ strbuf_addch(&path, '/');
+ strbuf_addstr(&path, de->d_name);
+ metapack_one(path.buf, type);
+ strbuf_setlen(&path, dirlen);
+ }
+
+ closedir(dh);
+ strbuf_release(&path);
+}
+
+int cmd_metapack(int argc, const char **argv, const char *prefix)
+{
+ int all = 0;
+ int type = 0;
+ struct option opts[] = {
+ OPT_BOOL(0, "all", &all, N_("create metapacks for all packs")),
+ OPT_BIT(0, "commits", &type, N_("create commit metapacks"),
+ METAPACK_COMMITS),
+ OPT_END()
+ };
+
+ argc = parse_options(argc, argv, prefix, opts, metapack_usage, 0);
+
+ if (all && argc)
+ usage_msg_opt(_("pack arguments do not make sense with --all"),
+ metapack_usage, opts);
+ if (!type)
+ usage_msg_opt(_("no metapack type specified"),
+ metapack_usage, opts);
+
+ if (all)
+ metapack_all(type);
+ else
+ for (; *argv; argv++)
+ metapack_one(*argv, type);
+
+ return 0;
+}
diff --git a/git-repack.sh b/git-repack.sh
index 7579331..e6a9773 100755
--- a/git-repack.sh
+++ b/git-repack.sh
@@ -180,7 +180,7 @@ then
do
case " $fullbases " in
*" $e "*) ;;
- *) rm -f "$e.pack" "$e.idx" "$e.keep" ;;
+ *) rm -f "$e.pack" "$e.idx" "$e.keep" "$e.commits";;
esac
done
)
diff --git a/git.c b/git.c
index b10c18b..f6e5552 100644
--- a/git.c
+++ b/git.c
@@ -365,6 +365,7 @@ static void handle_internal_command(int argc, const char **argv)
{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE },
{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE },
{ "merge-tree", cmd_merge_tree, RUN_SETUP },
+ { "metapack", cmd_metapack, RUN_SETUP },
{ "mktag", cmd_mktag, RUN_SETUP },
{ "mktree", cmd_mktree, RUN_SETUP },
{ "mv", cmd_mv, RUN_SETUP | NEED_WORK_TREE },
--
1.8.0.2.16.g72e2fc9
^ permalink raw reply related
* [PATCH 6/6] commit: look up commit info in metapack
From: Jeff King @ 2013-01-29 9:16 UTC (permalink / raw)
To: git; +Cc: Duy Nguyen, Shawn O. Pearce
In-Reply-To: <20130129091434.GA6975@sigill.intra.peff.net>
Now that we have the plumbing in place to generate and read
commit metapacks, we can hook them up to parse_commit to
fill in the traversal information much more quickly.
We only do so if save_commit_buffer is turned off;
otherwise, the callers will expect to be able to read
commit->buffer after parse_commit returns (and since our
cache obviously does not have that information, we must
leave it NULL). As callers learn to handle a NULL
commit->buffer, we can eventually relax this (while it might
seem like a useless no-op to use the cache if we are going
to load the commit anyway, many callers may first filter
based on the traversal, and end up loading the commit
message for only a subset of the commits).
With this patch (and having run "git metapack --all
--commits"), my best-of-five warm-cache "git rev-list
--count --all" traversal of linux-2.6.git drops from 4.219s
to 0.659s.
Similarly, cold-cache drops from 13.696s to 4.763s due to
the compactness of the cache data (but you are penalized, of
course, if you then want to actually look at the commit
messages, since you have not warmed them into the cache).
Signed-off-by: Jeff King <peff@peff.net>
---
commit.c | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/commit.c b/commit.c
index e8eb0ae..b326201 100644
--- a/commit.c
+++ b/commit.c
@@ -8,6 +8,7 @@
#include "notes.h"
#include "gpg-interface.h"
#include "mergesort.h"
+#include "commit-metapack.h"
static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
@@ -306,6 +307,24 @@ int parse_commit_buffer(struct commit *item, const void *buffer, unsigned long s
return 0;
}
+static int parse_commit_metapack(struct commit *item)
+{
+ unsigned char *tree, *p1, *p2;
+ uint32_t ts;
+
+ if (commit_metapack(item->object.sha1, &ts, &tree, &p1, &p2) < 0)
+ return -1;
+
+ item->date = ts;
+ item->tree = lookup_tree(tree);
+ commit_list_insert(lookup_commit(p1), &item->parents);
+ if (!is_null_sha1(p2))
+ commit_list_insert(lookup_commit(p2), &item->parents->next);
+
+ item->object.parsed = 1;
+ return 0;
+}
+
int parse_commit(struct commit *item)
{
enum object_type type;
@@ -317,6 +336,10 @@ int parse_commit(struct commit *item)
return -1;
if (item->object.parsed)
return 0;
+
+ if (!save_commit_buffer && !parse_commit_metapack(item))
+ return 0;
+
buffer = read_sha1_file(item->object.sha1, &type, &size);
if (!buffer)
return error("Could not read %s",
--
1.8.0.2.16.g72e2fc9
^ permalink raw reply related
* Re: [PATCH 2/6] strbuf: add string-chomping functions
From: Michael Haggerty @ 2013-01-29 10:15 UTC (permalink / raw)
To: Jeff King; +Cc: git, Duy Nguyen, Shawn O. Pearce
In-Reply-To: <20130129091540.GB9999@sigill.intra.peff.net>
On 01/29/2013 10:15 AM, Jeff King wrote:
> Sometimes it is handy to cut a trailing string off the end
> of a strbuf (e.g., a file extension). These helper functions
> make it a one-liner.
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> strbuf.c | 11 +++++++++++
> strbuf.h | 2 ++
> 2 files changed, 13 insertions(+)
>
> diff --git a/strbuf.c b/strbuf.c
> index 9a373be..8199ced 100644
> --- a/strbuf.c
> +++ b/strbuf.c
> @@ -106,6 +106,17 @@ void strbuf_ltrim(struct strbuf *sb)
> sb->buf[sb->len] = '\0';
> }
>
> +void strbuf_chompmem(struct strbuf *sb, const void *data, size_t len)
> +{
> + if (sb->len >= len && !memcmp(data, sb->buf + sb->len - len, len))
> + strbuf_setlen(sb, sb->len - len);
> +}
> +
> +void strbuf_chompstr(struct strbuf *sb, const char *str)
> +{
> + strbuf_chompmem(sb, str, strlen(str));
> +}
> +
> struct strbuf **strbuf_split_buf(const char *str, size_t slen,
> int terminator, int max)
> {
> diff --git a/strbuf.h b/strbuf.h
> index ecae4e2..3aeb815 100644
> --- a/strbuf.h
> +++ b/strbuf.h
> @@ -42,6 +42,8 @@ extern void strbuf_ltrim(struct strbuf *);
> extern void strbuf_trim(struct strbuf *);
> extern void strbuf_rtrim(struct strbuf *);
> extern void strbuf_ltrim(struct strbuf *);
> +extern void strbuf_chompmem(struct strbuf *, const void *, size_t);
> +extern void strbuf_chompstr(struct strbuf *, const char *);
> extern int strbuf_cmp(const struct strbuf *, const struct strbuf *);
>
> /*
>
It might be handy to have these functions return true/false based on
whether the suffix was actually found.
Please document the new functions in
Documentation/technical/api-strbuf.txt. Personally I would also
advocate a "docstring" in the header file, but obviously that preference
is the exception rather than the rule in the git project :-(
Michael
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
^ permalink raw reply
* Re: [PATCH 4/6] introduce a commit metapack
From: Michael Haggerty @ 2013-01-29 10:24 UTC (permalink / raw)
To: Jeff King; +Cc: git, Duy Nguyen, Shawn O. Pearce
In-Reply-To: <20130129091610.GD9999@sigill.intra.peff.net>
On 01/29/2013 10:16 AM, Jeff King wrote:
> When we are doing a commit traversal that does not need to
> look at the commit messages themselves (e.g., rev-list,
> merge-base, etc), we spend a lot of time accessing,
> decompressing, and parsing the commit objects just to find
> the parent and timestamp information. We can make a
> space-time tradeoff by caching that information on disk in a
> compact, uncompressed format.
>
> TODO: document on-disk format in Documentation/technical
> TODO: document API
Would this be a good place to add the commit generation number that is
so enthusiastically discussed on the mailing list from time to time?
Michael
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
^ permalink raw reply
* [PATCH] README: fix broken mailing list archive link
From: Ramkumar Ramachandra @ 2013-01-29 10:40 UTC (permalink / raw)
To: Git List
marc.theaimsgroup.com does not exist anymore, so replace it
with a link to the archive on GMane.
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
---
README | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/README b/README
index 49713ea..3aae16a 100644
--- a/README
+++ b/README
@@ -47,8 +47,8 @@ requests, comments and patches to git@vger.kernel.org (read
Documentation/SubmittingPatches for instructions on patch submission).
To subscribe to the list, send an email with just "subscribe git" in
the body to majordomo@vger.kernel.org. The mailing list archives are
-available at http://marc.theaimsgroup.com/?l=git and other archival
-sites.
+available at http://thread.gmane.org/gmane.comp.version-control.git/
+and other archival sites.
The messages titled "A note from the maintainer", "What's in
git.git (stable)" and "What's cooking in git.git (topics)" and
--
1.7.10.4
^ permalink raw reply related
* [PATCH] gitk-git/.gitignore: add rule for gitk-wish
From: Ramkumar Ramachandra @ 2013-01-29 10:52 UTC (permalink / raw)
To: Git List
8f26aa4 (Makefile: remove tracking of TCLTK_PATH, 2012-12-18) removed
"/gitk-git/gitk-wish" from the toplevel .gitignore, with the intent of
moving it to gitk-git/.gitignore in a later patch. This was never
realized.
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
---
Minor patch, so I didn't bother sending it through Paul.
gitk-git/.gitignore | 1 +
1 file changed, 1 insertion(+)
create mode 100644 gitk-git/.gitignore
diff --git a/gitk-git/.gitignore b/gitk-git/.gitignore
new file mode 100644
index 0000000..1dc38be
--- /dev/null
+++ b/gitk-git/.gitignore
@@ -0,0 +1 @@
+/gitk-wish
--
1.7.10.4
^ permalink raw reply related
* Re: [PATCH 0/2] optimizing pack access on "read only" fetch repos
From: Duy Nguyen @ 2013-01-29 11:01 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jeff King, git
In-Reply-To: <7vlibfxhit.fsf@alter.siamese.dyndns.org>
On Sun, Jan 27, 2013 at 1:32 PM, Junio C Hamano <gitster@pobox.com> wrote:
> I also wonder if we would be helped by another "repack" mode that
> coalesces small packs into a single one with minimum overhead, and
> run that often from "gc --auto", so that we do not end up having to
> have 50 packfiles.
>
> When we have 2 or more small and young packs, we could:
>
> - iterate over idx files for these packs to enumerate the objects
> to be packed, replacing read_object_list_from_stdin() step;
>
> - always choose to copy the data we have in these existing packs,
> instead of doing a full prepare_pack(); and
>
> - use the order the objects appear in the original packs, bypassing
> compute_write_order().
Isn't it easier and cheaper to create the "master index", something
like bup does?
--
Duy
^ permalink raw reply
* Re: [PATCH 2/6] strbuf: add string-chomping functions
From: Jeff King @ 2013-01-29 11:10 UTC (permalink / raw)
To: Michael Haggerty; +Cc: git, Duy Nguyen, Shawn O. Pearce
In-Reply-To: <5107A146.4000309@alum.mit.edu>
On Tue, Jan 29, 2013 at 11:15:34AM +0100, Michael Haggerty wrote:
> > +void strbuf_chompmem(struct strbuf *sb, const void *data, size_t len)
> > +{
> > + if (sb->len >= len && !memcmp(data, sb->buf + sb->len - len, len))
> > + strbuf_setlen(sb, sb->len - len);
> > +}
> > +
> > +void strbuf_chompstr(struct strbuf *sb, const char *str)
> > +{
> > + strbuf_chompmem(sb, str, strlen(str));
> > +}
> > +
> It might be handy to have these functions return true/false based on
> whether the suffix was actually found.
Yeah, that sounds reasonable.
> Please document the new functions in
> Documentation/technical/api-strbuf.txt. Personally I would also
> advocate a "docstring" in the header file, but obviously that preference
> is the exception rather than the rule in the git project :-(
Will do. I need to document the metapack functions, too, so I was thinking
about experimenting with some inline documentation systems.
-Peff
^ permalink raw reply
* Re: [PATCH 4/6] introduce a commit metapack
From: Jeff King @ 2013-01-29 11:13 UTC (permalink / raw)
To: Michael Haggerty; +Cc: git, Duy Nguyen, Shawn O. Pearce
In-Reply-To: <5107A36D.2050307@alum.mit.edu>
On Tue, Jan 29, 2013 at 11:24:45AM +0100, Michael Haggerty wrote:
> On 01/29/2013 10:16 AM, Jeff King wrote:
> > When we are doing a commit traversal that does not need to
> > look at the commit messages themselves (e.g., rev-list,
> > merge-base, etc), we spend a lot of time accessing,
> > decompressing, and parsing the commit objects just to find
> > the parent and timestamp information. We can make a
> > space-time tradeoff by caching that information on disk in a
> > compact, uncompressed format.
> >
> > TODO: document on-disk format in Documentation/technical
> > TODO: document API
>
> Would this be a good place to add the commit generation number that is
> so enthusiastically discussed on the mailing list from time to time?
Yes, that is one of my goals. We may even be able to just replace the
timestamp field in the cache with a generation number. When it gets
pretty-printed we pull it out of the commit message again anyway, so in
theory the only use inside "struct commit" is for ordering. But I
haven't looked at all of the use sites yet to be sure nobody is
depending on it being an actual date stamp.
-Peff
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox