* Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?,.gitacls? etc.
From: Jan Hudec @ 2007-09-05 18:38 UTC (permalink / raw)
To: Junio C Hamano
Cc: Dmitry Kakurin, Johannes Schindelin, Petr Baudis,
Git Mailing List
In-Reply-To: <7vk5r5jzpn.fsf@gitster.siamese.dyndns.org>
[-- Attachment #1: Type: text/plain, Size: 1982 bytes --]
On Wed, Sep 05, 2007 at 01:14:44 -0700, Junio C Hamano wrote:
> "Dmitry Kakurin" <dmitry.kakurin@gmail.com> writes:
>
> > I assert that since index does not have .gitattributes the one from
> > local directory should not be used.
> >
> > Think about dedicated build machine scenario: I have a machine that
> > always does sync + build. After every sync the local directory should
> > always be identical to what-was-committed.
>
> Thinking about the reason _why_ .gitattributes may be updated,
> one would notice that it is because somebody did this command
> sequence:
>
> git checkout ;# now work tree is clean
> edit .gitattributes ;# modify the attributes of a file
> edit file ;# edit the file attributes talks about
> git add file ;# this can be affected by .gitattributes
> git add .gitattributes ;# this is changed in the same commit
> git commit
>
> Now, should we always take .gitattributes from the index?
Yes, they should:
$ git checkout
$ edit .gitattributes
$ edit file
$ git add file
$ git commit ;# this does NOT have the changes to .gitattributes
the above case is a user error that can (at some cost) be detected:
$ git checkout
$ edit .gitattributes
$ edit file
$ git add file
$ git add .gitattributes
Warning! Changes to gitattributes affects handling of files scheduled for
commit. Please add following files again before commit:
file
$
It would be possible to special-case .gitattributes in add to:
- do diff between the old and new value of .gitattributes in index,
- list files changed in index compared to HEAD,
- match each of them to all patterns in the diff,
- if any matches, print the warning and list of matches.
It might be even possible to actually inspect the changes and apply those
that can be automatically (and not ask user to re-add), but some filters
loose information, so user interaction is needed to add good version.
--
Jan 'Bulb' Hudec <bulb@ucw.cz>
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Brandon Casey @ 2007-09-05 18:46 UTC (permalink / raw)
To: J. Bruce Fields; +Cc: Linus Torvalds, Git Mailing List
In-Reply-To: <20070905174427.GC13314@fieldses.org>
On Wed, 5 Sep 2007, J. Bruce Fields wrote:
> Well, this may just prove I'm an idiot, but one of the reasons I rarely
> run it is that I have trouble remembering exactly what it does; in
> particular,
>
> - does it prune anything that might be needed by a repo I
> cloned with -s?
YES! yikes.
This is about the best argument put forth so far for not automatically
running git-gc. Personally, I think git-gc should not remove unreferenced
objects without --prune (but I haven't done anything about it). But even
if git-gc was modified in this way, an occasional git-gc --prune would
still be necessary to remove all of the unreferenced and dangling objects
safely with a human thinking about the shared repo implications (unless
shared repo handling is modified).
-brandon
^ permalink raw reply
* Significant performance waste in git-svn and friends
From: Mike Hommey @ 2007-09-05 18:47 UTC (permalink / raw)
To: git
Hi,
Being a pervert abusing the way subversion doesn't deal with branches
and tags, I'm actually not a user of git-svn or git-svnimport, because
they just can't deal easily with my perversion. So I'm writing a script
to do the conversion for me, and since I also like to learn new things
when I'm coding, I'm writing it in ruby.
Anyways, one of the things I'm trying to convert is my svk repository
for debian packaging of xulrunner (so, a significant subset of the
mozilla tree), which doesn't involve a lot of revisions (around 280,
because I only imported releases or CVS snapshots), but involves a lot
of files (roughly 20k).
The first thing I noticed when twisting around the svk repo so that
git-svn could somehow import it a while ago, is that running git-svn
was in my case significantly slower than svnadmin dump | svnadmin load
(more than 2 times slower).
And now, with my own script, I got the same kind of "slowdown". So I
investigated it, and it didn't take long to realize that replacing
git-hash-object by a simple reimplementation in ruby was *way* faster.
git-hash-object being more than probably what you do the most when you
import a remote repository, it is not much of a surprise that forking
thousands of times is a huge performance waste.
So, just for the record, I did a lame hack of git-svn to see what kind
of speedup could happen in git-svn. You can find this lame hack as a
patch below. I did some tests (with a 1.5.2.1 release) and here are the
results, importing only the trunk (192 revisions), with no checkout, and
redirecting stdout to /dev/null:
original git-svn:
real 25m1.871s
user 8m51.593s
sys 12m31.659s
patched git-svn:
real 14m45.870s
user 7m31.928s
sys 4m1.047s
Some notes about the patch:
- I've not looked at the rest of the code to see if there was a way to
get the size of the file so that SHA-1 sum and compression could be
done in one pass and without copying the whole file in memory.
- The object creation in the .git/objects directory is not as safe as
what git-hash-object does.
Some notes about git-svn:
- A few lines above the patched zone, the file is already read once to
do the MD5 sum. It should be possible to do SHA-1, MD5 sums and
deflate in just one pass.
- It might be worth testing if git-cat-file is called a lot. If so,
implementing a simple git-cat-file equivalent that would work for
unpacked objects could improve speed.
The same things obviously apply to git-cvsimport and other scripts
calling git-hash-object a lot.
Mike
diff --git a/git-svn.perl b/git-svn.perl
index d3c8cd0..202c228 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -2417,6 +2417,8 @@ use warnings;
use Carp qw/croak/;
use IO::File qw//;
use Digest::MD5;
+use Digest::SHA1;
+use Compress::Zlib;
# file baton members: path, mode_a, mode_b, pool, fh, blob, base
sub new {
@@ -2603,15 +2605,26 @@ sub close_file {
$buf eq 'link ' or die "$path has mode 120000",
"but is not a link\n";
}
- defined(my $pid = open my $out,'-|') or die "Can't fork: $!\n";
- if (!$pid) {
- open STDIN, '<&', $fh or croak $!;
- exec qw/git-hash-object -w --stdin/ or croak $!;
+ my $size = 0;
+ my $buf = "";
+ while (my $read = sysread $fh, my $tmp, 4096) {
+ $size += $read;
+ $buf .= $tmp;
}
- chomp($hash = do { local $/; <$out> });
- close $out or croak $!;
+ my $sha1 = Digest::SHA1->new;
+ $sha1->add("blob $size\0");
+ $sha1->add($buf);
+ $hash = $sha1->hexdigest;
close $fh or croak $!;
$hash =~ /^[a-f\d]{40}$/ or die "not a sha1: $hash\n";
+ my $blob_dir = "$ENV{GIT_DIR}/objects/" . substr($hash, 0, 2);
+ my $blob_file = $blob_dir . "/" . substr($hash, 2);
+ if (! -f $blob_file) {
+ mkdir $blob_dir unless -d $blob_dir;
+ open BLOB, ">$blob_file";
+ print BLOB compress("blob $size\0" . $buf);
+ close BLOB;
+ }
close $fb->{base} or croak $!;
} else {
$hash = $fb->{blob} or die "no blob information\n";
^ permalink raw reply related
* Re: People unaware of the importance of "git gc"?
From: Nicolas Pitre @ 2007-09-05 18:54 UTC (permalink / raw)
To: Nix; +Cc: Steven Grimm, Linus Torvalds, Git Mailing List
In-Reply-To: <877in50y7p.fsf@hades.wkstn.nix>
On Wed, 5 Sep 2007, Nix wrote:
> I personally wonder if git-gc shouldn't use a proportional scheme, so
> that only some packs get repacked, maybe the smallest ones (and when
> they grow to the same size as the next largest one, the two get repacked
> into one). This has the singular advantage that you won't have to
> carefully drop .keep files everywhere or have to worry about your git-gc
> of 50K of loose objects suddenly deciding to repack 100Mb of packfiles
> and taking ages.
Not only that. Currently the "Counting objects" phase when running
git-gc on the Linux repo takes a significant amount of time, even if
there is little to repack.
If any kind of automatic repack is implemented, it should be an
incremental repacking only, not the full thing, i.e. git-repack without
-a, or git-pack-objects with --unpacked. The idea is to be the least
intrusive as possible. Also, object walking should be limited to
objects linked to a commit object which is itself unpacked in order to
cut on the time required to fully enumerate all objects.
This way a semi-packed state will always be preserved and should be good
enough. The full repacking should probably be left to manual execution
of git-gc.
Nicolas
^ permalink raw reply
* Re: [PATCH] Add a new lstat implementation based on Win32 API, and make stat use that implementation too.
From: David Kastrup @ 2007-09-05 19:01 UTC (permalink / raw)
To: Miklos Vajna; +Cc: git
In-Reply-To: <20070905160206.GY14853@genesis.frugalware.org>
[Vista and symbolic links]
Miklos Vajna <vmiklos@frugalware.org> writes:
> On Mon, Sep 03, 2007 at 01:32:07PM +0200, David Kastrup <dak@gnu.org> wrote:
>> >> And if I understand the documentation correctly, then
>> >
>> >> $ mkdir foo && cd foo
>> >> $ cat ../x
>> >> x: No such file or directory
>> >
>> > correct.
>>
>> Have you tested this, or is this from reading the documentation?
>
> umm, thanks for the notice, i was wrong:
>
> ----
> $ cat ../x
> this is makefile
> ----
>
> the situation what triggers the 'no such file' problem is:
>
> ----
> $ touch foo/Makefile
> $ mkdir bar
> $ ln -s foo/Makefile bar
> $ cd bar
> $ cat Makefile
> cat: Makefile: No such file or directory
> ----
This is under Vista? It would be the same under Unix. A good rule of
thumb is to create relative symbolic links _only_ when the current
work directory is identical with the target directory (and make it so
if it isn't). Alternatively, when one is completely awake and in full
possession of all one's mental facilities. The state that every geek
is in 90% of the time according to his own perception, and 9% of the
time according to the computer.
If you tested this on Vista, chances are that they only bungled the
documentation in this case (and of course, needing sysadmin privileges
for this would be just silly). Not without precedence.
--
David Kastrup, Kriemhildstr. 15, 44793 Bochum
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: David Kastrup @ 2007-09-05 19:09 UTC (permalink / raw)
To: Brandon Casey; +Cc: J. Bruce Fields, Linus Torvalds, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0709051339420.30020@torch.nrlssc.navy.mil>
Brandon Casey <casey@nrlssc.navy.mil> writes:
> On Wed, 5 Sep 2007, J. Bruce Fields wrote:
>
>> Well, this may just prove I'm an idiot, but one of the reasons I rarely
>> run it is that I have trouble remembering exactly what it does; in
>> particular,
>>
>> - does it prune anything that might be needed by a repo I
>> cloned with -s?
>
> YES! yikes.
>
> This is about the best argument put forth so far for not
> automatically running git-gc.
Well, it could also mean that if git finds a dead symbolic link when
looking up an object, it should check the corresponding link target
directory for a pack file with the respective object... and if it
finds such a pack file, create a link to it and use it.
--
David Kastrup, Kriemhildstr. 15, 44793 Bochum
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: J. Bruce Fields @ 2007-09-05 19:13 UTC (permalink / raw)
To: David Kastrup; +Cc: Brandon Casey, Linus Torvalds, Git Mailing List
In-Reply-To: <85642phqtn.fsf@lola.goethe.zz>
On Wed, Sep 05, 2007 at 09:09:40PM +0200, David Kastrup wrote:
> Brandon Casey <casey@nrlssc.navy.mil> writes:
>
> > On Wed, 5 Sep 2007, J. Bruce Fields wrote:
> >
> >> Well, this may just prove I'm an idiot, but one of the reasons I rarely
> >> run it is that I have trouble remembering exactly what it does; in
> >> particular,
> >>
> >> - does it prune anything that might be needed by a repo I
> >> cloned with -s?
> >
> > YES! yikes.
> >
> > This is about the best argument put forth so far for not
> > automatically running git-gc.
>
> Well, it could also mean that if git finds a dead symbolic link when
> looking up an object, it should check the corresponding link target
> directory for a pack file with the respective object... and if it
> finds such a pack file, create a link to it and use it.
One of the two of us is very confused about what "git-clone -s" does.
See the git-clone man page. I don't think symlinks are involved.
--b.
^ permalink raw reply
* [PATCH] Simplify write_tree using strbuf's.
From: Pierre Habouzit @ 2007-09-05 19:18 UTC (permalink / raw)
To: git; +Cc: Pierre Habouzit
In-Reply-To: <1189019923740-git-send-email-madcoder@debian.org>
Signed-off-by: Pierre Habouzit <madcoder@debian.org>
---
mktree.c | 22 +++++++---------------
1 files changed, 7 insertions(+), 15 deletions(-)
diff --git a/mktree.c b/mktree.c
index 86de5eb..3d11ac3 100644
--- a/mktree.c
+++ b/mktree.c
@@ -44,30 +44,22 @@ static int ent_compare(const void *a_, const void *b_)
static void write_tree(unsigned char *sha1)
{
- char *buffer;
- unsigned long size, offset;
+ struct strbuf buf = STRBUF_INIT;
+ size_t size;
int i;
qsort(entries, used, sizeof(*entries), ent_compare);
for (size = i = 0; i < used; i++)
size += 32 + entries[i]->len;
- buffer = xmalloc(size);
- offset = 0;
+ strbuf_grow(&buf, size);
for (i = 0; i < used; i++) {
struct treeent *ent = entries[i];
-
- if (offset + ent->len + 100 < size) {
- size = alloc_nr(offset + ent->len + 100);
- buffer = xrealloc(buffer, size);
- }
- offset += sprintf(buffer + offset, "%o ", ent->mode);
- offset += sprintf(buffer + offset, "%s", ent->name);
- buffer[offset++] = 0;
- hashcpy((unsigned char*)buffer + offset, ent->sha1);
- offset += 20;
+ strbuf_addf(&buf, "%o %s%c", ent->mode, ent->name, '\0');
+ strbuf_add(&buf, ent->sha1, 20);
}
- write_sha1_file(buffer, offset, tree_type, sha1);
+
+ write_sha1_file(buf.buf, buf.len, tree_type, sha1);
}
static const char mktree_usage[] = "git-mktree [-z]";
--
1.5.3
^ permalink raw reply related
* [PATCH] Simplify strbuf uses in archive-tar.c using the proper functions.
From: Pierre Habouzit @ 2007-09-05 19:18 UTC (permalink / raw)
To: git; +Cc: Pierre Habouzit
In-Reply-To: <11890199232110-git-send-email-madcoder@debian.org>
This is just cleaner way to deal with strbufs, using its API rather than
reinventing it in the module (e.g. strbuf_append_string is just the plain
strbuf_addstr function, and it was used to perform what strbuf_addch does
anyways).
---
archive-tar.c | 65 ++++++++++++++-------------------------------------------
1 files changed, 16 insertions(+), 49 deletions(-)
diff --git a/archive-tar.c b/archive-tar.c
index a0763c5..c84d7c0 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -78,19 +78,6 @@ static void write_trailer(void)
}
}
-static void strbuf_append_string(struct strbuf *sb, const char *s)
-{
- int slen = strlen(s);
- int total = sb->len + slen;
- if (total + 1 > sb->alloc) {
- sb->buf = xrealloc(sb->buf, total + 1);
- sb->alloc = total + 1;
- }
- memcpy(sb->buf + sb->len, s, slen);
- sb->len = total;
- sb->buf[total] = '\0';
-}
-
/*
* pax extended header records have the format "%u %s=%s\n". %u contains
* the size of the whole string (including the %u), the first %s is the
@@ -100,26 +87,17 @@ static void strbuf_append_string(struct strbuf *sb, const char *s)
static void strbuf_append_ext_header(struct strbuf *sb, const char *keyword,
const char *value, unsigned int valuelen)
{
- char *p;
- int len, total, tmp;
+ int len, tmp;
/* "%u %s=%s\n" */
len = 1 + 1 + strlen(keyword) + 1 + valuelen + 1;
for (tmp = len; tmp > 9; tmp /= 10)
len++;
- total = sb->len + len;
- if (total > sb->alloc) {
- sb->buf = xrealloc(sb->buf, total);
- sb->alloc = total;
- }
-
- p = sb->buf;
- p += sprintf(p, "%u %s=", len, keyword);
- memcpy(p, value, valuelen);
- p += valuelen;
- *p = '\n';
- sb->len = total;
+ strbuf_grow(sb, len);
+ strbuf_addf(sb, "%u %s=", len, keyword);
+ strbuf_add(sb, value, valuelen);
+ strbuf_addch(sb, '\n');
}
static unsigned int ustar_header_chksum(const struct ustar_header *header)
@@ -153,8 +131,7 @@ static void write_entry(const unsigned char *sha1, struct strbuf *path,
struct strbuf ext_header;
memset(&header, 0, sizeof(header));
- ext_header.buf = NULL;
- ext_header.len = ext_header.alloc = 0;
+ strbuf_init(&ext_header);
if (!sha1) {
*header.typeflag = TYPEFLAG_GLOBAL_HEADER;
@@ -225,8 +202,8 @@ static void write_entry(const unsigned char *sha1, struct strbuf *path,
if (ext_header.len > 0) {
write_entry(sha1, NULL, 0, ext_header.buf, ext_header.len);
- free(ext_header.buf);
}
+ strbuf_release(&ext_header);
write_blocked(&header, sizeof(header));
if (S_ISREG(mode) && buffer && size > 0)
write_blocked(buffer, size);
@@ -235,11 +212,11 @@ static void write_entry(const unsigned char *sha1, struct strbuf *path,
static void write_global_extended_header(const unsigned char *sha1)
{
struct strbuf ext_header;
- ext_header.buf = NULL;
- ext_header.len = ext_header.alloc = 0;
+
+ strbuf_init(&ext_header);
strbuf_append_ext_header(&ext_header, "comment", sha1_to_hex(sha1), 40);
write_entry(NULL, NULL, 0, ext_header.buf, ext_header.len);
- free(ext_header.buf);
+ strbuf_release(&ext_header);
}
static int git_tar_config(const char *var, const char *value)
@@ -260,28 +237,18 @@ static int write_tar_entry(const unsigned char *sha1,
const char *base, int baselen,
const char *filename, unsigned mode, int stage)
{
- static struct strbuf path;
+ static struct strbuf path = STRBUF_INIT;
int filenamelen = strlen(filename);
void *buffer;
enum object_type type;
unsigned long size;
- if (!path.alloc) {
- path.buf = xmalloc(PATH_MAX);
- path.alloc = PATH_MAX;
- path.len = path.eof = 0;
- }
- if (path.alloc < baselen + filenamelen + 1) {
- free(path.buf);
- path.buf = xmalloc(baselen + filenamelen + 1);
- path.alloc = baselen + filenamelen + 1;
- }
- memcpy(path.buf, base, baselen);
- memcpy(path.buf + baselen, filename, filenamelen);
- path.len = baselen + filenamelen;
- path.buf[path.len] = '\0';
+ strbuf_grow(&path, MAX(PATH_MAX, baselen + filenamelen + 1));
+ strbuf_reset(&path);
+ strbuf_add(&path, base, baselen);
+ strbuf_add(&path, filename, filenamelen);
if (S_ISDIR(mode) || S_ISGITLINK(mode)) {
- strbuf_append_string(&path, "/");
+ strbuf_addch(&path, '/');
buffer = NULL;
size = 0;
} else {
--
1.5.3
^ permalink raw reply related
* [PATCH] Use proper strbuf API, and also simplify cmd_data code.
From: Pierre Habouzit @ 2007-09-05 19:18 UTC (permalink / raw)
To: git; +Cc: Pierre Habouzit
In-Reply-To: <11890199232646-git-send-email-madcoder@debian.org>
This patch features the use of strbuf_detach, and prevent the programmer
to mess with allocation directly. The code is as efficent as before, just
more concise and more straightforward.
---
fast-import.c | 29 ++++++++++++-----------------
1 files changed, 12 insertions(+), 17 deletions(-)
diff --git a/fast-import.c b/fast-import.c
index 2f7baf4..1a02481 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -340,7 +340,7 @@ static struct tag *last_tag;
/* Input stream parsing */
static whenspec_type whenspec = WHENSPEC_RAW;
-static struct strbuf command_buf;
+static struct strbuf command_buf = STRBUF_INIT;
static int unread_command_buf;
static struct recent_command cmd_hist = {&cmd_hist, &cmd_hist, NULL};
static struct recent_command *cmd_tail = &cmd_hist;
@@ -1638,17 +1638,15 @@ static void cmd_mark(void)
static void *cmd_data (size_t *size)
{
- size_t length;
- char *buffer;
+ struct strbuf buffer = STRBUF_INIT;
if (prefixcmp(command_buf.buf, "data "))
die("Expected 'data n' command, found: %s", command_buf.buf);
if (!prefixcmp(command_buf.buf + 5, "<<")) {
char *term = xstrdup(command_buf.buf + 5 + 2);
- size_t sz = 8192, term_len = command_buf.len - 5 - 2;
- length = 0;
- buffer = xmalloc(sz);
+ size_t term_len = command_buf.len - 5 - 2;
+
for (;;) {
read_line(&command_buf, stdin, '\n');
if (command_buf.eof)
@@ -1656,21 +1654,18 @@ static void *cmd_data (size_t *size)
if (term_len == command_buf.len
&& !strcmp(term, command_buf.buf))
break;
- ALLOC_GROW(buffer, length + command_buf.len + 1, sz);
- memcpy(buffer + length,
- command_buf.buf,
- command_buf.len);
- length += command_buf.len;
- buffer[length++] = '\n';
+ strbuf_addbuf(&buffer, &command_buf);
+ strbuf_addch(&buffer, '\n');
}
free(term);
}
else {
- size_t n = 0;
+ size_t n = 0, length;
+
length = strtoul(command_buf.buf + 5, NULL, 10);
- buffer = xmalloc(length);
+
while (n < length) {
- size_t s = fread(buffer + n, 1, length - n, stdin);
+ size_t s = strbuf_fread(&buffer, length - n, stdin);
if (!s && feof(stdin))
die("EOF in data (%lu bytes remaining)",
(unsigned long)(length - n));
@@ -1679,8 +1674,8 @@ static void *cmd_data (size_t *size)
}
skip_optional_lf();
- *size = length;
- return buffer;
+ *size = buffer.len;
+ return strbuf_detach(&buffer);
}
static int validate_raw_date(const char *src, char *result, int maxlen)
--
1.5.3
^ permalink raw reply related
* [PATCH] Simplify strbuf uses in fast-import.c using the proper functions.
From: Pierre Habouzit @ 2007-09-05 19:18 UTC (permalink / raw)
To: git; +Cc: Pierre Habouzit
In-Reply-To: <11890199232128-git-send-email-madcoder@debian.org>
This is just cleaner way to deal with strbufs, using its API rather than
reinventing it in the module (e.g. strbuf_append_string is just the plain
strbuf_addstr function, and it was used to perform what strbuf_addch does
anyways).
---
archive-tar.c | 65 ++++++++++++++-------------------------------------------
1 files changed, 16 insertions(+), 49 deletions(-)
diff --git a/archive-tar.c b/archive-tar.c
index a0763c5..c84d7c0 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -78,19 +78,6 @@ static void write_trailer(void)
}
}
-static void strbuf_append_string(struct strbuf *sb, const char *s)
-{
- int slen = strlen(s);
- int total = sb->len + slen;
- if (total + 1 > sb->alloc) {
- sb->buf = xrealloc(sb->buf, total + 1);
- sb->alloc = total + 1;
- }
- memcpy(sb->buf + sb->len, s, slen);
- sb->len = total;
- sb->buf[total] = '\0';
-}
-
/*
* pax extended header records have the format "%u %s=%s\n". %u contains
* the size of the whole string (including the %u), the first %s is the
@@ -100,26 +87,17 @@ static void strbuf_append_string(struct strbuf *sb, const char *s)
static void strbuf_append_ext_header(struct strbuf *sb, const char *keyword,
const char *value, unsigned int valuelen)
{
- char *p;
- int len, total, tmp;
+ int len, tmp;
/* "%u %s=%s\n" */
len = 1 + 1 + strlen(keyword) + 1 + valuelen + 1;
for (tmp = len; tmp > 9; tmp /= 10)
len++;
- total = sb->len + len;
- if (total > sb->alloc) {
- sb->buf = xrealloc(sb->buf, total);
- sb->alloc = total;
- }
-
- p = sb->buf;
- p += sprintf(p, "%u %s=", len, keyword);
- memcpy(p, value, valuelen);
- p += valuelen;
- *p = '\n';
- sb->len = total;
+ strbuf_grow(sb, len);
+ strbuf_addf(sb, "%u %s=", len, keyword);
+ strbuf_add(sb, value, valuelen);
+ strbuf_addch(sb, '\n');
}
static unsigned int ustar_header_chksum(const struct ustar_header *header)
@@ -153,8 +131,7 @@ static void write_entry(const unsigned char *sha1, struct strbuf *path,
struct strbuf ext_header;
memset(&header, 0, sizeof(header));
- ext_header.buf = NULL;
- ext_header.len = ext_header.alloc = 0;
+ strbuf_init(&ext_header);
if (!sha1) {
*header.typeflag = TYPEFLAG_GLOBAL_HEADER;
@@ -225,8 +202,8 @@ static void write_entry(const unsigned char *sha1, struct strbuf *path,
if (ext_header.len > 0) {
write_entry(sha1, NULL, 0, ext_header.buf, ext_header.len);
- free(ext_header.buf);
}
+ strbuf_release(&ext_header);
write_blocked(&header, sizeof(header));
if (S_ISREG(mode) && buffer && size > 0)
write_blocked(buffer, size);
@@ -235,11 +212,11 @@ static void write_entry(const unsigned char *sha1, struct strbuf *path,
static void write_global_extended_header(const unsigned char *sha1)
{
struct strbuf ext_header;
- ext_header.buf = NULL;
- ext_header.len = ext_header.alloc = 0;
+
+ strbuf_init(&ext_header);
strbuf_append_ext_header(&ext_header, "comment", sha1_to_hex(sha1), 40);
write_entry(NULL, NULL, 0, ext_header.buf, ext_header.len);
- free(ext_header.buf);
+ strbuf_release(&ext_header);
}
static int git_tar_config(const char *var, const char *value)
@@ -260,28 +237,18 @@ static int write_tar_entry(const unsigned char *sha1,
const char *base, int baselen,
const char *filename, unsigned mode, int stage)
{
- static struct strbuf path;
+ static struct strbuf path = STRBUF_INIT;
int filenamelen = strlen(filename);
void *buffer;
enum object_type type;
unsigned long size;
- if (!path.alloc) {
- path.buf = xmalloc(PATH_MAX);
- path.alloc = PATH_MAX;
- path.len = path.eof = 0;
- }
- if (path.alloc < baselen + filenamelen + 1) {
- free(path.buf);
- path.buf = xmalloc(baselen + filenamelen + 1);
- path.alloc = baselen + filenamelen + 1;
- }
- memcpy(path.buf, base, baselen);
- memcpy(path.buf + baselen, filename, filenamelen);
- path.len = baselen + filenamelen;
- path.buf[path.len] = '\0';
+ strbuf_grow(&path, MAX(PATH_MAX, baselen + filenamelen + 1));
+ strbuf_reset(&path);
+ strbuf_add(&path, base, baselen);
+ strbuf_add(&path, filename, filenamelen);
if (S_ISDIR(mode) || S_ISGITLINK(mode)) {
- strbuf_append_string(&path, "/");
+ strbuf_addch(&path, '/');
buffer = NULL;
size = 0;
} else {
--
1.5.3
^ permalink raw reply related
* [PATCH] Further strbuf re-engineering.
From: Pierre Habouzit @ 2007-09-05 19:18 UTC (permalink / raw)
To: git; +Cc: Pierre Habouzit
In-Reply-To: <11890199233357-git-send-email-madcoder@debian.org>
Signed-off-by: Pierre Habouzit <madcoder@debian.org>
---
builtin-apply.c | 29 ++++++-----------------
builtin-blame.c | 34 +++++++++-------------------
builtin-commit-tree.c | 59 ++++++++++--------------------------------------
diff.c | 25 ++++++--------------
4 files changed, 40 insertions(+), 107 deletions(-)
diff --git a/builtin-apply.c b/builtin-apply.c
index 25b1447..4d8b270 100644
--- a/builtin-apply.c
+++ b/builtin-apply.c
@@ -12,6 +12,7 @@
#include "blob.h"
#include "delta.h"
#include "builtin.h"
+#include "strbuf.h"
/*
* --check turns on checking that the working tree matches the
@@ -181,34 +182,20 @@ static void say_patch_name(FILE *output, const char *pre, struct patch *patch, c
static void *read_patch_file(int fd, unsigned long *sizep)
{
- unsigned long size = 0, alloc = CHUNKSIZE;
- void *buffer = xmalloc(alloc);
+ struct strbuf buf = STRBUF_INIT;
- for (;;) {
- ssize_t nr = alloc - size;
- if (nr < 1024) {
- alloc += CHUNKSIZE;
- buffer = xrealloc(buffer, alloc);
- nr = alloc - size;
- }
- nr = xread(fd, (char *) buffer + size, nr);
- if (!nr)
- break;
- if (nr < 0)
- die("git-apply: read returned %s", strerror(errno));
- size += nr;
- }
- *sizep = size;
+ if (strbuf_read(&buf, fd) < 0)
+ die("git-apply: read returned %s", strerror(errno));
+ *sizep = buf.len;
/*
* Make sure that we have some slop in the buffer
* so that we can do speculative "memcmp" etc, and
* see to it that it is NUL-filled.
*/
- if (alloc < size + SLOP)
- buffer = xrealloc(buffer, size + SLOP);
- memset((char *) buffer + size, 0, SLOP);
- return buffer;
+ strbuf_grow(&buf, SLOP);
+ memset(buf.buf + buf.len, 0, SLOP);
+ return strbuf_detach(&buf);
}
static unsigned long linelen(const char *buffer, unsigned long size)
diff --git a/builtin-blame.c b/builtin-blame.c
index dc88a95..c2d36a1 100644
--- a/builtin-blame.c
+++ b/builtin-blame.c
@@ -18,6 +18,7 @@
#include "cache-tree.h"
#include "path-list.h"
#include "mailmap.h"
+#include "strbuf.h"
static char blame_usage[] =
"git-blame [-c] [-b] [-l] [--root] [-t] [-f] [-n] [-s] [-p] [-w] [-L n,m] [-S <revs-file>] [-M] [-C] [-C] [--contents <filename>] [--incremental] [commit] [--] file\n"
@@ -2001,11 +2002,10 @@ static struct commit *fake_working_tree_commit(const char *path, const char *con
struct commit *commit;
struct origin *origin;
unsigned char head_sha1[20];
- char *buf;
+ struct strbuf buf = STRBUF_INIT;
const char *ident;
int fd;
time_t now;
- unsigned long fin_size;
int size, len;
struct cache_entry *ce;
unsigned mode;
@@ -2026,6 +2026,7 @@ static struct commit *fake_working_tree_commit(const char *path, const char *con
if (!contents_from || strcmp("-", contents_from)) {
struct stat st;
const char *read_from;
+ unsigned long fin_size;
if (contents_from) {
if (stat(contents_from, &st) < 0)
@@ -2038,19 +2039,19 @@ static struct commit *fake_working_tree_commit(const char *path, const char *con
read_from = path;
}
fin_size = xsize_t(st.st_size);
- buf = xmalloc(fin_size+1);
mode = canon_mode(st.st_mode);
switch (st.st_mode & S_IFMT) {
case S_IFREG:
fd = open(read_from, O_RDONLY);
if (fd < 0)
die("cannot open %s", read_from);
- if (read_in_full(fd, buf, fin_size) != fin_size)
+ if (strbuf_read(&buf, fd) != xsize_t(st.st_size))
die("cannot read %s", read_from);
break;
case S_IFLNK:
- if (readlink(read_from, buf, fin_size+1) != fin_size)
+ if (readlink(read_from, buf.buf, buf.alloc) != fin_size)
die("cannot readlink %s", read_from);
+ buf.len = fin_size;
break;
default:
die("unsupported file type %s", read_from);
@@ -2059,26 +2060,13 @@ static struct commit *fake_working_tree_commit(const char *path, const char *con
else {
/* Reading from stdin */
contents_from = "standard input";
- buf = NULL;
- fin_size = 0;
mode = 0;
- while (1) {
- ssize_t cnt = 8192;
- buf = xrealloc(buf, fin_size + cnt);
- cnt = xread(0, buf + fin_size, cnt);
- if (cnt < 0)
- die("read error %s from stdin",
- strerror(errno));
- if (!cnt)
- break;
- fin_size += cnt;
- }
- buf = xrealloc(buf, fin_size + 1);
+ if (strbuf_read(&buf, 0) < 0)
+ die("read error %s from stdin", strerror(errno));
}
- buf[fin_size] = 0;
- origin->file.ptr = buf;
- origin->file.size = fin_size;
- pretend_sha1_file(buf, fin_size, OBJ_BLOB, origin->blob_sha1);
+ origin->file.ptr = buf.buf;
+ origin->file.size = buf.len;
+ pretend_sha1_file(buf.buf, buf.len, OBJ_BLOB, origin->blob_sha1);
commit->util = origin;
/*
diff --git a/builtin-commit-tree.c b/builtin-commit-tree.c
index ccbcbe3..ee74814 100644
--- a/builtin-commit-tree.c
+++ b/builtin-commit-tree.c
@@ -8,42 +8,13 @@
#include "tree.h"
#include "builtin.h"
#include "utf8.h"
+#include "strbuf.h"
#define BLOCKING (1ul << 14)
/*
* FIXME! Share the code with "write-tree.c"
*/
-static void init_buffer(char **bufp, unsigned int *sizep)
-{
- *bufp = xmalloc(BLOCKING);
- *sizep = 0;
-}
-
-static void add_buffer(char **bufp, unsigned int *sizep, const char *fmt, ...)
-{
- char one_line[2048];
- va_list args;
- int len;
- unsigned long alloc, size, newsize;
- char *buf;
-
- va_start(args, fmt);
- len = vsnprintf(one_line, sizeof(one_line), fmt, args);
- va_end(args);
- size = *sizep;
- newsize = size + len + 1;
- alloc = (size + 32767) & ~32767;
- buf = *bufp;
- if (newsize > alloc) {
- alloc = (newsize + 32767) & ~32767;
- buf = xrealloc(buf, alloc);
- *bufp = buf;
- }
- *sizep = newsize - 1;
- memcpy(buf + size, one_line, len);
-}
-
static void check_valid(unsigned char *sha1, enum object_type expect)
{
enum object_type type = sha1_object_info(sha1, NULL);
@@ -87,9 +58,7 @@ int cmd_commit_tree(int argc, const char **argv, const char *prefix)
int parents = 0;
unsigned char tree_sha1[20];
unsigned char commit_sha1[20];
- char comment[1000];
- char *buffer;
- unsigned int size;
+ struct strbuf buffer = STRBUF_INIT;
int encoding_is_utf8;
git_config(git_default_config);
@@ -118,8 +87,8 @@ int cmd_commit_tree(int argc, const char **argv, const char *prefix)
/* Not having i18n.commitencoding is the same as having utf-8 */
encoding_is_utf8 = is_encoding_utf8(git_commit_encoding);
- init_buffer(&buffer, &size);
- add_buffer(&buffer, &size, "tree %s\n", sha1_to_hex(tree_sha1));
+ strbuf_grow(&buffer, 8192); /* should avoid reallocs for the headers */
+ strbuf_addf(&buffer, "tree %s\n", sha1_to_hex(tree_sha1));
/*
* NOTE! This ordering means that the same exact tree merged with a
@@ -127,26 +96,24 @@ int cmd_commit_tree(int argc, const char **argv, const char *prefix)
* if everything else stays the same.
*/
for (i = 0; i < parents; i++)
- add_buffer(&buffer, &size, "parent %s\n", sha1_to_hex(parent_sha1[i]));
+ strbuf_addf(&buffer, "parent %s\n", sha1_to_hex(parent_sha1[i]));
/* Person/date information */
- add_buffer(&buffer, &size, "author %s\n", git_author_info(1));
- add_buffer(&buffer, &size, "committer %s\n", git_committer_info(1));
+ strbuf_addf(&buffer, "author %s\n", git_author_info(1));
+ strbuf_addf(&buffer, "committer %s\n", git_committer_info(1));
if (!encoding_is_utf8)
- add_buffer(&buffer, &size,
- "encoding %s\n", git_commit_encoding);
- add_buffer(&buffer, &size, "\n");
+ strbuf_addf(&buffer, "encoding %s\n", git_commit_encoding);
+ strbuf_addch(&buffer, '\n');
/* And add the comment */
- while (fgets(comment, sizeof(comment), stdin) != NULL)
- add_buffer(&buffer, &size, "%s", comment);
+ if (strbuf_read(&buffer, 0) < 0)
+ die("git-commit-tree: read returned %s", strerror(errno));
/* And check the encoding */
- buffer[size] = '\0';
- if (encoding_is_utf8 && !is_utf8(buffer))
+ if (encoding_is_utf8 && !is_utf8(buffer.buf))
fprintf(stderr, commit_utf8_warn);
- if (!write_sha1_file(buffer, size, commit_type, commit_sha1)) {
+ if (!write_sha1_file(buffer.buf, buffer.len, commit_type, commit_sha1)) {
printf("%s\n", sha1_to_hex(commit_sha1));
return 0;
}
diff --git a/diff.c b/diff.c
index 0d30d05..05721f8 100644
--- a/diff.c
+++ b/diff.c
@@ -9,6 +9,7 @@
#include "xdiff-interface.h"
#include "color.h"
#include "attr.h"
+#include "strbuf.h"
#ifdef NO_FAST_WORKING_DIRECTORY
#define FAST_WORKING_DIRECTORY 0
@@ -1546,25 +1547,15 @@ static int reuse_worktree_file(const char *name, const unsigned char *sha1, int
static int populate_from_stdin(struct diff_filespec *s)
{
#define INCREMENT 1024
- char *buf;
- unsigned long size;
- ssize_t got;
-
- size = 0;
- buf = NULL;
- while (1) {
- buf = xrealloc(buf, size + INCREMENT);
- got = xread(0, buf + size, INCREMENT);
- if (!got)
- break; /* EOF */
- if (got < 0)
- return error("error while reading from stdin %s",
+ struct strbuf buf = STRBUF_INIT;
+
+ if (strbuf_read(&buf, 0) < 0)
+ return error("error while reading from stdin %s",
strerror(errno));
- size += got;
- }
+
s->should_munmap = 0;
- s->data = buf;
- s->size = size;
+ s->size = buf.len;
+ s->data = strbuf_detach(&buf);
s->should_free = 1;
return 0;
}
--
1.5.3
^ permalink raw reply related
* [PATCH] Eradicate yet-another-buffer implementation in buitin-rerere.c
From: Pierre Habouzit @ 2007-09-05 19:18 UTC (permalink / raw)
To: git; +Cc: Pierre Habouzit
In-Reply-To: <1189019923943-git-send-email-madcoder@debian.org>
Signed-off-by: Pierre Habouzit <madcoder@debian.org>
---
builtin-rerere.c | 53 +++++++++++++++--------------------------------------
1 files changed, 15 insertions(+), 38 deletions(-)
diff --git a/builtin-rerere.c b/builtin-rerere.c
index 29d057c..2025004 100644
--- a/builtin-rerere.c
+++ b/builtin-rerere.c
@@ -1,6 +1,7 @@
#include "builtin.h"
#include "cache.h"
#include "path-list.h"
+#include "strbuf.h"
#include "xdiff/xdiff.h"
#include "xdiff-interface.h"
@@ -66,38 +67,14 @@ static int write_rr(struct path_list *rr, int out_fd)
return commit_lock_file(&write_lock);
}
-struct buffer {
- char *ptr;
- int nr, alloc;
-};
-
-static void append_line(struct buffer *buffer, const char *line)
-{
- int len = strlen(line);
-
- if (buffer->nr + len > buffer->alloc) {
- buffer->alloc = alloc_nr(buffer->nr + len);
- buffer->ptr = xrealloc(buffer->ptr, buffer->alloc);
- }
- memcpy(buffer->ptr + buffer->nr, line, len);
- buffer->nr += len;
-}
-
-static void clear_buffer(struct buffer *buffer)
-{
- free(buffer->ptr);
- buffer->ptr = NULL;
- buffer->nr = buffer->alloc = 0;
-}
-
static int handle_file(const char *path,
unsigned char *sha1, const char *output)
{
SHA_CTX ctx;
char buf[1024];
int hunk = 0, hunk_no = 0;
- struct buffer minus = { NULL, 0, 0 }, plus = { NULL, 0, 0 };
- struct buffer *one = &minus, *two = +
+ struct strbuf minus = STRBUF_INIT, plus = STRBUF_INIT;
+ struct strbuf *one = &minus, *two = +
FILE *f = fopen(path, "r");
FILE *out;
@@ -122,36 +99,36 @@ static int handle_file(const char *path,
else if (!prefixcmp(buf, "======="))
hunk = 2;
else if (!prefixcmp(buf, ">>>>>>> ")) {
- int one_is_longer = (one->nr > two->nr);
- int common_len = one_is_longer ? two->nr : one->nr;
- int cmp = memcmp(one->ptr, two->ptr, common_len);
+ int one_is_longer = (one->len > two->len);
+ int common_len = one_is_longer ? two->len : one->len;
+ int cmp = memcmp(one->buf, two->buf, common_len);
hunk_no++;
hunk = 0;
if ((cmp > 0) || ((cmp == 0) && one_is_longer)) {
- struct buffer *swap = one;
+ struct strbuf *swap = one;
one = two;
two = swap;
}
if (out) {
fputs("<<<<<<<\n", out);
- fwrite(one->ptr, one->nr, 1, out);
+ fwrite(one->buf, one->len, 1, out);
fputs("=======\n", out);
- fwrite(two->ptr, two->nr, 1, out);
+ fwrite(two->buf, two->len, 1, out);
fputs(">>>>>>>\n", out);
}
if (sha1) {
- SHA1_Update(&ctx, one->ptr, one->nr);
+ SHA1_Update(&ctx, one->buf, one->len);
SHA1_Update(&ctx, "\0", 1);
- SHA1_Update(&ctx, two->ptr, two->nr);
+ SHA1_Update(&ctx, two->buf, two->len);
SHA1_Update(&ctx, "\0", 1);
}
- clear_buffer(one);
- clear_buffer(two);
+ strbuf_release(one);
+ strbuf_release(two);
} else if (hunk == 1)
- append_line(one, buf);
+ strbuf_addstr(one, buf);
else if (hunk == 2)
- append_line(two, buf);
+ strbuf_addstr(two, buf);
else if (out)
fputs(buf, out);
}
--
1.5.3
^ permalink raw reply related
* [PATCH] More strbuf uses in cache-tree.c.
From: Pierre Habouzit @ 2007-09-05 19:18 UTC (permalink / raw)
To: git; +Cc: Pierre Habouzit
In-Reply-To: <1189019923662-git-send-email-madcoder@debian.org>
Should even be marginally faster.
Signed-off-by: Pierre Habouzit <madcoder@debian.org>
---
cache-tree.c | 57 ++++++++++++++++++++-------------------------------------
1 files changed, 20 insertions(+), 37 deletions(-)
diff --git a/cache-tree.c b/cache-tree.c
index 077f034..27ca062 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -1,4 +1,5 @@
#include "cache.h"
+#include "strbuf.h"
#include "tree.h"
#include "cache-tree.h"
@@ -235,8 +236,7 @@ static int update_one(struct cache_tree *it,
int missing_ok,
int dryrun)
{
- unsigned long size, offset;
- char *buffer;
+ struct strbuf buffer = STRBUF_INIT;
int i;
if (0 <= it->entry_count && has_sha1_file(it->sha1))
@@ -293,9 +293,7 @@ static int update_one(struct cache_tree *it,
/*
* Then write out the tree object for this level.
*/
- size = 8192;
- buffer = xmalloc(size);
- offset = 0;
+ strbuf_grow(&buffer, 8192);
for (i = 0; i < entries; i++) {
struct cache_entry *ce = cache[i];
@@ -332,15 +330,9 @@ static int update_one(struct cache_tree *it,
if (!ce->ce_mode)
continue; /* entry being removed */
- if (size < offset + entlen + 100) {
- size = alloc_nr(offset + entlen + 100);
- buffer = xrealloc(buffer, size);
- }
- offset += sprintf(buffer + offset,
- "%o %.*s", mode, entlen, path + baselen);
- buffer[offset++] = 0;
- hashcpy((unsigned char*)buffer + offset, sha1);
- offset += 20;
+ strbuf_grow(&buffer, entlen + 100);
+ strbuf_addf(&buffer, "%o %.*s%c", mode, entlen, path + baselen, '\0');
+ strbuf_add(&buffer, sha1, 20);
#if DEBUG
fprintf(stderr, "cache-tree update-one %o %.*s\n",
@@ -349,10 +341,10 @@ static int update_one(struct cache_tree *it,
}
if (dryrun)
- hash_sha1_file(buffer, offset, tree_type, it->sha1);
+ hash_sha1_file(buffer.buf, buffer.len, tree_type, it->sha1);
else
- write_sha1_file(buffer, offset, tree_type, it->sha1);
- free(buffer);
+ write_sha1_file(buffer.buf, buffer.len, tree_type, it->sha1);
+ strbuf_release(&buffer);
it->entry_count = i;
#if DEBUG
fprintf(stderr, "cache-tree update-one (%d ent, %d subtree) %s\n",
@@ -378,12 +370,10 @@ int cache_tree_update(struct cache_tree *it,
return 0;
}
-static void *write_one(struct cache_tree *it,
+static void write_one(struct cache_tree *it,
char *path,
int pathlen,
- char *buffer,
- unsigned long *size,
- unsigned long *offset)
+ struct strbuf *buffer)
{
int i;
@@ -393,13 +383,9 @@ static void *write_one(struct cache_tree *it,
* tree-sha1 (missing if invalid)
* subtree_nr "cache-tree" entries for subtrees.
*/
- if (*size < *offset + pathlen + 100) {
- *size = alloc_nr(*offset + pathlen + 100);
- buffer = xrealloc(buffer, *size);
- }
- *offset += sprintf(buffer + *offset, "%.*s%c%d %d\n",
- pathlen, path, 0,
- it->entry_count, it->subtree_nr);
+ strbuf_grow(buffer, pathlen + 100);
+ strbuf_add(buffer, path, pathlen);
+ strbuf_addf(buffer, "%c%d %d\n", 0, it->entry_count, it->subtree_nr);
#if DEBUG
if (0 <= it->entry_count)
@@ -412,8 +398,7 @@ static void *write_one(struct cache_tree *it,
#endif
if (0 <= it->entry_count) {
- hashcpy((unsigned char*)buffer + *offset, it->sha1);
- *offset += 20;
+ strbuf_add(buffer, it->sha1, 20);
}
for (i = 0; i < it->subtree_nr; i++) {
struct cache_tree_sub *down = it->down[i];
@@ -423,21 +408,19 @@ static void *write_one(struct cache_tree *it,
prev->name, prev->namelen) <= 0)
die("fatal - unsorted cache subtree");
}
- buffer = write_one(down->cache_tree, down->name, down->namelen,
- buffer, size, offset);
+ write_one(down->cache_tree, down->name, down->namelen, buffer);
}
- return buffer;
}
void *cache_tree_write(struct cache_tree *root, unsigned long *size_p)
{
char path[PATH_MAX];
- unsigned long size = 8192;
- char *buffer = xmalloc(size);
+ struct strbuf buffer = STRBUF_INIT;
- *size_p = 0;
path[0] = 0;
- return write_one(root, path, 0, buffer, &size, size_p);
+ write_one(root, path, 0, &buffer);
+ *size_p = buffer.len;
+ return strbuf_detach(&buffer);
}
static struct cache_tree *read_one(const char **buffer, unsigned long *size_p)
--
1.5.3
^ permalink raw reply related
* [PATCH] Rework strbuf API and semantics.
From: Pierre Habouzit @ 2007-09-05 19:18 UTC (permalink / raw)
To: git; +Cc: Pierre Habouzit
In-Reply-To: <20070905085720.GD31750@artemis.corp>
A strbuf can be used to store byte arrays, or as an extended string
library. The `buf' member can be passed to any C legacy string function,
because strbuf operations always ensure there is a terminating \0 at the end
of the buffer, not accounted in the `len' field of the structure.
A strbuf can be used to generate a string/buffer whose final size is not
really known, and then "strbuf_detach" can be used to get the built buffer,
and keep the wrapping "strbuf" structure usable for further work again.
Other interesting feature: strbuf_grow(sb, size) ensure that there is
enough allocated space in `sb' to put `size' new octets of data in the
buffer. It helps avoiding reallocating data for nothing when the problem the
strbuf helps to solve has a known typical size.
Signed-off-by: Pierre Habouzit <madcoder@debian.org>
---
archive-tar.c | 2 +-
fast-import.c | 15 ++++----
mktree.c | 4 +--
strbuf.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++++--------
strbuf.h | 42 ++++++++++++++++++++++-
5 files changed, 138 insertions(+), 28 deletions(-)
diff --git a/archive-tar.c b/archive-tar.c
index 66fe3e3..a0763c5 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -166,7 +166,7 @@ static void write_entry(const unsigned char *sha1, struct strbuf *path,
sprintf(header.name, "%s.paxheader", sha1_to_hex(sha1));
} else {
if (verbose)
- fprintf(stderr, "%.*s\n", path->len, path->buf);
+ fprintf(stderr, "%.*s\n", (int)path->len, path->buf);
if (S_ISDIR(mode) || S_ISGITLINK(mode)) {
*header.typeflag = TYPEFLAG_DIR;
mode = (mode | 0777) & ~tar_umask;
diff --git a/fast-import.c b/fast-import.c
index 078079d..2f7baf4 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1595,7 +1595,7 @@ static void read_next_command(void)
} else {
struct recent_command *rc;
- command_buf.buf = NULL;
+ strbuf_detach(&command_buf);
read_line(&command_buf, stdin, '\n');
if (command_buf.eof)
return;
@@ -1649,7 +1649,6 @@ static void *cmd_data (size_t *size)
size_t sz = 8192, term_len = command_buf.len - 5 - 2;
length = 0;
buffer = xmalloc(sz);
- command_buf.buf = NULL;
for (;;) {
read_line(&command_buf, stdin, '\n');
if (command_buf.eof)
@@ -1657,11 +1656,11 @@ static void *cmd_data (size_t *size)
if (term_len == command_buf.len
&& !strcmp(term, command_buf.buf))
break;
- ALLOC_GROW(buffer, length + command_buf.len, sz);
+ ALLOC_GROW(buffer, length + command_buf.len + 1, sz);
memcpy(buffer + length,
command_buf.buf,
- command_buf.len - 1);
- length += command_buf.len - 1;
+ command_buf.len);
+ length += command_buf.len;
buffer[length++] = '\n';
}
free(term);
@@ -2101,7 +2100,7 @@ static void cmd_new_commit(void)
}
/* file_change* */
- while (!command_buf.eof && command_buf.len > 1) {
+ while (!command_buf.eof && command_buf.len > 0) {
if (!prefixcmp(command_buf.buf, "M "))
file_change_m(b);
else if (!prefixcmp(command_buf.buf, "D "))
@@ -2256,7 +2255,7 @@ static void cmd_reset_branch(void)
else
b = new_branch(sp);
read_next_command();
- if (!cmd_from(b) && command_buf.len > 1)
+ if (!cmd_from(b) && command_buf.len > 0)
unread_command_buf = 1;
}
@@ -2273,7 +2272,7 @@ static void cmd_checkpoint(void)
static void cmd_progress(void)
{
- fwrite(command_buf.buf, 1, command_buf.len - 1, stdout);
+ fwrite(command_buf.buf, 1, command_buf.len, stdout);
fputc('\n', stdout);
fflush(stdout);
skip_optional_lf();
diff --git a/mktree.c b/mktree.c
index d86dde8..86de5eb 100644
--- a/mktree.c
+++ b/mktree.c
@@ -92,7 +92,6 @@ int main(int ac, char **av)
strbuf_init(&sb);
while (1) {
- int len;
char *ptr, *ntr;
unsigned mode;
enum object_type type;
@@ -101,7 +100,6 @@ int main(int ac, char **av)
read_line(&sb, stdin, line_termination);
if (sb.eof)
break;
- len = sb.len;
ptr = sb.buf;
/* Input is non-recursive ls-tree output format
* mode SP type SP sha1 TAB name
@@ -111,7 +109,7 @@ int main(int ac, char **av)
die("input format error: %s", sb.buf);
ptr = ntr + 1; /* type */
ntr = strchr(ptr, ' ');
- if (!ntr || sb.buf + len <= ntr + 41 ||
+ if (!ntr || sb.buf + sb.len <= ntr + 40 ||
ntr[41] != '\t' ||
get_sha1_hex(ntr + 1, sha1))
die("input format error: %s", sb.buf);
diff --git a/strbuf.c b/strbuf.c
index e33d06b..7866fbe 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -2,40 +2,115 @@
#include "strbuf.h"
void strbuf_init(struct strbuf *sb) {
- sb->buf = NULL;
- sb->eof = sb->alloc = sb->len = 0;
+ memset(sb, 0, sizeof(*sb));
}
-static void strbuf_begin(struct strbuf *sb) {
+void strbuf_release(struct strbuf *sb) {
free(sb->buf);
+ memset(sb, 0, sizeof(*sb));
+}
+
+void strbuf_reset(struct strbuf *sb) {
+ if (sb->len)
+ sb->buf[sb->len = 0] = '\0';
+}
+
+char *strbuf_detach(struct strbuf *sb) {
+ char *res = sb->buf;
strbuf_init(sb);
+ return res;
+}
+
+void strbuf_grow(struct strbuf *sb, size_t extra) {
+ if (sb->len + extra + 1 < sb->len)
+ die("you want to use way too much memory");
+ ALLOC_GROW(sb->buf, sb->len + extra + 1, sb->alloc);
+}
+
+void strbuf_add(struct strbuf *sb, const void *data, size_t len) {
+ strbuf_grow(sb, len);
+ memcpy(sb->buf + sb->len, data, len);
+ sb->len += len;
+ sb->buf[sb->len] = '\0';
+}
+
+void strbuf_addvf(struct strbuf *sb, const char *fmt, va_list ap)
+{
+ size_t len;
+ va_list ap2;
+
+ va_copy(ap2, ap);
+
+ len = vsnprintf(sb->buf + sb->len, sb->alloc - sb->len, fmt, ap);
+ if (len < 0) {
+ len = 0;
+ }
+ if (len >= sb->alloc - sb->len) {
+ strbuf_grow(sb, len);
+ len = vsnprintf(sb->buf + sb->len, sb->alloc - sb->len, fmt, ap2);
+ if (len >= sb->alloc - sb->len) {
+ len = sb->alloc - sb->len - 1;
+ }
+ }
+ sb->len = sb->len + len;
+ sb->buf[sb->len] = '\0';
}
-static void inline strbuf_add(struct strbuf *sb, int ch) {
- if (sb->alloc <= sb->len) {
- sb->alloc = sb->alloc * 3 / 2 + 16;
- sb->buf = xrealloc(sb->buf, sb->alloc);
+size_t strbuf_fread(struct strbuf *sb, size_t size, FILE *f) {
+ size_t res;
+
+ strbuf_grow(sb, size);
+ res = fread(sb->buf + sb->len, 1, size, f);
+ if (res > 0) {
+ sb->len += res;
+ sb->buf[sb->len] = '\0';
}
- sb->buf[sb->len++] = ch;
+ return res;
}
-static void strbuf_end(struct strbuf *sb) {
- strbuf_add(sb, 0);
+ssize_t strbuf_read(struct strbuf *sb, int fd)
+{
+ size_t oldlen = sb->len;
+
+ for (;;) {
+ ssize_t cnt;
+
+ strbuf_grow(sb, 8192);
+ cnt = xread(fd, sb->buf + sb->len, sb->alloc - sb->len - 1);
+ if (cnt < 0) {
+ sb->buf[sb->len = oldlen] = '\0';
+ return -1;
+ }
+ if (!cnt)
+ break;
+ sb->len += cnt;
+ }
+
+ sb->buf[sb->len] = '\0';
+ return sb->len - oldlen;
}
void read_line(struct strbuf *sb, FILE *fp, int term) {
int ch;
- strbuf_begin(sb);
if (feof(fp)) {
+ strbuf_release(sb);
sb->eof = 1;
return;
}
+
+ strbuf_reset(sb);
while ((ch = fgetc(fp)) != EOF) {
if (ch == term)
break;
- strbuf_add(sb, ch);
+ strbuf_grow(sb, 1);
+ sb->buf[sb->len++] = ch;
}
- if (ch == EOF && sb->len == 0)
+ if (ch == EOF && sb->len == 0) {
+ strbuf_release(sb);
sb->eof = 1;
- strbuf_end(sb);
+ }
+
+ strbuf_grow(sb, 1);
+ sb->buf[sb->len] = '\0';
}
+
diff --git a/strbuf.h b/strbuf.h
index 74cc012..db1e438 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -1,13 +1,51 @@
#ifndef STRBUF_H
#define STRBUF_H
struct strbuf {
- int alloc;
- int len;
+ size_t alloc;
+ size_t len;
int eof;
char *buf;
};
+#define STRBUF_INIT { 0, 0, 0, NULL }
+
+/* strbuf life cycle */
extern void strbuf_init(struct strbuf *);
+extern void strbuf_release(struct strbuf *);
+extern void strbuf_reset(struct strbuf *);
+extern char *strbuf_detach(struct strbuf *);
+
+
+extern void strbuf_grow(struct strbuf *, size_t);
+extern void strbuf_add(struct strbuf *, const void *, size_t);
+
+static inline void strbuf_addstr(struct strbuf *sb, const char *s) {
+ strbuf_add(sb, s, strlen(s));
+}
+static inline void strbuf_addbuf(struct strbuf *sb, struct strbuf *sb2) {
+ strbuf_add(sb, sb2->buf, sb2->len);
+}
+static inline void strbuf_addch(struct strbuf *sb, size_t c) {
+ strbuf_grow(sb, 1);
+ sb->buf[sb->len++] = c;
+ sb->buf[sb->len] = '\0';
+}
+
+__attribute__((format(printf,2,0)))
+extern void strbuf_addvf(struct strbuf *, const char *, va_list);
+
+static inline __attribute__((format(printf,2,3)))
+void strbuf_addf(struct strbuf *sb, const char *fmt, ...) {
+ va_list ap;
+ va_start(ap, fmt);
+ strbuf_addvf(sb, fmt, ap);
+ va_end(ap);
+}
+
+
+extern size_t strbuf_fread(struct strbuf *, size_t, FILE *);
+extern ssize_t strbuf_read(struct strbuf *, int fd);
+
extern void read_line(struct strbuf *, FILE *, int);
#endif /* STRBUF_H */
--
1.5.3
^ permalink raw reply related
* Re: [PATCH] Simplify strbuf uses in fast-import.c using the proper functions.
From: Pierre Habouzit @ 2007-09-05 19:21 UTC (permalink / raw)
To: git
In-Reply-To: <11890199232646-git-send-email-madcoder@debian.org>
[-- Attachment #1: Type: text/plain, Size: 570 bytes --]
Sorry, this one is a false positive :|
I should have been more careful.
On Wed, Sep 05, 2007 at 07:18:38PM +0000, Pierre Habouzit wrote:
> This is just cleaner way to deal with strbufs, using its API rather than
> reinventing it in the module (e.g. strbuf_append_string is just the plain
> strbuf_addstr function, and it was used to perform what strbuf_addch does
> anyways).
--
·O· Pierre Habouzit
··O madcoder@debian.org
OOO http://www.madism.org
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Mike Hommey @ 2007-09-05 19:20 UTC (permalink / raw)
To: David Kastrup
Cc: Brandon Casey, J. Bruce Fields, Linus Torvalds, Git Mailing List
In-Reply-To: <85642phqtn.fsf@lola.goethe.zz>
On Wed, Sep 05, 2007 at 09:09:40PM +0200, David Kastrup <dak@gnu.org> wrote:
> Brandon Casey <casey@nrlssc.navy.mil> writes:
>
> > On Wed, 5 Sep 2007, J. Bruce Fields wrote:
> >
> >> Well, this may just prove I'm an idiot, but one of the reasons I rarely
> >> run it is that I have trouble remembering exactly what it does; in
> >> particular,
> >>
> >> - does it prune anything that might be needed by a repo I
> >> cloned with -s?
> >
> > YES! yikes.
> >
> > This is about the best argument put forth so far for not
> > automatically running git-gc.
>
> Well, it could also mean that if git finds a dead symbolic link when
> looking up an object, it should check the corresponding link target
> directory for a pack file with the respective object... and if it
> finds such a pack file, create a link to it and use it.
The problem here is that the clone could be having refs on objects from
the origin that don't have refs left there. git-gc might, at some point,
prune these refs, and the clone would have dangling refs. That could
easily happen, for example, if you rebase a branch in the origin, but
still have a clone with the original branch.
Mike
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: David Kastrup @ 2007-09-05 19:43 UTC (permalink / raw)
To: J. Bruce Fields; +Cc: Brandon Casey, Linus Torvalds, Git Mailing List
In-Reply-To: <20070905191310.GE13314@fieldses.org>
"J. Bruce Fields" <bfields@fieldses.org> writes:
> On Wed, Sep 05, 2007 at 09:09:40PM +0200, David Kastrup wrote:
>> Brandon Casey <casey@nrlssc.navy.mil> writes:
>>
>> > On Wed, 5 Sep 2007, J. Bruce Fields wrote:
>> >
>> >> Well, this may just prove I'm an idiot, but one of the reasons I rarely
>> >> run it is that I have trouble remembering exactly what it does; in
>> >> particular,
>> >>
>> >> - does it prune anything that might be needed by a repo I
>> >> cloned with -s?
>> >
>> > YES! yikes.
>> >
>> > This is about the best argument put forth so far for not
>> > automatically running git-gc.
>>
>> Well, it could also mean that if git finds a dead symbolic link when
>> looking up an object, it should check the corresponding link target
>> directory for a pack file with the respective object... and if it
>> finds such a pack file, create a link to it and use it.
>
> One of the two of us is very confused about what "git-clone -s" does.
> See the git-clone man page. I don't think symlinks are involved.
Guilty as charged.
--
David Kastrup, Kriemhildstr. 15, 44793 Bochum
^ permalink raw reply
* [PATCH] Clean up .dotest if an error occurs before rebase work starts
From: Robin Rosenberg @ 2007-09-05 19:27 UTC (permalink / raw)
To: junkio; +Cc: git, Robin Rosenberg
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
---
git-rebase.sh | 18 ++++++++++++++----
1 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/git-rebase.sh b/git-rebase.sh
index 3bd66b0..81c48d1 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -253,23 +253,33 @@ else
fi
# The tree must be really really clean.
-git update-index --refresh || exit
+git update-index --refresh || {
+ rm -rf .dotest
+ exit 1
+}
+
diff=$(git diff-index --cached --name-status -r HEAD)
case "$diff" in
?*) echo "cannot rebase: your index is not up-to-date"
echo "$diff"
+ rm -rf .dotest
exit 1
;;
esac
# The upstream head must be given. Make sure it is valid.
upstream_name="$1"
-upstream=`git rev-parse --verify "${upstream_name}^0"` ||
- die "invalid upstream $upstream_name"
+upstream=`git rev-parse --verify "${upstream_name}^0"` || {
+ rm -rf .dotest
+ die "invalid upstream $upstream_name"
+}
# Make sure the branch to rebase onto is valid.
onto_name=${newbase-"$upstream_name"}
-onto=$(git rev-parse --verify "${onto_name}^0") || exit
+onto=$(git rev-parse --verify "${onto_name}^0") || {
+ rm -rf .dotest
+ exit 1
+}
# If a hook exists, give it a chance to interrupt
if test -x "$GIT_DIR/hooks/pre-rebase"
--
1.5.3
^ permalink raw reply related
* Re: best practice for a 3 devs team?
From: Jan Hudec @ 2007-09-05 19:50 UTC (permalink / raw)
To: Patrick Aljord; +Cc: git
In-Reply-To: <6b6419750708281520u12857b4j3e8f1ca0508f7d51@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 4037 bytes --]
On Tue, Aug 28, 2007 at 17:20:25 -0500, Patrick Aljord wrote:
> Hey all,
> We are 3 developers working on the same project. I was thinking what
> would be the best practice for us. We have just switched from svn and
> for now we are all working on the master branch that we push to a ssh
> git repository.
> I was thinking about maybe each of us could create our own branch like
> 'git branch <dev_name>' but then how would it go? Each one of us would
> push our own branch to the remote repository and each one of us could
> cherry pick the change from the remote branch of each developer. Is
> this the best way to do it? (I guess there is no best way but still :)
>
> any advice welcome
I'd vote against 1-to-1 mapping between developers and branches. It's
either needlessly inflexible or needlessly complicated, depending how quickly
the mainline changes and how stable it needs to be.
For project under heavy development (where occasional breakage won't hurt too
much and fast progress is desired), I'd recommend you to just keep pushing to
master everyone. This makes new changes immediately visible -- and tested --
by others, so you notice errors quickly. It also makes the workflow easiest,
since you only rebase on master and push it (I'd recommend rebasing work in
progress instead of merging to make the history easier to read). You can
occasionally create a feature branch in the central repository if you want
someone else to test your work in progress, but making your local
repositories readable by the others would work just as well.
On the other hand for project that is expected to provide stable releases,
especially if it's supposed to provide them often, something similar to what
Junio uses for git seems most reasonable. That is, you'd have a master (or
stable) branch for the oncoming release, one or more branches for testing and
a lot of feature branches that would come and go (yes -- in git when you
merge a feature branch for final, you can, and should, just remove it!). Each
developer could have a prefix assigned to avoid conflicts in feature branch
names (Junio currently uses initials of author as prefix).
For every logical change you do, you'd push a feature branch to the central
repository. Than somebody else would pick it up, review it and if it seems
sane, merge it to the most experimental branch. That branch would than be
subjected to testing. Junio currently reverts changes from this most
experimental branch (called 'pu' (proposed updates)) by deleting it,
branching it from 'next' again and re-merging all the still prospective
feature branches in again. This (re-creating a branch) is called rewinding
and you must be careful not to branch of it, or if you do, use rebase rather
than merge with it, but you should not base work on this experimental branch
anyway -- it only exists for creating the test build.
Now the individual feature branches that have seen enough testing in this
experimental branch will go, depending on how careful you need to be, either
to the master, or to a branch, that will be subjected to system test and
become next master after current release (this is the 'next' branch in git).
This branch should now be available for basing further development on, so you
may no longer rewind, so any bugs found here have to be fixed or reverted by
applying reverse patch to whatever one introduced them.
As I said, I'd in any case avoid having per-developer branches and keep
strictly per-feature (per-task). In git, commits don't belong to branches.
Branch is just a pointer to it's head commit. Once further commits are made
on top of it, it does not need any name to be accessible anymore. Therefore
once your features make it into 'master' (or even just 'next'), you can
remove their names -- and reuse them. This way you don't have to think twice
about the feature names, because they are temporary (besides you can also
rename them at any time).
--
Jan 'Bulb' Hudec <bulb@ucw.cz>
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: Git's database structure
From: Andy Parkins @ 2007-09-05 19:52 UTC (permalink / raw)
To: git; +Cc: Jon Smirl, Andreas Ericsson, Theodore Tso, Junio C Hamano
In-Reply-To: <9e4733910709050641j34d58683ra72caa52c56cdf0f@mail.gmail.com>
On Wednesday 2007, September 05, Jon Smirl wrote:
> The path name field needs to be moved back into the blobs to support
> alternative indexes. For example I want an index on the Signed-off-by
> field. I use this index to give me the SHAs for the blobs
> Signed-off-by a particular person. In the current design I have no way
> of recovering the path name for these blobs other than a brute force
> search following every path looking for the right SHA.
Erm, if that's your only way then you designed your index incorrectly.
1. Signed-Off-By lines appear in commits, so your index should be an index
of SOB name against commit hash
2. Lookup the commit for that commit hash. As usual this is blindlingly
git-fastic.
3. That commit blob contains a tree hash. Look it up. As usual this is
blindingly git-fastic
4. Start gathering blobs for that tree. Fast, fast, fast.
5. Any subtree objects you come across, goto 4.
This is not a brute force lookup and it's stuff that git is really good at
anyway.
I'm really not sure I see what problem you're trying to solve. Whatever
index you want, you could keep and maintain if you wanted to without
impacting git's core storage at all.
Andy
--
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Junio C Hamano @ 2007-09-05 20:01 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Nix, Steven Grimm, Linus Torvalds, Git Mailing List
In-Reply-To: <alpine.LFD.0.9999.0709051438460.21186@xanadu.home>
Nicolas Pitre <nico@cam.org> writes:
> Not only that. Currently the "Counting objects" phase when running
> git-gc on the Linux repo takes a significant amount of time, even if
> there is little to repack.
>
> If any kind of automatic repack is implemented, it should be an
> incremental repacking only, not the full thing, i.e. git-repack without
> -a, or git-pack-objects with --unpacked. The idea is to be the least
> intrusive as possible. Also, object walking should be limited to
> objects linked to a commit object which is itself unpacked in order to
> cut on the time required to fully enumerate all objects.
>
> This way a semi-packed state will always be preserved and should be good
> enough. The full repacking should probably be left to manual execution
> of git-gc.
Ok, how about doing something like this?
-- >8 -- snipsnap -- >8 -- clipcrap -- >8 --
Implement git gc --auto
This implements a new option "git gc --auto". When gc.auto is
set to a positive value, and the object database has accumulated
roughly that many number of loose objects, this runs a
lightweight version of "git gc". The primary difference from
the full "git gc" is that it does not pass "-a" option to "git
repack", which means we do not try to repack _everything_, but
only repack incrementally. We still do "git prune-packed". The
default threshold is arbitrarily set by yours truly to:
- not trigger it for fully unpacked git v0.99 history;
- do trigger it for fully unpacked git v1.0.0 history;
- not trigger it for incremental update to git v1.0.0 starting
from fully packed git v0.99 history.
This patch does not add invocation of the "auto repacking". It
is left to key Porcelain commands that could produce tons of
loose objects to add a call to "git gc --auto" after they are
done their work. Obvious candidates are:
git add
git fetch
git merge
git rebase
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
builtin-gc.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 63 insertions(+), 1 deletions(-)
diff --git a/builtin-gc.c b/builtin-gc.c
index 9397482..093b3dd 100644
--- a/builtin-gc.c
+++ b/builtin-gc.c
@@ -20,6 +20,7 @@ static const char builtin_gc_usage[] = "git-gc [--prune] [--aggressive]";
static int pack_refs = 1;
static int aggressive_window = -1;
+static int gc_auto_threshold = 6700;
#define MAX_ADD 10
static const char *argv_pack_refs[] = {"pack-refs", "--all", "--prune", NULL};
@@ -28,6 +29,8 @@ static const char *argv_repack[MAX_ADD] = {"repack", "-a", "-d", "-l", NULL};
static const char *argv_prune[] = {"prune", NULL};
static const char *argv_rerere[] = {"rerere", "gc", NULL};
+static const char *argv_repack_auto[] = {"repack", "-d", "-l", NULL};
+
static int gc_config(const char *var, const char *value)
{
if (!strcmp(var, "gc.packrefs")) {
@@ -41,6 +44,10 @@ static int gc_config(const char *var, const char *value)
aggressive_window = git_config_int(var, value);
return 0;
}
+ if (!strcmp(var, "gc.auto")) {
+ gc_auto_threshold = git_config_int(var, value);
+ return 0;
+ }
return git_default_config(var, value);
}
@@ -57,10 +64,49 @@ static void append_option(const char **cmd, const char *opt, int max_length)
cmd[i] = NULL;
}
+static int need_to_gc(void)
+{
+ /*
+ * Quickly check if a "gc" is needed, by estimating how
+ * many loose objects there are. Because SHA-1 is evenly
+ * distributed, we can check only one and get a reasonable
+ * estimate.
+ */
+ char path[PATH_MAX];
+ const char *objdir = get_object_directory();
+ DIR *dir;
+ struct dirent *ent;
+ int auto_threshold;
+ int num_loose = 0;
+ int needed = 0;
+
+ if (sizeof(path) <= snprintf(path, sizeof(path), "%s/17", objdir)) {
+ warning("insanely long object directory %.*s", 50, objdir);
+ return 0;
+ }
+ dir = opendir(path);
+ if (!dir)
+ return 0;
+
+ auto_threshold = (gc_auto_threshold + 255) / 256;
+ while ((ent = readdir(dir)) != NULL) {
+ if (strspn(ent->d_name, "0123456789abcdef") != 38 ||
+ ent->d_name[38] != '\0')
+ continue;
+ if (++num_loose > auto_threshold) {
+ needed = 1;
+ break;
+ }
+ }
+ closedir(dir);
+ return needed;
+}
+
int cmd_gc(int argc, const char **argv, const char *prefix)
{
int i;
int prune = 0;
+ int auto_gc = 0;
char buf[80];
git_config(gc_config);
@@ -82,12 +128,28 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
}
continue;
}
- /* perhaps other parameters later... */
+ if (!strcmp(arg, "--auto")) {
+ if (gc_auto_threshold <= 0)
+ return 0;
+ auto_gc = 1;
+ continue;
+ }
break;
}
if (i != argc)
usage(builtin_gc_usage);
+ if (auto_gc) {
+ /*
+ * Auto-gc should be least intrusive as possible.
+ */
+ prune = 0;
+ for (i = 0; i < ARRAY_SIZE(argv_repack_auto); i++)
+ argv_repack[i] = argv_repack_auto[i];
+ if (!need_to_gc())
+ return 0;
+ }
+
if (pack_refs && run_command_v_opt(argv_pack_refs, RUN_GIT_CMD))
return error(FAILED_RUN, argv_pack_refs[0]);
^ permalink raw reply related
* Re: [PATCH] Clean up .dotest if an error occurs before rebase work starts
From: Junio C Hamano @ 2007-09-05 20:07 UTC (permalink / raw)
To: Robin Rosenberg; +Cc: junkio, git
In-Reply-To: <1189020436633-git-send-email-robin.rosenberg@dewire.com>
It looks to me that we should instead delay the creation of
that .dotest directory until we determine that we _are_ going to
be able to run git-rebase, no?
^ permalink raw reply
* Re: People unaware of the importance of "git gc"?
From: Nicolas Pitre @ 2007-09-05 20:35 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Nix, Steven Grimm, Linus Torvalds, Git Mailing List
In-Reply-To: <7vr6lcj2zi.fsf@gitster.siamese.dyndns.org>
On Wed, 5 Sep 2007, Junio C Hamano wrote:
> Implement git gc --auto
>
> This implements a new option "git gc --auto". When gc.auto is
> set to a positive value, and the object database has accumulated
> roughly that many number of loose objects, this runs a
> lightweight version of "git gc". The primary difference from
> the full "git gc" is that it does not pass "-a" option to "git
> repack", which means we do not try to repack _everything_, but
> only repack incrementally. We still do "git prune-packed".
A big part of the repack cost is the counting of objects. I don't know
if --unpacked to git-pack-objects skips walking trees of a packed commit
object. If no then it probably should to gain a significant speed up,
or maybe a separate option should be created to actually imply this
loosened semantic.
> This patch does not add invocation of the "auto repacking". It
> is left to key Porcelain commands that could produce tons of
> loose objects to add a call to "git gc --auto" after they are
> done their work. Obvious candidates are:
>
> git add
Nope! 'git add' creates loose objects which are not yet reachable from
anywhere. They won't get repacked until a commit is made.
> git fetch
I think that would be a much better idea to simply decrease the
fetch.unpackLimit default value.
> git merge
> git rebase
and git commit. Which resumes it to commit creating operation.
Nicolas
^ permalink raw reply
* [PATCH] Invoke "git gc --auto" from "git add" and "git fetch"
From: Junio C Hamano @ 2007-09-05 20:37 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Nix, Steven Grimm, Linus Torvalds, Git Mailing List
In-Reply-To: <7vr6lcj2zi.fsf@gitster.siamese.dyndns.org>
This makes the two commands to call "git gc --auto" when they
are done.
I earlier said that obvious candidates also include merge and
rebase, but these are lot less frequent operations compared to
add, and more importantly, in a normal workflow they would
almost always happen after "git fetch" is done.
In other words, if you are downstream developer, the automatic
invocation in "git fetch" will take care of things for you, and
otherwise if you do not have an upstream, you would be doing
your own development, so "git add" to add your changes will take
care of the auto invocation for you.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
* This is obviously a follow-up to the previous one that allows
you to say "git gc --auto". I somewhat feel dirty about
calling cmd_gc() bypassing fork & exec from "git add",
though...
builtin-add.c | 2 ++
git-fetch.sh | 1 +
2 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/builtin-add.c b/builtin-add.c
index 105a9f0..8431c16 100644
--- a/builtin-add.c
+++ b/builtin-add.c
@@ -263,9 +263,11 @@ int cmd_add(int argc, const char **argv, const char *prefix)
finish:
if (active_cache_changed) {
+ const char *args[] = { "gc", "--auto", NULL };
if (write_cache(newfd, active_cache, active_nr) ||
close(newfd) || commit_locked_index(&lock_file))
die("Unable to write new index file");
+ cmd_gc(2, args, NULL);
}
return 0;
diff --git a/git-fetch.sh b/git-fetch.sh
index c3a2001..86050eb 100755
--- a/git-fetch.sh
+++ b/git-fetch.sh
@@ -375,3 +375,4 @@ case "$orig_head" in
fi
;;
esac
+git gc --auto
--
1.5.3.1.840.g0fedbc
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox