* Re: [PATCH 2/3] add new Git::Repo API
From: Lea Wiemann @ 2008-07-14 1:04 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, John Hawley, Jakub Narebski, Petr Baudis
In-Reply-To: <7vmyklpiqk.fsf@gitster.siamese.dyndns.org>
Junio C Hamano wrote:
> This probably is something the person who mentors you as a GSoC student
> should tell you, and shouldn't come from me,
No, thanks for telling me. I'm actually keeping John quite busy with
the server side on kernel.org, and wouldn't currently expect him to look
at the patches I'm sending.
> I would have preferred to see the first paragraph of yours stated more
> diplomaticly; it sounds rather unilateral ultimatum to me.
Apologies; I was trying to hold off developers who'd see duplicate
functionality and run off at me screaming "unacceptable! impure!!one".
I should've phrased it more carefully.
> This patch was not signed-off, but I assume it was simply forgotten
Yup. In the unlikely case it gets applied as-is, feel free to add it.
(I'll reply to Jakub's email shortly, unless I fall asleep in the
meantime. ^^)
^ permalink raw reply
* Re: [PATCH 2/3] add new Git::Repo API
From: Petr Baudis @ 2008-07-14 1:40 UTC (permalink / raw)
To: Lea Wiemann; +Cc: git, John Hawley, Jakub Narebski
In-Reply-To: <1215738665-5153-1-git-send-email-LeWiemann@gmail.com>
Thanks to Jakub for a nice writeup too. I will try to cover mainly
design issues and do not try to dive too much into technical arguments
about current Git.pm, since I agree with the general idea of designing
a new API; Jakub covered most of that pretty well, I think.
Maybe I sound too mentoring at places; I tried to tone that down as much
as possible, and of course what Lea should be working on is her and hers
official mentor call. On the other hand, if something general gets into
the Git tree, I'd like to make sure it's something we can live happily
with for long time, not just a hack tailored for gitweb caching.
On Fri, Jul 11, 2008 at 03:11:05AM +0200, Lea Wiemann wrote:
> This also adds the Git::Commit and Git::Tag classes, which are used by
> Git::Repo, the Git::Object base class, and the Git::RepoRoot helper
> factory class.
I really miss some more detailed writeup on the envisioned design here.
And if we are redoing the API in a better way, we better should have
some vision.
Most importantly, how is Git::Repo interacting with working copies, and
how is it going to interact with them as the new OO model shapes up?
You mention briefly Git::WC later, but it is not really clear how the
interaction should work.
First, I don't think it's good idea at all to put the pipe-related stuff
to Git::Repo - this is botched up API just like the current one. This
all is independent from any repository instances, in fact it's perfectly
valid to call standalone remote commands (like ls-remote or, actually,
clone).
Here is an idea: Introduce Git::Command object that will have very
general interface and look like
my $c = Git::Command->new(['git', '--git-dir=.', 'cat-file', \
'-p', 'bla'], {pipe_out=>1})
...
$c->close();
and a Git::CommandFactory with a nicer interface that would look like
my $cf = Git::CommandFactory->new('git', '--git-dir=.');
my $c = $cf->output_pipe('cat-file', '-p', 'bla');
$c->close();
Then, Git::Repo would have a single Git::CommandFactory instance
pre-initialized with the required calling convention, and returned by
e.g. cmd() method. Then, from the user POV, you would just:
my $repo = Git::Repo->new;
$repo->cmd->output_pipe('cat-file', '-p', 'bla');
Or am I overdoing it?
Git::Repo considers only bare repositories. Now, since "working copy" by
itself has nothing to do with Git and is just an ordinary directory
tree, I think Git::WC does not make that much sense, but something like
Git::Repo::Nonbare certainly would. This would be a Git::Repo subclass
with:
* Different constructor
* Different Git::CommandFactory instance
* Git::Index object aside the existing ones (like Git::Config,
Git::RefTree, ...)
* Some kind of wc_root() method to help directory navigation
And that pretty much covers it?
Another thing is clearly describing how error handling is going to work.
I have not much against ditching Error.pm, but just saying "die + eval"
does not cut it - how about possible sideband data? E.g. the failure
mode of Git.pm's command() method includes passing the error'd command
output in the exception object. How are we going to handle it? Now, it
might be actually okay to say that we _aren't_ going to handle this if
it is deemed unuseful, but that needs to be determined too. I don't know
off the top of my head.
> ---
> Here's some elaboration on why I didn't use or extend Git.pm.
>
> Please note before starting a reply to this: This is not an argument;
> I'm just explaining why I implemented it the way I did. So please
> don't try to argue with me about what I should or should have done.
> I'm not going to refactor Git::Repo to use Git.pm or vice versa; it's
> really a much more non-trivial task than you might think at first
> glance.
Frankly, I'm not really happy about this attitude. "In my dark corner,
I have developed this. Now behold - take it or leave it be; I'm not
going to argue."
That said, I do agree that better-structured object API would be
beneficial, and I think what you posted here is a good starting point.
However, I'm not going to settle with seeing a completely parallel and
independent code pop up alongside the existing Git.pm.
Instead, I believe the best course is to gradually translate all the
Git.pm functionality to the new OO model, leaving Git.pm as a
compatibility wrapper. Now, if you believe this is a non-trivial task,
please tell us why.
It should be actually very easy to start with moving all the pipe
functionality to Git::Command. This should let us get rid of a lot of
technical code, which can be reused in the cleanly rebuilt API. Aside
of the object management + error handling, the rest of Git.pm is then
actually rather boring and unimportant, especially given Jakub's
hopefully upcoming Git::Config.
I will be happy to send in patches factoring out the pipe handling to
Git::Command as soon as we agree about the missing design details and
the way of error handling. Then, making use of that in Git::Repo should
get us a long way ahead.
> Now I'm all in favor of re-using existing code, but refactoring
> Git.pm would have taken *much* longer than simply writing a new
> module. I'm working on caching for gitweb, not on implementing the
> next great Perl API for Git. (And Git::Repo isn't great, FTR.)
Wait, I can't make sense out of this paragraph. If Git.pm sucks, we can
work on new API. But we better _make_ it great. Or someone else comes by
next year and says "oh, but it's buggy and needs refactoring, let's
throw it away and redesign it!"
I agree that your main objective is caching for gitweb, but that's not
what everything revolves around for the rest of us. If you chose the way
of caching within the Git API and introducing the API to gitweb, I think
you should spend the effort to deal with the API properly now.
> diff --git a/perl/Git/Commit.pm b/perl/Git/Commit.pm
> new file mode 100644
> index 0000000..a9bc304
> --- /dev/null
> +++ b/perl/Git/Commit.pm
> @@ -0,0 +1,163 @@
> +=head1 NAME
> +
> +Git::Commit - Object-oriented interface to Git commit objects.
> +
> +=cut
> +
> +use strict;
> +use warnings;
> +
> +
> +package Git::Commit;
> +
> +use base qw(Git::Object);
> +
> +use constant _MESSAGE => 'M';
> +use constant _ENCODING => 'E';
> +use constant _TREE => 'T';
> +use constant _PARENTS => 'P';
> +use constant _AUTHOR => 'A';
> +use constant _COMMITTER => 'C';
What is the idea behind this? It may make sense to create numerical
constants like this and use arrays instead of hashes for "structs" in
order to gain extra performance. But if you are going to use hashes
anyway, why not actually key by sensible name directly?
> +=head1 METHODS
> +
> +=over
> +
> +=item $commit = Git::Commit->new($repo, $sha1)
Are we sure we don't want hash-based arguments instead? This is badly
extensible and inconsistent with the rest of the API.
> +Return a new Git::Commit instance for a commit object with $sha1 in
> +repository $repo.
> +
> +Calls to this method are free, since it does not check whether $sha1
> +exists and has the right type. However, accessing any of the commit
> +object's properties will fail if $sha1 is not a valid commit object.
This is nice idea, but I'd also provide a well-defined way for the user
to verify the object's validity at a good moment; basically, make load()
a public method. The user can deal with errors then and rely on
error-free behavior later.
> +Note that $sha1 must be the SHA1 of a commit object; tag objects are
> +not dereferenced.
Why not?
> +=item $obj->repo
> +
> +Return the Git::Repo instance this object was instantiated with.
> +
> +=item $obj->sha1
> +
> +Return the SHA1 of this commit object.
> +
> +=item $commit->tree
> +
> +Return the tree this commit object refers to.
> +
> +=item $commit->parents
> +
> +Return a list of zero or more parent commit objects. Note that commit
> +objects stringify to their respective SHA1s, so you can alternatively
> +treat this as a list of SHA1 strings.
> +
> +=item $commit->authors
author
> diff --git a/perl/Git/Object.pm b/perl/Git/Object.pm
> new file mode 100644
> index 0000000..a5126f9
> --- /dev/null
> +++ b/perl/Git/Object.pm
> @@ -0,0 +1,81 @@
> +=head1 NAME
> +
> +Git::Object - Object-oriented interface to Git objects (base class).
> +
> +=head1 DESCRIPTION
> +
> +Git::Object is a base class that provides access to commit, tag and
> +(unimplemented) tree objects. See L<Git::Commit> and L<Git::Tag>.
> +
> +Objects are loaded lazily, and hence instantiation is free. Objects
> +stringify to their SHA1s.
Maybe use the term 'Git database objects'? This way, it seems as if we
are talking about all Git/*.pm objects.
> +=cut
> +
> +
> +use strict;
> +use warnings;
> +
> +
> +package Git::Object;
> +
> +use base qw(Exporter);
> +
> +our @EXPORT = qw();
> +our @EXPORT_OK = qw();
> +
> +use overload
> + '""' => \&stringify;
(For the beholders: the overload pragma comes from 5.004.)
> +# Hash indices:
> +use constant _REPO => 'R';
> +use constant _SHA1 => 'H';
> +
> +=head1 METHODS
> +
> +=over
> +
> +=item Git::Object->new($repo, $sha1)
> +
> +Return a new Git::Object instance for the object with $sha1 in the
> +repository $repo (a Git::Repo instance).
> +
> +Note that this method does not check whether the object exists in the
> +repository. Trying to accessing its properties through a subclass
> +will fail if the object doesn't exist, however.
> +
> +=cut
> +
> +sub new {
> + my($class, $repo, $sha1) = @_;
> + die "$repo is not a Git::Repo instance" unless $repo->isa('Git::Repo');
> + my $self = {_REPO() => $repo, _SHA1() => $sha1};
> + return bless $self, $class;
> +}
> +
> +=item $obj->repo
> +
> +Return the Git::Repo instance this object was instantiated with.
> +
> +=cut
> +
> +sub repo {
> + shift->{_REPO()}
> +}
> +
> +=item $obj->sha1
> +
> +Return the SHA1 of this object.
> +
> +=cut
> +
> +sub sha1 {
> + shift->{_SHA1()}
> +}
> +
> +sub stringify {
> + shift->{_SHA1()}
> +}
> +
> +1;
Why not overload "" directly to sha1()?
> diff --git a/perl/Git/Repo.pm b/perl/Git/Repo.pm
> new file mode 100644
> index 0000000..5a4c7e0
> --- /dev/null
> +++ b/perl/Git/Repo.pm
> @@ -0,0 +1,397 @@
> +=head1 NAME
> +
> +Git::Repo - Read-only access to the Git repositories.
> +
> +WARNING: This module is in active development -- do not use it in any
> +production code as the API may change at any time.
> +
> +=head1 SYNOPSIS
> +
> + use Git::Repo;
> +
> + my $repo = Git::Repo->new(
> + directory => '/path/to/repository.git',
> + git_binary => '/usr/bin/git');
> + my ($sha1, $type, $size) = $repo->get_sha1('HEAD');
> + print "Last log message:\n\n" . $repo->get_commit($sha1)->message;
> +
> +=cut
> +
> +
> +use strict;
> +use warnings;
> +use 5.006002;
> +
> +
> +package Git::Repo;
> +
> +use Git::Tag;
> +use Git::Commit;
> +
> +use IPC::Open2 qw(open2);
> +use IO::Handle;
> +
> +use base qw(Exporter);
> +
> +our @EXPORT = qw();
> +our @EXPORT_OK = qw(assert_sha1 assert_opts);
> +
> +# Auxiliary subroutines
> +
> +sub assert_opts {
> + die "must have an even number of arguments for named options"
> + unless $#_ % 2;
> +}
> +
> +sub assert_sha1 {
> + my $sha1 = shift;
> + die "'$sha1' is not a SHA1 (need to use get_sha1?)"
> + unless $sha1 && $sha1 =~ /^[a-f0-9]{40}$/;
> +}
Pretend names with underscores, since they are internal?
> +=head1 METHODS
> +
> +=head2 Instantiation and basic methods
> +
> +=over
> +
> +=item $repo = Git::Repo->new(%opts)
> +
> +Return a new Git::Repo object. The following options are supported:
> +
> +=over
> +
> +=item 'directory'
> +
> +The directory of the repository (mandatory).
> +
> +Note that this option is working-copy agnostic; you need to
> +instantiate it with the working copy's .git directory as the
> +'directory' option.
I don't think making this mandatory is reasonable, since all the git
commands can automatically figure this out by themselves too; so can
Git::Repo easily by calling git rev-parse --git-dir.
> +=item 'git_binary'
> +
> +The name or full path of the git binary (default: 'git').
> +
> +=back
> +
> +Calling this method is free, since it does not check whether the
> +repository exists. Trying to access the repository through one of the
> +instance methods will fail if it doesn't exist though.
> +
> +Examples:
> +
> + $repo = Git::Repo->new(directory => '/path/to/repository.git');
> + $repo = Git::Repo->new(directory => '/path/to/working_copy/.git');
> +
> +=cut
> +
> +sub new {
> + my $class = shift;
> + assert_opts(@_);
> + my $self = {@_};
> + bless $self, $class;
> + die 'no directory given' unless $self->{directory};
> + return $self;
> +}
> +
> +=item $repo->repo_dir
> +
> +Return the directory of the repository (.../.git in case of a working
> +copy).
> +
> +=cut
> +
> +sub repo_dir {
> + shift->{directory}
> +}
> +
> +=item $repo->git_binary
> +
> +Return the name of or path to the git binary (used with exec).
> +
> +=cut
> +
> +sub git_binary {
> + shift->{git_binary}
> +}
--- CUT ---
> +=item $repo->version
..snip..
> +# Return the first items of the git command line, for instance
> +# qw(/usr/bin/git --git-dir=/path/to/repo.git).
> +sub _get_git_cmd {
> + my $self = shift;
> + return ($self->git_binary || 'git', '--git-dir=' . $self->repo_dir);
> +}
--- CUT ---
This definitely does not belong to a Git::Repo object.
BTW, I like the idea of reuse => 1 very much.
> +=back
> +
> +=head2 Inspecting the Repository
> +
> +=over
> +
> +=item $repo->cat_file($sha1)
> +
> +Return the ($type, $content) of the object identified by $sha1, or die
> +if no such object exists in the repository.
I don't think this is good combination of semantic and name. Since we
don't do the same thing as plain git cat-file, we might as well call it
cat_object() or even better get_object().
> +Note that you may want to use the higher-level methods get_commit and
> +get_tag instead.
..snip..
> +=item $repo->get_path($tree_sha1, $file_sha1)
Now we are quickly getting messy again. This should definitely live in
Git::Tree.
> +=item $repo->get_refs
Again, the refs should be properly integrated into the object structure.
--
Petr "Pasky" Baudis
GNU, n. An animal of South Africa, which in its domesticated state
resembles a horse, a buffalo and a stag. In its wild condition it is
something like a thunderbolt, an earthquake and a cyclone. -- A. Pierce
^ permalink raw reply
* [PATCH 0/4] Honor core.deltaBaseCacheLimit during index-pack
From: Shawn O. Pearce @ 2008-07-14 2:07 UTC (permalink / raw)
To: Junio C Hamano, Junio C Hamano
Cc: git, Stephan Hennig, Nicolas Pitre, Andreas Ericsson
In-Reply-To: <20080713011512.GB31050@spearce.org>
This should resolve the obscene memory usage of index-pack when
resolving deltas for very large files.
Shawn O. Pearce (4):
index-pack: Refactor base arguments of resolve_delta into a struct
index-pack: Chain the struct base_data on the stack for traversal
index-pack: Track the object_entry that creates each base_data
index-pack: Honor core.deltaBaseCacheLimit when resolving deltas
index-pack.c | 137 +++++++++++++++++++++++++++++++++++++++++++++------------
1 files changed, 108 insertions(+), 29 deletions(-)
^ permalink raw reply
* [PATCH 1/4] index-pack: Refactor base arguments of resolve_delta into a struct
From: Shawn O. Pearce @ 2008-07-14 2:07 UTC (permalink / raw)
To: Junio C Hamano, Junio C Hamano
Cc: git, Stephan Hennig, Nicolas Pitre, Andreas Ericsson
In-Reply-To: <20080713011512.GB31050@spearce.org>
We need to discard base objects which are not recently used if our
memory gets low, such as when we are unpacking a long delta chain
of a very large object.
To support tracking the available base objects we combine the
pointer and size into a struct. Future changes would allow the
data pointer to be free'd and marked NULL if memory gets low.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
index-pack.c | 60 +++++++++++++++++++++++++++++++--------------------------
1 files changed, 33 insertions(+), 27 deletions(-)
diff --git a/index-pack.c b/index-pack.c
index 25db5db..db03478 100644
--- a/index-pack.c
+++ b/index-pack.c
@@ -26,6 +26,11 @@ union delta_base {
off_t offset;
};
+struct base_data {
+ void *data;
+ unsigned long size;
+};
+
/*
* Even if sizeof(union delta_base) == 24 on 64-bit archs, we really want
* to memcmp() only the first 20 bytes.
@@ -426,25 +431,25 @@ static void sha1_object(const void *data, unsigned long size,
}
}
-static void resolve_delta(struct object_entry *delta_obj, void *base_data,
- unsigned long base_size, enum object_type type)
+static void resolve_delta(struct object_entry *delta_obj,
+ struct base_data *base_obj, enum object_type type)
{
void *delta_data;
unsigned long delta_size;
- void *result;
- unsigned long result_size;
union delta_base delta_base;
int j, first, last;
+ struct base_data result;
delta_obj->real_type = type;
delta_data = get_data_from_pack(delta_obj);
delta_size = delta_obj->size;
- result = patch_delta(base_data, base_size, delta_data, delta_size,
- &result_size);
+ result.data = patch_delta(base_obj->data, base_obj->size,
+ delta_data, delta_size,
+ &result.size);
free(delta_data);
- if (!result)
+ if (!result.data)
bad_object(delta_obj->idx.offset, "failed to apply delta");
- sha1_object(result, result_size, type, delta_obj->idx.sha1);
+ sha1_object(result.data, result.size, type, delta_obj->idx.sha1);
nr_resolved_deltas++;
hashcpy(delta_base.sha1, delta_obj->idx.sha1);
@@ -452,7 +457,7 @@ static void resolve_delta(struct object_entry *delta_obj, void *base_data,
for (j = first; j <= last; j++) {
struct object_entry *child = objects + deltas[j].obj_no;
if (child->real_type == OBJ_REF_DELTA)
- resolve_delta(child, result, result_size, type);
+ resolve_delta(child, &result, type);
}
}
@@ -462,11 +467,11 @@ static void resolve_delta(struct object_entry *delta_obj, void *base_data,
for (j = first; j <= last; j++) {
struct object_entry *child = objects + deltas[j].obj_no;
if (child->real_type == OBJ_OFS_DELTA)
- resolve_delta(child, result, result_size, type);
+ resolve_delta(child, &result, type);
}
}
- free(result);
+ free(result.data);
}
static int compare_delta_entry(const void *a, const void *b)
@@ -481,7 +486,6 @@ static void parse_pack_objects(unsigned char *sha1)
{
int i;
struct delta_entry *delta = deltas;
- void *data;
struct stat st;
/*
@@ -496,7 +500,7 @@ static void parse_pack_objects(unsigned char *sha1)
nr_objects);
for (i = 0; i < nr_objects; i++) {
struct object_entry *obj = &objects[i];
- data = unpack_raw_entry(obj, &delta->base);
+ void *data = unpack_raw_entry(obj, &delta->base);
obj->real_type = obj->type;
if (obj->type == OBJ_REF_DELTA || obj->type == OBJ_OFS_DELTA) {
nr_deltas++;
@@ -545,6 +549,7 @@ static void parse_pack_objects(unsigned char *sha1)
struct object_entry *obj = &objects[i];
union delta_base base;
int j, ref, ref_first, ref_last, ofs, ofs_first, ofs_last;
+ struct base_data base_obj;
if (obj->type == OBJ_REF_DELTA || obj->type == OBJ_OFS_DELTA)
continue;
@@ -555,22 +560,22 @@ static void parse_pack_objects(unsigned char *sha1)
ofs = !find_delta_children(&base, &ofs_first, &ofs_last);
if (!ref && !ofs)
continue;
- data = get_data_from_pack(obj);
+ base_obj.data = get_data_from_pack(obj);
+ base_obj.size = obj->size;
+
if (ref)
for (j = ref_first; j <= ref_last; j++) {
struct object_entry *child = objects + deltas[j].obj_no;
if (child->real_type == OBJ_REF_DELTA)
- resolve_delta(child, data,
- obj->size, obj->type);
+ resolve_delta(child, &base_obj, obj->type);
}
if (ofs)
for (j = ofs_first; j <= ofs_last; j++) {
struct object_entry *child = objects + deltas[j].obj_no;
if (child->real_type == OBJ_OFS_DELTA)
- resolve_delta(child, data,
- obj->size, obj->type);
+ resolve_delta(child, &base_obj, obj->type);
}
- free(data);
+ free(base_obj.data);
display_progress(progress, nr_resolved_deltas);
}
}
@@ -656,28 +661,29 @@ static void fix_unresolved_deltas(int nr_unresolved)
for (i = 0; i < n; i++) {
struct delta_entry *d = sorted_by_pos[i];
- void *data;
- unsigned long size;
enum object_type type;
int j, first, last;
+ struct base_data base_obj;
if (objects[d->obj_no].real_type != OBJ_REF_DELTA)
continue;
- data = read_sha1_file(d->base.sha1, &type, &size);
- if (!data)
+ base_obj.data = read_sha1_file(d->base.sha1, &type, &base_obj.size);
+ if (!base_obj.data)
continue;
find_delta_children(&d->base, &first, &last);
for (j = first; j <= last; j++) {
struct object_entry *child = objects + deltas[j].obj_no;
if (child->real_type == OBJ_REF_DELTA)
- resolve_delta(child, data, size, type);
+ resolve_delta(child, &base_obj, type);
}
- if (check_sha1_signature(d->base.sha1, data, size, typename(type)))
+ if (check_sha1_signature(d->base.sha1, base_obj.data,
+ base_obj.size, typename(type)))
die("local object %s is corrupt", sha1_to_hex(d->base.sha1));
- append_obj_to_pack(d->base.sha1, data, size, type);
- free(data);
+ append_obj_to_pack(d->base.sha1, base_obj.data,
+ base_obj.size, type);
+ free(base_obj.data);
display_progress(progress, nr_resolved_deltas);
}
free(sorted_by_pos);
--
1.5.6.2.393.g45096
^ permalink raw reply related
* [PATCH 2/4] index-pack: Chain the struct base_data on the stack for traversal
From: Shawn O. Pearce @ 2008-07-14 2:07 UTC (permalink / raw)
To: Junio C Hamano, Junio C Hamano
Cc: git, Stephan Hennig, Nicolas Pitre, Andreas Ericsson
In-Reply-To: <20080713011512.GB31050@spearce.org>
We need to release earlier inflated base objects when memory gets
low, which means we need to be able to walk up or down the stack
to locate the objects we want to release, and free their data.
The new link/unlink routines allow inserting and removing the struct
base_data during recursion inside resolve_delta, and the global
base_cache gives us the head of the chain (bottom of the stack)
so we can traverse it.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
index-pack.c | 34 +++++++++++++++++++++++++++++++---
1 files changed, 31 insertions(+), 3 deletions(-)
diff --git a/index-pack.c b/index-pack.c
index db03478..6c59fd3 100644
--- a/index-pack.c
+++ b/index-pack.c
@@ -27,6 +27,8 @@ union delta_base {
};
struct base_data {
+ struct base_data *base;
+ struct base_data *child;
void *data;
unsigned long size;
};
@@ -48,6 +50,7 @@ struct delta_entry
static struct object_entry *objects;
static struct delta_entry *deltas;
+static struct base_data *base_cache;
static int nr_objects;
static int nr_deltas;
static int nr_resolved_deltas;
@@ -216,6 +219,27 @@ static void bad_object(unsigned long offset, const char *format, ...)
die("pack has bad object at offset %lu: %s", offset, buf);
}
+static void link_base_data(struct base_data *base, struct base_data *c)
+{
+ if (base)
+ base->child = c;
+ else
+ base_cache = c;
+
+ c->base = base;
+ c->child = NULL;
+}
+
+static void unlink_base_data(struct base_data *c)
+{
+ struct base_data *base = c->base;
+ if (base)
+ base->child = NULL;
+ else
+ base_cache = NULL;
+ free(c->data);
+}
+
static void *unpack_entry_data(unsigned long offset, unsigned long size)
{
z_stream stream;
@@ -452,6 +476,8 @@ static void resolve_delta(struct object_entry *delta_obj,
sha1_object(result.data, result.size, type, delta_obj->idx.sha1);
nr_resolved_deltas++;
+ link_base_data(base_obj, &result);
+
hashcpy(delta_base.sha1, delta_obj->idx.sha1);
if (!find_delta_children(&delta_base, &first, &last)) {
for (j = first; j <= last; j++) {
@@ -471,7 +497,7 @@ static void resolve_delta(struct object_entry *delta_obj,
}
}
- free(result.data);
+ unlink_base_data(&result);
}
static int compare_delta_entry(const void *a, const void *b)
@@ -562,6 +588,7 @@ static void parse_pack_objects(unsigned char *sha1)
continue;
base_obj.data = get_data_from_pack(obj);
base_obj.size = obj->size;
+ link_base_data(NULL, &base_obj);
if (ref)
for (j = ref_first; j <= ref_last; j++) {
@@ -575,7 +602,7 @@ static void parse_pack_objects(unsigned char *sha1)
if (child->real_type == OBJ_OFS_DELTA)
resolve_delta(child, &base_obj, obj->type);
}
- free(base_obj.data);
+ unlink_base_data(&base_obj);
display_progress(progress, nr_resolved_deltas);
}
}
@@ -670,6 +697,7 @@ static void fix_unresolved_deltas(int nr_unresolved)
base_obj.data = read_sha1_file(d->base.sha1, &type, &base_obj.size);
if (!base_obj.data)
continue;
+ link_base_data(NULL, &base_obj);
find_delta_children(&d->base, &first, &last);
for (j = first; j <= last; j++) {
@@ -683,7 +711,7 @@ static void fix_unresolved_deltas(int nr_unresolved)
die("local object %s is corrupt", sha1_to_hex(d->base.sha1));
append_obj_to_pack(d->base.sha1, base_obj.data,
base_obj.size, type);
- free(base_obj.data);
+ unlink_base_data(&base_obj);
display_progress(progress, nr_resolved_deltas);
}
free(sorted_by_pos);
--
1.5.6.2.393.g45096
^ permalink raw reply related
* [PATCH 3/4] index-pack: Track the object_entry that creates each base_data
From: Shawn O. Pearce @ 2008-07-14 2:07 UTC (permalink / raw)
To: Junio C Hamano, Junio C Hamano
Cc: git, Stephan Hennig, Nicolas Pitre, Andreas Ericsson
In-Reply-To: <20080713011512.GB31050@spearce.org>
If we free the data stored within a base_data we need the struct
object_entry to get the data back again for use with another
dependent delta. Storing the object_entry* makes it simple to call
get_data_from_pack() to recover the compressed information.
This however means we must add the missing baes object to the end
of our packfile prior to calling resolve_delta() on each of the
dependent deltas. Adding the base first ensures we can read the
base back from the pack we indexing, as if it had been included by
the remote side.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
index-pack.c | 18 ++++++++++++------
1 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/index-pack.c b/index-pack.c
index 6c59fd3..7239e89 100644
--- a/index-pack.c
+++ b/index-pack.c
@@ -29,6 +29,7 @@ union delta_base {
struct base_data {
struct base_data *base;
struct base_data *child;
+ struct object_entry *obj;
void *data;
unsigned long size;
};
@@ -476,6 +477,7 @@ static void resolve_delta(struct object_entry *delta_obj,
sha1_object(result.data, result.size, type, delta_obj->idx.sha1);
nr_resolved_deltas++;
+ result.obj = delta_obj;
link_base_data(base_obj, &result);
hashcpy(delta_base.sha1, delta_obj->idx.sha1);
@@ -588,6 +590,7 @@ static void parse_pack_objects(unsigned char *sha1)
continue;
base_obj.data = get_data_from_pack(obj);
base_obj.size = obj->size;
+ base_obj.obj = obj;
link_base_data(NULL, &base_obj);
if (ref)
@@ -633,7 +636,8 @@ static int write_compressed(int fd, void *in, unsigned int size, uint32_t *obj_c
return size;
}
-static void append_obj_to_pack(const unsigned char *sha1, void *buf,
+static struct object_entry *append_obj_to_pack(
+ const unsigned char *sha1, void *buf,
unsigned long size, enum object_type type)
{
struct object_entry *obj = &objects[nr_objects++];
@@ -654,6 +658,7 @@ static void append_obj_to_pack(const unsigned char *sha1, void *buf,
obj[1].idx.offset = obj[0].idx.offset + n;
obj[1].idx.offset += write_compressed(output_fd, buf, size, &obj[0].idx.crc32);
hashcpy(obj->idx.sha1, sha1);
+ return obj;
}
static int delta_pos_compare(const void *_a, const void *_b)
@@ -697,6 +702,12 @@ static void fix_unresolved_deltas(int nr_unresolved)
base_obj.data = read_sha1_file(d->base.sha1, &type, &base_obj.size);
if (!base_obj.data)
continue;
+
+ if (check_sha1_signature(d->base.sha1, base_obj.data,
+ base_obj.size, typename(type)))
+ die("local object %s is corrupt", sha1_to_hex(d->base.sha1));
+ base_obj.obj = append_obj_to_pack(d->base.sha1, base_obj.data,
+ base_obj.size, type);
link_base_data(NULL, &base_obj);
find_delta_children(&d->base, &first, &last);
@@ -706,11 +717,6 @@ static void fix_unresolved_deltas(int nr_unresolved)
resolve_delta(child, &base_obj, type);
}
- if (check_sha1_signature(d->base.sha1, base_obj.data,
- base_obj.size, typename(type)))
- die("local object %s is corrupt", sha1_to_hex(d->base.sha1));
- append_obj_to_pack(d->base.sha1, base_obj.data,
- base_obj.size, type);
unlink_base_data(&base_obj);
display_progress(progress, nr_resolved_deltas);
}
--
1.5.6.2.393.g45096
^ permalink raw reply related
* [PATCH 4/4] index-pack: Honor core.deltaBaseCacheLimit when resolving deltas
From: Shawn O. Pearce @ 2008-07-14 2:07 UTC (permalink / raw)
To: Junio C Hamano, Junio C Hamano
Cc: git, Stephan Hennig, Nicolas Pitre, Andreas Ericsson
In-Reply-To: <20080713011512.GB31050@spearce.org>
If we are trying to resolve deltas for a long delta chain composed
of multi-megabyte objects we can easily run into requiring 500M+
of memory to hold each object in the chain on the call stack while
we recurse into the dependent objects and resolve them.
We now use a simple delta cache that discards objects near the
bottom of the call stack first, as they are the most least recently
used objects in this current delta chain. If we recurse out of a
chain we may find the base object is no longer available, as it was
free'd to keep memory under the deltaBaseCacheLimit. In such cases
we must unpack the base object again, which will require recursing
back to the root of the top of the delta chain as we released that
root first.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
index-pack.c | 43 +++++++++++++++++++++++++++++++++++++++++--
1 files changed, 41 insertions(+), 2 deletions(-)
diff --git a/index-pack.c b/index-pack.c
index 7239e89..84c9fc1 100644
--- a/index-pack.c
+++ b/index-pack.c
@@ -52,6 +52,7 @@ struct delta_entry
static struct object_entry *objects;
static struct delta_entry *deltas;
static struct base_data *base_cache;
+static size_t base_cache_used;
static int nr_objects;
static int nr_deltas;
static int nr_resolved_deltas;
@@ -220,6 +221,20 @@ static void bad_object(unsigned long offset, const char *format, ...)
die("pack has bad object at offset %lu: %s", offset, buf);
}
+static void prune_base_data(void)
+{
+ struct base_data *b = base_cache;
+ for (b = base_cache;
+ base_cache_used > delta_base_cache_limit && b && b->child;
+ b = b->child) {
+ if (b->data) {
+ free(b->data);
+ b->data = NULL;
+ base_cache_used -= b->size;
+ }
+ }
+}
+
static void link_base_data(struct base_data *base, struct base_data *c)
{
if (base)
@@ -229,6 +244,8 @@ static void link_base_data(struct base_data *base, struct base_data *c)
c->base = base;
c->child = NULL;
+ base_cache_used += c->size;
+ prune_base_data();
}
static void unlink_base_data(struct base_data *c)
@@ -238,7 +255,10 @@ static void unlink_base_data(struct base_data *c)
base->child = NULL;
else
base_cache = NULL;
- free(c->data);
+ if (c->data) {
+ free(c->data);
+ base_cache_used -= c->size;
+ }
}
static void *unpack_entry_data(unsigned long offset, unsigned long size)
@@ -456,6 +476,25 @@ static void sha1_object(const void *data, unsigned long size,
}
}
+static void *get_base_data(struct base_data *c)
+{
+ if (!c->data) {
+ struct object_entry *obj = c->obj;
+ void *raw = get_data_from_pack(obj);
+ if (obj->type == OBJ_REF_DELTA || obj->type == OBJ_OFS_DELTA) {
+ c->data = patch_delta(
+ get_base_data(c->base), c->base->size,
+ raw, obj->size,
+ &c->size);
+ free(raw);
+ if (!c->data)
+ bad_object(obj->idx.offset, "failed to apply delta");
+ } else
+ c->data = raw;
+ }
+ return c->data;
+}
+
static void resolve_delta(struct object_entry *delta_obj,
struct base_data *base_obj, enum object_type type)
{
@@ -468,7 +507,7 @@ static void resolve_delta(struct object_entry *delta_obj,
delta_obj->real_type = type;
delta_data = get_data_from_pack(delta_obj);
delta_size = delta_obj->size;
- result.data = patch_delta(base_obj->data, base_obj->size,
+ result.data = patch_delta(get_base_data(base_obj), base_obj->size,
delta_data, delta_size,
&result.size);
free(delta_data);
--
1.5.6.2.393.g45096
^ permalink raw reply related
* Re: [PATCH/Test] Build in merge is broken
From: Sverre Hvammen Johansen @ 2008-07-14 2:15 UTC (permalink / raw)
To: Miklos Vajna; +Cc: git, Junio C Hamano
In-Reply-To: <20080713184300.GF10347@genesis.frugalware.org>
Test cases for build in merge.
---
After applying Miklos's fix there still are some breakages. I have squashed in
another test case. c1 is merged with c1 and c2. Three parents are
recorded in the
merge commit object; c1, c1, and c2.
t/t7600-merge.sh | 22 ++++++++++++++++++++++
1 files changed, 22 insertions(+), 0 deletions(-)
diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
index 16f4608..80cfee6 100755
--- a/t/t7600-merge.sh
+++ b/t/t7600-merge.sh
@@ -465,4 +465,26 @@ test_expect_success 'merge log message' '
test_debug 'gitk --all'
+test_expect_success 'merge c1 with c0, c2, c0, and c1' '
+ git reset --hard c1 &&
+ git config branch.master.mergeoptions "" &&
+ test_tick &&
+ git merge c0 c2 c0 c1 &&
+ verify_merge file result.1-5 &&
+ verify_parents $c1 $c2
+'
+
+test_debug 'gitk --all'
+
+test_expect_success 'merge c1 with c1 and c2' '
+ git reset --hard c1 &&
+ git config branch.master.mergeoptions "" &&
+ test_tick &&
+ git merge c1 c2 &&
+ verify_merge file result.1-5 &&
+ verify_parents $c1 $c2
+'
+
+test_debug 'gitk --all'
+
test_done
--
1.5.5.54.gc6550
--
Sverre Hvammen Johansen
^ permalink raw reply related
* Re: [PATCH 0/4] Honor core.deltaBaseCacheLimit during index-pack
From: Nicolas Pitre @ 2008-07-14 2:27 UTC (permalink / raw)
To: Shawn O. Pearce
Cc: Junio C Hamano, Junio C Hamano, git, Stephan Hennig,
Andreas Ericsson
In-Reply-To: <1216001267-33235-1-git-send-email-spearce@spearce.org>
On Sun, 13 Jul 2008, Shawn O. Pearce wrote:
> This should resolve the obscene memory usage of index-pack when
> resolving deltas for very large files.
>
> Shawn O. Pearce (4):
> index-pack: Refactor base arguments of resolve_delta into a struct
> index-pack: Chain the struct base_data on the stack for traversal
> index-pack: Track the object_entry that creates each base_data
> index-pack: Honor core.deltaBaseCacheLimit when resolving deltas
Well...
I don't like this. Not your patch, but rather the direction this whole
thing took in the first place, and now we have to bolt extra complexity
on top.
I have the feeling this is not the best approach, especially since many
long delta chains are going deep and all those intermediate objects are
simply useless once the _only_ delta child has been resolved and
therefore could be freed right away instead.
But I have to sleep on it first.
Nicolas
^ permalink raw reply
* Re: [PATCH 2/3] add new Git::Repo API
From: Lea Wiemann @ 2008-07-14 2:29 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git, John Hawley, Petr Baudis
In-Reply-To: <200807140128.44923.jnareb@gmail.com>
Jakub Narebski wrote:
> I think it would be perhaps better to explain relationship and purpose
> of each class in more detail, including Git::Repo.
Noted, will do.
>> [Git.pm] tries to do (a) WC access, (b) repo access,
>> and (c) frontend error handling (with sensible error messages).
>
> I can see (b) and (c), but I have trouble seeing (a).
Well, Git.pm operates on working copies in the constructor (obviously),
but also wc_{path,subdir,chdir} and hash_and_insert_object.
>> every working copy has a repository associated with it
>
> Please remember that the opposite relation is also true.
True. *nods*
>> but I'd probably let [Git.pm] die a slow death
>
> I'm not so sure if it is a way to go. Most git commands wants to just
> invoke other git commands safely,
Good point. Perhaps the command functionality of Git.pm and Git::Repo
could be extracted into something like Git::Cmd.
> Non OO things, like ability to write print color('reset') . "\n";
> is also important.
Perhaps, though you might not get around some instantiation to specify
the semantics of the color command: Honor color configuration in
.gitconfig or .git/config? Honor non-terminal stdout? Honor command
line? I suspect that in the end non-OO functions end up being wrappers
around OO interfaces that simply specify a set of reasonable defaults.
> I'm not sure if using Error module was a good idea for
> frontend error handling.
As a general rule, I'd try to not use program exceptions as a means to
do frontend error handling, unless you're trying hard to keep the
frontend minimalist. Even if you don't care about i18n, different
frontends have different needs for their error reporting styles. Also,
things like failed SHA1-lookups might be an error to one frontend but
not an error to another frontend, so you'd have to implement an
exception hierarchy to make fine-granular catching possible.
On top of that, this kind of exception handling doesn't seem very much
like typical Perl style.
> How would you like to catch errors from frontend in Git::Repo and
> friends?
Handle them yourself -- Git::Repo doesn't die unless a fatal (i.e.
unexpected) error occurs:
($sha1, $type) = $repo->get_sha1('HEAD:/my/file');
if (! defined $sha1 || $type ne 'blob') { ... handle error ... }
$contents = $repo->cat_file($sha1);
... work with contents ...
Also note how there's one well-defined (and known) error point: $sha1
being undefined, or the $type being wrong. The $repo methods *cannot*
throw errors unless they're fatal, so you can for instance call cat_file
and assume that everything goes right.
> What is max_exit_code
It allows you call the git binary without dying if it exits with
non-zero status; see the cmd_output documentation for details.
The idea is that a non-zero exit status always indicates an internal
(fatal) error, unless you specify that it's OK.
>> - It's buggy and untested. Neither of these is a problem by itself,
>> but the combination is deadly.
>
> Haven't you added t/t9700-perl-git.sh?
Yes (and it alleviated the problem), but I couldn't test the areas where
the untestedness actually hits (e.g. the missing semicolon I mentioned).
IOW, t9700 is only testing the parts that are working anyway.
> What I worry about is that dependence on Git.pm or Git::Repo would make
> gitweb installation too hard for some.
If I'm not mistaken you can always drop the perl/Git directory next to
gitweb.cgi. (I'll add that to the installation notes.)
>> Unrelatedly, should I add copyright notices at the bottom of each perl
>> module so they are displayed in the perldoc/man pages?
>
> Well, most manpages have information about who made them... which means
> who was initial author, usually, and/or who is current maintainer.
I don't really care about being credited as the initial author, and I'm
honestly not sure if I'll be able to maintain the modules in the long run.
Should I perhaps add some note along the lines of "Direct questions and
patches to git@vger"?
^ permalink raw reply
* Re: [PATCH/Test] Build in merge is broken
From: Junio C Hamano @ 2008-07-14 2:53 UTC (permalink / raw)
To: Sverre Hvammen Johansen; +Cc: Miklos Vajna, git
In-Reply-To: <402c10cd0807131915u6567cba9h361d26d3dc003739@mail.gmail.com>
"Sverre Hvammen Johansen" <hvammen@gmail.com> writes:
> Test cases for build in merge.
> ---
Thanks.
Obviously this is not for application but to help Miklos and others to
help fixing the remaining issues.
Tonight's pu won't have this but that is only because I am currently deep
in today's integration session already.
^ permalink raw reply
* Re: [PATCH 0/4] Honor core.deltaBaseCacheLimit during index-pack
From: Shawn O. Pearce @ 2008-07-14 3:12 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Junio C Hamano, git, Stephan Hennig, Andreas Ericsson
In-Reply-To: <alpine.LFD.1.10.0807132220570.12484@xanadu.home>
Nicolas Pitre <nico@cam.org> wrote:
> On Sun, 13 Jul 2008, Shawn O. Pearce wrote:
>
> > This should resolve the obscene memory usage of index-pack when
> > resolving deltas for very large files.
>
> I don't like this. Not your patch, but rather the direction this whole
> thing took in the first place, and now we have to bolt extra complexity
> on top.
>
> I have the feeling this is not the best approach, especially since many
> long delta chains are going deep and all those intermediate objects are
> simply useless once the _only_ delta child has been resolved and
> therefore could be freed right away instead.
I thought about trying to free a base object if this is the last
resolve_delta() call which would require that data, but I realized
this is just a "tail-call optimization" and doesn't work in the
more general case. This patch series is the best solution I could
come up with to handle even the general case.
The only other alternative I can come up with is to change
pack-objects (or at least its defaults) so we don't generate these
massive delta chains. But there are already packs in the wild that
use these chains, resulting in performance problems for clients.
--
Shawn.
^ permalink raw reply
* [ANNOUNCE] GIT 1.5.6.3
From: Junio C Hamano @ 2008-07-14 3:13 UTC (permalink / raw)
To: git; +Cc: linux-kernel
In-Reply-To: <7vy74fo9t4.fsf@gitster.siamese.dyndns.org>
The latest maintenance release GIT 1.5.6.3 is available at the
usual places:
http://www.kernel.org/pub/software/scm/git/
git-1.5.6.3.tar.{gz,bz2} (source tarball)
git-htmldocs-1.5.6.3.tar.{gz,bz2} (preformatted docs)
git-manpages-1.5.6.3.tar.{gz,bz2} (preformatted docs)
The RPM binary packages for a few architectures are also provided
as courtesy. As the machine I cut them at kernel.org has been updated to FC9
recently, their filenames now have "fc9" in them.
RPMS/$arch/git-*-1.5.6.3-1.fc9.$arch.rpm (RPM)
The changes since the previous maintenance release is getting smaller.
About half the diff is in Documentation/ section.
Documentation/RelNotes-1.5.6.3.txt | 52 ++++++++++++++++++++++++++++++++
Documentation/git-cvsimport.txt | 6 ++++
Documentation/git-cvsserver.txt | 3 ++
Documentation/git-mailinfo.txt | 5 ++-
Documentation/git-rev-parse.txt | 4 +-
Documentation/git.txt | 3 +-
Documentation/gitattributes.txt | 33 ++++++++++----------
Documentation/gitcvs-migration.txt | 5 +++
Documentation/pretty-formats.txt | 8 ++--
RelNotes | 2 +-
builtin-clone.c | 8 +++--
builtin-commit.c | 11 ++----
builtin-log.c | 16 ++-------
builtin-mailinfo.c | 3 +-
builtin-send-pack.c | 3 +-
builtin-tag.c | 2 +-
contrib/completion/git-completion.bash | 30 ++++++++++++++++++
convert.c | 25 +++++----------
diff.c | 8 +----
git-am.sh | 3 +-
git-submodule.sh | 3 +-
git-svn.perl | 2 +
http.c | 36 +++++++--------------
path.c | 2 +-
read-cache.c | 16 ++++++++++
refs.c | 2 +-
run-command.c | 2 +
t/t0004-unwritable.sh | 19 ++++++-----
t/t1301-shared-repo.sh | 23 ++++++++++++++
t/t5404-tracking-branches.sh | 7 ++++
t/t5601-clone.sh | 20 +++++++++++-
transport.c | 4 ++-
32 files changed, 253 insertions(+), 113 deletions(-)
----------------------------------------------------------------
GIT v1.5.6.3 Release Notes
==========================
Fixes since v1.5.6.2
--------------------
* Setting core.sharerepository to traditional "true" value was supposed to make
the repository group writable but should not affect permission for others.
However, since 1.5.6, it was broken to drop permission for others when umask is
022, making the repository unreadable by others.
* Setting GIT_TRACE will report spawning of external process via run_command().
* Using an object with very deep delta chain pinned memory needed for extracting
intermediate base objects unnecessarily long, leading to excess memory usage.
* Bash completion script did not notice '--' marker on the command
line and tried the relatively slow "ref completion" even when
completing arguments after one.
* Registering a non-empty blob racily and then truncating the working
tree file for it confused "racy-git avoidance" logic into thinking
that the path is now unchanged.
* The section that describes attributes related to git-archive were placed
in a wrong place in the gitattributes(5) manual page.
* "git am" was not helpful to the users when it detected that the committer
information is not set up properly yet.
* "git clone" had a leftover debugging fprintf().
* "git clone -q" was not quiet enough as it used to and gave object count
and progress reports.
* "git clone" marked downloaded packfile with .keep; this could be a
good thing if the remote side is well packed but otherwise not,
especially for a project that is not really big.
* "git daemon" used to call syslog() from a signal handler, which
could raise signals of its own but generally is not reentrant. This
was fixed by restructuring the code to report syslog() after the handler
returns.
* When "git push" tries to remove a remote ref, and corresponding
tracking ref is missing, we used to report error (i.e. failure to
remove something that does not exist).
* "git mailinfo" (hence "git am") did not handle commit log messages in a
MIME multipart mail correctly.
Contains other various documentation fixes.
^ permalink raw reply
* Re: [PATCH] Added --export option to git-send-email.
From: Junio C Hamano @ 2008-07-14 3:13 UTC (permalink / raw)
To: Avery Pennarun; +Cc: Eduard - Gabriel Munteanu, ryan, git
In-Reply-To: <32541b130807131642k1382cf84gdf5e8bb8d2ff4ffe@mail.gmail.com>
"Avery Pennarun" <apenwarr@gmail.com> writes:
> If git is going to start actually producing mbox files (as opposed to
> just individual messages as it does now), it should probably
> explicitly take a stance on the issue...
Yeah, that makes one reason I should not be enthusiastic to take this
patch.
^ permalink raw reply
* Re: [PATCH] Added --export option to git-send-email.
From: Avery Pennarun @ 2008-07-14 3:41 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Eduard - Gabriel Munteanu, ryan, git
In-Reply-To: <7vtzetkviy.fsf@gitster.siamese.dyndns.org>
On 7/13/08, Junio C Hamano <gitster@pobox.com> wrote:
> "Avery Pennarun" <apenwarr@gmail.com> writes:
> > If git is going to start actually producing mbox files (as opposed to
> > just individual messages as it does now), it should probably
> > explicitly take a stance on the issue...
>
> Yeah, that makes one reason I should not be enthusiastic to take this
> patch.
Eduard: perhaps instead of a --export option, what about an option to
run an external program that will take the given email on stdin and
process it however you want? ... aha, it seems this already exists in
the --smtp-server option. So a shell script like this as your
--smtp-server should do what you want:
#!/bin/sh
exec sed s/'^\(>*From \)/>\1/'
Thus the result of git-send-email would be a series of emails
concatenated to stdout, with From lines escaped appropriately.
Have fun,
Avery
^ permalink raw reply
* [PATCH] Fix reduce_heads
From: Sverre Hvammen Johansen @ 2008-07-14 4:07 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Miklos Vajna, git
Reduce_heads used by build in merge failed to find the correct
heads in cases where duplicate heads are specified.
---
This should fix the last breakage I found. ([PATCH/Test] Build in
merge is broken)
I have squashed in Miklos's fix and the two tests cases to protect this.
I have several other test cases that were used for the --ff-only patch
I worked on.
Many of the cases for that patch gives protection not currently in any of the
existing tests. If there is interest for it I can squash them in as well.
commit.c | 8 +++++++-
t/t7600-merge.sh | 22 ++++++++++++++++++++++
2 files changed, 29 insertions(+), 1 deletions(-)
diff --git a/commit.c b/commit.c
index d20b14e..4cacf4c 100644
--- a/commit.c
+++ b/commit.c
@@ -746,8 +746,14 @@ struct commit_list *reduce_heads(struct commit_list *heads)
struct commit_list *q, *base;
num_other = 0;
+ for (q = heads; q != p; q = q->next) {
+ if (p->item == q->item)
+ break;
+ }
+ if (q != p)
+ continue;
for (q = heads; q; q = q->next) {
- if (p == q)
+ if (p->item == q->item)
continue;
other[num_other++] = q->item;
}
diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
index 16f4608..80cfee6 100755
--- a/t/t7600-merge.sh
+++ b/t/t7600-merge.sh
@@ -465,4 +465,26 @@ test_expect_success 'merge log message' '
test_debug 'gitk --all'
+test_expect_success 'merge c1 with c0, c2, c0, and c1' '
+ git reset --hard c1 &&
+ git config branch.master.mergeoptions "" &&
+ test_tick &&
+ git merge c0 c2 c0 c1 &&
+ verify_merge file result.1-5 &&
+ verify_parents $c1 $c2
+'
+
+test_debug 'gitk --all'
+
+test_expect_success 'merge c1 with c1 and c2' '
+ git reset --hard c1 &&
+ git config branch.master.mergeoptions "" &&
+ test_tick &&
+ git merge c1 c2 &&
+ verify_merge file result.1-5 &&
+ verify_parents $c1 $c2
+'
+
+test_debug 'gitk --all'
+
test_done
--
Sverre Hvammen Johansen
^ permalink raw reply related
* What's cooking in git.git (topics)
From: Junio C Hamano @ 2008-07-14 5:11 UTC (permalink / raw)
To: git
In-Reply-To: <7vabgqsc37.fsf@gitster.siamese.dyndns.org>
Here are the topics that have been cooking. Commits prefixed
with '-' are only in 'pu' while commits prefixed with '+' are
in 'next'.
The topics list the commits in reverse chronological order. The topics
meant to be merged to the maintenance series have "maint-" in their names.
I think most of the important stuff is already in 'next'. Let's start
talking about closing the merge window for 1.6.0.
----------------------------------------------------------------
[New Topics]
* sb/dashless (Sun Jul 13 15:36:15 2008 +0200) 3 commits
- Make usage strings dash-less
- t/: Use "test_must_fail git" instead of "! git"
- t/test-lib.sh: exit with small negagive int is ok with
test_must_fail
* mv/dashless (Fri Jul 11 02:12:06 2008 +0200) 4 commits
- make remove-dashes: apply to scripts and programs as well, not
just to builtins
- git-bisect: use dash-less form on git bisect log
- t1007-hash-object.sh: use quotes for the test description
- t0001-init.sh: change confusing directory name
* sp/maint-bash-completion-optim (Mon Jul 14 00:22:03 2008 +0000) 1 commit
+ bash completion: Append space after file names have been completed
Early parts are already merged to 'master' and need to be merged down to
maint as well, as this is about a "performance bug" that has been with us
almost forever.
* ag/rewrite_one (Sat Jul 12 22:00:57 2008 +0400) 1 commit
+ Fix quadratic performance in rewrite_one.
* sp/win (Fri Jul 11 18:52:42 2008 +0200) 3 commits
+ We need to check for msys as well as Windows in add--interactive.
+ Convert CR/LF to LF in tag signatures
+ Fixed text file auto-detection: treat EOF character 032 at the end
of file as printable
* js/merge-rr (Sat Jul 12 15:56:19 2008 +0100) 2 commits
+ Move MERGE_RR from .git/rr-cache/ into .git/
+ builtin-rerere: more carefully find conflict markers
* sb/rerere-lib (Wed Jul 9 14:58:57 2008 +0200) 2 commits
+ rerere: Separate libgit and builtin functions
+ builtin-rerere: more carefully find conflict markers
* ls/mailinfo (Sun Jul 13 20:30:12 2008 +0200) 3 commits
- git-mailinfo: use strbuf's instead of fixed buffers
- Add some useful functions for strbuf manipulation.
- Make some strbuf_*() struct strbuf arguments const.
* gi/cherry-cache (Sat Jul 12 20:14:51 2008 -0700) 1 commit
- cherry: cache patch-ids to avoid repeating work
This does not seem to pass tests even on its own.
* js/maint-pretty-mailmap (Sat Jul 12 00:28:18 2008 +0100) 1 commit
+ Add pretty format %aN which gives the author name, respecting
.mailmap
* js/more-win (Sun Jul 13 22:31:23 2008 +0200) 6 commits
- Allow add_path() to add non-existent directories to the path
- Allow the built-in exec path to be relative to the command
invocation path
- Fix relative built-in paths to be relative to the command
invocation
+ help (Windows): Display HTML in default browser using Windows'
shell API
+ help.c: Add support for htmldir relative to git_exec_path()
+ Move code interpreting path relative to exec-dir to new function
system_path()
The earlier parts are obvious; Dscho seemed to have some comments on the
later ones that are in 'pu'.
* lw/gitweb (Fri Jul 11 03:11:48 2008 +0200) 3 commits
- gitweb: use new Git::Repo API, and add optional caching
- Add new Git::Repo API
- gitweb: add test suite with Test::WWW::Mechanize::CGI
This does not pass t9710, at least for me X-<.
----------------------------------------------------------------
[Will merge to master soon]
* jc/rebase-orig-head (Tue Jul 8 00:12:22 2008 -0400) 2 commits
+ Documentation: mention ORIG_HEAD in am, merge, and rebase
+ Teach "am" and "rebase" to mark the original position with
ORIG_HEAD
* jc/branch-merged (Tue Jul 8 17:55:47 2008 -0700) 3 commits
+ branch --merged/--no-merged: allow specifying arbitrary commit
+ branch --contains: default to HEAD
+ parse-options: add PARSE_OPT_LASTARG_DEFAULT flag
This builds on top of the parse-options enhancement series that
has been cooking in 'next' for some time.
* om/rerere-careful (Mon Jul 7 14:42:48 2008 +0200) 1 commit
+ builtin-rerere: more carefully find conflict markers
* ls/maint-mailinfo-patch-label (Thu Jul 10 23:41:33 2008 +0200) 1 commit
+ git-mailinfo: Fix getting the subject from the in-body [PATCH]
line
----------------------------------------------------------------
[Actively Cooking]
* xx/merge-in-c-into-next (Wed Jul 9 13:51:46 2008 -0700) 4 commits
+ Teach git-merge -X<option> again.
+ Merge branch 'jc/merge-theirs' into xx/merge-in-c-into-next
+ builtin-merge.c: use parse_options_step() "incremental parsing"
machinery
+ Merge branch 'ph/parseopt-step-blame' into xx/merge-in-c-into-next
I've described what this is in a separate message.
* rs/imap (Wed Jul 9 22:29:02 2008 +0100) 5 commits
- Documentation: Improve documentation for git-imap-send(1)
- imap-send.c: more style fixes
- imap-send.c: style fixes
- git-imap-send: Support SSL
- git-imap-send: Allow the program to be run from subdirectories of
a git tree
Some people seem to prefer having this feature available also with gnutls.
If such a patch materializes soon, that would be good, but otherwise I'll
merge this as-is to 'next'. Such an enhancement can be done in-tree on
top of this series.
* jc/merge-theirs (Mon Jun 30 22:18:57 2008 -0700) 5 commits
+ Make "subtree" part more orthogonal to the rest of merge-
recursive.
+ Teach git-pull to pass -X<option> to git-merge
+ Teach git-merge to pass -X<option> to the backend strategy module
+ git-merge-recursive-{ours,theirs}
+ git-merge-file --ours, --theirs
Punting a merge by discarding your own work in conflicting parts but still
salvaging the parts that are cleanly automerged. It is likely that this
will result in nonsense mishmash, but somehow often people want this, so
here they are. The interface to the backends is updated so that you can
say "git merge -Xours -Xsubtree=foo/bar/baz -s recursive other" now.
* mv/merge-in-c (Sun Jul 13 08:13:55 2008 +0000) 19 commits
+ reduce_heads(): thinkofix
+ Add a new test for git-merge-resolve
+ t6021: add a new test for git-merge-resolve
+ Teach merge.log to "git-merge" again
+ Build in merge
+ Fix t7601-merge-pull-config.sh on AIX
+ git-commit-tree: make it usable from other builtins
+ Add new test case to ensure git-merge prepends the custom merge
message
+ Add new test case to ensure git-merge reduces octopus parents when
possible
+ Introduce reduce_heads()
+ Introduce get_merge_bases_many()
+ Add new test to ensure git-merge handles more than 25 refs.
+ Introduce get_octopus_merge_bases() in commit.c
+ git-fmt-merge-msg: make it usable from other builtins
+ Move read_cache_unmerged() to read-cache.c
+ Add new test to ensure git-merge handles pull.twohead and
pull.octopus
+ Move parse-options's skip_prefix() to git-compat-util.h
+ Move commit_list_count() to commit.c
+ Move split_cmdline() to alias.c
Sverre seems to have a yet another fixup on top of this that came late and
I haven't looked at.
----------------------------------------------------------------
[Graduated to "master"]
* js/pick-root (Fri Jul 4 16:19:52 2008 +0100) 1 commit
+ Allow cherry-picking root commits
* ab/bundle (Sat Jul 5 17:26:40 2008 -0400) 1 commit
+ Teach git-bundle to read revision arguments from stdin like git-
rev-list.
* sg/stash-k-i (Tue Jul 8 00:40:56 2008 -0700) 2 commits
+ Documentation: tweak use case in "git stash save --keep-index"
+ stash: introduce 'stash save --keep-index' option
One weakness of our "partial commit" workflow support used to be that the
user can incrementally build what is to be committed in the index but that
state cannot be tested as a whole in the working tree. This allows you to
temporarily stash the remaining changes in the working tree so that the
index state before running "stash save --keep-index" can be seen in the
working tree to be tested and then committed.
* am/stash-branch (Mon Jul 7 02:50:10 2008 +0530) 2 commits
+ Add a test for "git stash branch"
+ Implement "git stash branch <newbranch> <stash>"
Creates a new branch out of the stashed state, after returning from the
interrupt that forced you to create the stash in the first place.
* tr/add-i-e (Thu Jul 3 00:00:00 2008 +0200) 3 commits
+ git-add--interactive: manual hunk editing mode
+ git-add--interactive: remove hunk coalescing
+ git-add--interactive: replace hunk recounting with apply --recount
Adds 'e/dit' action to interactive add command.
* jc/report-tracking (Sun Jul 6 02:54:56 2008 -0700) 5 commits
+ branch -r -v: do not spit out garbage
+ stat_tracking_info(): clear object flags used during counting
+ git-branch -v: show the remote tracking statistics
+ git-status: show the remote tracking statistics
+ Refactor "tracking statistics" code used by "git checkout"
Makes the "your branch is ahead of the tracked one by N commits" logic and
messages available to other commands; status and branch are updated.
* ph/parseopt-step-blame (Wed Jul 9 23:38:34 2008 +0200) 18 commits
+ revisions: refactor handle_revision_opt into parse_revision_opt.
+ git-shortlog: migrate to parse-options partially.
+ git-blame: fix lapsus
+ git-blame: migrate to incremental parse-option [2/2]
+ git-blame: migrate to incremental parse-option [1/2]
+ revisions: split handle_revision_opt() from setup_revisions()
+ Merge branch 'jc/blame' (early part) into HEAD
+ parse-opt: add PARSE_OPT_KEEP_ARGV0 parser option.
+ parse-opt: fake short strings for callers to believe in.
+ parse-opt: do not print errors on unknown options, return -2
intead.
+ parse-opt: create parse_options_step.
+ parse-opt: Export a non NORETURN usage dumper.
+ parse-opt: have parse_options_{start,end}.
+ git-blame --reverse
+ builtin-blame.c: allow more than 16 parents
+ builtin-blame.c: move prepare_final() into a separate function.
+ rev-list --children
+ revision traversal: --children option
Became active again ;-) This probably is ready for 'master' already,
except for the last two which I only looked at the patch and have not
used heavily in production yet.
----------------------------------------------------------------
[On Hold]
* sg/merge-options (Sun Apr 6 03:23:47 2008 +0200) 1 commit
+ merge: remove deprecated summary and diffstat options and config
variables
This was previously in "will be in master soon" category, but it turns out
that the synonyms to the ones this one deletes are fairly new invention
that happend in 1.5.6 timeframe, and we cannot do this just yet. Perhaps
in 1.7.0.
* jc/dashless (Thu Jun 26 16:43:34 2008 -0700) 2 commits
+ Revert "Make clients ask for "git program" over ssh and local
transport"
+ Make clients ask for "git program" over ssh and local transport
This is the "botched" one. Will be resurrected during 1.7.0 or 1.8.0
timeframe.
* jk/renamelimit (Sat May 3 13:58:42 2008 -0700) 1 commit
- diff: enable "too large a rename" warning when -M/-C is explicitly
asked for
This would be the right thing to do for command line use, but gitk will be
hit due to tcl/tk's limitation, so I am holding this back for now.
----------------------------------------------------------------
[Stalled/Needs more work]
* sb/sequencer (Tue Jul 1 04:38:34 2008 +0200) 4 commits
. Migrate git-am to use git-sequencer
. Add git-sequencer test suite (t3350)
. Add git-sequencer prototype documentation
. Add git-sequencer shell prototype
* jc/grafts (Wed Jul 2 17:14:12 2008 -0700) 1 commit
- [BROKEN wrt shallow clones] Ignore graft during object transfer
Cloning or fetching from a repository from grafts did not send objects
that are hidden by grafts, but the commits in the resulting repository do
need these to pass fsck. This fixes object transfer to ignore grafts.
Another fix is needed to git-prune so that it ignores grafts but treats
commits that are mentioned in grafts as reachable.
* jc/blame (Wed Jun 4 22:58:40 2008 -0700) 2 commits
- blame: show "previous" information in --porcelain/--incremental
format
- git-blame: refactor code to emit "porcelain format" output
This is for peeling to see what's behind the blamed commit, which may or
may not help applications like gitweb.
^ permalink raw reply
* What's in git.git (stable)
From: Junio C Hamano @ 2008-07-14 5:33 UTC (permalink / raw)
To: git
In-Reply-To: <7v1w25xfd6.fsf@gitster.siamese.dyndns.org>
* The 'maint' branch is at 1.5.6.3
* The 'master' branch has these since the last announcement
in addition to what's in 1.5.6.3
Abhijit Menon-Sen (2):
Implement "git stash branch <newbranch> <stash>"
Add a test for "git stash branch"
Adam Brewster (1):
Teach git-bundle to read revision arguments from stdin like git-rev-list.
Brandon Casey (1):
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects
Eric Hanchrow (2):
user-manual: typo and grammar fixes
Documentation: fix broken "linkgit" links
Eric Raible (2):
Documentation: tweak use case in "git stash save --keep-index"
completion: add branch options --contains --merged --no-merged
Jeff King (2):
Allow per-command pager config
avoid null SHA1 in oldest reflog
Johannes Schindelin (2):
Teach "git apply" to prepend a prefix with "--root=<root>"
Allow cherry-picking root commits
Johannes Sixt (1):
Provide fallback definitions of PRIu32 and PRIx32
Junio C Hamano (17):
revision traversal: --children option
rev-list --children
builtin-blame.c: move prepare_final() into a separate function.
builtin-blame.c: allow more than 16 parents
git-blame --reverse
Per-ref reflog expiry configuration
Make default expiration period of reflog used for stash infinite
apply --root: thinkofix.
Refactor "tracking statistics" code used by "git checkout"
git-status: show the remote tracking statistics
git-branch -v: show the remote tracking statistics
stat_tracking_info(): clear object flags used during counting
branch -r -v: do not spit out garbage
git-apply --directory: make --root more similar to GNU diff
Tone down warning about GNU Interactive Tools
Documentation: update sections on naming revisions and revision ranges
apply: fix copy/rename breakage
Mark Levedahl (1):
install-doc-quick - use git --exec-path to find git-sh-setup
Mike Hommey (4):
Catch failures from t5540-http-push
Fix http-push test
Skip t5540-http-push test when USE_CURL_MULTI is undefined
Avoid apache complaining about lack of server's FQDN
Petr Baudis (1):
Git.pm: Add remote_refs() git-ls-remote frontend
Pierre Habouzit (12):
parse-opt: have parse_options_{start,end}.
parse-opt: Export a non NORETURN usage dumper.
parse-opt: create parse_options_step.
parse-opt: do not print errors on unknown options, return -2 intead.
parse-opt: fake short strings for callers to believe in.
parse-opt: add PARSE_OPT_KEEP_ARGV0 parser option.
revisions: split handle_revision_opt() from setup_revisions()
git-blame: migrate to incremental parse-option [1/2]
git-blame: migrate to incremental parse-option [2/2]
git-blame: fix lapsus
git-shortlog: migrate to parse-options partially.
revisions: refactor handle_revision_opt into parse_revision_opt.
Ramsay Allan Jones (3):
t9113-*.sh: provide user feedback when test skipped
t9100-git-svn-basic.sh: Fix determination of utf-8 locale
git-request-pull: replace call to deprecated peek-remote
Robert Shearman (1):
git-send-email: Fix authenticating on some servers when using TLS.
SZEDER Gábor (1):
stash: introduce 'stash save --keep-index' option
Shawn O. Pearce (3):
Correct pack memory leak causing git gc to try to exceed ulimit
bash completion: Improve responsiveness of git-log completion
bash completion: Don't offer "a.." as a completion for "a."
Stephan Beyer (3):
git-am/git-mailsplit: correct synopsis for reading from stdin
t3404: test two "preserve merges with -p" cases
Make rebase--interactive use OPTIONS_SPEC
Thomas Rast (3):
git-add--interactive: replace hunk recounting with apply --recount
git-add--interactive: remove hunk coalescing
git-add--interactive: manual hunk editing mode
^ permalink raw reply
* Re: [PATCH] bash completion: Fix the . -> .. revision range completion
From: Linus Torvalds @ 2008-07-14 5:38 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: Junio C Hamano, Petr Baudis, git
In-Reply-To: <20080714000021.GB13066@spearce.org>
On Mon, 14 Jul 2008, Shawn O. Pearce wrote:
> >
> > Does it fix this one too:
> >
> > git show origin/pu:Makef<tab>
> >
> > which totally screws up and becomes
> >
> > git show Makefile
> >
> > dropping the version specifier?
>
> Its unrelated to the issue you described above. But I just tested
> your Makefile completion case and it worked as expected, though
> I just found out it doesn't add a trailing space once there is a
> unique completion made. I'll look at another patch to fix that.
What version of bash do you have?
It definitely doesn't work for me with
GNU bash, version 3.2.33(1)-release (x86_64-redhat-linux-gnu)
and quite frankly, I don't see how it _can_ work.
Something like this seems to be totally missing, and definitely required.
How can you say that it works for you? I don't see how that is possible
even in theory? Did you actually test it?
Linus
---
contrib/completion/git-completion.bash | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 27332ed..0a87337 100755
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -290,7 +290,7 @@ __git_complete_file ()
ls="$ref"
;;
esac
- COMPREPLY=($(compgen -P "$pfx" \
+ COMPREPLY=($(compgen -P "$ref:$pfx" \
-W "$(git --git-dir="$(__gitdir)" ls-tree "$ls" \
| sed '/^100... blob /s,^.* ,,
/^040000 tree /{
^ permalink raw reply related
* A note from the maintainer
From: Junio C Hamano @ 2008-07-14 5:51 UTC (permalink / raw)
To: git
In-Reply-To: <7vbq1xx4y8.fsf@gitster.siamese.dyndns.org>
Welcome to git community.
This message talks about how git.git is managed, and how you can work
with it.
* IRC and Mailing list
Many active members of development community hang around on #git
IRC channel on Freenode. Its log is available at:
http://colabti.org/irclogger/irclogger_log/git
The development however is primarily done on the git mailing list
(git@vger.kernel.org). If you have patches, please send them to the
list, following Documentation/SubmittingPatches.
I usually try to read all patches posted to the list, and follow
almost all the discussions on the list, unless the topic is about an
obscure corner that I do not personally use. But I am obviously not
perfect. If you sent a patch that you did not hear from anybody for
three days, that is a very good indication that it was dropped on the
floor --- please do not hesitate to remind me.
The list archive is available at a few public sites as well:
http://news.gmane.org/gmane.comp.version-control.git/
http://marc.theaimsgroup.com/?l=git
http://www.spinics.net/lists/git/
and some people seem to prefer to read it over NNTP:
nntp://news.gmane.org/gmane.comp.version-control.git
When you point at a message in a mailing list archive, using
gmane is often the easiest to follow by readers, like this:
http://thread.gmane.org/gmane.comp.version-control.git/27/focus=217
as it also allows people who subscribe to the mailing list as
gmane newsgroup to "jump to" the article.
* Repositories, branches and documentation.
My public git.git repository is at:
git://git.kernel.org/pub/scm/git/git.git/
Immediately after I publish to the primary repository at kernel.org, I
also push into an alternate here:
git://repo.or.cz/alt-git.git/
Impatient people might have better luck with the latter one.
Their gitweb interfaces are found at:
http://git.kernel.org/?p=git/git.git
http://repo.or.cz/w/alt-git.git
There are three branches in git.git repository that are not
about the source tree of git: "todo", "html" and "man". The
first one was meant to contain TODO list for me, but I am not
good at maintaining such a list and it is not as often updated as
it could/should be. It also contains some helper scripts I use
to maintain git.
The "html" and "man" are autogenerated documentation from the
tip of the "master" branch; the tip of "html" is extracted to be
visible at kernel.org at:
http://www.kernel.org/pub/software/scm/git/docs/
The above URL is the top-level documentation page, and it has
links to documentation of older releases.
The script to maintain these two documentation branches are
found in "todo" branch as dodoc.sh, if you are interested. It
is a good demonstration of how to use an update hook to automate
a task.
There are four branches in git.git repository that track the
source tree of git: "master", "maint", "next", and "pu". I may
add more maintenance branches (e.g. "maint-1.5.4") if we have
hugely backward incompatible feature updates in the future to keep
an older release alive; I may not, but the distributed nature of
git means any volunteer can run a stable-tree like that herself.
The "master" branch is meant to contain what are very well
tested and ready to be used in a production setting. There
could occasionally be minor breakages or brown paper bag bugs
but they are not expected to be anything major, and more
importantly quickly and trivially fixable. Every now and
then, a "feature release" is cut from the tip of this branch and
they typically are named with three dotted decimal digits. The
last such release was 1.5.6 done on Jun 18th this year. You
can expect that the tip of the "master" branch is always more
stable than any of the released versions.
Whenever a feature release is made, "maint" branch is forked off
from "master" at that point. Obvious, safe and urgent fixes
after a feature release are applied to this branch and
maintenance releases are cut from it. The maintenance releases
are named with four dotted decimal, named after the feature
release they are updates to; the last such release was 1.5.6.3.
New features never go to this branch. This branch is also
merged into "master" to propagate the fixes forward.
A trivial and safe enhancement goes directly on top of "master".
A new development, either initiated by myself or more often by
somebody who found his or her own itch to scratch, does not
usually happen on "master", however. Instead, a separate topic
branch is forked from the tip of "master", and it first is
tested in isolation; I may make minimum fixups at this point.
Usually there are a handful such topic branches that are running
ahead of "master" in git.git repository. I do not publish the
tip of these branches in my public repository, however, partly
to keep the number of branches that downstream developers need
to worry about low, and primarily because I am lazy.
The quality of topic branches are judged primarily by the mailing list
discussions. Some of them start out as "good idea but obviously is
broken in some areas (e.g. breaks the existing testsuite)" and then
with some more work (either by the original contributor's effort or
help from other people on the list) becomes "more or less done and can
now be tested by wider audience". Luckily, most of them start out in
the latter, better shape.
The "next" branch is to merge and test topic branches in the
latter category. In general, the branch always contains the tip
of "master". It might not be quite rock-solid production ready,
but is expected to work more or less without major breakage. I
usually use "next" version of git for my own work, so it cannot
be _that_ broken to prevent me from pushing the changes out.
The "next" branch is where new and exciting things take place.
The two branches "master" and "maint" are never rewound, and
"next" usually will not be either (this automatically means the
topics that have been merged into "next" are usually not
rebased, and you can find the tip of topic branches you are
interested in from the output of "git log next"). You should be
able to safely track them.
After a feature release is made from "master", however, "next"
will be rebuilt from the tip of "master" using the surviving
topics. The commit that replaces the tip of the "next" will
have the identical tree, but it will have different ancestry
from the tip of "master". An announcement will be made to warn
people about such a rebasing.
The "pu" (proposed updates) branch bundles all the remainder of
topic branches. The "pu" branch, and topic branches that are
only in "pu", are subject to rebasing in general. By the above
definition of how "next" works, you can tell that this branch
will contain quite experimental and obviously broken stuff.
When a topic that was in "pu" proves to be in testable shape, it
graduates to "next". I do this with:
git checkout next
git merge that-topic-branch
Sometimes, an idea that looked promising turns out to be not so
good and the topic can be dropped from "pu" in such a case.
A topic that is in "next" is expected to be tweaked and fixed to
perfection before it is merged to "master" (that's why "master"
can be expected to stay very stable). Similarly to the above, I
do it with this:
git checkout master
git merge that-topic-branch
git branch -d that-topic-branch
Note that being in "next" is not a guarantee to appear in the
next release (being in "master" is such a guarantee, unless it
is later found seriously broken and reverted), or even in any
future release. There even were cases that topics needed
reverting a few commits in them before graduating to "master",
or a topic that already was in "next" were entirely reverted
from "next" because fatal flaws were found in them later.
Starting from v1.5.0, "master" and "maint" have release notes
for the next release in Documentation/RelNotes-* files, so that
I do not have to run around summarizing what happened just
before the release.
* Other people's trees, trusted lieutenants and credits.
Documentation/SubmittingPatches outlines who your changes should
be sent to. As described in contrib/README, I would delegate
fixes and enhancements in contrib/ area to primary contributors
of them.
Although the following are included in git.git repository, they
have their own authoritative repository and maintainers:
- git-gui/ comes from Shawn Pearce's git-gui project:
git://repo.or.cz/git-gui.git
- gitk-git/ comes from Paul Mackerras's gitk project:
git://git.kernel.org/pub/scm/gitk/gitk.git
I would like to thank everybody who helped to raise git into the
current shape. Especially I would like to thank the git list
regulars whose help I have relied on and expect to continue
relying on heavily:
- Linus on general design issues.
- Linus, Shawn Pearce, Johannes Schindelin, Nicolas Pitre,
René Scharfe and Jeff King on general implementation issues.
- Shawn and Nicolas Pitre on pack issues.
- Martin Langhoff and Frank Lichtenheld on cvsserver and cvsimport.
- Paul Mackerras on gitk.
- Eric Wong on git-svn.
- Simon Hausmann on git-p4.
- Jakub Narebski, Petr Baudis, and Luben Tuikov on gitweb.
- J. Bruce Fields on documentaton issues.
- Johannes Schindelin, Johannes Sixt and others for their effort
to move things forward on the Windows front. Most of the fruits
from their porting efforts have been merged to the mainline git.git
repository in 1.6.0 release.
- People on non-Linux platforms for keeping their eyes on
portability; especially, Randal Schwartz, Theodore Ts'o,
Jason Riedy, Thomas Glanzmann, but countless others as well.
* This document
The latest copy of this document is found in git.git repository,
on 'todo' branch, as MaintNotes.
^ permalink raw reply
* Re: [PATCH] bash completion: Fix the . -> .. revision range completion
From: Shawn O. Pearce @ 2008-07-14 5:57 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, Petr Baudis, git
In-Reply-To: <alpine.LFD.1.10.0807132210430.3305@woody.linux-foundation.org>
Linus Torvalds <torvalds@linux-foundation.org> wrote:
> On Mon, 14 Jul 2008, Shawn O. Pearce wrote:
> > >
> > > Does it fix this one too:
> > >
> > > git show origin/pu:Makef<tab>
> > >
> > > which totally screws up and becomes
> > >
> > > git show Makefile
> > >
> > > dropping the version specifier?
>
> What version of bash do you have?
GNU bash, version 3.2.17(1)-release (i386-apple-darwin9.0)
> It definitely doesn't work for me with
>
> GNU bash, version 3.2.33(1)-release (x86_64-redhat-linux-gnu)
So between .17 and .33 they changed something.
> Something like this seems to be totally missing, and definitely required.
> How can you say that it works for you? I don't see how that is possible
> even in theory? Did you actually test it?
Yes, of course dammit, I did test it before writing an email that
said "I tested this ...".
> diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
> index 27332ed..0a87337 100755
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
> @@ -290,7 +290,7 @@ __git_complete_file ()
> ls="$ref"
> ;;
> esac
> - COMPREPLY=($(compgen -P "$pfx" \
> + COMPREPLY=($(compgen -P "$ref:$pfx" \
> -W "$(git --git-dir="$(__gitdir)" ls-tree "$ls" \
> | sed '/^100... blob /s,^.* ,,
> /^040000 tree /{
In bash 3.2.17 your change above causes completion to include the
ref name _twice_:
git show jc/html:in<tab>
git show jc/html:jc/html:index.html
This is not good news because it means we need to perform a version
test to identify the correct behavior we need for this shell, and
we also have to figure out what version of 3.2.X this changed in.
*sigh*
--
Shawn.
^ permalink raw reply
* Re: [PATCH] bash completion: Fix the . -> .. revision range completion
From: Shawn O. Pearce @ 2008-07-14 6:27 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, Petr Baudis, git
In-Reply-To: <alpine.LFD.1.10.0807132210430.3305@woody.linux-foundation.org>
Linus Torvalds <torvalds@linux-foundation.org> wrote:
> On Mon, 14 Jul 2008, Shawn O. Pearce wrote:
> > >
> > > Does it fix this one too:
> > >
> > > git show origin/pu:Makef<tab>
> > >
> > > which totally screws up and becomes
> > >
> > > git show Makefile
> > >
> > > dropping the version specifier?
What is $COMP_WORDBREAKS set to in your shell? In mine it
appears to be:
" \"'@><=;|&(:"
which is the I believe the shell default.
Björn Steinbrink (doener on #git) is running bash 3.2.39 from
Debian and has the same setting, and the completion works correctly
there too. He reports that removing : from COMP_WORDBREAKS will
get the behavior you are reporting as broken.
I have to say, this sounds to me like you (or some package on your
system) modified COMP_WORDBREAKS away from the default that other
distributions use and that is what is breaking us here. Since we
can have only one setting for this variable in the shell I do not
thing it would be a good idea for our completion package to force
a specific setting upon the user.
Though we could try to detect : in there and if it is not present
use the workaround you posted. But I wonder if just asking the
user to include : is easier.
--
Shawn.
^ permalink raw reply
* Re: [StGit PATCH] Remove --undo flags from stg commands and docs
From: Karl Hasselström @ 2008-07-14 6:27 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
In-Reply-To: <b0943d9e0807120258g26e30c4bn5b7f77f69a0c3db3@mail.gmail.com>
On 2008-07-12 10:58:44 +0100, Catalin Marinas wrote:
> 2008/7/8 Karl Hasselström <kha@treskal.com>:
>
> > On 2008-07-07 21:54:01 +0100, Catalin Marinas wrote:
> >
> > > Anyway, have you heard of anyone else using sync apart from me?
> >
> > No, I haven't.
> >
> > I've tried to understand what it does, and as far as I can tell it
> > doesn't do quite what I want. (What I want is the ability to 3-way
> > merge StGit patch stacks, so that I can modify the same patch
> > stack in several places and merge back and forth. From what I
> > recall, the sync command is more like a 2-way merge -- that is, it
> > doesn't take the last common ancestor into account. But it's been
> > a while since I studied it.)
>
> No, the sync command is a 3-way merge between the top of a remote
> patch, the top of the current patch (current HEAD actually) and the
> bottom of the remote patch as ancestor. It also has a mode to
> synchronise with a patch file and it applies the patch onto the
> bottom of the current patch and does the merging between patch tops
> afterwards.
>
> This is not ideal as I have to deal with safe conflicts several time
> (maybe adding git-rerere support would help). If you have better
> ideas for keeping two stacks in sync, I'm happy to accept them (or
> maybe just a different workflow).
Yeah, the actual _tree merging_ is 3-way, but the _patch merging_ is
2-way, since you don't consider the patches' latest common ancestor.
So you run into problems much like what you'd get if you tried to
apply a patch, change it, then apply it again, etc.
When merging two branches with stack logs, you can find the stack log
of the merge base, and use that to 3-way merge the patches.
( To 3-way merge a patch:
1. You have the common ancestor (A), and the two diverged patches
(B and C).
2. Push A, B, and C separately onto the same commit. (If there are
conflicts at this stage, one probably wants to just ignore them
automatically, like merge-recursive does.) Now, A_b == B_b ==
C_b. (_b is the patch's bottom; _t is its top.)
3. 3-way merge A_t, B_t and C_t to get the new patch top.
4. Commit message etc. can be merged with a standard 3-way merge.
To merge an entire patch stack, first merge the stack bases (with an
ordinary 3-way merge, for example), then merge any changes is the
patch order, and then merge all the patches one by one.
As I've said before, I haven't actually tried this, so I don't know
if it'll work as intended. But it sounds right ... and it works out
OK for trivial cases like when the patch is only modified in one
branch, or when no patch bottoms are changed by step (2). )
--
Karl Hasselström, kha@treskal.com
www.treskal.com/kalle
^ permalink raw reply
* Re: [StGit PATCH 03/14] Write to a stack log when stack is modified
From: Karl Hasselström @ 2008-07-14 6:32 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
In-Reply-To: <b0943d9e0807120309jb01aa09se5749a19bc526240@mail.gmail.com>
On 2008-07-12 11:09:18 +0100, Catalin Marinas wrote:
> 2008/6/23 Karl Hasselström <kha@treskal.com>:
>
> > Two optimizations would hopefully make it fast:
> >
> > 1. If the patch's sha1 hasn't changed, we don't have to
> > regenerate the diff.
> >
> > 2. If the patch's sha1 has changed, but git apply was sufficient
> > during the merge stage, we can just reuse that patch. We do
> > have to write it to a blob, but we have already generated the
> > diff and don't need to do so again. (I've shamelessly stolen
> > your idea here.)
>
> It can be optimised a bit more to actually apply the diff in the
> blob directly rather than the current way of generating the diff
> (since we don't store the diff).
Ah. Yes, that might be faster. While git is supposedly very fast in
diffing two trees, it should be even faster in just reading a blob.
> > In most cases, (1) would make sure that only a small handful of
> > patches would need to be considered. In the cases where a lot of
> > patches are touched, such as rebase, (2) would provide a good
> > speedup (except for the cases where we had to call
> > merge-recursive, and those are slow anyway).
>
> I think it should work. Rebase is indeed my worry but it might be
> even faster for most of the patches to apply the blob than computing
> the diff. In my experience with the Linux kernel, full merge is
> rarely needed.
Yes. Now all we need is a good benchmarking system so that we know
we're heading in the right direction. I need to fix up and post that
script I used.
--
Karl Hasselström, kha@treskal.com
www.treskal.com/kalle
^ permalink raw reply
* Re: [StGit PATCH 2/2] Reuse the same temp index in a transaction
From: Karl Hasselström @ 2008-07-14 6:35 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
In-Reply-To: <b0943d9e0807120324l7674c010w9a8e4a0bbdeeee65@mail.gmail.com>
On 2008-07-12 11:24:24 +0100, Catalin Marinas wrote:
> 2008/7/2 Karl Hasselström <kha@treskal.com>:
>
> > Instead of making a new temp index every time we need one, just
> > keep reusing the same one. And keep track of which tree is
> > currently stored in it -- if we do several consecutive successful
> > pushes, it's always going to be the "right" tree so that we don't
> > have to call read-tree before each patch application.
> >
> > The motivation behind this change is of course that it makes
> > things faster.
> >
> > (The same simple test as in the previous patch -- pushing 250
> > patches in a 32k-file repository, with one file-level merge
> > necessary per push -- went from 0.36 to 0.19 seconds per patch
> > with this patch applied.)
>
> That's an impressive improvement (together with the previous patch).
> Is this with the new patch log infrastructure?
Yes, with all the bells and whistles -- in particular, with the
overhead of the simplified log. (But note that it's a synthetic
benchmark, and not a real linux-kernel patch series.)
--
Karl Hasselström, kha@treskal.com
www.treskal.com/kalle
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox