* Re: A generalization of git notes from blobs to trees - git metadata?
From: Jon Seymour @ 2010-02-07 9:41 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Jeff King, Junio C Hamano, Johan Herland, git
In-Reply-To: <m363699zn4.fsf@localhost.localdomain>
On Sun, Feb 7, 2010 at 8:15 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> Jon Seymour <jon.seymour@gmail.com> writes:
>
> [cut]
>
>> As I see it, the existing use of notes is a special instance of a more
>> general metadata capability in which the metadata is constrained to be
>> a single blob. If notes continued to be constrained in this way, there
>> is no reason to change anything with respect to its current userspace
>> behaviour. That said, most of the plumbing which enabled notes could
>> be generalized to enable the arbitrary tree case [ which admittedly, I
>> have yet to sell successfully !]
>>
>> In one sense, there is a sense in the merge issue doesn't exist. When
>> the maintainer publishes a tag no-one expects to have to deal with
>> downstream conflicting definitions of the tag. Likewise, if the
>> maintainer were to publish the /man and /html metadata trees (per my
>> previous example) for a release tag, anyone who received
>> /refs/metadata/doc would expect to receive the metadata trees as
>> published by the maintainer. Anyone who didn't wouldn't have to pull
>> /refs/metadata/doc.
>>
>> I can see there are use cases where multiple parties might want to
>> contribute metadata and I do not currently have a good solution to
>> that problem, but that is not to say there isn't one - surely it is
>> just a question of applying a little intellect creatively?
>
> Are you trying to repeat fail of Apple's / MacOS / HFS+ filesystem
> data/resource forks, and Microsoft's Alternate Data Streams in git? :-)
>
No I am not. I don't see why a metadata proposal is any more exposed
to subversive payloads than say, use of git merge -s ours [ a
subversive payload could be made reachable from a commit that
otherwise merges in favour of the legitimate source - who would know?
]
Really, I can't see why the rationale that makes a single blob used
for extending a commit message justified can't be used to justify
associating a metadata tree of arbitrary complexity to an arbitrary
sha1 object. What makes maintaining a mapping to a single blob
acceptable but maintaining a mapping to a tree unacceptable? Is there
really any fundamental difference?
jon.
^ permalink raw reply
* Re: [PATCH] archive: fix segfault from too long --format parameter
From: René Scharfe @ 2010-02-07 10:03 UTC (permalink / raw)
To: Jonathan Nieder; +Cc: Junio C Hamano, Dmitry Potapov, git
In-Reply-To: <20100207070811.GA26338@progeny.tock>
Am 07.02.2010 08:10, schrieb Jonathan Nieder:
> ‘git archive --format=<string of 25 characters or more>’ overflows a
> local buffer, producing a segfault here.
>
> The context: in commit 0f4b377 (git-archive: infer output format from
> filename when unspecified, 2009-09-14), the cmd_archive wrapper
> learned to produce a format argument for the local or remote archive
> machinery in a local buffer, but that code was missing a bounds check.
>
> So add the missing check. As a belt-and-suspenders measure, also use
> snprintf to make sure the copy afterwards does not overflow.
>
> Cc: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
> Cc: Dmitry Potapov <dpotapov@gmail.com>
> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
> ---
> I noticed this while reading over the archive code. Thoughts?
>
> builtin-archive.c | 4 +++-
> t/t5000-tar-tree.sh | 6 ++++++
> 2 files changed, 9 insertions(+), 1 deletions(-)
>
> diff --git a/builtin-archive.c b/builtin-archive.c
> index 3fb4136..94db00d 100644
> --- a/builtin-archive.c
> +++ b/builtin-archive.c
> @@ -107,7 +107,9 @@ int cmd_archive(int argc, const char **argv, const char *prefix)
> }
>
> if (format) {
> - sprintf(fmt_opt, "--format=%s", format);
> + if (strlen(format) > sizeof(fmt_opt) - sizeof("--format="))
> + die("git archive: format is too long: %.50s", format);
> + snprintf(fmt_opt, sizeof(fmt_opt), "--format=%s", format);
> /*
> * We have enough room in argv[] to muck it in place,
> * because either --format and/or --output must have
Thanks. I think this should go into 1.7.0. I'd use the same format
string for the error message as archive.c, i.e. "Unknown archive format
'%s'". Later I'd rather do this:
builtin-archive.c | 25 ++++++++++++-------------
1 files changed, 12 insertions(+), 13 deletions(-)
diff --git a/builtin-archive.c b/builtin-archive.c
index 3fb4136..ffe4f4a 100644
--- a/builtin-archive.c
+++ b/builtin-archive.c
@@ -70,7 +70,7 @@ static const char *format_from_name(const char *filename)
return NULL;
ext++;
if (!strcasecmp(ext, "zip"))
- return "zip";
+ return "--format=zip";
return NULL;
}
@@ -92,33 +92,32 @@ int cmd_archive(int argc, const char **argv, const char *prefix)
"retrieve the archive from remote repository <repo>"),
OPT_STRING(0, "exec", &exec, "cmd",
"path to the remote git-upload-archive command"),
- OPT_STRING(0, "format", &format, "fmt", "archive format"),
OPT_END()
};
- char fmt_opt[32];
argc = parse_options(argc, argv, prefix, local_opts, NULL,
PARSE_OPT_KEEP_ALL);
if (output) {
create_output_file(output);
- if (!format)
- format = format_from_name(output);
+ format = format_from_name(output);
}
if (format) {
- sprintf(fmt_opt, "--format=%s", format);
/*
* We have enough room in argv[] to muck it in place,
- * because either --format and/or --output must have
- * been given on the original command line if we get
- * to this point, and parse_options() must have eaten
- * it, i.e. we can add back one element to the array.
- * But argv[] may contain "--"; we should make it the
- * first option.
+ * because --output must have been given on the
+ * original command line if we get to this point, and
+ * parse_options() must have eaten it, i.e. we can add
+ * back one element to the array. We add a fake
+ * --format option at the beginning with the hint
+ * derived from our output filename. This way explicit
+ * --format options can override it, and the fake
+ * option is inserted before any "--" that might have
+ * been given.
*/
memmove(argv + 2, argv + 1, sizeof(*argv) * argc);
- argv[1] = fmt_opt;
+ argv[1] = format;
argv[++argc] = NULL;
}
^ permalink raw reply related
* Re: A generalization of git notes from blobs to trees - git metadata?
From: Jon Seymour @ 2010-02-07 10:15 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Jeff King, Junio C Hamano, Johan Herland, git
In-Reply-To: <2cfc40321002070141y36f62679id6ce72f924a635de@mail.gmail.com>
To explain a little further why the metadata concept is different to
the resource fork or alternate data stream concept.
These two concepts were based on the idea of associating metadata with
the name of the resource and preserving the metadata along with the
resource as the resource evolved.
This is not the intent with the metadata concept. Rather, the idea is
to annotate the content (whether it be a commit, tree or blob) with
other content (a tree in the general metadata case, or a blob in the
git notes case)
The use cases I have in mind relate to caching "expensive (or
impractical) to re-derive" results from an input. So, for example,
storing /man and /html trees for a given commit in a metadata commit
called "refs/metadata/doc" would be one case. Storing the foreign SCM
revision id that a git repo was pushed into would be another [ storing
it the commit message isn't an option because the commit has already
happened before the push ]. [ And granted: git notes can already be
used for this scenario ].
jon.
On Sun, Feb 7, 2010 at 8:41 PM, Jon Seymour <jon.seymour@gmail.com> wrote:
> On Sun, Feb 7, 2010 at 8:15 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>> Jon Seymour <jon.seymour@gmail.com> writes:
>>
>> [cut]
>>
>>> As I see it, the existing use of notes is a special instance of a more
>>> general metadata capability in which the metadata is constrained to be
>>> a single blob. If notes continued to be constrained in this way, there
>>> is no reason to change anything with respect to its current userspace
>>> behaviour. That said, most of the plumbing which enabled notes could
>>> be generalized to enable the arbitrary tree case [ which admittedly, I
>>> have yet to sell successfully !]
>>>
>>> In one sense, there is a sense in the merge issue doesn't exist. When
>>> the maintainer publishes a tag no-one expects to have to deal with
>>> downstream conflicting definitions of the tag. Likewise, if the
>>> maintainer were to publish the /man and /html metadata trees (per my
>>> previous example) for a release tag, anyone who received
>>> /refs/metadata/doc would expect to receive the metadata trees as
>>> published by the maintainer. Anyone who didn't wouldn't have to pull
>>> /refs/metadata/doc.
>>>
>>> I can see there are use cases where multiple parties might want to
>>> contribute metadata and I do not currently have a good solution to
>>> that problem, but that is not to say there isn't one - surely it is
>>> just a question of applying a little intellect creatively?
>>
>> Are you trying to repeat fail of Apple's / MacOS / HFS+ filesystem
>> data/resource forks, and Microsoft's Alternate Data Streams in git? :-)
>>
>
> No I am not. I don't see why a metadata proposal is any more exposed
> to subversive payloads than say, use of git merge -s ours [ a
> subversive payload could be made reachable from a commit that
> otherwise merges in favour of the legitimate source - who would know?
> ]
>
> Really, I can't see why the rationale that makes a single blob used
> for extending a commit message justified can't be used to justify
> associating a metadata tree of arbitrary complexity to an arbitrary
> sha1 object. What makes maintaining a mapping to a single blob
> acceptable but maintaining a mapping to a tree unacceptable? Is there
> really any fundamental difference?
>
> jon.
>
^ permalink raw reply
* Re: [PATCH] gitweb: Die if there are parsing errors in config file
From: J.H. @ 2010-02-07 10:53 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <20100207093744.29846.6468.stgit@localhost.localdomain>
I'd sign-off that, I've probably run into it a couple of times myself.
- John 'Warthog9' Hawley
On 02/07/2010 01:40 AM, Jakub Narebski wrote:
> Otherwise the errors can propagate, and show in damnest places, and
> you would spend your time chasing ghosts instead of debugging real
> problem (yes, it is from personal experience).
>
> This follows (parts of) advice in `perldoc -f do` documentation.
>
> Signed-off-by: Jakub Narebski <jnareb@gmail.com>
> ---
> This is fallout from my work on [split] "Gitweb output caching" series.
> Before I used `die $@ if $@;' in t/t9503/test_cache_interface.pl, tests
> failed for no discernable reason...
>
> So I think the same should be done for the gitweb config file.
>
> gitweb/gitweb.perl | 2 ++
> 1 files changed, 2 insertions(+), 0 deletions(-)
>
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 1f6978a..a5bc359 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -556,6 +556,8 @@ if (-e $GITWEB_CONFIG) {
> our $GITWEB_CONFIG_SYSTEM = $ENV{'GITWEB_CONFIG_SYSTEM'} || "++GITWEB_CONFIG_SYSTEM++";
> do $GITWEB_CONFIG_SYSTEM if -e $GITWEB_CONFIG_SYSTEM;
> }
> +# die if there are errors parsing config file
> +die $@ if $@;
>
> # Get loadavg of system, to compare against $maxload.
> # Currently it requires '/proc/loadavg' present to get loadavg;
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 1/4] gitweb: notes feature
From: Giuseppe Bilotta @ 2010-02-07 10:57 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git, Johannes Schindelin, Johan Herland, Junio C Hamano
In-Reply-To: <201002070220.36897.jnareb@gmail.com>
On Sun, Feb 7, 2010 at 2:20 AM, Jakub Narebski <jnareb@gmail.com> wrote:
> On Sat, 6 Feb 2010, Giuseppe Bilotta wrote:
>> On Sat, Feb 6, 2010 at 11:14 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>>> So it is to be single shell-glob / fnmatch (I think) compatible pattern,
>>> isn't it?
>>
>> Sort of. fnmatch doesn't do brace expansion, which is a pity IMO, but
>> that's just my personal preference.
>
> Well, fnmatch is what I think git uses for <pattern> e.g. for
> git-for-each-ref.
fnmatch is the function to use to match a single component. The
question is how to group components together in a single spec (aside
from shell globs and []); my personal choice would be brace expansion,
colon-separated was suggested by (IIRC) Junio. Of course, for gitweb
I'll go with whatever is chosen for core.
>> Colon-separated, fnmatched components is probably the easiest thing to
>> implement to have multiple refs. I'll go with whatever is chosen for
>> core.
>
> I think that having actual list of patterns in $feature{'notes'}{'default'}
> might be more clear; you would still need colon separated (or space
> separated) list of patterns in per-repo override in gitweb.notes config
> variable.
>
> So it would be
>
> $feature{'notes'}{'default'} = ['commits', '*svn*'];
> $feature{'notes'}{'override'} = 1;
>
> but
>
> [gitweb]
> notes = commits:*svn*
>
> Note that refs names cannot contain either colon ':' or space ' '
> (see git-check-ref-format).
That makes sense. Colon-separated values will probably be allowed in
$feature{'notes'} too, which will keep the code more consistent.
[...]
>>> The above fragment of code is tested that it works. You would probably
>>> need to replace dies with something less fatal...
>>
>> On the other hand, as mentioned by Junio, this approach is not
>> future-proof enough for any kind of fan-out schemes.
>
> On the third hand ;-P you propose below a trick to deal with fan-out
> schemes, assuming that they use 2-character component breaking.
>
> Also, perhaps "git notes show" should acquire --batch / --batch-check
> options, similar to git-cat-file's options of the same name?
There is little doubt that batch note processing should be available
in core, be it with git notes show --batch(-check), git ls-notes, or
both. I also agree with Johan that gitweb should use these mechanisms
as soon as they are available. The approaches we're discussing here
are basically a temporary workaround for the missing core support.
[...]
>> If we have a guarantee that the fan-outs follow a 2/[2/...] scheme,
>> the open2 approach might still be the best way to go, by just trying
>> not only namespace:xxxxx...xxx but also namespace:xx/xxxxx etc.
>> Horrible, but could still be coalesced in a single call. It mgiht also
>> be optimized to stop at the first successfull hit in a namespace.
>
> Nice trick! It seems like quite a good idea... but it would absolutely
> require using 'git cat-file --batch' rather than one git-show per try.
Oh, that's a given. git-show was just the only approach I could think
of not knowing about the batch processing git commands.
>> I'm not getting these in the repo I'm testing this. And I think this
>> is indeed the behavior of current git next
>
> Errr, I not made myself clear.
>
> I have added a note to a commit, using "git notes edit d6bbe7f". Now if
> you take a look at gitweb output for this commit (e.g. 'commit' view for
> this commit) using gitweb without your changes, you would see that it
> flattened notes at the bottom of the commit message (which I think is
> intended result by notes implementation).
>
> If you run the command that parse_commit runs, namely
>
> $ git rev-list --parents --header -z --max-count=1 \
> d6bbe7fd52058cdf0e48bec00701ae0f4861dcd3
>
> you would get (up to invisible NUL characters) the output shown above.
And that's what I don't get. git version 1.7.0.rc1.193.ge8618. If I
remember correctly, the behaviour of automatically displaying notes in
git log & friends was changed recently. So git log -1
e8618c52b5f815624251f048609744c9558d92a1 gives me the notes I put
there for testing, git rev-list --parents --header -z --max-count=1
does not.
--
Giuseppe "Oblomov" Bilotta
^ permalink raw reply
* Re: [PATCH 1/4] gitweb: notes feature
From: Jakub Narebski @ 2010-02-07 11:08 UTC (permalink / raw)
To: Johan Herland; +Cc: Giuseppe Bilotta, git, Johannes Schindelin, Junio C Hamano
In-Reply-To: <201002070248.03855.johan@herland.net>
On Sun, 7 February 2010, Johan Herland wrote:
> On Sunday 07 February 2010, Jakub Narebski wrote:
> > On Sat, 6 Feb 2010, Giuseppe Bilotta wrote:
> > > On the other hand, as mentioned by Junio, this approach is not
> > > future-proof enough for any kind of fan-out schemes.
> >
> > On the third hand ;-P you propose below a trick to deal with fan-out
> > schemes, assuming that they use 2-character component breaking.
>
> The current notes code (as it stands in 'pu') use only 2-character component
> breaking, and I don't see any other fanout mechanism being added anytime
> soon.
On one hand checking <notes-ref>:403f3b3e..., then <notes-ref>:40/3f3b3e...,
then <notes-ref>:40/3f/3b3e... etc. feels a bit kludgy...
On the other hand it would allow to support notes in gitweb even if git
binary does not have notes support (and yet notes did get somehow into
repository).
> > Also, perhaps "git notes show" should acquire --batch / --batch-check
> > options, similar to git-cat-file's options of the same name?
>
> I'd much rather have support for ^{notes} (or similar) in the rev-parse
> machinery, so that you could look up deadbeef's notes by passing
> "deadbeef^{notes}" to 'git cat-file --batch'.
+1. Good idea!
The only caveat is that if/when we support either of:
* allowing to specify multiple notes namespaces (e.g. multiple --notes-ref
or --notes option, or PATH-like GIT_NOTES_REF with colon-separated list
of multiple notes namespaces)
* allowing to have multiple notes per object ('tree' notes)
then <commit-ish>^{notes} would mean multiple objects. But it is not
much different from supported <commit-ish>^! and <commit-ish>^@ syntax.
[...]
> > > If we have a guarantee that the fan-outs follow a 2/[2/...] scheme,
> > > the open2 approach might still be the best way to go, by just trying
> > > not only namespace:xxxxx...xxx but also namespace:xx/xxxxx etc.
> > > Horrible, but could still be coalesced in a single call. It mgiht also
> > > be optimized to stop at the first successfull hit in a namespace.
> >
> > Nice trick! It seems like quite a good idea... but it would absolutely
> > require using 'git cat-file --batch' rather than one git-show per try.
>
> Still, I'd still much rather use the notes.c code itself for doing this
> since it should always be the fastest (not to mention future-proof) way of
> making lookups in the notes tree.
One of Giuseppe goals seems to be to support notes in gitweb even if used
git binary doesn't have support for notes (git-notes command, ^{notes}
extended SHA1 syntax).
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: [PATCH 1/4] gitweb: notes feature
From: Jakub Narebski @ 2010-02-07 11:11 UTC (permalink / raw)
To: Giuseppe Bilotta; +Cc: git, Johannes Schindelin, Johan Herland, Junio C Hamano
In-Reply-To: <cb7bb73a1002070257g8f1e59dvf124c8966cdf3270@mail.gmail.com>
Giuseppe Bilotta wrote:
> On Sun, Feb 7, 2010 at 2:20 AM, Jakub Narebski <jnareb@gmail.com> wrote:
> > Errr, I not made myself clear.
> >
> > I have added a note to a commit, using "git notes edit d6bbe7f". Now if
> > you take a look at gitweb output for this commit (e.g. 'commit' view for
> > this commit) using gitweb without your changes, you would see that it
> > flattened notes at the bottom of the commit message (which I think is
> > intended result by notes implementation).
> >
> > If you run the command that parse_commit runs, namely
> >
> > $ git rev-list --parents --header -z --max-count=1 \
> > d6bbe7fd52058cdf0e48bec00701ae0f4861dcd3
> >
> > you would get (up to invisible NUL characters) the output shown above.
>
> And that's what I don't get. git version 1.7.0.rc1.193.ge8618. If I
> remember correctly, the behaviour of automatically displaying notes in
> git log & friends was changed recently. So git log -1
> e8618c52b5f815624251f048609744c9558d92a1 gives me the notes I put
> there for testing, git rev-list --parents --header -z --max-count=1
> does not.
Ah, sorry. Git version 1.6.6.1 here.
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: [PATCH 1/4] gitweb: notes feature
From: Giuseppe Bilotta @ 2010-02-07 11:14 UTC (permalink / raw)
To: Johan Herland; +Cc: Jakub Narebski, git, Johannes Schindelin, Junio C Hamano
In-Reply-To: <201002070248.03855.johan@herland.net>
On Sun, Feb 7, 2010 at 2:48 AM, Johan Herland <johan@herland.net> wrote:
> On Sunday 07 February 2010, Jakub Narebski wrote:
>
>> Also, perhaps "git notes show" should acquire --batch / --batch-check
>> options, similar to git-cat-file's options of the same name?
>
> I'd much rather have support for ^{notes} (or similar) in the rev-parse
> machinery, so that you could look up deadbeef's notes by passing
> "deadbeef^{notes}" to 'git cat-file --batch'.
Maybe something like deadbeef@{notes[:namespace]}? The ability to
embed the notes namespace to use in the call is very useful to be able
to access all the notes with a single git call.
>> > If we have a guarantee that the fan-outs follow a 2/[2/...] scheme,
>> > the open2 approach might still be the best way to go, by just trying
>> > not only namespace:xxxxx...xxx but also namespace:xx/xxxxx etc.
>> > Horrible, but could still be coalesced in a single call. It mgiht also
>> > be optimized to stop at the first successfull hit in a namespace.
>>
>> Nice trick! It seems like quite a good idea... but it would absolutely
>> require using 'git cat-file --batch' rather than one git-show per try.
>
> Still, I'd still much rather use the notes.c code itself for doing this
> since it should always be the fastest (not to mention future-proof) way of
> making lookups in the notes tree.
I agree with you on this, btw. As I mentioned in the other message,
these would just be workarounds for the current lack of support of
these features in core. I'd probably try and have a go at the thing
myself, too (i.e. first implement the core functionality, and then use
it in gitweb), but I honestly don't feel confident enough to hack at
git core.
--
Giuseppe "Oblomov" Bilotta
^ permalink raw reply
* notes metadata?
From: Giuseppe Bilotta @ 2010-02-07 11:50 UTC (permalink / raw)
To: Git List; +Cc: Johan Herland, Junio C Hamano
Hello all,
ok, this may sound a little odd especially with the 'notes vs
metadata' thread going on, but I was wondering: do we store _any_ kind
of metadata _about_ the notes themselves? If I'm reading the code
correctly, we have neither author nor date information about the notes
themselves, so we don't know who added them or when. Is it too late to
suggest that this kind of metadata be added to notes? Making them
full-blown commit-style objects is probalby overengineered and wrong
under many points of view (not to mention probably incompatible with
current storage), but maybe we can set up a convention that notes
SHOULD be in pseudo-mbox format? This would mean that when a note is
created, the template starts with a 'From ' line including the user's
name & email and note creation date; when editing, the note is again
augmented with the new author name email and date. Of course the users
are then free do expunge the From lines if they don't want it (just
commenting it would be enough, of course). How does the idea sound?
--
Giuseppe "Oblomov" Bilotta
^ permalink raw reply
* Re: [RFC PATCH 00/10] gitweb: Simple file based output caching
From: Jakub Narebski @ 2010-02-07 12:35 UTC (permalink / raw)
To: J.H.; +Cc: git, John 'Warthog9' Hawley
In-Reply-To: <201002070056.31665.jnareb@gmail.com>
On Sun, 7 Feb 2010, Jakub Narebski wrote:
> There is new version of this series in gitweb/cache-kernel-v2 in my
> git/jnareb-git.git fork (clone) of git.git repository at repo.or.cz.
> Now all commits have proper description (for first series one had to
> read comment section in emails for commit description), [...]
Below there are commit messages for gitweb/cache-kernel-v2 branch after
rebase and fixups:
commit 560e2ab10d0f8457fbeca7a26814ff3e32396f7b
Author: Jakub Narebski <jnareb@gmail.com>
Date: Sun Feb 7 11:27:22 2010 +0100
gitweb: href(..., -path_info => 0|1)
If named boolean option -path_info is passed to href() subroutine, use
its value to decide whether to generate path_info URL form. If this
option is not passed, href() queries 'pathinfo' feature to check
whether to generate path_info URL (if generating path_info link is
possible at all).
href(-replay=>1, -path_info=>0) is meant to be used to generate a key
for caching gitweb output; alternate solution would be to use freeze()
from Storable (core module) on %input_params hash (or its reference),
e.g.:
$key = freeze \%input_params;
or other serialization technique.
While at it document extra options/flags to href().
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
gitweb/gitweb.perl | 7 ++++++-
1 files changed, 6 insertions(+), 1 deletions(-)
commit dd6e8dc27d5b799bd2a1aed03738195dfe3bc5e7
Author: Jakub Narebski <jnareb@gmail.com>
Date: Sun Feb 7 13:13:06 2010 +0100
gitweb/cache.pm - Very simple file based caching
This is first step towards implementing file based output (response)
caching layer that is used on such large sites as kernel.org.
This patch introduces GitwebCaching::SimpleFileCache package, which
follows Cache::Cache / CHI interface, although do not implement it
fully. The intent of following established convention is to be able
in the future to replace our simple file based cache e.g. by one using
memcached.
Like in original patch by John 'Warthog9' Hawley (J.H.) (the one this
commit intends to be incremental step to), the data is stored in the
case as-is, without adding metadata (like expiration date), and
without serialization (which means only scalar data).
To be implemented (from original patch by J.H.):
* cache expiration (based on file stats, current time and global
expiration time); currently elements in cache do not expire
* actually using this cache in gitweb, except error pages
* adaptive cache expiration, based on average system load
* optional locking interface, where only one process can update cache
(using flock)
* server-side progress indicator when waiting for filling cache,
which in turn requires separating situations (like snapshots and
other non-HTML responses) where we should not show 'please wait'
message
Possible extensions (beyond what was in original patch):
* (optionally) show information about cache utilization
* AJAX (JavaScript-based) progress indicator
* JavaScript code to update relative dates in cached output
* make cache size-aware (try to not exceed specified maximum size)
* utilize X-Sendfile header (or equivalent) to show cached data
(optional, as it makes sense only if web server supports sendfile
feature and have it enabled)
* variable expiration feature from CHI, allowing items to expire a bit
earlier than the stated expiration time to prevent cache miss
stampedes (although locking, if available, should take care of
this).
The code of GitwebCaching::SimpleFileCache package in gitweb/cache.pm
was heavily based on file-based cache in Cache::Cache package, i.e.
on Cache::FileCache, Cache::FileBackend and Cache::BaseCache, and on
file-based cache in CHI, i.e. on CHI::Driver::File and CHI::Driver
(including implementing atomic write, something that original patch
lacks).
This patch does not yet enable output caching in gitweb (it doesn't
have all required features yet); on the other hand it includes tests,
currently testing only cache Perl API.
Inspired-by-code-by: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
gitweb/cache.pm | 269 +++++++++++++++++++++++++++++++++++++++
t/t9503-gitweb-caching.sh | 32 +++++
t/t9503/test_cache_interface.pl | 84 ++++++++++++
t/test-lib.sh | 3 +
4 files changed, 388 insertions(+), 0 deletions(-)
create mode 100644 gitweb/cache.pm
create mode 100755 t/t9503-gitweb-caching.sh
create mode 100755 t/t9503/test_cache_interface.pl
commit 3914e7da792fec50fcc64c0e644d54cf4451703a
Author: Jakub Narebski <jnareb@gmail.com>
Date: Sun Feb 7 13:13:17 2010 +0100
gitweb/cache.pm - Stat-based cache expiration
Add stat-based cache expiration to file-based GitwebCache::SimpleFileCache.
Contrary to the way other caching interfaces such as Cache::Cache and CHI
do it, the time cache element expires in is _global_ value associated with
cache instance, and is not local property of cache entry. (Currently cache
entry does not store any metadata associated with entry... which means that
there is no need for serialization / marshalling / freezing and thawing.)
Default expire time is -1, which means never expire.
To check if cache entry is expired, GitwebCache::SimpleFileCache compares
difference between mtime (last modify time) of a cache file and current time
with (global) time to expire. It is done using CHI-compatible is_valid()
method.
Add some tests checking that expiring works correctly (on the level of API).
To be implemented (from original patch by J.H.):
* actually using this cache in gitweb, except error pages
* adaptive cache expiration, based on average system load
* optional locking interface, where only one process can update cache
(using flock)
* server-side progress indicator when waiting for filling cache,
which in turn requires separating situations (like snapshots and
other non-HTML responses) where we should not show 'please wait'
message
Inspired-by-code-by: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
gitweb/cache.pm | 34 ++++++++++++++++++++++++++++++++--
t/t9503/test_cache_interface.pl | 10 ++++++++++
2 files changed, 42 insertions(+), 2 deletions(-)
commit a55625cb0f2d6c08a28e774fd2ddb4e5347a24b3
Author: Jakub Narebski <jnareb@gmail.com>
Date: Sun Feb 7 13:13:27 2010 +0100
gitweb: Use Cache::Cache compatible (get, set) output caching
This commit actually adds output caching to gitweb, as we have now
minimal features required for it in GitwebCache::SimpleFileCache
(a 'dumb' but fast file-based cache engine). To enable cache you need
at least set $caching_enabled to true in gitweb config, and copy cache.pm
from gitweb/ alongside gitweb.cgi - this is described in more detail
in the new "Gitweb caching" section in gitweb/README
Currently cache support related subroutines in cache.pm (which are
outside GitwebCache::SimpleFileCache package) are not well separated
from gitweb script itself; cache.pm lacks encapsulation. cache.pm
assumes that there are href() subroutine and %actions variable, and
that there exist $actions{$action} (where $action is parameter passed
to cache_fetch), and it is a code reference (see also comments in
t/t9503/test_cache_interface.pl). This is remaining artifact from the
original patch by J.H. (which also had cache_fetch() subroutine).
Gitweb itself uses directly only cache_fetch, to get page from cache
or to generate page and save it to cache, and cache_stop, to be used
in die_error subroutine, as currently error pages are not cached.
The cache_fetch subroutine captures output (from STDOUT only, as
STDERR is usually logged) using either ->push_layer()/->pop_layer()
from PerlIO::Util submodule (if it is available), or by setting and
restoring *STDOUT. Note that only the former could be tested reliably
to be reliable in t9503 test!
Enabling caching causes the following additional changes to gitweb
output:
* Disables content-type negotiation (choosing between 'text/html'
mimetype and 'application/xhtml+xml') when caching, as there is no
content-type negotiation done when retrieving page from cache.
Use 'text/html' mimetype that can be used by all browsers.
* Disable timing info (how much time it took to generate original
page, and how many git commands it took), and in its place show when
page was originally generated (in GMT / UTC timezone).
Add basic tests of caching support to t9500-gitweb-standalone-no-errors
test: set $caching_enabled to true and check for errors for first time
run (generating cache) and second time run (retrieving from cache) for a
single view - summary view for a project.
If PerlIO::Util is available (see comments), test that cache_fetch
behaves correctly, namely that it saves and restores action output in
cache, and that it prints generated output or cached output.
To be implemented (from original patch by J.H.):
* adaptive cache expiration, based on average system load
* optional locking interface, where only one process can update cache
(using flock)
* server-side progress indicator when waiting for filling cache,
which in turn requires separating situations (like snapshots and
other non-HTML responses) where we should not show 'please wait'
message
Inspired-by-code-by: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
gitweb/README | 70 ++++++++++++++++++++++
gitweb/cache.pm | 78 ++++++++++++++++++++++++
gitweb/gitweb.perl | 102 ++++++++++++++++++++++++++++----
t/gitweb-lib.sh | 2 +
t/t9500-gitweb-standalone-no-errors.sh | 19 ++++++
t/t9503/test_cache_interface.pl | 93 +++++++++++++++++++++++++++++
6 files changed, 352 insertions(+), 12 deletions(-)
commit 3e471ebd31e881ce1439f23075378c2ec6b95e4d
Author: Jakub Narebski <jnareb@gmail.com>
Date: Sun Feb 7 13:13:31 2010 +0100
gitweb/cache.pm - Adaptive cache expiration time
Add to GitwebCache::SimpleFileCache support for adaptive lifetime
(cache expiration) control. Cache lifetime can be increased or
decreased by any factor, e.g. load average, through the definition
of the 'check_load' callback.
Note that using ->set_expires_in, or unsetting 'check_load' via
->set_check_load(undef) turns off adaptive caching.
Make gitweb automatically adjust cache lifetime by load, using
get_loadavg() function. Define and describe default parameters for
dynamic (adaptive) cache expiration time control.
There are some very basic tests of dynamic expiration time in t9503,
namely checking if dynamic expire time is within given upper and lower
bounds.
To be implemented (from original patch by J.H.):
* optional locking interface, where only one process can update cache
(using flock)
* server-side progress indicator when waiting for filling cache,
which in turn requires separating situations (like snapshots and
other non-HTML responses) where we should not show 'please wait'
message
Inspired-by-code-by: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
gitweb/cache.pm | 55 +++++++++++++++++++++++++++++++++++---
gitweb/gitweb.perl | 27 +++++++++++++++++-
t/t9503/test_cache_interface.pl | 22 +++++++++++++++
3 files changed, 97 insertions(+), 7 deletions(-)
commit 984390f99c33d82cd4ddbfa6e00c721d9e74cddb
Author: Jakub Narebski <jnareb@gmail.com>
Date: Sun Feb 7 13:13:52 2010 +0100
gitweb: Use CHI compatible (compute method) caching
If $cache provides CHI compatible ->compute($key, $code) method, use it
instead of Cache::Cache compatible ->get($key) and ->set($key, $data).
While at it, refactor regenerating cache into cache_calculate subroutine.
GitwebCache::SimpleFileCache provides 'compute' method, which currently
simply use 'get' and 'set' methods in proscribed manner. Nevertheless
'compute' method can be more flexible in choosing when to refresh cache,
and which process is to refresh/(re)generate cache entry. This method
would use (advisory) locking to prevent 'cache miss stampede' (aka
'stampeding herd') problem in the next commit.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
gitweb/cache.pm | 39 ++++++++++++++++++++++++++++++++++++---
1 files changed, 36 insertions(+), 3 deletions(-)
commit 7d0109e4379f5187364edf7c25cdbc5247609f64
Author: Jakub Narebski <jnareb@gmail.com>
Date: Sun Feb 7 13:18:14 2010 +0100
gitweb/cache.pm - Use locking to avoid 'cache miss stampede' problem
In the ->compute($key, $code) method from GitwebCache::SimpleFileCache,
use locking (via flock) to ensure that only one process would generate
data to update/fill-in cache; the rest would wait for the cache to
be (re)generated and would read data from cache.
Currently this feature can not be disabled (via %cache_options).
A test in t9503 shows that in the case where there are two clients
trying to simultaneously access non-existent or stale cache entry,
(and generating data takes (artifically) a bit of time), if they are
using ->compute method the data is (re)generated once, as opposed to
if those clients are just using ->get/->set methods.
To be implemented (from original patch by J.H.):
* background building, and showing stale cache
* server-side progress indicator when waiting for filling cache,
which in turn requires separating situations (like snapshots and
other non-HTML responses) where we should not show 'please wait'
message
Inspired-by-code-by: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
gitweb/cache.pm | 29 ++++++++++++++++-
t/t9503/test_cache_interface.pl | 65 +++++++++++++++++++++++++++++++++++++++
2 files changed, 92 insertions(+), 2 deletions(-)
commit e7985f69eb9000860b155939d5fd7040e30f682f
Author: Jakub Narebski <jnareb@gmail.com>
Date: Sun Feb 7 13:19:21 2010 +0100
gitweb/cache.pm - Serve stale data when waiting for filling cache
When process fails to acquire exclusive (writers) lock, then instead
of waiting for the other process to (re)generate and fill cache, serve
stale (expired) data from cache. This is of course possible only if
there is some stale data in cache for given key.
This feature of GitwebCache::SimpleFileCache is used only for an
->update($key, $code) method. It is controlled by 'max_lifetime'
cache parameter; you can set it to -1 to always serve stale data
if it exists, and you can set it to 0 (or any value smaller than
'expires_min') to turn this feature off.
This feature, as it is implemented currently, makes ->update() method a
bit assymetric with respect to process that acquired writers lock and
those processes that didn't, which can be seen in the new test in t9503.
The process that is to regenerate (refresh) data in cache must wait for
the data to be generated in full before showing anything to client, while
the other processes show stale (expired) data immediately. In order to
remove or reduce this assymetry gitweb would need to employ one of the two
alternate solutions. Either data should be (re)generated in background,
so that process that acquired writers lock would generate data in
background while serving stale data, or alternatively the process that
generates data should pass output to original STDOUT while capturing it
("tee" otput).
When developing this feature, ->is_valid() method acquired additional
extra optional parameter, where one cap pass expire time instead of using
cache-wode global expire time.
To be implemented (from original patch by J.H.):
* background building,
* server-side progress indicator when waiting for filling cache,
which in turn requires separating situations (like snapshots and
other non-HTML responses) where we should not show 'please wait'
message
Inspired-by-code-by: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
gitweb/cache.pm | 23 ++++++++++----
gitweb/gitweb.perl | 8 +++++
t/t9503/test_cache_interface.pl | 63 +++++++++++++++++++++++++++++++++++++-
3 files changed, 86 insertions(+), 8 deletions(-)
commit 19911970b8a811a6382e39a10b071bff1dd4bd70
Author: Jakub Narebski <jnareb@gmail.com>
Date: Sun Feb 7 13:20:46 2010 +0100
gitweb/cache.pm - Regenerate (refresh) cache in background
This commit removes assymetry in serving stale data (if it exists)
when regenerating cache in GitwebCache::SimpleFileCache. The process
that acquired exclusive (writers) lock, and is therefore selected to
be the one that (re)generates data to fill the cache, can now generate
data in background, while serving stale data.
This feature can be enabled or disabled on demand via 'background_cache'
cache parameter. It is turned on by default.
To be implemented (from original patch by J.H.):
* server-side progress indicator when waiting for filling cache,
which in turn requires separating situations (like snapshots and
other non-HTML responses) where we should not show 'please wait'
message
Inspired-by-code-by: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
gitweb/cache.pm | 36 +++++++++++++++++++++++++++++-------
gitweb/gitweb.perl | 9 +++++++++
t/t9503/test_cache_interface.pl | 14 ++++++++------
3 files changed, 46 insertions(+), 13 deletions(-)
commit ce97bb5bc1660f6d5c9b9be68c556ac94097978c
Author: Jakub Narebski <jnareb@gmail.com>
Date: Sun Feb 7 13:21:10 2010 +0100
gitweb: Show appropriate "Generating..." page when regenerating cache
When there exist stale/expired (but not too stale) version of
(re)generated page in cache, gitweb returns stale version (and updates
cache in background, assuming 'background_cache' is set to true value).
When there is no stale version suitable to serve the client, currently
we have to wait for the data to be generated in full before showing it.
Add to GitwebCache::SimpleFileCache, via 'generating_info' callback,
the ability to show user some activity indicator / progress bar, to
show that we are working on generating data.
Gitweb itself uses "Generating..." page as activity indicator, which
redirects (via <meta http-equiv="Refresh" ...>) to refreshed version
of the page after the cache is filled (via trick of not closing page
and therefore not closing connection till data is available in cache,
checked by getting shared/readers lock on lockfile for cache entry).
The git_generating_data_html() subroutine, which is used by gitweb
to implement this feature, is highly configurable: you can choose
initial delay, frequency of writing some data so that connection
won't get closed, and maximum time to wait for data in "Generating..."
page (see %generating_options hash).
Currently git_generating_data_html() contains hardcoded "whitelist" of
actions for which such HTML "Generating..." page makes sense.
This implements final feature from the original gitweb output caching
patch by J.H.
Inspired-by-code-by: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
gitweb/cache.pm | 23 +++++-
gitweb/gitweb.perl | 154 ++++++++++++++++++++++++++++++++++++++-
t/t9503/test_cache_interface.pl | 45 +++++++++++
3 files changed, 216 insertions(+), 6 deletions(-)
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: [PATCH 1/4] gitweb: notes feature
From: Jakub Narebski @ 2010-02-07 12:41 UTC (permalink / raw)
To: Giuseppe Bilotta; +Cc: Johan Herland, git, Johannes Schindelin, Junio C Hamano
In-Reply-To: <cb7bb73a1002070314t4f382d31k91423eac00a68715@mail.gmail.com>
On Sun, 7 Feb 2010, Giuseppe Bilotta wrote:
> On Sun, Feb 7, 2010 at 2:48 AM, Johan Herland <johan@herland.net> wrote:
> > On Sunday 07 February 2010, Jakub Narebski wrote:
> >
> > > Also, perhaps "git notes show" should acquire --batch / --batch-check
> > > options, similar to git-cat-file's options of the same name?
> >
> > I'd much rather have support for ^{notes} (or similar) in the rev-parse
> > machinery, so that you could look up deadbeef's notes by passing
> > "deadbeef^{notes}" to 'git cat-file --batch'.
>
> Maybe something like deadbeef@{notes[:namespace]}? The ability to
> embed the notes namespace to use in the call is very useful to be able
> to access all the notes with a single git call.
That is just bikeshedding, but I'd rather not use '@', which currently
is used only for _reflog_ based revision specifiers: [<ref>]@{<date>},
[<ref>]@{<n>}, @{-<n>}, for notes which are not reflog based.
We can use
echo <commit>^{notes} | git --notes-ref=<namespace> cat-file --batch
or perhaps
echo <commit>^{notes:<namespace>} | git cat-file --batch
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: how to apply patch?
From: Tay Ray Chuan @ 2010-02-07 13:00 UTC (permalink / raw)
To: Matt Di Pasquale; +Cc: git
In-Reply-To: <13f0168a1002062206p8da7dd0n5f1b353c33b344f3@mail.gmail.com>
Hi,
On Sun, Feb 7, 2010 at 2:06 PM, Matt Di Pasquale
<liveloveprosper@gmail.com> wrote:
> [snip]
> Also, is there an easy way to do something like this with a branch
> instead of a remote? Remote doesn't make sense to me conceptually (and
> I already have a remote repo setup that I push to from master).
I'm already doing this with branches. It's true that I used a remote
configuration, but the remote configuration ("[remote local]") says
that we're treating the local repository as the remote repo ("url =
.") and to "fetch" the dev branch (into FETCH_HEAD).
(btw, instead of '[remote local]' it should be '[remote "local"]'.)
I do this so that you can just type "git pull" without specifying the
branch, unlike when using git-rebase.
> Maybe
> I should have 2 different branches? One for dev & one for pro. How
> would I do that?
You should.
Your history for the two branches would look like this:
a---b---c dev
\
A---B---C pro
(if you're reading this in a non-fixed width font, pro branches off at "c".)
The commits in uppercase (A, B, C) are those that are applied onto dev
- for example, changing file paths, configuration, etc.
Let's say you've made some changes onto your dev branch. To update
your production site, you would rebase your pro changes onto the dev
branch (using the config and commands in the previous message), so
that you get this history:
a---b---c---d---e dev
\
A'---B'---C' pro
(pro branches off at dev's tip, "e".)
You have to keep in mind several issues. For example, if you're
updating file paths, when new files are added in the dev branch, you
need to update the pro side to modify the file paths in those files. I
was also assuming your dev branch fast-forwards cleanly, or else the
rebase fumbles.
--
Cheers,
Ray Chuan
^ permalink raw reply
* Re: notes metadata?
From: Jon Seymour @ 2010-02-07 13:27 UTC (permalink / raw)
To: Git List
In-Reply-To: <cb7bb73a1002070350j750287abl43de4d936a47acef@mail.gmail.com>
This is already done refs/notes/commits (or any other notes ref) is a
normal commit and so any edits to the notes are thus tracked in the
normal way.
To see this add a note, then use:
gitk notes/commits.
Regards,
jon.
On Sun, Feb 7, 2010 at 10:50 PM, Giuseppe Bilotta
<giuseppe.bilotta@gmail.com> wrote:
> Hello all,
>
> ok, this may sound a little odd especially with the 'notes vs
> metadata' thread going on, but I was wondering: do we store _any_ kind
> of metadata _about_ the notes themselves? If I'm reading the code
> correctly, we have neither author nor date information about the notes
> themselves, so we don't know who added them or when. Is it too late to
> suggest that this kind of metadata be added to notes? Making them
> full-blown commit-style objects is probalby overengineered and wrong
> under many points of view (not to mention probably incompatible with
> current storage), but maybe we can set up a convention that notes
> SHOULD be in pseudo-mbox format? This would mean that when a note is
> created, the template starts with a 'From ' line including the user's
> name & email and note creation date; when editing, the note is again
> augmented with the new author name email and date. Of course the users
> are then free do expunge the From lines if they don't want it (just
> commenting it would be enough, of course). How does the idea sound?
>
> --
> Giuseppe "Oblomov" Bilotta
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* [PATCH] Documentation/SubmittingPatches: fix Gmail workaround advice
From: Aaron Crane @ 2010-02-07 15:14 UTC (permalink / raw)
To: git; +Cc: Sverre Rabbelier, Jacob Helwig, David Aguilar, Jay Soffian
The suggested approach to dealing with Gmail's propensity for breaking
patches doesn't seem to work. Recommend an alternative technique which
does.
Signed-off-by: Aaron Crane <git@aaroncrane.co.uk>
---
Thanks to Sverre Rabbelier, Jacob Helwig, David Aguilar, and Jay Soffian for
their suggestions; the advice here comes from them.
Documentation/SubmittingPatches | 88 +++++++++++++++++++++++++--------------
1 files changed, 56 insertions(+), 32 deletions(-)
diff --git a/Documentation/SubmittingPatches b/Documentation/SubmittingPatches
index c686f86..4d3c45f 100644
--- a/Documentation/SubmittingPatches
+++ b/Documentation/SubmittingPatches
@@ -517,35 +517,59 @@ message, complete the addressing and subject fields, and press send.
Gmail
-----
-GMail does not appear to have any way to turn off line wrapping in the web
-interface, so this will mangle any emails that you send. You can however
-use any IMAP email client to connect to the google imap server, and forward
-the emails through that. Just make sure to disable line wrapping in that
-email client. Alternatively, use "git send-email" instead.
-
-Submitting properly formatted patches via Gmail is simple now that
-IMAP support is available. First, edit your ~/.gitconfig to specify your
-account settings:
-
-[imap]
- folder = "[Gmail]/Drafts"
- host = imaps://imap.gmail.com
- user = user@gmail.com
- pass = p4ssw0rd
- port = 993
- sslverify = false
-
-You might need to instead use: folder = "[Google Mail]/Drafts" if you get an error
-that the "Folder doesn't exist".
-
-Next, ensure that your Gmail settings are correct. In "Settings" the
-"Use Unicode (UTF-8) encoding for outgoing messages" should be checked.
-
-Once your commits are ready to send to the mailing list, run the following
-command to send the patch emails to your Gmail Drafts folder.
-
- $ git format-patch -M --stdout origin/master | git imap-send
-
-Go to your Gmail account, open the Drafts folder, find the patch email, fill
-in the To: and CC: fields and send away!
-
+Gmail does not appear to have any way to turn off line wrapping in the web
+interface, so this will mangle any emails that you send. Nor can you use
+"git imap-send" to upload an email to your Drafts folder; the web interface
+will still mangle your message when you send it.
+
+The best approach is to send the email using Gmail's SMTP submission
+servers. Configuring "git send-email" to do that looks like this:
+
+ [sendemail]
+ smtpencryption = tls
+ smtpserver = smtp.gmail.com
+ smtpuser = [YOURADDRESSHERE]@gmail.com
+ smtpserverport = 587
+
+Then sending will look like this:
+
+ $ git format-patch --no-color -C -M origin/master..topic -o outgoing/
+ $ git send-email --compose outgoing/00*
+
+"git send-email" will then prompt you for your password.
+
+However, "git send-email" needs the Net::SMTP::SSL Perl module to send to
+TLS-encrypted servers. On some operating systems (like Mac OS X with
+MacPorts), it may be hard to install that module. In such cases, the
+third-party "msmtp" program might be easier to install. If so, configure
+"git send-email" like this:
+
+ [sendemail]
+ smtpserver = /opt/local/bin/msmtp
+ # adjust the path if necessary
+
+and then configure msmtp to use the Gmail servers, in your ~/.msmtprc file:
+
+ # Set default values for all following accounts
+ defaults
+ tls on
+ tls_trust_file /opt/local/share/curl/curl-ca-bundle.crt
+ # with MacPorts, install the curl-ca-bundle port for this file
+
+ # Configure a "gmail" account
+ account gmail
+ host smtp.gmail.com
+ port 587
+ from [YOURADDRESSHERE]@gmail.com
+ auth on
+ user [YOURADDRESSHERE]@gmail.com
+
+ # Set the "gmail" account as the default
+ account default : gmail
+
+If you're using either Gnome or Mac OS X, msmtp will look up your password
+from your OS keychain; otherwise, or if that fails, it prompts you for your
+password.
+
+For more details, see
+http://git.wiki.kernel.org/index.php/GitTips#Using_gmail_to_send_your_patches
--
1.6.6.1
^ permalink raw reply related
* Re: git gc / git repack not removing unused objects?
From: Jon Nelson @ 2010-02-07 17:48 UTC (permalink / raw)
Cc: git
In-Reply-To: <alpine.LFD.2.00.1002061935180.1681@xanadu.home>
On Sat, Feb 6, 2010 at 7:16 PM, Nicolas Pitre <nico@fluxnic.net> wrote:
> On Sat, 6 Feb 2010, Jon Nelson wrote:
>
>> Last night, the repo size was 153G after removing some commits and
>> objects by way of git filter-branch.
>> I'm using "du -sh" in the .git directory to determine the disk usage.
>>
>> Before: 136G
>> git repack -dAl
>> After: 153G
>
> Why are you using -A instead of -a ?
As it turns out, I've been using both -a and -A. I suspect -A is a
typo on my part.
>> Then, just to make sure of some things, I changed nothing and simply
>> re-ran "git repack -dAl".
>> After: 167G
>
> Could you run 'git count-objects -v' before and after a repack in such
> cases as well?
Yes.
>> [pack]
>> packsizelimit = 256m
>
> Why are you using this?
I didn't want my pack files to be too huge. I've bumped that up to 2G.
>> pack.packsizelimit=2M
My ~/.gitconfig normally uses 2M for easy rsyncing. In this repo I
thought the value was overridden by the project's config (which was
specifying 256m and now specifies 2048m).
Suboptimal or not, it still doesn't explain why the repo grows with each repack.
Now running:
git count-objects -v ; git repack -ad ; git count-objects -v
--
Jon
^ permalink raw reply
* Re: [PATCH] Documentation/SubmittingPatches: fix Gmail workaround advice
From: Jay Soffian @ 2010-02-07 18:01 UTC (permalink / raw)
To: Aaron Crane; +Cc: git, Sverre Rabbelier, Jacob Helwig, David Aguilar
In-Reply-To: <1265555642-40204-1-git-send-email-git@aaroncrane.co.uk>
On Sun, Feb 7, 2010 at 10:14 AM, Aaron Crane <git@aaroncrane.co.uk> wrote:
> The suggested approach to dealing with Gmail's propensity for breaking
> patches doesn't seem to work. Recommend an alternative technique which
> does.
>
> Signed-off-by: Aaron Crane <git@aaroncrane.co.uk>
> ---
Thanks for writing this up. See inline.
> +However, "git send-email" needs the Net::SMTP::SSL Perl module to send to
> +TLS-encrypted servers. On some operating systems (like Mac OS X with
> +MacPorts), it may be hard to install that module. In such cases, the
> +third-party "msmtp" program might be easier to install.
Neither is harder to install than the other, I use msmtp primarily for
historical reasons (git didn't used to support SMTP over SSL/TLS) and
because it doesn't prompt me for my password. :-)
So I'd just say:
"Note that "git send-email" needs the Net::SMTP::SSL Perl module to
send to TLS-encrypted servers. Alternately, git can make use of the
third-party "msmtp" program like so:"
Thanks,
j.
^ permalink raw reply
* Re: [PATCH] Documentation/SubmittingPatches: fix Gmail workaround advice
From: Junio C Hamano @ 2010-02-07 18:37 UTC (permalink / raw)
To: Aaron Crane
Cc: git, Sverre Rabbelier, Jacob Helwig, David Aguilar, Jay Soffian
In-Reply-To: <1265555642-40204-1-git-send-email-git@aaroncrane.co.uk>
Aaron Crane <git@aaroncrane.co.uk> writes:
> The suggested approach to dealing with Gmail's propensity for breaking
> patches doesn't seem to work. Recommend an alternative technique which
> does.
>
> Signed-off-by: Aaron Crane <git@aaroncrane.co.uk>
> ---
> Thanks to Sverre Rabbelier, Jacob Helwig, David Aguilar, and Jay Soffian for
> their suggestions; the advice here comes from them.
Do you know _why_ it does not work? For example, does it not work _at
all_ for you? Or only certain things does not reliably work (iow, perhaps
you are seeing a bug in imap-send)? We might want to find who wrote the
imap-send recommendation and ask their input.
What I am trying to get at is to see if the current imap-send suggestion
is fundamentally unworkable with gmail. Perhaps they stopped supporting
imap. Perhaps the procedure never worked. I would be very happy to apply
your patch as is if that is the case.
But if the trouble you had is something fixable, I think removal of the
original text is unfriendly to people who have only imap reachability to
gmail---the description instead needs fixing.
^ permalink raw reply
* Re: [PATCH] Add a test for a problem in "rebase --whitespace=fix"
From: Junio C Hamano @ 2010-02-07 18:38 UTC (permalink / raw)
To: Björn Gustavsson; +Cc: git
In-Reply-To: <4B6E7564.7040109@gmail.com>
Björn Gustavsson <bgustavsson@gmail.com> writes:
> The command "git rebase --whitespace=fix HEAD~<N>" is supposed to
> only clean up trailing whitespace, and the expectation is that it
> cannot fail.
I don't know if the expectation is sound to begin with, for exactly the
reason you mention below.
> Unfortunately, if one commit adds a blank line at the end of a file
> and a subsequent commit adds more non-blank lines after the blank
> line,...
First, is this a condition that we want to change the behaviour to
"succeed" later?
Imagine that the gap between abc and def block in your example is much
larger to exceed the number of pre-context lines of your second patch
(usually 3), and imagine you are the "git apply --whitespace=fix" program
you have updated to "fix" the preceived problem. You know you earlier
might have stripped some blank lines at the EOF, but there is nothing that
tells you if you had only 3 blank lines, or you had even more. How many
blank lines will you be adding back?
I think one fundamental difference between stripping of blanks at EOL and
blanks at EOF is that the former, even after applying an earlier patch
with the whitespace fix, could be fudged to an unspecified number of
whitespaces while you are applying the second one, exactly because you
will strip them out from the output of the second one anyway. But the
latter will have to _appear_ in the result, as you have demonstrated, as a
gap between abc and def blocks in your example. Simply there is not
enough information to do so.
Around 1.6.6/1.6.5.3 timeframe, we have separated blank-at-{eol,eof} out
of trailing-space to allow users to keep the traling blank lines. Perhaps
you could demonstrate what is expected to work (and not bothering with
what is not ever expected to work) by changing the test like this.
I added one "trailing whitespace at EOL" example to keep the "strip
trailing whitespace" part working, by the way.
t/t3417-rebase-whitespace-fix.sh | 11 +++++++----
1 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/t/t3417-rebase-whitespace-fix.sh b/t/t3417-rebase-whitespace-fix.sh
index 55cbce7..0e366f8 100755
--- a/t/t3417-rebase-whitespace-fix.sh
+++ b/t/t3417-rebase-whitespace-fix.sh
@@ -7,9 +7,9 @@ This test runs git rebase --whitespace=fix and make sure that it works.
. ./test-lib.sh
-# prepare initial revision of "file" with a blank line at the end
-cat >file <<EOF
-a
+# prepare initial revision of "file" with a trailing blank and a blank line at the end
+sed -e 's/Z//' >file <<\EOF
+a Z
b
c
@@ -20,6 +20,7 @@ cat >expect <<EOF
a
b
c
+
EOF
# prepare second revision of "file"
@@ -33,7 +34,9 @@ e
f
EOF
-test_expect_failure 'blanks line at end of file; extend at end of file' '
+test_expect_success 'keep blanks line at end of file' '
+ git config core.whitespace -blank-at-eof &&
+
git commit --allow-empty -m "Initial empty commit" &&
git add file && git commit -m first &&
mv second file &&
^ permalink raw reply related
* Re: [PATCH 1/4] gitweb: notes feature
From: Junio C Hamano @ 2010-02-07 18:38 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Giuseppe Bilotta, Johan Herland, git, Johannes Schindelin
In-Reply-To: <201002071341.36440.jnareb@gmail.com>
Jakub Narebski <jnareb@gmail.com> writes:
> That is just bikeshedding, but I'd rather not use '@', which currently
> is used only for _reflog_ based revision specifiers: [<ref>]@{<date>},
> [<ref>]@{<n>}, @{-<n>}, for notes which are not reflog based.
Probably a nicer way to say the same thing is to avoid "reflog based"
which sounds like you are talking about an implementation detail.
A fundamental reason to favor your "bikeshedding" (I don't think it is a
bikeshedding---it is a sound argument against using "@{...}") is that the
at-brace notation applies to a ref, not to an arbitrary commit. Applying
@{yesterday} to an arbitrary commit does not make any sense.
Notes are fundamenally metainformation about an _object_ [*1*] and are not
metainformation about refs. Since whatever magic notation to denote notes
we choose wants to be applied to an arbitrary commit, it shouldn't be the
at-brace syntax.
[Footnote]
*1* Yes, I am aware of movements to misuse notes to annotate anything
after mapping it to a random SHA-1 value, but I think that is outside the
scope of notes. Our design decision should be based on supporting the
primary use of annotating an object, and that might still keep such a use
working, in which case that would be an added bonus. But our design
shouldn't be constrained by such a secondary use.
^ permalink raw reply
* Re: A generalization of git notes from blobs to trees - git metadata?
From: Junio C Hamano @ 2010-02-07 18:48 UTC (permalink / raw)
To: Jeff King; +Cc: Johan Herland, Jon Seymour, git
In-Reply-To: <20100207050255.GA17049@coredump.intra.peff.net>
Jeff King <peff@peff.net> writes:
> ... I think you would do better
> to simply store a tree sha1 inside the note blob, and callers who were
> interested in the tree contents could then dereference it and examine as
> they saw fit. The only caveat is that you need some way of telling git
> that the referenced trees are reachable and not to be pruned.
Thanks for a good summary. To paraphrase the idea, for the "pre-built
binaries" use case, I could update the dodoc.sh script (in 'todo'---that
is what autobuilds the html and man documentation and updates the
corresponding branches at k.org when I push things out to the master
branch) to add a note to the commit from 'master' the docs are generated
from, and the note would say which commits on html and man branches
correspond to that commit. That way, the referenced "trees" are of course
protected because they are reachable from html/man refs.
Right?
^ permalink raw reply
* Re: notes TODOs
From: Jakub Narebski @ 2010-02-07 19:05 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Giuseppe Bilotta, Johan Herland, git, Johannes Schindelin
In-Reply-To: <7vr5oy2qh3.fsf@alter.siamese.dyndns.org>
On Sat, 6 Feb 2010, Junio C Hamano wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
>
> > 'git notes' move [-f] <oldobject> <newobject>
>
> I suspect "copy" to keep the old one than "move" would be a lot more
> sensible, especially when you are talking about people (like me) who amend
> often. They cannot get it right in their first try by definition ;-), and
> their very original edition is sometimes easier to start from than their
> second edition, when they are trying to come up with the final edition of
> the commit. Using "move" to lose the notes from the old object will make
> it harder to go back to the original and start amending from there.
True. It would be better then to have "git notes copy ...", and perhaps
even a configuration option to have --amend copy notes to new version
automatically.
You can always get note before move with
GIT_NOTES_REF=refs/notes/commit^ <git command>
(assuming that you use default notes ref). But that would work for
"git show", but not for "git log --walk-reflogs".
P.S. I have tried to use 'git notes edit v1.6.6.1', and while it annotates
a tag object, the message in editor make it look like you are annotating
the commit it points to.
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: A generalization of git notes from blobs to trees - git metadata?
From: Jeff King @ 2010-02-07 19:18 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johan Herland, Jon Seymour, git
In-Reply-To: <7vsk9cdgpx.fsf@alter.siamese.dyndns.org>
On Sun, Feb 07, 2010 at 10:48:58AM -0800, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > ... I think you would do better
> > to simply store a tree sha1 inside the note blob, and callers who were
> > interested in the tree contents could then dereference it and examine as
> > they saw fit. The only caveat is that you need some way of telling git
> > that the referenced trees are reachable and not to be pruned.
>
> Thanks for a good summary. To paraphrase the idea, for the "pre-built
> binaries" use case, I could update the dodoc.sh script (in 'todo'---that
> is what autobuilds the html and man documentation and updates the
> corresponding branches at k.org when I push things out to the master
> branch) to add a note to the commit from 'master' the docs are generated
> from, and the note would say which commits on html and man branches
> correspond to that commit. That way, the referenced "trees" are of course
> protected because they are reachable from html/man refs.
>
> Right?
Yeah, I think that would work fine. I guess there are cases, though,
where somebody might not be keeping a linear history of noted trees in a
separate ref (the way you keep html/man refs). In which case they would
have to deal with the reachability problem separately. I can't think of
an example off the top of my head, though.
-Peff
^ permalink raw reply
* Re: A generalization of git notes from blobs to trees - git metadata?
From: Jeff King @ 2010-02-07 19:33 UTC (permalink / raw)
To: Jon Seymour; +Cc: Junio C Hamano, Johan Herland, git
In-Reply-To: <2cfc40321002062136q64f832aesd979c9cb22f3612@mail.gmail.com>
On Sun, Feb 07, 2010 at 04:36:59PM +1100, Jon Seymour wrote:
> As I see it, the existing use of notes is a special instance of a more
> general metadata capability in which the metadata is constrained to be
> a single blob. If notes continued to be constrained in this way, there
> is no reason to change anything with respect to its current userspace
> behaviour. That said, most of the plumbing which enabled notes could
> be generalized to enable the arbitrary tree case [ which admittedly, I
> have yet to sell successfully !]
I do agree that storing trees is a natural generalization of the current
notes implementation. Callers have to be made aware that they may see
trees, of course, but you could probably "demote" trees into their
representative sha1s for callers who were interested only in a blob
form.
But what I am concerned with is that generalizing may violate some
assumptions made about how notes work. Notes trees can re-balance
themselves to some degree, I thought (though I am pretty out of the loop
on current notes developments). So during merges we need to normalize
tree representations (though we probably already need to do that for the
blob case). We would also need to do some magic with rename detection
during merges. You would probably want rename detection _within_ a tree
stored as a note for a particular commit, but not between notes stored
for different commits.
Or perhaps you would not even want to do a tree-merge between notes at
all, and would rather see a conflict if two people noted two different
trees. This would make sense to me if you were doing something like
noting a build setup. If I note that commit X builds with a tree
pointing to version Y of the build tools, and you note that it builds
with version Z of the build tools, what should happen when we merge our
notes? I can imagine wanting a conflict, and resolving it to Y or Z
(perhaps whichever is more desirable). I can also see resolving it to Y
_and_ Z (iow, treating it like a list). But doing a merge on the two
trees of build tools (which are presumably somewhat immutable) is
probably not helpful.
Which to me argues in favor of adding the extra level of indirection.
The note should store the tree sha1, and those who want to treat it as a
tree can do so. Rename and merge issues just go away, as they operate on
the tree sha1 and not on the tree itself. And of course the
representation is just an implementation detail; you could still make a
"git metadata" wrapper to transparently store trees from the user's
perspective.
The only complication is that git doesn't know to follow those sha1s for
reachability analysis. In some cases that won't matter (like Junio's
html/man example), but I suspect in some it will. Perhaps there is some
way to flag the note entry as "this stores a sha1 that should be
followed by fsck, but not otherwise dereferenced".
I dunno. That is all just thinking out loud. It would help if we had
some really detailed concrete examples of notes being used in practice.
-Peff
^ permalink raw reply
* Re: [PATCH] Documentation/SubmittingPatches: fix Gmail workaround advice
From: Aaron Crane @ 2010-02-07 20:03 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Sverre Rabbelier, Jacob Helwig, David Aguilar, Jay Soffian
In-Reply-To: <7v8wb4gaef.fsf@alter.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> wrote:
> Aaron Crane <git@aaroncrane.co.uk> writes:
>> The suggested approach to dealing with Gmail's propensity for breaking
>> patches doesn't seem to work. Recommend an alternative technique which
>> does.
>
> Do you know _why_ it does not work? For example, does it not work _at
> all_ for you? Or only certain things does not reliably work (iow, perhaps
> you are seeing a bug in imap-send)?
What happens for me is that `git imap-send` successfully puts the
message into the gmail Drafts folder, but when it's sent it's been
line-wrapped (and any line-end spaces are deleted, which breaks diffs
containing empty context lines).
I've verified that it's not `git imap-send` at fault, by tweaking it
to tee all socket-written data to a local file; all whitespace in the
mail prepared locally is sent to Gmail unchanged. So Gmail is
mangling the message when it gets uploaded (or possibly when it's
opened in the web interface, or when it's sent).
I've tried this both with a vanilla Gmail account, and with Google
Apps For Your Domain Standard Edition, and I get the same behaviour.
> What I am trying to get at is to see if the current imap-send suggestion
> is fundamentally unworkable with gmail. Perhaps they stopped supporting
> imap. Perhaps the procedure never worked.
As far as I can determine, the current imap-send suggestion is
fundamentally unworkable with gmail at the moment. I don't know
whether it ever used to work, though I've been assuming that it did.
--
Aaron Crane ** http://aaroncrane.co.uk/
^ permalink raw reply
* Re: [PATCH 1/4] gitweb: notes feature
From: Giuseppe Bilotta @ 2010-02-07 20:11 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jakub Narebski, Johan Herland, git, Johannes Schindelin
In-Reply-To: <7vock0evs8.fsf@alter.siamese.dyndns.org>
On Sun, Feb 7, 2010 at 7:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> Notes are fundamenally metainformation about an _object_ [*1*] and are not
> metainformation about refs. Since whatever magic notation to denote notes
> we choose wants to be applied to an arbitrary commit, it shouldn't be the
> at-brace syntax.
Makes sense. ^{note[:namespace]} is ok for me too btw, although maybe
it looks a little off-base when compared with the tag indicator ^{}
which works, in a sense, in the opposite direction.
> [Footnote]
>
> *1* Yes, I am aware of movements to misuse notes to annotate anything
> after mapping it to a random SHA-1 value, but I think that is outside the
> scope of notes. Our design decision should be based on supporting the
> primary use of annotating an object, and that might still keep such a use
> working, in which case that would be an added bonus. But our design
> shouldn't be constrained by such a secondary use.
BTW, I still think that notes should be attachable to named refs (not
SHA-1, thus) too.
--
Giuseppe "Oblomov" Bilotta
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox