* Re: [PATCH 4/4] config: allow including config from repository blobs
From: Junio C Hamano @ 2012-01-27 5:30 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20120127004902.GA15257@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
> No, I meant it explicitly to be about this single user hating it. Note
> how the resulting commits are never pushed. It is purely a local
> override, but with the added bonus that history is tracked so you can
> merge in further changes from upstream.
> ...
> It also does allow "[include]ref = origin/meta:gitconfig" if you want to
> live dangerously. I consider that a feature, because it lets the user
> make the security tradeoff they deem appropriate.
While I do not think origin/meta:config is a sensible default, I actually
do think that:
[include]
ref = meta:gitconfig
[branch "meta"]
remote = origin
merge = refs/heads/meta
makes some sense. The earlier example with the in-tree dev_tools/config in
the same line of history as the usual source material to keep track of
private changes ("this single user hating it") was not realistic as it
forbids the user from sharing the rest of the source once she decides to
fork the config preference.
^ permalink raw reply
* Re: [PATCH 1/4] config: add include directive
From: Junio C Hamano @ 2012-01-27 5:23 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20120126225140.GB12855@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
> And then because of 1a and 2a, most programs should Just Work without
> any changes, but because of 1b and 2b, any special uses will have to
> decide manually whether they would want to allow includes.
>
> Does that make sense?
In short, the "git config" interface defaults to "--no-includes" when
reading from an explicit file with "-f" and "--includes" otherwise, which
sounds like a 100% sensible default to me.
> I had a similar thought while writing it, but hoped the sentence after
> (that you snipped) would make it clear.
I think the whole paragraph makes it reasonably clear; I was merely being
pedantic to see if you or others can come up with a clear and simple way
to rephrase it that can also satisfy such pedantry.
> How about:
>
> The included file is processed immediately, before any other
> directives from the surrounding file.
>
> What I wanted to make clear there is the ordering, which sometimes
> matters.
Hmm, I think the original is probably easier to read.
> The one use I think is to bundle a bunch of related config options, and
> then turn them on selectively.
> ...
> but with this patch, you can do:
>
> cat >~/.gitconfig.foo <<-\EOF
> [foo]
> one = 1
> two = 2
> EOF
> git -c include.path=$HOME/.gitconfig.foo blah
That is quite a sensible use case actually.
Thanks.
^ permalink raw reply
* Re: [PATCH 1/4] config: add include directive
From: Michael Haggerty @ 2012-01-27 5:07 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20120126073752.GA30474@sigill.intra.peff.net>
On 01/26/2012 08:37 AM, Jeff King wrote:
> [...]
> This patch introduces an include directive for config files.
> It looks like:
>
> [include]
> path = /path/to/file
I like it.
> diff --git a/cache.h b/cache.h
> index 10afd71..21bbb0a 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -1138,6 +1138,12 @@ extern const char *get_commit_output_encoding(void);
>
> extern int git_config_parse_parameter(const char *, config_fn_t fn, void *data);
>
> +struct git_config_include_data {
> + config_fn_t fn;
> + void *data;
> +};
> +int git_config_include(const char *name, const char *value, void *vdata);
> +
> extern const char *config_exclusive_filename;
>
> #define MAX_GITNAME (1000)
How about a short comment or two?
> diff --git a/config.c b/config.c
> index 40f9c6d..a6966c1 100644
> --- a/config.c
> +++ b/config.c
> [...]
> +int git_config_include(const char *name, const char *value, void *vdata)
> +{
> + const struct git_config_include_data *data = vdata;
> + const char *type;
> + int ret;
> +
> + /*
> + * Pass along all values, including "include" directives; this makes it
> + * possible to query information on the includes themselves.
> + */
> + ret = data->fn(name, value, data->data);
> + if (ret < 0)
> + return ret;
> +
> + if (prefixcmp(name, "include."))
> + return ret;
> + type = strrchr(name, '.') + 1;
> +
> + if (!strcmp(type, "path"))
> + ret = handle_path_include(value, vdata);
> +
> + return ret;
> +}
> +
Doesn't this code accept all keys of the form "include\.(.*\.)?path"
(e.g., "include.foo.path")? If that is your intention, then the
documentation should be fixed. If not, then a single strcmp(name,
"include.path") would seem sufficient.
> int git_config_early(config_fn_t fn, void *data, const char *repo_config)
> {
> int ret = 0, found = 0;
> const char *home = NULL;
> + struct git_config_include_data inc;
> +
> + inc.fn = fn;
> + inc.data = data;
> + fn = git_config_include;
> + data = &inc;
>
> /* Setting $GIT_CONFIG makes git read _only_ the given config file. */
> if (config_exclusive_filename)
The comment just after your addition should be adjusted, since now "the
given config file and any files that it includes" are read.
Michael
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
^ permalink raw reply
* Re: How to migrate a complex directory structure from SVN to GIT?
From: David Barr @ 2012-01-27 4:46 UTC (permalink / raw)
To: Jehan Bing; +Cc: git, Jonathan Nieder, Dmitry Ivankov, Ramkumar Ramachandra
In-Reply-To: <4F1713D4.9000602@orb.com>
> If however you have a more complex layout, you can use "git svn init", then
> edit .git/config to suit your needs, then run "git svn fetch".
> And by "suit your needs", I mean you can add multiple "fetch=..." lines.
> In my case, I ended up having one "fetch=..." for each trunk, branch and
> tag.
> It was not efficient, it took 2 weeks to convert <30k revisions, ~200
> branches/project, ~80 projects, but it works well enough for me.
>
> Jehan
This is why we need to breath life into the git-remote-svn effort.
We should be able to handle imports of that size in minutes.
(Cost of dumping from SVN aside.)
--
David Barr
^ permalink raw reply
* Re: Git svn migration does not work because fatal git checkout updating paths is incompatible with switching branches
From: David Barr @ 2012-01-27 4:34 UTC (permalink / raw)
To: Carlos Martín Nieto; +Cc: Christine Bauers, git
In-Reply-To: <1327518563.31804.82.camel@centaur.lab.cmartin.tk>
Hi,
A lot of work has occurred on the version of svn-fe within the git tree.
Jonathan Nieder is the nominal maintainer of that effort.
He has just requested that the most stable set of changes be merged.
--
David Barr
From Jonathan's pull request:
Junio, please pull
git://repo.or.cz/git/jrn.git svn-fe
to get the following changes since commit
ec014eac0e9e6f30cbbca616090fa2ecf74797e7:
Git 1.7.5 (2011-04-23 23:36:32 -0700)
up to c5bcbcdcfa1e2a1977497cb3a342c0365c8d78d6:
vcs-svn: reset first_commit_done in fast_export_init (2011-06-23
10:04:36 -0500)
On Thu, Jan 26, 2012 at 6:09 AM, Carlos Martín Nieto <cmn@elego.de> wrote:
> On Wed, 2012-01-25 at 19:04 +0100, Christine Bauers wrote:
>> Hi there,
>>
>> I´m trying to migrate a repository from svn to git which branches and
>> tags with the following migration script:
>>
>> git svn clone --no-metadata --stdlayout --A ../users.txt
>> svn://host/svn/project/subproject subproject
>>
>> cd subproject
>> git config svn.authorsfile ../../users.txt
>> git svn fetch
>>
>> git checkout -b branch1 remotes/branch1
>> git checkout -b branch2 remotes/branch2
>> git checkout -b branch3 remotes/branch3
>>
>> git checkout -b src_v1 remotes/tags/src
>> git checkout master
>> git tag src src_v1
>> git branch -D src_v1
>>
>> git checkout -b WebContent_v1 remotes/tags/WebContent
>> git checkout master
>> git tag WebContent WebContent_v1
>> git branch -D WebContent_v1
>>
>> and get the follwoing errors:
>>
>> W: Ignoring error from SVN, path probably does not exist: (160013):
>> Filesystem has no item: Datei nicht gefunden: Revision 8966, Pfad
>> »subproject«
>> W: Do not be alarmed at the above message git-svn is just searching
>> aggressively for old history.
>> This may take a while on large repositories
>> fatal: git checkout: updating paths is incompatible with switching branches.
>> Did you intend to checkout 'remotes/branch1' which can not be resolved
>> as commit?
>> fatal: git checkout: updating paths is incompatible with switching branches.
>> Did you intend to checkout 'remotes/branch2 which can not be resolved as
>> commit?
>> fatal: git checkout: updating paths is incompatible with switching branches.
>> Did you intend to checkout 'remotes/branch3' which can not be resolved
>> as commit?
>> fatal: git checkout: updating paths is incompatible with switching branches.
>> Did you intend to checkout 'remotes/tags/src' which can not be resolved
>> as commit?
>> error: pathspec 'master' did not match any file(s) known to git.
>> fatal: Failed to resolve 'src_v1' as a valid ref.
>> error: branch 'src_v1' not found.
>> fatal: git checkout: updating paths is incompatible with switching branches.
>> Did you intend to checkout 'remotes/tags/WebContent' which can not be
>> resolved as commit?
>> error: pathspec 'master' did not match any file(s) known to git.
>> fatal: Failed to resolve 'WebContent_v1' as a valid ref.
>> error: branch 'WebContent_v1' not found.
>>
>> How do I solve this problem?
>
> First try to figure out where the problem happens. It could be that
> git-svn isn't recognising the branches properly, or that the layout
> isn't what it expects or any number of things.
>
> What layout does the repo have? Does it correspond to what git-svn is
> expecting? All those error messages come from the fact that you're
> telling git some starting points that it can't find. Make sure those
> exist and they have the name you're giving. What does `git branch -a`
> say? You're presumably not giving us the real names, so we can't tell if
> there are problems there.
>
> If you're looking to migrate completely, something like
> svn-dump-fast-export ( https://github.com/barrbrain/svn-dump-fast-export
> ) might get you there better.
>
> cmn
^ permalink raw reply
* Re: [PATCH 4/4] config: allow including config from repository blobs
From: Nguyen Thai Ngoc Duy @ 2012-01-27 4:01 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20120126074208.GD30474@sigill.intra.peff.net>
On Thu, Jan 26, 2012 at 2:42 PM, Jeff King <peff@peff.net> wrote:
> +Security Considerations
> +~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Because git configuration may cause git to execute arbitrary shell
> +commands, it is important to verify any configuration you receive over
> +the network. In particular, it is not a good idea to point `include.ref`
> +directly at a remote tracking branch like `origin/master:shared-config`.
> +After a fetch, you have no way of inspecting the shared-config you have
> +just received without running git (and thus respecting the downloaded
> +config). Instead, you can create a local tag representing the last
> +verified version of the config, and only update the tag after inspecting
> +any new content.
It may be a good idea to tell users the ref include.ref points to has
been updated at the end of git-fetch. Showing a diff is even better.
--
Duy
^ permalink raw reply
* Re: [PATCH 4/4] config: allow including config from repository blobs
From: Nguyen Thai Ngoc Duy @ 2012-01-27 3:47 UTC (permalink / raw)
To: Jeff King; +Cc: Johannes Sixt, git
In-Reply-To: <4F211C0C.7060400@viscovery.net>
On Thu, Jan 26, 2012 at 4:25 PM, Johannes Sixt <j.sixt@viscovery.net> wrote:
> Am 1/26/2012 8:42, schrieb Jeff King:
>> +static int handle_ref_include(const char *ref, void *data)
>> +{
>> + unsigned char sha1[20];
>> + char *buf;
>> + unsigned long size;
>> + enum object_type type;
>> + int ret;
>> +
>> + if (get_sha1(ref, sha1))
>> + return 0;
>> + buf = read_sha1_file(sha1, &type, &size);
>> + if (!buf)
>> + return error("unable to read include ref '%s'", ref);
>> + if (type != OBJ_BLOB)
>> + return error("include ref '%s' is not a blob", ref);
>> +
>> + ret = git_config_from_buffer(git_config_include, data, ref, buf, size);
>> + free(buf);
>> + return ret;
>> +}
>
> What happens if a ref cannot be resolved, for example due to repository
> corruption? Does git just emit an error and then carries on, or does it
> always die? Can I run at least git-fsck in such a case?
Moreover, if I specify sha-1 in the config (it's discouraged but not
forbidden from the code), can git-prune remove the blob?
--
Duy
^ permalink raw reply
* Dynamic diff [Was: Re: [PATCH] merge: Make merge strategy message follow the diffstat]
From: Arnaud Lacombe @ 2012-01-27 3:24 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Git Mailing List
Hi,
On Fri, Jan 13, 2012 at 2:49 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Fri, Jan 13, 2012 at 11:27 AM, Nguyen Thai Ngoc Duy
> <pclouds@gmail.com> wrote:
>>
>> Still, diffstat from a fetch/pull is sometimes too verbose. It'd be
>> better if we have something that fit in one screen (dirstat or maybe
>> just a first few lines from diffstat then ellipsis) then refer users
>> to "git diff --stat HEAD@{1}" for more detail stat.
>
> Yeah, I've wanted that. Show the beginning, the end, and the summary
> line of the diffstat would be lovely.
>
> It would be lovely in "git commit" too. Not just
>
> Modified: filename
>
> but a diffstat that shows now many lines.
>
> And what I've *really* wanted is to actually see the diff itself if it
> is small. So some kind of "dynamic summary": for one-liners (or
> ten-liners), show the whole diff. For medium-sized changes, show the
> whole diffstat. And for really big changes, show an outline and the
> "768 files changed, 179851 lines added, 7630 lines removed" stats.
>
> IOW, whatever fits in, say, 50 lines or less.
>
> That would be absolutely lovely if somebody were to do it.
>
something like:
https://github.com/lacombar/git/commit/3d90830fc8730e4f0cad9974070c6e9241797489
?
I'm still not a huge fan of having the `shortstat' before the
cumulative `dirstat', and still have some trouble wiring correctly the
thresholds.
- Arnaud
> Linus
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PULL] svn-fe updates for master or next
From: David Barr @ 2012-01-27 1:03 UTC (permalink / raw)
To: Jonathan Nieder
Cc: Junio C Hamano, Scott Chacon, Brian Gernhardt, david,
Ramkumar Ramachandra, git, Dmitry Ivankov
In-Reply-To: <20120127004605.GA31538@burratino>
On Fri, Jan 27, 2012 at 11:46 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Jonathan Nieder wrote:
>
>> It simplifies svn-fe a great
>> deal and fulfills a longstanding wish: support for dumps with deltas
>> in them.
>
> Oh, and incremental imports, too. ;-)
Thank you, Jonathan. I forgot our prior discussion about avoiding yet
another history rewrite. This provides a nice cut-point.
I do think we need to gather Dmitry's work on svn-fe and propose a
front-end for core so that it is no longer relegated to contrib.
--
David Barr.
^ permalink raw reply
* Re: make install rewrites source files
From: Junio C Hamano @ 2012-01-27 0:49 UTC (permalink / raw)
To: Clemens Buchacher; +Cc: Hallvard Breien Furuseth, git
In-Reply-To: <20120126225231.GA14753@ecki>
Clemens Buchacher <drizzd@aon.at> writes:
> How about removing the profile-all target and making it a build option
> instead? To enable it, do the usual:
>
> echo PROFILE_BUILD=YesPlease >> config.mak
> echo prefix=... >> config.mak
> make
> su make install
Yeah, I would prefer something like that. We could even keep "profile-all"
target for b/c if we wanted to, no?
^ permalink raw reply
* Re: [PATCH 4/4] config: allow including config from repository blobs
From: Jeff King @ 2012-01-27 0:49 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vsjj20yog.fsf@alter.siamese.dyndns.org>
On Thu, Jan 26, 2012 at 04:35:59PM -0800, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > So yeah, if you are just going to copy it once, or even periodically, it
> > is not that big an advantage. And the example I gave using "git tag" did
> > just that. But I also wanted to allow more complex things, like this:
> >
> > # clone and inspect
> > git clone git://example.com/project.git
> > cd project
> > git show origin:devtools/std_gitconfig
> >
> > # well, that looks pretty good. But I'd like to tweak something.
> > git checkout -b config origin
> > $EDITOR devtools/std_gitconfig
> > git commit -a -m "drop the foo option, which I hate"
> >
> > # OK, let's use it now.
> > git config include.ref config:devtools/std_gitconfig
> >
> > # Weeks pass. Somebody else updates the std_gitconfig.
> > git fetch
> > # let's inspect the changes
> > git checkout config
> > git diff @{u} -- devtools/std_gitconfig
> > # looks good, let's merge (not copy!) them in
> > git merge @{u}
> >
> > This is obviously an advanced thing to be doing.
>
> The "which *I* hate" in the log message makes it sound as if it is a
> personal preference, but in fact this is more about maintaining the
> recommended configuration among participants, no?
No, I meant it explicitly to be about this single user hating it. Note
how the resulting commits are never pushed. It is purely a local
override, but with the added bonus that history is tracked so you can
merge in further changes from upstream.
Of course, you could also share it with others, or do whatever. Once
it's tracked by git, you can be as flexible as you like.
> And if you have the source of the configuration on a branch so that
> people can work on it among themselves, then "config.path =
> ../devtools/std_gitconfig" should be sufficient, no?
Yes, you _could_ just keep it in a branch, merge upstream's changes into
the branch, and then periodically copy it out to your .git directory.
But this removes that final step.
It also does allow "[include]ref = origin/meta:gitconfig" if you want to
live dangerously. I consider that a feature, because it lets the user
make the security tradeoff they deem appropriate. Yes, I want to have
git be secure by default, and yes I want to encourage awareness of the
issues in the documentation for the feature. But I suspect in practice
that many people fetch changes and run "make" without looking at them,
which is basically the exact same hole. If a user has already accepted
that risk, why deny them the convenience of accepting it somewhere else?
> The pros-and-cons between the volume of the change to read include from
> blobs and the benefit illustrated in the use case did not look too good to
> me, at least from the messages in this thread so far.
I didn't think the read-from-blob code was very big or complex (most of
the refactoring was to support parsing an arbitrary buffer, but I think
that may be a good thing to have in the long run, anyway).
But I guess that is all a matter of opinion.
-Peff
^ permalink raw reply
* Re: [PULL] svn-fe updates for master or next
From: Jonathan Nieder @ 2012-01-27 0:46 UTC (permalink / raw)
To: Junio C Hamano
Cc: David Barr, Scott Chacon, Brian Gernhardt, david,
Ramkumar Ramachandra, git, Dmitry Ivankov
In-Reply-To: <20120127003258.GA6946@burratino>
Jonathan Nieder wrote:
> It simplifies svn-fe a great
> deal and fulfills a longstanding wish: support for dumps with deltas
> in them.
Oh, and incremental imports, too. ;-)
^ permalink raw reply
* Re: [PATCH 4/4] config: allow including config from repository blobs
From: Junio C Hamano @ 2012-01-27 0:35 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20120126230054.GC12855@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
> So yeah, if you are just going to copy it once, or even periodically, it
> is not that big an advantage. And the example I gave using "git tag" did
> just that. But I also wanted to allow more complex things, like this:
>
> # clone and inspect
> git clone git://example.com/project.git
> cd project
> git show origin:devtools/std_gitconfig
>
> # well, that looks pretty good. But I'd like to tweak something.
> git checkout -b config origin
> $EDITOR devtools/std_gitconfig
> git commit -a -m "drop the foo option, which I hate"
>
> # OK, let's use it now.
> git config include.ref config:devtools/std_gitconfig
>
> # Weeks pass. Somebody else updates the std_gitconfig.
> git fetch
> # let's inspect the changes
> git checkout config
> git diff @{u} -- devtools/std_gitconfig
> # looks good, let's merge (not copy!) them in
> git merge @{u}
>
> This is obviously an advanced thing to be doing.
The "which *I* hate" in the log message makes it sound as if it is a
personal preference, but in fact this is more about maintaining the
recommended configuration among participants, no? And if you have the
source of the configuration on a branch so that people can work on it
among themselves, then "config.path = ../devtools/std_gitconfig" should be
sufficient, no?
The pros-and-cons between the volume of the change to read include from
blobs and the benefit illustrated in the use case did not look too good to
me, at least from the messages in this thread so far.
^ permalink raw reply
* [PULL] svn-fe updates for master or next
From: Jonathan Nieder @ 2012-01-27 0:33 UTC (permalink / raw)
To: Junio C Hamano
Cc: David Barr, Scott Chacon, Brian Gernhardt, david,
Ramkumar Ramachandra, git, Dmitry Ivankov
In-Reply-To: <20120127001041.GB6158@burratino>
Jonathan Nieder wrote:
> Test results from the svn-fe branch would be interesting. I am
> particularly nervous about asking Junio to pull changes to
> contrib/svn-fe that might break
Eh, you only live once.
Junio, please pull
git://repo.or.cz/git/jrn.git svn-fe
to get the following changes since commit ec014eac0e9e6f30cbbca616090fa2ecf74797e7:
Git 1.7.5 (2011-04-23 23:36:32 -0700)
up to c5bcbcdcfa1e2a1977497cb3a342c0365c8d78d6:
vcs-svn: reset first_commit_done in fast_export_init (2011-06-23 10:04:36 -0500)
I'd even be okay with pulling this for v1.7.9, but application for the
next release would also be fine with me. It simplifies svn-fe a great
deal and fulfills a longstanding wish: support for dumps with deltas
in them. The cost is that commandline usage of the svn-fe tool
becomes a little more complicated since it no longer keeps state
itself but instead reads blobs back from fast-import in order to copy
them between revisions and apply deltas to them.
Sorry I've taken so long to get to this, and thanks to David for the
prodding.
----------------------------------------------------------------
David Barr (9):
vcs-svn: set up channel to read fast-import cat-blob response
vcs-svn: quote paths correctly for ls command
vcs-svn: use mark from previous import for parent commit
vcs-svn: pass paths through to fast-import
vcs-svn: drop string_pool
vcs-svn: drop treap
vcs-svn: drop obj_pool
vcs-svn: avoid using ls command twice
vcs-svn: implement text-delta handling
Dmitry Ivankov (2):
vcs-svn: do not initialize report_buffer twice
vcs-svn: reset first_commit_done in fast_export_init
Jonathan Nieder (31):
vcs-svn: use higher mark numbers for blobs
vcs-svn: save marks for imported commits
vcs-svn: add a comment before each commit
vcs-svn: eliminate repo_tree structure
vcs-svn: handle filenames with dq correctly
Merge branch 'db/length-as-hash' (early part) into db/svn-fe-code-purge
Merge branch 'db/strbufs-for-metadata' into db/svn-fe-code-purge
Makefile: list one vcs-svn/xdiff object or header per line
vcs-svn: learn to maintain a sliding view of a file
vcs-svn: make buffer_read_binary API more convenient
vcs-svn: skeleton of an svn delta parser
vcs-svn: parse svndiff0 window header
vcs-svn: read the preimage when applying deltas
vcs-svn: read inline data from deltas
vcs-svn: read instructions from deltas
vcs-svn: implement copyfrom_data delta instruction
vcs-svn: verify that deltas consume all inline data
vcs-svn: let deltas use data from postimage
vcs-svn: let deltas use data from preimage
Merge commit 'v1.7.5' into svn-fe
Merge branch 'db/vcs-svn-incremental' into svn-fe
Merge branch 'db/svn-fe-code-purge' into svn-fe
Merge branch 'db/delta-applier' into db/text-delta
test-svn-fe: split off "test-svn-fe -d" into a separate function
vcs-svn: cap number of bytes read from sliding view
Merge branch 'db/delta-applier' into svn-fe
Merge branch 'db/delta-applier' into db/text-delta
vcs-svn: guard against overflow when computing preimage length
vcs-svn: avoid hangs from corrupt deltas
Merge branch 'db/text-delta' into svn-fe
Merge branch 'db/text-delta' into svn-fe
.gitignore | 3 -
Makefile | 58 +++++---
contrib/svn-fe/svn-fe.txt | 9 +-
t/t0080-vcs-svn.sh | 117 ---------------
t/t0081-line-buffer.sh | 106 +-------------
t/t9010-svn-fe.sh | 365 ++++++++++++++++++++++++++++++++++++++-------
t/t9011-svn-da.sh | 248 ++++++++++++++++++++++++++++++
test-obj-pool.c | 116 --------------
test-string-pool.c | 31 ----
test-svn-fe.c | 50 ++++++-
test-treap.c | 70 ---------
vcs-svn/LICENSE | 3 +-
vcs-svn/fast_export.c | 253 +++++++++++++++++++++++++++++--
vcs-svn/fast_export.h | 26 +++-
vcs-svn/line_buffer.c | 6 +-
vcs-svn/line_buffer.h | 2 +-
vcs-svn/obj_pool.h | 61 --------
vcs-svn/repo_tree.c | 330 ++++-------------------------------------
vcs-svn/repo_tree.h | 12 +-
vcs-svn/sliding_window.c | 79 ++++++++++
vcs-svn/sliding_window.h | 18 +++
vcs-svn/string_pool.c | 102 -------------
vcs-svn/string_pool.h | 11 --
vcs-svn/string_pool.txt | 43 ------
vcs-svn/svndiff.c | 308 ++++++++++++++++++++++++++++++++++++++
vcs-svn/svndiff.h | 10 ++
vcs-svn/svndump.c | 117 ++++++++++-----
vcs-svn/trp.h | 237 -----------------------------
vcs-svn/trp.txt | 109 --------------
29 files changed, 1434 insertions(+), 1466 deletions(-)
delete mode 100755 t/t0080-vcs-svn.sh
create mode 100755 t/t9011-svn-da.sh
delete mode 100644 test-obj-pool.c
delete mode 100644 test-string-pool.c
delete mode 100644 test-treap.c
delete mode 100644 vcs-svn/obj_pool.h
create mode 100644 vcs-svn/sliding_window.c
create mode 100644 vcs-svn/sliding_window.h
delete mode 100644 vcs-svn/string_pool.c
delete mode 100644 vcs-svn/string_pool.h
delete mode 100644 vcs-svn/string_pool.txt
create mode 100644 vcs-svn/svndiff.c
create mode 100644 vcs-svn/svndiff.h
delete mode 100644 vcs-svn/trp.h
delete mode 100644 vcs-svn/trp.txt
^ permalink raw reply
* Re: [PATCH 1/4] config: add include directive
From: Jeff King @ 2012-01-27 0:32 UTC (permalink / raw)
To: Ævar Arnfjörð Bjarmason; +Cc: git
In-Reply-To: <CACBZZX5_qjC6WZsZ9hKvSR5vQJPs=jgWn-R4EnWZGVq+RvjRyg@mail.gmail.com>
On Fri, Jan 27, 2012 at 01:02:52AM +0100, Ævar Arnfjörð Bjarmason wrote:
> On Thu, Jan 26, 2012 at 08:37, Jeff King <peff@peff.net> wrote:
> > This patch introduces an include directive for config files.
> > It looks like:
> >
> > [include]
> > path = /path/to/file
>
> Very nice, I'd been meaning to resurrect my gitconfig.d series, and
> this series implements a lot of the structural changes needed for that
> sort of thing.
Yeah, that seems like a reasonable thing to do. It could make life
easier for package managers (I think the only reason it has not come up
much is that there simply isn't a lot of third-party git config).
> What do you think of an option (e.g. include.gitconfig_d = true) that
> would cause git to look in:
>
> /etc/gitconfig.d/*
> ~/.gitconfig.d/*
> .git/config.d/*
Hmm. Is that really worth having an option? I.e., why not just always
check those directories?
I could see having
[include]
dir = /path/to/gitconfig.d
for non-standard directories, though (or perhaps even simpler, the
"path" directive should auto-detect a file versus a directory. Similarly
the "ref" form could detect and expand a tree).
-Peff
^ permalink raw reply
* Re: [linux.conf.au] VCS Interoperability
From: Jonathan Nieder @ 2012-01-27 0:10 UTC (permalink / raw)
To: David Barr
Cc: Scott Chacon, Brian Gernhardt, david, Ramkumar Ramachandra, git,
Junio C Hamano, Dmitry Ivankov
In-Reply-To: <CAFfmPPPvpbsYz9cjN6OspivCN3dbuPGOU7fyaVdnic3D4V855w@mail.gmail.com>
Hi,
David Barr wrote:
> After giving my talk, I feel compelled to reroll the historic vcs-svn series.
> I'm pushing as I go to my GitHub account:
>
> https://github.com/barrbrain/git/commits/svn-fe-reroll
Just some quick notes to save some time:
All the commits on the
git://repo.or.cz/git/jrn.git svn-fe
branch were known-good in the sense that they seemed sane enough to
build on and I do not think they need any reorganization or other
work. Maybe that could help bootstrap your efforts?
The svn-fe-pu branch includes some other patches: some optimizations
which are probably safe, the demo helper that allows "git clone
svn-alpha::<url>", some transport-helper patches to support the
latter, and so on. They are not vetted. If anyone sends patches from
that branch, or any other patch for that matter, to the list and cc-s
me, I'll be happy to review them. Here's a shortlog for convenience:
David Barr (2):
fast-import: allow object_table to grow dynamically
fast-import: allow atom_table to grow dynamically
Dmitry Ivankov (2):
Arrange a backflow pipe from fast-importer to remote helper stdin
Add alpha version of remote-svn helper
Jonathan Nieder (12):
work around ISO C incompatibility between data and function pointers
ensure initializer elements are computable at load time
enums: omit trailing comma for portability
notes: avoid C99-style comment
notes merge: eliminate OUTPUT macro
make sure initializers are computable at load time
fast-import: allow branch_table to grow dynamically
fast-import: use DIV_ROUND_UP
svn-fe: add a program that generates a notes-to-svn-revs mapping
svn-fe: do not rely on /bin/env utility to launch remote helper
svn-fe: use tabs to indent in remote helper script
remaining warnings
Test results from the svn-fe branch would be interesting. I am
particularly nervous about asking Junio to pull changes to
contrib/svn-fe that might break it at the same time as making the
interface changes needed for support of the "svnadmin dump
--incremental" format without much testing since it would be painful
to back them out. But probably that's moot, since after this long
while there still hasn't been anyone testing it.
Jonathan
^ permalink raw reply
* Re: [PATCH 1/4] config: add include directive
From: Ævar Arnfjörð Bjarmason @ 2012-01-27 0:02 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20120126073752.GA30474@sigill.intra.peff.net>
On Thu, Jan 26, 2012 at 08:37, Jeff King <peff@peff.net> wrote:
> This patch introduces an include directive for config files.
> It looks like:
>
> [include]
> path = /path/to/file
Very nice, I'd been meaning to resurrect my gitconfig.d series, and
this series implements a lot of the structural changes needed for that
sort of thing.
What do you think of an option (e.g. include.gitconfig_d = true) that
would cause git to look in:
/etc/gitconfig.d/*
~/.gitconfig.d/*
.git/config.d/*
As well as the usual:
/etc/gitconfig
~/.gitconfig
.git/config
It would make including third-party config easy since you could just
symlink it in, and it would follow the convention of a lot of other
programs that have a foo and a foo.d directory.
^ permalink raw reply
* Re: [linux.conf.au] VCS Interoperability
From: David Barr @ 2012-01-26 23:47 UTC (permalink / raw)
To: Scott Chacon
Cc: Brian Gernhardt, david, Ramkumar Ramachandra, git, Junio C Hamano,
Jonathan Nieder, Dmitry Ivankov
In-Reply-To: <CAP2yMaLHK2md=MHFmV--R6rmr4q3XuZxqsb2fUszMhssx3GDoA@mail.gmail.com>
>>>> 3. What are your thoughts on lib'ifying Git so that others can call
>>>> into it using an API?
>>>
>>> This is something that everyone agrees would be a good thing. There have been many people who have started projects to do so, but they have all stalled.
>>
>> I believe libgit2 is still under active development.
>>
>> http://libgit2.github.com
>>
>
> Yes, GitHub alone actually has 2 full time employees and 1 contractor
> who are entirely dedicated to the project. It also ships with the
> GitHub for Mac product, so it's used in production by tens of
> thousands every day. If any of you want to get involved, you can check
> out the mailing list (libgit2@librelist.org) or (probably more
> usefully) the GitHub project page:
>
> https://github.com/libgit2/libgit2
>
> Open tickets, contribute code, review PRs, etc.
I'm thinking libgit2 is where git features that have stablised are formalised.
On the other hand, git-core is where features are incubated.
I would like to see fast-import ported to libgit2 when it stabilises ;)
After giving my talk, I feel compelled to reroll the historic vcs-svn series.
I'm pushing as I go to my GitHub account:
https://github.com/barrbrain/git/commits/svn-fe-reroll
--
David Barr
^ permalink raw reply
* Re: How to reorganize git tree
From: Jakub Narebski @ 2012-01-26 23:27 UTC (permalink / raw)
To: Ben Walton; +Cc: Alan Edwards, git
In-Reply-To: <1327604128-sup-1697@pinkfloyd.chass.utoronto.ca>
Ben Walton <bwalton@artsci.utoronto.ca> writes:
> Excerpts from Alan Edwards's message of Thu Jan 26 13:35:05 -0500 2012:
>
> Hi Alan,
>
> > Any one have any suggestions?
>
> I think that git filter-branch is going to be your friend here. I
> won't prescribe the details as you should definitely read and
> understand the docs for this one, but things like
> --subdirectory-filter and/or --tree-filter will be of great use to
> you, I think.
Third-party git-subtree tool can also help here, I think.
--
Jakub Narebski
^ permalink raw reply
* Re: [PATCH 4/4] config: allow including config from repository blobs
From: Jeff King @ 2012-01-26 23:00 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vbopq2mk9.fsf@alter.siamese.dyndns.org>
On Thu, Jan 26, 2012 at 01:14:46PM -0800, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > +You can also include configuration from a blob stored in your repository
> > +by setting the special `include.ref` variable to the name of an object
> > +containing your configuration data (in the same format as a regular
> > +config file).
>
> Hmm, the concept is surely *fun*, but is this really worth it?
> [you could just tell people to copy the file into .git]
Yes, that does work. I liked the idea of putting it in the repo, though,
because it means you can actually _track_ the contents, including any
local changes you make.
So yeah, if you are just going to copy it once, or even periodically, it
is not that big an advantage. And the example I gave using "git tag" did
just that. But I also wanted to allow more complex things, like this:
# clone and inspect
git clone git://example.com/project.git
cd project
git show origin:devtools/std_gitconfig
# well, that looks pretty good. But I'd like to tweak something.
git checkout -b config origin
$EDITOR devtools/std_gitconfig
git commit -a -m "drop the foo option, which I hate"
# OK, let's use it now.
git config include.ref config:devtools/std_gitconfig
# Weeks pass. Somebody else updates the std_gitconfig.
git fetch
# let's inspect the changes
git checkout config
git diff @{u} -- devtools/std_gitconfig
# looks good, let's merge (not copy!) them in
git merge @{u}
This is obviously an advanced thing to be doing. But it's cool to me,
because even if you aren't working on a shared project, it is a means of
versioning your config.
-Peff
^ permalink raw reply
* Re: make install rewrites source files
From: Clemens Buchacher @ 2012-01-26 22:52 UTC (permalink / raw)
To: Hallvard Breien Furuseth; +Cc: Junio C Hamano, git
In-Reply-To: <hbf.20120123j61g@bombur.uio.no>
On Mon, Jan 23, 2012 at 09:57:51PM +0100, Hallvard Breien Furuseth wrote:
>
> 'profile-all' makes 'all' with different CFLAGS from those in
> Makefile.
How about removing the profile-all target and making it a build option
instead? To enable it, do the usual:
echo PROFILE_BUILD=YesPlease >> config.mak
echo prefix=... >> config.mak
make
su make install
In the Makefile, we would have
ifdef PROFILE_BUILD
all:
$(MAKE) CFLAGS=... -fprofile-generate ... all-one
$(MAKE) CFLAGS=... -fprofile-use ... all-one
else
all: all-one
endif
and each previous instance of 'all' replaced with 'all-one'.
Clemens
^ permalink raw reply
* Re: [PATCH 1/4] config: add include directive
From: Jeff King @ 2012-01-26 22:51 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vipjy2nbi.fsf@alter.siamese.dyndns.org>
On Thu, Jan 26, 2012 at 12:58:25PM -0800, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > Include directives are turned on for regular git config
> > parsing (i.e., when you call git_config()), as well as for
> > lookups via the "git config" program. They are not turned on
> > in other cases, including:
> >
> > 1. Parsing of other config-like files, like .gitmodules.
> > There isn't a real need, and I'd rather be conservative
> > and avoid unnecessary incompatibility or confusion.
>
> This is a good design decision, I think, but I am not sure where this "we
> do not let gitmodules to honor inclusion" is implemented in this patch. I
> would have expected a patch to "git-submodule.sh" that adds --no-includes
> where it invokes "git config"?
My goal was to leave git_config_from_file untouched, so that the code in
submodule.c:gitmodules_config would work without modification. And that
works, obviously.
However, I didn't think about the fact that git-submodule.sh would be
calling git-config separately, and that is accidentally changed by my
patch. Even if we changed git-submodule to use "git config
--no-includes" that would break any third-party scripts that use "git
config" to read git-like files.
But it would be nice for callers doing "git config foo.bar" to get the
includes by default. So maybe the right rule is:
1. In C:
a. git_config() respects includes automatically.
b. other callers do not do so automatically (e.g., gitmodules via
submodule.c).
(i.e., what is implemented by this patch)
2. Callers of git-config:
a. respect includes for lookup that checks all of the "normal"
config spots in sequence: .git/config, ~/.gitconfig, and
/etc/gitconfig. These are the shell equivalent of calling
git_config().
b. when we are looking in a specific file (via GIT_CONFIG or "git
config -f"), do not respect includes (but allow --includes if
the caller chooses). This specific file may be something like
.gitmodules. Or perhaps somebody is saying "no, I really just
want to know what is in _this_ file, not what the config
ecosystem tells me in general".
And then because of 1a and 2a, most programs should Just Work without
any changes, but because of 1b and 2b, any special uses will have to
decide manually whether they would want to allow includes.
Does that make sense?
> > +Includes
> > +~~~~~~~~
> > +
> > +You can include one config file from another by setting the special
> > +`include.path` variable to the name of the file to be included. The
> > +included file is expanded immediately, as if its contents had been
> > +found at the location of the include directive.
>
> I cannot offer better wording, but the first reaction I had to this was
> "Eh, if I include a file A that includes another file B, to which config
> file the '[include] path = B' directive found in file A relative to?"
>
> Logically it should be relative to A, and I think your implementation
> makes it so, but the above "as if its contents had been found at..."
> sounds as if it should be the same as writing '[include] path = B' in the
> original config file that included A.
Yes, it is relative to A. Anything else would be insane (because it
would mean "A" has to care about who is including it).
I had a similar thought while writing it, but hoped the sentence after
(that you snipped) would make it clear. But I see the "as if..." can be
ambiguous (because it's only mostly "as if"). I started trying to write
something like "with the exception of further includes, which..." but I
think that is just getting more confusing.
How about:
The included file is processed immediately, before any other
directives from the surrounding file.
What I wanted to make clear there is the ordering, which sometimes
matters.
> By the way, I was a bit surprised that you did not have to add the source
> stacking to git_config_from_file().
>
> It was added in 924aaf3 (config.c: Make git_config() work correctly when
> called recursively, 2011-06-16) to address the situation where
> git_config() ends up calling git_config() recursively.
Yeah. I remembered discussing that patch with Ramsay a few months ago,
and I was pleased that it just worked in this case.
> My gut feeling is that logically that issue shouldn't be limited to
> configuration that is coming from file, which in turn makes me suspect
> that the source stacking may be better in one level higher than
> git_config_from_file() so that the ones coming from parameters could
> also be treated as a different kind of source to read configuration
> from.
I do factor it out in the second patch. I also considered how
git_config_from_parameters doesn't share the same source marking. But it
follows a completely different code-path, as its syntax is totally
different. So there's really not much point in adding to the config_file
stack (if you had another source that could be included from a file, you
would want to mention it in the stack. But the command-line parameters
must always be the "root" of the stack, so it's sufficient to simply
have a NULL stack to mark that).
> But in practice, including from the command line (i.e. -c include.path=...)
> is somewhat crazy and we probably would not want to support it, so...
Actually, I made sure it works, and it's in the tests. It even complains
if you use a relative path (I thought about using $PWD, but that is too
insane, as git changes the working directory behind the scenes).
The one use I think is to bundle a bunch of related config options, and
then turn them on selectively. So you _could_ do:
cat >>~/.bashrc <<-\EOF
FOO_OPTIONS="-c foo.one=1 -c foo.two=2"
EOF
# without FOO
git blah
# with FOO
git $FOO_OPTIONS blah
but with this patch, you can do:
cat >~/.gitconfig.foo <<-\EOF
[foo]
one = 1
two = 2
EOF
git -c include.path=$HOME/.gitconfig.foo blah
which to my mind is a little more natural. I don't personally plan on
using it, but it was easy to implement (in fact, I'd have to write to
disallow it, since the include magic is in the callback handler).
-Peff
^ permalink raw reply
* Re: [PATCH 1/4] config: add include directive
From: Jeff King @ 2012-01-26 22:43 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johannes Sixt, git
In-Reply-To: <20120126222532.GA12855@sigill.intra.peff.net>
On Thu, Jan 26, 2012 at 05:25:32PM -0500, Jeff King wrote:
> There's a slight complication with when to clear the "I have seen this"
> mark for each source. If you are interested only in breaking cycles,
> then obviously you can just clear the marks as you pop the stack. But if
> you want to stop repeated inclusion down different branches of the
> include tree, you need to keep the marks until you're done (e.g., the
> same file referenced by .git/config and ~/.gitconfig). But you do still
> need to clear them, because we repeatedly call git_config with different
> callback functions, and we need to re-parse each time.
>
> I think it should be sufficient to make the marks live through
> git_config(), and get cleared after that (so all of .git/config,
> ~/.gitconfig, and /etc/gitconfig will only load a given include once,
> but another call to git_config will load it again).
Ugh, actually it's a little trickier. Because a git_config callback can
call git_config again, we want one set of marks per git_config
invocation, but we need to handle two simultaneous invocations. But I
think it's OK. We can stuff the data into the "git_config_include_data",
which has the proper scope.
-Peff
^ permalink raw reply
* Re: Finding all commits which modify a file
From: Junio C Hamano @ 2012-01-26 22:42 UTC (permalink / raw)
To: Neal Groothuis; +Cc: Santi Béjar, Linus Torvalds, git
In-Reply-To: <30433.38.96.167.131.1327508582.squirrel@mail.lo-cal.org>
"Neal Groothuis" <ngroot@lo-cal.org> writes:
> Is there a situation where checking for TREESAMEness before simplification
> is desirable and checking after would not be?
When you do not want to see a side branch that does not contribute to the
end result at all, obviously ;-). Outside that situation, before or after
should not make a difference, I would think.
^ permalink raw reply
* Re: [PATCH 1/4] config: add include directive
From: Jeff King @ 2012-01-26 22:25 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johannes Sixt, git
In-Reply-To: <7vmx9a2o23.fsf@alter.siamese.dyndns.org>
On Thu, Jan 26, 2012 at 12:42:28PM -0800, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > And no, I didn't do any cycle detection. We could either do:
> >
> > 1. Record some canonical name for each source we look at (probably
> > realpath() for files, and the sha1 for refs), and don't descend
> > into already-seen sources.
> >
> > 2. Simply provide a maximum depth, and don't include beyond it.
> >
> > The latter is much simpler to implement, but I think the former is a
> > little nicer for the user.
>
> Another thing I wondered after reading this patch was that it will be a
> rather common "mistake" to include the same file twice, one in ~/.gitconfig
> and then another in project specific .git/config, or more likely, people
> start using useful ones in ~their/.gitconfig, and then the site administrator
> by popular demand adding the same include in /etc/gitconfig to retroactively
> cause the same file included twice for them.
>
> Your first alternative solution should solve this case nicely as well, I
> would think.
Agreed. I'll implement that, then.
There's a slight complication with when to clear the "I have seen this"
mark for each source. If you are interested only in breaking cycles,
then obviously you can just clear the marks as you pop the stack. But if
you want to stop repeated inclusion down different branches of the
include tree, you need to keep the marks until you're done (e.g., the
same file referenced by .git/config and ~/.gitconfig). But you do still
need to clear them, because we repeatedly call git_config with different
callback functions, and we need to re-parse each time.
I think it should be sufficient to make the marks live through
git_config(), and get cleared after that (so all of .git/config,
~/.gitconfig, and /etc/gitconfig will only load a given include once,
but another call to git_config will load it again).
-Peff
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox