project wide: git config entry for [diff] renames=true

project wide: git config entry for [diff] renames=true

[Joe Perches]

On Thu, 2014-09-25 at 17:03 +0200, Greg Kroah-Hartman wrote:

> In the future, please generate a git "move" diff, which makes it easier
> to review, and prove that nothing really changed.  It also helps if the
> file is a bit different from what you diffed against, which in my case,
> was true.

Maybe it'd be possible to add 

[diff]
	renames = true

to the .git/config file.

but I don't find a mechanism to add anything to the
.git/config and have it be pulled.

Re: project wide: git config entry for [diff] renames=true

[Jeff King]

On Thu, Sep 25, 2014 at 08:48:31AM -0700, Joe Perches wrote:

> On Thu, 2014-09-25 at 17:03 +0200, Greg Kroah-Hartman wrote:
> 
> > In the future, please generate a git "move" diff, which makes it easier
> > to review, and prove that nothing really changed.  It also helps if the
> > file is a bit different from what you diffed against, which in my case,
> > was true.
> 
> Maybe it'd be possible to add 
> 
> [diff]
> 	renames = true
> 
> to the .git/config file.
> 
> but I don't find a mechanism to add anything to the
> .git/config and have it be pulled.

There is no such mechanism within git. We've resisted adding one because
of the danger of something like:

  [diff]
    external = rm -rf /

diff.renames is probably safe, but any config-sharing mechanism would
have to deal with either whitelisting, or providing some mechanism for
the puller to review changes before blindly following them.

-Peff

Re: project wide: git config entry for [diff] renames=true

[Joe Perches]

On Thu, 2014-09-25 at 14:00 -0400, Jeff King wrote:
> On Thu, Sep 25, 2014 at 08:48:31AM -0700, Joe Perches wrote:
> 
> > On Thu, 2014-09-25 at 17:03 +0200, Greg Kroah-Hartman wrote:
> > 
> > > In the future, please generate a git "move" diff, which makes it easier
> > > to review, and prove that nothing really changed.  It also helps if the
> > > file is a bit different from what you diffed against, which in my case,
> > > was true.
> > 
> > Maybe it'd be possible to add 
> > 
> > [diff]
> > 	renames = true
> > 
> > to the .git/config file.
> > 
> > but I don't find a mechanism to add anything to the
> > .git/config and have it be pulled.
> 
> There is no such mechanism within git. We've resisted adding one because
> of the danger of something like:
> 
>   [diff]
>     external = rm -rf /
> 
> diff.renames is probably safe, but any config-sharing mechanism would
> have to deal with either whitelisting, or providing some mechanism for
> the puller to review changes before blindly following them.

Another mechanism might be to add a repository
top level .gitconfig and add whatever to that.

Re: project wide: git config entry for [diff] renames=true

[Junio C Hamano]

Joe Perches <joe@perches.com> writes:

> On Thu, 2014-09-25 at 14:00 -0400, Jeff King wrote:
> ...
>> diff.renames is probably safe, but any config-sharing mechanism would
>> have to deal with either whitelisting, or providing some mechanism for
>> the puller to review changes before blindly following them.
>
> Another mechanism might be to add a repository
> top level .gitconfig and add whatever to that.

That could be smaller half of an implementation detail of one of the
two possibilities Jeff mentioned i.e. "mechanism for the puller to
review changes before blindly following".  It gives the transfer
part.  You still need a new mechanism to make that file that is
tracked in the repository to be used as part of your configuration
variable set after letting the puller to review and approve.

A puller who blindly trust the project could use the "include"
mechanism from your .git/config to include a file with a well-known
name that is tracked by the project _without_ review or approval.  I
doubt we would recommend that in an open source setting, though.

Re: project wide: git config entry for [diff] renames=true

[Junio C Hamano]

Jeff King <peff-AdEPDUrAXsQ@public.gmane.org> writes:

> There is no such mechanism within git. We've resisted adding one because
> of the danger of something like:
>
>   [diff]
>     external = rm -rf /
>
> diff.renames is probably safe, but any config-sharing mechanism would
> have to deal with either whitelisting, or providing some mechanism for
> the puller to review changes before blindly following them.

It might be useful to add a "safe include" feature, perhaps?  We
ship a small set of hardcoded default whitelist (diff.renames may be
included in there), and allow the user who do not want to be
affected to override it with

    [include]
        safe = !diff.renames

or even

    [config]
    	safe = !*

at the same time allow them to add what we do not hardcode to it
using the same mechanism, e.g.

    [config]
    	safe = merge.*

Then

    [include]
	safe
    	path = ../project.gitconfig

    [include]
    	path = $HOME/.gitconfig-variant1

would only allow the variables include.safe deems safe to affect
us from the in-tree file, and use everything from my personal set in
my home directory.



    	
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: project wide: git config entry for [diff] renames=true

[Junio C Hamano]

Junio C Hamano <gitster-e+AXbWqSrlAAvxtiuMwx3w@public.gmane.org>
writes:

> or even
>
>     [config]
>     	safe = !*
> ...

Gaah, I meant [include] in all places I spelled [config] in the
previous message.

[RFC/PATCH 0/2] Introduce safe-include config feature

[Rasmus Villemoes]

[trimming Ccs]

This is an attempt at implementing the suggested safe-include config
feature. It mostly has the semantics Junio suggested in the parent
post, but it does not directly extend the current include directive;
instead, it uses a separate safe-include directive. This is done so
that if a repository is used with both old and new versions of git,
the older versions will just silently ignore the safe-include, instead
of ignoring include.safe and then proceeding to processing "path =
../project.gitconfig".

Config variables are whitelisted using safe-include.whitelist; the
value is interpreted as a whitespace-separated list of, possibly
negated, patterns. Later patterns override earlier ones.

If the feature is deemed worthwhile and my approach is acceptable,
I'll go ahead and try to write some documentation. For now, there is
just a small test script.


Rasmus Villemoes (2):
  config: Add safe-include directive
  config: Add test of safe-include feature

 config.c                       | 91 +++++++++++++++++++++++++++++++++++++--
 t/t1309-config-safe-include.sh | 96 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 184 insertions(+), 3 deletions(-)
 create mode 100755 t/t1309-config-safe-include.sh

-- 
2.0.4

[RFC/PATCH 1/2] config: Add safe-include directive

[Rasmus Villemoes]

This adds a variant of the include directive, where only certain
config variables in the included files are honoured. The set of
honoured variables consists of those the user has mentioned in a
safe-include.whitelist directive, along with a small set of git.git
blessed ones.

This can, for example, be used by a project to supply a set of
suggested configuration variables, such as "diff.renames = true". The
project would provide these in e.g project.gitconfig, and the user then
has to explicitly opt-in by putting

[safe-include]
    path = ../project.gitconfig

into .git/config, possibly preceding the path directive with a
whitelist directive.

The problem with simply using the ordinary include directive for this
purpose is that certain configuration variables (e.g. diff.external)
can allow arbitrary programs to be run.

Older versions of git do not understand the safe-include directives,
so they will effectively just ignore them.

Obviously, we must ignore safe-include.whitelist directives when we
are processing a safe-included file.

Signed-off-by: Rasmus Villemoes <rv@rasmusvillemoes.dk>
---
 config.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 88 insertions(+), 3 deletions(-)

diff --git a/config.c b/config.c
index a677eb6..764cda1 100644
--- a/config.c
+++ b/config.c
@@ -11,6 +11,7 @@
 #include "quote.h"
 #include "hashmap.h"
 #include "string-list.h"
+#include "wildmatch.h"
 
 struct config_source {
 	struct config_source *prev;
@@ -39,6 +40,79 @@ static struct config_source *cf;
 
 static int zlib_compression_seen;
 
+struct safe_var {
+	struct safe_var *next;
+	const char *pattern;
+	int blacklisted;
+};
+
+static int safe_include_depth;
+static struct safe_var *safe_var_head;
+
+static const char *builtin_safe_patterns[] = {
+	"diff.renames",
+};
+
+static int config_name_is_safe(const char *var)
+{
+	struct safe_var *sv;
+	unsigned i;
+
+	for (sv = safe_var_head; sv; sv = sv->next) {
+		/* Handle malformed patterns? */
+		if (wildmatch(sv->pattern, var, WM_CASEFOLD, NULL) == WM_MATCH)
+			return !sv->blacklisted;
+	}
+	for (i = 0; i < ARRAY_SIZE(builtin_safe_patterns); ++i) {
+		if (wildmatch(builtin_safe_patterns[i], var, WM_CASEFOLD, NULL) == WM_MATCH)
+			return 1;
+	}
+
+	return 0;
+}
+
+static void config_add_safe_pattern(const char *p)
+{
+	struct safe_var *sv;
+	int blacklist = 0;
+
+	if (*p == '!') {
+		blacklist = 1;
+		++p;
+	}
+	if (!*p)
+		return;
+	sv = xmalloc(sizeof(*sv));
+	sv->pattern = xstrdup(p);
+	sv->blacklisted = blacklist;
+	sv->next = safe_var_head;
+	safe_var_head = sv;
+}
+
+static void config_add_safe_names(const char *value)
+{
+	char *patterns = xstrdup(value);
+	char *p, *save;
+
+	/*
+	 * This allows giving multiple patterns in a single line, e.g.
+	 *
+	 *     whitelist = !* foo.bar squirrel.*
+	 *
+	 * to override the builtin list of safe vars and only declare
+	 * foo.bar and the squirrel section safe. But it has the
+	 * obvious drawback that one cannot match subsection names
+	 * containing whitespace. The alternative is that the above
+	 * would have to be written on three separate whitelist lines.
+	 */
+	for (p = strtok_r(patterns, " \t", &save); p; p = strtok_r(NULL, " \t", &save)) {
+		config_add_safe_pattern(p);
+	}
+
+	free(patterns);
+}
+
+
 /*
  * Default config_set that contains key-value pairs from the usual set of config
  * config files (i.e repo specific .git/config, user wide ~/.gitconfig, XDG
@@ -142,12 +216,23 @@ int git_config_include(const char *var, const char *value, void *data)
 	 * Pass along all values, including "include" directives; this makes it
 	 * possible to query information on the includes themselves.
 	 */
-	ret = inc->fn(var, value, inc->data);
-	if (ret < 0)
-		return ret;
+	if (safe_include_depth == 0 || config_name_is_safe(var)) {
+		ret = inc->fn(var, value, inc->data);
+		if (ret < 0)
+			return ret;
+	}
 
 	if (!strcmp(var, "include.path"))
 		ret = handle_path_include(value, inc);
+	else if (safe_include_depth == 0
+		 && !strcmp(var, "safe-include.whitelist")) {
+		config_add_safe_names(value);
+	}
+	else if (!strcmp(var, "safe-include.path")) {
+		safe_include_depth++;
+		ret = handle_path_include(value, inc);
+		safe_include_depth--;
+	}
 	return ret;
 }
 
-- 
2.0.4

[RFC/PATCH 2/2] config: Add test of safe-include feature

[Rasmus Villemoes]

This adds a script for testing various aspects of the safe-include feature.

Signed-off-by: Rasmus Villemoes <rv@rasmusvillemoes.dk>
---
 t/t1309-config-safe-include.sh | 96 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 96 insertions(+)
 create mode 100755 t/t1309-config-safe-include.sh

diff --git a/t/t1309-config-safe-include.sh b/t/t1309-config-safe-include.sh
new file mode 100755
index 0000000..b8ccc94
--- /dev/null
+++ b/t/t1309-config-safe-include.sh
@@ -0,0 +1,96 @@
+#!/bin/sh
+
+test_description='test config file safe-include directives'
+. ./test-lib.sh
+
+
+test_expect_success 'blacklist by default' '
+	echo "[diff]external = badprog" >project &&
+	echo "[safe-include]path = project" >.gitconfig &&
+	test_must_fail git config diff.external
+'
+
+
+test_expect_success 'builtin safe rules' '
+	echo "[diff]renames = true" >project &&
+	echo "[safe-include]path = project" >.gitconfig &&
+	echo true >expect &&
+	git config diff.renames >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'user blacklist taking precedence' '
+	echo "[diff]renames = true" >project &&
+	cat >.gitconfig <<-\EOF &&
+	[diff]renames = false
+	[safe-include]whitelist = !diff.renames
+	[safe-include]path = project
+	EOF
+	echo false >expect &&
+	git config diff.renames >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'wildcard matching' '
+	cat >project <<-\EOF &&
+	[test]beer = true
+	[test]bar = true
+	[test]foo = true
+	EOF
+	cat >.gitconfig <<-\EOF &&
+	[safe-include]whitelist = test.b*r
+	[safe-include]path = project
+	EOF
+	printf "test.bar true\ntest.beer true\n" | sort >expect &&
+	git config --get-regexp "^test" | sort >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'ignore whitelist directives in safe-included files' '
+	cat >project <<-\EOF &&
+	[safe-include]whitelist = *
+	[diff]external = badprog
+	EOF
+	echo "[safe-include]path = project" >.gitconfig &&
+	test_must_fail git config diff.external
+'
+
+test_expect_success 'multiple whitelist/blacklist patterns in one line' '
+	cat >.gitconfig <<-\EOF &&
+	[safe-include]whitelist = !* foo.bar squirrel.* !squirrel.xyz
+	[safe-include]path = project
+	EOF
+	cat >project <<-\EOF &&
+	[diff]renames = true
+	[foo]bar = bar
+	[squirrel]abc = abc
+	[squirrel]xyz = xyz
+	EOF
+	test_must_fail git config diff.renames &&
+	test_must_fail git config squirrel.xyz &&
+	echo bar >expect &&
+	git config foo.bar >actual &&
+	test_cmp expect actual
+	echo abc >expect &&
+	git config squirrel.abc >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'case insensitivity' '
+	cat >.gitconfig <<-\EOF &&
+	[safe-include]whitelist = Test.Abc test.xyz
+	[safe-include]path = project
+	EOF
+	cat >project <<-\EOF &&
+	[test]abc = abc
+	[TeST]XyZ = XyZ
+	EOF
+	echo abc >expect &&
+	git config test.abc >actual &&
+	test_cmp expect actual &&
+	echo XyZ >expect &&
+	git config test.xyz >actual &&
+	test_cmp expect actual
+'
+
+test_done
-- 
2.0.4

Re: [RFC/PATCH 1/2] config: Add safe-include directive

[Junio C Hamano]

On Thu, Oct 2, 2014 at 6:37 PM, Rasmus Villemoes <rv@rasmusvillemoes.dk> wrote:
> This adds a variant of the include directive, where only certain
> config variables in the included files are honoured. The set of
> honoured variables consists of those the user has mentioned in a
> safe-include.whitelist directive, along with a small set of git.git
> blessed ones.
>
> This can, for example, be used by a project to supply a set of
> suggested configuration variables, such as "diff.renames = true". The
> project would provide these in e.g project.gitconfig, and the user then
> has to explicitly opt-in by putting
>
> [safe-include]
>     path = ../project.gitconfig
>
> into .git/config, possibly preceding the path directive with a
> whitelist directive.

Good thinking to protect against accidental inclusion by older versions of Git.

Even though I did allude to ../project.gitconfig in the original message, I
think there should probably be an explicit syntax to name a path that is
relative to the root level of the working tree. People do funky things using
$GIT_DIR and $GIT_WORK_TREE to break the ".. relative to the config
file is the root level of the working tree" assumption, and also a repository
can have a regular file ".git" that points at the real location of the directory
that has "config" in it, in which case its parent directory is very unlikely to
be the root level of the working tree.

That syntax _could_ be just a relative path (e.g. project.gitconfig names
the file with that name at the top-level of the working tree), and if we are
to do so, we should forbid any relative path that escapes from the working
tree (e.g. ../project.gitconfig is forbidden, but down/down/../../.gitconfig
could be OK as it is the same as .gitconfig). For that matter, anything with
/./ and /../ in it can safely be forbidden without losing functionality.

The reason why I think it is sufficient to take a relative path as relative
to the working tree is primarily because I do not see a reason why we
would want to do a safer inclusion of anything inside $GIT_DIR (which
would be the natural interpretation if the relatigve path is taken as relative
to the including config file, in the same way as how the regular include
is processed). But I could be missing some other useful use cases.

And we can allow absolute path, e.g. /etc/gitconfig, of course, but I'd
prefer to at least initially forbid an absolute path, due to the same worries
I have against the "unset some variables defined in /etc/gitconfig" topic
we discussed earlier today in a separate thread.

Re: [RFC/PATCH 1/2] config: Add safe-include directive

[Junio C Hamano]

On Thu, Oct 2, 2014 at 10:27 PM, Junio C Hamano <gitster@pobox.com> wrote:
> On Thu, Oct 2, 2014 at 6:37 PM, Rasmus Villemoes <rv@rasmusvillemoes.dk> wrote:
>> This adds a variant of the include directive, where only certain
>> config variables in the included files are honoured. The set of
>> honoured variables consists of those the user has mentioned in a
>> safe-include.whitelist directive, along with a small set of git.git
>> blessed ones.

Another design decision we would need to make is if it should be
allowed for a safe-included file to use safe-include directive to
include other files. Offhand I do not think of a reason we absolutely
need to support it, but there may be an interesting workflow enabled
if we did so. I dunno.

Re: [RFC/PATCH 1/2] config: Add safe-include directive

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Even though I did allude to ../project.gitconfig in the original message, I
> think there should probably be an explicit syntax to name a path that is
> relative to the root level of the working tree. People do funky things using
> $GIT_DIR and $GIT_WORK_TREE to break the ".. relative to the config
> file is the root level of the working tree" assumption, and also a repository
> can have a regular file ".git" that points at the real location of the directory
> that has "config" in it, in which case its parent directory is very unlikely to
> be the root level of the working tree.

There is another reason why I suspect that it may make the resulting
system more useful if we had a way to explicitly mark the path you
used to safeInclude (by the way, we do not do dashes in names for
configuration by convention) as referring to something inside the
project's working tree.  In a bare repository, we might want to grab
the blob at that path in HEAD (i.e. the project's primary branch)
and include its contents.

Re: [RFC/PATCH 1/2] config: Add safe-include directive

[Rasmus Villemoes]

Junio C Hamano <gitster@pobox.com> wrote:

> (by the way, we do not do dashes in names for configuration by
> convention)

OK. Actually, I now think I'd prefer a subsection [include "safe"], but
I don't have any strong preferences regarding the names.

> That syntax _could_ be just a relative path (e.g. project.gitconfig names
> the file with that name at the top-level of the working tree), and if we are
> to do so, we should forbid any relative path that escapes from the working
> tree (e.g. ../project.gitconfig is forbidden, but down/down/../../.gitconfig
> could be OK as it is the same as .gitconfig). For that matter, anything with
> /./ and /../ in it can safely be forbidden without losing functionality.

I agree that it would be most useful to interpret relative paths as
being relative to the working tree. I'm not sure what would be gained by
checking for ./ and ../ components, a symlink could easily be used to
circumvent that.

> And we can allow absolute path, e.g. /etc/gitconfig, of course, but I'd
> prefer to at least initially forbid an absolute path, due to the same worries
> I have against the "unset some variables defined in /etc/gitconfig" topic
> we discussed earlier today in a separate thread.

One might (ab)use the feature to only use some settings from a global
file, e.g.

[include "safe"]
    whitelist = !foo.*
    path = ~/extra.gitconfig

But I'm fine with forbidding absolute paths until someone actually comes
with such a use case.

> Another design decision we would need to make is if it should be
> allowed for a safe-included file to use safe-include directive to
> include other files. Offhand I do not think of a reason we absolutely
> need to support it, but there may be an interesting workflow enabled
> if we did so. I dunno.

After one level of safe-include, any safe-include can also be done as a
normal include (but one may need to spell the path differently if the
two included files are not both at the top of the working tree). One
could imagine a project supplying lots of defaults and splitting those
into separate files, each included from a single project.gitconfig.

Anyway, my proposal allows nesting includes and safe-includes inside
safe-includes; forbidding it would just be a matter of adding a
safe_include_depth == 0 check in two places. (Then safe_include_depth
probably could/should be renamed in_safe_include.) I think I have a
slight preference to allowing nested includes, but if absolute paths are
forbidden for safe-includes, they should also be forbidden for
include-inside-safe-include.

Rasmus

Re: [RFC/PATCH 1/2] config: Add safe-include directive

[Junio C Hamano]

Rasmus Villemoes <rv@rasmusvillemoes.dk> writes:

> Junio C Hamano <gitster@pobox.com> wrote:
>
>> (by the way, we do not do dashes in names for configuration by
>> convention)
>
> OK. Actually, I now think I'd prefer a subsection [include "safe"], but
> I don't have any strong preferences regarding the names.

I think Peff mentioned something about having the second level
between include and path, so I'll defer it to him.

>> That syntax _could_ be just a relative path (e.g. project.gitconfig names
>> the file with that name at the top-level of the working tree), and if we are
>> to do so, we should forbid any relative path that escapes from the working
>> tree (e.g. ../project.gitconfig is forbidden, but down/down/../../.gitconfig
>> could be OK as it is the same as .gitconfig). For that matter, anything with
>> /./ and /../ in it can safely be forbidden without losing functionality.
>
> I agree that it would be most useful to interpret relative paths as
> being relative to the working tree. I'm not sure what would be gained by
> checking for ./ and ../ components, a symlink could easily be used to
> circumvent that.

If the "limit to the the working tree" is the reason to suggest a
relative path to be taken as relative to the working tree, which my
suggestion clearly was, the reader should be intelligent enough to
infer that an implementation working in that mode should make sure
symlinks and any other means do not step outside it.

And as you noticed that, you apparently are ;-)

> One might (ab)use the feature to only use some settings from a global
> file, e.g.
>
> [include "safe"]
>     whitelist = !foo.*
>     path = ~/extra.gitconfig

You do not have to write something you do not want to use in your
own ~/extra.gitconfig that is under your $HOME/, so I'd prefer to
explicitly forbidding such a use case at least in the beginning.