[RFC] Fourth round of support for cloning submodules

[RFC] Fourth round of support for cloning submodules

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

This patch series implements a mechanism for cloning submodules.
Each submodule is specified by a 'submodule.<submodule>.url'
configuration option, e.g.,

bash-3.00$ ./git-config --remote=http://www.liacs.nl/~sverdool/isa.git --get-regexp 'submodule\..*\.url' 
submodule.cloog.url /home/sverdool/public_html/cloog.git
submodule.cloog.url http://www.liacs.nl/~sverdool/cloog.git

git-checkout will use the first url that works.
E.g., a

git clone --submodules ssh://liacs/~/public_html/isa.git

followed by

git checkout origin/submodule

(which only works for me), will use the first url, while a

git clone --submodules http://www.liacs.nl/~sverdool/isa.git

followed by

git checkout origin/submodule

will use the second.

The cloning of submodules is handled inside git-checkout.

git-checkout will now also checkout/fetch any submodule
than is present (i.e., the corresponding subdirectory has a .git)
irrespective of the --submodules option.

A forced checkout of the submodule directory, i.e.,

git checkout submodule

will (clone and) checkout the submodule.

Note that this is still WIP, so there is no need to remind
me that I still need to write documentation and tests.

skimo

[PATCH 01/22] git_connect: unset CONFIG_ENVIRONMENT in child

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 connect.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/connect.c b/connect.c
index 2f61ea3..8f8ff53 100644
--- a/connect.c
+++ b/connect.c
@@ -552,6 +552,7 @@ pid_t git_connect(int fd[2], char *url, const char *prog, int flags)
 		else {
 			unsetenv(ALTERNATE_DB_ENVIRONMENT);
 			unsetenv(DB_ENVIRONMENT);
+			unsetenv(CONFIG_ENVIRONMENT);
 			unsetenv(GIT_DIR_ENVIRONMENT);
 			unsetenv(GRAFT_ENVIRONMENT);
 			unsetenv(INDEX_ENVIRONMENT);
-- 
1.5.2.784.g5532e

[PATCH 02/22] Add dump-config

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

This command dumps the config of a repository and will be used
to read config options from a remote site.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 .gitignore                        |    1 +
 Documentation/cmd-list.perl       |    1 +
 Documentation/git-dump-config.txt |   37 +++++++++++++++++++++++++++++++++++++
 Makefile                          |    1 +
 daemon.c                          |    7 +++++++
 dump-config.c                     |   29 +++++++++++++++++++++++++++++
 6 files changed, 76 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/git-dump-config.txt
 create mode 100644 dump-config.c

diff --git a/.gitignore b/.gitignore
index 4dc0c39..d4e5492 100644
--- a/.gitignore
+++ b/.gitignore
@@ -38,6 +38,7 @@ git-diff-files
 git-diff-index
 git-diff-tree
 git-describe
+git-dump-config
 git-fast-import
 git-fetch
 git-fetch--tool
diff --git a/Documentation/cmd-list.perl b/Documentation/cmd-list.perl
index 443802a..fa04615 100755
--- a/Documentation/cmd-list.perl
+++ b/Documentation/cmd-list.perl
@@ -103,6 +103,7 @@ git-diff-files                          plumbinginterrogators
 git-diff-index                          plumbinginterrogators
 git-diff                                mainporcelain
 git-diff-tree                           plumbinginterrogators
+git-dump-config                         synchelpers
 git-fast-import				ancillarymanipulators
 git-fetch                               mainporcelain
 git-fetch-pack                          synchingrepositories
diff --git a/Documentation/git-dump-config.txt b/Documentation/git-dump-config.txt
new file mode 100644
index 0000000..370781c
--- /dev/null
+++ b/Documentation/git-dump-config.txt
@@ -0,0 +1,37 @@
+git-dump-config(1)
+====================
+
+NAME
+----
+git-dump-config - Dump config options
+
+
+SYNOPSIS
+--------
+'git-dump-config' <directory>
+
+DESCRIPTION
+-----------
+Invoked by 'git-config --remote' and dumps the config file to the
+other end over the git protocol.
+
+This command is usually not invoked directly by the end user.  The UI
+for the protocol is on the 'git-config' side, where it is used to get
+options from a remote repository.
+
+OPTIONS
+-------
+<directory>::
+	The repository to get the config options from.
+
+Author
+------
+Written by Sven Verdoolaege.
+
+Documentation
+--------------
+Documentation by Sven Verdoolaege.
+
+GIT
+---
+Part of the gitlink:git[7] suite
diff --git a/Makefile b/Makefile
index 35864ed..bd94f6d 100644
--- a/Makefile
+++ b/Makefile
@@ -240,6 +240,7 @@ PROGRAMS = \
 	git-fast-import$X \
 	git-merge-base$X \
 	git-daemon$X \
+	git-dump-config$X \
 	git-merge-index$X git-mktag$X git-mktree$X git-patch-id$X \
 	git-peek-remote$X git-receive-pack$X \
 	git-send-pack$X git-shell$X \
diff --git a/daemon.c b/daemon.c
index 674e30d..69afb60 100644
--- a/daemon.c
+++ b/daemon.c
@@ -378,10 +378,17 @@ static int receive_pack(void)
 	return -1;
 }
 
+static int dump_config(void)
+{
+	execl_git_cmd("dump-config", ".", NULL);
+	return -1;
+}
+
 static struct daemon_service daemon_service[] = {
 	{ "upload-archive", "uploadarch", upload_archive, 0, 1 },
 	{ "upload-pack", "uploadpack", upload_pack, 1, 1 },
 	{ "receive-pack", "receivepack", receive_pack, 0, 1 },
+	{ "dump-config", "dumpconfig", dump_config, 0, 1 },
 };
 
 static void enable_service(const char *name, int ena) {
diff --git a/dump-config.c b/dump-config.c
new file mode 100644
index 0000000..355920d
--- /dev/null
+++ b/dump-config.c
@@ -0,0 +1,29 @@
+#include "git-compat-util.h"
+#include "cache.h"
+#include "pkt-line.h"
+
+static const char dump_config_usage[] = "git-dump-config <dir>";
+
+static int dump_config(const char *var, const char *value)
+{
+	packet_write(1, "%s", var);
+	packet_write(1, "%s", value);
+	return 0;
+}
+
+int main(int argc, char **argv)
+{
+	char *dir;
+
+	if (argc != 2)
+		usage(dump_config_usage);
+
+	dir = argv[1];
+	if (!enter_repo(dir, 0))
+		die("'%s': unable to chdir or not a git archive", dir);
+
+	git_config(dump_config);
+	packet_flush(1);
+
+	return 0;
+}
-- 
1.5.2.784.g5532e

[PATCH 03/22] git-config: add --remote option for reading config from remote repo

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 Documentation/git-config.txt |   27 ++++++++++++++++---------
 builtin-config.c             |   44 ++++++++++++++++++++++++++++++++---------
 cache.h                      |    1 +
 config.c                     |   26 ++++++++++++++++++++++++
 4 files changed, 78 insertions(+), 20 deletions(-)

diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt
index 827a499..549ef4e 100644
--- a/Documentation/git-config.txt
+++ b/Documentation/git-config.txt
@@ -9,16 +9,16 @@ git-config - Get and set repository or global options
 SYNOPSIS
 --------
 [verse]
-'git-config' [--system | --global] name [value [value_regex]]
-'git-config' [--system | --global] --add name value
-'git-config' [--system | --global] --replace-all name [value [value_regex]]
-'git-config' [--system | --global] [type] --get name [value_regex]
-'git-config' [--system | --global] [type] --get-all name [value_regex]
-'git-config' [--system | --global] --unset name [value_regex]
-'git-config' [--system | --global] --unset-all name [value_regex]
-'git-config' [--system | --global] --rename-section old_name new_name
-'git-config' [--system | --global] --remove-section name
-'git-config' [--system | --global] -l | --list
+'git-config' [scope] name [value [value_regex]]
+'git-config' [scope] --add name value
+'git-config' [scope] --replace-all name [value [value_regex]]
+'git-config' [scope] [type] --get name [value_regex]
+'git-config' [scope] [type] --get-all name [value_regex]
+'git-config' [scope] --unset name [value_regex]
+'git-config' [scope] --unset-all name [value_regex]
+'git-config' [scope] --rename-section old_name new_name
+'git-config' [scope] --remove-section name
+'git-config' [scope] -l | --list
 
 DESCRIPTION
 -----------
@@ -33,6 +33,9 @@ existing values that match the regexp are updated or unset.  If
 you want to handle the lines that do *not* match the regex, just
 prepend a single exclamation mark in front (see EXAMPLES).
 
+The scope specifier can be either '--system', '--global' or
+'--remote=[<host>:]<directory>'.
+
 The type specifier can be either '--int' or '--bool', which will make
 'git-config' ensure that the variable(s) are of the given type and
 convert the value to the canonical form (simple decimal number for int,
@@ -81,6 +84,10 @@ OPTIONS
 	Use system-wide $(prefix)/etc/gitconfig rather than the repository
 	.git/config.
 
+--remote=[<host>:]<directory
+	Use remote config instead of the repository .git/config.
+	Only available for reading options.
+
 --remove-section::
 	Remove the given section from the configuration file.
 
diff --git a/builtin-config.c b/builtin-config.c
index b2515f7..3a1e86c 100644
--- a/builtin-config.c
+++ b/builtin-config.c
@@ -2,8 +2,10 @@
 #include "cache.h"
 
 static const char git_config_set_usage[] =
-"git-config [ --global | --system ] [ --bool | --int ] [--get | --get-all | --get-regexp | --replace-all | --add | --unset | --unset-all] name [value [value_regex]] | --rename-section old_name new_name | --remove-section name | --list";
+"git-config [ --global | --system | --remote=[<host>:]<directory ] "
+"[ --bool | --int ] [--get | --get-all | --get-regexp | --replace-all | --add | --unset | --unset-all] name [value [value_regex]] | --rename-section old_name new_name | --remove-section name | --list";
 
+static char *dest;
 static char *key;
 static regex_t *key_regexp;
 static regex_t *regexp;
@@ -104,15 +106,19 @@ static int get_value(const char* key_, const char* regex_)
 		}
 	}
 
-	if (do_all && system_wide)
-		git_config_from_file(show_config, system_wide);
-	if (do_all && global)
-		git_config_from_file(show_config, global);
-	git_config_from_file(show_config, local);
-	if (!do_all && !seen && global)
-		git_config_from_file(show_config, global);
-	if (!do_all && !seen && system_wide)
-		git_config_from_file(show_config, system_wide);
+	if (dest)
+		git_config_from_remote(show_config, dest);
+	else {
+		if (do_all && system_wide)
+			git_config_from_file(show_config, system_wide);
+		if (do_all && global)
+			git_config_from_file(show_config, global);
+		git_config_from_file(show_config, local);
+		if (!do_all && !seen && global)
+			git_config_from_file(show_config, global);
+		if (!do_all && !seen && system_wide)
+			git_config_from_file(show_config, system_wide);
+	}
 
 	free(key);
 	if (regexp) {
@@ -155,8 +161,14 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 		}
 		else if (!strcmp(argv[1], "--system"))
 			setenv("GIT_CONFIG", ETC_GITCONFIG, 1);
+		else if (!prefixcmp(argv[1], "--remote="))
+			dest = xstrdup(argv[1]+9);
 		else if (!strcmp(argv[1], "--rename-section")) {
 			int ret;
+			if (dest) {
+				fprintf(stderr, "Cannot rename on remote\n");
+				return 1;
+			}
 			if (argc != 4)
 				usage(git_config_set_usage);
 			ret = git_config_rename_section(argv[2], argv[3]);
@@ -170,6 +182,10 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 		}
 		else if (!strcmp(argv[1], "--remove-section")) {
 			int ret;
+			if (dest) {
+				fprintf(stderr, "Cannot remove on remote\n");
+				return 1;
+			}
 			if (argc != 3)
 				usage(git_config_set_usage);
 			ret = git_config_rename_section(argv[2], NULL);
@@ -191,6 +207,10 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 	case 2:
 		return get_value(argv[1], NULL);
 	case 3:
+		if (dest && prefixcmp(argv[1], "--get")) {
+			fprintf(stderr, "Cannot (un)set on remote\n");
+			return 1;
+		}
 		if (!strcmp(argv[1], "--unset"))
 			return git_config_set(argv[2], NULL);
 		else if (!strcmp(argv[1], "--unset-all"))
@@ -209,6 +229,10 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 
 			return git_config_set(argv[1], argv[2]);
 	case 4:
+		if (dest && prefixcmp(argv[1], "--get")) {
+			fprintf(stderr, "Cannot (un)set on remote\n");
+			return 1;
+		}
 		if (!strcmp(argv[1], "--unset"))
 			return git_config_set_multivar(argv[2], NULL, argv[3], 0);
 		else if (!strcmp(argv[1], "--unset-all"))
diff --git a/cache.h b/cache.h
index ec85d93..6ca65ac 100644
--- a/cache.h
+++ b/cache.h
@@ -499,6 +499,7 @@ extern int update_server_info(int);
 typedef int (*config_fn_t)(const char *, const char *);
 extern int git_default_config(const char *, const char *);
 extern int git_config_from_file(config_fn_t fn, const char *);
+extern int git_config_from_remote(config_fn_t fn, char *dest);
 extern int git_config(config_fn_t fn);
 extern int git_config_int(const char *, const char *);
 extern int git_config_bool(const char *, const char *);
diff --git a/config.c b/config.c
index 0614c2b..dbfae3f 100644
--- a/config.c
+++ b/config.c
@@ -6,9 +6,12 @@
  *
  */
 #include "cache.h"
+#include "pkt-line.h"
 
 #define MAXNAME (256)
 
+static const char *dumpconfig = "git-dump-config";
+
 static FILE *config_file;
 static const char *config_file_name;
 static int config_linenr;
@@ -403,6 +406,29 @@ int git_config_from_file(config_fn_t fn, const char *filename)
 	return ret;
 }
 
+int git_config_from_remote(config_fn_t fn, char *dest)
+{
+	int ret;
+	int fd[2];
+	pid_t pid;
+	static char var[MAXNAME];
+	static char value[1024];
+
+	pid = git_connect(fd, dest, dumpconfig);
+	if (pid < 0)
+		return 1;
+	ret = 0;
+	while (packet_read_line(fd[0], var, sizeof(var))) {
+		if (!packet_read_line(fd[0], value, sizeof(value)))
+			die("Missing value");
+		fn(var, value);
+	}
+	close(fd[0]);
+	close(fd[1]);
+	ret |= finish_connect(pid);
+	return !!ret;
+}
+
 int git_config(config_fn_t fn)
 {
 	int ret = 0;
-- 
1.5.2.784.g5532e

[PATCH 04/22] http.h: make fill_active_slots a function pointer

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

This allows us to use the methods provided by http.c
from within libgit, in particular config.c.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 http-fetch.c |    5 ++++-
 http-push.c  |    5 ++++-
 http.h       |    2 +-
 3 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/http-fetch.c b/http-fetch.c
index 09baedc..53fb2a9 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -317,7 +317,7 @@ static void release_object_request(struct object_request *obj_req)
 }
 
 #ifdef USE_CURL_MULTI
-void fill_active_slots(void)
+static void fetch_fill_active_slots(void)
 {
 	struct object_request *obj_req = object_queue_head;
 	struct active_request_slot *slot = active_queue_head;
@@ -1031,6 +1031,9 @@ int main(int argc, const char **argv)
 	}
 	url = argv[arg];
 
+#ifdef USE_CURL_MULTI
+	fill_active_slots = fetch_fill_active_slots;
+#endif
 	http_init();
 
 	no_pragma_header = curl_slist_append(no_pragma_header, "Pragma:");
diff --git a/http-push.c b/http-push.c
index 79d2c38..8f253fd 100644
--- a/http-push.c
+++ b/http-push.c
@@ -795,7 +795,7 @@ static void finish_request(struct transfer_request *request)
 }
 
 #ifdef USE_CURL_MULTI
-void fill_active_slots(void)
+static void push_fill_active_slots(void)
 {
 	struct transfer_request *request = request_queue_head;
 	struct transfer_request *next;
@@ -2356,6 +2356,9 @@ int main(int argc, char **argv)
 
 	memset(remote_dir_exists, -1, 256);
 
+#ifdef USE_CURL_MULTI
+	fill_active_slots = push_fill_active_slots;
+#endif
 	http_init();
 
 	no_pragma_header = curl_slist_append(no_pragma_header, "Pragma:");
diff --git a/http.h b/http.h
index 69b6b66..7a41cde 100644
--- a/http.h
+++ b/http.h
@@ -69,7 +69,7 @@ extern void finish_all_active_slots(void);
 extern void release_active_slot(struct active_request_slot *slot);
 
 #ifdef USE_CURL_MULTI
-extern void fill_active_slots(void);
+extern void (*fill_active_slots)(void);
 extern void step_active_slots(void);
 #endif
 
-- 
1.5.2.784.g5532e

[PATCH 05/22] git-config: read remote config files over HTTP

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 Makefile           |    7 ++++++-
 builtin-config.c   |    8 ++++++--
 config.c           |   16 +++++++++++++++-
 http.c             |   10 ++++++----
 http.h             |    2 +-
 http_config.h      |    1 +
 http_config_curl.c |   52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 http_config_none.c |    6 ++++++
 8 files changed, 93 insertions(+), 9 deletions(-)
 create mode 100644 http_config.h
 create mode 100644 http_config_curl.c
 create mode 100644 http_config_none.c

diff --git a/Makefile b/Makefile
index bd94f6d..1fa1896 100644
--- a/Makefile
+++ b/Makefile
@@ -320,7 +320,8 @@ LIB_OBJS = \
 	write_or_die.o trace.o list-objects.o grep.o match-trees.o \
 	alloc.o merge-file.o path-list.o help.o unpack-trees.o $(DIFF_OBJS) \
 	color.o wt-status.o archive-zip.o archive-tar.o shallow.o utf8.o \
-	convert.o attr.o decorate.o progress.o mailmap.o symlinks.o remote.o
+	convert.o attr.o decorate.o progress.o mailmap.o symlinks.o remote.o \
+	$(HTTP_CONFIG_OBJ)
 
 BUILTIN_OBJS = \
 	builtin-add.o \
@@ -527,6 +528,10 @@ ifndef NO_CURL
 	ifndef NO_EXPAT
 		EXPAT_LIBEXPAT = -lexpat
 	endif
+	HTTP_CONFIG_OBJ = http_config_curl.o http.o
+	EXTLIBS += $(CURL_LIBCURL)
+else
+	HTTP_CONFIG_OBJ = http_config_none.o
 endif
 
 ifndef NO_OPENSSL
diff --git a/builtin-config.c b/builtin-config.c
index 3a1e86c..7e18f73 100644
--- a/builtin-config.c
+++ b/builtin-config.c
@@ -147,8 +147,12 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 			type = T_INT;
 		else if (!strcmp(argv[1], "--bool"))
 			type = T_BOOL;
-		else if (!strcmp(argv[1], "--list") || !strcmp(argv[1], "-l"))
-			return git_config(show_all_config);
+		else if (!strcmp(argv[1], "--list") || !strcmp(argv[1], "-l")) {
+			if (dest)
+				return git_config_from_remote(show_all_config, dest);
+			else
+				return git_config(show_all_config);
+		}
 		else if (!strcmp(argv[1], "--global")) {
 			char *home = getenv("HOME");
 			if (home) {
diff --git a/config.c b/config.c
index dbfae3f..fc2162b 100644
--- a/config.c
+++ b/config.c
@@ -7,6 +7,7 @@
  */
 #include "cache.h"
 #include "pkt-line.h"
+#include "http_config.h"
 
 #define MAXNAME (256)
 
@@ -406,6 +407,16 @@ int git_config_from_file(config_fn_t fn, const char *filename)
 	return ret;
 }
 
+static int config_from_http(config_fn_t fn, char *dest)
+{
+	char config_temp[50];
+	if (git_http_fetch_config(dest, config_temp, sizeof(config_temp)))
+		return 1;
+	git_config_from_file(fn, config_temp);
+	unlink(config_temp);
+	return 0;
+}
+
 int git_config_from_remote(config_fn_t fn, char *dest)
 {
 	int ret;
@@ -414,7 +425,10 @@ int git_config_from_remote(config_fn_t fn, char *dest)
 	static char var[MAXNAME];
 	static char value[1024];
 
-	pid = git_connect(fd, dest, dumpconfig);
+	if (!prefixcmp(dest, "http://"))
+		return config_from_http(fn, dest);
+
+	pid = git_connect(fd, dest, dumpconfig, 0);
 	if (pid < 0)
 		return 1;
 	ret = 0;
diff --git a/http.c b/http.c
index ae27e0c..c8237cb 100644
--- a/http.c
+++ b/http.c
@@ -25,6 +25,8 @@ long curl_low_speed_limit = -1;
 long curl_low_speed_time = -1;
 int curl_ftp_no_epsv = 0;
 
+void (*fill_active_slots)(void) = NULL;
+
 struct curl_slist *pragma_header;
 
 struct active_request_slot *active_queue_head = NULL;
@@ -394,7 +396,8 @@ void step_active_slots(void)
 	} while (curlm_result == CURLM_CALL_MULTI_PERFORM);
 	if (num_transfers < active_requests) {
 		process_curl_messages();
-		fill_active_slots();
+		if (fill_active_slots)
+			fill_active_slots();
 	}
 }
 #endif
@@ -458,9 +461,8 @@ void release_active_slot(struct active_request_slot *slot)
 		curl_easy_cleanup(slot->curl);
 		slot->curl = NULL;
 	}
-#ifdef USE_CURL_MULTI
-	fill_active_slots();
-#endif
+	if (fill_active_slots)
+		fill_active_slots();
 }
 
 static void finish_active_slot(struct active_request_slot *slot)
diff --git a/http.h b/http.h
index 7a41cde..7f29ff8 100644
--- a/http.h
+++ b/http.h
@@ -68,8 +68,8 @@ extern void run_active_slot(struct active_request_slot *slot);
 extern void finish_all_active_slots(void);
 extern void release_active_slot(struct active_request_slot *slot);
 
-#ifdef USE_CURL_MULTI
 extern void (*fill_active_slots)(void);
+#ifdef USE_CURL_MULTI
 extern void step_active_slots(void);
 #endif
 
diff --git a/http_config.h b/http_config.h
new file mode 100644
index 0000000..25f5c19
--- /dev/null
+++ b/http_config.h
@@ -0,0 +1 @@
+int git_http_fetch_config(const char *repo, char *config_file, int len);
diff --git a/http_config_curl.c b/http_config_curl.c
new file mode 100644
index 0000000..88317cf
--- /dev/null
+++ b/http_config_curl.c
@@ -0,0 +1,52 @@
+#include "http_config.h"
+#include "http.h"
+
+int git_http_fetch_config(const char *repo, char *config, int config_len)
+{
+	char url[PATH_MAX];
+	int len = strlen(repo);
+
+	int fd;
+	FILE *configfile;
+	struct active_request_slot *slot;
+	struct slot_results results;
+
+	strcpy(url, repo);
+	while (len > 0 && url[len-1] == '/')
+		--len;
+	snprintf(url+len, sizeof(url)-len, "/config");
+
+	fd = git_mkstemp(config, config_len, ".config_XXXXXX");
+	if (fd >= 0)
+		configfile = fdopen(fd, "w");
+	if (fd < 0 || !configfile)
+		return error("Unable to open local file %s for config",
+			     config);
+
+	http_init();
+
+	slot = get_active_slot();
+	slot->results = &results;
+	curl_easy_setopt(slot->curl, CURLOPT_FILE, configfile);
+	curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite);
+	curl_easy_setopt(slot->curl, CURLOPT_URL, url);
+	slot->local = configfile;
+
+	if (start_active_slot(slot)) {
+		run_active_slot(slot);
+		if (results.curl_result != CURLE_OK) {
+			fclose(configfile);
+			warning("Unable to get config %s\n%s", url,
+				 curl_errorstr);
+		}
+	} else {
+		fclose(configfile);
+		return error("Unable to start request");
+	}
+
+	http_cleanup();
+
+	fclose(configfile);
+
+	return 0;
+}
diff --git a/http_config_none.c b/http_config_none.c
new file mode 100644
index 0000000..860ae84
--- /dev/null
+++ b/http_config_none.c
@@ -0,0 +1,6 @@
+#include "http_config.h"
+
+int git_http_fetch_config(const char *repo, char *config_file, int len)
+{
+	return error("Reading http config files not supported");
+}
-- 
1.5.2.784.g5532e

[PATCH 06/22] unpack-trees.c: pass cache_entry * to verify_absent rather than just the name

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

We will need the full cache_entry later to figure out if we are dealing
with a submodule.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 unpack-trees.c |   32 ++++++++++++++++----------------
 1 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/unpack-trees.c b/unpack-trees.c
index cac2411..3dac150 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -487,7 +487,7 @@ static int verify_clean_subdirectory(const char *path, const char *action,
  * We do not want to remove or overwrite a working tree file that
  * is not tracked, unless it is ignored.
  */
-static void verify_absent(const char *path, const char *action,
+static void verify_absent(struct cache_entry *ce, const char *action,
 		struct unpack_trees_options *o)
 {
 	struct stat st;
@@ -495,12 +495,12 @@ static void verify_absent(const char *path, const char *action,
 	if (o->index_only || o->reset || !o->update)
 		return;
 
-	if (!lstat(path, &st)) {
+	if (!lstat(ce->name, &st)) {
 		int cnt;
 
-		if (o->dir && excluded(o->dir, path))
+		if (o->dir && excluded(o->dir, ce->name))
 			/*
-			 * path is explicitly excluded, so it is Ok to
+			 * ce->name is explicitly excluded, so it is Ok to
 			 * overwrite it.
 			 */
 			return;
@@ -512,7 +512,7 @@ static void verify_absent(const char *path, const char *action,
 			 * files that are in "foo/" we would lose
 			 * it.
 			 */
-			cnt = verify_clean_subdirectory(path, action, o);
+			cnt = verify_clean_subdirectory(ce->name, action, o);
 
 			/*
 			 * If this removed entries from the index,
@@ -540,7 +540,7 @@ static void verify_absent(const char *path, const char *action,
 		 * delete this path, which is in a subdirectory that
 		 * is being replaced with a blob.
 		 */
-		cnt = cache_name_pos(path, strlen(path));
+		cnt = cache_name_pos(ce->name, strlen(ce->name));
 		if (0 <= cnt) {
 			struct cache_entry *ce = active_cache[cnt];
 			if (!ce_stage(ce) && !ce->ce_mode)
@@ -548,7 +548,7 @@ static void verify_absent(const char *path, const char *action,
 		}
 
 		die("Untracked working tree file '%s' "
-		    "would be %s by merge.", path, action);
+		    "would be %s by merge.", ce->name, action);
 	}
 }
 
@@ -572,7 +572,7 @@ static int merged_entry(struct cache_entry *merge, struct cache_entry *old,
 		}
 	}
 	else {
-		verify_absent(merge->name, "overwritten", o);
+		verify_absent(merge, "overwritten", o);
 		invalidate_ce_path(merge);
 	}
 
@@ -587,7 +587,7 @@ static int deleted_entry(struct cache_entry *ce, struct cache_entry *old,
 	if (old)
 		verify_uptodate(old, o);
 	else
-		verify_absent(ce->name, "removed", o);
+		verify_absent(ce, "removed", o);
 	ce->ce_mode = 0;
 	add_cache_entry(ce, ADD_CACHE_OK_TO_ADD|ADD_CACHE_OK_TO_REPLACE);
 	invalidate_ce_path(ce);
@@ -704,18 +704,18 @@ int threeway_merge(struct cache_entry **stages,
 	if (o->aggressive) {
 		int head_deleted = !head && !df_conflict_head;
 		int remote_deleted = !remote && !df_conflict_remote;
-		const char *path = NULL;
+		struct cache_entry *ce = NULL;
 
 		if (index)
-			path = index->name;
+			ce = index;
 		else if (head)
-			path = head->name;
+			ce = head;
 		else if (remote)
-			path = remote->name;
+			ce = remote;
 		else {
 			for (i = 1; i < o->head_idx; i++) {
 				if (stages[i] && stages[i] != o->df_conflict_entry) {
-					path = stages[i]->name;
+					ce = stages[i];
 					break;
 				}
 			}
@@ -730,8 +730,8 @@ int threeway_merge(struct cache_entry **stages,
 		    (remote_deleted && head && head_match)) {
 			if (index)
 				return deleted_entry(index, index, o);
-			else if (path && !head_deleted)
-				verify_absent(path, "removed", o);
+			else if (ce && !head_deleted)
+				verify_absent(ce, "removed", o);
 			return 0;
 		}
 		/*
-- 
1.5.2.784.g5532e

[PATCH 07/22] git-read-tree: take --submodules option

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

This option currently has no effect.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 Documentation/config.txt |    4 ++++
 builtin-read-tree.c      |   25 ++++++++++++++++++++++---
 cache.h                  |    3 ++-
 unpack-trees.c           |    1 +
 unpack-trees.h           |    1 +
 5 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 179cb17..5045443 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -261,6 +261,10 @@ core.excludeFile::
 	'.git/info/exclude', git looks into this file for patterns
 	of files which are not meant to be tracked.
 
+core.submodules
+	If true, gitlink:git-checkout[1] also checks out submodules.
+	False by default.
+
 alias.*::
 	Command aliases for the gitlink:git[1] command wrapper - e.g.
 	after defining "alias.last = cat-file commit HEAD", the invocation
diff --git a/builtin-read-tree.c b/builtin-read-tree.c
index 316fb0f..929dd95 100644
--- a/builtin-read-tree.c
+++ b/builtin-read-tree.c
@@ -87,14 +87,23 @@ static void prime_cache_tree(void)
 static const char read_tree_usage[] = "git-read-tree (<sha> | [[-m [--aggressive] | --reset | --prefix=<prefix>] [-u | -i]] [--exclude-per-directory=<gitignore>] [--index-output=<file>] <sha1> [<sha2> [<sha3>]])";
 
 static struct lock_file lock_file;
+static struct unpack_trees_options opts;
+
+static int git_read_tree_config(const char *var, const char *value)
+{
+	if (!strcmp(var, "core.submodules")) {
+		opts.submodules = git_config_bool(var, value);
+		return 0;
+	}
+
+	return git_default_config(var, value);
+}
 
 int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 {
 	int i, newfd, stage = 0;
 	unsigned char sha1[20];
-	struct unpack_trees_options opts;
 
-	memset(&opts, 0, sizeof(opts));
 	opts.head_idx = -1;
 
 	setup_git_directory();
@@ -102,7 +111,7 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 
 	newfd = hold_locked_index(&lock_file, 1);
 
-	git_config(git_default_config);
+	git_config(git_read_tree_config);
 
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
@@ -172,6 +181,16 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 			continue;
 		}
 
+		if (!strcmp(arg, "--no-submodules")) {
+			opts.submodules = 0;
+			continue;
+		}
+
+		if (!strcmp(arg, "--submodules")) {
+			opts.submodules = 1;
+			continue;
+		}
+
 		/* "-m" stands for "merge", meaning we start in stage 1 */
 		if (!strcmp(arg, "-m")) {
 			if (stage || opts.merge || opts.prefix)
diff --git a/cache.h b/cache.h
index 6ca65ac..873cb8d 100644
--- a/cache.h
+++ b/cache.h
@@ -406,7 +406,8 @@ struct checkout {
 	unsigned force:1,
 		 quiet:1,
 		 not_new:1,
-		 refresh_cache:1;
+		 refresh_cache:1,
+		 submodules:1;
 };
 
 extern int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *topath);
diff --git a/unpack-trees.c b/unpack-trees.c
index 3dac150..5fa637a 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -352,6 +352,7 @@ int unpack_trees(struct object_list *trees, struct unpack_trees_options *o)
 	state.force = 1;
 	state.quiet = 1;
 	state.refresh_cache = 1;
+	state.submodules = o->submodules;
 
 	o->merge_size = len;
 
diff --git a/unpack-trees.h b/unpack-trees.h
index fee7da4..21005d9 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -15,6 +15,7 @@ struct unpack_trees_options {
 	int trivial_merges_only;
 	int verbose_update;
 	int aggressive;
+	int submodules;
 	const char *prefix;
 	int pos;
 	struct dir_struct *dir;
-- 
1.5.2.784.g5532e

[PATCH 08/22] unpack-trees.c: assume submodules are clean

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

If the submodules are not clean, then we will get an error
when we actally do the checkout.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 unpack-trees.c |   43 ++++++++++++++++++++++++++++++++++---------
 1 files changed, 34 insertions(+), 9 deletions(-)

diff --git a/unpack-trees.c b/unpack-trees.c
index 5fa637a..2f2d9b9 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -5,6 +5,7 @@
 #include "cache-tree.h"
 #include "unpack-trees.h"
 #include "progress.h"
+#include "refs.h"
 
 #define DBRT_DEBUG 1
 
@@ -426,11 +427,24 @@ static void invalidate_ce_path(struct cache_entry *ce)
 		cache_tree_invalidate_path(active_cache_tree, ce->name);
 }
 
-static int verify_clean_subdirectory(const char *path, const char *action,
+/* Check that checking out ce->sha1 in subdir ce->name is not
+ * going to overwrite any working files.
+ *
+ * FIXME: implement this function, so we can detect problems
+ *        early, rather than waiting until we actually try to checkout
+ *        the submodules.
+ */
+static int verify_clean_submodule(struct cache_entry *ce, const char *action,
+				      struct unpack_trees_options *o)
+{
+	return 0;
+}
+
+static int verify_clean_subdirectory(struct cache_entry *ce, const char *action,
 				      struct unpack_trees_options *o)
 {
 	/*
-	 * we are about to extract "path"; we would not want to lose
+	 * we are about to extract "ce->name"; we would not want to lose
 	 * anything in the existing directory there.
 	 */
 	int namelen;
@@ -438,13 +452,24 @@ static int verify_clean_subdirectory(const char *path, const char *action,
 	struct dir_struct d;
 	char *pathbuf;
 	int cnt = 0;
+	unsigned char sha1[20];
+
+	if (S_ISGITLINK(ntohl(ce->ce_mode)) &&
+	    resolve_gitlink_ref(ce->name, "HEAD", sha1) == 0) {
+		/* If we are not going to update the submodule, then
+		 * we don't care.
+		 */
+		if (!hashcmp(sha1, ce->sha1))
+			return 0;
+		verify_clean_submodule(ce, action, o);
+	}
 
 	/*
 	 * First let's make sure we do not have a local modification
 	 * in that directory.
 	 */
-	namelen = strlen(path);
-	pos = cache_name_pos(path, namelen);
+	namelen = strlen(ce->name);
+	pos = cache_name_pos(ce->name, namelen);
 	if (0 <= pos)
 		return cnt; /* we have it as nondirectory */
 	pos = -pos - 1;
@@ -452,7 +477,7 @@ static int verify_clean_subdirectory(const char *path, const char *action,
 		struct cache_entry *ce = active_cache[i];
 		int len = ce_namelen(ce);
 		if (len < namelen ||
-		    strncmp(path, ce->name, namelen) ||
+		    strncmp(ce->name, ce->name, namelen) ||
 		    ce->name[namelen] != '/')
 			break;
 		/*
@@ -470,16 +495,16 @@ static int verify_clean_subdirectory(const char *path, const char *action,
 	 * present file that is not ignored.
 	 */
 	pathbuf = xmalloc(namelen + 2);
-	memcpy(pathbuf, path, namelen);
+	memcpy(pathbuf, ce->name, namelen);
 	strcpy(pathbuf+namelen, "/");
 
 	memset(&d, 0, sizeof(d));
 	if (o->dir)
 		d.exclude_per_dir = o->dir->exclude_per_dir;
-	i = read_directory(&d, path, pathbuf, namelen+1, NULL);
+	i = read_directory(&d, ce->name, pathbuf, namelen+1, NULL);
 	if (i)
 		die("Updating '%s' would lose untracked files in it",
-		    path);
+		    ce->name);
 	free(pathbuf);
 	return cnt;
 }
@@ -513,7 +538,7 @@ static void verify_absent(struct cache_entry *ce, const char *action,
 			 * files that are in "foo/" we would lose
 			 * it.
 			 */
-			cnt = verify_clean_subdirectory(ce->name, action, o);
+			cnt = verify_clean_subdirectory(ce, action, o);
 
 			/*
 			 * If this removed entries from the index,
-- 
1.5.2.784.g5532e

[PATCH 09/22] Add run_command_v_opt_cd: chdir into a directory before exec

[skimo]

From: Alex Riesen <raa.lkml@gmail.com>

It can make code simplier (no need to preserve cwd) and safer
(no chance the cwd of the current process is accidentally forgotten).

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
 run-command.c |   27 ++++++++++++++++++++++-----
 run-command.h |    2 ++
 2 files changed, 24 insertions(+), 5 deletions(-)

diff --git a/run-command.c b/run-command.c
index eff523e..043b570 100644
--- a/run-command.c
+++ b/run-command.c
@@ -73,6 +73,9 @@ int start_command(struct child_process *cmd)
 			close(cmd->out);
 		}
 
+		if (cmd->dir && chdir(cmd->dir))
+			die("exec %s: cd to %s failed (%s)", cmd->argv[0],
+			    cmd->dir, strerror(errno));
 		if (cmd->git_cmd) {
 			execv_git_cmd(cmd->argv);
 		} else {
@@ -133,13 +136,27 @@ int run_command(struct child_process *cmd)
 	return finish_command(cmd);
 }
 
+static void prepare_run_command_v_opt(struct child_process *cmd,
+				      const char **argv, int opt)
+{
+	memset(cmd, 0, sizeof(*cmd));
+	cmd->argv = argv;
+	cmd->no_stdin = opt & RUN_COMMAND_NO_STDIN ? 1 : 0;
+	cmd->git_cmd = opt & RUN_GIT_CMD ? 1 : 0;
+	cmd->stdout_to_stderr = opt & RUN_COMMAND_STDOUT_TO_STDERR ? 1 : 0;
+}
+
 int run_command_v_opt(const char **argv, int opt)
 {
 	struct child_process cmd;
-	memset(&cmd, 0, sizeof(cmd));
-	cmd.argv = argv;
-	cmd.no_stdin = opt & RUN_COMMAND_NO_STDIN ? 1 : 0;
-	cmd.git_cmd = opt & RUN_GIT_CMD ? 1 : 0;
-	cmd.stdout_to_stderr = opt & RUN_COMMAND_STDOUT_TO_STDERR ? 1 : 0;
+	prepare_run_command_v_opt(&cmd, argv, opt);
+	return run_command(&cmd);
+}
+
+int run_command_v_opt_cd(const char **argv, int opt, const char *dir)
+{
+	struct child_process cmd;
+	prepare_run_command_v_opt(&cmd, argv, opt);
+	cmd.dir = dir;
 	return run_command(&cmd);
 }
diff --git a/run-command.h b/run-command.h
index 3680ef9..cbd7484 100644
--- a/run-command.h
+++ b/run-command.h
@@ -16,6 +16,7 @@ struct child_process {
 	pid_t pid;
 	int in;
 	int out;
+	const char *dir;
 	unsigned close_in:1;
 	unsigned close_out:1;
 	unsigned no_stdin:1;
@@ -32,5 +33,6 @@ int run_command(struct child_process *);
 #define RUN_GIT_CMD	     2	/*If this is to be git sub-command */
 #define RUN_COMMAND_STDOUT_TO_STDERR 4
 int run_command_v_opt(const char **argv, int opt);
+int run_command_v_opt_cd(const char **argv, int opt, const char *dir);
 
 #endif
-- 
1.5.2.784.g5532e

[PATCH 10/22] run-command: optionally clear git environment

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 run-command.c |    9 +++++++++
 run-command.h |    2 ++
 2 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/run-command.c b/run-command.c
index 043b570..806af46 100644
--- a/run-command.c
+++ b/run-command.c
@@ -76,6 +76,14 @@ int start_command(struct child_process *cmd)
 		if (cmd->dir && chdir(cmd->dir))
 			die("exec %s: cd to %s failed (%s)", cmd->argv[0],
 			    cmd->dir, strerror(errno));
+		if (cmd->clear_git_env) {
+			unsetenv(ALTERNATE_DB_ENVIRONMENT);
+			unsetenv(DB_ENVIRONMENT);
+			unsetenv(CONFIG_ENVIRONMENT);
+			unsetenv(GIT_DIR_ENVIRONMENT);
+			unsetenv(GRAFT_ENVIRONMENT);
+			unsetenv(INDEX_ENVIRONMENT);
+		}
 		if (cmd->git_cmd) {
 			execv_git_cmd(cmd->argv);
 		} else {
@@ -144,6 +152,7 @@ static void prepare_run_command_v_opt(struct child_process *cmd,
 	cmd->no_stdin = opt & RUN_COMMAND_NO_STDIN ? 1 : 0;
 	cmd->git_cmd = opt & RUN_GIT_CMD ? 1 : 0;
 	cmd->stdout_to_stderr = opt & RUN_COMMAND_STDOUT_TO_STDERR ? 1 : 0;
+	cmd->clear_git_env = opt & RUN_COMMAND_CLEAR_GIT_ENV ? 1 : 0;
 }
 
 int run_command_v_opt(const char **argv, int opt)
diff --git a/run-command.h b/run-command.h
index cbd7484..7724118 100644
--- a/run-command.h
+++ b/run-command.h
@@ -23,6 +23,7 @@ struct child_process {
 	unsigned no_stdout:1;
 	unsigned git_cmd:1; /* if this is to be git sub-command */
 	unsigned stdout_to_stderr:1;
+	unsigned clear_git_env:1;
 };
 
 int start_command(struct child_process *);
@@ -32,6 +33,7 @@ int run_command(struct child_process *);
 #define RUN_COMMAND_NO_STDIN 1
 #define RUN_GIT_CMD	     2	/*If this is to be git sub-command */
 #define RUN_COMMAND_STDOUT_TO_STDERR 4
+#define RUN_COMMAND_CLEAR_GIT_ENV	(1 << 3)
 int run_command_v_opt(const char **argv, int opt);
 int run_command_v_opt_cd(const char **argv, int opt, const char *dir);
 
-- 
1.5.2.784.g5532e

Re: [PATCH 10/22] run-command: optionally clear git environment

[Alex Riesen]

On 5/24/07, skimo@liacs.nl <skimo@liacs.nl> wrote:
> +               if (cmd->clear_git_env) {
> +                       unsetenv(ALTERNATE_DB_ENVIRONMENT);
> +                       unsetenv(DB_ENVIRONMENT);
> +                       unsetenv(CONFIG_ENVIRONMENT);
> +                       unsetenv(GIT_DIR_ENVIRONMENT);
> +                       unsetenv(GRAFT_ENVIRONMENT);
> +                       unsetenv(INDEX_ENVIRONMENT);
> +               }

You might want to try the alternative approach from the recently
proposed patches to do the same, but more generic. Would
be less code, too.

Re: [PATCH 10/22] run-command: optionally clear git environment

[Shawn O. Pearce]

Alex Riesen <raa.lkml@gmail.com> wrote:
> On 5/24/07, skimo@liacs.nl <skimo@liacs.nl> wrote:
> >+               if (cmd->clear_git_env) {
> >+                       unsetenv(ALTERNATE_DB_ENVIRONMENT);
> >+                       unsetenv(DB_ENVIRONMENT);
> >+                       unsetenv(CONFIG_ENVIRONMENT);
> >+                       unsetenv(GIT_DIR_ENVIRONMENT);
> >+                       unsetenv(GRAFT_ENVIRONMENT);
> >+                       unsetenv(INDEX_ENVIRONMENT);
> >+               }
> 
> You might want to try the alternative approach from the recently
> proposed patches to do the same, but more generic. Would
> be less code, too.

Unfortunately Alex's approach means the caller must know the list of
"special Git envvars" that should be cleared when entering into a
subproject Git repository to execute a command.  That's horrible code
duplication in the callers of run_command, and is just asking for
trouble later when/if another magic environment variable is added.

As long as the above unsetenv list is, I'd really rather have a
specific clear_git_env bit in struct child_process, just so that the
callers don't have to be bothered with the precise list of names.
Of course declaring those names in a static const char** and
looping over it before doing Alex's env array thing would probably
be less code and let the two play along together rather nicely.

-- 
Shawn.

Re: [PATCH 10/22] run-command: optionally clear git environment

[Alex Riesen]

On 5/24/07, Shawn O. Pearce <spearce@spearce.org> wrote:
> Alex Riesen <raa.lkml@gmail.com> wrote:
> > On 5/24/07, skimo@liacs.nl <skimo@liacs.nl> wrote:
> > >+               if (cmd->clear_git_env) {
> > >+                       unsetenv(ALTERNATE_DB_ENVIRONMENT);
> > >+                       unsetenv(DB_ENVIRONMENT);
> > >+                       unsetenv(CONFIG_ENVIRONMENT);
> > >+                       unsetenv(GIT_DIR_ENVIRONMENT);
> > >+                       unsetenv(GRAFT_ENVIRONMENT);
> > >+                       unsetenv(INDEX_ENVIRONMENT);
> > >+               }
> >
> > You might want to try the alternative approach from the recently
> > proposed patches to do the same, but more generic. Would
> > be less code, too.
>
> Unfortunately Alex's approach means the caller must know the list of
> "special Git envvars" that should be cleared when entering into a
> subproject Git repository to execute a command.  That's horrible code
> duplication in the callers of run_command, and is just asking for
> trouble later when/if another magic environment variable is added.

#define GIT_ENV_LIST ALTERNATE_DB_ENVIRONMENT, \
            ...

[PATCH 11/22] entry.c: optionally checkout submodules

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Use run_command_v_opt_cd, as proposed by Alex Riesen.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 entry.c |   29 +++++++++++++++++++++++++++--
 1 files changed, 27 insertions(+), 2 deletions(-)

diff --git a/entry.c b/entry.c
index ae64764..7ba2241 100644
--- a/entry.c
+++ b/entry.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "blob.h"
+#include "run-command.h"
 
 static void create_directories(const char *path, const struct checkout *state)
 {
@@ -75,6 +76,31 @@ static void *read_blob_entry(struct cache_entry *ce, const char *path, unsigned
 	return NULL;
 }
 
+static int checkout_submodule(struct cache_entry *ce, const char *path, const struct checkout *state)
+{
+	const char *args[10];
+	int argc;
+	int err;
+
+	if (!state->submodules)
+		return 0;
+
+	argc = 0;
+	args[argc++] = "checkout";
+	if (state->force)
+	    args[argc++] = "-f";
+	args[argc++] = sha1_to_hex(ce->sha1);
+	args[argc] = NULL;
+
+	err = run_command_v_opt_cd(args, RUN_GIT_CMD|RUN_COMMAND_CLEAR_GIT_ENV,
+				   path);
+
+	if (err)
+		return error("failed to run git-checkout in submodule '%s'", path);
+
+	return 0;
+}
+
 static int write_entry(struct cache_entry *ce, char *path, const struct checkout *state, int to_tempfile)
 {
 	int fd;
@@ -193,9 +219,8 @@ int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *t
 		 */
 		unlink(path);
 		if (S_ISDIR(st.st_mode)) {
-			/* If it is a gitlink, leave it alone! */
 			if (S_ISGITLINK(ntohl(ce->ce_mode)))
-				return 0;
+				return checkout_submodule(ce, path, state);
 			if (!state->force)
 				return error("%s is a directory", path);
 			remove_subtree(path);
-- 
1.5.2.784.g5532e

Re: [PATCH 11/22] entry.c: optionally checkout submodules

[Alex Riesen]

On 5/24/07, skimo@liacs.nl <skimo@liacs.nl> wrote:

> +       args[argc++] = "checkout";
> +       if (state->force)
> +           args[argc++] = "-f";
> +       args[argc++] = sha1_to_hex(ce->sha1);
> +       args[argc] = NULL;

You should consider passing "-v" if the superprojects read-tree
had it. Some submodules will be annoyingly big

Re: [PATCH 11/22] entry.c: optionally checkout submodules

[Shawn O. Pearce]

Alex Riesen <raa.lkml@gmail.com> wrote:
> On 5/24/07, skimo@liacs.nl <skimo@liacs.nl> wrote:
> 
> >+       args[argc++] = "checkout";
> >+       if (state->force)
> >+           args[argc++] = "-f";
> >+       args[argc++] = sha1_to_hex(ce->sha1);
> >+       args[argc] = NULL;
> 
> You should consider passing "-v" if the superprojects read-tree
> had it. Some submodules will be annoyingly big

In 1.5.2 that -v shouldn't be necessary.  The read-tree should
start a timer, and if it has not reached 50% of its processing
within 2 seconds it starts showing progress.  Unless !istty(2),
in which case it just sits there, chugging away at your drive.

I'm actually really unhappy with our !istty(2) means disable
progress thing.  git-gui knows how to read and show the progress
meters, but nobody prints them anymore as 2 is a pipe.  I have the
same problem with a Java build tool that sometimes starts up an
expensive Git operation (like a clone over SSH of a 60+ MiB project).

I've been considering adding a GIT_ISTTY environment variable to
forcefully override the istty result, just to get the progress
meters turned back on...

-- 
Shawn.

Re: [PATCH 11/22] entry.c: optionally checkout submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 03:18:19AM -0400, Shawn O. Pearce wrote:
> Alex Riesen <raa.lkml@gmail.com> wrote:
> > On 5/24/07, skimo@liacs.nl <skimo@liacs.nl> wrote:
> > 
> > >+       args[argc++] = "checkout";
> > >+       if (state->force)
> > >+           args[argc++] = "-f";
> > >+       args[argc++] = sha1_to_hex(ce->sha1);
> > >+       args[argc] = NULL;
> > 
> > You should consider passing "-v" if the superprojects read-tree
> > had it. Some submodules will be annoyingly big
> 
> In 1.5.2 that -v shouldn't be necessary.  The read-tree should
> start a timer, and if it has not reached 50% of its processing
> within 2 seconds it starts showing progress.  Unless !istty(2),
> in which case it just sits there, chugging away at your drive.

Well, git-checkout.sh has this line:

    merge_error=$(git-read-tree $submodules -m -u --exclude-per-directory=.gitignore $old $new 2>&1) || (

so actually you don't see anything right now when a submodule
checkout is going on during a checkout of the supermodule.

skimo

Re: [PATCH 11/22] entry.c: optionally checkout submodules

[Alex Riesen]

On 5/24/07, Shawn O. Pearce <spearce@spearce.org> wrote:
> Alex Riesen <raa.lkml@gmail.com> wrote:
> > On 5/24/07, skimo@liacs.nl <skimo@liacs.nl> wrote:
> >
> > >+       args[argc++] = "checkout";
> > >+       if (state->force)
> > >+           args[argc++] = "-f";
> > >+       args[argc++] = sha1_to_hex(ce->sha1);
> > >+       args[argc] = NULL;
> >
> > You should consider passing "-v" if the superprojects read-tree
> > had it. Some submodules will be annoyingly big
>
> In 1.5.2 that -v shouldn't be necessary.  The read-tree should

It is necessary. Progress meters may not stay forever the only
thing to show when verbose. The code just stripped a part of
users command, how _can_ this be ok?!

> start a timer, and if it has not reached 50% of its processing
> within 2 seconds it starts showing progress.  Unless !istty(2),
> in which case it just sits there, chugging away at your drive.
>
> I'm actually really unhappy with our !istty(2) means disable
> progress thing.  git-gui knows how to read and show the progress
> meters, but nobody prints them anymore as 2 is a pipe.  I have the

Somebody does: just because some stupid script in the middle
did a 2>&1.

> same problem with a Java build tool that sometimes starts up an
> expensive Git operation (like a clone over SSH of a 60+ MiB project).

That said, many tools have explicit --progress switch. Maybe _this_ is
what you need, and not an override whether STDERR is on a tty.

BTW, you could have used ptys, at least on UNIX-like platforms.

Re: [PATCH 11/22] entry.c: optionally checkout submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 767 bytes --]

hoi :)

On Thu, May 24, 2007 at 03:18:19AM -0400, Shawn O. Pearce wrote:
> I'm actually really unhappy with our !istty(2) means disable
> progress thing.  git-gui knows how to read and show the progress
> meters, but nobody prints them anymore as 2 is a pipe.  I have the
> same problem with a Java build tool that sometimes starts up an
> expensive Git operation (like a clone over SSH of a 60+ MiB project).
> 
> I've been considering adding a GIT_ISTTY environment variable to
> forcefully override the istty result, just to get the progress
> meters turned back on...

or perhaps introduce GIT_PROGRESS to name a filedescriptor which then
_only_ gets all the progress information, in a format easily parseable
by other tools?

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH 11/22] entry.c: optionally checkout submodules

[Shawn O. Pearce]

Martin Waitz <tali@admingilde.org> wrote:
> 
> On Thu, May 24, 2007 at 03:18:19AM -0400, Shawn O. Pearce wrote:
> > I'm actually really unhappy with our !istty(2) means disable
> > progress thing.  git-gui knows how to read and show the progress
> > meters, but nobody prints them anymore as 2 is a pipe.  I have the
> > same problem with a Java build tool that sometimes starts up an
> > expensive Git operation (like a clone over SSH of a 60+ MiB project).
> > 
> > I've been considering adding a GIT_ISTTY environment variable to
> > forcefully override the istty result, just to get the progress
> > meters turned back on...
> 
> or perhaps introduce GIT_PROGRESS to name a filedescriptor which then
> _only_ gets all the progress information, in a format easily parseable
> by other tools?

Unfortunately that isn't Tcl friendly, and its *really* not Tcl
on Windows friendly as there we have a difficult time passing
environment variables from Tcl down into Cygwin forked processes.
That problem appears to be a glitch in how Cygwin's Tcl happens to be
implemented on Windows; its actually more a native Tcl than a Cygwin
process, especially when it comes to the builtin Tcl "exec" command.

But its a good idea.  Its just hard for me to take advantage of
it up in git-gui.  ;-)

-- 
Shawn.

[PATCH 12/22] git-checkout: pass --submodules option to git-read-tree

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 git-checkout.sh |   20 +++++++++++++++-----
 1 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/git-checkout.sh b/git-checkout.sh
index 6b6facf..162cef4 100755
--- a/git-checkout.sh
+++ b/git-checkout.sh
@@ -1,6 +1,6 @@
 #!/bin/sh
 
-USAGE='[-q] [-f] [-b <new_branch>] [-m] [<branch>] [<paths>...]'
+USAGE='[-q] [-f] [--submodules] [--no-submodules] [-b <new_branch>] [-m] [<branch>] [<paths>...]'
 SUBDIRECTORY_OK=Sometimes
 . git-sh-setup
 require_work_tree
@@ -16,6 +16,7 @@ track=
 newbranch=
 newbranch_log=
 merge=
+submodules=
 quiet=
 v=-v
 LF='
@@ -46,6 +47,15 @@ while [ "$#" != "0" ]; do
 	-m)
 		merge=1
 		;;
+	--su|--sub|--subm|--submo|--submod|--submodu|--submodul|\
+	--submodule|--submodules)
+		submodules="--submodules"
+		;;
+	--no-su|--no-sub|--no-subm|--no-submo|--no-submod|\
+	--no-submodu|--no-submodul|\
+	--no-submodule|--no-submodules)
+		submodules="--no-submodules"
+		;;
 	"-q")
 		quiet=1
 		v=
@@ -199,10 +209,10 @@ fi
 
 if [ "$force" ]
 then
-    git-read-tree $v --reset -u $new
+    git-read-tree $v $submodules --reset -u $new
 else
     git-update-index --refresh >/dev/null
-    merge_error=$(git-read-tree -m -u --exclude-per-directory=.gitignore $old $new 2>&1) || (
+    merge_error=$(git-read-tree $submodules -m -u --exclude-per-directory=.gitignore $old $new 2>&1) || (
 	case "$merge" in
 	'')
 		echo >&2 "$merge_error"
@@ -212,7 +222,7 @@ else
 	# Match the index to the working tree, and do a three-way.
     	git diff-files --name-only | git update-index --remove --stdin &&
 	work=`git write-tree` &&
-	git read-tree $v --reset -u $new || exit
+	git read-tree $v $submodules --reset -u $new || exit
 
 	eval GITHEAD_$new='${new_name:-${branch:-$new}}' &&
 	eval GITHEAD_$work=local &&
@@ -223,7 +233,7 @@ else
 	# this is not a real merge before committing, but just carrying
 	# the working tree changes along.
 	unmerged=`git ls-files -u`
-	git read-tree $v --reset $new
+	git read-tree $v $submodules --reset $new
 	case "$unmerged" in
 	'')	;;
 	*)
-- 
1.5.2.784.g5532e

[PATCH 13/22] git-read-tree: treat null commit as empty tree

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 builtin-read-tree.c |    9 ++++++---
 unpack-trees.c      |    3 +++
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/builtin-read-tree.c b/builtin-read-tree.c
index 929dd95..b9fcff7 100644
--- a/builtin-read-tree.c
+++ b/builtin-read-tree.c
@@ -17,9 +17,12 @@ static struct object_list *trees;
 
 static int list_tree(unsigned char *sha1)
 {
-	struct tree *tree = parse_tree_indirect(sha1);
-	if (!tree)
-		return -1;
+	struct tree *tree = NULL;
+	if (!is_null_sha1(sha1)) {
+		tree = parse_tree_indirect(sha1);
+		if (!tree)
+			return -1;
+	}
 	object_list_append(&tree->object, &trees);
 	return 0;
 }
diff --git a/unpack-trees.c b/unpack-trees.c
index 2f2d9b9..d5e458d 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -26,6 +26,9 @@ static struct tree_entry_list *create_tree_entry_list(struct tree *tree)
 	struct tree_entry_list *ret = NULL;
 	struct tree_entry_list **list_p = &ret;
 
+	if (!tree)
+		return ret;
+
 	if (!tree->object.parsed)
 		parse_tree(tree);
 
-- 
1.5.2.784.g5532e

[PATCH 14/22] git_config: add void * for callback data

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 archive-tar.c            |    6 +++---
 builtin-add.c            |    6 +++---
 builtin-apply.c          |    6 +++---
 builtin-blame.c          |    6 +++---
 builtin-branch.c         |   10 +++++-----
 builtin-cat-file.c       |    2 +-
 builtin-checkout-index.c |    2 +-
 builtin-commit-tree.c    |    2 +-
 builtin-config.c         |   21 +++++++++++----------
 builtin-diff-files.c     |    2 +-
 builtin-diff-index.c     |    2 +-
 builtin-diff-tree.c      |    2 +-
 builtin-diff.c           |    2 +-
 builtin-fmt-merge-msg.c  |    5 +++--
 builtin-gc.c             |    6 +++---
 builtin-init-db.c        |    4 ++--
 builtin-log.c            |   18 +++++++++---------
 builtin-ls-files.c       |    2 +-
 builtin-ls-tree.c        |    2 +-
 builtin-mailinfo.c       |    2 +-
 builtin-merge-base.c     |    2 +-
 builtin-mv.c             |    2 +-
 builtin-name-rev.c       |    2 +-
 builtin-pack-objects.c   |    6 +++---
 builtin-read-tree.c      |    9 +++++----
 builtin-reflog.c         |    7 ++++---
 builtin-rerere.c         |    6 +++---
 builtin-rev-list.c       |    2 +-
 builtin-rev-parse.c      |    2 +-
 builtin-revert.c         |    2 +-
 builtin-rm.c             |    2 +-
 builtin-runstatus.c      |    2 +-
 builtin-show-branch.c    |    7 ++++---
 builtin-symbolic-ref.c   |    2 +-
 builtin-unpack-objects.c |    2 +-
 builtin-update-index.c   |    2 +-
 builtin-update-ref.c     |    2 +-
 builtin-verify-pack.c    |    2 +-
 cache.h                  |   13 +++++++------
 config.c                 |   36 ++++++++++++++++++------------------
 connect.c                |    7 ++++---
 convert.c                |    5 +++--
 daemon.c                 |    4 ++--
 diff.c                   |    6 +++---
 diff.h                   |    2 +-
 dump-config.c            |    4 ++--
 fast-import.c            |    2 +-
 fetch-pack.c             |    6 +++---
 git.c                    |    4 ++--
 http-fetch.c             |    2 +-
 http.c                   |    6 +++---
 imap-send.c              |    4 ++--
 local-fetch.c            |    2 +-
 merge-recursive.c        |   10 +++++-----
 receive-pack.c           |    6 +++---
 remote.c                 |    4 ++--
 send-pack.c              |    2 +-
 setup.c                  |    5 +++--
 ssh-fetch.c              |    2 +-
 unpack-file.c            |    2 +-
 var.c                    |    8 ++++----
 wt-status.c              |    4 ++--
 wt-status.h              |    2 +-
 63 files changed, 163 insertions(+), 154 deletions(-)

diff --git a/archive-tar.c b/archive-tar.c
index 66fe3e3..9aef671 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -242,7 +242,7 @@ static void write_global_extended_header(const unsigned char *sha1)
 	free(ext_header.buf);
 }
 
-static int git_tar_config(const char *var, const char *value)
+static int git_tar_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "tar.umask")) {
 		if (!strcmp(value, "user")) {
@@ -253,7 +253,7 @@ static int git_tar_config(const char *var, const char *value)
 		}
 		return 0;
 	}
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static int write_tar_entry(const unsigned char *sha1,
@@ -300,7 +300,7 @@ int write_tar_archive(struct archiver_args *args)
 {
 	int plen = args->base ? strlen(args->base) : 0;
 
-	git_config(git_tar_config);
+	git_config(git_tar_config, NULL);
 
 	archive_time = args->time;
 	verbose = args->verbose;
diff --git a/builtin-add.c b/builtin-add.c
index 1591171..1b24c56 100644
--- a/builtin-add.c
+++ b/builtin-add.c
@@ -136,7 +136,7 @@ static void update(int verbose, const char **files)
 	run_diff_files(&rev, 0);
 }
 
-static int git_add_config(const char *var, const char *value)
+static int git_add_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "core.excludesfile")) {
 		if (!value)
@@ -145,7 +145,7 @@ static int git_add_config(const char *var, const char *value)
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static struct lock_file lock_file;
@@ -175,7 +175,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		exit(1);
 	}
 
-	git_config(git_add_config);
+	git_config(git_add_config, NULL);
 
 	newfd = hold_locked_index(&lock_file, 1);
 
diff --git a/builtin-apply.c b/builtin-apply.c
index e717898..f56435e 100644
--- a/builtin-apply.c
+++ b/builtin-apply.c
@@ -2726,13 +2726,13 @@ static int apply_patch(int fd, const char *filename, int inaccurate_eof)
 	return 0;
 }
 
-static int git_apply_config(const char *var, const char *value)
+static int git_apply_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "apply.whitespace")) {
 		apply_default_whitespace = xstrdup(value);
 		return 0;
 	}
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 
@@ -2748,7 +2748,7 @@ int cmd_apply(int argc, const char **argv, const char *unused_prefix)
 
 	prefix = setup_git_directory_gently(&is_not_gitdir);
 	prefix_length = prefix ? strlen(prefix) : 0;
-	git_config(git_apply_config);
+	git_config(git_apply_config, NULL);
 	if (apply_default_whitespace)
 		parse_whitespace_option(apply_default_whitespace);
 
diff --git a/builtin-blame.c b/builtin-blame.c
index 35471fc..0fb76ee 100644
--- a/builtin-blame.c
+++ b/builtin-blame.c
@@ -1973,7 +1973,7 @@ static void prepare_blame_range(struct scoreboard *sb,
 		usage(blame_usage);
 }
 
-static int git_blame_config(const char *var, const char *value)
+static int git_blame_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "blame.showroot")) {
 		show_root = git_config_bool(var, value);
@@ -1983,7 +1983,7 @@ static int git_blame_config(const char *var, const char *value)
 		blank_boundary = git_config_bool(var, value);
 		return 0;
 	}
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static struct commit *fake_working_tree_commit(const char *path, const char *contents_from)
@@ -2136,7 +2136,7 @@ int cmd_blame(int argc, const char **argv, const char *prefix)
 
 	cmd_is_annotate = !strcmp(argv[0], "annotate");
 
-	git_config(git_blame_config);
+	git_config(git_blame_config, NULL);
 	save_commit_buffer = 0;
 
 	opt = 0;
diff --git a/builtin-branch.c b/builtin-branch.c
index a5b6bbe..40cc830 100644
--- a/builtin-branch.c
+++ b/builtin-branch.c
@@ -55,7 +55,7 @@ static int parse_branch_color_slot(const char *var, int ofs)
 	die("bad config variable '%s'", var);
 }
 
-int git_branch_config(const char *var, const char *value)
+int git_branch_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "color.branch")) {
 		branch_use_color = git_config_colorbool(var, value);
@@ -69,7 +69,7 @@ int git_branch_config(const char *var, const char *value)
 	if (!strcmp(var, "branch.autosetupmerge"))
 		branch_track_remotes = git_config_bool(var, value);
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 const char *branch_get_color(enum color_branch ix)
@@ -356,7 +356,7 @@ static int get_remote_branch_name(const char *value)
 	return 0;
 }
 
-static int get_remote_config(const char *key, const char *value)
+static int get_remote_config(const char *key, const char *value, void *cb_data)
 {
 	const char *var;
 	if (prefixcmp(key, "remote."))
@@ -400,7 +400,7 @@ static void set_branch_defaults(const char *name, const char *real_ref)
 	start_ref = real_ref;
 	start_len = strlen(real_ref);
 	base_len = slash - real_ref;
-	git_config(get_remote_config);
+	git_config(get_remote_config, NULL);
 	if (!config_repo && !config_remote &&
 	    !prefixcmp(real_ref, "refs/heads/")) {
 		set_branch_merge(name, ".", real_ref);
@@ -538,7 +538,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 	int kinds = REF_LOCAL_BRANCH;
 	int i;
 
-	git_config(git_branch_config);
+	git_config(git_branch_config, NULL);
 	track = branch_track_remotes;
 
 	for (i = 1; i < argc; i++) {
diff --git a/builtin-cat-file.c b/builtin-cat-file.c
index f132d58..b488fad 100644
--- a/builtin-cat-file.c
+++ b/builtin-cat-file.c
@@ -85,7 +85,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix)
 	int opt;
 	const char *exp_type, *obj_name;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	if (argc != 3)
 		usage("git-cat-file [-t|-s|-e|-p|<type>] <sha1>");
 	exp_type = argv[1];
diff --git a/builtin-checkout-index.c b/builtin-checkout-index.c
index 8460f97..47a8b1b 100644
--- a/builtin-checkout-index.c
+++ b/builtin-checkout-index.c
@@ -168,7 +168,7 @@ int cmd_checkout_index(int argc, const char **argv, const char *prefix)
 	int read_from_stdin = 0;
 	int prefix_length;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	state.base_dir = "";
 	prefix_length = prefix ? strlen(prefix) : 0;
 
diff --git a/builtin-commit-tree.c b/builtin-commit-tree.c
index ccbcbe3..3439321 100644
--- a/builtin-commit-tree.c
+++ b/builtin-commit-tree.c
@@ -92,7 +92,7 @@ int cmd_commit_tree(int argc, const char **argv, const char *prefix)
 	unsigned int size;
 	int encoding_is_utf8;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	if (argc < 2)
 		usage(commit_tree_usage);
diff --git a/builtin-config.c b/builtin-config.c
index 7e18f73..7834e19 100644
--- a/builtin-config.c
+++ b/builtin-config.c
@@ -16,7 +16,7 @@ static int do_not_match;
 static int seen;
 static enum { T_RAW, T_INT, T_BOOL } type = T_RAW;
 
-static int show_all_config(const char *key_, const char *value_)
+static int show_all_config(const char *key_, const char *value_, void *cb_data)
 {
 	if (value_)
 		printf("%s=%s\n", key_, value_);
@@ -25,7 +25,7 @@ static int show_all_config(const char *key_, const char *value_)
 	return 0;
 }
 
-static int show_config(const char* key_, const char* value_)
+static int show_config(const char* key_, const char* value_, void *cb_data)
 {
 	char value[256];
 	const char *vptr = value;
@@ -107,17 +107,17 @@ static int get_value(const char* key_, const char* regex_)
 	}
 
 	if (dest)
-		git_config_from_remote(show_config, dest);
+		git_config_from_remote(show_config, dest, NULL);
 	else {
 		if (do_all && system_wide)
-			git_config_from_file(show_config, system_wide);
+			git_config_from_file(show_config, system_wide, NULL);
 		if (do_all && global)
-			git_config_from_file(show_config, global);
-		git_config_from_file(show_config, local);
+			git_config_from_file(show_config, global, NULL);
+		git_config_from_file(show_config, local, NULL);
 		if (!do_all && !seen && global)
-			git_config_from_file(show_config, global);
+			git_config_from_file(show_config, global, NULL);
 		if (!do_all && !seen && system_wide)
-			git_config_from_file(show_config, system_wide);
+			git_config_from_file(show_config, system_wide, NULL);
 	}
 
 	free(key);
@@ -149,9 +149,10 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 			type = T_BOOL;
 		else if (!strcmp(argv[1], "--list") || !strcmp(argv[1], "-l")) {
 			if (dest)
-				return git_config_from_remote(show_all_config, dest);
+				return git_config_from_remote(show_all_config,
+								dest, NULL);
 			else
-				return git_config(show_all_config);
+				return git_config(show_all_config, NULL);
 		}
 		else if (!strcmp(argv[1], "--global")) {
 			char *home = getenv("HOME");
diff --git a/builtin-diff-files.c b/builtin-diff-files.c
index 6cb30c8..017f4b9 100644
--- a/builtin-diff-files.c
+++ b/builtin-diff-files.c
@@ -21,7 +21,7 @@ int cmd_diff_files(int argc, const char **argv, const char *prefix)
 
 	prefix = setup_git_directory_gently(&nongit);
 	init_revisions(&rev, prefix);
-	git_config(git_default_config); /* no "diff" UI options */
+	git_config(git_default_config, NULL); /* no "diff" UI options */
 	rev.abbrev = 0;
 
 	if (!setup_diff_no_index(&rev, argc, argv, nongit, prefix))
diff --git a/builtin-diff-index.c b/builtin-diff-index.c
index d90eba9..6e92b78 100644
--- a/builtin-diff-index.c
+++ b/builtin-diff-index.c
@@ -17,7 +17,7 @@ int cmd_diff_index(int argc, const char **argv, const char *prefix)
 	int result;
 
 	init_revisions(&rev, prefix);
-	git_config(git_default_config); /* no "diff" UI options */
+	git_config(git_default_config, NULL); /* no "diff" UI options */
 	rev.abbrev = 0;
 
 	argc = setup_revisions(argc, argv, &rev, NULL);
diff --git a/builtin-diff-tree.c b/builtin-diff-tree.c
index 0b591c8..341edad 100644
--- a/builtin-diff-tree.c
+++ b/builtin-diff-tree.c
@@ -68,7 +68,7 @@ int cmd_diff_tree(int argc, const char **argv, const char *prefix)
 	int read_stdin = 0;
 
 	init_revisions(opt, prefix);
-	git_config(git_default_config); /* no "diff" UI options */
+	git_config(git_default_config, NULL); /* no "diff" UI options */
 	nr_sha1 = 0;
 	opt->abbrev = 0;
 	opt->diff = 1;
diff --git a/builtin-diff.c b/builtin-diff.c
index 7f367b6..906b698 100644
--- a/builtin-diff.c
+++ b/builtin-diff.c
@@ -220,7 +220,7 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
 	 */
 
 	prefix = setup_git_directory_gently(&nongit);
-	git_config(git_diff_ui_config);
+	git_config(git_diff_ui_config, NULL);
 	init_revisions(&rev, prefix);
 
 	if (!setup_diff_no_index(&rev, argc, argv, nongit, prefix))
diff --git a/builtin-fmt-merge-msg.c b/builtin-fmt-merge-msg.c
index 5c145d2..a156548 100644
--- a/builtin-fmt-merge-msg.c
+++ b/builtin-fmt-merge-msg.c
@@ -10,7 +10,8 @@ static const char *fmt_merge_msg_usage =
 
 static int merge_summary;
 
-static int fmt_merge_msg_config(const char *key, const char *value)
+static int fmt_merge_msg_config(const char *key, const char *value,
+				void *cb_data)
 {
 	if (!strcmp("merge.summary", key))
 		merge_summary = git_config_bool(key, value);
@@ -251,7 +252,7 @@ int cmd_fmt_merge_msg(int argc, const char **argv, const char *prefix)
 	unsigned char head_sha1[20];
 	const char *current_branch;
 
-	git_config(fmt_merge_msg_config);
+	git_config(fmt_merge_msg_config, NULL);
 
 	while (argc > 1) {
 		if (!strcmp(argv[1], "--summary"))
diff --git a/builtin-gc.c b/builtin-gc.c
index 8ea165a..164fe71 100644
--- a/builtin-gc.c
+++ b/builtin-gc.c
@@ -27,7 +27,7 @@ static const char *argv_repack[MAX_ADD] = {"repack", "-a", "-d", "-l", NULL};
 static const char *argv_prune[] = {"prune", NULL};
 static const char *argv_rerere[] = {"rerere", "gc", NULL};
 
-static int gc_config(const char *var, const char *value)
+static int gc_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "gc.packrefs")) {
 		if (!strcmp(value, "notbare"))
@@ -40,7 +40,7 @@ static int gc_config(const char *var, const char *value)
 		aggressive_window = git_config_int(var, value);
 		return 0;
 	}
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static void append_option(const char **cmd, const char *opt, int max_length)
@@ -62,7 +62,7 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
 	int prune = 0;
 	char buf[80];
 
-	git_config(gc_config);
+	git_config(gc_config, NULL);
 
 	if (pack_refs < 0)
 		pack_refs = !is_bare_repository();
diff --git a/builtin-init-db.c b/builtin-init-db.c
index 4df9fd0..11460b6 100644
--- a/builtin-init-db.c
+++ b/builtin-init-db.c
@@ -152,7 +152,7 @@ static void copy_templates(const char *git_dir, int len, const char *template_di
 	strcpy(template_path + template_len, "config");
 	repository_format_version = 0;
 	git_config_from_file(check_repository_format_version,
-			     template_path);
+			     template_path, NULL);
 	template_path[template_len] = 0;
 
 	if (repository_format_version &&
@@ -207,7 +207,7 @@ static int create_default_files(const char *git_dir, const char *template_path)
 	path[len] = 0;
 	copy_templates(path, len, template_path);
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	/*
 	 * We would have created the above under user's umask -- under
diff --git a/builtin-log.c b/builtin-log.c
index 3744712..2581ce4 100644
--- a/builtin-log.c
+++ b/builtin-log.c
@@ -93,20 +93,20 @@ static int cmd_log_walk(struct rev_info *rev)
 	return 0;
 }
 
-static int git_log_config(const char *var, const char *value)
+static int git_log_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "log.showroot")) {
 		default_show_root = git_config_bool(var, value);
 		return 0;
 	}
-	return git_diff_ui_config(var, value);
+	return git_diff_ui_config(var, value, NULL);
 }
 
 int cmd_whatchanged(int argc, const char **argv, const char *prefix)
 {
 	struct rev_info rev;
 
-	git_config(git_log_config);
+	git_config(git_log_config, NULL);
 	init_revisions(&rev, prefix);
 	rev.diff = 1;
 	rev.diffopt.recursive = 1;
@@ -155,7 +155,7 @@ int cmd_show(int argc, const char **argv, const char *prefix)
 	struct object_array_entry *objects;
 	int i, count, ret = 0;
 
-	git_config(git_log_config);
+	git_config(git_log_config, NULL);
 	init_revisions(&rev, prefix);
 	rev.diff = 1;
 	rev.diffopt.recursive = 1;
@@ -220,7 +220,7 @@ int cmd_log_reflog(int argc, const char **argv, const char *prefix)
 {
 	struct rev_info rev;
 
-	git_config(git_log_config);
+	git_config(git_log_config, NULL);
 	init_revisions(&rev, prefix);
 	init_reflog_walk(&rev.reflog_info);
 	rev.abbrev_commit = 1;
@@ -248,7 +248,7 @@ int cmd_log(int argc, const char **argv, const char *prefix)
 {
 	struct rev_info rev;
 
-	git_config(git_log_config);
+	git_config(git_log_config, NULL);
 	init_revisions(&rev, prefix);
 	rev.always_show_header = 1;
 	cmd_log_init(argc, argv, prefix, &rev);
@@ -268,7 +268,7 @@ static char *extra_headers = NULL;
 static int extra_headers_size = 0;
 static const char *fmt_patch_suffix = ".patch";
 
-static int git_format_config(const char *var, const char *value)
+static int git_format_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "format.headers")) {
 		int len;
@@ -291,7 +291,7 @@ static int git_format_config(const char *var, const char *value)
 	if (!strcmp(var, "diff.color") || !strcmp(var, "color.diff")) {
 		return 0;
 	}
-	return git_log_config(var, value);
+	return git_log_config(var, value, NULL);
 }
 
 
@@ -440,7 +440,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	char message_id[1024];
 	char ref_message_id[1024];
 
-	git_config(git_format_config);
+	git_config(git_format_config, NULL);
 	init_revisions(&rev, prefix);
 	rev.commit_format = CMIT_FMT_EMAIL;
 	rev.verbose_header = 1;
diff --git a/builtin-ls-files.c b/builtin-ls-files.c
index f7c066b..9bdadc4 100644
--- a/builtin-ls-files.c
+++ b/builtin-ls-files.c
@@ -347,7 +347,7 @@ int cmd_ls_files(int argc, const char **argv, const char *prefix)
 	memset(&dir, 0, sizeof(dir));
 	if (prefix)
 		prefix_offset = strlen(prefix);
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
diff --git a/builtin-ls-tree.c b/builtin-ls-tree.c
index cb4be4f..f97a357 100644
--- a/builtin-ls-tree.c
+++ b/builtin-ls-tree.c
@@ -124,7 +124,7 @@ int cmd_ls_tree(int argc, const char **argv, const char *prefix)
 	unsigned char sha1[20];
 	struct tree *tree;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	ls_tree_prefix = prefix;
 	if (prefix && *prefix)
 		chomp_prefix = strlen(prefix);
diff --git a/builtin-mailinfo.c b/builtin-mailinfo.c
index c95e477..82f7a6e 100644
--- a/builtin-mailinfo.c
+++ b/builtin-mailinfo.c
@@ -894,7 +894,7 @@ int cmd_mailinfo(int argc, const char **argv, const char *prefix)
 	/* NEEDSWORK: might want to do the optional .git/ directory
 	 * discovery
 	 */
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	def_charset = (git_commit_encoding ? git_commit_encoding : "utf-8");
 	metainfo_charset = def_charset;
diff --git a/builtin-merge-base.c b/builtin-merge-base.c
index e35d362..9f766c5 100644
--- a/builtin-merge-base.c
+++ b/builtin-merge-base.c
@@ -27,7 +27,7 @@ int cmd_merge_base(int argc, const char **argv, const char *prefix)
 	unsigned char rev1key[20], rev2key[20];
 	int show_all = 0;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	while (1 < argc && argv[1][0] == '-') {
 		const char *arg = argv[1];
diff --git a/builtin-mv.c b/builtin-mv.c
index 3563216..7d01203 100644
--- a/builtin-mv.c
+++ b/builtin-mv.c
@@ -75,7 +75,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)
 	struct path_list deleted = {NULL, 0, 0, 0};
 	struct path_list changed = {NULL, 0, 0, 0};
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	newfd = hold_locked_index(&lock_file, 1);
 	if (read_cache() < 0)
diff --git a/builtin-name-rev.c b/builtin-name-rev.c
index a639e2f..078db48 100644
--- a/builtin-name-rev.c
+++ b/builtin-name-rev.c
@@ -156,7 +156,7 @@ int cmd_name_rev(int argc, const char **argv, const char *prefix)
 	int as_is = 0, all = 0, transform_stdin = 0;
 	struct name_ref_data data = { 0, 0, NULL };
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	if (argc < 2)
 		usage(name_rev_usage);
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 19b0aa1..6c8cef1 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1587,7 +1587,7 @@ static void prepare_pack(int window, int depth)
 	free(delta_list);
 }
 
-static int git_pack_config(const char *k, const char *v)
+static int git_pack_config(const char *k, const char *v, void *cb_data)
 {
 	if(!strcmp(k, "pack.window")) {
 		window = git_config_int(k, v);
@@ -1607,7 +1607,7 @@ static int git_pack_config(const char *k, const char *v)
 		pack_compression_seen = 1;
 		return 0;
 	}
-	return git_default_config(k, v);
+	return git_default_config(k, v, NULL);
 }
 
 static void read_object_list_from_stdin(void)
@@ -1712,7 +1712,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	rp_av[1] = "--objects"; /* --thin will make it --objects-edge */
 	rp_ac = 2;
 
-	git_config(git_pack_config);
+	git_config(git_pack_config, NULL);
 	if (!pack_compression_seen && core_compression_seen)
 		pack_compression_level = core_compression_level;
 
diff --git a/builtin-read-tree.c b/builtin-read-tree.c
index b9fcff7..cec2021 100644
--- a/builtin-read-tree.c
+++ b/builtin-read-tree.c
@@ -92,14 +92,15 @@ static const char read_tree_usage[] = "git-read-tree (<sha> | [[-m [--aggressive
 static struct lock_file lock_file;
 static struct unpack_trees_options opts;
 
-static int git_read_tree_config(const char *var, const char *value)
+static int git_read_tree_config(const char *var, const char *value,
+				void *cb_data)
 {
 	if (!strcmp(var, "core.submodules")) {
 		opts.submodules = git_config_bool(var, value);
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
@@ -110,11 +111,11 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 	opts.head_idx = -1;
 
 	setup_git_directory();
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	newfd = hold_locked_index(&lock_file, 1);
 
-	git_config(git_read_tree_config);
+	git_config(git_read_tree_config, NULL);
 
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
diff --git a/builtin-reflog.c b/builtin-reflog.c
index ce093ca..c616a0a 100644
--- a/builtin-reflog.c
+++ b/builtin-reflog.c
@@ -281,14 +281,15 @@ static int expire_reflog(const char *ref, const unsigned char *sha1, int unused,
 	return status;
 }
 
-static int reflog_expire_config(const char *var, const char *value)
+static int reflog_expire_config(const char *var, const char *value,
+				void *cb_data)
 {
 	if (!strcmp(var, "gc.reflogexpire"))
 		default_reflog_expire = approxidate(value);
 	else if (!strcmp(var, "gc.reflogexpireunreachable"))
 		default_reflog_expire_unreachable = approxidate(value);
 	else
-		return git_default_config(var, value);
+		return git_default_config(var, value, NULL);
 	return 0;
 }
 
@@ -298,7 +299,7 @@ static int cmd_reflog_expire(int argc, const char **argv, const char *prefix)
 	unsigned long now = time(NULL);
 	int i, status, do_all;
 
-	git_config(reflog_expire_config);
+	git_config(reflog_expire_config, NULL);
 
 	save_commit_buffer = 0;
 	do_all = status = 0;
diff --git a/builtin-rerere.c b/builtin-rerere.c
index 8c2c8bd..5d1fe12 100644
--- a/builtin-rerere.c
+++ b/builtin-rerere.c
@@ -380,14 +380,14 @@ tail_optimization:
 	return write_rr(rr, fd);
 }
 
-static int git_rerere_config(const char *var, const char *value)
+static int git_rerere_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "gc.rerereresolved"))
 		cutoff_resolve = git_config_int(var, value);
 	else if (!strcmp(var, "gc.rerereunresolved"))
 		cutoff_noresolve = git_config_int(var, value);
 	else
-		return git_default_config(var, value);
+		return git_default_config(var, value, NULL);
 	return 0;
 }
 
@@ -400,7 +400,7 @@ int cmd_rerere(int argc, const char **argv, const char *prefix)
 	if (stat(git_path("rr-cache"), &st) || !S_ISDIR(st.st_mode))
 		return 0;
 
-	git_config(git_rerere_config);
+	git_config(git_rerere_config, NULL);
 
 	merge_rr_path = xstrdup(git_path("rr-cache/MERGE_RR"));
 	fd = hold_lock_file_for_update(&write_lock, merge_rr_path, 1);
diff --git a/builtin-rev-list.c b/builtin-rev-list.c
index ebf53f5..8700d37 100644
--- a/builtin-rev-list.c
+++ b/builtin-rev-list.c
@@ -468,7 +468,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 	int read_from_stdin = 0;
 	int bisect_show_vars = 0;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	init_revisions(&revs, prefix);
 	revs.abbrev = 0;
 	revs.commit_format = CMIT_FMT_UNSPECIFIED;
diff --git a/builtin-rev-parse.c b/builtin-rev-parse.c
index 37addb2..8e868c3 100644
--- a/builtin-rev-parse.c
+++ b/builtin-rev-parse.c
@@ -214,7 +214,7 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix)
 	int i, as_is = 0, verify = 0;
 	unsigned char sha1[20];
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
diff --git a/builtin-revert.c b/builtin-revert.c
index ea2f15b..aa26c27 100644
--- a/builtin-revert.c
+++ b/builtin-revert.c
@@ -239,7 +239,7 @@ static int revert_or_cherry_pick(int argc, const char **argv)
 	const char *message, *encoding;
 	const char *defmsg = xstrdup(git_path("MERGE_MSG"));
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	me = action == REVERT ? "revert" : "cherry-pick";
 	setenv(GIT_REFLOG_ACTION, me, 0);
 	parse_options(argc, argv);
diff --git a/builtin-rm.c b/builtin-rm.c
index 4a0bd93..7f03735 100644
--- a/builtin-rm.c
+++ b/builtin-rm.c
@@ -109,7 +109,7 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 	const char **pathspec;
 	char *seen;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	newfd = hold_locked_index(&lock_file, 1);
 
diff --git a/builtin-runstatus.c b/builtin-runstatus.c
index 4b489b1..d365d7d 100644
--- a/builtin-runstatus.c
+++ b/builtin-runstatus.c
@@ -11,7 +11,7 @@ int cmd_runstatus(int argc, const char **argv, const char *prefix)
 	struct wt_status s;
 	int i;
 
-	git_config(git_status_config);
+	git_config(git_status_config, NULL);
 	wt_status_prepare(&s);
 
 	for (i = 1; i < argc; i++) {
diff --git a/builtin-show-branch.c b/builtin-show-branch.c
index c892f1f..73565c8 100644
--- a/builtin-show-branch.c
+++ b/builtin-show-branch.c
@@ -531,7 +531,8 @@ static void append_one_rev(const char *av)
 	die("bad sha1 reference %s", av);
 }
 
-static int git_show_branch_config(const char *var, const char *value)
+static int git_show_branch_config(const char *var, const char *value,
+				  void *cb_data)
 {
 	if (!strcmp(var, "showbranch.default")) {
 		if (default_alloc <= default_num + 1) {
@@ -543,7 +544,7 @@ static int git_show_branch_config(const char *var, const char *value)
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static int omit_in_dense(struct commit *commit, struct commit **rev, int n)
@@ -607,7 +608,7 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 	int reflog = 0;
 	const char *reflog_base = NULL;
 
-	git_config(git_show_branch_config);
+	git_config(git_show_branch_config, NULL);
 
 	/* If nothing is specified, try the default first */
 	if (ac == 1 && default_num) {
diff --git a/builtin-symbolic-ref.c b/builtin-symbolic-ref.c
index d41b406..114c473 100644
--- a/builtin-symbolic-ref.c
+++ b/builtin-symbolic-ref.c
@@ -27,7 +27,7 @@ int cmd_symbolic_ref(int argc, const char **argv, const char *prefix)
 	int quiet = 0;
 	const char *msg = NULL;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	while (1 < argc) {
 		const char *arg = argv[1];
diff --git a/builtin-unpack-objects.c b/builtin-unpack-objects.c
index a6ff62f..73dd79f 100644
--- a/builtin-unpack-objects.c
+++ b/builtin-unpack-objects.c
@@ -341,7 +341,7 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix)
 	int i;
 	unsigned char sha1[20];
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	quiet = !isatty(2);
 
diff --git a/builtin-update-index.c b/builtin-update-index.c
index 509369e..ddb429b 100644
--- a/builtin-update-index.c
+++ b/builtin-update-index.c
@@ -570,7 +570,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
 	int lock_error = 0;
 	struct lock_file *lock_file;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	/* We can't free this memory, it becomes part of a linked list parsed atexit() */
 	lock_file = xcalloc(1, sizeof(struct lock_file));
diff --git a/builtin-update-ref.c b/builtin-update-ref.c
index feac2ed..782d4aa 100644
--- a/builtin-update-ref.c
+++ b/builtin-update-ref.c
@@ -14,7 +14,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 
 	delete = 0;
 	ref_flags = 0;
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	for (i = 1; i < argc; i++) {
 		if (!strcmp("-m", argv[i])) {
diff --git a/builtin-verify-pack.c b/builtin-verify-pack.c
index 4e31c27..8a59d14 100644
--- a/builtin-verify-pack.c
+++ b/builtin-verify-pack.c
@@ -55,7 +55,7 @@ int cmd_verify_pack(int argc, const char **argv, const char *prefix)
 	int no_more_options = 0;
 	int nothing_done = 1;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	while (1 < argc) {
 		if (!no_more_options && argv[1][0] == '-') {
 			if (!strcmp("-v", argv[1]))
diff --git a/cache.h b/cache.h
index 873cb8d..54d52cf 100644
--- a/cache.h
+++ b/cache.h
@@ -497,17 +497,18 @@ extern const char *packed_object_info_detail(struct packed_git *, off_t, unsigne
 /* Dumb servers support */
 extern int update_server_info(int);
 
-typedef int (*config_fn_t)(const char *, const char *);
-extern int git_default_config(const char *, const char *);
-extern int git_config_from_file(config_fn_t fn, const char *);
-extern int git_config_from_remote(config_fn_t fn, char *dest);
-extern int git_config(config_fn_t fn);
+typedef int (*config_fn_t)(const char *, const char *, void *cb_data);
+extern int git_default_config(const char *, const char *, void *cb_data);
+extern int git_config_from_file(config_fn_t fn, const char *, void *cb_data);
+extern int git_config_from_remote(config_fn_t fn, char *dest, void *cb_data);
+extern int git_config(config_fn_t fn, void *cb_data);
 extern int git_config_int(const char *, const char *);
 extern int git_config_bool(const char *, const char *);
 extern int git_config_set(const char *, const char *);
 extern int git_config_set_multivar(const char *, const char *, const char *, int);
 extern int git_config_rename_section(const char *, const char *);
-extern int check_repository_format_version(const char *var, const char *value);
+extern int check_repository_format_version(const char *var, const char *value,
+					   void *cb_data);
 
 #define MAX_GITNAME (1000)
 extern char git_default_email[MAX_GITNAME];
diff --git a/config.c b/config.c
index fc2162b..cab6f29 100644
--- a/config.c
+++ b/config.c
@@ -113,7 +113,7 @@ static inline int iskeychar(int c)
 	return isalnum(c) || c == '-';
 }
 
-static int get_value(config_fn_t fn, char *name, unsigned int len)
+static int get_value(config_fn_t fn, char *name, unsigned int len, void *cb_data)
 {
 	int c;
 	char *value;
@@ -141,7 +141,7 @@ static int get_value(config_fn_t fn, char *name, unsigned int len)
 		if (!value)
 			return -1;
 	}
-	return fn(name, value);
+	return fn(name, value, cb_data);
 }
 
 static int get_extended_base_var(char *name, int baselen, int c)
@@ -199,7 +199,7 @@ static int get_base_var(char *name)
 	}
 }
 
-static int git_parse_file(config_fn_t fn)
+static int git_parse_file(config_fn_t fn, void *cb_data)
 {
 	int comment = 0;
 	int baselen = 0;
@@ -231,7 +231,7 @@ static int git_parse_file(config_fn_t fn)
 		if (!isalpha(c))
 			break;
 		var[baselen] = tolower(c);
-		if (get_value(fn, var, baselen+1) < 0)
+		if (get_value(fn, var, baselen+1, cb_data) < 0)
 			break;
 	}
 	die("bad config file line %d in %s", config_linenr, config_file_name);
@@ -267,7 +267,7 @@ int git_config_bool(const char *name, const char *value)
 	return git_config_int(name, value) != 0;
 }
 
-int git_default_config(const char *var, const char *value)
+int git_default_config(const char *var, const char *value, void *cb_data)
 {
 	/* This needs a better name */
 	if (!strcmp(var, "core.filemode")) {
@@ -390,7 +390,7 @@ int git_default_config(const char *var, const char *value)
 	return 0;
 }
 
-int git_config_from_file(config_fn_t fn, const char *filename)
+int git_config_from_file(config_fn_t fn, const char *filename, void *cb_data)
 {
 	int ret;
 	FILE *f = fopen(filename, "r");
@@ -400,24 +400,24 @@ int git_config_from_file(config_fn_t fn, const char *filename)
 		config_file = f;
 		config_file_name = filename;
 		config_linenr = 1;
-		ret = git_parse_file(fn);
+		ret = git_parse_file(fn, cb_data);
 		fclose(f);
 		config_file_name = NULL;
 	}
 	return ret;
 }
 
-static int config_from_http(config_fn_t fn, char *dest)
+static int config_from_http(config_fn_t fn, char *dest, void *cb_data)
 {
 	char config_temp[50];
 	if (git_http_fetch_config(dest, config_temp, sizeof(config_temp)))
 		return 1;
-	git_config_from_file(fn, config_temp);
+	git_config_from_file(fn, config_temp, cb_data);
 	unlink(config_temp);
 	return 0;
 }
 
-int git_config_from_remote(config_fn_t fn, char *dest)
+int git_config_from_remote(config_fn_t fn, char *dest, void *cb_data)
 {
 	int ret;
 	int fd[2];
@@ -426,7 +426,7 @@ int git_config_from_remote(config_fn_t fn, char *dest)
 	static char value[1024];
 
 	if (!prefixcmp(dest, "http://"))
-		return config_from_http(fn, dest);
+		return config_from_http(fn, dest, cb_data);
 
 	pid = git_connect(fd, dest, dumpconfig, 0);
 	if (pid < 0)
@@ -435,7 +435,7 @@ int git_config_from_remote(config_fn_t fn, char *dest)
 	while (packet_read_line(fd[0], var, sizeof(var))) {
 		if (!packet_read_line(fd[0], value, sizeof(value)))
 			die("Missing value");
-		fn(var, value);
+		fn(var, value, cb_data);
 	}
 	close(fd[0]);
 	close(fd[1]);
@@ -443,7 +443,7 @@ int git_config_from_remote(config_fn_t fn, char *dest)
 	return !!ret;
 }
 
-int git_config(config_fn_t fn)
+int git_config(config_fn_t fn, void *cb_data)
 {
 	int ret = 0;
 	char *repo_config = NULL;
@@ -456,7 +456,7 @@ int git_config(config_fn_t fn)
 	filename = getenv(CONFIG_ENVIRONMENT);
 	if (!filename) {
 		if (!access(ETC_GITCONFIG, R_OK))
-			ret += git_config_from_file(fn, ETC_GITCONFIG);
+			ret += git_config_from_file(fn, ETC_GITCONFIG, cb_data);
 		home = getenv("HOME");
 		filename = getenv(CONFIG_LOCAL_ENVIRONMENT);
 		if (!filename)
@@ -466,11 +466,11 @@ int git_config(config_fn_t fn)
 	if (home) {
 		char *user_config = xstrdup(mkpath("%s/.gitconfig", home));
 		if (!access(user_config, R_OK))
-			ret = git_config_from_file(fn, user_config);
+			ret = git_config_from_file(fn, user_config, cb_data);
 		free(user_config);
 	}
 
-	ret += git_config_from_file(fn, filename);
+	ret += git_config_from_file(fn, filename, cb_data);
 	free(repo_config);
 	return ret;
 }
@@ -500,7 +500,7 @@ static int matches(const char* key, const char* value)
 		  !regexec(store.value_regex, value, 0, NULL, 0)));
 }
 
-static int store_aux(const char* key, const char* value)
+static int store_aux(const char* key, const char* value, void *cb_data)
 {
 	const char *ep;
 	size_t section_len;
@@ -836,7 +836,7 @@ int git_config_set_multivar(const char* key, const char* value,
 		 * As a side effect, we make sure to transform only a valid
 		 * existing config file.
 		 */
-		if (git_config_from_file(store_aux, config_filename)) {
+		if (git_config_from_file(store_aux, config_filename, NULL)) {
 			fprintf(stderr, "invalid config file\n");
 			free(store.key);
 			if (store.value_regex != NULL) {
diff --git a/connect.c b/connect.c
index 8f8ff53..fd14a4f 100644
--- a/connect.c
+++ b/connect.c
@@ -320,7 +320,8 @@ static char *git_proxy_command;
 static const char *rhost_name;
 static int rhost_len;
 
-static int git_proxy_command_options(const char *var, const char *value)
+static int git_proxy_command_options(const char *var, const char *value,
+				     void *cb_data)
 {
 	if (!strcmp(var, "core.gitproxy")) {
 		const char *for_pos;
@@ -364,7 +365,7 @@ static int git_proxy_command_options(const char *var, const char *value)
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static int git_use_proxy(const char *host)
@@ -372,7 +373,7 @@ static int git_use_proxy(const char *host)
 	rhost_name = host;
 	rhost_len = strlen(host);
 	git_proxy_command = getenv("GIT_PROXY_COMMAND");
-	git_config(git_proxy_command_options);
+	git_config(git_proxy_command_options, NULL);
 	rhost_name = NULL;
 	return (git_proxy_command && *git_proxy_command);
 }
diff --git a/convert.c b/convert.c
index 4b26b1a..6dde5fa 100644
--- a/convert.c
+++ b/convert.c
@@ -335,7 +335,8 @@ static struct convert_driver {
 	char *clean;
 } *user_convert, **user_convert_tail;
 
-static int read_convert_config(const char *var, const char *value)
+static int read_convert_config(const char *var, const char *value,
+				void *cb_data)
 {
 	const char *ep, *name;
 	int namelen;
@@ -402,7 +403,7 @@ static void setup_convert_check(struct git_attr_check *check)
 		attr_ident = git_attr("ident", 5);
 		attr_filter = git_attr("filter", 6);
 		user_convert_tail = &user_convert;
-		git_config(read_convert_config);
+		git_config(read_convert_config, NULL);
 	}
 	check[0].attr = attr_crlf;
 	check[1].attr = attr_ident;
diff --git a/daemon.c b/daemon.c
index 69afb60..35cbd09 100644
--- a/daemon.c
+++ b/daemon.c
@@ -284,7 +284,7 @@ struct daemon_service {
 static struct daemon_service *service_looking_at;
 static int service_enabled;
 
-static int git_daemon_config(const char *var, const char *value)
+static int git_daemon_config(const char *var, const char *value, void *cb_data)
 {
 	if (!prefixcmp(var, "daemon.") &&
 	    !strcmp(var + 7, service_looking_at->config_name)) {
@@ -334,7 +334,7 @@ static int run_service(struct interp *itable, struct daemon_service *service)
 	if (service->overridable) {
 		service_looking_at = service;
 		service_enabled = -1;
-		git_config(git_daemon_config);
+		git_config(git_daemon_config, NULL);
 		if (0 <= service_enabled)
 			enabled = service_enabled;
 	}
diff --git a/diff.c b/diff.c
index 0e26049..9f5e589 100644
--- a/diff.c
+++ b/diff.c
@@ -99,7 +99,7 @@ static int parse_lldiff_command(const char *var, const char *ep, const char *val
  * never be affected by the setting of diff.renames
  * the user happens to have in the configuration file.
  */
-int git_diff_ui_config(const char *var, const char *value)
+int git_diff_ui_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "diff.renamelimit")) {
 		diff_rename_limit_default = git_config_int(var, value);
@@ -131,7 +131,7 @@ int git_diff_ui_config(const char *var, const char *value)
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static char *quote_one(const char *str)
@@ -1760,7 +1760,7 @@ static const char *external_diff_attr(const char *name)
 
 			if (!user_diff_tail) {
 				user_diff_tail = &user_diff;
-				git_config(git_diff_ui_config);
+				git_config(git_diff_ui_config, NULL);
 			}
 			for (drv = user_diff; drv; drv = drv->next)
 				if (!strcmp(drv->name, value))
diff --git a/diff.h b/diff.h
index 63738c1..a715dff 100644
--- a/diff.h
+++ b/diff.h
@@ -161,7 +161,7 @@ extern int diff_scoreopt_parse(const char *opt);
 #define DIFF_SETUP_USE_CACHE		2
 #define DIFF_SETUP_USE_SIZE_CACHE	4
 
-extern int git_diff_ui_config(const char *var, const char *value);
+extern int git_diff_ui_config(const char *var, const char *value, void *cb_data);
 extern void diff_setup(struct diff_options *);
 extern int diff_opt_parse(struct diff_options *, const char **, int);
 extern int diff_setup_done(struct diff_options *);
diff --git a/dump-config.c b/dump-config.c
index 355920d..99dbeb6 100644
--- a/dump-config.c
+++ b/dump-config.c
@@ -4,7 +4,7 @@
 
 static const char dump_config_usage[] = "git-dump-config <dir>";
 
-static int dump_config(const char *var, const char *value)
+static int dump_config(const char *var, const char *value, void *cb_data)
 {
 	packet_write(1, "%s", var);
 	packet_write(1, "%s", value);
@@ -22,7 +22,7 @@ int main(int argc, char **argv)
 	if (!enter_repo(dir, 0))
 		die("'%s': unable to chdir or not a git archive", dir);
 
-	git_config(dump_config);
+	git_config(dump_config, NULL);
 	packet_flush(1);
 
 	return 0;
diff --git a/fast-import.c b/fast-import.c
index ffa00fd..599d045 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -2009,7 +2009,7 @@ int main(int argc, const char **argv)
 {
 	int i, show_stats = 1;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	alloc_objects(object_entry_alloc);
 	strbuf_init(&command_buf);
 	atom_table = xcalloc(atom_table_sz, sizeof(struct atom_str*));
diff --git a/fetch-pack.c b/fetch-pack.c
index aa59043..c7d24fe 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -646,7 +646,7 @@ static int remove_duplicates(int nr_heads, char **heads)
 	return dst;
 }
 
-static int fetch_pack_config(const char *var, const char *value)
+static int fetch_pack_config(const char *var, const char *value, void *cb_data)
 {
 	if (strcmp(var, "fetch.unpacklimit") == 0) {
 		fetch_unpack_limit = git_config_int(var, value);
@@ -658,7 +658,7 @@ static int fetch_pack_config(const char *var, const char *value)
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static struct lock_file lock;
@@ -672,7 +672,7 @@ int main(int argc, char **argv)
 	struct stat st;
 
 	setup_git_directory();
-	git_config(fetch_pack_config);
+	git_config(fetch_pack_config, NULL);
 
 	if (0 <= transfer_unpack_limit)
 		unpack_limit = transfer_unpack_limit;
diff --git a/git.c b/git.c
index 29b55a1..1a57934 100644
--- a/git.c
+++ b/git.c
@@ -87,7 +87,7 @@ static int handle_options(const char*** argv, int* argc)
 static const char *alias_command;
 static char *alias_string;
 
-static int git_alias_config(const char *var, const char *value)
+static int git_alias_config(const char *var, const char *value, void *cb_data)
 {
 	if (!prefixcmp(var, "alias.") && !strcmp(var + 6, alias_command)) {
 		alias_string = xstrdup(value);
@@ -158,7 +158,7 @@ static int handle_alias(int *argcp, const char ***argv)
 	subdir = setup_git_directory_gently(&nongit);
 
 	alias_command = (*argv)[0];
-	git_config(git_alias_config);
+	git_config(git_alias_config, NULL);
 	if (alias_string) {
 		if (alias_string[0] == '!') {
 			trace_printf("trace: alias to shell cmd: %s => %s\n",
diff --git a/http-fetch.c b/http-fetch.c
index 53fb2a9..bc87d4b 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -996,7 +996,7 @@ int main(int argc, const char **argv)
 	int rc = 0;
 
 	setup_git_directory();
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	while (arg < argc && argv[arg][0] == '-') {
 		if (argv[arg][1] == 't') {
diff --git a/http.c b/http.c
index c8237cb..fd359bf 100644
--- a/http.c
+++ b/http.c
@@ -97,7 +97,7 @@ static void process_curl_messages(void)
 }
 #endif
 
-static int http_options(const char *var, const char *value)
+static int http_options(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp("http.sslverify", var)) {
 		if (curl_ssl_verify == -1) {
@@ -164,7 +164,7 @@ static int http_options(const char *var, const char *value)
 	}
 
 	/* Fall back on the default ones */
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static CURL* get_curl_handle(void)
@@ -252,7 +252,7 @@ void http_init(void)
 	if (low_speed_time != NULL)
 		curl_low_speed_time = strtol(low_speed_time, NULL, 10);
 
-	git_config(http_options);
+	git_config(http_options, NULL);
 
 	if (curl_ssl_verify == -1)
 		curl_ssl_verify = 1;
diff --git a/imap-send.c b/imap-send.c
index 4283a4a..129776c 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -1255,7 +1255,7 @@ static imap_server_conf_t server =
 static char *imap_folder;
 
 static int
-git_imap_config(const char *key, const char *val)
+git_imap_config(const char *key, const char *val, void *cb_data)
 {
 	char imap_key[] = "imap.";
 
@@ -1300,7 +1300,7 @@ main(int argc, char **argv)
 	/* init the random number generator */
 	arc4_init();
 
-	git_config( git_imap_config );
+	git_config(git_imap_config, NULL);
 
 	if (!imap_folder) {
 		fprintf( stderr, "no imap store specified\n" );
diff --git a/local-fetch.c b/local-fetch.c
index 4b650ef..23d2cbe 100644
--- a/local-fetch.c
+++ b/local-fetch.c
@@ -204,7 +204,7 @@ int main(int argc, const char **argv)
 	int arg = 1;
 
 	setup_git_directory();
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	while (arg < argc && argv[arg][0] == '-') {
 		if (argv[arg][1] == 't')
diff --git a/merge-recursive.c b/merge-recursive.c
index 8f72b2c..f774342 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -851,7 +851,7 @@ static int ll_ext_merge(const struct ll_merge_driver *fn,
 static struct ll_merge_driver *ll_user_merge, **ll_user_merge_tail;
 static const char *default_ll_merge;
 
-static int read_merge_config(const char *var, const char *value)
+static int read_merge_config(const char *var, const char *value, void *cb_data)
 {
 	struct ll_merge_driver *fn;
 	const char *ep, *name;
@@ -940,7 +940,7 @@ static void initialize_ll_merge(void)
 	if (ll_user_merge_tail)
 		return;
 	ll_user_merge_tail = &ll_user_merge;
-	git_config(read_merge_config);
+	git_config(read_merge_config, NULL);
 }
 
 static const struct ll_merge_driver *find_ll_merge_driver(const char *merge_attr)
@@ -1696,13 +1696,13 @@ static struct commit *get_ref(const char *ref)
 	return (struct commit *)object;
 }
 
-static int merge_config(const char *var, const char *value)
+static int merge_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcasecmp(var, "merge.verbosity")) {
 		verbosity = git_config_int(var, value);
 		return 0;
 	}
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 int main(int argc, char *argv[])
@@ -1723,7 +1723,7 @@ int main(int argc, char *argv[])
 			subtree_merge = 1;
 	}
 
-	git_config(merge_config);
+	git_config(merge_config, NULL);
 	if (getenv("GIT_MERGE_VERBOSITY"))
 		verbosity = strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
 
diff --git a/receive-pack.c b/receive-pack.c
index d3c422b..feab9a4 100644
--- a/receive-pack.c
+++ b/receive-pack.c
@@ -18,7 +18,7 @@ static int report_status;
 static char capabilities[] = " report-status delete-refs ";
 static int capabilities_sent;
 
-static int receive_pack_config(const char *var, const char *value)
+static int receive_pack_config(const char *var, const char *value, void *cb_data)
 {
 	if (strcmp(var, "receive.denynonfastforwards") == 0) {
 		deny_non_fast_forwards = git_config_bool(var, value);
@@ -35,7 +35,7 @@ static int receive_pack_config(const char *var, const char *value)
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static int show_ref(const char *path, const unsigned char *sha1, int flag, void *cb_data)
@@ -484,7 +484,7 @@ int main(int argc, char **argv)
 	if (is_repository_shallow())
 		die("attempt to push into a shallow repository");
 
-	git_config(receive_pack_config);
+	git_config(receive_pack_config, NULL);
 
 	if (0 <= transfer_unpack_limit)
 		unpack_limit = transfer_unpack_limit;
diff --git a/remote.c b/remote.c
index 46fe8d9..4b8c872 100644
--- a/remote.c
+++ b/remote.c
@@ -148,7 +148,7 @@ static char *default_remote_name = NULL;
 static const char *current_branch = NULL;
 static int current_branch_len = 0;
 
-static int handle_config(const char *key, const char *value)
+static int handle_config(const char *key, const char *value, void *cb_data)
 {
 	const char *name;
 	const char *subkey;
@@ -215,7 +215,7 @@ static void read_config(void)
 		current_branch = head_ref + strlen("refs/heads/");
 		current_branch_len = strlen(current_branch);
 	}
-	git_config(handle_config);
+	git_config(handle_config, NULL);
 }
 
 static struct refspec *parse_ref_spec(int nr_refspec, const char **refspec)
diff --git a/send-pack.c b/send-pack.c
index 2c0b19b..3956ba0 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -372,7 +372,7 @@ int main(int argc, char **argv)
 	struct remote *remote = NULL;
 
 	setup_git_directory();
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	argv++;
 	for (i = 1; i < argc; i++, argv++) {
diff --git a/setup.c b/setup.c
index a45ea83..d3487aa 100644
--- a/setup.c
+++ b/setup.c
@@ -270,7 +270,8 @@ int git_config_perm(const char *var, const char *value)
 	return git_config_bool(var, value);
 }
 
-int check_repository_format_version(const char *var, const char *value)
+int check_repository_format_version(const char *var, const char *value,
+				    void *cb_data)
 {
        if (strcmp(var, "core.repositoryformatversion") == 0)
                repository_format_version = git_config_int(var, value);
@@ -281,7 +282,7 @@ int check_repository_format_version(const char *var, const char *value)
 
 int check_repository_format(void)
 {
-	git_config(check_repository_format_version);
+	git_config(check_repository_format_version, NULL);
 	if (GIT_REPO_VERSION < repository_format_version)
 		die ("Expected git repo version <= %d, found %d",
 		     GIT_REPO_VERSION, repository_format_version);
diff --git a/ssh-fetch.c b/ssh-fetch.c
index bdf51a7..ceb8b7d 100644
--- a/ssh-fetch.c
+++ b/ssh-fetch.c
@@ -125,7 +125,7 @@ int main(int argc, char **argv)
 	if (!prog) prog = "git-ssh-upload";
 
 	setup_git_directory();
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	while (arg < argc && argv[arg][0] == '-') {
 		if (argv[arg][1] == 't') {
diff --git a/unpack-file.c b/unpack-file.c
index 25c56b3..ade9926 100644
--- a/unpack-file.c
+++ b/unpack-file.c
@@ -33,7 +33,7 @@ int main(int argc, char **argv)
 		die("Not a valid object name %s", argv[1]);
 
 	setup_git_directory();
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	puts(create_temp_file(sha1));
 	return 0;
diff --git a/var.c b/var.c
index e585e59..8ec0fdd 100644
--- a/var.c
+++ b/var.c
@@ -39,13 +39,13 @@ static const char *read_var(const char *var)
 	return val;
 }
 
-static int show_config(const char *var, const char *value)
+static int show_config(const char *var, const char *value, void *cb_data)
 {
 	if (value)
 		printf("%s=%s\n", var, value);
 	else
 		printf("%s\n", var);
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 int main(int argc, char **argv)
@@ -59,11 +59,11 @@ int main(int argc, char **argv)
 	val = NULL;
 
 	if (strcmp(argv[1], "-l") == 0) {
-		git_config(show_config);
+		git_config(show_config, NULL);
 		list_vars();
 		return 0;
 	}
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	val = read_var(argv[1]);
 	if (!val)
 		usage(var_usage);
diff --git a/wt-status.c b/wt-status.c
index 4bfe8f1..af6e39d 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -349,7 +349,7 @@ void wt_status_print(struct wt_status *s)
 	}
 }
 
-int git_status_config(const char *k, const char *v)
+int git_status_config(const char *k, const char *v, void *cb_data)
 {
 	if (!strcmp(k, "status.color") || !strcmp(k, "color.status")) {
 		wt_status_use_color = git_config_colorbool(k, v);
@@ -365,5 +365,5 @@ int git_status_config(const char *k, const char *v)
 		excludes_file = xstrdup(v);
 		return 0;
 	}
-	return git_default_config(k, v);
+	return git_default_config(k, v, NULL);
 }
diff --git a/wt-status.h b/wt-status.h
index cfea4ae..242e9c5 100644
--- a/wt-status.h
+++ b/wt-status.h
@@ -21,7 +21,7 @@ struct wt_status {
 	int workdir_untracked;
 };
 
-int git_status_config(const char *var, const char *value);
+int git_status_config(const char *var, const char *value, void *cb_data);
 void wt_status_prepare(struct wt_status *s);
 void wt_status_print(struct wt_status *s);
 
-- 
1.5.2.784.g5532e

[PATCH 15/22] make redirecting stdout to /dev/null available via run_command_v_opt

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 run-command.c |    1 +
 run-command.h |    9 +++++----
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/run-command.c b/run-command.c
index 806af46..d5b8ba2 100644
--- a/run-command.c
+++ b/run-command.c
@@ -150,6 +150,7 @@ static void prepare_run_command_v_opt(struct child_process *cmd,
 	memset(cmd, 0, sizeof(*cmd));
 	cmd->argv = argv;
 	cmd->no_stdin = opt & RUN_COMMAND_NO_STDIN ? 1 : 0;
+	cmd->no_stdout = opt & RUN_COMMAND_NO_STDOUT ? 1 : 0;
 	cmd->git_cmd = opt & RUN_GIT_CMD ? 1 : 0;
 	cmd->stdout_to_stderr = opt & RUN_COMMAND_STDOUT_TO_STDERR ? 1 : 0;
 	cmd->clear_git_env = opt & RUN_COMMAND_CLEAR_GIT_ENV ? 1 : 0;
diff --git a/run-command.h b/run-command.h
index 7724118..e3e897a 100644
--- a/run-command.h
+++ b/run-command.h
@@ -30,10 +30,11 @@ int start_command(struct child_process *);
 int finish_command(struct child_process *);
 int run_command(struct child_process *);
 
-#define RUN_COMMAND_NO_STDIN 1
-#define RUN_GIT_CMD	     2	/*If this is to be git sub-command */
-#define RUN_COMMAND_STDOUT_TO_STDERR 4
-#define RUN_COMMAND_CLEAR_GIT_ENV	(1 << 3)
+#define RUN_COMMAND_NO_STDIN 		(1 << 0)
+#define RUN_COMMAND_NO_STDOUT 		(1 << 1)
+#define RUN_GIT_CMD			(1 << 2)	/* git sub-command */
+#define RUN_COMMAND_STDOUT_TO_STDERR 	(1 << 3)
+#define RUN_COMMAND_CLEAR_GIT_ENV	(1 << 4)
 int run_command_v_opt(const char **argv, int opt);
 int run_command_v_opt_cd(const char **argv, int opt, const char *dir);
 
-- 
1.5.2.784.g5532e

[PATCH 16/22] unpack-trees.c: optionally clone submodules for later checkout

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

When the --submodules option is specified and a submodule
to be checked out is not available locally, git-checkout will
search for submodule.<submodule>.url options in the remote
configuration and clone each submodule using the first url that
it can use from the local site.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 Documentation/config.txt |    3 +
 Makefile                 |    4 +-
 submodules.c             |  211 ++++++++++++++++++++++++++++++++++++++++++++++
 submodules.h             |    7 ++
 unpack-trees.c           |   50 +++++++++++
 5 files changed, 273 insertions(+), 2 deletions(-)
 create mode 100644 submodules.c
 create mode 100644 submodules.h

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 5045443..2a2e142 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -615,6 +615,9 @@ showbranch.default::
 	The default set of branches for gitlink:git-show-branch[1].
 	See gitlink:git-show-branch[1].
 
+submodule.<submodule>.url
+	The URL of a submodule.  See gitlink:git-clone[1].
+
 tar.umask::
 	By default, gitlink:git-tar-tree[1] sets file and directories modes
 	to 0666 or 0777. While this is both useful and acceptable for projects
diff --git a/Makefile b/Makefile
index 1fa1896..39bf2d4 100644
--- a/Makefile
+++ b/Makefile
@@ -298,7 +298,7 @@ LIB_H = \
 	run-command.h strbuf.h tag.h tree.h git-compat-util.h revision.h \
 	tree-walk.h log-tree.h dir.h path-list.h unpack-trees.h builtin.h \
 	utf8.h reflog-walk.h patch-ids.h attr.h decorate.h progress.h \
-	mailmap.h remote.h
+	mailmap.h remote.h submodules.h
 
 DIFF_OBJS = \
 	diff.o diff-lib.o diffcore-break.o diffcore-order.o \
@@ -321,7 +321,7 @@ LIB_OBJS = \
 	alloc.o merge-file.o path-list.o help.o unpack-trees.o $(DIFF_OBJS) \
 	color.o wt-status.o archive-zip.o archive-tar.o shallow.o utf8.o \
 	convert.o attr.o decorate.o progress.o mailmap.o symlinks.o remote.o \
-	$(HTTP_CONFIG_OBJ)
+	$(HTTP_CONFIG_OBJ) submodules.o
 
 BUILTIN_OBJS = \
 	builtin-add.o \
diff --git a/submodules.c b/submodules.c
new file mode 100644
index 0000000..44c0f2c
--- /dev/null
+++ b/submodules.c
@@ -0,0 +1,211 @@
+#include "cache.h"
+#include "refs.h"
+#include "submodules.h"
+#include "run-command.h"
+
+int is_checkedout_submodule(const char *path)
+{
+	unsigned char sha1[20];
+	return resolve_gitlink_ref(path, "HEAD", sha1) == 0;
+}
+
+struct key_val_list {
+	struct key_val_list *next;
+	char *key;
+	char *val;
+};
+
+static void free_key_val_list(struct key_val_list *list)
+{
+	struct key_val_list *next;
+	for (; list; list = next) {
+		next = list->next;
+		free(list->key);
+		free(list->val);
+		free(list);
+	}
+}
+
+static struct key_val_list *find_key_val_list(struct key_val_list *list,
+						const char *key)
+{
+	while (list && strcmp(list->key, key))
+		list = list->next;
+	return list;
+}
+
+struct collect_urls_data {
+	struct key_val_list **next;
+
+	const char *type;
+};
+
+static int collect_urls(const char *var, const char *value, void *cb_data)
+{
+	struct collect_urls_data *cb = (struct collect_urls_data*)cb_data;
+	int typelen = strlen(cb->type);
+	int len;
+	char *doturl;
+	struct key_val_list *item;
+
+	if (prefixcmp(var, cb->type))
+		return 0;
+
+	if (var[typelen] != '.')
+		return 0;
+
+	var += typelen+1;
+
+	doturl = strrchr(var, '.');
+	if (!doturl || strcmp(doturl, ".url"))
+		return 0;
+
+	len = doturl-var;
+	if (len <= 0)
+		return 0;
+
+	item = xmalloc(sizeof(struct key_val_list));
+	item->key = xmalloc(len+1);
+	memcpy(item->key, var, len);
+	item->key[len] = 0;
+	item->val = xstrdup(value);
+	item->next = NULL;
+	*cb->next = item;
+	cb->next = &item->next;
+
+	return 0;
+}
+
+static const char *local_URL(const char *remote, const char *url)
+{
+	static char local_url[PATH_MAX];
+
+	if (!prefixcmp(url, "https://"))
+		return url;
+
+	if (!prefixcmp(url, "http://"))
+		return url;
+
+	if (!prefixcmp(url, "ftp://"))
+		return url;
+
+	if (!prefixcmp(remote, "/"))
+		return url;
+
+	if (!prefixcmp(remote, "ssh://") && !prefixcmp(url, "/")) {
+		char *slash;
+		int len = strlen(url);
+
+		slash = strchr(remote+6, '/');
+		if (!slash || (slash-remote)+len+1 > sizeof(local_url))
+			return NULL;
+		memcpy(local_url, remote, slash-remote);
+		memcpy(local_url+(slash-remote), url, len+1);
+		return local_url;
+	}
+
+	return NULL;
+}
+
+static int fetch_submodule_urls(struct key_val_list **next_url)
+{
+	struct key_val_list *remotes = NULL;
+	struct collect_urls_data remotes_data = { &remotes, "remote" };
+	struct key_val_list *remote;
+	static char key[1024];
+
+	git_config(collect_urls, &remotes_data);
+	for (remote = remotes; remote; remote = remote->next) {
+		struct key_val_list *submodules = NULL;
+		struct collect_urls_data submodules_data =
+			{ &submodules, "submodule" };
+		struct key_val_list *submodule;
+		char *dest;
+
+		dest = xstrdup(remote->val);
+		git_config_from_remote(collect_urls, dest, &submodules_data);
+		free(dest);
+		for (submodule = submodules; submodule; submodule = submodule->next) {
+			const char *local_url;
+			struct key_val_list *item;
+
+			local_url = local_URL(remote->val, submodule->val);
+			if (!local_url)
+				continue;
+
+			if (snprintf(key, sizeof(key),
+			    "submodule.%s.url", submodule->key) > sizeof(key))
+				return error("submodule name too long");
+
+			git_config_set(key, local_url);
+
+			item = xmalloc(sizeof(struct key_val_list));
+			item->key = xstrdup(submodule->key);
+			item->val = xstrdup(local_url);
+			item->next = NULL;
+			*next_url = item;
+			next_url = &item->next;
+		}
+
+		free_key_val_list(submodules);
+	}
+
+	free_key_val_list(remotes);
+
+	return 0;
+}
+
+int clone_submodule(const char *submodule)
+{
+	struct key_val_list *submodules = NULL;
+	struct collect_urls_data submodules_data = { &submodules, "submodule" };
+	struct key_val_list *item;
+	char *path;
+	int err;
+	const char *args[10];
+	int argc;
+
+	git_config(collect_urls, &submodules_data);
+	item = find_key_val_list(submodules, submodule);
+	if (!item) {
+		err = fetch_submodule_urls(submodules_data.next);
+		if (err)
+			return err;
+		item = find_key_val_list(*submodules_data.next, submodule);
+		if (!item)
+			return error("don't know where to get submodule '%s'",
+					submodule);
+	}
+
+	path = git_path("submodules/%s", submodule);
+
+	argc = 0;
+	args[argc++] = "clone";
+	args[argc++] = "--submodules";
+	args[argc++] = "-n";
+	args[argc++] = item->val;
+	args[argc++] = path;
+	args[argc] = NULL;
+
+	err = run_command_v_opt(args, RUN_GIT_CMD|RUN_COMMAND_CLEAR_GIT_ENV);
+
+	path = git_path("submodules/%s/.git", submodule);
+
+	argc = 0;
+	args[argc++] = "update-ref";
+	args[argc++] = "--no-deref";
+	args[argc++] = "HEAD";
+	args[argc++] = "0000000000000000000000000000000000000000";
+	args[argc] = NULL;
+
+	if (!err)
+		err = run_command_v_opt_cd(args,
+				RUN_GIT_CMD|RUN_COMMAND_CLEAR_GIT_ENV, path);
+
+	if (err)
+		return error("failed to clone submodule '%s'", submodule);
+
+	free_key_val_list(submodules);
+
+	return 0;
+}
diff --git a/submodules.h b/submodules.h
new file mode 100644
index 0000000..bf3f118
--- /dev/null
+++ b/submodules.h
@@ -0,0 +1,7 @@
+#ifndef SUBMODULES_H
+#define SUBMODULES_H
+
+int is_checkedout_submodule(const char *path);
+int clone_submodule(const char *submodule);
+
+#endif
diff --git a/unpack-trees.c b/unpack-trees.c
index d5e458d..ddefb51 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -6,6 +6,7 @@
 #include "unpack-trees.h"
 #include "progress.h"
 #include "refs.h"
+#include "submodules.h"
 
 #define DBRT_DEBUG 1
 
@@ -804,6 +805,35 @@ int threeway_merge(struct cache_entry **stages,
 	return count;
 }
 
+static int ensure_submodule(struct cache_entry *ce,
+			    struct unpack_trees_options *o)
+{
+	struct stat st;
+	char *path;
+
+	if (!ce)
+		return 0;
+
+	if (!S_ISGITLINK(ntohl(ce->ce_mode)))
+		return 0;
+
+	if (!is_checkedout_submodule(ce->name) && !o->submodules)
+		return 0;
+
+	path = mkpath("%s/.git", ce->name);
+	if (lstat(path, &st)) {
+		path = git_path("submodules/%s/.git", ce->name);
+		if (lstat(path, &st)) {
+			if (clone_submodule(ce->name))
+				return -1;
+		}
+	}
+
+	/* Now check that the commit is available and fetch if needed */
+
+	return 0;
+}
+
 /*
  * Two-way merge.
  *
@@ -829,6 +859,17 @@ int twoway_merge(struct cache_entry **src,
 	if (newtree == o->df_conflict_entry)
 		newtree = NULL;
 
+	if (o->update) {
+		int err;
+		err = ensure_submodule(current, o);
+		if (!err)
+			err = ensure_submodule(oldtree, o);
+		if (!err)
+			err = ensure_submodule(newtree, o);
+		if (err)
+			return err;
+	}
+
 	if (current) {
 		if ((!oldtree && !newtree) || /* 4 and 5 */
 		    (!oldtree && newtree &&
@@ -905,6 +946,15 @@ int oneway_merge(struct cache_entry **src,
 		return error("Cannot do a oneway merge of %d trees",
 			     o->merge_size);
 
+	if (o->update) {
+		int err;
+		err = ensure_submodule(old, o);
+		if (!err)
+			err = ensure_submodule(a, o);
+		if (err)
+			return err;
+	}
+
 	if (!a)
 		return deleted_entry(old, old, o);
 	if (old && same(old, a)) {
-- 
1.5.2.784.g5532e

[PATCH 17/22] entry.c: optionally checkout newly cloned submodules

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 entry.c      |   23 ++++++++++++++++++++---
 submodules.c |   43 +++++++++++++++++++++++++++++++++++++++++++
 submodules.h |    1 +
 3 files changed, 64 insertions(+), 3 deletions(-)

diff --git a/entry.c b/entry.c
index 7ba2241..f3e0c59 100644
--- a/entry.c
+++ b/entry.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "blob.h"
 #include "run-command.h"
+#include "submodules.h"
 
 static void create_directories(const char *path, const struct checkout *state)
 {
@@ -82,7 +83,7 @@ static int checkout_submodule(struct cache_entry *ce, const char *path, const st
 	int argc;
 	int err;
 
-	if (!state->submodules)
+	if (!state->submodules && !is_checkedout_submodule(ce->name))
 		return 0;
 
 	argc = 0;
@@ -101,10 +102,25 @@ static int checkout_submodule(struct cache_entry *ce, const char *path, const st
 	return 0;
 }
 
+static int write_submodule(struct cache_entry *ce, char *path, const struct checkout *state)
+{
+	if (mkdir(path, 0777) < 0)
+		return error("git-checkout-index: cannot create subproject directory %s", path);
+
+	if (!state->submodules)
+		return 0;
+
+	if (attach_submodule(ce->name))
+		return -1;
+
+	return checkout_submodule(ce, path, state);
+}
+
 static int write_entry(struct cache_entry *ce, char *path, const struct checkout *state, int to_tempfile)
 {
 	int fd;
 	long wrote;
+	int err;
 
 	switch (ntohl(ce->ce_mode) & S_IFMT) {
 		char *buf, *new;
@@ -174,8 +190,9 @@ static int write_entry(struct cache_entry *ce, char *path, const struct checkout
 	case S_IFGITLINK:
 		if (to_tempfile)
 			return error("git-checkout-index: cannot create temporary subproject %s", path);
-		if (mkdir(path, 0777) < 0)
-			return error("git-checkout-index: cannot create subproject directory %s", path);
+		err = write_submodule(ce, path, state);
+		if (err)
+			return err;
 		break;
 	default:
 		return error("git-checkout-index: unknown file mode for %s", path);
diff --git a/submodules.c b/submodules.c
index 44c0f2c..325de33 100644
--- a/submodules.c
+++ b/submodules.c
@@ -209,3 +209,46 @@ int clone_submodule(const char *submodule)
 
 	return 0;
 }
+
+static const char *relativize_path(const char *path, const char *dest)
+{
+	static char relative_path[PATH_MAX];
+	int slashes;
+	const char *cp;
+	char *rp;
+
+	if (path[0] == '/')
+		return path;
+
+	for (slashes = 0, cp = strchr(dest, '/'); cp; cp = strchr(cp, '/')) {
+		++slashes;
+		while (*cp == '/')
+			++cp;
+	}
+	if (3 * slashes + strlen(path) + 1 > sizeof(relative_path))
+		die("path too long");
+
+	rp = relative_path;
+	while (slashes--) {
+		memcpy(rp, "../", 3);
+		rp += 3;
+	}
+	strcpy(rp, path);
+
+	return relative_path;
+}
+
+int attach_submodule(const char *submodule)
+{
+	struct stat st;
+	const char *submodule_dir, *dest;
+
+	submodule_dir = git_path("submodules/%s/.git", submodule);
+	if (lstat(submodule_dir, &st))
+		return error("submodule '%s' unavailable", submodule);
+
+	dest = mkpath("%s/.git", submodule);
+	submodule_dir = relativize_path(submodule_dir, dest);
+
+	return symlink(submodule_dir, dest);
+}
diff --git a/submodules.h b/submodules.h
index bf3f118..57432bb 100644
--- a/submodules.h
+++ b/submodules.h
@@ -3,5 +3,6 @@
 
 int is_checkedout_submodule(const char *path);
 int clone_submodule(const char *submodule);
+int attach_submodule(const char *submodule);
 
 #endif
-- 
1.5.2.784.g5532e

Re: [PATCH 17/22] entry.c: optionally checkout newly cloned submodules

[Johannes Sixt]

skimo@liacs.nl wrote:
> +static const char *relativize_path(const char *path, const char *dest)
> +{
> ...
> +}

I really get a head ache when I try to understand the purpose of this
function. A comment or two telling what it's trying to do would really
be helpful. But...

> +int attach_submodule(const char *submodule)
> +{
> +       struct stat st;
> +       const char *submodule_dir, *dest;
> +
> +       submodule_dir = git_path("submodules/%s/.git", submodule);
> +       if (lstat(submodule_dir, &st))
> +               return error("submodule '%s' unavailable", submodule);
> +
> +       dest = mkpath("%s/.git", submodule);
> +       submodule_dir = relativize_path(submodule_dir, dest);
> +
> +       return symlink(submodule_dir, dest);
> +}

I don't like this. Symlinks are a major nuisance on Windows (MinGW
port). Can't we just have this repo itself at this place instead of the
symlink?

-- Hannes

[PATCH 18/22] git-clone: add --submodules for cloning submodules

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

When the --submodules option is specified, git-clone will search
for submodule.<submodule>.url options in the remote configuration
and clone each submodule using the first url that it can use from
the local site.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 Documentation/git-clone.txt |    6 +++++-
 git-clone.sh                |   16 ++++++++++++++--
 2 files changed, 19 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 644bf12..565155b 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -11,7 +11,7 @@ SYNOPSIS
 [verse]
 'git-clone' [--template=<template_directory>] [-l [-s]] [-q] [-n] [--bare]
 	  [-o <name>] [-u <upload-pack>] [--reference <repository>]
-	  [--depth <depth>] <repository> [<directory>]
+	  [--depth <depth>] [--submodules] <repository> [<directory>]
 
 DESCRIPTION
 -----------
@@ -105,6 +105,10 @@ OPTIONS
 	with a long history, and would want to send in a fixes
 	as patches.
 
+--submodules::
+	Clone submodules specified in (remote) configuration parameters
+	submodule.<submodule>.url.
+
 <repository>::
 	The (possibly remote) repository to clone from.  It can
 	be any URL git-fetch supports.
diff --git a/git-clone.sh b/git-clone.sh
index fdd354f..a51b887 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -14,7 +14,7 @@ die() {
 }
 
 usage() {
-	die "Usage: $0 [--template=<template_directory>] [--reference <reference-repo>] [--bare] [-l [-s]] [-q] [-u <upload-pack>] [--origin <name>] [--depth <n>] [-n] <repo> [<dir>]"
+	die "Usage: $0 [--template=<template_directory>] [--reference <reference-repo>] [--bare] [-l [-s]] [-q] [-u <upload-pack>] [--origin <name>] [--depth <n>] [-n] [--submodules] <repo> [<dir>]"
 }
 
 get_repo_base() {
@@ -88,6 +88,7 @@ origin_override=
 use_separate_remote=t
 depth=
 no_progress=
+submodules=
 test -t 1 || no_progress=--no-progress
 while
 	case "$#,$1" in
@@ -138,6 +139,8 @@ while
 	*,--depth)
 		shift
 		depth="--depth=$1";;
+	*,--su|*,--sub|*,--subm|*,--submo|*,--submod|*,--submodu|*,--submodul|\
+	*,--submodule|*,--submodules) submodules="--submodules" ;;
 	*,-*) usage ;;
 	*) break ;;
 	esac
@@ -156,6 +159,10 @@ then
 	then
 		die '--bare and --origin $origin options are incompatible.'
 	fi
+	if test -n "$submodules"
+	then
+		die '--bare and --submodules origin options are incompatible.'
+	fi
 	no_checkout=yes
 	use_separate_remote=
 fi
@@ -401,10 +408,15 @@ then
 		git-config branch."$head_points_at".merge "refs/heads/$head_points_at"
 	esac
 
+	if test -n "$submodules"
+	then
+		git-config core.submodules true
+	fi
+
 	case "$no_checkout" in
 	'')
 		test "z$quiet" = z -a "z$no_progress" = z && v=-v || v=
-		git-read-tree -m -u $v HEAD HEAD
+		git-read-tree -m -u $v $submodules HEAD HEAD
 	esac
 fi
 rm -f "$GIT_DIR/CLONE_HEAD" "$GIT_DIR/REMOTE_HEAD"
-- 
1.5.2.784.g5532e

[PATCH 19/22] test for simple submodule checkout support

[skimo]

From: Martin Waitz <tali@admingilde.org>

Signed-off-by: Martin Waitz <tali@admingilde.org>
Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 t/t3041-subprojects-checkout.sh |   39 +++++++++++++++++++++++++++++++++++++++
 1 files changed, 39 insertions(+), 0 deletions(-)
 create mode 100755 t/t3041-subprojects-checkout.sh

diff --git a/t/t3041-subprojects-checkout.sh b/t/t3041-subprojects-checkout.sh
new file mode 100755
index 0000000..4b3cea9
--- /dev/null
+++ b/t/t3041-subprojects-checkout.sh
@@ -0,0 +1,39 @@
+#!/bin/sh
+
+test_description='submodule checkout'
+. ./test-lib.sh
+
+test_expect_success 'submodule creation' \
+    '(mkdir A && cd A &&
+      git init &&
+      echo 1 > a &&
+      git add a &&
+      git commit -m "create submodule" || exit $? )'
+
+test_expect_success 'Super module creation' \
+    'git add A &&
+     git commit -m "supermodule creation" &&
+     git branch one'
+
+test_expect_success 'submodule change' \
+    '(cd A &&
+      echo 2 > a &&
+      git add a &&
+      git commit -m "create submodule" || exit $? )'
+
+test_expect_success 'supermodule change' \
+    'git add A &&
+     git commit -m "supermodule creation"'
+
+test_expect_success 'supermodule switching branch' \
+    'git checkout one &&
+     echo 1 > expected &&
+     git diff expected A/a'
+
+test_expect_success 'supermodule reset' \
+    'git reset --hard master &&
+     echo 2 > expected &&
+     git diff expected A/a'
+
+
+test_done
-- 
1.5.2.784.g5532e

[PATCH 20/22] checkout_submodule: checkout submodule on forced checkout of submodule dir

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 entry.c |   19 ++++++++++++++++++-
 1 files changed, 18 insertions(+), 1 deletions(-)

diff --git a/entry.c b/entry.c
index f3e0c59..9c093ac 100644
--- a/entry.c
+++ b/entry.c
@@ -82,10 +82,20 @@ static int checkout_submodule(struct cache_entry *ce, const char *path, const st
 	const char *args[10];
 	int argc;
 	int err;
+	struct stat st;
+	char *gitdir;
 
-	if (!state->submodules && !is_checkedout_submodule(ce->name))
+	if (!state->force && !is_checkedout_submodule(ce->name))
 		return 0;
 
+	gitdir = mkpath("%s/.git", ce->name);
+	if (lstat(gitdir, &st)) {
+		if (clone_submodule(ce->name))
+			return -1;
+		if (attach_submodule(ce->name))
+			return -1;
+	}
+
 	argc = 0;
 	args[argc++] = "checkout";
 	if (state->force)
@@ -220,6 +230,13 @@ int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *t
 
 	if (!lstat(path, &st)) {
 		unsigned changed = ce_match_stat(ce, &st, 1);
+		/*
+		 *  If submodule has not been checked out, then force
+		 *  forces a check-out.
+		 */
+		if (state->force && S_ISGITLINK(ntohl(ce->ce_mode)) &&
+		    !is_checkedout_submodule(ce->name))
+			changed = 1;
 		if (!changed)
 			return 0;
 		if (!state->force) {
-- 
1.5.2.784.g5532e

[PATCH 21/22] run-command: optionally redirect stderr to /dev/null

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 run-command.c |    4 ++++
 run-command.h |    8 +++++---
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/run-command.c b/run-command.c
index d5b8ba2..a620be3 100644
--- a/run-command.c
+++ b/run-command.c
@@ -73,6 +73,9 @@ int start_command(struct child_process *cmd)
 			close(cmd->out);
 		}
 
+		if (cmd->no_stderr)
+			dup_devnull(2);
+
 		if (cmd->dir && chdir(cmd->dir))
 			die("exec %s: cd to %s failed (%s)", cmd->argv[0],
 			    cmd->dir, strerror(errno));
@@ -151,6 +154,7 @@ static void prepare_run_command_v_opt(struct child_process *cmd,
 	cmd->argv = argv;
 	cmd->no_stdin = opt & RUN_COMMAND_NO_STDIN ? 1 : 0;
 	cmd->no_stdout = opt & RUN_COMMAND_NO_STDOUT ? 1 : 0;
+	cmd->no_stderr = opt & RUN_COMMAND_NO_STDERR ? 1 : 0;
 	cmd->git_cmd = opt & RUN_GIT_CMD ? 1 : 0;
 	cmd->stdout_to_stderr = opt & RUN_COMMAND_STDOUT_TO_STDERR ? 1 : 0;
 	cmd->clear_git_env = opt & RUN_COMMAND_CLEAR_GIT_ENV ? 1 : 0;
diff --git a/run-command.h b/run-command.h
index e3e897a..85770e0 100644
--- a/run-command.h
+++ b/run-command.h
@@ -21,6 +21,7 @@ struct child_process {
 	unsigned close_out:1;
 	unsigned no_stdin:1;
 	unsigned no_stdout:1;
+	unsigned no_stderr:1;
 	unsigned git_cmd:1; /* if this is to be git sub-command */
 	unsigned stdout_to_stderr:1;
 	unsigned clear_git_env:1;
@@ -32,9 +33,10 @@ int run_command(struct child_process *);
 
 #define RUN_COMMAND_NO_STDIN 		(1 << 0)
 #define RUN_COMMAND_NO_STDOUT 		(1 << 1)
-#define RUN_GIT_CMD			(1 << 2)	/* git sub-command */
-#define RUN_COMMAND_STDOUT_TO_STDERR 	(1 << 3)
-#define RUN_COMMAND_CLEAR_GIT_ENV	(1 << 4)
+#define RUN_COMMAND_NO_STDERR 		(1 << 2)
+#define RUN_GIT_CMD			(1 << 3)	/* git sub-command */
+#define RUN_COMMAND_STDOUT_TO_STDERR 	(1 << 4)
+#define RUN_COMMAND_CLEAR_GIT_ENV	(1 << 5)
 int run_command_v_opt(const char **argv, int opt);
 int run_command_v_opt_cd(const char **argv, int opt, const char *dir);
 
-- 
1.5.2.784.g5532e

[PATCH 22/22] ensure_submodule: fetch missing revisions

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 t/t3042-subprojects-fetch.sh |   46 ++++++++++++++++++++++++++++++++++++++++++
 unpack-trees.c               |   26 +++++++++++++++++++++++
 2 files changed, 72 insertions(+), 0 deletions(-)
 create mode 100755 t/t3042-subprojects-fetch.sh

diff --git a/t/t3042-subprojects-fetch.sh b/t/t3042-subprojects-fetch.sh
new file mode 100755
index 0000000..1acc1bd
--- /dev/null
+++ b/t/t3042-subprojects-fetch.sh
@@ -0,0 +1,46 @@
+#!/bin/sh
+
+test_description='submodule fetch'
+. ./test-lib.sh
+
+test_create_repo orig
+
+test_expect_success 'submodule creation' \
+    '(mkdir orig/A && cd orig/A &&
+      git init &&
+      echo 1 > a &&
+      git add a &&
+      git commit -m "create submodule" || exit $? )'
+
+test_expect_success 'supermodule creation' \
+    '(cd orig &&
+     git add A &&
+     git commit -m "supermodule creation" &&
+     git branch one &&
+     git config 'submodule.A.url' $(pwd)/A || exit $?)'
+
+test_expect_success 'clone supermodule' \
+    'git clone --submodules orig clone &&
+     echo 1 > expected &&
+     git diff expected clone/A/a'
+
+test_expect_success 'submodule change' \
+    '(cd orig/A &&
+      echo 2 > a &&
+      git add a &&
+      git commit -m "change submodule" || exit $? )'
+
+test_expect_success 'supermodule change' \
+    '(cd orig &&
+     git add A &&
+     git commit -m "supermodule change" || exit $? )'
+
+test_expect_success 'pull changes' \
+    '(cd clone &&
+      git pull || exit $? )'
+
+test_expect_success 'check pulled changes' \
+    'echo 2 > expected &&
+     git diff expected clone/A/a'
+
+test_done
diff --git a/unpack-trees.c b/unpack-trees.c
index ddefb51..1a76a29 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -7,6 +7,7 @@
 #include "progress.h"
 #include "refs.h"
 #include "submodules.h"
+#include "run-command.h"
 
 #define DBRT_DEBUG 1
 
@@ -810,6 +811,9 @@ static int ensure_submodule(struct cache_entry *ce,
 {
 	struct stat st;
 	char *path;
+	const char *argv_check[10];
+	const char *argv_fetch[] = {"fetch", NULL};
+	int argc;
 
 	if (!ce)
 		return 0;
@@ -826,10 +830,32 @@ static int ensure_submodule(struct cache_entry *ce,
 		if (lstat(path, &st)) {
 			if (clone_submodule(ce->name))
 				return -1;
+			/* may have been overwritten */
+			path = git_path("submodules/%s/.git", ce->name);
 		}
 	}
 
 	/* Now check that the commit is available and fetch if needed */
+	argc = 0;
+	argv_check[argc++] = "cat-file";
+	argv_check[argc++] = "-t";
+	argv_check[argc++] = sha1_to_hex(ce->sha1);
+	argv_check[argc] = NULL;
+
+	if (run_command_v_opt_cd(argv_check,
+	    RUN_GIT_CMD|RUN_COMMAND_CLEAR_GIT_ENV|RUN_COMMAND_NO_STDOUT|
+	    RUN_COMMAND_NO_STDERR, path)) {
+		if (run_command_v_opt_cd(argv_fetch,
+		    RUN_GIT_CMD|RUN_COMMAND_CLEAR_GIT_ENV, path))
+			return error("Unable to fetch submodule '%s'", ce->name);
+
+		if (run_command_v_opt_cd(argv_check,
+		    RUN_GIT_CMD|RUN_COMMAND_CLEAR_GIT_ENV|RUN_COMMAND_NO_STDOUT|
+		    RUN_COMMAND_NO_STDERR, path))
+			return error(
+			    "Unable to fetch revision %s for submodule '%s'",
+			    sha1_to_hex(ce->sha1), ce->name);
+	}
 
 	return 0;
 }
-- 
1.5.2.784.g5532e

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, skimo@liacs.nl wrote:

> This patch series implements a mechanism for cloning submodules.
> Each submodule is specified by a 'submodule.<submodule>.url'
> configuration option, e.g.,
> 
> bash-3.00$ ./git-config --remote=http://www.liacs.nl/~sverdool/isa.git --get-regexp 'submodule\..*\.url' 
> submodule.cloog.url /home/sverdool/public_html/cloog.git
> submodule.cloog.url http://www.liacs.nl/~sverdool/cloog.git

I am sorry to complain so late in the game, but I am not really interested 
in submodules. However, what you say here is not a task for git-config 
IMHO, but rather for git-remote.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Thu, 24 May 2007, skimo@liacs.nl wrote:
>
>> This patch series implements a mechanism for cloning submodules.
>> Each submodule is specified by a 'submodule.<submodule>.url'
>> configuration option, e.g.,
>> 
>> bash-3.00$ ./git-config --remote=http://www.liacs.nl/~sverdool/isa.git --get-regexp 'submodule\..*\.url' 
>> submodule.cloog.url /home/sverdool/public_html/cloog.git
>> submodule.cloog.url http://www.liacs.nl/~sverdool/cloog.git
>
> I am sorry to complain so late in the game, but I am not really interested 
> in submodules. However, what you say here is not a task for git-config 
> IMHO, but rather for git-remote.

Honestly speaking, I do not think people have no business
peeking into configuratoin remote repository has, and it would
be preferrable that supermodule Porcelain stuff does not rely on
that.

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Wed, May 23, 2007 at 05:50:42PM -0700, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> > On Thu, 24 May 2007, skimo@liacs.nl wrote:
> >> bash-3.00$ ./git-config --remote=http://www.liacs.nl/~sverdool/isa.git --get-regexp 'submodule\..*\.url' 
> >> submodule.cloog.url /home/sverdool/public_html/cloog.git
> >> submodule.cloog.url http://www.liacs.nl/~sverdool/cloog.git
> >
> > I am sorry to complain so late in the game, but I am not really interested 
> > in submodules. However, what you say here is not a task for git-config 
> > IMHO, but rather for git-remote.
> 
> Honestly speaking, I do not think people have no business
> peeking into configuratoin remote repository has, and it would
> be preferrable that supermodule Porcelain stuff does not rely on
> that.

Maybe there are too many negations in that sentence, but are you
saying it is ok to look into the remote configuration or not?

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Shawn O. Pearce]

Sven Verdoolaege <skimo@kotnet.org> wrote:
> On Wed, May 23, 2007 at 05:50:42PM -0700, Junio C Hamano wrote:
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> > >
> > > I am sorry to complain so late in the game, but I am not really interested 
> > > in submodules. However, what you say here is not a task for git-config 
> > > IMHO, but rather for git-remote.
> > 
> > Honestly speaking, I do not think people have no business
> > peeking into configuratoin remote repository has, and it would
> > be preferrable that supermodule Porcelain stuff does not rely on
> > that.
> 
> Maybe there are too many negations in that sentence, but are you
> saying it is ok to look into the remote configuration or not?

I think its OK to look at *your* .git/config to see what is
configured for the remotes, (e.g. git config remote.origin.url)
but it is NOT OK to look at the *remote*'s .git/config to see what
they have configured.

Why?  Their configuration is their configuration.  Who knows what
they have stored there.  Look at the recent cvsserver config options,
there's now a lot of information about the SQL database that backs
cvsserver.  That stuff shouldn't be public.

If you want to publish something for a client to fetch, it should
be done by publishing a Git object referenced by a proper ref:
blob, tree, commit, tag, take your pick.

-- 
Shawn.

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 03:29:45AM -0400, Shawn O. Pearce wrote:
> Why?  Their configuration is their configuration.  Who knows what
> they have stored there.  Look at the recent cvsserver config options,
> there's now a lot of information about the SQL database that backs
> cvsserver.  That stuff shouldn't be public.

For http:// or rsync:// it's public already; for ssh://, if you are allowed
to access the git repo, you can read the config as well; for git://,
we can dump a predefined selection of configuration variables.

> If you want to publish something for a client to fetch, it should
> be done by publishing a Git object referenced by a proper ref:
> blob, tree, commit, tag, take your pick.

You mean like a tag "submodules" that points to a text file
describing the submodules?
That's a bit of a pain to set up since you would want that
to be independent of your project.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, Sven Verdoolaege wrote:

> On Thu, May 24, 2007 at 03:29:45AM -0400, Shawn O. Pearce wrote:
> > Why?  Their configuration is their configuration.  Who knows what
> > they have stored there.  Look at the recent cvsserver config options,
> > there's now a lot of information about the SQL database that backs
> > cvsserver.  That stuff shouldn't be public.
> 
> For http:// or rsync:// it's public already; for ssh://, if you are 
> allowed to access the git repo, you can read the config as well; for 
> git://, we can dump a predefined selection of configuration variables.

I sanitized a once-public repo, which was _not_ updated via http-push (in 
which case you'd not see a meaningful config anyway), where the 
permissions prevented the config from being read.

> > If you want to publish something for a client to fetch, it should be 
> > done by publishing a Git object referenced by a proper ref: blob, 
> > tree, commit, tag, take your pick.
> 
> You mean like a tag "submodules" that points to a text file
> describing the submodules?
> That's a bit of a pain to set up since you would want that
> to be independent of your project.

I could imagine this to be another extension of ls-remote.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 10:41:30AM +0100, Johannes Schindelin wrote:
> On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > You mean like a tag "submodules" that points to a text file
> > describing the submodules?
> > That's a bit of a pain to set up since you would want that
> > to be independent of your project.
> 
> I could imagine this to be another extension of ls-remote.

You mean extending upload-pack ?  Junio mentioned this possibility as well.
This only solves the git:// and ssh:// case though.
What to do with the other protocols?

Also, I don't really understand why it would be less of a hack
to add it to ls-remote than to add it to git-config.
The latter seems more natural to me.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, Sven Verdoolaege wrote:

> On Thu, May 24, 2007 at 10:41:30AM +0100, Johannes Schindelin wrote:
> > On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > > You mean like a tag "submodules" that points to a text file
> > > describing the submodules?
> > > That's a bit of a pain to set up since you would want that
> > > to be independent of your project.
> > 
> > I could imagine this to be another extension of ls-remote.
> 
> You mean extending upload-pack ?  Junio mentioned this possibility as well.
> This only solves the git:// and ssh:// case though.
> What to do with the other protocols?

As we do for the refs: put it into .git/info/refs. This file is already 
meant to "cache" the output of ls-remote for dumb protocols.

> Also, I don't really understand why it would be less of a hack
> to add it to ls-remote than to add it to git-config.
> The latter seems more natural to me.

No, not at all. git-config is about the local data. It is _meant_ to be 
private. Things you can receive by ls-remote are _meant_ to be public.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 12:02:41PM +0100, Johannes Schindelin wrote:
> On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > On Thu, May 24, 2007 at 10:41:30AM +0100, Johannes Schindelin wrote:
> > > I could imagine this to be another extension of ls-remote.
> > 
> > You mean extending upload-pack ?  Junio mentioned this possibility as well.
> > This only solves the git:// and ssh:// case though.
> > What to do with the other protocols?
> 
> As we do for the refs: put it into .git/info/refs. This file is already 
> meant to "cache" the output of ls-remote for dumb protocols.

OK... so what should git-update-server-info put in this file for submodules?
Or, equivalently, what should be the output of ls-remote?

Right now its a list of pairs of revs(sha1) and refs.
For submodules we want a connection between a submodule name
and one or more URLs where the submodule can be found.
How are you going to squeeze that into info/refs without confusing
older versions of git?

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, Sven Verdoolaege wrote:

> On Thu, May 24, 2007 at 12:02:41PM +0100, Johannes Schindelin wrote:
> > On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > > On Thu, May 24, 2007 at 10:41:30AM +0100, Johannes Schindelin wrote:
> > > > I could imagine this to be another extension of ls-remote.
> > > 
> > > You mean extending upload-pack ?  Junio mentioned this possibility as well.
> > > This only solves the git:// and ssh:// case though.
> > > What to do with the other protocols?
> > 
> > As we do for the refs: put it into .git/info/refs. This file is already 
> > meant to "cache" the output of ls-remote for dumb protocols.
> 
> OK... so what should git-update-server-info put in this file for submodules?
> Or, equivalently, what should be the output of ls-remote?
> 
> Right now its a list of pairs of revs(sha1) and refs.
> For submodules we want a connection between a submodule name
> and one or more URLs where the submodule can be found.
> How are you going to squeeze that into info/refs without confusing
> older versions of git?

I wonder if the "ref^{blub}" syntax could be used for that: change "blub" 
to the URL, or "sub:URL" or something.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 12:31:33PM +0100, Johannes Schindelin wrote:
> On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > OK... so what should git-update-server-info put in this file for submodules?
> > Or, equivalently, what should be the output of ls-remote?
> > 
> > Right now its a list of pairs of revs(sha1) and refs.
> > For submodules we want a connection between a submodule name
> > and one or more URLs where the submodule can be found.
> > How are you going to squeeze that into info/refs without confusing
> > older versions of git?
> 
> I wonder if the "ref^{blub}" syntax could be used for that: change "blub" 
> to the URL, or "sub:URL" or something.

Just to be clear, would it look like the following?

e8a6e39ecfbd391a54b9c3329fd3c6e33d745abd	refs/heads/bernstein
c5c64e3fe48302f0c4581985f9c68d615f7bcb4e	refs/heads/master
3fa7ded19a8da868d3af7c942f86358e6720f0c7	refs/heads/submodule
/home/sverdool/public_html/cloog.git	cloog^{URL}
http://www.liacs.nl/~sverdool/cloog.git	cloog^{URL}

Is there no code out there that expects the "rev" part to be
exactly 40 characters?
Or do you propose we put the URL in a blob and put the object sha1
in there.  If so, who's going to create these blobs for the git://
and ssh:// protocols?  upload-pack?

Thanks for the discussion, btw.
I hope we can come up with something that's acceptable to everyone.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, Sven Verdoolaege wrote:

> On Thu, May 24, 2007 at 12:31:33PM +0100, Johannes Schindelin wrote:
> > On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > > OK... so what should git-update-server-info put in this file for submodules?
> > > Or, equivalently, what should be the output of ls-remote?
> > > 
> > > Right now its a list of pairs of revs(sha1) and refs.
> > > For submodules we want a connection between a submodule name
> > > and one or more URLs where the submodule can be found.
> > > How are you going to squeeze that into info/refs without confusing
> > > older versions of git?
> > 
> > I wonder if the "ref^{blub}" syntax could be used for that: change "blub" 
> > to the URL, or "sub:URL" or something.
> 
> Just to be clear, would it look like the following?
> 
> e8a6e39ecfbd391a54b9c3329fd3c6e33d745abd	refs/heads/bernstein
> c5c64e3fe48302f0c4581985f9c68d615f7bcb4e	refs/heads/master
> 3fa7ded19a8da868d3af7c942f86358e6720f0c7	refs/heads/submodule
> /home/sverdool/public_html/cloog.git	cloog^{URL}
> http://www.liacs.nl/~sverdool/cloog.git	cloog^{URL}

I was more thinking about something like this:
3fa7ded19a8da868d3af7c942f86358e6720f0c7	refs/heads/submodule^{URL:/home/sverdool/public_html/cloog.git}

But then, I haven't really thought about it deeply.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Sixt]

Johannes Schindelin wrote:
> I was more thinking about something like this:
> 3fa7ded19a8da868d3af7c942f86358e6720f0c7        refs/heads/submodule^{URL:/home/sverdool/public_html/cloog.git}

where 3fa7ded... is the commit (gitlink) that appears in the tree?

-- Hannes

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, Johannes Sixt wrote:

> Johannes Schindelin wrote:
> > I was more thinking about something like this:
> > 3fa7ded19a8da868d3af7c942f86358e6720f0c7        refs/heads/submodule^{URL:/home/sverdool/public_html/cloog.git}
> 
> where 3fa7ded... is the commit (gitlink) that appears in the tree?

Yes.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 01:16:38PM +0100, Johannes Schindelin wrote:
> I was more thinking about something like this:
> 3fa7ded19a8da868d3af7c942f86358e6720f0c7	refs/heads/submodule^{URL:/home/sverdool/public_html/cloog.git}

As Johannes already indicated I don't think this rev makes sense.
I suppose we could just set it to 0.
I also don't think the URL should be associated to a ref.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, Sven Verdoolaege wrote:

> On Thu, May 24, 2007 at 01:16:38PM +0100, Johannes Schindelin wrote:
> > I was more thinking about something like this:
> > 3fa7ded19a8da868d3af7c942f86358e6720f0c7	refs/heads/submodule^{URL:/home/sverdool/public_html/cloog.git}
> 
> As Johannes already indicated I don't think this rev makes sense.

You like to confuse me, don't you?

> I suppose we could just set it to 0.
> I also don't think the URL should be associated to a ref.

It does not need to be.

But then, it is sort of a "subref": You could just clone the submodule in 
its own right, correct?

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 02:17:27PM +0100, Johannes Schindelin wrote:
> On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > I suppose we could just set it to 0.
> > I also don't think the URL should be associated to a ref.
> 
> It does not need to be.
> 
> But then, it is sort of a "subref": You could just clone the submodule in 
> its own right, correct?

Exactly, but the information we want is not associated to any
particular revision of the submodule.  It just points to the repo.
It's also not associated with any revision of the supermodule.
That information should go in a tracked .gitmodules file.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, Sven Verdoolaege wrote:

> On Thu, May 24, 2007 at 02:17:27PM +0100, Johannes Schindelin wrote:
> > On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > > I suppose we could just set it to 0.
> > > I also don't think the URL should be associated to a ref.
> > 
> > It does not need to be.
> > 
> > But then, it is sort of a "subref": You could just clone the submodule in 
> > its own right, correct?
> 
> Exactly, but the information we want is not associated to any
> particular revision of the submodule.  It just points to the repo.
> It's also not associated with any revision of the supermodule.
> That information should go in a tracked .gitmodules file.

It is not that expensive to just give the SHA-1 with the URL, and to 
introduce a new namespace, say

3f... submodule/path^{URL:blablub}

to say that the submodule which is connected in "HEAD:path" is available 
with the URL "blablub" and just so happens to be at commit "3f..." at the 
moment. Heck, you can even use this instead of expensive fetches to verify 
up-to-date, and even more, you can make sure that you are as up-to-date as 
the remote supermodule.

But then, I do not care about that deeply. Like I said, I haven't followed 
the discussions, but I really do not understand why an information as 
essential to a superproject is not contained in something like 
HEAD:.gitmodules. Git does not have to take it from there, it can still 
continue to take it from a local config in .git/, but .gitmodules can live 
in the tree happily, for the pleasure of the tools (if only to initialise 
the first version of the local config after clone).

Without some very intrusive surgery into the transport code of Git, in 22 
patches, which I am not at all comfortable with.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 02:52:25PM +0100, Johannes Schindelin wrote:
> It is not that expensive to just give the SHA-1 with the URL, and to 
> introduce a new namespace, say
> 
> 3f... submodule/path^{URL:blablub}
> 
> to say that the submodule which is connected in "HEAD:path" is available 
> with the URL "blablub" and just so happens to be at commit "3f..." at the 
> moment. Heck, you can even use this instead of expensive fetches to verify 
> up-to-date, and even more, you can make sure that you are as up-to-date as 
> the remote supermodule.

What about all the other branches?
Also, the submodule may not even be in any of the tips of the branches.

> Without some very intrusive surgery into the transport code of Git, in 22 
> patches, which I am not at all comfortable with.

You're not comfortable with the number or with the patches?
There's only about four or five that deal with this git-config stuff
and any other mechanism would have to implemented as well.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, Sven Verdoolaege wrote:

> On Thu, May 24, 2007 at 02:52:25PM +0100, Johannes Schindelin wrote:
> > It is not that expensive to just give the SHA-1 with the URL, and to 
> > introduce a new namespace, say
> > 
> > 3f... submodule/path^{URL:blablub}
> > 
> > to say that the submodule which is connected in "HEAD:path" is available 
> > with the URL "blablub" and just so happens to be at commit "3f..." at the 
> > moment. Heck, you can even use this instead of expensive fetches to verify 
> > up-to-date, and even more, you can make sure that you are as up-to-date as 
> > the remote supermodule.
> 
> What about all the other branches?
> Also, the submodule may not even be in any of the tips of the branches.

Okay, bad idea.

> > Without some very intrusive surgery into the transport code of Git, in 22 
> > patches, which I am not at all comfortable with.
> 
> You're not comfortable with the number or with the patches?
> There's only about four or five that deal with this git-config stuff
> and any other mechanism would have to implemented as well.

If it was even only one, which is as intrusive, I would still be 
uncomfortable. And yes, the large number does not make me want to review 
the series either.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Lars Hjemli]

On 5/24/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Thu, 24 May 2007, Sven Verdoolaege wrote:
>
> > On Thu, May 24, 2007 at 12:31:33PM +0100, Johannes Schindelin wrote:
> > > On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > > > OK... so what should git-update-server-info put in this file for submodules?
> > > > Or, equivalently, what should be the output of ls-remote?
> > > >
> > > > Right now its a list of pairs of revs(sha1) and refs.
> > > > For submodules we want a connection between a submodule name
> > > > and one or more URLs where the submodule can be found.
> > > > How are you going to squeeze that into info/refs without confusing
> > > > older versions of git?
> > >
> > > I wonder if the "ref^{blub}" syntax could be used for that: change "blub"
> > > to the URL, or "sub:URL" or something.
> >
> > Just to be clear, would it look like the following?
> >
> > e8a6e39ecfbd391a54b9c3329fd3c6e33d745abd      refs/heads/bernstein
> > c5c64e3fe48302f0c4581985f9c68d615f7bcb4e      refs/heads/master
> > 3fa7ded19a8da868d3af7c942f86358e6720f0c7      refs/heads/submodule
> > /home/sverdool/public_html/cloog.git  cloog^{URL}
> > http://www.liacs.nl/~sverdool/cloog.git       cloog^{URL}
>
> I was more thinking about something like this:
> 3fa7ded19a8da868d3af7c942f86358e6720f0c7        refs/heads/submodule^{URL:/home/sverdool/public_html/cloog.git}
>
> But then, I haven't really thought about it deeply.

I think the whole point of the 'remote config' stuff is to get an
unversioned/out-of-tree .gitmodules file, right?

If so, one could put this file into the object db and refer to it with
something like 'refs/tags/subproject-config' or even
'refs/misc/subproject-config'. Both of these refs will be found by
ls-remote and point to the object containing the suggested subproject
configuration.

I actually do something similar with the release tarballs for cgit: try
  git ls-remote git://hjemli.net/pub/git/cgit
to see the references to the tarballs which are displayed on the cgit
summary page.

But then again, I might have completely misunderstood the problem...

--
larsh

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 02:41:57PM +0200, Lars Hjemli wrote:
> I think the whole point of the 'remote config' stuff is to get an
> unversioned/out-of-tree .gitmodules file, right?

Yes.

> If so, one could put this file into the object db and refer to it with
> something like 'refs/tags/subproject-config' or even
> 'refs/misc/subproject-config'. Both of these refs will be found by
> ls-remote and point to the object containing the suggested subproject
> configuration.

That's a possibility, but then...

On Thu, May 24, 2007 at 01:43:54PM +0200, Sven Verdoolaege wrote:
> Or do you propose we put the URL in a blob and put the object sha1
> in there.  If so, who's going to create these blobs for the git://
> and ssh:// protocols?  upload-pack?

I don't think you can expect the user to this herself.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Lars Hjemli]

On 5/24/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> On Thu, May 24, 2007 at 02:41:57PM +0200, Lars Hjemli wrote:
> > I think the whole point of the 'remote config' stuff is to get an
> > unversioned/out-of-tree .gitmodules file, right?
>
> Yes.
>
> > If so, one could put this file into the object db and refer to it with
> > something like 'refs/tags/subproject-config' or even
> > 'refs/misc/subproject-config'. Both of these refs will be found by
> > ls-remote and point to the object containing the suggested subproject
> > configuration.
>
> That's a possibility, but then...
>
> On Thu, May 24, 2007 at 01:43:54PM +0200, Sven Verdoolaege wrote:
> > Or do you propose we put the URL in a blob and put the object sha1
> > in there.  If so, who's going to create these blobs for the git://
> > and ssh:// protocols?  upload-pack?
>
> I don't think you can expect the user to this herself.

Well, my point would be that we already have the necessary plumbing to
solve the problem (discovery and distribution of out-of-tree objects).
Some porcelain support on top of this is all that is needed (well,
almost. I just tried downloading my own tarball, and got an error
about the ref not pointing to a commit. But that shouldn't be too hard
to fix)

-- 
larsh

Re: [RFC] Fourth round of support for cloning submodules

[Junio C Hamano]

"Lars Hjemli" <hjemli@gmail.com> writes:

> I think the whole point of the 'remote config' stuff is to get an
> unversioned/out-of-tree .gitmodules file, right?

Why does this have to be out-of-tree and unversioned to begin
with?

When you are bootstrapping, you will start by a fetch/clone of
the superproject.  Why can't that tree contain necessary
information that is relevant to the superproject in question?

Isn't the information about which subprojects are used by the
superproject specific to each superproject, and also specific to
each version of the superproject (as a superproject can start
using more projects than it did before)?

Re: [RFC] Fourth round of support for cloning submodules

[Lars Hjemli]

On 5/24/07, Junio C Hamano <junkio@cox.net> wrote:
> "Lars Hjemli" <hjemli@gmail.com> writes:
>
> > I think the whole point of the 'remote config' stuff is to get an
> > unversioned/out-of-tree .gitmodules file, right?
>
> Why does this have to be out-of-tree and unversioned to begin
> with?

Probably to cater for subprojects moving away from the url mentioned
in the versioned .gitmodules file.

> When you are bootstrapping, you will start by a fetch/clone of
> the superproject.  Why can't that tree contain necessary
> information that is relevant to the superproject in question?

It sure can, and it would be the most natural solution. I just wanted
to mention an alternative to the 'git config --remote' solution.

--
larsh

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 10:13:58AM -0700, Junio C Hamano wrote:
> When you are bootstrapping, you will start by a fetch/clone of
> the superproject.  Why can't that tree contain necessary
> information that is relevant to the superproject in question?

Because git is a distributed SCM.

> Isn't the information about which subprojects are used by the
> superproject specific to each superproject, and also specific to
> each version of the superproject (as a superproject can start
> using more projects than it did before)?

Perhaps for some information, but not for the "preferred" URL
to get the subprojects from.  That may change in time (independently
of any changes in the superproject itself) and they
will be different for different copies of the same superproject.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Linus Torvalds]

On Thu, 24 May 2007, Junio C Hamano wrote:
> 
> Why does this have to be out-of-tree and unversioned to begin
> with?

I _really_ think that the right approach is to

 - have the submodules information under version control (and I'd 
   personally call it the ".gitmodules" file, but whatever)

   This gives you the defaults, and the ability to change them. Remember: 
   if you get some "config" information at "git clone" time, you're 
   *screwed* if the thing ever changes!

   So being version-controlled is not just a good idea. It's a 
   _requirement_ for working well.

 - have a way to *override* the version-controlled information using a 
   local config file.

   This is what you'd use to say "ignore the official information, it's 
   either slow (because the "official" server is on another continent) or 
   outdated (because the server went away)".

I actually really liked Junio's suggestion of

	[subproject "git://git.kernel.org/pub/linux-2.4.git"]
		URL = http://www.kernel.org/pub/linux-2.4.git

except I would actually make it a bit more generic, in that I don't think 
this kind of "URL rewriting" is necessarily even subproject-specific, but 
could be useful in general. IOW, it might be a good idea to have

	[url "git://git.kernel.org/pub/linux-2.4.git"]
		rewrite = "ssh://master.kernel.org/pub/linux-2.4.git"

or something - which allows people to set up automatic rewriting rules in 
case they have alternate ways of getting to better repositories (ie people 
might send me a "please pull" request with a pointer to the _public_ site, 
because they also Cc: the kernel mailing list, but since I have direct SSH 
access to the master site, _I_ might want to rewrite it to use that 
instead!).

Note how that example had nothing to do with subprojects per se: the URL 
rewriting is really another issue. But yes, I think it might also be worth 
it to actually be able to override the whole subproject data, ie also have

	[subproject "kernel/"]
		url = ...
		branch = xyzzy

and allow that kind of information in .git/config to _override_ any such 
entry in .gitmodules!

		Linus

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 10:40:52AM -0700, Linus Torvalds wrote:
> 
> 
> On Thu, 24 May 2007, Junio C Hamano wrote:
> > 
> > Why does this have to be out-of-tree and unversioned to begin
> > with?
> 
> I _really_ think that the right approach is to
> 
>  - have the submodules information under version control (and I'd 
>    personally call it the ".gitmodules" file, but whatever)
> 
>    This gives you the defaults, and the ability to change them. Remember: 
>    if you get some "config" information at "git clone" time, you're 
>    *screwed* if the thing ever changes!

If you allow an override, then I don't see how having the initial
information in the tree is any better.
When new information gets in from the tree, you're going to ignore it anyway.

What happens if the URL changes?
You have to modify .gitmodules in _every_ branch you have?

What if someone is working on his own branch of the superproject
that needs some changes in his own subproject?
He needs to modify .gitmodules, but when the changes go upstream,
this .gitmodules changes get merged as well.
Now imagine several developers doing this.
You end up continually having to modify .gitmodules.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Linus Torvalds]

On Thu, 24 May 2007, Sven Verdoolaege wrote:
> 
> If you allow an override, then I don't see how having the initial
> information in the tree is any better.
> When new information gets in from the tree, you're going to ignore it anyway.

Well, duh. "Paging Mr Ovious".

IF you have local overrides they get ignored. 

Which is what _should_ happen, of course.

> What if someone is working on his own branch of the superproject
> that needs some changes in his own subproject?
> He needs to modify .gitmodules, but when the changes go upstream,
> this .gitmodules changes get merged as well.
> Now imagine several developers doing this.
> You end up continually having to modify .gitmodules.

Ehh. What drugs are you on?

That's the whole point of having local overrides. You use them for local 
branches. You do _not_ use .gitmodules for those.

So ".gitmodules" is the default for people who don't do anything special. 
Only people who change the _default_ would ever change that.

I really don't understand or see your objections at all. You are making 
totally idiotic arguments BASED ON DOING OBVIOUSLY STUPID THINGS. That's 
not an argument.

		Linus

Re: [RFC] Fourth round of support for cloning submodules

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> On Thu, 24 May 2007, Sven Verdoolaege wrote:
>> ...
>> What if someone is working on his own branch of the superproject
>> that needs some changes in his own subproject?
>> He needs to modify .gitmodules, but when the changes go upstream,
>> this .gitmodules changes get merged as well.
>> Now imagine several developers doing this.
>> You end up continually having to modify .gitmodules.
>
> Ehh. What drugs are you on?
>
> That's the whole point of having local overrides. You use them for local 
> branches. You do _not_ use .gitmodules for those.
>
> So ".gitmodules" is the default for people who don't do anything special. 
> Only people who change the _default_ would ever change that.
>
> I really don't understand or see your objections at all. You are making 
> totally idiotic arguments BASED ON DOING OBVIOUSLY STUPID THINGS. That's 
> not an argument.

While I agree with the three-level thing that uses .gitmodules
and take the information from .git/config would solve the
problem, I do not think you need to shout.

While I disagree with some design decisions Sven's series made,
I am happy that the series is there for people to comment on.
It helps us identify the design issues by making the differences
of opinion people have on them stand out.

Re: [RFC] Fourth round of support for cloning submodules

[Lars Hjemli]

On 5/24/07, Junio C Hamano <junkio@cox.net> wrote:
> While I disagree with some design decisions Sven's series made,
> I am happy that the series is there for people to comment on.
> It helps us identify the design issues by making the differences
> of opinion people have on them stand out.
>

Possibly offtopic: I'm not so sure there should be any --submodules or
similar options to clone/fetch/checkout/merge/diff etc.

What I think would be nice is some porcelain support to manually init,
update and see the checked out version of selected subprojects, but as
standalone commands.

This would make it easier to start using subprojects, and it wouldn't
force any specific politics on it. And hopefully the experience of
actually using subprojects will make it more obvious how to add
extended support later on.

Just my 2c

--
larsh

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, Lars Hjemli wrote:

> On 5/24/07, Junio C Hamano <junkio@cox.net> wrote:
> > While I disagree with some design decisions Sven's series made,
> > I am happy that the series is there for people to comment on.
> > It helps us identify the design issues by making the differences
> > of opinion people have on them stand out.
> > 
> 
> Possibly offtopic: I'm not so sure there should be any --submodules or
> similar options to clone/fetch/checkout/merge/diff etc.
> 
> What I think would be nice is some porcelain support to manually init,
> update and see the checked out version of selected subprojects, but as
> standalone commands.

Yes, a la git-remote. I'd be much happier with that, too, especially since 
I think that this can be a relatively small and easy-to-review script. I 
really was intimidated by the long patch series, also because I thought 
that it shouldn't be that much work on top of what Linus did.

Ciao,
Dscho

P.S.: Linus paid for his Caps Lock key, so he might as well use it.

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, Sven Verdoolaege wrote:

> On Thu, May 24, 2007 at 10:40:52AM -0700, Linus Torvalds wrote:
> > 
> > 
> > On Thu, 24 May 2007, Junio C Hamano wrote:
> > > 
> > > Why does this have to be out-of-tree and unversioned to begin
> > > with?
> > 
> > I _really_ think that the right approach is to
> > 
> >  - have the submodules information under version control (and I'd 
> >    personally call it the ".gitmodules" file, but whatever)
> > 
> >    This gives you the defaults, and the ability to change them. Remember: 
> >    if you get some "config" information at "git clone" time, you're 
> >    *screwed* if the thing ever changes!
> 
> If you allow an override, then I don't see how having the initial
> information in the tree is any better.

Isn't that obvious? _Most_ people will _not_ override the information.

Plus, it is an easy solution to your problem, without having to touch a 
lot of real core parts of Git. Simple is beautiful. And less buggy.

> When new information gets in from the tree, you're going to ignore it 
> anyway.

No. Not if it is _not_ overridden.

> What happens if the URL changes?
> You have to modify .gitmodules in _every_ branch you have?

Oh yes. Exactly the same as when you have an INSTALL file and the URL of 
some project changes which your project depends on. That's life.

> What if someone is working on his own branch of the superproject
> that needs some changes in his own subproject?

Uhm. Then you can either just override that URL, _or_ you can branch from 
the superproject, changing .gitmodules as you wish.

> He needs to modify .gitmodules, but when the changes go upstream,
> this .gitmodules changes get merged as well.

If you change the superproject, that is. And then only if you don't have a 
to-merge branch. Your problem is not specific to superprojects, and it has 
been solved already.

> Now imagine several developers doing this.
> You end up continually having to modify .gitmodules.

No, just your config. Once.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

I'm obviously not going to stop anyone from putting URLs in .gitmodules
and I can see that it would be useful in practice, but I still think
that it doesn't matter where the submodule was located at any
given point in history (from the point of view of the superproject).
It only matters where the submodule is located now.

You may be bisecting a problem and you may need to clone a submodule
for a point in history when the submodule was placed somewhere else.
(You may not have had a need to checkout the submodule before,
or it may simply not be used in the current version of the supermodule.)
So, I think it would still be useful to have an optional additional
out-of-tree mechanism of getting usable URLs if the URLs in .gitmodules
or your local config don't work.

On Thu, May 24, 2007 at 07:11:31PM +0100, Johannes Schindelin wrote:
> On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > If you allow an override, then I don't see how having the initial
> > information in the tree is any better.
> 
> Isn't that obvious? _Most_ people will _not_ override the information.

You may be asked to.

	"Please clone from a mirror close to you and after cloning,
	 please change the URLs of these 54 submodules to a mirror
	 close to you."

> Plus, it is an easy solution to your problem, without having to touch a 
> lot of real core parts of Git. Simple is beautiful. And less buggy.

I'll take your word for it.

> > He needs to modify .gitmodules, but when the changes go upstream,
> > this .gitmodules changes get merged as well.
> 
> If you change the superproject, that is.

That's what I meant.  Plus, other users may want to pull from such forks too.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Junio C Hamano]

Sven Verdoolaege <skimo@kotnet.org> writes:

> I'm obviously not going to stop anyone from putting URLs in .gitmodules
> and I can see that it would be useful in practice, but I still think
> that it doesn't matter where the submodule was located at any
> given point in history (from the point of view of the superproject).
> It only matters where the submodule is located now.
>
> You may be bisecting a problem and you may need to clone a submodule
> for a point in history when the submodule was placed somewhere else.
> (You may not have had a need to checkout the submodule before,
> or it may simply not be used in the current version of the supermodule.)
> So, I think it would still be useful to have an optional additional
> out-of-tree mechanism of getting usable URLs if the URLs in .gitmodules
> or your local config don't work.

I thought that was already solved in my original two-level
strawman and can naturally be extended to the three-level
strawman.  What am I missing?

gmane=http://article.gmane.org/gmane.comp.version-control.git/

	$gmane/47502
	$gmane/47548
	$gmane/47621

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Fri, May 25, 2007 at 09:16:30AM -0700, Junio C Hamano wrote:
> Sven Verdoolaege <skimo@kotnet.org> writes:
> > So, I think it would still be useful to have an optional additional
> > out-of-tree mechanism of getting usable URLs if the URLs in .gitmodules
> > or your local config don't work.
> 
> I thought that was already solved in my original two-level
> strawman and can naturally be extended to the three-level
> strawman.  What am I missing?

Maybe I'm missing something but you only seem to talk about .gitmodules
and local config there, while this would be a way of automatically getting
URLs that are either not available in .gitmodules or the local config
or are outdated or should be overridden.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Sven Verdoolaege wrote:

> On Fri, May 25, 2007 at 09:16:30AM -0700, Junio C Hamano wrote:
> > Sven Verdoolaege <skimo@kotnet.org> writes:
> > > So, I think it would still be useful to have an optional additional
> > > out-of-tree mechanism of getting usable URLs if the URLs in .gitmodules
> > > or your local config don't work.
> > 
> > I thought that was already solved in my original two-level
> > strawman and can naturally be extended to the three-level
> > strawman.  What am I missing?
> 
> Maybe I'm missing something but you only seem to talk about .gitmodules
> and local config there, while this would be a way of automatically getting
> URLs that are either not available in .gitmodules or the local config
> or are outdated or should be overridden.

I do not see what you're getting at.

Either you want to fetch the URLs from an upstream, in which case they 
should be fetched. Why not in ".gitmodules"? If the URLs don't work, you 
should blame the maintainer, not fsck up the tool.

Or you want some sort of "fallback" set of URLs. I cannot think of any 
reasonable use case here.

Or you want to receive overrides, such as when a submodule has moved, so 
that it still works when you check out older versions (and do not have the 
objects locally already). But to make such overrides automatic wreaks 
havoc IMHO.

Also, this is rare enough that I'd rather not fsck up an elegant solution, 
where you have to treat such special cases manually, for something which 
affects "normal" users, which just track the project.

Maybe I'm wrong. But then, Josef is right, we can talk about that later. 
It's not like you need a feature like that from day one.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> I actually really liked Junio's suggestion of
>
> 	[subproject "git://git.kernel.org/pub/linux-2.4.git"]
> 		URL = http://www.kernel.org/pub/linux-2.4.git
> ...
> Note how that example had nothing to do with subprojects per se: the URL 
> rewriting is really another issue. But yes, I think it might also be worth 
> it to actually be able to override the whole subproject data, ie also have
>
> 	[subproject "kernel/"]
> 		url = ...
> 		branch = xyzzy
>
> and allow that kind of information in .git/config to _override_ any such 
> entry in .gitmodules!

I just sent out a rather longish message that peripherally
touches this and said I liked the three-level thing Steven Grimm
suggested, which is essentially what you wrote above.

I am reasonably happy that we are in agreement and I was not way
off in the design area by myself.

Re: [RFC] Fourth round of support for cloning submodules

[Josef Weidendorfer]

On Thursday 24 May 2007, Linus Torvalds wrote:
> 
> On Thu, 24 May 2007, Junio C Hamano wrote:
> > 
> > Why does this have to be out-of-tree and unversioned to begin
> > with?
> 
> I _really_ think that the right approach is to
> 
>  - have the submodules information under version control (and I'd 
>    personally call it the ".gitmodules" file, but whatever)
> 
>    This gives you the defaults, and the ability to change them.

Sorry to repeat the obvious.

I assume you talk about a versioned .gitmodules file tied to the
superproject history, and any fetch/pull would look into this
file from the current working directory to lookup the default URL.

Wouldn't this have the problem that when you check out an old
revision of the superproject you get out-of-date URLs, so that
a fetch does not work (without local overrides)?

IMHO we really need 2 kinds of submodule config:
(1) One bound to the superproject history
(2) One bound _not_ to the superproject history.

However, (2) could be put into its own, seperate history/branch,
similar to the "todo" branch in the git repository. The tip of the
submodule config history, (2) above, would e.g. be available as
"ref/submodulesconfig" (and could be checked out by "git-clone" into
a separate place like .git/submodulesconfig of the superproject for easy
editing).
Any "git-fetch" of the superproject would also fetch any changed defaults
in the submodulesconfig history.

This even allows to override any URL defaults for downstream clones
of a superproject repository, by commiting such changes as new
revision in the refs/submodulesconfig history.

Perhaps I am missing something obvious, but a history for project
configuration, completely separate from the project history itself,
seems useful in general to me.

Josef

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Josef Weidendorfer wrote:

> On Thursday 24 May 2007, Linus Torvalds wrote:
> > 
> > On Thu, 24 May 2007, Junio C Hamano wrote:
> > > 
> > > Why does this have to be out-of-tree and unversioned to begin
> > > with?
> > 
> > I _really_ think that the right approach is to
> > 
> >  - have the submodules information under version control (and I'd 
> >    personally call it the ".gitmodules" file, but whatever)
> > 
> >    This gives you the defaults, and the ability to change them.
> 
> Sorry to repeat the obvious.
> 
> I assume you talk about a versioned .gitmodules file tied to the
> superproject history, and any fetch/pull would look into this
> file from the current working directory to lookup the default URL.
> 
> Wouldn't this have the problem that when you check out an old
> revision of the superproject you get out-of-date URLs, so that
> a fetch does not work (without local overrides)?

If you check out an old revision, wouldn't you have that _already_, so it 
does not matter what URL is given in .gitmodules?

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Josef Weidendorfer]

On Friday 25 May 2007, Johannes Schindelin wrote:
> > I assume you talk about a versioned .gitmodules file tied to the
> > superproject history, and any fetch/pull would look into this
> > file from the current working directory to lookup the default URL.
> > 
> > Wouldn't this have the problem that when you check out an old
> > revision of the superproject you get out-of-date URLs, so that
> > a fetch does not work (without local overrides)?
> 
> If you check out an old revision, wouldn't you have that _already_, so it 
> does not matter what URL is given in .gitmodules?

Not necessary.
* Submodules can appear/disappear any time in the superproject.
Therefore, going back in time can make it necessary to have to clone
a submodule you did not have before.
* Not every submodule is interesting for every developer; therefore,
an important design-decision for submodules is to allow at git-clone time
to not clone some submodules at all. However, you can change your mind and
want to follow a given submodule later.

Above points make it necessary for the submodule feature to implement
lazy clones. You want be able to clone a submodule without error independent
of the superproject HEAD. IMHO it would be really bad porcelain design
to break out with an error each time and expect the USER to manually
override submodule URLs.

Another difference between in-tree .gitmodules and out-of-tree submodule
config is that for in-tree you any want to see configuration for submodules
which happen to be part of the superproject at this point in history.
For out-of-tree config, there has to be information for every submodule which
was referenced in the whole history of the superproject.

Josef

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Josef Weidendorfer wrote:

> On Friday 25 May 2007, Johannes Schindelin wrote:
> > > I assume you talk about a versioned .gitmodules file tied to the
> > > superproject history, and any fetch/pull would look into this
> > > file from the current working directory to lookup the default URL.
> > > 
> > > Wouldn't this have the problem that when you check out an old
> > > revision of the superproject you get out-of-date URLs, so that
> > > a fetch does not work (without local overrides)?
> > 
> > If you check out an old revision, wouldn't you have that _already_, so it 
> > does not matter what URL is given in .gitmodules?
> 
> Not necessary.
> * Submodules can appear/disappear any time in the superproject.
> Therefore, going back in time can make it necessary to have to clone
> a submodule you did not have before.
> * Not every submodule is interesting for every developer; therefore,
> an important design-decision for submodules is to allow at git-clone time
> to not clone some submodules at all. However, you can change your mind and
> want to follow a given submodule later.

Okay, so there are exceptions to the rule, just as everywhere.

We already talked about being able to override .gitmodules from 
.git/config. I think that should really, really be sufficient, as you 
cannot hope to have a one-size-fits-them-all solution for the exceptions 
you described. You'll have to cope with them manually anyway.

We should not design for the exception. Therefore I think the .gitmodules, 
overrideable by .git/config is sufficient.

And the point about my config being private still stands. You have no 
business looking into my config.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Fri, May 25, 2007 at 03:16:56PM +0100, Johannes Schindelin wrote:
> And the point about my config being private still stands. You have no 
> business looking into my config.

Put your private configuration in config and your public configuration
in submodulesconfig.
A public repo typically wouldn't have any submodules configuration
in config (or any other private information for that matter), but
we wouldn't read it anyway.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Sven Verdoolaege wrote:

> On Fri, May 25, 2007 at 03:16:56PM +0100, Johannes Schindelin wrote:
> > And the point about my config being private still stands. You have no 
> > business looking into my config.
> 
> Put your private configuration in config and your public configuration
> in submodulesconfig.
> A public repo typically wouldn't have any submodules configuration
> in config (or any other private information for that matter), but
> we wouldn't read it anyway.

Why complicate things? The information you want to put into 
submodulesconfig is an information you would most likely want to give 
cloners.

And as it happens, we _have_ a perfect transport mechanism for clones. You 
only have to put the information into the object database itself, and -- 
voila -- no need to change any core level plumbing.

That's why I keep harping on .gitmodules, since it _is_ the obvious 
correct, simple and elegant way to do things here.

So, just "s/\.git\/submodulesconfig/.gitmodules/g" in your head, and 
you're done.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Josef Weidendorfer]

On Friday 25 May 2007, Johannes Schindelin wrote:
> On Fri, 25 May 2007, Josef Weidendorfer wrote:
> > * Submodules can appear/disappear any time in the superproject.
> > Therefore, going back in time can make it necessary to have to clone
> > a submodule you did not have before.
> > * Not every submodule is interesting for every developer; therefore,
> > an important design-decision for submodules is to allow at git-clone time
> > to not clone some submodules at all. However, you can change your mind and
> > want to follow a given submodule later.
> 
> Okay, so there are exceptions to the rule, just as everywhere.

The question here is in how many superprojects the exception
will become the rule, which would make manual overriding quite
cumbersome.

However, the exact policy for finding a fitting URL for a submodule
is not a fundamental design decision, and can be incrementally
improved, depending on use cases.

I agree with Junio that a simply, basic and robust submodule
implementation currently is important as first goal.

> We should not design for the exception. Therefore I think the .gitmodules, 
> overrideable by .git/config is sufficient.
> 
> And the point about my config being private still stands. You have no 
> business looking into my config.

I totally agree.

Josef

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Josef Weidendorfer wrote:

> On Friday 25 May 2007, Johannes Schindelin wrote:
> > On Fri, 25 May 2007, Josef Weidendorfer wrote:
> >
> > > * Submodules can appear/disappear any time in the superproject. 
> > > Therefore, going back in time can make it necessary to have to clone 
> > > a submodule you did not have before.
> > >
> > > * Not every submodule is interesting for every developer; therefore, 
> > > an important design-decision for submodules is to allow at git-clone 
> > > time to not clone some submodules at all. However, you can change 
> > > your mind and want to follow a given submodule later.
> > 
> > Okay, so there are exceptions to the rule, just as everywhere.
> 
> The question here is in how many superprojects the exception will become 
> the rule, which would make manual overriding quite cumbersome.

I never disputed that manual overriding is cumbersome. However, I dispute 
that automatic overriding is _possible_.

> I agree with Junio that a simply, basic and robust submodule 
> implementation currently is important as first goal.

Good. Me, too.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Linus Torvalds]

On Fri, 25 May 2007, Josef Weidendorfer wrote:
>
> * Submodules can appear/disappear any time in the superproject.
> Therefore, going back in time can make it necessary to have to clone
> a submodule you did not have before.

You missed the obvious thing here: if a submodule 
appears/disappears/moves, then the module info *has* to be versioned. 
Otherwise your module setup will be *wrong*.

Think about it. If you move the kernel module somewhere else (say, you 
make it be in "old-kernel"), and then replace the "kernel" module with a 
_new_ submodule, if you don't version the .gitmodules file, you're simply 
_screwed_. You'll be loading the "kernel" module from the wrong place 
when you move around in history!

In contrast, if you version it, everything JUST WORKS.

So:

 - submodule information *has* to be versioned. If it isn't, it's broken. 

   It really _is_ that simple.

 - because some people will want to replace repository addresses etc, you 
   want to have a per-repo (or user, or system) *mapping* of that
   versioned submodule information, so that people can override the actual 
   location. But that doesn't obviate the need for the basic module 
   information having to be versioned.

So no, we cannot have the fundamental module info setup anywhere else than 
in something like .gitmodules.

		Linus

Re: [RFC] Fourth round of support for cloning submodules

[Josef Weidendorfer]

On Friday 25 May 2007, Linus Torvalds wrote:
 
> On Fri, 25 May 2007, Josef Weidendorfer wrote:
> >
> > * Submodules can appear/disappear any time in the superproject.
> > Therefore, going back in time can make it necessary to have to clone
> > a submodule you did not have before.
> 
> You missed the obvious thing here: if a submodule 
> appears/disappears/moves, then the module info *has* to be versioned.

I never refused the need for a versioned .gitmodules file in-tree;
I totally agree with you here. Perhaps I forgot to mention this
because I took it for granted.

>  - because some people will want to replace repository addresses etc, you 
>    want to have a per-repo (or user, or system) *mapping* of that
>    versioned submodule information,

Yes.
So now we have 2 levels of submodule configuration:
(1) In-tree, versioned together with the superproject tree
(2) Local overrides via .git/config.

AFAICS the whole discussion here is about whether we need
a further configuration level with different visibility:
one which in not versioned together with the superproject
history, but which can be cloned, ie. a superproject-wide
configuration (*1*).
And I can see a need here for submodule URLs.

But as I just said in another mail in this thread, that
would be an additional feature which can be added later
if we see that  - because of whatever strange use cases
for submodules - some git users are forced to constantly
adapt 50 URL rewritings in their local config.

Josef

*1* For project-wide configuration, independent of the
actual project history, a completely separate branch
can be used (lets say refs/projectconfig), which
contains one blob, the config file: "projectconfig".
By making "git-config" to also parse
"refs/projectconfig:projectconfig" before the other
config files if existing, this gives us a natural way
to provide project wide settings.

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Josef Weidendorfer wrote:

> AFAICS the whole discussion here is about whether we need
> a further configuration level with different visibility:
> one which in not versioned together with the superproject
> history, but which can be cloned, ie. a superproject-wide
> configuration (*1*).

I smell centralization here. _Forced_ centralization.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Josef Weidendorfer]

On Friday 25 May 2007, Johannes Schindelin wrote:
> Hi,
> 
> On Fri, 25 May 2007, Josef Weidendorfer wrote:
> 
> > AFAICS the whole discussion here is about whether we need
> > a further configuration level with different visibility:
> > one which in not versioned together with the superproject
> > history, but which can be cloned, ie. a superproject-wide
> > configuration (*1*).
> 
> I smell centralization here. _Forced_ centralization.

?

Everybody would be free to change the config in such a project
config branch, or use another version of it.

Hmm. Perhaps this is one of my complicating ideas ;-)

Josef

> 
> Ciao,
> Dscho
> 

Re: [RFC] Fourth round of support for cloning submodules

[Jakub Narebski]

Johannes Schindelin wrote:
> On Thu, 24 May 2007, Sven Verdoolaege wrote:
>> On Thu, May 24, 2007 at 12:31:33PM +0100, Johannes Schindelin wrote:
>>> On Thu, 24 May 2007, Sven Verdoolaege wrote:

>>>> OK... so what should git-update-server-info put in this file for submodules?
>>>> Or, equivalently, what should be the output of ls-remote?
>>>> 
>>>> Right now its a list of pairs of revs(sha1) and refs.
>>>> For submodules we want a connection between a submodule name
>>>> and one or more URLs where the submodule can be found.
>>>> How are you going to squeeze that into info/refs without confusing
>>>> older versions of git?
>>> 
>>> I wonder if the "ref^{blub}" syntax could be used for that: change "blub" 
>>> to the URL, or "sub:URL" or something.
>> 
>> Just to be clear, would it look like the following?
>> 
>> e8a6e39ecfbd391a54b9c3329fd3c6e33d745abd     refs/heads/bernstein
>> c5c64e3fe48302f0c4581985f9c68d615f7bcb4e     refs/heads/master
>> 3fa7ded19a8da868d3af7c942f86358e6720f0c7     refs/heads/submodule
>> /home/sverdool/public_html/cloog.git cloog^{URL}
>> http://www.liacs.nl/~sverdool/cloog.git      cloog^{URL}
> 
> I was more thinking about something like this:
> 3fa7ded19a8da868d3af7c942f86358e6720f0c7      refs/heads/submodule^{URL:/home/sverdool/public_html/cloog.git}
> 
> But then, I haven't really thought about it deeply.

I was thinking about the following:

  ref: refs/heads/master        HEAD
  e8a6e39ecfbd391a54b9c3329fd3c6e33d745abd      refs/heads/bernstein
  c5c64e3fe48302f0c4581985f9c68d615f7bcb4e      refs/heads/master
  3fa7ded19a8da868d3af7c942f86358e6720f0c7      refs/heads/master:submodule/path
  URL: /home/sverdool/public_html/cloog.git     refs/heads/master:submodule/path^{URL}
  URL: http://www.liacs.nl/~sverdool/cloog.git  refs/heads/master:submodule/path^{URL}

By the way, it would be nice if git-show-refs --deferefence used TAB
between sha1 and ref name, like in git-ls-remote / git-peek-remote,
otherwise I think git-show-refs output parsers would fail on [proposed]
"current branch" extension.

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Fri, 25 May 2007, Jakub Narebski wrote:

> Johannes Schindelin wrote:
> > On Thu, 24 May 2007, Sven Verdoolaege wrote:
> >> On Thu, May 24, 2007 at 12:31:33PM +0100, Johannes Schindelin wrote:
> >>> On Thu, 24 May 2007, Sven Verdoolaege wrote:
> 
> >>>> OK... so what should git-update-server-info put in this file for submodules?
> >>>> Or, equivalently, what should be the output of ls-remote?
> >>>> 
> >>>> Right now its a list of pairs of revs(sha1) and refs.
> >>>> For submodules we want a connection between a submodule name
> >>>> and one or more URLs where the submodule can be found.
> >>>> How are you going to squeeze that into info/refs without confusing
> >>>> older versions of git?
> >>> 
> >>> I wonder if the "ref^{blub}" syntax could be used for that: change "blub" 
> >>> to the URL, or "sub:URL" or something.
> >> 
> >> Just to be clear, would it look like the following?
> >> 
> >> e8a6e39ecfbd391a54b9c3329fd3c6e33d745abd     refs/heads/bernstein
> >> c5c64e3fe48302f0c4581985f9c68d615f7bcb4e     refs/heads/master
> >> 3fa7ded19a8da868d3af7c942f86358e6720f0c7     refs/heads/submodule
> >> /home/sverdool/public_html/cloog.git cloog^{URL}
> >> http://www.liacs.nl/~sverdool/cloog.git      cloog^{URL}
> > 
> > I was more thinking about something like this:
> > 3fa7ded19a8da868d3af7c942f86358e6720f0c7      refs/heads/submodule^{URL:/home/sverdool/public_html/cloog.git}
> > 
> > But then, I haven't really thought about it deeply.
> 
> I was thinking about the following:
> 
>   ref: refs/heads/master        HEAD
>   e8a6e39ecfbd391a54b9c3329fd3c6e33d745abd      refs/heads/bernstein
>   c5c64e3fe48302f0c4581985f9c68d615f7bcb4e      refs/heads/master
>   3fa7ded19a8da868d3af7c942f86358e6720f0c7      refs/heads/master:submodule/path
>   URL: /home/sverdool/public_html/cloog.git     refs/heads/master:submodule/path^{URL}
>   URL: http://www.liacs.nl/~sverdool/cloog.git  refs/heads/master:submodule/path^{URL}
> 
> By the way, it would be nice if git-show-refs --deferefence used TAB
> between sha1 and ref name, like in git-ls-remote / git-peek-remote,
> otherwise I think git-show-refs output parsers would fail on [proposed]
> "current branch" extension.

Well, I always hinted at it, and now officially retract my proposal. IMHO 
.gitmodules is a much better place for that. It _belongs_ in the 
repository, not just the config.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Santi Béjar]

On 5/24/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> On Thu, May 24, 2007 at 12:31:33PM +0100, Johannes Schindelin wrote:
> > On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > > OK... so what should git-update-server-info put in this file for submodules?
> > > Or, equivalently, what should be the output of ls-remote?
> > >
> > > Right now its a list of pairs of revs(sha1) and refs.
> > > For submodules we want a connection between a submodule name
> > > and one or more URLs where the submodule can be found.
> > > How are you going to squeeze that into info/refs without confusing
> > > older versions of git?
> >
> > I wonder if the "ref^{blub}" syntax could be used for that: change "blub"
> > to the URL, or "sub:URL" or something.
>
> Just to be clear, would it look like the following?
>
> e8a6e39ecfbd391a54b9c3329fd3c6e33d745abd        refs/heads/bernstein
> c5c64e3fe48302f0c4581985f9c68d615f7bcb4e        refs/heads/master
> 3fa7ded19a8da868d3af7c942f86358e6720f0c7        refs/heads/submodule
> /home/sverdool/public_html/cloog.git    cloog^{URL}
> http://www.liacs.nl/~sverdool/cloog.git cloog^{URL}
>
> Is there no code out there that expects the "rev" part to be
> exactly 40 characters?
> Or do you propose we put the URL in a blob and put the object sha1
> in there.  If so, who's going to create these blobs for the git://
> and ssh:// protocols?  upload-pack?

There was a thread about adding symrefs to this as:

ref: refs/heads/master    HEAD
c5c64e3fe48302f0c4581985f9c68d615f7bcb4e        refs/heads/master

we could extend this for subproject support as:

subproject: kernel:URL    refs/heads/master
subproject: gcc:URL    refs/heads/master

And it allows to define different URL for different branches
(linux-2.6.git for master and linux-2.4.git for old, for example).

Just my 0.02 cents.

Santi

>
> Thanks for the discussion, btw.
> I hope we can come up with something that's acceptable to everyone.
>
> skimo
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [RFC] Fourth round of support for cloning submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 1273 bytes --]

hoi :)

On Thu, May 24, 2007 at 12:02:41PM +0100, Johannes Schindelin wrote:
> On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > On Thu, May 24, 2007 at 10:41:30AM +0100, Johannes Schindelin wrote:
> > > On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > > > You mean like a tag "submodules" that points to a text file
> > > > describing the submodules?
> > > > That's a bit of a pain to set up since you would want that
> > > > to be independent of your project.
> > > 
> > > I could imagine this to be another extension of ls-remote.
> > 
> > You mean extending upload-pack ?  Junio mentioned this possibility as well.
> > This only solves the git:// and ssh:// case though.
> > What to do with the other protocols?
> 
> As we do for the refs: put it into .git/info/refs. This file is already 
> meant to "cache" the output of ls-remote for dumb protocols.

why on earth do you want to store a subproject SHA1 in the
superproject.git?

The subproject only exists as a part of the superproject tree and thus
there is no "the subproject SHA1 which is used by the superproject".
It just doesn't work as you can have several superproject branches with
different subproject.  And even the superproject can itself be a
subproject.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Sun, May 27, 2007 at 10:34:42PM +0200, Martin Waitz wrote:
> On Thu, May 24, 2007 at 12:02:41PM +0100, Johannes Schindelin wrote:
> > As we do for the refs: put it into .git/info/refs. This file is already 
> > meant to "cache" the output of ls-remote for dumb protocols.
> 
> why on earth do you want to store a subproject SHA1 in the
> superproject.git?

Did you read that sentence after writing it?

Anyway, we already concluded further down the thread that
it wouldn't make sense to put any particular subproject SHA-1
in the superproject's info/refs.

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 838 bytes --]

hoi :)

On Wed, May 23, 2007 at 05:50:42PM -0700, Junio C Hamano wrote:
> Honestly speaking, I do not think people have no business
> peeking into configuratoin remote repository has, and it would
> be preferrable that supermodule Porcelain stuff does not rely on
> that.

yes.
I would greatly appreciate having one standard repository format,
also for subprojects so that it is not possible to look into the
remote configuration.

However I do see that the site administrator might want to physically
separate the different subprojects which is difficult to support
in one big monolithic repository.  But if additional information
about the physical location of subprojects is neccessary, then
that should be stored in something like .git/info/subprojects or
similiar, instead of the .git/config.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 12:40:20AM +0100, Johannes Schindelin wrote:
> On Thu, 24 May 2007, skimo@liacs.nl wrote:
> > This patch series implements a mechanism for cloning submodules.
> > Each submodule is specified by a 'submodule.<submodule>.url'
> > configuration option, e.g.,
> > 
> > bash-3.00$ ./git-config --remote=http://www.liacs.nl/~sverdool/isa.git --get-regexp 'submodule\..*\.url' 
> > submodule.cloog.url /home/sverdool/public_html/cloog.git
> > submodule.cloog.url http://www.liacs.nl/~sverdool/cloog.git
> 
> I am sorry to complain so late in the game, but I am not really interested 
> in submodules. However, what you say here is not a task for git-config 
> IMHO, but rather for git-remote.

Hmmm... git-remote does only local configuration and never gets
any information from the other side.
What would be the interface and how would you get the information?

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, Sven Verdoolaege wrote:

> On Thu, May 24, 2007 at 12:40:20AM +0100, Johannes Schindelin wrote:
> > On Thu, 24 May 2007, skimo@liacs.nl wrote:
> > > This patch series implements a mechanism for cloning submodules.
> > > Each submodule is specified by a 'submodule.<submodule>.url'
> > > configuration option, e.g.,
> > > 
> > > bash-3.00$ ./git-config --remote=http://www.liacs.nl/~sverdool/isa.git --get-regexp 'submodule\..*\.url' 
> > > submodule.cloog.url /home/sverdool/public_html/cloog.git
> > > submodule.cloog.url http://www.liacs.nl/~sverdool/cloog.git
> > 
> > I am sorry to complain so late in the game, but I am not really interested 
> > in submodules. However, what you say here is not a task for git-config 
> > IMHO, but rather for git-remote.
> 
> Hmmm... git-remote does only local configuration and never gets
> any information from the other side.
> What would be the interface and how would you get the information?

I was complaining that git-config, which is purely a local beast, gets 
infected with even more obscure stuff. Junio mentions regularly that he 
does not trust git-config that much, and given the number of fixes we 
still get, I have to agree. So let's not put any more stuff into that 
kitchen sink, especially if it has nothing to do with the configuration of 
your repo.

Ciao,
Dscho

Re: [RFC] Fourth round of support for cloning submodules

[Sven Verdoolaege]

On Thu, May 24, 2007 at 10:35:33AM +0100, Johannes Schindelin wrote:
> I was complaining that git-config, which is purely a local beast, gets 
> infected with even more obscure stuff. Junio mentions regularly that he 
> does not trust git-config that much, and given the number of fixes we 
> still get, I have to agree. So let's not put any more stuff into that 
> kitchen sink, especially if it has nothing to do with the configuration of 
> your repo.

Well, the name has been changed from git-repo-config some time ago...

skimo

Re: [RFC] Fourth round of support for cloning submodules

[Petr Baudis]

On Thu, May 24, 2007 at 11:35:33AM CEST, Johannes Schindelin wrote:
> Hi,
> 
> On Thu, 24 May 2007, Sven Verdoolaege wrote:
> 
> > On Thu, May 24, 2007 at 12:40:20AM +0100, Johannes Schindelin wrote:
> > > On Thu, 24 May 2007, skimo@liacs.nl wrote:
> > > > This patch series implements a mechanism for cloning submodules.
> > > > Each submodule is specified by a 'submodule.<submodule>.url'
> > > > configuration option, e.g.,
> > > > 
> > > > bash-3.00$ ./git-config --remote=http://www.liacs.nl/~sverdool/isa.git --get-regexp 'submodule\..*\.url' 
> > > > submodule.cloog.url /home/sverdool/public_html/cloog.git
> > > > submodule.cloog.url http://www.liacs.nl/~sverdool/cloog.git
> > > 
> > > I am sorry to complain so late in the game, but I am not really interested 
> > > in submodules. However, what you say here is not a task for git-config 
> > > IMHO, but rather for git-remote.
> > 
> > Hmmm... git-remote does only local configuration and never gets
> > any information from the other side.
> > What would be the interface and how would you get the information?
> 
> I was complaining that git-config, which is purely a local beast, gets 
> infected with even more obscure stuff. Junio mentions regularly that he 
> does not trust git-config that much, and given the number of fixes we 
> still get, I have to agree. So let's not put any more stuff into that 
> kitchen sink, especially if it has nothing to do with the configuration of 
> your repo.

Then again, git-remote is purely a local beast too, isn't it?

We could use git-remote-config for accessing remote configuration (if
it's a good idea at all, which I'm not totally convinced about)... (And
hope people don't confuse it with stuff related to git-remote too much.)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: [RFC] Fourth round of support for cloning submodules

[Johannes Schindelin]

Hi,

On Thu, 24 May 2007, Petr Baudis wrote:

> On Thu, May 24, 2007 at 11:35:33AM CEST, Johannes Schindelin wrote:
> > Hi,
> > 
> > On Thu, 24 May 2007, Sven Verdoolaege wrote:
> > 
> > > On Thu, May 24, 2007 at 12:40:20AM +0100, Johannes Schindelin wrote:
> > > > On Thu, 24 May 2007, skimo@liacs.nl wrote:
> > > > > This patch series implements a mechanism for cloning submodules.
> > > > > Each submodule is specified by a 'submodule.<submodule>.url'
> > > > > configuration option, e.g.,
> > > > > 
> > > > > bash-3.00$ ./git-config --remote=http://www.liacs.nl/~sverdool/isa.git --get-regexp 'submodule\..*\.url' 
> > > > > submodule.cloog.url /home/sverdool/public_html/cloog.git
> > > > > submodule.cloog.url http://www.liacs.nl/~sverdool/cloog.git
> > > > 
> > > > I am sorry to complain so late in the game, but I am not really interested 
> > > > in submodules. However, what you say here is not a task for git-config 
> > > > IMHO, but rather for git-remote.
> > > 
> > > Hmmm... git-remote does only local configuration and never gets
> > > any information from the other side.
> > > What would be the interface and how would you get the information?
> > 
> > I was complaining that git-config, which is purely a local beast, gets 
> > infected with even more obscure stuff. Junio mentions regularly that he 
> > does not trust git-config that much, and given the number of fixes we 
> > still get, I have to agree. So let's not put any more stuff into that 
> > kitchen sink, especially if it has nothing to do with the configuration of 
> > your repo.
> 
> Then again, git-remote is purely a local beast too, isn't it?

No. Think "update" and "prune". Both access the remote site.

> We could use git-remote-config for accessing remote configuration (if 
> it's a good idea at all, which I'm not totally convinced about)... (And 
> hope people don't confuse it with stuff related to git-remote too much.)

As Junio _already_ said, people have no business looking in other people's 
configuration.

If you have something to publish, do so. But don't make the tool publish 
_everything_ by default.

Ciao,
Dscho