[RFC] Third round of support for cloning submodules

[RFC] Third round of support for cloning submodules

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

This patch series implements a mechanism for cloning submodules.
Each submodule is specified by a 'submodule.<submodule>.url'
configuration option, e.g.,

bash-3.00$ ./git-config --remote=http://www.liacs.nl/~sverdool/isa.git --get-regexp 'submodule\..*\.url' 
submodule.cloog.url /home/sverdool/public_html/cloog.git
submodule.cloog.url http://www.liacs.nl/~sverdool/cloog.git

git-checkout will use the first url that works.
E.g., a

git clone --submodules ssh://liacs/~/public_html/isa.git

followed by

git checkout origin/submodule

(which only works for me), will use the first url, while a

git clone --submodules http://www.liacs.nl/~sverdool/isa.git

followed by

git checkout origin/submodule

will use the second.

The cloning of submodules is now handled inside git-checkout.

I currently do not fetch after the initial clone, since
I'm not sure what ref to use for the revision I need to
fetch for the supermodule.
Suggestions are welcome.

Note that this is still WIP, so there is no need to remind
me that I still need to write documentation and tests.

skimo

[PATCH 01/15] Add dump-config

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

This command dumps the config of a repository and will be used
to read config options from a remote site.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 .gitignore                        |    1 +
 Documentation/cmd-list.perl       |    1 +
 Documentation/git-dump-config.txt |   37 +++++++++++++++++++++++++++++++++++++
 Makefile                          |    1 +
 daemon.c                          |    7 +++++++
 dump-config.c                     |   29 +++++++++++++++++++++++++++++
 6 files changed, 76 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/git-dump-config.txt
 create mode 100644 dump-config.c

diff --git a/.gitignore b/.gitignore
index 4dc0c39..d4e5492 100644
--- a/.gitignore
+++ b/.gitignore
@@ -38,6 +38,7 @@ git-diff-files
 git-diff-index
 git-diff-tree
 git-describe
+git-dump-config
 git-fast-import
 git-fetch
 git-fetch--tool
diff --git a/Documentation/cmd-list.perl b/Documentation/cmd-list.perl
index 443802a..fa04615 100755
--- a/Documentation/cmd-list.perl
+++ b/Documentation/cmd-list.perl
@@ -103,6 +103,7 @@ git-diff-files                          plumbinginterrogators
 git-diff-index                          plumbinginterrogators
 git-diff                                mainporcelain
 git-diff-tree                           plumbinginterrogators
+git-dump-config                         synchelpers
 git-fast-import				ancillarymanipulators
 git-fetch                               mainporcelain
 git-fetch-pack                          synchingrepositories
diff --git a/Documentation/git-dump-config.txt b/Documentation/git-dump-config.txt
new file mode 100644
index 0000000..370781c
--- /dev/null
+++ b/Documentation/git-dump-config.txt
@@ -0,0 +1,37 @@
+git-dump-config(1)
+====================
+
+NAME
+----
+git-dump-config - Dump config options
+
+
+SYNOPSIS
+--------
+'git-dump-config' <directory>
+
+DESCRIPTION
+-----------
+Invoked by 'git-config --remote' and dumps the config file to the
+other end over the git protocol.
+
+This command is usually not invoked directly by the end user.  The UI
+for the protocol is on the 'git-config' side, where it is used to get
+options from a remote repository.
+
+OPTIONS
+-------
+<directory>::
+	The repository to get the config options from.
+
+Author
+------
+Written by Sven Verdoolaege.
+
+Documentation
+--------------
+Documentation by Sven Verdoolaege.
+
+GIT
+---
+Part of the gitlink:git[7] suite
diff --git a/Makefile b/Makefile
index 29243c6..37eb861 100644
--- a/Makefile
+++ b/Makefile
@@ -240,6 +240,7 @@ PROGRAMS = \
 	git-fast-import$X \
 	git-merge-base$X \
 	git-daemon$X \
+	git-dump-config$X \
 	git-merge-index$X git-mktag$X git-mktree$X git-patch-id$X \
 	git-peek-remote$X git-receive-pack$X \
 	git-send-pack$X git-shell$X \
diff --git a/daemon.c b/daemon.c
index e74ecac..3e5ebf3 100644
--- a/daemon.c
+++ b/daemon.c
@@ -378,10 +378,17 @@ static int receive_pack(void)
 	return -1;
 }
 
+static int dump_config(void)
+{
+	execl_git_cmd("dump-config", ".", NULL);
+	return -1;
+}
+
 static struct daemon_service daemon_service[] = {
 	{ "upload-archive", "uploadarch", upload_archive, 0, 1 },
 	{ "upload-pack", "uploadpack", upload_pack, 1, 1 },
 	{ "receive-pack", "receivepack", receive_pack, 0, 1 },
+	{ "dump-config", "dumpconfig", dump_config, 0, 1 },
 };
 
 static void enable_service(const char *name, int ena) {
diff --git a/dump-config.c b/dump-config.c
new file mode 100644
index 0000000..355920d
--- /dev/null
+++ b/dump-config.c
@@ -0,0 +1,29 @@
+#include "git-compat-util.h"
+#include "cache.h"
+#include "pkt-line.h"
+
+static const char dump_config_usage[] = "git-dump-config <dir>";
+
+static int dump_config(const char *var, const char *value)
+{
+	packet_write(1, "%s", var);
+	packet_write(1, "%s", value);
+	return 0;
+}
+
+int main(int argc, char **argv)
+{
+	char *dir;
+
+	if (argc != 2)
+		usage(dump_config_usage);
+
+	dir = argv[1];
+	if (!enter_repo(dir, 0))
+		die("'%s': unable to chdir or not a git archive", dir);
+
+	git_config(dump_config);
+	packet_flush(1);
+
+	return 0;
+}
-- 
1.5.2.rc3.815.g8fc2

[PATCH 02/15] git-config: add --remote option for reading config from remote repo

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 Documentation/git-config.txt |   33 +++++++++++++++++++++---------
 builtin-config.c             |   44 ++++++++++++++++++++++++++++++++---------
 cache.h                      |    1 +
 config.c                     |   26 ++++++++++++++++++++++++
 4 files changed, 84 insertions(+), 20 deletions(-)

diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt
index 280ef20..76398ab 100644
--- a/Documentation/git-config.txt
+++ b/Documentation/git-config.txt
@@ -9,16 +9,25 @@ git-config - Get and set repository or global options
 SYNOPSIS
 --------
 [verse]
-'git-config' [--system | --global] [type] name [value [value_regex]]
-'git-config' [--system | --global] [type] --add name value
-'git-config' [--system | --global] [type] --replace-all name [value [value_regex]]
-'git-config' [--system | --global] [type] --get name [value_regex]
-'git-config' [--system | --global] [type] --get-all name [value_regex]
-'git-config' [--system | --global] [type] --unset name [value_regex]
-'git-config' [--system | --global] [type] --unset-all name [value_regex]
-'git-config' [--system | --global] [type] --rename-section old_name new_name
-'git-config' [--system | --global] [type] --remove-section name
-'git-config' [--system | --global] -l | --list
+'git-config' [--system | --global | --remote=[<host>:]<directory ]
+	     [type] name [value [value_regex]]
+'git-config' [--system | --global | --remote=[<host>:]<directory ]
+	     [type] --add name value
+'git-config' [--system | --global | --remote=[<host>:]<directory ]
+	     [type] --replace-all name [value [value_regex]]
+'git-config' [--system | --global | --remote=[<host>:]<directory ]
+	     [type] --get name [value_regex]
+'git-config' [--system | --global | --remote=[<host>:]<directory ]
+	     [type] --get-all name [value_regex]
+'git-config' [--system | --global | --remote=[<host>:]<directory ]
+	     [type] --unset name [value_regex]
+'git-config' [--system | --global | --remote=[<host>:]<directory ]
+	     [type] --unset-all name [value_regex]
+'git-config' [--system | --global | --remote=[<host>:]<directory ]
+	     [type] --rename-section old_name new_name
+'git-config' [--system | --global | --remote=[<host>:]<directory ]
+	     [type] --remove-section name
+'git-config' [--system | --global | --remote=[<host>:]<directory ] -l | --list
 
 DESCRIPTION
 -----------
@@ -80,6 +89,10 @@ OPTIONS
 	Use system-wide $(prefix)/etc/gitconfig rather than the repository
 	.git/config.
 
+--remote=[<host>:]<directory
+	Use remote config instead of the repository .git/config.
+	Only available for reading options.
+
 --remove-section::
 	Remove the given section from the configuration file.
 
diff --git a/builtin-config.c b/builtin-config.c
index b2515f7..3a1e86c 100644
--- a/builtin-config.c
+++ b/builtin-config.c
@@ -2,8 +2,10 @@
 #include "cache.h"
 
 static const char git_config_set_usage[] =
-"git-config [ --global | --system ] [ --bool | --int ] [--get | --get-all | --get-regexp | --replace-all | --add | --unset | --unset-all] name [value [value_regex]] | --rename-section old_name new_name | --remove-section name | --list";
+"git-config [ --global | --system | --remote=[<host>:]<directory ] "
+"[ --bool | --int ] [--get | --get-all | --get-regexp | --replace-all | --add | --unset | --unset-all] name [value [value_regex]] | --rename-section old_name new_name | --remove-section name | --list";
 
+static char *dest;
 static char *key;
 static regex_t *key_regexp;
 static regex_t *regexp;
@@ -104,15 +106,19 @@ static int get_value(const char* key_, const char* regex_)
 		}
 	}
 
-	if (do_all && system_wide)
-		git_config_from_file(show_config, system_wide);
-	if (do_all && global)
-		git_config_from_file(show_config, global);
-	git_config_from_file(show_config, local);
-	if (!do_all && !seen && global)
-		git_config_from_file(show_config, global);
-	if (!do_all && !seen && system_wide)
-		git_config_from_file(show_config, system_wide);
+	if (dest)
+		git_config_from_remote(show_config, dest);
+	else {
+		if (do_all && system_wide)
+			git_config_from_file(show_config, system_wide);
+		if (do_all && global)
+			git_config_from_file(show_config, global);
+		git_config_from_file(show_config, local);
+		if (!do_all && !seen && global)
+			git_config_from_file(show_config, global);
+		if (!do_all && !seen && system_wide)
+			git_config_from_file(show_config, system_wide);
+	}
 
 	free(key);
 	if (regexp) {
@@ -155,8 +161,14 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 		}
 		else if (!strcmp(argv[1], "--system"))
 			setenv("GIT_CONFIG", ETC_GITCONFIG, 1);
+		else if (!prefixcmp(argv[1], "--remote="))
+			dest = xstrdup(argv[1]+9);
 		else if (!strcmp(argv[1], "--rename-section")) {
 			int ret;
+			if (dest) {
+				fprintf(stderr, "Cannot rename on remote\n");
+				return 1;
+			}
 			if (argc != 4)
 				usage(git_config_set_usage);
 			ret = git_config_rename_section(argv[2], argv[3]);
@@ -170,6 +182,10 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 		}
 		else if (!strcmp(argv[1], "--remove-section")) {
 			int ret;
+			if (dest) {
+				fprintf(stderr, "Cannot remove on remote\n");
+				return 1;
+			}
 			if (argc != 3)
 				usage(git_config_set_usage);
 			ret = git_config_rename_section(argv[2], NULL);
@@ -191,6 +207,10 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 	case 2:
 		return get_value(argv[1], NULL);
 	case 3:
+		if (dest && prefixcmp(argv[1], "--get")) {
+			fprintf(stderr, "Cannot (un)set on remote\n");
+			return 1;
+		}
 		if (!strcmp(argv[1], "--unset"))
 			return git_config_set(argv[2], NULL);
 		else if (!strcmp(argv[1], "--unset-all"))
@@ -209,6 +229,10 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 
 			return git_config_set(argv[1], argv[2]);
 	case 4:
+		if (dest && prefixcmp(argv[1], "--get")) {
+			fprintf(stderr, "Cannot (un)set on remote\n");
+			return 1;
+		}
 		if (!strcmp(argv[1], "--unset"))
 			return git_config_set_multivar(argv[2], NULL, argv[3], 0);
 		else if (!strcmp(argv[1], "--unset-all"))
diff --git a/cache.h b/cache.h
index 65b4685..452aa89 100644
--- a/cache.h
+++ b/cache.h
@@ -501,6 +501,7 @@ extern int update_server_info(int);
 typedef int (*config_fn_t)(const char *, const char *);
 extern int git_default_config(const char *, const char *);
 extern int git_config_from_file(config_fn_t fn, const char *);
+extern int git_config_from_remote(config_fn_t fn, char *dest);
 extern int git_config(config_fn_t fn);
 extern int git_config_int(const char *, const char *);
 extern int git_config_bool(const char *, const char *);
diff --git a/config.c b/config.c
index 0614c2b..dbfae3f 100644
--- a/config.c
+++ b/config.c
@@ -6,9 +6,12 @@
  *
  */
 #include "cache.h"
+#include "pkt-line.h"
 
 #define MAXNAME (256)
 
+static const char *dumpconfig = "git-dump-config";
+
 static FILE *config_file;
 static const char *config_file_name;
 static int config_linenr;
@@ -403,6 +406,29 @@ int git_config_from_file(config_fn_t fn, const char *filename)
 	return ret;
 }
 
+int git_config_from_remote(config_fn_t fn, char *dest)
+{
+	int ret;
+	int fd[2];
+	pid_t pid;
+	static char var[MAXNAME];
+	static char value[1024];
+
+	pid = git_connect(fd, dest, dumpconfig);
+	if (pid < 0)
+		return 1;
+	ret = 0;
+	while (packet_read_line(fd[0], var, sizeof(var))) {
+		if (!packet_read_line(fd[0], value, sizeof(value)))
+			die("Missing value");
+		fn(var, value);
+	}
+	close(fd[0]);
+	close(fd[1]);
+	ret |= finish_connect(pid);
+	return !!ret;
+}
+
 int git_config(config_fn_t fn)
 {
 	int ret = 0;
-- 
1.5.2.rc3.815.g8fc2

Re: [PATCH 02/15] git-config: add --remote option for reading config from remote repo

[Frank Lichtenheld]

On Sun, May 20, 2007 at 08:04:35PM +0200, skimo@liacs.nl wrote:
> From: Sven Verdoolaege <skimo@kotnet.org>
> 
> Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
> ---
>  Documentation/git-config.txt |   33 +++++++++++++++++++++---------

All my old suggestions and corrections for the documentation
part still apply... should I repeat them?

Gruesse,
-- 
Frank Lichtenheld <frank@lichtenheld.de>
www: http://www.djpig.de/

Re: [PATCH 02/15] git-config: add --remote option for reading config from remote repo

[Sven Verdoolaege]

On Sun, May 20, 2007 at 08:11:55PM +0200, Frank Lichtenheld wrote:
> On Sun, May 20, 2007 at 08:04:35PM +0200, skimo@liacs.nl wrote:
> > From: Sven Verdoolaege <skimo@kotnet.org>
> > 
> > Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
> > ---
> >  Documentation/git-config.txt |   33 +++++++++++++++++++++---------
> 
> All my old suggestions and corrections for the documentation
> part still apply... should I repeat them?

I did the [scope] thing, but it seems I inadvertedly threw it out.
I guess I'll have to do it again.

Was there anything else?

skimo

Re: [PATCH 02/15] git-config: add --remote option for reading config from remote repo

[Frank Lichtenheld]

On Sun, May 20, 2007 at 09:44:48PM +0200, Sven Verdoolaege wrote:
> On Sun, May 20, 2007 at 08:11:55PM +0200, Frank Lichtenheld wrote:
> > On Sun, May 20, 2007 at 08:04:35PM +0200, skimo@liacs.nl wrote:
> > > From: Sven Verdoolaege <skimo@kotnet.org>
> > > 
> > > Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
> > > ---
> > >  Documentation/git-config.txt |   33 +++++++++++++++++++++---------
> > 
> > All my old suggestions and corrections for the documentation
> > part still apply... should I repeat them?
> 
> I did the [scope] thing, but it seems I inadvertedly threw it out.
> I guess I'll have to do it again.
> 
> Was there anything else?

You list --remote for all variants while it is not really supported for
them. Although this problem probably implicetly goes away if you use
scope and later explain what that means.

Gruesse,
-- 
Frank Lichtenheld <frank@lichtenheld.de>
www: http://www.djpig.de/

[PATCH 03/15] http.h: make fill_active_slots a function pointer

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

This allows us to use the methods provided by http.c
from within libgit, in particular config.c.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 http-fetch.c |    5 ++++-
 http-push.c  |    5 ++++-
 http.h       |    2 +-
 3 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/http-fetch.c b/http-fetch.c
index 09baedc..53fb2a9 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -317,7 +317,7 @@ static void release_object_request(struct object_request *obj_req)
 }
 
 #ifdef USE_CURL_MULTI
-void fill_active_slots(void)
+static void fetch_fill_active_slots(void)
 {
 	struct object_request *obj_req = object_queue_head;
 	struct active_request_slot *slot = active_queue_head;
@@ -1031,6 +1031,9 @@ int main(int argc, const char **argv)
 	}
 	url = argv[arg];
 
+#ifdef USE_CURL_MULTI
+	fill_active_slots = fetch_fill_active_slots;
+#endif
 	http_init();
 
 	no_pragma_header = curl_slist_append(no_pragma_header, "Pragma:");
diff --git a/http-push.c b/http-push.c
index e3f7675..d4c850b 100644
--- a/http-push.c
+++ b/http-push.c
@@ -794,7 +794,7 @@ static void finish_request(struct transfer_request *request)
 }
 
 #ifdef USE_CURL_MULTI
-void fill_active_slots(void)
+static void push_fill_active_slots(void)
 {
 	struct transfer_request *request = request_queue_head;
 	struct transfer_request *next;
@@ -2355,6 +2355,9 @@ int main(int argc, char **argv)
 
 	memset(remote_dir_exists, -1, 256);
 
+#ifdef USE_CURL_MULTI
+	fill_active_slots = push_fill_active_slots;
+#endif
 	http_init();
 
 	no_pragma_header = curl_slist_append(no_pragma_header, "Pragma:");
diff --git a/http.h b/http.h
index 69b6b66..7a41cde 100644
--- a/http.h
+++ b/http.h
@@ -69,7 +69,7 @@ extern void finish_all_active_slots(void);
 extern void release_active_slot(struct active_request_slot *slot);
 
 #ifdef USE_CURL_MULTI
-extern void fill_active_slots(void);
+extern void (*fill_active_slots)(void);
 extern void step_active_slots(void);
 #endif
 
-- 
1.5.2.rc3.815.g8fc2

[PATCH 04/15] git-config: read remote config files over HTTP

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 Makefile           |    7 ++++++-
 builtin-config.c   |    8 ++++++--
 config.c           |   16 +++++++++++++++-
 http.c             |   10 ++++++----
 http.h             |    2 +-
 http_config.h      |    1 +
 http_config_curl.c |   52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 http_config_none.c |    6 ++++++
 8 files changed, 93 insertions(+), 9 deletions(-)
 create mode 100644 http_config.h
 create mode 100644 http_config_curl.c
 create mode 100644 http_config_none.c

diff --git a/Makefile b/Makefile
index 37eb861..bce8514 100644
--- a/Makefile
+++ b/Makefile
@@ -319,7 +319,8 @@ LIB_OBJS = \
 	write_or_die.o trace.o list-objects.o grep.o match-trees.o \
 	alloc.o merge-file.o path-list.o help.o unpack-trees.o $(DIFF_OBJS) \
 	color.o wt-status.o archive-zip.o archive-tar.o shallow.o utf8.o \
-	convert.o attr.o decorate.o progress.o mailmap.o symlinks.o
+	convert.o attr.o decorate.o progress.o mailmap.o symlinks.o \
+	$(HTTP_CONFIG_OBJ)
 
 BUILTIN_OBJS = \
 	builtin-add.o \
@@ -526,6 +527,10 @@ ifndef NO_CURL
 	ifndef NO_EXPAT
 		EXPAT_LIBEXPAT = -lexpat
 	endif
+	HTTP_CONFIG_OBJ = http_config_curl.o http.o
+	EXTLIBS += $(CURL_LIBCURL)
+else
+	HTTP_CONFIG_OBJ = http_config_none.o
 endif
 
 ifndef NO_OPENSSL
diff --git a/builtin-config.c b/builtin-config.c
index 3a1e86c..7e18f73 100644
--- a/builtin-config.c
+++ b/builtin-config.c
@@ -147,8 +147,12 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 			type = T_INT;
 		else if (!strcmp(argv[1], "--bool"))
 			type = T_BOOL;
-		else if (!strcmp(argv[1], "--list") || !strcmp(argv[1], "-l"))
-			return git_config(show_all_config);
+		else if (!strcmp(argv[1], "--list") || !strcmp(argv[1], "-l")) {
+			if (dest)
+				return git_config_from_remote(show_all_config, dest);
+			else
+				return git_config(show_all_config);
+		}
 		else if (!strcmp(argv[1], "--global")) {
 			char *home = getenv("HOME");
 			if (home) {
diff --git a/config.c b/config.c
index dbfae3f..fc2162b 100644
--- a/config.c
+++ b/config.c
@@ -7,6 +7,7 @@
  */
 #include "cache.h"
 #include "pkt-line.h"
+#include "http_config.h"
 
 #define MAXNAME (256)
 
@@ -406,6 +407,16 @@ int git_config_from_file(config_fn_t fn, const char *filename)
 	return ret;
 }
 
+static int config_from_http(config_fn_t fn, char *dest)
+{
+	char config_temp[50];
+	if (git_http_fetch_config(dest, config_temp, sizeof(config_temp)))
+		return 1;
+	git_config_from_file(fn, config_temp);
+	unlink(config_temp);
+	return 0;
+}
+
 int git_config_from_remote(config_fn_t fn, char *dest)
 {
 	int ret;
@@ -414,7 +425,10 @@ int git_config_from_remote(config_fn_t fn, char *dest)
 	static char var[MAXNAME];
 	static char value[1024];
 
-	pid = git_connect(fd, dest, dumpconfig);
+	if (!prefixcmp(dest, "http://"))
+		return config_from_http(fn, dest);
+
+	pid = git_connect(fd, dest, dumpconfig, 0);
 	if (pid < 0)
 		return 1;
 	ret = 0;
diff --git a/http.c b/http.c
index ae27e0c..c8237cb 100644
--- a/http.c
+++ b/http.c
@@ -25,6 +25,8 @@ long curl_low_speed_limit = -1;
 long curl_low_speed_time = -1;
 int curl_ftp_no_epsv = 0;
 
+void (*fill_active_slots)(void) = NULL;
+
 struct curl_slist *pragma_header;
 
 struct active_request_slot *active_queue_head = NULL;
@@ -394,7 +396,8 @@ void step_active_slots(void)
 	} while (curlm_result == CURLM_CALL_MULTI_PERFORM);
 	if (num_transfers < active_requests) {
 		process_curl_messages();
-		fill_active_slots();
+		if (fill_active_slots)
+			fill_active_slots();
 	}
 }
 #endif
@@ -458,9 +461,8 @@ void release_active_slot(struct active_request_slot *slot)
 		curl_easy_cleanup(slot->curl);
 		slot->curl = NULL;
 	}
-#ifdef USE_CURL_MULTI
-	fill_active_slots();
-#endif
+	if (fill_active_slots)
+		fill_active_slots();
 }
 
 static void finish_active_slot(struct active_request_slot *slot)
diff --git a/http.h b/http.h
index 7a41cde..7f29ff8 100644
--- a/http.h
+++ b/http.h
@@ -68,8 +68,8 @@ extern void run_active_slot(struct active_request_slot *slot);
 extern void finish_all_active_slots(void);
 extern void release_active_slot(struct active_request_slot *slot);
 
-#ifdef USE_CURL_MULTI
 extern void (*fill_active_slots)(void);
+#ifdef USE_CURL_MULTI
 extern void step_active_slots(void);
 #endif
 
diff --git a/http_config.h b/http_config.h
new file mode 100644
index 0000000..25f5c19
--- /dev/null
+++ b/http_config.h
@@ -0,0 +1 @@
+int git_http_fetch_config(const char *repo, char *config_file, int len);
diff --git a/http_config_curl.c b/http_config_curl.c
new file mode 100644
index 0000000..88317cf
--- /dev/null
+++ b/http_config_curl.c
@@ -0,0 +1,52 @@
+#include "http_config.h"
+#include "http.h"
+
+int git_http_fetch_config(const char *repo, char *config, int config_len)
+{
+	char url[PATH_MAX];
+	int len = strlen(repo);
+
+	int fd;
+	FILE *configfile;
+	struct active_request_slot *slot;
+	struct slot_results results;
+
+	strcpy(url, repo);
+	while (len > 0 && url[len-1] == '/')
+		--len;
+	snprintf(url+len, sizeof(url)-len, "/config");
+
+	fd = git_mkstemp(config, config_len, ".config_XXXXXX");
+	if (fd >= 0)
+		configfile = fdopen(fd, "w");
+	if (fd < 0 || !configfile)
+		return error("Unable to open local file %s for config",
+			     config);
+
+	http_init();
+
+	slot = get_active_slot();
+	slot->results = &results;
+	curl_easy_setopt(slot->curl, CURLOPT_FILE, configfile);
+	curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite);
+	curl_easy_setopt(slot->curl, CURLOPT_URL, url);
+	slot->local = configfile;
+
+	if (start_active_slot(slot)) {
+		run_active_slot(slot);
+		if (results.curl_result != CURLE_OK) {
+			fclose(configfile);
+			warning("Unable to get config %s\n%s", url,
+				 curl_errorstr);
+		}
+	} else {
+		fclose(configfile);
+		return error("Unable to start request");
+	}
+
+	http_cleanup();
+
+	fclose(configfile);
+
+	return 0;
+}
diff --git a/http_config_none.c b/http_config_none.c
new file mode 100644
index 0000000..860ae84
--- /dev/null
+++ b/http_config_none.c
@@ -0,0 +1,6 @@
+#include "http_config.h"
+
+int git_http_fetch_config(const char *repo, char *config_file, int len)
+{
+	return error("Reading http config files not supported");
+}
-- 
1.5.2.rc3.815.g8fc2

[PATCH 05/15] unpack-trees.c: pass cache_entry * to verify_absent rather than just the name

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

We will need the full cache_entry later to figure out if we are dealing
with a submodule.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 unpack-trees.c |   32 ++++++++++++++++----------------
 1 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/unpack-trees.c b/unpack-trees.c
index 906ce69..317f656 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -491,7 +491,7 @@ static int verify_clean_subdirectory(const char *path, const char *action,
  * We do not want to remove or overwrite a working tree file that
  * is not tracked, unless it is ignored.
  */
-static void verify_absent(const char *path, const char *action,
+static void verify_absent(struct cache_entry *ce, const char *action,
 		struct unpack_trees_options *o)
 {
 	struct stat st;
@@ -499,12 +499,12 @@ static void verify_absent(const char *path, const char *action,
 	if (o->index_only || o->reset || !o->update)
 		return;
 
-	if (!lstat(path, &st)) {
+	if (!lstat(ce->name, &st)) {
 		int cnt;
 
-		if (o->dir && excluded(o->dir, path))
+		if (o->dir && excluded(o->dir, ce->name))
 			/*
-			 * path is explicitly excluded, so it is Ok to
+			 * ce->name is explicitly excluded, so it is Ok to
 			 * overwrite it.
 			 */
 			return;
@@ -516,7 +516,7 @@ static void verify_absent(const char *path, const char *action,
 			 * files that are in "foo/" we would lose
 			 * it.
 			 */
-			cnt = verify_clean_subdirectory(path, action, o);
+			cnt = verify_clean_subdirectory(ce->name, action, o);
 
 			/*
 			 * If this removed entries from the index,
@@ -544,7 +544,7 @@ static void verify_absent(const char *path, const char *action,
 		 * delete this path, which is in a subdirectory that
 		 * is being replaced with a blob.
 		 */
-		cnt = cache_name_pos(path, strlen(path));
+		cnt = cache_name_pos(ce->name, strlen(ce->name));
 		if (0 <= cnt) {
 			struct cache_entry *ce = active_cache[cnt];
 			if (!ce_stage(ce) && !ce->ce_mode)
@@ -552,7 +552,7 @@ static void verify_absent(const char *path, const char *action,
 		}
 
 		die("Untracked working tree file '%s' "
-		    "would be %s by merge.", path, action);
+		    "would be %s by merge.", ce->name, action);
 	}
 }
 
@@ -576,7 +576,7 @@ static int merged_entry(struct cache_entry *merge, struct cache_entry *old,
 		}
 	}
 	else {
-		verify_absent(merge->name, "overwritten", o);
+		verify_absent(merge, "overwritten", o);
 		invalidate_ce_path(merge);
 	}
 
@@ -591,7 +591,7 @@ static int deleted_entry(struct cache_entry *ce, struct cache_entry *old,
 	if (old)
 		verify_uptodate(old, o);
 	else
-		verify_absent(ce->name, "removed", o);
+		verify_absent(ce, "removed", o);
 	ce->ce_mode = 0;
 	add_cache_entry(ce, ADD_CACHE_OK_TO_ADD|ADD_CACHE_OK_TO_REPLACE);
 	invalidate_ce_path(ce);
@@ -708,18 +708,18 @@ int threeway_merge(struct cache_entry **stages,
 	if (o->aggressive) {
 		int head_deleted = !head && !df_conflict_head;
 		int remote_deleted = !remote && !df_conflict_remote;
-		const char *path = NULL;
+		struct cache_entry *ce = NULL;
 
 		if (index)
-			path = index->name;
+			ce = index;
 		else if (head)
-			path = head->name;
+			ce = head;
 		else if (remote)
-			path = remote->name;
+			ce = remote;
 		else {
 			for (i = 1; i < o->head_idx; i++) {
 				if (stages[i] && stages[i] != o->df_conflict_entry) {
-					path = stages[i]->name;
+					ce = stages[i];
 					break;
 				}
 			}
@@ -734,8 +734,8 @@ int threeway_merge(struct cache_entry **stages,
 		    (remote_deleted && head && head_match)) {
 			if (index)
 				return deleted_entry(index, index, o);
-			else if (path && !head_deleted)
-				verify_absent(path, "removed", o);
+			else if (ce && !head_deleted)
+				verify_absent(ce, "removed", o);
 			return 0;
 		}
 		/*
-- 
1.5.2.rc3.815.g8fc2

[PATCH 06/15] git-read-tree: take --submodules option

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

This option currently has no effect.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 Documentation/config.txt |    4 ++++
 builtin-read-tree.c      |   25 ++++++++++++++++++++++---
 cache.h                  |    3 ++-
 unpack-trees.c           |    1 +
 unpack-trees.h           |    1 +
 5 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index ee1c35e..5d891ac 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -256,6 +256,10 @@ You probably do not need to adjust this value.
 +
 Common unit suffixes of 'k', 'm', or 'g' are supported.
 
+core.submodules
+	If true, gitlink:git-checkout[1] also checks out submodules.
+	False by default.
+
 alias.*::
 	Command aliases for the gitlink:git[1] command wrapper - e.g.
 	after defining "alias.last = cat-file commit HEAD", the invocation
diff --git a/builtin-read-tree.c b/builtin-read-tree.c
index 316fb0f..929dd95 100644
--- a/builtin-read-tree.c
+++ b/builtin-read-tree.c
@@ -87,14 +87,23 @@ static void prime_cache_tree(void)
 static const char read_tree_usage[] = "git-read-tree (<sha> | [[-m [--aggressive] | --reset | --prefix=<prefix>] [-u | -i]] [--exclude-per-directory=<gitignore>] [--index-output=<file>] <sha1> [<sha2> [<sha3>]])";
 
 static struct lock_file lock_file;
+static struct unpack_trees_options opts;
+
+static int git_read_tree_config(const char *var, const char *value)
+{
+	if (!strcmp(var, "core.submodules")) {
+		opts.submodules = git_config_bool(var, value);
+		return 0;
+	}
+
+	return git_default_config(var, value);
+}
 
 int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 {
 	int i, newfd, stage = 0;
 	unsigned char sha1[20];
-	struct unpack_trees_options opts;
 
-	memset(&opts, 0, sizeof(opts));
 	opts.head_idx = -1;
 
 	setup_git_directory();
@@ -102,7 +111,7 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 
 	newfd = hold_locked_index(&lock_file, 1);
 
-	git_config(git_default_config);
+	git_config(git_read_tree_config);
 
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
@@ -172,6 +181,16 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 			continue;
 		}
 
+		if (!strcmp(arg, "--no-submodules")) {
+			opts.submodules = 0;
+			continue;
+		}
+
+		if (!strcmp(arg, "--submodules")) {
+			opts.submodules = 1;
+			continue;
+		}
+
 		/* "-m" stands for "merge", meaning we start in stage 1 */
 		if (!strcmp(arg, "-m")) {
 			if (stage || opts.merge || opts.prefix)
diff --git a/cache.h b/cache.h
index 452aa89..446030a 100644
--- a/cache.h
+++ b/cache.h
@@ -406,7 +406,8 @@ struct checkout {
 	unsigned force:1,
 		 quiet:1,
 		 not_new:1,
-		 refresh_cache:1;
+		 refresh_cache:1,
+		 submodules:1;
 };
 
 extern int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *topath);
diff --git a/unpack-trees.c b/unpack-trees.c
index 317f656..4497a46 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -352,6 +352,7 @@ int unpack_trees(struct object_list *trees, struct unpack_trees_options *o)
 	state.force = 1;
 	state.quiet = 1;
 	state.refresh_cache = 1;
+	state.submodules = o->submodules;
 
 	o->merge_size = len;
 
diff --git a/unpack-trees.h b/unpack-trees.h
index fee7da4..21005d9 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -15,6 +15,7 @@ struct unpack_trees_options {
 	int trivial_merges_only;
 	int verbose_update;
 	int aggressive;
+	int submodules;
 	const char *prefix;
 	int pos;
 	struct dir_struct *dir;
-- 
1.5.2.rc3.815.g8fc2

Re: [PATCH 06/15] git-read-tree: take --submodules option

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

hoi :)

what really is the motivation to suppress submodule checkout at this
level?  I can see that we need some per-submodule option for checkout,
but this should influence the actual checkout process and not
read-tree.

At least we really want to always update the index correctly and a
read-tree --no-submodules  which updates the index for submodules but
doesn't go into existing submodules just feels wrong.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH 06/15] git-read-tree: take --submodules option

[Sven Verdoolaege]

On Sun, May 20, 2007 at 11:24:04PM +0200, Martin Waitz wrote:
> hoi :)
> 
> what really is the motivation to suppress submodule checkout at this
> level?  I can see that we need some per-submodule option for checkout,
> but this should influence the actual checkout process and not
> read-tree.

It's only used with update is set.

> At least we really want to always update the index correctly and a
> read-tree --no-submodules  which updates the index for submodules but
> doesn't go into existing submodules just feels wrong.

It doesn't do that.

skimo

[PATCH 07/15] unpack-trees.c: assume submodules are clean

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

If the submodules are not clean, then we will get an error
when we actally do the checkout.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 unpack-trees.c |   43 ++++++++++++++++++++++++++++++++++---------
 1 files changed, 34 insertions(+), 9 deletions(-)

diff --git a/unpack-trees.c b/unpack-trees.c
index 4497a46..f3fe2dd 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -5,6 +5,7 @@
 #include "cache-tree.h"
 #include "unpack-trees.h"
 #include "progress.h"
+#include "refs.h"
 
 #define DBRT_DEBUG 1
 
@@ -430,11 +431,24 @@ static void invalidate_ce_path(struct cache_entry *ce)
 		cache_tree_invalidate_path(active_cache_tree, ce->name);
 }
 
-static int verify_clean_subdirectory(const char *path, const char *action,
+/* Check that checking out ce->sha1 in subdir ce->name is not
+ * going to overwrite any working files.
+ *
+ * FIXME: implement this function, so we can detect problems
+ *        early, rather than waiting until we actually try to checkout
+ *        the submodules.
+ */
+static int verify_clean_submodule(struct cache_entry *ce, const char *action,
+				      struct unpack_trees_options *o)
+{
+	return 0;
+}
+
+static int verify_clean_subdirectory(struct cache_entry *ce, const char *action,
 				      struct unpack_trees_options *o)
 {
 	/*
-	 * we are about to extract "path"; we would not want to lose
+	 * we are about to extract "ce->name"; we would not want to lose
 	 * anything in the existing directory there.
 	 */
 	int namelen;
@@ -442,13 +456,24 @@ static int verify_clean_subdirectory(const char *path, const char *action,
 	struct dir_struct d;
 	char *pathbuf;
 	int cnt = 0;
+	unsigned char sha1[20];
+
+	if (S_ISDIRLNK(ntohl(ce->ce_mode)) &&
+	    resolve_gitlink_ref(ce->name, "HEAD", sha1) == 0) {
+		/* If we are not going to update the submodule, then
+		 * we don't care.
+		 */
+		if (!o->submodules || !hashcmp(sha1, ce->sha1))
+			return 0;
+		verify_clean_submodule(ce, action, o);
+	}
 
 	/*
 	 * First let's make sure we do not have a local modification
 	 * in that directory.
 	 */
-	namelen = strlen(path);
-	pos = cache_name_pos(path, namelen);
+	namelen = strlen(ce->name);
+	pos = cache_name_pos(ce->name, namelen);
 	if (0 <= pos)
 		return cnt; /* we have it as nondirectory */
 	pos = -pos - 1;
@@ -456,7 +481,7 @@ static int verify_clean_subdirectory(const char *path, const char *action,
 		struct cache_entry *ce = active_cache[i];
 		int len = ce_namelen(ce);
 		if (len < namelen ||
-		    strncmp(path, ce->name, namelen) ||
+		    strncmp(ce->name, ce->name, namelen) ||
 		    ce->name[namelen] != '/')
 			break;
 		/*
@@ -474,16 +499,16 @@ static int verify_clean_subdirectory(const char *path, const char *action,
 	 * present file that is not ignored.
 	 */
 	pathbuf = xmalloc(namelen + 2);
-	memcpy(pathbuf, path, namelen);
+	memcpy(pathbuf, ce->name, namelen);
 	strcpy(pathbuf+namelen, "/");
 
 	memset(&d, 0, sizeof(d));
 	if (o->dir)
 		d.exclude_per_dir = o->dir->exclude_per_dir;
-	i = read_directory(&d, path, pathbuf, namelen+1, NULL);
+	i = read_directory(&d, ce->name, pathbuf, namelen+1, NULL);
 	if (i)
 		die("Updating '%s' would lose untracked files in it",
-		    path);
+		    ce->name);
 	free(pathbuf);
 	return cnt;
 }
@@ -517,7 +542,7 @@ static void verify_absent(struct cache_entry *ce, const char *action,
 			 * files that are in "foo/" we would lose
 			 * it.
 			 */
-			cnt = verify_clean_subdirectory(ce->name, action, o);
+			cnt = verify_clean_subdirectory(ce, action, o);
 
 			/*
 			 * If this removed entries from the index,
-- 
1.5.2.rc3.815.g8fc2

[PATCH 08/15] Add run_command_v_opt_cd: chdir into a directory before exec

[skimo]

From: Alex Riesen <raa.lkml@gmail.com>

It can make code simplier (no need to preserve cwd) and safer
(no chance the cwd of the current process is accidentally forgotten).

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
 run-command.c |   27 ++++++++++++++++++++++-----
 run-command.h |    2 ++
 2 files changed, 24 insertions(+), 5 deletions(-)

diff --git a/run-command.c b/run-command.c
index eff523e..043b570 100644
--- a/run-command.c
+++ b/run-command.c
@@ -73,6 +73,9 @@ int start_command(struct child_process *cmd)
 			close(cmd->out);
 		}
 
+		if (cmd->dir && chdir(cmd->dir))
+			die("exec %s: cd to %s failed (%s)", cmd->argv[0],
+			    cmd->dir, strerror(errno));
 		if (cmd->git_cmd) {
 			execv_git_cmd(cmd->argv);
 		} else {
@@ -133,13 +136,27 @@ int run_command(struct child_process *cmd)
 	return finish_command(cmd);
 }
 
+static void prepare_run_command_v_opt(struct child_process *cmd,
+				      const char **argv, int opt)
+{
+	memset(cmd, 0, sizeof(*cmd));
+	cmd->argv = argv;
+	cmd->no_stdin = opt & RUN_COMMAND_NO_STDIN ? 1 : 0;
+	cmd->git_cmd = opt & RUN_GIT_CMD ? 1 : 0;
+	cmd->stdout_to_stderr = opt & RUN_COMMAND_STDOUT_TO_STDERR ? 1 : 0;
+}
+
 int run_command_v_opt(const char **argv, int opt)
 {
 	struct child_process cmd;
-	memset(&cmd, 0, sizeof(cmd));
-	cmd.argv = argv;
-	cmd.no_stdin = opt & RUN_COMMAND_NO_STDIN ? 1 : 0;
-	cmd.git_cmd = opt & RUN_GIT_CMD ? 1 : 0;
-	cmd.stdout_to_stderr = opt & RUN_COMMAND_STDOUT_TO_STDERR ? 1 : 0;
+	prepare_run_command_v_opt(&cmd, argv, opt);
+	return run_command(&cmd);
+}
+
+int run_command_v_opt_cd(const char **argv, int opt, const char *dir)
+{
+	struct child_process cmd;
+	prepare_run_command_v_opt(&cmd, argv, opt);
+	cmd.dir = dir;
 	return run_command(&cmd);
 }
diff --git a/run-command.h b/run-command.h
index 3680ef9..cbd7484 100644
--- a/run-command.h
+++ b/run-command.h
@@ -16,6 +16,7 @@ struct child_process {
 	pid_t pid;
 	int in;
 	int out;
+	const char *dir;
 	unsigned close_in:1;
 	unsigned close_out:1;
 	unsigned no_stdin:1;
@@ -32,5 +33,6 @@ int run_command(struct child_process *);
 #define RUN_GIT_CMD	     2	/*If this is to be git sub-command */
 #define RUN_COMMAND_STDOUT_TO_STDERR 4
 int run_command_v_opt(const char **argv, int opt);
+int run_command_v_opt_cd(const char **argv, int opt, const char *dir);
 
 #endif
-- 
1.5.2.rc3.815.g8fc2

[PATCH 09/15] entry.c: optionally checkout submodules

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Use run_command_v_opt_cd, as proposed by Alex Riesen.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 entry.c |   32 ++++++++++++++++++++++++++++++--
 1 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/entry.c b/entry.c
index 82bf725..8c70a47 100644
--- a/entry.c
+++ b/entry.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "blob.h"
+#include "run-command.h"
 
 static void create_directories(const char *path, const struct checkout *state)
 {
@@ -75,6 +76,34 @@ static void *read_blob_entry(struct cache_entry *ce, const char *path, unsigned
 	return NULL;
 }
 
+static int checkout_submodule(struct cache_entry *ce, const char *path, const struct checkout *state)
+{
+	const char *gitdirenv;
+	const char *args[10];
+	int argc;
+	int err;
+
+	if (!state->submodules)
+		return 0;
+
+	argc = 0;
+	args[argc++] = "checkout";
+	if (state->force)
+	    args[argc++] = "-f";
+	args[argc++] = sha1_to_hex(ce->sha1);
+	args[argc] = NULL;
+
+	gitdirenv = getenv(GIT_DIR_ENVIRONMENT);
+	unsetenv(GIT_DIR_ENVIRONMENT);
+	err = run_command_v_opt_cd(args, RUN_GIT_CMD, path);
+	setenv(GIT_DIR_ENVIRONMENT, gitdirenv, 1);
+
+	if (err)
+		return error("failed to run git-checkout in submodule '%s'", path);
+
+	return 0;
+}
+
 static int write_entry(struct cache_entry *ce, char *path, const struct checkout *state, int to_tempfile)
 {
 	int fd;
@@ -193,9 +222,8 @@ int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *t
 		 */
 		unlink(path);
 		if (S_ISDIR(st.st_mode)) {
-			/* If it is a gitlink, leave it alone! */
 			if (S_ISDIRLNK(ntohl(ce->ce_mode)))
-				return 0;
+				return checkout_submodule(ce, path, state);
 			if (!state->force)
 				return error("%s is a directory", path);
 			remove_subtree(path);
-- 
1.5.2.rc3.815.g8fc2

Re: [PATCH 09/15] entry.c: optionally checkout submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 284 bytes --]

hoi :)

have you seen my patch to checkout submodules?

The submodule checkout should really check that the requested commit
is really available and have some other path for creating submodules
which are not currently checked out / able to be checked out.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH 09/15] entry.c: optionally checkout submodules

[Sven Verdoolaege]

On Sun, May 20, 2007 at 11:18:50PM +0200, Martin Waitz wrote:
> hoi :)
> 
> have you seen my patch to checkout submodules?

Not yet.  I'll look for it in the morning.

skimo

Re: [PATCH 09/15] entry.c: optionally checkout submodules

[Sven Verdoolaege]

On Sun, May 20, 2007 at 11:18:50PM +0200, Martin Waitz wrote:
> hoi :)
> 
> have you seen my patch to checkout submodules?
> 
> The submodule checkout should really check that the requested commit
> is really available and have some other path for creating submodules
> which are not currently checked out / able to be checked out.

If you are talking about your May 5th patch, then you weren't doing
that either.  You just checked if it _looked_ like a SHA1.

I check for the presence of the commit by calling git-cat-file.
Of course, this doesn't guarantee that the whole tree is available.

skimo

[PATCH 10/15] git-checkout: pass --submodules option to git-read-tree

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 git-checkout.sh |   20 +++++++++++++++-----
 1 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/git-checkout.sh b/git-checkout.sh
index 6b6facf..162cef4 100755
--- a/git-checkout.sh
+++ b/git-checkout.sh
@@ -1,6 +1,6 @@
 #!/bin/sh
 
-USAGE='[-q] [-f] [-b <new_branch>] [-m] [<branch>] [<paths>...]'
+USAGE='[-q] [-f] [--submodules] [--no-submodules] [-b <new_branch>] [-m] [<branch>] [<paths>...]'
 SUBDIRECTORY_OK=Sometimes
 . git-sh-setup
 require_work_tree
@@ -16,6 +16,7 @@ track=
 newbranch=
 newbranch_log=
 merge=
+submodules=
 quiet=
 v=-v
 LF='
@@ -46,6 +47,15 @@ while [ "$#" != "0" ]; do
 	-m)
 		merge=1
 		;;
+	--su|--sub|--subm|--submo|--submod|--submodu|--submodul|\
+	--submodule|--submodules)
+		submodules="--submodules"
+		;;
+	--no-su|--no-sub|--no-subm|--no-submo|--no-submod|\
+	--no-submodu|--no-submodul|\
+	--no-submodule|--no-submodules)
+		submodules="--no-submodules"
+		;;
 	"-q")
 		quiet=1
 		v=
@@ -199,10 +209,10 @@ fi
 
 if [ "$force" ]
 then
-    git-read-tree $v --reset -u $new
+    git-read-tree $v $submodules --reset -u $new
 else
     git-update-index --refresh >/dev/null
-    merge_error=$(git-read-tree -m -u --exclude-per-directory=.gitignore $old $new 2>&1) || (
+    merge_error=$(git-read-tree $submodules -m -u --exclude-per-directory=.gitignore $old $new 2>&1) || (
 	case "$merge" in
 	'')
 		echo >&2 "$merge_error"
@@ -212,7 +222,7 @@ else
 	# Match the index to the working tree, and do a three-way.
     	git diff-files --name-only | git update-index --remove --stdin &&
 	work=`git write-tree` &&
-	git read-tree $v --reset -u $new || exit
+	git read-tree $v $submodules --reset -u $new || exit
 
 	eval GITHEAD_$new='${new_name:-${branch:-$new}}' &&
 	eval GITHEAD_$work=local &&
@@ -223,7 +233,7 @@ else
 	# this is not a real merge before committing, but just carrying
 	# the working tree changes along.
 	unmerged=`git ls-files -u`
-	git read-tree $v --reset $new
+	git read-tree $v $submodules --reset $new
 	case "$unmerged" in
 	'')	;;
 	*)
-- 
1.5.2.rc3.815.g8fc2

[PATCH 11/15] git-read-tree: treat null commit as empty tree

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

---
 builtin-read-tree.c |    9 ++++++---
 unpack-trees.c      |    3 +++
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/builtin-read-tree.c b/builtin-read-tree.c
index 929dd95..b9fcff7 100644
--- a/builtin-read-tree.c
+++ b/builtin-read-tree.c
@@ -17,9 +17,12 @@ static struct object_list *trees;
 
 static int list_tree(unsigned char *sha1)
 {
-	struct tree *tree = parse_tree_indirect(sha1);
-	if (!tree)
-		return -1;
+	struct tree *tree = NULL;
+	if (!is_null_sha1(sha1)) {
+		tree = parse_tree_indirect(sha1);
+		if (!tree)
+			return -1;
+	}
 	object_list_append(&tree->object, &trees);
 	return 0;
 }
diff --git a/unpack-trees.c b/unpack-trees.c
index f3fe2dd..3dadebb 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -26,6 +26,9 @@ static struct tree_entry_list *create_tree_entry_list(struct tree *tree)
 	struct tree_entry_list *ret = NULL;
 	struct tree_entry_list **list_p = &ret;
 
+	if (!tree)
+		return ret;
+
 	if (!tree->object.parsed)
 		parse_tree(tree);
 
-- 
1.5.2.rc3.815.g8fc2

[PATCH 12/15] git_config: add void * for callback data

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

---
 archive-tar.c            |    6 +++---
 builtin-add.c            |    6 +++---
 builtin-apply.c          |    6 +++---
 builtin-blame.c          |    6 +++---
 builtin-branch.c         |   10 +++++-----
 builtin-cat-file.c       |    2 +-
 builtin-checkout-index.c |    2 +-
 builtin-commit-tree.c    |    2 +-
 builtin-config.c         |   21 +++++++++++----------
 builtin-diff-files.c     |    2 +-
 builtin-diff-index.c     |    2 +-
 builtin-diff-tree.c      |    2 +-
 builtin-diff.c           |    2 +-
 builtin-fmt-merge-msg.c  |    5 +++--
 builtin-gc.c             |    6 +++---
 builtin-init-db.c        |    4 ++--
 builtin-log.c            |   18 +++++++++---------
 builtin-ls-files.c       |    2 +-
 builtin-ls-tree.c        |    2 +-
 builtin-mailinfo.c       |    2 +-
 builtin-merge-base.c     |    2 +-
 builtin-mv.c             |    2 +-
 builtin-name-rev.c       |    2 +-
 builtin-pack-objects.c   |    6 +++---
 builtin-push.c           |    4 ++--
 builtin-read-tree.c      |    9 +++++----
 builtin-reflog.c         |    7 ++++---
 builtin-rerere.c         |    6 +++---
 builtin-rev-list.c       |    2 +-
 builtin-rev-parse.c      |    2 +-
 builtin-revert.c         |    2 +-
 builtin-rm.c             |    2 +-
 builtin-runstatus.c      |    2 +-
 builtin-show-branch.c    |    7 ++++---
 builtin-symbolic-ref.c   |    2 +-
 builtin-unpack-objects.c |    2 +-
 builtin-update-index.c   |    2 +-
 builtin-update-ref.c     |    2 +-
 builtin-verify-pack.c    |    2 +-
 cache.h                  |   13 +++++++------
 config.c                 |   36 ++++++++++++++++++------------------
 connect.c                |    7 ++++---
 convert.c                |    5 +++--
 daemon.c                 |    4 ++--
 diff.c                   |    6 +++---
 diff.h                   |    2 +-
 dump-config.c            |    4 ++--
 fast-import.c            |    2 +-
 fetch-pack.c             |    6 +++---
 git.c                    |    4 ++--
 http-fetch.c             |    2 +-
 http.c                   |    6 +++---
 imap-send.c              |    4 ++--
 local-fetch.c            |    2 +-
 merge-recursive.c        |   10 +++++-----
 receive-pack.c           |    6 +++---
 send-pack.c              |    2 +-
 setup.c                  |    5 +++--
 ssh-fetch.c              |    2 +-
 unpack-file.c            |    2 +-
 var.c                    |    8 ++++----
 wt-status.c              |    4 ++--
 wt-status.h              |    2 +-
 63 files changed, 163 insertions(+), 154 deletions(-)

diff --git a/archive-tar.c b/archive-tar.c
index 33e7657..595c153 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -242,7 +242,7 @@ static void write_global_extended_header(const unsigned char *sha1)
 	free(ext_header.buf);
 }
 
-static int git_tar_config(const char *var, const char *value)
+static int git_tar_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "tar.umask")) {
 		if (!strcmp(value, "user")) {
@@ -253,7 +253,7 @@ static int git_tar_config(const char *var, const char *value)
 		}
 		return 0;
 	}
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static int write_tar_entry(const unsigned char *sha1,
@@ -300,7 +300,7 @@ int write_tar_archive(struct archiver_args *args)
 {
 	int plen = args->base ? strlen(args->base) : 0;
 
-	git_config(git_tar_config);
+	git_config(git_tar_config, NULL);
 
 	archive_time = args->time;
 	verbose = args->verbose;
diff --git a/builtin-add.c b/builtin-add.c
index 1591171..1b24c56 100644
--- a/builtin-add.c
+++ b/builtin-add.c
@@ -136,7 +136,7 @@ static void update(int verbose, const char **files)
 	run_diff_files(&rev, 0);
 }
 
-static int git_add_config(const char *var, const char *value)
+static int git_add_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "core.excludesfile")) {
 		if (!value)
@@ -145,7 +145,7 @@ static int git_add_config(const char *var, const char *value)
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static struct lock_file lock_file;
@@ -175,7 +175,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		exit(1);
 	}
 
-	git_config(git_add_config);
+	git_config(git_add_config, NULL);
 
 	newfd = hold_locked_index(&lock_file, 1);
 
diff --git a/builtin-apply.c b/builtin-apply.c
index 0399743..deae199 100644
--- a/builtin-apply.c
+++ b/builtin-apply.c
@@ -2707,13 +2707,13 @@ static int apply_patch(int fd, const char *filename, int inaccurate_eof)
 	return 0;
 }
 
-static int git_apply_config(const char *var, const char *value)
+static int git_apply_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "apply.whitespace")) {
 		apply_default_whitespace = xstrdup(value);
 		return 0;
 	}
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 
@@ -2729,7 +2729,7 @@ int cmd_apply(int argc, const char **argv, const char *unused_prefix)
 
 	prefix = setup_git_directory_gently(&is_not_gitdir);
 	prefix_length = prefix ? strlen(prefix) : 0;
-	git_config(git_apply_config);
+	git_config(git_apply_config, NULL);
 	if (apply_default_whitespace)
 		parse_whitespace_option(apply_default_whitespace);
 
diff --git a/builtin-blame.c b/builtin-blame.c
index 35471fc..0fb76ee 100644
--- a/builtin-blame.c
+++ b/builtin-blame.c
@@ -1973,7 +1973,7 @@ static void prepare_blame_range(struct scoreboard *sb,
 		usage(blame_usage);
 }
 
-static int git_blame_config(const char *var, const char *value)
+static int git_blame_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "blame.showroot")) {
 		show_root = git_config_bool(var, value);
@@ -1983,7 +1983,7 @@ static int git_blame_config(const char *var, const char *value)
 		blank_boundary = git_config_bool(var, value);
 		return 0;
 	}
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static struct commit *fake_working_tree_commit(const char *path, const char *contents_from)
@@ -2136,7 +2136,7 @@ int cmd_blame(int argc, const char **argv, const char *prefix)
 
 	cmd_is_annotate = !strcmp(argv[0], "annotate");
 
-	git_config(git_blame_config);
+	git_config(git_blame_config, NULL);
 	save_commit_buffer = 0;
 
 	opt = 0;
diff --git a/builtin-branch.c b/builtin-branch.c
index 6bd5843..a0f4c23 100644
--- a/builtin-branch.c
+++ b/builtin-branch.c
@@ -55,7 +55,7 @@ static int parse_branch_color_slot(const char *var, int ofs)
 	die("bad config variable '%s'", var);
 }
 
-int git_branch_config(const char *var, const char *value)
+int git_branch_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "color.branch")) {
 		branch_use_color = git_config_colorbool(var, value);
@@ -69,7 +69,7 @@ int git_branch_config(const char *var, const char *value)
 	if (!strcmp(var, "branch.autosetupmerge"))
 		branch_track_remotes = git_config_bool(var, value);
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 const char *branch_get_color(enum color_branch ix)
@@ -356,7 +356,7 @@ static int get_remote_branch_name(const char *value)
 	return 0;
 }
 
-static int get_remote_config(const char *key, const char *value)
+static int get_remote_config(const char *key, const char *value, void *cb_data)
 {
 	const char *var;
 	if (prefixcmp(key, "remote."))
@@ -400,7 +400,7 @@ static void set_branch_defaults(const char *name, const char *real_ref)
 	start_ref = real_ref;
 	start_len = strlen(real_ref);
 	base_len = slash - real_ref;
-	git_config(get_remote_config);
+	git_config(get_remote_config, NULL);
 	if (!config_repo && !config_remote &&
 	    !prefixcmp(real_ref, "refs/heads/")) {
 		set_branch_merge(name, ".", real_ref);
@@ -538,7 +538,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 	int kinds = REF_LOCAL_BRANCH;
 	int i;
 
-	git_config(git_branch_config);
+	git_config(git_branch_config, NULL);
 	track = branch_track_remotes;
 
 	for (i = 1; i < argc; i++) {
diff --git a/builtin-cat-file.c b/builtin-cat-file.c
index f132d58..b488fad 100644
--- a/builtin-cat-file.c
+++ b/builtin-cat-file.c
@@ -85,7 +85,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix)
 	int opt;
 	const char *exp_type, *obj_name;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	if (argc != 3)
 		usage("git-cat-file [-t|-s|-e|-p|<type>] <sha1>");
 	exp_type = argv[1];
diff --git a/builtin-checkout-index.c b/builtin-checkout-index.c
index 8460f97..47a8b1b 100644
--- a/builtin-checkout-index.c
+++ b/builtin-checkout-index.c
@@ -168,7 +168,7 @@ int cmd_checkout_index(int argc, const char **argv, const char *prefix)
 	int read_from_stdin = 0;
 	int prefix_length;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	state.base_dir = "";
 	prefix_length = prefix ? strlen(prefix) : 0;
 
diff --git a/builtin-commit-tree.c b/builtin-commit-tree.c
index ccbcbe3..3439321 100644
--- a/builtin-commit-tree.c
+++ b/builtin-commit-tree.c
@@ -92,7 +92,7 @@ int cmd_commit_tree(int argc, const char **argv, const char *prefix)
 	unsigned int size;
 	int encoding_is_utf8;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	if (argc < 2)
 		usage(commit_tree_usage);
diff --git a/builtin-config.c b/builtin-config.c
index 7e18f73..7834e19 100644
--- a/builtin-config.c
+++ b/builtin-config.c
@@ -16,7 +16,7 @@ static int do_not_match;
 static int seen;
 static enum { T_RAW, T_INT, T_BOOL } type = T_RAW;
 
-static int show_all_config(const char *key_, const char *value_)
+static int show_all_config(const char *key_, const char *value_, void *cb_data)
 {
 	if (value_)
 		printf("%s=%s\n", key_, value_);
@@ -25,7 +25,7 @@ static int show_all_config(const char *key_, const char *value_)
 	return 0;
 }
 
-static int show_config(const char* key_, const char* value_)
+static int show_config(const char* key_, const char* value_, void *cb_data)
 {
 	char value[256];
 	const char *vptr = value;
@@ -107,17 +107,17 @@ static int get_value(const char* key_, const char* regex_)
 	}
 
 	if (dest)
-		git_config_from_remote(show_config, dest);
+		git_config_from_remote(show_config, dest, NULL);
 	else {
 		if (do_all && system_wide)
-			git_config_from_file(show_config, system_wide);
+			git_config_from_file(show_config, system_wide, NULL);
 		if (do_all && global)
-			git_config_from_file(show_config, global);
-		git_config_from_file(show_config, local);
+			git_config_from_file(show_config, global, NULL);
+		git_config_from_file(show_config, local, NULL);
 		if (!do_all && !seen && global)
-			git_config_from_file(show_config, global);
+			git_config_from_file(show_config, global, NULL);
 		if (!do_all && !seen && system_wide)
-			git_config_from_file(show_config, system_wide);
+			git_config_from_file(show_config, system_wide, NULL);
 	}
 
 	free(key);
@@ -149,9 +149,10 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 			type = T_BOOL;
 		else if (!strcmp(argv[1], "--list") || !strcmp(argv[1], "-l")) {
 			if (dest)
-				return git_config_from_remote(show_all_config, dest);
+				return git_config_from_remote(show_all_config,
+								dest, NULL);
 			else
-				return git_config(show_all_config);
+				return git_config(show_all_config, NULL);
 		}
 		else if (!strcmp(argv[1], "--global")) {
 			char *home = getenv("HOME");
diff --git a/builtin-diff-files.c b/builtin-diff-files.c
index 6cb30c8..017f4b9 100644
--- a/builtin-diff-files.c
+++ b/builtin-diff-files.c
@@ -21,7 +21,7 @@ int cmd_diff_files(int argc, const char **argv, const char *prefix)
 
 	prefix = setup_git_directory_gently(&nongit);
 	init_revisions(&rev, prefix);
-	git_config(git_default_config); /* no "diff" UI options */
+	git_config(git_default_config, NULL); /* no "diff" UI options */
 	rev.abbrev = 0;
 
 	if (!setup_diff_no_index(&rev, argc, argv, nongit, prefix))
diff --git a/builtin-diff-index.c b/builtin-diff-index.c
index d90eba9..6e92b78 100644
--- a/builtin-diff-index.c
+++ b/builtin-diff-index.c
@@ -17,7 +17,7 @@ int cmd_diff_index(int argc, const char **argv, const char *prefix)
 	int result;
 
 	init_revisions(&rev, prefix);
-	git_config(git_default_config); /* no "diff" UI options */
+	git_config(git_default_config, NULL); /* no "diff" UI options */
 	rev.abbrev = 0;
 
 	argc = setup_revisions(argc, argv, &rev, NULL);
diff --git a/builtin-diff-tree.c b/builtin-diff-tree.c
index 0b591c8..341edad 100644
--- a/builtin-diff-tree.c
+++ b/builtin-diff-tree.c
@@ -68,7 +68,7 @@ int cmd_diff_tree(int argc, const char **argv, const char *prefix)
 	int read_stdin = 0;
 
 	init_revisions(opt, prefix);
-	git_config(git_default_config); /* no "diff" UI options */
+	git_config(git_default_config, NULL); /* no "diff" UI options */
 	nr_sha1 = 0;
 	opt->abbrev = 0;
 	opt->diff = 1;
diff --git a/builtin-diff.c b/builtin-diff.c
index 7f367b6..906b698 100644
--- a/builtin-diff.c
+++ b/builtin-diff.c
@@ -220,7 +220,7 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
 	 */
 
 	prefix = setup_git_directory_gently(&nongit);
-	git_config(git_diff_ui_config);
+	git_config(git_diff_ui_config, NULL);
 	init_revisions(&rev, prefix);
 
 	if (!setup_diff_no_index(&rev, argc, argv, nongit, prefix))
diff --git a/builtin-fmt-merge-msg.c b/builtin-fmt-merge-msg.c
index 5c145d2..a156548 100644
--- a/builtin-fmt-merge-msg.c
+++ b/builtin-fmt-merge-msg.c
@@ -10,7 +10,8 @@ static const char *fmt_merge_msg_usage =
 
 static int merge_summary;
 
-static int fmt_merge_msg_config(const char *key, const char *value)
+static int fmt_merge_msg_config(const char *key, const char *value,
+				void *cb_data)
 {
 	if (!strcmp("merge.summary", key))
 		merge_summary = git_config_bool(key, value);
@@ -251,7 +252,7 @@ int cmd_fmt_merge_msg(int argc, const char **argv, const char *prefix)
 	unsigned char head_sha1[20];
 	const char *current_branch;
 
-	git_config(fmt_merge_msg_config);
+	git_config(fmt_merge_msg_config, NULL);
 
 	while (argc > 1) {
 		if (!strcmp(argv[1], "--summary"))
diff --git a/builtin-gc.c b/builtin-gc.c
index 8ea165a..164fe71 100644
--- a/builtin-gc.c
+++ b/builtin-gc.c
@@ -27,7 +27,7 @@ static const char *argv_repack[MAX_ADD] = {"repack", "-a", "-d", "-l", NULL};
 static const char *argv_prune[] = {"prune", NULL};
 static const char *argv_rerere[] = {"rerere", "gc", NULL};
 
-static int gc_config(const char *var, const char *value)
+static int gc_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "gc.packrefs")) {
 		if (!strcmp(value, "notbare"))
@@ -40,7 +40,7 @@ static int gc_config(const char *var, const char *value)
 		aggressive_window = git_config_int(var, value);
 		return 0;
 	}
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static void append_option(const char **cmd, const char *opt, int max_length)
@@ -62,7 +62,7 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
 	int prune = 0;
 	char buf[80];
 
-	git_config(gc_config);
+	git_config(gc_config, NULL);
 
 	if (pack_refs < 0)
 		pack_refs = !is_bare_repository();
diff --git a/builtin-init-db.c b/builtin-init-db.c
index 4df9fd0..11460b6 100644
--- a/builtin-init-db.c
+++ b/builtin-init-db.c
@@ -152,7 +152,7 @@ static void copy_templates(const char *git_dir, int len, const char *template_di
 	strcpy(template_path + template_len, "config");
 	repository_format_version = 0;
 	git_config_from_file(check_repository_format_version,
-			     template_path);
+			     template_path, NULL);
 	template_path[template_len] = 0;
 
 	if (repository_format_version &&
@@ -207,7 +207,7 @@ static int create_default_files(const char *git_dir, const char *template_path)
 	path[len] = 0;
 	copy_templates(path, len, template_path);
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	/*
 	 * We would have created the above under user's umask -- under
diff --git a/builtin-log.c b/builtin-log.c
index 3744712..2581ce4 100644
--- a/builtin-log.c
+++ b/builtin-log.c
@@ -93,20 +93,20 @@ static int cmd_log_walk(struct rev_info *rev)
 	return 0;
 }
 
-static int git_log_config(const char *var, const char *value)
+static int git_log_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "log.showroot")) {
 		default_show_root = git_config_bool(var, value);
 		return 0;
 	}
-	return git_diff_ui_config(var, value);
+	return git_diff_ui_config(var, value, NULL);
 }
 
 int cmd_whatchanged(int argc, const char **argv, const char *prefix)
 {
 	struct rev_info rev;
 
-	git_config(git_log_config);
+	git_config(git_log_config, NULL);
 	init_revisions(&rev, prefix);
 	rev.diff = 1;
 	rev.diffopt.recursive = 1;
@@ -155,7 +155,7 @@ int cmd_show(int argc, const char **argv, const char *prefix)
 	struct object_array_entry *objects;
 	int i, count, ret = 0;
 
-	git_config(git_log_config);
+	git_config(git_log_config, NULL);
 	init_revisions(&rev, prefix);
 	rev.diff = 1;
 	rev.diffopt.recursive = 1;
@@ -220,7 +220,7 @@ int cmd_log_reflog(int argc, const char **argv, const char *prefix)
 {
 	struct rev_info rev;
 
-	git_config(git_log_config);
+	git_config(git_log_config, NULL);
 	init_revisions(&rev, prefix);
 	init_reflog_walk(&rev.reflog_info);
 	rev.abbrev_commit = 1;
@@ -248,7 +248,7 @@ int cmd_log(int argc, const char **argv, const char *prefix)
 {
 	struct rev_info rev;
 
-	git_config(git_log_config);
+	git_config(git_log_config, NULL);
 	init_revisions(&rev, prefix);
 	rev.always_show_header = 1;
 	cmd_log_init(argc, argv, prefix, &rev);
@@ -268,7 +268,7 @@ static char *extra_headers = NULL;
 static int extra_headers_size = 0;
 static const char *fmt_patch_suffix = ".patch";
 
-static int git_format_config(const char *var, const char *value)
+static int git_format_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "format.headers")) {
 		int len;
@@ -291,7 +291,7 @@ static int git_format_config(const char *var, const char *value)
 	if (!strcmp(var, "diff.color") || !strcmp(var, "color.diff")) {
 		return 0;
 	}
-	return git_log_config(var, value);
+	return git_log_config(var, value, NULL);
 }
 
 
@@ -440,7 +440,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	char message_id[1024];
 	char ref_message_id[1024];
 
-	git_config(git_format_config);
+	git_config(git_format_config, NULL);
 	init_revisions(&rev, prefix);
 	rev.commit_format = CMIT_FMT_EMAIL;
 	rev.verbose_header = 1;
diff --git a/builtin-ls-files.c b/builtin-ls-files.c
index f7c066b..9bdadc4 100644
--- a/builtin-ls-files.c
+++ b/builtin-ls-files.c
@@ -347,7 +347,7 @@ int cmd_ls_files(int argc, const char **argv, const char *prefix)
 	memset(&dir, 0, sizeof(dir));
 	if (prefix)
 		prefix_offset = strlen(prefix);
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
diff --git a/builtin-ls-tree.c b/builtin-ls-tree.c
index 1cb4dca..104e98f 100644
--- a/builtin-ls-tree.c
+++ b/builtin-ls-tree.c
@@ -108,7 +108,7 @@ int cmd_ls_tree(int argc, const char **argv, const char *prefix)
 	unsigned char sha1[20];
 	struct tree *tree;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	ls_tree_prefix = prefix;
 	if (prefix && *prefix)
 		chomp_prefix = strlen(prefix);
diff --git a/builtin-mailinfo.c b/builtin-mailinfo.c
index c95e477..82f7a6e 100644
--- a/builtin-mailinfo.c
+++ b/builtin-mailinfo.c
@@ -894,7 +894,7 @@ int cmd_mailinfo(int argc, const char **argv, const char *prefix)
 	/* NEEDSWORK: might want to do the optional .git/ directory
 	 * discovery
 	 */
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	def_charset = (git_commit_encoding ? git_commit_encoding : "utf-8");
 	metainfo_charset = def_charset;
diff --git a/builtin-merge-base.c b/builtin-merge-base.c
index e35d362..9f766c5 100644
--- a/builtin-merge-base.c
+++ b/builtin-merge-base.c
@@ -27,7 +27,7 @@ int cmd_merge_base(int argc, const char **argv, const char *prefix)
 	unsigned char rev1key[20], rev2key[20];
 	int show_all = 0;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	while (1 < argc && argv[1][0] == '-') {
 		const char *arg = argv[1];
diff --git a/builtin-mv.c b/builtin-mv.c
index 3563216..7d01203 100644
--- a/builtin-mv.c
+++ b/builtin-mv.c
@@ -75,7 +75,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)
 	struct path_list deleted = {NULL, 0, 0, 0};
 	struct path_list changed = {NULL, 0, 0, 0};
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	newfd = hold_locked_index(&lock_file, 1);
 	if (read_cache() < 0)
diff --git a/builtin-name-rev.c b/builtin-name-rev.c
index ef16385..988426c 100644
--- a/builtin-name-rev.c
+++ b/builtin-name-rev.c
@@ -151,7 +151,7 @@ int cmd_name_rev(int argc, const char **argv, const char *prefix)
 	int as_is = 0, all = 0, transform_stdin = 0;
 	struct name_ref_data data = { 0, NULL };
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	if (argc < 2)
 		usage(name_rev_usage);
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index d165f10..e6ce67a 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1419,7 +1419,7 @@ static void prepare_pack(int window, int depth)
 	free(delta_list);
 }
 
-static int git_pack_config(const char *k, const char *v)
+static int git_pack_config(const char *k, const char *v, void *cb_data)
 {
 	if(!strcmp(k, "pack.window")) {
 		window = git_config_int(k, v);
@@ -1439,7 +1439,7 @@ static int git_pack_config(const char *k, const char *v)
 		pack_compression_seen = 1;
 		return 0;
 	}
-	return git_default_config(k, v);
+	return git_default_config(k, v, NULL);
 }
 
 static void read_object_list_from_stdin(void)
@@ -1549,7 +1549,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	rp_av[1] = "--objects"; /* --thin will make it --objects-edge */
 	rp_ac = 2;
 
-	git_config(git_pack_config);
+	git_config(git_pack_config, NULL);
 	if (!pack_compression_seen && core_compression_seen)
 		pack_compression_level = core_compression_level;
 
diff --git a/builtin-push.c b/builtin-push.c
index cb78401..a19be32 100644
--- a/builtin-push.c
+++ b/builtin-push.c
@@ -193,7 +193,7 @@ static int config_current_uri;
 static int config_get_refspecs;
 static int config_get_receivepack;
 
-static int get_remote_config(const char* key, const char* value)
+static int get_remote_config(const char* key, const char* value, void *cb_data)
 {
 	if (!prefixcmp(key, "remote.") &&
 	    !strncmp(key + 7, config_repo, config_repo_len)) {
@@ -230,7 +230,7 @@ static int get_config_remotes_uri(const char *repo, const char *uri[MAX_URI])
 	config_get_refspecs = !(refspec_nr || all || tags);
 	config_get_receivepack = (receivepack == NULL);
 
-	git_config(get_remote_config);
+	git_config(get_remote_config, NULL);
 	return config_current_uri;
 }
 
diff --git a/builtin-read-tree.c b/builtin-read-tree.c
index b9fcff7..cec2021 100644
--- a/builtin-read-tree.c
+++ b/builtin-read-tree.c
@@ -92,14 +92,15 @@ static const char read_tree_usage[] = "git-read-tree (<sha> | [[-m [--aggressive
 static struct lock_file lock_file;
 static struct unpack_trees_options opts;
 
-static int git_read_tree_config(const char *var, const char *value)
+static int git_read_tree_config(const char *var, const char *value,
+				void *cb_data)
 {
 	if (!strcmp(var, "core.submodules")) {
 		opts.submodules = git_config_bool(var, value);
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
@@ -110,11 +111,11 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 	opts.head_idx = -1;
 
 	setup_git_directory();
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	newfd = hold_locked_index(&lock_file, 1);
 
-	git_config(git_read_tree_config);
+	git_config(git_read_tree_config, NULL);
 
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
diff --git a/builtin-reflog.c b/builtin-reflog.c
index ce093ca..c616a0a 100644
--- a/builtin-reflog.c
+++ b/builtin-reflog.c
@@ -281,14 +281,15 @@ static int expire_reflog(const char *ref, const unsigned char *sha1, int unused,
 	return status;
 }
 
-static int reflog_expire_config(const char *var, const char *value)
+static int reflog_expire_config(const char *var, const char *value,
+				void *cb_data)
 {
 	if (!strcmp(var, "gc.reflogexpire"))
 		default_reflog_expire = approxidate(value);
 	else if (!strcmp(var, "gc.reflogexpireunreachable"))
 		default_reflog_expire_unreachable = approxidate(value);
 	else
-		return git_default_config(var, value);
+		return git_default_config(var, value, NULL);
 	return 0;
 }
 
@@ -298,7 +299,7 @@ static int cmd_reflog_expire(int argc, const char **argv, const char *prefix)
 	unsigned long now = time(NULL);
 	int i, status, do_all;
 
-	git_config(reflog_expire_config);
+	git_config(reflog_expire_config, NULL);
 
 	save_commit_buffer = 0;
 	do_all = status = 0;
diff --git a/builtin-rerere.c b/builtin-rerere.c
index 8c2c8bd..5d1fe12 100644
--- a/builtin-rerere.c
+++ b/builtin-rerere.c
@@ -380,14 +380,14 @@ tail_optimization:
 	return write_rr(rr, fd);
 }
 
-static int git_rerere_config(const char *var, const char *value)
+static int git_rerere_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "gc.rerereresolved"))
 		cutoff_resolve = git_config_int(var, value);
 	else if (!strcmp(var, "gc.rerereunresolved"))
 		cutoff_noresolve = git_config_int(var, value);
 	else
-		return git_default_config(var, value);
+		return git_default_config(var, value, NULL);
 	return 0;
 }
 
@@ -400,7 +400,7 @@ int cmd_rerere(int argc, const char **argv, const char *prefix)
 	if (stat(git_path("rr-cache"), &st) || !S_ISDIR(st.st_mode))
 		return 0;
 
-	git_config(git_rerere_config);
+	git_config(git_rerere_config, NULL);
 
 	merge_rr_path = xstrdup(git_path("rr-cache/MERGE_RR"));
 	fd = hold_lock_file_for_update(&write_lock, merge_rr_path, 1);
diff --git a/builtin-rev-list.c b/builtin-rev-list.c
index ebf53f5..8700d37 100644
--- a/builtin-rev-list.c
+++ b/builtin-rev-list.c
@@ -468,7 +468,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 	int read_from_stdin = 0;
 	int bisect_show_vars = 0;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	init_revisions(&revs, prefix);
 	revs.abbrev = 0;
 	revs.commit_format = CMIT_FMT_UNSPECIFIED;
diff --git a/builtin-rev-parse.c b/builtin-rev-parse.c
index 37addb2..8e868c3 100644
--- a/builtin-rev-parse.c
+++ b/builtin-rev-parse.c
@@ -214,7 +214,7 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix)
 	int i, as_is = 0, verify = 0;
 	unsigned char sha1[20];
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
diff --git a/builtin-revert.c b/builtin-revert.c
index ea2f15b..aa26c27 100644
--- a/builtin-revert.c
+++ b/builtin-revert.c
@@ -239,7 +239,7 @@ static int revert_or_cherry_pick(int argc, const char **argv)
 	const char *message, *encoding;
 	const char *defmsg = xstrdup(git_path("MERGE_MSG"));
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	me = action == REVERT ? "revert" : "cherry-pick";
 	setenv(GIT_REFLOG_ACTION, me, 0);
 	parse_options(argc, argv);
diff --git a/builtin-rm.c b/builtin-rm.c
index 4a0bd93..7f03735 100644
--- a/builtin-rm.c
+++ b/builtin-rm.c
@@ -109,7 +109,7 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 	const char **pathspec;
 	char *seen;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	newfd = hold_locked_index(&lock_file, 1);
 
diff --git a/builtin-runstatus.c b/builtin-runstatus.c
index 4b489b1..d365d7d 100644
--- a/builtin-runstatus.c
+++ b/builtin-runstatus.c
@@ -11,7 +11,7 @@ int cmd_runstatus(int argc, const char **argv, const char *prefix)
 	struct wt_status s;
 	int i;
 
-	git_config(git_status_config);
+	git_config(git_status_config, NULL);
 	wt_status_prepare(&s);
 
 	for (i = 1; i < argc; i++) {
diff --git a/builtin-show-branch.c b/builtin-show-branch.c
index c892f1f..73565c8 100644
--- a/builtin-show-branch.c
+++ b/builtin-show-branch.c
@@ -531,7 +531,8 @@ static void append_one_rev(const char *av)
 	die("bad sha1 reference %s", av);
 }
 
-static int git_show_branch_config(const char *var, const char *value)
+static int git_show_branch_config(const char *var, const char *value,
+				  void *cb_data)
 {
 	if (!strcmp(var, "showbranch.default")) {
 		if (default_alloc <= default_num + 1) {
@@ -543,7 +544,7 @@ static int git_show_branch_config(const char *var, const char *value)
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static int omit_in_dense(struct commit *commit, struct commit **rev, int n)
@@ -607,7 +608,7 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 	int reflog = 0;
 	const char *reflog_base = NULL;
 
-	git_config(git_show_branch_config);
+	git_config(git_show_branch_config, NULL);
 
 	/* If nothing is specified, try the default first */
 	if (ac == 1 && default_num) {
diff --git a/builtin-symbolic-ref.c b/builtin-symbolic-ref.c
index d41b406..114c473 100644
--- a/builtin-symbolic-ref.c
+++ b/builtin-symbolic-ref.c
@@ -27,7 +27,7 @@ int cmd_symbolic_ref(int argc, const char **argv, const char *prefix)
 	int quiet = 0;
 	const char *msg = NULL;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	while (1 < argc) {
 		const char *arg = argv[1];
diff --git a/builtin-unpack-objects.c b/builtin-unpack-objects.c
index a6ff62f..73dd79f 100644
--- a/builtin-unpack-objects.c
+++ b/builtin-unpack-objects.c
@@ -341,7 +341,7 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix)
 	int i;
 	unsigned char sha1[20];
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	quiet = !isatty(2);
 
diff --git a/builtin-update-index.c b/builtin-update-index.c
index 8f98991..023c2dc 100644
--- a/builtin-update-index.c
+++ b/builtin-update-index.c
@@ -570,7 +570,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
 	int lock_error = 0;
 	struct lock_file *lock_file;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	/* We can't free this memory, it becomes part of a linked list parsed atexit() */
 	lock_file = xcalloc(1, sizeof(struct lock_file));
diff --git a/builtin-update-ref.c b/builtin-update-ref.c
index feac2ed..782d4aa 100644
--- a/builtin-update-ref.c
+++ b/builtin-update-ref.c
@@ -14,7 +14,7 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 
 	delete = 0;
 	ref_flags = 0;
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	for (i = 1; i < argc; i++) {
 		if (!strcmp("-m", argv[i])) {
diff --git a/builtin-verify-pack.c b/builtin-verify-pack.c
index 4e31c27..8a59d14 100644
--- a/builtin-verify-pack.c
+++ b/builtin-verify-pack.c
@@ -55,7 +55,7 @@ int cmd_verify_pack(int argc, const char **argv, const char *prefix)
 	int no_more_options = 0;
 	int nothing_done = 1;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	while (1 < argc) {
 		if (!no_more_options && argv[1][0] == '-') {
 			if (!strcmp("-v", argv[1]))
diff --git a/cache.h b/cache.h
index 446030a..d56db20 100644
--- a/cache.h
+++ b/cache.h
@@ -499,17 +499,18 @@ extern const char *packed_object_info_detail(struct packed_git *, off_t, unsigne
 /* Dumb servers support */
 extern int update_server_info(int);
 
-typedef int (*config_fn_t)(const char *, const char *);
-extern int git_default_config(const char *, const char *);
-extern int git_config_from_file(config_fn_t fn, const char *);
-extern int git_config_from_remote(config_fn_t fn, char *dest);
-extern int git_config(config_fn_t fn);
+typedef int (*config_fn_t)(const char *, const char *, void *cb_data);
+extern int git_default_config(const char *, const char *, void *cb_data);
+extern int git_config_from_file(config_fn_t fn, const char *, void *cb_data);
+extern int git_config_from_remote(config_fn_t fn, char *dest, void *cb_data);
+extern int git_config(config_fn_t fn, void *cb_data);
 extern int git_config_int(const char *, const char *);
 extern int git_config_bool(const char *, const char *);
 extern int git_config_set(const char *, const char *);
 extern int git_config_set_multivar(const char *, const char *, const char *, int);
 extern int git_config_rename_section(const char *, const char *);
-extern int check_repository_format_version(const char *var, const char *value);
+extern int check_repository_format_version(const char *var, const char *value,
+					   void *cb_data);
 
 #define MAX_GITNAME (1000)
 extern char git_default_email[MAX_GITNAME];
diff --git a/config.c b/config.c
index fc2162b..cab6f29 100644
--- a/config.c
+++ b/config.c
@@ -113,7 +113,7 @@ static inline int iskeychar(int c)
 	return isalnum(c) || c == '-';
 }
 
-static int get_value(config_fn_t fn, char *name, unsigned int len)
+static int get_value(config_fn_t fn, char *name, unsigned int len, void *cb_data)
 {
 	int c;
 	char *value;
@@ -141,7 +141,7 @@ static int get_value(config_fn_t fn, char *name, unsigned int len)
 		if (!value)
 			return -1;
 	}
-	return fn(name, value);
+	return fn(name, value, cb_data);
 }
 
 static int get_extended_base_var(char *name, int baselen, int c)
@@ -199,7 +199,7 @@ static int get_base_var(char *name)
 	}
 }
 
-static int git_parse_file(config_fn_t fn)
+static int git_parse_file(config_fn_t fn, void *cb_data)
 {
 	int comment = 0;
 	int baselen = 0;
@@ -231,7 +231,7 @@ static int git_parse_file(config_fn_t fn)
 		if (!isalpha(c))
 			break;
 		var[baselen] = tolower(c);
-		if (get_value(fn, var, baselen+1) < 0)
+		if (get_value(fn, var, baselen+1, cb_data) < 0)
 			break;
 	}
 	die("bad config file line %d in %s", config_linenr, config_file_name);
@@ -267,7 +267,7 @@ int git_config_bool(const char *name, const char *value)
 	return git_config_int(name, value) != 0;
 }
 
-int git_default_config(const char *var, const char *value)
+int git_default_config(const char *var, const char *value, void *cb_data)
 {
 	/* This needs a better name */
 	if (!strcmp(var, "core.filemode")) {
@@ -390,7 +390,7 @@ int git_default_config(const char *var, const char *value)
 	return 0;
 }
 
-int git_config_from_file(config_fn_t fn, const char *filename)
+int git_config_from_file(config_fn_t fn, const char *filename, void *cb_data)
 {
 	int ret;
 	FILE *f = fopen(filename, "r");
@@ -400,24 +400,24 @@ int git_config_from_file(config_fn_t fn, const char *filename)
 		config_file = f;
 		config_file_name = filename;
 		config_linenr = 1;
-		ret = git_parse_file(fn);
+		ret = git_parse_file(fn, cb_data);
 		fclose(f);
 		config_file_name = NULL;
 	}
 	return ret;
 }
 
-static int config_from_http(config_fn_t fn, char *dest)
+static int config_from_http(config_fn_t fn, char *dest, void *cb_data)
 {
 	char config_temp[50];
 	if (git_http_fetch_config(dest, config_temp, sizeof(config_temp)))
 		return 1;
-	git_config_from_file(fn, config_temp);
+	git_config_from_file(fn, config_temp, cb_data);
 	unlink(config_temp);
 	return 0;
 }
 
-int git_config_from_remote(config_fn_t fn, char *dest)
+int git_config_from_remote(config_fn_t fn, char *dest, void *cb_data)
 {
 	int ret;
 	int fd[2];
@@ -426,7 +426,7 @@ int git_config_from_remote(config_fn_t fn, char *dest)
 	static char value[1024];
 
 	if (!prefixcmp(dest, "http://"))
-		return config_from_http(fn, dest);
+		return config_from_http(fn, dest, cb_data);
 
 	pid = git_connect(fd, dest, dumpconfig, 0);
 	if (pid < 0)
@@ -435,7 +435,7 @@ int git_config_from_remote(config_fn_t fn, char *dest)
 	while (packet_read_line(fd[0], var, sizeof(var))) {
 		if (!packet_read_line(fd[0], value, sizeof(value)))
 			die("Missing value");
-		fn(var, value);
+		fn(var, value, cb_data);
 	}
 	close(fd[0]);
 	close(fd[1]);
@@ -443,7 +443,7 @@ int git_config_from_remote(config_fn_t fn, char *dest)
 	return !!ret;
 }
 
-int git_config(config_fn_t fn)
+int git_config(config_fn_t fn, void *cb_data)
 {
 	int ret = 0;
 	char *repo_config = NULL;
@@ -456,7 +456,7 @@ int git_config(config_fn_t fn)
 	filename = getenv(CONFIG_ENVIRONMENT);
 	if (!filename) {
 		if (!access(ETC_GITCONFIG, R_OK))
-			ret += git_config_from_file(fn, ETC_GITCONFIG);
+			ret += git_config_from_file(fn, ETC_GITCONFIG, cb_data);
 		home = getenv("HOME");
 		filename = getenv(CONFIG_LOCAL_ENVIRONMENT);
 		if (!filename)
@@ -466,11 +466,11 @@ int git_config(config_fn_t fn)
 	if (home) {
 		char *user_config = xstrdup(mkpath("%s/.gitconfig", home));
 		if (!access(user_config, R_OK))
-			ret = git_config_from_file(fn, user_config);
+			ret = git_config_from_file(fn, user_config, cb_data);
 		free(user_config);
 	}
 
-	ret += git_config_from_file(fn, filename);
+	ret += git_config_from_file(fn, filename, cb_data);
 	free(repo_config);
 	return ret;
 }
@@ -500,7 +500,7 @@ static int matches(const char* key, const char* value)
 		  !regexec(store.value_regex, value, 0, NULL, 0)));
 }
 
-static int store_aux(const char* key, const char* value)
+static int store_aux(const char* key, const char* value, void *cb_data)
 {
 	const char *ep;
 	size_t section_len;
@@ -836,7 +836,7 @@ int git_config_set_multivar(const char* key, const char* value,
 		 * As a side effect, we make sure to transform only a valid
 		 * existing config file.
 		 */
-		if (git_config_from_file(store_aux, config_filename)) {
+		if (git_config_from_file(store_aux, config_filename, NULL)) {
 			fprintf(stderr, "invalid config file\n");
 			free(store.key);
 			if (store.value_regex != NULL) {
diff --git a/connect.c b/connect.c
index 2a26fdb..08cfac0 100644
--- a/connect.c
+++ b/connect.c
@@ -558,7 +558,8 @@ static char *git_proxy_command;
 static const char *rhost_name;
 static int rhost_len;
 
-static int git_proxy_command_options(const char *var, const char *value)
+static int git_proxy_command_options(const char *var, const char *value,
+				     void *cb_data)
 {
 	if (!strcmp(var, "core.gitproxy")) {
 		const char *for_pos;
@@ -602,7 +603,7 @@ static int git_proxy_command_options(const char *var, const char *value)
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static int git_use_proxy(const char *host)
@@ -610,7 +611,7 @@ static int git_use_proxy(const char *host)
 	rhost_name = host;
 	rhost_len = strlen(host);
 	git_proxy_command = getenv("GIT_PROXY_COMMAND");
-	git_config(git_proxy_command_options);
+	git_config(git_proxy_command_options, NULL);
 	rhost_name = NULL;
 	return (git_proxy_command && *git_proxy_command);
 }
diff --git a/convert.c b/convert.c
index 4b26b1a..6dde5fa 100644
--- a/convert.c
+++ b/convert.c
@@ -335,7 +335,8 @@ static struct convert_driver {
 	char *clean;
 } *user_convert, **user_convert_tail;
 
-static int read_convert_config(const char *var, const char *value)
+static int read_convert_config(const char *var, const char *value,
+				void *cb_data)
 {
 	const char *ep, *name;
 	int namelen;
@@ -402,7 +403,7 @@ static void setup_convert_check(struct git_attr_check *check)
 		attr_ident = git_attr("ident", 5);
 		attr_filter = git_attr("filter", 6);
 		user_convert_tail = &user_convert;
-		git_config(read_convert_config);
+		git_config(read_convert_config, NULL);
 	}
 	check[0].attr = attr_crlf;
 	check[1].attr = attr_ident;
diff --git a/daemon.c b/daemon.c
index 3e5ebf3..2d6302f 100644
--- a/daemon.c
+++ b/daemon.c
@@ -284,7 +284,7 @@ struct daemon_service {
 static struct daemon_service *service_looking_at;
 static int service_enabled;
 
-static int git_daemon_config(const char *var, const char *value)
+static int git_daemon_config(const char *var, const char *value, void *cb_data)
 {
 	if (!prefixcmp(var, "daemon.") &&
 	    !strcmp(var + 7, service_looking_at->config_name)) {
@@ -334,7 +334,7 @@ static int run_service(struct interp *itable, struct daemon_service *service)
 	if (service->overridable) {
 		service_looking_at = service;
 		service_enabled = -1;
-		git_config(git_daemon_config);
+		git_config(git_daemon_config, NULL);
 		if (0 <= service_enabled)
 			enabled = service_enabled;
 	}
diff --git a/diff.c b/diff.c
index 33297aa..8edfcc5 100644
--- a/diff.c
+++ b/diff.c
@@ -99,7 +99,7 @@ static int parse_lldiff_command(const char *var, const char *ep, const char *val
  * never be affected by the setting of diff.renames
  * the user happens to have in the configuration file.
  */
-int git_diff_ui_config(const char *var, const char *value)
+int git_diff_ui_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp(var, "diff.renamelimit")) {
 		diff_rename_limit_default = git_config_int(var, value);
@@ -131,7 +131,7 @@ int git_diff_ui_config(const char *var, const char *value)
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static char *quote_one(const char *str)
@@ -1762,7 +1762,7 @@ static const char *external_diff_attr(const char *name)
 
 			if (!user_diff_tail) {
 				user_diff_tail = &user_diff;
-				git_config(git_diff_ui_config);
+				git_config(git_diff_ui_config, NULL);
 			}
 			for (drv = user_diff; drv; drv = drv->next)
 				if (!strcmp(drv->name, value))
diff --git a/diff.h b/diff.h
index 63738c1..a715dff 100644
--- a/diff.h
+++ b/diff.h
@@ -161,7 +161,7 @@ extern int diff_scoreopt_parse(const char *opt);
 #define DIFF_SETUP_USE_CACHE		2
 #define DIFF_SETUP_USE_SIZE_CACHE	4
 
-extern int git_diff_ui_config(const char *var, const char *value);
+extern int git_diff_ui_config(const char *var, const char *value, void *cb_data);
 extern void diff_setup(struct diff_options *);
 extern int diff_opt_parse(struct diff_options *, const char **, int);
 extern int diff_setup_done(struct diff_options *);
diff --git a/dump-config.c b/dump-config.c
index 355920d..99dbeb6 100644
--- a/dump-config.c
+++ b/dump-config.c
@@ -4,7 +4,7 @@
 
 static const char dump_config_usage[] = "git-dump-config <dir>";
 
-static int dump_config(const char *var, const char *value)
+static int dump_config(const char *var, const char *value, void *cb_data)
 {
 	packet_write(1, "%s", var);
 	packet_write(1, "%s", value);
@@ -22,7 +22,7 @@ int main(int argc, char **argv)
 	if (!enter_repo(dir, 0))
 		die("'%s': unable to chdir or not a git archive", dir);
 
-	git_config(dump_config);
+	git_config(dump_config, NULL);
 	packet_flush(1);
 
 	return 0;
diff --git a/fast-import.c b/fast-import.c
index ffa00fd..599d045 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -2009,7 +2009,7 @@ int main(int argc, const char **argv)
 {
 	int i, show_stats = 1;
 
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	alloc_objects(object_entry_alloc);
 	strbuf_init(&command_buf);
 	atom_table = xcalloc(atom_table_sz, sizeof(struct atom_str*));
diff --git a/fetch-pack.c b/fetch-pack.c
index aa59043..c7d24fe 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -646,7 +646,7 @@ static int remove_duplicates(int nr_heads, char **heads)
 	return dst;
 }
 
-static int fetch_pack_config(const char *var, const char *value)
+static int fetch_pack_config(const char *var, const char *value, void *cb_data)
 {
 	if (strcmp(var, "fetch.unpacklimit") == 0) {
 		fetch_unpack_limit = git_config_int(var, value);
@@ -658,7 +658,7 @@ static int fetch_pack_config(const char *var, const char *value)
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static struct lock_file lock;
@@ -672,7 +672,7 @@ int main(int argc, char **argv)
 	struct stat st;
 
 	setup_git_directory();
-	git_config(fetch_pack_config);
+	git_config(fetch_pack_config, NULL);
 
 	if (0 <= transfer_unpack_limit)
 		unpack_limit = transfer_unpack_limit;
diff --git a/git.c b/git.c
index f200907..bbe09d5 100644
--- a/git.c
+++ b/git.c
@@ -87,7 +87,7 @@ static int handle_options(const char*** argv, int* argc)
 static const char *alias_command;
 static char *alias_string;
 
-static int git_alias_config(const char *var, const char *value)
+static int git_alias_config(const char *var, const char *value, void *cb_data)
 {
 	if (!prefixcmp(var, "alias.") && !strcmp(var + 6, alias_command)) {
 		alias_string = xstrdup(value);
@@ -158,7 +158,7 @@ static int handle_alias(int *argcp, const char ***argv)
 	subdir = setup_git_directory_gently(&nongit);
 
 	alias_command = (*argv)[0];
-	git_config(git_alias_config);
+	git_config(git_alias_config, NULL);
 	if (alias_string) {
 		if (alias_string[0] == '!') {
 			trace_printf("trace: alias to shell cmd: %s => %s\n",
diff --git a/http-fetch.c b/http-fetch.c
index 53fb2a9..bc87d4b 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -996,7 +996,7 @@ int main(int argc, const char **argv)
 	int rc = 0;
 
 	setup_git_directory();
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	while (arg < argc && argv[arg][0] == '-') {
 		if (argv[arg][1] == 't') {
diff --git a/http.c b/http.c
index c8237cb..fd359bf 100644
--- a/http.c
+++ b/http.c
@@ -97,7 +97,7 @@ static void process_curl_messages(void)
 }
 #endif
 
-static int http_options(const char *var, const char *value)
+static int http_options(const char *var, const char *value, void *cb_data)
 {
 	if (!strcmp("http.sslverify", var)) {
 		if (curl_ssl_verify == -1) {
@@ -164,7 +164,7 @@ static int http_options(const char *var, const char *value)
 	}
 
 	/* Fall back on the default ones */
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static CURL* get_curl_handle(void)
@@ -252,7 +252,7 @@ void http_init(void)
 	if (low_speed_time != NULL)
 		curl_low_speed_time = strtol(low_speed_time, NULL, 10);
 
-	git_config(http_options);
+	git_config(http_options, NULL);
 
 	if (curl_ssl_verify == -1)
 		curl_ssl_verify = 1;
diff --git a/imap-send.c b/imap-send.c
index 4283a4a..129776c 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -1255,7 +1255,7 @@ static imap_server_conf_t server =
 static char *imap_folder;
 
 static int
-git_imap_config(const char *key, const char *val)
+git_imap_config(const char *key, const char *val, void *cb_data)
 {
 	char imap_key[] = "imap.";
 
@@ -1300,7 +1300,7 @@ main(int argc, char **argv)
 	/* init the random number generator */
 	arc4_init();
 
-	git_config( git_imap_config );
+	git_config(git_imap_config, NULL);
 
 	if (!imap_folder) {
 		fprintf( stderr, "no imap store specified\n" );
diff --git a/local-fetch.c b/local-fetch.c
index 4b650ef..23d2cbe 100644
--- a/local-fetch.c
+++ b/local-fetch.c
@@ -204,7 +204,7 @@ int main(int argc, const char **argv)
 	int arg = 1;
 
 	setup_git_directory();
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	while (arg < argc && argv[arg][0] == '-') {
 		if (argv[arg][1] == 't')
diff --git a/merge-recursive.c b/merge-recursive.c
index 8f72b2c..f774342 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -851,7 +851,7 @@ static int ll_ext_merge(const struct ll_merge_driver *fn,
 static struct ll_merge_driver *ll_user_merge, **ll_user_merge_tail;
 static const char *default_ll_merge;
 
-static int read_merge_config(const char *var, const char *value)
+static int read_merge_config(const char *var, const char *value, void *cb_data)
 {
 	struct ll_merge_driver *fn;
 	const char *ep, *name;
@@ -940,7 +940,7 @@ static void initialize_ll_merge(void)
 	if (ll_user_merge_tail)
 		return;
 	ll_user_merge_tail = &ll_user_merge;
-	git_config(read_merge_config);
+	git_config(read_merge_config, NULL);
 }
 
 static const struct ll_merge_driver *find_ll_merge_driver(const char *merge_attr)
@@ -1696,13 +1696,13 @@ static struct commit *get_ref(const char *ref)
 	return (struct commit *)object;
 }
 
-static int merge_config(const char *var, const char *value)
+static int merge_config(const char *var, const char *value, void *cb_data)
 {
 	if (!strcasecmp(var, "merge.verbosity")) {
 		verbosity = git_config_int(var, value);
 		return 0;
 	}
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 int main(int argc, char *argv[])
@@ -1723,7 +1723,7 @@ int main(int argc, char *argv[])
 			subtree_merge = 1;
 	}
 
-	git_config(merge_config);
+	git_config(merge_config, NULL);
 	if (getenv("GIT_MERGE_VERBOSITY"))
 		verbosity = strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
 
diff --git a/receive-pack.c b/receive-pack.c
index d3c422b..feab9a4 100644
--- a/receive-pack.c
+++ b/receive-pack.c
@@ -18,7 +18,7 @@ static int report_status;
 static char capabilities[] = " report-status delete-refs ";
 static int capabilities_sent;
 
-static int receive_pack_config(const char *var, const char *value)
+static int receive_pack_config(const char *var, const char *value, void *cb_data)
 {
 	if (strcmp(var, "receive.denynonfastforwards") == 0) {
 		deny_non_fast_forwards = git_config_bool(var, value);
@@ -35,7 +35,7 @@ static int receive_pack_config(const char *var, const char *value)
 		return 0;
 	}
 
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 static int show_ref(const char *path, const unsigned char *sha1, int flag, void *cb_data)
@@ -484,7 +484,7 @@ int main(int argc, char **argv)
 	if (is_repository_shallow())
 		die("attempt to push into a shallow repository");
 
-	git_config(receive_pack_config);
+	git_config(receive_pack_config, NULL);
 
 	if (0 <= transfer_unpack_limit)
 		unpack_limit = transfer_unpack_limit;
diff --git a/send-pack.c b/send-pack.c
index 83ee87d..7895520 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -346,7 +346,7 @@ int main(int argc, char **argv)
 	pid_t pid;
 
 	setup_git_directory();
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	argv++;
 	for (i = 1; i < argc; i++, argv++) {
diff --git a/setup.c b/setup.c
index a45ea83..d3487aa 100644
--- a/setup.c
+++ b/setup.c
@@ -270,7 +270,8 @@ int git_config_perm(const char *var, const char *value)
 	return git_config_bool(var, value);
 }
 
-int check_repository_format_version(const char *var, const char *value)
+int check_repository_format_version(const char *var, const char *value,
+				    void *cb_data)
 {
        if (strcmp(var, "core.repositoryformatversion") == 0)
                repository_format_version = git_config_int(var, value);
@@ -281,7 +282,7 @@ int check_repository_format_version(const char *var, const char *value)
 
 int check_repository_format(void)
 {
-	git_config(check_repository_format_version);
+	git_config(check_repository_format_version, NULL);
 	if (GIT_REPO_VERSION < repository_format_version)
 		die ("Expected git repo version <= %d, found %d",
 		     GIT_REPO_VERSION, repository_format_version);
diff --git a/ssh-fetch.c b/ssh-fetch.c
index bdf51a7..ceb8b7d 100644
--- a/ssh-fetch.c
+++ b/ssh-fetch.c
@@ -125,7 +125,7 @@ int main(int argc, char **argv)
 	if (!prog) prog = "git-ssh-upload";
 
 	setup_git_directory();
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	while (arg < argc && argv[arg][0] == '-') {
 		if (argv[arg][1] == 't') {
diff --git a/unpack-file.c b/unpack-file.c
index 25c56b3..ade9926 100644
--- a/unpack-file.c
+++ b/unpack-file.c
@@ -33,7 +33,7 @@ int main(int argc, char **argv)
 		die("Not a valid object name %s", argv[1]);
 
 	setup_git_directory();
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 
 	puts(create_temp_file(sha1));
 	return 0;
diff --git a/var.c b/var.c
index e585e59..8ec0fdd 100644
--- a/var.c
+++ b/var.c
@@ -39,13 +39,13 @@ static const char *read_var(const char *var)
 	return val;
 }
 
-static int show_config(const char *var, const char *value)
+static int show_config(const char *var, const char *value, void *cb_data)
 {
 	if (value)
 		printf("%s=%s\n", var, value);
 	else
 		printf("%s\n", var);
-	return git_default_config(var, value);
+	return git_default_config(var, value, NULL);
 }
 
 int main(int argc, char **argv)
@@ -59,11 +59,11 @@ int main(int argc, char **argv)
 	val = NULL;
 
 	if (strcmp(argv[1], "-l") == 0) {
-		git_config(show_config);
+		git_config(show_config, NULL);
 		list_vars();
 		return 0;
 	}
-	git_config(git_default_config);
+	git_config(git_default_config, NULL);
 	val = read_var(argv[1]);
 	if (!val)
 		usage(var_usage);
diff --git a/wt-status.c b/wt-status.c
index a055990..6e40870 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -346,7 +346,7 @@ void wt_status_print(struct wt_status *s)
 	}
 }
 
-int git_status_config(const char *k, const char *v)
+int git_status_config(const char *k, const char *v, void *cb_data)
 {
 	if (!strcmp(k, "status.color") || !strcmp(k, "color.status")) {
 		wt_status_use_color = git_config_colorbool(k, v);
@@ -356,5 +356,5 @@ int git_status_config(const char *k, const char *v)
 		int slot = parse_status_slot(k, 13);
 		color_parse(v, k, wt_status_colors[slot]);
 	}
-	return git_default_config(k, v);
+	return git_default_config(k, v, NULL);
 }
diff --git a/wt-status.h b/wt-status.h
index cfea4ae..242e9c5 100644
--- a/wt-status.h
+++ b/wt-status.h
@@ -21,7 +21,7 @@ struct wt_status {
 	int workdir_untracked;
 };
 
-int git_status_config(const char *var, const char *value);
+int git_status_config(const char *var, const char *value, void *cb_data);
 void wt_status_prepare(struct wt_status *s);
 void wt_status_print(struct wt_status *s);
 
-- 
1.5.2.rc3.815.g8fc2

[PATCH 13/15] unpack-trees.c: optionally clone submodules for later checkout

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

When the --submodules option is specified and a submodule
to be checked out is not available locally, git-checkout will
search for submodule.<submodule>.url options in the remote
configuration and clone each submodule using the first url that
it can use from the local site.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 Documentation/config.txt |    3 +
 Makefile                 |    5 +-
 submodules.c             |  209 ++++++++++++++++++++++++++++++++++++++++++++++
 submodules.h             |    6 ++
 unpack-trees.c           |   51 +++++++++++
 5 files changed, 272 insertions(+), 2 deletions(-)
 create mode 100644 submodules.c
 create mode 100644 submodules.h

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 5d891ac..cee9e40 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -610,6 +610,9 @@ showbranch.default::
 	The default set of branches for gitlink:git-show-branch[1].
 	See gitlink:git-show-branch[1].
 
+submodule.<submodule>.url
+	The URL of a submodule.  See gitlink:git-clone[1].
+
 tar.umask::
 	By default, gitlink:git-tar-tree[1] sets file and directories modes
 	to 0666 or 0777. While this is both useful and acceptable for projects
diff --git a/Makefile b/Makefile
index bce8514..c79e636 100644
--- a/Makefile
+++ b/Makefile
@@ -297,7 +297,8 @@ LIB_H = \
 	diff.h object.h pack.h pkt-line.h quote.h refs.h list-objects.h sideband.h \
 	run-command.h strbuf.h tag.h tree.h git-compat-util.h revision.h \
 	tree-walk.h log-tree.h dir.h path-list.h unpack-trees.h builtin.h \
-	utf8.h reflog-walk.h patch-ids.h attr.h decorate.h progress.h mailmap.h
+	utf8.h reflog-walk.h patch-ids.h attr.h decorate.h progress.h mailmap.h \
+	submodules.h
 
 DIFF_OBJS = \
 	diff.o diff-lib.o diffcore-break.o diffcore-order.o \
@@ -320,7 +321,7 @@ LIB_OBJS = \
 	alloc.o merge-file.o path-list.o help.o unpack-trees.o $(DIFF_OBJS) \
 	color.o wt-status.o archive-zip.o archive-tar.o shallow.o utf8.o \
 	convert.o attr.o decorate.o progress.o mailmap.o symlinks.o \
-	$(HTTP_CONFIG_OBJ)
+	$(HTTP_CONFIG_OBJ) submodules.o
 
 BUILTIN_OBJS = \
 	builtin-add.o \
diff --git a/submodules.c b/submodules.c
new file mode 100644
index 0000000..5035a05
--- /dev/null
+++ b/submodules.c
@@ -0,0 +1,209 @@
+#include "cache.h"
+#include "submodules.h"
+#include "run-command.h"
+
+struct key_val_list {
+	struct key_val_list *next;
+	char *key;
+	char *val;
+};
+
+static void free_key_val_list(struct key_val_list *list)
+{
+	struct key_val_list *next;
+	for (; list; list = next) {
+		next = list->next;
+		free(list->key);
+		free(list->val);
+		free(list);
+	}
+}
+
+static struct key_val_list *find_key_val_list(struct key_val_list *list,
+						const char *key)
+{
+	while (list && strcmp(list->key, key))
+		list = list->next;
+	return list;
+}
+
+struct collect_urls_data {
+	struct key_val_list **next;
+
+	const char *type;
+};
+
+static int collect_urls(const char *var, const char *value, void *cb_data)
+{
+	struct collect_urls_data *cb = (struct collect_urls_data*)cb_data;
+	int typelen = strlen(cb->type);
+	int len;
+	char *doturl;
+	struct key_val_list *item;
+
+	if (prefixcmp(var, cb->type))
+		return 0;
+
+	if (var[typelen] != '.')
+		return 0;
+
+	var += typelen+1;
+
+	doturl = strrchr(var, '.');
+	if (!doturl || strcmp(doturl, ".url"))
+		return 0;
+
+	len = doturl-var;
+	if (len <= 0)
+		return 0;
+
+	item = xmalloc(sizeof(struct key_val_list));
+	item->key = xmalloc(len+1);
+	memcpy(item->key, var, len);
+	item->key[len] = 0;
+	item->val = xstrdup(value);
+	item->next = NULL;
+	*cb->next = item;
+	cb->next = &item->next;
+
+	return 0;
+}
+
+static const char *local_URL(const char *remote, const char *url)
+{
+	static char local_url[PATH_MAX];
+
+	if (!prefixcmp(url, "https://"))
+		return url;
+
+	if (!prefixcmp(url, "http://"))
+		return url;
+
+	if (!prefixcmp(url, "ftp://"))
+		return url;
+
+	if (!prefixcmp(remote, "/"))
+		return url;
+
+	if (!prefixcmp(remote, "ssh://") && !prefixcmp(url, "/")) {
+		char *slash;
+		int len = strlen(url);
+
+		slash = strchr(remote+6, '/');
+		if (!slash || (slash-remote)+len+1 > sizeof(local_url))
+			return NULL;
+		memcpy(local_url, remote, slash-remote);
+		memcpy(local_url+(slash-remote), url, len+1);
+		return local_url;
+	}
+
+	return NULL;
+}
+
+static int fetch_submodule_urls(struct key_val_list **next_url)
+{
+	struct key_val_list *remotes = NULL;
+	struct collect_urls_data remotes_data = { &remotes, "remote" };
+	struct key_val_list *remote;
+	static char key[1024];
+
+	git_config(collect_urls, &remotes_data);
+	for (remote = remotes; remote; remote = remote->next) {
+		struct key_val_list *submodules = NULL;
+		struct collect_urls_data submodules_data =
+			{ &submodules, "submodule" };
+		struct key_val_list *submodule;
+		char *dest;
+
+		dest = xstrdup(remote->val);
+		git_config_from_remote(collect_urls, dest, &submodules_data);
+		free(dest);
+		for (submodule = submodules; submodule; submodule = submodule->next) {
+			const char *local_url;
+			struct key_val_list *item;
+
+			local_url = local_URL(remote->val, submodule->val);
+			if (!local_url)
+				continue;
+
+			if (snprintf(key, sizeof(key),
+			    "submodule.%s.url", submodule->key) > sizeof(key))
+				return error("submodule name too long");
+
+			git_config_set(key, local_url);
+
+			item = xmalloc(sizeof(struct key_val_list));
+			item->key = xstrdup(submodule->key);
+			item->val = xstrdup(local_url);
+			item->next = NULL;
+			*next_url = item;
+			next_url = &item->next;
+		}
+
+		free_key_val_list(submodules);
+	}
+
+	free_key_val_list(remotes);
+
+	return 0;
+}
+
+int clone_submodule(const char *submodule)
+{
+	struct key_val_list *submodules = NULL;
+	struct collect_urls_data submodules_data = { &submodules, "submodule" };
+	struct key_val_list *item;
+	char *path;
+	int err;
+	const char *gitdirenv;
+	const char *args[10];
+	int argc;
+
+	git_config(collect_urls, &submodules_data);
+	item = find_key_val_list(submodules, submodule);
+	if (!item) {
+		err = fetch_submodule_urls(submodules_data.next);
+		if (err)
+			return err;
+		item = find_key_val_list(*submodules_data.next, submodule);
+		if (!item)
+			return error("don't know where to get submodule '%s'",
+					submodule);
+	}
+
+	path = git_path("submodules/%s", submodule);
+
+	argc = 0;
+	args[argc++] = "clone";
+	args[argc++] = "--submodules";
+	args[argc++] = "-n";
+	args[argc++] = item->val;
+	args[argc++] = path;
+	args[argc] = NULL;
+
+	gitdirenv = getenv(GIT_DIR_ENVIRONMENT);
+	unsetenv(GIT_DIR_ENVIRONMENT);
+	err = run_command_v_opt(args, RUN_GIT_CMD);
+
+	setenv(GIT_DIR_ENVIRONMENT,
+		git_path("submodules/%s/.git", submodule), 1);
+
+	argc = 0;
+	args[argc++] = "update-ref";
+	args[argc++] = "--no-deref";
+	args[argc++] = "HEAD";
+	args[argc++] = "0000000000000000000000000000000000000000";
+	args[argc] = NULL;
+
+	if (!err)
+		err = run_command_v_opt(args, RUN_GIT_CMD);
+
+	setenv(GIT_DIR_ENVIRONMENT, gitdirenv, 1);
+
+	if (err)
+		return error("failed to clone submodule '%s'", submodule);
+
+	free_key_val_list(submodules);
+
+	return 0;
+}
diff --git a/submodules.h b/submodules.h
new file mode 100644
index 0000000..56590e7
--- /dev/null
+++ b/submodules.h
@@ -0,0 +1,6 @@
+#ifndef SUBMODULES_H
+#define SUBMODULES_H
+
+int clone_submodule(const char *submodule);
+
+#endif
diff --git a/unpack-trees.c b/unpack-trees.c
index 3dadebb..f9865b9 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -6,6 +6,7 @@
 #include "unpack-trees.h"
 #include "progress.h"
 #include "refs.h"
+#include "submodules.h"
 
 #define DBRT_DEBUG 1
 
@@ -808,6 +809,36 @@ int threeway_merge(struct cache_entry **stages,
 	return count;
 }
 
+static int ensure_submodule(struct cache_entry *ce,
+			    struct unpack_trees_options *o)
+{
+	struct stat st;
+	char *path;
+
+	if (!ce)
+		return 0;
+
+	if (!S_ISDIRLNK(ntohl(ce->ce_mode)))
+		return 0;
+
+	path = mkpath("%s/.git", ce->name);
+	fprintf(stderr, "path: %s\n", path);
+	if (lstat(path, &st)) {
+		path = git_path("submodules/%s/.git", ce->name);
+		fprintf(stderr, "path: %s\n", path);
+		if (lstat(path, &st)) {
+			int err;
+			err = clone_submodule(ce->name);
+			if (err)
+				return err;
+		}
+	}
+
+	/* Now check that the commit is available and fetch if needed */
+
+	return 0;
+}
+
 /*
  * Two-way merge.
  *
@@ -833,6 +864,17 @@ int twoway_merge(struct cache_entry **src,
 	if (newtree == o->df_conflict_entry)
 		newtree = NULL;
 
+	if (o->submodules && o->update) {
+		int err;
+		err = ensure_submodule(current, o);
+		if (!err)
+			err = ensure_submodule(oldtree, o);
+		if (!err)
+			err = ensure_submodule(newtree, o);
+		if (err)
+			return err;
+	}
+
 	if (current) {
 		if ((!oldtree && !newtree) || /* 4 and 5 */
 		    (!oldtree && newtree &&
@@ -909,6 +951,15 @@ int oneway_merge(struct cache_entry **src,
 		return error("Cannot do a oneway merge of %d trees",
 			     o->merge_size);
 
+	if (o->submodules && o->update) {
+		int err;
+		err = ensure_submodule(old, o);
+		if (!err)
+			err = ensure_submodule(a, o);
+		if (err)
+			return err;
+	}
+
 	if (!a)
 		return deleted_entry(old, old, o);
 	if (old && same(old, a)) {
-- 
1.5.2.rc3.815.g8fc2

[PATCH 14/15] entry.c: optionall checkout newly cloned submodules

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

---
 entry.c |   58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 files changed, 56 insertions(+), 2 deletions(-)

diff --git a/entry.c b/entry.c
index 8c70a47..24bf161 100644
--- a/entry.c
+++ b/entry.c
@@ -104,10 +104,63 @@ static int checkout_submodule(struct cache_entry *ce, const char *path, const st
 	return 0;
 }
 
+static const char *relativize_path(const char *path, const char *dest)
+{
+	static char relative_path[PATH_MAX];
+	int slashes;
+	const char *cp;
+	char *rp;
+
+	if (path[0] == '/')
+		return path;
+
+	for (slashes = 0, cp = strchr(dest, '/'); cp; cp = strchr(cp, '/')) {
+		++slashes;
+		while (*cp == '/')
+			++cp;
+	}
+	if (3 * slashes + strlen(path) + 1 > sizeof(relative_path))
+		die("path too long");
+
+	rp = relative_path;
+	while (slashes--) {
+		memcpy(rp, "../", 3);
+		rp += 3;
+	}
+	strcpy(rp, path);
+
+	return relative_path;
+}
+
+static int write_submodule(struct cache_entry *ce, char *path, const struct checkout *state)
+{
+	struct stat st;
+	const char *submodule_dir, *dest;
+
+	if (mkdir(path, 0777) < 0)
+		return error("git-checkout-index: cannot create subproject directory %s", path);
+
+	if (!state->submodules)
+		return 0;
+
+	submodule_dir = git_path("submodules/%s/.git", ce->name);
+	if (lstat(submodule_dir, &st))
+		return error("submodule '%s' unavailable", ce->name);
+
+	dest = mkpath("%s/.git", ce->name);
+	submodule_dir = relativize_path(submodule_dir, dest);
+
+	if (symlink(submodule_dir, dest))
+		return -1;
+
+	return checkout_submodule(ce, path, state);
+}
+
 static int write_entry(struct cache_entry *ce, char *path, const struct checkout *state, int to_tempfile)
 {
 	int fd;
 	long wrote;
+	int err;
 
 	switch (ntohl(ce->ce_mode) & S_IFMT) {
 		char *buf, *new;
@@ -177,8 +230,9 @@ static int write_entry(struct cache_entry *ce, char *path, const struct checkout
 	case S_IFDIRLNK:
 		if (to_tempfile)
 			return error("git-checkout-index: cannot create temporary subproject %s", path);
-		if (mkdir(path, 0777) < 0)
-			return error("git-checkout-index: cannot create subproject directory %s", path);
+		err = write_submodule(ce, path, state);
+		if (err)
+			return err;
 		break;
 	default:
 		return error("git-checkout-index: unknown file mode for %s", path);
-- 
1.5.2.rc3.815.g8fc2

[PATCH 15/15] git-clone: add --submodules for cloning submodules

[skimo]

From: Sven Verdoolaege <skimo@kotnet.org>

When the --submodules option is specified, git-clone will search
for submodule.<submodule>.url options in the remote configuration
and clone each submodule using the first url that it can use from
the local site.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 Documentation/git-clone.txt |    6 +++++-
 git-clone.sh                |   16 ++++++++++++++--
 2 files changed, 19 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 644bf12..565155b 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -11,7 +11,7 @@ SYNOPSIS
 [verse]
 'git-clone' [--template=<template_directory>] [-l [-s]] [-q] [-n] [--bare]
 	  [-o <name>] [-u <upload-pack>] [--reference <repository>]
-	  [--depth <depth>] <repository> [<directory>]
+	  [--depth <depth>] [--submodules] <repository> [<directory>]
 
 DESCRIPTION
 -----------
@@ -105,6 +105,10 @@ OPTIONS
 	with a long history, and would want to send in a fixes
 	as patches.
 
+--submodules::
+	Clone submodules specified in (remote) configuration parameters
+	submodule.<submodule>.url.
+
 <repository>::
 	The (possibly remote) repository to clone from.  It can
 	be any URL git-fetch supports.
diff --git a/git-clone.sh b/git-clone.sh
index fdd354f..a51b887 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -14,7 +14,7 @@ die() {
 }
 
 usage() {
-	die "Usage: $0 [--template=<template_directory>] [--reference <reference-repo>] [--bare] [-l [-s]] [-q] [-u <upload-pack>] [--origin <name>] [--depth <n>] [-n] <repo> [<dir>]"
+	die "Usage: $0 [--template=<template_directory>] [--reference <reference-repo>] [--bare] [-l [-s]] [-q] [-u <upload-pack>] [--origin <name>] [--depth <n>] [-n] [--submodules] <repo> [<dir>]"
 }
 
 get_repo_base() {
@@ -88,6 +88,7 @@ origin_override=
 use_separate_remote=t
 depth=
 no_progress=
+submodules=
 test -t 1 || no_progress=--no-progress
 while
 	case "$#,$1" in
@@ -138,6 +139,8 @@ while
 	*,--depth)
 		shift
 		depth="--depth=$1";;
+	*,--su|*,--sub|*,--subm|*,--submo|*,--submod|*,--submodu|*,--submodul|\
+	*,--submodule|*,--submodules) submodules="--submodules" ;;
 	*,-*) usage ;;
 	*) break ;;
 	esac
@@ -156,6 +159,10 @@ then
 	then
 		die '--bare and --origin $origin options are incompatible.'
 	fi
+	if test -n "$submodules"
+	then
+		die '--bare and --submodules origin options are incompatible.'
+	fi
 	no_checkout=yes
 	use_separate_remote=
 fi
@@ -401,10 +408,15 @@ then
 		git-config branch."$head_points_at".merge "refs/heads/$head_points_at"
 	esac
 
+	if test -n "$submodules"
+	then
+		git-config core.submodules true
+	fi
+
 	case "$no_checkout" in
 	'')
 		test "z$quiet" = z -a "z$no_progress" = z && v=-v || v=
-		git-read-tree -m -u $v HEAD HEAD
+		git-read-tree -m -u $v $submodules HEAD HEAD
 	esac
 fi
 rm -f "$GIT_DIR/CLONE_HEAD" "$GIT_DIR/REMOTE_HEAD"
-- 
1.5.2.rc3.815.g8fc2

Re: [RFC] Third round of support for cloning submodules

[Junio C Hamano]

skimo@liacs.nl writes:

> This patch series implements a mechanism for cloning submodules.

Let me start by asking a few stupid questions.

> Each submodule is specified by a 'submodule.<submodule>.url'
> configuration option, e.g.,
>
> bash-3.00$ ./git-config --remote=http://www.liacs.nl/~sverdool/isa.git --get-regexp 'submodule\..*\.url' 
> submodule.cloog.url /home/sverdool/public_html/cloog.git
> submodule.cloog.url http://www.liacs.nl/~sverdool/cloog.git

You are priming the process by having these configuration
variables in the superproject to be cloned (i.e. this is done by
the owner of the superproject).

> git-checkout will use the first url that works.
> E.g., a
>
> git clone --submodules ssh://liacs/~/public_html/isa.git
>
> followed by
>
> git checkout origin/submodule
>
> (which only works for me), will use the first url, while a
>
> git clone --submodules http://www.liacs.nl/~sverdool/isa.git
>
> followed by
>
> git checkout origin/submodule
>
> will use the second.

What's the design like to make clone and checkout work together?
When you run the first clone with -n ("do not checkout"), what
should happen (I am not asking what your code does, but what the
desired behaviour should be)?  My take on that question is
"subproject cloning is done by checkout, not clone".

> The cloning of submodules is now handled inside git-checkout.

which I guess means your answer is the same as mine, which
is fine.

I am very worried about this big red switch that says "all
subprojects to be cloned and checked out, or nothing".  I think
this would not work well with projects that truly need
superproject support (i.e. very large ones, where most people
would not want to clone and check out every single subproject).

> I currently do not fetch after the initial clone, since
> I'm not sure what ref to use for the revision I need to
> fetch for the supermodule.

I think fetching inside the subproject can be safely done with
the default (i.e. refs/heads/*:refs/remotes/origin/*) of 1.5.0
or later, as long as we tell the users of the feature that they
should make sure that the commit referenced by superproject tree
entries are available with such a fetch, which is a sane thing
to require anyway.

The more important issue I think is at what point in the
superproject operation does a recursive checkout in a subproject
should happen, and how we should do the checkout.  Issues I can
think of offhand are (no way exhaustive):

 - Do we checkout a branch? if so which one?

 - Do we detach HEAD if the commit named by the superproject
   tree is not at the tip of the current branch of subproject?
   do we detach always even if the commit is at the tip?

 - What would we do when the subproject working tree is not
   clean?

 - How can a user decide which subproject to descend into and
   which subproject to ignore, and how does git remember the
   earlier decision made by the user without asking the same
   again, and how does a user express "now I want to also track
   that subproject I've ignored so far" and "now I am not
   interested in following that subproject anymore"?

So I tend to disagree with not having the indirection we
discussed on the other thread about .gitmodules, but I consider
it a minor detail of cloning, and it is not a major deal to me.

However, I agree with Alex that checkout semantics is a much
bigger deal, and would expect people (brighter than myself,
hopefully) to offer ideas.

Re: [RFC] Third round of support for cloning submodules

[Sven Verdoolaege]

On Sun, May 20, 2007 at 12:10:04PM -0700, Junio C Hamano wrote:
> > bash-3.00$ ./git-config --remote=http://www.liacs.nl/~sverdool/isa.git --get-regexp 'submodule\..*\.url' 
> > submodule.cloog.url /home/sverdool/public_html/cloog.git
> > submodule.cloog.url http://www.liacs.nl/~sverdool/cloog.git
> 
> You are priming the process by having these configuration
> variables in the superproject to be cloned (i.e. this is done by
> the owner of the superproject).

Is that a question?
The answer would be that if the user didn't put this information
in a config file, then git will try to get the information from
any remote it can get its hands on.

> What's the design like to make clone and checkout work together?
> When you run the first clone with -n ("do not checkout"), what
> should happen (I am not asking what your code does, but what the
> desired behaviour should be)?  My take on that question is
> "subproject cloning is done by checkout, not clone".

Then I guess you are not asking me.
I took the suggestion of doing the subproject cloning during checkout
from you.

> I am very worried about this big red switch that says "all
> subprojects to be cloned and checked out, or nothing".  I think
> this would not work well with projects that truly need
> superproject support (i.e. very large ones, where most people
> would not want to clone and check out every single subproject).

It's pretty easy to add a "submodule.*.skip" or "submodule.*.ignore".
Since the subcloning only happens at checkout, you could set these
before doing a checkout.

> > I currently do not fetch after the initial clone, since
> > I'm not sure what ref to use for the revision I need to
> > fetch for the supermodule.
> 
> I think fetching inside the subproject can be safely done with
> the default (i.e. refs/heads/*:refs/remotes/origin/*) of 1.5.0
> or later, as long as we tell the users of the feature that they
> should make sure that the commit referenced by superproject tree
> entries are available with such a fetch, which is a sane thing
> to require anyway.

Seems like a pretty strict requirement, but it's easy to implement,
so I guess I can do that in the fourth version.

> The more important issue I think is at what point in the
> superproject operation does a recursive checkout in a subproject
> should happen, and how we should do the checkout.  Issues I can
> think of offhand are (no way exhaustive):
> 
>  - Do we checkout a branch? if so which one?
> 
>  - Do we detach HEAD if the commit named by the superproject
>    tree is not at the tip of the current branch of subproject?
>    do we detach always even if the commit is at the tip?

I thought there was a consensus to detach the HEAD.
I don't have a strong opinion on this issue, but a detached
HEAD seems the most appropriate to me.

>  - What would we do when the subproject working tree is not
>    clean?

I was planning on adding a --dry-run to git-checkout.
The superproject would run this in each subproject before
doing the actual checkout of the superproject.

>  - How can a user decide which subproject to descend into and
>    which subproject to ignore, and how does git remember the
>    earlier decision made by the user without asking the same
>    again, and how does a user express "now I want to also track
>    that subproject I've ignored so far" and "now I am not
>    interested in following that subproject anymore"?

Just twiddle the "submodule.*.skip" option.

> So I tend to disagree with not having the indirection we
> discussed on the other thread about .gitmodules, but I consider
> it a minor detail of cloning, and it is not a major deal to me.

Some form of indirection is definitely required (although
not for my use of submodules) and I'll probably add it in a future round.

skimo

Re: [RFC] Third round of support for cloning submodules

[Alex Riesen]

Sven Verdoolaege, Sun, May 20, 2007 21:59:30 +0200:
> > I am very worried about this big red switch that says "all
> > subprojects to be cloned and checked out, or nothing".  I think
> > this would not work well with projects that truly need
> > superproject support (i.e. very large ones, where most people
> > would not want to clone and check out every single subproject).
> 
> It's pretty easy to add a "submodule.*.skip" or "submodule.*.ignore".
> Since the subcloning only happens at checkout, you could set these
> before doing a checkout.

And set them back after doing the checkout? And so for each and every
checkout operation? I suggest you try checkout something like KDE a
few times (well, assuming KDE is split in submodules. It isn't yet).
The point is - it is annoying. And if it is annoying, it wont be used
(as branches in CVS and merging in SVN).

> > The more important issue I think is at what point in the
> > superproject operation does a recursive checkout in a subproject
> > should happen, and how we should do the checkout.  Issues I can
> > think of offhand are (no way exhaustive):
> > 
> >  - Do we checkout a branch? if so which one?
> > 
> >  - Do we detach HEAD if the commit named by the superproject
> >    tree is not at the tip of the current branch of subproject?
> >    do we detach always even if the commit is at the tip?
> 
> I thought there was a consensus to detach the HEAD.
> I don't have a strong opinion on this issue, but a detached
> HEAD seems the most appropriate to me.

Me too. I actually believe it is the only way to do it. How can you
checkout a subproject to something else (to what a branch may point)
and to what the tree of superproject has? On the other side (in
subproject) - why are you, the superproject, allowed to screw the
references of the subproject?! It is independent, isn't it?!

> >  - What would we do when the subproject working tree is not
> >    clean?
> 
> I was planning on adding a --dry-run to git-checkout.
> The superproject would run this in each subproject before
> doing the actual checkout of the superproject.

Why not do exactly what we do now? Pass "-m" down to it, if it was
given to the top-level git-checkout.

Re: [RFC] Third round of support for cloning submodules

[Sven Verdoolaege]

On Sun, May 20, 2007 at 10:54:44PM +0200, Alex Riesen wrote:
> Sven Verdoolaege, Sun, May 20, 2007 21:59:30 +0200:
> > > I am very worried about this big red switch that says "all
> > > subprojects to be cloned and checked out, or nothing".  I think
> > > this would not work well with projects that truly need
> > > superproject support (i.e. very large ones, where most people
> > > would not want to clone and check out every single subproject).
> > 
> > It's pretty easy to add a "submodule.*.skip" or "submodule.*.ignore".
> > Since the subcloning only happens at checkout, you could set these
> > before doing a checkout.
> 
> And set them back after doing the checkout?

What do you mean?  Why would you set them back?
I guess I'm missing something.

> Me too. I actually believe it is the only way to do it. How can you
> checkout a subproject to something else (to what a branch may point)
> and to what the tree of superproject has? On the other side (in
> subproject) - why are you, the superproject, allowed to screw the
> references of the subproject?! It is independent, isn't it?!

Well... the subproject as a whole is independent of the superproject,
but the checkout in the superproject is not entirely independent.

> > >  - What would we do when the subproject working tree is not
> > >    clean?
> > 
> > I was planning on adding a --dry-run to git-checkout.
> > The superproject would run this in each subproject before
> > doing the actual checkout of the superproject.
> 
> Why not do exactly what we do now? Pass "-m" down to it, if it was
> given to the top-level git-checkout.

We want to be sure that all (selected) subprojects can be updated before
updating any, no?

skimo

Re: [RFC] Third round of support for cloning submodules

[Alex Riesen]

Sven Verdoolaege, Sun, May 20, 2007 23:09:54 +0200:
> On Sun, May 20, 2007 at 10:54:44PM +0200, Alex Riesen wrote:
> > Sven Verdoolaege, Sun, May 20, 2007 21:59:30 +0200:
> > > > I am very worried about this big red switch that says "all
> > > > subprojects to be cloned and checked out, or nothing".  I think
> > > > this would not work well with projects that truly need
> > > > superproject support (i.e. very large ones, where most people
> > > > would not want to clone and check out every single subproject).
> > > 
> > > It's pretty easy to add a "submodule.*.skip" or "submodule.*.ignore".
> > > Since the subcloning only happens at checkout, you could set these
> > > before doing a checkout.
> > 
> > And set them back after doing the checkout?
> 
> What do you mean?  Why would you set them back?

Why should I set them before doing a checkout?

> I guess I'm missing something.

"checkout" is an operation which is done often. It never had to be
configured before.

> > Me too. I actually believe it is the only way to do it. How can you
> > checkout a subproject to something else (to what a branch may point)
> > and to what the tree of superproject has? On the other side (in
> > subproject) - why are you, the superproject, allowed to screw the
> > references of the subproject?! It is independent, isn't it?!
> 
> Well... the subproject as a whole is independent of the superproject,
> but the checkout in the superproject is not entirely independent.
>

Junio was talking about branch in subproject, wasn't he?

> > > >  - What would we do when the subproject working tree is not
> > > >    clean?
> > > 
> > > I was planning on adding a --dry-run to git-checkout.
> > > The superproject would run this in each subproject before
> > > doing the actual checkout of the superproject.
> > 
> > Why not do exactly what we do now? Pass "-m" down to it, if it was
> > given to the top-level git-checkout.
> 
> We want to be sure that all (selected) subprojects can be updated before
> updating any, no?
> 

I guess passing "-m" to git-checkout _is_ an explicit permission from
the operator to perform a merge. Besides, it's visible: merge prints
something, user sees the "-m" in command history (or in script code).
Calling git-checkout twice even if we don't have to... it is kind of
ugly. Still need some dry-run this for normal case (checkout can be
modified to do this by default, I think).

Re: [RFC] Third round of support for cloning submodules

[Sven Verdoolaege]

On Sun, May 20, 2007 at 11:24:32PM +0200, Alex Riesen wrote:
> Sven Verdoolaege, Sun, May 20, 2007 23:09:54 +0200:
> > On Sun, May 20, 2007 at 10:54:44PM +0200, Alex Riesen wrote:
> > > Sven Verdoolaege, Sun, May 20, 2007 21:59:30 +0200:
> > > > > I am very worried about this big red switch that says "all
> > > > > subprojects to be cloned and checked out, or nothing".  I think
> > > > > this would not work well with projects that truly need
> > > > > superproject support (i.e. very large ones, where most people
> > > > > would not want to clone and check out every single subproject).
> > > > 
> > > > It's pretty easy to add a "submodule.*.skip" or "submodule.*.ignore".
> > > > Since the subcloning only happens at checkout, you could set these
> > > > before doing a checkout.
> > > 
> > > And set them back after doing the checkout?
> > 
> > What do you mean?  Why would you set them back?
> 
> Why should I set them before doing a checkout?
> 
> > I guess I'm missing something.
> 
> "checkout" is an operation which is done often. It never had to be
> configured before.

There is going to have to be *some* way of selecting which
subprojects you want to check out.  A config option that you
have to set only once (or not at all if you are happy with
the default) seems to be the easiest way.  You can have git-gui
set them for you if you want.

How would _you_ specify which subprojects to checkout ?

> > Well... the subproject as a whole is independent of the superproject,
> > but the checkout in the superproject is not entirely independent.
> >
> 
> Junio was talking about branch in subproject, wasn't he?

That's a local thing.

skimo

Re: [RFC] Third round of support for cloning submodules

[Alex Riesen]

Sven Verdoolaege, Sun, May 20, 2007 23:47:32 +0200:
> 
> How would _you_ specify which subprojects to checkout ?
> 

Aren't the ones which already have .git in them are kind of specified?

Re: [RFC] Third round of support for cloning submodules

[Sven Verdoolaege]

On Mon, May 21, 2007 at 12:26:21AM +0200, Alex Riesen wrote:
> Sven Verdoolaege, Sun, May 20, 2007 23:47:32 +0200:
> > 
> > How would _you_ specify which subprojects to checkout ?
> > 
> 
> Aren't the ones which already have .git in them are kind of specified?
> 

Would you always recurse into these submodules, regardless of
any option?
Or would you want two options, one for handling the submodules
you have explicitly marked someway and one for getting all submodules?

skimo

Re: [RFC] Third round of support for cloning submodules

[Josef Weidendorfer]

On Monday 21 May 2007, Sven Verdoolaege wrote:
> On Mon, May 21, 2007 at 12:26:21AM +0200, Alex Riesen wrote:
> > Sven Verdoolaege, Sun, May 20, 2007 23:47:32 +0200:
> > > 
> > > How would _you_ specify which subprojects to checkout ?
> > > 
> > 
> > Aren't the ones which already have .git in them are kind of specified?
> > 
> 
> Would you always recurse into these submodules, regardless of
> any option?
> Or would you want two options, one for handling the submodules
> you have explicitly marked someway and one for getting all submodules?

There should be a way for a superproject to specify useful sets of
subprojects for different developer roles, and these sets should be
versioned. It is also useful for a superproject to be able to say
"for this subproject to work, that other subprojects needs to be
checked out".

Both issues could be supported with a "dependson" setting in .gitmodules
(or better call this file ".gitprojects"?)

 [subproject "german-translation"]
    path = lang/german
    dependson = docbuilds

 [subproject "all-translations"]
    dependson = german-translation france-translation japanese-translation 

The syntax here only is RFC, including the fact that this example
puts the subproject identifier into the key, and the path as config.
If we do not go the .gitattributes way, IMHO this is more logical.

When cloning, one should be allowed to specify the subprojects one wants
to track, e.g.

 git-clone --subproject=all-translations ...

Josef

Re: [RFC] Third round of support for cloning submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 1116 bytes --]

hoi :)

On Mon, May 21, 2007 at 12:44:16PM +0200, Josef Weidendorfer wrote:
> There should be a way for a superproject to specify useful sets of
> subprojects for different developer roles, and these sets should be
> versioned. It is also useful for a superproject to be able to say
> "for this subproject to work, that other subprojects needs to be
> checked out".

What subprojects to use is the responsibility of the build system and
we should not step on its shoes too much.
We should provide a simple way to populate a submodule, but all the
dependency handling should really be done in the build system /
package handling system on top of git, IMHO.

Perhaps we can simply provide "cd $subproject && git clone" to
automatically fetch all needed stuff from a default location and
checkout that subproject.
Then we can integrate that command in bitbake and whatnot or start
a new configuration management system on top of git which uses
dependencies from Makefiles etc. to automatically check out the
right set of subprojects.  But that should really be on top of git.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Third round of support for cloning submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 1399 bytes --]

hoi :)

On Sun, May 20, 2007 at 10:54:44PM +0200, Alex Riesen wrote:
> Me too. I actually believe it is the only way to do it. How can you
> checkout a subproject to something else (to what a branch may point)
> and to what the tree of superproject has? On the other side (in
> subproject) - why are you, the superproject, allowed to screw the
> references of the subproject?! It is independent, isn't it?!

right.  except when you have some managed-by-superproject branch
which is known to be special ;-)

After all the submodule checkout is independent from its parent
repository, too -- so you don't screw anything *g*.


> > >  - What would we do when the subproject working tree is not
> > >    clean?
> > 
> > I was planning on adding a --dry-run to git-checkout.
> > The superproject would run this in each subproject before
> > doing the actual checkout of the superproject.
> 
> Why not do exactly what we do now? Pass "-m" down to it, if it was
> given to the top-level git-checkout.

sounds good.
With submodules we have to consider one extra level of merging.
-m in the supermodule also means that an automatic merge of the
dirlink entry should be done.  Which would execute git-merge in the
submodule.  And merging in a dirty tree is a challenge of its own.

So if local changes conflict with the checkout we should just error out.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Third round of support for cloning submodules

[Alex Riesen]

Martin Waitz, Sun, May 20, 2007 23:40:26 +0200:
> > > >  - What would we do when the subproject working tree is not
> > > >    clean?
> > > 
> > > I was planning on adding a --dry-run to git-checkout.
> > > The superproject would run this in each subproject before
> > > doing the actual checkout of the superproject.
> > 
> > Why not do exactly what we do now? Pass "-m" down to it, if it was
> > given to the top-level git-checkout.
> 
> sounds good.
> With submodules we have to consider one extra level of merging.
> -m in the supermodule also means that an automatic merge of the
> dirlink entry should be done.  Which would execute git-merge in the
> submodule.  And merging in a dirty tree is a challenge of its own.

But it is not a merge. It is a checkout. Being another operation it
may even be disallow merges of subprojects. Just plainly tell user
that this checkout is not possible because there are changes in
subprojects and in the pointer to this subproject in the upper level
superproject, and that the user should think about committing in
subproject first.

> So if local changes conflict with the checkout we should just error out.

On account of it being too complex. Always a good reason.

Re: [RFC] Third round of support for cloning submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 553 bytes --]

hoi :)

On Mon, May 21, 2007 at 12:24:10AM +0200, Alex Riesen wrote:
> But it is not a merge. It is a checkout. Being another operation it
> may even be disallow merges of subprojects. Just plainly tell user
> that this checkout is not possible because there are changes in
> subprojects and in the pointer to this subproject in the upper level
> superproject, and that the user should think about committing in
> subproject first.

If the user did commit and then you do a supermodule checkout -m you
will get a merge.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Third round of support for cloning submodules

[Alex Riesen]

Martin Waitz, Mon, May 21, 2007 00:55:22 +0200:
> hoi :)
> 
> On Mon, May 21, 2007 at 12:24:10AM +0200, Alex Riesen wrote:
> > But it is not a merge. It is a checkout. Being another operation it
> > may even be disallow merges of subprojects. Just plainly tell user
> > that this checkout is not possible because there are changes in
> > subprojects and in the pointer to this subproject in the upper level
> > superproject, and that the user should think about committing in
> > subproject first.
> 
> If the user did commit and then you do a supermodule checkout -m you
> will get a merge.
> 

Only if the user continue to use the last branch (or the detached
head) the subproject was on. He don't have to, he can even return to
the commit which does not conflict, unless he have to complicate
things.

Re: [RFC] Third round of support for cloning submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 538 bytes --]

hoi :)

On Mon, May 21, 2007 at 01:02:48AM +0200, Alex Riesen wrote:
> > If the user did commit and then you do a supermodule checkout -m you
> > will get a merge.
> 
> Only if the user continue to use the last branch (or the detached
> head) the subproject was on. He don't have to, he can even return to
> the commit which does not conflict, unless he have to complicate
> things.

just curious:
so you want to differenciate between a subproject HEAD which was
set by the superproject and other ones?

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Third round of support for cloning submodules

[Alex Riesen]

Martin Waitz, Mon, May 21, 2007 01:12:01 +0200:
> On Mon, May 21, 2007 at 01:02:48AM +0200, Alex Riesen wrote:
> > > If the user did commit and then you do a supermodule checkout -m you
> > > will get a merge.
> > 
> > Only if the user continue to use the last branch (or the detached
> > head) the subproject was on. He don't have to, he can even return to
> > the commit which does not conflict, unless he have to complicate
> > things.
> 
> just curious:
> so you want to differenciate between a subproject HEAD which was
> set by the superproject and other ones?

No. Why do you think that I want to do that?

Re: [RFC] Third round of support for cloning submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 1058 bytes --]

hoi :)

On Tue, May 22, 2007 at 11:54:23PM +0200, Alex Riesen wrote:
> Martin Waitz, Mon, May 21, 2007 01:12:01 +0200:
> > On Mon, May 21, 2007 at 01:02:48AM +0200, Alex Riesen wrote:
> > > > If the user did commit and then you do a supermodule checkout -m you
> > > > will get a merge.
> > > 
> > > Only if the user continue to use the last branch (or the detached
> > > head) the subproject was on. He don't have to, he can even return to
> > > the commit which does not conflict, unless he have to complicate
> > > things.
> > 
> > just curious:
> > so you want to differenciate between a subproject HEAD which was
> > set by the superproject and other ones?
> 
> No. Why do you think that I want to do that?

I think I simply was too tired to read. ;-)

Your 'Only if the user continues to use...' suggested that he has a
different option, namely going to a branch which is not controlled
by the superproject.

And of course the user can go back to another commit, after checkout -m
created a merge... ;-)

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Third round of support for cloning submodules

[Steven Grimm]

Sven Verdoolaege wrote:
> It's pretty easy to add a "submodule.*.skip" or "submodule.*.ignore".
> Since the subcloning only happens at checkout, you could set these
> before doing a checkout.
>   

Can I take this to mean that you intend the default behavior to be to 
check out all subprojects, with individual ones suppressed via 
configuration as needed?

Picture a corporate development environment with a common build system, 
a couple common libraries, and a bunch of separate products. Product 
developers will want to check out the build system (which would perhaps 
be in the superproject), the libraries they need, and their own product. 
They will rarely want to check out the other products, which could 
account for the vast majority of the subprojects, and certainly won't 
want to have to keep track of which new subprojects are appearing so 
they can add those to the exclude list.

In other words, "I want the superproject and these four subprojects and 
nothing else" should be a well-supported mode of operation. In many 
cases developers will already know at clone time exactly what they want.

If I'm misunderstanding your intent, then never mind. :)

-Steve

Re: [RFC] Third round of support for cloning submodules

[Sven Verdoolaege]

On Sun, May 20, 2007 at 05:39:10PM -0700, Steven Grimm wrote:
> Sven Verdoolaege wrote:
> >It's pretty easy to add a "submodule.*.skip" or "submodule.*.ignore".
> >Since the subcloning only happens at checkout, you could set these
> >before doing a checkout.
> >  
> 
> Can I take this to mean that you intend the default behavior to be to 
> check out all subprojects, with individual ones suppressed via 
> configuration as needed?

Or we could have a tri-state variable, with "yes" meaning handle
the submodule, "no" don't, and undefined meaning do whatever is
specified by the global submodules option.
Explicitly checking out a submodule could then set the variable to "yes".

skimo

Re: [RFC] Third round of support for cloning submodules

[Alex Riesen]

Steven Grimm, Mon, May 21, 2007 02:39:10 +0200:
> Sven Verdoolaege wrote:
> >It's pretty easy to add a "submodule.*.skip" or "submodule.*.ignore".
> >Since the subcloning only happens at checkout, you could set these
> >before doing a checkout.
> >  
> 
> Can I take this to mean that you intend the default behavior to be to 
> check out all subprojects, with individual ones suppressed via 
> configuration as needed?

Neither fetch nor checkout none of them and leave the subproject
fetch+checkout as exercise to the user?

Re: [RFC] Third round of support for cloning submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 2466 bytes --]

hoi :)

On Sun, May 20, 2007 at 12:10:04PM -0700, Junio C Hamano wrote:
> The more important issue I think is at what point in the
> superproject operation does a recursive checkout in a subproject
> should happen, and how we should do the checkout.

we should really move from our big clone thing to a simple
fetch+checkout wrapper.

And then integrate all the submodule logic into the actual checkout
step where it belongs.

We might also want to expand fetch to also fetch newly reachable
submodule commits (of a configurable subset of modules).

> Issues I can think of offhand are (no way exhaustive):
> 
>  - Do we checkout a branch? if so which one?

At least no off-the-shelf branch from the upstream repository of
the submodule.

To use some special branch allows to use normal git methods in
the submodule, too -- but I haven't been able to convince everybody
yet...  So let's get it to a state where people can play with it
in real projects and let's see.


>  - Do we detach HEAD if the commit named by the superproject
>    tree is not at the tip of the current branch of subproject?
>    do we detach always even if the commit is at the tip?

We must not mess with random upstream branches of the submodule
just because they happen to reference the same tip.
That would be too confusing.
Either use one special branch or detach.

>  - What would we do when the subproject working tree is not
>    clean?

The same as with normal files:
error out if something is changed which conflicts with the requested
update.

When we have a special managed-by-supermodule branch and the submodule
has another branch currently checked out we can entirely ignore this
issue.
This really allows the user to deliberately keep one module in an
unclean state.

>  - How can a user decide which subproject to descend into and
>    which subproject to ignore, and how does git remember the
>    earlier decision made by the user without asking the same
>    again, and how does a user express "now I want to also track
>    that subproject I've ignored so far" and "now I am not
>    interested in following that subproject anymore"?

I'd simply use explicit checkout of a submodule and removal of
the submodule to be a fine way to express the user's wish.
Of course we also need some way to say: populate everything
below "src/target" or similar.  But that is independent from
the rest.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Third round of support for cloning submodules

[Alex Riesen]

Martin Waitz, Mon, May 21, 2007 00:14:55 +0200:
> >  - Do we detach HEAD if the commit named by the superproject
> >    tree is not at the tip of the current branch of subproject?
> >    do we detach always even if the commit is at the tip?
> 
> We must not mess with random upstream branches of the submodule
> just because they happen to reference the same tip.
> That would be too confusing.

Strange. The very same reason I heard when I tried to explain why
branches are good. The people found them confusing, just like you now.
They preach Perforce, too.

> Either use one special branch or detach.

Why not just detach always?

> >  - What would we do when the subproject working tree is not
> >    clean?
> 
> The same as with normal files:
> error out if something is changed which conflicts with the requested
> update.

This is called tree-level merge. Done by -m option (it does more than
that, yes, but this one too). While at it we can do file-level merge
as well, why not?

> When we have a special managed-by-supermodule branch and the submodule
> has another branch currently checked out we can entirely ignore this
> issue.

Detached head isn't special enough?

> >  - How can a user decide which subproject to descend into and
> >    which subproject to ignore, and how does git remember the
> >    earlier decision made by the user without asking the same
> >    again, and how does a user express "now I want to also track
> >    that subproject I've ignored so far" and "now I am not
> >    interested in following that subproject anymore"?
> 
> I'd simply use explicit checkout of a submodule and removal of
> the submodule to be a fine way to express the user's wish.

The directory of the submodule will be back by the next
git-checkout-index (in the current implementation).
Not the .git's, so yes it is a fine way to express user's wish: he
just initialize the subprojects he wants (by whatever way).

Re: [RFC] Third round of support for cloning submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 2344 bytes --]

hoi :)

On Mon, May 21, 2007 at 12:58:10AM +0200, Alex Riesen wrote:
> Martin Waitz, Mon, May 21, 2007 00:14:55 +0200:
> > >  - Do we detach HEAD if the commit named by the superproject
> > >    tree is not at the tip of the current branch of subproject?
> > >    do we detach always even if the commit is at the tip?
> > 
> > We must not mess with random upstream branches of the submodule
> > just because they happen to reference the same tip.
> > That would be too confusing.
> 
> Strange. The very same reason I heard when I tried to explain why
> branches are good. The people found them confusing, just like you now.
> They preach Perforce, too.

Sorry, you lost me.

I didn't say that branches are bad but that guessing branch names based on
their tip is bad.


> > Either use one special branch or detach.
> 
> Why not just detach always?

Which is just another name for "unnamed special branch" ;-)
When you give it a name you can actually use it even after you switched
to another one.

> > >  - What would we do when the subproject working tree is not
> > >    clean?
> > 
> > The same as with normal files:
> > error out if something is changed which conflicts with the requested
> > update.
> 
> This is called tree-level merge. Done by -m option (it does more than
> that, yes, but this one too). While at it we can do file-level merge
> as well, why not?

It's not that easy, for submodules we have different levels of dirty:
 * submodule HEAD matches supermodule index, but submodule working
   directory is dirty.
   If the submodule update would touch any modified file then it should
   fail.
   If used with -m (or perhaps another option? after all this merge
   is in a submodule) then it could do the file-level merge for dirty
   files.
 * submodule HEAD does not match supermodule index
   normal checkout should error out if it would touch the submodule.
   checkout -m has to merge submodule HEAD
And of course:
 * index entry of submodule does not match the entry in supermodule HEAD.
   Same as for files.


> > When we have a special managed-by-supermodule branch and the submodule
> > has another branch currently checked out we can entirely ignore this
> > issue.
> 
> Detached head isn't special enough?

it's too special ;-)

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Third round of support for cloning submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 2224 bytes --]

hoi :)

On Sun, May 20, 2007 at 08:04:33PM +0200, skimo@liacs.nl wrote:
> This patch series implements a mechanism for cloning submodules.
> Each submodule is specified by a 'submodule.<submodule>.url'
> configuration option, e.g.,
> 
> bash-3.00$ ./git-config --remote=http://www.liacs.nl/~sverdool/isa.git --get-regexp 'submodule\..*\.url' 
> submodule.cloog.url /home/sverdool/public_html/cloog.git
> submodule.cloog.url http://www.liacs.nl/~sverdool/cloog.git

I really think we should try to find one standard method to
automatically find the right parent repository for a submodule,
based on the supermodules parent repository.

So e.g. that the submodule repository should stay in the same directory
or in some special subdirectory of the supermodule or even in the
same object store.
Then we can add a configuration layor on top, but there should always
be a sane default.


Things we have to think about:
 * we have to cope with moving / disappearing repositories.
 * we should support bare repositories even for superprojects.
   This can be done either by including the submodule objects in
   the bare repository directly or by linking them (e.g. with your
   config implementation)
 * we have to keep old submodules around forever,
   at least when we want to be able to recover old versions.
   Of course this is not required for all working copies as people
   only want to have a subset of needed modules.
   But for central synchronization repositories (probably the bare ones)
   this is really important.
 * If you remove the whole working directory I don't want to loose any
   data which is already committed, including submodules.

That leads to submodules which store their objects within the
supermodule .git directory, which would automatically obsolete the
need to specify explicit submodule URLs.  But I'm not quite sure
on how to really do this, despite having experimented a bit with it
(my module3 branch should still contain some brainstorming and code).


So back to your code: I don't like absolute URLs in the cloneable part
of the repository.  We should try to stay with relative ones which
can stay the same everywhere.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Third round of support for cloning submodules

[Sven Verdoolaege]

On Mon, May 21, 2007 at 12:52:53AM +0200, Martin Waitz wrote:
> That leads to submodules which store their objects within the
> supermodule .git directory,

My code clones submodules in .git/submodules/<submodule>, so
that could be a good default.

> which would automatically obsolete the
> need to specify explicit submodule URLs.

Absolutely not.  The subproject will likely have a life of its own.
If you export it on the same machine, then why would you have two
different URLs for the same project?
Also, the subproject will typically not even be on the same site,
so you _have_ to be able to specify a submodule URL.
(I noticed that I forgot the "git://" protocol; I'll add that in
the next round.)

> So back to your code: I don't like absolute URLs in the cloneable part
> of the repository.  We should try to stay with relative ones which
> can stay the same everywhere.

The problem with relative paths is that you don't know if the
URL the user gave you points to the working directory or the
git directory of the project, but I guess I can let dump-config
tell you where it found the config file.

skimo

Re: [RFC] Third round of support for cloning submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 1711 bytes --]

hoi :)

On Mon, May 21, 2007 at 10:54:19AM +0200, Sven Verdoolaege wrote:
> On Mon, May 21, 2007 at 12:52:53AM +0200, Martin Waitz wrote:
> > That leads to submodules which store their objects within the
> > supermodule .git directory,
> 
> My code clones submodules in .git/submodules/<submodule>, so
> that could be a good default.

good.

> > which would automatically obsolete the
> > need to specify explicit submodule URLs.
> 
> Absolutely not.  The subproject will likely have a life of its own.
> If you export it on the same machine, then why would you have two
> different URLs for the same project?
> Also, the subproject will typically not even be on the same site,
> so you _have_ to be able to specify a submodule URL.
> (I noticed that I forgot the "git://" protocol; I'll add that in
> the next round.)

Typically, you have to keep it on the same site because you have
some local adaptions which are only ment to be included within the
superproject.  Think about distributions which seldomly use upstream
software completely unmodified.
Being able to configure it for other URLs is nice but by default it
should work without.

> > So back to your code: I don't like absolute URLs in the cloneable part
> > of the repository.  We should try to stay with relative ones which
> > can stay the same everywhere.
> 
> The problem with relative paths is that you don't know if the
> URL the user gave you points to the working directory or the
> git directory of the project, but I guess I can let dump-config
> tell you where it found the config file.

We have already solved that in clone & fetch so I don't think this is a
real problem.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Third round of support for cloning submodules

[Sven Verdoolaege]

On Mon, May 21, 2007 at 12:07:16PM +0200, Martin Waitz wrote:
> > The problem with relative paths is that you don't know if the
> > URL the user gave you points to the working directory or the
> > git directory of the project, but I guess I can let dump-config
> > tell you where it found the config file.
> 
> We have already solved that in clone & fetch so I don't think this is a
> real problem.

How?  AFAICS, it's upload-pack of the *clonee* that determines
where the actual git dir is.  The cloner has no clue.

skimo

Re: [RFC] Third round of support for cloning submodules

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 438 bytes --]

hoi :)

On Mon, May 21, 2007 at 12:14:55PM +0200, Sven Verdoolaege wrote:
> How?  AFAICS, it's upload-pack of the *clonee* that determines
> where the actual git dir is.  The cloner has no clue.

we can just store the link to the workdir-or-gitdir and then try
it out every time we access it.  When clone stores the location
of the parent repository it doesn't know what it points to, but
fetch works, too.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Third round of support for cloning submodules

[Sven Verdoolaege]

On Mon, May 21, 2007 at 01:34:15PM +0200, Martin Waitz wrote:
> hoi :)
> 
> On Mon, May 21, 2007 at 12:14:55PM +0200, Sven Verdoolaege wrote:
> > How?  AFAICS, it's upload-pack of the *clonee* that determines
> > where the actual git dir is.  The cloner has no clue.
> 
> we can just store the link to the workdir-or-gitdir and then try
> it out every time we access it.  When clone stores the location
> of the parent repository it doesn't know what it points to, but
> fetch works, too.

clone is just init + fetch (well, almost).
The comment above applies to fetching as well.

skimo