Wishlist for a bundle-only transport mode

Wishlist for a bundle-only transport mode

[Santi Béjar]

Hi *,

  I have to work in a bundle-only mode, but there are some problems
with it, namely:

1) git-clone does not accept a bundle file, even if git-fetch does.
I've made a patch to use git-fetch in git-clone for this.

2) The bundles created with "git bundle" does not record the HEAD,
they resolve the symbolic name to a branch name.

3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:

[remote "bundle"]
  url = /file/to/bundle
  fetch = "+refs/heads/*:refs/remotes/bundle/*"

$ git push bundle

would create a bundle in /file/to/bundle with the same branches as a
normal git push, but considering the remote branches as the local
remotes/bundle/*

Thank you

Santi

Re: Wishlist for a bundle-only transport mode

[Jakub Narebski]

Santi Béjar wrote:

> 3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:
> 
> [remote "bundle"]
>   url = /file/to/bundle
>   fetch = "+refs/heads/*:refs/remotes/bundle/*"
> 
> $ git push bundle
> 
> would create a bundle in /file/to/bundle with the same branches as a
> normal git push, but considering the remote branches as the local
> remotes/bundle/*

And how you would differentiate between path meaning bundle, and
path meaning "local" protocol, i.e. git repository on the same
filesystem? "bundle = /file/to/bundle" perhaps...

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Wishlist for a bundle-only transport mode

[Santi Béjar]

On Nov 21, 2007 4:04 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> Santi Béjar wrote:
>
> > 3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:
> >
> > [remote "bundle"]
> > url = /file/to/bundle
> > fetch = "+refs/heads/*:refs/remotes/bundle/*"
> >
> > $ git push bundle
> >
> > would create a bundle in /file/to/bundle with the same branches as a
> > normal git push, but considering the remote branches as the local
> > remotes/bundle/*
>
> And how you would differentiate between path meaning bundle, and
> path meaning "local" protocol, i.e. git repository on the same
> filesystem? "bundle = /file/to/bundle" perhaps...

Good point. The git repository must exist before pushing, but the bundle:

1) Exist and is a file, do nothing by default, maybe requiere a --overwrite
2) Does not exist, create the bundle, but maybe you've written badly the url, so
    the best thing would be to say explicitly that you wanted a bundle
like you said.

Santi

Re: Wishlist for a bundle-only transport mode

[Johannes Schindelin]

Hi,

On Wed, 21 Nov 2007, Jakub Narebski wrote:

> Santi B?jar wrote:
> 
> > 3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:
> > 
> > [remote "bundle"]
> > ? url = /file/to/bundle
> > ? fetch = "+refs/heads/*:refs/remotes/bundle/*"
> > 
> > $ git push bundle
> > 
> > would create a bundle in /file/to/bundle with the same branches as a
> > normal git push, but considering the remote branches as the local
> > remotes/bundle/*
> 
> And how you would differentiate between path meaning bundle, and path 
> meaning "local" protocol, i.e. git repository on the same filesystem? 

Maybe because the git repository is specified as an existing directory?  
The bundle is specified as a (possibly non-existing) file...

Hth,
Dscho

Re: Wishlist for a bundle-only transport mode

[Johannes Schindelin]

Hi,

On Wed, 21 Nov 2007, Santi B?jar wrote:

> 1) git-clone does not accept a bundle file, even if git-fetch does. I've 
> made a patch to use git-fetch in git-clone for this.

This, along with rewriting git-clone as a very thin wrapper over git-init, 
-remote and -fetch, is a really low hanging fruit.

Or maybe go the full nine yards and build it in.  Should be a breeze now, 
given parse_options() and run_command().

> 2) The bundles created with "git bundle" does not record the HEAD, they 
> resolve the symbolic name to a branch name.

It imitates ls-remote output.

> 3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:
> 
> [remote "bundle"]
>   url = /file/to/bundle
>   fetch = "+refs/heads/*:refs/remotes/bundle/*"
> 
> $ git push bundle
> 
> would create a bundle in /file/to/bundle with the same branches as a
> normal git push, but considering the remote branches as the local
> remotes/bundle/*

Does not seem to be too complicated to add it to transport.c IMHO.  
Something similar to rsync_transport_push().  And I'd definitely refuse to 
overwrite any existing file.

Ciao,
Dscho

Re: Wishlist for a bundle-only transport mode

[Santi Béjar]

On Nov 21, 2007 4:59 PM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Wed, 21 Nov 2007, Santi B?jar wrote:
>
> > 1) git-clone does not accept a bundle file, even if git-fetch does. I've
> > made a patch to use git-fetch in git-clone for this.
>
> This, along with rewriting git-clone as a very thin wrapper over git-init,
> -remote and -fetch, is a really low hanging fruit.

For the basic/normal mode it can be a very thin wrapper but you have
to support --local, --shared, --reference...

> > 2) The bundles created with "git bundle" does not record the HEAD, they
> > resolve the symbolic name to a branch name.
>
> It imitates ls-remote output.

No, it does not.

With a newly created project with one commit:

$ git ls-remote git.git
b71992713c17c3a29f9566e1b50e8cf402375faf        HEAD
b71992713c17c3a29f9566e1b50e8cf402375faf        refs/heads/master

$ git bundle create git.bdl HEAD master

$ git bundle list-heads git.bdl
b71992713c17c3a29f9566e1b50e8cf402375faf refs/heads/master
b71992713c17c3a29f9566e1b50e8cf402375faf refs/heads/master

Santi

Re: Wishlist for a bundle-only transport mode

[Kristian Høgsberg]

On Wed, 2007-11-21 at 15:59 +0000, Johannes Schindelin wrote:
> Hi,
> 
> On Wed, 21 Nov 2007, Santi B?jar wrote:
> 
> > 1) git-clone does not accept a bundle file, even if git-fetch does. I've 
> > made a patch to use git-fetch in git-clone for this.
> 
> This, along with rewriting git-clone as a very thin wrapper over git-init, 
> -remote and -fetch, is a really low hanging fruit.
> 
> Or maybe go the full nine yards and build it in.  Should be a breeze now, 
> given parse_options() and run_command().

I started doing this, which is pretty stupid as I'm pressed to find time
to finish up builtin-commit.  Nevertheless, could you elaborate on the
plan here?  How would you rewrite it to just use remote and fetch?  I
just finished the boilerplate option parsing stuff (patch below) and
started reading the core of git-clone.sh, but if most of this can be
replaced I'd like to hear about it :)

cheers,
Kristian

>From ccfa6b7b6c0bad98201166db3f91760b423ab0c9 Mon Sep 17 00:00:00 2001
From: =?utf-8?q?Kristian=20H=C3=B8gsberg?= <krh@redhat.com>
Date: Sun, 18 Nov 2007 18:48:04 -0500
Subject: [PATCH] builtin-clone: first steps, up to and including
git-init.

---
 Makefile                                      |    2 +-
 builtin-clone.c                               |  286
+++++++++++++++++++++++++
 builtin-init-db.c                             |   90 ++++----
 builtin.h                                     |    1 +
 cache.h                                       |    4 +
 git-clone.sh => contrib/examples/git-clone.sh |    0 
 git.c                                         |    1 +
 7 files changed, 341 insertions(+), 43 deletions(-)
 create mode 100644 builtin-clone.c
 rename git-clone.sh => contrib/examples/git-clone.sh (100%)

diff --git a/Makefile b/Makefile
index 4d23d12..f39b7e8 100644
--- a/Makefile
+++ b/Makefile
@@ -213,7 +213,6 @@ BASIC_LDFLAGS =
 
 SCRIPT_SH = \
 	git-bisect.sh git-checkout.sh \
-	git-clone.sh \
 	git-merge-one-file.sh git-mergetool.sh git-parse-remote.sh \
 	git-pull.sh git-rebase.sh git-rebase--interactive.sh \
 	git-repack.sh git-request-pull.sh \
@@ -327,6 +326,7 @@ BUILTIN_OBJS = \
 	builtin-checkout-index.o \
 	builtin-check-ref-format.o \
 	builtin-clean.o \
+	builtin-clone.o \
 	builtin-commit.o \
 	builtin-commit-tree.o \
 	builtin-count-objects.o \
diff --git a/builtin-clone.c b/builtin-clone.c
new file mode 100644
index 0000000..8a569e0
--- /dev/null
+++ b/builtin-clone.c
@@ -0,0 +1,286 @@
+/*
+ * Builtin "git clone"
+ *
+ * Copyright (c) 2007 Kristian Høgsberg <krh@redhat.com>
+ * Based on git-commit.sh by Junio C Hamano and Linus Torvalds
+ *
+ * Clone a repository into a different directory that does not yet
exist.
+ */
+
+#include "cache.h"
+#include "parse-options.h"
+
+/*
+ * Implementation notes:
+ *  - dropping use-separate-remote and no-separate-remote compatibility
+ *
+ */
+static const char * const builtin_clone_usage[] = {
+	"git-clone [options] [--] <repo> [<dir>]",
+	NULL
+};
+
+static int option_quiet, option_no_checkout, option_bare;
+static int option_local, option_no_hardlinks, option_shared,
option_depth;
+static char *option_template, *option_reference, *option_origin;
+static char *option_upload_pack;
+
+static struct option builtin_clone_options[] = {
+	OPT__QUIET(&option_quiet),
+	OPT_BOOLEAN('n', "no-checkout", &option_no_checkout,
+		    "don't create a checkout"),
+	OPT_BOOLEAN(0, "bare", &option_bare, "create a bare repository"),
+	OPT_BOOLEAN(0, "naked", &option_bare, "create a bare repository"),
+	OPT_BOOLEAN('l', "local", &option_local,
+		    "to clone from a local repository"),
+	OPT_BOOLEAN(0, "no-hardlinks", &option_no_hardlinks,
+		    "don't use local hardlinks, always copy"),
+	OPT_BOOLEAN('s', "shared", &option_shared,
+		    "setup as shared repository"),
+	OPT_STRING(0, "template", &option_template, "path",
+		   "path the template repository"),
+	OPT_STRING(0, "reference", &option_reference, "repo",
+		   "reference repository"),
+	OPT_STRING('o', "origin", &option_origin, "branch",
+		   "use <branch> instead or 'origin' to track upstream"),
+	OPT_STRING('u', "upload-pack", &option_upload_pack, "path",
+		   "path to git-upload-pack on the remote"),
+	OPT_INTEGER(0, "depth", &option_depth,
+		    "create a shallow clone of that depth"),
+
+	OPT_END()
+};
+
+static char *get_repo_path(const char *repo)
+{
+	const char *path;
+	struct stat buf;
+
+	path = mkpath("%s/.git", repo);
+	if (!stat(path, &buf) && S_ISDIR(buf.st_mode))
+		return xstrdup(make_absolute_path(path));
+
+	path = mkpath("%s.git", repo);
+	if (!stat(path, &buf) && S_ISDIR(buf.st_mode))
+		return xstrdup(make_absolute_path(path));
+
+	if (!stat(repo, &buf) && S_ISDIR(buf.st_mode))
+		return xstrdup(make_absolute_path(repo));
+	
+	return NULL;
+}
+
+static char *guess_dir_name(const char *repo)
+{
+	const char *p, *start, *end, *limit;
+	int after_slash_or_colon;
+
+	/* Guess dir name from repository: strip trailing '/',
+	 * strip trailing '[:/]*git', strip leading '.*[/:]'. */
+
+	after_slash_or_colon = 1;
+	limit = repo + strlen(repo);
+	start = repo;
+	end = limit;
+	for (p = repo; p < limit; p++) {
+		if (!prefixcmp(p, ".git")) {
+			if (!after_slash_or_colon)
+				end = p;
+			p += 3;
+		} else if (*p == '/' || *p == ':') {
+			if (end == limit)
+				end = p; 
+			after_slash_or_colon = 1;
+		} else if (after_slash_or_colon) {
+			start = p;
+			end = limit;
+			after_slash_or_colon = 0;
+		}
+	}
+
+	return xstrndup(start, end - start);
+}
+
+static void
+setup_reference(void)
+{
+	struct stat buf;
+	const char *ref_git, *alternates;
+	int fd;
+
+	if (!option_reference)
+		return;
+
+	if (!stat(mkpath("%s/.git/objects", option_reference), &buf) &&
+	    S_ISDIR(buf.st_mode))
+		ref_git = mkpath("%s/.git", option_reference);
+	else if (!stat(mkpath("%s/objects", option_reference), &buf) &&
+		 S_ISDIR(buf.st_mode))
+		ref_git = option_reference;
+	else
+		die("reference repository '%s' is not a local directory.",
+		    option_reference);
+
+	ref_git = make_absolute_path(ref_git);
+	
+	fd = open(git_path("objects/info/alternates"),
+		  O_WRONLY | O_CREAT, 0666);
+	if (fd < 0)
+		die("could not create alternates file\n");
+	alternates = mkpath("%s/objects\n", ref_git);
+	write_or_die(fd, alternates, strlen(alternates));
+	if (close(fd) < 0)
+		die("could not close alternates file\n");
+
+	/* This part is going to be tricky, since it deals with two
+	 * repositories.  Most code in git works on global state
+	 * assumed to be coming from the one active repository, but
+	 * here we want to do for-each-ref in one repository and say
+	 * update-ref in another.  We could just say set_git_dir() and
+	 * then say for-each-ref and store the refs in an array and
+	 * the say set_git_dir() again to switch to the repository
+	 * we're creating and then write out the refs using update-ref. */
+
+	/*
+
+	echo "$ref_git/objects" >"$GIT_DIR/objects/info/alternates"
+	(
+		GIT_DIR="$ref_git" git for-each-ref \
+			--format='%(objectname) %(*objectname)'
+	) |
+	while read a b
+	do
+		test -z "$a" ||
+		git update-ref "refs/reference-tmp/$a" "$a"
+		test -z "$b" ||
+		git update-ref "refs/reference-tmp/$b" "$b"
+	done
+
+	*/
+}
+
+static void
+cleanup_reference(void)
+{
+	/*
+	  test -d "$GIT_DIR/refs/reference-tmp" &&
+		rm -fr "$GIT_DIR/refs/reference-tmp"
+	*/
+}
+
+static void
+clone_local(const char *path)
+{
+	/* local hardlink magic */
+	printf("local, from %s\n", path);
+}
+
+static void
+clone_rsync(const char *repo)
+{
+	printf("rsync, from %s\n", repo);
+	if (option_depth > 0)
+		die("shallow over rsync not supported");
+}
+
+static void
+clone_native(const char *repo)
+{
+		printf("native git, from %s\n", repo);
+}
+
+static void
+clone_curl(const char *repo)
+{
+	printf("curl, from %s\n", repo);
+	if (option_depth > 0)
+		die("shallow over rsync not supported");
+}
+
+int cmd_clone(int argc, const char **argv, const char *prefix)
+{
+	int use_local_hardlinks = 1;
+	int use_separate_remote = 1;
+	struct stat buf;
+	const char *repo, *work_tree, *git_dir;
+	char *path, *dir;
+
+	argc = parse_options(argc, argv, builtin_clone_options,
+			     builtin_clone_usage, 0);
+
+	if (argc == 0)
+		die("You must specify a repository to clone.");
+
+	if (option_no_hardlinks)
+		use_local_hardlinks = 0;
+
+	if (option_bare) {
+		if (option_origin)
+			die("--bare and --origin %s options are incompatible.",
+			    option_origin);
+		option_no_checkout = 1;
+		use_separate_remote = 0;
+	}
+
+	if (!option_origin)
+		option_origin = "origin";
+
+	repo = argv[0];
+	path = get_repo_path(repo);
+
+	if (argc == 2) {
+		dir = xstrdup(argv[1]);
+	} else {
+		dir = guess_dir_name(repo);
+	}
+
+	printf("repo: %s, path: %s\n", repo, path);
+	printf("dir: %s\n", dir);
+
+	if (!stat(dir, &buf))
+		die("destination directory '%s' already exists.", dir);
+
+	if (option_bare)
+		work_tree = NULL;
+	else {
+		work_tree = getenv("GIT_WORK_TREE");
+		if (work_tree && !stat(work_tree, &buf))
+			die("working tree '%s' already exists.", work_tree);
+	}
+
+	if (mkdir(dir, 0755))
+		die("could not create repository dir '%s'.", dir);
+	if (work_tree && mkdir(work_tree, 0755))
+		die("could not create work tree dir '%s'.", work_tree);
+
+	if (option_bare || work_tree)
+		git_dir = xstrdup(dir); 
+	else
+		git_dir = xstrdup(mkpath("%s/.git", dir));
+
+	set_git_dir(make_absolute_path(git_dir));
+	init_db(git_dir, option_template, option_quiet ? INIT_DB_QUIET : 0);
+
+	if (option_bare)
+		git_config_set("core.bare", "true");
+
+	setup_reference();
+
+	if (path != NULL) {
+		clone_local(path);
+	} else if (!prefixcmp(repo, "rsync://")) {
+		clone_rsync(repo);
+	} else if (!prefixcmp(repo, "https://") ||
+		   !prefixcmp(repo, "http://") ||
+		   !prefixcmp(repo, "ftp://")) {
+		clone_curl(repo);
+	} else {
+		clone_native(repo);
+	}
+
+	cleanup_reference();
+
+
+
+	return 0;
+}
diff --git a/builtin-init-db.c b/builtin-init-db.c
index e1393b8..0ddfb46 100644
--- a/builtin-init-db.c
+++ b/builtin-init-db.c
@@ -330,49 +330,11 @@ static void guess_repository_type(const char
*git_dir)
 	return;
 }
 
-static const char init_db_usage[] =
-"git-init [-q | --quiet] [--template=<template-directory>] [--shared]";
-
-/*
- * If you want to, you can share the DB area with any number of
branches.
- * That has advantages: you can save space by sharing all the SHA1
objects.
- * On the other hand, it might just make lookup slower and messier. You
- * be the judge.  The default case is to have one DB per managed
directory.
- */
-int cmd_init_db(int argc, const char **argv, const char *prefix)
+int init_db(const char *git_dir, const char *template_dir, unsigned int
flags)
 {
-	const char *git_dir;
 	const char *sha1_dir;
-	const char *template_dir = NULL;
 	char *path;
-	int len, i, reinit;
-	int quiet = 0;
-
-	for (i = 1; i < argc; i++, argv++) {
-		const char *arg = argv[1];
-		if (!prefixcmp(arg, "--template="))
-			template_dir = arg+11;
-		else if (!strcmp(arg, "--shared"))
-			shared_repository = PERM_GROUP;
-		else if (!prefixcmp(arg, "--shared="))
-			shared_repository = git_config_perm("arg", arg+9);
-		else if (!strcmp(arg, "-q") || !strcmp(arg, "--quiet"))
-		        quiet = 1;
-		else
-			usage(init_db_usage);
-	}
-
-	/*
-	 * GIT_WORK_TREE makes sense only in conjunction with GIT_DIR
-	 * without --bare.  Catch the error early.
-	 */
-	git_dir = getenv(GIT_DIR_ENVIRONMENT);
-	if ((!git_dir || is_bare_repository_cfg == 1)
-	    && getenv(GIT_WORK_TREE_ENVIRONMENT))
-		die("%s (or --work-tree=<directory>) not allowed without "
-		    "specifying %s (or --git-dir=<directory>)",
-		    GIT_WORK_TREE_ENVIRONMENT,
-		    GIT_DIR_ENVIRONMENT);
+	int len, reinit;
 
 	guess_repository_type(git_dir);
 
@@ -388,7 +350,6 @@ int cmd_init_db(int argc, const char **argv, const
char *prefix)
 	/*
 	 * Set up the default .git directory contents
 	 */
-	git_dir = getenv(GIT_DIR_ENVIRONMENT);
 	if (!git_dir)
 		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
 	safe_create_dir(git_dir, 0);
@@ -427,7 +388,7 @@ int cmd_init_db(int argc, const char **argv, const
char *prefix)
 		git_config_set("receive.denyNonFastforwards", "true");
 	}
 
-	if (!quiet)
+	if (!(flags & INIT_DB_QUIET))
 		printf("%s%s Git repository in %s/\n",
 		       reinit ? "Reinitialized existing" : "Initialized empty",
 		       shared_repository ? " shared" : "",
@@ -435,3 +396,48 @@ int cmd_init_db(int argc, const char **argv, const
char *prefix)
 
 	return 0;
 }
+
+static const char init_db_usage[] =
+"git-init [-q | --quiet] [--template=<template-directory>] [--shared]";
+
+/*
+ * If you want to, you can share the DB area with any number of
branches.
+ * That has advantages: you can save space by sharing all the SHA1
objects.
+ * On the other hand, it might just make lookup slower and messier. You
+ * be the judge.  The default case is to have one DB per managed
directory.
+ */
+int cmd_init_db(int argc, const char **argv, const char *prefix)
+{
+	const char *git_dir;
+	const char *template_dir = NULL;
+	unsigned int flags = 0;
+	int i;
+
+	for (i = 1; i < argc; i++, argv++) {
+		const char *arg = argv[1];
+		if (!prefixcmp(arg, "--template="))
+			template_dir = arg+11;
+		else if (!strcmp(arg, "--shared"))
+			shared_repository = PERM_GROUP;
+		else if (!prefixcmp(arg, "--shared="))
+			shared_repository = git_config_perm("arg", arg+9);
+		else if (!strcmp(arg, "-q") || !strcmp(arg, "--quiet"))
+		        flags |= INIT_DB_QUIET;
+		else
+			usage(init_db_usage);
+	}
+
+	/*
+	 * GIT_WORK_TREE makes sense only in conjunction with GIT_DIR
+	 * without --bare.  Catch the error early.
+	 */
+	git_dir = getenv(GIT_DIR_ENVIRONMENT);
+	if ((!git_dir || is_bare_repository_cfg == 1)
+	    && getenv(GIT_WORK_TREE_ENVIRONMENT))
+		die("%s (or --work-tree=<directory>) not allowed without "
+		    "specifying %s (or --git-dir=<directory>)",
+		    GIT_WORK_TREE_ENVIRONMENT,
+		    GIT_DIR_ENVIRONMENT);
+
+	return init_db(git_dir, template_dir, flags);
+}
diff --git a/builtin.h b/builtin.h
index 6f01e96..8e59778 100644
--- a/builtin.h
+++ b/builtin.h
@@ -24,6 +24,7 @@ extern int cmd_check_attr(int argc, const char **argv,
const char *prefix);
 extern int cmd_check_ref_format(int argc, const char **argv, const char
*prefix);
 extern int cmd_cherry(int argc, const char **argv, const char *prefix);
 extern int cmd_cherry_pick(int argc, const char **argv, const char
*prefix);
+extern int cmd_clone(int argc, const char **argv, const char *prefix);
 extern int cmd_clean(int argc, const char **argv, const char *prefix);
 extern int cmd_commit(int argc, const char **argv, const char *prefix);
 extern int cmd_commit_tree(int argc, const char **argv, const char
*prefix);
diff --git a/cache.h b/cache.h
index 510154b..8c5f6af 100644
--- a/cache.h
+++ b/cache.h
@@ -230,6 +230,10 @@ extern const char *prefix_filename(const char
*prefix, int len, const char *path
 extern void verify_filename(const char *prefix, const char *name);
 extern void verify_non_filename(const char *prefix, const char *name);
 
+#define INIT_DB_QUIET 0x0001
+
+extern int init_db(const char *git_dir, const char *template_dir,
unsigned int flags);
+
 #define alloc_nr(x) (((x)+16)*3/2)
 
 /*
diff --git a/git-clone.sh b/contrib/examples/git-clone.sh
similarity index 100%
rename from git-clone.sh
rename to contrib/examples/git-clone.sh
diff --git a/git.c b/git.c
index 9606937..9236833 100644
--- a/git.c
+++ b/git.c
@@ -293,6 +293,7 @@ static void handle_internal_command(int argc, const
char **argv)
 		{ "check-attr", cmd_check_attr, RUN_SETUP | NEED_WORK_TREE },
 		{ "cherry", cmd_cherry, RUN_SETUP },
 		{ "cherry-pick", cmd_cherry_pick, RUN_SETUP | NEED_WORK_TREE },
+		{ "clone", cmd_clone },
 		{ "clean", cmd_clean, RUN_SETUP | NEED_WORK_TREE },
 		{ "commit", cmd_commit, RUN_SETUP | NEED_WORK_TREE },
 		{ "commit-tree", cmd_commit_tree, RUN_SETUP },
-- 
1.5.3.4

Re: Wishlist for a bundle-only transport mode

[Johannes Schindelin]

Hi,

On Wed, 21 Nov 2007, Santi B?jar wrote:

> On Nov 21, 2007 4:59 PM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>
> > On Wed, 21 Nov 2007, Santi B?jar wrote:
> >
> > > 1) git-clone does not accept a bundle file, even if git-fetch does. 
> > > I've made a patch to use git-fetch in git-clone for this.
> >
> > This, along with rewriting git-clone as a very thin wrapper over 
> > git-init, -remote and -fetch, is a really low hanging fruit.
> 
> For the basic/normal mode it can be a very thin wrapper but you have to 
> support --local, --shared, --reference...

That is not all that difficult.

> > > 2) The bundles created with "git bundle" does not record the HEAD, 
> > > they resolve the symbolic name to a branch name.
> >
> > It imitates ls-remote output.
> 
> No, it does not.
> 
> With a newly created project with one commit:
> 
> $ git ls-remote git.git
> b71992713c17c3a29f9566e1b50e8cf402375faf        HEAD
> b71992713c17c3a29f9566e1b50e8cf402375faf        refs/heads/master
> 
> $ git bundle create git.bdl HEAD master
> 
> $ git bundle list-heads git.bdl
> b71992713c17c3a29f9566e1b50e8cf402375faf refs/heads/master
> b71992713c17c3a29f9566e1b50e8cf402375faf refs/heads/master

Ah, I misunderstood.  I thought you were expecting that the first line 
would read

	refs/heads/master	HEAD

Alas, this behaviour stems from dwim_ref() returning "refs/heads/master" 
as real ref.

I am not quite sure how to solve this, though.  Let me see what I can come 
up with.

Ciao,
Dscho
 

Re: Wishlist for a bundle-only transport mode

[Santi Béjar]

On Nov 21, 2007 5:36 PM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Wed, 21 Nov 2007, Santi B?jar wrote:
>
> > On Nov 21, 2007 4:59 PM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> >
> > > On Wed, 21 Nov 2007, Santi B?jar wrote:
> > >
> > > > 1) git-clone does not accept a bundle file, even if git-fetch does.
> > > > I've made a patch to use git-fetch in git-clone for this.
> > >
> > > This, along with rewriting git-clone as a very thin wrapper over
> > > git-init, -remote and -fetch, is a really low hanging fruit.
> >
> > For the basic/normal mode it can be a very thin wrapper but you have to
> > support --local, --shared, --reference...
>
> That is not all that difficult.

I did not say it was difficult, I said it was not a very thin wrapper.

>
> > > > 2) The bundles created with "git bundle" does not record the HEAD,
> > > > they resolve the symbolic name to a branch name.
> > >
> > > It imitates ls-remote output.
> >
> > No, it does not.
> >
> > With a newly created project with one commit:
> >
> > $ git ls-remote git.git
> > b71992713c17c3a29f9566e1b50e8cf402375faf        HEAD
> > b71992713c17c3a29f9566e1b50e8cf402375faf        refs/heads/master
> >
> > $ git bundle create git.bdl HEAD master
> >
> > $ git bundle list-heads git.bdl
> > b71992713c17c3a29f9566e1b50e8cf402375faf refs/heads/master
> > b71992713c17c3a29f9566e1b50e8cf402375faf refs/heads/master
>
> Ah, I misunderstood.

Maybe because of my non-native english :-)

>  I thought you were expecting that the first line
> would read
>
>         refs/heads/master       HEAD

or

ref: refs/heads/master        HEAD

That would be perfect, but it is a different story.

>
> Alas, this behaviour stems from dwim_ref() returning "refs/heads/master"
> as real ref.
>
> I am not quite sure how to solve this, though.  Let me see what I can come
> up with.

Thank you.

Santi

Re: Wishlist for a bundle-only transport mode

[Jakub Narebski]

Johannes Schindelin wrote:
> On Wed, 21 Nov 2007, Jakub Narebski wrote:
>> Santi Bejar wrote:
>> 
>>> 3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:
>>> 
>>> [remote "bundle"]
>>> 	url = /file/to/bundle
>>> 	fetch = "+refs/heads/*:refs/remotes/bundle/*"
>>> 
>>> $ git push bundle
>>> 
>>> would create a bundle in /file/to/bundle with the same branches as a
>>> normal git push, but considering the remote branches as the local
>>> remotes/bundle/*
>> 
>> And how you would differentiate between path meaning bundle, and path 
>> meaning "local" protocol, i.e. git repository on the same filesystem? 
> 
> Maybe because the git repository is specified as an existing directory?  
> The bundle is specified as a (possibly non-existing) file...

That has the disadvantage of pushing to bundle when you make an error
in the lastpart of path to existing repository.

After thinking about it a bit, I think it would be better to use 
bundle:// pseudoprotocol for the URL including config (bundle://file,
bundle:///path/to/bundle) and --bundle option to git push for
commandline.

-- 
Jakub Narebski
Poland

[PATCH] bundle create: keep symbolic refs' names instead of resolving them

[Johannes Schindelin]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2328 bytes --]


When creating a bundle, symbolic refs used to be resolved to the
non-symbolic refs they point to before being written to the list
of contained refs.  I.e. "git bundle create a1.bundle HEAD master"
would show something like

388afe7881b33102fada216dd07806728773c011        refs/heads/master
388afe7881b33102fada216dd07806728773c011        refs/heads/master

Introduce a special handling so that the symbolic refs are listed
with the names passed on the command line.

Noticed by Santi Béjar.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 bundle.c |   13 ++++++++-----
 1 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/bundle.c b/bundle.c
index e4d60cd..5e8242f 100644
--- a/bundle.c
+++ b/bundle.c
@@ -6,6 +6,7 @@
 #include "revision.h"
 #include "list-objects.h"
 #include "run-command.h"
+#include "refs.h"
 
 static const char bundle_signature[] = "# v2 git bundle\n";
 
@@ -231,12 +232,17 @@ int create_bundle(struct bundle_header *header, const char *path,
 	for (i = 0; i < revs.pending.nr; i++) {
 		struct object_array_entry *e = revs.pending.objects + i;
 		unsigned char sha1[20];
-		char *ref;
+		const char *ref;
+		int flag;
 
 		if (e->item->flags & UNINTERESTING)
 			continue;
-		if (dwim_ref(e->name, strlen(e->name), sha1, &ref) != 1)
+		ref = resolve_ref(e->name, sha1, 1, &flag);
+		if (!ref)
 			continue;
+		if (flag & REF_ISSYMREF)
+			ref = e->name;
+
 		/*
 		 * Make sure the refs we wrote out is correct; --max-count and
 		 * other limiting options could have prevented all the tips
@@ -249,7 +255,6 @@ int create_bundle(struct bundle_header *header, const char *path,
 		if (!(e->item->flags & SHOWN) && e->item->type == OBJ_COMMIT) {
 			warning("ref '%s' is excluded by the rev-list options",
 				e->name);
-			free(ref);
 			continue;
 		}
 		/*
@@ -280,7 +285,6 @@ int create_bundle(struct bundle_header *header, const char *path,
 				obj->flags |= SHOWN;
 				add_pending_object(&revs, obj, e->name);
 			}
-			free(ref);
 			continue;
 		}
 
@@ -289,7 +293,6 @@ int create_bundle(struct bundle_header *header, const char *path,
 		write_or_die(bundle_fd, " ", 1);
 		write_or_die(bundle_fd, ref, strlen(ref));
 		write_or_die(bundle_fd, "\n", 1);
-		free(ref);
 	}
 	if (!ref_count)
 		die ("Refusing to create empty bundle.");
-- 
1.5.3.6.1977.g54d30

Re: Wishlist for a bundle-only transport mode

[Johannes Schindelin]

Hi,

On Wed, 21 Nov 2007, Jakub Narebski wrote:

> Johannes Schindelin wrote:
> > On Wed, 21 Nov 2007, Jakub Narebski wrote:
> >> Santi Bejar wrote:
> >> 
> >>> 3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:
> >>> 
> >>> [remote "bundle"]
> >>> 	url = /file/to/bundle
> >>> 	fetch = "+refs/heads/*:refs/remotes/bundle/*"
> >>> 
> >>> $ git push bundle
> >>> 
> >>> would create a bundle in /file/to/bundle with the same branches as a
> >>> normal git push, but considering the remote branches as the local
> >>> remotes/bundle/*
> >> 
> >> And how you would differentiate between path meaning bundle, and path 
> >> meaning "local" protocol, i.e. git repository on the same filesystem? 
> > 
> > Maybe because the git repository is specified as an existing directory?  
> > The bundle is specified as a (possibly non-existing) file...
> 
> That has the disadvantage of pushing to bundle when you make an error
> in the lastpart of path to existing repository.

As I wrote in another reply, I would not allow overwriting an existing 
file.

> After thinking about it a bit, I think it would be better to use 
> bundle:// pseudoprotocol for the URL including config (bundle://file, 
> bundle:///path/to/bundle) and --bundle option to git push for 
> commandline.

I don't like that at all.

Specifying a non-existing file should be good enough.

Ciao,
Dscho

Re: Wishlist for a bundle-only transport mode

[Jakub Narebski]

[Cc: Santi Béjar <sbejar@gmail.com>, git@vger.kernel.org]

Santi Béjar wrote:

> 1) git-clone does not accept a bundle file, even if git-fetch does.
> I've made a patch to use git-fetch in git-clone for this.

[...] 
> 3) I can "git fetch" a bundle but I cannot "git push" a bundle

We would also need "git ls-remote <bundle>" and "git fsck <bundle>"
to demote git-bundle to porcelain status :-)

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Wishlist for a bundle-only transport mode

[Johannes Schindelin]

Hi,

On Wed, 21 Nov 2007, Kristian H?gsberg wrote:

> On Wed, 2007-11-21 at 15:59 +0000, Johannes Schindelin wrote:
> > Hi,
> > 
> > On Wed, 21 Nov 2007, Santi B?jar wrote:
> > 
> > > 1) git-clone does not accept a bundle file, even if git-fetch does. I've 
> > > made a patch to use git-fetch in git-clone for this.
> > 
> > This, along with rewriting git-clone as a very thin wrapper over git-init, 
> > -remote and -fetch, is a really low hanging fruit.
> > 
> > Or maybe go the full nine yards and build it in.  Should be a breeze now, 
> > given parse_options() and run_command().
> 
> I started doing this, which is pretty stupid as I'm pressed to find time 
> to finish up builtin-commit.  Nevertheless, could you elaborate on the 
> plan here?  How would you rewrite it to just use remote and fetch?  I 
> just finished the boilerplate option parsing stuff (patch below) and 
> started reading the core of git-clone.sh, but if most of this can be 
> replaced I'd like to hear about it :)

After handling the options like --reference, and except for special 
handling of a local clone (not via file:// protocol), it should be as easy 
as

	git remote add -f origin $url &&
	case "$no_checkout" in
	t)
		;;
	*)
		git checkout -f -b master remotes/origin/HEAD
		;;
	esac

(Of course I'd use run_command() for this.)

I would avoid at all costs to reimplement the different methods for the 
different protocols.

Ciao,
Dscho

Re: Wishlist for a bundle-only transport mode

[Jakub Narebski]

Johannes Schindelin wrote:
> On Wed, 21 Nov 2007, Jakub Narebski wrote:
>> Johannes Schindelin wrote:
>>> On Wed, 21 Nov 2007, Jakub Narebski wrote:
>>>> Santi Bejar wrote:
>>>> 
>>>>> 3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:
>>>>> 
>>>>> [remote "bundle"]
>>>>> 	url = /file/to/bundle
>>>>> 	fetch = "+refs/heads/*:refs/remotes/bundle/*"
>>>>> 
>>>>> $ git push bundle
>>>>> 
>>>>> would create a bundle in /file/to/bundle with the same branches as a
>>>>> normal git push, but considering the remote branches as the local
>>>>> remotes/bundle/*
>>>> 
>>>> And how you would differentiate between path meaning bundle, and path 
>>>> meaning "local" protocol, i.e. git repository on the same filesystem? 
>>> 
>>> Maybe because the git repository is specified as an existing directory?  
>>> The bundle is specified as a (possibly non-existing) file...
>> 
>> That has the disadvantage of pushing to bundle when you make an error
>> in the lastpart of path to existing repository.
> 
> As I wrote in another reply, I would not allow overwriting an existing 
> file.

> Specifying a non-existing file should be good enough.

What I meant here that if you do "git push /some/path/to/rpeo.git", with
mistake in the last part of path to repository, you would end up with
a bundle, and you would have to really watch what happened to catch
the error.

I'd rather use "git push bundle:///some/path/to/bundle" or 
"git push --bundle bundlename" to catch errors better.

Besides it should be IMHO be possible to overwrite bundle if you are
doing fast-forward push...

-- 
Jakub Narebski
Poland

Re: Wishlist for a bundle-only transport mode

[Johannes Schindelin]

Hi,

On Wed, 21 Nov 2007, Jakub Narebski wrote:

> Johannes Schindelin wrote:
> > On Wed, 21 Nov 2007, Jakub Narebski wrote:
> >
> >> That has the disadvantage of pushing to bundle when you make an error 
> >> in the lastpart of path to existing repository.
> > 
> > As I wrote in another reply, I would not allow overwriting an existing 
> > file.
> 
> > Specifying a non-existing file should be good enough.
> 
> What I meant here that if you do "git push /some/path/to/rpeo.git", with 
> mistake in the last part of path to repository, you would end up with a 
> bundle, and you would have to really watch what happened to catch the 
> error.

I use tab completion all the time, so this would not happen to me.  IMHO 
that is a lesser issue than to introduce a "protocol".

> I'd rather use "git push bundle:///some/path/to/bundle" or "git push 
> --bundle bundlename" to catch errors better.
> 
> Besides it should be IMHO be possible to overwrite bundle if you are 
> doing fast-forward push...

Not as far as I can see.  A push there would see what the bundle has 
already, and put them into the new bundle as _prerequisites_.  So the 
bundle would lose information.

BTW this was my gripe (that I decided not to make public earlier) with 
Santi's proposal to begin with: a push would not have any way to specify 
what the other side has already.  So I think "git push <bundle>" is the 
wrong way of creating a bundle.

Except if we add some cunning strategy not to overwrite, ever, but to 
create <bundle>.<n> with an incrementing <n>.  But that might be too much.

Ciao,
Dscho

Re: Wishlist for a bundle-only transport mode

[Johannes Schindelin]

Hi,

On Wed, 21 Nov 2007, Jakub Narebski wrote:

> We would also need "git ls-remote <bundle>" and "git fsck <bundle>" to 
> demote git-bundle to porcelain status :-)

git ls-remote <bundle> worked, last time I checked (15 seconds ago).  But 
git fsck?  Since when does something like "git fsck 
git://repo.or.cz/git.git" work?

Besides, "git bundle verify <bundle>" works quite well.

Ciao,
Dscho

Re: Wishlist for a bundle-only transport mode

[Jakub Narebski]

On Wed, 21 Nov 2007, Johannes Schindelin wrote:
> On Wed, 21 Nov 2007, Jakub Narebski wrote:
>> Johannes Schindelin wrote:
>>> On Wed, 21 Nov 2007, Jakub Narebski wrote:
>>>
>>>> That has the disadvantage of pushing to bundle when you make an error 
>>>> in the lastpart of path to existing repository.
>>> 
>>> As I wrote in another reply, I would not allow overwriting an existing 
>>> file.
>> 
>>> Specifying a non-existing file should be good enough.
>> 
>> What I meant here that if you do "git push /some/path/to/rpeo.git", with 
>> mistake in the last part of path to repository, you would end up with a 
>> bundle, and you would have to really watch what happened to catch the 
>> error.
> 
> I use tab completion all the time, so this would not happen to me.  IMHO 
> that is a lesser issue than to introduce a "protocol".

When I copy'n'paste pathname I sometimes catch return / linefeed in
the middle, ending with incorect pathname (which would create bundle).
Tab completon is not always solution (tab completion can be slow,
or nonexistent).

[cut]

But I agree that because "git create bundle" needs refspecs _and revlist_,
while "git push" gets only refspecs because it calculates revlist it would
be better to have to use "git bundle create" to create bundle.

Contrary to "git clone <bundle>" which should just work IMHO.

-- 
Jakub Narebski
Poland

Re: Wishlist for a bundle-only transport mode

[Santi Béjar]

On Nov 21, 2007 6:26 PM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Wed, 21 Nov 2007, Jakub Narebski wrote:
>
> > Johannes Schindelin wrote:
> > > On Wed, 21 Nov 2007, Jakub Narebski wrote:
> > >
> > >> That has the disadvantage of pushing to bundle when you make an error
> > >> in the lastpart of path to existing repository.
> > >
> > > As I wrote in another reply, I would not allow overwriting an existing
> > > file.
> >
> > > Specifying a non-existing file should be good enough.
> >
> > What I meant here that if you do "git push /some/path/to/rpeo.git", with
> > mistake in the last part of path to repository, you would end up with a
> > bundle, and you would have to really watch what happened to catch the
> > error.
>
> I use tab completion all the time, so this would not happen to me.  IMHO
> that is a lesser issue than to introduce a "protocol".
>
> > I'd rather use "git push bundle:///some/path/to/bundle" or "git push
> > --bundle bundlename" to catch errors better.

I would vote for the later, and a way to configure this in the config.

> >
> > Besides it should be IMHO be possible to overwrite bundle if you are
> > doing fast-forward push...
>
> Not as far as I can see.  A push there would see what the bundle has
> already, and put them into the new bundle as _prerequisites_.  So the
> bundle would lose information.

I prefer not to overwrite an existing bundle.

>
> BTW this was my gripe (that I decided not to make public earlier) with
> Santi's proposal to begin with: a push would not have any way to specify
> what the other side has already.  So I think "git push <bundle>" is the
> wrong way of creating a bundle.

Sorry but I do not understand this. I think this two lines could be equivalent:

git push --bundle bundle.bdl "refs/heads/master:refs/remotes/bundle/master"
git bundle create bundle.bdl refs/heads/master ^refs/remotes/bundle/master

>
> Except if we add some cunning strategy not to overwrite, ever, but to
> create <bundle>.<n> with an incrementing <n>.  But that might be too much.

That make sense.

Santi

Re: Wishlist for a bundle-only transport mode

[Junio C Hamano]

"Santi Béjar" <sbejar@gmail.com> writes:

> Sorry but I do not understand this. I think this two lines could be equivalent:
>
> git push --bundle bundle.bdl "refs/heads/master:refs/remotes/bundle/master"
> git bundle create bundle.bdl refs/heads/master ^refs/remotes/bundle/master

Interesting.

	$ git push $something ours:theirs

has defined semantics for any value of $something.

 * give $something objects it lacks, so that the object $ours
   become complete in there;

 * set the ref in $theirs to point at the object $ours.

So "git push bundle.bdl refspec" should:

 * First read bundle.bdl, to find out what objects it gives to
   the recipients;

 * Add missing objects to it to make it up to date wrt "ours",
   iow, fetching from the updated bundle would now give "ours";

 * Record the object "ours" as "theirs" ref, iow, listing
   the updated bundle would show "theis" ref pointing at that
   object.

If bundle.bdl does not exist yet, it is like pushing into a
freshly initialized empty repository.

Re: [PATCH] bundle create: keep symbolic refs' names instead of resolving them

[Johannes Schindelin]

Hi,

On Wed, 21 Nov 2007, Johannes Schindelin wrote:

> 
> When creating a bundle, symbolic refs used to be resolved to the
> non-symbolic refs they point to before being written to the list
> of contained refs.  I.e. "git bundle create a1.bundle HEAD master"
> would show something like
> 
> 388afe7881b33102fada216dd07806728773c011        refs/heads/master
> 388afe7881b33102fada216dd07806728773c011        refs/heads/master

Aaargh.

My patch broke "git bundle create x master".  I somehow believed that 
resolve_ref () would turn "master" into "refs/heads/master", which it does 
not do.

So please ignore this patch.

Ciao,
Dscho

[REPLACEMENT PATCH] bundle create: keep symbolic refs' names instead of resolving them

[Johannes Schindelin]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2101 bytes --]


When creating a bundle, symbolic refs used to be resolved to the
non-symbolic refs they point to before being written to the list
of contained refs.  I.e. "git bundle create a1.bundle HEAD master"
would show something like

388afe7881b33102fada216dd07806728773c011        refs/heads/master
388afe7881b33102fada216dd07806728773c011        refs/heads/master

instead of

388afe7881b33102fada216dd07806728773c011        HEAD
388afe7881b33102fada216dd07806728773c011        refs/heads/master

Introduce a special handling so that the symbolic refs are listed
with the names passed on the command line.

Noticed by Santi Béjar.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---

	... and this patch actually does not break the test suite.

 bundle.c |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/bundle.c b/bundle.c
index e4d60cd..9b9b916 100644
--- a/bundle.c
+++ b/bundle.c
@@ -6,6 +6,7 @@
 #include "revision.h"
 #include "list-objects.h"
 #include "run-command.h"
+#include "refs.h"
 
 static const char bundle_signature[] = "# v2 git bundle\n";
 
@@ -232,11 +233,17 @@ int create_bundle(struct bundle_header *header, const char *path,
 		struct object_array_entry *e = revs.pending.objects + i;
 		unsigned char sha1[20];
 		char *ref;
+		const char *display_ref;
+		int flag;
 
 		if (e->item->flags & UNINTERESTING)
 			continue;
 		if (dwim_ref(e->name, strlen(e->name), sha1, &ref) != 1)
 			continue;
+		if (!resolve_ref(e->name, sha1, 1, &flag))
+			flag = 0;
+		display_ref = (flag & REF_ISSYMREF) ? e->name : ref;
+
 		/*
 		 * Make sure the refs we wrote out is correct; --max-count and
 		 * other limiting options could have prevented all the tips
@@ -287,7 +294,7 @@ int create_bundle(struct bundle_header *header, const char *path,
 		ref_count++;
 		write_or_die(bundle_fd, sha1_to_hex(e->item->sha1), 40);
 		write_or_die(bundle_fd, " ", 1);
-		write_or_die(bundle_fd, ref, strlen(ref));
+		write_or_die(bundle_fd, display_ref, strlen(display_ref));
 		write_or_die(bundle_fd, "\n", 1);
 		free(ref);
 	}
-- 
1.5.3.6.1977.g54d30

Re: Wishlist for a bundle-only transport mode

[Jakub Narebski]

On Thu, 22 Nov 2007, Junio C Hamano :
> "Santi Béjar" <sbejar@gmail.com> writes:
> 
>> Sorry but I do not understand this. I think this two lines could be equivalent:
>>
>> git push --bundle bundle.bdl "refs/heads/master:refs/remotes/bundle/master"
>> git bundle create bundle.bdl refs/heads/master ^refs/remotes/bundle/master
> 
> Interesting.
> 
> 	$ git push $something ours:theirs
> 
> has defined semantics for any value of $something.
> 
>  * give $something objects it lacks, so that the object $ours
>    become complete in there;
> 
>  * set the ref in $theirs to point at the object $ours.
> 
> So "git push bundle.bdl refspec" should:
> 
>  * First read bundle.bdl, to find out what objects it gives to
>    the recipients;
> 
>  * Add missing objects to it to make it up to date wrt "ours",
>    iow, fetching from the updated bundle would now give "ours";
> 
>  * Record the object "ours" as "theirs" ref, iow, listing
>    the updated bundle would show "theirs" ref pointing at that
>    object.
> 
> If bundle.bdl does not exist yet, it is like pushing into a
> freshly initialized empty repository.

But for that I think "git bundle" should learn new subcommand:
"git bundle update", which would use refs existing in given
bundle as prerequisites, and either update bundle (create anew
or just concatenate next pack) or create incremental bundle.

Something similar to incremental mode or update command of
archivers...

-- 
Jakub Narebski
Poland

Re: Wishlist for a bundle-only transport mode

[Junio C Hamano]

Jakub Narebski <jnareb@gmail.com> writes:

> On Thu, 22 Nov 2007, Junio C Hamano :
>> "Santi Béjar" <sbejar@gmail.com> writes:
>> 
>>> Sorry but I do not understand this. I think this two lines could be equivalent:
>>>
>>> git push --bundle bundle.bdl "refs/heads/master:refs/remotes/bundle/master"
>>> git bundle create bundle.bdl refs/heads/master ^refs/remotes/bundle/master
>> 
>> Interesting.
>> 
>> 	$ git push $something ours:theirs
>> 
>> has defined semantics for any value of $something.
>> ...
>> If bundle.bdl does not exist yet, it is like pushing into a
>> freshly initialized empty repository.
>
> But for that I think "git bundle" should learn new subcommand:
> "git bundle update", which would use refs existing in given
> bundle as prerequisites, and either update bundle (create anew
> or just concatenate next pack) or create incremental bundle.
>
> Something similar to incremental mode or update command of
> archivers...

I was disagreeing with Santi's "'push --bundle' and 'bundle
create' can be equivalent".  They can't be, as "push" is always
"update" and never "create".  So I do not quite get your "But for
that I think"; I think you are just agreeing with me.

Even if we taught "push" to create (which I doubt would happen
due to its security and administrative implications), it would
not make the two any closer to being equivalent.  For them to
become equivalent, we would need to have "push" unlearn how to
update, which would never happen ;-).

Re: Wishlist for a bundle-only transport mode

[Jakub Narebski]

On Fri, 23 Nov 2007, Junio C Hamano wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
>> On Thu, 22 Nov 2007, Junio C Hamano :
>>> "Santi Béjar" <sbejar@gmail.com> writes:
>>> 
>>>> Sorry but I do not understand this. I think this two lines could be equivalent:
>>>>
>>>> git push --bundle bundle.bdl "refs/heads/master:refs/remotes/bundle/master"
>>>> git bundle create bundle.bdl refs/heads/master ^refs/remotes/bundle/master
>>> 
>>> Interesting.
>>> 
>>> 	$ git push $something ours:theirs
>>> 
>>> has defined semantics for any value of $something.
>>> ...
>>> If bundle.bdl does not exist yet, it is like pushing into a
>>> freshly initialized empty repository.
>>
>> But for that I think "git bundle" should learn new subcommand:
>> "git bundle update", which would use refs existing in given
>> bundle as prerequisites, and either update bundle (create anew
>> or just concatenate next pack) or create incremental bundle.
>>
>> Something similar to incremental mode or update command of
>> archivers...
> 
> I was disagreeing with Santi's "'push --bundle' and 'bundle
> create' can be equivalent".  They can't be, as "push" is always
> "update" and never "create".  So I do not quite get your "But for
> that I think"; I think you are just agreeing with me.

Bad choice of words on my part... and I didn't parse that you were
disagreeing as well as commenting. :-/

> Even if we taught "push" to create (which I doubt would happen
> due to its security and administrative implications), it would
> not make the two any closer to being equivalent.  For them to
> become equivalent, we would need to have "push" unlearn how to
> update, which would never happen ;-).

I also think that pushing to bundle might be not a good idea;
but I think that "git clone <bundle>" should just work.

Nevertheless I think that having "git bundle update" or 
"git bundle create --reference=<old bundle>" (or both) would be
a good thing, as it would free us from remembering or tagging when
last bundle was created.


Could anyone who uses "git bundle" actively raise a voice?

-- 
Jakub Narebski
Poland

Re: Wishlist for a bundle-only transport mode

[Santi Béjar]

On Nov 23, 2007 10:31 AM, Junio C Hamano <gitster@pobox.com> wrote:
>
> I was disagreeing with Santi's "'push --bundle' and 'bundle
> create' can be equivalent".  They can't be, as "push" is always
> "update" and never "create".  So I do not quite get your "But for
> that I think"; I think you are just agreeing with me.
>
> Even if we taught "push" to create (which I doubt would happen
> due to its security and administrative implications), it would
> not make the two any closer to being equivalent.  For them to
> become equivalent, we would need to have "push" unlearn how to
> update, which would never happen ;-).
>

OK. So git push will never understand bundles.

My motivation was to have a similar workflow with bundles as with
pushes, or at least as similar as possible. So another possible
workflow would be with a:

git bundle push [<bundle> [<refspec>]]

that creates a bundle

# Have a project with different branches (master, next, pu)
$ git bundle push bundle.bdl
# would create a complete bundle.bdl with all the local branches (with HEAD)

# You send it to the other person and he does:
$ git clone bundle.bdl project
$ cat .git/config
...
[remote "origin"]
        url = /path/to/bundle.bdl
        fetch = +refs/heads/*:refs/remotes/origin/*

# He makes changes, commits and so on and then he wants to send to back a bundle
$ git bundle push [<remote>]
# This creates an incremental bundle with the local branches (with
HEAD) as specified
# by remote.<remote>.* (possibly with an implicit "push" line
"refs/heads/*:refs/heads/*"
# And considering the remotes/origin/* as the bases for the bundle

# Then I got it and do a:
$ git remote add bundle /path/to/bundle
$ git fetch

# Now maybe I just want to send him the master branch
$ git bundle push origin master:master

# If I want to send him a complete bundle
$ git bundle push

The important part, for me, is to keep remote tracking branches to
know the bases for the next bundle. It does not matter if it is "git
push", "git bundle push", "git push --bundle", "git bundle update",
...

Santi

Re: Wishlist for a bundle-only transport mode

[Johannes Schindelin]

Hi,

On Fri, 23 Nov 2007, Santi B?jar wrote:

> My motivation was to have a similar workflow with bundles as with 
> pushes, or at least as similar as possible. So another possible workflow 
> would be with a:
> 
> git bundle push [<bundle> [<refspec>]]
> 
> that creates a bundle

What about

	git bundle create retort.bundle --all \
		--not $(git ls-remote the-other.bundle | cut -c1-40)

Hmm?

If you need that quite often, it should not be hard at all for you to add 
an option to the "create" named "--haves-from=<bundle>".

But I'd rather script it.

Ciao,
Dscho

Re: Wishlist for a bundle-only transport mode

[Junio C Hamano]

"Santi Béjar" <sbejar@gmail.com> writes:

> OK. So git push will never understand bundles.

Why not?  Because you will not code it?

I am not opposed to the notion of "pushing to a bundle to
update it."  I am just saying that "pushing to create" only for
bundle and not regular repository is very inconsistent.

Re: Wishlist for a bundle-only transport mode

[Santi Béjar]

On Nov 23, 2007 4:06 PM, Junio C Hamano <gitster@pobox.com> wrote:
> "Santi Béjar" <sbejar@gmail.com> writes:
>
> > OK. So git push will never understand bundles.
>
> Why not?

Sorry, I misunderstood you, because pushing to an existing
bundle loses information, and that is also inconsistent with pushing
to regular repositories.

>  Because you will not code it?
>
> I am not opposed to the notion of "pushing to a bundle to
> update it."  I am just saying that "pushing to create" only for
> bundle and not regular repository is very inconsistent.

Santi

Re: Wishlist for a bundle-only transport mode

[Junio C Hamano]

"Santi Béjar" <sbejar@gmail.com> writes:

>> > OK. So git push will never understand bundles.
>>
>> Why not?
>
> Sorry, I misunderstood you, because pushing to an existing
> bundle loses information, and that is also inconsistent with pushing
> to regular repositories.

Maybe I am missing something from the discussion, but what
information loss are you referring to?

Re: Wishlist for a bundle-only transport mode

[Santi Béjar]

On Nov 23, 2007 5:05 PM, Junio C Hamano <gitster@pobox.com> wrote:
> "Santi Béjar" <sbejar@gmail.com> writes:
>
> >> > OK. So git push will never understand bundles.
> >>
> >> Why not?
> >
> > Sorry, I misunderstood you, because pushing to an existing
> > bundle loses information, and that is also inconsistent with pushing
> > to regular repositories.
>
> Maybe I am missing something from the discussion, but what
> information loss are you referring to?
>

Because you create an incremental bundle, so all the objects in the
old bundle will
not be in the new bundle. But it can be considered the natural
behavior of bundles.

But anyway, pushing to an existing bundle is not very convenient (at
leas for me) because it forces you to keep the bundles. I think the
most convenient way would be to have remote tracking branching to
record the bases for future bundles.

For now, I will do an script that behaves like the "git bundle push"
that I described and then I'll report the pro/cons that it has.

Thank you all for the discussion, it was very fruitful.

Santi

P.D.: Am I the only one doing a bundle-only transport mode? If not,
can you comment on the different suggestions that have been said.
Thanks a lot.

Re: Wishlist for a bundle-only transport mode

[Jakub Narebski]

On Fri, 23 Nov 2007, Santi Béjar wrote:
> On Nov 23, 2007 4:06 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> "Santi Béjar" <sbejar@gmail.com> writes:
>>
>>> OK. So git push will never understand bundles.
>>
>> Why not?
> 
> Sorry, I misunderstood you, because pushing to an existing
> bundle loses information, and that is also inconsistent with pushing
> to regular repositories.

Pushing to regular repositories _adds_ information (adds objects
and advances refs). Junio proposed that "git push <bundle>" do the
same, which means getting refs from bundle, and adding objects
to bundle creating new bundle, which contain all the old one has,
and all what was created since then. What information would be lost?

Johannes provided alternate solution which you can use even now,
namely how to create "incremental" bundle, which has all objects
since last bundle was created. But this does not play with the way
push works with ordinary, regular repositories.

P.S. Code speaks louder than words ;-)
-- 
Jakub Narebski
Poland

Re: Wishlist for a bundle-only transport mode

[Junio C Hamano]

"Santi Béjar" <sbejar@gmail.com> writes:

> On Nov 23, 2007 5:05 PM, Junio C Hamano <gitster@pobox.com> wrote:
> ...
>> Maybe I am missing something from the discussion, but what
>> information loss are you referring to?
>
> Because you create an incremental bundle, so all the objects in the
> old bundle will
> not be in the new bundle. But it can be considered the natural
> behavior of bundles.

It might be natural if you are thinking within the limit of "git
bundle create", but it is not natural for "git push" at all.

I think treating a bundle as if it is a bare repository with a
funny representation wouldn't be so wrong, at least from the
user interface point of view.  IOW, I think it is natural for
these:

	$ git push $remote $refspec
        $ git fetch $remote $refspec

to work as "expected" when $remote is actually a local file that
is a bundle, and in fact, "git fetch" should already work that
way.

What's "expected" for a push?  You push from your repository the
objects needed to complete LHS of given $refspec into the
$remote, and then update the refs in the $remote specified by
the $refspec.  There is no deletion of existing objects from the
$remote.  That is what's expected for a push.

So if you want to implement "pushing into a bundle", the
implementation would be:

  * Find the required objects in the existing bundle.  If the
    bundle file does not exist, it might be natural to treat it
    as if you are pushing into an empty but initialized regular
    repository.  If we choose to do this, for a nonexistent
    bundle file, the set of required objects is an empty set.

  * Find the recorded heads in the existing bundle.  Add or
    replace them with the RHS of $refspecs being pushed to come
    up with the new set of heads for the updated bundle.  We
    would want to perform the ordinary "fast-forward" safety and
    reject a push as needed.

  * If there are HEADs in the updated bundle that the pushing
    repository does not have, fetch them (and their required
    objects) first, as it is necessary for the next step.

  * Run this pipeline in the pushing repository to generate a
    packdata stream:

    $ git rev-list --objects <heads in the updated bundle> \
    	--not <required objects in the bundle> |
      git pack-objects --stdout

    This packdata stream will be the payload of the updated
    bundle.

  * The updated bundle will require the same set of objects as
    the bundle before the update.

This is quite different from the way how the other "transports"
are implemented internally to push into usual repositories, but
that is perfectly fine.  What the end user sees will be
consistent if you implement "push into bundle" that way and that
is what matters.

	Note.  I am not saying that we _should_ allow pushing
	into a bundle to update.  I am just saying that if we
	were to implement "git push" into a bundle, that should
        behave as close as other push transports from the end
	uesr's point of view.

If you want different "object losing" semantics, "git bundle
create" to create a new bundle is already there for you.  You
just shouldn't overload that different semantics to "git push",
because that would confuse users without much gain.