[PATCH] Cogito: Support for implicit remote branches in cloned repositories

[PATCH] Cogito: Support for implicit remote branches in cloned repositories

[Josef Weidendorfer]

This allows to specify remote branch names from the
repository this one was cloned from (= remote repository of
"origin" branch), without explicitly adding a branch for them.
Therefore, these remote branches are called implicit.

Use them together with cg-fetch or cg-update.
Example: Your current repository was cloned with

  cg-clone git+ssh://remotehost:/path myrep

If this remote repository contains a branch "foo",
you can say inside of myrep:

  cg-fetch foo

This fetches "git+ssh://remotehost:/path#foo" into
a local branch "foo" (created if not existing).
Similarily, a "cg-update foo" will fetch the remote
branch and merge it into your current local branch.

After fetching from an implicit remote branch, cg-status
will show the corresponding local branch, but still without
an "R", because that is only shown for explicit branches.
The implicit remote branch in the example above can be
made explicit with

  cg-branch-add foo git+ssh://remotehost:/path#foo

Note that cg-update now always tries to fetch from a
remote repository, as every non-explicit branch
name is supposed to be an implicit remote branch.

Signed-off-by: Josef Weidendorfer <Josef.Weidendorfer@gmx.de>

---

This patch is RFC. IMHO, it simplifies the usage of Cogito
quite a lot. One difference of cloning with GIT vs. with
Cogito is that Git always clones all remote branches. This can
be limiting if you want to work with multiple repositories,
but allows you to immediatly work with all the branches.

Cogito instead clones only one remote branch, and requires
you to explicitly add further branches with "cg-branch-add".
This is not necessary with implicit remote branches, and
they should cover the standard use case which works with
branches from the repository you clone from.

The nice thing here is that the patch is really small for
the added functionality. If you accept it, I will provide
updates for the documentation and tutorial.

Josef

 cg-fetch  |   10 +++++++++-
 cg-update |    6 +-----
 2 files changed, 10 insertions(+), 6 deletions(-)

applies-to: 22948869bcf0ec216ed9aa14e1c5ecc22114d66b
769aa2148fc2b583b071354e8eef3dae74e09c14
diff --git a/cg-fetch b/cg-fetch
index 759488a..5a5aeb1 100755
--- a/cg-fetch
+++ b/cg-fetch
@@ -262,7 +262,15 @@ name=${ARGS[0]}
 
 [ "$name" ] || { [ -s "$_git/branches/origin" ] && name=origin; }
 [ "$name" ] || die "where to fetch from?"
-uri=$(cat "$_git/branches/$name" 2>/dev/null) || die "unknown branch: $name"
+uri=$(cat "$_git/branches/$name" 2>/dev/null) || \
+    { if [ -s "$_git/branches/origin" ]; then
+	uri=$(cat "$_git/branches/origin" 2>/dev/null)
+	uri="$(echo "$uri" | cut -d '#' -f 1)#$name"
+	echo Fetching implicit remote branch "$uri".
+      else
+        die "unknown branch: $name"
+      fi
+    }
 
 rembranch=
 if echo "$uri" | grep -q '#'; then
diff --git a/cg-update b/cg-update
index 96035c5..534bf4a 100755
--- a/cg-update
+++ b/cg-update
@@ -39,11 +39,7 @@ name=${ARGS[0]}
 [ "$name" ] || { [ -s $_git/branches/origin ] && name=origin; }
 [ "$name" ] || die "where to update from?"
 
-if [ -s "$_git/branches/$name" ]; then
-	cg-fetch $force $name || exit 1
-else
-	echo "Updating from a local branch."
-fi
+cg-fetch $force $name || exit 1
 echo
 echo "Applying changes..."
 cg-merge $name

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

[Junio C Hamano]

Josef Weidendorfer <Josef.Weidendorfer@gmx.de> writes:

> ... One difference of cloning with GIT vs. with
> Cogito is that Git always clones all remote branches. This can
> be limiting if you want to work with multiple repositories,
> but allows you to immediatly work with all the branches.

Three points, and two footnotes.

. After a git clone, you can still fetch from non-origin
  repository; you can also set up .git/remotes/somewhere-else if
  you pull from the non-origin repository regularly, so it is
  not really "limiting".  It is just being slightly less
  convenient, in that setting up for the origin is done for you
  by clone [*1*] while you have to arrange for non-origin
  repository yourself afterwards.

. The namespace under refs/heads is and always will be an issue.
  It is a local matter and how remote branches are named should
  not dictate what local branch names you can use in your
  repository, but that essentially is what happens after
  git-clone to users who do not rename those branches from the
  initial cloning.

  I once considered to give an option to clone to map the origin
  heads to .git/refs/heads/origin/{master,maint,pu,...}.  In
  hindsight that might have been cleaner.  Instead I just followed
  what Cogito already established, and mapped remote "master" to
  "origin".

. The namespace under refs/tags theoretically also has the same
  issue, but I suspect it would not matter too much in practice.
  The tags people fetches from remote tend to be release-point
  tags (e.g. v2.6.14) whose names implicitly follow an obvious
  (to humans) naming convention; when you name your temporary
  anchor points using lightweight tags, you can easily avoid
  name clashes with those "for other people" tags [*2*].

[Footnotes]

*1* Actually, even the setting up for the origin is done only
halfway -- it only arranges 'git fetch/pull' to fetch from the
master branch, and other branches are not tracked unless you
explicitly arrange them to be.  This is somewhat deliberate; the
refs/ namespace management is a local matter and you do not
necessarily want to keep tracking all the branches from origin.

*2* This becomes somewhat problematic when the tool
automatically follows/fetches tags, and that is why git-core
barebone Porcelainish requires an explicit 'git fetch --tags'.

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

[Josef Weidendorfer]

On Friday 04 November 2005 18:43, Junio C Hamano wrote:
> Josef Weidendorfer <Josef.Weidendorfer@gmx.de> writes:
> 
> > ... One difference of cloning with GIT vs. with
> > Cogito is that Git always clones all remote branches. This can
> > be limiting if you want to work with multiple repositories,
> > but allows you to immediatly work with all the branches.
> 
> Three points, and two footnotes.
> 
> . After a git clone, you can still fetch from non-origin
>   repository; you can also set up .git/remotes/somewhere-else if
>   you pull from the non-origin repository regularly, so it is
>   not really "limiting".

With "limiting" I only meant the fact that all the refs/heads names are
copied over, thus polluting your local namespace. Additionally, it can
be some transfer burden to copy all the objects instead only the objects
needed for one head.

Cogito does not have an issue with pollution of refs/heads namespace
as it only clones one head to the special "origin" head [*1*].
But as I said, the Git way can be more convenient. I think that the
proposed "implicit remote branches" are a way between: make it convenient
for Cogito users to fast access branches from the origin repository, but
pollute the name space only "a little".

[*1*] Still, it would be nice to specify the name of the branch tracking
the origin repository via on option in cg-clone (defaulting to "origin"),
and to also be able to specify the head which is branched off and you
are put on after cloning (defaulting to "master"); but I do not think
there is much use to such flexibility.

>   It is just being slightly less 
>   convenient, in that setting up for the origin is done for you
>   by clone [*1*] while you have to arrange for non-origin
>   repository yourself afterwards.

I do not think there is much difference in editing files under
.git/remotes, or using cg-branch-add.

> . The namespace under refs/heads is and always will be an issue.
>   It is a local matter and how remote branches are named should
>   not dictate what local branch names you can use in your
>   repository, but that essentially is what happens after
>   git-clone to users who do not rename those branches from the
>   initial cloning.

Yes, and that is nice about Cogito.

>   I once considered to give an option to clone to map the origin
>   heads to .git/refs/heads/origin/{master,maint,pu,...}.  In
>   hindsight that might have been cleaner.  Instead I just followed
>   what Cogito already established, and mapped remote "master" to
>   "origin".

That is a nice idea. What is the exact compatiblity problem with Cogito
here? Either a repository is cloned with Git or with Cogito, and
afterwards, both should be able to cope with any naming scheme.

Both Git and Cogito do setup local "master" for local work after cloning.
To still be able to track a remote "master", this is mapped to local
"origin". Am I right here?
What happens if the remote side itself has an "origin" head?

> [Footnotes]
> 
> *1* Actually, even the setting up for the origin is done only
> halfway -- it only arranges 'git fetch/pull' to fetch from the
> master branch, and other branches are not tracked unless you
> explicitly arrange them to be.  This is somewhat deliberate; the
> refs/ namespace management is a local matter and you do not
> necessarily want to keep tracking all the branches from origin.

Yes. I do not see any problem here with explicitly specifying a
head you want to track in "git fetch origin <head>". Still, I would
not call this syntax 'porcelainish'.

But we have Cogito for this.

> *2* This becomes somewhat problematic when the tool
> automatically follows/fetches tags, and that is why git-core
> barebone Porcelainish requires an explicit 'git fetch --tags'.

Ah, ok.

Josef

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

[Petr Baudis]

Dear diary, on Fri, Nov 04, 2005 at 06:43:04PM CET, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
>   I once considered to give an option to clone to map the origin
>   heads to .git/refs/heads/origin/{master,maint,pu,...}.  In
>   hindsight that might have been cleaner.  Instead I just followed
>   what Cogito already established, and mapped remote "master" to
>   "origin".

Well, that obviously works only when you resign on the workflow where
pull means fetch + merge, since your local master would conflict with
the remote master, therefore having nowhere to store the remote master.

> . The namespace under refs/tags theoretically also has the same
>   issue, but I suspect it would not matter too much in practice.
>   The tags people fetches from remote tend to be release-point
>   tags (e.g. v2.6.14) whose names implicitly follow an obvious
>   (to humans) naming convention; when you name your temporary
>   anchor points using lightweight tags, you can easily avoid
>   name clashes with those "for other people" tags [*2*].

I still believe we need the notion of private tags which shouldn't be
cloned.

Hmm. Wait.

All right. git-update-server-info ignores hidden refs, but referencing a
hidden ref works all right (unsurprisingly). So let's just codify that
private tags which shan't be fetched (unless requested explicitly) start
with a dot (/^\./) and we are all set...?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

> I still believe we need the notion of private tags which shouldn't be
> cloned.

I agree with you violently.  I just do not know what is the
right convention to tell which is private and which is not.

Fetching used to be a different matter, because we did not do
'git-fetch --tags' for a reason: you do not have any business
with my tags unless I tell you about them.  But now we tell
others about what tags we have, so...

> All right. git-update-server-info ignores hidden refs, but referencing a
> hidden ref works all right (unsurprisingly). So let's just codify that
> private tags which shan't be fetched (unless requested explicitly) start
> with a dot (/^\./) and we are all set...?

Except that I suspect refs.c::check_ref_format() will barf on it
instead of ignoring it, and obviously you would want it to do
two different things depending on what kind of operation you are
doing.  You for example would want to change git-branch or
git-tag not to refuse creating such "private" tag.  You would
still want git-upload-pack to show it to the other end, for
better common commit computation purposes, especially if the tag
is of lightweight kind, but would want git-clone to ignore
them.  It should be doable but we first need a plan.

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

[Linus Torvalds]

On Fri, 4 Nov 2005, Junio C Hamano wrote:
> 
> I agree with you violently.  I just do not know what is the
> right convention to tell which is private and which is not.

I don't like the dot idea, but it certainly _would_ make sense to have a 
local prefix.

We could even make it a _totally_ different namespace:

	.git/refs/local/heads
	.git/refs/local/tags

which means that even if you call a local head the same thing as a global 
one, both can still exist.

Then, make "origin" always be a local head.

That avoids the issue of the remote repo _also_ having an "origin" branch.

Locally, the "local" heads would always take precedence, with this trivial 
patch making sure of that.. 

If you want to access a global tag or head, you can always do so by using 
the full path ("refs/tags/tagname" is the global one, but "tags/tagname" 
or "tagname" would be the local one if one exists).

		Linus

---
diff --git a/sha1_name.c b/sha1_name.c
index be1755a..5daaa11 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -231,6 +231,8 @@ static int get_sha1_basic(const char *st
 	static const char *prefix[] = {
 		"",
 		"refs",
+		"refs/local/tags",
+		"refs/local/heads",
 		"refs/tags",
 		"refs/heads",
 		NULL

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> I don't like the dot idea, but it certainly _would_ make sense to have a 
> local prefix.
>
> We could even make it a _totally_ different namespace:
>
> 	.git/refs/local/heads
> 	.git/refs/local/tags
>
> which means that even if you call a local head the same thing as a global 
> one, both can still exist.
>
> Then, make "origin" always be a local head.

While that is logically the right thing to do, it is somewhat
confusing to see all those branch heads we obtained from the
remote side stored as our "local" branches -- meaning they are
private to us.

Just for discussion, I'd suggest a less structured approach as
an alternative, and making it just a convention.

 - clone by default [*1*] creates .git/refs/heads/origin/*
   heads; your "master" becomes my "origin/master".
   Interestingly but logically, if your repository was cloned
   from somewhere else, you would have "origin/master", which
   would become my "origin/origin/master".

 - clone creates remotes/origin that knows all branches from the
   origin as copied by the above process, as suggested by Daniel
   in the other thread.

Then we would have a recommended workflow example that uses
origin/* as reference branches after they are set up.  After
becoming familiar with this setup, when people fetch/pull from
more than one repository, they can configure things correctly by
emulating what clone did to the "origin" remote repository for
them.

[Footnote]

*1* I say "by default" here; I think there should be an
operating mode for git clone that acts as a straight copy, not
even the current "HEAD" -> "origin" renaming.  In fact that is
what you do with low-level git-clone-pack.

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

[Petr Baudis]

Dear diary, on Sun, Nov 06, 2005 at 10:11:10AM CET, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> Just for discussion, I'd suggest a less structured approach as
> an alternative, and making it just a convention.

The problem is that at least Cogito defaults to the 'origin' branch when
pulling/merging, so if you'll have 'origin/master' instead of 'origin',
I will have to hack that around. So it's not _just_ a convention but
something reflected in some code as well.

>  - clone by default [*1*] creates .git/refs/heads/origin/*
>    heads; your "master" becomes my "origin/master".
>    Interestingly but logically, if your repository was cloned
>    from somewhere else, you would have "origin/master", which
>    would become my "origin/origin/master".

This loses the pretty nice property that 'origin' refers to the HEAD
branch of the remote repository, no matter how is it called (well, it is
nice as long as the remote repository doesn't _switch_ the HEAD to some
non-superset branch; but repositories you're cloning from shouldn't do
that). So you should either have origin/HEAD locally, or make the
'master' branch more "socially canonical".

> *1* I say "by default" here; I think there should be an
> operating mode for git clone that acts as a straight copy, not
> even the current "HEAD" -> "origin" renaming.  In fact that is
> what you do with low-level git-clone-pack.

What would it be useful for?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

> This loses the pretty nice property that 'origin' refers to the HEAD
> branch of the remote repository,...

I think the code actually honors what your HEAD points at;
I did not go into that detail in the above description.

>> *1* I say "by default" here; I think there should be an
>> operating mode for git clone that acts as a straight copy, not
>> even the current "HEAD" -> "origin" renaming.  In fact that is
>> what you do with low-level git-clone-pack.
>
> What would it be useful for?

Cheap back-up, Linus style.

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

[Josef Weidendorfer]

On Tuesday 08 November 2005 00:21, Petr Baudis wrote:
> The problem is that at least Cogito defaults to the 'origin' branch when
> pulling/merging, so if you'll have 'origin/master' instead of 'origin',
> I will have to hack that around. So it's not _just_ a convention but
> something reflected in some code as well.

The new git-clone simply would not produce an "origin" branch at all, and I do
not think Cogito expects that a repository always has an "origin" branch.

The following fact (Git's default clone setup until now, Cogito's clone setup)

	"origin" tracks remote "master", to be merged into local "master"

has to be stored into some configuration file, and not to be set up implicitly
as it is done currently. Then you take this setup from the configuration, and
there never would be something hardcoded, and Junios suggestion would be a
convention.

Git already stores half of this into remotes/repo: the "Pull" lines specify
which local head tracks which remote head; the same with Cogito's .git/branches.
Missing in both cases is the information that "origin" maps to the local
development head "master". More common name for this information: "origin" is
the upstream for "master".

This information is useful at least for two cases:
(1) for specifying the default action to be done on "git-pull" or "cg-update",
and (2) try to prohibit accidental bogus merges.

The default action currently is either hardcoded (Cogito) or working somehow
by luck (Git: you have to be on "master" for git-pull).
AFAIK, Cogito prohibits bogus merges by enforcing to be either on "master"
or to be in a not-mergeable state (cg-seeking).

Merge relationship is useful to be specified not only for upstream direction,
but also for branches which branched off the current head and should be
merged back (e.g. topical branches). For this, multiple branches to be
merged with "git pull" should be specifyable.

Where to put the information about default merge relation?

Both .git/remotes and .git/branches are candidates, but merge relation
also makes sense for local branches, i.e. git-branch <branch> <origbranch>
should set up to merge from <origbranch> if you are on <branch> by default.
Similarly, it should set up a relation the other way to state that it is
useful to merge from <branch> into <origbranch> (this is for (2) above).

A possiblity would be to put the merge relationship into .git/config:

[master]
merge-canditate: origin
merge-default: origin

Problem here is that head names should not appear in key names,
because key names are case insensitive :-(
A possibility would be for config keys to become case sensitive after a ":"
(of course only for toplevel keys):

[head:master]
merge-default: origin

> >  - clone by default [*1*] creates .git/refs/heads/origin/*
> >    heads; your "master" becomes my "origin/master".
> >    Interestingly but logically, if your repository was cloned
> >    from somewhere else, you would have "origin/master", which
> >    would become my "origin/origin/master".
> 
> This loses the pretty nice property that 'origin' refers to the HEAD
> branch of the remote repository, no matter how is it called (well, it is
> nice as long as the remote repository doesn't _switch_ the HEAD to some
> non-superset branch; but repositories you're cloning from shouldn't do
> that). So you should either have origin/HEAD locally, or make the
> 'master' branch more "socially canonical".

Ah, so git-clone sets up a local development head for the remote HEAD, which
does not always have to be "master", right? This information would be available
by the proposed storing of merge relationship: local "master" wants get merges
from remote HEAD to be updated.

Josef