[PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Pavel Roskin]

Hello!

This patch changes cg-pull so that if the branch is not specified, it
takes origin's .git/HEAD without first trying .git/refs/heads/master.
This removes preferential treatment of the "master" branch, allowing the
upstream to use another name for the default branch.  To get the master
branch, users would have to append "#master" to the URL.

Local URL handling needs to be fixed to handle .git/HEAD properly, since
it's a symlink in the upstream directory.  A new flag "-b" for fetch_*
functions is introduced, meaning "dereference" (like in GNU cp).

To do: the code needs refactoring with better option handling.

Signed-off-by: Pavel Roskin <proski@gnu.org>

diff --git a/cg-pull b/cg-pull
--- a/cg-pull
+++ b/cg-pull
@@ -67,6 +67,8 @@ pull_progress()
 
 fetch_rsync()
 {
+	[ "$1" = "-b" ] && shift
+
 	redir=
 	if [ "$1" = "-i" ]; then # ignore-errors
 		redir="2>/dev/null"
@@ -108,6 +110,7 @@ pull_rsync()
 
 fetch_http()
 {
+	[ "$1" = "-b" ] && shift
 	[ "$1" = "-i" ] && shift
 	[ "$1" = "-s" ] && shift
 
@@ -158,6 +161,7 @@ pull_http()
 
 fetch_ssh()
 {
+	[ "$1" = "-b" ] && shift
 	[ "$1" = "-i" ] && shift
 	[ "$1" = "-s" ] && shift
 
@@ -205,6 +209,11 @@ fetch_local()
 	[ "$1" = "-s" ] && shift
 
 	cp_flags_l="-vdpR"
+	if [ "$1" = "-b" ]; then
+		cp_flags_l="-vb" # Dereference symlinks
+		shift
+	fi
+
 	if [ "$1" = "-u" ]; then
 		cp_flags_l="$cp_flags_l -fu$can_hardlink"
 		suggest_hardlink=
@@ -255,7 +264,7 @@ name=${ARGS[0]}
 [ "$name" ] || die "where to pull from?"
 uri=$(cat "$_git/branches/$name" 2>/dev/null) || die "unknown branch: $name"
 
-rembranch=master
+rembranch=
 if echo "$uri" | grep -q '#'; then
 	rembranch=$(echo $uri | cut -d '#' -f 2)
 	uri=$(echo $uri | cut -d '#' -f 1)
@@ -308,13 +317,13 @@ orig_head=
 
 
 mkdir -p $_git/refs/heads
-rsyncerr=
-$fetch -i "$uri/refs/heads/$rembranch" "$_git/refs/heads/.$name-pulling" || rsyncerr=1
-if [ "$rsyncerr" ] && [ "$rembranch" = "master" ]; then
-	rsyncerr=
-	$fetch -s "$uri/HEAD" "$_git/refs/heads/.$name-pulling" || rsyncerr=1
+if [ "$rembranch" ]; then
+	$fetch -i "$uri/refs/heads/$rembranch" "$_git/refs/heads/.$name-pulling" ||
+		die "unable to get the head pointer of branch $rembranch"
+else
+	$fetch -b -s "$uri/HEAD" "$_git/refs/heads/.$name-pulling" ||
+		die "unable to get the HEAD branch"
 fi
-[ "$rsyncerr" ] && die "unable to get the head pointer of branch $rembranch"
 
 new_head=$(cat "$_git/refs/heads/.$name-pulling")
 if ! [ "$symlinked" ]; then


-- 
Regards,
Pavel Roskin

Re: [PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Petr Baudis]

  Hello,

Dear diary, on Tue, Aug 23, 2005 at 11:33:16PM CEST, I got a letter
where Pavel Roskin <proski@gnu.org> said that...
> Local URL handling needs to be fixed to handle .git/HEAD properly, since
> it's a symlink in the upstream directory.  A new flag "-b" for fetch_*
> functions is introduced, meaning "dereference" (like in GNU cp).
> 
> @@ -205,6 +209,11 @@ fetch_local()
>  	[ "$1" = "-s" ] && shift
>  
>  	cp_flags_l="-vdpR"
> +	if [ "$1" = "-b" ]; then
> +		cp_flags_l="-vb" # Dereference symlinks
> +		shift
> +	fi
> +
>  	if [ "$1" = "-u" ]; then
>  		cp_flags_l="$cp_flags_l -fu$can_hardlink"
>  		suggest_hardlink=

  can you still remember why did you introduce this? In GNU cp
documentation, I can see just

       -b, --backup
              Make backups of files that are about to be overwritten or removed.

which doesn't make sense to me - -L dereferences symlinks.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: [PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Pavel Roskin]

Hi, Petr!

On Thu, 2005-11-10 at 20:24 +0100, Petr Baudis wrote:
>   can you still remember why did you introduce this? In GNU cp
> documentation, I can see just
> 
>        -b, --backup
>               Make backups of files that are about to be overwritten or removed.
> 
> which doesn't make sense to me - -L dereferences symlinks.

You are right, it must be my error.  Anyway, it was so long ago that I
would need to review and retest it.

While at that, let's stop using symlinks.  git doesn't use symlinks on
Cygwin.  I think git should use that code on all OSes, since the
benefits of using symlinks are minimal (I think the only benefits are
their atomicity and resolving the reference in the kernel rather than in
userspace).  Having more uniform code for all platforms would simplify
development and testing.  It could also reduce requirements for the
transport protocols.  Finally, symlinks could be still used by the users
(if they know what they are doing) - git and cogito would simply become
symlink agnostic.

When files are copied around, symlinks are pain to deal with.  They
require special handling to be preserved both for remote operation and
dereferenced for local operation (that's what my patch was intended to
do).  I'm not even considering what would happen when cloning from Linux
to Windows or vice versa.

Sure, it can wait until git 1.0, but it would be great to keep this goal
in mind.

Disclaimer - I'm not reading the git mailing list, so if it was
discussed, I'm sorry, I don't intend to restart that discussion - just
give me the pointer and I'll read it.

-- 
Regards,
Pavel Roskin

Re: [PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Josef Weidendorfer]

On Tuesday 23 August 2005 23:33, Pavel Roskin wrote:
> Hello!
> 
> This patch changes cg-pull so that if the branch is not specified, it
> takes origin's .git/HEAD without first trying .git/refs/heads/master.
> This removes preferential treatment of the "master" branch, allowing the
> upstream to use another name for the default branch.  To get the master
> branch, users would have to append "#master" to the URL.

I think this is wrong.

Why would anybody want to fetch the current upstream HEAD at cg-pull
time? If you do this, and you have no control on the upstream
repository, "origin" will jump randomly around after cg-pull, depending
on the current HEAD of the origin repository.

Besides, this is a incompatible change in cg-pull behavior: I am sure that
a lot of people did a cg-clone in the past without any specific remote branch,
and expect that cg-pull will update their origin to the remote "master".
Now, if they update to a new Cogito with this patch, the next cg-pull
sometimes will rebase their origin to another random branch, depending
on remote HEAD ?!

Or am I missing something here?

Josef

Re: [PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Petr Baudis]

Dear diary, on Fri, Nov 11, 2005 at 12:26:18AM CET, I got a letter
where Josef Weidendorfer <Josef.Weidendorfer@gmx.de> said that...
> On Tuesday 23 August 2005 23:33, Pavel Roskin wrote:
> > Hello!
> > 
> > This patch changes cg-pull so that if the branch is not specified, it
> > takes origin's .git/HEAD without first trying .git/refs/heads/master.
> > This removes preferential treatment of the "master" branch, allowing the
> > upstream to use another name for the default branch.  To get the master
> > branch, users would have to append "#master" to the URL.
> 
> I think this is wrong.
> 
> Why would anybody want to fetch the current upstream HEAD at cg-pull
> time? If you do this, and you have no control on the upstream
> repository, "origin" will jump randomly around after cg-pull, depending
> on the current HEAD of the origin repository.

In public repositories, you usually set the HEAD once and it stays
there, or when you switch it, you are really careful about it.

> Besides, this is a incompatible change in cg-pull behavior: I am sure that
> a lot of people did a cg-clone in the past without any specific remote branch,
> and expect that cg-pull will update their origin to the remote "master".
> Now, if they update to a new Cogito with this patch, the next cg-pull
> sometimes will rebase their origin to another random branch, depending
> on remote HEAD ?!

See above. The remote HEAD is supposed to be generally stable for public
repositories, pointing to the _default_ head for public consumption.
If you want something specific, specify it by the #branchname.

> Or am I missing something here?

That this is from 23 August and this change was in Cogito for several
major releases and noone ever complained. ;-)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: [PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Josef Weidendorfer]

On Friday 11 November 2005 00:40, Petr Baudis wrote:
> Dear diary, on Fri, Nov 11, 2005 at 12:26:18AM CET, I got a letter
> where Josef Weidendorfer <Josef.Weidendorfer@gmx.de> said that...
> > On Tuesday 23 August 2005 23:33, Pavel Roskin wrote:
> > > Hello!
> > > 
> > > This patch changes cg-pull so that if the branch is not specified, it
> > > takes origin's .git/HEAD without first trying .git/refs/heads/master.
> > > This removes preferential treatment of the "master" branch, allowing the
> > > upstream to use another name for the default branch.  To get the master
> > > branch, users would have to append "#master" to the URL.
> ...
> > Or am I missing something here?
> 
> That this is from 23 August and this change was in Cogito for several
> major releases and noone ever complained. ;-)

You catched me ;-)

Still I think my reasoning holds: cg-clone is also used locally, thus
cloning a repository on which you probably are working on, too.
If I want to branch off from master, I can use cg-clone to start a new
development branch. I suppose this is the suggested Cogito way, from
the README; and it makes Cogitos usage so simple, even with multiple
development branches: by using a cloned repository for each branch.

Of course, a Git user would use "git branch ...", and switch between
branches inside of one repository.

Josef

Re: [PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Petr Baudis]

Dear diary, on Fri, Nov 11, 2005 at 12:56:54AM CET, I got a letter
where Josef Weidendorfer <Josef.Weidendorfer@gmx.de> said that...
> Still I think my reasoning holds: cg-clone is also used locally, thus
> cloning a repository on which you probably are working on, too.

Yes, that's a valid point, but I still think for the usual case
defaulting to HEAD is the right thing to do. But it wasn't described
in the documentation and that's indeed a bug - fixed now.

> If I want to branch off from master, I can use cg-clone to start a new
> development branch. I suppose this is the suggested Cogito way, from
> the README; and it makes Cogitos usage so simple, even with multiple
> development branches: by using a cloned repository for each branch.

Yes, this is the recommended workflow. Well, if you plan to switch
branches randomly in your "central private repository", better explicitly
specify #master in the offsprings. ;-)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: [PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Pavel Roskin]

On Fri, 2005-11-11 at 00:40 +0100, Petr Baudis wrote: 
> Dear diary, on Fri, Nov 11, 2005 at 12:26:18AM CET, I got a letter
> where Josef Weidendorfer <Josef.Weidendorfer@gmx.de> said that...
> > Why would anybody want to fetch the current upstream HEAD at cg-pull
> > time? If you do this, and you have no control on the upstream
> > repository, "origin" will jump randomly around after cg-pull, depending
> > on the current HEAD of the origin repository.
> 
> In public repositories, you usually set the HEAD once and it stays
> there, or when you switch it, you are really careful about it.

Correct.  But if it's a concern, I think we could make some
improvements.  Following approaches could be tried (starting from top,
using following steps as a fallback):

1) Use recorded branch, i.e. the branch name that was used in cg-clone.

2) Check a file specifying the default branch for cloning (let's call it
e.g. .git/default_export).  Record branch name.

3) If cg-switch is implemented, another file would be likely used to
specify the default branch for local operations
(e.g. .git/default_branch).  Use it.  Record branch name.

4) If .git/HEAD can be recognized as link (we may need to eliminate
symlinks for that to work over remote transport), use the branch it
points to.  Record branch name.

5) If .git/HEAD cannot be recognized as link, use it as the branch to
clone.  Don't record the branch name.  (I'm not sure if we really want
that - it's good to know what branch we are using).

6) Use "master" branch.  Record branch name.

-- 
Regards,
Pavel Roskin

Re: [PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Josef Weidendorfer]

On Friday 11 November 2005 01:14, Pavel Roskin wrote:
> On Fri, 2005-11-11 at 00:40 +0100, Petr Baudis wrote: 
> > Dear diary, on Fri, Nov 11, 2005 at 12:26:18AM CET, I got a letter
> > where Josef Weidendorfer <Josef.Weidendorfer@gmx.de> said that...
> > > Why would anybody want to fetch the current upstream HEAD at cg-pull
> > > time?
> ...
> Correct.  But if it's a concern, I think we could make some
> improvements.  Following approaches could be tried (starting from top,
> using following steps as a fallback):
> 
> 1) Use recorded branch, i.e. the branch name that was used in cg-clone.

This is the "#..." part in .git/branches/origin, and it is already used.

I have nothing against using HEAD for cloning, but the branch name should
be recorded, such that cg-pull always fetches from the same branch.

I think 1) is enough if we add the detected branch name in the #... part
in .git/branches/origin at cg-clone time.

And "detectable" covers the local case, which was my concern here [*1*]

Josef

[*1*] after being told about the "stable HEAD" property of public repositories,
which is correct. I am not even sure that a HEAD makes sense
in public repositories: in my opinion HEAD has something to do with checked out
files, and a public repository usually does not have these. But perhaps
that's only me...

Re: [PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Pavel Roskin]

On Fri, 2005-11-11 at 02:13 +0100, Josef Weidendorfer wrote:
> > Correct.  But if it's a concern, I think we could make some
> > improvements.  Following approaches could be tried (starting from top,
> > using following steps as a fallback):
> > 
> > 1) Use recorded branch, i.e. the branch name that was used in cg-clone.
> 
> This is the "#..." part in .git/branches/origin, and it is already used.

I'm not sure if we understand each other, but "recorded" refers to
automatically determined branch name.  If cg-clone can determine the
branch name, that name should be saved and used for updates.

Whether the branch name is saved in .git/branches/ using hash notation
(old style) or in .git/remotes/ (new style) is irrelevant.

If cg-clone cannot determine the branch name, we may either support it,
or revert to "master".  I'm ambivalent about that.  Probably reverting
to "master" is better to simplify the code.  On the other hand, I can
imagine public repositories without the "master" branch.

> I have nothing against using HEAD for cloning, but the branch name should
> be recorded, such that cg-pull always fetches from the same branch.

To use HEAD, we'll need to port git from Cygwin, since readlink doesn't
work over http.

> I think 1) is enough if we add the detected branch name in the #... part
> in .git/branches/origin at cg-clone time.

I agree that the branch name should be saved explicitly using already
supported notation (if that's what you mean).

> And "detectable" covers the local case, which was my concern here [*1*]
> 
> Josef
> 
> [*1*] after being told about the "stable HEAD" property of public repositories,
> which is correct. I am not even sure that a HEAD makes sense
> in public repositories: in my opinion HEAD has something to do with checked out
> files, and a public repository usually does not have these. But perhaps
> that's only me...

That's why I mentioned the idea of having a separate file to indicate
the default branch for export.

-- 
Regards,
Pavel Roskin

Re: [PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Josef Weidendorfer]

On Friday 11 November 2005 05:53, Pavel Roskin wrote:
> I'm not sure if we understand each other, but "recorded" refers to
> automatically determined branch name.  If cg-clone can determine the
> branch name, that name should be saved and used for updates.
> 
> Whether the branch name is saved in .git/branches/ using hash notation
> (old style) or in .git/remotes/ (new style) is irrelevant.

Yes.
But for the conrete implementation it is relevant.

AFAIK, there is not really a "new" and "old" style, but more the
git way (attributes for remote repositories in .git/remotes) and the
Cogito way (attributes for a remote branch).

"Recording" talks about storing the name of the remote branch that
maps to the local "origin" branch, so I would vote for storing this
in .git/branches/origin.

For cg-clone, this is no problem because cg-clone writes this file itself.
Another thing is if you add later on a remote branch with cg-branch-add
without specifying a concrete remote branch name. Do we want the
record the branch name at the first cg-fetch for the future?

> That's why I mentioned the idea of having a separate file to indicate
> the default branch for export.

Agree.

Josef

Re: [PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Pavel Roskin]

On Fri, 2005-11-11 at 15:22 +0100, Josef Weidendorfer wrote:
> On Friday 11 November 2005 05:53, Pavel Roskin wrote:
> > I'm not sure if we understand each other, but "recorded" refers to
> > automatically determined branch name.  If cg-clone can determine the
> > branch name, that name should be saved and used for updates.
> > 
> > Whether the branch name is saved in .git/branches/ using hash notation
> > (old style) or in .git/remotes/ (new style) is irrelevant.
> 
> Yes.
> But for the conrete implementation it is relevant.
> 
> AFAIK, there is not really a "new" and "old" style, but more the
> git way (attributes for remote repositories in .git/remotes) and the
> Cogito way (attributes for a remote branch).

Switching to the "git way" is long overdue, since hashes can be used in
the actual path.

> "Recording" talks about storing the name of the remote branch that
> maps to the local "origin" branch, so I would vote for storing this
> in .git/branches/origin.

Yes, cg-clone should do its best to determine the branch name and to
save it for future use by cg-fetch.

> For cg-clone, this is no problem because cg-clone writes this file itself.
> Another thing is if you add later on a remote branch with cg-branch-add
> without specifying a concrete remote branch name. Do we want the
> record the branch name at the first cg-fetch for the future?

Good that you noticed that.

cg-fetch could do that, but maybe cg-branch-add would be an even better
place for that?  I mean, cg-branch-add could actually check the
repository and prompt the user which branch to use if there is no
obvious default.  I think, there should be a way to run cg-branch-add
without having it connect the repository or prompt the user, but the
default should be user-friendly.

-- 
Regards,
Pavel Roskin

Re: [PATCH] cg-pull to stop treating "master" specially, fix fetch_local for .git/HEAD

[Josef Weidendorfer]

On Friday 11 November 2005 16:26, Pavel Roskin wrote:
> > AFAIK, there is not really a "new" and "old" style, but more the
> > git way (attributes for remote repositories in .git/remotes) and the
> > Cogito way (attributes for a remote branch).
> 
> Switching to the "git way" is long overdue, since hashes can be used in
> the actual path.

Currently, Git does not store mappings of remote to local branches in
.git/remotes/ per se. "Pull" lines allow you to specify default actions
for git-fetch/git-pull, but even this is quite limited:
"git-fetch" defaults to fetching all heads mentioned, and "git-pull"
defaults to merging the first head on the first "Pull" line.

Because the git way is per-remote-repository, you always have to specify
the remote repository shortcut in git-fetch/git-pull/git-push, as there
is no implicit remote repository. With per-branch, it is far simpler as
you always have a current active branch which is HEAD. By using
.git/remotes, you more or less get a shortcut name for a remote repository.
Of course, Cogito could use this shortcut in the .git/branches/, too.

Another thing: Per branch configuration is not only about branches which
have a mapping to a remote file. I would *love* to be able to specify
merge candidates and default merge branches for every local branch.
Such configuration of local branches can also reside in .git/branches.

But you are right, hashes can be part of an path, so Cogito should use
multiple lines in .git/branches/ like Git does in .git/remotes.

So what are possible configuration items for a branch?
Branches can be of 2 kinds: either a branch should track a remote branch,
or it is used for local development.

A branch which is tracking a remote one does need
* the remote repository URL, or path (if local)
* the branch name to track from the remote repository

A branch used for development can have:
* set of local branch names which are candidates for merging
* set of local branch names for default merge action
* a default remote branch to use for pushing

> > For cg-clone, this is no problem because cg-clone writes this file itself.
> > Another thing is if you add later on a remote branch with cg-branch-add
> > without specifying a concrete remote branch name. Do we want the
> > record the branch name at the first cg-fetch for the future?
> 
> Good that you noticed that.
> 
> cg-fetch could do that, but maybe cg-branch-add would be an even better
> place for that?  I mean, cg-branch-add could actually check the
> repository and prompt the user which branch to use if there is no
> obvious default.  I think, there should be a way to run cg-branch-add
> without having it connect the repository or prompt the user, but the
> default should be user-friendly.

Yes.

Josef