New way of tracking remote branches -- question

New way of tracking remote branches -- question

[Luben Tuikov]

I can see that the remote heads are where they are supposed to be
but no local tracking heads are created (by default).  I had
to do this manually.

Old behavior was that git did that for you automatically.
So I suppose this is another newbie protection.

What is the now the new, accepted way of tracking locally
remote branches? (other than creating them manually in
refs/heads?)

Thanks,
   Luben

Re: New way of tracking remote branches -- question

[Junio C Hamano]

Luben Tuikov <ltuikov@yahoo.com> writes:

> I can see that the remote heads are where they are supposed to be
> but no local tracking heads are created (by default).  I had
> to do this manually.
>
> Old behavior was that git did that for you automatically.
> So I suppose this is another newbie protection.

A very fuzzily stated question which is hard to answer, but I do
not think it is another newbie protection, if it apparently is
actively hurting you.  Also the documentation may need to be
updated to teach you enough about how to achieve what you want.

You can see where remote heads are by doing what?  ls-remote?
"Old behaviour" for what configuration?

A fresh clone made with a recent version sets things up to track
all remote branches from the repository you cloned from under
remotes/origin/, and it even tracks new ones as they are added
at the remote, so you probably are doing something different
from the default configuration that has:

    remotes.origin.fetch = +refs/heads/*:refs/remotes/origin/*

Re: New way of tracking remote branches -- question

[Junio C Hamano]

Junio C Hamano <junkio@cox.net> writes:

> Luben Tuikov <ltuikov@yahoo.com> writes:
>
>> I can see that the remote heads are where they are supposed to be
>> but no local tracking heads are created (by default).  I had
>> to do this manually.
>>
>> Old behavior was that git did that for you automatically.
>> So I suppose this is another newbie protection.
>
> A very fuzzily stated question which is hard to answer, but I do
> not think it is another newbie protection, if it apparently is
> actively hurting you.  Also the documentation may need to be
> updated to teach you enough about how to achieve what you want.

Can you state the problem you observed about the recent git in a
way that is easier to debug?

For example, you could state:

	With older git (I verified that v1.3.0 still works like
	this), I used to be able to just say:

		$ git fetch

	(this is the exact command line -- I am not giving a URL
	nor even "origin" after "git fetch").  When the upstream
	created a new branch 'blah', the above command created a
	new local branch 'blah' automatically for me.  With the
	tip of 'master' (e27e609), this does not happen anymore.

	My configuration is that I have .git/remotes/origin file
	whose contents is ....  I do not have any remote.*.url,
	remote.*.fetch, nor branch.*.remote configuration variables.

to be more helpful.

I am not dismissing your message as whining.  You probably have
hit a regression while we adopted the BCP to encourage separate
remote layout, and I would like to understand the issue.

Re: New way of tracking remote branches -- question

[Luben Tuikov]

--- Junio C Hamano <junkio@cox.net> wrote:
> Junio C Hamano <junkio@cox.net> writes:
> 
> > Luben Tuikov <ltuikov@yahoo.com> writes:
> >
> >> I can see that the remote heads are where they are supposed to be
> >> but no local tracking heads are created (by default).  I had
> >> to do this manually.
> >>
> >> Old behavior was that git did that for you automatically.
> >> So I suppose this is another newbie protection.
> >
> > A very fuzzily stated question which is hard to answer, but I do
> > not think it is another newbie protection, if it apparently is
> > actively hurting you.  Also the documentation may need to be
> > updated to teach you enough about how to achieve what you want.
> 
> Can you state the problem you observed about the recent git in a
> way that is easier to debug?
> 
> For example, you could state:
> 
> 	With older git (I verified that v1.3.0 still works like
> 	this), I used to be able to just say:
> 
> 		$ git fetch
> 
> 	(this is the exact command line -- I am not giving a URL
> 	nor even "origin" after "git fetch").  When the upstream
> 	created a new branch 'blah', the above command created a
> 	new local branch 'blah' automatically for me.  With the
> 	tip of 'master' (e27e609), this does not happen anymore.
> 
> 	My configuration is that I have .git/remotes/origin file
> 	whose contents is ....  I do not have any remote.*.url,
> 	remote.*.fetch, nor branch.*.remote configuration variables.
> 
> to be more helpful.
> 
> I am not dismissing your message as whining.  You probably have
> hit a regression while we adopted the BCP to encourage separate
> remote layout, and I would like to understand the issue.

And I'm not whining.  It just that when I've done something 1000
times and all of a sudden I do the same thing and didn't see the
expected behaviour, I posted.

"git-pull" didn't "create" the branches in the place I was
expecting.  I.e. while they are in .git/refs/remotes/origin/
they are not in .git/refs/heads.

Then I manually created the heads in .git/refs/heads
and manually added that to .git/config, i.e. the [branch]
part.

I was hoping I wouldn't need to do that at all, as old
git-pull exposed remote branches, or I was expecting to
at least find a git command to do this 2nd additional
manual step for me.

    Luben

Re: New way of tracking remote branches -- question

[Luben Tuikov]

--- Junio C Hamano <junkio@cox.net> wrote:
> Luben Tuikov <ltuikov@yahoo.com> writes:
> 
> > I can see that the remote heads are where they are supposed to be
> > but no local tracking heads are created (by default).  I had
> > to do this manually.
> >
> > Old behavior was that git did that for you automatically.
> > So I suppose this is another newbie protection.
> 
> A very fuzzily stated question which is hard to answer, but I do
> not think it is another newbie protection, if it apparently is
> actively hurting you.  Also the documentation may need to be
> updated to teach you enough about how to achieve what you want.
> 
> You can see where remote heads are by doing what?  ls-remote?
> "Old behaviour" for what configuration?
> 
> A fresh clone made with a recent version sets things up to track
> all remote branches from the repository you cloned from under
> remotes/origin/, and it even tracks new ones as they are added
> at the remote, so you probably are doing something different
> from the default configuration that has:
> 
>     remotes.origin.fetch = +refs/heads/*:refs/remotes/origin/*

That's exactly what I have, but "git branch" shows only "master".
The other branches are indeed in refs/remotes/origin/ but I want
them in refs/heads/ so I had to do that manually by creating
the head and add this into .git/config.

Old behavior was more _convenient_.

    Luben

Re: New way of tracking remote branches -- question

[Junio C Hamano]

Luben Tuikov <ltuikov@yahoo.com> writes:

> I can see that the remote heads are where they are supposed to be
> but no local tracking heads are created (by default).  I had
> to do this manually.

Ah, after firing off a few messages, I think I now guessed
what you are talking about correctly.

You are talking about the separate remote layout "git clone"
creates, and talking about the fact that there is nothing in
refs/heads/ except master while refs/remotes/origin/ mirrors
what the remote side has faithfully, aren't you?

If that is the case, I can explain.  It is a good topic to talk
about.

Re: New way of tracking remote branches -- question

[Junio C Hamano]

Junio C Hamano <junkio@cox.net> writes:

> Luben Tuikov <ltuikov@yahoo.com> writes:
>
>> I can see that the remote heads are where they are supposed to be
>> but no local tracking heads are created (by default).  I had
>> to do this manually.
>
> Ah, after firing off a few messages, I think I now guessed
> what you are talking about correctly.
>
> You are talking about the separate remote layout "git clone"
> creates, and talking about the fact that there is nothing in
> refs/heads/ except master while refs/remotes/origin/ mirrors
> what the remote side has faithfully, aren't you?
>
> If that is the case, I can explain.  It is a good topic to talk
> about.

Disclaimer.  I am not very good at defending what I do not fully
agree with, and the separate remote is one of them, but because
it has become the default layout, let me try.

When cloning from a repository with a single branch (like
Linus's linux-2.6), the traditional layout created
heads/{origin,master}.  Origin is a tracking branch that you
were not supposed to build on top of (or reset to move around,
for that matter), and master is where you do your work (or you
do your work elsewhere and use master to integrate them).

The separate remote layout creates remotes/origin/{HEAD,master}
that you can still refer to with "origin" (because sha1_name
knows about "refs/remotes/%s/HEAD") and is a tracking branch.
It also creates heads/master which is where you do your work (or
integration).

So from the high level view, the usage has not changed for the
simplest case with the new layout.

When the other side have more than one branch (say, 'master' and
'next'), the story is different.  The traditional layout created
heads/next in addition.  This _is_ a tracking branch that you
were not supposed to build on top of or reset.

The problem is that "git branch" listing does not tell you the
differences between 'master' and 'next' with the traditional
layout.  They happen to be the same name as what the remote side
has, but they are VASTLY different.  'master' is *your* local
branch you can freely use to do whatever you want, and does not
have to have ANYTHING to do with the remote 'master' ('origin'
is the mirror of remote 'master', not 'master').  While 'next'
is the mirror of remote 'next' and you are not even supposed to
touch it.

The separate remote layout instead adds remotes/origin/next.  It
does not fork heads/next from it when a clone is made.  I guess
we could do that, and I do not have a strong preference myself.
I think other poeple on the list (especially the ones who wanted
separate remote layout) prefer not to have the automatic fork of
all remote branches.

I think the current way reflects the distributed nature of git
better.

 * You may not necessarily care what happens on the non-primary
   branch at the remote side.  You are not forced to have the
   same branch structure as the remote has.  People who happily
   track 'master' can even forget about other branches tracked
   in remotes/origin/ and have heads/master and nothing else in
   the local branch namespace.

 * Your heads/ namespace is of your own.  The repository clone
   sets up arranges the master branch to start from the same
   commit as where you cloned from, and sets up so that changes
   made on the master at the remote is merged into your master,
   but that is merely a convention that was deemed as the most
   common and the most convenient.

One inconvenience for disciplined people is that traditional
layout allowed "git checkout next" to compile and install what
the remote side placed on a non-primary branch 'next'.

If you wanted to do _ANY_ development on top of it (even if that
was just to set '#define DEBUG 1'), however, you would need to
branch off of it, say "git checkout -b my-next next", so the
above inconvenience is arguably very minor.  On the other hand,
the downside are the pollution of your own heads/ namespace
(having your forked branch and remote tracking branch means
heads/ would have next and pu from me and my-next and my-pu you
work on), and some people are not as careful and disciplined as
you are and have made mistakes of committing their own changes
while on such remote tracking branches (you can call the latter
part "newbie protection").  The next 'git fetch', especially
when it was forced _and_ was done while on 'next', was rather
unpleasant.  Of course, discipined people like you and me would
never do that, but we are not the only two people in the
universe ;-).

The issue of "checking the tracking branch out to look-and-see"
is going to be addressed by the upcoming detached HEAD support,
so personally I do not think it will be a huge problem.  You
would have to say "git checkout origin/next" for it with the new
layout.

Another thing the separate remote gives us is to make it easier
to interact with more than one remote, by having a new directory
next to remotes/origin/ directory.  If you used the traditional
layout, your heads/ namespace would explode because you would
have to have something like:

	heads/origin		-- from linux-2.6's master
        heads/origin-libata	-- from jgarzik libata ALL
	heads/origin-sii-lbt	-- from jgarzik libata sii-lbt
        ...

in a flat namespace intermixed with your own development
branches, instead of:

	remotes/origin/master	-- from linux-2.6's master
        remotes/libata/ALL
        remotes/libata/sii-lbt
	...

grouped together by where they come from (and they cannot
possibly be mucked around directly thanks to being in remotes/
hierarchy), separated from any of your own development branches
that are in heads/.

Also, this is only about the default layout clone makes, and the
refs are not packed, so you can easily move remotes/origin/*
(except 'HEAD') to heads/ immediately after making a clone and
set up remotes.*.fetch configuration to match the traditional
layout if that is what you find more convenient.  The operation
in a repository layed out in the traditional way is still fully
supported (and will continue to be -- that's what old timers
use), and there is not even a reason to change your work habit
by converting an existing repository.

Re: New way of tracking remote branches -- question

[Steven Grimm]

Junio C Hamano wrote:
> I think other poeple on the list (especially the ones who wanted
> separate remote layout) prefer not to have the automatic fork of
> all remote branches.
>   

I'm a big fan of separate remote, and while I don't necessarily think 
automatic fork should be the default behavior, it might be nice to have 
an option for it, including setting up the appropriate config entries so 
push/pull do the expected things.

If that doesn't sound palatable, maybe an option to specify which remote 
branch to check out by default -- that way you can tell someone to run a 
particular git-clone command to get a local copy of a specific branch of 
yours. We just had a need for that here today, in fact. The current 
behavior is pretty screwy when you want to tell someone to clone your 
development environment to look at what you're doing; they don't have a 
good way of knowing which of your branches they're going to end up with, 
since you might be working in any of them when they happen to run 
git-clone. You end up having to say, "Run git-clone, then run 
git-branch, and if you don't see '* xyz', then run 'git-checkout -b xyz 
origin/xyz'."

-Steve

Re: New way of tracking remote branches -- question

[Theodore Tso]

On Fri, Jan 05, 2007 at 05:11:04PM -0800, Junio C Hamano wrote:
>  * Your heads/ namespace is of your own.  The repository clone
>    sets up arranges the master branch to start from the same
>    commit as where you cloned from, and sets up so that changes
>    made on the master at the remote is merged into your master,
>    but that is merely a convention that was deemed as the most
>    common and the most convenient.

The other nice thing about the new scheme is that if new branches
appear in the remote repository, they automatically show up in
remote/origin/*.  Before, if new branches showed up after the initial
clone, you'd never know about it and it wouldn't be possible to
automatically create new branches in heads/ since it might conflict
with an originally existing branch in the local namespace.  So IMHO,
the separation of namespace is definitely a good thing.

						- Ted

Re: New way of tracking remote branches -- question

[Luben Tuikov]

> Junio C Hamano <junkio@cox.net> writes:
> 
> If you wanted to do _ANY_ development on top of it (even if that
> was just to set '#define DEBUG 1'), however, you would need to
> branch off of it, say "git checkout -b my-next next", so the
> above inconvenience is arguably very minor.  On the other hand,
> the downside are the pollution of your own heads/ namespace
> (having your forked branch and remote tracking branch means
> heads/ would have next and pu from me and my-next and my-pu you
> work on), and some people are not as careful and disciplined as
> you are and have made mistakes of committing their own changes
> while on such remote tracking branches (you can call the latter
> part "newbie protection").  The next 'git fetch', especially
> when it was forced _and_ was done while on 'next', was rather
> unpleasant.  Of course, discipined people like you and me would
> never do that, but we are not the only two people in the
> universe ;-).

LOL, I love your sense of humor here! :-)

> The issue of "checking the tracking branch out to look-and-see"
> is going to be addressed by the upcoming detached HEAD support,
> so personally I do not think it will be a huge problem.  You
> would have to say "git checkout origin/next" for it with the new
> layout.

I guess, what I want is still to be able to "see" remotes/
when I do "git branch".

Sometimes I don't care to check out the remote branch, I just
want to rev-list it.  Sometimes I do check it out so I can
take a look and when I pull, I do the required reset, but
as you said above :-) this requires discipline.

And of course if I want to do my own development, I branch
off, as I've done with "next", and never forget to "git-pull . next"
after I pull from you.  Again this requires discipline.

But what I appreciate in git, is this discipline.  The fact that
I'm in control of it and that I have to do it -- keeps my mind sharp.

I really think that the line separating porcelains from git
should be drawn at some point.

I think that this "automatic merge/newbie protection" policy,
should've been well suited to porcelains as opposed to
git.

> Another thing the separate remote gives us is to make it easier
> to interact with more than one remote, by having a new directory
> next to remotes/origin/ directory.  If you used the traditional
> layout, your heads/ namespace would explode because you would
> have to have something like:
> 
> 	heads/origin		-- from linux-2.6's master
>         heads/origin-libata	-- from jgarzik libata ALL
> 	heads/origin-sii-lbt	-- from jgarzik libata sii-lbt
>         ...
> 
> in a flat namespace intermixed with your own development
> branches, instead of:
> 
> 	remotes/origin/master	-- from linux-2.6's master
>         remotes/libata/ALL
>         remotes/libata/sii-lbt
> 	...
> 
> grouped together by where they come from (and they cannot
> possibly be mucked around directly thanks to being in remotes/
> hierarchy), separated from any of your own development branches
> that are in heads/.

I don't mind this at all, as long as I get to "see" it, when
I do normal git operations, like "git branch".

So maybe we should take it one step at a time and first
implement that, still exposing it to "git branch" and then
if porcelains want to establish a policy on top of this,
that's fine.  But low-level git should not hide this from me.

    Luben

> Also, this is only about the default layout clone makes, and the
> refs are not packed, so you can easily move remotes/origin/*
> (except 'HEAD') to heads/ immediately after making a clone and
> set up remotes.*.fetch configuration to match the traditional
> layout if that is what you find more convenient.  The operation

I do find that more convenient, but wouldn't like to have
to do that, nor to have a script that does it for me to have
to do it after I clone.

It would be most convenient when this per-repo layout is
implemented, yet I get to see this when I do "git-branch".

I guess, most convenent, yet "newbie-protection" would be
to disallow any commits to remotes/* in this way... I'd opt
for that wholeheartedly.

     Luben

Re: New way of tracking remote branches -- question

[Junio C Hamano]

Luben Tuikov <ltuikov@yahoo.com> writes:

> I guess, what I want is still to be able to "see" remotes/
> when I do "git branch".

Then do not use "remotes/" but hang everything underneath
"heads/" (or "git branch -a").

Of course, the next complaint from you would be that the latest
git-clone would not make the traditional layout, but ...

> I think that this "automatic merge/newbie protection" policy,
> should've been well suited to porcelains as opposed to
> git.

... at that point, honestly speaking, I feel that you should do
your own "spartan porcelain" that lays things out in a way you
prefer.

> So maybe we should take it one step at a time and first
> implement that, still exposing it to "git branch" and then
> if porcelains want to establish a policy on top of this,
> that's fine.  But low-level git should not hide this from me.

No low-level is hiding anything from you.

It used to be 'find .git/refs/ -type f' was the lowest level,
and these days git-for-each-ref is the lowest level.  "git
branch" has always been Porcelain.

> I really think that the line separating porcelains from git
> should be drawn at some point.

And drawing this line was done long time ago (notice different
sections in Documentation/git.txt).

While I've been claiming that I do not do Porcelain, the nature
of what are shipped with git have changed enough over time.  The
Porcelain-ish has long been "barely usable spartan set", hoping
that people would do different porcelains depending on their
needs, but that has never materialized, except perhaps a notable
example of StGIT.

I described my position on recent focus on Porcelain-ish in the
initial "digression" part of this:

http://thread.gmane.org/gmane.comp.version-control.git/34908/focus=34914

We've taken "one step at a time" by first introducing separate
remote layout as an option, taught 'git branch' the needed
options such as -a and -r, and then made this the default
layout.  It still looks _different_ to me from what I am used
to, but that's just traditionalist speaking and I cannot really
deny the advantage of the separate remote layout offers.  In
other words, we have been doing "one step at a time".

Re: New way of tracking remote branches -- question

[Luben Tuikov]

--- Junio C Hamano <junkio@cox.net> wrote:
> > I guess, what I want is still to be able to "see" remotes/
> > when I do "git branch".
> 
> Then do not use "remotes/" but hang everything underneath
> "heads/" (or "git branch -a").
> 
> Of course, the next complaint from you would be that the latest
> git-clone would not make the traditional layout, but ...

What I meant was something of the lines of "git branch -r".
As I said, I don't mind the new way of seprating remote
branches per repo as is done by remotes/*/ .

> We've taken "one step at a time" by first introducing separate
> remote layout as an option, taught 'git branch' the needed
> options such as -a and -r, and then made this the default
> layout.  It still looks _different_ to me from what I am used
> to, but that's just traditionalist speaking and I cannot really
> deny the advantage of the separate remote layout offers.  In
> other words, we have been doing "one step at a time".

Me neither.  I also think there is an advantage of separately
tracking remotes as per remotes/*/ , so that organization is
better, i.e. per remote repo.

    Luben

Re: New way of tracking remote branches -- question

[Martin Langhoff]

On 1/6/07, Junio C Hamano <junkio@cox.net> wrote:
> You are talking about the separate remote layout "git clone"
> creates, and talking about the fact that there is nothing in
> refs/heads/ except master while refs/remotes/origin/ mirrors
> what the remote side has faithfully, aren't you?
>
> If that is the case, I can explain.  It is a good topic to talk
> about.

It's confusing me too ;-) though I think I'm getting to understand it.
Still more comfortable with the cogito style, personally. The only
thing I see as a clear improvement in git "remotes" is that I can
fetch a bundle of heads for review. And that it can track heads that
rewind. All the magic that happens if you have multiple pull lines is
a bit scary, the kind of thing that should be disallowed or use a
different syntax.

One aspect that I am getting used to, but I still consider slightly
bogus is that as we have added a layer of indirection, what used to
point to a specific remote branch now points to a bundle or remotes
heads.

It is quite disconcerting to say git-pull origin, and see that pu is
being pulled in. And  fortunately I'm tracking repos that don't have
an 'origin' head. Otherwise I'd get origin/origin. :-(

[reading on now...]



martin