Automatically remote prune

Automatically remote prune

[John Tapsell]

Hi all,

  Is there any particular reason why "git update" doesn't
automatically remove remote branches that have been removed on the
server?  I keep getting questions about this from confused users.

  If there's strong resistance against that (there usually is for
changing any default behaviour), could we at least mark deleted
branches?
So maybe:

$> git branch -r
origin/blah1 [Deleted]
origin/blah2

(Some branches have been deleted on the remote server.  Use  "git
remote prune origin" to remove these)

Or something.

John

Re: Automatically remote prune

[Junio C Hamano]

John Tapsell <johnflux@gmail.com> writes:

> $> git branch -r
> origin/blah1 [Deleted]
> origin/blah2
>
> (Some branches have been deleted on the remote server.  Use  "git
> remote prune origin" to remove these)

There is no information locally available to do this, unless you are
willing to contact the remote every time somebody says "branch -r" (or
"branch -a").  I tend to think it is not very nice for the branch command
that has long been a "local" command to suddenly start talking to outside
world.

You could store necessary information somewhere else when you contacted
the remote the last time, but we need to consider what the benefits are to
give this information in the first place.a

The [Deleted] mark in your suggestion tells the user:

    This is already removed in the remote, and this tracking copy is the
    only way for you to view it ever again.  Do not run 'remote prune
    origin' blindly otherwise you will lose it.

But that is already possible with "git remote show origin", isn't it?

Re: Automatically remote prune

[John Tapsell]

2009/11/5 Junio C Hamano <gitster@pobox.com>:
> John Tapsell <johnflux@gmail.com> writes:

> You could store necessary information somewhere else when you contacted
> the remote the last time, but we need to consider what the benefits are to
> give this information in the first place.

We already get all this information on a "git fetch", no?  And then
promptly discard it.  Surely when we do "git fetch" .  So I'm talking
about just not ignoring the information we get from git fetch, but
present that information to the user.

John

Re: Automatically remote prune

[Shawn O. Pearce]

John Tapsell <johnflux@gmail.com> wrote:
> 2009/11/5 Junio C Hamano <gitster@pobox.com>:
> > John Tapsell <johnflux@gmail.com> writes:
> 
> > You could store necessary information somewhere else when you contacted
> > the remote the last time, but we need to consider what the benefits are to
> > give this information in the first place.
> 
> We already get all this information on a "git fetch", no?  And then
> promptly discard it.  Surely when we do "git fetch" .  So I'm talking
> about just not ignoring the information we get from git fetch, but
> present that information to the user.

Good point.  We currently don't have a provision to store this
information, but we could store a list of dead remote tracking
branches for reference later during `git branch -r`.

Its not a lot of data, it just has little perceived value to most
Git hackers because a remote branch disappears very infrequently
(if ever) for most of us.

-- 
Shawn.

Re: Automatically remote prune

[Junio C Hamano]

John Tapsell <johnflux@gmail.com> writes:

> 2009/11/5 Junio C Hamano <gitster@pobox.com>:
>> John Tapsell <johnflux@gmail.com> writes:
>
>> You could store necessary information somewhere else when you contacted
>> the remote the last time, but we need to consider what the benefits are to
>> give this information in the first place.
>
> We already get all this information on a "git fetch", no?

"what the benefits are to give this information _in the 'branch' output_"
was what I meant.  From the part you omitted from my message:

    The [Deleted] mark in your suggestion tells the user:

        This is already removed in the remote, and this tracking copy is the
        only way for you to view it ever again.  Do not run 'remote prune
        origin' blindly otherwise you will lose it.

There are two reasons I could think of that the user might want to know
this.

 (1) The user wants to keep the remotes/<origin>/* namespace clean (iow,
     the user does not care to keep old commits that were deemed bad by
     the remote side) by removing stale tracking refs.  But in this case,
     it is very unlikely that the user would use "git branch -d -r" to
     remove stale ones one-by-one after seeing '[Deleted]' label in the
     output from "git branch -r".  Rather he would run "git remote prune
     origin" to blindly remove them all.

 (2) The user does want to be careful not to lose commits that now only
     exists in his repository.  Perhaps he saw something worthwhile to
     base his topic later on.  But these stale remote tracking refs are
     not removed until the user runs "git remote prune origin".  As long
     as we give him a way to check what will be pruned before running "git
     remote prune", there is not much point in showing that information in
     output of "git branch -r".  There is no need to keep extra info by
     creating a new file in .git by "fetch". Nor showing that to the user
     when he does "fetch" either, for that matter.

A better approach to please the first class of audience may be to
introduce an option that tells fetch to cull tracking refs that are stale.
Then "branch -r" output will not show stale refs and there is no place
(nor need) to show [Deleted] labels.

Such an option won't be very useful for the second class of audience,
though.  For them we would need something else, and it would likely be an
enhancement to "git remote".  It would ask the other side what refs are no
longer there, and then check our local refspace to see if there are local
topics based on them (which would mean the user is already in a trouble)
and which ones are not forked locally at all (which may mean "it wasn't
interesting to the user, and we can safely remove it" or "the user was
interested in it, but hasn't got around to forking from it yet, being busy
working on something else").  I am unsure what should be done in the
latter case (i.e. lost remote refs haven't been touched locally) but am
just thinking aloud.

Re: Automatically remote prune

[Sitaram Chamarty]

On Thu, Nov 5, 2009 at 7:53 AM, Junio C Hamano <gitster@pobox.com> wrote:

>  (1) The user wants to keep the remotes/<origin>/* namespace clean (iow,

[snip]

>  (2) The user does want to be careful not to lose commits that now only

This whole discussion is a conflict between those two.  The current
system does the latter, safer, thing.  John's users are getting
confused because they *think* this means the remote still has those
refs.

At best an alias that does a prune before (or after) a fetch should do.

And I notice 'git gui' has an option to "prune during fetch" -- maybe
that should be an option that is also made available to the CLI fetch.

Re: Automatically remote prune

[John Tapsell]

2009/11/5 Junio C Hamano <gitster@pobox.com>:
> John Tapsell <johnflux@gmail.com> writes:
> "what the benefits are to give this information _in the 'branch' output_"
> was what I meant.  From the part you omitted from my message:

I omitted it just because, imho, it's not what I 'care about'.  I'm
not trying to help advanced users (Users that _want_ to keep
remotes/origin/* clean and users that _want_ to be careful to not lose
commits are both advanced users, imho).  I'm just interested in
reducing confusion for non-advanced users.  So either not-showing
removed remote branches by default, or showing them but marking them
as deleted.

> A better approach to please the first class of audience may be to
> introduce an option that tells fetch to cull tracking refs that are stale.
> Then "branch -r" output will not show stale refs and there is no place
> (nor need) to show [Deleted] labels.

If it's a non-default option, then it won't help the non-advanced users.

> Such an option won't be very useful for the second class of audience,
> though.  For them we would need something else, and it would likely be an
> enhancement to "git remote".

Which still leaves confusion when viewing "git branch -r" since they
would show up there still.


John

Re: Automatically remote prune

[Junio C Hamano]

John Tapsell <johnflux@gmail.com> writes:

> I omitted it just because, imho, it's not what I 'care about'.  I'm
> not trying to help advanced users (Users that _want_ to keep
> remotes/origin/* clean and users that _want_ to be careful to not lose
> commits are both advanced users, imho).  I'm just interested in
> reducing confusion for non-advanced users.

I _think_ you are saying that your non-advanced users expect "branch -r"
output to be in sync (to the extent possible without going over the
network every time it is run) with the remote side. It is the same thing
as keeping remotes/origin/* free of stale remote branches, i.e. they are
in the first camp.  There is nothing advanced about either of the camps.
The former wants the view to be in sync, the latter wants a way to avoid
information loss.

It is understandable to expect "branch -r" output to be in sync with the
other end, especially if one has CVS/SVN mentality but even if one
doesn't, I would say it is a reasonable thing to expect.

I am open to an optional feature to "git fetch' to prune them, but I would
not make it the default from day one.  When introducing a change that
causes information loss, our migration strategy has always been "Give an
option first, release and wait for two releases or so, and then start
discussing to change the default behaviour."

The necessary change to "git fetch" shouldn't be too hard to code, as we
are already doing this in mirror mode.

Re: Automatically remote prune

[Jay Soffian]

On Thu, Nov 5, 2009 at 3:05 PM, Junio C Hamano <gitster@pobox.com> wrote:
> The necessary change to "git fetch" shouldn't be too hard to code, as we
> are already doing this in mirror mode.

Actually, mirror mode applies to push only. Unless I'm missing
something obvious.

Thus, adding prune to builtin-fetch duplicates the logic from
builtin-remote, unless I do some refactoring. Hmph, I was hoping for a
15 minute patch. :-(

j.

Re: Automatically remote prune

[Petr Baudis]

On Thu, Nov 05, 2009 at 06:09:03PM -0500, Jay Soffian wrote:
> On Thu, Nov 5, 2009 at 3:05 PM, Junio C Hamano <gitster@pobox.com> wrote:
> > The necessary change to "git fetch" shouldn't be too hard to code, as we
> > are already doing this in mirror mode.
> 
> Actually, mirror mode applies to push only. Unless I'm missing
> something obvious.

Perhaps you are, mirror mode applies to fetch as well; that's how e.g.
repo.or.cz mirror mode is done currently.

				Petr "Pasky" Baudis

Re: Automatically remote prune

[Jay Soffian]

On Thu, Nov 5, 2009 at 7:17 PM, Petr Baudis <pasky@suse.cz> wrote:
> On Thu, Nov 05, 2009 at 06:09:03PM -0500, Jay Soffian wrote:
>> Actually, mirror mode applies to push only. Unless I'm missing
>> something obvious.
>
> Perhaps you are, mirror mode applies to fetch as well; that's how e.g.
> repo.or.cz mirror mode is done currently.

Can you please be more specific? There is no mention of mirror in
builtin-fetch.c, only builtin-push.c. Further, the docs for
"remote.<name>.mirror" indicate it's a push option only.

So please explain what you mean by "that's how repo.or.cz mirror mode is done".

Cluelessly yours,

j.

Re: Automatically remote prune

[Petr Baudis]

On Thu, Nov 05, 2009 at 07:38:39PM -0500, Jay Soffian wrote:
> On Thu, Nov 5, 2009 at 7:17 PM, Petr Baudis <pasky@suse.cz> wrote:
> > On Thu, Nov 05, 2009 at 06:09:03PM -0500, Jay Soffian wrote:
> >> Actually, mirror mode applies to push only. Unless I'm missing
> >> something obvious.
> >
> > Perhaps you are, mirror mode applies to fetch as well; that's how e.g.
> > repo.or.cz mirror mode is done currently.
> 
> Can you please be more specific? There is no mention of mirror in
> builtin-fetch.c, only builtin-push.c. Further, the docs for
> "remote.<name>.mirror" indicate it's a push option only.
> 
> So please explain what you mean by "that's how repo.or.cz mirror mode is done".

Ok, I got a bit confused there. :-) For fetches, it just sets up the
refspec to point to main namespace instead of refs/remote/, of course it
doesn't use the remote.*.mirror config. I'm not sure what Junio meant by
the change being easy, seeing that push uses a technical trick in
send-pack that doesn't seem to be straightforwardly applicable to
fetch... but I don't know this part of code too well.

-- 
				Petr "Pasky" Baudis
A lot of people have my books on their bookshelves.
That's the problem, they need to read them. -- Don Knuth

Re: Automatically remote prune

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

> On Thu, Nov 05, 2009 at 07:38:39PM -0500, Jay Soffian wrote:
>> On Thu, Nov 5, 2009 at 7:17 PM, Petr Baudis <pasky@suse.cz> wrote:
>> > On Thu, Nov 05, 2009 at 06:09:03PM -0500, Jay Soffian wrote:
>> >> Actually, mirror mode applies to push only. Unless I'm missing
>> >> something obvious.
>> >
>> > Perhaps you are, mirror mode applies to fetch as well...

Sorry, my thinko.  We do not do mirrors in fetch.

Re: Automatically remote prune

[Petr Baudis]

On Thu, Nov 05, 2009 at 05:05:50PM +0900, John Tapsell wrote:
> 2009/11/5 Junio C Hamano <gitster@pobox.com>:
> > John Tapsell <johnflux@gmail.com> writes:
> > "what the benefits are to give this information _in the 'branch' output_"
> > was what I meant.  From the part you omitted from my message:
> 
> I omitted it just because, imho, it's not what I 'care about'.  I'm
> not trying to help advanced users (Users that _want_ to keep
> remotes/origin/* clean and users that _want_ to be careful to not lose
> commits are both advanced users, imho).  I'm just interested in
> reducing confusion for non-advanced users.  So either not-showing
> removed remote branches by default, or showing them but marking them
> as deleted.

Maybe, if your users want to know "what branches does the remote repo
have" instead of "what branches does my 'image' of the remote repo has",
just teach them to run

	git remote show origin

instead of

	git branch -r origin

as mentioned earlier by Junio?

Of course, users would still need to realize the actual meaning of the
commands and the fact that Git _does_ keep a distinctive 'image of
remote repository', so this may not be the most intuitive solution.

-- 
				Petr "Pasky" Baudis
A lot of people have my books on their bookshelves.
That's the problem, they need to read them. -- Don Knuth

Re: Automatically remote prune

[Dmitry Potapov]

On Wed, Nov 04, 2009 at 06:23:57PM -0800, Junio C Hamano wrote:
> 
> There are two reasons I could think of that the user might want to know
> this.
> 
>  (1) The user wants to keep the remotes/<origin>/* namespace clean (iow,
>      the user does not care to keep old commits that were deemed bad by
>      the remote side) by removing stale tracking refs.  But in this case,
>      it is very unlikely that the user would use "git branch -d -r" to
>      remove stale ones one-by-one after seeing '[Deleted]' label in the
>      output from "git branch -r".  Rather he would run "git remote prune
>      origin" to blindly remove them all.
> 
>  (2) The user does want to be careful not to lose commits that now only
>      exists in his repository.  Perhaps he saw something worthwhile to
>      base his topic later on.  But these stale remote tracking refs are
>      not removed until the user runs "git remote prune origin".  As long
>      as we give him a way to check what will be pruned before running "git
>      remote prune", there is not much point in showing that information in
>      output of "git branch -r".  There is no need to keep extra info by
>      creating a new file in .git by "fetch". Nor showing that to the user
>      when he does "fetch" either, for that matter.
> 
> A better approach to please the first class of audience may be to
> introduce an option that tells fetch to cull tracking refs that are stale.

'git remote update' already has the --prune option, so it would be only
logical if 'git fetch' has it too... Perhaps, it would be useful to add
a configuration option to automatically prune remote branches on fetch.

> Such an option won't be very useful for the second class of audience,
> though.  For them we would need something else, and it would likely be an
> enhancement to "git remote".

'git remote prune' has --dry-run:

$ git remote prune --dry-run origin
Pruning origin
URL: git://git.kernel.org/pub/scm/git/git.git
 * [would prune] origin/offcuts
 * [would prune] origin/old-next

(Hmm... I did not know I had them in my git repo....)

So, it is possible to do now, but personally, I would prefer if 'git
fetch' would tell what references are removed. It may appear a bit too
chatty if someone has a lot of deleted references in a remote repo, but
I really do not see any good reason to keep stale branches. If you think
that some reference is useful and you want to preserve it then you can
create a local branch, but in 99.9% cases people want just to remove a
stale branch.

Actually, accumulation of stale branches is only half of the problem.
In some cases, the stale branch can mislead people. For instance, some
feature has been implemented and was merged to 'pu'. After that, this
feature was re-worked and later merged to the 'master' branch, after
that the feature branch was removed. Yet, some users may have the stale
branch pointing on the original version and may think that it is still
not merged and buggy...

BTW, the current behavior of 'git fetch' does not really protect you from
losing commits. It protects commits only in the case when the branch is
deleted, but not when it is force-updated, which feels inconsistent to
me. Personally, I would prefer if 'git fetch' removed branches, but when
it removes or force-updates some branch, it adds a record to the reflog.


Dmitry