Separate default remotes for pulling and pushing

Separate default remotes for pulling and pushing

[David Lee]

Hello,

I couldn't find a way to do this, but it seems it would be a fairly common use-case:

I want to set up different default remotes for pushing and pulling so that I can pull from remote a using "git pull" and push to remote b using "git push". This is particularly useful when committing to projects on github, since you may only push to your fork, but you only want to pull updates from the original.

Right now, I can either set the original as my default remote, in which case I must use "git push myfork", or I can set my fork as the default remote, in which case I must use "git pull original".

Please let me know if I'm overlooking a feature already present in git that allows me to have separate remotes for pull and push. Otherwise, I hope it gets implemented in the next version of git!

Thanks for reading,
David

Re: Separate default remotes for pulling and pushing

[Sverre Rabbelier]

Heya,

On Sat, May 7, 2011 at 10:10, David Lee <davidomundo@gmail.com> wrote:
> I want to set up different default remotes for pushing and pulling

See 'git remote set-url --push' [0].

[0] http://www.kernel.org/pub/software/scm/git/docs/git-remote.html

-- 
Cheers,

Sverre Rabbelier

Re: Separate default remotes for pulling and pushing

[Jeff King]

On Sat, May 07, 2011 at 11:50:35AM +0200, Sverre Rabbelier wrote:

> On Sat, May 7, 2011 at 10:10, David Lee <davidomundo@gmail.com> wrote:
> > I want to set up different default remotes for pushing and pulling
> 
> See 'git remote set-url --push' [0].
> 
> [0] http://www.kernel.org/pub/software/scm/git/docs/git-remote.html

That probably doesn't quite do what David wants. It's useful when the
URL for pushing and pulling a particular repository are different.

But if I understand it correctly, David has two _separate_ repositories.
And using remote.*.pushurl for that has some unwanted side effects,
because the tracking ref namespace (i.e., "remotes/origin/*") is shared
by both, even though their refs may not be at the same position.

For example, when pushing "refs/heads/master" to the remote, git will
update "refs/remotes/origin/master" to the pushed value. But that ref is
supposed to reflect the value of the last fetch from his "original"
repository, and now it doesn't. The ref value will flip back and forth
between what's in the two repositories as he pushes and fetches.

I don't think there is currently a way to do what he wants. I think it
would be useful for certain workflows. As I see it, there are a few
pretty common setups for remotes:

  1. Centralized; you push and pull from a single repo, shared with
     other developers. In this case, you call it "origin" and everything
     is easy.

  2. Decentralized, you're the maintainer; you pull from other people or
     apply patches via email. When you push, you push out to some
     publishing point. You'd probably call your publishing point
     "origin" (though I think Junio calls kernel.org "ko" and just types
     "git push ko"). You never pull from some single place all the time,
     so you don't care about a shorthand for it.

  3. Decentralized, you're a developer with a patch-based workflow; you
     fetch from the maintainer, then develop and submit patches via a
     mailing list. You call the maintainer's repo "origin" and pull from
     it automatically.  You never push, so it doesn't matter that
     there's no shorthand.

  4. Decentralized, you're a developer that publishes work via git. You
     call the upstream maintainer "origin", so fetches are convenient
     (and git does this for you at clone, after all). But pushing, even
     though you probably always push to the same central, does not have
     a convenient shorthand.

     This is David's case (and mine, and I suspect some other git
     developers who do enough work that they want to make it publicly
     available via git, or even just have backups). It's also encouraged
     by sites like github, where you might clone the upstream's
     repository, but then pushes your changes up to a personal "fork"
     to let others see and merge them.

So I think part of the reason we don't have such an option is that cases
1-3 are so common. But it would be a nice convenience for people in case
4.

-Peff

Re: Separate default remotes for pulling and pushing

[David Lee]

Hey Jeff,

Thanks for the super write-up.

> That probably doesn't quite do what David wants. It's useful when the
> URL for pushing and pulling a particular repository are different.
> 
> But if I understand it correctly, David has two _separate_ repositories.
> And using remote.*.pushurl for that has some unwanted side effects,
> because the tracking ref namespace (i.e., "remotes/origin/*") is shared
> by both, even though their refs may not be at the same position.
> 
> For example, when pushing "refs/heads/master" to the remote, git will
> update "refs/remotes/origin/master" to the pushed value. But that ref is
> supposed to reflect the value of the last fetch from his "original"
> repository, and now it doesn't. The ref value will flip back and forth
> between what's in the two repositories as he pushes and fetches.

I was actually using Sverre's recommendation with virtually no problems except for what you mention: the ref value flips back and forth.

>  4. Decentralized, you're a developer that publishes work via git. You
>     call the upstream maintainer "origin", so fetches are convenient
>     (and git does this for you at clone, after all). But pushing, even
>     though you probably always push to the same central, does not have
>     a convenient shorthand.

By "push to the same central", I assume you mean "push to the same mirror of origin".

> 
>     This is David's case (and mine, and I suspect some other git
>     developers who do enough work that they want to make it publicly
>     available via git, or even just have backups). It's also encouraged
>     by sites like github, where you might clone the upstream's
>     repository, but then pushes your changes up to a personal "fork"
>     to let others see and merge them.
> 
> So I think part of the reason we don't have such an option is that cases
> 1-3 are so common. But it would be a nice convenience for people in case
> 4.

I think github is making option 4 the dominant use case. In fact, in our workplace we have a similar workflow set up, where we pull from a central origin, but push to individual mirrors from where commits are reviewed, tested, and merged unto origin.

--David

Re: Separate default remotes for pulling and pushing

[Jeff King]

On Mon, May 09, 2011 at 01:34:22AM -0700, David Lee wrote:

> >  4. Decentralized, you're a developer that publishes work via git. You
> >     call the upstream maintainer "origin", so fetches are convenient
> >     (and git does this for you at clone, after all). But pushing, even
> >     though you probably always push to the same central, does not have
> >     a convenient shorthand.
> 
> By "push to the same central", I assume you mean "push to the same mirror of origin".

Yeah, sorry, that was supposed to be "central spot", but I think you got
the meaning.

> I think github is making option 4 the dominant use case. In fact, in
> our workplace we have a similar workflow set up, where we pull from a
> central origin, but push to individual mirrors from where commits are
> reviewed, tested, and merged unto origin.

Yeah, I think we will see more of that as decentralized workflows (and
the tools that support them) mature.

With respect to supporting an alternate default push destination, I'm
not sure what is the best change to make. If "origin" were simply the
default, I would say we should have a push.defaultRemote config that
lets you specify something else.

But it's not that simple. If you are on branch "foo", and you have
"branch.foo.remote" set in your config, then the value of that config
option becomes the default remote. Which makes some sense for pulling
(and there is the associated branch.*.merge config), but of course
pushing may not match.

But now we have precedence questions. If I have config like:

  [push]
    defaultRemote = my-mirror

  [branch "foo"]
    remote = origin
    merge = refs/heads/master

which remote should be the default for "git push"? Obviously if I'm not
on "foo", it should be my-mirror. But if I am, should push.defaultRemote
take precedence? Should there also be a branch.*.pushRemote config that
takes precedence over branch.*.remote?

I have to admit that I have never found the branch.*.remote config to be
useful for any of my workflows, so I am not really sure how people use
it.

-Peff

Re: Separate default remotes for pulling and pushing

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

>   4. Decentralized, you're a developer that publishes work via git. You
>      call the upstream maintainer "origin", so fetches are convenient
>      (and git does this for you at clone, after all). But pushing, even
>      though you probably always push to the same central, does not have
>      a convenient shorthand.
>
>      This is David's case (and mine, and I suspect some other git
>      developers who do enough work that they want to make it publicly
>      available via git, or even just have backups). It's also encouraged
>      by sites like github, where you might clone the upstream's
>      repository, but then pushes your changes up to a personal "fork"
>      to let others see and merge them.

In a sense, this is what I do as well.  As you mentioned, I push to "ko"
to publish, but when I "fetch" (or "pull") from "origin", I get the public
copy I have at kernel.org like everybody else, and I do fetch every time
after I push to "ko" to get the updated preformatted HTML and man page
branches.

I never felt it awkward that my fetch says "origin" and my push says "ko".
Even though I imagine that I could smash them together into a single
"origin", it would not give me anything, as I also push to push to
different places like repo.or.cz, sourceforge and github anyway.

While I see why some people might want to say "origin" for both in such a
set-up (when they do not push to multiple places like I do), I have a
feeling that it is a misguided wish that would make themselves unnecessary
confused than they already are, especially if the repositories used for
pushing and fetching are in reality different repositories (one good
example why it would be confusing is how remote tracking branches are
updated).

Re: Separate default remotes for pulling and pushing

[David Lee]

> But now we have precedence questions. If I have config like:
> 
>  [push]
>    defaultRemote = my-mirror
> 
>  [branch "foo"]
>    remote = origin
>    merge = refs/heads/master
> 
> which remote should be the default for "git push"? Obviously if I'm not
> on "foo", it should be my-mirror. But if I am, should push.defaultRemote
> take precedence? Should there also be a branch.*.pushRemote config that
> takes precedence over branch.*.remote?
> 
> I have to admit that I have never found the branch.*.remote config to be
> useful for any of my workflows, so I am not really sure how people use
> it.

What about removing the branch.*.remote config by default, and if it's not set, then it defaults to whatever the repo-wide setting is for defaultRemote? Then a branch.*.remote would override the defaultRemote, and a branch.*.pushRemote would again override that.

Re: Separate default remotes for pulling and pushing

[Jeff King]

On Mon, May 09, 2011 at 09:45:49AM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> >   4. Decentralized, you're a developer that publishes work via git. You
> >      call the upstream maintainer "origin", so fetches are convenient
> >      (and git does this for you at clone, after all). But pushing, even
> >      though you probably always push to the same central, does not have
> >      a convenient shorthand.
> >
> >      This is David's case (and mine, and I suspect some other git
> >      developers who do enough work that they want to make it publicly
> >      available via git, or even just have backups). It's also encouraged
> >      by sites like github, where you might clone the upstream's
> >      repository, but then pushes your changes up to a personal "fork"
> >      to let others see and merge them.
> 
> In a sense, this is what I do as well.  As you mentioned, I push to "ko"
> to publish, but when I "fetch" (or "pull") from "origin", I get the public
> copy I have at kernel.org like everybody else, and I do fetch every time
> after I push to "ko" to get the updated preformatted HTML and man page
> branches.

Interesting. Is your fetch from "ko" a no-op, or are you using it to
syncrhonize development between different machines?

> While I see why some people might want to say "origin" for both in such a
> set-up (when they do not push to multiple places like I do), I have a
> feeling that it is a misguided wish that would make themselves unnecessary
> confused than they already are, especially if the repositories used for
> pushing and fetching are in reality different repositories (one good
> example why it would be confusing is how remote tracking branches are
> updated).

I think it is important to note that calling them both "origin" is
definitely the wrong thing. The proposal is instead that "git push"
without a remote would default to something besides "origin". For people
who publish multiple places, it might even make sense for it to be an
iterative push to each place.

But this is all definitely a minor convenience. It is not that
world-shattering to type "git push my-fork", nor "git push my-fork &&
git push my-backup". Probably twice a week I accidentally try to push to
"git://git.kernel.org/pub/scm/git/git.git", but git make it clear that
is not allowed. :)

-Peff

Re: Separate default remotes for pulling and pushing

[Jeff King]

On Mon, May 09, 2011 at 12:01:53PM -0700, David Lee wrote:

> > But now we have precedence questions. If I have config like:
> > 
> >  [push]
> >    defaultRemote = my-mirror
> > 
> >  [branch "foo"]
> >    remote = origin
> >    merge = refs/heads/master
> > 
> > which remote should be the default for "git push"? Obviously if I'm not
> > on "foo", it should be my-mirror. But if I am, should push.defaultRemote
> > take precedence? Should there also be a branch.*.pushRemote config that
> > takes precedence over branch.*.remote?
> > 
> > I have to admit that I have never found the branch.*.remote config to be
> > useful for any of my workflows, so I am not really sure how people use
> > it.
> 
> What about removing the branch.*.remote config by default, and if it's
> not set, then it defaults to whatever the repo-wide setting is for
> defaultRemote? Then a branch.*.remote would override the
> defaultRemote, and a branch.*.pushRemote would again override that.

The branch.*.remote config does much more, though. For example, it is
part of the "upstream" branch config. So getting rid of it is definitely
not an option.

-Peff

Re: Separate default remotes for pulling and pushing

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

>> In a sense, this is what I do as well.  As you mentioned, I push to "ko"
>> to publish, but when I "fetch" (or "pull") from "origin", I get the public
>> copy I have at kernel.org like everybody else, and I do fetch every time
>> after I push to "ko" to get the updated preformatted HTML and man page
>> branches.
>
> Interesting. Is your fetch from "ko" a no-op, or are you using it to
> syncrhonize development between different machines?

Preformatted html and man pages are made when I push the integrated source
branches out, triggered from the post-update hook at kernel.org.  I do not
format them on my box.

I say "git push ko", take a sip of tea, and after k.org finishes the
asciidoc dance "git fetch ko" to get them all, before pushing
remotes/ko/{maint,master,next,pu,todo,html,man} to secondary sites.

> I think it is important to note that calling them both "origin" is
> definitely the wrong thing. The proposal is instead that "git push"
> without a remote would default to something besides "origin". For people
> who publish multiple places, it might even make sense for it to be an
> iterative push to each place.

Either we add branch.<name>.pushremote or push.defaultremote then?

Re: Separate default remotes for pulling and pushing

[Jay Soffian]

On Mon, May 9, 2011 at 6:04 PM, Jeff King <peff@peff.net> wrote:
> I think it is important to note that calling them both "origin" is
> definitely the wrong thing. The proposal is instead that "git push"
> without a remote would default to something besides "origin". For people
> who publish multiple places, it might even make sense for it to be an
> iterative push to each place.

While developing in a particular repo, I constantly have to push to
two compile machines. I just dropped a Makefile into the top of my
working tree:

all: push
amend:
	git amend -a
	git --no-pager diff @{1}
	make push
push:
	make -j 2 mac win
mac:
	git push mac
win:
	git push win

j.

Re: Separate default remotes for pulling and pushing

[Jeff King]

On Mon, May 09, 2011 at 03:46:19PM -0700, Junio C Hamano wrote:

> > Interesting. Is your fetch from "ko" a no-op, or are you using it to
> > syncrhonize development between different machines?
> 
> Preformatted html and man pages are made when I push the integrated source
> branches out, triggered from the post-update hook at kernel.org.  I do not
> format them on my box.

Ah, that makes sense.

> > I think it is important to note that calling them both "origin" is
> > definitely the wrong thing. The proposal is instead that "git push"
> > without a remote would default to something besides "origin". For people
> > who publish multiple places, it might even make sense for it to be an
> > iterative push to each place.
> 
> Either we add branch.<name>.pushremote or push.defaultremote then?

I don't think most people would want just branch.<name>.pushremote. They
are more likely to want to always push to some place, so
push.defaultremote is a better choice.

But that has weird precedence problems. We auto-create branch.*.remote,
so if branch-specific config takes precedence, their push.defaultremote
will almost never be used. And if branch-specific config _doesn't_ take
precedence, then that is weird and unlike almost every other part of
git.

Another option is to mark the remote with an explicit "don't push here,
push to this other remote instead" config, like:

  $ git remote add my-fork host:project.git
  $ git config remote.origin.pushremote my-fork
  $ git push ;# acts as if you did "git push my-fork"

And then when we default to a remote, either because it is "origin" or
because it is in branch.*.remote, then we will end up pushing to the
right place.

I'm unsure what should happen in the case of:

  $ git config remote.origin.pushremote my-fork
  $ git push origin

In the workflow we described, "origin" is simply not a push-able remote,
so it is a slight convenience to intercept all pushes for it and
transparently rewrite them to "my-fork". And it makes the concept of
that config slightly simpler. It basically becomes the equivalent of
remote.*.pushurl, except that we are clear that it is a totally separate
remote with different tracking branches.

But it does eliminate any workflow where "origin" is _sometimes_
pushable, but you almost always want to push somewhere else instead.
Because you have no way to push to "origin" now. I don't know of any
such workflow, but I hate to be restrictive for no good reason.

If it just handles "use this remote instead only in the case of a
default selection", then the name should probably be like
"defaultpushremote" or something.

-Peff

Re: Separate default remotes for pulling and pushing

[Jeff King]

On Tue, May 10, 2011 at 08:47:53AM -0400, Jay Soffian wrote:

> On Mon, May 9, 2011 at 6:04 PM, Jeff King <peff@peff.net> wrote:
> > I think it is important to note that calling them both "origin" is
> > definitely the wrong thing. The proposal is instead that "git push"
> > without a remote would default to something besides "origin". For people
> > who publish multiple places, it might even make sense for it to be an
> > iterative push to each place.
> 
> While developing in a particular repo, I constantly have to push to
> two compile machines. I just dropped a Makefile into the top of my
> working tree:
> 
> all: push
> amend:
> 	git amend -a
> 	git --no-pager diff @{1}
> 	make push
> push:
> 	make -j 2 mac win
> mac:
> 	git push mac
> win:
> 	git push win

Yeah, I have scripts to help with that sort of thing now. But that is
often an unsatisfactory solution, because either:

  1. Your Makefile is not version-controlled.

  2. Your are polluting the project history with stuff specific to your
     workflow. I would not get very far proposing that git.git's
     Makefile contain such a thing. :)

That's how Junio ended up with his "Meta" directory (and I have one,
too, but with drastically different things in it).

-Peff

Re: Separate default remotes for pulling and pushing

[Jay Soffian]

On Tue, May 10, 2011 at 4:20 PM, Jeff King <peff@peff.net> wrote:
> Yeah, I have scripts to help with that sort of thing now. But that is
> often an unsatisfactory solution, because either:
>
>  1. Your Makefile is not version-controlled.

Sure, but your .git/config is similarly not version controlled.

>
>  2. Your are polluting the project history with stuff specific to your
>     workflow. I would not get very far proposing that git.git's
>     Makefile contain such a thing. :)
>
> That's how Junio ended up with his "Meta" directory (and I have one,
> too, but with drastically different things in it).

As well. :-)

j.