[RFC] Moving "git remote add --mirror blah" functionality to "git clone --bare --origin=blah"

[RFC] Moving "git remote add --mirror blah" functionality to "git clone --bare --origin=blah"

[Paolo Bonzini]

In the thread "git remote update -> rejected" Junio and Johannes came to 
the conclusion that "--mirror means that you do not have local 
branches", because "you give control away to the other end on the ref 
namespace".  Furthermore, it was agreed that --mirror currently makes 
sense mostly (or only?) on a bare repository.

 From here I gather that if you have "git remote add --mirror", most 
likely the mirrored repository will be the only remote you have.  There 
is in general no point in having other remotes in a bare repository. 
And so there is no loss of generality if this remote is the "origin" remote.

Hence, my proposal is:

1) Add an option to "git clone", to be used with --bare, to create a 
mirror.  --bare already leaves the original refs in place, without 
moving them under refs/remotes/origin, so it makes sense to optionally 
create the remote.

Actually you don't need to add a new option.  Right now, specifying 
-o/--origin together with --bare would give an error; in my proposal, if 
you had "--bare --origin=source" a remote would be created in the config 
file like this:

[remote "source"]
	url = ...
	fetch = +refs/*:refs/*

2) Modify the --mirror option to "git remote add", so that it creates a 
remote *acting as a mirror of this one*.  The config would look like this:

[remote "mirror"]
         url = ...
         fetch = +refs/heads/*:refs/remotes/mirror/*
         push = +refs/heads/*:refs/heads/*
         push = +refs/tags/*:refs/tags/*
         mirror


Now, I know that this is not what was proposed in the original --mirror 
thread, and it is backwards-incompatible.  On the other hand, I think it 
has several advantages:

1) it does not add special cases such as "--mirror is only to be used in 
a --bare repository".  It moves the functionality to a combination of 
options that used to give an error.

2) it provides an easy-to-use UI to create a mirror of this repository 
somewhere.  With a patch I already posted, "git push" would 
automatically update this mirror.

3) overall, I think it makes a lot of sense: "git remote add" is about 
adding one remote of many, and "git clone" is about setting up a single 
special remote.  Now, you can have many remotes that are 
mirrored-from-here, while you can only have one remote that is 
mirrored-to-here.  So, I believe that "git remote add" should add 
mirrored-from-here remotes, while "git clone" should be used to create a 
mirrored-to-here remote.

Opinions?

Paolo

Re: [RFC] Moving "git remote add --mirror blah" functionality to "git clone --bare --origin=blah"

[Johannes Schindelin]

Hi,

On Wed, 23 Apr 2008, Paolo Bonzini wrote:

> In the thread "git remote update -> rejected" Junio and Johannes came to 
> the conclusion that "--mirror means that you do not have local 
> branches", because "you give control away to the other end on the ref 
> namespace".  Furthermore, it was agreed that --mirror currently makes 
> sense mostly (or only?) on a bare repository.

Yes.  But the opposite may not be true: if you plan to work on many 
workdirs, but the same repository, you might still want to have separate 
remotes layout.

In any case, I think that this discussion is too early: clone should be 
builtin first, making things much simpler because of being a very thin 
layer on top of init && remote && fetch.

But yes, there is a lot of work left on the builtin-clone to make it so.

Ciao,
Dscho

Re: [RFC] Moving "git remote add --mirror blah" functionality to "git clone --bare --origin=blah"

[Junio C Hamano]

Paolo Bonzini <bonzini@gnu.org> writes:

> In the thread "git remote update -> rejected" Junio and Johannes came
> to the conclusion that "--mirror means that you do not have local
> branches", because "you give control away to the other end on the ref
> namespace".  Furthermore, it was agreed that --mirror currently makes
> sense mostly (or only?) on a bare repository.
>
> From here I gather that if you have "git remote add --mirror", most
> likely the mirrored repository will be the only remote you have.
> There is in general no point in having other remotes in a bare
> repository. And so there is no loss of generality if this remote is
> the "origin" remote.
>
> Hence, my proposal is:
>
> 1) Add an option to "git clone", to be used with --bare, to create a
> mirror.  --bare already leaves the original refs in place, without
> moving them under refs/remotes/origin, so it makes sense to optionally
> create the remote.

I actually had a slightly different vision.  A mid-term goal should be to
reimplement "clone" that has a lot of code duplication with "fetch" and
"remote" in terms of "init + remote + fetch + checkout" [*1*].  For that
to happen, I suspect that "remote" needs to learn a few more tricks than
it currently knows (e.g. "figuring out the HEAD").

I would agree that it is useful for "clone" to create a bare repository in
a mode that can be used for further cloning by other repositories (perhaps
the former sits at the firewall boundary that it is cumbersome to cross by
the latter).  As you described above, we already have that (iow, "--bare").

And "remote add --mirror" would be an implementation detail to produce
such a clone.  It is mostly about fetching.

An option to add a back-up repository that you can maintain an exact
mirror of your working repository would be useful, but that is different
from what "remote add --mirror" is about.

[Footnote]

*1* In that sense, a more sensible order than rewriting "clone" in C in
its current form would be to make necessary enhancements to the components
in this sequence that need to implement clone, figure out how they should
fit together and first make "clone" a four-liner shell script. Then
rewriting the result in C may become more trivial.

Re: [RFC] Moving "git remote add --mirror blah" functionality to "git clone --bare --origin=blah"

[Daniel Barkalow]

On Wed, 23 Apr 2008, Junio C Hamano wrote:

> *1* In that sense, a more sensible order than rewriting "clone" in C in
> its current form would be to make necessary enhancements to the components
> in this sequence that need to implement clone, figure out how they should
> fit together and first make "clone" a four-liner shell script. Then
> rewriting the result in C may become more trivial.

clone can't be a four-liner shell script, because it has to come up with 
the name of the working directory for a non-bare clone, which doesn't make 
sense for any other program. Currently, a local clone is special in that 
it duplicates all of the objects without checking whether the destination 
already has them (because it can't have anything) or whether they're 
actually needed; fetch could do some similar stuff, but couldn't do quite 
that much optimization, because it's not likely the same special case.

Something needs to get the code to figure out the remote HEAD, and it 
shouldn't be "remote" for clone's use, because then clone would take an 
extra connection, since "fetch" obviously has to connect and "remote" 
would run before it and in a separate process.

It also needs to support --reference (list another repository's refs as 
your refs, because you have that repository as an alternate but don't have 
your own refs that use its objects), which is sort of plausible for fetch 
but much less useful.

Aside from that, it's obviously got quite a few lines of option parsing to 
get the right arguments to the things it calls, as well as determining 
whether to check out a branch (depending on what we did with the origin) 
and such.

	-Daniel
*This .sig left intentionally blank*

Re: [RFC] Moving "git remote add --mirror blah" functionality to "git clone --bare --origin=blah"

[Paolo Bonzini]

> *1* In that sense, a more sensible order than rewriting "clone" in C in
> its current form would be to make necessary enhancements to the components
> in this sequence that need to implement clone, figure out how they should
> fit together and first make "clone" a four-liner shell script. Then
> rewriting the result in C may become more trivial.

On the other hand, a C version may have the necessary APIs available to 
support this kind of enhancement, but the APIs may not be there for a 
shell script.

It seems useless to have an option in "git remote add" just because it 
might be necessary in a future refactoring of "git clone", but without a 
good use case beside that one -- because right now "git remote add 
--mirror" is close to useless: anyone who needs it 99% of the time knows 
how to hack the config, unlike people who just want a quick way to 
remotely backup of their repository.

Paolo

Re: [RFC] Moving "git remote add --mirror blah" functionality to "git clone --bare --origin=blah"

[Junio C Hamano]

Paolo Bonzini <bonzini@gnu.org> writes:

> It seems useless to have an option in "git remote add" just because it
> might be necessary in a future refactoring of "git clone", but without
> a good use case beside that one.

Ooo.  But I think that's the other way around.  It was prepared for that
purpose but people never followed through.  Check the archive around
commit 3894439 (Teach "git remote" a mirror mode, 2007-09-02).

> --mirror" is close to useless: anyone who needs it 99% of the time
> knows how to hack the config...

Eh, in that sense, "git remote" itself is useless, isn't it?  I have to
say that this particular point of your argument does not give us any new
or interesting insight to work from.

As I already said, I think "push to backup" option would be useful.  It
just cannot take over --mirror option, which means something different.

Re: [RFC] Moving "git remote add --mirror blah" functionality to "git clone --bare --origin=blah"

[Jeff King]

On Wed, Apr 23, 2008 at 12:56:41PM -0400, Daniel Barkalow wrote:

> Something needs to get the code to figure out the remote HEAD, and it 
> shouldn't be "remote" for clone's use, because then clone would take an 
> extra connection, since "fetch" obviously has to connect and "remote" 
> would run before it and in a separate process.

Actually, having "fetch" figure out the HEAD would be a nice feature,
since then you could track the remote's idea of HEAD as time passes.

AIUI, we could get by with a single connection over git:// with a
protocol extension.

-Peff

Re: [RFC] Moving "git remote add --mirror blah" functionality to "git clone --bare --origin=blah"

[Johannes Schindelin]

Hi,

On Wed, 23 Apr 2008, Junio C Hamano wrote:

> Paolo Bonzini <bonzini@gnu.org> writes:
> 
> > It seems useless to have an option in "git remote add" just because it 
> > might be necessary in a future refactoring of "git clone", but without 
> > a good use case beside that one.
> 
> Ooo.  But I think that's the other way around.

I think it is very much the other way round.  "git remote add" is meant 
for adding _additional_ remotes.  "git clone" just _happens_ to install an 
"origin" remote.

I even have a case where I used "--mirror" _twice_ in the same, _non_-bare 
repository.  It has merit, you know?

Ciao,
Dscho

Re: [RFC] Moving "git remote add --mirror blah" functionality to "git clone --bare --origin=blah"

[Paolo Bonzini]

> Ooo.  But I think that's the other way around.  It was prepared for that
> purpose but people never followed through.  Check the archive around
> commit 3894439 (Teach "git remote" a mirror mode, 2007-09-02).

Well, the thread is not very explicative.  Neither is the commit log 
message.  There is no info as to *why* it is useful.

After finding no particularly good info in the commit, and seeing that 
the functionality is unused within git's shell script porcelain, I wrote 
my RFC on the intuition that it is a failed experiment.  (The rest of 
this message is also assuming this possibly wrong intuition).

>> --mirror" is close to useless: anyone who needs it 99% of the time
>> knows how to hack the config...
> 
> Eh, in that sense, "git remote" itself is useless, isn't it?

Hm, bad sentence on my side.  My point was that my proposed improvements 
to clone:

   git clone --bare -o origin <url> foo.git

satisfy 99% of the usecase; you usually know beforehand if you're 
cloning for local development or for mirrors.   Anyone who *still* needs 
--mirror after that (i.e. needs two mirror-mode remotes, or needs it on 
non-bare repositories), 99% of the time knows how to hack the config. 
Actually in the latter case (non-bare repos) he might have to hack the 
config soon, if Dscho's patch to forbid this is polished and goes in.

In the end, this piece of functionality is almost unused (surely unused 
within git's shell porcelain, if not in the wild) and is dubiously 
placed for two reasons:

1) If the point of git remote is to shield from the config, what the 
user will learn is that "git remote" is about "git branch -r" (maybe 
he'll also learn "refs/remotes/something").  Now instead you have an 
option that goes under "git remote" just because it creates remote.* 
configuration options.  Indeed, "git remote add --mirror -f" will create 
*local* branches, so that "git branch -r" does not show them.

2) Similarly, "--mirror" suggests that the remote will go hand in hand 
with "git push" (that has the "--mirror" option, and this started my 
confusion) rather than "git fetch"/"git pull".  Indeed, even after my 
patch[1] to add some DWIM functionality for mirrors to "git push", the 
user will need to hack the config file manually to activate the DWIM.

Instead, my proposed meaning of "git remote add --mirror" would fix both 
discrepancies...

Paolo

[1] http://marc.info/?l=git&m=120877130932254&w=2