[PATCH v2 0/9] Clarify two uses of remote.*.fetch

[PATCH v2 0/9] Clarify two uses of remote.*.fetch

[Junio C Hamano]

The early part of this has been sent to the list and this round
contains updates based on review comments.  The new patches at the
end clarifies how remote.*.fetch configuration variables are used in
two conceptually different ways.

We would need a similar update for the "git push" side, which uses
remote.*.push configuration variables in the same two different
ways, but that is for a different week.

Junio C Hamano (8):
  fetch doc: update introductory part for clarity
  fetch doc: update note on '+' in front of the refspec
  fetch doc: remove notes on outdated "mixed layout"
  fetch doc: on pulling multiple refspecs
  fetch doc: update refspec format description
  fetch doc: remove "short-cut" section
  fetch doc: add a section on configured remote-tracking branches
  fetch: allow explicit --refmap to override configuration

Marc Branchaud (1):
  fetch doc: move FETCH_HEAD material lower and add an example

 Documentation/fetch-options.txt    |  8 ++++
 Documentation/git-fetch.txt        | 87 ++++++++++++++++++++++++++++++++------
 Documentation/pull-fetch-param.txt | 58 +++++++++----------------
 builtin/fetch.c                    | 35 +++++++++++++--
 t/t5510-fetch.sh                   | 37 ++++++++++++++++
 5 files changed, 171 insertions(+), 54 deletions(-)

-- 
2.0.0-511-g1433423

[PATCH v2 1/9] fetch doc: update introductory part for clarity

[Junio C Hamano]

 - "Branches" is a more common way to say "heads" in these days.

 - Remote-tracking branches are used a lot more these days and it is
   worth mentioning that it is one of the primary side effects of
   the command to update them.

 - Avoid "X. That means Y."  If Y is easier to understand to
   readers, just say that upfront.

 - Use of explicit refspec to fetch tags does not have much to do
   with turning "auto following" on or off.  It is a way to fetch
   tags that otherwise would not be fetched by auto-following.

Helped-by: Marc Branchaud <marcnarc@xiplink.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-fetch.txt | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)

diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index 5809aa4..78fe948 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -17,20 +17,22 @@ SYNOPSIS
 
 DESCRIPTION
 -----------
-Fetches named heads or tags from one or more other repositories,
-along with the objects necessary to complete them.
+Fetch branches and/or tags (collectively, "refs") from one or more
+other repositories, along with the objects necessary to complete
+their histories.
 
-The ref names and their object names of fetched refs are stored
-in `.git/FETCH_HEAD`.  This information is left for a later merge
-operation done by 'git merge'.
+The names of refs that are fetched, together with the object names
+they point at, are written to `.git/FETCH_HEAD`.  This information
+can be used to learn what was fetched.  In addition, the remote-tracking
+branches are updated (see description on <refspec> below for details).
 
-By default, tags are auto-followed.  This means that when fetching
-from a remote, any tags on the remote that point to objects that exist
-in the local repository are fetched.  The effect is to fetch tags that
+By default, any tag that points into the histories being fetched is
+also fetched; the effect is to fetch tags that
 point at branches that you are interested in.  This default behavior
-can be changed by using the --tags or --no-tags options, by
-configuring remote.<name>.tagopt, or by using a refspec that fetches
-tags explicitly.
+can be changed by using the --tags or --no-tags options or by
+configuring remote.<name>.tagopt.  By using a refspec that fetches tags
+explicitly, you can fetch tags that do not point into branches you
+are interested in as well.
 
 'git fetch' can fetch from either a single named repository,
 or from several repositories at once if <group> is given and
-- 
2.0.0-511-g1433423

[PATCH v2 2/9] fetch doc: move FETCH_HEAD material lower and add an example

[Junio C Hamano]

From: Marc Branchaud <marcnarc@xiplink.com>

Signed-off-by: Marc Branchaud <marcnarc@xiplink.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-fetch.txt | 29 +++++++++++++++++++++--------
 1 file changed, 21 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index 78fe948..06106b9 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -18,13 +18,9 @@ SYNOPSIS
 DESCRIPTION
 -----------
 Fetch branches and/or tags (collectively, "refs") from one or more
-other repositories, along with the objects necessary to complete
-their histories.
-
-The names of refs that are fetched, together with the object names
-they point at, are written to `.git/FETCH_HEAD`.  This information
-can be used to learn what was fetched.  In addition, the remote-tracking
-branches are updated (see description on <refspec> below for details).
+other repositories, along with the objects necessary to complete their
+histories.  Remote-tracking branches are updated (see the description
+of <refspec> below for ways to control this behavior).
 
 By default, any tag that points into the histories being fetched is
 also fetched; the effect is to fetch tags that
@@ -34,7 +30,7 @@ configuring remote.<name>.tagopt.  By using a refspec that fetches tags
 explicitly, you can fetch tags that do not point into branches you
 are interested in as well.
 
-'git fetch' can fetch from either a single named repository,
+'git fetch' can fetch from either a single named repository or URL,
 or from several repositories at once if <group> is given and
 there is a remotes.<group> entry in the configuration file.
 (See linkgit:git-config[1]).
@@ -42,6 +38,10 @@ there is a remotes.<group> entry in the configuration file.
 When no remote is specified, by default the `origin` remote will be used,
 unless there's an upstream branch configured for the current branch.
 
+The names of refs that are fetched, together with the object names
+they point at, are written to `.git/FETCH_HEAD`.  This information
+may be used by scripts or other git commands, such as linkgit:git-pull[1].
+
 OPTIONS
 -------
 include::fetch-options.txt[]
@@ -78,6 +78,19 @@ the local repository by fetching from the branches (respectively)
 The `pu` branch will be updated even if it is does not fast-forward,
 because it is prefixed with a plus sign; `tmp` will not be.
 
+* Peek at a remote's branch, without configuring the remote in your local
+repository:
++
+------------------------------------------------
+$ git fetch git://git.kernel.org/pub/scm/git/git.git maint
+$ git log FETCH_HEAD
+------------------------------------------------
++
+The first command fetches the `maint` branch from the repository at
+`git://git.kernel.org/pub/scm/git/git.git` and the second command uses
+`FETCH_HEAD` to examine the branch with linkgit:git-log[1].  The fetched
+objects will eventually be removed by git's built-in housekeeping (see
+linkgit:git-gc[1]).
 
 BUGS
 ----
-- 
2.0.0-511-g1433423

[PATCH v2 3/9] fetch doc: update note on '+' in front of the refspec

[Junio C Hamano]

While it is not *wrong* per-se to say that pulling a rewound/rebased
branch will lead to an unnecessary merge conflict, that is not what
the leading "+" sign to allow non-fast-forward update of remote-tracking
branch is at all.

Helped-by: Marc Branchaud <marcnarc@xiplink.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/pull-fetch-param.txt | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/Documentation/pull-fetch-param.txt b/Documentation/pull-fetch-param.txt
index 18cffc2..41474c5 100644
--- a/Documentation/pull-fetch-param.txt
+++ b/Documentation/pull-fetch-param.txt
@@ -24,15 +24,15 @@ is updated even if it does not result in a fast-forward
 update.
 +
 [NOTE]
-If the remote branch from which you want to pull is
-modified in non-linear ways such as being rewound and
-rebased frequently, then a pull will attempt a merge with
-an older version of itself, likely conflict, and fail.
-It is under these conditions that you would want to use
-the `+` sign to indicate non-fast-forward updates will
-be needed.  There is currently no easy way to determine
-or declare that a branch will be made available in a
-repository with this behavior; the pulling user simply
+When the remote branch you want to fetch is known to
+be rewound and rebased regularly, it is expected that
+its new tip will not be descendant of its previous tip
+(as stored in your remote-tracking branch the last time
+you fetched).  You would want
+to use the `+` sign to indicate non-fast-forward updates
+will be needed for such branches.  There is no way to
+determine or declare that a branch will be made available
+in a repository with this behavior; the pulling user simply
 must know this is the expected usage pattern for a branch.
 +
 [NOTE]
-- 
2.0.0-511-g1433423

Re: [PATCH v2 3/9] fetch doc: update note on '+' in front of the refspec

[Michael Haggerty]

On 06/04/2014 12:16 AM, Junio C Hamano wrote:
> While it is not *wrong* per-se to say that pulling a rewound/rebased
> branch will lead to an unnecessary merge conflict, that is not what
> the leading "+" sign to allow non-fast-forward update of remote-tracking
> branch is at all.
> 
> Helped-by: Marc Branchaud <marcnarc@xiplink.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  Documentation/pull-fetch-param.txt | 18 +++++++++---------
>  1 file changed, 9 insertions(+), 9 deletions(-)
> 
> diff --git a/Documentation/pull-fetch-param.txt b/Documentation/pull-fetch-param.txt
> index 18cffc2..41474c5 100644
> --- a/Documentation/pull-fetch-param.txt
> +++ b/Documentation/pull-fetch-param.txt
> @@ -24,15 +24,15 @@ is updated even if it does not result in a fast-forward
>  update.
>  +
>  [NOTE]
> -If the remote branch from which you want to pull is
> -modified in non-linear ways such as being rewound and
> -rebased frequently, then a pull will attempt a merge with
> -an older version of itself, likely conflict, and fail.
> -It is under these conditions that you would want to use
> -the `+` sign to indicate non-fast-forward updates will
> -be needed.  There is currently no easy way to determine
> -or declare that a branch will be made available in a
> -repository with this behavior; the pulling user simply
> +When the remote branch you want to fetch is known to
> +be rewound and rebased regularly, it is expected that
> +its new tip will not be descendant of its previous tip

s/will not be descendant/will not be a descendant/

to fix a typo, and maybe

s/will not be descendant/will sometimes not be a descendant/

because sometimes it *will* be a descendant.

> +(as stored in your remote-tracking branch the last time
> +you fetched).  You would want
> +to use the `+` sign to indicate non-fast-forward updates
> +will be needed for such branches.  There is no way to
> +determine or declare that a branch will be made available
> +in a repository with this behavior; the pulling user simply
>  must know this is the expected usage pattern for a branch.
>  +
>  [NOTE]
> 


-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

[PATCH v2 4/9] fetch doc: remove notes on outdated "mixed layout"

[Junio C Hamano]

In old days before Git 1.5, it was customery for "git fetch" to use
the same local branch namespace to keep track of the remote-tracking
branches, and it was necessary to tell users not to check them out
and commit on them.  Since everybody uses the separate remote layout
these days, there is no need to warn against the practice to check
out the right-hand side of <refspec> and build on it---the RHS is
typically not even a local branch.

Incidentally, this also kills one mention of "Pull:" line of
$GIT_DIR/remotes/* configuration, which is a lot less familiar to
new people than the more modern remote.*.fetch configuration
variable.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/pull-fetch-param.txt | 13 -------------
 1 file changed, 13 deletions(-)

diff --git a/Documentation/pull-fetch-param.txt b/Documentation/pull-fetch-param.txt
index 41474c5..40f8687 100644
--- a/Documentation/pull-fetch-param.txt
+++ b/Documentation/pull-fetch-param.txt
@@ -36,19 +36,6 @@ in a repository with this behavior; the pulling user simply
 must know this is the expected usage pattern for a branch.
 +
 [NOTE]
-You never do your own development on branches that appear
-on the right hand side of a <refspec> colon on `Pull:` lines;
-they are to be updated by 'git fetch'.  If you intend to do
-development derived from a remote branch `B`, have a `Pull:`
-line to track it (i.e. `Pull: B:remote-B`), and have a separate
-branch `my-B` to do your development on top of it.  The latter
-is created by `git branch my-B remote-B` (or its equivalent `git
-checkout -b my-B remote-B`).  Run `git fetch` to keep track of
-the progress of the remote side, and when you see something new
-on the remote branch, merge it into your development branch with
-`git pull . remote-B`, while you are on `my-B` branch.
-+
-[NOTE]
 There is a difference between listing multiple <refspec>
 directly on 'git pull' command line and having multiple
 `Pull:` <refspec> lines for a <repository> and running
-- 
2.0.0-511-g1433423

[PATCH v2 5/9] fetch doc: on pulling multiple refspecs

[Junio C Hamano]

Replace desription of old-style "Pull:" lines in remotes/
configuration with modern remote.*.fetch variables.

As this note applies only to "git pull", enable it only
in git-pull manual page.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/pull-fetch-param.txt | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/Documentation/pull-fetch-param.txt b/Documentation/pull-fetch-param.txt
index 40f8687..25af2ce 100644
--- a/Documentation/pull-fetch-param.txt
+++ b/Documentation/pull-fetch-param.txt
@@ -34,22 +34,26 @@ will be needed for such branches.  There is no way to
 determine or declare that a branch will be made available
 in a repository with this behavior; the pulling user simply
 must know this is the expected usage pattern for a branch.
+ifdef::git-pull[]
 +
 [NOTE]
 There is a difference between listing multiple <refspec>
 directly on 'git pull' command line and having multiple
-`Pull:` <refspec> lines for a <repository> and running
+`remote.<repository>.fetch` entries in your configuration
+for a <repository> and running
 'git pull' command without any explicit <refspec> parameters.
 <refspec> listed explicitly on the command line are always
 merged into the current branch after fetching.  In other words,
 if you list more than one remote refs, you would be making
-an Octopus.  While 'git pull' run without any explicit <refspec>
-parameter takes default <refspec>s from `Pull:` lines, it
+an Octopus merge. On the other hand, 'git pull' that is run
+without any explicit <refspec> parameter takes default
+<refspec>s from `remote.<repository>.fetch` configuration, it
 merges only the first <refspec> found into the current branch,
-after fetching all the remote refs.  This is because making an
+after fetching all the remote refs specified.  This is because making an
 Octopus from remote refs is rarely done, while keeping track
 of multiple remote heads in one-go by fetching more than one
 is often useful.
+endif::git-pull[]
 +
 Some short-cut notations are also supported.
 +
-- 
2.0.0-511-g1433423

Re: [PATCH v2 5/9] fetch doc: on pulling multiple refspecs

[Marc Branchaud]

On 14-06-03 06:16 PM, Junio C Hamano wrote:
> Replace desription of old-style "Pull:" lines in remotes/
> configuration with modern remote.*.fetch variables.
> 
> As this note applies only to "git pull", enable it only
> in git-pull manual page.
> 
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  Documentation/pull-fetch-param.txt | 12 ++++++++----
>  1 file changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/Documentation/pull-fetch-param.txt b/Documentation/pull-fetch-param.txt
> index 40f8687..25af2ce 100644
> --- a/Documentation/pull-fetch-param.txt
> +++ b/Documentation/pull-fetch-param.txt
> @@ -34,22 +34,26 @@ will be needed for such branches.  There is no way to
>  determine or declare that a branch will be made available
>  in a repository with this behavior; the pulling user simply
>  must know this is the expected usage pattern for a branch.
> +ifdef::git-pull[]
>  +
>  [NOTE]
>  There is a difference between listing multiple <refspec>
>  directly on 'git pull' command line and having multiple
> -`Pull:` <refspec> lines for a <repository> and running
> +`remote.<repository>.fetch` entries in your configuration
> +for a <repository> and running

s/running/running a/

>  'git pull' command without any explicit <refspec> parameters.
>  <refspec> listed explicitly on the command line are always

s/<refspec>/<refspec>s/

>  merged into the current branch after fetching.  In other words,
>  if you list more than one remote refs, you would be making

s/refs, you would be making/ref, 'git pull' will create/

> -an Octopus.  While 'git pull' run without any explicit <refspec>
> -parameter takes default <refspec>s from `Pull:` lines, it
> +an Octopus merge. On the other hand, 'git pull' that is run
> +without any explicit <refspec> parameter takes default
> +<refspec>s from `remote.<repository>.fetch` configuration, it
>  merges only the first <refspec> found into the current branch,
> -after fetching all the remote refs.  This is because making an
> +after fetching all the remote refs specified.  This is because making an

That "On the other hand" sentence is hard to parse.  How about

	On the other hand, if you do not list any remote refs, 'git pull'
	will fetch all the <refspec>s it finds in the
	`remote.<repository>.fetch` configuration and merge only the
	first <refspec> found into the current branch.

Actually, "first <refspec> found" is also wrong, isn't it?  I'm not sure I
can craft a succinct summary of pull's merge behaviour for this note!

>  Octopus from remote refs is rarely done, while keeping track
>  of multiple remote heads in one-go by fetching more than one
>  is often useful.
> +endif::git-pull[]
>  +
>  Some short-cut notations are also supported.
>  +

Hmmm, in my 2.0 man page output (asciidoc 8.6.9 on Ubuntu) there's no empty
line between the end of the note and the "Some short-cut notations" line,
which I think is inconsistent with the rest of the formatting.  The HTML
version looks fine though.  Is there some asciidoc-ese that would insert a
line there for the man format?

		M.

[PATCH v2 6/9] fetch doc: update refspec format description

[Junio C Hamano]

The text made it sound as if the leading plus is the only thing that
is optional, and forgot that <lhs> is the same as <lhs>:, i.e. fetch
it and do not store anywhere.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/pull-fetch-param.txt | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Documentation/pull-fetch-param.txt b/Documentation/pull-fetch-param.txt
index 25af2ce..a090d4a 100644
--- a/Documentation/pull-fetch-param.txt
+++ b/Documentation/pull-fetch-param.txt
@@ -15,6 +15,7 @@ endif::git-pull[]
 	The format of a <refspec> parameter is an optional plus
 	`+`, followed by the source ref <src>, followed
 	by a colon `:`, followed by the destination ref <dst>.
+	The colon can be omitted when <dst> is empty.
 +
 The remote ref that matches <src>
 is fetched, and if <dst> is not empty string, the local
-- 
2.0.0-511-g1433423

[PATCH v2 7/9] fetch doc: remove "short-cut" section

[Junio C Hamano]

It is misleading to mention that <ref> that does not store is to
fetch the ref into FETCH_HEAD, because a refspec that does store is
also to fetch the LHS into FETCH_HEAD.  It is doubly misleading to
list it as part of "short-cut".  <ref> stands for a refspec that has
it on the LHS with a colon and an empty RHS, and that definition
should be given at the beginning of the entry where the format is
defined.

Tentatively remove this misleading description, which leaves the
`tag <tag>` as the only true short-hand, so move it at the beginning
of the entry.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/pull-fetch-param.txt | 16 +++-------------
 1 file changed, 3 insertions(+), 13 deletions(-)

diff --git a/Documentation/pull-fetch-param.txt b/Documentation/pull-fetch-param.txt
index a090d4a..9e62434 100644
--- a/Documentation/pull-fetch-param.txt
+++ b/Documentation/pull-fetch-param.txt
@@ -17,6 +17,9 @@ endif::git-pull[]
 	by a colon `:`, followed by the destination ref <dst>.
 	The colon can be omitted when <dst> is empty.
 +
+`tag <tag>` means the same as `refs/tags/<tag>:refs/tags/<tag>`;
+it requests fetching everything up to the given tag.
++
 The remote ref that matches <src>
 is fetched, and if <dst> is not empty string, the local
 ref that matches it is fast-forwarded using <src>.
@@ -55,16 +58,3 @@ Octopus from remote refs is rarely done, while keeping track
 of multiple remote heads in one-go by fetching more than one
 is often useful.
 endif::git-pull[]
-+
-Some short-cut notations are also supported.
-+
-* `tag <tag>` means the same as `refs/tags/<tag>:refs/tags/<tag>`;
-  it requests fetching everything up to the given tag.
-ifndef::git-pull[]
-* A parameter <ref> without a colon fetches that ref into FETCH_HEAD,
-endif::git-pull[]
-ifdef::git-pull[]
-* A parameter <ref> without a colon merges <ref> into the current
-  branch,
-endif::git-pull[]
-  and updates the remote-tracking branches (if any).
-- 
2.0.0-511-g1433423

Re: [PATCH v2 7/9] fetch doc: remove "short-cut" section

[Marc Branchaud]

On 14-06-03 06:16 PM, Junio C Hamano wrote:
> It is misleading to mention that <ref> that does not store is to
> fetch the ref into FETCH_HEAD, because a refspec that does store is
> also to fetch the LHS into FETCH_HEAD.  It is doubly misleading to
> list it as part of "short-cut".  <ref> stands for a refspec that has
> it on the LHS with a colon and an empty RHS, and that definition
> should be given at the beginning of the entry where the format is
> defined.
> 
> Tentatively remove this misleading description, which leaves the
> `tag <tag>` as the only true short-hand, so move it at the beginning
> of the entry.

Well that neatly solves the missing empty line problem...  :)

		M.

[PATCH v2 8/9] fetch doc: add a section on configured remote-tracking branches

[Junio C Hamano]

To resurrect a misleading mention removed in the previous step,
add a section to explain how the remote-tracking configuration
interacts with the refspecs given as the command-line arguments.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-fetch.txt | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index 06106b9..d09736a 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -51,6 +51,49 @@ include::pull-fetch-param.txt[]
 include::urls-remotes.txt[]
 
 
+CONFIGURED REMOTE-TRACKING BRANCHES
+-----------------------------------
+
+You would often interact with the same remote repository by
+regularly and repeatedly fetching from it.  In order to keep track
+of the progress of such a remote repository, `git fetch` allows you
+to configure `remote.<repository>.fetch` configuration variable.
+
+Typically such a variable may look like this:
+
+------------------------------------------------
+[remote "origin"]
+	fetch = +refs/heads/*:refs/remotes/origin/*
+------------------------------------------------
+
+This configuration is used in two ways:
+
+* When `git fetch` command is run without specifying what branches
+  and/or tags to fetch on the command line, e.g. `git fetch origin`
+  or `git fetch`, the values configured to this variable are used as
+  the refspecs to be used to fetch.  The example above will fetch
+  all branches that exist on the `origin` (i.e. any ref that matches
+  the left-hand side of the value, `refs/heads/*`) and update the
+  corresponding remote-tracking branches in `refs/remotes/origin/*`
+  hierarchy.
+
+* When `git fetch` command is run with explicit branches and/or tags
+  to fetch on the command line, e.g. `git fetch origin master`, the
+  <refspec> given on the command line (e.g. `master` in the example,
+  which is a short-hand for `master:`, which in turn would mean
+  "fetch the 'master' branch but I do not explicitly say what
+  remote-tracking branch to update with it from the command line")
+  determines what are to be fetched, and the example command will
+  fetch _only_ the 'master' branch.  The values of the variable are
+  used to map the branch (i.e. `master`) to determine which
+  remote-tracking branch, if any, is updated.  When used in this
+  way, the values of the configuration variable do not have any
+  effect in deciding _what_ gets fetched (i.e. the values are not
+  used as refspecs when the command-line lists refspecs); they are
+  only used to decide _where_ the refs that are fetched are stored
+  by acting as a mapping.
+
+
 EXAMPLES
 --------
 
-- 
2.0.0-511-g1433423

Re: [PATCH v2 8/9] fetch doc: add a section on configured remote-tracking branches

[Marc Branchaud]

On 14-06-03 06:16 PM, Junio C Hamano wrote:
> To resurrect a misleading mention removed in the previous step,
> add a section to explain how the remote-tracking configuration
> interacts with the refspecs given as the command-line arguments.
> 
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  Documentation/git-fetch.txt | 43 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 43 insertions(+)
> 
> diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
> index 06106b9..d09736a 100644
> --- a/Documentation/git-fetch.txt
> +++ b/Documentation/git-fetch.txt
> @@ -51,6 +51,49 @@ include::pull-fetch-param.txt[]
>  include::urls-remotes.txt[]
>  
>  
> +CONFIGURED REMOTE-TRACKING BRANCHES
> +-----------------------------------
> +
> +You would often interact with the same remote repository by

s/would//

> +regularly and repeatedly fetching from it.  In order to keep track
> +of the progress of such a remote repository, `git fetch` allows you
> +to configure `remote.<repository>.fetch` configuration variable.

/variable/variables/

> +
> +Typically such a variable may look like this:
> +
> +------------------------------------------------
> +[remote "origin"]
> +	fetch = +refs/heads/*:refs/remotes/origin/*
> +------------------------------------------------
> +
> +This configuration is used in two ways:
> +
> +* When `git fetch` command is run without specifying what branches

s/command//

> +  and/or tags to fetch on the command line, e.g. `git fetch origin`
> +  or `git fetch`, the values configured to this variable are used as

s/values configured to this variable/`remote.<repository>.fetch` values/

> +  the refspecs to be used to fetch.  The example above will fetch

/to be used//

> +  all branches that exist on the `origin` (i.e. any ref that matches

s/on/in/

> +  the left-hand side of the value, `refs/heads/*`) and update the
> +  corresponding remote-tracking branches in `refs/remotes/origin/*`

s/in/in the/

> +  hierarchy.
> +
> +* When `git fetch` command is run with explicit branches and/or tags

s/command//

> +  to fetch on the command line, e.g. `git fetch origin master`, the
> +  <refspec> given on the command line (e.g. `master` in the example,
> +  which is a short-hand for `master:`, which in turn would mean
> +  "fetch the 'master' branch but I do not explicitly say what
> +  remote-tracking branch to update with it from the command line")
> +  determines what are to be fetched, and the example command will

Change "determines what are to be fetched" to "determines what gets fetched"
and move the phrase to before the parenthetical comment.

> +  fetch _only_ the 'master' branch.  The values of the variable are

s/values of the variable/`remote.<repository>.fetch` values/

> +  used to map the branch (i.e. `master`) to determine which

s/used to map the branch (i.e. `master`) to//

> +  remote-tracking branch, if any, is updated.  When used in this
> +  way, the values of the configuration variable do not have any

s/values of the configuration variable/`remote.<repository>.fetch` values/

		M.


> +  effect in deciding _what_ gets fetched (i.e. the values are not
> +  used as refspecs when the command-line lists refspecs); they are
> +  only used to decide _where_ the refs that are fetched are stored
> +  by acting as a mapping.
> +
> +
>  EXAMPLES
>  --------
>  
> 

Re: [PATCH v2 8/9] fetch doc: add a section on configured remote-tracking branches

[Junio C Hamano]

Marc Branchaud <mbranchaud@xiplink.com> writes:

[jc: omitted good suggestions I'll use in amending]

>> +  the refspecs to be used to fetch.  The example above will fetch
>
> /to be used//

I have a problem with that change, actually, because you do not
"fetch" refspec from anywhere.  A refspec is what is used to
determine what histories to fetch (i.e. left-hand side of it before
the colon) and which local refs to update with what is fetched
(i.e. right-hand side of it after the colon), and this description
of the traditional behaviour is meant to highlight the difference
from the second usage, which is relatively new since f2690487
(fetch: opportunistically update tracking refs, 2013-05-11),
i.e. how the variable is *not* used as a refspec when the command
line already has one.

Perhaps

    ... `remote.<repository>.fetch` values are used as the refspecs,
    i.e. they specify what refs to fetch and what local refs to
    update.

or something?

Re: [PATCH v2 8/9] fetch doc: add a section on configured remote-tracking branches

[Marc Branchaud]

On 14-06-04 06:17 PM, Junio C Hamano wrote:
> Marc Branchaud <mbranchaud@xiplink.com> writes:
> 
> [jc: omitted good suggestions I'll use in amending]
> 
>>> +  the refspecs to be used to fetch.  The example above will fetch
>>
>> /to be used//
> 
> I have a problem with that change, actually, because you do not
> "fetch" refspec from anywhere.  A refspec is what is used to
> determine what histories to fetch (i.e. left-hand side of it before
> the colon) and which local refs to update with what is fetched
> (i.e. right-hand side of it after the colon), and this description
> of the traditional behaviour is meant to highlight the difference
> from the second usage, which is relatively new since f2690487
> (fetch: opportunistically update tracking refs, 2013-05-11),
> i.e. how the variable is *not* used as a refspec when the command
> line already has one.
> 
> Perhaps
> 
>     ... `remote.<repository>.fetch` values are used as the refspecs,
>     i.e. they specify what refs to fetch and what local refs to
>     update.
> 
> or something?

s/what/which/ and I think that would be fine.

		M.

[PATCH] docs: Explain the purpose of fetch's and pull's <refspec> parameter.

[Marc Branchaud]

Signed-off-by: Marc Branchaud <marcnarc@xiplink.com>
---
 Documentation/git-fetch.txt        |  4 ++--
 Documentation/pull-fetch-param.txt | 17 ++++++++++++++---
 2 files changed, 16 insertions(+), 5 deletions(-)

On 14-06-04 06:17 PM, Junio C Hamano wrote:
> 
> Perhaps
> 
>     ... `remote.<repository>.fetch` values are used as the refspecs,
>     i.e. they specify what refs to fetch and what local refs to
>     update.
> 
> or something?

In fact, I like that so much I think it should be *the* description of
the <refspec> parameter.  Much better than just jumping straight into
the syntax.

This patch applies atop your 8/9.  I feel strongly that some kind of
reference should accompany this description, and your new CONFIGURED
REMOTE-TRACKING BRANCHES section seems like a good one for the fetch
variant, but since pull's variant doesn't have that section I just
made it link to fetch's doc.

(Also, I'm not sure if "CRTB" is a good link ID for your new section.)

		M.

diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index 4a735ab..a4cafa3 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -49,8 +49,8 @@ include::pull-fetch-param.txt[]
 include::urls-remotes.txt[]
 
 
-CONFIGURED REMOTE-TRACKING BRANCHES
------------------------------------
+CONFIGURED REMOTE-TRACKING BRANCHES[[CRTB]]
+-------------------------------------------
 
 You would often interact with the same remote repository by
 regularly and repeatedly fetching from it.  In order to keep track
diff --git a/Documentation/pull-fetch-param.txt b/Documentation/pull-fetch-param.txt
index 18cffc2..40304c6 100644
--- a/Documentation/pull-fetch-param.txt
+++ b/Documentation/pull-fetch-param.txt
@@ -12,9 +12,20 @@ ifndef::git-pull[]
 endif::git-pull[]
 
 <refspec>::
-	The format of a <refspec> parameter is an optional plus
-	`+`, followed by the source ref <src>, followed
-	by a colon `:`, followed by the destination ref <dst>.
+	Specifies which refs to fetch and which local refs to update.
+	<refspec> parameters are not normally specified on the command
+	line, but instead are read from `remote.<repository>.fetch`
+	configuration variables
+ifndef::git-pull[]
+	(see <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> below).
+endif::git-pull[]
+ifdef::git-pull[]
+	(see linkgit:git-fetch[1]).
+endif::git-pull[]
++
+The format of a <refspec> parameter is an optional plus
+`+`, followed by the source ref <src>, followed
+by a colon `:`, followed by the destination ref <dst>.
 +
 The remote ref that matches <src>
 is fetched, and if <dst> is not empty string, the local
-- 
2.0.0.dirty

Re: [PATCH] docs: Explain the purpose of fetch's and pull's <refspec> parameter.

[Junio C Hamano]

Marc Branchaud <marcnarc@xiplink.com> writes:

> This patch applies atop your 8/9.  I feel strongly that some kind of
> reference should accompany this description, and your new CONFIGURED
> REMOTE-TRACKING BRANCHES section seems like a good one for the fetch
> variant, but since pull's variant doesn't have that section I just
> made it link to fetch's doc.
>
> (Also, I'm not sure if "CRTB" is a good link ID for your new section.)

Nobody looks at these ids, hopefully ;-)

> diff --git a/Documentation/pull-fetch-param.txt b/Documentation/pull-fetch-param.txt
> index 18cffc2..40304c6 100644
> --- a/Documentation/pull-fetch-param.txt
> +++ b/Documentation/pull-fetch-param.txt
> @@ -12,9 +12,20 @@ ifndef::git-pull[]
>  endif::git-pull[]
>  
>  <refspec>::
> -	The format of a <refspec> parameter is an optional plus
> -	`+`, followed by the source ref <src>, followed
> -	by a colon `:`, followed by the destination ref <dst>.
> +	Specifies which refs to fetch and which local refs to update.

That is an improvement.  We should first say what it is and what it
is for before saying how you spell it and the above change is
exactly that.

> +	<refspec> parameters are not normally specified on the command
> +	line, but instead are read from `remote.<repository>.fetch`

I however am not sure if this is an improvement, especially the
"normally" part.  Those who respond to a git-pull-request output
might be fewer than those who send pull requests, but that does not
mean they are abnormal.

	The command line often omit <refspec> parameters when
	fetching or pulling from a remote you regularly interact
	with, in which case `remote.<repository>.fetch` values are
	used instead.

would be OK, though.

Later today I'll push out the series on 'pu' after amending them
with your comments so far.  It would be nice if you can reroll this
on top of the updated one ("git log --oneline --first-parent
master..pu" and find jc/fetch-pull-refmap in there).

Thanks.

[PATCH] docs: Explain the purpose of fetch's and pull's <refspec> parameter.

[Marc Branchaud]

Signed-off-by: Marc Branchaud <marcnarc@xiplink.com>
---
 Documentation/git-fetch.txt        |  4 ++--
 Documentation/pull-fetch-param.txt | 18 ++++++++++++++----
 2 files changed, 16 insertions(+), 6 deletions(-)

On 14-06-05 06:12 PM, Junio C Hamano wrote:
> 
>> +	<refspec> parameters are not normally specified on the command
>> +	line, but instead are read from `remote.<repository>.fetch`
> 
> I however am not sure if this is an improvement, especially the
> "normally" part.  Those who respond to a git-pull-request output
> might be fewer than those who send pull requests, but that does not
> mean they are abnormal.
> 
> 	The command line often omit <refspec> parameters when
> 	fetching or pulling from a remote you regularly interact
> 	with, in which case `remote.<repository>.fetch` values are
> 	used instead.
> 
> would be OK, though.
> 
> Later today I'll push out the series on 'pu' after amending them
> with your comments so far.  It would be nice if you can reroll this
> on top of the updated one ("git log --oneline --first-parent
> master..pu" and find jc/fetch-pull-refmap in there).

Here's the reroll.  I used completely different phrasing but I think
I still captured the gist of what you said.

(Apologies for the delay -- I was looking for jc/fetch-pull-refmap on pu,
not jc/fetch-push-refmap on next!)

		M.


diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index 629a1cf..8deb614 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -51,8 +51,8 @@ include::pull-fetch-param.txt[]
 include::urls-remotes.txt[]
 
 
-CONFIGURED REMOTE-TRACKING BRANCHES
------------------------------------
+CONFIGURED REMOTE-TRACKING BRANCHES[[CRTB]]
+-------------------------------------------
 
 You often interact with the same remote repository by
 regularly and repeatedly fetching from it.  In order to keep track
diff --git a/Documentation/pull-fetch-param.txt b/Documentation/pull-fetch-param.txt
index 4bff65b..1ebbf1d 100644
--- a/Documentation/pull-fetch-param.txt
+++ b/Documentation/pull-fetch-param.txt
@@ -12,10 +12,20 @@ ifndef::git-pull[]
 endif::git-pull[]
 
 <refspec>::
-	The format of a <refspec> parameter is an optional plus
-	`+`, followed by the source ref <src>, followed
-	by a colon `:`, followed by the destination ref <dst>.
-	The colon can be omitted when <dst> is empty.
+	Specifies which refs to fetch and which local refs to update.
+	When no <refspec>s appear on the command line, the refs to fetch
+	are read from `remote.<repository>.fetch` variables instead
+ifndef::git-pull[]
+	(see <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> below).
+endif::git-pull[]
+ifdef::git-pull[]
+	(see linkgit:git-fetch[1]).
+endif::git-pull[]
++
+The format of a <refspec> parameter is an optional plus
+`+`, followed by the source ref <src>, followed
+by a colon `:`, followed by the destination ref <dst>.
+The colon can be omitted when <dst> is empty.
 +
 `tag <tag>` means the same as `refs/tags/<tag>:refs/tags/<tag>`;
 it requests fetching everything up to the given tag.
-- 
2.0.0.dirty

[PATCH v2 9/9] fetch: allow explicit --refmap to override configuration

[Junio C Hamano]

Since the introduction of opportunisitic updates of remote-tracking
branches, started at around f2690487 (fetch: opportunistically
update tracking refs, 2013-05-11) with a few updates in v1.8.4 era,
the remote.*.fetch configuration always kicks in even when a refspec
to specify what to fetch is given on the command line, and there is
no way to disable or override it per-invocation.

Teach the command to pay attention to the --refmap=<lhs>:<rhs>
command-line options that can be used to override the use of
configured remote.*.fetch as the refmap.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
---
 Documentation/fetch-options.txt |  8 ++++++++
 Documentation/git-fetch.txt     |  3 +++
 builtin/fetch.c                 | 35 ++++++++++++++++++++++++++++++++---
 t/t5510-fetch.sh                | 37 +++++++++++++++++++++++++++++++++++++
 4 files changed, 80 insertions(+), 3 deletions(-)

diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 92c68c3..d5c6289 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -72,6 +72,14 @@ endif::git-pull[]
 	setting. See linkgit:git-config[1].
 
 ifndef::git-pull[]
+--refmap=<refspec>::
+	When fetching refs listed on the command line, use the
+	specified refspec (can be given more than once) to map the
+	refs to remote-tracking branches, instead of values of
+	`remote.*.fetch` configuration variable for the remote
+	repository.  See section on "Configured Remote-tracking
+	Branches" for details.
+
 -t::
 --tags::
 	Fetch all tags from the remote (i.e., fetch remote tags
diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index d09736a..96c44f9 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -93,6 +93,9 @@ This configuration is used in two ways:
   only used to decide _where_ the refs that are fetched are stored
   by acting as a mapping.
 
+The latter use of the configured values can be overridden by giving
+the `--refmap=<refspec>` parameter(s) on the command line.
+
 
 EXAMPLES
 --------
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 55f457c..dd46b61 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -45,6 +45,8 @@ static struct transport *gsecondary;
 static const char *submodule_prefix = "";
 static const char *recurse_submodules_default;
 static int shown_url = 0;
+static int refmap_alloc, refmap_nr;
+static const char **refmap_array;
 
 static int option_parse_recurse_submodules(const struct option *opt,
 				   const char *arg, int unset)
@@ -69,6 +71,19 @@ static int git_fetch_config(const char *k, const char *v, void *cb)
 	return 0;
 }
 
+static int parse_refmap_arg(const struct option *opt, const char *arg, int unset)
+{
+	ALLOC_GROW(refmap_array, refmap_nr + 1, refmap_alloc);
+
+	/*
+	 * "git fetch --refmap='' origin foo"
+	 * can be used to tell the command not to store anywhere
+	 */
+	if (*arg)
+		refmap_array[refmap_nr++] = arg;
+	return 0;
+}
+
 static struct option builtin_fetch_options[] = {
 	OPT__VERBOSITY(&verbosity),
 	OPT_BOOL(0, "all", &all,
@@ -107,6 +122,8 @@ static struct option builtin_fetch_options[] = {
 		   N_("default mode for recursion"), PARSE_OPT_HIDDEN },
 	OPT_BOOL(0, "update-shallow", &update_shallow,
 		 N_("accept refs that update .git/shallow")),
+	{ OPTION_CALLBACK, 0, "refmap", NULL, N_("refmap"),
+	  N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg },
 	OPT_END()
 };
 
@@ -278,6 +295,9 @@ static struct ref *get_ref_map(struct transport *transport,
 	const struct ref *remote_refs = transport_get_remote_refs(transport);
 
 	if (refspec_count) {
+		struct refspec *fetch_refspec;
+		int fetch_refspec_nr;
+
 		for (i = 0; i < refspec_count; i++) {
 			get_fetch_map(remote_refs, &refspecs[i], &tail, 0);
 			if (refspecs[i].dst && refspecs[i].dst[0])
@@ -307,12 +327,21 @@ static struct ref *get_ref_map(struct transport *transport,
 		 * by ref_remove_duplicates() in favor of one of these
 		 * opportunistic entries with FETCH_HEAD_IGNORE.
 		 */
-		for (i = 0; i < transport->remote->fetch_refspec_nr; i++)
-			get_fetch_map(ref_map, &transport->remote->fetch[i],
-				      &oref_tail, 1);
+		if (refmap_array) {
+			fetch_refspec = parse_fetch_refspec(refmap_nr, refmap_array);
+			fetch_refspec_nr = refmap_nr;
+		} else {
+			fetch_refspec = transport->remote->fetch;
+			fetch_refspec_nr = transport->remote->fetch_refspec_nr;
+		}
+
+		for (i = 0; i < fetch_refspec_nr; i++)
+			get_fetch_map(ref_map, &fetch_refspec[i], &oref_tail, 1);
 
 		if (tags == TAGS_SET)
 			get_fetch_map(remote_refs, tag_refspec, &tail, 0);
+	} else if (refmap_array) {
+		die("--refmap option is only meaningful with command-line refspec(s).");
 	} else {
 		/* Use the defaults */
 		struct remote *remote = transport->remote;
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 29d59ef..d78f320 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -447,6 +447,43 @@ test_expect_success 'explicit pull should update tracking' '
 	)
 '
 
+test_expect_success 'explicit --refmap is allowed only with command-line refspec' '
+	cd "$D" &&
+	(
+		cd three &&
+		test_must_fail git fetch --refmap="*:refs/remotes/none/*"
+	)
+'
+
+test_expect_success 'explicit --refmap option overrides remote.*.fetch' '
+	cd "$D" &&
+	git branch -f side &&
+	(
+		cd three &&
+		git update-ref refs/remotes/origin/master base-origin-master &&
+		o=$(git rev-parse --verify refs/remotes/origin/master) &&
+		git fetch --refmap="refs/heads/*:refs/remotes/other/*" origin master &&
+		n=$(git rev-parse --verify refs/remotes/origin/master) &&
+		test "$o" = "$n" &&
+		test_must_fail git rev-parse --verify refs/remotes/origin/side &&
+		git rev-parse --verify refs/remotes/other/master
+	)
+'
+
+test_expect_success 'explicitly empty --refmap option disables remote.*.fetch' '
+	cd "$D" &&
+	git branch -f side &&
+	(
+		cd three &&
+		git update-ref refs/remotes/origin/master base-origin-master &&
+		o=$(git rev-parse --verify refs/remotes/origin/master) &&
+		git fetch --refmap="" origin master &&
+		n=$(git rev-parse --verify refs/remotes/origin/master) &&
+		test "$o" = "$n" &&
+		test_must_fail git rev-parse --verify refs/remotes/origin/side
+	)
+'
+
 test_expect_success 'configured fetch updates tracking' '
 
 	cd "$D" &&
-- 
2.0.0-511-g1433423

Re: [PATCH v2 9/9] fetch: allow explicit --refmap to override configuration

[Marc Branchaud]

On 14-06-03 06:16 PM, Junio C Hamano wrote:
> Since the introduction of opportunisitic updates of remote-tracking
> branches, started at around f2690487 (fetch: opportunistically
> update tracking refs, 2013-05-11) with a few updates in v1.8.4 era,
> the remote.*.fetch configuration always kicks in even when a refspec
> to specify what to fetch is given on the command line, and there is
> no way to disable or override it per-invocation.
> 
> Teach the command to pay attention to the --refmap=<lhs>:<rhs>
> command-line options that can be used to override the use of
> configured remote.*.fetch as the refmap.

(Your 0/9 message merely said "The new patches at the
end clarifies how remote.*.fetch configuration variables are used in
two conceptually different ways." so I was not expecting fetch to get a new
option.)

> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> ---
>  Documentation/fetch-options.txt |  8 ++++++++
>  Documentation/git-fetch.txt     |  3 +++
>  builtin/fetch.c                 | 35 ++++++++++++++++++++++++++++++++---
>  t/t5510-fetch.sh                | 37 +++++++++++++++++++++++++++++++++++++
>  4 files changed, 80 insertions(+), 3 deletions(-)
> 
> diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
> index 92c68c3..d5c6289 100644
> --- a/Documentation/fetch-options.txt
> +++ b/Documentation/fetch-options.txt
> @@ -72,6 +72,14 @@ endif::git-pull[]
>  	setting. See linkgit:git-config[1].
>  
>  ifndef::git-pull[]
> +--refmap=<refspec>::
> +	When fetching refs listed on the command line, use the
> +	specified refspec (can be given more than once) to map the
> +	refs to remote-tracking branches, instead of values of

s/values/the values/

> +	`remote.*.fetch` configuration variable for the remote

s/variable/variables/

> +	repository.  See section on "Configured Remote-tracking
> +	Branches" for details.
> +
>  -t::
>  --tags::
>  	Fetch all tags from the remote (i.e., fetch remote tags
> diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
> index d09736a..96c44f9 100644
> --- a/Documentation/git-fetch.txt
> +++ b/Documentation/git-fetch.txt
> @@ -93,6 +93,9 @@ This configuration is used in two ways:
>    only used to decide _where_ the refs that are fetched are stored
>    by acting as a mapping.
>  
> +The latter use of the configured values can be overridden by giving

For consistency with my suggestions on 8/9:

s/configured values/`remote.<repository>.fetch` values/

		M.

> +the `--refmap=<refspec>` parameter(s) on the command line.
> +
>  
>  EXAMPLES
>  --------
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index 55f457c..dd46b61 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -45,6 +45,8 @@ static struct transport *gsecondary;
>  static const char *submodule_prefix = "";
>  static const char *recurse_submodules_default;
>  static int shown_url = 0;
> +static int refmap_alloc, refmap_nr;
> +static const char **refmap_array;
>  
>  static int option_parse_recurse_submodules(const struct option *opt,
>  				   const char *arg, int unset)
> @@ -69,6 +71,19 @@ static int git_fetch_config(const char *k, const char *v, void *cb)
>  	return 0;
>  }
>  
> +static int parse_refmap_arg(const struct option *opt, const char *arg, int unset)
> +{
> +	ALLOC_GROW(refmap_array, refmap_nr + 1, refmap_alloc);
> +
> +	/*
> +	 * "git fetch --refmap='' origin foo"
> +	 * can be used to tell the command not to store anywhere
> +	 */
> +	if (*arg)
> +		refmap_array[refmap_nr++] = arg;
> +	return 0;
> +}
> +
>  static struct option builtin_fetch_options[] = {
>  	OPT__VERBOSITY(&verbosity),
>  	OPT_BOOL(0, "all", &all,
> @@ -107,6 +122,8 @@ static struct option builtin_fetch_options[] = {
>  		   N_("default mode for recursion"), PARSE_OPT_HIDDEN },
>  	OPT_BOOL(0, "update-shallow", &update_shallow,
>  		 N_("accept refs that update .git/shallow")),
> +	{ OPTION_CALLBACK, 0, "refmap", NULL, N_("refmap"),
> +	  N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg },
>  	OPT_END()
>  };
>  
> @@ -278,6 +295,9 @@ static struct ref *get_ref_map(struct transport *transport,
>  	const struct ref *remote_refs = transport_get_remote_refs(transport);
>  
>  	if (refspec_count) {
> +		struct refspec *fetch_refspec;
> +		int fetch_refspec_nr;
> +
>  		for (i = 0; i < refspec_count; i++) {
>  			get_fetch_map(remote_refs, &refspecs[i], &tail, 0);
>  			if (refspecs[i].dst && refspecs[i].dst[0])
> @@ -307,12 +327,21 @@ static struct ref *get_ref_map(struct transport *transport,
>  		 * by ref_remove_duplicates() in favor of one of these
>  		 * opportunistic entries with FETCH_HEAD_IGNORE.
>  		 */
> -		for (i = 0; i < transport->remote->fetch_refspec_nr; i++)
> -			get_fetch_map(ref_map, &transport->remote->fetch[i],
> -				      &oref_tail, 1);
> +		if (refmap_array) {
> +			fetch_refspec = parse_fetch_refspec(refmap_nr, refmap_array);
> +			fetch_refspec_nr = refmap_nr;
> +		} else {
> +			fetch_refspec = transport->remote->fetch;
> +			fetch_refspec_nr = transport->remote->fetch_refspec_nr;
> +		}
> +
> +		for (i = 0; i < fetch_refspec_nr; i++)
> +			get_fetch_map(ref_map, &fetch_refspec[i], &oref_tail, 1);
>  
>  		if (tags == TAGS_SET)
>  			get_fetch_map(remote_refs, tag_refspec, &tail, 0);
> +	} else if (refmap_array) {
> +		die("--refmap option is only meaningful with command-line refspec(s).");
>  	} else {
>  		/* Use the defaults */
>  		struct remote *remote = transport->remote;
> diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
> index 29d59ef..d78f320 100755
> --- a/t/t5510-fetch.sh
> +++ b/t/t5510-fetch.sh
> @@ -447,6 +447,43 @@ test_expect_success 'explicit pull should update tracking' '
>  	)
>  '
>  
> +test_expect_success 'explicit --refmap is allowed only with command-line refspec' '
> +	cd "$D" &&
> +	(
> +		cd three &&
> +		test_must_fail git fetch --refmap="*:refs/remotes/none/*"
> +	)
> +'
> +
> +test_expect_success 'explicit --refmap option overrides remote.*.fetch' '
> +	cd "$D" &&
> +	git branch -f side &&
> +	(
> +		cd three &&
> +		git update-ref refs/remotes/origin/master base-origin-master &&
> +		o=$(git rev-parse --verify refs/remotes/origin/master) &&
> +		git fetch --refmap="refs/heads/*:refs/remotes/other/*" origin master &&
> +		n=$(git rev-parse --verify refs/remotes/origin/master) &&
> +		test "$o" = "$n" &&
> +		test_must_fail git rev-parse --verify refs/remotes/origin/side &&
> +		git rev-parse --verify refs/remotes/other/master
> +	)
> +'
> +
> +test_expect_success 'explicitly empty --refmap option disables remote.*.fetch' '
> +	cd "$D" &&
> +	git branch -f side &&
> +	(
> +		cd three &&
> +		git update-ref refs/remotes/origin/master base-origin-master &&
> +		o=$(git rev-parse --verify refs/remotes/origin/master) &&
> +		git fetch --refmap="" origin master &&
> +		n=$(git rev-parse --verify refs/remotes/origin/master) &&
> +		test "$o" = "$n" &&
> +		test_must_fail git rev-parse --verify refs/remotes/origin/side
> +	)
> +'
> +
>  test_expect_success 'configured fetch updates tracking' '
>  
>  	cd "$D" &&
> 

Re: [PATCH v2 9/9] fetch: allow explicit --refmap to override configuration

[Junio C Hamano]

Marc Branchaud <mbranchaud@xiplink.com> writes:

>> Teach the command to pay attention to the --refmap=<lhs>:<rhs>
>> command-line options that can be used to override the use of
>> configured remote.*.fetch as the refmap.
>
> (Your 0/9 message merely said "The new patches at the
> end clarifies how remote.*.fetch configuration variables are used in
> two conceptually different ways." so I was not expecting fetch to get a new
> option.)

This is more about conceptual consistency & completeness than new
and useful addition, in that configured values and the feature they
enable ought to be expressible and overridable from the command line
options but we so far lacked a way to trigger the "do not affect
what gets fetched, only affect where they go locally" feature, which
is offered by the second way to use remote.*.fetch variable.  I do
not think we absolutely need it and that is why it is at the end as
an optional addition.

[jc: again, good suggestions I'll use when amending snipped]

Thanks.

Re: [PATCH v2 9/9] fetch: allow explicit --refmap to override configuration

[Marc Branchaud]

On 14-06-04 06:28 PM, Junio C Hamano wrote:
> Marc Branchaud <mbranchaud@xiplink.com> writes:
> 
>>> Teach the command to pay attention to the --refmap=<lhs>:<rhs>
>>> command-line options that can be used to override the use of
>>> configured remote.*.fetch as the refmap.
>>
>> (Your 0/9 message merely said "The new patches at the
>> end clarifies how remote.*.fetch configuration variables are used in
>> two conceptually different ways." so I was not expecting fetch to get a new
>> option.)
> 
> This is more about conceptual consistency & completeness than new
> and useful addition, in that configured values and the feature they
> enable ought to be expressible and overridable from the command line
> options but we so far lacked a way to trigger the "do not affect
> what gets fetched, only affect where they go locally" feature, which
> is offered by the second way to use remote.*.fetch variable.  I do
> not think we absolutely need it and that is why it is at the end as
> an optional addition.

Ah, OK.

I don't have any objection to the option per se.  But I do wonder if there's
a need to add yet another knob to git just for completeness.  Has anyone ever
needed this?

		M.

Re: [PATCH v2 9/9] fetch: allow explicit --refmap to override configuration

[Junio C Hamano]

Marc Branchaud <marcnarc@xiplink.com> writes:

> I don't have any objection to the option per se.  But I do wonder if there's
> a need to add yet another knob to git just for completeness.  Has anyone ever
> needed this?

It is not a good yardstick, as everybody has survived without it
since Git's inception.  The right question to ask is: would it help
new use patterns, or improve existing use patterns?

Two possible scenarios I can think of offhand are

 * using an empty refmap to ensure that your "fetch" this time is
   really ephemeral without affecting the longer-term configured
   remote-tracking branches

 * grabbing only a few selected branches out of hundreds, e.g.

   $ git fetch https://github.com/gitster/git \
       --refmap=refs/heads/*:refs/remotes/jch/* maint master next +pu

   instead of having to spell its long-hand

   $ git fetch https://github.com/gitster/git \
       refs/heads/maint:refs/remotes/jch/maint \
       refs/heads/master:refs/remotes/jch/master \
       refs/heads/next:refs/remotes/jch/next \
       +refs/heads/pu:refs/remotes/jch/pu

but there may be more useful scenarios other people can come up
with ;-).

Re: [PATCH v2 9/9] fetch: allow explicit --refmap to override configuration

[Michael Haggerty]

On 06/05/2014 08:36 PM, Junio C Hamano wrote:
> Marc Branchaud <marcnarc@xiplink.com> writes:
> 
>> I don't have any objection to the option per se.  But I do wonder if there's
>> a need to add yet another knob to git just for completeness.  Has anyone ever
>> needed this?
> 
> It is not a good yardstick, as everybody has survived without it
> since Git's inception.  The right question to ask is: would it help
> new use patterns, or improve existing use patterns?

I agree that this feature is pretty esoteric and probably more cognitive
load than it's worth.  One of your use cases has workarounds shown below.

> Two possible scenarios I can think of offhand are
> 
>  * using an empty refmap to ensure that your "fetch" this time is
>    really ephemeral without affecting the longer-term configured
>    remote-tracking branches

Doesn't specifying an explicit URL get around the refspecs configured
for the remote?  E.g.,

    git fetch $(git config remote.github.url) master

Or if we had a way to temporarily unset multivalued configuration values
(which we may have soon thanks to the GSoC project of Tanay Abhra), one
could use

    git --unset=remote.github.fetch fetch github master

>  * grabbing only a few selected branches out of hundreds, e.g.
> 
>    $ git fetch https://github.com/gitster/git \
>        --refmap=refs/heads/*:refs/remotes/jch/* maint master next +pu
> 
>    instead of having to spell its long-hand
> 
>    $ git fetch https://github.com/gitster/git \
>        refs/heads/maint:refs/remotes/jch/maint \
>        refs/heads/master:refs/remotes/jch/master \
>        refs/heads/next:refs/remotes/jch/next \
>        +refs/heads/pu:refs/remotes/jch/pu

I'm not quite sure what your goal is here, but if you want to fetch some
branches on the fly without setting up a remote, then

    git -c remote.github.fetch='refs/heads/*:refs/remotes/jch/*' \
             fetch https://github.com/gitster/git maint master next +pu

should work, no?

> but there may be more useful scenarios other people can come up
> with ;-).

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu