[PATCH] user-manual: Talk about tracking third-party snapshots

[PATCH] user-manual: Talk about tracking third-party snapshots

[Michael Smith]

Add some sections about tracking third-party sources to the advanced
branching chapter. This might save some guesswork for tasks like
importing snapshots and tracking local changes.

Signed-off-by: Michael Smith <msmith@cbnco.com>
---

Years of heavy CVS abuse left me partly brain damaged, and some things that
are pretty easy in Git seemed like they should have been more complicated.
Hopefully this should make it painfully obvious how do to the equivalent
of CVS vendor branches.

 Documentation/user-manual.txt |  139 +++++++++++++++++++++++++++++++++++++++++
 1 files changed, 139 insertions(+), 0 deletions(-)

diff --git a/Documentation/user-manual.txt b/Documentation/user-manual.txt
index d99adc6..942f851 100644
--- a/Documentation/user-manual.txt
+++ b/Documentation/user-manual.txt
@@ -2711,6 +2711,145 @@ gitlink:git-config[1].
 See gitlink:git-config[1] for more details on the configuration
 options mentioned above.
 
+[[tracking-sources]]
+Tracking local changes to third-party sources
+---------------------------------------------
+
+[[tracking-sources-git]]
+Tracking changes when the upstream is a Git repository
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You don't have to do anything special to keep track of your changes to a
+Git project. All you need is a remote tracking branch for the upstream
+repository--either one of the remote branches created by
+gitlink:git-clone[1], or one you created according to
+<<fetching-individual-branches>>.
+
+Given a situation where you've merged the tracking branch's changes into
+your local branch, then made some more changes, as in the diagram below:
+
+................................................
+ o--o--a--c--e--f--g <-- origin/master (remote tracking branch)
+        \     \
+         b--d--m--h--i <-- master
+................................................
+
+You can view all your local changes--b, d, h, and i--with the
+gitlink:git-diff[1] command:
+
+------------------------------------------
+$ git diff origin/master...master
+------------------------------------------
+
+The three-dot `\...` tells gitlink:git-diff[1] to show the changes on the
+master branch since the last common ancestor with origin/master. (If you
+used two dots instead of three, you'd see the entire patch to go from
+origin/master to master, including reversing commits "f" and "g".)
+
+You can use the gitlink:git-cherry[1] command to display the commit
+IDs that are only present on your local branch, or only on the remote
+branch, respectively:
+
+------------------------------------------------
+$ git cherry -v origin/master master
++ 8ed8ff9315e36824e601659b168bbaad5e4d53ca b
++ 2bf7cdf2bef8e6f8b213634ce67dd01cc9e145e0 d
++ 14f97309ca82f742bc42d03fa4619a81973521a9 h
++ 4497730f04ed9849c807f2a5bf8f097f87636d3f i
+$ git cherry -v master origin/master
++ 8cc12a9763279d6f0c913ef47e0a996193aaa1c5 f
++ c793ea90311db286c0e22d227b494f09620aef3d g
+------------------------------------------------
+
+`git show <commit-id>` can display the full log message and patch for you.
+
+[[tracking-sources-snapshots]]
+Tracking changes when the upstream uses snapshots
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To track your changes to projects that aren't using Git, you can commit
+a snapshot or release onto a branch in your repository, then tag it. If
+you are used to vendor branches in CVS, you'll find this is similar,
+although CVS combines the commit, tag, and sometimes merge operations
+into one "import" command. Here's an example:
+
+------------------------------------------------
+$ mkdir project
+$ cd project
+$ git init
+$ cp -a ~/src/project-1.0/* .
+$ git add .
+$ git commit -m "Import Project v1.0"
+$ git tag v1.0
+$ git branch upstream
+------------------------------------------------
+
+Make your changes as usual on the "master" branch, or on topic branches
+that you merge to "master". For the next import, switch to the
+"upstream" branch, commit the new version, then switch back and merge in
+the changes:
+
+------------------------------------------------
+$ git checkout upstream
+$ rm -r *
+$ cp -a ~/src/project-1.1/* .
+      # "git add ." will catch the added and modified files
+$ git add .
+      # the "-a" flag will commit file deletions, too
+$ git commit -a -m "Import Project v1.1"
+$ git tag v1.1
+$ git checkout master
+$ git merge upstream
+------------------------------------------------
+
+If you are publishing your repository, you may also want to push the
+"upstream" branch and your tags.
+
+[[fixing-branch-ancestry]]
+Fixing branch ancestry
+~~~~~~~~~~~~~~~~~~~~~~
+
+Git can only do a sensible merge if it knows about a common ancestor
+between your local changes and the third-party sources. It needs to know
+the commit where your local changes and the third-party sources began to
+diverge--in other words, the last time they were merged. There are some
+cases where Git might not have a record of this merge:
+
+1. You imported CVS or Subversion vendor branch history into Git.
+Sometimes this can produce completely independent master and vendor
+branches with no merging between the two. All the changes are there,
+they just aren't linked by a merge. You can see this in `gitk --all` as
+two parallel development histories.
+2. You've been importing third-party tarballs or snapshots into Git, but
+now the upstream has switched to Git and you want to pull from their new
+repository. As far as Git knows, their branch is completely independent
+from yours, with no common ancestry.
+
+You can fix situations like these by doing a merge that isn't really a
+merge, using the "ours" merge strategy. Look through the history on the
+third-party branch and try to find the exact commit that matches the
+last snapshot you imported. Often there's a tag close to the commit, or
+on the commit, if you're lucky--but don't trust it blindly; check the
+diffs. Check out your local branch and tell Git about the relationship:
+
+------------------------------------------------
+$ git remote add upstreamgit git://upstream.org/project.git
+$ git fetch upstreamgit
+$ git tag
+v1.0
+v1.1
+v1.2
+$ git checkout master
+$ git merge --strategy=ours \
+    -m "Tie old v1.1 into our history by merging with strategy=ours." \
+    v1.1
+------------------------------------------------
+
+You'll see the branches merge together in `gitk --all` or `git
+show-branch master upstreamgit/master`.  Now you'll be able to merge any
+changes from the remote branch since v1.1 with `git merge
+upstreamgit/master`.
+
 
 [[git-concepts]]
 Git concepts
-- 
1.5.3.2.102.ge6eb7

Re: [PATCH] user-manual: Talk about tracking third-party snapshots

[Jakub Narebski]

Michael Smith wrote:

> +You can use the gitlink:git-cherry[1] command to display the commit
> +IDs that are only present on your local branch, or only on the remote
> +branch, respectively:

I think git-cherry is deprecated in favor of "git log --left-right" (with
appropriate format, for example '--abbrev-commit --pretty=oneline')

BTW. that means that git-cherry can be removed from git-help output,
I think.

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: [PATCH] user-manual: Talk about tracking third-party snapshots

[Sergei Organov]

Jakub Narebski <jnareb@gmail.com> writes:

> Michael Smith wrote:
>
>> +You can use the gitlink:git-cherry[1] command to display the commit
>> +IDs that are only present on your local branch, or only on the remote
>> +branch, respectively:
>
> I think git-cherry is deprecated in favor of "git log --left-right" (with
> appropriate format, for example '--abbrev-commit --pretty=oneline')
>
> BTW. that means that git-cherry can be removed from git-help output,
> I think.

And from core-tutorial.txt:1567:

4. Use `git cherry origin` to see which ones of your patches
   were accepted, and/or use `git rebase origin` to port your
   unmerged changes forward to the updated upstream.

???

-- 
Sergei.

Re: [PATCH] user-manual: Talk about tracking third-party snapshots

[Jakub Narebski]

Sergei Organov wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
> 
>> Michael Smith wrote:
>>
>>> +You can use the gitlink:git-cherry[1] command to display the commit
>>> +IDs that are only present on your local branch, or only on the remote
>>> +branch, respectively:
>>
>> I think git-cherry is deprecated in favor of "git log --left-right" (with
>> appropriate format, for example '--abbrev-commit --pretty=oneline')
>>
>> BTW. that means that git-cherry can be removed from git-help output,
>> I think.
> 
> And from core-tutorial.txt:1567:
> 
> 4. Use `git cherry origin` to see which ones of your patches
>    were accepted, and/or use `git rebase origin` to port your
>    unmerged changes forward to the updated upstream.
> 
> ???

On one hand, core-tutorial is old documentation, dating before
--left-right option to git-log.

On the other hand I have forgot that git-cherry does more throghout
checking if patch was accepted upstream (by checking changeset).


Sorry for the noise...
-- 
Jakub Narebski
Poland

Re: [PATCH] user-manual: Talk about tracking third-party snapshots

[Jan Hudec]

On Tue, Nov 13, 2007 at 15:07:05 +0100, Jakub Narebski wrote:
> Michael Smith wrote:
> 
> > +You can use the gitlink:git-cherry[1] command to display the commit
> > +IDs that are only present on your local branch, or only on the remote
> > +branch, respectively:
> 
> I think git-cherry is deprecated in favor of "git log --left-right" (with
> appropriate format, for example '--abbrev-commit --pretty=oneline')

git log has such option?

$ man git-log | grep -e --left-right; echo $?
1
$ git --version
git version 1.5.3.5

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

Re: [PATCH] user-manual: Talk about tracking third-party snapshots

[Jakub Narebski]

On Sat, Nov 17, 2007, Jan Hudec wrote:
> On Tue, Nov 13, 2007 at 15:07:05 +0100, Jakub Narebski wrote:
>> Michael Smith wrote:
>> 
>>> +You can use the gitlink:git-cherry[1] command to display the commit
>>> +IDs that are only present on your local branch, or only on the remote
>>> +branch, respectively:
>> 
>> I think git-cherry is deprecated in favor of "git log --left-right" (with
>> appropriate format, for example '--abbrev-commit --pretty=oneline')

Not true. git-cherry is more than --left-right, as it checks
if changesets matches, not if commit id matches.

> git log has such option?
> 
> $ man git-log | grep -e --left-right; echo $?
> 1
> $ git --version
> git version 1.5.3.5

It has, although it is hidden in git-rev-list(1) manpage. It is a bit
obscure corner...

-- 
Jakub Narebski
Poland

Re: [PATCH] user-manual: Talk about tracking third-party snapshots

[Jan Hudec]

On Sat, Nov 17, 2007 at 19:18:40 +0100, Jakub Narebski wrote:
> On Sat, Nov 17, 2007, Jan Hudec wrote:
> > On Tue, Nov 13, 2007 at 15:07:05 +0100, Jakub Narebski wrote:
> >> Michael Smith wrote:
> >> 
> >>> +You can use the gitlink:git-cherry[1] command to display the commit
> >>> +IDs that are only present on your local branch, or only on the remote
> >>> +branch, respectively:
> >> 
> >> I think git-cherry is deprecated in favor of "git log --left-right" (with
> >> appropriate format, for example '--abbrev-commit --pretty=oneline')
> 
> Not true. git-cherry is more than --left-right, as it checks
> if changesets matches, not if commit id matches.
> 
> > git log has such option?
> > 
> > $ man git-log | grep -e --left-right; echo $?
> > 1
> > $ git --version
> > git version 1.5.3.5
> 
> It has, although it is hidden in git-rev-list(1) manpage. It is a bit
> obscure corner...

I hope the new option parsing ifrastructure will take over quickly and start
to be used to generate the short help and probably even option section in the
man pages. It's unfortunately not the only option that is not mentioned in
the manual page of a command that has it.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

Re: [PATCH] user-manual: Talk about tracking third-party snapshots

[Jakub Narebski]

Jan Hudec wrote:
> On Sat, Nov 17, 2007 at 19:18:40 +0100, Jakub Narebski wrote:
>> On Sat, Nov 17, 2007, Jan Hudec wrote:

>>> git log has such option?
>>> 
>>> $ man git-log | grep -e --left-right; echo $?
>>> 1
>>> $ git --version
>>> git version 1.5.3.5
>> 
>> It has, although it is hidden in git-rev-list(1) manpage. It is a bit
>> obscure corner...
> 
> I hope the new option parsing ifrastructure will take over quickly and start
> to be used to generate the short help and probably even option section in the
> man pages. It's unfortunately not the only option that is not mentioned in
> the manual page of a command that has it.

Truth to be told _this_ one option I don't mean it is only
in git-rev-parse (which git-log references, together with git-diff)

-- 
Jakub Narebski
Poland

Re: [PATCH] user-manual: Talk about tracking third-party snapshots

[Junio C Hamano]

Michael Smith <msmith@cbnco.com> writes:

> +You can fix situations like these by doing a merge that isn't really a
> +merge, using the "ours" merge strategy. Look through the history on the
> +third-party branch and try to find the exact commit that matches the
> +last snapshot you imported. Often there's a tag close to the commit, or
> +on the commit, if you're lucky--but don't trust it blindly; check the
> +diffs. Check out your local branch and tell Git about the relationship:
> +
> +------------------------------------------------
> +$ git remote add upstreamgit git://upstream.org/project.git
> +$ git fetch upstreamgit
> +$ git tag
> +v1.0
> +v1.1
> +v1.2
> +$ git checkout master
> +$ git merge --strategy=ours \
> +    -m "Tie old v1.1 into our history by merging with strategy=ours." \
> +    v1.1
> +------------------------------------------------
> +
> +You'll see the branches merge together in `gitk --all` or `git
> +show-branch master upstreamgit/master`.  Now you'll be able to merge any
> +changes from the remote branch since v1.1 with `git merge
> +upstreamgit/master`.
> +

This would work only when your 'master' happens to be at v1.1
(and identical to it) isn't it?  Which means that as an example
it will be of very limited scope.

People would want to know "But my 'master' is _not_ at v1.1 but
is _based_ on v1.1.  How would I handle that case?" and the
above does not answer that question.

Even worse, most people are probably not careful enough to ask
the above question, but just say "Heh, my 'master' is based on
v1.1, so I'll blindly follow that example to bind the histories
together".

I did not find any technical problem in the other parts of your
description, but I did not read the resulting document from
cover to cover, so I do not know if your change fits in the
entire organization of the document very well.

Re: [PATCH] user-manual: Talk about tracking third-party snapshots

[Michael Smith]

On Tue, 13 Nov 2007, Junio C Hamano wrote:

> > +$ git checkout master
> > +$ git merge --strategy=ours \
> > +    -m "Tie old v1.1 into our history by merging with strategy=ours." \
> > +    v1.1

> This would work only when your 'master' happens to be at v1.1
> (and identical to it) isn't it?
> 
> People would want to know "But my 'master' is _not_ at v1.1 but
> is _based_ on v1.1.  How would I handle that case?"

That was actually my case - my master was based on v1.1. The command 
worked for me - I was able to merge in v1.2 from Git afterward - but maybe 
I just got lucky, so I'd be interested to know the right thing to do.

I think --strategy=ours just leaves the files as is on master and creates 
a merge commit with two parents: master and v1.1. So anything between v1.1 
and v1.2, on the upstream branch, is fair game to be merged next time.

So, say I started with this:

---------------
old snapshot branch: snap-v1.0---snap-v1.1
                       \           \
master:                 o---o---o---o--o-o

new Git upstream: o--o-v1.0-o-o-o-o-o-o-v1.1-o-o-v1.2
---------------

I could make the merge with --strategy=ours:
---------------
old snapshot branch: snap-v1.0---snap-v1.1
                       \           \
master:                 o---o---o---o--o-o--o
                                           /
new Git upstream: o--o-v1.0-o-o-o-o-o-o-v1.1-o-o-v1.2
---------------


Then I'd free to merge in v1.2:
---------------
master:                 o---o---o---o--o-o--o------o
                                           /      /
new Git upstream: o--o-v1.0-o-o-o-o-o-o-v1.1-o-o-v1.2
---------------

Hmm, looks like a toothbrush. But is it right?

Mike

Re: [PATCH] user-manual: Talk about tracking third-party snapshots

[J. Bruce Fields]

On Tue, Nov 13, 2007 at 11:25:12AM -0800, Junio C Hamano wrote:
> Michael Smith <msmith@cbnco.com> writes:
> 
> > +You can fix situations like these by doing a merge that isn't really a
> > +merge, using the "ours" merge strategy. Look through the history on the
> > +third-party branch and try to find the exact commit that matches the
> > +last snapshot you imported. Often there's a tag close to the commit, or
> > +on the commit, if you're lucky--but don't trust it blindly; check the
> > +diffs. Check out your local branch and tell Git about the relationship:
> > +
> > +------------------------------------------------
> > +$ git remote add upstreamgit git://upstream.org/project.git
> > +$ git fetch upstreamgit
> > +$ git tag
> > +v1.0
> > +v1.1
> > +v1.2
> > +$ git checkout master
> > +$ git merge --strategy=ours \
> > +    -m "Tie old v1.1 into our history by merging with strategy=ours." \
> > +    v1.1
> > +------------------------------------------------
> > +
> > +You'll see the branches merge together in `gitk --all` or `git
> > +show-branch master upstreamgit/master`.  Now you'll be able to merge any
> > +changes from the remote branch since v1.1 with `git merge
> > +upstreamgit/master`.
> > +
> 
> This would work only when your 'master' happens to be at v1.1
> (and identical to it) isn't it?  Which means that as an example
> it will be of very limited scope.
> 
> People would want to know "But my 'master' is _not_ at v1.1 but
> is _based_ on v1.1.  How would I handle that case?" and the
> above does not answer that question.
> 
> Even worse, most people are probably not careful enough to ask
> the above question, but just say "Heh, my 'master' is based on
> v1.1, so I'll blindly follow that example to bind the histories
> together".
> 
> I did not find any technical problem in the other parts of your
> description, but I did not read the resulting document from
> cover to cover, so I do not know if your change fits in the
> entire organization of the document very well.

A mention of git-cherry or --left-right someplace might make sense,
maybe in chapter 2 somewhere (maybe in "Browsing revisions"?), but it
might also be useful to discuss this kind of stuff in the "sharing
development with others" section.  (E.g. at the end of "Submitting
patches to a project", it might help to have some hints about how to
recognize when they've been accepted and what to do then.)

The trick of using a merge (possibly with -s ours) to tie together two
independent paths to the same state is more generally useful.  (I've
been using it sometimes to keep track of the revision history of a patch
series, for example).  I'm not sure where that should go.

I like the tips on importing snapshots or other history.  I've long been
wanting an "interoperating with other SCM's" section (rough beginnings
at git://linux-nfs.org/~bfields/git.git docwork-foreign-scms), and maybe
some of this would fit here.

All of these things seem more generally useful even to an audience that
hasn't worked with CVS vendor branches before, so I'd prefer
explanations that didn't reference the CVS stuff.  (With CVS-specific
stuff moved somewhere else, maybe to cvs-migration.txt.)

--b.

Re: [PATCH] user-manual: Talk about tracking third-party snapshots

[J. Bruce Fields]

On Tue, Nov 13, 2007 at 07:29:59AM -0500, Michael Smith wrote:
> +You can view all your local changes--b, d, h, and i--with the
> +gitlink:git-diff[1] command:
> +
> +------------------------------------------
> +$ git diff origin/master...master
> +------------------------------------------
> +
> +The three-dot `\...` tells gitlink:git-diff[1] to show the changes on the
> +master branch since the last common ancestor with origin/master. (If you
> +used two dots instead of three, you'd see the entire patch to go from
> +origin/master to master, including reversing commits "f" and "g".)

I missed the "..." thing when on my first attempt at the manual.  It
really should be mentioned in the "Generating diffs" section; I've added
the following to my

	git://linux-nfs.org/~bfields/git.git maint

--b.

>From 5b98d9bca16e19710380d2d03f704de9eb98621d Mon Sep 17 00:00:00 2001
From: J. Bruce Fields <bfields@citi.umich.edu>
Date: Sun, 18 Nov 2007 19:18:27 -0500
Subject: [PATCH] user-manual: mention "..." in "Generating diffs", etc.

We should mention the use of the "..." syntax for git-diff here.  The
note about the difference between diff and the combined output of
git-format-patch then no longer fits so well, so remove it.  Add a
reference to the git-format-patch[1] manpage.

Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
---
 Documentation/user-manual.txt |   15 +++++++++++----
 1 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/Documentation/user-manual.txt b/Documentation/user-manual.txt
index e399685..c027353 100644
--- a/Documentation/user-manual.txt
+++ b/Documentation/user-manual.txt
@@ -658,16 +658,23 @@ gitlink:git-diff[1]:
 $ git diff master..test
 -------------------------------------------------
 
-Sometimes what you want instead is a set of patches:
+That will produce the diff between the tips of the two branches.  If
+you'd prefer to find the diff from their common ancestor to test, you
+can use three dots instead of two:
+
+-------------------------------------------------
+$ git diff master...test
+-------------------------------------------------
+
+Sometimes what you want instead is a set of patches; for this you can
+use gitlink:git-format-patch[1]:
 
 -------------------------------------------------
 $ git format-patch master..test
 -------------------------------------------------
 
 will generate a file with a patch for each commit reachable from test
-but not from master.  Note that if master also has commits which are
-not reachable from test, then the combined result of these patches
-will not be the same as the diff produced by the git-diff example.
+but not from master.
 
 [[viewing-old-file-versions]]
 Viewing old file versions
-- 
1.5.3.5.561.g140d