How to update a cloned git repository

How to update a cloned git repository

[Joachim Schmitz]

Hi folks

Probably a beginner's question...

If I did a
    git clone git://guthub.com/git/git.git
and worked on some own branches of pu
    git checkout pu;git checkout -p mybranch;hack;hack;...;git commit -a -s
how to update my repository once the the one on github changed?
A plain
    git pull
or
    git fetch;git merge
keeps failing on my with lots of conflicts, none of which relate to any of 
the changes I did (and hence wouldn't know how to resolve)


Bye, Jojo 

Re: How to update a cloned git repository

[Matthieu Moy]

"Joachim Schmitz" <jojo@schmitz-digital.de> writes:

> Hi folks
>
> Probably a beginner's question...
>
> If I did a
>    git clone git://guthub.com/git/git.git
> and worked on some own branches of pu
>    git checkout pu;git checkout -p mybranch;

I guess you meant "git checkout -b mybranch" (not -p).

> hack;hack;...;git commit -a -s
>
> how to update my repository once the the one on github changed? A
> plain git pull or git fetch;git merge keeps failing on my with lots of
> conflicts, none of which relate to any of the changes I did (and hence
> wouldn't know how to resolve)

Short answer: don't work on pu. Work on master unless you have a good
reason not to.

Longer answer: the pu branch in git.git is often re-written, hence the
commit on which you started hacking once existed in git.git's pu, but it
probably no longer is.

You cloned this:

--A---B---C <-- origin/pu

Hacked to this

         origin/pu
          |
          v
--A---B---C---D---E <-- mybranch

and the next fetch resulted in something like this:

    B'---C'---D'---F <-- origin/pu
   /
--A---B---C---D---E <-- mybranch

while you could have expected that if origin/pu had just been
fast-forwarded with a new commit F:

            F <-- origin/pu
           /
--A---B---C---D---E <-- mybranch

As a result, "git merge" computes a common ancestor very far backward in
history. Instead of merging only your changes with new pu content, it
merges the old history of pu (plus your changes) with the new history of
pu, and you get spurious conflicts.

The solution is to rebase your changes (and only yours). My advice is to
rebase them on master, like this (replace 42 by the number of commits
you want to rebase in HEAD~42):

  git rebase HEAD~42 --onto origin/master

Once you did this, you can start using "git pull" (or "git pull
--rebase") as usual.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

RE: How to update a cloned git repository

[Joachim Schmitz]

> From: Matthieu Moy [mailto:Matthieu.Moy@grenoble-inp.fr]
> Sent: Tuesday, September 11, 2012 1:06 PM
> To: Joachim Schmitz
> Cc: git@vger.kernel.org
> Subject: Re: How to update a cloned git repository
> 
> "Joachim Schmitz" <jojo@schmitz-digital.de> writes:
> 
> > Hi folks
> >
> > Probably a beginner's question...
> >
> > If I did a
> >    git clone git://guthub.com/git/git.git
> > and worked on some own branches of pu
> >    git checkout pu;git checkout -p mybranch;
> 
> I guess you meant "git checkout -b mybranch" (not -p).

Yes, of course...

> > hack;hack;...;git commit -a -s
> >
> > how to update my repository once the the one on github changed? A
> > plain git pull or git fetch;git merge keeps failing on my with lots of
> > conflicts, none of which relate to any of the changes I did (and hence
> > wouldn't know how to resolve)
> 
> Short answer: don't work on pu. Work on master unless you have a good
> reason not to.

There are some changes in pu, that I need as the basis, namely my setitimer patch and my 2nd mkdir patch, which haven't yet made it
into the master branch (and in the setitimer case not out of pu)

> Longer answer: the pu branch in git.git is often re-written, hence the
> commit on which you started hacking once existed in git.git's pu, but it
> probably no longer is.
> 
> You cloned this:
> 
> --A---B---C <-- origin/pu
> 
> Hacked to this
> 
>          origin/pu
>           |
>           v
> --A---B---C---D---E <-- mybranch
> 
> and the next fetch resulted in something like this:
> 
>     B'---C'---D'---F <-- origin/pu
>    /
> --A---B---C---D---E <-- mybranch
> 
> while you could have expected that if origin/pu had just been
> fast-forwarded with a new commit F:
> 
>             F <-- origin/pu
>            /
> --A---B---C---D---E <-- mybranch
> 
> As a result, "git merge" computes a common ancestor very far backward in
> history. Instead of merging only your changes with new pu content, it
> merges the old history of pu (plus your changes) with the new history of
> pu, and you get spurious conflicts.
> 
> The solution is to rebase your changes (and only yours). My advice is to
> rebase them on master, like this (replace 42 by the number of commits
> you want to rebase in HEAD~42):
> 
>   git rebase HEAD~42 --onto origin/master

For pu this would be similar?

> Once you did this, you can start using "git pull" (or "git pull
> --rebase") as usual.


Like this?
git pull --rebase HEAD~42

So far I create patches, wiped out the entire repository, cloned, forked and applied the changes, pretty painful.

Re: How to update a cloned git repository

[Matthieu Moy]

"Joachim Schmitz" <jojo@schmitz-digital.de> writes:

>> From: Matthieu Moy [mailto:Matthieu.Moy@grenoble-inp.fr]
>>
>> Short answer: don't work on pu. Work on master unless you have a good
>> reason not to.
>
> There are some changes in pu, that I need as the basis, namely my
> setitimer patch and my 2nd mkdir patch, which haven't yet made it into
> the master branch (and in the setitimer case not out of pu)

Then, work on the tip of the topic branch you depend on instead of pu.
These are more stable, as they will be rewritten only if this particular
topic branch changes.

>>   git rebase HEAD~42 --onto origin/master
>
> For pu this would be similar?

Yes. If you insist in working on top of pu, you can rebase --onto
origin/pu.

> Like this?
> git pull --rebase HEAD~42

That would be "git fetch" and then "git rebase", as I don't think "git
pull --rebase" would allow you to specify the starting point for rebase.

> So far I create patches, wiped out the entire repository, cloned,
> forked and applied the changes, pretty painful.

This is conceptually similar to what "git rebase" does, but it does it
in a slightly more efficient way ;-).

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: How to update a cloned git repository

[Matthieu Moy]

[ Re-adding git@vger in Cc, I guess it was meant to be so ]

"Joachim Schmitz" <jojo@schmitz-digital.de> writes:

>> Then, work on the tip of the topic branch you depend on instead of pu.
>> These are more stable, as they will be rewritten only if this particular
>> topic branch changes.
>
> These are not available from git hub. Or are they? How?

I think they exist in some of the repos junio pushes to, but I don't
remember how/which one.

Anyway, you can easily get it from the commit that merges the branch
(it's the-merge-commit^1).

>> > Like this?
>> > git pull --rebase HEAD~42
>> 
>> That would be "git fetch" and then "git rebase", as I don't think "git
>> pull --rebase" would allow you to specify the starting point for rebase.
>
> OK, I'll try that next time then. Like this?
> 	git fetch;git rebase HEAD~42 --onto origin/pu

That should work, yes.

In general, when you have a somehow complex workflow, I recommand
fetch+(merge|rebase) over pull. It gives you more flexibility, and the
opportunity to check what you fetched before starting the merge.

>> > So far I create patches, wiped out the entire repository, cloned,
>> > forked and applied the changes, pretty painful.
>> 
>> This is conceptually similar to what "git rebase" does, but it does it
>> in a slightly more efficient way ;-).
>
> Indeed ;-)
>
>
>

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

RE: How to update a cloned git repository

[Joachim Schmitz]

> From: Matthieu Moy [mailto:Matthieu.Moy@grenoble-inp.fr]
> Sent: Tuesday, September 11, 2012 2:41 PM
> To: Joachim Schmitz
> Cc: git
> Subject: Re: How to update a cloned git repository
> 
> [ Re-adding git@vger in Cc, I guess it was meant to be so ]


Oops, yes it was meant to.

> "Joachim Schmitz" <jojo@schmitz-digital.de> writes:
> 
> >> Then, work on the tip of the topic branch you depend on instead of pu.
> >> These are more stable, as they will be rewritten only if this particular
> >> topic branch changes.
> >
> > These are not available from git hub. Or are they? How?
> 
> I think they exist in some of the repos junio pushes to, but I don't
> remember how/which one.
> 
> Anyway, you can easily get it from the commit that merges the branch
> (it's the-merge-commit^1).
> 
> >> > Like this?
> >> > git pull --rebase HEAD~42
> >>
> >> That would be "git fetch" and then "git rebase", as I don't think "git
> >> pull --rebase" would allow you to specify the starting point for rebase.
> >
> > OK, I'll try that next time then. Like this?
> > 	git fetch;git rebase HEAD~42 --onto origin/pu
> 
> That should work, yes.
> 
> In general, when you have a somehow complex workflow, I recommand
> fetch+(merge|rebase) over pull. It gives you more flexibility, and the
> opportunity to check what you fetched before starting the merge.

OK, thanks.

I'm sure I'll have more questions later though ;-)

Re: How to update a cloned git repository

[Erik Faye-Lund]

On Tue, Sep 11, 2012 at 2:40 PM, Matthieu Moy
<Matthieu.Moy@grenoble-inp.fr> wrote:
> [ Re-adding git@vger in Cc, I guess it was meant to be so ]
>
> "Joachim Schmitz" <jojo@schmitz-digital.de> writes:
>
>>> Then, work on the tip of the topic branch you depend on instead of pu.
>>> These are more stable, as they will be rewritten only if this particular
>>> topic branch changes.
>>
>> These are not available from git hub. Or are they? How?
>
> I think they exist in some of the repos junio pushes to, but I don't
> remember how/which one.
>

You can usually find them here: https://github.com/gitster/git/branches

And Joachim's branch in particular:
https://github.com/gitster/git/tree/js/compat-itimer

Re: How to update a cloned git repository

[Junio C Hamano]

Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:

> [ Re-adding git@vger in Cc, I guess it was meant to be so ]
>
> "Joachim Schmitz" <jojo@schmitz-digital.de> writes:
>
>>> Then, work on the tip of the topic branch you depend on instead of pu.
>>> These are more stable, as they will be rewritten only if this particular
>>> topic branch changes.
>>
>> These are not available from git hub. Or are they? How?
>
> I think they exist in some of the repos junio pushes to, but I don't
> remember how/which one.
>
> Anyway, you can easily get it from the commit that merges the branch
> (it's the-merge-commit^1).

I think it is the-merge-commit^2; contrib/git-resurrect.sh might be
of interest, too.

Re: How to update a cloned git repository

[Matthieu Moy]

Junio C Hamano <gitster@pobox.com> writes:

> Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:
>
>> Anyway, you can easily get it from the commit that merges the branch
>> (it's the-merge-commit^1).
>
> I think it is the-merge-commit^2;

Right. I forgot that Git counted parents from 1, not 0.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

RE: How to update a cloned git repository

[Joachim Schmitz]

> From: Junio C Hamano [mailto:gitster@pobox.com]
> Sent: Tuesday, September 11, 2012 6:06 PM
> To: Matthieu Moy
> Cc: Joachim Schmitz; git
> Subject: Re: How to update a cloned git repository
> 
> Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:
> 
> > [ Re-adding git@vger in Cc, I guess it was meant to be so ]
> >
> > "Joachim Schmitz" <jojo@schmitz-digital.de> writes:
> >
> >>> Then, work on the tip of the topic branch you depend on instead of pu.
> >>> These are more stable, as they will be rewritten only if this particular
> >>> topic branch changes.
> >>
> >> These are not available from git hub. Or are they? How?
> >
> > I think they exist in some of the repos junio pushes to, but I don't
> > remember how/which one.
> >
> > Anyway, you can easily get it from the commit that merges the branch
> > (it's the-merge-commit^1).
> 
> I think it is the-merge-commit^2; contrib/git-resurrect.sh might be
> of interest, too.

Sorry you lost me, this is greek to me...

Re: How to update a cloned git repository

[Matthieu Moy]

"Joachim Schmitz" <jojo@schmitz-digital.de> writes:

>> I think it is the-merge-commit^2; contrib/git-resurrect.sh might be
>> of interest, too.
>
> Sorry you lost me, this is greek to me...

A commit is an object that contain pointers to its parents. The root
commit has no parent. For ordinary commits, there is one parent which is
the commit on top of which it was created. For merge commits, there are
N commits, the first is the one on top of which the merge was created,
and the N-1 next ones are the commits being merged.

commit^ => first parent of commit
commit^1 => same
commit^2 => second parent, i.e. the one merged in commit.

See in git.git:

$ git show cb10ae9433126ef <---------------------------- one commit to study
commit cb10ae9433126efbc4dcc46779d7ef2fe6b1f597
Merge: 13b608a 9aeaab6   <------------------------------- list of parents
Author: Junio C Hamano <gitster@pobox.com>
Date:   Tue Sep 11 15:57:04 2012 -0700

    Merge branch 'jc/maint-blame-no-such-path' into pu
    
    * jc/maint-blame-no-such-path:
      blame: allow "blame file" in the middle of a conflicted merge

$ git show cb10ae9433126ef^1 <-------------------- "previous commit" in origin/pu
commit 13b608a063ce861929322e6bb3862b5364f3fbcf
Merge: fa17a26 bdee397
Author: Junio C Hamano <gitster@pobox.com>
Date:   Tue Sep 11 11:50:44 2012 -0700

    Merge branch 'dg/run-command-child-cleanup' into pu
    
    * dg/run-command-child-cleanup:
      run-command.c: fix broken list iteration in clear_child_for_cleanup

$ git show cb10ae9433126ef^2 <----------------- commit being merged by cb10ae9433126ef
commit 9aeaab6811dce596b4f6141d76f5300359bfd009
Author: Junio C Hamano <gitster@pobox.com>
Date:   Tue Sep 11 14:30:03 2012 -0700

    blame: allow "blame file" in the middle of a conflicted merge
[...]

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: How to update a cloned git repository

[Sitaram Chamarty]

On Tue, Sep 11, 2012 at 4:47 PM, Joachim Schmitz
<jojo@schmitz-digital.de> wrote:

> Like this?
> git pull --rebase HEAD~42
>
> So far I create patches, wiped out the entire repository, cloned, forked and applied the changes, pretty painful.

I think a 'git pull --rebase' should usually work even for 'pu'.

But sometimes pu may have changes that take away the basis for your
patch (i.e, not just a restructure) then you'd get conflicts.

Re: How to update a cloned git repository

[Junio C Hamano]

"Joachim Schmitz" <jojo@schmitz-digital.de> writes:

>> From: Matthieu Moy [mailto:Matthieu.Moy@grenoble-inp.fr]
>> ..
>> Short answer: don't work on pu. Work on master unless you have a good
>> reason not to.
>
> There are some changes in pu, that I need as the basis, namely my
> setitimer patch and my 2nd mkdir patch, which haven't yet made it
> into the master branch (and in the setitimer case not out of pu)

And that is not a good reason, either.

In general, it is a good habit to get into to base your changes on
the oldest point release they may want to be used with.  For
example, if you really wanted to, you could make sure your Tandem
changes can be back-merged all the way down to v1.7.6 by forking
your own branch from there, queuing your changes like mkdir, itimer
on top.  And you develop and test your changes on that branch,
without pulling from or rebasing it on top of my tree where random
other things happen that won't affect you an iota.  A recent change
to add the new "--set-upstream-to" option to "branch" command does
not have any platform-specific bits, and for the purpose of the
"port to Tandem" topic, keeping up to date with such a change in my
tree is pointless, for example.  To make sure that the result is
still usable with recent releases, you can create a throw-away merge
between your work (that is forked from a stable base) and my tip
every once in a while, test the result, and discard the throw-away
merge when you are done.  Any breakage in your series you find in
such an integration test is to be fixed on your branch, not on top
of such a throw-away merge.

It might be the case that nobody cares if the resulting patches will
not apply to and usable with 'master' or older integration branches,
so in the particular case of "Tandem", choosing a stable fork point
that is older than 'master' may not make much sense, though.

RE: How to update a cloned git repository

[Joachim Schmitz]

> -----Original Message-----
> From: Junio C Hamano [mailto:gitster@pobox.com]
> Sent: Tuesday, September 11, 2012 6:01 PM
> To: Joachim Schmitz
> Cc: 'Matthieu Moy'; git@vger.kernel.org
> Subject: Re: How to update a cloned git repository
> 
> "Joachim Schmitz" <jojo@schmitz-digital.de> writes:
> 
> >> From: Matthieu Moy [mailto:Matthieu.Moy@grenoble-inp.fr]
> >> ..
> >> Short answer: don't work on pu. Work on master unless you have a good
> >> reason not to.
> >
> > There are some changes in pu, that I need as the basis, namely my
> > setitimer patch and my 2nd mkdir patch, which haven't yet made it
> > into the master branch (and in the setitimer case not out of pu)
> 
> And that is not a good reason, either.
> 
> In general, it is a good habit to get into to base your changes on
> the oldest point release they may want to be used with.  For
> example, if you really wanted to, you could make sure your Tandem
> changes can be back-merged all the way down to v1.7.6 by forking

The first version I ever made available for NonStop is 1.5.12, so no reason for me to go back.
On the other hand I see nothing in my patches that would not easily backport to much older releases, as the code I touched is either
newly created by me (e.g. compat/mkdir.c) or pretty old (compar/win32/poll.c).

> your own branch from there, queuing your changes like mkdir, itimer
> on top.  And you develop and test your changes on that branch,
> without pulling from or rebasing it on top of my tree where random
> other things happen that won't affect you an iota.  A recent change
> to add the new "--set-upstream-to" option to "branch" command does
> not have any platform-specific bits, and for the purpose of the
> "port to Tandem" topic, keeping up to date with such a change in my
> tree is pointless, for example.  To make sure that the result is
> still usable with recent releases, you can create a throw-away merge
> between your work (that is forked from a stable base) and my tip
> every once in a while, test the result, and discard the throw-away
> merge when you are done.  Any breakage in your series you find in
> such an integration test is to be fixed on your branch, not on top
> of such a throw-away merge.
> 
> It might be the case that nobody cares if the resulting patches will
> not apply to and usable with 'master' or older integration branches,
> so in the particular case of "Tandem", choosing a stable fork point
> that is older than 'master' may not make much sense, though.

My goal here is that the next stable release has as much of my patches integrated as possible, so I have much less work when
porting, ideals just hitting 'make'...

So far my poll patches are still needed. And then, but not earlier, my plain NonStop specific ones (like a section in Makefile and
some adjustments in git-compat-utils.h) , these don't make much sense earlier.

Bye, Jojo