rsync update appears broken now

rsync update appears broken now

[Randal L. Schwartz]

Doing my daily git-pull now broke in this way (using yesterday's git version):

sent 1196 bytes  received 155984 bytes  4555.94 bytes/sec
total size is 4511741  speedup is 28.70
* committish: 6e1c6c103c522d01829f3a63992a023ff031e851
  branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
* refs/heads/origin: does not fast forward to branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git;
  not updating.
Trying really trivial in-index merge...
fatal: Merge requires file-level merging
Nope.
Trying simple merge.
Simple merge failed, trying Automatic merge.
Auto-merging fetch-pack.c.
merge: warning: conflicts during merge
ERROR: Merge conflict in fetch-pack.c.
fatal: merge program failed
Automatic merge failed; fix up by hand


-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: rsync update appears broken now

[Alex Riesen]

On 20 Oct 2005 05:47:42 -0700, Randal L. Schwartz <merlyn@stonehenge.com> wrote:
>
> Doing my daily git-pull now broke in this way (using yesterday's git version):
>
> sent 1196 bytes  received 155984 bytes  4555.94 bytes/sec
> total size is 4511741  speedup is 28.70
> * committish: 6e1c6c103c522d01829f3a63992a023ff031e851
>   branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
> * refs/heads/origin: does not fast forward to branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git;
>   not updating.
> Trying really trivial in-index merge...
> fatal: Merge requires file-level merging
> Nope.
> Trying simple merge.
> Simple merge failed, trying Automatic merge.
> Auto-merging fetch-pack.c.
> merge: warning: conflicts during merge
> ERROR: Merge conflict in fetch-pack.c.
> fatal: merge program failed
> Automatic merge failed; fix up by hand

Absolutely normal pull into a changed repository. Just fix the
conflict (in fetch-pack.c, look for >>>), git-update-index the file
and commit. Doesn't look like a problem at all.

Re: rsync update appears broken now

[Randal L. Schwartz]

>>>>> "Alex" == Alex Riesen <raa.lkml@gmail.com> writes:

Alex> Absolutely normal pull into a changed repository. Just fix the
Alex> conflict (in fetch-pack.c, look for >>>), git-update-index the file
Alex> and commit. Doesn't look like a problem at all.

What do you mean "changed repository"?  This is my git image, and I'm
not working on git.  I made no changes.

Thus, something broken?

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: rsync update appears broken now

[Alex Riesen]

On 20 Oct 2005 07:12:53 -0700, Randal L. Schwartz <merlyn@stonehenge.com> wrote:
> Alex> Absolutely normal pull into a changed repository. Just fix the
> Alex> conflict (in fetch-pack.c, look for >>>), git-update-index the file
> Alex> and commit. Doesn't look like a problem at all.
>
> What do you mean "changed repository"?  This is my git image, and I'm
> not working on git.  I made no changes.
>
> Thus, something broken?
>

Not necessarily (I see this every morning, but I know precisely I
touched the sources).
Maybe local damage? What does git-status show?

Re: rsync update appears broken now

[Linus Torvalds]

On Thu, 20 Oct 2005, Alex Riesen wrote:
> 
> Absolutely normal pull into a changed repository. Just fix the
> conflict (in fetch-pack.c, look for >>>), git-update-index the file
> and commit. Doesn't look like a problem at all.

No, that's wrong.

The fact is, Junio has screwed up his repository, and if you merge, you'll 
never have something that matches _his_ repository.

To fix this, _Junio_ needs to fix his repository.

For those of us who have a separate branch to track the original (I call 
mine "parent), this tells the story:

	* refs/heads/parent: does not fast forward to branch 'master' of master.kernel.org:/pub/scm/git/git;
	  not updating.

IOW, it looks like Junio has screwed up his "master" branch on kernel.org 
and it no longer contains what it used to contain (and this is not a 
mirroring problem - I'm using "master.kernel.org" with no mirrors in 
between).

Doing a

	git fetch parent master:new-junio

followed by a

	gitk parent new-junio

shows that Junio seems to have re-based his "master" branch by removing 
his old top-most entry, and restarted the "master" branch, which is wrong, 
wrong, wrong.

Junio, please don't do that. It really screws people up. Now people can't 
fetch your head any more, and can't track you, because your branch isn't 
stable any more. I know you're very used to doing so in your "pu" branch, 
but it's _wrong_. It's wrong in "pu" too, but at least there you have an 
excuse.

Reparenting things is ok in _private_ branches, but it's not ok in public 
ones, since _others_ will have worked or at least downloaded the previous 
state.

The fact is, "distributed" fundamentally means "mistakes cannot be 
undone". Mistakes have to be _fixed_, not removed. Because the mistakes 
have already percolated to others.

			Linus

Re: rsync update appears broken now

[Junio C Hamano]

Linus Torvalds <torvalds <at> osdl.org> writes:

> For those of us who have a separate branch to track the original (I call 
> mine "parent), this tells the story:
> 
> 	* refs/heads/parent: does not fast forward to branch 'master' of
master.kernel.org:/pub/scm/git/git;
> 	  not updating.

Yes, I screwed up.  One honest mistake and another stupid one.

Note that "those of us" above include everybody who did "git-clone" to
set up the repository and left the default "Pull: master:origin" line in
.git/refs/remotes/origin.

> Junio, please don't do that. It really screws people up. Now people can't 
> fetch your head any more, and can't track you, because your branch isn't 
> stable any more.

Sorry, and yes that was the "honest mistake" part.  I know I should never
rewind beyond what I already have pushed out, and usually I carefully follow
that rule.

But this time I forgot that I pushed something out and also failed to re-check
what I pushed out before rewinding (the check is done by fetching from
master.kernel.org myself).

And another stupid mistake part was when I pushed out the result.  send-pack
correctly refused to update the master side, but I forced it without much
thinking.

Sorry about these mistakes.

Having said that, all is not lost.  Your message made it sound like the
new head was completely re-rooted and there is no common commit (which
almost gave me a heart attack), but I do not think it is that bad.  It is
more like pulling from two places.

               -----(A) head merlyn and everybody
             /          pulled from kernel.org previously
  --- common ------------------------------------(B) head rebased and pushed
                                                     out by mistake

The "parent" head you described in your message is A above, and what is on
kernel.org is B.  A is known as "origin" in the default setup git-clone makes.

If you have *no* further development on top of (A), the recovery is simple.

   $ git checkout master ;# make sure you are on your "master" branch.
   $ git fetch origin +master:origin ;# force "origin" to be (B)
   $ git reset --hard origin ;# reset your "master" branch to (B)

If you had any further development on top of (A), then the graph would look
like:
               -----(A)---------------(X) your tip
             /
  --- common ----------------------(B)

The merge conflict message you got when you pulled is result of trying to
merge (X) and (B).

There are two things we could do.

One is to forward port your changes between (A) and (X) on top of (B):

   $ git checkout master ;# make sure you are on your "master" branch.
   $ git tag anchor-a origin ;# stash (A) away
   $ git fetch origin +master:origin ;# force "origin" to be (B)
   $ git checkout -b rescue-stupid-junio origin
   $ rm -fr .dotest
   $ git format-patch -k --stdout anchor-a..master | git am -k -3

Depending on the changes between A and X, forward porting may result in
conflicting/unapplicable patches.  The individual changes are found in
.dotest/ directory and you can re-run 'git am -i' after fixing up the
conflicts.

Then validate the tip of the result to make sure forward porting did not lose
your changes:

   $ git diff origin..rescue-stupid-junio

If the result look OK, then we can make this your master branch.

   $ git checkout master
   $ git reset --hard rescue-stupid-junio

and clean things up by

   $ rm -f .git/refs/tags/anchor-a
   $ git branch -d rescue-stupid-junio

Another thing we could do is to treat as if this stupid maintainer briefly
had a twin who did concurrent development, and merge (X) and (B).

   $ git checkout master
   $ git pull origin +master:origin

This, as Merlyn and you saw, probably would result in merge conflicts which
need to be resolved manually.

Unfortunately I do not have access right now to commits between "common" and
(A) in the above picture (I am at work), so I cannot try any of the above out
myself.

Re: rsync update appears broken now

[Linus Torvalds]

On Thu, 20 Oct 2005, Junio C Hamano wrote:
> 
> Having said that, all is not lost.  Your message made it sound like the
> new head was completely re-rooted and there is no common commit (which
> almost gave me a heart attack), but I do not think it is that bad.  It is
> more like pulling from two places.

No. 

What you _must_ do to fix things up is that _you_ merge your current head 
with your old head, and then you push out the result.

At that point:

>                -----(A) head merlyn and everybody
>              /          pulled from kernel.org previously
>   --- common ------------------------------------(B) head rebased and pushed
>                                                      out by mistake

You should have a (C) that is the merge of both A and B (or a later 
commit, if you have other commits merged).

Now you can push out that merge, and everybody will be happy again: 
they'll see somethign that is a proper superset of whichever HEAD they 
happened to pull.

Don't make everybody else try to fix up your mistakes for you. It's easy 
enough for you to just fix it up, and everybody else can just pull.

Of course, anybody who did a pull and then did the merge by hand will 
always have their own private merge, and they should probably just undo 
that. But undoing _private_ stuff is ok.

		Linus

Re: rsync update appears broken now

[Junio Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> What you _must_ do to fix things up is that _you_ merge your current head 
> with your old head, and then you push out the result.

Yes.  That is what I will do when able (not right now).

I posted a workaround because I am at work and I do not have
access to kernel.org machine from here, and I did not pull
yesterday at work so I do not have an access to that lost side
branch until later today (I think I still have it in my private
repo, either on my home machine or my ~/git on hera).

> At that point:
>
>>                -----(A) head merlyn and everybody
>>              /          pulled from kernel.org previously
>>   --- common ------------------------------------(B) head rebased and pushed
>>                                                      out by mistake

Mind telling me the (A) commit ID if you know it?

Re: rsync update appears broken now

[Johannes Schindelin]

Hi,

On Thu, 20 Oct 2005, Junio Hamano wrote:

> > At that point:
> >
> >>                -----(A) head merlyn and everybody
> >>              /          pulled from kernel.org previously
> >>   --- common ------------------------------------(B) head rebased and pushed
> >>                                                      out by mistake
> 
> Mind telling me the (A) commit ID if you know it?

ea5a65a59916503d2a14369c46b1023384d51645

Also, people having merged with (A) would not have to undo that merge as 
was suggested. The new (C) commit would contain the revert of (A).

Ciao,
Dscho

Re: rsync update appears broken now

[Linus Torvalds]

On Fri, 21 Oct 2005, Johannes Schindelin wrote:
> 
> Also, people having merged with (A) would not have to undo that merge as 
> was suggested. The new (C) commit would contain the revert of (A).

Oh, they'll need to revert if they want to match Junios history. Otherwise 
they'll always generate yet another merge when they pull.

Doesn't matter that the _tree_ state may be the same, the only thing that 
matters is that their history will be different, and thus they will never 
be able to fast-forwared to Junio's tree unless they revert their local 
merge.

		Linus

Re: rsync update appears broken now

[Johannes Schindelin]

Hi,

On Thu, 20 Oct 2005, Linus Torvalds wrote:

> On Fri, 21 Oct 2005, Johannes Schindelin wrote:
> > 
> > Also, people having merged with (A) would not have to undo that merge as 
> > was suggested. The new (C) commit would contain the revert of (A).
> 
> Oh, they'll need to revert if they want to match Junios history. Otherwise 
> they'll always generate yet another merge when they pull.

So?

> Doesn't matter that the _tree_ state may be the same, the only thing that 
> matters is that their history will be different, and thus they will never 
> be able to fast-forwared to Junio's tree unless they revert their local 
> merge.

It does not have to be a fast-forward. After all, what is another merge? 
Since that merge does not have Junio as committer, close inspection of 
the commit will reveal that.

Of course, if you did not have local changes and merged, you might want to 
do a clean "git-fetch kernel.org +master:parent".

So, no need for a revert.

Ciao,
Dscho

Re: rsync update appears broken now

[Linus Torvalds]

On Fri, 21 Oct 2005, Johannes Schindelin wrote:
> > 
> > Oh, they'll need to revert if they want to match Junios history. Otherwise 
> > they'll always generate yet another merge when they pull.
> 
> So?

History matters. History matters as much as the data does.

> It does not have to be a fast-forward. After all, what is another merge? 
> Since that merge does not have Junio as committer, close inspection of 
> the commit will reveal that.

It _does_ have to be a fast-forward, if you expect the tree to have the 
same content as Junio's.

Even if it merges everything automatically, if the history is different, 
it could in theory at least merge _differently_ than what Junio had. Plus 
you'll have a really ugly version history for no good reason.

But yes, it will _work_. It just won't work the way people expect it to, 
so they'll make bug-reports about not being able to merge automatically, 
and others will say that they can't re-create it and blame the wrong 
person.

		Linus

Re: rsync update appears broken now

[Petr Baudis]

Dear diary, on Fri, Oct 21, 2005 at 02:19:42AM CEST, I got a letter
where Linus Torvalds <torvalds@osdl.org> told me that...
> On Fri, 21 Oct 2005, Johannes Schindelin wrote:
> > It does not have to be a fast-forward. After all, what is another merge? 
> > Since that merge does not have Junio as committer, close inspection of 
> > the commit will reveal that.
> 
> It _does_ have to be a fast-forward, if you expect the tree to have the 
> same content as Junio's.
> 
> Even if it merges everything automatically, if the history is different, 
> it could in theory at least merge _differently_ than what Junio had. Plus 
> you'll have a really ugly version history for no good reason.

And if Junio will want to merge with many of those people, you will
better have to get a dualhead for your gitk.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: rsync update appears broken now

[Linus Torvalds]

On Thu, 20 Oct 2005, Junio Hamano wrote:
> 
> Mind telling me the (A) commit ID if you know it?

The latest one I have is ea5a65a59916503d2a14369c46b1023384d51645, but if 
you had more pushed out at some point that I just didn't happen to pick 
up, that may not be the top-most (A).

		Linus

Re: rsync update appears broken now

[Junio C Hamano]

Linus Torvalds <torvalds <at> osdl.org> writes:

> The latest one I have is ea5a65a59916503d2a14369c46b1023384d51645, but if 
> you had more pushed out at some point that I just didn't happen to pick 
> up, that may not be the top-most (A).

I am reasonably sure that the screw-up was only rewinding one commit too much.
I've done the merge so things should look better once mirrors catch up.

Thanks for your help.

Re: rsync update appears broken now

[Linus Torvalds]

On Fri, 21 Oct 2005, Junio C Hamano wrote:
> 
> I am reasonably sure that the screw-up was only rewinding one commit too much.
> I've done the merge so things should look better once mirrors catch up.

Yup, works at least for me. Thx,

		Linus

Re: rsync update appears broken now

[Randal L. Schwartz]

>>>>> "Linus" == Linus Torvalds <torvalds@osdl.org> writes:

Linus> On Fri, 21 Oct 2005, Junio C Hamano wrote:
>> 
>> I am reasonably sure that the screw-up was only rewinding one commit too much.
>> I've done the merge so things should look better once mirrors catch up.

Linus> Yup, works at least for me. Thx,

Doesn't yet work for me.  What do I need to do now?

    * committish: 4ae22d96fe9248dac4f26b1fc91154ba5e879799
      branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
    * refs/heads/origin: same as branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
    Updating from ea5a65a59916503d2a14369c46b1023384d51645 to 4ae22d96fe9248dac4f26b1fc91154ba5e879799.
    fetch-pack.c: needs update
    fatal: Entry 'daemon.c' would be overwritten by merge. Cannot merge.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: rsync update appears broken now

[Linus Torvalds]

On Fri, 21 Oct 2005, Randal L. Schwartz wrote:
> 
> Doesn't yet work for me.  What do I need to do now?
> 
>     * committish: 4ae22d96fe9248dac4f26b1fc91154ba5e879799
>       branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
>     * refs/heads/origin: same as branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
>     Updating from ea5a65a59916503d2a14369c46b1023384d51645 to 4ae22d96fe9248dac4f26b1fc91154ba5e879799.
>     fetch-pack.c: needs update

Your previous failed pull had left the unfinished merge around, so now 
when you try to pull and it tries to update, it can't. 

Do a 

	git checkout -f

first to reset your tree to your old HEAD, and then try again. It should 
work then.

(Or do "git reset --hard", which should end up doing the same thing)

		Linus

Re: rsync update appears broken now

[Morten Welinder]

I see the very same with an http pull.

Morten

Re: rsync update appears broken now

[dave morgan]

On Thu, 20 Oct 2005 10:15:58 -0400, Morten Welinder
<mwelinder@gmail.com> wrote:

>I see the very same with an http pull.
>
>Morten
>-
>To unsubscribe from this list: send the line "unsubscribe git" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

I got the same with a git:// update

origin  git://git.kernel.org/pub/scm/git/git.git	

david@tower2:~/git$ cg-update
Up to date.

Applying changes...
Merging f8765797a41a39f4dfc7030098c38283e6461a83 ->
6e1c6c103c522d01829f3a63992a023ff031e851
        to ea5a65a59916503d2a14369c46b1023384d51645...
... Auto-merging fetch-pack.c
merge: warning: conflicts during merge

        Conflicts during merge. Do cg-commit after resolving them.

here is status after pull
david@tower2:~/git$ cg-status
Heads:
   >master      ea5a65a59916503d2a14369c46b1023384d51645
  R origin      6e1c6c103c522d01829f3a63992a023ff031e851

? git-build-rev-cache
? git-checkout-cache
? git-convert-cache
? git-diff-cache
? git-diff-helper
? git-export
? git-fsck-cache
? git-http-pull
? git-local-pull
? git-merge-cache
? git-rev-tree
? git-show-rev-cache
? git-update-cache

Dave

Re: rsync update appears broken now

[Stephen C. Tweedie]

Hi,

On Thu, 2005-10-20 at 05:47 -0700, Randal L. Schwartz wrote:
> Doing my daily git-pull now broke in this way (using yesterday's git version):
> ...
> * committish: 6e1c6c103c522d01829f3a63992a023ff031e851
>   branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
> * refs/heads/origin: does not fast forward to branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git;
>   not updating.

Seen here too.  My HEAD and HEAD^ as of yesterday's pull was:

commit ea5a65a59916503d2a14369c46b1023384d51645
Author: Junio C Hamano <junkio@cox.net>
Date:   Tue Oct 18 18:42:19 2005 -0700

    Do not ask for objects known to be complete.

    On top of optimization by Linus not to ask refs that already match, we
    can walk our refs and not issue "want" for things that are known to be
    reachable from them.

    Signed-off-by: Junio C Hamano <junkio@cox.net>

commit f8765797a41a39f4dfc7030098c38283e6461a83
Author: Junio C Hamano <junkio@cox.net>
Date:   Tue Oct 18 18:42:14 2005 -0700

    Even when overwriting tags, report if they are changed or not.

    Signed-off-by: Junio C Hamano <junkio@cox.net>

Today, looking at git-web I can see that that HEAD^ commit is still
there, but yesterday's HEAD is simply not in the commit chain any more:
there are a couple of other commits and then the "Do not ask for objects
known to be complete" commit appears with SHA
49bb805e69f97e75472e54a68e9eb24e08dee011.

Somebody broke this by rsyncing a non-superset tree to kernel.org,
maybe?

"git reset --hard HEAD^" got rid of the missing HEAD commit and allowed
me to pull from the new commit chain, but that really shouldn't be
necessary.

--Stephen