"warning: no common commits" triggered due to change of remote's IP address?

"warning: no common commits" triggered due to change of remote's IP address?

[Brent Goodrick]

Hi,

I had this setup in my .git/config on my satellite machine:

[remote "origin"]
	url = 192.168.2.3:git.repos/environ.git
	fetch = +refs/heads/home:refs/remotes/origin/home

192.168.2.3 is the IP address of the main machine and repo, and is
internal to my LAN at home. My workflow is fairly simple: At home, on
my internal LAN, I usually update via these commands:

  git fetch
  git branch # <-- verify I am on the "home" branch
  git merge origin/home  # <-- merge origin/home into home and fix conflicts

I needed to do a git fetch of that same repository while out at an
Internet cafe (via ssh). I know ahead of time that I might have had a
couple of files out of date between my satellite machines repo and my
origin repo, but certainly not megabytes of data in difference. I want
to pull those differences to the satellite machines repo to continue
to work.

On the satellite machine, I simply did the fetch manually by changing
the IP address to be the WAN internet IP address of my ssh daemon I'm
running at home:

  gitw fetch 88.99.100.101:git.repos/environ.git
+refs/heads/home:refs/remotes/origin/home

My expectation at this point is that, since I've changed only the IP
address, and kept everything else the same, git should be smart enough
to compare SHA1 values only and not download the entire remote repo
just to do that comparison.

But I was quite surprised to find that it was pulling down tons of data:

  warning: no common commits
  remote: Counting objects: 2473, done.
  remote: Compressing objects: 100% (2199/2199), done.
  Receiving objects:  83% (2077/2473), 66.58 MiB | 67 KiB/s     C-c C-c

At this point, for some reason I can't explain, the network got very
slow (network bandwidth limiting?). So, I terminated the git fetch
(the C-c C-c above), thinking that I can repair this when I get back
to my LAN.

So my questions are:

 1. Will terminating the git fetch like I did leave the satellite repo
    in an inconsistent state? If so, is my only choice to start
    a new repo from scratch on the satellite machine, or is there some
    repair mechanism?

 2. Why did git conclude that there was no common commits?

Thanks,
Brent

Re: "warning: no common commits" triggered due to change of remote's IP address?

[Thomas Rast]

[-- Attachment #1: Type: text/plain, Size: 2099 bytes --]

Brent Goodrick wrote:
> My expectation at this point is that, since I've changed only the IP
> address, and kept everything else the same, git should be smart enough
> to compare SHA1 values only and not download the entire remote repo
> just to do that comparison.
> 
> But I was quite surprised to find that it was pulling down tons of data:

Git doesn't care about the details of the transport; during its
handshake, lists of the available refs are exchanged, and then used to
determine the common commits.  So I'm also rather surprised.

However, your use of + refspecs in

>   gitw fetch 88.99.100.101:git.repos/environ.git
> +refs/heads/home:refs/remotes/origin/home

makes me wonder: have you rewritten the repo hosting 'home' between
two fetches?  Using (especially, but not only) git-filter-branch can
easily render your history disjoint from the pre-filtering state.

>   warning: no common commits

Either your history is very short and really has no common commits
whatsoever, or it gave up because of the 256 revision limit during
find_common().

IIRC it walks by date, so it is enough to make 256 local commits with
a new timestamp to hit that limit.  I posted some experimental code
two months ago that would use a bisection algorithm, so that it is
harder to hit the limit and faster at detecting disjoint history, but
nobody had the time to review it.

I'll follow up with the patch if you want to try it.  The problem is
it's quite large _and_ I don't run any git servers where I could give
it a good testing.  You'll have to apply it to both sides.

(It does have the nice side-effect of saving the uploading side a bit
of work.)

>  1. Will terminating the git fetch like I did leave the satellite repo
>     in an inconsistent state? If so, is my only choice to start
>     a new repo from scratch on the satellite machine, or is there some
>     repair mechanism?

It will just leave a temporary pack file that git-gc will eventually
remove.  You can just try another fetch later.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: "warning: no common commits" triggered due to change of remote's IP address?

[Brent Goodrick]

On Sun, Mar 1, 2009 at 1:20 PM, Thomas Rast <trast@student.ethz.ch> wrote:
> However, your use of + refspecs in
>
>>   gitw fetch 88.99.100.101:git.repos/environ.git
>> +refs/heads/home:refs/remotes/origin/home
>
> makes me wonder: have you rewritten the repo hosting 'home' between
> two fetches?  Using (especially, but not only) git-filter-branch can
> easily render your history disjoint from the pre-filtering state.
>
>>   warning: no common commits
>
> Either your history is very short and really has no common commits
> whatsoever, or it gave up because of the 256 revision limit during
> find_common().

Hmmm, maybe, without knowing it. Originally, that section of the
.git/config file had "*"'s where "home" was. To clarify, the original
was:

[remote "origin"]
	url = <some_ip_address>:git.repos/environ.git
	fetch = +refs/heads/*:refs/remotes/origin/*

and the current one is now:

[remote "origin"]
	url = <some_ip_address>:git.repos/environ.git
	fetch = +refs/heads/home:refs/remotes/origin/home

Maybe I had made that change and this is the first time I am doing a
fetch to using that change. I thinking that was the cause of this,
because I retried doing a fetch into a separate throw-away repo with
just the change of IP address, and it did not need to fetch anything
more. I had not executed git-filter-branch at all.

>>  1. Will terminating the git fetch like I did leave the satellite repo
>>     in an inconsistent state? If so, is my only choice to start
>>     a new repo from scratch on the satellite machine, or is there some
>>     repair mechanism?
>
> It will just leave a temporary pack file that git-gc will eventually
> remove.  You can just try another fetch later.

Good, that is what I would have expected.

Brent

Re: "warning: no common commits" triggered due to change of remote's IP address?

[Thomas Rast]

[-- Attachment #1: Type: text/plain, Size: 1935 bytes --]

Brent Goodrick wrote:
> On Sun, Mar 1, 2009 at 1:20 PM, Thomas Rast <trast@student.ethz.ch> wrote:
> > [...] have you rewritten the repo hosting 'home' between
> > two fetches?  Using (especially, but not only) git-filter-branch can
> > easily render your history disjoint from the pre-filtering state.
> 
> Hmmm, maybe, without knowing it.

It's rather hard to rewrite history without knowing it, there are big
warnings all over the relevant tools' manpages...

> Originally, that section of the
> .git/config file had "*"'s where "home" was. To clarify, the original
> was:
> 
> [remote "origin"]
> 	url = <some_ip_address>:git.repos/environ.git
> 	fetch = +refs/heads/*:refs/remotes/origin/*
> 
> and the current one is now:
> 
> [remote "origin"]
> 	url = <some_ip_address>:git.repos/environ.git
> 	fetch = +refs/heads/home:refs/remotes/origin/home
> 
> Maybe I had made that change and this is the first time I am doing a
> fetch to using that change. I thinking that was the cause of this,
> because I retried doing a fetch into a separate throw-away repo with
> just the change of IP address, and it did not need to fetch anything
> more. I had not executed git-filter-branch at all.

Ironically I cannot reproduce this except with my "own" version that
includes the patch I posted yesterday.  I'll have to look into why it
fails to list any refs to the remote.  In the meantime please
disregard that patch.

If you still have a repo that can reproduce the problem, please keep a
copy for future investigation, and then try

  git fetch-pack -v $url refs/remotes/origin/home 2>&1 \
  | git name-rev --stdin

The -v will dump a lot of output about the common commit search.  The
message "giving up" indicates that you hit the 256 commit limit; if
that doesn't appear, please include the full output so that we can see
where it stops.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: "warning: no common commits" triggered due to change of remote's IP address?

[Thomas Rast]

[-- Attachment #1: Type: text/plain, Size: 434 bytes --]

Thomas Rast wrote:
> If you still have a repo that can reproduce the problem, please keep a
> copy for future investigation, and then try
> 
>   git fetch-pack -v $url refs/remotes/origin/home 2>&1 \
>   | git name-rev --stdin

Actually this should name the remote's idea of the ref, i.e.,

  git fetch-pack -v $url refs/heads/home 2>&1 \
  | git name-rev --stdin

Sorry.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: "warning: no common commits" triggered due to change of remote's IP address?

[Brent Goodrick]

On Mon, Mar 2, 2009 at 12:56 AM, Thomas Rast <trast@student.ethz.ch> wrote:
> Actually this should name the remote's idea of the ref, i.e.,
>
>  git fetch-pack -v $url refs/heads/home 2>&1 \
>  | git name-rev --stdin

Sorry, but I had since done a git fetch from home using the local IP
address before I realized that you would want to try more
experiments on it. But that "local" git-fetch did not emit the
"warning: no common commits" message and was fast. Maybe the majority
of the fetch I had started at the Internet cafe but terminated had
done a lion share of the work already, and the speedy local fetch
finished up the leftovers.

But for completeness sake, I did as you said on the now (mostly) up to
date repo:

  git fetch-pack -v <remote_ip_address>:git.repos/environ.git
refs/heads/home 2>&1 \
   | git name-rev --stdin
  Server supports multi_ack
  Server supports side-band-64k
  Marking 67cb0521a93778a9d9c4d8f4608f2c6c796a7558 (home) as complete
  already have 67cb0521a93778a9d9c4d8f4608f2c6c796a7558 (home) (refs/heads/home)
  67cb0521a93778a9d9c4d8f4608f2c6c796a7558 (home) refs/heads/home

where <remote_ip_address> is the remote IP address, and not the usual
local IP address that is referenced in .git/config.

What does "Marking ... as complete" mean?

Is the git-fetch-pack method shown above the only way to get a report
of what git finds out of date? Seems that git-fetch-pack actually
downloads objects, instead of just reporting what it would do without
actually doing it.

Brent