git repository modified after migration

git repository modified after migration

[Yang Yu]

I migrated a 11G git repository converted from svn on a host with
Debian 8.2, reiserfs, git 2.1.4 to a host with Ubuntu 12.04.5 LTS,
xfs, git 2.6.4. After the migration, `git status` showing a good
amount of files modified.

I did the transfer with
1) `rsync -azP`, after noticing the modified files I ran `rsync -avH
--delete` but it did not correct the problem
2) tar zcf, then on the destination download the tar.gz (served by
nginx) with wget

Both had the same result. But the original repository was still clean.

I did some comparison between "modified" and original files
a) same hash (md5sum, shasum)
b) same permission (-rw-r--r-- 1 )
c) same encoding and line termination (UTF-8 Unicode (with BOM) text,
with CRLF line terminators)
d) no git attributes for either


On the destination host, I ran `git checkout` on each of those
modified files. After one pass I got less modified files. Repeating
`git checkout` on remaining files for a few more times, finally I got
a clean repository on the destination host.

What could have caused git to consider those files as modified? And
why multiple `git checkout` again the same file was necessary?

Thanks.


Yang

Re: git repository modified after migration

[Michael J Gruber]

Yang Yu venit, vidit, dixit 25.12.2015 21:49:
> I migrated a 11G git repository converted from svn on a host with
> Debian 8.2, reiserfs, git 2.1.4 to a host with Ubuntu 12.04.5 LTS,
> xfs, git 2.6.4. After the migration, `git status` showing a good
> amount of files modified.
> 
> I did the transfer with
> 1) `rsync -azP`, after noticing the modified files I ran `rsync -avH
> --delete` but it did not correct the problem
> 2) tar zcf, then on the destination download the tar.gz (served by
> nginx) with wget
> 
> Both had the same result. But the original repository was still clean.
> 
> I did some comparison between "modified" and original files
> a) same hash (md5sum, shasum)
> b) same permission (-rw-r--r-- 1 )
> c) same encoding and line termination (UTF-8 Unicode (with BOM) text,
> with CRLF line terminators)
> d) no git attributes for either
> 
> 
> On the destination host, I ran `git checkout` on each of those
> modified files. After one pass I got less modified files. Repeating
> `git checkout` on remaining files for a few more times, finally I got
> a clean repository on the destination host.
> 
> What could have caused git to consider those files as modified? And
> why multiple `git checkout` again the same file was necessary?
> 
> Thanks.
> 
> 
> Yang

This happens whenever the "stat" information changes, e.g. due to
changed device numbering and such. "git reset --hard" would have been
the quickiest way to reset the stat cache/index - after git diff, of
course ;)

Michael

Re: git repository modified after migration

[Yang Yu]

On Mon, Jan 11, 2016 at 9:04 AM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> This happens whenever the "stat" information changes, e.g. due to
> changed device numbering and such. "git reset --hard" would have been
> the quickiest way to reset the stat cache/index - after git diff, of
> course ;)
>


Can you go into a bit more details about this? Why does it only affect
certain files (always these filed modified) on every host I tried?
Thanks.

Re: git repository modified after migration

[Junio C Hamano]

Michael J Gruber <git@drmicha.warpmail.net> writes:

> This happens whenever the "stat" information changes, e.g. due to
> changed device numbering and such. "git reset --hard" would have been
> the quickiest way to reset the stat cache/index - after git diff, of
> course ;)

That does not quite explain why 'git status' reported modified files
in the first place.  It would have refreshed the cached stat info in
the index as the first thing to do.  "git status" I think is the
recommended way these days ("update-index --refresh" for us old
timers) to nondestructively correct the cached stat information
discrepancy caused by "cp -R".

If you need to resort to "reset --hard", then there is something
else going on.

Re: git repository modified after migration

[Michael J Gruber]

Junio C Hamano venit, vidit, dixit 11.01.2016 19:19:
> Michael J Gruber <git@drmicha.warpmail.net> writes:
> 
>> This happens whenever the "stat" information changes, e.g. due to
>> changed device numbering and such. "git reset --hard" would have been
>> the quickiest way to reset the stat cache/index - after git diff, of
>> course ;)
> 
> That does not quite explain why 'git status' reported modified files
> in the first place.  It would have refreshed the cached stat info in
> the index as the first thing to do.  "git status" I think is the
> recommended way these days ("update-index --refresh" for us old
> timers) to nondestructively correct the cached stat information
> discrepancy caused by "cp -R".
> 
> If you need to resort to "reset --hard", then there is something
> else going on.

Back than when I had the same problem with git repos on removable file
systems (if I remember correctly) git status did not correct that
information. It may be different now.

Michael

Re: git repository modified after migration

[Junio C Hamano]

Michael J Gruber <git@drmicha.warpmail.net> writes:

> Junio C Hamano venit, vidit, dixit 11.01.2016 19:19:
>> Michael J Gruber <git@drmicha.warpmail.net> writes:
>> 
>>> This happens whenever the "stat" information changes, e.g. due to
>>> changed device numbering and such. "git reset --hard" would have been
>>> the quickiest way to reset the stat cache/index - after git diff, of
>>> course ;)
>> 
>> That does not quite explain why 'git status' reported modified files
>> in the first place.  It would have refreshed the cached stat info in
>> the index as the first thing to do.  "git status" I think is the
>> recommended way these days ("update-index --refresh" for us old
>> timers) to nondestructively correct the cached stat information
>> discrepancy caused by "cp -R".
>> 
>> If you need to resort to "reset --hard", then there is something
>> else going on.
>
> Back than when I had the same problem with git repos on removable file
> systems (if I remember correctly) git status did not correct that
> information. It may be different now.

I do not recall we did anything specific to help removable devices,
so if the report is coming from a filesystem on a removable device
we may be seeing the same symptom.

I somehow doubt that is the case, though.

A not-entirely-inplausible theory is that the index in the original
repository somehow marks modified entries as clean, fooling "status"
that is run in the original repository into reporting that nothing
changed.  Because "cp -R" into a different location forces the
content-level checking, "status" run in the copy notices that they
are indeed different and tells the true story.  If that is the case,
it would be interesting to see how the index got into that state.