On refreshing the index

On refreshing the index

[Sergio Callegari]

Hi,

I have recently received instructions on how to refresh the index...
However, it looks like I've run into a case where the refresh is not as
successful as expected...

Here it is

If I run git status, git runs filters on a couple of opendocument files for
which a filter is defined

GIT_TRACE=1 git status
trace: built-in: git 'status'
trace: run_command: 'rezip -p ODF_UNCOMPRESS2'
trace: exec: 'sh' '-c' 'rezip -p ODF_UNCOMPRESS2' 'rezip -p ODF_UNCOMPRESS2'
trace: run_command: 'rezip -p ODF_UNCOMPRESS2'
trace: exec: 'sh' '-c' 'rezip -p ODF_UNCOMPRESS2' 'rezip -p ODF_UNCOMPRESS2'
# On branch M05
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#       WIP/
#       program.txt
#       program.txt~
nothing added to commit but untracked files present (use "git add" to track)

 ... but there is actually nothing new...
 so I try to update the index

GIT_TRACE=1 git update-index --refresh
trace: built-in: git 'update-index' '--refresh'
trace: run_command: 'rezip -p ODF_UNCOMPRESS2'
trace: exec: 'sh' '-c' 'rezip -p ODF_UNCOMPRESS2' 'rezip -p ODF_UNCOMPRESS2'
trace: run_command: 'rezip -p ODF_UNCOMPRESS2'
trace: exec: 'sh' '-c' 'rezip -p ODF_UNCOMPRESS2' 'rezip -p ODF_UNCOMPRESS2'

Is now status any faster?

GIT_TRACE=1 git status
trace: built-in: git 'status'
trace: run_command: 'rezip -p ODF_UNCOMPRESS2'
trace: exec: 'sh' '-c' 'rezip -p ODF_UNCOMPRESS2' 'rezip -p ODF_UNCOMPRESS2'
trace: run_command: 'rezip -p ODF_UNCOMPRESS2'
trace: exec: 'sh' '-c' 'rezip -p ODF_UNCOMPRESS2' 'rezip -p ODF_UNCOMPRESS2'
# On branch M05
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#       WIP/
#       program.txt
#       program.txt~
nothing added to commit but untracked files present (use "git add" to track)

No, the filters are run all the same...
What about git diff?

GIT_TRACE=1 git diff
trace: built-in: git 'diff'
trace: run_command: 'less'
trace: exec: 'less'
trace: run_command: 'rezip -p ODF_UNCOMPRESS2'
trace: exec: 'sh' '-c' 'rezip -p ODF_UNCOMPRESS2' 'rezip -p ODF_UNCOMPRESS2'
trace: run_command: 'rezip -p ODF_UNCOMPRESS2'
trace: exec: 'sh' '-c' 'rezip -p ODF_UNCOMPRESS2' 'rezip -p ODF_UNCOMPRESS2'

... really there is no change ...
But git keeps running the filters on these two files (note that there are other
OO files in the repo, and with the others, the filters are not run).

This happened on 1.7.0.2 after switching branches.

Sergio

Re: On refreshing the index

[Junio C Hamano]

Sergio Callegari <sergio.callegari@gmail.com> writes:

> If I run git status, git runs filters on a couple of opendocument files for
> which a filter is defined
>
> GIT_TRACE=1 git status
> trace: built-in: git 'status'
> trace: run_command: 'rezip -p ODF_UNCOMPRESS2'
> trace: exec: 'sh' '-c' 'rezip -p ODF_UNCOMPRESS2' 'rezip -p ODF_UNCOMPRESS2'
> trace: run_command: 'rezip -p ODF_UNCOMPRESS2'
> trace: exec: 'sh' '-c' 'rezip -p ODF_UNCOMPRESS2' 'rezip -p ODF_UNCOMPRESS2'
> # On branch M05
> # Untracked files:
> #   (use "git add <file>..." to include in what will be committed)
> #
> #       WIP/
> #       program.txt
> #       program.txt~
> nothing added to commit but untracked files present (use "git add" to track)

What does "git diff-files" and/or "git diff-index HEAD" say at this point?
If they do not say there are no difference, that means that the file on
the filesystem and the blob registered in the index are different, even
though after transmogrified with rezip (whatever it does) these two
different blobs may look the same.

"update-index --refresh" marks index entries for paths whose files on the
filesystem are identical to the blobs registered for them in the index as
up-to-date, so that later comparison do not have to inspect contents
(instead, it can notice something has changed by looking at stat data such
as file timestamps).

I think the difference between "may look the same" and "identical" is what
you are seeing.  Try "git add" on those paths and see what happens.

Re: On refreshing the index

[Sergio Callegari]

Junio C Hamano wrote:
> Sergio Callegari <sergio.callegari@gmail.com> writes:
>
>   
>> If I run git status, git runs filters on a couple of opendocument files for
>> which a filter is defined
>>
>> GIT_TRACE=1 git status
>> trace: built-in: git 'status'
>> trace: run_command: 'rezip -p ODF_UNCOMPRESS2'
>> trace: exec: 'sh' '-c' 'rezip -p ODF_UNCOMPRESS2' 'rezip -p ODF_UNCOMPRESS2'
>> trace: run_command: 'rezip -p ODF_UNCOMPRESS2'
>> trace: exec: 'sh' '-c' 'rezip -p ODF_UNCOMPRESS2' 'rezip -p ODF_UNCOMPRESS2'
>> # On branch M05
>> # Untracked files:
>> #   (use "git add <file>..." to include in what will be committed)
>> #
>> #       WIP/
>> #       program.txt
>> #       program.txt~
>> nothing added to commit but untracked files present (use "git add" to track)
>>     
>
> What does "git diff-files" and/or "git diff-index HEAD" say at this point?
> If they do not say there are no difference, that means that the file on
> the filesystem and the blob registered in the index are different, even
> though after transmogrified with rezip (whatever it does) these two
> different blobs may look the same.
>   
Neither git diff-files" and/or "git diff-index HEAD say nothing at this 
point...

> I think the difference between "may look the same" and "identical" is what
> you are seeing.  Try "git add" on those paths and see what happens.
>   
Is there any way I can find out which file is the guilty one since git 
diff-files says nothing? E.g. a trace telling me on what is the filter 
being called?

BTW... some notes that may be useful...

1) rezip is a mere recompressor.  It takes a zip file and re-creates it 
at zero compression, so that the git delta logic can do a good job on it 
on repacking.  I've found this useful on zip files, openoffice files, 
jar files, etc.

2) if I clone outside git, git update-index --refresh is always ok at 
making git status fast (i.e. not running expensive filters).

3) The problem always happens when I switch branches, right after the 
switch.

Sergio