Possible timestamp problems with diff-files?

Possible timestamp problems with diff-files?

[Marc Strapetz]

For our Git client, we are invoking

git diff-files--quiet --ignore-submodules

immediately after a commit of *all* changes. Hence, the expected exit
code would be 0 (because there are no changes). A user has now reported
that for commits with many changes, exit code is sometimes 1. For the
last incident, the commit was started at 15:24:11,820 and finished at
15:24:12,329, diff-files was invoked at 15:24:12,455 and failed with
exit code 1 at 15:24:21,394. A subsequent diff-files succeeded, so I'm
wondering now, if that could be a timestamp problem (maybe related to
the Index)?

Note that there are a couple of threads running in the client itself, so
I guess actual running time of diff-files was much shorter than 9
seconds and chances are that some parts of diff-files were run in the
same second 15:24:12.

System configuration is:
Linux splitter 2.6.38-11-server #48-Ubuntu SMP Fri Jul 29 19:20:32 UTC
2011 x86_64 x86_64 x86_64 GNU/Linux

git version 1.7.4.1

--
Best regards,
Marc Strapetz
=============
syntevo GmbH
http://www.syntevo.com
http://blog.syntevo.com

Re: Possible timestamp problems with diff-files?

[Jeff King]

On Tue, Sep 20, 2011 at 12:30:53PM +0200, Marc Strapetz wrote:

> For our Git client, we are invoking
> 
> git diff-files--quiet --ignore-submodules
> 
> immediately after a commit of *all* changes. Hence, the expected exit
> code would be 0 (because there are no changes). A user has now reported
> that for commits with many changes, exit code is sometimes 1. For the
> last incident, the commit was started at 15:24:11,820 and finished at
> 15:24:12,329, diff-files was invoked at 15:24:12,455 and failed with
> exit code 1 at 15:24:21,394. A subsequent diff-files succeeded, so I'm
> wondering now, if that could be a timestamp problem (maybe related to
> the Index)?

diff-files is scriptable plumbing, which means it is up to the script
writer to decide exactly when the index should be refreshed with respect
to the working tree files (because doing so could be kind of expensive,
as it needs to stat every file in the working tree). Have you tried
running "git update-index --refresh" just before your diff-files?

-Peff

Re: Possible timestamp problems with diff-files?

[Marc Strapetz]

On 20.09.2011 19:54, Jeff King wrote:
> On Tue, Sep 20, 2011 at 12:30:53PM +0200, Marc Strapetz wrote:
> 
>> For our Git client, we are invoking
>>
>> git diff-files--quiet --ignore-submodules
>>
>> immediately after a commit of *all* changes. Hence, the expected exit
>> code would be 0 (because there are no changes). A user has now reported
>> that for commits with many changes, exit code is sometimes 1. For the
>> last incident, the commit was started at 15:24:11,820 and finished at
>> 15:24:12,329, diff-files was invoked at 15:24:12,455 and failed with
>> exit code 1 at 15:24:21,394. A subsequent diff-files succeeded, so I'm
>> wondering now, if that could be a timestamp problem (maybe related to
>> the Index)?
> 
> diff-files is scriptable plumbing, which means it is up to the script
> writer to decide exactly when the index should be refreshed with respect
> to the working tree files (because doing so could be kind of expensive,
> as it needs to stat every file in the working tree). Have you tried
> running "git update-index --refresh" just before your diff-files?

My point is that "git diff-files --quiet" seems to returns 1 and when
invoked a short time later (without modifying working tree or invoking
any other Git command) it returns 0. This would indicate a bug in Git, I
guess. I think we can add more debug logging to our client to track down
that problem. However, to do that I'd need some input on what to log?

--
Best regards,
Marc Strapetz
=============
syntevo GmbH
http://www.syntevo.com
http://blog.syntevo.com

Re: Possible timestamp problems with diff-files?

[Junio C Hamano]

Marc Strapetz <marc.strapetz@syntevo.com> writes:

> On 20.09.2011 19:54, Jeff King wrote:
>> On Tue, Sep 20, 2011 at 12:30:53PM +0200, Marc Strapetz wrote:
>> 
>>> For our Git client, we are invoking
>>>
>>> git diff-files--quiet --ignore-submodules
>>>
>>> immediately after a commit of *all* changes. Hence, the expected exit
>>> code would be 0 (because there are no changes). A user has now reported
>>> that for commits with many changes, exit code is sometimes 1. For the
>>> last incident, the commit was started at 15:24:11,820 and finished at
>>> 15:24:12,329, diff-files was invoked at 15:24:12,455 and failed with
>>> exit code 1 at 15:24:21,394. A subsequent diff-files succeeded, so I'm
>>> wondering now, if that could be a timestamp problem (maybe related to
>>> the Index)?

What peff said already.

If you do not refresh the cached stat information, diff-files may report
"they differ" for a path that is otherwise unchanged without looking at
the contents of such a path to notice that the only difference is the
cached stat information (the whole and only point of having the cached
stat information is to avoid looking at the contents). Also, it may look
at the contents of such a path if it has a reason to suspect that the file
might have changed if it cannot tell from the cached stat information
(look for "racy-git" if you are really interested).

Update the cached stat information before you use plumbing commands in
your script.

Re: Possible timestamp problems with diff-files?

[Robin Rosenberg]

Junio C Hamano skrev 2011-09-21 23.33:
> Marc Strapetz<marc.strapetz@syntevo.com>  writes:
>
>> On 20.09.2011 19:54, Jeff King wrote:
>>> On Tue, Sep 20, 2011 at 12:30:53PM +0200, Marc Strapetz wrote:
>>>
>>>> For our Git client, we are invoking
>>>>
>>>> git diff-files--quiet --ignore-submodules
>>>>
>>>> immediately after a commit of *all* changes. Hence, the expected exit
>>>> code would be 0 (because there are no changes). A user has now reported
>>>> that for commits with many changes, exit code is sometimes 1. For the
>>>> last incident, the commit was started at 15:24:11,820 and finished at
>>>> 15:24:12,329, diff-files was invoked at 15:24:12,455 and failed with
>>>> exit code 1 at 15:24:21,394. A subsequent diff-files succeeded, so I'm
>>>> wondering now, if that could be a timestamp problem (maybe related to
>>>> the Index)?
>
> What peff said already.
>
> If you do not refresh the cached stat information, diff-files may report
> "they differ" for a path that is otherwise unchanged without looking at
> the contents of such a path to notice that the only difference is the
> cached stat information (the whole and only point of having the cached
> stat information is to avoid looking at the contents). Also, it may look
> at the contents of such a path if it has a reason to suspect that the file
> might have changed if it cannot tell from the cached stat information
> (look for "racy-git" if you are really interested).
>
> Update the cached stat information before you use plumbing commands in
> your script.

Using git diff instead of git diff-files would do just that, unless I am
missing something.

-- robin