[PATCH v2] Improve --assume-unchanged in the git update-index man page

[PATCH v2] Improve --assume-unchanged in the git update-index man page

[Philip Oakley]

This version 2 patch squashes in Junio's comments ($gmane/260915) and tidies up
the commit message. Previous series at $gmane/260901.

In $gmane/260837 the --assume-unchanged flag was reported as buggy because
of a misunderstanding about what it is being promised.

This clarifies the current situation, rather than change the code as proposed
by Duy in $gmane/260865.

Philip Oakley (1):
  doc: make clear --assume-unchanged's user contract

 Documentation/git-update-index.txt | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

-- 
1.9.4.msysgit.0

[PATCH v2] doc: make clear --assume-unchanged's user contract

[Philip Oakley]

Many users misunderstand the --assume-unchanged contract, believing
it means Git won't look at the flagged file.

Be explicit that the --assume-unchanged contract is by the user that
they will NOT change the file so that Git does not need to look (and
expend, for example, lstat(2) cycles)

Mentioning "Git stops checking" does not help the reader, as it is
only one possible consequence of what that assumption allows Git to
do, but

   (1) there are things other than "stop checking" that Git can do
       based on that assumption; and
   (2) Git is not obliged to stop checking; it merely is allowed to.

Also, this is a single flag bit, correct the plural to singular, and
the verb, accordingly.

Drop the stale and incorrect information about "poor-man's ignore",
which is not what this flag bit is about at all.

Signed-off-by: Philip Oakley <philipoakley@iee.org>
---
 Documentation/git-update-index.txt | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt
index e0a8702..da1ccbc 100644
--- a/Documentation/git-update-index.txt
+++ b/Documentation/git-update-index.txt
@@ -78,20 +78,18 @@ OPTIONS
         Set the execute permissions on the updated files.
 
 --[no-]assume-unchanged::
-	When these flags are specified, the object names recorded
-	for the paths are not updated.  Instead, these options
-	set and unset the "assume unchanged" bit for the
-	paths.  When the "assume unchanged" bit is on, Git stops
-	checking the working tree files for possible
-	modifications, so you need to manually unset the bit to
-	tell Git when you change the working tree file. This is
+	When this flag is specified, the object names recorded
+	for the paths are not updated.  Instead, this option
+	sets/unsets the "assume unchanged" bit for the
+	paths.  When the "assume unchanged" bit is on, the user
+	promises not to change the file and allows Git to assume
+	that the working tree file matches what is recorded in
+	the index.  If you want to change the working tree file,
+	you need to unset the bit to tell Git.  This is
 	sometimes helpful when working with a big project on a
 	filesystem that has very slow lstat(2) system call
 	(e.g. cifs).
 +
-This option can be also used as a coarse file-level mechanism
-to ignore uncommitted changes in tracked files (akin to what
-`.gitignore` does for untracked files).
 Git will fail (gracefully) in case it needs to modify this file
 in the index e.g. when merging in a commit;
 thus, in case the assumed-untracked file is changed upstream,
-- 
1.9.4.msysgit.0

Re: [PATCH v2] doc: make clear --assume-unchanged's user contract

[Sérgio Basto]

On Sáb, 2014-12-06 at 15:04 +0000, Philip Oakley wrote: 
> Many users misunderstand the --assume-unchanged contract, believing
> it means Git won't look at the flagged file.
> 
> Be explicit that the --assume-unchanged contract is by the user that
> they will NOT change the file so that Git does not need to look (and
> expend, for example, lstat(2) cycles)
> 
> Mentioning "Git stops checking" does not help the reader, as it is
> only one possible consequence of what that assumption allows Git to
> do, but
> 
>    (1) there are things other than "stop checking" that Git can do
>        based on that assumption; and
>    (2) Git is not obliged to stop checking; it merely is allowed to.
> 
> Also, this is a single flag bit, correct the plural to singular, and
> the verb, accordingly.
> 
> Drop the stale and incorrect information about "poor-man's ignore",
> which is not what this flag bit is about at all.
> 
> Signed-off-by: Philip Oakley <philipoakley@iee.org>
> ---
>  Documentation/git-update-index.txt | 18 ++++++++----------
>  1 file changed, 8 insertions(+), 10 deletions(-)
> 
> diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt
> index e0a8702..da1ccbc 100644
> --- a/Documentation/git-update-index.txt
> +++ b/Documentation/git-update-index.txt
> @@ -78,20 +78,18 @@ OPTIONS
>          Set the execute permissions on the updated files.
>  
>  --[no-]assume-unchanged::
> -	When these flags are specified, the object names recorded
> -	for the paths are not updated.  Instead, these options
> -	set and unset the "assume unchanged" bit for the
> -	paths.  When the "assume unchanged" bit is on, Git stops
> -	checking the working tree files for possible
> -	modifications, so you need to manually unset the bit to
> -	tell Git when you change the working tree file. This is
> +	When this flag is specified, the object names recorded
> +	for the paths are not updated.  Instead, this option
> +	sets/unsets the "assume unchanged" bit for the
> +	paths.  When the "assume unchanged" bit is on, the user
> +	promises not to change the file and allows Git to assume
> +	that the working tree file matches what is recorded in
> +	the index.  If you want to change the working tree file,
> +	you need to unset the bit to tell Git.  This is
>  	sometimes helpful when working with a big project on a
>  	filesystem that has very slow lstat(2) system call
>  	(e.g. cifs).
>  +
> -This option can be also used as a coarse file-level mechanism
> -to ignore uncommitted changes in tracked files (akin to what
> -`.gitignore` does for untracked files).
>  Git will fail (gracefully) in case it needs to modify this file
>  in the index e.g. when merging in a commit;
>  thus, in case the assumed-untracked file is changed upstream,

I don't understand why you insist that we have a contract, 
when : 
"git diff .", "git diff -a" and "git commit -a" have a different
behavior of "git commit ." , this is not about any contract this is
about coherency and be user friendly . 

At least if you want keep things like that, wrote in doc, clearly, that
assume-unchanged flag *is not*, to git ignoring changes in tracked files
and currently not ignore files for git commit <path> and may not work in
other cases . 

Also don't understand why --assumed-untracked shouldn't deal with
changed files instead fallback in "the user promises not to change the
file" and sometimes works others not. 

Also if this is the contract when a file is different from commit,
should warning the user that is not in contract (modify files that are
assumed-untracked ) 


Thanks, 
-- 
Sérgio M. B.

Re: [PATCH v2] doc: make clear --assume-unchanged's user contract

[Philip Oakley]

From: "Sérgio Basto" <sergio@serjux.com>
> On Sáb, 2014-12-06 at 15:04 +0000, Philip Oakley wrote:
>> Many users misunderstand the --assume-unchanged contract, believing
>> it means Git won't look at the flagged file.
>>
>> Be explicit that the --assume-unchanged contract is by the user that
>> they will NOT change the file so that Git does not need to look (and
>> expend, for example, lstat(2) cycles)
>>
>> Mentioning "Git stops checking" does not help the reader, as it is
>> only one possible consequence of what that assumption allows Git to
>> do, but
>>
>>    (1) there are things other than "stop checking" that Git can do
>>        based on that assumption; and
>>    (2) Git is not obliged to stop checking; it merely is allowed to.
>>
>> Also, this is a single flag bit, correct the plural to singular, and
>> the verb, accordingly.
>>
>> Drop the stale and incorrect information about "poor-man's ignore",
>> which is not what this flag bit is about at all.
>>
>> Signed-off-by: Philip Oakley <philipoakley@iee.org>
>> ---
>>  Documentation/git-update-index.txt | 18 ++++++++----------
>>  1 file changed, 8 insertions(+), 10 deletions(-)
>>
>> diff --git a/Documentation/git-update-index.txt 
>> b/Documentation/git-update-index.txt
>> index e0a8702..da1ccbc 100644
>> --- a/Documentation/git-update-index.txt
>> +++ b/Documentation/git-update-index.txt
>> @@ -78,20 +78,18 @@ OPTIONS
>>          Set the execute permissions on the updated files.
>>
>>  --[no-]assume-unchanged::
>> - When these flags are specified, the object names recorded
>> - for the paths are not updated.  Instead, these options
>> - set and unset the "assume unchanged" bit for the
>> - paths.  When the "assume unchanged" bit is on, Git stops
>> - checking the working tree files for possible
>> - modifications, so you need to manually unset the bit to
>> - tell Git when you change the working tree file. This is
>> + When this flag is specified, the object names recorded
>> + for the paths are not updated.  Instead, this option
>> + sets/unsets the "assume unchanged" bit for the
>> + paths.  When the "assume unchanged" bit is on, the user
>> + promises not to change the file and allows Git to assume
>> + that the working tree file matches what is recorded in
>> + the index.  If you want to change the working tree file,
>> + you need to unset the bit to tell Git.  This is
>>  sometimes helpful when working with a big project on a
>>  filesystem that has very slow lstat(2) system call
>>  (e.g. cifs).
>>  +
>> -This option can be also used as a coarse file-level mechanism
>> -to ignore uncommitted changes in tracked files (akin to what
>> -`.gitignore` does for untracked files).
>>  Git will fail (gracefully) in case it needs to modify this file
>>  in the index e.g. when merging in a commit;
>>  thus, in case the assumed-untracked file is changed upstream,
>
> I don't understand why you insist that we have a contract,

The misunderstanding (which even I had until recently), was the same as 
your misunderstanding, and is also repeated (wrongly) in the 
documentation that the flag is about ignoring changes to files.

That is completely wrong, and not why the flag was invented (as I 
understand it).

The flag was invented because Git bran very slowly on some file systems 
(e.g. cifs lstat(2) system calls) to the point that it was deemed 
unacceptable. The solution was to tell git to stop doing those lstat 
calls because the user had flagged it's unchanged nature.

The problem was then that, as a side effect, in the main, Git wouldn't 
notice any changes to such marked files (but some commands do notice;-) 
so some people used it as a "poor man's ignore" flag, and that 
misinformation stuck. Unfortunately you have been caught in that trap.

While many seek code (change) based solutions, I see clearer 
documentation (especially for the intermediate rather than expert users) 
as a good first step. People do read the documentation, though often in 
retrospect, so my patch seeks to clarify the existing situation. That is 
"please git, update the index, you can assume these files are 
unchanged".

Another thread has pointed out that the misunderstanding as also 
repeated on another page, so I'll need to see if a patch is forthcoming 
for that!

> when :
> "git diff .", "git diff -a" and "git commit -a" have a different
> behavior of "git commit ." , this is not about any contract this is
> about coherency and be user friendly .
>
> At least if you want keep things like that, wrote in doc, clearly, 
> that
> assume-unchanged flag *is not*, to git ignoring changes in tracked 
> files
> and currently not ignore files for git commit <path> and may not work 
> in
> other cases .
>
> Also don't understand why --assumed-untracked shouldn't deal with
> changed files instead fallback in "the user promises not to change the
> file" and sometimes works others not.
>
> Also if this is the contract when a file is different from commit,
> should warning the user that is not in contract (modify files that are
> assumed-untracked )
>
I hope my explanation helps in explaining how we have come to this 
perhaps unhelpful situation.
--
Philip 

Re: [PATCH v2] doc: make clear --assume-unchanged's user contract

[Philip Oakley]

From: "Philip Oakley" <philipoakley@iee.org>
> From: "Sérgio Basto" <sergio@serjux.com>
>> On Sáb, 2014-12-06 at 15:04 +0000, Philip Oakley wrote:
>>> Many users misunderstand the --assume-unchanged contract, believing
>>> it means Git won't look at the flagged file.
[...]
> retrospect, so my patch seeks to clarify the existing situation. That 
> is "please git, update the index, you can assume these files are 
> unchanged ... ".
... and if you don't  trust me, then you can even go and check 
[Honestly,  you can trust me I'm a developer]"
--
the extra bit.
Philip

Re: [PATCH v2] doc: make clear --assume-unchanged's user contract

[Michael J Gruber]

Sérgio Basto schrieb am 09.12.2014 um 04:43:
> On Sáb, 2014-12-06 at 15:04 +0000, Philip Oakley wrote: 
>> Many users misunderstand the --assume-unchanged contract, believing
>> it means Git won't look at the flagged file.
>>
>> Be explicit that the --assume-unchanged contract is by the user that
>> they will NOT change the file so that Git does not need to look (and
>> expend, for example, lstat(2) cycles)
>>
>> Mentioning "Git stops checking" does not help the reader, as it is
>> only one possible consequence of what that assumption allows Git to
>> do, but
>>
>>    (1) there are things other than "stop checking" that Git can do
>>        based on that assumption; and
>>    (2) Git is not obliged to stop checking; it merely is allowed to.
>>
>> Also, this is a single flag bit, correct the plural to singular, and
>> the verb, accordingly.
>>
>> Drop the stale and incorrect information about "poor-man's ignore",
>> which is not what this flag bit is about at all.
>>
>> Signed-off-by: Philip Oakley <philipoakley@iee.org>
>> ---
>>  Documentation/git-update-index.txt | 18 ++++++++----------
>>  1 file changed, 8 insertions(+), 10 deletions(-)
>>
>> diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt
>> index e0a8702..da1ccbc 100644
>> --- a/Documentation/git-update-index.txt
>> +++ b/Documentation/git-update-index.txt
>> @@ -78,20 +78,18 @@ OPTIONS
>>          Set the execute permissions on the updated files.
>>  
>>  --[no-]assume-unchanged::
>> -	When these flags are specified, the object names recorded
>> -	for the paths are not updated.  Instead, these options
>> -	set and unset the "assume unchanged" bit for the
>> -	paths.  When the "assume unchanged" bit is on, Git stops
>> -	checking the working tree files for possible
>> -	modifications, so you need to manually unset the bit to
>> -	tell Git when you change the working tree file. This is
>> +	When this flag is specified, the object names recorded
>> +	for the paths are not updated.  Instead, this option
>> +	sets/unsets the "assume unchanged" bit for the
>> +	paths.  When the "assume unchanged" bit is on, the user
>> +	promises not to change the file and allows Git to assume
>> +	that the working tree file matches what is recorded in
>> +	the index.  If you want to change the working tree file,
>> +	you need to unset the bit to tell Git.  This is
>>  	sometimes helpful when working with a big project on a
>>  	filesystem that has very slow lstat(2) system call
>>  	(e.g. cifs).
>>  +
>> -This option can be also used as a coarse file-level mechanism
>> -to ignore uncommitted changes in tracked files (akin to what
>> -`.gitignore` does for untracked files).
>>  Git will fail (gracefully) in case it needs to modify this file
>>  in the index e.g. when merging in a commit;
>>  thus, in case the assumed-untracked file is changed upstream,
> 
> I don't understand why you insist that we have a contract, 

Buy setting the bit, you are making the promise to Git: "You can assume
the file is unchanged without even checking."

> when : 
> "git diff .", "git diff -a" and "git commit -a" have a different
> behavior of "git commit ." , this is not about any contract this is
> about coherency and be user friendly . 

Git does not make the promise that it will not check.

> At least if you want keep things like that, wrote in doc, clearly, that
> assume-unchanged flag *is not*, to git ignoring changes in tracked files
> and currently not ignore files for git commit <path> and may not work in
> other cases . 
> 
> Also don't understand why --assumed-untracked shouldn't deal with
> changed files instead fallback in "the user promises not to change the
> file" and sometimes works others not. 
> 
> Also if this is the contract when a file is different from commit,
> should warning the user that is not in contract (modify files that are
> assumed-untracked ) 
> 
> 
> Thanks, 
> 

git update-index is a plumbing command, not a user frontend. If you use
it and bring workdir/index into an inconsistent state it's simply the
wrong use of a plumbing tool. Things tend to break when you use a
plumbing tool incorrectly ;)

That being said, there is some wrong advice in gitignore.txt that we
should remove.

In git-update-index.txt, we could try and spell this out even more clearly:

..allows Git to assume... in the index; nonetheless Git may check the
working tree file under some circumstances.

And maybe we could specify in all man pages the category of a command,
or a warning for plumbing commands ("plumbing - use at own risk").

Michael

[PATCH] gitignore.txt: do not suggest assume-unchanged

[Michael J Gruber]

git-update-index --assume-unchanged was never meant to ignore changes
to tracked files (only to spare some stats). So do not suggest it
as a means to achieve that.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
 Documentation/gitignore.txt | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/Documentation/gitignore.txt b/Documentation/gitignore.txt
index 09e82c3..4fd0442 100644
--- a/Documentation/gitignore.txt
+++ b/Documentation/gitignore.txt
@@ -138,9 +138,6 @@ NOTES
 The purpose of gitignore files is to ensure that certain files
 not tracked by Git remain untracked.
 
-To ignore uncommitted changes in a file that is already tracked,
-use 'git update-index {litdd}assume-unchanged'.
-
 To stop tracking a file that is currently tracked, use
 'git rm --cached'.
 
@@ -203,7 +200,6 @@ everything within `foo/bar`):
 SEE ALSO
 --------
 linkgit:git-rm[1],
-linkgit:git-update-index[1],
 linkgit:gitrepository-layout[5],
 linkgit:git-check-ignore[1]
 
-- 
2.2.0.345.g7041aac

Re: [PATCH v2] doc: make clear --assume-unchanged's user contract

[Junio C Hamano]

Sérgio Basto <sergio@serjux.com> writes:

> Also don't understand why --assumed-untracked shouldn't deal with
> changed files instead fallback in "the user promises not to change the
> file" and sometimes works others not. 
>
> Also if this is the contract when a file is different from commit,
> should warning the user that is not in contract (modify files that are
> assumed-untracked ) 

Unfortunately, that is not even possible in the case where
assume-untracked is meant to be used without breaking the use case
it was invented to support.  The user flips the bit so that Git does
not have to traverse the working tree to run lstat(2) on all of them
in order to see if some have changes relative to the index.

If we cannot trust that bit and need to verify, how would we do
that?

Think.

Yes, by running lstat(2) on these files that the user promised not
to touch.  That just defeats the sole objective the feature was
invented for in the first place.

Having said all that, I know what you wish to have, and I am not
fundamentally opposed to a change to add a new feature to order Git
to pretend that changes that may exist in the working tree are not
there.  It's just that assume-unchanged bit is not that.  It is a
way to allow Git that it can assume the files in the working tree
are not changed.  It is a permission, not a command.

This is a tangent, but somebody may want to check the "Git will fail
(gracefully)" bit in the documentation Philip's documentation patch
did not remove.  Such a detection is not absolutely necessary, and
the paragraph may be describing a wishful thinking by somebody who
misunderstood --assume-unchanged with a cursory observation of what
happened for limited test cases back when the documentation was
added, in which case that paragraph may also want to go.

Re: [PATCH] gitignore.txt: do not suggest assume-unchanged

[Jonathan Nieder]

Michael J Gruber wrote:

> git-update-index --assume-unchanged was never meant to ignore changes
> to tracked files (only to spare some stats). So do not suggest it
> as a means to achieve that.
[...]
> +++ b/Documentation/gitignore.txt
> @@ -138,9 +138,6 @@ NOTES
>  The purpose of gitignore files is to ensure that certain files
>  not tracked by Git remain untracked.
>  
> -To ignore uncommitted changes in a file that is already tracked,
> -use 'git update-index {litdd}assume-unchanged'.
> -
>  To stop tracking a file that is currently tracked, use
>  'git rm --cached'.

Makes sense.

But we need some advice to replace the paragraph you are deleting.  Is
the idea something like

	Git will not ignore uncommitted changes in a file that is already tracked.
	If you have time to work on that, please contact git@vger.kernel.org.

(perhaps without that second line)?

Thanks,
Jonathan

Re: [PATCH v2] doc: make clear --assume-unchanged's user contract

[Sérgio Basto]

On Ter, 2014-12-09 at 16:44 -0800, Junio C Hamano wrote: 
> Sérgio Basto <sergio@serjux.com> writes:
> 
> > Also don't understand why --assumed-untracked shouldn't deal with
> > changed files instead fallback in "the user promises not to change the
> > file" and sometimes works others not. 
> >
> > Also if this is the contract when a file is different from commit,
> > should warning the user that is not in contract (modify files that are
> > assumed-untracked ) 
> 
> Unfortunately, that is not even possible in the case where
> assume-untracked is meant to be used without breaking the use case
> it was invented to support. 

OK, but when we do: git --assumed-untracked <file> and file is already
modified. we could do an warning .

May also should mention in documentation that --assumed-untracked  is
not to deal with "local versions of tracked config files in git" (if you
don't) 

Meanwhile I read https://gist.github.com/canton7/1423106 
I adopt
https://gist.github.com/canton7/1423106#if-you-cant-modify-your-application 
item 1 

Thanks for your patience


>  The user flips the bit so that Git does
> not have to traverse the working tree to run lstat(2) on all of them
> in order to see if some have changes relative to the index.
> 
> If we cannot trust that bit and need to verify, how would we do
> that?
> 
> Think.
> 
> Yes, by running lstat(2) on these files that the user promised not
> to touch.  That just defeats the sole objective the feature was
> invented for in the first place.
> 
> Having said all that, I know what you wish to have, and I am not
> fundamentally opposed to a change to add a new feature to order Git
> to pretend that changes that may exist in the working tree are not
> there. 
> It's just that assume-unchanged bit is not that.  It is a
> way to allow Git that it can assume the files in the working tree
> are not changed.  It is a permission, not a command.
> 
> This is a tangent, but somebody may want to check the "Git will fail
> (gracefully)" bit in the documentation Philip's documentation patch
> did not remove.  Such a detection is not absolutely necessary, and
> the paragraph may be describing a wishful thinking by somebody who
> misunderstood --assume-unchanged with a cursory observation of what
> happened for limited test cases back when the documentation was
> added, in which case that paragraph may also want to go.
> 
> 
> 
> 
> 
> 

-- 
Sérgio M. B.

Re: [PATCH] gitignore.txt: do not suggest assume-unchanged

[Michael J Gruber]

Jonathan Nieder schrieb am 10.12.2014 um 02:06:
> Michael J Gruber wrote:
> 
>> git-update-index --assume-unchanged was never meant to ignore changes
>> to tracked files (only to spare some stats). So do not suggest it
>> as a means to achieve that.
> [...]
>> +++ b/Documentation/gitignore.txt
>> @@ -138,9 +138,6 @@ NOTES
>>  The purpose of gitignore files is to ensure that certain files
>>  not tracked by Git remain untracked.
>>  
>> -To ignore uncommitted changes in a file that is already tracked,
>> -use 'git update-index {litdd}assume-unchanged'.
>> -
>>  To stop tracking a file that is currently tracked, use
>>  'git rm --cached'.
> 
> Makes sense.
> 
> But we need some advice to replace the paragraph you are deleting.  Is
> the idea something like
> 
> 	Git will not ignore uncommitted changes in a file that is already tracked.
> 	If you have time to work on that, please contact git@vger.kernel.org.
> 
> (perhaps without that second line)?
> 
> Thanks,
> Jonathan

I seems you're missing that info only because you know what used to be
there.

If you reread - trying to read only what is there now - you will see
that the text clearly says: gitignore files are about ignoring untracked
files to ensure that they remain untracked.

But maybe I'm somewhat professionally preoccupied :)

That is: Even if it clearly says it is about untracked files, it doesn't
hurt to say what it is not about (namely tracked files). It's just that
it is not about many other things either.

Michael