gitignore design

gitignore design

[llucianf]

why gitignore doesnt simply work like in cvs where if you put something in
the ignore file, those stuff are simply ignored from that point without
having to remove them from repo?


--
View this message in context: http://git.661346.n2.nabble.com/gitignore-design-tp6632987p6632987.html
Sent from the git mailing list archive at Nabble.com.

Re: gitignore design

[Ferry Huberts]

On 07/29/2011 12:20 PM, llucianf wrote:
> why gitignore doesnt simply work like in cvs where if you put something in
> the ignore file, those stuff are simply ignored from that point without
> having to remove them from repo?
> 

because when it's in the repo you obviously want to track it...
tracking trumps ignoring


if you now suddenly do not want to track it anymore you have to remove
it and ignore it.


-- 
Ferry Huberts

Re: gitignore design

[llucianf]

sorry but this is not always the case. there are plenty of cases (project
files is most common example) in which i need files in repo but i do not
need git to track them. so why i cant just simply enumerate those project
files into .gitignore and 'persuade'  git to simply forget about them?


--
View this message in context: http://git.661346.n2.nabble.com/gitignore-design-tp6632987p6633274.html
Sent from the git mailing list archive at Nabble.com.

Re: gitignore design

[Ferry Huberts]

On 07/29/2011 02:01 PM, llucianf wrote:
> sorry but this is not always the case. there are plenty of cases (project
> files is most common example) in which i need files in repo but i do not
> need git to track them. so why i cant just simply enumerate those project

this is a contradiction: you want them tracked (they're in the repo) but
you do not want them tracked.

you can't have it both ways.

git works this way.

 git != cvs

-- 
Ferry Huberts

Re: gitignore design

[llucianf]

is not a contradiction. i need project files into repo so whenever i clone
the project on diff machine i need them there.
but during development i dont need them to be 'taken into account' by git. 
the purpose of this topic is for me to understand why git dont use the
simple cvs approach on this matter.
why i cant just enumerate some files into .gitignore file and have git
simply ignore them without removing them from repo?

--
View this message in context: http://git.661346.n2.nabble.com/gitignore-design-tp6632987p6633332.html
Sent from the git mailing list archive at Nabble.com.

Re: gitignore design

[Jakub Narebski]

llucianf <llucianf@gmail.com> writes:
> Ferry Huberts <mailings@hupie.com> writes:
> > On 07/29/2011 12:20 PM, llucianf wrote:

> > > why gitignore doesnt simply work like in cvs where if you put something in
> > > the ignore file, those stuff are simply ignored from that point without
> > > having to remove them from repo?

Are you sure that cvsignore file makes CVS to ignore changes in
_tracked_ files (in repository)?  CVS manual says:

  Ignoring files via cvsignore
  ============================

     There are certain file names that frequently occur inside your
  working copy, but that you don't want to put under CVS control.
  Examples are all the object files that you get while you compile your
  sources.  Normally, when you run `cvs update', it prints a line for
  each file it encounters that it doesn't know about.

Tracked files (in repository) are by definition not unknown to version
control system, be it CVS or Git.
 
> > because when it's in the repo you obviously want to track it...
> > tracking trumps ignoring
> > 
> > if you now suddenly do not want to track it anymore you have to remove
> > it and ignore it.
>
> sorry but this is not always the case.

But it is _usually_ the case, and that is why gitignore works as it
does, being only about _untracked_ files.

>                                        there are plenty of cases (project
> files is most common example) in which i need files in repo but i do not
> need git to track them. so why i cant just simply enumerate those project
> files into .gitignore and 'persuade'  git to simply forget about them?

For that you can use 'assume-unachanged' mechanism (note: it is local
to repository).  The gitignore(7) manpage says:

  NOTES
     The purpose of gitignore files is to ensure that certain files not tracked
     by git remain untracked.

     To ignore uncommitted changes in a file that is already tracked, use
     `git update-index --assume-unchanged <file>`.


Or just don't use 'git commit -a' and commit with dirty tree.

-- 
Jakub Narębski

Re: gitignore design

[Jakub Narebski]

llucianf <llucianf@gmail.com> writes:

> is not a contradiction. i need project files into repo so whenever i clone
> the project on diff machine i need them there.
> but during development i dont need them to be 'taken into account' by git. 

The usual solution is to put _template_ under version control (for
example <projfile>.sample), and have build system create initial
custom version during first build.  Local project file is untracked
and ignored, and project file template is tracked.

> the purpose of this topic is for me to understand why git dont use the
> simple cvs approach on this matter.

Are you sure that is how CVS does it?  Because from what I read in CVS
manual, CVS and Git approach to ignore files are the same wrt. already
tracked files.

> why i cant just enumerate some files into .gitignore file and have git
> simply ignore them without removing them from repo?

There is assume-unchanged mechanism... which is _explicitely_
mentioned in "Notes" section of gitignore(7) manpage.  RTFM, please.

-- 
Jakub Narębski

Re: gitignore design

[llucianf]

im sure cvs doesnt require you to remove files from repo in order to ignore
them. i used cvs for years and its ingonre policy is simple and effective.
you just put the files/patterns into ignore file and things happen aka they
are ignored.
with this very intelligent git this simple thing is not so simple. of course
there are workarounds (like the template example you gave) but they are
clumsy.
im just trying to understand why git ignore mechanism cant just read the
.gitignore file and obey to those ignore rules without asking you to do
fancy voodoo operations such removing those files from repo.


--
View this message in context: http://git.661346.n2.nabble.com/gitignore-design-tp6632987p6633412.html
Sent from the git mailing list archive at Nabble.com.

Re: gitignore design

[Jakub Narebski]

llucianf <llucianf@gmail.com> writes:

> im sure cvs doesnt require you to remove files from repo in order to ignore
> them. i used cvs for years and its ingonre policy is simple and effective.
> you just put the files/patterns into ignore file and things happen aka they
> are ignored.

_Untracked_ files are ignored.  Tracked files are not, even with CVS.

  $ echo file2.txt >>.cvsignore
  $ echo "3 line"  >>file2.txt

Now 'cvs status' shows file as 

  File: file2.txt         Status: Locally Modified

and 'cvs diff' shows changes.

CVS 1.11.19

[And damn, how hard it was to check this in CVS as compared to
 checking similar things with Git].

> with this very intelligent git this simple thing is not so simple. of course
> there are workarounds (like the template example you gave) but they are
> clumsy.

They are correct and better solutions than ignoring changes.

Ignoring changes to tracked files is much more rare than having broad
ignore file, and tracking some files that match ignore patterns (but
note that you must use "git add --force" to add/track ignored file).

> im just trying to understand why git ignore mechanism cant just read the
> .gitignore file and obey to those ignore rules without asking you to do
> fancy voodoo operations such removing those files from repo.

Please read carefully: I mentioned 'ASSUME-UNCHANGED' mechanism in
both of my posts, haven't I?

-- 
Jakub Narębski

Re: gitignore design

[Johannes Sixt]

Am 7/29/2011 14:19, schrieb Jakub Narebski:
> For that you can use 'assume-unachanged' mechanism (note: it is local
> to repository).  The gitignore(7) manpage says:
> 
>   NOTES
>      The purpose of gitignore files is to ensure that certain files not tracked
>      by git remain untracked.
> 
>      To ignore uncommitted changes in a file that is already tracked, use
>      `git update-index --assume-unchanged <file>`.

This statement in our documentation is *wrong*!! Please do not suggest it
for cases like the OP's!

See the discussion of assume-unchanged in git-update-index: This bit
actually means that git may assume that the file was not changed, and it
can take the worktree's data when it otherwise would have to unpack the
index's data. IOW, using it for the purposes that the OP would need is
*dangerous*.

-- Hannes

Re: gitignore design

[Jakub Narebski]

Johannes Sixt <j.sixt@viscovery.net> writes:
> Am 7/29/2011 14:19, schrieb Jakub Narebski:
> > For that you can use 'assume-unachanged' mechanism (note: it is local
> > to repository).  The gitignore(7) manpage says:
> > 
> >   NOTES
> >      The purpose of gitignore files is to ensure that certain files not tracked
> >      by git remain untracked.
> > 
> >      To ignore uncommitted changes in a file that is already tracked, use
> >      `git update-index --assume-unchanged <file>`.
> 
> This statement in our documentation is *wrong*!! Please do not suggest it
> for cases like the OP's!
> 
> See the discussion of assume-unchanged in git-update-index: This bit
> actually means that git may assume that the file was not changed, and it
> can take the worktree's data when it otherwise would have to unpack the
> index's data. IOW, using it for the purposes that the OP would need is
> *dangerous*.

Are you sure?  It seems to work as I thought it would.

  $ git init
  Initialized empty Git repository in /tmp/jnareb/test/.git/
  [master!test]$ echo foo >foo
  [master!test]$ echo bar >bar
  [master!test]$ echo bar >.gitignore
  [master!test]$ git add .
  [master!test]$ git commit -m Initial
  [master (root-commit) 522267b] Initial
   2 files changed, 2 insertions(+), 0 deletions(-)
   create mode 100644 .gitignore
   create mode 100644 foo
  [master!test]$ git add -f bar
  [master!test]$ git commit -m 'Add bar (ignored)'
  [master a708f70] Add bar (ignored)
   1 files changed, 1 insertions(+), 0 deletions(-)
   create mode 100644 bar
  [master!test]$ echo foo >>foo
  [master!test]$ echo bar >>bar
  [master!test]$ git status -s
   M bar
   M foo
  [master!test]$ git update-index --assume-unchanged bar
  [master!test]$ git status -s
   M foo
  [master!test]$ git commit -a -m "assume-unchanged bar, both changed"
  [master ec74f8e] assume-unchanged bar, both changed
   1 files changed, 1 insertions(+), 0 deletions(-)
  [master!test]$ git status -s
  [master!test]$ git show
  commit ec74f8e3f3f819bba22453324d7659fe8dd253e8
  Author: Jakub Narebski <jnareb@gmail.com>
  Date:   Fri Jul 29 44 2011 +0200
  
      assume-unchanged bar, both changed
  
  diff --git a/foo b/foo
  index 257cc56..0d55bed 100644
  --- a/foo
  +++ b/foo
  @@ -1 +1,2 @@
   foo
  +foo

Notice that change to 'bar' didn't get comitted.

-- 
Jakub Narębski

Re: gitignore design

[Johannes Sixt]

Am 7/29/2011 15:19, schrieb Jakub Narebski:
> Are you sure?  It seems to work as I thought it would.
> [...]
> Notice that change to 'bar' didn't get comitted.

Of course, it didn't get committed, you promised not to change it, so why
should git commit it?

However, your example does not show the dangerous part. git-commit is not
dangerous. But you might run into trouble when git has to merge content
into the worktree or index; in this case, git may decide to just read the
file instead of to unpack an object - assuming that the content on disk is
identical to the unpacked object (it will do so because with
--assume-unchanged you promised not to change the file). If you broke your
promise, you get to what you deserve ;)

No code reference, sorry, because I'm just parrotting what I've read
elsewhere on the list, for example,
http://thread.gmane.org/gmane.comp.version-control.git/146082/focus=146353

-- Hannes

Re: gitignore design

[Ferry Huberts]

On 07/29/2011 02:57 PM, Jakub Narebski wrote:
> llucianf <llucianf@gmail.com> writes:
> 
>> im sure cvs doesnt require you to remove files from repo in order to ignore
>> them. i used cvs for years and its ingonre policy is simple and effective.
>> you just put the files/patterns into ignore file and things happen aka they
>> are ignored.
> 
> _Untracked_ files are ignored.  Tracked files are not, even with CVS.


yeah llucianf, you could have checked this yourself...


the way it works now is safer: if someone by accident puts a tracked
file in an ignore file then the file is still not ignored.
you actually _want_ to see the file changes since you are after all
tracking the file.

git takes a safe route here.

having the file ignored while it is still in the repo can lead to very
very strange situation.

suppose someone puts a complicated ignore pattern in an ignore file that
has a mistake in it. then 'real' files might be ignored by the pattern
by git will still see the changes.

but to come back to cvs: cvs appears to do the same as git does so your
complaint is unfounded.

> 
>   $ echo file2.txt >>.cvsignore
>   $ echo "3 line"  >>file2.txt
> 
> Now 'cvs status' shows file as 
> 
>   File: file2.txt         Status: Locally Modified
> 
> and 'cvs diff' shows changes.
> 
> CVS 1.11.19
> 
> [And damn, how hard it was to check this in CVS as compared to
>  checking similar things with Git].
> 
>> with this very intelligent git this simple thing is not so simple. of course
>> there are workarounds (like the template example you gave) but they are
>> clumsy.
> 
> They are correct and better solutions than ignoring changes.
> 
> Ignoring changes to tracked files is much more rare than having broad
> ignore file, and tracking some files that match ignore patterns (but
> note that you must use "git add --force" to add/track ignored file).
> 
>> im just trying to understand why git ignore mechanism cant just read the
>> .gitignore file and obey to those ignore rules without asking you to do
>> fancy voodoo operations such removing those files from repo.
> 
> Please read carefully: I mentioned 'ASSUME-UNCHANGED' mechanism in
> both of my posts, haven't I?
> 


-- 
Ferry Huberts

Re: gitignore design

[Philip Oakley]

The answer is that you can use
   git add --force <filename>
to force git to take a copy of the current version of that file and place it 
in the staging area, despite it being in the ignore list.

 Being *in* the staging area means that that specific version will be copied 
into each commit until you ask git to remove that copy (with git rm 
<filename> from the staging area, or update that copy, or whatever.

This depends on how (i.e. the options) you add files before a commit, or do 
the commit itself, etc. For example the 'git add -A .' option will notice 
all the changes, including any updates to your <filename> (which may not be 
what you want).

Git is flexible enough to do what you need. It can be hard to see the effect 
of all the options especially if someone has shown you a 'magic' command, 
but hasn't fully explained the consequences. Many things are only obvious in 
retrospect.

Philip Oakley
Scotland
----- Original Message ----- 
From: "llucianf" <llucianf@gmail.com>
To: <git@vger.kernel.org>
Sent: Friday, July 29, 2011 11:20 AM
Subject: gitignore design


> why gitignore doesnt simply work like in cvs where if you put something in
> the ignore file, those stuff are simply ignored from that point without
> having to remove them from repo?
>
>
> --
> View this message in context: 
> http://git.661346.n2.nabble.com/gitignore-design-tp6632987p6632987.html
> Sent from the git mailing list archive at Nabble.com.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________
>
>
> -----
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 10.0.1390 / Virus Database: 1518/3793 - Release Date: 07/28/11
> 

Re: gitignore design

[Jakub Narebski]

On Fri, 29 Jul 2011, Johannes Sixt napisał:
> Am 7/29/2011 15:19, schrieb Jakub Narebski:

> > Are you sure?  It seems to work as I thought it would.
> > [...]
> > Notice that change to 'bar' didn't get comitted.
> 
> Of course, it didn't get committed, you promised not to change it, so why
> should git commit it?
> 
> However, your example does not show the dangerous part. git-commit is not
> dangerous. But you might run into trouble when git has to merge content
> into the worktree or index; in this case, git may decide to just read the
> file instead of to unpack an object - assuming that the content on disk is
> identical to the unpacked object (it will do so because with
> --assume-unchanged you promised not to change the file). If you broke your
> promise, you get to what you deserve ;)

True, it is *assume-unchanged*, not ignore-changes bit; though the latter
would be also possible to implement, I think... but having some file not
changing and marking it as such for better performance is saner use case
than tracking some file but not really tracking it.
 
> No code reference, sorry, because I'm just parrotting what I've read
> elsewhere on the list, for example,
> http://thread.gmane.org/gmane.comp.version-control.git/146082/focus=146353

Well, there is hint that there might be problems, but not really says
that they are, and where (if one is lying about assume unchanged by changing
assume-unchanged file).

-- 
Jakub Narębski
Poland

Re: gitignore design

[Nguyen Thai Ngoc Duy]

2011/7/30 Jakub Narebski <jnareb@gmail.com>
> True, it is *assume-unchanged*, not ignore-changes bit; though the latter
> would be also possible to implement, I think... but having some file not
> changing and marking it as such for better performance is saner use case
> than tracking some file but not really tracking it.

If you just want to ignore some files (or paths), then adding
--exclude option to diff machinery may be a better option.
--assume-unchanged is too low-level, and not really convenient to use.

> > No code reference, sorry, because I'm just parrotting what I've read
> > elsewhere on the list, for example,
> > http://thread.gmane.org/gmane.comp.version-control.git/146082/focus=146353
>
> Well, there is hint that there might be problems, but not really says
> that they are, and where (if one is lying about assume unchanged by changing
> assume-unchanged file).

There were problems in the past. See aecda37 (do not overwrite files
marked "assume unchanged" - 2010-05-01)

The only place that relies on checking files uptodate (which can be
faked by assume-unchanged bit) before overwriting them is
unpack-trees.c, verify_update_1(). With that fix in place, I think
assume-unchanged bit is safe now. However, as long as we don't
explicitly state that we will not carelessly overwrite
assume-unchanged files, there are still chances that some
optimizations in future may make it dangerous again.
--
Duy

Re: gitignore design

[Piotr Krukowiecki]

On Sat, Jul 30, 2011 at 5:10 AM, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
> 2011/7/30 Jakub Narebski <jnareb@gmail.com>
>> True, it is *assume-unchanged*, not ignore-changes bit; though the latter
>> would be also possible to implement, I think... but having some file not
>> changing and marking it as such for better performance is saner use case
>> than tracking some file but not really tracking it.
>
> If you just want to ignore some files (or paths), then adding
> --exclude option to diff machinery may be a better option.
> --assume-unchanged is too low-level, and not really convenient to use.

I want to have "exclude tracked" feature too, for the same reason as
llucianf wrote. I have project files which are
tracked but when I use them locally they might get changed (if I
modify some options).

The template project files are some solution, but they need changes to
the build process, which might not always be possible. Also, with
templates you won't have updates (like new project options) in your
local copy automatically.

The project files are updated very rarely so handling updates should
not be a big problem.

I think the "exclude tracked" option might work like this (all of this
with "unless asked explicitly" assumption):
- diff, stat do not show differences
- add, commit do not add changes
- stash do not stash changes
- merge tries to merge changes from upstream. If there's a conflict
you'll have to resolve it. OR
- merge does not touch your locally unchanged files if they have
changes - if upstream have changes you'll have to make your local
changes disappear (copy file, checkout orig file, merge, analyze what
changed, copy your local changed file back)
- checkout (different branch) - probably same as merge


>> > No code reference, sorry, because I'm just parrotting what I've read
>> > elsewhere on the list, for example,
>> > http://thread.gmane.org/gmane.comp.version-control.git/146082/focus=146353
>>
>> Well, there is hint that there might be problems, but not really says
>> that they are, and where (if one is lying about assume unchanged by changing
>> assume-unchanged file).
>
> There were problems in the past. See aecda37 (do not overwrite files
> marked "assume unchanged" - 2010-05-01)
>
> The only place that relies on checking files uptodate (which can be
> faked by assume-unchanged bit) before overwriting them is
> unpack-trees.c, verify_update_1(). With that fix in place, I think
> assume-unchanged bit is safe now. However, as long as we don't
> explicitly state that we will not carelessly overwrite
> assume-unchanged files, there are still chances that some
> optimizations in future may make it dangerous again.

I was using assume-unchanged for some time but stopped after some
weird problems during updates. I'm not sure if this was caused by this
or by sparse-checkout (and I use git-svn too). Anyway, after stopping
using assume-unchanged and sparse-checkout mysterious problems
disappeared.


-- 
Piotr Krukowiecki

Re: gitignore design

[Nguyen Thai Ngoc Duy]

On Sat, Jul 30, 2011 at 1:45 PM, Piotr Krukowiecki
<piotr.krukowiecki@gmail.com> wrote:
> I was using assume-unchanged for some time but stopped after some
> weird problems during updates. I'm not sure if this was caused by this
> or by sparse-checkout (and I use git-svn too). Anyway, after stopping
> using assume-unchanged and sparse-checkout mysterious problems
> disappeared.

I'm interested in the problems you had (even better if you found a way
to reproduce).
-- 
Duy

Re: gitignore design

[Piotr Krukowiecki]

On Sat, Jul 30, 2011 at 3:22 PM, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
> On Sat, Jul 30, 2011 at 1:45 PM, Piotr Krukowiecki
> <piotr.krukowiecki@gmail.com> wrote:
>> I was using assume-unchanged for some time but stopped after some
>> weird problems during updates. I'm not sure if this was caused by this
>> or by sparse-checkout (and I use git-svn too). Anyway, after stopping
>> using assume-unchanged and sparse-checkout mysterious problems
>> disappeared.
>
> I'm interested in the problems you had (even better if you found a way
> to reproduce).

Sorry, but that was some time ago and I don't remember details.
I could try using it again (which one are you interested in, sparse-checkout or
assume-unchanged?) and report back if I find any problems.


-- 
Piotr Krukowiecki

Re: gitignore design

[Clemens Buchacher]

On Sat, Jul 30, 2011 at 08:22:14PM +0700, Nguyen Thai Ngoc Duy wrote:
> On Sat, Jul 30, 2011 at 1:45 PM, Piotr Krukowiecki
> <piotr.krukowiecki@gmail.com> wrote:
> > I was using assume-unchanged for some time but stopped after some
> > weird problems during updates. I'm not sure if this was caused by this
> > or by sparse-checkout (and I use git-svn too). Anyway, after stopping
> > using assume-unchanged and sparse-checkout mysterious problems
> > disappeared.
> 
> I'm interested in the problems you had (even better if you found a way
> to reproduce).

Hi, Same here.

Concerning the OP's question, I've also written this FAQ entry,
which explains two methods to deal with the problem:

 https://git.wiki.kernel.org/index.php/GitFaq#How_do_I_tell_git_to_ignore_tracked_files.3F

It also mentions assume-unchanged and sparse checkout as a third
option, but it warns that these features were not designed for that
purpose. Apart from the bug fixed in aecda37 I have never had a
problem with them myself. But it's not a particularly convenient
solution in any case.

As far as a possible 'exclude untracked' mechanism is concerned, I
am not sure that is a good thing to have. It's like saying "I want
to track changes to those files, but I do not want to commit
changes by default (e.g. commit -a, add -u etc.)." That may sound
reasonable at first, but I think the desire to ignore changes to
tracked files usually indicates a design problem. And it can almost
always be solved using either option (a) or (b) from the FAQ entry
above.

On the other hand, such a feature bears some risk.  The repository
is not guaranteed to be in a certain state, even if git status is
empty.  You could still have ignored changes somewhere. And it's
all too easy to forget about those. I know I always forget about
the rules I have in my .git/info/exclude.

Maybe I should not be trying to protect users from shooting
themselves in the foot. But I would be very curious to hear from
the OP why options (a) or (b) above are not a solution for his use
case, before adding yet another "dangerous" mechanism to git.

Clemens