Feature Request - Hide ignored files before checkout

Feature Request - Hide ignored files before checkout

[Matthew Ciancio]

To whom it may concern,

I am not sure if this is the right place to send this, but I couldn't find
anything on the web that seemed official, so here goes.

Imagine this scenario:

1) You have a Git repo with two branches (branchA and branchB), which are
currently identical.
2) Checkout to branch.
3) Create file foo.txt, stage it and commit it.
4) Create file ignore.txt and add it to the ".gitignore" file of branchB so
that it is successfully ignored by Git.
5) Checkout to branchA.

Problem: ignore.txt does not "disappear" like foo.txt does and is now just
sitting in branchA (and now any other branch I checkout into).

When I first started using Git, I genuinely thought this was a bug, because
it seems so logical to me that ignore files should hide/reappear just like
tracked files do, when switching branches.
I have been told ways of circumventing this (using commits and un-commits OR
using stash), but my reason for avoiding commits is: say you have binary/OS
specific files which really do not belong in the commit logs (even locally)
and hence should be ignored. What if you want those files in only one branch
and not all?
Stashing doesn't seem appropriate either, because it would get messy.

Do you think this warrants a feature request?

If so, I was thinking that maybe the .gitignore file could have a flag after
each entry, to indicate whether the file(s)/folder(s) should have this new
feature or not (that way it would cater for everyone, but I can't see why
you wouldn't want this behaviour). 

If you like this idea and submit this as a feature request, please respond
with a link of the official request so that I can follow it, or provide me
with a link to submit it myself.

P.S. Here is a forum post I made on StackOverflow about the issue:
http://stackoverflow.com/questions/13761682/gitignore-hide-ignored-files-dur
ing-checkout

Kind regards,
Matt

Re: Feature Request - Hide ignored files before checkout

[Chris Rorvick]

On Fri, Dec 7, 2012 at 6:50 PM, Matthew Ciancio
<matthew.ciancio16@gmail.com> wrote:
> Imagine this scenario:
>
> 1) You have a Git repo with two branches (branchA and branchB), which are
> currently identical.
> 2) Checkout to branch.
> 3) Create file foo.txt, stage it and commit it.
> 4) Create file ignore.txt and add it to the ".gitignore" file of branchB so
> that it is successfully ignored by Git.
> 5) Checkout to branchA.
>
> Problem: ignore.txt does not "disappear" like foo.txt does

When you say "disappear", do you mean "deleted from the file system"
or something else?

> and is now just
> sitting in branchA (and now any other branch I checkout into).

It's not in branchA, it's just no longer ignored because your changes
to .gitignore were effectively reverted by jumping back to the commit
that branchA points to.

> When I first started using Git, I genuinely thought this was a bug, because
> it seems so logical to me that ignore files should hide/reappear just like
> tracked files do, when switching branches.

"hide/reappear" is the equivalent to saying "deleted/created" in the
case of a tracked file in your working tree.  But how would Git cause
an untracked file to reappear?  By definition, it doesn't know
anything about the file.

> P.S. Here is a forum post I made on StackOverflow about the issue:
> http://stackoverflow.com/questions/13761682/gitignore-hide-ignored-files-dur
> ing-checkout

I posted an answer there, too, but I'm not sure I fully understand the
problem.  The top-voted answer doesn't make much sense, though.

Thanks,

Chris

Re: Feature Request - Hide ignored files before checkout

[Chris Rorvick]

On Sat, Dec 8, 2012 at 6:06 PM, Matthew Ciancio
<matthew.ciancio16@gmail.com> wrote:
> Hi Chris,
>
> Yes, I don't think I have explained myself well enough.
>
> When I say "disappear" I do not mean "get deleted", I mean: go out of view
> just like foo.txt does, as it is committed to branchB and not merged into
> branchA.
>
> So I am saying that I think .gitignored files should behave partly like
> committed and un-merged files, in the sense that they disappear when
> checking out to a different branch.
> I don't want to commit these files (which would give me the behaviour I
> want), because they are binary/OS specific and really do not belong in the
> repository, BUT I need them to run/build certain committed files.
>
> To be concrete: I want ignore.txt to be ignored in branchB and hence
> disappear (in the same way that foo.txt will), when checking out to branchA.
> When I checkout back to branchB I want ignore.txt to reappear (in the same
> way that foo.txt will).
>
> I understand why this behaviour is not happening (because my .gitignore
> files are different between the branches), but I am saying that I would like
> to have the option to keep my .gitignore'd files local to the branch they
> are in.
>
> E.g. I currently have a branch with all these binary files that are required
> to run an application on my OS, but when I checkout to another branch I do
> not need or want those binary files anymore (at least not until I checkout
> back into the branch I just came from).
>
> Please tell me if that still doesn't make sense.

Hi Matthew,

Cc'ing the list to benefit from the review of others.

Not wanting the files in the repository seems to be in conflict with
the desire to have them under its control (i.e., disappear/reappear
behavior.)  I understand not wanting to commit dependencies, but why
do you need them to disappear?  Why not just put them somewhere where
they can be used when needed and left alone when not?

If you do want this behavior, it seems like you should just commit the
files on the respective branch.  Maybe someone else will have a better
idea, though.

Chris

RE: Feature Request - Hide ignored files before checkout

[Matthew Ciancio]

I'm glad we are on the same page now and thanks for bringing in others.

" Not wanting the files in the repository seems to be in conflict with the
desire to have them under its control (i.e., disappear/reappear behavior.) "
Sorry, when I said I didn't want them in the repository, I meant I wanted
them there (in the repository folder), but not within Git commit logs and
not being tracked, etc.
Committing binary/OS specific files just doesn't sit right with me and I am
sure many others.

"... but why do you need them to disappear?  Why not just put them somewhere
where they can be used when needed and left alone when not?"
I could do that (keep them in a folder that will be left alone), but in my
specific case it would mean the executable files (ignored files placed in
this new folder) will not execute because of the change in location and
hence would require a lot of code change to suit.

>From my perspective I don't see the harm in having this as a new feature
(via flags in the .gitignore file, if you don't want to make it default
behaviour). Is there some reason I don't know about, maybe to do with the
Git source code?

-----Original Message-----
From: chris.rorvick@gmail.com [mailto:chris.rorvick@gmail.com] On Behalf Of
Chris Rorvick
Sent: Sunday, 9 December 2012 4:54 PM
To: Matthew Ciancio
Cc: git@vger.kernel.org
Subject: Re: Feature Request - Hide ignored files before checkout

On Sat, Dec 8, 2012 at 6:06 PM, Matthew Ciancio
<matthew.ciancio16@gmail.com> wrote:
> Hi Chris,
>
> Yes, I don't think I have explained myself well enough.
>
> When I say "disappear" I do not mean "get deleted", I mean: go out of 
> view just like foo.txt does, as it is committed to branchB and not 
> merged into branchA.
>
> So I am saying that I think .gitignored files should behave partly 
> like committed and un-merged files, in the sense that they disappear 
> when checking out to a different branch.
> I don't want to commit these files (which would give me the behaviour 
> I want), because they are binary/OS specific and really do not belong 
> in the repository, BUT I need them to run/build certain committed files.
>
> To be concrete: I want ignore.txt to be ignored in branchB and hence 
> disappear (in the same way that foo.txt will), when checking out to
branchA.
> When I checkout back to branchB I want ignore.txt to reappear (in the 
> same way that foo.txt will).
>
> I understand why this behaviour is not happening (because my 
> .gitignore files are different between the branches), but I am saying 
> that I would like to have the option to keep my .gitignore'd files 
> local to the branch they are in.
>
> E.g. I currently have a branch with all these binary files that are 
> required to run an application on my OS, but when I checkout to 
> another branch I do not need or want those binary files anymore (at 
> least not until I checkout back into the branch I just came from).
>
> Please tell me if that still doesn't make sense.

Hi Matthew,

Cc'ing the list to benefit from the review of others.

Not wanting the files in the repository seems to be in conflict with the
desire to have them under its control (i.e., disappear/reappear
behavior.)  I understand not wanting to commit dependencies, but why do you
need them to disappear?  Why not just put them somewhere where they can be
used when needed and left alone when not?

If you do want this behavior, it seems like you should just commit the files
on the respective branch.  Maybe someone else will have a better idea,
though.

Chris

Re: Feature Request - Hide ignored files before checkout

[Junio C Hamano]

Chris Rorvick <chris@rorvick.com> writes:

> It's not in branchA, it's just no longer ignored because your changes
> to .gitignore were effectively reverted by jumping back to the commit
> that branchA points to.
> ...
> "hide/reappear" is the equivalent to saying "deleted/created" in the
> case of a tracked file in your working tree.  But how would Git cause
> an untracked file to reappear?  By definition, it doesn't know
> anything about the file.

Nicely explained. To make something simply disappear, you could
remove it, but that is obviously not enough to make it reappear.  It
has to be stashed away somewhere before it gets removed, and in the
context of (any) SCM, that is done by committing.

You may have Mac and Windows branches, each of which needs to link
with vendor supplied object file blackbox.o with the rest of the
source.  It is understandable if a project does not want to mix such
platform specific black box binaries in the history of the source.

But that does not necessarily mean the project can totally ignore
what specific black box binary was meant to be used with the rest of
the source.

After you released the v1.0 of your product for both Macintosh and
Windows, the vendor may supply updated versions of the blackbox.o
binary for these platforms, and you would start working toward v1.1
of your product using these updated copies of objects.  Then you
find problems in the released v1.0 software.  Without keeping track
of which version of the object was used to build the released v1.0,
you cannot diagnose, build and test a maintenance update v1.0.1.

The project may add new Macintosh (or Windows) developers.  You can
tell new Macintosh developers to clone and checkout mac branch, and
in the same e-mail, give them the untracked blackbox.o file for that
platform, but you have to rely on human not making mistakes (you may
mistakenly send Windows version of blackbox.o to him, you may send
stale Macintosh version, the developer may misplace the new one and
keep using the stale one, etc. etc.).

Some people commit blackbox.o on each platform-specific branch, or
all branches share blackbox-win.o and blackbox-mac.o, only one of
which is used at any given branch, for this exact reason.

The project, for licensing reasons, may not have rights to
distribute such a blackbox object file along with its sources, but
the vendor of the blackbox object may allow individual developer to
download and link it from vendor's site.  In such a case, the
project would not want to (and is not allowed to) commit such object
file.  One approach I have seen used in such a case is to arrange
the build procedure so that these individual developers can drop
such an external object next to the project directory, and refer to
it as ../blackbox.o when linking.

So "these files are moved away from the working tree upon checking
another branch out, and moved back into the working tree upon
checking out this branch" is pretty much outside the scope of any
SCM.  It is not very interesting, as it is not necessary to solve
any real world problem.

Of course, the users can do whatever moving/copying/renaming of
untracked files in their post-checkout hook to be run when a new
branch is checked out.

RE: Feature Request - Hide ignored files before checkout

[Matthew Ciancio]

I appreciate your involvement, Mr Hamano.

You have made me realise that my intentions were flawed from the beginning,
because I had been misusing the branch feature.

Thank you for your time.

-----Original Message-----
From: Junio C Hamano [mailto:gitster@pobox.com] 
Sent: Sunday, 9 December 2012 8:04 PM
To: Chris Rorvick
Cc: Matthew Ciancio; git@vger.kernel.org
Subject: Re: Feature Request - Hide ignored files before checkout

Chris Rorvick <chris@rorvick.com> writes:

> It's not in branchA, it's just no longer ignored because your changes 
> to .gitignore were effectively reverted by jumping back to the commit 
> that branchA points to.
> ...
> "hide/reappear" is the equivalent to saying "deleted/created" in the 
> case of a tracked file in your working tree.  But how would Git cause 
> an untracked file to reappear?  By definition, it doesn't know 
> anything about the file.

Nicely explained. To make something simply disappear, you could remove it,
but that is obviously not enough to make it reappear.  It has to be stashed
away somewhere before it gets removed, and in the context of (any) SCM, that
is done by committing.

You may have Mac and Windows branches, each of which needs to link with
vendor supplied object file blackbox.o with the rest of the source.  It is
understandable if a project does not want to mix such platform specific
black box binaries in the history of the source.

But that does not necessarily mean the project can totally ignore what
specific black box binary was meant to be used with the rest of the source.

After you released the v1.0 of your product for both Macintosh and Windows,
the vendor may supply updated versions of the blackbox.o binary for these
platforms, and you would start working toward v1.1 of your product using
these updated copies of objects.  Then you find problems in the released
v1.0 software.  Without keeping track of which version of the object was
used to build the released v1.0, you cannot diagnose, build and test a
maintenance update v1.0.1.

The project may add new Macintosh (or Windows) developers.  You can tell new
Macintosh developers to clone and checkout mac branch, and in the same
e-mail, give them the untracked blackbox.o file for that platform, but you
have to rely on human not making mistakes (you may mistakenly send Windows
version of blackbox.o to him, you may send stale Macintosh version, the
developer may misplace the new one and keep using the stale one, etc. etc.).

Some people commit blackbox.o on each platform-specific branch, or all
branches share blackbox-win.o and blackbox-mac.o, only one of which is used
at any given branch, for this exact reason.

The project, for licensing reasons, may not have rights to distribute such a
blackbox object file along with its sources, but the vendor of the blackbox
object may allow individual developer to download and link it from vendor's
site.  In such a case, the project would not want to (and is not allowed to)
commit such object file.  One approach I have seen used in such a case is to
arrange the build procedure so that these individual developers can drop
such an external object next to the project directory, and refer to it as
../blackbox.o when linking.

So "these files are moved away from the working tree upon checking another
branch out, and moved back into the working tree upon checking out this
branch" is pretty much outside the scope of any SCM.  It is not very
interesting, as it is not necessary to solve any real world problem.

Of course, the users can do whatever moving/copying/renaming of untracked
files in their post-checkout hook to be run when a new branch is checked
out.

Re: Feature Request - Hide ignored files before checkout

[Andrew Ardill]

Hi Matt,

On 8 December 2012 11:50, Matthew Ciancio <matthew.ciancio16@gmail.com> wrote:
> Problem: ignore.txt does not "disappear" like foo.txt does and is now just
> sitting in branchA (and now any other branch I checkout into).
>
> When I first started using Git, I genuinely thought this was a bug, because
> it seems so logical to me that ignore files should hide/reappear just like
> tracked files do, when switching branches.

Let me address this by asking a few questions; *why* do files
hide/reappear, what is the mechanism behind that and does it really
make sense to apply it to ignored files.

For each commit, git stores a snapshot of your files. When we switch
branches we are telling git to restore the previously saved snapshot
so we can work with those files. This means resetting the working
directory so that it looks like what we had committed; git will delete
files that were part of the current checked out snapshot but not the
new one, and create files that need to be created. As a convenience to
users, files that are not tracked are left 'as-is' when switching
branches.

So we see that in order to hide/reappear a file it has to be tracked
in a snapshot, and so has to be committed *somewhere*. An ignored file
is by definition not included in commits, and furthermore you hope to
keep these files out of your commit history.

> I have been told ways of circumventing this (using commits and un-commits OR
> using stash), but my reason for avoiding commits is: say you have binary/OS
> specific files which really do not belong in the commit logs (even locally)
> and hence should be ignored. What if you want those files in only one branch
> and not all?
> Stashing doesn't seem appropriate either, because it would get messy.

I am not sure how viable a suggestion this is, but perhaps you can
have two separate repositories, one tracking your standard branches,
and another tracking the ignored files. These repositories could be
kept in sync through submodules or some similar mechanism. This could
also allow you to, for example, publish the histories of these
independently, for example releasing the non-ignored repository
publicly.
I haven't heard of anyone doing this, but if you need to keep the
history clean it might be a way of achieving it.
I also don't know what the implications of checking out two
repositories into the same tree might be, or even if git would allow
it in general (maybe if you ignored everything belonging to the other
repository?) In any case, this solution could quickly become messy,
but if carefully controlled might solve your problem. Then again,
maybe you can achieve what you want using more 'traditional' git
workflows.

Regards,

Andrew Ardill

RE: Feature Request - Hide ignored files before checkout

[Matthew Ciancio]

Thanks for explaining that Andrew. I guess that was my intention: to have an "ignored file snapshot", but I can see now that it goes against Git's definitions and is not really needed.

I have overcome the problem by re-organising my repository and "... using more 'traditional' git workflows.".

-----Original Message-----
From: Andrew Ardill [mailto:andrew.ardill@gmail.com] 
Sent: Monday, 10 December 2012 12:46 PM
To: Matthew Ciancio
Cc: git@vger.kernel.org
Subject: Re: Feature Request - Hide ignored files before checkout

Hi Matt,

On 8 December 2012 11:50, Matthew Ciancio <matthew.ciancio16@gmail.com> wrote:
> Problem: ignore.txt does not "disappear" like foo.txt does and is now 
> just sitting in branchA (and now any other branch I checkout into).
>
> When I first started using Git, I genuinely thought this was a bug, 
> because it seems so logical to me that ignore files should 
> hide/reappear just like tracked files do, when switching branches.

Let me address this by asking a few questions; *why* do files hide/reappear, what is the mechanism behind that and does it really make sense to apply it to ignored files.

For each commit, git stores a snapshot of your files. When we switch branches we are telling git to restore the previously saved snapshot so we can work with those files. This means resetting the working directory so that it looks like what we had committed; git will delete files that were part of the current checked out snapshot but not the new one, and create files that need to be created. As a convenience to users, files that are not tracked are left 'as-is' when switching branches.

So we see that in order to hide/reappear a file it has to be tracked in a snapshot, and so has to be committed *somewhere*. An ignored file is by definition not included in commits, and furthermore you hope to keep these files out of your commit history.

> I have been told ways of circumventing this (using commits and 
> un-commits OR using stash), but my reason for avoiding commits is: say 
> you have binary/OS specific files which really do not belong in the 
> commit logs (even locally) and hence should be ignored. What if you 
> want those files in only one branch and not all?
> Stashing doesn't seem appropriate either, because it would get messy.

I am not sure how viable a suggestion this is, but perhaps you can have two separate repositories, one tracking your standard branches, and another tracking the ignored files. These repositories could be kept in sync through submodules or some similar mechanism. This could also allow you to, for example, publish the histories of these independently, for example releasing the non-ignored repository publicly.
I haven't heard of anyone doing this, but if you need to keep the history clean it might be a way of achieving it.
I also don't know what the implications of checking out two repositories into the same tree might be, or even if git would allow it in general (maybe if you ignored everything belonging to the other
repository?) In any case, this solution could quickly become messy, but if carefully controlled might solve your problem. Then again, maybe you can achieve what you want using more 'traditional' git workflows.

Regards,

Andrew Ardill

Re: Feature Request - Hide ignored files before checkout

[Erik Faye-Lund]

On Mon, Dec 10, 2012 at 3:01 AM, Matthew Ciancio
<matthew.ciancio16@gmail.com> wrote:
> Thanks for explaining that Andrew. I guess that was my intention: to have an "ignored file snapshot", but I can see now that it goes against Git's definitions and is not really needed.
>

I have played around with the idea of backing up files deleted by
git-clean in the object database, maintained by a reflog (similar to
git-stash). I did this to protect my code against my fat fingers, but
perhaps this could also have been useful in your case?

https://github.com/kusma/git/tree/work/clean-backup