Tracking the untracked

Tracking the untracked

[Giuseppe Bilotta]

<shameless plug alert />

On Wednesday 06 May 2009 03:19, Geoff Russell wrote:
> Basically, I'm trying to find a way of having a huge bunch of stuff in
> my repository and
> tracked, but which doesn't get pushed to the central program repository .. which
> has always just been source for us .. I figured I could stick it on a
> branch which doesn't get pushed.
> But that doesn't work for the reason mentioned.
>
> One solution would be to have a class of files that is "unversioned
> but tracked".
>
> Alternatively we could just bite the bullet and track everything.

I do realize I'm a month late in replying to this email, so I guess in
the end you bit the bullet, but I was perusing the mailing list
archive and this post catched my attention because it's a possible use
case for Zit, the single-file Git-based tracker I've been developing
on and off ( http://git.or.cz/gitwiki/InterfacesFrontendsAndTools#Zit
). I have never thought about using Zit this way, but I see no
problem. Create a .zit directory (by default zit tracks each file
using a git dir named .filename.git, but if .zit/ exists it'll use
.zit/filename.git), add that and all tracked but untracked files to
the gitignore file, and you'll be able to track them with no noise in
your standard git repository.

The only possible underside is that with zit each file is tracked
independently, so if you want to track them all together you'll have
to think of something else. Also, zit isn't very rename-friendly
(yet), so it might not be that good a choice for you if you do a lot
of renaming.


-- 
Giuseppe "Oblomov" Bilotta

Tracking the untracked

[Geoff Russell]

Bug or feature? I don't know.

On the master branch I have some untracked files e.g., object modules,
executables.

I create a branch B1 and add+commit the untracked files.

When I switch back to the master (git checkout master), the untracked
files are no longer where I left them.

Basically, I'm trying to find a way of having a huge bunch of stuff in
my repository and
tracked, but which doesn't get pushed to the central program repository .. which
has always just been source for us .. I figured I could stick it on a
branch which doesn't get pushed.
But that doesn't work for the reason mentioned.

One solution would be to have a class of files that is "unversioned
but tracked".

Alternatively we could just bite the bullet and track everything.

Any ideas?

Cheers,
Geoff Russell

Re: Tracking the untracked

[Johannes Schindelin]

Hi,

On Wed, 6 May 2009, Geoff Russell wrote:

> Bug or feature? I don't know.
> 
> On the master branch I have some untracked files e.g., object modules,
> executables.
> 
> I create a branch B1 and add+commit the untracked files.
> 
> When I switch back to the master (git checkout master), the untracked
> files are no longer where I left them.

This is exactly what Git is supposed to do.  You were on branch B1, which 
has the files tracked, and you do not have the files tracked in master, so 
when switching to master, they should be removed.

Probably you have to rethink what you are trying to do.

Ciao,
Dscho

Re: Tracking the untracked

[Thomas Rast]

[-- Attachment #1: Type: text/plain, Size: 1727 bytes --]

Geoff Russell wrote:
> Bug or feature? I don't know.

Feature.

> On the master branch I have some untracked files e.g., object modules,
> executables.
> 
> I create a branch B1 and add+commit the untracked files.
> 
> When I switch back to the master (git checkout master), the untracked
> files are no longer where I left them.
[...]
> One solution would be to have a class of files that is "unversioned
> but tracked".

Note that "versioned" and "tracked" mean the same thing in Git --
both denote the class of files it cares about.

Your build products became tracked (w.r.t. the then-state of the
repository, on branch B1) the second you added them to the index with
'git add'.  Git then cares about them, and among many other things
will look at them whenever you change branches.  Since they're not
present in the target branch 'master', they are removed from the work
tree.

> Basically, I'm trying to find a way of having a huge bunch of stuff in
> my repository and
> tracked, but which doesn't get pushed to the central program repository .. which
> has always just been source for us .. I figured I could stick it on a
> branch which doesn't get pushed.
> But that doesn't work for the reason mentioned.

Most people just put their build products in .gitignore so that they
stop showing up under "untracked files" in 'git status'.  (They'll
still be untracked!)

Of course this means the object files for source that actually changed
between the branches have to be rebuilt.  However, Git takes great
care to not touch any source files that are the same, so that the
builds are usually quite fast even after a branch switch.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Tracking the untracked

[Geoff Russell]

On 5/6/09, Thomas Rast <trast@student.ethz.ch> wrote:
> Geoff Russell wrote:
>  > Bug or feature? I don't know.
>
>
> Feature.
>
>
>  > On the master branch I have some untracked files e.g., object modules,
>  > executables.
>  >
>  > I create a branch B1 and add+commit the untracked files.
>  >
>  > When I switch back to the master (git checkout master), the untracked
>  > files are no longer where I left them.
>
> [...]
>
> > One solution would be to have a class of files that is "unversioned
>  > but tracked".
>
>
> Note that "versioned" and "tracked" mean the same thing in Git --
>  both denote the class of files it cares about.
>
>  Your build products became tracked (w.r.t. the then-state of the
>  repository, on branch B1) the second you added them to the index with
>  'git add'.  Git then cares about them, and among many other things
>  will look at them whenever you change branches.  Since they're not
>  present in the target branch 'master', they are removed from the work
>  tree.
>
>
>  > Basically, I'm trying to find a way of having a huge bunch of stuff in
>  > my repository and
>  > tracked, but which doesn't get pushed to the central program repository .. which
>  > has always just been source for us .. I figured I could stick it on a
>  > branch which doesn't get pushed.
>  > But that doesn't work for the reason mentioned.
>
>
> Most people just put their build products in .gitignore so that they
>  stop showing up under "untracked files" in 'git status'.  (They'll
>  still be untracked!)
>
>  Of course this means the object files for source that actually changed
>  between the branches have to be rebuilt.  However, Git takes great
>  care to not touch any source files that are the same, so that the
>  builds are usually quite fast even after a branch switch.
>

Ok, its clearly a policy choice.  But suppose I have an untracked
file and I do "git some-command" then I don't expect git to touch what
it doesn't know about. I.e., "git add x" shouldn't delete the untracked
file y. That seems sensible. But now "git checkout branch" behaves
quite differently in just deleting stuff that it doesn't own (i.e., is
untracked).

Anyway, I'll rethink.

Cheers,
Geoff.

>
>  --
>  Thomas Rast
>  trast@{inf,student}.ethz.ch
>
>


-- 
6 Fifth Ave,
St Morris, S.A. 5068
Australia
Ph: 041 8805 184 / 08 8332 5069

Re: Tracking the untracked

[Jakub Narebski]

Geoff Russell <geoffrey.russell@gmail.com> writes:
> On 5/6/09, Thomas Rast <trast@student.ethz.ch> wrote:
> > Geoff Russell wrote:

> > > Bug or feature? I don't know.
> >
> >
> > Feature.

[...]
> Ok, its clearly a policy choice.  But suppose I have an untracked
> file and I do "git some-command" then I don't expect git to touch what
> it doesn't know about. I.e., "git add x" shouldn't delete the untracked
> file y. That seems sensible. But now "git checkout branch" behaves
> quite differently in just deleting stuff that it doesn't own (i.e., is
> untracked).
> 
> Anyway, I'll rethink.

First, did you consider just .gitignore'ing untracked files,
and if they are compilation products use cc-cache instead?

Second, the way git treats untracked files is simple: do not
lose information.  If a file is tracked, or to be more exact
specific contents of a file is in repository, then deleting
it would not remove information.

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: Tracking the untracked

[Thomas Rast]

[-- Attachment #1: Type: text/plain, Size: 2531 bytes --]

Geoff Russell wrote:
>

Please keep the Cc list, it's customary around here and helps us find
the mails addressed to us.

> On 5/6/09, Thomas Rast <trast@student.ethz.ch> wrote:
> >  Your build products became tracked (w.r.t. the then-state of the
> >  repository, on branch B1) the second you added them to the index with
> >  'git add'.  Git then cares about them, and among many other things
> >  will look at them whenever you change branches.  Since they're not
> >  present in the target branch 'master', they are removed from the work
> >  tree.
[...]
> Ok, its clearly a policy choice.  But suppose I have an untracked
> file and I do "git some-command" then I don't expect git to touch what
> it doesn't know about. I.e., "git add x" shouldn't delete the untracked
> file y. That seems sensible. But now "git checkout branch" behaves
> quite differently in just deleting stuff that it doesn't own (i.e., is
> untracked).
> 
> Anyway, I'll rethink.

They weren't untracked!

This is roughly what Dscho said, but I'll try to explain it in more
detail:

- In the first step, when switching from master to B1, your build
  products were indeed untracked (not part of either master or B1 or
  the index).  So Git leaves them alone.

- By adding (git add $file) your files to the index, you made them
  tracked.[1]

- By making a commit, you let the files "officially" belong to the
  current branch (B1).  So with respect to the _current_ branch B1,
  they _were_ tracked.

- When you switch branches in the last step, Git sees that B1 has
  these files, but master doesn't, so it removes them to bring the
  worktree into the state that master has.

- Consider what happens if you were to recompile at this point, so
  that you again have the _untracked_ (w.r.t. the 'master' branch)
  build products, and then check out B1 again.  Git faces the choice
  of either overwriting your worktree files (and losing them) or not
  overwriting (and not doing the checkout right), so it will abort.
  This is a corollary of what Jakub pointed out, Git will not lose
  information unless told to.


[1] If you were to switch branches at this point, without making a
commit, Git tries to "carry over" the addition of files to the target
branch, unless it conflicts with files in that target branch, so the
index is in a slightly different position than the HEAD.  Still,
anything that is listed in the index is considered tracked.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Tracking the untracked

[Geoff Russell]

On 5/7/09, Thomas Rast <trast@student.ethz.ch> wrote:
> Geoff Russell wrote:
> [...]
>  - Consider what happens if you were to recompile at this point, so
>   that you again have the _untracked_ (w.r.t. the 'master' branch)
>   build products, and then check out B1 again.  Git faces the choice
>   of either overwriting your worktree files (and losing them) or not
>   overwriting (and not doing the checkout right), so it will abort.
>   This is a corollary of what Jakub pointed out, Git will not lose
>   information unless told to.

Understood. Upon reflection, I can't think of a better alternative.

Cheers and thanks for the help,

Geoff.