git stash data loss

git stash data loss

[Aleksandr Pryimak]

Hello.

It is believed that
git stash
git stash pop
is safe to-do operation. But sometimes it is not. For example here is an example http://stackoverflow.com/questions/11680453/git-stash-where-did-the-files-in-my-directory-go/11680645

I also recreated it

aleksandr@beast:/tmp/test$ git init
Initialized empty Git repository in /tmp/test/.git/
aleksandr@beast:/tmp/test$ touch x
aleksandr@beast:/tmp/test$ git add x
aleksandr@beast:/tmp/test$ git commit -m "Initial"
[master (root-commit) d3569a0] Initial
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 x
aleksandr@beast:/tmp/test$ rm x
aleksandr@beast:/tmp/test$ mkdir x/
aleksandr@beast:/tmp/test$ ls
x
aleksandr@beast:/tmp/test$ git stash
Saved working directory and index state WIP on master: d3569a0 Initial
HEAD is now at d3569a0 Initial
aleksandr@beast:/tmp/test$ ls
x
aleksandr@beast:/tmp/test$ git stash pop
Removing x
# On branch master
# Changes not staged for commit:
#   (use "git add/rm <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	deleted:    x
#
no changes added to commit (use "git add" and/or "git commit -a")
Dropped refs/stash@{0} (c500443ae16cf0d846b195cb97eb388dec5f440e)
aleksandr@beast:/tmp/test$ ls
aleksandr@beast:/tmp/test$ git --version
git version 1.7.5.4

--
Aleksandr Pryimak

Re: git stash data loss

[Jeff King]

On Fri, Jul 27, 2012 at 07:06:08AM +0400, Aleksandr Pryimak wrote:

> I also recreated it
> 
> aleksandr@beast:/tmp/test$ git init
> Initialized empty Git repository in /tmp/test/.git/
> aleksandr@beast:/tmp/test$ touch x
> aleksandr@beast:/tmp/test$ git add x
> aleksandr@beast:/tmp/test$ git commit -m "Initial"
> [master (root-commit) d3569a0] Initial
>  0 files changed, 0 insertions(+), 0 deletions(-)
>  create mode 100644 x

OK, so we have "x" as a tracked file.

> aleksandr@beast:/tmp/test$ rm x
> aleksandr@beast:/tmp/test$ mkdir x/
> aleksandr@beast:/tmp/test$ ls
> x

And then we remove it in favor of a directory. Note that git does not
track directories directly, only files inside them. So from git's
perspective, the working tree has no content in it at all.

> aleksandr@beast:/tmp/test$ git stash
> Saved working directory and index state WIP on master: d3569a0 Initial
> HEAD is now at d3569a0 Initial

And now we stash it. That will stash the working tree removal of x. It
will not stash anything about the new directory x, because it has no
content inside it.

> aleksandr@beast:/tmp/test$ ls
> x
> aleksandr@beast:/tmp/test$ git stash pop
> Removing x
> # On branch master
> # Changes not staged for commit:
> #   (use "git add/rm <file>..." to update what will be committed)
> #   (use "git checkout -- <file>..." to discard changes in working directory)
> #
> #	deleted:    x
> #
> no changes added to commit (use "git add" and/or "git commit -a")
> Dropped refs/stash@{0} (c500443ae16cf0d846b195cb97eb388dec5f440e)
> aleksandr@beast:/tmp/test$ ls

And your stash pop restores the deletion of "x". It does _not_ reinstate
the directory "x", because as I stated above, that is not part of the
stash.

So what is the data loss? That the "mkdir x" was lost? That has nothing
to do with stash, but rather the fact that git does not track empty
directories. You could see the same thing with:
 
  mkdir x && git commit -a

which will not record your mkdir at all. In other words, this is by
design.

If we put actual files inside "x", which git does track, then they would
be part of the stash, and should be properly retained. But they're not:

  $ rm x && mkdir x && echo foo >x/file

Now we have some precious contents in the form of "x/file". They are
untracked by git, but git should be careful about removing them.

  $ git stash
  Saved working directory and index state WIP on master: 2d32d3a initial
  HEAD is now at 2d32d3a initial
  $ ls -l x
  -rw-r--r-- 1 peff peff 0 Jul 27 09:19 x
  $ git stash show --raw
  :100644 000000 e69de29... 0000000... D  x

Now this _is_ data loss. Stash blows away untracked files inside the
directory, but does not record them in the resulting stash. And that
should be fixed.

With "-u", I'd expect stash to include the untracked file x/foo in the
stash. Without "-u", I'd expect stash to fail, since it would be
deleting untracked files.

-Peff

Re: git stash data loss

[Jeff King]

On Fri, Jul 27, 2012 at 09:29:53AM -0400, Jeff King wrote:

> If we put actual files inside "x", which git does track, then they would
> be part of the stash, and should be properly retained. But they're not:
> 
>   $ rm x && mkdir x && echo foo >x/file
> 
> Now we have some precious contents in the form of "x/file". They are
> untracked by git, but git should be careful about removing them.
> 
>   $ git stash
>   Saved working directory and index state WIP on master: 2d32d3a initial
>   HEAD is now at 2d32d3a initial
>   $ ls -l x
>   -rw-r--r-- 1 peff peff 0 Jul 27 09:19 x
>   $ git stash show --raw
>   :100644 000000 e69de29... 0000000... D  x
> 
> Now this _is_ data loss. Stash blows away untracked files inside the
> directory, but does not record them in the resulting stash. And that
> should be fixed.

Hrm. The problem is that after creating the stash, we then run "git
reset --hard" to drop the changes that we just stashed. But that is not
always accurate. It will not usually touch untracked files, but it might
if they have D/F conflicts with tracked files. So we need to replace
that "git reset --hard" with some safer command that will notice we are
about to overwrite untracked files. But I am not sure what that command
would be.

-Peff

Re: git stash data loss

[Aleksandr Priymak]

Thanks Jeff for the comment. You are right. I oversimplified the use-case ( forget to fill the directory )

Sent from my iPad

On Jul 27, 2012, at 5:29 PM, Jeff King <peff@peff.net> wrote:

> On Fri, Jul 27, 2012 at 07:06:08AM +0400, Aleksandr Pryimak wrote:
> 
>> I also recreated it
>> 
>> aleksandr@beast:/tmp/test$ git init
>> Initialized empty Git repository in /tmp/test/.git/
>> aleksandr@beast:/tmp/test$ touch x
>> aleksandr@beast:/tmp/test$ git add x
>> aleksandr@beast:/tmp/test$ git commit -m "Initial"
>> [master (root-commit) d3569a0] Initial
>> 0 files changed, 0 insertions(+), 0 deletions(-)
>> create mode 100644 x
> 
> OK, so we have "x" as a tracked file.
> 
>> aleksandr@beast:/tmp/test$ rm x
>> aleksandr@beast:/tmp/test$ mkdir x/
>> aleksandr@beast:/tmp/test$ ls
>> x
> 
> And then we remove it in favor of a directory. Note that git does not
> track directories directly, only files inside them. So from git's
> perspective, the working tree has no content in it at all.
> 
>> aleksandr@beast:/tmp/test$ git stash
>> Saved working directory and index state WIP on master: d3569a0 Initial
>> HEAD is now at d3569a0 Initial
> 
> And now we stash it. That will stash the working tree removal of x. It
> will not stash anything about the new directory x, because it has no
> content inside it.
> 
>> aleksandr@beast:/tmp/test$ ls
>> x
>> aleksandr@beast:/tmp/test$ git stash pop
>> Removing x
>> # On branch master
>> # Changes not staged for commit:
>> #   (use "git add/rm <file>..." to update what will be committed)
>> #   (use "git checkout -- <file>..." to discard changes in working directory)
>> #
>> #    deleted:    x
>> #
>> no changes added to commit (use "git add" and/or "git commit -a")
>> Dropped refs/stash@{0} (c500443ae16cf0d846b195cb97eb388dec5f440e)
>> aleksandr@beast:/tmp/test$ ls
> 
> And your stash pop restores the deletion of "x". It does _not_ reinstate
> the directory "x", because as I stated above, that is not part of the
> stash.
> 
> So what is the data loss? That the "mkdir x" was lost? That has nothing
> to do with stash, but rather the fact that git does not track empty
> directories. You could see the same thing with:
> 
>  mkdir x && git commit -a
> 
> which will not record your mkdir at all. In other words, this is by
> design.
> 
> If we put actual files inside "x", which git does track, then they would
> be part of the stash, and should be properly retained. But they're not:
> 
>  $ rm x && mkdir x && echo foo >x/file
> 
> Now we have some precious contents in the form of "x/file". They are
> untracked by git, but git should be careful about removing them.
> 
>  $ git stash
>  Saved working directory and index state WIP on master: 2d32d3a initial
>  HEAD is now at 2d32d3a initial
>  $ ls -l x
>  -rw-r--r-- 1 peff peff 0 Jul 27 09:19 x
>  $ git stash show --raw
>  :100644 000000 e69de29... 0000000... D  x
> 
> Now this _is_ data loss. Stash blows away untracked files inside the
> directory, but does not record them in the resulting stash. And that
> should be fixed.
> 
> With "-u", I'd expect stash to include the untracked file x/foo in the
> stash. Without "-u", I'd expect stash to fail, since it would be
> deleting untracked files.
> 
> -Peff

Re: git stash data loss

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Fri, Jul 27, 2012 at 09:29:53AM -0400, Jeff King wrote:
>
>> If we put actual files inside "x", which git does track, then they would
>> be part of the stash, and should be properly retained. But they're not:
>> 
>>   $ rm x && mkdir x && echo foo >x/file
>> 
>> Now we have some precious contents in the form of "x/file". They are
>> untracked by git, but git should be careful about removing them.
>> 
>>   $ git stash
>>   Saved working directory and index state WIP on master: 2d32d3a initial
>>   HEAD is now at 2d32d3a initial
>>   $ ls -l x
>>   -rw-r--r-- 1 peff peff 0 Jul 27 09:19 x
>>   $ git stash show --raw
>>   :100644 000000 e69de29... 0000000... D  x
>> 
>> Now this _is_ data loss. Stash blows away untracked files inside the
>> directory, but does not record them in the resulting stash. And that
>> should be fixed.
>
> Hrm. The problem is that after creating the stash, we then run "git
> reset --hard" to drop the changes that we just stashed. But that is not
> always accurate. It will not usually touch untracked files, but it might
> if they have D/F conflicts with tracked files. So we need to replace
> that "git reset --hard" with some safer command that will notice we are
> about to overwrite untracked files. But I am not sure what that command
> would be.

Is this something we still want to keep track of?

As readers can probably guess, I am trying to come up with a list of
loose ends for the next cycle.  This may be one of the low-hanging
ones.

Re: git stash data loss

[Jeff King]

On Thu, Aug 16, 2012 at 08:30:48PM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > On Fri, Jul 27, 2012 at 09:29:53AM -0400, Jeff King wrote:
> >
> >> If we put actual files inside "x", which git does track, then they would
> >> be part of the stash, and should be properly retained. But they're not:
> >> 
> >>   $ rm x && mkdir x && echo foo >x/file
> >> 
> >> Now we have some precious contents in the form of "x/file". They are
> >> untracked by git, but git should be careful about removing them.
> >> 
> >>   $ git stash
> >>   Saved working directory and index state WIP on master: 2d32d3a initial
> >>   HEAD is now at 2d32d3a initial
> >>   $ ls -l x
> >>   -rw-r--r-- 1 peff peff 0 Jul 27 09:19 x
> >>   $ git stash show --raw
> >>   :100644 000000 e69de29... 0000000... D  x
> >> 
> >> Now this _is_ data loss. Stash blows away untracked files inside the
> >> directory, but does not record them in the resulting stash. And that
> >> should be fixed.
> >
> > Hrm. The problem is that after creating the stash, we then run "git
> > reset --hard" to drop the changes that we just stashed. But that is not
> > always accurate. It will not usually touch untracked files, but it might
> > if they have D/F conflicts with tracked files. So we need to replace
> > that "git reset --hard" with some safer command that will notice we are
> > about to overwrite untracked files. But I am not sure what that command
> > would be.
> 
> Is this something we still want to keep track of?

Yeah, I think it is worth fixing. It's a somewhat rare case, but data
loss is bad. I was hoping you would respond with "...and here is the
magical incantation of git commands to make the working directory look
like we want". I couldn't come up with one. We may need a new option to
reset or read-tree.

-Peff

Re: git stash data loss

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

>> > Hrm. The problem is that after creating the stash, we then run "git
>> > reset --hard" to drop the changes that we just stashed. But that is not
>> > always accurate. It will not usually touch untracked files, but it might
>> > if they have D/F conflicts with tracked files. So we need to replace
>> > that "git reset --hard" with some safer command that will notice we are
>> > about to overwrite untracked files. But I am not sure what that command
>> > would be.
>> 
>> Is this something we still want to keep track of?
>
> Yeah, I think it is worth fixing. It's a somewhat rare case, but data
> loss is bad. I was hoping you would respond with "...and here is the
> magical incantation of git commands to make the working directory look
> like we want". I couldn't come up with one. We may need a new option to
> reset or read-tree.

ls-files has an ancient option to show the files "killed"; perhaps
that is the closest thing to what you are looking for.