Experience with Recovering From User Error (And suggestions for improvements)

Experience with Recovering From User Error (And suggestions for improvements)

[Armin Ronacher]

Hi,

Long story short: I failed big time yesterday with accidentally 
executing git reset hard in the wrong terminal window but managed to 
recover my changes from the staging area by manually examining blobs 
touched recently.

After that however I figured I might want to add a precaution for myself 
that would have helped there.  git fsck is quite nice, but unfortunately 
it does not help if you do not have a commit.  So I figured it might be 
nice to create a dangling backup commit before a reset which would have 
helped me.  Unfortunately there is currently no good way to hook into 
git reset.

Things I noticed in the process:

*   for recovering blobs, going through the objects itself was more
     useful because they were all recent changes and as such I could
     order by timestamp.  git fsck will not provide any timestamps
     (which generally makes sense, but made it quite useless for me)
*   Recovering from blobs is painful, it would be nice if git reset
     --hard made a dangling dummy commit before :)
*   There is no pre-commit hook which could be used to implement the
     previous suggestion.

Would it make sense to introduce a `pre-commit` hook for this sort of 
thing or even create a dummy commit by default?  I did a quick googling 
around and it looks like I was not the first person who made this 
mistake.  Github's windows client even creates dangling backup commits 
in what appears to be fixed time intervals.

I understand that ultimately this was a user error on my part, but it 
seems like a small change that could save a lot of frustration.


Regards,
Armin

Re: Experience with Recovering From User Error (And suggestions for improvements)

[Ævar Arnfjörð Bjarmason]

On Mon, Feb 16, 2015 at 11:41 AM, Armin Ronacher
<armin.ronacher@active-4.com> wrote:
> Long story short: I failed big time yesterday with accidentally executing
> git reset hard in the wrong terminal window but managed to recover my
> changes from the staging area by manually examining blobs touched recently.
>
> After that however I figured I might want to add a precaution for myself
> that would have helped there.  git fsck is quite nice, but unfortunately it
> does not help if you do not have a commit.  So I figured it might be nice to
> create a dangling backup commit before a reset which would have helped me.
> Unfortunately there is currently no good way to hook into git reset.
>
> Things I noticed in the process:
>
> *   for recovering blobs, going through the objects itself was more
>     useful because they were all recent changes and as such I could
>     order by timestamp.  git fsck will not provide any timestamps
>     (which generally makes sense, but made it quite useless for me)
> *   Recovering from blobs is painful, it would be nice if git reset
>     --hard made a dangling dummy commit before :)
> *   There is no pre-commit hook which could be used to implement the
>     previous suggestion.
>
> Would it make sense to introduce a `pre-commit` hook for this sort of thing
> or even create a dummy commit by default?  I did a quick googling around and
> it looks like I was not the first person who made this mistake.  Github's
> windows client even creates dangling backup commits in what appears to be
> fixed time intervals.
>
> I understand that ultimately this was a user error on my part, but it seems
> like a small change that could save a lot of frustration.

Something like "can we have a hook for every change in the working
tree" has come up in the past, but has been defeated by performance
concerns. "git reset --hard" is a low-level-ish operation, and it's
really useful to be able to quickly reset the working tree to some
state no matter what, and without creating extra commits or whatever.

We should definitely make recovery like this harder, but is there a
reason for why you don't use "git reset --keep" instead of --hard?
It'll keep any local changes to your index/staging area, and reset the
files that don't conflict, if there's any conflicts the operation will
be aborted.

If we created such hooks for "git reset --hard" we'd just need to
expose some other thing as that low-level operation (and break scripts
that already rely on it doing the minimal "yes I want to change the
tree no matter what" thing), and then we'd just be back to square one
in a few years when users started using "git reset --really-hard" (or
whatever the flag would be).

Re: Experience with Recovering From User Error (And suggestions for improvements)

[Ævar Arnfjörð Bjarmason]

On Mon, Feb 16, 2015 at 1:09 PM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> On Mon, Feb 16, 2015 at 11:41 AM, Armin Ronacher
> <armin.ronacher@active-4.com> wrote:
>> Long story short: I failed big time yesterday with accidentally executing
>> git reset hard in the wrong terminal window but managed to recover my
>> changes from the staging area by manually examining blobs touched recently.
>>
>> After that however I figured I might want to add a precaution for myself
>> that would have helped there.  git fsck is quite nice, but unfortunately it
>> does not help if you do not have a commit.  So I figured it might be nice to
>> create a dangling backup commit before a reset which would have helped me.
>> Unfortunately there is currently no good way to hook into git reset.
>>
>> Things I noticed in the process:
>>
>> *   for recovering blobs, going through the objects itself was more
>>     useful because they were all recent changes and as such I could
>>     order by timestamp.  git fsck will not provide any timestamps
>>     (which generally makes sense, but made it quite useless for me)
>> *   Recovering from blobs is painful, it would be nice if git reset
>>     --hard made a dangling dummy commit before :)
>> *   There is no pre-commit hook which could be used to implement the
>>     previous suggestion.
>>
>> Would it make sense to introduce a `pre-commit` hook for this sort of thing
>> or even create a dummy commit by default?  I did a quick googling around and
>> it looks like I was not the first person who made this mistake.  Github's
>> windows client even creates dangling backup commits in what appears to be
>> fixed time intervals.
>>
>> I understand that ultimately this was a user error on my part, but it seems
>> like a small change that could save a lot of frustration.
>
> Something like "can we have a hook for every change in the working
> tree" has come up in the past, but has been defeated by performance
> concerns. "git reset --hard" is a low-level-ish operation, and it's
> really useful to be able to quickly reset the working tree to some
> state no matter what, and without creating extra commits or whatever.
>
> We should definitely make recovery like this harder, but is there a
> reason for why you don't use "git reset --keep" instead of --hard?
> It'll keep any local changes to your index/staging area, and reset the
> files that don't conflict, if there's any conflicts the operation will
> be aborted.

"Recovery like this easier", i.e. make it easier to get back
previously staged commits / blobs.

> If we created such hooks for "git reset --hard" we'd just need to
> expose some other thing as that low-level operation (and break scripts
> that already rely on it doing the minimal "yes I want to change the
> tree no matter what" thing), and then we'd just be back to square one
> in a few years when users started using "git reset --really-hard" (or
> whatever the flag would be).

Re: Experience with Recovering From User Error (And suggestions for improvements)

[Duy Nguyen]

On Mon, Feb 16, 2015 at 7:10 PM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>> We should definitely make recovery like this harder, but is there a
>> reason for why you don't use "git reset --keep" instead of --hard?
>> It'll keep any local changes to your index/staging area, and reset the
>> files that don't conflict, if there's any conflicts the operation will
>> be aborted.
>
> "Recovery like this easier", i.e. make it easier to get back
> previously staged commits / blobs.

I started with git-undo (or what's its name) a while back (*). The
idea is for dangerous commands like this we could save some data back,
which would be pruned after some time. Saving stuff in index is quite
easy because they are already in object database. For this reset
--hard, we may need to hash/store some more blobs. I think it's worth
the safety. Not sure if anyone's interested in continuing that work.

(*) found it: http://thread.gmane.org/gmane.comp.version-control.git/231621/focus=231879
-- 
Duy

Re: Experience with Recovering From User Error (And suggestions for improvements)

[Armin Ronacher]

Hi,

On 16/02/15 13:09, Ævar Arnfjörð Bjarmason wrote:
> We should definitely make recovery like this harder, but is there a
> reason for why you don't use "git reset --keep" instead of --hard?
This was only the second time in years of git usage that the reset was 
incorrectly done.  I suppose at this point I might try to retrain my 
muscle memory to type something else :)

> If we created such hooks for "git reset --hard" we'd just need to
> expose some other thing as that low-level operation (and break scripts
> that already rely on it doing the minimal "yes I want to change the
> tree no matter what" thing), and then we'd just be back to square one
> in a few years when users started using "git reset --really-hard" (or
> whatever the flag would be).
I don't think that's necessary, I don't think it would make the 
operation much slower to just make a dangling commit and write out a few 
blobs.  The garbage collect will soon enough take care of that data 
anyways.  But I guess that would need testing on large trees to see how 
bad that goes.

I might look into the git undo thing that was mentioned.


Regards,
Armin

Re: Experience with Recovering From User Error (And suggestions for improvements)

[Michael J Gruber]

Armin Ronacher venit, vidit, dixit 16.02.2015 14:29:
> Hi,
> 
> On 16/02/15 13:09, Ævar Arnfjörð Bjarmason wrote:
>> We should definitely make recovery like this harder, but is there a
>> reason for why you don't use "git reset --keep" instead of --hard?
> This was only the second time in years of git usage that the reset was 
> incorrectly done.  I suppose at this point I might try to retrain my 
> muscle memory to type something else :)
> 
>> If we created such hooks for "git reset --hard" we'd just need to
>> expose some other thing as that low-level operation (and break scripts
>> that already rely on it doing the minimal "yes I want to change the
>> tree no matter what" thing), and then we'd just be back to square one
>> in a few years when users started using "git reset --really-hard" (or
>> whatever the flag would be).
> I don't think that's necessary, I don't think it would make the 
> operation much slower to just make a dangling commit and write out a few 
> blobs.  The garbage collect will soon enough take care of that data 
> anyways.  But I guess that would need testing on large trees to see how 
> bad that goes.
> 
> I might look into the git undo thing that was mentioned.
> 
> 
> Regards,
> Armin
> 

Are you concerned about the index only, not unstaged worktree changes?

In this case, keeping a reflog for the index may help, and it would
somehow fit into the overall concept.

Otherwise, we would basically need a full stash before a hard reset.
That's not the first time where we could need a distinction between
"command run by user" and "command run by script". For the former, we
could allow overriding default options, re-aliasing internal commands,
adding expensive safety hooks. For the latter we can't.

It's just that we don't have such a concept yet (other than checking tty).

Michael

Re: Experience with Recovering From User Error (And suggestions for improvements)

[Michael J Gruber]

Kyle J. McKay venit, vidit, dixit 19.02.2015 02:17:
> On Feb 18, 2015, at 01:46, Michael J Gruber wrote:
>> Armin Ronacher venit, vidit, dixit 16.02.2015 14:29:
>>> Hi,
>>>
>>> On 16/02/15 13:09, Ævar Arnfjörð Bjarmason wrote:
>>>> We should definitely make recovery like this harder, but is there a
>>>> reason for why you don't use "git reset --keep" instead of --hard?
>>> This was only the second time in years of git usage that the reset  
>>> was
>>> incorrectly done.  I suppose at this point I might try to retrain my
>>> muscle memory to type something else :)
>>>
>>>> If we created such hooks for "git reset --hard" we'd just need to
>>>> expose some other thing as that low-level operation (and break  
>>>> scripts
>>>> that already rely on it doing the minimal "yes I want to change the
>>>> tree no matter what" thing), and then we'd just be back to square  
>>>> one
>>>> in a few years when users started using "git reset --really- 
>>>> hard" (or
>>>> whatever the flag would be).
>>> I don't think that's necessary, I don't think it would make the
>>> operation much slower to just make a dangling commit and write out  
>>> a few
>>> blobs.  The garbage collect will soon enough take care of that data
>>> anyways.  But I guess that would need testing on large trees to see  
>>> how
>>> bad that goes.
>>>
>>> I might look into the git undo thing that was mentioned.
>>>
>>> Regards,
>>> Armin
>>>
>>
>> Are you concerned about the index only, not unstaged worktree changes?
>>
>> In this case, keeping a reflog for the index may help, and it would
>> somehow fit into the overall concept.
> 
> There was this concept of a git stash checkpoint to save work in  
> progress without creating a normal commit that I read about some time  
> ago (blog? Git book? -- don't recall) that was basically just this:
> 
>    git stash save
>    git stash apply
> 
> The problem with that is that it touches the working tree and can  
> trigger rebuilds etc.  However, when I ran across the undocumented  
> "git stash create" command I was able to write a simple git-checkpoint  
> script [1] that creates a new stash entry without touching the index  
> or working tree which I find quite handy from time to time.

I think that would make for a nice additional command/mode that we could
support for git-stash.sh. Alle the pieces are there.

> So I think that what Armin originally asked for (create a dangling  
> commit of changes before reset --hard) could be accomplished simply by  
> running:
> 
>    git checkpoint && git stash drop
> 
>> Otherwise, we would basically need a full stash before a hard reset.
>> That's not the first time where we could need a distinction between
>> "command run by user" and "command run by script". For the former, we
>> could allow overriding default options, re-aliasing internal commands,
>> adding expensive safety hooks. For the latter we can't.
>>
>> It's just that we don't have such a concept yet (other than checking  
>> tty).
> 
> But of course plugging that into git reset somehow is indeed the  
> problem since you cannot alias/redefine git commands.
> 
> -Kyle
> 
> [1] https://gist.github.com/mackyle/83b1ba13e263356bdab0

Also, "git stash create" does the tree creation and object creation that
we wanted to avoid at least for scripts.

And "git reset --hard-but-safe" suffers from the user education problems
that have been mentioned already.

Michael

[RFD/PATCH] stash: introduce checkpoint mode

[Michael J Gruber]

"git stash save" performs the steps "create-store-reset". Often,
users try to use "stash save" as a way to to save their current state
(index, worktree) before an operation like "checkout/reset --patch" they
don't feel confident about, and are forced to do "git stash save && git
stash apply".

Provide an extra mode that does "create-store" only without the reset,
so that one can "ceckpoint" the sate and keep working on it.

Suggested-by: "Kyle J. McKay" <mackyle@gmail.com>
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---

Notes:
    I'm not sure about how to best expose this mode:
    
    git stash checkpoint
    git stash save --checkpoint
    
    Maybe it is best to document the former and rename "--checkpoint"
    to "--no-reset"?
    
    Also, a "safe return" to a checkpoint probably requires
    
    git reset --hard && git stash pop
    
    although "git stash pop" will do in many cases. Should we provide a shortcut
    "restore" which does the reset-and-pop?

 git-stash.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/git-stash.sh b/git-stash.sh
index d4cf818..42f140c 100755
--- a/git-stash.sh
+++ b/git-stash.sh
@@ -193,12 +193,16 @@ store_stash () {
 }
 
 save_stash () {
+	checkpoint=
 	keep_index=
 	patch_mode=
 	untracked=
 	while test $# != 0
 	do
 		case "$1" in
+		-c|--checkpoint)
+			checkpoint=t
+			;;
 		-k|--keep-index)
 			keep_index=t
 			;;
@@ -267,6 +271,11 @@ save_stash () {
 	die "$(gettext "Cannot save the current status")"
 	say Saved working directory and index state "$stash_msg"
 
+	if test -n "$checkpoint"
+	then
+		exit 0
+	fi
+
 	if test -z "$patch_mode"
 	then
 		git reset --hard ${GIT_QUIET:+-q}
@@ -576,6 +585,10 @@ save)
 	shift
 	save_stash "$@"
 	;;
+checkpoint)
+	shift
+	save_stash "--checkpoint" "$@"
+	;;
 apply)
 	shift
 	apply_stash "$@"
-- 
2.3.0.191.ge77e8b9

Re: [RFD/PATCH] stash: introduce checkpoint mode

[Kyle J. McKay]

On Feb 19, 2015, at 04:34, Michael J Gruber wrote:

> "git stash save" performs the steps "create-store-reset". Often,
> users try to use "stash save" as a way to to save their current state
> (index, worktree) before an operation like "checkout/reset --patch"  
> they
> don't feel confident about, and are forced to do "git stash save &&  
> git
> stash apply".
>
> Provide an extra mode that does "create-store" only without the reset,
> so that one can "ceckpoint" the sate and keep working on it.

s/sate/state/

> Suggested-by: "Kyle J. McKay" <mackyle@gmail.com>
> Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
> ---
>
> Notes:
>    I'm not sure about how to best expose this mode:
>
>    git stash checkpoint
>    git stash save --checkpoint
>
>    Maybe it is best to document the former and rename "--checkpoint"
>    to "--no-reset"?

Once the user figures out that "save" is really "save-and-reset" I  
think "--no-reset" makes more sense.

It certainly seems more discoverable via an explicit "checkpoint"  
command though, but that's really just an alias so maybe it's better  
left up to the user to make one.

There would need to be some updated docs (git-stash.txt) to go with  
the change...

>    Also, a "safe return" to a checkpoint probably requires
>
>    git reset --hard && git stash pop
>
>    although "git stash pop" will do in many cases. Should we provide  
> a shortcut
>    "restore" which does the reset-and-pop?

What about a shortcut to "reset-and-apply" as well?

I have often been frustrated when "git stash apply" refuses to work  
because I have changes that would be stepped on and there's no --force  
option like git checkout has.  I end up doing a reset just so I can  
run stash apply.

What about if git stash apply/pop grokked a --force option?  That  
would seem to eliminate the need for a "reset-and-pop"/"reset-and- 
apply" shortcut while also being useful to non-checkpoint stashes as  
well.

> git-stash.sh | 13 +++++++++++++
> 1 file changed, 13 insertions(+)
>
> diff --git a/git-stash.sh b/git-stash.sh
> index d4cf818..42f140c 100755
> --- a/git-stash.sh
> +++ b/git-stash.sh
> @@ -193,12 +193,16 @@ store_stash () {
> }
>
> save_stash () {
> +	checkpoint=
> 	keep_index=
> 	patch_mode=
> 	untracked=
> 	while test $# != 0
> 	do
> 		case "$1" in
> +		-c|--checkpoint)
> +			checkpoint=t
> +			;;
> 		-k|--keep-index)
> 			keep_index=t
> 			;;
> @@ -267,6 +271,11 @@ save_stash () {
> 	die "$(gettext "Cannot save the current status")"
> 	say Saved working directory and index state "$stash_msg"
>
> +	if test -n "$checkpoint"
> +	then
> +		exit 0
> +	fi
> +
> 	if test -z "$patch_mode"
> 	then
> 		git reset --hard ${GIT_QUIET:+-q}
> @@ -576,6 +585,10 @@ save)
> 	shift
> 	save_stash "$@"
> 	;;
> +checkpoint)
> +	shift
> +	save_stash "--checkpoint" "$@"
> +	;;
> apply)
> 	shift
> 	apply_stash "$@"
> -- 

Otherwise this looks good.  A very small change to add the  
functionality.

-Kyle

Re: [RFD/PATCH] stash: introduce checkpoint mode

[Junio C Hamano]

"Kyle J. McKay" <mackyle@gmail.com> writes:

> What about a shortcut to "reset-and-apply" as well?
>
> I have often been frustrated when "git stash apply" refuses to work
> because I have changes that would be stepped on and there's no --force
> option like git checkout has.  I end up doing a reset just so I can
> run stash apply.

Doesn't that cut both ways, though?

A single step short-cut, done in any way other than a more explicit
way such as "git reset --hard && git stash apply" (e.g. "git stash
reset-and-apply" or "git stash apply --force") that makes it crystal
clear that the user _is_ discarding, has a risk of encouraging users
to form a dangerous habit of invoking the short-cut without thinking
and leading to "oops, I didn't mean that!".

Re: [RFD/PATCH] stash: introduce checkpoint mode

[Kyle J. McKay]

On Feb 19, 2015, at 09:49, Junio C Hamano wrote:
> "Kyle J. McKay" <mackyle@gmail.com> writes:
>
>> What about a shortcut to "reset-and-apply" as well?
>>
>> I have often been frustrated when "git stash apply" refuses to work
>> because I have changes that would be stepped on and there's no -- 
>> force
>> option like git checkout has.  I end up doing a reset just so I can
>> run stash apply.
>
> Doesn't that cut both ways, though?
>
> A single step short-cut, done in any way other than a more explicit
> way such as "git reset --hard && git stash apply" (e.g. "git stash
> reset-and-apply" or "git stash apply --force") that makes it crystal
> clear that the user _is_ discarding, has a risk of encouraging users
> to form a dangerous habit of invoking the short-cut without thinking
> and leading to "oops, I didn't mean that!".

Does that reasoning not also apply to the plethora of commands that  
take "--force" already?

I didn't check them all, but tag, checkout, push and branch  
immediately come to mind.  Why is it okay for all those other commands  
to have a --force mode, but not git stash?