git 1.4.0 usability problem

git 1.4.0 usability problem

[Jeff Garzik]

Now that kernel 2.6.17 is out, I updated all my repositories to be based 
against that kernel.  And for each repository I updated, my merge was 
rejected, due to an error similar to:

> fatal: Untracked working tree file '.gitignore' would be overwritten by merge.

I am only able to merge if I delete files in the working directory, so 
that git stops complaining on merge.

This behavior is new with git 1.4.0, which Fedora Extras just added.  I 
verified that merges work as expected in git 1.3.3, the last version 
Fedora Extras shipped prior to 1.4.0.

This behavior is a definite regression, that impacts workflow :(

Here is how to reproduce:

git clone -l $url/torvalds/linux-2.6.git tmp-2.6
cd tmp-2.6
cp .git/refs/tags/v2.6.12 .git/refs/heads/tmp
git checkout -f tmp
git pull . master
# watch OBVIOUS FAST-FORWARD MERGE complain about untracked
# working tree files

Regards,

	Jeff

Re: git 1.4.0 usability problem

[Ryan Anderson]

On Sun, Jun 18, 2006 at 09:40:06AM -0400, Jeff Garzik wrote:
> Now that kernel 2.6.17 is out, I updated all my repositories to be based 
> against that kernel.  And for each repository I updated, my merge was 
> rejected, due to an error similar to:
> 
> >fatal: Untracked working tree file '.gitignore' would be overwritten by 
> >merge.
> 
> I am only able to merge if I delete files in the working directory, so 
> that git stops complaining on merge.
> 
> This behavior is new with git 1.4.0, which Fedora Extras just added.  I 
> verified that merges work as expected in git 1.3.3, the last version 
> Fedora Extras shipped prior to 1.4.0.
> 
> This behavior is a definite regression, that impacts workflow :(
> 
> Here is how to reproduce:
> 
> git clone -l $url/torvalds/linux-2.6.git tmp-2.6

At this point you have master checked out, and recorded properly in the
index.

> cd tmp-2.6
> cp .git/refs/tags/v2.6.12 .git/refs/heads/tmp
> git checkout -f tmp

Here, you throw that index away, ignore the contents of the working
tree, and checkout tmp.

> git pull . master
> # watch OBVIOUS FAST-FORWARD MERGE complain about untracked
> # working tree files

At this point, you have a working tree containing files leftover from
the checkout of master, but which are totally unknown to the 2.6.12
tree, and so are untracked.  The fast-forward is trying hard not to
overwrite things it shouldn't be messing with, and so complains.

The fix is to drop the "-f" from git checkout, and things should work
correctly.  ("-f" should really not be a normal thing to use. For
switching branches, "git checkout" should be sufficient, and should
result ina working tree that doesn't contain nearly as many potential
conflict sources.

-- 

Ryan Anderson
  sometimes Pug Majere

Re: git 1.4.0 usability problem

[Junio C Hamano]

Jeff Garzik <jeff@garzik.org> writes:

> Now that kernel 2.6.17 is out, I updated all my repositories to be
> based against that kernel.  And for each repository I updated, my
> merge was rejected, due to an error similar to:
>
>> fatal: Untracked working tree file '.gitignore' would be overwritten by merge.
>
> I am only able to merge if I delete files in the working directory, so
> that git stops complaining on merge.
>
> This behavior is new with git 1.4.0, which Fedora Extras just added.
> I verified that merges work as expected in git 1.3.3, the last version
> Fedora Extras shipped prior to 1.4.0.
>
> This behavior is a definite regression, that impacts workflow :(
>
> Here is how to reproduce:
>
> git clone -l $url/torvalds/linux-2.6.git tmp-2.6
> cd tmp-2.6
> cp .git/refs/tags/v2.6.12 .git/refs/heads/tmp
> git checkout -f tmp
> git pull . master

I was not happy with this change myself when I saw the extent of
damage it caused to the existing testsuite that was loosely
written (fcc387db9bc453dc7e07a262873481af2ee9e5c8 introduced
this change, and the needed changes to the testsuite can be seen
in the same commit).

This was done in response to this problem report:

        From: Santi <sbejar@gmail.com>
        Subject: Merge with local conflicts in new files
        Date: Wed, 17 May 2006 00:00:10 +0200
        Message-ID: <8aa486160605161500m1dd8428cj@mail.gmail.com>

        Hi *,

              In the case of:

        - You merge from a branch with new files
        - You have these files in the working directory
        - You do not have these files in the HEAD.

          The end result is that you lose the content of these files.

It is an improvement not to lose untracked files, and this is
consistent with how "read-tree -m -u" tries to protect your
changes to tracked files.  When moving around in the history of
the same project without using "git checkout" (sans -f),
however, it is bound to cause the above trouble whenever your
version switching involves created/deleted files.

I am open to suggestions to make this check easily overridable.
I suspect it should be sufficient to disable verify_absent()
check in builtin-read-tree.c when the user tells us to do so,
but the issue is how.  I can think of a few ways:

 (1) define an environment variable, return from verify_absent()
     without checking when that variable is set, and have the
     user run

     	$ GIT_UNTRACKED_CLOBBER_OK=t git pull

     when clobbering is desired.

 (2) In addition to the above, modify Porcelainish commands such
     as checkout, pull, and merge to set the environment variable
     when --clobber-ok flag is given to them.

 (3) define a new flag --clobber-ok to git-read-tree, pass it
     around from Porcelainish commands that would eventually
     call "read-tree -m -u".

I suspect the last one would be quite intrusive.  Independent of
the above, we could have a configuration item "core.clobberok = true"
to always disable the check for the repository.

It might be better to give the user a way to recover from the
situation while keeping things still safer than before by not
giving the above "clobber-ok" flag.  For example, "read-tree -m
-u" currently dies on the first "refraining from clobbering
untracked file" message, but there is not an obvious way to list
all untracked files that will be clobbered by the operation so
that you can make sure they are something you do not care about
and remove them yourself before retrying.

Re: git 1.4.0 usability problem

[Junio C Hamano]

Jeff Garzik <jeff@garzik.org> writes:

> Here is how to reproduce:

This is not related to the "not clobbering untracked files"
safety valve under discussion, but one thing I noticed.

> git clone -l $url/torvalds/linux-2.6.git tmp-2.6
> cd tmp-2.6
> cp .git/refs/tags/v2.6.12 .git/refs/heads/tmp
> git checkout -f tmp

This should never have been supported.  At this point tmp is a
tag object that is under heads/ -- a definite no-no.  We should
make checkout more careful to complain about it.

Doing

        git update-ref refs/heads/tmp $(git rev-parse v2.6.12^0)

instead of "cp" is kosher, and

	git-rev-parse v2.6.12^0 >.git/refs/heads/tmp

is OK under the current implementation of refs.

> git pull . master
> # watch OBVIOUS FAST-FORWARD MERGE complain about untracked
> # working tree files

In any case, here is a patch I think would alleviate your
original problem.

Sorry for the trouble.  I really did not want to disrupt the
workflow of old timers in the name of making it safer for new
people.  Could you comment on whether this is an acceptable
approach?

-- >8 --
[PATCH] Conditionally loosen "no clobber untracked files" safety valve.

This introduces a new configuration item "core.oktoclobber" to
control how untracked working tree file is handled during branch
switching.

The safety valve introduced during 1.4.0 development cycle
refrains from checking out a file that exists in the working
tree, not in the current HEAD tree and exists in the branch we
are switching to, in order to prevent accidental and
irreversible lossage of user data.  This can be controlled by
having core.oktoclobber configuration item:

 - When core.oktoclobber is set to "false" (the default),
   untracked working tree files are never overwritten.

 - When core.oktoclobber is set to "true", the check is
   disabled.

 - When core.oktoclobber is set to "ask", and both standard
   input and standard error streams are connected to the
   terminal, we ask the user if it is OK to clobber.  You can
   answer:

	y: to allow clobbering only one path; the question is
	   asked again for other paths.
        n: to stop the operation.
	a: to allow clobbering any untracked files; the question
	   is not asked again.

   If the configuration item is set to "ask" but the program is
   not talking to a terminal, it refrains from clobbering the
   untracked files.

Signed-off-by: Junio C Hamano <junkio@cox.net>

---

 builtin-read-tree.c |   43 ++++++++++++++++++++++++++++++++++++++++++-
 cache.h             |    6 ++++++
 config.c            |   20 ++++++++++++++++++++
 environment.c       |    1 +
 4 files changed, 69 insertions(+), 1 deletions(-)

diff --git a/builtin-read-tree.c b/builtin-read-tree.c
index 04506da..7a7018c 100644
--- a/builtin-read-tree.c
+++ b/builtin-read-tree.c
@@ -477,6 +477,46 @@ static void invalidate_ce_path(struct ca
 		cache_tree_invalidate_path(active_cache_tree, ce->name);
 }
 
+static int ok_to_clobber_untracked(const char *path, const char *action)
+{
+	switch (ok_to_clobber_untracked_files) {
+	case NEVER_CLOBBER:
+		return 0;
+	case OK_TO_CLOBBER:
+		return 1;
+	case ASK_TO_CLOBBER:
+		if (isatty(0) && isatty(2)) {
+			char answer[1024];
+			while (1) {
+				answer[0] = '\0';
+				fprintf(stderr,
+					"Untracked working tree file '%s' is"
+					" about to be %s.  Is it OK "
+					"[y]es/[n]o/yes to [a]ll? ",
+					path, action);
+				fgets(answer, sizeof(answer), stdin);
+				switch (answer[0]) {
+				case 'y': case 'Y':
+					return 1;
+				case 'n': case 'N':
+					return 0;
+				case 'a': case 'A':
+					ok_to_clobber_untracked_files =
+						OK_TO_CLOBBER;
+					return 1;
+				default:
+					fprintf(stderr,
+						"I do not understand.\n");
+				}
+			}
+		}
+		else {
+			ok_to_clobber_untracked_files = NEVER_CLOBBER;
+			return 0;
+		}
+	}
+}
+
 /*
  * We do not want to remove or overwrite a working tree file that
  * is not tracked.
@@ -485,7 +525,8 @@ static void verify_absent(const char *pa
 {
 	struct stat st;
 
-	if (index_only || reset || !update)
+	if (index_only || reset || !update ||
+	    ok_to_clobber_untracked(path, action))
 		return;
 	if (!lstat(path, &st))
 		die("Untracked working tree file '%s' "
diff --git a/cache.h b/cache.h
index f630cf4..7468440 100644
--- a/cache.h
+++ b/cache.h
@@ -183,6 +183,12 @@ extern int log_all_ref_updates;
 extern int warn_ambiguous_refs;
 extern int diff_rename_limit_default;
 extern int shared_repository;
+enum ok_to_clobber {
+	NEVER_CLOBBER = 0,
+	OK_TO_CLOBBER,
+	ASK_TO_CLOBBER
+};
+extern enum ok_to_clobber ok_to_clobber_untracked_files;
 extern const char *apply_default_whitespace;
 
 #define GIT_REPO_VERSION 0
diff --git a/config.c b/config.c
index 984c75f..13f5f4f 100644
--- a/config.c
+++ b/config.c
@@ -251,6 +251,21 @@ int git_config_bool(const char *name, co
 	return git_config_int(name, value) != 0;
 }
 
+static enum ok_to_clobber git_config_clobber(const char *var, const char *value)
+{
+	if (!strcasecmp(value, "ask"))
+		return ASK_TO_CLOBBER;
+	if (!strcasecmp(value, "yes"))
+		return OK_TO_CLOBBER;
+	if (!strcasecmp(value, "ok"))
+		return NEVER_CLOBBER;
+	if (!strcasecmp(value, "no"))
+		return NEVER_CLOBBER;
+	if (git_config_bool(var, value))
+		return OK_TO_CLOBBER;
+	return NEVER_CLOBBER;
+}
+
 int git_default_config(const char *var, const char *value)
 {
 	/* This needs a better name */
@@ -279,6 +294,11 @@ int git_default_config(const char *var, 
 		return 0;
 	}
 
+	if (!strcmp(var, "core.oktoclobber")) {
+		ok_to_clobber_untracked_files = git_config_clobber(var, value);
+		return 0;
+	}
+
 	if (!strcmp(var, "user.name")) {
 		safe_strncpy(git_default_name, value, sizeof(git_default_name));
 		return 0;
diff --git a/environment.c b/environment.c
index 2e79eab..c388b5b 100644
--- a/environment.c
+++ b/environment.c
@@ -19,6 +19,7 @@ int warn_ambiguous_refs = 1;
 int repository_format_version = 0;
 char git_commit_encoding[MAX_ENCODING_LENGTH] = "utf-8";
 int shared_repository = 0;
+extern enum ok_to_clobber ok_to_clobber_untracked_files = NEVER_CLOBBER;
 const char *apply_default_whitespace = NULL;
 
 static char *git_dir, *git_object_dir, *git_index_file, *git_refs_dir,

Re: git 1.4.0 usability problem

[Ryan Anderson]

On Sun, Jun 18, 2006 at 09:43:00AM -0700, Ryan Anderson wrote:
> 
> The fix is to drop the "-f" from git checkout, and things should work
> correctly.  ("-f" should really not be a normal thing to use. For
> switching branches, "git checkout" should be sufficient, and should
> result ina working tree that doesn't contain nearly as many potential
> conflict sources.

I wrote all of that from memory, but I figured I should really test it:

$ git branch tmp v2.6.12
$ git checkout tmp
$ git pull . master
Updating from 9ee1c939d1cb936b1f98e8d81aeffab57bae46ab to
553698f944ed715dfe023b4cef07601f0ce735f0
Checking files out...
 100% (14284/14284) done
 Fast forward
(Watch insanely large diffstat blow by)
$ git status
# On branch refs/heads/tmp
nothing to commit
$ git branch
  master
  origin
  ryan
* tmp

So, I think all you need to do is drop the "-f" from the call to
checkout, and the issues will be fixed, for your particular use case.




-- 

Ryan Anderson
  sometimes Pug Majere

Re: git 1.4.0 usability problem

[Jeff Garzik]

Junio C Hamano wrote:
> Jeff Garzik <jeff@garzik.org> writes:
> 
>> Here is how to reproduce:
> 
> This is not related to the "not clobbering untracked files"
> safety valve under discussion, but one thing I noticed.
> 
>> git clone -l $url/torvalds/linux-2.6.git tmp-2.6
>> cd tmp-2.6
>> cp .git/refs/tags/v2.6.12 .git/refs/heads/tmp
>> git checkout -f tmp
> 
> This should never have been supported.  At this point tmp is a
> tag object that is under heads/ -- a definite no-no.  We should
> make checkout more careful to complain about it.
> 
> Doing
> 
>         git update-ref refs/heads/tmp $(git rev-parse v2.6.12^0)
> 
> instead of "cp" is kosher, and
> 
> 	git-rev-parse v2.6.12^0 >.git/refs/heads/tmp
> 
> is OK under the current implementation of refs.

Sorry about that.  The contrived example produced the same results as 
the real-world example (updating jgarzik/{libata-dev,scsilun-2.6}.git 
branches).


>> git pull . master
>> # watch OBVIOUS FAST-FORWARD MERGE complain about untracked
>> # working tree files
> 
> In any case, here is a patch I think would alleviate your
> original problem.
> 
> Sorry for the trouble.  I really did not want to disrupt the
> workflow of old timers in the name of making it safer for new
> people.  Could you comment on whether this is an acceptable
> approach?
> 
> -- >8 --
> [PATCH] Conditionally loosen "no clobber untracked files" safety valve.
> 
> This introduces a new configuration item "core.oktoclobber" to
> control how untracked working tree file is handled during branch
> switching.
> 
> The safety valve introduced during 1.4.0 development cycle
> refrains from checking out a file that exists in the working
> tree, not in the current HEAD tree and exists in the branch we
> are switching to, in order to prevent accidental and
> irreversible lossage of user data.  This can be controlled by
> having core.oktoclobber configuration item:

I'm a bit under the weather today, so I must defer thinking about this. 
  :)  But if what Ryan says is true, about simply needing to ditch the 
"-f" argument I habitually pass to 'git checkout', would that alleviate 
the need for a patch?

FWIW, my workflow is

	cd /repos
	cd linux-2.6
	git pull
	cd ../libata-dev
	git checkout -f master	# guarantee any WIP goes away
	git pull ../linux-2.6	# update vanilla branch
	git checkout -f upstream# switch to working branch,
				# guarantee any WIP goes away.
	git pull . master	# pull latest upstream updates
	build/test/etc.
	git checkout -f sii-m15w # switch to topic-specific branch,
				 # whose parent is always #upstream
	git pull . upstream
	build/test/etc.
	repeat for several topics (on-going devel branches)
	git checkout -f -b ALL upstream	# create everything-together
					# test branch
	git pull . sii-m15w
	git pull . topicB
	git pull . topicC
	build/test/etc.
	git checkout -f master
	./push		# calls 'git push --force --all $url'

More tomorrow,

	Jeff

Re: git 1.4.0 usability problem

[Junio C Hamano]

Jeff Garzik <jeff@garzik.org> writes:

> But if what Ryan says is true, about simply needing to ditch
> the "-f" argument I habitually pass to 'git checkout', would that
> alleviate the need for a patch?

To a certain degree, yes.

But I suspect (I am not a kernel person so I can only speculate)
in the kernel workflow you would often pick up a patch from the
list, apply it to your working tree (without applying it to your
index, IOW with "patch -p1" or "git apply", not with "git apply
--index"), and then decide to pull from somewhere else while
your working tree is dirty (but index is not).  The patch might
have created a new file or two, and the pull may also contain a
commit that applied the same patch in question.  The no-clobber
check would trigger in such a case preventing you from pulling,
and neither "checkout" nor "checkout -f" would clean these new
files that you have not told git about.

> FWIW, my workflow is
>
> 	cd /repos
> 	cd linux-2.6
> 	git pull
> 	cd ../libata-dev
> 	git checkout -f master	# guarantee any WIP goes away

We kept saying "with checkout -f any dirty state goes away from
your working tree".  It is true only with respect to the files
git knows about.  The trouble you experienced was about
untracked files -- files git does not know about, and they will
be left behind.

So if path F is in test branch head and linus branch head, but
not in your master branch head, and you have checked out test in
your working tree, even if path F in the working tree is clean:

	git checkout -f master

will leave F behind.  If you pull from linus at this point, the
check would trigger.  Running "git checkout master" without -f
would however remove it and you would not have the problem.
That is what Ryan's suggestion is about.

However, if you have a patch you got from somebody on the net
that creates F (maybe it is the same patch linus accepted
recently) while on your master branch, and you tried to examine
it by applying it to your working tree with "patch -p1" or "git
apply", your working tree will have F that is not in index (and
in your branch head).  In that state if you pull from linus, the
no-clobber check triggers.  In this case, neither "git checkout
master", "git checkout -f master", nor "git reset --hard master"
would remove F, because git does not even know about it, so
pulling from linus would fail.  "git clean" would removes F, so
it may not be a big deal, but it is rather a heavy-handed
operation that removes all crufts, so you may find it a
not-so-useful workaround (I certainly would, and that is the
primary reason I rarely use "git clean" myself).

It's a bit sad situation.  One of the useful feature of git is
that you can continue working in a dirty working tree as long as
your index is clean and your local changes do not interfere with
a merge, patch application, or branch switching.  Strictly
speaking, this no-clobber check _is_ about a local change that
does interfere with the operation, so from theoretical point of
view it is a good safety measure, but at the same time we did
not consider untracked files as precious until recently, and I
suspect that "the same patch applied elsewhere to create the
same file" pattern is reasonably common that this safety valve
may interfere the work more often than it may help avoiding
mistakes.

Re: git 1.4.0 usability problem

[Jeff Garzik]

Here's a real world example of the 1.4.0 change breaking a merge:

("netdev-2.6" == local clone of kernel.org/...jgarzik/netdev-2.6.git)
[jgarzik@pretzel netdev-2.6]$ git branch
   ALL
   e100-sbit
* master
   upstream
   upstream-linus

[jgarzik@pretzel netdev-2.6]$ git pull /spare/repo/linux-2.6
Generating pack...
Done counting 3427 objects.
Result has 2510 objects.
Deltifying 2510 objects.
  100% (2510/2510) done
Unpacking 2510 objects
Total 2510, written 2510 (delta 2024), reused 0 (delta 0)
  100% (2510/2510) done
Updating from 427abfa28afedffadfca9dd8b067eb6d36bac53f to 
25f42b6af09e34c3f92107b36b5aa6edc2fdba2f
fatal: Untracked working tree file 'drivers/net/myri10ge/Makefile' would 
be overwritten by merge.

EXPLANATION:

* drivers/net/myri10ge/Makefile exists in latest Linus kernel tree, 
stored locally in /spare/repo/linux-2.6.
* drivers/net/myri10ge/Makefile exists in netdev-2.6#upstream and 
netdev-2.6#upstream-linus branches.
* drivers/net/myri10ge/Makefile does not exist in current branch, 
netdev-2.6#master.

Re: git 1.4.0 usability problem

[Ryan Anderson]

On Tue, Jun 20, 2006 at 04:36:46AM -0400, Jeff Garzik wrote:
> Here's a real world example of the 1.4.0 change breaking a merge:
> 
> ("netdev-2.6" == local clone of kernel.org/...jgarzik/netdev-2.6.git)
> [jgarzik@pretzel netdev-2.6]$ git branch
>   ALL
>   e100-sbit
> * master
>   upstream
>   upstream-linus
> 
> [jgarzik@pretzel netdev-2.6]$ git pull /spare/repo/linux-2.6
> Generating pack...
> Done counting 3427 objects.
> Result has 2510 objects.
> Deltifying 2510 objects.
>  100% (2510/2510) done
> Unpacking 2510 objects
> Total 2510, written 2510 (delta 2024), reused 0 (delta 0)
>  100% (2510/2510) done
> Updating from 427abfa28afedffadfca9dd8b067eb6d36bac53f to 
> 25f42b6af09e34c3f92107b36b5aa6edc2fdba2f
> fatal: Untracked working tree file 'drivers/net/myri10ge/Makefile' would 
> be overwritten by merge.
> 
> EXPLANATION:
> 
> * drivers/net/myri10ge/Makefile exists in latest Linus kernel tree, 
> stored locally in /spare/repo/linux-2.6.
> * drivers/net/myri10ge/Makefile exists in netdev-2.6#upstream and 
> netdev-2.6#upstream-linus branches.
> * drivers/net/myri10ge/Makefile does not exist in current branch, 
> netdev-2.6#master.

If that is the case, how did it get into the directory, then?

I suspect the history we're missing is this:

git checkout upstream
...
git checkout -f master

If you have a clean tree, there's not really a reason to use -f.  It
actively hurts you by confusing git as to what state your directory is
in. (If you don't have a clean tree, well, using -f will orphan new
files that Git knows about, and overwrite the changes on ones it does,
so it's still rather inconsistent, and probably not at all what you
intend.)

Instead of using "git checkout -f" to abandon any WIP, try something
like this:

	git branch throw-away HEAD
	git checkout throw-away
	git commit -a -m "throw away"
	git checkout $1
	git branch -D throw-away


i.e:
$ sed -i -e 's/PHONY/YNOHP/g' Makefile
$ git branch throw-away HEAD
$ git checkout throw-away
$ git diff |diffstat
 Makefile |   60 ++++++++++++++++++++++++++++++------------------------------
$ git commit -a -m "throw-away"
$ git checkout master
$ git branch -d throw-away
The branch 'throw-away' is not a strict subset of your current HEAD.
If you are sure you want to delete it, run 'git branch -D throw-away'.
$ git branch -D throw-away
Deleted branch throw-away.
$ git diff | diffstat
0 files changed
$ git branch throw-away HEAD
$ cp Makefile Makefile-ryan
$ git add Makefile-ryan 
$ git checkout throw-away
$ git diff | diffstat
 0 files changed
$ git status
# On branch refs/heads/throw-away
#
# Updated but not checked in:
#   (will commit)
#
#	new file: Makefile-ryan
#
$ git commit -a -m "throw-away"
$ git checkout master
$ git status
nothing to commit

Note that using plain "checkout" should be noticeably faster, too.





-- 

Ryan Anderson
  sometimes Pug Majere

Re: git 1.4.0 usability problem

[Junio C Hamano]

Jeff Garzik <jeff@garzik.org> writes:

> Here's a real world example of the 1.4.0 change breaking a merge:
>
> ("netdev-2.6" == local clone of kernel.org/...jgarzik/netdev-2.6.git)
> [jgarzik@pretzel netdev-2.6]$ git branch
>   ALL
>   e100-sbit
> * master
>   upstream
>   upstream-linus
>
> [jgarzik@pretzel netdev-2.6]$ git pull /spare/repo/linux-2.6
> ...
> Updating from 427abfa28afedffadfca9dd8b067eb6d36bac53f to
> 25f42b6af09e34c3f92107b36b5aa6edc2fdba2f
> fatal: Untracked working tree file 'drivers/net/myri10ge/Makefile'
> would be overwritten by merge.
>
> EXPLANATION:
>
> * drivers/net/myri10ge/Makefile exists in latest Linus kernel tree,
> stored locally in /spare/repo/linux-2.6.
> * drivers/net/myri10ge/Makefile exists in netdev-2.6#upstream and
> netdev-2.6#upstream-linus branches.
> * drivers/net/myri10ge/Makefile does not exist in current branch,
> netdev-2.6#master.

Requesting one more bit of explanation.

When you did this pull, you were on your "master" branch.  Whose
HEAD does not have the myri10ge/Makefile in its tree.  And you
did not have that path in the index either (otherwise it would
not have said "untracked working tree file").  Did you have that
file in your working tree?  And if so how/why?

If you did something like this before ending up in your "master"
branch, I think you would have such a file:

	$ git branch ;# you were in upstream-linus branch
          ALL
          e100-sbit
          master
          upstream
        * upstream-linus
        $ git diff      ;# and were up-to-date -- no output nor
        $ git diff HEAD ;# local changes
        $ git checkout -f master ;# but you switched to master with -f

With "checkout -f", your local changes to files that appear in
the "master" branch will be overwritten, but the files that are
supposed to disappear because you are switching out of the
current branch (e.g. myri10ge/Makefile) are left behind.

It probably would make sense to change "checkout -f" so that it
removes such files from your working tree to support this
particular usage.  We kept saying "with -f local dirty state
will be gone", but "checkout -f" does not do as thorough job as
"reset --hard".  Changing "checkout -f next-branch" to do the
moral equivalent of "reset --hard HEAD && checkout next-branch"
would solve this problem, and I do not think changing it that
way would not have any negative impact.

I suspect this patch would do exactly that...

diff --git a/git-checkout.sh b/git-checkout.sh
index 564117f..193f6c5 100755
--- a/git-checkout.sh
+++ b/git-checkout.sh
@@ -137,7 +137,7 @@ # what we already had
 
 if [ "$force" ]
 then
-    git-read-tree --reset $new &&
+    git-read-tree --reset -u $new &&
 	git-checkout-index -q -f -u -a
 else
     git-update-index --refresh >/dev/null

[PATCH] checkout -f: do not leave untracked working tree files.

[Junio C Hamano]

Earlier we did not consider untracked working tree files
"precious", but we have always considered them fair game to
clobber.  These days, branch switching by read-tree is more
careful and tries to protect untracked working tree files.  This
caused the following workflow to stop working:

	git checkout one-branch-with-file-F
	git checkout -f another-without-file-F
	git pull . one-branch-with-file-F

Because the second checkout leaves F from the previous state as
untracked file in the working tree, the merge would fail, trying
to protect F from being clobbered.

This changes "git checkout -f" to remove working tree files that
are known to git in the switched-from state but do not exist in
the switched-to state, borrowing the same logic from "reset --hard".

Signed-off-by: Junio C Hamano <junkio@cox.net>
---

 * I am going to bed without trying this out since it is very
   late tonight.  Might be in "pu" or "next" tomorrow depending
   on my mood.  If this works out for Jeff, I can simply drop
   the "core.oktoclobber = ask" patch from my topics -- although
   I kind of liked that one ;-).

 git-checkout.sh |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/git-checkout.sh b/git-checkout.sh
index 564117f..77c2593 100755
--- a/git-checkout.sh
+++ b/git-checkout.sh
@@ -137,8 +137,7 @@ # what we already had
 
 if [ "$force" ]
 then
-    git-read-tree --reset $new &&
-	git-checkout-index -q -f -u -a
+    git-read-tree --reset -u $new
 else
     git-update-index --refresh >/dev/null
     merge_error=$(git-read-tree -m -u $old $new 2>&1) || (
-- 
1.4.0.g59268

Re: [PATCH] checkout -f: do not leave untracked working tree files.

[Santi Béjar]

Junio C Hamano <junkio@cox.net> writes:

> Earlier we did not consider untracked working tree files
> "precious", but we have always considered them fair game to
> clobber.  These days, branch switching by read-tree is more
> careful and tries to protect untracked working tree files.  This
> caused the following workflow to stop working:
>
> 	git checkout one-branch-with-file-F
> 	git checkout -f another-without-file-F
> 	git pull . one-branch-with-file-F
>
> Because the second checkout leaves F from the previous state as
> untracked file in the working tree, the merge would fail, trying
> to protect F from being clobbered.
>
> This changes "git checkout -f" to remove working tree files that
> are known to git in the switched-from state but do not exist in
> the switched-to state, borrowing the same logic from "reset --hard".
>

I agree with this patch (without testing).

Another thing to take into account is that, for this particular
case/sequence, the untracked file-F is exactly the same as the one from
the pull, so you are not overwritting that file and it could succeed.

    Santi

Re: [PATCH] checkout -f: do not leave untracked working tree files.

[Junio C Hamano]

Santi Béjar <sbejar@gmail.com> writes:

> Another thing to take into account is that, for this particular
> case/sequence, the untracked file-F is exactly the same as the one from
> the pull, so you are not overwritting that file and it could succeed.

Yes and no.  The untracked file is not even known to git when
pull is happening (that is the definition of "untracked").  Yes,
we could see if the contents happen to match and choose to
overwrite, but I think it is often more likely that the contents
do not match that it is not worth it (I have not examined what
it would involve to add that check yet, so it might turn out to
be a cheap check but I doubt it).

Re: [PATCH] checkout -f: do not leave untracked working tree files.

[Alexander Litvinov]

Good news. I have a habbit to switch branches using two commands:

git checkout -f second-branch
git clean -d -q

Now this will work with single command. Thanks.

Re: [PATCH] checkout -f: do not leave untracked working tree files.

[Junio C Hamano]

Alexander Litvinov <lan@academsoft.ru> writes:

> Good news. I have a habbit to switch branches using two commands:
>
> git checkout -f second-branch
> git clean -d -q
>
> Now this will work with single command. Thanks.

"will" meaning "you suspect" or "you tried and confirmed"?

Re: [PATCH] checkout -f: do not leave untracked working tree files.

[Alexander Litvinov]

> "will" meaning "you suspect" or "you tried and confirmed"?
:-) I suspect.

Lets test. Your pu branch (5af05efbeaa06005596129fb111a739a87f8a883) really 
works. I have cvs repo and use git for daily work. I have head and 2 branches 
from cvs and store CVS/* files in git too. Usualy this operation pruduce a 
lot of CVS/Tag files:

git checkout -f cvs-v52 (this is one of by cvs's branch)
cvs -q up -dP (nothing have been updated)
git checkout -f master (this is a dev branch from cvs head)
git status

pu branch nuke them when use git checkout -f !

Re: [PATCH] checkout -f: do not leave untracked working tree files.

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1173 bytes --]

On Tue, 20 Jun 2006 02:50:08 -0700, Junio C Hamano wrote:
> 
> Earlier we did not consider untracked working tree files
> "precious", but we have always considered them fair game to
> clobber.  These days, branch switching by read-tree is more
> careful and tries to protect untracked working tree files.  This
> caused the following workflow to stop working:
> 
> 	git checkout one-branch-with-file-F
> 	git checkout -f another-without-file-F
> 	git pull . one-branch-with-file-F

Another one that a colleague of mine hit is:

	git checkout -b branch-without-file branch-with-file
	git rm some-file
	# Allow for some external changes on branch-with-file
	git pull . branch-with-file

One possibility for fixing this case is to make git-rm delete the file
by default, (that is, act as if the current -f option is
passed). There's no real safety concern unless the file is dirty,
(could require -f again for that case).

>                 If this works out for Jeff, I can simply drop
>    the "core.oktoclobber = ask" patch from my topics -- although
>    I kind of liked that one ;-).

Fixing the behavior instead of adding configuration is definitely a
good plan.

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]