[PATCH 1/2] init-db.c: honor case on case preserving fs

[PATCH 1/2] init-db.c: honor case on case preserving fs

[Reuben Hawkins]

Most case-insensitive filesystems are case-preserving. In these
filesystems (such as HFS+ on OS X) you can name a file Filename.txt,
then rename the file to FileName.txt.  That file will be accessible
by both filenames, but the case is otherwise honored.  We don't want
to have git ignore case on these case-preserving filesystem
implementations.

This change adds an additional check in init-db.c before
automatically setting core.ignorecase to true.

This fixes a problem where if you import an hg repository, using
git-remote-hg, on two OSX systems, one with a case-sensitive fs and
the other with case-insensitive fs, the sha1 commit ids of the
repositories diverge on commits where a file was renamed, but only
the case in the filename changes (for example renaming
Filename.cpp -> FileName.cpp).

The alternative solutions are
* to set ignore_case to 0 in fast-import.c at runtime
* explicitly use strcmp, rather than strcmp_icase (also in
  fast-import.c)
* completely rework ignorecase into something that can handle more
  options (true, false, sometimes, maybe, partially, etc...)

Signed-off-by: Reuben Hawkins <reubenhwk@gmail.com>
---
 builtin/init-db.c |   43 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

diff --git a/builtin/init-db.c b/builtin/init-db.c
index 78aa387..34f09d8 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -288,8 +288,47 @@ static int create_default_files(const char *template_path)
 		/* Check if the filesystem is case-insensitive */
 		path[len] = 0;
 		strcpy(path + len, "CoNfIg");
-		if (!access(path, F_OK))
-			git_config_set("core.ignorecase", "true");
+		if (!access(path, F_OK)) {
+			/*
+			 * This filesystem is at least partially case-insensitive.  Let's
+			 * find out if this filesystem is completely case-insensitive.
+			 *
+			 * Create a CamelCase file here, make sure readdir reads a
+			 * CamelCase file below.
+			 */
+			int completely_insensitive_fs = 1;
+			char const * const case_check_filename = ".CaseCheck";
+			struct dirent *dirent;
+			FILE *case_file;
+			DIR *dir;
+
+			path[len] = 0;
+			strcpy(path + len, case_check_filename);
+			case_file = fopen(path, "w");
+			if (!case_file)
+				die_errno(_("cannot open '%s'"), path);
+			fclose(case_file);
+
+			path[len] = 0;
+			dir = opendir(path);
+			if (!dir)
+				die_errno(_("cannot opendir '%s'"), path);
+
+			while ( (dirent = readdir(dir)) ) {
+				if (0 == strcmp(case_check_filename, dirent->d_name)) {
+					completely_insensitive_fs = 0;
+					break;
+				}
+			}
+
+			closedir(dir);
+			path[len] = 0;
+			strcpy(path + len, case_check_filename);
+			unlink(path);
+
+			if (completely_insensitive_fs)
+				git_config_set("core.ignorecase", "true");
+		}
 		probe_utf8_pathname_composition(path, len);
 	}
 
-- 
1.7.9.5

[PATCH 2/2] init-db.c: factor out probe_case_sensitivity func

[Reuben Hawkins]

create_default_files was getting a bit long and indented.  This
particular bit of code got long enough to be its own function.

Signed-off-by: Reuben Hawkins <reubenhwk@gmail.com>
---
 builtin/init-db.c |   97 +++++++++++++++++++++++++++++------------------------
 1 file changed, 53 insertions(+), 44 deletions(-)

diff --git a/builtin/init-db.c b/builtin/init-db.c
index 34f09d8..bcf7f4c 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -177,6 +177,58 @@ static int git_init_db_config(const char *k, const char *v, void *cb)
 	return 0;
 }
 
+static void probe_case_sensitivity(char *path, size_t len)
+{
+	int completely_insensitive_fs = 1;
+	char const * const case_check_filename = ".CaseCheck";
+	struct dirent *dirent;
+	FILE *case_file;
+	DIR *dir;
+
+	/*
+	 * If we can't access "config" by the name "CoNfIg", this fs is
+	 * case sensitive...so do nothing and return.
+	 */
+	path[len] = 0;
+	strcpy(path + len, "CoNfIg");
+	if (access(path, F_OK))
+		return;
+
+	/*
+	 * This filesystem is at least partially case-insensitive.  Let's
+	 * find out if this filesystem is completely case-insensitive.
+	 *
+	 * Create a CamelCase file here, make sure readdir reads a
+	 * CamelCase file below.
+	 */
+	path[len] = 0;
+	strcpy(path + len, case_check_filename);
+	case_file = fopen(path, "w");
+	if (!case_file)
+		die_errno(_("cannot open '%s'"), path);
+	fclose(case_file);
+
+	path[len] = 0;
+	dir = opendir(path);
+	if (!dir)
+		die_errno(_("cannot opendir '%s'"), path);
+
+	while ( (dirent = readdir(dir)) ) {
+		if (0 == strcmp(case_check_filename, dirent->d_name)) {
+			completely_insensitive_fs = 0;
+			break;
+		}
+	}
+
+	closedir(dir);
+	path[len] = 0;
+	strcpy(path + len, case_check_filename);
+	unlink(path);
+
+	if (completely_insensitive_fs)
+		git_config_set("core.ignorecase", "true");
+}
+
 static int create_default_files(const char *template_path)
 {
 	const char *git_dir = get_git_dir();
@@ -285,50 +337,7 @@ static int create_default_files(const char *template_path)
 		else
 			git_config_set("core.symlinks", "false");
 
-		/* Check if the filesystem is case-insensitive */
-		path[len] = 0;
-		strcpy(path + len, "CoNfIg");
-		if (!access(path, F_OK)) {
-			/*
-			 * This filesystem is at least partially case-insensitive.  Let's
-			 * find out if this filesystem is completely case-insensitive.
-			 *
-			 * Create a CamelCase file here, make sure readdir reads a
-			 * CamelCase file below.
-			 */
-			int completely_insensitive_fs = 1;
-			char const * const case_check_filename = ".CaseCheck";
-			struct dirent *dirent;
-			FILE *case_file;
-			DIR *dir;
-
-			path[len] = 0;
-			strcpy(path + len, case_check_filename);
-			case_file = fopen(path, "w");
-			if (!case_file)
-				die_errno(_("cannot open '%s'"), path);
-			fclose(case_file);
-
-			path[len] = 0;
-			dir = opendir(path);
-			if (!dir)
-				die_errno(_("cannot opendir '%s'"), path);
-
-			while ( (dirent = readdir(dir)) ) {
-				if (0 == strcmp(case_check_filename, dirent->d_name)) {
-					completely_insensitive_fs = 0;
-					break;
-				}
-			}
-
-			closedir(dir);
-			path[len] = 0;
-			strcpy(path + len, case_check_filename);
-			unlink(path);
-
-			if (completely_insensitive_fs)
-				git_config_set("core.ignorecase", "true");
-		}
+		probe_case_sensitivity(path, len);
 		probe_utf8_pathname_composition(path, len);
 	}
 
-- 
1.7.9.5

Re: [PATCH 1/2] init-db.c: honor case on case preserving fs

[Torsten Bögershausen]

On 2014-02-01 10.14, Reuben Hawkins wrote:
> Most case-insensitive filesystems are case-preserving. In these
> filesystems (such as HFS+ on OS X) you can name a file Filename.txt,
> then rename the file to FileName.txt.  That file will be accessible
> by both filenames, but the case is otherwise honored.  We don't want
> to have git ignore case on these case-preserving filesystem
> implementations.

Yes, we want.
Because the file system will treat "Filename.txt" and "FileName.txt"
the same.
Whatever is on disc, the OS will not distinguish them.
(On a case-insensitive HFS+ partition).

And when core.ignorecase == true, Git does the same what the OS does,
ignore the case.

Could you describe the problems more in detail ?

Could you supply a test case, (or a short script) which shows
the problem and makes it reproducable for others?

Which problems does your patch solve, which can not be solved
by setting core.ignorecase==false manually?
 
/Torsten

Re: [PATCH 1/2] init-db.c: honor case on case preserving fs

[Dmitry Potapov]

On Sat, Feb 1, 2014 at 1:14 PM, Reuben Hawkins <reubenhwk@gmail.com> wrote:
> Most case-insensitive filesystems are case-preserving. In these
> filesystems (such as HFS+ on OS X) you can name a file Filename.txt,
> then rename the file to FileName.txt.  That file will be accessible
> by both filenames, but the case is otherwise honored.  We don't want
> to have git ignore case on these case-preserving filesystem
> implementations.

Actually, we want to ignore case on ANY case-insensitive file system whether
it is a case-preserving or not.

Let's suppose we have a central repository where Filename.txt is renamed
as FileName.txt. Now a user pulls changes from that repository on Windows.

With core.ignorecase = true, everything works perfectly:

$ git pull
remote: Counting objects: 3, done.
remote: Total 2 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (2/2), done.
From z:/shared/test2/
   3d885e6..ce5deef  master     -> origin/master
Updating 3d885e6..ce5deef
Fast-forward
 Filename.txt => FileName.txt | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename Filename.txt => FileName.txt (100%)

$ ls
FileName.txt

Now, if you try to do the same thing with core.ignorecase = false,
you get this:

$ git pull
remote: Counting objects: 3, done.
remote: Total 2 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (2/2), done.
From z:/shared/test2/
   3d885e6..ce5deef  master     -> origin/master
Updating 3d885e6..ce5deef
error: The following untracked working tree files would be overwritten by merge:
        FileName.txt
Please move or remove them before you can merge.
Aborting

I have tested this with git version 1.8.3.msysgit.0, but I don't think
it is any different with the latest version.

> This fixes a problem where if you import an hg repository, using
> git-remote-hg,

Perhaps, it fixes that case, but it breaks a far more common case.
So I believe that git-remote-hg should be corrected and not init-db.c


Dmitry

Re: [PATCH 1/2] init-db.c: honor case on case preserving fs

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

> On 2014-02-01 10.14, Reuben Hawkins wrote:
>> Most case-insensitive filesystems are case-preserving. In these
>> filesystems (such as HFS+ on OS X) you can name a file Filename.txt,
>> then rename the file to FileName.txt.  That file will be accessible
>> by both filenames, but the case is otherwise honored.  We don't want
>> to have git ignore case on these case-preserving filesystem
>> implementations.
>
> Yes, we want.
> Because the file system will treat "Filename.txt" and "FileName.txt"
> the same.

Another important thing to remember is that we cannot have these two
files at the same time on such a filesystem.

Somebody may have Filename.txt in the commit at the tip of the
history, you clone/fetch and check it out, and you will have
Filename.txt with the original contents.  We do not try to corrupt
the filename on core.ignorecase filesystem by any canonicalization.

But then you may edit that file, and you either deliberately or
without knowing (because some of your tools do this behind your
back) may end up saving the result as FileName.txt.  What happens?

When we ask "what is the contents of Filename.txt now?" (using the
original name still in the index) to the underlying system, we will
be given what you placed in FileName.txt.  We won't see "You do not
have Filename.txt, but you now have FileName.txt".

And that is the behaviour the end users (of not Git, but of a
platform with such a filesystem) do expect from their tools.  They
do not want to see "You no longer have Filename.txt, and you have a
new file FileName.txt".

Now think what "git add Filename.txt" should do at that point?  It
should not say "I was told to add Filename.txt, but there is no such
file, so I'll add nothing".  If you run "git add -u Filename.txt",
it should not say "I was told to add Filename.txt, but there is no
such file, so I'll remove existing Filename.txt from the index".

It must pick up the updated contents from your new FileName.txt,
update the index entry "Filename.txt", and the next "git commit"
must record it as an update to the same file.

If you are on the other hand trying to correct an earlier mistake of
having named the file "Filename.txt" but you now want to rename it
"FileName.txt", the above behaviour by core.ignorecase may make it a
bit cumbersome to do.  You can first remove it from the index and
then re-add it, I would think, as a workaround.  Having to do a
"workaround" is unfortunate but it is an unavoidable consequence of
having to choose between the two and having to pick one.  Most of
the time you do not want such a rename (or rather, the loss of the
file Filename.txt and the creation of the unrelated FileName.txt)
and a change from Filename.txt to FileName.txt is most likely to be
a mistake in the platform tool that mucked with the files on your
filesystem, so we choose to make it easy for the user not to be
disturbed by such a change.

Re: [PATCH 1/2] init-db.c: honor case on case preserving fs

[Dmitry Potapov]

On 2/2/14, Reuben Hawkins <reubenhwk@gmail.com> wrote:
>>
> This is a separate issue which core.ignorecase is sweeping under the rug.
>  When you get this error message, do an 'ls' and you'll see there isn't an
> untracked file called 'FileName.txt'.  There is, however, a tracked file
> called 'Filename.txt'.  Because there aren't really any untracked files,
> the merge should be able to complete without overwriting anything.

I am afraid but it is exactly the same issue, because it never happens
on any case-sensitive system. IIRC, Git stats 'FileName.txt' to verify
whether it exists or not, and the file system tells that this file
exists, because it ignores case. So the option is called 'ignorecase'.

No one meant this option to be about whether the file system preserves
case or not, because Git does not care about it and assumes that on
any case-insensitive filesystem, the right case is in the index, which is
usually better, because otherwise people can often change filenames
inadvertently.

I don't know much about importers, but I have looked at the source code
of fast-import.c and I do not see ignore_case there, though it may affect
it indirectly through some functions that it uses. AFAIK, fast-import does
not rely on the filesystem, it should always work with ignorecase=false.


Dmitry