[PATCH 0/7] Second part of msysgit/unicode

[PATCH 0/7] Second part of msysgit/unicode

[Stepan Kasal]

This is the second part of the time-proven unicode suport branch from msysgit.
This batch is a collection of small independent changes, limited to mingw.c.
The only exception is the last patch: it changes gitk and git-gui.

(The third and last part of msysgit/unicode branch will all be about
environment handling.)

Johannes Schindelin (1):
  Let mingw_execve() return an int

Karsten Blees (6):
  Win32: simplify internal mingw_spawn* APIs
  Win32: fix potential multi-threading issue
  MinGW: disable CRT command line globbing
  Win32: Unicode arguments (outgoing)
  Win32: Unicode arguments (incoming)
    - also contains small part of 8c452ca Win32: use low-level memory alloc...
  Unicode file name support (gitk and git-gui)

 compat/mingw.c          | 94 +++++++++++++++++++++++++++++++++++++------------
 git-gui/git-gui.sh      | 11 +++---
 git-gui/lib/browser.tcl |  2 +-
 git-gui/lib/index.tcl   |  6 ++--
 gitk-git/gitk           | 15 ++++----
 5 files changed, 90 insertions(+), 38 deletions(-)

-- 
2.0.0.9635.g0be03cb

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[PATCH 1/7] Let mingw_execve() return an int

[Stepan Kasal]

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Mon, 28 May 2012 21:21:39 -0500

This is in the great tradition of POSIX. Original fix by Olivier Refalo.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
---
 compat/mingw.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index d242557..7da73fa 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -1019,7 +1019,7 @@ static int try_shell_exec(const char *cmd, char *const *argv, char **env)
 	return pid;
 }
 
-static void mingw_execve(const char *cmd, char *const *argv, char *const *env)
+static int mingw_execve(const char *cmd, char *const *argv, char *const *env)
 {
 	/* check if git_command is a shell script */
 	if (!try_shell_exec(cmd, argv, (char **)env)) {
@@ -1027,11 +1027,12 @@ static void mingw_execve(const char *cmd, char *const *argv, char *const *env)
 
 		pid = mingw_spawnve(cmd, (const char **)argv, (char **)env, 0);
 		if (pid < 0)
-			return;
+			return -1;
 		if (waitpid(pid, &status, 0) < 0)
 			status = 255;
 		exit(status);
 	}
+	return -1;
 }
 
 int mingw_execvp(const char *cmd, char *const *argv)
-- 
2.0.0.9635.g0be03cb

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[PATCH 2/7] Win32: simplify internal mingw_spawn* APIs

[Stepan Kasal]

From: Karsten Blees <blees@dcon.de>
Date: Fri, 25 Nov 2011 21:33:17 +0100

The only public spawn function that needs to tweak the environment is
mingw_spawnvpe (called from start_command). Nevertheless, all internal
spawn* functions take an env parameter and needlessly pass the global
char **environ around. Remove the env parameter where it's not needed.

This removes the internal mingw_execve abstraction, which is no longer
needed.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
---
 compat/mingw.c | 23 ++++++++---------------
 1 file changed, 8 insertions(+), 15 deletions(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index 7da73fa..1c0b153 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -941,10 +941,9 @@ static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
 	return (pid_t)pi.dwProcessId;
 }
 
-static pid_t mingw_spawnve(const char *cmd, const char **argv, char **env,
-			   int prepend_cmd)
+static pid_t mingw_spawnv(const char *cmd, const char **argv, int prepend_cmd)
 {
-	return mingw_spawnve_fd(cmd, argv, env, NULL, prepend_cmd, 0, 1, 2);
+	return mingw_spawnve_fd(cmd, argv, environ, NULL, prepend_cmd, 0, 1, 2);
 }
 
 pid_t mingw_spawnvpe(const char *cmd, const char **argv, char **env,
@@ -986,7 +985,7 @@ pid_t mingw_spawnvpe(const char *cmd, const char **argv, char **env,
 	return pid;
 }
 
-static int try_shell_exec(const char *cmd, char *const *argv, char **env)
+static int try_shell_exec(const char *cmd, char *const *argv)
 {
 	const char *interpr = parse_interpreter(cmd);
 	char **path;
@@ -1004,7 +1003,7 @@ static int try_shell_exec(const char *cmd, char *const *argv, char **env)
 		argv2 = xmalloc(sizeof(*argv) * (argc+1));
 		argv2[0] = (char *)cmd;	/* full path to the script file */
 		memcpy(&argv2[1], &argv[1], sizeof(*argv) * argc);
-		pid = mingw_spawnve(prog, argv2, env, 1);
+		pid = mingw_spawnv(prog, argv2, 1);
 		if (pid >= 0) {
 			int status;
 			if (waitpid(pid, &status, 0) < 0)
@@ -1019,13 +1018,13 @@ static int try_shell_exec(const char *cmd, char *const *argv, char **env)
 	return pid;
 }
 
-static int mingw_execve(const char *cmd, char *const *argv, char *const *env)
+int mingw_execv(const char *cmd, char *const *argv)
 {
 	/* check if git_command is a shell script */
-	if (!try_shell_exec(cmd, argv, (char **)env)) {
+	if (!try_shell_exec(cmd, argv)) {
 		int pid, status;
 
-		pid = mingw_spawnve(cmd, (const char **)argv, (char **)env, 0);
+		pid = mingw_spawnv(cmd, (const char **)argv, 0);
 		if (pid < 0)
 			return -1;
 		if (waitpid(pid, &status, 0) < 0)
@@ -1041,7 +1040,7 @@ int mingw_execvp(const char *cmd, char *const *argv)
 	char *prog = path_lookup(cmd, path, 0);
 
 	if (prog) {
-		mingw_execve(prog, argv, environ);
+		mingw_execv(prog, argv);
 		free(prog);
 	} else
 		errno = ENOENT;
@@ -1050,12 +1049,6 @@ int mingw_execvp(const char *cmd, char *const *argv)
 	return -1;
 }
 
-int mingw_execv(const char *cmd, char *const *argv)
-{
-	mingw_execve(cmd, argv, environ);
-	return -1;
-}
-
 int mingw_kill(pid_t pid, int sig)
 {
 	if (pid > 0 && sig == SIGTERM) {
-- 
2.0.0.9635.g0be03cb

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[PATCH 3/7] Win32: fix potential multi-threading issue

[Stepan Kasal]

From: Karsten Blees <blees@dcon.de>
Date: Fri, 7 Jan 2011 18:04:16 +0100

...by removing a static buffer in do_stat_internal.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
---
 compat/mingw.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index 1c0b153..6849815 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -441,7 +441,7 @@ static int do_lstat(int follow, const char *file_name, struct stat *buf)
 static int do_stat_internal(int follow, const char *file_name, struct stat *buf)
 {
 	int namelen;
-	static char alt_name[PATH_MAX];
+	char alt_name[PATH_MAX];
 
 	if (!do_lstat(follow, file_name, buf))
 		return 0;
-- 
2.0.0.9635.g0be03cb

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[PATCH 4/7] MinGW: disable CRT command line globbing

[Stepan Kasal]

From: Karsten Blees <blees@dcon.de>
Date: Fri, 7 Jan 2011 19:52:20 +0100

MingwRT listens to _CRT_glob to decide if __getmainargs should
perform globbing, with the default being that it should.
Unfortunately, __getmainargs globbing is sub-par; for instance
patterns like "*.c" will only match c-sources in the current
directory.

Disable __getmainargs' command line wildcard expansion, so these
patterns will be left untouched, and handled by Git's superior
built-in globbing instead.

MSVC defaults to no globbing, so we don't need to do anything
in that case.

This fixes t5505 and t7810.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
---
 compat/mingw.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/compat/mingw.c b/compat/mingw.c
index 6849815..1140a13 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -1927,6 +1927,12 @@ int xwcstoutf(char *utf, const wchar_t *wcs, size_t utflen)
 	return -1;
 }
 
+/*
+ * Disable MSVCRT command line wildcard expansion (__getmainargs called from
+ * mingw startup code, see init.c in mingw runtime).
+ */
+int _CRT_glob = 0;
+
 void mingw_startup()
 {
 	/* copy executable name to argv[0] */
-- 
2.0.0.9635.g0be03cb

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[PATCH 5/7] Win32: Unicode arguments (outgoing)

[Stepan Kasal]

From: Karsten Blees <blees@dcon.de>
Date: Sun, 16 Jan 2011 18:27:53 +0100

Convert command line arguments from UTF-8 to UTF-16 when creating other
processes.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
---
 compat/mingw.c | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index 1140a13..8a7b047 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -831,9 +831,10 @@ static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
 			      const char *dir,
 			      int prepend_cmd, int fhin, int fhout, int fherr)
 {
-	STARTUPINFO si;
+	STARTUPINFOW si;
 	PROCESS_INFORMATION pi;
 	struct strbuf envblk, args;
+	wchar_t wcmd[MAX_PATH], wdir[MAX_PATH], *wargs;
 	unsigned flags;
 	BOOL ret;
 
@@ -869,6 +870,11 @@ static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
 	si.hStdOutput = winansi_get_osfhandle(fhout);
 	si.hStdError = winansi_get_osfhandle(fherr);
 
+	if (xutftowcs_path(wcmd, cmd) < 0)
+		return -1;
+	if (dir && xutftowcs_path(wdir, dir) < 0)
+		return -1;
+
 	/* concatenate argv, quoting args as we go */
 	strbuf_init(&args, 0);
 	if (prepend_cmd) {
@@ -886,6 +892,10 @@ static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
 			free(quoted);
 	}
 
+	wargs = xmalloc((2 * args.len + 1) * sizeof(wchar_t));
+	xutftowcs(wargs, args.buf, 2 * args.len + 1);
+	strbuf_release(&args);
+
 	if (env) {
 		int count = 0;
 		char **e, **sorted_env;
@@ -907,12 +917,12 @@ static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
 	}
 
 	memset(&pi, 0, sizeof(pi));
-	ret = CreateProcess(cmd, args.buf, NULL, NULL, TRUE, flags,
-		env ? envblk.buf : NULL, dir, &si, &pi);
+	ret = CreateProcessW(wcmd, wargs, NULL, NULL, TRUE, flags,
+		env ? envblk.buf : NULL, dir ? wdir : NULL, &si, &pi);
 
 	if (env)
 		strbuf_release(&envblk);
-	strbuf_release(&args);
+	free(wargs);
 
 	if (!ret) {
 		errno = ENOENT;
-- 
2.0.0.9635.g0be03cb

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[PATCH 6/7] Win32: Unicode arguments (incoming)

[Stepan Kasal]

From: Karsten Blees <blees@dcon.de>
Date: Sun, 16 Jan 2011 18:28:27 +0100

Convert command line arguments from UTF-16 to UTF-8 on startup.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
---
 compat/mingw.c | 42 ++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 40 insertions(+), 2 deletions(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index 8a7b047..3baaa4d 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -1943,10 +1943,48 @@ int xwcstoutf(char *utf, const wchar_t *wcs, size_t utflen)
  */
 int _CRT_glob = 0;
 
+typedef struct {
+	int newmode;
+} _startupinfo;
+
+extern int __wgetmainargs(int *argc, wchar_t ***argv, wchar_t ***env, int glob,
+		_startupinfo *si);
+
+static NORETURN void die_startup()
+{
+	fputs("fatal: not enough memory for initialization", stderr);
+	exit(128);
+}
+
 void mingw_startup()
 {
-	/* copy executable name to argv[0] */
-	__argv[0] = xstrdup(_pgmptr);
+	int i, len, maxlen, argc;
+	char *buffer;
+	wchar_t **wenv, **wargv;
+	_startupinfo si;
+
+	/* get wide char arguments and environment */
+	si.newmode = 0;
+	if (__wgetmainargs(&argc, &wargv, &wenv, _CRT_glob, &si) < 0)
+		die_startup();
+
+	/* determine size of argv and environ conversion buffer */
+	maxlen = wcslen(_wpgmptr);
+	for (i = 1; i < argc; i++)
+		maxlen = max(maxlen, wcslen(wargv[i]));
+
+	/* allocate buffer (wchar_t encodes to max 3 UTF-8 bytes) */
+	maxlen = 3 * maxlen + 1;
+	buffer = xmalloc(maxlen);
+
+	/* convert command line arguments and environment to UTF-8 */
+	len = xwcstoutf(buffer, _wpgmptr, maxlen);
+	__argv[0] = xmemdupz(buffer, len);
+	for (i = 1; i < argc; i++) {
+		len = xwcstoutf(buffer, wargv[i], maxlen);
+		__argv[i] = xmemdupz(buffer, len);
+	}
+	free(buffer);
 
 	/* initialize critical section for waitpid pinfo_t list */
 	InitializeCriticalSection(&pinfo_cs);
-- 
2.0.0.9635.g0be03cb

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[PATCH 7/7] Unicode file name support (gitk and git-gui)

[Stepan Kasal]

From: Karsten Blees <blees@dcon.de>
Date: Sat, 4 Feb 2012 21:54:36 +0100

Assumes file names in git tree objects are UTF-8 encoded.

On most unix systems, the system encoding (and thus the TCL system
encoding) will be UTF-8, so file names will be displayed correctly.

On Windows, it is impossible to set the system encoding to UTF-8. Changing
the TCL system encoding (via 'encoding system ...', e.g. in the startup
code) is explicitly discouraged by the TCL docs.

Change gitk and git-gui functions dealing with file names to always convert
from and to UTF-8.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
---
 git-gui/git-gui.sh      | 11 +++++++----
 git-gui/lib/browser.tcl |  2 +-
 git-gui/lib/index.tcl   |  6 +++---
 gitk-git/gitk           | 15 ++++++++-------
 4 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/git-gui/git-gui.sh b/git-gui/git-gui.sh
index b186329..f9c942c 100755
--- a/git-gui/git-gui.sh
+++ b/git-gui/git-gui.sh
@@ -548,6 +548,9 @@ proc git {args} {
 
 	_trace_exec [concat $opt $cmdp $args]
 	set result [eval exec $opt $cmdp $args]
+	if {[encoding system] != "utf-8"} {
+		set result [encoding convertfrom utf-8 [encoding convertto $result]]
+	}
 	if {$::_trace} {
 		puts stderr "< $result"
 	}
@@ -1104,7 +1107,7 @@ git-version proc _parse_config {arr_name args} {
 				[list git_read config] \
 				$args \
 				[list --null --list]]
-			fconfigure $fd_rc -translation binary
+			fconfigure $fd_rc -translation binary -encoding utf-8
 			set buf [read $fd_rc]
 			close $fd_rc
 		}
@@ -1682,7 +1685,7 @@ proc read_diff_index {fd after} {
 		set i [split [string range $buf_rdi $c [expr {$z1 - 2}]] { }]
 		set p [string range $buf_rdi $z1 [expr {$z2 - 1}]]
 		merge_state \
-			[encoding convertfrom $p] \
+			[encoding convertfrom utf-8 $p] \
 			[lindex $i 4]? \
 			[list [lindex $i 0] [lindex $i 2]] \
 			[list]
@@ -1715,7 +1718,7 @@ proc read_diff_files {fd after} {
 		set i [split [string range $buf_rdf $c [expr {$z1 - 2}]] { }]
 		set p [string range $buf_rdf $z1 [expr {$z2 - 1}]]
 		merge_state \
-			[encoding convertfrom $p] \
+			[encoding convertfrom utf-8 $p] \
 			?[lindex $i 4] \
 			[list] \
 			[list [lindex $i 0] [lindex $i 2]]
@@ -1738,7 +1741,7 @@ proc read_ls_others {fd after} {
 	set pck [split $buf_rlo "\0"]
 	set buf_rlo [lindex $pck end]
 	foreach p [lrange $pck 0 end-1] {
-		set p [encoding convertfrom $p]
+		set p [encoding convertfrom utf-8 $p]
 		if {[string index $p end] eq {/}} {
 			set p [string range $p 0 end-1]
 		}
diff --git a/git-gui/lib/browser.tcl b/git-gui/lib/browser.tcl
index 0328338..555db89 100644
--- a/git-gui/lib/browser.tcl
+++ b/git-gui/lib/browser.tcl
@@ -197,7 +197,7 @@ method _ls {tree_id {name {}}} {
 	$w conf -state disabled
 
 	set fd [git_read ls-tree -z $tree_id]
-	fconfigure $fd -blocking 0 -translation binary -encoding binary
+	fconfigure $fd -blocking 0 -translation binary -encoding utf-8
 	fileevent $fd readable [cb _read $fd]
 }
 
diff --git a/git-gui/lib/index.tcl b/git-gui/lib/index.tcl
index 74a81a7..d10ffe9 100644
--- a/git-gui/lib/index.tcl
+++ b/git-gui/lib/index.tcl
@@ -115,7 +115,7 @@ proc write_update_indexinfo {fd pathList totalCnt batch after} {
 		set info [lindex $s 2]
 		if {$info eq {}} continue
 
-		puts -nonewline $fd "$info\t[encoding convertto $path]\0"
+		puts -nonewline $fd "$info\t[encoding convertto utf-8 $path]\0"
 		display_file $path $new
 	}
 
@@ -186,7 +186,7 @@ proc write_update_index {fd pathList totalCnt batch after} {
 		?M {set new M_}
 		?? {continue}
 		}
-		puts -nonewline $fd "[encoding convertto $path]\0"
+		puts -nonewline $fd "[encoding convertto utf-8 $path]\0"
 		display_file $path $new
 	}
 
@@ -247,7 +247,7 @@ proc write_checkout_index {fd pathList totalCnt batch after} {
 		?M -
 		?T -
 		?D {
-			puts -nonewline $fd "[encoding convertto $path]\0"
+			puts -nonewline $fd "[encoding convertto utf-8 $path]\0"
 			display_file $path ?_
 		}
 		}
diff --git a/gitk-git/gitk b/gitk-git/gitk
index f6efaa6..0301608 100755
--- a/gitk-git/gitk
+++ b/gitk-git/gitk
@@ -7529,7 +7529,7 @@ proc gettreeline {gtf id} {
 	if {[string index $fname 0] eq "\""} {
 	    set fname [lindex $fname 0]
 	}
-	set fname [encoding convertfrom $fname]
+	set fname [encoding convertfrom utf-8 $fname]
 	lappend treefilelist($id) $fname
     }
     if {![eof $gtf]} {
@@ -7791,7 +7791,7 @@ proc gettreediffline {gdtf ids} {
 	    if {[string index $file 0] eq "\""} {
 		set file [lindex $file 0]
 	    }
-	    set file [encoding convertfrom $file]
+	    set file [encoding convertfrom utf-8 $file]
 	    if {$file ne [lindex $treediff end]} {
 		lappend treediff $file
 		lappend sublist $file
@@ -7936,7 +7936,7 @@ proc makediffhdr {fname ids} {
     global ctext curdiffstart treediffs diffencoding
     global ctext_file_names jump_to_here targetline diffline
 
-    set fname [encoding convertfrom $fname]
+    set fname [encoding convertfrom utf-8 $fname]
     set diffencoding [get_path_encoding $fname]
     set i [lsearch -exact $treediffs($ids) $fname]
     if {$i >= 0} {
@@ -7993,7 +7993,7 @@ proc parseblobdiffline {ids line} {
 
     if {![string compare -length 5 "diff " $line]} {
 	if {![regexp {^diff (--cc|--git) } $line m type]} {
-	    set line [encoding convertfrom $line]
+	    set line [encoding convertfrom utf-8 $line]
 	    $ctext insert end "$line\n" hunksep
 	    continue
 	}
@@ -8040,7 +8040,7 @@ proc parseblobdiffline {ids line} {
 	makediffhdr $fname $ids
 
     } elseif {![string compare -length 16 "* Unmerged path " $line]} {
-	set fname [encoding convertfrom [string range $line 16 end]]
+	set fname [encoding convertfrom utf-8 [string range $line 16 end]]
 	$ctext insert end "\n"
 	set curdiffstart [$ctext index "end - 1c"]
 	lappend ctext_file_names $fname
@@ -8095,7 +8095,7 @@ proc parseblobdiffline {ids line} {
 	    if {[string index $fname 0] eq "\""} {
 		set fname [lindex $fname 0]
 	    }
-	    set fname [encoding convertfrom $fname]
+	    set fname [encoding convertfrom utf-8 $fname]
 	    set i [lsearch -exact $treediffs($ids) $fname]
 	    if {$i >= 0} {
 		setinlist difffilestart $i $curdiffstart
@@ -8114,6 +8114,7 @@ proc parseblobdiffline {ids line} {
 	    set diffinhdr 0
 	    return
 	}
+	set line [encoding convertfrom utf-8 $line]
 	$ctext insert end "$line\n" filesep
 
     } else {
@@ -11902,7 +11903,7 @@ proc cache_gitattr {attr pathlist} {
 	    foreach row [split $rlist "\n"] {
 		if {[regexp "(.*): $attr: (.*)" $row m path value]} {
 		    if {[string index $path 0] eq "\""} {
-			set path [encoding convertfrom [lindex $path 0]]
+			set path [encoding convertfrom utf-8 [lindex $path 0]]
 		    }
 		    set path_attr_cache($attr,$path) $value
 		}
-- 
2.0.0.9635.g0be03cb

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [PATCH 0/7] Second part of msysgit/unicode

[Karsten Blees]

Am 11.06.2014 11:37, schrieb Stepan Kasal:
> This is the second part of the time-proven unicode suport branch from msysgit.
> This batch is a collection of small independent changes, limited to mingw.c.
> The only exception is the last patch: it changes gitk and git-gui.
> 

I'm missing the other two "Unicode file name" patches (and "Win32: fix detection
of empty directories in is_dir_empty", probably squashed). If gitk and git-gui
expect file names to be UTF-8, git.exe should do so as well, don't you think?

Otherwise OK (and AFAICT, there's only environment stuff left).

>   Win32: Unicode arguments (incoming)

Note: This depends on "MSVC: link dynamically to the CRT", which was sent
separately (and is already in master - good).

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [PATCH 0/7] Second part of msysgit/unicode

[Stepan Kasal]

Hello Karsten,

On Tue, Jun 17, 2014 at 11:06:52AM +0200, Karsten Blees wrote:
> Am 11.06.2014 11:37, schrieb Stepan Kasal:
> > This is the second part of the time-proven unicode suport branch from msysgit.
> > This batch is a collection of small independent changes, limited to mingw.c.
> > The only exception is the last patch: it changes gitk and git-gui.
> 
> I'm missing the other two "Unicode file name" patches (and "Win32: fix detection

indeed.  I noticed that after sending the plan quoted above.
Luckily, the gitk/git-gui patch was not accepted and has to be
resubmitted.

So the plan for future submissions is:

1) two "Unicode file name" patches (with "fix... is_dir_empty"
squashed)
2) the environament patches from your unicode branch (several
patches)
3) "color term" (and env. var. TERM); updated according to your
instructions, thus sent separately after the series
4) resubmit gitk / git-gui (have separate maintainers)

This is work in progress, I suppose to mail 1) and 2) in a few days.

Stepan

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [PATCH 0/7] Second part of msysgit/unicode

[Junio C Hamano]

Stepan Kasal <kasal@ucw.cz> writes:

> Hello Karsten,
>
> On Tue, Jun 17, 2014 at 11:06:52AM +0200, Karsten Blees wrote:
>> Am 11.06.2014 11:37, schrieb Stepan Kasal:
>> > This is the second part of the time-proven unicode suport branch from msysgit.
>> > This batch is a collection of small independent changes, limited to mingw.c.
>> > The only exception is the last patch: it changes gitk and git-gui.
>> 
>> I'm missing the other two "Unicode file name" patches (and "Win32: fix detection
>
> indeed.  I noticed that after sending the plan quoted above.
> Luckily, the gitk/git-gui patch was not accepted and has to be
> resubmitted.
>
> So the plan for future submissions is:
>
> 1) two "Unicode file name" patches (with "fix... is_dir_empty"
> squashed)
> 2) the environament patches from your unicode branch (several
> patches)
> 3) "color term" (and env. var. TERM); updated according to your
> instructions, thus sent separately after the series
> 4) resubmit gitk / git-gui (have separate maintainers)
>
> This is work in progress, I suppose to mail 1) and 2) in a few days.
>
> Stepan

In the meantime, are Windows folks happy with the four topics queued
on 'pu' so far?  I would like to start moving them down to 'next'
and to 'master' soonish.

They consist of these individual patches:

    $ git shortlog ^master \
      sk/mingw-dirent \
      sk/mingw-main \
      sk/mingw-uni-console \
      sk/mingw-unicode-spawn-args
    Johannes Schindelin (1):
          Win32: let mingw_execve() return an int

    Karsten Blees (18):
          Win32 dirent: remove unused dirent.d_ino member
          Win32 dirent: remove unused dirent.d_reclen member
          Win32 dirent: change FILENAME_MAX to MAX_PATH
          Win32 dirent: clarify #include directives
          Win32 dirent: improve dirent implementation
          Win32: move main macro to a function
          Win32: support Unicode console output
          Win32: detect console streams more reliably
          Win32: warn if the console font doesn't support Unicode
          Win32: add Unicode conversion functions
          Win32: Thread-safe windows console output
          Win32: fix broken pipe detection
          Win32: reliably detect console pipe handles
          Win32: simplify internal mingw_spawn* APIs
          Win32: fix potential multi-threading issue
          MinGW: disable CRT command line globbing
          Win32: Unicode arguments (outgoing)
          Win32: Unicode arguments (incoming)

    Stepan Kasal (1):
          mingw: avoid const warning

Thanks.

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Re: [PATCH 0/7] Second part of msysgit/unicode

[Johannes Sixt]

Am 18.06.2014 19:33, schrieb Junio C Hamano:
> In the meantime, are Windows folks happy with the four topics queued
> on 'pu' so far?  I would like to start moving them down to 'next'
> and to 'master' soonish.
> 
> They consist of these individual patches:
> 
>     $ git shortlog ^master \
>       sk/mingw-dirent \
>       sk/mingw-main \
>       sk/mingw-uni-console \
>       sk/mingw-unicode-spawn-args

Topic sk/test-cmp-bin revealed a new breakage in t5000-tar-tree,
specifically, the penultimate test "remote tar.gz is allowed by
default". I have yet to find out what it is (I suspect a LF-CRLF
conversion issue) and whether it is in connection with one of these topics.

I haven't had a chance to test the topics in the field. In particular, I
have a few files with Shift-JIS content (but ASCII file names), and I
would like to see how well I fare with the unicode topics in this situation.

-- Hannes

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.