BUG: "git add --interactive" does not work with UTF-8 filenames

BUG: "git add --interactive" does not work with UTF-8 filenames

[Teemu Likonen]

I just learnt that "git add --interactive" does not work with UTF-8 
encoded filenames. Maybe it's about all 8-bit characters.

The menu invoked with command

  git add -i <file>

does not even show this <file> if the name contains UTF-8 characters. 
Command and menu

  git add -i

shows the file but nothing can be done with it. Neither "patch" 
nor "update" does work, for example.


As a side note/rant: Git tends to show non-Ascii characters in filenames 
as \xxx escape sequences. It's not a big problem but surely it's not 
very nice to read if (say) half of the word's letters are escaped like 
this. I believe Git is mostly used for tracking source codes where 
Ascii-only filenames are a common practice. I'm using Git to develop 
Finnish spell-checking system (called Voikko) and we have some 
non-Ascii filenames in the repo.

Re: BUG: "git add --interactive" does not work with UTF-8 filenames

[Jakub Narebski]

Teemu Likonen <tlikonen@iki.fi> writes:

> As a side note/rant: Git tends to show non-Ascii characters in filenames 
> as \xxx escape sequences. It's not a big problem but surely it's not 
> very nice to read if (say) half of the word's letters are escaped like 
> this. I believe Git is mostly used for tracking source codes where 
> Ascii-only filenames are a common practice. I'm using Git to develop 
> Finnish spell-checking system (called Voikko) and we have some 
> non-Ascii filenames in the repo.

git-config(1):

  core.quotepath::
	The commands that output paths (e.g. `ls-files`,
	`diff`), when not given the `-z` option, will quote
	"unusual" characters in the pathname by enclosing the
	pathname in a double-quote pair and with backslashes the
	same way strings in C source code are quoted.  If this
	variable is set to false, the bytes higher than 0x80 are
	not quoted but output as verbatim.  Note that double
	quote, backslash and control characters are always
	quoted without `-z` regardless of the setting of this
	variable.

Since 1.5.2.2 (I think), or at least 1.5.3, so you have new enough
version of git (git is now at 1.5.5-rc2).

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: BUG: "git add --interactive" does not work with UTF-8 filenames if core.quotepath=true

[Teemu Likonen]

Jakub Narebski kirjoitti:

> git-config(1):
>
>   core.quotepath::

Ah, thanks! This whole bug disappeared when I set core.quotepath=false 
i.e. "git add -i" works fine with UTF-8 filenames too.

However, the bug remains when core.quotepath=true or it's unconfigured.

Re: BUG: "git add --interactive" does not work with UTF-8 filenames

[Junio C Hamano]

Jakub Narebski <jnareb@gmail.com> writes:

> git-config(1):
>
>   core.quotepath::
> 	The commands that output paths (e.g. `ls-files`,
> 	`diff`), when not given the `-z` option, will quote
> 	"unusual" characters in the pathname by enclosing the
> 	pathname in a double-quote pair and with backslashes the
> 	same way strings in C source code are quoted.  If this
> 	variable is set to false, the bytes higher than 0x80 are
> 	not quoted but output as verbatim.  Note that double
> 	quote, backslash and control characters are always
> 	quoted without `-z` regardless of the setting of this
> 	variable.
>
> Since 1.5.2.2 (I think), or at least 1.5.3, so you have new enough
> version of git (git is now at 1.5.5-rc2).

Yes, but I do not think "add -i" unwraps the path quoting.  I think it
should, but I do not think I bothered to.  Because it is trivial to do by
any aspiring git hacker wannabes, I left it as an exercise to readers when
I did the "interactive" as a quick-and-dirty hack (aka "demonstration").

Somehow nobody found an itch to improve it until now, but it seems that we
found a volunteer with the itch ;-)

Re: BUG: "git add --interactive" does not work with UTF-8 filenames

[Jakub Narebski]

Junio C Hamano wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
> 
> > git-config(1):
> >
> >   core.quotepath::
> > 	The commands that output paths (e.g. `ls-files`,
> > 	`diff`), when not given the `-z` option, will quote
> > 	"unusual" characters in the pathname by enclosing the
> > 	pathname in a double-quote pair and with backslashes the
> > 	same way strings in C source code are quoted.  If this
> > 	variable is set to false, the bytes higher than 0x80 are
> > 	not quoted but output as verbatim.  Note that double
> > 	quote, backslash and control characters are always
> > 	quoted without `-z` regardless of the setting of this
> > 	variable.
> >
> > Since 1.5.2.2 (I think), or at least 1.5.3, so you have new enough
> > version of git (git is now at 1.5.5-rc2).
> 
> Yes, but I do not think "add -i" unwraps the path quoting.  I think it
> should, but I do not think I bothered to.  Because it is trivial to do by
> any aspiring git hacker wannabes, I left it as an exercise to readers when
> I did the "interactive" as a quick-and-dirty hack (aka "demonstration").

By the way, the code to unwrap the path quoting can be found in gitweb
as unescape() subroutine... or git-add--interactive can use '-z' switch.

-- 
Jakub Narebski
Poland

Re: BUG: "git add --interactive" does not work with UTF-8 filenames

[Junio C Hamano]

Jakub Narebski <jnareb@gmail.com> writes:

> By the way, the code to unwrap the path quoting can be found in gitweb
> as unescape() subroutine... or git-add--interactive can use '-z' switch.

Yes, but if you use '-z' to read from plumbing, you would need your own
wrapping on the UI side to protect your output from control characters
(most notably LF) embedded in pathnames.

Also I had an impression that the compatibility implementation of
run_cmd_pipe() had some issues with NUL terminated list.

[PATCH] Describe the bug about handling non-Ascii filenames in 'git add -i'

[Teemu Likonen]

The command 'git add -i' (i.e. the interactive mode) can't handle
files which have non-Ascii characters in their name unless the option
core.quotepath is set to false. This patch adds a description of the bug
to the git-add.txt manual.
---

Junio C Hamano kirjoitti (1.4.2008 klo 17.41):

> Jakub Narebski <jnareb@gmail.com> writes:
> 
> > git-config(1):
> >
> >   core.quotepath::
> > 	The commands that output paths (e.g. `ls-files`,
> > 	`diff`), when not given the `-z` option, will quote
> > 	"unusual" characters in the pathname by enclosing the
> > 	pathname in a double-quote pair and with backslashes the
> > 	same way strings in C source code are quoted.  If this
> > 	variable is set to false, the bytes higher than 0x80 are
> > 	not quoted but output as verbatim.  Note that double
> > 	quote, backslash and control characters are always
> > 	quoted without `-z` regardless of the setting of this
> > 	variable.
> 
> Yes, but I do not think "add -i" unwraps the path quoting.  I think it
> should, but I do not think I bothered to.  Because it is trivial to do
> by any aspiring git hacker wannabes, I left it as an exercise to
> readers when I did the "interactive" as a quick-and-dirty hack (aka
> "demonstration").
> 
> Somehow nobody found an itch to improve it until now, but it seems
> that we found a volunteer with the itch ;-)

Unfortunately we found a non-programmer :) My itch (both of them)
disappeared pretty much at the moment I set core.quotepath to false
(thanks Jakub). But this patch is the kind of contribution I can do at
the present moment.


 Documentation/git-add.txt |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/Documentation/git-add.txt b/Documentation/git-add.txt
index c751a17..f085b15 100644
--- a/Documentation/git-add.txt
+++ b/Documentation/git-add.txt
@@ -226,6 +226,13 @@ diff::
   This lets you review what will be committed (i.e. between
   HEAD and index).
 
+Bugs
+----
+Files with non-Ascii characters in their name don't work in the interactive
+mode (see above) unless option 'core.quotepath' is set to false. Setting
+'core.quotepath' to false may be a good idea anyway if you need to handle
+non-Ascii filenames. See linkgit:git-config[1] for more information about this
+option.
 
 See Also
 --------
-- 
1.5.5.rc2.15.g4db6

Re: [PATCH] Describe the bug about handling non-Ascii filenames in 'git add -i'

[Junio C Hamano]

This is not limited to non-ascii.  You would also need to warn about HT,
LF and double-quotes in the filenames.

I'd call this a limitation, though.