gettext CTYPE for libc

gettext CTYPE for libc

[Trần Ngọc Quân]

Hello,

$ mkdir xyz
$ cd xyz
$ rmdir ../xyz
$ git status
fatal: Unable to read current working directory: Kh?ng c? t?p tin ho?c
th? m?c nh? v?y

So, somthing wrong with our charset.

$ strace git status 2>&1 | grep open
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
open("/lib/i386-linux-gnu/libz.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/lib/i386-linux-gnu/libcrypto.so.1.0.0", O_RDONLY|O_CLOEXEC) = 3
open("/lib/i386-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
open("/lib/i386-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
open("/lib/i386-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
open("/dev/null", O_RDWR|O_LARGEFILE)   = 3
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
open("/usr/share/locale/vi_VN/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/vi/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/usr/share/locale-langpack/vi_VN/LC_MESSAGES/libc.mo", O_RDONLY) =
-1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/vi/LC_MESSAGES/libc.mo", O_RDONLY) = 3
open("/usr/lib/i386-linux-gnu/gconv/gconv-modules.cache", O_RDONLY) = 3

We will see, this string come from libc.mo
$ gettext --domain=libc "No such file or directory"
Không có tập tin hoặc thư mục như vậy

in git's gettext.c, it not allow CTYPE="" for all domain, so we will set
this one individually. In this ex. I set it for libc:

$ git diff
diff --git a/gettext.c b/gettext.c
index 71e9545..abd3978 100644
--- a/gettext.c
+++ b/gettext.c
@@ -115,6 +115,7 @@ static void init_gettext_charset(const char *domain)
        setlocale(LC_CTYPE, "");
        charset = locale_charset();
        bind_textdomain_codeset(domain, charset);
+       bind_textdomain_codeset("libc", charset);
        setlocale(LC_CTYPE, "C");
 }

And it work as I expect!

-- 
Trần Ngọc Quân.

Re: gettext CTYPE for libc

[Thomas Rast]

Trần Ngọc Quân <vnwildman@gmail.com> writes:

> $ git status
> fatal: Unable to read current working directory: Kh?ng c? t?p tin ho?c
> th? m?c nh? v?y
>
> So, somthing wrong with our charset.
[...]
> $ gettext --domain=libc "No such file or directory"
> Không có tập tin hoặc thư mục như vậy
>
> in git's gettext.c, it not allow CTYPE="" for all domain, so we will set
> this one individually. In this ex. I set it for libc:
>
> $ git diff
> diff --git a/gettext.c b/gettext.c
> index 71e9545..abd3978 100644
> --- a/gettext.c
> +++ b/gettext.c
> @@ -115,6 +115,7 @@ static void init_gettext_charset(const char *domain)
>         setlocale(LC_CTYPE, "");
>         charset = locale_charset();
>         bind_textdomain_codeset(domain, charset);
> +       bind_textdomain_codeset("libc", charset);
>         setlocale(LC_CTYPE, "C");
>  }

Do you know why this "suddenly" broke?  The long comment in
init_gettext_charset() suggests that the *existing* code is there to
handle exactly this problem, and apparently it doesn't.  Why?  Has libc
moved the perror() strings into a separate domain in some version?

-- 
Thomas Rast
tr@thomasrast.ch

Re: gettext CTYPE for libc

[Trần Ngọc Quân]

On 24/11/2013 16:05, Thomas Rast wrote:
> Trần Ngọc Quân <vnwildman@gmail.com> writes:
>
>> $ git status
>> fatal: Unable to read current working directory: Kh?ng c? t?p tin ho?c
>> th? m?c nh? v?y
>>
>> So, somthing wrong with our charset.
> [...]
> Do you know why this "suddenly" broke?
I think git set CTYPE="C" for libc, so charset become  7-bit ASCII, but
it don't set LC_MESSAGES="C" for libc and libc will get this one from
system variable.
> The long comment in
> init_gettext_charset() suggests that the *existing* code is there to
> handle exactly this problem, and apparently it doesn't.  Why?  Has libc
> moved the perror() strings into a separate domain in some version?
See setlocale(3) [1]
I'm a newbie in GIT. I'm not sure about git work correctly [2] if set
git's charset to same with system.
I don't think libc moved perror() string in separate domain. It use its
own domain.

[1]  http://man7.org/linux/man-pages/man3/setlocale.3.html
[2]  incorrect if some function need work in ASCII mode

-- 
Trần Ngọc Quân.

Re: gettext CTYPE for libc

[Duy Nguyen]

Jonathan I see you participated in bug 6530, mentioned in the big
comment block in init_gettext_charset(). The bug seems fixed since
glibc 2.17. Do you think it's ok to revert the workaround if we detect
the running glibc is fixed (or if it does not run glibc at all)? I
think we could use gnu_get_libc_version() to detect it.

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=6530

On Sat, Nov 23, 2013 at 9:15 AM, Trần Ngọc Quân <vnwildman@gmail.com> wrote:
> Hello,
>
> $ mkdir xyz
> $ cd xyz
> $ rmdir ../xyz
> $ git status
> fatal: Unable to read current working directory: Kh?ng c? t?p tin ho?c
> th? m?c nh? v?y
>
> So, somthing wrong with our charset.
>
> $ strace git status 2>&1 | grep open
> open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
> open("/lib/i386-linux-gnu/libz.so.1", O_RDONLY|O_CLOEXEC) = 3
> open("/lib/i386-linux-gnu/libcrypto.so.1.0.0", O_RDONLY|O_CLOEXEC) = 3
> open("/lib/i386-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
> open("/lib/i386-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
> open("/lib/i386-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
> open("/dev/null", O_RDWR|O_LARGEFILE)   = 3
> open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
> open("/usr/share/locale/vi_VN/LC_MESSAGES/libc.mo", O_RDONLY) = -1
> ENOENT (No such file or directory)
> open("/usr/share/locale/vi/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
> (No such file or directory)
> open("/usr/share/locale-langpack/vi_VN/LC_MESSAGES/libc.mo", O_RDONLY) =
> -1 ENOENT (No such file or directory)
> open("/usr/share/locale-langpack/vi/LC_MESSAGES/libc.mo", O_RDONLY) = 3
> open("/usr/lib/i386-linux-gnu/gconv/gconv-modules.cache", O_RDONLY) = 3
>
> We will see, this string come from libc.mo
> $ gettext --domain=libc "No such file or directory"
> Không có tập tin hoặc thư mục như vậy
>
> in git's gettext.c, it not allow CTYPE="" for all domain, so we will set
> this one individually. In this ex. I set it for libc:
>
> $ git diff
> diff --git a/gettext.c b/gettext.c
> index 71e9545..abd3978 100644
> --- a/gettext.c
> +++ b/gettext.c
> @@ -115,6 +115,7 @@ static void init_gettext_charset(const char *domain)
>         setlocale(LC_CTYPE, "");
>         charset = locale_charset();
>         bind_textdomain_codeset(domain, charset);
> +       bind_textdomain_codeset("libc", charset);
>         setlocale(LC_CTYPE, "C");
>  }
>
> And it work as I expect!
>
> --
> Trần Ngọc Quân.
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Duy

Re: gettext CTYPE for libc

[Jonathan Nieder]

Duy Nguyen wrote:

>             Do you think it's ok to revert the workaround if we detect
> the running glibc is fixed (or if it does not run glibc at all)? I
> think we could use gnu_get_libc_version() to detect it.

That would be wonderful.

Thanks,
Jonathan