Problem with i18n (input)

Problem with i18n (input)

[Otavio Salvador]

Hello,

I've been in trouble to get proper support for input in OE. We're
having problem to support input of international charsets on X. We've
include locale packages and we have the system properly translated,
however the input of latin1 chars (e.g. éçóã...) doesn't work.

Here goes the list of packages (locale related) that are installed on
the testing image, for reference:

glibc-binary-localedata-pt-br_2.6.1-r6_i586.ipk
glibc-charmap-iso-8859-1_2.6.1-r6_i586.ipk
glibc-gconv_2.6.1-r6_i586.ipk
glibc-gconv-iso8859-1_2.6.1-r6_i586.ipk
glibc-localedata-i18n_2.6.1-r6_i586.ipk
glibc-localedata-iso14651-t1_2.6.1-r6_i586.ipk
glibc-localedata-iso14651-t1-common_2.6.1-r6_i586.ipk
glibc-localedata-translit-circle_2.6.1-r6_i586.ipk
glibc-localedata-translit-cjk-compat_2.6.1-r6_i586.ipk
glibc-localedata-translit-compat_2.6.1-r6_i586.ipk
glibc-localedata-translit-font_2.6.1-r6_i586.ipk
glibc-localedata-translit-fraction_2.6.1-r6_i586.ipk
glibc-localedata-translit-narrow_2.6.1-r6_i586.ipk
glibc-localedata-translit-neutral_2.6.1-r6_i586.ipk
glibc-localedata-translit-small_2.6.1-r6_i586.ipk
glibc-localedata-translit-wide_2.6.1-r6_i586.ipk
libx11-locale_1:1.1.3-r3_i586.ipk
locale-base-pt-br_2.6.1-r6_i586.ipk
thunar-locale-pt-br_0.9.0-r0_i586.ipk
xfce-terminal-locale-pt-br_0.2.8-r0_i586.ipk
xfce4-panel-locale-pt-br_4.4.2-r0_i586.ipk
xfce4-session-locale-pt-br_4.4.2-r0_i586.ipk
xfdesktop-locale-pt-br_4.4.2-r0_i586.ipk
xfwm4-locale-pt-br_4.4.2-r0_i586.ipk

Any idea?

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Problem with i18n (input)

[Phil Blundell]

On Mon, 2008-06-30 at 15:36 -0300, Otavio Salvador wrote:
> I've been in trouble to get proper support for input in OE. We're
> having problem to support input of international charsets on X. We've
> include locale packages and we have the system properly translated,
> however the input of latin1 chars (e.g. éçóã...) doesn't work.

Can you be any more specific about what exactly goes wrong?  Are you
basically saying that the compose key doesn't work, or are you talking
about something different?  If compose, does it not work at all (i.e.
just enters "e'" as separate keystrokes, as if you hadn't pressed
compose in the first place) or does it swallow the two keystrokes and
then give you nothing, or garbage, out?  If the former, have you
verified (e.g. with xev) that the compose key itself is correctly mapped
and generating events with the right keysym?

Also, what applications are you trying to input these characters into?
This is an area that is not terribly well standardised in X and
different toolkits have different behaviour.

Also also, what build of libx11 do you have?  Is it a "diet" version or
the full thing?

p.

Re: Problem with i18n (input)

[Otavio Salvador]

Phil Blundell <pb@reciva.com> writes:

> On Mon, 2008-06-30 at 15:36 -0300, Otavio Salvador wrote:
>> I've been in trouble to get proper support for input in OE. We're
>> having problem to support input of international charsets on X. We've
>> include locale packages and we have the system properly translated,
>> however the input of latin1 chars (e.g. éçóã...) doesn't work.
>
> Can you be any more specific about what exactly goes wrong?  Are you
> basically saying that the compose key doesn't work, or are you talking
> about something different?  If compose, does it not work at all (i.e.
> just enters "e'" as separate keystrokes, as if you hadn't pressed
> compose in the first place) or does it swallow the two keystrokes and
> then give you nothing, or garbage, out?  If the former, have you
> verified (e.g. with xev) that the compose key itself is correctly mapped
> and generating events with the right keysym?

When I press "'e" it gives me nothing. When I run it inside xev it
gives me:

KeyPress event, serial 30, synthetic NO, window 0x1400001,
    root 0x3a, subw 0x0, time 3675299458, (149,171), root:(572,473),
    state 0x0, keycode 34 (keysym 0xfe51, dead_acute), same_screen YES,
    XLookupString gives 1 bytes: (b4) "�"
    XFilterEvent returns: False

KeyRelease event, serial 30, synthetic NO, window 0x1400001,
    root 0x3a, subw 0x0, time 3675299519, (149,171), root:(572,473),
    state 0x0, keycode 34 (keysym 0xfe51, dead_acute), same_screen YES,
    XLookupString gives 1 bytes: (b4) "�"
    XFilterEvent returns: False

KeyPress event, serial 30, synthetic NO, window 0x1400001,
    root 0x3a, subw 0x0, time 3675302404, (149,171), root:(572,473),
    state 0x0, keycode 26 (keysym 0x65, e), same_screen YES,
    XLookupString gives 1 bytes: (65) "e"
    XFilterEvent returns: False

KeyRelease event, serial 30, synthetic NO, window 0x1400001,
    root 0x3a, subw 0x0, time 3675302404, (149,171), root:(572,473),
    state 0x0, keycode 26 (keysym 0x65, e), same_screen YES,
    XLookupString gives 1 bytes: (65) "e"
    XFilterEvent returns: False

> Also, what applications are you trying to input these characters into?
> This is an area that is not terribly well standardised in X and
> different toolkits have different behaviour.

We used xfce4-terminal for testing. It uses GTK and works fine in my
Debian system but fails in OE.

> Also also, what build of libx11 do you have?  Is it a "diet" version or
> the full thing?

I'm using the full thing:

libx11-6_1:1.1.3-r3_i586.ipk
libx11-locale_1:1.1.3-r3_i586.ipk

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Problem with i18n (input)

[Phil Blundell]

On Mon, 2008-06-30 at 17:02 -0300, Otavio Salvador wrote:
> When I press "'e" it gives me nothing. When I run it inside xev it
> gives me:
> 
> KeyPress event, serial 30, synthetic NO, window 0x1400001,
>     root 0x3a, subw 0x0, time 3675299458, (149,171), root:(572,473),
>     state 0x0, keycode 34 (keysym 0xfe51, dead_acute), same_screen YES,
>     XLookupString gives 1 bytes: (b4) "�"
>     XFilterEvent returns: False
> 
> KeyRelease event, serial 30, synthetic NO, window 0x1400001,
>     root 0x3a, subw 0x0, time 3675299519, (149,171), root:(572,473),
>     state 0x0, keycode 34 (keysym 0xfe51, dead_acute), same_screen YES,
>     XLookupString gives 1 bytes: (b4) "�"
>     XFilterEvent returns: False
> 
> KeyPress event, serial 30, synthetic NO, window 0x1400001,
>     root 0x3a, subw 0x0, time 3675302404, (149,171), root:(572,473),
>     state 0x0, keycode 26 (keysym 0x65, e), same_screen YES,
>     XLookupString gives 1 bytes: (65) "e"
>     XFilterEvent returns: False
> 
> KeyRelease event, serial 30, synthetic NO, window 0x1400001,
>     root 0x3a, subw 0x0, time 3675302404, (149,171), root:(572,473),
>     state 0x0, keycode 26 (keysym 0x65, e), same_screen YES,
>     XLookupString gives 1 bytes: (65) "e"
>     XFilterEvent returns: False

This output means that XIM isn't enabled and hence you aren't getting
any compose handling at the Xlib level.  That shouldn't matter in your
case, though, because GTK does its own compose handling and XIM is
basically just dead weight.

If you see nothing at all when you type "'e" then that would suggest
that GTK is correctly interpreting the dead key and generating the right
keycode internally, but that something is then going wrong after that.
(If it didn't understand the dead acute at all then you would see a
plain unadorned "e".)

Unfortunately I don't think there is any convenient xev equivalent that
will show you gtk's inner workings, so you might have to resort to the
debugger to figure out what's going on here.  I do have a vague
recollection from when I last worked on this stuff that gtk was quite
sensitive to locale settings.  Presumably locale-base-pt-br is a UTF8
locale, right?  Do you have "LANG=pt_BR" set in your environment?

p.

Re: Problem with i18n (input)

[Otavio Salvador]

Phil Blundell <pb@reciva.com> writes:

> On Mon, 2008-06-30 at 17:02 -0300, Otavio Salvador wrote:
>> When I press "'e" it gives me nothing. When I run it inside xev it
>> gives me:
>> 
>> KeyPress event, serial 30, synthetic NO, window 0x1400001,
>>     root 0x3a, subw 0x0, time 3675299458, (149,171), root:(572,473),
>>     state 0x0, keycode 34 (keysym 0xfe51, dead_acute), same_screen YES,
>>     XLookupString gives 1 bytes: (b4) "�"
>>     XFilterEvent returns: False
>> 
>> KeyRelease event, serial 30, synthetic NO, window 0x1400001,
>>     root 0x3a, subw 0x0, time 3675299519, (149,171), root:(572,473),
>>     state 0x0, keycode 34 (keysym 0xfe51, dead_acute), same_screen YES,
>>     XLookupString gives 1 bytes: (b4) "�"
>>     XFilterEvent returns: False
>> 
>> KeyPress event, serial 30, synthetic NO, window 0x1400001,
>>     root 0x3a, subw 0x0, time 3675302404, (149,171), root:(572,473),
>>     state 0x0, keycode 26 (keysym 0x65, e), same_screen YES,
>>     XLookupString gives 1 bytes: (65) "e"
>>     XFilterEvent returns: False
>> 
>> KeyRelease event, serial 30, synthetic NO, window 0x1400001,
>>     root 0x3a, subw 0x0, time 3675302404, (149,171), root:(572,473),
>>     state 0x0, keycode 26 (keysym 0x65, e), same_screen YES,
>>     XLookupString gives 1 bytes: (65) "e"
>>     XFilterEvent returns: False
>
> This output means that XIM isn't enabled and hence you aren't getting
> any compose handling at the Xlib level.  That shouldn't matter in your
> case, though, because GTK does its own compose handling and XIM is
> basically just dead weight.

Any way for me to enable it?

> If you see nothing at all when you type "'e" then that would suggest
> that GTK is correctly interpreting the dead key and generating the right
> keycode internally, but that something is then going wrong after that.
> (If it didn't understand the dead acute at all then you would see a
> plain unadorned "e".)

Humm, confusing ...

> Unfortunately I don't think there is any convenient xev equivalent that
> will show you gtk's inner workings, so you might have to resort to the
> debugger to figure out what's going on here.  I do have a vague
> recollection from when I last worked on this stuff that gtk was quite
> sensitive to locale settings.  Presumably locale-base-pt-br is a UTF8
> locale, right?  Do you have "LANG=pt_BR" set in your environment?

Yep .. I've used: LC_ALL=pt_BR.

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Problem with i18n (input)

[Phil Blundell]

On Mon, 2008-06-30 at 17:54 -0300, Otavio Salvador wrote:
> Any way for me to enable it?

What do you have listed for pt_BR in /usr/share/X11/locale/compose.dir?
It should contain a reference to one of
the /usr/share/X11/locale/XXX/Compose files; check that this is in place
and readable.  Might be worth using "strace xev" to see if X is making
any effort to open the right files.

p.

Re: Problem with i18n (input)

[Otavio Salvador]

Phil Blundell <pb@reciva.com> writes:

> On Mon, 2008-06-30 at 17:54 -0300, Otavio Salvador wrote:
>> Any way for me to enable it?
>
> What do you have listed for pt_BR in /usr/share/X11/locale/compose.dir?
> It should contain a reference to one of
> the /usr/share/X11/locale/XXX/Compose files; check that this is in place
> and readable.  Might be worth using "strace xev" to see if X is making
> any effort to open the right files.

I have pt_BR.UTF-8.

I've tryed it on both, a regular Debian system and it does load the
Compose file. At OE image, it does not.

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Problem with i18n (input)

[Phil Blundell]

On Mon, 2008-06-30 at 18:42 -0300, Otavio Salvador wrote:
> I've tryed it on both, a regular Debian system and it does load the
> Compose file. At OE image, it does not.

Is it even loading compose.dir on your OE image?

p.

Re: Problem with i18n (input)

[Otavio Salvador]

Phil Blundell <pb@reciva.com> writes:

> On Mon, 2008-06-30 at 18:42 -0300, Otavio Salvador wrote:
>> I've tryed it on both, a regular Debian system and it does load the
>> Compose file. At OE image, it does not.
>
> Is it even loading compose.dir on your OE image?

It's not.

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Problem with i18n (input)

[Phil Blundell]

On Tue, 2008-07-01 at 15:19 -0300, Otavio Salvador wrote:
> Phil Blundell <pb@reciva.com> writes:
> 
> > On Mon, 2008-06-30 at 18:42 -0300, Otavio Salvador wrote:
> >> I've tryed it on both, a regular Debian system and it does load the
> >> Compose file. At OE image, it does not.
> >
> > Is it even loading compose.dir on your OE image?
> 
> It's not.

Hum.  How about locale.dir, does it load that one?

p.

Re: Problem with i18n (input)

[Otavio Salvador]

Phil Blundell <pb@reciva.com> writes:

> On Tue, 2008-07-01 at 15:19 -0300, Otavio Salvador wrote:
>> Phil Blundell <pb@reciva.com> writes:
>> 
>> > On Mon, 2008-06-30 at 18:42 -0300, Otavio Salvador wrote:
>> >> I've tryed it on both, a regular Debian system and it does load the
>> >> Compose file. At OE image, it does not.
>> >
>> > Is it even loading compose.dir on your OE image?
>> 
>> It's not.
>
> Hum.  How about locale.dir, does it load that one?

It does.

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Problem with i18n (input)

[Otavio Salvador]

Otavio Salvador <otavio@debian.org> writes:

> Phil Blundell <pb@reciva.com> writes:
>
>> On Tue, 2008-07-01 at 15:19 -0300, Otavio Salvador wrote:
>>> Phil Blundell <pb@reciva.com> writes:
>>> 
>>> > On Mon, 2008-06-30 at 18:42 -0300, Otavio Salvador wrote:
>>> >> I've tryed it on both, a regular Debian system and it does load the
>>> >> Compose file. At OE image, it does not.
>>> >
>>> > Is it even loading compose.dir on your OE image?
>>> 
>>> It's not.
>>
>> Hum.  How about locale.dir, does it load that one?
>
> It does.

root@localhost:~# DISPLAY=:0 LC_ALL=pt_BR strace xev 2>&1 | grep X11
open("/usr/lib/libX11.so.6", O_RDONLY)  = 3
connect(3, {sa_family=AF_FILE, path="/tmp/.X11-unix/X0"}, 19) = 0
open("/usr/share/X11/locale/locale.alias", O_RDONLY) = 4
read(4, "#\t$XdotOrg: lib/X11/nls/locale.a"..., 1024) = 1024
open("/usr/share/X11/locale/locale.dir", O_RDONLY) = 4
read(4, "#\t$XdotOrg: lib/X11/nls/locale.d"..., 1024) = 1024
access("/usr/share/X11/locale/iso8859-1/XLC_LOCALE", R_OK) = 0
open("/usr/share/X11/locale/iso8859-1/XLC_LOCALE", O_RDONLY) = 4
open("/usr/share/X11/locale/locale.alias", O_RDONLY) = 4
read(4, "#\t$XdotOrg: lib/X11/nls/locale.a"..., 1024) = 1024
open("/usr/share/X11/locale/locale.dir", O_RDONLY) = 4
read(4, "#\t$XdotOrg: lib/X11/nls/locale.d"..., 1024) = 1024
access("/usr/share/X11/locale/iso8859-1/XLC_LOCALE", R_OK) = 0
open("/usr/share/X11/locale/iso8859-1/XLC_LOCALE", O_RDONLY) = 4

This is the whole output.

I fail to see why it doesn't even tries to open compose.dir. Any idea?

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Problem with i18n (input)

[Stefano Regno]

I would study to learn how to build a toolchain, could sameone tell me 
where is the file in openebedded project to read how you developped the 
toolchain please?

Regrets Stefano

-- 
Peace cannot be kept by force; it can only be achieved by understanding --Albert Einstein