[PATCH] Fix autodetection of UNICODE keymap

[PATCH] Fix autodetection of UNICODE keymap

[Andrey Borzenkov]

Original console_init from RH initscripts supports KEYTABLE variable
which dynamically determines whether to load UNICODE keymap depending
on current LANG value.

Scripts from 10i18n are more simplistic and assume the right keymap is
statically defined in configuration files.

Mandriva depends on console_init behavior and is using KEYTABLE
exclusively, so under new 10i18n non-UNICDE keymap is always loaded.

Add support for KEYTABLE to dynamically determine whether to install
UNICODE or non-UNICODE keymap version. This should probably go into
run-time 10i18n/console_init instead to be fully compatible.

Signed-off-by: Andrey Borzenkov <arvidjaar-JGs/UdohzUI@public.gmane.org>

---

 modules.d/10i18n/README  |    2 ++
 modules.d/10i18n/install |   54 +++++++++++++++++++++++++++++-----------------
 2 files changed, 36 insertions(+), 20 deletions(-)

diff --git a/modules.d/10i18n/README b/modules.d/10i18n/README
index c1878a7..11a7315 100644
--- a/modules.d/10i18n/README
+++ b/modules.d/10i18n/README
@@ -47,6 +47,8 @@ The following variables are used by i18n install script and at initramfs
 runtime:
 
     KEYMAP - keyboard translation table loaded by loadkeys
+    KEYTABLE - base name for keyboard translation table; if UNICODE is
+	true, Unicode version will be loaded. Overrides KEYMAP.
     EXT_KEYMAPS - list of extra keymaps to bo loaded (sep. by space)
     UNICODE - boolean, indicating UTF-8 mode
     SYSFONT - console font
diff --git a/modules.d/10i18n/install b/modules.d/10i18n/install
index 83999e5..de719b6 100755
--- a/modules.d/10i18n/install
+++ b/modules.d/10i18n/install
@@ -91,13 +91,42 @@ install_all_kbd() {
     dracut_install gzip bzip2
 }
 
+check_unicode() {
+    if [[ ${UNICODE} ]]
+    then
+        if [[ ${UNICODE^^} = YES || ${UNICODE} = 1 ]]
+        then
+            UNICODE=1
+        elif [[ ${UNICODE^^} = NO || ${UNICODE} = 0 ]]
+        then
+            UNICODE=0
+        else
+            UNICODE=''
+        fi
+    fi
+    if [[ ! ${UNICODE} && ${LANG^^} =~ .*\.UTF-?8 ]]
+    then
+        UNICODE=1
+    fi
+}
+
 install_local_keyboard() {
     local map
 
-    eval $(gather_vars ${keyboard_vars})
-
     # Gentoo user may have KEYMAP set to something like "-u pl2",
     KEYMAP=${KEYMAP#-* }
+
+    # KEYTABLE is a bit special - it defines base keymap name and UNICODE
+    # determines whether non-UNICODE or UNICODE version is used
+
+    if [[ ${KEYTABLE} ]]; then
+	if [[ ${UNICODE} == 1 ]]; then
+	    [[ ${KEYTABLE} =~ .*\.uni.* ]] || KEYTABLE=${KEYTABLE%.map*}.uni
+	fi
+
+	KEYMAP=${KEYTABLE}
+    fi
+
     # I'm not sure of the purpose of UNIKEYMAP and GRP_TOGGLE.  They were in
     # original redhat-i18n module.  Anyway it won't hurt.
     EXT_KEYMAPS+=\ ${UNIKEYMAP}\ ${GRP_TOGGLE}
@@ -118,7 +147,6 @@ install_local_keyboard() {
 }
 
 install_local_i18n() {
-    eval $(gather_vars ${i18n_vars})
 
     [[ ${SYSFONT} ]] || SYSFONT=${DEFAULT_SYSFONT}
     SYSFONT=${SYSFONT%.psf*}
@@ -136,23 +164,6 @@ install_local_i18n() {
         inst ${kbddir}/unimaps/${UNIMAP}.uni
     fi
 
-    if [[ ${UNICODE} ]]
-    then
-        if [[ ${UNICODE^^} = YES || ${UNICODE} = 1 ]]
-        then
-            UNICODE=1
-        elif [[ ${UNICODE^^} = NO || ${UNICODE} = 0 ]]
-        then
-            UNICODE=0
-        else
-            UNICODE=''
-        fi
-    fi
-    if [[ ! ${UNICODE} && ${LANG^^} =~ .*\.UTF-?8 ]]
-    then
-        UNICODE=1
-    fi
-
     mksubdirs ${I18N_CONF}
     print_vars LC_ALL LANG UNICODE SYSFONT CONTRANS UNIMAP >> ${I18N_CONF}
 }
@@ -187,6 +198,9 @@ then
 
     if [[ ${hostonly} ]]
     then
+	eval $(gather_vars ${keyboard_vars})
+	eval $(gather_vars ${i18n_vars})
+	check_unicode
         install_local_keyboard
         install_local_i18n
     else

Re: [PATCH] Fix autodetection of UNICODE keymap

[Harald Hoyer]

On 08/11/2010 07:47 PM, Andrey Borzenkov wrote:
> +
> +    # KEYTABLE is a bit special - it defines base keymap name and UNICODE
> +    # determines whether non-UNICODE or UNICODE version is used
> +
> +    if [[ ${KEYTABLE} ]]; then
> +	if [[ ${UNICODE} == 1 ]]; then
> +	    [[ ${KEYTABLE} =~ .*\.uni.* ]] || KEYTABLE=${KEYTABLE%.map*}.uni
> +	fi
> +
> +	KEYMAP=${KEYTABLE}
> +    fi

On Fedora we would end with:

KEYTABLE="de-latin1-nodeadkeys"

at

KEYMAP="de-latin1-nodeadkeys.uni"

which does not exist.

Re: [PATCH] Fix autodetection of UNICODE keymap

[Harald Hoyer]

On 08/12/2010 01:05 PM, Harald Hoyer wrote:
> On 08/11/2010 07:47 PM, Andrey Borzenkov wrote:
>> +
>> + # KEYTABLE is a bit special - it defines base keymap name and UNICODE
>> + # determines whether non-UNICODE or UNICODE version is used
>> +
>> + if [[ ${KEYTABLE} ]]; then
>> + if [[ ${UNICODE} == 1 ]]; then
>> + [[ ${KEYTABLE} =~ .*\.uni.* ]] || KEYTABLE=${KEYTABLE%.map*}.uni
>> + fi
>> +
>> + KEYMAP=${KEYTABLE}
>> + fi
>
> On Fedora we would end with:
>
> KEYTABLE="de-latin1-nodeadkeys"
>
> at
>
> KEYMAP="de-latin1-nodeadkeys.uni"
>
> which does not exist.

Fedora's console_init calls "loadkeys" with "-u", if its LANG has .UTF-8

Re[2]: [PATCH] Fix autodetection of UNICODE keymap

[Andrey Borzenkov]

Thu, 12 Aug 2010 13:08:25 +0200 письмо от Harald Hoyer <harald-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>:

> On 08/12/2010 01:05 PM, Harald Hoyer wrote:
> > On 08/11/2010 07:47 PM, Andrey Borzenkov wrote:
> >> +
> >> + # KEYTABLE is a bit special - it defines base keymap name and UNICODE
> >> + # determines whether non-UNICODE or UNICODE version is used
> >> +
> >> + if [[ ${KEYTABLE} ]]; then
> >> + if [[ ${UNICODE} == 1 ]]; then
> >> + [[ ${KEYTABLE} =~ .*\.uni.* ]] || KEYTABLE=${KEYTABLE%.map*}.uni
> >> + fi
> >> +
> >> + KEYMAP=${KEYTABLE}
> >> + fi
> >
> > On Fedora we would end with:
> >
> > KEYTABLE="de-latin1-nodeadkeys"
> >
> > at
> >
> > KEYMAP="de-latin1-nodeadkeys.uni"
> >
> > which does not exist.
> 

Could you send me your /etc/sysconfig/keyboard (or where these are defined)?

So, in other words - you have UTF-8 environment but load non-UNICODE keyboard?

> Fedora's console_init calls "loadkeys" with "-u", if its LANG has .UTF-8

It does not help here, unfortunately. Which version of initscripts is it?

Re: [PATCH] Fix autodetection of UNICODE keymap

[Amadeusz Żołnowski]

[-- Attachment #1: Type: text/plain, Size: 2230 bytes --]

Hello Andrey,

Andrey Borzenkov <arvidjaar-JGs/UdohzUI@public.gmane.org>:
> Original console_init from RH initscripts supports KEYTABLE variable
> which dynamically determines whether to load UNICODE keymap depending
> on current LANG value.
> 
> Scripts from 10i18n are more simplistic and assume the right keymap is
> statically defined in configuration files.

10i18n install script determines UTF-8 mode in basis of UNICODE
variable value or LANG value.  console_init doesn't have to perform any
auto-detection again, although it might be useful to check LANG again
instead of or additionally to checking UNICODE kernel cmdline arg.
Maybe I'll add sth like this.


> Add support for KEYTABLE to dynamically determine whether to install
> UNICODE or non-UNICODE keymap version. This should probably go into
> run-time 10i18n/console_init instead to be fully compatible.

-- CUT --

> +    KEYTABLE - base name for keyboard translation table; if UNICODE
> is
> +	true, Unicode version will be loaded. Overrides KEYMAP.

-- CUT --

> +    # KEYTABLE is a bit special - it defines base keymap name and
> UNICODE
> +    # determines whether non-UNICODE or UNICODE version is used
> +
> +    if [[ ${KEYTABLE} ]]; then
> +	if [[ ${UNICODE} == 1 ]]; then
> +	    [[ ${KEYTABLE} =~ .*\.uni.* ]] ||
> KEYTABLE=${KEYTABLE%.map*}.uni
> +	fi
> +
> +	KEYMAP=${KEYTABLE}
> +    fi
> +

What does *.uni* have to do with KEYMAP? Maybe I misunderstand it, but
we set up keyboard with two things:
  1) loadkeys to which we provide map from keymaps directory
     (where are no *.uni* files) and optionally '-u' which is UTF-8
     switch;
  2) kbd_mode to which we provide '-u' (UTF-8) or '-a' (ASCII).

*.uni* files are in unimaps directory. Those files are used by setfont.

Please explain me what is really the KEYTABLE variable and please check
out if you can achieve what you want with mappings in config file. Have
you read the README file?

If you have to resist with KEYTABLE, please publish archive with
configs where those vars are set up in your system, kbd dirs and init
scripts which set up those things.


Cheers,
Amadeusz Żołnowski

-- 
PGP key: 1024D/C284750D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [PATCH] Fix autodetection of UNICODE keymap

[Harald Hoyer]

On 08/12/2010 01:48 PM, Andrey Borzenkov wrote:
>
> Thu, 12 Aug 2010 13:08:25 +0200 письмо от Harald Hoyer<harald-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>:
>
>> On 08/12/2010 01:05 PM, Harald Hoyer wrote:
>>> On 08/11/2010 07:47 PM, Andrey Borzenkov wrote:
>>>> +
>>>> + # KEYTABLE is a bit special - it defines base keymap name and UNICODE
>>>> + # determines whether non-UNICODE or UNICODE version is used
>>>> +
>>>> + if [[ ${KEYTABLE} ]]; then
>>>> + if [[ ${UNICODE} == 1 ]]; then
>>>> + [[ ${KEYTABLE} =~ .*\.uni.* ]] || KEYTABLE=${KEYTABLE%.map*}.uni
>>>> + fi
>>>> +
>>>> + KEYMAP=${KEYTABLE}
>>>> + fi
>>>
>>> On Fedora we would end with:
>>>
>>> KEYTABLE="de-latin1-nodeadkeys"
>>>
>>> at
>>>
>>> KEYMAP="de-latin1-nodeadkeys.uni"
>>>
>>> which does not exist.
>>
>
> Could you send me your /etc/sysconfig/keyboard (or where these are defined)?
>
> So, in other words - you have UTF-8 environment but load non-UNICODE keyboard?
>
>> Fedora's console_init calls "loadkeys" with "-u", if its LANG has .UTF-8
>
> It does not help here, unfortunately. Which version of initscripts is it?
> --
>

http://git.fedorahosted.org/git/?p=initscripts.git;a=blob;f=src/console_init.c

It's been like this forever...

http://git.fedorahosted.org/git/?p=initscripts.git;a=history;f=src/console_init.c

# loadkeys -u de-latin1-nodeadkeys
Loading /lib/kbd/keymaps/i386/qwertz/de-latin1-nodeadkeys.map.gz

Re: [PATCH] Fix autodetection of UNICODE keymap

[Andrey Borzenkov]

[-- Attachment #1: Type: Text/Plain, Size: 2255 bytes --]

On Thursday 12 of August 2010 16:28:28 Harald Hoyer wrote:
> On 08/12/2010 01:48 PM, Andrey Borzenkov wrote:
> > Thu, 12 Aug 2010 13:08:25 +0200 письмо от Harald 
Hoyer<harald-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>:
> >> On 08/12/2010 01:05 PM, Harald Hoyer wrote:
> >>> On 08/11/2010 07:47 PM, Andrey Borzenkov wrote:
> >>>> +
> >>>> + # KEYTABLE is a bit special - it defines base keymap name and
> >>>> UNICODE + # determines whether non-UNICODE or UNICODE version
> >>>> is used +
> >>>> + if [[ ${KEYTABLE} ]]; then
> >>>> + if [[ ${UNICODE} == 1 ]]; then
> >>>> + [[ ${KEYTABLE} =~ .*\.uni.* ]] ||
> >>>> KEYTABLE=${KEYTABLE%.map*}.uni + fi
> >>>> +
> >>>> + KEYMAP=${KEYTABLE}
> >>>> + fi
> >>> 
> >>> On Fedora we would end with:
> >>> 
> >>> KEYTABLE="de-latin1-nodeadkeys"
> >>> 
> >>> at
> >>> 
> >>> KEYMAP="de-latin1-nodeadkeys.uni"
> >>> 
> >>> which does not exist.
> > 
> > Could you send me your /etc/sysconfig/keyboard (or where these are
> > defined)?
> > 
> > So, in other words - you have UTF-8 environment but load
> > non-UNICODE keyboard?
> > 
> >> Fedora's console_init calls "loadkeys" with "-u", if its LANG has
> >> .UTF-8
> > 
> > It does not help here, unfortunately. Which version of initscripts
> > is it? --
> 
> http://git.fedorahosted.org/git/?p=initscripts.git;a=blob;f=src/conso
> le_init.c
> 
> It's been like this forever...
> 
> http://git.fedorahosted.org/git/?p=initscripts.git;a=history;f=src/co
> nsole_init.c
> 
> # loadkeys -u de-latin1-nodeadkeys
> Loading /lib/kbd/keymaps/i386/qwertz/de-latin1-nodeadkeys.map.gz

OK, so

- KEYTABLE handling is Mandriva specific patch. Sorry for confusion
- loadkeys -u fails misereably for all Russian keymaps that use 8 bit 
character set. This is generic kbd issue that is not easy to fix (in 
short - loadkeys treats all single byte characters as latin1. So it 
converts into UNICODE just fine - but not Russian UNICODE, rather Latin 
one ...). 

This means there is no way to have single Russian keymap that works 
either in 8 bit or UTF-8 character sets. But Mandriva tools use single 
KEYTABLE for defining keyboard layout ...

I think I just leave it as distro-specific patch for now then.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]