[Qemu-devel] Keysymbol interpretation missing in QEMU's VNC server?

[Qemu-devel] Keysymbol interpretation missing in QEMU's VNC server?

[Fabian Holler]

Hello,

I'm not sure if I found a bug in QEMU's VNC keyboard layout mapping or
if it's a general problem in the implemented VNC server:

Scenario:
QEMU started with: "-k de"
Keyboard layout in VM: de
Keyboard layout from Client OS: us

What i expect:
I type the '/' character on the Client OS (key left from the right-shift-key) on US layout.
Keysymbol '/' is send over VNC to the QEMU.
QEMU lookup in the de keyboard mapping table for the character '/' and
should find the scancodes for the keys shift+'7'.
The Scancodes for shift and '7' are send to the VM's emulated keyboard
controller and the '/' appears in the VM.

But what actually happens is that the '7' character shows up in the VM.
It seems that QEMU misses to also generate the Shift Scancode.

Is this a general problem in the VNC server implementation?
So that an interpretation (http://tools.ietf.org/html/rfc6143#section-7.5.4)
of the received keysymbols to add eg an additional shift keypress
never happens?

If yes would a QEMU patch that adds keysymbol interpretation have a
chance to be merged in the upstream version?
Or are there reasons that it isn't a good idea to interpret the
keysymbols for the scancode conversion?


regards

Fabian

[Qemu-devel] Keysymbol interpretation missing in QEMU's VNC server?

[Fabian Holler]

Hello,

I'm not sure if I found a bug in QEMU's VNC keyboard layout mapping or
if it's a general problem in the implemented VNC server:

Scenario:
QEMU started with: "-k de"
Keyboard layout in VM: de
Keyboard layout from Client OS: us

What i expect:
I type the '/' character on the Client OS (key left from the right-shift-key) on US layout.
Keysymbol '/' is send over VNC to the QEMU.
QEMU lookup in the de keyboard mapping table for the character '/' and
should find the scancodes for the keys shift+'7'.
The Scancodes for shift and '7' are send to the VM's emulated keyboard
controller and the '/' appears in the VM.

But what actually happens is that the '7' character shows up in the VM.
It seems that QEMU misses to also generate the Shift Scancode.

Is this a general problem in the VNC server implementation?
So that an interpretation (http://tools.ietf.org/html/rfc6143#section-7.5.4)
of the received keysymbols to add eg an additional shift keypress
never happens?

If yes, would a QEMU patch that adds keysymbol interpretation have a
chance to be merged into upstream?
Or are there reasons that it isn't a good idea to interpret the
keysymbols for the scancode conversion?


regards

Fabian

Re: [Qemu-devel] Keysymbol interpretation missing in QEMU's VNC server?

[Philipp Hahn]

[-- Attachment #1: Type: text/plain, Size: 929 bytes --]

Hello Fabian,

On Wednesday 07 March 2012 11:53:57 Fabian Holler wrote:
> I'm not sure if I found a bug in QEMU's VNC keyboard layout mapping or
> if it's a general problem in the implemented VNC server:

Daniel P. Berrangé has written some very good blog entries about the VNC 
problem: 
<http://berrange.com/posts/2010/07/04/more-than-you-or-i-ever-wanted-to-know-about-virtual-keyboard-handling/>
They're quiet technical, but after reading them I at least concluded that VNC 
is normally just broken.
I recommend using virt-viewer, with which I have mostly no such problems.

Sincerely
Philipp
-- 
Philipp Hahn           Open Source Software Engineer      hahn@univention.de
Univention GmbH        Linux for Your Business        fon: +49 421 22 232- 0
Mary-Somerville-Str.1  D-28359 Bremen                 fax: +49 421 22 232-99
                                                   http://www.univention.de/

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [Qemu-devel] Keysymbol interpretation missing in QEMU's VNC server?

[Fabian Holler]

Hello Philipp,

thanks for your answer.

On Thu, Mar 08, 2012 at 09:24:13PM +0100, Philipp Hahn wrote:
> On Wednesday 07 March 2012 11:53:57 Fabian Holler wrote:
> > I'm not sure if I found a bug in QEMU's VNC keyboard layout mapping or
> > if it's a general problem in the implemented VNC server:
> 
> Daniel P. Berrangé has written some very good blog entries about the VNC 
> problem: 
> <http://berrange.com/posts/2010/07/04/more-than-you-or-i-ever-wanted-to-know-about-virtual-keyboard-handling/>
> They're quiet technical, but after reading them I at least concluded that VNC 
> is normally just broken.

I already discovered Berrangès articles, helped me very lot to
understand the problem.

> I recommend using virt-viewer, with which I have mostly no such problems.

Ah, never heard of it, it seems also be gtk-vnc based.
I already tried gtk-vnc and it works but we need a Web based solution
for our customers.
I found Java VNC (http://www.jdams.org/java-vnc), which also sends the
QEMU extended key event messages (based on TigerVNC). But in the JVM is
the same old problem that the scan code/key code isn't known.

So our current plan is to let the customer choose his
local keyboard layout in the applet and recover the key code for the key
symbol (+ add/remove if needed metakey events like shift) on the client.

Similar like QEMU's "-k" option, but on the client-side +
interpretation. We will only can support a limited number of keyboard
layouts but changing the layout without restarting QEMU will be
possible. :-)


regards

Fabian

Re: [Qemu-devel] Keysymbol interpretation missing in QEMU's VNC server?

[Anthony Liguori]

On 03/07/2012 04:53 AM, Fabian Holler wrote:
> Hello,
>
> I'm not sure if I found a bug in QEMU's VNC keyboard layout mapping or
> if it's a general problem in the implemented VNC server:
>
> Scenario:
> QEMU started with: "-k de"
> Keyboard layout in VM: de
> Keyboard layout from Client OS: us
>
> What i expect:
> I type the '/' character on the Client OS (key left from the right-shift-key) on US layout.
> Keysymbol '/' is send over VNC to the QEMU.
> QEMU lookup in the de keyboard mapping table for the character '/' and
> should find the scancodes for the keys shift+'7'.
> The Scancodes for shift and '7'

This does not exist.  There is no such thing as "Scancodes for shift and '7'".

Instead, what's sent to the Client OS is literally, "the key at the fourth 
column, second row".

The sticker that is on that key is unknown to the physical keyboard.  When you 
select a keyboard type in an OS, you're literally telling the OS what labels are 
on the physical keyboard.

Yes, it's almost mind blowing that in 2012, keyboards don't tell you what the 
labels on the keys are... but they don't.

When you tell QEMU to use '-k de', what you're telling it is to figure out what 
the right key is on a de keyboard for the label '/' which may be "the key at the 
second 8th column, second row" on a de keyboard, but on a US keyboard, it's a 7.

There's really nothing that can be done about this.  The way gtk-vnc fixes this 
is by obtaining the actual scancode from the user's keyboard.  But you can't get 
this in Java in an applet AFAIK.

Regards,

Anthony Liguori

  are send to the VM's emulated keyboard
> controller and the '/' appears in the VM.
>
> But what actually happens is that the '7' character shows up in the VM.
> It seems that QEMU misses to also generate the Shift Scancode.
>
> Is this a general problem in the VNC server implementation?
> So that an interpretation (http://tools.ietf.org/html/rfc6143#section-7.5.4)
> of the received keysymbols to add eg an additional shift keypress
> never happens?
>
> If yes would a QEMU patch that adds keysymbol interpretation have a
> chance to be merged in the upstream version?
> Or are there reasons that it isn't a good idea to interpret the
> keysymbols for the scancode conversion?
>
>
> regards
>
> Fabian
>

Re: [Qemu-devel] Keysymbol interpretation missing in QEMU's VNC server?

[Fabian Holler]

Hello Anthony,


On Thu, Mar 08, 2012 at 03:30:18PM -0600, Anthony Liguori wrote:
> On 03/07/2012 04:53 AM, Fabian Holler wrote:
> >Hello,
> >
> >I'm not sure if I found a bug in QEMU's VNC keyboard layout mapping or
> >if it's a general problem in the implemented VNC server:
> >
> >Scenario:
> >QEMU started with: "-k de"
> >Keyboard layout in VM: de
> >Keyboard layout from Client OS: us
> >
> >What i expect:
> >I type the '/' character on the Client OS (key left from the right-shift-key) on US layout.
> >Keysymbol '/' is send over VNC to the QEMU.
> >QEMU lookup in the de keyboard mapping table for the character '/' and
> >should find the scancodes for the keys shift+'7'.
> >The Scancodes for shift and '7'
> 
> This does not exist.  There is no such thing as "Scancodes for shift and '7'".
> 
> Instead, what's sent to the Client OS is literally, "the key at the
> fourth column, second row".

Yes, I know.
What i actually meant was: with the knowledge of the used
keyboard layout and the key symbol QEMU could figure out that it has to
send scan codes for the keys that are labeled SHIFT & 7 on a keyboard
with DE layout.
Afaik QEMU does this already for the 7 labeled key. But it doesn't
remove/adds additional needed metakey presses like eg Shift.
Or I'm wrong?

The idea was to add these interpretation to also add/remove additional
metakey scancodes for the VM if needed.

> There's really nothing that can be done about this.  The way gtk-vnc
> fixes this is by obtaining the actual scancode from the user's
> keyboard.  But you can't get this in Java in an applet AFAIK.

The same should work in a Java Applet but you also have to figure out
the used keyboard layout and handle metakeys.


regards

Fabian

Re: [Qemu-devel] Keysymbol interpretation missing in QEMU's VNC server?

[Erik Rull]

On March 8, 2012 at 11:01 PM Fabian Holler <fabian.holler@profitbricks.com>
wrote:

> Hello Anthony,
>
>
> On Thu, Mar 08, 2012 at 03:30:18PM -0600, Anthony Liguori wrote:
> > On 03/07/2012 04:53 AM, Fabian Holler wrote:
> > >Hello,
> > >
> > >I'm not sure if I found a bug in QEMU's VNC keyboard layout mapping or
> > >if it's a general problem in the implemented VNC server:
> > >
> > >Scenario:
> > >QEMU started with: "-k de"
> > >Keyboard layout in VM: de
> > >Keyboard layout from Client OS: us
> > >
> > >What i expect:
> > >I type the '/' character on the Client OS (key left from the
right-shift-key) on US layout.
> > >Keysymbol '/' is send over VNC to the QEMU.
> > >QEMU lookup in the de keyboard mapping table for the character '/' and
> > >should find the scancodes for the keys shift+'7'.
> > >The Scancodes for shift and '7'
> >
> > This does not exist.  There is no such thing as "Scancodes for shift
and '7'".
> >
> > Instead, what's sent to the Client OS is literally, "the key at the
> > fourth column, second row".
>
> Yes, I know.
> What i actually meant was: with the knowledge of the used
> keyboard layout and the key symbol QEMU could figure out that it has to
> send scan codes for the keys that are labeled SHIFT & 7 on a keyboard
> with DE layout.
> Afaik QEMU does this already for the 7 labeled key. But it doesn't
> remove/adds additional needed metakey presses like eg Shift.
> Or I'm wrong?
>
> The idea was to add these interpretation to also add/remove additional
> metakey scancodes for the VM if needed.
>
> > There's really nothing that can be done about this.  The way gtk-vnc
> > fixes this is by obtaining the actual scancode from the user's
> > keyboard.  But you can't get this in Java in an applet AFAIK.
>
> The same should work in a Java Applet but you also have to figure out
> the used keyboard layout and handle metakeys.
>
>
> regards
>
> Fabian
>


Hi all,

I'm experiencing the same issues.

When using the QEMU VNC server (which I would prefer to Remote Desktop or a
VNC server installed on the guest) I get problems when pressing special
character keys on the remote keyboard. QEMU reports on the console:
Warning: no scancode found for keysym 180
or
Warning: no scancode found for keysym 223

I just want to route all keypresses to the guest without interfering with
the native QEMU key layout. Is that possible? When running QEMU via SDL and
a real monitor and keyboard attached, I have no problems with the different
keyboard language layouts (I tested US, DE, ES, FR). Well - there I switch
the keyboard language layout in Windows for having all keypresses handled
correctly (in most cases, Windows detects the correct keyboard layout
automatically), but there is no change needed in the QEMU command line when
having different hardware attached!

Any ideas how to resolve that? How does the QEMU VNC server receive the key
presses? In the same manner as the direct way does by getting scancodes or
via "real" characters?
My requirement for the VNC usage is, that all clients that connect via VNC
can work independently of their client keyboard layout. The manual switch
in Windows for getting the proper key translation is fine for the clients
that use VNC.

Best regards,

Erik

Re: [Qemu-devel] Keysymbol interpretation missing in QEMU's VNC server?

[Erik Rull]

 On May 23, 2012 at 10:16 AM Erik Rull <erik.rull@rdsoftware.de> wrote:

>
> On March 8, 2012 at 11:01 PM Fabian Holler
<fabian.holler@profitbricks.com>
> wrote:
>
> > Hello Anthony,
> >
> >
> > On Thu, Mar 08, 2012 at 03:30:18PM -0600, Anthony Liguori wrote:
> > > On 03/07/2012 04:53 AM, Fabian Holler wrote:
> > > >Hello,
> > > >
> > > >I'm not sure if I found a bug in QEMU's VNC keyboard layout mapping
or
> > > >if it's a general problem in the implemented VNC server:
> > > >
> > > >Scenario:
> > > >QEMU started with: "-k de"
> > > >Keyboard layout in VM: de
> > > >Keyboard layout from Client OS: us
> > > >
> > > >What i expect:
> > > >I type the '/' character on the Client OS (key left from the
> right-shift-key) on US layout.
> > > >Keysymbol '/' is send over VNC to the QEMU.
> > > >QEMU lookup in the de keyboard mapping table for the character '/'
and
> > > >should find the scancodes for the keys shift+'7'.
> > > >The Scancodes for shift and '7'
> > >
> > > This does not exist.  There is no such thing as "Scancodes for shift
> and '7'".
> > >
> > > Instead, what's sent to the Client OS is literally, "the key at the
> > > fourth column, second row".
> >
> > Yes, I know.
> > What i actually meant was: with the knowledge of the used
> > keyboard layout and the key symbol QEMU could figure out that it has to
> > send scan codes for the keys that are labeled SHIFT & 7 on a keyboard
> > with DE layout.
> > Afaik QEMU does this already for the 7 labeled key. But it doesn't
> > remove/adds additional needed metakey presses like eg Shift.
> > Or I'm wrong?
> >
> > The idea was to add these interpretation to also add/remove additional
> > metakey scancodes for the VM if needed.
> >
> > > There's really nothing that can be done about this.  The way gtk-vnc
> > > fixes this is by obtaining the actual scancode from the user's
> > > keyboard.  But you can't get this in Java in an applet AFAIK.
> >
> > The same should work in a Java Applet but you also have to figure out
> > the used keyboard layout and handle metakeys.
> >
> >
> > regards
> >
> > Fabian
> >
>
>
> Hi all,
>
> I'm experiencing the same issues.
>
> When using the QEMU VNC server (which I would prefer to Remote Desktop or
a
> VNC server installed on the guest) I get problems when pressing special
> character keys on the remote keyboard. QEMU reports on the console:
> Warning: no scancode found for keysym 180
> or
> Warning: no scancode found for keysym 223
>
> I just want to route all keypresses to the guest without interfering with
> the native QEMU key layout. Is that possible? When running QEMU via SDL
and
> a real monitor and keyboard attached, I have no problems with the
different
> keyboard language layouts (I tested US, DE, ES, FR). Well - there I
switch
> the keyboard language layout in Windows for having all keypresses handled
> correctly (in most cases, Windows detects the correct keyboard layout
> automatically), but there is no change needed in the QEMU command line
when
> having different hardware attached!
>
> Any ideas how to resolve that? How does the QEMU VNC server receive the
key
> presses? In the same manner as the direct way does by getting scancodes
or
> via "real" characters?
> My requirement for the VNC usage is, that all clients that connect via
VNC
> can work independently of their client keyboard layout. The manual switch
> in Windows for getting the proper key translation is fine for the clients
> that use VNC.
>
> Best regards,
>
> Erik
>

Just a small addition: The AltGr Keys are not routed correctly through VNC!
I had a look at the "de" keymap, it looks as if they are described (e.g.
backslash \) which is placed on the german keyboard on the key right to the
0 key (german key ssharp) and accessed via AltGr + this key. Same with }
which is at the 0 key position accessed via AltGr + 0. The shifted keys and
the Windows standard functions Ctrl + C / Ctrl + V works as well.
They are not lost completely but the AltGr press gets lost! So when
pressing AltGr + 0, I get only a 0 on the guest instead of a "}".

Any ideas?

Best regards,

Erik

Re: [Qemu-devel] Keysymbol interpretation missing in QEMU's VNC server?

[Gonglei (Arei)]

Hi,

> Just a small addition: The AltGr Keys are not routed correctly through VNC!
> I had a look at the "de" keymap, it looks as if they are described (e.g.
> backslash \) which is placed on the german keyboard on the key right to the
> 0 key (german key ssharp) and accessed via AltGr + this key. Same with }
> which is at the 0 key position accessed via AltGr + 0. The shifted keys and
> the Windows standard functions Ctrl + C / Ctrl + V works as well.
> They are not lost completely but the AltGr press gets lost! So when
> pressing AltGr + 0, I get only a 0 on the guest instead of a "}".
> 
> Any ideas?

By googled, I found the topic. and I encountered the same probleam.
I am convinced that the AltGr key conflicted with the QEMU console switch.
On the German keyboard, the AltGr as same with Ctrl + Alt on the US keyborad.
Same as you said, the { accessed via AltGr + 7. But at present, the Ctrl + Alt + 1..9
are reserved as QEMU console switch. When we push down the AltGr + 7,
do_key_event function will reset key and enter console_select for AltGr (Ctrl + Alt), 
and finally we get the last key "7" instead of a "{".

the Qemu code(qemu/ui/vnc.c):
static void do_key_event(VncState *vs, int down, int keycode, int sym)
{
    /* QEMU console switch */
    switch(keycode) {
    case 0x2a:                          /* Left Shift */
    case 0x36:                          /* Right Shift */
    case 0x1d:                          /* Left CTRL */
    case 0x9d:                          /* Right CTRL */
    case 0x38:                          /* Left ALT */
    case 0xb8:                          /* Right ALT */
        if (down)
            vs->modifiers_state[keycode] = 1;
        else
            vs->modifiers_state[keycode] = 0;
        break;
    case 0x02 ... 0x0a: /* '1' to '9' keys */
        if (down && vs->modifiers_state[0x1d] && vs->modifiers_state[0x38]) {
            /* Reset the modifiers sent to the current console */
            reset_keys(vs);
            console_select(keycode - 0x02);
            return;
        }
        break;
    ... ...
}

Best regards,
-Gonglei

Re: [Qemu-devel] Keysymbol interpretation missing in QEMU's VNC server?

[Gonglei (Arei)]

Hi,
Would you give me some advice. Thanks!

I thank we can only need reserve four Qemu console,
Ctrl +1 for graphic show, Ctrl+2 for monitor console,
Ctrl + 3 for serial, Ctrl+4 for parallel. And then will not conflict with "de" keymay.

Best regards,
-Gonglei


> -----Original Message-----
> From: Gonglei (Arei)
> Sent: Tuesday, January 14, 2014 4:45 PM
> To: 'erik.rull@rdsoftware.de'; 'Anthony Liguori'; 'Gerd Hoffmann'
> Cc: Luonengjun; Huangweidong (Hardware); qemu-devel@nongnu.org
> Subject: Re: [Qemu-devel] Keysymbol interpretation missing in QEMU's VNC
> server?
> 
> Hi,
> 
> > Just a small addition: The AltGr Keys are not routed correctly through VNC!
> > I had a look at the "de" keymap, it looks as if they are described (e.g.
> > backslash \) which is placed on the german keyboard on the key right to the
> > 0 key (german key ssharp) and accessed via AltGr + this key. Same with }
> > which is at the 0 key position accessed via AltGr + 0. The shifted keys and
> > the Windows standard functions Ctrl + C / Ctrl + V works as well.
> > They are not lost completely but the AltGr press gets lost! So when
> > pressing AltGr + 0, I get only a 0 on the guest instead of a "}".
> >
> > Any ideas?
> 
> By googled, I found the topic. and I encountered the same probleam.
> I am convinced that the AltGr key conflicted with the QEMU console switch.
> On the German keyboard, the AltGr as same with Ctrl + Alt on the US keyborad.
> Same as you said, the { accessed via AltGr + 7. But at present, the Ctrl + Alt +
> 1..9
> are reserved as QEMU console switch. When we push down the AltGr + 7,
> do_key_event function will reset key and enter console_select for AltGr (Ctrl +
> Alt),
> and finally we get the last key "7" instead of a "{".
> 
> the Qemu code(qemu/ui/vnc.c):
> static void do_key_event(VncState *vs, int down, int keycode, int sym)
> {
>     /* QEMU console switch */
>     switch(keycode) {
>     case 0x2a:                          /* Left Shift */
>     case 0x36:                          /* Right Shift */
>     case 0x1d:                          /* Left CTRL */
>     case 0x9d:                          /* Right CTRL */
>     case 0x38:                          /* Left ALT */
>     case 0xb8:                          /* Right ALT */
>         if (down)
>             vs->modifiers_state[keycode] = 1;
>         else
>             vs->modifiers_state[keycode] = 0;
>         break;
>     case 0x02 ... 0x0a: /* '1' to '9' keys */
>         if (down && vs->modifiers_state[0x1d] && vs->modifiers_state[0x38])
> {
>             /* Reset the modifiers sent to the current console */
>             reset_keys(vs);
>             console_select(keycode - 0x02);
>             return;
>         }
>         break;
>     ... ...
> }
> 
> Best regards,
> -Gonglei
>