[Qemu-devel] QEMU RFB (vnc) driver

[Qemu-devel] QEMU RFB (vnc) driver

[Matthew Mastracci]

[-- Attachment #1: Type: text/plain, Size: 985 bytes --]

I'm working on adding RFB support for QEMU.  I've attached my
work-in-progress here.  You'll need to compile and link against
libvncserver to make it work- I'm not an autoconf wizard, so I just
hacked my local version for now.

The whole thing works surprisingly well.  I managed to talk to QEMU on
my Linux box from my LAN-connected Windows box with amazing performance.

It's a bit of a pain using the mouse over VNC since everything is done
using relative position packets.  It might be possible to emulate a
Synaptics PS/2 touchpad (which supports absolute positioning and is
supported under Windows and X) to make this a bit easier, but I'm not
certain how difficult that would be.  It might be easier to whip up a
special VNC client that supports mouse capturing.  :)

I've also left the keymapping as a "work in progress" until I find an
easy way to map from the vnc codes (which seem to be X codes) to the
PC-101 scancodes.  Any ideas?

-- 
Matthew Mastracci <matt@aclaro.com>

[-- Attachment #2: vnc.c --]
[-- Type: text/x-c, Size: 3496 bytes --]

/*
 * QEMU VNC display driver
 * 
 * Copyright (c) 2003 Fabrice Bellard
 * 
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
 * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 * THE SOFTWARE.
 */
#include "vl.h"

#include <rfb/rfb.h>

#ifndef _WIN32
#include <signal.h>
#endif

rfbScreenInfoPtr rfbScreen;
int last_x_position;
int last_y_position;

static void vnc_update(DisplayState *ds, int x, int y, int w, int h)
{
//    printf("updating x=%d y=%d w=%d h=%d\n", x, y, w, h);
    rfbMarkRectAsModified(rfbScreen, x, y, x+w, y+h);
}

static void vnc_resize(DisplayState *ds, int w, int h)
{
    printf("resize! w=%i h=%i\n", w, h);
    ds->data = rfbScreen->frameBuffer + ( 1024 - w ) + ( 768 - h ) * 1024;
    ds->linesize = 1024*2;
    ds->depth = 16;
    rfbFillRect(rfbScreen, 0, 0, 1023, 767, 0);
}

static void vnc_refresh(DisplayState *ds)
{
    vga_update_display();  
//    rfbProcessEvents(rfbScreen,1000000);
}

static void vnc_pointer(int buttonMask, int x, int y, rfbClientPtr cl)
{
    int dx, dy, buttons;
    dx = x - last_x_position;
    last_x_position = x;
    dy = y - last_y_position;
    last_y_position = y;

	buttons = 0;

	if ( buttonMask & 1 )
		buttons |= MOUSE_EVENT_LBUTTON;
	if ( buttonMask & 2 )
		buttons |= MOUSE_EVENT_MBUTTON;
	if ( buttonMask & 4 )
		buttons |= MOUSE_EVENT_RBUTTON;

    kbd_mouse_event(dx, dy, 0, buttons);
}

static void vnc_keyboard(rfbBool down, rfbKeySym key, rfbClientPtr cl)
{
	if (key == 'A')
		if (down)
			kbd_put_keycode(0x38);
		else
			kbd_put_keycode(0x38 | 0x80);
}

void vnc_display_init(DisplayState *ds)
{
    int flags;
    last_x_position = 0;
    last_y_position = 0;

    rfbScreen = rfbGetScreen(0, NULL, 1024, 768, 5, 3, 2);
    rfbScreen->frameBuffer = (char*)malloc(1074*768*2);
    rfbScreen->rfbServerFormat.bitsPerPixel = 16;
    rfbScreen->rfbServerFormat.depth = 16;
    rfbScreen->rfbServerFormat.bigEndian = 0;
    rfbScreen->rfbServerFormat.trueColour = 1;
    rfbScreen->rfbServerFormat.redMax = (1 << 5) - 1;
    rfbScreen->rfbServerFormat.greenMax = (1 << 6) - 1;
    rfbScreen->rfbServerFormat.blueMax = (1 << 5) - 1;
    rfbScreen->rfbServerFormat.redShift = 11;
    rfbScreen->rfbServerFormat.greenShift = 5;
    rfbScreen->rfbServerFormat.blueShift = 0;

	rfbScreen->ptrAddEvent = vnc_pointer;
	rfbScreen->kbdAddEvent = vnc_keyboard;

    rfbInitServer(rfbScreen);
	rfbRunEventLoop(rfbScreen, 1000000, TRUE);

    vnc_resize(ds, 640, 480);

    ds->dpy_update = vnc_update;
    ds->dpy_resize = vnc_resize;
    ds->dpy_refresh = vnc_refresh;
}

Re: [Qemu-devel] QEMU RFB (vnc) driver

[Brad Campbell]

Matthew Mastracci wrote:
> I'm working on adding RFB support for QEMU.  I've attached my
> work-in-progress here.  You'll need to compile and link against
> libvncserver to make it work- I'm not an autoconf wizard, so I just
> hacked my local version for now.
> 
> The whole thing works surprisingly well.  I managed to talk to QEMU on
> my Linux box from my LAN-connected Windows box with amazing performance.
> 
> It's a bit of a pain using the mouse over VNC since everything is done
> using relative position packets.  It might be possible to emulate a
> Synaptics PS/2 touchpad (which supports absolute positioning and is
> supported under Windows and X) to make this a bit easier, but I'm not
> certain how difficult that would be.  It might be easier to whip up a
> special VNC client that supports mouse capturing.  :)
> 

Just out of interest, the mouse behaves just as badly running qemu under an X vnc server session.

Oh, and did I mention, nice work!

Regards,
Brad

[Qemu-devel] Re: QEMU RFB (vnc) driver

[Matthew Mastracci]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

Brad Campbell wrote:

> Matthew Mastracci wrote:
>
>> It's a bit of a pain using the mouse over VNC since everything is done
>> using relative position packets.  It might be possible to emulate a
>> Synaptics PS/2 touchpad (which supports absolute positioning and is
>> supported under Windows and X) to make this a bit easier, but I'm not
>> certain how difficult that would be.  It might be easier to whip up a
>> special VNC client that supports mouse capturing.  :)
>
>
> Just out of interest, the mouse behaves just as badly running qemu 
> under an X vnc server session.
>
> Oh, and did I mention, nice work!

Thanks!  The mouse issue is certainly a problem - there's no tracking 
between the virtual VNC cursor and the real guest cursor. 

I hope that having a host/guest interface port (see the other recent 
thread) will make the whole thing a lot easier.  If we know where the 
guest's cursor is, we can figure out the relative movement required to 
get us there.
 
It's tough for me to post it as a diff -u against CVS because I'm so 
weak with autoconf-related changes.  I had to hack my own makefile just 
to get it to compile.  I'll look into what it would take to turn it into 
a full-fledged patch.

Matt.  

[-- Attachment #2: matt.vcf --]
[-- Type: text/x-vcard, Size: 286 bytes --]

begin:vcard
fn:Matthew Mastracci
n:Mastracci;Matthew
org:aclaro Softworks, inc.
adr:;;1900 a - 11 St. SE;Calgary;Alberta;T2H 3G2;Canada
email;internet:matt@aclaro.com
title:Software Developer
tel;work:(403) 299-6612
x-mozilla-html:FALSE
url:http://www.aclaro.com
version:2.1
end:vcard

Re: [Qemu-devel] QEMU RFB (vnc) driver

[Fabrice Bellard]

Brad Campbell wrote:
> Matthew Mastracci wrote:
> 
>> I'm working on adding RFB support for QEMU.  I've attached my
>> work-in-progress here.  You'll need to compile and link against
>> libvncserver to make it work- I'm not an autoconf wizard, so I just
>> hacked my local version for now.
>>
>> The whole thing works surprisingly well.  I managed to talk to QEMU on
>> my Linux box from my LAN-connected Windows box with amazing performance.
>>
>> It's a bit of a pain using the mouse over VNC since everything is done
>> using relative position packets.  It might be possible to emulate a
>> Synaptics PS/2 touchpad (which supports absolute positioning and is
>> supported under Windows and X) to make this a bit easier, but I'm not
>> certain how difficult that would be.  It might be easier to whip up a
>> special VNC client that supports mouse capturing.  :)

I vote for the Synaptics PS/2 touchpad emulation. From the Linux kernel 
source, it does not seem difficult.

Another topic is the keycode conversions. Even with SDL, a solution like 
bochs or rdesktop (with an option to give the host keyboard mapping) is 
needed. That's why I did not merge the various patch submitted to 
'improve' the SDL keyboard emulation.

Fabrice.

Re: [Qemu-devel] QEMU RFB (vnc) driver

[Johannes Schindelin]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2628 bytes --]

Hi,

First of all: QEMU is wonderful. It is blazingly fast, yet very small!

Second, my sincere apologies to Matthew: I hacked together my own version
of what you did. Attached is a patch which does about the same thing.

A few differences, though:

	- I stole a bit from wine, so I have a compile time choice of a
	  keymap. This certainly has room to improve.

	- I actually change the screen resolution. There is support for
	  this in LibVNCServer since version 0.4, and a few clients
	  support it. If the client does not support the change, a warning
	  is issued.

	- It is configurable:
		./configure --enable-vnc
	  will change the output completely to vnc, and
		./configure --enable-vnc-and-sdl
	  will use both simultaneously!

	- It still contains debug messages for every key you hit (it also
	  has those messages in SDL so I can work out why VNC behaves
	  differently).

On Wed, 5 May 2004, Fabrice Bellard wrote:

> Brad Campbell wrote:
> > Matthew Mastracci wrote:
> >> It's a bit of a pain using the mouse over VNC since everything is done
> >> using relative position packets.  It might be possible to emulate a
> >> Synaptics PS/2 touchpad (which supports absolute positioning and is
> >> supported under Windows and X) to make this a bit easier, but I'm not
> >> certain how difficult that would be.  It might be easier to whip up a
> >> special VNC client that supports mouse capturing.  :)

Yes, but the whole idea of VNC is that you can use any client ever written
with every VNC server ever to be written. This is certainly a new concept
in the Microsoft world, where you have to upgrade to the new office
version, because somebody decided to send you a Word document.

> I vote for the Synaptics PS/2 touchpad emulation. From the Linux kernel
> source, it does not seem difficult.

Me, too. But for me, it seems difficult. I have an idea where to start,
but I don't know too many things about mouses and their protocols.

Another idea hit me: The main problem with the mouse seems to be that it's
speed. If I can convince the guest OS to not accelerate, and adjust the
virtual speed so that it matches the real speed, it should be much better.
What do you think?

> Another topic is the keycode conversions. Even with SDL, a solution like
> bochs or rdesktop (with an option to give the host keyboard mapping) is
> needed. That's why I did not merge the various patch submitted to
> 'improve' the SDL keyboard emulation.

As I started, we could continue. We could just use the keymaps the wine
guys collected. (I tried to convince the rdesktop people of the same, but
they didn't bite).

Ciao,
Dscho

[-- Attachment #2: Type: APPLICATION/x-gunzip, Size: 5807 bytes --]

Re: [Qemu-devel] QEMU RFB (vnc) driver

[Johannes Schindelin]

Hi again,

you should always read again your mail before sending! I forgot to mention
that in order to use my patch, you have to have libvncserver-config in
your PATH. Also, preconfigured is the German keyboard layout (as my native
language is German). You can change this in vnc.c by defining
CURRENT_KEY_LAYOUT differently.

<plug type='shameless'>
If things don't work out for you and you suspect your version of
LibVNCServer is the reason: get a new one at
	http://libvncserver.sf.net/LibVNCServer-0.7pre.tar.gz
</plug>

Ciao,
Dscho

[Qemu-devel] Re: QEMU RFB (vnc) driver

[Matthew Mastracci]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]


> I vote for the Synaptics PS/2 touchpad emulation. From the Linux 
> kernel source, it does not seem difficult.
>
> Another topic is the keycode conversions. Even with SDL, a solution 
> like bochs or rdesktop (with an option to give the host keyboard 
> mapping) is needed. That's why I did not merge the various patch 
> submitted to 'improve' the SDL keyboard emulation.
>
It looks like the Synaptics emulation would require an additional driver 
on the Windows side, but Linux/X should support it (and detect it) 
natively.  I wonder how hard it is to write a mouse (or mouse filter 
driver) for Windows to support this sort of device. 

Two other possiblities might be to use the host/guest interface, or to 
implement USB support and create a HID mouse driver with absolute 
positioning support.

Matt.

[-- Attachment #2: matt.vcf --]
[-- Type: text/x-vcard, Size: 286 bytes --]

begin:vcard
fn:Matthew Mastracci
n:Mastracci;Matthew
org:aclaro Softworks, inc.
adr:;;1900 a - 11 St. SE;Calgary;Alberta;T2H 3G2;Canada
email;internet:matt@aclaro.com
title:Software Developer
tel;work:(403) 299-6612
x-mozilla-html:FALSE
url:http://www.aclaro.com
version:2.1
end:vcard

Re: [Qemu-devel] QEMU RFB (vnc) driver

[Brad Campbell]

Johannes Schindelin wrote:
> Hi,
> 
> First of all: QEMU is wonderful. It is blazingly fast, yet very small!
> 
> Second, my sincere apologies to Matthew: I hacked together my own version
> of what you did. Attached is a patch which does about the same thing.
> 
> A few differences, though:
> 
> 	- I stole a bit from wine, so I have a compile time choice of a
> 	  keymap. This certainly has room to improve.
> 
> 	- I actually change the screen resolution. There is support for
> 	  this in LibVNCServer since version 0.4, and a few clients
> 	  support it. If the client does not support the change, a warning
> 	  is issued.

Err. No warning here.

06/05/2004 10:56:02 Got connection from client 192.168.2.80
06/05/2004 10:56:02   other clients:
06/05/2004 10:56:02 Protocol version 3.3
06/05/2004 10:56:02 Pixel format for client 192.168.2.80:
06/05/2004 10:56:02   16 bpp, depth 16, little endian
06/05/2004 10:56:02   true colour: max r 31 g 63 b 31, shift r 11 g 5 b 0
06/05/2004 10:56:02 no translation needed
06/05/2004 10:56:02 Using tight encoding for client 192.168.2.80
06/05/2004 10:56:02 Using compression level 1 for client 192.168.2.80
06/05/2004 10:56:02 Using image quality level 6 for client 192.168.2.80
06/05/2004 10:56:02 Enabling X-style cursor updates for client 192.168.2.80
06/05/2004 10:56:02 Enabling full-color cursor updates for client 192.168.2.80
06/05/2004 10:56:02 Enabling cursor position updates for client 192.168.2.80
06/05/2004 10:56:02 Enabling LastRect protocol extension for client 192.168.2.80
06/05/2004 10:56:22 Warning: Client 192.168.2.80 does not support NewFBSize!
Segmentation fault

I upgraded to a later vnc client that supported NewFBSize and this became a non-issue

> 
> 	- It is configurable:
> 		./configure --enable-vnc
> 	  will change the output completely to vnc, and
> 		./configure --enable-vnc-and-sdl
> 	  will use both simultaneously!

Neato! Only one quirk. --enable-vnc-and-sdl does not compile because it does not compile sdl.c


rm -f libqemu.a
ar rcs libqemu.a exec.o translate-all.o cpu-exec.o translate.o op.o helper.o helper2.o 
translate-copy.o disas.o  i386-dis.o
gcc  -static -Wl,-T,/home/brad/src/qemu/i386-vl.ld  -o qemu-fast vl.o osdep.o block.o monitor.o 
ide.o ne2000.o pckbd.o vga.o sb16.o dma.o oss.o fdc.o mc146818rtc.o serial.o i8259.o i8254.o pc.o 
gdbstub.o vnc.o libqemu.a  -lm  -L/usr/local/libvnc/lib -lvncserver -lnsl -lpthread -ljpeg -lz -lutil
/usr/local/libvnc/lib/libvncserver.a(httpd.o)(.text+0x450): In function `httpProcessInput':
/home/brad/src/LibVNCServer-0.7pre/httpd.c:221: warning: Using 'getpwuid' in statically linked 
applications requires at runtime the shared libraries from the glibc version used for linking
/usr/local/libvnc/lib/libvncserver.a(sockets.o)(.text+0x1017): In function `ConnectToTcpAddr':
/home/brad/src/LibVNCServer-0.7pre/sockets.c:574: warning: Using 'gethostbyname' in statically 
linked applications requires at runtime the shared libraries from the glibc version used for linking
vnc.o(.text+0x40e): In function `vnc_display_init':
/home/brad/src/qemu/vnc.c:164: undefined reference to `sdl_display_init'
collect2: ld returned 1 exit status
make[1]: *** [qemu-fast] Error 1
make[1]: Leaving directory `/home/brad/src/qemu/i386'
make: *** [all] Error 1

I'll keep playing with this one..

Regards,
Brad

Re: [Qemu-devel] Re: QEMU RFB (vnc) driver

[Filip Navara]

----- PŮVODNÍ ZPRÁVA -----
Od: "Matthew Mastracci" <matt@aclaro.com>
Komu: qemu-devel@nongnu.org
Předmět: [Qemu-devel] Re: QEMU RFB (vnc) driver
Datum: 6.5.2004 - 0:21:47

> It looks like the Synaptics emulation would require an
> additional driver 
> on the Windows side, but Linux/X should support it (and
> detect it) 
> natively.  I wonder how hard it is to write a mouse (or
> mouse filter 
> driver) for Windows to support this sort of device. 

It's not hard. I'll write it if Fabrice decides to implement the
QEMU side of the emulation. BTW, will there be an option to
switch between emulating standard PS/2 mouse and this Touchpad?

Regards,
Filip


-- 
Využijte speciální nabídky VOLNÝ ADSL na zkoušku. Za měsíční
paušál a vratnou zálohu máte 30 dní možnost otestovat připojení k
internetu rychlostí 512/128 kbit/s. Více na
http://sluzby.volny.cz/product/adsl/adsl_trial/

[Qemu-devel] Re: QEMU RFB (vnc) driver

[Matthew Mastracci]

[-- Attachment #1: Type: text/plain, Size: 383 bytes --]

Johannes Schindelin wrote:

>First of all: QEMU is wonderful. It is blazingly fast, yet very small!
>
>Second, my sincere apologies to Matthew: I hacked together my own version
>of what you did. Attached is a patch which does about the same thing.
>  
>
No worries - it's good to have others interesting in supporting this 
feature.  I'll give your patch a try myself tonight.

Matt

[-- Attachment #2: matt.vcf --]
[-- Type: text/x-vcard, Size: 286 bytes --]

begin:vcard
fn:Matthew Mastracci
n:Mastracci;Matthew
org:aclaro Softworks, inc.
adr:;;1900 a - 11 St. SE;Calgary;Alberta;T2H 3G2;Canada
email;internet:matt@aclaro.com
title:Software Developer
tel;work:(403) 299-6612
x-mozilla-html:FALSE
url:http://www.aclaro.com
version:2.1
end:vcard

[Qemu-devel] Re: QEMU RFB (vnc) driver

[Matthew Mastracci]

[-- Attachment #1: Type: text/plain, Size: 458 bytes --]

Filip Navara wrote:

>It's not hard. I'll write it if Fabrice decides to implement the
>QEMU side of the emulation. BTW, will there be an option to
>switch between emulating standard PS/2 mouse and this Touchpad?
>
>  
>
The Synaptics touchpad appears as a standard PS/2 mouse until the "magic 
knock sequence" is applied to the PS/2 registers.  Once this has 
happened, it starts reporting more info like absolute X/Y position and 
finger pressure.

Matt.


[-- Attachment #2: matt.vcf --]
[-- Type: text/x-vcard, Size: 286 bytes --]

begin:vcard
fn:Matthew Mastracci
n:Mastracci;Matthew
org:aclaro Softworks, inc.
adr:;;1900 a - 11 St. SE;Calgary;Alberta;T2H 3G2;Canada
email;internet:matt@aclaro.com
title:Software Developer
tel;work:(403) 299-6612
x-mozilla-html:FALSE
url:http://www.aclaro.com
version:2.1
end:vcard

Re: [Qemu-devel] QEMU RFB (vnc) driver

[Fabrice Bellard]

Some comments about your patch:

- The configure option '--enable-vnc-and-sdl' must be suppressed. QEMU 
can handle several display interfaces (it has already two). So it is 
better to add a command line option '-vnc' to QEMU to select the VNC 
server display. Another command line option can be added to have both 
sdl and vnc display. Having configure time options _must_ be avoided as 
most users won't recompile QEMU.

- An option '-k keyb_type' must be added to select the type of keyboard.

Fabrice.

Johannes Schindelin wrote:
> Hi,
> 
> First of all: QEMU is wonderful. It is blazingly fast, yet very small!
> 
> Second, my sincere apologies to Matthew: I hacked together my own version
> of what you did. Attached is a patch which does about the same thing.
> 
> A few differences, though:
> 
> 	- I stole a bit from wine, so I have a compile time choice of a
> 	  keymap. This certainly has room to improve.
> 
> 	- I actually change the screen resolution. There is support for
> 	  this in LibVNCServer since version 0.4, and a few clients
> 	  support it. If the client does not support the change, a warning
> 	  is issued.
> 
> 	- It is configurable:
> 		./configure --enable-vnc
> 	  will change the output completely to vnc, and
> 		./configure --enable-vnc-and-sdl
> 	  will use both simultaneously!
> 
> 	- It still contains debug messages for every key you hit (it also
> 	  has those messages in SDL so I can work out why VNC behaves
> 	  differently).

Re: [Qemu-devel] QEMU RFB (vnc) driver

[Johannes Schindelin]

Hi,

On Sat, 8 May 2004, Fabrice Bellard wrote:
> - The configure option '--enable-vnc-and-sdl' must be suppressed. QEMU
> can handle several display interfaces (it has already two). So it is
> better to add a command line option '-vnc' to QEMU to select the VNC
> server display. Another command line option can be added to have both
> sdl and vnc display. Having configure time options _must_ be avoided as
> most users won't recompile QEMU.

Agree. These settings were only to get it work, and to debug. Whenever SDL
did something VNC couldn't, it was a fine feature to try the other
interface on the same qemu instance at once... I will change the compile
time options to command line options in the next few days.

> - An option '-k keyb_type' must be added to select the type of keyboard.

This I can do. But what about SDL? The keyboard support so far is only for
VNC, because it does not work at all otherwise.

Ciao,
Dscho