[Qemu-devel] [RFC] qemu-gui based on wxWidgets and libvncclient

[Qemu-devel] [RFC] qemu-gui based on wxWidgets and libvncclient

[Marc André Tanner]

Hi,

During the last few weeks i was playing around with the idea to
implement a GUI for qemu and so this is what i came up with.

Since i am still pretty new to C/C++ development (in fact i would call
this my first real world C++ application) and not familiar with qemu
internals i decided to implement the GUI independent of qemu itself. So
from a technically point of view i just call qemu with arguments to
redirect the monitor to stdio and to export the display to vnc (-monitor
stdio -vnc display). I then try to capture and embed the vnc display
within the GUI that's where libvncclient[1] comes in.

It's this last step which doesn't yet work as expected. It is extremely
slow and/or crashes after some time. But since i am not that experienced
in both multi threaded gui and wxWidgets programming i am stuck.

So i hope someone from the list can help me here, the two most important
files for this issue are vncpanel.cpp and vncdisplaythread.cpp.

My current development snapshot can be found at 
    http://www.brain-dump.org/projects/qemu-gui/doku.php/download
but don't expect it to actually work it's still buggy and in pre alpha
state.

Any comments, ideas, patches and so on are highly appreciated.

At this point i would like to thank Johannes Schindelin for his help
with the integration of libvncclient.

Thanks,

Marc

[1] http://libvncserver.sourceforge.net/

Re: [Qemu-devel] [RFC] qemu-gui based on wxWidgets and libvncclient

[Martin Bochnig]

+1


Marc André Tanner wrote:

>Hi,
>
>During the last few weeks i was playing around with the idea to
>implement a GUI for qemu and so this is what i came up with.
>
>Since i am still pretty new to C/C++ development (in fact i would call
>this my first real world C++ application) and not familiar with qemu
>internals i decided to implement the GUI independent of qemu itself. So
>from a technically point of view i just call qemu with arguments to
>redirect the monitor to stdio and to export the display to vnc (-monitor
>stdio -vnc display). I then try to capture and embed the vnc display
>within the GUI that's where libvncclient[1] comes in.
>
>It's this last step which doesn't yet work as expected. It is extremely
>slow and/or crashes after some time. But since i am not that experienced
>in both multi threaded gui and wxWidgets programming i am stuck.
>
>So i hope someone from the list can help me here, the two most important
>files for this issue are vncpanel.cpp and vncdisplaythread.cpp.
>
>My current development snapshot can be found at 
>    http://www.brain-dump.org/projects/qemu-gui/doku.php/download
>but don't expect it to actually work it's still buggy and in pre alpha
>state.
>
>Any comments, ideas, patches and so on are highly appreciated.
>
>At this point i would like to thank Johannes Schindelin for his help
>with the integration of libvncclient.
>
>Thanks,
>
>Marc
>
>[1] http://libvncserver.sourceforge.net/
>
>
>
>_______________________________________________
>Qemu-devel mailing list
>Qemu-devel@nongnu.org
>http://lists.nongnu.org/mailman/listinfo/qemu-devel
>
>
>  
>

[Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[Anthony Liguori]

Marc André Tanner wrote:
> Hi,
> 
> During the last few weeks i was playing around with the idea to
> implement a GUI for qemu and so this is what i came up with.

Cool.  You should checkout virt-manager.  It has much the same mission 
as what you describe.

> Since i am still pretty new to C/C++ development (in fact i would call
> this my first real world C++ application) and not familiar with qemu
> internals i decided to implement the GUI independent of qemu itself. So
> from a technically point of view i just call qemu with arguments to
> redirect the monitor to stdio and to export the display to vnc (-monitor
> stdio -vnc display). I then try to capture and embed the vnc display
> within the GUI that's where libvncclient[1] comes in.

I have no experience with libvncclient.  I think the general idea of 
using VNC to create an external QEMU GUI is a good one.  Previously, I 
had posted some patches for a shmem GUI that used a custom control 
channel.  After hacking on that for a little bit based on some feedback, 
I've come full circle and am now under the view that extending VNC is a 
better long term approach.

So, I've reserved some pseudo-encodings and a client message type and am 
now working on some VNC extensions to enable better QEMU integration.

My current client code is available at:

http://hg.codemonkey.ws/vnc-gui/

The VNC extensions are still a work in progress but documentation is here:

http://tocm.wikidot.com/vncextensions

Regards,

Anthony Liguori

> It's this last step which doesn't yet work as expected. It is extremely
> slow and/or crashes after some time. But since i am not that experienced
> in both multi threaded gui and wxWidgets programming i am stuck.
> 
> So i hope someone from the list can help me here, the two most important
> files for this issue are vncpanel.cpp and vncdisplaythread.cpp.
> 
> My current development snapshot can be found at 
>     http://www.brain-dump.org/projects/qemu-gui/doku.php/download
> but don't expect it to actually work it's still buggy and in pre alpha
> state.
> 
> Any comments, ideas, patches and so on are highly appreciated.
> 
> At this point i would like to thank Johannes Schindelin for his help
> with the integration of libvncclient.
> 
> Thanks,
> 
> Marc
> 
> [1] http://libvncserver.sourceforge.net/

Re: [Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[Fabrice Bellard]

Anthony Liguori wrote:

> I have no experience with libvncclient.  I think the general idea of 
> using VNC to create an external QEMU GUI is a good one.  Previously, I 
> had posted some patches for a shmem GUI that used a custom control 
> channel.  After hacking on that for a little bit based on some feedback, 
> I've come full circle and am now under the view that extending VNC is a 
> better long term approach.
> 
> So, I've reserved some pseudo-encodings and a client message type and am 
> now working on some VNC extensions to enable better QEMU integration.
> 
> My current client code is available at:
> 
> http://hg.codemonkey.ws/vnc-gui/
> 
> The VNC extensions are still a work in progress but documentation is here:
> 
> http://tocm.wikidot.com/vncextensions

I think this is a good idea. First one detail: for Pointertypechange, it 
would be better to use the value 32768 as 0 instead of 32767 or to use a 
16 bit two complement value.

Another point is that you should consider adding audio support. I can 
help you on that (maybe malc would be interested too !). A simple format 
could be 4 bit ADPCM at fixed frequency. Optionally A more advanced 
codec such as Vorbis could be used.

Regards,

Fabrice.

Re: [Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[Marc André Tanner]

On Sun, 2006-10-15 at 22:56 -0500, Anthony Liguori wrote:
> Marc André Tanner wrote:
> > Hi,
> > 
> > During the last few weeks i was playing around with the idea to
> > implement a GUI for qemu and so this is what i came up with.
> 
> Cool.  You should checkout virt-manager.  It has much the same mission 
> as what you describe.

Ok, i will check it out, what is the state of the libvirt backend for
qemu?

> > Since i am still pretty new to C/C++ development (in fact i would call
> > this my first real world C++ application) and not familiar with qemu
> > internals i decided to implement the GUI independent of qemu itself. So
> > from a technically point of view i just call qemu with arguments to
> > redirect the monitor to stdio and to export the display to vnc (-monitor
> > stdio -vnc display). I then try to capture and embed the vnc display
> > within the GUI that's where libvncclient[1] comes in.
> 
> I have no experience with libvncclient.  I think the general idea of 
> using VNC to create an external QEMU GUI is a good one.  

Yep, although the performance will obviously be a bit slower.

> Previously, I 
> had posted some patches for a shmem GUI that used a custom control 
> channel.  After hacking on that for a little bit based on some feedback, 
> I've come full circle and am now under the view that extending VNC is a 
> better long term approach.
> 
> So, I've reserved some pseudo-encodings and a client message type and am 
> now working on some VNC extensions to enable better QEMU integration.
> 
> My current client code is available at:
> 
> http://hg.codemonkey.ws/vnc-gui/
> 
> The VNC extensions are still a work in progress but documentation is here:
> 
> http://tocm.wikidot.com/vncextensions
> 
> Regards,
> 
> Anthony Liguori

If i find some time, i will take a look at it.

Anyway if someone has comments about my libvncclient based code, please
share them.

Regards,

Marc

Re: [Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[Daniel P. Berrange]

On Tue, Oct 17, 2006 at 11:10:00PM +0200, Marc Andr? Tanner wrote:
> On Sun, 2006-10-15 at 22:56 -0500, Anthony Liguori wrote:
> > Marc André Tanner wrote:
> > > Hi,
> > > 
> > > During the last few weeks i was playing around with the idea to
> > > implement a GUI for qemu and so this is what i came up with.
> > 
> > Cool.  You should checkout virt-manager.  It has much the same mission 
> > as what you describe.
> 
> Ok, i will check it out, what is the state of the libvirt backend for
> qemu?

I did an initial prototype patch to validate the design / approach. This now
needs to be cleaned up to a state suitable for inclusion in libvirt CVS repo.
My goal is to get QEMU support in an official libvirt release within the
next 3-6 months, so that I can make it into Fedora Core 7.

There'll be a little incremental work to virt-manager once the libvirt QEMU
bits are done, but I don't expect that to cover anything difficult - just
tweaking the install wizard to enable the options that are relevant for 
QEMU VMs.

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

[Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[Anthony Liguori]

Fabrice Bellard wrote:
> Anthony Liguori wrote:
> 
>> I have no experience with libvncclient.  I think the general idea of 
>> using VNC to create an external QEMU GUI is a good one.  Previously, I 
>> had posted some patches for a shmem GUI that used a custom control 
>> channel.  After hacking on that for a little bit based on some 
>> feedback, I've come full circle and am now under the view that 
>> extending VNC is a better long term approach.
>>
>> So, I've reserved some pseudo-encodings and a client message type and 
>> am now working on some VNC extensions to enable better QEMU integration.
>>
>> My current client code is available at:
>>
>> http://hg.codemonkey.ws/vnc-gui/
>>
>> The VNC extensions are still a work in progress but documentation is 
>> here:
>>
>> http://tocm.wikidot.com/vncextensions
> 
> I think this is a good idea. First one detail: for Pointertypechange, it 
> would be better to use the value 32768 as 0 instead of 32767 or to use a 
> 16 bit two complement value.

Changing the U16's to S16's is a really good idea.  I'll make the change.

> Another point is that you should consider adding audio support. I can 
> help you on that (maybe malc would be interested too !). A simple format 
> could be 4 bit ADPCM at fixed frequency. Optionally A more advanced 
> codec such as Vorbis could be used.

I know very little about audio so help would be appreciated.  I
currently have 16 pseudo-encodings and a client/server message reserved
so we have ample room to extend VNC.

Regards,

Anthony Liguori

> Regards,
> 
> Fabrice.

Re: [Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[Johannes Schindelin]

Hi,

On Tue, 17 Oct 2006, Fabrice Bellard wrote:

> Another point is that you should consider adding audio support. I can 
> help you on that (maybe malc would be interested too !). A simple format 
> could be 4 bit ADPCM at fixed frequency. Optionally A more advanced 
> codec such as Vorbis could be used.

Please, please not _another_ vnc hack! The RFB protocol loses all of its 
appeal if everybody has proprietary, incompatible extensions! If you 
bother to search Google for prior art, here is a link:

http://www.ks.uni-freiburg.de/php_arbeitdet.php?id=75

Yes, it is in German, but it already works. No need to add yet another 
obscure extension.

Ciao,
Dscho

Re: [Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[malc]

On Wed, 18 Oct 2006, Johannes Schindelin wrote:

> Hi,
>
> On Tue, 17 Oct 2006, Fabrice Bellard wrote:
>
>> Another point is that you should consider adding audio support. I can
>> help you on that (maybe malc would be interested too !). A simple format
>> could be 4 bit ADPCM at fixed frequency. Optionally A more advanced
>> codec such as Vorbis could be used.

I am certainly ready to give a helping hand to anyone who attempts
something to that effect.

>
> Please, please not _another_ vnc hack! The RFB protocol loses all of its
> appeal if everybody has proprietary, incompatible extensions! If you
> bother to search Google for prior art, here is a link:
>
> http://www.ks.uni-freiburg.de/php_arbeitdet.php?id=75
>
> Yes, it is in German, but it already works. No need to add yet another
> obscure extension.

Schade das mein Deutsch ...ehh... isn't basically up to scratch.

--
vale

[Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[Anthony Liguori]

Johannes Schindelin wrote:
> Hi,
> 
> On Tue, 17 Oct 2006, Fabrice Bellard wrote:
> 
>> Another point is that you should consider adding audio support. I can 
>> help you on that (maybe malc would be interested too !). A simple format 
>> could be 4 bit ADPCM at fixed frequency. Optionally A more advanced 
>> codec such as Vorbis could be used.
> 
> Please, please not _another_ vnc hack! The RFB protocol loses all of its 
> appeal if everybody has proprietary, incompatible extensions! If you 
> bother to search Google for prior art, here is a link:
> 
> http://www.ks.uni-freiburg.de/php_arbeitdet.php?id=75
> 
> Yes, it is in German, but it already works. No need to add yet another 
> obscure extension.

My German is not very good, but from what I was able to read, it seems 
like they are just using VNC to transmit the port information for doing 
esd forwarding.  Presumably this saved a lot of work because no server 
side code was needed.

Regards,

Anthony Liguori

> Ciao,
> Dscho

Re: [Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[Oliver Gerlich]

Here's a partial translation/explanation of the link that Johannes sent 
some days ago, concerning remote sound in VNC. The final documentation 
(in german) can be found at 
http://www.ks.uni-freiburg.de/download/studienarbeit/SS05/08-05-TonSpur-DRichter/Dokumentation/Dokumentation.pdf

According to that PDF, sound is transmitted by capturing it with a Java 
application, then converting it using JMF (Java Media Framework) and 
sending it out as RTP stream. A Java client receives the RTP stream and 
plays it. As far as I understand, handshaking is done with an own TCP 
connection, while actual RTP packets are sent via UDP. During 
handshaking, connection speed is measured (see 6.1.2) to decide on the 
encoding (see 6.2.2).

There's also an older document at 
http://www.ks.uni-freiburg.de/download/studienarbeit/WS03/Ton-Spur-VNC.pdf 
which indeed describes a way to use ESD for remote sound.


Regards,
Oliver


Anthony Liguori wrote:
> Johannes Schindelin wrote:
>> Hi,
>>
>> On Tue, 17 Oct 2006, Fabrice Bellard wrote:
>>
>>> Another point is that you should consider adding audio support. I can 
>>> help you on that (maybe malc would be interested too !). A simple 
>>> format could be 4 bit ADPCM at fixed frequency. Optionally A more 
>>> advanced codec such as Vorbis could be used.
>>
>> Please, please not _another_ vnc hack! The RFB protocol loses all of 
>> its appeal if everybody has proprietary, incompatible extensions! If 
>> you bother to search Google for prior art, here is a link:
>>
>> http://www.ks.uni-freiburg.de/php_arbeitdet.php?id=75
>>
>> Yes, it is in German, but it already works. No need to add yet another 
>> obscure extension.
> 
> My German is not very good, but from what I was able to read, it seems 
> like they are just using VNC to transmit the port information for doing 
> esd forwarding.  Presumably this saved a lot of work because no server 
> side code was needed.
> 
> Regards,
> 
> Anthony Liguori
> 
>> Ciao,
>> Dscho
> 
> 
> 
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel
> 
> 

Re: [Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[Jorge Luis Zapata Muga]

On 10/18/06, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Tue, 17 Oct 2006, Fabrice Bellard wrote:
>
> > Another point is that you should consider adding audio support. I can
> > help you on that (maybe malc would be interested too !). A simple format
> > could be 4 bit ADPCM at fixed frequency. Optionally A more advanced
> > codec such as Vorbis could be used.
>
> Please, please not _another_ vnc hack! The RFB protocol loses all of its
> appeal if everybody has proprietary, incompatible extensions! If you
> bother to search Google for prior art, here is a link:
>
> http://www.ks.uni-freiburg.de/php_arbeitdet.php?id=75
>
> Yes, it is in German, but it already works. No need to add yet another
> obscure extension.
>

Hello all,

Does qemu actually needs a vnc hacked backend to support a gui version
of it? I think a better/cleaner solution is what it is proposed here:
http://lists.gnu.org/archive/html/qemu-devel/2005-02/msg00075.html
as it will let developers build their own frontend, of course qemu
could have its own "official" gui in the main source tree.


> Ciao,
> Dscho
>
>
>
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel
>

Re: [Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[Fabrice Bellard]

Anthony Liguori wrote:
> Johannes Schindelin wrote:
> 
>> Hi,
>>
>> On Tue, 17 Oct 2006, Fabrice Bellard wrote:
>>
>>> Another point is that you should consider adding audio support. I can 
>>> help you on that (maybe malc would be interested too !). A simple 
>>> format could be 4 bit ADPCM at fixed frequency. Optionally A more 
>>> advanced codec such as Vorbis could be used.
>>
>>
>> Please, please not _another_ vnc hack! The RFB protocol loses all of 
>> its appeal if everybody has proprietary, incompatible extensions! If 
>> you bother to search Google for prior art, here is a link:
>>
>> http://www.ks.uni-freiburg.de/php_arbeitdet.php?id=75
>>
>> Yes, it is in German, but it already works. No need to add yet another 
>> obscure extension.
> 
> 
> My German is not very good, but from what I was able to read, it seems 
> like they are just using VNC to transmit the port information for doing 
> esd forwarding.  Presumably this saved a lot of work because no server 
> side code was needed.

I would prefer a self contained solution using the same TCP connection 
as the rest of the protocol.

Regards,

Fabrice.

Re: [Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[Stefan Weil]

>>>
>>> Please, please not _another_ vnc hack! The RFB protocol loses all of
>>> its appeal if everybody has proprietary, incompatible extensions! If
>>> you bother to search Google for prior art, here is a link:
>>>
>>> http://www.ks.uni-freiburg.de/php_arbeitdet.php?id=75
>>>
>>> Yes, it is in German, but it already works. No need to add yet
>>> another obscure extension.
>>
>>
>> My German is not very good, but from what I was able to read, it
>> seems like they are just using VNC to transmit the port information
>> for doing esd forwarding.  Presumably this saved a lot of work
>> because no server side code was needed.
>
> I would prefer a self contained solution using the same TCP connection
> as the rest of the protocol.
>
> Regards,
>
> Fabrice.
>
>
>
Video and audio data have different needs, so two connections
would allow one transfering real-time data (maybe without error
correction) for audio, the other can transfer VNC video data
(no need for high priority real-time, but for error free transmission).

Of course, sometimes a single connection makes things much easier
(for example behind a firewall)...

Regards,

Stefan

VNC audio extension, was Re: [Qemu-devel] Re: [RFC] qemu-gui based on wxWidgets and libvncclient

[Johannes Schindelin]

Hi,

On Wed, 18 Oct 2006, Anthony Liguori wrote:

> Johannes Schindelin wrote:
> > Hi,
> > 
> > On Tue, 17 Oct 2006, Fabrice Bellard wrote:
> > 
> > > Another point is that you should consider adding audio support. I can help
> > > you on that (maybe malc would be interested too !). A simple format could
> > > be 4 bit ADPCM at fixed frequency. Optionally A more advanced codec such
> > > as Vorbis could be used.
> > 
> > Please, please not _another_ vnc hack! The RFB protocol loses all of its
> > appeal if everybody has proprietary, incompatible extensions! If you bother
> > to search Google for prior art, here is a link:
> > 
> > http://www.ks.uni-freiburg.de/php_arbeitdet.php?id=75
> > 
> > Yes, it is in German, but it already works. No need to add yet another
> > obscure extension.
> 
> My German is not very good, but from what I was able to read, it seems like
> they are just using VNC to transmit the port information for doing esd
> forwarding.  Presumably this saved a lot of work because no server side code
> was needed.

Actually, it does not use ESD forwarding. The ESD part was meant for audio 
_capture_, since the project was to transport also audio from a real 
computer.

But you are correct about the opening a port part. It uses RTP, not TCP, 
to ensure that the latency is low (which makes sense). The essence of this 
work is (AFAIUI) that

	- Java is not the right language to do the job (which I don't 
	  agree with), and
	- it makes no sense to integrate the data stream into the VNC
	  data stream, since you cannot sensibly multiplex the video
	  and audio stream (because they have different goals).

Hth,
Dscho