[Qemu-devel] Sensible VNC encodings

[Qemu-devel] Sensible VNC encodings

[Johannes Schindelin]

Hi,

for those interested, I finally broke down and updated my personal fork of 
QEmu using LibVNCServer.  Amongst the benefits are Tight, ZRLE and ZYWRLE 
encodings.  Note that TLS is not supported with LibVNCServer, but nobody 
in her right mind uses VNC without an SSH tunnel anyway.

Fortunately, Git helps me tremendously in maintaining my fork, since I do 
not expect the mainline to have any usable VNC support anytime soon.  If 
you're interested, look at the "vnc" branch in 
http://repo.or.cz/w/qemu/dscho.git/

Ciao,
Dscho

Re: [Qemu-devel] Sensible VNC encodings

[Anthony Liguori]

Johannes Schindelin wrote:
> Hi,
>
> for those interested, I finally broke down and updated my personal fork of 
> QEmu using LibVNCServer.  Amongst the benefits are Tight, ZRLE and ZYWRLE 
> encodings.  Note that TLS is not supported with LibVNCServer, but nobody 
> in her right mind uses VNC without an SSH tunnel anyway.
>   

It's funny that you say that because in my observations, hextile + an 
compressed SSH tunnel does just about as well as Tight.

Regards,

Anthony Liguori

> Fortunately, Git helps me tremendously in maintaining my fork, since I do 
> not expect the mainline to have any usable VNC support anytime soon.  If 
> you're interested, look at the "vnc" branch in 
> http://repo.or.cz/w/qemu/dscho.git/
>
> Ciao,
> Dscho
>
>
>
>   

Re: [Qemu-devel] Sensible VNC encodings

[Alexander Graf]

On May 21, 2008, at 3:20 AM, Johannes Schindelin wrote:

> Hi,
>
> for those interested, I finally broke down and updated my personal  
> fork of
> QEmu using LibVNCServer.  Amongst the benefits are Tight, ZRLE and  
> ZYWRLE
> encodings.  Note that TLS is not supported with LibVNCServer, but  
> nobody
> in her right mind uses VNC without an SSH tunnel anyway.

It might be a good idea to have this as an additional output, not as a  
replacement for the VNC support as it is now. Think of a system, where  
libvncserver is not available.

Another thing that came to my mind while reading this was librdp. It  
should be easily feasible to make qemu support RDP with that.

So what we really need would be a "video output" command line option,  
similar to the -vga or -display switch that was suggested earlier, but  
that handles the host's video output. This way you could just plug  
your libvncserver based output in and maybe even have a chance to get  
that upstream, as it does not break anything.

I personally prefer reusage of code, so the general idea of using  
libvncserver is great!

Alex

>
>
> Fortunately, Git helps me tremendously in maintaining my fork, since  
> I do
> not expect the mainline to have any usable VNC support anytime  
> soon.  If
> you're interested, look at the "vnc" branch in
> http://repo.or.cz/w/qemu/dscho.git/
>
> Ciao,
> Dscho
>
>
>

Re: [Qemu-devel] Sensible VNC encodings

[Johannes Schindelin]

Hi,

On Wed, 21 May 2008, Alexander Graf wrote:

> On May 21, 2008, at 3:20 AM, Johannes Schindelin wrote:
> 
> >for those interested, I finally broke down and updated my personal fork 
> >of QEmu using LibVNCServer.  Amongst the benefits are Tight, ZRLE and 
> >ZYWRLE encodings.  Note that TLS is not supported with LibVNCServer, 
> >but nobody in her right mind uses VNC without an SSH tunnel anyway.
> 
> It might be a good idea to have this as an additional output, not as a 
> replacement for the VNC support as it is now. Think of a system, where 
> libvncserver is not available.

I am long past pushing for inclusion.

> Another thing that came to my mind while reading this was librdp. It 
> should be easily feasible to make qemu support RDP with that.

That was my original plan with LibVNCServer, merging it with xrdp.  
Unfortunately, the xrdp made some unfortunate design decisions which made 
this next to impossible.

Ciao,
Dscho

Re: [Qemu-devel] Sensible VNC encodings

[Johannes Schindelin]

Hi,

On Tue, 20 May 2008, Anthony Liguori wrote:

> Johannes Schindelin wrote:
>
> > for those interested, I finally broke down and updated my personal 
> > fork of QEmu using LibVNCServer.  Amongst the benefits are Tight, ZRLE 
> > and ZYWRLE encodings.  Note that TLS is not supported with 
> > LibVNCServer, but nobody in her right mind uses VNC without an SSH 
> > tunnel anyway.
> 
> It's funny that you say that because in my observations, hextile + an 
> compressed SSH tunnel does just about as well as Tight.

Clearly, you have never waited one minute for a full screen update from a 
machine half around the planet.

That is what happens here with hextile + an compressed SSH tunnel, just in 
case you thought this was pure speculation.

It is pretty obvious why, too: generic compression, such as SSH's, will 
never outperform specialized compression, such as tight compression.

Ciao,
Dscho

Re: [Qemu-devel] Sensible VNC encodings

[Daniel P. Berrange]

On Wed, May 21, 2008 at 07:55:37AM +0200, Alexander Graf wrote:
> 
> On May 21, 2008, at 3:20 AM, Johannes Schindelin wrote:
> 
> >Hi,
> >
> >for those interested, I finally broke down and updated my personal  
> >fork of
> >QEmu using LibVNCServer.  Amongst the benefits are Tight, ZRLE and  
> >ZYWRLE
> >encodings.  Note that TLS is not supported with LibVNCServer, but  
> >nobody
> >in her right mind uses VNC without an SSH tunnel anyway.
> 
> It might be a good idea to have this as an additional output, not as a  
> replacement for the VNC support as it is now. Think of a system, where  
> libvncserver is not available.

[snip]

> I personally prefer reusage of code, so the general idea of using  
> libvncserver is great!

My experiance with libvncserver has not been at all pleasant, finding it
to be seriously crash happy and near impossible to debug, particuarly
with its multithread code. We used to use it in Xen for the paravirt
framebuffer server but because of its unreliability, libvncserver was
ripped out and now Xen hooks into the QEMU VNC server code instead.
While QEMU's existing code may not support as many encodings, it is it
clearly written, easy to debug and doesn't crash. If there's more 
encodings we need to support for performance then IMHO they should be
added to the existing code. 

Dan.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] Sensible VNC encodings

[Anthony Liguori]

Johannes Schindelin wrote:
> Hi,
>
> On Tue, 20 May 2008, Anthony Liguori wrote:
>
> Clearly, you have never waited one minute for a full screen update from a 
> machine half around the planet.
>
> That is what happens here with hextile + an compressed SSH tunnel, just in 
> case you thought this was pure speculation.
>
> It is pretty obvious why, too: generic compression, such as SSH's, will 
> never outperform specialized compression, such as tight compression.
>   

Tight uses zlib compression in combination with a palette mechanism.  
Hextile is a palette mechanism so adding zlib compression to it (via 
ssh) makes it pretty comparable to Tight.

Regards,

Anthony Liguori

> Ciao,
> Dscho
>
>   

Re: [Qemu-devel] Sensible VNC encodings

[Johannes Schindelin]

Hi,

On Wed, 21 May 2008, Daniel P. Berrange wrote:

> My experiance with libvncserver has not been at all pleasant, finding it 
> to be seriously crash happy and near impossible to debug, particuarly 
> with its multithread code. We used to use it in Xen for the paravirt 
> framebuffer server but because of its unreliability, libvncserver was 
> ripped out and now Xen hooks into the QEMU VNC server code instead.

Of course, you are free to do what you want to do.  But I have to wonder 
why I, the maintainer of LibVNCServer, did not know a _thing_ about this 
before you wrote this, and you did not even direct that mail to me.  Seems 
you were not serious about wanting these bugs fixed.

Sheesh,
Dscho

Re: [Qemu-devel] Sensible VNC encodings

[Johannes Schindelin]

Hi,

On Wed, 21 May 2008, Anthony Liguori wrote:

> Johannes Schindelin wrote:
>
> > On Tue, 20 May 2008, Anthony Liguori wrote:
> >
> > Clearly, you have never waited one minute for a full screen update 
> > from a machine half around the planet.
> >
> > That is what happens here with hextile + an compressed SSH tunnel, 
> > just in case you thought this was pure speculation.
> >
> > It is pretty obvious why, too: generic compression, such as SSH's, 
> > will never outperform specialized compression, such as tight 
> > compression.
> 
> Tight uses zlib compression in combination with a palette mechanism.  
> Hextile is a palette mechanism so adding zlib compression to it (via 
> ssh) makes it pretty comparable to Tight.

I'm awfully short on time, so this is only a hint to the explanation:

Tight compression can fall back to a Jpeg scheme which is lossy.  You 
obviously did not know that, and obviously this is what I have to use to 
make VNC usable.

Hth,
Dscho

Re: [Qemu-devel] Sensible VNC encodings

[Anthony Liguori]

Johannes Schindelin wrote:
> Hi,
>
> On Wed, 21 May 2008, Anthony Liguori wrote:
>
>   
>> Johannes Schindelin wrote:
>>
>>     
>>> On Tue, 20 May 2008, Anthony Liguori wrote:
>>>
>>> Clearly, you have never waited one minute for a full screen update 
>>> from a machine half around the planet.
>>>
>>> That is what happens here with hextile + an compressed SSH tunnel, 
>>> just in case you thought this was pure speculation.
>>>
>>> It is pretty obvious why, too: generic compression, such as SSH's, 
>>> will never outperform specialized compression, such as tight 
>>> compression.
>>>       
>> Tight uses zlib compression in combination with a palette mechanism.  
>> Hextile is a palette mechanism so adding zlib compression to it (via 
>> ssh) makes it pretty comparable to Tight.
>>     
>
> I'm awfully short on time, so this is only a hint to the explanation:
>
> Tight compression can fall back to a Jpeg scheme which is lossy.  You 
> obviously did not know that, and obviously this is what I have to use to 
> make VNC usable.
>   

I clearly did seeing that I implemented Tight compression support in 
gtk-vnc.  The jpeg compression is optional and requires the use of 
additional pseudo-encodings to activate it.

I really don't like the idea of using a lossy encoding though.  I think 
you would find that the builtin VNC + compression over SSH does better 
than Tight + jpeg because of the fact that in the builtin VNC server, we 
reduce the size of the update regions before encoding.  libvncserver 
always has to operate at scanline granularities so it's update regions 
are much larger.

Regards,

Anthony Liguori

> Hth,
> Dscho
>
>   

Re: [Qemu-devel] Sensible VNC encodings

[Jamie Lokier]

Anthony Liguori wrote:
> I really don't like the idea of using a lossy encoding though.

Lossy encoding sounds very useful if you're on a very slow link and it
would take 30s to display a major update even with the most brilliant
of lossless compressions.  Having a blurry, progressively updated
image until it's exact would sometimes be useful.

-- Jamie

Re: [Qemu-devel] Sensible VNC encodings

[Anthony Liguori]

Johannes Schindelin wrote:
> Hi,
>
> On Wed, 21 May 2008, Daniel P. Berrange wrote:
>
>   
>> My experiance with libvncserver has not been at all pleasant, finding it 
>> to be seriously crash happy and near impossible to debug, particuarly 
>> with its multithread code. We used to use it in Xen for the paravirt 
>> framebuffer server but because of its unreliability, libvncserver was 
>> ripped out and now Xen hooks into the QEMU VNC server code instead.
>>     
>
> Of course, you are free to do what you want to do.  But I have to wonder 
> why I, the maintainer of LibVNCServer, did not know a _thing_ about this 
> before you wrote this, and you did not even direct that mail to me.  Seems 
> you were not serious about wanting these bugs fixed.
>   

You should have as I've told you this repeatedly.  I wrote the original 
vncfb (and this was *after* having written the QEMU VNC server) that 
used libvncserver.  Our experiences with it have been very unpleasant.  
The update regions are too big making mouse movement very slow.  The 
locking is terribly incorrect.  The problem with libvncserver is it's 
not an actual library.  You took the standalone VNC server and attempted 
to fit it into a library--but it was never designed for that.

You forked RealVNC, made a crappy library out of it, and now are 
complaining here because people aren't flocking to your forked copy of 
RealVNC.  Please, stop trolling qemu-devel about this and stop sending 
me private flames about it.  This issue was pointless to discuss years ago.

Regards,

Anthony Liguori

> Sheesh,
> Dscho
>
>
>
>   

Re: [Qemu-devel] Sensible VNC encodings

[Anthony Liguori]

Jamie Lokier wrote:
> Anthony Liguori wrote:
>   
>> I really don't like the idea of using a lossy encoding though.
>>     
>
> Lossy encoding sounds very useful if you're on a very slow link and it
> would take 30s to display a major update even with the most brilliant
> of lossless compressions.  Having a blurry, progressively updated
> image until it's exact would sometimes be useful.
>   

VNC has a builtin notion of "lossy" encoding via SetPixelFormat.  You 
can use it to reduce the overall color depth.  Hextile et. al. are also 
paletted encodings so I doubt there are a lot of circumstances where 
jpeg compression really saves you anything.

If someone wants to actually do some experiments and can post numbers 
where jpeg compression provides the best bandwidth/quality, I'd be happy 
to implement it.  FWIW, Tight is not part of the RFB spec.  ZRLE (which 
is meant to replace it) does not support JPEG encoding at all.

Regards,

Anthony Liguori

> -- Jamie
>
>
>