Re: [Qemu-devel] [PATCH] Add JPEG encoding to VNC server

[Anthony Liguori <anthony@codemonkey.ws>]

Jamie Lokier wrote:
> Alexander Graf wrote:
>   
>> While this might sound like it renders the whole implementation useless, it
>> does make sense to implement it nevertheless. I have some ideas to implement
>> progressive encodings for video.
>>
>> So when we'd detect that one region is updated a lot in a short
>> about of time with content that zlib can't really handle well, we'd
>> just send a really low quality JPEG first and then send the update
>> after a timer if the region wasn't updated within that timeframe.
>>     
>
> Detecting video regions may also be possible if virtual video overlay
> hardware can be offered.  It would probably require a special guest
> driver.  virtio-video :-)

vmware-vga can do it already.

I don't want to send JPEG, I'd rather send YUV.  The nice thing about an 
overlay is that you get the yuv data directly and you get small update 
regions (which is a feature of vmware-vga).  The nice thing is that you 
get pretty close to mpeg encoding with that.  If a blit is used for tile 
motion, that's even better.

Also, hardware scaling could be coded effectively too.

>  Then again the trend, if you believe in the
> <video> element demos in web browsers, is likely towards OpenGL-driven
> video frame rendering because it's more versatile.
>
> Sending JPEG quality first and a lossless update after is an
> interesting idea, even for non-video.  On a slow link, seeing boxes
> and buttons appear quickly might be quicker to click through a GUI, as
> you can see where to click even if the text and decorations are a bit
> fuzzy for a moment.  Especially if you already know what to expect.
>
> Is it possible to use any of the lossless or "near-lossless" JPEG
> encodings, and would that be any more efficient than VNC's usual
> lossless encoding?
>   

near-lossless is going to result in very large images.  I'm quite sure 
it's worse than even hextile.

OTOH, zlib compressed raw tiles is probably pretty close to as good as 
hextile and it's quite easy to do.  Alex has already done it once for 
the zlib encoding :-)

Regards,

Anthony Liguori