* [Qemu-devel] [PATCH v2 00/10] vnc-encodings
@ 2010-05-18 12:49 Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 01/10] vnc: refactor set_encodings Corentin Chary
` (9 more replies)
0 siblings, 10 replies; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 12:49 UTC (permalink / raw)
To: qemu-devel; +Cc: Corentin Chary, Anthony Liguori, Alexander Graf, Adam Litke
Hi,
I this series you'll find zlib fixes and a new encoding: tight.
Tight implementation is inspired (stolen ?) from libvncserver/tightvnc/tigervnc,
but rewritten to match QEMU VNC implementation and coding style.
Tight encoding still lacks gradient and jpeg, but some results [1] [2] show
that it already performs better than zlib and hextile.
Thanks,
[1] http://xf.iksaif.net/blog/index.php?post/2010/05/11/GsoC-2010-QEMU%3A-First-%28dumb%29-tight-benchmarks
[2] http://xf.iksaif.net/blog/index.php?post/2010/05/18/QEMU%3A-some-charts-on-tight-zlib-and-hextile
Since v1: added commit descriptions for all patchs
Corentin Chary (10):
vnc: refactor set_encodings
vnc: really call zlib if we want zlib
vnc: only use a single zlib stream
vnc: adjust compression zstream level
vnc: add basic tight support
vnc: add support for tight fill encoding
vnc: don't clear zlib stream on set_encoding
vnc: remove memory leaks in zlib and tight encoding
vnc: tight: add palette encoding
vnc: update copyrights for vnc-encoding-tight.c
Makefile | 2 +
Makefile.objs | 1 +
vnc-encoding-hextile.c | 5 +-
vnc-encoding-tight.c | 945 ++++++++++++++++++++++++++++++++++++++++++++++++
vnc-encoding-tight.h | 199 ++++++++++
vnc-encoding-zlib.c | 43 ++-
vnc.c | 79 +++--
vnc.h | 24 +-
8 files changed, 1245 insertions(+), 53 deletions(-)
create mode 100644 vnc-encoding-tight.c
create mode 100644 vnc-encoding-tight.h
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] [PATCH v2 01/10] vnc: refactor set_encodings
2010-05-18 12:49 [Qemu-devel] [PATCH v2 00/10] vnc-encodings Corentin Chary
@ 2010-05-18 12:49 ` Corentin Chary
2010-05-18 12:54 ` [Qemu-devel] " Alexander Graf
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 02/10] vnc: really call zlib if we want zlib Corentin Chary
` (8 subsequent siblings)
9 siblings, 1 reply; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 12:49 UTC (permalink / raw)
To: qemu-devel; +Cc: Corentin Chary, Anthony Liguori, Alexander Graf, Adam Litke
Create a new set_encoding() function to remove
duplicated code in set_encodings().
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
vnc.c | 19 ++++++++++---------
1 files changed, 10 insertions(+), 9 deletions(-)
diff --git a/vnc.c b/vnc.c
index 1f7ad73..a91c3a3 100644
--- a/vnc.c
+++ b/vnc.c
@@ -1587,6 +1587,13 @@ static void send_ext_audio_ack(VncState *vs)
vnc_flush(vs);
}
+static void set_encoding(VncState *vs, int encoding)
+{
+ if (vs->vnc_encoding == -1) {
+ vs->vnc_encoding = encoding;
+ }
+}
+
static void set_encodings(VncState *vs, int32_t *encodings, size_t n_encodings)
{
int i;
@@ -1603,24 +1610,18 @@ static void set_encodings(VncState *vs, int32_t *encodings, size_t n_encodings)
enc = encodings[i];
switch (enc) {
case VNC_ENCODING_RAW:
- if (vs->vnc_encoding != -1) {
- vs->vnc_encoding = enc;
- }
+ set_encoding(vs, enc);
break;
case VNC_ENCODING_COPYRECT:
vs->features |= VNC_FEATURE_COPYRECT_MASK;
break;
case VNC_ENCODING_HEXTILE:
vs->features |= VNC_FEATURE_HEXTILE_MASK;
- if (vs->vnc_encoding != -1) {
- vs->vnc_encoding = enc;
- }
+ set_encoding(vs, enc);
break;
case VNC_ENCODING_ZLIB:
vs->features |= VNC_FEATURE_ZLIB_MASK;
- if (vs->vnc_encoding != -1) {
- vs->vnc_encoding = enc;
- }
+ set_encoding(vs, enc);
break;
case VNC_ENCODING_DESKTOPRESIZE:
vs->features |= VNC_FEATURE_RESIZE_MASK;
--
1.7.0.2
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [Qemu-devel] [PATCH v2 02/10] vnc: really call zlib if we want zlib
2010-05-18 12:49 [Qemu-devel] [PATCH v2 00/10] vnc-encodings Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 01/10] vnc: refactor set_encodings Corentin Chary
@ 2010-05-18 12:49 ` Corentin Chary
2010-05-18 12:54 ` [Qemu-devel] " Alexander Graf
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 03/10] vnc: only use a single zlib stream Corentin Chary
` (7 subsequent siblings)
9 siblings, 1 reply; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 12:49 UTC (permalink / raw)
To: qemu-devel; +Cc: Corentin Chary, Anthony Liguori, Alexander Graf, Adam Litke
send_framebuffer_update() was calling hextile instead of zlib
since commit 70a4568fe0c5a64adaa3da5030b7109e5199e692.
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
vnc.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/vnc.c b/vnc.c
index a91c3a3..d0c0d00 100644
--- a/vnc.c
+++ b/vnc.c
@@ -655,7 +655,7 @@ static void send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
{
switch(vs->vnc_encoding) {
case VNC_ENCODING_ZLIB:
- vnc_hextile_send_framebuffer_update(vs, x, y, w, h);
+ vnc_zlib_send_framebuffer_update(vs, x, y, w, h);
break;
case VNC_ENCODING_HEXTILE:
vnc_framebuffer_update(vs, x, y, w, h, VNC_ENCODING_HEXTILE);
--
1.7.0.2
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [Qemu-devel] [PATCH v2 03/10] vnc: only use a single zlib stream
2010-05-18 12:49 [Qemu-devel] [PATCH v2 00/10] vnc-encodings Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 01/10] vnc: refactor set_encodings Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 02/10] vnc: really call zlib if we want zlib Corentin Chary
@ 2010-05-18 12:49 ` Corentin Chary
2010-05-18 12:51 ` [Qemu-devel] " Alexander Graf
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 04/10] vnc: adjust compression zstream level Corentin Chary
` (6 subsequent siblings)
9 siblings, 1 reply; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 12:49 UTC (permalink / raw)
To: qemu-devel; +Cc: Corentin Chary, Anthony Liguori, Alexander Graf, Adam Litke
According to http://tigervnc.org/cgi-bin/rfbproto#zlib-encoding
zlib encoding only uses a single stream. Current implementation defines
4 streams but only uses the first one. Remove them and only use a single
stream.
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
vnc-encoding-zlib.c | 12 +++++-------
vnc.h | 2 +-
2 files changed, 6 insertions(+), 8 deletions(-)
diff --git a/vnc-encoding-zlib.c b/vnc-encoding-zlib.c
index 4a495ad..6a16a79 100644
--- a/vnc-encoding-zlib.c
+++ b/vnc-encoding-zlib.c
@@ -54,9 +54,9 @@ static void vnc_zlib_start(VncState *vs)
vs->output = vs->zlib;
}
-static int vnc_zlib_stop(VncState *vs, int stream_id)
+static int vnc_zlib_stop(VncState *vs)
{
- z_streamp zstream = &vs->zlib_stream[stream_id];
+ z_streamp zstream = &vs->zlib_stream;
int previous_out;
// switch back to normal output/zlib buffers
@@ -70,7 +70,7 @@ static int vnc_zlib_stop(VncState *vs, int stream_id)
if (zstream->opaque != vs) {
int err;
- VNC_DEBUG("VNC: initializing zlib stream %d\n", stream_id);
+ VNC_DEBUG("VNC: initializing zlib stream\n");
VNC_DEBUG("VNC: opaque = %p | vs = %p\n", zstream->opaque, vs);
zstream->zalloc = zalloc;
zstream->zfree = zfree;
@@ -122,7 +122,7 @@ void vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
// compress the stream
vnc_zlib_start(vs);
vnc_raw_send_framebuffer_update(vs, x, y, w, h);
- bytes_written = vnc_zlib_stop(vs, 0);
+ bytes_written = vnc_zlib_stop(vs);
if (bytes_written == -1)
return;
@@ -136,7 +136,5 @@ void vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
void vnc_zlib_init(VncState *vs)
{
- int i;
- for (i=0; i<(sizeof(vs->zlib_stream) / sizeof(z_stream)); i++)
- vs->zlib_stream[i].opaque = NULL;
+ vs->zlib_stream.opaque = NULL;
}
diff --git a/vnc.h b/vnc.h
index 1aa71b0..dfdb240 100644
--- a/vnc.h
+++ b/vnc.h
@@ -173,7 +173,7 @@ struct VncState
/* Zlib */
Buffer zlib;
Buffer zlib_tmp;
- z_stream zlib_stream[4];
+ z_stream zlib_stream;
Notifier mouse_mode_notifier;
--
1.7.0.2
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [Qemu-devel] [PATCH v2 04/10] vnc: adjust compression zstream level
2010-05-18 12:49 [Qemu-devel] [PATCH v2 00/10] vnc-encodings Corentin Chary
` (2 preceding siblings ...)
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 03/10] vnc: only use a single zlib stream Corentin Chary
@ 2010-05-18 12:49 ` Corentin Chary
2010-05-18 12:55 ` [Qemu-devel] " Alexander Graf
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 05/10] vnc: add basic tight support Corentin Chary
` (5 subsequent siblings)
9 siblings, 1 reply; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 12:49 UTC (permalink / raw)
To: qemu-devel; +Cc: Corentin Chary, Anthony Liguori, Alexander Graf, Adam Litke
Adjust zlib compression level if needed by calling deflateParams.
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
vnc-encoding-zlib.c | 9 ++++++++-
vnc.h | 1 +
2 files changed, 9 insertions(+), 1 deletions(-)
diff --git a/vnc-encoding-zlib.c b/vnc-encoding-zlib.c
index 6a16a79..29dd1b7 100644
--- a/vnc-encoding-zlib.c
+++ b/vnc-encoding-zlib.c
@@ -83,10 +83,17 @@ static int vnc_zlib_stop(VncState *vs)
return -1;
}
+ vs->zlib_level = vs->tight_compression;
zstream->opaque = vs;
}
- // XXX what to do if tight_compression changed in between?
+ if (vs->tight_compression != vs->zlib_level) {
+ if (deflateParams(zstream, vs->tight_compression,
+ Z_DEFAULT_STRATEGY) != Z_OK) {
+ return -1;
+ }
+ vs->zlib_level = vs->tight_compression;
+ }
// reserve memory in output buffer
buffer_reserve(&vs->output, vs->zlib.offset + 64);
diff --git a/vnc.h b/vnc.h
index dfdb240..96f3fe7 100644
--- a/vnc.h
+++ b/vnc.h
@@ -174,6 +174,7 @@ struct VncState
Buffer zlib;
Buffer zlib_tmp;
z_stream zlib_stream;
+ int zlib_level;
Notifier mouse_mode_notifier;
--
1.7.0.2
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [Qemu-devel] [PATCH v2 05/10] vnc: add basic tight support
2010-05-18 12:49 [Qemu-devel] [PATCH v2 00/10] vnc-encodings Corentin Chary
` (3 preceding siblings ...)
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 04/10] vnc: adjust compression zstream level Corentin Chary
@ 2010-05-18 12:49 ` Corentin Chary
2010-05-18 13:33 ` [Qemu-devel] " Anthony Liguori
2010-05-18 13:49 ` Alexander Graf
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 06/10] vnc: add support for tight fill encoding Corentin Chary
` (4 subsequent siblings)
9 siblings, 2 replies; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 12:49 UTC (permalink / raw)
To: qemu-devel; +Cc: Corentin Chary, Anthony Liguori, Alexander Graf, Adam Litke
Add support for tight encoding [1]. This patch only add support
for "basic" tight compression without any filter.
[1] http://tigervnc.org/cgi-bin/rfbproto#tight-encoding.
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
Makefile | 2 +
Makefile.objs | 1 +
vnc-encoding-hextile.c | 5 +-
vnc-encoding-tight.c | 287 ++++++++++++++++++++++++++++++++++++++++++++++++
vnc-encoding-tight.h | 199 +++++++++++++++++++++++++++++++++
vnc-encoding-zlib.c | 17 ++-
vnc.c | 37 +++++--
vnc.h | 17 +++-
8 files changed, 543 insertions(+), 22 deletions(-)
create mode 100644 vnc-encoding-tight.c
create mode 100644 vnc-encoding-tight.h
diff --git a/Makefile b/Makefile
index eb9e02b..99d9c5d 100644
--- a/Makefile
+++ b/Makefile
@@ -124,6 +124,8 @@ vnc-encoding-zlib.o: vnc.h
vnc-encoding-hextile.o: vnc.h
+vnc-encoding-tight.o: vnc.h vnc-encoding-tight.h
+
curses.o: curses.c keymaps.h curses_keys.h
bt-host.o: QEMU_CFLAGS += $(BLUEZ_CFLAGS)
diff --git a/Makefile.objs b/Makefile.objs
index acbaf22..070ee09 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -104,6 +104,7 @@ common-obj-$(CONFIG_SDL) += sdl.o sdl_zoom.o x_keymap.o
common-obj-$(CONFIG_CURSES) += curses.o
common-obj-y += vnc.o acl.o d3des.o
common-obj-y += vnc-encoding-zlib.o vnc-encoding-hextile.o
+common-obj-y += vnc-encoding-tight.o
common-obj-y += iov.o
common-obj-$(CONFIG_VNC_TLS) += vnc-tls.o vnc-auth-vencrypt.o
common-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
diff --git a/vnc-encoding-hextile.c b/vnc-encoding-hextile.c
index a01c5e2..728f25e 100644
--- a/vnc-encoding-hextile.c
+++ b/vnc-encoding-hextile.c
@@ -62,8 +62,8 @@ static void hextile_enc_cord(uint8_t *ptr, int x, int y, int w, int h)
#undef BPP
#undef GENERIC
-void vnc_hextile_send_framebuffer_update(VncState *vs, int x,
- int y, int w, int h)
+int vnc_hextile_send_framebuffer_update(VncState *vs, int x,
+ int y, int w, int h)
{
int i, j;
int has_fg, has_bg;
@@ -83,6 +83,7 @@ void vnc_hextile_send_framebuffer_update(VncState *vs, int x,
free(last_fg);
free(last_bg);
+ return 1;
}
void vnc_hextile_set_pixel_conversion(VncState *vs, int generic)
diff --git a/vnc-encoding-tight.c b/vnc-encoding-tight.c
new file mode 100644
index 0000000..0d328c2
--- /dev/null
+++ b/vnc-encoding-tight.c
@@ -0,0 +1,287 @@
+/*
+ * QEMU VNC display driver: tight encoding
+ *
+ * Copyright (C) 2006 Anthony Liguori <anthony@codemonkey.ws>
+ * Copyright (C) 2006 Fabrice Bellard
+ * Copyright (C) 2009 Red Hat, Inc
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+
+#include <stdbool.h>
+
+#include "vnc.h"
+#include "vnc-encoding-tight.h"
+
+/* Compression level stuff. The following array contains various
+ encoder parameters for each of 10 compression levels (0..9).
+ Last three parameters correspond to JPEG quality levels (0..9). */
+
+static const struct {
+ int max_rect_size, max_rect_width;
+ int mono_min_rect_size, gradient_min_rect_size;
+ int idx_zlib_level, mono_zlib_level, raw_zlib_level, gradient_zlib_level;
+ int gradient_threshold, gradient_threshold24;
+ int idx_max_colors_divisor;
+ int jpeg_quality, jpeg_threshold, jpeg_threshold24;
+} tight_conf[] = {
+ { 512, 32, 6, 65536, 0, 0, 0, 0, 0, 0, 4, 5, 10000, 23000 },
+ { 2048, 128, 6, 65536, 1, 1, 1, 0, 0, 0, 8, 10, 8000, 18000 },
+ { 6144, 256, 8, 65536, 3, 3, 2, 0, 0, 0, 24, 15, 6500, 15000 },
+ { 10240, 1024, 12, 65536, 5, 5, 3, 0, 0, 0, 32, 25, 5000, 12000 },
+ { 16384, 2048, 12, 65536, 6, 6, 4, 0, 0, 0, 32, 37, 4000, 10000 },
+ { 32768, 2048, 12, 4096, 7, 7, 5, 4, 150, 380, 32, 50, 3000, 8000 },
+ { 65536, 2048, 16, 4096, 7, 7, 6, 4, 170, 420, 48, 60, 2000, 5000 },
+ { 65536, 2048, 16, 4096, 8, 8, 7, 5, 180, 450, 64, 70, 1000, 2500 },
+ { 65536, 2048, 32, 8192, 9, 9, 8, 6, 190, 475, 64, 75, 500, 1200 },
+ { 65536, 2048, 32, 8192, 9, 9, 9, 6, 200, 500, 96, 80, 200, 500 }
+};
+
+static int tight_init_stream(VncState *vs, int stream_id,
+ int level, int strategy)
+{
+ z_streamp zstream = &vs->tight_stream[stream_id];
+
+ if (zstream->opaque != vs) {
+ int err;
+
+ VNC_DEBUG("VNC: TIGHT: initializing zlib stream %d\n", stream_id);
+ VNC_DEBUG("VNC: TIGHT: opaque = %p | vs = %p\n", zstream->opaque, vs);
+ zstream->zalloc = vnc_zlib_zalloc;
+ zstream->zfree = vnc_zlib_zfree;
+
+ err = deflateInit2(zstream, level, Z_DEFLATED, MAX_WBITS,
+ MAX_MEM_LEVEL, strategy);
+
+ if (err != Z_OK) {
+ fprintf(stderr, "VNC: error initializing zlib\n");
+ return -1;
+ }
+
+ vs->tight_levels[stream_id] = level;
+ zstream->opaque = vs;
+ }
+
+ if (vs->tight_levels[stream_id] != level) {
+ if (deflateParams (zstream, level, strategy) != Z_OK) {
+ return -1;
+ }
+ vs->tight_levels[stream_id] = level;
+ }
+ return 0;
+}
+
+static void tight_send_compact_size(VncState *vs, size_t len)
+{
+ int lpc = 0;
+ int bytes = 0;
+ char buf[3] = {0, 0, 0};
+
+ buf[bytes++] = len & 0x7F;
+ if (len > 0x7F) {
+ buf[bytes-1] |= 0x80;
+ buf[bytes++] = len >> 7 & 0x7F;
+ if (len > 0x3FFF) {
+ buf[bytes-1] |= 0x80;
+ buf[bytes++] = len >> 14 & 0xFF;
+ }
+ }
+ for(lpc = 0; lpc < bytes; lpc++) {
+ vnc_write_u8(vs, buf[lpc]);
+ }
+}
+
+static int tight_compress_data(VncState *vs, int stream_id, size_t bytes,
+ int level, int strategy)
+{
+ z_streamp zstream = &vs->tight_stream[stream_id];
+ int previous_out;
+
+ if (bytes < VNC_TIGHT_MIN_TO_COMPRESS) {
+ vnc_write(vs, vs->tight.buffer, vs->tight.offset);
+ return bytes;
+ }
+
+ if (tight_init_stream(vs, stream_id, level, strategy))
+ return -1;
+
+ // reserve memory in output buffer
+ buffer_reserve(&vs->tight_zlib, bytes + 64);
+
+ // set pointers
+ zstream->next_in = vs->tight.buffer;
+ zstream->avail_in = vs->tight.offset;
+ zstream->next_out = vs->tight_zlib.buffer + vs->tight_zlib.offset;
+ zstream->avail_out = vs->tight_zlib.capacity - vs->tight_zlib.offset;
+ zstream->data_type = Z_BINARY;
+ previous_out = zstream->total_out;
+
+ // start encoding
+ if (deflate(zstream, Z_SYNC_FLUSH) != Z_OK) {
+ fprintf(stderr, "VNC: error during tight compression\n");
+ return -1;
+ }
+
+ vs->tight_zlib.offset = vs->tight_zlib.capacity - zstream->avail_out;
+ bytes = zstream->total_out - previous_out;
+
+ tight_send_compact_size(vs, bytes);
+ vnc_write(vs, vs->tight_zlib.buffer, bytes);
+
+ buffer_reset(&vs->tight_zlib);
+
+ return bytes;
+}
+
+/*
+ * Subencoding implementations.
+ */
+static void tight_pack24(VncState *vs, size_t count)
+{
+ unsigned char *buf;
+ uint32_t *buf32;
+ uint32_t pix;
+ int rshift, gshift, bshift;
+
+ buf = vs->tight.buffer;
+ buf32 = (uint32_t *)buf;
+
+ if ((vs->clientds.flags & QEMU_BIG_ENDIAN_FLAG) ==
+ (vs->ds->surface->flags & QEMU_BIG_ENDIAN_FLAG)) {
+ rshift = vs->clientds.pf.rshift;
+ gshift = vs->clientds.pf.gshift;
+ bshift = vs->clientds.pf.bshift;
+ } else {
+ rshift = 24 - vs->clientds.pf.rshift;
+ gshift = 24 - vs->clientds.pf.gshift;
+ bshift = 24 - vs->clientds.pf.bshift;
+ }
+
+ vs->tight.offset = count * 3;
+
+ while (count--) {
+ pix = *buf32++;
+ *buf++ = (char)(pix >> rshift);
+ *buf++ = (char)(pix >> gshift);
+ *buf++ = (char)(pix >> bshift);
+ }
+}
+
+static int send_full_color_rect(VncState *vs, int w, int h)
+{
+ int stream = 0;
+ size_t bytes;
+
+ vnc_write_u8(vs, stream << 4); /* no flushing, no filter */
+
+ if (vs->tight_pixel24) {
+ tight_pack24(vs, w * h);
+ bytes = 3;
+ } else {
+ bytes = vs->clientds.pf.bytes_per_pixel;
+ }
+
+ bytes = tight_compress_data(vs, stream, w * h * bytes,
+ tight_conf[vs->tight_compression].raw_zlib_level,
+ Z_DEFAULT_STRATEGY);
+
+ return (bytes >= 0);
+}
+
+static void vnc_tight_start(VncState *vs)
+{
+ buffer_reset(&vs->tight);
+
+ // make the output buffer be the zlib buffer, so we can compress it later
+ vs->tight_tmp = vs->output;
+ vs->output = vs->tight;
+}
+
+static void vnc_tight_stop(VncState *vs)
+{
+ // switch back to normal output/zlib buffers
+ vs->tight = vs->output;
+ vs->output = vs->tight_tmp;
+}
+
+static int send_sub_rect(VncState *vs, int x, int y, int w, int h)
+{
+ vnc_framebuffer_update(vs, x, y, w, h, VNC_ENCODING_TIGHT);
+
+ /*
+ * Convert pixels and store them in vs->tight
+ * We will probably rework that later, probably
+ * when adding other sub-encodings
+ */
+ vnc_tight_start(vs);
+ vnc_raw_send_framebuffer_update(vs, x, y, w, h);
+ vnc_tight_stop(vs);
+
+ return send_full_color_rect(vs, w, h);
+}
+
+static int send_rect_simple(VncState *vs, int x, int y, int w, int h)
+{
+ int max_size, max_width;
+ int max_sub_width, max_sub_height;
+ int dx, dy;
+ int rw, rh;
+ int n = 0;
+
+ max_size = tight_conf[vs->tight_compression].max_rect_size;
+ max_width = tight_conf[vs->tight_compression].max_rect_width;
+
+ if (w > max_width || w * h > max_size) {
+ max_sub_width = (w > max_width) ? max_width : w;
+ max_sub_height = max_size / max_sub_width;
+
+ for (dy = 0; dy < h; dy += max_sub_height) {
+ for (dx = 0; dx < w; dx += max_width) {
+ rw = (dx + max_sub_width < w) ? max_sub_width : w - dx;
+ rh = (dy + max_sub_height < h) ? max_sub_height : h - dy;
+ n += send_sub_rect(vs, x+dx, y+dy, rw, rh);
+ }
+ }
+ } else {
+ n += send_sub_rect(vs, x, y, w, h);
+ }
+
+ return n;
+}
+
+void vnc_tight_init(VncState *vs)
+{
+ int i;
+ for (i=0; i<ARRAY_SIZE(vs->tight_stream); i++)
+ vs->tight_stream[i].opaque = NULL;
+}
+
+int vnc_tight_send_framebuffer_update(VncState *vs, int x, int y,
+ int w, int h)
+{
+ if (vs->clientds.pf.bytes_per_pixel == 4 && vs->clientds.pf.rmax == 0xFF &&
+ vs->clientds.pf.bmax == 0xFF && vs->clientds.pf.gmax == 0xFF) {
+ vs->tight_pixel24 = true;
+ } else {
+ vs->tight_pixel24 = false;
+ }
+
+ return send_rect_simple(vs, x, y, w, h);
+}
diff --git a/vnc-encoding-tight.h b/vnc-encoding-tight.h
new file mode 100644
index 0000000..108e184
--- /dev/null
+++ b/vnc-encoding-tight.h
@@ -0,0 +1,199 @@
+/*
+ * QEMU VNC display driver: tight encoding
+ *
+ * Copyright (C) 2006 Anthony Liguori <anthony@codemonkey.ws>
+ * Copyright (C) 2006 Fabrice Bellard
+ * Copyright (C) 2009 Red Hat, Inc
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#ifndef __QEMU_VNC_ENCODING_TIGHT_H
+# define __QEMU_VNC_ENCODING_TIGHT_H
+
+/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+ * Tight Encoding.
+ *
+ *-- The first byte of each Tight-encoded rectangle is a "compression control
+ * byte". Its format is as follows (bit 0 is the least significant one):
+ *
+ * bit 0: if 1, then compression stream 0 should be reset;
+ * bit 1: if 1, then compression stream 1 should be reset;
+ * bit 2: if 1, then compression stream 2 should be reset;
+ * bit 3: if 1, then compression stream 3 should be reset;
+ * bits 7-4: if 1000 (0x08), then the compression type is "fill",
+ * if 1001 (0x09), then the compression type is "jpeg",
+ * if 0xxx, then the compression type is "basic",
+ * values greater than 1001 are not valid.
+ *
+ * If the compression type is "basic", then bits 6..4 of the
+ * compression control byte (those xxx in 0xxx) specify the following:
+ *
+ * bits 5-4: decimal representation is the index of a particular zlib
+ * stream which should be used for decompressing the data;
+ * bit 6: if 1, then a "filter id" byte is following this byte.
+ *
+ *-- The data that follows after the compression control byte described
+ * above depends on the compression type ("fill", "jpeg" or "basic").
+ *
+ *-- If the compression type is "fill", then the only pixel value follows, in
+ * client pixel format (see NOTE 1). This value applies to all pixels of the
+ * rectangle.
+ *
+ *-- If the compression type is "jpeg", the following data stream looks like
+ * this:
+ *
+ * 1..3 bytes: data size (N) in compact representation;
+ * N bytes: JPEG image.
+ *
+ * Data size is compactly represented in one, two or three bytes, according
+ * to the following scheme:
+ *
+ * 0xxxxxxx (for values 0..127)
+ * 1xxxxxxx 0yyyyyyy (for values 128..16383)
+ * 1xxxxxxx 1yyyyyyy zzzzzzzz (for values 16384..4194303)
+ *
+ * Here each character denotes one bit, xxxxxxx are the least significant 7
+ * bits of the value (bits 0-6), yyyyyyy are bits 7-13, and zzzzzzzz are the
+ * most significant 8 bits (bits 14-21). For example, decimal value 10000
+ * should be represented as two bytes: binary 10010000 01001110, or
+ * hexadecimal 90 4E.
+ *
+ *-- If the compression type is "basic" and bit 6 of the compression control
+ * byte was set to 1, then the next (second) byte specifies "filter id" which
+ * tells the decoder what filter type was used by the encoder to pre-process
+ * pixel data before the compression. The "filter id" byte can be one of the
+ * following:
+ *
+ * 0: no filter ("copy" filter);
+ * 1: "palette" filter;
+ * 2: "gradient" filter.
+ *
+ *-- If bit 6 of the compression control byte is set to 0 (no "filter id"
+ * byte), or if the filter id is 0, then raw pixel values in the client
+ * format (see NOTE 1) will be compressed. See below details on the
+ * compression.
+ *
+ *-- The "gradient" filter pre-processes pixel data with a simple algorithm
+ * which converts each color component to a difference between a "predicted"
+ * intensity and the actual intensity. Such a technique does not affect
+ * uncompressed data size, but helps to compress photo-like images better.
+ * Pseudo-code for converting intensities to differences is the following:
+ *
+ * P[i,j] := V[i-1,j] + V[i,j-1] - V[i-1,j-1];
+ * if (P[i,j] < 0) then P[i,j] := 0;
+ * if (P[i,j] > MAX) then P[i,j] := MAX;
+ * D[i,j] := V[i,j] - P[i,j];
+ *
+ * Here V[i,j] is the intensity of a color component for a pixel at
+ * coordinates (i,j). MAX is the maximum value of intensity for a color
+ * component.
+ *
+ *-- The "palette" filter converts true-color pixel data to indexed colors
+ * and a palette which can consist of 2..256 colors. If the number of colors
+ * is 2, then each pixel is encoded in 1 bit, otherwise 8 bits is used to
+ * encode one pixel. 1-bit encoding is performed such way that the most
+ * significant bits correspond to the leftmost pixels, and each raw of pixels
+ * is aligned to the byte boundary. When "palette" filter is used, the
+ * palette is sent before the pixel data. The palette begins with an unsigned
+ * byte which value is the number of colors in the palette minus 1 (i.e. 1
+ * means 2 colors, 255 means 256 colors in the palette). Then follows the
+ * palette itself which consist of pixel values in client pixel format (see
+ * NOTE 1).
+ *
+ *-- The pixel data is compressed using the zlib library. But if the data
+ * size after applying the filter but before the compression is less then 12,
+ * then the data is sent as is, uncompressed. Four separate zlib streams
+ * (0..3) can be used and the decoder should read the actual stream id from
+ * the compression control byte (see NOTE 2).
+ *
+ * If the compression is not used, then the pixel data is sent as is,
+ * otherwise the data stream looks like this:
+ *
+ * 1..3 bytes: data size (N) in compact representation;
+ * N bytes: zlib-compressed data.
+ *
+ * Data size is compactly represented in one, two or three bytes, just like
+ * in the "jpeg" compression method (see above).
+ *
+ *-- NOTE 1. If the color depth is 24, and all three color components are
+ * 8-bit wide, then one pixel in Tight encoding is always represented by
+ * three bytes, where the first byte is red component, the second byte is
+ * green component, and the third byte is blue component of the pixel color
+ * value. This applies to colors in palettes as well.
+ *
+ *-- NOTE 2. The decoder must reset compression streams' states before
+ * decoding the rectangle, if some of bits 0,1,2,3 in the compression control
+ * byte are set to 1. Note that the decoder must reset zlib streams even if
+ * the compression type is "fill" or "jpeg".
+ *
+ *-- NOTE 3. The "gradient" filter and "jpeg" compression may be used only
+ * when bits-per-pixel value is either 16 or 32, not 8.
+ *
+ *-- NOTE 4. The width of any Tight-encoded rectangle cannot exceed 2048
+ * pixels. If a rectangle is wider, it must be split into several rectangles
+ * and each one should be encoded separately.
+ *
+ */
+
+#define VNC_TIGHT_EXPLICIT_FILTER 0x04
+#define VNC_TIGHT_FILL 0x08
+#define VNC_TIGHT_JPEG 0x09
+#define VNC_TIGHT_MAX_SUBENCODING 0x09
+
+/* Filters to improve compression efficiency */
+#define VNC_TIGHT_FILTER_COPY 0x00
+#define VNC_TIGHT_FILTER_PALETTE 0x01
+#define VNC_TIGHT_FILTER_GRADIENT 0x02
+
+/* Note: The following constant should not be changed. */
+#define VNC_TIGHT_MIN_TO_COMPRESS 12
+
+/* The parameters below may be adjusted. */
+#define VNC_TIGHT_MIN_SPLIT_RECT_SIZE 4096
+#define VNC_TIGHT_MIN_SOLID_SUBRECT_SIZE 2048
+#define VNC_TIGHT_MAX_SPLIT_TILE_SIZE 16
+
+/*
+ * min()/max()/clamp() macros that also do
+ * strict type-checking.. See the
+ * "unnecessary" pointer comparison.
+ */
+#define min(x, y) ({ \
+ typeof(x) _min1 = (x); \
+ typeof(y) _min2 = (y); \
+ (void) (&_min1 == &_min2); \
+ _min1 < _min2 ? _min1 : _min2; })
+
+#define max(x, y) ({ \
+ typeof(x) _max1 = (x); \
+ typeof(y) _max2 = (y); \
+ (void) (&_max1 == &_max2); \
+ _max1 > _max2 ? _max1 : _max2; })
+
+#define clamp(val, min, max) ({ \
+ typeof(val) __val = (val); \
+ typeof(min) __min = (min); \
+ typeof(max) __max = (max); \
+ (void) (&__val == &__min); \
+ (void) (&__val == &__max); \
+ __val = __val < __min ? __min: __val; \
+ __val > __max ? __max: __val; })
+
+#endif /* __QEMU_VNC_ENCODING_TIGHT_H */
diff --git a/vnc-encoding-zlib.c b/vnc-encoding-zlib.c
index 29dd1b7..17a61bf 100644
--- a/vnc-encoding-zlib.c
+++ b/vnc-encoding-zlib.c
@@ -28,7 +28,7 @@
#define ZALLOC_ALIGNMENT 16
-static void *zalloc(void *x, unsigned items, unsigned size)
+void *vnc_zlib_zalloc(void *x, unsigned items, unsigned size)
{
void *p;
@@ -40,7 +40,7 @@ static void *zalloc(void *x, unsigned items, unsigned size)
return (p);
}
-static void zfree(void *x, void *addr)
+void vnc_zlib_zfree(void *x, void *addr)
{
qemu_free(addr);
}
@@ -72,8 +72,8 @@ static int vnc_zlib_stop(VncState *vs)
VNC_DEBUG("VNC: initializing zlib stream\n");
VNC_DEBUG("VNC: opaque = %p | vs = %p\n", zstream->opaque, vs);
- zstream->zalloc = zalloc;
- zstream->zfree = zfree;
+ zstream->zalloc = vnc_zlib_zalloc;
+ zstream->zfree = vnc_zlib_zfree;
err = deflateInit2(zstream, vs->tight_compression, Z_DEFLATED, MAX_WBITS,
MAX_MEM_LEVEL, Z_DEFAULT_STRATEGY);
@@ -116,7 +116,7 @@ static int vnc_zlib_stop(VncState *vs)
return zstream->total_out - previous_out;
}
-void vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
+int vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
{
int old_offset, new_offset, bytes_written;
@@ -131,14 +131,17 @@ void vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
vnc_raw_send_framebuffer_update(vs, x, y, w, h);
bytes_written = vnc_zlib_stop(vs);
- if (bytes_written == -1)
- return;
+ if (bytes_written == -1) {
+ return 0;
+ }
// hack in the size
new_offset = vs->output.offset;
vs->output.offset = old_offset;
vnc_write_u32(vs, bytes_written);
vs->output.offset = new_offset;
+
+ return 1;
}
void vnc_zlib_init(VncState *vs)
diff --git a/vnc.c b/vnc.c
index d0c0d00..c7b76ff 100644
--- a/vnc.c
+++ b/vnc.c
@@ -638,7 +638,7 @@ static void vnc_write_pixels_generic(VncState *vs, void *pixels1, int size)
}
}
-void vnc_raw_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
+int vnc_raw_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
{
int i;
uint8_t *row;
@@ -649,23 +649,31 @@ void vnc_raw_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
vs->write_pixels(vs, row, w * ds_get_bytes_per_pixel(vs->ds));
row += ds_get_linesize(vs->ds);
}
+ return 1;
}
-static void send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
+static int send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
{
+ int n = 0;
+
switch(vs->vnc_encoding) {
case VNC_ENCODING_ZLIB:
- vnc_zlib_send_framebuffer_update(vs, x, y, w, h);
+ n = vnc_zlib_send_framebuffer_update(vs, x, y, w, h);
break;
case VNC_ENCODING_HEXTILE:
vnc_framebuffer_update(vs, x, y, w, h, VNC_ENCODING_HEXTILE);
- vnc_hextile_send_framebuffer_update(vs, x, y, w, h);
+ n = vnc_hextile_send_framebuffer_update(vs, x, y, w, h);
+ break;
+ case VNC_ENCODING_TIGHT:
+ n = vnc_tight_send_framebuffer_update(vs, x, y, w, h);
break;
default:
vnc_framebuffer_update(vs, x, y, w, h, VNC_ENCODING_RAW);
- vnc_raw_send_framebuffer_update(vs, x, y, w, h);
+ n = vnc_raw_send_framebuffer_update(vs, x, y, w, h);
break;
}
+
+ return n;
}
static void vnc_copy(VncState *vs, int src_x, int src_y, int dst_x, int dst_y, int w, int h)
@@ -776,6 +784,7 @@ static int vnc_update_client(VncState *vs, int has_dirty)
int y;
int n_rectangles;
int saved_offset;
+ int n;
if (vs->output.offset && !vs->audio_cap && !vs->force_update)
/* kernel send buffers are full -> drop frames to throttle */
@@ -808,16 +817,18 @@ static int vnc_update_client(VncState *vs, int has_dirty)
} else {
if (last_x != -1) {
int h = find_and_clear_dirty_height(vs, y, last_x, x);
- send_framebuffer_update(vs, last_x * 16, y, (x - last_x) * 16, h);
- n_rectangles++;
+ n = send_framebuffer_update(vs, last_x * 16, y,
+ (x - last_x) * 16, h);
+ n_rectangles += n;
}
last_x = -1;
}
}
if (last_x != -1) {
int h = find_and_clear_dirty_height(vs, y, last_x, x);
- send_framebuffer_update(vs, last_x * 16, y, (x - last_x) * 16, h);
- n_rectangles++;
+ n = send_framebuffer_update(vs, last_x * 16, y,
+ (x - last_x) * 16, h);
+ n_rectangles += n;
}
}
vs->output.buffer[saved_offset] = (n_rectangles >> 8) & 0xFF;
@@ -1600,8 +1611,10 @@ static void set_encodings(VncState *vs, int32_t *encodings, size_t n_encodings)
unsigned int enc = 0;
vnc_zlib_init(vs);
+ vnc_tight_init(vs);
vs->features = 0;
- vs->vnc_encoding = -1;
+ if (!vs->vnc_encoding)
+ vs->vnc_encoding = -1;
vs->tight_compression = 9;
vs->tight_quality = 9;
vs->absolute = -1;
@@ -1619,6 +1632,10 @@ static void set_encodings(VncState *vs, int32_t *encodings, size_t n_encodings)
vs->features |= VNC_FEATURE_HEXTILE_MASK;
set_encoding(vs, enc);
break;
+ case VNC_ENCODING_TIGHT:
+ vs->features |= VNC_FEATURE_TIGHT_MASK;
+ set_encoding(vs, enc);
+ break;
case VNC_ENCODING_ZLIB:
vs->features |= VNC_FEATURE_ZLIB_MASK;
set_encoding(vs, enc);
diff --git a/vnc.h b/vnc.h
index 96f3fe7..ec7f481 100644
--- a/vnc.h
+++ b/vnc.h
@@ -166,6 +166,12 @@ struct VncState
/* Tight */
uint8_t tight_quality;
uint8_t tight_compression;
+ uint8_t tight_pixel24;
+ Buffer tight;
+ Buffer tight_tmp;
+ Buffer tight_zlib;
+ int tight_levels[4];
+ z_stream tight_stream[4];
/* Hextile */
VncSendHextileTile *send_hextile_tile;
@@ -391,13 +397,18 @@ void vnc_framebuffer_update(VncState *vs, int x, int y, int w, int h,
void vnc_convert_pixel(VncState *vs, uint8_t *buf, uint32_t v);
/* Encodings */
-void vnc_raw_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
+int vnc_raw_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
-void vnc_hextile_send_framebuffer_update(VncState *vs, int x,
+int vnc_hextile_send_framebuffer_update(VncState *vs, int x,
int y, int w, int h);
void vnc_hextile_set_pixel_conversion(VncState *vs, int generic);
void vnc_zlib_init(VncState *vs);
-void vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
+void *vnc_zlib_zalloc(void *x, unsigned items, unsigned size);
+void vnc_zlib_zfree(void *x, void *addr);
+int vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
+
+void vnc_tight_init(VncState *vs);
+int vnc_tight_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
#endif /* __QEMU_VNC_H */
--
1.7.0.2
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [Qemu-devel] [PATCH v2 06/10] vnc: add support for tight fill encoding
2010-05-18 12:49 [Qemu-devel] [PATCH v2 00/10] vnc-encodings Corentin Chary
` (4 preceding siblings ...)
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 05/10] vnc: add basic tight support Corentin Chary
@ 2010-05-18 12:49 ` Corentin Chary
2010-05-18 13:34 ` [Qemu-devel] " Anthony Liguori
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 07/10] vnc: don't clear zlib stream on set_encoding Corentin Chary
` (3 subsequent siblings)
9 siblings, 1 reply; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 12:49 UTC (permalink / raw)
To: qemu-devel; +Cc: Corentin Chary, Anthony Liguori, Alexander Graf, Adam Litke
Fill encoding detects rectangle using only one color and send only
one pixel value.
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
vnc-encoding-tight.c | 245 +++++++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 244 insertions(+), 1 deletions(-)
diff --git a/vnc-encoding-tight.c b/vnc-encoding-tight.c
index 0d328c2..5ea90c0 100644
--- a/vnc-encoding-tight.c
+++ b/vnc-encoding-tight.c
@@ -54,6 +54,135 @@ static const struct {
{ 65536, 2048, 32, 8192, 9, 9, 9, 6, 200, 500, 96, 80, 200, 500 }
};
+/*
+ * Check if a rectangle is all of the same color. If needSameColor is
+ * set to non-zero, then also check that its color equals to the
+ * *colorPtr value. The result is 1 if the test is successfull, and in
+ * that case new color will be stored in *colorPtr.
+ */
+
+#define DEFINE_CHECK_SOLID_FUNCTION(bpp) \
+ \
+ static bool \
+ check_solid_tile##bpp(VncState *vs, int x, int y, int w, int h, \
+ uint32_t* color, bool samecolor) \
+ { \
+ VncDisplay *vd = vs->vd; \
+ uint##bpp##_t *fbptr; \
+ uint##bpp##_t c; \
+ int dx, dy; \
+ \
+ fbptr = (uint##bpp##_t *) \
+ (vd->server->data + y * ds_get_linesize(vs->ds) + \
+ x * ds_get_bytes_per_pixel(vs->ds)); \
+ \
+ c = *fbptr; \
+ if (samecolor && (uint32_t)c != *color) \
+ return false; \
+ \
+ for (dy = 0; dy < h; dy++) { \
+ for (dx = 0; dx < w; dx++) { \
+ if (c != fbptr[dx]) \
+ return false; \
+ } \
+ fbptr = (uint##bpp##_t *) \
+ ((uint8_t *)fbptr + ds_get_linesize(vs->ds)); \
+ } \
+ \
+ *color = (uint32_t)c; \
+ return true; \
+ }
+
+DEFINE_CHECK_SOLID_FUNCTION(32)
+DEFINE_CHECK_SOLID_FUNCTION(16)
+DEFINE_CHECK_SOLID_FUNCTION(8)
+
+static bool check_solid_tile(VncState *vs, int x, int y, int w, int h,
+ uint32_t* color, bool samecolor)
+{
+ VncDisplay *vd = vs->vd;
+
+ switch(vd->server->pf.bytes_per_pixel) {
+ case 4:
+ return check_solid_tile32(vs, x, y, w, h, color, samecolor);
+ case 2:
+ return check_solid_tile16(vs, x, y, w, h, color, samecolor);
+ default:
+ return check_solid_tile8(vs, x, y, w, h, color, samecolor);
+ }
+}
+
+static void find_best_solid_area(VncState *vs, int x, int y, int w, int h,
+ uint32_t color, int *w_ptr, int *h_ptr)
+{
+ int dx, dy, dw, dh;
+ int w_prev;
+ int w_best = 0, h_best = 0;
+
+ w_prev = w;
+
+ for (dy = y; dy < y + h; dy += VNC_TIGHT_MAX_SPLIT_TILE_SIZE) {
+
+ dh = min(VNC_TIGHT_MAX_SPLIT_TILE_SIZE, y + h - dy);
+ dw = min(VNC_TIGHT_MAX_SPLIT_TILE_SIZE, w_prev);
+
+ if (!check_solid_tile(vs, x, dy, dw, dh, &color, true))
+ break;
+
+ for (dx = x + dw; dx < x + w_prev;) {
+ dw = min(VNC_TIGHT_MAX_SPLIT_TILE_SIZE, x + w_prev - dx);
+
+ if (!check_solid_tile(vs, dx, dy, dw, dh, &color, true))
+ break;
+ dx += dw;
+ }
+
+ w_prev = dx - x;
+ if (w_prev * (dy + dh - y) > w_best * h_best) {
+ w_best = w_prev;
+ h_best = dy + dh - y;
+ }
+ }
+
+ *w_ptr = w_best;
+ *h_ptr = h_best;
+}
+
+static void extend_solid_area(VncState *vs, int x, int y, int w, int h,
+ uint32_t color, int *x_ptr, int *y_ptr,
+ int *w_ptr, int *h_ptr)
+{
+ int cx, cy;
+
+ /* Try to extend the area upwards. */
+ for ( cy = *y_ptr - 1;
+ cy >= y && check_solid_tile(vs, *x_ptr, cy, *w_ptr, 1, &color, true);
+ cy-- );
+ *h_ptr += *y_ptr - (cy + 1);
+ *y_ptr = cy + 1;
+
+ /* ... downwards. */
+ for ( cy = *y_ptr + *h_ptr;
+ cy < y + h &&
+ check_solid_tile(vs, *x_ptr, cy, *w_ptr, 1, &color, true);
+ cy++ );
+ *h_ptr += cy - (*y_ptr + *h_ptr);
+
+ /* ... to the left. */
+ for ( cx = *x_ptr - 1;
+ cx >= x && check_solid_tile(vs, cx, *y_ptr, 1, *h_ptr, &color, true);
+ cx-- );
+ *w_ptr += *x_ptr - (cx + 1);
+ *x_ptr = cx + 1;
+
+ /* ... to the right. */
+ for ( cx = *x_ptr + *w_ptr;
+ cx < x + w &&
+ check_solid_tile(vs, cx, *y_ptr, 1, *h_ptr, &color, true);
+ cx++ );
+ *w_ptr += cx - (*x_ptr + *w_ptr);
+}
+
static int tight_init_stream(VncState *vs, int stream_id,
int level, int strategy)
{
@@ -205,6 +334,23 @@ static int send_full_color_rect(VncState *vs, int w, int h)
return (bytes >= 0);
}
+static int send_solid_rect(VncState *vs)
+{
+ size_t bytes;
+
+ vnc_write_u8(vs, VNC_TIGHT_FILL << 4); /* no flushing, no filter */
+
+ if (vs->tight_pixel24) {
+ tight_pack24(vs, 1);
+ bytes = 3;
+ } else {
+ bytes = vs->clientds.pf.bytes_per_pixel;
+ }
+
+ vnc_write(vs, vs->tight.buffer, bytes);
+ return 1;
+}
+
static void vnc_tight_start(VncState *vs)
{
buffer_reset(&vs->tight);
@@ -237,6 +383,17 @@ static int send_sub_rect(VncState *vs, int x, int y, int w, int h)
return send_full_color_rect(vs, w, h);
}
+static int send_sub_rect_solid(VncState *vs, int x, int y, int w, int h)
+{
+ vnc_framebuffer_update(vs, x, y, w, h, VNC_ENCODING_TIGHT);
+
+ vnc_tight_start(vs);
+ vnc_raw_send_framebuffer_update(vs, x, y, w, h);
+ vnc_tight_stop(vs);
+
+ return send_solid_rect(vs);
+}
+
static int send_rect_simple(VncState *vs, int x, int y, int w, int h)
{
int max_size, max_width;
@@ -266,6 +423,82 @@ static int send_rect_simple(VncState *vs, int x, int y, int w, int h)
return n;
}
+static int find_large_solid_color_rect(VncState *vs, int x, int y,
+ int w, int h, int max_rows)
+{
+ int dx, dy, dw, dh;
+ int n = 0;
+
+ /* Try to find large solid-color areas and send them separately. */
+
+ for (dy = y; dy < y + h; dy += VNC_TIGHT_MAX_SPLIT_TILE_SIZE) {
+
+ /* If a rectangle becomes too large, send its upper part now. */
+
+ if (dy - y >= max_rows) {
+ n += send_rect_simple(vs, x, y, w, max_rows);
+ y += max_rows;
+ h -= max_rows;
+ }
+
+ dh = min(VNC_TIGHT_MAX_SPLIT_TILE_SIZE, (y + h - dy));
+
+ for (dx = x; dx < x + w; dx += VNC_TIGHT_MAX_SPLIT_TILE_SIZE) {
+ uint32_t color_value;
+ int x_best, y_best, w_best, h_best;
+
+ dw = min(VNC_TIGHT_MAX_SPLIT_TILE_SIZE, (x + w - dx));
+
+ if (!check_solid_tile(vs, dx, dy, dw, dh, &color_value, false))
+ continue ;
+
+ /* Get dimensions of solid-color area. */
+
+ find_best_solid_area(vs, dx, dy, w - (dx - x), h - (dy - y),
+ color_value, &w_best, &h_best);
+
+ /* Make sure a solid rectangle is large enough
+ (or the whole rectangle is of the same color). */
+
+ if (w_best * h_best != w * h &&
+ w_best * h_best < VNC_TIGHT_MIN_SOLID_SUBRECT_SIZE)
+ continue;
+
+ /* Try to extend solid rectangle to maximum size. */
+
+ x_best = dx; y_best = dy;
+ extend_solid_area(vs, x, y, w, h, color_value,
+ &x_best, &y_best, &w_best, &h_best);
+
+ /* Send rectangles at top and left to solid-color area. */
+
+ if (y_best != y)
+ n += send_rect_simple(vs, x, y, w, y_best-y);
+ if (x_best != x)
+ n += vnc_tight_send_framebuffer_update(vs, x, y_best,
+ x_best-x, h_best);
+
+ /* Send solid-color rectangle. */
+ n += send_sub_rect_solid(vs, x_best, y_best, w_best, h_best);
+
+ /* Send remaining rectangles (at right and bottom). */
+
+ if (x_best + w_best != x + w)
+ n += vnc_tight_send_framebuffer_update(vs, x_best+w_best,
+ y_best,
+ w-(x_best-x)-w_best,
+ h_best);
+ if (y_best + h_best != y + h)
+ n += vnc_tight_send_framebuffer_update(vs, x, y_best+h_best,
+ w, h-(y_best-y)-h_best);
+
+ /* Return after all recursive calls are done. */
+ return n;
+ }
+ }
+ return n + send_rect_simple(vs, x, y, w, h);
+}
+
void vnc_tight_init(VncState *vs)
{
int i;
@@ -276,6 +509,8 @@ void vnc_tight_init(VncState *vs)
int vnc_tight_send_framebuffer_update(VncState *vs, int x, int y,
int w, int h)
{
+ int max_rows;
+
if (vs->clientds.pf.bytes_per_pixel == 4 && vs->clientds.pf.rmax == 0xFF &&
vs->clientds.pf.bmax == 0xFF && vs->clientds.pf.gmax == 0xFF) {
vs->tight_pixel24 = true;
@@ -283,5 +518,13 @@ int vnc_tight_send_framebuffer_update(VncState *vs, int x, int y,
vs->tight_pixel24 = false;
}
- return send_rect_simple(vs, x, y, w, h);
+ if (w * h < VNC_TIGHT_MIN_SPLIT_RECT_SIZE)
+ return send_rect_simple(vs, x, y, w, h);
+
+ /* Calculate maximum number of rows in one non-solid rectangle. */
+
+ max_rows = tight_conf[vs->tight_compression].max_rect_size;
+ max_rows /= min(tight_conf[vs->tight_compression].max_rect_width, w);
+
+ return find_large_solid_color_rect(vs, x, y, w, h, max_rows);
}
--
1.7.0.2
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [Qemu-devel] [PATCH v2 07/10] vnc: don't clear zlib stream on set_encoding
2010-05-18 12:49 [Qemu-devel] [PATCH v2 00/10] vnc-encodings Corentin Chary
` (5 preceding siblings ...)
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 06/10] vnc: add support for tight fill encoding Corentin Chary
@ 2010-05-18 12:49 ` Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 08/10] vnc: remove memory leaks in zlib and tight encoding Corentin Chary
` (2 subsequent siblings)
9 siblings, 0 replies; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 12:49 UTC (permalink / raw)
To: qemu-devel; +Cc: Corentin Chary, Anthony Liguori, Alexander Graf, Adam Litke
On init, values are already NULL, but we shouldn't try
to reset them each time a client send a set encoding
command because this break everything. For example,
libvncclient re-send a set encoding command if the
framebuffer is resized.
This fix framebuffer resizing for tight and zlib encodings.
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
vnc-encoding-tight.c | 7 -------
vnc-encoding-zlib.c | 5 -----
vnc.c | 2 --
vnc.h | 2 --
4 files changed, 0 insertions(+), 16 deletions(-)
diff --git a/vnc-encoding-tight.c b/vnc-encoding-tight.c
index 5ea90c0..229927e 100644
--- a/vnc-encoding-tight.c
+++ b/vnc-encoding-tight.c
@@ -499,13 +499,6 @@ static int find_large_solid_color_rect(VncState *vs, int x, int y,
return n + send_rect_simple(vs, x, y, w, h);
}
-void vnc_tight_init(VncState *vs)
-{
- int i;
- for (i=0; i<ARRAY_SIZE(vs->tight_stream); i++)
- vs->tight_stream[i].opaque = NULL;
-}
-
int vnc_tight_send_framebuffer_update(VncState *vs, int x, int y,
int w, int h)
{
diff --git a/vnc-encoding-zlib.c b/vnc-encoding-zlib.c
index 17a61bf..6b68540 100644
--- a/vnc-encoding-zlib.c
+++ b/vnc-encoding-zlib.c
@@ -143,8 +143,3 @@ int vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
return 1;
}
-
-void vnc_zlib_init(VncState *vs)
-{
- vs->zlib_stream.opaque = NULL;
-}
diff --git a/vnc.c b/vnc.c
index c7b76ff..f660c10 100644
--- a/vnc.c
+++ b/vnc.c
@@ -1610,8 +1610,6 @@ static void set_encodings(VncState *vs, int32_t *encodings, size_t n_encodings)
int i;
unsigned int enc = 0;
- vnc_zlib_init(vs);
- vnc_tight_init(vs);
vs->features = 0;
if (!vs->vnc_encoding)
vs->vnc_encoding = -1;
diff --git a/vnc.h b/vnc.h
index ec7f481..ea6e70b 100644
--- a/vnc.h
+++ b/vnc.h
@@ -403,12 +403,10 @@ int vnc_hextile_send_framebuffer_update(VncState *vs, int x,
int y, int w, int h);
void vnc_hextile_set_pixel_conversion(VncState *vs, int generic);
-void vnc_zlib_init(VncState *vs);
void *vnc_zlib_zalloc(void *x, unsigned items, unsigned size);
void vnc_zlib_zfree(void *x, void *addr);
int vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
-void vnc_tight_init(VncState *vs);
int vnc_tight_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
#endif /* __QEMU_VNC_H */
--
1.7.0.2
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [Qemu-devel] [PATCH v2 08/10] vnc: remove memory leaks in zlib and tight encoding
2010-05-18 12:49 [Qemu-devel] [PATCH v2 00/10] vnc-encodings Corentin Chary
` (6 preceding siblings ...)
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 07/10] vnc: don't clear zlib stream on set_encoding Corentin Chary
@ 2010-05-18 12:49 ` Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 09/10] vnc: tight: add palette encoding Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 10/10] vnc: update copyrights for vnc-encoding-tight.c Corentin Chary
9 siblings, 0 replies; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 12:49 UTC (permalink / raw)
To: qemu-devel; +Cc: Corentin Chary, Anthony Liguori, Alexander Graf, Adam Litke
Free buffers used by tight and zlib encodings, lso introduce
a new helper to free buffers: buffer_free()
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
vnc-encoding-tight.c | 11 +++++++++++
vnc-encoding-zlib.c | 8 ++++++++
vnc.c | 23 ++++++++++++++---------
vnc.h | 4 +++-
4 files changed, 36 insertions(+), 10 deletions(-)
diff --git a/vnc-encoding-tight.c b/vnc-encoding-tight.c
index 229927e..44984f9 100644
--- a/vnc-encoding-tight.c
+++ b/vnc-encoding-tight.c
@@ -499,6 +499,17 @@ static int find_large_solid_color_rect(VncState *vs, int x, int y,
return n + send_rect_simple(vs, x, y, w, h);
}
+void vnc_tight_clear(VncState *vs)
+{
+ int i;
+ for (i=0; i<ARRAY_SIZE(vs->tight_stream); i++)
+ if (vs->tight_stream[i].opaque)
+ deflateEnd(&vs->tight_stream[i]);
+
+ buffer_free(&vs->tight);
+ buffer_free(&vs->tight_zlib);
+}
+
int vnc_tight_send_framebuffer_update(VncState *vs, int x, int y,
int w, int h)
{
diff --git a/vnc-encoding-zlib.c b/vnc-encoding-zlib.c
index 6b68540..e5a431e 100644
--- a/vnc-encoding-zlib.c
+++ b/vnc-encoding-zlib.c
@@ -143,3 +143,11 @@ int vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
return 1;
}
+
+void vnc_zlib_clear(VncState *vs)
+{
+ if (vs->zlib_stream.opaque)
+ deflateEnd(&vs->zlib_stream);
+
+ buffer_free(&vs->zlib);
+}
diff --git a/vnc.c b/vnc.c
index f660c10..0dac736 100644
--- a/vnc.c
+++ b/vnc.c
@@ -503,7 +503,15 @@ uint8_t *buffer_end(Buffer *buffer)
void buffer_reset(Buffer *buffer)
{
- buffer->offset = 0;
+ buffer->offset = 0;
+}
+
+void buffer_free(Buffer *buffer)
+{
+ qemu_free(buffer->buffer);
+ buffer->offset = 0;
+ buffer->capacity = 0;
+ buffer->buffer = NULL;
}
void buffer_append(Buffer *buffer, const void *data, size_t len)
@@ -922,17 +930,14 @@ static void vnc_disconnect_finish(VncState *vs)
{
vnc_qmp_event(vs, QEVENT_VNC_DISCONNECTED);
- if (vs->input.buffer) {
- qemu_free(vs->input.buffer);
- vs->input.buffer = NULL;
- }
- if (vs->output.buffer) {
- qemu_free(vs->output.buffer);
- vs->output.buffer = NULL;
- }
+ buffer_free(&vs->input);
+ buffer_free(&vs->output);
qobject_decref(vs->info);
+ vnc_tight_clear(vs);
+ vnc_zlib_clear(vs);
+
#ifdef CONFIG_VNC_TLS
vnc_tls_client_cleanup(vs);
#endif /* CONFIG_VNC_TLS */
diff --git a/vnc.h b/vnc.h
index ea6e70b..18a4e8b 100644
--- a/vnc.h
+++ b/vnc.h
@@ -383,7 +383,7 @@ int buffer_empty(Buffer *buffer);
uint8_t *buffer_end(Buffer *buffer);
void buffer_reset(Buffer *buffer);
void buffer_append(Buffer *buffer, const void *data, size_t len);
-
+void buffer_free(Buffer *buffer);
/* Misc helpers */
@@ -403,10 +403,12 @@ int vnc_hextile_send_framebuffer_update(VncState *vs, int x,
int y, int w, int h);
void vnc_hextile_set_pixel_conversion(VncState *vs, int generic);
+void vnc_zlib_clear(VncState *vs);
void *vnc_zlib_zalloc(void *x, unsigned items, unsigned size);
void vnc_zlib_zfree(void *x, void *addr);
int vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
+void vnc_tight_clear(VncState *vs);
int vnc_tight_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
#endif /* __QEMU_VNC_H */
--
1.7.0.2
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [Qemu-devel] [PATCH v2 09/10] vnc: tight: add palette encoding
2010-05-18 12:49 [Qemu-devel] [PATCH v2 00/10] vnc-encodings Corentin Chary
` (7 preceding siblings ...)
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 08/10] vnc: remove memory leaks in zlib and tight encoding Corentin Chary
@ 2010-05-18 12:49 ` Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 10/10] vnc: update copyrights for vnc-encoding-tight.c Corentin Chary
9 siblings, 0 replies; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 12:49 UTC (permalink / raw)
To: qemu-devel; +Cc: Corentin Chary, Anthony Liguori, Alexander Graf, Adam Litke
Add palette tight encoding. Palette encoding will try to count the number
of colors for a given rectangle, and if this number is low enough
(< 256) it will send the palette + the rectangle with indexed colors.
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
vnc-encoding-tight.c | 430 ++++++++++++++++++++++++++++++++++++++++++++++++--
1 files changed, 418 insertions(+), 12 deletions(-)
diff --git a/vnc-encoding-tight.c b/vnc-encoding-tight.c
index 44984f9..b836569 100644
--- a/vnc-encoding-tight.c
+++ b/vnc-encoding-tight.c
@@ -27,6 +27,8 @@
#include <stdbool.h>
+#include "qdict.h"
+#include "qint.h"
#include "vnc.h"
#include "vnc-encoding-tight.h"
@@ -55,6 +57,279 @@ static const struct {
};
/*
+ * Code to determine how many different colors used in rectangle.
+ */
+
+static void tight_palette_rgb2buf(uint32_t rgb, int bpp, uint8_t buf[6])
+{
+ memset(buf, 0, 6);
+
+ if (bpp == 32) {
+ buf[0] = ((rgb >> 24) & 0xFF);
+ buf[1] = ((rgb >> 16) & 0xFF);
+ buf[2] = ((rgb >> 8) & 0xFF);
+ buf[3] = ((rgb >> 0) & 0xFF);
+ buf[4] = ((buf[0] & 1) == 0) << 3 | ((buf[1] & 1) == 0) << 2;
+ buf[4]|= ((buf[2] & 1) == 0) << 1 | ((buf[3] & 1) == 0) << 0;
+ buf[0] |= 1;
+ buf[1] |= 1;
+ buf[2] |= 1;
+ buf[3] |= 1;
+ }
+ if (bpp == 16) {
+ buf[0] = ((rgb >> 8) & 0xFF);
+ buf[1] = ((rgb >> 0) & 0xFF);
+ buf[2] = ((buf[0] & 1) == 0) << 1 | ((buf[1] & 1) == 0) << 0;
+ buf[0] |= 1;
+ buf[1] |= 1;
+ }
+}
+
+static uint32_t tight_palette_buf2rgb(int bpp, const uint8_t *buf)
+{
+ uint32_t rgb = 0;
+
+ if (bpp == 32) {
+ rgb |= ((buf[0] & ~1) | !((buf[4] >> 3) & 1)) << 24;
+ rgb |= ((buf[1] & ~1) | !((buf[4] >> 2) & 1)) << 16;
+ rgb |= ((buf[2] & ~1) | !((buf[4] >> 1) & 1)) << 8;
+ rgb |= ((buf[3] & ~1) | !((buf[4] >> 0) & 1)) << 0;
+ }
+ if (bpp == 16) {
+ rgb |= ((buf[0] & ~1) | !((buf[2] >> 1) & 1)) << 8;
+ rgb |= ((buf[1] & ~1) | !((buf[2] >> 0) & 1)) << 0;
+ }
+ return rgb;
+}
+
+
+static int tight_palette_insert(QDict *palette, uint32_t rgb, int bpp, int max)
+{
+ uint8_t key[6];
+ int idx = qdict_size(palette);
+ bool present;
+
+ tight_palette_rgb2buf(rgb, bpp, key);
+ present = qdict_haskey(palette, (char *)key);
+ if (idx >= max && !present) {
+ return 0;
+ }
+ if (!present)
+ qdict_put(palette, (char *)key, qint_from_int(idx));
+ return qdict_size(palette);
+}
+
+#define DEFINE_FILL_PALETTE_FUNCTION(bpp) \
+ \
+ static int \
+ tight_fill_palette##bpp(VncState *vs, int x, int y, \
+ int max, size_t count, \
+ uint32_t *bg, uint32_t *fg, \
+ struct QDict **palette) { \
+ uint##bpp##_t *data; \
+ uint##bpp##_t c0, c1, ci; \
+ int i, n0, n1; \
+ \
+ data = (uint##bpp##_t *)vs->tight.buffer; \
+ \
+ c0 = data[0]; \
+ i = 1; \
+ while (i < count && data[i] == c0) \
+ i++; \
+ if (i >= count) { \
+ *bg = *fg = c0; \
+ return 1; \
+ } \
+ \
+ if (max < 2) { \
+ return 0; \
+ } \
+ \
+ n0 = i; \
+ c1 = data[i]; \
+ n1 = 0; \
+ for (i++; i < count; i++) { \
+ ci = data[i]; \
+ if (ci == c0) { \
+ n0++; \
+ } else if (ci == c1) { \
+ n1++; \
+ } else \
+ break; \
+ } \
+ if (i >= count) { \
+ if (n0 > n1) { \
+ *bg = (uint32_t)c0; \
+ *fg = (uint32_t)c1; \
+ } else { \
+ *bg = (uint32_t)c1; \
+ *fg = (uint32_t)c0; \
+ } \
+ return 2; \
+ } \
+ \
+ if (max == 2) { \
+ return 0; \
+ } \
+ \
+ *palette = qdict_new(); \
+ tight_palette_insert(*palette, c0, bpp, max); \
+ tight_palette_insert(*palette, c1, bpp, max); \
+ \
+ for (i++; i < count; i++) { \
+ if (data[i] == ci) { \
+ continue; \
+ } else { \
+ if (!tight_palette_insert(*palette, (uint32_t)ci, \
+ bpp, max)) { \
+ return 0; \
+ } \
+ ci = data[i]; \
+ } \
+ } \
+ \
+ return qdict_size(*palette); \
+ }
+
+DEFINE_FILL_PALETTE_FUNCTION(8)
+DEFINE_FILL_PALETTE_FUNCTION(16)
+DEFINE_FILL_PALETTE_FUNCTION(32)
+
+static int tight_fill_palette(VncState *vs, int x, int y,
+ size_t count, uint32_t *bg, uint32_t *fg,
+ struct QDict **palette)
+{
+ int max;
+
+ max = count / tight_conf[vs->tight_compression].idx_max_colors_divisor;
+ if (max < 2 && count >= tight_conf[vs->tight_compression].mono_min_rect_size)
+ max = 2;
+ if (max >= 256)
+ max = 256;
+
+ switch(vs->clientds.pf.bytes_per_pixel) {
+ case 4:
+ return tight_fill_palette32(vs, x, y, max, count, bg, fg, palette);
+ case 2:
+ return tight_fill_palette16(vs, x, y, max, count, bg, fg, palette);
+ default:
+ max = 2;
+ return tight_fill_palette8(vs, x, y, max, count, bg, fg, palette);
+ }
+ return 0;
+}
+
+/* Callback to dump a palette with qdict_iter
+static void print_palette(const char *key, QObject *obj, void *opaque)
+{
+ uint8_t idx = qint_get_int(qobject_to_qint(obj));
+ uint32_t rgb = tight_palette_buf2rgb(32, (uint8_t *)key);
+
+ fprintf(stderr, "%.2x ", (unsigned char)*key);
+ while (*key++)
+ fprintf(stderr, "%.2x ", (unsigned char)*key);
+
+ fprintf(stderr, ": idx: %x rgb: %x\n", idx, rgb);
+}
+*/
+
+/*
+ * Converting truecolor samples into palette indices.
+ */
+#define DEFINE_IDX_ENCODE_FUNCTION(bpp) \
+ \
+ static void \
+ tight_encode_indexed_rect##bpp(uint8_t *buf, int count, \
+ struct QDict *palette) { \
+ uint##bpp##_t *src; \
+ uint##bpp##_t rgb; \
+ uint8_t key[6]; \
+ int rep = 0; \
+ uint8_t idx; \
+ \
+ src = (uint##bpp##_t *) buf; \
+ \
+ count -= 1; \
+ while (count--) { \
+ rgb = *src++; \
+ rep = 0; \
+ while (count && *src == rgb) { \
+ rep++, src++, count--; \
+ } \
+ tight_palette_rgb2buf(rgb, bpp, key); \
+ if (!qdict_haskey(palette, (char *)key)) { \
+ /* \
+ * Should never happen, but don't break everything \
+ * if it does, use the first color instead \
+ */ \
+ idx = 0; \
+ } else { \
+ idx = qdict_get_int(palette, (char *)key); \
+ } \
+ while (rep >= 0) { \
+ *buf++ = idx; \
+ rep--; \
+ } \
+ } \
+ }
+
+DEFINE_IDX_ENCODE_FUNCTION(16)
+DEFINE_IDX_ENCODE_FUNCTION(32)
+
+#define DEFINE_MONO_ENCODE_FUNCTION(bpp) \
+ \
+ static void \
+ tight_encode_mono_rect##bpp(uint8_t *buf, int w, int h, \
+ uint##bpp##_t bg, uint##bpp##_t fg) { \
+ uint##bpp##_t *ptr; \
+ unsigned int value, mask; \
+ int aligned_width; \
+ int x, y, bg_bits; \
+ \
+ ptr = (uint##bpp##_t *) buf; \
+ aligned_width = w - w % 8; \
+ \
+ for (y = 0; y < h; y++) { \
+ for (x = 0; x < aligned_width; x += 8) { \
+ for (bg_bits = 0; bg_bits < 8; bg_bits++) { \
+ if (*ptr++ != bg) \
+ break; \
+ } \
+ if (bg_bits == 8) { \
+ *buf++ = 0; \
+ continue; \
+ } \
+ mask = 0x80 >> bg_bits; \
+ value = mask; \
+ for (bg_bits++; bg_bits < 8; bg_bits++) { \
+ mask >>= 1; \
+ if (*ptr++ != bg) { \
+ value |= mask; \
+ } \
+ } \
+ *buf++ = (uint8_t)value; \
+ } \
+ \
+ mask = 0x80; \
+ value = 0; \
+ if (x >= w) \
+ continue; \
+ \
+ for (; x < w; x++) { \
+ if (*ptr++ != bg) { \
+ value |= mask; \
+ } \
+ mask >>= 1; \
+ } \
+ *buf++ = (uint8_t)value; \
+ } \
+ }
+
+DEFINE_MONO_ENCODE_FUNCTION(8)
+DEFINE_MONO_ENCODE_FUNCTION(16)
+DEFINE_MONO_ENCODE_FUNCTION(32)
+
+/*
* Check if a rectangle is all of the same color. If needSameColor is
* set to non-zero, then also check that its color equals to the
* *colorPtr value. The result is 1 if the test is successfull, and in
@@ -282,14 +557,12 @@ static int tight_compress_data(VncState *vs, int stream_id, size_t bytes,
/*
* Subencoding implementations.
*/
-static void tight_pack24(VncState *vs, size_t count)
+static void tight_pack24(VncState *vs, uint8_t *buf, size_t count, size_t *ret)
{
- unsigned char *buf;
uint32_t *buf32;
uint32_t pix;
int rshift, gshift, bshift;
- buf = vs->tight.buffer;
buf32 = (uint32_t *)buf;
if ((vs->clientds.flags & QEMU_BIG_ENDIAN_FLAG) ==
@@ -303,7 +576,8 @@ static void tight_pack24(VncState *vs, size_t count)
bshift = 24 - vs->clientds.pf.bshift;
}
- vs->tight.offset = count * 3;
+ if (ret)
+ *ret = count * 3;
while (count--) {
pix = *buf32++;
@@ -321,7 +595,7 @@ static int send_full_color_rect(VncState *vs, int w, int h)
vnc_write_u8(vs, stream << 4); /* no flushing, no filter */
if (vs->tight_pixel24) {
- tight_pack24(vs, w * h);
+ tight_pack24(vs, vs->tight.buffer, w * h, &vs->tight.offset);
bytes = 3;
} else {
bytes = vs->clientds.pf.bytes_per_pixel;
@@ -341,7 +615,7 @@ static int send_solid_rect(VncState *vs)
vnc_write_u8(vs, VNC_TIGHT_FILL << 4); /* no flushing, no filter */
if (vs->tight_pixel24) {
- tight_pack24(vs, 1);
+ tight_pack24(vs, vs->tight.buffer, 1, &vs->tight.offset);
bytes = 3;
} else {
bytes = vs->clientds.pf.bytes_per_pixel;
@@ -351,6 +625,126 @@ static int send_solid_rect(VncState *vs)
return 1;
}
+static int send_mono_rect(VncState *vs, int w, int h, uint32_t bg, uint32_t fg)
+{
+ size_t bytes;
+ int stream = 1;
+ int level = tight_conf[vs->tight_compression].mono_zlib_level;
+
+ bytes = ((w + 7) / 8) * h;
+
+ vnc_write_u8(vs, (stream | VNC_TIGHT_EXPLICIT_FILTER) << 4);
+ vnc_write_u8(vs, VNC_TIGHT_FILTER_PALETTE);
+ vnc_write_u8(vs, 1);
+
+ switch(vs->clientds.pf.bytes_per_pixel) {
+ case 4:
+ {
+ uint32_t buf[2] = {bg, fg};
+ size_t ret = sizeof (buf);
+
+ if (vs->tight_pixel24) {
+ tight_pack24(vs, (unsigned char*)buf, 2, &ret);
+ }
+ vnc_write(vs, buf, ret);
+
+ tight_encode_mono_rect32(vs->tight.buffer, w, h, bg, fg);
+ break;
+ }
+ case 2:
+ vnc_write(vs, &bg, 2);
+ vnc_write(vs, &fg, 2);
+ tight_encode_mono_rect16(vs->tight.buffer, w, h, bg, fg);
+ break;
+ default:
+ vnc_write_u8(vs, bg);
+ vnc_write_u8(vs, fg);
+ tight_encode_mono_rect8(vs->tight.buffer, w, h, bg, fg);
+ break;
+ }
+ vs->tight.offset = bytes;
+
+ bytes = tight_compress_data(vs, stream, bytes, level, Z_DEFAULT_STRATEGY);
+ return (bytes >= 0);
+}
+
+struct palette_cb_priv {
+ VncState *vs;
+ uint8_t *header;
+};
+
+static void write_palette(const char *key, QObject *obj, void *opaque)
+{
+ struct palette_cb_priv *priv = opaque;
+ VncState *vs = priv->vs;
+ uint32_t bytes = vs->clientds.pf.bytes_per_pixel;
+ uint8_t idx = qint_get_int(qobject_to_qint(obj));
+
+ if (bytes == 4) {
+ uint32_t color = tight_palette_buf2rgb(32, (uint8_t *)key);
+
+ ((uint32_t*)priv->header)[idx] = color;
+ } else {
+ uint16_t color = tight_palette_buf2rgb(16, (uint8_t *)key);
+
+ ((uint16_t*)priv->header)[idx] = color;
+ }
+}
+
+static int send_palette_rect(VncState *vs, int w, int h, struct QDict *palette)
+{
+ int stream = 2;
+ int level = tight_conf[vs->tight_compression].idx_zlib_level;
+ int colors;
+ size_t bytes;
+
+ colors = qdict_size(palette);
+
+ vnc_write_u8(vs, (stream | VNC_TIGHT_EXPLICIT_FILTER) << 4);
+ vnc_write_u8(vs, VNC_TIGHT_FILTER_PALETTE);
+ vnc_write_u8(vs, colors - 1);
+
+ switch(vs->clientds.pf.bytes_per_pixel) {
+ case 4:
+ {
+ size_t old_offset, offset;
+ uint32_t header[qdict_size(palette)];
+ struct palette_cb_priv priv = { vs, (uint8_t *)header };
+
+ old_offset = vs->output.offset;
+ qdict_iter(palette, write_palette, &priv);
+ vnc_write(vs, header, sizeof(header));
+
+ if (vs->tight_pixel24) {
+ tight_pack24(vs, vs->output.buffer + old_offset, colors, &offset);
+ vs->output.offset = old_offset + offset;
+ }
+
+ tight_encode_indexed_rect32(vs->tight.buffer, w * h, palette);
+ break;
+ }
+ case 2:
+ {
+ uint16_t header[qdict_size(palette)];
+ struct palette_cb_priv priv = { vs, (uint8_t *)header };
+
+ qdict_iter(palette, write_palette, &priv);
+ vnc_write(vs, header, sizeof(header));
+ tight_encode_indexed_rect16(vs->tight.buffer, w * h, palette);
+ break;
+ }
+ default:
+ return -1; /* No palette for 8bits colors */
+ break;
+ }
+ bytes = w * h;
+ vs->tight.offset = bytes;
+
+ bytes = tight_compress_data(vs, stream, bytes,
+ level, Z_DEFAULT_STRATEGY);
+ return (bytes >= 0);
+}
+
static void vnc_tight_start(VncState *vs)
{
buffer_reset(&vs->tight);
@@ -369,18 +763,30 @@ static void vnc_tight_stop(VncState *vs)
static int send_sub_rect(VncState *vs, int x, int y, int w, int h)
{
+ struct QDict *palette = NULL;
+ uint32_t bg = 0, fg = 0;
+ int colors;
+ int ret = 0;
+
vnc_framebuffer_update(vs, x, y, w, h, VNC_ENCODING_TIGHT);
- /*
- * Convert pixels and store them in vs->tight
- * We will probably rework that later, probably
- * when adding other sub-encodings
- */
vnc_tight_start(vs);
vnc_raw_send_framebuffer_update(vs, x, y, w, h);
vnc_tight_stop(vs);
- return send_full_color_rect(vs, w, h);
+ colors = tight_fill_palette(vs, x, y, w * h, &fg, &bg, &palette);
+
+ if (colors == 0) {
+ ret = send_full_color_rect(vs, w, h);
+ } else if (colors == 1) {
+ ret = send_solid_rect(vs);
+ } else if (colors == 2) {
+ ret = send_mono_rect(vs, w, h, bg, fg);
+ } else if (colors <= 256) {
+ ret = send_palette_rect(vs, w, h, palette);
+ }
+ QDECREF(palette);
+ return ret;
}
static int send_sub_rect_solid(VncState *vs, int x, int y, int w, int h)
--
1.7.0.2
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [Qemu-devel] [PATCH v2 10/10] vnc: update copyrights for vnc-encoding-tight.c
2010-05-18 12:49 [Qemu-devel] [PATCH v2 00/10] vnc-encodings Corentin Chary
` (8 preceding siblings ...)
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 09/10] vnc: tight: add palette encoding Corentin Chary
@ 2010-05-18 12:49 ` Corentin Chary
2010-05-18 13:37 ` [Qemu-devel] " Anthony Liguori
9 siblings, 1 reply; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 12:49 UTC (permalink / raw)
To: qemu-devel; +Cc: Corentin Chary, Anthony Liguori, Alexander Graf, Adam Litke
Add missing copyrights in vnc-encoding-tight.c, since the code is
mostly inspired from libvncserver/tight.c.
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
vnc-encoding-tight.c | 5 +++++
1 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/vnc-encoding-tight.c b/vnc-encoding-tight.c
index b836569..0c661a9 100644
--- a/vnc-encoding-tight.c
+++ b/vnc-encoding-tight.c
@@ -1,9 +1,14 @@
/*
* QEMU VNC display driver: tight encoding
*
+ * From libvncserver/tight.c
+ * Copyright (C) 1999 AT&T Laboratories Cambridge. All Rights Reserved.
+ * Copyright (C) 2000, 2001 Const Kaplinsky. All Rights Reserved.
+ *
* Copyright (C) 2006 Anthony Liguori <anthony@codemonkey.ws>
* Copyright (C) 2006 Fabrice Bellard
* Copyright (C) 2009 Red Hat, Inc
+ * Copyright (C) 2019 Corentin Chary <corentin.chary@gmail.com>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
--
1.7.0.2
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 03/10] vnc: only use a single zlib stream
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 03/10] vnc: only use a single zlib stream Corentin Chary
@ 2010-05-18 12:51 ` Alexander Graf
2010-05-18 13:39 ` Anthony Liguori
0 siblings, 1 reply; 30+ messages in thread
From: Alexander Graf @ 2010-05-18 12:51 UTC (permalink / raw)
To: Corentin Chary; +Cc: Anthony Liguori, qemu-devel, Adam Litke
Corentin Chary wrote:
> According to http://tigervnc.org/cgi-bin/rfbproto#zlib-encoding
> zlib encoding only uses a single stream. Current implementation defines
> 4 streams but only uses the first one. Remove them and only use a single
> stream.
>
How about when we start to implement zrle or zhextile? Wouldn't those
need different streams?
Alex
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 01/10] vnc: refactor set_encodings
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 01/10] vnc: refactor set_encodings Corentin Chary
@ 2010-05-18 12:54 ` Alexander Graf
2010-05-18 18:09 ` Corentin Chary
0 siblings, 1 reply; 30+ messages in thread
From: Alexander Graf @ 2010-05-18 12:54 UTC (permalink / raw)
To: Corentin Chary; +Cc: Anthony Liguori, qemu-devel, Adam Litke
Corentin Chary wrote:
> Create a new set_encoding() function to remove
> duplicated code in set_encodings().
>
> Signed-off-by: Corentin Chary <corentincj@iksaif.net>
>
Acked-by: Alexander Graf <agraf@suse.de>
Keep in mind that this still means that the last requested compression
algorithm is used, which is reverse according to spec.
Alex
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 02/10] vnc: really call zlib if we want zlib
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 02/10] vnc: really call zlib if we want zlib Corentin Chary
@ 2010-05-18 12:54 ` Alexander Graf
0 siblings, 0 replies; 30+ messages in thread
From: Alexander Graf @ 2010-05-18 12:54 UTC (permalink / raw)
To: Corentin Chary; +Cc: Anthony Liguori, qemu-devel, Adam Litke
Corentin Chary wrote:
> send_framebuffer_update() was calling hextile instead of zlib
> since commit 70a4568fe0c5a64adaa3da5030b7109e5199e692.
>
> Signed-off-by: Corentin Chary <corentincj@iksaif.net>
>
Acked-by: Alexander Graf <agraf@suse.de>
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 04/10] vnc: adjust compression zstream level
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 04/10] vnc: adjust compression zstream level Corentin Chary
@ 2010-05-18 12:55 ` Alexander Graf
0 siblings, 0 replies; 30+ messages in thread
From: Alexander Graf @ 2010-05-18 12:55 UTC (permalink / raw)
To: Corentin Chary; +Cc: Anthony Liguori, qemu-devel, Adam Litke
Corentin Chary wrote:
> Adjust zlib compression level if needed by calling deflateParams.
>
> Signed-off-by: Corentin Chary <corentincj@iksaif.net>
>
Wow, I didn't know it was that easy :).
Acked-by: Alexander Graf <agraf@suse.de>
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 05/10] vnc: add basic tight support
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 05/10] vnc: add basic tight support Corentin Chary
@ 2010-05-18 13:33 ` Anthony Liguori
2010-05-18 13:49 ` Alexander Graf
1 sibling, 0 replies; 30+ messages in thread
From: Anthony Liguori @ 2010-05-18 13:33 UTC (permalink / raw)
To: Corentin Chary; +Cc: Adam Litke, qemu-devel, Alexander Graf
On 05/18/2010 07:49 AM, Corentin Chary wrote:
> Add support for tight encoding [1]. This patch only add support
> for "basic" tight compression without any filter.
>
> [1] http://tigervnc.org/cgi-bin/rfbproto#tight-encoding.
>
> Signed-off-by: Corentin Chary<corentincj@iksaif.net>
> ---
> Makefile | 2 +
> Makefile.objs | 1 +
> vnc-encoding-hextile.c | 5 +-
> vnc-encoding-tight.c | 287 ++++++++++++++++++++++++++++++++++++++++++++++++
> vnc-encoding-tight.h | 199 +++++++++++++++++++++++++++++++++
> vnc-encoding-zlib.c | 17 ++-
> vnc.c | 37 +++++--
> vnc.h | 17 +++-
> 8 files changed, 543 insertions(+), 22 deletions(-)
> create mode 100644 vnc-encoding-tight.c
> create mode 100644 vnc-encoding-tight.h
>
> diff --git a/Makefile b/Makefile
> index eb9e02b..99d9c5d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -124,6 +124,8 @@ vnc-encoding-zlib.o: vnc.h
>
> vnc-encoding-hextile.o: vnc.h
>
> +vnc-encoding-tight.o: vnc.h vnc-encoding-tight.h
> +
> curses.o: curses.c keymaps.h curses_keys.h
>
> bt-host.o: QEMU_CFLAGS += $(BLUEZ_CFLAGS)
> diff --git a/Makefile.objs b/Makefile.objs
> index acbaf22..070ee09 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -104,6 +104,7 @@ common-obj-$(CONFIG_SDL) += sdl.o sdl_zoom.o x_keymap.o
> common-obj-$(CONFIG_CURSES) += curses.o
> common-obj-y += vnc.o acl.o d3des.o
> common-obj-y += vnc-encoding-zlib.o vnc-encoding-hextile.o
> +common-obj-y += vnc-encoding-tight.o
> common-obj-y += iov.o
> common-obj-$(CONFIG_VNC_TLS) += vnc-tls.o vnc-auth-vencrypt.o
> common-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
> diff --git a/vnc-encoding-hextile.c b/vnc-encoding-hextile.c
> index a01c5e2..728f25e 100644
> --- a/vnc-encoding-hextile.c
> +++ b/vnc-encoding-hextile.c
> @@ -62,8 +62,8 @@ static void hextile_enc_cord(uint8_t *ptr, int x, int y, int w, int h)
> #undef BPP
> #undef GENERIC
>
> -void vnc_hextile_send_framebuffer_update(VncState *vs, int x,
> - int y, int w, int h)
> +int vnc_hextile_send_framebuffer_update(VncState *vs, int x,
> + int y, int w, int h)
> {
> int i, j;
> int has_fg, has_bg;
> @@ -83,6 +83,7 @@ void vnc_hextile_send_framebuffer_update(VncState *vs, int x,
> free(last_fg);
> free(last_bg);
>
> + return 1;
> }
>
> void vnc_hextile_set_pixel_conversion(VncState *vs, int generic)
> diff --git a/vnc-encoding-tight.c b/vnc-encoding-tight.c
> new file mode 100644
> index 0000000..0d328c2
> --- /dev/null
> +++ b/vnc-encoding-tight.c
> @@ -0,0 +1,287 @@
> +/*
> + * QEMU VNC display driver: tight encoding
> + *
> + * Copyright (C) 2006 Anthony Liguori<anthony@codemonkey.ws>
> + * Copyright (C) 2006 Fabrice Bellard
> + * Copyright (C) 2009 Red Hat, Inc
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this software and associated documentation files (the "Software"), to deal
> + * in the Software without restriction, including without limitation the rights
> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> + * copies of the Software, and to permit persons to whom the Software is
> + * furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
> + * THE SOFTWARE.
> + */
> +
> +
> +#include<stdbool.h>
> +
> +#include "vnc.h"
> +#include "vnc-encoding-tight.h"
> +
> +/* Compression level stuff. The following array contains various
> + encoder parameters for each of 10 compression levels (0..9).
> + Last three parameters correspond to JPEG quality levels (0..9). */
> +
> +static const struct {
> + int max_rect_size, max_rect_width;
> + int mono_min_rect_size, gradient_min_rect_size;
> + int idx_zlib_level, mono_zlib_level, raw_zlib_level, gradient_zlib_level;
> + int gradient_threshold, gradient_threshold24;
> + int idx_max_colors_divisor;
> + int jpeg_quality, jpeg_threshold, jpeg_threshold24;
> +} tight_conf[] = {
> + { 512, 32, 6, 65536, 0, 0, 0, 0, 0, 0, 4, 5, 10000, 23000 },
> + { 2048, 128, 6, 65536, 1, 1, 1, 0, 0, 0, 8, 10, 8000, 18000 },
> + { 6144, 256, 8, 65536, 3, 3, 2, 0, 0, 0, 24, 15, 6500, 15000 },
> + { 10240, 1024, 12, 65536, 5, 5, 3, 0, 0, 0, 32, 25, 5000, 12000 },
> + { 16384, 2048, 12, 65536, 6, 6, 4, 0, 0, 0, 32, 37, 4000, 10000 },
> + { 32768, 2048, 12, 4096, 7, 7, 5, 4, 150, 380, 32, 50, 3000, 8000 },
> + { 65536, 2048, 16, 4096, 7, 7, 6, 4, 170, 420, 48, 60, 2000, 5000 },
> + { 65536, 2048, 16, 4096, 8, 8, 7, 5, 180, 450, 64, 70, 1000, 2500 },
> + { 65536, 2048, 32, 8192, 9, 9, 8, 6, 190, 475, 64, 75, 500, 1200 },
> + { 65536, 2048, 32, 8192, 9, 9, 9, 6, 200, 500, 96, 80, 200, 500 }
> +};
>
>
I assume this was borrowed from somewhere so we need to preserve the
original copyright statements.
> +static int tight_init_stream(VncState *vs, int stream_id,
> + int level, int strategy)
> +{
> + z_streamp zstream =&vs->tight_stream[stream_id];
> +
> + if (zstream->opaque != vs) {
> + int err;
>
IIUC, if zstream->opaque != vs, then it == NULL? I think the
affirmative check would make more sense.
> + VNC_DEBUG("VNC: TIGHT: initializing zlib stream %d\n", stream_id);
> + VNC_DEBUG("VNC: TIGHT: opaque = %p | vs = %p\n", zstream->opaque, vs);
> + zstream->zalloc = vnc_zlib_zalloc;
> + zstream->zfree = vnc_zlib_zfree;
> +
> + err = deflateInit2(zstream, level, Z_DEFLATED, MAX_WBITS,
> + MAX_MEM_LEVEL, strategy);
> +
> + if (err != Z_OK) {
> + fprintf(stderr, "VNC: error initializing zlib\n");
> + return -1;
> + }
> +
> + vs->tight_levels[stream_id] = level;
> + zstream->opaque = vs;
> + }
> +
> + if (vs->tight_levels[stream_id] != level) {
> + if (deflateParams (zstream, level, strategy) != Z_OK) {
>
There's an extra space in the function call.
> + return -1;
> + }
> + vs->tight_levels[stream_id] = level;
> + }
> + return 0;
> +}
> +
> +static void tight_send_compact_size(VncState *vs, size_t len)
> +{
> + int lpc = 0;
> + int bytes = 0;
> + char buf[3] = {0, 0, 0};
> +
> + buf[bytes++] = len& 0x7F;
> + if (len> 0x7F) {
> + buf[bytes-1] |= 0x80;
> + buf[bytes++] = len>> 7& 0x7F;
>
I'd suggest using parenthesis here for readability.
> + if (len> 0x3FFF) {
> + buf[bytes-1] |= 0x80;
> + buf[bytes++] = len>> 14& 0xFF;
>
Here too.
> + }
> + }
> + for(lpc = 0; lpc< bytes; lpc++) {
>
Missing space.
> + vnc_write_u8(vs, buf[lpc]);
> + }
> +}
> +
> +static int tight_compress_data(VncState *vs, int stream_id, size_t bytes,
> + int level, int strategy)
> +{
> + z_streamp zstream =&vs->tight_stream[stream_id];
> + int previous_out;
> +
> + if (bytes< VNC_TIGHT_MIN_TO_COMPRESS) {
> + vnc_write(vs, vs->tight.buffer, vs->tight.offset);
> + return bytes;
> + }
> +
> + if (tight_init_stream(vs, stream_id, level, strategy))
> + return -1;
>
Missing {}s.
> + // reserve memory in output buffer
> + buffer_reserve(&vs->tight_zlib, bytes + 64);
>
Please avoid C99 comments.
> + // set pointers
> + zstream->next_in = vs->tight.buffer;
> + zstream->avail_in = vs->tight.offset;
> + zstream->next_out = vs->tight_zlib.buffer + vs->tight_zlib.offset;
> + zstream->avail_out = vs->tight_zlib.capacity - vs->tight_zlib.offset;
> + zstream->data_type = Z_BINARY;
> + previous_out = zstream->total_out;
> +
> + // start encoding
> + if (deflate(zstream, Z_SYNC_FLUSH) != Z_OK) {
> + fprintf(stderr, "VNC: error during tight compression\n");
> + return -1;
> + }
> +
> + vs->tight_zlib.offset = vs->tight_zlib.capacity - zstream->avail_out;
> + bytes = zstream->total_out - previous_out;
> +
> + tight_send_compact_size(vs, bytes);
> + vnc_write(vs, vs->tight_zlib.buffer, bytes);
> +
> + buffer_reset(&vs->tight_zlib);
> +
> + return bytes;
> +}
> +
> +/*
> + * Subencoding implementations.
> + */
> +static void tight_pack24(VncState *vs, size_t count)
> +{
> + unsigned char *buf;
> + uint32_t *buf32;
> + uint32_t pix;
> + int rshift, gshift, bshift;
> +
> + buf = vs->tight.buffer;
> + buf32 = (uint32_t *)buf;
> +
> + if ((vs->clientds.flags& QEMU_BIG_ENDIAN_FLAG) ==
> + (vs->ds->surface->flags& QEMU_BIG_ENDIAN_FLAG)) {
> + rshift = vs->clientds.pf.rshift;
> + gshift = vs->clientds.pf.gshift;
> + bshift = vs->clientds.pf.bshift;
> + } else {
> + rshift = 24 - vs->clientds.pf.rshift;
> + gshift = 24 - vs->clientds.pf.gshift;
> + bshift = 24 - vs->clientds.pf.bshift;
>
Shouldn't this be 32 instead of 24?
> + }
> +
> + vs->tight.offset = count * 3;
> +
> + while (count--) {
> + pix = *buf32++;
> + *buf++ = (char)(pix>> rshift);
> + *buf++ = (char)(pix>> gshift);
> + *buf++ = (char)(pix>> bshift);
> + }
> +}
> +
> +static int send_full_color_rect(VncState *vs, int w, int h)
> +{
> + int stream = 0;
> + size_t bytes;
> +
> + vnc_write_u8(vs, stream<< 4); /* no flushing, no filter */
> +
> + if (vs->tight_pixel24) {
> + tight_pack24(vs, w * h);
> + bytes = 3;
> + } else {
> + bytes = vs->clientds.pf.bytes_per_pixel;
> + }
> +
> + bytes = tight_compress_data(vs, stream, w * h * bytes,
> + tight_conf[vs->tight_compression].raw_zlib_level,
> + Z_DEFAULT_STRATEGY);
> +
> + return (bytes>= 0);
> +}
> +
> +static void vnc_tight_start(VncState *vs)
> +{
> + buffer_reset(&vs->tight);
> +
> + // make the output buffer be the zlib buffer, so we can compress it later
> + vs->tight_tmp = vs->output;
> + vs->output = vs->tight;
> +}
> +
> +static void vnc_tight_stop(VncState *vs)
> +{
> + // switch back to normal output/zlib buffers
> + vs->tight = vs->output;
> + vs->output = vs->tight_tmp;
> +}
> +
> +static int send_sub_rect(VncState *vs, int x, int y, int w, int h)
> +{
> + vnc_framebuffer_update(vs, x, y, w, h, VNC_ENCODING_TIGHT);
> +
> + /*
> + * Convert pixels and store them in vs->tight
> + * We will probably rework that later, probably
> + * when adding other sub-encodings
> + */
> + vnc_tight_start(vs);
> + vnc_raw_send_framebuffer_update(vs, x, y, w, h);
> + vnc_tight_stop(vs);
> +
> + return send_full_color_rect(vs, w, h);
> +}
> +
> +static int send_rect_simple(VncState *vs, int x, int y, int w, int h)
> +{
> + int max_size, max_width;
> + int max_sub_width, max_sub_height;
> + int dx, dy;
> + int rw, rh;
> + int n = 0;
> +
> + max_size = tight_conf[vs->tight_compression].max_rect_size;
> + max_width = tight_conf[vs->tight_compression].max_rect_width;
> +
> + if (w> max_width || w * h> max_size) {
> + max_sub_width = (w> max_width) ? max_width : w;
>
We have a MAX() macro.
> + max_sub_height = max_size / max_sub_width;
> +
> + for (dy = 0; dy< h; dy += max_sub_height) {
> + for (dx = 0; dx< w; dx += max_width) {
> + rw = (dx + max_sub_width< w) ? max_sub_width : w - dx;
> + rh = (dy + max_sub_height< h) ? max_sub_height : h - dy;
> + n += send_sub_rect(vs, x+dx, y+dy, rw, rh);
> + }
> + }
> + } else {
> + n += send_sub_rect(vs, x, y, w, h);
> + }
> +
> + return n;
> +}
> +
> +void vnc_tight_init(VncState *vs)
> +{
> + int i;
> + for (i=0; i<ARRAY_SIZE(vs->tight_stream); i++)
> + vs->tight_stream[i].opaque = NULL;
>
Missing {}s
> +}
> +
> +int vnc_tight_send_framebuffer_update(VncState *vs, int x, int y,
> + int w, int h)
> +{
> + if (vs->clientds.pf.bytes_per_pixel == 4&& vs->clientds.pf.rmax == 0xFF&&
> + vs->clientds.pf.bmax == 0xFF&& vs->clientds.pf.gmax == 0xFF) {
> + vs->tight_pixel24 = true;
> + } else {
> + vs->tight_pixel24 = false;
> + }
> +
> + return send_rect_simple(vs, x, y, w, h);
> +}
> diff --git a/vnc-encoding-tight.h b/vnc-encoding-tight.h
> new file mode 100644
> index 0000000..108e184
> --- /dev/null
> +++ b/vnc-encoding-tight.h
> @@ -0,0 +1,199 @@
> +/*
> + * QEMU VNC display driver: tight encoding
> + *
> + * Copyright (C) 2006 Anthony Liguori<anthony@codemonkey.ws>
> + * Copyright (C) 2006 Fabrice Bellard
> + * Copyright (C) 2009 Red Hat, Inc
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this software and associated documentation files (the "Software"), to deal
> + * in the Software without restriction, including without limitation the rights
> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> + * copies of the Software, and to permit persons to whom the Software is
> + * furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
> + * THE SOFTWARE.
> + */
> +
> +#ifndef __QEMU_VNC_ENCODING_TIGHT_H
> +# define __QEMU_VNC_ENCODING_TIGHT_H
>
Please avoid the __ prefix and the space in #define.
> +/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> + * Tight Encoding.
> + *
> + *-- The first byte of each Tight-encoded rectangle is a "compression control
> + * byte". Its format is as follows (bit 0 is the least significant one):
> + *
> + * bit 0: if 1, then compression stream 0 should be reset;
> + * bit 1: if 1, then compression stream 1 should be reset;
> + * bit 2: if 1, then compression stream 2 should be reset;
> + * bit 3: if 1, then compression stream 3 should be reset;
> + * bits 7-4: if 1000 (0x08), then the compression type is "fill",
> + * if 1001 (0x09), then the compression type is "jpeg",
> + * if 0xxx, then the compression type is "basic",
> + * values greater than 1001 are not valid.
> + *
> + * If the compression type is "basic", then bits 6..4 of the
> + * compression control byte (those xxx in 0xxx) specify the following:
> + *
> + * bits 5-4: decimal representation is the index of a particular zlib
> + * stream which should be used for decompressing the data;
> + * bit 6: if 1, then a "filter id" byte is following this byte.
> + *
> + *-- The data that follows after the compression control byte described
> + * above depends on the compression type ("fill", "jpeg" or "basic").
> + *
> + *-- If the compression type is "fill", then the only pixel value follows, in
> + * client pixel format (see NOTE 1). This value applies to all pixels of the
> + * rectangle.
> + *
> + *-- If the compression type is "jpeg", the following data stream looks like
> + * this:
> + *
> + * 1..3 bytes: data size (N) in compact representation;
> + * N bytes: JPEG image.
> + *
> + * Data size is compactly represented in one, two or three bytes, according
> + * to the following scheme:
> + *
> + * 0xxxxxxx (for values 0..127)
> + * 1xxxxxxx 0yyyyyyy (for values 128..16383)
> + * 1xxxxxxx 1yyyyyyy zzzzzzzz (for values 16384..4194303)
> + *
> + * Here each character denotes one bit, xxxxxxx are the least significant 7
> + * bits of the value (bits 0-6), yyyyyyy are bits 7-13, and zzzzzzzz are the
> + * most significant 8 bits (bits 14-21). For example, decimal value 10000
> + * should be represented as two bytes: binary 10010000 01001110, or
> + * hexadecimal 90 4E.
> + *
> + *-- If the compression type is "basic" and bit 6 of the compression control
> + * byte was set to 1, then the next (second) byte specifies "filter id" which
> + * tells the decoder what filter type was used by the encoder to pre-process
> + * pixel data before the compression. The "filter id" byte can be one of the
> + * following:
> + *
> + * 0: no filter ("copy" filter);
> + * 1: "palette" filter;
> + * 2: "gradient" filter.
> + *
> + *-- If bit 6 of the compression control byte is set to 0 (no "filter id"
> + * byte), or if the filter id is 0, then raw pixel values in the client
> + * format (see NOTE 1) will be compressed. See below details on the
> + * compression.
> + *
> + *-- The "gradient" filter pre-processes pixel data with a simple algorithm
> + * which converts each color component to a difference between a "predicted"
> + * intensity and the actual intensity. Such a technique does not affect
> + * uncompressed data size, but helps to compress photo-like images better.
> + * Pseudo-code for converting intensities to differences is the following:
> + *
> + * P[i,j] := V[i-1,j] + V[i,j-1] - V[i-1,j-1];
> + * if (P[i,j]< 0) then P[i,j] := 0;
> + * if (P[i,j]> MAX) then P[i,j] := MAX;
> + * D[i,j] := V[i,j] - P[i,j];
> + *
> + * Here V[i,j] is the intensity of a color component for a pixel at
> + * coordinates (i,j). MAX is the maximum value of intensity for a color
> + * component.
> + *
> + *-- The "palette" filter converts true-color pixel data to indexed colors
> + * and a palette which can consist of 2..256 colors. If the number of colors
> + * is 2, then each pixel is encoded in 1 bit, otherwise 8 bits is used to
> + * encode one pixel. 1-bit encoding is performed such way that the most
> + * significant bits correspond to the leftmost pixels, and each raw of pixels
> + * is aligned to the byte boundary. When "palette" filter is used, the
> + * palette is sent before the pixel data. The palette begins with an unsigned
> + * byte which value is the number of colors in the palette minus 1 (i.e. 1
> + * means 2 colors, 255 means 256 colors in the palette). Then follows the
> + * palette itself which consist of pixel values in client pixel format (see
> + * NOTE 1).
> + *
> + *-- The pixel data is compressed using the zlib library. But if the data
> + * size after applying the filter but before the compression is less then 12,
> + * then the data is sent as is, uncompressed. Four separate zlib streams
> + * (0..3) can be used and the decoder should read the actual stream id from
> + * the compression control byte (see NOTE 2).
> + *
> + * If the compression is not used, then the pixel data is sent as is,
> + * otherwise the data stream looks like this:
> + *
> + * 1..3 bytes: data size (N) in compact representation;
> + * N bytes: zlib-compressed data.
> + *
> + * Data size is compactly represented in one, two or three bytes, just like
> + * in the "jpeg" compression method (see above).
> + *
> + *-- NOTE 1. If the color depth is 24, and all three color components are
> + * 8-bit wide, then one pixel in Tight encoding is always represented by
> + * three bytes, where the first byte is red component, the second byte is
> + * green component, and the third byte is blue component of the pixel color
> + * value. This applies to colors in palettes as well.
> + *
> + *-- NOTE 2. The decoder must reset compression streams' states before
> + * decoding the rectangle, if some of bits 0,1,2,3 in the compression control
> + * byte are set to 1. Note that the decoder must reset zlib streams even if
> + * the compression type is "fill" or "jpeg".
> + *
> + *-- NOTE 3. The "gradient" filter and "jpeg" compression may be used only
> + * when bits-per-pixel value is either 16 or 32, not 8.
> + *
> + *-- NOTE 4. The width of any Tight-encoded rectangle cannot exceed 2048
> + * pixels. If a rectangle is wider, it must be split into several rectangles
> + * and each one should be encoded separately.
> + *
> + */
>
Definitely need to carry the original copyright here.
> +#define VNC_TIGHT_EXPLICIT_FILTER 0x04
> +#define VNC_TIGHT_FILL 0x08
> +#define VNC_TIGHT_JPEG 0x09
> +#define VNC_TIGHT_MAX_SUBENCODING 0x09
> +
> +/* Filters to improve compression efficiency */
> +#define VNC_TIGHT_FILTER_COPY 0x00
> +#define VNC_TIGHT_FILTER_PALETTE 0x01
> +#define VNC_TIGHT_FILTER_GRADIENT 0x02
> +
> +/* Note: The following constant should not be changed. */
> +#define VNC_TIGHT_MIN_TO_COMPRESS 12
> +
> +/* The parameters below may be adjusted. */
> +#define VNC_TIGHT_MIN_SPLIT_RECT_SIZE 4096
> +#define VNC_TIGHT_MIN_SOLID_SUBRECT_SIZE 2048
> +#define VNC_TIGHT_MAX_SPLIT_TILE_SIZE 16
> +
> +/*
> + * min()/max()/clamp() macros that also do
> + * strict type-checking.. See the
> + * "unnecessary" pointer comparison.
> + */
> +#define min(x, y) ({ \
> + typeof(x) _min1 = (x); \
> + typeof(y) _min2 = (y); \
> + (void) (&_min1 ==&_min2); \
> + _min1< _min2 ? _min1 : _min2; })
> +
> +#define max(x, y) ({ \
> + typeof(x) _max1 = (x); \
> + typeof(y) _max2 = (y); \
> + (void) (&_max1 ==&_max2); \
> + _max1> _max2 ? _max1 : _max2; })
> +
> +#define clamp(val, min, max) ({ \
> + typeof(val) __val = (val); \
> + typeof(min) __min = (min); \
> + typeof(max) __max = (max); \
> + (void) (&__val ==&__min); \
> + (void) (&__val ==&__max); \
> + __val = __val< __min ? __min: __val; \
> + __val> __max ? __max: __val; })
>
You can drop these macros.
Regards,
Anthony Liguori
> +
> +#endif /* __QEMU_VNC_ENCODING_TIGHT_H */
> diff --git a/vnc-encoding-zlib.c b/vnc-encoding-zlib.c
> index 29dd1b7..17a61bf 100644
> --- a/vnc-encoding-zlib.c
> +++ b/vnc-encoding-zlib.c
> @@ -28,7 +28,7 @@
>
> #define ZALLOC_ALIGNMENT 16
>
> -static void *zalloc(void *x, unsigned items, unsigned size)
> +void *vnc_zlib_zalloc(void *x, unsigned items, unsigned size)
> {
> void *p;
>
> @@ -40,7 +40,7 @@ static void *zalloc(void *x, unsigned items, unsigned size)
> return (p);
> }
>
> -static void zfree(void *x, void *addr)
> +void vnc_zlib_zfree(void *x, void *addr)
> {
> qemu_free(addr);
> }
> @@ -72,8 +72,8 @@ static int vnc_zlib_stop(VncState *vs)
>
> VNC_DEBUG("VNC: initializing zlib stream\n");
> VNC_DEBUG("VNC: opaque = %p | vs = %p\n", zstream->opaque, vs);
> - zstream->zalloc = zalloc;
> - zstream->zfree = zfree;
> + zstream->zalloc = vnc_zlib_zalloc;
> + zstream->zfree = vnc_zlib_zfree;
>
> err = deflateInit2(zstream, vs->tight_compression, Z_DEFLATED, MAX_WBITS,
> MAX_MEM_LEVEL, Z_DEFAULT_STRATEGY);
> @@ -116,7 +116,7 @@ static int vnc_zlib_stop(VncState *vs)
> return zstream->total_out - previous_out;
> }
>
> -void vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
> +int vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
> {
> int old_offset, new_offset, bytes_written;
>
> @@ -131,14 +131,17 @@ void vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
> vnc_raw_send_framebuffer_update(vs, x, y, w, h);
> bytes_written = vnc_zlib_stop(vs);
>
> - if (bytes_written == -1)
> - return;
> + if (bytes_written == -1) {
> + return 0;
> + }
>
> // hack in the size
> new_offset = vs->output.offset;
> vs->output.offset = old_offset;
> vnc_write_u32(vs, bytes_written);
> vs->output.offset = new_offset;
> +
> + return 1;
> }
>
> void vnc_zlib_init(VncState *vs)
> diff --git a/vnc.c b/vnc.c
> index d0c0d00..c7b76ff 100644
> --- a/vnc.c
> +++ b/vnc.c
> @@ -638,7 +638,7 @@ static void vnc_write_pixels_generic(VncState *vs, void *pixels1, int size)
> }
> }
>
> -void vnc_raw_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
> +int vnc_raw_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
> {
> int i;
> uint8_t *row;
> @@ -649,23 +649,31 @@ void vnc_raw_send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
> vs->write_pixels(vs, row, w * ds_get_bytes_per_pixel(vs->ds));
> row += ds_get_linesize(vs->ds);
> }
> + return 1;
> }
>
> -static void send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
> +static int send_framebuffer_update(VncState *vs, int x, int y, int w, int h)
> {
> + int n = 0;
> +
> switch(vs->vnc_encoding) {
> case VNC_ENCODING_ZLIB:
> - vnc_zlib_send_framebuffer_update(vs, x, y, w, h);
> + n = vnc_zlib_send_framebuffer_update(vs, x, y, w, h);
> break;
> case VNC_ENCODING_HEXTILE:
> vnc_framebuffer_update(vs, x, y, w, h, VNC_ENCODING_HEXTILE);
> - vnc_hextile_send_framebuffer_update(vs, x, y, w, h);
> + n = vnc_hextile_send_framebuffer_update(vs, x, y, w, h);
> + break;
> + case VNC_ENCODING_TIGHT:
> + n = vnc_tight_send_framebuffer_update(vs, x, y, w, h);
> break;
> default:
> vnc_framebuffer_update(vs, x, y, w, h, VNC_ENCODING_RAW);
> - vnc_raw_send_framebuffer_update(vs, x, y, w, h);
> + n = vnc_raw_send_framebuffer_update(vs, x, y, w, h);
> break;
> }
> +
> + return n;
> }
>
> static void vnc_copy(VncState *vs, int src_x, int src_y, int dst_x, int dst_y, int w, int h)
> @@ -776,6 +784,7 @@ static int vnc_update_client(VncState *vs, int has_dirty)
> int y;
> int n_rectangles;
> int saved_offset;
> + int n;
>
> if (vs->output.offset&& !vs->audio_cap&& !vs->force_update)
> /* kernel send buffers are full -> drop frames to throttle */
> @@ -808,16 +817,18 @@ static int vnc_update_client(VncState *vs, int has_dirty)
> } else {
> if (last_x != -1) {
> int h = find_and_clear_dirty_height(vs, y, last_x, x);
> - send_framebuffer_update(vs, last_x * 16, y, (x - last_x) * 16, h);
> - n_rectangles++;
> + n = send_framebuffer_update(vs, last_x * 16, y,
> + (x - last_x) * 16, h);
> + n_rectangles += n;
> }
> last_x = -1;
> }
> }
> if (last_x != -1) {
> int h = find_and_clear_dirty_height(vs, y, last_x, x);
> - send_framebuffer_update(vs, last_x * 16, y, (x - last_x) * 16, h);
> - n_rectangles++;
> + n = send_framebuffer_update(vs, last_x * 16, y,
> + (x - last_x) * 16, h);
> + n_rectangles += n;
> }
> }
> vs->output.buffer[saved_offset] = (n_rectangles>> 8)& 0xFF;
> @@ -1600,8 +1611,10 @@ static void set_encodings(VncState *vs, int32_t *encodings, size_t n_encodings)
> unsigned int enc = 0;
>
> vnc_zlib_init(vs);
> + vnc_tight_init(vs);
> vs->features = 0;
> - vs->vnc_encoding = -1;
> + if (!vs->vnc_encoding)
> + vs->vnc_encoding = -1;
> vs->tight_compression = 9;
> vs->tight_quality = 9;
> vs->absolute = -1;
> @@ -1619,6 +1632,10 @@ static void set_encodings(VncState *vs, int32_t *encodings, size_t n_encodings)
> vs->features |= VNC_FEATURE_HEXTILE_MASK;
> set_encoding(vs, enc);
> break;
> + case VNC_ENCODING_TIGHT:
> + vs->features |= VNC_FEATURE_TIGHT_MASK;
> + set_encoding(vs, enc);
> + break;
> case VNC_ENCODING_ZLIB:
> vs->features |= VNC_FEATURE_ZLIB_MASK;
> set_encoding(vs, enc);
> diff --git a/vnc.h b/vnc.h
> index 96f3fe7..ec7f481 100644
> --- a/vnc.h
> +++ b/vnc.h
> @@ -166,6 +166,12 @@ struct VncState
> /* Tight */
> uint8_t tight_quality;
> uint8_t tight_compression;
> + uint8_t tight_pixel24;
> + Buffer tight;
> + Buffer tight_tmp;
> + Buffer tight_zlib;
> + int tight_levels[4];
> + z_stream tight_stream[4];
>
> /* Hextile */
> VncSendHextileTile *send_hextile_tile;
> @@ -391,13 +397,18 @@ void vnc_framebuffer_update(VncState *vs, int x, int y, int w, int h,
> void vnc_convert_pixel(VncState *vs, uint8_t *buf, uint32_t v);
>
> /* Encodings */
> -void vnc_raw_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
> +int vnc_raw_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
>
> -void vnc_hextile_send_framebuffer_update(VncState *vs, int x,
> +int vnc_hextile_send_framebuffer_update(VncState *vs, int x,
> int y, int w, int h);
> void vnc_hextile_set_pixel_conversion(VncState *vs, int generic);
>
> void vnc_zlib_init(VncState *vs);
> -void vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
> +void *vnc_zlib_zalloc(void *x, unsigned items, unsigned size);
> +void vnc_zlib_zfree(void *x, void *addr);
> +int vnc_zlib_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
> +
> +void vnc_tight_init(VncState *vs);
> +int vnc_tight_send_framebuffer_update(VncState *vs, int x, int y, int w, int h);
>
> #endif /* __QEMU_VNC_H */
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 06/10] vnc: add support for tight fill encoding
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 06/10] vnc: add support for tight fill encoding Corentin Chary
@ 2010-05-18 13:34 ` Anthony Liguori
0 siblings, 0 replies; 30+ messages in thread
From: Anthony Liguori @ 2010-05-18 13:34 UTC (permalink / raw)
To: Corentin Chary; +Cc: Adam Litke, qemu-devel, Alexander Graf
On 05/18/2010 07:49 AM, Corentin Chary wrote:
> Fill encoding detects rectangle using only one color and send only
> one pixel value.
>
> Signed-off-by: Corentin Chary<corentincj@iksaif.net>
> ---
> vnc-encoding-tight.c | 245 +++++++++++++++++++++++++++++++++++++++++++++++++-
> 1 files changed, 244 insertions(+), 1 deletions(-)
>
> diff --git a/vnc-encoding-tight.c b/vnc-encoding-tight.c
> index 0d328c2..5ea90c0 100644
> --- a/vnc-encoding-tight.c
> +++ b/vnc-encoding-tight.c
> @@ -54,6 +54,135 @@ static const struct {
> { 65536, 2048, 32, 8192, 9, 9, 9, 6, 200, 500, 96, 80, 200, 500 }
> };
>
> +/*
> + * Check if a rectangle is all of the same color. If needSameColor is
> + * set to non-zero, then also check that its color equals to the
> + * *colorPtr value. The result is 1 if the test is successfull, and in
> + * that case new color will be stored in *colorPtr.
> + */
> +
> +#define DEFINE_CHECK_SOLID_FUNCTION(bpp) \
> + \
> + static bool \
> + check_solid_tile##bpp(VncState *vs, int x, int y, int w, int h, \
> + uint32_t* color, bool samecolor) \
> + { \
> + VncDisplay *vd = vs->vd; \
> + uint##bpp##_t *fbptr; \
> + uint##bpp##_t c; \
> + int dx, dy; \
> + \
> + fbptr = (uint##bpp##_t *) \
> + (vd->server->data + y * ds_get_linesize(vs->ds) + \
> + x * ds_get_bytes_per_pixel(vs->ds)); \
> + \
> + c = *fbptr; \
> + if (samecolor&& (uint32_t)c != *color) \
> + return false; \
> + \
> + for (dy = 0; dy< h; dy++) { \
> + for (dx = 0; dx< w; dx++) { \
> + if (c != fbptr[dx]) \
> + return false; \
> + } \
> + fbptr = (uint##bpp##_t *) \
> + ((uint8_t *)fbptr + ds_get_linesize(vs->ds)); \
> + } \
> + \
> + *color = (uint32_t)c; \
> + return true; \
> + }
> +
> +DEFINE_CHECK_SOLID_FUNCTION(32)
> +DEFINE_CHECK_SOLID_FUNCTION(16)
> +DEFINE_CHECK_SOLID_FUNCTION(8)
> +
> +static bool check_solid_tile(VncState *vs, int x, int y, int w, int h,
> + uint32_t* color, bool samecolor)
> +{
> + VncDisplay *vd = vs->vd;
> +
> + switch(vd->server->pf.bytes_per_pixel) {
> + case 4:
> + return check_solid_tile32(vs, x, y, w, h, color, samecolor);
> + case 2:
> + return check_solid_tile16(vs, x, y, w, h, color, samecolor);
> + default:
> + return check_solid_tile8(vs, x, y, w, h, color, samecolor);
> + }
> +}
> +
> +static void find_best_solid_area(VncState *vs, int x, int y, int w, int h,
> + uint32_t color, int *w_ptr, int *h_ptr)
> +{
> + int dx, dy, dw, dh;
> + int w_prev;
> + int w_best = 0, h_best = 0;
> +
> + w_prev = w;
> +
> + for (dy = y; dy< y + h; dy += VNC_TIGHT_MAX_SPLIT_TILE_SIZE) {
> +
> + dh = min(VNC_TIGHT_MAX_SPLIT_TILE_SIZE, y + h - dy);
> + dw = min(VNC_TIGHT_MAX_SPLIT_TILE_SIZE, w_prev);
> +
> + if (!check_solid_tile(vs, x, dy, dw, dh,&color, true))
> + break;
> +
> + for (dx = x + dw; dx< x + w_prev;) {
> + dw = min(VNC_TIGHT_MAX_SPLIT_TILE_SIZE, x + w_prev - dx);
> +
> + if (!check_solid_tile(vs, dx, dy, dw, dh,&color, true))
> + break;
> + dx += dw;
> + }
> +
> + w_prev = dx - x;
> + if (w_prev * (dy + dh - y)> w_best * h_best) {
> + w_best = w_prev;
> + h_best = dy + dh - y;
> + }
> + }
> +
> + *w_ptr = w_best;
> + *h_ptr = h_best;
> +}
> +
> +static void extend_solid_area(VncState *vs, int x, int y, int w, int h,
> + uint32_t color, int *x_ptr, int *y_ptr,
> + int *w_ptr, int *h_ptr)
> +{
> + int cx, cy;
> +
> + /* Try to extend the area upwards. */
> + for ( cy = *y_ptr - 1;
> + cy>= y&& check_solid_tile(vs, *x_ptr, cy, *w_ptr, 1,&color, true);
> + cy-- );
> + *h_ptr += *y_ptr - (cy + 1);
> + *y_ptr = cy + 1;
>
>
Please check this file against CODING_STYLE. There are extra spaces in
the for()s and missing {}s.
Regards,
Anthony Liguori
> + /* ... downwards. */
> + for ( cy = *y_ptr + *h_ptr;
> + cy< y + h&&
> + check_solid_tile(vs, *x_ptr, cy, *w_ptr, 1,&color, true);
> + cy++ );
> + *h_ptr += cy - (*y_ptr + *h_ptr);
> +
> + /* ... to the left. */
> + for ( cx = *x_ptr - 1;
> + cx>= x&& check_solid_tile(vs, cx, *y_ptr, 1, *h_ptr,&color, true);
> + cx-- );
> + *w_ptr += *x_ptr - (cx + 1);
> + *x_ptr = cx + 1;
> +
> + /* ... to the right. */
> + for ( cx = *x_ptr + *w_ptr;
> + cx< x + w&&
> + check_solid_tile(vs, cx, *y_ptr, 1, *h_ptr,&color, true);
> + cx++ );
> + *w_ptr += cx - (*x_ptr + *w_ptr);
> +}
> +
> static int tight_init_stream(VncState *vs, int stream_id,
> int level, int strategy)
> {
> @@ -205,6 +334,23 @@ static int send_full_color_rect(VncState *vs, int w, int h)
> return (bytes>= 0);
> }
>
> +static int send_solid_rect(VncState *vs)
> +{
> + size_t bytes;
> +
> + vnc_write_u8(vs, VNC_TIGHT_FILL<< 4); /* no flushing, no filter */
> +
> + if (vs->tight_pixel24) {
> + tight_pack24(vs, 1);
> + bytes = 3;
> + } else {
> + bytes = vs->clientds.pf.bytes_per_pixel;
> + }
> +
> + vnc_write(vs, vs->tight.buffer, bytes);
> + return 1;
> +}
> +
> static void vnc_tight_start(VncState *vs)
> {
> buffer_reset(&vs->tight);
> @@ -237,6 +383,17 @@ static int send_sub_rect(VncState *vs, int x, int y, int w, int h)
> return send_full_color_rect(vs, w, h);
> }
>
> +static int send_sub_rect_solid(VncState *vs, int x, int y, int w, int h)
> +{
> + vnc_framebuffer_update(vs, x, y, w, h, VNC_ENCODING_TIGHT);
> +
> + vnc_tight_start(vs);
> + vnc_raw_send_framebuffer_update(vs, x, y, w, h);
> + vnc_tight_stop(vs);
> +
> + return send_solid_rect(vs);
> +}
> +
> static int send_rect_simple(VncState *vs, int x, int y, int w, int h)
> {
> int max_size, max_width;
> @@ -266,6 +423,82 @@ static int send_rect_simple(VncState *vs, int x, int y, int w, int h)
> return n;
> }
>
> +static int find_large_solid_color_rect(VncState *vs, int x, int y,
> + int w, int h, int max_rows)
> +{
> + int dx, dy, dw, dh;
> + int n = 0;
> +
> + /* Try to find large solid-color areas and send them separately. */
> +
> + for (dy = y; dy< y + h; dy += VNC_TIGHT_MAX_SPLIT_TILE_SIZE) {
> +
> + /* If a rectangle becomes too large, send its upper part now. */
> +
> + if (dy - y>= max_rows) {
> + n += send_rect_simple(vs, x, y, w, max_rows);
> + y += max_rows;
> + h -= max_rows;
> + }
> +
> + dh = min(VNC_TIGHT_MAX_SPLIT_TILE_SIZE, (y + h - dy));
> +
> + for (dx = x; dx< x + w; dx += VNC_TIGHT_MAX_SPLIT_TILE_SIZE) {
> + uint32_t color_value;
> + int x_best, y_best, w_best, h_best;
> +
> + dw = min(VNC_TIGHT_MAX_SPLIT_TILE_SIZE, (x + w - dx));
> +
> + if (!check_solid_tile(vs, dx, dy, dw, dh,&color_value, false))
> + continue ;
> +
> + /* Get dimensions of solid-color area. */
> +
> + find_best_solid_area(vs, dx, dy, w - (dx - x), h - (dy - y),
> + color_value,&w_best,&h_best);
> +
> + /* Make sure a solid rectangle is large enough
> + (or the whole rectangle is of the same color). */
> +
> + if (w_best * h_best != w * h&&
> + w_best * h_best< VNC_TIGHT_MIN_SOLID_SUBRECT_SIZE)
> + continue;
> +
> + /* Try to extend solid rectangle to maximum size. */
> +
> + x_best = dx; y_best = dy;
> + extend_solid_area(vs, x, y, w, h, color_value,
> +&x_best,&y_best,&w_best,&h_best);
> +
> + /* Send rectangles at top and left to solid-color area. */
> +
> + if (y_best != y)
> + n += send_rect_simple(vs, x, y, w, y_best-y);
> + if (x_best != x)
> + n += vnc_tight_send_framebuffer_update(vs, x, y_best,
> + x_best-x, h_best);
> +
> + /* Send solid-color rectangle. */
> + n += send_sub_rect_solid(vs, x_best, y_best, w_best, h_best);
> +
> + /* Send remaining rectangles (at right and bottom). */
> +
> + if (x_best + w_best != x + w)
> + n += vnc_tight_send_framebuffer_update(vs, x_best+w_best,
> + y_best,
> + w-(x_best-x)-w_best,
> + h_best);
> + if (y_best + h_best != y + h)
> + n += vnc_tight_send_framebuffer_update(vs, x, y_best+h_best,
> + w, h-(y_best-y)-h_best);
> +
> + /* Return after all recursive calls are done. */
> + return n;
> + }
> + }
> + return n + send_rect_simple(vs, x, y, w, h);
> +}
> +
> void vnc_tight_init(VncState *vs)
> {
> int i;
> @@ -276,6 +509,8 @@ void vnc_tight_init(VncState *vs)
> int vnc_tight_send_framebuffer_update(VncState *vs, int x, int y,
> int w, int h)
> {
> + int max_rows;
> +
> if (vs->clientds.pf.bytes_per_pixel == 4&& vs->clientds.pf.rmax == 0xFF&&
> vs->clientds.pf.bmax == 0xFF&& vs->clientds.pf.gmax == 0xFF) {
> vs->tight_pixel24 = true;
> @@ -283,5 +518,13 @@ int vnc_tight_send_framebuffer_update(VncState *vs, int x, int y,
> vs->tight_pixel24 = false;
> }
>
> - return send_rect_simple(vs, x, y, w, h);
> + if (w * h< VNC_TIGHT_MIN_SPLIT_RECT_SIZE)
> + return send_rect_simple(vs, x, y, w, h);
> +
> + /* Calculate maximum number of rows in one non-solid rectangle. */
> +
> + max_rows = tight_conf[vs->tight_compression].max_rect_size;
> + max_rows /= min(tight_conf[vs->tight_compression].max_rect_width, w);
> +
> + return find_large_solid_color_rect(vs, x, y, w, h, max_rows);
> }
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 10/10] vnc: update copyrights for vnc-encoding-tight.c
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 10/10] vnc: update copyrights for vnc-encoding-tight.c Corentin Chary
@ 2010-05-18 13:37 ` Anthony Liguori
2010-05-18 13:47 ` Chris Krumme
2010-05-18 17:26 ` Blue Swirl
0 siblings, 2 replies; 30+ messages in thread
From: Anthony Liguori @ 2010-05-18 13:37 UTC (permalink / raw)
To: Corentin Chary; +Cc: Adam Litke, qemu-devel, Alexander Graf
On 05/18/2010 07:49 AM, Corentin Chary wrote:
> Add missing copyrights in vnc-encoding-tight.c, since the code is
> mostly inspired from libvncserver/tight.c.
>
> Signed-off-by: Corentin Chary<corentincj@iksaif.net>
> ---
> vnc-encoding-tight.c | 5 +++++
> 1 files changed, 5 insertions(+), 0 deletions(-)
>
> diff --git a/vnc-encoding-tight.c b/vnc-encoding-tight.c
> index b836569..0c661a9 100644
> --- a/vnc-encoding-tight.c
> +++ b/vnc-encoding-tight.c
> @@ -1,9 +1,14 @@
> /*
> * QEMU VNC display driver: tight encoding
> *
> + * From libvncserver/tight.c
> + * Copyright (C) 1999 AT&T Laboratories Cambridge. All Rights Reserved.
> + * Copyright (C) 2000, 2001 Const Kaplinsky. All Rights Reserved.
> + *
> * Copyright (C) 2006 Anthony Liguori<anthony@codemonkey.ws>
> * Copyright (C) 2006 Fabrice Bellard
> * Copyright (C) 2009 Red Hat, Inc
> + * Copyright (C) 2019 Corentin Chary<corentin.chary@gmail.com>
> *
> * Permission is hereby granted, free of charge, to any person obtaining a copy
> * of this software and associated documentation files (the "Software"), to deal
>
This should come earlier in the series and also needs to carry a LICENSE.
If it's truly under 'All Rights Reserved', then we cannot incorporate
this code into QEMU.
Regards,
Anthony Liguori
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 03/10] vnc: only use a single zlib stream
2010-05-18 12:51 ` [Qemu-devel] " Alexander Graf
@ 2010-05-18 13:39 ` Anthony Liguori
2010-05-18 13:52 ` Alexander Graf
0 siblings, 1 reply; 30+ messages in thread
From: Anthony Liguori @ 2010-05-18 13:39 UTC (permalink / raw)
To: Alexander Graf; +Cc: Corentin Chary, qemu-devel, Adam Litke
On 05/18/2010 07:51 AM, Alexander Graf wrote:
> Corentin Chary wrote:
>
>> According to http://tigervnc.org/cgi-bin/rfbproto#zlib-encoding
>> zlib encoding only uses a single stream. Current implementation defines
>> 4 streams but only uses the first one. Remove them and only use a single
>> stream.
>>
>>
> How about when we start to implement zrle or zhextile? Wouldn't those
> need different streams?
>
Only tight uses multiple streams. zrle just uses one.
zhextile doesn't exist and if it did, it wouldn't be better than Tight
because Tight is essentially zhextile :-)
Regards,
Anthony Liguori
> Alex
>
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [Qemu-devel] Re: [PATCH v2 10/10] vnc: update copyrights for vnc-encoding-tight.c
2010-05-18 13:37 ` [Qemu-devel] " Anthony Liguori
@ 2010-05-18 13:47 ` Chris Krumme
2010-05-18 17:26 ` Blue Swirl
1 sibling, 0 replies; 30+ messages in thread
From: Chris Krumme @ 2010-05-18 13:47 UTC (permalink / raw)
To: qemu-devel
On 05/18/2010 08:37 AM, Anthony Liguori wrote:
> On 05/18/2010 07:49 AM, Corentin Chary wrote:
>> Add missing copyrights in vnc-encoding-tight.c, since the code is
>> mostly inspired from libvncserver/tight.c.
>>
>> Signed-off-by: Corentin Chary<corentincj@iksaif.net>
>> ---
>> vnc-encoding-tight.c | 5 +++++
>> 1 files changed, 5 insertions(+), 0 deletions(-)
>>
>> diff --git a/vnc-encoding-tight.c b/vnc-encoding-tight.c
>> index b836569..0c661a9 100644
>> --- a/vnc-encoding-tight.c
>> +++ b/vnc-encoding-tight.c
>> @@ -1,9 +1,14 @@
>> /*
>> * QEMU VNC display driver: tight encoding
>> *
>> + * From libvncserver/tight.c
>> + * Copyright (C) 1999 AT&T Laboratories Cambridge. All Rights
>> Reserved.
>> + * Copyright (C) 2000, 2001 Const Kaplinsky. All Rights Reserved.
>> + *
>> * Copyright (C) 2006 Anthony Liguori<anthony@codemonkey.ws>
>> * Copyright (C) 2006 Fabrice Bellard
>> * Copyright (C) 2009 Red Hat, Inc
>> + * Copyright (C) 2019 Corentin Chary<corentin.chary@gmail.com>
Maybe the laws are different in our future. 9 years from now this whole
IP issue may be resolved. (or you may want to correct the year also.)
Thanks
Chris
>> *
>> * Permission is hereby granted, free of charge, to any person
>> obtaining a copy
>> * of this software and associated documentation files (the
>> "Software"), to deal
>
> This should come earlier in the series and also needs to carry a LICENSE.
>
> If it's truly under 'All Rights Reserved', then we cannot incorporate
> this code into QEMU.
>
> Regards,
>
> Anthony Liguori
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 05/10] vnc: add basic tight support
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 05/10] vnc: add basic tight support Corentin Chary
2010-05-18 13:33 ` [Qemu-devel] " Anthony Liguori
@ 2010-05-18 13:49 ` Alexander Graf
2010-05-18 13:54 ` Anthony Liguori
1 sibling, 1 reply; 30+ messages in thread
From: Alexander Graf @ 2010-05-18 13:49 UTC (permalink / raw)
To: Corentin Chary; +Cc: Anthony Liguori, qemu-devel, Adam Litke
Corentin Chary wrote:
> Add support for tight encoding [1]. This patch only add support
> for "basic" tight compression without any filter.
>
> [1] http://tigervnc.org/cgi-bin/rfbproto#tight-encoding.
>
> Signed-off-by: Corentin Chary <corentincj@iksaif.net>
> ---
> Makefile | 2 +
> Makefile.objs | 1 +
> vnc-encoding-hextile.c | 5 +-
> vnc-encoding-tight.c | 287 ++++++++++++++++++++++++++++++++++++++++++++++++
> vnc-encoding-tight.h | 199 +++++++++++++++++++++++++++++++++
> vnc-encoding-zlib.c | 17 ++-
> vnc.c | 37 +++++--
> vnc.h | 17 +++-
> 8 files changed, 543 insertions(+), 22 deletions(-)
> create mode 100644 vnc-encoding-tight.c
> create mode 100644 vnc-encoding-tight.h
>
> diff --git a/Makefile b/Makefile
> index eb9e02b..99d9c5d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -124,6 +124,8 @@ vnc-encoding-zlib.o: vnc.h
>
> vnc-encoding-hextile.o: vnc.h
>
> +vnc-encoding-tight.o: vnc.h vnc-encoding-tight.h
> +
> curses.o: curses.c keymaps.h curses_keys.h
>
> bt-host.o: QEMU_CFLAGS += $(BLUEZ_CFLAGS)
> diff --git a/Makefile.objs b/Makefile.objs
> index acbaf22..070ee09 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -104,6 +104,7 @@ common-obj-$(CONFIG_SDL) += sdl.o sdl_zoom.o x_keymap.o
> common-obj-$(CONFIG_CURSES) += curses.o
> common-obj-y += vnc.o acl.o d3des.o
> common-obj-y += vnc-encoding-zlib.o vnc-encoding-hextile.o
> +common-obj-y += vnc-encoding-tight.o
> common-obj-y += iov.o
> common-obj-$(CONFIG_VNC_TLS) += vnc-tls.o vnc-auth-vencrypt.o
> common-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
> diff --git a/vnc-encoding-hextile.c b/vnc-encoding-hextile.c
> index a01c5e2..728f25e 100644
> --- a/vnc-encoding-hextile.c
> +++ b/vnc-encoding-hextile.c
> @@ -62,8 +62,8 @@ static void hextile_enc_cord(uint8_t *ptr, int x, int y, int w, int h)
> #undef BPP
> #undef GENERIC
>
> -void vnc_hextile_send_framebuffer_update(VncState *vs, int x,
> - int y, int w, int h)
> +int vnc_hextile_send_framebuffer_update(VncState *vs, int x,
> + int y, int w, int h)
>
IMHO this API change should go into a separate patch.
> {
> int i, j;
> int has_fg, has_bg;
> @@ -83,6 +83,7 @@ void vnc_hextile_send_framebuffer_update(VncState *vs, int x,
> free(last_fg);
> free(last_bg);
>
> + return 1;
> }
>
> void vnc_hextile_set_pixel_conversion(VncState *vs, int generic)
>
>
[...]
> diff --git a/vnc.h b/vnc.h
> index 96f3fe7..ec7f481 100644
> --- a/vnc.h
> +++ b/vnc.h
> @@ -166,6 +166,12 @@ struct VncState
> /* Tight */
> uint8_t tight_quality;
> uint8_t tight_compression;
> + uint8_t tight_pixel24;
> + Buffer tight;
> + Buffer tight_tmp;
> + Buffer tight_zlib;
> + int tight_levels[4];
> + z_stream tight_stream[4];
>
You just shrunk down things to a single stream for zlib, so why go with
4 here? Did I miss anything?
Alex
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 03/10] vnc: only use a single zlib stream
2010-05-18 13:39 ` Anthony Liguori
@ 2010-05-18 13:52 ` Alexander Graf
2010-05-18 13:55 ` Anthony Liguori
0 siblings, 1 reply; 30+ messages in thread
From: Alexander Graf @ 2010-05-18 13:52 UTC (permalink / raw)
To: Anthony Liguori; +Cc: Corentin Chary, qemu-devel, Adam Litke
Anthony Liguori wrote:
> On 05/18/2010 07:51 AM, Alexander Graf wrote:
>> Corentin Chary wrote:
>>
>>> According to http://tigervnc.org/cgi-bin/rfbproto#zlib-encoding
>>> zlib encoding only uses a single stream. Current implementation defines
>>> 4 streams but only uses the first one. Remove them and only use a
>>> single
>>> stream.
>>>
>>>
>> How about when we start to implement zrle or zhextile? Wouldn't those
>> need different streams?
>>
>
> Only tight uses multiple streams. zrle just uses one.
Ah, that's why I put it in there back then. I see :).
Alex
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 05/10] vnc: add basic tight support
2010-05-18 13:49 ` Alexander Graf
@ 2010-05-18 13:54 ` Anthony Liguori
0 siblings, 0 replies; 30+ messages in thread
From: Anthony Liguori @ 2010-05-18 13:54 UTC (permalink / raw)
To: Alexander Graf; +Cc: Corentin Chary, qemu-devel, Adam Litke
On 05/18/2010 08:49 AM, Alexander Graf wrote:
>> diff --git a/vnc.h b/vnc.h
>> index 96f3fe7..ec7f481 100644
>> --- a/vnc.h
>> +++ b/vnc.h
>> @@ -166,6 +166,12 @@ struct VncState
>> /* Tight */
>> uint8_t tight_quality;
>> uint8_t tight_compression;
>> + uint8_t tight_pixel24;
>> + Buffer tight;
>> + Buffer tight_tmp;
>> + Buffer tight_zlib;
>> + int tight_levels[4];
>> + z_stream tight_stream[4];
>>
>>
> You just shrunk down things to a single stream for zlib, so why go with
> 4 here? Did I miss anything?
>
Tight requires 4 streams.
Regards,
Anthony Liguori
> Alex
>
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 03/10] vnc: only use a single zlib stream
2010-05-18 13:52 ` Alexander Graf
@ 2010-05-18 13:55 ` Anthony Liguori
2010-05-18 13:56 ` Alexander Graf
0 siblings, 1 reply; 30+ messages in thread
From: Anthony Liguori @ 2010-05-18 13:55 UTC (permalink / raw)
To: Alexander Graf; +Cc: Corentin Chary, qemu-devel, Adam Litke
On 05/18/2010 08:52 AM, Alexander Graf wrote:
> Anthony Liguori wrote:
>
>> On 05/18/2010 07:51 AM, Alexander Graf wrote:
>>
>>> Corentin Chary wrote:
>>>
>>>
>>>> According to http://tigervnc.org/cgi-bin/rfbproto#zlib-encoding
>>>> zlib encoding only uses a single stream. Current implementation defines
>>>> 4 streams but only uses the first one. Remove them and only use a
>>>> single
>>>> stream.
>>>>
>>>>
>>>>
>>> How about when we start to implement zrle or zhextile? Wouldn't those
>>> need different streams?
>>>
>>>
>> Only tight uses multiple streams. zrle just uses one.
>>
> Ah, that's why I put it in there back then. I see :).
>
Keep in mind, tight's 4 streams are not the same as zrle or zlib's 1
stream. A server is free to send both Tight updates and zrle updates if
the client supports it. The client is going to expect that zrle is a
different compression stream than any of the 4 tight streams.
That's why you can't use the Tight compression level selection messages
to choose the zlib compression level btw.
Regards,
Anthony Liguori
> Alex
>
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 03/10] vnc: only use a single zlib stream
2010-05-18 13:55 ` Anthony Liguori
@ 2010-05-18 13:56 ` Alexander Graf
2010-05-18 13:58 ` Anthony Liguori
0 siblings, 1 reply; 30+ messages in thread
From: Alexander Graf @ 2010-05-18 13:56 UTC (permalink / raw)
To: Anthony Liguori; +Cc: Corentin Chary, qemu-devel, Adam Litke
Anthony Liguori wrote:
> On 05/18/2010 08:52 AM, Alexander Graf wrote:
>> Anthony Liguori wrote:
>>
>>> On 05/18/2010 07:51 AM, Alexander Graf wrote:
>>>
>>>> Corentin Chary wrote:
>>>>
>>>>
>>>>> According to http://tigervnc.org/cgi-bin/rfbproto#zlib-encoding
>>>>> zlib encoding only uses a single stream. Current implementation
>>>>> defines
>>>>> 4 streams but only uses the first one. Remove them and only use a
>>>>> single
>>>>> stream.
>>>>>
>>>>>
>>>>>
>>>> How about when we start to implement zrle or zhextile? Wouldn't those
>>>> need different streams?
>>>>
>>>>
>>> Only tight uses multiple streams. zrle just uses one.
>>>
>> Ah, that's why I put it in there back then. I see :).
>>
>
> Keep in mind, tight's 4 streams are not the same as zrle or zlib's 1
> stream. A server is free to send both Tight updates and zrle updates
> if the client supports it. The client is going to expect that zrle is
> a different compression stream than any of the 4 tight streams.
>
> That's why you can't use the Tight compression level selection
> messages to choose the zlib compression level btw.
So patch 4 is invalid?
Alex
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 03/10] vnc: only use a single zlib stream
2010-05-18 13:56 ` Alexander Graf
@ 2010-05-18 13:58 ` Anthony Liguori
0 siblings, 0 replies; 30+ messages in thread
From: Anthony Liguori @ 2010-05-18 13:58 UTC (permalink / raw)
To: Alexander Graf; +Cc: Corentin Chary, qemu-devel, Adam Litke
On 05/18/2010 08:56 AM, Alexander Graf wrote:
> So patch 4 is invalid?
>
Technically, yes although I'm not sure that it would necessarily break a
client (since the client's request of compression levels is really just
advisory).
Regards,
Anthony Liguori
> Alex
>
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [Qemu-devel] Re: [PATCH v2 10/10] vnc: update copyrights for vnc-encoding-tight.c
2010-05-18 13:37 ` [Qemu-devel] " Anthony Liguori
2010-05-18 13:47 ` Chris Krumme
@ 2010-05-18 17:26 ` Blue Swirl
1 sibling, 0 replies; 30+ messages in thread
From: Blue Swirl @ 2010-05-18 17:26 UTC (permalink / raw)
To: Anthony Liguori; +Cc: Corentin Chary, Alexander Graf, qemu-devel, Adam Litke
On 5/18/10, Anthony Liguori <aliguori@linux.vnet.ibm.com> wrote:
> On 05/18/2010 07:49 AM, Corentin Chary wrote:
>
> > Add missing copyrights in vnc-encoding-tight.c, since the code is
> > mostly inspired from libvncserver/tight.c.
> >
> > Signed-off-by: Corentin Chary<corentincj@iksaif.net>
> > ---
> > vnc-encoding-tight.c | 5 +++++
> > 1 files changed, 5 insertions(+), 0 deletions(-)
> >
> > diff --git a/vnc-encoding-tight.c b/vnc-encoding-tight.c
> > index b836569..0c661a9 100644
> > --- a/vnc-encoding-tight.c
> > +++ b/vnc-encoding-tight.c
> > @@ -1,9 +1,14 @@
> > /*
> > * QEMU VNC display driver: tight encoding
> > *
> > + * From libvncserver/tight.c
> > + * Copyright (C) 1999 AT&T Laboratories Cambridge. All Rights Reserved.
> > + * Copyright (C) 2000, 2001 Const Kaplinsky. All Rights Reserved.
> > + *
> > * Copyright (C) 2006 Anthony Liguori<anthony@codemonkey.ws>
> > * Copyright (C) 2006 Fabrice Bellard
> > * Copyright (C) 2009 Red Hat, Inc
> > + * Copyright (C) 2019 Corentin
> Chary<corentin.chary@gmail.com>
> > *
> > * Permission is hereby granted, free of charge, to any person obtaining
> a copy
> > * of this software and associated documentation files (the "Software"),
> to deal
> >
> >
>
> This should come earlier in the series and also needs to carry a LICENSE.
>
> If it's truly under 'All Rights Reserved', then we cannot incorporate this
> code into QEMU.
Yes we can. I made the same mistake earlier:
http://lists.gnu.org/archive/html/qemu-devel/2009-02/msg01105.html
http://en.wikipedia.org/wiki/All_rights_reserved
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 01/10] vnc: refactor set_encodings
2010-05-18 12:54 ` [Qemu-devel] " Alexander Graf
@ 2010-05-18 18:09 ` Corentin Chary
2010-05-18 18:18 ` Alexander Graf
0 siblings, 1 reply; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 18:09 UTC (permalink / raw)
To: Alexander Graf; +Cc: Anthony Liguori, qemu-devel, Adam Litke
On Tue, May 18, 2010 at 2:54 PM, Alexander Graf <agraf@suse.de> wrote:
> Corentin Chary wrote:
>> Create a new set_encoding() function to remove
>> duplicated code in set_encodings().
>>
>> Signed-off-by: Corentin Chary <corentincj@iksaif.net>
>>
>
> Acked-by: Alexander Graf <agraf@suse.de>
>
> Keep in mind that this still means that the last requested compression
> algorithm is used, which is reverse according to spec.
>
I didn't notice that the loop was reversed, and that the right
encoding was selected *before* my patch.
I think the right thing to do is to reverse my previous patch and add
a comment at the begining of the loop.
I can also just make a patch to reverse the loop.
--
Corentin Chary
http://xf.iksaif.net
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 01/10] vnc: refactor set_encodings
2010-05-18 18:09 ` Corentin Chary
@ 2010-05-18 18:18 ` Alexander Graf
2010-05-18 19:23 ` Corentin Chary
0 siblings, 1 reply; 30+ messages in thread
From: Alexander Graf @ 2010-05-18 18:18 UTC (permalink / raw)
To: Corentin Chary; +Cc: Anthony Liguori, qemu-devel, Adam Litke
On 18.05.2010, at 20:09, Corentin Chary wrote:
> On Tue, May 18, 2010 at 2:54 PM, Alexander Graf <agraf@suse.de> wrote:
>> Corentin Chary wrote:
>>> Create a new set_encoding() function to remove
>>> duplicated code in set_encodings().
>>>
>>> Signed-off-by: Corentin Chary <corentincj@iksaif.net>
>>>
>>
>> Acked-by: Alexander Graf <agraf@suse.de>
>>
>> Keep in mind that this still means that the last requested compression
>> algorithm is used, which is reverse according to spec.
>>
>
> I didn't notice that the loop was reversed, and that the right
> encoding was selected *before* my patch.
> I think the right thing to do is to reverse my previous patch and add
> a comment at the begining of the loop.
I don't see your patch reversing the logic?
Alex
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Qemu-devel] Re: [PATCH v2 01/10] vnc: refactor set_encodings
2010-05-18 18:18 ` Alexander Graf
@ 2010-05-18 19:23 ` Corentin Chary
0 siblings, 0 replies; 30+ messages in thread
From: Corentin Chary @ 2010-05-18 19:23 UTC (permalink / raw)
To: Alexander Graf; +Cc: Anthony Liguori, qemu-devel, Adam Litke
> I don't see your patch reversing the logic?
>
> Alex
>
>
Before my patch (not this one, but
http://git.qemu.org/qemu.git/commit/?id=14eb8b6829ad9dee7035de729e083844a425f274
), we looped from the last encoding to the first (for (i = n_encodings
- 1; i >= 0; i--)), so the code was ok.
Commit 14eb8b6829ad9dee7035de729e083844a425f274 was wrong because I
wanted `if (encoding == -1) encoding = enc`, not
`if (encoding != -1) encoding = enc`. With the current code the
encoding will never be set and will always be -1.
And this patch (refactor set_encoding) is also wrong, because it will
only set the first encoding (which is in fact the last, because we
start from the end of the array).
I believe that the right thing to do is to revert
14eb8b6829ad9dee7035de729e083844a425f274 and add some comments to
explain why we loop in reverse order.
--
Corentin Chary
http://xf.iksaif.net
^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2010-05-18 19:24 UTC | newest]
Thread overview: 30+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-05-18 12:49 [Qemu-devel] [PATCH v2 00/10] vnc-encodings Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 01/10] vnc: refactor set_encodings Corentin Chary
2010-05-18 12:54 ` [Qemu-devel] " Alexander Graf
2010-05-18 18:09 ` Corentin Chary
2010-05-18 18:18 ` Alexander Graf
2010-05-18 19:23 ` Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 02/10] vnc: really call zlib if we want zlib Corentin Chary
2010-05-18 12:54 ` [Qemu-devel] " Alexander Graf
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 03/10] vnc: only use a single zlib stream Corentin Chary
2010-05-18 12:51 ` [Qemu-devel] " Alexander Graf
2010-05-18 13:39 ` Anthony Liguori
2010-05-18 13:52 ` Alexander Graf
2010-05-18 13:55 ` Anthony Liguori
2010-05-18 13:56 ` Alexander Graf
2010-05-18 13:58 ` Anthony Liguori
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 04/10] vnc: adjust compression zstream level Corentin Chary
2010-05-18 12:55 ` [Qemu-devel] " Alexander Graf
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 05/10] vnc: add basic tight support Corentin Chary
2010-05-18 13:33 ` [Qemu-devel] " Anthony Liguori
2010-05-18 13:49 ` Alexander Graf
2010-05-18 13:54 ` Anthony Liguori
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 06/10] vnc: add support for tight fill encoding Corentin Chary
2010-05-18 13:34 ` [Qemu-devel] " Anthony Liguori
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 07/10] vnc: don't clear zlib stream on set_encoding Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 08/10] vnc: remove memory leaks in zlib and tight encoding Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 09/10] vnc: tight: add palette encoding Corentin Chary
2010-05-18 12:49 ` [Qemu-devel] [PATCH v2 10/10] vnc: update copyrights for vnc-encoding-tight.c Corentin Chary
2010-05-18 13:37 ` [Qemu-devel] " Anthony Liguori
2010-05-18 13:47 ` Chris Krumme
2010-05-18 17:26 ` Blue Swirl
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).