* [Qemu-devel] [PATCH] vnc: Fix tight_detect_smooth_image() for lossless case
@ 2014-02-21 15:42 Markus Armbruster
2014-03-10 10:10 ` Markus Armbruster
0 siblings, 1 reply; 3+ messages in thread
From: Markus Armbruster @ 2014-02-21 15:42 UTC (permalink / raw)
To: qemu-devel; +Cc: kraxel
VncTight member uint8_t quality is either (uint8_t)-1 for lossless or
less than 10 for lossy.
tight_detect_smooth_image() first promotes it to int, then compares
with -1. Always unequal, so we always execute the lossy code. Reads
beyond tight_conf[] and returns crap when quality is actually
lossless.
Compare to (uint8_t)-1 instead, like we do elsewhere.
Spotted by Coverity.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
ui/vnc-enc-tight.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/ui/vnc-enc-tight.c b/ui/vnc-enc-tight.c
index e6966ae..59b59c0 100644
--- a/ui/vnc-enc-tight.c
+++ b/ui/vnc-enc-tight.c
@@ -330,7 +330,7 @@ tight_detect_smooth_image(VncState *vs, int w, int h)
} else {
errors = tight_detect_smooth_image16(vs, w, h);
}
- if (quality != -1) {
+ if (quality != (uint8_t)-1) {
return (errors < tight_conf[quality].jpeg_threshold);
}
return (errors < tight_conf[compression].gradient_threshold);
--
1.8.1.4
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [Qemu-devel] [PATCH] vnc: Fix tight_detect_smooth_image() for lossless case
2014-02-21 15:42 [Qemu-devel] [PATCH] vnc: Fix tight_detect_smooth_image() for lossless case Markus Armbruster
@ 2014-03-10 10:10 ` Markus Armbruster
2014-03-10 10:23 ` Gerd Hoffmann
0 siblings, 1 reply; 3+ messages in thread
From: Markus Armbruster @ 2014-03-10 10:10 UTC (permalink / raw)
To: qemu-devel; +Cc: kraxel
Ping?
Markus Armbruster <armbru@redhat.com> writes:
> VncTight member uint8_t quality is either (uint8_t)-1 for lossless or
> less than 10 for lossy.
>
> tight_detect_smooth_image() first promotes it to int, then compares
> with -1. Always unequal, so we always execute the lossy code. Reads
> beyond tight_conf[] and returns crap when quality is actually
> lossless.
>
> Compare to (uint8_t)-1 instead, like we do elsewhere.
>
> Spotted by Coverity.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> ui/vnc-enc-tight.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/ui/vnc-enc-tight.c b/ui/vnc-enc-tight.c
> index e6966ae..59b59c0 100644
> --- a/ui/vnc-enc-tight.c
> +++ b/ui/vnc-enc-tight.c
> @@ -330,7 +330,7 @@ tight_detect_smooth_image(VncState *vs, int w, int h)
> } else {
> errors = tight_detect_smooth_image16(vs, w, h);
> }
> - if (quality != -1) {
> + if (quality != (uint8_t)-1) {
> return (errors < tight_conf[quality].jpeg_threshold);
> }
> return (errors < tight_conf[compression].gradient_threshold);
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2014-03-10 10:23 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-02-21 15:42 [Qemu-devel] [PATCH] vnc: Fix tight_detect_smooth_image() for lossless case Markus Armbruster
2014-03-10 10:10 ` Markus Armbruster
2014-03-10 10:23 ` Gerd Hoffmann
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).