[PULL 12/26] audio: prevent an integer overflow in resampling code

[Gerd Hoffmann <kraxel@redhat.com>]

From: Volker Rümelin <vr_qemu@t-online.de>

There are corner cases where rate->opos can overflow. For
example, if QEMU is started with -audiodev pa,id=audio0,
out.frequency=11025 -device ich9-intel-hda -device hda-duplex,
audiodev=audio0 and the guest plays audio with a sampling
frequency of 44100Hz, rate->opos will overflow after 27.05h
and the audio stream will be silent for a long time.

To prevent a rate->opos and also a rate->ipos overflow, both
are wrapped around after a short time. The wrap around point
rate->ipos >= 0x10001 is an arbitrarily selected value and can
be any small value, 0 and 1 included.

The comment that an ipos overflow will result in an infinite
loop has been removed, because in this case the resampling code
only generates no more output samples and the audio stream stalls.
However, there is no infinite loop.

Signed-off-by: Volker Rümelin <vr_qemu@t-online.de>
Message-Id: <20220923183640.8314-12-vr_qemu@t-online.de>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 audio/rate_template.h | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/audio/rate_template.h b/audio/rate_template.h
index f94c940c61b1..b432719ebbaa 100644
--- a/audio/rate_template.h
+++ b/audio/rate_template.h
@@ -72,11 +72,6 @@ void NAME (void *opaque, struct st_sample *ibuf, struct st_sample *obuf,
             ilast = *ibuf++;
             rate->ipos++;
 
-            /* if ipos overflow, there is  a infinite loop */
-            if (rate->ipos == 0xffffffff) {
-                rate->ipos = 1;
-                rate->opos = rate->opos & 0xffffffff;
-            }
             /* See if we finished the input buffer yet */
             if (ibuf >= iend) {
                 goto the_end;
@@ -85,6 +80,12 @@ void NAME (void *opaque, struct st_sample *ibuf, struct st_sample *obuf,
 
         icur = *ibuf;
 
+        /* wrap ipos and opos around long before they overflow */
+        if (rate->ipos >= 0x10001) {
+            rate->ipos = 1;
+            rate->opos &= 0xffffffff;
+        }
+
         /* interpolate */
 #ifdef FLOAT_MIXENG
 #ifdef RECIPROCAL
-- 
2.37.3