[Qemu-devel] [PATCH v2] build: Work around SIZE_MAX bug in OSX headers

[Eric Blake <eblake@redhat.com>]

C99 requires SIZE_MAX to be declared with the same type as the
integral promotion of size_t, but OSX mistakenly defines it as
an 'unsigned long long' expression even though size_t is only
'unsigned long'.  Rather than futzing around with whether size_t
is 32- or 64-bits wide (which would be needed if we cared about
using SIZE_T in a #if expression), let the compiler get the right
type for us by virtue of integer promotion.

See also https://patchwork.ozlabs.org/patch/542327/ for an
instance where the wrong type trips us up if we don't fix it
for good in osdep.h.

Some versions of glibc make a similar mistake with SSIZE_MAX; the
goal is that the approach of this patch could be copied to work
around that problem if it ever becomes important to us.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v1 was here:
https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg02520.html

The topic recently came up again, and I noticed this patch sitting
on one of my older branches, so I've taken another shot at it.
https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg00950.html

v2: rewrite into a configure check (not sure if directly adding a
-D to QEMU_CFLAGS is the best, so advice welcome)

I lack easy access to a Mac box, so this is untested as to whether
it actually solves the issue...
---
 include/qemu/osdep.h |  8 ++++++++
 configure            | 13 +++++++++++++
 2 files changed, 21 insertions(+)

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index 9e9fa61..22667f6 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -141,6 +141,14 @@ extern int daemon(int, int);
 # error Unknown pointer size
 #endif

+/* Mac OSX has a <stdint.h> bug that incorrectly defines SIZE_MAX with
+ * the wrong type. Our replacement isn't usable in preprocessor
+ * expressions, but it is sufficient for our needs. */
+#ifdef SIZE_MAX_BROKEN
+#undef SIZE_MAX
+#define SIZE_MAX ((sizeof(char)) * -1)
+#endif
+
 #ifndef MIN
 #define MIN(a, b) (((a) < (b)) ? (a) : (b))
 #endif
diff --git a/configure b/configure
index 5751d8e..137ebb8 100755
--- a/configure
+++ b/configure
@@ -1725,6 +1725,19 @@ if test "$cocoa" = "yes"; then
     sdl=no
 fi

+# Some versions of Mac OS X incorrectly define SIZE_MAX
+cat > $TMPC << EOF
+#include <stdint.h>
+#include <stdio.h>
+int main(int argc, char *argv[]) {
+    return printf("%zu", SIZE_MAX);
+}
+EOF
+
+if ! compile_object -Werror ; then
+    QEMU_CFLAGS="-DSIZE_MAX_BROKEN $QEMU_CFLAGS"
+fi
+
 ##########################################
 # L2TPV3 probe

-- 
2.7.4