[patch] printf.3: Prevent signed integer overflow in example

[Tobias Stoeckmann <tobias@stoeckmann.org>]

The function make_message illustrates how to use vsnprintf to determine
the required amount of memory for a specific format and its arguments.

If make_message is called with a format which will use exactly INT_MAX
characters (excluding '\0'), then the size++ calculation will overflow
the signed integer "size", which is an undefined behaviour in C.

Since malloc and vsnprintf rightfully take a size_t argument, I decided
to use a size_t variable for size calculation. Therefore, this patched
code uses variables of the same data types as expected by function
arguments.

Proof of concept (tested on Linux/glibc amd64):

int main() { make_message("%647s%2147483000s", "", ""); }

If the code is compiled with address sanitizer (gcc -fsanitize=address)
you can see the following line, assuming that a signed integer overflow
simply leads to INT_MIN:

==3094==WARNING: AddressSanitizer failed to allocate 0xffffffff80000000 bytes
---
 man3/printf.3 | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/man3/printf.3 b/man3/printf.3
index 50e136ba6..827d9cbae 100644
--- a/man3/printf.3
+++ b/man3/printf.3
@@ -1132,29 +1132,32 @@ To allocate a sufficiently large string and print into it
 char *
 make_message(const char *fmt, ...)
 {
-    int size = 0;
+    int n = 0;
+    size_t size = 0;
     char *p = NULL;
     va_list ap;
 
     /* Determine required size */
 
     va_start(ap, fmt);
-    size = vsnprintf(p, size, fmt, ap);
+    n = vsnprintf(p, size, fmt, ap);
     va_end(ap);
 
-    if (size < 0)
+    if (n < 0)
         return NULL;
 
-    size++;             /* For '\e0' */
+    /* One extra byte for '\e0' */
+
+    size = (size_t) n + 1;
     p = malloc(size);
     if (p == NULL)
         return NULL;
 
     va_start(ap, fmt);
-    size = vsnprintf(p, size, fmt, ap);
+    n = vsnprintf(p, size, fmt, ap);
     va_end(ap);
 
-    if (size < 0) {
+    if (n < 0) {
         free(p);
         return NULL;
     }
-- 
2.26.2