Adding a realloc() usage note to the malloc(3) manual page

Adding a realloc() usage note to the malloc(3) manual page

[Michael Kerrisk (man-pages)]

Hello all,

I plan to add a note on the correct usage of realloc() to the
malloc(3) manual page. I would be happy to receive comments and
improvements to the text below.

Thanks,

Michael


   Correct usage of realloc() (and reallocarray())
       Since, on the one hand, realloc() (and  reallocarray())  may  move
       the block of memory, and on the other, it may return NULL on fail‐
       ure and leave the memory contents and location unchanged,  correct
       usage is something like the following:

           void *ptr, *nptr;
           ptr = malloc(origsize);
           ...
           /* In the following, we presume 'newsize' is not 0.
              (If 'newsize' is zero, realloc() may return NULL,
              and that is not an error.) */

           nptr = realloc(ptr, newsize);
           if (nptr == NULL) {
               /* Handle error; the block pointed to by 'ptr' is
                  still usable. */
           } else {
               /* realloc() succeeded; update 'ptr' to point to
                  the (possibly moved) block. */
               ptr = nptr;
           }

       Similar remarks apply for reallocarray().
-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: Adding a realloc() usage note to the malloc(3) manual page

[Florian Weimer]

* Michael Kerrisk:

>    Correct usage of realloc() (and reallocarray())
>        Since, on the one hand, realloc() (and  reallocarray())  may  move
>        the block of memory, and on the other, it may return NULL on fail‐
>        ure and leave the memory contents and location unchanged,  correct
>        usage is something like the following:
>
>            void *ptr, *nptr;
>            ptr = malloc(origsize);
>            ...
>            /* In the following, we presume 'newsize' is not 0.
>               (If 'newsize' is zero, realloc() may return NULL,
>               and that is not an error.) */
>
>            nptr = realloc(ptr, newsize);
>            if (nptr == NULL) {
>                /* Handle error; the block pointed to by 'ptr' is
>                   still usable. */
>            } else {
>                /* realloc() succeeded; update 'ptr' to point to
>                   the (possibly moved) block. */
>                ptr = nptr;
>            }

Maybe add this to the else branch: “The original value of ptr itself and
the memory it pointed to are invalid at this point.”

That is, the somewhat common idiom of adjusting internal pointers in the
allocation to point to the new allocation is invalid.

Thanks,
Florian

Re: Adding a realloc() usage note to the malloc(3) manual page

[Paul Eggert]

On 8/31/21 12:07 AM, Florian Weimer wrote:
> the somewhat common idiom of adjusting internal pointers in the
> allocation to point to the new allocation is invalid.

Good point. Also, the example call to malloc should check the return value.

Something like this, perhaps:

   char *ptr = malloc(origsize);
   if (ptr == NULL)
     return NULL;
   char *p = ptr + some_random_value();

   /* In the following, we presume 'newsize' is not 0.
      (If 'newsize' is zero, realloc() may return NULL,
      and that is not an error.) */

   ptrdiff_t p_offset = p - ptr;
   char *nptr = realloc(ptr, newsize);
   if (nptr == NULL) {
     /* Handle error; the block pointed to by 'ptr' is
        still usable. */
   } else {
     /* realloc() succeeded; update 'ptr' and 'p' to point to
        the (possibly moved) block.  'p += nptr - ptr; ptr = nptr;'
        would be invalid here, since 'ptr' is invalid immediately
        after the successful realloc().  */
     ptr = nptr;
     p = nptr + p_offset;
   }

Re: Adding a realloc() usage note to the malloc(3) manual page

[Michael Kerrisk (man-pages)]

On Tue, 31 Aug 2021 at 11:29, Paul Eggert <eggert@cs.ucla.edu> wrote:
>
> On 8/31/21 12:07 AM, Florian Weimer wrote:
> > the somewhat common idiom of adjusting internal pointers in the
> > allocation to point to the new allocation is invalid.
>
> Good point.

Yes, thanks, Florian.

> Also, the example call to malloc should check the return value.

yes...

> Something like this, perhaps:>
>    char *ptr = malloc(origsize);
>    if (ptr == NULL)
>      return NULL;
>    char *p = ptr + some_random_value();
>
>    /* In the following, we presume 'newsize' is not 0.
>       (If 'newsize' is zero, realloc() may return NULL,
>       and that is not an error.) */
>
>    ptrdiff_t p_offset = p - ptr;
>    char *nptr = realloc(ptr, newsize);
>    if (nptr == NULL) {
>      /* Handle error; the block pointed to by 'ptr' is
>         still usable. */
>    } else {
>      /* realloc() succeeded; update 'ptr' and 'p' to point to
>         the (possibly moved) block.  'p += nptr - ptr; ptr = nptr;'
>         would be invalid here, since 'ptr' is invalid immediately
>         after the successful realloc().  */

I don't get something here. Obviously, '*ptr' is invalid after a
successful realloc(). But why is 'ptr' invalid?

Thanks,

Michael


>      ptr = nptr;
>      p = nptr + p_offset;
>    }



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: Adding a realloc() usage note to the malloc(3) manual page

[Paul Eggert]

On 9/1/21 5:21 PM, Michael Kerrisk (man-pages) wrote:
> Obviously, '*ptr' is invalid after a
> successful realloc(). But why is 'ptr' invalid?

The C standard says that once a program has freed a non-null pointer, 
the program cannot look at the pointer's value any more. It can't copy 
the value to another pointer, and it can't even test whether the value 
is null. A debugging implementation can trap any use of the pointer's value.

This rule is not an issue for production uses of glibc realloc. However, 
it could at least in theory affect debugging implementations like gcc 
-fsanitize=address, because the rule can help catch bugs in programs. 
And I vaguely recall talk that a few non-glibc platforms (IBM i, 
perhaps?) enforce the rule.

Re: Adding a realloc() usage note to the malloc(3) manual page

[Michael Kerrisk (man-pages)]

Hello Paul,

On Thu, 2 Sept 2021 at 03:23, Paul Eggert <eggert@cs.ucla.edu> wrote:
>
> On 9/1/21 5:21 PM, Michael Kerrisk (man-pages) wrote:
> > Obviously, '*ptr' is invalid after a
> > successful realloc(). But why is 'ptr' invalid?
>
> The C standard says that once a program has freed a non-null pointer,
> the program cannot look at the pointer's value any more. It can't copy
> the value to another pointer, and it can't even test whether the value
> is null. A debugging implementation can trap any use of the pointer's value.

Thanks. I wondered if it was something like that, but there was
nothing obvious in the specification for free().

But, just to confirm, I suppose that you are talking about this
sentence (which I did finally discover) from Section 6.2.4 ("Storage
durations of objects") in the C11 standard:

[[
The value of a pointer becomes indeterminate when the object it points
to (or just past) reaches the end of its lifetime.
]]

?

> This rule is not an issue for production uses of glibc realloc. However,
> it could at least in theory affect debugging implementations like gcc
> -fsanitize=address, because the rule can help catch bugs in programs.
> And I vaguely recall talk that a few non-glibc platforms (IBM i,
> perhaps?) enforce the rule.

Thanks for the background.

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: Adding a realloc() usage note to the malloc(3) manual page

[Paul Eggert]

On 9/1/21 10:22 PM, Michael Kerrisk (man-pages) wrote:

> I suppose that you are talking about this
> sentence (which I did finally discover) from Section 6.2.4 ("Storage
> durations of objects") in the C11 standard:
> 
> [[
> The value of a pointer becomes indeterminate when the object it points
> to (or just past) reaches the end of its lifetime.
> ]]

Although I haven't checked the standard, that does sound like the sort 
of thing I was talking about, yes.