Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[Markus Armbruster]

malc <av1474@comtv.ru> writes:

> On Tue, 29 Dec 2009, Jamie Lokier wrote:
>
>> malc wrote:
>> > On Mon, 28 Dec 2009, Jamie Lokier wrote:
>> > 
>> > > Aurelien Jarno wrote:
>> > > > This fixes the loading of a stripped kernel with zero malloc disabled.
>> > > 
>> > > *Raises an eyebrow*
>> > > 
>> > > Even though there's different perspectives over whether qemu_malloc(0)
>> > > should be allowed, inherited from ambiguity over malloc(0),
>> > > realloc(p,0) has always had a standard, well-defined meaning.
>> > 
>> > No.
>> > http://groups.google.com/group/comp.std.c/browse_thread/thread/4e9af8847613d71f/6f75ad22e0768a0b?q=realloc++group:comp.std.c#6f75ad22e0768a0b
>> 
>> Wow, thanks for that.  It's a real surprise.  Looks like C99's own
>> rationale is not consistent with itself on the subject, and differs
>> from C90 where the "standard, well-defined meaning" I referred to was
>> defined.
>
> Yep.

No, this is a misinterpretation of the C99 standard, made possible by
its poor wording.  The C99 Rationale is perfectly clear, though:

    7.20.3.4 The realloc function

    A null first argument is permissible.  If the first argument is not
    null, and the second argument is 0, then the call frees the memory
    pointed to by the first argument, and a null argument may be
    returned; [...]

This is hardly surprising, because anything else would break working C89
programs, and that would squarely contradict the standard's mission, as
explained in the Rationale:

    Avoid "quiet changes."  Any change to widespread practice altering
    the meaning of existing code causes problems.  Changes that cause
    code to be so ill-formed as to require diagnostic messages are at
    least easy to detect.  As much as seemed possible consistent with its
    other goals, the C89 Committee avoided changes that quietly alter
    one valid program to another with different semantics, that cause a
    working program to work differently without notice.  In important
    places where this principle is violated, both the C89 Rationale and
    this Rationale point out a QUIET CHANGE.

The only quiet change in the Rationale regarding malloc() & friends is
this one:

                          QUIET CHANGE IN C89

    A program which relies on size-zero allocation requests returning a
    non-null pointer will behave differently.

For what it's worth, I checked my facts with a member of the C
committee.

>> See also http://c-faq.com/malloc/reallocnull.html which says "and the
>> related realloc(..., 0), which frees" and has references at the end.
>> See, it's not just me :-)
>
> Nope not just you, even glibc still uses C90 behaviour.

Because it reads the standard correctly, just like the comp.lang.c FAQ
Aurelien quoted.

[...]

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[malc]

On Thu, 21 Jan 2010, Markus Armbruster wrote:

> malc <av1474@comtv.ru> writes:
> 
> > On Tue, 29 Dec 2009, Jamie Lokier wrote:
> >
> >> malc wrote:
> >> > On Mon, 28 Dec 2009, Jamie Lokier wrote:
> >> > 
> >> > > Aurelien Jarno wrote:
> >> > > > This fixes the loading of a stripped kernel with zero malloc disabled.
> >> > > 
> >> > > *Raises an eyebrow*
> >> > > 
> >> > > Even though there's different perspectives over whether qemu_malloc(0)
> >> > > should be allowed, inherited from ambiguity over malloc(0),
> >> > > realloc(p,0) has always had a standard, well-defined meaning.
> >> > 
> >> > No.
> >> > http://groups.google.com/group/comp.std.c/browse_thread/thread/4e9af8847613d71f/6f75ad22e0768a0b?q=realloc++group:comp.std.c#6f75ad22e0768a0b
> >> 
> >> Wow, thanks for that.  It's a real surprise.  Looks like C99's own
> >> rationale is not consistent with itself on the subject, and differs
> >> from C90 where the "standard, well-defined meaning" I referred to was
> >> defined.
> >
> > Yep.
> 
> No, this is a misinterpretation of the C99 standard, made possible by
> its poor wording.  The C99 Rationale is perfectly clear, though:

You have to show the flaw in Hallvard B Furuseth's analysis to claim
that it's a misinterpretation. And unlike the standard rationale is
non normative.

[..snip..]

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[Markus Armbruster]

malc <av1474@comtv.ru> writes:

> On Thu, 21 Jan 2010, Markus Armbruster wrote:
>
>> malc <av1474@comtv.ru> writes:
>> 
>> > On Tue, 29 Dec 2009, Jamie Lokier wrote:
>> >
>> >> malc wrote:
>> >> > On Mon, 28 Dec 2009, Jamie Lokier wrote:
>> >> > 
>> >> > > Aurelien Jarno wrote:
>> >> > > > This fixes the loading of a stripped kernel with zero malloc disabled.
>> >> > > 
>> >> > > *Raises an eyebrow*
>> >> > > 
>> >> > > Even though there's different perspectives over whether qemu_malloc(0)
>> >> > > should be allowed, inherited from ambiguity over malloc(0),
>> >> > > realloc(p,0) has always had a standard, well-defined meaning.
>> >> > 
>> >> > No.
>> >> > http://groups.google.com/group/comp.std.c/browse_thread/thread/4e9af8847613d71f/6f75ad22e0768a0b?q=realloc++group:comp.std.c#6f75ad22e0768a0b
>> >> 
>> >> Wow, thanks for that.  It's a real surprise.  Looks like C99's own
>> >> rationale is not consistent with itself on the subject, and differs
>> >> from C90 where the "standard, well-defined meaning" I referred to was
>> >> defined.
>> >
>> > Yep.
>> 
>> No, this is a misinterpretation of the C99 standard, made possible by
>> its poor wording.  The C99 Rationale is perfectly clear, though:
>
> You have to show the flaw in Hallvard B Furuseth's analysis to claim
> that it's a misinterpretation. And unlike the standard rationale is
> non normative.
>
> [..snip..]

I did.  If that doesn't convince you, I'll gladly wait for the Technical
Corrigendum that'll put this rather absurd misreading to rest.

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[malc]

On Thu, 21 Jan 2010, Markus Armbruster wrote:

> malc <av1474@comtv.ru> writes:
> 
> > On Thu, 21 Jan 2010, Markus Armbruster wrote:

[..snip..]

> >> No, this is a misinterpretation of the C99 standard, made possible by
> >> its poor wording.  The C99 Rationale is perfectly clear, though:
> >
> > You have to show the flaw in Hallvard B Furuseth's analysis to claim
> > that it's a misinterpretation. And unlike the standard rationale is
> > non normative.
> >
> > [..snip..]
> 
> I did.  If that doesn't convince you, I'll gladly wait for the Technical
> Corrigendum that'll put this rather absurd misreading to rest.

If you did, then, i guess, i've missed it, here's the whole analysis,
please point what and where is wrong:

[quote: http://groups.google.com/group/comp.std.c/browse_thread/thread/4e9af8847613d71f/6f75ad22e0768a0b?q=realloc++group:comp.std.c#6f75ad22e0768a0b]

C90 said realloc(non-null, 0) did free().  C99 inverted that, saying it
does not:

The only place where 7.20.3.4 (The realloc function) mentions that
realloc may free the old object, is in the case where it returns another
object.  7.20.3 (Memory management functions) says zero-sized allocation
returns NULL, but 7.20.3.4 overrides that.

Could we have the original behavior back, please?  I've seen people say
the current definition is unintentional.  And it conflicts with the C99
Rationale:

   7.20.3.4 The realloc function

   (...)  If the first argument is not null, and the second argument is
   0, then the call frees the memory pointed to by the first argument,

though that goes on with

   and a null argument may be returned; C99 is consistent with the
   policy of not allowing zero-sized objects.

Is that supposed to mean no new object is returned but the return value
is indeterminate, or does it mean that it might free the old object and
return an inaccessible new object like malloc(0)?

Repeating from old realloc(non-null, 0) discussions:

In the latter case a program which sees a NULL return from
realloc(non-null, 0) cannot know if the old object was freed or not -
i.e. it cannot know if the NULL was a failure return (from allocating
the new object) or a success return (after freeing the old object).

Which is exactly the situation for a portable program which sees such a
NULL return today - it cannot know if it was a C99 failure return or a
C90 success return.  Even if the language is C99, the library might be
C90. 

[end quote]

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[Markus Armbruster]

malc <av1474@comtv.ru> writes:

> On Thu, 21 Jan 2010, Markus Armbruster wrote:
>
>> malc <av1474@comtv.ru> writes:
>> 
>> > On Thu, 21 Jan 2010, Markus Armbruster wrote:
>
> [..snip..]
>
>> >> No, this is a misinterpretation of the C99 standard, made possible by
>> >> its poor wording.  The C99 Rationale is perfectly clear, though:
>> >
>> > You have to show the flaw in Hallvard B Furuseth's analysis to claim
>> > that it's a misinterpretation. And unlike the standard rationale is
>> > non normative.
>> >
>> > [..snip..]
>> 
>> I did.  If that doesn't convince you, I'll gladly wait for the Technical
>> Corrigendum that'll put this rather absurd misreading to rest.
>
> If you did, then, i guess, i've missed it, here's the whole analysis,
> please point what and where is wrong:
>
> [quote: http://groups.google.com/group/comp.std.c/browse_thread/thread/4e9af8847613d71f/6f75ad22e0768a0b?q=realloc++group:comp.std.c#6f75ad22e0768a0b]
>
> C90 said realloc(non-null, 0) did free().  C99 inverted that, saying it
> does not:
>
> The only place where 7.20.3.4 (The realloc function) mentions that
> realloc may free the old object, is in the case where it returns another
> object.

7.20.3.4 starts with the simple, unconditional statement:

    The realloc function deallocates the old object pointed to by ptr
    and returns a pointer to a new object that has the size specified by
    size.

The only place where it limits this clause is here:

    If memory for the new object cannot be allocated, the old object is
    not deallocated and its value is unchanged.

Therefore, saying that realloc() frees the old object only when "it
returns another object" is imprecise.  What the standard *actually* says
is realloc() frees unless "the new object cannot be allocated".

The question is whether returning null for a zero-sized allocation is to
be considered "the new object cannot be allocated".  The claim that
realloc(p, 0) may not free(p) relies on the answer to that question
being "yes".

However, the Rationale maintains that the answer is "no".  Which is
consistent with the previous iteration of the standard, common usage
predating the standard, and thus with the standard's mission to avoid
"quiet changes".

>          7.20.3 (Memory management functions) says zero-sized allocation
> returns NULL, but 7.20.3.4 overrides that.
>
> Could we have the original behavior back, please?  I've seen people say
> the current definition is unintentional.  And it conflicts with the C99
> Rationale:
>
>    7.20.3.4 The realloc function
>
>    (...)  If the first argument is not null, and the second argument is
>    0, then the call frees the memory pointed to by the first argument,
>
> though that goes on with
>
>    and a null argument may be returned; C99 is consistent with the
>    policy of not allowing zero-sized objects.
>
> Is that supposed to mean no new object is returned but the return value
> is indeterminate, or does it mean that it might free the old object and
> return an inaccessible new object like malloc(0)?

It means it must free the old object and may return either null or an
inaccessible new object, just like malloc(0).

The "policy of not allowing zero-sized objects" refers back to 7.20.3:

    Some implementations have returned non-null values for allocation
    requests of zero bytes.  Although this strategy has the theoretical
    advantage of distinguishing between "nothing" and "zero" (an
    unallocated pointer vs. a pointer to zero-length space), it has the
    more compelling theoretical disadvantage of requiring the concept of
    a zero-length object.  Since such objects cannot be declared, the
    only way they could come into existence would be through such
    allocation requests.

    The C89 Committee decided not to accept the idea of zero-length
    objects.  The allocation functions may therefore return a null
    pointer for an allocation request of zero bytes.  Note that this
    treatment does not preclude the paradigm outlined above.

> Repeating from old realloc(non-null, 0) discussions:
>
> In the latter case a program which sees a NULL return from
> realloc(non-null, 0) cannot know if the old object was freed or not -
> i.e. it cannot know if the NULL was a failure return (from allocating
> the new object) or a success return (after freeing the old object).
>
> Which is exactly the situation for a portable program which sees such a
> NULL return today - it cannot know if it was a C99 failure return or a
> C90 success return.  Even if the language is C99, the library might be
> C90. 
>
> [end quote]

If your reading of the standard leads to conclusions like "breaks most
existing programs in an unfixable way", in absence of any evidence of
such breakage in practice, then it's maybe time to double-check it
against the Rationale.

If C99 made the sky fall, we'd be done being crushed by now :)

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[malc]

On Fri, 22 Jan 2010, Markus Armbruster wrote:

> malc <av1474@comtv.ru> writes:
>
> > On Thu, 21 Jan 2010, Markus Armbruster wrote:
> >
> >> malc <av1474@comtv.ru> writes:
> >>
> >> > On Thu, 21 Jan 2010, Markus Armbruster wrote:
> >
> > [..snip..]
> >
> >> >> No, this is a misinterpretation of the C99 standard, made possible by
> >> >> its poor wording.  The C99 Rationale is perfectly clear, though:
> >> >
> >> > You have to show the flaw in Hallvard B Furuseth's analysis to claim
> >> > that it's a misinterpretation. And unlike the standard rationale is
> >> > non normative.
> >> >
> >> > [..snip..]
> >>
> >> I did.  If that doesn't convince you, I'll gladly wait for the Technical
> >> Corrigendum that'll put this rather absurd misreading to rest.
> >
> > If you did, then, i guess, i've missed it, here's the whole analysis,
> > please point what and where is wrong:
> >
> > [quote: http://groups.google.com/group/comp.std.c/browse_thread/thread/4e9af8847613d71f/6f75ad22e0768a0b?q=realloc++group:comp.std.c#6f75ad22e0768a0b]
> >
> > C90 said realloc(non-null, 0) did free().  C99 inverted that, saying it
> > does not:
> >
> > The only place where 7.20.3.4 (The realloc function) mentions that
> > realloc may free the old object, is in the case where it returns another
> > object.
>
> 7.20.3.4 starts with the simple, unconditional statement:
>
>     The realloc function deallocates the old object pointed to by ptr
>     and returns a pointer to a new object that has the size specified by
>     size.
>
> The only place where it limits this clause is here:
>
>     If memory for the new object cannot be allocated, the old object is
>     not deallocated and its value is unchanged.
>
> Therefore, saying that realloc() frees the old object only when "it
> returns another object" is imprecise.  What the standard *actually* says
> is realloc() frees unless "the new object cannot be allocated".

It says, btw for some reason you've snipped the last part of the paragraph
in one of you previous posts:

      Returns

       [#4] The realloc function  returns  a  pointer  to  the  new |
       object  (which  may  have the same value as a pointer to the |
       old object), or a null pointer if the new object  could  not |
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       be allocated.
       ^^^^^^^^^^^^^

^^ marks what you've snipped and under this GLIBC's behaviour is in
error, since

a. GLIBC returns NULL, which is only sanctioned if the new object
   is not allocated

b. From a) and #4 follows that the old object wasn't freed

>
> The question is whether returning null for a zero-sized allocation is to
> be considered "the new object cannot be allocated".  The claim that
> realloc(p, 0) may not free(p) relies on the answer to that question
> being "yes".

And the answer is "yes" if one follows simple logic.

> However, the Rationale maintains that the answer is "no".  Which is
> consistent with the previous iteration of the standard, common usage
> predating the standard, and thus with the standard's mission to avoid
> "quiet changes".

We really have to wait for TC.

[..snip..]

>
> If your reading of the standard leads to conclusions like "breaks most
> existing programs in an unfixable way", in absence of any evidence of
> such breakage in practice, then it's maybe time to double-check it
> against the Rationale.
>
> If C99 made the sky fall, we'd be done being crushed by now :)
>

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[Jamie Lokier]

Markus Armbruster wrote:
> malc <av1474@comtv.ru> writes:
> 
> > On Tue, 29 Dec 2009, Jamie Lokier wrote:
> >
> >> malc wrote:
> >> > On Mon, 28 Dec 2009, Jamie Lokier wrote:
> >> > 
> >> > > Aurelien Jarno wrote:
> >> > > > This fixes the loading of a stripped kernel with zero malloc disabled.
> >> > > 
> >> > > *Raises an eyebrow*
> >> > > 
> >> > > Even though there's different perspectives over whether qemu_malloc(0)
> >> > > should be allowed, inherited from ambiguity over malloc(0),
> >> > > realloc(p,0) has always had a standard, well-defined meaning.
> >> > 
> >> > No.
> >> > http://groups.google.com/group/comp.std.c/browse_thread/thread/4e9af8847613d71f/6f75ad22e0768a0b?q=realloc++group:comp.std.c#6f75ad22e0768a0b
> >> 
> >> Wow, thanks for that.  It's a real surprise.  Looks like C99's own
> >> rationale is not consistent with itself on the subject, and differs
> >> from C90 where the "standard, well-defined meaning" I referred to was
> >> defined.
> >
> > Yep.
> 
> No, this is a misinterpretation of the C99 standard, made possible by
> its poor wording.  The C99 Rationale is perfectly clear, though:
> 
>     7.20.3.4 The realloc function
> 
>     A null first argument is permissible.  If the first argument is not
>     null, and the second argument is 0, then the call frees the memory
>     pointed to by the first argument, and a null argument may be
>     returned; [...]

The rationale above does not match C89 behaviour.  It says the call
frees the memory, but it does not forbid the call from then proceeding
to do the same as malloc(0) and return a non-NULL pointer.  It's quite
explicit: a null argument *may* be returned.  Which means the
rationale does not require realloc(p,0) to do the same as C89, which
always frees the memory and doesn't allocate anything.

> This is hardly surprising, because anything else would break working C89
> programs, and that would squarely contradict the standard's mission,

Understood.  But it doesn't really matter what's intended or what's
misinterpreted.  If there are any significant implementations out
there based on the "misinterpretation", or even based on the
rationale, that's enough of a reason to not depend on realloc(p,0).

-- Jamie

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[malc]

On Thu, 21 Jan 2010, Jamie Lokier wrote:

> Markus Armbruster wrote:
> > malc <av1474@comtv.ru> writes:
> > 
> > > On Tue, 29 Dec 2009, Jamie Lokier wrote:
> > >
> > >> malc wrote:
> > >> > On Mon, 28 Dec 2009, Jamie Lokier wrote:
> > >> > 
> > >> > > Aurelien Jarno wrote:
> > >> > > > This fixes the loading of a stripped kernel with zero malloc disabled.
> > >> > > 
> > >> > > *Raises an eyebrow*
> > >> > > 
> > >> > > Even though there's different perspectives over whether qemu_malloc(0)
> > >> > > should be allowed, inherited from ambiguity over malloc(0),
> > >> > > realloc(p,0) has always had a standard, well-defined meaning.
> > >> > 
> > >> > No.
> > >> > http://groups.google.com/group/comp.std.c/browse_thread/thread/4e9af8847613d71f/6f75ad22e0768a0b?q=realloc++group:comp.std.c#6f75ad22e0768a0b
> > >> 
> > >> Wow, thanks for that.  It's a real surprise.  Looks like C99's own
> > >> rationale is not consistent with itself on the subject, and differs
> > >> from C90 where the "standard, well-defined meaning" I referred to was
> > >> defined.
> > >
> > > Yep.
> > 
> > No, this is a misinterpretation of the C99 standard, made possible by
> > its poor wording.  The C99 Rationale is perfectly clear, though:
> > 
> >     7.20.3.4 The realloc function
> > 
> >     A null first argument is permissible.  If the first argument is not
> >     null, and the second argument is 0, then the call frees the memory
> >     pointed to by the first argument, and a null argument may be
> >     returned; [...]
> 
> The rationale above does not match C89 behaviour.  It says the call
> frees the memory, but it does not forbid the call from then proceeding
> to do the same as malloc(0) and return a non-NULL pointer.  It's quite
> explicit: a null argument *may* be returned.  Which means the
> rationale does not require realloc(p,0) to do the same as C89, which
> always frees the memory and doesn't allocate anything.
> 
> > This is hardly surprising, because anything else would break working C89
> > programs, and that would squarely contradict the standard's mission,
> 
> Understood.  But it doesn't really matter what's intended or what's
> misinterpreted.  If there are any significant implementations out
> there based on the "misinterpretation", or even based on the
> rationale, that's enough of a reason to not depend on realloc(p,0).
> 

My sentiment exactly.

An example:

Dinkum Unabridged Library was certified by Perennial 
(http://peren.com/pages/aboutus_set.htm) to conform
to ISO/IEC 9899:1999.

Documentation for realloc:
http://www.dinkumware.com/manuals/?manual=compleat&Search=realloc&page=stdlib.html#realloc

Hallvard B Furuseth analysis fully applies i believe...

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[Markus Armbruster]

malc <av1474@comtv.ru> writes:

> On Thu, 21 Jan 2010, Jamie Lokier wrote:
>
>> Markus Armbruster wrote:
>> > malc <av1474@comtv.ru> writes:
>> > 
>> > > On Tue, 29 Dec 2009, Jamie Lokier wrote:
>> > >
>> > >> malc wrote:
>> > >> > On Mon, 28 Dec 2009, Jamie Lokier wrote:
>> > >> > 
>> > >> > > Aurelien Jarno wrote:
>> > >> > > > This fixes the loading of a stripped kernel with zero malloc disabled.
>> > >> > > 
>> > >> > > *Raises an eyebrow*
>> > >> > > 
>> > >> > > Even though there's different perspectives over whether qemu_malloc(0)
>> > >> > > should be allowed, inherited from ambiguity over malloc(0),
>> > >> > > realloc(p,0) has always had a standard, well-defined meaning.
>> > >> > 
>> > >> > No.
>> > >> > http://groups.google.com/group/comp.std.c/browse_thread/thread/4e9af8847613d71f/6f75ad22e0768a0b?q=realloc++group:comp.std.c#6f75ad22e0768a0b
>> > >> 
>> > >> Wow, thanks for that.  It's a real surprise.  Looks like C99's own
>> > >> rationale is not consistent with itself on the subject, and differs
>> > >> from C90 where the "standard, well-defined meaning" I referred to was
>> > >> defined.
>> > >
>> > > Yep.
>> > 
>> > No, this is a misinterpretation of the C99 standard, made possible by
>> > its poor wording.  The C99 Rationale is perfectly clear, though:
>> > 
>> >     7.20.3.4 The realloc function
>> > 
>> >     A null first argument is permissible.  If the first argument is not
>> >     null, and the second argument is 0, then the call frees the memory
>> >     pointed to by the first argument, and a null argument may be
>> >     returned; [...]
>> 
>> The rationale above does not match C89 behaviour.  It says the call
>> frees the memory, but it does not forbid the call from then proceeding
>> to do the same as malloc(0) and return a non-NULL pointer.  It's quite
>> explicit: a null argument *may* be returned.  Which means the
>> rationale does not require realloc(p,0) to do the same as C89, which
>> always frees the memory and doesn't allocate anything.
>> 
>> > This is hardly surprising, because anything else would break working C89
>> > programs, and that would squarely contradict the standard's mission,
>> 
>> Understood.  But it doesn't really matter what's intended or what's
>> misinterpreted.  If there are any significant implementations out
>> there based on the "misinterpretation", or even based on the
>> rationale, that's enough of a reason to not depend on realloc(p,0).
>> 
>
> My sentiment exactly.
>
> An example:
>
> Dinkum Unabridged Library was certified by Perennial 
> (http://peren.com/pages/aboutus_set.htm) to conform
> to ISO/IEC 9899:1999.
>
> Documentation for realloc:
> http://www.dinkumware.com/manuals/?manual=compleat&Search=realloc&page=stdlib.html#realloc
>
> Hallvard B Furuseth analysis fully applies i believe...

Regardless of whether his analysis applies to this implementation's
documentation or not: what does this implementation actually do?

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[malc]

On Fri, 22 Jan 2010, Markus Armbruster wrote:

> malc <av1474@comtv.ru> writes:
> 
> > On Thu, 21 Jan 2010, Jamie Lokier wrote:
> >
> >> Markus Armbruster wrote:
> >> > malc <av1474@comtv.ru> writes:
> >> > 
> >> > > On Tue, 29 Dec 2009, Jamie Lokier wrote:
> >> > >
> >> > >> malc wrote:
> >> > >> > On Mon, 28 Dec 2009, Jamie Lokier wrote:
> >> > >> > 
> >> > >> > > Aurelien Jarno wrote:
> >> > >> > > > This fixes the loading of a stripped kernel with zero malloc disabled.
> >> > >> > > 
> >> > >> > > *Raises an eyebrow*
> >> > >> > > 
> >> > >> > > Even though there's different perspectives over whether qemu_malloc(0)
> >> > >> > > should be allowed, inherited from ambiguity over malloc(0),
> >> > >> > > realloc(p,0) has always had a standard, well-defined meaning.
> >> > >> > 
> >> > >> > No.
> >> > >> > http://groups.google.com/group/comp.std.c/browse_thread/thread/4e9af8847613d71f/6f75ad22e0768a0b?q=realloc++group:comp.std.c#6f75ad22e0768a0b
> >> > >> 
> >> > >> Wow, thanks for that.  It's a real surprise.  Looks like C99's own
> >> > >> rationale is not consistent with itself on the subject, and differs
> >> > >> from C90 where the "standard, well-defined meaning" I referred to was
> >> > >> defined.
> >> > >
> >> > > Yep.
> >> > 
> >> > No, this is a misinterpretation of the C99 standard, made possible by
> >> > its poor wording.  The C99 Rationale is perfectly clear, though:
> >> > 
> >> >     7.20.3.4 The realloc function
> >> > 
> >> >     A null first argument is permissible.  If the first argument is not
> >> >     null, and the second argument is 0, then the call frees the memory
> >> >     pointed to by the first argument, and a null argument may be
> >> >     returned; [...]
> >> 
> >> The rationale above does not match C89 behaviour.  It says the call
> >> frees the memory, but it does not forbid the call from then proceeding
> >> to do the same as malloc(0) and return a non-NULL pointer.  It's quite
> >> explicit: a null argument *may* be returned.  Which means the
> >> rationale does not require realloc(p,0) to do the same as C89, which
> >> always frees the memory and doesn't allocate anything.
> >> 
> >> > This is hardly surprising, because anything else would break working C89
> >> > programs, and that would squarely contradict the standard's mission,
> >> 
> >> Understood.  But it doesn't really matter what's intended or what's
> >> misinterpreted.  If there are any significant implementations out
> >> there based on the "misinterpretation", or even based on the
> >> rationale, that's enough of a reason to not depend on realloc(p,0).
> >> 
> >
> > My sentiment exactly.
> >
> > An example:
> >
> > Dinkum Unabridged Library was certified by Perennial 
> > (http://peren.com/pages/aboutus_set.htm) to conform
> > to ISO/IEC 9899:1999.
> >
> > Documentation for realloc:
> > http://www.dinkumware.com/manuals/?manual=compleat&Search=realloc&page=stdlib.html#realloc
> >
> > Hallvard B Furuseth analysis fully applies i believe...
> 
> Regardless of whether his analysis applies to this implementation's
> documentation or not: what does this implementation actually do?
> 

I'd guess it follows it's own documentation, but i don't have it to
check, anyone?

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[Markus Armbruster]

Jamie Lokier <jamie@shareable.org> writes:

> Markus Armbruster wrote:
>> malc <av1474@comtv.ru> writes:
>> 
>> > On Tue, 29 Dec 2009, Jamie Lokier wrote:
>> >
>> >> malc wrote:
>> >> > On Mon, 28 Dec 2009, Jamie Lokier wrote:
>> >> > 
>> >> > > Aurelien Jarno wrote:
>> >> > > > This fixes the loading of a stripped kernel with zero malloc disabled.
>> >> > > 
>> >> > > *Raises an eyebrow*
>> >> > > 
>> >> > > Even though there's different perspectives over whether qemu_malloc(0)
>> >> > > should be allowed, inherited from ambiguity over malloc(0),
>> >> > > realloc(p,0) has always had a standard, well-defined meaning.
>> >> > 
>> >> > No.
>> >> > http://groups.google.com/group/comp.std.c/browse_thread/thread/4e9af8847613d71f/6f75ad22e0768a0b?q=realloc++group:comp.std.c#6f75ad22e0768a0b
>> >> 
>> >> Wow, thanks for that.  It's a real surprise.  Looks like C99's own
>> >> rationale is not consistent with itself on the subject, and differs
>> >> from C90 where the "standard, well-defined meaning" I referred to was
>> >> defined.
>> >
>> > Yep.
>> 
>> No, this is a misinterpretation of the C99 standard, made possible by
>> its poor wording.  The C99 Rationale is perfectly clear, though:
>> 
>>     7.20.3.4 The realloc function
>> 
>>     A null first argument is permissible.  If the first argument is not
>>     null, and the second argument is 0, then the call frees the memory
>>     pointed to by the first argument, and a null argument may be
>>     returned; [...]
>
> The rationale above does not match C89 behaviour.  It says the call
> frees the memory, but it does not forbid the call from then proceeding
> to do the same as malloc(0) and return a non-NULL pointer.  It's quite
> explicit: a null argument *may* be returned.  Which means the
> rationale does not require realloc(p,0) to do the same as C89, which
> always frees the memory and doesn't allocate anything.

I didn't claim there's *no* difference between C89 and C99.  In fact,
the Rationale nicely documents the change:

    A new feature of C99: the realloc function was changed to make it
    clear that the pointed-to object is deallocated, a new object is
    allocated, and the content of the new object is the same as that of
    the old object up to the lesser of the two sizes.  C89 attempted to
    specify that the new object was the same object as the old object
    but might have a different address.  This conflicts with other parts
    of the Standard that assume that the address of an object is
    constant during its lifetime.  Also, implementations that support an
    actual allocation when the size is zero do not necessarily return a
    null pointer for this case.  C89 appeared to require a null return
    value, and the Committee felt that this was too restrictive.

So C99 permits realloc(p, 0) to return a non-null value.  Regardless, it
still *requires* it to free(p).

>> This is hardly surprising, because anything else would break working C89
>> programs, and that would squarely contradict the standard's mission,
>
> Understood.  But it doesn't really matter what's intended or what's
> misinterpreted.  If there are any significant implementations out
> there based on the "misinterpretation", or even based on the
> rationale, that's enough of a reason to not depend on realloc(p,0).

There are none.

I don't really care how scared QEMU is of realloc(p, 0).  I just want to
correct the misinformation on the standard being spread on this list.

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[Jamie Lokier]

Markus Armbruster wrote:
> I didn't claim there's *no* difference between C89 and C99.  In fact,
> the Rationale nicely documents the change:
> 
>     [snipped]
>     Also, implementations that support an
>     actual allocation when the size is zero do not necessarily return a
>     null pointer for this case.  C89 appeared to require a null return
>     value, and the Committee felt that this was too restrictive.
> 
> So C99 permits realloc(p, 0) to return a non-null value.  Regardless, it
> still *requires* it to free(p).

Nobody disagrees that it does free(p).

The question is whether it may _follow_ the free(p) with malloc(0) and
return that, in which case the returned pointer must eventually be
passed to a subsequent free() to avoid leaks, or if it only doess
free(p) and a non-null result must be ignored.

For either meaning of non-null result, there are valid C89 programs which
will break, either leaking or calling free() on an invalid address.

> I just want to correct the misinformation on the standard being
> spread on this list.

I can't tell from your writing which misinformation you mean.

The only thing in question is the (new in C99) possibility of non-null
result and what to do with one (that it does free(p) is not in doubt),
and I'm afriad the sections you quoted firmly support the non-null
result change, and perpetuate the ambiguity of it's meaning.

In any case, there is no doubt, from the possibiliy of non-null result
alone (which is clear), that is already enough to make some valid C89
code misbehave.

The ambiguity of a non-null result (i.e. whether it is equivalent to
malloc(0) and the caller must free it later, or it is something the
caller must ignore because the realloc(p,0) call is equivalent to
free(p) only) is what makes it seem unsafe to call realloc(p,0) at all.

I don't want to argue and I really appreciate your clarification if
you know something.  If there is misinformation, it would be good to
correct it, in which case I don't think you have succeeded.

Unfortunately I can't tell from your mail what you think the meaning
of a non-null result is.

-- Jamie

Re: [Qemu-devel] [PATCH] loader: don't call realloc(O) when no symbols are present

[Markus Armbruster]

Jamie Lokier <jamie@shareable.org> writes:

> Markus Armbruster wrote:
>> I didn't claim there's *no* difference between C89 and C99.  In fact,
>> the Rationale nicely documents the change:
>> 
>>     [snipped]
>>     Also, implementations that support an
>>     actual allocation when the size is zero do not necessarily return a
>>     null pointer for this case.  C89 appeared to require a null return
>>     value, and the Committee felt that this was too restrictive.
>> 
>> So C99 permits realloc(p, 0) to return a non-null value.  Regardless, it
>> still *requires* it to free(p).
>
> Nobody disagrees that it does free(p).

Apparently malc does.

> The question is whether it may _follow_ the free(p) with malloc(0) and
> return that, in which case the returned pointer must eventually be
> passed to a subsequent free() to avoid leaks, or if it only doess
> free(p) and a non-null result must be ignored.
>
> For either meaning of non-null result, there are valid C89 programs which
> will break, either leaking or calling free() on an invalid address.
>
>> I just want to correct the misinformation on the standard being
>> spread on this list.
>
> I can't tell from your writing which misinformation you mean.

I mean the claim that realloc(p, 0) does not always free(p).

> The only thing in question is the (new in C99) possibility of non-null
> result and what to do with one (that it does free(p) is not in doubt),
> and I'm afriad the sections you quoted firmly support the non-null
> result change, and perpetuate the ambiguity of it's meaning.
>
> In any case, there is no doubt, from the possibiliy of non-null result
> alone (which is clear), that is already enough to make some valid C89
> code misbehave.
>
> The ambiguity of a non-null result (i.e. whether it is equivalent to
> malloc(0) and the caller must free it later, or it is something the
> caller must ignore because the realloc(p,0) call is equivalent to
> free(p) only) is what makes it seem unsafe to call realloc(p,0) at all.
>
> I don't want to argue and I really appreciate your clarification if
> you know something.  If there is misinformation, it would be good to
> correct it, in which case I don't think you have succeeded.
>
> Unfortunately I can't tell from your mail what you think the meaning
> of a non-null result is.

There is really nothing special about a non-null value returned by
realloc(p, 0).  Quoting C99 7.20.3.4:

    The realloc function returns a pointer to the new object (which may
    have the same value as a pointer to the old object), or a null
    pointer [...]

So this is really the same deal as with malloc():

* If you get a non-null value, you should pass it to free() to avoid
  memory leaks (actually, you may pass any value you get to free, since
  free() does nothing for null arguments).

* Checking for failure is tricky, because size argument zero may get you
  a null value.

There is no "ambiguity of a non-null result".

You are right on C99 breaking programs that assumed realloc(p, 0)
*always* returned null.  Quoting the C99 Rationale (emphasis mine):

    Also, implementations that support an actual allocation when the
    size is zero do not necessarily return a null pointer for this case.
    C89 *appeared* to require a null return value, and the Committee
    felt that this was too restrictive.

And this is where C89 7.10.3.4 appears to require it, but in fact
doesn't:

    Description
    [...]
    If ptr is a null pointer, the realloc function behaves like the
    malloc function for the specified size.  Otherwise, if ptr does not
    match a pointer earlier returned by the calloc, malloc, or realloc
    function, or if the space has been deallocated by a call to the free
    or realloc function, the behavior is undefined.  If the space cannot
    be allocated, the object pointed to by ptr is unchanged.  If size is
    zero and ptr is not a null pointer, the object it points to is
    freed.

The last sentence specifies only that the old object is freed.  It
doesn't specify return of a null value.  The return value is covered by
the *next* section:

    Returns

    The realloc function returns either a null pointer or a a pointer to
    the possibly moved allocated space.

For C99, the committee made the fact that realloc(p, 0) need not return
null explicit.  Unfortunately, they messed up the wording, which has led
a few people to believe the spec for realloc() changed incompatibly in
C99.  It did not.

Anything still unclear?


The practical question is of course whether realloc(p, 0) can be used
safely in portable programs.  It can, in the same way it's always been
used:

* If it yields a non-null value, you pass it to free()

* If it yields a null value, this means "out of memory" only if you
  asked for a non-zero size.

Note that such a portable program doesn't need to know how the
implementation defines behavior for zero arguments.