Re: [wish] Flexible array members in unions

Re: [wish] Flexible array members in unions

[Kees Cook]

On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
> On 5/11/23 18:07, Alejandro Colomar wrote:
> [...]
> > Would you allow flexible array members in unions?  Is there any
> > strong reason to disallow them?

Yes please!! And alone in a struct, too.

AFAICT, there is no mechanical/architectural reason to disallow them
(especially since they _can_ be constructed with some fancy tricks,
and they behave as expected.) My understanding is that it's disallowed
due to an overly strict reading of the very terse language that created
flexible arrays in C99.

> [...]
> Currently, the Linux kernel has to go through some hoops due to this
> restriction:
> 
> 
> $ grepc -tm __DECLARE_FLEX_ARRAY *
> include/uapi/linux/stddef.h:42:
> #define __DECLARE_FLEX_ARRAY(TYPE, NAME)	\
> 	struct { \
> 		struct { } __empty_ ## NAME; \
> 		TYPE NAME[]; \
> 	}

Yes, we've had to do this as we eradicate all the fake flexible arrays
in the kernel which cause endless bugs[1]. Additionally, we'll be
using -fstrict-flex-arrays=3 soon to let existing array bounds checking
mitigations gain coverage over trailing arrays. All of this means that
we're converting a lot of code that is happily using dynamically sized
arrays in unions, etc.

[1] https://people.kernel.org/kees/bounded-flexible-arrays-in-c

-- 
Kees Cook

Re: [wish] Flexible array members in unions

[Joseph Myers]

On Thu, 11 May 2023, Kees Cook via Gcc wrote:

> On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
> > On 5/11/23 18:07, Alejandro Colomar wrote:
> > [...]
> > > Would you allow flexible array members in unions?  Is there any
> > > strong reason to disallow them?
> 
> Yes please!! And alone in a struct, too.
> 
> AFAICT, there is no mechanical/architectural reason to disallow them
> (especially since they _can_ be constructed with some fancy tricks,
> and they behave as expected.) My understanding is that it's disallowed
> due to an overly strict reading of the very terse language that created
> flexible arrays in C99.

Standard C has no such thing as a zero-size object or type, which would 
lead to problems with a struct or union that only contains a flexible 
array member there.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [wish] Flexible array members in unions

[Kees Cook]

On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote:
> On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> 
> > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
> > > On 5/11/23 18:07, Alejandro Colomar wrote:
> > > [...]
> > > > Would you allow flexible array members in unions?  Is there any
> > > > strong reason to disallow them?
> > 
> > Yes please!! And alone in a struct, too.
> > 
> > AFAICT, there is no mechanical/architectural reason to disallow them
> > (especially since they _can_ be constructed with some fancy tricks,
> > and they behave as expected.) My understanding is that it's disallowed
> > due to an overly strict reading of the very terse language that created
> > flexible arrays in C99.
> 
> Standard C has no such thing as a zero-size object or type, which would 
> lead to problems with a struct or union that only contains a flexible 
> array member there.

Ah-ha, okay. That root cause makes sense now.

Why are zero-sized objects missing in Standard C? Or, perhaps, the better
question is: what's needed to support the idea of a zero-sized object?

-- 
Kees Cook

Re: [wish] Flexible array members in unions

[Joseph Myers]

On Thu, 11 May 2023, Kees Cook via Gcc wrote:

> Why are zero-sized objects missing in Standard C? Or, perhaps, the better
> question is: what's needed to support the idea of a zero-sized object?

Zero-sized objects break the principle that different objects have 
different addresses, and the principle of being able to subtract pointers 
to different elements of an array.  There would also be serious C++ 
compatibility concerns, since C++ allows a struct with no members but it 
has nonzero size, unlike the GNU C extension where a struct with no 
members has size zero.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [wish] Flexible array members in unions

[Kees Cook]

On Thu, May 11, 2023 at 09:43:49PM +0000, Joseph Myers wrote:
> On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> 
> > Why are zero-sized objects missing in Standard C? Or, perhaps, the better
> > question is: what's needed to support the idea of a zero-sized object?
> 
> Zero-sized objects break the principle that different objects have 
> different addresses, and the principle of being able to subtract pointers 
> to different elements of an array.  There would also be serious C++ 
> compatibility concerns, since C++ allows a struct with no members but it 
> has nonzero size, unlike the GNU C extension where a struct with no 
> members has size zero.

Okay, understood. If this is a C-only thing, we can ignore the C++
impact. What depends on the "different objects have different addresses"
principle? And why do unions not break this -- they could point to the
same locations within the object? And don't flexible arrays already need
special handling in this regard?

-- 
Kees Cook

Re: [wish] Flexible array members in unions

[Joseph Myers]

On Thu, 11 May 2023, Kees Cook via Gcc wrote:

> Okay, understood. If this is a C-only thing, we can ignore the C++
> impact.

We're a lot more careful lately in WG14 about checking for C++ 
compatibility issues and expecting approval from the liaison group for 
anything with possible compatibility concerns for syntax in the common 
subset of C and C++.  So, no, we can't ignore the C++ impact for adding 
empty types; it would need careful consideration in the liaison group.

> What depends on the "different objects have different addresses"
> principle? And why do unions not break this -- they could point to the
> same locations within the object? And don't flexible arrays already need
> special handling in this regard?

"including a pointer to an object and a subobject at its beginning" and 
"one is a pointer to one past the end of one array object and the other is 
a pointer to the start of a different array object that happens to 
immediately follow the first array object in the address space" are both 
cases included in the semantics for comparison operators.  If you allow 
zero-size objects you get more special cases there (and quite possibly 
affect optimizations based on points-to analysis that can determine 
pointers are based on different objects, if an object is not known at 
compile time to have nonzero size).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [wish] Flexible array members in unions

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 2964 bytes --]

Hi Joseph, Kees,

On 5/12/23 00:52, Joseph Myers wrote:
> On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> 
>> Okay, understood. If this is a C-only thing, we can ignore the C++
>> impact.
> 
> We're a lot more careful lately in WG14 about checking for C++ 
> compatibility issues and expecting approval from the liaison group for 
> anything with possible compatibility concerns for syntax in the common 
> subset of C and C++.  So, no, we can't ignore the C++ impact for adding 
> empty types; it would need careful consideration in the liaison group.
> 
>> What depends on the "different objects have different addresses"
>> principle? And why do unions not break this -- they could point to the
>> same locations within the object? And don't flexible arrays already need
>> special handling in this regard?
> 
> "including a pointer to an object and a subobject at its beginning" and 
> "one is a pointer to one past the end of one array object and the other is 
> a pointer to the start of a different array object that happens to 
> immediately follow the first array object in the address space" are both 
> cases included in the semantics for comparison operators.  If you allow 
> zero-size objects you get more special cases there (and quite possibly 
> affect optimizations based on points-to analysis that can determine 
> pointers are based on different objects, if an object is not known at 
> compile time to have nonzero size).

Since GNU C already supports empty structs, how about allowing that in GCC
with no intention of adding it to ISO C?  We'll see how good it behaves in
GCC, and if so suggest it for inclusion in the standard.

Why should GNU C, which allows empty structures, and de facto supports
flexible arrays in empty structs and in unions (via the empty preceeding
struct and the wrapper struct tricks, as the kernel does), shouldn't
support them officially without tricks?

Apart from violating artificial rules that disallow that use of flexible
arrays, I believe the example program I provided in the first post
doesn't violate aliasing rules or other similar rules that would result
in optimization conflicts.  Does it?

About zero-sized types, a union consisting of only flexible-array members
would effectively be a zero-sized type, so GCC should have similar issues
having this in unions and in empty structs.  So far, I don't see GCC
complaining about such horrible thing as an array of empty structs:

$ cat arr.c
#include <stdio.h>

struct s {};

struct s x[10];

int
main(void)
{
	printf("x:    %zu, %p\n", sizeof(x), &x);
	printf("x[3]: %zu, %p\n", sizeof(x[3]), &x[3]);
}
$ gcc-13 arr.c -Wall -Wextra
$ ./a.out 
x:    0, 0x55c5f6d72019
x[3]: 0, 0x55c5f6d72019



So, in GNU C land, I think it is reasonable to add this feature.

Cheers,
Alex

-- 
<http://www.alejandro-colomar.es/>
GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [wish] Flexible array members in unions

[Richard Biener]

On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote:
>
> On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote:
> > On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> >
> > > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
> > > > On 5/11/23 18:07, Alejandro Colomar wrote:
> > > > [...]
> > > > > Would you allow flexible array members in unions?  Is there any
> > > > > strong reason to disallow them?
> > >
> > > Yes please!! And alone in a struct, too.
> > >
> > > AFAICT, there is no mechanical/architectural reason to disallow them
> > > (especially since they _can_ be constructed with some fancy tricks,
> > > and they behave as expected.) My understanding is that it's disallowed
> > > due to an overly strict reading of the very terse language that created
> > > flexible arrays in C99.
> >
> > Standard C has no such thing as a zero-size object or type, which would
> > lead to problems with a struct or union that only contains a flexible
> > array member there.
>
> Ah-ha, okay. That root cause makes sense now.

Hmm. but then the workaround

struct X {
  int n;
  union u {
      char at_least_size_one;
      int iarr[];
      short sarr[];
  };
};

doesn't work either.  We could make that a GNU extension without
adverse effects?

Richard.

> Why are zero-sized objects missing in Standard C? Or, perhaps, the better
> question is: what's needed to support the idea of a zero-sized object?
>
> --
> Kees Cook

Re: [wish] Flexible array members in unions

[Qing Zhao]

> On May 12, 2023, at 2:16 AM, Richard Biener via Gcc <gcc@gcc.gnu.org> wrote:
> 
> On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc <gcc@gcc.gnu.org> wrote:
>> 
>> On Thu, May 11, 2023 at 08:53:52PM +0000, Joseph Myers wrote:
>>> On Thu, 11 May 2023, Kees Cook via Gcc wrote:
>>> 
>>>> On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
>>>>> On 5/11/23 18:07, Alejandro Colomar wrote:
>>>>> [...]
>>>>>> Would you allow flexible array members in unions?  Is there any
>>>>>> strong reason to disallow them?
>>>> 
>>>> Yes please!! And alone in a struct, too.
>>>> 
>>>> AFAICT, there is no mechanical/architectural reason to disallow them
>>>> (especially since they _can_ be constructed with some fancy tricks,
>>>> and they behave as expected.) My understanding is that it's disallowed
>>>> due to an overly strict reading of the very terse language that created
>>>> flexible arrays in C99.
>>> 
>>> Standard C has no such thing as a zero-size object or type, which would
>>> lead to problems with a struct or union that only contains a flexible
>>> array member there.
>> 
>> Ah-ha, okay. That root cause makes sense now.
> 
> Hmm. but then the workaround
> 
> struct X {
>  int n;
>  union u {
>      char at_least_size_one;
>      int iarr[];
>      short sarr[];
>  };
> };
> 
> doesn't work either.  We could make that a GNU extension without
> adverse effects?

I think that this might be  a very nice extension, which addresses the standard C’s restriction  on the zero-size object, and also can resolve kernel’s need. (And also other users’s similar programming need?)
And maybe it’s also possible to add such extension later to Standard C?

Similar as flexible array member in Standard C, we should limit such union as the last field of another structure.  (Since basically this union can be treated
As a flexible array member)

Qing

> 
> Richard.
> 
>> Why are zero-sized objects missing in Standard C? Or, perhaps, the better
>> question is: what's needed to support the idea of a zero-sized object?
>> 
>> --
>> Kees Cook