Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 4181 bytes --]

[CC += linux-man@, since we're discussing an API documented there, and
 the manual page would also need to be updated]

Hi Xi,  Jakub,

On Fri, Jul 05, 2024 at 09:38:21PM GMT, Xi Ruoyao wrote:
> On Fri, 2024-07-05 at 15:03 +0200, Alejandro Colomar wrote:
> > ISO C specifies these APIs as accepting a restricted pointer in their
> > first parameter:
> > 
> > $ stdc c99 strtol
> > long int strtol(const char *restrict nptr, char **restrict endptr, int base);
> > $ stdc c11 strtol
> > long int strtol(const char *restrict nptr, char **restrict endptr, int base);
> > 
> > However, it should be considered a defect in ISO C.  It's common to see
> > code that aliases it:
> > 
> > 	char str[] = "10 20";
> > 
> > 	p = str;
> > 	a = strtol(p, &p, 0);  // Let's ignore error handling for
> > 	b = strtol(p, &p, 0);  // simplicity.
> 
> Why this is wrong?
> 
> During the execution of strtol() the only expression accessing the
> object "p" is *endptr.  When the body of strtol() refers "nptr" it
> accesses a different object, not "p".

<http://port70.net/~nsz/c/c11/n1570.html#6.7.3p8>

Theoretically, 'restrict' is defined in terms of accesses, not just
references, so it's fine for strtol(3) to hold two references of p in
restrict pointers.  That is, the following code is valid:

	int
	dumb(int *restrict a, int *restrict also_a)
	{
		// We don't access the objects
		return a == also_a;
	}

	int
	main(void)
	{
		int x = 3;

		return dumb(&x, &x);
	}

However, in practice that's dumb.  The caller cannot know that the
function doesn't access the object, so it must be cautious and enable
-Wrestrict, which should be paranoid and do not allow passing references
to the same object in different arguments, just in case the function
decides to access to objects.  Of course, GCC reports a diagnostic for
the previous code:

	$ cc -Wall -Wextra dumb.c 
	dumb.c: In function ‘main’:
	dumb.c:13:21: warning: passing argument 1 to ‘restrict’-qualified parameter aliases with argument 2 [-Wrestrict]
	   13 |         return dumb(&x, &x);
	      |                     ^~  ~~

... even when there's no UB, since the object is not being accessed.

But when the thing gets non-trivial, as in strtol(3), GCC misses the
-Wrestrict diagnostic, as reported in
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112833>.

Let's write a reproducer by altering the dumb.c program from above, with
just another reference:

	int
	dumb2(int *restrict a, int *restrict *restrict ap)
	{
		// We don't access the objects
		return a == *ap;
	}

	int
	main(void)
	{
		int x = 3;
		int *xp = &x;

		return dumb2(&x, &xp);
	}

GCC doesn't report anything bad here, even though it's basically the
same as the program from above:

	$ cc -Wall -Wextra dumb2.c
	$

Again, there's no UB, but we really want to be cautious and get a
diagnostic as callers, just in case the callee decides to access the
object; we never know.

So, GCC should be patched to report a warning in the program above.
That will also cause strtol(3) to start issuing warnings in use cases
like the one I showed.

Even further, let's try something really weird: inequality comparison,
which is only defined for pointers to the same array object:

	int
	dumb3(int *restrict a, int *restrict *restrict ap)
	{
		// We don't access the objects
		return a > *ap;
	}

	int
	main(void)
	{
		int x = 3;
		int *xp = &x;

		return dumb3(&x, &xp);
	}

The behavior is still defined, since the obnjects are not accessed, but
the compiler should really warn, on both sides:

-  The caller is passing references to the same object in restricted
   parameters, which is a red flag.

-  The callee is comparing for inequality pointers that should, under
   normal circumstances, cause Undefined Behavior.


> And if this is really wrong you should report it to WG14 before changing
> glibc.

Well, I don't know how to report that defect to WG14.  If you help me,
I'll be pleased to do so.  Do they have a public mailing list or
anything like that?


Cheers,
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Martin Uecker]

Am Freitag, dem 05.07.2024 um 16:37 +0200 schrieb Alejandro Colomar via Gcc:
> [CC += linux-man@, since we're discussing an API documented there, and
>  the manual page would also need to be updated]
> 
> Hi Xi,  Jakub,
> 
> On Fri, Jul 05, 2024 at 09:38:21PM GMT, Xi Ruoyao wrote:
> > On Fri, 2024-07-05 at 15:03 +0200, Alejandro Colomar wrote:
> > > ISO C specifies these APIs as accepting a restricted pointer in their
> > > first parameter:
> > > 
> > > $ stdc c99 strtol
> > > long int strtol(const char *restrict nptr, char **restrict endptr, int base);
> > > $ stdc c11 strtol
> > > long int strtol(const char *restrict nptr, char **restrict endptr, int base);
> > > 
> > > However, it should be considered a defect in ISO C.  It's common to see
> > > code that aliases it:
> > > 
> > > 	char str[] = "10 20";
> > > 
> > > 	p = str;
> > > 	a = strtol(p, &p, 0);  // Let's ignore error handling for
> > > 	b = strtol(p, &p, 0);  // simplicity.
> > 
> > Why this is wrong?
> > 
> > During the execution of strtol() the only expression accessing the
> > object "p" is *endptr.  When the body of strtol() refers "nptr" it
> > accesses a different object, not "p".
> 
> <http://port70.net/~nsz/c/c11/n1570.html#6.7.3p8>
> 
> Theoretically, 'restrict' is defined in terms of accesses, not just
> references, so it's fine for strtol(3) to hold two references of p in
> restrict pointers.  That is, the following code is valid:
> 
> 	int
> 	dumb(int *restrict a, int *restrict also_a)
> 	{
> 		// We don't access the objects
> 		return a == also_a;
> 	}
> 
> 	int
> 	main(void)
> 	{
> 		int x = 3;
> 
> 		return dumb(&x, &x);
> 	}
> 
> However, in practice that's dumb.  The caller cannot know that the
> function doesn't access the object, so it must be cautious and enable
> -Wrestrict, which should be paranoid and do not allow passing references
> to the same object in different arguments, just in case the function
> decides to access to objects.  Of course, GCC reports a diagnostic for
> the previous code:
> 
> 	$ cc -Wall -Wextra dumb.c 
> 	dumb.c: In function ‘main’:
> 	dumb.c:13:21: warning: passing argument 1 to ‘restrict’-qualified parameter aliases with argument 2 [-Wrestrict]
> 	   13 |         return dumb(&x, &x);
> 	      |                     ^~  ~~
> 
> ... even when there's no UB, since the object is not being accessed.
> 
> But when the thing gets non-trivial, as in strtol(3), GCC misses the
> -Wrestrict diagnostic, as reported in
> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112833>.
> 
> Let's write a reproducer by altering the dumb.c program from above, with
> just another reference:
> 
> 	int
> 	dumb2(int *restrict a, int *restrict *restrict ap)
> 	{
> 		// We don't access the objects
> 		return a == *ap;
> 	}
> 
> 	int
> 	main(void)
> 	{
> 		int x = 3;
> 		int *xp = &x;
> 
> 		return dumb2(&x, &xp);
> 	}
> 
> GCC doesn't report anything bad here, even though it's basically the
> same as the program from above:
> 
> 	$ cc -Wall -Wextra dumb2.c
> 	$

strtol does have  a "char * restrict * restrict" though, so the
situation is different.   A "char **" and a "const char *"
shouldn't alias anyway. 


> 
> Again, there's no UB, but we really want to be cautious and get a
> diagnostic as callers, just in case the callee decides to access the
> object; we never know.
> 
> So, GCC should be patched to report a warning in the program above.
> That will also cause strtol(3) to start issuing warnings in use cases
> like the one I showed.
> 
> Even further, let's try something really weird: inequality comparison,
> which is only defined for pointers to the same array object:
> 
> 	int
> 	dumb3(int *restrict a, int *restrict *restrict ap)
> 	{
> 		// We don't access the objects
> 		return a > *ap;
> 	}
> 
> 	int
> 	main(void)
> 	{
> 		int x = 3;
> 		int *xp = &x;
> 
> 		return dumb3(&x, &xp);
> 	}
> 
> The behavior is still defined, since the obnjects are not accessed, but
> the compiler should really warn, on both sides:
> 
> -  The caller is passing references to the same object in restricted
>    parameters, which is a red flag.
> 
> -  The callee is comparing for inequality pointers that should, under
>    normal circumstances, cause Undefined Behavior.
> 
> 
> > And if this is really wrong you should report it to WG14 before changing
> > glibc.
> 
> Well, I don't know how to report that defect to WG14.  If you help me,
> I'll be pleased to do so.  Do they have a public mailing list or
> anything like that?

One can submit clarification or change requests:

https://www.open-std.org/jtc1/sc22/wg14/www/contributing.html

Martin

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2801 bytes --]

Hi Martin,

On Fri, Jul 05, 2024 at 05:02:15PM GMT, Martin Uecker wrote:
> > But when the thing gets non-trivial, as in strtol(3), GCC misses the
> > -Wrestrict diagnostic, as reported in
> > <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112833>.
> > 
> > Let's write a reproducer by altering the dumb.c program from above, with
> > just another reference:
> > 
> > 	int
> > 	dumb2(int *restrict a, int *restrict *restrict ap)
> > 	{
> > 		// We don't access the objects
> > 		return a == *ap;
> > 	}
> > 
> > 	int
> > 	main(void)
> > 	{
> > 		int x = 3;
> > 		int *xp = &x;
> > 
> > 		return dumb2(&x, &xp);
> > 	}
> > 
> > GCC doesn't report anything bad here, even though it's basically the
> > same as the program from above:
> > 
> > 	$ cc -Wall -Wextra dumb2.c
> > 	$
> 
> strtol does have  a "char * restrict * restrict" though, so the
> situation is different.   A "char **" and a "const char *"
> shouldn't alias anyway. 

Pedantically, it is actually declared as 'char **restrict' (the inner
one is not declared as restrict, even though it will be restricted,
since there are no other unrestricted pointers).

I've written functions that more closely resemble strtol(3), to show
that in the end they all share the same issue regarding const-ness:

	$ cat d.c 
	int d(const char *restrict ca, char *restrict a)
	{
		return ca > a;
	}

	int main(void)
	{
		char x = 3;
		char *xp = &x;
		d(xp, xp);
	}
	$ cc -Wall -Wextra d.c 
	d.c: In function ‘main’:
	d.c:10:9: warning: passing argument 2 to ‘restrict’-qualified parameter aliases with argument 1 [-Wrestrict]
	   10 |         d(xp, xp);
	      |         ^

This trivial program causes a diagnostic.  (Although I think the '>'
should also cause a diagnostic!!)

Let's add a reference, to resemble strtol(3):

	$ cat d2.c 
	int d2(const char *restrict ca, char *restrict *restrict ap)
	{
		return ca > *ap;
	}

	int main(void)
	{
		char x = 3;
		char *xp = &x;
		d2(xp, &xp);
	}
	$ cc -Wall -Wextra d2.c 
	$ 

Why does this not cause a -Wrestrict diagnostic, while d.c does?  How
are these programs any different regarding pointer restrict-ness?

> > Well, I don't know how to report that defect to WG14.  If you help me,
> > I'll be pleased to do so.  Do they have a public mailing list or
> > anything like that?
> 
> One can submit clarification or change requests:
> 
> https://www.open-std.org/jtc1/sc22/wg14/www/contributing.html

Thanks!  Will do.  Anyway, I think this should be discussed in glibc/gcc
in parallel, since it's clearly a missed diagnostic, and possibly a
dangerous use of restrict if the compiler does any assumptions that
shouldn't be done.

Have a lovely day!
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Martin Uecker]

Am Freitag, dem 05.07.2024 um 17:23 +0200 schrieb Alejandro Colomar:
> Hi Martin,
> 
> On Fri, Jul 05, 2024 at 05:02:15PM GMT, Martin Uecker wrote:
> > > But when the thing gets non-trivial, as in strtol(3), GCC misses the
> > > -Wrestrict diagnostic, as reported in
> > > <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112833>.
> > > 
> > > Let's write a reproducer by altering the dumb.c program from above, with
> > > just another reference:
> > > 
> > > 	int
> > > 	dumb2(int *restrict a, int *restrict *restrict ap)
> > > 	{
> > > 		// We don't access the objects
> > > 		return a == *ap;
> > > 	}
> > > 
> > > 	int
> > > 	main(void)
> > > 	{
> > > 		int x = 3;
> > > 		int *xp = &x;
> > > 
> > > 		return dumb2(&x, &xp);
> > > 	}
> > > 
> > > GCC doesn't report anything bad here, even though it's basically the
> > > same as the program from above:
> > > 
> > > 	$ cc -Wall -Wextra dumb2.c
> > > 	$
> > 
> > strtol does have  a "char * restrict * restrict" though, so the
> > situation is different.   A "char **" and a "const char *"
> > shouldn't alias anyway. 
> 
> Pedantically, it is actually declared as 'char **restrict' (the inner
> one is not declared as restrict, even though it will be restricted,
> since there are no other unrestricted pointers).
> 
> I've written functions that more closely resemble strtol(3), to show
> that in the end they all share the same issue regarding const-ness:
> 
> 	$ cat d.c 
> 	int d(const char *restrict ca, char *restrict a)
> 	{
> 		return ca > a;
> 	}
> 
> 	int main(void)
> 	{
> 		char x = 3;
> 		char *xp = &x;
> 		d(xp, xp);
> 	}
> 	$ cc -Wall -Wextra d.c 
> 	d.c: In function ‘main’:
> 	d.c:10:9: warning: passing argument 2 to ‘restrict’-qualified parameter aliases with argument 1 [-Wrestrict]
> 	   10 |         d(xp, xp);
> 	      |         ^
> 
> This trivial program causes a diagnostic.  (Although I think the '>'
> should also cause a diagnostic!!)
> 
> Let's add a reference, to resemble strtol(3):
> 
> 	$ cat d2.c 
> 	int d2(const char *restrict ca, char *restrict *restrict ap)
> 	{
> 		return ca > *ap;
> 	}
> 
> 	int main(void)
> 	{
> 		char x = 3;
> 		char *xp = &x;
> 		d2(xp, &xp);
> 	}
> 	$ cc -Wall -Wextra d2.c 
> 	$ 
> 
> Why does this not cause a -Wrestrict diagnostic, while d.c does?  How
> are these programs any different regarding pointer restrict-ness?

It would require data flow anaylsis to produce the diagnostic while
the first can simply be diagnosed by comparing arguments.

Martin

> 
> > > Well, I don't know how to report that defect to WG14.  If you help me,
> > > I'll be pleased to do so.  Do they have a public mailing list or
> > > anything like that?
> > 
> > One can submit clarification or change requests:
> > 
> > https://www.open-std.org/jtc1/sc22/wg14/www/contributing.html
> 
> Thanks!  Will do.  Anyway, I think this should be discussed in glibc/gcc
> in parallel, since it's clearly a missed diagnostic, and possibly a
> dangerous use of restrict if the compiler does any assumptions that
> shouldn't be done.
> 
> Have a lovely day!
> Alex
> 

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2790 bytes --]

Hi Martin,

On Fri, Jul 05, 2024 at 05:34:55PM GMT, Martin Uecker wrote:
> > I've written functions that more closely resemble strtol(3), to show
> > that in the end they all share the same issue regarding const-ness:

(Above I meant s/const/restrict/)

> > 
> > 	$ cat d.c 
> > 	int d(const char *restrict ca, char *restrict a)
> > 	{
> > 		return ca > a;
> > 	}
> > 
> > 	int main(void)
> > 	{
> > 		char x = 3;
> > 		char *xp = &x;
> > 		d(xp, xp);
> > 	}
> > 	$ cc -Wall -Wextra d.c 
> > 	d.c: In function ‘main’:
> > 	d.c:10:9: warning: passing argument 2 to ‘restrict’-qualified parameter aliases with argument 1 [-Wrestrict]
> > 	   10 |         d(xp, xp);
> > 	      |         ^
> > 
> > This trivial program causes a diagnostic.  (Although I think the '>'
> > should also cause a diagnostic!!)
> > 
> > Let's add a reference, to resemble strtol(3):
> > 
> > 	$ cat d2.c 
> > 	int d2(const char *restrict ca, char *restrict *restrict ap)
> > 	{
> > 		return ca > *ap;
> > 	}
> > 
> > 	int main(void)
> > 	{
> > 		char x = 3;
> > 		char *xp = &x;
> > 		d2(xp, &xp);
> > 	}
> > 	$ cc -Wall -Wextra d2.c 
> > 	$ 
> > 
> > Why does this not cause a -Wrestrict diagnostic, while d.c does?  How
> > are these programs any different regarding pointer restrict-ness?
> 
> It would require data flow anaylsis to produce the diagnostic while
> the first can simply be diagnosed by comparing arguments.

Agree.  It seems like a task for -fanalyzer.

	$ cc -Wall -Wextra -fanalyzer -fuse-linker-plugin -flto d2.c
	$

I'm unable to trigger that at all.  It's probably not implemented, I
guess.  I've updated the bug report
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112833> to change the
component to 'analyzer'.

At least, I hope there's consensus that while current GCC doesn't warn
about this, ideally it should, which means it should warn for valid uses
of strtol(3), which means strtol(3) should be fixed, in all of ISO,
POSIX, and glibc.

Cheers,
Alex

> > > > Well, I don't know how to report that defect to WG14.  If you help me,
> > > > I'll be pleased to do so.  Do they have a public mailing list or
> > > > anything like that?
> > > 
> > > One can submit clarification or change requests:
> > > 
> > > https://www.open-std.org/jtc1/sc22/wg14/www/contributing.html

P.S.:

I've sent a mail to UNE (the Spanish National Body for ISO), and
asked them about joining WG14.  Let's see what they say.

P.S. 2:

I'm also preparing a paper; would you mind championing it if I'm not yet
able to do it when it's ready?

P.S. 3:

Do you know of any Spanish member of WG14?  Maybe I can talk with them
to have more information about how they work.

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Xi Ruoyao]

On Fri, 2024-07-05 at 17:53 +0200, Alejandro Colomar wrote:
> At least, I hope there's consensus that while current GCC doesn't warn
> about this, ideally it should, which means it should warn for valid uses
> of strtol(3), which means strtol(3) should be fixed, in all of ISO,
> POSIX, and glibc.

It **shouldn't**.  strtol will only violate restrict if it's wrongly
implemented, or something dumb is done like "strtol((const char*) &p,
&p, 0)".

See my previous reply.

-- 
Xi Ruoyao <xry111@xry111.site>
School of Aerospace Science and Technology, Xidian University

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Xi Ruoyao]

On Sat, 2024-07-06 at 00:01 +0800, Xi Ruoyao wrote:
> On Fri, 2024-07-05 at 17:53 +0200, Alejandro Colomar wrote:
> > At least, I hope there's consensus that while current GCC doesn't warn
> > about this, ideally it should, which means it should warn for valid uses
> > of strtol(3), which means strtol(3) should be fixed, in all of ISO,
> > POSIX, and glibc.
> 
> It **shouldn't**.  strtol will only violate restrict if it's wrongly
> implemented, or something dumb is done like "strtol((const char*) &p,
> &p, 0)".
> 
> See my previous reply.

Also even if we'll introduce an over-eager warning having many
positives, there are still many possibilities besides changing the
standard to satisfy such a warning.  For example, g++ -Wdangling-
reference has many false positives (IMO it's over-eager) so a
no_dangling attribute has been added to allow suppressing it.

-- 
Xi Ruoyao <xry111@xry111.site>
School of Aerospace Science and Technology, Xidian University

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Jonathan Wakely]

On Fri, 5 Jul 2024 at 17:02, Xi Ruoyao via Gcc <gcc@gcc.gnu.org> wrote:
>
> On Fri, 2024-07-05 at 17:53 +0200, Alejandro Colomar wrote:
> > At least, I hope there's consensus that while current GCC doesn't warn
> > about this, ideally it should, which means it should warn for valid uses
> > of strtol(3), which means strtol(3) should be fixed, in all of ISO,
> > POSIX, and glibc.
>
> It **shouldn't**.  strtol will only violate restrict if it's wrongly
> implemented, or something dumb is done like "strtol((const char*) &p,
> &p, 0)".
>
> See my previous reply.

Right, is there a valid use of strtol where a warning would be justified?

Showing that you can contrive a case where a const char* restrict and
char** restrict can alias doesn't mean there's a problem with strtol.

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Martin Uecker]

Am Freitag, dem 05.07.2024 um 17:24 +0100 schrieb Jonathan Wakely:
> On Fri, 5 Jul 2024 at 17:02, Xi Ruoyao via Gcc <gcc@gcc.gnu.org> wrote:
> > 
> > On Fri, 2024-07-05 at 17:53 +0200, Alejandro Colomar wrote:
> > > At least, I hope there's consensus that while current GCC doesn't warn
> > > about this, ideally it should, which means it should warn for valid uses
> > > of strtol(3), which means strtol(3) should be fixed, in all of ISO,
> > > POSIX, and glibc.
> > 
> > It **shouldn't**.  strtol will only violate restrict if it's wrongly
> > implemented, or something dumb is done like "strtol((const char*) &p,
> > &p, 0)".
> > 
> > See my previous reply.
> 
> Right, is there a valid use of strtol where a warning would be justified?
> 
> Showing that you can contrive a case where a const char* restrict and
> char** restrict can alias doesn't mean there's a problem with strtol.

I think his point is that a const char* restrict and something which
is stored in a char* whose address is then passed can alias and there
a warning would make sense in other situations.   

But I am also not convinced removing restrict would be an improvement.
It would make more sense to have an annotation that indicates that
endptr is only used as output.

Martin  

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 2174 bytes --]

Hi,

On Fri, Jul 05, 2024 at 06:30:50PM GMT, Martin Uecker wrote:
> Am Freitag, dem 05.07.2024 um 17:24 +0100 schrieb Jonathan Wakely:
> > On Fri, 5 Jul 2024 at 17:02, Xi Ruoyao via Gcc <gcc@gcc.gnu.org> wrote:
> > > 
> > > On Fri, 2024-07-05 at 17:53 +0200, Alejandro Colomar wrote:
> > > > At least, I hope there's consensus that while current GCC doesn't warn
> > > > about this, ideally it should, which means it should warn for valid uses
> > > > of strtol(3), which means strtol(3) should be fixed, in all of ISO,
> > > > POSIX, and glibc.
> > > 
> > > It **shouldn't**.  strtol will only violate restrict if it's wrongly
> > > implemented, or something dumb is done like "strtol((const char*) &p,
> > > &p, 0)".
> > > 
> > > See my previous reply.

That's not right.  See my reply to yours, Xi.  The restrict in

	char **endptr

already prevents calls such as strtol(x, x, 0).

The restrict in

	const char *nptr

provides nothing.

> > 
> > Right, is there a valid use of strtol where a warning would be justified?

Is there any valid reason to have restrict in the _first_ parameter of
strtol(3)?  Other than "ISO C says so"?  I'll take my beef with ISO C to
WG14, and hopefully get that fixed.  Can we please discuss this
technically, ignoring the existence of ISO C, for the time being?

> > Showing that you can contrive a case where a const char* restrict and
> > char** restrict can alias doesn't mean there's a problem with strtol.
> 
> I think his point is that a const char* restrict and something which
> is stored in a char* whose address is then passed can alias and there
> a warning would make sense in other situations.   

Indeed.

> But I am also not convinced removing restrict would be an improvement.
> It would make more sense to have an annotation that indicates that
> endptr is only used as output.

What is the benefit of keeping restrict there?  It doesn't provide any
benefits, AFAICS.

I've prepared a paper for wg14.  I'll ask for a number, but will attach
it here already.  I also attach the man(7) source code for it.

Cheers,
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #1.2: strtol.man --]
[-- Type: application/x-troff-man, Size: 13705 bytes --]

[-- Attachment #1.3: strtol.pdf --]
[-- Type: application/pdf, Size: 13379 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Jonathan Wakely]

On Fri, 5 Jul 2024 at 20:28, Alejandro Colomar <alx@kernel.org> wrote:
>
> Hi,
>
> On Fri, Jul 05, 2024 at 06:30:50PM GMT, Martin Uecker wrote:
> > Am Freitag, dem 05.07.2024 um 17:24 +0100 schrieb Jonathan Wakely:
> > > On Fri, 5 Jul 2024 at 17:02, Xi Ruoyao via Gcc <gcc@gcc.gnu.org> wrote:
> > > >
> > > > On Fri, 2024-07-05 at 17:53 +0200, Alejandro Colomar wrote:
> > > > > At least, I hope there's consensus that while current GCC doesn't warn
> > > > > about this, ideally it should, which means it should warn for valid uses
> > > > > of strtol(3), which means strtol(3) should be fixed, in all of ISO,
> > > > > POSIX, and glibc.
> > > >
> > > > It **shouldn't**.  strtol will only violate restrict if it's wrongly
> > > > implemented, or something dumb is done like "strtol((const char*) &p,
> > > > &p, 0)".
> > > >
> > > > See my previous reply.
>
> That's not right.  See my reply to yours, Xi.  The restrict in
>
>         char **endptr
>
> already prevents calls such as strtol(x, x, 0).

That seems to contradict footnote 153 in C23.

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1501 bytes --]

Hi Jonathan,

On Fri, Jul 05, 2024 at 08:38:15PM GMT, Jonathan Wakely wrote:
> On Fri, 5 Jul 2024 at 20:28, Alejandro Colomar <alx@kernel.org> wrote:
> >
> > Hi,
> >
> > On Fri, Jul 05, 2024 at 06:30:50PM GMT, Martin Uecker wrote:
> > > Am Freitag, dem 05.07.2024 um 17:24 +0100 schrieb Jonathan Wakely:
> > > > On Fri, 5 Jul 2024 at 17:02, Xi Ruoyao via Gcc <gcc@gcc.gnu.org> wrote:
> > > > >
> > > > > On Fri, 2024-07-05 at 17:53 +0200, Alejandro Colomar wrote:
> > > > > > At least, I hope there's consensus that while current GCC doesn't warn
> > > > > > about this, ideally it should, which means it should warn for valid uses
> > > > > > of strtol(3), which means strtol(3) should be fixed, in all of ISO,
> > > > > > POSIX, and glibc.
> > > > >
> > > > > It **shouldn't**.  strtol will only violate restrict if it's wrongly
> > > > > implemented, or something dumb is done like "strtol((const char*) &p,
> > > > > &p, 0)".
> > > > >
> > > > > See my previous reply.
> >
> > That's not right.  See my reply to yours, Xi.  The restrict in
> >
> >         char **endptr
> >
> > already prevents calls such as strtol(x, x, 0).
> 
> That seems to contradict footnote 153 in C23.

Did you mean a different footnote number?

Here's 153 in N3047:

153) An implementation can delay the choice of which integer type until
     all enumeration constants have been seen.

which seems completely unrelated.

Cheers,
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Jonathan Wakely]

On Fri, 5 Jul 2024 at 20:47, Alejandro Colomar <alx@kernel.org> wrote:
>
> Hi Jonathan,
>
> On Fri, Jul 05, 2024 at 08:38:15PM GMT, Jonathan Wakely wrote:
> > On Fri, 5 Jul 2024 at 20:28, Alejandro Colomar <alx@kernel.org> wrote:
> > >
> > > Hi,
> > >
> > > On Fri, Jul 05, 2024 at 06:30:50PM GMT, Martin Uecker wrote:
> > > > Am Freitag, dem 05.07.2024 um 17:24 +0100 schrieb Jonathan Wakely:
> > > > > On Fri, 5 Jul 2024 at 17:02, Xi Ruoyao via Gcc <gcc@gcc.gnu.org> wrote:
> > > > > >
> > > > > > On Fri, 2024-07-05 at 17:53 +0200, Alejandro Colomar wrote:
> > > > > > > At least, I hope there's consensus that while current GCC doesn't warn
> > > > > > > about this, ideally it should, which means it should warn for valid uses
> > > > > > > of strtol(3), which means strtol(3) should be fixed, in all of ISO,
> > > > > > > POSIX, and glibc.
> > > > > >
> > > > > > It **shouldn't**.  strtol will only violate restrict if it's wrongly
> > > > > > implemented, or something dumb is done like "strtol((const char*) &p,
> > > > > > &p, 0)".
> > > > > >
> > > > > > See my previous reply.
> > >
> > > That's not right.  See my reply to yours, Xi.  The restrict in
> > >
> > >         char **endptr
> > >
> > > already prevents calls such as strtol(x, x, 0).
> >
> > That seems to contradict footnote 153 in C23.
>
> Did you mean a different footnote number?

No.

>
> Here's 153 in N3047:

That draft is nearly two years old.

>
> 153) An implementation can delay the choice of which integer type until
>      all enumeration constants have been seen.
>
> which seems completely unrelated.

Because you're looking at a draft from nearly two years ago. Try N3220.

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 3018 bytes --]

Hi Jonathan,

On Fri, Jul 05, 2024 at 08:52:30PM GMT, Jonathan Wakely wrote:
> > > > > > > It **shouldn't**.  strtol will only violate restrict if it's wrongly
> > > > > > > implemented, or something dumb is done like "strtol((const char*) &p,
> > > > > > > &p, 0)".
> > > > > > >
> > > > > > > See my previous reply.
> > > >
> > > > That's not right.  See my reply to yours, Xi.  The restrict in
> > > >
> > > >         char **endptr
> > > >
> > > > already prevents calls such as strtol(x, x, 0).
> > >
> > > That seems to contradict footnote 153 in C23.
> >
> > Did you mean a different footnote number?
> 
> No.
> 
> >
> > Here's 153 in N3047:
> 
> That draft is nearly two years old.
> 
> >
> > 153) An implementation can delay the choice of which integer type until
> >      all enumeration constants have been seen.
> >
> > which seems completely unrelated.
> 
> Because you're looking at a draft from nearly two years ago. Try N3220.

Ahhh, sorry!  Indeed.

Let's quote it here, for others to not need to find it:

153) In other words, E depends on the value of P itself rather than on
     the value of an object referenced indirectly through P.  For
     example, if identifier p has type (int **restrict), then the
     pointer expressions p and p+1 are based on the restricted pointer
     object designated by p, but the pointer expressions *p and p[1] are
     not.

I don't think footnote 153 is problematic here.

Let's have this prototype:

	long int
	alx_strtol(const char *nptr, char **restrict endptr, int base);

and let's discuss some example of bad usage:

	char str[] = "1";

	s = str;
	alx_strtol(s, (char **)s, 0);

According to 153, the pointer expression endptr is based on the
restricted pointer object, but *endptr and nptr are not.

The user has passed s as endptr, and also s as nptr.  Thus, the object
s is being accessed via a restricted pointer, endptr, and a
non-restricted one, nptr.  That's UB.

Let's see a different example of bad usage:

	char str[] = "1";

	s = str;
	alx_strtol((char *)&s, &s, 0);

For similar reasons, it's also UB.

The compiler diagnoses both:

	$ cat r.c
	long alx_strtol(const char *s, char **restrict endp, int base);

	int main(void)
	{
		char x = 3;
		char *xp = &x;

		alx_strtol(xp, &xp, 0);  // Fine.
		alx_strtol(xp, (char **) xp, 0);  // Bug.
		alx_strtol((char *) &xp, &xp, 0);  // Bug.
	}
	$ cc -Wall -Wextra -S r.c
	r.c: In function ‘main’:
	r.c:9:24: warning: passing argument 2 to ‘restrict’-qualified parameter aliases with argument 1 [-Wrestrict]
	    9 |         alx_strtol(xp, (char **) xp, 0);  // Bug.
	      |                        ^~~~~~~~~~~~
	r.c:10:34: warning: passing argument 2 to ‘restrict’-qualified parameter aliases with argument 1 [-Wrestrict]
	   10 |         alx_strtol((char *) &xp, &xp, 0);  // Bug.
	      |                    ~~~~~~~~~~~~  ^~~

Cheers,
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Emanuele Torre]

On Fri, Jul 05, 2024 at 08:52:30PM +0100, Jonathan Wakely wrote:
> On Fri, 5 Jul 2024 at 20:47, Alejandro Colomar <alx@kernel.org> wrote:
> >
> > Here's 153 in N3047:
> 
> That draft is nearly two years old.
> 
> >
> > 153) An implementation can delay the choice of which integer type until
> >      all enumeration constants have been seen.
> >
> > which seems completely unrelated.
> 
> Because you're looking at a draft from nearly two years ago. Try N3220.

That is 6.7.3.1p3:



In what follows, a pointer expression E is said to be based on object P
    if (at some sequence point in the execution of B prior to the
    evaluation of E) modifying P to point to a copy of the array object
    into which it formerly pointed would change the value of E.153) Note
    that "based" is defined only for expressions with pointer types.

Footnote 153) In other words, E depends on the value of P itself rather
    than on the value of an object referenced indirectly through P. For
    example, if identifier p has type (int **restrict), then the pointer
    expressions p and p+1 are based on the restricted pointer object
    designated by p, but the pointer expressions *p and p[1] are not.



Which would be the same paragraph of the same section on N3047, but
footnote number 168.

o/
 emanuele6

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Ben Boeckel]

On Fri, Jul 05, 2024 at 22:15:44 +0200, Emanuele Torre via Gcc wrote:
> That is 6.7.3.1p3:
> 
> 
> 
> In what follows, a pointer expression E is said to be based on object P
>     if (at some sequence point in the execution of B prior to the
>     evaluation of E) modifying P to point to a copy of the array object
>     into which it formerly pointed would change the value of E.153) Note
>     that "based" is defined only for expressions with pointer types.
> 
> Footnote 153) In other words, E depends on the value of P itself rather
>     than on the value of an object referenced indirectly through P. For
>     example, if identifier p has type (int **restrict), then the pointer
>     expressions p and p+1 are based on the restricted pointer object
>     designated by p, but the pointer expressions *p and p[1] are not.
> 
> 
> 
> Which would be the same paragraph of the same section on N3047, but
> footnote number 168.

Obviously, we need better types on our standardese pointers. Jonathan
used a pointer of type `footnote _N3220*` while Alejandro was expecting
a pointer of type `footnote _N3047*`. Of course, given that they end up
referring to the same thing, this may have interesting implications when
they are used with `restrict`.

--Ben

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Martin Uecker]

Am Freitag, dem 05.07.2024 um 21:28 +0200 schrieb Alejandro Colomar:

...
> 
> > > Showing that you can contrive a case where a const char* restrict and
> > > char** restrict can alias doesn't mean there's a problem with strtol.
> > 
> > I think his point is that a const char* restrict and something which
> > is stored in a char* whose address is then passed can alias and there
> > a warning would make sense in other situations.   
> 
> Indeed.
> 
> > But I am also not convinced removing restrict would be an improvement.
> > It would make more sense to have an annotation that indicates that
> > endptr is only used as output.
> 
> What is the benefit of keeping restrict there?  It doesn't provide any
> benefits, AFAICS.

Not really I think.  I am generally not a fan of restrict.
IMHO it is misdesigned and I would like to see it replaced
with something better.  But I also not convinced it really
helps to remove it here.

If we marked endptr as "write_only" (which it might already
be) then a future warning mechanism for -Wrestrict could
ignore the content of *endptr. 

Martin

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Jonathan Wakely]

On Fri, 5 Jul 2024 at 21:26, Martin Uecker <muecker@gwdg.de> wrote:
>
> Am Freitag, dem 05.07.2024 um 21:28 +0200 schrieb Alejandro Colomar:
>
> ...
> >
> > > > Showing that you can contrive a case where a const char* restrict and
> > > > char** restrict can alias doesn't mean there's a problem with strtol.
> > >
> > > I think his point is that a const char* restrict and something which
> > > is stored in a char* whose address is then passed can alias and there
> > > a warning would make sense in other situations.
> >
> > Indeed.
> >
> > > But I am also not convinced removing restrict would be an improvement.
> > > It would make more sense to have an annotation that indicates that
> > > endptr is only used as output.
> >
> > What is the benefit of keeping restrict there?  It doesn't provide any
> > benefits, AFAICS.
>
> Not really I think.  I am generally not a fan of restrict.
> IMHO it is misdesigned and I would like to see it replaced
> with something better.  But I also not convinced it really
> helps to remove it here.
>
> If we marked endptr as "write_only" (which it might already
> be) then a future warning mechanism for -Wrestrict could
> ignore the content of *endptr.


That seems more useful. Add semantic information instead of taking it
away. If the concern is a hypothetical future compiler warning that
would give false positives for perfectly valid uses of strtol, then
the problem is the compiler warning, not strtol. If additional
information can be added to avoid the false positives (and also
possibly optimize the code better), then that wouldn't require a
change to the standard motivated by a hypothetical compiler warning.

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2130 bytes --]

On Fri, Jul 05, 2024 at 09:28:46PM GMT, Jonathan Wakely wrote:
> On Fri, 5 Jul 2024 at 21:26, Martin Uecker <muecker@gwdg.de> wrote:
> >
> > Am Freitag, dem 05.07.2024 um 21:28 +0200 schrieb Alejandro Colomar:
> >
> > ...
> > >
> > > > > Showing that you can contrive a case where a const char* restrict and
> > > > > char** restrict can alias doesn't mean there's a problem with strtol.
> > > >
> > > > I think his point is that a const char* restrict and something which
> > > > is stored in a char* whose address is then passed can alias and there
> > > > a warning would make sense in other situations.
> > >
> > > Indeed.
> > >
> > > > But I am also not convinced removing restrict would be an improvement.
> > > > It would make more sense to have an annotation that indicates that
> > > > endptr is only used as output.
> > >
> > > What is the benefit of keeping restrict there?  It doesn't provide any
> > > benefits, AFAICS.
> >
> > Not really I think.  I am generally not a fan of restrict.
> > IMHO it is misdesigned and I would like to see it replaced
> > with something better.  But I also not convinced it really
> > helps to remove it here.
> >
> > If we marked endptr as "write_only" (which it might already
> > be) then a future warning mechanism for -Wrestrict could
> > ignore the content of *endptr.
> 
> 
> That seems more useful. Add semantic information instead of taking it
> away.

How does restrict on nptr (or conversely on *endptr) add any semantic
information?  Can you please phrase the semantic information provided by
it?  How is it useful to the caller?  How is it useful to the callee?

Cheers,
Alex

> If the concern is a hypothetical future compiler warning that
> would give false positives for perfectly valid uses of strtol, then
> the problem is the compiler warning, not strtol. If additional
> information can be added to avoid the false positives (and also
> possibly optimize the code better), then that wouldn't require a
> change to the standard motivated by a hypothetical compiler warning.

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1833 bytes --]

On Fri, Jul 05, 2024 at 09:28:46PM GMT, Jonathan Wakely wrote:
> > If we marked endptr as "write_only" (which it might already
> > be) then a future warning mechanism for -Wrestrict could
> > ignore the content of *endptr.
> 
> 
> That seems more useful. Add semantic information instead of taking it
> away. If the concern is a hypothetical future compiler warning that
> would give false positives for perfectly valid uses of strtol, then
> the problem is the compiler warning, not strtol. If additional
> information can be added to avoid the false positives (and also
> possibly optimize the code better), then that wouldn't require a
> change to the standard motivated by a hypothetical compiler warning.

Let me be a little bit sarcastic.

If so, let's take down -Wrestrict at all, because it triggers false
positives at the same rate.  How is it even in -Wall and not just
-Wextra?

Here's a false positive:

	$ cat d.c
	int is_same_pointer(const char *restrict ca, char *restrict a)
	{
		return ca == a;
	}

	int main(void)
	{
		char x = 3;
		char *xp = &x;
		is_same_pointer(xp, xp);
	}
	$ cc -Wall d.c
	d.c: In function ‘main’:
	d.c:10:9: warning: passing argument 2 to ‘restrict’-qualified parameter aliases with argument 1 [-Wrestrict]
	   10 |         d(xp, xp);
	      |         ^~~~~~~~~~~~~~~

It's impossible to know if a use of restrict causes UB without reading
both the source code of the caller and the callee, so except for
-fanalyzer, it's impossible to diagnose something with certainty.

So, it's certainly not something we want in -Wall.

Or should I remove the 'restrict' qualifier from that function, loosing
"precious" semantic information, just because the compiler doesn't like
it?

Cheers,
Alex


-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Jonathan Wakely]

On Fri, 5 Jul 2024 at 21:55, Alejandro Colomar <alx@kernel.org> wrote:
>
> On Fri, Jul 05, 2024 at 09:28:46PM GMT, Jonathan Wakely wrote:
> > > If we marked endptr as "write_only" (which it might already
> > > be) then a future warning mechanism for -Wrestrict could
> > > ignore the content of *endptr.
> >
> >
> > That seems more useful. Add semantic information instead of taking it
> > away. If the concern is a hypothetical future compiler warning that
> > would give false positives for perfectly valid uses of strtol, then
> > the problem is the compiler warning, not strtol. If additional
> > information can be added to avoid the false positives (and also
> > possibly optimize the code better), then that wouldn't require a
> > change to the standard motivated by a hypothetical compiler warning.
>
> Let me be a little bit sarcastic.
>
> If so, let's take down -Wrestrict at all, because it triggers false
> positives at the same rate.  How is it even in -Wall and not just
> -Wextra?
>
> Here's a false positive:
>
>         $ cat d.c
>         int is_same_pointer(const char *restrict ca, char *restrict a)
>         {
>                 return ca == a;

This is a strawman argument, because all your example functions have
been pretty useless and/or not good uses of restrict.

Yes, if you put restrict on functions where you don't ever access any
objects through the pointers, the restrict qualifiers are misleading
and so compilers might give bad warnings for your bad API.

>         }
>
>         int main(void)
>         {
>                 char x = 3;
>                 char *xp = &x;
>                 is_same_pointer(xp, xp);
>         }
>         $ cc -Wall d.c
>         d.c: In function ‘main’:
>         d.c:10:9: warning: passing argument 2 to ‘restrict’-qualified parameter aliases with argument 1 [-Wrestrict]
>            10 |         d(xp, xp);
>               |         ^~~~~~~~~~~~~~~
>
> It's impossible to know if a use of restrict causes UB without reading
> both the source code of the caller and the callee, so except for
> -fanalyzer, it's impossible to diagnose something with certainty.
>
> So, it's certainly not something we want in -Wall.
>
> Or should I remove the 'restrict' qualifier from that function, loosing
> "precious" semantic information, just because the compiler doesn't like
> it?
>
> Cheers,
> Alex
>
>
> --
> <https://www.alejandro-colomar.es/>

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 3619 bytes --]

Hi Jonathan,

On Fri, Jul 05, 2024 at 10:39:52PM GMT, Jonathan Wakely wrote:
> On Fri, 5 Jul 2024 at 21:55, Alejandro Colomar <alx@kernel.org> wrote:
> >
> > On Fri, Jul 05, 2024 at 09:28:46PM GMT, Jonathan Wakely wrote:
> > > > If we marked endptr as "write_only" (which it might already
> > > > be) then a future warning mechanism for -Wrestrict could
> > > > ignore the content of *endptr.
> > >
> > >
> > > That seems more useful. Add semantic information instead of taking it
> > > away. If the concern is a hypothetical future compiler warning that
> > > would give false positives for perfectly valid uses of strtol, then
> > > the problem is the compiler warning, not strtol. If additional
> > > information can be added to avoid the false positives (and also
> > > possibly optimize the code better), then that wouldn't require a
> > > change to the standard motivated by a hypothetical compiler warning.
> >
> > Let me be a little bit sarcastic.
> >
> > If so, let's take down -Wrestrict at all, because it triggers false
> > positives at the same rate.  How is it even in -Wall and not just
> > -Wextra?
> >
> > Here's a false positive:
> >
> >         $ cat d.c
> >         int is_same_pointer(const char *restrict ca, char *restrict a)
> >         {
> >                 return ca == a;
> 
> This is a strawman argument, because all your example functions have
> been pretty useless and/or not good uses of restrict.
> 
> Yes, if you put restrict on functions where you don't ever access any
> objects through the pointers, the restrict qualifiers are misleading

That's precisely the case with strtol(3): it doesn't access any objects
through *endptr, and so that pointer need not be restrict.

Then, nptr is a read-only pointer, so is doesn't matter either if it's
accessed or not.

Let's say we add as many attributes as possible to strtol(3):

[[gnu::access(read_only, 1)]]
[[gnu::access(write_only, 1)]]
[[gnu::leaf]]
[[gnu::nothrow]]
[[gnu::null_terminated_string_arg(1)]]
 // [[gnu::access(none, *1)]]
long
alx_strtol(const char *nptr, char **_Nullable restrict endp, int base);

Let's say we could mark *endptr as a 'access(none)' pointer, since it's
not accessed.  Let's say we do that with [[gnu::access(none, *1)]].

Then, do you think the information of that prototype is any different
than a prototype with restrict on the remaining pointers?

[[gnu::access(read_only, 1)]]
[[gnu::access(write_only, 1)]]
[[gnu::leaf]]
[[gnu::nothrow]]
[[gnu::null_terminated_string_arg(1)]]
 // [[gnu::access(none, *1)]]
long
alx_strtol(const char *restrict nptr,
           char *restrict *_Nullable restrict endp, int base);

I don't think so.  Since *endptr is access(none), it certainly cannot
access nptr, and thus the qualifier on nptr is superfluous.

And even without the hypothetical [[gnu::access(none, *1)]]:

-  The callee doesn't care about restrict, because it doesn't access
   any objects via *endptr, so it certainly knows that nptr can be read
   without any concerns about optimization.

-  The caller can't know if strtol(3) accesses *endptr, or nptr, and so
   it can't optimize.  Unless it passes an uninitialized value in
   *endptr, which means the caller knows for sure that nptr won't be
   written, regardless of restrict on it or not.

Please, describe what's the information you think is being added by
having restrict on nptr, on how it would be lost if we remove it.

Cheers,
Alex

> and so compilers might give bad warnings for your bad API.

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 3999 bytes --]

On Sat, Jul 06, 2024 at 12:02:06AM GMT, Alejandro Colomar wrote:
> Hi Jonathan,
> 
> On Fri, Jul 05, 2024 at 10:39:52PM GMT, Jonathan Wakely wrote:
> > On Fri, 5 Jul 2024 at 21:55, Alejandro Colomar <alx@kernel.org> wrote:
> > >
> > > On Fri, Jul 05, 2024 at 09:28:46PM GMT, Jonathan Wakely wrote:
> > > > > If we marked endptr as "write_only" (which it might already
> > > > > be) then a future warning mechanism for -Wrestrict could
> > > > > ignore the content of *endptr.
> > > >
> > > >
> > > > That seems more useful. Add semantic information instead of taking it
> > > > away. If the concern is a hypothetical future compiler warning that
> > > > would give false positives for perfectly valid uses of strtol, then
> > > > the problem is the compiler warning, not strtol. If additional
> > > > information can be added to avoid the false positives (and also
> > > > possibly optimize the code better), then that wouldn't require a
> > > > change to the standard motivated by a hypothetical compiler warning.
> > >
> > > Let me be a little bit sarcastic.
> > >
> > > If so, let's take down -Wrestrict at all, because it triggers false
> > > positives at the same rate.  How is it even in -Wall and not just
> > > -Wextra?
> > >
> > > Here's a false positive:
> > >
> > >         $ cat d.c
> > >         int is_same_pointer(const char *restrict ca, char *restrict a)
> > >         {
> > >                 return ca == a;
> > 
> > This is a strawman argument, because all your example functions have
> > been pretty useless and/or not good uses of restrict.
> > 
> > Yes, if you put restrict on functions where you don't ever access any
> > objects through the pointers, the restrict qualifiers are misleading
> 
> That's precisely the case with strtol(3): it doesn't access any objects
> through *endptr, and so that pointer need not be restrict.
> 
> Then, nptr is a read-only pointer, so is doesn't matter either if it's
> accessed or not.
> 
> Let's say we add as many attributes as possible to strtol(3):
> 
> [[gnu::access(read_only, 1)]]
> [[gnu::access(write_only, 1)]]

s/1/2/

> [[gnu::leaf]]
> [[gnu::nothrow]]
> [[gnu::null_terminated_string_arg(1)]]
>  // [[gnu::access(none, *1)]]

s/*1/*2/

> long
> alx_strtol(const char *nptr, char **_Nullable restrict endp, int base);
> 
> Let's say we could mark *endptr as a 'access(none)' pointer, since it's
> not accessed.  Let's say we do that with [[gnu::access(none, *1)]].

s/*1/*2/

> Then, do you think the information of that prototype is any different
> than a prototype with restrict on the remaining pointers?
> 
> [[gnu::access(read_only, 1)]]
> [[gnu::access(write_only, 1)]]

s/1/2/

> [[gnu::leaf]]
> [[gnu::nothrow]]
> [[gnu::null_terminated_string_arg(1)]]
>  // [[gnu::access(none, *1)]]

s/*1/*2/

> long
> alx_strtol(const char *restrict nptr,
>            char *restrict *_Nullable restrict endp, int base);
> 
> I don't think so.  Since *endptr is access(none), it certainly cannot
> access nptr, and thus the qualifier on nptr is superfluous.
> 
> And even without the hypothetical [[gnu::access(none, *1)]]:

s/*1/*2/

> 
> -  The callee doesn't care about restrict, because it doesn't access
>    any objects via *endptr, so it certainly knows that nptr can be read
>    without any concerns about optimization.
> 
> -  The caller can't know if strtol(3) accesses *endptr, or nptr, and so
>    it can't optimize.  Unless it passes an uninitialized value in
>    *endptr, which means the caller knows for sure that nptr won't be
>    written, regardless of restrict on it or not.
> 
> Please, describe what's the information you think is being added by
> having restrict on nptr, on how it would be lost if we remove it.
> 
> Cheers,
> Alex
> 
> > and so compilers might give bad warnings for your bad API.
> 
> -- 
> <https://www.alejandro-colomar.es/>



-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Xi Ruoyao]

On Sat, 2024-07-06 at 00:02 +0200, Alejandro Colomar wrote:
> That's precisely the case with strtol(3): it doesn't access any objects
> through *endptr, and so that pointer need not be restrict.
> 
> Then, nptr is a read-only pointer, so is doesn't matter either if it's
> accessed or not.

Restrict allows to reorder any writes to other objects with an read from
nptr then. In strtol at least errno can be written, and depending on the
implementation of locale things there may be more.

TBAA does not help here because char aliases with anything.

-- 
Xi Ruoyao <xry111@xry111.site>
School of Aerospace Science and Technology, Xidian University

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Xi Ruoyao]

On Sat, 2024-07-06 at 10:24 +0800, Xi Ruoyao wrote:
> On Sat, 2024-07-06 at 00:02 +0200, Alejandro Colomar wrote:
> > That's precisely the case with strtol(3): it doesn't access any objects
> > through *endptr, and so that pointer need not be restrict.
> > 
> > Then, nptr is a read-only pointer, so is doesn't matter either if it's
> > accessed or not.
> 
> Restrict allows to reorder any writes to other objects with an read from
> nptr then. In strtol at least errno can be written, and depending on the
> implementation of locale things there may be more.
> 
> TBAA does not help here because char aliases with anything.

Also in the implementation of strtol if it passes nptr to another
auxiliary function and that function does not have some fancy const or
access attributes, the compiler will assume that function may write into
the buffer pointed by nptr because in C you can actually write via a
const T * unless it really points to a const T (not non-qualified T).

BTW among your list:

> > [[gnu::access(read_only, 1)]]
> > [[gnu::access(write_only, 2)]]
> > [[gnu::leaf]]
> > [[gnu::nothrow]]
> > [[gnu::null_terminated_string_arg(1)]]

IMO we should add these access attributes, they'll definitely help the
optimization (like, optimize away the initialization of a pointer).

We already have __THROW which expands to nothrow and leaf.

I'm not sure if null_terminated_string_arg is correct: is the following
invalid or not?

char p[] = {'1', ')'};
char *q;
strtol(p, &q, 10);
assert(q == &p[1]);

If this is invalid we should have null_terminated_string_arg so at least
we'll get a warning against this.

-- 
Xi Ruoyao <xry111@xry111.site>
School of Aerospace Science and Technology, Xidian University

[[gnu::null_terminated_string_arg(1)]] on strtol(1) (was: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like) functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1658 bytes --]

Hi Xi,

On Sat, Jul 06, 2024 at 10:39:41AM GMT, Xi Ruoyao wrote:
> BTW among your list:
> 
> > > [[gnu::access(read_only, 1)]]
> > > [[gnu::access(write_only, 2)]]
> > > [[gnu::leaf]]
> > > [[gnu::nothrow]]
> > > [[gnu::null_terminated_string_arg(1)]]
> 
> IMO we should add these access attributes, they'll definitely help the
> optimization (like, optimize away the initialization of a pointer).
> 
> We already have __THROW which expands to nothrow and leaf.
> 
> I'm not sure if null_terminated_string_arg is correct: is the following
> invalid or not?
> 
> char p[] = {'1', ')'};
> char *q;
> strtol(p, &q, 10);
> assert(q == &p[1]);
> 
> If this is invalid we should have null_terminated_string_arg so at least
> we'll get a warning against this.

ISO C says:

"""
The strtol, strtoll, strtoul, and strtoull functions convert the initial
portion of the string pointed to by nptr to long int, long long int,
unsigned long int, and unsigned long long int representation,
respectively.  First, they decompose the input string into three parts:
an initial, possibly empty, sequence of white-space characters (as
specified by the isspace function), a subject sequence resembling an
integer represented in some radix determined by the value of base, and a
final string of one or more unrecognized characters, including the
terminating null character of the input string.  Then, they attempt to
convert the subject sequence to an integer, and return the result.
"""
<http://port70.net/~nsz/c/c11/n1570.html#7.22.1.4p2>

I'd say it's a string.

Have a lovely day!
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 3013 bytes --]

Hi Xi,

On Sat, Jul 06, 2024 at 10:24:16AM GMT, Xi Ruoyao wrote:
> On Sat, 2024-07-06 at 00:02 +0200, Alejandro Colomar wrote:
> > That's precisely the case with strtol(3): it doesn't access any objects
> > through *endptr, and so that pointer need not be restrict.
> > 
> > Then, nptr is a read-only pointer, so is doesn't matter either if it's
> > accessed or not.
> 
> Restrict allows to reorder any writes to other objects with an read from
> nptr then. In strtol at least errno can be written, and depending on the
> implementation of locale things there may be more.

This does not apply here, I think.  Let's include errno in the list of
objects that strtol(3) takes, and list their access modes:

-  nptr		access(read_only)
-  *endptr	access(none)
-  endptr	access(read_write) [it checks for NULL; I had forgotten]
-  errno	access(read_write)

In the callee:
~~~~~~~~~~~~~~

The access modes are known by the callee, because of course it knows
what it does, so even without the attributes, it knows that.

strtol(3) cannot write to errno until it has parsed the string.  And
once it knows it has failed (so wants to set errno), it has no reasons
to read nptr again.  Thus, even without knowing if 'errno' and 'nptr'
are the same thing, there's nothing that could be optimized.

*endptr is access(none), so it is implicitly restricted even without
specifying restrict on it; the callee couldn't care less about it.

endptr is 'restrict', so it can be treated separately from the rest.

In the caller:
~~~~~~~~~~~~~~

We can't specify the access mode of *endptr nor errno, so the caller
must assume they are read_write.

endptr is 'restrict', but this is only useful for having warnings
against dumb things such as strtol(x, x, 0).  Other than that, the
caller knows what it has passed in endptr, so it knows it's a different
thing.

The caller also knows that it hasn't passed errno as nptr.

Then we must make a distinction in what the caller passes in *endptr:

*endptr is uninitialized:
~~~~~~~~~~~~~~~~~~~~~~~~~

The caller knows that nptr is restricted even without the qualifier,
since all other objects are either restricted, or known to be different.

*endptr points to the same thing as nptr:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Regardless of the 'restrict' qualifier being specified or not, the
caller has no way to determine if the callee accesses the object via
nptr or via *endptr, so it must assume the worst case: *endptr; and so
it must assume it could have written to it (since *endptr is non-const
--and even if it were const, as you said, it means nothing--).


So, even considering errno in the game, I don't see any difference if we
specify nptr to be restrict or not.

Thanks for the feedback!  I'll incorporate the discussion about errno in
the paper for WG14.

Have a lovely day!
Alex

> 
> TBAA does not help here because char aliases with anything.

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 3403 bytes --]

On Sat, Jul 06, 2024 at 08:10:28AM GMT, Alejandro Colomar wrote:
> Hi Xi,
> 
> On Sat, Jul 06, 2024 at 10:24:16AM GMT, Xi Ruoyao wrote:
> > On Sat, 2024-07-06 at 00:02 +0200, Alejandro Colomar wrote:
> > > That's precisely the case with strtol(3): it doesn't access any objects
> > > through *endptr, and so that pointer need not be restrict.
> > > 
> > > Then, nptr is a read-only pointer, so is doesn't matter either if it's
> > > accessed or not.
> > 
> > Restrict allows to reorder any writes to other objects with an read from
> > nptr then. In strtol at least errno can be written, and depending on the
> > implementation of locale things there may be more.
> 
> This does not apply here, I think.  Let's include errno in the list of
> objects that strtol(3) takes, and list their access modes:
> 
> -  nptr		access(read_only)
> -  *endptr	access(none)
> -  endptr	access(read_write) [it checks for NULL; I had forgotten]

Sorry, I was right the first time; it's write_only.  The NULL check is
on the pointer; not the pointee.

> -  errno	access(read_write)
> 
> In the callee:
> ~~~~~~~~~~~~~~
> 
> The access modes are known by the callee, because of course it knows
> what it does, so even without the attributes, it knows that.
> 
> strtol(3) cannot write to errno until it has parsed the string.  And
> once it knows it has failed (so wants to set errno), it has no reasons
> to read nptr again.  Thus, even without knowing if 'errno' and 'nptr'
> are the same thing, there's nothing that could be optimized.
> 
> *endptr is access(none), so it is implicitly restricted even without
> specifying restrict on it; the callee couldn't care less about it.
> 
> endptr is 'restrict', so it can be treated separately from the rest.
> 
> In the caller:
> ~~~~~~~~~~~~~~
> 
> We can't specify the access mode of *endptr nor errno, so the caller
> must assume they are read_write.
> 
> endptr is 'restrict', but this is only useful for having warnings
> against dumb things such as strtol(x, x, 0).  Other than that, the
> caller knows what it has passed in endptr, so it knows it's a different
> thing.
> 
> The caller also knows that it hasn't passed errno as nptr.
> 
> Then we must make a distinction in what the caller passes in *endptr:
> 
> *endptr is uninitialized:
> ~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> The caller knows that nptr is restricted even without the qualifier,
> since all other objects are either restricted, or known to be different.
> 
> *endptr points to the same thing as nptr:
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> Regardless of the 'restrict' qualifier being specified or not, the
> caller has no way to determine if the callee accesses the object via
> nptr or via *endptr, so it must assume the worst case: *endptr; and so
> it must assume it could have written to it (since *endptr is non-const
> --and even if it were const, as you said, it means nothing--).
> 
> 
> So, even considering errno in the game, I don't see any difference if we
> specify nptr to be restrict or not.
> 
> Thanks for the feedback!  I'll incorporate the discussion about errno in
> the paper for WG14.
> 
> Have a lovely day!
> Alex
> 
> > 
> > TBAA does not help here because char aliases with anything.
> 
> -- 
> <https://www.alejandro-colomar.es/>



-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Sam James]

Jonathan Wakely via Gcc <gcc@gcc.gnu.org> writes:

> On Fri, 5 Jul 2024 at 17:02, Xi Ruoyao via Gcc <gcc@gcc.gnu.org> wrote:
>>
>> On Fri, 2024-07-05 at 17:53 +0200, Alejandro Colomar wrote:
>> > At least, I hope there's consensus that while current GCC doesn't warn
>> > about this, ideally it should, which means it should warn for valid uses
>> > of strtol(3), which means strtol(3) should be fixed, in all of ISO,
>> > POSIX, and glibc.
>>
>> It **shouldn't**.  strtol will only violate restrict if it's wrongly
>> implemented, or something dumb is done like "strtol((const char*) &p,
>> &p, 0)".
>>
>> See my previous reply.
>
> Right, is there a valid use of strtol where a warning would be justified?
>
> Showing that you can contrive a case where a const char* restrict and
> char** restrict can alias doesn't mean there's a problem with strtol.

I still don't understand why it'd be appropriate for GCC and glibc to
override this without it even being *brought to* the committee, either.

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Martin Uecker]

Am Freitag, dem 05.07.2024 um 17:53 +0200 schrieb Alejandro Colomar:
> Hi Martin,
> 
> On Fri, Jul 05, 2024 at 05:34:55PM GMT, Martin Uecker wrote:
> > > I've written functions that more closely resemble strtol(3), to show
> > > that in the end they all share the same issue regarding const-ness:
> 
> (Above I meant s/const/restrict/)
> 
> > > 
> > > 	$ cat d.c 
> > > 	int d(const char *restrict ca, char *restrict a)
> > > 	{
> > > 		return ca > a;
> > > 	}
> > > 
> > > 	int main(void)
> > > 	{
> > > 		char x = 3;
> > > 		char *xp = &x;
> > > 		d(xp, xp);
> > > 	}
> > > 	$ cc -Wall -Wextra d.c 
> > > 	d.c: In function ‘main’:
> > > 	d.c:10:9: warning: passing argument 2 to ‘restrict’-qualified parameter aliases with argument 1 [-Wrestrict]
> > > 	   10 |         d(xp, xp);
> > > 	      |         ^
> > > 
> > > This trivial program causes a diagnostic.  (Although I think the '>'
> > > should also cause a diagnostic!!)
> > > 
> > > Let's add a reference, to resemble strtol(3):
> > > 
> > > 	$ cat d2.c 
> > > 	int d2(const char *restrict ca, char *restrict *restrict ap)
> > > 	{
> > > 		return ca > *ap;
> > > 	}
> > > 
> > > 	int main(void)
> > > 	{
> > > 		char x = 3;
> > > 		char *xp = &x;
> > > 		d2(xp, &xp);
> > > 	}
> > > 	$ cc -Wall -Wextra d2.c 
> > > 	$ 
> > > 
> > > Why does this not cause a -Wrestrict diagnostic, while d.c does?  How
> > > are these programs any different regarding pointer restrict-ness?
> > 
> > It would require data flow anaylsis to produce the diagnostic while
> > the first can simply be diagnosed by comparing arguments.
> 
> Agree.  It seems like a task for -fanalyzer.
> 
> 	$ cc -Wall -Wextra -fanalyzer -fuse-linker-plugin -flto d2.c
> 	$
> 
> I'm unable to trigger that at all.  It's probably not implemented, I
> guess.  I've updated the bug report
> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112833> to change the
> component to 'analyzer'.
> 
> At least, I hope there's consensus that while current GCC doesn't warn
> about this, ideally it should, which means it should warn for valid uses
> of strtol(3), which means strtol(3) should be fixed, in all of ISO,
> POSIX, and glibc.

I am not sure. 

> 
> > > > > Well, I don't know how to report that defect to WG14.  If you help me,
> > > > > I'll be pleased to do so.  Do they have a public mailing list or
> > > > > anything like that?
> > > > 
> > > > One can submit clarification or change requests:
> > > > 
> > > > https://www.open-std.org/jtc1/sc22/wg14/www/contributing.html
> 
> P.S.:
> 
> I've sent a mail to UNE (the Spanish National Body for ISO), and
> asked them about joining WG14.  Let's see what they say.
> 
> P.S. 2:
> 
> I'm also preparing a paper; would you mind championing it if I'm not yet
> able to do it when it's ready?

Guests can present too.

> 
> P.S. 3:
> 
> Do you know of any Spanish member of WG14?  Maybe I can talk with them
> to have more information about how they work.

You could ask Miguel Ojeda.

Martin

> 

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Jonathan Wakely]

On Fri, 5 Jul 2024 at 16:54, Alejandro Colomar via Gcc <gcc@gcc.gnu.org> wrote:
> At least, I hope there's consensus that while current GCC doesn't warn
> about this, ideally it should, which means it should warn for valid uses
> of strtol(3), which means strtol(3) should be fixed, in all of ISO,
> POSIX, and glibc.

I'm not convinced. It doesn't look like anybody else is convinced. I
wouldn't call that consensus.

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Richard Earnshaw (lists)]

On 05/07/2024 17:11, Jonathan Wakely via Gcc wrote:
> On Fri, 5 Jul 2024 at 16:54, Alejandro Colomar via Gcc <gcc@gcc.gnu.org> wrote:
>> At least, I hope there's consensus that while current GCC doesn't warn
>> about this, ideally it should, which means it should warn for valid uses
>> of strtol(3), which means strtol(3) should be fixed, in all of ISO,
>> POSIX, and glibc.
> 
> I'm not convinced. It doesn't look like anybody else is convinced. I
> wouldn't call that consensus.

And what's more this prototype is defined by the C standard.  If you have a beef with that, then you should take it up with the committee; I can't see GCC wanting to be different on this.

R.

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[LIU Hao]

[-- Attachment #1.1: Type: text/plain, Size: 730 bytes --]

在 2024-07-05 23:23, Alejandro Colomar via Gcc 写道:
> Hi Martin,
> 
> On Fri, Jul 05, 2024 at 05:02:15PM GMT, Martin Uecker wrote:
>>> But when the thing gets non-trivial, as in strtol(3), GCC misses the
>>> -Wrestrict diagnostic, as reported in

A pointer to `char` can alias any object, so in theory one could write code that looks like below. 
This piece of code is probably nonsense, but it illustrates the exact necessity of the `restrict` 
qualifiers:

    char*
    dumb(char* p)
      {
        strtol((const char*) &p, &p, 0);
        return p;
      }

    // warning: passing argument 2 to 'restrict'-qualified parameter
    // aliases with argument 1






-- 
Best regards,
LIU Hao


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Xi Ruoyao]

On Fri, 2024-07-05 at 17:23 +0200, Alejandro Colomar wrote:
> > strtol does have  a "char * restrict * restrict" though, so the
> > situation is different.   A "char **" and a "const char *"
> > shouldn't alias anyway. 
> 
> Pedantically, it is actually declared as 'char **restrict' (the inner
> one is not declared as restrict, even though it will be restricted,
> since there are no other unrestricted pointers).

So how's the following implementation of strtol (10-based, no negative
number handling, no overflow handling, ASCII-only) wrong?

long int my_strtol(const char *restrict nptr, char **restrict endptr)
{
  long ret = 0;

  while (isdigit(*nptr))
    ret = ret * 10 + (*nptr++ - '0');

  *endptr = (char *)nptr;
  return ret;
}

There's no dumb thing, there's no restrict violation (unless it's called
in a stupid way, see below), and there **shouldn't** be a -Wrestrict
warning.

If you do

char *x = NULL;
strtol((char *)&x, &x, 10);

it'll violate restrict.  Nobody sane should write this, and it's warned
anyway:

t.c: In function 'main':
t.c:6:28: warning: passing argument 2 to 'restrict'-qualified parameter
aliases with argument 1 [-Wrestrict]
    6 |         strtol((char *)&x, &x, 10);
      |                ~~~~~~~~~~  ^~

-- 
Xi Ruoyao <xry111@xry111.site>
School of Aerospace Science and Technology, Xidian University

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 3023 bytes --]

Hi Xi,

On Fri, Jul 05, 2024 at 11:55:05PM GMT, Xi Ruoyao wrote:
> On Fri, 2024-07-05 at 17:23 +0200, Alejandro Colomar wrote:
> > > strtol does have  a "char * restrict * restrict" though, so the
> > > situation is different.   A "char **" and a "const char *"
> > > shouldn't alias anyway. 
> > 
> > Pedantically, it is actually declared as 'char **restrict' (the inner
> > one is not declared as restrict, even though it will be restricted,
> > since there are no other unrestricted pointers).
> 
> So how's the following implementation of strtol (10-based, no negative
> number handling, no overflow handling, ASCII-only) wrong?
> 
> long int my_strtol(const char *restrict nptr, char **restrict endptr)
> {
>   long ret = 0;
> 
>   while (isdigit(*nptr))
>     ret = ret * 10 + (*nptr++ - '0');
> 
>   *endptr = (char *)nptr;
>   return ret;
> }
> 
> There's no dumb thing, there's no restrict violation (unless it's called
> in a stupid way, see below), and there **shouldn't** be a -Wrestrict
> warning.
> 
> If you do
> 
> char *x = NULL;
> strtol((char *)&x, &x, 10);

The restrict in `char **restrict endptr` already protects you from this.
You don't need to make the first parameter also restricted.  See:

	$ cat r.c 
	long alx_strtol(const char *nptr, char **restrict endp);

	int main(void)
	{
		char x = 3;
		char *xp = &x;

		alx_strtol(xp, &xp);  // Fine.
		alx_strtol(xp, (char **) xp);  // Bug.
		alx_strtol((char *) &xp, &xp);  // Bug.
	}
	$ cc -Wall -Wextra -S r.c 
	r.c: In function ‘main’:
	r.c:9:24: warning: passing argument 2 to ‘restrict’-qualified parameter aliases with argument 1 [-Wrestrict]
	    9 |         alx_strtol(xp, (char **) xp);  // Bug.
	      |                        ^~~~~~~~~~~~
	r.c:10:34: warning: passing argument 2 to ‘restrict’-qualified parameter aliases with argument 1 [-Wrestrict]
	   10 |         alx_strtol((char *) &xp, &xp);  // Bug.
	      |                    ~~~~~~~~~~~~  ^~~

Using my proposed prototype wouldn't case any warnings with a powerful
-fanalyzer that would be able to emit diagnostics with the current
prototype.

In strtol(3), there are 3 pointers:

-  nptr
-  *endptr
-  endptr

The first two should be allowed to alias each other (and at the end of
the call, they definitely alias each other).  It is only the third one
which must not alias any of the other, which is why my patch (v2) keeps
that restrict, but removes the other one.

Does that make sense?

Cheers,
Alex

> 
> it'll violate restrict.  Nobody sane should write this, and it's warned
> anyway:
> 
> t.c: In function 'main':
> t.c:6:28: warning: passing argument 2 to 'restrict'-qualified parameter
> aliases with argument 1 [-Wrestrict]
>     6 |         strtol((char *)&x, &x, 10);
>       |                ~~~~~~~~~~  ^~
> 
> -- 
> Xi Ruoyao <xry111@xry111.site>
> School of Aerospace Science and Technology, Xidian University

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 3516 bytes --]

On Fri, Jul 05, 2024 at 06:32:26PM GMT, Alejandro Colomar wrote:
> Hi Xi,
> 
> On Fri, Jul 05, 2024 at 11:55:05PM GMT, Xi Ruoyao wrote:
> > On Fri, 2024-07-05 at 17:23 +0200, Alejandro Colomar wrote:
> > > > strtol does have  a "char * restrict * restrict" though, so the
> > > > situation is different.   A "char **" and a "const char *"
> > > > shouldn't alias anyway. 
> > > 
> > > Pedantically, it is actually declared as 'char **restrict' (the inner
> > > one is not declared as restrict, even though it will be restricted,
> > > since there are no other unrestricted pointers).
> > 
> > So how's the following implementation of strtol (10-based, no negative
> > number handling, no overflow handling, ASCII-only) wrong?
> > 
> > long int my_strtol(const char *restrict nptr, char **restrict endptr)
> > {
> >   long ret = 0;
> > 
> >   while (isdigit(*nptr))
> >     ret = ret * 10 + (*nptr++ - '0');
> > 
> >   *endptr = (char *)nptr;
> >   return ret;
> > }
> > 
> > There's no dumb thing, there's no restrict violation (unless it's called
> > in a stupid way, see below), and there **shouldn't** be a -Wrestrict
> > warning.
> > 
> > If you do
> > 
> > char *x = NULL;
> > strtol((char *)&x, &x, 10);
> 
> The restrict in `char **restrict endptr` already protects you from this.
> You don't need to make the first parameter also restricted.  See:
> 
> 	$ cat r.c 
> 	long alx_strtol(const char *nptr, char **restrict endp);
> 
> 	int main(void)
> 	{
> 		char x = 3;
> 		char *xp = &x;
> 
> 		alx_strtol(xp, &xp);  // Fine.
> 		alx_strtol(xp, (char **) xp);  // Bug.
> 		alx_strtol((char *) &xp, &xp);  // Bug.
> 	}
> 	$ cc -Wall -Wextra -S r.c 
> 	r.c: In function ‘main’:
> 	r.c:9:24: warning: passing argument 2 to ‘restrict’-qualified parameter aliases with argument 1 [-Wrestrict]
> 	    9 |         alx_strtol(xp, (char **) xp);  // Bug.
> 	      |                        ^~~~~~~~~~~~
> 	r.c:10:34: warning: passing argument 2 to ‘restrict’-qualified parameter aliases with argument 1 [-Wrestrict]
> 	   10 |         alx_strtol((char *) &xp, &xp);  // Bug.
> 	      |                    ~~~~~~~~~~~~  ^~~
> 
> Using my proposed prototype wouldn't case any warnings with a powerful
> -fanalyzer that would be able to emit diagnostics with the current
> prototype.
> 
> In strtol(3), there are 3 pointers:
> 
> -  nptr
> -  *endptr
> -  endptr
> 
> The first two should be allowed to alias each other (and at the end of
> the call, they definitely alias each other).  It is only the third one
> which must not alias any of the other, which is why my patch (v2) keeps

(Whoops; forget about that v2; that was about a similar patch to
strsep(3).  In this case we're in patch v1; which already had that into
consideration.  Please read it as s/v2/v1/.)

> that restrict, but removes the other one.
> 
> Does that make sense?
> 
> Cheers,
> Alex
> 
> > 
> > it'll violate restrict.  Nobody sane should write this, and it's warned
> > anyway:
> > 
> > t.c: In function 'main':
> > t.c:6:28: warning: passing argument 2 to 'restrict'-qualified parameter
> > aliases with argument 1 [-Wrestrict]
> >     6 |         strtol((char *)&x, &x, 10);
> >       |                ~~~~~~~~~~  ^~
> > 
> > -- 
> > Xi Ruoyao <xry111@xry111.site>
> > School of Aerospace Science and Technology, Xidian University
> 
> -- 
> <https://www.alejandro-colomar.es/>



-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[WG14] Request for document number; strtol restrictness

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 341 bytes --]

Hi,

I have a paper for removing restrict from the first parameter of
strtol(3) et al.  The title is

	strtol(3) et al. should’t have a restricted first parameter.

If it helps, I already have a draft of the paper, which I attach (both
the PDF, and the man(7) source).

Cheers,
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #1.2: strtol.man --]
[-- Type: application/x-troff-man, Size: 13763 bytes --]

[-- Attachment #1.3: strtol.pdf --]
[-- Type: application/pdf, Size: 13445 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

RE: [WG14] Request for document number; strtol restrictness

[Daniel Plakosh]

Alex,

Your document number is below:

n3294 - strtol(3) et al. shouldn't have a restricted first parameter

Please return the updated document with this number

Best regards,

Dan

Technical Director - Enabling Mission Capability at Scale
Principal Member of the Technical Staff
Software Engineering Institute
Carnegie Mellon University
4500 Fifth Avenue
Pittsburgh, PA 15213
WORK: 412-268-7197
CELL: 412-427-4606

-----Original Message-----
From: Alejandro Colomar <alx@kernel.org> 
Sent: Friday, July 5, 2024 3:42 PM
To: dplakosh@cert.org
Cc: Martin Uecker <muecker@gwdg.de>; Jonathan Wakely <jwakely.gcc@gmail.com>; Xi Ruoyao <xry111@xry111.site>; Jakub Jelinek <jakub@redhat.com>; libc-alpha@sourceware.org; gcc@gcc.gnu.org; Paul Eggert <eggert@cs.ucla.edu>; linux-man@vger.kernel.org; LIU Hao <lh_mouse@126.com>; Richard Earnshaw <Richard.Earnshaw@arm.com>; Sam James <sam@gentoo.org>
Subject: [WG14] Request for document number; strtol restrictness

Hi,

I have a paper for removing restrict from the first parameter of
strtol(3) et al.  The title is

	strtol(3) et al. should’t have a restricted first parameter.

If it helps, I already have a draft of the paper, which I attach (both the PDF, and the man(7) source).

Cheers,
Alex

--
<https://www.alejandro-colomar.es/>

Re: [WG14] Request for document number; strtol restrictness

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1694 bytes --]

Hi Daniel,

On Sun, Jul 07, 2024 at 03:46:48PM GMT, Daniel Plakosh wrote:
> Alex,
> 
> Your document number is below:
> 
> n3294 - strtol(3) et al. shouldn't have a restricted first parameter
> 
> Please return the updated document with this number

Am I allowed to retitle the paper?

  n3294 - [[noalias()]] function attribute as a replacement of restrict

Sorry for any inconveniences.

Thanks,
Alex

> 
> Best regards,
> 
> Dan
> 
> Technical Director - Enabling Mission Capability at Scale
> Principal Member of the Technical Staff
> Software Engineering Institute
> Carnegie Mellon University
> 4500 Fifth Avenue
> Pittsburgh, PA 15213
> WORK: 412-268-7197
> CELL: 412-427-4606
> 
> -----Original Message-----
> From: Alejandro Colomar <alx@kernel.org> 
> Sent: Friday, July 5, 2024 3:42 PM
> To: dplakosh@cert.org
> Cc: Martin Uecker <muecker@gwdg.de>; Jonathan Wakely <jwakely.gcc@gmail.com>; Xi Ruoyao <xry111@xry111.site>; Jakub Jelinek <jakub@redhat.com>; libc-alpha@sourceware.org; gcc@gcc.gnu.org; Paul Eggert <eggert@cs.ucla.edu>; linux-man@vger.kernel.org; LIU Hao <lh_mouse@126.com>; Richard Earnshaw <Richard.Earnshaw@arm.com>; Sam James <sam@gentoo.org>
> Subject: [WG14] Request for document number; strtol restrictness
> 
> Hi,
> 
> I have a paper for removing restrict from the first parameter of
> strtol(3) et al.  The title is
> 
> 	strtol(3) et al. should’t have a restricted first parameter.
> 
> If it helps, I already have a draft of the paper, which I attach (both the PDF, and the man(7) source).
> 
> Cheers,
> Alex
> 
> --
> <https://www.alejandro-colomar.es/>

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

RE: [WG14] Request for document number; strtol restrictness

[Daniel Plakosh]

Alejandro,

Sure please remind me when you submit

Best regards,

Dan

Technical Director - Enabling Mission Capability at Scale
Principal Member of the Technical Staff
Software Engineering Institute
Carnegie Mellon University
4500 Fifth Avenue
Pittsburgh, PA 15213
WORK: 412-268-7197
CELL: 412-427-4606

-----Original Message-----
From: Alejandro Colomar <alx@kernel.org> 
Sent: Tuesday, July 09, 2024 3:00 PM
To: Daniel Plakosh <dplakosh@sei.cmu.edu>
Cc: dplakosh@cert.org; Martin Uecker <muecker@gwdg.de>; Jonathan Wakely <jwakely.gcc@gmail.com>; Xi Ruoyao <xry111@xry111.site>; Jakub Jelinek <jakub@redhat.com>; libc-alpha@sourceware.org; gcc@gcc.gnu.org; Paul Eggert <eggert@cs.ucla.edu>; linux-man@vger.kernel.org; LIU Hao <lh_mouse@126.com>; Richard Earnshaw <Richard.Earnshaw@arm.com>; Sam James <sam@gentoo.org>
Subject: Re: [WG14] Request for document number; strtol restrictness

Hi Daniel,

On Sun, Jul 07, 2024 at 03:46:48PM GMT, Daniel Plakosh wrote:
> Alex,
> 
> Your document number is below:
> 
> n3294 - strtol(3) et al. shouldn't have a restricted first parameter
> 
> Please return the updated document with this number

Am I allowed to retitle the paper?

  n3294 - [[noalias()]] function attribute as a replacement of restrict

Sorry for any inconveniences.

Thanks,
Alex

> 
> Best regards,
> 
> Dan
> 
> Technical Director - Enabling Mission Capability at Scale Principal 
> Member of the Technical Staff Software Engineering Institute Carnegie 
> Mellon University
> 4500 Fifth Avenue
> Pittsburgh, PA 15213
> WORK: 412-268-7197
> CELL: 412-427-4606
> 
> -----Original Message-----
> From: Alejandro Colomar <alx@kernel.org>
> Sent: Friday, July 5, 2024 3:42 PM
> To: dplakosh@cert.org
> Cc: Martin Uecker <muecker@gwdg.de>; Jonathan Wakely 
> <jwakely.gcc@gmail.com>; Xi Ruoyao <xry111@xry111.site>; Jakub Jelinek 
> <jakub@redhat.com>; libc-alpha@sourceware.org; gcc@gcc.gnu.org; Paul 
> Eggert <eggert@cs.ucla.edu>; linux-man@vger.kernel.org; LIU Hao 
> <lh_mouse@126.com>; Richard Earnshaw <Richard.Earnshaw@arm.com>; Sam 
> James <sam@gentoo.org>
> Subject: [WG14] Request for document number; strtol restrictness
> 
> Hi,
> 
> I have a paper for removing restrict from the first parameter of
> strtol(3) et al.  The title is
> 
> 	strtol(3) et al. should’t have a restricted first parameter.
> 
> If it helps, I already have a draft of the paper, which I attach (both the PDF, and the man(7) source).
> 
> Cheers,
> Alex
> 
> --
> <https://www.alejandro-colomar.es/>

--
<https://www.alejandro-colomar.es/>

WG14 paper for removing restrict from nptr in strtol(3)

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 236 bytes --]

Hi,

I've incorporated feedback, and here's a new revision, let's call it
v0.2, of the draft for a WG14 paper.  I've attached the man(7) source,
and the generated PDF.

Cheers,
Alex


-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #1.2: strtol.man --]
[-- Type: application/x-troff-man, Size: 19199 bytes --]

[-- Attachment #1.3: strtol.pdf --]
[-- Type: application/pdf, Size: 16964 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Martin Uecker]

Hi Alejandro,

if in caller it is known that endptr has access mode "write_only"
then it can conclude that the content of *endptr has access mode
"none", couldn't it?

You also need to discuss backwards compatibility.  Changing
the type of those functions can break valid programs.  You would
need to make a case that this is unlikely to affect any real
world program.

Martin

Am Sonntag, dem 07.07.2024 um 03:58 +0200 schrieb Alejandro Colomar:
> Hi,
> 
> I've incorporated feedback, and here's a new revision, let's call it
> v0.2, of the draft for a WG14 paper.  I've attached the man(7) source,
> and the generated PDF.
> 
> Cheers,
> Alex
> 
> 

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1912 bytes --]

Hi Martin,

On Sun, Jul 07, 2024 at 09:15:23AM GMT, Martin Uecker wrote:
> 
> Hi Alejandro,
> 
> if in caller it is known that endptr has access mode "write_only"
> then it can conclude that the content of *endptr has access mode
> "none", couldn't it?

Hmmmm.  I think you're correct.  I'll incorporate that and see how it
affects the caller.

At first glance, I think it would result in

	nptr	access(read_only)	alias *endptr
	endptr	access(write_only)	unique
	errno	access(read_write)	unique
	*endptr	access(none)		alias nptr

Which is actually having perfect information, regardless of 'restrict'
on nptr.  :-)

> You also need to discuss backwards compatibility.  Changing
> the type of those functions can break valid programs.

I might be forgetting about other possibilities, but the only one I had
in mind that could break API would be function pointers.  However, a
small experiment seems to say it doesn't:

	$ cat strtolp.c 
	#include <stdlib.h>

	long
	alx_strtol(const char *nptr, char **restrict endptr, int base)
	{
		return strtol(nptr, endptr, base);
	}

	typedef long (*strtolp_t)(const char *restrict nptr,
				  char **restrict endptr, int base);
	typedef long (*strtolpnr_t)(const char *nptr,
				   char **restrict endptr, int base);

	int
	main(void)
	{
		[[maybe_unused]] strtolp_t    a = &strtol;
		[[maybe_unused]] strtolpnr_t  b = &strtol;
		[[maybe_unused]] strtolp_t    c = &alx_strtol;
		[[maybe_unused]] strtolpnr_t  d = &alx_strtol;
	}

	$ cc -Wall -Wextra strtolp.c 
	$

Anyway, I'll say that it doesn't seem to break API.

>  You would
> need to make a case that this is unlikely to affect any real
> world program.

If you have something else in mind that could break API, please let me
know, and I'll add it to the experiments.

Thanks!

Have a lovely day!
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Martin Uecker]

Am Sonntag, dem 07.07.2024 um 13:07 +0200 schrieb Alejandro Colomar via Gcc:
> Hi Martin,
> 
> On Sun, Jul 07, 2024 at 09:15:23AM GMT, Martin Uecker wrote:
> > 
> > Hi Alejandro,
> > 
> > if in caller it is known that endptr has access mode "write_only"
> > then it can conclude that the content of *endptr has access mode
> > "none", couldn't it?
> 
> Hmmmm.  I think you're correct.  I'll incorporate that and see how it
> affects the caller.
> 
> At first glance, I think it would result in
> 
> 	nptr	access(read_only)	alias *endptr
> 	endptr	access(write_only)	unique
> 	errno	access(read_write)	unique
> 	*endptr	access(none)		alias nptr
> 
> Which is actually having perfect information, regardless of 'restrict'
> on nptr.  :-)

Yes, but my point is that even with "restrict" a smarter
compiler could then also be smart enough not to warn even
when *endptr aliases nptr.

> 
> > You also need to discuss backwards compatibility.  Changing
> > the type of those functions can break valid programs.
> 
> I might be forgetting about other possibilities, but the only one I had
> in mind that could break API would be function pointers.  However, a
> small experiment seems to say it doesn't:

Right, the outermost qualifiers are ignored, so this is not a
compatibility problem.  So I think this is not an issue, but
it is worth pointing it out.

Martin

> 
> 	$ cat strtolp.c 
> 	#include <stdlib.h>
> 
> 	long
> 	alx_strtol(const char *nptr, char **restrict endptr, int base)
> 	{
> 		return strtol(nptr, endptr, base);
> 	}
> 
> 	typedef long (*strtolp_t)(const char *restrict nptr,
> 				  char **restrict endptr, int base);
> 	typedef long (*strtolpnr_t)(const char *nptr,
> 				   char **restrict endptr, int base);
> 
> 	int
> 	main(void)
> 	{
> 		[[maybe_unused]] strtolp_t    a = &strtol;
> 		[[maybe_unused]] strtolpnr_t  b = &strtol;
> 		[[maybe_unused]] strtolp_t    c = &alx_strtol;
> 		[[maybe_unused]] strtolpnr_t  d = &alx_strtol;
> 	}
> 
> 	$ cc -Wall -Wextra strtolp.c 
> 	$
> 
> Anyway, I'll say that it doesn't seem to break API.
> 
> >  You would
> > need to make a case that this is unlikely to affect any real
> > world program.
> 
> If you have something else in mind that could break API, please let me
> know, and I'll add it to the experiments.
> 
> Thanks!
> 
> Have a lovely day!
> Alex
> 

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1643 bytes --]

Hi Martin,

On Sun, Jul 07, 2024 at 02:21:17PM GMT, Martin Uecker wrote:
> Am Sonntag, dem 07.07.2024 um 13:07 +0200 schrieb Alejandro Colomar via Gcc:
> > Which is actually having perfect information, regardless of 'restrict'
> > on nptr.  :-)
> 
> Yes, but my point is that even with "restrict" a smarter
> compiler could then also be smart enough not to warn even
> when *endptr aliases nptr.

Hmmm, this is a valid argument.  I feel less strongly about this
proposal now.

I'll document this in the proposal.

Your analyzer would need to be more complex to be able to not trigger
false positives here, but it's possible, so I guess I'm happy with
either case.

Still, removing restrict from strtol(3) would allow to change the
semantics of restrict to be more restrictive (and easier to understand),
so that passing aliasing pointers as restrict pointers would already be
Undefined Behavior, regardless of the accesses by the callee.

But yeah, either way it's good, as far as strtol(3) and gcc-20 are
concerned.  :)

Have a lovely day!
Alex

> > > You also need to discuss backwards compatibility.  Changing
> > > the type of those functions can break valid programs.
> > 
> > I might be forgetting about other possibilities, but the only one I had
> > in mind that could break API would be function pointers.  However, a
> > small experiment seems to say it doesn't:
> 
> Right, the outermost qualifiers are ignored, so this is not a
> compatibility problem.  So I think this is not an issue, but
> it is worth pointing it out.

Yup.

> 
> Martin

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Paul Eggert]

On 7/7/24 03:58, Alejandro Colomar wrote:

> I've incorporated feedback, and here's a new revision, let's call it
> v0.2, of the draft for a WG14 paper.
Although I have not followed the email discussion closely, I read v0.2 
and think that as stated there is little chance that its proposal will 
be accepted.

Fundamentally the proposal is trying to say that there are two styles X 
and Y for declaring strtol and similar functions, and that although both 
styles are correct in some sense, style Y is better than style X. 
However, the advantages of Y are not clearly stated and the advantages 
of style X over Y are not admitted, so the proposal is not making the 
case clearly and fairly.

One other thing: to maximize the chance of a proposal being accepted, 
please tailor it for its expected readership. The C committee is expert 
on ‘restrict’, so don’t try to define ‘restrict’ in your own way. Unless 
merely repeating the language of the standard, any definition given for 
‘restrict’ is likely to cause the committee to quibble with the 
restatement of the standard wording. (It is OK to mention some 
corollaries of the standard definition, so long as the corollaries are 
not immediately obvious.)

Here are some comments about the proposal. At the start these comments 
are detailed; towards the end, as I could see the direction the proposal 
was headed and was convinced it wouldn’t be accepted as stated, the 
comments are less detailed.


"The API may copy"

One normally doesn’t think of the application programming interface as 
copying. Please replace the phrase “the API” with “the caller” or “the 
callee” as appropriate. (Although ‘restrict’ can be used in places other 
than function parameters, I don’t think the proposal is concerned about 
those cases and so it doesn’t need to go into that.)


"To avoid violations of for example C11::6.5.16.1p3,"

Code that violates C11::6.5.16.1p3 will do so regardless of whether 
‘restrict’ is present. I would not mention C11::6.5.16.1p3 as it’s a red 
herring. Fundamentally, ‘restrict’ is not about the consequences of 
caching when one does overlapping moves; it’s about caching in a more 
general sense.


“As long as an object is only accessed via one restricted pointer, other 
restricted pointers are allowed to point to the same object.”

“only accessed” → “accessed only”


“This is less strict than I think it should be, but this proposal 
doesn’t attempt to change that definition.”

I would omit this sentence and all similar sentences. Don’t distract the 
reader with other potential proposals. The proposal as it stands is 
complicated enough.


“return ca > a;”
“return ca > *ap;”

I fail to understand why these examples are present. It’s not simply 
that nobody writes code like that: the examples are not on point. I 
would remove the entire programs containing them, along with the 
sections that discuss them. When writing to the C committee one can 
assume the reader is expert in ‘restrict’, there is no need for examples 
such as these.


“strtol(3) accepts 4 objects via pointer parameters and global variables.”

Omit the “(3)”, here and elsewhere, as the audience is the C standard 
committee.

“accepts” is a strange word to use here: normally one says “accepts” to 
talk about parameters, not global variables. Also, “global variables” is 
not right here. The C standard allows strtol, for example, to read and 
write an internal static cache. (Yes, that would be weird, but it’s 
allowed.) I suggest rephrasing this sentence to talk about accessing, 
not accepting.


“endptr	access(write_only) ... *endptr access(none)”

This is true for glibc, but it’s not necessarily true for all conforming 
strtol implementations. If endptr is non-null, a conforming strtol 
implementation can both read and write *endptr; it can also both read 
and write **endptr. (Although it would need to write before reading, 
reading is allowed.)


“This qualifier helps catch obvious bugs such as strtol(p, p, 0) and 
strtol(&p, &p, 0) .”

No it doesn’t. Ordinary type checking catches those obvious bugs, and 
‘restrict’ provides no additional help there. Please complicate the 
examples to make the point more clearly.


“The caller knows that errno doesn’t alias any of the function arguments.”

Only because all args are declared with ‘restrict’. So if the proposal 
is accepted, the caller doesn’t necessarily know that.


“The callee knows that *endptr is not accessed.”

This is true for glibc, but not necessarily true for every conforming 
strtol implementation.


“It might seem that it’s a problem that the callee doesn’t know if nptr 
can alias errno or not. However, the callee will not write to the latter 
directly until it knows it has failed,”

Again this is true for glibc, but not necessarily true for every 
conforming strtol implementation.

To my mind this is the most serious objection. The current standard 
prohibits calls like strtol((char *) &errno, 0, 0). The proposal would 
relax the standard to allow such calls. In other words, the proposal 
would constrain implementations to support such calls. Why is this 
change worth making? Real-world programs do not make calls like that.


“But nothing prohibits those internal helper functions to specify that 
nptr is restrict and thus distinct from errno.”

Although true, it’s also the case that the C standard does not *require* 
internal helper functions to use ‘restrict’. All that matters is the 
accesses. So I’m not sure what the point of this statement is.


“m = strtol(p, &p, 0); An analyzer more powerful than the current ones
could extend the current -Wrestrict diagnostic to also diagnose this case.”

Why would an analyzer want to do that? This case is a perfectly normal 
thing to do and it has well-defined behavior.


“To prevent triggering diagnostics in a powerful analyzer that would be 
smart enough to diagnose the example function g(), the prototype of 
strtol(3) should be changed to ‘long int strtol(const char *nptr, char 
**restrict endptr, int base);’”

Sorry, but the case has not been made to make any such change to 
strtol’s prototype. On the contrary, what I’m mostly gathering from the 
discussion is that ‘restrict’ can be confusing, which is not news.

n3220 §6.7.4.2 examples 5 through 7 demonstrate that the C committee has 
thought through the points you’re making. (These examples were not 
present in C11.) This may help to explain why the standard specifies 
strtol with ‘restrict’ on both arguments.

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 13035 bytes --]

Hi Paul,

On Sun, Jul 07, 2024 at 12:42:51PM GMT, Paul Eggert wrote:
> On 7/7/24 03:58, Alejandro Colomar wrote:
> 
> > I've incorporated feedback, and here's a new revision, let's call it
> > v0.2, of the draft for a WG14 paper.
> Although I have not followed the email discussion closely, I read v0.2 and
> think that as stated there is little chance that its proposal will be
> accepted.

Thanks for reading thoroughly, and the feedback!

> Fundamentally the proposal is trying to say that there are two styles X and
> Y for declaring strtol and similar functions, and that although both styles
> are correct in some sense, style Y is better than style X. However, the
> advantages of Y are not clearly stated and the advantages of style X over Y
> are not admitted, so the proposal is not making the case clearly and fairly.
> 
> One other thing: to maximize the chance of a proposal being accepted, please
> tailor it for its expected readership. The C committee is expert on
> ‘restrict’, so don’t try to define ‘restrict’ in your own way. Unless merely
> repeating the language of the standard, any definition given for ‘restrict’
> is likely to cause the committee to quibble with the restatement of the
> standard wording. (It is OK to mention some corollaries of the standard
> definition, so long as the corollaries are not immediately obvious.)
> 
> Here are some comments about the proposal. At the start these comments are
> detailed; towards the end, as I could see the direction the proposal was
> headed and was convinced it wouldn’t be accepted as stated, the comments are
> less detailed.
> 
> 
> "The API may copy"
> 
> One normally doesn’t think of the application programming interface as
> copying. Please replace the phrase “the API” with “the caller” or “the
> callee” as appropriate. (Although ‘restrict’ can be used in places other
> than function parameters, I don’t think the proposal is concerned about
> those cases and so it doesn’t need to go into that.)

Ok.

> "To avoid violations of for example C11::6.5.16.1p3,"
> 
> Code that violates C11::6.5.16.1p3 will do so regardless of whether
> ‘restrict’ is present. I would not mention C11::6.5.16.1p3 as it’s a red
> herring. Fundamentally, ‘restrict’ is not about the consequences of caching
> when one does overlapping moves; it’s about caching in a more general sense.

The violations are UB regardless of restrict, but consistent use of
restrict allows the caller to have a rough model of what the callee will
do with the objects, and prevent those violations via compiler
diagnostics.  I've reworded that part to make it more clear why I'm
mentioning that.

> “As long as an object is only accessed via one restricted pointer, other
> restricted pointers are allowed to point to the same object.”
> 
> “only accessed” → “accessed only”

Ok.

> “This is less strict than I think it should be, but this proposal doesn’t
> attempt to change that definition.”
> 
> I would omit this sentence and all similar sentences. Don’t distract the
> reader with other potential proposals. The proposal as it stands is
> complicated enough.

Ok.

> “return ca > a;”
> “return ca > *ap;”
> 
> I fail to understand why these examples are present. It’s not simply that
> nobody writes code like that: the examples are not on point. I would remove
> the entire programs containing them, along with the sections that discuss
> them. When writing to the C committee one can assume the reader is expert in
> ‘restrict’, there is no need for examples such as these.

Those are examples of how consistent use of restrict can --or could, in
the case of g()-- detect, via compiler diagnostics, (likely) violations
of seemingly unrelated parts of the standard, such as the referenced
C11::6.5.16.1p3, or in this case, C11::6.5.8p5.  

> “strtol(3) accepts 4 objects via pointer parameters and global variables.”
> 
> Omit the “(3)”, here and elsewhere, as the audience is the C standard
> committee.

The C standard committee doesn't know about historic use of (3)?  That
predates the standard, and they built on top of that (C originated in
Unix).  While they probably don't care about it anymore, I expect my
paper to be read by other audience, including GCC and glibc, and I
prefer to keep it readable for that audience.  I expect the standard
committee to at least have a rough idea of the existence of this syntax,
and respect it, even if they don't use it or like it.

> “accepts” is a strange word to use here: normally one says “accepts” to talk
> about parameters, not global variables.

The thing is, strtol(3) does not actually access *endptr.  I thought
that might cause more confusion than using "accepts".

> Also, “global variables” is not
> right here. The C standard allows strtol, for example, to read and write an
> internal static cache. (Yes, that would be weird, but it’s allowed.)

That's not part of the API.  A user must not access internal static
cache, and so the implementation is free to assume that it doesn't,
regardless of the use of restrict in the API, so it is not relevant for
the purpose of this discussion, I think.

> I
> suggest rephrasing this sentence to talk about accessing, not accepting.

I don't want to use accessing, for it would be inconsistent with later
saying that *endptr is not accessed.  However, I'm open to other
suggested terms that might be more appropriate than both.

> “endptr	access(write_only) ... *endptr access(none)”
> 
> This is true for glibc, but it’s not necessarily true for all conforming
> strtol implementations. If endptr is non-null, a conforming strtol
> implementation can both read and write *endptr;

It can't, I think.  It's perfectly valid to pass an uninitialized
endptr, which means the callee must not read the original value.

	char *end;
	strtol("0", &end, 0);

If strtol(3) would be allowed to read it, the user would need to
initialize it.

> it can also both read and
> write **endptr. (Although it would need to write before reading, reading is
> allowed.)

Here, we need to consider two separate objects.  The object pointed-to
by *endptr _before_ the object pointed to by endptr is written to, and
the object pointed-to by *endptr _after_ the object pointed to by endptr
is written to.

For the former (the original *endptr):

	Since *endptr might be uninitialized, strtol(3) must NOT access
	the object pointed to by an uninitialized pointer.

For the latter (the final *endptr):

	The callee cannot write to it, since the specification of the
	function is that the string will not be modified.  And in any
	case, such an access is ultimately derived from nptr, not from
	*endptr, so it does not matter for the discussion of *endptr.

Of course, that's derived from the specification of the function, and
not from its prototype, since ISO C doesn't provide such detailed
prototypes (since it doesn't have the [[gnu::access()]] attribute).  But
the standard must abide by its own specification of functions, anyway.

> “This qualifier helps catch obvious bugs such as strtol(p, p, 0) and
> strtol(&p, &p, 0) .”
> 
> No it doesn’t. Ordinary type checking catches those obvious bugs, and
> ‘restrict’ provides no additional help there. Please complicate the examples
> to make the point more clearly.

To be pedantic, I didn't specify the type of p, so it might be (void *),
and thus avoid type checking at all.  However, to avoid second guessing
from the standards committee, I'll add casts, to make it more obvious
that restrict is catching those.

> “The caller knows that errno doesn’t alias any of the function arguments.”
> 
> Only because all args are declared with ‘restrict’. So if the proposal is
> accepted, the caller doesn’t necessarily know that.

Not really.  The caller has created the string (or has received it via a
restricted pointer), and so it knows it's not derived from errno.

	char buf[LINE_MAX + 1];

	fgets(...);
	n = strtol(buf, ...);

This caller knows with certainty that errno does not alias buf.  Of
course, in some complex cases, it might not know, but I ommitted that
for simplicity.  And in any case, I don't think any optimizations are
affected by that in the caller.

> 
> 
> “The callee knows that *endptr is not accessed.”
> 
> This is true for glibc, but not necessarily true for every conforming strtol
> implementation.

The original *endptr may be uninitialized, and so must not be accessed.

> “It might seem that it’s a problem that the callee doesn’t know if nptr can
> alias errno or not. However, the callee will not write to the latter
> directly until it knows it has failed,”
> 
> Again this is true for glibc, but not necessarily true for every conforming
> strtol implementation.

An implementation is free to set errno = EDEADLK in the middle of it, as
long as it later removes that.  However, I don't see how it would make
any sense.

If that's done, it's probably done via a helper internal function, which
as said below, can use restrict for nptr, and thus know with certainty
that nptr is distinct from errno.

If that's done directly in the body of strtol(3) (the only place where
it's not known that nptr is distinct from errno) we can probably agree
that the implementation is doing that just for fun, and doesn't care
about optimization, and thus we can safely ignore it.

> To my mind this is the most serious objection. The current standard
> prohibits calls like strtol((char *) &errno, 0, 0). The proposal would relax
> the standard to allow such calls. In other words, the proposal would
> constrain implementations to support such calls.

I don't think it does.  ISO C specifies that strtol(3) takes a string as
its first parameter, and errno is not (unless you do this:).

	(char *)&errno = "111";

Okay, let's assume you're allowed to do that, since a char* can alias
anything.

I still don't think ISO C constrains implementations to allow passing
(char *)&errno as a char*, just because it's not restrict.  Let's find
an ISO C function that accepts a non-restrict string:

	int system(const char *string);

Does ISO C constrain implementations to support system((char *)&errno)?
I don't think so.  Maybe it does implicitly because of a defect in the
wording, but even then it's widely understood that it doesn't.

> Why is this change worth
> making? Real-world programs do not make calls like that.

Because it makes analysis of 'restrict' more consistent.  The obvious
improvement of GCC's analyzer to catch restrict violations will trigger
false positives in normal uses of strtol(3).

> “But nothing prohibits those internal helper functions to specify that nptr
> is restrict and thus distinct from errno.”
> 
> Although true, it’s also the case that the C standard does not *require*
> internal helper functions to use ‘restrict’. All that matters is the
> accesses. So I’m not sure what the point of this statement is.

If an implementation wants to optimize, it should be careful and use
restrict.  If it doesn't, then it can't complain that ISO C doesn't
allow it to.  It's actually allowed to optimize, but it has to do some
work for it.

> “m = strtol(p, &p, 0); An analyzer more powerful than the current ones
> could extend the current -Wrestrict diagnostic to also diagnose this case.”
> 
> Why would an analyzer want to do that? This case is a perfectly normal thing
> to do and it has well-defined behavior.

Because without an analyzer, restrict cannot emit many useful
diagnostics.  It's a qualifier that's all about data flow analysis, and
normal diagnostics aren't able to do that.

A qualifier that enables optimizations but doesn't enable diagnostics is
quite dangerous, and probably better not used.  If however, the analyzer
emits advanced diagnostics for misuses of it, then it's a good
qualifier.

Have a lovely day!
Alex

> 
> “To prevent triggering diagnostics in a powerful analyzer that would be
> smart enough to diagnose the example function g(), the prototype of
> strtol(3) should be changed to ‘long int strtol(const char *nptr, char
> **restrict endptr, int base);’”
> 
> Sorry, but the case has not been made to make any such change to strtol’s
> prototype. On the contrary, what I’m mostly gathering from the discussion is
> that ‘restrict’ can be confusing, which is not news.
> 
> n3220 §6.7.4.2 examples 5 through 7 demonstrate that the C committee has
> thought through the points you’re making. (These examples were not present
> in C11.) This may help to explain why the standard specifies strtol with
> ‘restrict’ on both arguments.
> 

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Paul Eggert]

On 7/7/24 14:42, Alejandro Colomar wrote:
> On Sun, Jul 07, 2024 at 12:42:51PM GMT, Paul Eggert wrote:
>> Also, “global variables” is not
>> right here. The C standard allows strtol, for example, to read and write an
>> internal static cache. (Yes, that would be weird, but it’s allowed.)
> 
> That's not part of the API.  A user must not access internal static
> cache

Although true in the normal (sane) case, as an extension the 
implementation can make such a static cache visible to the user, and in 
this case the caller must not pass cache addresses as arguments to strtol.

For other functions this point is not purely academic. For example, the 
C standard specifies the signature "FILE *fopen(const char *restrict, 
const char *restrict);". If I understand your argument correctly, it 
says that the "restrict"s can be omitted there without changing the set 
of valid programs. But that can't be right, as omitting the "restrict"s 
would make the following code be valid in any platform where sizeof(int)>1:

    char *p = (char *) &errno;
    p[0] = 'r';
    p[1] = 0;
    FILE *f = fopen (p, p);

even though the current standard says this code is invalid.


>> “endptr	access(write_only) ... *endptr access(none)”
>>
>> This is true for glibc, but it’s not necessarily true for all conforming
>> strtol implementations. If endptr is non-null, a conforming strtol
>> implementation can both read and write *endptr;
> 
> It can't, I think.  It's perfectly valid to pass an uninitialized
> endptr, which means the callee must not read the original value.

Sure, but the callee can do something silly like "*endptr = p + 1; 
*endptr = *endptr - 1;". That is, it can read *endptr after writing it, 
without any undefined behavior. (And if the callee is written in 
assembly language it can read *endptr even before writing it - but I 
digress.)

The point is that it is not correct to say that *endptr cannot be read 
from; it can. Similarly for **endptr.


> Here, we need to consider two separate objects.  The object pointed-to
> by *endptr _before_ the object pointed to by endptr is written to, and
> the object pointed-to by *endptr _after_ the object pointed to by endptr
> is written to.

Those are not the only possibilities. The C standard also permits strtol 
to set *endptr to some other pointer value, not pointing anywhere into 
the string being scanned, so long as it sets *endptr correctly before it 
returns.


>> “The caller knows that errno doesn’t alias any of the function arguments.”
>>
>> Only because all args are declared with ‘restrict’. So if the proposal is
>> accepted, the caller doesn’t necessarily know that.
> 
> Not really.  The caller has created the string (or has received it via a
> restricted pointer)

v0.2 doesn't state the assumption that the caller either created the 
string or received it via a restricted pointer. If this assumption were 
stated clearly, that would address the objection here.


>> “The callee knows that *endptr is not accessed.”
>>
>> This is true for glibc, but not necessarily true for every conforming strtol
>> implementation.
> 
> The original *endptr may be uninitialized, and so must not be accessed.

**endptr can be read once the callee sets *endptr. **endptr can even be 
written, if the callee temporarily sets *endptr to point to a writable 
buffer; admittedly this would be weird but it's allowed.


>> “It might seem that it’s a problem that the callee doesn’t know if nptr can
>> alias errno or not. However, the callee will not write to the latter
>> directly until it knows it has failed,”
>>
>> Again this is true for glibc, but not necessarily true for every conforming
>> strtol implementation.
> 
> An implementation is free to set errno = EDEADLK in the middle of it, as
> long as it later removes that.  However, I don't see how it would make
> any sense.

It could make sense in some cases. Here the spec is a bit tricky, but an 
implementation is allowed to set errno = EINVAL first thing, and then 
set errno to some other nonzero value if it determines that the 
arguments are valid. I wouldn't implement strtol that way, but I can see 
where someone else might do that.


> Let's find
> an ISO C function that accepts a non-restrict string:
> 
> 	int system(const char *string);
> 
> Does ISO C constrain implementations to support system((char *)&errno)?
> I don't think so.  Maybe it does implicitly because of a defect in the
> wording, but even then it's widely understood that it doesn't.

'system' is a special case since the C standard says 'system' can do 
pretty much anything it likes. That being said, I agree that 
implementations shouldn't need to support calls like atol((char *) 
&errno). Certainly the C standard's description of atol, which defines 
atol's behavior in terms of a call to strtol, means that atol's argument 
in practice must follow the 'restrict' rules.

Perhaps we should report this sort of thing as a defect in the standard. 
It is odd, for example, that fopen's two arguments are both const char 
*restrict, but system's argument lacks the "restrict".


>> Why is this change worth
>> making? Real-world programs do not make calls like that.
> 
> Because it makes analysis of 'restrict' more consistent.  The obvious
> improvement of GCC's analyzer to catch restrict violations will trigger
> false positives in normal uses of strtol(3).

v0.2 does not support this line of reasoning. On the contrary, v0.2 
suggests that a compiler should diagnose calls like "strtol(p, &p, 0)", 
which would be wrong as that call is perfectly reasonable.

Another way to put it: v0.2 does not clearly state the advantages of the 
proposed change, and in at least one area what it states as an advantage 
would actually be a disadvantage.


>> “m = strtol(p, &p, 0); An analyzer more powerful than the current ones
>> could extend the current -Wrestrict diagnostic to also diagnose this case.”
>>
>> Why would an analyzer want to do that? This case is a perfectly normal thing
>> to do and it has well-defined behavior.
> 
> Because without an analyzer, restrict cannot emit many useful
> diagnostics.  It's a qualifier that's all about data flow analysis, and
> normal diagnostics aren't able to do that.
> 
> A qualifier that enables optimizations but doesn't enable diagnostics is
> quite dangerous, and probably better not used.  If however, the analyzer
> emits advanced diagnostics for misuses of it, then it's a good
> qualifier.

Sorry, but I don't understand what you're trying to say here. Really, I 
can't make heads or tails of it. As-is, 'restrict' can be useful both 
for optimization and for generating diagnostics, and GCC does both of 
these things right now even if you don't use -fanalyzer.

Perhaps adding an example or two would help explain your point. But 
they'd need to be better examples than what's in v0.2 because v0.2 is 
unclear about this quality-of-diagnostics issue, as it relates to strtol.

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 13354 bytes --]

Hi Paul,

On Sun, Jul 07, 2024 at 07:30:43PM GMT, Paul Eggert wrote:
> On 7/7/24 14:42, Alejandro Colomar wrote:
> > On Sun, Jul 07, 2024 at 12:42:51PM GMT, Paul Eggert wrote:
> > > Also, “global variables” is not
> > > right here. The C standard allows strtol, for example, to read and write an
> > > internal static cache. (Yes, that would be weird, but it’s allowed.)
> > 
> > That's not part of the API.  A user must not access internal static
> > cache
> 
> Although true in the normal (sane) case, as an extension the implementation
> can make such a static cache visible to the user, and in this case the
> caller must not pass cache addresses as arguments to strtol.
> 
> For other functions this point is not purely academic. For example, the C
> standard specifies the signature "FILE *fopen(const char *restrict, const
> char *restrict);". If I understand your argument correctly, it says that the
> "restrict"s can be omitted there without changing the set of valid programs.

No, I didn't say that restrict can be removed from fopen(3).

What I say is that in functions that accept pointers that alias each
other, those aliasing pointers should not be restrict.  Usually,
pointers that alias are accessed, and thus they are not specified as
restrict, such as in memmove(3).  However, a small set of functions
accept pointers that alias each other, but one of them is never
accessed; in those few cases, restrict was added to the parameters in
ISO C, but I claim it would be better removed.  We're lucky, and the
small set of functions where this happens don't seem to use any state,
so we don't need to care about implementations using internal buffers
that are passed somehow to the user..

From ISO C, IIRC, the only examples are strtol(3) et al.  Another
example is Plan9's seprint(3) family of functions.  However, Plan9
doesn't use restrict, so it doesn't have it.

> But that can't be right, as omitting the "restrict"s would make the
> following code be valid in any platform where sizeof(int)>1:
> 
>    char *p = (char *) &errno;
>    p[0] = 'r';
>    p[1] = 0;
>    FILE *f = fopen (p, p);
> 
> even though the current standard says this code is invalid.

No, I wouldn't remove any of the restrict qualifiers in fopen(3).

Only from pointers that alias an access(none) pointer.

> > > “endptr	access(write_only) ... *endptr access(none)”
> > > 
> > > This is true for glibc, but it’s not necessarily true for all conforming
> > > strtol implementations. If endptr is non-null, a conforming strtol
> > > implementation can both read and write *endptr;
> > 
> > It can't, I think.  It's perfectly valid to pass an uninitialized
> > endptr, which means the callee must not read the original value.
> 
> Sure, but the callee can do something silly like "*endptr = p + 1; *endptr =
> *endptr - 1;". That is, it can read *endptr after writing it, without any
> undefined behavior. (And if the callee is written in assembly language it
> can read *endptr even before writing it - but I digress.)

But once you modify the pointer provenance, you don't care anymore about
it.  We need to consider the pointers that a function receives, which
are the ones the callee needs to know their provenance.  Of course, a
callee knows what it does, and so doesn't need restrict in local
variables.

C23/N3220::6.7.4.1p9 says:

	An object that is accessed through a restrict-qualified pointer
	has a special association with that pointer.  This association,
	defined in 6.7.4.2, requires that all accesses to that object
	use, directly or indirectly, the value of that pointer.

When you set *endptr = nptr + x, and use the lvalue **endptr, you're
still accessing the object indirectly using the value of nptr.

So, strtol(3) gets 4 objects, let's call them A, B, C, and D.

A is gotten via its pointer nptr.
B is gotten via its pointer endptr.
C is gotten via its pointer *endptr.
D is gotten via the global variable errno.

Object A may be the same as object C.
Object B is unique inside the callee.  Its pointer endptr must be
restrict-qualified to denote its uniqueness.
Object D is unique, but there's no way to specify that.

Object C must NOT be read or written.  The function is of course allowed
to set *endptr to whatever it likes, and then access it however it
likes, but object C must still NOT be accessed, since its pointer may be
uninitialized, and thus point to no object at all.


Maybe I should use abstract names for the objects, to avoid confusing
them with the pointer variables that are used to pass them?

The formal definition of restrict refers to the "object into which it
formerly [in the list of parameter declarations of a function
definition] pointed".  I'm not 100% certain, because this formal
definition is quite unreadable, though.  The more I read it, the less
sure I am about it.

BTW, I noticed something I didn't know:

	If L is used to access the value of the object X that it
	designates, and X is also modified (by any means), then the
	following requirements apply: T shall not be const-qualified

This reads to me as "const variables are not writable when they are
accessed via a restricted pointer; casting away is not enough".  Am I
reading this correctly?

> The point is that it is not correct to say that *endptr cannot be read from;
> it can. Similarly for **endptr.

Some better wording: the object pointed-to by *endptr at function entry
cannot be accessed.

> > Here, we need to consider two separate objects.  The object pointed-to
> > by *endptr _before_ the object pointed to by endptr is written to, and
> > the object pointed-to by *endptr _after_ the object pointed to by endptr
> > is written to.
> 
> Those are not the only possibilities. The C standard also permits strtol to
> set *endptr to some other pointer value, not pointing anywhere into the
> string being scanned, so long as it sets *endptr correctly before it
> returns.

Let's reword.  The initial object pointed-to by it, and everything else.

> > > “The caller knows that errno doesn’t alias any of the function arguments.”
> > > 
> > > Only because all args are declared with ‘restrict’. So if the proposal is
> > > accepted, the caller doesn’t necessarily know that.
> > 
> > Not really.  The caller has created the string (or has received it via a
> > restricted pointer)
> 
> v0.2 doesn't state the assumption that the caller either created the string
> or received it via a restricted pointer. If this assumption were stated
> clearly, that would address the objection here.

Ok.

> > > “The callee knows that *endptr is not accessed.”
> > > 
> > > This is true for glibc, but not necessarily true for every conforming strtol
> > > implementation.
> > 
> > The original *endptr may be uninitialized, and so must not be accessed.
> 
> **endptr can be read once the callee sets *endptr. **endptr can even be
> written, if the callee temporarily sets *endptr to point to a writable
> buffer; admittedly this would be weird but it's allowed.

The object originally pointed-to by *endptr (C) is what we care about.
Subsequently reusing the same pointer variable for pointing to different
objects is uninteresting for the purposes of knowing which objects are
accessed and in which way.

> > > “It might seem that it’s a problem that the callee doesn’t know if nptr can
> > > alias errno or not. However, the callee will not write to the latter
> > > directly until it knows it has failed,”
> > > 
> > > Again this is true for glibc, but not necessarily true for every conforming
> > > strtol implementation.
> > 
> > An implementation is free to set errno = EDEADLK in the middle of it, as
> > long as it later removes that.  However, I don't see how it would make
> > any sense.
> 
> It could make sense in some cases. Here the spec is a bit tricky, but an
> implementation is allowed to set errno = EINVAL first thing, and then set
> errno to some other nonzero value if it determines that the arguments are
> valid. I wouldn't implement strtol that way, but I can see where someone
> else might do that.

In any case an implementation is not obliged to pessimize strtol(3).  It
is only allowed to.  Should we not allow them to do so?

> > Let's find
> > an ISO C function that accepts a non-restrict string:
> > 
> > 	int system(const char *string);
> > 
> > Does ISO C constrain implementations to support system((char *)&errno)?
> > I don't think so.  Maybe it does implicitly because of a defect in the
> > wording, but even then it's widely understood that it doesn't.
> 
> 'system' is a special case since the C standard says 'system' can do pretty
> much anything it likes. That being said, I agree that implementations
> shouldn't need to support calls like atol((char *) &errno). Certainly the C
> standard's description of atol, which defines atol's behavior in terms of a
> call to strtol, means that atol's argument in practice must follow the
> 'restrict' rules.

Let's take a simpler one: rename(2).  Is it allowed to receive &errno?
Hopefully not.

> 
> Perhaps we should report this sort of thing as a defect in the standard. It
> is odd, for example, that fopen's two arguments are both const char
> *restrict, but system's argument lacks the "restrict".
> 

Meh, I don't care enough, I think.

> > > Why is this change worth
> > > making? Real-world programs do not make calls like that.
> > 
> > Because it makes analysis of 'restrict' more consistent.  The obvious
> > improvement of GCC's analyzer to catch restrict violations will trigger
> > false positives in normal uses of strtol(3).
> 
> v0.2 does not support this line of reasoning. On the contrary, v0.2 suggests
> that a compiler should diagnose calls like "strtol(p, &p, 0)", which would
> be wrong as that call is perfectly reasonable.

That call is perfectly, reasonable, which is why I suggest that the
standard should modify the prototype so that strtol(p, &p, 0), which is
a reasonable call, should not be warned by a compiler that would
diagnose such calls.

That is, just by reading the prototypes:

	void foo(int *restrict x, int **restrict p);

and

	void bar(int *x, int **restrict endp);

one should be able to determine that

	foo(p, &p);

is probably causing UB (and thus trigger a warning) but

	bar(p, &p);

is fine.

> 
> Another way to put it: v0.2 does not clearly state the advantages of the
> proposed change, and in at least one area what it states as an advantage
> would actually be a disadvantage.

The advantage is having more information in the caller.  As a caller, I
want to distinguish calls where it's ok to pass pointers that alias, and
where not.  And I want my compiler to be able to help me there.

If restrict is overapplied, then an analyzer cannot determine that.  Or
as Martin noted, it can, if it takes both the restrict qualifiers _and_
the access attributes into account, and performs some non-trivial
deduction.  I'd rather have a simple analyzer, which will provide for
less false positives and negatives.

> > > “m = strtol(p, &p, 0); An analyzer more powerful than the current ones
> > > could extend the current -Wrestrict diagnostic to also diagnose this case.”
> > > 
> > > Why would an analyzer want to do that? This case is a perfectly normal thing
> > > to do and it has well-defined behavior.
> > 
> > Because without an analyzer, restrict cannot emit many useful
> > diagnostics.  It's a qualifier that's all about data flow analysis, and
> > normal diagnostics aren't able to do that.
> > 
> > A qualifier that enables optimizations but doesn't enable diagnostics is
> > quite dangerous, and probably better not used.  If however, the analyzer
> > emits advanced diagnostics for misuses of it, then it's a good
> > qualifier.
> 
> Sorry, but I don't understand what you're trying to say here. Really, I
> can't make heads or tails of it. As-is, 'restrict' can be useful both for
> optimization and for generating diagnostics, and GCC does both of these
> things right now even if you don't use -fanalyzer.

GCC can only catch the most obvious violations of restrict.

	#include <string.h>

	typedef struct {
		int x;
	} T;

	[[gnu::access(read_only, 1)]]
	[[gnu::access(read_only, 2)]]
	void
	replace(T *restrict *restrict ls, const T *restrict new, size_t pos)
	{
		memcpy(ls[pos], new, sizeof(T));
	}

	void
	f(T *restrict *restrict ls)
	{
		replace(ls, ls[0], 1);
	}

	$ gcc-14 -Wall -Wextra -fanalyzer replace.c -S
	$

The above program causes UB, and uses an API that is so similar to
strtol(3), that for writing an analyzer that triggers on this code, it
will trigger on strtol(3) too.  Only if it's smart enough to also
consider the GNU access attributes, it will be able to differentiate
the two cases, as Martin suggested.

> Perhaps adding an example or two would help explain your point. But they'd
> need to be better examples than what's in v0.2 because v0.2 is unclear about
> this quality-of-diagnostics issue, as it relates to strtol.

Maybe the above?

Cheers,
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Paul Eggert]

On 7/8/24 00:52, Alejandro Colomar wrote:
 > a small set of functions
 > accept pointers that alias each other, but one of them is never
 > accessed; in those few cases, restrict was added to the parameters in
 > ISO C, but I claim it would be better removed.

Are these aliasing pointers the nptr and initial *endptr of strtol? That 
is, are you saying that last line in the following example, which is 
currently invalid, should become valid and should be implementable as 
‘end = s; long l = 0;’?

    char *end;
    char *s = (char *) &end;
    *s = '\0';
    long l = strtol (s, &end, 0);

If so, I fail to see the motivation for the proposed change, as nobody 
writes (or should write) code like that. And if not, evidently I 
misunderstand the proposal.


 > the small set of functions where this happens don't seem to use any 
state,
 > so we don't need to care about implementations using internal buffers
 > that are passed somehow to the user.

For strtol (nptr, endptr, 10) evidently the state that’s of concern is 
the value of *endptr. But here it’s possible that an implementation 
could use that state, even if the source code of the implementation's 
strtol does not. For example, suppose the key part of the base 10 
implementation in strtol.c is this:

     bool overflow = false;
     long int n = 0;
     for (; '0' <= *nptr && *nptr <= '9'; nptr++)
       {
	overflow |= ckd_mul (&n, n, 10);
	overflow |= ckd_add (&n, n, *nptr - '0');
       }
     *endptr = (char *) nptr;
     ... more code goes here ...

Currently, on typical platforms where CHAR_WIDTH < INT_WIDTH and 
INT_WIDTH == UINT_WIDTH, the C standard lets the compiler to compile 
this code as if it were the following instead.

     bool overflow = false;
     long int n = 0;
     *endptr = (char *) nptr;
     unsigned int digit = *nptr++ - '0';
     if (digit <= 9)
       {
	n = digit;
	while ((digit = *nptr++ - '0') <= 9)
	  {
	    overflow |= ckd_mul (&n, n, 10);
	    overflow |= ckd_add (&n, n, digit);
	  }
	*endptr = (char *) nptr;
       }
     ... more code goes here ...

This sort of thing might make sense on some architectures. However, the 
proposed change would not allow this optimization, because it’s invalid 
when nptr points into *endptr.

For strtol I suppose this is not that big a deal; strtol is kinda slow 
anyway so who cares if it’s a bit slower? But surely we wouldn’t want to 
give up even this minor performance win unless we get something in 
return, and I’m still not seeing what we get in return.


 > Maybe I should use abstract names for the objects, to avoid confusing
 > them with the pointer variables that are used to pass them?

That might help, yes, since v0.2 is unclear on this point.


 > this formal
 > definition is quite unreadable, though.  The more I read it, the less
 > sure I am about it.

Yes, it’s lovely isn’t it? One must understand what the C committee
intended in order to read and understand that part of the standard.


 > 	If L is used to access the value of the object X that it
 > 	designates, and X is also modified (by any means), then the
 > 	following requirements apply: T shall not be const-qualified
 >
 > This reads to me as "const variables are not writable when they are
 > accessed via a restricted pointer; casting away is not enough".  Am I
 > reading this correctly?

In that quoted statement, the restricted pointer is not allowed to be 
pointer-to-const. However, I’m not quite sure what your question means, 
as the phrase “const variables” does not appear in the standard. Perhaps 
give an example to clarify the question?


 >> an implementation is allowed to set errno = EINVAL first thing, and 
then set
 >> errno to some other nonzero value if it determines that the 
arguments are
 >> valid. I wouldn't implement strtol that way, but I can see where someone
 >> else might do that.
 >
 > In any case an implementation is not obliged to pessimize strtol(3).  It
 > is only allowed to.  Should we not allow them to do so?

Of course the standard should allow suboptimal implementations. However, 
I’m not sure what the point of the question is. The “errno = EINVAL 
first thing” comment says that removing ‘restrict’ obliges the 
implementation to support obviously-bogus calls like strtol(&errno, 
...), which might make the implementation less efficient. I don’t see 
how the question is relevant to that comment.


 > Let's take a simpler one: rename(2).  Is it allowed to receive &errno?
 > Hopefully not.

I agree with that hope, but the current C standard seems to allow it. I 
think we both agree this is a defect in the standard.


 >>>> Why is this change worth
 >>>> making? Real-world programs do not make calls like that.
 >>>
 >>> Because it makes analysis of 'restrict' more consistent.  The obvious
 >>> improvement of GCC's analyzer to catch restrict violations will trigger
 >>> false positives in normal uses of strtol(3).
 >>
 >> v0.2 does not support this line of reasoning. On the contrary, v0.2 
suggests
 >> that a compiler should diagnose calls like "strtol(p, &p, 0)", which 
would
 >> be wrong as that call is perfectly reasonable.
 >
 > That call is perfectly, reasonable, which is why I suggest that the
 > standard should modify the prototype so that strtol(p, &p, 0), which is
 > a reasonable call, should not be warned by a compiler that would
 > diagnose such calls.

Of course they shouldn’t warn. But where are these compilers?

v0.2 asserts that “An analyzer more powerful than the current ones could 
extend the current -Wrestrict diagnostic to also diagnose this case.” 
But why would an analyzer want to do that? v0.2 doesn’t say.

The proposal merely asks to change prototypes for the C standard 
functions strtol, strtoul, etc. But if that is the only change needed 
then why bother? C compilers already do special-case analysis for 
functions defined by the C standard, and they can suppress undesirable 
diagnostics for these special cases.

If you’ve identified a more general problem with ‘restrict’ then welcome 
to the club! The experts already know it’s confusing and limited, and 
are discussing about whether and how to improve things in the next C 
standard. I am sure you’d be welcome to those discussions.


 > That is, just by reading the prototypes:
 >
 > 	void foo(int *restrict x, int **restrict p);
 >
 > and
 >
 > 	void bar(int *x, int **restrict endp);
 >
 > one should be able to determine that
 >
 > 	foo(p, &p);
 >
 > is probably causing UB (and thus trigger a warning) but
 >
 > 	bar(p, &p);
 >
 > is fine.

Sure, but this is a discussion we should be having with the compiler 
writers, no?

Is this the main motivation for the proposal? If so, how would weakening 
the spec for strtol etc. affect that discussion with the compiler 
writers? v0.2 does not make this clear.


 >> Another way to put it: v0.2 does not clearly state the advantages of the
 >> proposed change, and in at least one area what it states as an advantage
 >> would actually be a disadvantage.
 >
 > The advantage is having more information in the caller.  As a caller, I
 > want to distinguish calls where it's ok to pass pointers that alias, and
 > where not.  And I want my compiler to be able to help me there.

I’m still not understanding. Removing ‘restrict’ from strtol’s first arg 
gives the caller less information, not more.


 > I'd rather have a simple analyzer, which will provide for
 > less false positives and negatives.

The C committee appears to have the opposite opinion, as when they were 
asked about this matter they added Examples 5 through 7 to what is now 
§6.7.4.2 (Formal definition of restrict). These examples say that 
Example 2 (which uses ‘restrict’ on all arguments) is the simplest and 
most effective way to use ‘restrict’, even though a smarter compiler can 
still make some good inferences when some pointer args are ‘restrict’ 
and others are merely pointers to const.

If the proposal is disagreeing with Examples 5 through 7, this point 
needs to be thoroughly discussed in the proposal.


 > GCC can only catch the most obvious violations of restrict.

Yes, but I fail to see how changing the API for strtol etc. would 
improve that situation.


 > 	#include <string.h>
 >
 > 	typedef struct {
 > 		int x;
 > 	} T;
 >
 > 	[[gnu::access(read_only, 1)]]
 > 	[[gnu::access(read_only, 2)]]
 > 	void
 > 	replace(T *restrict *restrict ls, const T *restrict new, size_t pos)
 > 	{
 > 		memcpy(ls[pos], new, sizeof(T));
 > 	}
 >
 > 	void
 > 	f(T *restrict *restrict ls)
 > 	{
 > 		replace(ls, ls[0], 1);
 > 	}
 >
 > 	$ gcc-14 -Wall -Wextra -fanalyzer replace.c -S
 > 	$
 >
 > The above program causes UB,

It’s not a complete program and I don’t see the undefined behavior. If 
behavior is undefined because it violates the [[gnu::access(...)]] 
restrictions, that is not the sort of example that would convince the C 
standardization committee; they’d want to see a standard C program.

I tried to write a standard C program to illustrate the issue, and came 
up with the following.

   #include <string.h>

   typedef struct { int x; } T;

   static void
   replace (T *restrict const *restrict ls,
	   T const *restrict new, size_t pos)
   {
     memcpy (ls[pos], new, sizeof (T));
   }

   static void
   f (T *restrict const *restrict ls)
   {
     replace (ls, ls[0], 1);
   }

   int
   main ()
   {
     T u = {100}, v = {200}, *a[2] = {&u, &v};
     f (a);
     return a[0]->x ^ a[1]->x;
   }

However, I still don’t see undefined behavior there. gcc -O2 compiles 
this as if it were ‘int main () { return 0; }’ which is the only 
possible correct behavior.


 > writing an analyzer that triggers on this code, it
 > will trigger on strtol(3) too.

Sorry, I’m still not following the motivation for the proposed change. 
It appears to be something like “if we removed ‘restrict’ from strtol 
etc’s first arg, compilers could generate better ‘restrict’ diagnostics 
everywhere” but none if this is clear or making sense to me. And if I’m 
missing the point I have little doubt that the C committee will miss it too.

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 11552 bytes --]

Hi Paul,

On Tue, Jul 09, 2024 at 02:09:24PM GMT, Paul Eggert wrote:
> On 7/8/24 00:52, Alejandro Colomar wrote:
> > a small set of functions
> > accept pointers that alias each other, but one of them is never
> > accessed; in those few cases, restrict was added to the parameters in
> > ISO C, but I claim it would be better removed.
> 
> Are these aliasing pointers the nptr and initial *endptr of strtol?

Yes.

> That is,
> are you saying that last line in the following example, which is currently
> invalid, should become valid and should be implementable as ‘end = s; long l
> = 0;’?

No.  I don't think this is a consequence of the previous statement.

> 
>    char *end;
>    char *s = (char *) &end;
>    *s = '\0';
>    long l = strtol (s, &end, 0);
> 
> If so, I fail to see the motivation for the proposed change, as nobody
> writes (or should write) code like that. And if not, evidently I
> misunderstand the proposal.

My proposal is:

	 long int
	-strtol(const char *restrict nptr, char **restrict endptr, int base);
	+strtol(const char *nptr, char **restrict endptr, int base);

My proposal doesn't make valid the example above.  To make that example
valid, you'd need:

	long int
	strtol(const char *nptr, char **endptr, int base);

Because in the example above, you're aliasing nptr with endptr, not with
*endptr.  Thus, endptr cannot be a restricted pointer for that example
to be valid.

[... snip ...]

I'm not sure I understood that part, but it's probably a consequence of
the misuderstanding from above.  Let's ignore it for now, and please
resend if you think it's still a concern.

> 
> > Maybe I should use abstract names for the objects, to avoid confusing
> > them with the pointer variables that are used to pass them?
> 
> That might help, yes, since v0.2 is unclear on this point.

Ok; will do.

> > this formal
> > definition is quite unreadable, though.  The more I read it, the less
> > sure I am about it.
> 
> Yes, it’s lovely isn’t it? One must understand what the C committee
> intended in order to read and understand that part of the standard.

:-)

> > 	If L is used to access the value of the object X that it
> > 	designates, and X is also modified (by any means), then the
> > 	following requirements apply: T shall not be const-qualified
> >
> > This reads to me as "const variables are not writable when they are
> > accessed via a restricted pointer; casting away is not enough".  Am I
> > reading this correctly?
> 
> In that quoted statement, the restricted pointer is not allowed to be
> pointer-to-const. However, I’m not quite sure what your question means, as
> the phrase “const variables” does not appear in the standard. Perhaps give
> an example to clarify the question?

I should have said

"An object pointed to by a pointer-to-const cannot be written if the
pointer is a restricted one; casting const away is not enough."

Is this interpretation of restrict correct?

> >> an implementation is allowed to set errno = EINVAL first thing, and then
> set
> >> errno to some other nonzero value if it determines that the arguments are
> >> valid. I wouldn't implement strtol that way, but I can see where someone
> >> else might do that.
> >
> > In any case an implementation is not obliged to pessimize strtol(3).  It
> > is only allowed to.  Should we not allow them to do so?
> 
> Of course the standard should allow suboptimal implementations. However, I’m
> not sure what the point of the question is. The “errno = EINVAL first thing”
> comment says that removing ‘restrict’ obliges the implementation to support
> obviously-bogus calls like strtol(&errno, ...), which might make the
> implementation less efficient.

See for example how musl implements strtol(3):

$ grepc strtox src/stdlib/strtol.c
src/stdlib/strtol.c:static unsigned long long strtox(const char *s, char **p, int base, unsigned long long lim)
{
	FILE f;
	sh_fromstring(&f, s);
	shlim(&f, 0);
	unsigned long long y = __intscan(&f, base, 1, lim);
	if (p) {
		size_t cnt = shcnt(&f);
		*p = (char *)s + cnt;
	}
	return y;
}

The work is done within __intscan(), which could be prototyped as

	hidden unsigned long long
	__intscan(FILE *restrict, unsigned, int, unsigned long long);

And now you're able to optimize internally, since thanks to that helper
function you know it doesn't alias errno, regardless of the external
API.


BTW, now I remember that strtol(3) says:

ERRORS
     This function does not modify errno on success.

Which means that setting errno at function start wouldn't make much
sense.  Although there's probably a contrived way of doing it and still
be conformant (plus, I think ISO C doesn't say that about errno).

> I don’t see how the question is relevant to
> that comment.
> 
> 
> > Let's take a simpler one: rename(2).  Is it allowed to receive &errno?
> > Hopefully not.
> 
> I agree with that hope, but the current C standard seems to allow it. I
> think we both agree this is a defect in the standard.

Yup.  :)

> >>>> Why is this change worth
> >>>> making? Real-world programs do not make calls like that.
> >>>
> >>> Because it makes analysis of 'restrict' more consistent.  The obvious
> >>> improvement of GCC's analyzer to catch restrict violations will trigger
> >>> false positives in normal uses of strtol(3).
> >>
> >> v0.2 does not support this line of reasoning. On the contrary, v0.2
> suggests
> >> that a compiler should diagnose calls like "strtol(p, &p, 0)", which
> would
> >> be wrong as that call is perfectly reasonable.
> >
> > That call is perfectly, reasonable, which is why I suggest that the
> > standard should modify the prototype so that strtol(p, &p, 0), which is
> > a reasonable call, should not be warned by a compiler that would
> > diagnose such calls.
> 
> Of course they shouldn’t warn. But where are these compilers?
> 
> v0.2 asserts that “An analyzer more powerful than the current ones could
> extend the current -Wrestrict diagnostic to also diagnose this case.” But
> why would an analyzer want to do that? v0.2 doesn’t say.

True.

> The proposal merely asks to change prototypes for the C standard functions
> strtol, strtoul, etc. But if that is the only change needed then why bother?
> C compilers already do special-case analysis for functions defined by the C
> standard, and they can suppress undesirable diagnostics for these special
> cases.
> 
> If you’ve identified a more general problem with ‘restrict’ then welcome to
> the club! The experts already know it’s confusing and limited, and are
> discussing about whether and how to improve things in the next C standard. I
> am sure you’d be welcome to those discussions.

Thanks!  I'm thinking I'll drop my proposal and redirection it into
replacing restrict by something better.

> > That is, just by reading the prototypes:
> >
> > 	void foo(int *restrict x, int **restrict p);
> >
> > and
> >
> > 	void bar(int *x, int **restrict endp);
> >
> > one should be able to determine that
> >
> > 	foo(p, &p);
> >
> > is probably causing UB (and thus trigger a warning) but
> >
> > 	bar(p, &p);
> >
> > is fine.
> 
> Sure, but this is a discussion we should be having with the compiler
> writers, no?
> 
> Is this the main motivation for the proposal?

Yep.

> If so, how would weakening the
> spec for strtol etc. affect that discussion with the compiler writers? v0.2
> does not make this clear.
> 
> 
> >> Another way to put it: v0.2 does not clearly state the advantages of the
> >> proposed change, and in at least one area what it states as an advantage
> >> would actually be a disadvantage.
> >
> > The advantage is having more information in the caller.  As a caller, I
> > want to distinguish calls where it's ok to pass pointers that alias, and
> > where not.  And I want my compiler to be able to help me there.
> 
> I’m still not understanding. Removing ‘restrict’ from strtol’s first arg
> gives the caller less information, not more.

Actually, the caller seems to have perfect information about strtol(3),
regardless of restrict.  (As long as strtol(3) uses gnu access attributes.)

However, in this paragraph, I meant not about strtol(3), but in general:
If a caller know if two arguments to a function are allowed to alias
just by seeing the uses of restrict in the prototype, it is allowed to
turn on strict diagnostics about it to catch UB.

> > I'd rather have a simple analyzer, which will provide for
> > less false positives and negatives.
> 
> The C committee appears to have the opposite opinion, as when they were
> asked about this matter they added Examples 5 through 7 to what is now
> §6.7.4.2 (Formal definition of restrict). These examples say that Example 2
> (which uses ‘restrict’ on all arguments) is the simplest and most effective
> way to use ‘restrict’, even though a smarter compiler can still make some
> good inferences when some pointer args are ‘restrict’ and others are merely
> pointers to const.
> 
> If the proposal is disagreeing with Examples 5 through 7, this point needs
> to be thoroughly discussed in the proposal.

My proposal is thinking now that restrict is a dead end, and must be
replaced by something better.

> > GCC can only catch the most obvious violations of restrict.
> 
> Yes, but I fail to see how changing the API for strtol etc. would improve
> that situation.
> 
> 
> > 	#include <string.h>
> >
> > 	typedef struct {
> > 		int x;
> > 	} T;
> >
> > 	[[gnu::access(read_only, 1)]]
> > 	[[gnu::access(read_only, 2)]]
> > 	void
> > 	replace(T *restrict *restrict ls, const T *restrict new, size_t pos)
> > 	{
> > 		memcpy(ls[pos], new, sizeof(T));
> > 	}
> >
> > 	void
> > 	f(T *restrict *restrict ls)
> > 	{
> > 		replace(ls, ls[0], 1);
> > 	}
> >
> > 	$ gcc-14 -Wall -Wextra -fanalyzer replace.c -S
> > 	$
> >
> > The above program causes UB,
> 
> It’s not a complete program and I don’t see the undefined behavior.

I should have said s/program/code/

> If
> behavior is undefined because it violates the [[gnu::access(...)]]
> restrictions,

It does not violate the gnu::access restrictions.  It actually only
reads the objects pointed to by ls and new.  It is the object pointed to
by *ls the one which is written to, but that's fine.

When I wrote it, I was thinking that the behavior was undefined because
the object pointed to by *ls is aliased by the object pointed to by new.
However, it is not UB; I forgot that restrict doesn't care if the
pointer aliases; it only cares if an access does alias, which does not
happen.  Let's s/0/1/ in that code to make it UB.

If you s/0/1/ in my code, it is UB.  I'd like a substitute for restrict
to reject that code because both new and ls are derived from the same
pointer in the caller.  That is, I'd like passing two references to the
same object is UB, via some attribute; regardless of accesses.  More or
less what Rust does, but opt-in in a controlled way.

> that is not the sort of example that would convince the C
> standardization committee; they’d want to see a standard C program.
> 
> I tried to write a standard C program to illustrate the issue, and came up
> with the following.

[...]

Have a lovely day!
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: WG14 paper for removing restrict from nptr in strtol(3)

[David Malcolm]

On Sun, 2024-07-07 at 14:42 +0200, Alejandro Colomar wrote:
> Hi Paul,
> 
> On Sun, Jul 07, 2024 at 12:42:51PM GMT, Paul Eggert wrote:
> > On 7/7/24 03:58, Alejandro Colomar wrote:
> > 
> > > I've incorporated feedback, and here's a new revision, let's call
> > > it
> > > v0.2, of the draft for a WG14 paper.
> > Although I have not followed the email discussion closely, I read
> > v0.2 and
> > think that as stated there is little chance that its proposal will
> > be
> > accepted.
> 
> Thanks for reading thoroughly, and the feedback!
> 
> > Fundamentally the proposal is trying to say that there are two
> > styles X and
> > Y for declaring strtol and similar functions, and that although
> > both styles
> > are correct in some sense, style Y is better than style X. However,
> > the
> > advantages of Y are not clearly stated and the advantages of style
> > X over Y
> > are not admitted, so the proposal is not making the case clearly
> > and fairly.
> > 
> > One other thing: to maximize the chance of a proposal being
> > accepted, please
> > tailor it for its expected readership. The C committee is expert on
> > ‘restrict’, so don’t try to define ‘restrict’ in your own way.
> > Unless merely
> > repeating the language of the standard, any definition given for
> > ‘restrict’
> > is likely to cause the committee to quibble with the restatement of
> > the
> > standard wording. (It is OK to mention some corollaries of the
> > standard
> > definition, so long as the corollaries are not immediately
> > obvious.)
> > 
> > Here are some comments about the proposal. At the start these
> > comments are
> > detailed; towards the end, as I could see the direction the
> > proposal was
> > headed and was convinced it wouldn’t be accepted as stated, the
> > comments are
> > less detailed.
> > 
> > 
> > "The API may copy"
> > 
> > One normally doesn’t think of the application programming interface
> > as
> > copying. Please replace the phrase “the API” with “the caller” or
> > “the
> > callee” as appropriate. (Although ‘restrict’ can be used in places
> > other
> > than function parameters, I don’t think the proposal is concerned
> > about
> > those cases and so it doesn’t need to go into that.)
> 
> Ok.
> 
> > "To avoid violations of for example C11::6.5.16.1p3,"
> > 
> > Code that violates C11::6.5.16.1p3 will do so regardless of whether
> > ‘restrict’ is present. I would not mention C11::6.5.16.1p3 as it’s
> > a red
> > herring. Fundamentally, ‘restrict’ is not about the consequences of
> > caching
> > when one does overlapping moves; it’s about caching in a more
> > general sense.
> 
> The violations are UB regardless of restrict, but consistent use of
> restrict allows the caller to have a rough model of what the callee
> will
> do with the objects, and prevent those violations via compiler
> diagnostics.  I've reworded that part to make it more clear why I'm
> mentioning that.
> 
> > “As long as an object is only accessed via one restricted pointer,
> > other
> > restricted pointers are allowed to point to the same object.”
> > 
> > “only accessed” → “accessed only”
> 
> Ok.
> 
> > “This is less strict than I think it should be, but this proposal
> > doesn’t
> > attempt to change that definition.”
> > 
> > I would omit this sentence and all similar sentences. Don’t
> > distract the
> > reader with other potential proposals. The proposal as it stands is
> > complicated enough.
> 
> Ok.
> 
> > “return ca > a;”
> > “return ca > *ap;”
> > 
> > I fail to understand why these examples are present. It’s not
> > simply that
> > nobody writes code like that: the examples are not on point. I
> > would remove
> > the entire programs containing them, along with the sections that
> > discuss
> > them. When writing to the C committee one can assume the reader is
> > expert in
> > ‘restrict’, there is no need for examples such as these.
> 
> Those are examples of how consistent use of restrict can --or could,
> in
> the case of g()-- detect, via compiler diagnostics, (likely)
> violations
> of seemingly unrelated parts of the standard, such as the referenced
> C11::6.5.16.1p3, or in this case, C11::6.5.8p5.  
> 
> > “strtol(3) accepts 4 objects via pointer parameters and global
> > variables.”
> > 
> > Omit the “(3)”, here and elsewhere, as the audience is the C
> > standard
> > committee.
> 
> The C standard committee doesn't know about historic use of (3)? 
> That
> predates the standard, and they built on top of that (C originated in
> Unix).  While they probably don't care about it anymore, I expect my
> paper to be read by other audience, including GCC and glibc, and I
> prefer to keep it readable for that audience.  I expect the standard
> committee to at least have a rough idea of the existence of this
> syntax,
> and respect it, even if they don't use it or like it.
> 
> > “accepts” is a strange word to use here: normally one says
> > “accepts” to talk
> > about parameters, not global variables.
> 
> The thing is, strtol(3) does not actually access *endptr.  I thought
> that might cause more confusion than using "accepts".
> 
> > Also, “global variables” is not
> > right here. The C standard allows strtol, for example, to read and
> > write an
> > internal static cache. (Yes, that would be weird, but it’s
> > allowed.)
> 
> That's not part of the API.  A user must not access internal static
> cache, and so the implementation is free to assume that it doesn't,
> regardless of the use of restrict in the API, so it is not relevant
> for
> the purpose of this discussion, I think.
> 
> > I
> > suggest rephrasing this sentence to talk about accessing, not
> > accepting.
> 
> I don't want to use accessing, for it would be inconsistent with
> later
> saying that *endptr is not accessed.  However, I'm open to other
> suggested terms that might be more appropriate than both.
> 
> > “endptr access(write_only) ... *endptr access(none)”
> > 
> > This is true for glibc, but it’s not necessarily true for all
> > conforming
> > strtol implementations. If endptr is non-null, a conforming strtol
> > implementation can both read and write *endptr;
> 
> It can't, I think.  It's perfectly valid to pass an uninitialized
> endptr, which means the callee must not read the original value.
> 
>         char *end;
>         strtol("0", &end, 0);
> 
> If strtol(3) would be allowed to read it, the user would need to
> initialize it.
> 
> > it can also both read and
> > write **endptr. (Although it would need to write before reading,
> > reading is
> > allowed.)
> 
> Here, we need to consider two separate objects.  The object pointed-
> to
> by *endptr _before_ the object pointed to by endptr is written to,
> and
> the object pointed-to by *endptr _after_ the object pointed to by
> endptr
> is written to.
> 
> For the former (the original *endptr):
> 
>         Since *endptr might be uninitialized, strtol(3) must NOT
> access
>         the object pointed to by an uninitialized pointer.
> 
> For the latter (the final *endptr):
> 
>         The callee cannot write to it, since the specification of the
>         function is that the string will not be modified.  And in any
>         case, such an access is ultimately derived from nptr, not
> from
>         *endptr, so it does not matter for the discussion of *endptr.
> 
> Of course, that's derived from the specification of the function, and
> not from its prototype, since ISO C doesn't provide such detailed
> prototypes (since it doesn't have the [[gnu::access()]] attribute). 
> But
> the standard must abide by its own specification of functions,
> anyway.
> 
> > “This qualifier helps catch obvious bugs such as strtol(p, p, 0)
> > and
> > strtol(&p, &p, 0) .”
> > 
> > No it doesn’t. Ordinary type checking catches those obvious bugs,
> > and
> > ‘restrict’ provides no additional help there. Please complicate the
> > examples
> > to make the point more clearly.
> 
> To be pedantic, I didn't specify the type of p, so it might be (void
> *),
> and thus avoid type checking at all.  However, to avoid second
> guessing
> from the standards committee, I'll add casts, to make it more obvious
> that restrict is catching those.
> 
> > “The caller knows that errno doesn’t alias any of the function
> > arguments.”
> > 
> > Only because all args are declared with ‘restrict’. So if the
> > proposal is
> > accepted, the caller doesn’t necessarily know that.
> 
> Not really.  The caller has created the string (or has received it
> via a
> restricted pointer), and so it knows it's not derived from errno.
> 
>         char buf[LINE_MAX + 1];
> 
>         fgets(...);
>         n = strtol(buf, ...);
> 
> This caller knows with certainty that errno does not alias buf.  Of
> course, in some complex cases, it might not know, but I ommitted that
> for simplicity.  And in any case, I don't think any optimizations are
> affected by that in the caller.
> 
> > 
> > 
> > “The callee knows that *endptr is not accessed.”
> > 
> > This is true for glibc, but not necessarily true for every
> > conforming strtol
> > implementation.
> 
> The original *endptr may be uninitialized, and so must not be
> accessed.
> 
> > “It might seem that it’s a problem that the callee doesn’t know if
> > nptr can
> > alias errno or not. However, the callee will not write to the
> > latter
> > directly until it knows it has failed,”
> > 
> > Again this is true for glibc, but not necessarily true for every
> > conforming
> > strtol implementation.
> 
> An implementation is free to set errno = EDEADLK in the middle of it,
> as
> long as it later removes that.  However, I don't see how it would
> make
> any sense.
> 
> If that's done, it's probably done via a helper internal function,
> which
> as said below, can use restrict for nptr, and thus know with
> certainty
> that nptr is distinct from errno.
> 
> If that's done directly in the body of strtol(3) (the only place
> where
> it's not known that nptr is distinct from errno) we can probably
> agree
> that the implementation is doing that just for fun, and doesn't care
> about optimization, and thus we can safely ignore it.
> 
> > To my mind this is the most serious objection. The current standard
> > prohibits calls like strtol((char *) &errno, 0, 0). The proposal
> > would relax
> > the standard to allow such calls. In other words, the proposal
> > would
> > constrain implementations to support such calls.
> 
> I don't think it does.  ISO C specifies that strtol(3) takes a string
> as
> its first parameter, and errno is not (unless you do this:).
> 
>         (char *)&errno = "111";
> 
> Okay, let's assume you're allowed to do that, since a char* can alias
> anything.
> 
> I still don't think ISO C constrains implementations to allow passing
> (char *)&errno as a char*, just because it's not restrict.  Let's
> find
> an ISO C function that accepts a non-restrict string:
> 
>         int system(const char *string);
> 
> Does ISO C constrain implementations to support system((char
> *)&errno)?
> I don't think so.  Maybe it does implicitly because of a defect in
> the
> wording, but even then it's widely understood that it doesn't.
> 
> > Why is this change worth
> > making? Real-world programs do not make calls like that.
> 
> Because it makes analysis of 'restrict' more consistent.  The obvious
> improvement of GCC's analyzer to catch restrict violations will
> trigger
> false positives in normal uses of strtol(3).

Hi Alejandro

I'm author/maintainer of GCC's -fanalyzer option, which is presumably
why you CCed me on this.  One of my GSoC 2022 students (Tim Lange)
looked at making use of 'restrict' in -fanalyzer, see e.g. 
https://lists.gnu.org/archive/html/bug-gnulib/2022-07/msg00062.html

Based on Paul's comment here:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99860#c2 (and its
references) I came to the conclusion at the time that we should work on
something else, as the meaning of 'restrict' is too ambiguous.

Later, I added a new -Wanalyzer-overlapping-buffers warning in GCC 14,
which simply has a hardcoded set of standard library functions that it
"knows" to warn about.

Has the C standard clarified the meaning of 'restrict' since that
discussion?  Without that, I wasn't planning to touch 'restrict' in
GCC's -fanalyzer.

Sorry if I'm missing anything here; I confess I've skimmed through
parts of this thread.

Dave







> 
> > “But nothing prohibits those internal helper functions to specify
> > that nptr
> > is restrict and thus distinct from errno.”
> > 
> > Although true, it’s also the case that the C standard does not
> > *require*
> > internal helper functions to use ‘restrict’. All that matters is
> > the
> > accesses. So I’m not sure what the point of this statement is.
> 
> If an implementation wants to optimize, it should be careful and use
> restrict.  If it doesn't, then it can't complain that ISO C doesn't
> allow it to.  It's actually allowed to optimize, but it has to do
> some
> work for it.
> 
> > “m = strtol(p, &p, 0); An analyzer more powerful than the current
> > ones
> > could extend the current -Wrestrict diagnostic to also diagnose
> > this case.”
> > 
> > Why would an analyzer want to do that? This case is a perfectly
> > normal thing
> > to do and it has well-defined behavior.
> 
> Because without an analyzer, restrict cannot emit many useful
> diagnostics.  It's a qualifier that's all about data flow analysis,
> and
> normal diagnostics aren't able to do that.
> 
> A qualifier that enables optimizations but doesn't enable diagnostics
> is
> quite dangerous, and probably better not used.  If however, the
> analyzer
> emits advanced diagnostics for misuses of it, then it's a good
> qualifier.
> 
> Have a lovely day!
> Alex
> 
> > 
> > “To prevent triggering diagnostics in a powerful analyzer that
> > would be
> > smart enough to diagnose the example function g(), the prototype of
> > strtol(3) should be changed to ‘long int strtol(const char *nptr,
> > char
> > **restrict endptr, int base);’”
> > 
> > Sorry, but the case has not been made to make any such change to
> > strtol’s
> > prototype. On the contrary, what I’m mostly gathering from the
> > discussion is
> > that ‘restrict’ can be confusing, which is not news.
> > 
> > n3220 §6.7.4.2 examples 5 through 7 demonstrate that the C
> > committee has
> > thought through the points you’re making. (These examples were not
> > present
> > in C11.) This may help to explain why the standard specifies strtol
> > with
> > ‘restrict’ on both arguments.
> > 
> 

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2342 bytes --]

On Mon, Jul 08, 2024 at 10:30:48AM GMT, David Malcolm wrote:
> > > Why is this change worth
> > > making? Real-world programs do not make calls like that.
> > 
> > Because it makes analysis of 'restrict' more consistent.  The obvious
> > improvement of GCC's analyzer to catch restrict violations will
> > trigger
> > false positives in normal uses of strtol(3).
> 
> Hi Alejandro

Hi Dave,

> I'm author/maintainer of GCC's -fanalyzer option, which is presumably
> why you CCed me on this.

Yup.

> One of my GSoC 2022 students (Tim Lange)
> looked at making use of 'restrict' in -fanalyzer, see e.g. 
> https://lists.gnu.org/archive/html/bug-gnulib/2022-07/msg00062.html
> 
> Based on Paul's comment here:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99860#c2 (and its
> references) I came to the conclusion at the time that we should work on
> something else, as the meaning of 'restrict' is too ambiguous.

restrict, as of the formal definition of ISO C is useless crap.  The
more I read it, the more I agree.

restrict, as of what -Wrestrict warns about, seems a reasonable thing.

How about a [[gnu::restrict()]] attribute, similar to [[gnu::access()]],
which is simpler than the qualifier?  Since restrict is only meaningful
in function boundaries, it would make sense to have a function
attribute.  We don't want a qualifier that must follow discarding rules.

And then have it mean something strict, such as: The object pointed to
by the pointer is not pointed to by any other pointer; period.

This definition is already what -Wrestrict seems to understand.

> Later, I added a new -Wanalyzer-overlapping-buffers warning in GCC 14,
> which simply has a hardcoded set of standard library functions that it
> "knows" to warn about.

Hmmm, so it doesn't help at all for anything other than libc.  Ok.

> Has the C standard clarified the meaning of 'restrict' since that
> discussion?  Without that, I wasn't planning to touch 'restrict' in
> GCC's -fanalyzer.

Meh; no they didn't.  I understand.  That's why I don't like innovations
in ISO C, and prefer that implementations innovate with real stuff.

> Sorry if I'm missing anything here; I confess I've skimmed through
> parts of this thread.

Nope; all's good.

> 
> Dave

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Martin Uecker]

Am Montag, dem 08.07.2024 um 17:01 +0200 schrieb Alejandro Colomar:
> On Mon, Jul 08, 2024 at 10:30:48AM GMT, David Malcolm wrote:

...
> And then have it mean something strict, such as: The object pointed to
> by the pointer is not pointed to by any other pointer; period.
> 
> This definition is already what -Wrestrict seems to understand.

One of the main uses of restrict is scientific computing. In this
context such a definition of "restrict" would not work for many 
important use cases. But I agree that for warning purposes the
definition of "restrict" in ISO C is not helpful.

> 
> > Later, I added a new -Wanalyzer-overlapping-buffers warning in GCC 14,
> > which simply has a hardcoded set of standard library functions that it
> > "knows" to warn about.
> 
> Hmmm, so it doesn't help at all for anything other than libc.  Ok.
> 
> > Has the C standard clarified the meaning of 'restrict' since that
> > discussion?  Without that, I wasn't planning to touch 'restrict' in
> > GCC's -fanalyzer.
> 
> Meh; no they didn't.  

There were examples added in C23 and there are now several papers
being under discussion.


> I understand.  That's why I don't like innovations
> in ISO C, and prefer that implementations innovate with real stuff.

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1371 bytes --]

Hi Martin,

On Mon, Jul 08, 2024 at 06:05:08PM GMT, Martin Uecker wrote:
> Am Montag, dem 08.07.2024 um 17:01 +0200 schrieb Alejandro Colomar:
> > On Mon, Jul 08, 2024 at 10:30:48AM GMT, David Malcolm wrote:
> 
> ...
> > And then have it mean something strict, such as: The object pointed to
> > by the pointer is not pointed to by any other pointer; period.
> > 
> > This definition is already what -Wrestrict seems to understand.
> 
> One of the main uses of restrict is scientific computing. In this
> context such a definition of "restrict" would not work for many 
> important use cases. But I agree that for warning purposes the
> definition of "restrict" in ISO C is not helpful.

Do you have some examples of functions where this matters and is
important?  I'm curious to see them.  Maybe we find some alternative.

> > > Has the C standard clarified the meaning of 'restrict' since that
> > > discussion?  Without that, I wasn't planning to touch 'restrict' in
> > > GCC's -fanalyzer.
> > 
> > Meh; no they didn't.  
> 
> There were examples added in C23 and there are now several papers
> being under discussion.

Hmm, yeah, the examples help with the formal definition.  I was thinking
of the definition itself, which I still find quite confusing.  :-)

Have a lovely night!
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Martin Uecker]

Am Montag, dem 08.07.2024 um 22:17 +0200 schrieb Alejandro Colomar:
> Hi Martin,
> 
> On Mon, Jul 08, 2024 at 06:05:08PM GMT, Martin Uecker wrote:
> > Am Montag, dem 08.07.2024 um 17:01 +0200 schrieb Alejandro Colomar:
> > > On Mon, Jul 08, 2024 at 10:30:48AM GMT, David Malcolm wrote:
> > 
> > ...
> > > And then have it mean something strict, such as: The object pointed to
> > > by the pointer is not pointed to by any other pointer; period.
> > > 
> > > This definition is already what -Wrestrict seems to understand.
> > 
> > One of the main uses of restrict is scientific computing. In this
> > context such a definition of "restrict" would not work for many 
> > important use cases. But I agree that for warning purposes the
> > definition of "restrict" in ISO C is not helpful.
> 
> Do you have some examples of functions where this matters and is
> important?  I'm curious to see them.  Maybe we find some alternative.

In many numerical algorithms you want to operate on
different parts of the same array object.  E.g. for matrix
decompositions you want to take a row / column and add it 
to another. Other examples are algorithms that decompose
some input (.e.g. high and low band in a wavelet transform)
and store it into the same output array, etc.

Without new notation for strided array slicing, one
fundamentally needs the flexibility of restrict that
only guarantuees that actual accesses do not conflict.

But this then implies that one can not use restrict as a
contract specification on function prototypes, but has
to analyze the implementation of a function to see if
it is used correctly.  But I would not see it as a design 
problem of restrict. It was simply not the intended use 
case when originally designed. 


> 
> > > > Has the C standard clarified the meaning of 'restrict' since that
> > > > discussion?  Without that, I wasn't planning to touch 'restrict' in
> > > > GCC's -fanalyzer.
> > > 
> > > Meh; no they didn't.  
> > 
> > There were examples added in C23 and there are now several papers
> > being under discussion.
> 
> Hmm, yeah, the examples help with the formal definition.  I was thinking
> of the definition itself, which I still find quite confusing.  :-)

Indeed.

Martin

> 
> Have a lovely night!
> Alex
> 

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 3078 bytes --]

Hi Martin,

On Tue, Jul 09, 2024 at 07:58:40AM GMT, Martin Uecker wrote:
> Am Montag, dem 08.07.2024 um 22:17 +0200 schrieb Alejandro Colomar:
> > Hi Martin,
> > 
> > On Mon, Jul 08, 2024 at 06:05:08PM GMT, Martin Uecker wrote:
> > > Am Montag, dem 08.07.2024 um 17:01 +0200 schrieb Alejandro Colomar:
> > > > On Mon, Jul 08, 2024 at 10:30:48AM GMT, David Malcolm wrote:
> > > 
> > > ...
> > > > And then have it mean something strict, such as: The object pointed to
> > > > by the pointer is not pointed to by any other pointer; period.
> > > > 
> > > > This definition is already what -Wrestrict seems to understand.
> > > 
> > > One of the main uses of restrict is scientific computing. In this
> > > context such a definition of "restrict" would not work for many 
> > > important use cases. But I agree that for warning purposes the
> > > definition of "restrict" in ISO C is not helpful.
> > 
> > Do you have some examples of functions where this matters and is
> > important?  I'm curious to see them.  Maybe we find some alternative.
> 
> In many numerical algorithms you want to operate on
> different parts of the same array object.  E.g. for matrix
> decompositions you want to take a row / column and add it 
> to another. Other examples are algorithms that decompose
> some input (.e.g. high and low band in a wavelet transform)
> and store it into the same output array, etc.
> 
> Without new notation for strided array slicing, one

I'll have to remove dust from my old proposal of [.nmemb]?  :)

> fundamentally needs the flexibility of restrict that
> only guarantuees that actual accesses do not conflict.

I guess a combination of [.nmemb] (or the third argument of
[[gnu::access()]] and [[alx::noalias]] could be good enough for such a
use case?

	[[alx::noalias(1)]] [[alx::noalias(2)]]
	void add(int a[.n], const int b[.n], n);

	//or

	[[alx::noalias(1)]] [[alx::noalias(2)]]
	[[gnu::access(read_write, 1, 3)]] [[gnu::access(read_only, 2, 3)]]
	void add(int a[], const int b[], n);

	mad(&arr[0], &arr[50], 50);

The caller should be able to know that while the pointers can alias
within the caller, they don't alias within the callee, since the callee
has no right to access past the specified bound*.

*  The standard would have to tighten bounds in function interfaces,
   since right now, the value is just ignored if it doesn't come with
   'static' (which I never understood), and if specified with 'static',
   it's just a lower bound not a strict bound.  I would propose changing
   the meaning of [N] in a function prototype to mean a strict bound.

Does that make sense?

Have a lovely day!
Alex

> But this then implies that one can not use restrict as a
> contract specification on function prototypes, but has
> to analyze the implementation of a function to see if
> it is used correctly.  But I would not see it as a design 
> problem of restrict. It was simply not the intended use 
> case when originally designed. 

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: WG14 paper for removing restrict from nptr in strtol(3)

[David Malcolm]

On Mon, 2024-07-08 at 17:01 +0200, Alejandro Colomar wrote:
> On Mon, Jul 08, 2024 at 10:30:48AM GMT, David Malcolm wrote:
> > > > Why is this change worth
> > > > making? Real-world programs do not make calls like that.
> > > 
> > > Because it makes analysis of 'restrict' more consistent.  The
> > > obvious
> > > improvement of GCC's analyzer to catch restrict violations will
> > > trigger
> > > false positives in normal uses of strtol(3).
> > 
> > Hi Alejandro
> 
> Hi Dave,
> 
> > I'm author/maintainer of GCC's -fanalyzer option, which is
> > presumably
> > why you CCed me on this.
> 
> Yup.
> 
> > One of my GSoC 2022 students (Tim Lange)
> > looked at making use of 'restrict' in -fanalyzer, see e.g. 
> > https://lists.gnu.org/archive/html/bug-gnulib/2022-07/msg00062.html
> > 
> > Based on Paul's comment here:
> > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99860#c2 (and its
> > references) I came to the conclusion at the time that we should
> > work on
> > something else, as the meaning of 'restrict' is too ambiguous.
> 
> restrict, as of the formal definition of ISO C is useless crap.  The
> more I read it, the more I agree.

Please note that "useless crap" was your wording, not mine.

> 
> restrict, as of what -Wrestrict warns about, seems a reasonable
> thing.
> 
> How about a [[gnu::restrict()]] attribute, similar to
> [[gnu::access()]],
> which is simpler than the qualifier?  Since restrict is only
> meaningful
> in function boundaries, it would make sense to have a function
> attribute.  We don't want a qualifier that must follow discarding
> rules.

If it doesn't have the same meaning as "restrict" then perhaps call the
proposed attribute something other than "restrict"?

That said, I don't have strong opinions on any of this, except to note
that I have more than enough *other* work on improvements to GCC's
static analyzer and usability to keep me busy, so getting sucked into
discussion/implementation on 'restrict' is something I want to avoid,
and -Wanalyzer-overlapping-buffers is getting the job done for me at
the moment.

[...snip...]

Hope this is constructive; sorry again if I missed anything due to only
skimming the thread

Dave

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1698 bytes --]

Hi Dave,

On Mon, Jul 08, 2024 at 06:48:51PM GMT, David Malcolm wrote:
> > restrict, as of the formal definition of ISO C is useless crap.  The
> > more I read it, the more I agree.
> 
> Please note that "useless crap" was your wording, not mine.

Yup.  :)

> 
> > 
> > restrict, as of what -Wrestrict warns about, seems a reasonable
> > thing.
> > 
> > How about a [[gnu::restrict()]] attribute, similar to
> > [[gnu::access()]],
> > which is simpler than the qualifier?  Since restrict is only
> > meaningful
> > in function boundaries, it would make sense to have a function
> > attribute.  We don't want a qualifier that must follow discarding
> > rules.
> 
> If it doesn't have the same meaning as "restrict" then perhaps call the
> proposed attribute something other than "restrict"?

Yup, I was thinking that maybe noalias is a better name.

> 
> That said, I don't have strong opinions on any of this, except to note
> that I have more than enough *other* work on improvements to GCC's
> static analyzer and usability to keep me busy, so getting sucked into
> discussion/implementation on 'restrict' is something I want to avoid,
> and -Wanalyzer-overlapping-buffers is getting the job done for me at
> the moment.
> 
> [...snip...]
> 
> Hope this is constructive; sorry again if I missed anything due to only
> skimming the thread

It is.  I don't want you to work on this if you don't have time or
interest.  Just having the idea floating aroud, and if somebody finds
time to have a look at it in the next decade, maybe try it.  :-)

Does that make sense?

Cheers,
Alex

> 
> Dave
> 
> 

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Jakub Jelinek]

On Tue, Jul 09, 2024 at 11:07:59AM +0200, Alejandro Colomar wrote:
> > > restrict, as of what -Wrestrict warns about, seems a reasonable
> > > thing.
> > > 
> > > How about a [[gnu::restrict()]] attribute, similar to
> > > [[gnu::access()]],
> > > which is simpler than the qualifier?  Since restrict is only
> > > meaningful
> > > in function boundaries, it would make sense to have a function
> > > attribute.  We don't want a qualifier that must follow discarding
> > > rules.
> > 
> > If it doesn't have the same meaning as "restrict" then perhaps call the
> > proposed attribute something other than "restrict"?
> 
> Yup, I was thinking that maybe noalias is a better name.

Name is one thing, but you'd also need to clearly define what it means.
When restrict is access based, it is clear what it means.

If you want something else which is not based on accesses and which should
allow warnings in the callers, I suppose you need to specify not just the
pointer but the extent as well (and maybe stride) or that it is an '\0'
terminated string, because if you want to say that for
void foo (char *, const char *, int);
the 2 pointers don't really alias, the size information is missing.  So,
shall the new warning warn on
struct S { char a[1024]; char b[1024]; } s;
foo (s.a, s.b, 512);
or not?  Or foo (s.a, s.a + 512, 512);

	Jakub

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2439 bytes --]

Hi Jakub,

On Tue, Jul 09, 2024 at 11:18:11AM GMT, Jakub Jelinek wrote:
> On Tue, Jul 09, 2024 at 11:07:59AM +0200, Alejandro Colomar wrote:
> > Yup, I was thinking that maybe noalias is a better name.
> 
> Name is one thing, but you'd also need to clearly define what it means.
> When restrict is access based, it is clear what it means.
> 
> If you want something else which is not based on accesses and which should
> allow warnings in the callers, I suppose you need to specify not just the
> pointer but the extent as well (and maybe stride) or that it is an '\0'

Agree.  Here's how I'd define it as an attribute:

noalias

	The noalias function attribute specifies that the pointer to
	which it applies is the only reference to the array object that
	it points to (except that a pointer to one past the last
	element may overlap another object).

	If the number of elements is specified with array notation, the
	array object to be considered is a subobject of the original
	array object, which is limited to the number of elements
	specified in the function prototype.

	Example:

	[[alx::noalias(1)]] [[alx::noalias(2)]]
	[[gnu::access(read_write, 1)]] [[gnu::access(read_only, 2)]]
	void add_inplace(int a[n], const int b[n], size_t n);

	char arr[100] = ...;

	add_inplace(arr, arr + 50, 50);

	In the example above, the parameters a and b don't alias inside
	the function, since the subobjects of 50 elements do not overlap
	eachother, even though they are one single array object to the
	outer function.

It may need some adjustment, to avoid conflicts with other parts of
ISO C, but this is the idea I have in mind.

> terminated string, because if you want to say that for
> void foo (char *, const char *, int);
> the 2 pointers don't really alias, the size information is missing.  So,
> shall the new warning warn on
> struct S { char a[1024]; char b[1024]; } s;
> foo (s.a, s.b, 512);

This does not need clarification of bounds.  You're passing separate
objects, and thus cannot alias (except that maybe you're able to cast
to the struct type, and then access s.b from a pointer derived from
s.a; I never know that rule too well).

> or not?  Or foo (s.a, s.a + 512, 512);

According to the definition I provide in this email, the above is just
fine.

Thanks!

Have a lovely day!
Alex

> 
> 	Jakub
> 
> 

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2752 bytes --]

On Tue, Jul 09, 2024 at 12:28:18PM GMT, Alejandro Colomar wrote:
> Hi Jakub,
> 
> On Tue, Jul 09, 2024 at 11:18:11AM GMT, Jakub Jelinek wrote:
> > On Tue, Jul 09, 2024 at 11:07:59AM +0200, Alejandro Colomar wrote:
> > > Yup, I was thinking that maybe noalias is a better name.
> > 
> > Name is one thing, but you'd also need to clearly define what it means.
> > When restrict is access based, it is clear what it means.
> > 
> > If you want something else which is not based on accesses and which should
> > allow warnings in the callers, I suppose you need to specify not just the
> > pointer but the extent as well (and maybe stride) or that it is an '\0'
> 
> Agree.  Here's how I'd define it as an attribute:
> 
> noalias
> 
> 	The noalias function attribute specifies that the pointer to
> 	which it applies is the only reference to the array object that
> 	it points to (except that a pointer to one past the last
> 	element may overlap another object).
> 
> 	If the number of elements is specified with array notation, the
> 	array object to be considered is a subobject of the original
> 	array object, which is limited to the number of elements
> 	specified in the function prototype.
> 
> 	Example:
> 
> 	[[alx::noalias(1)]] [[alx::noalias(2)]]
> 	[[gnu::access(read_write, 1)]] [[gnu::access(read_only, 2)]]
> 	void add_inplace(int a[n], const int b[n], size_t n);

Ooops, I meant 'n' to be the first parameter.

> 
> 	char arr[100] = ...;
> 
> 	add_inplace(arr, arr + 50, 50);
> 
> 	In the example above, the parameters a and b don't alias inside
> 	the function, since the subobjects of 50 elements do not overlap
> 	eachother, even though they are one single array object to the
> 	outer function.
> 
> It may need some adjustment, to avoid conflicts with other parts of
> ISO C, but this is the idea I have in mind.
> 
> > terminated string, because if you want to say that for
> > void foo (char *, const char *, int);
> > the 2 pointers don't really alias, the size information is missing.  So,
> > shall the new warning warn on
> > struct S { char a[1024]; char b[1024]; } s;
> > foo (s.a, s.b, 512);
> 
> This does not need clarification of bounds.  You're passing separate
> objects, and thus cannot alias (except that maybe you're able to cast
> to the struct type, and then access s.b from a pointer derived from
> s.a; I never know that rule too well).
> 
> > or not?  Or foo (s.a, s.a + 512, 512);
> 
> According to the definition I provide in this email, the above is just
> fine.
> 
> Thanks!
> 
> Have a lovely day!
> Alex
> 
> > 
> > 	Jakub
> > 
> > 
> 
> -- 
> <https://www.alejandro-colomar.es/>



-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 11769 bytes --]

Here's a proposal for adding a function attribute for replacing
the restrict restrict qualifier.  It's v0.3 of n3294 (now we have a
document number).

I was going to name it [[noalias()]], but I thought that it would be
possible to mark several pointers as possibly referencing the same
object, and then the name [[restrict()]] made more sense.

It's based on a proposal I sent to Martin recently in this discussion.

Do you have any feedback for this?

I've attached the man(7) source and the resulting PDF, and below goes
a plain text rendering (formatting is lost).


Have a lovely night!
Alex

---

N3294 (WG14)                Proposal for C2y               N3294 (WG14)

Name
     n3294  - The [[restrict()]] function attribute as a replacement of
     the restrict qualifier

Category
     Feature and deprecation.

Author
     Alejandro  Colomar  Andres;  maintainer  of  the  Linux  man-pages
     project.

   Cc
     GNU C library
     GNU Compiler Collection
     Linux man‐pages
     Paul Eggert
     Xi Ruoyao
     Jakub Jelinek
     Martin Uecker
     LIU Hao
     Jonathan Wakely
     Richard Earnshaw
     Sam James
     Emanuele Torre
     Ben Boeckel
     "Eissfeldt, Heiko"
     David Malcolm

Description
   restrict qualifier
     The  restrict  qualifier is not useful for diagnostics.  Being de‐
     fined in terms of accesses, the API is not enough for a caller  to
     know what the function will do with the objects it receives.

     That is, a caller cannot know if the following call is correct:

            void f(const int *restrict a, int **restrict b);

            f(a, &a);

     Having  no way to determine if a call will result in Undefined Be‐
     havior makes it a dangerous qualifier.

     The reader might notice that this prototype and call is very simi‐
     lar to the prototype of strtol(3), and the use reminds of a  rela‐
     tively common use of that function.

   Diagnostics
     A good replacement of the restrict qualifier should allow to spec‐
     ify  in  the  API of the following function that it doesn’t accept
     pointers that alias.

            void
            replace(const T *restrict new, T **restrict ls, size_t pos)
            {
                 memcpy(ls[pos], new, sizeof(T));
            }

     This proposal suggests the following:

            [[restrict(1)]] [[restrict(2)]]
            void
            replace(const T *restrict new, T **restrict ls, size_t pos);

            replace(arr[3], arr, 2);  // UB; can be diagnosed

   Qualifiers
     It is also unfortunate that restrict  is  a  qualifier,  since  it
     doesn’t  follow  the  rules  that  apply  to all other qualifiers.
     While it is discarded easily, its  semantics  make  it  as  if  it
     couldn’t be discarded.

   Function attribute
     The purpose of restrict is to

     •  Allow functions to optimize based on the knowledge that certain
        objects are not accessed by any other object in the same scope;
        usually a function boundary, which is the most opaque boundary,
        and where this information is not otherwise available.

     •  Diagnose  calls  that  would result in Undefined Behavior under
        this memory model.

     Qualifiers don’t seem to be good for  carrying  this  information,
     but  function attributes are precisely for adding information that
     cannot be expressed by just using the type system.

     An attribute would need to be more strict than the restrict quali‐
     fier to allow diagnosing non‐trivial cases, such as the call shown
     above.

     A caller only knows what the callee receives,  not  what  it  does
     with  it.  Thus, for diagnostics to work, the semantics of a func‐
     tion attribute should be specified in terms of what a function  is
     allowed to receive.

   [[restrict]]
     The  [[restrict]] function attribute specifies that the pointer to
     which it applies is the only reference  to  the  array  object  to
     which  it  points (except that a pointer to one past the last ele‐
     ment may overlap another object).

     If the number of elements is specified with array  notation  or  a
     compiler‐specific  attribute, the array object to be considered is
     a subobject of the original array object, which is limited by  the
     number of elementsspecified in the function prototype.

     For the following prototype:

            [[restrict(1)]] [[restrict(2)]]
            void add_inplace(size_t n, int a[n], const int b[n]);

     In  the following calls, the caller is able to determine with cer‐
     tainty if the behavior is defined or undefined:

            char  a[100] = ...;
            char  b[50] = ...;

            add_inplace(50, a, a + 50);  // Ok
            add_inplace(50, a, b);       // Ok
            add_inplace(50, a, a);       // UB

     In the first of the three calls, the parameters don’t alias inside
     the function, since the subobjects of 50 elements do  not  overlap
     each  other,  even  though they are one single array object to the
     outer function.

   Optimizations
     This function attribute allows similar  optimizations  than  those
     allowed by the restrict qualifier.

     strtol(3)
            In  some  cases, such as the strtol(3) function, the proto‐
            type will be different, since this  attribute  is  stricter
            than restrict, and can’t be applied to the same parameters.
            For example, the prototype for strtol(3) would be

                   [[restrict(2)]]
                   long
                   strtol(const char *str, char **endp, int base);

            This  could  affect optimizations, since now it’s not clear
            to the implementation that str is not modified by any other
            reference.   Compiler‐specific  attributes  can  help  with
            that.   For example, the [[gnu::access()]] attribute can be
            used in this function to give more information:

                   [[restrict(2)]]
                   [[gnu::access(read_only, 1)]]
                   [[gnu::access(write_only, 2)]]
                   long
                   strtol(const char *str, char **endp, int base);

            The fact that endp is write‐only  lets  the  callee  deduce
            that *endp cannot be used to write to the string (since the
            callee is not allowed to inspect *endp).

            Another  concern  is  that  a global variable such as errno
            might alias the string.  This is already a concern in  sev‐
            eral  ISO  C  calls, such as rename(2).  But in the case of
            strtol(3), it would be a regression.   There  are  ways  to
            overcome  that, such as designing helper functions in a way
            that the attribute can be applied to add extra information.

            It is important that diagnostics are easy to determine,  to
            avoid  false negatives and false positives, so that code is
            easily safe.  Optimizations, while important, need  not  be
            as  easy  to  apply  as  diagnostics.  If an implementation
            wants to be optimal, it will do the extra  work  for  being
            fast.

     Multiple aliasing pointers
            In  some cases, it might be useful to allow specifying that
            some pointers may alias each other, but not others.

     Strings
            Another way to determine that str cannot be aliased by  any
            other  object  such  as  errno would be to use an attribute
            that marks  str  as  a  string.   An  object  of  type  int
            shouldn’t  be  allowed to represent a string, so regardless
            of character types being allowed to alias any  other  type,
            an  attribute such as [[gnu::null_terminated_string_arg()]]
            might be used to determine that the global errno  does  not
            alias the string.

   Deprecation
     The restrict qualifier would be deprecated by this attribute, sim‐
     ilar  to how the noreturn function specifier was superseded by the
     [[noreturn]] function attribute.

   Backwards compatibility
     Removing the restrict qualifier from function prototypes does  not
     cause  problems  in  most functions.  Only functions with restrict
     applied to a pointee would  have  incompatible  definitions.   The
     only standard functions where this would happen are:

     tmpfile_s()
     fopen_s()
     freopen_s()

     Those  functions  are  not  widely  adopted,  so the problem would
     likely be minimal.

Proposal
   6.7.13.x The restrict function attribute
     Constraints
            The restrict attribute shall be applied to a function.

            A 1‐based index can be specified in an  attribute  argument
            clause,  to  associate the attribute with the corresponding
            parameter of the function, which must be of a pointer type.

            (Optional.)  Several indices can be specified, separated by
            commas.

            The attribute can be applied  several  times  to  the  same
            function, to mark several parameters with the attribute.

            (Optional.)   The argument attribute clause may be omitted,
            which is equivalent to specifying the  attribute  once  for
            each parameter that is a pointer.

     Semantics
            If  a  function is defined with the restrict attribute, the
            corresponding parameter shall be the only reference to  the
            array  object  that it points to.  If the function receives
            another reference to the same array object, the behavior is
            undefined.  If  the  function  accesses  the  array  object
            through  an  lvalue  that is not derived from that pointer,
            the behavior is undefined.

            (Optional.)  If more than one parameters are  specified  in
            the  same  attribute  argument  clause,  then  all of those
            pointers are allowed to point to the same array object.

            If the number of elements is specified with array  notation
            (or  a compiler‐specific attribute), the array object to be
            considered for aliasing is a sub‐object of the original ar‐
            ray object, limited by the number  of  elements  specifiedr
            [1].

     [1]  For the following prototype:

                 [[restrict(1)]] [[restrict(2)]]
                 void f(size_t n, int a[n], const int b[n]);

          In  the  the following calls, the caller is able to determine
          if the behavior is defined or undefined:

                 char a[100] = /*...*/;
                 char b[50] = /*...*/;

                 f(50, a, a + 50);  // Ok
                 f(50, a, b);       // UB; a diagnostic is recommended
                 f(50, a, a + 2);   // UB; a diagnostic is recommended

History
     Revisions of this paper:

     0.1    Original draft for removing restrict from the first parame‐
            ter of strtol(3).

     0.2    Incorporate feedback from glibc and gcc mailing lists.

     0.3    Re‐purpose,  to  deprecate  restrict  and   propose   [[re‐
            strict()]] instead.

See also
     The original discussion about restrict and strtol(3).

ISO/IEC 9899                   2024‐07‐09                  N3294 (WG14)


-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #1.2: restrict.man --]
[-- Type: application/x-troff-man, Size: 10361 bytes --]

[-- Attachment #1.3: restrict.pdf --]
[-- Type: application/pdf, Size: 14496 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Joseph Myers]

[-- Attachment #1: Type: text/plain, Size: 3092 bytes --]

On Wed, 10 Jul 2024, Alejandro Colomar via Gcc wrote:

>    6.7.13.x The restrict function attribute
>      Constraints
>             The restrict attribute shall be applied to a function.
> 
>             A 1‐based index can be specified in an  attribute  argument
>             clause,  to  associate the attribute with the corresponding
>             parameter of the function, which must be of a pointer type.

It's more appropriate to say "shall", and you need a requirement for the 
pointer to be a pointer to a complete object type (it makes no sense with 
function pointers, or void).  That is, something like "If an attribute 
argument clause is present, it shall have the form:

  ( constant-expression )

The constant expression shall be an integer constant expression with 
positive value.  It shall be the index, counting starting from 1, of a 
function parameter whose type is a pointer to a complete object type.".

(That might not quite be sufficient - there are the usual questions of 
exactly *when* the type needs to be complete, if it's completed part way 
through the function definition, but the standard already doesn't tend to 
specify such things very precisely.)

>             (Optional.)   The argument attribute clause may be omitted,
>             which is equivalent to specifying the  attribute  once  for
>             each parameter that is a pointer.

For each parameter that is a pointer to a complete object type, or should 
there be a constraint violation in this case if some are pointers to such 
types and some are pointers to other types?

>             If the number of elements is specified with array  notation
>             (or  a compiler‐specific attribute), the array object to be
>             considered for aliasing is a sub‐object of the original ar‐
>             ray object, limited by the number  of  elements  specifiedr
>             [1].

This is semantically problematic in the absence of something like N2906 
(different declarations could use different numbers of elements), and even 
N2906 wouldn't help for the VLA case.

>      [1]  For the following prototype:
> 
>                  [[restrict(1)]] [[restrict(2)]]
>                  void f(size_t n, int a[n], const int b[n]);

That declaration currently means

  void f(size_t n, int a[*], const int b[*]);

(that is, the expression giving a VLA size is ignored).  It's equivalent 
to e.g.

  void f(size_t n, int a[n + foo()], const int b[n + bar()]);

where because the size expressions are never evaluated and there's no time 
defined for evaluation, it's far from clear what anything talking about 
them giving an array size would even mean.


I know that "noalias" was included in some C89 drafts but removed from the 
final standard after objections.  Maybe someone who was around then could 
explain what "noalias" was, what the problems with it were and how it 
differs from "restrict", so we can make sure that any new proposals in 
this area don't suffer from whatever the perceived deficiencies of 
"noalias" were?

-- 
Joseph S. Myers
josmyers@redhat.com

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 809 bytes --]

At 2024-07-26T16:24:14+0000, Joseph Myers wrote:
> I know that "noalias" was included in some C89 drafts but removed from
> the final standard after objections.  Maybe someone who was around
> then could explain what "noalias" was, what the problems with it were

For this part, I think the source most often cited is Dennis Ritchie's
thunderbolt aimed directly at "noalias".

https://www.lysator.liu.se/c/dmr-on-noalias.html

> and how it differs from "restrict",

I can only disqualify myself as an authority here.

> To comprehensively address this demands so we can make sure that any
> new proposals in this area don't suffer from whatever the perceived
> deficiencies of "noalias" were?

I think it would be valuable to get such a discussion into the rationale
of the next C standard.

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1445 bytes --]

Hi Branden!

On Fri, Jul 26, 2024 at 11:35:51AM GMT, G. Branden Robinson wrote:
> At 2024-07-26T16:24:14+0000, Joseph Myers wrote:
> > I know that "noalias" was included in some C89 drafts but removed from
> > the final standard after objections.  Maybe someone who was around
> > then could explain what "noalias" was, what the problems with it were
> 
> For this part, I think the source most often cited is Dennis Ritchie's
> thunderbolt aimed directly at "noalias".
> 
> https://www.lysator.liu.se/c/dmr-on-noalias.html

Thanks!  It seems Dennis's concern was that it was a qualifier.
Probably the reason why restrict ended up being a qualifier on the
pointer (and thus easily ignored), instead of the pointee (it would have
caused the problems that Dennis mentioned and which anyone can guess).

Since I'm suggesting an attribute, we are pretty much safe from type
rules, and thus safe from Dennis's concerns, I think.

Have a lovely night!
Alex

> 
> > and how it differs from "restrict",
> 
> I can only disqualify myself as an authority here.
> 
> > To comprehensively address this demands so we can make sure that any
> > new proposals in this area don't suffer from whatever the perceived
> > deficiencies of "noalias" were?
> 
> I think it would be valuable to get such a discussion into the rationale
> of the next C standard.
> 
> Regards,
> Branden

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Paul Eggert]

On 7/26/24 09:24, Joseph Myers wrote:
>   Maybe someone who was around then could
> explain what "noalias" was, what the problems with it were and how it
> differs from "restrict"

You can get a hint by reading Dennis Ritchie's 1988 email with the 
unforgettable bottom line "Noalias must go. This is non-negotiable."

https://www.lysator.liu.se/c/dmr-on-noalias.html

... and this is partly why I haven't read Alejandro's proposal. Fiddling 
with 'restrict' should be done only very carefully and only if there are 
really important advantages to messing around with it.

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 5073 bytes --]

Hi Joseph,

On Fri, Jul 26, 2024 at 04:24:14PM GMT, Joseph Myers wrote:
> On Wed, 10 Jul 2024, Alejandro Colomar via Gcc wrote:
> 
> >    6.7.13.x The restrict function attribute
> >      Constraints
> >             The restrict attribute shall be applied to a function.
> > 
> >             A 1‐based index can be specified in an  attribute  argument
> >             clause,  to  associate the attribute with the corresponding
> >             parameter of the function, which must be of a pointer type.
> 
> It's more appropriate to say "shall", and you need a requirement for the 
> pointer to be a pointer to a complete object type (it makes no sense with 
> function pointers, or void).

I don't see why it should not apply to void*.  memcpy(3) should get this
attribute:

	[[alx::restrict(1)]]
	[[alx::restrict(2)]]
	void *memcpy(void *dst, const void *src, size_t n);

The index to which the text above refers is that '(1)' and '(2)'.

>  That is, something like "If an attribute 
> argument clause is present, it shall have the form:
> 
>   ( constant-expression )
> 
> The constant expression shall be an integer constant expression with 
> positive value.  It shall be the index, counting starting from 1, of a 
> function parameter whose type is a pointer to a complete object type.".
> 
> (That might not quite be sufficient - there are the usual questions of 
> exactly *when* the type needs to be complete, if it's completed part way 
> through the function definition, but the standard already doesn't tend to 
> specify such things very precisely.)
> 
> >             (Optional.)   The argument attribute clause may be omitted,
> >             which is equivalent to specifying the  attribute  once  for
> >             each parameter that is a pointer.
> 
> For each parameter that is a pointer to a complete object type, or should 
> there be a constraint violation in this case if some are pointers to such 
> types and some are pointers to other types?
> 
> >             If the number of elements is specified with array  notation
> >             (or  a compiler‐specific attribute), the array object to be
> >             considered for aliasing is a sub‐object of the original ar‐
> >             ray object, limited by the number  of  elements  specifiedr
> >             [1].
> 
> This is semantically problematic in the absence of something like N2906 
> (different declarations could use different numbers of elements),

Agree.  I think arrays should be fixed in C.  n2906 is a good step
towards that.  Thanks Martin!  :)

BTW, the author of n2529 didn't follow up, right?  I'd like that in, so
I'll prepare something after n2906 is merged.  Martin, would you mind
pinging me about it?

For what this [[alx::restrict]] proposal is concerned, I'd wait after
n2906 is merged for proposing that extension.

> and even 
> N2906 wouldn't help for the VLA case.

I'd basically propose that [3] or [n] means the same as [static 3] and
[static n], except for the nonnull implications of static.  Is there any
such paper?  I'm interested in presenting one for that.

Maybe it would also be interesting to wait after n2906 for that too.

> >      [1]  For the following prototype:
> > 
> >                  [[restrict(1)]] [[restrict(2)]]
> >                  void f(size_t n, int a[n], const int b[n]);
> 
> That declaration currently means
> 
>   void f(size_t n, int a[*], const int b[*]);

Yeah, that should be fixed in the standard.

I'll keep that extension of restrict out of a proposal until array
parameters are fixed in that regard.

> (that is, the expression giving a VLA size is ignored).  It's equivalent 
> to e.g.
> 
>   void f(size_t n, int a[n + foo()], const int b[n + bar()]);
> 
> where because the size expressions are never evaluated and there's no time 
> defined for evaluation, it's far from clear what anything talking about 
> them giving an array size would even mean.

Yup.

> I know that "noalias" was included in some C89 drafts but removed from the 
> final standard after objections.  Maybe someone who was around then could 
> explain what "noalias" was, what the problems with it were and how it 
> differs from "restrict", so we can make sure that any new proposals in 
> this area don't suffer from whatever the perceived deficiencies of 
> "noalias" were?

As I said in reply to Branden's response, it seems Dennis's concern was
that the noalias proposal was a qualifier, which admittedly makes little
sense (very much like the problems restrict has, but applied to the
pointee, which makes them much worse).

That in fact led me recently to think that an _Optional qualifier
(similar to Clang's _Nullable) as is being proposed at the moment in
n3222 is similarly DOA.  Those qualities of pointers are attributes,
which cannot be specified in the type system.

Have a lovely night!
Alex


> 
> -- 
> Joseph S. Myers
> josmyers@redhat.com


-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Joseph Myers]

On Fri, 26 Jul 2024, Alejandro Colomar via Gcc wrote:

> I don't see why it should not apply to void*.  memcpy(3) should get this
> attribute:
> 
> 	[[alx::restrict(1)]]
> 	[[alx::restrict(2)]]
> 	void *memcpy(void *dst, const void *src, size_t n);

That would disallow copying between disjoint subarrays within the same 
toplevel object (and there's no way to specify an array size for void *), 
which hardly seems right.

> BTW, the author of n2529 didn't follow up, right?  I'd like that in, so
> I'll prepare something after n2906 is merged.  Martin, would you mind
> pinging me about it?

See reflector message SC22WG14.18575, 17 Nov 2020 (the former convenor 
replying when I asked about just that paper).  As far as I know the author 
has not yet provided an updated version / asked for it to be added to a 
meeting agenda.

-- 
Joseph S. Myers
josmyers@redhat.com

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2188 bytes --]

Hi Joseph,

On Fri, Jul 26, 2024 at 08:30:33PM GMT, Joseph Myers wrote:
> On Fri, 26 Jul 2024, Alejandro Colomar via Gcc wrote:
> 
> > I don't see why it should not apply to void*.  memcpy(3) should get this
> > attribute:
> > 
> > 	[[alx::restrict(1)]]
> > 	[[alx::restrict(2)]]
> > 	void *memcpy(void *dst, const void *src, size_t n);
> 
> That would disallow copying between disjoint subarrays within the same 
> toplevel object (and there's no way to specify an array size for void *), 
> which hardly seems right.

Hmmm, I sometimes forget that ISO C is so painful about void.

Has WG14 discussed in the past about the GNU extension that defines
sizeof(void) == 1?

Maybe wording that also considers compiler-specific attributes and
extensions would allow for the following:

	[[gnu::access(write_only, 1, 3)]]
	[[gnu::access(read_only, 2, 3)]]
	[[alx::restrict(1)]]
	[[alx::restrict(2)]]
	void *memcpy(void *dst, const void *src, size_t n);

The GNU attribute specifies the number of elements of the subarrays, and
the GNU extension sizeof(void)==1 specifies the size of each element.
That gives us the size of the subarrays to be considered for the
restrictness.

So, ISO C wouldn't be allowed to mark malloc(3) as [[alx::restrict]]
(unless they add these GNU extensions), but GNU C could.

> > BTW, the author of n2529 didn't follow up, right?  I'd like that in, so
> > I'll prepare something after n2906 is merged.  Martin, would you mind
> > pinging me about it?
> 
> See reflector message SC22WG14.18575, 17 Nov 2020 (the former convenor 
> replying when I asked about just that paper).

Where can I find reflector messages?

>  As far as I know the author 
> has not yet provided an updated version / asked for it to be added to a 
> meeting agenda.

I think you mentioned that to me some time ago.  I guess I'll take over
then.  I'll ask for a number to propose _Nitems().

And another one to propose that [n] means the same as [static n] except
for the nonnull property of static.

Have a lovely night!
Alex

> 
> -- 
> Joseph S. Myers
> josmyers@redhat.com
> 
> 

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Joseph Myers]

On Fri, 26 Jul 2024, Alejandro Colomar via Gcc wrote:

> > See reflector message SC22WG14.18575, 17 Nov 2020 (the former convenor 
> > replying when I asked about just that paper).
> 
> Where can I find reflector messages?

https://www.open-std.org/jtc1/sc22/wg14/18575

> And another one to propose that [n] means the same as [static n] except
> for the nonnull property of static.

I'm not convinced that introducing extra undefined behavior for things 
that have been valid since C89 (which would be the effect of such a change 
for any code that passes a smaller array) is a good idea - the general 
mood is to *reduce* undefined behavior.

-- 
Joseph S. Myers
josmyers@redhat.com

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1570 bytes --]

On Fri, Jul 26, 2024 at 09:22:42PM GMT, Joseph Myers wrote:
> On Fri, 26 Jul 2024, Alejandro Colomar via Gcc wrote:
> 
> > > See reflector message SC22WG14.18575, 17 Nov 2020 (the former convenor 
> > > replying when I asked about just that paper).
> > 
> > Where can I find reflector messages?
> 
> https://www.open-std.org/jtc1/sc22/wg14/18575

Thanks!

> 
> > And another one to propose that [n] means the same as [static n] except
> > for the nonnull property of static.
> 
> I'm not convinced that introducing extra undefined behavior for things 
> that have been valid since C89 (which would be the effect of such a change 
> for any code that passes a smaller array) is a good idea - the general 
> mood is to *reduce* undefined behavior.

While [n] has always _officially_ meant the same as [], it has never
made any sense to write code like that.  Unofficially, it has always
meant the obvious thing.

Maybe if GNU C compilers (GCC and Clang) add it first as an extension,
adding diagnostics, it would help.

Does anyone know of any existing code that uses [n] for meaning anything
other than "n elements are available to the function"?

Functions that specify [n] most likely (definitely?) already mean that
n elements are accessed, and thus passing something different than n
elements results in UB one way or another.  Having the compiler enforce
that via diagnostics and UB is probably an improvement.

Cheers,
Alex

> 
> -- 
> Joseph S. Myers
> josmyers@redhat.com
> 

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Martin Uecker]

Am Freitag, dem 26.07.2024 um 23:49 +0200 schrieb Alejandro Colomar via Gcc:
> On Fri, Jul 26, 2024 at 09:22:42PM GMT, Joseph Myers wrote:
> > On Fri, 26 Jul 2024, Alejandro Colomar via Gcc wrote:
> > 
> > > > See reflector message SC22WG14.18575, 17 Nov 2020 (the former convenor 
> > > > replying when I asked about just that paper).
> > > 
> > > Where can I find reflector messages?
> > 
> > https://www.open-std.org/jtc1/sc22/wg14/18575
> 
> Thanks!
> 
> > 
> > > And another one to propose that [n] means the same as [static n] except
> > > for the nonnull property of static.
> > 
> > I'm not convinced that introducing extra undefined behavior for things 
> > that have been valid since C89 (which would be the effect of such a change 
> > for any code that passes a smaller array) is a good idea - the general 
> > mood is to *reduce* undefined behavior.
> 
> While [n] has always _officially_ meant the same as [], it has never
> made any sense to write code like that.  Unofficially, it has always
> meant the obvious thing.
> 
> Maybe if GNU C compilers (GCC and Clang) add it first as an extension,
> adding diagnostics, it would help.

Both GCC and Clang already have such diagnostics and/or run-time checks:

https://godbolt.org/z/MPnxqb9h7

Martin


> 
> Does anyone know of any existing code that uses [n] for meaning anything
> other than "n elements are available to the function"?
> 
> Functions that specify [n] most likely (definitely?) already mean that
> n elements are accessed, and thus passing something different than n
> elements results in UB one way or another.  Having the compiler enforce
> that via diagnostics and UB is probably an improvement.
> 
> Cheers,
> Alex
> 
> > 
> > -- 
> > Joseph S. Myers
> > josmyers@redhat.com
> > 
> 

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 545 bytes --]

On Sat, Jul 27, 2024 at 12:03:20AM GMT, Martin Uecker wrote:
> > Maybe if GNU C compilers (GCC and Clang) add it first as an extension,
> > adding diagnostics, it would help.
> 
> Both GCC and Clang already have such diagnostics and/or run-time checks:
> 
> https://godbolt.org/z/MPnxqb9h7

Hi Martin,

I guess that's prior art enough to make this UB in ISO C.  Is there any
paper for this already?  Does any of your paper cover that?  Should I
prepare one?

Have a lovely night!
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Martin Uecker]

Am Samstag, dem 27.07.2024 um 00:26 +0200 schrieb Alejandro Colomar:
> On Sat, Jul 27, 2024 at 12:03:20AM GMT, Martin Uecker wrote:
> > > Maybe if GNU C compilers (GCC and Clang) add it first as an extension,
> > > adding diagnostics, it would help.
> > 
> > Both GCC and Clang already have such diagnostics and/or run-time checks:
> > 
> > https://godbolt.org/z/MPnxqb9h7
> 
> Hi Martin,
> 
> I guess that's prior art enough to make this UB in ISO C.  Is there any
> paper for this already?  Does any of your paper cover that?  Should I
> prepare one?
> 

What do you mean by "this"?  Adding UB would likely see a lot
of opposition, even where this could enable run-time checks.  

N2906 would make 

int foo(char f[4]);
int foo(char f[5]);

a constraint violation (although having those types be incompatible
could also cause UB indirectly, this would not be its main effect).

So I think brining a new version of this paper forward would be
a possible next step, addressing the issues raised in the past.

Martin

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1769 bytes --]

Hi Martin,

On Sat, Jul 27, 2024 at 12:59:34AM GMT, Martin Uecker wrote:
> Am Samstag, dem 27.07.2024 um 00:26 +0200 schrieb Alejandro Colomar:
> > On Sat, Jul 27, 2024 at 12:03:20AM GMT, Martin Uecker wrote:
> > > > Maybe if GNU C compilers (GCC and Clang) add it first as an extension,
> > > > adding diagnostics, it would help.
> > > 
> > > Both GCC and Clang already have such diagnostics and/or run-time checks:
> > > 
> > > https://godbolt.org/z/MPnxqb9h7
> > 
> > Hi Martin,
> > 
> > I guess that's prior art enough to make this UB in ISO C.  Is there any
> > paper for this already?  Does any of your paper cover that?  Should I
> > prepare one?
> > 
> 
> What do you mean by "this"?

Adding UB.

>  Adding UB would likely see a lot
> of opposition,

But UB allows for safer code.  It's the lack of UB what reduces the
quality of diagnostics, which results in worse code.  I understand it
will see opposition, so we better wait for the path to be prepared
(i.e., n2906 already merged before presenting a paper), but once that's
done, I'd try to add UB.

> even where this could enable run-time checks.  

(And build-time too.)

> N2906 would make 
> 
> int foo(char f[4]);
> int foo(char f[5]);
> 
> a constraint violation (although having those types be incompatible
> could also cause UB indirectly, this would not be its main effect).
> 
> So I think brining a new version of this paper forward would be
> a possible next step, addressing the issues raised in the past.

Yeah, that would be a good next step.  And when the array type is part
of the function type, it'll be easier to convince that [n] can only mean
[n].

Have a lovely day!
Alex

> Martin
> 

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]