nolibc patches, still possible for 6.5 ?

nolibc patches, still possible for 6.5 ?

[Willy Tarreau]

Hello Paul,

Thomas and Zhangjin have provided significant nolibc cleanups, and
fixes, as well as preparation work to later support riscv32.

These consist in the following main series:
  - generalization of stackprotector to other archs that were not
    previously supported (riscv, mips, loongarch, arm, arm64)

  - general cleanups of the makefile, test report output, deduplication
    of certain tests

  - slightly better compliance of some tests performed on certain syscalls
    (e.g. no longer pass (void*)1 to gettimeofday() since glibc hates it).

  - add support for nanoseconds in stat() and statx()

  - fixes for some syscalls (e.g. ppoll() has 5 arguments not 4)

  - fixes around limits.h and  INT_MAX / INT_FAST64_MAX

I rebased the whole series on top of your latest dev branch (d19a9ca3d5)
and it works fine for all archs.

I don't know if you're still planning on merging new stuff in this area
for 6.5 or not (since I know that it involves new series of tests on your
side as well), but given that Zhangjin will engage into deeper changes
later for riscv32 that will likely imply to update more syscalls to use
the time64 ones, I would prefer to split the cleanups from the hard stuff,
but I'll let you judge based on the current state of what's pending for
6.5.

In any case I'm putting all this here for now (not for merge yet):

   git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20230604-nolibc-rv32+stkp6

I'd like Thomas and Zhangjin to perform a last check to confirm they're
OK with this final integration.

Thanks!
Willy

Re: nolibc patches, still possible for 6.5 ?

[Paul E. McKenney]

On Sun, Jun 04, 2023 at 03:20:11PM +0200, Willy Tarreau wrote:
> Hello Paul,
> 
> Thomas and Zhangjin have provided significant nolibc cleanups, and
> fixes, as well as preparation work to later support riscv32.
> 
> These consist in the following main series:
>   - generalization of stackprotector to other archs that were not
>     previously supported (riscv, mips, loongarch, arm, arm64)
> 
>   - general cleanups of the makefile, test report output, deduplication
>     of certain tests
> 
>   - slightly better compliance of some tests performed on certain syscalls
>     (e.g. no longer pass (void*)1 to gettimeofday() since glibc hates it).
> 
>   - add support for nanoseconds in stat() and statx()
> 
>   - fixes for some syscalls (e.g. ppoll() has 5 arguments not 4)
> 
>   - fixes around limits.h and  INT_MAX / INT_FAST64_MAX
> 
> I rebased the whole series on top of your latest dev branch (d19a9ca3d5)
> and it works fine for all archs.
> 
> I don't know if you're still planning on merging new stuff in this area
> for 6.5 or not (since I know that it involves new series of tests on your
> side as well), but given that Zhangjin will engage into deeper changes
> later for riscv32 that will likely imply to update more syscalls to use
> the time64 ones, I would prefer to split the cleanups from the hard stuff,
> but I'll let you judge based on the current state of what's pending for
> 6.5.
> 
> In any case I'm putting all this here for now (not for merge yet):
> 
>    git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20230604-nolibc-rv32+stkp6
> 
> I'd like Thomas and Zhangjin to perform a last check to confirm they're
> OK with this final integration.

Given that the testing converges by the end of this week, I can't see
any reason why these cannot make v6.5.  (There were some kernel test
robot complaints as well, valid or not I am not sure.)

							Thanx, Paul

Re: nolibc patches, still possible for 6.5 ?

[Willy Tarreau]

On Sun, Jun 04, 2023 at 03:57:54PM -0700, Paul E. McKenney wrote:
> On Sun, Jun 04, 2023 at 03:20:11PM +0200, Willy Tarreau wrote:
> > Hello Paul,
> > 
> > Thomas and Zhangjin have provided significant nolibc cleanups, and
> > fixes, as well as preparation work to later support riscv32.
> > 
> > These consist in the following main series:
> >   - generalization of stackprotector to other archs that were not
> >     previously supported (riscv, mips, loongarch, arm, arm64)
> > 
> >   - general cleanups of the makefile, test report output, deduplication
> >     of certain tests
> > 
> >   - slightly better compliance of some tests performed on certain syscalls
> >     (e.g. no longer pass (void*)1 to gettimeofday() since glibc hates it).
> > 
> >   - add support for nanoseconds in stat() and statx()
> > 
> >   - fixes for some syscalls (e.g. ppoll() has 5 arguments not 4)
> > 
> >   - fixes around limits.h and  INT_MAX / INT_FAST64_MAX
> > 
> > I rebased the whole series on top of your latest dev branch (d19a9ca3d5)
> > and it works fine for all archs.
> > 
> > I don't know if you're still planning on merging new stuff in this area
> > for 6.5 or not (since I know that it involves new series of tests on your
> > side as well), but given that Zhangjin will engage into deeper changes
> > later for riscv32 that will likely imply to update more syscalls to use
> > the time64 ones, I would prefer to split the cleanups from the hard stuff,
> > but I'll let you judge based on the current state of what's pending for
> > 6.5.
> > 
> > In any case I'm putting all this here for now (not for merge yet):
> > 
> >    git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20230604-nolibc-rv32+stkp6
> > 
> > I'd like Thomas and Zhangjin to perform a last check to confirm they're
> > OK with this final integration.
> 
> Given that the testing converges by the end of this week, I can't see
> any reason why these cannot make v6.5.

Perfect, thank you!

> (There were some kernel test
> robot complaints as well, valid or not I am not sure.)

You mean in relation with nolibc stuff (or nolibc-test) or something
totally different ?

Best regards,
Willy

Re: nolibc patches, still possible for 6.5 ?

[Paul E. McKenney]

On Mon, Jun 05, 2023 at 05:26:19AM +0200, Willy Tarreau wrote:
> On Sun, Jun 04, 2023 at 03:57:54PM -0700, Paul E. McKenney wrote:
> > On Sun, Jun 04, 2023 at 03:20:11PM +0200, Willy Tarreau wrote:
> > > Hello Paul,
> > > 
> > > Thomas and Zhangjin have provided significant nolibc cleanups, and
> > > fixes, as well as preparation work to later support riscv32.
> > > 
> > > These consist in the following main series:
> > >   - generalization of stackprotector to other archs that were not
> > >     previously supported (riscv, mips, loongarch, arm, arm64)
> > > 
> > >   - general cleanups of the makefile, test report output, deduplication
> > >     of certain tests
> > > 
> > >   - slightly better compliance of some tests performed on certain syscalls
> > >     (e.g. no longer pass (void*)1 to gettimeofday() since glibc hates it).
> > > 
> > >   - add support for nanoseconds in stat() and statx()
> > > 
> > >   - fixes for some syscalls (e.g. ppoll() has 5 arguments not 4)
> > > 
> > >   - fixes around limits.h and  INT_MAX / INT_FAST64_MAX
> > > 
> > > I rebased the whole series on top of your latest dev branch (d19a9ca3d5)
> > > and it works fine for all archs.
> > > 
> > > I don't know if you're still planning on merging new stuff in this area
> > > for 6.5 or not (since I know that it involves new series of tests on your
> > > side as well), but given that Zhangjin will engage into deeper changes
> > > later for riscv32 that will likely imply to update more syscalls to use
> > > the time64 ones, I would prefer to split the cleanups from the hard stuff,
> > > but I'll let you judge based on the current state of what's pending for
> > > 6.5.
> > > 
> > > In any case I'm putting all this here for now (not for merge yet):
> > > 
> > >    git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20230604-nolibc-rv32+stkp6
> > > 
> > > I'd like Thomas and Zhangjin to perform a last check to confirm they're
> > > OK with this final integration.
> > 
> > Given that the testing converges by the end of this week, I can't see
> > any reason why these cannot make v6.5.
> 
> Perfect, thank you!
> 
> > (There were some kernel test
> > robot complaints as well, valid or not I am not sure.)
> 
> You mean in relation with nolibc stuff (or nolibc-test) or something
> totally different ?

Apologies, this was me being confused and failing to look closely.

The complaints were not about nolibc, but rather about my patches that
they were on top of.  Not your problem!

And please let me know when the next batch from your tree are ready to go.
(You might have been saying that they were in your recent emails, but
I thought I should double-check.)

							Thanx, Paul

Re: nolibc patches, still possible for 6.5 ?

[Willy Tarreau]

On Wed, Jun 07, 2023 at 02:03:17PM -0700, Paul E. McKenney wrote:
> > > (There were some kernel test
> > > robot complaints as well, valid or not I am not sure.)
> > 
> > You mean in relation with nolibc stuff (or nolibc-test) or something
> > totally different ?
> 
> Apologies, this was me being confused and failing to look closely.
> 
> The complaints were not about nolibc, but rather about my patches that
> they were on top of.  Not your problem!

Ah no problem :-)

> And please let me know when the next batch from your tree are ready to go.
> (You might have been saying that they were in your recent emails, but
> I thought I should double-check.)

No pb, I just sent it while you were writing and our emails have crossed :-)

In short, it's ready now with branch 20230606-nolibc-rv32+stkp7a but if you
need any more info (more detailed summary, a public repost of the whole
series etc), just let me know. And I faced 2 kernel build errors on s390x
and riscv about rcu_task something, though you might be interested :-/

Thanks,
Willy

Re: nolibc patches, still possible for 6.5 ?

[Paul E. McKenney]

On Wed, Jun 07, 2023 at 11:22:54PM +0200, Willy Tarreau wrote:
> On Wed, Jun 07, 2023 at 02:03:17PM -0700, Paul E. McKenney wrote:
> > > > (There were some kernel test
> > > > robot complaints as well, valid or not I am not sure.)
> > > 
> > > You mean in relation with nolibc stuff (or nolibc-test) or something
> > > totally different ?
> > 
> > Apologies, this was me being confused and failing to look closely.
> > 
> > The complaints were not about nolibc, but rather about my patches that
> > they were on top of.  Not your problem!
> 
> Ah no problem :-)
> 
> > And please let me know when the next batch from your tree are ready to go.
> > (You might have been saying that they were in your recent emails, but
> > I thought I should double-check.)
> 
> No pb, I just sent it while you were writing and our emails have crossed :-)
> 
> In short, it's ready now with branch 20230606-nolibc-rv32+stkp7a but if you
> need any more info (more detailed summary, a public repost of the whole
> series etc), just let me know. And I faced 2 kernel build errors on s390x
> and riscv about rcu_task something, though you might be interested :-/

And I pulled them in and got this from "make run":

138 test(s) passed, 0 skipped, 0 failed.[    2.416045] reboot: Power down

And this from "make run-user":

136 test(s) passed, 2 skipped, 0 failed. See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out

And run.out looks as it has before, so all looks good at this end.

Thus, unless you tell me otherwise, I will move these to my nolibc branch
for the upcoming merge window.

							Thanx, Paul

Re: nolibc patches, still possible for 6.5 ?

[Willy Tarreau]

On Wed, Jun 07, 2023 at 04:06:37PM -0700, Paul E. McKenney wrote:
> On Wed, Jun 07, 2023 at 11:22:54PM +0200, Willy Tarreau wrote:
> > On Wed, Jun 07, 2023 at 02:03:17PM -0700, Paul E. McKenney wrote:
> > > > > (There were some kernel test
> > > > > robot complaints as well, valid or not I am not sure.)
> > > > 
> > > > You mean in relation with nolibc stuff (or nolibc-test) or something
> > > > totally different ?
> > > 
> > > Apologies, this was me being confused and failing to look closely.
> > > 
> > > The complaints were not about nolibc, but rather about my patches that
> > > they were on top of.  Not your problem!
> > 
> > Ah no problem :-)
> > 
> > > And please let me know when the next batch from your tree are ready to go.
> > > (You might have been saying that they were in your recent emails, but
> > > I thought I should double-check.)
> > 
> > No pb, I just sent it while you were writing and our emails have crossed :-)
> > 
> > In short, it's ready now with branch 20230606-nolibc-rv32+stkp7a but if you
> > need any more info (more detailed summary, a public repost of the whole
> > series etc), just let me know. And I faced 2 kernel build errors on s390x
> > and riscv about rcu_task something, though you might be interested :-/
> 
> And I pulled them in and got this from "make run":
> 
> 138 test(s) passed, 0 skipped, 0 failed.[    2.416045] reboot: Power down
> 
> And this from "make run-user":
> 
> 136 test(s) passed, 2 skipped, 0 failed. See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out
> 
> And run.out looks as it has before, so all looks good at this end.
> 
> Thus, unless you tell me otherwise, I will move these to my nolibc branch
> for the upcoming merge window.

Perfect, thank you very much Paul!

Willy

Re: nolibc patches, still possible for 6.5 ?

[Paul E. McKenney]

On Thu, Jun 08, 2023 at 06:32:04AM +0200, Willy Tarreau wrote:
> On Wed, Jun 07, 2023 at 04:06:37PM -0700, Paul E. McKenney wrote:
> > On Wed, Jun 07, 2023 at 11:22:54PM +0200, Willy Tarreau wrote:
> > > On Wed, Jun 07, 2023 at 02:03:17PM -0700, Paul E. McKenney wrote:
> > > > > > (There were some kernel test
> > > > > > robot complaints as well, valid or not I am not sure.)
> > > > > 
> > > > > You mean in relation with nolibc stuff (or nolibc-test) or something
> > > > > totally different ?
> > > > 
> > > > Apologies, this was me being confused and failing to look closely.
> > > > 
> > > > The complaints were not about nolibc, but rather about my patches that
> > > > they were on top of.  Not your problem!
> > > 
> > > Ah no problem :-)
> > > 
> > > > And please let me know when the next batch from your tree are ready to go.
> > > > (You might have been saying that they were in your recent emails, but
> > > > I thought I should double-check.)
> > > 
> > > No pb, I just sent it while you were writing and our emails have crossed :-)
> > > 
> > > In short, it's ready now with branch 20230606-nolibc-rv32+stkp7a but if you
> > > need any more info (more detailed summary, a public repost of the whole
> > > series etc), just let me know. And I faced 2 kernel build errors on s390x
> > > and riscv about rcu_task something, though you might be interested :-/
> > 
> > And I pulled them in and got this from "make run":
> > 
> > 138 test(s) passed, 0 skipped, 0 failed.[    2.416045] reboot: Power down
> > 
> > And this from "make run-user":
> > 
> > 136 test(s) passed, 2 skipped, 0 failed. See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out
> > 
> > And run.out looks as it has before, so all looks good at this end.
> > 
> > Thus, unless you tell me otherwise, I will move these to my nolibc branch
> > for the upcoming merge window.
> 
> Perfect, thank you very much Paul!

And done!

							Thanx, Paul

Re: nolibc patches, still possible for 6.5 ?

[Willy Tarreau]

Hello Paul,

On Sun, Jun 04, 2023 at 03:57:54PM -0700, Paul E. McKenney wrote:
> On Sun, Jun 04, 2023 at 03:20:11PM +0200, Willy Tarreau wrote:
> > Hello Paul,
> > 
> > Thomas and Zhangjin have provided significant nolibc cleanups, and
> > fixes, as well as preparation work to later support riscv32.
> > 
> > These consist in the following main series:
> >   - generalization of stackprotector to other archs that were not
> >     previously supported (riscv, mips, loongarch, arm, arm64)
> > 
> >   - general cleanups of the makefile, test report output, deduplication
> >     of certain tests
> > 
> >   - slightly better compliance of some tests performed on certain syscalls
> >     (e.g. no longer pass (void*)1 to gettimeofday() since glibc hates it).
> > 
> >   - add support for nanoseconds in stat() and statx()
> > 
> >   - fixes for some syscalls (e.g. ppoll() has 5 arguments not 4)
> > 
> >   - fixes around limits.h and  INT_MAX / INT_FAST64_MAX
> > 
> > I rebased the whole series on top of your latest dev branch (d19a9ca3d5)
> > and it works fine for all archs.
> > 
> > I don't know if you're still planning on merging new stuff in this area
> > for 6.5 or not (since I know that it involves new series of tests on your
> > side as well), but given that Zhangjin will engage into deeper changes
> > later for riscv32 that will likely imply to update more syscalls to use
> > the time64 ones, I would prefer to split the cleanups from the hard stuff,
> > but I'll let you judge based on the current state of what's pending for
> > 6.5.
> > 
> > In any case I'm putting all this here for now (not for merge yet):
> > 
> >    git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20230604-nolibc-rv32+stkp6
> > 
> > I'd like Thomas and Zhangjin to perform a last check to confirm they're
> > OK with this final integration.
> 
> Given that the testing converges by the end of this week, I can't see
> any reason why these cannot make v6.5.  (There were some kernel test
> robot complaints as well, valid or not I am not sure.)

After Thomas' and Zhangjin's reviews and checks, I could run a mostly
complete check:
  - arm64, i386, x86_64 show 100% success
  - arm, mips: 100% success, stackprotector skipped
  - s390x, riscv64: run-user OK, kernel build fails (see below)
  - loongarch: build OK, just not executed (need to upgrade my qemu
    and I hate doing it late when some tests results are needed)

Regarding the build failure affecting s390x and riscv64, it's a regular
kernel resulting from "make defconfig". For both archs, I'm getting this
failure:

  In file included from kernel/rcu/update.c:649:
  kernel/rcu/tasks.h: In function 'get_rcu_tasks_gp_kthread':
    CC      fs/kernfs/dir.o
    CC      security/bpf/hooks.o
  kernel/rcu/tasks.h:1939:16: error: 'rcu_tasks' undeclared (first use in this function)
   1939 |         return rcu_tasks.kthread_ptr;
        |                ^~~~~~~~~
  kernel/rcu/tasks.h:1939:16: note: each undeclared identifier is reported only once for each function it appears in
  kernel/rcu/tasks.h:1940:1: error: control reaches end of non-void function [-Werror=return-type]
   1940 | }
        | ^
  cc1: some warnings being treated as errors

I rebased the branch on top of 6.4-rc5 and got the same. I'm building
with gcc-11.3.0 from kernel.org. I'm not sure whether this comes from
my build environment or recent changes to the kernel, but I'm sure I
haven't seen that error during 6.3-rc cycle. However, given that
Zhangjin seems to have successfully built it for riscv, there might
be something odd on my side.

Given that this build issue is not dependent on the selftest, I'm fine
with the branch getting merged as-is, and can provide feedback on this
build error if needed:

  git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20230606-nolibc-rv32+stkp7a

Just let me know if you prefer that I resend the whole series or need
more info etc, as usual.

Thank you!
Willy

Re: nolibc patches, still possible for 6.5 ?

[Paul E. McKenney]

On Wed, Jun 07, 2023 at 11:19:43PM +0200, Willy Tarreau wrote:
> Hello Paul,
> 
> On Sun, Jun 04, 2023 at 03:57:54PM -0700, Paul E. McKenney wrote:
> > On Sun, Jun 04, 2023 at 03:20:11PM +0200, Willy Tarreau wrote:
> > > Hello Paul,
> > > 
> > > Thomas and Zhangjin have provided significant nolibc cleanups, and
> > > fixes, as well as preparation work to later support riscv32.
> > > 
> > > These consist in the following main series:
> > >   - generalization of stackprotector to other archs that were not
> > >     previously supported (riscv, mips, loongarch, arm, arm64)
> > > 
> > >   - general cleanups of the makefile, test report output, deduplication
> > >     of certain tests
> > > 
> > >   - slightly better compliance of some tests performed on certain syscalls
> > >     (e.g. no longer pass (void*)1 to gettimeofday() since glibc hates it).
> > > 
> > >   - add support for nanoseconds in stat() and statx()
> > > 
> > >   - fixes for some syscalls (e.g. ppoll() has 5 arguments not 4)
> > > 
> > >   - fixes around limits.h and  INT_MAX / INT_FAST64_MAX
> > > 
> > > I rebased the whole series on top of your latest dev branch (d19a9ca3d5)
> > > and it works fine for all archs.
> > > 
> > > I don't know if you're still planning on merging new stuff in this area
> > > for 6.5 or not (since I know that it involves new series of tests on your
> > > side as well), but given that Zhangjin will engage into deeper changes
> > > later for riscv32 that will likely imply to update more syscalls to use
> > > the time64 ones, I would prefer to split the cleanups from the hard stuff,
> > > but I'll let you judge based on the current state of what's pending for
> > > 6.5.
> > > 
> > > In any case I'm putting all this here for now (not for merge yet):
> > > 
> > >    git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20230604-nolibc-rv32+stkp6
> > > 
> > > I'd like Thomas and Zhangjin to perform a last check to confirm they're
> > > OK with this final integration.
> > 
> > Given that the testing converges by the end of this week, I can't see
> > any reason why these cannot make v6.5.  (There were some kernel test
> > robot complaints as well, valid or not I am not sure.)
> 
> After Thomas' and Zhangjin's reviews and checks, I could run a mostly
> complete check:
>   - arm64, i386, x86_64 show 100% success
>   - arm, mips: 100% success, stackprotector skipped
>   - s390x, riscv64: run-user OK, kernel build fails (see below)
>   - loongarch: build OK, just not executed (need to upgrade my qemu
>     and I hate doing it late when some tests results are needed)

Very good!

> Regarding the build failure affecting s390x and riscv64, it's a regular
> kernel resulting from "make defconfig". For both archs, I'm getting this
> failure:
> 
>   In file included from kernel/rcu/update.c:649:
>   kernel/rcu/tasks.h: In function 'get_rcu_tasks_gp_kthread':
>     CC      fs/kernfs/dir.o
>     CC      security/bpf/hooks.o
>   kernel/rcu/tasks.h:1939:16: error: 'rcu_tasks' undeclared (first use in this function)
>    1939 |         return rcu_tasks.kthread_ptr;
>         |                ^~~~~~~~~
>   kernel/rcu/tasks.h:1939:16: note: each undeclared identifier is reported only once for each function it appears in
>   kernel/rcu/tasks.h:1940:1: error: control reaches end of non-void function [-Werror=return-type]
>    1940 | }
>         | ^
>   cc1: some warnings being treated as errors
> 
> I rebased the branch on top of 6.4-rc5 and got the same. I'm building
> with gcc-11.3.0 from kernel.org. I'm not sure whether this comes from
> my build environment or recent changes to the kernel, but I'm sure I
> haven't seen that error during 6.3-rc cycle. However, given that
> Zhangjin seems to have successfully built it for riscv, there might
> be something odd on my side.

That line of code is in rcu/dev but not in mainline yet.  In fact, it
is not yet in -next.

But it is a bug.  One that my Kconfig laziness hid from me.  Easy fix,
but it is clearly time for me to stop being lazy about that part of the
Kconfig setup.  :-/

So thank you for reporting it!

> Given that this build issue is not dependent on the selftest, I'm fine
> with the branch getting merged as-is, and can provide feedback on this
> build error if needed:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20230606-nolibc-rv32+stkp7a
> 
> Just let me know if you prefer that I resend the whole series or need
> more info etc, as usual.

I will pull it from your tree, test it, and if all goes well, rebase it
on my existing nolibc stack.

Longer term, both to avoid you having to deal with RCU bugs and to make
it easier to have multiple administrative nolibc maintainers, it might
work better for you to base your stack on vX.y-rc1.  That way, I could
just pull directly from your tree.

This works because you buffer up the commits and test them, which
makes it completely reasonable for me to simply pull your new stack
and merge them in.  Which also means that if there are multiple nolibc
administrative maintainers, we have exactly the same set of nolibc
commits in our respective trees, right down to the SHA-1 hashes.

This approach is used a lot, for example, back when my RCU patches
went through Ingo Molnar, he pulled from my tree so that mainline's RCU
patches were identical to mine, again, right down to the SHA-1 hashes.

This is something to think about for some upcoming cycle, given that
we are already pretty much set up for the upcoming merge window.

Your choice, either way works for me.

							Thanx, Paul

Re: nolibc patches, still possible for 6.5 ?

[Willy Tarreau]

On Wed, Jun 07, 2023 at 03:58:01PM -0700, Paul E. McKenney wrote:
> > Regarding the build failure affecting s390x and riscv64, it's a regular
> > kernel resulting from "make defconfig". For both archs, I'm getting this
> > failure:
> > 
> >   In file included from kernel/rcu/update.c:649:
> >   kernel/rcu/tasks.h: In function 'get_rcu_tasks_gp_kthread':
> >     CC      fs/kernfs/dir.o
> >     CC      security/bpf/hooks.o
> >   kernel/rcu/tasks.h:1939:16: error: 'rcu_tasks' undeclared (first use in this function)
> >    1939 |         return rcu_tasks.kthread_ptr;
> >         |                ^~~~~~~~~
> >   kernel/rcu/tasks.h:1939:16: note: each undeclared identifier is reported only once for each function it appears in
> >   kernel/rcu/tasks.h:1940:1: error: control reaches end of non-void function [-Werror=return-type]
> >    1940 | }
> >         | ^
> >   cc1: some warnings being treated as errors
> > 
> > I rebased the branch on top of 6.4-rc5 and got the same. I'm building
> > with gcc-11.3.0 from kernel.org. I'm not sure whether this comes from
> > my build environment or recent changes to the kernel, but I'm sure I
> > haven't seen that error during 6.3-rc cycle. However, given that
> > Zhangjin seems to have successfully built it for riscv, there might
> > be something odd on my side.
> 
> That line of code is in rcu/dev but not in mainline yet.  In fact, it
> is not yet in -next.
> 
> But it is a bug.  One that my Kconfig laziness hid from me.  Easy fix,
> but it is clearly time for me to stop being lazy about that part of the
> Kconfig setup.  :-/
> 
> So thank you for reporting it!

Great, I'm happy that it cuold be used to spot a real bug ;-)

> Longer term, both to avoid you having to deal with RCU bugs and to make
> it easier to have multiple administrative nolibc maintainers, it might
> work better for you to base your stack on vX.y-rc1.  That way, I could
> just pull directly from your tree.
(...)
> This is something to think about for some upcoming cycle, given that
> we are already pretty much set up for the upcoming merge window.

Yes I think it makes sense now. Initially tiny changes had implications
on rcutorture and needed to be properly sequenced but that's no longer
the case and we can indeed simplify this. And it will force us to gather
all patches in one single series, which is also easier to review/discuss.

So that works for me.

Thanks!
Willy

Re: nolibc patches, still possible for 6.5 ?

[Paul E. McKenney]

On Thu, Jun 08, 2023 at 06:36:33AM +0200, Willy Tarreau wrote:
> On Wed, Jun 07, 2023 at 03:58:01PM -0700, Paul E. McKenney wrote:
> > > Regarding the build failure affecting s390x and riscv64, it's a regular
> > > kernel resulting from "make defconfig". For both archs, I'm getting this
> > > failure:
> > > 
> > >   In file included from kernel/rcu/update.c:649:
> > >   kernel/rcu/tasks.h: In function 'get_rcu_tasks_gp_kthread':
> > >     CC      fs/kernfs/dir.o
> > >     CC      security/bpf/hooks.o
> > >   kernel/rcu/tasks.h:1939:16: error: 'rcu_tasks' undeclared (first use in this function)
> > >    1939 |         return rcu_tasks.kthread_ptr;
> > >         |                ^~~~~~~~~
> > >   kernel/rcu/tasks.h:1939:16: note: each undeclared identifier is reported only once for each function it appears in
> > >   kernel/rcu/tasks.h:1940:1: error: control reaches end of non-void function [-Werror=return-type]
> > >    1940 | }
> > >         | ^
> > >   cc1: some warnings being treated as errors
> > > 
> > > I rebased the branch on top of 6.4-rc5 and got the same. I'm building
> > > with gcc-11.3.0 from kernel.org. I'm not sure whether this comes from
> > > my build environment or recent changes to the kernel, but I'm sure I
> > > haven't seen that error during 6.3-rc cycle. However, given that
> > > Zhangjin seems to have successfully built it for riscv, there might
> > > be something odd on my side.
> > 
> > That line of code is in rcu/dev but not in mainline yet.  In fact, it
> > is not yet in -next.
> > 
> > But it is a bug.  One that my Kconfig laziness hid from me.  Easy fix,
> > but it is clearly time for me to stop being lazy about that part of the
> > Kconfig setup.  :-/
> > 
> > So thank you for reporting it!
> 
> Great, I'm happy that it cuold be used to spot a real bug ;-)
> 
> > Longer term, both to avoid you having to deal with RCU bugs and to make
> > it easier to have multiple administrative nolibc maintainers, it might
> > work better for you to base your stack on vX.y-rc1.  That way, I could
> > just pull directly from your tree.
> (...)
> > This is something to think about for some upcoming cycle, given that
> > we are already pretty much set up for the upcoming merge window.
> 
> Yes I think it makes sense now. Initially tiny changes had implications
> on rcutorture and needed to be properly sequenced but that's no longer
> the case and we can indeed simplify this. And it will force us to gather
> all patches in one single series, which is also easier to review/discuss.
> 
> So that works for me.

Very good!

Actually, in theory, I could replace my current stack with a direct
pull of your stack and get a head start on this process.  Let me see
how I feel about making this switch on Monday.  ;-)

							Thanx, Paul

Re: nolibc patches, still possible for 6.5 ?

[Zhangjin Wu]

Hi, Willy

> Hello Paul,
> 
> On Sun, Jun 04, 2023 at 03:57:54PM -0700, Paul E. McKenney wrote:
> > On Sun, Jun 04, 2023 at 03:20:11PM +0200, Willy Tarreau wrote:
> > > Hello Paul,
> > > 
> > > Thomas and Zhangjin have provided significant nolibc cleanups, and
> > > fixes, as well as preparation work to later support riscv32.
> > > 
> (...)
> > 
> > Given that the testing converges by the end of this week, I can't see
> > any reason why these cannot make v6.5.  (There were some kernel test
> > robot complaints as well, valid or not I am not sure.)
> 
> After Thomas' and Zhangjin's reviews and checks, I could run a mostly
> complete check:
>   - arm64, i386, x86_64 show 100% success
>   - arm, mips: 100% success, stackprotector skipped
>   - s390x, riscv64: run-user OK, kernel build fails (see below)
>   - loongarch: build OK, just not executed (need to upgrade my qemu
>     and I hate doing it late when some tests results are needed)
> 
> Regarding the build failure affecting s390x and riscv64, it's a regular
> kernel resulting from "make defconfig". For both archs, I'm getting this
> failure:
> 
>   In file included from kernel/rcu/update.c:649:
>   kernel/rcu/tasks.h: In function 'get_rcu_tasks_gp_kthread':
>     CC      fs/kernfs/dir.o
>     CC      security/bpf/hooks.o
>   kernel/rcu/tasks.h:1939:16: error: 'rcu_tasks' undeclared (first use in this function)
>    1939 |         return rcu_tasks.kthread_ptr;
>         |                ^~~~~~~~~
>   kernel/rcu/tasks.h:1939:16: note: each undeclared identifier is reported only once for each function it appears in
>   kernel/rcu/tasks.h:1940:1: error: control reaches end of non-void function [-Werror=return-type]
>    1940 | }
>         | ^
>   cc1: some warnings being treated as errors
> 
> I rebased the branch on top of 6.4-rc5 and got the same. I'm building
> with gcc-11.3.0 from kernel.org. I'm not sure whether this comes from
> my build environment or recent changes to the kernel, but I'm sure I
> haven't seen that error during 6.3-rc cycle. However, given that
> Zhangjin seems to have successfully built it for riscv, there might
> be something odd on my side.
>

Sorry, The reason is that to speed up the kernel build+nolibc tests, a
local small config was customized for the first report, it may not
trigger the above issue, in the later report about the v4 rv32 compile
support [1], I did test 'run' target with the default defconfig and
reported the same failure as bove, this is the note I pasted there:

  ---
  Did compile test for aarch64, rv32 and rv64, include run-user and run.

  Note, this is required with the default config from the
  20230606-nolibc-rv32+stkp7a branch of [5]:

    diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h
    index ce02bb09651b..72bd8fe0cad6 100644
    --- a/kernel/rcu/tasks.h
    +++ b/kernel/rcu/tasks.h
    @@ -1934,11 +1934,13 @@ void show_rcu_tasks_gp_kthreads(void)
     }
     #endif /* #ifndef CONFIG_TINY_RCU */

    +#ifdef CONFIG_TASKS_RCU
     struct task_struct *get_rcu_tasks_gp_kthread(void)
     {
            return rcu_tasks.kthread_ptr;
     }
     EXPORT_SYMBOL_GPL(get_rcu_tasks_gp_kthread);
    +#endif

     #ifdef CONFIG_PROVE_RCU
     struct rcu_tasks_test_desc {
  ---

I have seen Paul have known the cause of the above issue in your later
discussion and you found a better solution to avoid such failures,
congrats!

[1]: https://lore.kernel.org/linux-riscv/cover.1686128703.git.falcon@tinylab.org/T/#t

Best regards,
Zhangjin

> Given that this build issue is not dependent on the selftest, I'm fine
> with the branch getting merged as-is, and can provide feedback on this
> build error if needed:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20230606-nolibc-rv32+stkp7a
> 
> Just let me know if you prefer that I resend the whole series or need
> more info etc, as usual.
> 
> Thank you!
> Willy

Re: nolibc patches, still possible for 6.5 ?

[Willy Tarreau]

Hi Zhangjin,

On Thu, Jun 08, 2023 at 04:13:37PM +0800, Zhangjin Wu wrote:
> > I rebased the branch on top of 6.4-rc5 and got the same. I'm building
> > with gcc-11.3.0 from kernel.org. I'm not sure whether this comes from
> > my build environment or recent changes to the kernel, but I'm sure I
> > haven't seen that error during 6.3-rc cycle. However, given that
> > Zhangjin seems to have successfully built it for riscv, there might
> > be something odd on my side.
> >
> 
> Sorry, The reason is that to speed up the kernel build+nolibc tests, a
> local small config was customized for the first report,

You don't have to be sorry for this! The more diversity in configs, the
more bugs are discovered before they hit users.

> it may not
> trigger the above issue, in the later report about the v4 rv32 compile
> support [1], I did test 'run' target with the default defconfig and
> reported the same failure as bove, this is the note I pasted there:
> 
>   ---
>   Did compile test for aarch64, rv32 and rv64, include run-user and run.
> 
>   Note, this is required with the default config from the
>   20230606-nolibc-rv32+stkp7a branch of [5]:
> 
>     diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h
>     index ce02bb09651b..72bd8fe0cad6 100644
>     --- a/kernel/rcu/tasks.h
>     +++ b/kernel/rcu/tasks.h
>     @@ -1934,11 +1934,13 @@ void show_rcu_tasks_gp_kthreads(void)
>      }
>      #endif /* #ifndef CONFIG_TINY_RCU */
> 
>     +#ifdef CONFIG_TASKS_RCU
>      struct task_struct *get_rcu_tasks_gp_kthread(void)
>      {
>             return rcu_tasks.kthread_ptr;
>      }
>      EXPORT_SYMBOL_GPL(get_rcu_tasks_gp_kthread);
>     +#endif
> 
>      #ifdef CONFIG_PROVE_RCU
>      struct rcu_tasks_test_desc {
>   ---

Indeed, I initially didn't make the connection regarding this, because
you mentioned the fix but not the problem, so I didn't remember to take
any particular care about this.

> I have seen Paul have known the cause of the above issue in your later
> discussion and you found a better solution to avoid such failures,
> congrats!

Yes apparently Paul is aware of it so that's all what I wanted to make
sure of.

Thanks!
Willy

PS: I'll hopefully check your other series later today

Re: nolibc patches, still possible for 6.5 ?

[Paul E. McKenney]

On Thu, Jun 08, 2023 at 10:21:57AM +0200, Willy Tarreau wrote:
> Hi Zhangjin,
> 
> On Thu, Jun 08, 2023 at 04:13:37PM +0800, Zhangjin Wu wrote:
> > > I rebased the branch on top of 6.4-rc5 and got the same. I'm building
> > > with gcc-11.3.0 from kernel.org. I'm not sure whether this comes from
> > > my build environment or recent changes to the kernel, but I'm sure I
> > > haven't seen that error during 6.3-rc cycle. However, given that
> > > Zhangjin seems to have successfully built it for riscv, there might
> > > be something odd on my side.
> > >
> > 
> > Sorry, The reason is that to speed up the kernel build+nolibc tests, a
> > local small config was customized for the first report,
> 
> You don't have to be sorry for this! The more diversity in configs, the
> more bugs are discovered before they hit users.

And I couldn't agree more!  Again, thank you both!

							Thanx, Paul

> > it may not
> > trigger the above issue, in the later report about the v4 rv32 compile
> > support [1], I did test 'run' target with the default defconfig and
> > reported the same failure as bove, this is the note I pasted there:
> > 
> >   ---
> >   Did compile test for aarch64, rv32 and rv64, include run-user and run.
> > 
> >   Note, this is required with the default config from the
> >   20230606-nolibc-rv32+stkp7a branch of [5]:
> > 
> >     diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h
> >     index ce02bb09651b..72bd8fe0cad6 100644
> >     --- a/kernel/rcu/tasks.h
> >     +++ b/kernel/rcu/tasks.h
> >     @@ -1934,11 +1934,13 @@ void show_rcu_tasks_gp_kthreads(void)
> >      }
> >      #endif /* #ifndef CONFIG_TINY_RCU */
> > 
> >     +#ifdef CONFIG_TASKS_RCU
> >      struct task_struct *get_rcu_tasks_gp_kthread(void)
> >      {
> >             return rcu_tasks.kthread_ptr;
> >      }
> >      EXPORT_SYMBOL_GPL(get_rcu_tasks_gp_kthread);
> >     +#endif
> > 
> >      #ifdef CONFIG_PROVE_RCU
> >      struct rcu_tasks_test_desc {
> >   ---
> 
> Indeed, I initially didn't make the connection regarding this, because
> you mentioned the fix but not the problem, so I didn't remember to take
> any particular care about this.
> 
> > I have seen Paul have known the cause of the above issue in your later
> > discussion and you found a better solution to avoid such failures,
> > congrats!
> 
> Yes apparently Paul is aware of it so that's all what I wanted to make
> sure of.
> 
> Thanks!
> Willy
> 
> PS: I'll hopefully check your other series later today

Re: nolibc patches, still possible for 6.5 ?

[Thomas Weißschuh]

On 2023-06-04 15:20:11+0200, Willy Tarreau wrote:
> Hello Paul,
> 
> Thomas and Zhangjin have provided significant nolibc cleanups, and
> fixes, as well as preparation work to later support riscv32.
> 
> These consist in the following main series:
>   - generalization of stackprotector to other archs that were not
>     previously supported (riscv, mips, loongarch, arm, arm64)
> 
>   - general cleanups of the makefile, test report output, deduplication
>     of certain tests
> 
>   - slightly better compliance of some tests performed on certain syscalls
>     (e.g. no longer pass (void*)1 to gettimeofday() since glibc hates it).
> 
>   - add support for nanoseconds in stat() and statx()
> 
>   - fixes for some syscalls (e.g. ppoll() has 5 arguments not 4)
> 
>   - fixes around limits.h and  INT_MAX / INT_FAST64_MAX
> 
> I rebased the whole series on top of your latest dev branch (d19a9ca3d5)
> and it works fine for all archs.
> 
> I don't know if you're still planning on merging new stuff in this area
> for 6.5 or not (since I know that it involves new series of tests on your
> side as well), but given that Zhangjin will engage into deeper changes
> later for riscv32 that will likely imply to update more syscalls to use
> the time64 ones, I would prefer to split the cleanups from the hard stuff,
> but I'll let you judge based on the current state of what's pending for
> 6.5.
> 
> In any case I'm putting all this here for now (not for merge yet):
> 
>    git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20230604-nolibc-rv32+stkp6
> 
> I'd like Thomas and Zhangjin to perform a last check to confirm they're
> OK with this final integration.

(Talking about 20230605-nolibc-rv32+stkp7,
but the same issue is in stkp6)

I noticed that the commit introducing prctl() comes *after* the one
making use of it.

8389cb4b55ae ("selftests/nolibc: prevent coredumps during test execution")
033c01b5a8f4 ("tools/nolibc: add support for prctl()")

This can lead to compiler errors and break bisectability.

They just need to be swapped.

Thomas

Re: nolibc patches, still possible for 6.5 ?

[Willy Tarreau]

On Mon, Jun 05, 2023 at 12:44:03PM +0200, Thomas Weißschuh wrote:
> (Talking about 20230605-nolibc-rv32+stkp7,
> but the same issue is in stkp6)
> 
> I noticed that the commit introducing prctl() comes *after* the one
> making use of it.
> 
> 8389cb4b55ae ("selftests/nolibc: prevent coredumps during test execution")
> 033c01b5a8f4 ("tools/nolibc: add support for prctl()")
> 
> This can lead to compiler errors and break bisectability.
> 
> They just need to be swapped.

just done in 20230606-nolibc-rv32+stkp7a, thank you!

Willy