[PATCH 00/14] RCU torture changes for v6.16

[PATCH 00/14] RCU torture changes for v6.16

[Joel Fernandes]

Hi,

Please find the upcoming miscellaneous RCU changes. The changes can also
be found at:

        git://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git torture.2025.04.18a

Thanks.

Joel Fernandes (2):
  rcutorture: Perform more frequent testing of ->gpwrap
  rcutorture: Fix issue with re-using old images on ARM64

Paul E. McKenney (12):
  rcutorture: Make srcu_lockdep.sh check kernel Kconfig
  rcutorture: Make srcu_lockdep.sh check reader-conflict handling
  rcutorture: Split out beginning and end from rcu_torture_one_read()
  rcutorture: Make torture.sh --do-rt use CONFIG_PREEMPT_RT
  rcutorture: Add tests for SRCU up/down reader primitives
  rcutorture: Pull rcu_torture_updown() loop body into new function
  rcutorture: Comment invocations of tick_dep_set_task()
  rcutorture: Complain if an ->up_read() is delayed more than 10 seconds
  rcutorture: Check for ->up_read() without matching ->down_read()
  checkpatch: Deprecate srcu_read_lock_lite() and
    srcu_read_unlock_lite()
  torture: Add --do-{,no-}normal to torture.sh
  torture: Add testing of RCU's Rust bindings to torture.sh

 kernel/rcu/rcu.h                              |   4 +
 kernel/rcu/rcutorture.c                       | 440 +++++++++++++++---
 kernel/rcu/tree.c                             |  34 +-
 kernel/rcu/tree.h                             |   1 +
 scripts/checkpatch.pl                         |   2 +
 .../rcutorture/bin/kvm-test-1-run.sh          |   2 +-
 .../selftests/rcutorture/bin/srcu_lockdep.sh  |  42 +-
 .../selftests/rcutorture/bin/torture.sh       |  89 +++-
 8 files changed, 537 insertions(+), 77 deletions(-)

-- 
2.43.0

[PATCH 12/14] torture: Add testing of RCU's Rust bindings to torture.sh

[Joel Fernandes]

From: "Paul E. McKenney" <paulmck@kernel.org>

This commit adds a --do-rcu-rust parameter to torture.sh, which invokes
a rust_doctests_kernel kunit run.  Note that kunit wants a clean source
tree, so this runs "make mrproper", which might come as a surprise to
some users.  Should there be a --mrproper parameter to torture.sh to make
the user explicitly ask for it?

Co-developed-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
---
 .../selftests/rcutorture/bin/torture.sh       | 45 +++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/tools/testing/selftests/rcutorture/bin/torture.sh b/tools/testing/selftests/rcutorture/bin/torture.sh
index 475f758f6216..e03fdaca89b3 100755
--- a/tools/testing/selftests/rcutorture/bin/torture.sh
+++ b/tools/testing/selftests/rcutorture/bin/torture.sh
@@ -59,6 +59,7 @@ do_clocksourcewd=yes
 do_rt=yes
 do_rcutasksflavors=yes
 do_srcu_lockdep=yes
+do_rcu_rust=no
 
 # doyesno - Helper function for yes/no arguments
 function doyesno () {
@@ -89,6 +90,7 @@ usage () {
 	echo "       --do-rcutorture / --do-no-rcutorture / --no-rcutorture"
 	echo "       --do-refscale / --do-no-refscale / --no-refscale"
 	echo "       --do-rt / --do-no-rt / --no-rt"
+	echo "       --do-rcu-rust / --do-no-rcu-rust / --no-rcu-rust"
 	echo "       --do-scftorture / --do-no-scftorture / --no-scftorture"
 	echo "       --do-srcu-lockdep / --do-no-srcu-lockdep / --no-srcu-lockdep"
 	echo "       --duration [ <minutes> | <hours>h | <days>d ]"
@@ -191,6 +193,9 @@ do
 	--do-rt|--do-no-rt|--no-rt)
 		do_rt=`doyesno "$1" --do-rt`
 		;;
+	--do-rcu-rust|--do-no-rcu-rust|--no-rcu-rust)
+		do_rcu_rust=`doyesno "$1" --do-rcu-rust`
+		;;
 	--do-scftorture|--do-no-scftorture|--no-scftorture)
 		do_scftorture=`doyesno "$1" --do-scftorture`
 		;;
@@ -485,6 +490,46 @@ then
 	torture_set "rcurttorture-exp" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --kconfig "CONFIG_PREEMPT_RT=y CONFIG_EXPERT=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_FULL=y CONFIG_RCU_NOCB_CPU=y" --trust-make
 fi
 
+if test "$do_rcu_rust" = "yes"
+then
+	echo " --- do-rcu-rust:" Start `date` | tee -a $T/log
+	rrdir="tools/testing/selftests/rcutorture/res/$ds/results-rcu-rust"
+	mkdir -p "$rrdir"
+	echo " --- make LLVM=1 rustavailable " | tee -a $rrdir/log > $rrdir/rustavailable.out
+	make LLVM=1 rustavailable > $T/rustavailable.out 2>&1
+	retcode=$?
+	echo $retcode > $rrdir/rustavailable.exitcode
+	cat $T/rustavailable.out | tee -a $rrdir/log >> $rrdir/rustavailable.out 2>&1
+	buildphase=rustavailable
+	if test "$retcode" -eq 0
+	then
+		echo " --- Running 'make mrproper' in order to run kunit." | tee -a $rrdir/log > $rrdir/mrproper.out
+		make mrproper > $rrdir/mrproper.out 2>&1
+		retcode=$?
+		echo $retcode > $rrdir/mrproper.exitcode
+		buildphase=mrproper
+	fi
+	if test "$retcode" -eq 0
+	then
+		echo " --- Running rust_doctests_kernel." | tee -a $rrdir/log > $rrdir/rust_doctests_kernel.out
+		./tools/testing/kunit/kunit.py run --make_options LLVM=1 --make_options CLIPPY=1 --arch arm64 --kconfig_add CONFIG_SMP=y --kconfig_add CONFIG_WERROR=y --kconfig_add CONFIG_RUST=y rust_doctests_kernel >> $rrdir/rust_doctests_kernel.out 2>&1
+		# @@@ Remove "--arch arm64" in order to test on native architecture?
+		# @@@ Analyze $rrdir/rust_doctests_kernel.out contents?
+		retcode=$?
+		echo $retcode > $rrdir/rust_doctests_kernel.exitcode
+		buildphase=rust_doctests_kernel
+	fi
+	if test "$retcode" -eq 0
+	then
+		echo "rcu-rust($retcode)" $rrdir >> $T/successes
+		echo Success >> $rrdir/log
+	else
+		echo "rcu-rust($retcode)" $rrdir >> $T/failures
+		echo " --- rcu-rust Test summary:" >> $rrdir/log
+		echo " --- Summary: Exit code $retcode from $buildphase, see $rrdir/$buildphase.out" >> $rrdir/log
+	fi
+fi
+
 if test "$do_srcu_lockdep" = "yes"
 then
 	echo " --- do-srcu-lockdep:" Start `date` | tee -a $T/log
-- 
2.43.0

[PATCH 12/12] torture: Add testing of RCU's Rust bindings to torture.sh

[Joel Fernandes]

From: "Paul E. McKenney" <paulmck@kernel.org>

This commit adds a --do-rcu-rust parameter to torture.sh, which invokes
a rust_doctests_kernel kunit run.  Note that kunit wants a clean source
tree, so this runs "make mrproper", which might come as a surprise to
some users.  Should there be a --mrproper parameter to torture.sh to make
the user explicitly ask for it?

Co-developed-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
---
 .../selftests/rcutorture/bin/torture.sh       | 45 +++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/tools/testing/selftests/rcutorture/bin/torture.sh b/tools/testing/selftests/rcutorture/bin/torture.sh
index 5ccd60a563be..6d1a84f3f631 100755
--- a/tools/testing/selftests/rcutorture/bin/torture.sh
+++ b/tools/testing/selftests/rcutorture/bin/torture.sh
@@ -59,6 +59,7 @@ do_clocksourcewd=yes
 do_rt=yes
 do_rcutasksflavors=yes
 do_srcu_lockdep=yes
+do_rcu_rust=no
 
 # doyesno - Helper function for yes/no arguments
 function doyesno () {
@@ -89,6 +90,7 @@ usage () {
 	echo "       --do-rcutorture / --do-no-rcutorture / --no-rcutorture"
 	echo "       --do-refscale / --do-no-refscale / --no-refscale"
 	echo "       --do-rt / --do-no-rt / --no-rt"
+	echo "       --do-rcu-rust / --do-no-rcu-rust / --no-rcu-rust"
 	echo "       --do-scftorture / --do-no-scftorture / --no-scftorture"
 	echo "       --do-srcu-lockdep / --do-no-srcu-lockdep / --no-srcu-lockdep"
 	echo "       --duration [ <minutes> | <hours>h | <days>d ]"
@@ -191,6 +193,9 @@ do
 	--do-rt|--do-no-rt|--no-rt)
 		do_rt=`doyesno "$1" --do-rt`
 		;;
+	--do-rcu-rust|--do-no-rcu-rust|--no-rcu-rust)
+		do_rcu_rust=`doyesno "$1" --do-rcu-rust`
+		;;
 	--do-scftorture|--do-no-scftorture|--no-scftorture)
 		do_scftorture=`doyesno "$1" --do-scftorture`
 		;;
@@ -485,6 +490,46 @@ then
 	torture_set "rcurttorture-exp" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --kconfig "CONFIG_PREEMPT_RT=y CONFIG_EXPERT=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_FULL=y" --trust-make
 fi
 
+if test "$do_rcu_rust" = "yes"
+then
+	echo " --- do-rcu-rust:" Start `date` | tee -a $T/log
+	rrdir="tools/testing/selftests/rcutorture/res/$ds/results-rcu-rust"
+	mkdir -p "$rrdir"
+	echo " --- make LLVM=1 rustavailable " | tee -a $rrdir/log > $rrdir/rustavailable.out
+	make LLVM=1 rustavailable > $T/rustavailable.out 2>&1
+	retcode=$?
+	echo $retcode > $rrdir/rustavailable.exitcode
+	cat $T/rustavailable.out | tee -a $rrdir/log >> $rrdir/rustavailable.out 2>&1
+	buildphase=rustavailable
+	if test "$retcode" -eq 0
+	then
+		echo " --- Running 'make mrproper' in order to run kunit." | tee -a $rrdir/log > $rrdir/mrproper.out
+		make mrproper > $rrdir/mrproper.out 2>&1
+		retcode=$?
+		echo $retcode > $rrdir/mrproper.exitcode
+		buildphase=mrproper
+	fi
+	if test "$retcode" -eq 0
+	then
+		echo " --- Running rust_doctests_kernel." | tee -a $rrdir/log > $rrdir/rust_doctests_kernel.out
+		./tools/testing/kunit/kunit.py run --make_options LLVM=1 --make_options CLIPPY=1 --arch arm64 --kconfig_add CONFIG_SMP=y --kconfig_add CONFIG_WERROR=y --kconfig_add CONFIG_RUST=y rust_doctests_kernel >> $rrdir/rust_doctests_kernel.out 2>&1
+		# @@@ Remove "--arch arm64" in order to test on native architecture?
+		# @@@ Analyze $rrdir/rust_doctests_kernel.out contents?
+		retcode=$?
+		echo $retcode > $rrdir/rust_doctests_kernel.exitcode
+		buildphase=rust_doctests_kernel
+	fi
+	if test "$retcode" -eq 0
+	then
+		echo "rcu-rust($retcode)" $rrdir >> $T/successes
+		echo Success >> $rrdir/log
+	else
+		echo "rcu-rust($retcode)" $rrdir >> $T/failures
+		echo " --- rcu-rust Test summary:" >> $rrdir/log
+		echo " --- Summary: Exit code $retcode from $buildphase, see $rrdir/$buildphase.out" >> $rrdir/log
+	fi
+fi
+
 if test "$do_srcu_lockdep" = "yes"
 then
 	echo " --- do-srcu-lockdep:" Start `date` | tee -a $T/log
-- 
2.43.0

Re: [PATCH 00/14] RCU torture changes for v6.16

[Joel Fernandes]

On 4/18/2025 12:09 PM, Joel Fernandes wrote:
> Hi,
> 
> Please find the upcoming miscellaneous RCU changes. The changes can also

FYI: Sorry, s/miscellaneous/torture-related/ here.

Also please ignore patches with "XX/12" in their subject, that was an error on
my part.

The rest of the series stands good for perusal.

thanks,

 - Joel

Re: [PATCH 12/14] torture: Add testing of RCU's Rust bindings to torture.sh

[Miguel Ojeda]

On Fri, Apr 18, 2025 at 6:10 PM Joel Fernandes <joelagnelf@nvidia.com> wrote:
>
> a rust_doctests_kernel kunit run.  Note that kunit wants a clean source
> tree, so this runs "make mrproper", which might come as a surprise to
> some users.  Should there be a --mrproper parameter to torture.sh to make
> the user explicitly ask for it?

One may run KUnit without using `kunit.py`, i.e. by enabling the
Kconfigs and booting the kernel. Would that help?

Cheers,
Miguel

Re: [PATCH 12/14] torture: Add testing of RCU's Rust bindings to torture.sh

[Paul E. McKenney]

On Fri, Apr 18, 2025 at 07:31:53PM +0200, Miguel Ojeda wrote:
> On Fri, Apr 18, 2025 at 6:10 PM Joel Fernandes <joelagnelf@nvidia.com> wrote:
> >
> > a rust_doctests_kernel kunit run.  Note that kunit wants a clean source
> > tree, so this runs "make mrproper", which might come as a surprise to
> > some users.  Should there be a --mrproper parameter to torture.sh to make
> > the user explicitly ask for it?
> 
> One may run KUnit without using `kunit.py`, i.e. by enabling the
> Kconfigs and booting the kernel. Would that help?

Potentially?

Suppose we fired up a guest OS and captured the console output.  Is there
a way to make that guest OS shut down automatically at the end of the
test and to extract the test results?

In the short term, I am good with this the current state of this being
default-disabled and those of us wishing to run it regularly adjusting
our scripts accordingly.

But, longer term, if there is a better way...

							Thanx, Paul

Re: [PATCH 12/14] torture: Add testing of RCU's Rust bindings to torture.sh

[Miguel Ojeda]

On Fri, Apr 18, 2025 at 8:04 PM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> Suppose we fired up a guest OS and captured the console output.  Is there
> a way to make that guest OS shut down automatically at the end of the
> test and to extract the test results?

Ah, sorry, I thought you were already doing something like that, i.e.
that perhaps you could reuse some kernel build you already had and
avoiding a full rebuild/`mrproper`. The KUnit Python script uses QEMU
and parses the results; e.g. you could look for the results lines
like:

    # Totals: pass:133 fail:0 skip:0 total:133
    ok 2 rust_doctests_kernel

Cheers,
Miguel

Re: [PATCH 12/14] torture: Add testing of RCU's Rust bindings to torture.sh

[Paul E. McKenney]

On Fri, Apr 18, 2025 at 08:32:46PM +0200, Miguel Ojeda wrote:
> On Fri, Apr 18, 2025 at 8:04 PM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > Suppose we fired up a guest OS and captured the console output.  Is there
> > a way to make that guest OS shut down automatically at the end of the
> > test and to extract the test results?
> 
> Ah, sorry, I thought you were already doing something like that, i.e.
> that perhaps you could reuse some kernel build you already had and
> avoiding a full rebuild/`mrproper`. The KUnit Python script uses QEMU
> and parses the results; e.g. you could look for the results lines
> like:
> 
>     # Totals: pass:133 fail:0 skip:0 total:133
>     ok 2 rust_doctests_kernel

Alternatively, I could clone a copy of the current archive into a
temporary directory, "make mrproper" there, run kunit normally, then
clean up the temporary directory.  Extra storage, but quite a bit
more robust and user-friendly.

Other thoughts?

							Thanx, Paul

Re: [12/14] torture: Add testing of RCU's Rust bindings to torture.sh

[Paul E. McKenney]

On Fri, Apr 18, 2025 at 10:29:19PM -0000, Joel Fernandes wrote:
> Hello, Paul,
> 
> On Fri, 18 Apr 2025 22:26:17 GMT, "Paul E. McKenney" wrote:
> > On Fri, Apr 18, 2025 at 08:32:46PM +0200, Miguel Ojeda wrote:
> > > On Fri, Apr 18, 2025 at 8:04 PM Paul E. McKenney <paulmck@kernel.org> wrot
> > e:
> > > >
> > > > Suppose we fired up a guest OS and captured the console output.  Is ther
> > e
> > > > a way to make that guest OS shut down automatically at the end of the
> > > > test and to extract the test results?
> > > 
> > > Ah, sorry, I thought you were already doing something like that, i.e.
> > > that perhaps you could reuse some kernel build you already had and
> > > avoiding a full rebuild/mrproper. The KUnit Python script uses QEMU
> > > and parses the results; e.g. you could look for the results lines
> > > like:
> > > 
> > >     # Totals: pass:133 fail:0 skip:0 total:133
> > >     ok 2 rust_doctests_kernel
> > 
> > Alternatively, I could clone a copy of the current archive into a
> > temporary directory, "make mrproper" there, run kunit normally, then
> > clean up the temporary directory.  Extra storage, but quite a bit
> > more robust and user-friendly.
> 
> Just to be on the same page, is the concern about the
> slowness of mrproper or that it kills the kernel build
> artifacts requiring a clean build?

It blows away things like tags and cscope databases.  Me, I have my
cscope databases elsewhere, but lots of people build them for their
live git repos.  And they are (quite understandably) unhappy when their
source-code browsing tools are blown away by some random test doing an
unsuspected "make mrproper".  ;-)

> What kind of improvement are we looking for and why would
> this patch in its current form not work?

For the near term, I am OK with its current form because it is
non-default.  Longer term, though, we need it to be tested by default,
and that means making it avoid undoing people's work.  My short-term
approach is to enable this test on my employer's test systems, which
have Rust set up correctly, and skip it on my laptop, which has a strange
FrankenRust due to my early playing around with that language.

							Thanx, Paul

Re: [12/14] torture: Add testing of RCU's Rust bindings to torture.sh

[Joel Fernandes]

Hi Paul,

On 4/18/2025 6:45 PM, Paul E. McKenney wrote:
>>>>> Suppose we fired up a guest OS and captured the console output.  Is there
>>>>> a way to make that guest OS shut down automatically at the end of the
>>>>> test and to extract the test results?
>>>> Ah, sorry, I thought you were already doing something like that, i.e.
>>>> that perhaps you could reuse some kernel build you already had and
>>>> avoiding a full rebuild/mrproper. The KUnit Python script uses QEMU
>>>> and parses the results; e.g. you could look for the results lines
>>>> like:
>>>>
>>>>     # Totals: pass:133 fail:0 skip:0 total:133
>>>>     ok 2 rust_doctests_kernel
>>>>
>>> Alternatively, I could clone a copy of the current archive into a
>>> temporary directory, "make mrproper" there, run kunit normally, then
>>> clean up the temporary directory.  Extra storage, but quite a bit
>>> more robust and user-friendly.
>>>
>> Just to be on the same page, is the concern about the
>> slowness of mrproper or that it kills the kernel build
>> artifacts requiring a clean build?
>
> It blows away things like tags and cscope databases.  Me, I have my
> cscope databases elsewhere, but lots of people build them for their
> live git repos.  And they are (quite understandably) unhappy when their
> source-code browsing tools are blown away by some random test doing an
> unsuspected "make mrproper".  😉

Cool. One thing is, running other test modes in torture.sh also reconfigures the
kernel and potentially recompiles the entire kernel as a result. So if someone
is already having their own kernel build, running torture.sh already may cause
them to have to do a full rebuild, AFAICS. But yes, and to your point, the
cscope DB and stuff may get blown away for additional disruption.

> 
>> What kind of improvement are we looking for and why would
>> this patch in its current form not work?
> For the near term, I am OK with its current form because it is
> non-default.  Longer term, though, we need it to be tested by default,
> and that means making it avoid undoing people's work.  My short-term
> approach is to enable this test on my employer's test systems, which
> have Rust set up correctly, and skip it on my laptop, which has a strange
> FrankenRust due to my early playing around with that language.
> 
Or we teach kunit.py to not require a mrproper? :-) I wonder why it needs to do
that. I may run into that too considering my other kernel project requires me to
mess around with rust.

thanks,

 - Joel

Re: [12/14] torture: Add testing of RCU's Rust bindings to torture.sh

[Paul E. McKenney]

On Sat, Apr 19, 2025 at 06:24:56PM -0400, Joel Fernandes wrote:
> Hi Paul,
> 
> On 4/18/2025 6:45 PM, Paul E. McKenney wrote:
> >>>>> Suppose we fired up a guest OS and captured the console output.  Is there
> >>>>> a way to make that guest OS shut down automatically at the end of the
> >>>>> test and to extract the test results?
> >>>> Ah, sorry, I thought you were already doing something like that, i.e.
> >>>> that perhaps you could reuse some kernel build you already had and
> >>>> avoiding a full rebuild/mrproper. The KUnit Python script uses QEMU
> >>>> and parses the results; e.g. you could look for the results lines
> >>>> like:
> >>>>
> >>>>     # Totals: pass:133 fail:0 skip:0 total:133
> >>>>     ok 2 rust_doctests_kernel
> >>>>
> >>> Alternatively, I could clone a copy of the current archive into a
> >>> temporary directory, "make mrproper" there, run kunit normally, then
> >>> clean up the temporary directory.  Extra storage, but quite a bit
> >>> more robust and user-friendly.
> >>>
> >> Just to be on the same page, is the concern about the
> >> slowness of mrproper or that it kills the kernel build
> >> artifacts requiring a clean build?
> >
> > It blows away things like tags and cscope databases.  Me, I have my
> > cscope databases elsewhere, but lots of people build them for their
> > live git repos.  And they are (quite understandably) unhappy when their
> > source-code browsing tools are blown away by some random test doing an
> > unsuspected "make mrproper".  😉
> 
> Cool. One thing is, running other test modes in torture.sh also reconfigures the
> kernel and potentially recompiles the entire kernel as a result. So if someone
> is already having their own kernel build, running torture.sh already may cause
> them to have to do a full rebuild, AFAICS. But yes, and to your point, the
> cscope DB and stuff may get blown away for additional disruption.
> 
> > 
> >> What kind of improvement are we looking for and why would
> >> this patch in its current form not work?
> > For the near term, I am OK with its current form because it is
> > non-default.  Longer term, though, we need it to be tested by default,
> > and that means making it avoid undoing people's work.  My short-term
> > approach is to enable this test on my employer's test systems, which
> > have Rust set up correctly, and skip it on my laptop, which has a strange
> > FrankenRust due to my early playing around with that language.
> > 
> Or we teach kunit.py to not require a mrproper? :-) I wonder why it needs to do
> that. I may run into that too considering my other kernel project requires me to
> mess around with rust.

They might have a good reason for that "make mrproper", but it does
sound like a good avenue to investigate.  Otherwise, as noted earlier, my
thought is to clone the repo into a temp directory, run "make mrproper"
there, continue being a non-special user of kunit, and avoid messing up
people's databases and tags.

							Thanx, Paul