Re: [PATCH] selftests: settings: tests can be in subsubdirs

[Kees Cook <keescook@chromium.org>]

On Thu, Nov 21, 2019 at 05:32:42PM +0100, Matthieu Baerts wrote:
> Hi Shuah,
> 
> First, thank you for maintaining the Kernel Selftest framework!
> 
> On 22/10/2019 19:12, Matthieu Baerts wrote:
> > Commit 852c8cbf34d3 (selftests/kselftest/runner.sh: Add 45 second
> > timeout per test) adds support for a new per-test-directory "settings"
> > file. But this only works for tests not in a sub-subdirectories, e.g.
> > 
> >   - tools/testing/selftests/rtc (rtc) is OK,
> >   - tools/testing/selftests/net/mptcp (net/mptcp) is not.
> > 
> > We have to increase the timeout for net/mptcp tests which are not
> > upstreamed yet but this fix is valid for other tests if they need to add
> > a "settings" file, see the full list with:
> > 
> >    tools/testing/selftests/*/*/**/Makefile
> > 
> > Note that this patch changes the text header message printed at the end
> > of the execution but this text is modified only for the tests that are
> > in sub-subdirectories, e.g.
> > 
> >    ok 1 selftests: net/mptcp: mptcp_connect.sh
> > 
> > Before we had:
> > 
> >    ok 1 selftests: mptcp: mptcp_connect.sh
> > 
> > But showing the full target name is probably better, just in case a
> > subsubdir has the same name as another one in another subdirectory.
> > 
> > Fixes: 852c8cbf34d3 (selftests/kselftest/runner.sh: Add 45 second timeout per test)
> > Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
> Sorry to bother you again with this but by chance, did you have a look at
> the patch below? :)
> 
> It doesn't only fix an issue with MPTCP, not in the kernel yet. But it also
> fixes the issue of taking the right "settings" file (if available) for any
> other tests in a sub-directory, e.g.:
> 
>   drivers/dma-buf
>   filesystems/binderfs
>   net/forwarding
>   networking/timestamping
> 
> But I guess all tests in powerpc/* dirs and others.

Thanks for the ping! I missed this patch when you originally sent it.
Yes, this make sense to me:

Reviewed-by: Kees Cook <keescook@chromium.org>

As an improvement on this, I wonder if we need to walk all directories
between $BASEDIR and $DIR? Actually, let me write this and send it...

-Kees

> 
> Cheers,
> Matt
> 
> > ---
> >   tools/testing/selftests/kselftest/runner.sh | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
> > index 84de7bc74f2c..0d7a89901ef7 100644
> > --- a/tools/testing/selftests/kselftest/runner.sh
> > +++ b/tools/testing/selftests/kselftest/runner.sh
> > @@ -90,7 +90,7 @@ run_one()
> >   run_many()
> >   {
> >   	echo "TAP version 13"
> > -	DIR=$(basename "$PWD")
> > +	DIR="${PWD#${BASE_DIR}/}"
> >   	test_num=0
> >   	total=$(echo "$@" | wc -w)
> >   	echo "1..$total"
> > 
> 
> -- 
> Matthieu Baerts | R&D Engineer
> matthieu.baerts@tessares.net
> Tessares SA | Hybrid Access Solutions
> www.tessares.net
> 1 Avenue Jean Monnet, 1348 Louvain-la-Neuve, Belgium

-- 
Kees Cook