[lttng-dev] [lttng-tools] Removal of root_regression tests

[lttng-dev] [lttng-tools] Removal of root_regression tests

[Marcel Hamer via lttng-dev]

Hello,

Since version v2.12.9 of lttng-tools the root_regression file has been emptied
to make the tests part of the 'make check' sequence instead. 

We were always actively using that test file as part of our regression testing. 
In our case we are working in a cross-compilation environment, where the run.sh
script was used on target for testing and as such not at compile time. It is not
easy to run a make check sequence on a target.

It is now also a bit unclear which tests actually require root access and which
tests do not. I understood this was the reason the file was called
'root_regression'?

Some questions that get raised because of this:

- Is there now an alternative way to run regressions on target in case of a
  cross-compilation environment? 
- Would there be a possibility to fill the 'root_regression' file again and
  possibly revert this change?
- How are tests now identified that require root access?

This has been as part of commit 9e2d9d2bae015e6748ebf8575ea602ee0fe65c62 in
lttng-tools.

Thank you in advance!

Kind regards,

Marcel
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [lttng-tools] Removal of root_regression tests

[Jonathan Rajotte-Julien via lttng-dev]

Hi Marcel,

----- Original Message -----
> From: "Marcel Hamer via lttng-dev" <lttng-dev@lists.lttng.org>
> To: "lttng-dev" <lttng-dev@lists.lttng.org>
> Sent: Monday, 13 June, 2022 07:49:39
> Subject: [lttng-dev] [lttng-tools] Removal of root_regression tests

> Hello,
> 
> Since version v2.12.9 of lttng-tools the root_regression file has been emptied
> to make the tests part of the 'make check' sequence instead.
> 
> We were always actively using that test file as part of our regression testing.
> In our case we are working in a cross-compilation environment, where the run.sh
> script was used on target for testing and as such not at compile time. It is not
> easy to run a make check sequence on a target.

I would suggest that you take a look at how OpenEmbedded does it with ptest AFAIK it match your requirements:

https://github.com/openembedded/openembedded-core/blob/c7e2901eacf3dcbd0c5bb91d2cc1d467b4a9aaf7/meta/recipes-kernel/lttng/lttng-tools_2.13.7.bb#L75

> 
> It is now also a bit unclear which tests actually require root access and which
> tests do not. I understood this was the reason the file was called
> 'root_regression'?

Yes when the tests suites primarily used `prove` via run.sh.

We have been slowly moving away from it for a good time and now mostly use the Automake test harness as much as possible.

The worse that will happen if you run a test that required root as a non-root user is that `skip` tap output will be emitted.

> 
> Some questions that get raised because of this:
> 
> - Is there now an alternative way to run regressions on target in case of a
>  cross-compilation environment?

AFAIU, this is out of scope of the lttng project. Still, I would recommend that you see how yocto/oe do it with ptest.

> - Would there be a possibility to fill the 'root_regression' file again and
>  possibly revert this change?

Feel free to do it out-of-tree. I doubt that we are the only project that WindRiver handles that uses
the automake test harness and that do not provide a easy way to run on-target for cross-compilation testing.

A quick grep with "isroot" should get you 95% there.

> - How are tests now identified that require root access?

All tests that require root access test for it at runtime

Something along:

regression/tools/streaming/test_high_throughput_limits:

 if [ "$(id -u)" == "0" ]; then
 	isroot=1
 else
	 isroot=0
 fi

 skip $isroot "Root access is needed to set bandwidth limits. Skipping all tests." $NUM_TESTS ||
 {
 ...
    Tests are done here.
 }

Cheers
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [lttng-tools] Removal of root_regression tests

[Marcel Hamer via lttng-dev]

Hello Jonathan,

On Mon, Jun 13, 2022 at 11:21:49AM -0400, Jonathan Rajotte-Julien wrote:
> [Please note: This e-mail is from an EXTERNAL e-mail address]
> 
> Hi Marcel,
> 
> ----- Original Message -----
> > From: "Marcel Hamer via lttng-dev" <lttng-dev@lists.lttng.org>
> > To: "lttng-dev" <lttng-dev@lists.lttng.org>
> > Sent: Monday, 13 June, 2022 07:49:39
> > Subject: [lttng-dev] [lttng-tools] Removal of root_regression tests
> 
> > Hello,
> >
> > Since version v2.12.9 of lttng-tools the root_regression file has been emptied
> > to make the tests part of the 'make check' sequence instead.
> >
> > We were always actively using that test file as part of our regression testing.
> > In our case we are working in a cross-compilation environment, where the run.sh
> > script was used on target for testing and as such not at compile time. It is not
> > easy to run a make check sequence on a target.
> 
> I would suggest that you take a look at how OpenEmbedded does it with ptest AFAIK it match your requirements:
> 
> https://github.com/openembedded/openembedded-core/blob/c7e2901eacf3dcbd0c5bb91d2cc1d467b4a9aaf7/meta/recipes-kernel/lttng/lttng-tools_2.13.7.bb#L75
> 

That is a very good suggestion. I guess we were a bit too focused on our existing
solution of using run.sh. We will look into this.

> >
> > It is now also a bit unclear which tests actually require root access and which
> > tests do not. I understood this was the reason the file was called
> > 'root_regression'?
> 
> Yes when the tests suites primarily used `prove` via run.sh.
> 
> We have been slowly moving away from it for a good time and now mostly use the Automake test harness as much as possible.
> 
> The worse that will happen if you run a test that required root as a non-root user is that `skip` tap output will be emitted.
> 
> >
> > Some questions that get raised because of this:
> >
> > - Is there now an alternative way to run regressions on target in case of a
> >  cross-compilation environment?
> 
> AFAIU, this is out of scope of the lttng project. Still, I would recommend that you see how yocto/oe do it with ptest.
> 
> > - Would there be a possibility to fill the 'root_regression' file again and
> >  possibly revert this change?
> 
> Feel free to do it out-of-tree. I doubt that we are the only project that WindRiver handles that uses
> the automake test harness and that do not provide a easy way to run on-target for cross-compilation testing.

Yes, you are right and that is a fair point. We will look into the ptest
solution.

> 
> A quick grep with "isroot" should get you 95% there.
> 
> > - How are tests now identified that require root access?
> 
> All tests that require root access test for it at runtime
> 
> Something along:
> 
> regression/tools/streaming/test_high_throughput_limits:
> 
>  if [ "$(id -u)" == "0" ]; then
>         isroot=1
>  else
>          isroot=0
>  fi
> 
>  skip $isroot "Root access is needed to set bandwidth limits. Skipping all tests." $NUM_TESTS ||
>  {
>  ...
>     Tests are done here.
>  }
> 
> Cheers

Thanks for the tip on how to identify test cases that require root privileges.

Kind regards,

Marcel
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev