Re: [OE-core][PATCH] selftest: allow '-R' and '-r' be used together

[Alexandre Belloni <alexandre.belloni@bootlin.com>]

On 24/11/2022 12:24:38+0000, Chen Qi wrote:
> Hi Alexandre,
> 
> Yes, I'm aware of this behavior regression.
> With this patch, 'oe-selftest -R xxx' needs to be changed to 'oe-selftest -a -R xxx'.
> 

Yes, my point is that there is no -R in the command line used by the AB

> I'm not sure if this is useful for other people. Because how people use oe-selftest is unknown to me.
> For me, I want to run something like 'oe-selftest -r A -R A.B.C'.
> 
> Regards,
> Qi
> 
> -----Original Message-----
> From: Alexandre Belloni <alexandre.belloni@bootlin.com> 
> Sent: Thursday, November 24, 2022 7:35 PM
> To: Chen, Qi <Qi.Chen@windriver.com>
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core][PATCH] selftest: allow '-R' and '-r' be used together
> 
> Hello,
> 
> On 23/11/2022 18:55:28-0800, Chen Qi wrote:
> > For now, there's no easy way to skip some specified tests when using 
> > 'oe-selftest -r'. This is because '-R' and '-r' are in the mutually 
> > exclusive group. Looking at the codes, the codes have already allowed 
> > running specified tests and skipping specified tests at the same time. 
> > So move '-R' out of the mutually exclusive group and change its help 
> > message.
> > 
> > After this change, the following command could be used.
> >   oe-selftest -r A -R A.B.C
> > 
> > This does introduce a behavior regression.
> > Previous 'oe-selftest -R xxx' needs to now be changed to 'oe-selftest 
> > -a -R xxx'.
> > 
> 
> This fails on the autobuilders which uses:
> 
> oe-selftest --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/4395/steps/14/logs/stdio
> 
> > Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> > ---
> >  meta/lib/oeqa/selftest/context.py | 6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
> > 
> > diff --git a/meta/lib/oeqa/selftest/context.py 
> > b/meta/lib/oeqa/selftest/context.py
> > index 78c7a467e2..c7dd03ce37 100644
> > --- a/meta/lib/oeqa/selftest/context.py
> > +++ b/meta/lib/oeqa/selftest/context.py
> > @@ -154,9 +154,6 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
> >          group.add_argument('-a', '--run-all-tests', default=False,
> >                  action="store_true", dest="run_all_tests",
> >                  help='Run all (unhidden) tests')
> > -        group.add_argument('-R', '--skip-tests', required=False, action='store',
> > -                nargs='+', dest="skips", default=None,
> > -                help='Run all (unhidden) tests except the ones specified. Format should be <module>[.<class>[.<test_method>]]')
> >          group.add_argument('-r', '--run-tests', required=False, action='store',
> >                  nargs='+', dest="run_tests", default=None,
> >                  help='Select what tests to run (modules, classes or 
> > test methods). Format should be: <module>.<class>.<test_method>') @@ -171,6 +168,9 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
> >                  action="store_true", default=False,
> >                  help='List all available tests.')
> >  
> > +        parser.add_argument('-R', '--skip-tests', required=False, action='store',
> > +                nargs='+', dest="skips", default=None,
> > +                help='Skip the tests specified. Format should be 
> > + <module>[.<class>[.<test_method>]]')
> >          parser.add_argument('-j', '--num-processes', dest='processes', action='store',
> >                  type=int, help="number of processes to execute in 
> > parallel with")
> >  
> > --
> > 2.37.1
> > 
> 
> > 
> > 
> > 
> 
> 
> -- 
> Alexandre Belloni, co-owner and COO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#173743): https://lists.openembedded.org/g/openembedded-core/message/173743
> Mute This Topic: https://lists.openembedded.org/mt/95231396/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com