Re: [PATCH v5 5/5] dts: add crypto test decorator

[Thomas Monjalon <thomas@monjalon.net>]

12/03/2026 20:54, Patrick Robb:
> On Thu, Mar 12, 2026 at 2:08 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > 06/03/2026 17:40, Andrew Bailey:
> > > Currently, a test case is decorated to signify whether to use Scapy or
> > > TREX. This change allows test suites that do not use a traffic generator
> > > to avoid the overhead of initializing either one.
> > [...]
> > >                      if (tt.test_type is TestCaseType.FUNCTIONAL and self.func)
> > >                      or (tt.test_type is TestCaseType.PERFORMANCE and self.perf)
> > > +                    or (tt.test_type is TestCaseType.CRYPTO and self.crypto)
> >
> > That's a bit strange to read,
> > because a crypto test can be functional or performance.
> > I suppose we can re-discuss the classification of the tests.
> > If the only need here is about the traffic generator,
> 
> In my opinion this is not quite true. There is another (and more
> important) need associated with the test categories, which is allowing
> users to run only a sub-category of tests according to their interest
> or hardware setup. I.e. we can expect some people to show up and
> desire only to run functional testcases (perhaps they have a minimal
> hardware setup which is not good for performance, or they are only
> working on resolving functional issues) and the test type categories
> allow the users this flexibility with regards to what kinds of DTS
> tests are run.
> 
> > we could make it "raw input" or something like that?
> >
> >
> 
> In any case I agree with Thomas and Andrew that the situation with
> regards to test type classification/organization could use a review
> and a tweak in the 26.07 release. Right now every single crypto_test
> is a performance test (i.e. it runs a crypto workload, compares the
> Gbps output against the Gbps baseline the user set in
> tests_config.yaml for each testcast). But, some open questions for the
> future:
> 
> 1. Is there a need for users to select only functional crypto cases vs
> only perf crypto cases? Or we can just leave all crypto cases in 1
> bucket?
> 2. If the above is true, should we switch the testcase decorators over to:
> 
>     a. ethdev_func
>     b. ethdev_perf
>     c. cryptodev_func
>     d. cryptodev_perf
> 
> and every testcase gets one of these decorators
> 
> OR
> 
> We apply multiple decorators to each testcase (i.e. add both
> crypto_test and func_test to a testcase) and have the framework code
> read the whole set of testcase type decorators and setup accordingly.
> 
> Thomas, is what we have right now in terms of testcase classification
> okay for the 26.03 release, and we can tweak it in 26.07, or do you
> want us to do something in the immediate term?

There is no urgency.
I'm just opening the discussion for future.