Re: [PATCH 3/6] selftests/resctrl: Simplify benchmark parameter passing

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Ilpo,

On 8/30/24 4:13 AM, Ilpo Järvinen wrote:
> On Thu, 29 Aug 2024, Reinette Chatre wrote:
> 
>> The benchmark used during the CMT, MBM, and MBA tests can be provided by
>> the user via (-b) parameter to the tests, if not provided the default
>> "fill_buf" benchmark is used.
>>
>> The "fill_buf" benchmark requires parameters and these are managed as
>> an array of strings.
>>
>> Using an array of strings to manage the "fill_buf" parameters is
>> complex because it requires transformations to/from strings at every
>> producer and consumer. This is made worse for the individual tests
>> where the default benchmark parameters values may not be appropriate and
>> additional data wrangling is required. For example, the CMT test
>> duplicates the entire array of strings in order to replace one of
>> the parameters.
>>
>> Replace the "array of strings" parameters used for "fill_buf" with a
>> struct that contains the "fill_buf" parameters that can be used directly
>> without transformations to/from strings. Make these parameters
>> part of the parameters associated with each test so that each test can
>> set benchmark parameters that are appropriate for it.
>>
>> Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
>> ---

...

> 
> If I didn't miss anything important, this change takes away the ability to
> alter fill_buf's parameters using -b option which to me felt the most
> useful way to use that parameter. The current code of course was lacks
> many safeguards for that case but still felt an useful feature.

hmmm ... thank you for pointing this out. I did not consider this use case.
I have never received feedback on how these tests are used  and
if folks even use "-b", for "fill_buf" parameter changes or something else.

> 
> I suggest that while parsing -b parameter, check if it starts with
> "fill_buf", and if it does, parse the argument into fill_buf_param in
> user_params which will override the default fill_buf parameters.
> 
> While parsing, adding new sanity checks wouldn't be a bad idea.
> 
> It might be some parameters might be better to be overridden always by the
> tests, e.g. "once" but specifying "operation" (W instead or R) or
> "buf_size" seems okay use cases to me.
> 

Will do. Thank you for the suggestion.

Reinette