Re: [PATCH 2/2] perf test: Fix JSON format linter test checks

[Michael Petlan <mpetlan@redhat.com>]

On Sun, 22 Jan 2023, Namhyung Kim wrote:
> On Fri, Jan 20, 2023 at 6:00 AM Michael Petlan <mpetlan@redhat.com> wrote:
> >
> > The test fails on CPUs with topdown metrics, where it is common to print
> > two metrics per line. Since these are included in default event set for
> > `perf stat -- something`, it causes the "no args" subtest to fail due to
> > unexpected member count. We need to accept 7 or 9 members in that case.
> >
> > Coming to that, counting commas in the JSON line and consider it a number
> > of elements is incorrect and misleading. There should be +1 element than
> > the count of commas, while also, commas can potentially appear in the
> > event names. Count " : " substrings rather, since these better fit to
> > what should be actually counted.
> 
> I'm also curious if ":" could appear in the event/metric names.
> Probably more chances to have ":" than "," in the name, no?

Yes, ':' can appear in the event names, but not " : " (with the spaces
around).

Michael

> 
> Thanks,
> Namhyung
> 
> 
> >
> > Fixes: 0c343af2a2f8 ("perf test: JSON format checking")
> >
> > Signed-off-by: Michael Petlan <mpetlan@redhat.com>
> > ---
> >  .../tests/shell/lib/perf_json_output_lint.py     | 16 ++++++++--------
> >  1 file changed, 8 insertions(+), 8 deletions(-)
> >
> > diff --git a/tools/perf/tests/shell/lib/perf_json_output_lint.py b/tools/perf/tests/shell/lib/perf_json_output_lint.py
> > index d90f8d102eb9..4f1bbb3f07ec 100644
> > --- a/tools/perf/tests/shell/lib/perf_json_output_lint.py
> > +++ b/tools/perf/tests/shell/lib/perf_json_output_lint.py
> > @@ -40,17 +40,17 @@ def is_counter_value(num):
> >    return isfloat(num) or num == '<not counted>' or num == '<not supported>'
> >
> >  def check_json_output(expected_items):
> > -  if expected_items != -1:
> > +  if expected_items:
> >      for line in Lines:
> >        if 'failed' not in line:
> >          count = 0
> > -        count = line.count(',')
> > -        if count != expected_items and count >= 1 and count <= 3 and 'metric-value' in line:
> > +        count = line.count(' : ')
> > +        if count not in expected_items and count >= 2 and count <= 4 and 'metric-value' in line:
> >            # Events that generate >1 metric may have isolated metric
> >            # values and possibly other prefixes like interval, core and
> >            # aggregate-number.
> >            continue
> > -        if count != expected_items:
> > +        if count not in expected_items:
> >            raise RuntimeError(f'wrong number of fields. counted {count} expected {expected_items}'
> >                               f' in \'{line}\'')
> >    checks = {
> > @@ -82,14 +82,14 @@ def check_json_output(expected_items):
> >
> >  try:
> >    if args.no_args or args.system_wide or args.event:
> > -    expected_items = 6
> > +    expected_items = [7, 9]
> >    elif args.interval or args.per_thread or args.system_wide_no_aggr:
> > -    expected_items = 7
> > +    expected_items = [8, 10]
> >    elif args.per_core or args.per_socket or args.per_node or args.per_die:
> > -    expected_items = 8
> > +    expected_items = [9, 11]
> >    else:
> >      # If no option is specified, don't check the number of items.
> > -    expected_items = -1
> > +    expected_items = []
> >    check_json_output(expected_items)
> >  except:
> >    print('Test failed for input:\n' + '\n'.join(Lines))
> > --
> > 2.18.4
> >
> 
>