Re: perf test pmu-events question

[John Garry <john.g.garry@oracle.com>]

On 23/06/2023 16:48, Ian Rogers wrote:
> On Fri, Jun 23, 2023 at 8:20 AM John Garry <john.g.garry@oracle.com> wrote:
>>
>> Hi Ian,
>>
>> I was doing some re-write for pmu sys events metric support and I want
>> to add more testcases - we don't really have any.
>>
>> So I noticed that tests/pmu-events.c::test__parsing() can pick up on an
>> invalid metric for the test CPU, like M3 as it references itself.
>> However we can't detect whether event aliases referenced in the metric
>> exist for the CPU. Is it worth testing for this? Would it be sane to
>> scan the associated CPU event tables to see if we can find the event
>> there? Or do we already have some per-non-test-CPU metric validator
>> already in the testcases (I could not see it)?
> 

Hi Ian,

Thanks for getting back to me.

> 
> Hi John,
> 
> we always need more tests :-)

Sure

> For Intel there has historically been a
> problem that the names in metrics and the event data will get out of
> alignment. There is a best effort test during the metric generation
> that this doesn't happen any more here:
> https://urldefense.com/v3/__https://github.com/intel/perfmon/blob/main/scripts/create_perf_json.py*L1171-L1188__;Iw!!ACWV5N9M2RV99hQ!OKyD56aIReczgaPkGD3MZ0yZ-WFe7qXps1aFyF6pibEN8nk0r4N5Y87MgV0twed_V8E8l7kjJd88_AjTGW8$

Does that validate alias event support for the CPU? It seems that the 
specific lines mentioned are just checking for keywords.

> But still things get messed up, such as PMU names being mismatched -
> when this happens the JSON events just don't appear at runtime. We
> could try to replicate the event matching logic at creation time in
> jevents.py and that should do what you mention and be
> multi-architecture.

OK, that could work. It is limited to testing just aliases, which I 
suppose has some value.

However I am not sure how well this would suit testing sys event metric 
support, where I want to ensure that a specific event alias is chosen 
for a metric in the same pmu_sys_events struct.

> The metric test tests every metric produces an
> output, and this fails when event parsing fails, so this has
> traditionally been the way to find bad metrics. 
>We do have build time
> tests like:
> https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/pmu-events/Build?h=perf-tools-next*n30__;Iw!!ACWV5N9M2RV99hQ!OKyD56aIReczgaPkGD3MZ0yZ-WFe7qXps1aFyF6pibEN8nk0r4N5Y87MgV0twed_V8E8l7kjJd882X6fEUE$

This just tests invalid metric format, right?

> 
> Intel don't have JSON encodings for the TSX events and so they use
> sysfs events and this is what motivated the has_event test:
> https://urldefense.com/v3/__https://lore.kernel.org/linux-perf-users/20230623151016.4193660-1-irogers@google.com/__;!!ACWV5N9M2RV99hQ!OKyD56aIReczgaPkGD3MZ0yZ-WFe7qXps1aFyF6pibEN8nk0r4N5Y87MgV0twed_V8E8l7kjJd88rRd1z5c$
> 
> Some other thoughts in this area:
> 1) I'd like to get sysfs snapshots so that we can get things like
> parse-events coverage of heterogeneous PMUs without needing such a
> system.

That idea would seem good to testing other sort of uncore PMUs - for 
example, since I don't have access to a system which supports sys 
events, I need to bodge the kernel to support some fake PMU kernel 
driver for things like smmu pmcg.

> 2) The largest part of the perf binary is the json data.

oh

> I'd like to
> convert the json data to a sysfs style, then compress it in a way
> where we benefit from duplication between files (zip can't do this,
> but tgz or 7zip can). libarchive can load these formats from memory.
> The current json data on x86 converts to about 7MB of read-only data,
> but the same json files compress down to about 1.5MB. The perf binary
> is about 3MB of code, so doing the compression should hopefully make
> the file size about half what it currently is and do something to stem
> the problem of the binary slowly growing over time with new
> architecture releases.

ok, sounds reasonable if slow binary loading is really a problem.

> 3) I'm hoping we can do something like a unionfs with the in memory
> sysfs, on disk sysfs tar balls and the like, and the machine's actual
> sysfs so that we can just have one consistent view in the tool.
> 4) I want to make PMUs lazier so that we don't need to eagerly load
> all the events. The cycles event doesn't need json or sysfs data, but
> it is all still loaded due to wanting the PMU. This faults in a lot of
> JSON data on x86 that is then never used.
> 
That should be pretty easy to do by only needing to setup event aliases 
et al when we have events which are not from known config terms, right?

Thanks,
John