Re: NMI received for unknown reason when running perf with IBS on AMD

[Ravi Bangoria <ravi.bangoria@amd.com>]

Hi Arnaldo, Guilherme,

>> [443324.404938] Uhhuh. NMI received for unknown reason 2c on CPU 9.
>> [443324.404940] Dazed and confused, but trying to continue

This is a known hw bug in Zen2:
https://lore.kernel.org/lkml/e08e33d5-4f6d-91aa-f335-9404d16a983c@amd.com

You won't see it on later Zen processors, or at least the possibility of
hitting it is very less.

> (gdb) print perf_evlist__first(evlist)->attr.precise_ip
> $4 = 2
> (gdb)
> 
> So it seems to be using IBS:

Correct. More details below.

> Humm:
> 
>> # cpu pmu capabilities: max_precise=0
> 
> For me:
> 
> [root@five ~]# perf report --header-only | grep "cpu pmu capabilities"
> # cpu pmu capabilities: max_precise=0

Although confusing, this is technically correct because AMD core pmu
does not support precise mode.

However, as a special case, 3 core pmu events: cpu-cycles, r076 (same as
cpu-cycles) and r0C1 (micro-ops) are supported with attr.precise_ip > 0,
and they are forwarded to IBS OP pmu. All other core pmu events with
attr.precise_ip > 0 fails with -EINVAL.

> [root@five ~]# cat /sys/devices/cpu/caps/max_precise
> 0
> [root@five ~]#
> 
> Ravi, this probably is the max_precise for the "core" PMU, but the "ibs"
> one, that gets used when :pp and :p (but not :ppp) is used, i.e. that:
> 
> +++ b/arch/x86/events/amd/core.c
> @@ -374,7 +374,7 @@ static int amd_pmu_hw_config(struct perf_event *event)
> 
>         /* pass precise event sampling to ibs: */
>         if (event->attr.precise_ip && get_ibs_caps())
> -               return -ENOENT;
> +               return forward_event_to_ibs(event);
> 
> Thing..
> 
> Shouldn't we have some sort of max_precise cap for IBS so that we could
> use it and not have this confusing "max_precise=0" for the core PMU but
> accept attr.precise_ip = 1 and 2?

Unlike Intel PEBS which provides levels of precision, AMD core pmu is
inherently non-precise and IBS is inherently precise. So max_precise value
is irrelevant for IBS. I mean ibs_op//, ibs_op//p, ibs_op//pp and
ibs_op//ppp are all same.

Thanks,
Ravi

PS: I don't know the history behind not supporting attr.precise_ip = 3 when
event is forwarded from core pmu to IBS. This check was added long back in
2012. Robert might know.