Re: NMI received for unknown reason when running perf with IBS on AMD

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Em Tue, Oct 17, 2023 at 01:31:30PM +0530, Ravi Bangoria escreveu:
> Hi Arnaldo, Guilherme,
> 
> >> [443324.404938] Uhhuh. NMI received for unknown reason 2c on CPU 9.
> >> [443324.404940] Dazed and confused, but trying to continue
> 
> This is a known hw bug in Zen2:
> https://lore.kernel.org/lkml/e08e33d5-4f6d-91aa-f335-9404d16a983c@amd.com
> 
> You won't see it on later Zen processors, or at least the possibility of
> hitting it is very less.

Do you think we can detect that it is a zen2 processor, that it has this
problem, and then the tools can just avoid using precise_ip != 0? I.e.
"cycles:P" becomes plain "cycles" and asking for, say, "cycles:p" states
that this isn't possible on Zen2?
 
> > (gdb) print perf_evlist__first(evlist)->attr.precise_ip
> > $4 = 2
> > (gdb)

> > So it seems to be using IBS:
 
> Correct. More details below.
 
> > Humm:

> >> # cpu pmu capabilities: max_precise=0

> > For me:

> > [root@five ~]# perf report --header-only | grep "cpu pmu capabilities"
> > # cpu pmu capabilities: max_precise=0
 
> Although confusing, this is technically correct because AMD core pmu
> does not support precise mode.
 
> However, as a special case, 3 core pmu events: cpu-cycles, r076 (same as
> cpu-cycles) and r0C1 (micro-ops) are supported with attr.precise_ip > 0,
> and they are forwarded to IBS OP pmu. All other core pmu events with
> attr.precise_ip > 0 fails with -EINVAL.

If we can programatically detect that IBS is present we can add a note
right after that 'perf report --header-only' line with this information.
 
> > [root@five ~]# cat /sys/devices/cpu/caps/max_precise
> > 0
> > [root@five ~]#

> > Ravi, this probably is the max_precise for the "core" PMU, but the "ibs"
> > one, that gets used when :pp and :p (but not :ppp) is used, i.e. that:
> > 
> > +++ b/arch/x86/events/amd/core.c
> > @@ -374,7 +374,7 @@ static int amd_pmu_hw_config(struct perf_event *event)
> > 
> >         /* pass precise event sampling to ibs: */
> >         if (event->attr.precise_ip && get_ibs_caps())
> > -               return -ENOENT;
> > +               return forward_event_to_ibs(event);
> > 
> > Thing..
> > 
> > Shouldn't we have some sort of max_precise cap for IBS so that we could
> > use it and not have this confusing "max_precise=0" for the core PMU but
> > accept attr.precise_ip = 1 and 2?
 
> Unlike Intel PEBS which provides levels of precision, AMD core pmu is
> inherently non-precise and IBS is inherently precise. So max_precise value
> is irrelevant for IBS. I mean ibs_op//, ibs_op//p, ibs_op//pp and
> ibs_op//ppp are all same.

This part can go to the man page, where :p, :P is documented.
 
> Thanks,
> Ravi
 
> PS: I don't know the history behind not supporting attr.precise_ip = 3 when
> event is forwarded from core pmu to IBS. This check was added long back in
> 2012. Robert might know.

Robert?