Re: [igt-dev] [PATCH i-g-t] tests/perf_pmu: Compare against requested freq in frequency subtest

["Dixit, Ashutosh" <ashutosh.dixit@intel.com>]

On Wed, 09 Nov 2022 17:37:18 -0800, Belgaumkar, Vinay wrote:
>
> On 11/8/2022 5:53 PM, Dixit, Ashutosh wrote:
> > On Tue, 08 Nov 2022 13:02:33 -0800, Belgaumkar, Vinay wrote:
> >>
> >> On 11/8/2022 1:24 AM, Tvrtko Ursulin wrote:
> >>> On 08/11/2022 01:31, Dixit, Ashutosh wrote:
> >>>> On Mon, 07 Nov 2022 16:57:24 -0800, Belgaumkar, Vinay wrote:
> >>>>> On 11/7/2022 4:22 PM, Dixit, Ashutosh wrote:
> >>>>>> On Mon, 07 Nov 2022 16:18:31 -0800, Dixit, Ashutosh wrote:
> >>>>>> Hi Vinay,
> >>>>>>
> >>>>>> A question for you below.
> >>>>>>
> >>>>>>> So I submitted this patch to repro the issue and to print out the
> >>>>>>> requested
> >>>>>>> freq from sysfs:
> >>>>>>>
> >>>>>>> https://patchwork.freedesktop.org/series/110630/
> >>>>>>>
> >>>>>>> And we can see the output here:
> >>>>>>>
> >>>>>>> https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8061/bat-dg2-11/igt@perf_pmu@frequency.html
> >>>>>>>
> >>>>>>> ```
> >>>>>>> IGT-Version: 1.26-g1bef4d081 (x86_64) (Linux:
> >>>>>>> 6.1.0-rc4-CI_DRM_12352-gc55ac6a74bd1+ x86_64)
> >>>>>>> Starting subtest: frequency
> >>>>>>> Frequency: min=300, max=2050, boost=2050 MHz
> >>>>>>> Min frequency: requested 349.7, actual 349.7
> >>>>>>> Max frequency: requested 2048.0, actual 2048.0
> >>>>>>> Sysfs requested: min 350, max 2050
> >>>>>>> Stack trace:
> >>>>>>>      #0 ../../../usr/src/igt-gpu-tools/lib/igt_core.c:1908
> >>>>>>> __igt_fail_assert()
> >>>>>>>      #1 ../../../usr/src/igt-gpu-tools/tests/i915/perf_pmu.c:1656
> >>>>>>> __igt_unique____real_main2147()
> >>>>>>>      #2 ../../../usr/src/igt-gpu-tools/tests/i915/perf_pmu.c:2147
> >>>>>>> main()
> >>>>>>>      #3 [__libc_start_main+0xf3]
> >>>>>>>      #4 [_start+0x2e]
> >>>>>>> Subtest frequency: FAIL (2.212s)
> >>>>>>> ```
> >>>>>>>
> >>>>>>> So we clearly see the requested freq from sysfs is indeed 350 MHz so
> >>>>>>> SLPC/PCODE is not honoring the set min == max == boost freq (and PMU
> >>>>>>> is
> >>>>>>> measuring what sysfs is showing). In general PCODE is the final
> >>>>>>> arbiter in
> >>>>>>> such cases and we do occasionally see instances where set freq limits
> >>>>>>> are
> >>>>>>> not honored.
> >>>>>>>
> >>>>>>> I would say if igt@perf_pmu@frequency is testing freq measured by PMU
> >>>>>>> then
> >>>>>>> the patch below is correct. Whether SLPC/PCODE is honoring the set
> >>>>>>> freq
> >>>>>>> limits should be tested in a SLPC test (which we also have).
> >>>>>> igt@perf_pmu@frequency sets 'min == max == boost == 300 MHz' but we
> >>>>>> still
> >>>>>> see the requested freq to be 350 MHz. Do we have a SLPC test covering
> >>>>>> this
> >>>>>> scenario or should we add one? This is failing on one of the DG2's.
> >>>>> Does adding a delay help (around 20 ms for the h2g to go through
> >>>>> typically)?
> >>>> There is no delay but the test calls gem_quiescent_gpu() after setting
> >>>> 'min
> >>>> == max == boost == 300 MHz' and then launches a spinner. We are checking
> >>>> the requested freq 500 ms after the spinner is started (so plenty of
> >>>> time
> >>>> for the h2g) and still the requested freq is 350 MHz.
> >>>>
> >>>>> Also, is there a workload running when we change the min=max=boost to
> >>>>> 300?
> >>>> No, the workload is started after setting the freq and calling
> >>>> gem_quiescent_gpu().
> >>> Implication here seems to be that gem_quiescent_gpu() would need to cover
> >>> H2G communication - does it?
> >> No, more like there needs to be a WL running in order for SLPC to actively
> >> make changes to the requested frequency.
> > Well here we set the freq's first and later when the WL runs FW should
> > select an appropriate requested freq.
> >
> >>>>> We already check these things in our SLPC selftests.
> >>> Is it then expected to respect the 300MHz max in this case? Or if it
> >>> can't, should it be reflected in the sysfs readback?
> >> It should respect the 300 Mhz. The only question in my mind is regarding
> >> efficient frequency. Can we print out what the RP1 is here?
> > RP1 is also 300. We can see it here:
> >
> > https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8071/bat-dg2-11/igt@perf_pmu@frequency.html
> >
> > with this patch:
> >
> > https://patchwork.freedesktop.org/series/110630/#rev3
> >
> > In this case, min == max == 300, but requested freq is 400 (previously it
> > was 350).
>
> Ok, this might be happening due to the following -
>
> 1. We have efficient frequency enabled now, so GuC will use that instead of
> min even on light loads etc.
>
> 2. It is also known that this efficient frequency is "dynamic", especially
> on DG2.
>
> 3. When we set min freq to a value lower than efficient, we will turn off
> efficient frequency usage. But, here min = efficient = 300, so it will
> remain ON.
>
> This is why we see 350 or even 400 sometimes as we are not bound to the min
> (or even a single freq level) when efficient freq usage is allowed.

Hi Vinay, thanks for the explanation, makes sense.

> The solution for this case may be to turn off efficient frequency
> forcibly for this test and see if that helps. We do that in the selftests
> to ensure proper frequency bounds.

I don't believe we can do this from userland, can we?

Thanks.
--
Ashutosh