[PATCH i-g-t v2 00/10] Add per-client engine utilization tests

[PATCH i-g-t v2 00/10] Add per-client engine utilization tests

[Umesh Nerlige Ramappa]

Port per client engine utilization tests from i915 to xe.

v2:
- Add a test to destroy exec queue before capturing sample
- Include review comments from Lucas/Riana

Opens from v1 review
---
1) Documenting mapping for TEST_BUSY/TEST_TRAILING_IDLE etc to tests
- Not seeing a clean way to do this since some tests iterate over all engine and
  some over virtual engines and then some tests omit hwe/width/count params.
  Would leave it as is if acceptable.

2) Busyness > 100%
- sampling RING_TIMESTAMP before utilization values is likely leading to this
  error. Ideally RING TIMESTAMP should be sampled before 1st sample and after 2nd
  sample, but that's not feasible. It can only be one way for each sample -
  either before or after.

Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>

Umesh Nerlige Ramappa (10):
  tests/intel/xe_drm_fdinfo: Rename basic to basic-memory test
  tests/intel/xe_drm_fdinfo: Add basic-engine-utilization test
  tests/intel/xe_drm_fdinfo: Add helper to read utilization for all
    classes
  tests/intel/xe_drm_fdinfo: Add helpers for spinning batches
  tests/intel/xe_drm_fdinfo: Add single engine tests
  tests/intel/xe_drm_fdinfo: Add tests to verify all class utilization
  tests/intel/xe_drm_fdinfo: Add an iterator for virtual engines
  tests/intel/xe_drm_fdinfo: Add tests for virtual engines
  tests/intel/xe_drm_fdinfo: Add tests for parallel engines
  tests/intel/xe_drm_fdinfo: Ensure queue destroy records load correctly

 tests/intel/xe_drm_fdinfo.c | 587 +++++++++++++++++++++++++++++++++++-
 1 file changed, 577 insertions(+), 10 deletions(-)

-- 
2.38.1

[PATCH i-g-t v2 01/10] tests/intel/xe_drm_fdinfo: Rename basic to basic-memory test

[Umesh Nerlige Ramappa]

Existing basic test is checking memory info in fdinfo. Rename
accordingly.

Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
---
 tests/intel/xe_drm_fdinfo.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
index 2ceafba24..f0aa23e2f 100644
--- a/tests/intel/xe_drm_fdinfo.c
+++ b/tests/intel/xe_drm_fdinfo.c
@@ -22,8 +22,8 @@
  * Feature: SMI, core
  * Test category: SysMan
  *
- * SUBTEST: basic
- * Description: Check if basic fdinfo content is present
+ * SUBTEST: basic-memory
+ * Description: Check if basic fdinfo content is present for memory
  *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
@@ -259,7 +259,7 @@ static void test_total_resident(int xe)
 	xe_vm_destroy(xe, vm);
 }
 
-static void basic(int xe)
+static void basic_memory(int xe)
 {
 	struct drm_xe_mem_region *memregion;
 	uint64_t memreg = all_memory_regions(xe), region;
@@ -299,9 +299,9 @@ igt_main
 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
 	}
 
-	igt_describe("Check if basic fdinfo content is present");
-	igt_subtest("basic")
-		basic(xe);
+	igt_describe("Check if basic fdinfo content is present for memory");
+	igt_subtest("basic-memory")
+		basic_memory(xe);
 
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
-- 
2.38.1

Re: [PATCH i-g-t v2 01/10] tests/intel/xe_drm_fdinfo: Rename basic to basic-memory test

[Lucas De Marchi]

On Tue, Jul 02, 2024 at 05:25:23PM GMT, Umesh Nerlige Ramappa wrote:
>Existing basic test is checking memory info in fdinfo. Rename
>accordingly.
>
>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>---
> tests/intel/xe_drm_fdinfo.c | 12 ++++++------
> 1 file changed, 6 insertions(+), 6 deletions(-)
>
>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>index 2ceafba24..f0aa23e2f 100644
>--- a/tests/intel/xe_drm_fdinfo.c
>+++ b/tests/intel/xe_drm_fdinfo.c

I was also expecting a change to the testlist... but it seems we don't
really have this there.

https://intel-gfx-ci.01.org/tree/intel-xe/index.html?testfilter=xe_drm_fdinfo
returns an empty result so it seems we are not even running these tests?

Cc'ing some people. For the changes here:

Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>

thanks
Lucas De Marchi

>@@ -22,8 +22,8 @@
>  * Feature: SMI, core
>  * Test category: SysMan
>  *
>- * SUBTEST: basic
>- * Description: Check if basic fdinfo content is present
>+ * SUBTEST: basic-memory
>+ * Description: Check if basic fdinfo content is present for memory
>  *
>  * SUBTEST: drm-total-resident
>  * Description: Create and compare total and resident memory consumption by client
>@@ -259,7 +259,7 @@ static void test_total_resident(int xe)
> 	xe_vm_destroy(xe, vm);
> }
>
>-static void basic(int xe)
>+static void basic_memory(int xe)
> {
> 	struct drm_xe_mem_region *memregion;
> 	uint64_t memreg = all_memory_regions(xe), region;
>@@ -299,9 +299,9 @@ igt_main
> 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
> 	}
>
>-	igt_describe("Check if basic fdinfo content is present");
>-	igt_subtest("basic")
>-		basic(xe);
>+	igt_describe("Check if basic fdinfo content is present for memory");
>+	igt_subtest("basic-memory")
>+		basic_memory(xe);
>
> 	igt_describe("Create and compare total and resident memory consumption by client");
> 	igt_subtest("drm-total-resident")
>-- 
>2.38.1
>

Re: [PATCH i-g-t v2 01/10] tests/intel/xe_drm_fdinfo: Rename basic to basic-memory test

[Umesh Nerlige Ramappa]

On Thu, Jul 11, 2024 at 07:00:40AM -0500, Lucas De Marchi wrote:
>On Tue, Jul 02, 2024 at 05:25:23PM GMT, Umesh Nerlige Ramappa wrote:
>>Existing basic test is checking memory info in fdinfo. Rename
>>accordingly.
>>
>>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>>---
>>tests/intel/xe_drm_fdinfo.c | 12 ++++++------
>>1 file changed, 6 insertions(+), 6 deletions(-)
>>
>>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>>index 2ceafba24..f0aa23e2f 100644
>>--- a/tests/intel/xe_drm_fdinfo.c
>>+++ b/tests/intel/xe_drm_fdinfo.c
>
>I was also expecting a change to the testlist... but it seems we don't
>really have this there.
>
>https://intel-gfx-ci.01.org/tree/intel-xe/index.html?testfilter=xe_drm_fdinfo
>returns an empty result so it seems we are not even running these tests?

Strange, for rev2 I actually saw these new tests run in the shards list 
and was expecting the same for newer revs.

https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11334/shards-all.html?testfilter=xe_drm_fdinfo

Umesh

>
>Cc'ing some people. For the changes here:
>
>Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
>
>thanks
>Lucas De Marchi
>
>>@@ -22,8 +22,8 @@
>> * Feature: SMI, core
>> * Test category: SysMan
>> *
>>- * SUBTEST: basic
>>- * Description: Check if basic fdinfo content is present
>>+ * SUBTEST: basic-memory
>>+ * Description: Check if basic fdinfo content is present for memory
>> *
>> * SUBTEST: drm-total-resident
>> * Description: Create and compare total and resident memory consumption by client
>>@@ -259,7 +259,7 @@ static void test_total_resident(int xe)
>>	xe_vm_destroy(xe, vm);
>>}
>>
>>-static void basic(int xe)
>>+static void basic_memory(int xe)
>>{
>>	struct drm_xe_mem_region *memregion;
>>	uint64_t memreg = all_memory_regions(xe), region;
>>@@ -299,9 +299,9 @@ igt_main
>>		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
>>	}
>>
>>-	igt_describe("Check if basic fdinfo content is present");
>>-	igt_subtest("basic")
>>-		basic(xe);
>>+	igt_describe("Check if basic fdinfo content is present for memory");
>>+	igt_subtest("basic-memory")
>>+		basic_memory(xe);
>>
>>	igt_describe("Create and compare total and resident memory consumption by client");
>>	igt_subtest("drm-total-resident")
>>-- 
>>2.38.1
>>

Re: [PATCH i-g-t v2 01/10] tests/intel/xe_drm_fdinfo: Rename basic to basic-memory test

[Lucas De Marchi]

On Thu, Jul 11, 2024 at 10:39:28AM GMT, Umesh Nerlige Ramappa wrote:
>On Thu, Jul 11, 2024 at 07:00:40AM -0500, Lucas De Marchi wrote:
>>On Tue, Jul 02, 2024 at 05:25:23PM GMT, Umesh Nerlige Ramappa wrote:
>>>Existing basic test is checking memory info in fdinfo. Rename
>>>accordingly.
>>>
>>>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>>>---
>>>tests/intel/xe_drm_fdinfo.c | 12 ++++++------
>>>1 file changed, 6 insertions(+), 6 deletions(-)
>>>
>>>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>>>index 2ceafba24..f0aa23e2f 100644
>>>--- a/tests/intel/xe_drm_fdinfo.c
>>>+++ b/tests/intel/xe_drm_fdinfo.c
>>
>>I was also expecting a change to the testlist... but it seems we don't
>>really have this there.
>>
>>https://intel-gfx-ci.01.org/tree/intel-xe/index.html?testfilter=xe_drm_fdinfo
>>returns an empty result so it seems we are not even running these tests?
>
>Strange, for rev2 I actually saw these new tests run in the shards 
>list and was expecting the same for newer revs.
>
>https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11334/shards-all.html?testfilter=xe_drm_fdinfo

yeah... if I use the shards-all.html I can see it.. and that one doesn't
need a testlist since it's actually "all the tests".

Lucas De Marchi

>
>Umesh
>
>>
>>Cc'ing some people. For the changes here:
>>
>>Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
>>
>>thanks
>>Lucas De Marchi
>>
>>>@@ -22,8 +22,8 @@
>>>* Feature: SMI, core
>>>* Test category: SysMan
>>>*
>>>- * SUBTEST: basic
>>>- * Description: Check if basic fdinfo content is present
>>>+ * SUBTEST: basic-memory
>>>+ * Description: Check if basic fdinfo content is present for memory
>>>*
>>>* SUBTEST: drm-total-resident
>>>* Description: Create and compare total and resident memory consumption by client
>>>@@ -259,7 +259,7 @@ static void test_total_resident(int xe)
>>>	xe_vm_destroy(xe, vm);
>>>}
>>>
>>>-static void basic(int xe)
>>>+static void basic_memory(int xe)
>>>{
>>>	struct drm_xe_mem_region *memregion;
>>>	uint64_t memreg = all_memory_regions(xe), region;
>>>@@ -299,9 +299,9 @@ igt_main
>>>		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
>>>	}
>>>
>>>-	igt_describe("Check if basic fdinfo content is present");
>>>-	igt_subtest("basic")
>>>-		basic(xe);
>>>+	igt_describe("Check if basic fdinfo content is present for memory");
>>>+	igt_subtest("basic-memory")
>>>+		basic_memory(xe);
>>>
>>>	igt_describe("Create and compare total and resident memory consumption by client");
>>>	igt_subtest("drm-total-resident")
>>>-- 
>>>2.38.1
>>>

Re: [PATCH i-g-t v2 01/10] tests/intel/xe_drm_fdinfo: Rename basic to basic-memory test

[Kamil Konieczny]

Hi Lucas,
On 2024-07-11 at 15:36:16 -0500, Lucas De Marchi wrote:
> On Thu, Jul 11, 2024 at 10:39:28AM GMT, Umesh Nerlige Ramappa wrote:
> > On Thu, Jul 11, 2024 at 07:00:40AM -0500, Lucas De Marchi wrote:
> > > On Tue, Jul 02, 2024 at 05:25:23PM GMT, Umesh Nerlige Ramappa wrote:
> > > > Existing basic test is checking memory info in fdinfo. Rename
> > > > accordingly.
> > > > 
> > > > Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
> > > > ---
> > > > tests/intel/xe_drm_fdinfo.c | 12 ++++++------
> > > > 1 file changed, 6 insertions(+), 6 deletions(-)
> > > > 
> > > > diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
> > > > index 2ceafba24..f0aa23e2f 100644
> > > > --- a/tests/intel/xe_drm_fdinfo.c
> > > > +++ b/tests/intel/xe_drm_fdinfo.c
> > > 
> > > I was also expecting a change to the testlist... but it seems we don't
> > > really have this there.
> > > 
> > > https://intel-gfx-ci.01.org/tree/intel-xe/index.html?testfilter=xe_drm_fdinfo
> > > returns an empty result so it seems we are not even running these tests?
> > 
> > Strange, for rev2 I actually saw these new tests run in the shards list
> > and was expecting the same for newer revs.
> > 
> > https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11334/shards-all.html?testfilter=xe_drm_fdinfo
> 
> yeah... if I use the shards-all.html I can see it.. and that one doesn't
> need a testlist since it's actually "all the tests".

We need change only for hard-coded xe-fast-feedback or if it was
in blocklist. Btw is this correct:

xe.blocklist.txt:igt@drm_fdinfo

imho this should also be running in our tests? Or am I missing something?

Regards,
Kamil

> 
> Lucas De Marchi
> 
> > 
> > Umesh
> > 
> > > 
> > > Cc'ing some people. For the changes here:
> > > 
> > > Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
> > > 
> > > thanks
> > > Lucas De Marchi
> > > 
> > > > @@ -22,8 +22,8 @@
> > > > * Feature: SMI, core
> > > > * Test category: SysMan
> > > > *
> > > > - * SUBTEST: basic
> > > > - * Description: Check if basic fdinfo content is present
> > > > + * SUBTEST: basic-memory
> > > > + * Description: Check if basic fdinfo content is present for memory
> > > > *
> > > > * SUBTEST: drm-total-resident
> > > > * Description: Create and compare total and resident memory consumption by client
> > > > @@ -259,7 +259,7 @@ static void test_total_resident(int xe)
> > > > 	xe_vm_destroy(xe, vm);
> > > > }
> > > > 
> > > > -static void basic(int xe)
> > > > +static void basic_memory(int xe)
> > > > {
> > > > 	struct drm_xe_mem_region *memregion;
> > > > 	uint64_t memreg = all_memory_regions(xe), region;
> > > > @@ -299,9 +299,9 @@ igt_main
> > > > 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
> > > > 	}
> > > > 
> > > > -	igt_describe("Check if basic fdinfo content is present");
> > > > -	igt_subtest("basic")
> > > > -		basic(xe);
> > > > +	igt_describe("Check if basic fdinfo content is present for memory");
> > > > +	igt_subtest("basic-memory")
> > > > +		basic_memory(xe);
> > > > 
> > > > 	igt_describe("Create and compare total and resident memory consumption by client");
> > > > 	igt_subtest("drm-total-resident")
> > > > -- 
> > > > 2.38.1
> > > > 

[PATCH i-g-t v2 02/10] tests/intel/xe_drm_fdinfo: Add basic-engine-utilization test

[Umesh Nerlige Ramappa]

Add the basic test for engine utilization.

Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
---
 tests/intel/xe_drm_fdinfo.c | 32 +++++++++++++++++++++++++++++---
 1 file changed, 29 insertions(+), 3 deletions(-)

diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
index f0aa23e2f..b64bef339 100644
--- a/tests/intel/xe_drm_fdinfo.c
+++ b/tests/intel/xe_drm_fdinfo.c
@@ -14,17 +14,20 @@
 #include "xe/xe_spin.h"
 /**
  * TEST: xe drm fdinfo
- * Description: Read and verify drm client memory consumption using fdinfo
+ * Description: Read and verify drm client memory consumption and engine utilization using fdinfo
  * Category: Core
  * Mega feature: General Core features
  * Sub-category: driver
- * Functionality: Per client memory statistics
+ * Functionality: Per client memory and engine utilization statistics
  * Feature: SMI, core
  * Test category: SysMan
  *
  * SUBTEST: basic-memory
  * Description: Check if basic fdinfo content is present for memory
  *
+ * SUBTEST: basic-engine-utilization
+ * Description: Check if basic fdinfo content is present for engine utilization
+ *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
  *
@@ -35,10 +38,17 @@
  * Description: Create and compare active memory consumption by client
  */
 
-IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption using fdinfo");
+IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine utilization using fdinfo");
 
 #define BO_SIZE (65536)
 
+static const char *engine_map[] = {
+	"rcs",
+	"bcs",
+	"vcs",
+	"vecs",
+	"ccs",
+};
 /* Subtests */
 static void test_active(int fd, struct drm_xe_engine *engine)
 {
@@ -287,6 +297,18 @@ static void basic_memory(int xe)
 	}
 }
 
+static void basic_engine_utilization(int xe)
+{
+	struct drm_client_fdinfo info = { };
+	unsigned int ret;
+
+	ret = igt_parse_drm_fdinfo(xe, &info, engine_map,
+				   ARRAY_SIZE(engine_map), NULL, 0);
+	igt_assert_f(ret != 0, "failed with err:%d\n", errno);
+	igt_assert(!strcmp(info.driver, "xe"));
+	igt_require(info.num_engines);
+}
+
 igt_main
 {
 	int xe;
@@ -303,6 +325,10 @@ igt_main
 	igt_subtest("basic-memory")
 		basic_memory(xe);
 
+	igt_describe("Check if basic fdinfo content is present for engine utilization");
+	igt_subtest("basic-engine-utilization")
+		basic_engine_utilization(xe);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
-- 
2.38.1

Re: [PATCH i-g-t v2 02/10] tests/intel/xe_drm_fdinfo: Add basic-engine-utilization test

[Lucas De Marchi]

On Tue, Jul 02, 2024 at 05:25:24PM GMT, Umesh Nerlige Ramappa wrote:
>Add the basic test for engine utilization.
>
>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>


Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>

Lucas De Marchi

Re: [PATCH i-g-t v2 02/10] tests/intel/xe_drm_fdinfo: Add basic-engine-utilization test

[Lucas De Marchi]

On Thu, Jul 11, 2024 at 07:03:38AM GMT, Lucas De Marchi wrote:
>On Tue, Jul 02, 2024 at 05:25:24PM GMT, Umesh Nerlige Ramappa wrote:
>>Add the basic test for engine utilization.
>>
>>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>
>
>Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>

applied these first 2 patches while I think (after vacations) about the
rest. Also sent some minor fixes while studying this code:
https://patchwork.freedesktop.org/series/136713/

Lucas De Marchi

>
>Lucas De Marchi

[PATCH i-g-t v2 03/10] tests/intel/xe_drm_fdinfo: Add helper to read utilization for all classes

[Umesh Nerlige Ramappa]

Add a helper that captures utilization for all classes.

Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
---
 tests/intel/xe_drm_fdinfo.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
index b64bef339..c697f3e53 100644
--- a/tests/intel/xe_drm_fdinfo.c
+++ b/tests/intel/xe_drm_fdinfo.c
@@ -42,6 +42,11 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 
 #define BO_SIZE (65536)
 
+struct pceu_cycles {
+	uint64_t cycles;
+	uint64_t total_cycles;
+};
+
 static const char *engine_map[] = {
 	"rcs",
 	"bcs",
@@ -49,6 +54,21 @@ static const char *engine_map[] = {
 	"vecs",
 	"ccs",
 };
+static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
+{
+	struct drm_client_fdinfo info = { };
+	int class;
+
+	igt_assert(pceu);
+	igt_assert(igt_parse_drm_fdinfo(xe, &info, engine_map,
+					ARRAY_SIZE(engine_map), NULL, 0));
+
+	xe_for_each_engine_class(class) {
+		pceu[class].cycles = info.cycles[class];
+		pceu[class].total_cycles = info.total_cycles[class];
+	}
+}
+
 /* Subtests */
 static void test_active(int fd, struct drm_xe_engine *engine)
 {
-- 
2.38.1

Re: [PATCH i-g-t v2 03/10] tests/intel/xe_drm_fdinfo: Add helper to read utilization for all classes

[Lucas De Marchi]

On Tue, Jul 02, 2024 at 05:25:25PM GMT, Umesh Nerlige Ramappa wrote:
>Add a helper that captures utilization for all classes.
>
>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>---
> tests/intel/xe_drm_fdinfo.c | 20 ++++++++++++++++++++
> 1 file changed, 20 insertions(+)
>
>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>index b64bef339..c697f3e53 100644
>--- a/tests/intel/xe_drm_fdinfo.c
>+++ b/tests/intel/xe_drm_fdinfo.c
>@@ -42,6 +42,11 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
>
> #define BO_SIZE (65536)
>
>+struct pceu_cycles {
>+	uint64_t cycles;
>+	uint64_t total_cycles;
>+};
>+
> static const char *engine_map[] = {
> 	"rcs",
> 	"bcs",
>@@ -49,6 +54,21 @@ static const char *engine_map[] = {
> 	"vecs",
> 	"ccs",
> };

nit: missing newline here, otherwise:

>+static void read_engine_cycles(int xe, struct pceu_cycles *pceu)

I think that would be better to give the compiler a hint about the
size of the array:

static void read_engine_cycles(int xe,
			       struct pceu_cycles pceu[static DRM_XE_ENGINE_CLASS_COMPUTE + 1])

(it would not be a bad idea to have a local __IGT_DRM_XE_ENGINE_CLASS_LAST
too, but outside of the scope here)

Otherwise,

// Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>

Lucas De Marchi

>+{
>+	struct drm_client_fdinfo info = { };
>+	int class;
>+
>+	igt_assert(pceu);
>+	igt_assert(igt_parse_drm_fdinfo(xe, &info, engine_map,
>+					ARRAY_SIZE(engine_map), NULL, 0));
>+
>+	xe_for_each_engine_class(class) {
>+		pceu[class].cycles = info.cycles[class];
>+		pceu[class].total_cycles = info.total_cycles[class];
>+	}
>+}
>+
> /* Subtests */
> static void test_active(int fd, struct drm_xe_engine *engine)
> {
>-- 
>2.38.1
>

Re: [PATCH i-g-t v2 03/10] tests/intel/xe_drm_fdinfo: Add helper to read utilization for all classes

[Umesh Nerlige Ramappa]

On Thu, Jul 11, 2024 at 08:40:46AM -0500, Lucas De Marchi wrote:
>On Tue, Jul 02, 2024 at 05:25:25PM GMT, Umesh Nerlige Ramappa wrote:
>>Add a helper that captures utilization for all classes.
>>
>>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>>---
>>tests/intel/xe_drm_fdinfo.c | 20 ++++++++++++++++++++
>>1 file changed, 20 insertions(+)
>>
>>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>>index b64bef339..c697f3e53 100644
>>--- a/tests/intel/xe_drm_fdinfo.c
>>+++ b/tests/intel/xe_drm_fdinfo.c
>>@@ -42,6 +42,11 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
>>
>>#define BO_SIZE (65536)
>>
>>+struct pceu_cycles {
>>+	uint64_t cycles;
>>+	uint64_t total_cycles;
>>+};
>>+
>>static const char *engine_map[] = {
>>	"rcs",
>>	"bcs",
>>@@ -49,6 +54,21 @@ static const char *engine_map[] = {
>>	"vecs",
>>	"ccs",
>>};
>
>nit: missing newline here, otherwise:

will add

>
>>+static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
>
>I think that would be better to give the compiler a hint about the
>size of the array:
>
>static void read_engine_cycles(int xe,
>			       struct pceu_cycles pceu[static DRM_XE_ENGINE_CLASS_COMPUTE + 1])

Never knew static could be placed there. Will add.

>
>(it would not be a bad idea to have a local __IGT_DRM_XE_ENGINE_CLASS_LAST
>too, but outside of the scope here)
>
>Otherwise,
>
>// Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>

Thanks,
Umesh
>
>Lucas De Marchi
>
>>+{
>>+	struct drm_client_fdinfo info = { };
>>+	int class;
>>+
>>+	igt_assert(pceu);
>>+	igt_assert(igt_parse_drm_fdinfo(xe, &info, engine_map,
>>+					ARRAY_SIZE(engine_map), NULL, 0));
>>+
>>+	xe_for_each_engine_class(class) {
>>+		pceu[class].cycles = info.cycles[class];
>>+		pceu[class].total_cycles = info.total_cycles[class];
>>+	}
>>+}
>>+
>>/* Subtests */
>>static void test_active(int fd, struct drm_xe_engine *engine)
>>{
>>-- 
>>2.38.1
>>

Re: [PATCH i-g-t v2 03/10] tests/intel/xe_drm_fdinfo: Add helper to read utilization for all classes

[Lucas De Marchi]

On Thu, Jul 11, 2024 at 12:54:49PM GMT, Umesh Nerlige Ramappa wrote:
>On Thu, Jul 11, 2024 at 08:40:46AM -0500, Lucas De Marchi wrote:
>>On Tue, Jul 02, 2024 at 05:25:25PM GMT, Umesh Nerlige Ramappa wrote:
>>>Add a helper that captures utilization for all classes.
>>>
>>>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>>>---
>>>tests/intel/xe_drm_fdinfo.c | 20 ++++++++++++++++++++
>>>1 file changed, 20 insertions(+)
>>>
>>>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>>>index b64bef339..c697f3e53 100644
>>>--- a/tests/intel/xe_drm_fdinfo.c
>>>+++ b/tests/intel/xe_drm_fdinfo.c
>>>@@ -42,6 +42,11 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
>>>
>>>#define BO_SIZE (65536)
>>>
>>>+struct pceu_cycles {
>>>+	uint64_t cycles;
>>>+	uint64_t total_cycles;
>>>+};
>>>+
>>>static const char *engine_map[] = {
>>>	"rcs",
>>>	"bcs",
>>>@@ -49,6 +54,21 @@ static const char *engine_map[] = {
>>>	"vecs",
>>>	"ccs",
>>>};
>>
>>nit: missing newline here, otherwise:
>
>will add
>
>>
>>>+static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
>>
>>I think that would be better to give the compiler a hint about the
>>size of the array:
>>
>>static void read_engine_cycles(int xe,
>>			       struct pceu_cycles pceu[static DRM_XE_ENGINE_CLASS_COMPUTE + 1])
>
>Never knew static could be placed there. Will add.
>
>>
>>(it would not be a bad idea to have a local __IGT_DRM_XE_ENGINE_CLASS_LAST
>>too, but outside of the scope here)
>>
>>Otherwise,
>>
>>// Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>

I was checking if we could apply this and rework on top. But both this
and the next commit have the same issue of adding a static helper
without a real caller (so we get warnings).  I will keep looking if I
can split this differently to get things going.

Lucas De Marchi

>
>Thanks,
>Umesh
>>
>>Lucas De Marchi
>>
>>>+{
>>>+	struct drm_client_fdinfo info = { };
>>>+	int class;
>>>+
>>>+	igt_assert(pceu);
>>>+	igt_assert(igt_parse_drm_fdinfo(xe, &info, engine_map,
>>>+					ARRAY_SIZE(engine_map), NULL, 0));
>>>+
>>>+	xe_for_each_engine_class(class) {
>>>+		pceu[class].cycles = info.cycles[class];
>>>+		pceu[class].total_cycles = info.total_cycles[class];
>>>+	}
>>>+}
>>>+
>>>/* Subtests */
>>>static void test_active(int fd, struct drm_xe_engine *engine)
>>>{
>>>-- 
>>>2.38.1
>>>

[PATCH i-g-t v2 04/10] tests/intel/xe_drm_fdinfo: Add helpers for spinning batches

[Umesh Nerlige Ramappa]

Add helpers for submitting batches and waiting for them to start.

v2: Remove xe prefixes from the structures and helpers (Lucas)

Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
---
 tests/intel/xe_drm_fdinfo.c | 135 ++++++++++++++++++++++++++++++++++++
 1 file changed, 135 insertions(+)

diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
index c697f3e53..037f25e53 100644
--- a/tests/intel/xe_drm_fdinfo.c
+++ b/tests/intel/xe_drm_fdinfo.c
@@ -54,6 +54,17 @@ static const char *engine_map[] = {
 	"vecs",
 	"ccs",
 };
+
+static const uint64_t batch_addr[] = {
+	0x170000,
+	0x180000,
+	0x190000,
+	0x1a0000,
+	0x1b0000,
+	0x1c0000,
+	0x1d0000,
+	0x1e0000,
+};
 static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
 {
 	struct drm_client_fdinfo info = { };
@@ -329,6 +340,130 @@ static void basic_engine_utilization(int xe)
 	igt_require(info.num_engines);
 }
 
+#define MAX_PARALLEL 8
+struct spin_ctx {
+	uint32_t vm;
+	uint64_t addr[MAX_PARALLEL];
+	struct drm_xe_sync sync[2];
+	struct drm_xe_exec exec;
+	uint32_t exec_queue;
+	size_t bo_size;
+	uint32_t bo;
+	struct xe_spin *spin;
+	struct xe_spin_opts spin_opts;
+	bool ended;
+	uint16_t class;
+	uint16_t width;
+	uint16_t num_placements;
+};
+
+static struct spin_ctx *
+spin_ctx_init(int fd, struct drm_xe_engine_class_instance *hwe, uint32_t vm,
+	      uint16_t width, uint16_t num_placements)
+{
+	struct spin_ctx *ctx = calloc(1, sizeof(*ctx));
+
+	igt_assert(width && num_placements &&
+		   (width == 1 || num_placements == 1));
+
+	igt_assert(width <= MAX_PARALLEL);
+
+	ctx->class = hwe->engine_class;
+	ctx->width = width;
+	ctx->num_placements = num_placements;
+	ctx->vm = vm;
+	for (int i = 0; i < ctx->width; i++)
+		ctx->addr[i] = batch_addr[hwe->engine_class];
+
+	ctx->exec.num_batch_buffer = width;
+	ctx->exec.num_syncs = 2;
+	ctx->exec.syncs = to_user_pointer(ctx->sync);
+
+	ctx->sync[0].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
+	ctx->sync[0].flags = DRM_XE_SYNC_FLAG_SIGNAL;
+	ctx->sync[0].handle = syncobj_create(fd, 0);
+
+	ctx->sync[1].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
+	ctx->sync[1].flags = DRM_XE_SYNC_FLAG_SIGNAL;
+	ctx->sync[1].handle = syncobj_create(fd, 0);
+
+	ctx->bo_size = sizeof(struct xe_spin);
+	ctx->bo_size = xe_bb_size(fd, ctx->bo_size);
+	ctx->bo = xe_bo_create(fd, ctx->vm, ctx->bo_size,
+			       vram_if_possible(fd, hwe->gt_id),
+			       DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM);
+	ctx->spin = xe_bo_map(fd, ctx->bo, ctx->bo_size);
+
+	igt_assert_eq(__xe_exec_queue_create(fd, ctx->vm, width, num_placements,
+					     hwe, 0, &ctx->exec_queue), 0);
+
+	xe_vm_bind_async(fd, ctx->vm, 0, ctx->bo, 0, ctx->addr[0], ctx->bo_size,
+			 ctx->sync, 1);
+
+	return ctx;
+}
+
+static void
+spin_sync_start(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx)
+		return;
+
+	ctx->spin_opts.addr = ctx->addr[0];
+	ctx->spin_opts.preempt = true;
+	xe_spin_init(ctx->spin, &ctx->spin_opts);
+
+	/* re-use sync[0] for exec */
+	ctx->sync[0].flags &= ~DRM_XE_SYNC_FLAG_SIGNAL;
+
+	ctx->exec.exec_queue_id = ctx->exec_queue;
+	if (ctx->width > 1)
+		ctx->exec.address = to_user_pointer(ctx->addr);
+	else
+		ctx->exec.address = ctx->addr[0];
+	xe_exec(fd, &ctx->exec);
+
+	xe_spin_wait_started(ctx->spin);
+	igt_assert(!syncobj_wait(fd, &ctx->sync[1].handle, 1, 1, 0, NULL));
+
+	igt_debug("%s: spinner started\n", engine_map[ctx->class]);
+}
+
+static void
+spin_sync_end(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx || ctx->ended)
+		return;
+
+	xe_spin_end(ctx->spin);
+
+	igt_assert(syncobj_wait(fd, &ctx->sync[1].handle, 1, INT64_MAX, 0, NULL));
+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
+
+	ctx->sync[0].flags |= DRM_XE_SYNC_FLAG_SIGNAL;
+	xe_vm_unbind_async(fd, ctx->vm, 0, 0, ctx->addr[0], ctx->bo_size, ctx->sync, 1);
+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
+
+	ctx->ended = true;
+	igt_debug("%s: spinner ended\n", engine_map[ctx->class]);
+}
+
+static void
+spin_ctx_destroy(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx)
+		return;
+
+	syncobj_destroy(fd, ctx->sync[0].handle);
+	syncobj_destroy(fd, ctx->sync[1].handle);
+	xe_exec_queue_destroy(fd, ctx->exec_queue);
+
+	munmap(ctx->spin, ctx->bo_size);
+	gem_close(fd, ctx->bo);
+
+	free(ctx);
+}
+
 igt_main
 {
 	int xe;
-- 
2.38.1

Re: [PATCH i-g-t v2 04/10] tests/intel/xe_drm_fdinfo: Add helpers for spinning batches

[Lucas De Marchi]

On Tue, Jul 02, 2024 at 05:25:26PM GMT, Umesh Nerlige Ramappa wrote:
>Add helpers for submitting batches and waiting for them to start.
>
>v2: Remove xe prefixes from the structures and helpers (Lucas)
>
>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>---
> tests/intel/xe_drm_fdinfo.c | 135 ++++++++++++++++++++++++++++++++++++
> 1 file changed, 135 insertions(+)
>
>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>index c697f3e53..037f25e53 100644
>--- a/tests/intel/xe_drm_fdinfo.c
>+++ b/tests/intel/xe_drm_fdinfo.c
>@@ -54,6 +54,17 @@ static const char *engine_map[] = {
> 	"vecs",
> 	"ccs",
> };
>+
>+static const uint64_t batch_addr[] = {
>+	0x170000,
>+	0x180000,
>+	0x190000,
>+	0x1a0000,
>+	0x1b0000,
>+	0x1c0000,
>+	0x1d0000,
>+	0x1e0000,
>+};

where are these numbers coming from? I'm looking at
tests/intel/xe_spin_batch.c and the lib/xe/xe_spin.c implementation and
I don't understand why we need to add so many helpers and handcrafted
things in this particular test.

+Matt Brost, +Zbigniew Kempczyński to help help here


Lucas De Marchi

> static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
> {
> 	struct drm_client_fdinfo info = { };
>@@ -329,6 +340,130 @@ static void basic_engine_utilization(int xe)
> 	igt_require(info.num_engines);
> }
>
>+#define MAX_PARALLEL 8
>+struct spin_ctx {
>+	uint32_t vm;
>+	uint64_t addr[MAX_PARALLEL];
>+	struct drm_xe_sync sync[2];
>+	struct drm_xe_exec exec;
>+	uint32_t exec_queue;
>+	size_t bo_size;
>+	uint32_t bo;
>+	struct xe_spin *spin;
>+	struct xe_spin_opts spin_opts;
>+	bool ended;
>+	uint16_t class;
>+	uint16_t width;
>+	uint16_t num_placements;
>+};
>+
>+static struct spin_ctx *
>+spin_ctx_init(int fd, struct drm_xe_engine_class_instance *hwe, uint32_t vm,
>+	      uint16_t width, uint16_t num_placements)
>+{
>+	struct spin_ctx *ctx = calloc(1, sizeof(*ctx));
>+
>+	igt_assert(width && num_placements &&
>+		   (width == 1 || num_placements == 1));
>+
>+	igt_assert(width <= MAX_PARALLEL);
>+
>+	ctx->class = hwe->engine_class;
>+	ctx->width = width;
>+	ctx->num_placements = num_placements;
>+	ctx->vm = vm;
>+	for (int i = 0; i < ctx->width; i++)
>+		ctx->addr[i] = batch_addr[hwe->engine_class];
>+
>+	ctx->exec.num_batch_buffer = width;
>+	ctx->exec.num_syncs = 2;
>+	ctx->exec.syncs = to_user_pointer(ctx->sync);
>+
>+	ctx->sync[0].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
>+	ctx->sync[0].flags = DRM_XE_SYNC_FLAG_SIGNAL;
>+	ctx->sync[0].handle = syncobj_create(fd, 0);
>+
>+	ctx->sync[1].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
>+	ctx->sync[1].flags = DRM_XE_SYNC_FLAG_SIGNAL;
>+	ctx->sync[1].handle = syncobj_create(fd, 0);
>+
>+	ctx->bo_size = sizeof(struct xe_spin);
>+	ctx->bo_size = xe_bb_size(fd, ctx->bo_size);
>+	ctx->bo = xe_bo_create(fd, ctx->vm, ctx->bo_size,
>+			       vram_if_possible(fd, hwe->gt_id),
>+			       DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM);
>+	ctx->spin = xe_bo_map(fd, ctx->bo, ctx->bo_size);
>+
>+	igt_assert_eq(__xe_exec_queue_create(fd, ctx->vm, width, num_placements,
>+					     hwe, 0, &ctx->exec_queue), 0);
>+
>+	xe_vm_bind_async(fd, ctx->vm, 0, ctx->bo, 0, ctx->addr[0], ctx->bo_size,
>+			 ctx->sync, 1);
>+
>+	return ctx;
>+}
>+
>+static void
>+spin_sync_start(int fd, struct spin_ctx *ctx)
>+{
>+	if (!ctx)
>+		return;
>+
>+	ctx->spin_opts.addr = ctx->addr[0];
>+	ctx->spin_opts.preempt = true;
>+	xe_spin_init(ctx->spin, &ctx->spin_opts);
>+
>+	/* re-use sync[0] for exec */
>+	ctx->sync[0].flags &= ~DRM_XE_SYNC_FLAG_SIGNAL;
>+
>+	ctx->exec.exec_queue_id = ctx->exec_queue;
>+	if (ctx->width > 1)
>+		ctx->exec.address = to_user_pointer(ctx->addr);
>+	else
>+		ctx->exec.address = ctx->addr[0];
>+	xe_exec(fd, &ctx->exec);
>+
>+	xe_spin_wait_started(ctx->spin);
>+	igt_assert(!syncobj_wait(fd, &ctx->sync[1].handle, 1, 1, 0, NULL));
>+
>+	igt_debug("%s: spinner started\n", engine_map[ctx->class]);
>+}
>+
>+static void
>+spin_sync_end(int fd, struct spin_ctx *ctx)
>+{
>+	if (!ctx || ctx->ended)
>+		return;
>+
>+	xe_spin_end(ctx->spin);
>+
>+	igt_assert(syncobj_wait(fd, &ctx->sync[1].handle, 1, INT64_MAX, 0, NULL));
>+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
>+
>+	ctx->sync[0].flags |= DRM_XE_SYNC_FLAG_SIGNAL;
>+	xe_vm_unbind_async(fd, ctx->vm, 0, 0, ctx->addr[0], ctx->bo_size, ctx->sync, 1);
>+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
>+
>+	ctx->ended = true;
>+	igt_debug("%s: spinner ended\n", engine_map[ctx->class]);
>+}
>+
>+static void
>+spin_ctx_destroy(int fd, struct spin_ctx *ctx)
>+{
>+	if (!ctx)
>+		return;
>+
>+	syncobj_destroy(fd, ctx->sync[0].handle);
>+	syncobj_destroy(fd, ctx->sync[1].handle);
>+	xe_exec_queue_destroy(fd, ctx->exec_queue);
>+
>+	munmap(ctx->spin, ctx->bo_size);
>+	gem_close(fd, ctx->bo);
>+
>+	free(ctx);
>+}
>+
> igt_main
> {
> 	int xe;
>-- 
>2.38.1
>

Re: [PATCH i-g-t v2 04/10] tests/intel/xe_drm_fdinfo: Add helpers for spinning batches

[Umesh Nerlige Ramappa]

On Thu, Jul 11, 2024 at 07:46:23AM -0500, Lucas De Marchi wrote:
>On Tue, Jul 02, 2024 at 05:25:26PM GMT, Umesh Nerlige Ramappa wrote:
>>Add helpers for submitting batches and waiting for them to start.
>>
>>v2: Remove xe prefixes from the structures and helpers (Lucas)
>>
>>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>>---
>>tests/intel/xe_drm_fdinfo.c | 135 ++++++++++++++++++++++++++++++++++++
>>1 file changed, 135 insertions(+)
>>
>>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>>index c697f3e53..037f25e53 100644
>>--- a/tests/intel/xe_drm_fdinfo.c
>>+++ b/tests/intel/xe_drm_fdinfo.c
>>@@ -54,6 +54,17 @@ static const char *engine_map[] = {
>>	"vecs",
>>	"ccs",
>>};
>>+
>>+static const uint64_t batch_addr[] = {
>>+	0x170000,
>>+	0x180000,
>>+	0x190000,
>>+	0x1a0000,
>>+	0x1b0000,
>>+	0x1c0000,
>>+	0x1d0000,
>>+	0x1e0000,
>>+};
>
>where are these numbers coming from?

The hardcoded number above are arbitrary, although I don't know if 
that's an issue. I did see similar numbers in use with other xe tests.

>I'm looking at
>tests/intel/xe_spin_batch.c and the lib/xe/xe_spin.c implementation and
>I don't understand why we need to add so many helpers and handcrafted
>things in this particular test.

These helpers are just a break down of something like xe_exec_balancer.c 
-> test_exec(). The helpers are still using functions from xe_spin.c.  

The breakdown helps control some execution elements for batches like (1) 
using a single address space when running batches on all classes, (2) 
configuring width and num_placements for parallel/virtual cases and (3) 
provide some flexibility to test utilization when the batch has started 
running.

Regards,
Umesh

>
>+Matt Brost, +Zbigniew Kempczyński to help help here
>
>
>Lucas De Marchi
>
>>static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
>>{
>>	struct drm_client_fdinfo info = { };
>>@@ -329,6 +340,130 @@ static void basic_engine_utilization(int xe)
>>	igt_require(info.num_engines);
>>}
>>
>>+#define MAX_PARALLEL 8
>>+struct spin_ctx {
>>+	uint32_t vm;
>>+	uint64_t addr[MAX_PARALLEL];
>>+	struct drm_xe_sync sync[2];
>>+	struct drm_xe_exec exec;
>>+	uint32_t exec_queue;
>>+	size_t bo_size;
>>+	uint32_t bo;
>>+	struct xe_spin *spin;
>>+	struct xe_spin_opts spin_opts;
>>+	bool ended;
>>+	uint16_t class;
>>+	uint16_t width;
>>+	uint16_t num_placements;
>>+};
>>+
>>+static struct spin_ctx *
>>+spin_ctx_init(int fd, struct drm_xe_engine_class_instance *hwe, uint32_t vm,
>>+	      uint16_t width, uint16_t num_placements)
>>+{
>>+	struct spin_ctx *ctx = calloc(1, sizeof(*ctx));
>>+
>>+	igt_assert(width && num_placements &&
>>+		   (width == 1 || num_placements == 1));
>>+
>>+	igt_assert(width <= MAX_PARALLEL);
>>+
>>+	ctx->class = hwe->engine_class;
>>+	ctx->width = width;
>>+	ctx->num_placements = num_placements;
>>+	ctx->vm = vm;
>>+	for (int i = 0; i < ctx->width; i++)
>>+		ctx->addr[i] = batch_addr[hwe->engine_class];
>>+
>>+	ctx->exec.num_batch_buffer = width;
>>+	ctx->exec.num_syncs = 2;
>>+	ctx->exec.syncs = to_user_pointer(ctx->sync);
>>+
>>+	ctx->sync[0].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
>>+	ctx->sync[0].flags = DRM_XE_SYNC_FLAG_SIGNAL;
>>+	ctx->sync[0].handle = syncobj_create(fd, 0);
>>+
>>+	ctx->sync[1].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
>>+	ctx->sync[1].flags = DRM_XE_SYNC_FLAG_SIGNAL;
>>+	ctx->sync[1].handle = syncobj_create(fd, 0);
>>+
>>+	ctx->bo_size = sizeof(struct xe_spin);
>>+	ctx->bo_size = xe_bb_size(fd, ctx->bo_size);
>>+	ctx->bo = xe_bo_create(fd, ctx->vm, ctx->bo_size,
>>+			       vram_if_possible(fd, hwe->gt_id),
>>+			       DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM);
>>+	ctx->spin = xe_bo_map(fd, ctx->bo, ctx->bo_size);
>>+
>>+	igt_assert_eq(__xe_exec_queue_create(fd, ctx->vm, width, num_placements,
>>+					     hwe, 0, &ctx->exec_queue), 0);
>>+
>>+	xe_vm_bind_async(fd, ctx->vm, 0, ctx->bo, 0, ctx->addr[0], ctx->bo_size,
>>+			 ctx->sync, 1);
>>+
>>+	return ctx;
>>+}
>>+
>>+static void
>>+spin_sync_start(int fd, struct spin_ctx *ctx)
>>+{
>>+	if (!ctx)
>>+		return;
>>+
>>+	ctx->spin_opts.addr = ctx->addr[0];
>>+	ctx->spin_opts.preempt = true;
>>+	xe_spin_init(ctx->spin, &ctx->spin_opts);
>>+
>>+	/* re-use sync[0] for exec */
>>+	ctx->sync[0].flags &= ~DRM_XE_SYNC_FLAG_SIGNAL;
>>+
>>+	ctx->exec.exec_queue_id = ctx->exec_queue;
>>+	if (ctx->width > 1)
>>+		ctx->exec.address = to_user_pointer(ctx->addr);
>>+	else
>>+		ctx->exec.address = ctx->addr[0];
>>+	xe_exec(fd, &ctx->exec);
>>+
>>+	xe_spin_wait_started(ctx->spin);
>>+	igt_assert(!syncobj_wait(fd, &ctx->sync[1].handle, 1, 1, 0, NULL));
>>+
>>+	igt_debug("%s: spinner started\n", engine_map[ctx->class]);
>>+}
>>+
>>+static void
>>+spin_sync_end(int fd, struct spin_ctx *ctx)
>>+{
>>+	if (!ctx || ctx->ended)
>>+		return;
>>+
>>+	xe_spin_end(ctx->spin);
>>+
>>+	igt_assert(syncobj_wait(fd, &ctx->sync[1].handle, 1, INT64_MAX, 0, NULL));
>>+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
>>+
>>+	ctx->sync[0].flags |= DRM_XE_SYNC_FLAG_SIGNAL;
>>+	xe_vm_unbind_async(fd, ctx->vm, 0, 0, ctx->addr[0], ctx->bo_size, ctx->sync, 1);
>>+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
>>+
>>+	ctx->ended = true;
>>+	igt_debug("%s: spinner ended\n", engine_map[ctx->class]);
>>+}
>>+
>>+static void
>>+spin_ctx_destroy(int fd, struct spin_ctx *ctx)
>>+{
>>+	if (!ctx)
>>+		return;
>>+
>>+	syncobj_destroy(fd, ctx->sync[0].handle);
>>+	syncobj_destroy(fd, ctx->sync[1].handle);
>>+	xe_exec_queue_destroy(fd, ctx->exec_queue);
>>+
>>+	munmap(ctx->spin, ctx->bo_size);
>>+	gem_close(fd, ctx->bo);
>>+
>>+	free(ctx);
>>+}
>>+
>>igt_main
>>{
>>	int xe;
>>-- 
>>2.38.1
>>

Re: [PATCH i-g-t v2 04/10] tests/intel/xe_drm_fdinfo: Add helpers for spinning batches

[Matthew Brost]

On Thu, Jul 11, 2024 at 11:29:00AM -0700, Umesh Nerlige Ramappa wrote:
> On Thu, Jul 11, 2024 at 07:46:23AM -0500, Lucas De Marchi wrote:
> > On Tue, Jul 02, 2024 at 05:25:26PM GMT, Umesh Nerlige Ramappa wrote:
> > > Add helpers for submitting batches and waiting for them to start.
> > > 
> > > v2: Remove xe prefixes from the structures and helpers (Lucas)
> > > 
> > > Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
> > > ---
> > > tests/intel/xe_drm_fdinfo.c | 135 ++++++++++++++++++++++++++++++++++++
> > > 1 file changed, 135 insertions(+)
> > > 
> > > diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
> > > index c697f3e53..037f25e53 100644
> > > --- a/tests/intel/xe_drm_fdinfo.c
> > > +++ b/tests/intel/xe_drm_fdinfo.c
> > > @@ -54,6 +54,17 @@ static const char *engine_map[] = {
> > > 	"vecs",
> > > 	"ccs",
> > > };
> > > +
> > > +static const uint64_t batch_addr[] = {
> > > +	0x170000,
> > > +	0x180000,
> > > +	0x190000,
> > > +	0x1a0000,
> > > +	0x1b0000,
> > > +	0x1c0000,
> > > +	0x1d0000,
> > > +	0x1e0000,
> > > +};
> > 
> > where are these numbers coming from?
> 
> The hardcoded number above are arbitrary, although I don't know if that's an
> issue. I did see similar numbers in use with other xe tests.
>

See below, I think you can just use a single address when setting up the
cork (see below).
 
> > I'm looking at
> > tests/intel/xe_spin_batch.c and the lib/xe/xe_spin.c implementation and
> > I don't understand why we need to add so many helpers and handcrafted
> > things in this particular test.
> 
> These helpers are just a break down of something like xe_exec_balancer.c ->
> test_exec(). The helpers are still using functions from xe_spin.c.
> 
> The breakdown helps control some execution elements for batches like (1)
> using a single address space when running batches on all classes, (2)
> configuring width and num_placements for parallel/virtual cases and (3)
> provide some flexibility to test utilization when the batch has started
> running.
> 

I see what you are trying to do, create spinners for virtual and
parallel exec queues.

I think the proper place to put this would be in xe_spin.c adding 1 or
2 new functions that look like xe_cork_init() but for virtual and
parallel exec queues. Then just use existing cork functions for
everything else.

I think that will work.

I don't think any IGTs actually use the cork stuff yet (several probably
should) but I did post an IGT recently [1] that made use of this.

Matt

[1] https://patchwork.freedesktop.org/patch/600254/?series=135143&rev=1

> Regards,
> Umesh
> 
> > 
> > +Matt Brost, +Zbigniew Kempczyński to help help here
> > 
> > 
> > Lucas De Marchi
> > 
> > > static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
> > > {
> > > 	struct drm_client_fdinfo info = { };
> > > @@ -329,6 +340,130 @@ static void basic_engine_utilization(int xe)
> > > 	igt_require(info.num_engines);
> > > }
> > > 
> > > +#define MAX_PARALLEL 8
> > > +struct spin_ctx {
> > > +	uint32_t vm;
> > > +	uint64_t addr[MAX_PARALLEL];
> > > +	struct drm_xe_sync sync[2];
> > > +	struct drm_xe_exec exec;
> > > +	uint32_t exec_queue;
> > > +	size_t bo_size;
> > > +	uint32_t bo;
> > > +	struct xe_spin *spin;
> > > +	struct xe_spin_opts spin_opts;
> > > +	bool ended;
> > > +	uint16_t class;
> > > +	uint16_t width;
> > > +	uint16_t num_placements;
> > > +};
> > > +
> > > +static struct spin_ctx *
> > > +spin_ctx_init(int fd, struct drm_xe_engine_class_instance *hwe, uint32_t vm,
> > > +	      uint16_t width, uint16_t num_placements)
> > > +{
> > > +	struct spin_ctx *ctx = calloc(1, sizeof(*ctx));
> > > +
> > > +	igt_assert(width && num_placements &&
> > > +		   (width == 1 || num_placements == 1));
> > > +
> > > +	igt_assert(width <= MAX_PARALLEL);
> > > +
> > > +	ctx->class = hwe->engine_class;
> > > +	ctx->width = width;
> > > +	ctx->num_placements = num_placements;
> > > +	ctx->vm = vm;
> > > +	for (int i = 0; i < ctx->width; i++)
> > > +		ctx->addr[i] = batch_addr[hwe->engine_class];
> > > +
> > > +	ctx->exec.num_batch_buffer = width;
> > > +	ctx->exec.num_syncs = 2;
> > > +	ctx->exec.syncs = to_user_pointer(ctx->sync);
> > > +
> > > +	ctx->sync[0].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
> > > +	ctx->sync[0].flags = DRM_XE_SYNC_FLAG_SIGNAL;
> > > +	ctx->sync[0].handle = syncobj_create(fd, 0);
> > > +
> > > +	ctx->sync[1].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
> > > +	ctx->sync[1].flags = DRM_XE_SYNC_FLAG_SIGNAL;
> > > +	ctx->sync[1].handle = syncobj_create(fd, 0);
> > > +
> > > +	ctx->bo_size = sizeof(struct xe_spin);
> > > +	ctx->bo_size = xe_bb_size(fd, ctx->bo_size);
> > > +	ctx->bo = xe_bo_create(fd, ctx->vm, ctx->bo_size,
> > > +			       vram_if_possible(fd, hwe->gt_id),
> > > +			       DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM);
> > > +	ctx->spin = xe_bo_map(fd, ctx->bo, ctx->bo_size);
> > > +
> > > +	igt_assert_eq(__xe_exec_queue_create(fd, ctx->vm, width, num_placements,
> > > +					     hwe, 0, &ctx->exec_queue), 0);
> > > +
> > > +	xe_vm_bind_async(fd, ctx->vm, 0, ctx->bo, 0, ctx->addr[0], ctx->bo_size,
> > > +			 ctx->sync, 1);
> > > +
> > > +	return ctx;
> > > +}
> > > +
> > > +static void
> > > +spin_sync_start(int fd, struct spin_ctx *ctx)
> > > +{
> > > +	if (!ctx)
> > > +		return;
> > > +
> > > +	ctx->spin_opts.addr = ctx->addr[0];
> > > +	ctx->spin_opts.preempt = true;
> > > +	xe_spin_init(ctx->spin, &ctx->spin_opts);
> > > +
> > > +	/* re-use sync[0] for exec */
> > > +	ctx->sync[0].flags &= ~DRM_XE_SYNC_FLAG_SIGNAL;
> > > +
> > > +	ctx->exec.exec_queue_id = ctx->exec_queue;
> > > +	if (ctx->width > 1)
> > > +		ctx->exec.address = to_user_pointer(ctx->addr);
> > > +	else
> > > +		ctx->exec.address = ctx->addr[0];
> > > +	xe_exec(fd, &ctx->exec);
> > > +
> > > +	xe_spin_wait_started(ctx->spin);
> > > +	igt_assert(!syncobj_wait(fd, &ctx->sync[1].handle, 1, 1, 0, NULL));
> > > +
> > > +	igt_debug("%s: spinner started\n", engine_map[ctx->class]);
> > > +}
> > > +
> > > +static void
> > > +spin_sync_end(int fd, struct spin_ctx *ctx)
> > > +{
> > > +	if (!ctx || ctx->ended)
> > > +		return;
> > > +
> > > +	xe_spin_end(ctx->spin);
> > > +
> > > +	igt_assert(syncobj_wait(fd, &ctx->sync[1].handle, 1, INT64_MAX, 0, NULL));
> > > +	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
> > > +
> > > +	ctx->sync[0].flags |= DRM_XE_SYNC_FLAG_SIGNAL;
> > > +	xe_vm_unbind_async(fd, ctx->vm, 0, 0, ctx->addr[0], ctx->bo_size, ctx->sync, 1);
> > > +	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
> > > +
> > > +	ctx->ended = true;
> > > +	igt_debug("%s: spinner ended\n", engine_map[ctx->class]);
> > > +}
> > > +
> > > +static void
> > > +spin_ctx_destroy(int fd, struct spin_ctx *ctx)
> > > +{
> > > +	if (!ctx)
> > > +		return;
> > > +
> > > +	syncobj_destroy(fd, ctx->sync[0].handle);
> > > +	syncobj_destroy(fd, ctx->sync[1].handle);
> > > +	xe_exec_queue_destroy(fd, ctx->exec_queue);
> > > +
> > > +	munmap(ctx->spin, ctx->bo_size);
> > > +	gem_close(fd, ctx->bo);
> > > +
> > > +	free(ctx);
> > > +}
> > > +
> > > igt_main
> > > {
> > > 	int xe;
> > > -- 
> > > 2.38.1
> > > 

Re: [PATCH i-g-t v2 04/10] tests/intel/xe_drm_fdinfo: Add helpers for spinning batches

[Lucas De Marchi]

On Fri, Jul 12, 2024 at 12:04:31AM GMT, Matthew Brost wrote:
>On Thu, Jul 11, 2024 at 11:29:00AM -0700, Umesh Nerlige Ramappa wrote:
>> On Thu, Jul 11, 2024 at 07:46:23AM -0500, Lucas De Marchi wrote:
>> > On Tue, Jul 02, 2024 at 05:25:26PM GMT, Umesh Nerlige Ramappa wrote:
>> > > Add helpers for submitting batches and waiting for them to start.
>> > >
>> > > v2: Remove xe prefixes from the structures and helpers (Lucas)
>> > >
>> > > Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>> > > ---
>> > > tests/intel/xe_drm_fdinfo.c | 135 ++++++++++++++++++++++++++++++++++++
>> > > 1 file changed, 135 insertions(+)
>> > >
>> > > diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>> > > index c697f3e53..037f25e53 100644
>> > > --- a/tests/intel/xe_drm_fdinfo.c
>> > > +++ b/tests/intel/xe_drm_fdinfo.c
>> > > @@ -54,6 +54,17 @@ static const char *engine_map[] = {
>> > > 	"vecs",
>> > > 	"ccs",
>> > > };
>> > > +
>> > > +static const uint64_t batch_addr[] = {
>> > > +	0x170000,
>> > > +	0x180000,
>> > > +	0x190000,
>> > > +	0x1a0000,
>> > > +	0x1b0000,
>> > > +	0x1c0000,
>> > > +	0x1d0000,
>> > > +	0x1e0000,
>> > > +};
>> >
>> > where are these numbers coming from?
>>
>> The hardcoded number above are arbitrary, although I don't know if that's an
>> issue. I did see similar numbers in use with other xe tests.
>>
>
>See below, I think you can just use a single address when setting up the
>cork (see below).

because each cork uses a different vm rather than the shared vm used
here, right?

>
>> > I'm looking at
>> > tests/intel/xe_spin_batch.c and the lib/xe/xe_spin.c implementation and
>> > I don't understand why we need to add so many helpers and handcrafted
>> > things in this particular test.
>>
>> These helpers are just a break down of something like xe_exec_balancer.c ->
>> test_exec(). The helpers are still using functions from xe_spin.c.
>>
>> The breakdown helps control some execution elements for batches like (1)
>> using a single address space when running batches on all classes, (2)
>> configuring width and num_placements for parallel/virtual cases and (3)
>> provide some flexibility to test utilization when the batch has started
>> running.
>>
>
>I see what you are trying to do, create spinners for virtual and
>parallel exec queues.
>
>I think the proper place to put this would be in xe_spin.c adding 1 or
>2 new functions that look like xe_cork_init() but for virtual and
>parallel exec queues. Then just use existing cork functions for
>everything else.
>
>I think that will work.

It seems like one problem with cork is that it exec on init.  Maybe
adding a xe_cork_start_sync() so we can separate the setup part from the
start part would be ok?

We could do _init() + _wait_started(), but with the current state of the
test we are trying to minimize the jitter to calculate the time in which
it ran. Although I was thinking... if we stop checking the cpu time and
only rely on the gpu time we collect from fdinfo the igt asserts should
be more reliable. And another thing we could use is the time saved by
the gpu in the spin itself.

I sent a few patches to better abstract the parallel/virtual setup,
similarly to what you suggested in another thread:
https://patchwork.freedesktop.org/series/137317/

I think that should mostly take care about the parallel/virtual tests
added here.  For the single() ones, I squashed locally the patches 3-5
(because otherwise they create warnings) and I'm thinking on merging
them mostly as is. Then I add the missing cork stuff and migrate the
test to use that instead of the local functions. Thoughts?

Long term I think we could do:

	s/xe_spin/xe_spin_bath/
	s/xe_cork/xe_spinner/

Thoughts?

Thanks
Lucas De Marchi

>
>I don't think any IGTs actually use the cork stuff yet (several probably
>should) but I did post an IGT recently [1] that made use of this.
>
>Matt
>
>[1] https://patchwork.freedesktop.org/patch/600254/?series=135143&rev=1
>
>> Regards,
>> Umesh
>>
>> >
>> > +Matt Brost, +Zbigniew Kempczyński to help help here
>> >
>> >
>> > Lucas De Marchi
>> >
>> > > static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
>> > > {
>> > > 	struct drm_client_fdinfo info = { };
>> > > @@ -329,6 +340,130 @@ static void basic_engine_utilization(int xe)
>> > > 	igt_require(info.num_engines);
>> > > }
>> > >
>> > > +#define MAX_PARALLEL 8
>> > > +struct spin_ctx {
>> > > +	uint32_t vm;
>> > > +	uint64_t addr[MAX_PARALLEL];
>> > > +	struct drm_xe_sync sync[2];
>> > > +	struct drm_xe_exec exec;
>> > > +	uint32_t exec_queue;
>> > > +	size_t bo_size;
>> > > +	uint32_t bo;
>> > > +	struct xe_spin *spin;
>> > > +	struct xe_spin_opts spin_opts;
>> > > +	bool ended;
>> > > +	uint16_t class;
>> > > +	uint16_t width;
>> > > +	uint16_t num_placements;
>> > > +};
>> > > +
>> > > +static struct spin_ctx *
>> > > +spin_ctx_init(int fd, struct drm_xe_engine_class_instance *hwe, uint32_t vm,
>> > > +	      uint16_t width, uint16_t num_placements)
>> > > +{
>> > > +	struct spin_ctx *ctx = calloc(1, sizeof(*ctx));
>> > > +
>> > > +	igt_assert(width && num_placements &&
>> > > +		   (width == 1 || num_placements == 1));
>> > > +
>> > > +	igt_assert(width <= MAX_PARALLEL);
>> > > +
>> > > +	ctx->class = hwe->engine_class;
>> > > +	ctx->width = width;
>> > > +	ctx->num_placements = num_placements;
>> > > +	ctx->vm = vm;
>> > > +	for (int i = 0; i < ctx->width; i++)
>> > > +		ctx->addr[i] = batch_addr[hwe->engine_class];
>> > > +
>> > > +	ctx->exec.num_batch_buffer = width;
>> > > +	ctx->exec.num_syncs = 2;
>> > > +	ctx->exec.syncs = to_user_pointer(ctx->sync);
>> > > +
>> > > +	ctx->sync[0].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
>> > > +	ctx->sync[0].flags = DRM_XE_SYNC_FLAG_SIGNAL;
>> > > +	ctx->sync[0].handle = syncobj_create(fd, 0);
>> > > +
>> > > +	ctx->sync[1].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
>> > > +	ctx->sync[1].flags = DRM_XE_SYNC_FLAG_SIGNAL;
>> > > +	ctx->sync[1].handle = syncobj_create(fd, 0);
>> > > +
>> > > +	ctx->bo_size = sizeof(struct xe_spin);
>> > > +	ctx->bo_size = xe_bb_size(fd, ctx->bo_size);
>> > > +	ctx->bo = xe_bo_create(fd, ctx->vm, ctx->bo_size,
>> > > +			       vram_if_possible(fd, hwe->gt_id),
>> > > +			       DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM);
>> > > +	ctx->spin = xe_bo_map(fd, ctx->bo, ctx->bo_size);
>> > > +
>> > > +	igt_assert_eq(__xe_exec_queue_create(fd, ctx->vm, width, num_placements,
>> > > +					     hwe, 0, &ctx->exec_queue), 0);
>> > > +
>> > > +	xe_vm_bind_async(fd, ctx->vm, 0, ctx->bo, 0, ctx->addr[0], ctx->bo_size,
>> > > +			 ctx->sync, 1);
>> > > +
>> > > +	return ctx;
>> > > +}
>> > > +
>> > > +static void
>> > > +spin_sync_start(int fd, struct spin_ctx *ctx)
>> > > +{
>> > > +	if (!ctx)
>> > > +		return;
>> > > +
>> > > +	ctx->spin_opts.addr = ctx->addr[0];
>> > > +	ctx->spin_opts.preempt = true;
>> > > +	xe_spin_init(ctx->spin, &ctx->spin_opts);
>> > > +
>> > > +	/* re-use sync[0] for exec */
>> > > +	ctx->sync[0].flags &= ~DRM_XE_SYNC_FLAG_SIGNAL;
>> > > +
>> > > +	ctx->exec.exec_queue_id = ctx->exec_queue;
>> > > +	if (ctx->width > 1)
>> > > +		ctx->exec.address = to_user_pointer(ctx->addr);
>> > > +	else
>> > > +		ctx->exec.address = ctx->addr[0];
>> > > +	xe_exec(fd, &ctx->exec);
>> > > +
>> > > +	xe_spin_wait_started(ctx->spin);
>> > > +	igt_assert(!syncobj_wait(fd, &ctx->sync[1].handle, 1, 1, 0, NULL));
>> > > +
>> > > +	igt_debug("%s: spinner started\n", engine_map[ctx->class]);
>> > > +}
>> > > +
>> > > +static void
>> > > +spin_sync_end(int fd, struct spin_ctx *ctx)
>> > > +{
>> > > +	if (!ctx || ctx->ended)
>> > > +		return;
>> > > +
>> > > +	xe_spin_end(ctx->spin);
>> > > +
>> > > +	igt_assert(syncobj_wait(fd, &ctx->sync[1].handle, 1, INT64_MAX, 0, NULL));
>> > > +	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
>> > > +
>> > > +	ctx->sync[0].flags |= DRM_XE_SYNC_FLAG_SIGNAL;
>> > > +	xe_vm_unbind_async(fd, ctx->vm, 0, 0, ctx->addr[0], ctx->bo_size, ctx->sync, 1);
>> > > +	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
>> > > +
>> > > +	ctx->ended = true;
>> > > +	igt_debug("%s: spinner ended\n", engine_map[ctx->class]);
>> > > +}
>> > > +
>> > > +static void
>> > > +spin_ctx_destroy(int fd, struct spin_ctx *ctx)
>> > > +{
>> > > +	if (!ctx)
>> > > +		return;
>> > > +
>> > > +	syncobj_destroy(fd, ctx->sync[0].handle);
>> > > +	syncobj_destroy(fd, ctx->sync[1].handle);
>> > > +	xe_exec_queue_destroy(fd, ctx->exec_queue);
>> > > +
>> > > +	munmap(ctx->spin, ctx->bo_size);
>> > > +	gem_close(fd, ctx->bo);
>> > > +
>> > > +	free(ctx);
>> > > +}
>> > > +
>> > > igt_main
>> > > {
>> > > 	int xe;
>> > > --
>> > > 2.38.1
>> > >

[PATCH i-g-t v2 05/10] tests/intel/xe_drm_fdinfo: Add single engine tests

[Umesh Nerlige Ramappa]

Add simple tests that submit work to one engine and measure utilization
per class.

v2:
- Drop measured_usleep since return value is not used
- s/parallel engines/parallel submission/ in comment
- Use NSEC_PER_SEC for batch_duration_ns
- Percent should not be > 100
- Check utilization for both clients for isolation case

Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
---
 tests/intel/xe_drm_fdinfo.c | 97 +++++++++++++++++++++++++++++++++++++
 1 file changed, 97 insertions(+)

diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
index 037f25e53..410c885e7 100644
--- a/tests/intel/xe_drm_fdinfo.c
+++ b/tests/intel/xe_drm_fdinfo.c
@@ -28,6 +28,15 @@
  * SUBTEST: basic-engine-utilization
  * Description: Check if basic fdinfo content is present for engine utilization
  *
+ * SUBTEST: drm-idle
+ * Description: Check that engines show no load when idle
+ *
+ * SUBTEST: drm-busy-idle
+ * Description: Check that engines show load when idle after busy
+ *
+ * SUBTEST: drm-busy-idle-isolation
+ * Description: Check that engine load does not spill over to other drm clients
+ *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
  *
@@ -42,11 +51,18 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 
 #define BO_SIZE (65536)
 
+/* flag masks */
+#define TEST_BUSY		(1 << 0)
+#define TEST_TRAILING_IDLE	(1 << 1)
+#define TEST_ISOLATION		(1 << 2)
+
 struct pceu_cycles {
 	uint64_t cycles;
 	uint64_t total_cycles;
 };
 
+const unsigned long batch_duration_ns = (1 * NSEC_PER_SEC) / 2;
+
 static const char *engine_map[] = {
 	"rcs",
 	"bcs",
@@ -464,8 +480,77 @@ spin_ctx_destroy(int fd, struct spin_ctx *ctx)
 	free(ctx);
 }
 
+static void
+check_results(struct pceu_cycles *s1, struct pceu_cycles *s2,
+	      int class, int width, unsigned int flags)
+{
+	double percent;
+
+	igt_debug("%s: sample 1: cycles %lu, total_cycles %lu\n",
+		  engine_map[class], s1[class].cycles, s1[class].total_cycles);
+	igt_debug("%s: sample 2: cycles %lu, total_cycles %lu\n",
+		  engine_map[class], s2[class].cycles, s2[class].total_cycles);
+
+	percent = ((s2[class].cycles - s1[class].cycles) * 100) /
+		  ((s2[class].total_cycles + 1) - s1[class].total_cycles);
+
+	/* for parallel submission scale the busyness with width */
+	percent = percent / width;
+
+	igt_debug("%s: percent: %f\n", engine_map[class], percent);
+
+	if (flags & TEST_BUSY)
+		igt_assert(percent >= 95 && percent <= 100);
+	else
+		igt_assert(!percent);
+}
+
+static void
+single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
+       unsigned int flags)
+{
+	struct pceu_cycles pceu1[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct spin_ctx *ctx = NULL;
+	uint32_t vm;
+	int new_fd;
+
+	if (flags & TEST_ISOLATION)
+		new_fd = drm_reopen_driver(fd);
+
+	vm = xe_vm_create(fd, 0, 0);
+	if (flags & TEST_BUSY) {
+		ctx = spin_ctx_init(fd, hwe, vm, width, count);
+		spin_sync_start(fd, ctx);
+	}
+
+	read_engine_cycles(fd, pceu1[0]);
+	if (flags & TEST_ISOLATION)
+		read_engine_cycles(new_fd, pceu1[1]);
+
+	usleep(batch_duration_ns / 1000);
+	if (flags & TEST_TRAILING_IDLE)
+		spin_sync_end(fd, ctx);
+
+	read_engine_cycles(fd, pceu2[0]);
+	if (flags & TEST_ISOLATION)
+		read_engine_cycles(new_fd, pceu2[1]);
+
+	check_results(pceu1[0], pceu2[0], hwe->engine_class, width, flags);
+
+	if (flags & TEST_ISOLATION) {
+		check_results(pceu1[1], pceu2[1], hwe->engine_class, width, 0);
+		close(new_fd);
+	}
+
+	spin_sync_end(fd, ctx);
+	spin_ctx_destroy(fd, ctx);
+	xe_vm_destroy(fd, vm);
+}
+
 igt_main
 {
+	struct drm_xe_engine_class_instance *hwe;
 	int xe;
 
 	igt_fixture {
@@ -484,6 +569,18 @@ igt_main
 	igt_subtest("basic-engine-utilization")
 		basic_engine_utilization(xe);
 
+	igt_subtest("drm-idle")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, 0);
+
+	igt_subtest("drm-busy-idle")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-busy-idle-isolation")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
-- 
2.38.1

Re: [PATCH i-g-t v2 05/10] tests/intel/xe_drm_fdinfo: Add single engine tests

[Lucas De Marchi]

On Tue, Jul 02, 2024 at 05:25:27PM GMT, Umesh Nerlige Ramappa wrote:
>Add simple tests that submit work to one engine and measure utilization
>per class.
>
>v2:
>- Drop measured_usleep since return value is not used
>- s/parallel engines/parallel submission/ in comment
>- Use NSEC_PER_SEC for batch_duration_ns
>- Percent should not be > 100

but as we chatted later in the review, it's actually possible to have it
greater than 100, so I think we need to leave some room

>- Check utilization for both clients for isolation case
>
>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>---
> tests/intel/xe_drm_fdinfo.c | 97 +++++++++++++++++++++++++++++++++++++
> 1 file changed, 97 insertions(+)
>
>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>index 037f25e53..410c885e7 100644
>--- a/tests/intel/xe_drm_fdinfo.c
>+++ b/tests/intel/xe_drm_fdinfo.c
>@@ -28,6 +28,15 @@
>  * SUBTEST: basic-engine-utilization
>  * Description: Check if basic fdinfo content is present for engine utilization
>  *
>+ * SUBTEST: drm-idle
>+ * Description: Check that engines show no load when idle
>+ *
>+ * SUBTEST: drm-busy-idle
>+ * Description: Check that engines show load when idle after busy
>+ *
>+ * SUBTEST: drm-busy-idle-isolation
>+ * Description: Check that engine load does not spill over to other drm clients
>+ *
>  * SUBTEST: drm-total-resident
>  * Description: Create and compare total and resident memory consumption by client
>  *
>@@ -42,11 +51,18 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
>
> #define BO_SIZE (65536)
>
>+/* flag masks */
>+#define TEST_BUSY		(1 << 0)
>+#define TEST_TRAILING_IDLE	(1 << 1)
>+#define TEST_ISOLATION		(1 << 2)
>+
> struct pceu_cycles {
> 	uint64_t cycles;
> 	uint64_t total_cycles;
> };
>
>+const unsigned long batch_duration_ns = (1 * NSEC_PER_SEC) / 2;
>+
> static const char *engine_map[] = {
> 	"rcs",
> 	"bcs",
>@@ -464,8 +480,77 @@ spin_ctx_destroy(int fd, struct spin_ctx *ctx)
> 	free(ctx);
> }
>
>+static void
>+check_results(struct pceu_cycles *s1, struct pceu_cycles *s2,
>+	      int class, int width, unsigned int flags)
>+{
>+	double percent;
>+
>+	igt_debug("%s: sample 1: cycles %lu, total_cycles %lu\n",
>+		  engine_map[class], s1[class].cycles, s1[class].total_cycles);
>+	igt_debug("%s: sample 2: cycles %lu, total_cycles %lu\n",
>+		  engine_map[class], s2[class].cycles, s2[class].total_cycles);
>+
>+	percent = ((s2[class].cycles - s1[class].cycles) * 100) /
>+		  ((s2[class].total_cycles + 1) - s1[class].total_cycles);
>+
>+	/* for parallel submission scale the busyness with width */
>+	percent = percent / width;
>+
>+	igt_debug("%s: percent: %f\n", engine_map[class], percent);
>+
>+	if (flags & TEST_BUSY)
>+		igt_assert(percent >= 95 && percent <= 100);

<= 105 like you had before seems good, otherwise we may have too much
noise in CI.  What are the typical numbers you're seeing?

>+	else
>+		igt_assert(!percent);
>+}
>+
>+static void
>+single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
>+       unsigned int flags)
>+{
>+	struct pceu_cycles pceu1[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
>+	struct pceu_cycles pceu2[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];

nit: either name it pceu_start/pceu_end:

pceu_start[0] is the pceu for client0 on start
pceu_start[1] is the pceu for client1 on start
pceu_end[0] is the pceu for client0 on end
pceu_end[1] is the pceu for client1 on end

or pceu_client0/pceu_client1 if grouping by client

pceu_client0[0] is the pceu for client0 on start
pceu_client0[1] is the pceu for client0 on end
pceu_client1[1] is the pceu for client1 on start
pceu_client1[2] is the pceu for client1 on end

... this makes it less prone to typos.

But see below as I think 2 samples are not sufficient.

>+	struct spin_ctx *ctx = NULL;
>+	uint32_t vm;
>+	int new_fd;
>+
>+	if (flags & TEST_ISOLATION)
>+		new_fd = drm_reopen_driver(fd);
>+
>+	vm = xe_vm_create(fd, 0, 0);
>+	if (flags & TEST_BUSY) {
>+		ctx = spin_ctx_init(fd, hwe, vm, width, count);
>+		spin_sync_start(fd, ctx);
>+	}
>+
>+	read_engine_cycles(fd, pceu1[0]);
>+	if (flags & TEST_ISOLATION)
>+		read_engine_cycles(new_fd, pceu1[1]);
>+
>+	usleep(batch_duration_ns / 1000);

NSEC_PER_USEC

>+	if (flags & TEST_TRAILING_IDLE)
>+		spin_sync_end(fd, ctx);
>+
>+	read_engine_cycles(fd, pceu2[0]);

... and here we have another source of noise.... when doing
TEST_TRAILING_IDLE, should we read the first sample after the spin
ended, sleep, and read the second sample?

but then how are we testing all at the same time here? I can't see any
igt_assert() for the trailing idle test and it will actually impact the
busy test since it will wait the spin to end.... Am I missing anything
here? I think we need to add a third sample when testing trailing idle
and ensure:

considering the indexing as sample[client][idx]

	sample[0][1] - sample[0][0] == 100% if BUSY else 0%
	sample[0][2] - sample[0][1] == 0%  -> idle on trailing test

and for isolation:

	sample[1][1] - sample[0][0] == 0%
	sample[1][2] - sample[1][1] == 0%

>+	if (flags & TEST_ISOLATION)
>+		read_engine_cycles(new_fd, pceu2[1]);
>+
>+	check_results(pceu1[0], pceu2[0], hwe->engine_class, width, flags);
>+
>+	if (flags & TEST_ISOLATION) {
>+		check_results(pceu1[1], pceu2[1], hwe->engine_class, width, 0);
>+		close(new_fd);
>+	}
>+
>+	spin_sync_end(fd, ctx);
>+	spin_ctx_destroy(fd, ctx);

if !BUSY you are not even calling spin_ctx_init()

Lucas De Marchi

>+	xe_vm_destroy(fd, vm);
>+}
>+
> igt_main
> {
>+	struct drm_xe_engine_class_instance *hwe;
> 	int xe;
>
> 	igt_fixture {
>@@ -484,6 +569,18 @@ igt_main
> 	igt_subtest("basic-engine-utilization")
> 		basic_engine_utilization(xe);
>
>+	igt_subtest("drm-idle")
>+		xe_for_each_engine(xe, hwe)
>+			single(xe, hwe, 1, 1, 0);
>+
>+	igt_subtest("drm-busy-idle")
>+		xe_for_each_engine(xe, hwe)
>+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
>+
>+	igt_subtest("drm-busy-idle-isolation")
>+		xe_for_each_engine(xe, hwe)
>+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
>+
> 	igt_describe("Create and compare total and resident memory consumption by client");
> 	igt_subtest("drm-total-resident")
> 		test_total_resident(xe);
>-- 
>2.38.1
>

Re: [PATCH i-g-t v2 05/10] tests/intel/xe_drm_fdinfo: Add single engine tests

[Umesh Nerlige Ramappa]

On Thu, Jul 11, 2024 at 08:20:17AM -0500, Lucas De Marchi wrote:
>On Tue, Jul 02, 2024 at 05:25:27PM GMT, Umesh Nerlige Ramappa wrote:
>>Add simple tests that submit work to one engine and measure utilization
>>per class.
>>
>>v2:
>>- Drop measured_usleep since return value is not used
>>- s/parallel engines/parallel submission/ in comment
>>- Use NSEC_PER_SEC for batch_duration_ns
>>- Percent should not be > 100
>
>but as we chatted later in the review, it's actually possible to have it
>greater than 100, so I think we need to leave some room
>
>>- Check utilization for both clients for isolation case
>>
>>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>>---
>>tests/intel/xe_drm_fdinfo.c | 97 +++++++++++++++++++++++++++++++++++++
>>1 file changed, 97 insertions(+)
>>
>>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>>index 037f25e53..410c885e7 100644
>>--- a/tests/intel/xe_drm_fdinfo.c
>>+++ b/tests/intel/xe_drm_fdinfo.c
>>@@ -28,6 +28,15 @@
>> * SUBTEST: basic-engine-utilization
>> * Description: Check if basic fdinfo content is present for engine utilization
>> *
>>+ * SUBTEST: drm-idle
>>+ * Description: Check that engines show no load when idle
>>+ *
>>+ * SUBTEST: drm-busy-idle
>>+ * Description: Check that engines show load when idle after busy
>>+ *
>>+ * SUBTEST: drm-busy-idle-isolation
>>+ * Description: Check that engine load does not spill over to other drm clients
>>+ *
>> * SUBTEST: drm-total-resident
>> * Description: Create and compare total and resident memory consumption by client
>> *
>>@@ -42,11 +51,18 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
>>
>>#define BO_SIZE (65536)
>>
>>+/* flag masks */
>>+#define TEST_BUSY		(1 << 0)
>>+#define TEST_TRAILING_IDLE	(1 << 1)
>>+#define TEST_ISOLATION		(1 << 2)
>>+
>>struct pceu_cycles {
>>	uint64_t cycles;
>>	uint64_t total_cycles;
>>};
>>
>>+const unsigned long batch_duration_ns = (1 * NSEC_PER_SEC) / 2;
>>+
>>static const char *engine_map[] = {
>>	"rcs",
>>	"bcs",
>>@@ -464,8 +480,77 @@ spin_ctx_destroy(int fd, struct spin_ctx *ctx)
>>	free(ctx);
>>}
>>
>>+static void
>>+check_results(struct pceu_cycles *s1, struct pceu_cycles *s2,
>>+	      int class, int width, unsigned int flags)
>>+{
>>+	double percent;
>>+
>>+	igt_debug("%s: sample 1: cycles %lu, total_cycles %lu\n",
>>+		  engine_map[class], s1[class].cycles, s1[class].total_cycles);
>>+	igt_debug("%s: sample 2: cycles %lu, total_cycles %lu\n",
>>+		  engine_map[class], s2[class].cycles, s2[class].total_cycles);
>>+
>>+	percent = ((s2[class].cycles - s1[class].cycles) * 100) /
>>+		  ((s2[class].total_cycles + 1) - s1[class].total_cycles);
>>+
>>+	/* for parallel submission scale the busyness with width */
>>+	percent = percent / width;
>>+
>>+	igt_debug("%s: percent: %f\n", engine_map[class], percent);
>>+
>>+	if (flags & TEST_BUSY)
>>+		igt_assert(percent >= 95 && percent <= 100);
>
><= 105 like you had before seems good, otherwise we may have too much
>noise in CI.  What are the typical numbers you're seeing?

I see 101 whenever it fails. Also I see it only when I run batches on 
all classes (all_busy_all/most_busy_all etc.). The difference in 
numerator and denominator deltas is close to 5ms though. Not convinced 
that we should see 100 still because we are ending the spinners (and 
waiting for fences and all) before taking sample 2. Still debugging a 
little more on this one.

>
>>+	else
>>+		igt_assert(!percent);
>>+}
>>+
>>+static void
>>+single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
>>+       unsigned int flags)
>>+{
>>+	struct pceu_cycles pceu1[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
>>+	struct pceu_cycles pceu2[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
>
>nit: either name it pceu_start/pceu_end:
>
>pceu_start[0] is the pceu for client0 on start
>pceu_start[1] is the pceu for client1 on start
>pceu_end[0] is the pceu for client0 on end
>pceu_end[1] is the pceu for client1 on end
>
>or pceu_client0/pceu_client1 if grouping by client
>
>pceu_client0[0] is the pceu for client0 on start
>pceu_client0[1] is the pceu for client0 on end
>pceu_client1[1] is the pceu for client1 on start
>pceu_client1[2] is the pceu for client1 on end
>
>... this makes it less prone to typos.
>
>But see below as I think 2 samples are not sufficient.
>
>>+	struct spin_ctx *ctx = NULL;
>>+	uint32_t vm;
>>+	int new_fd;
>>+
>>+	if (flags & TEST_ISOLATION)
>>+		new_fd = drm_reopen_driver(fd);
>>+
>>+	vm = xe_vm_create(fd, 0, 0);
>>+	if (flags & TEST_BUSY) {
>>+		ctx = spin_ctx_init(fd, hwe, vm, width, count);
>>+		spin_sync_start(fd, ctx);
>>+	}
>>+
>>+	read_engine_cycles(fd, pceu1[0]);
>>+	if (flags & TEST_ISOLATION)
>>+		read_engine_cycles(new_fd, pceu1[1]);
>>+
>>+	usleep(batch_duration_ns / 1000);
>
>NSEC_PER_USEC
>
>>+	if (flags & TEST_TRAILING_IDLE)
>>+		spin_sync_end(fd, ctx);
>>+
>>+	read_engine_cycles(fd, pceu2[0]);
>
>... and here we have another source of noise.... when doing
>TEST_TRAILING_IDLE, should we read the first sample after the spin
>ended, sleep, and read the second sample?

That would always make utilization = 0% (numerator delta is 0). We 
should take sample 1 when the spinner has started and sample 2 when it 
has ended (for PCEU).

TEST_TRAILING_IDLE is only valid when TEST_BUSY is also specified.  
Maybe I should enforce that in the test. This is the flow for common 
flag configs:

1) flag = TEST_BUSY | TEST_TRAILING_IDLE
- wait till spinner starts
- take sample 1
- usleep
- end the batch
- take sample 2.
- check_results (expected 100% util)

2) flag = TEST_BUSY
- wait till spinner starts
- take sample 1
- usleep
- take sample 2.
- end the batch
- check_results (expected 100% util)

3) flag = 0 means truly IDLE since there is no spinner.

>
>but then how are we testing all at the same time here? I can't see any
>igt_assert() for the trailing idle test and it will actually impact the
>busy test since it will wait the spin to end.... Am I missing anything
>here?

Case (2) above does not exist for PCEU tests because CTX_TIMESTAMP is 
only updates when the context switches out, which means we must end the 
spinner before taking the second sample (or preempt it, but ending is 
simpler). This series only has tests for case (1) and (3) implemented.

Case (2) is applicable for a completely different feature which is still 
wip - per engine utilization via kernel perf framework.

>I think we need to add a third sample when testing trailing idle
>and ensure:
>
>considering the indexing as sample[client][idx]
>
>	sample[0][1] - sample[0][0] == 100% if BUSY else 0%
>	sample[0][2] - sample[0][1] == 0%  -> idle on trailing test
>
>and for isolation:
>
>	sample[1][1] - sample[0][0] == 0%
>	sample[1][2] - sample[1][1] == 0%
>
>>+	if (flags & TEST_ISOLATION)
>>+		read_engine_cycles(new_fd, pceu2[1]);
>>+
>>+	check_results(pceu1[0], pceu2[0], hwe->engine_class, width, flags);

This check_results will check for TEST_BUSY case.

>>+
>>+	if (flags & TEST_ISOLATION) {
>>+		check_results(pceu1[1], pceu2[1], hwe->engine_class, width, 0);

This check_results will check for idle case (flags = 0).

>>+		close(new_fd);
>>+	}
>>+
>>+	spin_sync_end(fd, ctx);
>>+	spin_ctx_destroy(fd, ctx);
>
>if !BUSY you are not even calling spin_ctx_init()

:) yeah, sorry about the readability here. The spin_sync_end and 
spin_ctx_destroy bail out if the ctx is NULL which is the case for 
!BUSY. I can add and explicit check to not call these instead.

Thanks,
Umesh
>
>Lucas De Marchi
>
>>+	xe_vm_destroy(fd, vm);
>>+}
>>+
>>igt_main
>>{
>>+	struct drm_xe_engine_class_instance *hwe;
>>	int xe;
>>
>>	igt_fixture {
>>@@ -484,6 +569,18 @@ igt_main
>>	igt_subtest("basic-engine-utilization")
>>		basic_engine_utilization(xe);
>>
>>+	igt_subtest("drm-idle")
>>+		xe_for_each_engine(xe, hwe)
>>+			single(xe, hwe, 1, 1, 0);
>>+
>>+	igt_subtest("drm-busy-idle")
>>+		xe_for_each_engine(xe, hwe)
>>+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
>>+
>>+	igt_subtest("drm-busy-idle-isolation")
>>+		xe_for_each_engine(xe, hwe)
>>+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
>>+
>>	igt_describe("Create and compare total and resident memory consumption by client");
>>	igt_subtest("drm-total-resident")
>>		test_total_resident(xe);
>>-- 
>>2.38.1
>>

Re: [PATCH i-g-t v2 05/10] tests/intel/xe_drm_fdinfo: Add single engine tests

[Lucas De Marchi]

On Thu, Jul 11, 2024 at 12:13:22PM GMT, Umesh Nerlige Ramappa wrote:
>On Thu, Jul 11, 2024 at 08:20:17AM -0500, Lucas De Marchi wrote:
>>On Tue, Jul 02, 2024 at 05:25:27PM GMT, Umesh Nerlige Ramappa wrote:
>>>Add simple tests that submit work to one engine and measure utilization
>>>per class.
>>>
>>>v2:
>>>- Drop measured_usleep since return value is not used
>>>- s/parallel engines/parallel submission/ in comment
>>>- Use NSEC_PER_SEC for batch_duration_ns
>>>- Percent should not be > 100
>>
>>but as we chatted later in the review, it's actually possible to have it
>>greater than 100, so I think we need to leave some room
>>
>>>- Check utilization for both clients for isolation case
>>>
>>>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>>>---
>>>tests/intel/xe_drm_fdinfo.c | 97 +++++++++++++++++++++++++++++++++++++
>>>1 file changed, 97 insertions(+)
>>>
>>>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>>>index 037f25e53..410c885e7 100644
>>>--- a/tests/intel/xe_drm_fdinfo.c
>>>+++ b/tests/intel/xe_drm_fdinfo.c
>>>@@ -28,6 +28,15 @@
>>>* SUBTEST: basic-engine-utilization
>>>* Description: Check if basic fdinfo content is present for engine utilization
>>>*
>>>+ * SUBTEST: drm-idle
>>>+ * Description: Check that engines show no load when idle
>>>+ *
>>>+ * SUBTEST: drm-busy-idle
>>>+ * Description: Check that engines show load when idle after busy
>>>+ *
>>>+ * SUBTEST: drm-busy-idle-isolation
>>>+ * Description: Check that engine load does not spill over to other drm clients
>>>+ *
>>>* SUBTEST: drm-total-resident
>>>* Description: Create and compare total and resident memory consumption by client
>>>*
>>>@@ -42,11 +51,18 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
>>>
>>>#define BO_SIZE (65536)
>>>
>>>+/* flag masks */
>>>+#define TEST_BUSY		(1 << 0)
>>>+#define TEST_TRAILING_IDLE	(1 << 1)
>>>+#define TEST_ISOLATION		(1 << 2)
>>>+
>>>struct pceu_cycles {
>>>	uint64_t cycles;
>>>	uint64_t total_cycles;
>>>};
>>>
>>>+const unsigned long batch_duration_ns = (1 * NSEC_PER_SEC) / 2;
>>>+
>>>static const char *engine_map[] = {
>>>	"rcs",
>>>	"bcs",
>>>@@ -464,8 +480,77 @@ spin_ctx_destroy(int fd, struct spin_ctx *ctx)
>>>	free(ctx);
>>>}
>>>
>>>+static void
>>>+check_results(struct pceu_cycles *s1, struct pceu_cycles *s2,
>>>+	      int class, int width, unsigned int flags)
>>>+{
>>>+	double percent;
>>>+
>>>+	igt_debug("%s: sample 1: cycles %lu, total_cycles %lu\n",
>>>+		  engine_map[class], s1[class].cycles, s1[class].total_cycles);
>>>+	igt_debug("%s: sample 2: cycles %lu, total_cycles %lu\n",
>>>+		  engine_map[class], s2[class].cycles, s2[class].total_cycles);
>>>+
>>>+	percent = ((s2[class].cycles - s1[class].cycles) * 100) /
>>>+		  ((s2[class].total_cycles + 1) - s1[class].total_cycles);
>>>+
>>>+	/* for parallel submission scale the busyness with width */
>>>+	percent = percent / width;
>>>+
>>>+	igt_debug("%s: percent: %f\n", engine_map[class], percent);
>>>+
>>>+	if (flags & TEST_BUSY)
>>>+		igt_assert(percent >= 95 && percent <= 100);
>>
>><= 105 like you had before seems good, otherwise we may have too much
>>noise in CI.  What are the typical numbers you're seeing?
>
>I see 101 whenever it fails. Also I see it only when I run batches on 
>all classes (all_busy_all/most_busy_all etc.). The difference in 
>numerator and denominator deltas is close to 5ms though. Not convinced 
>that we should see 100 still because we are ending the spinners (and 
>waiting for fences and all) before taking sample 2. Still debugging a 
>little more on this one.
>
>>
>>>+	else
>>>+		igt_assert(!percent);
>>>+}
>>>+
>>>+static void
>>>+single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
>>>+       unsigned int flags)
>>>+{
>>>+	struct pceu_cycles pceu1[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
>>>+	struct pceu_cycles pceu2[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
>>
>>nit: either name it pceu_start/pceu_end:
>>
>>pceu_start[0] is the pceu for client0 on start
>>pceu_start[1] is the pceu for client1 on start
>>pceu_end[0] is the pceu for client0 on end
>>pceu_end[1] is the pceu for client1 on end
>>
>>or pceu_client0/pceu_client1 if grouping by client
>>
>>pceu_client0[0] is the pceu for client0 on start
>>pceu_client0[1] is the pceu for client0 on end
>>pceu_client1[1] is the pceu for client1 on start
>>pceu_client1[2] is the pceu for client1 on end
>>
>>... this makes it less prone to typos.
>>
>>But see below as I think 2 samples are not sufficient.
>>
>>>+	struct spin_ctx *ctx = NULL;
>>>+	uint32_t vm;
>>>+	int new_fd;
>>>+
>>>+	if (flags & TEST_ISOLATION)
>>>+		new_fd = drm_reopen_driver(fd);
>>>+
>>>+	vm = xe_vm_create(fd, 0, 0);
>>>+	if (flags & TEST_BUSY) {
>>>+		ctx = spin_ctx_init(fd, hwe, vm, width, count);
>>>+		spin_sync_start(fd, ctx);
>>>+	}
>>>+
>>>+	read_engine_cycles(fd, pceu1[0]);
>>>+	if (flags & TEST_ISOLATION)
>>>+		read_engine_cycles(new_fd, pceu1[1]);
>>>+
>>>+	usleep(batch_duration_ns / 1000);
>>
>>NSEC_PER_USEC
>>
>>>+	if (flags & TEST_TRAILING_IDLE)
>>>+		spin_sync_end(fd, ctx);
>>>+
>>>+	read_engine_cycles(fd, pceu2[0]);
>>
>>... and here we have another source of noise.... when doing
>>TEST_TRAILING_IDLE, should we read the first sample after the spin
>>ended, sleep, and read the second sample?
>
>That would always make utilization = 0% (numerator delta is 0). We 
>should take sample 1 when the spinner has started and sample 2 when it 
>has ended (for PCEU).
>
>TEST_TRAILING_IDLE is only valid when TEST_BUSY is also specified.  
>Maybe I should enforce that in the test. This is the flow for common 
>flag configs:
>
>1) flag = TEST_BUSY | TEST_TRAILING_IDLE
>- wait till spinner starts
>- take sample 1
>- usleep
>- end the batch
>- take sample 2.
>- check_results (expected 100% util)

I think the way the test is written, the expectation is wrong.
By absurd:

1) flag = TEST_BUSY | TEST_TRAILING_IDLE
- wait till spinner starts
- take sample 1
- usleep
- end the batch
		<--- long preempt here emulated with
		     usleep(500000)
- take sample 2.
- check_results

So we'd still expect the same ballpark number of delta ticks, but
utilization overall would go down as now you are ending the batch
and sleeping a little more before taking another sample. In the example
above I'd expect utilization to be 50%

>
>2) flag = TEST_BUSY
>- wait till spinner starts
>- take sample 1
>- usleep
>- take sample 2.
>- end the batch
>- check_results (expected 100% util)
>
>3) flag = 0 means truly IDLE since there is no spinner.
>
>>
>>but then how are we testing all at the same time here? I can't see any
>>igt_assert() for the trailing idle test and it will actually impact the
>>busy test since it will wait the spin to end.... Am I missing anything
>>here?
>
>Case (2) above does not exist for PCEU tests because CTX_TIMESTAMP is 
>only updates when the context switches out, which means we must end 
>the spinner before taking the second sample (or preempt it, but ending 
>is simpler). This series only has tests for case (1) and (3) 
>implemented.

true, we are only reading the value from the context image, but does the
hardware update it only on context switch? What I had understood was
that the value was updated while the context was running. Otherwise I
think it's utterly broken:

Considering
	s* are the samples
	c* are the context timestamps
	g* are the gpu timestamps

	- not running
	| running


          g0 g1                    g2        g3   g4 
          c0 c1                    c2        c3   c4
ctx0 ----||||||||||||||||||||||||||||||||||||-------
             ^                     ^              ^
             s0                    s1             s3                    

For simplicity, assuming g0 == c0 == 0

in s0: g1 == 3
        c1 == 0

in s1: g2 == 25
        c2 == 0

-> utilization returns 0%

in s3: g4 == 40
        c4 == c3 == 35


-> utilization returns 35 / (40 - 25) == 233%

how can we get this right if the value is not updated for running
contexts?

Lucas De Marchi

>
>Case (2) is applicable for a completely different feature which is 
>still wip - per engine utilization via kernel perf framework.
>
>>I think we need to add a third sample when testing trailing idle
>>and ensure:
>>
>>considering the indexing as sample[client][idx]
>>
>>	sample[0][1] - sample[0][0] == 100% if BUSY else 0%
>>	sample[0][2] - sample[0][1] == 0%  -> idle on trailing test
>>
>>and for isolation:
>>
>>	sample[1][1] - sample[0][0] == 0%
>>	sample[1][2] - sample[1][1] == 0%
>>
>>>+	if (flags & TEST_ISOLATION)
>>>+		read_engine_cycles(new_fd, pceu2[1]);
>>>+
>>>+	check_results(pceu1[0], pceu2[0], hwe->engine_class, width, flags);
>
>This check_results will check for TEST_BUSY case.
>
>>>+
>>>+	if (flags & TEST_ISOLATION) {
>>>+		check_results(pceu1[1], pceu2[1], hwe->engine_class, width, 0);
>
>This check_results will check for idle case (flags = 0).
>
>>>+		close(new_fd);
>>>+	}
>>>+
>>>+	spin_sync_end(fd, ctx);
>>>+	spin_ctx_destroy(fd, ctx);
>>
>>if !BUSY you are not even calling spin_ctx_init()
>
>:) yeah, sorry about the readability here. The spin_sync_end and 
>spin_ctx_destroy bail out if the ctx is NULL which is the case for 
>!BUSY. I can add and explicit check to not call these instead.
>
>Thanks,
>Umesh
>>
>>Lucas De Marchi
>>
>>>+	xe_vm_destroy(fd, vm);
>>>+}
>>>+
>>>igt_main
>>>{
>>>+	struct drm_xe_engine_class_instance *hwe;
>>>	int xe;
>>>
>>>	igt_fixture {
>>>@@ -484,6 +569,18 @@ igt_main
>>>	igt_subtest("basic-engine-utilization")
>>>		basic_engine_utilization(xe);
>>>
>>>+	igt_subtest("drm-idle")
>>>+		xe_for_each_engine(xe, hwe)
>>>+			single(xe, hwe, 1, 1, 0);
>>>+
>>>+	igt_subtest("drm-busy-idle")
>>>+		xe_for_each_engine(xe, hwe)
>>>+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
>>>+
>>>+	igt_subtest("drm-busy-idle-isolation")
>>>+		xe_for_each_engine(xe, hwe)
>>>+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
>>>+
>>>	igt_describe("Create and compare total and resident memory consumption by client");
>>>	igt_subtest("drm-total-resident")
>>>		test_total_resident(xe);
>>>-- 
>>>2.38.1
>>>

Re: [PATCH i-g-t v2 05/10] tests/intel/xe_drm_fdinfo: Add single engine tests

[Umesh Nerlige Ramappa]

On Thu, Jul 11, 2024 at 04:34:53PM -0500, Lucas De Marchi wrote:
>On Thu, Jul 11, 2024 at 12:13:22PM GMT, Umesh Nerlige Ramappa wrote:
>>On Thu, Jul 11, 2024 at 08:20:17AM -0500, Lucas De Marchi wrote:
>>>On Tue, Jul 02, 2024 at 05:25:27PM GMT, Umesh Nerlige Ramappa wrote:
>>>>Add simple tests that submit work to one engine and measure utilization
>>>>per class.
>>>>
>>>>v2:
>>>>- Drop measured_usleep since return value is not used
>>>>- s/parallel engines/parallel submission/ in comment
>>>>- Use NSEC_PER_SEC for batch_duration_ns
>>>>- Percent should not be > 100
>>>
>>>but as we chatted later in the review, it's actually possible to have it
>>>greater than 100, so I think we need to leave some room
>>>
>>>>- Check utilization for both clients for isolation case
>>>>
>>>>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>>>>---
>>>>tests/intel/xe_drm_fdinfo.c | 97 +++++++++++++++++++++++++++++++++++++
>>>>1 file changed, 97 insertions(+)
>>>>
>>>>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>>>>index 037f25e53..410c885e7 100644
>>>>--- a/tests/intel/xe_drm_fdinfo.c
>>>>+++ b/tests/intel/xe_drm_fdinfo.c
>>>>@@ -28,6 +28,15 @@
>>>>* SUBTEST: basic-engine-utilization
>>>>* Description: Check if basic fdinfo content is present for engine utilization
>>>>*
>>>>+ * SUBTEST: drm-idle
>>>>+ * Description: Check that engines show no load when idle
>>>>+ *
>>>>+ * SUBTEST: drm-busy-idle
>>>>+ * Description: Check that engines show load when idle after busy
>>>>+ *
>>>>+ * SUBTEST: drm-busy-idle-isolation
>>>>+ * Description: Check that engine load does not spill over to other drm clients
>>>>+ *
>>>>* SUBTEST: drm-total-resident
>>>>* Description: Create and compare total and resident memory consumption by client
>>>>*
>>>>@@ -42,11 +51,18 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
>>>>
>>>>#define BO_SIZE (65536)
>>>>
>>>>+/* flag masks */
>>>>+#define TEST_BUSY		(1 << 0)
>>>>+#define TEST_TRAILING_IDLE	(1 << 1)
>>>>+#define TEST_ISOLATION		(1 << 2)
>>>>+
>>>>struct pceu_cycles {
>>>>	uint64_t cycles;
>>>>	uint64_t total_cycles;
>>>>};
>>>>
>>>>+const unsigned long batch_duration_ns = (1 * NSEC_PER_SEC) / 2;
>>>>+
>>>>static const char *engine_map[] = {
>>>>	"rcs",
>>>>	"bcs",
>>>>@@ -464,8 +480,77 @@ spin_ctx_destroy(int fd, struct spin_ctx *ctx)
>>>>	free(ctx);
>>>>}
>>>>
>>>>+static void
>>>>+check_results(struct pceu_cycles *s1, struct pceu_cycles *s2,
>>>>+	      int class, int width, unsigned int flags)
>>>>+{
>>>>+	double percent;
>>>>+
>>>>+	igt_debug("%s: sample 1: cycles %lu, total_cycles %lu\n",
>>>>+		  engine_map[class], s1[class].cycles, s1[class].total_cycles);
>>>>+	igt_debug("%s: sample 2: cycles %lu, total_cycles %lu\n",
>>>>+		  engine_map[class], s2[class].cycles, s2[class].total_cycles);
>>>>+
>>>>+	percent = ((s2[class].cycles - s1[class].cycles) * 100) /
>>>>+		  ((s2[class].total_cycles + 1) - s1[class].total_cycles);
>>>>+
>>>>+	/* for parallel submission scale the busyness with width */
>>>>+	percent = percent / width;
>>>>+
>>>>+	igt_debug("%s: percent: %f\n", engine_map[class], percent);
>>>>+
>>>>+	if (flags & TEST_BUSY)
>>>>+		igt_assert(percent >= 95 && percent <= 100);
>>>
>>><= 105 like you had before seems good, otherwise we may have too much
>>>noise in CI.  What are the typical numbers you're seeing?
>>
>>I see 101 whenever it fails. Also I see it only when I run batches 
>>on all classes (all_busy_all/most_busy_all etc.). The difference in 
>>numerator and denominator deltas is close to 5ms though. Not 
>>convinced that we should see 100 still because we are ending the 
>>spinners (and waiting for fences and all) before taking sample 2. 
>>Still debugging a little more on this one.
>>
>>>
>>>>+	else
>>>>+		igt_assert(!percent);
>>>>+}
>>>>+
>>>>+static void
>>>>+single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
>>>>+       unsigned int flags)
>>>>+{
>>>>+	struct pceu_cycles pceu1[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
>>>>+	struct pceu_cycles pceu2[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
>>>
>>>nit: either name it pceu_start/pceu_end:
>>>
>>>pceu_start[0] is the pceu for client0 on start
>>>pceu_start[1] is the pceu for client1 on start
>>>pceu_end[0] is the pceu for client0 on end
>>>pceu_end[1] is the pceu for client1 on end
>>>
>>>or pceu_client0/pceu_client1 if grouping by client
>>>
>>>pceu_client0[0] is the pceu for client0 on start
>>>pceu_client0[1] is the pceu for client0 on end
>>>pceu_client1[1] is the pceu for client1 on start
>>>pceu_client1[2] is the pceu for client1 on end
>>>
>>>... this makes it less prone to typos.
>>>
>>>But see below as I think 2 samples are not sufficient.
>>>
>>>>+	struct spin_ctx *ctx = NULL;
>>>>+	uint32_t vm;
>>>>+	int new_fd;
>>>>+
>>>>+	if (flags & TEST_ISOLATION)
>>>>+		new_fd = drm_reopen_driver(fd);
>>>>+
>>>>+	vm = xe_vm_create(fd, 0, 0);
>>>>+	if (flags & TEST_BUSY) {
>>>>+		ctx = spin_ctx_init(fd, hwe, vm, width, count);
>>>>+		spin_sync_start(fd, ctx);
>>>>+	}
>>>>+
>>>>+	read_engine_cycles(fd, pceu1[0]);
>>>>+	if (flags & TEST_ISOLATION)
>>>>+		read_engine_cycles(new_fd, pceu1[1]);
>>>>+
>>>>+	usleep(batch_duration_ns / 1000);
>>>
>>>NSEC_PER_USEC
>>>
>>>>+	if (flags & TEST_TRAILING_IDLE)
>>>>+		spin_sync_end(fd, ctx);
>>>>+
>>>>+	read_engine_cycles(fd, pceu2[0]);
>>>
>>>... and here we have another source of noise.... when doing
>>>TEST_TRAILING_IDLE, should we read the first sample after the spin
>>>ended, sleep, and read the second sample?
>>
>>That would always make utilization = 0% (numerator delta is 0). We 
>>should take sample 1 when the spinner has started and sample 2 when 
>>it has ended (for PCEU).
>>
>>TEST_TRAILING_IDLE is only valid when TEST_BUSY is also specified.  
>>Maybe I should enforce that in the test. This is the flow for common 
>>flag configs:
>>
>>1) flag = TEST_BUSY | TEST_TRAILING_IDLE
>>- wait till spinner starts
>>- take sample 1
>>- usleep
>>- end the batch
>>- take sample 2.
>>- check_results (expected 100% util)
>
>I think the way the test is written, the expectation is wrong.
>By absurd:
>
>1) flag = TEST_BUSY | TEST_TRAILING_IDLE
>- wait till spinner starts
>- take sample 1
>- usleep
>- end the batch
>		<--- long preempt here emulated with
>		     usleep(500000)
>- take sample 2.
>- check_results
>
>So we'd still expect the same ballpark number of delta ticks, but
>utilization overall would go down as now you are ending the batch
>and sleeping a little more before taking another sample. In the example
>above I'd expect utilization to be 50%

Indeed. if preempted after ending batch, then the numbers will be off, 
but that would be for this test. In other words, the test will not be 
able to verify this accurately. In reality, user will see the correct 
utilization - 50%.

>
>>
>>2) flag = TEST_BUSY
>>- wait till spinner starts
>>- take sample 1
>>- usleep
>>- take sample 2.
>>- end the batch
>>- check_results (expected 100% util)
>>
>>3) flag = 0 means truly IDLE since there is no spinner.
>>
>>>
>>>but then how are we testing all at the same time here? I can't see any
>>>igt_assert() for the trailing idle test and it will actually impact the
>>>busy test since it will wait the spin to end.... Am I missing anything
>>>here?
>>
>>Case (2) above does not exist for PCEU tests because CTX_TIMESTAMP 
>>is only updates when the context switches out, which means we must 
>>end the spinner before taking the second sample (or preempt it, but 
>>ending is simpler). This series only has tests for case (1) and (3) 
>>implemented.
>
>true, we are only reading the value from the context image, but does the
>hardware update it only on context switch? What I had understood was
>that the value was updated while the context was running. Otherwise I
>think it's utterly broken:

CTX_TIMESTAMP in the context image is only updated on a context switch.  

>
>Considering
>	s* are the samples
>	c* are the context timestamps
>	g* are the gpu timestamps
>
>	- not running
>	| running
>
>
>         g0 g1                    g2        g3   g4
>         c0 c1                    c2        c3   c4
>ctx0 ----||||||||||||||||||||||||||||||||||||-------
>            ^                     ^              ^
>            s0                    s1             s3
>
>For simplicity, assuming g0 == c0 == 0
>
>in s0: g1 == 3
>       c1 == 0
>
>in s1: g2 == 25
>       c2 == 0
>
>-> utilization returns 0%
>
>in s3: g4 == 40
>       c4 == c3 == 35
>
>
>-> utilization returns 35 / (40 - 25) == 233%

well... I think that's the issue with > 100% utilization that I see with 
tests!! If sample 1 is delayed, then calculations are off.

>
>how can we get this right if the value is not updated for running
>contexts?

I don't think we can. This is an inherent limitation of the HW counter 
that we are exposing. It is updated only when context switches happen 
and can be reliable only for frequently switching contexts. Tools would 
have to use some sort of averaging algorithm over a period of time to 
make sense of the data.

I think this sort of sampling affects any counters that we export that 
are updated infrequently. You will run into internediate values that are 
larger than 100%. Results may need to be averaged out.

Although, for this specific case, we should try to support getting c2 as 
well since that would be required for long-running contexts anyways. GuC 
had some level of support for this but with some limitations. I would 
have to dig that up and consult GuC if we can use/extend that.

I would probably not merge the IGTs, but would appreciate it if they can 
still be reviewed. I would still use the same tests once the above 
support is added.

Thanks,
Umesh

>
>Lucas De Marchi
>
>>
>>Case (2) is applicable for a completely different feature which is 
>>still wip - per engine utilization via kernel perf framework.
>>
>>>I think we need to add a third sample when testing trailing idle
>>>and ensure:
>>>
>>>considering the indexing as sample[client][idx]
>>>
>>>	sample[0][1] - sample[0][0] == 100% if BUSY else 0%
>>>	sample[0][2] - sample[0][1] == 0%  -> idle on trailing test
>>>
>>>and for isolation:
>>>
>>>	sample[1][1] - sample[0][0] == 0%
>>>	sample[1][2] - sample[1][1] == 0%
>>>
>>>>+	if (flags & TEST_ISOLATION)
>>>>+		read_engine_cycles(new_fd, pceu2[1]);
>>>>+
>>>>+	check_results(pceu1[0], pceu2[0], hwe->engine_class, width, flags);
>>
>>This check_results will check for TEST_BUSY case.
>>
>>>>+
>>>>+	if (flags & TEST_ISOLATION) {
>>>>+		check_results(pceu1[1], pceu2[1], hwe->engine_class, width, 0);
>>
>>This check_results will check for idle case (flags = 0).
>>
>>>>+		close(new_fd);
>>>>+	}
>>>>+
>>>>+	spin_sync_end(fd, ctx);
>>>>+	spin_ctx_destroy(fd, ctx);
>>>
>>>if !BUSY you are not even calling spin_ctx_init()
>>
>>:) yeah, sorry about the readability here. The spin_sync_end and 
>>spin_ctx_destroy bail out if the ctx is NULL which is the case for 
>>!BUSY. I can add and explicit check to not call these instead.
>>
>>Thanks,
>>Umesh
>>>
>>>Lucas De Marchi
>>>
>>>>+	xe_vm_destroy(fd, vm);
>>>>+}
>>>>+
>>>>igt_main
>>>>{
>>>>+	struct drm_xe_engine_class_instance *hwe;
>>>>	int xe;
>>>>
>>>>	igt_fixture {
>>>>@@ -484,6 +569,18 @@ igt_main
>>>>	igt_subtest("basic-engine-utilization")
>>>>		basic_engine_utilization(xe);
>>>>
>>>>+	igt_subtest("drm-idle")
>>>>+		xe_for_each_engine(xe, hwe)
>>>>+			single(xe, hwe, 1, 1, 0);
>>>>+
>>>>+	igt_subtest("drm-busy-idle")
>>>>+		xe_for_each_engine(xe, hwe)
>>>>+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
>>>>+
>>>>+	igt_subtest("drm-busy-idle-isolation")
>>>>+		xe_for_each_engine(xe, hwe)
>>>>+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
>>>>+
>>>>	igt_describe("Create and compare total and resident memory consumption by client");
>>>>	igt_subtest("drm-total-resident")
>>>>		test_total_resident(xe);
>>>>-- 
>>>>2.38.1
>>>>

[PATCH i-g-t v2 06/10] tests/intel/xe_drm_fdinfo: Add tests to verify all class utilization

[Umesh Nerlige Ramappa]

Verify utilization for all classes with varying loads.

v2:
- Drop unused ISOLATION flag in some tests.
- s/measured_usleep/usleep
- remove xe_ prefix from helpers/structures
- Drop unused parameters to all_busy_check_all
- s/_class/class

Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
---
 tests/intel/xe_drm_fdinfo.c | 130 ++++++++++++++++++++++++++++++++++++
 1 file changed, 130 insertions(+)

diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
index 410c885e7..f2051c422 100644
--- a/tests/intel/xe_drm_fdinfo.c
+++ b/tests/intel/xe_drm_fdinfo.c
@@ -37,6 +37,15 @@
  * SUBTEST: drm-busy-idle-isolation
  * Description: Check that engine load does not spill over to other drm clients
  *
+ * SUBTEST: drm-busy-idle-check-all
+ * Description: Check that only the target engine shows load when idle after busy
+ *
+ * SUBTEST: drm-most-busy-idle-check-all
+ * Description: Check that only the target engine shows idle and all others are busy
+ *
+ * SUBTEST: drm-all-busy-idle-check-all
+ * Description: Check that all engines show busy when all are loaded
+ *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
  *
@@ -548,6 +557,116 @@ single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
 	xe_vm_destroy(fd, vm);
 }
 
+static void
+busy_check_all(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
+	       unsigned int flags)
+{
+	struct pceu_cycles pceu1[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct spin_ctx *ctx = NULL;
+	uint32_t vm;
+	int class;
+
+	vm = xe_vm_create(fd, 0, 0);
+	if (flags & TEST_BUSY) {
+		ctx = spin_ctx_init(fd, hwe, vm, width, count);
+		spin_sync_start(fd, ctx);
+	}
+
+	read_engine_cycles(fd, pceu1);
+	usleep(batch_duration_ns / 1000);
+	if (flags & TEST_TRAILING_IDLE)
+		spin_sync_end(fd, ctx);
+	read_engine_cycles(fd, pceu2);
+
+	xe_for_each_engine_class(class)
+		check_results(pceu1, pceu2, class, width,
+			      hwe->engine_class == class ? flags : 0);
+
+	spin_sync_end(fd, ctx);
+	spin_ctx_destroy(fd, ctx);
+	xe_vm_destroy(fd, vm);
+}
+
+static void
+most_busy_check_all(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
+		    unsigned int flags)
+{
+	struct pceu_cycles pceu1[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct spin_ctx *ctx[DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};
+	struct drm_xe_engine_class_instance *_hwe;
+	uint32_t vm;
+	int class;
+
+	vm = xe_vm_create(fd, 0, 0);
+	if (flags & TEST_BUSY) {
+		/* spin on one hwe per class except the target class hwes */
+		xe_for_each_engine(fd, _hwe) {
+			int _class = _hwe->engine_class;
+
+			if (_class == hwe->engine_class || ctx[_class])
+				continue;
+
+			ctx[_class] = spin_ctx_init(fd, _hwe, vm, width, count);
+			spin_sync_start(fd, ctx[_class]);
+		}
+	}
+
+	read_engine_cycles(fd, pceu1);
+	usleep(batch_duration_ns / 1000);
+	if (flags & TEST_TRAILING_IDLE)
+		xe_for_each_engine_class(class)
+			spin_sync_end(fd, ctx[class]);
+	read_engine_cycles(fd, pceu2);
+
+	xe_for_each_engine_class(class) {
+		check_results(pceu1, pceu2, class, width,
+			      hwe->engine_class == class ? 0 : flags);
+		spin_sync_end(fd, ctx[class]);
+		spin_ctx_destroy(fd, ctx[class]);
+	}
+	xe_vm_destroy(fd, vm);
+}
+
+static void
+all_busy_check_all(int fd, unsigned int flags)
+{
+	struct pceu_cycles pceu1[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct spin_ctx *ctx[DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};
+	struct drm_xe_engine_class_instance *hwe;
+	uint32_t vm;
+	int class;
+
+	vm = xe_vm_create(fd, 0, 0);
+	if (flags & TEST_BUSY) {
+		/* spin on one hwe per class */
+		xe_for_each_engine(fd, hwe) {
+			class = hwe->engine_class;
+
+			if (ctx[class])
+				continue;
+
+			ctx[class] = spin_ctx_init(fd, hwe, vm, 1, 1);
+			spin_sync_start(fd, ctx[class]);
+		}
+	}
+
+	read_engine_cycles(fd, pceu1);
+	usleep(batch_duration_ns / 1000);
+	if (flags & TEST_TRAILING_IDLE)
+		xe_for_each_engine_class(class)
+			spin_sync_end(fd, ctx[class]);
+	read_engine_cycles(fd, pceu2);
+
+	xe_for_each_engine_class(class) {
+		check_results(pceu1, pceu2, class, 1, flags);
+		spin_sync_end(fd, ctx[class]);
+		spin_ctx_destroy(fd, ctx[class]);
+	}
+	xe_vm_destroy(fd, vm);
+}
 igt_main
 {
 	struct drm_xe_engine_class_instance *hwe;
@@ -581,6 +700,17 @@ igt_main
 		xe_for_each_engine(xe, hwe)
 			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
 
+	igt_subtest("drm-busy-idle-check-all")
+		xe_for_each_engine(xe, hwe)
+			busy_check_all(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-most-busy-idle-check-all")
+		xe_for_each_engine(xe, hwe)
+			most_busy_check_all(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-all-busy-idle-check-all")
+		all_busy_check_all(xe, TEST_BUSY | TEST_TRAILING_IDLE);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
-- 
2.38.1

[PATCH i-g-t v2 07/10] tests/intel/xe_drm_fdinfo: Add an iterator for virtual engines

[Umesh Nerlige Ramappa]

Add a helper iterator for virtual engines.

Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
---
 tests/intel/xe_drm_fdinfo.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
index f2051c422..9d3d1b285 100644
--- a/tests/intel/xe_drm_fdinfo.c
+++ b/tests/intel/xe_drm_fdinfo.c
@@ -90,6 +90,34 @@ static const uint64_t batch_addr[] = {
 	0x1d0000,
 	0x1e0000,
 };
+
+#define MAX_GTS 2
+#define MAX_INSTANCE 9
+struct virtual_hwe {
+	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
+	int count;
+} vhwe[MAX_GTS][DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};
+
+static void list_virtual_engines(int fd)
+{
+	struct drm_xe_engine_class_instance *hwe;
+
+	xe_for_each_engine(fd, hwe) {
+		struct virtual_hwe *v;
+
+		igt_assert(hwe->gt_id < MAX_GTS);
+		igt_assert(hwe->engine_class < DRM_XE_ENGINE_CLASS_COMPUTE + 1);
+		v = &vhwe[hwe->gt_id][hwe->engine_class];
+
+		igt_assert(v->count < MAX_INSTANCE);
+		v->eci[v->count++] = *hwe;
+	}
+}
+#define xe_for_each_multi_engine(__fd, __hwe, __count) \
+	for (int igt_unique(gt) = 0; igt_unique(gt) < MAX_GTS; igt_unique(gt)++) \
+		for (int igt_unique(c) = 0; igt_unique(c) < DRM_XE_ENGINE_CLASS_COMPUTE + 1; igt_unique(c)++) \
+			for_if((__hwe = &vhwe[igt_unique(gt)][igt_unique(c)].eci[0]) && ((__count = vhwe[igt_unique(gt)][igt_unique(c)].count) > 1))
+
 static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
 {
 	struct drm_client_fdinfo info = { };
@@ -678,6 +706,7 @@ igt_main
 		xe = drm_open_driver(DRIVER_XE);
 		igt_require_xe(xe);
 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
+		list_virtual_engines(xe);
 	}
 
 	igt_describe("Check if basic fdinfo content is present for memory");
-- 
2.38.1

Re: [PATCH i-g-t v2 07/10] tests/intel/xe_drm_fdinfo: Add an iterator for virtual engines

[Lucas De Marchi]

On Tue, Jul 02, 2024 at 05:25:29PM GMT, Umesh Nerlige Ramappa wrote:
>Add a helper iterator for virtual engines.
>
>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>---
> tests/intel/xe_drm_fdinfo.c | 29 +++++++++++++++++++++++++++++
> 1 file changed, 29 insertions(+)
>
>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>index f2051c422..9d3d1b285 100644
>--- a/tests/intel/xe_drm_fdinfo.c
>+++ b/tests/intel/xe_drm_fdinfo.c
>@@ -90,6 +90,34 @@ static const uint64_t batch_addr[] = {
> 	0x1d0000,
> 	0x1e0000,
> };
>+
>+#define MAX_GTS 2
>+#define MAX_INSTANCE 9
>+struct virtual_hwe {
>+	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
>+	int count;
>+} vhwe[MAX_GTS][DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};

there's no concept of "virtual engine". I think this is something
brought over from i915? Is it related to the width when submitting?

Please add a comment for the confused reader and it's probably worth
this abstraction to be outside this test. +Matt Brost

>+
>+static void list_virtual_engines(int fd)

list? shouldn't it be virtual_engines_init()?

Lucas De Marchi

>+{
>+	struct drm_xe_engine_class_instance *hwe;
>+
>+	xe_for_each_engine(fd, hwe) {
>+		struct virtual_hwe *v;
>+
>+		igt_assert(hwe->gt_id < MAX_GTS);
>+		igt_assert(hwe->engine_class < DRM_XE_ENGINE_CLASS_COMPUTE + 1);
>+		v = &vhwe[hwe->gt_id][hwe->engine_class];
>+
>+		igt_assert(v->count < MAX_INSTANCE);
>+		v->eci[v->count++] = *hwe;
>+	}
>+}
>+#define xe_for_each_multi_engine(__fd, __hwe, __count) \
>+	for (int igt_unique(gt) = 0; igt_unique(gt) < MAX_GTS; igt_unique(gt)++) \
>+		for (int igt_unique(c) = 0; igt_unique(c) < DRM_XE_ENGINE_CLASS_COMPUTE + 1; igt_unique(c)++) \
>+			for_if((__hwe = &vhwe[igt_unique(gt)][igt_unique(c)].eci[0]) && ((__count = vhwe[igt_unique(gt)][igt_unique(c)].count) > 1))
>+
> static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
> {
> 	struct drm_client_fdinfo info = { };
>@@ -678,6 +706,7 @@ igt_main
> 		xe = drm_open_driver(DRIVER_XE);
> 		igt_require_xe(xe);
> 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
>+		list_virtual_engines(xe);
> 	}
>
> 	igt_describe("Check if basic fdinfo content is present for memory");
>-- 
>2.38.1
>

Re: [PATCH i-g-t v2 07/10] tests/intel/xe_drm_fdinfo: Add an iterator for virtual engines

[Umesh Nerlige Ramappa]

On Thu, Jul 11, 2024 at 08:34:03AM -0500, Lucas De Marchi wrote:
>On Tue, Jul 02, 2024 at 05:25:29PM GMT, Umesh Nerlige Ramappa wrote:
>>Add a helper iterator for virtual engines.
>>
>>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>>---
>>tests/intel/xe_drm_fdinfo.c | 29 +++++++++++++++++++++++++++++
>>1 file changed, 29 insertions(+)
>>
>>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>>index f2051c422..9d3d1b285 100644
>>--- a/tests/intel/xe_drm_fdinfo.c
>>+++ b/tests/intel/xe_drm_fdinfo.c
>>@@ -90,6 +90,34 @@ static const uint64_t batch_addr[] = {
>>	0x1d0000,
>>	0x1e0000,
>>};
>>+
>>+#define MAX_GTS 2
>>+#define MAX_INSTANCE 9
>>+struct virtual_hwe {
>>+	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
>>+	int count;
>>+} vhwe[MAX_GTS][DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};
>
>there's no concept of "virtual engine". I think this is something
>brought over from i915? Is it related to the width when submitting?
>
>Please add a comment for the confused reader and it's probably worth
>this abstraction to be outside this test. +Matt Brost

width is for parallel submission and num_placements is for virtual.

>
>>+
>>+static void list_virtual_engines(int fd)
>
>list? shouldn't it be virtual_engines_init()?

Right, I will change that.

Thanks,
Umesh

>
>Lucas De Marchi
>
>>+{
>>+	struct drm_xe_engine_class_instance *hwe;
>>+
>>+	xe_for_each_engine(fd, hwe) {
>>+		struct virtual_hwe *v;
>>+
>>+		igt_assert(hwe->gt_id < MAX_GTS);
>>+		igt_assert(hwe->engine_class < DRM_XE_ENGINE_CLASS_COMPUTE + 1);
>>+		v = &vhwe[hwe->gt_id][hwe->engine_class];
>>+
>>+		igt_assert(v->count < MAX_INSTANCE);
>>+		v->eci[v->count++] = *hwe;
>>+	}
>>+}
>>+#define xe_for_each_multi_engine(__fd, __hwe, __count) \
>>+	for (int igt_unique(gt) = 0; igt_unique(gt) < MAX_GTS; igt_unique(gt)++) \
>>+		for (int igt_unique(c) = 0; igt_unique(c) < DRM_XE_ENGINE_CLASS_COMPUTE + 1; igt_unique(c)++) \
>>+			for_if((__hwe = &vhwe[igt_unique(gt)][igt_unique(c)].eci[0]) && ((__count = vhwe[igt_unique(gt)][igt_unique(c)].count) > 1))
>>+
>>static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
>>{
>>	struct drm_client_fdinfo info = { };
>>@@ -678,6 +706,7 @@ igt_main
>>		xe = drm_open_driver(DRIVER_XE);
>>		igt_require_xe(xe);
>>		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
>>+		list_virtual_engines(xe);
>>	}
>>
>>	igt_describe("Check if basic fdinfo content is present for memory");
>>-- 
>>2.38.1
>>

Re: [PATCH i-g-t v2 07/10] tests/intel/xe_drm_fdinfo: Add an iterator for virtual engines

[Lucas De Marchi]

On Thu, Jul 11, 2024 at 11:31:37AM GMT, Umesh Nerlige Ramappa wrote:
>On Thu, Jul 11, 2024 at 08:34:03AM -0500, Lucas De Marchi wrote:
>>On Tue, Jul 02, 2024 at 05:25:29PM GMT, Umesh Nerlige Ramappa wrote:
>>>Add a helper iterator for virtual engines.
>>>
>>>Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>>>---
>>>tests/intel/xe_drm_fdinfo.c | 29 +++++++++++++++++++++++++++++
>>>1 file changed, 29 insertions(+)
>>>
>>>diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
>>>index f2051c422..9d3d1b285 100644
>>>--- a/tests/intel/xe_drm_fdinfo.c
>>>+++ b/tests/intel/xe_drm_fdinfo.c
>>>@@ -90,6 +90,34 @@ static const uint64_t batch_addr[] = {
>>>	0x1d0000,
>>>	0x1e0000,
>>>};
>>>+
>>>+#define MAX_GTS 2
>>>+#define MAX_INSTANCE 9
>>>+struct virtual_hwe {
>>>+	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
>>>+	int count;
>>>+} vhwe[MAX_GTS][DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};
>>
>>there's no concept of "virtual engine". I think this is something
>>brought over from i915? Is it related to the width when submitting?
>>
>>Please add a comment for the confused reader and it's probably worth
>>this abstraction to be outside this test. +Matt Brost
>
>width is for parallel submission and num_placements is for virtual.

ok... let's also s/virtual engines/virtual exec_queues/  where
appropriate. Like we have in xe_exec_balancer.c

can we also use the helper in xe_exec_balancer?


Lucas De Marchi

>
>>
>>>+
>>>+static void list_virtual_engines(int fd)
>>
>>list? shouldn't it be virtual_engines_init()?
>
>Right, I will change that.
>
>Thanks,
>Umesh
>
>>
>>Lucas De Marchi
>>
>>>+{
>>>+	struct drm_xe_engine_class_instance *hwe;
>>>+
>>>+	xe_for_each_engine(fd, hwe) {
>>>+		struct virtual_hwe *v;
>>>+
>>>+		igt_assert(hwe->gt_id < MAX_GTS);
>>>+		igt_assert(hwe->engine_class < DRM_XE_ENGINE_CLASS_COMPUTE + 1);
>>>+		v = &vhwe[hwe->gt_id][hwe->engine_class];
>>>+
>>>+		igt_assert(v->count < MAX_INSTANCE);
>>>+		v->eci[v->count++] = *hwe;
>>>+	}
>>>+}
>>>+#define xe_for_each_multi_engine(__fd, __hwe, __count) \
>>>+	for (int igt_unique(gt) = 0; igt_unique(gt) < MAX_GTS; igt_unique(gt)++) \
>>>+		for (int igt_unique(c) = 0; igt_unique(c) < DRM_XE_ENGINE_CLASS_COMPUTE + 1; igt_unique(c)++) \
>>>+			for_if((__hwe = &vhwe[igt_unique(gt)][igt_unique(c)].eci[0]) && ((__count = vhwe[igt_unique(gt)][igt_unique(c)].count) > 1))
>>>+
>>>static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
>>>{
>>>	struct drm_client_fdinfo info = { };
>>>@@ -678,6 +706,7 @@ igt_main
>>>		xe = drm_open_driver(DRIVER_XE);
>>>		igt_require_xe(xe);
>>>		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
>>>+		list_virtual_engines(xe);
>>>	}
>>>
>>>	igt_describe("Check if basic fdinfo content is present for memory");
>>>-- 
>>>2.38.1
>>>

Re: [PATCH i-g-t v2 07/10] tests/intel/xe_drm_fdinfo: Add an iterator for virtual engines

[Matthew Brost]

On Thu, Jul 11, 2024 at 05:30:14PM -0500, Lucas De Marchi wrote:
> On Thu, Jul 11, 2024 at 11:31:37AM GMT, Umesh Nerlige Ramappa wrote:
> > On Thu, Jul 11, 2024 at 08:34:03AM -0500, Lucas De Marchi wrote:
> > > On Tue, Jul 02, 2024 at 05:25:29PM GMT, Umesh Nerlige Ramappa wrote:
> > > > Add a helper iterator for virtual engines.
> > > > 
> > > > Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
> > > > ---
> > > > tests/intel/xe_drm_fdinfo.c | 29 +++++++++++++++++++++++++++++
> > > > 1 file changed, 29 insertions(+)
> > > > 
> > > > diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
> > > > index f2051c422..9d3d1b285 100644
> > > > --- a/tests/intel/xe_drm_fdinfo.c
> > > > +++ b/tests/intel/xe_drm_fdinfo.c
> > > > @@ -90,6 +90,34 @@ static const uint64_t batch_addr[] = {
> > > > 	0x1d0000,
> > > > 	0x1e0000,
> > > > };
> > > > +
> > > > +#define MAX_GTS 2
> > > > +#define MAX_INSTANCE 9
> > > > +struct virtual_hwe {
> > > > +	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
> > > > +	int count;
> > > > +} vhwe[MAX_GTS][DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};
> > > 
> > > there's no concept of "virtual engine". I think this is something
> > > brought over from i915? Is it related to the width when submitting?
> > > 
> > > Please add a comment for the confused reader and it's probably worth
> > > this abstraction to be outside this test. +Matt Brost
> > 
> > width is for parallel submission and num_placements is for virtual.
> 
> ok... let's also s/virtual engines/virtual exec_queues/  where
> appropriate. Like we have in xe_exec_balancer.c
> 
> can we also use the helper in xe_exec_balancer?

I'm with Lucas that this looks overly complicated.

Perhaps this code from xe_exec_balancer.c can be moved to helper
somewhere?

194         xe_for_each_engine(fd, hwe) {
195                 if (hwe->engine_class != class || hwe->gt_id != gt)
196                         continue;
197
198                 eci[num_placements++] = *hwe;
199         }
200         if (num_placements < 2)
201                 return;

This code collects the hwe instances for a class / gt + calculates the
number of placements. Once you have that calling __xe_exec_queue_create
is pretty simple.

So something like this:

int
xe_exec_queue_virtaul_setup(int fd, struct drm_xe_engine_class_instance *eci,
			    int class, int gt)
{
	struct drm_xe_engine_class_instance *hwe;
	int num_placements = 0;

        xe_for_each_engine(fd, hwe) {
                 if (hwe->engine_class != class || hwe->gt_id != gt)
                         continue;

                 eci[num_placements++] = *hwe;
        }

	return num_placements;
}

Then your test function is something like:

int foo_test(int fd, int class, int gt)
{
	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
	int num_placement;

	num_placement = xe_exec_queue_virtaul_setup(fd, eci, class, gt);
	if (num_placement < 2)
		return;

	/* Do test */
}

Then your main IGT loop is:

	igt_subtest("foo_test")
		xe_for_each_gt(fd, gt)
			xe_for_each_engine_class(class)
				foo_test(fd, class, gt);

This is the pattern in xe_exec_balancer.c. I don't think we should
reinvent this (it is helpful if IGTs all look fairly similar so if you
unstand one, you probably understand most).

Matt

> 
> 
> Lucas De Marchi
> 
> > 
> > > 
> > > > +
> > > > +static void list_virtual_engines(int fd)
> > > 
> > > list? shouldn't it be virtual_engines_init()?
> > 
> > Right, I will change that.
> > 
> > Thanks,
> > Umesh
> > 
> > > 
> > > Lucas De Marchi
> > > 
> > > > +{
> > > > +	struct drm_xe_engine_class_instance *hwe;
> > > > +
> > > > +	xe_for_each_engine(fd, hwe) {
> > > > +		struct virtual_hwe *v;
> > > > +
> > > > +		igt_assert(hwe->gt_id < MAX_GTS);
> > > > +		igt_assert(hwe->engine_class < DRM_XE_ENGINE_CLASS_COMPUTE + 1);
> > > > +		v = &vhwe[hwe->gt_id][hwe->engine_class];
> > > > +
> > > > +		igt_assert(v->count < MAX_INSTANCE);
> > > > +		v->eci[v->count++] = *hwe;
> > > > +	}
> > > > +}
> > > > +#define xe_for_each_multi_engine(__fd, __hwe, __count) \
> > > > +	for (int igt_unique(gt) = 0; igt_unique(gt) < MAX_GTS; igt_unique(gt)++) \
> > > > +		for (int igt_unique(c) = 0; igt_unique(c) < DRM_XE_ENGINE_CLASS_COMPUTE + 1; igt_unique(c)++) \
> > > > +			for_if((__hwe = &vhwe[igt_unique(gt)][igt_unique(c)].eci[0]) && ((__count = vhwe[igt_unique(gt)][igt_unique(c)].count) > 1))
> > > > +
> > > > static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
> > > > {
> > > > 	struct drm_client_fdinfo info = { };
> > > > @@ -678,6 +706,7 @@ igt_main
> > > > 		xe = drm_open_driver(DRIVER_XE);
> > > > 		igt_require_xe(xe);
> > > > 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
> > > > +		list_virtual_engines(xe);
> > > > 	}
> > > > 
> > > > 	igt_describe("Check if basic fdinfo content is present for memory");
> > > > -- 
> > > > 2.38.1
> > > > 

[PATCH i-g-t v2 08/10] tests/intel/xe_drm_fdinfo: Add tests for virtual engines

[Umesh Nerlige Ramappa]

Add tests to verify utilization on virtual engines.

v2:
- s/measured_usleep/usleep
- Remove xe_ prefix from helpers/structures
- Remove ISOLATION case for multi_all_busy_check_all

Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
---
 tests/intel/xe_drm_fdinfo.c | 71 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 70 insertions(+), 1 deletion(-)

diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
index 9d3d1b285..5a3cd776c 100644
--- a/tests/intel/xe_drm_fdinfo.c
+++ b/tests/intel/xe_drm_fdinfo.c
@@ -46,6 +46,18 @@
  * SUBTEST: drm-all-busy-idle-check-all
  * Description: Check that all engines show busy when all are loaded
  *
+ * SUBTEST: drm-virtual-idle
+ * Description: Check that virtual engines show no load when idle
+ *
+ * SUBTEST: drm-virtual-busy-idle
+ * Description: Check that virtual engines show load when idle after busy
+ *
+ * SUBTEST: drm-virtual-all-busy-idle-check-all
+ * Description: Check that all virtual engines show busy when loaded
+ *
+ * SUBTEST: drm-virtual-busy-idle-isolation
+ * Description: Check that virtual engine load does not spill over to other drm clients
+ *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
  *
@@ -695,10 +707,52 @@ all_busy_check_all(int fd, unsigned int flags)
 	}
 	xe_vm_destroy(fd, vm);
 }
+
+static void
+multi_all_busy_check_all(int fd, unsigned int flags)
+{
+	struct pceu_cycles pceu1[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct spin_ctx *ctx[DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};
+	struct drm_xe_engine_class_instance *hwe;
+	int count, class;
+	uint32_t vm;
+
+	vm = xe_vm_create(fd, 0, 0);
+	if (flags & TEST_BUSY) {
+		xe_for_each_multi_engine(fd, hwe, count) {
+			class = hwe->engine_class;
+			ctx[class] = spin_ctx_init(fd, hwe, vm,
+						      1,
+						      count);
+			spin_sync_start(fd, ctx[class]);
+		}
+	}
+
+	read_engine_cycles(fd, pceu1);
+	usleep(batch_duration_ns / 1000);
+	if (flags & TEST_TRAILING_IDLE) {
+		xe_for_each_multi_engine(fd, hwe, count) {
+			spin_sync_end(fd, ctx[hwe->engine_class]);
+		}
+	}
+	read_engine_cycles(fd, pceu2);
+
+	xe_for_each_multi_engine(fd, hwe, count) {
+		class = hwe->engine_class;
+		check_results(pceu1, pceu2, class,
+			      1,
+			      flags);
+		spin_sync_end(fd, ctx[class]);
+		spin_ctx_destroy(fd, ctx[class]);
+	}
+	xe_vm_destroy(fd, vm);
+}
+
 igt_main
 {
 	struct drm_xe_engine_class_instance *hwe;
-	int xe;
+	int xe, count;
 
 	igt_fixture {
 		struct drm_client_fdinfo info = { };
@@ -740,6 +794,21 @@ igt_main
 	igt_subtest("drm-all-busy-idle-check-all")
 		all_busy_check_all(xe, TEST_BUSY | TEST_TRAILING_IDLE);
 
+	igt_subtest("drm-virtual-idle")
+		xe_for_each_multi_engine(xe, hwe, count)
+			single(xe, hwe, 1, count, 0);
+
+	igt_subtest("drm-virtual-busy-idle")
+		xe_for_each_multi_engine(xe, hwe, count)
+			single(xe, hwe, 1, count, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-virtual-all-busy-idle-check-all")
+		multi_all_busy_check_all(xe, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-virtual-busy-idle-isolation")
+		xe_for_each_multi_engine(xe, hwe, count)
+			single(xe, hwe, 1, count, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
-- 
2.38.1

[PATCH i-g-t v2 09/10] tests/intel/xe_drm_fdinfo: Add tests for parallel engines

[Umesh Nerlige Ramappa]

Add tests to verify utilization on parallel engines.

v2:
- s/measured_usleep/usleep
- remove xe_ prefix from helpers/structures

Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
---
 tests/intel/xe_drm_fdinfo.c | 34 +++++++++++++++++++++++++++++++---
 1 file changed, 31 insertions(+), 3 deletions(-)

diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
index 5a3cd776c..f1e06991c 100644
--- a/tests/intel/xe_drm_fdinfo.c
+++ b/tests/intel/xe_drm_fdinfo.c
@@ -58,6 +58,18 @@
  * SUBTEST: drm-virtual-busy-idle-isolation
  * Description: Check that virtual engine load does not spill over to other drm clients
  *
+ * SUBTEST: drm-parallel-idle
+ * Description: Check that parallel engines show no load when idle
+ *
+ * SUBTEST: drm-parallel-busy-idle
+ * Description: Check that parallel engines show load when idle after busy
+ *
+ * SUBTEST: drm-parallel-all-busy-idle-check-all
+ * Description: Check that all parallel engines show busy when loaded
+ *
+ * SUBTEST: drm-parallel-busy-idle-isolation
+ * Description: Check that parallel engine load does not spill over to other drm clients
+ *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
  *
@@ -76,6 +88,7 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 #define TEST_BUSY		(1 << 0)
 #define TEST_TRAILING_IDLE	(1 << 1)
 #define TEST_ISOLATION		(1 << 2)
+#define EXEC_PARALLEL		(1 << 16)
 
 struct pceu_cycles {
 	uint64_t cycles;
@@ -723,8 +736,8 @@ multi_all_busy_check_all(int fd, unsigned int flags)
 		xe_for_each_multi_engine(fd, hwe, count) {
 			class = hwe->engine_class;
 			ctx[class] = spin_ctx_init(fd, hwe, vm,
-						      1,
-						      count);
+						   flags & EXEC_PARALLEL ? count : 1,
+						   flags & EXEC_PARALLEL ? 1 : count);
 			spin_sync_start(fd, ctx[class]);
 		}
 	}
@@ -741,7 +754,7 @@ multi_all_busy_check_all(int fd, unsigned int flags)
 	xe_for_each_multi_engine(fd, hwe, count) {
 		class = hwe->engine_class;
 		check_results(pceu1, pceu2, class,
-			      1,
+			      flags & EXEC_PARALLEL ? count : 1,
 			      flags);
 		spin_sync_end(fd, ctx[class]);
 		spin_ctx_destroy(fd, ctx[class]);
@@ -809,6 +822,21 @@ igt_main
 		xe_for_each_multi_engine(xe, hwe, count)
 			single(xe, hwe, 1, count, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
 
+	igt_subtest("drm-parallel-idle")
+		xe_for_each_multi_engine(xe, hwe, count)
+			single(xe, hwe, count, 1, 0);
+
+	igt_subtest("drm-parallel-busy-idle")
+		xe_for_each_multi_engine(xe, hwe, count)
+			single(xe, hwe, count, 1, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-parallel-all-busy-idle-check-all")
+		multi_all_busy_check_all(xe, TEST_BUSY | TEST_TRAILING_IDLE | EXEC_PARALLEL);
+
+	igt_subtest("drm-parallel-busy-idle-isolation")
+		xe_for_each_multi_engine(xe, hwe, count)
+			single(xe, hwe, count, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
-- 
2.38.1

[PATCH i-g-t v2 10/10] tests/intel/xe_drm_fdinfo: Ensure queue destroy records load correctly

[Umesh Nerlige Ramappa]

If the exec queue is destroyed before recording the utilization, make
sure the load is correctly captured.

v2:
- s/measured_usleep/usleep
- remove xe_ prefix from helpers/structures

Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
---
 tests/intel/xe_drm_fdinfo.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/tests/intel/xe_drm_fdinfo.c b/tests/intel/xe_drm_fdinfo.c
index f1e06991c..b3c865757 100644
--- a/tests/intel/xe_drm_fdinfo.c
+++ b/tests/intel/xe_drm_fdinfo.c
@@ -70,6 +70,9 @@
  * SUBTEST: drm-parallel-busy-idle-isolation
  * Description: Check that parallel engine load does not spill over to other drm clients
  *
+ * SUBTEST: drm-busy-exec-queue-destroy-idle
+ * Description: Destroy exec queue before idle and ensure load is accurate
+ *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
  *
@@ -567,6 +570,32 @@ check_results(struct pceu_cycles *s1, struct pceu_cycles *s2,
 		igt_assert(!percent);
 }
 
+static void
+single_destroy_queue(int fd, struct drm_xe_engine_class_instance *hwe)
+{
+	struct pceu_cycles pceu1[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct spin_ctx *ctx = NULL;
+	uint32_t vm;
+
+	vm = xe_vm_create(fd, 0, 0);
+	ctx = spin_ctx_init(fd, hwe, vm, 1, 1);
+	spin_sync_start(fd, ctx);
+
+	read_engine_cycles(fd, pceu1);
+	usleep(batch_duration_ns / 1000);
+
+	/* destroy queue before sampling again */
+	spin_sync_end(fd, ctx);
+	spin_ctx_destroy(fd, ctx);
+
+	read_engine_cycles(fd, pceu2);
+
+	xe_vm_destroy(fd, vm);
+
+	check_results(pceu1, pceu2, hwe->engine_class, 1, TEST_BUSY);
+}
+
 static void
 single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
        unsigned int flags)
@@ -837,6 +866,10 @@ igt_main
 		xe_for_each_multi_engine(xe, hwe, count)
 			single(xe, hwe, count, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
 
+	igt_subtest("drm-busy-exec-queue-destroy-idle")
+		xe_for_each_engine(xe, hwe)
+			single_destroy_queue(xe, hwe);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
-- 
2.38.1

✗ Fi.CI.BUILD: failure for Add per-client engine utilization tests (rev3)

[Patchwork]

== Series Details ==

Series: Add per-client engine utilization tests (rev3)
URL   : https://patchwork.freedesktop.org/series/135204/
State : failure

== Summary ==

IGT patchset build failed on latest successful build
d0d71f374c95a89a3bdcd104c7d8c2043f79e37a tests/intel/xe_intel_bb: Reduce render subtest execution time

Tail of build.log:

✓ CI.xeBAT: success for Add per-client engine utilization tests (rev4)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 1488 bytes --]

== Series Details ==

Series: Add per-client engine utilization tests (rev4)
URL   : https://patchwork.freedesktop.org/series/135204/
State : success

== Summary ==

CI Bug Log - changes from XEIGT_7916_BAT -> XEIGTPW_11367_BAT
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  

Participating hosts (5 -> 5)
------------------------------

  No changes in participating hosts

Known issues
------------

  Here are the changes found in XEIGTPW_11367_BAT that come from known issues:

### IGT changes ###

#### Possible fixes ####

  * igt@xe_gt_freq@freq_basic_api:
    - bat-adlp-7:         [FAIL][1] -> [PASS][2]
   [1]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/bat-adlp-7/igt@xe_gt_freq@freq_basic_api.html
   [2]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/bat-adlp-7/igt@xe_gt_freq@freq_basic_api.html

  


Build changes
-------------

  * IGT: IGT_7916 -> IGTPW_11367
  * Linux: xe-1562-2bf4c5214910654a6a50082c90f0f4596ea6b0c6 -> xe-1565-55e18f9be7b75c5226a57eff4a3b3eb6110815c6

  IGTPW_11367: 11367
  IGT_7916: 971cfc9d6afc8242db25d1eabd4ae00890d9144a @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  xe-1562-2bf4c5214910654a6a50082c90f0f4596ea6b0c6: 2bf4c5214910654a6a50082c90f0f4596ea6b0c6
  xe-1565-55e18f9be7b75c5226a57eff4a3b3eb6110815c6: 55e18f9be7b75c5226a57eff4a3b3eb6110815c6

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/index.html

[-- Attachment #2: Type: text/html, Size: 2071 bytes --]

✗ Fi.CI.BAT: failure for Add per-client engine utilization tests (rev4)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 4432 bytes --]

== Series Details ==

Series: Add per-client engine utilization tests (rev4)
URL   : https://patchwork.freedesktop.org/series/135204/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_15034 -> IGTPW_11367
====================================================

Summary
-------

  **FAILURE**

  Serious unknown changes coming with IGTPW_11367 absolutely need to be
  verified manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in IGTPW_11367, please notify your bug team (I915-ci-infra@lists.freedesktop.org) to allow them
  to document this new failure mode, which will reduce false positives in CI.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/index.html

Participating hosts (42 -> 36)
------------------------------

  Missing    (6): fi-kbl-7567u bat-dg1-7 fi-snb-2520m fi-elk-e7500 bat-jsl-3 bat-arlh-2 

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in IGTPW_11367:

### IGT changes ###

#### Possible regressions ####

  * igt@i915_selftest@live@gt_lrc:
    - fi-kbl-x1275:       [PASS][1] -> [INCOMPLETE][2]
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_15034/fi-kbl-x1275/igt@i915_selftest@live@gt_lrc.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/fi-kbl-x1275/igt@i915_selftest@live@gt_lrc.html

  
#### Suppressed ####

  The following results come from untrusted machines, tests, or statuses.
  They do not affect the overall result.

  * igt@i915_selftest@live@gt_lrc:
    - {bat-twl-2}:        [PASS][3] -> [INCOMPLETE][4]
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_15034/bat-twl-2/igt@i915_selftest@live@gt_lrc.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/bat-twl-2/igt@i915_selftest@live@gt_lrc.html

  
Known issues
------------

  Here are the changes found in IGTPW_11367 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@i915_selftest@live@execlists:
    - fi-bsw-n3050:       [PASS][5] -> [ABORT][6] ([i915#10594])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_15034/fi-bsw-n3050/igt@i915_selftest@live@execlists.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/fi-bsw-n3050/igt@i915_selftest@live@execlists.html

  * igt@i915_selftest@live@workarounds:
    - bat-adlp-6:         [PASS][7] -> [INCOMPLETE][8] ([i915#9413])
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_15034/bat-adlp-6/igt@i915_selftest@live@workarounds.html
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/bat-adlp-6/igt@i915_selftest@live@workarounds.html

  * igt@kms_pipe_crc_basic@hang-read-crc@pipe-b-dp-1:
    - bat-dg2-8:          [PASS][9] -> [FAIL][10] ([i915#11379])
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_15034/bat-dg2-8/igt@kms_pipe_crc_basic@hang-read-crc@pipe-b-dp-1.html
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/bat-dg2-8/igt@kms_pipe_crc_basic@hang-read-crc@pipe-b-dp-1.html

  
#### Possible fixes ####

  * igt@kms_frontbuffer_tracking@basic:
    - bat-arls-2:         [DMESG-WARN][11] ([i915#7507]) -> [PASS][12]
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_15034/bat-arls-2/igt@kms_frontbuffer_tracking@basic.html
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/bat-arls-2/igt@kms_frontbuffer_tracking@basic.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [i915#10594]: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/10594
  [i915#11379]: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/11379
  [i915#180]: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/180
  [i915#1982]: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/1982
  [i915#7507]: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/7507
  [i915#9413]: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/9413


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_7916 -> IGTPW_11367

  CI-20190529: 20190529
  CI_DRM_15034: 55e18f9be7b75c5226a57eff4a3b3eb6110815c6 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_11367: 11367
  IGT_7916: 971cfc9d6afc8242db25d1eabd4ae00890d9144a @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/index.html

[-- Attachment #2: Type: text/html, Size: 5044 bytes --]

✓ CI.xeFULL: success for Add per-client engine utilization tests (rev4)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 25628 bytes --]

== Series Details ==

Series: Add per-client engine utilization tests (rev4)
URL   : https://patchwork.freedesktop.org/series/135204/
State : success

== Summary ==

CI Bug Log - changes from XEIGT_7916_full -> XEIGTPW_11367_full
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  

Participating hosts (3 -> 3)
------------------------------

  No changes in participating hosts

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in XEIGTPW_11367_full:

### IGT changes ###

#### Possible regressions ####

  * {igt@kms_getfb} (NEW):
    - {shard-lnl}:        NOTRUN -> [INCOMPLETE][1]
   [1]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-1/igt@kms_getfb.html

  * {igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle} (NEW):
    - shard-dg2-set2:     NOTRUN -> [FAIL][2] +1 other test fail
   [2]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle.html

  * {igt@xe_drm_fdinfo@drm-most-busy-idle-check-all} (NEW):
    - {shard-lnl}:        NOTRUN -> [FAIL][3]
   [3]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-8/igt@xe_drm_fdinfo@drm-most-busy-idle-check-all.html

  
#### Suppressed ####

  The following results come from untrusted machines, tests, or statuses.
  They do not affect the overall result.

  * igt@kms_chamelium_frames@hdmi-cmp-planes-random:
    - {shard-lnl}:        [SKIP][4] ([Intel XE#373]) -> [INCOMPLETE][5]
   [4]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-lnl-5/igt@kms_chamelium_frames@hdmi-cmp-planes-random.html
   [5]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-7/igt@kms_chamelium_frames@hdmi-cmp-planes-random.html

  
New tests
---------

  New tests have been introduced between XEIGT_7916_full and XEIGTPW_11367_full:

### New IGT tests (18) ###

  * igt@kms_getfb:
    - Statuses : 1 incomplete(s)
    - Exec time: [0.0] s

  * igt@xe_drm_fdinfo@basic-engine-utilization:
    - Statuses : 2 pass(s)
    - Exec time: [0.0] s

  * igt@xe_drm_fdinfo@basic-memory:
    - Statuses : 2 pass(s)
    - Exec time: [0.0] s

  * igt@xe_drm_fdinfo@drm-all-busy-idle-check-all:
    - Statuses : 2 pass(s)
    - Exec time: [0.51] s

  * igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle:
    - Statuses : 1 fail(s)
    - Exec time: [0.53] s

  * igt@xe_drm_fdinfo@drm-busy-idle:
    - Statuses : 2 pass(s)
    - Exec time: [2.54, 3.54] s

  * igt@xe_drm_fdinfo@drm-busy-idle-check-all:
    - Statuses : 2 pass(s)
    - Exec time: [2.55, 3.55] s

  * igt@xe_drm_fdinfo@drm-busy-idle-isolation:
    - Statuses : 1 pass(s)
    - Exec time: [3.54] s

  * igt@xe_drm_fdinfo@drm-idle:
    - Statuses : 2 pass(s)
    - Exec time: [2.51, 3.52] s

  * igt@xe_drm_fdinfo@drm-most-busy-idle-check-all:
    - Statuses : 2 fail(s)
    - Exec time: [1.04, 1.56] s

  * igt@xe_drm_fdinfo@drm-parallel-all-busy-idle-check-all:
    - Statuses : 1 pass(s)
    - Exec time: [0.51] s

  * igt@xe_drm_fdinfo@drm-parallel-busy-idle:
    - Statuses : 2 pass(s)
    - Exec time: [0.0, 1.01] s

  * igt@xe_drm_fdinfo@drm-parallel-busy-idle-isolation:
    - Statuses : 1 pass(s)
    - Exec time: [1.01] s

  * igt@xe_drm_fdinfo@drm-parallel-idle:
    - Statuses : 2 pass(s)
    - Exec time: [0.0, 1.00] s

  * igt@xe_drm_fdinfo@drm-virtual-all-busy-idle-check-all:
    - Statuses : 2 pass(s)
    - Exec time: [0.50, 0.51] s

  * igt@xe_drm_fdinfo@drm-virtual-busy-idle:
    - Statuses : 2 pass(s)
    - Exec time: [0.0, 1.01] s

  * igt@xe_drm_fdinfo@drm-virtual-busy-idle-isolation:
    - Statuses : 2 pass(s)
    - Exec time: [0.0, 1.01] s

  * igt@xe_drm_fdinfo@drm-virtual-idle:
    - Statuses : 2 pass(s)
    - Exec time: [0.0, 1.00] s

  

Known issues
------------

  Here are the changes found in XEIGTPW_11367_full that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@core_setmaster@master-drop-set-shared-fd:
    - shard-dg2-set2:     [PASS][6] -> [DMESG-WARN][7] ([Intel XE#1162] / [Intel XE#1214])
   [6]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-434/igt@core_setmaster@master-drop-set-shared-fd.html
   [7]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@core_setmaster@master-drop-set-shared-fd.html

  * igt@kms_async_flips@async-flip-with-page-flip-events:
    - shard-dg2-set2:     [PASS][8] -> [INCOMPLETE][9] ([Intel XE#1150] / [Intel XE#1195])
   [8]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-466/igt@kms_async_flips@async-flip-with-page-flip-events.html
   [9]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-436/igt@kms_async_flips@async-flip-with-page-flip-events.html

  * igt@kms_async_flips@async-flip-with-page-flip-events@pipe-d-hdmi-a-6-x:
    - shard-dg2-set2:     [PASS][10] -> [INCOMPLETE][11] ([Intel XE#1195]) +2 other tests incomplete
   [10]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-466/igt@kms_async_flips@async-flip-with-page-flip-events@pipe-d-hdmi-a-6-x.html
   [11]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-436/igt@kms_async_flips@async-flip-with-page-flip-events@pipe-d-hdmi-a-6-x.html

  * igt@kms_big_fb@linear-8bpp-rotate-270:
    - shard-dg2-set2:     NOTRUN -> [SKIP][12] ([Intel XE#1201] / [Intel XE#316])
   [12]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@kms_big_fb@linear-8bpp-rotate-270.html

  * igt@kms_big_fb@yf-tiled-64bpp-rotate-90:
    - shard-dg2-set2:     NOTRUN -> [SKIP][13] ([Intel XE#1124] / [Intel XE#1201]) +2 other tests skip
   [13]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-466/igt@kms_big_fb@yf-tiled-64bpp-rotate-90.html

  * igt@kms_ccs@ccs-on-another-bo-y-tiled-gen12-rc-ccs:
    - shard-dg2-set2:     NOTRUN -> [SKIP][14] ([Intel XE#1201] / [Intel XE#455] / [Intel XE#787]) +9 other tests skip
   [14]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@kms_ccs@ccs-on-another-bo-y-tiled-gen12-rc-ccs.html

  * igt@kms_ccs@crc-sprite-planes-basic-y-tiled-gen12-mc-ccs@pipe-b-hdmi-a-6:
    - shard-dg2-set2:     NOTRUN -> [SKIP][15] ([Intel XE#1201] / [Intel XE#787]) +34 other tests skip
   [15]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@kms_ccs@crc-sprite-planes-basic-y-tiled-gen12-mc-ccs@pipe-b-hdmi-a-6.html

  * igt@kms_chamelium_color@ctm-0-50:
    - shard-dg2-set2:     NOTRUN -> [SKIP][16] ([Intel XE#1201] / [Intel XE#306])
   [16]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-434/igt@kms_chamelium_color@ctm-0-50.html

  * igt@kms_chamelium_hpd@hdmi-hpd-fast:
    - shard-dg2-set2:     NOTRUN -> [SKIP][17] ([Intel XE#1201] / [Intel XE#373])
   [17]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@kms_chamelium_hpd@hdmi-hpd-fast.html

  * igt@kms_cursor_crc@cursor-offscreen-512x512:
    - shard-dg2-set2:     NOTRUN -> [SKIP][18] ([Intel XE#1201] / [Intel XE#308]) +1 other test skip
   [18]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-434/igt@kms_cursor_crc@cursor-offscreen-512x512.html

  * igt@kms_cursor_legacy@torture-move:
    - shard-dg2-set2:     [PASS][19] -> [DMESG-WARN][20] ([Intel XE#1214] / [Intel XE#877]) +1 other test dmesg-warn
   [19]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-466/igt@kms_cursor_legacy@torture-move.html
   [20]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@kms_cursor_legacy@torture-move.html

  * igt@kms_frontbuffer_tracking@drrs-2p-primscrn-cur-indfb-draw-mmap-wc:
    - shard-dg2-set2:     NOTRUN -> [SKIP][21] ([Intel XE#1201] / [Intel XE#651]) +11 other tests skip
   [21]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-436/igt@kms_frontbuffer_tracking@drrs-2p-primscrn-cur-indfb-draw-mmap-wc.html

  * igt@kms_frontbuffer_tracking@psr-2p-primscrn-spr-indfb-draw-mmap-wc:
    - shard-dg2-set2:     NOTRUN -> [SKIP][22] ([Intel XE#1201] / [Intel XE#653]) +9 other tests skip
   [22]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@kms_frontbuffer_tracking@psr-2p-primscrn-spr-indfb-draw-mmap-wc.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-factor-0-25:
    - shard-dg2-set2:     NOTRUN -> [SKIP][23] ([Intel XE#1201] / [Intel XE#305] / [Intel XE#455]) +1 other test skip
   [23]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-433/igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-factor-0-25.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-factor-0-25@pipe-c-hdmi-a-6:
    - shard-dg2-set2:     NOTRUN -> [SKIP][24] ([Intel XE#1201] / [Intel XE#305]) +2 other tests skip
   [24]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-433/igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-factor-0-25@pipe-c-hdmi-a-6.html

  * igt@kms_pm_backlight@bad-brightness:
    - shard-dg2-set2:     NOTRUN -> [SKIP][25] ([Intel XE#1201] / [Intel XE#870])
   [25]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@kms_pm_backlight@bad-brightness.html

  * igt@kms_psr2_sf@overlay-plane-update-continuous-sf:
    - shard-dg2-set2:     NOTRUN -> [SKIP][26] ([Intel XE#1201] / [Intel XE#1489])
   [26]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-466/igt@kms_psr2_sf@overlay-plane-update-continuous-sf.html

  * igt@kms_psr@fbc-psr2-primary-blt:
    - shard-dg2-set2:     NOTRUN -> [SKIP][27] ([Intel XE#1201] / [Intel XE#929]) +3 other tests skip
   [27]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@kms_psr@fbc-psr2-primary-blt.html

  * igt@kms_rotation_crc@primary-rotation-270:
    - shard-dg2-set2:     NOTRUN -> [SKIP][28] ([Intel XE#1201] / [Intel XE#327])
   [28]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-436/igt@kms_rotation_crc@primary-rotation-270.html

  * igt@kms_setmode@basic-clone-single-crtc:
    - shard-dg2-set2:     NOTRUN -> [SKIP][29] ([Intel XE#1201] / [Intel XE#455]) +1 other test skip
   [29]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-433/igt@kms_setmode@basic-clone-single-crtc.html

  * igt@kms_writeback@writeback-fb-id-xrgb2101010:
    - shard-dg2-set2:     NOTRUN -> [SKIP][30] ([Intel XE#1201] / [Intel XE#756])
   [30]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-433/igt@kms_writeback@writeback-fb-id-xrgb2101010.html

  * igt@xe_evict@evict-cm-threads-large:
    - shard-dg2-set2:     [PASS][31] -> [TIMEOUT][32] ([Intel XE#1473] / [Intel XE#392])
   [31]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-463/igt@xe_evict@evict-cm-threads-large.html
   [32]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-433/igt@xe_evict@evict-cm-threads-large.html

  * igt@xe_evict@evict-threads-large:
    - shard-dg2-set2:     [PASS][33] -> [INCOMPLETE][34] ([Intel XE#1195] / [Intel XE#1473] / [Intel XE#392]) +1 other test incomplete
   [33]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-434/igt@xe_evict@evict-threads-large.html
   [34]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-464/igt@xe_evict@evict-threads-large.html

  * igt@xe_exec_fault_mode@twice-bindexecqueue-rebind-imm:
    - shard-dg2-set2:     NOTRUN -> [SKIP][35] ([Intel XE#1201] / [Intel XE#288]) +8 other tests skip
   [35]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-464/igt@xe_exec_fault_mode@twice-bindexecqueue-rebind-imm.html

  * igt@xe_live_ktest@xe_migrate:
    - shard-dg2-set2:     [PASS][36] -> [SKIP][37] ([Intel XE#1192] / [Intel XE#1201])
   [36]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-463/igt@xe_live_ktest@xe_migrate.html
   [37]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-466/igt@xe_live_ktest@xe_migrate.html

  * igt@xe_pm@s3-vm-bind-unbind-all:
    - shard-dg2-set2:     [PASS][38] -> [DMESG-WARN][39] ([Intel XE#1162] / [Intel XE#1214] / [Intel XE#1941])
   [38]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-464/igt@xe_pm@s3-vm-bind-unbind-all.html
   [39]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-464/igt@xe_pm@s3-vm-bind-unbind-all.html

  * igt@xe_pm@s3-vm-bind-userptr:
    - shard-dg2-set2:     [PASS][40] -> [DMESG-WARN][41] ([Intel XE#1214] / [Intel XE#1551] / [Intel XE#569])
   [40]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-435/igt@xe_pm@s3-vm-bind-userptr.html
   [41]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-436/igt@xe_pm@s3-vm-bind-userptr.html

  * igt@xe_vm@mixed-userptr-binds-1611661312:
    - shard-dg2-set2:     [PASS][42] -> [DMESG-WARN][43] ([Intel XE#1214]) +4 other tests dmesg-warn
   [42]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-436/igt@xe_vm@mixed-userptr-binds-1611661312.html
   [43]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@xe_vm@mixed-userptr-binds-1611661312.html

  
#### Possible fixes ####

  * igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-180:
    - {shard-lnl}:        [FAIL][44] ([Intel XE#1659]) -> [PASS][45]
   [44]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-lnl-8/igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-180.html
   [45]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-2/igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-180.html

  * igt@kms_pipe_crc_basic@suspend-read-crc:
    - shard-dg2-set2:     [DMESG-WARN][46] ([Intel XE#1162] / [Intel XE#1214]) -> [PASS][47]
   [46]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-434/igt@kms_pipe_crc_basic@suspend-read-crc.html
   [47]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@kms_pipe_crc_basic@suspend-read-crc.html

  * {igt@kms_plane@plane-position-covered@pipe-a-plane-1}:
    - {shard-lnl}:        [DMESG-FAIL][48] ([Intel XE#324]) -> [PASS][49] +1 other test pass
   [48]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-lnl-5/igt@kms_plane@plane-position-covered@pipe-a-plane-1.html
   [49]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-2/igt@kms_plane@plane-position-covered@pipe-a-plane-1.html

  * igt@kms_universal_plane@cursor-fb-leak:
    - shard-dg2-set2:     [FAIL][50] ([Intel XE#771] / [Intel XE#899]) -> [PASS][51]
   [50]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-433/igt@kms_universal_plane@cursor-fb-leak.html
   [51]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@kms_universal_plane@cursor-fb-leak.html

  * igt@kms_universal_plane@cursor-fb-leak@pipe-c-hdmi-a-6:
    - shard-dg2-set2:     [FAIL][52] ([Intel XE#899]) -> [PASS][53] +1 other test pass
   [52]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-433/igt@kms_universal_plane@cursor-fb-leak@pipe-c-hdmi-a-6.html
   [53]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@kms_universal_plane@cursor-fb-leak@pipe-c-hdmi-a-6.html

  * igt@xe_evict@evict-beng-threads-large:
    - shard-dg2-set2:     [TIMEOUT][54] ([Intel XE#1473]) -> [PASS][55]
   [54]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-464/igt@xe_evict@evict-beng-threads-large.html
   [55]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-436/igt@xe_evict@evict-beng-threads-large.html

  * igt@xe_evict@evict-mixed-many-threads-small:
    - shard-dg2-set2:     [INCOMPLETE][56] ([Intel XE#1195] / [Intel XE#1473]) -> [PASS][57]
   [56]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-466/igt@xe_evict@evict-mixed-many-threads-small.html
   [57]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@xe_evict@evict-mixed-many-threads-small.html

  * igt@xe_exec_basic@many-execqueues-bindexecqueue-rebind:
    - {shard-lnl}:        [FAIL][58] -> [PASS][59]
   [58]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-lnl-5/igt@xe_exec_basic@many-execqueues-bindexecqueue-rebind.html
   [59]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-4/igt@xe_exec_basic@many-execqueues-bindexecqueue-rebind.html

  * igt@xe_live_ktest@xe_bo:
    - {shard-lnl}:        [SKIP][60] ([Intel XE#1192]) -> [PASS][61]
   [60]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-lnl-4/igt@xe_live_ktest@xe_bo.html
   [61]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-6/igt@xe_live_ktest@xe_bo.html

  * igt@xe_module_load@reload:
    - shard-dg2-set2:     [DMESG-WARN][62] ([Intel XE#1214]) -> [PASS][63] +7 other tests pass
   [62]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-463/igt@xe_module_load@reload.html
   [63]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@xe_module_load@reload.html

  * igt@xe_pm@s2idle-vm-bind-userptr:
    - shard-dg2-set2:     [INCOMPLETE][64] ([Intel XE#1195] / [Intel XE#1694]) -> [PASS][65]
   [64]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-435/igt@xe_pm@s2idle-vm-bind-userptr.html
   [65]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-464/igt@xe_pm@s2idle-vm-bind-userptr.html

  * igt@xe_pm@s3-vm-bind-prefetch:
    - shard-dg2-set2:     [DMESG-WARN][66] ([Intel XE#1214] / [Intel XE#569]) -> [PASS][67]
   [66]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-434/igt@xe_pm@s3-vm-bind-prefetch.html
   [67]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-434/igt@xe_pm@s3-vm-bind-prefetch.html

  
#### Warnings ####

  * igt@kms_content_protection@lic-type-0:
    - shard-dg2-set2:     [INCOMPLETE][68] ([Intel XE#1195]) -> [FAIL][69] ([Intel XE#1204]) +1 other test fail
   [68]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-435/igt@kms_content_protection@lic-type-0.html
   [69]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@kms_content_protection@lic-type-0.html

  * igt@kms_plane@plane-panning-bottom-right-suspend@pipe-b:
    - shard-dg2-set2:     [DMESG-FAIL][70] ([Intel XE#1551]) -> [FAIL][71] ([Intel XE#616]) +1 other test fail
   [70]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-433/igt@kms_plane@plane-panning-bottom-right-suspend@pipe-b.html
   [71]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@kms_plane@plane-panning-bottom-right-suspend@pipe-b.html

  * igt@xe_evict@evict-mixed-threads-large:
    - shard-dg2-set2:     [TIMEOUT][72] ([Intel XE#1473] / [Intel XE#392]) -> [DMESG-FAIL][73] ([Intel XE#482])
   [72]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-463/igt@xe_evict@evict-mixed-threads-large.html
   [73]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-466/igt@xe_evict@evict-mixed-threads-large.html

  * igt@xe_live_ktest@xe_mocs:
    - shard-dg2-set2:     [FAIL][74] ([Intel XE#1999]) -> [SKIP][75] ([Intel XE#1192] / [Intel XE#1201])
   [74]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-463/igt@xe_live_ktest@xe_mocs.html
   [75]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-464/igt@xe_live_ktest@xe_mocs.html

  * igt@xe_wedged@wedged-at-any-timeout:
    - shard-dg2-set2:     [DMESG-WARN][76] ([Intel XE#1214] / [Intel XE#1760]) -> [DMESG-FAIL][77] ([Intel XE#1760])
   [76]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-433/igt@xe_wedged@wedged-at-any-timeout.html
   [77]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-464/igt@xe_wedged@wedged-at-any-timeout.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [Intel XE#1124]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1124
  [Intel XE#1127]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1127
  [Intel XE#1128]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1128
  [Intel XE#1150]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1150
  [Intel XE#1162]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1162
  [Intel XE#1192]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1192
  [Intel XE#1195]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1195
  [Intel XE#1201]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1201
  [Intel XE#1204]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1204
  [Intel XE#1214]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1214
  [Intel XE#1392]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1392
  [Intel XE#1397]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1397
  [Intel XE#1399]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1399
  [Intel XE#1401]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1401
  [Intel XE#1406]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1406
  [Intel XE#1407]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1407
  [Intel XE#1413]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1413
  [Intel XE#1421]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1421
  [Intel XE#1424]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1424
  [Intel XE#1430]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1430
  [Intel XE#1435]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1435
  [Intel XE#1437]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1437
  [Intel XE#1442]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1442
  [Intel XE#1447]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1447
  [Intel XE#1473]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1473
  [Intel XE#1489]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1489
  [Intel XE#1551]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1551
  [Intel XE#1659]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1659
  [Intel XE#1694]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1694
  [Intel XE#1745]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1745
  [Intel XE#1760]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1760
  [Intel XE#1941]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1941
  [Intel XE#1948]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1948
  [Intel XE#1999]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1999
  [Intel XE#2191]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2191
  [Intel XE#288]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/288
  [Intel XE#305]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/305
  [Intel XE#306]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/306
  [Intel XE#307]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/307
  [Intel XE#308]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/308
  [Intel XE#309]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/309
  [Intel XE#316]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/316
  [Intel XE#323]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/323
  [Intel XE#324]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/324
  [Intel XE#327]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/327
  [Intel XE#330]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/330
  [Intel XE#346]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/346
  [Intel XE#352]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/352
  [Intel XE#366]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/366
  [Intel XE#367]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/367
  [Intel XE#373]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/373
  [Intel XE#392]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/392
  [Intel XE#455]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/455
  [Intel XE#482]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/482
  [Intel XE#498]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/498
  [Intel XE#569]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/569
  [Intel XE#584]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/584
  [Intel XE#599]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/599
  [Intel XE#616]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/616
  [Intel XE#651]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/651
  [Intel XE#653]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/653
  [Intel XE#656]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/656
  [Intel XE#688]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/688
  [Intel XE#756]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/756
  [Intel XE#771]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/771
  [Intel XE#787]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/787
  [Intel XE#870]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/870
  [Intel XE#877]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/877
  [Intel XE#886]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/886
  [Intel XE#899]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/899
  [Intel XE#929]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/929
  [Intel XE#944]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/944


Build changes
-------------

  * IGT: IGT_7916 -> IGTPW_11367
  * Linux: xe-1562-2bf4c5214910654a6a50082c90f0f4596ea6b0c6 -> xe-1565-55e18f9be7b75c5226a57eff4a3b3eb6110815c6

  IGTPW_11367: 11367
  IGT_7916: 971cfc9d6afc8242db25d1eabd4ae00890d9144a @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  xe-1562-2bf4c5214910654a6a50082c90f0f4596ea6b0c6: 2bf4c5214910654a6a50082c90f0f4596ea6b0c6
  xe-1565-55e18f9be7b75c5226a57eff4a3b3eb6110815c6: 55e18f9be7b75c5226a57eff4a3b3eb6110815c6

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/index.html

[-- Attachment #2: Type: text/html, Size: 28887 bytes --]