[igt-dev] [PATCH i-g-t v2 0/8] D3Cold Tool & IGT

[igt-dev] [PATCH i-g-t v2 0/8] D3Cold Tool & IGT

[Anshuman Gupta]

This series adds support for D3Cold Tool and IGT.

v2 rev has addressed the review comment mentioned in 
commit version log.

Anshuman Gupta (8):
  test/i915_pm_rpm: Add placement to gem-{mmap-type, execbuf}
  lib/igt_device: Get gfx PCI card root port
  lib/igt_pm: D3Cold runtime pm infrastructure
  tools: Add intel_pm_rpm tool
  i915_pm_rpm: Add D3Cold basic subtest
  i915_pm_rpm: Extend gem_exec_stress test with D3Cold
  i915_pm_rpm: Extend gem_execbuf test with D3Cold
  i915_pm_rpm: Extend gem-mmap-type test with D3Cold

 lib/igt_device.c         |  28 ++++
 lib/igt_device.h         |   1 +
 lib/igt_pm.c             | 290 +++++++++++++++++++++++++++++++++++++++
 lib/igt_pm.h             |  23 ++++
 tests/i915/i915_pm_rpm.c | 190 +++++++++++++++++++++++--
 tools/intel_pm_rpm.c     | 206 +++++++++++++++++++++++++++
 tools/meson.build        |   1 +
 7 files changed, 731 insertions(+), 8 deletions(-)
 create mode 100644 tools/intel_pm_rpm.c

-- 
2.26.2

[igt-dev] [PATCH i-g-t v2 1/8] test/i915_pm_rpm: Add placement to gem-{mmap-type, execbuf}

[Anshuman Gupta]

Add memory region placement to gem-{mmap-type, execbuf} subtest.
This will be useful to extend these sub-test for D3Cold.

Cc: Chris Wilson <chris.p.wilson@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 tests/i915/i915_pm_rpm.c | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/tests/i915/i915_pm_rpm.c b/tests/i915/i915_pm_rpm.c
index 461730e8a..d2bce5826 100644
--- a/tests/i915/i915_pm_rpm.c
+++ b/tests/i915/i915_pm_rpm.c
@@ -1069,7 +1069,8 @@ static void debugfs_forcewake_user_subtest(void)
 	igt_assert(wait_for_suspended());
 }
 
-static void gem_mmap_args(const struct mmap_offset *t)
+static void gem_mmap_args(const struct mmap_offset *t,
+			  struct drm_i915_gem_memory_class_instance *mem_regions)
 {
 	int i;
 	uint32_t handle;
@@ -1079,7 +1080,7 @@ static void gem_mmap_args(const struct mmap_offset *t)
 	/* Create, map and set data while the device is active. */
 	enable_one_screen_and_wait(&ms_data);
 
-	handle = gem_create(drm_fd, buf_size);
+	handle = gem_create_in_memory_region_list(drm_fd, buf_size, mem_regions, 1);
 
 	gem_buf = __gem_mmap_offset(drm_fd, handle, 0, buf_size,
 				    PROT_READ | PROT_WRITE, t->type);
@@ -1290,7 +1291,7 @@ static void submit_blt_cmd(uint32_t dst_handle, int dst_size,
 }
 
 /* Make sure we can submit a batch buffer and verify its result. */
-static void gem_execbuf_subtest(void)
+static void gem_execbuf_subtest(struct drm_i915_gem_memory_class_instance *mem_regions)
 {
 	int x, y;
 	uint32_t handle;
@@ -1308,7 +1309,7 @@ static void gem_execbuf_subtest(void)
 	/* Create and set data while the device is active. */
 	enable_one_screen_and_wait(&ms_data);
 
-	handle = gem_create(drm_fd, dst_size);
+	handle = gem_create_in_memory_region_list(drm_fd, dst_size, mem_regions, 1);
 
 	cpu_buf = malloc(dst_size);
 	igt_assert(cpu_buf);
@@ -2083,15 +2084,21 @@ igt_main_args("", long_options, help_str, opt_handler, NULL)
 	/* GEM */
 	igt_subtest_with_dynamic("gem-mmap-type") {
 		for_each_mmap_offset_type(drm_fd, t) {
-			igt_dynamic_f("%s", t->name)
-				gem_mmap_args(t);
+			for_each_memory_region(r, drm_fd) {
+				igt_dynamic_f("%s-%s", t->name, r->name)
+				gem_mmap_args(t, &r->ci);
+			}
 		}
 	}
 
 	igt_subtest("gem-pread")
 		gem_pread_subtest();
-	igt_subtest("gem-execbuf")
-		gem_execbuf_subtest();
+	igt_subtest_with_dynamic("gem-execbuf") {
+		for_each_memory_region(r, drm_fd) {
+			igt_dynamic_f("%s", r->name)
+				gem_execbuf_subtest(&r->ci);
+		}
+	}
 	igt_subtest("gem-idle")
 		gem_idle_subtest();
 	igt_subtest("gem-evict-pwrite") {
-- 
2.26.2

[igt-dev] [PATCH i-g-t v2 2/8] lib/igt_device: Get gfx PCI card root port

[Anshuman Gupta]

Utility library function to get the Gfx Card PCI topology
root port pci device using libpaciaccess.
root port will be used to get the real ACPI D state.

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 lib/igt_device.c | 28 ++++++++++++++++++++++++++++
 lib/igt_device.h |  1 +
 2 files changed, 29 insertions(+)

diff --git a/lib/igt_device.c b/lib/igt_device.c
index 9794a9928..fddfba72c 100644
--- a/lib/igt_device.c
+++ b/lib/igt_device.c
@@ -257,3 +257,31 @@ struct pci_device *igt_device_get_pci_device(int fd)
 
 	return pci_dev;
 }
+
+/**
+ * igt_device_get_pci_root_port:
+ * @fd: the device.
+ *
+ * Looks up the graphics pci device root port using libpciaccess.
+ *
+ * Returns:
+ * The root port pci_device.
+ */
+struct pci_device *
+igt_device_get_pci_root_port(int fd)
+{
+	struct pci_device *pci_dev, *prev;
+
+	pci_dev = __igt_device_get_pci_device(fd, 0);
+	igt_require(pci_dev);
+
+	while (pci_dev) {
+		prev = pci_dev;
+		pci_dev = pci_device_get_parent_bridge(pci_dev);
+	}
+
+	igt_debug("Root Port PCI device %04x:%02x:%02x.%01x\n",
+		  prev->domain, prev->bus, prev->dev, prev->func);
+
+	return prev;
+}
diff --git a/lib/igt_device.h b/lib/igt_device.h
index 1aaa840e2..800a0fcc3 100644
--- a/lib/igt_device.h
+++ b/lib/igt_device.h
@@ -34,5 +34,6 @@ void igt_device_drop_master(int fd);
 int igt_device_get_card_index(int fd);
 struct pci_device *igt_device_get_pci_device(int fd);
 struct pci_device *__igt_device_get_pci_device(int fd, unsigned int vf_id);
+struct pci_device *igt_device_get_pci_root_port(int fd);
 
 #endif /* __IGT_DEVICE_H__ */
-- 
2.26.2

[igt-dev] [PATCH i-g-t v2 3/8] lib/igt_pm: D3Cold runtime pm infrastructure

[Anshuman Gupta]

Enable gfx card pci devices runtime pm for all pci devices
and bridge under the topology of Gfx Card root port.

Added a library function to get the PCI root port ACPI
D state and to print the pci card devices runtime pm
status.

v2:
- Save pci dev power attr to dynamically allocated array. [Rodrigo]
- Set autosuspend delay to 0 on supported devices. [Rodrigo]
- %s/else if/else in  igt_pm_get_acpi_real_d_state. [Kamil]

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
---
 lib/igt_pm.c | 290 +++++++++++++++++++++++++++++++++++++++++++++++++++
 lib/igt_pm.h |  23 ++++
 2 files changed, 313 insertions(+)

diff --git a/lib/igt_pm.c b/lib/igt_pm.c
index b409ec463..20296c40b 100644
--- a/lib/igt_pm.c
+++ b/lib/igt_pm.c
@@ -28,6 +28,7 @@
 #include <fcntl.h>
 #include <stdio.h>
 #include <limits.h>
+#include <pciaccess.h>
 #include <stdlib.h>
 #include <string.h>
 #include <unistd.h>
@@ -75,6 +76,7 @@ enum {
 #define MIN_POWER_STR		"min_power\n"
 /* Remember to fix this if adding longer strings */
 #define MAX_POLICY_STRLEN	strlen(MAX_PERFORMANCE_STR)
+#define MAX_PCI_DEVICES		256
 int8_t *__sata_pm_policies;
 int __scsi_host_cnt;
 
@@ -856,3 +858,291 @@ bool i915_output_is_lpsp_capable(int drm_fd, igt_output_t *output)
 
 	return strstr(buf, "LPSP: capable");
 }
+
+/**
+ * igt_pm_acpi_d3cold_supported:
+ * @pci_dev: root port pci_dev.
+ * Check ACPI D3Cold support.
+ *
+ * Returns:
+ * True if ACPI D3Cold supported otherwise false.
+ */
+bool igt_pm_acpi_d3cold_supported(struct pci_device *pci_dev)
+{
+	char name[PATH_MAX];
+	int dir, fd;
+
+	snprintf(name, PATH_MAX,
+		 "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/firmware_node",
+		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
+
+	dir = open(name, O_RDONLY);
+	igt_require(dir > 0);
+
+	/* BIOS need to enable ACPI D3Cold Support.*/
+	fd = openat(dir, "real_power_state", O_RDONLY);
+	if (fd < 0 && errno == ENOENT)
+		return false;
+
+	igt_require(fd > 0);
+
+	return true;
+}
+
+/**
+ * igt_pm_get_acpi_real_d_state:
+ * @pci_dev: root port pci_dev.
+ * Get ACPI D state for a given root port.
+ *
+ * Returns:
+ * igt_acpi_d_state state.
+ */
+enum igt_acpi_d_state
+igt_pm_get_acpi_real_d_state(struct pci_device *pci_dev)
+{
+	char name[PATH_MAX];
+	char acpi_d_state[64];
+	int fd, n_read;
+
+	snprintf(name, PATH_MAX,
+		 "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/firmware_node/real_power_state",
+		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
+
+	fd = open(name, O_RDONLY);
+	if (fd < 0)
+		return IGT_ACPI_UNKNOWN_STATE;
+
+	n_read = read(fd, acpi_d_state, sizeof(acpi_d_state) - 1);
+	igt_assert(n_read >= 0);
+	acpi_d_state[n_read] = '\0';
+	close(fd);
+
+	if (strncmp(acpi_d_state, "D0\n", n_read) == 0)
+		return IGT_ACPI_D0;
+	if  (strncmp(acpi_d_state, "D1\n", n_read) == 0)
+		return IGT_ACPI_D1;
+	if  (strncmp(acpi_d_state, "D2\n", n_read) == 0)
+		return IGT_ACPI_D2;
+	if  (strncmp(acpi_d_state, "D3hot\n", n_read) == 0)
+		return IGT_ACPI_D3Hot;
+	if  (strncmp(acpi_d_state, "D3cold\n", n_read) == 0)
+		return IGT_ACPI_D3Cold;
+
+	return IGT_ACPI_UNKNOWN_STATE;
+}
+
+static struct igt_pm_pci_dev_pwrattr  *__pci_dev_pwrattr;
+static int __igt_pm_pci_dev_cnt;
+
+static void __igt_pm_pci_card_exit_handler(int sig)
+{
+	igt_pm_restore_pci_card_runtime_pm();
+}
+
+static int igt_pm_get_power_attr_fd(struct pci_device *pci_dev, const char *attr)
+{
+	char name[PATH_MAX];
+	int fd;
+
+	snprintf(name, PATH_MAX, "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/power/%s",
+		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func, attr);
+
+	fd = open(name, O_RDWR);
+	igt_assert_f(fd >= 0, "Can't open %s\n", attr);
+
+	return fd;
+}
+
+static bool igt_pm_save_power_attr(int fd, char *attr, int len, bool autosuspend)
+{
+	int size;
+
+	size = read(fd, attr, len - 1);
+
+	if (autosuspend) {
+		if (size < 0)
+			return false;
+	} else {
+		igt_assert(size > 0);
+	}
+
+	attr[size] = '\0';
+	strchomp(attr);
+
+	return true;
+}
+
+static void igt_pm_setup_power_attr(int fd, const char *val)
+{
+	int size, len;
+	char buf[6];
+
+	len = strlen(val);
+
+	size = write(fd, val, len);
+	igt_assert_eq(size, len);
+	lseek(fd, 0, SEEK_SET);
+	size = read(fd, buf, ARRAY_SIZE(buf));
+	igt_assert_eq(size, len);
+	igt_assert(strncmp(buf, val, len) == 0);
+}
+
+static void
+__igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev, int index)
+{
+	int control_fd, delay_fd, control_size, delay_size;
+	char *control, *delay;
+
+	igt_assert(index <  MAX_PCI_DEVICES);
+
+	if (!(index % 16)) {
+		__pci_dev_pwrattr =
+			realloc(__pci_dev_pwrattr,
+				sizeof(struct igt_pm_pci_dev_pwrattr) * (index / 16 + 1) * 16);
+	}
+
+	control = __pci_dev_pwrattr[index].control;
+	control_size =  sizeof(__pci_dev_pwrattr[index].control);
+	delay = __pci_dev_pwrattr[index].autosuspend_delay;
+	delay_size =  sizeof(__pci_dev_pwrattr[index].autosuspend_delay);
+	__pci_dev_pwrattr[index].pci_dev = pci_dev;
+	__pci_dev_pwrattr[index].autosuspend_supported = true;
+
+	delay_fd = igt_pm_get_power_attr_fd(pci_dev, "autosuspend_delay_ms");
+	control_fd = igt_pm_get_power_attr_fd(pci_dev, "control");
+
+	if (!igt_pm_save_power_attr(delay_fd, delay, delay_size, true)) {
+		__pci_dev_pwrattr[index].autosuspend_supported = false;
+		igt_debug("PCI '%04x:%02x:%02x.%01x' doesn't support auto_suspend\n",
+			  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
+	}
+
+	igt_pm_save_power_attr(control_fd, control, control_size, false);
+	igt_debug("PCI '%04x:%02x:%02x.%01x' Saved 'control, autosuspend_delay_ms' as '%s, %s'\n",
+		  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func, control,
+		  __pci_dev_pwrattr[index].autosuspend_supported ? delay : "NA");
+
+	if (!index)
+		igt_install_exit_handler(__igt_pm_pci_card_exit_handler);
+
+	if (__pci_dev_pwrattr[index].autosuspend_supported)
+		igt_pm_setup_power_attr(delay_fd, "0\n");
+
+	igt_pm_setup_power_attr(control_fd, "auto\n");
+
+	close(delay_fd);
+	close(control_fd);
+}
+
+/**
+ * igt_pm_setup_pci_card_runtime_pm:
+ * @pci_dev: root port pci_dev.
+ * Setup runtime PM for all PCI endpoints devices for a given root port by
+ * enabling runtime suspend and setting autosuspend_delay_ms to zero.
+ * It also saves power control attribute for all PCI endpoints
+ * devices under given root port.
+ */
+void igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev)
+{
+	int primary, secondary, subordinate, ret;
+	struct pci_device_iterator *iter;
+	struct pci_device *dev;
+	int i;
+
+	ret = pci_device_get_bridge_buses(pci_dev, &primary, &secondary, &subordinate);
+	igt_assert(!ret);
+
+	ret = pci_system_init();
+	igt_assert(!ret);
+
+	iter = pci_slot_match_iterator_create(NULL);
+	/* Setup runtime pm for PCI root port */
+	__igt_pm_setup_pci_card_runtime_pm(pci_dev, i++);
+	while ((dev = pci_device_next(iter)) != NULL) {
+		if (dev->bus >= secondary && dev->bus <= subordinate)
+			__igt_pm_setup_pci_card_runtime_pm(dev, i++);
+	}
+
+	__igt_pm_pci_dev_cnt = i;
+}
+
+static void
+igt_pm_restore_power_attr(struct pci_device *pci_dev, const char *attr, char *val, int len)
+{
+	int fd;
+
+	igt_debug("PCI '%04x:%02x:%02x.%01x' Restoring %s attr to '%s'\n",
+		  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func, attr, val);
+
+	fd = igt_pm_get_power_attr_fd(pci_dev, attr);
+	igt_assert(write(fd, val, len) == len);
+
+	close(fd);
+}
+
+/**
+ * igt_pm_restore_pci_card_runtime_pm:
+ * Restore control and autosuspend_delay_ms power attribute for all
+ * PCI endpoints devices under gfx root port, which were saved earlier
+ * by igt_pm_setup_pci_card_runtime_pm().
+ */
+void igt_pm_restore_pci_card_runtime_pm(void)
+{
+	int i = 0;
+
+	if (!__pci_dev_pwrattr)
+		return;
+
+	for (i = 0; i < __igt_pm_pci_dev_cnt; i++) {
+		igt_pm_restore_power_attr(__pci_dev_pwrattr[i].pci_dev, "control",
+					  __pci_dev_pwrattr[i].control,
+					  sizeof(__pci_dev_pwrattr[i].control));
+
+		if (!__pci_dev_pwrattr[i].autosuspend_supported)
+			continue;
+
+		igt_pm_restore_power_attr(__pci_dev_pwrattr[i].pci_dev, "autosuspend_delay_ms",
+					  __pci_dev_pwrattr[i].autosuspend_delay,
+					  sizeof(__pci_dev_pwrattr[i].autosuspend_delay));
+	}
+
+	pci_system_cleanup();
+	free(__pci_dev_pwrattr);
+	__pci_dev_pwrattr = NULL;
+}
+
+static void igt_pm_print_pci_dev_runtime_status(struct pci_device *pci_dev)
+{
+	char name[PATH_MAX], runtime_status[64];
+	int fd, n_read;
+
+	snprintf(name, PATH_MAX, "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/power/runtime_status",
+		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
+
+	fd = open(name, O_RDONLY);
+	igt_assert_f(fd >= 0, "Can't open runtime_status\n");
+
+	n_read = read(fd, runtime_status, sizeof(runtime_status) - 1);
+	igt_assert(n_read >= 0);
+	runtime_status[n_read] = '\0';
+	igt_info("runtime suspend status for PCI '%04x:%02x:%02x.%01x' %s\n",
+		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func, runtime_status);
+	close(fd);
+}
+
+/**
+ * igt_pm_print_pci_card_runtime_status:
+ * @pci_dev: root port pci_dev.
+ * Print runtime suspend status for all PCI endpoints devices for a given
+ * root port.
+ */
+void igt_pm_print_pci_card_runtime_status(void)
+{
+	int i = 0;
+
+	if (!__pci_dev_pwrattr)
+		return;
+
+	for (i = 0; i < __igt_pm_pci_dev_cnt; i++)
+		igt_pm_print_pci_dev_runtime_status(__pci_dev_pwrattr[i].pci_dev);
+}
diff --git a/lib/igt_pm.h b/lib/igt_pm.h
index 162d3ca3c..898178670 100644
--- a/lib/igt_pm.h
+++ b/lib/igt_pm.h
@@ -46,6 +46,23 @@ enum igt_runtime_pm_status {
 	IGT_RUNTIME_PM_STATUS_UNKNOWN,
 };
 
+/* PCI ACPI firmware node real state */
+enum igt_acpi_d_state {
+	IGT_ACPI_D0,
+	IGT_ACPI_D1,
+	IGT_ACPI_D2,
+	IGT_ACPI_D3Hot,
+	IGT_ACPI_D3Cold,
+	IGT_ACPI_UNKNOWN_STATE,
+};
+
+struct	igt_pm_pci_dev_pwrattr {
+	struct pci_device *pci_dev;
+	char control[64];
+	bool autosuspend_supported;
+	char autosuspend_delay[64];
+};
+
 bool igt_setup_runtime_pm(int device);
 void igt_disable_runtime_pm(void);
 void igt_restore_runtime_pm(void);
@@ -54,5 +71,11 @@ bool igt_wait_for_pm_status(enum igt_runtime_pm_status status);
 bool igt_pm_dmc_loaded(int debugfs);
 bool igt_pm_pc8_plus_residencies_enabled(int msr_fd);
 bool i915_output_is_lpsp_capable(int drm_fd, igt_output_t *output);
+bool igt_pm_acpi_d3cold_supported(struct pci_device *pci_dev);
+enum igt_acpi_d_state
+igt_pm_get_acpi_real_d_state(struct pci_device *pci_dev);
+void igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev);
+void igt_pm_restore_pci_card_runtime_pm(void);
+void igt_pm_print_pci_card_runtime_status(void);
 
 #endif /* IGT_PM_H */
-- 
2.26.2

Re: [igt-dev] [PATCH i-g-t v2 3/8] lib/igt_pm: D3Cold runtime pm infrastructure

[Rodrigo Vivi]

On Fri, Apr 29, 2022 at 02:24:20PM +0530, Anshuman Gupta wrote:
> Enable gfx card pci devices runtime pm for all pci devices
> and bridge under the topology of Gfx Card root port.
> 
> Added a library function to get the PCI root port ACPI
> D state and to print the pci card devices runtime pm
> status.
> 
> v2:
> - Save pci dev power attr to dynamically allocated array. [Rodrigo]
> - Set autosuspend delay to 0 on supported devices. [Rodrigo]
> - %s/else if/else in  igt_pm_get_acpi_real_d_state. [Kamil]
> 
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
> ---
>  lib/igt_pm.c | 290 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  lib/igt_pm.h |  23 ++++
>  2 files changed, 313 insertions(+)
> 
> diff --git a/lib/igt_pm.c b/lib/igt_pm.c
> index b409ec463..20296c40b 100644
> --- a/lib/igt_pm.c
> +++ b/lib/igt_pm.c
> @@ -28,6 +28,7 @@
>  #include <fcntl.h>
>  #include <stdio.h>
>  #include <limits.h>
> +#include <pciaccess.h>
>  #include <stdlib.h>
>  #include <string.h>
>  #include <unistd.h>
> @@ -75,6 +76,7 @@ enum {
>  #define MIN_POWER_STR		"min_power\n"
>  /* Remember to fix this if adding longer strings */
>  #define MAX_POLICY_STRLEN	strlen(MAX_PERFORMANCE_STR)
> +#define MAX_PCI_DEVICES		256
>  int8_t *__sata_pm_policies;
>  int __scsi_host_cnt;
>  
> @@ -856,3 +858,291 @@ bool i915_output_is_lpsp_capable(int drm_fd, igt_output_t *output)
>  
>  	return strstr(buf, "LPSP: capable");
>  }
> +
> +/**
> + * igt_pm_acpi_d3cold_supported:
> + * @pci_dev: root port pci_dev.
> + * Check ACPI D3Cold support.
> + *
> + * Returns:
> + * True if ACPI D3Cold supported otherwise false.
> + */
> +bool igt_pm_acpi_d3cold_supported(struct pci_device *pci_dev)
> +{
> +	char name[PATH_MAX];
> +	int dir, fd;
> +
> +	snprintf(name, PATH_MAX,
> +		 "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/firmware_node",
> +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> +
> +	dir = open(name, O_RDONLY);
> +	igt_require(dir > 0);
> +
> +	/* BIOS need to enable ACPI D3Cold Support.*/
> +	fd = openat(dir, "real_power_state", O_RDONLY);
> +	if (fd < 0 && errno == ENOENT)
> +		return false;
> +
> +	igt_require(fd > 0);
> +
> +	return true;
> +}
> +
> +/**
> + * igt_pm_get_acpi_real_d_state:
> + * @pci_dev: root port pci_dev.
> + * Get ACPI D state for a given root port.
> + *
> + * Returns:
> + * igt_acpi_d_state state.
> + */
> +enum igt_acpi_d_state
> +igt_pm_get_acpi_real_d_state(struct pci_device *pci_dev)
> +{
> +	char name[PATH_MAX];
> +	char acpi_d_state[64];
> +	int fd, n_read;
> +
> +	snprintf(name, PATH_MAX,
> +		 "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/firmware_node/real_power_state",
> +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> +
> +	fd = open(name, O_RDONLY);
> +	if (fd < 0)
> +		return IGT_ACPI_UNKNOWN_STATE;
> +
> +	n_read = read(fd, acpi_d_state, sizeof(acpi_d_state) - 1);
> +	igt_assert(n_read >= 0);
> +	acpi_d_state[n_read] = '\0';
> +	close(fd);
> +
> +	if (strncmp(acpi_d_state, "D0\n", n_read) == 0)
> +		return IGT_ACPI_D0;
> +	if  (strncmp(acpi_d_state, "D1\n", n_read) == 0)
> +		return IGT_ACPI_D1;
> +	if  (strncmp(acpi_d_state, "D2\n", n_read) == 0)
> +		return IGT_ACPI_D2;
> +	if  (strncmp(acpi_d_state, "D3hot\n", n_read) == 0)
> +		return IGT_ACPI_D3Hot;
> +	if  (strncmp(acpi_d_state, "D3cold\n", n_read) == 0)
> +		return IGT_ACPI_D3Cold;
> +
> +	return IGT_ACPI_UNKNOWN_STATE;
> +}
> +
> +static struct igt_pm_pci_dev_pwrattr  *__pci_dev_pwrattr;
> +static int __igt_pm_pci_dev_cnt;

looking at this now I believe I prefer the static global one than this
reallocated one.

I saw that we have other examples in igt lib that follows same patter so that
should be okay.

But honestly what I don't like is the global at all.
I see that you have the exit handler below that takes care of the unset,
but I believe as a "lib" this should be independent for symmetric reasons.

what I had in mind was something to be called at setup fixture and clean up
fixture. you save locally your pci config and then you clean that up on
restore...

But now seeing other global cases in the core of igt lib I don't care much
anymore...

But probably cleaner with the static now... and sorry about the back and forth
on this...

> +
> +static void __igt_pm_pci_card_exit_handler(int sig)
> +{
> +	igt_pm_restore_pci_card_runtime_pm();
> +}
> +
> +static int igt_pm_get_power_attr_fd(struct pci_device *pci_dev, const char *attr)
> +{
> +	char name[PATH_MAX];
> +	int fd;
> +
> +	snprintf(name, PATH_MAX, "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/power/%s",
> +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func, attr);
> +
> +	fd = open(name, O_RDWR);
> +	igt_assert_f(fd >= 0, "Can't open %s\n", attr);
> +
> +	return fd;
> +}
> +
> +static bool igt_pm_save_power_attr(int fd, char *attr, int len, bool autosuspend)
> +{
> +	int size;
> +
> +	size = read(fd, attr, len - 1);
> +
> +	if (autosuspend) {
> +		if (size < 0)
> +			return false;
> +	} else {
> +		igt_assert(size > 0);
> +	}
> +
> +	attr[size] = '\0';
> +	strchomp(attr);
> +
> +	return true;
> +}
> +
> +static void igt_pm_setup_power_attr(int fd, const char *val)
> +{
> +	int size, len;
> +	char buf[6];
> +
> +	len = strlen(val);
> +
> +	size = write(fd, val, len);
> +	igt_assert_eq(size, len);
> +	lseek(fd, 0, SEEK_SET);
> +	size = read(fd, buf, ARRAY_SIZE(buf));
> +	igt_assert_eq(size, len);
> +	igt_assert(strncmp(buf, val, len) == 0);
> +}
> +
> +static void
> +__igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev, int index)
> +{
> +	int control_fd, delay_fd, control_size, delay_size;
> +	char *control, *delay;
> +
> +	igt_assert(index <  MAX_PCI_DEVICES);
> +
> +	if (!(index % 16)) {
> +		__pci_dev_pwrattr =
> +			realloc(__pci_dev_pwrattr,
> +				sizeof(struct igt_pm_pci_dev_pwrattr) * (index / 16 + 1) * 16);
> +	}
> +
> +	control = __pci_dev_pwrattr[index].control;
> +	control_size =  sizeof(__pci_dev_pwrattr[index].control);
> +	delay = __pci_dev_pwrattr[index].autosuspend_delay;
> +	delay_size =  sizeof(__pci_dev_pwrattr[index].autosuspend_delay);
> +	__pci_dev_pwrattr[index].pci_dev = pci_dev;
> +	__pci_dev_pwrattr[index].autosuspend_supported = true;
> +
> +	delay_fd = igt_pm_get_power_attr_fd(pci_dev, "autosuspend_delay_ms");
> +	control_fd = igt_pm_get_power_attr_fd(pci_dev, "control");
> +
> +	if (!igt_pm_save_power_attr(delay_fd, delay, delay_size, true)) {
> +		__pci_dev_pwrattr[index].autosuspend_supported = false;
> +		igt_debug("PCI '%04x:%02x:%02x.%01x' doesn't support auto_suspend\n",
> +			  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> +	}
> +
> +	igt_pm_save_power_attr(control_fd, control, control_size, false);
> +	igt_debug("PCI '%04x:%02x:%02x.%01x' Saved 'control, autosuspend_delay_ms' as '%s, %s'\n",
> +		  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func, control,
> +		  __pci_dev_pwrattr[index].autosuspend_supported ? delay : "NA");
> +
> +	if (!index)
> +		igt_install_exit_handler(__igt_pm_pci_card_exit_handler);
> +
> +	if (__pci_dev_pwrattr[index].autosuspend_supported)
> +		igt_pm_setup_power_attr(delay_fd, "0\n");
> +
> +	igt_pm_setup_power_attr(control_fd, "auto\n");
> +
> +	close(delay_fd);
> +	close(control_fd);
> +}
> +
> +/**
> + * igt_pm_setup_pci_card_runtime_pm:
> + * @pci_dev: root port pci_dev.
> + * Setup runtime PM for all PCI endpoints devices for a given root port by
> + * enabling runtime suspend and setting autosuspend_delay_ms to zero.
> + * It also saves power control attribute for all PCI endpoints
> + * devices under given root port.
> + */
> +void igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev)
> +{
> +	int primary, secondary, subordinate, ret;
> +	struct pci_device_iterator *iter;
> +	struct pci_device *dev;
> +	int i;
> +
> +	ret = pci_device_get_bridge_buses(pci_dev, &primary, &secondary, &subordinate);
> +	igt_assert(!ret);
> +
> +	ret = pci_system_init();
> +	igt_assert(!ret);
> +
> +	iter = pci_slot_match_iterator_create(NULL);
> +	/* Setup runtime pm for PCI root port */
> +	__igt_pm_setup_pci_card_runtime_pm(pci_dev, i++);
> +	while ((dev = pci_device_next(iter)) != NULL) {
> +		if (dev->bus >= secondary && dev->bus <= subordinate)
> +			__igt_pm_setup_pci_card_runtime_pm(dev, i++);
> +	}
> +
> +	__igt_pm_pci_dev_cnt = i;
> +}
> +
> +static void
> +igt_pm_restore_power_attr(struct pci_device *pci_dev, const char *attr, char *val, int len)
> +{
> +	int fd;
> +
> +	igt_debug("PCI '%04x:%02x:%02x.%01x' Restoring %s attr to '%s'\n",
> +		  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func, attr, val);
> +
> +	fd = igt_pm_get_power_attr_fd(pci_dev, attr);
> +	igt_assert(write(fd, val, len) == len);
> +
> +	close(fd);
> +}
> +
> +/**
> + * igt_pm_restore_pci_card_runtime_pm:
> + * Restore control and autosuspend_delay_ms power attribute for all
> + * PCI endpoints devices under gfx root port, which were saved earlier
> + * by igt_pm_setup_pci_card_runtime_pm().
> + */
> +void igt_pm_restore_pci_card_runtime_pm(void)
> +{
> +	int i = 0;
> +
> +	if (!__pci_dev_pwrattr)
> +		return;
> +
> +	for (i = 0; i < __igt_pm_pci_dev_cnt; i++) {
> +		igt_pm_restore_power_attr(__pci_dev_pwrattr[i].pci_dev, "control",
> +					  __pci_dev_pwrattr[i].control,
> +					  sizeof(__pci_dev_pwrattr[i].control));
> +
> +		if (!__pci_dev_pwrattr[i].autosuspend_supported)
> +			continue;
> +
> +		igt_pm_restore_power_attr(__pci_dev_pwrattr[i].pci_dev, "autosuspend_delay_ms",
> +					  __pci_dev_pwrattr[i].autosuspend_delay,
> +					  sizeof(__pci_dev_pwrattr[i].autosuspend_delay));
> +	}
> +
> +	pci_system_cleanup();
> +	free(__pci_dev_pwrattr);
> +	__pci_dev_pwrattr = NULL;
> +}
> +
> +static void igt_pm_print_pci_dev_runtime_status(struct pci_device *pci_dev)
> +{
> +	char name[PATH_MAX], runtime_status[64];
> +	int fd, n_read;
> +
> +	snprintf(name, PATH_MAX, "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/power/runtime_status",
> +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> +
> +	fd = open(name, O_RDONLY);
> +	igt_assert_f(fd >= 0, "Can't open runtime_status\n");
> +
> +	n_read = read(fd, runtime_status, sizeof(runtime_status) - 1);
> +	igt_assert(n_read >= 0);
> +	runtime_status[n_read] = '\0';
> +	igt_info("runtime suspend status for PCI '%04x:%02x:%02x.%01x' %s\n",
> +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func, runtime_status);
> +	close(fd);
> +}
> +
> +/**
> + * igt_pm_print_pci_card_runtime_status:
> + * @pci_dev: root port pci_dev.
> + * Print runtime suspend status for all PCI endpoints devices for a given
> + * root port.
> + */
> +void igt_pm_print_pci_card_runtime_status(void)
> +{
> +	int i = 0;
> +
> +	if (!__pci_dev_pwrattr)
> +		return;
> +
> +	for (i = 0; i < __igt_pm_pci_dev_cnt; i++)
> +		igt_pm_print_pci_dev_runtime_status(__pci_dev_pwrattr[i].pci_dev);
> +}
> diff --git a/lib/igt_pm.h b/lib/igt_pm.h
> index 162d3ca3c..898178670 100644
> --- a/lib/igt_pm.h
> +++ b/lib/igt_pm.h
> @@ -46,6 +46,23 @@ enum igt_runtime_pm_status {
>  	IGT_RUNTIME_PM_STATUS_UNKNOWN,
>  };
>  
> +/* PCI ACPI firmware node real state */
> +enum igt_acpi_d_state {
> +	IGT_ACPI_D0,
> +	IGT_ACPI_D1,
> +	IGT_ACPI_D2,
> +	IGT_ACPI_D3Hot,
> +	IGT_ACPI_D3Cold,
> +	IGT_ACPI_UNKNOWN_STATE,
> +};
> +
> +struct	igt_pm_pci_dev_pwrattr {
> +	struct pci_device *pci_dev;
> +	char control[64];
> +	bool autosuspend_supported;
> +	char autosuspend_delay[64];
> +};
> +
>  bool igt_setup_runtime_pm(int device);
>  void igt_disable_runtime_pm(void);
>  void igt_restore_runtime_pm(void);
> @@ -54,5 +71,11 @@ bool igt_wait_for_pm_status(enum igt_runtime_pm_status status);
>  bool igt_pm_dmc_loaded(int debugfs);
>  bool igt_pm_pc8_plus_residencies_enabled(int msr_fd);
>  bool i915_output_is_lpsp_capable(int drm_fd, igt_output_t *output);
> +bool igt_pm_acpi_d3cold_supported(struct pci_device *pci_dev);
> +enum igt_acpi_d_state
> +igt_pm_get_acpi_real_d_state(struct pci_device *pci_dev);
> +void igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev);
> +void igt_pm_restore_pci_card_runtime_pm(void);
> +void igt_pm_print_pci_card_runtime_status(void);
>  
>  #endif /* IGT_PM_H */
> -- 
> 2.26.2
> 

Re: [igt-dev] [PATCH i-g-t v2 3/8] lib/igt_pm: D3Cold runtime pm infrastructure

[Gupta, Anshuman]

> -----Original Message-----
> From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> Sent: Friday, April 29, 2022 8:18 PM
> To: Gupta, Anshuman <anshuman.gupta@intel.com>
> Cc: igt-dev@lists.freedesktop.org; kamil.konieczny@linux.intel.com; Latvala,
> Petri <petri.latvala@intel.com>; Nilawar, Badal <badal.nilawar@intel.com>;
> Ewins, Jon <jon.ewins@intel.com>
> Subject: Re: [PATCH i-g-t v2 3/8] lib/igt_pm: D3Cold runtime pm infrastructure
> 
> On Fri, Apr 29, 2022 at 02:24:20PM +0530, Anshuman Gupta wrote:
> > Enable gfx card pci devices runtime pm for all pci devices and bridge
> > under the topology of Gfx Card root port.
> >
> > Added a library function to get the PCI root port ACPI D state and to
> > print the pci card devices runtime pm status.
> >
> > v2:
> > - Save pci dev power attr to dynamically allocated array. [Rodrigo]
> > - Set autosuspend delay to 0 on supported devices. [Rodrigo]
> > - %s/else if/else in  igt_pm_get_acpi_real_d_state. [Kamil]
> >
> > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
> > ---
> >  lib/igt_pm.c | 290
> > +++++++++++++++++++++++++++++++++++++++++++++++++++
> >  lib/igt_pm.h |  23 ++++
> >  2 files changed, 313 insertions(+)
> >
> > diff --git a/lib/igt_pm.c b/lib/igt_pm.c index b409ec463..20296c40b
> > 100644
> > --- a/lib/igt_pm.c
> > +++ b/lib/igt_pm.c
> > @@ -28,6 +28,7 @@
> >  #include <fcntl.h>
> >  #include <stdio.h>
> >  #include <limits.h>
> > +#include <pciaccess.h>
> >  #include <stdlib.h>
> >  #include <string.h>
> >  #include <unistd.h>
> > @@ -75,6 +76,7 @@ enum {
> >  #define MIN_POWER_STR		"min_power\n"
> >  /* Remember to fix this if adding longer strings */
> >  #define MAX_POLICY_STRLEN	strlen(MAX_PERFORMANCE_STR)
> > +#define MAX_PCI_DEVICES		256
> >  int8_t *__sata_pm_policies;
> >  int __scsi_host_cnt;
> >
> > @@ -856,3 +858,291 @@ bool i915_output_is_lpsp_capable(int drm_fd,
> > igt_output_t *output)
> >
> >  	return strstr(buf, "LPSP: capable");  }
> > +
> > +/**
> > + * igt_pm_acpi_d3cold_supported:
> > + * @pci_dev: root port pci_dev.
> > + * Check ACPI D3Cold support.
> > + *
> > + * Returns:
> > + * True if ACPI D3Cold supported otherwise false.
> > + */
> > +bool igt_pm_acpi_d3cold_supported(struct pci_device *pci_dev) {
> > +	char name[PATH_MAX];
> > +	int dir, fd;
> > +
> > +	snprintf(name, PATH_MAX,
> > +
> "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/firmware_node",
> > +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> > +
> > +	dir = open(name, O_RDONLY);
> > +	igt_require(dir > 0);
> > +
> > +	/* BIOS need to enable ACPI D3Cold Support.*/
> > +	fd = openat(dir, "real_power_state", O_RDONLY);
> > +	if (fd < 0 && errno == ENOENT)
> > +		return false;
> > +
> > +	igt_require(fd > 0);
> > +
> > +	return true;
> > +}
> > +
> > +/**
> > + * igt_pm_get_acpi_real_d_state:
> > + * @pci_dev: root port pci_dev.
> > + * Get ACPI D state for a given root port.
> > + *
> > + * Returns:
> > + * igt_acpi_d_state state.
> > + */
> > +enum igt_acpi_d_state
> > +igt_pm_get_acpi_real_d_state(struct pci_device *pci_dev) {
> > +	char name[PATH_MAX];
> > +	char acpi_d_state[64];
> > +	int fd, n_read;
> > +
> > +	snprintf(name, PATH_MAX,
> > +
> "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/firmware_node/real_power_stat
> e",
> > +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> > +
> > +	fd = open(name, O_RDONLY);
> > +	if (fd < 0)
> > +		return IGT_ACPI_UNKNOWN_STATE;
> > +
> > +	n_read = read(fd, acpi_d_state, sizeof(acpi_d_state) - 1);
> > +	igt_assert(n_read >= 0);
> > +	acpi_d_state[n_read] = '\0';
> > +	close(fd);
> > +
> > +	if (strncmp(acpi_d_state, "D0\n", n_read) == 0)
> > +		return IGT_ACPI_D0;
> > +	if  (strncmp(acpi_d_state, "D1\n", n_read) == 0)
> > +		return IGT_ACPI_D1;
> > +	if  (strncmp(acpi_d_state, "D2\n", n_read) == 0)
> > +		return IGT_ACPI_D2;
> > +	if  (strncmp(acpi_d_state, "D3hot\n", n_read) == 0)
> > +		return IGT_ACPI_D3Hot;
> > +	if  (strncmp(acpi_d_state, "D3cold\n", n_read) == 0)
> > +		return IGT_ACPI_D3Cold;
> > +
> > +	return IGT_ACPI_UNKNOWN_STATE;
> > +}
> > +
> > +static struct igt_pm_pci_dev_pwrattr  *__pci_dev_pwrattr; static int
> > +__igt_pm_pci_dev_cnt;
> 
> looking at this now I believe I prefer the static global one than this reallocated
> one.
> 
> I saw that we have other examples in igt lib that follows same patter so that
> should be okay.
Reallocated buffer approach also referred from __igt_pm_enable_sata_link_power_management()
Igt_pm.c.  IMHO this will be good as compare to global buffer, unless i missed any issues around it ? 
> 
> But honestly what I don't like is the global at all.
> I see that you have the exit handler below that takes care of the unset, but I
> believe as a "lib" this should be independent for symmetric reasons.
> 
> what I had in mind was something to be called at setup fixture and clean up
> fixture. you save locally your pci config and then you clean that up on restore...
AFAIK igt_fixture() approach to restore the power attribute state may not work in certain cases when task is
getting killed due to some raised signal. That is where igt exit handler have importance.
If we would have a void *data argument to pass in igt exit handler , then  we could get rid of global.
Thanks,
Anshuman Gupta.
> 
> But now seeing other global cases in the core of igt lib I don't care much
> anymore...
> 
> But probably cleaner with the static now... and sorry about the back and forth
> on this...
> 
> > +
> > +static void __igt_pm_pci_card_exit_handler(int sig) {
> > +	igt_pm_restore_pci_card_runtime_pm();
> > +}
> > +
> > +static int igt_pm_get_power_attr_fd(struct pci_device *pci_dev, const
> > +char *attr) {
> > +	char name[PATH_MAX];
> > +	int fd;
> > +
> > +	snprintf(name, PATH_MAX,
> "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/power/%s",
> > +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func,
> attr);
> > +
> > +	fd = open(name, O_RDWR);
> > +	igt_assert_f(fd >= 0, "Can't open %s\n", attr);
> > +
> > +	return fd;
> > +}
> > +
> > +static bool igt_pm_save_power_attr(int fd, char *attr, int len, bool
> > +autosuspend) {
> > +	int size;
> > +
> > +	size = read(fd, attr, len - 1);
> > +
> > +	if (autosuspend) {
> > +		if (size < 0)
> > +			return false;
> > +	} else {
> > +		igt_assert(size > 0);
> > +	}
> > +
> > +	attr[size] = '\0';
> > +	strchomp(attr);
> > +
> > +	return true;
> > +}
> > +
> > +static void igt_pm_setup_power_attr(int fd, const char *val) {
> > +	int size, len;
> > +	char buf[6];
> > +
> > +	len = strlen(val);
> > +
> > +	size = write(fd, val, len);
> > +	igt_assert_eq(size, len);
> > +	lseek(fd, 0, SEEK_SET);
> > +	size = read(fd, buf, ARRAY_SIZE(buf));
> > +	igt_assert_eq(size, len);
> > +	igt_assert(strncmp(buf, val, len) == 0); }
> > +
> > +static void
> > +__igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev, int
> > +index) {
> > +	int control_fd, delay_fd, control_size, delay_size;
> > +	char *control, *delay;
> > +
> > +	igt_assert(index <  MAX_PCI_DEVICES);
> > +
> > +	if (!(index % 16)) {
> > +		__pci_dev_pwrattr =
> > +			realloc(__pci_dev_pwrattr,
> > +				sizeof(struct igt_pm_pci_dev_pwrattr) * (index
> / 16 + 1) * 16);
> > +	}
> > +
> > +	control = __pci_dev_pwrattr[index].control;
> > +	control_size =  sizeof(__pci_dev_pwrattr[index].control);
> > +	delay = __pci_dev_pwrattr[index].autosuspend_delay;
> > +	delay_size =  sizeof(__pci_dev_pwrattr[index].autosuspend_delay);
> > +	__pci_dev_pwrattr[index].pci_dev = pci_dev;
> > +	__pci_dev_pwrattr[index].autosuspend_supported = true;
> > +
> > +	delay_fd = igt_pm_get_power_attr_fd(pci_dev,
> "autosuspend_delay_ms");
> > +	control_fd = igt_pm_get_power_attr_fd(pci_dev, "control");
> > +
> > +	if (!igt_pm_save_power_attr(delay_fd, delay, delay_size, true)) {
> > +		__pci_dev_pwrattr[index].autosuspend_supported = false;
> > +		igt_debug("PCI '%04x:%02x:%02x.%01x' doesn't support
> auto_suspend\n",
> > +			  pci_dev->domain, pci_dev->bus, pci_dev->dev,
> pci_dev->func);
> > +	}
> > +
> > +	igt_pm_save_power_attr(control_fd, control, control_size, false);
> > +	igt_debug("PCI '%04x:%02x:%02x.%01x' Saved 'control,
> autosuspend_delay_ms' as '%s, %s'\n",
> > +		  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func,
> control,
> > +		  __pci_dev_pwrattr[index].autosuspend_supported ? delay :
> "NA");
> > +
> > +	if (!index)
> > +		igt_install_exit_handler(__igt_pm_pci_card_exit_handler);
> > +
> > +	if (__pci_dev_pwrattr[index].autosuspend_supported)
> > +		igt_pm_setup_power_attr(delay_fd, "0\n");
> > +
> > +	igt_pm_setup_power_attr(control_fd, "auto\n");
> > +
> > +	close(delay_fd);
> > +	close(control_fd);
> > +}
> > +
> > +/**
> > + * igt_pm_setup_pci_card_runtime_pm:
> > + * @pci_dev: root port pci_dev.
> > + * Setup runtime PM for all PCI endpoints devices for a given root
> > +port by
> > + * enabling runtime suspend and setting autosuspend_delay_ms to zero.
> > + * It also saves power control attribute for all PCI endpoints
> > + * devices under given root port.
> > + */
> > +void igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev) {
> > +	int primary, secondary, subordinate, ret;
> > +	struct pci_device_iterator *iter;
> > +	struct pci_device *dev;
> > +	int i;
> > +
> > +	ret = pci_device_get_bridge_buses(pci_dev, &primary, &secondary,
> &subordinate);
> > +	igt_assert(!ret);
> > +
> > +	ret = pci_system_init();
> > +	igt_assert(!ret);
> > +
> > +	iter = pci_slot_match_iterator_create(NULL);
> > +	/* Setup runtime pm for PCI root port */
> > +	__igt_pm_setup_pci_card_runtime_pm(pci_dev, i++);
> > +	while ((dev = pci_device_next(iter)) != NULL) {
> > +		if (dev->bus >= secondary && dev->bus <= subordinate)
> > +			__igt_pm_setup_pci_card_runtime_pm(dev, i++);
> > +	}
> > +
> > +	__igt_pm_pci_dev_cnt = i;
> > +}
> > +
> > +static void
> > +igt_pm_restore_power_attr(struct pci_device *pci_dev, const char
> > +*attr, char *val, int len) {
> > +	int fd;
> > +
> > +	igt_debug("PCI '%04x:%02x:%02x.%01x' Restoring %s attr to '%s'\n",
> > +		  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func,
> attr,
> > +val);
> > +
> > +	fd = igt_pm_get_power_attr_fd(pci_dev, attr);
> > +	igt_assert(write(fd, val, len) == len);
> > +
> > +	close(fd);
> > +}
> > +
> > +/**
> > + * igt_pm_restore_pci_card_runtime_pm:
> > + * Restore control and autosuspend_delay_ms power attribute for all
> > + * PCI endpoints devices under gfx root port, which were saved
> > +earlier
> > + * by igt_pm_setup_pci_card_runtime_pm().
> > + */
> > +void igt_pm_restore_pci_card_runtime_pm(void)
> > +{
> > +	int i = 0;
> > +
> > +	if (!__pci_dev_pwrattr)
> > +		return;
> > +
> > +	for (i = 0; i < __igt_pm_pci_dev_cnt; i++) {
> > +		igt_pm_restore_power_attr(__pci_dev_pwrattr[i].pci_dev,
> "control",
> > +					  __pci_dev_pwrattr[i].control,
> > +					  sizeof(__pci_dev_pwrattr[i].control));
> > +
> > +		if (!__pci_dev_pwrattr[i].autosuspend_supported)
> > +			continue;
> > +
> > +		igt_pm_restore_power_attr(__pci_dev_pwrattr[i].pci_dev,
> "autosuspend_delay_ms",
> > +
> __pci_dev_pwrattr[i].autosuspend_delay,
> > +
> sizeof(__pci_dev_pwrattr[i].autosuspend_delay));
> > +	}
> > +
> > +	pci_system_cleanup();
> > +	free(__pci_dev_pwrattr);
> > +	__pci_dev_pwrattr = NULL;
> > +}
> > +
> > +static void igt_pm_print_pci_dev_runtime_status(struct pci_device
> > +*pci_dev) {
> > +	char name[PATH_MAX], runtime_status[64];
> > +	int fd, n_read;
> > +
> > +	snprintf(name, PATH_MAX,
> "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/power/runtime_status",
> > +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> > +
> > +	fd = open(name, O_RDONLY);
> > +	igt_assert_f(fd >= 0, "Can't open runtime_status\n");
> > +
> > +	n_read = read(fd, runtime_status, sizeof(runtime_status) - 1);
> > +	igt_assert(n_read >= 0);
> > +	runtime_status[n_read] = '\0';
> > +	igt_info("runtime suspend status for PCI '%04x:%02x:%02x.%01x' %s\n",
> > +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func,
> runtime_status);
> > +	close(fd);
> > +}
> > +
> > +/**
> > + * igt_pm_print_pci_card_runtime_status:
> > + * @pci_dev: root port pci_dev.
> > + * Print runtime suspend status for all PCI endpoints devices for a
> > +given
> > + * root port.
> > + */
> > +void igt_pm_print_pci_card_runtime_status(void)
> > +{
> > +	int i = 0;
> > +
> > +	if (!__pci_dev_pwrattr)
> > +		return;
> > +
> > +	for (i = 0; i < __igt_pm_pci_dev_cnt; i++)
> > +
> 	igt_pm_print_pci_dev_runtime_status(__pci_dev_pwrattr[i].pci_dev);
> > +}
> > diff --git a/lib/igt_pm.h b/lib/igt_pm.h index 162d3ca3c..898178670
> > 100644
> > --- a/lib/igt_pm.h
> > +++ b/lib/igt_pm.h
> > @@ -46,6 +46,23 @@ enum igt_runtime_pm_status {
> >  	IGT_RUNTIME_PM_STATUS_UNKNOWN,
> >  };
> >
> > +/* PCI ACPI firmware node real state */ enum igt_acpi_d_state {
> > +	IGT_ACPI_D0,
> > +	IGT_ACPI_D1,
> > +	IGT_ACPI_D2,
> > +	IGT_ACPI_D3Hot,
> > +	IGT_ACPI_D3Cold,
> > +	IGT_ACPI_UNKNOWN_STATE,
> > +};
> > +
> > +struct	igt_pm_pci_dev_pwrattr {
> > +	struct pci_device *pci_dev;
> > +	char control[64];
> > +	bool autosuspend_supported;
> > +	char autosuspend_delay[64];
> > +};
> > +
> >  bool igt_setup_runtime_pm(int device);  void
> > igt_disable_runtime_pm(void);  void igt_restore_runtime_pm(void); @@
> > -54,5 +71,11 @@ bool igt_wait_for_pm_status(enum igt_runtime_pm_status
> > status);  bool igt_pm_dmc_loaded(int debugfs);  bool
> > igt_pm_pc8_plus_residencies_enabled(int msr_fd);  bool
> > i915_output_is_lpsp_capable(int drm_fd, igt_output_t *output);
> > +bool igt_pm_acpi_d3cold_supported(struct pci_device *pci_dev); enum
> > +igt_acpi_d_state igt_pm_get_acpi_real_d_state(struct pci_device
> > +*pci_dev); void igt_pm_setup_pci_card_runtime_pm(struct pci_device
> > +*pci_dev); void igt_pm_restore_pci_card_runtime_pm(void);
> > +void igt_pm_print_pci_card_runtime_status(void);
> >
> >  #endif /* IGT_PM_H */
> > --
> > 2.26.2
> >

Re: [igt-dev] [PATCH i-g-t v2 3/8] lib/igt_pm: D3Cold runtime pm infrastructure

[Nilawar, Badal]

On 29-04-2022 14:24, Anshuman Gupta wrote:
> Enable gfx card pci devices runtime pm for all pci devices
> and bridge under the topology of Gfx Card root port.
> 
> Added a library function to get the PCI root port ACPI
> D state and to print the pci card devices runtime pm
> status.
> 
> v2:
> - Save pci dev power attr to dynamically allocated array. [Rodrigo]
> - Set autosuspend delay to 0 on supported devices. [Rodrigo]
> - %s/else if/else in  igt_pm_get_acpi_real_d_state. [Kamil]
> 
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
> ---
>   lib/igt_pm.c | 290 +++++++++++++++++++++++++++++++++++++++++++++++++++
>   lib/igt_pm.h |  23 ++++
>   2 files changed, 313 insertions(+)
> 
> diff --git a/lib/igt_pm.c b/lib/igt_pm.c
> index b409ec463..20296c40b 100644
> --- a/lib/igt_pm.c
> +++ b/lib/igt_pm.c
> @@ -28,6 +28,7 @@
>   #include <fcntl.h>
>   #include <stdio.h>
>   #include <limits.h>
> +#include <pciaccess.h>
>   #include <stdlib.h>
>   #include <string.h>
>   #include <unistd.h>
> @@ -75,6 +76,7 @@ enum {
>   #define MIN_POWER_STR		"min_power\n"
>   /* Remember to fix this if adding longer strings */
>   #define MAX_POLICY_STRLEN	strlen(MAX_PERFORMANCE_STR)
> +#define MAX_PCI_DEVICES		256
>   int8_t *__sata_pm_policies;
>   int __scsi_host_cnt;
>   
> @@ -856,3 +858,291 @@ bool i915_output_is_lpsp_capable(int drm_fd, igt_output_t *output)
>   
>   	return strstr(buf, "LPSP: capable");
>   }
> +
> +/**
> + * igt_pm_acpi_d3cold_supported:
> + * @pci_dev: root port pci_dev.
> + * Check ACPI D3Cold support.
> + *
> + * Returns:
> + * True if ACPI D3Cold supported otherwise false.
> + */
> +bool igt_pm_acpi_d3cold_supported(struct pci_device *pci_dev)
> +{
> +	char name[PATH_MAX];
> +	int dir, fd;
> +
> +	snprintf(name, PATH_MAX,
> +		 "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/firmware_node",
> +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> +
> +	dir = open(name, O_RDONLY);
> +	igt_require(dir > 0);
> +
> +	/* BIOS need to enable ACPI D3Cold Support.*/
> +	fd = openat(dir, "real_power_state", O_RDONLY);
> +	if (fd < 0 && errno == ENOENT)
> +		return false;
> +
> +	igt_require(fd > 0);
> +
> +	return true;
> +}
> +
> +/**
> + * igt_pm_get_acpi_real_d_state:
> + * @pci_dev: root port pci_dev.
> + * Get ACPI D state for a given root port.
> + *
> + * Returns:
> + * igt_acpi_d_state state.
> + */
> +enum igt_acpi_d_state
> +igt_pm_get_acpi_real_d_state(struct pci_device *pci_dev)
> +{
> +	char name[PATH_MAX];
> +	char acpi_d_state[64];
> +	int fd, n_read;
> +
> +	snprintf(name, PATH_MAX,
> +		 "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/firmware_node/real_power_state",
> +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> +
> +	fd = open(name, O_RDONLY);
> +	if (fd < 0)
> +		return IGT_ACPI_UNKNOWN_STATE;
> +
> +	n_read = read(fd, acpi_d_state, sizeof(acpi_d_state) - 1);
> +	igt_assert(n_read >= 0);
> +	acpi_d_state[n_read] = '\0';
> +	close(fd);
> +
> +	if (strncmp(acpi_d_state, "D0\n", n_read) == 0)
> +		return IGT_ACPI_D0;
> +	if  (strncmp(acpi_d_state, "D1\n", n_read) == 0)
> +		return IGT_ACPI_D1;
> +	if  (strncmp(acpi_d_state, "D2\n", n_read) == 0)
> +		return IGT_ACPI_D2;
> +	if  (strncmp(acpi_d_state, "D3hot\n", n_read) == 0)
> +		return IGT_ACPI_D3Hot;
> +	if  (strncmp(acpi_d_state, "D3cold\n", n_read) == 0)
> +		return IGT_ACPI_D3Cold;
> +
> +	return IGT_ACPI_UNKNOWN_STATE;
> +}
> +
> +static struct igt_pm_pci_dev_pwrattr  *__pci_dev_pwrattr;
> +static int __igt_pm_pci_dev_cnt;
> +
> +static void __igt_pm_pci_card_exit_handler(int sig)
> +{
> +	igt_pm_restore_pci_card_runtime_pm();
> +}
> +
> +static int igt_pm_get_power_attr_fd(struct pci_device *pci_dev, const char *attr)
> +{
> +	char name[PATH_MAX];
> +	int fd;
> +
> +	snprintf(name, PATH_MAX, "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/power/%s",
> +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func, attr);
> +
> +	fd = open(name, O_RDWR);
> +	igt_assert_f(fd >= 0, "Can't open %s\n", attr);
> +
> +	return fd;
> +}
> +
> +static bool igt_pm_save_power_attr(int fd, char *attr, int len, bool autosuspend)
> +{
> +	int size;
> +
> +	size = read(fd, attr, len - 1);
> +
> +	if (autosuspend) {
> +		if (size < 0)
> +			return false;
> +	} else {
> +		igt_assert(size > 0);
> +	}
> +
> +	attr[size] = '\0';
> +	strchomp(attr);
> +
> +	return true;
> +}
> +
> +static void igt_pm_setup_power_attr(int fd, const char *val)
> +{
> +	int size, len;
> +	char buf[6];
> +
> +	len = strlen(val);
> +
> +	size = write(fd, val, len);
> +	igt_assert_eq(size, len);
> +	lseek(fd, 0, SEEK_SET);
> +	size = read(fd, buf, ARRAY_SIZE(buf));
> +	igt_assert_eq(size, len);
> +	igt_assert(strncmp(buf, val, len) == 0);
> +}
> +
> +static void
> +__igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev, int index)
> +{
> +	int control_fd, delay_fd, control_size, delay_size;
> +	char *control, *delay;
> +
> +	igt_assert(index <  MAX_PCI_DEVICES) > +
> +	if (!(index % 16)) {
> +		__pci_dev_pwrattr =
> +			realloc(__pci_dev_pwrattr,
> +				sizeof(struct igt_pm_pci_dev_pwrattr) * (index / 16 + 1) * 16);
> +	}
> +
> +	control = __pci_dev_pwrattr[index].control;
> +	control_size =  sizeof(__pci_dev_pwrattr[index].control);
> +	delay = __pci_dev_pwrattr[index].autosuspend_delay;
> +	delay_size =  sizeof(__pci_dev_pwrattr[index].autosuspend_delay);
> +	__pci_dev_pwrattr[index].pci_dev = pci_dev;
> +	__pci_dev_pwrattr[index].autosuspend_supported = true;
> +
> +	delay_fd = igt_pm_get_power_attr_fd(pci_dev, "autosuspend_delay_ms");
> +	control_fd = igt_pm_get_power_attr_fd(pci_dev, "control");
> +
> +	if (!igt_pm_save_power_attr(delay_fd, delay, delay_size, true)) {
> +		__pci_dev_pwrattr[index].autosuspend_supported = false;
> +		igt_debug("PCI '%04x:%02x:%02x.%01x' doesn't support auto_suspend\n",
> +			  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> +	}
> +
> +	igt_pm_save_power_attr(control_fd, control, control_size, false);
> +	igt_debug("PCI '%04x:%02x:%02x.%01x' Saved 'control, autosuspend_delay_ms' as '%s, %s'\n",
> +		  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func, control,
> +		  __pci_dev_pwrattr[index].autosuspend_supported ? delay : "NA");
> +
> +	if (!index)
> +		igt_install_exit_handler(__igt_pm_pci_card_exit_handler);
> +
> +	if (__pci_dev_pwrattr[index].autosuspend_supported)
> +		igt_pm_setup_power_attr(delay_fd, "0\n");
> +
> +	igt_pm_setup_power_attr(control_fd, "auto\n");
> +
> +	close(delay_fd);
> +	close(control_fd);
> +}
> +
> +/**
> + * igt_pm_setup_pci_card_runtime_pm:
> + * @pci_dev: root port pci_dev.
> + * Setup runtime PM for all PCI endpoints devices for a given root port by
> + * enabling runtime suspend and setting autosuspend_delay_ms to zero.
> + * It also saves power control attribute for all PCI endpoints
> + * devices under given root port.
> + */
> +void igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev)
> +{
> +	int primary, secondary, subordinate, ret;
> +	struct pci_device_iterator *iter;
> +	struct pci_device *dev;
> +	int i;
i should be initialized to 0;
> +
> +	ret = pci_device_get_bridge_buses(pci_dev, &primary, &secondary, &subordinate);
> +	igt_assert(!ret);
> +
> +	ret = pci_system_init();
> +	igt_assert(!ret);
> +
> +	iter = pci_slot_match_iterator_create(NULL);
> +	/* Setup runtime pm for PCI root port */
> +	__igt_pm_setup_pci_card_runtime_pm(pci_dev, i++);
> +	while ((dev = pci_device_next(iter)) != NULL) { > +		if (dev->bus >= secondary && dev->bus <= subordinate)
> +			__igt_pm_setup_pci_card_runtime_pm(dev, i++);
> +	}Searching of pci_device would be little faster if we use itr->mode = 
match_slot and pass iter->match.slot.bus to pci_device_next()

Thanks,
Badal Nilawar
> +
> +	__igt_pm_pci_dev_cnt = i;
> +}
> +
> +static void
> +igt_pm_restore_power_attr(struct pci_device *pci_dev, const char *attr, char *val, int len)
> +{
> +	int fd;
> +
> +	igt_debug("PCI '%04x:%02x:%02x.%01x' Restoring %s attr to '%s'\n",
> +		  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func, attr, val);
> +
> +	fd = igt_pm_get_power_attr_fd(pci_dev, attr);
> +	igt_assert(write(fd, val, len) == len);
> +
> +	close(fd);
> +}
> +
> +/**
> + * igt_pm_restore_pci_card_runtime_pm:
> + * Restore control and autosuspend_delay_ms power attribute for all
> + * PCI endpoints devices under gfx root port, which were saved earlier
> + * by igt_pm_setup_pci_card_runtime_pm().
> + */
> +void igt_pm_restore_pci_card_runtime_pm(void)
> +{
> +	int i = 0;
> +
> +	if (!__pci_dev_pwrattr)
> +		return;
> +
> +	for (i = 0; i < __igt_pm_pci_dev_cnt; i++) {
> +		igt_pm_restore_power_attr(__pci_dev_pwrattr[i].pci_dev, "control",
> +					  __pci_dev_pwrattr[i].control,
> +					  sizeof(__pci_dev_pwrattr[i].control));
> +
> +		if (!__pci_dev_pwrattr[i].autosuspend_supported)
> +			continue;
> +
> +		igt_pm_restore_power_attr(__pci_dev_pwrattr[i].pci_dev, "autosuspend_delay_ms",
> +					  __pci_dev_pwrattr[i].autosuspend_delay,
> +					  sizeof(__pci_dev_pwrattr[i].autosuspend_delay));
> +	}
> +
> +	pci_system_cleanup();
> +	free(__pci_dev_pwrattr);
> +	__pci_dev_pwrattr = NULL;
> +}
> +
> +static void igt_pm_print_pci_dev_runtime_status(struct pci_device *pci_dev)
> +{
> +	char name[PATH_MAX], runtime_status[64];
> +	int fd, n_read;
> +
> +	snprintf(name, PATH_MAX, "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/power/runtime_status",
> +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> +
> +	fd = open(name, O_RDONLY);
> +	igt_assert_f(fd >= 0, "Can't open runtime_status\n");
> +
> +	n_read = read(fd, runtime_status, sizeof(runtime_status) - 1);
> +	igt_assert(n_read >= 0);
> +	runtime_status[n_read] = '\0';
> +	igt_info("runtime suspend status for PCI '%04x:%02x:%02x.%01x' %s\n",
> +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func, runtime_status);
> +	close(fd);
> +}
> +
> +/**
> + * igt_pm_print_pci_card_runtime_status:
> + * @pci_dev: root port pci_dev.
> + * Print runtime suspend status for all PCI endpoints devices for a given
> + * root port.
> + */
> +void igt_pm_print_pci_card_runtime_status(void)
> +{
> +	int i = 0;
> +
> +	if (!__pci_dev_pwrattr)
> +		return;
> +
> +	for (i = 0; i < __igt_pm_pci_dev_cnt; i++)
> +		igt_pm_print_pci_dev_runtime_status(__pci_dev_pwrattr[i].pci_dev);
> +}
> diff --git a/lib/igt_pm.h b/lib/igt_pm.h
> index 162d3ca3c..898178670 100644
> --- a/lib/igt_pm.h
> +++ b/lib/igt_pm.h
> @@ -46,6 +46,23 @@ enum igt_runtime_pm_status {
>   	IGT_RUNTIME_PM_STATUS_UNKNOWN,
>   };
>   
> +/* PCI ACPI firmware node real state */
> +enum igt_acpi_d_state {
> +	IGT_ACPI_D0,
> +	IGT_ACPI_D1,
> +	IGT_ACPI_D2,
> +	IGT_ACPI_D3Hot,
> +	IGT_ACPI_D3Cold,
> +	IGT_ACPI_UNKNOWN_STATE,
> +};
> +
> +struct	igt_pm_pci_dev_pwrattr {
> +	struct pci_device *pci_dev;
> +	char control[64];
> +	bool autosuspend_supported;
> +	char autosuspend_delay[64];
> +};
> +
>   bool igt_setup_runtime_pm(int device);
>   void igt_disable_runtime_pm(void);
>   void igt_restore_runtime_pm(void);
> @@ -54,5 +71,11 @@ bool igt_wait_for_pm_status(enum igt_runtime_pm_status status);
>   bool igt_pm_dmc_loaded(int debugfs);
>   bool igt_pm_pc8_plus_residencies_enabled(int msr_fd);
>   bool i915_output_is_lpsp_capable(int drm_fd, igt_output_t *output);
> +bool igt_pm_acpi_d3cold_supported(struct pci_device *pci_dev);
> +enum igt_acpi_d_state
> +igt_pm_get_acpi_real_d_state(struct pci_device *pci_dev);
> +void igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev);
> +void igt_pm_restore_pci_card_runtime_pm(void);
> +void igt_pm_print_pci_card_runtime_status(void);
>   
>   #endif /* IGT_PM_H */

Re: [igt-dev] [PATCH i-g-t v2 3/8] lib/igt_pm: D3Cold runtime pm infrastructure

[Gupta, Anshuman]

> -----Original Message-----
> From: Nilawar, Badal <badal.nilawar@intel.com>
> Sent: Saturday, April 30, 2022 12:43 AM
> To: Gupta, Anshuman <anshuman.gupta@intel.com>; igt-
> dev@lists.freedesktop.org
> Cc: kamil.konieczny@linux.intel.com; Vivi, Rodrigo <rodrigo.vivi@intel.com>;
> Latvala, Petri <petri.latvala@intel.com>; Ewins, Jon <jon.ewins@intel.com>
> Subject: Re: [PATCH i-g-t v2 3/8] lib/igt_pm: D3Cold runtime pm infrastructure
> 
> 
> 
> On 29-04-2022 14:24, Anshuman Gupta wrote:
> > Enable gfx card pci devices runtime pm for all pci devices and bridge
> > under the topology of Gfx Card root port.
> >
> > Added a library function to get the PCI root port ACPI D state and to
> > print the pci card devices runtime pm status.
> >
> > v2:
> > - Save pci dev power attr to dynamically allocated array. [Rodrigo]
> > - Set autosuspend delay to 0 on supported devices. [Rodrigo]
> > - %s/else if/else in  igt_pm_get_acpi_real_d_state. [Kamil]
> >
> > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
> > ---
> >   lib/igt_pm.c | 290
> +++++++++++++++++++++++++++++++++++++++++++++++++++
> >   lib/igt_pm.h |  23 ++++
> >   2 files changed, 313 insertions(+)
> >
> > diff --git a/lib/igt_pm.c b/lib/igt_pm.c index b409ec463..20296c40b
> > 100644
> > --- a/lib/igt_pm.c
> > +++ b/lib/igt_pm.c
> > @@ -28,6 +28,7 @@
> >   #include <fcntl.h>
> >   #include <stdio.h>
> >   #include <limits.h>
> > +#include <pciaccess.h>
> >   #include <stdlib.h>
> >   #include <string.h>
> >   #include <unistd.h>
> > @@ -75,6 +76,7 @@ enum {
> >   #define MIN_POWER_STR		"min_power\n"
> >   /* Remember to fix this if adding longer strings */
> >   #define MAX_POLICY_STRLEN	strlen(MAX_PERFORMANCE_STR)
> > +#define MAX_PCI_DEVICES		256
> >   int8_t *__sata_pm_policies;
> >   int __scsi_host_cnt;
> >
> > @@ -856,3 +858,291 @@ bool i915_output_is_lpsp_capable(int drm_fd,
> > igt_output_t *output)
> >
> >   	return strstr(buf, "LPSP: capable");
> >   }
> > +
> > +/**
> > + * igt_pm_acpi_d3cold_supported:
> > + * @pci_dev: root port pci_dev.
> > + * Check ACPI D3Cold support.
> > + *
> > + * Returns:
> > + * True if ACPI D3Cold supported otherwise false.
> > + */
> > +bool igt_pm_acpi_d3cold_supported(struct pci_device *pci_dev) {
> > +	char name[PATH_MAX];
> > +	int dir, fd;
> > +
> > +	snprintf(name, PATH_MAX,
> > +
> "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/firmware_node",
> > +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> > +
> > +	dir = open(name, O_RDONLY);
> > +	igt_require(dir > 0);
> > +
> > +	/* BIOS need to enable ACPI D3Cold Support.*/
> > +	fd = openat(dir, "real_power_state", O_RDONLY);
> > +	if (fd < 0 && errno == ENOENT)
> > +		return false;
> > +
> > +	igt_require(fd > 0);
> > +
> > +	return true;
> > +}
> > +
> > +/**
> > + * igt_pm_get_acpi_real_d_state:
> > + * @pci_dev: root port pci_dev.
> > + * Get ACPI D state for a given root port.
> > + *
> > + * Returns:
> > + * igt_acpi_d_state state.
> > + */
> > +enum igt_acpi_d_state
> > +igt_pm_get_acpi_real_d_state(struct pci_device *pci_dev) {
> > +	char name[PATH_MAX];
> > +	char acpi_d_state[64];
> > +	int fd, n_read;
> > +
> > +	snprintf(name, PATH_MAX,
> > +
> "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/firmware_node/real_power_stat
> e",
> > +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> > +
> > +	fd = open(name, O_RDONLY);
> > +	if (fd < 0)
> > +		return IGT_ACPI_UNKNOWN_STATE;
> > +
> > +	n_read = read(fd, acpi_d_state, sizeof(acpi_d_state) - 1);
> > +	igt_assert(n_read >= 0);
> > +	acpi_d_state[n_read] = '\0';
> > +	close(fd);
> > +
> > +	if (strncmp(acpi_d_state, "D0\n", n_read) == 0)
> > +		return IGT_ACPI_D0;
> > +	if  (strncmp(acpi_d_state, "D1\n", n_read) == 0)
> > +		return IGT_ACPI_D1;
> > +	if  (strncmp(acpi_d_state, "D2\n", n_read) == 0)
> > +		return IGT_ACPI_D2;
> > +	if  (strncmp(acpi_d_state, "D3hot\n", n_read) == 0)
> > +		return IGT_ACPI_D3Hot;
> > +	if  (strncmp(acpi_d_state, "D3cold\n", n_read) == 0)
> > +		return IGT_ACPI_D3Cold;
> > +
> > +	return IGT_ACPI_UNKNOWN_STATE;
> > +}
> > +
> > +static struct igt_pm_pci_dev_pwrattr  *__pci_dev_pwrattr; static int
> > +__igt_pm_pci_dev_cnt;
> > +
> > +static void __igt_pm_pci_card_exit_handler(int sig) {
> > +	igt_pm_restore_pci_card_runtime_pm();
> > +}
> > +
> > +static int igt_pm_get_power_attr_fd(struct pci_device *pci_dev, const
> > +char *attr) {
> > +	char name[PATH_MAX];
> > +	int fd;
> > +
> > +	snprintf(name, PATH_MAX,
> "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/power/%s",
> > +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func,
> attr);
> > +
> > +	fd = open(name, O_RDWR);
> > +	igt_assert_f(fd >= 0, "Can't open %s\n", attr);
> > +
> > +	return fd;
> > +}
> > +
> > +static bool igt_pm_save_power_attr(int fd, char *attr, int len, bool
> > +autosuspend) {
> > +	int size;
> > +
> > +	size = read(fd, attr, len - 1);
> > +
> > +	if (autosuspend) {
> > +		if (size < 0)
> > +			return false;
> > +	} else {
> > +		igt_assert(size > 0);
> > +	}
> > +
> > +	attr[size] = '\0';
> > +	strchomp(attr);
> > +
> > +	return true;
> > +}
> > +
> > +static void igt_pm_setup_power_attr(int fd, const char *val) {
> > +	int size, len;
> > +	char buf[6];
> > +
> > +	len = strlen(val);
> > +
> > +	size = write(fd, val, len);
> > +	igt_assert_eq(size, len);
> > +	lseek(fd, 0, SEEK_SET);
> > +	size = read(fd, buf, ARRAY_SIZE(buf));
> > +	igt_assert_eq(size, len);
> > +	igt_assert(strncmp(buf, val, len) == 0); }
> > +
> > +static void
> > +__igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev, int
> > +index) {
> > +	int control_fd, delay_fd, control_size, delay_size;
> > +	char *control, *delay;
> > +
> > +	igt_assert(index <  MAX_PCI_DEVICES) > +
> > +	if (!(index % 16)) {
> > +		__pci_dev_pwrattr =
> > +			realloc(__pci_dev_pwrattr,
> > +				sizeof(struct igt_pm_pci_dev_pwrattr) * (index
> / 16 + 1) * 16);
> > +	}
> > +
> > +	control = __pci_dev_pwrattr[index].control;
> > +	control_size =  sizeof(__pci_dev_pwrattr[index].control);
> > +	delay = __pci_dev_pwrattr[index].autosuspend_delay;
> > +	delay_size =  sizeof(__pci_dev_pwrattr[index].autosuspend_delay);
> > +	__pci_dev_pwrattr[index].pci_dev = pci_dev;
> > +	__pci_dev_pwrattr[index].autosuspend_supported = true;
> > +
> > +	delay_fd = igt_pm_get_power_attr_fd(pci_dev,
> "autosuspend_delay_ms");
> > +	control_fd = igt_pm_get_power_attr_fd(pci_dev, "control");
> > +
> > +	if (!igt_pm_save_power_attr(delay_fd, delay, delay_size, true)) {
> > +		__pci_dev_pwrattr[index].autosuspend_supported = false;
> > +		igt_debug("PCI '%04x:%02x:%02x.%01x' doesn't support
> auto_suspend\n",
> > +			  pci_dev->domain, pci_dev->bus, pci_dev->dev,
> pci_dev->func);
> > +	}
> > +
> > +	igt_pm_save_power_attr(control_fd, control, control_size, false);
> > +	igt_debug("PCI '%04x:%02x:%02x.%01x' Saved 'control,
> autosuspend_delay_ms' as '%s, %s'\n",
> > +		  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func,
> control,
> > +		  __pci_dev_pwrattr[index].autosuspend_supported ? delay :
> "NA");
> > +
> > +	if (!index)
> > +		igt_install_exit_handler(__igt_pm_pci_card_exit_handler);
> > +
> > +	if (__pci_dev_pwrattr[index].autosuspend_supported)
> > +		igt_pm_setup_power_attr(delay_fd, "0\n");
> > +
> > +	igt_pm_setup_power_attr(control_fd, "auto\n");
> > +
> > +	close(delay_fd);
> > +	close(control_fd);
> > +}
> > +
> > +/**
> > + * igt_pm_setup_pci_card_runtime_pm:
> > + * @pci_dev: root port pci_dev.
> > + * Setup runtime PM for all PCI endpoints devices for a given root
> > +port by
> > + * enabling runtime suspend and setting autosuspend_delay_ms to zero.
> > + * It also saves power control attribute for all PCI endpoints
> > + * devices under given root port.
> > + */
> > +void igt_pm_setup_pci_card_runtime_pm(struct pci_device *pci_dev) {
> > +	int primary, secondary, subordinate, ret;
> > +	struct pci_device_iterator *iter;
> > +	struct pci_device *dev;
> > +	int i;
> i should be initialized to 0;
Thanks for catching this, I missed it while rebasing it. It was initialized in v1 patch. 
> > +
> > +	ret = pci_device_get_bridge_buses(pci_dev, &primary, &secondary,
> &subordinate);
> > +	igt_assert(!ret);
> > +
> > +	ret = pci_system_init();
> > +	igt_assert(!ret);
> > +
> > +	iter = pci_slot_match_iterator_create(NULL);
> > +	/* Setup runtime pm for PCI root port */
> > +	__igt_pm_setup_pci_card_runtime_pm(pci_dev, i++);
> > +	while ((dev = pci_device_next(iter)) != NULL) { > +		if (dev->bus >=
> secondary && dev->bus <= subordinate)
> > +			__igt_pm_setup_pci_card_runtime_pm(dev, i++);
> > +	}Searching of pci_device would be little faster if we use itr->mode
> > +=
> match_slot and pass iter->match.slot.bus to pci_device_next()
Iter members are not accessible outside libpciaccess.
So we do need to create iter each time , when do we want to increment the secondary bus.
Creating iter every time will not search faster.
Thanks,
Anshuman Gupta. 
> 
> Thanks,
> Badal Nilawar
> > +
> > +	__igt_pm_pci_dev_cnt = i;
> > +}
> > +
> > +static void
> > +igt_pm_restore_power_attr(struct pci_device *pci_dev, const char
> > +*attr, char *val, int len) {
> > +	int fd;
> > +
> > +	igt_debug("PCI '%04x:%02x:%02x.%01x' Restoring %s attr to '%s'\n",
> > +		  pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func,
> attr,
> > +val);
> > +
> > +	fd = igt_pm_get_power_attr_fd(pci_dev, attr);
> > +	igt_assert(write(fd, val, len) == len);
> > +
> > +	close(fd);
> > +}
> > +
> > +/**
> > + * igt_pm_restore_pci_card_runtime_pm:
> > + * Restore control and autosuspend_delay_ms power attribute for all
> > + * PCI endpoints devices under gfx root port, which were saved
> > +earlier
> > + * by igt_pm_setup_pci_card_runtime_pm().
> > + */
> > +void igt_pm_restore_pci_card_runtime_pm(void)
> > +{
> > +	int i = 0;
> > +
> > +	if (!__pci_dev_pwrattr)
> > +		return;
> > +
> > +	for (i = 0; i < __igt_pm_pci_dev_cnt; i++) {
> > +		igt_pm_restore_power_attr(__pci_dev_pwrattr[i].pci_dev,
> "control",
> > +					  __pci_dev_pwrattr[i].control,
> > +					  sizeof(__pci_dev_pwrattr[i].control));
> > +
> > +		if (!__pci_dev_pwrattr[i].autosuspend_supported)
> > +			continue;
> > +
> > +		igt_pm_restore_power_attr(__pci_dev_pwrattr[i].pci_dev,
> "autosuspend_delay_ms",
> > +
> __pci_dev_pwrattr[i].autosuspend_delay,
> > +
> sizeof(__pci_dev_pwrattr[i].autosuspend_delay));
> > +	}
> > +
> > +	pci_system_cleanup();
> > +	free(__pci_dev_pwrattr);
> > +	__pci_dev_pwrattr = NULL;
> > +}
> > +
> > +static void igt_pm_print_pci_dev_runtime_status(struct pci_device
> > +*pci_dev) {
> > +	char name[PATH_MAX], runtime_status[64];
> > +	int fd, n_read;
> > +
> > +	snprintf(name, PATH_MAX,
> "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/power/runtime_status",
> > +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func);
> > +
> > +	fd = open(name, O_RDONLY);
> > +	igt_assert_f(fd >= 0, "Can't open runtime_status\n");
> > +
> > +	n_read = read(fd, runtime_status, sizeof(runtime_status) - 1);
> > +	igt_assert(n_read >= 0);
> > +	runtime_status[n_read] = '\0';
> > +	igt_info("runtime suspend status for PCI '%04x:%02x:%02x.%01x' %s\n",
> > +		 pci_dev->domain, pci_dev->bus, pci_dev->dev, pci_dev->func,
> runtime_status);
> > +	close(fd);
> > +}
> > +
> > +/**
> > + * igt_pm_print_pci_card_runtime_status:
> > + * @pci_dev: root port pci_dev.
> > + * Print runtime suspend status for all PCI endpoints devices for a
> > +given
> > + * root port.
> > + */
> > +void igt_pm_print_pci_card_runtime_status(void)
> > +{
> > +	int i = 0;
> > +
> > +	if (!__pci_dev_pwrattr)
> > +		return;
> > +
> > +	for (i = 0; i < __igt_pm_pci_dev_cnt; i++)
> > +
> 	igt_pm_print_pci_dev_runtime_status(__pci_dev_pwrattr[i].pci_dev);
> > +}
> > diff --git a/lib/igt_pm.h b/lib/igt_pm.h index 162d3ca3c..898178670
> > 100644
> > --- a/lib/igt_pm.h
> > +++ b/lib/igt_pm.h
> > @@ -46,6 +46,23 @@ enum igt_runtime_pm_status {
> >   	IGT_RUNTIME_PM_STATUS_UNKNOWN,
> >   };
> >
> > +/* PCI ACPI firmware node real state */ enum igt_acpi_d_state {
> > +	IGT_ACPI_D0,
> > +	IGT_ACPI_D1,
> > +	IGT_ACPI_D2,
> > +	IGT_ACPI_D3Hot,
> > +	IGT_ACPI_D3Cold,
> > +	IGT_ACPI_UNKNOWN_STATE,
> > +};
> > +
> > +struct	igt_pm_pci_dev_pwrattr {
> > +	struct pci_device *pci_dev;
> > +	char control[64];
> > +	bool autosuspend_supported;
> > +	char autosuspend_delay[64];
> > +};
> > +
> >   bool igt_setup_runtime_pm(int device);
> >   void igt_disable_runtime_pm(void);
> >   void igt_restore_runtime_pm(void);
> > @@ -54,5 +71,11 @@ bool igt_wait_for_pm_status(enum
> igt_runtime_pm_status status);
> >   bool igt_pm_dmc_loaded(int debugfs);
> >   bool igt_pm_pc8_plus_residencies_enabled(int msr_fd);
> >   bool i915_output_is_lpsp_capable(int drm_fd, igt_output_t *output);
> > +bool igt_pm_acpi_d3cold_supported(struct pci_device *pci_dev); enum
> > +igt_acpi_d_state igt_pm_get_acpi_real_d_state(struct pci_device
> > +*pci_dev); void igt_pm_setup_pci_card_runtime_pm(struct pci_device
> > +*pci_dev); void igt_pm_restore_pci_card_runtime_pm(void);
> > +void igt_pm_print_pci_card_runtime_status(void);
> >
> >   #endif /* IGT_PM_H */

[igt-dev] [PATCH i-g-t v2 4/8] tools: Add intel_pm_rpm tool

[Anshuman Gupta]

intel_pm_rpm tool is a debug tool. It can be use to setup
and prepare the gfx card to go to D3Cold.
It also provide the debug option to disable all display and
prepare device to enter to runtime suspend.

v2:
- Removed IS_DGFX() condition.

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 tools/intel_pm_rpm.c | 206 +++++++++++++++++++++++++++++++++++++++++++
 tools/meson.build    |   1 +
 2 files changed, 207 insertions(+)
 create mode 100644 tools/intel_pm_rpm.c

diff --git a/tools/intel_pm_rpm.c b/tools/intel_pm_rpm.c
new file mode 100644
index 000000000..bf8212b3c
--- /dev/null
+++ b/tools/intel_pm_rpm.c
@@ -0,0 +1,206 @@
+/*
+ * Copyright © 2022 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include <errno.h>
+#include <getopt.h>
+#include <glib.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdbool.h>
+#include "igt.h"
+#include "igt_device.h"
+#include "igt_device_scan.h"
+#include "igt_pm.h"
+
+typedef struct {
+	int drm_fd;
+	int debugfs_fd;
+	uint32_t devid;
+	drmModeResPtr res;
+	igt_display_t display;
+} data_t;
+
+const char *help_str =
+	"  --disable-display\t\tDisable all screen and try to go into runtime pm.\n"
+	"  --setup-d3cold\t\tPrepare dgfx gfx card to enter runtime D3Cold.\n"
+	"  --help\t\tProvide help. Provide card name with IGT_DEVICE=drm:/dev/dri/card*.";
+static struct option long_options[] = {
+	{"disable-display", 0, 0, 'd'},
+	{"setup-d3cold", 0, 0, 's'},
+	{"help", 0, 0, 'h'},
+	{ 0, 0, 0, 0 }
+};
+
+const char *optstr = "dsh";
+
+static void usage(const char *name)
+{
+	igt_info("Usage: %s [options]\n", name);
+	igt_info("%s\n", help_str);
+}
+
+static void disable_all_displays(data_t *data)
+{
+	igt_output_t *output;
+
+	for (int i = 0; i < data->display.n_outputs; i++) {
+		output = &data->display.outputs[i];
+		igt_output_set_pipe(output, PIPE_NONE);
+		igt_display_commit(&data->display);
+	}
+}
+
+static void setup_gfx_card_d3cold(data_t *data)
+{
+	struct pci_device *root;
+	int d_state;
+
+	root = igt_device_get_pci_root_port(data->drm_fd);
+
+	if (!igt_pm_acpi_d3cold_supported(root)) {
+		igt_info("D3Cold isn't supported on Root port %04x:%02x:%02x.%01x\n",
+			 root->domain, root->bus, root->dev, root->func);
+		return;
+	}
+
+	disable_all_displays(data);
+	igt_pm_setup_pci_card_runtime_pm(root);
+	sleep(1);
+	d_state = igt_pm_get_acpi_real_d_state(root);
+
+	if (d_state == IGT_ACPI_D3Cold) {
+		igt_info("D3Cold achieved for root port %04x:%02x:%02x.%01x\n",
+			 root->domain, root->bus, root->dev, root->func);
+	} else {
+		igt_pm_print_pci_card_runtime_status();
+		igt_info("D3Cold not achieved yet. Please monitor %04x:%02x:%02x.%01x real_power_state\n",
+			 root->domain, root->bus, root->dev, root->func);
+	}
+}
+
+int main(int argc, char *argv[])
+{
+	bool disable_display = false, setup_d3cold = false;
+	struct igt_device_card card;
+	char *env_device = NULL;
+	int c, option_index = 0;
+	data_t data = {};
+	int ret = 0;
+
+	if (argc <= 1) {
+		usage(argv[0]);
+		return EXIT_SUCCESS;
+	}
+
+	env_device = getenv("IGT_DEVICE");
+	igt_devices_scan(false);
+
+	if (env_device) {
+		if (!igt_device_card_match(env_device, &card)) {
+			igt_warn("No device found for the env_device\n");
+			ret = EXIT_FAILURE;
+			goto exit;
+		}
+	} else {
+		if (!igt_device_find_first_i915_discrete_card(&card)) {
+			igt_warn("No discrete gpu found\n");
+			ret = EXIT_FAILURE;
+			goto exit;
+		}
+	}
+
+	while ((c = getopt_long(argc, argv, optstr,
+				long_options, &option_index)) != -1) {
+		switch (c) {
+		case 'd':
+			disable_display = true;
+			break;
+		case 's':
+			setup_d3cold = true;
+			break;
+		default:
+		case 'h':
+			usage(argv[0]);
+			ret = EXIT_SUCCESS;
+			goto exit;
+		}
+	}
+
+	data.drm_fd = igt_open_card(&card);
+	if (data.drm_fd  >= 0) {
+		igt_info("Device %s successfully opened\n", card.card);
+	} else {
+		igt_warn("Cannot open card %s device\n", card.card);
+		ret = EXIT_FAILURE;
+		goto exit;
+	}
+
+	data.debugfs_fd = igt_debugfs_dir(data.drm_fd);
+	data.devid = intel_get_drm_devid(data.drm_fd);
+	igt_setup_runtime_pm(data.drm_fd);
+
+	data.res = drmModeGetResources(data.drm_fd);
+	if (data.res) {
+		kmstest_set_vt_graphics_mode();
+		igt_display_require(&data.display, data.drm_fd);
+
+		/* i915 disables RPM in case DMC is not loaded on kms supported cards */
+		if (!igt_pm_dmc_loaded(data.debugfs_fd)) {
+			igt_warn("dmc fw is not loaded, no runtime pm\n");
+			ret = EXIT_FAILURE;
+			goto exit;
+		}
+	}
+
+	if (disable_display) {
+		disable_all_displays(&data);
+		if (!igt_wait_for_pm_status(IGT_RUNTIME_PM_STATUS_SUSPENDED)) {
+			__igt_debugfs_dump(data.drm_fd, "i915_runtime_pm_status", IGT_LOG_INFO);
+			ret = EXIT_FAILURE;
+			goto exit;
+		}
+
+		igt_info("Device runtime suspended, Useful for debugging.\n"
+			 "Hit CTRL-C to exit\n");
+		/* Don't return useful for debugging */
+		while (1)
+			sleep(600);
+	}
+
+	if (setup_d3cold)
+		setup_gfx_card_d3cold(&data);
+
+exit:
+	igt_restore_runtime_pm();
+
+	if (data.res)
+		igt_display_fini(&data.display);
+
+	close(data.debugfs_fd);
+	close(data.drm_fd);
+	igt_devices_free();
+
+	return ret;
+}
diff --git a/tools/meson.build b/tools/meson.build
index 771d0b9e3..24d0ea714 100644
--- a/tools/meson.build
+++ b/tools/meson.build
@@ -28,6 +28,7 @@ tools_progs = [
 	'intel_lid',
 	'intel_opregion_decode',
 	'intel_panel_fitter',
+	'intel_pm_rpm',
 	'intel_reg_checker',
 	'intel_residency',
 	'intel_stepping',
-- 
2.26.2

[igt-dev] [PATCH i-g-t v2 5/8] i915_pm_rpm: Add D3Cold basic subtest

[Anshuman Gupta]

Add support for D3Cold basic subtest.
It setup and prepares the gfx PCI card for D3Cold
and checks the ACPI D3Cold state.

v2:
- Removed IS_DGFX() condition.
- Added igt_describe(). [Kamil]

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 tests/i915/i915_pm_rpm.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tests/i915/i915_pm_rpm.c b/tests/i915/i915_pm_rpm.c
index d2bce5826..31f9712a8 100644
--- a/tests/i915/i915_pm_rpm.c
+++ b/tests/i915/i915_pm_rpm.c
@@ -1534,6 +1534,28 @@ __noreturn static void stay_subtest(void)
 		sleep(600);
 }
 
+static void d3cold_basic_subtest(void)
+{
+	struct pci_device *root;
+	bool result;
+
+	root = igt_device_get_pci_root_port(drm_fd);
+	igt_require(igt_pm_acpi_d3cold_supported(root));
+	igt_pm_setup_pci_card_runtime_pm(root);
+	disable_all_screens_and_wait(&ms_data);
+
+	result = igt_wait(igt_pm_get_acpi_real_d_state(root) == IGT_ACPI_D3Cold, 10000, 500);
+
+	if (!result) {
+		igt_info("D3Cold not achieved for root port %04x:%02x:%02x.%01x\n",
+			 root->domain, root->bus, root->dev, root->func);
+		igt_pm_print_pci_card_runtime_status();
+	}
+
+	igt_assert(result);
+	igt_pm_restore_pci_card_runtime_pm();
+}
+
 static void system_suspend_subtest(int state, int debug)
 {
 	disable_all_screens_and_wait(&ms_data);
@@ -2065,6 +2087,10 @@ igt_main_args("", long_options, help_str, opt_handler, NULL)
 		igt_subtest("stay")
 			stay_subtest();
 
+	igt_describe("Validate gfx root port d3cold state");
+	igt_subtest("d3cold-basic")
+		d3cold_basic_subtest();
+
 	/* Essential things */
 	igt_subtest("drm-resources-equal")
 		drm_resources_equal_subtest();
-- 
2.26.2

[igt-dev] [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test with D3Cold

[Anshuman Gupta]

Added d3cold dynamic subtest to gem_exec_stress with device class
memory region. It test both D3Cold-{VRAM_SR, Off} by using
d3cold_sr_lmem_threshold i915_params debugfs.

v2:
- Two different subtest d3cold-off and d3cold-vrsr. [Rodrigo]
- commit header modification.

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
---
 tests/i915/i915_pm_rpm.c | 97 ++++++++++++++++++++++++++++++++++------
 1 file changed, 83 insertions(+), 14 deletions(-)

diff --git a/tests/i915/i915_pm_rpm.c b/tests/i915/i915_pm_rpm.c
index 31f9712a8..1ef76f562 100644
--- a/tests/i915/i915_pm_rpm.c
+++ b/tests/i915/i915_pm_rpm.c
@@ -85,6 +85,7 @@ enum plane_type {
 #define WAIT_PC8_RES	2
 #define WAIT_EXTRA	4
 #define USE_DPMS	8
+#define WAIT_D3COLD	16
 
 int drm_fd, msr_fd, pc8_status_fd;
 int debugfs;
@@ -215,6 +216,21 @@ static bool wait_for_suspended(void)
 	}
 }
 
+static bool wait_for_d3cold(struct pci_device *root)
+{
+	bool d3colded;
+
+	d3colded = igt_wait(igt_pm_get_acpi_real_d_state(root) == IGT_ACPI_D3Cold, 10000, 500);
+
+	if (!d3colded) {
+		igt_info("D3Cold not achieved for root port %04x:%02x:%02x.%01x\n",
+			 root->domain, root->bus, root->dev, root->func);
+		igt_pm_print_pci_card_runtime_status();
+	}
+
+	return d3colded;
+}
+
 static bool wait_for_active(void)
 {
 	if (has_pc8 && !has_runtime_pm)
@@ -744,6 +760,30 @@ static void test_i2c(struct mode_set_data *data)
 			"There is an EDID mismatch between i2c and DRM!\n");
 }
 
+static int get_d3cold_sr_lmem_threshold(int dir)
+{
+	int param_dir, lmem_threshold, ret;
+
+	param_dir = openat(dir, "i915_params", O_RDONLY);
+	ret = igt_sysfs_scanf(param_dir, "d3cold_sr_lmem_threshold", "%d", &lmem_threshold);
+	igt_assert(ret >= 0);
+
+	close(param_dir);
+	return lmem_threshold;
+}
+
+static int set_d3cold_sr_lmem_threshold(int dir, int val)
+{
+	int param_dir, ret;
+
+	param_dir = openat(dir, "i915_params", O_RDONLY);
+	ret = igt_sysfs_printf(param_dir, "d3cold_sr_lmem_threshold", "%d", val);
+	igt_assert(ret > 0);
+
+	close(param_dir);
+	return ret;
+}
+
 static void setup_pc8(void)
 {
 	has_pc8 = false;
@@ -1069,6 +1109,17 @@ static void debugfs_forcewake_user_subtest(void)
 	igt_assert(wait_for_suspended());
 }
 
+static struct pci_device *setup_d3cold_and_get_root_port(void)
+{
+	struct pci_device *root;
+
+	root = igt_device_get_pci_root_port(drm_fd);
+	igt_require(igt_pm_acpi_d3cold_supported(root));
+	igt_pm_setup_pci_card_runtime_pm(root);
+
+	return root;
+}
+
 static void gem_mmap_args(const struct mmap_offset *t,
 			  struct drm_i915_gem_memory_class_instance *mem_regions)
 {
@@ -1392,6 +1443,7 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
 	int i;
 	int batch_size = 4 * sizeof(uint32_t);
 	uint32_t batch_buf[batch_size];
+	struct pci_device *root;
 	uint32_t handle;
 	struct drm_i915_gem_execbuffer2 execbuf = {};
 	struct drm_i915_gem_exec_object2 objs[1] = {{}};
@@ -1408,6 +1460,9 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
 	batch_buf[i++] = MI_NOOP;
 	igt_assert(i * sizeof(uint32_t) == batch_size);
 
+	if (wait_flags & WAIT_D3COLD)
+		root = setup_d3cold_and_get_root_port();
+
 	disable_all_screens_and_wait(&ms_data);
 
 	/* PC8 test is only applicable to igfx  */
@@ -1433,6 +1488,9 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
 			/* clean up idle work */
 			igt_drop_caches_set(drm_fd, DROP_IDLE);
 			igt_assert(wait_for_suspended());
+			if (wait_flags & WAIT_D3COLD)
+				igt_assert(wait_for_d3cold(root));
+
 		}
 		if (wait_flags & WAIT_PC8_RES)
 			igt_assert(pc8_plus_residency_changed(30));
@@ -1440,6 +1498,9 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
 			sleep(5);
 	}
 
+	if (wait_flags & WAIT_D3COLD)
+		igt_pm_restore_pci_card_runtime_pm();
+
 	gem_close(drm_fd, handle);
 }
 
@@ -1537,22 +1598,10 @@ __noreturn static void stay_subtest(void)
 static void d3cold_basic_subtest(void)
 {
 	struct pci_device *root;
-	bool result;
 
-	root = igt_device_get_pci_root_port(drm_fd);
-	igt_require(igt_pm_acpi_d3cold_supported(root));
-	igt_pm_setup_pci_card_runtime_pm(root);
+	root = setup_d3cold_and_get_root_port();
 	disable_all_screens_and_wait(&ms_data);
-
-	result = igt_wait(igt_pm_get_acpi_real_d_state(root) == IGT_ACPI_D3Cold, 10000, 500);
-
-	if (!result) {
-		igt_info("D3Cold not achieved for root port %04x:%02x:%02x.%01x\n",
-			 root->domain, root->bus, root->dev, root->func);
-		igt_pm_print_pci_card_runtime_status();
-	}
-
-	igt_assert(result);
+	igt_assert(wait_for_d3cold(root));
 	igt_pm_restore_pci_card_runtime_pm();
 }
 
@@ -2206,11 +2255,31 @@ igt_main_args("", long_options, help_str, opt_handler, NULL)
 	igt_describe("Validate execbuf submission while exercising rpm "
 		     "suspend/resume cycles.");
 	igt_subtest_with_dynamic("gem-execbuf-stress") {
+		int lmem_threshold;
+
 		for_each_memory_region(r, drm_fd) {
 			igt_dynamic_f("%s", r->name)
 				gem_execbuf_stress_subtest(rounds, WAIT_STATUS, &r->ci);
 			igt_dynamic_f("%s-%s", "extra-wait", r->name)
 				gem_execbuf_stress_subtest(rounds, WAIT_STATUS | WAIT_EXTRA, &r->ci);
+
+			if (r->ci.memory_class == I915_MEMORY_CLASS_SYSTEM)
+				continue;
+
+			lmem_threshold = get_d3cold_sr_lmem_threshold(debugfs);
+			igt_dynamic_f("%s-%s", "d3cold-off", r->name) {
+				set_d3cold_sr_lmem_threshold(debugfs, 300);
+				gem_execbuf_stress_subtest(rounds, WAIT_STATUS | WAIT_D3COLD,
+							   &r->ci);
+				set_d3cold_sr_lmem_threshold(debugfs, lmem_threshold);
+			}
+
+			igt_dynamic_f("%s-%s", "d3cold-vrsr", r->name) {
+				set_d3cold_sr_lmem_threshold(debugfs, 0);
+				gem_execbuf_stress_subtest(rounds, WAIT_STATUS | WAIT_D3COLD,
+							   &r->ci);
+				set_d3cold_sr_lmem_threshold(debugfs, lmem_threshold);
+			}
 		}
 	}
 
-- 
2.26.2

Re: [igt-dev] [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test with D3Cold

[Rodrigo Vivi]

On Fri, Apr 29, 2022 at 02:24:23PM +0530, Anshuman Gupta wrote:
> Added d3cold dynamic subtest to gem_exec_stress with device class
> memory region. It test both D3Cold-{VRAM_SR, Off} by using
> d3cold_sr_lmem_threshold i915_params debugfs.
> 
> v2:
> - Two different subtest d3cold-off and d3cold-vrsr. [Rodrigo]
> - commit header modification.
> 
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
> ---
>  tests/i915/i915_pm_rpm.c | 97 ++++++++++++++++++++++++++++++++++------
>  1 file changed, 83 insertions(+), 14 deletions(-)
> 
> diff --git a/tests/i915/i915_pm_rpm.c b/tests/i915/i915_pm_rpm.c
> index 31f9712a8..1ef76f562 100644
> --- a/tests/i915/i915_pm_rpm.c
> +++ b/tests/i915/i915_pm_rpm.c
> @@ -85,6 +85,7 @@ enum plane_type {
>  #define WAIT_PC8_RES	2
>  #define WAIT_EXTRA	4
>  #define USE_DPMS	8
> +#define WAIT_D3COLD	16
>  
>  int drm_fd, msr_fd, pc8_status_fd;
>  int debugfs;
> @@ -215,6 +216,21 @@ static bool wait_for_suspended(void)
>  	}
>  }
>  
> +static bool wait_for_d3cold(struct pci_device *root)
> +{
> +	bool d3colded;
> +
> +	d3colded = igt_wait(igt_pm_get_acpi_real_d_state(root) == IGT_ACPI_D3Cold, 10000, 500);
> +
> +	if (!d3colded) {
> +		igt_info("D3Cold not achieved for root port %04x:%02x:%02x.%01x\n",
> +			 root->domain, root->bus, root->dev, root->func);
> +		igt_pm_print_pci_card_runtime_status();
> +	}
> +
> +	return d3colded;
> +}
> +
>  static bool wait_for_active(void)
>  {
>  	if (has_pc8 && !has_runtime_pm)
> @@ -744,6 +760,30 @@ static void test_i2c(struct mode_set_data *data)
>  			"There is an EDID mismatch between i2c and DRM!\n");
>  }
>  
> +static int get_d3cold_sr_lmem_threshold(int dir)
> +{
> +	int param_dir, lmem_threshold, ret;
> +
> +	param_dir = openat(dir, "i915_params", O_RDONLY);
> +	ret = igt_sysfs_scanf(param_dir, "d3cold_sr_lmem_threshold", "%d", &lmem_threshold);
> +	igt_assert(ret >= 0);
> +
> +	close(param_dir);
> +	return lmem_threshold;
> +}
> +
> +static int set_d3cold_sr_lmem_threshold(int dir, int val)
> +{
> +	int param_dir, ret;
> +
> +	param_dir = openat(dir, "i915_params", O_RDONLY);
> +	ret = igt_sysfs_printf(param_dir, "d3cold_sr_lmem_threshold", "%d", val);
> +	igt_assert(ret > 0);
> +
> +	close(param_dir);
> +	return ret;
> +}
> +
>  static void setup_pc8(void)
>  {
>  	has_pc8 = false;
> @@ -1069,6 +1109,17 @@ static void debugfs_forcewake_user_subtest(void)
>  	igt_assert(wait_for_suspended());
>  }
>  
> +static struct pci_device *setup_d3cold_and_get_root_port(void)
> +{
> +	struct pci_device *root;
> +
> +	root = igt_device_get_pci_root_port(drm_fd);
> +	igt_require(igt_pm_acpi_d3cold_supported(root));
> +	igt_pm_setup_pci_card_runtime_pm(root);
> +
> +	return root;
> +}
> +
>  static void gem_mmap_args(const struct mmap_offset *t,
>  			  struct drm_i915_gem_memory_class_instance *mem_regions)
>  {
> @@ -1392,6 +1443,7 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
>  	int i;
>  	int batch_size = 4 * sizeof(uint32_t);
>  	uint32_t batch_buf[batch_size];
> +	struct pci_device *root;
>  	uint32_t handle;
>  	struct drm_i915_gem_execbuffer2 execbuf = {};
>  	struct drm_i915_gem_exec_object2 objs[1] = {{}};
> @@ -1408,6 +1460,9 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
>  	batch_buf[i++] = MI_NOOP;
>  	igt_assert(i * sizeof(uint32_t) == batch_size);
>  
> +	if (wait_flags & WAIT_D3COLD)
> +		root = setup_d3cold_and_get_root_port();
> +
>  	disable_all_screens_and_wait(&ms_data);
>  
>  	/* PC8 test is only applicable to igfx  */
> @@ -1433,6 +1488,9 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
>  			/* clean up idle work */
>  			igt_drop_caches_set(drm_fd, DROP_IDLE);
>  			igt_assert(wait_for_suspended());
> +			if (wait_flags & WAIT_D3COLD)
> +				igt_assert(wait_for_d3cold(root));
> +
>  		}
>  		if (wait_flags & WAIT_PC8_RES)
>  			igt_assert(pc8_plus_residency_changed(30));
> @@ -1440,6 +1498,9 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
>  			sleep(5);
>  	}
>  
> +	if (wait_flags & WAIT_D3COLD)
> +		igt_pm_restore_pci_card_runtime_pm();
> +
>  	gem_close(drm_fd, handle);
>  }
>  
> @@ -1537,22 +1598,10 @@ __noreturn static void stay_subtest(void)
>  static void d3cold_basic_subtest(void)
>  {
>  	struct pci_device *root;
> -	bool result;
>  
> -	root = igt_device_get_pci_root_port(drm_fd);
> -	igt_require(igt_pm_acpi_d3cold_supported(root));
> -	igt_pm_setup_pci_card_runtime_pm(root);
> +	root = setup_d3cold_and_get_root_port();
>  	disable_all_screens_and_wait(&ms_data);
> -
> -	result = igt_wait(igt_pm_get_acpi_real_d_state(root) == IGT_ACPI_D3Cold, 10000, 500);
> -
> -	if (!result) {
> -		igt_info("D3Cold not achieved for root port %04x:%02x:%02x.%01x\n",
> -			 root->domain, root->bus, root->dev, root->func);
> -		igt_pm_print_pci_card_runtime_status();
> -	}
> -
> -	igt_assert(result);
> +	igt_assert(wait_for_d3cold(root));
>  	igt_pm_restore_pci_card_runtime_pm();
>  }
>  
> @@ -2206,11 +2255,31 @@ igt_main_args("", long_options, help_str, opt_handler, NULL)
>  	igt_describe("Validate execbuf submission while exercising rpm "
>  		     "suspend/resume cycles.");
>  	igt_subtest_with_dynamic("gem-execbuf-stress") {
> +		int lmem_threshold;
> +
>  		for_each_memory_region(r, drm_fd) {
>  			igt_dynamic_f("%s", r->name)
>  				gem_execbuf_stress_subtest(rounds, WAIT_STATUS, &r->ci);
>  			igt_dynamic_f("%s-%s", "extra-wait", r->name)
>  				gem_execbuf_stress_subtest(rounds, WAIT_STATUS | WAIT_EXTRA, &r->ci);
> +
> +			if (r->ci.memory_class == I915_MEMORY_CLASS_SYSTEM)
> +				continue;
> +
> +			lmem_threshold = get_d3cold_sr_lmem_threshold(debugfs);

since we have just one global threashold and not one per memory region we can avoid the
extra reads in here by moving it ouside.

> +			igt_dynamic_f("%s-%s", "d3cold-off", r->name) {
> +				set_d3cold_sr_lmem_threshold(debugfs, 300);
> +				gem_execbuf_stress_subtest(rounds, WAIT_STATUS | WAIT_D3COLD,
> +							   &r->ci);
> +				set_d3cold_sr_lmem_threshold(debugfs, lmem_threshold);

but also we should probably do the save/restore with igt_fixture?

> +			}
> +
> +			igt_dynamic_f("%s-%s", "d3cold-vrsr", r->name) {
> +				set_d3cold_sr_lmem_threshold(debugfs, 0);
> +				gem_execbuf_stress_subtest(rounds, WAIT_STATUS | WAIT_D3COLD,
> +							   &r->ci);
> +				set_d3cold_sr_lmem_threshold(debugfs, lmem_threshold);
> +			}
>  		}
>  	}
>  
> -- 
> 2.26.2
> 

Re: [igt-dev] [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test with D3Cold

[Gupta, Anshuman]

> -----Original Message-----
> From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> Sent: Friday, April 29, 2022 6:37 PM
> To: Gupta, Anshuman <anshuman.gupta@intel.com>
> Cc: igt-dev@lists.freedesktop.org; kamil.konieczny@linux.intel.com; Latvala,
> Petri <petri.latvala@intel.com>; Nilawar, Badal <badal.nilawar@intel.com>;
> Ewins, Jon <jon.ewins@intel.com>
> Subject: Re: [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test
> with D3Cold
> 
> On Fri, Apr 29, 2022 at 02:24:23PM +0530, Anshuman Gupta wrote:
> > Added d3cold dynamic subtest to gem_exec_stress with device class
> > memory region. It test both D3Cold-{VRAM_SR, Off} by using
> > d3cold_sr_lmem_threshold i915_params debugfs.
> >
> > v2:
> > - Two different subtest d3cold-off and d3cold-vrsr. [Rodrigo]
> > - commit header modification.
> >
> > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
> > ---
> >  tests/i915/i915_pm_rpm.c | 97
> > ++++++++++++++++++++++++++++++++++------
> >  1 file changed, 83 insertions(+), 14 deletions(-)
> >
> > diff --git a/tests/i915/i915_pm_rpm.c b/tests/i915/i915_pm_rpm.c index
> > 31f9712a8..1ef76f562 100644
> > --- a/tests/i915/i915_pm_rpm.c
> > +++ b/tests/i915/i915_pm_rpm.c
> > @@ -85,6 +85,7 @@ enum plane_type {
> >  #define WAIT_PC8_RES	2
> >  #define WAIT_EXTRA	4
> >  #define USE_DPMS	8
> > +#define WAIT_D3COLD	16
> >
> >  int drm_fd, msr_fd, pc8_status_fd;
> >  int debugfs;
> > @@ -215,6 +216,21 @@ static bool wait_for_suspended(void)
> >  	}
> >  }
> >
> > +static bool wait_for_d3cold(struct pci_device *root) {
> > +	bool d3colded;
> > +
> > +	d3colded = igt_wait(igt_pm_get_acpi_real_d_state(root) ==
> > +IGT_ACPI_D3Cold, 10000, 500);
> > +
> > +	if (!d3colded) {
> > +		igt_info("D3Cold not achieved for root port
> %04x:%02x:%02x.%01x\n",
> > +			 root->domain, root->bus, root->dev, root->func);
> > +		igt_pm_print_pci_card_runtime_status();
> > +	}
> > +
> > +	return d3colded;
> > +}
> > +
> >  static bool wait_for_active(void)
> >  {
> >  	if (has_pc8 && !has_runtime_pm)
> > @@ -744,6 +760,30 @@ static void test_i2c(struct mode_set_data *data)
> >  			"There is an EDID mismatch between i2c and DRM!\n");
> }
> >
> > +static int get_d3cold_sr_lmem_threshold(int dir) {
> > +	int param_dir, lmem_threshold, ret;
> > +
> > +	param_dir = openat(dir, "i915_params", O_RDONLY);
> > +	ret = igt_sysfs_scanf(param_dir, "d3cold_sr_lmem_threshold", "%d",
> &lmem_threshold);
> > +	igt_assert(ret >= 0);
> > +
> > +	close(param_dir);
> > +	return lmem_threshold;
> > +}
> > +
> > +static int set_d3cold_sr_lmem_threshold(int dir, int val) {
> > +	int param_dir, ret;
> > +
> > +	param_dir = openat(dir, "i915_params", O_RDONLY);
> > +	ret = igt_sysfs_printf(param_dir, "d3cold_sr_lmem_threshold", "%d",
> val);
> > +	igt_assert(ret > 0);
> > +
> > +	close(param_dir);
> > +	return ret;
> > +}
> > +
> >  static void setup_pc8(void)
> >  {
> >  	has_pc8 = false;
> > @@ -1069,6 +1109,17 @@ static void debugfs_forcewake_user_subtest(void)
> >  	igt_assert(wait_for_suspended());
> >  }
> >
> > +static struct pci_device *setup_d3cold_and_get_root_port(void)
> > +{
> > +	struct pci_device *root;
> > +
> > +	root = igt_device_get_pci_root_port(drm_fd);
> > +	igt_require(igt_pm_acpi_d3cold_supported(root));
> > +	igt_pm_setup_pci_card_runtime_pm(root);
> > +
> > +	return root;
> > +}
> > +
> >  static void gem_mmap_args(const struct mmap_offset *t,
> >  			  struct drm_i915_gem_memory_class_instance
> *mem_regions)  { @@
> > -1392,6 +1443,7 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
> >  	int i;
> >  	int batch_size = 4 * sizeof(uint32_t);
> >  	uint32_t batch_buf[batch_size];
> > +	struct pci_device *root;
> >  	uint32_t handle;
> >  	struct drm_i915_gem_execbuffer2 execbuf = {};
> >  	struct drm_i915_gem_exec_object2 objs[1] = {{}}; @@ -1408,6 +1460,9
> > @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
> >  	batch_buf[i++] = MI_NOOP;
> >  	igt_assert(i * sizeof(uint32_t) == batch_size);
> >
> > +	if (wait_flags & WAIT_D3COLD)
> > +		root = setup_d3cold_and_get_root_port();
> > +
> >  	disable_all_screens_and_wait(&ms_data);
> >
> >  	/* PC8 test is only applicable to igfx  */ @@ -1433,6 +1488,9 @@
> > gem_execbuf_stress_subtest(int rounds, int wait_flags,
> >  			/* clean up idle work */
> >  			igt_drop_caches_set(drm_fd, DROP_IDLE);
> >  			igt_assert(wait_for_suspended());
> > +			if (wait_flags & WAIT_D3COLD)
> > +				igt_assert(wait_for_d3cold(root));
> > +
> >  		}
> >  		if (wait_flags & WAIT_PC8_RES)
> >  			igt_assert(pc8_plus_residency_changed(30));
> > @@ -1440,6 +1498,9 @@ gem_execbuf_stress_subtest(int rounds, int
> wait_flags,
> >  			sleep(5);
> >  	}
> >
> > +	if (wait_flags & WAIT_D3COLD)
> > +		igt_pm_restore_pci_card_runtime_pm();
> > +
> >  	gem_close(drm_fd, handle);
> >  }
> >
> > @@ -1537,22 +1598,10 @@ __noreturn static void stay_subtest(void)
> > static void d3cold_basic_subtest(void)  {
> >  	struct pci_device *root;
> > -	bool result;
> >
> > -	root = igt_device_get_pci_root_port(drm_fd);
> > -	igt_require(igt_pm_acpi_d3cold_supported(root));
> > -	igt_pm_setup_pci_card_runtime_pm(root);
> > +	root = setup_d3cold_and_get_root_port();
> >  	disable_all_screens_and_wait(&ms_data);
> > -
> > -	result = igt_wait(igt_pm_get_acpi_real_d_state(root) ==
> IGT_ACPI_D3Cold, 10000, 500);
> > -
> > -	if (!result) {
> > -		igt_info("D3Cold not achieved for root port
> %04x:%02x:%02x.%01x\n",
> > -			 root->domain, root->bus, root->dev, root->func);
> > -		igt_pm_print_pci_card_runtime_status();
> > -	}
> > -
> > -	igt_assert(result);
> > +	igt_assert(wait_for_d3cold(root));
> >  	igt_pm_restore_pci_card_runtime_pm();
> >  }
> >
> > @@ -2206,11 +2255,31 @@ igt_main_args("", long_options, help_str,
> opt_handler, NULL)
> >  	igt_describe("Validate execbuf submission while exercising rpm "
> >  		     "suspend/resume cycles.");
> >  	igt_subtest_with_dynamic("gem-execbuf-stress") {
> > +		int lmem_threshold;
> > +
> >  		for_each_memory_region(r, drm_fd) {
> >  			igt_dynamic_f("%s", r->name)
> >  				gem_execbuf_stress_subtest(rounds,
> WAIT_STATUS, &r->ci);
> >  			igt_dynamic_f("%s-%s", "extra-wait", r->name)
> >  				gem_execbuf_stress_subtest(rounds,
> WAIT_STATUS | WAIT_EXTRA,
> > &r->ci);
> > +
> > +			if (r->ci.memory_class ==
> I915_MEMORY_CLASS_SYSTEM)
> > +				continue;
> > +
> > +			lmem_threshold =
> get_d3cold_sr_lmem_threshold(debugfs);
> 
> since we have just one global threashold and not one per memory region we can
> avoid the extra reads in here by moving it ouside.
> 
> > +			igt_dynamic_f("%s-%s", "d3cold-off", r->name) {
> > +				set_d3cold_sr_lmem_threshold(debugfs, 300);
> > +				gem_execbuf_stress_subtest(rounds,
> WAIT_STATUS | WAIT_D3COLD,
> > +							   &r->ci);
> > +				set_d3cold_sr_lmem_threshold(debugfs,
> lmem_threshold);
> 
> but also we should probably do the save/restore with igt_fixture?
We can do once we have this param in kernel , otherwise CI will break for all i915_pm_rpm tests. 
Thanks,
Anshuman Gupta.
> 
> > +			}
> > +
> > +			igt_dynamic_f("%s-%s", "d3cold-vrsr", r->name) {
> > +				set_d3cold_sr_lmem_threshold(debugfs, 0);
> > +				gem_execbuf_stress_subtest(rounds,
> WAIT_STATUS | WAIT_D3COLD,
> > +							   &r->ci);
> > +				set_d3cold_sr_lmem_threshold(debugfs,
> lmem_threshold);
> > +			}
> >  		}
> >  	}
> >
> > --
> > 2.26.2
> >

Re: [igt-dev] [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test with D3Cold

[Petri Latvala]

On Fri, Apr 29, 2022 at 04:10:15PM +0300, Gupta, Anshuman wrote:
> 
> 
> > -----Original Message-----
> > From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > Sent: Friday, April 29, 2022 6:37 PM
> > To: Gupta, Anshuman <anshuman.gupta@intel.com>
> > Cc: igt-dev@lists.freedesktop.org; kamil.konieczny@linux.intel.com; Latvala,
> > Petri <petri.latvala@intel.com>; Nilawar, Badal <badal.nilawar@intel.com>;
> > Ewins, Jon <jon.ewins@intel.com>
> > Subject: Re: [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test
> > with D3Cold
> > 
> > On Fri, Apr 29, 2022 at 02:24:23PM +0530, Anshuman Gupta wrote:
> > > Added d3cold dynamic subtest to gem_exec_stress with device class
> > > memory region. It test both D3Cold-{VRAM_SR, Off} by using
> > > d3cold_sr_lmem_threshold i915_params debugfs.
> > >
> > > v2:
> > > - Two different subtest d3cold-off and d3cold-vrsr. [Rodrigo]
> > > - commit header modification.
> > >
> > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
> > > ---
> > >  tests/i915/i915_pm_rpm.c | 97
> > > ++++++++++++++++++++++++++++++++++------
> > >  1 file changed, 83 insertions(+), 14 deletions(-)
> > >
> > > diff --git a/tests/i915/i915_pm_rpm.c b/tests/i915/i915_pm_rpm.c index
> > > 31f9712a8..1ef76f562 100644
> > > --- a/tests/i915/i915_pm_rpm.c
> > > +++ b/tests/i915/i915_pm_rpm.c
> > > @@ -85,6 +85,7 @@ enum plane_type {
> > >  #define WAIT_PC8_RES	2
> > >  #define WAIT_EXTRA	4
> > >  #define USE_DPMS	8
> > > +#define WAIT_D3COLD	16
> > >
> > >  int drm_fd, msr_fd, pc8_status_fd;
> > >  int debugfs;
> > > @@ -215,6 +216,21 @@ static bool wait_for_suspended(void)
> > >  	}
> > >  }
> > >
> > > +static bool wait_for_d3cold(struct pci_device *root) {
> > > +	bool d3colded;
> > > +
> > > +	d3colded = igt_wait(igt_pm_get_acpi_real_d_state(root) ==
> > > +IGT_ACPI_D3Cold, 10000, 500);
> > > +
> > > +	if (!d3colded) {
> > > +		igt_info("D3Cold not achieved for root port
> > %04x:%02x:%02x.%01x\n",
> > > +			 root->domain, root->bus, root->dev, root->func);
> > > +		igt_pm_print_pci_card_runtime_status();
> > > +	}
> > > +
> > > +	return d3colded;
> > > +}
> > > +
> > >  static bool wait_for_active(void)
> > >  {
> > >  	if (has_pc8 && !has_runtime_pm)
> > > @@ -744,6 +760,30 @@ static void test_i2c(struct mode_set_data *data)
> > >  			"There is an EDID mismatch between i2c and DRM!\n");
> > }
> > >
> > > +static int get_d3cold_sr_lmem_threshold(int dir) {
> > > +	int param_dir, lmem_threshold, ret;
> > > +
> > > +	param_dir = openat(dir, "i915_params", O_RDONLY);
> > > +	ret = igt_sysfs_scanf(param_dir, "d3cold_sr_lmem_threshold", "%d",
> > &lmem_threshold);
> > > +	igt_assert(ret >= 0);
> > > +
> > > +	close(param_dir);
> > > +	return lmem_threshold;
> > > +}
> > > +
> > > +static int set_d3cold_sr_lmem_threshold(int dir, int val) {
> > > +	int param_dir, ret;
> > > +
> > > +	param_dir = openat(dir, "i915_params", O_RDONLY);
> > > +	ret = igt_sysfs_printf(param_dir, "d3cold_sr_lmem_threshold", "%d",
> > val);
> > > +	igt_assert(ret > 0);
> > > +
> > > +	close(param_dir);
> > > +	return ret;
> > > +}
> > > +
> > >  static void setup_pc8(void)
> > >  {
> > >  	has_pc8 = false;
> > > @@ -1069,6 +1109,17 @@ static void debugfs_forcewake_user_subtest(void)
> > >  	igt_assert(wait_for_suspended());
> > >  }
> > >
> > > +static struct pci_device *setup_d3cold_and_get_root_port(void)
> > > +{
> > > +	struct pci_device *root;
> > > +
> > > +	root = igt_device_get_pci_root_port(drm_fd);
> > > +	igt_require(igt_pm_acpi_d3cold_supported(root));
> > > +	igt_pm_setup_pci_card_runtime_pm(root);
> > > +
> > > +	return root;
> > > +}
> > > +
> > >  static void gem_mmap_args(const struct mmap_offset *t,
> > >  			  struct drm_i915_gem_memory_class_instance
> > *mem_regions)  { @@
> > > -1392,6 +1443,7 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
> > >  	int i;
> > >  	int batch_size = 4 * sizeof(uint32_t);
> > >  	uint32_t batch_buf[batch_size];
> > > +	struct pci_device *root;
> > >  	uint32_t handle;
> > >  	struct drm_i915_gem_execbuffer2 execbuf = {};
> > >  	struct drm_i915_gem_exec_object2 objs[1] = {{}}; @@ -1408,6 +1460,9
> > > @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
> > >  	batch_buf[i++] = MI_NOOP;
> > >  	igt_assert(i * sizeof(uint32_t) == batch_size);
> > >
> > > +	if (wait_flags & WAIT_D3COLD)
> > > +		root = setup_d3cold_and_get_root_port();
> > > +
> > >  	disable_all_screens_and_wait(&ms_data);
> > >
> > >  	/* PC8 test is only applicable to igfx  */ @@ -1433,6 +1488,9 @@
> > > gem_execbuf_stress_subtest(int rounds, int wait_flags,
> > >  			/* clean up idle work */
> > >  			igt_drop_caches_set(drm_fd, DROP_IDLE);
> > >  			igt_assert(wait_for_suspended());
> > > +			if (wait_flags & WAIT_D3COLD)
> > > +				igt_assert(wait_for_d3cold(root));
> > > +
> > >  		}
> > >  		if (wait_flags & WAIT_PC8_RES)
> > >  			igt_assert(pc8_plus_residency_changed(30));
> > > @@ -1440,6 +1498,9 @@ gem_execbuf_stress_subtest(int rounds, int
> > wait_flags,
> > >  			sleep(5);
> > >  	}
> > >
> > > +	if (wait_flags & WAIT_D3COLD)
> > > +		igt_pm_restore_pci_card_runtime_pm();
> > > +
> > >  	gem_close(drm_fd, handle);
> > >  }
> > >
> > > @@ -1537,22 +1598,10 @@ __noreturn static void stay_subtest(void)
> > > static void d3cold_basic_subtest(void)  {
> > >  	struct pci_device *root;
> > > -	bool result;
> > >
> > > -	root = igt_device_get_pci_root_port(drm_fd);
> > > -	igt_require(igt_pm_acpi_d3cold_supported(root));
> > > -	igt_pm_setup_pci_card_runtime_pm(root);
> > > +	root = setup_d3cold_and_get_root_port();
> > >  	disable_all_screens_and_wait(&ms_data);
> > > -
> > > -	result = igt_wait(igt_pm_get_acpi_real_d_state(root) ==
> > IGT_ACPI_D3Cold, 10000, 500);
> > > -
> > > -	if (!result) {
> > > -		igt_info("D3Cold not achieved for root port
> > %04x:%02x:%02x.%01x\n",
> > > -			 root->domain, root->bus, root->dev, root->func);
> > > -		igt_pm_print_pci_card_runtime_status();
> > > -	}
> > > -
> > > -	igt_assert(result);
> > > +	igt_assert(wait_for_d3cold(root));
> > >  	igt_pm_restore_pci_card_runtime_pm();
> > >  }
> > >
> > > @@ -2206,11 +2255,31 @@ igt_main_args("", long_options, help_str,
> > opt_handler, NULL)
> > >  	igt_describe("Validate execbuf submission while exercising rpm "
> > >  		     "suspend/resume cycles.");
> > >  	igt_subtest_with_dynamic("gem-execbuf-stress") {
> > > +		int lmem_threshold;
> > > +
> > >  		for_each_memory_region(r, drm_fd) {
> > >  			igt_dynamic_f("%s", r->name)
> > >  				gem_execbuf_stress_subtest(rounds,
> > WAIT_STATUS, &r->ci);
> > >  			igt_dynamic_f("%s-%s", "extra-wait", r->name)
> > >  				gem_execbuf_stress_subtest(rounds,
> > WAIT_STATUS | WAIT_EXTRA,
> > > &r->ci);
> > > +
> > > +			if (r->ci.memory_class ==
> > I915_MEMORY_CLASS_SYSTEM)
> > > +				continue;
> > > +
> > > +			lmem_threshold =
> > get_d3cold_sr_lmem_threshold(debugfs);
> > 
> > since we have just one global threashold and not one per memory region we can
> > avoid the extra reads in here by moving it ouside.
> > 
> > > +			igt_dynamic_f("%s-%s", "d3cold-off", r->name) {
> > > +				set_d3cold_sr_lmem_threshold(debugfs, 300);
> > > +				gem_execbuf_stress_subtest(rounds,
> > WAIT_STATUS | WAIT_D3COLD,
> > > +							   &r->ci);
> > > +				set_d3cold_sr_lmem_threshold(debugfs,
> > lmem_threshold);
> > 
> > but also we should probably do the save/restore with igt_fixture?
> We can do once we have this param in kernel , otherwise CI will break for all i915_pm_rpm tests. 

Does this mean LTS kernels are expected to be broken?

-- 
Petri Latvala

Re: [igt-dev] [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test with D3Cold

[Rodrigo Vivi]

On Fri, Apr 29, 2022 at 09:10:15AM -0400, Gupta, Anshuman wrote:
> 
> 
> > -----Original Message-----
> > From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > Sent: Friday, April 29, 2022 6:37 PM
> > To: Gupta, Anshuman <anshuman.gupta@intel.com>
> > Cc: igt-dev@lists.freedesktop.org; kamil.konieczny@linux.intel.com; Latvala,
> > Petri <petri.latvala@intel.com>; Nilawar, Badal <badal.nilawar@intel.com>;
> > Ewins, Jon <jon.ewins@intel.com>
> > Subject: Re: [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test
> > with D3Cold
> > 
> > On Fri, Apr 29, 2022 at 02:24:23PM +0530, Anshuman Gupta wrote:
> > > Added d3cold dynamic subtest to gem_exec_stress with device class
> > > memory region. It test both D3Cold-{VRAM_SR, Off} by using
> > > d3cold_sr_lmem_threshold i915_params debugfs.
> > >
> > > v2:
> > > - Two different subtest d3cold-off and d3cold-vrsr. [Rodrigo]
> > > - commit header modification.
> > >
> > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
> > > ---
> > >  tests/i915/i915_pm_rpm.c | 97
> > > ++++++++++++++++++++++++++++++++++------
> > >  1 file changed, 83 insertions(+), 14 deletions(-)
> > >
> > > diff --git a/tests/i915/i915_pm_rpm.c b/tests/i915/i915_pm_rpm.c index
> > > 31f9712a8..1ef76f562 100644
> > > --- a/tests/i915/i915_pm_rpm.c
> > > +++ b/tests/i915/i915_pm_rpm.c
> > > @@ -85,6 +85,7 @@ enum plane_type {
> > >  #define WAIT_PC8_RES	2
> > >  #define WAIT_EXTRA	4
> > >  #define USE_DPMS	8
> > > +#define WAIT_D3COLD	16
> > >
> > >  int drm_fd, msr_fd, pc8_status_fd;
> > >  int debugfs;
> > > @@ -215,6 +216,21 @@ static bool wait_for_suspended(void)
> > >  	}
> > >  }
> > >
> > > +static bool wait_for_d3cold(struct pci_device *root) {
> > > +	bool d3colded;
> > > +
> > > +	d3colded = igt_wait(igt_pm_get_acpi_real_d_state(root) ==
> > > +IGT_ACPI_D3Cold, 10000, 500);
> > > +
> > > +	if (!d3colded) {
> > > +		igt_info("D3Cold not achieved for root port
> > %04x:%02x:%02x.%01x\n",
> > > +			 root->domain, root->bus, root->dev, root->func);
> > > +		igt_pm_print_pci_card_runtime_status();
> > > +	}
> > > +
> > > +	return d3colded;
> > > +}
> > > +
> > >  static bool wait_for_active(void)
> > >  {
> > >  	if (has_pc8 && !has_runtime_pm)
> > > @@ -744,6 +760,30 @@ static void test_i2c(struct mode_set_data *data)
> > >  			"There is an EDID mismatch between i2c and DRM!\n");
> > }
> > >
> > > +static int get_d3cold_sr_lmem_threshold(int dir) {
> > > +	int param_dir, lmem_threshold, ret;
> > > +
> > > +	param_dir = openat(dir, "i915_params", O_RDONLY);
> > > +	ret = igt_sysfs_scanf(param_dir, "d3cold_sr_lmem_threshold", "%d",
> > &lmem_threshold);
> > > +	igt_assert(ret >= 0);
> > > +
> > > +	close(param_dir);
> > > +	return lmem_threshold;
> > > +}
> > > +
> > > +static int set_d3cold_sr_lmem_threshold(int dir, int val) {
> > > +	int param_dir, ret;
> > > +
> > > +	param_dir = openat(dir, "i915_params", O_RDONLY);
> > > +	ret = igt_sysfs_printf(param_dir, "d3cold_sr_lmem_threshold", "%d",
> > val);
> > > +	igt_assert(ret > 0);
> > > +
> > > +	close(param_dir);
> > > +	return ret;
> > > +}
> > > +
> > >  static void setup_pc8(void)
> > >  {
> > >  	has_pc8 = false;
> > > @@ -1069,6 +1109,17 @@ static void debugfs_forcewake_user_subtest(void)
> > >  	igt_assert(wait_for_suspended());
> > >  }
> > >
> > > +static struct pci_device *setup_d3cold_and_get_root_port(void)
> > > +{
> > > +	struct pci_device *root;
> > > +
> > > +	root = igt_device_get_pci_root_port(drm_fd);
> > > +	igt_require(igt_pm_acpi_d3cold_supported(root));
> > > +	igt_pm_setup_pci_card_runtime_pm(root);
> > > +
> > > +	return root;
> > > +}
> > > +
> > >  static void gem_mmap_args(const struct mmap_offset *t,
> > >  			  struct drm_i915_gem_memory_class_instance
> > *mem_regions)  { @@
> > > -1392,6 +1443,7 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
> > >  	int i;
> > >  	int batch_size = 4 * sizeof(uint32_t);
> > >  	uint32_t batch_buf[batch_size];
> > > +	struct pci_device *root;
> > >  	uint32_t handle;
> > >  	struct drm_i915_gem_execbuffer2 execbuf = {};
> > >  	struct drm_i915_gem_exec_object2 objs[1] = {{}}; @@ -1408,6 +1460,9
> > > @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
> > >  	batch_buf[i++] = MI_NOOP;
> > >  	igt_assert(i * sizeof(uint32_t) == batch_size);
> > >
> > > +	if (wait_flags & WAIT_D3COLD)
> > > +		root = setup_d3cold_and_get_root_port();
> > > +
> > >  	disable_all_screens_and_wait(&ms_data);
> > >
> > >  	/* PC8 test is only applicable to igfx  */ @@ -1433,6 +1488,9 @@
> > > gem_execbuf_stress_subtest(int rounds, int wait_flags,
> > >  			/* clean up idle work */
> > >  			igt_drop_caches_set(drm_fd, DROP_IDLE);
> > >  			igt_assert(wait_for_suspended());
> > > +			if (wait_flags & WAIT_D3COLD)
> > > +				igt_assert(wait_for_d3cold(root));
> > > +
> > >  		}
> > >  		if (wait_flags & WAIT_PC8_RES)
> > >  			igt_assert(pc8_plus_residency_changed(30));
> > > @@ -1440,6 +1498,9 @@ gem_execbuf_stress_subtest(int rounds, int
> > wait_flags,
> > >  			sleep(5);
> > >  	}
> > >
> > > +	if (wait_flags & WAIT_D3COLD)
> > > +		igt_pm_restore_pci_card_runtime_pm();
> > > +
> > >  	gem_close(drm_fd, handle);
> > >  }
> > >
> > > @@ -1537,22 +1598,10 @@ __noreturn static void stay_subtest(void)
> > > static void d3cold_basic_subtest(void)  {
> > >  	struct pci_device *root;
> > > -	bool result;
> > >
> > > -	root = igt_device_get_pci_root_port(drm_fd);
> > > -	igt_require(igt_pm_acpi_d3cold_supported(root));
> > > -	igt_pm_setup_pci_card_runtime_pm(root);
> > > +	root = setup_d3cold_and_get_root_port();
> > >  	disable_all_screens_and_wait(&ms_data);
> > > -
> > > -	result = igt_wait(igt_pm_get_acpi_real_d_state(root) ==
> > IGT_ACPI_D3Cold, 10000, 500);
> > > -
> > > -	if (!result) {
> > > -		igt_info("D3Cold not achieved for root port
> > %04x:%02x:%02x.%01x\n",
> > > -			 root->domain, root->bus, root->dev, root->func);
> > > -		igt_pm_print_pci_card_runtime_status();
> > > -	}
> > > -
> > > -	igt_assert(result);
> > > +	igt_assert(wait_for_d3cold(root));
> > >  	igt_pm_restore_pci_card_runtime_pm();
> > >  }
> > >
> > > @@ -2206,11 +2255,31 @@ igt_main_args("", long_options, help_str,
> > opt_handler, NULL)
> > >  	igt_describe("Validate execbuf submission while exercising rpm "
> > >  		     "suspend/resume cycles.");
> > >  	igt_subtest_with_dynamic("gem-execbuf-stress") {
> > > +		int lmem_threshold;
> > > +
> > >  		for_each_memory_region(r, drm_fd) {
> > >  			igt_dynamic_f("%s", r->name)
> > >  				gem_execbuf_stress_subtest(rounds,
> > WAIT_STATUS, &r->ci);
> > >  			igt_dynamic_f("%s-%s", "extra-wait", r->name)
> > >  				gem_execbuf_stress_subtest(rounds,
> > WAIT_STATUS | WAIT_EXTRA,
> > > &r->ci);
> > > +
> > > +			if (r->ci.memory_class ==
> > I915_MEMORY_CLASS_SYSTEM)
> > > +				continue;
> > > +
> > > +			lmem_threshold =
> > get_d3cold_sr_lmem_threshold(debugfs);
> > 
> > since we have just one global threashold and not one per memory region we can
> > avoid the extra reads in here by moving it ouside.
> > 
> > > +			igt_dynamic_f("%s-%s", "d3cold-off", r->name) {
> > > +				set_d3cold_sr_lmem_threshold(debugfs, 300);
> > > +				gem_execbuf_stress_subtest(rounds,
> > WAIT_STATUS | WAIT_D3COLD,
> > > +							   &r->ci);
> > > +				set_d3cold_sr_lmem_threshold(debugfs,
> > lmem_threshold);
> > 
> > but also we should probably do the save/restore with igt_fixture?
> We can do once we have this param in kernel , otherwise CI will break for all i915_pm_rpm tests.

I don't get it... why?
Why can't we get_d3cold_sr_lmem_threshold(debugfs); at setup_environment()
and set_d3cold_sr_lmem_threshold(debugfs, lmem_threshold) at teardown_environment()
?

> Thanks,
> Anshuman Gupta.
> > 
> > > +			}
> > > +
> > > +			igt_dynamic_f("%s-%s", "d3cold-vrsr", r->name) {
> > > +				set_d3cold_sr_lmem_threshold(debugfs, 0);
> > > +				gem_execbuf_stress_subtest(rounds,
> > WAIT_STATUS | WAIT_D3COLD,
> > > +							   &r->ci);
> > > +				set_d3cold_sr_lmem_threshold(debugfs,
> > lmem_threshold);
> > > +			}
> > >  		}
> > >  	}
> > >
> > > --
> > > 2.26.2
> > >

Re: [igt-dev] [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test with D3Cold

[Gupta, Anshuman]

> -----Original Message-----
> From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> Sent: Friday, April 29, 2022 7:13 PM
> To: Gupta, Anshuman <anshuman.gupta@intel.com>
> Cc: igt-dev@lists.freedesktop.org; kamil.konieczny@linux.intel.com; Latvala,
> Petri <petri.latvala@intel.com>; Nilawar, Badal <badal.nilawar@intel.com>;
> Ewins, Jon <jon.ewins@intel.com>
> Subject: Re: [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test
> with D3Cold
> 
> On Fri, Apr 29, 2022 at 09:10:15AM -0400, Gupta, Anshuman wrote:
> >
> >
> > > -----Original Message-----
> > > From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > > Sent: Friday, April 29, 2022 6:37 PM
> > > To: Gupta, Anshuman <anshuman.gupta@intel.com>
> > > Cc: igt-dev@lists.freedesktop.org; kamil.konieczny@linux.intel.com;
> > > Latvala, Petri <petri.latvala@intel.com>; Nilawar, Badal
> > > <badal.nilawar@intel.com>; Ewins, Jon <jon.ewins@intel.com>
> > > Subject: Re: [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend
> > > gem_exec_stress test with D3Cold
> > >
> > > On Fri, Apr 29, 2022 at 02:24:23PM +0530, Anshuman Gupta wrote:
> > > > Added d3cold dynamic subtest to gem_exec_stress with device class
> > > > memory region. It test both D3Cold-{VRAM_SR, Off} by using
> > > > d3cold_sr_lmem_threshold i915_params debugfs.
> > > >
> > > > v2:
> > > > - Two different subtest d3cold-off and d3cold-vrsr. [Rodrigo]
> > > > - commit header modification.
> > > >
> > > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
> > > > ---
> > > >  tests/i915/i915_pm_rpm.c | 97
> > > > ++++++++++++++++++++++++++++++++++------
> > > >  1 file changed, 83 insertions(+), 14 deletions(-)
> > > >
> > > > diff --git a/tests/i915/i915_pm_rpm.c b/tests/i915/i915_pm_rpm.c
> > > > index
> > > > 31f9712a8..1ef76f562 100644
> > > > --- a/tests/i915/i915_pm_rpm.c
> > > > +++ b/tests/i915/i915_pm_rpm.c
> > > > @@ -85,6 +85,7 @@ enum plane_type {
> > > >  #define WAIT_PC8_RES	2
> > > >  #define WAIT_EXTRA	4
> > > >  #define USE_DPMS	8
> > > > +#define WAIT_D3COLD	16
> > > >
> > > >  int drm_fd, msr_fd, pc8_status_fd;  int debugfs; @@ -215,6
> > > > +216,21 @@ static bool wait_for_suspended(void)
> > > >  	}
> > > >  }
> > > >
> > > > +static bool wait_for_d3cold(struct pci_device *root) {
> > > > +	bool d3colded;
> > > > +
> > > > +	d3colded = igt_wait(igt_pm_get_acpi_real_d_state(root) ==
> > > > +IGT_ACPI_D3Cold, 10000, 500);
> > > > +
> > > > +	if (!d3colded) {
> > > > +		igt_info("D3Cold not achieved for root port
> > > %04x:%02x:%02x.%01x\n",
> > > > +			 root->domain, root->bus, root->dev, root->func);
> > > > +		igt_pm_print_pci_card_runtime_status();
> > > > +	}
> > > > +
> > > > +	return d3colded;
> > > > +}
> > > > +
> > > >  static bool wait_for_active(void)  {
> > > >  	if (has_pc8 && !has_runtime_pm)
> > > > @@ -744,6 +760,30 @@ static void test_i2c(struct mode_set_data *data)
> > > >  			"There is an EDID mismatch between i2c and DRM!\n");
> > > }
> > > >
> > > > +static int get_d3cold_sr_lmem_threshold(int dir) {
> > > > +	int param_dir, lmem_threshold, ret;
> > > > +
> > > > +	param_dir = openat(dir, "i915_params", O_RDONLY);
> > > > +	ret = igt_sysfs_scanf(param_dir, "d3cold_sr_lmem_threshold", "%d",
> > > &lmem_threshold);
> > > > +	igt_assert(ret >= 0);
> > > > +
> > > > +	close(param_dir);
> > > > +	return lmem_threshold;
> > > > +}
> > > > +
> > > > +static int set_d3cold_sr_lmem_threshold(int dir, int val) {
> > > > +	int param_dir, ret;
> > > > +
> > > > +	param_dir = openat(dir, "i915_params", O_RDONLY);
> > > > +	ret = igt_sysfs_printf(param_dir, "d3cold_sr_lmem_threshold", "%d",
> > > val);
> > > > +	igt_assert(ret > 0);
> > > > +
> > > > +	close(param_dir);
> > > > +	return ret;
> > > > +}
> > > > +
> > > >  static void setup_pc8(void)
> > > >  {
> > > >  	has_pc8 = false;
> > > > @@ -1069,6 +1109,17 @@ static void
> debugfs_forcewake_user_subtest(void)
> > > >  	igt_assert(wait_for_suspended());
> > > >  }
> > > >
> > > > +static struct pci_device *setup_d3cold_and_get_root_port(void)
> > > > +{
> > > > +	struct pci_device *root;
> > > > +
> > > > +	root = igt_device_get_pci_root_port(drm_fd);
> > > > +	igt_require(igt_pm_acpi_d3cold_supported(root));
> > > > +	igt_pm_setup_pci_card_runtime_pm(root);
> > > > +
> > > > +	return root;
> > > > +}
> > > > +
> > > >  static void gem_mmap_args(const struct mmap_offset *t,
> > > >  			  struct drm_i915_gem_memory_class_instance
> > > *mem_regions)  { @@
> > > > -1392,6 +1443,7 @@ gem_execbuf_stress_subtest(int rounds, int
> wait_flags,
> > > >  	int i;
> > > >  	int batch_size = 4 * sizeof(uint32_t);
> > > >  	uint32_t batch_buf[batch_size];
> > > > +	struct pci_device *root;
> > > >  	uint32_t handle;
> > > >  	struct drm_i915_gem_execbuffer2 execbuf = {};
> > > >  	struct drm_i915_gem_exec_object2 objs[1] = {{}}; @@ -1408,6 +1460,9
> > > > @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
> > > >  	batch_buf[i++] = MI_NOOP;
> > > >  	igt_assert(i * sizeof(uint32_t) == batch_size);
> > > >
> > > > +	if (wait_flags & WAIT_D3COLD)
> > > > +		root = setup_d3cold_and_get_root_port();
> > > > +
> > > >  	disable_all_screens_and_wait(&ms_data);
> > > >
> > > >  	/* PC8 test is only applicable to igfx  */ @@ -1433,6 +1488,9 @@
> > > > gem_execbuf_stress_subtest(int rounds, int wait_flags,
> > > >  			/* clean up idle work */
> > > >  			igt_drop_caches_set(drm_fd, DROP_IDLE);
> > > >  			igt_assert(wait_for_suspended());
> > > > +			if (wait_flags & WAIT_D3COLD)
> > > > +				igt_assert(wait_for_d3cold(root));
> > > > +
> > > >  		}
> > > >  		if (wait_flags & WAIT_PC8_RES)
> > > >  			igt_assert(pc8_plus_residency_changed(30));
> > > > @@ -1440,6 +1498,9 @@ gem_execbuf_stress_subtest(int rounds, int
> > > wait_flags,
> > > >  			sleep(5);
> > > >  	}
> > > >
> > > > +	if (wait_flags & WAIT_D3COLD)
> > > > +		igt_pm_restore_pci_card_runtime_pm();
> > > > +
> > > >  	gem_close(drm_fd, handle);
> > > >  }
> > > >
> > > > @@ -1537,22 +1598,10 @@ __noreturn static void stay_subtest(void)
> > > > static void d3cold_basic_subtest(void)  {
> > > >  	struct pci_device *root;
> > > > -	bool result;
> > > >
> > > > -	root = igt_device_get_pci_root_port(drm_fd);
> > > > -	igt_require(igt_pm_acpi_d3cold_supported(root));
> > > > -	igt_pm_setup_pci_card_runtime_pm(root);
> > > > +	root = setup_d3cold_and_get_root_port();
> > > >  	disable_all_screens_and_wait(&ms_data);
> > > > -
> > > > -	result = igt_wait(igt_pm_get_acpi_real_d_state(root) ==
> > > IGT_ACPI_D3Cold, 10000, 500);
> > > > -
> > > > -	if (!result) {
> > > > -		igt_info("D3Cold not achieved for root port
> > > %04x:%02x:%02x.%01x\n",
> > > > -			 root->domain, root->bus, root->dev, root->func);
> > > > -		igt_pm_print_pci_card_runtime_status();
> > > > -	}
> > > > -
> > > > -	igt_assert(result);
> > > > +	igt_assert(wait_for_d3cold(root));
> > > >  	igt_pm_restore_pci_card_runtime_pm();
> > > >  }
> > > >
> > > > @@ -2206,11 +2255,31 @@ igt_main_args("", long_options, help_str,
> > > opt_handler, NULL)
> > > >  	igt_describe("Validate execbuf submission while exercising rpm "
> > > >  		     "suspend/resume cycles.");
> > > >  	igt_subtest_with_dynamic("gem-execbuf-stress") {
> > > > +		int lmem_threshold;
> > > > +
> > > >  		for_each_memory_region(r, drm_fd) {
> > > >  			igt_dynamic_f("%s", r->name)
> > > >  				gem_execbuf_stress_subtest(rounds,
> > > WAIT_STATUS, &r->ci);
> > > >  			igt_dynamic_f("%s-%s", "extra-wait", r->name)
> > > >  				gem_execbuf_stress_subtest(rounds,
> > > WAIT_STATUS | WAIT_EXTRA,
> > > > &r->ci);
> > > > +
> > > > +			if (r->ci.memory_class ==
> > > I915_MEMORY_CLASS_SYSTEM)
> > > > +				continue;
> > > > +
> > > > +			lmem_threshold =
> > > get_d3cold_sr_lmem_threshold(debugfs);
> > >
> > > since we have just one global threashold and not one per memory region we
> can
> > > avoid the extra reads in here by moving it ouside.
> > >
> > > > +			igt_dynamic_f("%s-%s", "d3cold-off", r->name) {
> > > > +				set_d3cold_sr_lmem_threshold(debugfs, 300);
> > > > +				gem_execbuf_stress_subtest(rounds,
> > > WAIT_STATUS | WAIT_D3COLD,
> > > > +							   &r->ci);
> > > > +				set_d3cold_sr_lmem_threshold(debugfs,
> > > lmem_threshold);
> > >
> > > but also we should probably do the save/restore with igt_fixture?
> > We can do once we have this param in kernel , otherwise CI will break for all
> i915_pm_rpm tests.
> 
> I don't get it... why?
> Why can't we get_d3cold_sr_lmem_threshold(debugfs); at setup_environment()
> and set_d3cold_sr_lmem_threshold(debugfs, lmem_threshold) at
> teardown_environment()
We have a assertion here on ret. 
ret = igt_sysfs_scanf(param_dir, "d3cold_sr_lmem_threshold", "%d", &lmem_threshold);
igt_assert(ret >= 0);
so this will fail , until underlying kernel implement the d3cold_sr_lmem_threshold  module_param.
I need to remove igt_assertion from above to to keep {get,set}_ get_d3cold_sr_lmem_threshold()
in igt_fixture().

Thanks,
Anshuman Gupta.
> ?
> 
> > Thanks,
> > Anshuman Gupta.
> > >
> > > > +			}
> > > > +
> > > > +			igt_dynamic_f("%s-%s", "d3cold-vrsr", r->name) {
> > > > +				set_d3cold_sr_lmem_threshold(debugfs, 0);
> > > > +				gem_execbuf_stress_subtest(rounds,
> > > WAIT_STATUS | WAIT_D3COLD,
> > > > +							   &r->ci);
> > > > +				set_d3cold_sr_lmem_threshold(debugfs,
> > > lmem_threshold);
> > > > +			}
> > > >  		}
> > > >  	}
> > > >
> > > > --
> > > > 2.26.2
> > > >

Re: [igt-dev] [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test with D3Cold

[Rodrigo Vivi]

On Fri, Apr 29, 2022 at 10:57:25AM -0400, Gupta, Anshuman wrote:
> 
> 
> > -----Original Message-----
> > From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > Sent: Friday, April 29, 2022 7:13 PM
> > To: Gupta, Anshuman <anshuman.gupta@intel.com>
> > Cc: igt-dev@lists.freedesktop.org; kamil.konieczny@linux.intel.com; Latvala,
> > Petri <petri.latvala@intel.com>; Nilawar, Badal <badal.nilawar@intel.com>;
> > Ewins, Jon <jon.ewins@intel.com>
> > Subject: Re: [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test
> > with D3Cold
> > 
> > On Fri, Apr 29, 2022 at 09:10:15AM -0400, Gupta, Anshuman wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > > > Sent: Friday, April 29, 2022 6:37 PM
> > > > To: Gupta, Anshuman <anshuman.gupta@intel.com>
> > > > Cc: igt-dev@lists.freedesktop.org; kamil.konieczny@linux.intel.com;
> > > > Latvala, Petri <petri.latvala@intel.com>; Nilawar, Badal
> > > > <badal.nilawar@intel.com>; Ewins, Jon <jon.ewins@intel.com>
> > > > Subject: Re: [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend
> > > > gem_exec_stress test with D3Cold
> > > >
> > > > On Fri, Apr 29, 2022 at 02:24:23PM +0530, Anshuman Gupta wrote:
> > > > > Added d3cold dynamic subtest to gem_exec_stress with device class
> > > > > memory region. It test both D3Cold-{VRAM_SR, Off} by using
> > > > > d3cold_sr_lmem_threshold i915_params debugfs.
> > > > >
> > > > > v2:
> > > > > - Two different subtest d3cold-off and d3cold-vrsr. [Rodrigo]
> > > > > - commit header modification.
> > > > >
> > > > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > > Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
> > > > > ---
> > > > >  tests/i915/i915_pm_rpm.c | 97
> > > > > ++++++++++++++++++++++++++++++++++------
> > > > >  1 file changed, 83 insertions(+), 14 deletions(-)
> > > > >
> > > > > diff --git a/tests/i915/i915_pm_rpm.c b/tests/i915/i915_pm_rpm.c
> > > > > index
> > > > > 31f9712a8..1ef76f562 100644
> > > > > --- a/tests/i915/i915_pm_rpm.c
> > > > > +++ b/tests/i915/i915_pm_rpm.c
> > > > > @@ -85,6 +85,7 @@ enum plane_type {
> > > > >  #define WAIT_PC8_RES	2
> > > > >  #define WAIT_EXTRA	4
> > > > >  #define USE_DPMS	8
> > > > > +#define WAIT_D3COLD	16
> > > > >
> > > > >  int drm_fd, msr_fd, pc8_status_fd;  int debugfs; @@ -215,6
> > > > > +216,21 @@ static bool wait_for_suspended(void)
> > > > >  	}
> > > > >  }
> > > > >
> > > > > +static bool wait_for_d3cold(struct pci_device *root) {
> > > > > +	bool d3colded;
> > > > > +
> > > > > +	d3colded = igt_wait(igt_pm_get_acpi_real_d_state(root) ==
> > > > > +IGT_ACPI_D3Cold, 10000, 500);
> > > > > +
> > > > > +	if (!d3colded) {
> > > > > +		igt_info("D3Cold not achieved for root port
> > > > %04x:%02x:%02x.%01x\n",
> > > > > +			 root->domain, root->bus, root->dev, root->func);
> > > > > +		igt_pm_print_pci_card_runtime_status();
> > > > > +	}
> > > > > +
> > > > > +	return d3colded;
> > > > > +}
> > > > > +
> > > > >  static bool wait_for_active(void)  {
> > > > >  	if (has_pc8 && !has_runtime_pm)
> > > > > @@ -744,6 +760,30 @@ static void test_i2c(struct mode_set_data *data)
> > > > >  			"There is an EDID mismatch between i2c and DRM!\n");
> > > > }
> > > > >
> > > > > +static int get_d3cold_sr_lmem_threshold(int dir) {
> > > > > +	int param_dir, lmem_threshold, ret;
> > > > > +
> > > > > +	param_dir = openat(dir, "i915_params", O_RDONLY);
> > > > > +	ret = igt_sysfs_scanf(param_dir, "d3cold_sr_lmem_threshold", "%d",
> > > > &lmem_threshold);
> > > > > +	igt_assert(ret >= 0);
> > > > > +
> > > > > +	close(param_dir);
> > > > > +	return lmem_threshold;
> > > > > +}
> > > > > +
> > > > > +static int set_d3cold_sr_lmem_threshold(int dir, int val) {
> > > > > +	int param_dir, ret;
> > > > > +
> > > > > +	param_dir = openat(dir, "i915_params", O_RDONLY);
> > > > > +	ret = igt_sysfs_printf(param_dir, "d3cold_sr_lmem_threshold", "%d",
> > > > val);
> > > > > +	igt_assert(ret > 0);
> > > > > +
> > > > > +	close(param_dir);
> > > > > +	return ret;
> > > > > +}
> > > > > +
> > > > >  static void setup_pc8(void)
> > > > >  {
> > > > >  	has_pc8 = false;
> > > > > @@ -1069,6 +1109,17 @@ static void
> > debugfs_forcewake_user_subtest(void)
> > > > >  	igt_assert(wait_for_suspended());
> > > > >  }
> > > > >
> > > > > +static struct pci_device *setup_d3cold_and_get_root_port(void)
> > > > > +{
> > > > > +	struct pci_device *root;
> > > > > +
> > > > > +	root = igt_device_get_pci_root_port(drm_fd);
> > > > > +	igt_require(igt_pm_acpi_d3cold_supported(root));
> > > > > +	igt_pm_setup_pci_card_runtime_pm(root);
> > > > > +
> > > > > +	return root;
> > > > > +}
> > > > > +
> > > > >  static void gem_mmap_args(const struct mmap_offset *t,
> > > > >  			  struct drm_i915_gem_memory_class_instance
> > > > *mem_regions)  { @@
> > > > > -1392,6 +1443,7 @@ gem_execbuf_stress_subtest(int rounds, int
> > wait_flags,
> > > > >  	int i;
> > > > >  	int batch_size = 4 * sizeof(uint32_t);
> > > > >  	uint32_t batch_buf[batch_size];
> > > > > +	struct pci_device *root;
> > > > >  	uint32_t handle;
> > > > >  	struct drm_i915_gem_execbuffer2 execbuf = {};
> > > > >  	struct drm_i915_gem_exec_object2 objs[1] = {{}}; @@ -1408,6 +1460,9
> > > > > @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
> > > > >  	batch_buf[i++] = MI_NOOP;
> > > > >  	igt_assert(i * sizeof(uint32_t) == batch_size);
> > > > >
> > > > > +	if (wait_flags & WAIT_D3COLD)
> > > > > +		root = setup_d3cold_and_get_root_port();
> > > > > +
> > > > >  	disable_all_screens_and_wait(&ms_data);
> > > > >
> > > > >  	/* PC8 test is only applicable to igfx  */ @@ -1433,6 +1488,9 @@
> > > > > gem_execbuf_stress_subtest(int rounds, int wait_flags,
> > > > >  			/* clean up idle work */
> > > > >  			igt_drop_caches_set(drm_fd, DROP_IDLE);
> > > > >  			igt_assert(wait_for_suspended());
> > > > > +			if (wait_flags & WAIT_D3COLD)
> > > > > +				igt_assert(wait_for_d3cold(root));
> > > > > +
> > > > >  		}
> > > > >  		if (wait_flags & WAIT_PC8_RES)
> > > > >  			igt_assert(pc8_plus_residency_changed(30));
> > > > > @@ -1440,6 +1498,9 @@ gem_execbuf_stress_subtest(int rounds, int
> > > > wait_flags,
> > > > >  			sleep(5);
> > > > >  	}
> > > > >
> > > > > +	if (wait_flags & WAIT_D3COLD)
> > > > > +		igt_pm_restore_pci_card_runtime_pm();
> > > > > +
> > > > >  	gem_close(drm_fd, handle);
> > > > >  }
> > > > >
> > > > > @@ -1537,22 +1598,10 @@ __noreturn static void stay_subtest(void)
> > > > > static void d3cold_basic_subtest(void)  {
> > > > >  	struct pci_device *root;
> > > > > -	bool result;
> > > > >
> > > > > -	root = igt_device_get_pci_root_port(drm_fd);
> > > > > -	igt_require(igt_pm_acpi_d3cold_supported(root));
> > > > > -	igt_pm_setup_pci_card_runtime_pm(root);
> > > > > +	root = setup_d3cold_and_get_root_port();
> > > > >  	disable_all_screens_and_wait(&ms_data);
> > > > > -
> > > > > -	result = igt_wait(igt_pm_get_acpi_real_d_state(root) ==
> > > > IGT_ACPI_D3Cold, 10000, 500);
> > > > > -
> > > > > -	if (!result) {
> > > > > -		igt_info("D3Cold not achieved for root port
> > > > %04x:%02x:%02x.%01x\n",
> > > > > -			 root->domain, root->bus, root->dev, root->func);
> > > > > -		igt_pm_print_pci_card_runtime_status();
> > > > > -	}
> > > > > -
> > > > > -	igt_assert(result);
> > > > > +	igt_assert(wait_for_d3cold(root));
> > > > >  	igt_pm_restore_pci_card_runtime_pm();
> > > > >  }
> > > > >
> > > > > @@ -2206,11 +2255,31 @@ igt_main_args("", long_options, help_str,
> > > > opt_handler, NULL)
> > > > >  	igt_describe("Validate execbuf submission while exercising rpm "
> > > > >  		     "suspend/resume cycles.");
> > > > >  	igt_subtest_with_dynamic("gem-execbuf-stress") {
> > > > > +		int lmem_threshold;
> > > > > +
> > > > >  		for_each_memory_region(r, drm_fd) {
> > > > >  			igt_dynamic_f("%s", r->name)
> > > > >  				gem_execbuf_stress_subtest(rounds,
> > > > WAIT_STATUS, &r->ci);
> > > > >  			igt_dynamic_f("%s-%s", "extra-wait", r->name)
> > > > >  				gem_execbuf_stress_subtest(rounds,
> > > > WAIT_STATUS | WAIT_EXTRA,
> > > > > &r->ci);
> > > > > +
> > > > > +			if (r->ci.memory_class ==
> > > > I915_MEMORY_CLASS_SYSTEM)
> > > > > +				continue;
> > > > > +
> > > > > +			lmem_threshold =
> > > > get_d3cold_sr_lmem_threshold(debugfs);
> > > >
> > > > since we have just one global threashold and not one per memory region we
> > can
> > > > avoid the extra reads in here by moving it ouside.
> > > >
> > > > > +			igt_dynamic_f("%s-%s", "d3cold-off", r->name) {
> > > > > +				set_d3cold_sr_lmem_threshold(debugfs, 300);
> > > > > +				gem_execbuf_stress_subtest(rounds,
> > > > WAIT_STATUS | WAIT_D3COLD,
> > > > > +							   &r->ci);
> > > > > +				set_d3cold_sr_lmem_threshold(debugfs,
> > > > lmem_threshold);
> > > >
> > > > but also we should probably do the save/restore with igt_fixture?
> > > We can do once we have this param in kernel , otherwise CI will break for all
> > i915_pm_rpm tests.
> > 
> > I don't get it... why?
> > Why can't we get_d3cold_sr_lmem_threshold(debugfs); at setup_environment()
> > and set_d3cold_sr_lmem_threshold(debugfs, lmem_threshold) at
> > teardown_environment()
> We have a assertion here on ret. 
> ret = igt_sysfs_scanf(param_dir, "d3cold_sr_lmem_threshold", "%d", &lmem_threshold);

A few things I just noticed:

1. (probably not related to this patch)
sysfs? can we use these igt_sysfs funct for the debugfs?
if so we could probably rename them...

2. param_dir is not a good name. that's the file pathname already, not dir.

> igt_assert(ret >= 0);
> so this will fail , until underlying kernel implement the d3cold_sr_lmem_threshold  module_param.

if the underlying kernel doesn't have the debug fs won't this igt_assert fail as well?

why would it be different with module_param?

Petri's question is pertinent, won't this break in LTS anyway?

> I need to remove igt_assertion from above to to keep {get,set}_ get_d3cold_sr_lmem_threshold()
> in igt_fixture().

don't we need to remove it anyway?
in the read inside the loop aren't you breaking the execution of the following memory region subtests?

> 
> Thanks,
> Anshuman Gupta.
> > ?
> > 
> > > Thanks,
> > > Anshuman Gupta.
> > > >
> > > > > +			}
> > > > > +
> > > > > +			igt_dynamic_f("%s-%s", "d3cold-vrsr", r->name) {
> > > > > +				set_d3cold_sr_lmem_threshold(debugfs, 0);
> > > > > +				gem_execbuf_stress_subtest(rounds,
> > > > WAIT_STATUS | WAIT_D3COLD,
> > > > > +							   &r->ci);
> > > > > +				set_d3cold_sr_lmem_threshold(debugfs,
> > > > lmem_threshold);
> > > > > +			}
> > > > >  		}
> > > > >  	}
> > > > >
> > > > > --
> > > > > 2.26.2
> > > > >

Re: [igt-dev] [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test with D3Cold

[Gupta, Anshuman]

> -----Original Message-----
> From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> Sent: Friday, April 29, 2022 9:31 PM
> To: Gupta, Anshuman <anshuman.gupta@intel.com>
> Cc: igt-dev@lists.freedesktop.org; kamil.konieczny@linux.intel.com; Latvala,
> Petri <petri.latvala@intel.com>; Nilawar, Badal <badal.nilawar@intel.com>;
> Ewins, Jon <jon.ewins@intel.com>
> Subject: Re: [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend gem_exec_stress test
> with D3Cold
> 
> On Fri, Apr 29, 2022 at 10:57:25AM -0400, Gupta, Anshuman wrote:
> >
> >
> > > -----Original Message-----
> > > From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > > Sent: Friday, April 29, 2022 7:13 PM
> > > To: Gupta, Anshuman <anshuman.gupta@intel.com>
> > > Cc: igt-dev@lists.freedesktop.org; kamil.konieczny@linux.intel.com;
> > > Latvala, Petri <petri.latvala@intel.com>; Nilawar, Badal
> > > <badal.nilawar@intel.com>; Ewins, Jon <jon.ewins@intel.com>
> > > Subject: Re: [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend
> > > gem_exec_stress test with D3Cold
> > >
> > > On Fri, Apr 29, 2022 at 09:10:15AM -0400, Gupta, Anshuman wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > > > > Sent: Friday, April 29, 2022 6:37 PM
> > > > > To: Gupta, Anshuman <anshuman.gupta@intel.com>
> > > > > Cc: igt-dev@lists.freedesktop.org;
> > > > > kamil.konieczny@linux.intel.com; Latvala, Petri
> > > > > <petri.latvala@intel.com>; Nilawar, Badal
> > > > > <badal.nilawar@intel.com>; Ewins, Jon <jon.ewins@intel.com>
> > > > > Subject: Re: [PATCH i-g-t v2 6/8] i915_pm_rpm: Extend
> > > > > gem_exec_stress test with D3Cold
> > > > >
> > > > > On Fri, Apr 29, 2022 at 02:24:23PM +0530, Anshuman Gupta wrote:
> > > > > > Added d3cold dynamic subtest to gem_exec_stress with device
> > > > > > class memory region. It test both D3Cold-{VRAM_SR, Off} by
> > > > > > using d3cold_sr_lmem_threshold i915_params debugfs.
> > > > > >
> > > > > > v2:
> > > > > > - Two different subtest d3cold-off and d3cold-vrsr. [Rodrigo]
> > > > > > - commit header modification.
> > > > > >
> > > > > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > > > Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
> > > > > > ---
> > > > > >  tests/i915/i915_pm_rpm.c | 97
> > > > > > ++++++++++++++++++++++++++++++++++------
> > > > > >  1 file changed, 83 insertions(+), 14 deletions(-)
> > > > > >
> > > > > > diff --git a/tests/i915/i915_pm_rpm.c
> > > > > > b/tests/i915/i915_pm_rpm.c index
> > > > > > 31f9712a8..1ef76f562 100644
> > > > > > --- a/tests/i915/i915_pm_rpm.c
> > > > > > +++ b/tests/i915/i915_pm_rpm.c
> > > > > > @@ -85,6 +85,7 @@ enum plane_type {
> > > > > >  #define WAIT_PC8_RES	2
> > > > > >  #define WAIT_EXTRA	4
> > > > > >  #define USE_DPMS	8
> > > > > > +#define WAIT_D3COLD	16
> > > > > >
> > > > > >  int drm_fd, msr_fd, pc8_status_fd;  int debugfs; @@ -215,6
> > > > > > +216,21 @@ static bool wait_for_suspended(void)
> > > > > >  	}
> > > > > >  }
> > > > > >
> > > > > > +static bool wait_for_d3cold(struct pci_device *root) {
> > > > > > +	bool d3colded;
> > > > > > +
> > > > > > +	d3colded = igt_wait(igt_pm_get_acpi_real_d_state(root) ==
> > > > > > +IGT_ACPI_D3Cold, 10000, 500);
> > > > > > +
> > > > > > +	if (!d3colded) {
> > > > > > +		igt_info("D3Cold not achieved for root port
> > > > > %04x:%02x:%02x.%01x\n",
> > > > > > +			 root->domain, root->bus, root->dev, root-
> >func);
> > > > > > +		igt_pm_print_pci_card_runtime_status();
> > > > > > +	}
> > > > > > +
> > > > > > +	return d3colded;
> > > > > > +}
> > > > > > +
> > > > > >  static bool wait_for_active(void)  {
> > > > > >  	if (has_pc8 && !has_runtime_pm) @@ -744,6 +760,30 @@
> static
> > > > > > void test_i2c(struct mode_set_data *data)
> > > > > >  			"There is an EDID mismatch between i2c and
> DRM!\n");
> > > > > }
> > > > > >
> > > > > > +static int get_d3cold_sr_lmem_threshold(int dir) {
> > > > > > +	int param_dir, lmem_threshold, ret;
> > > > > > +
> > > > > > +	param_dir = openat(dir, "i915_params", O_RDONLY);
> > > > > > +	ret = igt_sysfs_scanf(param_dir, "d3cold_sr_lmem_threshold",
> > > > > > +"%d",
> > > > > &lmem_threshold);
> > > > > > +	igt_assert(ret >= 0);
> > > > > > +
> > > > > > +	close(param_dir);
> > > > > > +	return lmem_threshold;
> > > > > > +}
> > > > > > +
> > > > > > +static int set_d3cold_sr_lmem_threshold(int dir, int val) {
> > > > > > +	int param_dir, ret;
> > > > > > +
> > > > > > +	param_dir = openat(dir, "i915_params", O_RDONLY);
> > > > > > +	ret = igt_sysfs_printf(param_dir,
> > > > > > +"d3cold_sr_lmem_threshold", "%d",
> > > > > val);
> > > > > > +	igt_assert(ret > 0);
> > > > > > +
> > > > > > +	close(param_dir);
> > > > > > +	return ret;
> > > > > > +}
> > > > > > +
> > > > > >  static void setup_pc8(void)
> > > > > >  {
> > > > > >  	has_pc8 = false;
> > > > > > @@ -1069,6 +1109,17 @@ static void
> > > debugfs_forcewake_user_subtest(void)
> > > > > >  	igt_assert(wait_for_suspended());  }
> > > > > >
> > > > > > +static struct pci_device
> > > > > > +*setup_d3cold_and_get_root_port(void)
> > > > > > +{
> > > > > > +	struct pci_device *root;
> > > > > > +
> > > > > > +	root = igt_device_get_pci_root_port(drm_fd);
> > > > > > +	igt_require(igt_pm_acpi_d3cold_supported(root));
> > > > > > +	igt_pm_setup_pci_card_runtime_pm(root);
> > > > > > +
> > > > > > +	return root;
> > > > > > +}
> > > > > > +
> > > > > >  static void gem_mmap_args(const struct mmap_offset *t,
> > > > > >  			  struct drm_i915_gem_memory_class_instance
> > > > > *mem_regions)  { @@
> > > > > > -1392,6 +1443,7 @@ gem_execbuf_stress_subtest(int rounds, int
> > > wait_flags,
> > > > > >  	int i;
> > > > > >  	int batch_size = 4 * sizeof(uint32_t);
> > > > > >  	uint32_t batch_buf[batch_size];
> > > > > > +	struct pci_device *root;
> > > > > >  	uint32_t handle;
> > > > > >  	struct drm_i915_gem_execbuffer2 execbuf = {};
> > > > > >  	struct drm_i915_gem_exec_object2 objs[1] = {{}}; @@ -1408,6
> > > > > > +1460,9 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
> > > > > >  	batch_buf[i++] = MI_NOOP;
> > > > > >  	igt_assert(i * sizeof(uint32_t) == batch_size);
> > > > > >
> > > > > > +	if (wait_flags & WAIT_D3COLD)
> > > > > > +		root = setup_d3cold_and_get_root_port();
> > > > > > +
> > > > > >  	disable_all_screens_and_wait(&ms_data);
> > > > > >
> > > > > >  	/* PC8 test is only applicable to igfx  */ @@ -1433,6
> > > > > > +1488,9 @@ gem_execbuf_stress_subtest(int rounds, int wait_flags,
> > > > > >  			/* clean up idle work */
> > > > > >  			igt_drop_caches_set(drm_fd, DROP_IDLE);
> > > > > >  			igt_assert(wait_for_suspended());
> > > > > > +			if (wait_flags & WAIT_D3COLD)
> > > > > > +				igt_assert(wait_for_d3cold(root));
> > > > > > +
> > > > > >  		}
> > > > > >  		if (wait_flags & WAIT_PC8_RES)
> > > > > >  			igt_assert(pc8_plus_residency_changed(30));
> > > > > > @@ -1440,6 +1498,9 @@ gem_execbuf_stress_subtest(int rounds,
> > > > > > int
> > > > > wait_flags,
> > > > > >  			sleep(5);
> > > > > >  	}
> > > > > >
> > > > > > +	if (wait_flags & WAIT_D3COLD)
> > > > > > +		igt_pm_restore_pci_card_runtime_pm();
> > > > > > +
> > > > > >  	gem_close(drm_fd, handle);
> > > > > >  }
> > > > > >
> > > > > > @@ -1537,22 +1598,10 @@ __noreturn static void
> > > > > > stay_subtest(void) static void d3cold_basic_subtest(void)  {
> > > > > >  	struct pci_device *root;
> > > > > > -	bool result;
> > > > > >
> > > > > > -	root = igt_device_get_pci_root_port(drm_fd);
> > > > > > -	igt_require(igt_pm_acpi_d3cold_supported(root));
> > > > > > -	igt_pm_setup_pci_card_runtime_pm(root);
> > > > > > +	root = setup_d3cold_and_get_root_port();
> > > > > >  	disable_all_screens_and_wait(&ms_data);
> > > > > > -
> > > > > > -	result = igt_wait(igt_pm_get_acpi_real_d_state(root) ==
> > > > > IGT_ACPI_D3Cold, 10000, 500);
> > > > > > -
> > > > > > -	if (!result) {
> > > > > > -		igt_info("D3Cold not achieved for root port
> > > > > %04x:%02x:%02x.%01x\n",
> > > > > > -			 root->domain, root->bus, root->dev, root-
> >func);
> > > > > > -		igt_pm_print_pci_card_runtime_status();
> > > > > > -	}
> > > > > > -
> > > > > > -	igt_assert(result);
> > > > > > +	igt_assert(wait_for_d3cold(root));
> > > > > >  	igt_pm_restore_pci_card_runtime_pm();
> > > > > >  }
> > > > > >
> > > > > > @@ -2206,11 +2255,31 @@ igt_main_args("", long_options,
> > > > > > help_str,
> > > > > opt_handler, NULL)
> > > > > >  	igt_describe("Validate execbuf submission while exercising rpm
> "
> > > > > >  		     "suspend/resume cycles.");
> > > > > >  	igt_subtest_with_dynamic("gem-execbuf-stress") {
> > > > > > +		int lmem_threshold;
> > > > > > +
> > > > > >  		for_each_memory_region(r, drm_fd) {
> > > > > >  			igt_dynamic_f("%s", r->name)
> > > > > >  				gem_execbuf_stress_subtest(rounds,
> > > > > WAIT_STATUS, &r->ci);
> > > > > >  			igt_dynamic_f("%s-%s", "extra-wait", r->name)
> > > > > >  				gem_execbuf_stress_subtest(rounds,
> > > > > WAIT_STATUS | WAIT_EXTRA,
> > > > > > &r->ci);
> > > > > > +
> > > > > > +			if (r->ci.memory_class ==
> > > > > I915_MEMORY_CLASS_SYSTEM)
> > > > > > +				continue;
> > > > > > +
> > > > > > +			lmem_threshold =
> > > > > get_d3cold_sr_lmem_threshold(debugfs);
> > > > >
> > > > > since we have just one global threashold and not one per memory
> > > > > region we
> > > can
> > > > > avoid the extra reads in here by moving it ouside.
> > > > >
> > > > > > +			igt_dynamic_f("%s-%s", "d3cold-off", r->name)
> {
> > > > > > +
> 	set_d3cold_sr_lmem_threshold(debugfs, 300);
> > > > > > +				gem_execbuf_stress_subtest(rounds,
> > > > > WAIT_STATUS | WAIT_D3COLD,
> > > > > > +							   &r->ci);
> > > > > > +
> 	set_d3cold_sr_lmem_threshold(debugfs,
> > > > > lmem_threshold);
> > > > >
> > > > > but also we should probably do the save/restore with igt_fixture?
> > > > We can do once we have this param in kernel , otherwise CI will
> > > > break for all
> > > i915_pm_rpm tests.
> > >
> > > I don't get it... why?
> > > Why can't we get_d3cold_sr_lmem_threshold(debugfs); at
> > > setup_environment() and set_d3cold_sr_lmem_threshold(debugfs,
> > > lmem_threshold) at
> > > teardown_environment()
> > We have a assertion here on ret.
> > ret = igt_sysfs_scanf(param_dir, "d3cold_sr_lmem_threshold", "%d",
> > &lmem_threshold);
> 
> A few things I just noticed:
> 
> 1. (probably not related to this patch)
> sysfs? can we use these igt_sysfs funct for the debugfs?
> if so we could probably rename them...
> 
> 2. param_dir is not a good name. that's the file pathname already, not dir.
> 
> > igt_assert(ret >= 0);
> > so this will fail , until underlying kernel implement the
> d3cold_sr_lmem_threshold  module_param.
> 
> if the underlying kernel doesn't have the debug fs won't this igt_assert fail as
> well?
> 
> why would it be different with module_param?
> 
> Petri's question is pertinent, won't this break in LTS anyway?
> 
> > I need to remove igt_assertion from above to to keep {get,set}_
> > get_d3cold_sr_lmem_threshold() in igt_fixture().
> 
> don't we need to remove it anyway?
> in the read inside the loop aren't you breaking the execution of the following
> memory region subtests?
Yes, got that point. Thanks for explaining that.
I will remove the assertion from that point.
Br,
Thanks,
Anshuman Gupta.
> 
> >
> > Thanks,
> > Anshuman Gupta.
> > > ?
> > >
> > > > Thanks,
> > > > Anshuman Gupta.
> > > > >
> > > > > > +			}
> > > > > > +
> > > > > > +			igt_dynamic_f("%s-%s", "d3cold-vrsr", r-
> >name) {
> > > > > > +
> 	set_d3cold_sr_lmem_threshold(debugfs, 0);
> > > > > > +				gem_execbuf_stress_subtest(rounds,
> > > > > WAIT_STATUS | WAIT_D3COLD,
> > > > > > +							   &r->ci);
> > > > > > +
> 	set_d3cold_sr_lmem_threshold(debugfs,
> > > > > lmem_threshold);
> > > > > > +			}
> > > > > >  		}
> > > > > >  	}
> > > > > >
> > > > > > --
> > > > > > 2.26.2
> > > > > >

[igt-dev] [PATCH i-g-t v2 7/8] i915_pm_rpm: Extend gem_execbuf test with D3Cold

[Anshuman Gupta]

Added d3cold dynamic subtest to gem_execbuf with device class
memory region. It test both D3Cold-{VRAM_SR, Off} by using
d3cold_sr_lmem_threshold i915_params debugfs.

v2:
- Two different subtest d3cold-off and d3cold-vrsr. [Rodrigo]

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
---
 tests/i915/i915_pm_rpm.c | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)

diff --git a/tests/i915/i915_pm_rpm.c b/tests/i915/i915_pm_rpm.c
index 1ef76f562..97a84359b 100644
--- a/tests/i915/i915_pm_rpm.c
+++ b/tests/i915/i915_pm_rpm.c
@@ -1342,10 +1342,11 @@ static void submit_blt_cmd(uint32_t dst_handle, int dst_size,
 }
 
 /* Make sure we can submit a batch buffer and verify its result. */
-static void gem_execbuf_subtest(struct drm_i915_gem_memory_class_instance *mem_regions)
+static void gem_execbuf_subtest(struct drm_i915_gem_memory_class_instance *mem_regions, bool d3cold)
 {
 	int x, y;
 	uint32_t handle;
+	struct pci_device *root;
 	int bpp = 4;
 	int pitch = 128 * bpp;
 	int dst_size = 128 * 128 * bpp; /* 128x128 square */
@@ -1367,6 +1368,9 @@ static void gem_execbuf_subtest(struct drm_i915_gem_memory_class_instance *mem_r
 	memset(cpu_buf, 0, dst_size);
 	gem_write(drm_fd, handle, 0, cpu_buf, dst_size);
 
+	if (d3cold)
+		root = setup_d3cold_and_get_root_port();
+
 	/* Now suspend and try it. */
 	disable_all_screens_and_wait(&ms_data);
 
@@ -1375,8 +1379,15 @@ static void gem_execbuf_subtest(struct drm_i915_gem_memory_class_instance *mem_r
 		       &presumed_offset);
 	igt_assert(wait_for_suspended());
 
+	if (d3cold)
+		igt_assert(wait_for_d3cold(root));
+
 	gem_read(drm_fd, handle, 0, cpu_buf, dst_size);
 	igt_assert(wait_for_suspended());
+
+	if (d3cold)
+		igt_assert(wait_for_d3cold(root));
+
 	for (y = 0; y < 128; y++) {
 		for (x = 0; x < 128; x++) {
 			uint32_t px = cpu_buf[y * 128 + x];
@@ -1415,6 +1426,9 @@ static void gem_execbuf_subtest(struct drm_i915_gem_memory_class_instance *mem_r
 
 	disable_all_screens_and_wait(&ms_data);
 
+	if (d3cold)
+		igt_assert(wait_for_d3cold(root));
+
 	memset(cpu_buf, 0, dst_size);
 	gem_read(drm_fd, handle, 0, cpu_buf, dst_size);
 	for (y = 0; y < 128; y++) {
@@ -2169,9 +2183,27 @@ igt_main_args("", long_options, help_str, opt_handler, NULL)
 	igt_subtest("gem-pread")
 		gem_pread_subtest();
 	igt_subtest_with_dynamic("gem-execbuf") {
+		int lmem_threshold;
+
 		for_each_memory_region(r, drm_fd) {
 			igt_dynamic_f("%s", r->name)
-				gem_execbuf_subtest(&r->ci);
+				gem_execbuf_subtest(&r->ci, false);
+
+			if (r->ci.memory_class == I915_MEMORY_CLASS_SYSTEM)
+				continue;
+
+			lmem_threshold = get_d3cold_sr_lmem_threshold(debugfs);
+			igt_dynamic_f("%s-%s", "d3cold-off", r->name) {
+				set_d3cold_sr_lmem_threshold(debugfs, 300);
+				gem_execbuf_subtest(&r->ci, true);
+				set_d3cold_sr_lmem_threshold(debugfs, lmem_threshold);
+			}
+
+			igt_dynamic_f("%s-%s", "d3cold-vrsr", r->name) {
+				set_d3cold_sr_lmem_threshold(debugfs, 0);
+				gem_execbuf_subtest(&r->ci, true);
+				set_d3cold_sr_lmem_threshold(debugfs, lmem_threshold);
+			}
 		}
 	}
 	igt_subtest("gem-idle")
-- 
2.26.2

[igt-dev] [PATCH i-g-t v2 8/8] i915_pm_rpm: Extend gem-mmap-type test with D3Cold

[Anshuman Gupta]

Added d3cold dynamic subtest to gem-mmap-type with device class
memory region. It test both D3Cold-{VRAM_SR, Off} by using
d3cold_sr_lmem_threshold i915_params debugfs.

v2:
- Two different subtest d3cold-off and d3cold-vrsr. [Rodrigo]

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
---
 tests/i915/i915_pm_rpm.c | 44 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 42 insertions(+), 2 deletions(-)

diff --git a/tests/i915/i915_pm_rpm.c b/tests/i915/i915_pm_rpm.c
index 97a84359b..8ca6ba2c8 100644
--- a/tests/i915/i915_pm_rpm.c
+++ b/tests/i915/i915_pm_rpm.c
@@ -1121,11 +1121,12 @@ static struct pci_device *setup_d3cold_and_get_root_port(void)
 }
 
 static void gem_mmap_args(const struct mmap_offset *t,
-			  struct drm_i915_gem_memory_class_instance *mem_regions)
+			  struct drm_i915_gem_memory_class_instance *mem_regions, bool d3cold)
 {
 	int i;
 	uint32_t handle;
 	int buf_size = 8192;
+	struct pci_device *root;
 	uint8_t *gem_buf;
 
 	/* Create, map and set data while the device is active. */
@@ -1143,17 +1144,29 @@ static void gem_mmap_args(const struct mmap_offset *t,
 	for (i = 0; i < buf_size; i++)
 		igt_assert(gem_buf[i] == (i & 0xFF));
 
+	if (d3cold)
+		root = setup_d3cold_and_get_root_port();
+
 	/* Now suspend, read and modify. */
 	disable_all_screens_and_wait(&ms_data);
 
+	if (d3cold)
+		igt_assert(wait_for_d3cold(root));
+
 	for (i = 0; i < buf_size; i++)
 		igt_assert(gem_buf[i] == (i & 0xFF));
 	igt_assert(wait_for_suspended());
 
+	if (d3cold)
+		igt_assert(wait_for_d3cold(root));
+
 	for (i = 0; i < buf_size; i++)
 		gem_buf[i] = (~i & 0xFF);
 	igt_assert(wait_for_suspended());
 
+	if (d3cold)
+		igt_assert(wait_for_d3cold(root));
+
 	/* Now resume and see if it's still there. */
 	enable_one_screen_and_wait(&ms_data);
 	for (i = 0; i < buf_size; i++)
@@ -1165,12 +1178,18 @@ static void gem_mmap_args(const struct mmap_offset *t,
 	 * suspended. */
 	disable_all_screens_and_wait(&ms_data);
 
+	if (d3cold)
+		igt_assert(wait_for_d3cold(root));
+
 	gem_buf = __gem_mmap_offset(drm_fd, handle, 0, buf_size,
 				    PROT_READ | PROT_WRITE, t->type);
 	igt_require(gem_buf);
 
 	igt_assert(wait_for_suspended());
 
+	if (d3cold)
+		igt_assert(wait_for_d3cold(root));
+
 	for (i = 0; i < buf_size; i++)
 		gem_buf[i] = i & 0xFF;
 
@@ -1179,6 +1198,9 @@ static void gem_mmap_args(const struct mmap_offset *t,
 
 	igt_assert(wait_for_suspended());
 
+	if (d3cold)
+		igt_assert(wait_for_d3cold(root));
+
 	/* Resume and check if it's still there. */
 	enable_one_screen_and_wait(&ms_data);
 	for (i = 0; i < buf_size; i++)
@@ -2172,10 +2194,28 @@ igt_main_args("", long_options, help_str, opt_handler, NULL)
 
 	/* GEM */
 	igt_subtest_with_dynamic("gem-mmap-type") {
+		int lmem_threshold;
+
 		for_each_mmap_offset_type(drm_fd, t) {
 			for_each_memory_region(r, drm_fd) {
 				igt_dynamic_f("%s-%s", t->name, r->name)
-				gem_mmap_args(t, &r->ci);
+				gem_mmap_args(t, &r->ci, false);
+
+				if (r->ci.memory_class == I915_MEMORY_CLASS_SYSTEM)
+					continue;
+
+				lmem_threshold = get_d3cold_sr_lmem_threshold(debugfs);
+				igt_dynamic_f("%s-%s-%s", t->name, "d3cold-off", r->name) {
+					set_d3cold_sr_lmem_threshold(debugfs, 300);
+					gem_mmap_args(t, &r->ci, true);
+					set_d3cold_sr_lmem_threshold(debugfs, lmem_threshold);
+				}
+
+				igt_dynamic_f("%s-%s-%s", t->name, "d3cold-vrsr", r->name) {
+					set_d3cold_sr_lmem_threshold(debugfs, 0);
+					gem_mmap_args(t, &r->ci, true);
+					set_d3cold_sr_lmem_threshold(debugfs, lmem_threshold);
+				}
 			}
 		}
 	}
-- 
2.26.2

[igt-dev] ✓ Fi.CI.BAT: success for D3Cold Tool & IGT (rev2)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 8203 bytes --]

== Series Details ==

Series: D3Cold Tool & IGT (rev2)
URL   : https://patchwork.freedesktop.org/series/102780/
State : success

== Summary ==

CI Bug Log - changes from IGT_6462 -> IGTPW_7024
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/index.html

Participating hosts (44 -> 43)
------------------------------

  Additional (2): bat-adlm-1 fi-rkl-11600 
  Missing    (3): fi-bsw-cyan bat-rpls-1 fi-icl-u2 

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in IGTPW_7024:

### IGT changes ###

#### Suppressed ####

  The following results come from untrusted machines, tests, or statuses.
  They do not affect the overall result.

  * igt@i915_selftest@live@gt_timelines:
    - {bat-adlm-1}:       NOTRUN -> [INCOMPLETE][1]
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/bat-adlm-1/igt@i915_selftest@live@gt_timelines.html

  
Known issues
------------

  Here are the changes found in IGTPW_7024 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_huc_copy@huc-copy:
    - fi-rkl-11600:       NOTRUN -> [SKIP][2] ([i915#2190])
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-rkl-11600/igt@gem_huc_copy@huc-copy.html

  * igt@gem_lmem_swapping@basic:
    - fi-rkl-11600:       NOTRUN -> [SKIP][3] ([i915#4613]) +3 similar issues
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-rkl-11600/igt@gem_lmem_swapping@basic.html

  * igt@gem_tiled_pread_basic:
    - fi-rkl-11600:       NOTRUN -> [SKIP][4] ([i915#3282])
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-rkl-11600/igt@gem_tiled_pread_basic.html

  * igt@i915_pm_backlight@basic-brightness:
    - fi-rkl-11600:       NOTRUN -> [SKIP][5] ([i915#3012])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-rkl-11600/igt@i915_pm_backlight@basic-brightness.html

  * igt@i915_selftest@live@gt_pm:
    - fi-tgl-1115g4:      [PASS][6] -> [DMESG-FAIL][7] ([i915#3987])
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/fi-tgl-1115g4/igt@i915_selftest@live@gt_pm.html
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-tgl-1115g4/igt@i915_selftest@live@gt_pm.html

  * igt@i915_selftest@live@hangcheck:
    - fi-bdw-5557u:       NOTRUN -> [INCOMPLETE][8] ([i915#3921])
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-bdw-5557u/igt@i915_selftest@live@hangcheck.html

  * igt@kms_chamelium@dp-crc-fast:
    - fi-rkl-11600:       NOTRUN -> [SKIP][9] ([fdo#111827]) +8 similar issues
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-rkl-11600/igt@kms_chamelium@dp-crc-fast.html

  * igt@kms_cursor_legacy@basic-busy-flip-before-cursor-atomic:
    - fi-rkl-11600:       NOTRUN -> [SKIP][10] ([i915#4070] / [i915#4103]) +1 similar issue
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-rkl-11600/igt@kms_cursor_legacy@basic-busy-flip-before-cursor-atomic.html

  * igt@kms_force_connector_basic@force-load-detect:
    - fi-rkl-11600:       NOTRUN -> [SKIP][11] ([fdo#109285] / [i915#4098])
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-rkl-11600/igt@kms_force_connector_basic@force-load-detect.html

  * igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d:
    - fi-rkl-11600:       NOTRUN -> [SKIP][12] ([i915#4070] / [i915#533])
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-rkl-11600/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d.html

  * igt@kms_psr@primary_mmap_gtt:
    - fi-rkl-11600:       NOTRUN -> [SKIP][13] ([i915#1072]) +3 similar issues
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-rkl-11600/igt@kms_psr@primary_mmap_gtt.html

  * igt@kms_setmode@basic-clone-single-crtc:
    - fi-rkl-11600:       NOTRUN -> [SKIP][14] ([i915#3555] / [i915#4098])
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-rkl-11600/igt@kms_setmode@basic-clone-single-crtc.html

  * igt@prime_vgem@basic-userptr:
    - fi-rkl-11600:       NOTRUN -> [SKIP][15] ([i915#3301] / [i915#3708])
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-rkl-11600/igt@prime_vgem@basic-userptr.html

  * igt@prime_vgem@basic-write:
    - fi-rkl-11600:       NOTRUN -> [SKIP][16] ([i915#3291] / [i915#3708]) +2 similar issues
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-rkl-11600/igt@prime_vgem@basic-write.html

  
#### Possible fixes ####

  * igt@gem_ringfill@basic-all:
    - {bat-dg2-9}:        [TIMEOUT][17] ([i915#5647]) -> [PASS][18]
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/bat-dg2-9/igt@gem_ringfill@basic-all.html
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/bat-dg2-9/igt@gem_ringfill@basic-all.html

  * igt@i915_selftest@live@mman:
    - fi-bdw-5557u:       [INCOMPLETE][19] ([i915#5704]) -> [PASS][20]
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/fi-bdw-5557u/igt@i915_selftest@live@mman.html
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/fi-bdw-5557u/igt@i915_selftest@live@mman.html

  * igt@kms_busy@basic@flip:
    - {bat-adlp-6}:       [DMESG-WARN][21] ([i915#3576]) -> [PASS][22]
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/bat-adlp-6/igt@kms_busy@basic@flip.html
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/bat-adlp-6/igt@kms_busy@basic@flip.html

  * igt@kms_flip@basic-plain-flip@a-edp1:
    - bat-adlp-4:         [DMESG-WARN][23] ([i915#3576]) -> [PASS][24]
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/bat-adlp-4/igt@kms_flip@basic-plain-flip@a-edp1.html
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/bat-adlp-4/igt@kms_flip@basic-plain-flip@a-edp1.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285
  [fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
  [i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
  [i915#1155]: https://gitlab.freedesktop.org/drm/intel/issues/1155
  [i915#2190]: https://gitlab.freedesktop.org/drm/intel/issues/2190
  [i915#3012]: https://gitlab.freedesktop.org/drm/intel/issues/3012
  [i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282
  [i915#3291]: https://gitlab.freedesktop.org/drm/intel/issues/3291
  [i915#3301]: https://gitlab.freedesktop.org/drm/intel/issues/3301
  [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
  [i915#3576]: https://gitlab.freedesktop.org/drm/intel/issues/3576
  [i915#3708]: https://gitlab.freedesktop.org/drm/intel/issues/3708
  [i915#3921]: https://gitlab.freedesktop.org/drm/intel/issues/3921
  [i915#3987]: https://gitlab.freedesktop.org/drm/intel/issues/3987
  [i915#4070]: https://gitlab.freedesktop.org/drm/intel/issues/4070
  [i915#4098]: https://gitlab.freedesktop.org/drm/intel/issues/4098
  [i915#4103]: https://gitlab.freedesktop.org/drm/intel/issues/4103
  [i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312
  [i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
  [i915#5153]: https://gitlab.freedesktop.org/drm/intel/issues/5153
  [i915#5270]: https://gitlab.freedesktop.org/drm/intel/issues/5270
  [i915#533]: https://gitlab.freedesktop.org/drm/intel/issues/533
  [i915#5356]: https://gitlab.freedesktop.org/drm/intel/issues/5356
  [i915#5647]: https://gitlab.freedesktop.org/drm/intel/issues/5647
  [i915#5704]: https://gitlab.freedesktop.org/drm/intel/issues/5704


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_6462 -> IGTPW_7024

  CI-20190529: 20190529
  CI_DRM_11550: 56b089ae03ef8ea8ab7f474eaa70367898891ef0 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_7024: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/index.html
  IGT_6462: ca6db54be6bf51958ad99bb319bdb514ae58eb25 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git


Testlist changes
----------------

+igt@i915_pm_rpm@d3cold-basic

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/index.html

[-- Attachment #2: Type: text/html, Size: 9213 bytes --]

[igt-dev] ✓ Fi.CI.IGT: success for D3Cold Tool & IGT (rev2)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 58706 bytes --]

== Series Details ==

Series: D3Cold Tool & IGT (rev2)
URL   : https://patchwork.freedesktop.org/series/102780/
State : success

== Summary ==

CI Bug Log - changes from IGT_6462_full -> IGTPW_7024_full
====================================================

Summary
-------

  **WARNING**

  Minor unknown changes coming with IGTPW_7024_full need to be verified
  manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in IGTPW_7024_full, please notify your bug team to allow them
  to document this new failure mode, which will reduce false positives in CI.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/index.html

Participating hosts (10 -> 8)
------------------------------

  Missing    (2): shard-skl shard-dg1 

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in IGTPW_7024_full:

### IGT changes ###

#### Possible regressions ####

  * {igt@i915_pm_rpm@d3cold-basic} (NEW):
    - shard-iclb:         NOTRUN -> [SKIP][1]
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb2/igt@i915_pm_rpm@d3cold-basic.html

  * {igt@i915_pm_rpm@gem-mmap-type@wc-smem0} (NEW):
    - {shard-rkl}:        NOTRUN -> [SKIP][2] +4 similar issues
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-2/igt@i915_pm_rpm@gem-mmap-type@wc-smem0.html

  
#### Warnings ####

  * igt@kms_psr2_su@page_flip-p010:
    - shard-iclb:         [SKIP][3] ([fdo#109642] / [fdo#111068] / [i915#658]) -> [FAIL][4]
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb8/igt@kms_psr2_su@page_flip-p010.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb2/igt@kms_psr2_su@page_flip-p010.html

  
New tests
---------

  New tests have been introduced between IGT_6462_full and IGTPW_7024_full:

### New IGT tests (5) ###

  * igt@i915_pm_rpm@d3cold-basic:
    - Statuses : 7 skip(s)
    - Exec time: [0.0, 0.79] s

  * igt@i915_pm_rpm@gem-mmap-type@gtt-smem0:
    - Statuses : 6 pass(s) 1 skip(s)
    - Exec time: [0.0, 14.96] s

  * igt@i915_pm_rpm@gem-mmap-type@uc-smem0:
    - Statuses : 6 pass(s) 1 skip(s)
    - Exec time: [0.0, 6.43] s

  * igt@i915_pm_rpm@gem-mmap-type@wb-smem0:
    - Statuses : 6 pass(s) 1 skip(s)
    - Exec time: [0.0, 6.42] s

  * igt@i915_pm_rpm@gem-mmap-type@wc-smem0:
    - Statuses : 6 pass(s) 1 skip(s)
    - Exec time: [0.0, 6.99] s

  

Known issues
------------

  Here are the changes found in IGTPW_7024_full that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@feature_discovery@display-4x:
    - shard-tglb:         NOTRUN -> [SKIP][5] ([i915#1839]) +1 similar issue
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb7/igt@feature_discovery@display-4x.html

  * igt@gem_ccs@block-copy-compressed:
    - shard-tglb:         NOTRUN -> [SKIP][6] ([i915#3555] / [i915#5325])
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb5/igt@gem_ccs@block-copy-compressed.html

  * igt@gem_ccs@block-copy-uncompressed:
    - shard-iclb:         NOTRUN -> [SKIP][7] ([i915#5327])
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb6/igt@gem_ccs@block-copy-uncompressed.html

  * igt@gem_ccs@ctrl-surf-copy-new-ctx:
    - shard-tglb:         NOTRUN -> [SKIP][8] ([i915#5325])
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb3/igt@gem_ccs@ctrl-surf-copy-new-ctx.html

  * igt@gem_create@create-massive:
    - shard-apl:          NOTRUN -> [DMESG-WARN][9] ([i915#4991])
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl1/igt@gem_create@create-massive.html

  * igt@gem_ctx_sseu@engines:
    - shard-tglb:         NOTRUN -> [SKIP][10] ([i915#280])
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb3/igt@gem_ctx_sseu@engines.html

  * igt@gem_exec_fair@basic-pace@vcs0:
    - shard-kbl:          [PASS][11] -> [FAIL][12] ([i915#2842])
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-kbl4/igt@gem_exec_fair@basic-pace@vcs0.html
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl6/igt@gem_exec_fair@basic-pace@vcs0.html
    - shard-tglb:         [PASS][13] -> [FAIL][14] ([i915#2842])
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-tglb3/igt@gem_exec_fair@basic-pace@vcs0.html
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb8/igt@gem_exec_fair@basic-pace@vcs0.html

  * igt@gem_exec_flush@basic-uc-pro-default:
    - shard-snb:          [PASS][15] -> [SKIP][16] ([fdo#109271]) +4 similar issues
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-snb2/igt@gem_exec_flush@basic-uc-pro-default.html
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-snb6/igt@gem_exec_flush@basic-uc-pro-default.html

  * igt@gem_exec_whisper@basic-queues-all:
    - shard-glk:          NOTRUN -> [DMESG-WARN][17] ([i915#118])
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk1/igt@gem_exec_whisper@basic-queues-all.html

  * igt@gem_lmem_swapping@basic:
    - shard-apl:          NOTRUN -> [SKIP][18] ([fdo#109271] / [i915#4613]) +2 similar issues
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl1/igt@gem_lmem_swapping@basic.html

  * igt@gem_lmem_swapping@smem-oom:
    - shard-kbl:          NOTRUN -> [SKIP][19] ([fdo#109271] / [i915#4613]) +4 similar issues
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl7/igt@gem_lmem_swapping@smem-oom.html
    - shard-iclb:         NOTRUN -> [SKIP][20] ([i915#4613]) +1 similar issue
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb7/igt@gem_lmem_swapping@smem-oom.html

  * igt@gem_lmem_swapping@verify-random:
    - shard-tglb:         NOTRUN -> [SKIP][21] ([i915#4613]) +4 similar issues
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb7/igt@gem_lmem_swapping@verify-random.html

  * igt@gem_mmap_gtt@coherency:
    - shard-tglb:         NOTRUN -> [SKIP][22] ([fdo#111656])
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb1/igt@gem_mmap_gtt@coherency.html

  * igt@gem_pread@exhaustion:
    - shard-glk:          NOTRUN -> [WARN][23] ([i915#2658])
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk4/igt@gem_pread@exhaustion.html

  * igt@gem_pxp@create-regular-context-1:
    - shard-iclb:         NOTRUN -> [SKIP][24] ([i915#4270]) +2 similar issues
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb8/igt@gem_pxp@create-regular-context-1.html

  * igt@gem_pxp@protected-encrypted-src-copy-not-readible:
    - shard-tglb:         NOTRUN -> [SKIP][25] ([i915#4270]) +4 similar issues
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb1/igt@gem_pxp@protected-encrypted-src-copy-not-readible.html

  * igt@gem_render_copy@y-tiled-to-vebox-x-tiled:
    - shard-iclb:         NOTRUN -> [SKIP][26] ([i915#768])
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb6/igt@gem_render_copy@y-tiled-to-vebox-x-tiled.html

  * igt@gem_softpin@evict-snoop:
    - shard-iclb:         NOTRUN -> [SKIP][27] ([fdo#109312])
   [27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb8/igt@gem_softpin@evict-snoop.html
    - shard-tglb:         NOTRUN -> [SKIP][28] ([fdo#109312])
   [28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb8/igt@gem_softpin@evict-snoop.html

  * igt@gem_userptr_blits@dmabuf-sync:
    - shard-kbl:          NOTRUN -> [SKIP][29] ([fdo#109271] / [i915#3323])
   [29]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl6/igt@gem_userptr_blits@dmabuf-sync.html
    - shard-iclb:         NOTRUN -> [SKIP][30] ([i915#3323])
   [30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb3/igt@gem_userptr_blits@dmabuf-sync.html
    - shard-apl:          NOTRUN -> [SKIP][31] ([fdo#109271] / [i915#3323])
   [31]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl3/igt@gem_userptr_blits@dmabuf-sync.html
    - shard-glk:          NOTRUN -> [SKIP][32] ([fdo#109271] / [i915#3323])
   [32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk8/igt@gem_userptr_blits@dmabuf-sync.html
    - shard-tglb:         NOTRUN -> [SKIP][33] ([i915#3323])
   [33]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb1/igt@gem_userptr_blits@dmabuf-sync.html

  * igt@gem_userptr_blits@unsync-unmap-cycles:
    - shard-tglb:         NOTRUN -> [SKIP][34] ([i915#3297]) +2 similar issues
   [34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb5/igt@gem_userptr_blits@unsync-unmap-cycles.html
    - shard-iclb:         NOTRUN -> [SKIP][35] ([i915#3297]) +1 similar issue
   [35]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb7/igt@gem_userptr_blits@unsync-unmap-cycles.html

  * igt@gen7_exec_parse@basic-allowed:
    - shard-tglb:         NOTRUN -> [SKIP][36] ([fdo#109289]) +3 similar issues
   [36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb6/igt@gen7_exec_parse@basic-allowed.html

  * igt@gen9_exec_parse@allowed-all:
    - shard-apl:          [PASS][37] -> [DMESG-WARN][38] ([i915#5566] / [i915#716])
   [37]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-apl2/igt@gen9_exec_parse@allowed-all.html
   [38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl2/igt@gen9_exec_parse@allowed-all.html

  * igt@gen9_exec_parse@bb-start-cmd:
    - shard-tglb:         NOTRUN -> [SKIP][39] ([i915#2527] / [i915#2856]) +6 similar issues
   [39]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb2/igt@gen9_exec_parse@bb-start-cmd.html

  * igt@gen9_exec_parse@valid-registers:
    - shard-iclb:         NOTRUN -> [SKIP][40] ([i915#2856]) +1 similar issue
   [40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb4/igt@gen9_exec_parse@valid-registers.html

  * {igt@i915_pm_rpm@d3cold-basic} (NEW):
    - shard-tglb:         NOTRUN -> [SKIP][41] ([i915#2411])
   [41]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb8/igt@i915_pm_rpm@d3cold-basic.html

  * igt@i915_pm_rpm@dpms-non-lpsp:
    - shard-tglb:         NOTRUN -> [SKIP][42] ([fdo#111644] / [i915#1397] / [i915#2411])
   [42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb2/igt@i915_pm_rpm@dpms-non-lpsp.html

  * igt@i915_pm_sseu@full-enable:
    - shard-tglb:         NOTRUN -> [SKIP][43] ([i915#4387])
   [43]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb5/igt@i915_pm_sseu@full-enable.html
    - shard-iclb:         NOTRUN -> [SKIP][44] ([i915#4387])
   [44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb2/igt@i915_pm_sseu@full-enable.html

  * igt@i915_query@query-topology-known-pci-ids:
    - shard-tglb:         NOTRUN -> [SKIP][45] ([fdo#109303])
   [45]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb6/igt@i915_query@query-topology-known-pci-ids.html

  * igt@i915_query@query-topology-unsupported:
    - shard-tglb:         NOTRUN -> [SKIP][46] ([fdo#109302])
   [46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb7/igt@i915_query@query-topology-unsupported.html

  * igt@kms_big_fb@4-tiled-max-hw-stride-32bpp-rotate-0:
    - shard-tglb:         NOTRUN -> [SKIP][47] ([i915#5286]) +4 similar issues
   [47]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb7/igt@kms_big_fb@4-tiled-max-hw-stride-32bpp-rotate-0.html

  * igt@kms_big_fb@4-tiled-max-hw-stride-32bpp-rotate-180-hflip:
    - shard-iclb:         NOTRUN -> [SKIP][48] ([i915#5286]) +1 similar issue
   [48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb6/igt@kms_big_fb@4-tiled-max-hw-stride-32bpp-rotate-180-hflip.html

  * igt@kms_big_fb@linear-8bpp-rotate-90:
    - shard-tglb:         NOTRUN -> [SKIP][49] ([fdo#111614]) +2 similar issues
   [49]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb7/igt@kms_big_fb@linear-8bpp-rotate-90.html
    - shard-iclb:         NOTRUN -> [SKIP][50] ([fdo#110725] / [fdo#111614])
   [50]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb8/igt@kms_big_fb@linear-8bpp-rotate-90.html

  * igt@kms_big_fb@yf-tiled-8bpp-rotate-90:
    - shard-tglb:         NOTRUN -> [SKIP][51] ([fdo#111615]) +9 similar issues
   [51]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb1/igt@kms_big_fb@yf-tiled-8bpp-rotate-90.html

  * igt@kms_big_fb@yf-tiled-max-hw-stride-32bpp-rotate-0-hflip-async-flip:
    - shard-kbl:          NOTRUN -> [SKIP][52] ([fdo#109271]) +232 similar issues
   [52]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl4/igt@kms_big_fb@yf-tiled-max-hw-stride-32bpp-rotate-0-hflip-async-flip.html

  * igt@kms_big_joiner@basic:
    - shard-tglb:         NOTRUN -> [SKIP][53] ([i915#2705]) +1 similar issue
   [53]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb1/igt@kms_big_joiner@basic.html
    - shard-iclb:         NOTRUN -> [SKIP][54] ([i915#2705]) +1 similar issue
   [54]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb3/igt@kms_big_joiner@basic.html

  * igt@kms_ccs@pipe-a-bad-rotation-90-y_tiled_gen12_mc_ccs:
    - shard-glk:          NOTRUN -> [SKIP][55] ([fdo#109271] / [i915#3886]) +3 similar issues
   [55]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk3/igt@kms_ccs@pipe-a-bad-rotation-90-y_tiled_gen12_mc_ccs.html

  * igt@kms_ccs@pipe-a-random-ccs-data-yf_tiled_ccs:
    - shard-tglb:         NOTRUN -> [SKIP][56] ([fdo#111615] / [i915#3689]) +8 similar issues
   [56]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb5/igt@kms_ccs@pipe-a-random-ccs-data-yf_tiled_ccs.html

  * igt@kms_ccs@pipe-b-bad-rotation-90-y_tiled_ccs:
    - shard-tglb:         NOTRUN -> [SKIP][57] ([i915#3689]) +5 similar issues
   [57]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb6/igt@kms_ccs@pipe-b-bad-rotation-90-y_tiled_ccs.html

  * igt@kms_ccs@pipe-b-ccs-on-another-bo-y_tiled_gen12_mc_ccs:
    - shard-tglb:         NOTRUN -> [SKIP][58] ([i915#3689] / [i915#3886]) +3 similar issues
   [58]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb5/igt@kms_ccs@pipe-b-ccs-on-another-bo-y_tiled_gen12_mc_ccs.html

  * igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc:
    - shard-kbl:          NOTRUN -> [SKIP][59] ([fdo#109271] / [i915#3886]) +6 similar issues
   [59]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl7/igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc.html
    - shard-apl:          NOTRUN -> [SKIP][60] ([fdo#109271] / [i915#3886]) +10 similar issues
   [60]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl6/igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc.html
    - shard-iclb:         NOTRUN -> [SKIP][61] ([fdo#109278] / [i915#3886]) +2 similar issues
   [61]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb3/igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc.html

  * igt@kms_ccs@pipe-d-bad-rotation-90-y_tiled_gen12_rc_ccs_cc:
    - shard-iclb:         NOTRUN -> [SKIP][62] ([fdo#109278]) +25 similar issues
   [62]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb5/igt@kms_ccs@pipe-d-bad-rotation-90-y_tiled_gen12_rc_ccs_cc.html

  * igt@kms_ccs@pipe-d-random-ccs-data-y_tiled_gen12_mc_ccs:
    - shard-glk:          NOTRUN -> [SKIP][63] ([fdo#109271]) +58 similar issues
   [63]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk2/igt@kms_ccs@pipe-d-random-ccs-data-y_tiled_gen12_mc_ccs.html

  * igt@kms_chamelium@hdmi-hpd-storm:
    - shard-kbl:          NOTRUN -> [SKIP][64] ([fdo#109271] / [fdo#111827]) +19 similar issues
   [64]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl6/igt@kms_chamelium@hdmi-hpd-storm.html

  * igt@kms_color@pipe-b-deep-color:
    - shard-tglb:         NOTRUN -> [SKIP][65] ([i915#3555]) +1 similar issue
   [65]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb7/igt@kms_color@pipe-b-deep-color.html

  * igt@kms_color@pipe-d-ctm-0-5:
    - shard-iclb:         NOTRUN -> [SKIP][66] ([fdo#109278] / [i915#1149])
   [66]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb4/igt@kms_color@pipe-d-ctm-0-5.html

  * igt@kms_color_chamelium@pipe-b-ctm-max:
    - shard-tglb:         NOTRUN -> [SKIP][67] ([fdo#109284] / [fdo#111827]) +21 similar issues
   [67]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb5/igt@kms_color_chamelium@pipe-b-ctm-max.html
    - shard-snb:          NOTRUN -> [SKIP][68] ([fdo#109271] / [fdo#111827]) +5 similar issues
   [68]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-snb7/igt@kms_color_chamelium@pipe-b-ctm-max.html

  * igt@kms_color_chamelium@pipe-b-ctm-red-to-blue:
    - shard-iclb:         NOTRUN -> [SKIP][69] ([fdo#109284] / [fdo#111827]) +8 similar issues
   [69]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb3/igt@kms_color_chamelium@pipe-b-ctm-red-to-blue.html

  * igt@kms_color_chamelium@pipe-c-ctm-0-25:
    - shard-apl:          NOTRUN -> [SKIP][70] ([fdo#109271] / [fdo#111827]) +24 similar issues
   [70]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl4/igt@kms_color_chamelium@pipe-c-ctm-0-25.html

  * igt@kms_color_chamelium@pipe-d-degamma:
    - shard-glk:          NOTRUN -> [SKIP][71] ([fdo#109271] / [fdo#111827]) +8 similar issues
   [71]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk4/igt@kms_color_chamelium@pipe-d-degamma.html
    - shard-iclb:         NOTRUN -> [SKIP][72] ([fdo#109278] / [fdo#109284] / [fdo#111827])
   [72]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb7/igt@kms_color_chamelium@pipe-d-degamma.html

  * igt@kms_content_protection@atomic:
    - shard-apl:          NOTRUN -> [TIMEOUT][73] ([i915#1319]) +1 similar issue
   [73]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl3/igt@kms_content_protection@atomic.html

  * igt@kms_content_protection@mei_interface:
    - shard-tglb:         NOTRUN -> [SKIP][74] ([fdo#109300])
   [74]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb2/igt@kms_content_protection@mei_interface.html

  * igt@kms_content_protection@type1:
    - shard-iclb:         NOTRUN -> [SKIP][75] ([fdo#109300] / [fdo#111066])
   [75]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb7/igt@kms_content_protection@type1.html
    - shard-tglb:         NOTRUN -> [SKIP][76] ([i915#1063])
   [76]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb1/igt@kms_content_protection@type1.html

  * igt@kms_content_protection@uevent:
    - shard-apl:          NOTRUN -> [FAIL][77] ([i915#2105])
   [77]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl7/igt@kms_content_protection@uevent.html

  * igt@kms_cursor_crc@pipe-b-cursor-32x10-sliding:
    - shard-tglb:         NOTRUN -> [SKIP][78] ([i915#3359]) +8 similar issues
   [78]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb8/igt@kms_cursor_crc@pipe-b-cursor-32x10-sliding.html

  * igt@kms_cursor_crc@pipe-b-cursor-32x32-sliding:
    - shard-tglb:         NOTRUN -> [SKIP][79] ([i915#3319]) +1 similar issue
   [79]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb3/igt@kms_cursor_crc@pipe-b-cursor-32x32-sliding.html

  * igt@kms_cursor_crc@pipe-b-cursor-512x170-offscreen:
    - shard-iclb:         NOTRUN -> [SKIP][80] ([fdo#109278] / [fdo#109279]) +1 similar issue
   [80]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb1/igt@kms_cursor_crc@pipe-b-cursor-512x170-offscreen.html

  * igt@kms_cursor_crc@pipe-b-cursor-512x512-rapid-movement:
    - shard-tglb:         NOTRUN -> [SKIP][81] ([fdo#109279] / [i915#3359]) +4 similar issues
   [81]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb7/igt@kms_cursor_crc@pipe-b-cursor-512x512-rapid-movement.html

  * igt@kms_cursor_legacy@2x-cursor-vs-flip-legacy:
    - shard-iclb:         NOTRUN -> [SKIP][82] ([fdo#109274] / [fdo#109278]) +1 similar issue
   [82]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb3/igt@kms_cursor_legacy@2x-cursor-vs-flip-legacy.html

  * igt@kms_cursor_legacy@cursorb-vs-flipb-atomic:
    - shard-tglb:         NOTRUN -> [SKIP][83] ([fdo#109274] / [fdo#111825]) +14 similar issues
   [83]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb6/igt@kms_cursor_legacy@cursorb-vs-flipb-atomic.html

  * igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions:
    - shard-glk:          [PASS][84] -> [FAIL][85] ([i915#2346])
   [84]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-glk6/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html
   [85]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk2/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html

  * igt@kms_cursor_legacy@short-busy-flip-before-cursor-atomic-transitions-varying-size:
    - shard-tglb:         NOTRUN -> [SKIP][86] ([i915#4103]) +1 similar issue
   [86]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb7/igt@kms_cursor_legacy@short-busy-flip-before-cursor-atomic-transitions-varying-size.html

  * igt@kms_draw_crc@draw-method-rgb565-blt-4tiled:
    - shard-iclb:         NOTRUN -> [SKIP][87] ([i915#5287]) +2 similar issues
   [87]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb5/igt@kms_draw_crc@draw-method-rgb565-blt-4tiled.html

  * igt@kms_draw_crc@draw-method-rgb565-mmap-gtt-ytiled:
    - shard-snb:          NOTRUN -> [SKIP][88] ([fdo#109271]) +92 similar issues
   [88]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-snb4/igt@kms_draw_crc@draw-method-rgb565-mmap-gtt-ytiled.html

  * igt@kms_draw_crc@draw-method-rgb565-pwrite-4tiled:
    - shard-tglb:         NOTRUN -> [SKIP][89] ([i915#5287]) +3 similar issues
   [89]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb3/igt@kms_draw_crc@draw-method-rgb565-pwrite-4tiled.html

  * igt@kms_flip@2x-nonexisting-fb:
    - shard-iclb:         NOTRUN -> [SKIP][90] ([fdo#109274]) +1 similar issue
   [90]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb8/igt@kms_flip@2x-nonexisting-fb.html

  * igt@kms_flip@flip-vs-suspend-interruptible@c-dp1:
    - shard-apl:          [PASS][91] -> [DMESG-WARN][92] ([i915#180]) +1 similar issue
   [91]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-apl3/igt@kms_flip@flip-vs-suspend-interruptible@c-dp1.html
   [92]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl1/igt@kms_flip@flip-vs-suspend-interruptible@c-dp1.html

  * igt@kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling:
    - shard-iclb:         [PASS][93] -> [SKIP][94] ([i915#3701])
   [93]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb1/igt@kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling.html
   [94]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb2/igt@kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling.html

  * igt@kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling:
    - shard-tglb:         NOTRUN -> [SKIP][95] ([i915#2587])
   [95]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb5/igt@kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling.html

  * igt@kms_frontbuffer_tracking@psr-2p-primscrn-shrfb-plflip-blt:
    - shard-iclb:         NOTRUN -> [SKIP][96] ([fdo#109280]) +23 similar issues
   [96]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb8/igt@kms_frontbuffer_tracking@psr-2p-primscrn-shrfb-plflip-blt.html

  * igt@kms_frontbuffer_tracking@psr-2p-scndscrn-pri-indfb-draw-mmap-gtt:
    - shard-tglb:         NOTRUN -> [SKIP][97] ([fdo#109280] / [fdo#111825]) +46 similar issues
   [97]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb1/igt@kms_frontbuffer_tracking@psr-2p-scndscrn-pri-indfb-draw-mmap-gtt.html

  * igt@kms_pipe_crc_basic@nonblocking-crc-pipe-d-frame-sequence:
    - shard-kbl:          NOTRUN -> [SKIP][98] ([fdo#109271] / [i915#533]) +3 similar issues
   [98]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl4/igt@kms_pipe_crc_basic@nonblocking-crc-pipe-d-frame-sequence.html
    - shard-glk:          NOTRUN -> [SKIP][99] ([fdo#109271] / [i915#533])
   [99]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk1/igt@kms_pipe_crc_basic@nonblocking-crc-pipe-d-frame-sequence.html

  * igt@kms_pipe_crc_basic@suspend-read-crc-pipe-d:
    - shard-apl:          NOTRUN -> [SKIP][100] ([fdo#109271] / [i915#533]) +4 similar issues
   [100]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl2/igt@kms_pipe_crc_basic@suspend-read-crc-pipe-d.html

  * igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes:
    - shard-apl:          NOTRUN -> [DMESG-WARN][101] ([i915#180])
   [101]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl1/igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes.html

  * igt@kms_plane_alpha_blend@pipe-a-alpha-transparent-fb:
    - shard-kbl:          NOTRUN -> [FAIL][102] ([i915#265])
   [102]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl6/igt@kms_plane_alpha_blend@pipe-a-alpha-transparent-fb.html

  * igt@kms_plane_alpha_blend@pipe-b-alpha-basic:
    - shard-apl:          NOTRUN -> [FAIL][103] ([fdo#108145] / [i915#265]) +1 similar issue
   [103]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl1/igt@kms_plane_alpha_blend@pipe-b-alpha-basic.html

  * igt@kms_plane_alpha_blend@pipe-c-constant-alpha-max:
    - shard-glk:          NOTRUN -> [FAIL][104] ([fdo#108145] / [i915#265])
   [104]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk7/igt@kms_plane_alpha_blend@pipe-c-constant-alpha-max.html
    - shard-kbl:          NOTRUN -> [FAIL][105] ([fdo#108145] / [i915#265])
   [105]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl4/igt@kms_plane_alpha_blend@pipe-c-constant-alpha-max.html

  * igt@kms_plane_lowres@pipe-a-tiling-none:
    - shard-tglb:         NOTRUN -> [SKIP][106] ([i915#3536]) +3 similar issues
   [106]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb1/igt@kms_plane_lowres@pipe-a-tiling-none.html

  * igt@kms_plane_lowres@pipe-c-tiling-y:
    - shard-iclb:         NOTRUN -> [SKIP][107] ([i915#3536])
   [107]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb1/igt@kms_plane_lowres@pipe-c-tiling-y.html

  * igt@kms_plane_scaling@scaler-with-pixel-format-unity-scaling@pipe-b-edp-1-scaler-with-pixel-format:
    - shard-iclb:         [PASS][108] -> [INCOMPLETE][109] ([i915#5395])
   [108]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb1/igt@kms_plane_scaling@scaler-with-pixel-format-unity-scaling@pipe-b-edp-1-scaler-with-pixel-format.html
   [109]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb2/igt@kms_plane_scaling@scaler-with-pixel-format-unity-scaling@pipe-b-edp-1-scaler-with-pixel-format.html

  * igt@kms_psr2_su@frontbuffer-xrgb8888:
    - shard-iclb:         [PASS][110] -> [SKIP][111] ([fdo#109642] / [fdo#111068] / [i915#658])
   [110]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb2/igt@kms_psr2_su@frontbuffer-xrgb8888.html
   [111]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb4/igt@kms_psr2_su@frontbuffer-xrgb8888.html

  * igt@kms_psr2_su@page_flip-xrgb8888:
    - shard-apl:          NOTRUN -> [SKIP][112] ([fdo#109271] / [i915#658]) +1 similar issue
   [112]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl4/igt@kms_psr2_su@page_flip-xrgb8888.html

  * igt@kms_psr@psr2_cursor_blt:
    - shard-iclb:         NOTRUN -> [SKIP][113] ([fdo#109441]) +2 similar issues
   [113]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb1/igt@kms_psr@psr2_cursor_blt.html

  * igt@kms_psr@psr2_sprite_blt:
    - shard-tglb:         NOTRUN -> [FAIL][114] ([i915#132] / [i915#3467]) +5 similar issues
   [114]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb5/igt@kms_psr@psr2_sprite_blt.html

  * igt@kms_psr@psr2_sprite_mmap_gtt:
    - shard-iclb:         [PASS][115] -> [SKIP][116] ([fdo#109441]) +1 similar issue
   [115]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb2/igt@kms_psr@psr2_sprite_mmap_gtt.html
   [116]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb7/igt@kms_psr@psr2_sprite_mmap_gtt.html

  * igt@kms_rotation_crc@primary-yf-tiled-reflect-x-0:
    - shard-tglb:         NOTRUN -> [SKIP][117] ([fdo#111615] / [i915#5289])
   [117]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb7/igt@kms_rotation_crc@primary-yf-tiled-reflect-x-0.html

  * igt@kms_sysfs_edid_timing:
    - shard-kbl:          NOTRUN -> [FAIL][118] ([IGT#2])
   [118]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl3/igt@kms_sysfs_edid_timing.html

  * igt@kms_vblank@pipe-a-ts-continuation-dpms-suspend:
    - shard-kbl:          [PASS][119] -> [INCOMPLETE][120] ([i915#3614])
   [119]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-kbl4/igt@kms_vblank@pipe-a-ts-continuation-dpms-suspend.html
   [120]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl4/igt@kms_vblank@pipe-a-ts-continuation-dpms-suspend.html

  * igt@kms_writeback@writeback-pixel-formats:
    - shard-apl:          NOTRUN -> [SKIP][121] ([fdo#109271] / [i915#2437])
   [121]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl7/igt@kms_writeback@writeback-pixel-formats.html

  * igt@nouveau_crc@pipe-b-ctx-flip-skip-current-frame:
    - shard-apl:          NOTRUN -> [SKIP][122] ([fdo#109271]) +212 similar issues
   [122]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl4/igt@nouveau_crc@pipe-b-ctx-flip-skip-current-frame.html

  * igt@nouveau_crc@pipe-b-source-rg:
    - shard-iclb:         NOTRUN -> [SKIP][123] ([i915#2530]) +2 similar issues
   [123]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb7/igt@nouveau_crc@pipe-b-source-rg.html

  * igt@nouveau_crc@pipe-c-source-outp-complete:
    - shard-tglb:         NOTRUN -> [SKIP][124] ([i915#2530]) +3 similar issues
   [124]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb2/igt@nouveau_crc@pipe-c-source-outp-complete.html

  * igt@perf@per-context-mode-unprivileged:
    - shard-iclb:         NOTRUN -> [SKIP][125] ([fdo#109289])
   [125]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb1/igt@perf@per-context-mode-unprivileged.html

  * igt@prime_nv_api@i915_nv_import_twice:
    - shard-iclb:         NOTRUN -> [SKIP][126] ([fdo#109291]) +1 similar issue
   [126]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb5/igt@prime_nv_api@i915_nv_import_twice.html

  * igt@prime_nv_pcopy@test3_1:
    - shard-tglb:         NOTRUN -> [SKIP][127] ([fdo#109291]) +5 similar issues
   [127]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb2/igt@prime_nv_pcopy@test3_1.html

  * igt@prime_vgem@fence-read-hang:
    - shard-tglb:         NOTRUN -> [SKIP][128] ([fdo#109295])
   [128]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb7/igt@prime_vgem@fence-read-hang.html
    - shard-iclb:         NOTRUN -> [SKIP][129] ([fdo#109295])
   [129]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb8/igt@prime_vgem@fence-read-hang.html

  * igt@syncobj_timeline@transfer-timeline-point:
    - shard-iclb:         NOTRUN -> [DMESG-FAIL][130] ([i915#5098])
   [130]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb4/igt@syncobj_timeline@transfer-timeline-point.html
    - shard-kbl:          NOTRUN -> [DMESG-FAIL][131] ([i915#5098])
   [131]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl6/igt@syncobj_timeline@transfer-timeline-point.html
    - shard-apl:          NOTRUN -> [DMESG-FAIL][132] ([i915#5098])
   [132]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl2/igt@syncobj_timeline@transfer-timeline-point.html
    - shard-glk:          NOTRUN -> [DMESG-FAIL][133] ([i915#5098])
   [133]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk8/igt@syncobj_timeline@transfer-timeline-point.html
    - shard-tglb:         NOTRUN -> [DMESG-FAIL][134] ([i915#5098])
   [134]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb3/igt@syncobj_timeline@transfer-timeline-point.html
    - shard-snb:          NOTRUN -> [DMESG-FAIL][135] ([i915#5098])
   [135]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-snb4/igt@syncobj_timeline@transfer-timeline-point.html

  * igt@sysfs_clients@fair-0:
    - shard-tglb:         NOTRUN -> [SKIP][136] ([i915#2994]) +1 similar issue
   [136]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb3/igt@sysfs_clients@fair-0.html
    - shard-iclb:         NOTRUN -> [SKIP][137] ([i915#2994])
   [137]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb4/igt@sysfs_clients@fair-0.html

  * igt@sysfs_clients@sema-25:
    - shard-kbl:          NOTRUN -> [SKIP][138] ([fdo#109271] / [i915#2994]) +2 similar issues
   [138]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl4/igt@sysfs_clients@sema-25.html

  
#### Possible fixes ####

  * igt@feature_discovery@psr2:
    - {shard-rkl}:        [SKIP][139] ([i915#658]) -> [PASS][140]
   [139]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-4/igt@feature_discovery@psr2.html
   [140]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@feature_discovery@psr2.html

  * igt@gem_busy@close-race:
    - shard-glk:          [INCOMPLETE][141] -> [PASS][142]
   [141]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-glk3/igt@gem_busy@close-race.html
   [142]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk4/igt@gem_busy@close-race.html

  * igt@gem_eio@unwedge-stress:
    - shard-iclb:         [TIMEOUT][143] ([i915#3070]) -> [PASS][144]
   [143]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb1/igt@gem_eio@unwedge-stress.html
   [144]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb3/igt@gem_eio@unwedge-stress.html

  * igt@gem_exec_fair@basic-flow@rcs0:
    - shard-tglb:         [FAIL][145] ([i915#2842]) -> [PASS][146]
   [145]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-tglb8/igt@gem_exec_fair@basic-flow@rcs0.html
   [146]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb3/igt@gem_exec_fair@basic-flow@rcs0.html

  * igt@gem_exec_fair@basic-none-solo@rcs0:
    - shard-apl:          [FAIL][147] ([i915#2842]) -> [PASS][148]
   [147]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-apl6/igt@gem_exec_fair@basic-none-solo@rcs0.html
   [148]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl7/igt@gem_exec_fair@basic-none-solo@rcs0.html

  * igt@gem_exec_fair@basic-pace@rcs0:
    - shard-kbl:          [FAIL][149] ([i915#2842]) -> [PASS][150]
   [149]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-kbl4/igt@gem_exec_fair@basic-pace@rcs0.html
   [150]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl6/igt@gem_exec_fair@basic-pace@rcs0.html

  * igt@gem_exec_fair@basic-pace@vecs0:
    - shard-glk:          [FAIL][151] ([i915#2842]) -> [PASS][152] +3 similar issues
   [151]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-glk4/igt@gem_exec_fair@basic-pace@vecs0.html
   [152]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk2/igt@gem_exec_fair@basic-pace@vecs0.html

  * igt@gem_exec_fair@basic-throttle@rcs0:
    - shard-iclb:         [FAIL][153] ([i915#2849]) -> [PASS][154]
   [153]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb8/igt@gem_exec_fair@basic-throttle@rcs0.html
   [154]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb3/igt@gem_exec_fair@basic-throttle@rcs0.html

  * igt@gem_exec_flush@basic-uc-rw-default:
    - shard-snb:          [SKIP][155] ([fdo#109271]) -> [PASS][156] +1 similar issue
   [155]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-snb6/igt@gem_exec_flush@basic-uc-rw-default.html
   [156]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-snb5/igt@gem_exec_flush@basic-uc-rw-default.html

  * igt@i915_pm_dc@dc6-psr:
    - shard-iclb:         [FAIL][157] ([i915#454]) -> [PASS][158]
   [157]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb6/igt@i915_pm_dc@dc6-psr.html
   [158]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb1/igt@i915_pm_dc@dc6-psr.html

  * igt@i915_pm_rpm@cursor-dpms:
    - {shard-rkl}:        [SKIP][159] ([i915#1849]) -> [PASS][160]
   [159]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-4/igt@i915_pm_rpm@cursor-dpms.html
   [160]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@i915_pm_rpm@cursor-dpms.html

  * igt@kms_async_flips@async-flip-with-page-flip-events:
    - {shard-rkl}:        [SKIP][161] ([i915#1845] / [i915#4098]) -> [PASS][162] +15 similar issues
   [161]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-4/igt@kms_async_flips@async-flip-with-page-flip-events.html
   [162]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_async_flips@async-flip-with-page-flip-events.html

  * igt@kms_atomic@atomic_plane_damage:
    - {shard-rkl}:        [SKIP][163] ([i915#4098]) -> [PASS][164]
   [163]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-4/igt@kms_atomic@atomic_plane_damage.html
   [164]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_atomic@atomic_plane_damage.html

  * igt@kms_color@pipe-b-ctm-0-5:
    - {shard-rkl}:        [SKIP][165] ([i915#1149] / [i915#1849] / [i915#4070] / [i915#4098]) -> [PASS][166] +1 similar issue
   [165]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-2/igt@kms_color@pipe-b-ctm-0-5.html
   [166]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_color@pipe-b-ctm-0-5.html

  * igt@kms_cursor_crc@pipe-b-cursor-suspend:
    - {shard-rkl}:        [SKIP][167] ([fdo#112022] / [i915#4070]) -> [PASS][168] +3 similar issues
   [167]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-4/igt@kms_cursor_crc@pipe-b-cursor-suspend.html
   [168]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_cursor_crc@pipe-b-cursor-suspend.html

  * igt@kms_cursor_crc@pipe-d-cursor-suspend:
    - {shard-tglu}:       [INCOMPLETE][169] ([i915#1982]) -> [PASS][170]
   [169]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-tglu-3/igt@kms_cursor_crc@pipe-d-cursor-suspend.html
   [170]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglu-2/igt@kms_cursor_crc@pipe-d-cursor-suspend.html

  * igt@kms_cursor_edge_walk@pipe-b-256x256-left-edge:
    - {shard-rkl}:        [SKIP][171] ([i915#1849] / [i915#4070] / [i915#4098]) -> [PASS][172] +2 similar issues
   [171]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-2/igt@kms_cursor_edge_walk@pipe-b-256x256-left-edge.html
   [172]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_cursor_edge_walk@pipe-b-256x256-left-edge.html

  * igt@kms_cursor_edge_walk@pipe-c-128x128-top-edge:
    - shard-glk:          [FAIL][173] ([i915#1888]) -> [PASS][174]
   [173]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-glk6/igt@kms_cursor_edge_walk@pipe-c-128x128-top-edge.html
   [174]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk7/igt@kms_cursor_edge_walk@pipe-c-128x128-top-edge.html

  * igt@kms_cursor_legacy@cursora-vs-flipa-legacy:
    - {shard-rkl}:        [SKIP][175] ([fdo#111825] / [i915#4070]) -> [PASS][176] +1 similar issue
   [175]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-4/igt@kms_cursor_legacy@cursora-vs-flipa-legacy.html
   [176]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_cursor_legacy@cursora-vs-flipa-legacy.html

  * igt@kms_cursor_legacy@flip-vs-cursor-toggle:
    - shard-iclb:         [FAIL][177] ([i915#2346]) -> [PASS][178]
   [177]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb7/igt@kms_cursor_legacy@flip-vs-cursor-toggle.html
   [178]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb1/igt@kms_cursor_legacy@flip-vs-cursor-toggle.html

  * igt@kms_draw_crc@draw-method-xrgb2101010-blt-ytiled:
    - {shard-rkl}:        [SKIP][179] ([i915#4098] / [i915#4369]) -> [PASS][180] +1 similar issue
   [179]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-4/igt@kms_draw_crc@draw-method-xrgb2101010-blt-ytiled.html
   [180]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_draw_crc@draw-method-xrgb2101010-blt-ytiled.html

  * igt@kms_draw_crc@draw-method-xrgb2101010-render-untiled:
    - {shard-rkl}:        [SKIP][181] ([fdo#111314] / [i915#4098] / [i915#4369]) -> [PASS][182] +2 similar issues
   [181]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-5/igt@kms_draw_crc@draw-method-xrgb2101010-render-untiled.html
   [182]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_draw_crc@draw-method-xrgb2101010-render-untiled.html

  * igt@kms_flip@flip-vs-suspend@c-dp1:
    - shard-kbl:          [INCOMPLETE][183] ([i915#3614]) -> [PASS][184]
   [183]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-kbl3/igt@kms_flip@flip-vs-suspend@c-dp1.html
   [184]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-kbl6/igt@kms_flip@flip-vs-suspend@c-dp1.html

  * igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling:
    - shard-iclb:         [SKIP][185] ([i915#3701]) -> [PASS][186]
   [185]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb2/igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling.html
   [186]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb6/igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling.html

  * igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-indfb-draw-mmap-gtt:
    - {shard-rkl}:        [SKIP][187] ([i915#1849] / [i915#4098]) -> [PASS][188] +14 similar issues
   [187]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-4/igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-indfb-draw-mmap-gtt.html
   [188]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-indfb-draw-mmap-gtt.html

  * igt@kms_invalid_mode@bad-hsync-end:
    - {shard-rkl}:        [SKIP][189] ([i915#4278]) -> [PASS][190]
   [189]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-1/igt@kms_invalid_mode@bad-hsync-end.html
   [190]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_invalid_mode@bad-hsync-end.html

  * igt@kms_plane@plane-position-hole@pipe-b-planes:
    - {shard-rkl}:        [SKIP][191] ([i915#3558]) -> [PASS][192] +1 similar issue
   [191]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-1/igt@kms_plane@plane-position-hole@pipe-b-planes.html
   [192]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_plane@plane-position-hole@pipe-b-planes.html

  * igt@kms_plane_alpha_blend@pipe-b-alpha-opaque-fb:
    - {shard-rkl}:        [SKIP][193] ([i915#4070] / [i915#4098]) -> [PASS][194]
   [193]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-4/igt@kms_plane_alpha_blend@pipe-b-alpha-opaque-fb.html
   [194]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_plane_alpha_blend@pipe-b-alpha-opaque-fb.html

  * igt@kms_psr@primary_mmap_cpu:
    - {shard-rkl}:        [SKIP][195] ([i915#1072]) -> [PASS][196]
   [195]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-4/igt@kms_psr@primary_mmap_cpu.html
   [196]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_psr@primary_mmap_cpu.html

  * igt@kms_psr@psr2_primary_page_flip:
    - shard-iclb:         [SKIP][197] ([fdo#109441]) -> [PASS][198] +1 similar issue
   [197]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb3/igt@kms_psr@psr2_primary_page_flip.html
   [198]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb2/igt@kms_psr@psr2_primary_page_flip.html

  * igt@kms_psr_stress_test@invalidate-primary-flip-overlay:
    - {shard-rkl}:        [SKIP][199] ([i915#5461]) -> [PASS][200]
   [199]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-2/igt@kms_psr_stress_test@invalidate-primary-flip-overlay.html
   [200]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_psr_stress_test@invalidate-primary-flip-overlay.html

  * igt@kms_rotation_crc@primary-rotation-270:
    - shard-glk:          [FAIL][201] ([i915#1888] / [i915#65]) -> [PASS][202]
   [201]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-glk2/igt@kms_rotation_crc@primary-rotation-270.html
   [202]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-glk9/igt@kms_rotation_crc@primary-rotation-270.html

  * igt@kms_universal_plane@universal-plane-pipe-b-functional:
    - {shard-rkl}:        [SKIP][203] ([i915#1845] / [i915#4070] / [i915#4098]) -> [PASS][204]
   [203]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-rkl-1/igt@kms_universal_plane@universal-plane-pipe-b-functional.html
   [204]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-rkl-6/igt@kms_universal_plane@universal-plane-pipe-b-functional.html

  * igt@kms_vblank@pipe-c-ts-continuation-suspend:
    - shard-apl:          [DMESG-WARN][205] ([i915#180]) -> [PASS][206] +1 similar issue
   [205]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-apl2/igt@kms_vblank@pipe-c-ts-continuation-suspend.html
   [206]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl4/igt@kms_vblank@pipe-c-ts-continuation-suspend.html

  * igt@testdisplay:
    - {shard-tglu}:       [DMESG-WARN][207] ([i915#4941]) -> [PASS][208]
   [207]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-tglu-5/igt@testdisplay.html
   [208]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglu-1/igt@testdisplay.html

  
#### Warnings ####

  * igt@gem_eio@unwedge-stress:
    - shard-tglb:         [TIMEOUT][209] ([i915#3063]) -> [FAIL][210] ([i915#5784])
   [209]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-tglb6/igt@gem_eio@unwedge-stress.html
   [210]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-tglb2/igt@gem_eio@unwedge-stress.html

  * igt@gem_exec_balancer@parallel-out-fence:
    - shard-iclb:         [SKIP][211] ([i915#4525]) -> [DMESG-WARN][212] ([i915#5614])
   [211]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb7/igt@gem_exec_balancer@parallel-out-fence.html
   [212]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb2/igt@gem_exec_balancer@parallel-out-fence.html

  * igt@kms_psr2_sf@cursor-plane-update-sf:
    - shard-iclb:         [SKIP][213] ([fdo#111068] / [i915#658]) -> [SKIP][214] ([i915#2920])
   [213]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb4/igt@kms_psr2_sf@cursor-plane-update-sf.html
   [214]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb2/igt@kms_psr2_sf@cursor-plane-update-sf.html

  * igt@kms_psr2_sf@overlay-plane-move-continuous-sf:
    - shard-iclb:         [SKIP][215] ([i915#2920]) -> [SKIP][216] ([i915#658])
   [215]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-iclb2/igt@kms_psr2_sf@overlay-plane-move-continuous-sf.html
   [216]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-iclb7/igt@kms_psr2_sf@overlay-plane-move-continuous-sf.html

  * igt@runner@aborted:
    - shard-apl:          ([FAIL][217], [FAIL][218], [FAIL][219], [FAIL][220], [FAIL][221], [FAIL][222], [FAIL][223], [FAIL][224]) ([i915#180] / [i915#3002] / [i915#4312] / [i915#5257]) -> ([FAIL][225], [FAIL][226], [FAIL][227], [FAIL][228], [FAIL][229], [FAIL][230], [FAIL][231], [FAIL][232]) ([fdo#109271] / [i915#180] / [i915#3002] / [i915#4312] / [i915#5257])
   [217]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-apl6/igt@runner@aborted.html
   [218]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-apl4/igt@runner@aborted.html
   [219]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-apl4/igt@runner@aborted.html
   [220]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-apl3/igt@runner@aborted.html
   [221]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-apl8/igt@runner@aborted.html
   [222]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-apl2/igt@runner@aborted.html
   [223]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-apl2/igt@runner@aborted.html
   [224]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6462/shard-apl1/igt@runner@aborted.html
   [225]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl1/igt@runner@aborted.html
   [226]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl3/igt@runner@aborted.html
   [227]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl8/igt@runner@aborted.html
   [228]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl2/igt@runner@aborted.html
   [229]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl1/igt@runner@aborted.html
   [230]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl3/igt@runner@aborted.html
   [231]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl2/igt@runner@aborted.html
   [232]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/shard-apl1/igt@runner@aborted.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [IGT#2]: https://gitlab.freedesktop.org/drm/igt-gpu-tools/issues/2
  [fdo#108145]: https://bugs.freedesktop.org/show_bug.cgi?id=108145
  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#109274]: https://bugs.freedesktop.org/show_bug.cgi?id=109274
  [fdo#109278]: https://bugs.freedesktop.org/show_bug.cgi?id=109278
  [fdo#109279]: https://bugs.freedesktop.org/show_bug.cgi?id=109279
  [fdo#109280]: https://bugs.freedesktop.org/show_bug.cgi?id=109280
  [fdo#109283]: https://bugs.freedesktop.org/show_bug.cgi?id=109283
  [fdo#109284]: https://bugs.freedesktop.org/show_bug.cgi?id=109284
  [fdo#109289]: https://bugs.freedesktop.org/show_bug.cgi?id=109289
  [fdo#109291]: https://bugs.freedesktop.org/show_bug.cgi?id=109291
  [fdo#109295]: https://bugs.freedesktop.org/show_bug.cgi?id=109295
  [fdo#109300]: https://bugs.freedesktop.org/show_bug.cgi?id=109300
  [fdo#109302]: https://bugs.freedesktop.org/show_bug.cgi?id=109302
  [fdo#109303]: https://bugs.freedesktop.org/show_bug.cgi?id=109303
  [fdo#109308]: https://bugs.freedesktop.org/show_bug.cgi?id=109308
  [fdo#109312]: https://bugs.freedesktop.org/show_bug.cgi?id=109312
  [fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
  [fdo#109642]: https://bugs.freedesktop.org/show_bug.cgi?id=109642
  [fdo#110189]: https://bugs.freedesktop.org/show_bug.cgi?id=110189
  [fdo#110542]: https://bugs.freedesktop.org/show_bug.cgi?id=110542
  [fdo#110723]: https://bugs.freedesktop.org/show_bug.cgi?id=110723
  [fdo#110725]: https://bugs.freedesktop.org/show_bug.cgi?id=110725
  [fdo#111066]: https://bugs.freedesktop.org/show_bug.cgi?id=111066
  [fdo#111068]: https://bugs.freedesktop.org/show_bug.cgi?id=111068
  [fdo#111314]: https://bugs.freedesktop.org/show_bug.cgi?id=111314
  [fdo#111614]: https://bugs.freedesktop.org/show_bug.cgi?id=111614
  [fdo#111615]: https://bugs.freedesktop.org/show_bug.cgi?id=111615
  [fdo#111644]: https://bugs.freedesktop.org/show_bug.cgi?id=111644
  [fdo#111656]: https://bugs.freedesktop.org/show_bug.cgi?id=111656
  [fdo#111825]: https://bugs.freedesktop.org/show_bug.cgi?id=111825
  [fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
  [fdo#112022]: https://bugs.freedesktop.org/show_bug.cgi?id=112022
  [fdo#112054]: https://bugs.freedesktop.org/show_bug.cgi?id=112054
  [fdo#112283]: https://bugs.freedesktop.org/show_bug.cgi?id=112283
  [i915#1063]: https://gitlab.freedesktop.org/drm/intel/issues/1063
  [i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
  [i915#1149]: https://gitlab.freedesktop.org/drm/intel/issues/1149
  [i915#1155]: https://gitlab.freedesktop.org/drm/intel/issues/1155
  [i915#118]: https://gitlab.freedesktop.org/drm/intel/issues/118
  [i915#1319]: https://gitlab.freedesktop.org/drm/intel/issues/1319
  [i915#132]: https://gitlab.freedesktop.org/drm/intel/issues/132
  [i915#1397]: https://gitlab.freedesktop.org/drm/intel/issues/1397
  [i915#1769]: https://gitlab.freedesktop.org/drm/intel/issues/1769
  [i915#180]: https://gitlab.freedesktop.org/drm/intel/issues/180
  [i915#1825]: https://gitlab.freedesktop.org/drm/intel/issues/1825
  [i915#1839]: https://gitlab.freedesktop.org/drm/intel/issues/1839
  [i915#1845]: https://gitlab.freedesktop.org/drm/intel/issues/1845
  [i915#1849]: https://gitlab.freedesktop.org/drm/intel/issues/1849
  [i915#1888]: https://gitlab.freedesktop.org/drm/intel/issues/1888
  [i915#1902]: https://gitlab.freedesktop.org/drm/intel/issues/1902
  [i915#1911]: https://gitlab.freedesktop.org/drm/intel/issues/1911
  [i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
  [i915#2105]: https://gitlab.freedesktop.org/drm/intel/issues/2105
  [i915#2346]: https://gitlab.freedesktop.org/drm/intel/issues/2346
  [i915#2411]: https://gitlab.freedesktop.org/drm/intel/issues/2411
  [i915#2436]: https://gitlab.freedesktop.org/drm/intel/issues/2436
  [i915#2437]: https://gitlab.freedesktop.org/drm/intel/issues/2437
  [i915#2527]: https://gitlab.freedesktop.org/drm/intel/issues/2527
  [i915#2530]: https://gitlab.freedesktop.org/drm/intel/issues/2530
  [i915#2582]: https://gitlab.freedesktop.org/drm/intel/issues/2582
  [i915#2587]: https://gitlab.freedesktop.org/drm/intel/issues/2587
  [i915#265]: https://gitlab.freedesktop.org/drm/intel/issues/265
  [i915#2658]: https://gitlab.freedesktop.org/drm/intel/issues/2658
  [i915#2705]: https://gitlab.freedesktop.org/drm/intel/issues/2705
  [i915#280]: https://gitlab.freedesktop.org/drm/intel/issues/280
  [i915#2842]: https://gitlab.freedesktop.org/drm/intel/issues/2842
  [i915#2849]: https://gitlab.freedesktop.org/drm/intel/issues/2849
  [i915#2856]: https://gitlab.freedesktop.org/drm/intel/issues/2856
  [i915#2920]: https://gitlab.freedesktop.org/drm/intel/issues/2920
  [i915#2994]: https://gitlab.freedesktop.org/drm/intel/issues/2994
  [i915#3002]: https://gitlab.freedesktop.org/drm/intel/issues/3002
  [i915#3012]: https://gitlab.freedesktop.org/drm/intel/issues/3012
  [i915#3063]: https://gitlab.freedesktop.org/drm/intel/issues/3063
  [i915#3070]: https://gitlab.freedesktop.org/drm/intel/issues/3070
  [i915#3116]: https://gitlab.freedesktop.org/drm/intel/issues/3116
  [i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282
  [i915#3297]: https://gitlab.freedesktop.org/drm/intel/issues/3297
  [i915#3299]: https://gitlab.freedesktop.org/drm/intel/issues/3299
  [i915#3319]: https://gitlab.freedesktop.org/drm/intel/issues/3319
  [i915#3323]: https://gitlab.freedesktop.org/drm/intel/issues/3323
  [i915#3359]: https://gitlab.freedesktop.org/drm/intel/issues/3359
  [i915#3376]: https://gitlab.freedesktop.org/drm/intel/issues/3376
  [i915#3464]: https://gitlab.freedesktop.org/drm/intel/issues/3464
  [i915#3467]: https://gitlab.freedesktop.org/drm/intel/issues/3467
  [i915#3469]: https://gitlab.freedesktop.org/drm/intel/issues/3469
  [i915#3536]: https://gitlab.freedesktop.org/drm/intel/issues/3536
  [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
  [i915#3558]: https://gitlab.freedesktop.org/drm/intel/issues/3558
  [i915#3614]: https://gitlab.freedesktop.org/drm/intel/issues/3614
  [i915#3637]: https://gitlab.freedesktop.org/drm/intel/issues/3637
  [i915#3638]: https://gitlab.freedesktop.org/drm/intel/issues/3638
  [i915#3689]: https://gitlab.freedesktop.org/drm/intel/issues/3689
  [i915#3701]: https://gitlab.freedesktop.org/drm/intel/issues/3701
  [i915#3734]: https://gitlab.freedesktop.org/drm/intel/issues/3734
  [i915#3742]: https://gitlab.freedesktop.org/drm/intel/issues/3742
  [i915#3826]: https://gitlab.freedesktop.org/drm/intel/issues/3826
  [i915#3886]: https://gitlab.freedesktop.org/drm/intel/issues/3886
  [i915#3955]: https://gitlab.freedesktop.org/drm/intel/issues/3955
  [i915#4070]: https://gitlab.freedesktop.org/drm/intel/issues/4070
  [i915#4098]: https://gitlab.freedesktop.org/drm/intel/issues/4098
  [i915#4103]: https://gitlab.freedesktop.org/drm/intel/issues/4103
  [i915#4270]: https://gitlab.freedesktop.org/drm/intel/issues/4270
  [i915#4278]: https://gitlab.freedesktop.org/drm/intel/issues/4278
  [i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312
  [i915#4369]: https://gitlab.freedesktop.org/drm/intel/issues/4369
  [i915#4387]: https://gitlab.freedesktop.org/drm/intel/issues/4387
  [i915#4525]: https://gitlab.freedesktop.org/drm/intel/issues/4525
  [i915#454]: https://gitlab.freedesktop.org/drm/intel/issues/454
  [i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
  [i915#4941]: https://gitlab.freedesktop.org/drm/intel/issues/4941
  [i915#4991]: https://gitlab.freedesktop.org/drm/intel/issues/4991
  [i915#5076]: https://gitlab.freedesktop.org/drm/intel/issues/5076
  [i915#5098]: https://gitlab.freedesktop.org/drm/intel/issues/5098
  [i915#5257]: https://gitlab.freedesktop.org/drm/intel/issues/5257
  [i915#5286]: https://gitlab.freedesktop.org/drm/intel/issues/5286
  [i915#5287]: https://gitlab.freedesktop.org/drm/intel/issues/5287
  [i915#5289]: https://gitlab.freedesktop.org/drm/intel/issues/5289
  [i915#5325]: https://gitlab.freedesktop.org/drm/intel/issues/5325
  [i915#5327]: https://gitlab.freedesktop.org/drm/intel/issues/5327
  [i915#533]: https://gitlab.freedesktop.org/drm/intel/issues/533
  [i915#5395]: https://gitlab.freedesktop.org/drm/intel/issues/5395
  [i915#5461]: https://gitlab.freedesktop.org/drm/intel/issues/5461
  [i915#5566]: https://gitlab.freedesktop.org/drm/intel/issues/5566
  [i915#5614]: https://gitlab.freedesktop.org/drm/intel/issues/5614
  [i915#5784]: https://gitlab.freedesktop.org/drm/intel/issues/5784
  [i915#65]: https://gitlab.freedesktop.org/drm/intel/issues/65
  [i915#658]: https://gitlab.freedesktop.org/drm/intel/issues/658
  [i915#716]: https://gitlab.freedesktop.org/drm/intel/issues/716
  [i915#768]: https://gitlab.freedesktop.org/drm/intel/issues/768


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_6462 -> IGTPW_7024

  CI-20190529: 20190529
  CI_DRM_11550: 56b089ae03ef8ea8ab7f474eaa70367898891ef0 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_7024: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/index.html
  IGT_6462: ca6db54be6bf51958ad99bb319bdb514ae58eb25 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7024/index.html

[-- Attachment #2: Type: text/html, Size: 71425 bytes --]