* ✓ CI.Patch_applied: success for drm/xe/pmu: Enable PMU interface (rev9)
2024-06-13 10:04 [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Riana Tauro
@ 2024-06-13 9:57 ` Patchwork
2024-06-13 9:57 ` ✗ CI.checkpatch: warning " Patchwork
` (8 subsequent siblings)
9 siblings, 0 replies; 32+ messages in thread
From: Patchwork @ 2024-06-13 9:57 UTC (permalink / raw)
To: Iddamsetty, Aravind; +Cc: intel-xe
== Series Details ==
Series: drm/xe/pmu: Enable PMU interface (rev9)
URL : https://patchwork.freedesktop.org/series/119504/
State : success
== Summary ==
=== Applying kernel patches on branch 'drm-tip' with base: ===
Base commit: dea4e8a3a088 drm-tip: 2024y-06m-13d-07h-32m-18s UTC integration manifest
=== git am output follows ===
.git/rebase-apply/patch:785: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
Applying: drm/xe: Get GT clock to nanosecs
Applying: drm/xe/pmu: Enable PMU interface
^ permalink raw reply [flat|nested] 32+ messages in thread
* ✗ CI.checkpatch: warning for drm/xe/pmu: Enable PMU interface (rev9)
2024-06-13 10:04 [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Riana Tauro
2024-06-13 9:57 ` ✓ CI.Patch_applied: success for drm/xe/pmu: Enable PMU interface (rev9) Patchwork
@ 2024-06-13 9:57 ` Patchwork
2024-06-13 9:59 ` ✓ CI.KUnit: success " Patchwork
` (7 subsequent siblings)
9 siblings, 0 replies; 32+ messages in thread
From: Patchwork @ 2024-06-13 9:57 UTC (permalink / raw)
To: Iddamsetty, Aravind; +Cc: intel-xe
== Series Details ==
Series: drm/xe/pmu: Enable PMU interface (rev9)
URL : https://patchwork.freedesktop.org/series/119504/
State : warning
== Summary ==
+ KERNEL=/kernel
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools mt
Cloning into 'mt'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ git -C mt rev-list -n1 origin/master
51ce9f6cd981d42d7467409d7dbc559a450abc1e
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ git log -n1
commit 334e94dc4406ba922f6ffdecc86368fcc39c3542
Author: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Date: Thu Jun 13 15:34:11 2024 +0530
drm/xe/pmu: Enable PMU interface
There are a set of engine group busyness counters provided by HW which are
perfect fit to be exposed via PMU perf events.
BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
events can be listed using:
perf list
xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
and can be read using:
perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
time counts unit events
1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
The pmu base implementation is taken from i915.
v2:
Store last known value when device is awake return that while the GT is
suspended and then update the driver copy when read during awake.
v3:
1. drop init_samples, as storing counters before going to suspend should
be sufficient.
2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
dropped helpers to store and read samples.
3. use xe_device_mem_access_get_if_ongoing to check if device is active
before reading the OA registers.
4. dropped format attr as no longer needed
5. introduce xe_pmu_suspend to call engine_group_busyness_store
6. few other nits.
v4: minor nits.
v5: take forcewake when accessing the OAG registers
v6:
1. drop engine_busyness_sample_type
2. update UAPI documentation
v7:
1. update UAPI documentation
2. drop MEDIA_GT specific change for media busyness counter.
v8:
1. rebase
2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
3. remove interrupts pmu event
v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
+ /mt/dim checkpatch dea4e8a3a0889d80afc874513cca0d7f7a25bbcd drm-intel
db9cd9cccc7d drm/xe: Get GT clock to nanosecs
334e94dc4406 drm/xe/pmu: Enable PMU interface
Traceback (most recent call last):
File "scripts/spdxcheck.py", line 6, in <module>
from ply import lex, yacc
ModuleNotFoundError: No module named 'ply'
Traceback (most recent call last):
File "scripts/spdxcheck.py", line 6, in <module>
from ply import lex, yacc
ModuleNotFoundError: No module named 'ply'
Traceback (most recent call last):
File "scripts/spdxcheck.py", line 6, in <module>
from ply import lex, yacc
ModuleNotFoundError: No module named 'ply'
-:179: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating?
#179:
new file mode 100644
total: 0 errors, 1 warnings, 0 checks, 837 lines checked
^ permalink raw reply [flat|nested] 32+ messages in thread
* ✓ CI.KUnit: success for drm/xe/pmu: Enable PMU interface (rev9)
2024-06-13 10:04 [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Riana Tauro
2024-06-13 9:57 ` ✓ CI.Patch_applied: success for drm/xe/pmu: Enable PMU interface (rev9) Patchwork
2024-06-13 9:57 ` ✗ CI.checkpatch: warning " Patchwork
@ 2024-06-13 9:59 ` Patchwork
2024-06-13 10:04 ` [PATCH v9 1/2] drm/xe: Get GT clock to nanosecs Riana Tauro
` (6 subsequent siblings)
9 siblings, 0 replies; 32+ messages in thread
From: Patchwork @ 2024-06-13 9:59 UTC (permalink / raw)
To: Iddamsetty, Aravind; +Cc: intel-xe
== Series Details ==
Series: drm/xe/pmu: Enable PMU interface (rev9)
URL : https://patchwork.freedesktop.org/series/119504/
State : success
== Summary ==
+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig
[09:57:57] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[09:58:01] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make ARCH=um O=.kunit --jobs=48
../lib/iomap.c:156:5: warning: no previous prototype for ‘ioread64_lo_hi’ [-Wmissing-prototypes]
156 | u64 ioread64_lo_hi(const void __iomem *addr)
| ^~~~~~~~~~~~~~
../lib/iomap.c:163:5: warning: no previous prototype for ‘ioread64_hi_lo’ [-Wmissing-prototypes]
163 | u64 ioread64_hi_lo(const void __iomem *addr)
| ^~~~~~~~~~~~~~
../lib/iomap.c:170:5: warning: no previous prototype for ‘ioread64be_lo_hi’ [-Wmissing-prototypes]
170 | u64 ioread64be_lo_hi(const void __iomem *addr)
| ^~~~~~~~~~~~~~~~
../lib/iomap.c:178:5: warning: no previous prototype for ‘ioread64be_hi_lo’ [-Wmissing-prototypes]
178 | u64 ioread64be_hi_lo(const void __iomem *addr)
| ^~~~~~~~~~~~~~~~
../lib/iomap.c:264:6: warning: no previous prototype for ‘iowrite64_lo_hi’ [-Wmissing-prototypes]
264 | void iowrite64_lo_hi(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~
../lib/iomap.c:272:6: warning: no previous prototype for ‘iowrite64_hi_lo’ [-Wmissing-prototypes]
272 | void iowrite64_hi_lo(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~
../lib/iomap.c:280:6: warning: no previous prototype for ‘iowrite64be_lo_hi’ [-Wmissing-prototypes]
280 | void iowrite64be_lo_hi(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~~~
../lib/iomap.c:288:6: warning: no previous prototype for ‘iowrite64be_hi_lo’ [-Wmissing-prototypes]
288 | void iowrite64be_hi_lo(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~~~
[09:58:27] Starting KUnit Kernel (1/1)...
[09:58:27] ============================================================
Running tests with:
$ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt
[09:58:27] =================== guc_dbm (7 subtests) ===================
[09:58:27] [PASSED] test_empty
[09:58:27] [PASSED] test_default
[09:58:27] ======================== test_size ========================
[09:58:27] [PASSED] 4
[09:58:27] [PASSED] 8
[09:58:27] [PASSED] 32
[09:58:27] [PASSED] 256
[09:58:27] ==================== [PASSED] test_size ====================
[09:58:27] ======================= test_reuse ========================
[09:58:27] [PASSED] 4
[09:58:27] [PASSED] 8
[09:58:27] [PASSED] 32
[09:58:27] [PASSED] 256
[09:58:27] =================== [PASSED] test_reuse ====================
[09:58:27] =================== test_range_overlap ====================
[09:58:27] [PASSED] 4
[09:58:27] [PASSED] 8
[09:58:27] [PASSED] 32
[09:58:27] [PASSED] 256
[09:58:27] =============== [PASSED] test_range_overlap ================
[09:58:27] =================== test_range_compact ====================
[09:58:27] [PASSED] 4
[09:58:27] [PASSED] 8
[09:58:27] [PASSED] 32
[09:58:27] [PASSED] 256
[09:58:27] =============== [PASSED] test_range_compact ================
[09:58:27] ==================== test_range_spare =====================
[09:58:27] [PASSED] 4
[09:58:27] [PASSED] 8
[09:58:27] [PASSED] 32
[09:58:27] [PASSED] 256
[09:58:27] ================ [PASSED] test_range_spare =================
[09:58:27] ===================== [PASSED] guc_dbm =====================
[09:58:27] =================== guc_idm (6 subtests) ===================
[09:58:27] [PASSED] bad_init
[09:58:27] [PASSED] no_init
[09:58:27] [PASSED] init_fini
[09:58:27] [PASSED] check_used
[09:58:27] [PASSED] check_quota
[09:58:27] [PASSED] check_all
[09:58:27] ===================== [PASSED] guc_idm =====================
[09:58:27] ================== no_relay (3 subtests) ===================
[09:58:27] [PASSED] xe_drops_guc2pf_if_not_ready
[09:58:27] [PASSED] xe_drops_guc2vf_if_not_ready
[09:58:27] [PASSED] xe_rejects_send_if_not_ready
[09:58:27] ==================== [PASSED] no_relay =====================
[09:58:27] ================== pf_relay (14 subtests) ==================
[09:58:27] [PASSED] pf_rejects_guc2pf_too_short
[09:58:27] [PASSED] pf_rejects_guc2pf_too_long
[09:58:27] [PASSED] pf_rejects_guc2pf_no_payload
[09:58:27] [PASSED] pf_fails_no_payload
[09:58:27] [PASSED] pf_fails_bad_origin
[09:58:27] [PASSED] pf_fails_bad_type
[09:58:27] [PASSED] pf_txn_reports_error
[09:58:27] [PASSED] pf_txn_sends_pf2guc
[09:58:27] [PASSED] pf_sends_pf2guc
[09:58:27] [SKIPPED] pf_loopback_nop
[09:58:27] [SKIPPED] pf_loopback_echo
[09:58:27] [SKIPPED] pf_loopback_fail
[09:58:27] [SKIPPED] pf_loopback_busy
[09:58:27] [SKIPPED] pf_loopback_retry
[09:58:27] ==================== [PASSED] pf_relay =====================
[09:58:27] ================== vf_relay (3 subtests) ===================
[09:58:27] [PASSED] vf_rejects_guc2vf_too_short
[09:58:27] [PASSED] vf_rejects_guc2vf_too_long
[09:58:27] [PASSED] vf_rejects_guc2vf_no_payload
[09:58:27] ==================== [PASSED] vf_relay =====================
[09:58:27] ================= pf_service (11 subtests) =================
[09:58:27] [PASSED] pf_negotiate_any
[09:58:27] [PASSED] pf_negotiate_base_match
[09:58:27] [PASSED] pf_negotiate_base_newer
[09:58:27] [PASSED] pf_negotiate_base_next
[09:58:27] [SKIPPED] pf_negotiate_base_older
[09:58:27] [PASSED] pf_negotiate_base_prev
[09:58:27] [PASSED] pf_negotiate_latest_match
[09:58:27] [PASSED] pf_negotiate_latest_newer
[09:58:27] [PASSED] pf_negotiate_latest_next
[09:58:27] [SKIPPED] pf_negotiate_latest_older
[09:58:27] [SKIPPED] pf_negotiate_latest_prev
[09:58:27] =================== [PASSED] pf_service ====================
[09:58:27] ===================== lmtt (1 subtest) =====================
[09:58:27] ======================== test_ops =========================
[09:58:27] [PASSED] 2-level
[09:58:27] [PASSED] multi-level
[09:58:27] ==================== [PASSED] test_ops =====================
[09:58:27] ====================== [PASSED] lmtt =======================
[09:58:27] ==================== xe_bo (2 subtests) ====================
[09:58:27] [SKIPPED] xe_ccs_migrate_kunit
[09:58:27] [SKIPPED] xe_bo_evict_kunit
[09:58:27] ===================== [SKIPPED] xe_bo ======================
[09:58:27] ================== xe_dma_buf (1 subtest) ==================
[09:58:27] [SKIPPED] xe_dma_buf_kunit
[09:58:27] =================== [SKIPPED] xe_dma_buf ===================
[09:58:27] ================== xe_migrate (1 subtest) ==================
[09:58:27] [SKIPPED] xe_migrate_sanity_kunit
[09:58:27] =================== [SKIPPED] xe_migrate ===================
[09:58:27] =================== xe_mocs (2 subtests) ===================
[09:58:27] [SKIPPED] xe_live_mocs_kernel_kunit
[09:58:27] [SKIPPED] xe_live_mocs_reset_kunit
[09:58:27] ==================== [SKIPPED] xe_mocs =====================
[09:58:27] ==================== args (11 subtests) ====================
[09:58:27] [PASSED] count_args_test
[09:58:27] [PASSED] call_args_example
[09:58:27] [PASSED] call_args_test
[09:58:27] [PASSED] drop_first_arg_example
[09:58:27] [PASSED] drop_first_arg_test
[09:58:27] [PASSED] first_arg_example
[09:58:27] [PASSED] first_arg_test
[09:58:27] [PASSED] last_arg_example
[09:58:27] [PASSED] last_arg_test
[09:58:27] [PASSED] pick_arg_example
[09:58:27] [PASSED] sep_comma_example
[09:58:27] ====================== [PASSED] args =======================
[09:58:27] =================== xe_pci (2 subtests) ====================
[09:58:27] [PASSED] xe_gmdid_graphics_ip
[09:58:27] [PASSED] xe_gmdid_media_ip
[09:58:27] ===================== [PASSED] xe_pci ======================
[09:58:27] ==================== xe_rtp (1 subtest) ====================
[09:58:27] ================== xe_rtp_process_tests ===================
[09:58:27] [PASSED] coalesce-same-reg
[09:58:27] [PASSED] no-match-no-add
[09:58:27] [PASSED] no-match-no-add-multiple-rules
[09:58:27] [PASSED] two-regs-two-entries
[09:58:27] [PASSED] clr-one-set-other
[09:58:27] [PASSED] set-field
[09:58:27] [PASSED] conflict-duplicate
[09:58:27] [PASSED] conflict-not-disjoint
[09:58:27] [PASSED] conflict-reg-type
[09:58:27] ============== [PASSED] xe_rtp_process_tests ===============
stty: 'standard input': Inappropriate ioctl for device
[09:58:27] ===================== [PASSED] xe_rtp ======================
[09:58:27] ==================== xe_wa (1 subtest) =====================
[09:58:27] ======================== xe_wa_gt =========================
[09:58:27] [PASSED] TIGERLAKE (B0)
[09:58:27] [PASSED] DG1 (A0)
[09:58:27] [PASSED] DG1 (B0)
[09:58:27] [PASSED] ALDERLAKE_S (A0)
[09:58:27] [PASSED] ALDERLAKE_S (B0)
[09:58:27] [PASSED] ALDERLAKE_S (C0)
[09:58:27] [PASSED] ALDERLAKE_S (D0)
[09:58:27] [PASSED] ALDERLAKE_P (A0)
[09:58:27] [PASSED] ALDERLAKE_P (B0)
[09:58:27] [PASSED] ALDERLAKE_P (C0)
[09:58:27] [PASSED] ALDERLAKE_S_RPLS (D0)
[09:58:27] [PASSED] ALDERLAKE_P_RPLU (E0)
[09:58:27] [PASSED] DG2_G10 (C0)
[09:58:27] [PASSED] DG2_G11 (B1)
[09:58:27] [PASSED] DG2_G12 (A1)
[09:58:27] [PASSED] METEORLAKE (g:A0, m:A0)
[09:58:27] [PASSED] METEORLAKE (g:A0, m:A0)
[09:58:27] [PASSED] METEORLAKE (g:A0, m:A0)
[09:58:27] [PASSED] LUNARLAKE (g:A0, m:A0)
[09:58:27] [PASSED] LUNARLAKE (g:B0, m:A0)
[09:58:27] ==================== [PASSED] xe_wa_gt =====================
[09:58:27] ====================== [PASSED] xe_wa ======================
[09:58:27] ============================================================
[09:58:27] Testing complete. Ran 109 tests: passed: 95, skipped: 14
[09:58:27] Elapsed time: 30.240s total, 4.231s configuring, 25.735s building, 0.231s running
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/tests/.kunitconfig
[09:58:27] Configuring KUnit Kernel ...
Regenerating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[09:58:29] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make ARCH=um O=.kunit --jobs=48
../lib/iomap.c:156:5: warning: no previous prototype for ‘ioread64_lo_hi’ [-Wmissing-prototypes]
156 | u64 ioread64_lo_hi(const void __iomem *addr)
| ^~~~~~~~~~~~~~
../lib/iomap.c:163:5: warning: no previous prototype for ‘ioread64_hi_lo’ [-Wmissing-prototypes]
163 | u64 ioread64_hi_lo(const void __iomem *addr)
| ^~~~~~~~~~~~~~
../lib/iomap.c:170:5: warning: no previous prototype for ‘ioread64be_lo_hi’ [-Wmissing-prototypes]
170 | u64 ioread64be_lo_hi(const void __iomem *addr)
| ^~~~~~~~~~~~~~~~
../lib/iomap.c:178:5: warning: no previous prototype for ‘ioread64be_hi_lo’ [-Wmissing-prototypes]
178 | u64 ioread64be_hi_lo(const void __iomem *addr)
| ^~~~~~~~~~~~~~~~
../lib/iomap.c:264:6: warning: no previous prototype for ‘iowrite64_lo_hi’ [-Wmissing-prototypes]
264 | void iowrite64_lo_hi(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~
../lib/iomap.c:272:6: warning: no previous prototype for ‘iowrite64_hi_lo’ [-Wmissing-prototypes]
272 | void iowrite64_hi_lo(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~
../lib/iomap.c:280:6: warning: no previous prototype for ‘iowrite64be_lo_hi’ [-Wmissing-prototypes]
280 | void iowrite64be_lo_hi(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~~~
../lib/iomap.c:288:6: warning: no previous prototype for ‘iowrite64be_hi_lo’ [-Wmissing-prototypes]
288 | void iowrite64be_hi_lo(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~~~
[09:58:50] Starting KUnit Kernel (1/1)...
[09:58:50] ============================================================
Running tests with:
$ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt
[09:58:51] ============ drm_test_pick_cmdline (2 subtests) ============
[09:58:51] [PASSED] drm_test_pick_cmdline_res_1920_1080_60
[09:58:51] =============== drm_test_pick_cmdline_named ===============
[09:58:51] [PASSED] NTSC
[09:58:51] [PASSED] NTSC-J
[09:58:51] [PASSED] PAL
[09:58:51] [PASSED] PAL-M
[09:58:51] =========== [PASSED] drm_test_pick_cmdline_named ===========
[09:58:51] ============== [PASSED] drm_test_pick_cmdline ==============
[09:58:51] ================== drm_buddy (7 subtests) ==================
[09:58:51] [PASSED] drm_test_buddy_alloc_limit
[09:58:51] [PASSED] drm_test_buddy_alloc_optimistic
[09:58:51] [PASSED] drm_test_buddy_alloc_pessimistic
[09:58:51] [PASSED] drm_test_buddy_alloc_pathological
[09:58:51] [PASSED] drm_test_buddy_alloc_contiguous
[09:58:51] [PASSED] drm_test_buddy_alloc_clear
[09:58:51] [PASSED] drm_test_buddy_alloc_range_bias
[09:58:51] ==================== [PASSED] drm_buddy ====================
[09:58:51] ============= drm_cmdline_parser (40 subtests) =============
[09:58:51] [PASSED] drm_test_cmdline_force_d_only
[09:58:51] [PASSED] drm_test_cmdline_force_D_only_dvi
[09:58:51] [PASSED] drm_test_cmdline_force_D_only_hdmi
[09:58:51] [PASSED] drm_test_cmdline_force_D_only_not_digital
[09:58:51] [PASSED] drm_test_cmdline_force_e_only
[09:58:51] [PASSED] drm_test_cmdline_res
[09:58:51] [PASSED] drm_test_cmdline_res_vesa
[09:58:51] [PASSED] drm_test_cmdline_res_vesa_rblank
[09:58:51] [PASSED] drm_test_cmdline_res_rblank
[09:58:51] [PASSED] drm_test_cmdline_res_bpp
[09:58:51] [PASSED] drm_test_cmdline_res_refresh
[09:58:51] [PASSED] drm_test_cmdline_res_bpp_refresh
[09:58:51] [PASSED] drm_test_cmdline_res_bpp_refresh_interlaced
[09:58:51] [PASSED] drm_test_cmdline_res_bpp_refresh_margins
[09:58:51] [PASSED] drm_test_cmdline_res_bpp_refresh_force_off
[09:58:51] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on
[09:58:51] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on_analog
[09:58:51] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on_digital
[09:58:51] [PASSED] drm_test_cmdline_res_bpp_refresh_interlaced_margins_force_on
[09:58:51] [PASSED] drm_test_cmdline_res_margins_force_on
[09:58:51] [PASSED] drm_test_cmdline_res_vesa_margins
[09:58:51] [PASSED] drm_test_cmdline_name
[09:58:51] [PASSED] drm_test_cmdline_name_bpp
[09:58:51] [PASSED] drm_test_cmdline_name_option
[09:58:51] [PASSED] drm_test_cmdline_name_bpp_option
[09:58:51] [PASSED] drm_test_cmdline_rotate_0
[09:58:51] [PASSED] drm_test_cmdline_rotate_90
[09:58:51] [PASSED] drm_test_cmdline_rotate_180
[09:58:51] [PASSED] drm_test_cmdline_rotate_270
[09:58:51] [PASSED] drm_test_cmdline_hmirror
[09:58:51] [PASSED] drm_test_cmdline_vmirror
[09:58:51] [PASSED] drm_test_cmdline_margin_options
[09:58:51] [PASSED] drm_test_cmdline_multiple_options
[09:58:51] [PASSED] drm_test_cmdline_bpp_extra_and_option
[09:58:51] [PASSED] drm_test_cmdline_extra_and_option
[09:58:51] [PASSED] drm_test_cmdline_freestanding_options
[09:58:51] [PASSED] drm_test_cmdline_freestanding_force_e_and_options
[09:58:51] [PASSED] drm_test_cmdline_panel_orientation
[09:58:51] ================ drm_test_cmdline_invalid =================
[09:58:51] [PASSED] margin_only
[09:58:51] [PASSED] interlace_only
[09:58:51] [PASSED] res_missing_x
[09:58:51] [PASSED] res_missing_y
[09:58:51] [PASSED] res_bad_y
[09:58:51] [PASSED] res_missing_y_bpp
[09:58:51] [PASSED] res_bad_bpp
[09:58:51] [PASSED] res_bad_refresh
[09:58:51] [PASSED] res_bpp_refresh_force_on_off
[09:58:51] [PASSED] res_invalid_mode
[09:58:51] [PASSED] res_bpp_wrong_place_mode
[09:58:51] [PASSED] name_bpp_refresh
[09:58:51] [PASSED] name_refresh
[09:58:51] [PASSED] name_refresh_wrong_mode
[09:58:51] [PASSED] name_refresh_invalid_mode
[09:58:51] [PASSED] rotate_multiple
[09:58:51] [PASSED] rotate_invalid_val
[09:58:51] [PASSED] rotate_truncated
[09:58:51] [PASSED] invalid_option
[09:58:51] [PASSED] invalid_tv_option
[09:58:51] [PASSED] truncated_tv_option
[09:58:51] ============ [PASSED] drm_test_cmdline_invalid =============
[09:58:51] =============== drm_test_cmdline_tv_options ===============
[09:58:51] [PASSED] NTSC
[09:58:51] [PASSED] NTSC_443
[09:58:51] [PASSED] NTSC_J
[09:58:51] [PASSED] PAL
[09:58:51] [PASSED] PAL_M
[09:58:51] [PASSED] PAL_N
[09:58:51] [PASSED] SECAM
[09:58:51] =========== [PASSED] drm_test_cmdline_tv_options ===========
[09:58:51] =============== [PASSED] drm_cmdline_parser ================
[09:58:51] ========== drmm_connector_hdmi_init (19 subtests) ==========
[09:58:51] [PASSED] drm_test_connector_hdmi_init_valid
[09:58:51] [PASSED] drm_test_connector_hdmi_init_bpc_8
[09:58:51] [PASSED] drm_test_connector_hdmi_init_bpc_10
[09:58:51] [PASSED] drm_test_connector_hdmi_init_bpc_12
[09:58:51] [PASSED] drm_test_connector_hdmi_init_bpc_invalid
[09:58:51] [PASSED] drm_test_connector_hdmi_init_bpc_null
[09:58:51] [PASSED] drm_test_connector_hdmi_init_formats_empty
[09:58:51] [PASSED] drm_test_connector_hdmi_init_formats_no_rgb
[09:58:51] [PASSED] drm_test_connector_hdmi_init_null_ddc
[09:58:51] [PASSED] drm_test_connector_hdmi_init_null_product
[09:58:51] [PASSED] drm_test_connector_hdmi_init_null_vendor
[09:58:51] [PASSED] drm_test_connector_hdmi_init_product_length_exact
[09:58:51] [PASSED] drm_test_connector_hdmi_init_product_length_too_long
[09:58:51] [PASSED] drm_test_connector_hdmi_init_product_valid
[09:58:51] [PASSED] drm_test_connector_hdmi_init_vendor_length_exact
[09:58:51] [PASSED] drm_test_connector_hdmi_init_vendor_length_too_long
[09:58:51] [PASSED] drm_test_connector_hdmi_init_vendor_valid
[09:58:51] ========= drm_test_connector_hdmi_init_type_valid =========
[09:58:51] [PASSED] HDMI-A
[09:58:51] [PASSED] HDMI-B
[09:58:51] ===== [PASSED] drm_test_connector_hdmi_init_type_valid =====
[09:58:51] ======== drm_test_connector_hdmi_init_type_invalid ========
[09:58:51] [PASSED] Unknown
[09:58:51] [PASSED] VGA
[09:58:51] [PASSED] DVI-I
[09:58:51] [PASSED] DVI-D
[09:58:51] [PASSED] DVI-A
[09:58:51] [PASSED] Composite
[09:58:51] [PASSED] SVIDEO
[09:58:51] [PASSED] LVDS
[09:58:51] [PASSED] Component
[09:58:51] [PASSED] DIN
[09:58:51] [PASSED] DP
[09:58:51] [PASSED] TV
[09:58:51] [PASSED] eDP
[09:58:51] [PASSED] Virtual
[09:58:51] [PASSED] DSI
[09:58:51] [PASSED] DPI
[09:58:51] [PASSED] Writeback
[09:58:51] [PASSED] SPI
[09:58:51] [PASSED] USB
[09:58:51] ==== [PASSED] drm_test_connector_hdmi_init_type_invalid ====
[09:58:51] ============ [PASSED] drmm_connector_hdmi_init =============
[09:58:51] ============= drmm_connector_init (3 subtests) =============
[09:58:51] [PASSED] drm_test_drmm_connector_init
[09:58:51] [PASSED] drm_test_drmm_connector_init_null_ddc
[09:58:51] ========= drm_test_drmm_connector_init_type_valid =========
[09:58:51] [PASSED] Unknown
[09:58:51] [PASSED] VGA
[09:58:51] [PASSED] DVI-I
[09:58:51] [PASSED] DVI-D
[09:58:51] [PASSED] DVI-A
[09:58:51] [PASSED] Composite
[09:58:51] [PASSED] SVIDEO
[09:58:51] [PASSED] LVDS
[09:58:51] [PASSED] Component
[09:58:51] [PASSED] DIN
[09:58:51] [PASSED] DP
[09:58:51] [PASSED] HDMI-A
[09:58:51] [PASSED] HDMI-B
[09:58:51] [PASSED] TV
[09:58:51] [PASSED] eDP
[09:58:51] [PASSED] Virtual
[09:58:51] [PASSED] DSI
[09:58:51] [PASSED] DPI
[09:58:51] [PASSED] Writeback
[09:58:51] [PASSED] SPI
[09:58:51] [PASSED] USB
[09:58:51] ===== [PASSED] drm_test_drmm_connector_init_type_valid =====
[09:58:51] =============== [PASSED] drmm_connector_init ===============
[09:58:51] = drm_connector_attach_broadcast_rgb_property (2 subtests) =
[09:58:51] [PASSED] drm_test_drm_connector_attach_broadcast_rgb_property
[09:58:51] [PASSED] drm_test_drm_connector_attach_broadcast_rgb_property_hdmi_connector
[09:58:51] === [PASSED] drm_connector_attach_broadcast_rgb_property ===
[09:58:51] ========== drm_get_tv_mode_from_name (2 subtests) ==========
[09:58:51] ========== drm_test_get_tv_mode_from_name_valid ===========
[09:58:51] [PASSED] NTSC
[09:58:51] [PASSED] NTSC-443
[09:58:51] [PASSED] NTSC-J
[09:58:51] [PASSED] PAL
[09:58:51] [PASSED] PAL-M
[09:58:51] [PASSED] PAL-N
[09:58:51] [PASSED] SECAM
[09:58:51] ====== [PASSED] drm_test_get_tv_mode_from_name_valid =======
[09:58:51] [PASSED] drm_test_get_tv_mode_from_name_truncated
[09:58:51] ============ [PASSED] drm_get_tv_mode_from_name ============
[09:58:51] = drm_test_connector_hdmi_compute_mode_clock (12 subtests) =
[09:58:51] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb
[09:58:51] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_10bpc
[09:58:51] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_10bpc_vic_1
[09:58:51] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_12bpc
[09:58:51] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_12bpc_vic_1
[09:58:51] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_double
[09:58:51] = drm_test_connector_hdmi_compute_mode_clock_yuv420_valid =
[09:58:51] [PASSED] VIC 96
[09:58:51] [PASSED] VIC 97
[09:58:51] [PASSED] VIC 101
[09:58:51] [PASSED] VIC 102
[09:58:51] [PASSED] VIC 106
[09:58:51] [PASSED] VIC 107
[09:58:51] === [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv420_valid ===
[09:58:51] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv420_10_bpc
[09:58:51] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv420_12_bpc
[09:58:51] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv422_8_bpc
[09:58:51] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv422_10_bpc
[09:58:51] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv422_12_bpc
[09:58:51] === [PASSED] drm_test_connector_hdmi_compute_mode_clock ====
[09:58:51] == drm_hdmi_connector_get_broadcast_rgb_name (2 subtests) ==
[09:58:51] === drm_test_drm_hdmi_connector_get_broadcast_rgb_name ====
[09:58:51] [PASSED] Automatic
[09:58:51] [PASSED] Full
[09:58:51] [PASSED] Limited 16:235
[09:58:51] === [PASSED] drm_test_drm_hdmi_connector_get_broadcast_rgb_name ===
[09:58:51] [PASSED] drm_test_drm_hdmi_connector_get_broadcast_rgb_name_invalid
[09:58:51] ==== [PASSED] drm_hdmi_connector_get_broadcast_rgb_name ====
[09:58:51] == drm_hdmi_connector_get_output_format_name (2 subtests) ==
[09:58:51] === drm_test_drm_hdmi_connector_get_output_format_name ====
[09:58:51] [PASSED] RGB
[09:58:51] [PASSED] YUV 4:2:0
[09:58:51] [PASSED] YUV 4:2:2
[09:58:51] [PASSED] YUV 4:4:4
[09:58:51] === [PASSED] drm_test_drm_hdmi_connector_get_output_format_name ===
[09:58:51] [PASSED] drm_test_drm_hdmi_connector_get_output_format_name_invalid
[09:58:51] ==== [PASSED] drm_hdmi_connector_get_output_format_name ====
[09:58:51] ============= drm_damage_helper (21 subtests) ==============
[09:58:51] [PASSED] drm_test_damage_iter_no_damage
[09:58:51] [PASSED] drm_test_damage_iter_no_damage_fractional_src
[09:58:51] [PASSED] drm_test_damage_iter_no_damage_src_moved
[09:58:51] [PASSED] drm_test_damage_iter_no_damage_fractional_src_moved
[09:58:51] [PASSED] drm_test_damage_iter_no_damage_not_visible
[09:58:51] [PASSED] drm_test_damage_iter_no_damage_no_crtc
[09:58:51] [PASSED] drm_test_damage_iter_no_damage_no_fb
[09:58:51] [PASSED] drm_test_damage_iter_simple_damage
[09:58:51] [PASSED] drm_test_damage_iter_single_damage
[09:58:51] [PASSED] drm_test_damage_iter_single_damage_intersect_src
[09:58:51] [PASSED] drm_test_damage_iter_single_damage_outside_src
[09:58:51] [PASSED] drm_test_damage_iter_single_damage_fractional_src
[09:58:51] [PASSED] drm_test_damage_iter_single_damage_intersect_fractional_src
[09:58:51] [PASSED] drm_test_damage_iter_single_damage_outside_fractional_src
[09:58:51] [PASSED] drm_test_damage_iter_single_damage_src_moved
[09:58:51] [PASSED] drm_test_damage_iter_single_damage_fractional_src_moved
[09:58:51] [PASSED] drm_test_damage_iter_damage
[09:58:51] [PASSED] drm_test_damage_iter_damage_one_intersect
[09:58:51] [PASSED] drm_test_damage_iter_damage_one_outside
[09:58:51] [PASSED] drm_test_damage_iter_damage_src_moved
[09:58:51] [PASSED] drm_test_damage_iter_damage_not_visible
[09:58:51] ================ [PASSED] drm_damage_helper ================
[09:58:51] ============== drm_dp_mst_helper (3 subtests) ==============
[09:58:51] ============== drm_test_dp_mst_calc_pbn_mode ==============
[09:58:51] [PASSED] Clock 154000 BPP 30 DSC disabled
[09:58:51] [PASSED] Clock 234000 BPP 30 DSC disabled
[09:58:51] [PASSED] Clock 297000 BPP 24 DSC disabled
[09:58:51] [PASSED] Clock 332880 BPP 24 DSC enabled
[09:58:51] [PASSED] Clock 324540 BPP 24 DSC enabled
[09:58:51] ========== [PASSED] drm_test_dp_mst_calc_pbn_mode ==========
[09:58:51] ============== drm_test_dp_mst_calc_pbn_div ===============
[09:58:51] [PASSED] Link rate 2000000 lane count 4
[09:58:51] [PASSED] Link rate 2000000 lane count 2
[09:58:51] [PASSED] Link rate 2000000 lane count 1
[09:58:51] [PASSED] Link rate 1350000 lane count 4
[09:58:51] [PASSED] Link rate 1350000 lane count 2
[09:58:51] [PASSED] Link rate 1350000 lane count 1
[09:58:51] [PASSED] Link rate 1000000 lane count 4
[09:58:51] [PASSED] Link rate 1000000 lane count 2
[09:58:51] [PASSED] Link rate 1000000 lane count 1
[09:58:51] [PASSED] Link rate 810000 lane count 4
[09:58:51] [PASSED] Link rate 810000 lane count 2
[09:58:51] [PASSED] Link rate 810000 lane count 1
[09:58:51] [PASSED] Link rate 540000 lane count 4
[09:58:51] [PASSED] Link rate 540000 lane count 2
[09:58:51] [PASSED] Link rate 540000 lane count 1
[09:58:51] [PASSED] Link rate 270000 lane count 4
[09:58:51] [PASSED] Link rate 270000 lane count 2
[09:58:51] [PASSED] Link rate 270000 lane count 1
[09:58:51] [PASSED] Link rate 162000 lane count 4
[09:58:51] [PASSED] Link rate 162000 lane count 2
[09:58:51] [PASSED] Link rate 162000 lane count 1
[09:58:51] ========== [PASSED] drm_test_dp_mst_calc_pbn_div ===========
[09:58:51] ========= drm_test_dp_mst_sideband_msg_req_decode =========
[09:58:51] [PASSED] DP_ENUM_PATH_RESOURCES with port number
[09:58:51] [PASSED] DP_POWER_UP_PHY with port number
[09:58:51] [PASSED] DP_POWER_DOWN_PHY with port number
[09:58:51] [PASSED] DP_ALLOCATE_PAYLOAD with SDP stream sinks
[09:58:51] [PASSED] DP_ALLOCATE_PAYLOAD with port number
[09:58:51] [PASSED] DP_ALLOCATE_PAYLOAD with VCPI
[09:58:51] [PASSED] DP_ALLOCATE_PAYLOAD with PBN
[09:58:51] [PASSED] DP_QUERY_PAYLOAD with port number
[09:58:51] [PASSED] DP_QUERY_PAYLOAD with VCPI
[09:58:51] [PASSED] DP_REMOTE_DPCD_READ with port number
[09:58:51] [PASSED] DP_REMOTE_DPCD_READ with DPCD address
[09:58:51] [PASSED] DP_REMOTE_DPCD_READ with max number of bytes
[09:58:51] [PASSED] DP_REMOTE_DPCD_WRITE with port number
[09:58:51] [PASSED] DP_REMOTE_DPCD_WRITE with DPCD address
[09:58:51] [PASSED] DP_REMOTE_DPCD_WRITE with data array
[09:58:51] [PASSED] DP_REMOTE_I2C_READ with port number
[09:58:51] [PASSED] DP_REMOTE_I2C_READ with I2C device ID
[09:58:51] [PASSED] DP_REMOTE_I2C_READ with transactions array
[09:58:51] [PASSED] DP_REMOTE_I2C_WRITE with port number
[09:58:51] [PASSED] DP_REMOTE_I2C_WRITE with I2C device ID
[09:58:51] [PASSED] DP_REMOTE_I2C_WRITE with data array
[09:58:51] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream ID
[09:58:51] [PASSED] DP_QUERY_STREAM_ENC_STATUS with client ID
[09:58:51] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream event
[09:58:51] [PASSED] DP_QUERY_STREAM_ENC_STATUS with valid stream event
[09:58:51] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream behavior
[09:58:51] [PASSED] DP_QUERY_STREAM_ENC_STATUS with a valid stream behavior
[09:58:51] ===== [PASSED] drm_test_dp_mst_sideband_msg_req_decode =====
[09:58:51] ================ [PASSED] drm_dp_mst_helper ================
[09:58:51] ================== drm_exec (7 subtests) ===================
[09:58:51] [PASSED] sanitycheck
[09:58:51] [PASSED] test_lock
[09:58:51] [PASSED] test_lock_unlock
[09:58:51] [PASSED] test_duplicates
[09:58:51] [PASSED] test_prepare
[09:58:51] [PASSED] test_prepare_array
[09:58:51] [PASSED] test_multiple_loops
[09:58:51] ==================== [PASSED] drm_exec =====================
[09:58:51] =========== drm_format_helper_test (17 subtests) ===========
[09:58:51] ============== drm_test_fb_xrgb8888_to_gray8 ==============
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ========== [PASSED] drm_test_fb_xrgb8888_to_gray8 ==========
[09:58:51] ============= drm_test_fb_xrgb8888_to_rgb332 ==============
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb332 ==========
[09:58:51] ============= drm_test_fb_xrgb8888_to_rgb565 ==============
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb565 ==========
[09:58:51] ============ drm_test_fb_xrgb8888_to_xrgb1555 =============
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ======== [PASSED] drm_test_fb_xrgb8888_to_xrgb1555 =========
[09:58:51] ============ drm_test_fb_xrgb8888_to_argb1555 =============
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ======== [PASSED] drm_test_fb_xrgb8888_to_argb1555 =========
[09:58:51] ============ drm_test_fb_xrgb8888_to_rgba5551 =============
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ======== [PASSED] drm_test_fb_xrgb8888_to_rgba5551 =========
[09:58:51] ============= drm_test_fb_xrgb8888_to_rgb888 ==============
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb888 ==========
[09:58:51] ============ drm_test_fb_xrgb8888_to_argb8888 =============
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ======== [PASSED] drm_test_fb_xrgb8888_to_argb8888 =========
[09:58:51] =========== drm_test_fb_xrgb8888_to_xrgb2101010 ===========
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ======= [PASSED] drm_test_fb_xrgb8888_to_xrgb2101010 =======
[09:58:51] =========== drm_test_fb_xrgb8888_to_argb2101010 ===========
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ======= [PASSED] drm_test_fb_xrgb8888_to_argb2101010 =======
[09:58:51] ============== drm_test_fb_xrgb8888_to_mono ===============
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ========== [PASSED] drm_test_fb_xrgb8888_to_mono ===========
[09:58:51] ==================== drm_test_fb_swab =====================
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ================ [PASSED] drm_test_fb_swab =================
[09:58:51] ============ drm_test_fb_xrgb8888_to_xbgr8888 =============
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ======== [PASSED] drm_test_fb_xrgb8888_to_xbgr8888 =========
[09:58:51] ============ drm_test_fb_xrgb8888_to_abgr8888 =============
[09:58:51] [PASSED] single_pixel_source_buffer
[09:58:51] [PASSED] single_pixel_clip_rectangle
[09:58:51] [PASSED] well_known_colors
[09:58:51] [PASSED] destination_pitch
[09:58:51] ======== [PASSED] drm_test_fb_xrgb8888_to_abgr8888 =========
[09:58:51] ================= drm_test_fb_clip_offset =================
[09:58:51] [PASSED] pass through
[09:58:51] [PASSED] horizontal offset
[09:58:51] [PASSED] vertical offset
[09:58:51] [PASSED] horizontal and vertical offset
[09:58:51] [PASSED] horizontal offset (custom pitch)
[09:58:51] [PASSED] vertical offset (custom pitch)
[09:58:51] [PASSED] horizontal and vertical offset (custom pitch)
[09:58:51] ============= [PASSED] drm_test_fb_clip_offset =============
[09:58:51] ============== drm_test_fb_build_fourcc_list ==============
[09:58:51] [PASSED] no native formats
[09:58:51] [PASSED] XRGB8888 as native format
[09:58:51] [PASSED] remove duplicates
[09:58:51] [PASSED] convert alpha formats
[09:58:51] [PASSED] random formats
[09:58:51] ========== [PASSED] drm_test_fb_build_fourcc_list ==========
[09:58:51] =================== drm_test_fb_memcpy ====================
[09:58:51] [PASSED] single_pixel_source_buffer: XR24 little-endian (0x34325258)
[09:58:51] [PASSED] single_pixel_source_buffer: XRA8 little-endian (0x38415258)
[09:58:51] [PASSED] single_pixel_source_buffer: YU24 little-endian (0x34325559)
[09:58:51] [PASSED] single_pixel_clip_rectangle: XB24 little-endian (0x34324258)
[09:58:51] [PASSED] single_pixel_clip_rectangle: XRA8 little-endian (0x38415258)
[09:58:51] [PASSED] single_pixel_clip_rectangle: YU24 little-endian (0x34325559)
[09:58:51] [PASSED] well_known_colors: XB24 little-endian (0x34324258)
[09:58:51] [PASSED] well_known_colors: XRA8 little-endian (0x38415258)
[09:58:51] [PASSED] well_known_colors: YU24 little-endian (0x34325559)
[09:58:51] [PASSED] destination_pitch: XB24 little-endian (0x34324258)
[09:58:51] [PASSED] destination_pitch: XRA8 little-endian (0x38415258)
[09:58:51] [PASSED] destination_pitch: YU24 little-endian (0x34325559)
[09:58:51] =============== [PASSED] drm_test_fb_memcpy ================
[09:58:51] ============= [PASSED] drm_format_helper_test ==============
[09:58:51] ================= drm_format (18 subtests) =================
[09:58:51] [PASSED] drm_test_format_block_width_invalid
[09:58:51] [PASSED] drm_test_format_block_width_one_plane
[09:58:51] [PASSED] drm_test_format_block_width_two_plane
[09:58:51] [PASSED] drm_test_format_block_width_three_plane
[09:58:51] [PASSED] drm_test_format_block_width_tiled
[09:58:51] [PASSED] drm_test_format_block_height_invalid
[09:58:51] [PASSED] drm_test_format_block_height_one_plane
[09:58:51] [PASSED] drm_test_format_block_height_two_plane
[09:58:51] [PASSED] drm_test_format_block_height_three_plane
[09:58:51] [PASSED] drm_test_format_block_height_tiled
[09:58:51] [PASSED] drm_test_format_min_pitch_invalid
[09:58:51] [PASSED] drm_test_format_min_pitch_one_plane_8bpp
[09:58:51] [PASSED] drm_test_format_min_pitch_one_plane_16bpp
[09:58:51] [PASSED] drm_test_format_min_pitch_one_plane_24bpp
[09:58:51] [PASSED] drm_test_format_min_pitch_one_plane_32bpp
[09:58:51] [PASSED] drm_test_format_min_pitch_two_plane
[09:58:51] [PASSED] drm_test_format_min_pitch_three_plane_8bpp
[09:58:51] [PASSED] drm_test_format_min_pitch_tiled
[09:58:51] =================== [PASSED] drm_format ====================
[09:58:51] =============== drm_framebuffer (1 subtest) ================
[09:58:51] =============== drm_test_framebuffer_create ===============
[09:58:51] [PASSED] ABGR8888 normal sizes
[09:58:51] [PASSED] ABGR8888 max sizes
[09:58:51] [PASSED] ABGR8888 pitch greater than min required
[09:58:51] [PASSED] ABGR8888 pitch less than min required
[09:58:51] [PASSED] ABGR8888 Invalid width
[09:58:51] [PASSED] ABGR8888 Invalid buffer handle
[09:58:51] [PASSED] No pixel format
[09:58:51] [PASSED] ABGR8888 Width 0
[09:58:51] [PASSED] ABGR8888 Height 0
[09:58:51] [PASSED] ABGR8888 Out of bound height * pitch combination
[09:58:51] [PASSED] ABGR8888 Large buffer offset
[09:58:51] [PASSED] ABGR8888 Set DRM_MODE_FB_MODIFIERS without modifiers
[09:58:51] [PASSED] ABGR8888 Valid buffer modifier
[09:58:51] [PASSED] ABGR8888 Invalid buffer modifier(DRM_FORMAT_MOD_SAMSUNG_64_32_TILE)
[09:58:51] [PASSED] ABGR8888 Extra pitches without DRM_MODE_FB_MODIFIERS
[09:58:51] [PASSED] ABGR8888 Extra pitches with DRM_MODE_FB_MODIFIERS
[09:58:51] [PASSED] NV12 Normal sizes
[09:58:51] [PASSED] NV12 Max sizes
[09:58:51] [PASSED] NV12 Invalid pitch
[09:58:51] [PASSED] NV12 Invalid modifier/missing DRM_MODE_FB_MODIFIERS flag
[09:58:51] [PASSED] NV12 different modifier per-plane
[09:58:51] [PASSED] NV12 with DRM_FORMAT_MOD_SAMSUNG_64_32_TILE
[09:58:51] [PASSED] NV12 Valid modifiers without DRM_MODE_FB_MODIFIERS
[09:58:51] [PASSED] NV12 Modifier for inexistent plane
[09:58:51] [PASSED] NV12 Handle for inexistent plane
[09:58:51] [PASSED] NV12 Handle for inexistent plane without DRM_MODE_FB_MODIFIERS
[09:58:51] [PASSED] YVU420 DRM_MODE_FB_MODIFIERS set without modifier
[09:58:51] [PASSED] YVU420 Normal sizes
[09:58:51] [PASSED] YVU420 Max sizes
[09:58:51] [PASSED] YVU420 Invalid pitch
[09:58:51] [PASSED] YVU420 Different pitches
[09:58:51] [PASSED] YVU420 Different buffer offsets/pitches
[09:58:51] [PASSED] YVU420 Modifier set just for plane 0, without DRM_MODE_FB_MODIFIERS
[09:58:51] [PASSED] YVU420 Modifier set just for planes 0, 1, without DRM_MODE_FB_MODIFIERS
[09:58:51] [PASSED] YVU420 Modifier set just for plane 0, 1, with DRM_MODE_FB_MODIFIERS
[09:58:51] [PASSED] YVU420 Valid modifier
[09:58:51] [PASSED] YVU420 Different modifiers per plane
[09:58:51] [PASSED] YVU420 Modifier for inexistent plane
[09:58:51] [PASSED] X0L2 Normal sizes
[09:58:51] [PASSED] X0L2 Max sizes
[09:58:51] [PASSED] X0L2 Invalid pitch
[09:58:51] [PASSED] X0L2 Pitch greater than minimum required
[09:58:51] [PASSED] X0L2 Handle for inexistent plane
[09:58:51] [PASSED] X0L2 Offset for inexistent plane, without DRM_MODE_FB_MODIFIERS set
[09:58:51] [PASSED] X0L2 Modifier without DRM_MODE_FB_MODIFIERS set
[09:58:51] [PASSED] X0L2 Valid modifier
[09:58:51] [PASSED] X0L2 Modifier for inexistent plane
[09:58:51] =========== [PASSED] drm_test_framebuffer_create ===========
[09:58:51] ================= [PASSED] drm_framebuffer =================
[09:58:51] ================ drm_gem_shmem (8 subtests) ================
[09:58:51] [PASSED] drm_gem_shmem_test_obj_create
[09:58:51] [PASSED] drm_gem_shmem_test_obj_create_private
[09:58:51] [PASSED] drm_gem_shmem_test_pin_pages
[09:58:51] [PASSED] drm_gem_shmem_test_vmap
[09:58:51] [PASSED] drm_gem_shmem_test_get_pages_sgt
[09:58:51] [PASSED] drm_gem_shmem_test_get_sg_table
[09:58:51] [PASSED] drm_gem_shmem_test_madvise
[09:58:51] [PASSED] drm_gem_shmem_test_purge
[09:58:51] ================== [PASSED] drm_gem_shmem ==================
[09:58:51] === drm_atomic_helper_connector_hdmi_check (22 subtests) ===
[09:58:51] [PASSED] drm_test_check_broadcast_rgb_auto_cea_mode
[09:58:51] [PASSED] drm_test_check_broadcast_rgb_auto_cea_mode_vic_1
[09:58:51] [PASSED] drm_test_check_broadcast_rgb_full_cea_mode
[09:58:51] [PASSED] drm_test_check_broadcast_rgb_full_cea_mode_vic_1
[09:58:51] [PASSED] drm_test_check_broadcast_rgb_limited_cea_mode
[09:58:51] [PASSED] drm_test_check_broadcast_rgb_limited_cea_mode_vic_1
[09:58:51] [PASSED] drm_test_check_broadcast_rgb_crtc_mode_changed
[09:58:51] [PASSED] drm_test_check_broadcast_rgb_crtc_mode_not_changed
[09:58:51] [PASSED] drm_test_check_hdmi_funcs_reject_rate
[09:58:51] [PASSED] drm_test_check_max_tmds_rate_bpc_fallback
[09:58:51] [PASSED] drm_test_check_max_tmds_rate_format_fallback
[09:58:51] [PASSED] drm_test_check_output_bpc_crtc_mode_changed
[09:58:51] [PASSED] drm_test_check_output_bpc_crtc_mode_not_changed
[09:58:51] [PASSED] drm_test_check_output_bpc_dvi
[09:58:51] [PASSED] drm_test_check_output_bpc_format_vic_1
[09:58:51] [PASSED] drm_test_check_output_bpc_format_display_8bpc_only
[09:58:51] [PASSED] drm_test_check_output_bpc_format_display_rgb_only
[09:58:51] [PASSED] drm_test_check_output_bpc_format_driver_8bpc_only
[09:58:51] [PASSED] drm_test_check_output_bpc_format_driver_rgb_only
[09:58:51] [PASSED] drm_test_check_tmds_char_rate_rgb_8bpc
[09:58:51] [PASSED] drm_test_check_tmds_char_rate_rgb_10bpc
[09:58:51] [PASSED] drm_test_check_tmds_char_rate_rgb_12bpc
[09:58:51] ===== [PASSED] drm_atomic_helper_connector_hdmi_check ======
[09:58:51] === drm_atomic_helper_connector_hdmi_reset (6 subtests) ====
[09:58:51] [PASSED] drm_test_check_broadcast_rgb_value
[09:58:51] [PASSED] drm_test_check_bpc_8_value
[09:58:51] [PASSED] drm_test_check_bpc_10_value
[09:58:51] [PASSED] drm_test_check_bpc_12_value
[09:58:51] [PASSED] drm_test_check_format_value
[09:58:51] [PASSED] drm_test_check_tmds_char_value
[09:58:51] ===== [PASSED] drm_atomic_helper_connector_hdmi_reset ======
[09:58:51] ================= drm_managed (2 subtests) =================
[09:58:51] [PASSED] drm_test_managed_release_action
[09:58:51] [PASSED] drm_test_managed_run_action
[09:58:51] =================== [PASSED] drm_managed ===================
[09:58:51] =================== drm_mm (6 subtests) ====================
[09:58:51] [PASSED] drm_test_mm_init
[09:58:51] [PASSED] drm_test_mm_debug
[09:58:51] [PASSED] drm_test_mm_align32
[09:58:51] [PASSED] drm_test_mm_align64
[09:58:51] [PASSED] drm_test_mm_lowest
[09:58:51] [PASSED] drm_test_mm_highest
[09:58:51] ===================== [PASSED] drm_mm ======================
[09:58:51] ============= drm_modes_analog_tv (4 subtests) =============
[09:58:51] [PASSED] drm_test_modes_analog_tv_ntsc_480i
[09:58:51] [PASSED] drm_test_modes_analog_tv_ntsc_480i_inlined
[09:58:51] [PASSED] drm_test_modes_analog_tv_pal_576i
[09:58:51] [PASSED] drm_test_modes_analog_tv_pal_576i_inlined
[09:58:51] =============== [PASSED] drm_modes_analog_tv ===============
[09:58:51] ============== drm_plane_helper (2 subtests) ===============
[09:58:51] =============== drm_test_check_plane_state ================
[09:58:51] [PASSED] clipping_simple
[09:58:51] [PASSED] clipping_rotate_reflect
[09:58:51] [PASSED] positioning_simple
[09:58:51] [PASSED] upscaling
[09:58:51] [PASSED] downscaling
[09:58:51] [PASSED] rounding1
[09:58:51] [PASSED] rounding2
[09:58:51] [PASSED] rounding3
[09:58:51] [PASSED] rounding4
[09:58:51] =========== [PASSED] drm_test_check_plane_state ============
[09:58:51] =========== drm_test_check_invalid_plane_state ============
[09:58:51] [PASSED] positioning_invalid
[09:58:51] [PASSED] upscaling_invalid
[09:58:51] [PASSED] downscaling_invalid
[09:58:51] ======= [PASSED] drm_test_check_invalid_plane_state ========
[09:58:51] ================ [PASSED] drm_plane_helper =================
stty: 'standard input': Inappropriate ioctl for device
[09:58:51] ====== drm_connector_helper_tv_get_modes (1 subtest) =======
[09:58:51] ====== drm_test_connector_helper_tv_get_modes_check =======
[09:58:51] [PASSED] None
[09:58:51] [PASSED] PAL
[09:58:51] [PASSED] NTSC
[09:58:51] [PASSED] Both, NTSC Default
[09:58:51] [PASSED] Both, PAL Default
[09:58:51] [PASSED] Both, NTSC Default, with PAL on command-line
[09:58:51] [PASSED] Both, PAL Default, with NTSC on command-line
[09:58:51] == [PASSED] drm_test_connector_helper_tv_get_modes_check ===
[09:58:51] ======== [PASSED] drm_connector_helper_tv_get_modes ========
[09:58:51] ================== drm_rect (9 subtests) ===================
[09:58:51] [PASSED] drm_test_rect_clip_scaled_div_by_zero
[09:58:51] [PASSED] drm_test_rect_clip_scaled_not_clipped
[09:58:51] [PASSED] drm_test_rect_clip_scaled_clipped
[09:58:51] [PASSED] drm_test_rect_clip_scaled_signed_vs_unsigned
[09:58:51] ================= drm_test_rect_intersect =================
[09:58:51] [PASSED] top-left x bottom-right: 2x2+1+1 x 2x2+0+0
[09:58:51] [PASSED] top-right x bottom-left: 2x2+0+0 x 2x2+1-1
[09:58:51] [PASSED] bottom-left x top-right: 2x2+1-1 x 2x2+0+0
[09:58:51] [PASSED] bottom-right x top-left: 2x2+0+0 x 2x2+1+1
[09:58:51] [PASSED] right x left: 2x1+0+0 x 3x1+1+0
[09:58:51] [PASSED] left x right: 3x1+1+0 x 2x1+0+0
[09:58:51] [PASSED] up x bottom: 1x2+0+0 x 1x3+0-1
[09:58:51] [PASSED] bottom x up: 1x3+0-1 x 1x2+0+0
[09:58:51] [PASSED] touching corner: 1x1+0+0 x 2x2+1+1
[09:58:51] [PASSED] touching side: 1x1+0+0 x 1x1+1+0
[09:58:51] [PASSED] equal rects: 2x2+0+0 x 2x2+0+0
[09:58:51] [PASSED] inside another: 2x2+0+0 x 1x1+1+1
[09:58:51] [PASSED] far away: 1x1+0+0 x 1x1+3+6
[09:58:51] [PASSED] points intersecting: 0x0+5+10 x 0x0+5+10
[09:58:51] [PASSED] points not intersecting: 0x0+0+0 x 0x0+5+10
[09:58:51] ============= [PASSED] drm_test_rect_intersect =============
[09:58:51] ================ drm_test_rect_calc_hscale ================
[09:58:51] [PASSED] normal use
[09:58:51] [PASSED] out of max range
[09:58:51] [PASSED] out of min range
[09:58:51] [PASSED] zero dst
[09:58:51] [PASSED] negative src
[09:58:51] [PASSED] negative dst
[09:58:51] ============ [PASSED] drm_test_rect_calc_hscale ============
[09:58:51] ================ drm_test_rect_calc_vscale ================
[09:58:51] [PASSED] normal use
[09:58:51] [PASSED] out of max range
[09:58:51] [PASSED] out of min range
[09:58:51] [PASSED] zero dst
[09:58:51] [PASSED] negative src
[09:58:51] [PASSED] negative dst
[09:58:51] ============ [PASSED] drm_test_rect_calc_vscale ============
[09:58:51] ================== drm_test_rect_rotate ===================
[09:58:51] [PASSED] reflect-x
[09:58:51] [PASSED] reflect-y
[09:58:51] [PASSED] rotate-0
[09:58:51] [PASSED] rotate-90
[09:58:51] [PASSED] rotate-180
[09:58:51] [PASSED] rotate-270
[09:58:51] ============== [PASSED] drm_test_rect_rotate ===============
[09:58:51] ================ drm_test_rect_rotate_inv =================
[09:58:51] [PASSED] reflect-x
[09:58:51] [PASSED] reflect-y
[09:58:51] [PASSED] rotate-0
[09:58:51] [PASSED] rotate-90
[09:58:51] [PASSED] rotate-180
[09:58:51] [PASSED] rotate-270
[09:58:51] ============ [PASSED] drm_test_rect_rotate_inv =============
[09:58:51] ==================== [PASSED] drm_rect =====================
[09:58:51] ============================================================
[09:58:51] Testing complete. Ran 511 tests: passed: 511
[09:58:51] Elapsed time: 23.748s total, 1.736s configuring, 21.792s building, 0.204s running
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/ttm/tests/.kunitconfig
stty: 'standard input': Inappropriate ioctl for device
[09:58:51] Configuring KUnit Kernel ...
Regenerating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[09:58:52] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make ARCH=um O=.kunit --jobs=48
[09:59:01] Starting KUnit Kernel (1/1)...
[09:59:01] ============================================================
Running tests with:
$ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt
[09:59:01] ================= ttm_device (5 subtests) ==================
[09:59:01] [PASSED] ttm_device_init_basic
[09:59:01] [PASSED] ttm_device_init_multiple
[09:59:01] [PASSED] ttm_device_fini_basic
[09:59:01] [PASSED] ttm_device_init_no_vma_man
[09:59:01] ================== ttm_device_init_pools ==================
[09:59:01] [PASSED] No DMA allocations, no DMA32 required
[09:59:01] [PASSED] DMA allocations, DMA32 required
[09:59:01] [PASSED] No DMA allocations, DMA32 required
[09:59:01] [PASSED] DMA allocations, no DMA32 required
[09:59:01] ============== [PASSED] ttm_device_init_pools ==============
[09:59:01] =================== [PASSED] ttm_device ====================
[09:59:01] ================== ttm_pool (8 subtests) ===================
[09:59:01] ================== ttm_pool_alloc_basic ===================
[09:59:01] [PASSED] One page
[09:59:01] [PASSED] More than one page
[09:59:01] [PASSED] Above the allocation limit
[09:59:01] [PASSED] One page, with coherent DMA mappings enabled
[09:59:01] [PASSED] Above the allocation limit, with coherent DMA mappings enabled
[09:59:01] ============== [PASSED] ttm_pool_alloc_basic ===============
[09:59:01] ============== ttm_pool_alloc_basic_dma_addr ==============
[09:59:01] [PASSED] One page
[09:59:01] [PASSED] More than one page
[09:59:01] [PASSED] Above the allocation limit
[09:59:01] [PASSED] One page, with coherent DMA mappings enabled
[09:59:01] [PASSED] Above the allocation limit, with coherent DMA mappings enabled
[09:59:01] ========== [PASSED] ttm_pool_alloc_basic_dma_addr ==========
[09:59:01] [PASSED] ttm_pool_alloc_order_caching_match
[09:59:01] [PASSED] ttm_pool_alloc_caching_mismatch
[09:59:01] [PASSED] ttm_pool_alloc_order_mismatch
[09:59:01] [PASSED] ttm_pool_free_dma_alloc
[09:59:01] [PASSED] ttm_pool_free_no_dma_alloc
[09:59:01] [PASSED] ttm_pool_fini_basic
[09:59:01] ==================== [PASSED] ttm_pool =====================
[09:59:01] ================ ttm_resource (8 subtests) =================
[09:59:01] ================= ttm_resource_init_basic =================
[09:59:01] [PASSED] Init resource in TTM_PL_SYSTEM
[09:59:01] [PASSED] Init resource in TTM_PL_VRAM
[09:59:01] [PASSED] Init resource in a private placement
[09:59:01] [PASSED] Init resource in TTM_PL_SYSTEM, set placement flags
[09:59:01] ============= [PASSED] ttm_resource_init_basic =============
[09:59:01] [PASSED] ttm_resource_init_pinned
[09:59:01] [PASSED] ttm_resource_fini_basic
[09:59:01] [PASSED] ttm_resource_manager_init_basic
[09:59:01] [PASSED] ttm_resource_manager_usage_basic
[09:59:01] [PASSED] ttm_resource_manager_set_used_basic
[09:59:01] [PASSED] ttm_sys_man_alloc_basic
[09:59:01] [PASSED] ttm_sys_man_free_basic
[09:59:01] ================== [PASSED] ttm_resource ===================
[09:59:01] =================== ttm_tt (10 subtests) ===================
[09:59:01] ==================== ttm_tt_init_basic ====================
[09:59:01] [PASSED] Page-aligned size
[09:59:01] [PASSED] Extra pages requested
[09:59:01] ================ [PASSED] ttm_tt_init_basic ================
[09:59:01] [PASSED] ttm_tt_init_misaligned
[09:59:01] [PASSED] ttm_tt_fini_basic
[09:59:01] [PASSED] ttm_tt_fini_sg
[09:59:01] [PASSED] ttm_tt_fini_shmem
[09:59:01] [PASSED] ttm_tt_create_basic
[09:59:01] [PASSED] ttm_tt_create_invalid_bo_type
[09:59:01] [PASSED] ttm_tt_create_ttm_exists
[09:59:01] [PASSED] ttm_tt_create_failed
[09:59:01] [PASSED] ttm_tt_destroy_basic
[09:59:01] ===================== [PASSED] ttm_tt ======================
[09:59:01] =================== ttm_bo (14 subtests) ===================
[09:59:01] =========== ttm_bo_reserve_optimistic_no_ticket ===========
[09:59:01] [PASSED] Cannot be interrupted and sleeps
[09:59:01] [PASSED] Cannot be interrupted, locks straight away
[09:59:01] [PASSED] Can be interrupted, sleeps
[09:59:01] ======= [PASSED] ttm_bo_reserve_optimistic_no_ticket =======
[09:59:01] [PASSED] ttm_bo_reserve_locked_no_sleep
[09:59:01] [PASSED] ttm_bo_reserve_no_wait_ticket
[09:59:01] [PASSED] ttm_bo_reserve_double_resv
[09:59:01] [PASSED] ttm_bo_reserve_interrupted
[09:59:01] [PASSED] ttm_bo_reserve_deadlock
[09:59:01] [PASSED] ttm_bo_unreserve_basic
[09:59:01] [PASSED] ttm_bo_unreserve_pinned
[09:59:01] [PASSED] ttm_bo_unreserve_bulk
[09:59:01] [PASSED] ttm_bo_put_basic
[09:59:01] [PASSED] ttm_bo_put_shared_resv
[09:59:01] [PASSED] ttm_bo_pin_basic
[09:59:01] [PASSED] ttm_bo_pin_unpin_resource
[09:59:01] [PASSED] ttm_bo_multiple_pin_one_unpin
[09:59:01] ===================== [PASSED] ttm_bo ======================
[09:59:01] ============================================================
[09:59:01] Testing complete. Ran 62 tests: passed: 62
[09:59:01] Elapsed time: 10.503s total, 1.722s configuring, 8.561s building, 0.187s running
+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface
@ 2024-06-13 10:04 Riana Tauro
2024-06-13 9:57 ` ✓ CI.Patch_applied: success for drm/xe/pmu: Enable PMU interface (rev9) Patchwork
` (9 more replies)
0 siblings, 10 replies; 32+ messages in thread
From: Riana Tauro @ 2024-06-13 10:04 UTC (permalink / raw)
To: intel-xe
Cc: riana.tauro, anshuman.gupta, ashutosh.dixit, aravind.iddamsetty,
rodrigo.vivi, umesh.nerlige.ramappa, krishnaiah.bommu
There are a set of engine group busyness counters provided by HW which are
perfect fit to be exposed via PMU perf events.
BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
events can be listed using:
perf list
xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
and can be read using:
perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
time counts unit events
1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
The pmu base implementation is taken from i915.
v9:
1. replace drmm_add_action_or_reset with devm
2. add doc to xe_gt_clock_cycles_to_ns
v8:
1. rebase
2. drop interrupt pmu event
v7:
1. update UAPI documentation
2. drop MEDIA_GT specific change for media busyness counter.
v6:
1. drop engine_busyness_sample_type
2. update UAPI documentation
v5:
1. Use spinlock in forcewake instead of mutex
2. take forcewake when accessing the OAG registers
v4: minor nits.
v3:
1. drop init_samples, as storing counters before going to suspend should
be sufficient.
2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
dropped helpers to store and read samples.
3. use xe_device_mem_access_get_if_ongoing to check if device is active
before reading the OA registers.
4. dropped format attr as no longer needed
5. introduce xe_pmu_suspend to call engine_group_busyness_store
6. few other nits.
v2:
Store last known value when device is awake return that while the GT is
suspended and then update the driver copy when read during awake.
Aravind Iddamsetty (2):
drm/xe: Get GT clock to nanosecs
drm/xe/pmu: Enable PMU interface
drivers/gpu/drm/xe/Makefile | 2 +
drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
drivers/gpu/drm/xe/xe_device.c | 2 +
drivers/gpu/drm/xe/xe_device_types.h | 4 +
drivers/gpu/drm/xe/xe_gt.c | 2 +
drivers/gpu/drm/xe/xe_gt_clock.c | 13 +
drivers/gpu/drm/xe/xe_gt_clock.h | 2 +-
drivers/gpu/drm/xe/xe_module.c | 5 +
drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_pmu.h | 26 ++
drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
include/uapi/drm/xe_drm.h | 39 ++
12 files changed, 797 insertions(+), 1 deletion(-)
create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
--
2.40.0
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v9 1/2] drm/xe: Get GT clock to nanosecs
2024-06-13 10:04 [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Riana Tauro
` (2 preceding siblings ...)
2024-06-13 9:59 ` ✓ CI.KUnit: success " Patchwork
@ 2024-06-13 10:04 ` Riana Tauro
2024-06-14 14:57 ` Lucas De Marchi
2024-06-13 10:04 ` [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface Riana Tauro
` (5 subsequent siblings)
9 siblings, 1 reply; 32+ messages in thread
From: Riana Tauro @ 2024-06-13 10:04 UTC (permalink / raw)
To: intel-xe
Cc: riana.tauro, anshuman.gupta, ashutosh.dixit, aravind.iddamsetty,
rodrigo.vivi, umesh.nerlige.ramappa, krishnaiah.bommu
From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Helper to convert GT clock cycles to nanoseconds.
v2: Use DIV_ROUND_CLOSEST_ULL helper(Ashutosh)
v3: rename xe_gt_clock_interval_to_ns to xe_gt_clock_cycles_to_ns
v4: rebase
v5: add doc
Reviewed-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
---
drivers/gpu/drm/xe/xe_gt_clock.c | 13 +++++++++++++
drivers/gpu/drm/xe/xe_gt_clock.h | 2 +-
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/xe/xe_gt_clock.c b/drivers/gpu/drm/xe/xe_gt_clock.c
index 86c2d62b4bdc..17bd8851ef3c 100644
--- a/drivers/gpu/drm/xe/xe_gt_clock.c
+++ b/drivers/gpu/drm/xe/xe_gt_clock.c
@@ -99,3 +99,16 @@ u64 xe_gt_clock_interval_to_ms(struct xe_gt *gt, u64 count)
{
return div_u64_roundup(count * MSEC_PER_SEC, gt->info.reference_clock);
}
+
+/**
+ * xe_gt_clock_cycles_to_ns - Convert sampled GT clock ticks to nsec
+ *
+ * @gt: GT instance
+ * @count: count of GT clock ticks
+ *
+ * Returns: time in nsec
+ */
+u64 xe_gt_clock_cycles_to_ns(const struct xe_gt *gt, u64 count)
+{
+ return DIV_ROUND_CLOSEST_ULL(count * NSEC_PER_SEC, gt->info.reference_clock);
+}
diff --git a/drivers/gpu/drm/xe/xe_gt_clock.h b/drivers/gpu/drm/xe/xe_gt_clock.h
index 3adeb7baaca4..2bbef839f19f 100644
--- a/drivers/gpu/drm/xe/xe_gt_clock.h
+++ b/drivers/gpu/drm/xe/xe_gt_clock.h
@@ -12,5 +12,5 @@ struct xe_gt;
int xe_gt_clock_init(struct xe_gt *gt);
u64 xe_gt_clock_interval_to_ms(struct xe_gt *gt, u64 count);
-
+u64 xe_gt_clock_cycles_to_ns(const struct xe_gt *gt, u64 count);
#endif
--
2.40.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-13 10:04 [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Riana Tauro
` (3 preceding siblings ...)
2024-06-13 10:04 ` [PATCH v9 1/2] drm/xe: Get GT clock to nanosecs Riana Tauro
@ 2024-06-13 10:04 ` Riana Tauro
2024-06-14 16:15 ` Lucas De Marchi
` (2 more replies)
2024-06-13 10:11 ` ✓ CI.Build: success for drm/xe/pmu: Enable PMU interface (rev9) Patchwork
` (4 subsequent siblings)
9 siblings, 3 replies; 32+ messages in thread
From: Riana Tauro @ 2024-06-13 10:04 UTC (permalink / raw)
To: intel-xe
Cc: riana.tauro, anshuman.gupta, ashutosh.dixit, aravind.iddamsetty,
rodrigo.vivi, umesh.nerlige.ramappa, krishnaiah.bommu
From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
There are a set of engine group busyness counters provided by HW which are
perfect fit to be exposed via PMU perf events.
BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
events can be listed using:
perf list
xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
and can be read using:
perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
time counts unit events
1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
The pmu base implementation is taken from i915.
v2:
Store last known value when device is awake return that while the GT is
suspended and then update the driver copy when read during awake.
v3:
1. drop init_samples, as storing counters before going to suspend should
be sufficient.
2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
dropped helpers to store and read samples.
3. use xe_device_mem_access_get_if_ongoing to check if device is active
before reading the OA registers.
4. dropped format attr as no longer needed
5. introduce xe_pmu_suspend to call engine_group_busyness_store
6. few other nits.
v4: minor nits.
v5: take forcewake when accessing the OAG registers
v6:
1. drop engine_busyness_sample_type
2. update UAPI documentation
v7:
1. update UAPI documentation
2. drop MEDIA_GT specific change for media busyness counter.
v8:
1. rebase
2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
3. remove interrupts pmu event
v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
---
drivers/gpu/drm/xe/Makefile | 2 +
drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
drivers/gpu/drm/xe/xe_device.c | 2 +
drivers/gpu/drm/xe/xe_device_types.h | 4 +
drivers/gpu/drm/xe/xe_gt.c | 2 +
drivers/gpu/drm/xe/xe_module.c | 5 +
drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_pmu.h | 26 ++
drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
include/uapi/drm/xe_drm.h | 39 ++
10 files changed, 783 insertions(+)
create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index cbf961b90237..83bf1e07669b 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
i915-display/skl_universal_plane.o \
i915-display/skl_watermark.o
+xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
+
ifeq ($(CONFIG_ACPI),y)
xe-$(CONFIG_DRM_XE_DISPLAY) += \
i915-display/intel_acpi.o \
diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
index 47c26c37608d..22821dcd4e1b 100644
--- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
+++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
@@ -390,6 +390,11 @@
#define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
#define GLOBAL_INVALIDATION_MODE REG_BIT(2)
+#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
+#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
+#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
+#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
+
#define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
#define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index 64691a56d59c..bb00c8c9ec9b 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
xe_hwmon_register(xe);
+ xe_pmu_register(&xe->pmu);
+
return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
err_fini_display:
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 52bc461171d5..a5dba7325cf1 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -18,6 +18,7 @@
#include "xe_lmtt_types.h"
#include "xe_memirq_types.h"
#include "xe_platform_types.h"
+#include "xe_pmu.h"
#include "xe_pt_types.h"
#include "xe_sriov_types.h"
#include "xe_step_types.h"
@@ -473,6 +474,9 @@ struct xe_device {
int mode;
} wedged;
+ /** @pmu: performance monitoring unit */
+ struct xe_pmu pmu;
+
/* private: */
#if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
index 57d84751e160..477d0ae5f230 100644
--- a/drivers/gpu/drm/xe/xe_gt.c
+++ b/drivers/gpu/drm/xe/xe_gt.c
@@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
if (err)
goto err_msg;
+ xe_pmu_suspend(gt);
+
err = xe_uc_suspend(>->uc);
if (err)
goto err_force_wake;
diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
index 3edeb30d5ccb..26f814f97fc2 100644
--- a/drivers/gpu/drm/xe/xe_module.c
+++ b/drivers/gpu/drm/xe/xe_module.c
@@ -11,6 +11,7 @@
#include "xe_drv.h"
#include "xe_hw_fence.h"
#include "xe_pci.h"
+#include "xe_pmu.h"
#include "xe_sched_job.h"
struct xe_modparam xe_modparam = {
@@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
.init = xe_sched_job_module_init,
.exit = xe_sched_job_module_exit,
},
+ {
+ .init = xe_pmu_init,
+ .exit = xe_pmu_exit,
+ },
{
.init = xe_register_pci_driver,
.exit = xe_unregister_pci_driver,
diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
new file mode 100644
index 000000000000..64960a358af2
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu.c
@@ -0,0 +1,631 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#include <drm/drm_drv.h>
+#include <drm/drm_managed.h>
+#include <drm/xe_drm.h>
+
+#include "regs/xe_gt_regs.h"
+#include "xe_device.h"
+#include "xe_force_wake.h"
+#include "xe_gt_clock.h"
+#include "xe_mmio.h"
+#include "xe_macros.h"
+#include "xe_pm.h"
+
+static cpumask_t xe_pmu_cpumask;
+static unsigned int xe_pmu_target_cpu = -1;
+
+static unsigned int config_gt_id(const u64 config)
+{
+ return config >> __XE_PMU_GT_SHIFT;
+}
+
+static u64 config_counter(const u64 config)
+{
+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
+}
+
+static void xe_pmu_event_destroy(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+
+ drm_WARN_ON(&xe->drm, event->parent);
+
+ drm_dev_put(&xe->drm);
+}
+
+static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
+{
+ u64 val;
+
+ switch (sample_type) {
+ case __XE_SAMPLE_RENDER_GROUP_BUSY:
+ val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
+ break;
+ case __XE_SAMPLE_COPY_GROUP_BUSY:
+ val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
+ break;
+ case __XE_SAMPLE_MEDIA_GROUP_BUSY:
+ val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
+ break;
+ case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
+ val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
+ break;
+ default:
+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
+ }
+
+ return xe_gt_clock_cycles_to_ns(gt, val * 16);
+}
+
+static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
+{
+ int sample_type = config_counter(config);
+ const unsigned int gt_id = gt->info.id;
+ struct xe_device *xe = gt->tile->xe;
+ struct xe_pmu *pmu = &xe->pmu;
+ unsigned long flags;
+ bool device_awake;
+ u64 val;
+
+ device_awake = xe_pm_runtime_get_if_active(xe);
+ if (device_awake) {
+ XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
+ val = __engine_group_busyness_read(gt, sample_type);
+ XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
+ xe_pm_runtime_put(xe);
+ }
+
+ spin_lock_irqsave(&pmu->lock, flags);
+
+ if (device_awake)
+ pmu->sample[gt_id][sample_type] = val;
+ else
+ val = pmu->sample[gt_id][sample_type];
+
+ spin_unlock_irqrestore(&pmu->lock, flags);
+
+ return val;
+}
+
+static void engine_group_busyness_store(struct xe_gt *gt)
+{
+ struct xe_pmu *pmu = >->tile->xe->pmu;
+ unsigned int gt_id = gt->info.id;
+ unsigned long flags;
+ int i;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+
+ for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
+ pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
+
+ spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+static int
+config_status(struct xe_device *xe, u64 config)
+{
+ unsigned int gt_id = config_gt_id(config);
+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
+
+ if (gt_id >= XE_PMU_MAX_GT)
+ return -ENOENT;
+
+ switch (config_counter(config)) {
+ case XE_PMU_RENDER_GROUP_BUSY(0):
+ case XE_PMU_COPY_GROUP_BUSY(0):
+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
+ if (gt->info.type == XE_GT_TYPE_MEDIA)
+ return -ENOENT;
+ break;
+ case XE_PMU_MEDIA_GROUP_BUSY(0):
+ if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
+ return -ENOENT;
+ break;
+ default:
+ return -ENOENT;
+ }
+
+ return 0;
+}
+
+static int xe_pmu_event_init(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+ int ret;
+
+ if (pmu->closed)
+ return -ENODEV;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ /* unsupported modes and filters */
+ if (event->attr.sample_period) /* no sampling */
+ return -EINVAL;
+
+ if (has_branch_stack(event))
+ return -EOPNOTSUPP;
+
+ if (event->cpu < 0)
+ return -EINVAL;
+
+ /* only allow running on one cpu at a time */
+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
+ return -EINVAL;
+
+ ret = config_status(xe, event->attr.config);
+ if (ret)
+ return ret;
+
+ if (!event->parent) {
+ drm_dev_get(&xe->drm);
+ event->destroy = xe_pmu_event_destroy;
+ }
+
+ return 0;
+}
+
+static u64 __xe_pmu_event_read(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ const unsigned int gt_id = config_gt_id(event->attr.config);
+ const u64 config = event->attr.config;
+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
+ u64 val;
+
+ switch (config_counter(config)) {
+ case XE_PMU_RENDER_GROUP_BUSY(0):
+ case XE_PMU_COPY_GROUP_BUSY(0):
+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
+ case XE_PMU_MEDIA_GROUP_BUSY(0):
+ val = engine_group_busyness_read(gt, config);
+ break;
+ default:
+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
+ }
+
+ return val;
+}
+
+static void xe_pmu_event_read(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct hw_perf_event *hwc = &event->hw;
+ struct xe_pmu *pmu = &xe->pmu;
+ u64 prev, new;
+
+ if (pmu->closed) {
+ event->hw.state = PERF_HES_STOPPED;
+ return;
+ }
+again:
+ prev = local64_read(&hwc->prev_count);
+ new = __xe_pmu_event_read(event);
+
+ if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
+ goto again;
+
+ local64_add(new - prev, &event->count);
+}
+
+static void xe_pmu_enable(struct perf_event *event)
+{
+ /*
+ * Store the current counter value so we can report the correct delta
+ * for all listeners. Even when the event was already enabled and has
+ * an existing non-zero value.
+ */
+ local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
+}
+
+static void xe_pmu_event_start(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (pmu->closed)
+ return;
+
+ xe_pmu_enable(event);
+ event->hw.state = 0;
+}
+
+static void xe_pmu_event_stop(struct perf_event *event, int flags)
+{
+ if (flags & PERF_EF_UPDATE)
+ xe_pmu_event_read(event);
+
+ event->hw.state = PERF_HES_STOPPED;
+}
+
+static int xe_pmu_event_add(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (pmu->closed)
+ return -ENODEV;
+
+ if (flags & PERF_EF_START)
+ xe_pmu_event_start(event, flags);
+
+ return 0;
+}
+
+static void xe_pmu_event_del(struct perf_event *event, int flags)
+{
+ xe_pmu_event_stop(event, PERF_EF_UPDATE);
+}
+
+static int xe_pmu_event_event_idx(struct perf_event *event)
+{
+ return 0;
+}
+
+struct xe_ext_attribute {
+ struct device_attribute attr;
+ unsigned long val;
+};
+
+static ssize_t xe_pmu_event_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct xe_ext_attribute *eattr;
+
+ eattr = container_of(attr, struct xe_ext_attribute, attr);
+ return sprintf(buf, "config=0x%lx\n", eattr->val);
+}
+
+static ssize_t cpumask_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
+}
+
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *xe_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group xe_pmu_cpumask_attr_group = {
+ .attrs = xe_cpumask_attrs,
+};
+
+#define __event(__counter, __name, __unit) \
+{ \
+ .counter = (__counter), \
+ .name = (__name), \
+ .unit = (__unit), \
+}
+
+static struct xe_ext_attribute *
+add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
+{
+ sysfs_attr_init(&attr->attr.attr);
+ attr->attr.attr.name = name;
+ attr->attr.attr.mode = 0444;
+ attr->attr.show = xe_pmu_event_show;
+ attr->val = config;
+
+ return ++attr;
+}
+
+static struct perf_pmu_events_attr *
+add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
+ const char *str)
+{
+ sysfs_attr_init(&attr->attr.attr);
+ attr->attr.attr.name = name;
+ attr->attr.attr.mode = 0444;
+ attr->attr.show = perf_event_sysfs_show;
+ attr->event_str = str;
+
+ return ++attr;
+}
+
+static struct attribute **
+create_event_attributes(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ static const struct {
+ unsigned int counter;
+ const char *name;
+ const char *unit;
+ } events[] = {
+ __event(0, "render-group-busy", "ns"),
+ __event(1, "copy-group-busy", "ns"),
+ __event(2, "media-group-busy", "ns"),
+ __event(3, "any-engine-group-busy", "ns"),
+ };
+
+ struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
+ struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
+ struct attribute **attr = NULL, **attr_iter;
+ unsigned int count = 0;
+ unsigned int i, j;
+ struct xe_gt *gt;
+
+ /* Count how many counters we will be exposing. */
+ for_each_gt(gt, xe, j) {
+ for (i = 0; i < ARRAY_SIZE(events); i++) {
+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
+
+ if (!config_status(xe, config))
+ count++;
+ }
+ }
+
+ /* Allocate attribute objects and table. */
+ xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
+ if (!xe_attr)
+ goto err_alloc;
+
+ pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
+ if (!pmu_attr)
+ goto err_alloc;
+
+ /* Max one pointer of each attribute type plus a termination entry. */
+ attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
+ if (!attr)
+ goto err_alloc;
+
+ xe_iter = xe_attr;
+ pmu_iter = pmu_attr;
+ attr_iter = attr;
+
+ for_each_gt(gt, xe, j) {
+ for (i = 0; i < ARRAY_SIZE(events); i++) {
+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
+ char *str;
+
+ if (config_status(xe, config))
+ continue;
+
+ str = kasprintf(GFP_KERNEL, "%s-gt%u",
+ events[i].name, j);
+ if (!str)
+ goto err;
+
+ *attr_iter++ = &xe_iter->attr.attr;
+ xe_iter = add_xe_attr(xe_iter, str, config);
+
+ if (events[i].unit) {
+ str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
+ events[i].name, j);
+ if (!str)
+ goto err;
+
+ *attr_iter++ = &pmu_iter->attr.attr;
+ pmu_iter = add_pmu_attr(pmu_iter, str,
+ events[i].unit);
+ }
+ }
+ }
+
+ pmu->xe_attr = xe_attr;
+ pmu->pmu_attr = pmu_attr;
+
+ return attr;
+
+err:
+ for (attr_iter = attr; *attr_iter; attr_iter++)
+ kfree((*attr_iter)->name);
+
+err_alloc:
+ kfree(attr);
+ kfree(xe_attr);
+ kfree(pmu_attr);
+
+ return NULL;
+}
+
+static void free_event_attributes(struct xe_pmu *pmu)
+{
+ struct attribute **attr_iter = pmu->events_attr_group.attrs;
+
+ for (; *attr_iter; attr_iter++)
+ kfree((*attr_iter)->name);
+
+ kfree(pmu->events_attr_group.attrs);
+ kfree(pmu->xe_attr);
+ kfree(pmu->pmu_attr);
+
+ pmu->events_attr_group.attrs = NULL;
+ pmu->xe_attr = NULL;
+ pmu->pmu_attr = NULL;
+}
+
+static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
+{
+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
+
+ /* Select the first online CPU as a designated reader. */
+ if (cpumask_empty(&xe_pmu_cpumask))
+ cpumask_set_cpu(cpu, &xe_pmu_cpumask);
+
+ return 0;
+}
+
+static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
+{
+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
+ unsigned int target = xe_pmu_target_cpu;
+
+ /*
+ * Unregistering an instance generates a CPU offline event which we must
+ * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
+ */
+ if (pmu->closed)
+ return 0;
+
+ if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
+ target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
+
+ /* Migrate events if there is a valid target */
+ if (target < nr_cpu_ids) {
+ cpumask_set_cpu(target, &xe_pmu_cpumask);
+ xe_pmu_target_cpu = target;
+ }
+ }
+
+ if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
+ perf_pmu_migrate_context(&pmu->base, cpu, target);
+ pmu->cpuhp.cpu = target;
+ }
+
+ return 0;
+}
+
+static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
+
+int xe_pmu_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
+ "perf/x86/intel/xe:online",
+ xe_pmu_cpu_online,
+ xe_pmu_cpu_offline);
+ if (ret < 0)
+ pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
+ ret);
+ else
+ cpuhp_slot = ret;
+
+ return 0;
+}
+
+void xe_pmu_exit(void)
+{
+ if (cpuhp_slot != CPUHP_INVALID)
+ cpuhp_remove_multi_state(cpuhp_slot);
+}
+
+static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
+{
+ if (cpuhp_slot == CPUHP_INVALID)
+ return -EINVAL;
+
+ return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
+}
+
+static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
+{
+ cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
+}
+
+void xe_pmu_suspend(struct xe_gt *gt)
+{
+ engine_group_busyness_store(gt);
+}
+
+static void xe_pmu_unregister(void *arg)
+{
+ struct xe_pmu *pmu = arg;
+
+ if (!pmu->base.event_init)
+ return;
+
+ /*
+ * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
+ * ensures all currently executing ones will have exited before we
+ * proceed with unregistration.
+ */
+ pmu->closed = true;
+ synchronize_rcu();
+
+ xe_pmu_unregister_cpuhp_state(pmu);
+
+ perf_pmu_unregister(&pmu->base);
+ pmu->base.event_init = NULL;
+ kfree(pmu->base.attr_groups);
+ kfree(pmu->name);
+ free_event_attributes(pmu);
+}
+
+void xe_pmu_register(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ const struct attribute_group *attr_groups[] = {
+ &pmu->events_attr_group,
+ &xe_pmu_cpumask_attr_group,
+ NULL
+ };
+
+ int ret = -ENOMEM;
+
+ spin_lock_init(&pmu->lock);
+ pmu->cpuhp.cpu = -1;
+
+ pmu->name = kasprintf(GFP_KERNEL,
+ "xe_%s",
+ dev_name(xe->drm.dev));
+ if (pmu->name)
+ /* tools/perf reserves colons as special. */
+ strreplace((char *)pmu->name, ':', '_');
+
+ if (!pmu->name)
+ goto err;
+
+ pmu->events_attr_group.name = "events";
+ pmu->events_attr_group.attrs = create_event_attributes(pmu);
+ if (!pmu->events_attr_group.attrs)
+ goto err_name;
+
+ pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
+ GFP_KERNEL);
+ if (!pmu->base.attr_groups)
+ goto err_attr;
+
+ pmu->base.module = THIS_MODULE;
+ pmu->base.task_ctx_nr = perf_invalid_context;
+ pmu->base.event_init = xe_pmu_event_init;
+ pmu->base.add = xe_pmu_event_add;
+ pmu->base.del = xe_pmu_event_del;
+ pmu->base.start = xe_pmu_event_start;
+ pmu->base.stop = xe_pmu_event_stop;
+ pmu->base.read = xe_pmu_event_read;
+ pmu->base.event_idx = xe_pmu_event_event_idx;
+
+ ret = perf_pmu_register(&pmu->base, pmu->name, -1);
+ if (ret)
+ goto err_groups;
+
+ ret = xe_pmu_register_cpuhp_state(pmu);
+ if (ret)
+ goto err_unreg;
+
+ ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
+ if (ret)
+ goto err_cpuhp;
+
+ return;
+
+err_cpuhp:
+ xe_pmu_unregister_cpuhp_state(pmu);
+err_unreg:
+ perf_pmu_unregister(&pmu->base);
+err_groups:
+ kfree(pmu->base.attr_groups);
+err_attr:
+ pmu->base.event_init = NULL;
+ free_event_attributes(pmu);
+err_name:
+ kfree(pmu->name);
+err:
+ drm_notice(&xe->drm, "Failed to register PMU!\n");
+}
diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
new file mode 100644
index 000000000000..8afa256f9dac
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#ifndef _XE_PMU_H_
+#define _XE_PMU_H_
+
+#include "xe_pmu_types.h"
+
+struct xe_gt;
+
+#if IS_ENABLED(CONFIG_PERF_EVENTS)
+int xe_pmu_init(void);
+void xe_pmu_exit(void);
+void xe_pmu_register(struct xe_pmu *pmu);
+void xe_pmu_suspend(struct xe_gt *gt);
+#else
+static inline int xe_pmu_init(void) { return 0; }
+static inline void xe_pmu_exit(void) {}
+static inline void xe_pmu_register(struct xe_pmu *pmu) {}
+static inline void xe_pmu_suspend(struct xe_gt *gt) {}
+#endif
+
+#endif
+
diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
new file mode 100644
index 000000000000..e86e8d7e0356
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
@@ -0,0 +1,67 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#ifndef _XE_PMU_TYPES_H_
+#define _XE_PMU_TYPES_H_
+
+#include <linux/perf_event.h>
+#include <linux/spinlock_types.h>
+#include <uapi/drm/xe_drm.h>
+
+enum {
+ __XE_SAMPLE_RENDER_GROUP_BUSY,
+ __XE_SAMPLE_COPY_GROUP_BUSY,
+ __XE_SAMPLE_MEDIA_GROUP_BUSY,
+ __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
+ __XE_NUM_PMU_SAMPLERS
+};
+
+#define XE_PMU_MAX_GT 2
+
+struct xe_pmu {
+ /**
+ * @cpuhp: Struct used for CPU hotplug handling.
+ */
+ struct {
+ struct hlist_node node;
+ unsigned int cpu;
+ } cpuhp;
+ /**
+ * @base: PMU base.
+ */
+ struct pmu base;
+ /**
+ * @closed: xe is unregistering.
+ */
+ bool closed;
+ /**
+ * @name: Name as registered with perf core.
+ */
+ const char *name;
+ /**
+ * @lock: Lock protecting enable mask and ref count handling.
+ */
+ spinlock_t lock;
+ /**
+ * @sample: Current and previous (raw) counters.
+ *
+ * These counters are updated when the device is awake.
+ */
+ u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
+ /**
+ * @events_attr_group: Device events attribute group.
+ */
+ struct attribute_group events_attr_group;
+ /**
+ * @xe_attr: Memory block holding device attributes.
+ */
+ void *xe_attr;
+ /**
+ * @pmu_attr: Memory block holding device attributes.
+ */
+ void *pmu_attr;
+};
+
+#endif
diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
index d7b0903c22b2..07ca545354f7 100644
--- a/include/uapi/drm/xe_drm.h
+++ b/include/uapi/drm/xe_drm.h
@@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
__u64 reserved[2];
};
+/**
+ * DOC: XE PMU event config IDs
+ *
+ * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in xe_drm.h
+ * in 'struct perf_event_attr' as part of perf_event_open syscall to read a
+ * particular event.
+ *
+ * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
+ *
+ * .. code-block:: C
+ *
+ * struct perf_event_attr attr;
+ * long long count;
+ * int cpu = 0;
+ * int fd;
+ *
+ * memset(&attr, 0, sizeof(struct perf_event_attr));
+ * attr.type = type; // eg: /sys/bus/event_source/devices/xe_0000_56_00.0/type
+ * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
+ * attr.use_clockid = 1;
+ * attr.clockid = CLOCK_MONOTONIC;
+ * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
+ *
+ * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
+ */
+
+/*
+ * Top bits of every counter are GT id.
+ */
+#define __XE_PMU_GT_SHIFT (56)
+
+#define ___XE_PMU_OTHER(gt, x) \
+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
+
+#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
+#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
+#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
+#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
+
#if defined(__cplusplus)
}
#endif
--
2.40.0
^ permalink raw reply related [flat|nested] 32+ messages in thread
* ✓ CI.Build: success for drm/xe/pmu: Enable PMU interface (rev9)
2024-06-13 10:04 [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Riana Tauro
` (4 preceding siblings ...)
2024-06-13 10:04 ` [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface Riana Tauro
@ 2024-06-13 10:11 ` Patchwork
2024-06-13 10:13 ` ✗ CI.Hooks: failure " Patchwork
` (3 subsequent siblings)
9 siblings, 0 replies; 32+ messages in thread
From: Patchwork @ 2024-06-13 10:11 UTC (permalink / raw)
To: Iddamsetty, Aravind; +Cc: intel-xe
== Series Details ==
Series: drm/xe/pmu: Enable PMU interface (rev9)
URL : https://patchwork.freedesktop.org/series/119504/
State : success
== Summary ==
lib/modules/6.10.0-rc3-xe/kernel/sound/core/seq/
lib/modules/6.10.0-rc3-xe/kernel/sound/core/seq/snd-seq.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/core/snd-seq-device.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/core/snd-hwdep.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/core/snd.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/core/snd-pcm.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/core/snd-compress.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/core/snd-timer.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soundcore.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/intel/
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/intel/atom/
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/intel/atom/snd-soc-sst-atom-hifi2-platform.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/intel/atom/sst/
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/intel/atom/sst/snd-intel-sst-acpi.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/intel/atom/sst/snd-intel-sst-core.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/intel/common/
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/intel/common/snd-soc-acpi-intel-match.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/amd/
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/amd/snd-acp-config.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/intel/
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/intel/snd-sof-pci-intel-tgl.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/intel/snd-sof-intel-hda-mlink.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/intel/snd-sof-pci-intel-cnl.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/intel/snd-sof-pci-intel-lnl.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/intel/snd-sof-intel-hda-common.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/intel/snd-sof-intel-hda-generic.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/intel/snd-sof-intel-hda.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/intel/snd-sof-pci-intel-mtl.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/amd/
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/amd/snd-sof-amd-renoir.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/amd/snd-sof-amd-acp.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/snd-sof-utils.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/snd-sof-pci.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/snd-sof.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/snd-sof-probes.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/xtensa/
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/sof/xtensa/snd-sof-xtensa-dsp.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/snd-soc-core.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/snd-soc-acpi.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/codecs/
lib/modules/6.10.0-rc3-xe/kernel/sound/soc/codecs/snd-soc-hdac-hda.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/hda/
lib/modules/6.10.0-rc3-xe/kernel/sound/hda/snd-intel-sdw-acpi.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/hda/ext/
lib/modules/6.10.0-rc3-xe/kernel/sound/hda/ext/snd-hda-ext-core.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/hda/snd-intel-dspcfg.ko
lib/modules/6.10.0-rc3-xe/kernel/sound/hda/snd-hda-core.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/kernel/
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/kernel/msr.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/kernel/cpuid.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/crypto/
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/crypto/sha512-ssse3.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/crypto/crct10dif-pclmul.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/crypto/ghash-clmulni-intel.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/crypto/sha1-ssse3.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/crypto/crc32-pclmul.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/crypto/sha256-ssse3.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/crypto/aesni-intel.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/crypto/polyval-clmulni.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/events/
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/events/intel/
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/events/intel/intel-cstate.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/events/rapl.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/kvm/
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/kvm/kvm.ko
lib/modules/6.10.0-rc3-xe/kernel/arch/x86/kvm/kvm-intel.ko
lib/modules/6.10.0-rc3-xe/kernel/crypto/
lib/modules/6.10.0-rc3-xe/kernel/crypto/crypto_simd.ko
lib/modules/6.10.0-rc3-xe/kernel/crypto/cmac.ko
lib/modules/6.10.0-rc3-xe/kernel/crypto/ccm.ko
lib/modules/6.10.0-rc3-xe/kernel/crypto/cryptd.ko
lib/modules/6.10.0-rc3-xe/kernel/crypto/polyval-generic.ko
lib/modules/6.10.0-rc3-xe/kernel/crypto/async_tx/
lib/modules/6.10.0-rc3-xe/kernel/crypto/async_tx/async_xor.ko
lib/modules/6.10.0-rc3-xe/kernel/crypto/async_tx/async_tx.ko
lib/modules/6.10.0-rc3-xe/kernel/crypto/async_tx/async_memcpy.ko
lib/modules/6.10.0-rc3-xe/kernel/crypto/async_tx/async_pq.ko
lib/modules/6.10.0-rc3-xe/kernel/crypto/async_tx/async_raid6_recov.ko
lib/modules/6.10.0-rc3-xe/build
lib/modules/6.10.0-rc3-xe/modules.alias.bin
lib/modules/6.10.0-rc3-xe/modules.builtin
lib/modules/6.10.0-rc3-xe/modules.softdep
lib/modules/6.10.0-rc3-xe/modules.alias
lib/modules/6.10.0-rc3-xe/modules.order
lib/modules/6.10.0-rc3-xe/modules.symbols
lib/modules/6.10.0-rc3-xe/modules.dep.bin
+ mv kernel-nodebug.tar.gz ..
+ cd ..
+ rm -rf archive
++ date +%s
+ echo -e '\e[0Ksection_end:1718273447:package_x86_64_nodebug\r\e[0K'
+ sync
^[[0Ksection_end:1718273447:package_x86_64_nodebug
^[[0K
+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel
^ permalink raw reply [flat|nested] 32+ messages in thread
* ✗ CI.Hooks: failure for drm/xe/pmu: Enable PMU interface (rev9)
2024-06-13 10:04 [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Riana Tauro
` (5 preceding siblings ...)
2024-06-13 10:11 ` ✓ CI.Build: success for drm/xe/pmu: Enable PMU interface (rev9) Patchwork
@ 2024-06-13 10:13 ` Patchwork
2024-06-13 10:14 ` ✓ CI.checksparse: success " Patchwork
` (2 subsequent siblings)
9 siblings, 0 replies; 32+ messages in thread
From: Patchwork @ 2024-06-13 10:13 UTC (permalink / raw)
To: Iddamsetty, Aravind; +Cc: intel-xe
== Series Details ==
Series: drm/xe/pmu: Enable PMU interface (rev9)
URL : https://patchwork.freedesktop.org/series/119504/
State : failure
== Summary ==
run-parts: executing /workspace/ci/hooks/00-showenv
+ export
+ grep -Ei '(^|\W)CI_'
declare -x CI_KERNEL_BUILD_DIR="/workspace/kernel/build64-default"
declare -x CI_KERNEL_SRC_DIR="/workspace/kernel"
declare -x CI_TOOLS_SRC_DIR="/workspace/ci"
declare -x CI_WORKSPACE_DIR="/workspace"
run-parts: executing /workspace/ci/hooks/10-build-W1
+ SRC_DIR=/workspace/kernel
+ RESTORE_DISPLAY_CONFIG=0
+ '[' -n /workspace/kernel/build64-default ']'
+ BUILD_DIR=/workspace/kernel/build64-default
+ cd /workspace/kernel
++ nproc
+ make -j48 O=/workspace/kernel/build64-default modules_prepare
make[1]: Entering directory '/workspace/kernel/build64-default'
GEN Makefile
UPD include/generated/compile.h
UPD include/config/kernel.release
mkdir -p /workspace/kernel/build64-default/tools/objtool && make O=/workspace/kernel/build64-default subdir=tools/objtool --no-print-directory -C objtool
UPD include/generated/utsrelease.h
HOSTCC /workspace/kernel/build64-default/tools/objtool/fixdep.o
CALL ../scripts/checksyscalls.sh
HOSTLD /workspace/kernel/build64-default/tools/objtool/fixdep-in.o
LINK /workspace/kernel/build64-default/tools/objtool/fixdep
INSTALL libsubcmd_headers
CC /workspace/kernel/build64-default/tools/objtool/libsubcmd/exec-cmd.o
CC /workspace/kernel/build64-default/tools/objtool/libsubcmd/help.o
CC /workspace/kernel/build64-default/tools/objtool/libsubcmd/pager.o
CC /workspace/kernel/build64-default/tools/objtool/libsubcmd/parse-options.o
CC /workspace/kernel/build64-default/tools/objtool/libsubcmd/run-command.o
CC /workspace/kernel/build64-default/tools/objtool/libsubcmd/sigchain.o
CC /workspace/kernel/build64-default/tools/objtool/libsubcmd/subcmd-config.o
LD /workspace/kernel/build64-default/tools/objtool/libsubcmd/libsubcmd-in.o
AR /workspace/kernel/build64-default/tools/objtool/libsubcmd/libsubcmd.a
CC /workspace/kernel/build64-default/tools/objtool/weak.o
CC /workspace/kernel/build64-default/tools/objtool/check.o
CC /workspace/kernel/build64-default/tools/objtool/special.o
CC /workspace/kernel/build64-default/tools/objtool/builtin-check.o
CC /workspace/kernel/build64-default/tools/objtool/elf.o
CC /workspace/kernel/build64-default/tools/objtool/objtool.o
CC /workspace/kernel/build64-default/tools/objtool/orc_gen.o
CC /workspace/kernel/build64-default/tools/objtool/orc_dump.o
CC /workspace/kernel/build64-default/tools/objtool/libstring.o
CC /workspace/kernel/build64-default/tools/objtool/libctype.o
CC /workspace/kernel/build64-default/tools/objtool/str_error_r.o
CC /workspace/kernel/build64-default/tools/objtool/librbtree.o
CC /workspace/kernel/build64-default/tools/objtool/arch/x86/special.o
CC /workspace/kernel/build64-default/tools/objtool/arch/x86/decode.o
CC /workspace/kernel/build64-default/tools/objtool/arch/x86/orc.o
LD /workspace/kernel/build64-default/tools/objtool/arch/x86/objtool-in.o
LD /workspace/kernel/build64-default/tools/objtool/objtool-in.o
LINK /workspace/kernel/build64-default/tools/objtool/objtool
make[1]: Leaving directory '/workspace/kernel/build64-default'
++ nproc
+ make -j48 O=/workspace/kernel/build64-default W=1 drivers/gpu/drm/xe
make[1]: Entering directory '/workspace/kernel/build64-default'
make[2]: Nothing to be done for 'drivers/gpu/drm/xe'.
make[1]: Leaving directory '/workspace/kernel/build64-default'
run-parts: executing /workspace/ci/hooks/11-build-32b
+++ realpath /workspace/ci/hooks/11-build-32b
++ dirname /workspace/ci/hooks/11-build-32b
+ THIS_SCRIPT_DIR=/workspace/ci/hooks
+ SRC_DIR=/workspace/kernel
+ TOOLS_SRC_DIR=/workspace/ci
+ '[' -n /workspace/kernel/build64-default ']'
+ BUILD_DIR=/workspace/kernel/build64-default
+ BUILD_DIR=/workspace/kernel/build64-default/build32
+ cd /workspace/kernel
+ mkdir -p /workspace/kernel/build64-default/build32
++ nproc
+ make -j48 ARCH=i386 O=/workspace/kernel/build64-default/build32 defconfig
make[1]: Entering directory '/workspace/kernel/build64-default/build32'
GEN Makefile
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
LEX scripts/kconfig/lexer.lex.c
YACC scripts/kconfig/parser.tab.[ch]
HOSTCC scripts/kconfig/menu.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTCC scripts/kconfig/util.o
HOSTCC scripts/kconfig/lexer.lex.o
HOSTCC scripts/kconfig/parser.tab.o
HOSTLD scripts/kconfig/conf
*** Default configuration is based on 'i386_defconfig'
#
# configuration written to .config
#
make[1]: Leaving directory '/workspace/kernel/build64-default/build32'
+ cd /workspace/kernel/build64-default/build32
+ /workspace/kernel/scripts/kconfig/merge_config.sh .config /workspace/ci/kernel/10-xe.fragment
Using .config as base
Merging /workspace/ci/kernel/10-xe.fragment
Value of CONFIG_DRM_XE is redefined by fragment /workspace/ci/kernel/10-xe.fragment:
Previous value: # CONFIG_DRM_XE is not set
New value: CONFIG_DRM_XE=m
Value of CONFIG_SND_DEBUG is redefined by fragment /workspace/ci/kernel/10-xe.fragment:
Previous value: # CONFIG_SND_DEBUG is not set
New value: CONFIG_SND_DEBUG=y
Value of CONFIG_SND_HDA_INTEL is redefined by fragment /workspace/ci/kernel/10-xe.fragment:
Previous value: CONFIG_SND_HDA_INTEL=y
New value: CONFIG_SND_HDA_INTEL=m
Value of CONFIG_SND_HDA_CODEC_HDMI is redefined by fragment /workspace/ci/kernel/10-xe.fragment:
Previous value: # CONFIG_SND_HDA_CODEC_HDMI is not set
New value: CONFIG_SND_HDA_CODEC_HDMI=m
GEN Makefile
WARNING: unmet direct dependencies detected for FB_IOMEM_HELPERS
Depends on [n]: HAS_IOMEM [=y] && FB_CORE [=n]
Selected by [m]:
- DRM_XE_DISPLAY [=y] && HAS_IOMEM [=y] && DRM [=y] && DRM_XE [=m] && DRM_XE [=m]=m [=m]
#
# configuration written to .config
#
Value requested for CONFIG_HAVE_UID16 not in final .config
Requested value: CONFIG_HAVE_UID16=y
Actual value:
Value requested for CONFIG_UID16 not in final .config
Requested value: CONFIG_UID16=y
Actual value:
Value requested for CONFIG_X86_32 not in final .config
Requested value: CONFIG_X86_32=y
Actual value:
Value requested for CONFIG_OUTPUT_FORMAT not in final .config
Requested value: CONFIG_OUTPUT_FORMAT="elf32-i386"
Actual value: CONFIG_OUTPUT_FORMAT="elf64-x86-64"
Value requested for CONFIG_ARCH_MMAP_RND_BITS_MIN not in final .config
Requested value: CONFIG_ARCH_MMAP_RND_BITS_MIN=8
Actual value: CONFIG_ARCH_MMAP_RND_BITS_MIN=28
Value requested for CONFIG_ARCH_MMAP_RND_BITS_MAX not in final .config
Requested value: CONFIG_ARCH_MMAP_RND_BITS_MAX=16
Actual value: CONFIG_ARCH_MMAP_RND_BITS_MAX=32
Value requested for CONFIG_PGTABLE_LEVELS not in final .config
Requested value: CONFIG_PGTABLE_LEVELS=2
Actual value: CONFIG_PGTABLE_LEVELS=5
Value requested for CONFIG_X86_BIGSMP not in final .config
Requested value: # CONFIG_X86_BIGSMP is not set
Actual value:
Value requested for CONFIG_X86_INTEL_QUARK not in final .config
Requested value: # CONFIG_X86_INTEL_QUARK is not set
Actual value:
Value requested for CONFIG_X86_RDC321X not in final .config
Requested value: # CONFIG_X86_RDC321X is not set
Actual value:
Value requested for CONFIG_X86_32_NON_STANDARD not in final .config
Requested value: # CONFIG_X86_32_NON_STANDARD is not set
Actual value:
Value requested for CONFIG_X86_32_IRIS not in final .config
Requested value: # CONFIG_X86_32_IRIS is not set
Actual value:
Value requested for CONFIG_M486SX not in final .config
Requested value: # CONFIG_M486SX is not set
Actual value:
Value requested for CONFIG_M486 not in final .config
Requested value: # CONFIG_M486 is not set
Actual value:
Value requested for CONFIG_M586 not in final .config
Requested value: # CONFIG_M586 is not set
Actual value:
Value requested for CONFIG_M586TSC not in final .config
Requested value: # CONFIG_M586TSC is not set
Actual value:
Value requested for CONFIG_M586MMX not in final .config
Requested value: # CONFIG_M586MMX is not set
Actual value:
Value requested for CONFIG_M686 not in final .config
Requested value: CONFIG_M686=y
Actual value:
Value requested for CONFIG_MPENTIUMII not in final .config
Requested value: # CONFIG_MPENTIUMII is not set
Actual value:
Value requested for CONFIG_MPENTIUMIII not in final .config
Requested value: # CONFIG_MPENTIUMIII is not set
Actual value:
Value requested for CONFIG_MPENTIUMM not in final .config
Requested value: # CONFIG_MPENTIUMM is not set
Actual value:
Value requested for CONFIG_MPENTIUM4 not in final .config
Requested value: # CONFIG_MPENTIUM4 is not set
Actual value:
Value requested for CONFIG_MK6 not in final .config
Requested value: # CONFIG_MK6 is not set
Actual value:
Value requested for CONFIG_MK7 not in final .config
Requested value: # CONFIG_MK7 is not set
Actual value:
Value requested for CONFIG_MCRUSOE not in final .config
Requested value: # CONFIG_MCRUSOE is not set
Actual value:
Value requested for CONFIG_MEFFICEON not in final .config
Requested value: # CONFIG_MEFFICEON is not set
Actual value:
Value requested for CONFIG_MWINCHIPC6 not in final .config
Requested value: # CONFIG_MWINCHIPC6 is not set
Actual value:
Value requested for CONFIG_MWINCHIP3D not in final .config
Requested value: # CONFIG_MWINCHIP3D is not set
Actual value:
Value requested for CONFIG_MELAN not in final .config
Requested value: # CONFIG_MELAN is not set
Actual value:
Value requested for CONFIG_MGEODEGX1 not in final .config
Requested value: # CONFIG_MGEODEGX1 is not set
Actual value:
Value requested for CONFIG_MGEODE_LX not in final .config
Requested value: # CONFIG_MGEODE_LX is not set
Actual value:
Value requested for CONFIG_MCYRIXIII not in final .config
Requested value: # CONFIG_MCYRIXIII is not set
Actual value:
Value requested for CONFIG_MVIAC3_2 not in final .config
Requested value: # CONFIG_MVIAC3_2 is not set
Actual value:
Value requested for CONFIG_MVIAC7 not in final .config
Requested value: # CONFIG_MVIAC7 is not set
Actual value:
Value requested for CONFIG_X86_GENERIC not in final .config
Requested value: # CONFIG_X86_GENERIC is not set
Actual value:
Value requested for CONFIG_X86_INTERNODE_CACHE_SHIFT not in final .config
Requested value: CONFIG_X86_INTERNODE_CACHE_SHIFT=5
Actual value: CONFIG_X86_INTERNODE_CACHE_SHIFT=6
Value requested for CONFIG_X86_L1_CACHE_SHIFT not in final .config
Requested value: CONFIG_X86_L1_CACHE_SHIFT=5
Actual value: CONFIG_X86_L1_CACHE_SHIFT=6
Value requested for CONFIG_X86_USE_PPRO_CHECKSUM not in final .config
Requested value: CONFIG_X86_USE_PPRO_CHECKSUM=y
Actual value:
Value requested for CONFIG_X86_MINIMUM_CPU_FAMILY not in final .config
Requested value: CONFIG_X86_MINIMUM_CPU_FAMILY=6
Actual value: CONFIG_X86_MINIMUM_CPU_FAMILY=64
Value requested for CONFIG_CPU_SUP_TRANSMETA_32 not in final .config
Requested value: CONFIG_CPU_SUP_TRANSMETA_32=y
Actual value:
Value requested for CONFIG_CPU_SUP_VORTEX_32 not in final .config
Requested value: CONFIG_CPU_SUP_VORTEX_32=y
Actual value:
Value requested for CONFIG_HPET_TIMER not in final .config
Requested value: # CONFIG_HPET_TIMER is not set
Actual value: CONFIG_HPET_TIMER=y
Value requested for CONFIG_NR_CPUS_RANGE_END not in final .config
Requested value: CONFIG_NR_CPUS_RANGE_END=8
Actual value: CONFIG_NR_CPUS_RANGE_END=512
Value requested for CONFIG_NR_CPUS_DEFAULT not in final .config
Requested value: CONFIG_NR_CPUS_DEFAULT=8
Actual value: CONFIG_NR_CPUS_DEFAULT=64
Value requested for CONFIG_X86_ANCIENT_MCE not in final .config
Requested value: # CONFIG_X86_ANCIENT_MCE is not set
Actual value:
Value requested for CONFIG_X86_LEGACY_VM86 not in final .config
Requested value: # CONFIG_X86_LEGACY_VM86 is not set
Actual value:
Value requested for CONFIG_X86_ESPFIX32 not in final .config
Requested value: CONFIG_X86_ESPFIX32=y
Actual value:
Value requested for CONFIG_TOSHIBA not in final .config
Requested value: # CONFIG_TOSHIBA is not set
Actual value:
Value requested for CONFIG_X86_REBOOTFIXUPS not in final .config
Requested value: # CONFIG_X86_REBOOTFIXUPS is not set
Actual value:
Value requested for CONFIG_MICROCODE_INITRD32 not in final .config
Requested value: CONFIG_MICROCODE_INITRD32=y
Actual value:
Value requested for CONFIG_NOHIGHMEM not in final .config
Requested value: # CONFIG_NOHIGHMEM is not set
Actual value:
Value requested for CONFIG_HIGHMEM4G not in final .config
Requested value: CONFIG_HIGHMEM4G=y
Actual value:
Value requested for CONFIG_HIGHMEM64G not in final .config
Requested value: # CONFIG_HIGHMEM64G is not set
Actual value:
Value requested for CONFIG_PAGE_OFFSET not in final .config
Requested value: CONFIG_PAGE_OFFSET=0xC0000000
Actual value:
Value requested for CONFIG_HIGHMEM not in final .config
Requested value: CONFIG_HIGHMEM=y
Actual value:
Value requested for CONFIG_X86_PAE not in final .config
Requested value: # CONFIG_X86_PAE is not set
Actual value:
Value requested for CONFIG_ARCH_FLATMEM_ENABLE not in final .config
Requested value: CONFIG_ARCH_FLATMEM_ENABLE=y
Actual value:
Value requested for CONFIG_ARCH_SELECT_MEMORY_MODEL not in final .config
Requested value: CONFIG_ARCH_SELECT_MEMORY_MODEL=y
Actual value:
Value requested for CONFIG_ILLEGAL_POINTER_VALUE not in final .config
Requested value: CONFIG_ILLEGAL_POINTER_VALUE=0
Actual value: CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
Value requested for CONFIG_HIGHPTE not in final .config
Requested value: # CONFIG_HIGHPTE is not set
Actual value:
Value requested for CONFIG_COMPAT_VDSO not in final .config
Requested value: # CONFIG_COMPAT_VDSO is not set
Actual value:
Value requested for CONFIG_FUNCTION_PADDING_CFI not in final .config
Requested value: CONFIG_FUNCTION_PADDING_CFI=0
Actual value: CONFIG_FUNCTION_PADDING_CFI=11
Value requested for CONFIG_FUNCTION_PADDING_BYTES not in final .config
Requested value: CONFIG_FUNCTION_PADDING_BYTES=4
Actual value: CONFIG_FUNCTION_PADDING_BYTES=16
Value requested for CONFIG_APM not in final .config
Requested value: # CONFIG_APM is not set
Actual value:
Value requested for CONFIG_X86_POWERNOW_K6 not in final .config
Requested value: # CONFIG_X86_POWERNOW_K6 is not set
Actual value:
Value requested for CONFIG_X86_POWERNOW_K7 not in final .config
Requested value: # CONFIG_X86_POWERNOW_K7 is not set
Actual value:
Value requested for CONFIG_X86_GX_SUSPMOD not in final .config
Requested value: # CONFIG_X86_GX_SUSPMOD is not set
Actual value:
Value requested for CONFIG_X86_SPEEDSTEP_ICH not in final .config
Requested value: # CONFIG_X86_SPEEDSTEP_ICH is not set
Actual value:
Value requested for CONFIG_X86_SPEEDSTEP_SMI not in final .config
Requested value: # CONFIG_X86_SPEEDSTEP_SMI is not set
Actual value:
Value requested for CONFIG_X86_CPUFREQ_NFORCE2 not in final .config
Requested value: # CONFIG_X86_CPUFREQ_NFORCE2 is not set
Actual value:
Value requested for CONFIG_X86_LONGRUN not in final .config
Requested value: # CONFIG_X86_LONGRUN is not set
Actual value:
Value requested for CONFIG_X86_LONGHAUL not in final .config
Requested value: # CONFIG_X86_LONGHAUL is not set
Actual value:
Value requested for CONFIG_X86_E_POWERSAVER not in final .config
Requested value: # CONFIG_X86_E_POWERSAVER is not set
Actual value:
Value requested for CONFIG_PCI_GOBIOS not in final .config
Requested value: # CONFIG_PCI_GOBIOS is not set
Actual value:
Value requested for CONFIG_PCI_GOMMCONFIG not in final .config
Requested value: # CONFIG_PCI_GOMMCONFIG is not set
Actual value:
Value requested for CONFIG_PCI_GODIRECT not in final .config
Requested value: # CONFIG_PCI_GODIRECT is not set
Actual value:
Value requested for CONFIG_PCI_GOANY not in final .config
Requested value: CONFIG_PCI_GOANY=y
Actual value:
Value requested for CONFIG_PCI_BIOS not in final .config
Requested value: CONFIG_PCI_BIOS=y
Actual value:
Value requested for CONFIG_ISA not in final .config
Requested value: # CONFIG_ISA is not set
Actual value:
Value requested for CONFIG_SCx200 not in final .config
Requested value: # CONFIG_SCx200 is not set
Actual value:
Value requested for CONFIG_OLPC not in final .config
Requested value: # CONFIG_OLPC is not set
Actual value:
Value requested for CONFIG_ALIX not in final .config
Requested value: # CONFIG_ALIX is not set
Actual value:
Value requested for CONFIG_NET5501 not in final .config
Requested value: # CONFIG_NET5501 is not set
Actual value:
Value requested for CONFIG_GEOS not in final .config
Requested value: # CONFIG_GEOS is not set
Actual value:
Value requested for CONFIG_COMPAT_32 not in final .config
Requested value: CONFIG_COMPAT_32=y
Actual value:
Value requested for CONFIG_HAVE_ATOMIC_IOMAP not in final .config
Requested value: CONFIG_HAVE_ATOMIC_IOMAP=y
Actual value:
Value requested for CONFIG_ARCH_32BIT_OFF_T not in final .config
Requested value: CONFIG_ARCH_32BIT_OFF_T=y
Actual value:
Value requested for CONFIG_ARCH_WANT_IPC_PARSE_VERSION not in final .config
Requested value: CONFIG_ARCH_WANT_IPC_PARSE_VERSION=y
Actual value:
Value requested for CONFIG_MODULES_USE_ELF_REL not in final .config
Requested value: CONFIG_MODULES_USE_ELF_REL=y
Actual value:
Value requested for CONFIG_ARCH_MMAP_RND_BITS not in final .config
Requested value: CONFIG_ARCH_MMAP_RND_BITS=8
Actual value: CONFIG_ARCH_MMAP_RND_BITS=28
Value requested for CONFIG_CLONE_BACKWARDS not in final .config
Requested value: CONFIG_CLONE_BACKWARDS=y
Actual value:
Value requested for CONFIG_OLD_SIGSUSPEND3 not in final .config
Requested value: CONFIG_OLD_SIGSUSPEND3=y
Actual value:
Value requested for CONFIG_OLD_SIGACTION not in final .config
Requested value: CONFIG_OLD_SIGACTION=y
Actual value:
Value requested for CONFIG_ARCH_SPLIT_ARG64 not in final .config
Requested value: CONFIG_ARCH_SPLIT_ARG64=y
Actual value:
Value requested for CONFIG_FUNCTION_ALIGNMENT not in final .config
Requested value: CONFIG_FUNCTION_ALIGNMENT=4
Actual value: CONFIG_FUNCTION_ALIGNMENT=16
Value requested for CONFIG_SELECT_MEMORY_MODEL not in final .config
Requested value: CONFIG_SELECT_MEMORY_MODEL=y
Actual value:
Value requested for CONFIG_FLATMEM_MANUAL not in final .config
Requested value: CONFIG_FLATMEM_MANUAL=y
Actual value:
Value requested for CONFIG_SPARSEMEM_MANUAL not in final .config
Requested value: # CONFIG_SPARSEMEM_MANUAL is not set
Actual value:
Value requested for CONFIG_FLATMEM not in final .config
Requested value: CONFIG_FLATMEM=y
Actual value:
Value requested for CONFIG_SPARSEMEM_STATIC not in final .config
Requested value: CONFIG_SPARSEMEM_STATIC=y
Actual value:
Value requested for CONFIG_BOUNCE not in final .config
Requested value: CONFIG_BOUNCE=y
Actual value:
Value requested for CONFIG_KMAP_LOCAL not in final .config
Requested value: CONFIG_KMAP_LOCAL=y
Actual value:
Value requested for CONFIG_HOTPLUG_PCI_COMPAQ not in final .config
Requested value: # CONFIG_HOTPLUG_PCI_COMPAQ is not set
Actual value:
Value requested for CONFIG_HOTPLUG_PCI_IBM not in final .config
Requested value: # CONFIG_HOTPLUG_PCI_IBM is not set
Actual value:
Value requested for CONFIG_EFI_CAPSULE_QUIRK_QUARK_CSH not in final .config
Requested value: CONFIG_EFI_CAPSULE_QUIRK_QUARK_CSH=y
Actual value:
Value requested for CONFIG_PCH_PHUB not in final .config
Requested value: # CONFIG_PCH_PHUB is not set
Actual value:
Value requested for CONFIG_SCSI_NSP32 not in final .config
Requested value: # CONFIG_SCSI_NSP32 is not set
Actual value:
Value requested for CONFIG_PATA_CS5520 not in final .config
Requested value: # CONFIG_PATA_CS5520 is not set
Actual value:
Value requested for CONFIG_PATA_CS5530 not in final .config
Requested value: # CONFIG_PATA_CS5530 is not set
Actual value:
Value requested for CONFIG_PATA_CS5535 not in final .config
Requested value: # CONFIG_PATA_CS5535 is not set
Actual value:
Value requested for CONFIG_PATA_CS5536 not in final .config
Requested value: # CONFIG_PATA_CS5536 is not set
Actual value:
Value requested for CONFIG_PATA_SC1200 not in final .config
Requested value: # CONFIG_PATA_SC1200 is not set
Actual value:
Value requested for CONFIG_PCH_GBE not in final .config
Requested value: # CONFIG_PCH_GBE is not set
Actual value:
Value requested for CONFIG_INPUT_WISTRON_BTNS not in final .config
Requested value: # CONFIG_INPUT_WISTRON_BTNS is not set
Actual value:
Value requested for CONFIG_SERIAL_TIMBERDALE not in final .config
Requested value: # CONFIG_SERIAL_TIMBERDALE is not set
Actual value:
Value requested for CONFIG_SERIAL_PCH_UART not in final .config
Requested value: # CONFIG_SERIAL_PCH_UART is not set
Actual value:
Value requested for CONFIG_HW_RANDOM_GEODE not in final .config
Requested value: CONFIG_HW_RANDOM_GEODE=y
Actual value:
Value requested for CONFIG_SONYPI not in final .config
Requested value: # CONFIG_SONYPI is not set
Actual value:
Value requested for CONFIG_PC8736x_GPIO not in final .config
Requested value: # CONFIG_PC8736x_GPIO is not set
Actual value:
Value requested for CONFIG_NSC_GPIO not in final .config
Requested value: # CONFIG_NSC_GPIO is not set
Actual value:
Value requested for CONFIG_I2C_EG20T not in final .config
Requested value: # CONFIG_I2C_EG20T is not set
Actual value:
Value requested for CONFIG_SCx200_ACB not in final .config
Requested value: # CONFIG_SCx200_ACB is not set
Actual value:
Value requested for CONFIG_PTP_1588_CLOCK_PCH not in final .config
Requested value: # CONFIG_PTP_1588_CLOCK_PCH is not set
Actual value:
Value requested for CONFIG_SBC8360_WDT not in final .config
Requested value: # CONFIG_SBC8360_WDT is not set
Actual value:
Value requested for CONFIG_SBC7240_WDT not in final .config
Requested value: # CONFIG_SBC7240_WDT is not set
Actual value:
Value requested for CONFIG_MFD_CS5535 not in final .config
Requested value: # CONFIG_MFD_CS5535 is not set
Actual value:
Value requested for CONFIG_AGP_ALI not in final .config
Requested value: # CONFIG_AGP_ALI is not set
Actual value:
Value requested for CONFIG_AGP_ATI not in final .config
Requested value: # CONFIG_AGP_ATI is not set
Actual value:
Value requested for CONFIG_AGP_AMD not in final .config
Requested value: # CONFIG_AGP_AMD is not set
Actual value:
Value requested for CONFIG_AGP_NVIDIA not in final .config
Requested value: # CONFIG_AGP_NVIDIA is not set
Actual value:
Value requested for CONFIG_AGP_SWORKS not in final .config
Requested value: # CONFIG_AGP_SWORKS is not set
Actual value:
Value requested for CONFIG_AGP_EFFICEON not in final .config
Requested value: # CONFIG_AGP_EFFICEON is not set
Actual value:
Value requested for CONFIG_SND_PCM not in final .config
Requested value: CONFIG_SND_PCM=y
Actual value: CONFIG_SND_PCM=m
Value requested for CONFIG_SND_HWDEP not in final .config
Requested value: CONFIG_SND_HWDEP=y
Actual value: CONFIG_SND_HWDEP=m
Value requested for CONFIG_SND_DYNAMIC_MINORS not in final .config
Requested value: # CONFIG_SND_DYNAMIC_MINORS is not set
Actual value: CONFIG_SND_DYNAMIC_MINORS=y
Value requested for CONFIG_SND_CS5530 not in final .config
Requested value: # CONFIG_SND_CS5530 is not set
Actual value:
Value requested for CONFIG_SND_CS5535AUDIO not in final .config
Requested value: # CONFIG_SND_CS5535AUDIO is not set
Actual value:
Value requested for CONFIG_SND_SIS7019 not in final .config
Requested value: # CONFIG_SND_SIS7019 is not set
Actual value:
Value requested for CONFIG_SND_HDA not in final .config
Requested value: CONFIG_SND_HDA=y
Actual value: CONFIG_SND_HDA=m
Value requested for CONFIG_SND_HDA_CORE not in final .config
Requested value: CONFIG_SND_HDA_CORE=y
Actual value: CONFIG_SND_HDA_CORE=m
Value requested for CONFIG_SND_INTEL_DSP_CONFIG not in final .config
Requested value: CONFIG_SND_INTEL_DSP_CONFIG=y
Actual value: CONFIG_SND_INTEL_DSP_CONFIG=m
Value requested for CONFIG_SND_INTEL_SOUNDWIRE_ACPI not in final .config
Requested value: CONFIG_SND_INTEL_SOUNDWIRE_ACPI=y
Actual value: CONFIG_SND_INTEL_SOUNDWIRE_ACPI=m
Value requested for CONFIG_LEDS_OT200 not in final .config
Requested value: # CONFIG_LEDS_OT200 is not set
Actual value:
Value requested for CONFIG_PCH_DMA not in final .config
Requested value: # CONFIG_PCH_DMA is not set
Actual value:
Value requested for CONFIG_CLKSRC_I8253 not in final .config
Requested value: CONFIG_CLKSRC_I8253=y
Actual value:
Value requested for CONFIG_MAILBOX not in final .config
Requested value: # CONFIG_MAILBOX is not set
Actual value: CONFIG_MAILBOX=y
Value requested for CONFIG_CRYPTO_SERPENT_SSE2_586 not in final .config
Requested value: # CONFIG_CRYPTO_SERPENT_SSE2_586 is not set
Actual value:
Value requested for CONFIG_CRYPTO_TWOFISH_586 not in final .config
Requested value: # CONFIG_CRYPTO_TWOFISH_586 is not set
Actual value:
Value requested for CONFIG_CRYPTO_DEV_GEODE not in final .config
Requested value: # CONFIG_CRYPTO_DEV_GEODE is not set
Actual value:
Value requested for CONFIG_CRYPTO_DEV_HIFN_795X not in final .config
Requested value: # CONFIG_CRYPTO_DEV_HIFN_795X is not set
Actual value:
Value requested for CONFIG_CRYPTO_LIB_POLY1305_RSIZE not in final .config
Requested value: CONFIG_CRYPTO_LIB_POLY1305_RSIZE=1
Actual value: CONFIG_CRYPTO_LIB_POLY1305_RSIZE=11
Value requested for CONFIG_AUDIT_GENERIC not in final .config
Requested value: CONFIG_AUDIT_GENERIC=y
Actual value:
Value requested for CONFIG_GENERIC_VDSO_32 not in final .config
Requested value: CONFIG_GENERIC_VDSO_32=y
Actual value:
Value requested for CONFIG_DEBUG_KMAP_LOCAL not in final .config
Requested value: # CONFIG_DEBUG_KMAP_LOCAL is not set
Actual value:
Value requested for CONFIG_DEBUG_HIGHMEM not in final .config
Requested value: # CONFIG_DEBUG_HIGHMEM is not set
Actual value:
Value requested for CONFIG_HAVE_DEBUG_STACKOVERFLOW not in final .config
Requested value: CONFIG_HAVE_DEBUG_STACKOVERFLOW=y
Actual value:
Value requested for CONFIG_DEBUG_STACKOVERFLOW not in final .config
Requested value: # CONFIG_DEBUG_STACKOVERFLOW is not set
Actual value:
Value requested for CONFIG_HAVE_FUNCTION_GRAPH_TRACER not in final .config
Requested value: CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
Actual value:
Value requested for CONFIG_HAVE_FUNCTION_GRAPH_RETVAL not in final .config
Requested value: CONFIG_HAVE_FUNCTION_GRAPH_RETVAL=y
Actual value:
Value requested for CONFIG_DRM_KUNIT_TEST not in final .config
Requested value: CONFIG_DRM_KUNIT_TEST=m
Actual value:
Value requested for CONFIG_DRM_XE_WERROR not in final .config
Requested value: CONFIG_DRM_XE_WERROR=y
Actual value:
Value requested for CONFIG_DRM_XE_DEBUG not in final .config
Requested value: CONFIG_DRM_XE_DEBUG=y
Actual value:
Value requested for CONFIG_DRM_XE_DEBUG_MEM not in final .config
Requested value: CONFIG_DRM_XE_DEBUG_MEM=y
Actual value:
Value requested for CONFIG_DRM_XE_KUNIT_TEST not in final .config
Requested value: CONFIG_DRM_XE_KUNIT_TEST=m
Actual value:
++ nproc
+ make -j48 ARCH=i386 olddefconfig
GEN Makefile
WARNING: unmet direct dependencies detected for FB_IOMEM_HELPERS
Depends on [n]: HAS_IOMEM [=y] && FB_CORE [=n]
Selected by [m]:
- DRM_XE_DISPLAY [=y] && HAS_IOMEM [=y] && DRM [=y] && DRM_XE [=m] && DRM_XE [=m]=m [=m]
#
# configuration written to .config
#
++ nproc
+ make -j48 ARCH=i386
SYNC include/config/auto.conf.cmd
GEN Makefile
WARNING: unmet direct dependencies detected for FB_IOMEM_HELPERS
Depends on [n]: HAS_IOMEM [=y] && FB_CORE [=n]
Selected by [m]:
- DRM_XE_DISPLAY [=y] && HAS_IOMEM [=y] && DRM [=y] && DRM_XE [=m] && DRM_XE [=m]=m [=m]
WARNING: unmet direct dependencies detected for FB_IOMEM_HELPERS
Depends on [n]: HAS_IOMEM [=y] && FB_CORE [=n]
Selected by [m]:
- DRM_XE_DISPLAY [=y] && HAS_IOMEM [=y] && DRM [=y] && DRM_XE [=m] && DRM_XE [=m]=m [=m]
WARNING: unmet direct dependencies detected for FB_IOMEM_HELPERS
Depends on [n]: HAS_IOMEM [=y] && FB_CORE [=n]
Selected by [m]:
- DRM_XE_DISPLAY [=y] && HAS_IOMEM [=y] && DRM [=y] && DRM_XE [=m] && DRM_XE [=m]=m [=m]
GEN Makefile
WRAP arch/x86/include/generated/uapi/asm/bpf_perf_event.h
WRAP arch/x86/include/generated/uapi/asm/errno.h
WRAP arch/x86/include/generated/uapi/asm/fcntl.h
WRAP arch/x86/include/generated/uapi/asm/ioctl.h
WRAP arch/x86/include/generated/uapi/asm/ioctls.h
WRAP arch/x86/include/generated/uapi/asm/ipcbuf.h
UPD include/generated/uapi/linux/version.h
WRAP arch/x86/include/generated/uapi/asm/param.h
WRAP arch/x86/include/generated/uapi/asm/poll.h
WRAP arch/x86/include/generated/uapi/asm/resource.h
WRAP arch/x86/include/generated/uapi/asm/socket.h
WRAP arch/x86/include/generated/uapi/asm/sockios.h
WRAP arch/x86/include/generated/uapi/asm/termbits.h
WRAP arch/x86/include/generated/uapi/asm/types.h
WRAP arch/x86/include/generated/uapi/asm/termios.h
SYSHDR arch/x86/include/generated/uapi/asm/unistd_64.h
SYSHDR arch/x86/include/generated/uapi/asm/unistd_x32.h
SYSHDR arch/x86/include/generated/uapi/asm/unistd_32.h
SYSTBL arch/x86/include/generated/asm/syscalls_32.h
WRAP arch/x86/include/generated/asm/early_ioremap.h
UPD include/generated/compile.h
WRAP arch/x86/include/generated/asm/mcs_spinlock.h
WRAP arch/x86/include/generated/asm/irq_regs.h
WRAP arch/x86/include/generated/asm/kmap_size.h
WRAP arch/x86/include/generated/asm/local64.h
WRAP arch/x86/include/generated/asm/mmiowb.h
WRAP arch/x86/include/generated/asm/module.lds.h
HOSTCC arch/x86/tools/relocs_32.o
WRAP arch/x86/include/generated/asm/rwonce.h
WRAP arch/x86/include/generated/asm/unaligned.h
HOSTCC arch/x86/tools/relocs_64.o
HOSTCC arch/x86/tools/relocs_common.o
HOSTCC scripts/kallsyms
HOSTCC scripts/sorttable
HOSTCC scripts/asn1_compiler
HOSTCC scripts/selinux/genheaders/genheaders
HOSTCC scripts/selinux/mdp/mdp
HOSTLD arch/x86/tools/relocs
UPD include/config/kernel.release
UPD include/generated/utsrelease.h
CC scripts/mod/empty.o
HOSTCC scripts/mod/mk_elfconfig
CC scripts/mod/devicetable-offsets.s
UPD scripts/mod/devicetable-offsets.h
MKELF scripts/mod/elfconfig.h
HOSTCC scripts/mod/modpost.o
HOSTCC scripts/mod/file2alias.o
HOSTCC scripts/mod/sumversion.o
HOSTCC scripts/mod/symsearch.o
HOSTLD scripts/mod/modpost
CC kernel/bounds.s
CHKSHA1 /workspace/kernel/include/linux/atomic/atomic-arch-fallback.h
CHKSHA1 /workspace/kernel/include/linux/atomic/atomic-instrumented.h
CHKSHA1 /workspace/kernel/include/linux/atomic/atomic-long.h
UPD include/generated/timeconst.h
UPD include/generated/bounds.h
CC arch/x86/kernel/asm-offsets.s
UPD include/generated/asm-offsets.h
CALL /workspace/kernel/scripts/checksyscalls.sh
LDS scripts/module.lds
CC init/main.o
HOSTCC usr/gen_init_cpio
CC init/do_mounts.o
CC init/do_mounts_initrd.o
CC security/commoncap.o
CC init/initramfs.o
CC certs/system_keyring.o
UPD init/utsversion-tmp.h
CC block/bdev.o
CC io_uring/io_uring.o
CC init/calibrate.o
CC security/lsm_syscalls.o
CC block/fops.o
CC ipc/util.o
CC init/init_task.o
CC arch/x86/realmode/init.o
CC io_uring/opdef.o
CC ipc/msgutil.o
CC mm/filemap.o
AS arch/x86/lib/atomic64_cx8_32.o
CC security/min_addr.o
CC block/bio.o
CC arch/x86/power/cpu.o
AR arch/x86/crypto/built-in.a
CC arch/x86/pci/i386.o
CC block/elevator.o
CC arch/x86/video/video-common.o
CC security/integrity/iint.o
AS arch/x86/realmode/rm/header.o
CC block/partitions/core.o
CC security/keys/gc.o
GEN security/selinux/flask.h security/selinux/av_permissions.h
AR arch/x86/net/built-in.a
CC arch/x86/mm/pat/set_memory.o
AR drivers/cache/built-in.a
AR virt/lib/built-in.a
CC fs/nfs_common/nfsacl.o
CC lib/math/div64.o
AR sound/i2c/other/built-in.a
CC net/core/sock.o
CC sound/core/seq/seq.o
CC lib/math/gcd.o
CC security/selinux/avc.o
CC arch/x86/events/amd/core.o
AR arch/x86/platform/atom/built-in.a
AR virt/built-in.a
AR sound/i2c/built-in.a
AR arch/x86/virt/svm/built-in.a
AR arch/x86/platform/ce4100/built-in.a
CC fs/notify/dnotify/dnotify.o
AS arch/x86/lib/checksum_32.o
CC arch/x86/kernel/fpu/init.o
AR drivers/irqchip/built-in.a
CC net/core/request_sock.o
CC arch/x86/lib/cmdline.o
AR arch/x86/virt/vmx/built-in.a
AS arch/x86/realmode/rm/trampoline_32.o
AR drivers/bus/mhi/built-in.a
CC arch/x86/entry/vdso/vma.o
CC kernel/sched/core.o
AR drivers/bus/built-in.a
AR sound/drivers/opl3/built-in.a
CC arch/x86/platform/efi/memmap.o
AR sound/drivers/opl4/built-in.a
AR arch/x86/virt/built-in.a
CC block/blk-core.o
AR drivers/pwm/built-in.a
AR sound/drivers/mpu401/built-in.a
AS arch/x86/realmode/rm/stack.o
AS arch/x86/realmode/rm/reboot.o
CC arch/x86/entry/vdso/extable.o
CC crypto/asymmetric_keys/asymmetric_type.o
AR sound/drivers/vx/built-in.a
CC drivers/pci/msi/pcidev_msi.o
AR sound/drivers/pcsp/built-in.a
CC lib/math/lcm.o
AS arch/x86/realmode/rm/wakeup_asm.o
AR sound/drivers/built-in.a
AS arch/x86/lib/cmpxchg8b_emu.o
CC drivers/video/console/dummycon.o
CC arch/x86/realmode/rm/wakemain.o
CC arch/x86/lib/cpu.o
CC lib/math/int_log.o
CC arch/x86/kernel/cpu/mce/core.o
CC arch/x86/realmode/rm/video-mode.o
CC arch/x86/kernel/cpu/mtrr/mtrr.o
CC lib/math/int_pow.o
GEN usr/initramfs_data.cpio
COPY usr/initramfs_inc_data
HOSTCC certs/extract-cert
AS usr/initramfs_data.o
CC arch/x86/kernel/fpu/bugs.o
CC lib/math/int_sqrt.o
AR usr/built-in.a
CC arch/x86/kernel/cpu/mtrr/if.o
AS arch/x86/realmode/rm/copy.o
AS arch/x86/realmode/rm/bioscall.o
CC arch/x86/realmode/rm/regs.o
CC lib/math/reciprocal_div.o
CC arch/x86/kernel/fpu/core.o
CC arch/x86/realmode/rm/video-vga.o
CC sound/core/seq/seq_lock.o
CC sound/core/seq/seq_clientmgr.o
AR arch/x86/video/built-in.a
CC lib/math/rational.o
CC arch/x86/lib/delay.o
AR arch/x86/platform/geode/built-in.a
CC kernel/sched/fair.o
CC arch/x86/realmode/rm/video-vesa.o
CERT certs/x509_certificate_list
CERT certs/signing_key.x509
AS certs/system_certificates.o
CC arch/x86/events/intel/core.o
CC fs/nfs_common/grace.o
CC arch/x86/events/zhaoxin/core.o
AR certs/built-in.a
CC security/integrity/integrity_audit.o
CC kernel/sched/build_policy.o
CC arch/x86/pci/init.o
CC security/security.o
CC drivers/pci/msi/api.o
CC init/version.o
CC arch/x86/realmode/rm/video-bios.o
CC drivers/pci/msi/msi.o
CC drivers/video/console/vgacon.o
CC security/keys/key.o
CC crypto/asymmetric_keys/restrict.o
AR fs/notify/dnotify/built-in.a
PASYMS arch/x86/realmode/rm/pasyms.h
CC crypto/asymmetric_keys/signature.o
LDS arch/x86/realmode/rm/realmode.lds
CC fs/notify/inotify/inotify_fsnotify.o
CC arch/x86/power/hibernate_32.o
LD arch/x86/realmode/rm/realmode.elf
RELOCS arch/x86/realmode/rm/realmode.relocs
CC arch/x86/platform/efi/quirks.o
OBJCOPY arch/x86/realmode/rm/realmode.bin
AS arch/x86/realmode/rmpiggy.o
CC sound/core/seq/seq_memory.o
LDS arch/x86/entry/vdso/vdso32/vdso32.lds
AR arch/x86/realmode/built-in.a
AS arch/x86/lib/getuser.o
CC fs/notify/inotify/inotify_user.o
AS arch/x86/entry/vdso/vdso32/note.o
CC block/partitions/msdos.o
AS arch/x86/power/hibernate_asm_32.o
CC block/partitions/efi.o
GEN arch/x86/lib/inat-tables.c
CC arch/x86/events/core.o
AS arch/x86/entry/vdso/vdso32/system_call.o
AR lib/math/built-in.a
AS arch/x86/entry/vdso/vdso32/sigreturn.o
CC arch/x86/kernel/cpu/mce/severity.o
CC net/core/skbuff.o
CC lib/crypto/mpi/generic_mpih-lshift.o
CC lib/crypto/mpi/generic_mpih-mul1.o
CC arch/x86/entry/vdso/vdso32/vclock_gettime.o
CC arch/x86/lib/insn-eval.o
CC arch/x86/kernel/cpu/mtrr/generic.o
CC ipc/msg.o
CC arch/x86/entry/vdso/vdso32/vgetcpu.o
CC arch/x86/events/probe.o
CC arch/x86/events/amd/lbr.o
AR arch/x86/platform/iris/built-in.a
CC security/keys/keyring.o
CC ipc/sem.o
CC ipc/shm.o
CC lib/crypto/mpi/generic_mpih-mul2.o
CC crypto/asymmetric_keys/public_key.o
CC arch/x86/pci/pcbios.o
CC arch/x86/kernel/fpu/regset.o
ASN.1 crypto/asymmetric_keys/x509.asn1.[ch]
ASN.1 crypto/asymmetric_keys/x509_akid.asn1.[ch]
CC arch/x86/mm/pat/memtype.o
CC crypto/asymmetric_keys/x509_loader.o
AR security/integrity/built-in.a
CC security/lsm_audit.o
CC ipc/syscall.o
AR fs/nfs_common/built-in.a
CC arch/x86/kernel/fpu/signal.o
AR init/built-in.a
CC security/device_cgroup.o
CC arch/x86/power/hibernate.o
AR arch/x86/events/zhaoxin/built-in.a
CC block/blk-sysfs.o
CC block/blk-flush.o
CC block/blk-settings.o
CC block/blk-ioc.o
CC security/selinux/hooks.o
CC lib/crypto/mpi/generic_mpih-mul3.o
CC kernel/sched/build_utility.o
HOSTCC arch/x86/entry/vdso/vdso2c
CC drivers/pci/msi/irqdomain.o
CC sound/core/sound.o
AR drivers/video/console/built-in.a
AR sound/isa/ad1816a/built-in.a
CC arch/x86/platform/efi/efi.o
CC drivers/video/backlight/backlight.o
AR sound/isa/ad1848/built-in.a
AR sound/isa/cs423x/built-in.a
CC fs/iomap/trace.o
CC fs/iomap/iter.o
AR sound/isa/es1688/built-in.a
CC lib/crypto/mpi/generic_mpih-rshift.o
CC sound/core/seq/seq_queue.o
AR sound/isa/galaxy/built-in.a
CC arch/x86/lib/insn.o
AR sound/isa/gus/built-in.a
AR drivers/video/fbdev/core/built-in.a
AR fs/notify/inotify/built-in.a
AR block/partitions/built-in.a
AR sound/isa/msnd/built-in.a
AR drivers/video/fbdev/omap/built-in.a
AR fs/notify/fanotify/built-in.a
AR sound/isa/opti9xx/built-in.a
CC drivers/video/aperture.o
CC fs/notify/fsnotify.o
AR drivers/video/fbdev/omap2/omapfb/dss/built-in.a
AR sound/isa/sb/built-in.a
AR drivers/video/fbdev/omap2/omapfb/displays/built-in.a
CC arch/x86/kernel/cpu/mce/genpool.o
CC arch/x86/events/amd/ibs.o
AR drivers/video/fbdev/omap2/omapfb/built-in.a
AR sound/isa/wavefront/built-in.a
CC crypto/asymmetric_keys/x509_public_key.o
AR drivers/video/fbdev/omap2/built-in.a
AR sound/isa/wss/built-in.a
AR drivers/video/fbdev/built-in.a
AR sound/isa/built-in.a
CC fs/notify/notification.o
CC arch/x86/kernel/cpu/mtrr/cleanup.o
CC sound/core/init.o
CC arch/x86/entry/vdso/vdso32-setup.o
CC arch/x86/pci/mmconfig_32.o
AR arch/x86/power/built-in.a
CC arch/x86/events/intel/bts.o
CC block/blk-map.o
CC arch/x86/lib/kaslr.o
CC arch/x86/pci/direct.o
CC arch/x86/events/intel/ds.o
CC lib/crypto/memneq.o
CC arch/x86/kernel/cpu/mtrr/amd.o
CC arch/x86/mm/pat/memtype_interval.o
CC arch/x86/kernel/fpu/xstate.o
CC lib/crypto/utils.o
CC security/keys/keyctl.o
VDSO arch/x86/entry/vdso/vdso32.so.dbg
OBJCOPY arch/x86/entry/vdso/vdso32.so
VDSO2C arch/x86/entry/vdso/vdso-image-32.c
CC arch/x86/lib/memcpy_32.o
CC arch/x86/entry/vdso/vdso-image-32.o
CC lib/crypto/mpi/generic_mpih-sub1.o
ASN.1 crypto/asymmetric_keys/pkcs7.asn1.[ch]
CC lib/crypto/chacha.o
AS arch/x86/lib/memmove_32.o
CC drivers/video/cmdline.o
CC arch/x86/lib/misc.o
AR drivers/pci/msi/built-in.a
CC fs/quota/dquot.o
CC fs/quota/quota_v2.o
CC drivers/pci/pcie/portdrv.o
CC arch/x86/lib/pc-conf-reg.o
CC fs/quota/quota_tree.o
CC drivers/pci/hotplug/pci_hotplug_core.o
CC crypto/asymmetric_keys/pkcs7_trust.o
CC arch/x86/kernel/cpu/mce/intel.o
CC block/blk-merge.o
CC sound/core/seq/seq_fifo.o
AR drivers/pci/controller/dwc/built-in.a
AR drivers/video/backlight/built-in.a
AR drivers/pci/controller/mobiveil/built-in.a
CC lib/crypto/aes.o
AR drivers/pci/controller/built-in.a
CC crypto/api.o
CC drivers/pci/pcie/rcec.o
CC drivers/pci/pcie/aspm.o
AR arch/x86/entry/vdso/built-in.a
AR drivers/pci/switch/built-in.a
AS arch/x86/entry/entry.o
AR arch/x86/entry/vsyscall/built-in.a
CC fs/notify/group.o
CC crypto/asymmetric_keys/pkcs7_verify.o
CC kernel/locking/mutex.o
AS arch/x86/lib/putuser.o
AS arch/x86/entry/entry_32.o
CC io_uring/kbuf.o
CC arch/x86/platform/efi/efi_32.o
AS arch/x86/lib/retpoline.o
CC arch/x86/lib/string_32.o
CC ipc/ipc_sysctl.o
CC arch/x86/kernel/cpu/mtrr/cyrix.o
CC arch/x86/entry/syscall_32.o
CC arch/x86/pci/mmconfig-shared.o
CC arch/x86/entry/common.o
CC arch/x86/lib/strstr_32.o
CC arch/x86/events/utils.o
AS arch/x86/entry/thunk.o
AR arch/x86/mm/pat/built-in.a
CC arch/x86/lib/usercopy.o
CC arch/x86/mm/init.o
CC arch/x86/lib/usercopy_32.o
CC sound/core/seq/seq_prioq.o
CC fs/notify/mark.o
CC lib/crypto/mpi/generic_mpih-add1.o
CC arch/x86/events/intel/knc.o
CC mm/mempool.o
CC fs/iomap/buffered-io.o
CC arch/x86/events/amd/uncore.o
CC crypto/asymmetric_keys/x509.asn1.o
CC crypto/asymmetric_keys/x509_akid.asn1.o
CC sound/core/seq/seq_timer.o
CC arch/x86/lib/msr-smp.o
CC kernel/locking/semaphore.o
CC fs/quota/quota.o
CC arch/x86/lib/cache-smp.o
CC crypto/asymmetric_keys/x509_cert_parser.o
CC lib/crypto/arc4.o
CC drivers/video/nomodeset.o
CC arch/x86/kernel/cpu/mce/amd.o
CC kernel/locking/rwsem.o
CC ipc/mqueue.o
CC crypto/asymmetric_keys/pkcs7.asn1.o
CC drivers/pci/pcie/pme.o
CC crypto/asymmetric_keys/pkcs7_parser.o
CC ipc/namespace.o
CC arch/x86/kernel/cpu/mtrr/centaur.o
CC fs/quota/kqid.o
CC block/blk-timeout.o
CC security/selinux/selinuxfs.o
CC lib/crypto/gf128mul.o
CC drivers/pci/hotplug/acpi_pcihp.o
AS arch/x86/platform/efi/efi_stub_32.o
CC arch/x86/lib/msr.o
AS arch/x86/lib/msr-reg.o
CC security/keys/permission.o
CC arch/x86/platform/efi/runtime-map.o
AR arch/x86/kernel/fpu/built-in.a
CC lib/zlib_inflate/inffast.o
CC arch/x86/kernel/cpu/microcode/core.o
CC lib/crypto/mpi/ec.o
CC arch/x86/kernel/cpu/microcode/intel.o
CC drivers/video/hdmi.o
CC lib/zlib_inflate/inflate.o
CC sound/core/seq/seq_system.o
CC arch/x86/kernel/acpi/boot.o
CC lib/zlib_inflate/infutil.o
CC arch/x86/mm/init_32.o
CC arch/x86/events/intel/lbr.o
CC arch/x86/pci/fixup.o
CC mm/oom_kill.o
CC arch/x86/mm/fault.o
CC arch/x86/pci/acpi.o
CC fs/notify/fdinfo.o
AR crypto/asymmetric_keys/built-in.a
CC crypto/cipher.o
CC sound/core/seq/seq_ports.o
CC arch/x86/kernel/cpu/cacheinfo.o
CC arch/x86/kernel/cpu/mtrr/legacy.o
CC mm/fadvise.o
CC crypto/compress.o
CC arch/x86/kernel/cpu/scattered.o
CC lib/crypto/blake2s.o
CC security/selinux/netlink.o
AR arch/x86/entry/built-in.a
CC lib/zlib_deflate/deflate.o
AR drivers/pci/pcie/built-in.a
CC lib/zlib_deflate/deftree.o
CC io_uring/rsrc.o
CC lib/zlib_deflate/deflate_syms.o
CC net/core/datagram.o
CC kernel/locking/percpu-rwsem.o
AR drivers/pci/hotplug/built-in.a
CC drivers/pci/access.o
CC lib/crypto/mpi/mpicoder.o
CC crypto/algapi.o
CC block/blk-lib.o
CC security/keys/process_keys.o
CC arch/x86/kernel/apic/apic.o
AR arch/x86/platform/efi/built-in.a
AR arch/x86/events/amd/built-in.a
CC lib/zlib_inflate/inftrees.o
AR arch/x86/kernel/cpu/mtrr/built-in.a
CC arch/x86/platform/intel/iosf_mbi.o
CC lib/zlib_inflate/inflate_syms.o
CC arch/x86/events/rapl.o
CC arch/x86/kernel/apic/apic_common.o
CC lib/crypto/blake2s-generic.o
CC arch/x86/lib/msr-reg-export.o
CC arch/x86/kernel/cpu/microcode/amd.o
CC fs/quota/netlink.o
AS arch/x86/lib/hweight.o
AR drivers/video/built-in.a
CC arch/x86/lib/iomem.o
AR drivers/idle/built-in.a
CC arch/x86/kernel/cpu/topology_common.o
AR drivers/char/ipmi/built-in.a
CC lib/crypto/sha1.o
AR fs/notify/built-in.a
CC drivers/acpi/acpica/dsargs.o
AR drivers/acpi/pmic/built-in.a
CC drivers/pnp/pnpacpi/core.o
AR drivers/amba/built-in.a
CC ipc/mq_sysctl.o
AR drivers/clk/actions/built-in.a
AR drivers/clk/analogbits/built-in.a
CC drivers/dma/dw/core.o
CC drivers/acpi/acpica/dscontrol.o
CC arch/x86/events/msr.o
AR drivers/clk/bcm/built-in.a
CC drivers/acpi/acpica/dsdebug.o
AR lib/zlib_inflate/built-in.a
AR drivers/clk/imgtec/built-in.a
CC arch/x86/pci/legacy.o
CC arch/x86/pci/irq.o
AR drivers/clk/imx/built-in.a
AR drivers/clk/ingenic/built-in.a
CC arch/x86/pci/common.o
CC sound/core/seq/seq_info.o
CC kernel/locking/spinlock.o
CC arch/x86/mm/ioremap.o
AR drivers/clk/mediatek/built-in.a
AR drivers/clk/microchip/built-in.a
CC arch/x86/pci/early.o
AR drivers/clk/mstar/built-in.a
CC arch/x86/mm/extable.o
AR drivers/clk/mvebu/built-in.a
CC arch/x86/pci/bus_numa.o
CC arch/x86/kernel/acpi/sleep.o
AR drivers/clk/ralink/built-in.a
CC arch/x86/pci/amd_bus.o
AR drivers/clk/renesas/built-in.a
CC lib/crypto/sha256.o
CC arch/x86/lib/atomic64_32.o
AR drivers/clk/socfpga/built-in.a
CC arch/x86/kernel/cpu/mce/threshold.o
AR lib/zlib_deflate/built-in.a
CC drivers/acpi/acpica/dsfield.o
AR drivers/clk/sophgo/built-in.a
CC lib/crypto/mpi/mpi-add.o
AR drivers/clk/sprd/built-in.a
AR drivers/clk/starfive/built-in.a
CC drivers/dma/dw/dw.o
CC arch/x86/events/intel/p4.o
CC arch/x86/lib/inat.o
AR drivers/clk/sunxi-ng/built-in.a
AR drivers/clk/ti/built-in.a
AR ipc/built-in.a
AS arch/x86/kernel/acpi/wakeup_32.o
CC arch/x86/kernel/acpi/cstate.o
AR drivers/clk/versatile/built-in.a
CC arch/x86/events/intel/p6.o
AR drivers/clk/xilinx/built-in.a
CC arch/x86/events/intel/pt.o
AR drivers/clk/built-in.a
CC lib/crypto/mpi/mpi-bit.o
AR arch/x86/platform/intel/built-in.a
CC kernel/power/qos.o
AR arch/x86/platform/intel-mid/built-in.a
CC kernel/power/main.o
AR arch/x86/platform/intel-quark/built-in.a
CC drivers/pci/bus.o
AR arch/x86/platform/olpc/built-in.a
AR arch/x86/lib/built-in.a
AR arch/x86/platform/scx200/built-in.a
AR arch/x86/lib/lib.a
CC block/blk-mq.o
AR arch/x86/platform/ts5500/built-in.a
CC kernel/power/console.o
CC arch/x86/events/intel/uncore.o
AR arch/x86/platform/uv/built-in.a
CC drivers/pnp/pnpacpi/rsparser.o
AR arch/x86/platform/built-in.a
CC kernel/locking/osq_lock.o
CC arch/x86/events/intel/uncore_nhmex.o
CC arch/x86/mm/mmap.o
CC fs/iomap/direct-io.o
CC security/selinux/nlmsgtab.o
CC arch/x86/mm/pgtable.o
AR arch/x86/kernel/cpu/microcode/built-in.a
CC fs/iomap/fiemap.o
CC arch/x86/kernel/cpu/topology_ext.o
AR fs/quota/built-in.a
CC arch/x86/events/intel/uncore_snb.o
CC sound/core/seq/seq_dummy.o
CC fs/proc/task_mmu.o
CC arch/x86/events/intel/uncore_snbep.o
CC security/keys/request_key.o
CC kernel/locking/qspinlock.o
CC fs/proc/inode.o
CC drivers/acpi/acpica/dsinit.o
CC arch/x86/kernel/kprobes/core.o
LDS arch/x86/kernel/vmlinux.lds
CC crypto/scatterwalk.o
CC crypto/proc.o
CC arch/x86/kernel/kprobes/opt.o
CC mm/maccess.o
AR kernel/sched/built-in.a
CC drivers/pnp/core.o
CC lib/lzo/lzo1x_compress.o
CC drivers/dma/hsu/hsu.o
CC io_uring/notif.o
CC drivers/pnp/card.o
AR drivers/dma/idxd/built-in.a
CC block/blk-mq-tag.o
CC lib/lzo/lzo1x_decompress_safe.o
CC mm/page-writeback.o
AR arch/x86/kernel/acpi/built-in.a
CC block/blk-stat.o
CC net/core/stream.o
CC kernel/printk/printk.o
CC drivers/pnp/driver.o
CC lib/crypto/mpi/mpi-cmp.o
CC drivers/dma/dw/idma32.o
CC kernel/printk/conopt.o
CC net/core/scm.o
CC drivers/acpi/acpica/dsmethod.o
CC arch/x86/mm/physaddr.o
CC mm/folio-compat.o
CC kernel/printk/printk_safe.o
CC kernel/locking/rtmutex_api.o
AR sound/core/seq/built-in.a
AR arch/x86/pci/built-in.a
CC sound/core/memory.o
CC kernel/irq/irqdesc.o
CC drivers/pci/probe.o
CC block/blk-mq-sysfs.o
CC drivers/pci/host-bridge.o
CC drivers/dma/dw/acpi.o
CC arch/x86/kernel/apic/apic_noop.o
CC kernel/printk/nbcon.o
AR arch/x86/kernel/cpu/mce/built-in.a
CC drivers/acpi/dptf/int340x_thermal.o
CC arch/x86/kernel/cpu/topology_amd.o
CC kernel/printk/printk_ringbuffer.o
CC crypto/aead.o
AR lib/lzo/built-in.a
AR drivers/pnp/pnpacpi/built-in.a
CC kernel/locking/qrwlock.o
CC mm/readahead.o
CC drivers/pnp/resource.o
CC kernel/rcu/update.o
CC kernel/rcu/sync.o
CC security/selinux/netif.o
CC kernel/power/process.o
CC security/keys/request_key_auth.o
CC kernel/rcu/srcutree.o
CC drivers/pnp/manager.o
CC drivers/pnp/support.o
CC drivers/acpi/acpica/dsmthdat.o
CC fs/proc/root.o
CC fs/proc/base.o
CC mm/swap.o
CC fs/iomap/seek.o
CC lib/lz4/lz4_decompress.o
CC lib/crypto/mpi/mpi-sub-ui.o
CC arch/x86/mm/tlb.o
AR arch/x86/kernel/kprobes/built-in.a
AR drivers/dma/hsu/built-in.a
CC lib/zstd/zstd_decompress_module.o
CC fs/kernfs/mount.o
CC lib/zstd/decompress/huf_decompress.o
CC fs/kernfs/inode.o
CC kernel/power/suspend.o
CC arch/x86/kernel/apic/ipi.o
CC kernel/printk/sysctl.o
CC mm/truncate.o
CC lib/zstd/decompress/zstd_ddict.o
CC arch/x86/kernel/cpu/common.o
CC sound/core/control.o
AR drivers/acpi/dptf/built-in.a
CC kernel/rcu/tree.o
CC lib/zstd/decompress/zstd_decompress.o
CC kernel/irq/handle.o
CC kernel/irq/manage.o
CC arch/x86/mm/cpu_entry_area.o
CC arch/x86/kernel/cpu/rdrand.o
CC lib/crypto/mpi/mpi-div.o
AR drivers/dma/dw/built-in.a
CC kernel/irq/spurious.o
CC drivers/acpi/acpica/dsobject.o
AR drivers/dma/mediatek/built-in.a
CC drivers/pnp/interface.o
CC drivers/acpi/acpica/dsopcode.o
AR drivers/dma/qcom/built-in.a
CC kernel/rcu/rcu_segcblist.o
AR drivers/dma/ti/built-in.a
CC io_uring/tctx.o
CC io_uring/filetable.o
AR drivers/dma/xilinx/built-in.a
CC drivers/dma/dmaengine.o
CC block/blk-mq-cpumap.o
CC arch/x86/events/intel/uncore_discovery.o
AR kernel/locking/built-in.a
CC arch/x86/events/intel/cstate.o
CC drivers/dma/virt-dma.o
CC security/keys/user_defined.o
CC fs/proc/generic.o
CC lib/crypto/mpi/mpi-inv.o
CC net/core/gen_stats.o
CC crypto/geniv.o
CC arch/x86/kernel/apic/vector.o
CC fs/iomap/swapfile.o
CC fs/proc/array.o
CC kernel/irq/resend.o
CC drivers/acpi/acpica/dspkginit.o
CC kernel/power/hibernate.o
CC block/blk-mq-sched.o
CC mm/vmscan.o
CC kernel/irq/chip.o
CC drivers/pnp/quirks.o
CC fs/proc/fd.o
CC fs/kernfs/dir.o
CC net/core/gen_estimator.o
CC drivers/dma/acpi-dma.o
CC kernel/power/snapshot.o
CC arch/x86/kernel/apic/init.o
CC arch/x86/mm/maccess.o
CC security/selinux/netnode.o
CC drivers/pnp/system.o
CC security/keys/proc.o
CC security/selinux/netport.o
CC fs/kernfs/file.o
CC security/keys/sysctl.o
CC drivers/pci/remove.o
AR lib/lz4/built-in.a
CC mm/shrinker.o
CC mm/shmem.o
CC security/selinux/status.o
CC block/ioctl.o
CC lib/crypto/mpi/mpi-mod.o
CC lib/zstd/decompress/zstd_decompress_block.o
CC drivers/acpi/acpica/dsutils.o
AR kernel/printk/built-in.a
CC crypto/lskcipher.o
CC crypto/skcipher.o
CC arch/x86/mm/pgprot.o
CC lib/crypto/mpi/mpi-mul.o
CC lib/xz/xz_dec_syms.o
CC io_uring/rw.o
CC lib/dim/dim.o
CC io_uring/net.o
CC lib/fonts/fonts.o
CC lib/zstd/zstd_common_module.o
AR arch/x86/events/intel/built-in.a
AR fs/iomap/built-in.a
AR arch/x86/events/built-in.a
CC arch/x86/kernel/apic/hw_nmi.o
CC lib/xz/xz_dec_stream.o
CC crypto/seqiv.o
CC lib/dim/net_dim.o
CC security/keys/keyctl_pkey.o
AR drivers/pnp/built-in.a
CC net/ethernet/eth.o
CC sound/core/misc.o
CC lib/crypto/mpi/mpih-cmp.o
CC kernel/irq/dummychip.o
AR drivers/soc/apple/built-in.a
AR drivers/dma/built-in.a
AR drivers/soc/aspeed/built-in.a
CC drivers/acpi/acpica/dswexec.o
AR drivers/soc/bcm/built-in.a
CC drivers/virtio/virtio.o
AR drivers/soc/fsl/built-in.a
CC lib/xz/xz_dec_lzma2.o
AR drivers/soc/fujitsu/built-in.a
CC drivers/tty/vt/vt_ioctl.o
CC drivers/tty/vt/vc_screen.o
AR drivers/soc/hisilicon/built-in.a
CC arch/x86/kernel/cpu/match.o
CC drivers/char/hw_random/core.o
AR drivers/soc/imx/built-in.a
AR drivers/soc/ixp4xx/built-in.a
CC drivers/pci/pci.o
CC drivers/pci/pci-driver.o
AR drivers/soc/loongson/built-in.a
AS arch/x86/kernel/head_32.o
CC fs/proc/proc_tty.o
CC lib/fonts/font_8x16.o
CC drivers/acpi/x86/apple.o
AR drivers/soc/mediatek/built-in.a
CC drivers/acpi/x86/cmos_rtc.o
AR drivers/soc/microchip/built-in.a
CC arch/x86/mm/pgtable_32.o
CC arch/x86/mm/iomap_32.o
AR drivers/soc/nuvoton/built-in.a
AR drivers/soc/pxa/built-in.a
AR drivers/soc/amlogic/built-in.a
CC drivers/virtio/virtio_ring.o
AR drivers/soc/qcom/built-in.a
CC net/core/net_namespace.o
CC lib/dim/rdma_dim.o
AR drivers/soc/renesas/built-in.a
CC net/core/secure_seq.o
AR drivers/soc/rockchip/built-in.a
AR drivers/soc/sunxi/built-in.a
CC net/core/flow_dissector.o
AR drivers/soc/ti/built-in.a
AR drivers/soc/xilinx/built-in.a
CC lib/xz/xz_dec_bcj.o
AR drivers/soc/built-in.a
CC mm/util.o
CC arch/x86/kernel/apic/io_apic.o
CC drivers/acpi/tables.o
CC kernel/irq/devres.o
AR drivers/iommu/amd/built-in.a
AR drivers/gpu/host1x/built-in.a
AR drivers/iommu/intel/built-in.a
CC kernel/irq/autoprobe.o
CC drivers/acpi/acpica/dswload.o
CC fs/kernfs/symlink.o
AR drivers/iommu/arm/arm-smmu/built-in.a
CC arch/x86/kernel/apic/msi.o
CC arch/x86/kernel/cpu/bugs.o
AR drivers/iommu/arm/arm-smmu-v3/built-in.a
CC drivers/iommu/iommu.o
AR drivers/iommu/iommufd/built-in.a
CC block/genhd.o
AR drivers/iommu/arm/built-in.a
CC drivers/char/hw_random/intel-rng.o
AR security/keys/built-in.a
CC drivers/acpi/x86/lpss.o
AR lib/fonts/built-in.a
CC crypto/echainiv.o
CC arch/x86/kernel/apic/probe_32.o
CC lib/crypto/mpi/mpih-div.o
CC drivers/char/hw_random/amd-rng.o
CC drivers/char/hw_random/geode-rng.o
CC security/selinux/ss/ebitmap.o
AR lib/dim/built-in.a
CC sound/core/device.o
CC kernel/irq/irqdomain.o
AR drivers/gpu/drm/tests/built-in.a
AR sound/pci/ac97/built-in.a
AR sound/pci/ali5451/built-in.a
AR drivers/gpu/drm/arm/built-in.a
AR sound/pci/asihpi/built-in.a
CC drivers/gpu/drm/display/drm_display_helper_mod.o
AR sound/pci/au88x0/built-in.a
CC drivers/gpu/drm/display/drm_dp_dual_mode_helper.o
AR sound/pci/aw2/built-in.a
CC drivers/gpu/drm/display/drm_dp_helper.o
CC drivers/acpi/x86/s2idle.o
AR sound/pci/ctxfi/built-in.a
CC crypto/ahash.o
AR sound/pci/ca0106/built-in.a
CC fs/proc/cmdline.o
CC crypto/shash.o
AR sound/pci/cs46xx/built-in.a
AR lib/xz/built-in.a
AR sound/pci/cs5535audio/built-in.a
CC lib/argv_split.o
AR sound/pci/lola/built-in.a
CC arch/x86/mm/hugetlbpage.o
CC drivers/acpi/acpica/dswload2.o
AR sound/pci/lx6464es/built-in.a
CC drivers/acpi/acpica/dswscope.o
AR sound/pci/echoaudio/built-in.a
AR sound/pci/emu10k1/built-in.a
CC drivers/pci/search.o
CC drivers/tty/hvc/hvc_console.o
AR sound/pci/hda/built-in.a
CC kernel/power/swap.o
CC [M] sound/pci/hda/hda_bind.o
CC lib/bug.o
CC kernel/irq/proc.o
CC drivers/tty/vt/selection.o
CC drivers/char/hw_random/via-rng.o
CC sound/core/info.o
AR sound/pci/ice1712/built-in.a
CC io_uring/poll.o
CC lib/zstd/common/debug.o
AR net/802/built-in.a
AR fs/kernfs/built-in.a
CC lib/zstd/common/entropy_common.o
CC io_uring/uring_cmd.o
CC lib/zstd/common/error_private.o
CC fs/sysfs/file.o
CC fs/sysfs/dir.o
CC drivers/acpi/x86/utils.o
CC fs/devpts/inode.o
CC crypto/akcipher.o
CC drivers/acpi/x86/blacklist.o
CC fs/sysfs/symlink.o
CC fs/netfs/buffered_read.o
AR net/ethernet/built-in.a
CC drivers/acpi/acpica/dswstate.o
CC fs/sysfs/mount.o
CC fs/proc/consoles.o
CC lib/crypto/mpi/mpih-mul.o
CC fs/sysfs/group.o
CC kernel/power/user.o
CC drivers/iommu/iommu-traces.o
CC drivers/tty/vt/keyboard.o
AR drivers/char/hw_random/built-in.a
CC drivers/char/agp/backend.o
CC lib/zstd/common/fse_decompress.o
CC arch/x86/mm/dump_pagetables.o
CC drivers/char/agp/generic.o
CC drivers/iommu/iommu-sysfs.o
CC kernel/power/poweroff.o
CC lib/buildid.o
AR drivers/gpu/vga/built-in.a
CC drivers/virtio/virtio_anchor.o
CC lib/clz_tab.o
CC fs/netfs/buffered_write.o
CC [M] sound/pci/hda/hda_codec.o
CC drivers/iommu/dma-iommu.o
CC io_uring/openclose.o
CC kernel/irq/migration.o
CC crypto/sig.o
CC drivers/acpi/acpica/evevent.o
AR drivers/acpi/x86/built-in.a
CC block/ioprio.o
CC [M] sound/pci/hda/hda_jack.o
CC fs/proc/cpuinfo.o
CC drivers/char/agp/isoch.o
CC lib/zstd/common/zstd_common.o
CC drivers/tty/vt/vt.o
CC security/selinux/ss/hashtab.o
CC io_uring/sqpoll.o
AR drivers/tty/hvc/built-in.a
CC drivers/iommu/iova.o
CC drivers/acpi/osi.o
CC crypto/kpp.o
AR sound/ppc/built-in.a
CC net/core/sysctl_net_core.o
CC sound/core/isadma.o
CC mm/mmzone.o
AR sound/arm/built-in.a
AR sound/sh/built-in.a
AR fs/devpts/built-in.a
CC lib/crypto/mpi/mpi-pow.o
CC drivers/gpu/drm/display/drm_dp_mst_topology.o
CC arch/x86/kernel/cpu/aperfmperf.o
AR sound/synth/emux/built-in.a
AR arch/x86/kernel/apic/built-in.a
AR sound/synth/built-in.a
AR lib/zstd/built-in.a
AR fs/sysfs/built-in.a
CC fs/netfs/direct_read.o
CC block/badblocks.o
AR sound/usb/misc/built-in.a
CC drivers/char/agp/amd64-agp.o
CC block/blk-rq-qos.o
CC kernel/irq/cpuhotplug.o
AR sound/usb/usx2y/built-in.a
CC net/core/dev.o
AR sound/usb/caiaq/built-in.a
CC drivers/acpi/acpica/evgpe.o
CC lib/cmdline.o
AR sound/usb/6fire/built-in.a
AR sound/usb/hiface/built-in.a
CC arch/x86/kernel/head32.o
AR sound/usb/bcd2000/built-in.a
AR sound/usb/built-in.a
CC security/selinux/ss/symtab.o
CC kernel/irq/pm.o
CC block/disk-events.o
CC drivers/virtio/virtio_pci_modern_dev.o
CC drivers/virtio/virtio_pci_legacy_dev.o
CC drivers/char/mem.o
CC [M] sound/pci/hda/hda_auto_parser.o
CC drivers/pci/rom.o
AR kernel/power/built-in.a
CC arch/x86/mm/highmem_32.o
AR kernel/livepatch/built-in.a
CC drivers/virtio/virtio_pci_modern.o
CC fs/proc/devices.o
CC kernel/dma/mapping.o
CC kernel/entry/common.o
CC security/selinux/ss/sidtab.o
CC sound/core/vmaster.o
CC kernel/entry/syscall_user_dispatch.o
AR kernel/rcu/built-in.a
COPY drivers/tty/vt/defkeymap.c
CC lib/cpumask.o
CC kernel/module/main.o
CC lib/ctype.o
CC [M] sound/pci/hda/hda_sysfs.o
CC lib/dec_and_lock.o
CC mm/vmstat.o
CC lib/decompress.o
CC drivers/acpi/acpica/evgpeblk.o
CC drivers/char/agp/intel-agp.o
CC kernel/module/strict_rwx.o
CC fs/proc/interrupts.o
CC drivers/char/agp/intel-gtt.o
CC fs/proc/loadavg.o
ASN.1 crypto/rsapubkey.asn1.[ch]
ASN.1 crypto/rsaprivkey.asn1.[ch]
CC io_uring/xattr.o
CC crypto/rsa.o
CC drivers/pci/setup-res.o
CC lib/crypto/mpi/mpiutil.o
CC arch/x86/kernel/cpu/cpuid-deps.o
CC drivers/pci/irq.o
CC lib/decompress_bunzip2.o
CC lib/decompress_inflate.o
CC mm/backing-dev.o
CC drivers/acpi/acpica/evgpeinit.o
CC io_uring/nop.o
CC crypto/rsa_helper.o
CC net/core/dev_addr_lists.o
CC drivers/virtio/virtio_pci_common.o
CC drivers/tty/serial/8250/8250_core.o
CC drivers/connector/cn_queue.o
AR arch/x86/mm/built-in.a
CC kernel/irq/msi.o
CC drivers/connector/connector.o
CC kernel/irq/affinity.o
AR sound/firewire/built-in.a
CC drivers/char/random.o
CC drivers/char/misc.o
CC fs/netfs/direct_write.o
CC drivers/tty/serial/serial_core.o
CC drivers/connector/cn_proc.o
CC sound/core/ctljack.o
CC crypto/rsa-pkcs1pad.o
CC drivers/virtio/virtio_pci_legacy.o
AR drivers/iommu/built-in.a
CC block/blk-ia-ranges.o
CC drivers/tty/serial/serial_base_bus.o
CC arch/x86/kernel/cpu/umwait.o
CC kernel/module/kmod.o
CC drivers/char/virtio_console.o
CC drivers/virtio/virtio_pci_admin_legacy_io.o
CC kernel/module/tree_lookup.o
CC fs/netfs/io.o
CC fs/proc/meminfo.o
CC [M] sound/pci/hda/hda_controller.o
CC drivers/acpi/acpica/evgpeutil.o
AR sound/sparc/built-in.a
CC kernel/dma/direct.o
AR sound/spi/built-in.a
AR sound/parisc/built-in.a
CC kernel/module/kallsyms.o
CC drivers/acpi/acpica/evglock.o
CC drivers/virtio/virtio_input.o
CC kernel/dma/ops_helpers.o
CC drivers/char/hpet.o
AR lib/crypto/mpi/built-in.a
AR lib/crypto/built-in.a
CC io_uring/fs.o
CC drivers/tty/vt/consolemap.o
CC sound/core/jack.o
CC kernel/irq/matrix.o
CC drivers/pci/vpd.o
AR kernel/entry/built-in.a
CC lib/decompress_unlz4.o
CC kernel/time/time.o
CC fs/netfs/iterator.o
CC io_uring/splice.o
CC security/selinux/ss/avtab.o
AR sound/pcmcia/vx/built-in.a
AR sound/pcmcia/pdaudiocf/built-in.a
AR sound/pcmcia/built-in.a
CC drivers/acpi/acpica/evhandler.o
CC drivers/acpi/acpica/evmisc.o
CC fs/ext4/balloc.o
AR drivers/char/agp/built-in.a
CC drivers/pci/setup-bus.o
CC fs/ext4/bitmap.o
CC fs/jbd2/transaction.o
CC drivers/char/nvram.o
CC fs/ext4/block_validity.o
MKCAP arch/x86/kernel/cpu/capflags.c
CC [M] sound/pci/hda/hda_proc.o
CC drivers/virtio/virtio_dma_buf.o
CC crypto/acompress.o
CC fs/ext4/dir.o
CC block/early-lookup.o
CC fs/proc/stat.o
CC fs/proc/uptime.o
CC arch/x86/kernel/ebda.o
CC mm/mm_init.o
CC [M] sound/pci/hda/hda_hwdep.o
CC drivers/tty/serial/8250/8250_pnp.o
CC lib/decompress_unlzma.o
CC kernel/dma/dummy.o
CC fs/netfs/locking.o
CC crypto/scompress.o
CC kernel/time/timer.o
CC kernel/time/hrtimer.o
AR sound/mips/built-in.a
CC kernel/dma/remap.o
AR drivers/connector/built-in.a
CC drivers/acpi/acpica/evregion.o
AR sound/soc/built-in.a
CC kernel/time/timekeeping.o
CC kernel/time/ntp.o
CC drivers/tty/serial/8250/8250_port.o
CC sound/core/timer.o
CC fs/ext4/ext4_jbd2.o
CC drivers/pci/vc.o
CC drivers/pci/mmap.o
CC fs/ext4/extents.o
HOSTCC drivers/tty/vt/conmakehash
CC fs/proc/util.o
CC fs/ext4/extents_status.o
CC drivers/acpi/acpica/evrgnini.o
AR drivers/virtio/built-in.a
CC kernel/time/clocksource.o
CC sound/core/hrtimer.o
CC lib/decompress_unlzo.o
CC net/core/dst.o
CC kernel/module/procfs.o
CC io_uring/sync.o
AR drivers/char/built-in.a
CC kernel/module/sysfs.o
CC fs/ext4/file.o
CC crypto/algboss.o
CC drivers/tty/vt/defkeymap.o
CC fs/proc/version.o
CC crypto/testmgr.o
CC [M] sound/pci/hda/patch_hdmi.o
CC block/bounce.o
CC drivers/base/power/sysfs.o
CC [M] sound/pci/hda/hda_eld.o
CC fs/netfs/main.o
CC drivers/base/power/generic_ops.o
AR kernel/irq/built-in.a
CC fs/ext4/fsmap.o
CC fs/ext4/fsync.o
AR kernel/dma/built-in.a
CC drivers/base/power/common.o
CC drivers/base/power/qos.o
CONMK drivers/tty/vt/consolemap_deftbl.c
CC drivers/tty/vt/consolemap_deftbl.o
CC security/selinux/ss/policydb.o
AR drivers/tty/vt/built-in.a
CC security/selinux/ss/services.o
CC drivers/gpu/drm/display/drm_dsc_helper.o
CC sound/core/seq_device.o
CC drivers/acpi/acpica/evsci.o
CC drivers/pci/devres.o
CC arch/x86/kernel/platform-quirks.o
CC io_uring/msg_ring.o
CC fs/ext4/hash.o
AR sound/atmel/built-in.a
CC crypto/cmac.o
CC crypto/hmac.o
CC fs/ext4/ialloc.o
CC security/selinux/ss/conditional.o
CC security/selinux/ss/mls.o
CC lib/decompress_unxz.o
CC fs/proc/softirqs.o
CC fs/jbd2/commit.o
CC net/core/netevent.o
CC mm/percpu.o
CC fs/jbd2/recovery.o
CC drivers/pci/proc.o
AR sound/hda/built-in.a
CC [M] sound/hda/hda_bus_type.o
CC drivers/acpi/acpica/evxface.o
AR kernel/module/built-in.a
AR sound/x86/built-in.a
CC drivers/base/power/runtime.o
CC drivers/base/power/wakeirq.o
CC drivers/pci/pci-sysfs.o
AR drivers/tty/ipwireless/built-in.a
CC kernel/time/jiffies.o
CC kernel/time/timer_list.o
CC block/bsg.o
CC arch/x86/kernel/process_32.o
CC fs/proc/namespaces.o
CC kernel/time/timeconv.o
CC io_uring/advise.o
CC drivers/gpu/drm/display/drm_hdcp_helper.o
CC [M] sound/core/hwdep.o
CC lib/decompress_unzstd.o
CC fs/netfs/misc.o
CC fs/netfs/objects.o
CC arch/x86/kernel/cpu/powerflags.o
CC kernel/time/timecounter.o
CC crypto/crypto_null.o
CC lib/dump_stack.o
CC drivers/gpu/drm/display/drm_hdmi_helper.o
CC kernel/time/alarmtimer.o
CC kernel/time/posix-timers.o
CC drivers/tty/serial/8250/8250_dma.o
CC crypto/md5.o
CC drivers/acpi/acpica/evxfevnt.o
CC crypto/sha256_generic.o
CC fs/netfs/write_collect.o
CC arch/x86/kernel/cpu/topology.o
CC drivers/acpi/osl.o
CC drivers/tty/serial/8250/8250_dwlib.o
CC [M] sound/hda/hdac_bus.o
CC fs/proc/self.o
CC block/blk-cgroup.o
CC drivers/base/power/main.o
CC drivers/tty/tty_io.o
CC net/core/neighbour.o
CC drivers/tty/serial/8250/8250_pcilib.o
CC arch/x86/kernel/cpu/proc.o
CC drivers/acpi/utils.o
CC fs/jbd2/checkpoint.o
CC net/core/rtnetlink.o
CC drivers/acpi/acpica/evxfgpe.o
CC drivers/gpu/drm/display/drm_scdc_helper.o
CC fs/ramfs/inode.o
CC [M] sound/core/pcm.o
CC [M] sound/core/pcm_native.o
CC [M] sound/core/pcm_lib.o
CC kernel/futex/core.o
CC security/selinux/ss/context.o
CC arch/x86/kernel/cpu/feat_ctl.o
CC arch/x86/kernel/cpu/intel.o
CC io_uring/epoll.o
CC kernel/futex/syscalls.o
CC io_uring/statx.o
CC lib/earlycpio.o
CC crypto/sha512_generic.o
CC kernel/futex/pi.o
CC lib/extable.o
CC lib/flex_proportions.o
CC drivers/pci/slot.o
CC [M] sound/pci/hda/hda_intel.o
LD [M] sound/pci/hda/snd-hda-codec.o
CC fs/proc/thread_self.o
LD [M] sound/pci/hda/snd-hda-codec-hdmi.o
CC fs/proc/proc_sysctl.o
CC crypto/sha3_generic.o
CC arch/x86/kernel/cpu/intel_pconfig.o
CC drivers/acpi/acpica/evxfregn.o
CC drivers/acpi/acpica/exconcat.o
CC arch/x86/kernel/cpu/tsx.o
CC [M] sound/hda/hdac_device.o
CC crypto/ecb.o
CC drivers/tty/serial/8250/8250_early.o
CC arch/x86/kernel/cpu/intel_epb.o
CC drivers/acpi/acpica/exconfig.o
CC drivers/acpi/acpica/exconvrt.o
CC [M] sound/hda/hdac_sysfs.o
CC net/core/utils.o
CC security/selinux/netlabel.o
CC net/core/link_watch.o
CC kernel/time/posix-cpu-timers.o
CC lib/idr.o
CC drivers/acpi/acpica/excreate.o
CC fs/ramfs/file-mmu.o
AR drivers/gpu/drm/display/built-in.a
CC drivers/gpu/drm/ttm/ttm_tt.o
CC drivers/gpu/drm/ttm/ttm_bo.o
CC drivers/acpi/acpica/exdebug.o
CC [M] sound/hda/hdac_regmap.o
CC drivers/gpu/drm/i915/i915_config.o
CC [M] sound/hda/hdac_controller.o
CC drivers/tty/serial/8250/8250_exar.o
CC io_uring/timeout.o
CC io_uring/fdinfo.o
CC io_uring/cancel.o
CC fs/netfs/write_issue.o
CC [M] sound/hda/hdac_stream.o
CC kernel/time/posix-clock.o
CC drivers/gpu/drm/i915/i915_driver.o
CC fs/hugetlbfs/inode.o
CC drivers/gpu/drm/i915/i915_drm_client.o
CC kernel/time/itimer.o
CC crypto/cbc.o
CC fs/jbd2/revoke.o
CC kernel/time/clockevents.o
CC fs/jbd2/journal.o
CC kernel/futex/requeue.o
CC drivers/pci/pci-acpi.o
CC mm/slab_common.o
CC io_uring/waitid.o
CC fs/fat/cache.o
CC [M] sound/core/pcm_misc.o
CC lib/irq_regs.o
CC drivers/acpi/acpica/exdump.o
CC io_uring/register.o
CC block/blk-ioprio.o
CC drivers/acpi/acpica/exfield.o
CC drivers/base/power/wakeup.o
CC [M] sound/hda/array.o
CC lib/is_single_threaded.o
CC lib/klist.o
CC [M] sound/hda/hdmi_chmap.o
AR fs/ramfs/built-in.a
CC crypto/ctr.o
CC drivers/tty/n_tty.o
CC drivers/acpi/acpica/exfldio.o
CC net/core/filter.o
CC [M] sound/hda/trace.o
CC kernel/time/tick-common.o
CC crypto/gcm.o
CC net/core/sock_diag.o
CC io_uring/truncate.o
CC fs/proc/proc_net.o
CC kernel/futex/waitwake.o
CC net/core/dev_ioctl.o
LD [M] sound/pci/hda/snd-hda-intel.o
CC io_uring/memmap.o
CC io_uring/io-wq.o
AR sound/pci/korg1212/built-in.a
AR sound/pci/mixart/built-in.a
CC io_uring/futex.o
CC drivers/tty/serial/8250/8250_lpss.o
AR sound/pci/nm256/built-in.a
CC lib/kobject.o
AR sound/pci/oxygen/built-in.a
CC drivers/gpu/drm/ttm/ttm_bo_util.o
AR sound/pci/pcxhr/built-in.a
AR sound/pci/riptide/built-in.a
AR sound/pci/rme9652/built-in.a
CC drivers/acpi/acpica/exmisc.o
AR sound/pci/trident/built-in.a
CC kernel/time/tick-broadcast.o
AR sound/pci/ymfpci/built-in.a
CC fs/fat/dir.o
CC kernel/time/tick-broadcast-hrtimer.o
AR security/selinux/built-in.a
AR sound/pci/vx222/built-in.a
CC [M] sound/hda/hdac_component.o
AR sound/xen/built-in.a
AR sound/pci/built-in.a
AR security/built-in.a
CC [M] sound/hda/hdac_i915.o
CC io_uring/napi.o
CC fs/fat/fatent.o
CC fs/ext4/indirect.o
CC [M] sound/hda/intel-dsp-config.o
CC fs/proc/kcore.o
CC mm/compaction.o
CC block/blk-iolatency.o
CC net/core/tso.o
CC drivers/pci/iomap.o
CC fs/fat/file.o
AR fs/netfs/built-in.a
AR sound/virtio/built-in.a
CC arch/x86/kernel/cpu/amd.o
CC block/blk-iocost.o
CC drivers/pci/quirks.o
AR fs/hugetlbfs/built-in.a
CC net/sched/sch_generic.o
CC net/netlink/af_netlink.o
AR net/bpf/built-in.a
CC drivers/gpu/drm/i915/i915_getparam.o
CC drivers/acpi/acpica/exmutex.o
CC net/sched/sch_mq.o
CC net/netlink/genetlink.o
CC lib/kobject_uevent.o
AR kernel/futex/built-in.a
CC kernel/cgroup/cgroup.o
CC kernel/trace/trace_clock.o
CC fs/proc/vmcore.o
CC drivers/tty/serial/8250/8250_mid.o
CC kernel/bpf/core.o
CC drivers/tty/serial/8250/8250_pci.o
CC [M] sound/core/pcm_memory.o
CC kernel/trace/ring_buffer.o
CC [M] sound/core/memalloc.o
CC kernel/trace/trace.o
CC kernel/events/core.o
CC [M] sound/hda/intel-nhlt.o
CC [M] sound/core/pcm_timer.o
CC crypto/ccm.o
CC drivers/gpu/drm/ttm/ttm_bo_vm.o
CC drivers/base/power/wakeup_stats.o
CC drivers/acpi/acpica/exnames.o
CC drivers/gpu/drm/ttm/ttm_module.o
CC drivers/gpu/drm/ttm/ttm_execbuf_util.o
CC kernel/fork.o
CC kernel/time/tick-oneshot.o
CC kernel/time/tick-sched.o
CC fs/isofs/namei.o
LD [M] sound/core/snd-hwdep.o
CC drivers/tty/serial/8250/8250_pericom.o
CC fs/isofs/inode.o
CC kernel/trace/trace_output.o
CC kernel/trace/trace_seq.o
CC fs/proc/kmsg.o
CC drivers/acpi/reboot.o
CC fs/fat/inode.o
CC drivers/acpi/acpica/exoparg1.o
CC net/core/sock_reuseport.o
CC arch/x86/kernel/cpu/hygon.o
CC drivers/base/power/trace.o
CC fs/ext4/inline.o
CC [M] sound/hda/intel-sdw-acpi.o
CC fs/nfs/client.o
CC mm/show_mem.o
CC mm/shmem_quota.o
CC fs/nfs/dir.o
CC drivers/gpu/drm/i915/i915_ioctl.o
CC drivers/gpu/drm/i915/i915_irq.o
AR sound/core/built-in.a
CC drivers/gpu/drm/i915/i915_mitigations.o
CC net/sched/sch_frag.o
AR io_uring/built-in.a
CC kernel/time/timer_migration.o
CC fs/fat/misc.o
CC drivers/gpu/drm/ttm/ttm_range_manager.o
CC block/mq-deadline.o
AR fs/jbd2/built-in.a
CC net/sched/sch_api.o
CC kernel/events/ring_buffer.o
CC drivers/tty/serial/serial_ctrl.o
CC fs/exportfs/expfs.o
CC lib/logic_pio.o
CC fs/proc/page.o
LD [M] sound/core/snd-pcm.o
CC kernel/trace/trace_stat.o
CC fs/lockd/clntlock.o
CC kernel/trace/trace_printk.o
CC arch/x86/kernel/cpu/centaur.o
CC crypto/aes_generic.o
CC drivers/acpi/acpica/exoparg2.o
LD [M] sound/hda/snd-hda-core.o
LD [M] sound/hda/snd-intel-dspcfg.o
LD [M] sound/hda/snd-intel-sdw-acpi.o
CC sound/sound_core.o
CC fs/lockd/clntproc.o
AR drivers/tty/serial/8250/built-in.a
CC drivers/tty/serial/serial_port.o
CC fs/nfs/file.o
AR drivers/base/power/built-in.a
CC mm/interval_tree.o
CC drivers/base/firmware_loader/builtin/main.o
CC drivers/base/firmware_loader/main.o
CC arch/x86/kernel/cpu/transmeta.o
CC drivers/gpu/drm/ttm/ttm_resource.o
CC drivers/acpi/acpica/exoparg3.o
CC fs/isofs/dir.o
CC fs/fat/nfs.o
CC fs/isofs/util.o
CC sound/last.o
CC drivers/pci/pci-label.o
CC lib/maple_tree.o
AR fs/exportfs/built-in.a
CC lib/memcat_p.o
CC drivers/base/regmap/regmap.o
CC net/ethtool/ioctl.o
CC fs/nls/nls_base.o
AR drivers/base/test/built-in.a
CC net/ethtool/common.o
AR fs/unicode/built-in.a
CC fs/autofs/init.o
AR drivers/base/firmware_loader/builtin/built-in.a
AR fs/proc/built-in.a
CC fs/autofs/inode.o
CC lib/nmi_backtrace.o
CC fs/isofs/rock.o
CC fs/autofs/root.o
CC crypto/crc32c_generic.o
CC mm/list_lru.o
CC net/core/fib_notifier.o
CC net/core/xdp.o
CC net/core/flow_offload.o
CC drivers/tty/serial/earlycon.o
CC net/core/gro.o
CC drivers/acpi/acpica/exoparg6.o
CC kernel/cgroup/rstat.o
CC net/netlink/policy.o
AR sound/built-in.a
CC drivers/gpu/drm/i915/i915_module.o
CC arch/x86/kernel/cpu/zhaoxin.o
CC drivers/gpu/drm/i915/i915_params.o
CC fs/nls/nls_cp437.o
CC lib/objpool.o
CC crypto/authenc.o
CC fs/isofs/export.o
CC fs/lockd/clntxdr.o
CC fs/ext4/inode.o
CC block/kyber-iosched.o
CC fs/fat/namei_vfat.o
CC fs/fat/namei_msdos.o
CC drivers/gpu/drm/i915/i915_pci.o
CC net/ethtool/netlink.o
CC drivers/pci/vgaarb.o
CC fs/nls/nls_ascii.o
CC mm/workingset.o
AR drivers/base/firmware_loader/built-in.a
CC fs/ext4/ioctl.o
CC drivers/gpu/drm/ttm/ttm_pool.o
CC drivers/acpi/acpica/exprep.o
CC drivers/gpu/drm/i915/i915_scatterlist.o
CC fs/nfs/getroot.o
CC kernel/time/vsyscall.o
CC arch/x86/kernel/cpu/vortex.o
CC arch/x86/kernel/cpu/perfctr-watchdog.o
AR kernel/bpf/built-in.a
CC net/ethtool/bitset.o
CC fs/lockd/host.o
CC fs/autofs/symlink.o
AR drivers/gpu/drm/renesas/rcar-du/built-in.a
AR drivers/gpu/drm/renesas/rz-du/built-in.a
AR drivers/gpu/drm/renesas/built-in.a
CC mm/debug.o
CC lib/plist.o
CC drivers/gpu/drm/i915/i915_suspend.o
AR drivers/tty/serial/built-in.a
CC fs/isofs/joliet.o
CC drivers/tty/tty_ioctl.o
CC drivers/tty/tty_ldisc.o
CC fs/nls/nls_iso8859-1.o
CC kernel/events/callchain.o
AR drivers/gpu/drm/omapdrm/built-in.a
AR drivers/gpu/drm/tilcdc/built-in.a
CC drivers/tty/tty_buffer.o
CC drivers/gpu/drm/virtio/virtgpu_drv.o
CC block/blk-mq-pci.o
CC drivers/acpi/acpica/exregion.o
CC fs/nfs/inode.o
CC kernel/time/timekeeping_debug.o
AR net/netlink/built-in.a
CC net/sched/sch_blackhole.o
AR drivers/gpu/drm/imx/built-in.a
AR drivers/gpu/drm/i2c/built-in.a
AR drivers/gpu/drm/panel/built-in.a
AR drivers/gpu/drm/bridge/analogix/built-in.a
AR drivers/gpu/drm/bridge/cadence/built-in.a
AR drivers/gpu/drm/bridge/imx/built-in.a
AR drivers/gpu/drm/bridge/synopsys/built-in.a
AR drivers/gpu/drm/bridge/built-in.a
AR drivers/gpu/drm/hisilicon/built-in.a
AR drivers/gpu/drm/mxsfb/built-in.a
AR drivers/gpu/drm/tiny/built-in.a
AR drivers/gpu/drm/xlnx/built-in.a
CC drivers/gpu/drm/virtio/virtgpu_kms.o
AR drivers/gpu/drm/gud/built-in.a
CC drivers/gpu/drm/virtio/virtgpu_gem.o
CC fs/nfs/super.o
CC fs/nls/nls_utf8.o
CC drivers/gpu/drm/virtio/virtgpu_vram.o
AR drivers/gpu/drm/solomon/built-in.a
CC crypto/authencesn.o
CC drivers/gpu/drm/virtio/virtgpu_display.o
CC drivers/acpi/acpica/exresnte.o
CC mm/gup.o
CC arch/x86/kernel/cpu/vmware.o
CC [M] drivers/gpu/drm/scheduler/sched_main.o
CC drivers/gpu/drm/i915/i915_switcheroo.o
CC drivers/acpi/acpica/exresolv.o
CC fs/autofs/waitq.o
CC drivers/gpu/drm/ttm/ttm_device.o
AR drivers/pci/built-in.a
AR fs/fat/built-in.a
CC mm/mmap_lock.o
CC drivers/acpi/acpica/exresop.o
CC [M] drivers/gpu/drm/xe/xe_bb.o
CC fs/isofs/compress.o
CC fs/9p/vfs_super.o
CC kernel/trace/pid_list.o
AR fs/nls/built-in.a
CC [M] drivers/gpu/drm/xe/xe_bo.o
AR fs/hostfs/built-in.a
CC drivers/acpi/acpica/exserial.o
CC drivers/acpi/acpica/exstore.o
CC fs/9p/vfs_inode.o
CC crypto/lzo.o
CC fs/9p/vfs_inode_dotl.o
CC drivers/acpi/acpica/exstoren.o
CC drivers/gpu/drm/ttm/ttm_sys_manager.o
CC kernel/time/namespace.o
CC drivers/tty/tty_port.o
CC kernel/exec_domain.o
CC fs/debugfs/inode.o
CC fs/tracefs/inode.o
CC fs/debugfs/file.o
CC drivers/gpu/drm/ttm/ttm_agp_backend.o
CC kernel/cgroup/namespace.o
CC crypto/lzo-rle.o
CC arch/x86/kernel/cpu/hypervisor.o
CC fs/lockd/svc.o
CC [M] drivers/gpu/drm/xe/xe_bo_evict.o
CC fs/lockd/svclock.o
CC fs/tracefs/event_inode.o
CC drivers/gpu/drm/virtio/virtgpu_vq.o
CC block/blk-mq-virtio.o
CC net/sched/cls_api.o
CC fs/lockd/svcshare.o
CC net/ethtool/strset.o
CC drivers/gpu/drm/virtio/virtgpu_fence.o
CC fs/lockd/svcproc.o
CC drivers/block/loop.o
CC drivers/acpi/acpica/exstorob.o
CC drivers/block/virtio_blk.o
CC net/ethtool/linkinfo.o
CC arch/x86/kernel/cpu/mshyperv.o
CC net/sched/act_api.o
CC lib/radix-tree.o
CC fs/autofs/expire.o
CC fs/9p/vfs_addr.o
CC crypto/rng.o
CC drivers/base/regmap/regcache.o
AR fs/isofs/built-in.a
CC fs/lockd/svcsubs.o
CC drivers/gpu/drm/i915/i915_sysfs.o
CC net/netfilter/core.o
CC net/ipv4/netfilter/nf_defrag_ipv4.o
AR kernel/time/built-in.a
CC net/ipv4/route.o
AR drivers/gpu/drm/ttm/built-in.a
CC kernel/trace/trace_sched_switch.o
CC kernel/panic.o
CC kernel/trace/trace_nop.o
CC net/xfrm/xfrm_policy.o
CC drivers/tty/tty_mutex.o
CC drivers/acpi/acpica/exsystem.o
CC net/xfrm/xfrm_state.o
CC kernel/cgroup/cgroup-v1.o
CC drivers/acpi/acpica/extrace.o
CC [M] drivers/gpu/drm/scheduler/sched_fence.o
CC drivers/base/component.o
CC net/unix/af_unix.o
CC fs/autofs/dev-ioctl.o
CC block/blk-mq-debugfs.o
AR fs/debugfs/built-in.a
CC [M] drivers/gpu/drm/xe/xe_debugfs.o
CC lib/ratelimit.o
CC mm/highmem.o
CC [M] drivers/gpu/drm/xe/xe_devcoredump.o
AR fs/tracefs/built-in.a
CC net/ipv6/netfilter/ip6_tables.o
CC net/ipv6/af_inet6.o
CC drivers/acpi/acpica/exutils.o
CC net/ipv6/netfilter/ip6table_filter.o
CC [M] drivers/gpu/drm/xe/xe_device.o
CC [M] drivers/gpu/drm/scheduler/sched_entity.o
CC fs/9p/vfs_file.o
CC drivers/acpi/acpica/hwacpi.o
CC net/packet/af_packet.o
CC drivers/tty/tty_ldsem.o
CC crypto/drbg.o
CC [M] fs/efivarfs/inode.o
CC [M] fs/efivarfs/file.o
CC arch/x86/kernel/cpu/debugfs.o
CC drivers/tty/tty_baudrate.o
CC drivers/gpu/drm/virtio/virtgpu_object.o
CC fs/lockd/mon.o
CC net/ethtool/linkmodes.o
CC drivers/base/regmap/regcache-rbtree.o
CC net/sched/sch_fifo.o
CC drivers/tty/tty_jobctrl.o
CC drivers/gpu/drm/i915/i915_utils.o
CC drivers/acpi/nvs.o
CC kernel/events/hw_breakpoint.o
CC fs/ext4/mballoc.o
CC kernel/events/uprobes.o
CC net/ipv4/netfilter/nf_reject_ipv4.o
CC fs/lockd/trace.o
CC drivers/acpi/acpica/hwesleep.o
CC drivers/gpu/drm/drm_aperture.o
AR fs/autofs/built-in.a
CC drivers/gpu/drm/virtio/virtgpu_debugfs.o
AR drivers/block/built-in.a
CC [M] fs/efivarfs/super.o
CC [M] fs/efivarfs/vars.o
CC block/blk-pm.o
CC kernel/trace/blktrace.o
CC arch/x86/kernel/cpu/capflags.o
CC net/core/netdev-genl.o
CC net/netfilter/nf_log.o
AR drivers/misc/eeprom/built-in.a
CC fs/nfs/io.o
AR arch/x86/kernel/cpu/built-in.a
AR drivers/mfd/built-in.a
CC arch/x86/kernel/signal.o
AR drivers/misc/cb710/built-in.a
CC fs/nfs/direct.o
LD [M] drivers/gpu/drm/scheduler/gpu-sched.o
CC drivers/tty/n_null.o
CC net/core/netdev-genl-gen.o
AR drivers/misc/ti-st/built-in.a
CC net/netfilter/nf_queue.o
CC fs/nfs/pagelist.o
AR drivers/misc/lis3lv02d/built-in.a
AR drivers/misc/cardreader/built-in.a
CC drivers/acpi/acpica/hwgpe.o
AR drivers/misc/built-in.a
CC drivers/gpu/drm/virtio/virtgpu_plane.o
CC mm/memory.o
CC [M] drivers/gpu/drm/xe/xe_device_sysfs.o
CC fs/9p/vfs_dir.o
CC kernel/trace/trace_events.o
CC net/ethtool/rss.o
CC kernel/cgroup/freezer.o
CC kernel/cpu.o
CC drivers/base/regmap/regcache-flat.o
CC crypto/jitterentropy.o
CC kernel/exit.o
CC net/ethtool/linkstate.o
CC crypto/jitterentropy-kcapi.o
CC net/ethtool/debug.o
CC crypto/ghash-generic.o
CC net/sched/cls_cgroup.o
CC kernel/trace/trace_export.o
CC drivers/gpu/drm/i915/intel_clock_gating.o
CC drivers/base/regmap/regcache-maple.o
CC drivers/gpu/drm/drm_atomic.o
CC drivers/acpi/acpica/hwregs.o
CC drivers/tty/pty.o
CC drivers/tty/tty_audit.o
LD [M] fs/efivarfs/efivarfs.o
CC net/ethtool/wol.o
CC block/holder.o
CC crypto/hash_info.o
CC kernel/trace/trace_event_perf.o
CC fs/lockd/xdr.o
CC net/ipv6/netfilter/ip6table_mangle.o
CC kernel/softirq.o
CC fs/9p/vfs_dentry.o
CC drivers/tty/sysrq.o
CC fs/lockd/clnt4xdr.o
CC net/ipv6/anycast.o
CC net/xfrm/xfrm_hash.o
CC arch/x86/kernel/signal_32.o
CC drivers/gpu/drm/virtio/virtgpu_ioctl.o
CC [M] drivers/gpu/drm/xe/xe_dma_buf.o
CC crypto/rsapubkey.asn1.o
CC lib/rbtree.o
CC drivers/gpu/drm/virtio/virtgpu_prime.o
CC crypto/rsaprivkey.asn1.o
AR crypto/built-in.a
CC net/xfrm/xfrm_input.o
CC fs/9p/v9fs.o
CC drivers/acpi/acpica/hwsleep.o
CC net/netfilter/nf_sockopt.o
CC fs/lockd/xdr4.o
CC kernel/cgroup/legacy_freezer.o
CC net/ipv4/netfilter/ip_tables.o
CC drivers/base/regmap/regmap-debugfs.o
AR kernel/events/built-in.a
CC lib/seq_buf.o
CC kernel/trace/trace_events_filter.o
CC kernel/trace/trace_events_trigger.o
CC net/unix/garbage.o
CC fs/ext4/migrate.o
CC net/unix/sysctl_net_unix.o
CC net/xfrm/xfrm_output.o
CC fs/9p/fid.o
CC drivers/gpu/drm/virtio/virtgpu_trace_points.o
AR block/built-in.a
CC drivers/acpi/acpica/hwvalid.o
CC net/ipv4/netfilter/iptable_filter.o
CC net/core/gso.o
CC drivers/acpi/acpica/hwxface.o
CC drivers/gpu/drm/drm_atomic_uapi.o
CC drivers/gpu/drm/virtio/virtgpu_submit.o
CC net/sched/ematch.o
CC mm/mincore.o
CC kernel/cgroup/pids.o
CC kernel/trace/trace_eprobe.o
CC net/ethtool/features.o
CC mm/mlock.o
CC arch/x86/kernel/traps.o
CC drivers/gpu/drm/i915/intel_device_info.o
CC drivers/gpu/drm/i915/intel_memory_region.o
AR drivers/tty/built-in.a
CC net/ipv4/inetpeer.o
CC mm/mmap.o
AR drivers/nfc/built-in.a
CC drivers/gpu/drm/drm_auth.o
CC drivers/gpu/drm/drm_blend.o
CC [M] drivers/gpu/drm/xe/xe_drm_client.o
CC net/ipv4/protocol.o
CC lib/siphash.o
CC drivers/acpi/acpica/hwxfsleep.o
CC kernel/trace/trace_kprobe.o
AR drivers/base/regmap/built-in.a
CC net/ethtool/privflags.o
CC drivers/base/core.o
CC drivers/acpi/acpica/hwpci.o
CC net/ipv6/netfilter/nf_defrag_ipv6_hooks.o
CC net/netfilter/utils.o
CC net/ipv4/ip_input.o
CC arch/x86/kernel/idt.o
CC fs/9p/xattr.o
CC kernel/resource.o
CC fs/lockd/svc4proc.o
CC kernel/cgroup/rdma.o
CC net/core/net-sysfs.o
CC net/ethtool/rings.o
CC fs/nfs/read.o
CC net/ethtool/channels.o
CC fs/lockd/procfs.o
AR net/packet/built-in.a
CC fs/ext4/mmp.o
AR net/dsa/built-in.a
AR drivers/gpu/drm/virtio/built-in.a
CC net/sunrpc/auth_gss/auth_gss.o
CC drivers/acpi/acpica/nsaccess.o
CC drivers/gpu/drm/drm_bridge.o
AR net/unix/built-in.a
CC lib/string.o
CC net/sunrpc/auth_gss/gss_generic_token.o
CC kernel/trace/error_report-traces.o
CC kernel/trace/power-traces.o
AR net/wireless/tests/built-in.a
CC kernel/trace/rpm-traces.o
CC net/wireless/core.o
CC net/wireless/sysfs.o
AR net/mac80211/tests/built-in.a
CC net/mac80211/main.o
CC net/netlabel/netlabel_user.o
CC net/mac80211/status.o
AR net/sched/built-in.a
CC mm/mmu_gather.o
CC net/netlabel/netlabel_kapi.o
CC lib/timerqueue.o
CC net/netlabel/netlabel_domainhash.o
CC net/wireless/radiotap.o
CC net/ipv4/netfilter/iptable_mangle.o
CC net/ipv4/netfilter/ipt_REJECT.o
CC net/xfrm/xfrm_sysctl.o
AR fs/9p/built-in.a
CC net/xfrm/xfrm_replay.o
CC fs/open.o
CC drivers/gpu/drm/drm_cache.o
CC drivers/gpu/drm/drm_client.o
CC drivers/acpi/acpica/nsalloc.o
CC lib/vsprintf.o
CC drivers/gpu/drm/i915/intel_pcode.o
CC arch/x86/kernel/irq.o
CC kernel/trace/trace_dynevent.o
CC drivers/gpu/drm/i915/intel_region_ttm.o
CC kernel/cgroup/cpuset.o
CC arch/x86/kernel/irq_32.o
CC drivers/gpu/drm/i915/intel_runtime_pm.o
CC net/netlabel/netlabel_addrlist.o
CC [M] drivers/gpu/drm/xe/xe_exec.o
CC net/sunrpc/auth_gss/gss_mech_switch.o
CC drivers/acpi/acpica/nsarguments.o
CC mm/mprotect.o
CC net/ipv6/netfilter/nf_conntrack_reasm.o
CC net/wireless/util.o
CC mm/mremap.o
CC net/netfilter/nfnetlink.o
CC mm/msync.o
CC net/mac80211/driver-ops.o
AR fs/lockd/built-in.a
CC net/ethtool/coalesce.o
CC net/netfilter/nfnetlink_log.o
CC mm/page_vma_mapped.o
CC net/ipv6/netfilter/nf_reject_ipv6.o
CC kernel/trace/trace_probe.o
CC drivers/gpu/drm/i915/intel_sbi.o
CC net/wireless/reg.o
CC fs/read_write.o
CC net/xfrm/xfrm_device.o
CC drivers/acpi/acpica/nsconvert.o
CC net/xfrm/xfrm_algo.o
CC fs/file_table.o
CC kernel/trace/trace_uprobe.o
CC kernel/trace/rethook.o
CC mm/pagewalk.o
CC drivers/gpu/drm/i915/intel_step.o
CC drivers/gpu/drm/i915/intel_uncore.o
CC [M] net/ipv4/netfilter/iptable_nat.o
CC kernel/sysctl.o
CC net/ipv4/ip_fragment.o
CC drivers/gpu/drm/i915/intel_wakeref.o
CC net/xfrm/xfrm_user.o
CC [M] drivers/gpu/drm/xe/xe_execlist.o
CC drivers/acpi/acpica/nsdump.o
CC net/ipv6/netfilter/ip6t_ipv6header.o
CC net/core/hotdata.o
CC drivers/acpi/acpica/nseval.o
CC fs/ext4/move_extent.o
CC net/netlabel/netlabel_mgmt.o
CC drivers/gpu/drm/i915/vlv_sideband.o
CC fs/ext4/namei.o
CC net/ethtool/pause.o
CC lib/win_minmax.o
CC fs/nfs/symlink.o
CC drivers/base/bus.o
CC mm/pgtable-generic.o
CC net/ethtool/eee.o
CC drivers/gpu/drm/i915/vlv_suspend.o
CC net/ipv6/netfilter/ip6t_REJECT.o
CC drivers/gpu/drm/i915/soc/intel_dram.o
CC fs/super.o
CC net/ethtool/tsinfo.o
CC drivers/acpi/acpica/nsinit.o
CC net/sunrpc/auth_gss/svcauth_gss.o
CC net/sunrpc/auth_gss/gss_rpc_upcall.o
CC fs/char_dev.o
CC [M] drivers/gpu/drm/xe/xe_exec_queue.o
CC [M] drivers/gpu/drm/xe/xe_force_wake.o
CC kernel/cgroup/misc.o
CC kernel/capability.o
CC arch/x86/kernel/dumpstack_32.o
CC net/netfilter/nf_conntrack_core.o
CC net/ethtool/cabletest.o
CC mm/rmap.o
CC fs/nfs/unlink.o
CC drivers/acpi/acpica/nsload.o
CC drivers/gpu/drm/i915/soc/intel_gmch.o
CC net/core/net-procfs.o
CC net/sunrpc/auth_gss/gss_rpc_xdr.o
CC net/core/netpoll.o
AR net/ipv4/netfilter/built-in.a
CC kernel/cgroup/debug.o
CC mm/vmalloc.o
CC net/ipv6/ip6_output.o
CC drivers/base/dd.o
CC drivers/acpi/acpica/nsnames.o
CC drivers/gpu/drm/i915/soc/intel_pch.o
CC drivers/gpu/drm/i915/i915_memcpy.o
CC drivers/gpu/drm/i915/i915_mm.o
CC net/ipv6/ip6_input.o
CC net/ipv4/ip_forward.o
CC lib/xarray.o
CC net/ipv4/ip_options.o
CC drivers/acpi/acpica/nsobject.o
CC drivers/gpu/drm/i915/i915_sw_fence.o
CC lib/lockref.o
CC net/ethtool/tunnels.o
CC arch/x86/kernel/time.o
CC net/netlabel/netlabel_unlabeled.o
CC net/ipv4/ip_output.o
CC fs/nfs/write.o
CC [M] drivers/gpu/drm/xe/xe_ggtt.o
CC drivers/acpi/acpica/nsparse.o
AR kernel/trace/built-in.a
CC fs/ext4/page-io.o
AR drivers/dax/hmem/built-in.a
CC kernel/ptrace.o
CC [M] drivers/gpu/drm/xe/xe_gpu_scheduler.o
AR drivers/dax/built-in.a
CC net/ipv4/ip_sockglue.o
CC drivers/acpi/acpica/nspredef.o
CC lib/bcd.o
AR net/ipv6/netfilter/built-in.a
CC net/rfkill/core.o
CC lib/sort.o
CC drivers/gpu/drm/i915/i915_sw_fence_work.o
CC drivers/gpu/drm/i915/i915_syncmap.o
CC net/rfkill/input.o
CC lib/parser.o
CC net/sunrpc/auth_gss/trace.o
CC net/ipv6/addrconf.o
CC lib/debug_locks.o
CC arch/x86/kernel/ioport.o
CC drivers/acpi/acpica/nsprepkg.o
CC mm/process_vm_access.o
CC fs/ext4/readpage.o
CC net/ipv4/inet_hashtables.o
CC net/mac80211/sta_info.o
AR kernel/cgroup/built-in.a
CC kernel/user.o
CC mm/page_alloc.o
CC fs/stat.o
CC net/ipv4/inet_timewait_sock.o
CC drivers/base/syscore.o
CC net/wireless/scan.o
HOSTCC drivers/gpu/drm/xe/xe_gen_wa_oob
CC net/core/fib_rules.o
CC lib/random32.o
CC net/mac80211/wep.o
CC drivers/gpu/drm/i915/i915_user_extensions.o
CC net/mac80211/aead_api.o
CC net/mac80211/wpa.o
CC drivers/acpi/wakeup.o
CC drivers/gpu/drm/drm_client_modeset.o
CC drivers/gpu/drm/i915/i915_debugfs.o
CC drivers/acpi/acpica/nsrepair.o
CC net/core/net-traces.o
CC kernel/signal.o
CC [M] drivers/gpu/drm/xe/xe_gsc_proxy.o
CC lib/bust_spinlocks.o
CC drivers/acpi/sleep.o
AR net/xfrm/built-in.a
CC drivers/gpu/drm/i915/i915_debugfs_params.o
CC lib/kasprintf.o
CC fs/ext4/resize.o
AR net/rfkill/built-in.a
CC drivers/gpu/drm/i915/i915_pmu.o
CC arch/x86/kernel/dumpstack.o
CC net/ethtool/fec.o
CC lib/bitmap.o
CC kernel/sys.o
CC drivers/gpu/drm/i915/gt/gen2_engine_cs.o
CC mm/init-mm.o
CC net/ethtool/eeprom.o
CC net/sunrpc/auth_gss/gss_krb5_mech.o
CC kernel/umh.o
CC drivers/acpi/acpica/nsrepair2.o
CC net/netlabel/netlabel_cipso_v4.o
CC fs/ext4/super.o
CC arch/x86/kernel/nmi.o
CC fs/nfs/namespace.o
CC net/ipv4/inet_connection_sock.o
CC net/netfilter/nf_conntrack_standalone.o
CC net/core/selftests.o
CC net/ipv4/tcp.o
CC drivers/base/driver.o
CC lib/scatterlist.o
CC net/netlabel/netlabel_calipso.o
CC net/ethtool/stats.o
CC drivers/acpi/acpica/nssearch.o
CC mm/memblock.o
CC net/core/ptp_classifier.o
CC drivers/gpu/drm/i915/gt/gen6_engine_cs.o
CC net/9p/mod.o
CC drivers/gpu/drm/i915/gt/gen6_ppgtt.o
CC [M] drivers/gpu/drm/xe/xe_gsc_submit.o
CC arch/x86/kernel/ldt.o
CC lib/list_sort.o
CC drivers/acpi/acpica/nsutils.o
CC net/sunrpc/auth_gss/gss_krb5_seal.o
CC net/dns_resolver/dns_key.o
CC net/dns_resolver/dns_query.o
CC drivers/base/class.o
CC net/core/netprio_cgroup.o
CC drivers/gpu/drm/drm_color_mgmt.o
CC net/wireless/nl80211.o
CC kernel/workqueue.o
CC fs/ext4/symlink.o
CC fs/ext4/sysfs.o
CC net/mac80211/scan.o
CC lib/uuid.o
CC lib/iov_iter.o
CC net/wireless/mlme.o
CC net/wireless/ibss.o
CC drivers/gpu/drm/drm_connector.o
CC kernel/pid.o
CC drivers/acpi/device_sysfs.o
CC net/9p/client.o
CC fs/ext4/xattr.o
CC fs/exec.o
CC net/sunrpc/auth_gss/gss_krb5_unseal.o
CC drivers/gpu/drm/drm_crtc.o
CC net/9p/error.o
CC drivers/acpi/acpica/nswalk.o
CC drivers/gpu/drm/i915/gt/gen7_renderclear.o
CC net/netfilter/nf_conntrack_expect.o
CC net/netfilter/nf_conntrack_helper.o
CC kernel/task_work.o
AR net/netlabel/built-in.a
CC lib/clz_ctz.o
CC fs/nfs/mount_clnt.o
CC net/ethtool/phc_vclocks.o
AR net/dns_resolver/built-in.a
CC net/ipv4/tcp_input.o
CC net/handshake/alert.o
CC net/devres.o
CC kernel/extable.o
CC kernel/params.o
CC net/ipv4/tcp_output.o
CC arch/x86/kernel/setup.o
CC net/handshake/genl.o
CC net/socket.o
CC drivers/base/platform.o
CC [M] drivers/gpu/drm/xe/xe_gt.o
CC net/handshake/netlink.o
CC net/wireless/sme.o
CC drivers/acpi/acpica/nsxfeval.o
CC net/netfilter/nf_conntrack_proto.o
CC drivers/dma-buf/dma-buf.o
CC drivers/acpi/device_pm.o
CC fs/ext4/xattr_hurd.o
CC drivers/acpi/acpica/nsxfname.o
CC net/ethtool/mm.o
CC net/sunrpc/auth_gss/gss_krb5_wrap.o
CC drivers/gpu/drm/i915/gt/gen8_engine_cs.o
CC kernel/kthread.o
CC net/ipv4/tcp_timer.o
CC net/9p/protocol.o
CC drivers/acpi/proc.o
CC drivers/gpu/drm/i915/gt/gen8_ppgtt.o
CC drivers/acpi/acpica/nsxfobj.o
CC drivers/base/cpu.o
CC drivers/base/firmware.o
CC net/ipv4/tcp_ipv4.o
CC net/9p/trans_common.o
CC mm/slub.o
CC net/ethtool/module.o
CC net/wireless/chan.o
CC [M] drivers/gpu/drm/xe/xe_gt_ccs_mode.o
CC kernel/sys_ni.o
CC fs/nfs/nfstrace.o
CC net/netfilter/nf_conntrack_proto_generic.o
CC net/sunrpc/auth_gss/gss_krb5_crypto.o
CC net/sysctl_net.o
CC arch/x86/kernel/x86_init.o
CC drivers/gpu/drm/drm_displayid.o
CC drivers/acpi/acpica/psargs.o
CC drivers/dma-buf/dma-fence.o
CC [M] drivers/gpu/drm/xe/xe_gt_clock.o
CC fs/ext4/xattr_trusted.o
CC mm/madvise.o
CC drivers/dma-buf/dma-fence-array.o
CC lib/bsearch.o
CC drivers/gpu/drm/drm_drv.o
CC arch/x86/kernel/i8259.o
CC drivers/base/init.o
CC net/handshake/request.o
CC net/handshake/tlshd.o
CC net/ipv6/addrlabel.o
CC fs/pipe.o
CC net/9p/trans_fd.o
CC net/handshake/trace.o
CC net/netfilter/nf_conntrack_proto_tcp.o
CC drivers/gpu/drm/i915/gt/intel_breadcrumbs.o
CC lib/find_bit.o
CC drivers/acpi/acpica/psloop.o
CC fs/ext4/xattr_user.o
CC net/wireless/ethtool.o
CC net/ethtool/pse-pd.o
CC fs/namei.o
CC drivers/dma-buf/dma-fence-chain.o
CC drivers/base/map.o
CC net/netfilter/nf_conntrack_proto_udp.o
CC [M] drivers/gpu/drm/xe/xe_gt_debugfs.o
CC net/wireless/mesh.o
CC lib/llist.o
CC kernel/nsproxy.o
CC net/core/netclassid_cgroup.o
CC arch/x86/kernel/irqinit.o
CC kernel/notifier.o
CC drivers/dma-buf/dma-fence-unwrap.o
CC fs/ext4/fast_commit.o
CC lib/lwq.o
CC net/netfilter/nf_conntrack_proto_icmp.o
CC drivers/gpu/drm/i915/gt/intel_context.o
CC drivers/acpi/acpica/psobject.o
CC arch/x86/kernel/jump_label.o
CC mm/page_io.o
CC net/sunrpc/auth_gss/gss_krb5_keys.o
CC drivers/acpi/bus.o
CC drivers/acpi/glue.o
CC lib/memweight.o
CC fs/ext4/orphan.o
CC drivers/acpi/acpica/psopcode.o
CC drivers/base/devres.o
CC net/mac80211/offchannel.o
CC drivers/gpu/drm/drm_dumb_buffers.o
CC lib/kfifo.o
CC drivers/gpu/drm/drm_edid.o
CC lib/percpu-refcount.o
CC drivers/dma-buf/dma-resv.o
CC fs/ext4/acl.o
CC drivers/dma-buf/sync_file.o
CC drivers/acpi/scan.o
CC net/ipv6/route.o
CC drivers/acpi/acpica/psopinfo.o
CC fs/fcntl.o
CC arch/x86/kernel/irq_work.o
CC net/9p/trans_virtio.o
CC net/wireless/ap.o
CC [M] drivers/gpu/drm/xe/xe_gt_freq.o
CC mm/swap_state.o
CC net/ethtool/plca.o
CC arch/x86/kernel/probe_roms.o
AR drivers/cxl/core/built-in.a
AR drivers/cxl/built-in.a
CC drivers/macintosh/mac_hid.o
CC net/core/dst_cache.o
CC kernel/ksysfs.o
CC [M] drivers/gpu/drm/xe/xe_gt_idle.o
CC drivers/gpu/drm/i915/gt/intel_context_sseu.o
CC drivers/acpi/acpica/psparse.o
AR net/sunrpc/auth_gss/built-in.a
CC net/core/gro_cells.o
AR net/handshake/built-in.a
CC net/sunrpc/clnt.o
CC net/core/failover.o
CC mm/swapfile.o
CC mm/swap_slots.o
CC drivers/base/attribute_container.o
CC kernel/cred.o
CC net/ipv4/tcp_minisocks.o
CC net/ipv4/tcp_cong.o
CC lib/rhashtable.o
CC drivers/acpi/mipi-disco-img.o
CC net/ipv4/tcp_metrics.o
AR drivers/scsi/pcmcia/built-in.a
CC net/netfilter/nf_conntrack_extend.o
CC drivers/scsi/scsi.o
AR drivers/dma-buf/built-in.a
AR drivers/nvme/common/built-in.a
CC drivers/acpi/resource.o
AR drivers/nvme/host/built-in.a
AR drivers/nvme/target/built-in.a
AR drivers/nvme/built-in.a
CC kernel/reboot.o
CC net/mac80211/ht.o
CC fs/nfs/export.o
AR drivers/macintosh/built-in.a
CC [M] drivers/gpu/drm/xe/xe_gt_mcr.o
CC drivers/acpi/acpica/psscope.o
CC fs/nfs/sysfs.o
CC arch/x86/kernel/sys_ia32.o
CC kernel/async.o
CC drivers/acpi/acpi_processor.o
CC drivers/base/transport_class.o
CC net/ipv6/ip6_fib.o
CC net/ipv4/tcp_fastopen.o
CC arch/x86/kernel/ksysfs.o
CC drivers/gpu/drm/i915/gt/intel_engine_cs.o
CC drivers/acpi/acpica/pstree.o
AR net/ethtool/built-in.a
CC net/sunrpc/xprt.o
CC [M] drivers/gpu/drm/xe/xe_gt_pagefault.o
CC net/mac80211/agg-tx.o
CC fs/nfs/fs_context.o
CC kernel/range.o
CC net/mac80211/agg-rx.o
CC drivers/ata/libata-core.o
AR net/9p/built-in.a
CC mm/dmapool.o
CC fs/ext4/xattr_security.o
CC drivers/ata/libata-scsi.o
CC net/mac80211/vht.o
CC drivers/base/topology.o
CC net/ipv6/ipv6_sockglue.o
AR net/core/built-in.a
CC net/mac80211/he.o
CC drivers/base/container.o
CC net/ipv6/ndisc.o
CC net/ipv4/tcp_rate.o
CC drivers/acpi/acpica/psutils.o
CC lib/base64.o
CC kernel/smpboot.o
CC arch/x86/kernel/bootflag.o
CC net/netfilter/nf_conntrack_acct.o
CC lib/once.o
CC arch/x86/kernel/e820.o
CC net/mac80211/s1g.o
CC net/wireless/trace.o
CC arch/x86/kernel/pci-dma.o
CC net/sunrpc/socklib.o
CC drivers/acpi/processor_core.o
CC net/ipv6/udp.o
CC net/wireless/ocb.o
CC net/ipv6/udplite.o
CC drivers/acpi/acpica/pswalk.o
CC drivers/scsi/hosts.o
AR drivers/net/phy/qcom/built-in.a
CC drivers/net/phy/mdio-boardinfo.o
CC drivers/firewire/init_ohci1394_dma.o
CC mm/hugetlb.o
CC drivers/base/property.o
CC drivers/cdrom/cdrom.o
CC lib/refcount.o
CC kernel/ucount.o
CC net/sunrpc/xprtsock.o
CC drivers/ata/libata-eh.o
CC drivers/scsi/scsi_ioctl.o
CC drivers/acpi/acpica/psxface.o
CC lib/rcuref.o
CC fs/nfs/nfsroot.o
CC net/ipv4/tcp_recovery.o
CC drivers/acpi/processor_pdc.o
CC drivers/net/phy/stubs.o
CC mm/mmu_notifier.o
CC net/mac80211/ibss.o
CC [M] drivers/gpu/drm/xe/xe_gt_sysfs.o
CC net/mac80211/iface.o
CC net/ipv4/tcp_ulp.o
CC lib/usercopy.o
CC net/mac80211/link.o
CC net/wireless/pmsr.o
CC kernel/regset.o
CC net/netfilter/nf_conntrack_seqadj.o
AR drivers/firewire/built-in.a
GEN net/wireless/shipped-certs.c
AR drivers/auxdisplay/built-in.a
CC drivers/pcmcia/cs.o
CC drivers/acpi/acpica/rsaddr.o
CC drivers/usb/common/common.o
CC arch/x86/kernel/quirks.o
CC drivers/pcmcia/socket_sysfs.o
CC drivers/usb/common/debug.o
CC drivers/input/serio/serio.o
CC lib/errseq.o
CC drivers/pcmcia/cardbus.o
CC drivers/input/serio/i8042.o
CC drivers/input/serio/serport.o
CC drivers/acpi/ec.o
CC lib/bucket_locks.o
CC kernel/ksyms_common.o
CC drivers/acpi/acpica/rscalc.o
CC net/mac80211/rate.o
CC net/mac80211/michael.o
CC drivers/base/cacheinfo.o
CC drivers/scsi/scsicam.o
CC drivers/gpu/drm/i915/gt/intel_engine_heartbeat.o
CC net/ipv6/raw.o
CC drivers/net/phy/mdio_devres.o
CC drivers/ata/libata-transport.o
CC [M] drivers/gpu/drm/xe/xe_gt_throttle.o
CC drivers/gpu/drm/i915/gt/intel_engine_pm.o
CC drivers/pcmcia/ds.o
AR drivers/usb/common/built-in.a
CC drivers/usb/core/usb.o
AR drivers/net/pse-pd/built-in.a
CC arch/x86/kernel/kdebugfs.o
CC drivers/gpu/drm/i915/gt/intel_engine_user.o
CC mm/migrate.o
AR drivers/usb/phy/built-in.a
CC lib/generic-radix-tree.o
CC drivers/net/mdio/acpi_mdio.o
CC fs/nfs/sysctl.o
CC drivers/acpi/acpica/rscreate.o
CC kernel/groups.o
CC fs/nfs/nfs3super.o
CC lib/bitmap-str.o
AR drivers/cdrom/built-in.a
CC drivers/usb/mon/mon_main.o
CC net/netfilter/nf_conntrack_proto_icmpv6.o
CC drivers/input/keyboard/atkbd.o
CC net/ipv4/tcp_offload.o
AR fs/ext4/built-in.a
CC net/mac80211/tkip.o
CC net/netfilter/nf_conntrack_netlink.o
CC mm/page_counter.o
CC drivers/input/mouse/psmouse-base.o
AR drivers/input/joystick/built-in.a
CC drivers/scsi/scsi_error.o
CC arch/x86/kernel/alternative.o
CC drivers/scsi/scsi_lib.o
CC drivers/acpi/acpica/rsdumpinfo.o
CC drivers/acpi/acpica/rsinfo.o
CC drivers/base/swnode.o
CC drivers/acpi/dock.o
CC drivers/gpu/drm/i915/gt/intel_execlists_submission.o
CC drivers/ata/libata-trace.o
CC drivers/input/serio/libps2.o
CC drivers/net/phy/phy.o
CC drivers/acpi/pci_root.o
CC net/ipv4/tcp_plb.o
CC net/ipv6/icmp.o
CC net/ipv6/mcast.o
CC [M] drivers/gpu/drm/xe/xe_gt_tlb_invalidation.o
CC drivers/acpi/pci_link.o
CC drivers/usb/core/hub.o
CC drivers/usb/mon/mon_stat.o
CC drivers/acpi/acpica/rsio.o
CC lib/string_helpers.o
CC kernel/kcmp.o
CC drivers/net/mdio/fwnode_mdio.o
CC drivers/usb/core/hcd.o
CC drivers/usb/core/urb.o
CC drivers/pcmcia/pcmcia_resource.o
CC net/netfilter/nf_conntrack_ftp.o
CC [M] drivers/gpu/drm/xe/xe_gt_topology.o
CC net/ipv6/reassembly.o
CC fs/nfs/nfs3client.o
GEN xe_wa_oob.c xe_wa_oob.h
CC net/ipv6/tcp_ipv6.o
CC arch/x86/kernel/i8253.o
AR drivers/input/keyboard/built-in.a
CC drivers/acpi/acpica/rsirq.o
CC arch/x86/kernel/hw_breakpoint.o
AR drivers/input/serio/built-in.a
CC drivers/rtc/lib.o
CC drivers/base/auxiliary.o
CC drivers/usb/mon/mon_text.o
CC drivers/input/mouse/synaptics.o
CC drivers/rtc/class.o
CC net/netfilter/nf_conntrack_irc.o
CC fs/nfs/nfs3proc.o
CC mm/hugetlb_cgroup.o
CC drivers/gpu/drm/drm_eld.o
CC drivers/acpi/acpica/rslist.o
CC net/mac80211/aes_cmac.o
CC drivers/i2c/algos/i2c-algo-bit.o
CC drivers/gpu/drm/drm_encoder.o
CC drivers/gpu/drm/drm_file.o
CC drivers/i2c/busses/i2c-i801.o
CC kernel/freezer.o
CC arch/x86/kernel/tsc.o
CC lib/hexdump.o
CC drivers/acpi/acpica/rsmemory.o
CC drivers/ata/libata-sata.o
CC lib/kstrtox.o
AR drivers/net/mdio/built-in.a
CC net/ipv4/datagram.o
CC drivers/base/devtmpfs.o
AR drivers/net/pcs/built-in.a
CC net/mac80211/aes_gmac.o
CC mm/early_ioremap.o
CC net/mac80211/fils_aead.o
CC fs/nfs/nfs3xdr.o
CC net/mac80211/cfg.o
AR drivers/i3c/built-in.a
CC drivers/scsi/constants.o
CC drivers/acpi/pci_irq.o
CC drivers/ata/libata-sff.o
CC drivers/net/phy/phy-c45.o
CC [M] drivers/gpu/drm/xe/xe_guc_ct.o
CC net/sunrpc/sched.o
CC drivers/net/phy/phy-core.o
CC drivers/acpi/acpica/rsmisc.o
CC drivers/pcmcia/cistpl.o
CC drivers/rtc/interface.o
AR drivers/media/i2c/built-in.a
CC arch/x86/kernel/tsc_msr.o
AR drivers/media/tuners/built-in.a
AR drivers/media/rc/keymaps/built-in.a
AR drivers/media/rc/built-in.a
AR drivers/media/common/b2c2/built-in.a
AR drivers/media/platform/allegro-dvt/built-in.a
CC drivers/usb/mon/mon_bin.o
AR drivers/media/common/saa7146/built-in.a
AR drivers/media/platform/amlogic/meson-ge2d/built-in.a
AR drivers/media/common/siano/built-in.a
AR drivers/media/platform/amlogic/built-in.a
AR drivers/media/common/v4l2-tpg/built-in.a
AR drivers/media/platform/amphion/built-in.a
CC lib/iomap.o
AR drivers/media/common/videobuf2/built-in.a
AR drivers/media/platform/aspeed/built-in.a
AR drivers/media/common/built-in.a
AR drivers/media/platform/atmel/built-in.a
AR drivers/media/platform/broadcom/built-in.a
CC net/ipv4/raw.o
CC net/ipv4/udp.o
AR drivers/media/platform/cadence/built-in.a
CC kernel/profile.o
AR drivers/media/platform/chips-media/coda/built-in.a
AR drivers/media/platform/chips-media/wave5/built-in.a
CC kernel/stacktrace.o
AR drivers/media/platform/chips-media/built-in.a
CC drivers/acpi/acpi_apd.o
CC kernel/dma.o
AR drivers/media/platform/intel/built-in.a
CC net/ipv4/udplite.o
AR drivers/media/platform/marvell/built-in.a
CC net/ipv4/udp_offload.o
CC net/mac80211/ethtool.o
AR drivers/media/platform/mediatek/jpeg/built-in.a
CC drivers/scsi/scsi_lib_dma.o
AR drivers/media/platform/mediatek/mdp/built-in.a
AR drivers/i2c/algos/built-in.a
AR drivers/media/platform/mediatek/vcodec/common/built-in.a
CC mm/secretmem.o
AR drivers/media/platform/mediatek/vcodec/encoder/built-in.a
AR drivers/media/platform/mediatek/vpu/built-in.a
AR drivers/media/platform/mediatek/vcodec/decoder/built-in.a
CC kernel/smp.o
AR drivers/media/platform/mediatek/vcodec/built-in.a
AR drivers/media/platform/mediatek/mdp3/built-in.a
AR drivers/media/platform/mediatek/built-in.a
CC net/ipv6/ping.o
CC mm/hmm.o
CC drivers/acpi/acpica/rsserial.o
AR drivers/media/platform/microchip/built-in.a
CC net/netfilter/nf_conntrack_sip.o
AR drivers/media/platform/nuvoton/built-in.a
CC drivers/gpu/drm/drm_fourcc.o
AR drivers/media/platform/nvidia/tegra-vde/built-in.a
AR drivers/media/pci/ttpci/built-in.a
AR drivers/media/platform/nvidia/built-in.a
AR drivers/media/pci/b2c2/built-in.a
AR drivers/media/platform/nxp/dw100/built-in.a
AR drivers/media/pci/pluto2/built-in.a
AR drivers/media/platform/nxp/imx-jpeg/built-in.a
AR drivers/media/pci/dm1105/built-in.a
CC drivers/input/mouse/focaltech.o
AR drivers/media/platform/nxp/imx8-isi/built-in.a
AR drivers/media/platform/nxp/built-in.a
CC drivers/base/module.o
AR drivers/media/pci/pt1/built-in.a
CC drivers/gpu/drm/drm_framebuffer.o
AR drivers/media/pci/pt3/built-in.a
CC drivers/gpu/drm/drm_gem.o
AR drivers/media/platform/qcom/camss/built-in.a
AR drivers/media/pci/mantis/built-in.a
AR drivers/media/pci/ngene/built-in.a
CC mm/memfd.o
AR drivers/media/platform/qcom/venus/built-in.a
AR drivers/media/platform/qcom/built-in.a
AR drivers/media/pci/ddbridge/built-in.a
AR drivers/media/pci/saa7146/built-in.a
AR drivers/media/pci/smipcie/built-in.a
AR drivers/media/platform/renesas/rcar-vin/built-in.a
CC arch/x86/kernel/io_delay.o
AR drivers/media/platform/rockchip/rga/built-in.a
AR drivers/media/pci/netup_unidvb/built-in.a
AR drivers/media/platform/renesas/rzg2l-cru/built-in.a
AR drivers/media/platform/rockchip/rkisp1/built-in.a
CC net/ipv4/arp.o
AR drivers/i2c/busses/built-in.a
AR drivers/media/pci/intel/ipu3/built-in.a
AR drivers/media/platform/renesas/vsp1/built-in.a
AR drivers/media/platform/rockchip/built-in.a
AR drivers/media/platform/renesas/built-in.a
CC net/ipv4/icmp.o
AR drivers/i2c/muxes/built-in.a
AR drivers/media/pci/intel/ivsc/built-in.a
CC net/ipv4/devinet.o
AR drivers/media/pci/intel/built-in.a
CC drivers/i2c/i2c-boardinfo.o
AR drivers/media/pci/built-in.a
AR drivers/media/platform/samsung/exynos-gsc/built-in.a
CC drivers/gpu/drm/drm_ioctl.o
AR drivers/media/platform/samsung/exynos4-is/built-in.a
CC drivers/gpu/drm/i915/gt/intel_ggtt.o
AR drivers/media/platform/samsung/s3c-camif/built-in.a
CC drivers/gpu/drm/drm_lease.o
CC drivers/acpi/acpica/rsutils.o
AR drivers/media/platform/samsung/s5p-g2d/built-in.a
CC kernel/uid16.o
AR drivers/media/platform/samsung/s5p-jpeg/built-in.a
CC kernel/kallsyms.o
AR drivers/media/platform/samsung/s5p-mfc/built-in.a
CC lib/iomap_copy.o
CC drivers/rtc/nvmem.o
AR drivers/media/platform/samsung/built-in.a
AR drivers/pps/clients/built-in.a
CC drivers/gpu/drm/drm_managed.o
AR drivers/pps/generators/built-in.a
AR drivers/media/platform/st/sti/bdisp/built-in.a
CC drivers/pps/pps.o
AR drivers/media/platform/st/sti/c8sectpfe/built-in.a
CC net/ipv4/af_inet.o
AR drivers/media/platform/st/sti/delta/built-in.a
AR drivers/media/platform/st/sti/hva/built-in.a
AR drivers/media/platform/st/stm32/built-in.a
CC drivers/base/devcoredump.o
AR drivers/media/platform/st/built-in.a
CC drivers/i2c/i2c-core-base.o
CC lib/devres.o
CC arch/x86/kernel/rtc.o
AR drivers/media/platform/sunxi/sun4i-csi/built-in.a
CC drivers/scsi/scsi_scan.o
AR drivers/media/platform/sunxi/sun6i-csi/built-in.a
AR drivers/usb/mon/built-in.a
AR drivers/media/platform/sunxi/sun6i-mipi-csi2/built-in.a
CC drivers/i2c/i2c-core-smbus.o
AR drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/built-in.a
CC drivers/net/phy/phy_device.o
AR drivers/media/platform/sunxi/sun8i-di/built-in.a
AR drivers/media/platform/sunxi/sun8i-rotate/built-in.a
AR drivers/media/platform/sunxi/built-in.a
AR drivers/media/platform/ti/am437x/built-in.a
CC drivers/input/mouse/alps.o
AR drivers/media/platform/ti/cal/built-in.a
CC drivers/usb/core/message.o
AR drivers/media/platform/ti/vpe/built-in.a
AR drivers/media/platform/verisilicon/built-in.a
CC drivers/usb/host/pci-quirks.o
AR drivers/media/platform/ti/davinci/built-in.a
CC drivers/usb/host/ehci-hcd.o
CC drivers/pcmcia/pcmcia_cis.o
CC drivers/acpi/acpica/rsxface.o
CC drivers/usb/host/ehci-pci.o
AR drivers/media/platform/ti/j721e-csi2rx/built-in.a
AR drivers/media/platform/ti/omap/built-in.a
CC [M] drivers/gpu/drm/xe/xe_guc_db_mgr.o
AR drivers/media/platform/ti/omap3isp/built-in.a
AR drivers/media/platform/ti/built-in.a
CC drivers/usb/host/ohci-hcd.o
AR drivers/media/platform/via/built-in.a
AR drivers/media/platform/xilinx/built-in.a
CC mm/ptdump.o
AR drivers/media/platform/built-in.a
CC drivers/rtc/dev.o
AR drivers/media/usb/b2c2/built-in.a
AR drivers/media/usb/dvb-usb/built-in.a
GEN drivers/scsi/scsi_devinfo_tbl.c
CC drivers/scsi/scsi_devinfo.o
AR drivers/media/usb/dvb-usb-v2/built-in.a
CC drivers/pcmcia/rsrc_mgr.o
AR drivers/media/usb/s2255/built-in.a
CC lib/check_signature.o
AR drivers/media/usb/siano/built-in.a
CC drivers/pcmcia/rsrc_nonstatic.o
AR drivers/media/usb/ttusb-budget/built-in.a
CC drivers/pps/kapi.o
CC [M] drivers/gpu/drm/xe/xe_guc_debugfs.o
AR drivers/media/usb/ttusb-dec/built-in.a
AR drivers/media/usb/built-in.a
CC net/ipv6/exthdrs.o
AR drivers/media/mmc/siano/built-in.a
AR drivers/media/mmc/built-in.a
AR drivers/media/firewire/built-in.a
AR drivers/media/spi/built-in.a
CC kernel/acct.o
AR drivers/media/test-drivers/built-in.a
CC arch/x86/kernel/resource.o
AS arch/x86/kernel/irqflags.o
CC [M] drivers/gpu/drm/xe/xe_guc_hwconfig.o
AR drivers/media/built-in.a
CC lib/interval_tree.o
CC drivers/base/platform-msi.o
CC kernel/vmcore_info.o
CC kernel/elfcorehdr.o
CC drivers/acpi/acpica/tbdata.o
CC drivers/acpi/acpica/tbfadt.o
CC drivers/rtc/proc.o
CC drivers/pps/sysfs.o
CC drivers/gpu/drm/drm_mm.o
CC drivers/ata/libata-pmp.o
CC arch/x86/kernel/static_call.o
CC net/ipv4/igmp.o
CC lib/assoc_array.o
CC mm/execmem.o
CC [M] drivers/gpu/drm/xe/xe_guc_id_mgr.o
CC drivers/base/physical_location.o
CC fs/nfs/nfs3acl.o
AR drivers/net/ethernet/3com/built-in.a
CC drivers/net/ethernet/8390/ne2k-pci.o
AR drivers/net/ethernet/adaptec/built-in.a
AR drivers/net/ethernet/agere/built-in.a
AR drivers/pps/built-in.a
AR drivers/net/usb/built-in.a
CC drivers/base/trace.o
AR drivers/net/wireless/admtek/built-in.a
AR drivers/net/wireless/ath/built-in.a
CC drivers/ptp/ptp_clock.o
AR drivers/net/ethernet/alacritech/built-in.a
AR drivers/net/wireless/atmel/built-in.a
CC drivers/ptp/ptp_chardev.o
CC drivers/ptp/ptp_sysfs.o
AR drivers/net/wireless/broadcom/built-in.a
AR drivers/net/wireless/intel/built-in.a
AR drivers/net/ethernet/alteon/built-in.a
AR drivers/net/ethernet/amazon/built-in.a
AR drivers/net/wireless/intersil/built-in.a
CC drivers/net/mii.o
CC drivers/acpi/acpica/tbfind.o
CC drivers/ptp/ptp_vclock.o
CC drivers/acpi/acpica/tbinstal.o
AR drivers/net/wireless/marvell/built-in.a
AR drivers/net/wireless/mediatek/built-in.a
AR drivers/net/wireless/microchip/built-in.a
CC drivers/rtc/sysfs.o
AR drivers/net/wireless/purelifi/built-in.a
CC fs/nfs/nfs4proc.o
AR drivers/net/wireless/quantenna/built-in.a
AR drivers/net/wireless/ralink/built-in.a
CC net/netfilter/nf_nat_core.o
CC [M] drivers/gpu/drm/xe/xe_guc_klv_helpers.o
AR drivers/net/wireless/realtek/built-in.a
CC drivers/scsi/scsi_sysctl.o
CC arch/x86/kernel/process.o
AR drivers/net/wireless/rsi/built-in.a
CC net/netfilter/nf_nat_proto.o
CC drivers/acpi/acpica/tbprint.o
AR drivers/net/wireless/silabs/built-in.a
AR drivers/net/wireless/st/built-in.a
CC drivers/pcmcia/yenta_socket.o
AR drivers/net/wireless/ti/built-in.a
CC drivers/rtc/rtc-mc146818-lib.o
AR drivers/net/wireless/zydas/built-in.a
AR drivers/net/wireless/virtual/built-in.a
AR drivers/net/wireless/built-in.a
CC drivers/net/loopback.o
CC drivers/i2c/i2c-core-acpi.o
CC drivers/net/netconsole.o
AR mm/built-in.a
AR drivers/net/ethernet/amd/built-in.a
CC drivers/gpu/drm/drm_mode_config.o
AR drivers/net/ethernet/aquantia/built-in.a
CC drivers/gpu/drm/drm_mode_object.o
CC net/ipv4/fib_frontend.o
CC drivers/acpi/acpica/tbutils.o
CC drivers/acpi/acpica/tbxface.o
CC drivers/gpu/drm/i915/gt/intel_ggtt_fencing.o
CC kernel/crash_reserve.o
CC lib/bitrev.o
CC drivers/scsi/scsi_proc.o
CC net/ipv4/fib_semantics.o
CC drivers/usb/core/driver.o
CC drivers/input/mouse/byd.o
CC lib/crc-ccitt.o
CC [M] drivers/gpu/drm/xe/xe_guc_log.o
CC drivers/i2c/i2c-smbus.o
AR drivers/net/ethernet/arc/built-in.a
CC drivers/usb/host/ohci-pci.o
CC net/netfilter/nf_nat_helper.o
AR drivers/base/built-in.a
CC drivers/ata/libata-acpi.o
CC kernel/kexec_core.o
CC drivers/rtc/rtc-cmos.o
CC drivers/net/phy/linkmode.o
CC drivers/ata/libata-pata-timings.o
CC drivers/net/phy/mdio_bus.o
CC drivers/ata/ahci.o
CC net/ipv4/fib_trie.o
CC drivers/acpi/acpica/tbxfload.o
CC drivers/power/supply/power_supply_core.o
CC drivers/hwmon/hwmon.o
CC drivers/power/supply/power_supply_sysfs.o
CC drivers/power/supply/power_supply_leds.o
CC drivers/ptp/ptp_kvm_x86.o
CC drivers/net/ethernet/8390/8390.o
CC lib/crc16.o
CC net/ipv6/datagram.o
CC drivers/power/supply/power_supply_hwmon.o
CC drivers/acpi/acpica/tbxfroot.o
CC fs/ioctl.o
CC drivers/usb/class/usblp.o
CC fs/readdir.o
CC drivers/input/mouse/logips2pp.o
CC drivers/scsi/scsi_debugfs.o
CC fs/nfs/nfs4xdr.o
CC drivers/gpu/drm/drm_modes.o
CC drivers/usb/host/uhci-hcd.o
HOSTCC lib/gen_crc32table
CC drivers/ptp/ptp_kvm_common.o
CC drivers/usb/host/xhci.o
CC drivers/usb/host/xhci-mem.o
CC drivers/net/virtio_net.o
AR drivers/i2c/built-in.a
CC drivers/acpi/acpica/utaddress.o
AR drivers/thermal/broadcom/built-in.a
AR drivers/watchdog/built-in.a
AR drivers/thermal/samsung/built-in.a
CC drivers/scsi/scsi_trace.o
CC lib/xxhash.o
CC drivers/thermal/intel/intel_tcc.o
AR drivers/thermal/st/built-in.a
CC drivers/thermal/intel/therm_throt.o
CC [M] drivers/gpu/drm/xe/xe_guc_pc.o
AR drivers/input/tablet/built-in.a
AR drivers/pcmcia/built-in.a
CC [M] drivers/thermal/intel/x86_pkg_temp_thermal.o
CC net/ipv6/ip6_flowlabel.o
CC drivers/ata/libahci.o
CC lib/genalloc.o
CC fs/select.o
CC drivers/usb/core/config.o
CC net/ipv4/fib_notifier.o
CC drivers/gpu/drm/i915/gt/intel_gt.o
AR drivers/power/supply/built-in.a
AR drivers/power/built-in.a
CC drivers/usb/core/file.o
CC drivers/scsi/scsi_logging.o
CC drivers/usb/core/buffer.o
CC arch/x86/kernel/ptrace.o
AR drivers/rtc/built-in.a
CC drivers/net/phy/mdio_device.o
CC drivers/usb/host/xhci-ext-caps.o
CC drivers/acpi/acpica/utalloc.o
CC drivers/input/mouse/lifebook.o
CC kernel/crash_core.o
CC drivers/net/phy/swphy.o
CC drivers/usb/core/sysfs.o
CC net/netfilter/nf_nat_masquerade.o
CC net/mac80211/rx.o
CC drivers/scsi/scsi_pm.o
AR drivers/usb/class/built-in.a
AR drivers/input/touchscreen/built-in.a
CC net/ipv4/inet_fragment.o
AR drivers/net/ethernet/8390/built-in.a
AR drivers/net/ethernet/asix/built-in.a
AR drivers/ptp/built-in.a
AR drivers/net/ethernet/atheros/built-in.a
CC net/ipv4/ping.o
CC fs/dcache.o
CC drivers/ata/ata_piix.o
CC drivers/ata/pata_amd.o
AR drivers/net/ethernet/cadence/built-in.a
CC lib/percpu_counter.o
CC fs/inode.o
CC drivers/net/ethernet/broadcom/bnx2.o
AR drivers/hwmon/built-in.a
CC fs/attr.o
AR drivers/net/ethernet/brocade/built-in.a
CC drivers/net/ethernet/broadcom/tg3.o
CC drivers/acpi/acpica/utascii.o
CC drivers/md/md.o
CC net/netfilter/nf_nat_ftp.o
CC net/ipv4/ip_tunnel_core.o
CC drivers/scsi/scsi_bsg.o
CC drivers/usb/core/endpoint.o
CC drivers/acpi/acpica/utbuffer.o
CC drivers/input/mouse/trackpoint.o
CC drivers/usb/storage/scsiglue.o
AR drivers/thermal/intel/built-in.a
AR drivers/thermal/qcom/built-in.a
CC drivers/usb/storage/protocol.o
AR drivers/thermal/tegra/built-in.a
CC drivers/usb/storage/transport.o
AR drivers/thermal/mediatek/built-in.a
CC drivers/thermal/thermal_core.o
CC lib/audit.o
CC [M] drivers/gpu/drm/xe/xe_guc_submit.o
CC net/netfilter/nf_nat_irc.o
CC drivers/usb/core/devio.o
CC drivers/input/mouse/cypress_ps2.o
CC kernel/kexec.o
CC net/ipv4/gre_offload.o
CC drivers/net/phy/fixed_phy.o
CC arch/x86/kernel/tls.o
CC drivers/gpu/drm/drm_modeset_lock.o
CC drivers/acpi/acpica/utcksum.o
CC drivers/usb/storage/usb.o
CC drivers/usb/core/notify.o
CC net/ipv6/inet6_connection_sock.o
CC drivers/acpi/acpica/utcopy.o
CC drivers/usb/host/xhci-ring.o
CC net/netfilter/nf_nat_sip.o
CC net/ipv4/metrics.o
CC fs/bad_inode.o
AR drivers/net/ethernet/cavium/common/built-in.a
CC lib/syscall.o
AR drivers/net/ethernet/cavium/thunder/built-in.a
CC drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.o
CC drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.o
CC drivers/acpi/acpica/utexcep.o
CC drivers/acpi/acpica/utdebug.o
CC drivers/scsi/scsi_common.o
AR drivers/net/ethernet/cavium/liquidio/built-in.a
AR drivers/net/ethernet/chelsio/built-in.a
CC drivers/md/md-bitmap.o
AR drivers/net/ethernet/cavium/octeon/built-in.a
AR drivers/net/ethernet/cavium/built-in.a
CC drivers/usb/core/generic.o
CC drivers/input/mouse/psmouse-smbus.o
CC drivers/usb/storage/initializers.o
CC drivers/acpi/acpica/utdecode.o
CC drivers/usb/core/quirks.o
CC drivers/thermal/thermal_sysfs.o
CC drivers/ata/pata_oldpiix.o
CC drivers/usb/host/xhci-hub.o
CC drivers/usb/host/xhci-dbg.o
CC drivers/acpi/acpica/utdelete.o
CC fs/file.o
CC drivers/gpu/drm/drm_plane.o
CC kernel/utsname.o
CC kernel/pid_namespace.o
CC fs/nfs/nfs4state.o
CC arch/x86/kernel/step.o
CC drivers/md/md-autodetect.o
CC drivers/ata/pata_sch.o
CC fs/filesystems.o
CC drivers/scsi/scsi_transport_spi.o
AR drivers/usb/misc/built-in.a
CC lib/errname.o
CC drivers/usb/core/devices.o
CC drivers/net/phy/realtek.o
CC lib/nlattr.o
CC lib/cpu_rmap.o
CC net/wireless/shipped-certs.o
CC drivers/ata/pata_mpiix.o
CC drivers/usb/early/ehci-dbgp.o
CC lib/dynamic_queue_limits.o
CC drivers/acpi/acpica/uterror.o
CC fs/nfs/nfs4renewd.o
CC drivers/scsi/virtio_scsi.o
CC net/ipv6/udp_offload.o
CC kernel/stop_machine.o
CC net/ipv6/seg6.o
CC drivers/acpi/acpica/uteval.o
CC net/ipv4/netlink.o
AR drivers/input/mouse/built-in.a
AR drivers/input/misc/built-in.a
CC drivers/thermal/thermal_trip.o
CC drivers/input/input.o
CC drivers/usb/host/xhci-trace.o
CC fs/nfs/nfs4super.o
CC drivers/usb/storage/sierra_ms.o
CC drivers/gpu/drm/i915/gt/intel_gt_clock_utils.o
CC arch/x86/kernel/i8237.o
CC lib/glob.o
CC drivers/usb/core/phy.o
CC drivers/acpi/acpica/utglobal.o
CC drivers/usb/host/xhci-debugfs.o
CC arch/x86/kernel/stacktrace.o
CC net/netfilter/x_tables.o
CC fs/namespace.o
CC net/sunrpc/auth.o
CC kernel/audit.o
CC drivers/ata/ata_generic.o
CC drivers/usb/core/port.o
CC drivers/gpu/drm/i915/gt/intel_gt_debugfs.o
CC drivers/usb/core/hcd-pci.o
CC drivers/usb/core/usb-acpi.o
CC drivers/thermal/thermal_helpers.o
CC lib/strncpy_from_user.o
CC drivers/usb/host/xhci-pci.o
CC drivers/acpi/acpica/uthex.o
CC drivers/acpi/acpica/utids.o
CC fs/nfs/nfs4file.o
CC [M] drivers/gpu/drm/xe/xe_heci_gsc.o
CC [M] drivers/gpu/drm/xe/xe_hw_engine.o
CC arch/x86/kernel/reboot.o
AR drivers/usb/early/built-in.a
CC drivers/cpufreq/cpufreq.o
CC drivers/scsi/sd.o
AR net/wireless/built-in.a
CC fs/seq_file.o
CC drivers/cpufreq/freq_table.o
AR drivers/net/phy/built-in.a
CC fs/nfs/delegation.o
CC net/sunrpc/auth_null.o
CC arch/x86/kernel/msr.o
CC drivers/usb/storage/option_ms.o
CC drivers/acpi/acpica/utinit.o
CC drivers/gpu/drm/i915/gt/intel_gt_engines_debugfs.o
CC drivers/scsi/sr.o
CC drivers/net/net_failover.o
CC arch/x86/kernel/cpuid.o
CC kernel/auditfilter.o
CC drivers/thermal/thermal_hwmon.o
CC net/sunrpc/auth_tls.o
CC net/ipv4/nexthop.o
CC drivers/gpu/drm/i915/gt/intel_gt_irq.o
CC net/ipv6/fib6_notifier.o
CC fs/nfs/nfs4idmap.o
CC lib/strnlen_user.o
CC lib/net_utils.o
CC drivers/acpi/acpica/utlock.o
CC fs/nfs/callback.o
CC arch/x86/kernel/early-quirks.o
AR drivers/ata/built-in.a
CC arch/x86/kernel/smp.o
CC net/sunrpc/auth_unix.o
CC drivers/cpufreq/cpufreq_performance.o
CC arch/x86/kernel/smpboot.o
CC net/ipv6/rpl.o
CC drivers/cpufreq/cpufreq_userspace.o
AR drivers/usb/core/built-in.a
CC [M] drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.o
CC drivers/acpi/acpi_platform.o
CC net/ipv4/udp_tunnel_stub.o
CC drivers/input/input-compat.o
CC drivers/scsi/sr_ioctl.o
CC drivers/thermal/gov_step_wise.o
CC drivers/acpi/acpica/utmath.o
CC net/mac80211/spectmgmt.o
CC net/mac80211/tx.o
CC arch/x86/kernel/tsc_sync.o
CC kernel/auditsc.o
CC drivers/usb/storage/usual-tables.o
CC drivers/cpuidle/governors/menu.o
AR drivers/mmc/built-in.a
CC drivers/cpuidle/governors/haltpoll.o
CC lib/sg_pool.o
CC drivers/cpuidle/cpuidle.o
CC lib/stackdepot.o
CC net/sunrpc/svc.o
CC net/sunrpc/svcsock.o
CC drivers/acpi/acpica/utmisc.o
CC fs/nfs/callback_xdr.o
CC drivers/cpuidle/driver.o
CC drivers/thermal/gov_user_space.o
CC drivers/input/input-mt.o
CC drivers/gpu/drm/drm_prime.o
CC net/netfilter/xt_tcpudp.o
CC drivers/gpu/drm/i915/gt/intel_gt_mcr.o
CC net/netfilter/xt_CONNSECMARK.o
CC net/netfilter/xt_NFLOG.o
CC fs/xattr.o
CC net/sunrpc/svcauth.o
CC arch/x86/kernel/setup_percpu.o
CC drivers/gpu/drm/i915/gt/intel_gt_pm.o
CC drivers/input/input-poller.o
CC drivers/gpu/drm/i915/gt/intel_gt_pm_debugfs.o
AR drivers/usb/host/built-in.a
CC drivers/gpu/drm/i915/gt/intel_gt_pm_irq.o
CC drivers/gpu/drm/i915/gt/intel_gt_requests.o
AR drivers/usb/storage/built-in.a
AR drivers/usb/built-in.a
CC drivers/input/ff-core.o
CC drivers/acpi/acpica/utmutex.o
AR drivers/ufs/built-in.a
CC net/sunrpc/svcauth_unix.o
CC arch/x86/kernel/mpparse.o
CC [M] drivers/gpu/drm/xe/xe_hw_fence.o
CC net/ipv6/ioam6.o
CC lib/asn1_decoder.o
AR drivers/leds/trigger/built-in.a
AR drivers/leds/blink/built-in.a
CC net/ipv4/ip_tunnel.o
CC net/sunrpc/addr.o
AR drivers/leds/simple/built-in.a
CC drivers/leds/led-core.o
CC drivers/acpi/acpica/utnonansi.o
AR drivers/thermal/built-in.a
CC drivers/leds/led-class.o
CC kernel/audit_watch.o
CC drivers/cpuidle/governor.o
AR drivers/firmware/arm_ffa/built-in.a
AR drivers/firmware/arm_scmi/built-in.a
AR drivers/firmware/broadcom/built-in.a
AR drivers/firmware/cirrus/built-in.a
AR drivers/firmware/meson/built-in.a
CC kernel/audit_fsnotify.o
AR drivers/firmware/microchip/built-in.a
CC drivers/cpufreq/cpufreq_ondemand.o
CC drivers/cpufreq/cpufreq_governor.o
GEN lib/oid_registry_data.c
CC drivers/firmware/efi/efi-bgrt.o
CC kernel/audit_tree.o
CC drivers/firmware/efi/libstub/efi-stub-helper.o
CC arch/x86/kernel/trace_clock.o
CC net/mac80211/key.o
CC lib/ucs2_string.o
AR drivers/cpuidle/governors/built-in.a
CC net/mac80211/util.o
CC drivers/leds/led-triggers.o
CC drivers/acpi/acpica/utobject.o
CC net/ipv6/sysctl_net_ipv6.o
CC drivers/firmware/efi/libstub/gop.o
CC net/ipv4/sysctl_net_ipv4.o
CC drivers/input/touchscreen.o
CC net/netfilter/xt_SECMARK.o
CC drivers/input/ff-memless.o
CC net/ipv6/xfrm6_policy.o
CC fs/nfs/callback_proc.o
CC fs/nfs/nfs4namespace.o
CC net/netfilter/xt_TCPMSS.o
CC net/sunrpc/rpcb_clnt.o
CC drivers/scsi/sr_vendor.o
CC drivers/cpuidle/sysfs.o
CC drivers/gpu/drm/i915/gt/intel_gt_sysfs.o
CC drivers/gpu/drm/drm_print.o
CC lib/sbitmap.o
CC kernel/kprobes.o
CC net/ipv4/proc.o
CC drivers/acpi/acpica/utosi.o
CC net/mac80211/parse.o
CC arch/x86/kernel/trace.o
CC net/mac80211/wme.o
CC net/netfilter/xt_conntrack.o
CC drivers/cpufreq/cpufreq_governor_attr_set.o
CC drivers/cpufreq/acpi-cpufreq.o
CC arch/x86/kernel/rethook.o
CC drivers/md/dm.o
CC fs/libfs.o
CC fs/nfs/nfs4getroot.o
CC drivers/md/dm-table.o
CC net/ipv4/fib_rules.o
CC arch/x86/kernel/vmcore_info_32.o
CC net/ipv4/ipmr.o
AR drivers/leds/built-in.a
CC arch/x86/kernel/machine_kexec_32.o
CC kernel/seccomp.o
CC fs/nfs/nfs4client.o
CC [M] drivers/gpu/drm/xe/xe_huc.o
AR drivers/crypto/stm32/built-in.a
AR drivers/crypto/xilinx/built-in.a
AR drivers/crypto/hisilicon/built-in.a
AR drivers/crypto/intel/keembay/built-in.a
AR drivers/crypto/intel/ixp4xx/built-in.a
AR drivers/crypto/intel/built-in.a
CC drivers/firmware/efi/libstub/secureboot.o
CC drivers/cpuidle/poll_state.o
AR drivers/crypto/starfive/built-in.a
CC drivers/acpi/acpica/utownerid.o
CC drivers/input/sparse-keymap.o
AR drivers/crypto/built-in.a
CC drivers/firmware/efi/libstub/tpm.o
CC net/netfilter/xt_policy.o
CC drivers/scsi/sg.o
CC lib/group_cpus.o
CC net/ipv4/ipmr_base.o
CC drivers/cpuidle/cpuidle-haltpoll.o
CC net/ipv4/syncookies.o
CC drivers/cpufreq/amd-pstate.o
CC net/ipv4/tunnel4.o
CC drivers/md/dm-target.o
CC net/mac80211/chan.o
CC net/netfilter/xt_state.o
CC drivers/acpi/acpica/utpredef.o
AS arch/x86/kernel/relocate_kernel_32.o
CC lib/fw_table.o
CC net/ipv6/xfrm6_state.o
CC drivers/gpu/drm/i915/gt/intel_gt_sysfs_pm.o
CC fs/nfs/nfs4session.o
CC drivers/input/vivaldi-fmap.o
CC drivers/md/dm-linear.o
AR drivers/net/ethernet/cisco/built-in.a
CC arch/x86/kernel/crash_dump_32.o
AR drivers/net/ethernet/cortina/built-in.a
CC arch/x86/kernel/crash.o
CC arch/x86/kernel/module.o
AR drivers/net/ethernet/dec/tulip/built-in.a
CC drivers/cpufreq/amd-pstate-trace.o
AR drivers/net/ethernet/dec/built-in.a
CC net/ipv4/ipconfig.o
AR drivers/cpuidle/built-in.a
AR lib/lib.a
CC net/sunrpc/timer.o
CC drivers/clocksource/acpi_pm.o
CC drivers/clocksource/i8253.o
CC drivers/acpi/acpica/utresdecode.o
CC drivers/firmware/efi/libstub/file.o
CC arch/x86/kernel/doublefault_32.o
CC drivers/firmware/efi/libstub/mem.o
GEN lib/crc32table.h
CC fs/nfs/dns_resolve.o
CC [M] drivers/gpu/drm/xe/xe_huc_debugfs.o
CC drivers/hid/usbhid/hid-core.o
CC [M] drivers/gpu/drm/xe/xe_irq.o
CC lib/oid_registry.o
CC drivers/input/input-leds.o
CC drivers/hid/usbhid/hiddev.o
CC kernel/relay.o
CC drivers/acpi/acpica/utresrc.o
CC drivers/scsi/scsi_sysfs.o
AR drivers/firmware/imx/built-in.a
AR drivers/firmware/psci/built-in.a
CC [M] net/netfilter/nf_log_syslog.o
AR drivers/firmware/qcom/built-in.a
AR drivers/firmware/smccc/built-in.a
CC fs/nfs/nfs4trace.o
CC kernel/utsname_sysctl.o
AR drivers/platform/surface/built-in.a
AR drivers/platform/x86/amd/built-in.a
CC fs/nfs/nfs4sysctl.o
AR drivers/platform/x86/intel/built-in.a
CC drivers/platform/x86/wmi.o
CC drivers/mailbox/mailbox.o
AR drivers/firmware/tegra/built-in.a
AR drivers/perf/built-in.a
CC net/sunrpc/xdr.o
CC drivers/platform/x86/wmi-bmof.o
CC arch/x86/kernel/early_printk.o
CC arch/x86/kernel/hpet.o
CC fs/fs-writeback.o
AR drivers/hwtracing/intel_th/built-in.a
CC lib/crc32.o
CC fs/pnode.o
CC drivers/acpi/acpica/utstate.o
CC drivers/hid/hid-core.o
CC [M] net/netfilter/xt_mark.o
AR drivers/clocksource/built-in.a
CC drivers/acpi/acpica/utstring.o
CC drivers/firmware/efi/libstub/random.o
CC [M] net/netfilter/xt_nat.o
CC net/ipv6/xfrm6_input.o
AR drivers/android/built-in.a
AR drivers/net/ethernet/dlink/built-in.a
CC kernel/delayacct.o
CC drivers/cpufreq/intel_pstate.o
CC arch/x86/kernel/amd_nb.o
CC drivers/gpu/drm/drm_property.o
CC drivers/input/evdev.o
AR drivers/nvmem/layouts/built-in.a
CC drivers/nvmem/core.o
CC arch/x86/kernel/kvm.o
CC drivers/platform/x86/eeepc-laptop.o
CC drivers/gpu/drm/i915/gt/intel_gtt.o
CC net/ipv6/xfrm6_output.o
AR drivers/net/ethernet/emulex/built-in.a
CC drivers/md/dm-stripe.o
AR drivers/firmware/xilinx/built-in.a
CC drivers/firmware/dmi_scan.o
CC drivers/acpi/acpica/utstrsuppt.o
CC drivers/gpu/drm/i915/gt/intel_llc.o
CC drivers/hid/usbhid/hid-pidff.o
CC net/ipv6/xfrm6_protocol.o
CC net/mac80211/trace.o
CC drivers/mailbox/pcc.o
CC [M] drivers/gpu/drm/xe/xe_lrc.o
CC drivers/gpu/drm/i915/gt/intel_lrc.o
CC drivers/firmware/efi/efi.o
CC kernel/taskstats.o
AR lib/built-in.a
CC kernel/tsacct.o
CC net/ipv6/netfilter.o
CC net/sunrpc/sunrpc_syms.o
CC drivers/gpu/drm/i915/gt/intel_migrate.o
CC fs/splice.o
CC kernel/tracepoint.o
CC drivers/platform/x86/p2sb.o
CC drivers/firmware/efi/libstub/randomalloc.o
CC drivers/acpi/acpica/utstrtoul64.o
CC net/ipv4/netfilter.o
CC net/ipv4/tcp_cubic.o
CC fs/sync.o
CC net/ipv6/proc.o
CC drivers/gpu/drm/i915/gt/intel_mocs.o
CC arch/x86/kernel/kvmclock.o
CC drivers/hid/hid-input.o
AR drivers/scsi/built-in.a
CC drivers/firmware/dmi-id.o
CC net/mac80211/mlme.o
AR drivers/mailbox/built-in.a
AR drivers/net/ethernet/engleder/built-in.a
CC net/mac80211/tdls.o
CC drivers/acpi/acpi_pnp.o
CC drivers/acpi/acpica/utxface.o
CC drivers/md/dm-ioctl.o
CC drivers/acpi/power.o
CC net/mac80211/ocb.o
AR drivers/input/built-in.a
CC net/mac80211/airtime.o
AR drivers/nvmem/built-in.a
CC fs/utimes.o
CC drivers/gpu/drm/i915/gt/intel_ppgtt.o
CC fs/d_path.o
CC [M] net/netfilter/xt_LOG.o
CC drivers/gpu/drm/i915/gt/intel_rc6.o
AR drivers/platform/x86/built-in.a
AR drivers/platform/built-in.a
CC drivers/gpu/drm/i915/gt/intel_region_lmem.o
CC drivers/firmware/efi/libstub/pci.o
AR drivers/hid/usbhid/built-in.a
CC drivers/acpi/acpica/utxfinit.o
CC drivers/gpu/drm/i915/gt/intel_renderstate.o
CC drivers/gpu/drm/i915/gt/intel_reset.o
CC drivers/gpu/drm/i915/gt/intel_ring.o
CC drivers/acpi/event.o
CC kernel/irq_work.o
CC arch/x86/kernel/paravirt.o
CC drivers/gpu/drm/drm_syncobj.o
AR drivers/net/ethernet/broadcom/built-in.a
AR drivers/net/ethernet/ezchip/built-in.a
AR drivers/net/ethernet/fujitsu/built-in.a
CC net/sunrpc/cache.o
CC net/ipv6/syncookies.o
AR drivers/net/ethernet/fungible/built-in.a
CC fs/stack.o
CC drivers/firmware/efi/vars.o
AR drivers/net/ethernet/google/built-in.a
CC drivers/firmware/efi/reboot.o
AR drivers/net/ethernet/huawei/built-in.a
AR drivers/net/ethernet/i825xx/built-in.a
CC drivers/net/ethernet/intel/e1000/e1000_main.o
CC drivers/net/ethernet/intel/e1000/e1000_hw.o
CC fs/fs_struct.o
CC [M] net/netfilter/xt_MASQUERADE.o
CC drivers/net/ethernet/intel/e1000/e1000_ethtool.o
CC drivers/acpi/acpica/utxferror.o
CC drivers/acpi/evged.o
CC drivers/gpu/drm/i915/gt/intel_ring_submission.o
CC drivers/gpu/drm/i915/gt/intel_rps.o
CC fs/statfs.o
CC drivers/gpu/drm/i915/gt/intel_sa_media.o
CC drivers/gpu/drm/i915/gt/intel_sseu.o
CC net/mac80211/eht.o
CC net/ipv4/tcp_sigpool.o
CC fs/fs_pin.o
AR drivers/cpufreq/built-in.a
CC fs/nsfs.o
CC drivers/acpi/sysfs.o
CC drivers/firmware/efi/libstub/skip_spaces.o
CC kernel/static_call.o
CC drivers/acpi/acpica/utxfmutex.o
CC fs/fs_types.o
CC net/mac80211/led.o
CC drivers/firmware/efi/memattr.o
CC drivers/firmware/efi/libstub/lib-cmdline.o
CC drivers/acpi/property.o
CC drivers/hid/hid-quirks.o
CC net/mac80211/pm.o
CC arch/x86/kernel/pvclock.o
CC arch/x86/kernel/pcspeaker.o
CC drivers/firmware/efi/libstub/lib-ctype.o
CC drivers/firmware/efi/libstub/alignedmem.o
CC drivers/gpu/drm/i915/gt/intel_sseu_debugfs.o
CC drivers/acpi/debugfs.o
CC drivers/acpi/acpi_lpat.o
CC [M] net/netfilter/xt_addrtype.o
CC drivers/gpu/drm/i915/gt/intel_timeline.o
CC drivers/hid/hid-debug.o
CC kernel/padata.o
CC drivers/net/ethernet/intel/e1000/e1000_param.o
AR drivers/acpi/acpica/built-in.a
CC fs/fs_context.o
CC drivers/acpi/acpi_pcc.o
CC drivers/gpu/drm/drm_sysfs.o
CC drivers/firmware/memmap.o
CC arch/x86/kernel/check.o
CC drivers/firmware/efi/libstub/relocate.o
CC fs/fs_parser.o
CC net/sunrpc/rpc_pipe.o
CC drivers/md/dm-io.o
CC drivers/md/dm-kcopyd.o
CC net/sunrpc/sysfs.o
CC net/ipv6/calipso.o
AR drivers/net/ethernet/microsoft/built-in.a
CC net/ipv6/ah6.o
CC drivers/acpi/ac.o
CC net/ipv6/esp6.o
CC drivers/hid/hidraw.o
CC arch/x86/kernel/uprobes.o
CC net/ipv4/cipso_ipv4.o
CC net/sunrpc/svc_xprt.o
CC drivers/acpi/button.o
CC arch/x86/kernel/perf_regs.o
AR fs/nfs/built-in.a
CC drivers/hid/hid-generic.o
CC drivers/net/ethernet/intel/e1000e/82571.o
CC fs/fsopen.o
CC drivers/net/ethernet/intel/e1000e/ich8lan.o
CC drivers/net/ethernet/intel/e100.o
CC net/mac80211/rc80211_minstrel_ht.o
CC arch/x86/kernel/tracepoint.o
CC net/ipv6/sit.o
CC net/mac80211/wbrf.o
CC drivers/firmware/efi/libstub/printk.o
CC kernel/jump_label.o
CC drivers/gpu/drm/i915/gt/intel_tlb.o
CC drivers/acpi/fan_core.o
CC drivers/acpi/fan_attr.o
CC net/ipv4/xfrm4_policy.o
CC drivers/acpi/acpi_video.o
CC drivers/acpi/video_detect.o
CC drivers/gpu/drm/i915/gt/intel_wopcm.o
CC drivers/acpi/processor_driver.o
CC drivers/gpu/drm/i915/gt/intel_workarounds.o
CC drivers/acpi/processor_thermal.o
CC drivers/gpu/drm/drm_trace_points.o
CC fs/init.o
CC [M] drivers/gpu/drm/xe/xe_migrate.o
CC fs/kernel_read_file.o
CC arch/x86/kernel/itmt.o
AR drivers/net/ethernet/litex/built-in.a
AR net/netfilter/built-in.a
AR drivers/net/ethernet/marvell/octeon_ep/built-in.a
CC net/ipv4/xfrm4_state.o
AR drivers/net/ethernet/marvell/octeon_ep_vf/built-in.a
AR drivers/net/ethernet/marvell/octeontx2/built-in.a
CC net/ipv4/xfrm4_input.o
CC net/ipv6/addrconf_core.o
CC fs/mnt_idmapping.o
AR drivers/net/ethernet/marvell/prestera/built-in.a
CC kernel/context_tracking.o
CC drivers/net/ethernet/marvell/sky2.o
CC net/ipv4/xfrm4_output.o
CC drivers/hid/hid-a4tech.o
CC drivers/md/dm-sysfs.o
CC drivers/gpu/drm/i915/gt/shmem_utils.o
CC drivers/acpi/processor_idle.o
CC drivers/md/dm-stats.o
CC net/ipv6/exthdrs_core.o
CC drivers/firmware/efi/libstub/vsprintf.o
CC drivers/md/dm-rq.o
CC drivers/firmware/efi/libstub/x86-stub.o
STUBCPY drivers/firmware/efi/libstub/alignedmem.stub.o
CC net/ipv6/ip6_checksum.o
CC kernel/iomem.o
CC drivers/acpi/processor_throttling.o
CC drivers/acpi/processor_perflib.o
CC net/ipv6/ip6_icmp.o
AR drivers/net/ethernet/mellanox/built-in.a
CC arch/x86/kernel/umip.o
STUBCPY drivers/firmware/efi/libstub/efi-stub-helper.stub.o
CC net/ipv4/xfrm4_protocol.o
CC net/ipv6/output_core.o
CC net/sunrpc/xprtmultipath.o
CC net/sunrpc/stats.o
CC kernel/rseq.o
CC fs/remap_range.o
CC fs/pidfs.o
CC [M] drivers/gpu/drm/xe/xe_mmio.o
CC drivers/net/ethernet/intel/e1000e/80003es2lan.o
CC drivers/md/dm-io-rewind.o
CC drivers/gpu/drm/i915/gt/sysfs_engines.o
CC drivers/hid/hid-apple.o
CC drivers/hid/hid-belkin.o
AR drivers/net/ethernet/intel/e1000/built-in.a
CC drivers/hid/hid-cherry.o
CC drivers/hid/hid-chicony.o
CC drivers/hid/hid-cypress.o
CC drivers/net/ethernet/intel/e1000e/mac.o
CC [M] drivers/gpu/drm/xe/xe_mocs.o
CC net/sunrpc/sysctl.o
CC drivers/acpi/container.o
CC drivers/acpi/thermal_lib.o
CC drivers/firmware/efi/tpm.o
CC drivers/gpu/drm/drm_vblank.o
CC net/ipv6/protocol.o
CC drivers/md/dm-builtin.o
AR drivers/net/ethernet/micrel/built-in.a
STUBCPY drivers/firmware/efi/libstub/file.stub.o
CC drivers/acpi/thermal.o
STUBCPY drivers/firmware/efi/libstub/gop.stub.o
CC drivers/md/dm-raid1.o
STUBCPY drivers/firmware/efi/libstub/lib-cmdline.stub.o
STUBCPY drivers/firmware/efi/libstub/lib-ctype.stub.o
CC drivers/gpu/drm/drm_vblank_work.o
CC fs/buffer.o
STUBCPY drivers/firmware/efi/libstub/mem.stub.o
CC fs/mpage.o
STUBCPY drivers/firmware/efi/libstub/pci.stub.o
CC fs/proc_namespace.o
STUBCPY drivers/firmware/efi/libstub/printk.stub.o
STUBCPY drivers/firmware/efi/libstub/random.stub.o
STUBCPY drivers/firmware/efi/libstub/randomalloc.stub.o
CC net/ipv6/ip6_offload.o
STUBCPY drivers/firmware/efi/libstub/relocate.stub.o
STUBCPY drivers/firmware/efi/libstub/secureboot.stub.o
STUBCPY drivers/firmware/efi/libstub/skip_spaces.stub.o
STUBCPY drivers/firmware/efi/libstub/tpm.stub.o
CC arch/x86/kernel/unwind_frame.o
STUBCPY drivers/firmware/efi/libstub/vsprintf.stub.o
CC drivers/md/dm-log.o
CC drivers/gpu/drm/i915/gt/intel_ggtt_gmch.o
STUBCPY drivers/firmware/efi/libstub/x86-stub.stub.o
CC drivers/acpi/nhlt.o
CC drivers/gpu/drm/drm_vma_manager.o
AR drivers/firmware/efi/libstub/lib.a
CC drivers/acpi/acpi_memhotplug.o
CC drivers/firmware/efi/memmap.o
CC drivers/net/ethernet/intel/e1000e/manage.o
CC drivers/hid/hid-ezkey.o
AR kernel/built-in.a
CC drivers/acpi/ioapic.o
CC net/ipv6/tcpv6_offload.o
CC drivers/md/dm-region-hash.o
CC drivers/hid/hid-gyration.o
CC drivers/md/dm-zero.o
CC drivers/hid/hid-ite.o
CC drivers/net/ethernet/intel/e1000e/nvm.o
CC fs/direct-io.o
CC [M] drivers/gpu/drm/xe/xe_module.o
AR net/ipv4/built-in.a
CC [M] drivers/gpu/drm/xe/xe_pat.o
CC drivers/hid/hid-kensington.o
CC [M] drivers/gpu/drm/xe/xe_pci.o
CC fs/eventpoll.o
CC fs/anon_inodes.o
CC [M] drivers/gpu/drm/xe/xe_pcode.o
CC drivers/hid/hid-lg.o
CC drivers/acpi/battery.o
CC drivers/gpu/drm/i915/gt/gen6_renderstate.o
AR drivers/net/ethernet/microchip/built-in.a
AR drivers/net/ethernet/mscc/built-in.a
AR drivers/net/ethernet/myricom/built-in.a
AR drivers/net/ethernet/natsemi/built-in.a
CC drivers/gpu/drm/drm_writeback.o
AR drivers/net/ethernet/neterion/built-in.a
AR drivers/net/ethernet/netronome/built-in.a
CC drivers/net/ethernet/intel/e1000e/phy.o
CC drivers/acpi/bgrt.o
CC drivers/gpu/drm/i915/gt/gen7_renderstate.o
AR drivers/net/ethernet/ni/built-in.a
CC drivers/gpu/drm/i915/gt/gen8_renderstate.o
CC drivers/net/ethernet/intel/e1000e/param.o
CC net/ipv6/exthdrs_offload.o
AR arch/x86/kernel/built-in.a
AR arch/x86/built-in.a
CC drivers/net/ethernet/intel/e1000e/ethtool.o
CC drivers/firmware/efi/capsule.o
CC drivers/firmware/efi/esrt.o
CC net/ipv6/inet6_hashtables.o
CC drivers/hid/hid-lgff.o
CC drivers/firmware/efi/runtime-wrappers.o
CC fs/signalfd.o
CC drivers/net/ethernet/intel/e1000e/netdev.o
CC drivers/hid/hid-lg4ff.o
CC drivers/hid/hid-lg-g15.o
CC [M] drivers/gpu/drm/xe/xe_pm.o
CC fs/timerfd.o
CC fs/eventfd.o
CC drivers/acpi/spcr.o
CC drivers/hid/hid-microsoft.o
CC drivers/net/ethernet/intel/e1000e/ptp.o
CC drivers/gpu/drm/i915/gt/gen9_renderstate.o
CC drivers/net/ethernet/nvidia/forcedeth.o
CC drivers/gpu/drm/i915/gem/i915_gem_busy.o
CC fs/aio.o
CC [M] drivers/gpu/drm/xe/xe_preempt_fence.o
CC [M] drivers/gpu/drm/xe/xe_pt.o
CC drivers/gpu/drm/drm_panel.o
CC [M] drivers/gpu/drm/xe/xe_pt_walk.o
AR drivers/md/built-in.a
CC [M] drivers/gpu/drm/xe/xe_query.o
CC drivers/firmware/efi/capsule-loader.o
CC fs/locks.o
CC net/ipv6/mcast_snoop.o
CC [M] drivers/gpu/drm/xe/xe_range_fence.o
CC drivers/gpu/drm/drm_pci.o
CC drivers/gpu/drm/i915/gem/i915_gem_clflush.o
CC drivers/gpu/drm/i915/gem/i915_gem_context.o
CC drivers/gpu/drm/drm_debugfs.o
CC [M] drivers/gpu/drm/xe/xe_reg_sr.o
CC drivers/firmware/efi/earlycon.o
CC fs/binfmt_misc.o
CC drivers/gpu/drm/i915/gem/i915_gem_create.o
CC drivers/hid/hid-monterey.o
CC drivers/hid/hid-ntrig.o
CC drivers/gpu/drm/drm_debugfs_crc.o
CC drivers/gpu/drm/drm_panel_orientation_quirks.o
CC drivers/gpu/drm/drm_buddy.o
AR drivers/net/ethernet/marvell/built-in.a
CC [M] drivers/gpu/drm/xe/xe_reg_whitelist.o
AR drivers/net/ethernet/oki-semi/built-in.a
AR net/sunrpc/built-in.a
CC drivers/hid/hid-pl.o
CC drivers/hid/hid-petalynx.o
CC drivers/gpu/drm/drm_gem_shmem_helper.o
CC [M] drivers/gpu/drm/xe/xe_rtp.o
AR drivers/acpi/built-in.a
CC [M] drivers/gpu/drm/xe/xe_ring_ops.o
CC [M] drivers/gpu/drm/xe/xe_sa.o
CC [M] drivers/gpu/drm/xe/xe_sched_job.o
CC [M] drivers/gpu/drm/xe/xe_step.o
CC drivers/hid/hid-redragon.o
CC drivers/gpu/drm/drm_atomic_helper.o
CC fs/binfmt_script.o
CC fs/binfmt_elf.o
CC fs/mbcache.o
CC drivers/hid/hid-samsung.o
CC [M] drivers/gpu/drm/xe/xe_sync.o
CC drivers/gpu/drm/drm_atomic_state_helper.o
CC drivers/hid/hid-sony.o
CC drivers/hid/hid-sunplus.o
CC [M] drivers/gpu/drm/xe/xe_tile.o
CC drivers/gpu/drm/drm_bridge_connector.o
CC drivers/gpu/drm/i915/gem/i915_gem_dmabuf.o
CC drivers/gpu/drm/i915/gem/i915_gem_domain.o
CC drivers/gpu/drm/drm_crtc_helper.o
CC drivers/hid/hid-topseed.o
CC fs/posix_acl.o
CC [M] drivers/gpu/drm/xe/xe_tile_sysfs.o
CC drivers/gpu/drm/i915/gem/i915_gem_execbuffer.o
AR drivers/firmware/efi/built-in.a
CC drivers/gpu/drm/i915/gem/i915_gem_internal.o
AR drivers/firmware/built-in.a
CC drivers/gpu/drm/i915/gem/i915_gem_lmem.o
AR net/ipv6/built-in.a
CC drivers/gpu/drm/i915/gem/i915_gem_mman.o
CC [M] drivers/gpu/drm/xe/xe_trace.o
AR drivers/net/ethernet/packetengines/built-in.a
CC drivers/gpu/drm/drm_damage_helper.o
AR drivers/net/ethernet/qlogic/built-in.a
AR drivers/net/ethernet/qualcomm/emac/built-in.a
AR drivers/net/ethernet/qualcomm/built-in.a
CC drivers/gpu/drm/drm_encoder_slave.o
CC drivers/net/ethernet/realtek/8139too.o
CC drivers/net/ethernet/realtek/r8169_main.o
AR drivers/net/ethernet/renesas/built-in.a
CC drivers/net/ethernet/realtek/r8169_firmware.o
CC drivers/gpu/drm/drm_flip_work.o
CC drivers/net/ethernet/realtek/r8169_phy_config.o
CC fs/coredump.o
CC fs/drop_caches.o
CC drivers/gpu/drm/drm_format_helper.o
CC fs/sysctls.o
CC drivers/gpu/drm/i915/gem/i915_gem_object.o
CC fs/fhandle.o
CC [M] drivers/gpu/drm/xe/xe_trace_bo.o
CC drivers/gpu/drm/i915/gem/i915_gem_pages.o
CC [M] drivers/gpu/drm/xe/xe_trace_guc.o
CC drivers/gpu/drm/drm_gem_atomic_helper.o
CC drivers/gpu/drm/drm_gem_framebuffer_helper.o
CC drivers/gpu/drm/drm_kms_helper_common.o
CC drivers/gpu/drm/drm_modeset_helper.o
CC drivers/gpu/drm/i915/gem/i915_gem_phys.o
CC drivers/gpu/drm/drm_plane_helper.o
CC drivers/gpu/drm/i915/gem/i915_gem_pm.o
CC [M] drivers/gpu/drm/xe/xe_ttm_sys_mgr.o
CC drivers/gpu/drm/drm_probe_helper.o
CC [M] drivers/gpu/drm/xe/xe_ttm_stolen_mgr.o
CC drivers/gpu/drm/drm_rect.o
CC [M] drivers/gpu/drm/xe/xe_ttm_vram_mgr.o
CC [M] drivers/gpu/drm/xe/xe_tuning.o
CC [M] drivers/gpu/drm/xe/xe_uc.o
CC [M] drivers/gpu/drm/xe/xe_uc_debugfs.o
CC [M] drivers/gpu/drm/xe/xe_uc_fw.o
CC drivers/gpu/drm/drm_self_refresh_helper.o
CC drivers/gpu/drm/drm_simple_kms_helper.o
CC drivers/gpu/drm/bridge/panel.o
AR drivers/net/ethernet/rdc/built-in.a
AR drivers/net/ethernet/rocker/built-in.a
AR drivers/net/ethernet/samsung/built-in.a
CC drivers/gpu/drm/drm_mipi_dsi.o
AR drivers/net/ethernet/seeq/built-in.a
CC [M] drivers/gpu/drm/drm_exec.o
AR net/mac80211/built-in.a
AR drivers/hid/built-in.a
CC [M] drivers/gpu/drm/drm_gpuvm.o
AR net/built-in.a
CC [M] drivers/gpu/drm/xe/xe_vm.o
CC drivers/gpu/drm/i915/gem/i915_gem_region.o
CC drivers/gpu/drm/i915/gem/i915_gem_shmem.o
CC [M] drivers/gpu/drm/drm_suballoc.o
CC drivers/gpu/drm/i915/gem/i915_gem_shrinker.o
CC [M] drivers/gpu/drm/drm_gem_ttm_helper.o
CC drivers/gpu/drm/i915/gem/i915_gem_stolen.o
CC drivers/gpu/drm/i915/gem/i915_gem_throttle.o
CC [M] drivers/gpu/drm/xe/xe_vram.o
CC [M] drivers/gpu/drm/xe/xe_vram_freq.o
CC [M] drivers/gpu/drm/xe/xe_wait_user_fence.o
CC [M] drivers/gpu/drm/xe/xe_wa.o
CC drivers/gpu/drm/i915/gem/i915_gem_tiling.o
CC drivers/gpu/drm/i915/gem/i915_gem_ttm.o
CC [M] drivers/gpu/drm/xe/xe_wopcm.o
CC [M] drivers/gpu/drm/xe/xe_hmm.o
CC [M] drivers/gpu/drm/xe/xe_hwmon.o
CC drivers/gpu/drm/i915/gem/i915_gem_ttm_move.o
CC drivers/gpu/drm/i915/gem/i915_gem_ttm_pm.o
CC drivers/gpu/drm/i915/gem/i915_gem_userptr.o
CC [M] drivers/gpu/drm/xe/xe_gt_sriov_vf.o
CC [M] drivers/gpu/drm/xe/xe_gt_sriov_vf_debugfs.o
CC [M] drivers/gpu/drm/xe/xe_guc_relay.o
CC drivers/gpu/drm/i915/gem/i915_gem_wait.o
CC drivers/gpu/drm/i915/gem/i915_gemfs.o
LD [M] drivers/gpu/drm/drm_suballoc_helper.o
CC [M] drivers/gpu/drm/xe/xe_memirq.o
CC drivers/gpu/drm/i915/i915_active.o
CC [M] drivers/gpu/drm/xe/xe_sriov.o
CC drivers/gpu/drm/i915/i915_cmd_parser.o
CC drivers/gpu/drm/i915/i915_deps.o
CC drivers/gpu/drm/i915/i915_gem.o
CC [M] drivers/gpu/drm/xe/display/ext/i915_irq.o
CC drivers/gpu/drm/i915/i915_gem_evict.o
AR drivers/net/ethernet/silan/built-in.a
CC [M] drivers/gpu/drm/xe/display/ext/i915_utils.o
CC [M] drivers/gpu/drm/xe/display/intel_fb_bo.o
LD [M] drivers/gpu/drm/drm_ttm_helper.o
AR drivers/net/ethernet/sis/built-in.a
CC drivers/gpu/drm/i915/i915_gem_gtt.o
CC [M] drivers/gpu/drm/xe/display/intel_fbdev_fb.o
AR drivers/net/ethernet/sfc/built-in.a
CC [M] drivers/gpu/drm/xe/display/xe_display.o
AR drivers/net/ethernet/nvidia/built-in.a
CC [M] drivers/gpu/drm/xe/display/xe_display_misc.o
AR drivers/net/ethernet/smsc/built-in.a
CC [M] drivers/gpu/drm/xe/display/xe_display_rps.o
AR drivers/net/ethernet/socionext/built-in.a
AR drivers/net/ethernet/stmicro/built-in.a
CC drivers/gpu/drm/i915/i915_gem_ww.o
AR drivers/net/ethernet/sun/built-in.a
AR drivers/net/ethernet/tehuti/built-in.a
CC [M] drivers/gpu/drm/xe/display/xe_dsb_buffer.o
CC [M] drivers/gpu/drm/xe/display/xe_fb_pin.o
AR drivers/net/ethernet/ti/built-in.a
AR drivers/net/ethernet/vertexcom/built-in.a
AR fs/built-in.a
AR drivers/net/ethernet/via/built-in.a
AR drivers/net/ethernet/wangxun/built-in.a
CC drivers/gpu/drm/i915/i915_query.o
AR drivers/net/ethernet/wiznet/built-in.a
AR drivers/net/ethernet/xilinx/built-in.a
AR drivers/net/ethernet/xircom/built-in.a
AR drivers/net/ethernet/pensando/built-in.a
AR drivers/net/ethernet/synopsys/built-in.a
CC drivers/gpu/drm/i915/i915_request.o
CC [M] drivers/gpu/drm/xe/display/xe_hdcp_gsc.o
CC drivers/gpu/drm/i915/i915_scheduler.o
CC [M] drivers/gpu/drm/xe/display/xe_plane_initial.o
In file included from /workspace/kernel/include/drm/drm_mm.h:51,
from /workspace/kernel/drivers/gpu/drm/i915/gem/i915_gem_stolen.c:10:
/workspace/kernel/drivers/gpu/drm/i915/gem/i915_gem_stolen.c: In function ‘i915_gem_stolen_lmem_setup’:
/workspace/kernel/drivers/gpu/drm/i915/gem/i915_gem_stolen.c:941:5: error: format ‘%lli’ expects argument of type ‘long long int’, but argument 5 has type ‘resource_size_t’ {aka ‘unsigned int’} [-Werror=format=]
941 | "Disabling stolen memory support due to OOB placement: lmem_size = %lli vs dsm_base = %lli\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
942 | lmem_size, dsm_base);
| ~~~~~~~~~
| |
| resource_size_t {aka unsigned int}
/workspace/kernel/include/drm/drm_print.h:424:32: note: in definition of macro ‘drm_dev_dbg’
424 | __drm_dev_dbg(NULL, dev, cat, fmt, ##__VA_ARGS__)
| ^~~
/workspace/kernel/include/drm/drm_print.h:524:32: note: in expansion of macro ‘drm_dbg_driver’
524 | #define drm_dbg(drm, fmt, ...) drm_dbg_driver(drm, fmt, ##__VA_ARGS__)
| ^~~~~~~~~~~~~~
/workspace/kernel/drivers/gpu/drm/i915/gem/i915_gem_stolen.c:940:4: note: in expansion of macro ‘drm_dbg’
940 | drm_dbg(&i915->drm,
| ^~~~~~~
/workspace/kernel/drivers/gpu/drm/i915/gem/i915_gem_stolen.c:941:75: note: format string is defined here
941 | "Disabling stolen memory support due to OOB placement: lmem_size = %lli vs dsm_base = %lli\n",
| ~~~^
| |
| long long int
| %i
In file included from /workspace/kernel/include/drm/drm_mm.h:51,
from /workspace/kernel/drivers/gpu/drm/i915/gem/i915_gem_stolen.c:10:
/workspace/kernel/drivers/gpu/drm/i915/gem/i915_gem_stolen.c:941:5: error: format ‘%lli’ expects argument of type ‘long long int’, but argument 6 has type ‘resource_size_t’ {aka ‘unsigned int’} [-Werror=format=]
941 | "Disabling stolen memory support due to OOB placement: lmem_size = %lli vs dsm_base = %lli\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
942 | lmem_size, dsm_base);
| ~~~~~~~~
| |
| resource_size_t {aka unsigned int}
/workspace/kernel/include/drm/drm_print.h:424:32: note: in definition of macro ‘drm_dev_dbg’
424 | __drm_dev_dbg(NULL, dev, cat, fmt, ##__VA_ARGS__)
| ^~~
/workspace/kernel/include/drm/drm_print.h:524:32: note: in expansion of macro ‘drm_dbg_driver’
524 | #define drm_dbg(drm, fmt, ...) drm_dbg_driver(drm, fmt, ##__VA_ARGS__)
| ^~~~~~~~~~~~~~
/workspace/kernel/drivers/gpu/drm/i915/gem/i915_gem_stolen.c:940:4: note: in expansion of macro ‘drm_dbg’
940 | drm_dbg(&i915->drm,
| ^~~~~~~
/workspace/kernel/drivers/gpu/drm/i915/gem/i915_gem_stolen.c:941:94: note: format string is defined here
941 | "Disabling stolen memory support due to OOB placement: lmem_size = %lli vs dsm_base = %lli\n",
| ~~~^
| |
| long long int
| %i
CC [M] drivers/gpu/drm/xe/display/xe_tdf.o
CC [M] drivers/gpu/drm/xe/i915-soc/intel_dram.o
CC [M] drivers/gpu/drm/xe/i915-soc/intel_pch.o
CC drivers/gpu/drm/i915/i915_trace_points.o
CC [M] drivers/gpu/drm/xe/i915-display/icl_dsi.o
CC drivers/gpu/drm/i915/i915_ttm_buddy_manager.o
CC drivers/gpu/drm/i915/i915_vma.o
CC drivers/gpu/drm/i915/i915_vma_resource.o
CC drivers/gpu/drm/i915/gt/uc/intel_gsc_fw.o
CC drivers/gpu/drm/i915/gt/uc/intel_gsc_proxy.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_alpm.o
cc1: all warnings being treated as errors
make[6]: *** [/workspace/kernel/scripts/Makefile.build:244: drivers/gpu/drm/i915/gem/i915_gem_stolen.o] Error 1
make[6]: *** Waiting for unfinished jobs....
CC [M] drivers/gpu/drm/xe/i915-display/intel_atomic.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_atomic_plane.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_audio.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_backlight.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_bios.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_bw.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_cdclk.o
AR drivers/net/ethernet/intel/e1000e/built-in.a
CC [M] drivers/gpu/drm/xe/i915-display/intel_color.o
AR drivers/net/ethernet/intel/built-in.a
CC [M] drivers/gpu/drm/xe/i915-display/intel_combo_phy.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_connector.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_crtc.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_crtc_state_dump.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_cursor.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_cx0_phy.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_ddi.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_ddi_buf_trans.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_display.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_display_device.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_display_driver.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_display_irq.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_display_params.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_display_power.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_display_power_map.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_display_power_well.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_display_trace.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_display_wa.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dkl_phy.o
AR drivers/net/ethernet/realtek/built-in.a
AR drivers/net/ethernet/built-in.a
CC [M] drivers/gpu/drm/xe/i915-display/intel_dmc.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dp_aux.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dp.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dp_aux_backlight.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dp_hdcp.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dp_link_training.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dp_mst.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dpll.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dpll_mgr.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dpt_common.o
AR drivers/net/built-in.a
CC [M] drivers/gpu/drm/xe/i915-display/intel_drrs.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dsb.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dsi.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dsi_dcs_backlight.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dsi_vbt.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_fb.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_fbc.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_fdi.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_fifo_underrun.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_frontbuffer.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_global_state.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_gmbus.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_hdcp.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_hdcp_gsc_message.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_hdmi.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_hotplug.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_hotplug_irq.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_hti.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_link_bw.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_lspcon.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_modeset_lock.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_modeset_setup.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_modeset_verify.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_panel.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_pmdemand.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_pps.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_psr.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_qp_tables.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_quirks.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_snps_phy.o
make[5]: *** [/workspace/kernel/scripts/Makefile.build:485: drivers/gpu/drm/i915] Error 2
make[5]: *** Waiting for unfinished jobs....
CC [M] drivers/gpu/drm/xe/i915-display/intel_tc.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_vblank.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_vdsc.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_vga.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_vrr.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_dmc_wl.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_wm.o
CC [M] drivers/gpu/drm/xe/i915-display/skl_scaler.o
CC [M] drivers/gpu/drm/xe/i915-display/skl_universal_plane.o
CC [M] drivers/gpu/drm/xe/i915-display/skl_watermark.o
CC [M] drivers/gpu/drm/xe/xe_pmu.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_acpi.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_opregion.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_display_debugfs.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_display_debugfs_params.o
CC [M] drivers/gpu/drm/xe/i915-display/intel_pipe_crc.o
CC [M] drivers/gpu/drm/xe/xe_gsc.o
CC [M] drivers/gpu/drm/xe/xe_guc.o
CC [M] drivers/gpu/drm/xe/xe_guc_ads.o
LD [M] drivers/gpu/drm/xe/xe.o
make[4]: *** [/workspace/kernel/scripts/Makefile.build:485: drivers/gpu/drm] Error 2
make[3]: *** [/workspace/kernel/scripts/Makefile.build:485: drivers/gpu] Error 2
make[2]: *** [/workspace/kernel/scripts/Makefile.build:485: drivers] Error 2
make[1]: *** [/workspace/kernel/Makefile:1934: .] Error 2
make: *** [/workspace/kernel/Makefile:240: __sub-make] Error 2
run-parts: /workspace/ci/hooks/11-build-32b exited with return code 2
^ permalink raw reply [flat|nested] 32+ messages in thread
* ✓ CI.checksparse: success for drm/xe/pmu: Enable PMU interface (rev9)
2024-06-13 10:04 [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Riana Tauro
` (6 preceding siblings ...)
2024-06-13 10:13 ` ✗ CI.Hooks: failure " Patchwork
@ 2024-06-13 10:14 ` Patchwork
2024-06-13 11:48 ` ✓ CI.BAT: " Patchwork
2024-06-13 18:39 ` [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Dixit, Ashutosh
9 siblings, 0 replies; 32+ messages in thread
From: Patchwork @ 2024-06-13 10:14 UTC (permalink / raw)
To: Iddamsetty, Aravind; +Cc: intel-xe
== Series Details ==
Series: drm/xe/pmu: Enable PMU interface (rev9)
URL : https://patchwork.freedesktop.org/series/119504/
State : success
== Summary ==
+ trap cleanup EXIT
+ KERNEL=/kernel
+ MT=/root/linux/maintainer-tools
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools /root/linux/maintainer-tools
Cloning into '/root/linux/maintainer-tools'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ make -C /root/linux/maintainer-tools
make: Entering directory '/root/linux/maintainer-tools'
cc -O2 -g -Wextra -o remap-log remap-log.c
make: Leaving directory '/root/linux/maintainer-tools'
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ /root/linux/maintainer-tools/dim sparse --fast dea4e8a3a0889d80afc874513cca0d7f7a25bbcd
Sparse version: 0.6.1 (Ubuntu: 0.6.1-2build1)
Fast mode used, each commit won't be checked separately.
Okay!
+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel
^ permalink raw reply [flat|nested] 32+ messages in thread
* ✓ CI.BAT: success for drm/xe/pmu: Enable PMU interface (rev9)
2024-06-13 10:04 [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Riana Tauro
` (7 preceding siblings ...)
2024-06-13 10:14 ` ✓ CI.checksparse: success " Patchwork
@ 2024-06-13 11:48 ` Patchwork
2024-06-13 18:39 ` [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Dixit, Ashutosh
9 siblings, 0 replies; 32+ messages in thread
From: Patchwork @ 2024-06-13 11:48 UTC (permalink / raw)
To: Iddamsetty, Aravind; +Cc: intel-xe
[-- Attachment #1: Type: text/plain, Size: 951 bytes --]
== Series Details ==
Series: drm/xe/pmu: Enable PMU interface (rev9)
URL : https://patchwork.freedesktop.org/series/119504/
State : success
== Summary ==
CI Bug Log - changes from xe-1456-dea4e8a3a0889d80afc874513cca0d7f7a25bbcd_BAT -> xe-pw-119504v9_BAT
====================================================
Summary
-------
**SUCCESS**
No regressions found.
Participating hosts (5 -> 5)
------------------------------
No changes in participating hosts
Changes
-------
No changes found
Build changes
-------------
* Linux: xe-1456-dea4e8a3a0889d80afc874513cca0d7f7a25bbcd -> xe-pw-119504v9
IGT_7884: 3c02edb64e6feef931f1df9898d129b0a92a39eb @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
xe-1456-dea4e8a3a0889d80afc874513cca0d7f7a25bbcd: dea4e8a3a0889d80afc874513cca0d7f7a25bbcd
xe-pw-119504v9: 119504v9
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-119504v9/index.html
[-- Attachment #2: Type: text/html, Size: 1499 bytes --]
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface
2024-06-13 10:04 [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Riana Tauro
` (8 preceding siblings ...)
2024-06-13 11:48 ` ✓ CI.BAT: " Patchwork
@ 2024-06-13 18:39 ` Dixit, Ashutosh
2024-06-14 5:50 ` Riana Tauro
9 siblings, 1 reply; 32+ messages in thread
From: Dixit, Ashutosh @ 2024-06-13 18:39 UTC (permalink / raw)
To: Riana Tauro
Cc: intel-xe, anshuman.gupta, aravind.iddamsetty, rodrigo.vivi,
umesh.nerlige.ramappa, krishnaiah.bommu
On Thu, 13 Jun 2024 03:04:09 -0700, Riana Tauro wrote:
>
Hi Riana,
> There are a set of engine group busyness counters provided by HW which are
> perfect fit to be exposed via PMU perf events.
>
> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>
> events can be listed using:
> perf list
> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
PMU patches merged previously were dropped in 90a8b23f9b85 ("drm/xe/pmu:
Remove PMU from Xe till uapi is finalized") because PMU uapi was expected
to change. Why are we re-posting these old patches again now, without
including the planned uapi changes?
Thanks.
--
Ashutosh
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface
2024-06-13 18:39 ` [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Dixit, Ashutosh
@ 2024-06-14 5:50 ` Riana Tauro
2024-06-14 15:34 ` Dixit, Ashutosh
0 siblings, 1 reply; 32+ messages in thread
From: Riana Tauro @ 2024-06-14 5:50 UTC (permalink / raw)
To: Dixit, Ashutosh
Cc: intel-xe, anshuman.gupta, aravind.iddamsetty, rodrigo.vivi,
umesh.nerlige.ramappa, krishnaiah.bommu
Hi Ashutosh
On 6/14/2024 12:09 AM, Dixit, Ashutosh wrote:
> On Thu, 13 Jun 2024 03:04:09 -0700, Riana Tauro wrote:
>>
>
> Hi Riana,
>
>> There are a set of engine group busyness counters provided by HW which are
>> perfect fit to be exposed via PMU perf events.
>>
>> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>
>> events can be listed using:
>> perf list
>> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>
> PMU patches merged previously were dropped in 90a8b23f9b85 ("drm/xe/pmu:
> Remove PMU from Xe till uapi is finalized") because PMU uapi was expected
> to change. Why are we re-posting these old patches again now, without
> including the planned uapi changes?
The uapi changes were dropped and there are no other upcoming changes
for Group busyness. So re-posted the old series.
Thanks,
Riana
>
> Thanks.
> --
> Ashutosh
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 1/2] drm/xe: Get GT clock to nanosecs
2024-06-13 10:04 ` [PATCH v9 1/2] drm/xe: Get GT clock to nanosecs Riana Tauro
@ 2024-06-14 14:57 ` Lucas De Marchi
0 siblings, 0 replies; 32+ messages in thread
From: Lucas De Marchi @ 2024-06-14 14:57 UTC (permalink / raw)
To: Riana Tauro
Cc: intel-xe, anshuman.gupta, ashutosh.dixit, aravind.iddamsetty,
rodrigo.vivi, umesh.nerlige.ramappa, krishnaiah.bommu,
Matthew Brost
On Thu, Jun 13, 2024 at 03:34:10PM GMT, Riana Tauro wrote:
>From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>
>Helper to convert GT clock cycles to nanoseconds.
>
>v2: Use DIV_ROUND_CLOSEST_ULL helper(Ashutosh)
>v3: rename xe_gt_clock_interval_to_ns to xe_gt_clock_cycles_to_ns
>v4: rebase
>v5: add doc
>
>Reviewed-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
>Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
>Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>---
> drivers/gpu/drm/xe/xe_gt_clock.c | 13 +++++++++++++
> drivers/gpu/drm/xe/xe_gt_clock.h | 2 +-
> 2 files changed, 14 insertions(+), 1 deletion(-)
>
>diff --git a/drivers/gpu/drm/xe/xe_gt_clock.c b/drivers/gpu/drm/xe/xe_gt_clock.c
>index 86c2d62b4bdc..17bd8851ef3c 100644
>--- a/drivers/gpu/drm/xe/xe_gt_clock.c
>+++ b/drivers/gpu/drm/xe/xe_gt_clock.c
>@@ -99,3 +99,16 @@ u64 xe_gt_clock_interval_to_ms(struct xe_gt *gt, u64 count)
> {
> return div_u64_roundup(count * MSEC_PER_SEC, gt->info.reference_clock);
> }
>+
>+/**
>+ * xe_gt_clock_cycles_to_ns - Convert sampled GT clock ticks to nsec
>+ *
>+ * @gt: GT instance
>+ * @count: count of GT clock ticks
>+ *
>+ * Returns: time in nsec
>+ */
>+u64 xe_gt_clock_cycles_to_ns(const struct xe_gt *gt, u64 count)
>+{
>+ return DIV_ROUND_CLOSEST_ULL(count * NSEC_PER_SEC, gt->info.reference_clock);
other than the changed unit (nsec vs msec), this is the same as the function above,
no reason for a different function name.
nit: we should be using the longer nsec/msec/sec convention rather than
ns/ms as it makes less ambiguous in code, matches other macros/functions.
Suggestion:
s/xe_gt_clock_interval_to_ms/xe_gt_clock_cycles_to_msec/ and make it use
DIV_ROUND_CLOSEST_ULL.
s/xe_gt_clock_cycles_to_ns/xe_gt_clock_cycles_to_nsec/.
+Matt Brost to ack
Lucas De Marchi
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface
2024-06-14 5:50 ` Riana Tauro
@ 2024-06-14 15:34 ` Dixit, Ashutosh
2024-06-20 19:15 ` Umesh Nerlige Ramappa
0 siblings, 1 reply; 32+ messages in thread
From: Dixit, Ashutosh @ 2024-06-14 15:34 UTC (permalink / raw)
To: Riana Tauro
Cc: intel-xe, anshuman.gupta, aravind.iddamsetty, rodrigo.vivi,
umesh.nerlige.ramappa, krishnaiah.bommu
On Thu, 13 Jun 2024 22:50:39 -0700, Riana Tauro wrote:
>
> Hi Ashutosh
>
> On 6/14/2024 12:09 AM, Dixit, Ashutosh wrote:
> > On Thu, 13 Jun 2024 03:04:09 -0700, Riana Tauro wrote:
> >>
> >
> > Hi Riana,
> >
> >> There are a set of engine group busyness counters provided by HW which are
> >> perfect fit to be exposed via PMU perf events.
> >>
> >> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
> >>
> >> events can be listed using:
> >> perf list
> >> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
> >> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
> >> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
> >> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
> >
> > PMU patches merged previously were dropped in 90a8b23f9b85 ("drm/xe/pmu:
> > Remove PMU from Xe till uapi is finalized") because PMU uapi was expected
> > to change. Why are we re-posting these old patches again now, without
> > including the planned uapi changes?
>
> The uapi changes were dropped and there are no other upcoming changes for
> Group busyness. So re-posted the old series.
What happened to VF busyness (which is why I thought the uapi was going to
change)?
>
> Thanks,
> Riana
> >
> > Thanks.
> > --
> > Ashutosh
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-13 10:04 ` [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface Riana Tauro
@ 2024-06-14 16:15 ` Lucas De Marchi
2024-06-14 16:38 ` Tvrtko Ursulin
2024-06-14 20:54 ` Ghimiray, Himal Prasad
2024-06-20 19:52 ` Umesh Nerlige Ramappa
2 siblings, 1 reply; 32+ messages in thread
From: Lucas De Marchi @ 2024-06-14 16:15 UTC (permalink / raw)
To: Riana Tauro
Cc: intel-xe, anshuman.gupta, ashutosh.dixit, aravind.iddamsetty,
rodrigo.vivi, umesh.nerlige.ramappa, krishnaiah.bommu,
tvrtko.ursulin
On Thu, Jun 13, 2024 at 03:34:11PM GMT, Riana Tauro wrote:
>From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>
>There are a set of engine group busyness counters provided by HW which are
>perfect fit to be exposed via PMU perf events.
>
>BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>
>events can be listed using:
>perf list
> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>
>and can be read using:
>
>perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
> time counts unit events
> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>
>The pmu base implementation is taken from i915.
>
>v2:
>Store last known value when device is awake return that while the GT is
>suspended and then update the driver copy when read during awake.
>
>v3:
>1. drop init_samples, as storing counters before going to suspend should
>be sufficient.
>2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>dropped helpers to store and read samples.
>3. use xe_device_mem_access_get_if_ongoing to check if device is active
>before reading the OA registers.
>4. dropped format attr as no longer needed
>5. introduce xe_pmu_suspend to call engine_group_busyness_store
>6. few other nits.
>
>v4: minor nits.
>
>v5: take forcewake when accessing the OAG registers
>
>v6:
>1. drop engine_busyness_sample_type
>2. update UAPI documentation
>
>v7:
>1. update UAPI documentation
>2. drop MEDIA_GT specific change for media busyness counter.
>
>v8:
>1. rebase
>2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>3. remove interrupts pmu event
>
>v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>
>Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
first s-o-b should match the author in the patch. According to the
"From" above, author is set to Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
I wil leave some nits about the implementation and focus on the main
concept being added here.
...
>diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>new file mode 100644
>index 000000000000..64960a358af2
>--- /dev/null
>+++ b/drivers/gpu/drm/xe/xe_pmu.c
>@@ -0,0 +1,631 @@
>+// SPDX-License-Identifier: MIT
>+/*
>+ * Copyright © 2024 Intel Corporation
>+ */
>+
>+#include <drm/drm_drv.h>
>+#include <drm/drm_managed.h>
>+#include <drm/xe_drm.h>
>+
>+#include "regs/xe_gt_regs.h"
>+#include "xe_device.h"
>+#include "xe_force_wake.h"
>+#include "xe_gt_clock.h"
>+#include "xe_mmio.h"
>+#include "xe_macros.h"
>+#include "xe_pm.h"
>+
>+static cpumask_t xe_pmu_cpumask;
>+static unsigned int xe_pmu_target_cpu = -1;
>+
>+static unsigned int config_gt_id(const u64 config)
>+{
>+ return config >> __XE_PMU_GT_SHIFT;
>+}
>+
>+static u64 config_counter(const u64 config)
>+{
>+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>+}
>+
>+static void xe_pmu_event_destroy(struct perf_event *event)
>+{
>+ struct xe_device *xe =
>+ container_of(event->pmu, typeof(*xe), pmu.base);
>+
>+ drm_WARN_ON(&xe->drm, event->parent);
>+
>+ drm_dev_put(&xe->drm);
>+}
>+
>+static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
>+{
>+ u64 val;
>+
>+ switch (sample_type) {
>+ case __XE_SAMPLE_RENDER_GROUP_BUSY:
>+ val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>+ break;
>+ case __XE_SAMPLE_COPY_GROUP_BUSY:
>+ val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>+ break;
>+ case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>+ val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>+ break;
>+ case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>+ val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>+ break;
>+ default:
>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>+ }
>+
>+ return xe_gt_clock_cycles_to_ns(gt, val * 16);
>+}
>+
>+static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>+{
>+ int sample_type = config_counter(config);
>+ const unsigned int gt_id = gt->info.id;
>+ struct xe_device *xe = gt->tile->xe;
>+ struct xe_pmu *pmu = &xe->pmu;
>+ unsigned long flags;
>+ bool device_awake;
>+ u64 val;
>+
>+ device_awake = xe_pm_runtime_get_if_active(xe);
>+ if (device_awake) {
>+ XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>+ val = __engine_group_busyness_read(gt, sample_type);
>+ XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>+ xe_pm_runtime_put(xe);
>+ }
>+
>+ spin_lock_irqsave(&pmu->lock, flags);
>+
>+ if (device_awake)
>+ pmu->sample[gt_id][sample_type] = val;
>+ else
>+ val = pmu->sample[gt_id][sample_type];
>+
>+ spin_unlock_irqrestore(&pmu->lock, flags);
>+
>+ return val;
>+}
>+
>+static void engine_group_busyness_store(struct xe_gt *gt)
>+{
>+ struct xe_pmu *pmu = >->tile->xe->pmu;
>+ unsigned int gt_id = gt->info.id;
>+ unsigned long flags;
>+ int i;
>+
>+ spin_lock_irqsave(&pmu->lock, flags);
>+
>+ for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>+ pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>+
>+ spin_unlock_irqrestore(&pmu->lock, flags);
>+}
>+
>+static int
>+config_status(struct xe_device *xe, u64 config)
>+{
>+ unsigned int gt_id = config_gt_id(config);
>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>+
>+ if (gt_id >= XE_PMU_MAX_GT)
>+ return -ENOENT;
>+
>+ switch (config_counter(config)) {
>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>+ case XE_PMU_COPY_GROUP_BUSY(0):
>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>+ if (gt->info.type == XE_GT_TYPE_MEDIA)
>+ return -ENOENT;
>+ break;
>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>+ if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
>+ return -ENOENT;
>+ break;
>+ default:
>+ return -ENOENT;
>+ }
>+
>+ return 0;
>+}
>+
>+static int xe_pmu_event_init(struct perf_event *event)
>+{
>+ struct xe_device *xe =
>+ container_of(event->pmu, typeof(*xe), pmu.base);
>+ struct xe_pmu *pmu = &xe->pmu;
>+ int ret;
>+
>+ if (pmu->closed)
>+ return -ENODEV;
>+
>+ if (event->attr.type != event->pmu->type)
>+ return -ENOENT;
>+
>+ /* unsupported modes and filters */
>+ if (event->attr.sample_period) /* no sampling */
>+ return -EINVAL;
>+
>+ if (has_branch_stack(event))
>+ return -EOPNOTSUPP;
>+
>+ if (event->cpu < 0)
>+ return -EINVAL;
>+
>+ /* only allow running on one cpu at a time */
>+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>+ return -EINVAL;
>+
>+ ret = config_status(xe, event->attr.config);
>+ if (ret)
>+ return ret;
>+
>+ if (!event->parent) {
>+ drm_dev_get(&xe->drm);
>+ event->destroy = xe_pmu_event_destroy;
>+ }
>+
>+ return 0;
>+}
>+
>+static u64 __xe_pmu_event_read(struct perf_event *event)
>+{
>+ struct xe_device *xe =
>+ container_of(event->pmu, typeof(*xe), pmu.base);
>+ const unsigned int gt_id = config_gt_id(event->attr.config);
>+ const u64 config = event->attr.config;
>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>+ u64 val;
>+
>+ switch (config_counter(config)) {
>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>+ case XE_PMU_COPY_GROUP_BUSY(0):
>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>+ val = engine_group_busyness_read(gt, config);
>+ break;
>+ default:
>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>+ }
>+
>+ return val;
>+}
>+
>+static void xe_pmu_event_read(struct perf_event *event)
>+{
>+ struct xe_device *xe =
>+ container_of(event->pmu, typeof(*xe), pmu.base);
>+ struct hw_perf_event *hwc = &event->hw;
>+ struct xe_pmu *pmu = &xe->pmu;
>+ u64 prev, new;
>+
>+ if (pmu->closed) {
>+ event->hw.state = PERF_HES_STOPPED;
>+ return;
>+ }
>+again:
>+ prev = local64_read(&hwc->prev_count);
>+ new = __xe_pmu_event_read(event);
so... when we enable a perf counter with the example in the cover
letter:
perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
then we will have the following call chain:
xe_pmu_event_read()
__xe_pmu_event_read()
engine_group_busyness_read()
engine_group_busyness_read()
__engine_group_busyness_read()
xe_mmio_read32()
At a frequency up to sysctl kernel.perf_event_max_sample_rate (~50kHz
in my distro). The event itself is recorded to the ring buffer if it
changed.
The HW interface we are using is to simply read XE_OAG_*_BUSY_FREE.
At the same time we are trying to add new uapi called xe_perf (that has
nothing to do with perf, sigh) that exposes OA as streams with ioctl.
Also, at the same time we already do per-client engine utilization and
with the usual very few clients, why couldn't userspace just aggregate
the values per engine? Is it really useful to expose this HW counter to
userspace? From a quick look, it doesn't seem that much more accurate
to use the global (per-engine-class) counter. The bad non-performant
part IMO of the per-client side is the client-discovery via /proc
Also not clear to me why we need the percpu part if we are collecting
the GPU counters. It's more than 10 years I last played with the
implementation side of perf counters, so I'm needing a refresh while
looking at this.
FYI I started to look to thi series because of the problem reported in
i915 wrt perf counters while unbinding the device:
https://lore.kernel.org/lkml/20240115170120.662220-1-tvrtko.ursulin@linux.intel.com/T/#me72abfa2771e6fc94b167ce47efdbf391cc313ab
and
https://lore.kernel.org/all/20240213180302.47266-1-umesh.nerlige.ramappa@intel.com/
Tvrtko, any additional feedback you got from the perf/core side for that
series?
Lucas De Marchi
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-14 16:15 ` Lucas De Marchi
@ 2024-06-14 16:38 ` Tvrtko Ursulin
0 siblings, 0 replies; 32+ messages in thread
From: Tvrtko Ursulin @ 2024-06-14 16:38 UTC (permalink / raw)
To: Lucas De Marchi, Riana Tauro
Cc: intel-xe, anshuman.gupta, ashutosh.dixit, aravind.iddamsetty,
rodrigo.vivi, umesh.nerlige.ramappa, krishnaiah.bommu
On 14/06/2024 17:15, Lucas De Marchi wrote:
> On Thu, Jun 13, 2024 at 03:34:11PM GMT, Riana Tauro wrote:
>> From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>
>> There are a set of engine group busyness counters provided by HW which
>> are
>> perfect fit to be exposed via PMU perf events.
>>
>> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>
>> events can be listed using:
>> perf list
>> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>
>> and can be read using:
>>
>> perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
>> time counts unit events
>> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>
>> The pmu base implementation is taken from i915.
>>
>> v2:
>> Store last known value when device is awake return that while the GT is
>> suspended and then update the driver copy when read during awake.
>>
>> v3:
>> 1. drop init_samples, as storing counters before going to suspend should
>> be sufficient.
>> 2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>> dropped helpers to store and read samples.
>> 3. use xe_device_mem_access_get_if_ongoing to check if device is active
>> before reading the OA registers.
>> 4. dropped format attr as no longer needed
>> 5. introduce xe_pmu_suspend to call engine_group_busyness_store
>> 6. few other nits.
>>
>> v4: minor nits.
>>
>> v5: take forcewake when accessing the OAG registers
>>
>> v6:
>> 1. drop engine_busyness_sample_type
>> 2. update UAPI documentation
>>
>> v7:
>> 1. update UAPI documentation
>> 2. drop MEDIA_GT specific change for media busyness counter.
>>
>> v8:
>> 1. rebase
>> 2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>> 3. remove interrupts pmu event
>>
>> v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>>
>> Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>> Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>
> first s-o-b should match the author in the patch. According to the
> "From" above, author is set to Aravind Iddamsetty
> <aravind.iddamsetty@linux.intel.com>
>
> I wil leave some nits about the implementation and focus on the main
> concept being added here.
>
> ...
>
>> diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>> new file mode 100644
>> index 000000000000..64960a358af2
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu.c
>> @@ -0,0 +1,631 @@
>> +// SPDX-License-Identifier: MIT
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#include <drm/drm_drv.h>
>> +#include <drm/drm_managed.h>
>> +#include <drm/xe_drm.h>
>> +
>> +#include "regs/xe_gt_regs.h"
>> +#include "xe_device.h"
>> +#include "xe_force_wake.h"
>> +#include "xe_gt_clock.h"
>> +#include "xe_mmio.h"
>> +#include "xe_macros.h"
>> +#include "xe_pm.h"
>> +
>> +static cpumask_t xe_pmu_cpumask;
>> +static unsigned int xe_pmu_target_cpu = -1;
>> +
>> +static unsigned int config_gt_id(const u64 config)
>> +{
>> + return config >> __XE_PMU_GT_SHIFT;
>> +}
>> +
>> +static u64 config_counter(const u64 config)
>> +{
>> + return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>> +}
>> +
>> +static void xe_pmu_event_destroy(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> +
>> + drm_WARN_ON(&xe->drm, event->parent);
>> +
>> + drm_dev_put(&xe->drm);
>> +}
>> +
>> +static u64 __engine_group_busyness_read(struct xe_gt *gt, int
>> sample_type)
>> +{
>> + u64 val;
>> +
>> + switch (sample_type) {
>> + case __XE_SAMPLE_RENDER_GROUP_BUSY:
>> + val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>> + break;
>> + case __XE_SAMPLE_COPY_GROUP_BUSY:
>> + val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>> + break;
>> + case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>> + val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>> + break;
>> + case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>> + val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>> + break;
>> + default:
>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>> + }
>> +
>> + return xe_gt_clock_cycles_to_ns(gt, val * 16);
>> +}
>> +
>> +static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>> +{
>> + int sample_type = config_counter(config);
>> + const unsigned int gt_id = gt->info.id;
>> + struct xe_device *xe = gt->tile->xe;
>> + struct xe_pmu *pmu = &xe->pmu;
>> + unsigned long flags;
>> + bool device_awake;
>> + u64 val;
>> +
>> + device_awake = xe_pm_runtime_get_if_active(xe);
>> + if (device_awake) {
>> + XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>> + val = __engine_group_busyness_read(gt, sample_type);
>> + XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>> + xe_pm_runtime_put(xe);
>> + }
>> +
>> + spin_lock_irqsave(&pmu->lock, flags);
>> +
>> + if (device_awake)
>> + pmu->sample[gt_id][sample_type] = val;
>> + else
>> + val = pmu->sample[gt_id][sample_type];
>> +
>> + spin_unlock_irqrestore(&pmu->lock, flags);
>> +
>> + return val;
>> +}
>> +
>> +static void engine_group_busyness_store(struct xe_gt *gt)
>> +{
>> + struct xe_pmu *pmu = >->tile->xe->pmu;
>> + unsigned int gt_id = gt->info.id;
>> + unsigned long flags;
>> + int i;
>> +
>> + spin_lock_irqsave(&pmu->lock, flags);
>> +
>> + for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <=
>> __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>> + pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>> +
>> + spin_unlock_irqrestore(&pmu->lock, flags);
>> +}
>> +
>> +static int
>> +config_status(struct xe_device *xe, u64 config)
>> +{
>> + unsigned int gt_id = config_gt_id(config);
>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>> +
>> + if (gt_id >= XE_PMU_MAX_GT)
>> + return -ENOENT;
>> +
>> + switch (config_counter(config)) {
>> + case XE_PMU_RENDER_GROUP_BUSY(0):
>> + case XE_PMU_COPY_GROUP_BUSY(0):
>> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>> + if (gt->info.type == XE_GT_TYPE_MEDIA)
>> + return -ENOENT;
>> + break;
>> + case XE_PMU_MEDIA_GROUP_BUSY(0):
>> + if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) |
>> BIT(XE_HW_ENGINE_VECS0))))
>> + return -ENOENT;
>> + break;
>> + default:
>> + return -ENOENT;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int xe_pmu_event_init(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> + int ret;
>> +
>> + if (pmu->closed)
>> + return -ENODEV;
>> +
>> + if (event->attr.type != event->pmu->type)
>> + return -ENOENT;
>> +
>> + /* unsupported modes and filters */
>> + if (event->attr.sample_period) /* no sampling */
>> + return -EINVAL;
>> +
>> + if (has_branch_stack(event))
>> + return -EOPNOTSUPP;
>> +
>> + if (event->cpu < 0)
>> + return -EINVAL;
>> +
>> + /* only allow running on one cpu at a time */
>> + if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>> + return -EINVAL;
>> +
>> + ret = config_status(xe, event->attr.config);
>> + if (ret)
>> + return ret;
>> +
>> + if (!event->parent) {
>> + drm_dev_get(&xe->drm);
>> + event->destroy = xe_pmu_event_destroy;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static u64 __xe_pmu_event_read(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + const unsigned int gt_id = config_gt_id(event->attr.config);
>> + const u64 config = event->attr.config;
>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>> + u64 val;
>> +
>> + switch (config_counter(config)) {
>> + case XE_PMU_RENDER_GROUP_BUSY(0):
>> + case XE_PMU_COPY_GROUP_BUSY(0):
>> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>> + case XE_PMU_MEDIA_GROUP_BUSY(0):
>> + val = engine_group_busyness_read(gt, config);
>> + break;
>> + default:
>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>> + }
>> +
>> + return val;
>> +}
>> +
>> +static void xe_pmu_event_read(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct hw_perf_event *hwc = &event->hw;
>> + struct xe_pmu *pmu = &xe->pmu;
>> + u64 prev, new;
>> +
>> + if (pmu->closed) {
>> + event->hw.state = PERF_HES_STOPPED;
>> + return;
>> + }
>> +again:
>> + prev = local64_read(&hwc->prev_count);
>> + new = __xe_pmu_event_read(event);
>
>
> so... when we enable a perf counter with the example in the cover
> letter:
>
> perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
>
> then we will have the following call chain:
>
> xe_pmu_event_read()
> __xe_pmu_event_read()
> engine_group_busyness_read()
> engine_group_busyness_read()
> __engine_group_busyness_read()
> xe_mmio_read32()
>
> At a frequency up to sysctl kernel.perf_event_max_sample_rate (~50kHz
> in my distro). The event itself is recorded to the ring buffer if it
> changed.
No, only once a second in this example. I forget the terminology..
Sampling is for per-task PMU drivers which i915 and this are not.
> The HW interface we are using is to simply read XE_OAG_*_BUSY_FREE.
> At the same time we are trying to add new uapi called xe_perf (that has
> nothing to do with perf, sigh) that exposes OA as streams with ioctl.
>
> Also, at the same time we already do per-client engine utilization and
> with the usual very few clients, why couldn't userspace just aggregate
> the values per engine? Is it really useful to expose this HW counter to
> userspace? From a quick look, it doesn't seem that much more accurate
> to use the global (per-engine-class) counter. The bad non-performant
> part IMO of the per-client side is the client-discovery via /proc
Once upon a time we were saying why we need these new group busyness
thingies when we got per engine, but semantics are different and people
(L0) were very insisting :shrug:
IMHO group busyness is weak and next to useless for end users and
intel_gpu_top like tools. But it is not a discussion for me.
Aggregating fdinfo stats would IMO be better semantics but wasteful and
not showing in-kernel stuff like maybe clearing and migrations via blitter.
> Also not clear to me why we need the percpu part if we are collecting
> the GPU counters. It's more than 10 years I last played with the
> implementation side of perf counters, so I'm needing a refresh while
> looking at this.
Per cpu.. you mean cpu hotplug handling? Perf core mandates it.
> FYI I started to look to thi series because of the problem reported in
> i915 wrt perf counters while unbinding the device:
>
> https://lore.kernel.org/lkml/20240115170120.662220-1-tvrtko.ursulin@linux.intel.com/T/#me72abfa2771e6fc94b167ce47efdbf391cc313ab
>
> and
>
> https://lore.kernel.org/all/20240213180302.47266-1-umesh.nerlige.ramappa@intel.com/
>
> Tvrtko, any additional feedback you got from the perf/core side for that
> series?
Good that you found it in the archives otherwise I could be thinking I
sent it to /dev/null. Does that answer you? :)
IMO best solution is to fix the perf core for hot unbind and have i915
like counters. Other options are all flawed and weak.
Regards,
Tvrtko
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-13 10:04 ` [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface Riana Tauro
2024-06-14 16:15 ` Lucas De Marchi
@ 2024-06-14 20:54 ` Ghimiray, Himal Prasad
2024-06-27 5:21 ` Riana Tauro
2024-06-20 19:52 ` Umesh Nerlige Ramappa
2 siblings, 1 reply; 32+ messages in thread
From: Ghimiray, Himal Prasad @ 2024-06-14 20:54 UTC (permalink / raw)
To: Riana Tauro, intel-xe
Cc: anshuman.gupta, ashutosh.dixit, aravind.iddamsetty, rodrigo.vivi,
umesh.nerlige.ramappa, krishnaiah.bommu
On 13-06-2024 15:34, Riana Tauro wrote:
> From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>
> There are a set of engine group busyness counters provided by HW which are
> perfect fit to be exposed via PMU perf events.
>
> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>
> events can be listed using:
> perf list
> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>
> and can be read using:
>
> perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
> time counts unit events
> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>
> The pmu base implementation is taken from i915.
>
> v2:
> Store last known value when device is awake return that while the GT is
> suspended and then update the driver copy when read during awake.
>
> v3:
> 1. drop init_samples, as storing counters before going to suspend should
> be sufficient.
> 2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
> dropped helpers to store and read samples.
> 3. use xe_device_mem_access_get_if_ongoing to check if device is active
> before reading the OA registers.
> 4. dropped format attr as no longer needed
> 5. introduce xe_pmu_suspend to call engine_group_busyness_store
> 6. few other nits.
>
> v4: minor nits.
>
> v5: take forcewake when accessing the OAG registers
>
> v6:
> 1. drop engine_busyness_sample_type
> 2. update UAPI documentation
>
> v7:
> 1. update UAPI documentation
> 2. drop MEDIA_GT specific change for media busyness counter.
>
> v8:
> 1. rebase
> 2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
> 3. remove interrupts pmu event
>
> v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>
> Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
> Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
> Signed-off-by: Riana Tauro <riana.tauro@intel.com>
> ---
> drivers/gpu/drm/xe/Makefile | 2 +
> drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
> drivers/gpu/drm/xe/xe_device.c | 2 +
> drivers/gpu/drm/xe/xe_device_types.h | 4 +
> drivers/gpu/drm/xe/xe_gt.c | 2 +
> drivers/gpu/drm/xe/xe_module.c | 5 +
> drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
> drivers/gpu/drm/xe/xe_pmu.h | 26 ++
> drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
> include/uapi/drm/xe_drm.h | 39 ++
> 10 files changed, 783 insertions(+)
> create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
> create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
> create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>
> diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
> index cbf961b90237..83bf1e07669b 100644
> --- a/drivers/gpu/drm/xe/Makefile
> +++ b/drivers/gpu/drm/xe/Makefile
> @@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
> i915-display/skl_universal_plane.o \
> i915-display/skl_watermark.o
>
> +xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
> +
> ifeq ($(CONFIG_ACPI),y)
> xe-$(CONFIG_DRM_XE_DISPLAY) += \
> i915-display/intel_acpi.o \
> diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
> index 47c26c37608d..22821dcd4e1b 100644
> --- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
> +++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
> @@ -390,6 +390,11 @@
> #define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
> #define GLOBAL_INVALIDATION_MODE REG_BIT(2)
>
> +#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
> +#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
> +#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
> +#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
> +
> #define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
> #define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
>
> diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
> index 64691a56d59c..bb00c8c9ec9b 100644
> --- a/drivers/gpu/drm/xe/xe_device.c
> +++ b/drivers/gpu/drm/xe/xe_device.c
> @@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
>
> xe_hwmon_register(xe);
>
> + xe_pmu_register(&xe->pmu);
> +
> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>
> err_fini_display:
> diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> index 52bc461171d5..a5dba7325cf1 100644
> --- a/drivers/gpu/drm/xe/xe_device_types.h
> +++ b/drivers/gpu/drm/xe/xe_device_types.h
> @@ -18,6 +18,7 @@
> #include "xe_lmtt_types.h"
> #include "xe_memirq_types.h"
> #include "xe_platform_types.h"
> +#include "xe_pmu.h"
> #include "xe_pt_types.h"
> #include "xe_sriov_types.h"
> #include "xe_step_types.h"
> @@ -473,6 +474,9 @@ struct xe_device {
> int mode;
> } wedged;
>
> + /** @pmu: performance monitoring unit */
> + struct xe_pmu pmu;
> +
> /* private: */
>
> #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
> diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
> index 57d84751e160..477d0ae5f230 100644
> --- a/drivers/gpu/drm/xe/xe_gt.c
> +++ b/drivers/gpu/drm/xe/xe_gt.c
> @@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
> if (err)
> goto err_msg;
>
> + xe_pmu_suspend(gt);
> +
> err = xe_uc_suspend(>->uc);
> if (err)
> goto err_force_wake;
> diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
> index 3edeb30d5ccb..26f814f97fc2 100644
> --- a/drivers/gpu/drm/xe/xe_module.c
> +++ b/drivers/gpu/drm/xe/xe_module.c
> @@ -11,6 +11,7 @@
> #include "xe_drv.h"
> #include "xe_hw_fence.h"
> #include "xe_pci.h"
> +#include "xe_pmu.h"
> #include "xe_sched_job.h"
>
> struct xe_modparam xe_modparam = {
> @@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
> .init = xe_sched_job_module_init,
> .exit = xe_sched_job_module_exit,
> },
> + {
> + .init = xe_pmu_init,
> + .exit = xe_pmu_exit,
> + },
> {
> .init = xe_register_pci_driver,
> .exit = xe_unregister_pci_driver,
> diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
> new file mode 100644
> index 000000000000..64960a358af2
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_pmu.c
> @@ -0,0 +1,631 @@
> +// SPDX-License-Identifier: MIT
> +/*
> + * Copyright © 2024 Intel Corporation
> + */
> +
> +#include <drm/drm_drv.h>
> +#include <drm/drm_managed.h>
> +#include <drm/xe_drm.h>
> +
> +#include "regs/xe_gt_regs.h"
> +#include "xe_device.h"
> +#include "xe_force_wake.h"
> +#include "xe_gt_clock.h"
> +#include "xe_mmio.h"
> +#include "xe_macros.h"
> +#include "xe_pm.h"
> +
> +static cpumask_t xe_pmu_cpumask;
> +static unsigned int xe_pmu_target_cpu = -1;
> +
> +static unsigned int config_gt_id(const u64 config)
> +{
> + return config >> __XE_PMU_GT_SHIFT;
> +}
> +
> +static u64 config_counter(const u64 config)
> +{
> + return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
> +}
> +
> +static void xe_pmu_event_destroy(struct perf_event *event)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> +
> + drm_WARN_ON(&xe->drm, event->parent);
> +
> + drm_dev_put(&xe->drm);
> +}
> +
> +static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
> +{
> + u64 val;
> +
> + switch (sample_type) {
> + case __XE_SAMPLE_RENDER_GROUP_BUSY:
> + val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
> + break;
> + case __XE_SAMPLE_COPY_GROUP_BUSY:
> + val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
> + break;
> + case __XE_SAMPLE_MEDIA_GROUP_BUSY:
> + val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
> + break;
> + case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
> + val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
> + break;
> + default:
> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
> + }
> +
> + return xe_gt_clock_cycles_to_ns(gt, val * 16);
> +}
> +
> +static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
> +{
> + int sample_type = config_counter(config);
> + const unsigned int gt_id = gt->info.id;
> + struct xe_device *xe = gt->tile->xe;
> + struct xe_pmu *pmu = &xe->pmu;
> + unsigned long flags;
> + bool device_awake;
> + u64 val;
> +
> + device_awake = xe_pm_runtime_get_if_active(xe);
> + if (device_awake) {
> + XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
> + val = __engine_group_busyness_read(gt, sample_type);
> + XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
> + xe_pm_runtime_put(xe);
> + }
> +
> + spin_lock_irqsave(&pmu->lock, flags);
> +
> + if (device_awake)
> + pmu->sample[gt_id][sample_type] = val;
> + else
> + val = pmu->sample[gt_id][sample_type];
> +
> + spin_unlock_irqrestore(&pmu->lock, flags);
> +
> + return val;
> +}
> +
> +static void engine_group_busyness_store(struct xe_gt *gt)
> +{
> + struct xe_pmu *pmu = >->tile->xe->pmu;
> + unsigned int gt_id = gt->info.id;
> + unsigned long flags;
> + int i;
> +
> + spin_lock_irqsave(&pmu->lock, flags);
> +
> + for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
> + pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
> +
> + spin_unlock_irqrestore(&pmu->lock, flags);
> +}
> +
> +static int
> +config_status(struct xe_device *xe, u64 config)
> +{
> + unsigned int gt_id = config_gt_id(config);
> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
> +
> + if (gt_id >= XE_PMU_MAX_GT)
> + return -ENOENT;
> +
> + switch (config_counter(config)) {
> + case XE_PMU_RENDER_GROUP_BUSY(0):
> + case XE_PMU_COPY_GROUP_BUSY(0):
> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
> + if (gt->info.type == XE_GT_TYPE_MEDIA)
> + return -ENOENT;
> + break;
> + case XE_PMU_MEDIA_GROUP_BUSY(0):
> + if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
> + return -ENOENT;
> + break;
> + default:
> + return -ENOENT;
> + }
> +
> + return 0;
> +}
> +
> +static int xe_pmu_event_init(struct perf_event *event)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct xe_pmu *pmu = &xe->pmu;
> + int ret;
> +
> + if (pmu->closed)
> + return -ENODEV;
> +
> + if (event->attr.type != event->pmu->type)
> + return -ENOENT;
> +
> + /* unsupported modes and filters */
> + if (event->attr.sample_period) /* no sampling */
> + return -EINVAL;
> +
> + if (has_branch_stack(event))
> + return -EOPNOTSUPP;
> +
> + if (event->cpu < 0)
> + return -EINVAL;
> +
> + /* only allow running on one cpu at a time */
> + if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
> + return -EINVAL;
> +
> + ret = config_status(xe, event->attr.config);
> + if (ret)
> + return ret;
> +
> + if (!event->parent) {
> + drm_dev_get(&xe->drm);
> + event->destroy = xe_pmu_event_destroy;
> + }
> +
> + return 0;
> +}
> +
> +static u64 __xe_pmu_event_read(struct perf_event *event)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + const unsigned int gt_id = config_gt_id(event->attr.config);
> + const u64 config = event->attr.config;
> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
> + u64 val;
> +
> + switch (config_counter(config)) {
> + case XE_PMU_RENDER_GROUP_BUSY(0):
> + case XE_PMU_COPY_GROUP_BUSY(0):
> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
> + case XE_PMU_MEDIA_GROUP_BUSY(0):
> + val = engine_group_busyness_read(gt, config);
> + break;
> + default:
> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
> + }
> +
> + return val;
> +}
> +
> +static void xe_pmu_event_read(struct perf_event *event)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct hw_perf_event *hwc = &event->hw;
> + struct xe_pmu *pmu = &xe->pmu;
> + u64 prev, new;
> +
> + if (pmu->closed) {
> + event->hw.state = PERF_HES_STOPPED;
> + return;
> + }
> +again:
> + prev = local64_read(&hwc->prev_count);
> + new = __xe_pmu_event_read(event);
> +
> + if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
> + goto again;
> +
> + local64_add(new - prev, &event->count);
> +}
> +
> +static void xe_pmu_enable(struct perf_event *event)
> +{
> + /*
> + * Store the current counter value so we can report the correct delta
> + * for all listeners. Even when the event was already enabled and has
> + * an existing non-zero value.
> + */
> + local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
> +}
> +
> +static void xe_pmu_event_start(struct perf_event *event, int flags)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct xe_pmu *pmu = &xe->pmu;
> +
> + if (pmu->closed)
> + return;
> +
> + xe_pmu_enable(event);
> + event->hw.state = 0;
> +}
> +
> +static void xe_pmu_event_stop(struct perf_event *event, int flags)
> +{
> + if (flags & PERF_EF_UPDATE)
> + xe_pmu_event_read(event);
> +
> + event->hw.state = PERF_HES_STOPPED;
> +}
> +
> +static int xe_pmu_event_add(struct perf_event *event, int flags)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct xe_pmu *pmu = &xe->pmu;
> +
> + if (pmu->closed)
> + return -ENODEV;
> +
> + if (flags & PERF_EF_START)
> + xe_pmu_event_start(event, flags);
> +
> + return 0;
> +}
> +
> +static void xe_pmu_event_del(struct perf_event *event, int flags)
> +{
> + xe_pmu_event_stop(event, PERF_EF_UPDATE);
> +}
> +
> +static int xe_pmu_event_event_idx(struct perf_event *event)
> +{
> + return 0;
> +}
> +
> +struct xe_ext_attribute {
> + struct device_attribute attr;
> + unsigned long val;
> +};
> +
> +static ssize_t xe_pmu_event_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + struct xe_ext_attribute *eattr;
> +
> + eattr = container_of(attr, struct xe_ext_attribute, attr);
> + return sprintf(buf, "config=0x%lx\n", eattr->val);
> +}
> +
> +static ssize_t cpumask_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
> +}
> +
> +static DEVICE_ATTR_RO(cpumask);
> +
> +static struct attribute *xe_cpumask_attrs[] = {
> + &dev_attr_cpumask.attr,
> + NULL,
> +};
> +
> +static const struct attribute_group xe_pmu_cpumask_attr_group = {
> + .attrs = xe_cpumask_attrs,
> +};
> +
> +#define __event(__counter, __name, __unit) \
> +{ \
> + .counter = (__counter), \
> + .name = (__name), \
> + .unit = (__unit), \
> +}
> +
> +static struct xe_ext_attribute *
> +add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
> +{
> + sysfs_attr_init(&attr->attr.attr);
> + attr->attr.attr.name = name;
> + attr->attr.attr.mode = 0444;
> + attr->attr.show = xe_pmu_event_show;
> + attr->val = config;
> +
> + return ++attr;
> +}
> +
> +static struct perf_pmu_events_attr *
> +add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
> + const char *str)
> +{
> + sysfs_attr_init(&attr->attr.attr);
> + attr->attr.attr.name = name;
> + attr->attr.attr.mode = 0444;
> + attr->attr.show = perf_event_sysfs_show;
> + attr->event_str = str;
> +
> + return ++attr;
> +}
> +
> +static struct attribute **
> +create_event_attributes(struct xe_pmu *pmu)
> +{
> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
> + static const struct {
> + unsigned int counter;
> + const char *name;
> + const char *unit;
> + } events[] = {
> + __event(0, "render-group-busy", "ns"),
> + __event(1, "copy-group-busy", "ns"),
> + __event(2, "media-group-busy", "ns"),
> + __event(3, "any-engine-group-busy", "ns"),
> + };
> +
> + struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
> + struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
> + struct attribute **attr = NULL, **attr_iter;
> + unsigned int count = 0;
> + unsigned int i, j;
> + struct xe_gt *gt;
> +
> + /* Count how many counters we will be exposing. */
> + for_each_gt(gt, xe, j) {
> + for (i = 0; i < ARRAY_SIZE(events); i++) {
> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
> +
> + if (!config_status(xe, config))
> + count++;
> + }
> + }
> +
> + /* Allocate attribute objects and table. */
> + xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
> + if (!xe_attr)
> + goto err_alloc;
> +
> + pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
> + if (!pmu_attr)
> + goto err_alloc;
> +
> + /* Max one pointer of each attribute type plus a termination entry. */
> + attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
> + if (!attr)
> + goto err_alloc;
> +
> + xe_iter = xe_attr;
> + pmu_iter = pmu_attr;
> + attr_iter = attr;
> +
> + for_each_gt(gt, xe, j) {
> + for (i = 0; i < ARRAY_SIZE(events); i++) {
> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
> + char *str;
> +
> + if (config_status(xe, config))
> + continue;
> +
> + str = kasprintf(GFP_KERNEL, "%s-gt%u",
> + events[i].name, j);
> + if (!str)
> + goto err;
> +
> + *attr_iter++ = &xe_iter->attr.attr;
> + xe_iter = add_xe_attr(xe_iter, str, config);
> +
> + if (events[i].unit) {
> + str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
> + events[i].name, j);
> + if (!str)
> + goto err;
> +
> + *attr_iter++ = &pmu_iter->attr.attr;
> + pmu_iter = add_pmu_attr(pmu_iter, str,
> + events[i].unit);
> + }
> + }
> + }
> +
> + pmu->xe_attr = xe_attr;
> + pmu->pmu_attr = pmu_attr;
> +
> + return attr;
> +
> +err:
> + for (attr_iter = attr; *attr_iter; attr_iter++)
> + kfree((*attr_iter)->name);
> +
> +err_alloc:
> + kfree(attr);
> + kfree(xe_attr);
> + kfree(pmu_attr);
> +
> + return NULL;
> +}
> +
> +static void free_event_attributes(struct xe_pmu *pmu)
> +{
> + struct attribute **attr_iter = pmu->events_attr_group.attrs;
> +
> + for (; *attr_iter; attr_iter++)
> + kfree((*attr_iter)->name);
> +
> + kfree(pmu->events_attr_group.attrs);
> + kfree(pmu->xe_attr);
> + kfree(pmu->pmu_attr);
> +
> + pmu->events_attr_group.attrs = NULL;
> + pmu->xe_attr = NULL;
> + pmu->pmu_attr = NULL;
> +}
> +
> +static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
> +{
> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
> +
> + /* Select the first online CPU as a designated reader. */
> + if (cpumask_empty(&xe_pmu_cpumask))
> + cpumask_set_cpu(cpu, &xe_pmu_cpumask);
> +
> + return 0;
> +}
> +
> +static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
> +{
> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
> + unsigned int target = xe_pmu_target_cpu;
> +
> + /*
> + * Unregistering an instance generates a CPU offline event which we must
> + * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
> + */
> + if (pmu->closed)
> + return 0;
> +
> + if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
> + target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
> +
> + /* Migrate events if there is a valid target */
> + if (target < nr_cpu_ids) {
> + cpumask_set_cpu(target, &xe_pmu_cpumask);
> + xe_pmu_target_cpu = target;
> + }
> + }
> +
> + if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
> + perf_pmu_migrate_context(&pmu->base, cpu, target);
> + pmu->cpuhp.cpu = target;
> + }
> +
> + return 0;
> +}
> +
> +static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
> +
> +int xe_pmu_init(void)
> +{
> + int ret;
> +
> + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
> + "perf/x86/intel/xe:online",
> + xe_pmu_cpu_online,
> + xe_pmu_cpu_offline);
> + if (ret < 0)
> + pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
> + ret);
> + else
> + cpuhp_slot = ret;
> +
> + return 0;
> +}
> +
> +void xe_pmu_exit(void)
> +{
> + if (cpuhp_slot != CPUHP_INVALID)
> + cpuhp_remove_multi_state(cpuhp_slot);
> +}
> +
> +static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
> +{
> + if (cpuhp_slot == CPUHP_INVALID)
> + return -EINVAL;
> +
> + return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
> +}
> +
> +static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
> +{
> + cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
> +}
> +
> +void xe_pmu_suspend(struct xe_gt *gt)
> +{
> + engine_group_busyness_store(gt);
> +}
> +
> +static void xe_pmu_unregister(void *arg)
> +{
> + struct xe_pmu *pmu = arg;
> +
> + if (!pmu->base.event_init)
> + return;
> +
> + /*
> + * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
> + * ensures all currently executing ones will have exited before we
> + * proceed with unregistration.
> + */
> + pmu->closed = true;
> + synchronize_rcu();
> +
> + xe_pmu_unregister_cpuhp_state(pmu);
> +
> + perf_pmu_unregister(&pmu->base);
> + pmu->base.event_init = NULL;
> + kfree(pmu->base.attr_groups);
> + kfree(pmu->name);
> + free_event_attributes(pmu);
> +}
> +
> +void xe_pmu_register(struct xe_pmu *pmu)
> +{
> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
> + const struct attribute_group *attr_groups[] = {
> + &pmu->events_attr_group,
> + &xe_pmu_cpumask_attr_group,
> + NULL
> + };
> +
> + int ret = -ENOMEM;
> +
> + spin_lock_init(&pmu->lock);
> + pmu->cpuhp.cpu = -1;
> +
> + pmu->name = kasprintf(GFP_KERNEL,
> + "xe_%s",
> + dev_name(xe->drm.dev));
> + if (pmu->name)
> + /* tools/perf reserves colons as special. */
> + strreplace((char *)pmu->name, ':', '_');
> +
> + if (!pmu->name)
> + goto err;
> +
> + pmu->events_attr_group.name = "events";
> + pmu->events_attr_group.attrs = create_event_attributes(pmu);
> + if (!pmu->events_attr_group.attrs)
> + goto err_name;
> +
> + pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
> + GFP_KERNEL);
> + if (!pmu->base.attr_groups)
> + goto err_attr;
> +
> + pmu->base.module = THIS_MODULE;
> + pmu->base.task_ctx_nr = perf_invalid_context;
> + pmu->base.event_init = xe_pmu_event_init;
> + pmu->base.add = xe_pmu_event_add;
> + pmu->base.del = xe_pmu_event_del;
> + pmu->base.start = xe_pmu_event_start;
> + pmu->base.stop = xe_pmu_event_stop;
> + pmu->base.read = xe_pmu_event_read;
> + pmu->base.event_idx = xe_pmu_event_event_idx;
> +
> + ret = perf_pmu_register(&pmu->base, pmu->name, -1);
> + if (ret)
> + goto err_groups;
> +
> + ret = xe_pmu_register_cpuhp_state(pmu);
> + if (ret)
> + goto err_unreg;
> +
> + ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
> + if (ret)
> + goto err_cpuhp;
> +
> + return;
> +
> +err_cpuhp:
> + xe_pmu_unregister_cpuhp_state(pmu);
> +err_unreg:
> + perf_pmu_unregister(&pmu->base);
> +err_groups:
> + kfree(pmu->base.attr_groups);
> +err_attr:
> + pmu->base.event_init = NULL;
> + free_event_attributes(pmu);
> +err_name:
> + kfree(pmu->name);
Needs fix. double free incase of devm_add_action_or_reset failure.
> +err:
> + drm_notice(&xe->drm, "Failed to register PMU!\n");
> +}
> diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
> new file mode 100644
> index 000000000000..8afa256f9dac
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_pmu.h
> @@ -0,0 +1,26 @@
> +/* SPDX-License-Identifier: MIT */
> +/*
> + * Copyright © 2024 Intel Corporation
> + */
> +
> +#ifndef _XE_PMU_H_
> +#define _XE_PMU_H_
> +
> +#include "xe_pmu_types.h"
> +
> +struct xe_gt;
> +
> +#if IS_ENABLED(CONFIG_PERF_EVENTS)
> +int xe_pmu_init(void);
> +void xe_pmu_exit(void);
> +void xe_pmu_register(struct xe_pmu *pmu);
> +void xe_pmu_suspend(struct xe_gt *gt);
> +#else
> +static inline int xe_pmu_init(void) { return 0; }
> +static inline void xe_pmu_exit(void) {}
> +static inline void xe_pmu_register(struct xe_pmu *pmu) {}
> +static inline void xe_pmu_suspend(struct xe_gt *gt) {}
> +#endif
> +
> +#endif
> +
> diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
> new file mode 100644
> index 000000000000..e86e8d7e0356
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_pmu_types.h
> @@ -0,0 +1,67 @@
> +/* SPDX-License-Identifier: MIT */
> +/*
> + * Copyright © 2024 Intel Corporation
> + */
> +
> +#ifndef _XE_PMU_TYPES_H_
> +#define _XE_PMU_TYPES_H_
> +
> +#include <linux/perf_event.h>
> +#include <linux/spinlock_types.h>
> +#include <uapi/drm/xe_drm.h>
> +
> +enum {
> + __XE_SAMPLE_RENDER_GROUP_BUSY,
> + __XE_SAMPLE_COPY_GROUP_BUSY,
> + __XE_SAMPLE_MEDIA_GROUP_BUSY,
> + __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
> + __XE_NUM_PMU_SAMPLERS
> +};
> +
> +#define XE_PMU_MAX_GT 2
> +
> +struct xe_pmu {
> + /**
> + * @cpuhp: Struct used for CPU hotplug handling.
> + */
> + struct {
> + struct hlist_node node;
> + unsigned int cpu;
> + } cpuhp;
> + /**
> + * @base: PMU base.
> + */
> + struct pmu base;
> + /**
> + * @closed: xe is unregistering.
> + */
> + bool closed;
> + /**
> + * @name: Name as registered with perf core.
> + */
> + const char *name;
> + /**
> + * @lock: Lock protecting enable mask and ref count handling.
> + */
> + spinlock_t lock;
> + /**
> + * @sample: Current and previous (raw) counters.
> + *
> + * These counters are updated when the device is awake.
> + */
> + u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
> + /**
> + * @events_attr_group: Device events attribute group.
> + */
> + struct attribute_group events_attr_group;
> + /**
> + * @xe_attr: Memory block holding device attributes.
> + */
> + void *xe_attr;
> + /**
> + * @pmu_attr: Memory block holding device attributes.
> + */
> + void *pmu_attr;
> +};
> +
> +#endif
> diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
> index d7b0903c22b2..07ca545354f7 100644
> --- a/include/uapi/drm/xe_drm.h
> +++ b/include/uapi/drm/xe_drm.h
> @@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
> __u64 reserved[2];
> };
>
> +/**
> + * DOC: XE PMU event config IDs
> + *
> + * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in xe_drm.h
> + * in 'struct perf_event_attr' as part of perf_event_open syscall to read a
> + * particular event.
> + *
> + * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
> + *
> + * .. code-block:: C
> + *
> + * struct perf_event_attr attr;
> + * long long count;
> + * int cpu = 0;
> + * int fd;
> + *
> + * memset(&attr, 0, sizeof(struct perf_event_attr));
> + * attr.type = type; // eg: /sys/bus/event_source/devices/xe_0000_56_00.0/type
> + * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
> + * attr.use_clockid = 1;
> + * attr.clockid = CLOCK_MONOTONIC;
> + * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
> + *
> + * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
> + */
> +
> +/*
> + * Top bits of every counter are GT id.
> + */
> +#define __XE_PMU_GT_SHIFT (56)
> +
> +#define ___XE_PMU_OTHER(gt, x) \
> + (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
> +
> +#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
> +#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
> +#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
> +#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
> +
> #if defined(__cplusplus)
> }
> #endif
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface
2024-06-14 15:34 ` Dixit, Ashutosh
@ 2024-06-20 19:15 ` Umesh Nerlige Ramappa
2024-06-21 4:29 ` Aravind Iddamsetty
0 siblings, 1 reply; 32+ messages in thread
From: Umesh Nerlige Ramappa @ 2024-06-20 19:15 UTC (permalink / raw)
To: Dixit, Ashutosh
Cc: Riana Tauro, intel-xe, anshuman.gupta, aravind.iddamsetty,
rodrigo.vivi, krishnaiah.bommu
On Fri, Jun 14, 2024 at 08:34:30AM -0700, Dixit, Ashutosh wrote:
>On Thu, 13 Jun 2024 22:50:39 -0700, Riana Tauro wrote:
>>
>> Hi Ashutosh
>>
>> On 6/14/2024 12:09 AM, Dixit, Ashutosh wrote:
>> > On Thu, 13 Jun 2024 03:04:09 -0700, Riana Tauro wrote:
>> >>
>> >
>> > Hi Riana,
>> >
>> >> There are a set of engine group busyness counters provided by HW which are
>> >> perfect fit to be exposed via PMU perf events.
>> >>
>> >> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>> >>
>> >> events can be listed using:
>> >> perf list
>> >> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>> >> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>> >> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>> >> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>> >
>> > PMU patches merged previously were dropped in 90a8b23f9b85 ("drm/xe/pmu:
>> > Remove PMU from Xe till uapi is finalized") because PMU uapi was expected
>> > to change. Why are we re-posting these old patches again now, without
>> > including the planned uapi changes?
>>
>> The uapi changes were dropped and there are no other upcoming changes for
>> Group busyness. So re-posted the old series.
>
>What happened to VF busyness (which is why I thought the uapi was going to
>change)?
There are no plans to support group busyness from a VF, so we are just
exporting group busyness to maintain parity with i915 for Native/PF
behavior. The only change would be to expose the counters in ticks
rather than ns. I still have to look at this series to see if that's
happening.
Regards,
Umesh
>
>>
>> Thanks,
>> Riana
>> >
>> > Thanks.
>> > --
>> > Ashutosh
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-13 10:04 ` [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface Riana Tauro
2024-06-14 16:15 ` Lucas De Marchi
2024-06-14 20:54 ` Ghimiray, Himal Prasad
@ 2024-06-20 19:52 ` Umesh Nerlige Ramappa
2024-06-27 6:49 ` Aravind Iddamsetty
2024-06-28 15:55 ` Lucas De Marchi
2 siblings, 2 replies; 32+ messages in thread
From: Umesh Nerlige Ramappa @ 2024-06-20 19:52 UTC (permalink / raw)
To: Riana Tauro
Cc: intel-xe, anshuman.gupta, ashutosh.dixit, aravind.iddamsetty,
rodrigo.vivi, krishnaiah.bommu, lucas.demarchi
On Thu, Jun 13, 2024 at 03:34:11PM +0530, Riana Tauro wrote:
>From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>
>There are a set of engine group busyness counters provided by HW which are
>perfect fit to be exposed via PMU perf events.
>
>BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>
>events can be listed using:
>perf list
> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>
>and can be read using:
>
>perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
> time counts unit events
> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>
>The pmu base implementation is taken from i915.
>
>v2:
>Store last known value when device is awake return that while the GT is
>suspended and then update the driver copy when read during awake.
>
>v3:
>1. drop init_samples, as storing counters before going to suspend should
>be sufficient.
>2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>dropped helpers to store and read samples.
>3. use xe_device_mem_access_get_if_ongoing to check if device is active
>before reading the OA registers.
>4. dropped format attr as no longer needed
>5. introduce xe_pmu_suspend to call engine_group_busyness_store
>6. few other nits.
>
>v4: minor nits.
>
>v5: take forcewake when accessing the OAG registers
>
>v6:
>1. drop engine_busyness_sample_type
>2. update UAPI documentation
>
>v7:
>1. update UAPI documentation
>2. drop MEDIA_GT specific change for media busyness counter.
>
>v8:
>1. rebase
>2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>3. remove interrupts pmu event
>
>v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>
>Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
>Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>---
> drivers/gpu/drm/xe/Makefile | 2 +
> drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
> drivers/gpu/drm/xe/xe_device.c | 2 +
> drivers/gpu/drm/xe/xe_device_types.h | 4 +
> drivers/gpu/drm/xe/xe_gt.c | 2 +
> drivers/gpu/drm/xe/xe_module.c | 5 +
> drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
> drivers/gpu/drm/xe/xe_pmu.h | 26 ++
> drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
> include/uapi/drm/xe_drm.h | 39 ++
> 10 files changed, 783 insertions(+)
> create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
> create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
> create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>
>diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>index cbf961b90237..83bf1e07669b 100644
>--- a/drivers/gpu/drm/xe/Makefile
>+++ b/drivers/gpu/drm/xe/Makefile
>@@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
> i915-display/skl_universal_plane.o \
> i915-display/skl_watermark.o
>
>+xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>+
> ifeq ($(CONFIG_ACPI),y)
> xe-$(CONFIG_DRM_XE_DISPLAY) += \
> i915-display/intel_acpi.o \
>diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>index 47c26c37608d..22821dcd4e1b 100644
>--- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>+++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>@@ -390,6 +390,11 @@
> #define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
> #define GLOBAL_INVALIDATION_MODE REG_BIT(2)
>
>+#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
>+#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
>+#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
>+#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
>+
> #define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
> #define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
>
>diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
>index 64691a56d59c..bb00c8c9ec9b 100644
>--- a/drivers/gpu/drm/xe/xe_device.c
>+++ b/drivers/gpu/drm/xe/xe_device.c
>@@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
>
> xe_hwmon_register(xe);
>
>+ xe_pmu_register(&xe->pmu);
>+
> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>
> err_fini_display:
>diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
>index 52bc461171d5..a5dba7325cf1 100644
>--- a/drivers/gpu/drm/xe/xe_device_types.h
>+++ b/drivers/gpu/drm/xe/xe_device_types.h
>@@ -18,6 +18,7 @@
> #include "xe_lmtt_types.h"
> #include "xe_memirq_types.h"
> #include "xe_platform_types.h"
>+#include "xe_pmu.h"
> #include "xe_pt_types.h"
> #include "xe_sriov_types.h"
> #include "xe_step_types.h"
>@@ -473,6 +474,9 @@ struct xe_device {
> int mode;
> } wedged;
>
>+ /** @pmu: performance monitoring unit */
>+ struct xe_pmu pmu;
>+
> /* private: */
>
> #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
>diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
>index 57d84751e160..477d0ae5f230 100644
>--- a/drivers/gpu/drm/xe/xe_gt.c
>+++ b/drivers/gpu/drm/xe/xe_gt.c
>@@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
> if (err)
> goto err_msg;
>
>+ xe_pmu_suspend(gt);
>+
> err = xe_uc_suspend(>->uc);
> if (err)
> goto err_force_wake;
>diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
>index 3edeb30d5ccb..26f814f97fc2 100644
>--- a/drivers/gpu/drm/xe/xe_module.c
>+++ b/drivers/gpu/drm/xe/xe_module.c
>@@ -11,6 +11,7 @@
> #include "xe_drv.h"
> #include "xe_hw_fence.h"
> #include "xe_pci.h"
>+#include "xe_pmu.h"
> #include "xe_sched_job.h"
>
> struct xe_modparam xe_modparam = {
>@@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
> .init = xe_sched_job_module_init,
> .exit = xe_sched_job_module_exit,
> },
>+ {
>+ .init = xe_pmu_init,
>+ .exit = xe_pmu_exit,
>+ },
> {
> .init = xe_register_pci_driver,
> .exit = xe_unregister_pci_driver,
>diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>new file mode 100644
>index 000000000000..64960a358af2
>--- /dev/null
>+++ b/drivers/gpu/drm/xe/xe_pmu.c
>@@ -0,0 +1,631 @@
>+// SPDX-License-Identifier: MIT
>+/*
>+ * Copyright © 2024 Intel Corporation
>+ */
>+
>+#include <drm/drm_drv.h>
>+#include <drm/drm_managed.h>
>+#include <drm/xe_drm.h>
>+
>+#include "regs/xe_gt_regs.h"
>+#include "xe_device.h"
>+#include "xe_force_wake.h"
>+#include "xe_gt_clock.h"
>+#include "xe_mmio.h"
>+#include "xe_macros.h"
>+#include "xe_pm.h"
>+
>+static cpumask_t xe_pmu_cpumask;
>+static unsigned int xe_pmu_target_cpu = -1;
>+
>+static unsigned int config_gt_id(const u64 config)
>+{
>+ return config >> __XE_PMU_GT_SHIFT;
>+}
>+
>+static u64 config_counter(const u64 config)
>+{
>+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>+}
>+
>+static void xe_pmu_event_destroy(struct perf_event *event)
>+{
>+ struct xe_device *xe =
>+ container_of(event->pmu, typeof(*xe), pmu.base);
>+
>+ drm_WARN_ON(&xe->drm, event->parent);
>+
>+ drm_dev_put(&xe->drm);
>+}
>+
>+static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
>+{
>+ u64 val;
>+
>+ switch (sample_type) {
>+ case __XE_SAMPLE_RENDER_GROUP_BUSY:
>+ val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>+ break;
>+ case __XE_SAMPLE_COPY_GROUP_BUSY:
>+ val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>+ break;
>+ case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>+ val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>+ break;
>+ case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>+ val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>+ break;
>+ default:
>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>+ }
>+
>+ return xe_gt_clock_cycles_to_ns(gt, val * 16);
>+}
>+
>+static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>+{
>+ int sample_type = config_counter(config);
>+ const unsigned int gt_id = gt->info.id;
>+ struct xe_device *xe = gt->tile->xe;
>+ struct xe_pmu *pmu = &xe->pmu;
>+ unsigned long flags;
>+ bool device_awake;
>+ u64 val;
>+
>+ device_awake = xe_pm_runtime_get_if_active(xe);
>+ if (device_awake) {
>+ XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>+ val = __engine_group_busyness_read(gt, sample_type);
>+ XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>+ xe_pm_runtime_put(xe);
>+ }
>+
>+ spin_lock_irqsave(&pmu->lock, flags);
>+
>+ if (device_awake)
>+ pmu->sample[gt_id][sample_type] = val;
>+ else
>+ val = pmu->sample[gt_id][sample_type];
>+
>+ spin_unlock_irqrestore(&pmu->lock, flags);
>+
>+ return val;
>+}
>+
>+static void engine_group_busyness_store(struct xe_gt *gt)
>+{
>+ struct xe_pmu *pmu = >->tile->xe->pmu;
>+ unsigned int gt_id = gt->info.id;
>+ unsigned long flags;
>+ int i;
>+
>+ spin_lock_irqsave(&pmu->lock, flags);
>+
>+ for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>+ pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>+
>+ spin_unlock_irqrestore(&pmu->lock, flags);
>+}
>+
>+static int
>+config_status(struct xe_device *xe, u64 config)
>+{
>+ unsigned int gt_id = config_gt_id(config);
>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>+
>+ if (gt_id >= XE_PMU_MAX_GT)
>+ return -ENOENT;
>+
>+ switch (config_counter(config)) {
>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>+ case XE_PMU_COPY_GROUP_BUSY(0):
>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>+ if (gt->info.type == XE_GT_TYPE_MEDIA)
>+ return -ENOENT;
>+ break;
>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>+ if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
>+ return -ENOENT;
>+ break;
>+ default:
>+ return -ENOENT;
>+ }
>+
>+ return 0;
>+}
>+
>+static int xe_pmu_event_init(struct perf_event *event)
>+{
>+ struct xe_device *xe =
>+ container_of(event->pmu, typeof(*xe), pmu.base);
>+ struct xe_pmu *pmu = &xe->pmu;
>+ int ret;
>+
>+ if (pmu->closed)
>+ return -ENODEV;
>+
>+ if (event->attr.type != event->pmu->type)
>+ return -ENOENT;
>+
>+ /* unsupported modes and filters */
>+ if (event->attr.sample_period) /* no sampling */
>+ return -EINVAL;
>+
>+ if (has_branch_stack(event))
>+ return -EOPNOTSUPP;
>+
>+ if (event->cpu < 0)
>+ return -EINVAL;
>+
>+ /* only allow running on one cpu at a time */
>+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>+ return -EINVAL;
>+
>+ ret = config_status(xe, event->attr.config);
>+ if (ret)
>+ return ret;
>+
>+ if (!event->parent) {
>+ drm_dev_get(&xe->drm);
>+ event->destroy = xe_pmu_event_destroy;
>+ }
>+
>+ return 0;
>+}
>+
>+static u64 __xe_pmu_event_read(struct perf_event *event)
>+{
>+ struct xe_device *xe =
>+ container_of(event->pmu, typeof(*xe), pmu.base);
>+ const unsigned int gt_id = config_gt_id(event->attr.config);
>+ const u64 config = event->attr.config;
>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>+ u64 val;
>+
>+ switch (config_counter(config)) {
>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>+ case XE_PMU_COPY_GROUP_BUSY(0):
>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>+ val = engine_group_busyness_read(gt, config);
>+ break;
>+ default:
>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>+ }
>+
>+ return val;
>+}
>+
>+static void xe_pmu_event_read(struct perf_event *event)
>+{
>+ struct xe_device *xe =
>+ container_of(event->pmu, typeof(*xe), pmu.base);
>+ struct hw_perf_event *hwc = &event->hw;
>+ struct xe_pmu *pmu = &xe->pmu;
>+ u64 prev, new;
>+
>+ if (pmu->closed) {
>+ event->hw.state = PERF_HES_STOPPED;
>+ return;
>+ }
>+again:
>+ prev = local64_read(&hwc->prev_count);
>+ new = __xe_pmu_event_read(event);
>+
>+ if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>+ goto again;
>+
>+ local64_add(new - prev, &event->count);
>+}
>+
>+static void xe_pmu_enable(struct perf_event *event)
>+{
>+ /*
>+ * Store the current counter value so we can report the correct delta
>+ * for all listeners. Even when the event was already enabled and has
>+ * an existing non-zero value.
>+ */
>+ local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>+}
>+
>+static void xe_pmu_event_start(struct perf_event *event, int flags)
>+{
>+ struct xe_device *xe =
>+ container_of(event->pmu, typeof(*xe), pmu.base);
>+ struct xe_pmu *pmu = &xe->pmu;
>+
>+ if (pmu->closed)
>+ return;
>+
>+ xe_pmu_enable(event);
>+ event->hw.state = 0;
>+}
>+
>+static void xe_pmu_event_stop(struct perf_event *event, int flags)
>+{
>+ if (flags & PERF_EF_UPDATE)
>+ xe_pmu_event_read(event);
>+
>+ event->hw.state = PERF_HES_STOPPED;
>+}
>+
>+static int xe_pmu_event_add(struct perf_event *event, int flags)
>+{
>+ struct xe_device *xe =
>+ container_of(event->pmu, typeof(*xe), pmu.base);
>+ struct xe_pmu *pmu = &xe->pmu;
>+
>+ if (pmu->closed)
>+ return -ENODEV;
>+
>+ if (flags & PERF_EF_START)
>+ xe_pmu_event_start(event, flags);
>+
>+ return 0;
>+}
>+
>+static void xe_pmu_event_del(struct perf_event *event, int flags)
>+{
>+ xe_pmu_event_stop(event, PERF_EF_UPDATE);
>+}
>+
>+static int xe_pmu_event_event_idx(struct perf_event *event)
>+{
>+ return 0;
>+}
>+
>+struct xe_ext_attribute {
>+ struct device_attribute attr;
>+ unsigned long val;
>+};
>+
>+static ssize_t xe_pmu_event_show(struct device *dev,
>+ struct device_attribute *attr, char *buf)
>+{
>+ struct xe_ext_attribute *eattr;
>+
>+ eattr = container_of(attr, struct xe_ext_attribute, attr);
>+ return sprintf(buf, "config=0x%lx\n", eattr->val);
>+}
>+
>+static ssize_t cpumask_show(struct device *dev,
>+ struct device_attribute *attr, char *buf)
>+{
>+ return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>+}
>+
>+static DEVICE_ATTR_RO(cpumask);
>+
>+static struct attribute *xe_cpumask_attrs[] = {
>+ &dev_attr_cpumask.attr,
>+ NULL,
>+};
>+
>+static const struct attribute_group xe_pmu_cpumask_attr_group = {
>+ .attrs = xe_cpumask_attrs,
>+};
>+
>+#define __event(__counter, __name, __unit) \
>+{ \
>+ .counter = (__counter), \
>+ .name = (__name), \
>+ .unit = (__unit), \
>+}
>+
>+static struct xe_ext_attribute *
>+add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>+{
>+ sysfs_attr_init(&attr->attr.attr);
>+ attr->attr.attr.name = name;
>+ attr->attr.attr.mode = 0444;
>+ attr->attr.show = xe_pmu_event_show;
>+ attr->val = config;
>+
>+ return ++attr;
>+}
>+
>+static struct perf_pmu_events_attr *
>+add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>+ const char *str)
>+{
>+ sysfs_attr_init(&attr->attr.attr);
>+ attr->attr.attr.name = name;
>+ attr->attr.attr.mode = 0444;
>+ attr->attr.show = perf_event_sysfs_show;
>+ attr->event_str = str;
>+
>+ return ++attr;
>+}
>+
>+static struct attribute **
>+create_event_attributes(struct xe_pmu *pmu)
>+{
>+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>+ static const struct {
>+ unsigned int counter;
>+ const char *name;
>+ const char *unit;
>+ } events[] = {
>+ __event(0, "render-group-busy", "ns"),
>+ __event(1, "copy-group-busy", "ns"),
>+ __event(2, "media-group-busy", "ns"),
>+ __event(3, "any-engine-group-busy", "ns"),
>+ };
>+
>+ struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>+ struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>+ struct attribute **attr = NULL, **attr_iter;
>+ unsigned int count = 0;
>+ unsigned int i, j;
>+ struct xe_gt *gt;
>+
>+ /* Count how many counters we will be exposing. */
>+ for_each_gt(gt, xe, j) {
>+ for (i = 0; i < ARRAY_SIZE(events); i++) {
>+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>+
>+ if (!config_status(xe, config))
>+ count++;
>+ }
>+ }
>+
>+ /* Allocate attribute objects and table. */
>+ xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>+ if (!xe_attr)
>+ goto err_alloc;
>+
>+ pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>+ if (!pmu_attr)
>+ goto err_alloc;
>+
>+ /* Max one pointer of each attribute type plus a termination entry. */
>+ attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>+ if (!attr)
>+ goto err_alloc;
>+
>+ xe_iter = xe_attr;
>+ pmu_iter = pmu_attr;
>+ attr_iter = attr;
>+
>+ for_each_gt(gt, xe, j) {
>+ for (i = 0; i < ARRAY_SIZE(events); i++) {
>+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>+ char *str;
>+
>+ if (config_status(xe, config))
>+ continue;
>+
>+ str = kasprintf(GFP_KERNEL, "%s-gt%u",
>+ events[i].name, j);
>+ if (!str)
>+ goto err;
>+
>+ *attr_iter++ = &xe_iter->attr.attr;
>+ xe_iter = add_xe_attr(xe_iter, str, config);
>+
>+ if (events[i].unit) {
>+ str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>+ events[i].name, j);
>+ if (!str)
>+ goto err;
>+
>+ *attr_iter++ = &pmu_iter->attr.attr;
>+ pmu_iter = add_pmu_attr(pmu_iter, str,
>+ events[i].unit);
>+ }
>+ }
>+ }
>+
>+ pmu->xe_attr = xe_attr;
>+ pmu->pmu_attr = pmu_attr;
>+
>+ return attr;
>+
>+err:
>+ for (attr_iter = attr; *attr_iter; attr_iter++)
>+ kfree((*attr_iter)->name);
>+
>+err_alloc:
>+ kfree(attr);
>+ kfree(xe_attr);
>+ kfree(pmu_attr);
>+
>+ return NULL;
>+}
>+
>+static void free_event_attributes(struct xe_pmu *pmu)
>+{
>+ struct attribute **attr_iter = pmu->events_attr_group.attrs;
>+
>+ for (; *attr_iter; attr_iter++)
>+ kfree((*attr_iter)->name);
>+
>+ kfree(pmu->events_attr_group.attrs);
>+ kfree(pmu->xe_attr);
>+ kfree(pmu->pmu_attr);
>+
>+ pmu->events_attr_group.attrs = NULL;
>+ pmu->xe_attr = NULL;
>+ pmu->pmu_attr = NULL;
>+}
>+
>+static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>+{
>+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>+
>+ /* Select the first online CPU as a designated reader. */
>+ if (cpumask_empty(&xe_pmu_cpumask))
>+ cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>+
>+ return 0;
>+}
>+
>+static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>+{
>+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>+ unsigned int target = xe_pmu_target_cpu;
>+
>+ /*
>+ * Unregistering an instance generates a CPU offline event which we must
>+ * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>+ */
>+ if (pmu->closed)
>+ return 0;
>+
>+ if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>+ target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>+
>+ /* Migrate events if there is a valid target */
>+ if (target < nr_cpu_ids) {
>+ cpumask_set_cpu(target, &xe_pmu_cpumask);
>+ xe_pmu_target_cpu = target;
>+ }
>+ }
>+
>+ if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>+ perf_pmu_migrate_context(&pmu->base, cpu, target);
>+ pmu->cpuhp.cpu = target;
>+ }
>+
>+ return 0;
>+}
>+
>+static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>+
>+int xe_pmu_init(void)
>+{
>+ int ret;
>+
>+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>+ "perf/x86/intel/xe:online",
>+ xe_pmu_cpu_online,
>+ xe_pmu_cpu_offline);
>+ if (ret < 0)
>+ pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>+ ret);
>+ else
>+ cpuhp_slot = ret;
>+
>+ return 0;
>+}
>+
>+void xe_pmu_exit(void)
>+{
>+ if (cpuhp_slot != CPUHP_INVALID)
>+ cpuhp_remove_multi_state(cpuhp_slot);
>+}
>+
>+static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>+{
>+ if (cpuhp_slot == CPUHP_INVALID)
>+ return -EINVAL;
>+
>+ return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>+}
>+
>+static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>+{
>+ cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>+}
>+
>+void xe_pmu_suspend(struct xe_gt *gt)
>+{
>+ engine_group_busyness_store(gt);
>+}
>+
>+static void xe_pmu_unregister(void *arg)
>+{
>+ struct xe_pmu *pmu = arg;
>+
>+ if (!pmu->base.event_init)
>+ return;
>+
>+ /*
>+ * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
>+ * ensures all currently executing ones will have exited before we
>+ * proceed with unregistration.
>+ */
>+ pmu->closed = true;
>+ synchronize_rcu();
>+
>+ xe_pmu_unregister_cpuhp_state(pmu);
>+
>+ perf_pmu_unregister(&pmu->base);
>+ pmu->base.event_init = NULL;
>+ kfree(pmu->base.attr_groups);
>+ kfree(pmu->name);
>+ free_event_attributes(pmu);
>+}
>+
>+void xe_pmu_register(struct xe_pmu *pmu)
>+{
>+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>+ const struct attribute_group *attr_groups[] = {
>+ &pmu->events_attr_group,
>+ &xe_pmu_cpumask_attr_group,
>+ NULL
>+ };
>+
>+ int ret = -ENOMEM;
>+
>+ spin_lock_init(&pmu->lock);
>+ pmu->cpuhp.cpu = -1;
>+
>+ pmu->name = kasprintf(GFP_KERNEL,
>+ "xe_%s",
>+ dev_name(xe->drm.dev));
>+ if (pmu->name)
>+ /* tools/perf reserves colons as special. */
>+ strreplace((char *)pmu->name, ':', '_');
>+
>+ if (!pmu->name)
>+ goto err;
>+
>+ pmu->events_attr_group.name = "events";
>+ pmu->events_attr_group.attrs = create_event_attributes(pmu);
>+ if (!pmu->events_attr_group.attrs)
>+ goto err_name;
>+
>+ pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>+ GFP_KERNEL);
>+ if (!pmu->base.attr_groups)
>+ goto err_attr;
>+
>+ pmu->base.module = THIS_MODULE;
>+ pmu->base.task_ctx_nr = perf_invalid_context;
>+ pmu->base.event_init = xe_pmu_event_init;
>+ pmu->base.add = xe_pmu_event_add;
>+ pmu->base.del = xe_pmu_event_del;
>+ pmu->base.start = xe_pmu_event_start;
>+ pmu->base.stop = xe_pmu_event_stop;
>+ pmu->base.read = xe_pmu_event_read;
>+ pmu->base.event_idx = xe_pmu_event_event_idx;
>+
>+ ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>+ if (ret)
>+ goto err_groups;
>+
>+ ret = xe_pmu_register_cpuhp_state(pmu);
>+ if (ret)
>+ goto err_unreg;
>+
>+ ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>+ if (ret)
>+ goto err_cpuhp;
>+
>+ return;
>+
>+err_cpuhp:
>+ xe_pmu_unregister_cpuhp_state(pmu);
>+err_unreg:
>+ perf_pmu_unregister(&pmu->base);
>+err_groups:
>+ kfree(pmu->base.attr_groups);
>+err_attr:
>+ pmu->base.event_init = NULL;
>+ free_event_attributes(pmu);
>+err_name:
>+ kfree(pmu->name);
>+err:
>+ drm_notice(&xe->drm, "Failed to register PMU!\n");
>+}
>diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>new file mode 100644
>index 000000000000..8afa256f9dac
>--- /dev/null
>+++ b/drivers/gpu/drm/xe/xe_pmu.h
>@@ -0,0 +1,26 @@
>+/* SPDX-License-Identifier: MIT */
>+/*
>+ * Copyright © 2024 Intel Corporation
>+ */
>+
>+#ifndef _XE_PMU_H_
>+#define _XE_PMU_H_
>+
>+#include "xe_pmu_types.h"
>+
>+struct xe_gt;
>+
>+#if IS_ENABLED(CONFIG_PERF_EVENTS)
>+int xe_pmu_init(void);
>+void xe_pmu_exit(void);
>+void xe_pmu_register(struct xe_pmu *pmu);
>+void xe_pmu_suspend(struct xe_gt *gt);
>+#else
>+static inline int xe_pmu_init(void) { return 0; }
>+static inline void xe_pmu_exit(void) {}
>+static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>+static inline void xe_pmu_suspend(struct xe_gt *gt) {}
>+#endif
>+
>+#endif
>+
>diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
>new file mode 100644
>index 000000000000..e86e8d7e0356
>--- /dev/null
>+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>@@ -0,0 +1,67 @@
>+/* SPDX-License-Identifier: MIT */
>+/*
>+ * Copyright © 2024 Intel Corporation
>+ */
>+
>+#ifndef _XE_PMU_TYPES_H_
>+#define _XE_PMU_TYPES_H_
>+
>+#include <linux/perf_event.h>
>+#include <linux/spinlock_types.h>
>+#include <uapi/drm/xe_drm.h>
>+
>+enum {
>+ __XE_SAMPLE_RENDER_GROUP_BUSY,
>+ __XE_SAMPLE_COPY_GROUP_BUSY,
>+ __XE_SAMPLE_MEDIA_GROUP_BUSY,
>+ __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
>+ __XE_NUM_PMU_SAMPLERS
>+};
>+
>+#define XE_PMU_MAX_GT 2
>+
>+struct xe_pmu {
>+ /**
>+ * @cpuhp: Struct used for CPU hotplug handling.
>+ */
>+ struct {
>+ struct hlist_node node;
>+ unsigned int cpu;
>+ } cpuhp;
>+ /**
>+ * @base: PMU base.
>+ */
>+ struct pmu base;
>+ /**
>+ * @closed: xe is unregistering.
>+ */
>+ bool closed;
>+ /**
>+ * @name: Name as registered with perf core.
>+ */
>+ const char *name;
>+ /**
>+ * @lock: Lock protecting enable mask and ref count handling.
>+ */
>+ spinlock_t lock;
>+ /**
>+ * @sample: Current and previous (raw) counters.
>+ *
>+ * These counters are updated when the device is awake.
>+ */
>+ u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>+ /**
>+ * @events_attr_group: Device events attribute group.
>+ */
>+ struct attribute_group events_attr_group;
>+ /**
>+ * @xe_attr: Memory block holding device attributes.
>+ */
>+ void *xe_attr;
>+ /**
>+ * @pmu_attr: Memory block holding device attributes.
>+ */
>+ void *pmu_attr;
>+};
>+
>+#endif
>diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>index d7b0903c22b2..07ca545354f7 100644
>--- a/include/uapi/drm/xe_drm.h
>+++ b/include/uapi/drm/xe_drm.h
>@@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
> __u64 reserved[2];
> };
>
>+/**
>+ * DOC: XE PMU event config IDs
>+ *
>+ * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in xe_drm.h
>+ * in 'struct perf_event_attr' as part of perf_event_open syscall to read a
>+ * particular event.
>+ *
>+ * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
>+ *
>+ * .. code-block:: C
>+ *
>+ * struct perf_event_attr attr;
>+ * long long count;
>+ * int cpu = 0;
>+ * int fd;
>+ *
>+ * memset(&attr, 0, sizeof(struct perf_event_attr));
>+ * attr.type = type; // eg: /sys/bus/event_source/devices/xe_0000_56_00.0/type
>+ * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
>+ * attr.use_clockid = 1;
>+ * attr.clockid = CLOCK_MONOTONIC;
>+ * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
>+ *
>+ * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
>+ */
>+
>+/*
>+ * Top bits of every counter are GT id.
>+ */
>+#define __XE_PMU_GT_SHIFT (56)
>+
>+#define ___XE_PMU_OTHER(gt, x) \
>+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>+
>+#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
>+#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
>+#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
>+#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
+ Lucas for inputs
We should align this to the interface planned for other PMU busyness
counters as well as how we do PCEU. i.e.
1) counters are in ticks
2) total time in ticks is also exported to the user.
For 1), I would just append TICKS to the counter names and drop the
conversion to _ns in __engine_group_busyness_read(). Also, drop the
patch that adds this conversion helper.
For 2) define a new counter - total active ticks that would return the
'CPU' timestamp converted to gpu ticks. The reason I am insisting on CPU
timestamp here is because we want to have a time base that is ticking
even when the GPU is idle.
Regards,
Umesh
>+
> #if defined(__cplusplus)
> }
> #endif
>--
>2.40.0
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface
2024-06-20 19:15 ` Umesh Nerlige Ramappa
@ 2024-06-21 4:29 ` Aravind Iddamsetty
2024-06-21 18:32 ` Umesh Nerlige Ramappa
0 siblings, 1 reply; 32+ messages in thread
From: Aravind Iddamsetty @ 2024-06-21 4:29 UTC (permalink / raw)
To: Umesh Nerlige Ramappa, Dixit, Ashutosh
Cc: Riana Tauro, intel-xe, anshuman.gupta, rodrigo.vivi,
krishnaiah.bommu
On 21/06/24 00:45, Umesh Nerlige Ramappa wrote:
> On Fri, Jun 14, 2024 at 08:34:30AM -0700, Dixit, Ashutosh wrote:
>> On Thu, 13 Jun 2024 22:50:39 -0700, Riana Tauro wrote:
>>>
>>> Hi Ashutosh
>>>
>>> On 6/14/2024 12:09 AM, Dixit, Ashutosh wrote:
>>> > On Thu, 13 Jun 2024 03:04:09 -0700, Riana Tauro wrote:
>>> >>
>>> >
>>> > Hi Riana,
>>> >
>>> >> There are a set of engine group busyness counters provided by HW which are
>>> >> perfect fit to be exposed via PMU perf events.
>>> >>
>>> >> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>> >>
>>> >> events can be listed using:
>>> >> perf list
>>> >> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>>> >> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>>> >> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>>> >> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>> >
>>> > PMU patches merged previously were dropped in 90a8b23f9b85 ("drm/xe/pmu:
>>> > Remove PMU from Xe till uapi is finalized") because PMU uapi was expected
>>> > to change. Why are we re-posting these old patches again now, without
>>> > including the planned uapi changes?
>>>
>>> The uapi changes were dropped and there are no other upcoming changes for
>>> Group busyness. So re-posted the old series.
>>
>> What happened to VF busyness (which is why I thought the uapi was going to
>> change)?
>
> There are no plans to support group busyness from a VF, so we are just exporting group busyness to maintain parity with i915 for Native/PF behavior. The only change would be to expose the counters in ticks rather than ns. I still have to look at this series to see if that's happening.
how will the counter ticks be used to get the busyness in time
Regards,
Aravind.
>
> Regards,
> Umesh
>>
>>>
>>> Thanks,
>>> Riana
>>> >
>>> > Thanks.
>>> > --
>>> > Ashutosh
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface
2024-06-21 4:29 ` Aravind Iddamsetty
@ 2024-06-21 18:32 ` Umesh Nerlige Ramappa
2024-06-24 10:27 ` Aravind Iddamsetty
0 siblings, 1 reply; 32+ messages in thread
From: Umesh Nerlige Ramappa @ 2024-06-21 18:32 UTC (permalink / raw)
To: Aravind Iddamsetty
Cc: Dixit, Ashutosh, Riana Tauro, intel-xe, anshuman.gupta,
rodrigo.vivi, krishnaiah.bommu
On Fri, Jun 21, 2024 at 09:59:11AM +0530, Aravind Iddamsetty wrote:
>
>On 21/06/24 00:45, Umesh Nerlige Ramappa wrote:
>> On Fri, Jun 14, 2024 at 08:34:30AM -0700, Dixit, Ashutosh wrote:
>>> On Thu, 13 Jun 2024 22:50:39 -0700, Riana Tauro wrote:
>>>>
>>>> Hi Ashutosh
>>>>
>>>> On 6/14/2024 12:09 AM, Dixit, Ashutosh wrote:
>>>> > On Thu, 13 Jun 2024 03:04:09 -0700, Riana Tauro wrote:
>>>> >>
>>>> >
>>>> > Hi Riana,
>>>> >
>>>> >> There are a set of engine group busyness counters provided by HW which are
>>>> >> perfect fit to be exposed via PMU perf events.
>>>> >>
>>>> >> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>>> >>
>>>> >> events can be listed using:
>>>> >> perf list
>>>> >> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>>>> >> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>>>> >> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>>>> >> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>>> >
>>>> > PMU patches merged previously were dropped in 90a8b23f9b85 ("drm/xe/pmu:
>>>> > Remove PMU from Xe till uapi is finalized") because PMU uapi was expected
>>>> > to change. Why are we re-posting these old patches again now, without
>>>> > including the planned uapi changes?
>>>>
>>>> The uapi changes were dropped and there are no other upcoming changes for
>>>> Group busyness. So re-posted the old series.
>>>
>>> What happened to VF busyness (which is why I thought the uapi was going to
>>> change)?
>>
>> There are no plans to support group busyness from a VF, so we are just exporting group busyness to maintain parity with i915 for Native/PF behavior. The only change would be to expose the counters in ticks rather than ns. I still have to look at this series to see if that's happening.
>
>how will the counter ticks be used to get the busyness in time
There will be no concept of "busyness in time units". User would just
take 2 samples and calculate busyness as a percentage. The counters will
be unitless. More info on equation here:
https://spec.oneapi.io/level-zero/1.9.3/sysman/api.html#_CPPv418zes_engine_stats_t
Percent utilization is calculated by taking two snapshots (s1, s2) and
using the equation:
util = (s2.activeTime - s1.activeTime) / (s2.timestamp - s1.timestamp)
Note that zesEngineGetActivity is used to get single as well as group
engine busyness from the KMD.
For single engine busyness, KMD will get the information from GuC and
that would be a pair of counters - ActiveTicks and TotalActiveTicks
corresponding to activeTime and timestamp in the above equation.
For group engine busyness, we should follow the same semantics.
Regards,
Umesh
>
>Regards,
>Aravind.
>>
>> Regards,
>> Umesh
>>>
>>>>
>>>> Thanks,
>>>> Riana
>>>> >
>>>> > Thanks.
>>>> > --
>>>> > Ashutosh
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface
2024-06-21 18:32 ` Umesh Nerlige Ramappa
@ 2024-06-24 10:27 ` Aravind Iddamsetty
2024-06-24 23:23 ` Umesh Nerlige Ramappa
0 siblings, 1 reply; 32+ messages in thread
From: Aravind Iddamsetty @ 2024-06-24 10:27 UTC (permalink / raw)
To: Umesh Nerlige Ramappa
Cc: Dixit, Ashutosh, Riana Tauro, intel-xe, anshuman.gupta,
rodrigo.vivi, krishnaiah.bommu, tvrtko.ursulin, Joonas Lahtinen
On 22/06/24 00:02, Umesh Nerlige Ramappa wrote:
> On Fri, Jun 21, 2024 at 09:59:11AM +0530, Aravind Iddamsetty wrote:
>>
>> On 21/06/24 00:45, Umesh Nerlige Ramappa wrote:
>>> On Fri, Jun 14, 2024 at 08:34:30AM -0700, Dixit, Ashutosh wrote:
>>>> On Thu, 13 Jun 2024 22:50:39 -0700, Riana Tauro wrote:
>>>>>
>>>>> Hi Ashutosh
>>>>>
>>>>> On 6/14/2024 12:09 AM, Dixit, Ashutosh wrote:
>>>>> > On Thu, 13 Jun 2024 03:04:09 -0700, Riana Tauro wrote:
>>>>> >>
>>>>> >
>>>>> > Hi Riana,
>>>>> >
>>>>> >> There are a set of engine group busyness counters provided by HW which are
>>>>> >> perfect fit to be exposed via PMU perf events.
>>>>> >>
>>>>> >> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>>>> >>
>>>>> >> events can be listed using:
>>>>> >> perf list
>>>>> >> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>>>>> >> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>>>>> >> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>>>>> >> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>>>> >
>>>>> > PMU patches merged previously were dropped in 90a8b23f9b85 ("drm/xe/pmu:
>>>>> > Remove PMU from Xe till uapi is finalized") because PMU uapi was expected
>>>>> > to change. Why are we re-posting these old patches again now, without
>>>>> > including the planned uapi changes?
>>>>>
>>>>> The uapi changes were dropped and there are no other upcoming changes for
>>>>> Group busyness. So re-posted the old series.
>>>>
>>>> What happened to VF busyness (which is why I thought the uapi was going to
>>>> change)?
>>>
>>> There are no plans to support group busyness from a VF, so we are just exporting group busyness to maintain parity with i915 for Native/PF behavior. The only change would be to expose the counters in ticks rather than ns. I still have to look at this series to see if that's happening.
>>
>> how will the counter ticks be used to get the busyness in time
>
> There will be no concept of "busyness in time units". User would just take 2 samples and calculate busyness as a percentage. The counters will be unitless. More info on equation here:
> https://spec.oneapi.io/level-zero/1.9.3/sysman/api.html#_CPPv418zes_engine_stats_t
>
> Percent utilization is calculated by taking two snapshots (s1, s2) and using the equation:
>
> util = (s2.activeTime - s1.activeTime) / (s2.timestamp - s1.timestamp)
>
> Note that zesEngineGetActivity is used to get single as well as group engine busyness from the KMD.
>
> For single engine busyness, KMD will get the information from GuC and that would be a pair of counters - ActiveTicks and TotalActiveTicks corresponding to activeTime and timestamp in the above equation.
>
> For group engine busyness, we should follow the same semantics.
does that mean that the counter itself doesn't have any relevance i mean it doesn't present any outcome to the user and rather one should use some formula to get a usable metric.
so from PMU interface point of view can we expose such a interface, my question because most of the interfaces I found in PMU are directly usable by user.
Thanks,
Aravind.
>
> Regards,
> Umesh
>
>>
>> Regards,
>> Aravind.
>>>
>>> Regards,
>>> Umesh
>>>>
>>>>>
>>>>> Thanks,
>>>>> Riana
>>>>> >
>>>>> > Thanks.
>>>>> > --
>>>>> > Ashutosh
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface
2024-06-24 10:27 ` Aravind Iddamsetty
@ 2024-06-24 23:23 ` Umesh Nerlige Ramappa
0 siblings, 0 replies; 32+ messages in thread
From: Umesh Nerlige Ramappa @ 2024-06-24 23:23 UTC (permalink / raw)
To: Aravind Iddamsetty
Cc: Dixit, Ashutosh, Riana Tauro, intel-xe, anshuman.gupta,
rodrigo.vivi, krishnaiah.bommu, tvrtko.ursulin, Joonas Lahtinen
On Mon, Jun 24, 2024 at 03:57:18PM +0530, Aravind Iddamsetty wrote:
>
>On 22/06/24 00:02, Umesh Nerlige Ramappa wrote:
>> On Fri, Jun 21, 2024 at 09:59:11AM +0530, Aravind Iddamsetty wrote:
>>>
>>> On 21/06/24 00:45, Umesh Nerlige Ramappa wrote:
>>>> On Fri, Jun 14, 2024 at 08:34:30AM -0700, Dixit, Ashutosh wrote:
>>>>> On Thu, 13 Jun 2024 22:50:39 -0700, Riana Tauro wrote:
>>>>>>
>>>>>> Hi Ashutosh
>>>>>>
>>>>>> On 6/14/2024 12:09 AM, Dixit, Ashutosh wrote:
>>>>>> > On Thu, 13 Jun 2024 03:04:09 -0700, Riana Tauro wrote:
>>>>>> >>
>>>>>> >
>>>>>> > Hi Riana,
>>>>>> >
>>>>>> >> There are a set of engine group busyness counters provided by HW which are
>>>>>> >> perfect fit to be exposed via PMU perf events.
>>>>>> >>
>>>>>> >> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>>>>> >>
>>>>>> >> events can be listed using:
>>>>>> >> perf list
>>>>>> >> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>>>>>> >> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>>>>>> >> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>>>>>> >> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>>>>> >
>>>>>> > PMU patches merged previously were dropped in 90a8b23f9b85 ("drm/xe/pmu:
>>>>>> > Remove PMU from Xe till uapi is finalized") because PMU uapi was expected
>>>>>> > to change. Why are we re-posting these old patches again now, without
>>>>>> > including the planned uapi changes?
>>>>>>
>>>>>> The uapi changes were dropped and there are no other upcoming changes for
>>>>>> Group busyness. So re-posted the old series.
>>>>>
>>>>> What happened to VF busyness (which is why I thought the uapi was going to
>>>>> change)?
>>>>
>>>> There are no plans to support group busyness from a VF, so we are just exporting group busyness to maintain parity with i915 for Native/PF behavior. The only change would be to expose the counters in ticks rather than ns. I still have to look at this series to see if that's happening.
>>>
>>> how will the counter ticks be used to get the busyness in time
>>
>> There will be no concept of "busyness in time units". User would just take 2 samples and calculate busyness as a percentage. The counters will be unitless. More info on equation here:
>> https://spec.oneapi.io/level-zero/1.9.3/sysman/api.html#_CPPv418zes_engine_stats_t
>>
>> Percent utilization is calculated by taking two snapshots (s1, s2) and using the equation:
>>
>> util = (s2.activeTime - s1.activeTime) / (s2.timestamp - s1.timestamp)
>>
>> Note that zesEngineGetActivity is used to get single as well as group engine busyness from the KMD.
>>
>> For single engine busyness, KMD will get the information from GuC and that would be a pair of counters - ActiveTicks and TotalActiveTicks corresponding to activeTime and timestamp in the above equation.
>>
>> For group engine busyness, we should follow the same semantics.
>
>does that mean that the counter itself doesn't have any relevance i mean it doesn't present any outcome to the user and rather one should use some formula to get a usable metric.
Correct.
>
>so from PMU interface point of view can we expose such a interface,
It's still a counter that ticks in one direction, so I believe we should
be okay using the PMU interface.
> my question because most of the interfaces I found in PMU are directly usable by user.
That's implementation specific. I don't see PMU imposing that
requirement on us. I agree that it would be intuitive to use the
counters directly, but this interface is required to work with SRIOV.
For example, in a VF, the user cannot use the single counter directly to
determine how busy an engine was. The user also needs to know what
fraction of wall time the VF ran for. That would help calculate per
VF/VM based engine utilization/percentage.
Regards,
Umesh
>
>Thanks,
>Aravind.
>>
>> Regards,
>> Umesh
>>
>>>
>>> Regards,
>>> Aravind.
>>>>
>>>> Regards,
>>>> Umesh
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Riana
>>>>>> >
>>>>>> > Thanks.
>>>>>> > --
>>>>>> > Ashutosh
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-14 20:54 ` Ghimiray, Himal Prasad
@ 2024-06-27 5:21 ` Riana Tauro
0 siblings, 0 replies; 32+ messages in thread
From: Riana Tauro @ 2024-06-27 5:21 UTC (permalink / raw)
To: Ghimiray, Himal Prasad, intel-xe
Cc: anshuman.gupta, ashutosh.dixit, aravind.iddamsetty, rodrigo.vivi,
umesh.nerlige.ramappa, krishnaiah.bommu
On 6/15/2024 2:24 AM, Ghimiray, Himal Prasad wrote:
>
>
> On 13-06-2024 15:34, Riana Tauro wrote:
>> From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>
>> There are a set of engine group busyness counters provided by HW which
>> are
>> perfect fit to be exposed via PMU perf events.
>>
>> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>
>> events can be listed using:
>> perf list
>> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>
>> and can be read using:
>>
>> perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
>> time counts unit events
>> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>
>> The pmu base implementation is taken from i915.
>>
>> v2:
>> Store last known value when device is awake return that while the GT is
>> suspended and then update the driver copy when read during awake.
>>
>> v3:
>> 1. drop init_samples, as storing counters before going to suspend should
>> be sufficient.
>> 2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>> dropped helpers to store and read samples.
>> 3. use xe_device_mem_access_get_if_ongoing to check if device is active
>> before reading the OA registers.
>> 4. dropped format attr as no longer needed
>> 5. introduce xe_pmu_suspend to call engine_group_busyness_store
>> 6. few other nits.
>>
>> v4: minor nits.
>>
>> v5: take forcewake when accessing the OAG registers
>>
>> v6:
>> 1. drop engine_busyness_sample_type
>> 2. update UAPI documentation
>>
>> v7:
>> 1. update UAPI documentation
>> 2. drop MEDIA_GT specific change for media busyness counter.
>>
>> v8:
>> 1. rebase
>> 2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>> 3. remove interrupts pmu event
>>
>> v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>>
>> Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>> Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>> Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
>> Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>> ---
>> drivers/gpu/drm/xe/Makefile | 2 +
>> drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
>> drivers/gpu/drm/xe/xe_device.c | 2 +
>> drivers/gpu/drm/xe/xe_device_types.h | 4 +
>> drivers/gpu/drm/xe/xe_gt.c | 2 +
>> drivers/gpu/drm/xe/xe_module.c | 5 +
>> drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
>> drivers/gpu/drm/xe/xe_pmu.h | 26 ++
>> drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
>> include/uapi/drm/xe_drm.h | 39 ++
>> 10 files changed, 783 insertions(+)
>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
>> create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>>
>> diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>> index cbf961b90237..83bf1e07669b 100644
>> --- a/drivers/gpu/drm/xe/Makefile
>> +++ b/drivers/gpu/drm/xe/Makefile
>> @@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
>> i915-display/skl_universal_plane.o \
>> i915-display/skl_watermark.o
>> +xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>> +
>> ifeq ($(CONFIG_ACPI),y)
>> xe-$(CONFIG_DRM_XE_DISPLAY) += \
>> i915-display/intel_acpi.o \
>> diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>> b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>> index 47c26c37608d..22821dcd4e1b 100644
>> --- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>> +++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>> @@ -390,6 +390,11 @@
>> #define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
>> #define GLOBAL_INVALIDATION_MODE REG_BIT(2)
>> +#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
>> +#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
>> +#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
>> +#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
>> +
>> #define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188,
>> XE_REG_OPTION_MASKED)
>> #define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
>> diff --git a/drivers/gpu/drm/xe/xe_device.c
>> b/drivers/gpu/drm/xe/xe_device.c
>> index 64691a56d59c..bb00c8c9ec9b 100644
>> --- a/drivers/gpu/drm/xe/xe_device.c
>> +++ b/drivers/gpu/drm/xe/xe_device.c
>> @@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
>> xe_hwmon_register(xe);
>> + xe_pmu_register(&xe->pmu);
>> +
>> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize,
>> xe);
>> err_fini_display:
>> diff --git a/drivers/gpu/drm/xe/xe_device_types.h
>> b/drivers/gpu/drm/xe/xe_device_types.h
>> index 52bc461171d5..a5dba7325cf1 100644
>> --- a/drivers/gpu/drm/xe/xe_device_types.h
>> +++ b/drivers/gpu/drm/xe/xe_device_types.h
>> @@ -18,6 +18,7 @@
>> #include "xe_lmtt_types.h"
>> #include "xe_memirq_types.h"
>> #include "xe_platform_types.h"
>> +#include "xe_pmu.h"
>> #include "xe_pt_types.h"
>> #include "xe_sriov_types.h"
>> #include "xe_step_types.h"
>> @@ -473,6 +474,9 @@ struct xe_device {
>> int mode;
>> } wedged;
>> + /** @pmu: performance monitoring unit */
>> + struct xe_pmu pmu;
>> +
>> /* private: */
>> #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
>> diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
>> index 57d84751e160..477d0ae5f230 100644
>> --- a/drivers/gpu/drm/xe/xe_gt.c
>> +++ b/drivers/gpu/drm/xe/xe_gt.c
>> @@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
>> if (err)
>> goto err_msg;
>> + xe_pmu_suspend(gt);
>> +
>> err = xe_uc_suspend(>->uc);
>> if (err)
>> goto err_force_wake;
>> diff --git a/drivers/gpu/drm/xe/xe_module.c
>> b/drivers/gpu/drm/xe/xe_module.c
>> index 3edeb30d5ccb..26f814f97fc2 100644
>> --- a/drivers/gpu/drm/xe/xe_module.c
>> +++ b/drivers/gpu/drm/xe/xe_module.c
>> @@ -11,6 +11,7 @@
>> #include "xe_drv.h"
>> #include "xe_hw_fence.h"
>> #include "xe_pci.h"
>> +#include "xe_pmu.h"
>> #include "xe_sched_job.h"
>> struct xe_modparam xe_modparam = {
>> @@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
>> .init = xe_sched_job_module_init,
>> .exit = xe_sched_job_module_exit,
>> },
>> + {
>> + .init = xe_pmu_init,
>> + .exit = xe_pmu_exit,
>> + },
>> {
>> .init = xe_register_pci_driver,
>> .exit = xe_unregister_pci_driver,
>> diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>> new file mode 100644
>> index 000000000000..64960a358af2
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu.c
>> @@ -0,0 +1,631 @@
>> +// SPDX-License-Identifier: MIT
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#include <drm/drm_drv.h>
>> +#include <drm/drm_managed.h>
>> +#include <drm/xe_drm.h>
>> +
>> +#include "regs/xe_gt_regs.h"
>> +#include "xe_device.h"
>> +#include "xe_force_wake.h"
>> +#include "xe_gt_clock.h"
>> +#include "xe_mmio.h"
>> +#include "xe_macros.h"
>> +#include "xe_pm.h"
>> +
>> +static cpumask_t xe_pmu_cpumask;
>> +static unsigned int xe_pmu_target_cpu = -1;
>> +
>> +static unsigned int config_gt_id(const u64 config)
>> +{
>> + return config >> __XE_PMU_GT_SHIFT;
>> +}
>> +
>> +static u64 config_counter(const u64 config)
>> +{
>> + return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>> +}
>> +
>> +static void xe_pmu_event_destroy(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> +
>> + drm_WARN_ON(&xe->drm, event->parent);
>> +
>> + drm_dev_put(&xe->drm);
>> +}
>> +
>> +static u64 __engine_group_busyness_read(struct xe_gt *gt, int
>> sample_type)
>> +{
>> + u64 val;
>> +
>> + switch (sample_type) {
>> + case __XE_SAMPLE_RENDER_GROUP_BUSY:
>> + val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>> + break;
>> + case __XE_SAMPLE_COPY_GROUP_BUSY:
>> + val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>> + break;
>> + case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>> + val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>> + break;
>> + case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>> + val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>> + break;
>> + default:
>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>> + }
>> +
>> + return xe_gt_clock_cycles_to_ns(gt, val * 16);
>> +}
>> +
>> +static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>> +{
>> + int sample_type = config_counter(config);
>> + const unsigned int gt_id = gt->info.id;
>> + struct xe_device *xe = gt->tile->xe;
>> + struct xe_pmu *pmu = &xe->pmu;
>> + unsigned long flags;
>> + bool device_awake;
>> + u64 val;
>> +
>> + device_awake = xe_pm_runtime_get_if_active(xe);
>> + if (device_awake) {
>> + XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>> + val = __engine_group_busyness_read(gt, sample_type);
>> + XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>> + xe_pm_runtime_put(xe);
>> + }
>> +
>> + spin_lock_irqsave(&pmu->lock, flags);
>> +
>> + if (device_awake)
>> + pmu->sample[gt_id][sample_type] = val;
>> + else
>> + val = pmu->sample[gt_id][sample_type];
>> +
>> + spin_unlock_irqrestore(&pmu->lock, flags);
>> +
>> + return val;
>> +}
>> +
>> +static void engine_group_busyness_store(struct xe_gt *gt)
>> +{
>> + struct xe_pmu *pmu = >->tile->xe->pmu;
>> + unsigned int gt_id = gt->info.id;
>> + unsigned long flags;
>> + int i;
>> +
>> + spin_lock_irqsave(&pmu->lock, flags);
>> +
>> + for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <=
>> __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>> + pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>> +
>> + spin_unlock_irqrestore(&pmu->lock, flags);
>> +}
>> +
>> +static int
>> +config_status(struct xe_device *xe, u64 config)
>> +{
>> + unsigned int gt_id = config_gt_id(config);
>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>> +
>> + if (gt_id >= XE_PMU_MAX_GT)
>> + return -ENOENT;
>> +
>> + switch (config_counter(config)) {
>> + case XE_PMU_RENDER_GROUP_BUSY(0):
>> + case XE_PMU_COPY_GROUP_BUSY(0):
>> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>> + if (gt->info.type == XE_GT_TYPE_MEDIA)
>> + return -ENOENT;
>> + break;
>> + case XE_PMU_MEDIA_GROUP_BUSY(0):
>> + if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) |
>> BIT(XE_HW_ENGINE_VECS0))))
>> + return -ENOENT;
>> + break;
>> + default:
>> + return -ENOENT;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int xe_pmu_event_init(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> + int ret;
>> +
>> + if (pmu->closed)
>> + return -ENODEV;
>> +
>> + if (event->attr.type != event->pmu->type)
>> + return -ENOENT;
>> +
>> + /* unsupported modes and filters */
>> + if (event->attr.sample_period) /* no sampling */
>> + return -EINVAL;
>> +
>> + if (has_branch_stack(event))
>> + return -EOPNOTSUPP;
>> +
>> + if (event->cpu < 0)
>> + return -EINVAL;
>> +
>> + /* only allow running on one cpu at a time */
>> + if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>> + return -EINVAL;
>> +
>> + ret = config_status(xe, event->attr.config);
>> + if (ret)
>> + return ret;
>> +
>> + if (!event->parent) {
>> + drm_dev_get(&xe->drm);
>> + event->destroy = xe_pmu_event_destroy;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static u64 __xe_pmu_event_read(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + const unsigned int gt_id = config_gt_id(event->attr.config);
>> + const u64 config = event->attr.config;
>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>> + u64 val;
>> +
>> + switch (config_counter(config)) {
>> + case XE_PMU_RENDER_GROUP_BUSY(0):
>> + case XE_PMU_COPY_GROUP_BUSY(0):
>> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>> + case XE_PMU_MEDIA_GROUP_BUSY(0):
>> + val = engine_group_busyness_read(gt, config);
>> + break;
>> + default:
>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>> + }
>> +
>> + return val;
>> +}
>> +
>> +static void xe_pmu_event_read(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct hw_perf_event *hwc = &event->hw;
>> + struct xe_pmu *pmu = &xe->pmu;
>> + u64 prev, new;
>> +
>> + if (pmu->closed) {
>> + event->hw.state = PERF_HES_STOPPED;
>> + return;
>> + }
>> +again:
>> + prev = local64_read(&hwc->prev_count);
>> + new = __xe_pmu_event_read(event);
>> +
>> + if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>> + goto again;
>> +
>> + local64_add(new - prev, &event->count);
>> +}
>> +
>> +static void xe_pmu_enable(struct perf_event *event)
>> +{
>> + /*
>> + * Store the current counter value so we can report the correct
>> delta
>> + * for all listeners. Even when the event was already enabled and
>> has
>> + * an existing non-zero value.
>> + */
>> + local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>> +}
>> +
>> +static void xe_pmu_event_start(struct perf_event *event, int flags)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> +
>> + if (pmu->closed)
>> + return;
>> +
>> + xe_pmu_enable(event);
>> + event->hw.state = 0;
>> +}
>> +
>> +static void xe_pmu_event_stop(struct perf_event *event, int flags)
>> +{
>> + if (flags & PERF_EF_UPDATE)
>> + xe_pmu_event_read(event);
>> +
>> + event->hw.state = PERF_HES_STOPPED;
>> +}
>> +
>> +static int xe_pmu_event_add(struct perf_event *event, int flags)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> +
>> + if (pmu->closed)
>> + return -ENODEV;
>> +
>> + if (flags & PERF_EF_START)
>> + xe_pmu_event_start(event, flags);
>> +
>> + return 0;
>> +}
>> +
>> +static void xe_pmu_event_del(struct perf_event *event, int flags)
>> +{
>> + xe_pmu_event_stop(event, PERF_EF_UPDATE);
>> +}
>> +
>> +static int xe_pmu_event_event_idx(struct perf_event *event)
>> +{
>> + return 0;
>> +}
>> +
>> +struct xe_ext_attribute {
>> + struct device_attribute attr;
>> + unsigned long val;
>> +};
>> +
>> +static ssize_t xe_pmu_event_show(struct device *dev,
>> + struct device_attribute *attr, char *buf)
>> +{
>> + struct xe_ext_attribute *eattr;
>> +
>> + eattr = container_of(attr, struct xe_ext_attribute, attr);
>> + return sprintf(buf, "config=0x%lx\n", eattr->val);
>> +}
>> +
>> +static ssize_t cpumask_show(struct device *dev,
>> + struct device_attribute *attr, char *buf)
>> +{
>> + return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>> +}
>> +
>> +static DEVICE_ATTR_RO(cpumask);
>> +
>> +static struct attribute *xe_cpumask_attrs[] = {
>> + &dev_attr_cpumask.attr,
>> + NULL,
>> +};
>> +
>> +static const struct attribute_group xe_pmu_cpumask_attr_group = {
>> + .attrs = xe_cpumask_attrs,
>> +};
>> +
>> +#define __event(__counter, __name, __unit) \
>> +{ \
>> + .counter = (__counter), \
>> + .name = (__name), \
>> + .unit = (__unit), \
>> +}
>> +
>> +static struct xe_ext_attribute *
>> +add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>> +{
>> + sysfs_attr_init(&attr->attr.attr);
>> + attr->attr.attr.name = name;
>> + attr->attr.attr.mode = 0444;
>> + attr->attr.show = xe_pmu_event_show;
>> + attr->val = config;
>> +
>> + return ++attr;
>> +}
>> +
>> +static struct perf_pmu_events_attr *
>> +add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>> + const char *str)
>> +{
>> + sysfs_attr_init(&attr->attr.attr);
>> + attr->attr.attr.name = name;
>> + attr->attr.attr.mode = 0444;
>> + attr->attr.show = perf_event_sysfs_show;
>> + attr->event_str = str;
>> +
>> + return ++attr;
>> +}
>> +
>> +static struct attribute **
>> +create_event_attributes(struct xe_pmu *pmu)
>> +{
>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>> + static const struct {
>> + unsigned int counter;
>> + const char *name;
>> + const char *unit;
>> + } events[] = {
>> + __event(0, "render-group-busy", "ns"),
>> + __event(1, "copy-group-busy", "ns"),
>> + __event(2, "media-group-busy", "ns"),
>> + __event(3, "any-engine-group-busy", "ns"),
>> + };
>> +
>> + struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>> + struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>> + struct attribute **attr = NULL, **attr_iter;
>> + unsigned int count = 0;
>> + unsigned int i, j;
>> + struct xe_gt *gt;
>> +
>> + /* Count how many counters we will be exposing. */
>> + for_each_gt(gt, xe, j) {
>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>> +
>> + if (!config_status(xe, config))
>> + count++;
>> + }
>> + }
>> +
>> + /* Allocate attribute objects and table. */
>> + xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>> + if (!xe_attr)
>> + goto err_alloc;
>> +
>> + pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>> + if (!pmu_attr)
>> + goto err_alloc;
>> +
>> + /* Max one pointer of each attribute type plus a termination
>> entry. */
>> + attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>> + if (!attr)
>> + goto err_alloc;
>> +
>> + xe_iter = xe_attr;
>> + pmu_iter = pmu_attr;
>> + attr_iter = attr;
>> +
>> + for_each_gt(gt, xe, j) {
>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>> + char *str;
>> +
>> + if (config_status(xe, config))
>> + continue;
>> +
>> + str = kasprintf(GFP_KERNEL, "%s-gt%u",
>> + events[i].name, j);
>> + if (!str)
>> + goto err;
>> +
>> + *attr_iter++ = &xe_iter->attr.attr;
>> + xe_iter = add_xe_attr(xe_iter, str, config);
>> +
>> + if (events[i].unit) {
>> + str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>> + events[i].name, j);
>> + if (!str)
>> + goto err;
>> +
>> + *attr_iter++ = &pmu_iter->attr.attr;
>> + pmu_iter = add_pmu_attr(pmu_iter, str,
>> + events[i].unit);
>> + }
>> + }
>> + }
>> +
>> + pmu->xe_attr = xe_attr;
>> + pmu->pmu_attr = pmu_attr;
>> +
>> + return attr;
>> +
>> +err:
>> + for (attr_iter = attr; *attr_iter; attr_iter++)
>> + kfree((*attr_iter)->name);
>> +
>> +err_alloc:
>> + kfree(attr);
>> + kfree(xe_attr);
>> + kfree(pmu_attr);
>> +
>> + return NULL;
>> +}
>> +
>> +static void free_event_attributes(struct xe_pmu *pmu)
>> +{
>> + struct attribute **attr_iter = pmu->events_attr_group.attrs;
>> +
>> + for (; *attr_iter; attr_iter++)
>> + kfree((*attr_iter)->name);
>> +
>> + kfree(pmu->events_attr_group.attrs);
>> + kfree(pmu->xe_attr);
>> + kfree(pmu->pmu_attr);
>> +
>> + pmu->events_attr_group.attrs = NULL;
>> + pmu->xe_attr = NULL;
>> + pmu->pmu_attr = NULL;
>> +}
>> +
>> +static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>> +{
>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu),
>> cpuhp.node);
>> +
>> + /* Select the first online CPU as a designated reader. */
>> + if (cpumask_empty(&xe_pmu_cpumask))
>> + cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>> +
>> + return 0;
>> +}
>> +
>> +static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>> +{
>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu),
>> cpuhp.node);
>> + unsigned int target = xe_pmu_target_cpu;
>> +
>> + /*
>> + * Unregistering an instance generates a CPU offline event which
>> we must
>> + * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>> + */
>> + if (pmu->closed)
>> + return 0;
>> +
>> + if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>> + target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>> +
>> + /* Migrate events if there is a valid target */
>> + if (target < nr_cpu_ids) {
>> + cpumask_set_cpu(target, &xe_pmu_cpumask);
>> + xe_pmu_target_cpu = target;
>> + }
>> + }
>> +
>> + if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>> + perf_pmu_migrate_context(&pmu->base, cpu, target);
>> + pmu->cpuhp.cpu = target;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>> +
>> +int xe_pmu_init(void)
>> +{
>> + int ret;
>> +
>> + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>> + "perf/x86/intel/xe:online",
>> + xe_pmu_cpu_online,
>> + xe_pmu_cpu_offline);
>> + if (ret < 0)
>> + pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>> + ret);
>> + else
>> + cpuhp_slot = ret;
>> +
>> + return 0;
>> +}
>> +
>> +void xe_pmu_exit(void)
>> +{
>> + if (cpuhp_slot != CPUHP_INVALID)
>> + cpuhp_remove_multi_state(cpuhp_slot);
>> +}
>> +
>> +static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>> +{
>> + if (cpuhp_slot == CPUHP_INVALID)
>> + return -EINVAL;
>> +
>> + return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>> +}
>> +
>> +static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>> +{
>> + cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>> +}
>> +
>> +void xe_pmu_suspend(struct xe_gt *gt)
>> +{
>> + engine_group_busyness_store(gt);
>> +}
>> +
>> +static void xe_pmu_unregister(void *arg)
>> +{
>> + struct xe_pmu *pmu = arg;
>> +
>> + if (!pmu->base.event_init)
>> + return;
>> +
>> + /*
>> + * "Disconnect" the PMU callbacks - since all are atomic
>> synchronize_rcu
>> + * ensures all currently executing ones will have exited before we
>> + * proceed with unregistration.
>> + */
>> + pmu->closed = true;
>> + synchronize_rcu();
>> +
>> + xe_pmu_unregister_cpuhp_state(pmu);
>> +
>> + perf_pmu_unregister(&pmu->base);
>> + pmu->base.event_init = NULL;
>> + kfree(pmu->base.attr_groups);
>> + kfree(pmu->name);
>> + free_event_attributes(pmu);
>> +}
>> +
>> +void xe_pmu_register(struct xe_pmu *pmu)
>> +{
>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>> + const struct attribute_group *attr_groups[] = {
>> + &pmu->events_attr_group,
>> + &xe_pmu_cpumask_attr_group,
>> + NULL
>> + };
>> +
>> + int ret = -ENOMEM;
>> +
>> + spin_lock_init(&pmu->lock);
>> + pmu->cpuhp.cpu = -1;
>> +
>> + pmu->name = kasprintf(GFP_KERNEL,
>> + "xe_%s",
>> + dev_name(xe->drm.dev));
>> + if (pmu->name)
>> + /* tools/perf reserves colons as special. */
>> + strreplace((char *)pmu->name, ':', '_');
>> +
>> + if (!pmu->name)
>> + goto err;
>> +
>> + pmu->events_attr_group.name = "events";
>> + pmu->events_attr_group.attrs = create_event_attributes(pmu);
>> + if (!pmu->events_attr_group.attrs)
>> + goto err_name;
>> +
>> + pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>> + GFP_KERNEL);
>> + if (!pmu->base.attr_groups)
>> + goto err_attr;
>> +
>> + pmu->base.module = THIS_MODULE;
>> + pmu->base.task_ctx_nr = perf_invalid_context;
>> + pmu->base.event_init = xe_pmu_event_init;
>> + pmu->base.add = xe_pmu_event_add;
>> + pmu->base.del = xe_pmu_event_del;
>> + pmu->base.start = xe_pmu_event_start;
>> + pmu->base.stop = xe_pmu_event_stop;
>> + pmu->base.read = xe_pmu_event_read;
>> + pmu->base.event_idx = xe_pmu_event_event_idx;
>> +
>> + ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>> + if (ret)
>> + goto err_groups;
>> +
>> + ret = xe_pmu_register_cpuhp_state(pmu);
>> + if (ret)
>> + goto err_unreg;
>> +
>> + ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>> + if (ret)
>> + goto err_cpuhp;
>> +
>> + return;
>> +
>> +err_cpuhp:
>> + xe_pmu_unregister_cpuhp_state(pmu);
>> +err_unreg:
>> + perf_pmu_unregister(&pmu->base);
>> +err_groups:
>> + kfree(pmu->base.attr_groups);
>> +err_attr:
>> + pmu->base.event_init = NULL;
>> + free_event_attributes(pmu);
>> +err_name:
>> + kfree(pmu->name);
>
> Needs fix. double free incase of devm_add_action_or_reset failure.
Thanks for catching this. Will fix it.
>
>> +err:
>> + drm_notice(&xe->drm, "Failed to register PMU!\n");
>> +}
>> diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>> new file mode 100644
>> index 000000000000..8afa256f9dac
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu.h
>> @@ -0,0 +1,26 @@
>> +/* SPDX-License-Identifier: MIT */
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#ifndef _XE_PMU_H_
>> +#define _XE_PMU_H_
>> +
>> +#include "xe_pmu_types.h"
>> +
>> +struct xe_gt;
>> +
>> +#if IS_ENABLED(CONFIG_PERF_EVENTS)
>> +int xe_pmu_init(void);
>> +void xe_pmu_exit(void);
>> +void xe_pmu_register(struct xe_pmu *pmu);
>> +void xe_pmu_suspend(struct xe_gt *gt);
>> +#else
>> +static inline int xe_pmu_init(void) { return 0; }
>> +static inline void xe_pmu_exit(void) {}
>> +static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>> +static inline void xe_pmu_suspend(struct xe_gt *gt) {}
>> +#endif
>> +
>> +#endif
>> +
>> diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h
>> b/drivers/gpu/drm/xe/xe_pmu_types.h
>> new file mode 100644
>> index 000000000000..e86e8d7e0356
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>> @@ -0,0 +1,67 @@
>> +/* SPDX-License-Identifier: MIT */
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#ifndef _XE_PMU_TYPES_H_
>> +#define _XE_PMU_TYPES_H_
>> +
>> +#include <linux/perf_event.h>
>> +#include <linux/spinlock_types.h>
>> +#include <uapi/drm/xe_drm.h>
>> +
>> +enum {
>> + __XE_SAMPLE_RENDER_GROUP_BUSY,
>> + __XE_SAMPLE_COPY_GROUP_BUSY,
>> + __XE_SAMPLE_MEDIA_GROUP_BUSY,
>> + __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
>> + __XE_NUM_PMU_SAMPLERS
>> +};
>> +
>> +#define XE_PMU_MAX_GT 2
>> +
>> +struct xe_pmu {
>> + /**
>> + * @cpuhp: Struct used for CPU hotplug handling.
>> + */
>> + struct {
>> + struct hlist_node node;
>> + unsigned int cpu;
>> + } cpuhp;
>> + /**
>> + * @base: PMU base.
>> + */
>> + struct pmu base;
>> + /**
>> + * @closed: xe is unregistering.
>> + */
>> + bool closed;
>> + /**
>> + * @name: Name as registered with perf core.
>> + */
>> + const char *name;
>> + /**
>> + * @lock: Lock protecting enable mask and ref count handling.
>> + */
>> + spinlock_t lock;
>> + /**
>> + * @sample: Current and previous (raw) counters.
>> + *
>> + * These counters are updated when the device is awake.
>> + */
>> + u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>> + /**
>> + * @events_attr_group: Device events attribute group.
>> + */
>> + struct attribute_group events_attr_group;
>> + /**
>> + * @xe_attr: Memory block holding device attributes.
>> + */
>> + void *xe_attr;
>> + /**
>> + * @pmu_attr: Memory block holding device attributes.
>> + */
>> + void *pmu_attr;
>> +};
>> +
>> +#endif
>> diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>> index d7b0903c22b2..07ca545354f7 100644
>> --- a/include/uapi/drm/xe_drm.h
>> +++ b/include/uapi/drm/xe_drm.h
>> @@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
>> __u64 reserved[2];
>> };
>> +/**
>> + * DOC: XE PMU event config IDs
>> + *
>> + * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in
>> xe_drm.h
>> + * in 'struct perf_event_attr' as part of perf_event_open syscall to
>> read a
>> + * particular event.
>> + *
>> + * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
>> + *
>> + * .. code-block:: C
>> + *
>> + * struct perf_event_attr attr;
>> + * long long count;
>> + * int cpu = 0;
>> + * int fd;
>> + *
>> + * memset(&attr, 0, sizeof(struct perf_event_attr));
>> + * attr.type = type; // eg:
>> /sys/bus/event_source/devices/xe_0000_56_00.0/type
>> + * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
>> + * attr.use_clockid = 1;
>> + * attr.clockid = CLOCK_MONOTONIC;
>> + * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
>> + *
>> + * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
>> + */
>> +
>> +/*
>> + * Top bits of every counter are GT id.
>> + */
>> +#define __XE_PMU_GT_SHIFT (56)
>> +
>> +#define ___XE_PMU_OTHER(gt, x) \
>> + (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>> +
>> +#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
>> +#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
>> +#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
>> +#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
>> +
>> #if defined(__cplusplus)
>> }
>> #endif
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-20 19:52 ` Umesh Nerlige Ramappa
@ 2024-06-27 6:49 ` Aravind Iddamsetty
2024-06-27 16:05 ` Umesh Nerlige Ramappa
2024-06-28 15:55 ` Lucas De Marchi
1 sibling, 1 reply; 32+ messages in thread
From: Aravind Iddamsetty @ 2024-06-27 6:49 UTC (permalink / raw)
To: Umesh Nerlige Ramappa, Riana Tauro
Cc: intel-xe, anshuman.gupta, ashutosh.dixit, rodrigo.vivi,
krishnaiah.bommu, lucas.demarchi, Joonas Lahtinen
On 21/06/24 01:22, Umesh Nerlige Ramappa wrote:
> On Thu, Jun 13, 2024 at 03:34:11PM +0530, Riana Tauro wrote:
>> From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>
>> There are a set of engine group busyness counters provided by HW which are
>> perfect fit to be exposed via PMU perf events.
>>
>> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>
>> events can be listed using:
>> perf list
>> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>
>> and can be read using:
>>
>> perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
>> time counts unit events
>> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>
>> The pmu base implementation is taken from i915.
>>
>> v2:
>> Store last known value when device is awake return that while the GT is
>> suspended and then update the driver copy when read during awake.
>>
>> v3:
>> 1. drop init_samples, as storing counters before going to suspend should
>> be sufficient.
>> 2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>> dropped helpers to store and read samples.
>> 3. use xe_device_mem_access_get_if_ongoing to check if device is active
>> before reading the OA registers.
>> 4. dropped format attr as no longer needed
>> 5. introduce xe_pmu_suspend to call engine_group_busyness_store
>> 6. few other nits.
>>
>> v4: minor nits.
>>
>> v5: take forcewake when accessing the OAG registers
>>
>> v6:
>> 1. drop engine_busyness_sample_type
>> 2. update UAPI documentation
>>
>> v7:
>> 1. update UAPI documentation
>> 2. drop MEDIA_GT specific change for media busyness counter.
>>
>> v8:
>> 1. rebase
>> 2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>> 3. remove interrupts pmu event
>>
>> v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>>
>> Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>> Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>> Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
>> Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>> ---
>> drivers/gpu/drm/xe/Makefile | 2 +
>> drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
>> drivers/gpu/drm/xe/xe_device.c | 2 +
>> drivers/gpu/drm/xe/xe_device_types.h | 4 +
>> drivers/gpu/drm/xe/xe_gt.c | 2 +
>> drivers/gpu/drm/xe/xe_module.c | 5 +
>> drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
>> drivers/gpu/drm/xe/xe_pmu.h | 26 ++
>> drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
>> include/uapi/drm/xe_drm.h | 39 ++
>> 10 files changed, 783 insertions(+)
>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
>> create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>>
>> diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>> index cbf961b90237..83bf1e07669b 100644
>> --- a/drivers/gpu/drm/xe/Makefile
>> +++ b/drivers/gpu/drm/xe/Makefile
>> @@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
>> i915-display/skl_universal_plane.o \
>> i915-display/skl_watermark.o
>>
>> +xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>> +
>> ifeq ($(CONFIG_ACPI),y)
>> xe-$(CONFIG_DRM_XE_DISPLAY) += \
>> i915-display/intel_acpi.o \
>> diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>> index 47c26c37608d..22821dcd4e1b 100644
>> --- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>> +++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>> @@ -390,6 +390,11 @@
>> #define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
>> #define GLOBAL_INVALIDATION_MODE REG_BIT(2)
>>
>> +#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
>> +#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
>> +#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
>> +#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
>> +
>> #define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
>> #define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
>>
>> diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
>> index 64691a56d59c..bb00c8c9ec9b 100644
>> --- a/drivers/gpu/drm/xe/xe_device.c
>> +++ b/drivers/gpu/drm/xe/xe_device.c
>> @@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
>>
>> xe_hwmon_register(xe);
>>
>> + xe_pmu_register(&xe->pmu);
>> +
>> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>>
>> err_fini_display:
>> diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
>> index 52bc461171d5..a5dba7325cf1 100644
>> --- a/drivers/gpu/drm/xe/xe_device_types.h
>> +++ b/drivers/gpu/drm/xe/xe_device_types.h
>> @@ -18,6 +18,7 @@
>> #include "xe_lmtt_types.h"
>> #include "xe_memirq_types.h"
>> #include "xe_platform_types.h"
>> +#include "xe_pmu.h"
>> #include "xe_pt_types.h"
>> #include "xe_sriov_types.h"
>> #include "xe_step_types.h"
>> @@ -473,6 +474,9 @@ struct xe_device {
>> int mode;
>> } wedged;
>>
>> + /** @pmu: performance monitoring unit */
>> + struct xe_pmu pmu;
>> +
>> /* private: */
>>
>> #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
>> diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
>> index 57d84751e160..477d0ae5f230 100644
>> --- a/drivers/gpu/drm/xe/xe_gt.c
>> +++ b/drivers/gpu/drm/xe/xe_gt.c
>> @@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
>> if (err)
>> goto err_msg;
>>
>> + xe_pmu_suspend(gt);
>> +
>> err = xe_uc_suspend(>->uc);
>> if (err)
>> goto err_force_wake;
>> diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
>> index 3edeb30d5ccb..26f814f97fc2 100644
>> --- a/drivers/gpu/drm/xe/xe_module.c
>> +++ b/drivers/gpu/drm/xe/xe_module.c
>> @@ -11,6 +11,7 @@
>> #include "xe_drv.h"
>> #include "xe_hw_fence.h"
>> #include "xe_pci.h"
>> +#include "xe_pmu.h"
>> #include "xe_sched_job.h"
>>
>> struct xe_modparam xe_modparam = {
>> @@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
>> .init = xe_sched_job_module_init,
>> .exit = xe_sched_job_module_exit,
>> },
>> + {
>> + .init = xe_pmu_init,
>> + .exit = xe_pmu_exit,
>> + },
>> {
>> .init = xe_register_pci_driver,
>> .exit = xe_unregister_pci_driver,
>> diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>> new file mode 100644
>> index 000000000000..64960a358af2
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu.c
>> @@ -0,0 +1,631 @@
>> +// SPDX-License-Identifier: MIT
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#include <drm/drm_drv.h>
>> +#include <drm/drm_managed.h>
>> +#include <drm/xe_drm.h>
>> +
>> +#include "regs/xe_gt_regs.h"
>> +#include "xe_device.h"
>> +#include "xe_force_wake.h"
>> +#include "xe_gt_clock.h"
>> +#include "xe_mmio.h"
>> +#include "xe_macros.h"
>> +#include "xe_pm.h"
>> +
>> +static cpumask_t xe_pmu_cpumask;
>> +static unsigned int xe_pmu_target_cpu = -1;
>> +
>> +static unsigned int config_gt_id(const u64 config)
>> +{
>> + return config >> __XE_PMU_GT_SHIFT;
>> +}
>> +
>> +static u64 config_counter(const u64 config)
>> +{
>> + return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>> +}
>> +
>> +static void xe_pmu_event_destroy(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> +
>> + drm_WARN_ON(&xe->drm, event->parent);
>> +
>> + drm_dev_put(&xe->drm);
>> +}
>> +
>> +static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
>> +{
>> + u64 val;
>> +
>> + switch (sample_type) {
>> + case __XE_SAMPLE_RENDER_GROUP_BUSY:
>> + val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>> + break;
>> + case __XE_SAMPLE_COPY_GROUP_BUSY:
>> + val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>> + break;
>> + case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>> + val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>> + break;
>> + case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>> + val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>> + break;
>> + default:
>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>> + }
>> +
>> + return xe_gt_clock_cycles_to_ns(gt, val * 16);
>> +}
>> +
>> +static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>> +{
>> + int sample_type = config_counter(config);
>> + const unsigned int gt_id = gt->info.id;
>> + struct xe_device *xe = gt->tile->xe;
>> + struct xe_pmu *pmu = &xe->pmu;
>> + unsigned long flags;
>> + bool device_awake;
>> + u64 val;
>> +
>> + device_awake = xe_pm_runtime_get_if_active(xe);
>> + if (device_awake) {
>> + XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>> + val = __engine_group_busyness_read(gt, sample_type);
>> + XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>> + xe_pm_runtime_put(xe);
>> + }
>> +
>> + spin_lock_irqsave(&pmu->lock, flags);
>> +
>> + if (device_awake)
>> + pmu->sample[gt_id][sample_type] = val;
>> + else
>> + val = pmu->sample[gt_id][sample_type];
>> +
>> + spin_unlock_irqrestore(&pmu->lock, flags);
>> +
>> + return val;
>> +}
>> +
>> +static void engine_group_busyness_store(struct xe_gt *gt)
>> +{
>> + struct xe_pmu *pmu = >->tile->xe->pmu;
>> + unsigned int gt_id = gt->info.id;
>> + unsigned long flags;
>> + int i;
>> +
>> + spin_lock_irqsave(&pmu->lock, flags);
>> +
>> + for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>> + pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>> +
>> + spin_unlock_irqrestore(&pmu->lock, flags);
>> +}
>> +
>> +static int
>> +config_status(struct xe_device *xe, u64 config)
>> +{
>> + unsigned int gt_id = config_gt_id(config);
>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>> +
>> + if (gt_id >= XE_PMU_MAX_GT)
>> + return -ENOENT;
>> +
>> + switch (config_counter(config)) {
>> + case XE_PMU_RENDER_GROUP_BUSY(0):
>> + case XE_PMU_COPY_GROUP_BUSY(0):
>> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>> + if (gt->info.type == XE_GT_TYPE_MEDIA)
>> + return -ENOENT;
>> + break;
>> + case XE_PMU_MEDIA_GROUP_BUSY(0):
>> + if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
>> + return -ENOENT;
>> + break;
>> + default:
>> + return -ENOENT;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int xe_pmu_event_init(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> + int ret;
>> +
>> + if (pmu->closed)
>> + return -ENODEV;
>> +
>> + if (event->attr.type != event->pmu->type)
>> + return -ENOENT;
>> +
>> + /* unsupported modes and filters */
>> + if (event->attr.sample_period) /* no sampling */
>> + return -EINVAL;
>> +
>> + if (has_branch_stack(event))
>> + return -EOPNOTSUPP;
>> +
>> + if (event->cpu < 0)
>> + return -EINVAL;
>> +
>> + /* only allow running on one cpu at a time */
>> + if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>> + return -EINVAL;
>> +
>> + ret = config_status(xe, event->attr.config);
>> + if (ret)
>> + return ret;
>> +
>> + if (!event->parent) {
>> + drm_dev_get(&xe->drm);
>> + event->destroy = xe_pmu_event_destroy;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static u64 __xe_pmu_event_read(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + const unsigned int gt_id = config_gt_id(event->attr.config);
>> + const u64 config = event->attr.config;
>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>> + u64 val;
>> +
>> + switch (config_counter(config)) {
>> + case XE_PMU_RENDER_GROUP_BUSY(0):
>> + case XE_PMU_COPY_GROUP_BUSY(0):
>> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>> + case XE_PMU_MEDIA_GROUP_BUSY(0):
>> + val = engine_group_busyness_read(gt, config);
>> + break;
>> + default:
>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>> + }
>> +
>> + return val;
>> +}
>> +
>> +static void xe_pmu_event_read(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct hw_perf_event *hwc = &event->hw;
>> + struct xe_pmu *pmu = &xe->pmu;
>> + u64 prev, new;
>> +
>> + if (pmu->closed) {
>> + event->hw.state = PERF_HES_STOPPED;
>> + return;
>> + }
>> +again:
>> + prev = local64_read(&hwc->prev_count);
>> + new = __xe_pmu_event_read(event);
>> +
>> + if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>> + goto again;
>> +
>> + local64_add(new - prev, &event->count);
>> +}
>> +
>> +static void xe_pmu_enable(struct perf_event *event)
>> +{
>> + /*
>> + * Store the current counter value so we can report the correct delta
>> + * for all listeners. Even when the event was already enabled and has
>> + * an existing non-zero value.
>> + */
>> + local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>> +}
>> +
>> +static void xe_pmu_event_start(struct perf_event *event, int flags)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> +
>> + if (pmu->closed)
>> + return;
>> +
>> + xe_pmu_enable(event);
>> + event->hw.state = 0;
>> +}
>> +
>> +static void xe_pmu_event_stop(struct perf_event *event, int flags)
>> +{
>> + if (flags & PERF_EF_UPDATE)
>> + xe_pmu_event_read(event);
>> +
>> + event->hw.state = PERF_HES_STOPPED;
>> +}
>> +
>> +static int xe_pmu_event_add(struct perf_event *event, int flags)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> +
>> + if (pmu->closed)
>> + return -ENODEV;
>> +
>> + if (flags & PERF_EF_START)
>> + xe_pmu_event_start(event, flags);
>> +
>> + return 0;
>> +}
>> +
>> +static void xe_pmu_event_del(struct perf_event *event, int flags)
>> +{
>> + xe_pmu_event_stop(event, PERF_EF_UPDATE);
>> +}
>> +
>> +static int xe_pmu_event_event_idx(struct perf_event *event)
>> +{
>> + return 0;
>> +}
>> +
>> +struct xe_ext_attribute {
>> + struct device_attribute attr;
>> + unsigned long val;
>> +};
>> +
>> +static ssize_t xe_pmu_event_show(struct device *dev,
>> + struct device_attribute *attr, char *buf)
>> +{
>> + struct xe_ext_attribute *eattr;
>> +
>> + eattr = container_of(attr, struct xe_ext_attribute, attr);
>> + return sprintf(buf, "config=0x%lx\n", eattr->val);
>> +}
>> +
>> +static ssize_t cpumask_show(struct device *dev,
>> + struct device_attribute *attr, char *buf)
>> +{
>> + return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>> +}
>> +
>> +static DEVICE_ATTR_RO(cpumask);
>> +
>> +static struct attribute *xe_cpumask_attrs[] = {
>> + &dev_attr_cpumask.attr,
>> + NULL,
>> +};
>> +
>> +static const struct attribute_group xe_pmu_cpumask_attr_group = {
>> + .attrs = xe_cpumask_attrs,
>> +};
>> +
>> +#define __event(__counter, __name, __unit) \
>> +{ \
>> + .counter = (__counter), \
>> + .name = (__name), \
>> + .unit = (__unit), \
>> +}
>> +
>> +static struct xe_ext_attribute *
>> +add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>> +{
>> + sysfs_attr_init(&attr->attr.attr);
>> + attr->attr.attr.name = name;
>> + attr->attr.attr.mode = 0444;
>> + attr->attr.show = xe_pmu_event_show;
>> + attr->val = config;
>> +
>> + return ++attr;
>> +}
>> +
>> +static struct perf_pmu_events_attr *
>> +add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>> + const char *str)
>> +{
>> + sysfs_attr_init(&attr->attr.attr);
>> + attr->attr.attr.name = name;
>> + attr->attr.attr.mode = 0444;
>> + attr->attr.show = perf_event_sysfs_show;
>> + attr->event_str = str;
>> +
>> + return ++attr;
>> +}
>> +
>> +static struct attribute **
>> +create_event_attributes(struct xe_pmu *pmu)
>> +{
>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>> + static const struct {
>> + unsigned int counter;
>> + const char *name;
>> + const char *unit;
>> + } events[] = {
>> + __event(0, "render-group-busy", "ns"),
>> + __event(1, "copy-group-busy", "ns"),
>> + __event(2, "media-group-busy", "ns"),
>> + __event(3, "any-engine-group-busy", "ns"),
>> + };
>> +
>> + struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>> + struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>> + struct attribute **attr = NULL, **attr_iter;
>> + unsigned int count = 0;
>> + unsigned int i, j;
>> + struct xe_gt *gt;
>> +
>> + /* Count how many counters we will be exposing. */
>> + for_each_gt(gt, xe, j) {
>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>> +
>> + if (!config_status(xe, config))
>> + count++;
>> + }
>> + }
>> +
>> + /* Allocate attribute objects and table. */
>> + xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>> + if (!xe_attr)
>> + goto err_alloc;
>> +
>> + pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>> + if (!pmu_attr)
>> + goto err_alloc;
>> +
>> + /* Max one pointer of each attribute type plus a termination entry. */
>> + attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>> + if (!attr)
>> + goto err_alloc;
>> +
>> + xe_iter = xe_attr;
>> + pmu_iter = pmu_attr;
>> + attr_iter = attr;
>> +
>> + for_each_gt(gt, xe, j) {
>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>> + char *str;
>> +
>> + if (config_status(xe, config))
>> + continue;
>> +
>> + str = kasprintf(GFP_KERNEL, "%s-gt%u",
>> + events[i].name, j);
>> + if (!str)
>> + goto err;
>> +
>> + *attr_iter++ = &xe_iter->attr.attr;
>> + xe_iter = add_xe_attr(xe_iter, str, config);
>> +
>> + if (events[i].unit) {
>> + str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>> + events[i].name, j);
>> + if (!str)
>> + goto err;
>> +
>> + *attr_iter++ = &pmu_iter->attr.attr;
>> + pmu_iter = add_pmu_attr(pmu_iter, str,
>> + events[i].unit);
>> + }
>> + }
>> + }
>> +
>> + pmu->xe_attr = xe_attr;
>> + pmu->pmu_attr = pmu_attr;
>> +
>> + return attr;
>> +
>> +err:
>> + for (attr_iter = attr; *attr_iter; attr_iter++)
>> + kfree((*attr_iter)->name);
>> +
>> +err_alloc:
>> + kfree(attr);
>> + kfree(xe_attr);
>> + kfree(pmu_attr);
>> +
>> + return NULL;
>> +}
>> +
>> +static void free_event_attributes(struct xe_pmu *pmu)
>> +{
>> + struct attribute **attr_iter = pmu->events_attr_group.attrs;
>> +
>> + for (; *attr_iter; attr_iter++)
>> + kfree((*attr_iter)->name);
>> +
>> + kfree(pmu->events_attr_group.attrs);
>> + kfree(pmu->xe_attr);
>> + kfree(pmu->pmu_attr);
>> +
>> + pmu->events_attr_group.attrs = NULL;
>> + pmu->xe_attr = NULL;
>> + pmu->pmu_attr = NULL;
>> +}
>> +
>> +static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>> +{
>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>> +
>> + /* Select the first online CPU as a designated reader. */
>> + if (cpumask_empty(&xe_pmu_cpumask))
>> + cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>> +
>> + return 0;
>> +}
>> +
>> +static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>> +{
>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>> + unsigned int target = xe_pmu_target_cpu;
>> +
>> + /*
>> + * Unregistering an instance generates a CPU offline event which we must
>> + * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>> + */
>> + if (pmu->closed)
>> + return 0;
>> +
>> + if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>> + target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>> +
>> + /* Migrate events if there is a valid target */
>> + if (target < nr_cpu_ids) {
>> + cpumask_set_cpu(target, &xe_pmu_cpumask);
>> + xe_pmu_target_cpu = target;
>> + }
>> + }
>> +
>> + if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>> + perf_pmu_migrate_context(&pmu->base, cpu, target);
>> + pmu->cpuhp.cpu = target;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>> +
>> +int xe_pmu_init(void)
>> +{
>> + int ret;
>> +
>> + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>> + "perf/x86/intel/xe:online",
>> + xe_pmu_cpu_online,
>> + xe_pmu_cpu_offline);
>> + if (ret < 0)
>> + pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>> + ret);
>> + else
>> + cpuhp_slot = ret;
>> +
>> + return 0;
>> +}
>> +
>> +void xe_pmu_exit(void)
>> +{
>> + if (cpuhp_slot != CPUHP_INVALID)
>> + cpuhp_remove_multi_state(cpuhp_slot);
>> +}
>> +
>> +static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>> +{
>> + if (cpuhp_slot == CPUHP_INVALID)
>> + return -EINVAL;
>> +
>> + return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>> +}
>> +
>> +static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>> +{
>> + cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>> +}
>> +
>> +void xe_pmu_suspend(struct xe_gt *gt)
>> +{
>> + engine_group_busyness_store(gt);
>> +}
>> +
>> +static void xe_pmu_unregister(void *arg)
>> +{
>> + struct xe_pmu *pmu = arg;
>> +
>> + if (!pmu->base.event_init)
>> + return;
>> +
>> + /*
>> + * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
>> + * ensures all currently executing ones will have exited before we
>> + * proceed with unregistration.
>> + */
>> + pmu->closed = true;
>> + synchronize_rcu();
>> +
>> + xe_pmu_unregister_cpuhp_state(pmu);
>> +
>> + perf_pmu_unregister(&pmu->base);
>> + pmu->base.event_init = NULL;
>> + kfree(pmu->base.attr_groups);
>> + kfree(pmu->name);
>> + free_event_attributes(pmu);
>> +}
>> +
>> +void xe_pmu_register(struct xe_pmu *pmu)
>> +{
>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>> + const struct attribute_group *attr_groups[] = {
>> + &pmu->events_attr_group,
>> + &xe_pmu_cpumask_attr_group,
>> + NULL
>> + };
>> +
>> + int ret = -ENOMEM;
>> +
>> + spin_lock_init(&pmu->lock);
>> + pmu->cpuhp.cpu = -1;
>> +
>> + pmu->name = kasprintf(GFP_KERNEL,
>> + "xe_%s",
>> + dev_name(xe->drm.dev));
>> + if (pmu->name)
>> + /* tools/perf reserves colons as special. */
>> + strreplace((char *)pmu->name, ':', '_');
>> +
>> + if (!pmu->name)
>> + goto err;
>> +
>> + pmu->events_attr_group.name = "events";
>> + pmu->events_attr_group.attrs = create_event_attributes(pmu);
>> + if (!pmu->events_attr_group.attrs)
>> + goto err_name;
>> +
>> + pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>> + GFP_KERNEL);
>> + if (!pmu->base.attr_groups)
>> + goto err_attr;
>> +
>> + pmu->base.module = THIS_MODULE;
>> + pmu->base.task_ctx_nr = perf_invalid_context;
>> + pmu->base.event_init = xe_pmu_event_init;
>> + pmu->base.add = xe_pmu_event_add;
>> + pmu->base.del = xe_pmu_event_del;
>> + pmu->base.start = xe_pmu_event_start;
>> + pmu->base.stop = xe_pmu_event_stop;
>> + pmu->base.read = xe_pmu_event_read;
>> + pmu->base.event_idx = xe_pmu_event_event_idx;
>> +
>> + ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>> + if (ret)
>> + goto err_groups;
>> +
>> + ret = xe_pmu_register_cpuhp_state(pmu);
>> + if (ret)
>> + goto err_unreg;
>> +
>> + ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>> + if (ret)
>> + goto err_cpuhp;
>> +
>> + return;
>> +
>> +err_cpuhp:
>> + xe_pmu_unregister_cpuhp_state(pmu);
>> +err_unreg:
>> + perf_pmu_unregister(&pmu->base);
>> +err_groups:
>> + kfree(pmu->base.attr_groups);
>> +err_attr:
>> + pmu->base.event_init = NULL;
>> + free_event_attributes(pmu);
>> +err_name:
>> + kfree(pmu->name);
>> +err:
>> + drm_notice(&xe->drm, "Failed to register PMU!\n");
>> +}
>> diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>> new file mode 100644
>> index 000000000000..8afa256f9dac
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu.h
>> @@ -0,0 +1,26 @@
>> +/* SPDX-License-Identifier: MIT */
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#ifndef _XE_PMU_H_
>> +#define _XE_PMU_H_
>> +
>> +#include "xe_pmu_types.h"
>> +
>> +struct xe_gt;
>> +
>> +#if IS_ENABLED(CONFIG_PERF_EVENTS)
>> +int xe_pmu_init(void);
>> +void xe_pmu_exit(void);
>> +void xe_pmu_register(struct xe_pmu *pmu);
>> +void xe_pmu_suspend(struct xe_gt *gt);
>> +#else
>> +static inline int xe_pmu_init(void) { return 0; }
>> +static inline void xe_pmu_exit(void) {}
>> +static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>> +static inline void xe_pmu_suspend(struct xe_gt *gt) {}
>> +#endif
>> +
>> +#endif
>> +
>> diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
>> new file mode 100644
>> index 000000000000..e86e8d7e0356
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>> @@ -0,0 +1,67 @@
>> +/* SPDX-License-Identifier: MIT */
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#ifndef _XE_PMU_TYPES_H_
>> +#define _XE_PMU_TYPES_H_
>> +
>> +#include <linux/perf_event.h>
>> +#include <linux/spinlock_types.h>
>> +#include <uapi/drm/xe_drm.h>
>> +
>> +enum {
>> + __XE_SAMPLE_RENDER_GROUP_BUSY,
>> + __XE_SAMPLE_COPY_GROUP_BUSY,
>> + __XE_SAMPLE_MEDIA_GROUP_BUSY,
>> + __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
>> + __XE_NUM_PMU_SAMPLERS
>> +};
>> +
>> +#define XE_PMU_MAX_GT 2
>> +
>> +struct xe_pmu {
>> + /**
>> + * @cpuhp: Struct used for CPU hotplug handling.
>> + */
>> + struct {
>> + struct hlist_node node;
>> + unsigned int cpu;
>> + } cpuhp;
>> + /**
>> + * @base: PMU base.
>> + */
>> + struct pmu base;
>> + /**
>> + * @closed: xe is unregistering.
>> + */
>> + bool closed;
>> + /**
>> + * @name: Name as registered with perf core.
>> + */
>> + const char *name;
>> + /**
>> + * @lock: Lock protecting enable mask and ref count handling.
>> + */
>> + spinlock_t lock;
>> + /**
>> + * @sample: Current and previous (raw) counters.
>> + *
>> + * These counters are updated when the device is awake.
>> + */
>> + u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>> + /**
>> + * @events_attr_group: Device events attribute group.
>> + */
>> + struct attribute_group events_attr_group;
>> + /**
>> + * @xe_attr: Memory block holding device attributes.
>> + */
>> + void *xe_attr;
>> + /**
>> + * @pmu_attr: Memory block holding device attributes.
>> + */
>> + void *pmu_attr;
>> +};
>> +
>> +#endif
>> diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>> index d7b0903c22b2..07ca545354f7 100644
>> --- a/include/uapi/drm/xe_drm.h
>> +++ b/include/uapi/drm/xe_drm.h
>> @@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
>> __u64 reserved[2];
>> };
>>
>> +/**
>> + * DOC: XE PMU event config IDs
>> + *
>> + * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in xe_drm.h
>> + * in 'struct perf_event_attr' as part of perf_event_open syscall to read a
>> + * particular event.
>> + *
>> + * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
>> + *
>> + * .. code-block:: C
>> + *
>> + * struct perf_event_attr attr;
>> + * long long count;
>> + * int cpu = 0;
>> + * int fd;
>> + *
>> + * memset(&attr, 0, sizeof(struct perf_event_attr));
>> + * attr.type = type; // eg: /sys/bus/event_source/devices/xe_0000_56_00.0/type
>> + * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
>> + * attr.use_clockid = 1;
>> + * attr.clockid = CLOCK_MONOTONIC;
>> + * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
>> + *
>> + * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
>> + */
>> +
>> +/*
>> + * Top bits of every counter are GT id.
>> + */
>> +#define __XE_PMU_GT_SHIFT (56)
>> +
>> +#define ___XE_PMU_OTHER(gt, x) \
>> + (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>> +
>> +#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
>> +#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
>> +#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
>> +#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
>
> + Lucas for inputs
>
> We should align this to the interface planned for other PMU busyness counters as well as how we do PCEU. i.e.
>
> 1) counters are in ticks
> 2) total time in ticks is also exported to the user.
>
> For 1), I would just append TICKS to the counter names and drop the conversion to _ns in __engine_group_busyness_read(). Also, drop the patch that adds this conversion helper.
>
> For 2) define a new counter - total active ticks that would return the 'CPU' timestamp converted to gpu ticks. The reason I am insisting on CPU timestamp here is because we want to have a time base that is ticking even when the GPU is idle.
why can't we expose what the HW presents[1] to us via register and leave the interpretation to userspace.
Thanks,
Aravind.
>
> Regards,
> Umesh
>
>> +
>> #if defined(__cplusplus)
>> }
>> #endif
>> --
>> 2.40.0
>>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-27 6:49 ` Aravind Iddamsetty
@ 2024-06-27 16:05 ` Umesh Nerlige Ramappa
2024-06-28 9:41 ` Aravind Iddamsetty
0 siblings, 1 reply; 32+ messages in thread
From: Umesh Nerlige Ramappa @ 2024-06-27 16:05 UTC (permalink / raw)
To: Aravind Iddamsetty
Cc: Riana Tauro, intel-xe, anshuman.gupta, ashutosh.dixit,
rodrigo.vivi, krishnaiah.bommu, lucas.demarchi, Joonas Lahtinen
On Thu, Jun 27, 2024 at 12:19:44PM +0530, Aravind Iddamsetty wrote:
>
>On 21/06/24 01:22, Umesh Nerlige Ramappa wrote:
>> On Thu, Jun 13, 2024 at 03:34:11PM +0530, Riana Tauro wrote:
>>> From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>>
>>> There are a set of engine group busyness counters provided by HW which are
>>> perfect fit to be exposed via PMU perf events.
>>>
>>> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>>
>>> events can be listed using:
>>> perf list
>>> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>>> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>>> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>>> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>>
>>> and can be read using:
>>>
>>> perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
>>> time counts unit events
>>> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>
>>> The pmu base implementation is taken from i915.
>>>
>>> v2:
>>> Store last known value when device is awake return that while the GT is
>>> suspended and then update the driver copy when read during awake.
>>>
>>> v3:
>>> 1. drop init_samples, as storing counters before going to suspend should
>>> be sufficient.
>>> 2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>>> dropped helpers to store and read samples.
>>> 3. use xe_device_mem_access_get_if_ongoing to check if device is active
>>> before reading the OA registers.
>>> 4. dropped format attr as no longer needed
>>> 5. introduce xe_pmu_suspend to call engine_group_busyness_store
>>> 6. few other nits.
>>>
>>> v4: minor nits.
>>>
>>> v5: take forcewake when accessing the OAG registers
>>>
>>> v6:
>>> 1. drop engine_busyness_sample_type
>>> 2. update UAPI documentation
>>>
>>> v7:
>>> 1. update UAPI documentation
>>> 2. drop MEDIA_GT specific change for media busyness counter.
>>>
>>> v8:
>>> 1. rebase
>>> 2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>>> 3. remove interrupts pmu event
>>>
>>> v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>>>
>>> Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>> Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>> Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
>>> Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>>> ---
>>> drivers/gpu/drm/xe/Makefile | 2 +
>>> drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
>>> drivers/gpu/drm/xe/xe_device.c | 2 +
>>> drivers/gpu/drm/xe/xe_device_types.h | 4 +
>>> drivers/gpu/drm/xe/xe_gt.c | 2 +
>>> drivers/gpu/drm/xe/xe_module.c | 5 +
>>> drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
>>> drivers/gpu/drm/xe/xe_pmu.h | 26 ++
>>> drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
>>> include/uapi/drm/xe_drm.h | 39 ++
>>> 10 files changed, 783 insertions(+)
>>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
>>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
>>> create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>>>
>>> diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>>> index cbf961b90237..83bf1e07669b 100644
>>> --- a/drivers/gpu/drm/xe/Makefile
>>> +++ b/drivers/gpu/drm/xe/Makefile
>>> @@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
>>> i915-display/skl_universal_plane.o \
>>> i915-display/skl_watermark.o
>>>
>>> +xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>>> +
>>> ifeq ($(CONFIG_ACPI),y)
>>> xe-$(CONFIG_DRM_XE_DISPLAY) += \
>>> i915-display/intel_acpi.o \
>>> diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>> index 47c26c37608d..22821dcd4e1b 100644
>>> --- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>> +++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>> @@ -390,6 +390,11 @@
>>> #define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
>>> #define GLOBAL_INVALIDATION_MODE REG_BIT(2)
>>>
>>> +#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
>>> +#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
>>> +#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
>>> +#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
>>> +
>>> #define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
>>> #define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
>>>
>>> diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
>>> index 64691a56d59c..bb00c8c9ec9b 100644
>>> --- a/drivers/gpu/drm/xe/xe_device.c
>>> +++ b/drivers/gpu/drm/xe/xe_device.c
>>> @@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
>>>
>>> xe_hwmon_register(xe);
>>>
>>> + xe_pmu_register(&xe->pmu);
>>> +
>>> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>>>
>>> err_fini_display:
>>> diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
>>> index 52bc461171d5..a5dba7325cf1 100644
>>> --- a/drivers/gpu/drm/xe/xe_device_types.h
>>> +++ b/drivers/gpu/drm/xe/xe_device_types.h
>>> @@ -18,6 +18,7 @@
>>> #include "xe_lmtt_types.h"
>>> #include "xe_memirq_types.h"
>>> #include "xe_platform_types.h"
>>> +#include "xe_pmu.h"
>>> #include "xe_pt_types.h"
>>> #include "xe_sriov_types.h"
>>> #include "xe_step_types.h"
>>> @@ -473,6 +474,9 @@ struct xe_device {
>>> int mode;
>>> } wedged;
>>>
>>> + /** @pmu: performance monitoring unit */
>>> + struct xe_pmu pmu;
>>> +
>>> /* private: */
>>>
>>> #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
>>> diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
>>> index 57d84751e160..477d0ae5f230 100644
>>> --- a/drivers/gpu/drm/xe/xe_gt.c
>>> +++ b/drivers/gpu/drm/xe/xe_gt.c
>>> @@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
>>> if (err)
>>> goto err_msg;
>>>
>>> + xe_pmu_suspend(gt);
>>> +
>>> err = xe_uc_suspend(>->uc);
>>> if (err)
>>> goto err_force_wake;
>>> diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
>>> index 3edeb30d5ccb..26f814f97fc2 100644
>>> --- a/drivers/gpu/drm/xe/xe_module.c
>>> +++ b/drivers/gpu/drm/xe/xe_module.c
>>> @@ -11,6 +11,7 @@
>>> #include "xe_drv.h"
>>> #include "xe_hw_fence.h"
>>> #include "xe_pci.h"
>>> +#include "xe_pmu.h"
>>> #include "xe_sched_job.h"
>>>
>>> struct xe_modparam xe_modparam = {
>>> @@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
>>> .init = xe_sched_job_module_init,
>>> .exit = xe_sched_job_module_exit,
>>> },
>>> + {
>>> + .init = xe_pmu_init,
>>> + .exit = xe_pmu_exit,
>>> + },
>>> {
>>> .init = xe_register_pci_driver,
>>> .exit = xe_unregister_pci_driver,
>>> diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>>> new file mode 100644
>>> index 000000000000..64960a358af2
>>> --- /dev/null
>>> +++ b/drivers/gpu/drm/xe/xe_pmu.c
>>> @@ -0,0 +1,631 @@
>>> +// SPDX-License-Identifier: MIT
>>> +/*
>>> + * Copyright © 2024 Intel Corporation
>>> + */
>>> +
>>> +#include <drm/drm_drv.h>
>>> +#include <drm/drm_managed.h>
>>> +#include <drm/xe_drm.h>
>>> +
>>> +#include "regs/xe_gt_regs.h"
>>> +#include "xe_device.h"
>>> +#include "xe_force_wake.h"
>>> +#include "xe_gt_clock.h"
>>> +#include "xe_mmio.h"
>>> +#include "xe_macros.h"
>>> +#include "xe_pm.h"
>>> +
>>> +static cpumask_t xe_pmu_cpumask;
>>> +static unsigned int xe_pmu_target_cpu = -1;
>>> +
>>> +static unsigned int config_gt_id(const u64 config)
>>> +{
>>> + return config >> __XE_PMU_GT_SHIFT;
>>> +}
>>> +
>>> +static u64 config_counter(const u64 config)
>>> +{
>>> + return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>>> +}
>>> +
>>> +static void xe_pmu_event_destroy(struct perf_event *event)
>>> +{
>>> + struct xe_device *xe =
>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>> +
>>> + drm_WARN_ON(&xe->drm, event->parent);
>>> +
>>> + drm_dev_put(&xe->drm);
>>> +}
>>> +
>>> +static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
>>> +{
>>> + u64 val;
>>> +
>>> + switch (sample_type) {
>>> + case __XE_SAMPLE_RENDER_GROUP_BUSY:
>>> + val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>>> + break;
>>> + case __XE_SAMPLE_COPY_GROUP_BUSY:
>>> + val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>>> + break;
>>> + case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>>> + val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>>> + break;
>>> + case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>>> + val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>>> + break;
>>> + default:
>>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>> + }
>>> +
>>> + return xe_gt_clock_cycles_to_ns(gt, val * 16);
>>> +}
>>> +
>>> +static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>>> +{
>>> + int sample_type = config_counter(config);
>>> + const unsigned int gt_id = gt->info.id;
>>> + struct xe_device *xe = gt->tile->xe;
>>> + struct xe_pmu *pmu = &xe->pmu;
>>> + unsigned long flags;
>>> + bool device_awake;
>>> + u64 val;
>>> +
>>> + device_awake = xe_pm_runtime_get_if_active(xe);
>>> + if (device_awake) {
>>> + XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>>> + val = __engine_group_busyness_read(gt, sample_type);
>>> + XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>>> + xe_pm_runtime_put(xe);
>>> + }
>>> +
>>> + spin_lock_irqsave(&pmu->lock, flags);
>>> +
>>> + if (device_awake)
>>> + pmu->sample[gt_id][sample_type] = val;
>>> + else
>>> + val = pmu->sample[gt_id][sample_type];
>>> +
>>> + spin_unlock_irqrestore(&pmu->lock, flags);
>>> +
>>> + return val;
>>> +}
>>> +
>>> +static void engine_group_busyness_store(struct xe_gt *gt)
>>> +{
>>> + struct xe_pmu *pmu = >->tile->xe->pmu;
>>> + unsigned int gt_id = gt->info.id;
>>> + unsigned long flags;
>>> + int i;
>>> +
>>> + spin_lock_irqsave(&pmu->lock, flags);
>>> +
>>> + for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>>> + pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>>> +
>>> + spin_unlock_irqrestore(&pmu->lock, flags);
>>> +}
>>> +
>>> +static int
>>> +config_status(struct xe_device *xe, u64 config)
>>> +{
>>> + unsigned int gt_id = config_gt_id(config);
>>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>> +
>>> + if (gt_id >= XE_PMU_MAX_GT)
>>> + return -ENOENT;
>>> +
>>> + switch (config_counter(config)) {
>>> + case XE_PMU_RENDER_GROUP_BUSY(0):
>>> + case XE_PMU_COPY_GROUP_BUSY(0):
>>> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>> + if (gt->info.type == XE_GT_TYPE_MEDIA)
>>> + return -ENOENT;
>>> + break;
>>> + case XE_PMU_MEDIA_GROUP_BUSY(0):
>>> + if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
>>> + return -ENOENT;
>>> + break;
>>> + default:
>>> + return -ENOENT;
>>> + }
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static int xe_pmu_event_init(struct perf_event *event)
>>> +{
>>> + struct xe_device *xe =
>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>> + struct xe_pmu *pmu = &xe->pmu;
>>> + int ret;
>>> +
>>> + if (pmu->closed)
>>> + return -ENODEV;
>>> +
>>> + if (event->attr.type != event->pmu->type)
>>> + return -ENOENT;
>>> +
>>> + /* unsupported modes and filters */
>>> + if (event->attr.sample_period) /* no sampling */
>>> + return -EINVAL;
>>> +
>>> + if (has_branch_stack(event))
>>> + return -EOPNOTSUPP;
>>> +
>>> + if (event->cpu < 0)
>>> + return -EINVAL;
>>> +
>>> + /* only allow running on one cpu at a time */
>>> + if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>>> + return -EINVAL;
>>> +
>>> + ret = config_status(xe, event->attr.config);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + if (!event->parent) {
>>> + drm_dev_get(&xe->drm);
>>> + event->destroy = xe_pmu_event_destroy;
>>> + }
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static u64 __xe_pmu_event_read(struct perf_event *event)
>>> +{
>>> + struct xe_device *xe =
>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>> + const unsigned int gt_id = config_gt_id(event->attr.config);
>>> + const u64 config = event->attr.config;
>>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>> + u64 val;
>>> +
>>> + switch (config_counter(config)) {
>>> + case XE_PMU_RENDER_GROUP_BUSY(0):
>>> + case XE_PMU_COPY_GROUP_BUSY(0):
>>> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>> + case XE_PMU_MEDIA_GROUP_BUSY(0):
>>> + val = engine_group_busyness_read(gt, config);
>>> + break;
>>> + default:
>>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>> + }
>>> +
>>> + return val;
>>> +}
>>> +
>>> +static void xe_pmu_event_read(struct perf_event *event)
>>> +{
>>> + struct xe_device *xe =
>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>> + struct hw_perf_event *hwc = &event->hw;
>>> + struct xe_pmu *pmu = &xe->pmu;
>>> + u64 prev, new;
>>> +
>>> + if (pmu->closed) {
>>> + event->hw.state = PERF_HES_STOPPED;
>>> + return;
>>> + }
>>> +again:
>>> + prev = local64_read(&hwc->prev_count);
>>> + new = __xe_pmu_event_read(event);
>>> +
>>> + if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>>> + goto again;
>>> +
>>> + local64_add(new - prev, &event->count);
>>> +}
>>> +
>>> +static void xe_pmu_enable(struct perf_event *event)
>>> +{
>>> + /*
>>> + * Store the current counter value so we can report the correct delta
>>> + * for all listeners. Even when the event was already enabled and has
>>> + * an existing non-zero value.
>>> + */
>>> + local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>>> +}
>>> +
>>> +static void xe_pmu_event_start(struct perf_event *event, int flags)
>>> +{
>>> + struct xe_device *xe =
>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>> + struct xe_pmu *pmu = &xe->pmu;
>>> +
>>> + if (pmu->closed)
>>> + return;
>>> +
>>> + xe_pmu_enable(event);
>>> + event->hw.state = 0;
>>> +}
>>> +
>>> +static void xe_pmu_event_stop(struct perf_event *event, int flags)
>>> +{
>>> + if (flags & PERF_EF_UPDATE)
>>> + xe_pmu_event_read(event);
>>> +
>>> + event->hw.state = PERF_HES_STOPPED;
>>> +}
>>> +
>>> +static int xe_pmu_event_add(struct perf_event *event, int flags)
>>> +{
>>> + struct xe_device *xe =
>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>> + struct xe_pmu *pmu = &xe->pmu;
>>> +
>>> + if (pmu->closed)
>>> + return -ENODEV;
>>> +
>>> + if (flags & PERF_EF_START)
>>> + xe_pmu_event_start(event, flags);
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static void xe_pmu_event_del(struct perf_event *event, int flags)
>>> +{
>>> + xe_pmu_event_stop(event, PERF_EF_UPDATE);
>>> +}
>>> +
>>> +static int xe_pmu_event_event_idx(struct perf_event *event)
>>> +{
>>> + return 0;
>>> +}
>>> +
>>> +struct xe_ext_attribute {
>>> + struct device_attribute attr;
>>> + unsigned long val;
>>> +};
>>> +
>>> +static ssize_t xe_pmu_event_show(struct device *dev,
>>> + struct device_attribute *attr, char *buf)
>>> +{
>>> + struct xe_ext_attribute *eattr;
>>> +
>>> + eattr = container_of(attr, struct xe_ext_attribute, attr);
>>> + return sprintf(buf, "config=0x%lx\n", eattr->val);
>>> +}
>>> +
>>> +static ssize_t cpumask_show(struct device *dev,
>>> + struct device_attribute *attr, char *buf)
>>> +{
>>> + return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>>> +}
>>> +
>>> +static DEVICE_ATTR_RO(cpumask);
>>> +
>>> +static struct attribute *xe_cpumask_attrs[] = {
>>> + &dev_attr_cpumask.attr,
>>> + NULL,
>>> +};
>>> +
>>> +static const struct attribute_group xe_pmu_cpumask_attr_group = {
>>> + .attrs = xe_cpumask_attrs,
>>> +};
>>> +
>>> +#define __event(__counter, __name, __unit) \
>>> +{ \
>>> + .counter = (__counter), \
>>> + .name = (__name), \
>>> + .unit = (__unit), \
>>> +}
>>> +
>>> +static struct xe_ext_attribute *
>>> +add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>>> +{
>>> + sysfs_attr_init(&attr->attr.attr);
>>> + attr->attr.attr.name = name;
>>> + attr->attr.attr.mode = 0444;
>>> + attr->attr.show = xe_pmu_event_show;
>>> + attr->val = config;
>>> +
>>> + return ++attr;
>>> +}
>>> +
>>> +static struct perf_pmu_events_attr *
>>> +add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>>> + const char *str)
>>> +{
>>> + sysfs_attr_init(&attr->attr.attr);
>>> + attr->attr.attr.name = name;
>>> + attr->attr.attr.mode = 0444;
>>> + attr->attr.show = perf_event_sysfs_show;
>>> + attr->event_str = str;
>>> +
>>> + return ++attr;
>>> +}
>>> +
>>> +static struct attribute **
>>> +create_event_attributes(struct xe_pmu *pmu)
>>> +{
>>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>> + static const struct {
>>> + unsigned int counter;
>>> + const char *name;
>>> + const char *unit;
>>> + } events[] = {
>>> + __event(0, "render-group-busy", "ns"),
>>> + __event(1, "copy-group-busy", "ns"),
>>> + __event(2, "media-group-busy", "ns"),
>>> + __event(3, "any-engine-group-busy", "ns"),
>>> + };
>>> +
>>> + struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>>> + struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>>> + struct attribute **attr = NULL, **attr_iter;
>>> + unsigned int count = 0;
>>> + unsigned int i, j;
>>> + struct xe_gt *gt;
>>> +
>>> + /* Count how many counters we will be exposing. */
>>> + for_each_gt(gt, xe, j) {
>>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>> +
>>> + if (!config_status(xe, config))
>>> + count++;
>>> + }
>>> + }
>>> +
>>> + /* Allocate attribute objects and table. */
>>> + xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>>> + if (!xe_attr)
>>> + goto err_alloc;
>>> +
>>> + pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>>> + if (!pmu_attr)
>>> + goto err_alloc;
>>> +
>>> + /* Max one pointer of each attribute type plus a termination entry. */
>>> + attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>>> + if (!attr)
>>> + goto err_alloc;
>>> +
>>> + xe_iter = xe_attr;
>>> + pmu_iter = pmu_attr;
>>> + attr_iter = attr;
>>> +
>>> + for_each_gt(gt, xe, j) {
>>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>> + char *str;
>>> +
>>> + if (config_status(xe, config))
>>> + continue;
>>> +
>>> + str = kasprintf(GFP_KERNEL, "%s-gt%u",
>>> + events[i].name, j);
>>> + if (!str)
>>> + goto err;
>>> +
>>> + *attr_iter++ = &xe_iter->attr.attr;
>>> + xe_iter = add_xe_attr(xe_iter, str, config);
>>> +
>>> + if (events[i].unit) {
>>> + str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>>> + events[i].name, j);
>>> + if (!str)
>>> + goto err;
>>> +
>>> + *attr_iter++ = &pmu_iter->attr.attr;
>>> + pmu_iter = add_pmu_attr(pmu_iter, str,
>>> + events[i].unit);
>>> + }
>>> + }
>>> + }
>>> +
>>> + pmu->xe_attr = xe_attr;
>>> + pmu->pmu_attr = pmu_attr;
>>> +
>>> + return attr;
>>> +
>>> +err:
>>> + for (attr_iter = attr; *attr_iter; attr_iter++)
>>> + kfree((*attr_iter)->name);
>>> +
>>> +err_alloc:
>>> + kfree(attr);
>>> + kfree(xe_attr);
>>> + kfree(pmu_attr);
>>> +
>>> + return NULL;
>>> +}
>>> +
>>> +static void free_event_attributes(struct xe_pmu *pmu)
>>> +{
>>> + struct attribute **attr_iter = pmu->events_attr_group.attrs;
>>> +
>>> + for (; *attr_iter; attr_iter++)
>>> + kfree((*attr_iter)->name);
>>> +
>>> + kfree(pmu->events_attr_group.attrs);
>>> + kfree(pmu->xe_attr);
>>> + kfree(pmu->pmu_attr);
>>> +
>>> + pmu->events_attr_group.attrs = NULL;
>>> + pmu->xe_attr = NULL;
>>> + pmu->pmu_attr = NULL;
>>> +}
>>> +
>>> +static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>>> +{
>>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>> +
>>> + /* Select the first online CPU as a designated reader. */
>>> + if (cpumask_empty(&xe_pmu_cpumask))
>>> + cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>>> +{
>>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>> + unsigned int target = xe_pmu_target_cpu;
>>> +
>>> + /*
>>> + * Unregistering an instance generates a CPU offline event which we must
>>> + * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>>> + */
>>> + if (pmu->closed)
>>> + return 0;
>>> +
>>> + if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>>> + target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>>> +
>>> + /* Migrate events if there is a valid target */
>>> + if (target < nr_cpu_ids) {
>>> + cpumask_set_cpu(target, &xe_pmu_cpumask);
>>> + xe_pmu_target_cpu = target;
>>> + }
>>> + }
>>> +
>>> + if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>>> + perf_pmu_migrate_context(&pmu->base, cpu, target);
>>> + pmu->cpuhp.cpu = target;
>>> + }
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>>> +
>>> +int xe_pmu_init(void)
>>> +{
>>> + int ret;
>>> +
>>> + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>>> + "perf/x86/intel/xe:online",
>>> + xe_pmu_cpu_online,
>>> + xe_pmu_cpu_offline);
>>> + if (ret < 0)
>>> + pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>>> + ret);
>>> + else
>>> + cpuhp_slot = ret;
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +void xe_pmu_exit(void)
>>> +{
>>> + if (cpuhp_slot != CPUHP_INVALID)
>>> + cpuhp_remove_multi_state(cpuhp_slot);
>>> +}
>>> +
>>> +static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>>> +{
>>> + if (cpuhp_slot == CPUHP_INVALID)
>>> + return -EINVAL;
>>> +
>>> + return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>>> +}
>>> +
>>> +static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>>> +{
>>> + cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>>> +}
>>> +
>>> +void xe_pmu_suspend(struct xe_gt *gt)
>>> +{
>>> + engine_group_busyness_store(gt);
>>> +}
>>> +
>>> +static void xe_pmu_unregister(void *arg)
>>> +{
>>> + struct xe_pmu *pmu = arg;
>>> +
>>> + if (!pmu->base.event_init)
>>> + return;
>>> +
>>> + /*
>>> + * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
>>> + * ensures all currently executing ones will have exited before we
>>> + * proceed with unregistration.
>>> + */
>>> + pmu->closed = true;
>>> + synchronize_rcu();
>>> +
>>> + xe_pmu_unregister_cpuhp_state(pmu);
>>> +
>>> + perf_pmu_unregister(&pmu->base);
>>> + pmu->base.event_init = NULL;
>>> + kfree(pmu->base.attr_groups);
>>> + kfree(pmu->name);
>>> + free_event_attributes(pmu);
>>> +}
>>> +
>>> +void xe_pmu_register(struct xe_pmu *pmu)
>>> +{
>>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>> + const struct attribute_group *attr_groups[] = {
>>> + &pmu->events_attr_group,
>>> + &xe_pmu_cpumask_attr_group,
>>> + NULL
>>> + };
>>> +
>>> + int ret = -ENOMEM;
>>> +
>>> + spin_lock_init(&pmu->lock);
>>> + pmu->cpuhp.cpu = -1;
>>> +
>>> + pmu->name = kasprintf(GFP_KERNEL,
>>> + "xe_%s",
>>> + dev_name(xe->drm.dev));
>>> + if (pmu->name)
>>> + /* tools/perf reserves colons as special. */
>>> + strreplace((char *)pmu->name, ':', '_');
>>> +
>>> + if (!pmu->name)
>>> + goto err;
>>> +
>>> + pmu->events_attr_group.name = "events";
>>> + pmu->events_attr_group.attrs = create_event_attributes(pmu);
>>> + if (!pmu->events_attr_group.attrs)
>>> + goto err_name;
>>> +
>>> + pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>>> + GFP_KERNEL);
>>> + if (!pmu->base.attr_groups)
>>> + goto err_attr;
>>> +
>>> + pmu->base.module = THIS_MODULE;
>>> + pmu->base.task_ctx_nr = perf_invalid_context;
>>> + pmu->base.event_init = xe_pmu_event_init;
>>> + pmu->base.add = xe_pmu_event_add;
>>> + pmu->base.del = xe_pmu_event_del;
>>> + pmu->base.start = xe_pmu_event_start;
>>> + pmu->base.stop = xe_pmu_event_stop;
>>> + pmu->base.read = xe_pmu_event_read;
>>> + pmu->base.event_idx = xe_pmu_event_event_idx;
>>> +
>>> + ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>>> + if (ret)
>>> + goto err_groups;
>>> +
>>> + ret = xe_pmu_register_cpuhp_state(pmu);
>>> + if (ret)
>>> + goto err_unreg;
>>> +
>>> + ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>>> + if (ret)
>>> + goto err_cpuhp;
>>> +
>>> + return;
>>> +
>>> +err_cpuhp:
>>> + xe_pmu_unregister_cpuhp_state(pmu);
>>> +err_unreg:
>>> + perf_pmu_unregister(&pmu->base);
>>> +err_groups:
>>> + kfree(pmu->base.attr_groups);
>>> +err_attr:
>>> + pmu->base.event_init = NULL;
>>> + free_event_attributes(pmu);
>>> +err_name:
>>> + kfree(pmu->name);
>>> +err:
>>> + drm_notice(&xe->drm, "Failed to register PMU!\n");
>>> +}
>>> diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>>> new file mode 100644
>>> index 000000000000..8afa256f9dac
>>> --- /dev/null
>>> +++ b/drivers/gpu/drm/xe/xe_pmu.h
>>> @@ -0,0 +1,26 @@
>>> +/* SPDX-License-Identifier: MIT */
>>> +/*
>>> + * Copyright © 2024 Intel Corporation
>>> + */
>>> +
>>> +#ifndef _XE_PMU_H_
>>> +#define _XE_PMU_H_
>>> +
>>> +#include "xe_pmu_types.h"
>>> +
>>> +struct xe_gt;
>>> +
>>> +#if IS_ENABLED(CONFIG_PERF_EVENTS)
>>> +int xe_pmu_init(void);
>>> +void xe_pmu_exit(void);
>>> +void xe_pmu_register(struct xe_pmu *pmu);
>>> +void xe_pmu_suspend(struct xe_gt *gt);
>>> +#else
>>> +static inline int xe_pmu_init(void) { return 0; }
>>> +static inline void xe_pmu_exit(void) {}
>>> +static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>>> +static inline void xe_pmu_suspend(struct xe_gt *gt) {}
>>> +#endif
>>> +
>>> +#endif
>>> +
>>> diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
>>> new file mode 100644
>>> index 000000000000..e86e8d7e0356
>>> --- /dev/null
>>> +++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>>> @@ -0,0 +1,67 @@
>>> +/* SPDX-License-Identifier: MIT */
>>> +/*
>>> + * Copyright © 2024 Intel Corporation
>>> + */
>>> +
>>> +#ifndef _XE_PMU_TYPES_H_
>>> +#define _XE_PMU_TYPES_H_
>>> +
>>> +#include <linux/perf_event.h>
>>> +#include <linux/spinlock_types.h>
>>> +#include <uapi/drm/xe_drm.h>
>>> +
>>> +enum {
>>> + __XE_SAMPLE_RENDER_GROUP_BUSY,
>>> + __XE_SAMPLE_COPY_GROUP_BUSY,
>>> + __XE_SAMPLE_MEDIA_GROUP_BUSY,
>>> + __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
>>> + __XE_NUM_PMU_SAMPLERS
>>> +};
>>> +
>>> +#define XE_PMU_MAX_GT 2
>>> +
>>> +struct xe_pmu {
>>> + /**
>>> + * @cpuhp: Struct used for CPU hotplug handling.
>>> + */
>>> + struct {
>>> + struct hlist_node node;
>>> + unsigned int cpu;
>>> + } cpuhp;
>>> + /**
>>> + * @base: PMU base.
>>> + */
>>> + struct pmu base;
>>> + /**
>>> + * @closed: xe is unregistering.
>>> + */
>>> + bool closed;
>>> + /**
>>> + * @name: Name as registered with perf core.
>>> + */
>>> + const char *name;
>>> + /**
>>> + * @lock: Lock protecting enable mask and ref count handling.
>>> + */
>>> + spinlock_t lock;
>>> + /**
>>> + * @sample: Current and previous (raw) counters.
>>> + *
>>> + * These counters are updated when the device is awake.
>>> + */
>>> + u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>>> + /**
>>> + * @events_attr_group: Device events attribute group.
>>> + */
>>> + struct attribute_group events_attr_group;
>>> + /**
>>> + * @xe_attr: Memory block holding device attributes.
>>> + */
>>> + void *xe_attr;
>>> + /**
>>> + * @pmu_attr: Memory block holding device attributes.
>>> + */
>>> + void *pmu_attr;
>>> +};
>>> +
>>> +#endif
>>> diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>>> index d7b0903c22b2..07ca545354f7 100644
>>> --- a/include/uapi/drm/xe_drm.h
>>> +++ b/include/uapi/drm/xe_drm.h
>>> @@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
>>> __u64 reserved[2];
>>> };
>>>
>>> +/**
>>> + * DOC: XE PMU event config IDs
>>> + *
>>> + * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in xe_drm.h
>>> + * in 'struct perf_event_attr' as part of perf_event_open syscall to read a
>>> + * particular event.
>>> + *
>>> + * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
>>> + *
>>> + * .. code-block:: C
>>> + *
>>> + * struct perf_event_attr attr;
>>> + * long long count;
>>> + * int cpu = 0;
>>> + * int fd;
>>> + *
>>> + * memset(&attr, 0, sizeof(struct perf_event_attr));
>>> + * attr.type = type; // eg: /sys/bus/event_source/devices/xe_0000_56_00.0/type
>>> + * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
>>> + * attr.use_clockid = 1;
>>> + * attr.clockid = CLOCK_MONOTONIC;
>>> + * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
>>> + *
>>> + * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
>>> + */
>>> +
>>> +/*
>>> + * Top bits of every counter are GT id.
>>> + */
>>> +#define __XE_PMU_GT_SHIFT (56)
>>> +
>>> +#define ___XE_PMU_OTHER(gt, x) \
>>> + (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>>> +
>>> +#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
>>> +#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
>>> +#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
>>> +#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
>>
>> + Lucas for inputs
>>
>> We should align this to the interface planned for other PMU busyness counters as well as how we do PCEU. i.e.
>>
>> 1) counters are in ticks
>> 2) total time in ticks is also exported to the user.
>>
>> For 1), I would just append TICKS to the counter names and drop the conversion to _ns in __engine_group_busyness_read(). Also, drop the patch that adds this conversion helper.
>>
>> For 2) define a new counter - total active ticks that would return the 'CPU' timestamp converted to gpu ticks. The reason I am insisting on CPU timestamp here is because we want to have a time base that is ticking even when the GPU is idle.
>
>why can't we expose what the HW presents[1] to us via register and leave the interpretation to userspace.
HW is indeed exposing ticks. In this case, I am suggesting to expose
that directly to the user, so I think you are saying the same.
As for interpretation, we need to make sure it works consistently in
SRIOV. The L0 API for group busyness itself imposes the requirement for
another counter to make sense of [1]. This additional counter has always
existed, but in prior implementations, it was just using the CPU time in
the equation. The CPU sample time is always returned in all the PMU
counters. With SRIOV, it will still be CPU time, but only the time that
a VF executed for and that information is only available to the KMD/GuC.
Without that information, interpreting the ticks in [1] will not be
meaningful.
Regards,
Umesh
>
>Thanks,
>Aravind.
>>
>> Regards,
>> Umesh
>>
>>> +
>>> #if defined(__cplusplus)
>>> }
>>> #endif
>>> --
>>> 2.40.0
>>>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-27 16:05 ` Umesh Nerlige Ramappa
@ 2024-06-28 9:41 ` Aravind Iddamsetty
2024-06-28 16:36 ` Umesh Nerlige Ramappa
0 siblings, 1 reply; 32+ messages in thread
From: Aravind Iddamsetty @ 2024-06-28 9:41 UTC (permalink / raw)
To: Umesh Nerlige Ramappa
Cc: Riana Tauro, intel-xe, anshuman.gupta, ashutosh.dixit,
rodrigo.vivi, krishnaiah.bommu, lucas.demarchi, Joonas Lahtinen
On 27/06/24 21:35, Umesh Nerlige Ramappa wrote:
> On Thu, Jun 27, 2024 at 12:19:44PM +0530, Aravind Iddamsetty wrote:
>>
>> On 21/06/24 01:22, Umesh Nerlige Ramappa wrote:
>>> On Thu, Jun 13, 2024 at 03:34:11PM +0530, Riana Tauro wrote:
>>>> From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>>>
>>>> There are a set of engine group busyness counters provided by HW which are
>>>> perfect fit to be exposed via PMU perf events.
>>>>
>>>> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>>>
>>>> events can be listed using:
>>>> perf list
>>>> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>>>> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>>>> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>>>> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>>>
>>>> and can be read using:
>>>>
>>>> perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
>>>> time counts unit events
>>>> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>
>>>> The pmu base implementation is taken from i915.
>>>>
>>>> v2:
>>>> Store last known value when device is awake return that while the GT is
>>>> suspended and then update the driver copy when read during awake.
>>>>
>>>> v3:
>>>> 1. drop init_samples, as storing counters before going to suspend should
>>>> be sufficient.
>>>> 2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>>>> dropped helpers to store and read samples.
>>>> 3. use xe_device_mem_access_get_if_ongoing to check if device is active
>>>> before reading the OA registers.
>>>> 4. dropped format attr as no longer needed
>>>> 5. introduce xe_pmu_suspend to call engine_group_busyness_store
>>>> 6. few other nits.
>>>>
>>>> v4: minor nits.
>>>>
>>>> v5: take forcewake when accessing the OAG registers
>>>>
>>>> v6:
>>>> 1. drop engine_busyness_sample_type
>>>> 2. update UAPI documentation
>>>>
>>>> v7:
>>>> 1. update UAPI documentation
>>>> 2. drop MEDIA_GT specific change for media busyness counter.
>>>>
>>>> v8:
>>>> 1. rebase
>>>> 2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>>>> 3. remove interrupts pmu event
>>>>
>>>> v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>>>>
>>>> Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>>> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>>> Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>>> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>>> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>>> Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
>>>> Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>>>> ---
>>>> drivers/gpu/drm/xe/Makefile | 2 +
>>>> drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
>>>> drivers/gpu/drm/xe/xe_device.c | 2 +
>>>> drivers/gpu/drm/xe/xe_device_types.h | 4 +
>>>> drivers/gpu/drm/xe/xe_gt.c | 2 +
>>>> drivers/gpu/drm/xe/xe_module.c | 5 +
>>>> drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
>>>> drivers/gpu/drm/xe/xe_pmu.h | 26 ++
>>>> drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
>>>> include/uapi/drm/xe_drm.h | 39 ++
>>>> 10 files changed, 783 insertions(+)
>>>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
>>>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
>>>> create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>>>>
>>>> diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>>>> index cbf961b90237..83bf1e07669b 100644
>>>> --- a/drivers/gpu/drm/xe/Makefile
>>>> +++ b/drivers/gpu/drm/xe/Makefile
>>>> @@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
>>>> i915-display/skl_universal_plane.o \
>>>> i915-display/skl_watermark.o
>>>>
>>>> +xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>>>> +
>>>> ifeq ($(CONFIG_ACPI),y)
>>>> xe-$(CONFIG_DRM_XE_DISPLAY) += \
>>>> i915-display/intel_acpi.o \
>>>> diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>> index 47c26c37608d..22821dcd4e1b 100644
>>>> --- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>> +++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>> @@ -390,6 +390,11 @@
>>>> #define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
>>>> #define GLOBAL_INVALIDATION_MODE REG_BIT(2)
>>>>
>>>> +#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
>>>> +#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
>>>> +#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
>>>> +#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
>>>> +
>>>> #define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
>>>> #define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
>>>>
>>>> diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
>>>> index 64691a56d59c..bb00c8c9ec9b 100644
>>>> --- a/drivers/gpu/drm/xe/xe_device.c
>>>> +++ b/drivers/gpu/drm/xe/xe_device.c
>>>> @@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
>>>>
>>>> xe_hwmon_register(xe);
>>>>
>>>> + xe_pmu_register(&xe->pmu);
>>>> +
>>>> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>>>>
>>>> err_fini_display:
>>>> diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
>>>> index 52bc461171d5..a5dba7325cf1 100644
>>>> --- a/drivers/gpu/drm/xe/xe_device_types.h
>>>> +++ b/drivers/gpu/drm/xe/xe_device_types.h
>>>> @@ -18,6 +18,7 @@
>>>> #include "xe_lmtt_types.h"
>>>> #include "xe_memirq_types.h"
>>>> #include "xe_platform_types.h"
>>>> +#include "xe_pmu.h"
>>>> #include "xe_pt_types.h"
>>>> #include "xe_sriov_types.h"
>>>> #include "xe_step_types.h"
>>>> @@ -473,6 +474,9 @@ struct xe_device {
>>>> int mode;
>>>> } wedged;
>>>>
>>>> + /** @pmu: performance monitoring unit */
>>>> + struct xe_pmu pmu;
>>>> +
>>>> /* private: */
>>>>
>>>> #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
>>>> diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
>>>> index 57d84751e160..477d0ae5f230 100644
>>>> --- a/drivers/gpu/drm/xe/xe_gt.c
>>>> +++ b/drivers/gpu/drm/xe/xe_gt.c
>>>> @@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
>>>> if (err)
>>>> goto err_msg;
>>>>
>>>> + xe_pmu_suspend(gt);
>>>> +
>>>> err = xe_uc_suspend(>->uc);
>>>> if (err)
>>>> goto err_force_wake;
>>>> diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
>>>> index 3edeb30d5ccb..26f814f97fc2 100644
>>>> --- a/drivers/gpu/drm/xe/xe_module.c
>>>> +++ b/drivers/gpu/drm/xe/xe_module.c
>>>> @@ -11,6 +11,7 @@
>>>> #include "xe_drv.h"
>>>> #include "xe_hw_fence.h"
>>>> #include "xe_pci.h"
>>>> +#include "xe_pmu.h"
>>>> #include "xe_sched_job.h"
>>>>
>>>> struct xe_modparam xe_modparam = {
>>>> @@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
>>>> .init = xe_sched_job_module_init,
>>>> .exit = xe_sched_job_module_exit,
>>>> },
>>>> + {
>>>> + .init = xe_pmu_init,
>>>> + .exit = xe_pmu_exit,
>>>> + },
>>>> {
>>>> .init = xe_register_pci_driver,
>>>> .exit = xe_unregister_pci_driver,
>>>> diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>>>> new file mode 100644
>>>> index 000000000000..64960a358af2
>>>> --- /dev/null
>>>> +++ b/drivers/gpu/drm/xe/xe_pmu.c
>>>> @@ -0,0 +1,631 @@
>>>> +// SPDX-License-Identifier: MIT
>>>> +/*
>>>> + * Copyright © 2024 Intel Corporation
>>>> + */
>>>> +
>>>> +#include <drm/drm_drv.h>
>>>> +#include <drm/drm_managed.h>
>>>> +#include <drm/xe_drm.h>
>>>> +
>>>> +#include "regs/xe_gt_regs.h"
>>>> +#include "xe_device.h"
>>>> +#include "xe_force_wake.h"
>>>> +#include "xe_gt_clock.h"
>>>> +#include "xe_mmio.h"
>>>> +#include "xe_macros.h"
>>>> +#include "xe_pm.h"
>>>> +
>>>> +static cpumask_t xe_pmu_cpumask;
>>>> +static unsigned int xe_pmu_target_cpu = -1;
>>>> +
>>>> +static unsigned int config_gt_id(const u64 config)
>>>> +{
>>>> + return config >> __XE_PMU_GT_SHIFT;
>>>> +}
>>>> +
>>>> +static u64 config_counter(const u64 config)
>>>> +{
>>>> + return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>>>> +}
>>>> +
>>>> +static void xe_pmu_event_destroy(struct perf_event *event)
>>>> +{
>>>> + struct xe_device *xe =
>>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>>> +
>>>> + drm_WARN_ON(&xe->drm, event->parent);
>>>> +
>>>> + drm_dev_put(&xe->drm);
>>>> +}
>>>> +
>>>> +static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
>>>> +{
>>>> + u64 val;
>>>> +
>>>> + switch (sample_type) {
>>>> + case __XE_SAMPLE_RENDER_GROUP_BUSY:
>>>> + val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>>>> + break;
>>>> + case __XE_SAMPLE_COPY_GROUP_BUSY:
>>>> + val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>>>> + break;
>>>> + case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>>>> + val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>>>> + break;
>>>> + case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>>>> + val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>>>> + break;
>>>> + default:
>>>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>>> + }
>>>> +
>>>> + return xe_gt_clock_cycles_to_ns(gt, val * 16);
>>>> +}
>>>> +
>>>> +static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>>>> +{
>>>> + int sample_type = config_counter(config);
>>>> + const unsigned int gt_id = gt->info.id;
>>>> + struct xe_device *xe = gt->tile->xe;
>>>> + struct xe_pmu *pmu = &xe->pmu;
>>>> + unsigned long flags;
>>>> + bool device_awake;
>>>> + u64 val;
>>>> +
>>>> + device_awake = xe_pm_runtime_get_if_active(xe);
>>>> + if (device_awake) {
>>>> + XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>>>> + val = __engine_group_busyness_read(gt, sample_type);
>>>> + XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>>>> + xe_pm_runtime_put(xe);
>>>> + }
>>>> +
>>>> + spin_lock_irqsave(&pmu->lock, flags);
>>>> +
>>>> + if (device_awake)
>>>> + pmu->sample[gt_id][sample_type] = val;
>>>> + else
>>>> + val = pmu->sample[gt_id][sample_type];
>>>> +
>>>> + spin_unlock_irqrestore(&pmu->lock, flags);
>>>> +
>>>> + return val;
>>>> +}
>>>> +
>>>> +static void engine_group_busyness_store(struct xe_gt *gt)
>>>> +{
>>>> + struct xe_pmu *pmu = >->tile->xe->pmu;
>>>> + unsigned int gt_id = gt->info.id;
>>>> + unsigned long flags;
>>>> + int i;
>>>> +
>>>> + spin_lock_irqsave(&pmu->lock, flags);
>>>> +
>>>> + for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>>>> + pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>>>> +
>>>> + spin_unlock_irqrestore(&pmu->lock, flags);
>>>> +}
>>>> +
>>>> +static int
>>>> +config_status(struct xe_device *xe, u64 config)
>>>> +{
>>>> + unsigned int gt_id = config_gt_id(config);
>>>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>>> +
>>>> + if (gt_id >= XE_PMU_MAX_GT)
>>>> + return -ENOENT;
>>>> +
>>>> + switch (config_counter(config)) {
>>>> + case XE_PMU_RENDER_GROUP_BUSY(0):
>>>> + case XE_PMU_COPY_GROUP_BUSY(0):
>>>> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>>> + if (gt->info.type == XE_GT_TYPE_MEDIA)
>>>> + return -ENOENT;
>>>> + break;
>>>> + case XE_PMU_MEDIA_GROUP_BUSY(0):
>>>> + if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
>>>> + return -ENOENT;
>>>> + break;
>>>> + default:
>>>> + return -ENOENT;
>>>> + }
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static int xe_pmu_event_init(struct perf_event *event)
>>>> +{
>>>> + struct xe_device *xe =
>>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>>> + struct xe_pmu *pmu = &xe->pmu;
>>>> + int ret;
>>>> +
>>>> + if (pmu->closed)
>>>> + return -ENODEV;
>>>> +
>>>> + if (event->attr.type != event->pmu->type)
>>>> + return -ENOENT;
>>>> +
>>>> + /* unsupported modes and filters */
>>>> + if (event->attr.sample_period) /* no sampling */
>>>> + return -EINVAL;
>>>> +
>>>> + if (has_branch_stack(event))
>>>> + return -EOPNOTSUPP;
>>>> +
>>>> + if (event->cpu < 0)
>>>> + return -EINVAL;
>>>> +
>>>> + /* only allow running on one cpu at a time */
>>>> + if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>>>> + return -EINVAL;
>>>> +
>>>> + ret = config_status(xe, event->attr.config);
>>>> + if (ret)
>>>> + return ret;
>>>> +
>>>> + if (!event->parent) {
>>>> + drm_dev_get(&xe->drm);
>>>> + event->destroy = xe_pmu_event_destroy;
>>>> + }
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static u64 __xe_pmu_event_read(struct perf_event *event)
>>>> +{
>>>> + struct xe_device *xe =
>>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>>> + const unsigned int gt_id = config_gt_id(event->attr.config);
>>>> + const u64 config = event->attr.config;
>>>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>>> + u64 val;
>>>> +
>>>> + switch (config_counter(config)) {
>>>> + case XE_PMU_RENDER_GROUP_BUSY(0):
>>>> + case XE_PMU_COPY_GROUP_BUSY(0):
>>>> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>>> + case XE_PMU_MEDIA_GROUP_BUSY(0):
>>>> + val = engine_group_busyness_read(gt, config);
>>>> + break;
>>>> + default:
>>>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>>> + }
>>>> +
>>>> + return val;
>>>> +}
>>>> +
>>>> +static void xe_pmu_event_read(struct perf_event *event)
>>>> +{
>>>> + struct xe_device *xe =
>>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>>> + struct hw_perf_event *hwc = &event->hw;
>>>> + struct xe_pmu *pmu = &xe->pmu;
>>>> + u64 prev, new;
>>>> +
>>>> + if (pmu->closed) {
>>>> + event->hw.state = PERF_HES_STOPPED;
>>>> + return;
>>>> + }
>>>> +again:
>>>> + prev = local64_read(&hwc->prev_count);
>>>> + new = __xe_pmu_event_read(event);
>>>> +
>>>> + if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>>>> + goto again;
>>>> +
>>>> + local64_add(new - prev, &event->count);
>>>> +}
>>>> +
>>>> +static void xe_pmu_enable(struct perf_event *event)
>>>> +{
>>>> + /*
>>>> + * Store the current counter value so we can report the correct delta
>>>> + * for all listeners. Even when the event was already enabled and has
>>>> + * an existing non-zero value.
>>>> + */
>>>> + local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>>>> +}
>>>> +
>>>> +static void xe_pmu_event_start(struct perf_event *event, int flags)
>>>> +{
>>>> + struct xe_device *xe =
>>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>>> + struct xe_pmu *pmu = &xe->pmu;
>>>> +
>>>> + if (pmu->closed)
>>>> + return;
>>>> +
>>>> + xe_pmu_enable(event);
>>>> + event->hw.state = 0;
>>>> +}
>>>> +
>>>> +static void xe_pmu_event_stop(struct perf_event *event, int flags)
>>>> +{
>>>> + if (flags & PERF_EF_UPDATE)
>>>> + xe_pmu_event_read(event);
>>>> +
>>>> + event->hw.state = PERF_HES_STOPPED;
>>>> +}
>>>> +
>>>> +static int xe_pmu_event_add(struct perf_event *event, int flags)
>>>> +{
>>>> + struct xe_device *xe =
>>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>>> + struct xe_pmu *pmu = &xe->pmu;
>>>> +
>>>> + if (pmu->closed)
>>>> + return -ENODEV;
>>>> +
>>>> + if (flags & PERF_EF_START)
>>>> + xe_pmu_event_start(event, flags);
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static void xe_pmu_event_del(struct perf_event *event, int flags)
>>>> +{
>>>> + xe_pmu_event_stop(event, PERF_EF_UPDATE);
>>>> +}
>>>> +
>>>> +static int xe_pmu_event_event_idx(struct perf_event *event)
>>>> +{
>>>> + return 0;
>>>> +}
>>>> +
>>>> +struct xe_ext_attribute {
>>>> + struct device_attribute attr;
>>>> + unsigned long val;
>>>> +};
>>>> +
>>>> +static ssize_t xe_pmu_event_show(struct device *dev,
>>>> + struct device_attribute *attr, char *buf)
>>>> +{
>>>> + struct xe_ext_attribute *eattr;
>>>> +
>>>> + eattr = container_of(attr, struct xe_ext_attribute, attr);
>>>> + return sprintf(buf, "config=0x%lx\n", eattr->val);
>>>> +}
>>>> +
>>>> +static ssize_t cpumask_show(struct device *dev,
>>>> + struct device_attribute *attr, char *buf)
>>>> +{
>>>> + return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>>>> +}
>>>> +
>>>> +static DEVICE_ATTR_RO(cpumask);
>>>> +
>>>> +static struct attribute *xe_cpumask_attrs[] = {
>>>> + &dev_attr_cpumask.attr,
>>>> + NULL,
>>>> +};
>>>> +
>>>> +static const struct attribute_group xe_pmu_cpumask_attr_group = {
>>>> + .attrs = xe_cpumask_attrs,
>>>> +};
>>>> +
>>>> +#define __event(__counter, __name, __unit) \
>>>> +{ \
>>>> + .counter = (__counter), \
>>>> + .name = (__name), \
>>>> + .unit = (__unit), \
>>>> +}
>>>> +
>>>> +static struct xe_ext_attribute *
>>>> +add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>>>> +{
>>>> + sysfs_attr_init(&attr->attr.attr);
>>>> + attr->attr.attr.name = name;
>>>> + attr->attr.attr.mode = 0444;
>>>> + attr->attr.show = xe_pmu_event_show;
>>>> + attr->val = config;
>>>> +
>>>> + return ++attr;
>>>> +}
>>>> +
>>>> +static struct perf_pmu_events_attr *
>>>> +add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>>>> + const char *str)
>>>> +{
>>>> + sysfs_attr_init(&attr->attr.attr);
>>>> + attr->attr.attr.name = name;
>>>> + attr->attr.attr.mode = 0444;
>>>> + attr->attr.show = perf_event_sysfs_show;
>>>> + attr->event_str = str;
>>>> +
>>>> + return ++attr;
>>>> +}
>>>> +
>>>> +static struct attribute **
>>>> +create_event_attributes(struct xe_pmu *pmu)
>>>> +{
>>>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>>> + static const struct {
>>>> + unsigned int counter;
>>>> + const char *name;
>>>> + const char *unit;
>>>> + } events[] = {
>>>> + __event(0, "render-group-busy", "ns"),
>>>> + __event(1, "copy-group-busy", "ns"),
>>>> + __event(2, "media-group-busy", "ns"),
>>>> + __event(3, "any-engine-group-busy", "ns"),
>>>> + };
>>>> +
>>>> + struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>>>> + struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>>>> + struct attribute **attr = NULL, **attr_iter;
>>>> + unsigned int count = 0;
>>>> + unsigned int i, j;
>>>> + struct xe_gt *gt;
>>>> +
>>>> + /* Count how many counters we will be exposing. */
>>>> + for_each_gt(gt, xe, j) {
>>>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>>>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>>> +
>>>> + if (!config_status(xe, config))
>>>> + count++;
>>>> + }
>>>> + }
>>>> +
>>>> + /* Allocate attribute objects and table. */
>>>> + xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>>>> + if (!xe_attr)
>>>> + goto err_alloc;
>>>> +
>>>> + pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>>>> + if (!pmu_attr)
>>>> + goto err_alloc;
>>>> +
>>>> + /* Max one pointer of each attribute type plus a termination entry. */
>>>> + attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>>>> + if (!attr)
>>>> + goto err_alloc;
>>>> +
>>>> + xe_iter = xe_attr;
>>>> + pmu_iter = pmu_attr;
>>>> + attr_iter = attr;
>>>> +
>>>> + for_each_gt(gt, xe, j) {
>>>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>>>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>>> + char *str;
>>>> +
>>>> + if (config_status(xe, config))
>>>> + continue;
>>>> +
>>>> + str = kasprintf(GFP_KERNEL, "%s-gt%u",
>>>> + events[i].name, j);
>>>> + if (!str)
>>>> + goto err;
>>>> +
>>>> + *attr_iter++ = &xe_iter->attr.attr;
>>>> + xe_iter = add_xe_attr(xe_iter, str, config);
>>>> +
>>>> + if (events[i].unit) {
>>>> + str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>>>> + events[i].name, j);
>>>> + if (!str)
>>>> + goto err;
>>>> +
>>>> + *attr_iter++ = &pmu_iter->attr.attr;
>>>> + pmu_iter = add_pmu_attr(pmu_iter, str,
>>>> + events[i].unit);
>>>> + }
>>>> + }
>>>> + }
>>>> +
>>>> + pmu->xe_attr = xe_attr;
>>>> + pmu->pmu_attr = pmu_attr;
>>>> +
>>>> + return attr;
>>>> +
>>>> +err:
>>>> + for (attr_iter = attr; *attr_iter; attr_iter++)
>>>> + kfree((*attr_iter)->name);
>>>> +
>>>> +err_alloc:
>>>> + kfree(attr);
>>>> + kfree(xe_attr);
>>>> + kfree(pmu_attr);
>>>> +
>>>> + return NULL;
>>>> +}
>>>> +
>>>> +static void free_event_attributes(struct xe_pmu *pmu)
>>>> +{
>>>> + struct attribute **attr_iter = pmu->events_attr_group.attrs;
>>>> +
>>>> + for (; *attr_iter; attr_iter++)
>>>> + kfree((*attr_iter)->name);
>>>> +
>>>> + kfree(pmu->events_attr_group.attrs);
>>>> + kfree(pmu->xe_attr);
>>>> + kfree(pmu->pmu_attr);
>>>> +
>>>> + pmu->events_attr_group.attrs = NULL;
>>>> + pmu->xe_attr = NULL;
>>>> + pmu->pmu_attr = NULL;
>>>> +}
>>>> +
>>>> +static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>>>> +{
>>>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>>> +
>>>> + /* Select the first online CPU as a designated reader. */
>>>> + if (cpumask_empty(&xe_pmu_cpumask))
>>>> + cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>>>> +{
>>>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>>> + unsigned int target = xe_pmu_target_cpu;
>>>> +
>>>> + /*
>>>> + * Unregistering an instance generates a CPU offline event which we must
>>>> + * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>>>> + */
>>>> + if (pmu->closed)
>>>> + return 0;
>>>> +
>>>> + if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>>>> + target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>>>> +
>>>> + /* Migrate events if there is a valid target */
>>>> + if (target < nr_cpu_ids) {
>>>> + cpumask_set_cpu(target, &xe_pmu_cpumask);
>>>> + xe_pmu_target_cpu = target;
>>>> + }
>>>> + }
>>>> +
>>>> + if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>>>> + perf_pmu_migrate_context(&pmu->base, cpu, target);
>>>> + pmu->cpuhp.cpu = target;
>>>> + }
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>>>> +
>>>> +int xe_pmu_init(void)
>>>> +{
>>>> + int ret;
>>>> +
>>>> + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>>>> + "perf/x86/intel/xe:online",
>>>> + xe_pmu_cpu_online,
>>>> + xe_pmu_cpu_offline);
>>>> + if (ret < 0)
>>>> + pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>>>> + ret);
>>>> + else
>>>> + cpuhp_slot = ret;
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +void xe_pmu_exit(void)
>>>> +{
>>>> + if (cpuhp_slot != CPUHP_INVALID)
>>>> + cpuhp_remove_multi_state(cpuhp_slot);
>>>> +}
>>>> +
>>>> +static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>>>> +{
>>>> + if (cpuhp_slot == CPUHP_INVALID)
>>>> + return -EINVAL;
>>>> +
>>>> + return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>>>> +}
>>>> +
>>>> +static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>>>> +{
>>>> + cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>>>> +}
>>>> +
>>>> +void xe_pmu_suspend(struct xe_gt *gt)
>>>> +{
>>>> + engine_group_busyness_store(gt);
>>>> +}
>>>> +
>>>> +static void xe_pmu_unregister(void *arg)
>>>> +{
>>>> + struct xe_pmu *pmu = arg;
>>>> +
>>>> + if (!pmu->base.event_init)
>>>> + return;
>>>> +
>>>> + /*
>>>> + * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
>>>> + * ensures all currently executing ones will have exited before we
>>>> + * proceed with unregistration.
>>>> + */
>>>> + pmu->closed = true;
>>>> + synchronize_rcu();
>>>> +
>>>> + xe_pmu_unregister_cpuhp_state(pmu);
>>>> +
>>>> + perf_pmu_unregister(&pmu->base);
>>>> + pmu->base.event_init = NULL;
>>>> + kfree(pmu->base.attr_groups);
>>>> + kfree(pmu->name);
>>>> + free_event_attributes(pmu);
>>>> +}
>>>> +
>>>> +void xe_pmu_register(struct xe_pmu *pmu)
>>>> +{
>>>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>>> + const struct attribute_group *attr_groups[] = {
>>>> + &pmu->events_attr_group,
>>>> + &xe_pmu_cpumask_attr_group,
>>>> + NULL
>>>> + };
>>>> +
>>>> + int ret = -ENOMEM;
>>>> +
>>>> + spin_lock_init(&pmu->lock);
>>>> + pmu->cpuhp.cpu = -1;
>>>> +
>>>> + pmu->name = kasprintf(GFP_KERNEL,
>>>> + "xe_%s",
>>>> + dev_name(xe->drm.dev));
>>>> + if (pmu->name)
>>>> + /* tools/perf reserves colons as special. */
>>>> + strreplace((char *)pmu->name, ':', '_');
>>>> +
>>>> + if (!pmu->name)
>>>> + goto err;
>>>> +
>>>> + pmu->events_attr_group.name = "events";
>>>> + pmu->events_attr_group.attrs = create_event_attributes(pmu);
>>>> + if (!pmu->events_attr_group.attrs)
>>>> + goto err_name;
>>>> +
>>>> + pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>>>> + GFP_KERNEL);
>>>> + if (!pmu->base.attr_groups)
>>>> + goto err_attr;
>>>> +
>>>> + pmu->base.module = THIS_MODULE;
>>>> + pmu->base.task_ctx_nr = perf_invalid_context;
>>>> + pmu->base.event_init = xe_pmu_event_init;
>>>> + pmu->base.add = xe_pmu_event_add;
>>>> + pmu->base.del = xe_pmu_event_del;
>>>> + pmu->base.start = xe_pmu_event_start;
>>>> + pmu->base.stop = xe_pmu_event_stop;
>>>> + pmu->base.read = xe_pmu_event_read;
>>>> + pmu->base.event_idx = xe_pmu_event_event_idx;
>>>> +
>>>> + ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>>>> + if (ret)
>>>> + goto err_groups;
>>>> +
>>>> + ret = xe_pmu_register_cpuhp_state(pmu);
>>>> + if (ret)
>>>> + goto err_unreg;
>>>> +
>>>> + ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>>>> + if (ret)
>>>> + goto err_cpuhp;
>>>> +
>>>> + return;
>>>> +
>>>> +err_cpuhp:
>>>> + xe_pmu_unregister_cpuhp_state(pmu);
>>>> +err_unreg:
>>>> + perf_pmu_unregister(&pmu->base);
>>>> +err_groups:
>>>> + kfree(pmu->base.attr_groups);
>>>> +err_attr:
>>>> + pmu->base.event_init = NULL;
>>>> + free_event_attributes(pmu);
>>>> +err_name:
>>>> + kfree(pmu->name);
>>>> +err:
>>>> + drm_notice(&xe->drm, "Failed to register PMU!\n");
>>>> +}
>>>> diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>>>> new file mode 100644
>>>> index 000000000000..8afa256f9dac
>>>> --- /dev/null
>>>> +++ b/drivers/gpu/drm/xe/xe_pmu.h
>>>> @@ -0,0 +1,26 @@
>>>> +/* SPDX-License-Identifier: MIT */
>>>> +/*
>>>> + * Copyright © 2024 Intel Corporation
>>>> + */
>>>> +
>>>> +#ifndef _XE_PMU_H_
>>>> +#define _XE_PMU_H_
>>>> +
>>>> +#include "xe_pmu_types.h"
>>>> +
>>>> +struct xe_gt;
>>>> +
>>>> +#if IS_ENABLED(CONFIG_PERF_EVENTS)
>>>> +int xe_pmu_init(void);
>>>> +void xe_pmu_exit(void);
>>>> +void xe_pmu_register(struct xe_pmu *pmu);
>>>> +void xe_pmu_suspend(struct xe_gt *gt);
>>>> +#else
>>>> +static inline int xe_pmu_init(void) { return 0; }
>>>> +static inline void xe_pmu_exit(void) {}
>>>> +static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>>>> +static inline void xe_pmu_suspend(struct xe_gt *gt) {}
>>>> +#endif
>>>> +
>>>> +#endif
>>>> +
>>>> diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
>>>> new file mode 100644
>>>> index 000000000000..e86e8d7e0356
>>>> --- /dev/null
>>>> +++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>>>> @@ -0,0 +1,67 @@
>>>> +/* SPDX-License-Identifier: MIT */
>>>> +/*
>>>> + * Copyright © 2024 Intel Corporation
>>>> + */
>>>> +
>>>> +#ifndef _XE_PMU_TYPES_H_
>>>> +#define _XE_PMU_TYPES_H_
>>>> +
>>>> +#include <linux/perf_event.h>
>>>> +#include <linux/spinlock_types.h>
>>>> +#include <uapi/drm/xe_drm.h>
>>>> +
>>>> +enum {
>>>> + __XE_SAMPLE_RENDER_GROUP_BUSY,
>>>> + __XE_SAMPLE_COPY_GROUP_BUSY,
>>>> + __XE_SAMPLE_MEDIA_GROUP_BUSY,
>>>> + __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
>>>> + __XE_NUM_PMU_SAMPLERS
>>>> +};
>>>> +
>>>> +#define XE_PMU_MAX_GT 2
>>>> +
>>>> +struct xe_pmu {
>>>> + /**
>>>> + * @cpuhp: Struct used for CPU hotplug handling.
>>>> + */
>>>> + struct {
>>>> + struct hlist_node node;
>>>> + unsigned int cpu;
>>>> + } cpuhp;
>>>> + /**
>>>> + * @base: PMU base.
>>>> + */
>>>> + struct pmu base;
>>>> + /**
>>>> + * @closed: xe is unregistering.
>>>> + */
>>>> + bool closed;
>>>> + /**
>>>> + * @name: Name as registered with perf core.
>>>> + */
>>>> + const char *name;
>>>> + /**
>>>> + * @lock: Lock protecting enable mask and ref count handling.
>>>> + */
>>>> + spinlock_t lock;
>>>> + /**
>>>> + * @sample: Current and previous (raw) counters.
>>>> + *
>>>> + * These counters are updated when the device is awake.
>>>> + */
>>>> + u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>>>> + /**
>>>> + * @events_attr_group: Device events attribute group.
>>>> + */
>>>> + struct attribute_group events_attr_group;
>>>> + /**
>>>> + * @xe_attr: Memory block holding device attributes.
>>>> + */
>>>> + void *xe_attr;
>>>> + /**
>>>> + * @pmu_attr: Memory block holding device attributes.
>>>> + */
>>>> + void *pmu_attr;
>>>> +};
>>>> +
>>>> +#endif
>>>> diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>>>> index d7b0903c22b2..07ca545354f7 100644
>>>> --- a/include/uapi/drm/xe_drm.h
>>>> +++ b/include/uapi/drm/xe_drm.h
>>>> @@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
>>>> __u64 reserved[2];
>>>> };
>>>>
>>>> +/**
>>>> + * DOC: XE PMU event config IDs
>>>> + *
>>>> + * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in xe_drm.h
>>>> + * in 'struct perf_event_attr' as part of perf_event_open syscall to read a
>>>> + * particular event.
>>>> + *
>>>> + * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
>>>> + *
>>>> + * .. code-block:: C
>>>> + *
>>>> + * struct perf_event_attr attr;
>>>> + * long long count;
>>>> + * int cpu = 0;
>>>> + * int fd;
>>>> + *
>>>> + * memset(&attr, 0, sizeof(struct perf_event_attr));
>>>> + * attr.type = type; // eg: /sys/bus/event_source/devices/xe_0000_56_00.0/type
>>>> + * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
>>>> + * attr.use_clockid = 1;
>>>> + * attr.clockid = CLOCK_MONOTONIC;
>>>> + * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
>>>> + *
>>>> + * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
>>>> + */
>>>> +
>>>> +/*
>>>> + * Top bits of every counter are GT id.
>>>> + */
>>>> +#define __XE_PMU_GT_SHIFT (56)
>>>> +
>>>> +#define ___XE_PMU_OTHER(gt, x) \
>>>> + (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>>>> +
>>>> +#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
>>>> +#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
>>>> +#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
>>>> +#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
>>>
>>> + Lucas for inputs
>>>
>>> We should align this to the interface planned for other PMU busyness counters as well as how we do PCEU. i.e.
>>>
>>> 1) counters are in ticks
>>> 2) total time in ticks is also exported to the user.
>>>
>>> For 1), I would just append TICKS to the counter names and drop the conversion to _ns in __engine_group_busyness_read(). Also, drop the patch that adds this conversion helper.
>>>
>>> For 2) define a new counter - total active ticks that would return the 'CPU' timestamp converted to gpu ticks. The reason I am insisting on CPU timestamp here is because we want to have a time base that is ticking even when the GPU is idle.
>>
>> why can't we expose what the HW presents[1] to us via register and leave the interpretation to userspace.
>
> HW is indeed exposing ticks. In this case, I am suggesting to expose that directly to the user, so I think you are saying the same.
correct
>
> As for interpretation, we need to make sure it works consistently in SRIOV. The L0 API for group busyness itself imposes the requirement for another counter to make sense of [1]. This additional counter has always existed, but in prior implementations, it was just using the CPU time in the equation. The CPU sample time is always returned in all the PMU counters. With SRIOV, it will still be CPU time, but only the time that a VF executed for and that information is only available to the KMD/GuC. Without that information, interpreting the ticks in [1] will not be meaningful.
I hope in SRIOV case you are mentioning about accessing these counters from PF right? if it is from VF I'm not sure if these registers are accessible.
Regards,
Aravind.
>
> Regards,
> Umesh
>
>
>>
>> Thanks,
>> Aravind.
>>>
>>> Regards,
>>> Umesh
>>>
>>>> +
>>>> #if defined(__cplusplus)
>>>> }
>>>> #endif
>>>> --
>>>> 2.40.0
>>>>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-20 19:52 ` Umesh Nerlige Ramappa
2024-06-27 6:49 ` Aravind Iddamsetty
@ 2024-06-28 15:55 ` Lucas De Marchi
2024-06-28 16:52 ` Umesh Nerlige Ramappa
1 sibling, 1 reply; 32+ messages in thread
From: Lucas De Marchi @ 2024-06-28 15:55 UTC (permalink / raw)
To: Umesh Nerlige Ramappa
Cc: Riana Tauro, intel-xe, anshuman.gupta, ashutosh.dixit,
aravind.iddamsetty, rodrigo.vivi, krishnaiah.bommu
On Thu, Jun 20, 2024 at 12:52:05PM GMT, Umesh Nerlige Ramappa wrote:
>On Thu, Jun 13, 2024 at 03:34:11PM +0530, Riana Tauro wrote:
>>From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>
>>There are a set of engine group busyness counters provided by HW which are
>>perfect fit to be exposed via PMU perf events.
>>
>>BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>
>>events can be listed using:
>>perf list
>> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>
>>and can be read using:
>>
>>perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
>> time counts unit events
>> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>
>>The pmu base implementation is taken from i915.
>>
>>v2:
>>Store last known value when device is awake return that while the GT is
>>suspended and then update the driver copy when read during awake.
>>
>>v3:
>>1. drop init_samples, as storing counters before going to suspend should
>>be sufficient.
>>2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>>dropped helpers to store and read samples.
>>3. use xe_device_mem_access_get_if_ongoing to check if device is active
>>before reading the OA registers.
>>4. dropped format attr as no longer needed
>>5. introduce xe_pmu_suspend to call engine_group_busyness_store
>>6. few other nits.
>>
>>v4: minor nits.
>>
>>v5: take forcewake when accessing the OAG registers
>>
>>v6:
>>1. drop engine_busyness_sample_type
>>2. update UAPI documentation
>>
>>v7:
>>1. update UAPI documentation
>>2. drop MEDIA_GT specific change for media busyness counter.
>>
>>v8:
>>1. rebase
>>2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>>3. remove interrupts pmu event
>>
>>v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>>
>>Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
>>Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>>---
>>drivers/gpu/drm/xe/Makefile | 2 +
>>drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
>>drivers/gpu/drm/xe/xe_device.c | 2 +
>>drivers/gpu/drm/xe/xe_device_types.h | 4 +
>>drivers/gpu/drm/xe/xe_gt.c | 2 +
>>drivers/gpu/drm/xe/xe_module.c | 5 +
>>drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
>>drivers/gpu/drm/xe/xe_pmu.h | 26 ++
>>drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
>>include/uapi/drm/xe_drm.h | 39 ++
>>10 files changed, 783 insertions(+)
>>create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
>>create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
>>create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>>
>>diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>>index cbf961b90237..83bf1e07669b 100644
>>--- a/drivers/gpu/drm/xe/Makefile
>>+++ b/drivers/gpu/drm/xe/Makefile
>>@@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
>> i915-display/skl_universal_plane.o \
>> i915-display/skl_watermark.o
>>
>>+xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>>+
>>ifeq ($(CONFIG_ACPI),y)
>> xe-$(CONFIG_DRM_XE_DISPLAY) += \
>> i915-display/intel_acpi.o \
>>diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>index 47c26c37608d..22821dcd4e1b 100644
>>--- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>+++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>@@ -390,6 +390,11 @@
>>#define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
>>#define GLOBAL_INVALIDATION_MODE REG_BIT(2)
>>
>>+#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
>>+#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
>>+#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
>>+#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
>>+
>>#define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
>>#define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
>>
>>diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
>>index 64691a56d59c..bb00c8c9ec9b 100644
>>--- a/drivers/gpu/drm/xe/xe_device.c
>>+++ b/drivers/gpu/drm/xe/xe_device.c
>>@@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
>>
>> xe_hwmon_register(xe);
>>
>>+ xe_pmu_register(&xe->pmu);
>>+
>> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>>
>>err_fini_display:
>>diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
>>index 52bc461171d5..a5dba7325cf1 100644
>>--- a/drivers/gpu/drm/xe/xe_device_types.h
>>+++ b/drivers/gpu/drm/xe/xe_device_types.h
>>@@ -18,6 +18,7 @@
>>#include "xe_lmtt_types.h"
>>#include "xe_memirq_types.h"
>>#include "xe_platform_types.h"
>>+#include "xe_pmu.h"
>>#include "xe_pt_types.h"
>>#include "xe_sriov_types.h"
>>#include "xe_step_types.h"
>>@@ -473,6 +474,9 @@ struct xe_device {
>> int mode;
>> } wedged;
>>
>>+ /** @pmu: performance monitoring unit */
>>+ struct xe_pmu pmu;
>>+
>> /* private: */
>>
>>#if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
>>diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
>>index 57d84751e160..477d0ae5f230 100644
>>--- a/drivers/gpu/drm/xe/xe_gt.c
>>+++ b/drivers/gpu/drm/xe/xe_gt.c
>>@@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
>> if (err)
>> goto err_msg;
>>
>>+ xe_pmu_suspend(gt);
>>+
>> err = xe_uc_suspend(>->uc);
>> if (err)
>> goto err_force_wake;
>>diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
>>index 3edeb30d5ccb..26f814f97fc2 100644
>>--- a/drivers/gpu/drm/xe/xe_module.c
>>+++ b/drivers/gpu/drm/xe/xe_module.c
>>@@ -11,6 +11,7 @@
>>#include "xe_drv.h"
>>#include "xe_hw_fence.h"
>>#include "xe_pci.h"
>>+#include "xe_pmu.h"
>>#include "xe_sched_job.h"
>>
>>struct xe_modparam xe_modparam = {
>>@@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
>> .init = xe_sched_job_module_init,
>> .exit = xe_sched_job_module_exit,
>> },
>>+ {
>>+ .init = xe_pmu_init,
>>+ .exit = xe_pmu_exit,
>>+ },
>> {
>> .init = xe_register_pci_driver,
>> .exit = xe_unregister_pci_driver,
>>diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>>new file mode 100644
>>index 000000000000..64960a358af2
>>--- /dev/null
>>+++ b/drivers/gpu/drm/xe/xe_pmu.c
>>@@ -0,0 +1,631 @@
>>+// SPDX-License-Identifier: MIT
>>+/*
>>+ * Copyright © 2024 Intel Corporation
>>+ */
>>+
>>+#include <drm/drm_drv.h>
>>+#include <drm/drm_managed.h>
>>+#include <drm/xe_drm.h>
>>+
>>+#include "regs/xe_gt_regs.h"
>>+#include "xe_device.h"
>>+#include "xe_force_wake.h"
>>+#include "xe_gt_clock.h"
>>+#include "xe_mmio.h"
>>+#include "xe_macros.h"
>>+#include "xe_pm.h"
>>+
>>+static cpumask_t xe_pmu_cpumask;
>>+static unsigned int xe_pmu_target_cpu = -1;
>>+
>>+static unsigned int config_gt_id(const u64 config)
>>+{
>>+ return config >> __XE_PMU_GT_SHIFT;
>>+}
>>+
>>+static u64 config_counter(const u64 config)
>>+{
>>+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>>+}
>>+
>>+static void xe_pmu_event_destroy(struct perf_event *event)
>>+{
>>+ struct xe_device *xe =
>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>+
>>+ drm_WARN_ON(&xe->drm, event->parent);
>>+
>>+ drm_dev_put(&xe->drm);
>>+}
>>+
>>+static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
>>+{
>>+ u64 val;
>>+
>>+ switch (sample_type) {
>>+ case __XE_SAMPLE_RENDER_GROUP_BUSY:
>>+ val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>>+ break;
>>+ case __XE_SAMPLE_COPY_GROUP_BUSY:
>>+ val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>>+ break;
>>+ case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>>+ val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>>+ break;
>>+ case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>>+ val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>>+ break;
>>+ default:
>>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>+ }
>>+
>>+ return xe_gt_clock_cycles_to_ns(gt, val * 16);
>>+}
>>+
>>+static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>>+{
>>+ int sample_type = config_counter(config);
>>+ const unsigned int gt_id = gt->info.id;
>>+ struct xe_device *xe = gt->tile->xe;
>>+ struct xe_pmu *pmu = &xe->pmu;
>>+ unsigned long flags;
>>+ bool device_awake;
>>+ u64 val;
>>+
>>+ device_awake = xe_pm_runtime_get_if_active(xe);
>>+ if (device_awake) {
>>+ XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>>+ val = __engine_group_busyness_read(gt, sample_type);
>>+ XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>>+ xe_pm_runtime_put(xe);
>>+ }
>>+
>>+ spin_lock_irqsave(&pmu->lock, flags);
>>+
>>+ if (device_awake)
>>+ pmu->sample[gt_id][sample_type] = val;
>>+ else
>>+ val = pmu->sample[gt_id][sample_type];
>>+
>>+ spin_unlock_irqrestore(&pmu->lock, flags);
>>+
>>+ return val;
>>+}
>>+
>>+static void engine_group_busyness_store(struct xe_gt *gt)
>>+{
>>+ struct xe_pmu *pmu = >->tile->xe->pmu;
>>+ unsigned int gt_id = gt->info.id;
>>+ unsigned long flags;
>>+ int i;
>>+
>>+ spin_lock_irqsave(&pmu->lock, flags);
>>+
>>+ for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>>+ pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>>+
>>+ spin_unlock_irqrestore(&pmu->lock, flags);
>>+}
>>+
>>+static int
>>+config_status(struct xe_device *xe, u64 config)
>>+{
>>+ unsigned int gt_id = config_gt_id(config);
>>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>+
>>+ if (gt_id >= XE_PMU_MAX_GT)
>>+ return -ENOENT;
>>+
>>+ switch (config_counter(config)) {
>>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>>+ case XE_PMU_COPY_GROUP_BUSY(0):
>>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>+ if (gt->info.type == XE_GT_TYPE_MEDIA)
>>+ return -ENOENT;
>>+ break;
>>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>>+ if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
>>+ return -ENOENT;
>>+ break;
>>+ default:
>>+ return -ENOENT;
>>+ }
>>+
>>+ return 0;
>>+}
>>+
>>+static int xe_pmu_event_init(struct perf_event *event)
>>+{
>>+ struct xe_device *xe =
>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>+ struct xe_pmu *pmu = &xe->pmu;
>>+ int ret;
>>+
>>+ if (pmu->closed)
>>+ return -ENODEV;
>>+
>>+ if (event->attr.type != event->pmu->type)
>>+ return -ENOENT;
>>+
>>+ /* unsupported modes and filters */
>>+ if (event->attr.sample_period) /* no sampling */
>>+ return -EINVAL;
>>+
>>+ if (has_branch_stack(event))
>>+ return -EOPNOTSUPP;
>>+
>>+ if (event->cpu < 0)
>>+ return -EINVAL;
>>+
>>+ /* only allow running on one cpu at a time */
>>+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>>+ return -EINVAL;
>>+
>>+ ret = config_status(xe, event->attr.config);
>>+ if (ret)
>>+ return ret;
>>+
>>+ if (!event->parent) {
>>+ drm_dev_get(&xe->drm);
>>+ event->destroy = xe_pmu_event_destroy;
>>+ }
>>+
>>+ return 0;
>>+}
>>+
>>+static u64 __xe_pmu_event_read(struct perf_event *event)
>>+{
>>+ struct xe_device *xe =
>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>+ const unsigned int gt_id = config_gt_id(event->attr.config);
>>+ const u64 config = event->attr.config;
>>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>+ u64 val;
>>+
>>+ switch (config_counter(config)) {
>>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>>+ case XE_PMU_COPY_GROUP_BUSY(0):
>>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>>+ val = engine_group_busyness_read(gt, config);
>>+ break;
>>+ default:
>>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>+ }
>>+
>>+ return val;
>>+}
>>+
>>+static void xe_pmu_event_read(struct perf_event *event)
>>+{
>>+ struct xe_device *xe =
>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>+ struct hw_perf_event *hwc = &event->hw;
>>+ struct xe_pmu *pmu = &xe->pmu;
>>+ u64 prev, new;
>>+
>>+ if (pmu->closed) {
>>+ event->hw.state = PERF_HES_STOPPED;
>>+ return;
>>+ }
>>+again:
>>+ prev = local64_read(&hwc->prev_count);
>>+ new = __xe_pmu_event_read(event);
>>+
>>+ if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>>+ goto again;
>>+
>>+ local64_add(new - prev, &event->count);
>>+}
>>+
>>+static void xe_pmu_enable(struct perf_event *event)
>>+{
>>+ /*
>>+ * Store the current counter value so we can report the correct delta
>>+ * for all listeners. Even when the event was already enabled and has
>>+ * an existing non-zero value.
>>+ */
>>+ local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>>+}
>>+
>>+static void xe_pmu_event_start(struct perf_event *event, int flags)
>>+{
>>+ struct xe_device *xe =
>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>+ struct xe_pmu *pmu = &xe->pmu;
>>+
>>+ if (pmu->closed)
>>+ return;
>>+
>>+ xe_pmu_enable(event);
>>+ event->hw.state = 0;
>>+}
>>+
>>+static void xe_pmu_event_stop(struct perf_event *event, int flags)
>>+{
>>+ if (flags & PERF_EF_UPDATE)
>>+ xe_pmu_event_read(event);
>>+
>>+ event->hw.state = PERF_HES_STOPPED;
>>+}
>>+
>>+static int xe_pmu_event_add(struct perf_event *event, int flags)
>>+{
>>+ struct xe_device *xe =
>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>+ struct xe_pmu *pmu = &xe->pmu;
>>+
>>+ if (pmu->closed)
>>+ return -ENODEV;
>>+
>>+ if (flags & PERF_EF_START)
>>+ xe_pmu_event_start(event, flags);
>>+
>>+ return 0;
>>+}
>>+
>>+static void xe_pmu_event_del(struct perf_event *event, int flags)
>>+{
>>+ xe_pmu_event_stop(event, PERF_EF_UPDATE);
>>+}
>>+
>>+static int xe_pmu_event_event_idx(struct perf_event *event)
>>+{
>>+ return 0;
>>+}
>>+
>>+struct xe_ext_attribute {
>>+ struct device_attribute attr;
>>+ unsigned long val;
>>+};
>>+
>>+static ssize_t xe_pmu_event_show(struct device *dev,
>>+ struct device_attribute *attr, char *buf)
>>+{
>>+ struct xe_ext_attribute *eattr;
>>+
>>+ eattr = container_of(attr, struct xe_ext_attribute, attr);
>>+ return sprintf(buf, "config=0x%lx\n", eattr->val);
>>+}
>>+
>>+static ssize_t cpumask_show(struct device *dev,
>>+ struct device_attribute *attr, char *buf)
>>+{
>>+ return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>>+}
>>+
>>+static DEVICE_ATTR_RO(cpumask);
>>+
>>+static struct attribute *xe_cpumask_attrs[] = {
>>+ &dev_attr_cpumask.attr,
>>+ NULL,
>>+};
>>+
>>+static const struct attribute_group xe_pmu_cpumask_attr_group = {
>>+ .attrs = xe_cpumask_attrs,
>>+};
>>+
>>+#define __event(__counter, __name, __unit) \
>>+{ \
>>+ .counter = (__counter), \
>>+ .name = (__name), \
>>+ .unit = (__unit), \
>>+}
>>+
>>+static struct xe_ext_attribute *
>>+add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>>+{
>>+ sysfs_attr_init(&attr->attr.attr);
>>+ attr->attr.attr.name = name;
>>+ attr->attr.attr.mode = 0444;
>>+ attr->attr.show = xe_pmu_event_show;
>>+ attr->val = config;
>>+
>>+ return ++attr;
>>+}
>>+
>>+static struct perf_pmu_events_attr *
>>+add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>>+ const char *str)
>>+{
>>+ sysfs_attr_init(&attr->attr.attr);
>>+ attr->attr.attr.name = name;
>>+ attr->attr.attr.mode = 0444;
>>+ attr->attr.show = perf_event_sysfs_show;
>>+ attr->event_str = str;
>>+
>>+ return ++attr;
>>+}
>>+
>>+static struct attribute **
>>+create_event_attributes(struct xe_pmu *pmu)
>>+{
>>+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>+ static const struct {
>>+ unsigned int counter;
>>+ const char *name;
>>+ const char *unit;
>>+ } events[] = {
>>+ __event(0, "render-group-busy", "ns"),
>>+ __event(1, "copy-group-busy", "ns"),
>>+ __event(2, "media-group-busy", "ns"),
>>+ __event(3, "any-engine-group-busy", "ns"),
>>+ };
>>+
>>+ struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>>+ struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>>+ struct attribute **attr = NULL, **attr_iter;
>>+ unsigned int count = 0;
>>+ unsigned int i, j;
>>+ struct xe_gt *gt;
>>+
>>+ /* Count how many counters we will be exposing. */
>>+ for_each_gt(gt, xe, j) {
>>+ for (i = 0; i < ARRAY_SIZE(events); i++) {
>>+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>+
>>+ if (!config_status(xe, config))
>>+ count++;
>>+ }
>>+ }
>>+
>>+ /* Allocate attribute objects and table. */
>>+ xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>>+ if (!xe_attr)
>>+ goto err_alloc;
>>+
>>+ pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>>+ if (!pmu_attr)
>>+ goto err_alloc;
>>+
>>+ /* Max one pointer of each attribute type plus a termination entry. */
>>+ attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>>+ if (!attr)
>>+ goto err_alloc;
>>+
>>+ xe_iter = xe_attr;
>>+ pmu_iter = pmu_attr;
>>+ attr_iter = attr;
>>+
>>+ for_each_gt(gt, xe, j) {
>>+ for (i = 0; i < ARRAY_SIZE(events); i++) {
>>+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>+ char *str;
>>+
>>+ if (config_status(xe, config))
>>+ continue;
>>+
>>+ str = kasprintf(GFP_KERNEL, "%s-gt%u",
>>+ events[i].name, j);
>>+ if (!str)
>>+ goto err;
>>+
>>+ *attr_iter++ = &xe_iter->attr.attr;
>>+ xe_iter = add_xe_attr(xe_iter, str, config);
>>+
>>+ if (events[i].unit) {
>>+ str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>>+ events[i].name, j);
>>+ if (!str)
>>+ goto err;
>>+
>>+ *attr_iter++ = &pmu_iter->attr.attr;
>>+ pmu_iter = add_pmu_attr(pmu_iter, str,
>>+ events[i].unit);
>>+ }
>>+ }
>>+ }
>>+
>>+ pmu->xe_attr = xe_attr;
>>+ pmu->pmu_attr = pmu_attr;
>>+
>>+ return attr;
>>+
>>+err:
>>+ for (attr_iter = attr; *attr_iter; attr_iter++)
>>+ kfree((*attr_iter)->name);
>>+
>>+err_alloc:
>>+ kfree(attr);
>>+ kfree(xe_attr);
>>+ kfree(pmu_attr);
>>+
>>+ return NULL;
>>+}
>>+
>>+static void free_event_attributes(struct xe_pmu *pmu)
>>+{
>>+ struct attribute **attr_iter = pmu->events_attr_group.attrs;
>>+
>>+ for (; *attr_iter; attr_iter++)
>>+ kfree((*attr_iter)->name);
>>+
>>+ kfree(pmu->events_attr_group.attrs);
>>+ kfree(pmu->xe_attr);
>>+ kfree(pmu->pmu_attr);
>>+
>>+ pmu->events_attr_group.attrs = NULL;
>>+ pmu->xe_attr = NULL;
>>+ pmu->pmu_attr = NULL;
>>+}
>>+
>>+static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>>+{
>>+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>+
>>+ /* Select the first online CPU as a designated reader. */
>>+ if (cpumask_empty(&xe_pmu_cpumask))
>>+ cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>>+
>>+ return 0;
>>+}
>>+
>>+static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>>+{
>>+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>+ unsigned int target = xe_pmu_target_cpu;
>>+
>>+ /*
>>+ * Unregistering an instance generates a CPU offline event which we must
>>+ * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>>+ */
>>+ if (pmu->closed)
>>+ return 0;
>>+
>>+ if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>>+ target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>>+
>>+ /* Migrate events if there is a valid target */
>>+ if (target < nr_cpu_ids) {
>>+ cpumask_set_cpu(target, &xe_pmu_cpumask);
>>+ xe_pmu_target_cpu = target;
>>+ }
>>+ }
>>+
>>+ if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>>+ perf_pmu_migrate_context(&pmu->base, cpu, target);
>>+ pmu->cpuhp.cpu = target;
>>+ }
>>+
>>+ return 0;
>>+}
>>+
>>+static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>>+
>>+int xe_pmu_init(void)
>>+{
>>+ int ret;
>>+
>>+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>>+ "perf/x86/intel/xe:online",
>>+ xe_pmu_cpu_online,
>>+ xe_pmu_cpu_offline);
>>+ if (ret < 0)
>>+ pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>>+ ret);
>>+ else
>>+ cpuhp_slot = ret;
>>+
>>+ return 0;
>>+}
>>+
>>+void xe_pmu_exit(void)
>>+{
>>+ if (cpuhp_slot != CPUHP_INVALID)
>>+ cpuhp_remove_multi_state(cpuhp_slot);
>>+}
>>+
>>+static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>>+{
>>+ if (cpuhp_slot == CPUHP_INVALID)
>>+ return -EINVAL;
>>+
>>+ return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>>+}
>>+
>>+static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>>+{
>>+ cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>>+}
>>+
>>+void xe_pmu_suspend(struct xe_gt *gt)
>>+{
>>+ engine_group_busyness_store(gt);
>>+}
>>+
>>+static void xe_pmu_unregister(void *arg)
>>+{
>>+ struct xe_pmu *pmu = arg;
>>+
>>+ if (!pmu->base.event_init)
>>+ return;
>>+
>>+ /*
>>+ * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
>>+ * ensures all currently executing ones will have exited before we
>>+ * proceed with unregistration.
>>+ */
>>+ pmu->closed = true;
>>+ synchronize_rcu();
>>+
>>+ xe_pmu_unregister_cpuhp_state(pmu);
>>+
>>+ perf_pmu_unregister(&pmu->base);
>>+ pmu->base.event_init = NULL;
>>+ kfree(pmu->base.attr_groups);
>>+ kfree(pmu->name);
>>+ free_event_attributes(pmu);
>>+}
>>+
>>+void xe_pmu_register(struct xe_pmu *pmu)
>>+{
>>+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>+ const struct attribute_group *attr_groups[] = {
>>+ &pmu->events_attr_group,
>>+ &xe_pmu_cpumask_attr_group,
>>+ NULL
>>+ };
>>+
>>+ int ret = -ENOMEM;
>>+
>>+ spin_lock_init(&pmu->lock);
>>+ pmu->cpuhp.cpu = -1;
>>+
>>+ pmu->name = kasprintf(GFP_KERNEL,
>>+ "xe_%s",
>>+ dev_name(xe->drm.dev));
>>+ if (pmu->name)
>>+ /* tools/perf reserves colons as special. */
>>+ strreplace((char *)pmu->name, ':', '_');
>>+
>>+ if (!pmu->name)
>>+ goto err;
>>+
>>+ pmu->events_attr_group.name = "events";
>>+ pmu->events_attr_group.attrs = create_event_attributes(pmu);
>>+ if (!pmu->events_attr_group.attrs)
>>+ goto err_name;
>>+
>>+ pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>>+ GFP_KERNEL);
>>+ if (!pmu->base.attr_groups)
>>+ goto err_attr;
>>+
>>+ pmu->base.module = THIS_MODULE;
>>+ pmu->base.task_ctx_nr = perf_invalid_context;
>>+ pmu->base.event_init = xe_pmu_event_init;
>>+ pmu->base.add = xe_pmu_event_add;
>>+ pmu->base.del = xe_pmu_event_del;
>>+ pmu->base.start = xe_pmu_event_start;
>>+ pmu->base.stop = xe_pmu_event_stop;
>>+ pmu->base.read = xe_pmu_event_read;
>>+ pmu->base.event_idx = xe_pmu_event_event_idx;
>>+
>>+ ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>>+ if (ret)
>>+ goto err_groups;
>>+
>>+ ret = xe_pmu_register_cpuhp_state(pmu);
>>+ if (ret)
>>+ goto err_unreg;
>>+
>>+ ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>>+ if (ret)
>>+ goto err_cpuhp;
>>+
>>+ return;
>>+
>>+err_cpuhp:
>>+ xe_pmu_unregister_cpuhp_state(pmu);
>>+err_unreg:
>>+ perf_pmu_unregister(&pmu->base);
>>+err_groups:
>>+ kfree(pmu->base.attr_groups);
>>+err_attr:
>>+ pmu->base.event_init = NULL;
>>+ free_event_attributes(pmu);
>>+err_name:
>>+ kfree(pmu->name);
>>+err:
>>+ drm_notice(&xe->drm, "Failed to register PMU!\n");
>>+}
>>diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>>new file mode 100644
>>index 000000000000..8afa256f9dac
>>--- /dev/null
>>+++ b/drivers/gpu/drm/xe/xe_pmu.h
>>@@ -0,0 +1,26 @@
>>+/* SPDX-License-Identifier: MIT */
>>+/*
>>+ * Copyright © 2024 Intel Corporation
>>+ */
>>+
>>+#ifndef _XE_PMU_H_
>>+#define _XE_PMU_H_
>>+
>>+#include "xe_pmu_types.h"
>>+
>>+struct xe_gt;
>>+
>>+#if IS_ENABLED(CONFIG_PERF_EVENTS)
>>+int xe_pmu_init(void);
>>+void xe_pmu_exit(void);
>>+void xe_pmu_register(struct xe_pmu *pmu);
>>+void xe_pmu_suspend(struct xe_gt *gt);
>>+#else
>>+static inline int xe_pmu_init(void) { return 0; }
>>+static inline void xe_pmu_exit(void) {}
>>+static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>>+static inline void xe_pmu_suspend(struct xe_gt *gt) {}
>>+#endif
>>+
>>+#endif
>>+
>>diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
>>new file mode 100644
>>index 000000000000..e86e8d7e0356
>>--- /dev/null
>>+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>>@@ -0,0 +1,67 @@
>>+/* SPDX-License-Identifier: MIT */
>>+/*
>>+ * Copyright © 2024 Intel Corporation
>>+ */
>>+
>>+#ifndef _XE_PMU_TYPES_H_
>>+#define _XE_PMU_TYPES_H_
>>+
>>+#include <linux/perf_event.h>
>>+#include <linux/spinlock_types.h>
>>+#include <uapi/drm/xe_drm.h>
>>+
>>+enum {
>>+ __XE_SAMPLE_RENDER_GROUP_BUSY,
>>+ __XE_SAMPLE_COPY_GROUP_BUSY,
>>+ __XE_SAMPLE_MEDIA_GROUP_BUSY,
>>+ __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
>>+ __XE_NUM_PMU_SAMPLERS
>>+};
>>+
>>+#define XE_PMU_MAX_GT 2
>>+
>>+struct xe_pmu {
>>+ /**
>>+ * @cpuhp: Struct used for CPU hotplug handling.
>>+ */
>>+ struct {
>>+ struct hlist_node node;
>>+ unsigned int cpu;
>>+ } cpuhp;
>>+ /**
>>+ * @base: PMU base.
>>+ */
>>+ struct pmu base;
>>+ /**
>>+ * @closed: xe is unregistering.
>>+ */
>>+ bool closed;
>>+ /**
>>+ * @name: Name as registered with perf core.
>>+ */
>>+ const char *name;
>>+ /**
>>+ * @lock: Lock protecting enable mask and ref count handling.
>>+ */
>>+ spinlock_t lock;
>>+ /**
>>+ * @sample: Current and previous (raw) counters.
>>+ *
>>+ * These counters are updated when the device is awake.
>>+ */
>>+ u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>>+ /**
>>+ * @events_attr_group: Device events attribute group.
>>+ */
>>+ struct attribute_group events_attr_group;
>>+ /**
>>+ * @xe_attr: Memory block holding device attributes.
>>+ */
>>+ void *xe_attr;
>>+ /**
>>+ * @pmu_attr: Memory block holding device attributes.
>>+ */
>>+ void *pmu_attr;
>>+};
>>+
>>+#endif
>>diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>>index d7b0903c22b2..07ca545354f7 100644
>>--- a/include/uapi/drm/xe_drm.h
>>+++ b/include/uapi/drm/xe_drm.h
>>@@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
>> __u64 reserved[2];
>>};
>>
>>+/**
>>+ * DOC: XE PMU event config IDs
>>+ *
>>+ * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in xe_drm.h
>>+ * in 'struct perf_event_attr' as part of perf_event_open syscall to read a
>>+ * particular event.
>>+ *
>>+ * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
>>+ *
>>+ * .. code-block:: C
>>+ *
>>+ * struct perf_event_attr attr;
>>+ * long long count;
>>+ * int cpu = 0;
>>+ * int fd;
>>+ *
>>+ * memset(&attr, 0, sizeof(struct perf_event_attr));
>>+ * attr.type = type; // eg: /sys/bus/event_source/devices/xe_0000_56_00.0/type
>>+ * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
>>+ * attr.use_clockid = 1;
>>+ * attr.clockid = CLOCK_MONOTONIC;
>>+ * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
>>+ *
>>+ * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
>>+ */
>>+
>>+/*
>>+ * Top bits of every counter are GT id.
>>+ */
>>+#define __XE_PMU_GT_SHIFT (56)
>>+
>>+#define ___XE_PMU_OTHER(gt, x) \
>>+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>>+
>>+#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
>>+#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
>>+#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
>>+#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
>
>+ Lucas for inputs
>
>We should align this to the interface planned for other PMU busyness
>counters as well as how we do PCEU. i.e.
>
>1) counters are in ticks
>2) total time in ticks is also exported to the user.
>
>For 1), I would just append TICKS to the counter names and drop the
this uses perf and as such I believe we should use the terms used by
perf.
$ sudo perf stat sleep 1
Performance counter stats for 'sleep 1':
0.91 msec task-clock # 0.001 CPUs utilized
1 context-switches # 1.096 K/sec
0 cpu-migrations # 0.000 /sec
72 page-faults # 78.924 K/sec
------> 2,033,156 cycles # 2.229 GHz
1,560,992 instructions # 0.77 insn per cycle
290,814 branches # 318.779 M/sec
10,449 branch-misses # 3.59% of all branches
1.001580466 seconds time elapsed
0.000000000 seconds user
0.001545000 seconds sys
so... s/ticks/cycles/
I think I said that before, but what's up with all these "group" in the
names? It's confusing since apparently group and engine class are mixed.
We are also missing proper kernel-doc in xe_pmu.c
Lucas De Marchi
>conversion to _ns in __engine_group_busyness_read(). Also, drop the
>patch that adds this conversion helper.
>
>For 2) define a new counter - total active ticks that would return the
>'CPU' timestamp converted to gpu ticks. The reason I am insisting on
>CPU timestamp here is because we want to have a time base that is
>ticking even when the GPU is idle.
>
>Regards,
>Umesh
>
>>+
>>#if defined(__cplusplus)
>>}
>>#endif
>>--
>>2.40.0
>>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-28 9:41 ` Aravind Iddamsetty
@ 2024-06-28 16:36 ` Umesh Nerlige Ramappa
0 siblings, 0 replies; 32+ messages in thread
From: Umesh Nerlige Ramappa @ 2024-06-28 16:36 UTC (permalink / raw)
To: Aravind Iddamsetty
Cc: Riana Tauro, intel-xe, anshuman.gupta, ashutosh.dixit,
rodrigo.vivi, krishnaiah.bommu, lucas.demarchi, Joonas Lahtinen
On Fri, Jun 28, 2024 at 03:11:02PM +0530, Aravind Iddamsetty wrote:
>
>On 27/06/24 21:35, Umesh Nerlige Ramappa wrote:
>> On Thu, Jun 27, 2024 at 12:19:44PM +0530, Aravind Iddamsetty wrote:
>>>
>>> On 21/06/24 01:22, Umesh Nerlige Ramappa wrote:
>>>> On Thu, Jun 13, 2024 at 03:34:11PM +0530, Riana Tauro wrote:
>>>>> From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>>>>
>>>>> There are a set of engine group busyness counters provided by HW which are
>>>>> perfect fit to be exposed via PMU perf events.
>>>>>
>>>>> BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>>>>
>>>>> events can be listed using:
>>>>> perf list
>>>>> xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>>>>> xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>>>>> xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>>>>> xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>>>>
>>>>> and can be read using:
>>>>>
>>>>> perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
>>>>> time counts unit events
>>>>> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>>
>>>>> The pmu base implementation is taken from i915.
>>>>>
>>>>> v2:
>>>>> Store last known value when device is awake return that while the GT is
>>>>> suspended and then update the driver copy when read during awake.
>>>>>
>>>>> v3:
>>>>> 1. drop init_samples, as storing counters before going to suspend should
>>>>> be sufficient.
>>>>> 2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>>>>> dropped helpers to store and read samples.
>>>>> 3. use xe_device_mem_access_get_if_ongoing to check if device is active
>>>>> before reading the OA registers.
>>>>> 4. dropped format attr as no longer needed
>>>>> 5. introduce xe_pmu_suspend to call engine_group_busyness_store
>>>>> 6. few other nits.
>>>>>
>>>>> v4: minor nits.
>>>>>
>>>>> v5: take forcewake when accessing the OAG registers
>>>>>
>>>>> v6:
>>>>> 1. drop engine_busyness_sample_type
>>>>> 2. update UAPI documentation
>>>>>
>>>>> v7:
>>>>> 1. update UAPI documentation
>>>>> 2. drop MEDIA_GT specific change for media busyness counter.
>>>>>
>>>>> v8:
>>>>> 1. rebase
>>>>> 2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>>>>> 3. remove interrupts pmu event
>>>>>
>>>>> v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>>>>>
>>>>> Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>>>> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>>>> Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>>>> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>>>> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>>>> Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
>>>>> Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>>>>> ---
>>>>> drivers/gpu/drm/xe/Makefile | 2 +
>>>>> drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
>>>>> drivers/gpu/drm/xe/xe_device.c | 2 +
>>>>> drivers/gpu/drm/xe/xe_device_types.h | 4 +
>>>>> drivers/gpu/drm/xe/xe_gt.c | 2 +
>>>>> drivers/gpu/drm/xe/xe_module.c | 5 +
>>>>> drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
>>>>> drivers/gpu/drm/xe/xe_pmu.h | 26 ++
>>>>> drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
>>>>> include/uapi/drm/xe_drm.h | 39 ++
>>>>> 10 files changed, 783 insertions(+)
>>>>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
>>>>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
>>>>> create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>>>>>
>>>>> diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>>>>> index cbf961b90237..83bf1e07669b 100644
>>>>> --- a/drivers/gpu/drm/xe/Makefile
>>>>> +++ b/drivers/gpu/drm/xe/Makefile
>>>>> @@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
>>>>> i915-display/skl_universal_plane.o \
>>>>> i915-display/skl_watermark.o
>>>>>
>>>>> +xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>>>>> +
>>>>> ifeq ($(CONFIG_ACPI),y)
>>>>> xe-$(CONFIG_DRM_XE_DISPLAY) += \
>>>>> i915-display/intel_acpi.o \
>>>>> diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>>> index 47c26c37608d..22821dcd4e1b 100644
>>>>> --- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>>> +++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>>> @@ -390,6 +390,11 @@
>>>>> #define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
>>>>> #define GLOBAL_INVALIDATION_MODE REG_BIT(2)
>>>>>
>>>>> +#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
>>>>> +#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
>>>>> +#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
>>>>> +#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
>>>>> +
>>>>> #define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
>>>>> #define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
>>>>> index 64691a56d59c..bb00c8c9ec9b 100644
>>>>> --- a/drivers/gpu/drm/xe/xe_device.c
>>>>> +++ b/drivers/gpu/drm/xe/xe_device.c
>>>>> @@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
>>>>>
>>>>> xe_hwmon_register(xe);
>>>>>
>>>>> + xe_pmu_register(&xe->pmu);
>>>>> +
>>>>> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>>>>>
>>>>> err_fini_display:
>>>>> diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
>>>>> index 52bc461171d5..a5dba7325cf1 100644
>>>>> --- a/drivers/gpu/drm/xe/xe_device_types.h
>>>>> +++ b/drivers/gpu/drm/xe/xe_device_types.h
>>>>> @@ -18,6 +18,7 @@
>>>>> #include "xe_lmtt_types.h"
>>>>> #include "xe_memirq_types.h"
>>>>> #include "xe_platform_types.h"
>>>>> +#include "xe_pmu.h"
>>>>> #include "xe_pt_types.h"
>>>>> #include "xe_sriov_types.h"
>>>>> #include "xe_step_types.h"
>>>>> @@ -473,6 +474,9 @@ struct xe_device {
>>>>> int mode;
>>>>> } wedged;
>>>>>
>>>>> + /** @pmu: performance monitoring unit */
>>>>> + struct xe_pmu pmu;
>>>>> +
>>>>> /* private: */
>>>>>
>>>>> #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
>>>>> diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
>>>>> index 57d84751e160..477d0ae5f230 100644
>>>>> --- a/drivers/gpu/drm/xe/xe_gt.c
>>>>> +++ b/drivers/gpu/drm/xe/xe_gt.c
>>>>> @@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
>>>>> if (err)
>>>>> goto err_msg;
>>>>>
>>>>> + xe_pmu_suspend(gt);
>>>>> +
>>>>> err = xe_uc_suspend(>->uc);
>>>>> if (err)
>>>>> goto err_force_wake;
>>>>> diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
>>>>> index 3edeb30d5ccb..26f814f97fc2 100644
>>>>> --- a/drivers/gpu/drm/xe/xe_module.c
>>>>> +++ b/drivers/gpu/drm/xe/xe_module.c
>>>>> @@ -11,6 +11,7 @@
>>>>> #include "xe_drv.h"
>>>>> #include "xe_hw_fence.h"
>>>>> #include "xe_pci.h"
>>>>> +#include "xe_pmu.h"
>>>>> #include "xe_sched_job.h"
>>>>>
>>>>> struct xe_modparam xe_modparam = {
>>>>> @@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
>>>>> .init = xe_sched_job_module_init,
>>>>> .exit = xe_sched_job_module_exit,
>>>>> },
>>>>> + {
>>>>> + .init = xe_pmu_init,
>>>>> + .exit = xe_pmu_exit,
>>>>> + },
>>>>> {
>>>>> .init = xe_register_pci_driver,
>>>>> .exit = xe_unregister_pci_driver,
>>>>> diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>>>>> new file mode 100644
>>>>> index 000000000000..64960a358af2
>>>>> --- /dev/null
>>>>> +++ b/drivers/gpu/drm/xe/xe_pmu.c
>>>>> @@ -0,0 +1,631 @@
>>>>> +// SPDX-License-Identifier: MIT
>>>>> +/*
>>>>> + * Copyright © 2024 Intel Corporation
>>>>> + */
>>>>> +
>>>>> +#include <drm/drm_drv.h>
>>>>> +#include <drm/drm_managed.h>
>>>>> +#include <drm/xe_drm.h>
>>>>> +
>>>>> +#include "regs/xe_gt_regs.h"
>>>>> +#include "xe_device.h"
>>>>> +#include "xe_force_wake.h"
>>>>> +#include "xe_gt_clock.h"
>>>>> +#include "xe_mmio.h"
>>>>> +#include "xe_macros.h"
>>>>> +#include "xe_pm.h"
>>>>> +
>>>>> +static cpumask_t xe_pmu_cpumask;
>>>>> +static unsigned int xe_pmu_target_cpu = -1;
>>>>> +
>>>>> +static unsigned int config_gt_id(const u64 config)
>>>>> +{
>>>>> + return config >> __XE_PMU_GT_SHIFT;
>>>>> +}
>>>>> +
>>>>> +static u64 config_counter(const u64 config)
>>>>> +{
>>>>> + return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>>>>> +}
>>>>> +
>>>>> +static void xe_pmu_event_destroy(struct perf_event *event)
>>>>> +{
>>>>> + struct xe_device *xe =
>>>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>>>> +
>>>>> + drm_WARN_ON(&xe->drm, event->parent);
>>>>> +
>>>>> + drm_dev_put(&xe->drm);
>>>>> +}
>>>>> +
>>>>> +static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
>>>>> +{
>>>>> + u64 val;
>>>>> +
>>>>> + switch (sample_type) {
>>>>> + case __XE_SAMPLE_RENDER_GROUP_BUSY:
>>>>> + val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>>>>> + break;
>>>>> + case __XE_SAMPLE_COPY_GROUP_BUSY:
>>>>> + val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>>>>> + break;
>>>>> + case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>>>>> + val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>>>>> + break;
>>>>> + case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>>>>> + val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>>>>> + break;
>>>>> + default:
>>>>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>>>> + }
>>>>> +
>>>>> + return xe_gt_clock_cycles_to_ns(gt, val * 16);
>>>>> +}
>>>>> +
>>>>> +static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>>>>> +{
>>>>> + int sample_type = config_counter(config);
>>>>> + const unsigned int gt_id = gt->info.id;
>>>>> + struct xe_device *xe = gt->tile->xe;
>>>>> + struct xe_pmu *pmu = &xe->pmu;
>>>>> + unsigned long flags;
>>>>> + bool device_awake;
>>>>> + u64 val;
>>>>> +
>>>>> + device_awake = xe_pm_runtime_get_if_active(xe);
>>>>> + if (device_awake) {
>>>>> + XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>>>>> + val = __engine_group_busyness_read(gt, sample_type);
>>>>> + XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>>>>> + xe_pm_runtime_put(xe);
>>>>> + }
>>>>> +
>>>>> + spin_lock_irqsave(&pmu->lock, flags);
>>>>> +
>>>>> + if (device_awake)
>>>>> + pmu->sample[gt_id][sample_type] = val;
>>>>> + else
>>>>> + val = pmu->sample[gt_id][sample_type];
>>>>> +
>>>>> + spin_unlock_irqrestore(&pmu->lock, flags);
>>>>> +
>>>>> + return val;
>>>>> +}
>>>>> +
>>>>> +static void engine_group_busyness_store(struct xe_gt *gt)
>>>>> +{
>>>>> + struct xe_pmu *pmu = >->tile->xe->pmu;
>>>>> + unsigned int gt_id = gt->info.id;
>>>>> + unsigned long flags;
>>>>> + int i;
>>>>> +
>>>>> + spin_lock_irqsave(&pmu->lock, flags);
>>>>> +
>>>>> + for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>>>>> + pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>>>>> +
>>>>> + spin_unlock_irqrestore(&pmu->lock, flags);
>>>>> +}
>>>>> +
>>>>> +static int
>>>>> +config_status(struct xe_device *xe, u64 config)
>>>>> +{
>>>>> + unsigned int gt_id = config_gt_id(config);
>>>>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>>>> +
>>>>> + if (gt_id >= XE_PMU_MAX_GT)
>>>>> + return -ENOENT;
>>>>> +
>>>>> + switch (config_counter(config)) {
>>>>> + case XE_PMU_RENDER_GROUP_BUSY(0):
>>>>> + case XE_PMU_COPY_GROUP_BUSY(0):
>>>>> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>>>> + if (gt->info.type == XE_GT_TYPE_MEDIA)
>>>>> + return -ENOENT;
>>>>> + break;
>>>>> + case XE_PMU_MEDIA_GROUP_BUSY(0):
>>>>> + if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
>>>>> + return -ENOENT;
>>>>> + break;
>>>>> + default:
>>>>> + return -ENOENT;
>>>>> + }
>>>>> +
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> +static int xe_pmu_event_init(struct perf_event *event)
>>>>> +{
>>>>> + struct xe_device *xe =
>>>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>>>> + struct xe_pmu *pmu = &xe->pmu;
>>>>> + int ret;
>>>>> +
>>>>> + if (pmu->closed)
>>>>> + return -ENODEV;
>>>>> +
>>>>> + if (event->attr.type != event->pmu->type)
>>>>> + return -ENOENT;
>>>>> +
>>>>> + /* unsupported modes and filters */
>>>>> + if (event->attr.sample_period) /* no sampling */
>>>>> + return -EINVAL;
>>>>> +
>>>>> + if (has_branch_stack(event))
>>>>> + return -EOPNOTSUPP;
>>>>> +
>>>>> + if (event->cpu < 0)
>>>>> + return -EINVAL;
>>>>> +
>>>>> + /* only allow running on one cpu at a time */
>>>>> + if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>>>>> + return -EINVAL;
>>>>> +
>>>>> + ret = config_status(xe, event->attr.config);
>>>>> + if (ret)
>>>>> + return ret;
>>>>> +
>>>>> + if (!event->parent) {
>>>>> + drm_dev_get(&xe->drm);
>>>>> + event->destroy = xe_pmu_event_destroy;
>>>>> + }
>>>>> +
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> +static u64 __xe_pmu_event_read(struct perf_event *event)
>>>>> +{
>>>>> + struct xe_device *xe =
>>>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>>>> + const unsigned int gt_id = config_gt_id(event->attr.config);
>>>>> + const u64 config = event->attr.config;
>>>>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>>>> + u64 val;
>>>>> +
>>>>> + switch (config_counter(config)) {
>>>>> + case XE_PMU_RENDER_GROUP_BUSY(0):
>>>>> + case XE_PMU_COPY_GROUP_BUSY(0):
>>>>> + case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>>>> + case XE_PMU_MEDIA_GROUP_BUSY(0):
>>>>> + val = engine_group_busyness_read(gt, config);
>>>>> + break;
>>>>> + default:
>>>>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>>>> + }
>>>>> +
>>>>> + return val;
>>>>> +}
>>>>> +
>>>>> +static void xe_pmu_event_read(struct perf_event *event)
>>>>> +{
>>>>> + struct xe_device *xe =
>>>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>>>> + struct hw_perf_event *hwc = &event->hw;
>>>>> + struct xe_pmu *pmu = &xe->pmu;
>>>>> + u64 prev, new;
>>>>> +
>>>>> + if (pmu->closed) {
>>>>> + event->hw.state = PERF_HES_STOPPED;
>>>>> + return;
>>>>> + }
>>>>> +again:
>>>>> + prev = local64_read(&hwc->prev_count);
>>>>> + new = __xe_pmu_event_read(event);
>>>>> +
>>>>> + if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>>>>> + goto again;
>>>>> +
>>>>> + local64_add(new - prev, &event->count);
>>>>> +}
>>>>> +
>>>>> +static void xe_pmu_enable(struct perf_event *event)
>>>>> +{
>>>>> + /*
>>>>> + * Store the current counter value so we can report the correct delta
>>>>> + * for all listeners. Even when the event was already enabled and has
>>>>> + * an existing non-zero value.
>>>>> + */
>>>>> + local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>>>>> +}
>>>>> +
>>>>> +static void xe_pmu_event_start(struct perf_event *event, int flags)
>>>>> +{
>>>>> + struct xe_device *xe =
>>>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>>>> + struct xe_pmu *pmu = &xe->pmu;
>>>>> +
>>>>> + if (pmu->closed)
>>>>> + return;
>>>>> +
>>>>> + xe_pmu_enable(event);
>>>>> + event->hw.state = 0;
>>>>> +}
>>>>> +
>>>>> +static void xe_pmu_event_stop(struct perf_event *event, int flags)
>>>>> +{
>>>>> + if (flags & PERF_EF_UPDATE)
>>>>> + xe_pmu_event_read(event);
>>>>> +
>>>>> + event->hw.state = PERF_HES_STOPPED;
>>>>> +}
>>>>> +
>>>>> +static int xe_pmu_event_add(struct perf_event *event, int flags)
>>>>> +{
>>>>> + struct xe_device *xe =
>>>>> + container_of(event->pmu, typeof(*xe), pmu.base);
>>>>> + struct xe_pmu *pmu = &xe->pmu;
>>>>> +
>>>>> + if (pmu->closed)
>>>>> + return -ENODEV;
>>>>> +
>>>>> + if (flags & PERF_EF_START)
>>>>> + xe_pmu_event_start(event, flags);
>>>>> +
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> +static void xe_pmu_event_del(struct perf_event *event, int flags)
>>>>> +{
>>>>> + xe_pmu_event_stop(event, PERF_EF_UPDATE);
>>>>> +}
>>>>> +
>>>>> +static int xe_pmu_event_event_idx(struct perf_event *event)
>>>>> +{
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> +struct xe_ext_attribute {
>>>>> + struct device_attribute attr;
>>>>> + unsigned long val;
>>>>> +};
>>>>> +
>>>>> +static ssize_t xe_pmu_event_show(struct device *dev,
>>>>> + struct device_attribute *attr, char *buf)
>>>>> +{
>>>>> + struct xe_ext_attribute *eattr;
>>>>> +
>>>>> + eattr = container_of(attr, struct xe_ext_attribute, attr);
>>>>> + return sprintf(buf, "config=0x%lx\n", eattr->val);
>>>>> +}
>>>>> +
>>>>> +static ssize_t cpumask_show(struct device *dev,
>>>>> + struct device_attribute *attr, char *buf)
>>>>> +{
>>>>> + return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>>>>> +}
>>>>> +
>>>>> +static DEVICE_ATTR_RO(cpumask);
>>>>> +
>>>>> +static struct attribute *xe_cpumask_attrs[] = {
>>>>> + &dev_attr_cpumask.attr,
>>>>> + NULL,
>>>>> +};
>>>>> +
>>>>> +static const struct attribute_group xe_pmu_cpumask_attr_group = {
>>>>> + .attrs = xe_cpumask_attrs,
>>>>> +};
>>>>> +
>>>>> +#define __event(__counter, __name, __unit) \
>>>>> +{ \
>>>>> + .counter = (__counter), \
>>>>> + .name = (__name), \
>>>>> + .unit = (__unit), \
>>>>> +}
>>>>> +
>>>>> +static struct xe_ext_attribute *
>>>>> +add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>>>>> +{
>>>>> + sysfs_attr_init(&attr->attr.attr);
>>>>> + attr->attr.attr.name = name;
>>>>> + attr->attr.attr.mode = 0444;
>>>>> + attr->attr.show = xe_pmu_event_show;
>>>>> + attr->val = config;
>>>>> +
>>>>> + return ++attr;
>>>>> +}
>>>>> +
>>>>> +static struct perf_pmu_events_attr *
>>>>> +add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>>>>> + const char *str)
>>>>> +{
>>>>> + sysfs_attr_init(&attr->attr.attr);
>>>>> + attr->attr.attr.name = name;
>>>>> + attr->attr.attr.mode = 0444;
>>>>> + attr->attr.show = perf_event_sysfs_show;
>>>>> + attr->event_str = str;
>>>>> +
>>>>> + return ++attr;
>>>>> +}
>>>>> +
>>>>> +static struct attribute **
>>>>> +create_event_attributes(struct xe_pmu *pmu)
>>>>> +{
>>>>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>>>> + static const struct {
>>>>> + unsigned int counter;
>>>>> + const char *name;
>>>>> + const char *unit;
>>>>> + } events[] = {
>>>>> + __event(0, "render-group-busy", "ns"),
>>>>> + __event(1, "copy-group-busy", "ns"),
>>>>> + __event(2, "media-group-busy", "ns"),
>>>>> + __event(3, "any-engine-group-busy", "ns"),
>>>>> + };
>>>>> +
>>>>> + struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>>>>> + struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>>>>> + struct attribute **attr = NULL, **attr_iter;
>>>>> + unsigned int count = 0;
>>>>> + unsigned int i, j;
>>>>> + struct xe_gt *gt;
>>>>> +
>>>>> + /* Count how many counters we will be exposing. */
>>>>> + for_each_gt(gt, xe, j) {
>>>>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>>>>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>>>> +
>>>>> + if (!config_status(xe, config))
>>>>> + count++;
>>>>> + }
>>>>> + }
>>>>> +
>>>>> + /* Allocate attribute objects and table. */
>>>>> + xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>>>>> + if (!xe_attr)
>>>>> + goto err_alloc;
>>>>> +
>>>>> + pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>>>>> + if (!pmu_attr)
>>>>> + goto err_alloc;
>>>>> +
>>>>> + /* Max one pointer of each attribute type plus a termination entry. */
>>>>> + attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>>>>> + if (!attr)
>>>>> + goto err_alloc;
>>>>> +
>>>>> + xe_iter = xe_attr;
>>>>> + pmu_iter = pmu_attr;
>>>>> + attr_iter = attr;
>>>>> +
>>>>> + for_each_gt(gt, xe, j) {
>>>>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>>>>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>>>> + char *str;
>>>>> +
>>>>> + if (config_status(xe, config))
>>>>> + continue;
>>>>> +
>>>>> + str = kasprintf(GFP_KERNEL, "%s-gt%u",
>>>>> + events[i].name, j);
>>>>> + if (!str)
>>>>> + goto err;
>>>>> +
>>>>> + *attr_iter++ = &xe_iter->attr.attr;
>>>>> + xe_iter = add_xe_attr(xe_iter, str, config);
>>>>> +
>>>>> + if (events[i].unit) {
>>>>> + str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>>>>> + events[i].name, j);
>>>>> + if (!str)
>>>>> + goto err;
>>>>> +
>>>>> + *attr_iter++ = &pmu_iter->attr.attr;
>>>>> + pmu_iter = add_pmu_attr(pmu_iter, str,
>>>>> + events[i].unit);
>>>>> + }
>>>>> + }
>>>>> + }
>>>>> +
>>>>> + pmu->xe_attr = xe_attr;
>>>>> + pmu->pmu_attr = pmu_attr;
>>>>> +
>>>>> + return attr;
>>>>> +
>>>>> +err:
>>>>> + for (attr_iter = attr; *attr_iter; attr_iter++)
>>>>> + kfree((*attr_iter)->name);
>>>>> +
>>>>> +err_alloc:
>>>>> + kfree(attr);
>>>>> + kfree(xe_attr);
>>>>> + kfree(pmu_attr);
>>>>> +
>>>>> + return NULL;
>>>>> +}
>>>>> +
>>>>> +static void free_event_attributes(struct xe_pmu *pmu)
>>>>> +{
>>>>> + struct attribute **attr_iter = pmu->events_attr_group.attrs;
>>>>> +
>>>>> + for (; *attr_iter; attr_iter++)
>>>>> + kfree((*attr_iter)->name);
>>>>> +
>>>>> + kfree(pmu->events_attr_group.attrs);
>>>>> + kfree(pmu->xe_attr);
>>>>> + kfree(pmu->pmu_attr);
>>>>> +
>>>>> + pmu->events_attr_group.attrs = NULL;
>>>>> + pmu->xe_attr = NULL;
>>>>> + pmu->pmu_attr = NULL;
>>>>> +}
>>>>> +
>>>>> +static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>>>>> +{
>>>>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>>>> +
>>>>> + /* Select the first online CPU as a designated reader. */
>>>>> + if (cpumask_empty(&xe_pmu_cpumask))
>>>>> + cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>>>>> +
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> +static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>>>>> +{
>>>>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>>>> + unsigned int target = xe_pmu_target_cpu;
>>>>> +
>>>>> + /*
>>>>> + * Unregistering an instance generates a CPU offline event which we must
>>>>> + * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>>>>> + */
>>>>> + if (pmu->closed)
>>>>> + return 0;
>>>>> +
>>>>> + if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>>>>> + target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>>>>> +
>>>>> + /* Migrate events if there is a valid target */
>>>>> + if (target < nr_cpu_ids) {
>>>>> + cpumask_set_cpu(target, &xe_pmu_cpumask);
>>>>> + xe_pmu_target_cpu = target;
>>>>> + }
>>>>> + }
>>>>> +
>>>>> + if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>>>>> + perf_pmu_migrate_context(&pmu->base, cpu, target);
>>>>> + pmu->cpuhp.cpu = target;
>>>>> + }
>>>>> +
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> +static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>>>>> +
>>>>> +int xe_pmu_init(void)
>>>>> +{
>>>>> + int ret;
>>>>> +
>>>>> + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>>>>> + "perf/x86/intel/xe:online",
>>>>> + xe_pmu_cpu_online,
>>>>> + xe_pmu_cpu_offline);
>>>>> + if (ret < 0)
>>>>> + pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>>>>> + ret);
>>>>> + else
>>>>> + cpuhp_slot = ret;
>>>>> +
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> +void xe_pmu_exit(void)
>>>>> +{
>>>>> + if (cpuhp_slot != CPUHP_INVALID)
>>>>> + cpuhp_remove_multi_state(cpuhp_slot);
>>>>> +}
>>>>> +
>>>>> +static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>>>>> +{
>>>>> + if (cpuhp_slot == CPUHP_INVALID)
>>>>> + return -EINVAL;
>>>>> +
>>>>> + return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>>>>> +}
>>>>> +
>>>>> +static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>>>>> +{
>>>>> + cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>>>>> +}
>>>>> +
>>>>> +void xe_pmu_suspend(struct xe_gt *gt)
>>>>> +{
>>>>> + engine_group_busyness_store(gt);
>>>>> +}
>>>>> +
>>>>> +static void xe_pmu_unregister(void *arg)
>>>>> +{
>>>>> + struct xe_pmu *pmu = arg;
>>>>> +
>>>>> + if (!pmu->base.event_init)
>>>>> + return;
>>>>> +
>>>>> + /*
>>>>> + * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
>>>>> + * ensures all currently executing ones will have exited before we
>>>>> + * proceed with unregistration.
>>>>> + */
>>>>> + pmu->closed = true;
>>>>> + synchronize_rcu();
>>>>> +
>>>>> + xe_pmu_unregister_cpuhp_state(pmu);
>>>>> +
>>>>> + perf_pmu_unregister(&pmu->base);
>>>>> + pmu->base.event_init = NULL;
>>>>> + kfree(pmu->base.attr_groups);
>>>>> + kfree(pmu->name);
>>>>> + free_event_attributes(pmu);
>>>>> +}
>>>>> +
>>>>> +void xe_pmu_register(struct xe_pmu *pmu)
>>>>> +{
>>>>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>>>> + const struct attribute_group *attr_groups[] = {
>>>>> + &pmu->events_attr_group,
>>>>> + &xe_pmu_cpumask_attr_group,
>>>>> + NULL
>>>>> + };
>>>>> +
>>>>> + int ret = -ENOMEM;
>>>>> +
>>>>> + spin_lock_init(&pmu->lock);
>>>>> + pmu->cpuhp.cpu = -1;
>>>>> +
>>>>> + pmu->name = kasprintf(GFP_KERNEL,
>>>>> + "xe_%s",
>>>>> + dev_name(xe->drm.dev));
>>>>> + if (pmu->name)
>>>>> + /* tools/perf reserves colons as special. */
>>>>> + strreplace((char *)pmu->name, ':', '_');
>>>>> +
>>>>> + if (!pmu->name)
>>>>> + goto err;
>>>>> +
>>>>> + pmu->events_attr_group.name = "events";
>>>>> + pmu->events_attr_group.attrs = create_event_attributes(pmu);
>>>>> + if (!pmu->events_attr_group.attrs)
>>>>> + goto err_name;
>>>>> +
>>>>> + pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>>>>> + GFP_KERNEL);
>>>>> + if (!pmu->base.attr_groups)
>>>>> + goto err_attr;
>>>>> +
>>>>> + pmu->base.module = THIS_MODULE;
>>>>> + pmu->base.task_ctx_nr = perf_invalid_context;
>>>>> + pmu->base.event_init = xe_pmu_event_init;
>>>>> + pmu->base.add = xe_pmu_event_add;
>>>>> + pmu->base.del = xe_pmu_event_del;
>>>>> + pmu->base.start = xe_pmu_event_start;
>>>>> + pmu->base.stop = xe_pmu_event_stop;
>>>>> + pmu->base.read = xe_pmu_event_read;
>>>>> + pmu->base.event_idx = xe_pmu_event_event_idx;
>>>>> +
>>>>> + ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>>>>> + if (ret)
>>>>> + goto err_groups;
>>>>> +
>>>>> + ret = xe_pmu_register_cpuhp_state(pmu);
>>>>> + if (ret)
>>>>> + goto err_unreg;
>>>>> +
>>>>> + ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>>>>> + if (ret)
>>>>> + goto err_cpuhp;
>>>>> +
>>>>> + return;
>>>>> +
>>>>> +err_cpuhp:
>>>>> + xe_pmu_unregister_cpuhp_state(pmu);
>>>>> +err_unreg:
>>>>> + perf_pmu_unregister(&pmu->base);
>>>>> +err_groups:
>>>>> + kfree(pmu->base.attr_groups);
>>>>> +err_attr:
>>>>> + pmu->base.event_init = NULL;
>>>>> + free_event_attributes(pmu);
>>>>> +err_name:
>>>>> + kfree(pmu->name);
>>>>> +err:
>>>>> + drm_notice(&xe->drm, "Failed to register PMU!\n");
>>>>> +}
>>>>> diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>>>>> new file mode 100644
>>>>> index 000000000000..8afa256f9dac
>>>>> --- /dev/null
>>>>> +++ b/drivers/gpu/drm/xe/xe_pmu.h
>>>>> @@ -0,0 +1,26 @@
>>>>> +/* SPDX-License-Identifier: MIT */
>>>>> +/*
>>>>> + * Copyright © 2024 Intel Corporation
>>>>> + */
>>>>> +
>>>>> +#ifndef _XE_PMU_H_
>>>>> +#define _XE_PMU_H_
>>>>> +
>>>>> +#include "xe_pmu_types.h"
>>>>> +
>>>>> +struct xe_gt;
>>>>> +
>>>>> +#if IS_ENABLED(CONFIG_PERF_EVENTS)
>>>>> +int xe_pmu_init(void);
>>>>> +void xe_pmu_exit(void);
>>>>> +void xe_pmu_register(struct xe_pmu *pmu);
>>>>> +void xe_pmu_suspend(struct xe_gt *gt);
>>>>> +#else
>>>>> +static inline int xe_pmu_init(void) { return 0; }
>>>>> +static inline void xe_pmu_exit(void) {}
>>>>> +static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>>>>> +static inline void xe_pmu_suspend(struct xe_gt *gt) {}
>>>>> +#endif
>>>>> +
>>>>> +#endif
>>>>> +
>>>>> diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
>>>>> new file mode 100644
>>>>> index 000000000000..e86e8d7e0356
>>>>> --- /dev/null
>>>>> +++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>>>>> @@ -0,0 +1,67 @@
>>>>> +/* SPDX-License-Identifier: MIT */
>>>>> +/*
>>>>> + * Copyright © 2024 Intel Corporation
>>>>> + */
>>>>> +
>>>>> +#ifndef _XE_PMU_TYPES_H_
>>>>> +#define _XE_PMU_TYPES_H_
>>>>> +
>>>>> +#include <linux/perf_event.h>
>>>>> +#include <linux/spinlock_types.h>
>>>>> +#include <uapi/drm/xe_drm.h>
>>>>> +
>>>>> +enum {
>>>>> + __XE_SAMPLE_RENDER_GROUP_BUSY,
>>>>> + __XE_SAMPLE_COPY_GROUP_BUSY,
>>>>> + __XE_SAMPLE_MEDIA_GROUP_BUSY,
>>>>> + __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
>>>>> + __XE_NUM_PMU_SAMPLERS
>>>>> +};
>>>>> +
>>>>> +#define XE_PMU_MAX_GT 2
>>>>> +
>>>>> +struct xe_pmu {
>>>>> + /**
>>>>> + * @cpuhp: Struct used for CPU hotplug handling.
>>>>> + */
>>>>> + struct {
>>>>> + struct hlist_node node;
>>>>> + unsigned int cpu;
>>>>> + } cpuhp;
>>>>> + /**
>>>>> + * @base: PMU base.
>>>>> + */
>>>>> + struct pmu base;
>>>>> + /**
>>>>> + * @closed: xe is unregistering.
>>>>> + */
>>>>> + bool closed;
>>>>> + /**
>>>>> + * @name: Name as registered with perf core.
>>>>> + */
>>>>> + const char *name;
>>>>> + /**
>>>>> + * @lock: Lock protecting enable mask and ref count handling.
>>>>> + */
>>>>> + spinlock_t lock;
>>>>> + /**
>>>>> + * @sample: Current and previous (raw) counters.
>>>>> + *
>>>>> + * These counters are updated when the device is awake.
>>>>> + */
>>>>> + u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>>>>> + /**
>>>>> + * @events_attr_group: Device events attribute group.
>>>>> + */
>>>>> + struct attribute_group events_attr_group;
>>>>> + /**
>>>>> + * @xe_attr: Memory block holding device attributes.
>>>>> + */
>>>>> + void *xe_attr;
>>>>> + /**
>>>>> + * @pmu_attr: Memory block holding device attributes.
>>>>> + */
>>>>> + void *pmu_attr;
>>>>> +};
>>>>> +
>>>>> +#endif
>>>>> diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>>>>> index d7b0903c22b2..07ca545354f7 100644
>>>>> --- a/include/uapi/drm/xe_drm.h
>>>>> +++ b/include/uapi/drm/xe_drm.h
>>>>> @@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
>>>>> __u64 reserved[2];
>>>>> };
>>>>>
>>>>> +/**
>>>>> + * DOC: XE PMU event config IDs
>>>>> + *
>>>>> + * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in xe_drm.h
>>>>> + * in 'struct perf_event_attr' as part of perf_event_open syscall to read a
>>>>> + * particular event.
>>>>> + *
>>>>> + * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
>>>>> + *
>>>>> + * .. code-block:: C
>>>>> + *
>>>>> + * struct perf_event_attr attr;
>>>>> + * long long count;
>>>>> + * int cpu = 0;
>>>>> + * int fd;
>>>>> + *
>>>>> + * memset(&attr, 0, sizeof(struct perf_event_attr));
>>>>> + * attr.type = type; // eg: /sys/bus/event_source/devices/xe_0000_56_00.0/type
>>>>> + * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
>>>>> + * attr.use_clockid = 1;
>>>>> + * attr.clockid = CLOCK_MONOTONIC;
>>>>> + * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
>>>>> + *
>>>>> + * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
>>>>> + */
>>>>> +
>>>>> +/*
>>>>> + * Top bits of every counter are GT id.
>>>>> + */
>>>>> +#define __XE_PMU_GT_SHIFT (56)
>>>>> +
>>>>> +#define ___XE_PMU_OTHER(gt, x) \
>>>>> + (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>>>>> +
>>>>> +#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
>>>>> +#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
>>>>> +#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
>>>>> +#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
>>>>
>>>> + Lucas for inputs
>>>>
>>>> We should align this to the interface planned for other PMU busyness counters as well as how we do PCEU. i.e.
>>>>
>>>> 1) counters are in ticks
>>>> 2) total time in ticks is also exported to the user.
>>>>
>>>> For 1), I would just append TICKS to the counter names and drop the conversion to _ns in __engine_group_busyness_read(). Also, drop the patch that adds this conversion helper.
>>>>
>>>> For 2) define a new counter - total active ticks that would return the 'CPU' timestamp converted to gpu ticks. The reason I am insisting on CPU timestamp here is because we want to have a time base that is ticking even when the GPU is idle.
>>>
>>> why can't we expose what the HW presents[1] to us via register and leave the interpretation to userspace.
>>
>> HW is indeed exposing ticks. In this case, I am suggesting to expose that directly to the user, so I think you are saying the same.
>correct
>>
>> As for interpretation, we need to make sure it works consistently in SRIOV. The L0 API for group busyness itself imposes the requirement for another counter to make sense of [1]. This additional counter has always existed, but in prior implementations, it was just using the CPU time in the equation. The CPU sample time is always returned in all the PMU counters. With SRIOV, it will still be CPU time, but only the time that a VF executed for and that information is only available to the KMD/GuC. Without that information, interpreting the ticks in [1] will not be meaningful.
>
>I hope in SRIOV case you are mentioning about accessing these counters from PF right? if it is from VF I'm not sure if these registers are accessible.
Correct, PF only.
Regards,
Umesh
>
>Regards,
>Aravind.
>>
>> Regards,
>> Umesh
>>
>>
>>>
>>> Thanks,
>>> Aravind.
>>>>
>>>> Regards,
>>>> Umesh
>>>>
>>>>> +
>>>>> #if defined(__cplusplus)
>>>>> }
>>>>> #endif
>>>>> --
>>>>> 2.40.0
>>>>>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-28 15:55 ` Lucas De Marchi
@ 2024-06-28 16:52 ` Umesh Nerlige Ramappa
2024-06-28 18:24 ` Lucas De Marchi
0 siblings, 1 reply; 32+ messages in thread
From: Umesh Nerlige Ramappa @ 2024-06-28 16:52 UTC (permalink / raw)
To: Lucas De Marchi
Cc: Riana Tauro, intel-xe, anshuman.gupta, ashutosh.dixit,
aravind.iddamsetty, rodrigo.vivi, krishnaiah.bommu
On Fri, Jun 28, 2024 at 10:55:06AM -0500, Lucas De Marchi wrote:
>On Thu, Jun 20, 2024 at 12:52:05PM GMT, Umesh Nerlige Ramappa wrote:
>>On Thu, Jun 13, 2024 at 03:34:11PM +0530, Riana Tauro wrote:
>>>From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>>
>>>There are a set of engine group busyness counters provided by HW which are
>>>perfect fit to be exposed via PMU perf events.
>>>
>>>BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>>
>>>events can be listed using:
>>>perf list
>>>xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>>>xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>>>xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>>>xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>>
>>>and can be read using:
>>>
>>>perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
>>> time counts unit events
>>> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>
>>>The pmu base implementation is taken from i915.
>>>
>>>v2:
>>>Store last known value when device is awake return that while the GT is
>>>suspended and then update the driver copy when read during awake.
>>>
>>>v3:
>>>1. drop init_samples, as storing counters before going to suspend should
>>>be sufficient.
>>>2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>>>dropped helpers to store and read samples.
>>>3. use xe_device_mem_access_get_if_ongoing to check if device is active
>>>before reading the OA registers.
>>>4. dropped format attr as no longer needed
>>>5. introduce xe_pmu_suspend to call engine_group_busyness_store
>>>6. few other nits.
>>>
>>>v4: minor nits.
>>>
>>>v5: take forcewake when accessing the OAG registers
>>>
>>>v6:
>>>1. drop engine_busyness_sample_type
>>>2. update UAPI documentation
>>>
>>>v7:
>>>1. update UAPI documentation
>>>2. drop MEDIA_GT specific change for media busyness counter.
>>>
>>>v8:
>>>1. rebase
>>>2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>>>3. remove interrupts pmu event
>>>
>>>v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>>>
>>>Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>>Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>>Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>>Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>>Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>>Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
>>>Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>>>---
>>>drivers/gpu/drm/xe/Makefile | 2 +
>>>drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
>>>drivers/gpu/drm/xe/xe_device.c | 2 +
>>>drivers/gpu/drm/xe/xe_device_types.h | 4 +
>>>drivers/gpu/drm/xe/xe_gt.c | 2 +
>>>drivers/gpu/drm/xe/xe_module.c | 5 +
>>>drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
>>>drivers/gpu/drm/xe/xe_pmu.h | 26 ++
>>>drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
>>>include/uapi/drm/xe_drm.h | 39 ++
>>>10 files changed, 783 insertions(+)
>>>create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
>>>create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
>>>create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>>>
>>>diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>>>index cbf961b90237..83bf1e07669b 100644
>>>--- a/drivers/gpu/drm/xe/Makefile
>>>+++ b/drivers/gpu/drm/xe/Makefile
>>>@@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
>>> i915-display/skl_universal_plane.o \
>>> i915-display/skl_watermark.o
>>>
>>>+xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>>>+
>>>ifeq ($(CONFIG_ACPI),y)
>>> xe-$(CONFIG_DRM_XE_DISPLAY) += \
>>> i915-display/intel_acpi.o \
>>>diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>index 47c26c37608d..22821dcd4e1b 100644
>>>--- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>+++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>@@ -390,6 +390,11 @@
>>>#define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
>>>#define GLOBAL_INVALIDATION_MODE REG_BIT(2)
>>>
>>>+#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
>>>+#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
>>>+#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
>>>+#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
>>>+
>>>#define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
>>>#define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
>>>
>>>diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
>>>index 64691a56d59c..bb00c8c9ec9b 100644
>>>--- a/drivers/gpu/drm/xe/xe_device.c
>>>+++ b/drivers/gpu/drm/xe/xe_device.c
>>>@@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
>>>
>>> xe_hwmon_register(xe);
>>>
>>>+ xe_pmu_register(&xe->pmu);
>>>+
>>> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>>>
>>>err_fini_display:
>>>diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
>>>index 52bc461171d5..a5dba7325cf1 100644
>>>--- a/drivers/gpu/drm/xe/xe_device_types.h
>>>+++ b/drivers/gpu/drm/xe/xe_device_types.h
>>>@@ -18,6 +18,7 @@
>>>#include "xe_lmtt_types.h"
>>>#include "xe_memirq_types.h"
>>>#include "xe_platform_types.h"
>>>+#include "xe_pmu.h"
>>>#include "xe_pt_types.h"
>>>#include "xe_sriov_types.h"
>>>#include "xe_step_types.h"
>>>@@ -473,6 +474,9 @@ struct xe_device {
>>> int mode;
>>> } wedged;
>>>
>>>+ /** @pmu: performance monitoring unit */
>>>+ struct xe_pmu pmu;
>>>+
>>> /* private: */
>>>
>>>#if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
>>>diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
>>>index 57d84751e160..477d0ae5f230 100644
>>>--- a/drivers/gpu/drm/xe/xe_gt.c
>>>+++ b/drivers/gpu/drm/xe/xe_gt.c
>>>@@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
>>> if (err)
>>> goto err_msg;
>>>
>>>+ xe_pmu_suspend(gt);
>>>+
>>> err = xe_uc_suspend(>->uc);
>>> if (err)
>>> goto err_force_wake;
>>>diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
>>>index 3edeb30d5ccb..26f814f97fc2 100644
>>>--- a/drivers/gpu/drm/xe/xe_module.c
>>>+++ b/drivers/gpu/drm/xe/xe_module.c
>>>@@ -11,6 +11,7 @@
>>>#include "xe_drv.h"
>>>#include "xe_hw_fence.h"
>>>#include "xe_pci.h"
>>>+#include "xe_pmu.h"
>>>#include "xe_sched_job.h"
>>>
>>>struct xe_modparam xe_modparam = {
>>>@@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
>>> .init = xe_sched_job_module_init,
>>> .exit = xe_sched_job_module_exit,
>>> },
>>>+ {
>>>+ .init = xe_pmu_init,
>>>+ .exit = xe_pmu_exit,
>>>+ },
>>> {
>>> .init = xe_register_pci_driver,
>>> .exit = xe_unregister_pci_driver,
>>>diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>>>new file mode 100644
>>>index 000000000000..64960a358af2
>>>--- /dev/null
>>>+++ b/drivers/gpu/drm/xe/xe_pmu.c
>>>@@ -0,0 +1,631 @@
>>>+// SPDX-License-Identifier: MIT
>>>+/*
>>>+ * Copyright © 2024 Intel Corporation
>>>+ */
>>>+
>>>+#include <drm/drm_drv.h>
>>>+#include <drm/drm_managed.h>
>>>+#include <drm/xe_drm.h>
>>>+
>>>+#include "regs/xe_gt_regs.h"
>>>+#include "xe_device.h"
>>>+#include "xe_force_wake.h"
>>>+#include "xe_gt_clock.h"
>>>+#include "xe_mmio.h"
>>>+#include "xe_macros.h"
>>>+#include "xe_pm.h"
>>>+
>>>+static cpumask_t xe_pmu_cpumask;
>>>+static unsigned int xe_pmu_target_cpu = -1;
>>>+
>>>+static unsigned int config_gt_id(const u64 config)
>>>+{
>>>+ return config >> __XE_PMU_GT_SHIFT;
>>>+}
>>>+
>>>+static u64 config_counter(const u64 config)
>>>+{
>>>+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>>>+}
>>>+
>>>+static void xe_pmu_event_destroy(struct perf_event *event)
>>>+{
>>>+ struct xe_device *xe =
>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>+
>>>+ drm_WARN_ON(&xe->drm, event->parent);
>>>+
>>>+ drm_dev_put(&xe->drm);
>>>+}
>>>+
>>>+static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
>>>+{
>>>+ u64 val;
>>>+
>>>+ switch (sample_type) {
>>>+ case __XE_SAMPLE_RENDER_GROUP_BUSY:
>>>+ val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>>>+ break;
>>>+ case __XE_SAMPLE_COPY_GROUP_BUSY:
>>>+ val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>>>+ break;
>>>+ case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>>>+ val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>>>+ break;
>>>+ case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>>>+ val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>>>+ break;
>>>+ default:
>>>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>>+ }
>>>+
>>>+ return xe_gt_clock_cycles_to_ns(gt, val * 16);
>>>+}
>>>+
>>>+static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>>>+{
>>>+ int sample_type = config_counter(config);
>>>+ const unsigned int gt_id = gt->info.id;
>>>+ struct xe_device *xe = gt->tile->xe;
>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>+ unsigned long flags;
>>>+ bool device_awake;
>>>+ u64 val;
>>>+
>>>+ device_awake = xe_pm_runtime_get_if_active(xe);
>>>+ if (device_awake) {
>>>+ XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>>>+ val = __engine_group_busyness_read(gt, sample_type);
>>>+ XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>>>+ xe_pm_runtime_put(xe);
>>>+ }
>>>+
>>>+ spin_lock_irqsave(&pmu->lock, flags);
>>>+
>>>+ if (device_awake)
>>>+ pmu->sample[gt_id][sample_type] = val;
>>>+ else
>>>+ val = pmu->sample[gt_id][sample_type];
>>>+
>>>+ spin_unlock_irqrestore(&pmu->lock, flags);
>>>+
>>>+ return val;
>>>+}
>>>+
>>>+static void engine_group_busyness_store(struct xe_gt *gt)
>>>+{
>>>+ struct xe_pmu *pmu = >->tile->xe->pmu;
>>>+ unsigned int gt_id = gt->info.id;
>>>+ unsigned long flags;
>>>+ int i;
>>>+
>>>+ spin_lock_irqsave(&pmu->lock, flags);
>>>+
>>>+ for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>>>+ pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>>>+
>>>+ spin_unlock_irqrestore(&pmu->lock, flags);
>>>+}
>>>+
>>>+static int
>>>+config_status(struct xe_device *xe, u64 config)
>>>+{
>>>+ unsigned int gt_id = config_gt_id(config);
>>>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>>+
>>>+ if (gt_id >= XE_PMU_MAX_GT)
>>>+ return -ENOENT;
>>>+
>>>+ switch (config_counter(config)) {
>>>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>>>+ case XE_PMU_COPY_GROUP_BUSY(0):
>>>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>>+ if (gt->info.type == XE_GT_TYPE_MEDIA)
>>>+ return -ENOENT;
>>>+ break;
>>>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>>>+ if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
>>>+ return -ENOENT;
>>>+ break;
>>>+ default:
>>>+ return -ENOENT;
>>>+ }
>>>+
>>>+ return 0;
>>>+}
>>>+
>>>+static int xe_pmu_event_init(struct perf_event *event)
>>>+{
>>>+ struct xe_device *xe =
>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>+ int ret;
>>>+
>>>+ if (pmu->closed)
>>>+ return -ENODEV;
>>>+
>>>+ if (event->attr.type != event->pmu->type)
>>>+ return -ENOENT;
>>>+
>>>+ /* unsupported modes and filters */
>>>+ if (event->attr.sample_period) /* no sampling */
>>>+ return -EINVAL;
>>>+
>>>+ if (has_branch_stack(event))
>>>+ return -EOPNOTSUPP;
>>>+
>>>+ if (event->cpu < 0)
>>>+ return -EINVAL;
>>>+
>>>+ /* only allow running on one cpu at a time */
>>>+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>>>+ return -EINVAL;
>>>+
>>>+ ret = config_status(xe, event->attr.config);
>>>+ if (ret)
>>>+ return ret;
>>>+
>>>+ if (!event->parent) {
>>>+ drm_dev_get(&xe->drm);
>>>+ event->destroy = xe_pmu_event_destroy;
>>>+ }
>>>+
>>>+ return 0;
>>>+}
>>>+
>>>+static u64 __xe_pmu_event_read(struct perf_event *event)
>>>+{
>>>+ struct xe_device *xe =
>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>+ const unsigned int gt_id = config_gt_id(event->attr.config);
>>>+ const u64 config = event->attr.config;
>>>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>>+ u64 val;
>>>+
>>>+ switch (config_counter(config)) {
>>>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>>>+ case XE_PMU_COPY_GROUP_BUSY(0):
>>>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>>>+ val = engine_group_busyness_read(gt, config);
>>>+ break;
>>>+ default:
>>>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>>+ }
>>>+
>>>+ return val;
>>>+}
>>>+
>>>+static void xe_pmu_event_read(struct perf_event *event)
>>>+{
>>>+ struct xe_device *xe =
>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>+ struct hw_perf_event *hwc = &event->hw;
>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>+ u64 prev, new;
>>>+
>>>+ if (pmu->closed) {
>>>+ event->hw.state = PERF_HES_STOPPED;
>>>+ return;
>>>+ }
>>>+again:
>>>+ prev = local64_read(&hwc->prev_count);
>>>+ new = __xe_pmu_event_read(event);
>>>+
>>>+ if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>>>+ goto again;
>>>+
>>>+ local64_add(new - prev, &event->count);
>>>+}
>>>+
>>>+static void xe_pmu_enable(struct perf_event *event)
>>>+{
>>>+ /*
>>>+ * Store the current counter value so we can report the correct delta
>>>+ * for all listeners. Even when the event was already enabled and has
>>>+ * an existing non-zero value.
>>>+ */
>>>+ local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>>>+}
>>>+
>>>+static void xe_pmu_event_start(struct perf_event *event, int flags)
>>>+{
>>>+ struct xe_device *xe =
>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>+
>>>+ if (pmu->closed)
>>>+ return;
>>>+
>>>+ xe_pmu_enable(event);
>>>+ event->hw.state = 0;
>>>+}
>>>+
>>>+static void xe_pmu_event_stop(struct perf_event *event, int flags)
>>>+{
>>>+ if (flags & PERF_EF_UPDATE)
>>>+ xe_pmu_event_read(event);
>>>+
>>>+ event->hw.state = PERF_HES_STOPPED;
>>>+}
>>>+
>>>+static int xe_pmu_event_add(struct perf_event *event, int flags)
>>>+{
>>>+ struct xe_device *xe =
>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>+
>>>+ if (pmu->closed)
>>>+ return -ENODEV;
>>>+
>>>+ if (flags & PERF_EF_START)
>>>+ xe_pmu_event_start(event, flags);
>>>+
>>>+ return 0;
>>>+}
>>>+
>>>+static void xe_pmu_event_del(struct perf_event *event, int flags)
>>>+{
>>>+ xe_pmu_event_stop(event, PERF_EF_UPDATE);
>>>+}
>>>+
>>>+static int xe_pmu_event_event_idx(struct perf_event *event)
>>>+{
>>>+ return 0;
>>>+}
>>>+
>>>+struct xe_ext_attribute {
>>>+ struct device_attribute attr;
>>>+ unsigned long val;
>>>+};
>>>+
>>>+static ssize_t xe_pmu_event_show(struct device *dev,
>>>+ struct device_attribute *attr, char *buf)
>>>+{
>>>+ struct xe_ext_attribute *eattr;
>>>+
>>>+ eattr = container_of(attr, struct xe_ext_attribute, attr);
>>>+ return sprintf(buf, "config=0x%lx\n", eattr->val);
>>>+}
>>>+
>>>+static ssize_t cpumask_show(struct device *dev,
>>>+ struct device_attribute *attr, char *buf)
>>>+{
>>>+ return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>>>+}
>>>+
>>>+static DEVICE_ATTR_RO(cpumask);
>>>+
>>>+static struct attribute *xe_cpumask_attrs[] = {
>>>+ &dev_attr_cpumask.attr,
>>>+ NULL,
>>>+};
>>>+
>>>+static const struct attribute_group xe_pmu_cpumask_attr_group = {
>>>+ .attrs = xe_cpumask_attrs,
>>>+};
>>>+
>>>+#define __event(__counter, __name, __unit) \
>>>+{ \
>>>+ .counter = (__counter), \
>>>+ .name = (__name), \
>>>+ .unit = (__unit), \
>>>+}
>>>+
>>>+static struct xe_ext_attribute *
>>>+add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>>>+{
>>>+ sysfs_attr_init(&attr->attr.attr);
>>>+ attr->attr.attr.name = name;
>>>+ attr->attr.attr.mode = 0444;
>>>+ attr->attr.show = xe_pmu_event_show;
>>>+ attr->val = config;
>>>+
>>>+ return ++attr;
>>>+}
>>>+
>>>+static struct perf_pmu_events_attr *
>>>+add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>>>+ const char *str)
>>>+{
>>>+ sysfs_attr_init(&attr->attr.attr);
>>>+ attr->attr.attr.name = name;
>>>+ attr->attr.attr.mode = 0444;
>>>+ attr->attr.show = perf_event_sysfs_show;
>>>+ attr->event_str = str;
>>>+
>>>+ return ++attr;
>>>+}
>>>+
>>>+static struct attribute **
>>>+create_event_attributes(struct xe_pmu *pmu)
>>>+{
>>>+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>>+ static const struct {
>>>+ unsigned int counter;
>>>+ const char *name;
>>>+ const char *unit;
>>>+ } events[] = {
>>>+ __event(0, "render-group-busy", "ns"),
>>>+ __event(1, "copy-group-busy", "ns"),
>>>+ __event(2, "media-group-busy", "ns"),
>>>+ __event(3, "any-engine-group-busy", "ns"),
>>>+ };
>>>+
>>>+ struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>>>+ struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>>>+ struct attribute **attr = NULL, **attr_iter;
>>>+ unsigned int count = 0;
>>>+ unsigned int i, j;
>>>+ struct xe_gt *gt;
>>>+
>>>+ /* Count how many counters we will be exposing. */
>>>+ for_each_gt(gt, xe, j) {
>>>+ for (i = 0; i < ARRAY_SIZE(events); i++) {
>>>+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>>+
>>>+ if (!config_status(xe, config))
>>>+ count++;
>>>+ }
>>>+ }
>>>+
>>>+ /* Allocate attribute objects and table. */
>>>+ xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>>>+ if (!xe_attr)
>>>+ goto err_alloc;
>>>+
>>>+ pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>>>+ if (!pmu_attr)
>>>+ goto err_alloc;
>>>+
>>>+ /* Max one pointer of each attribute type plus a termination entry. */
>>>+ attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>>>+ if (!attr)
>>>+ goto err_alloc;
>>>+
>>>+ xe_iter = xe_attr;
>>>+ pmu_iter = pmu_attr;
>>>+ attr_iter = attr;
>>>+
>>>+ for_each_gt(gt, xe, j) {
>>>+ for (i = 0; i < ARRAY_SIZE(events); i++) {
>>>+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>>+ char *str;
>>>+
>>>+ if (config_status(xe, config))
>>>+ continue;
>>>+
>>>+ str = kasprintf(GFP_KERNEL, "%s-gt%u",
>>>+ events[i].name, j);
>>>+ if (!str)
>>>+ goto err;
>>>+
>>>+ *attr_iter++ = &xe_iter->attr.attr;
>>>+ xe_iter = add_xe_attr(xe_iter, str, config);
>>>+
>>>+ if (events[i].unit) {
>>>+ str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>>>+ events[i].name, j);
>>>+ if (!str)
>>>+ goto err;
>>>+
>>>+ *attr_iter++ = &pmu_iter->attr.attr;
>>>+ pmu_iter = add_pmu_attr(pmu_iter, str,
>>>+ events[i].unit);
>>>+ }
>>>+ }
>>>+ }
>>>+
>>>+ pmu->xe_attr = xe_attr;
>>>+ pmu->pmu_attr = pmu_attr;
>>>+
>>>+ return attr;
>>>+
>>>+err:
>>>+ for (attr_iter = attr; *attr_iter; attr_iter++)
>>>+ kfree((*attr_iter)->name);
>>>+
>>>+err_alloc:
>>>+ kfree(attr);
>>>+ kfree(xe_attr);
>>>+ kfree(pmu_attr);
>>>+
>>>+ return NULL;
>>>+}
>>>+
>>>+static void free_event_attributes(struct xe_pmu *pmu)
>>>+{
>>>+ struct attribute **attr_iter = pmu->events_attr_group.attrs;
>>>+
>>>+ for (; *attr_iter; attr_iter++)
>>>+ kfree((*attr_iter)->name);
>>>+
>>>+ kfree(pmu->events_attr_group.attrs);
>>>+ kfree(pmu->xe_attr);
>>>+ kfree(pmu->pmu_attr);
>>>+
>>>+ pmu->events_attr_group.attrs = NULL;
>>>+ pmu->xe_attr = NULL;
>>>+ pmu->pmu_attr = NULL;
>>>+}
>>>+
>>>+static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>>>+{
>>>+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>>+
>>>+ /* Select the first online CPU as a designated reader. */
>>>+ if (cpumask_empty(&xe_pmu_cpumask))
>>>+ cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>>>+
>>>+ return 0;
>>>+}
>>>+
>>>+static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>>>+{
>>>+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>>+ unsigned int target = xe_pmu_target_cpu;
>>>+
>>>+ /*
>>>+ * Unregistering an instance generates a CPU offline event which we must
>>>+ * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>>>+ */
>>>+ if (pmu->closed)
>>>+ return 0;
>>>+
>>>+ if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>>>+ target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>>>+
>>>+ /* Migrate events if there is a valid target */
>>>+ if (target < nr_cpu_ids) {
>>>+ cpumask_set_cpu(target, &xe_pmu_cpumask);
>>>+ xe_pmu_target_cpu = target;
>>>+ }
>>>+ }
>>>+
>>>+ if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>>>+ perf_pmu_migrate_context(&pmu->base, cpu, target);
>>>+ pmu->cpuhp.cpu = target;
>>>+ }
>>>+
>>>+ return 0;
>>>+}
>>>+
>>>+static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>>>+
>>>+int xe_pmu_init(void)
>>>+{
>>>+ int ret;
>>>+
>>>+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>>>+ "perf/x86/intel/xe:online",
>>>+ xe_pmu_cpu_online,
>>>+ xe_pmu_cpu_offline);
>>>+ if (ret < 0)
>>>+ pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>>>+ ret);
>>>+ else
>>>+ cpuhp_slot = ret;
>>>+
>>>+ return 0;
>>>+}
>>>+
>>>+void xe_pmu_exit(void)
>>>+{
>>>+ if (cpuhp_slot != CPUHP_INVALID)
>>>+ cpuhp_remove_multi_state(cpuhp_slot);
>>>+}
>>>+
>>>+static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>>>+{
>>>+ if (cpuhp_slot == CPUHP_INVALID)
>>>+ return -EINVAL;
>>>+
>>>+ return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>>>+}
>>>+
>>>+static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>>>+{
>>>+ cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>>>+}
>>>+
>>>+void xe_pmu_suspend(struct xe_gt *gt)
>>>+{
>>>+ engine_group_busyness_store(gt);
>>>+}
>>>+
>>>+static void xe_pmu_unregister(void *arg)
>>>+{
>>>+ struct xe_pmu *pmu = arg;
>>>+
>>>+ if (!pmu->base.event_init)
>>>+ return;
>>>+
>>>+ /*
>>>+ * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
>>>+ * ensures all currently executing ones will have exited before we
>>>+ * proceed with unregistration.
>>>+ */
>>>+ pmu->closed = true;
>>>+ synchronize_rcu();
>>>+
>>>+ xe_pmu_unregister_cpuhp_state(pmu);
>>>+
>>>+ perf_pmu_unregister(&pmu->base);
>>>+ pmu->base.event_init = NULL;
>>>+ kfree(pmu->base.attr_groups);
>>>+ kfree(pmu->name);
>>>+ free_event_attributes(pmu);
>>>+}
>>>+
>>>+void xe_pmu_register(struct xe_pmu *pmu)
>>>+{
>>>+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>>+ const struct attribute_group *attr_groups[] = {
>>>+ &pmu->events_attr_group,
>>>+ &xe_pmu_cpumask_attr_group,
>>>+ NULL
>>>+ };
>>>+
>>>+ int ret = -ENOMEM;
>>>+
>>>+ spin_lock_init(&pmu->lock);
>>>+ pmu->cpuhp.cpu = -1;
>>>+
>>>+ pmu->name = kasprintf(GFP_KERNEL,
>>>+ "xe_%s",
>>>+ dev_name(xe->drm.dev));
>>>+ if (pmu->name)
>>>+ /* tools/perf reserves colons as special. */
>>>+ strreplace((char *)pmu->name, ':', '_');
>>>+
>>>+ if (!pmu->name)
>>>+ goto err;
>>>+
>>>+ pmu->events_attr_group.name = "events";
>>>+ pmu->events_attr_group.attrs = create_event_attributes(pmu);
>>>+ if (!pmu->events_attr_group.attrs)
>>>+ goto err_name;
>>>+
>>>+ pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>>>+ GFP_KERNEL);
>>>+ if (!pmu->base.attr_groups)
>>>+ goto err_attr;
>>>+
>>>+ pmu->base.module = THIS_MODULE;
>>>+ pmu->base.task_ctx_nr = perf_invalid_context;
>>>+ pmu->base.event_init = xe_pmu_event_init;
>>>+ pmu->base.add = xe_pmu_event_add;
>>>+ pmu->base.del = xe_pmu_event_del;
>>>+ pmu->base.start = xe_pmu_event_start;
>>>+ pmu->base.stop = xe_pmu_event_stop;
>>>+ pmu->base.read = xe_pmu_event_read;
>>>+ pmu->base.event_idx = xe_pmu_event_event_idx;
>>>+
>>>+ ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>>>+ if (ret)
>>>+ goto err_groups;
>>>+
>>>+ ret = xe_pmu_register_cpuhp_state(pmu);
>>>+ if (ret)
>>>+ goto err_unreg;
>>>+
>>>+ ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>>>+ if (ret)
>>>+ goto err_cpuhp;
>>>+
>>>+ return;
>>>+
>>>+err_cpuhp:
>>>+ xe_pmu_unregister_cpuhp_state(pmu);
>>>+err_unreg:
>>>+ perf_pmu_unregister(&pmu->base);
>>>+err_groups:
>>>+ kfree(pmu->base.attr_groups);
>>>+err_attr:
>>>+ pmu->base.event_init = NULL;
>>>+ free_event_attributes(pmu);
>>>+err_name:
>>>+ kfree(pmu->name);
>>>+err:
>>>+ drm_notice(&xe->drm, "Failed to register PMU!\n");
>>>+}
>>>diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>>>new file mode 100644
>>>index 000000000000..8afa256f9dac
>>>--- /dev/null
>>>+++ b/drivers/gpu/drm/xe/xe_pmu.h
>>>@@ -0,0 +1,26 @@
>>>+/* SPDX-License-Identifier: MIT */
>>>+/*
>>>+ * Copyright © 2024 Intel Corporation
>>>+ */
>>>+
>>>+#ifndef _XE_PMU_H_
>>>+#define _XE_PMU_H_
>>>+
>>>+#include "xe_pmu_types.h"
>>>+
>>>+struct xe_gt;
>>>+
>>>+#if IS_ENABLED(CONFIG_PERF_EVENTS)
>>>+int xe_pmu_init(void);
>>>+void xe_pmu_exit(void);
>>>+void xe_pmu_register(struct xe_pmu *pmu);
>>>+void xe_pmu_suspend(struct xe_gt *gt);
>>>+#else
>>>+static inline int xe_pmu_init(void) { return 0; }
>>>+static inline void xe_pmu_exit(void) {}
>>>+static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>>>+static inline void xe_pmu_suspend(struct xe_gt *gt) {}
>>>+#endif
>>>+
>>>+#endif
>>>+
>>>diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
>>>new file mode 100644
>>>index 000000000000..e86e8d7e0356
>>>--- /dev/null
>>>+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>>>@@ -0,0 +1,67 @@
>>>+/* SPDX-License-Identifier: MIT */
>>>+/*
>>>+ * Copyright © 2024 Intel Corporation
>>>+ */
>>>+
>>>+#ifndef _XE_PMU_TYPES_H_
>>>+#define _XE_PMU_TYPES_H_
>>>+
>>>+#include <linux/perf_event.h>
>>>+#include <linux/spinlock_types.h>
>>>+#include <uapi/drm/xe_drm.h>
>>>+
>>>+enum {
>>>+ __XE_SAMPLE_RENDER_GROUP_BUSY,
>>>+ __XE_SAMPLE_COPY_GROUP_BUSY,
>>>+ __XE_SAMPLE_MEDIA_GROUP_BUSY,
>>>+ __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
>>>+ __XE_NUM_PMU_SAMPLERS
>>>+};
>>>+
>>>+#define XE_PMU_MAX_GT 2
>>>+
>>>+struct xe_pmu {
>>>+ /**
>>>+ * @cpuhp: Struct used for CPU hotplug handling.
>>>+ */
>>>+ struct {
>>>+ struct hlist_node node;
>>>+ unsigned int cpu;
>>>+ } cpuhp;
>>>+ /**
>>>+ * @base: PMU base.
>>>+ */
>>>+ struct pmu base;
>>>+ /**
>>>+ * @closed: xe is unregistering.
>>>+ */
>>>+ bool closed;
>>>+ /**
>>>+ * @name: Name as registered with perf core.
>>>+ */
>>>+ const char *name;
>>>+ /**
>>>+ * @lock: Lock protecting enable mask and ref count handling.
>>>+ */
>>>+ spinlock_t lock;
>>>+ /**
>>>+ * @sample: Current and previous (raw) counters.
>>>+ *
>>>+ * These counters are updated when the device is awake.
>>>+ */
>>>+ u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>>>+ /**
>>>+ * @events_attr_group: Device events attribute group.
>>>+ */
>>>+ struct attribute_group events_attr_group;
>>>+ /**
>>>+ * @xe_attr: Memory block holding device attributes.
>>>+ */
>>>+ void *xe_attr;
>>>+ /**
>>>+ * @pmu_attr: Memory block holding device attributes.
>>>+ */
>>>+ void *pmu_attr;
>>>+};
>>>+
>>>+#endif
>>>diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>>>index d7b0903c22b2..07ca545354f7 100644
>>>--- a/include/uapi/drm/xe_drm.h
>>>+++ b/include/uapi/drm/xe_drm.h
>>>@@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
>>> __u64 reserved[2];
>>>};
>>>
>>>+/**
>>>+ * DOC: XE PMU event config IDs
>>>+ *
>>>+ * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in xe_drm.h
>>>+ * in 'struct perf_event_attr' as part of perf_event_open syscall to read a
>>>+ * particular event.
>>>+ *
>>>+ * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
>>>+ *
>>>+ * .. code-block:: C
>>>+ *
>>>+ * struct perf_event_attr attr;
>>>+ * long long count;
>>>+ * int cpu = 0;
>>>+ * int fd;
>>>+ *
>>>+ * memset(&attr, 0, sizeof(struct perf_event_attr));
>>>+ * attr.type = type; // eg: /sys/bus/event_source/devices/xe_0000_56_00.0/type
>>>+ * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
>>>+ * attr.use_clockid = 1;
>>>+ * attr.clockid = CLOCK_MONOTONIC;
>>>+ * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
>>>+ *
>>>+ * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
>>>+ */
>>>+
>>>+/*
>>>+ * Top bits of every counter are GT id.
>>>+ */
>>>+#define __XE_PMU_GT_SHIFT (56)
>>>+
>>>+#define ___XE_PMU_OTHER(gt, x) \
>>>+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>>>+
>>>+#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
>>>+#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
>>>+#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
>>>+#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
>>
>>+ Lucas for inputs
>>
>>We should align this to the interface planned for other PMU busyness
>>counters as well as how we do PCEU. i.e.
>>
>>1) counters are in ticks
>>2) total time in ticks is also exported to the user.
>>
>>For 1), I would just append TICKS to the counter names and drop the
>
>this uses perf and as such I believe we should use the terms used by
>perf.
>
>$ sudo perf stat sleep 1
>
> Performance counter stats for 'sleep 1':
>
> 0.91 msec task-clock # 0.001 CPUs utilized
> 1 context-switches # 1.096 K/sec
> 0 cpu-migrations # 0.000 /sec
> 72 page-faults # 78.924 K/sec
>------> 2,033,156 cycles # 2.229 GHz
> 1,560,992 instructions # 0.77 insn per cycle
> 290,814 branches # 318.779 M/sec
> 10,449 branch-misses # 3.59% of all branches
>
> 1.001580466 seconds time elapsed
>
> 0.000000000 seconds user
> 0.001545000 seconds sys
>
>so... s/ticks/cycles/
>
>I think I said that before, but what's up with all these "group" in the
>names? It's confusing since apparently group and engine class are mixed.
These are counters defined in the HW and indicate busyness of a group of
engines (spanning multiple classes) rather than a single engine. The
free running counters are directly read from HW.
Single engine busyness is a different API and wip.
Regards,
Umesh
>
>We are also missing proper kernel-doc in xe_pmu.c
>
>Lucas De Marchi
>
>>conversion to _ns in __engine_group_busyness_read(). Also, drop the
>>patch that adds this conversion helper.
>>
>>For 2) define a new counter - total active ticks that would return
>>the 'CPU' timestamp converted to gpu ticks. The reason I am
>>insisting on CPU timestamp here is because we want to have a time
>>base that is ticking even when the GPU is idle.
>>
>>Regards,
>>Umesh
>>
>>>+
>>>#if defined(__cplusplus)
>>>}
>>>#endif
>>>--
>>>2.40.0
>>>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-28 16:52 ` Umesh Nerlige Ramappa
@ 2024-06-28 18:24 ` Lucas De Marchi
2024-06-28 18:49 ` Umesh Nerlige Ramappa
0 siblings, 1 reply; 32+ messages in thread
From: Lucas De Marchi @ 2024-06-28 18:24 UTC (permalink / raw)
To: Umesh Nerlige Ramappa
Cc: Riana Tauro, intel-xe, anshuman.gupta, ashutosh.dixit,
aravind.iddamsetty, rodrigo.vivi, krishnaiah.bommu
On Fri, Jun 28, 2024 at 09:52:36AM GMT, Umesh Nerlige Ramappa wrote:
>On Fri, Jun 28, 2024 at 10:55:06AM -0500, Lucas De Marchi wrote:
>>On Thu, Jun 20, 2024 at 12:52:05PM GMT, Umesh Nerlige Ramappa wrote:
>>>On Thu, Jun 13, 2024 at 03:34:11PM +0530, Riana Tauro wrote:
>>>>From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>>>
>>>>There are a set of engine group busyness counters provided by HW which are
>>>>perfect fit to be exposed via PMU perf events.
>>>>
>>>>BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>>>
>>>>events can be listed using:
>>>>perf list
>>>>xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>>>>xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>>>>xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>>>>xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>>>
>>>>and can be read using:
>>>>
>>>>perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
>>>> time counts unit events
>>>> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>> 10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>
>>>>The pmu base implementation is taken from i915.
>>>>
>>>>v2:
>>>>Store last known value when device is awake return that while the GT is
>>>>suspended and then update the driver copy when read during awake.
>>>>
>>>>v3:
>>>>1. drop init_samples, as storing counters before going to suspend should
>>>>be sufficient.
>>>>2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>>>>dropped helpers to store and read samples.
>>>>3. use xe_device_mem_access_get_if_ongoing to check if device is active
>>>>before reading the OA registers.
>>>>4. dropped format attr as no longer needed
>>>>5. introduce xe_pmu_suspend to call engine_group_busyness_store
>>>>6. few other nits.
>>>>
>>>>v4: minor nits.
>>>>
>>>>v5: take forcewake when accessing the OAG registers
>>>>
>>>>v6:
>>>>1. drop engine_busyness_sample_type
>>>>2. update UAPI documentation
>>>>
>>>>v7:
>>>>1. update UAPI documentation
>>>>2. drop MEDIA_GT specific change for media busyness counter.
>>>>
>>>>v8:
>>>>1. rebase
>>>>2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>>>>3. remove interrupts pmu event
>>>>
>>>>v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>>>>
>>>>Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>>>Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>>>Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>>>Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>>>Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>>>Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
>>>>Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>>>>---
>>>>drivers/gpu/drm/xe/Makefile | 2 +
>>>>drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
>>>>drivers/gpu/drm/xe/xe_device.c | 2 +
>>>>drivers/gpu/drm/xe/xe_device_types.h | 4 +
>>>>drivers/gpu/drm/xe/xe_gt.c | 2 +
>>>>drivers/gpu/drm/xe/xe_module.c | 5 +
>>>>drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
>>>>drivers/gpu/drm/xe/xe_pmu.h | 26 ++
>>>>drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
>>>>include/uapi/drm/xe_drm.h | 39 ++
>>>>10 files changed, 783 insertions(+)
>>>>create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
>>>>create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
>>>>create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>>>>
>>>>diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>>>>index cbf961b90237..83bf1e07669b 100644
>>>>--- a/drivers/gpu/drm/xe/Makefile
>>>>+++ b/drivers/gpu/drm/xe/Makefile
>>>>@@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
>>>> i915-display/skl_universal_plane.o \
>>>> i915-display/skl_watermark.o
>>>>
>>>>+xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>>>>+
>>>>ifeq ($(CONFIG_ACPI),y)
>>>> xe-$(CONFIG_DRM_XE_DISPLAY) += \
>>>> i915-display/intel_acpi.o \
>>>>diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>>index 47c26c37608d..22821dcd4e1b 100644
>>>>--- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>>+++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>>@@ -390,6 +390,11 @@
>>>>#define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
>>>>#define GLOBAL_INVALIDATION_MODE REG_BIT(2)
>>>>
>>>>+#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
>>>>+#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
>>>>+#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
>>>>+#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
>>>>+
>>>>#define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
>>>>#define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
>>>>
>>>>diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
>>>>index 64691a56d59c..bb00c8c9ec9b 100644
>>>>--- a/drivers/gpu/drm/xe/xe_device.c
>>>>+++ b/drivers/gpu/drm/xe/xe_device.c
>>>>@@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
>>>>
>>>> xe_hwmon_register(xe);
>>>>
>>>>+ xe_pmu_register(&xe->pmu);
>>>>+
>>>> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>>>>
>>>>err_fini_display:
>>>>diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
>>>>index 52bc461171d5..a5dba7325cf1 100644
>>>>--- a/drivers/gpu/drm/xe/xe_device_types.h
>>>>+++ b/drivers/gpu/drm/xe/xe_device_types.h
>>>>@@ -18,6 +18,7 @@
>>>>#include "xe_lmtt_types.h"
>>>>#include "xe_memirq_types.h"
>>>>#include "xe_platform_types.h"
>>>>+#include "xe_pmu.h"
>>>>#include "xe_pt_types.h"
>>>>#include "xe_sriov_types.h"
>>>>#include "xe_step_types.h"
>>>>@@ -473,6 +474,9 @@ struct xe_device {
>>>> int mode;
>>>> } wedged;
>>>>
>>>>+ /** @pmu: performance monitoring unit */
>>>>+ struct xe_pmu pmu;
>>>>+
>>>> /* private: */
>>>>
>>>>#if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
>>>>diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
>>>>index 57d84751e160..477d0ae5f230 100644
>>>>--- a/drivers/gpu/drm/xe/xe_gt.c
>>>>+++ b/drivers/gpu/drm/xe/xe_gt.c
>>>>@@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
>>>> if (err)
>>>> goto err_msg;
>>>>
>>>>+ xe_pmu_suspend(gt);
>>>>+
>>>> err = xe_uc_suspend(>->uc);
>>>> if (err)
>>>> goto err_force_wake;
>>>>diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
>>>>index 3edeb30d5ccb..26f814f97fc2 100644
>>>>--- a/drivers/gpu/drm/xe/xe_module.c
>>>>+++ b/drivers/gpu/drm/xe/xe_module.c
>>>>@@ -11,6 +11,7 @@
>>>>#include "xe_drv.h"
>>>>#include "xe_hw_fence.h"
>>>>#include "xe_pci.h"
>>>>+#include "xe_pmu.h"
>>>>#include "xe_sched_job.h"
>>>>
>>>>struct xe_modparam xe_modparam = {
>>>>@@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
>>>> .init = xe_sched_job_module_init,
>>>> .exit = xe_sched_job_module_exit,
>>>> },
>>>>+ {
>>>>+ .init = xe_pmu_init,
>>>>+ .exit = xe_pmu_exit,
>>>>+ },
>>>> {
>>>> .init = xe_register_pci_driver,
>>>> .exit = xe_unregister_pci_driver,
>>>>diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>>>>new file mode 100644
>>>>index 000000000000..64960a358af2
>>>>--- /dev/null
>>>>+++ b/drivers/gpu/drm/xe/xe_pmu.c
>>>>@@ -0,0 +1,631 @@
>>>>+// SPDX-License-Identifier: MIT
>>>>+/*
>>>>+ * Copyright © 2024 Intel Corporation
>>>>+ */
>>>>+
>>>>+#include <drm/drm_drv.h>
>>>>+#include <drm/drm_managed.h>
>>>>+#include <drm/xe_drm.h>
>>>>+
>>>>+#include "regs/xe_gt_regs.h"
>>>>+#include "xe_device.h"
>>>>+#include "xe_force_wake.h"
>>>>+#include "xe_gt_clock.h"
>>>>+#include "xe_mmio.h"
>>>>+#include "xe_macros.h"
>>>>+#include "xe_pm.h"
>>>>+
>>>>+static cpumask_t xe_pmu_cpumask;
>>>>+static unsigned int xe_pmu_target_cpu = -1;
>>>>+
>>>>+static unsigned int config_gt_id(const u64 config)
>>>>+{
>>>>+ return config >> __XE_PMU_GT_SHIFT;
>>>>+}
>>>>+
>>>>+static u64 config_counter(const u64 config)
>>>>+{
>>>>+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>>>>+}
>>>>+
>>>>+static void xe_pmu_event_destroy(struct perf_event *event)
>>>>+{
>>>>+ struct xe_device *xe =
>>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>>+
>>>>+ drm_WARN_ON(&xe->drm, event->parent);
>>>>+
>>>>+ drm_dev_put(&xe->drm);
>>>>+}
>>>>+
>>>>+static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
>>>>+{
>>>>+ u64 val;
>>>>+
>>>>+ switch (sample_type) {
>>>>+ case __XE_SAMPLE_RENDER_GROUP_BUSY:
>>>>+ val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>>>>+ break;
>>>>+ case __XE_SAMPLE_COPY_GROUP_BUSY:
>>>>+ val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>>>>+ break;
>>>>+ case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>>>>+ val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>>>>+ break;
>>>>+ case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>>>>+ val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>>>>+ break;
>>>>+ default:
>>>>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>>>+ }
>>>>+
>>>>+ return xe_gt_clock_cycles_to_ns(gt, val * 16);
>>>>+}
>>>>+
>>>>+static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>>>>+{
>>>>+ int sample_type = config_counter(config);
>>>>+ const unsigned int gt_id = gt->info.id;
>>>>+ struct xe_device *xe = gt->tile->xe;
>>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>>+ unsigned long flags;
>>>>+ bool device_awake;
>>>>+ u64 val;
>>>>+
>>>>+ device_awake = xe_pm_runtime_get_if_active(xe);
>>>>+ if (device_awake) {
>>>>+ XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>>>>+ val = __engine_group_busyness_read(gt, sample_type);
>>>>+ XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>>>>+ xe_pm_runtime_put(xe);
>>>>+ }
>>>>+
>>>>+ spin_lock_irqsave(&pmu->lock, flags);
>>>>+
>>>>+ if (device_awake)
>>>>+ pmu->sample[gt_id][sample_type] = val;
>>>>+ else
>>>>+ val = pmu->sample[gt_id][sample_type];
>>>>+
>>>>+ spin_unlock_irqrestore(&pmu->lock, flags);
>>>>+
>>>>+ return val;
>>>>+}
>>>>+
>>>>+static void engine_group_busyness_store(struct xe_gt *gt)
>>>>+{
>>>>+ struct xe_pmu *pmu = >->tile->xe->pmu;
>>>>+ unsigned int gt_id = gt->info.id;
>>>>+ unsigned long flags;
>>>>+ int i;
>>>>+
>>>>+ spin_lock_irqsave(&pmu->lock, flags);
>>>>+
>>>>+ for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>>>>+ pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>>>>+
>>>>+ spin_unlock_irqrestore(&pmu->lock, flags);
>>>>+}
>>>>+
>>>>+static int
>>>>+config_status(struct xe_device *xe, u64 config)
>>>>+{
>>>>+ unsigned int gt_id = config_gt_id(config);
>>>>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>>>+
>>>>+ if (gt_id >= XE_PMU_MAX_GT)
>>>>+ return -ENOENT;
>>>>+
>>>>+ switch (config_counter(config)) {
>>>>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>>>>+ case XE_PMU_COPY_GROUP_BUSY(0):
>>>>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>>>+ if (gt->info.type == XE_GT_TYPE_MEDIA)
>>>>+ return -ENOENT;
>>>>+ break;
>>>>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>>>>+ if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
>>>>+ return -ENOENT;
>>>>+ break;
>>>>+ default:
>>>>+ return -ENOENT;
>>>>+ }
>>>>+
>>>>+ return 0;
>>>>+}
>>>>+
>>>>+static int xe_pmu_event_init(struct perf_event *event)
>>>>+{
>>>>+ struct xe_device *xe =
>>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>>+ int ret;
>>>>+
>>>>+ if (pmu->closed)
>>>>+ return -ENODEV;
>>>>+
>>>>+ if (event->attr.type != event->pmu->type)
>>>>+ return -ENOENT;
>>>>+
>>>>+ /* unsupported modes and filters */
>>>>+ if (event->attr.sample_period) /* no sampling */
>>>>+ return -EINVAL;
>>>>+
>>>>+ if (has_branch_stack(event))
>>>>+ return -EOPNOTSUPP;
>>>>+
>>>>+ if (event->cpu < 0)
>>>>+ return -EINVAL;
>>>>+
>>>>+ /* only allow running on one cpu at a time */
>>>>+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>>>>+ return -EINVAL;
>>>>+
>>>>+ ret = config_status(xe, event->attr.config);
>>>>+ if (ret)
>>>>+ return ret;
>>>>+
>>>>+ if (!event->parent) {
>>>>+ drm_dev_get(&xe->drm);
>>>>+ event->destroy = xe_pmu_event_destroy;
>>>>+ }
>>>>+
>>>>+ return 0;
>>>>+}
>>>>+
>>>>+static u64 __xe_pmu_event_read(struct perf_event *event)
>>>>+{
>>>>+ struct xe_device *xe =
>>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>>+ const unsigned int gt_id = config_gt_id(event->attr.config);
>>>>+ const u64 config = event->attr.config;
>>>>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>>>+ u64 val;
>>>>+
>>>>+ switch (config_counter(config)) {
>>>>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>>>>+ case XE_PMU_COPY_GROUP_BUSY(0):
>>>>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>>>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>>>>+ val = engine_group_busyness_read(gt, config);
>>>>+ break;
>>>>+ default:
>>>>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>>>+ }
>>>>+
>>>>+ return val;
>>>>+}
>>>>+
>>>>+static void xe_pmu_event_read(struct perf_event *event)
>>>>+{
>>>>+ struct xe_device *xe =
>>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>>+ struct hw_perf_event *hwc = &event->hw;
>>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>>+ u64 prev, new;
>>>>+
>>>>+ if (pmu->closed) {
>>>>+ event->hw.state = PERF_HES_STOPPED;
>>>>+ return;
>>>>+ }
>>>>+again:
>>>>+ prev = local64_read(&hwc->prev_count);
>>>>+ new = __xe_pmu_event_read(event);
>>>>+
>>>>+ if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>>>>+ goto again;
>>>>+
>>>>+ local64_add(new - prev, &event->count);
>>>>+}
>>>>+
>>>>+static void xe_pmu_enable(struct perf_event *event)
>>>>+{
>>>>+ /*
>>>>+ * Store the current counter value so we can report the correct delta
>>>>+ * for all listeners. Even when the event was already enabled and has
>>>>+ * an existing non-zero value.
>>>>+ */
>>>>+ local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>>>>+}
>>>>+
>>>>+static void xe_pmu_event_start(struct perf_event *event, int flags)
>>>>+{
>>>>+ struct xe_device *xe =
>>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>>+
>>>>+ if (pmu->closed)
>>>>+ return;
>>>>+
>>>>+ xe_pmu_enable(event);
>>>>+ event->hw.state = 0;
>>>>+}
>>>>+
>>>>+static void xe_pmu_event_stop(struct perf_event *event, int flags)
>>>>+{
>>>>+ if (flags & PERF_EF_UPDATE)
>>>>+ xe_pmu_event_read(event);
>>>>+
>>>>+ event->hw.state = PERF_HES_STOPPED;
>>>>+}
>>>>+
>>>>+static int xe_pmu_event_add(struct perf_event *event, int flags)
>>>>+{
>>>>+ struct xe_device *xe =
>>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>>+
>>>>+ if (pmu->closed)
>>>>+ return -ENODEV;
>>>>+
>>>>+ if (flags & PERF_EF_START)
>>>>+ xe_pmu_event_start(event, flags);
>>>>+
>>>>+ return 0;
>>>>+}
>>>>+
>>>>+static void xe_pmu_event_del(struct perf_event *event, int flags)
>>>>+{
>>>>+ xe_pmu_event_stop(event, PERF_EF_UPDATE);
>>>>+}
>>>>+
>>>>+static int xe_pmu_event_event_idx(struct perf_event *event)
>>>>+{
>>>>+ return 0;
>>>>+}
>>>>+
>>>>+struct xe_ext_attribute {
>>>>+ struct device_attribute attr;
>>>>+ unsigned long val;
>>>>+};
>>>>+
>>>>+static ssize_t xe_pmu_event_show(struct device *dev,
>>>>+ struct device_attribute *attr, char *buf)
>>>>+{
>>>>+ struct xe_ext_attribute *eattr;
>>>>+
>>>>+ eattr = container_of(attr, struct xe_ext_attribute, attr);
>>>>+ return sprintf(buf, "config=0x%lx\n", eattr->val);
>>>>+}
>>>>+
>>>>+static ssize_t cpumask_show(struct device *dev,
>>>>+ struct device_attribute *attr, char *buf)
>>>>+{
>>>>+ return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>>>>+}
>>>>+
>>>>+static DEVICE_ATTR_RO(cpumask);
>>>>+
>>>>+static struct attribute *xe_cpumask_attrs[] = {
>>>>+ &dev_attr_cpumask.attr,
>>>>+ NULL,
>>>>+};
>>>>+
>>>>+static const struct attribute_group xe_pmu_cpumask_attr_group = {
>>>>+ .attrs = xe_cpumask_attrs,
>>>>+};
>>>>+
>>>>+#define __event(__counter, __name, __unit) \
>>>>+{ \
>>>>+ .counter = (__counter), \
>>>>+ .name = (__name), \
>>>>+ .unit = (__unit), \
>>>>+}
>>>>+
>>>>+static struct xe_ext_attribute *
>>>>+add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>>>>+{
>>>>+ sysfs_attr_init(&attr->attr.attr);
>>>>+ attr->attr.attr.name = name;
>>>>+ attr->attr.attr.mode = 0444;
>>>>+ attr->attr.show = xe_pmu_event_show;
>>>>+ attr->val = config;
>>>>+
>>>>+ return ++attr;
>>>>+}
>>>>+
>>>>+static struct perf_pmu_events_attr *
>>>>+add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>>>>+ const char *str)
>>>>+{
>>>>+ sysfs_attr_init(&attr->attr.attr);
>>>>+ attr->attr.attr.name = name;
>>>>+ attr->attr.attr.mode = 0444;
>>>>+ attr->attr.show = perf_event_sysfs_show;
>>>>+ attr->event_str = str;
>>>>+
>>>>+ return ++attr;
>>>>+}
>>>>+
>>>>+static struct attribute **
>>>>+create_event_attributes(struct xe_pmu *pmu)
>>>>+{
>>>>+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>>>+ static const struct {
>>>>+ unsigned int counter;
>>>>+ const char *name;
>>>>+ const char *unit;
>>>>+ } events[] = {
>>>>+ __event(0, "render-group-busy", "ns"),
>>>>+ __event(1, "copy-group-busy", "ns"),
>>>>+ __event(2, "media-group-busy", "ns"),
>>>>+ __event(3, "any-engine-group-busy", "ns"),
>>>>+ };
>>>>+
>>>>+ struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>>>>+ struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>>>>+ struct attribute **attr = NULL, **attr_iter;
>>>>+ unsigned int count = 0;
>>>>+ unsigned int i, j;
>>>>+ struct xe_gt *gt;
>>>>+
>>>>+ /* Count how many counters we will be exposing. */
>>>>+ for_each_gt(gt, xe, j) {
>>>>+ for (i = 0; i < ARRAY_SIZE(events); i++) {
>>>>+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>>>+
>>>>+ if (!config_status(xe, config))
>>>>+ count++;
>>>>+ }
>>>>+ }
>>>>+
>>>>+ /* Allocate attribute objects and table. */
>>>>+ xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>>>>+ if (!xe_attr)
>>>>+ goto err_alloc;
>>>>+
>>>>+ pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>>>>+ if (!pmu_attr)
>>>>+ goto err_alloc;
>>>>+
>>>>+ /* Max one pointer of each attribute type plus a termination entry. */
>>>>+ attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>>>>+ if (!attr)
>>>>+ goto err_alloc;
>>>>+
>>>>+ xe_iter = xe_attr;
>>>>+ pmu_iter = pmu_attr;
>>>>+ attr_iter = attr;
>>>>+
>>>>+ for_each_gt(gt, xe, j) {
>>>>+ for (i = 0; i < ARRAY_SIZE(events); i++) {
>>>>+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>>>+ char *str;
>>>>+
>>>>+ if (config_status(xe, config))
>>>>+ continue;
>>>>+
>>>>+ str = kasprintf(GFP_KERNEL, "%s-gt%u",
>>>>+ events[i].name, j);
>>>>+ if (!str)
>>>>+ goto err;
>>>>+
>>>>+ *attr_iter++ = &xe_iter->attr.attr;
>>>>+ xe_iter = add_xe_attr(xe_iter, str, config);
>>>>+
>>>>+ if (events[i].unit) {
>>>>+ str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>>>>+ events[i].name, j);
>>>>+ if (!str)
>>>>+ goto err;
>>>>+
>>>>+ *attr_iter++ = &pmu_iter->attr.attr;
>>>>+ pmu_iter = add_pmu_attr(pmu_iter, str,
>>>>+ events[i].unit);
>>>>+ }
>>>>+ }
>>>>+ }
>>>>+
>>>>+ pmu->xe_attr = xe_attr;
>>>>+ pmu->pmu_attr = pmu_attr;
>>>>+
>>>>+ return attr;
>>>>+
>>>>+err:
>>>>+ for (attr_iter = attr; *attr_iter; attr_iter++)
>>>>+ kfree((*attr_iter)->name);
>>>>+
>>>>+err_alloc:
>>>>+ kfree(attr);
>>>>+ kfree(xe_attr);
>>>>+ kfree(pmu_attr);
>>>>+
>>>>+ return NULL;
>>>>+}
>>>>+
>>>>+static void free_event_attributes(struct xe_pmu *pmu)
>>>>+{
>>>>+ struct attribute **attr_iter = pmu->events_attr_group.attrs;
>>>>+
>>>>+ for (; *attr_iter; attr_iter++)
>>>>+ kfree((*attr_iter)->name);
>>>>+
>>>>+ kfree(pmu->events_attr_group.attrs);
>>>>+ kfree(pmu->xe_attr);
>>>>+ kfree(pmu->pmu_attr);
>>>>+
>>>>+ pmu->events_attr_group.attrs = NULL;
>>>>+ pmu->xe_attr = NULL;
>>>>+ pmu->pmu_attr = NULL;
>>>>+}
>>>>+
>>>>+static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>>>>+{
>>>>+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>>>+
>>>>+ /* Select the first online CPU as a designated reader. */
>>>>+ if (cpumask_empty(&xe_pmu_cpumask))
>>>>+ cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>>>>+
>>>>+ return 0;
>>>>+}
>>>>+
>>>>+static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>>>>+{
>>>>+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>>>+ unsigned int target = xe_pmu_target_cpu;
>>>>+
>>>>+ /*
>>>>+ * Unregistering an instance generates a CPU offline event which we must
>>>>+ * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>>>>+ */
>>>>+ if (pmu->closed)
>>>>+ return 0;
>>>>+
>>>>+ if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>>>>+ target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>>>>+
>>>>+ /* Migrate events if there is a valid target */
>>>>+ if (target < nr_cpu_ids) {
>>>>+ cpumask_set_cpu(target, &xe_pmu_cpumask);
>>>>+ xe_pmu_target_cpu = target;
>>>>+ }
>>>>+ }
>>>>+
>>>>+ if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>>>>+ perf_pmu_migrate_context(&pmu->base, cpu, target);
>>>>+ pmu->cpuhp.cpu = target;
>>>>+ }
>>>>+
>>>>+ return 0;
>>>>+}
>>>>+
>>>>+static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>>>>+
>>>>+int xe_pmu_init(void)
>>>>+{
>>>>+ int ret;
>>>>+
>>>>+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>>>>+ "perf/x86/intel/xe:online",
>>>>+ xe_pmu_cpu_online,
>>>>+ xe_pmu_cpu_offline);
>>>>+ if (ret < 0)
>>>>+ pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>>>>+ ret);
>>>>+ else
>>>>+ cpuhp_slot = ret;
>>>>+
>>>>+ return 0;
>>>>+}
>>>>+
>>>>+void xe_pmu_exit(void)
>>>>+{
>>>>+ if (cpuhp_slot != CPUHP_INVALID)
>>>>+ cpuhp_remove_multi_state(cpuhp_slot);
>>>>+}
>>>>+
>>>>+static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>>>>+{
>>>>+ if (cpuhp_slot == CPUHP_INVALID)
>>>>+ return -EINVAL;
>>>>+
>>>>+ return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>>>>+}
>>>>+
>>>>+static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>>>>+{
>>>>+ cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>>>>+}
>>>>+
>>>>+void xe_pmu_suspend(struct xe_gt *gt)
>>>>+{
>>>>+ engine_group_busyness_store(gt);
>>>>+}
>>>>+
>>>>+static void xe_pmu_unregister(void *arg)
>>>>+{
>>>>+ struct xe_pmu *pmu = arg;
>>>>+
>>>>+ if (!pmu->base.event_init)
>>>>+ return;
>>>>+
>>>>+ /*
>>>>+ * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
>>>>+ * ensures all currently executing ones will have exited before we
>>>>+ * proceed with unregistration.
>>>>+ */
>>>>+ pmu->closed = true;
>>>>+ synchronize_rcu();
>>>>+
>>>>+ xe_pmu_unregister_cpuhp_state(pmu);
>>>>+
>>>>+ perf_pmu_unregister(&pmu->base);
>>>>+ pmu->base.event_init = NULL;
>>>>+ kfree(pmu->base.attr_groups);
>>>>+ kfree(pmu->name);
>>>>+ free_event_attributes(pmu);
>>>>+}
>>>>+
>>>>+void xe_pmu_register(struct xe_pmu *pmu)
>>>>+{
>>>>+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>>>+ const struct attribute_group *attr_groups[] = {
>>>>+ &pmu->events_attr_group,
>>>>+ &xe_pmu_cpumask_attr_group,
>>>>+ NULL
>>>>+ };
>>>>+
>>>>+ int ret = -ENOMEM;
>>>>+
>>>>+ spin_lock_init(&pmu->lock);
>>>>+ pmu->cpuhp.cpu = -1;
>>>>+
>>>>+ pmu->name = kasprintf(GFP_KERNEL,
>>>>+ "xe_%s",
>>>>+ dev_name(xe->drm.dev));
>>>>+ if (pmu->name)
>>>>+ /* tools/perf reserves colons as special. */
>>>>+ strreplace((char *)pmu->name, ':', '_');
>>>>+
>>>>+ if (!pmu->name)
>>>>+ goto err;
>>>>+
>>>>+ pmu->events_attr_group.name = "events";
>>>>+ pmu->events_attr_group.attrs = create_event_attributes(pmu);
>>>>+ if (!pmu->events_attr_group.attrs)
>>>>+ goto err_name;
>>>>+
>>>>+ pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>>>>+ GFP_KERNEL);
>>>>+ if (!pmu->base.attr_groups)
>>>>+ goto err_attr;
>>>>+
>>>>+ pmu->base.module = THIS_MODULE;
>>>>+ pmu->base.task_ctx_nr = perf_invalid_context;
>>>>+ pmu->base.event_init = xe_pmu_event_init;
>>>>+ pmu->base.add = xe_pmu_event_add;
>>>>+ pmu->base.del = xe_pmu_event_del;
>>>>+ pmu->base.start = xe_pmu_event_start;
>>>>+ pmu->base.stop = xe_pmu_event_stop;
>>>>+ pmu->base.read = xe_pmu_event_read;
>>>>+ pmu->base.event_idx = xe_pmu_event_event_idx;
>>>>+
>>>>+ ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>>>>+ if (ret)
>>>>+ goto err_groups;
>>>>+
>>>>+ ret = xe_pmu_register_cpuhp_state(pmu);
>>>>+ if (ret)
>>>>+ goto err_unreg;
>>>>+
>>>>+ ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>>>>+ if (ret)
>>>>+ goto err_cpuhp;
>>>>+
>>>>+ return;
>>>>+
>>>>+err_cpuhp:
>>>>+ xe_pmu_unregister_cpuhp_state(pmu);
>>>>+err_unreg:
>>>>+ perf_pmu_unregister(&pmu->base);
>>>>+err_groups:
>>>>+ kfree(pmu->base.attr_groups);
>>>>+err_attr:
>>>>+ pmu->base.event_init = NULL;
>>>>+ free_event_attributes(pmu);
>>>>+err_name:
>>>>+ kfree(pmu->name);
>>>>+err:
>>>>+ drm_notice(&xe->drm, "Failed to register PMU!\n");
>>>>+}
>>>>diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>>>>new file mode 100644
>>>>index 000000000000..8afa256f9dac
>>>>--- /dev/null
>>>>+++ b/drivers/gpu/drm/xe/xe_pmu.h
>>>>@@ -0,0 +1,26 @@
>>>>+/* SPDX-License-Identifier: MIT */
>>>>+/*
>>>>+ * Copyright © 2024 Intel Corporation
>>>>+ */
>>>>+
>>>>+#ifndef _XE_PMU_H_
>>>>+#define _XE_PMU_H_
>>>>+
>>>>+#include "xe_pmu_types.h"
>>>>+
>>>>+struct xe_gt;
>>>>+
>>>>+#if IS_ENABLED(CONFIG_PERF_EVENTS)
>>>>+int xe_pmu_init(void);
>>>>+void xe_pmu_exit(void);
>>>>+void xe_pmu_register(struct xe_pmu *pmu);
>>>>+void xe_pmu_suspend(struct xe_gt *gt);
>>>>+#else
>>>>+static inline int xe_pmu_init(void) { return 0; }
>>>>+static inline void xe_pmu_exit(void) {}
>>>>+static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>>>>+static inline void xe_pmu_suspend(struct xe_gt *gt) {}
>>>>+#endif
>>>>+
>>>>+#endif
>>>>+
>>>>diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
>>>>new file mode 100644
>>>>index 000000000000..e86e8d7e0356
>>>>--- /dev/null
>>>>+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>>>>@@ -0,0 +1,67 @@
>>>>+/* SPDX-License-Identifier: MIT */
>>>>+/*
>>>>+ * Copyright © 2024 Intel Corporation
>>>>+ */
>>>>+
>>>>+#ifndef _XE_PMU_TYPES_H_
>>>>+#define _XE_PMU_TYPES_H_
>>>>+
>>>>+#include <linux/perf_event.h>
>>>>+#include <linux/spinlock_types.h>
>>>>+#include <uapi/drm/xe_drm.h>
>>>>+
>>>>+enum {
>>>>+ __XE_SAMPLE_RENDER_GROUP_BUSY,
>>>>+ __XE_SAMPLE_COPY_GROUP_BUSY,
>>>>+ __XE_SAMPLE_MEDIA_GROUP_BUSY,
>>>>+ __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
>>>>+ __XE_NUM_PMU_SAMPLERS
>>>>+};
>>>>+
>>>>+#define XE_PMU_MAX_GT 2
>>>>+
>>>>+struct xe_pmu {
>>>>+ /**
>>>>+ * @cpuhp: Struct used for CPU hotplug handling.
>>>>+ */
>>>>+ struct {
>>>>+ struct hlist_node node;
>>>>+ unsigned int cpu;
>>>>+ } cpuhp;
>>>>+ /**
>>>>+ * @base: PMU base.
>>>>+ */
>>>>+ struct pmu base;
>>>>+ /**
>>>>+ * @closed: xe is unregistering.
>>>>+ */
>>>>+ bool closed;
>>>>+ /**
>>>>+ * @name: Name as registered with perf core.
>>>>+ */
>>>>+ const char *name;
>>>>+ /**
>>>>+ * @lock: Lock protecting enable mask and ref count handling.
>>>>+ */
>>>>+ spinlock_t lock;
>>>>+ /**
>>>>+ * @sample: Current and previous (raw) counters.
>>>>+ *
>>>>+ * These counters are updated when the device is awake.
>>>>+ */
>>>>+ u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>>>>+ /**
>>>>+ * @events_attr_group: Device events attribute group.
>>>>+ */
>>>>+ struct attribute_group events_attr_group;
>>>>+ /**
>>>>+ * @xe_attr: Memory block holding device attributes.
>>>>+ */
>>>>+ void *xe_attr;
>>>>+ /**
>>>>+ * @pmu_attr: Memory block holding device attributes.
>>>>+ */
>>>>+ void *pmu_attr;
>>>>+};
>>>>+
>>>>+#endif
>>>>diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>>>>index d7b0903c22b2..07ca545354f7 100644
>>>>--- a/include/uapi/drm/xe_drm.h
>>>>+++ b/include/uapi/drm/xe_drm.h
>>>>@@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
>>>> __u64 reserved[2];
>>>>};
>>>>
>>>>+/**
>>>>+ * DOC: XE PMU event config IDs
>>>>+ *
>>>>+ * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in xe_drm.h
>>>>+ * in 'struct perf_event_attr' as part of perf_event_open syscall to read a
>>>>+ * particular event.
>>>>+ *
>>>>+ * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
>>>>+ *
>>>>+ * .. code-block:: C
>>>>+ *
>>>>+ * struct perf_event_attr attr;
>>>>+ * long long count;
>>>>+ * int cpu = 0;
>>>>+ * int fd;
>>>>+ *
>>>>+ * memset(&attr, 0, sizeof(struct perf_event_attr));
>>>>+ * attr.type = type; // eg: /sys/bus/event_source/devices/xe_0000_56_00.0/type
>>>>+ * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
>>>>+ * attr.use_clockid = 1;
>>>>+ * attr.clockid = CLOCK_MONOTONIC;
>>>>+ * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
>>>>+ *
>>>>+ * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
>>>>+ */
>>>>+
>>>>+/*
>>>>+ * Top bits of every counter are GT id.
>>>>+ */
>>>>+#define __XE_PMU_GT_SHIFT (56)
>>>>+
>>>>+#define ___XE_PMU_OTHER(gt, x) \
>>>>+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>>>>+
>>>>+#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
>>>>+#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
>>>>+#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
>>>>+#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
>>>
>>>+ Lucas for inputs
>>>
>>>We should align this to the interface planned for other PMU
>>>busyness counters as well as how we do PCEU. i.e.
>>>
>>>1) counters are in ticks
>>>2) total time in ticks is also exported to the user.
>>>
>>>For 1), I would just append TICKS to the counter names and drop the
>>
>>this uses perf and as such I believe we should use the terms used by
>>perf.
>>
>>$ sudo perf stat sleep 1
>>
>>Performance counter stats for 'sleep 1':
>>
>> 0.91 msec task-clock # 0.001 CPUs utilized
>> 1 context-switches # 1.096 K/sec
>> 0 cpu-migrations # 0.000 /sec
>> 72 page-faults # 78.924 K/sec
>>------> 2,033,156 cycles # 2.229 GHz
>> 1,560,992 instructions # 0.77 insn per cycle
>> 290,814 branches # 318.779 M/sec
>> 10,449 branch-misses # 3.59% of all branches
>>
>> 1.001580466 seconds time elapsed
>>
>> 0.000000000 seconds user
>> 0.001545000 seconds sys
>>
>>so... s/ticks/cycles/
>>
>>I think I said that before, but what's up with all these "group" in the
>>names? It's confusing since apparently group and engine class are mixed.
>
>These are counters defined in the HW and indicate busyness of a group
>of engines (spanning multiple classes) rather than a single engine.
these would really need to be documented. What we are really exposing
are:
#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
Bspec 46729 for OAG_RENDER_BUSY_FREE:
This register counts the time that any render engine is busy.
Bspec 46560 for OAG_BLT_BUSY_FREE:
This register counts the time thatBLT engine is busy
These first 2 match their respective classes
Bspec 46559 for OAG_ANY_MEDIA_FF_BUSY_FREE:
This register counts the time that any media fixed function is busy.
Bspec 46722 for OAG_RC0_ANY_ENGINE_BUSY_FREE:
This register counts the time that any engine is truly busy (not simply
powered up).
And these other 2 span to different classes, already shown by the use of
"any". I don't understand how "group" is helping. It's not how the spec
documents it. I'd expect "Group" to allow to group arbitrary engines
with e.g. a mask.
Lucas De Marchi
>The free running counters are directly read from HW.
>
>Single engine busyness is a different API and wip.
>
>Regards,
>Umesh
>>
>>We are also missing proper kernel-doc in xe_pmu.c
>>
>>Lucas De Marchi
>>
>>>conversion to _ns in __engine_group_busyness_read(). Also, drop
>>>the patch that adds this conversion helper.
>>>
>>>For 2) define a new counter - total active ticks that would return
>>>the 'CPU' timestamp converted to gpu ticks. The reason I am
>>>insisting on CPU timestamp here is because we want to have a time
>>>base that is ticking even when the GPU is idle.
>>>
>>>Regards,
>>>Umesh
>>>
>>>>+
>>>>#if defined(__cplusplus)
>>>>}
>>>>#endif
>>>>--
>>>>2.40.0
>>>>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface
2024-06-28 18:24 ` Lucas De Marchi
@ 2024-06-28 18:49 ` Umesh Nerlige Ramappa
0 siblings, 0 replies; 32+ messages in thread
From: Umesh Nerlige Ramappa @ 2024-06-28 18:49 UTC (permalink / raw)
To: Lucas De Marchi
Cc: Riana Tauro, intel-xe, anshuman.gupta, ashutosh.dixit,
aravind.iddamsetty, rodrigo.vivi, krishnaiah.bommu
On Fri, Jun 28, 2024 at 01:24:42PM -0500, Lucas De Marchi wrote:
>On Fri, Jun 28, 2024 at 09:52:36AM GMT, Umesh Nerlige Ramappa wrote:
>>On Fri, Jun 28, 2024 at 10:55:06AM -0500, Lucas De Marchi wrote:
>>>On Thu, Jun 20, 2024 at 12:52:05PM GMT, Umesh Nerlige Ramappa wrote:
>>>>On Thu, Jun 13, 2024 at 03:34:11PM +0530, Riana Tauro wrote:
>>>>>From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>>>>
>>>>>There are a set of engine group busyness counters provided by HW which are
>>>>>perfect fit to be exposed via PMU perf events.
>>>>>
>>>>>BSPEC: 46559, 46560, 46722, 46729, 52071, 71028
>>>>>
>>>>>events can be listed using:
>>>>>perf list
>>>>>xe_0000_03_00.0/any-engine-group-busy-gt0/ [Kernel PMU event]
>>>>>xe_0000_03_00.0/copy-group-busy-gt0/ [Kernel PMU event]
>>>>>xe_0000_03_00.0/media-group-busy-gt0/ [Kernel PMU event]
>>>>>xe_0000_03_00.0/render-group-busy-gt0/ [Kernel PMU event]
>>>>>
>>>>>and can be read using:
>>>>>
>>>>>perf stat -e "xe_0000_8c_00.0/render-group-busy-gt0/" -I 1000
>>>>> time counts unit events
>>>>> 1.001139062 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 2.003294678 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 3.005199582 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 4.007076497 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 5.008553068 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 6.010531563 43520 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 7.012468029 44800 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 8.013463515 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>> 9.015300183 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>>10.017233010 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>>10.971934120 0 ns xe_0000_8c_00.0/render-group-busy-gt0/
>>>>>
>>>>>The pmu base implementation is taken from i915.
>>>>>
>>>>>v2:
>>>>>Store last known value when device is awake return that while the GT is
>>>>>suspended and then update the driver copy when read during awake.
>>>>>
>>>>>v3:
>>>>>1. drop init_samples, as storing counters before going to suspend should
>>>>>be sufficient.
>>>>>2. ported the "drm/i915/pmu: Make PMU sample array two-dimensional" and
>>>>>dropped helpers to store and read samples.
>>>>>3. use xe_device_mem_access_get_if_ongoing to check if device is active
>>>>>before reading the OA registers.
>>>>>4. dropped format attr as no longer needed
>>>>>5. introduce xe_pmu_suspend to call engine_group_busyness_store
>>>>>6. few other nits.
>>>>>
>>>>>v4: minor nits.
>>>>>
>>>>>v5: take forcewake when accessing the OAG registers
>>>>>
>>>>>v6:
>>>>>1. drop engine_busyness_sample_type
>>>>>2. update UAPI documentation
>>>>>
>>>>>v7:
>>>>>1. update UAPI documentation
>>>>>2. drop MEDIA_GT specific change for media busyness counter.
>>>>>
>>>>>v8:
>>>>>1. rebase
>>>>>2. replace mem_access_if_ongoing with xe_pm_runtime_get_if_active
>>>>>3. remove interrupts pmu event
>>>>>
>>>>>v9: replace drmm_add_action_or_reset with devm (Matthew Auld)
>>>>>
>>>>>Co-developed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>>>>Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>>>>Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>>>>Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>>>>>Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>>>>Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
>>>>>Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>>>>>---
>>>>>drivers/gpu/drm/xe/Makefile | 2 +
>>>>>drivers/gpu/drm/xe/regs/xe_gt_regs.h | 5 +
>>>>>drivers/gpu/drm/xe/xe_device.c | 2 +
>>>>>drivers/gpu/drm/xe/xe_device_types.h | 4 +
>>>>>drivers/gpu/drm/xe/xe_gt.c | 2 +
>>>>>drivers/gpu/drm/xe/xe_module.c | 5 +
>>>>>drivers/gpu/drm/xe/xe_pmu.c | 631 +++++++++++++++++++++++++++
>>>>>drivers/gpu/drm/xe/xe_pmu.h | 26 ++
>>>>>drivers/gpu/drm/xe/xe_pmu_types.h | 67 +++
>>>>>include/uapi/drm/xe_drm.h | 39 ++
>>>>>10 files changed, 783 insertions(+)
>>>>>create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
>>>>>create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
>>>>>create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>>>>>
>>>>>diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>>>>>index cbf961b90237..83bf1e07669b 100644
>>>>>--- a/drivers/gpu/drm/xe/Makefile
>>>>>+++ b/drivers/gpu/drm/xe/Makefile
>>>>>@@ -278,6 +278,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
>>>>> i915-display/skl_universal_plane.o \
>>>>> i915-display/skl_watermark.o
>>>>>
>>>>>+xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>>>>>+
>>>>>ifeq ($(CONFIG_ACPI),y)
>>>>> xe-$(CONFIG_DRM_XE_DISPLAY) += \
>>>>> i915-display/intel_acpi.o \
>>>>>diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>>>index 47c26c37608d..22821dcd4e1b 100644
>>>>>--- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>>>+++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
>>>>>@@ -390,6 +390,11 @@
>>>>>#define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
>>>>>#define GLOBAL_INVALIDATION_MODE REG_BIT(2)
>>>>>
>>>>>+#define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
>>>>>+#define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
>>>>>+#define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
>>>>>+#define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
>>>>>+
>>>>>#define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
>>>>>#define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
>>>>>
>>>>>diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
>>>>>index 64691a56d59c..bb00c8c9ec9b 100644
>>>>>--- a/drivers/gpu/drm/xe/xe_device.c
>>>>>+++ b/drivers/gpu/drm/xe/xe_device.c
>>>>>@@ -668,6 +668,8 @@ int xe_device_probe(struct xe_device *xe)
>>>>>
>>>>> xe_hwmon_register(xe);
>>>>>
>>>>>+ xe_pmu_register(&xe->pmu);
>>>>>+
>>>>> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>>>>>
>>>>>err_fini_display:
>>>>>diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
>>>>>index 52bc461171d5..a5dba7325cf1 100644
>>>>>--- a/drivers/gpu/drm/xe/xe_device_types.h
>>>>>+++ b/drivers/gpu/drm/xe/xe_device_types.h
>>>>>@@ -18,6 +18,7 @@
>>>>>#include "xe_lmtt_types.h"
>>>>>#include "xe_memirq_types.h"
>>>>>#include "xe_platform_types.h"
>>>>>+#include "xe_pmu.h"
>>>>>#include "xe_pt_types.h"
>>>>>#include "xe_sriov_types.h"
>>>>>#include "xe_step_types.h"
>>>>>@@ -473,6 +474,9 @@ struct xe_device {
>>>>> int mode;
>>>>> } wedged;
>>>>>
>>>>>+ /** @pmu: performance monitoring unit */
>>>>>+ struct xe_pmu pmu;
>>>>>+
>>>>> /* private: */
>>>>>
>>>>>#if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
>>>>>diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
>>>>>index 57d84751e160..477d0ae5f230 100644
>>>>>--- a/drivers/gpu/drm/xe/xe_gt.c
>>>>>+++ b/drivers/gpu/drm/xe/xe_gt.c
>>>>>@@ -782,6 +782,8 @@ int xe_gt_suspend(struct xe_gt *gt)
>>>>> if (err)
>>>>> goto err_msg;
>>>>>
>>>>>+ xe_pmu_suspend(gt);
>>>>>+
>>>>> err = xe_uc_suspend(>->uc);
>>>>> if (err)
>>>>> goto err_force_wake;
>>>>>diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
>>>>>index 3edeb30d5ccb..26f814f97fc2 100644
>>>>>--- a/drivers/gpu/drm/xe/xe_module.c
>>>>>+++ b/drivers/gpu/drm/xe/xe_module.c
>>>>>@@ -11,6 +11,7 @@
>>>>>#include "xe_drv.h"
>>>>>#include "xe_hw_fence.h"
>>>>>#include "xe_pci.h"
>>>>>+#include "xe_pmu.h"
>>>>>#include "xe_sched_job.h"
>>>>>
>>>>>struct xe_modparam xe_modparam = {
>>>>>@@ -74,6 +75,10 @@ static const struct init_funcs init_funcs[] = {
>>>>> .init = xe_sched_job_module_init,
>>>>> .exit = xe_sched_job_module_exit,
>>>>> },
>>>>>+ {
>>>>>+ .init = xe_pmu_init,
>>>>>+ .exit = xe_pmu_exit,
>>>>>+ },
>>>>> {
>>>>> .init = xe_register_pci_driver,
>>>>> .exit = xe_unregister_pci_driver,
>>>>>diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>>>>>new file mode 100644
>>>>>index 000000000000..64960a358af2
>>>>>--- /dev/null
>>>>>+++ b/drivers/gpu/drm/xe/xe_pmu.c
>>>>>@@ -0,0 +1,631 @@
>>>>>+// SPDX-License-Identifier: MIT
>>>>>+/*
>>>>>+ * Copyright © 2024 Intel Corporation
>>>>>+ */
>>>>>+
>>>>>+#include <drm/drm_drv.h>
>>>>>+#include <drm/drm_managed.h>
>>>>>+#include <drm/xe_drm.h>
>>>>>+
>>>>>+#include "regs/xe_gt_regs.h"
>>>>>+#include "xe_device.h"
>>>>>+#include "xe_force_wake.h"
>>>>>+#include "xe_gt_clock.h"
>>>>>+#include "xe_mmio.h"
>>>>>+#include "xe_macros.h"
>>>>>+#include "xe_pm.h"
>>>>>+
>>>>>+static cpumask_t xe_pmu_cpumask;
>>>>>+static unsigned int xe_pmu_target_cpu = -1;
>>>>>+
>>>>>+static unsigned int config_gt_id(const u64 config)
>>>>>+{
>>>>>+ return config >> __XE_PMU_GT_SHIFT;
>>>>>+}
>>>>>+
>>>>>+static u64 config_counter(const u64 config)
>>>>>+{
>>>>>+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>>>>>+}
>>>>>+
>>>>>+static void xe_pmu_event_destroy(struct perf_event *event)
>>>>>+{
>>>>>+ struct xe_device *xe =
>>>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>>>+
>>>>>+ drm_WARN_ON(&xe->drm, event->parent);
>>>>>+
>>>>>+ drm_dev_put(&xe->drm);
>>>>>+}
>>>>>+
>>>>>+static u64 __engine_group_busyness_read(struct xe_gt *gt, int sample_type)
>>>>>+{
>>>>>+ u64 val;
>>>>>+
>>>>>+ switch (sample_type) {
>>>>>+ case __XE_SAMPLE_RENDER_GROUP_BUSY:
>>>>>+ val = xe_mmio_read32(gt, XE_OAG_RENDER_BUSY_FREE);
>>>>>+ break;
>>>>>+ case __XE_SAMPLE_COPY_GROUP_BUSY:
>>>>>+ val = xe_mmio_read32(gt, XE_OAG_BLT_BUSY_FREE);
>>>>>+ break;
>>>>>+ case __XE_SAMPLE_MEDIA_GROUP_BUSY:
>>>>>+ val = xe_mmio_read32(gt, XE_OAG_ANY_MEDIA_FF_BUSY_FREE);
>>>>>+ break;
>>>>>+ case __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY:
>>>>>+ val = xe_mmio_read32(gt, XE_OAG_RC0_ANY_ENGINE_BUSY_FREE);
>>>>>+ break;
>>>>>+ default:
>>>>>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>>>>+ }
>>>>>+
>>>>>+ return xe_gt_clock_cycles_to_ns(gt, val * 16);
>>>>>+}
>>>>>+
>>>>>+static u64 engine_group_busyness_read(struct xe_gt *gt, u64 config)
>>>>>+{
>>>>>+ int sample_type = config_counter(config);
>>>>>+ const unsigned int gt_id = gt->info.id;
>>>>>+ struct xe_device *xe = gt->tile->xe;
>>>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>>>+ unsigned long flags;
>>>>>+ bool device_awake;
>>>>>+ u64 val;
>>>>>+
>>>>>+ device_awake = xe_pm_runtime_get_if_active(xe);
>>>>>+ if (device_awake) {
>>>>>+ XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
>>>>>+ val = __engine_group_busyness_read(gt, sample_type);
>>>>>+ XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
>>>>>+ xe_pm_runtime_put(xe);
>>>>>+ }
>>>>>+
>>>>>+ spin_lock_irqsave(&pmu->lock, flags);
>>>>>+
>>>>>+ if (device_awake)
>>>>>+ pmu->sample[gt_id][sample_type] = val;
>>>>>+ else
>>>>>+ val = pmu->sample[gt_id][sample_type];
>>>>>+
>>>>>+ spin_unlock_irqrestore(&pmu->lock, flags);
>>>>>+
>>>>>+ return val;
>>>>>+}
>>>>>+
>>>>>+static void engine_group_busyness_store(struct xe_gt *gt)
>>>>>+{
>>>>>+ struct xe_pmu *pmu = >->tile->xe->pmu;
>>>>>+ unsigned int gt_id = gt->info.id;
>>>>>+ unsigned long flags;
>>>>>+ int i;
>>>>>+
>>>>>+ spin_lock_irqsave(&pmu->lock, flags);
>>>>>+
>>>>>+ for (i = __XE_SAMPLE_RENDER_GROUP_BUSY; i <= __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY; i++)
>>>>>+ pmu->sample[gt_id][i] = __engine_group_busyness_read(gt, i);
>>>>>+
>>>>>+ spin_unlock_irqrestore(&pmu->lock, flags);
>>>>>+}
>>>>>+
>>>>>+static int
>>>>>+config_status(struct xe_device *xe, u64 config)
>>>>>+{
>>>>>+ unsigned int gt_id = config_gt_id(config);
>>>>>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>>>>+
>>>>>+ if (gt_id >= XE_PMU_MAX_GT)
>>>>>+ return -ENOENT;
>>>>>+
>>>>>+ switch (config_counter(config)) {
>>>>>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>>>>>+ case XE_PMU_COPY_GROUP_BUSY(0):
>>>>>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>>>>+ if (gt->info.type == XE_GT_TYPE_MEDIA)
>>>>>+ return -ENOENT;
>>>>>+ break;
>>>>>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>>>>>+ if (!(gt->info.engine_mask & (BIT(XE_HW_ENGINE_VCS0) | BIT(XE_HW_ENGINE_VECS0))))
>>>>>+ return -ENOENT;
>>>>>+ break;
>>>>>+ default:
>>>>>+ return -ENOENT;
>>>>>+ }
>>>>>+
>>>>>+ return 0;
>>>>>+}
>>>>>+
>>>>>+static int xe_pmu_event_init(struct perf_event *event)
>>>>>+{
>>>>>+ struct xe_device *xe =
>>>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>>>+ int ret;
>>>>>+
>>>>>+ if (pmu->closed)
>>>>>+ return -ENODEV;
>>>>>+
>>>>>+ if (event->attr.type != event->pmu->type)
>>>>>+ return -ENOENT;
>>>>>+
>>>>>+ /* unsupported modes and filters */
>>>>>+ if (event->attr.sample_period) /* no sampling */
>>>>>+ return -EINVAL;
>>>>>+
>>>>>+ if (has_branch_stack(event))
>>>>>+ return -EOPNOTSUPP;
>>>>>+
>>>>>+ if (event->cpu < 0)
>>>>>+ return -EINVAL;
>>>>>+
>>>>>+ /* only allow running on one cpu at a time */
>>>>>+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>>>>>+ return -EINVAL;
>>>>>+
>>>>>+ ret = config_status(xe, event->attr.config);
>>>>>+ if (ret)
>>>>>+ return ret;
>>>>>+
>>>>>+ if (!event->parent) {
>>>>>+ drm_dev_get(&xe->drm);
>>>>>+ event->destroy = xe_pmu_event_destroy;
>>>>>+ }
>>>>>+
>>>>>+ return 0;
>>>>>+}
>>>>>+
>>>>>+static u64 __xe_pmu_event_read(struct perf_event *event)
>>>>>+{
>>>>>+ struct xe_device *xe =
>>>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>>>+ const unsigned int gt_id = config_gt_id(event->attr.config);
>>>>>+ const u64 config = event->attr.config;
>>>>>+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>>>>>+ u64 val;
>>>>>+
>>>>>+ switch (config_counter(config)) {
>>>>>+ case XE_PMU_RENDER_GROUP_BUSY(0):
>>>>>+ case XE_PMU_COPY_GROUP_BUSY(0):
>>>>>+ case XE_PMU_ANY_ENGINE_GROUP_BUSY(0):
>>>>>+ case XE_PMU_MEDIA_GROUP_BUSY(0):
>>>>>+ val = engine_group_busyness_read(gt, config);
>>>>>+ break;
>>>>>+ default:
>>>>>+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>>>>>+ }
>>>>>+
>>>>>+ return val;
>>>>>+}
>>>>>+
>>>>>+static void xe_pmu_event_read(struct perf_event *event)
>>>>>+{
>>>>>+ struct xe_device *xe =
>>>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>>>+ struct hw_perf_event *hwc = &event->hw;
>>>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>>>+ u64 prev, new;
>>>>>+
>>>>>+ if (pmu->closed) {
>>>>>+ event->hw.state = PERF_HES_STOPPED;
>>>>>+ return;
>>>>>+ }
>>>>>+again:
>>>>>+ prev = local64_read(&hwc->prev_count);
>>>>>+ new = __xe_pmu_event_read(event);
>>>>>+
>>>>>+ if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>>>>>+ goto again;
>>>>>+
>>>>>+ local64_add(new - prev, &event->count);
>>>>>+}
>>>>>+
>>>>>+static void xe_pmu_enable(struct perf_event *event)
>>>>>+{
>>>>>+ /*
>>>>>+ * Store the current counter value so we can report the correct delta
>>>>>+ * for all listeners. Even when the event was already enabled and has
>>>>>+ * an existing non-zero value.
>>>>>+ */
>>>>>+ local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>>>>>+}
>>>>>+
>>>>>+static void xe_pmu_event_start(struct perf_event *event, int flags)
>>>>>+{
>>>>>+ struct xe_device *xe =
>>>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>>>+
>>>>>+ if (pmu->closed)
>>>>>+ return;
>>>>>+
>>>>>+ xe_pmu_enable(event);
>>>>>+ event->hw.state = 0;
>>>>>+}
>>>>>+
>>>>>+static void xe_pmu_event_stop(struct perf_event *event, int flags)
>>>>>+{
>>>>>+ if (flags & PERF_EF_UPDATE)
>>>>>+ xe_pmu_event_read(event);
>>>>>+
>>>>>+ event->hw.state = PERF_HES_STOPPED;
>>>>>+}
>>>>>+
>>>>>+static int xe_pmu_event_add(struct perf_event *event, int flags)
>>>>>+{
>>>>>+ struct xe_device *xe =
>>>>>+ container_of(event->pmu, typeof(*xe), pmu.base);
>>>>>+ struct xe_pmu *pmu = &xe->pmu;
>>>>>+
>>>>>+ if (pmu->closed)
>>>>>+ return -ENODEV;
>>>>>+
>>>>>+ if (flags & PERF_EF_START)
>>>>>+ xe_pmu_event_start(event, flags);
>>>>>+
>>>>>+ return 0;
>>>>>+}
>>>>>+
>>>>>+static void xe_pmu_event_del(struct perf_event *event, int flags)
>>>>>+{
>>>>>+ xe_pmu_event_stop(event, PERF_EF_UPDATE);
>>>>>+}
>>>>>+
>>>>>+static int xe_pmu_event_event_idx(struct perf_event *event)
>>>>>+{
>>>>>+ return 0;
>>>>>+}
>>>>>+
>>>>>+struct xe_ext_attribute {
>>>>>+ struct device_attribute attr;
>>>>>+ unsigned long val;
>>>>>+};
>>>>>+
>>>>>+static ssize_t xe_pmu_event_show(struct device *dev,
>>>>>+ struct device_attribute *attr, char *buf)
>>>>>+{
>>>>>+ struct xe_ext_attribute *eattr;
>>>>>+
>>>>>+ eattr = container_of(attr, struct xe_ext_attribute, attr);
>>>>>+ return sprintf(buf, "config=0x%lx\n", eattr->val);
>>>>>+}
>>>>>+
>>>>>+static ssize_t cpumask_show(struct device *dev,
>>>>>+ struct device_attribute *attr, char *buf)
>>>>>+{
>>>>>+ return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>>>>>+}
>>>>>+
>>>>>+static DEVICE_ATTR_RO(cpumask);
>>>>>+
>>>>>+static struct attribute *xe_cpumask_attrs[] = {
>>>>>+ &dev_attr_cpumask.attr,
>>>>>+ NULL,
>>>>>+};
>>>>>+
>>>>>+static const struct attribute_group xe_pmu_cpumask_attr_group = {
>>>>>+ .attrs = xe_cpumask_attrs,
>>>>>+};
>>>>>+
>>>>>+#define __event(__counter, __name, __unit) \
>>>>>+{ \
>>>>>+ .counter = (__counter), \
>>>>>+ .name = (__name), \
>>>>>+ .unit = (__unit), \
>>>>>+}
>>>>>+
>>>>>+static struct xe_ext_attribute *
>>>>>+add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>>>>>+{
>>>>>+ sysfs_attr_init(&attr->attr.attr);
>>>>>+ attr->attr.attr.name = name;
>>>>>+ attr->attr.attr.mode = 0444;
>>>>>+ attr->attr.show = xe_pmu_event_show;
>>>>>+ attr->val = config;
>>>>>+
>>>>>+ return ++attr;
>>>>>+}
>>>>>+
>>>>>+static struct perf_pmu_events_attr *
>>>>>+add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>>>>>+ const char *str)
>>>>>+{
>>>>>+ sysfs_attr_init(&attr->attr.attr);
>>>>>+ attr->attr.attr.name = name;
>>>>>+ attr->attr.attr.mode = 0444;
>>>>>+ attr->attr.show = perf_event_sysfs_show;
>>>>>+ attr->event_str = str;
>>>>>+
>>>>>+ return ++attr;
>>>>>+}
>>>>>+
>>>>>+static struct attribute **
>>>>>+create_event_attributes(struct xe_pmu *pmu)
>>>>>+{
>>>>>+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>>>>+ static const struct {
>>>>>+ unsigned int counter;
>>>>>+ const char *name;
>>>>>+ const char *unit;
>>>>>+ } events[] = {
>>>>>+ __event(0, "render-group-busy", "ns"),
>>>>>+ __event(1, "copy-group-busy", "ns"),
>>>>>+ __event(2, "media-group-busy", "ns"),
>>>>>+ __event(3, "any-engine-group-busy", "ns"),
>>>>>+ };
>>>>>+
>>>>>+ struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>>>>>+ struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>>>>>+ struct attribute **attr = NULL, **attr_iter;
>>>>>+ unsigned int count = 0;
>>>>>+ unsigned int i, j;
>>>>>+ struct xe_gt *gt;
>>>>>+
>>>>>+ /* Count how many counters we will be exposing. */
>>>>>+ for_each_gt(gt, xe, j) {
>>>>>+ for (i = 0; i < ARRAY_SIZE(events); i++) {
>>>>>+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>>>>+
>>>>>+ if (!config_status(xe, config))
>>>>>+ count++;
>>>>>+ }
>>>>>+ }
>>>>>+
>>>>>+ /* Allocate attribute objects and table. */
>>>>>+ xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>>>>>+ if (!xe_attr)
>>>>>+ goto err_alloc;
>>>>>+
>>>>>+ pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>>>>>+ if (!pmu_attr)
>>>>>+ goto err_alloc;
>>>>>+
>>>>>+ /* Max one pointer of each attribute type plus a termination entry. */
>>>>>+ attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>>>>>+ if (!attr)
>>>>>+ goto err_alloc;
>>>>>+
>>>>>+ xe_iter = xe_attr;
>>>>>+ pmu_iter = pmu_attr;
>>>>>+ attr_iter = attr;
>>>>>+
>>>>>+ for_each_gt(gt, xe, j) {
>>>>>+ for (i = 0; i < ARRAY_SIZE(events); i++) {
>>>>>+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>>>>>+ char *str;
>>>>>+
>>>>>+ if (config_status(xe, config))
>>>>>+ continue;
>>>>>+
>>>>>+ str = kasprintf(GFP_KERNEL, "%s-gt%u",
>>>>>+ events[i].name, j);
>>>>>+ if (!str)
>>>>>+ goto err;
>>>>>+
>>>>>+ *attr_iter++ = &xe_iter->attr.attr;
>>>>>+ xe_iter = add_xe_attr(xe_iter, str, config);
>>>>>+
>>>>>+ if (events[i].unit) {
>>>>>+ str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>>>>>+ events[i].name, j);
>>>>>+ if (!str)
>>>>>+ goto err;
>>>>>+
>>>>>+ *attr_iter++ = &pmu_iter->attr.attr;
>>>>>+ pmu_iter = add_pmu_attr(pmu_iter, str,
>>>>>+ events[i].unit);
>>>>>+ }
>>>>>+ }
>>>>>+ }
>>>>>+
>>>>>+ pmu->xe_attr = xe_attr;
>>>>>+ pmu->pmu_attr = pmu_attr;
>>>>>+
>>>>>+ return attr;
>>>>>+
>>>>>+err:
>>>>>+ for (attr_iter = attr; *attr_iter; attr_iter++)
>>>>>+ kfree((*attr_iter)->name);
>>>>>+
>>>>>+err_alloc:
>>>>>+ kfree(attr);
>>>>>+ kfree(xe_attr);
>>>>>+ kfree(pmu_attr);
>>>>>+
>>>>>+ return NULL;
>>>>>+}
>>>>>+
>>>>>+static void free_event_attributes(struct xe_pmu *pmu)
>>>>>+{
>>>>>+ struct attribute **attr_iter = pmu->events_attr_group.attrs;
>>>>>+
>>>>>+ for (; *attr_iter; attr_iter++)
>>>>>+ kfree((*attr_iter)->name);
>>>>>+
>>>>>+ kfree(pmu->events_attr_group.attrs);
>>>>>+ kfree(pmu->xe_attr);
>>>>>+ kfree(pmu->pmu_attr);
>>>>>+
>>>>>+ pmu->events_attr_group.attrs = NULL;
>>>>>+ pmu->xe_attr = NULL;
>>>>>+ pmu->pmu_attr = NULL;
>>>>>+}
>>>>>+
>>>>>+static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>>>>>+{
>>>>>+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>>>>+
>>>>>+ /* Select the first online CPU as a designated reader. */
>>>>>+ if (cpumask_empty(&xe_pmu_cpumask))
>>>>>+ cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>>>>>+
>>>>>+ return 0;
>>>>>+}
>>>>>+
>>>>>+static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>>>>>+{
>>>>>+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>>>>>+ unsigned int target = xe_pmu_target_cpu;
>>>>>+
>>>>>+ /*
>>>>>+ * Unregistering an instance generates a CPU offline event which we must
>>>>>+ * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>>>>>+ */
>>>>>+ if (pmu->closed)
>>>>>+ return 0;
>>>>>+
>>>>>+ if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>>>>>+ target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>>>>>+
>>>>>+ /* Migrate events if there is a valid target */
>>>>>+ if (target < nr_cpu_ids) {
>>>>>+ cpumask_set_cpu(target, &xe_pmu_cpumask);
>>>>>+ xe_pmu_target_cpu = target;
>>>>>+ }
>>>>>+ }
>>>>>+
>>>>>+ if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>>>>>+ perf_pmu_migrate_context(&pmu->base, cpu, target);
>>>>>+ pmu->cpuhp.cpu = target;
>>>>>+ }
>>>>>+
>>>>>+ return 0;
>>>>>+}
>>>>>+
>>>>>+static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>>>>>+
>>>>>+int xe_pmu_init(void)
>>>>>+{
>>>>>+ int ret;
>>>>>+
>>>>>+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>>>>>+ "perf/x86/intel/xe:online",
>>>>>+ xe_pmu_cpu_online,
>>>>>+ xe_pmu_cpu_offline);
>>>>>+ if (ret < 0)
>>>>>+ pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>>>>>+ ret);
>>>>>+ else
>>>>>+ cpuhp_slot = ret;
>>>>>+
>>>>>+ return 0;
>>>>>+}
>>>>>+
>>>>>+void xe_pmu_exit(void)
>>>>>+{
>>>>>+ if (cpuhp_slot != CPUHP_INVALID)
>>>>>+ cpuhp_remove_multi_state(cpuhp_slot);
>>>>>+}
>>>>>+
>>>>>+static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>>>>>+{
>>>>>+ if (cpuhp_slot == CPUHP_INVALID)
>>>>>+ return -EINVAL;
>>>>>+
>>>>>+ return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>>>>>+}
>>>>>+
>>>>>+static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>>>>>+{
>>>>>+ cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>>>>>+}
>>>>>+
>>>>>+void xe_pmu_suspend(struct xe_gt *gt)
>>>>>+{
>>>>>+ engine_group_busyness_store(gt);
>>>>>+}
>>>>>+
>>>>>+static void xe_pmu_unregister(void *arg)
>>>>>+{
>>>>>+ struct xe_pmu *pmu = arg;
>>>>>+
>>>>>+ if (!pmu->base.event_init)
>>>>>+ return;
>>>>>+
>>>>>+ /*
>>>>>+ * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
>>>>>+ * ensures all currently executing ones will have exited before we
>>>>>+ * proceed with unregistration.
>>>>>+ */
>>>>>+ pmu->closed = true;
>>>>>+ synchronize_rcu();
>>>>>+
>>>>>+ xe_pmu_unregister_cpuhp_state(pmu);
>>>>>+
>>>>>+ perf_pmu_unregister(&pmu->base);
>>>>>+ pmu->base.event_init = NULL;
>>>>>+ kfree(pmu->base.attr_groups);
>>>>>+ kfree(pmu->name);
>>>>>+ free_event_attributes(pmu);
>>>>>+}
>>>>>+
>>>>>+void xe_pmu_register(struct xe_pmu *pmu)
>>>>>+{
>>>>>+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>>>>>+ const struct attribute_group *attr_groups[] = {
>>>>>+ &pmu->events_attr_group,
>>>>>+ &xe_pmu_cpumask_attr_group,
>>>>>+ NULL
>>>>>+ };
>>>>>+
>>>>>+ int ret = -ENOMEM;
>>>>>+
>>>>>+ spin_lock_init(&pmu->lock);
>>>>>+ pmu->cpuhp.cpu = -1;
>>>>>+
>>>>>+ pmu->name = kasprintf(GFP_KERNEL,
>>>>>+ "xe_%s",
>>>>>+ dev_name(xe->drm.dev));
>>>>>+ if (pmu->name)
>>>>>+ /* tools/perf reserves colons as special. */
>>>>>+ strreplace((char *)pmu->name, ':', '_');
>>>>>+
>>>>>+ if (!pmu->name)
>>>>>+ goto err;
>>>>>+
>>>>>+ pmu->events_attr_group.name = "events";
>>>>>+ pmu->events_attr_group.attrs = create_event_attributes(pmu);
>>>>>+ if (!pmu->events_attr_group.attrs)
>>>>>+ goto err_name;
>>>>>+
>>>>>+ pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>>>>>+ GFP_KERNEL);
>>>>>+ if (!pmu->base.attr_groups)
>>>>>+ goto err_attr;
>>>>>+
>>>>>+ pmu->base.module = THIS_MODULE;
>>>>>+ pmu->base.task_ctx_nr = perf_invalid_context;
>>>>>+ pmu->base.event_init = xe_pmu_event_init;
>>>>>+ pmu->base.add = xe_pmu_event_add;
>>>>>+ pmu->base.del = xe_pmu_event_del;
>>>>>+ pmu->base.start = xe_pmu_event_start;
>>>>>+ pmu->base.stop = xe_pmu_event_stop;
>>>>>+ pmu->base.read = xe_pmu_event_read;
>>>>>+ pmu->base.event_idx = xe_pmu_event_event_idx;
>>>>>+
>>>>>+ ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>>>>>+ if (ret)
>>>>>+ goto err_groups;
>>>>>+
>>>>>+ ret = xe_pmu_register_cpuhp_state(pmu);
>>>>>+ if (ret)
>>>>>+ goto err_unreg;
>>>>>+
>>>>>+ ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>>>>>+ if (ret)
>>>>>+ goto err_cpuhp;
>>>>>+
>>>>>+ return;
>>>>>+
>>>>>+err_cpuhp:
>>>>>+ xe_pmu_unregister_cpuhp_state(pmu);
>>>>>+err_unreg:
>>>>>+ perf_pmu_unregister(&pmu->base);
>>>>>+err_groups:
>>>>>+ kfree(pmu->base.attr_groups);
>>>>>+err_attr:
>>>>>+ pmu->base.event_init = NULL;
>>>>>+ free_event_attributes(pmu);
>>>>>+err_name:
>>>>>+ kfree(pmu->name);
>>>>>+err:
>>>>>+ drm_notice(&xe->drm, "Failed to register PMU!\n");
>>>>>+}
>>>>>diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>>>>>new file mode 100644
>>>>>index 000000000000..8afa256f9dac
>>>>>--- /dev/null
>>>>>+++ b/drivers/gpu/drm/xe/xe_pmu.h
>>>>>@@ -0,0 +1,26 @@
>>>>>+/* SPDX-License-Identifier: MIT */
>>>>>+/*
>>>>>+ * Copyright © 2024 Intel Corporation
>>>>>+ */
>>>>>+
>>>>>+#ifndef _XE_PMU_H_
>>>>>+#define _XE_PMU_H_
>>>>>+
>>>>>+#include "xe_pmu_types.h"
>>>>>+
>>>>>+struct xe_gt;
>>>>>+
>>>>>+#if IS_ENABLED(CONFIG_PERF_EVENTS)
>>>>>+int xe_pmu_init(void);
>>>>>+void xe_pmu_exit(void);
>>>>>+void xe_pmu_register(struct xe_pmu *pmu);
>>>>>+void xe_pmu_suspend(struct xe_gt *gt);
>>>>>+#else
>>>>>+static inline int xe_pmu_init(void) { return 0; }
>>>>>+static inline void xe_pmu_exit(void) {}
>>>>>+static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>>>>>+static inline void xe_pmu_suspend(struct xe_gt *gt) {}
>>>>>+#endif
>>>>>+
>>>>>+#endif
>>>>>+
>>>>>diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
>>>>>new file mode 100644
>>>>>index 000000000000..e86e8d7e0356
>>>>>--- /dev/null
>>>>>+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>>>>>@@ -0,0 +1,67 @@
>>>>>+/* SPDX-License-Identifier: MIT */
>>>>>+/*
>>>>>+ * Copyright © 2024 Intel Corporation
>>>>>+ */
>>>>>+
>>>>>+#ifndef _XE_PMU_TYPES_H_
>>>>>+#define _XE_PMU_TYPES_H_
>>>>>+
>>>>>+#include <linux/perf_event.h>
>>>>>+#include <linux/spinlock_types.h>
>>>>>+#include <uapi/drm/xe_drm.h>
>>>>>+
>>>>>+enum {
>>>>>+ __XE_SAMPLE_RENDER_GROUP_BUSY,
>>>>>+ __XE_SAMPLE_COPY_GROUP_BUSY,
>>>>>+ __XE_SAMPLE_MEDIA_GROUP_BUSY,
>>>>>+ __XE_SAMPLE_ANY_ENGINE_GROUP_BUSY,
>>>>>+ __XE_NUM_PMU_SAMPLERS
>>>>>+};
>>>>>+
>>>>>+#define XE_PMU_MAX_GT 2
>>>>>+
>>>>>+struct xe_pmu {
>>>>>+ /**
>>>>>+ * @cpuhp: Struct used for CPU hotplug handling.
>>>>>+ */
>>>>>+ struct {
>>>>>+ struct hlist_node node;
>>>>>+ unsigned int cpu;
>>>>>+ } cpuhp;
>>>>>+ /**
>>>>>+ * @base: PMU base.
>>>>>+ */
>>>>>+ struct pmu base;
>>>>>+ /**
>>>>>+ * @closed: xe is unregistering.
>>>>>+ */
>>>>>+ bool closed;
>>>>>+ /**
>>>>>+ * @name: Name as registered with perf core.
>>>>>+ */
>>>>>+ const char *name;
>>>>>+ /**
>>>>>+ * @lock: Lock protecting enable mask and ref count handling.
>>>>>+ */
>>>>>+ spinlock_t lock;
>>>>>+ /**
>>>>>+ * @sample: Current and previous (raw) counters.
>>>>>+ *
>>>>>+ * These counters are updated when the device is awake.
>>>>>+ */
>>>>>+ u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>>>>>+ /**
>>>>>+ * @events_attr_group: Device events attribute group.
>>>>>+ */
>>>>>+ struct attribute_group events_attr_group;
>>>>>+ /**
>>>>>+ * @xe_attr: Memory block holding device attributes.
>>>>>+ */
>>>>>+ void *xe_attr;
>>>>>+ /**
>>>>>+ * @pmu_attr: Memory block holding device attributes.
>>>>>+ */
>>>>>+ void *pmu_attr;
>>>>>+};
>>>>>+
>>>>>+#endif
>>>>>diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>>>>>index d7b0903c22b2..07ca545354f7 100644
>>>>>--- a/include/uapi/drm/xe_drm.h
>>>>>+++ b/include/uapi/drm/xe_drm.h
>>>>>@@ -1370,6 +1370,45 @@ struct drm_xe_wait_user_fence {
>>>>> __u64 reserved[2];
>>>>>};
>>>>>
>>>>>+/**
>>>>>+ * DOC: XE PMU event config IDs
>>>>>+ *
>>>>>+ * Check 'man perf_event_open' to use the ID's XE_PMU_XXXX listed in xe_drm.h
>>>>>+ * in 'struct perf_event_attr' as part of perf_event_open syscall to read a
>>>>>+ * particular event.
>>>>>+ *
>>>>>+ * For example to open the XE_PMU_RENDER_GROUP_BUSY(0):
>>>>>+ *
>>>>>+ * .. code-block:: C
>>>>>+ *
>>>>>+ * struct perf_event_attr attr;
>>>>>+ * long long count;
>>>>>+ * int cpu = 0;
>>>>>+ * int fd;
>>>>>+ *
>>>>>+ * memset(&attr, 0, sizeof(struct perf_event_attr));
>>>>>+ * attr.type = type; // eg: /sys/bus/event_source/devices/xe_0000_56_00.0/type
>>>>>+ * attr.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED;
>>>>>+ * attr.use_clockid = 1;
>>>>>+ * attr.clockid = CLOCK_MONOTONIC;
>>>>>+ * attr.config = XE_PMU_RENDER_GROUP_BUSY(0);
>>>>>+ *
>>>>>+ * fd = syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
>>>>>+ */
>>>>>+
>>>>>+/*
>>>>>+ * Top bits of every counter are GT id.
>>>>>+ */
>>>>>+#define __XE_PMU_GT_SHIFT (56)
>>>>>+
>>>>>+#define ___XE_PMU_OTHER(gt, x) \
>>>>>+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>>>>>+
>>>>>+#define XE_PMU_RENDER_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 0)
>>>>>+#define XE_PMU_COPY_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 1)
>>>>>+#define XE_PMU_MEDIA_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 2)
>>>>>+#define XE_PMU_ANY_ENGINE_GROUP_BUSY(gt) ___XE_PMU_OTHER(gt, 3)
>>>>
>>>>+ Lucas for inputs
>>>>
>>>>We should align this to the interface planned for other PMU
>>>>busyness counters as well as how we do PCEU. i.e.
>>>>
>>>>1) counters are in ticks
>>>>2) total time in ticks is also exported to the user.
>>>>
>>>>For 1), I would just append TICKS to the counter names and drop the
>>>
>>>this uses perf and as such I believe we should use the terms used by
>>>perf.
>>>
>>>$ sudo perf stat sleep 1
>>>
>>>Performance counter stats for 'sleep 1':
>>>
>>> 0.91 msec task-clock # 0.001 CPUs utilized
>>> 1 context-switches # 1.096 K/sec
>>> 0 cpu-migrations # 0.000 /sec
>>> 72 page-faults # 78.924 K/sec
>>>------> 2,033,156 cycles # 2.229 GHz
>>> 1,560,992 instructions # 0.77 insn per cycle
>>> 290,814 branches # 318.779 M/sec
>>> 10,449 branch-misses # 3.59% of all branches
>>>
>>> 1.001580466 seconds time elapsed
>>>
>>> 0.000000000 seconds user
>>> 0.001545000 seconds sys
>>>
>>>so... s/ticks/cycles/
>>>
>>>I think I said that before, but what's up with all these "group" in the
>>>names? It's confusing since apparently group and engine class are mixed.
>>
>>These are counters defined in the HW and indicate busyness of a
>>group of engines (spanning multiple classes) rather than a single
>>engine.
>
>these would really need to be documented. What we are really exposing
>are:
>
> #define XE_OAG_RC0_ANY_ENGINE_BUSY_FREE XE_REG(0xdb80)
> #define XE_OAG_ANY_MEDIA_FF_BUSY_FREE XE_REG(0xdba0)
> #define XE_OAG_BLT_BUSY_FREE XE_REG(0xdbbc)
> #define XE_OAG_RENDER_BUSY_FREE XE_REG(0xdbdc)
>
>Bspec 46729 for OAG_RENDER_BUSY_FREE:
>This register counts the time that any render engine is busy.
>
>Bspec 46560 for OAG_BLT_BUSY_FREE:
>This register counts the time thatBLT engine is busy
>
>These first 2 match their respective classes
>
>Bspec 46559 for OAG_ANY_MEDIA_FF_BUSY_FREE:
>This register counts the time that any media fixed function is busy.
>
>Bspec 46722 for OAG_RC0_ANY_ENGINE_BUSY_FREE:
>This register counts the time that any engine is truly busy (not simply
>powered up).
>
>And these other 2 span to different classes, already shown by the use of
>"any". I don't understand how "group" is helping. It's not how the spec
>documents it. I'd expect "Group" to allow to group arbitrary engines
>with e.g. a mask.
I have no real preference here. If group is misleading, we should drop
that. AFAIR, the feature itself was referred to as group engine busyness
and may have carried over that nomenclature to the API.
Regards,
Umesh
>
>Lucas De Marchi
>
>>The free running counters are directly read from HW.
>>
>>Single engine busyness is a different API and wip.
>>
>>Regards,
>>Umesh
>>>
>>>We are also missing proper kernel-doc in xe_pmu.c
>>>
>>>Lucas De Marchi
>>>
>>>>conversion to _ns in __engine_group_busyness_read(). Also, drop
>>>>the patch that adds this conversion helper.
>>>>
>>>>For 2) define a new counter - total active ticks that would
>>>>return the 'CPU' timestamp converted to gpu ticks. The reason I
>>>>am insisting on CPU timestamp here is because we want to have a
>>>>time base that is ticking even when the GPU is idle.
>>>>
>>>>Regards,
>>>>Umesh
>>>>
>>>>>+
>>>>>#if defined(__cplusplus)
>>>>>}
>>>>>#endif
>>>>>--
>>>>>2.40.0
>>>>>
^ permalink raw reply [flat|nested] 32+ messages in thread
end of thread, other threads:[~2024-06-28 18:50 UTC | newest]
Thread overview: 32+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-06-13 10:04 [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Riana Tauro
2024-06-13 9:57 ` ✓ CI.Patch_applied: success for drm/xe/pmu: Enable PMU interface (rev9) Patchwork
2024-06-13 9:57 ` ✗ CI.checkpatch: warning " Patchwork
2024-06-13 9:59 ` ✓ CI.KUnit: success " Patchwork
2024-06-13 10:04 ` [PATCH v9 1/2] drm/xe: Get GT clock to nanosecs Riana Tauro
2024-06-14 14:57 ` Lucas De Marchi
2024-06-13 10:04 ` [PATCH v9 2/2] drm/xe/pmu: Enable PMU interface Riana Tauro
2024-06-14 16:15 ` Lucas De Marchi
2024-06-14 16:38 ` Tvrtko Ursulin
2024-06-14 20:54 ` Ghimiray, Himal Prasad
2024-06-27 5:21 ` Riana Tauro
2024-06-20 19:52 ` Umesh Nerlige Ramappa
2024-06-27 6:49 ` Aravind Iddamsetty
2024-06-27 16:05 ` Umesh Nerlige Ramappa
2024-06-28 9:41 ` Aravind Iddamsetty
2024-06-28 16:36 ` Umesh Nerlige Ramappa
2024-06-28 15:55 ` Lucas De Marchi
2024-06-28 16:52 ` Umesh Nerlige Ramappa
2024-06-28 18:24 ` Lucas De Marchi
2024-06-28 18:49 ` Umesh Nerlige Ramappa
2024-06-13 10:11 ` ✓ CI.Build: success for drm/xe/pmu: Enable PMU interface (rev9) Patchwork
2024-06-13 10:13 ` ✗ CI.Hooks: failure " Patchwork
2024-06-13 10:14 ` ✓ CI.checksparse: success " Patchwork
2024-06-13 11:48 ` ✓ CI.BAT: " Patchwork
2024-06-13 18:39 ` [PATCH v9 0/2] drm/xe/pmu: Enable PMU interface Dixit, Ashutosh
2024-06-14 5:50 ` Riana Tauro
2024-06-14 15:34 ` Dixit, Ashutosh
2024-06-20 19:15 ` Umesh Nerlige Ramappa
2024-06-21 4:29 ` Aravind Iddamsetty
2024-06-21 18:32 ` Umesh Nerlige Ramappa
2024-06-24 10:27 ` Aravind Iddamsetty
2024-06-24 23:23 ` Umesh Nerlige Ramappa
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox