From: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
To: Chris Wilson <chris@chris-wilson.co.uk>, intel-gfx@lists.freedesktop.org
Subject: Re: [PATCH] RFC drm/i915/pmu: Avoid sleeping rpm_get under atomic PMU read
Date: Mon, 5 Feb 2018 09:47:52 +0000 [thread overview]
Message-ID: <009586ed-8292-05b5-d925-86ca7a0c7363@linux.intel.com> (raw)
In-Reply-To: <20180205091138.32740-1-chris@chris-wilson.co.uk>
On 05/02/2018 09:11, Chris Wilson wrote:
> The pmu_event_read() callchain is in an atomic context which then
Call-chain.
> forbids us sleeping, such as required to wake the device up for rpm.
> In this case, we can only read our registers iff the device is already
s/iff/if/
> awake. In the case of rc6 this means that we cannot report the latest
> values when the whole system is asleep.
>
> <3>[ 101.309358] BUG: sleeping function called from invalid context at drivers/base/power/runtime.c:1034
> <3>[ 101.309502] in_atomic(): 1, irqs_disabled(): 1, pid: 1513, name: perf_pmu
> <4>[ 101.309558] 3 locks held by perf_pmu/1513:
> <4>[ 101.309563] #0: (&cpuctx_mutex){+.+.}, at: [<00000000c3ec2708>] perf_event_ctx_lock_nested+0xbc/0x1d0
> <4>[ 101.309598] #1: (&event->child_mutex){+.+.}, at: [<000000004cd02bc9>] __perf_event_read_value+0x32/0xf0
> <4>[ 101.309624] #2: (&cpuctx_lock){....}, at: [<000000002343d2dc>] __perf_event_read+0x64/0x2e0
> <4>[ 101.309653] irq event stamp: 25750
> <4>[ 101.309662] hardirqs last enabled at (25749): [<00000000c4700020>] entry_SYSCALL_64_fastpath+0x5/0x8f
> <4>[ 101.309670] hardirqs last disabled at (25750): [<00000000e34beea3>] generic_exec_single+0x90/0xe0
> <4>[ 101.309678] softirqs last enabled at (25466): [<00000000bebe2149>] __do_softirq+0x3a1/0x4aa
> <4>[ 101.309687] softirqs last disabled at (25443): [<00000000062fd6b7>] irq_exit+0xa4/0xb0
> <3>[ 101.309692] Preemption disabled at:
> <4>[ 101.309699] [<000000005125cef8>] perf_event_read+0x98/0x2e0
> <4>[ 101.309741] CPU: 0 PID: 1513 Comm: perf_pmu Tainted: G U 4.15.0-CI-CI_DRM_3717+ #1
> <4>[ 101.309747] Hardware name: /NUC6CAYB, BIOS AYAPLCEL.86A.0040.2017.0619.1722 06/19/2017
> <4>[ 101.309752] Call Trace:
> <4>[ 101.309764] dump_stack+0x5f/0x86
> <4>[ 101.309774] ___might_sleep+0x1d9/0x240
> <4>[ 101.309788] __pm_runtime_resume+0x75/0x80
> <4>[ 101.309870] intel_runtime_pm_get+0x1c/0xa0 [i915]
> <4>[ 101.309938] __i915_pmu_event_read+0x54/0x1c0 [i915]
> <4>[ 101.310008] i915_pmu_event_read+0x1e/0x40 [i915]
> <4>[ 101.310019] __perf_event_read+0x2b8/0x2e0
> <4>[ 101.310031] ? __perf_event_output_stop+0xa0/0xa0
> <4>[ 101.310040] generic_exec_single+0x96/0xe0
> <4>[ 101.310050] ? __perf_event_output_stop+0xa0/0xa0
> <4>[ 101.310058] smp_call_function_single+0x88/0x120
> <4>[ 101.310076] ? perf_event_read+0xed/0x2e0
> <4>[ 101.310083] ? preempt_count_add+0x44/0x90
> <4>[ 101.310090] perf_event_read+0xed/0x2e0
> <4>[ 101.310106] __perf_event_read_value+0x3c/0xf0
> <4>[ 101.310119] perf_read+0x15e/0x290
> <4>[ 101.310130] ? lock_acquire+0xaf/0x200
> <4>[ 101.310143] ? free_object+0x6e/0x90
> <4>[ 101.310156] __vfs_read+0x1e/0x120
> <4>[ 101.310166] ? _raw_spin_unlock_irqrestore+0x39/0x60
> <4>[ 101.310174] ? free_object+0x6e/0x90
> <4>[ 101.310184] ? entry_SYSCALL_64_fastpath+0x5/0x8f
> <4>[ 101.310195] vfs_read+0x9e/0x150
> <4>[ 101.310206] SyS_read+0x40/0xa0
> <4>[ 101.310219] entry_SYSCALL_64_fastpath+0x22/0x8f
> <4>[ 101.310226] RIP: 0033:0x7f361c4276d0
> <4>[ 101.310232] RSP: 002b:00007ffcce301ee8 EFLAGS: 00000246
> <3>[ 101.311022] BUG: scheduling while atomic: perf_pmu/1513/0x00000004
> <4>[ 101.311082] 3 locks held by perf_pmu/1513:
> <4>[ 101.311088] #0: (&cpuctx_mutex){+.+.}, at: [<00000000c3ec2708>] perf_event_ctx_lock_nested+0xbc/0x1d0
> <4>[ 101.311124] #1: (&event->child_mutex){+.+.}, at: [<000000004cd02bc9>] __perf_event_read_value+0x32/0xf0
> <4>[ 101.311152] #2: (&cpuctx_lock){+.+.}, at: [<000000002343d2dc>] __perf_event_read+0x64/0x2e0
> <4>[ 101.311180] Modules linked in: snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic x86_pkg_temp_thermal coretemp crct10dif_pclmul crc32_pclmul ghash_clmulni_intel i915 lpc_ich r8169 mii snd_hda_intel snd_hda_codec snd_hwdep snd_hda_core snd_pcm pinctrl_broxton pinctrl_intel mei_me prime_numbers mei
> <3>[ 101.311314] Preemption disabled at:
> <4>[ 101.311321] [<000000005125cef8>] perf_event_read+0x98/0x2e0
> <4>[ 101.311362] CPU: 0 PID: 1513 Comm: perf_pmu Tainted: G U W 4.15.0-CI-CI_DRM_3717+ #1
> <4>[ 101.311368] Hardware name: /NUC6CAYB, BIOS AYAPLCEL.86A.0040.2017.0619.1722 06/19/2017
> <4>[ 101.311374] Call Trace:
> <4>[ 101.311385] dump_stack+0x5f/0x86
> <4>[ 101.311394] ? perf_event_read+0x98/0x2e0
> <4>[ 101.311402] __schedule_bug+0x7a/0xd0
> <4>[ 101.311413] __schedule+0x84c/0xaf0
> <4>[ 101.311424] ? mark_held_locks+0x64/0x90
> <4>[ 101.311432] ? _raw_spin_unlock_irqrestore+0x4c/0x60
> <4>[ 101.311446] schedule+0x37/0x90
> <4>[ 101.311456] schedule_timeout+0x1da/0x520
> <4>[ 101.311471] ? collect_expired_timers+0xa0/0xa0
> <4>[ 101.311485] ? msleep+0x1d/0x40
> <4>[ 101.311497] ? pci_restore_standard_config+0x40/0x40
> <4>[ 101.311504] msleep+0x34/0x40
> <4>[ 101.311514] pci_raw_set_power_state+0x18f/0x220
> <4>[ 101.311526] pci_set_power_state+0x94/0x110
> <4>[ 101.311537] pci_restore_standard_config+0x22/0x40
> <4>[ 101.311545] pci_pm_runtime_resume+0x3d/0x90
> <4>[ 101.311556] __rpm_callback+0xb1/0x1e0
> <4>[ 101.311566] ? pci_restore_standard_config+0x40/0x40
> <4>[ 101.311577] rpm_callback+0x1a/0x70
> <4>[ 101.311586] ? pci_restore_standard_config+0x40/0x40
> <4>[ 101.311594] rpm_resume+0x4d7/0x770
> <4>[ 101.311615] __pm_runtime_resume+0x42/0x80
> <4>[ 101.311700] intel_runtime_pm_get+0x1c/0xa0 [i915]
> <4>[ 101.311769] __i915_pmu_event_read+0x54/0x1c0 [i915]
> <4>[ 101.311840] i915_pmu_event_read+0x1e/0x40 [i915]
> <4>[ 101.311852] __perf_event_read+0x2b8/0x2e0
> <4>[ 101.311864] ? __perf_event_output_stop+0xa0/0xa0
> <4>[ 101.311874] generic_exec_single+0x96/0xe0
> <4>[ 101.311883] ? __perf_event_output_stop+0xa0/0xa0
> <4>[ 101.311891] smp_call_function_single+0x88/0x120
> <4>[ 101.311910] ? perf_event_read+0xed/0x2e0
> <4>[ 101.311916] ? preempt_count_add+0x44/0x90
> <4>[ 101.311924] perf_event_read+0xed/0x2e0
> <4>[ 101.311940] __perf_event_read_value+0x3c/0xf0
> <4>[ 101.311954] perf_read+0x15e/0x290
> <4>[ 101.311963] ? lock_acquire+0xaf/0x200
> <4>[ 101.311975] ? free_object+0x6e/0x90
> <4>[ 101.311989] __vfs_read+0x1e/0x120
> <4>[ 101.311999] ? _raw_spin_unlock_irqrestore+0x39/0x60
> <4>[ 101.312007] ? free_object+0x6e/0x90
> <4>[ 101.312018] ? entry_SYSCALL_64_fastpath+0x5/0x8f
> <4>[ 101.312028] vfs_read+0x9e/0x150
> <4>[ 101.312040] SyS_read+0x40/0xa0
> <4>[ 101.312054] entry_SYSCALL_64_fastpath+0x22/0x8f
> <4>[ 101.312060] RIP: 0033:0x7f361c4276d0
> <4>[ 101.312066] RSP: 002b:00007ffcce301ee8 EFLAGS: 00000246
>
> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=104943
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> ---
> drivers/gpu/drm/i915/i915_pmu.c | 29 +++++++++++++++++------------
> drivers/gpu/drm/i915/i915_pmu.h | 1 +
> 2 files changed, 18 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_pmu.c b/drivers/gpu/drm/i915/i915_pmu.c
> index ecb0198bfb7a..9c50b173e9a1 100644
> --- a/drivers/gpu/drm/i915/i915_pmu.c
> +++ b/drivers/gpu/drm/i915/i915_pmu.c
> @@ -423,18 +423,23 @@ static u64 __i915_pmu_event_read(struct perf_event *event)
> val = count_interrupts(i915);
> break;
> case I915_PMU_RC6_RESIDENCY:
> - intel_runtime_pm_get(i915);
> - val = intel_rc6_residency_ns(i915,
> - IS_VALLEYVIEW(i915) ?
> - VLV_GT_RENDER_RC6 :
> - GEN6_GT_GFX_RC6);
> - if (HAS_RC6p(i915))
> - val += intel_rc6_residency_ns(i915,
> - GEN6_GT_GFX_RC6p);
> - if (HAS_RC6pp(i915))
> - val += intel_rc6_residency_ns(i915,
> - GEN6_GT_GFX_RC6pp);
> - intel_runtime_pm_put(i915);
> + if (intel_runtime_pm_get_if_in_use(i915)) {
> + val = intel_rc6_residency_ns(i915,
> + IS_VALLEYVIEW(i915) ?
> + VLV_GT_RENDER_RC6 :
> + GEN6_GT_GFX_RC6);
> + if (HAS_RC6p(i915))
> + val += intel_rc6_residency_ns(i915,
> + GEN6_GT_GFX_RC6p);
> + if (HAS_RC6pp(i915))
> + val += intel_rc6_residency_ns(i915,
> + GEN6_GT_GFX_RC6pp);
> + intel_runtime_pm_put(i915);
> +
> + i915->pmu.sample[__I915_SAMPLE_RC6].cur = val;
> + } else {
> + val = i915->pmu.sample[__I915_SAMPLE_RC6].cur;
> + }
Downside is that idle system, when auto-suspended, will show 0% RC6. But
I have no alternative ideas. :(
> break;
> }
> }
> diff --git a/drivers/gpu/drm/i915/i915_pmu.h b/drivers/gpu/drm/i915/i915_pmu.h
> index 5a2e013a56bb..249983ed3f08 100644
> --- a/drivers/gpu/drm/i915/i915_pmu.h
> +++ b/drivers/gpu/drm/i915/i915_pmu.h
> @@ -27,6 +27,7 @@
> enum {
> __I915_SAMPLE_FREQ_ACT = 0,
> __I915_SAMPLE_FREQ_REQ,
> + __I915_SAMPLE_RC6,
> __I915_NUM_PMU_SAMPLERS
> };
>
>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
next prev parent reply other threads:[~2018-02-05 9:47 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-05 9:11 [PATCH] RFC drm/i915/pmu: Avoid sleeping rpm_get under atomic PMU read Chris Wilson
2018-02-05 9:40 ` ✓ Fi.CI.BAT: success for " Patchwork
2018-02-05 9:47 ` Tvrtko Ursulin [this message]
2018-02-05 9:50 ` [PATCH] " Chris Wilson
2018-02-05 10:31 ` ✓ Fi.CI.IGT: success for " Patchwork
2018-02-05 11:45 ` Tvrtko Ursulin
2018-02-05 13:18 ` Chris Wilson
2018-02-05 13:37 ` Tvrtko Ursulin
2018-02-05 13:43 ` Chris Wilson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=009586ed-8292-05b5-d925-86ca7a0c7363@linux.intel.com \
--to=tvrtko.ursulin@linux.intel.com \
--cc=chris@chris-wilson.co.uk \
--cc=intel-gfx@lists.freedesktop.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.