From: Jon Hunter <jonathanh@nvidia.com>
To: Juri Lelli <juri.lelli@redhat.com>
Cc: Thierry Reding <treding@nvidia.com>,
Waiman Long <longman@redhat.com>, Tejun Heo <tj@kernel.org>,
Johannes Weiner <hannes@cmpxchg.org>,
Michal Koutny <mkoutny@suse.com>, Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Vincent Guittot <vincent.guittot@linaro.org>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
Valentin Schneider <vschneid@redhat.com>,
Phil Auld <pauld@redhat.com>, Qais Yousef <qyousef@layalina.io>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
"Joel Fernandes (Google)" <joel@joelfernandes.org>,
Suleiman Souhlal <suleiman@google.com>,
Aashish Sharma <shraash@google.com>,
Shin Kawamura <kawasin@google.com>,
Vineeth Remanan Pillai <vineeth@bitbyteword.org>,
linux-kernel@vger.kernel.org, cgroups@vger.kernel.org,
"linux-tegra@vger.kernel.org" <linux-tegra@vger.kernel.org>
Subject: Re: [PATCH v2 3/2] sched/deadline: Check bandwidth overflow earlier for hotplug
Date: Wed, 5 Feb 2025 16:56:30 +0000 [thread overview]
Message-ID: <8572b3bc-46ec-4180-ba55-aa6b9ab7502b@nvidia.com> (raw)
In-Reply-To: <Z6M5fQB9P1_bDF7A@jlelli-thinkpadt14gen4.remote.csb>
Hi Juri,
On 05/02/2025 10:12, Juri Lelli wrote:
> On 05/02/25 07:53, Juri Lelli wrote:
>> On 03/02/25 11:01, Jon Hunter wrote:
>>> Hi Juri,
>>>
>>> On 16/01/2025 15:55, Juri Lelli wrote:
>>>> On 16/01/25 13:14, Jon Hunter wrote:
>>
>> ...
>>
>>>>> [ 210.595431] dl_bw_manage: cpu=5 cap=3072 fair_server_bw=52428 total_bw=209712 dl_bw_cpus=4
>>>>> [ 210.606269] dl_bw_manage: cpu=4 cap=2048 fair_server_bw=52428 total_bw=157284 dl_bw_cpus=3
>>>>> [ 210.617281] dl_bw_manage: cpu=3 cap=1024 fair_server_bw=52428 total_bw=104856 dl_bw_cpus=2
>>>>> [ 210.627205] dl_bw_manage: cpu=2 cap=1024 fair_server_bw=52428 total_bw=262140 dl_bw_cpus=2
>>>>> [ 210.637752] dl_bw_manage: cpu=1 cap=0 fair_server_bw=52428 total_bw=262140 dl_bw_cpus=1
>>>> ^
>>>> Different than before but still not what I expected. Looks like there
>>>> are conditions/path I currently cannot replicate on my setup, so more
>>>> thinking. Unfortunately I will be out traveling next week, so this
>>>> might required a bit of time.
>>>
>>>
>>> I see that this is now in the mainline and our board is still failing to
>>> suspend. Let me know if there is anything else you need me to test.
>>
>> Ah, can you actually add 'sched_verbose' and to your kernel cmdline? It
>> should print our additional debug info on the console when domains get
>> reconfigured by hotplug/suspends, e.g.
>>
>> dl_bw_manage: cpu=3 cap=3072 fair_server_bw=52428 total_bw=209712 dl_bw_cpus=4
>> CPU0 attaching NULL sched-domain.
>> CPU3 attaching NULL sched-domain.
>> CPU4 attaching NULL sched-domain.
>> CPU5 attaching NULL sched-domain.
>> CPU0 attaching sched-domain(s):
>> domain-0: span=0,4-5 level=MC
>> groups: 0:{ span=0 cap=766 }, 4:{ span=4 cap=908 }, 5:{ span=5 cap=989 }
>> CPU4 attaching sched-domain(s):
>> domain-0: span=0,4-5 level=MC
>> groups: 4:{ span=4 cap=908 }, 5:{ span=5 cap=989 }, 0:{ span=0 cap=766 }
>> CPU5 attaching sched-domain(s):
>> domain-0: span=0,4-5 level=MC
>> groups: 5:{ span=5 cap=989 }, 0:{ span=0 cap=766 }, 4:{ span=4 cap=908 }
>> root domain span: 0,4-5
>> rd 0,4-5: Checking EAS, CPUs do not have asymmetric capacities
>> psci: CPU3 killed (polled 0 ms)
>>
>> Can you please share this information as well if you are able to collect
>> it (while still running with my last proposed fix)?
>
> Also, if you don't mind, add the following on top of the existing
> changes.
>
> Just to be sure we don't get out of sync, I pushed current set to
>
> https://github.com/jlelli/linux.git experimental/dl-debug
Thanks! That did make it easier :-)
Here is what I see ...
[ 53.823979] PM: suspend entry (deep)
[ 53.827715] Filesystems sync: 0.000 seconds
[ 53.832859] Freezing user space processes
[ 53.838132] Freezing user space processes completed (elapsed 0.001 seconds)
[ 53.845118] OOM killer disabled.
[ 53.848348] Freezing remaining freezable tasks
[ 53.853884] Freezing remaining freezable tasks completed (elapsed 0.001 seconds)
[ 53.900686] tegra-xusb 3530000.usb: Firmware timestamp: 2020-07-06 13:39:28 UTC
[ 53.918492] dwc-eth-dwmac 2490000.ethernet eth0: Link is Down
[ 53.962316] Disabling non-boot CPUs ...
[ 53.966192] dl_bw_manage: cpu=5 cap=3072 fair_server_bw=52428 total_bw=209712 dl_bw_cpus=4 type=DYN span=0,3-5
[ 53.966231] CPU0 attaching NULL sched-domain.
[ 53.980574] span=1-2
[ 53.982767] CPU3 attaching NULL sched-domain.
[ 53.987119] span=0-2
[ 53.989309] CPU4 attaching NULL sched-domain.
[ 53.993662] span=0-3
[ 53.995853] CPU5 attaching NULL sched-domain.
[ 54.000206] span=0-4
[ 54.002433] CPU0 attaching sched-domain(s):
[ 54.006614] domain-0: span=0,3-4 level=MC
[ 54.010711] groups: 0:{ span=0 cap=1022 }, 3:{ span=3 cap=1022 }, 4:{ span=4 }
[ 54.018126] CPU3 attaching sched-domain(s):
[ 54.022307] domain-0: span=0,3-4 level=MC
[ 54.026404] groups: 3:{ span=3 cap=1022 }, 4:{ span=4 }, 0:{ span=0 cap=1023 }
[ 54.033821] CPU4 attaching sched-domain(s):
[ 54.038001] domain-0: span=0,3-4 level=MC
[ 54.042098] groups: 4:{ span=4 }, 0:{ span=0 cap=1023 }, 3:{ span=3 cap=1022 }
[ 54.049508] root domain span: 0,3-4
[ 54.052997] default domain span: 1-2,5
[ 54.056756] rd 0,3-4: Checking EAS, CPUs do not have asymmetric capacities
[ 54.064688] psci: CPU5 killed (polled 0 ms)
[ 54.069495] dl_bw_manage: cpu=4 cap=2048 fair_server_bw=52428 total_bw=157284 dl_bw_cpus=3 type=DYN span=0,3-4
[ 54.069547] CPU0 attaching NULL sched-domain.
[ 54.083910] span=1-2,5
[ 54.086277] CPU3 attaching NULL sched-domain.
[ 54.090633] span=0-2,5
[ 54.092999] CPU4 attaching NULL sched-domain.
[ 54.097351] span=0-3,5
[ 54.099756] CPU0 attaching sched-domain(s):
[ 54.103941] domain-0: span=0,3 level=MC
[ 54.107865] groups: 0:{ span=0 }, 3:{ span=3 cap=1023 }
[ 54.113279] CPU3 attaching sched-domain(s):
[ 54.117459] domain-0: span=0,3 level=MC
[ 54.121382] groups: 3:{ span=3 cap=1023 }, 0:{ span=0 }
[ 54.126793] root domain span: 0,3
[ 54.130109] default domain span: 1-2,4-5
[ 54.134040] rd 0,3: Checking EAS, CPUs do not have asymmetric capacities
[ 54.141597] psci: CPU4 killed (polled 0 ms)
[ 54.146819] dl_bw_manage: cpu=3 cap=1024 fair_server_bw=52428 total_bw=104856 dl_bw_cpus=2 type=DYN span=0,3
[ 54.156727] CPU0 attaching NULL sched-domain.
[ 54.161086] span=1-2,4-5
[ 54.163632] CPU3 attaching NULL sched-domain.
[ 54.167988] span=0-2,4-5
[ 54.170553] CPU0 attaching NULL sched-domain.
[ 54.174909] span=0-5
[ 54.177096] root domain span: 0
[ 54.180239] default domain span: 1-5
[ 54.183821] rd 0: Checking EAS, CPUs do not have asymmetric capacities
[ 54.191728] psci: CPU3 killed (polled 4 ms)
[ 54.196389] dl_bw_manage: cpu=2 cap=1024 fair_server_bw=52428 total_bw=262140 dl_bw_cpus=2 type=DEF span=1-5
[ 54.196518] rd 0: Checking EAS, CPUs do not have asymmetric capacities
[ 54.214816] psci: CPU2 killed (polled 0 ms)
[ 54.219411] dl_bw_manage: cpu=1 cap=0 fair_server_bw=52428 total_bw=262140 dl_bw_cpus=1 type=DEF span=1-5
[ 54.219493] Error taking CPU1 down: -16
[ 54.232948] Non-boot CPUs are not disabled
[ 54.237046] Enabling non-boot CPUs ...
[ 54.241216] Detected PIPT I-cache on CPU2
[ 54.245258] CPU features: SANITY CHECK: Unexpected variation in SYS_CTR_EL0. Boot CPU: 0x0000008444c004, CPU2: 0x0000009444c004
[ 54.256744] CPU features: SANITY CHECK: Unexpected variation in SYS_ID_AA64DFR0_EL1. Boot CPU: 0x00000010305106, CPU2: 0x00000010305116
[ 54.268954] CPU features: SANITY CHECK: Unexpected variation in SYS_ID_DFR0_EL1. Boot CPU: 0x00000003010066, CPU2: 0x00000003001066
[ 54.280865] CPU2: Booted secondary processor 0x0000000001 [0x4e0f0030]
[ 54.288270] rd 0: Checking EAS, CPUs do not have asymmetric capacities
[ 54.295061] CPU2 is up
[ 54.297599] Detected PIPT I-cache on CPU3
[ 54.301642] CPU3: Booted secondary processor 0x0000000101 [0x411fd073]
[ 54.308419] CPU0 attaching NULL sched-domain.
[ 54.312786] span=1-5
[ 54.315031] CPU0 attaching sched-domain(s):
[ 54.319220] domain-0: span=0,3 level=MC
[ 54.323145] groups: 0:{ span=0 }, 3:{ span=3 cap=1016 }
[ 54.328564] CPU3 attaching sched-domain(s):
[ 54.332746] domain-0: span=0,3 level=MC
[ 54.336671] groups: 3:{ span=3 cap=1016 }, 0:{ span=0 }
[ 54.342080] root domain span: 0,3
[ 54.345405] default domain span: 1-2,4-5
[ 54.349338] rd 0,3: Checking EAS, CPUs do not have asymmetric capacities
[ 54.356122] CPU3 is up
[ 54.358649] Detected PIPT I-cache on CPU4
[ 54.362677] CPU4: Booted secondary processor 0x0000000102 [0x411fd073]
[ 54.369399] CPU0 attaching NULL sched-domain.
[ 54.373767] span=1-2,4-5
[ 54.376310] CPU3 attaching NULL sched-domain.
[ 54.380667] span=0-2,4-5
[ 54.383251] CPU0 attaching sched-domain(s):
[ 54.387439] domain-0: span=0,3-4 level=MC
[ 54.391538] groups: 0:{ span=0 }, 3:{ span=3 cap=1021 }, 4:{ span=4 }
[ 54.398173] CPU3 attaching sched-domain(s):
[ 54.402356] domain-0: span=0,3-4 level=MC
[ 54.406456] groups: 3:{ span=3 cap=1021 }, 4:{ span=4 }, 0:{ span=0 }
[ 54.413090] CPU4 attaching sched-domain(s):
[ 54.417271] domain-0: span=0,3-4 level=MC
[ 54.421373] groups: 4:{ span=4 }, 0:{ span=0 }, 3:{ span=3 cap=1021 }
[ 54.428005] root domain span: 0,3-4
[ 54.431503] default domain span: 1-2,5
[ 54.435259] rd 0,3-4: Checking EAS, CPUs do not have asymmetric capacities
[ 54.442287] CPU4 is up
[ 54.444821] Detected PIPT I-cache on CPU5
[ 54.448848] CPU5: Booted secondary processor 0x0000000103 [0x411fd073]
[ 54.455574] CPU0 attaching NULL sched-domain.
[ 54.459950] span=1-2,5
[ 54.462315] CPU3 attaching NULL sched-domain.
[ 54.466674] span=0-2,5
[ 54.469042] CPU4 attaching NULL sched-domain.
[ 54.473401] span=0-3,5
[ 54.475812] CPU0 attaching sched-domain(s):
[ 54.480000] domain-0: span=0,3-5 level=MC
[ 54.484099] groups: 0:{ span=0 }, 3:{ span=3 }, 4:{ span=4 }, 5:{ span=5 }
[ 54.491171] CPU3 attaching sched-domain(s):
[ 54.495352] domain-0: span=0,3-5 level=MC
[ 54.499452] groups: 3:{ span=3 }, 4:{ span=4 }, 5:{ span=5 }, 0:{ span=0 }
[ 54.506519] CPU4 attaching sched-domain(s):
[ 54.510703] domain-0: span=0,3-5 level=MC
[ 54.514800] groups: 4:{ span=4 }, 5:{ span=5 }, 0:{ span=0 }, 3:{ span=3 }
[ 54.521869] CPU5 attaching sched-domain(s):
[ 54.526050] domain-0: span=0,3-5 level=MC
[ 54.530150] groups: 5:{ span=5 }, 0:{ span=0 }, 3:{ span=3 }, 4:{ span=4 }
[ 54.537217] root domain span: 0,3-5
[ 54.540716] default domain span: 1-2
[ 54.544303] rd 0,3-5: Checking EAS, CPUs do not have asymmetric capacities
[ 54.551393] rd 0,3-5: Checking EAS, CPUs do not have asymmetric capacities
[ 54.558281] CPU5 is up
[ 54.568000] dwc-eth-dwmac 2490000.ethernet eth0: configuring for phy/rgmii link mode
[ 55.585391] dwc-eth-dwmac 2490000.ethernet: Failed to reset the dma
[ 55.591664] dwc-eth-dwmac 2490000.ethernet eth0: stmmac_hw_setup: DMA engine initialization failed
[ 55.600905] dwc-eth-dwmac 2490000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 55.615015] usb-conn-gpio 3520000.padctl:ports:usb2-0:connector: repeated role: device
[ 55.617967] tegra-xusb 3530000.usb: Firmware timestamp: 2020-07-06 13:39:28 UTC
[ 55.655665] OOM killer enabled.
[ 55.658813] Restarting tasks ... done.
[ 55.664082] random: crng reseeded on system resumption
[ 55.664403] VDDIO_SDMMC3_AP: voltage operation not allowed
[ 55.674862] PM: suspend exit
--
nvpublic
next prev parent reply other threads:[~2025-02-05 16:56 UTC|newest]
Thread overview: 80+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-14 14:28 [PATCH v2 0/2] Fix DEADLINE bandwidth accounting in root domain changes and hotplug Juri Lelli
2024-11-14 14:28 ` [PATCH v2 1/2] sched/deadline: Restore dl_server bandwidth on non-destructive root domain changes Juri Lelli
2024-11-14 15:56 ` Phil Auld
2024-11-14 14:28 ` [PATCH v2 2/2] sched/deadline: Correctly account for allocated bandwidth during hotplug Juri Lelli
2024-11-14 15:58 ` Phil Auld
2024-12-06 10:43 ` Dan Carpenter
2024-12-09 14:20 ` Juri Lelli
2024-11-14 15:48 ` [PATCH v2 0/2] Fix DEADLINE bandwidth accounting in root domain changes and hotplug Waiman Long
2024-11-14 16:14 ` Juri Lelli
2024-11-14 18:16 ` Waiman Long
2024-11-14 18:43 ` Phil Auld
2024-11-15 11:48 ` [PATCH v2 3/2] sched/deadline: Check bandwidth overflow earlier for hotplug Juri Lelli
2025-01-10 11:52 ` Jon Hunter
2025-01-10 15:45 ` Juri Lelli
2025-01-10 18:40 ` Jon Hunter
2025-01-13 9:32 ` Juri Lelli
2025-01-13 13:53 ` Jon Hunter
2025-01-14 13:52 ` Jon Hunter
2025-01-14 14:02 ` Juri Lelli
2025-01-15 16:10 ` Juri Lelli
2025-01-16 13:14 ` Jon Hunter
2025-01-16 15:55 ` Juri Lelli
2025-02-03 11:01 ` Jon Hunter
2025-02-04 17:26 ` Juri Lelli
2025-02-05 6:53 ` Juri Lelli
2025-02-05 10:12 ` Juri Lelli
2025-02-05 16:56 ` Jon Hunter [this message]
2025-02-06 9:29 ` Juri Lelli
2025-02-07 10:38 ` Jon Hunter
2025-02-07 13:38 ` Dietmar Eggemann
2025-02-07 14:04 ` Jon Hunter
2025-02-07 15:55 ` Christian Loehle
2025-02-10 17:09 ` Juri Lelli
2025-02-11 8:36 ` Dietmar Eggemann
2025-02-11 9:21 ` Juri Lelli
2025-02-11 10:43 ` Dietmar Eggemann
2025-02-11 10:15 ` Christian Loehle
2025-02-11 10:42 ` Juri Lelli
2025-02-12 18:22 ` Dietmar Eggemann
2025-02-13 6:20 ` Juri Lelli
2025-02-13 12:27 ` Christian Loehle
2025-02-13 13:33 ` Juri Lelli
2025-02-13 13:38 ` Christian Loehle
2025-02-13 14:51 ` Juri Lelli
2025-02-13 14:57 ` Christian Loehle
2025-02-16 16:33 ` Qais Yousef
2025-02-17 14:52 ` Juri Lelli
2025-02-22 23:59 ` Qais Yousef
2025-02-24 9:27 ` Juri Lelli
2025-02-25 0:02 ` Qais Yousef
2025-02-25 9:46 ` Juri Lelli
2025-02-25 10:09 ` Christian Loehle
2025-02-12 23:01 ` Jon Hunter
2025-02-13 6:16 ` Juri Lelli
2025-02-13 9:53 ` Jon Hunter
2025-02-14 10:05 ` Jon Hunter
2025-02-17 16:08 ` Juri Lelli
2025-02-17 16:10 ` Jon Hunter
2025-02-17 16:25 ` Juri Lelli
2025-02-18 9:58 ` Juri Lelli
2025-02-18 10:30 ` Juri Lelli
2025-02-18 14:12 ` Dietmar Eggemann
2025-02-18 14:18 ` Juri Lelli
2025-02-19 9:29 ` Dietmar Eggemann
2025-02-19 10:02 ` Juri Lelli
2025-02-19 11:23 ` Jon Hunter
2025-02-19 13:09 ` Dietmar Eggemann
2025-02-19 18:14 ` Dietmar Eggemann
2025-02-20 10:40 ` Juri Lelli
2025-02-20 15:25 ` Juri Lelli
2025-02-21 11:56 ` Jon Hunter
2025-02-21 14:45 ` Dietmar Eggemann
2025-02-24 13:53 ` Dietmar Eggemann
2025-02-24 14:03 ` Juri Lelli
2025-02-24 23:39 ` Jon Hunter
2025-02-25 9:48 ` Juri Lelli
2025-03-03 14:17 ` Jon Hunter
2025-03-03 16:00 ` Juri Lelli
2025-02-07 14:04 ` Jon Hunter
2025-02-07 15:52 ` Juri Lelli
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8572b3bc-46ec-4180-ba55-aa6b9ab7502b@nvidia.com \
--to=jonathanh@nvidia.com \
--cc=bigeasy@linutronix.de \
--cc=bsegall@google.com \
--cc=cgroups@vger.kernel.org \
--cc=dietmar.eggemann@arm.com \
--cc=hannes@cmpxchg.org \
--cc=joel@joelfernandes.org \
--cc=juri.lelli@redhat.com \
--cc=kawasin@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tegra@vger.kernel.org \
--cc=longman@redhat.com \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=mkoutny@suse.com \
--cc=pauld@redhat.com \
--cc=peterz@infradead.org \
--cc=qyousef@layalina.io \
--cc=rostedt@goodmis.org \
--cc=shraash@google.com \
--cc=suleiman@google.com \
--cc=tj@kernel.org \
--cc=treding@nvidia.com \
--cc=vincent.guittot@linaro.org \
--cc=vineeth@bitbyteword.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox