Linux Documentation
 help / color / mirror / Atom feed
From: Shrikanth Hegde <sshegde@linux.ibm.com>
To: Yury Norov <ynorov@nvidia.com>
Cc: linux-kernel@vger.kernel.org, mingo@kernel.org,
	peterz@infradead.org, juri.lelli@redhat.com,
	vincent.guittot@linaro.org, yury.norov@gmail.com,
	kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net,
	tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com,
	seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com,
	rostedt@goodmis.org, dietmar.eggemann@arm.com,
	maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com,
	chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org,
	arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com,
	tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org,
	rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com,
	linux-doc@vger.kernel.org
Subject: Re: [PATCH v6 20/23] virt/steal_monitor: Act on steal values at regular intervals
Date: Thu, 9 Jul 2026 10:08:01 +0530	[thread overview]
Message-ID: <3962bf6a-23c7-42a3-bd72-76580c0a9297@linux.ibm.com> (raw)
In-Reply-To: <ak6sO7eAfC-AWEWe@yury>



On 7/9/26 1:29 AM, Yury Norov wrote:
> On Wed, Jul 08, 2026 at 10:25:09PM +0530, Shrikanth Hegde wrote:
>>
>>
>> On 7/8/26 9:43 PM, Yury Norov wrote:
>>> On Wed, Jul 08, 2026 at 04:03:18PM +0530, Shrikanth Hegde wrote:
>>>>
>>>>
>>>> On 7/7/26 12:46 PM, Shrikanth Hegde wrote:
>>>>>
>>>>>
>>>>> On 7/7/26 2:03 AM, Yury Norov wrote:
>>>>>> On Wed, Jul 01, 2026 at 07:46:51PM +0530, Shrikanth Hegde wrote:
>>>>>>> This is the steal_monitor core functionality done in periodic work
>>>>>>>
>>>>>>> - Calculate the steal_ratio. It is multiplied by 100 to consider the
>>>>>>>      fractional values of steal time. I.e 10 means 0.1% steal time.
>>>>>>> - If steal value is higher than high threshold, call the method
>>>>>>> to reduce
>>>>>>>      the preferred CPUs.
>>>>>>> - If steal value is lower or equal to low threshold, call the method to
>>>>>>>      increase the preferred CPUs.
>>>>>>> - If the steal value is in between, no action is taken.
>>>>>>> - Save the values for next delta calculations.
>>>>>>>
>>>>>>> Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
>>>>>>> ---
>>>>>>> v5->v6:
>>>>>>> - Address u64 overflow concerns.
>>>>>>>
>>>>>>>     drivers/virt/steal_monitor/sm_core.c | 33 ++++++++++++++++++++++++++++
>>>>>>>     1 file changed, 33 insertions(+)
>>>>>>>
>>>>>>> diff --git a/drivers/virt/steal_monitor/sm_core.c
>>>>>>> b/drivers/virt/ steal_monitor/sm_core.c
>>>>>>> index b499faa61010..7b7435f79b85 100644
>>>>>>> --- a/drivers/virt/steal_monitor/sm_core.c
>>>>>>> +++ b/drivers/virt/steal_monitor/sm_core.c
>>>>>>> @@ -34,6 +34,37 @@ MODULE_PARM_DESC(low_threshold,
>>>>>>>     static void compute_preferred_cpus_work(struct work_struct *work)
>>>>>>>     {
>>>>>>> +    u64 curr_steal, delta_steal, delta_ns, steal_ratio;
>>>>>>> +    ktime_t now;
>>>>>>> +
>>>>>>> +    curr_steal = get_system_steal_time();
>>>>>>> +    now = ktime_get();
>>>>>>> +
>>>>>>> +    /* get the deltas */
>>>>>>> +    delta_steal = curr_steal > sm_core_ctx.prev_steal ?
>>>>>>> +              curr_steal - sm_core_ctx.prev_steal : 0;
>>>>>>> +    delta_ns = max_t(u64, ktime_to_ns(ktime_sub(now,
>>>>>>> sm_core_ctx.prev_time)), 1);
>>>>>>
>>>>>> The below return on '!delta_ns' makes this max(...) useless, right?
>>>>>> Regardless, if the time between 2 measures is less then 1ns, I
>>>>>> believe, the whole measure is not trustworthy
>>>>>>
>>>>>
>>>>> Seeing sashiko comment on divide by zero, i forgot I had max above there :(
>>>>>
>>>>>>> +
>>>>>>> +    /* Update for next calculation */
>>>>>>> +    sm_core_ctx.prev_steal = curr_steal;
>>>>>>> +    sm_core_ctx.prev_time = now;
>>>>>>
>>>>>> So below return should go prior to this update, because 'now' is actually
>>>>>> the same as 'prev_time', right?
>>>>>>
>>>>>> I don't understand why 'now' can be so close to prev_time, because
>>>>>> you've scheduled this callback on the regular interval. But if that's
>>>>>> possible, can you explain that and do like this at the very beginning
>>>>>> of the function:
>>>>>>
>>>>
>>>> I don't think it is possible.
>>>>
>>>>>>            now = ktime_get();
>>>>>>            if (unlikely(now < sm_core_ctx.prev_time +
>>>>>> sm_core_ctx.interval / 2)) {
>>>>>>                    pr_warn(...);
>>>>>>                    return;
>>>>>>            }
>>>>>
>>>>> Pretty much, just have to requeue the work.
>>>>>
>>>>>>
>>>>>> And if it's a never-happen condition, just use WARN_ON().
>>>>>>
>>>>
>>>> I have decided keep just max_t() to ensure no divide by zero happens.
>>>>
>>>> Issue with putting sm_core_ctx.interval_ms check is that, one may edit
>>>> it while the work is queued, but not yet called. It would print wrongly
>>>> in that case.
>>>
>>> Who is that 'one'? If it's userspace, you can simply disallow changing
>>> the interval. If it's kernel, you're already compromised, and
>>> everything is not trustworthy.
>>>
>>> I would simply disallow to change any parameters while the driver is
>>> loaded. If I need to change something - I unload it and load with new
>>> interval, or whatever.
>>
>> It is user writing to /sys/module/steal_monitor/parameters/interval_ms
>> that i thought of. So user.
>>
>> If we disable then those checks can be valid.
>>
>>>
>>>> Even if work function gets called prematurely, it should be fine since the
>>>> steal also increases accordingly. No issues in delta logic.
>>>>
>>>> So capping to 1 to ensure no divide by 0 happens due to scaling of
>>>> delta_ns is good enough.
>>> If there's 1 nanosecond between two calls, your statistics is very
>>> inaccurate, pretty much a noise. You want to make decision based on
>>> noise. That's simply wrong.
>>
>>
>> yes, that makes sense too. I have kept interval_ms/8 as safe bet.
>> Return will not queue the timer and effectively disables it.
>>
>>          ktime_t now;
>>
>>          now = ktime_get();
>>          delta_ns = ktime_to_ns(ktime_sub(now, sm_core_ctx.prev_time));
>>
>>          if (unlikely(delta_ns < sm_core_ctx.interval_ms * ((u64)NSEC_PER_MSEC >> 3))) {
>>                  pr_err("work scheduled too soon delta_time: %llu ns. Abort steal_monitor.\n",
>>                         delta_ns);
>>
>>                  guard(cpus_read_lock)();
>>                  cpumask_copy(&__cpu_preferred_mask, cpu_active_mask);
>>                  return;
>>          }
> 
> Don't overthink it. Simply don't allow user to change the parameters
> while the driver is running. It may only cause troubles to those
> compiling the driver into the Image. Those compiling it as a module
> will only have to reload the driver. This is how most of the kernel
> works.

Ok. I have made it that way.

I will just put a pr_err for delta_ns less than 1ms. If anyone hits it,
then we can bring in the logic to abort etc.

PS: I wasn;t able to hit it even with 1ms as interval and check for delta_ns as (1ms/8)

> 
> So, when the driver is loaded, make sanity checks for all user parameters.
> Interval, for example must be between 10ms and 10s. If user wants 20s,

I have it as 10ms and 100s in current v7.

> pr_err() something and fail to load. After the driver is loaded, don't
> allow to change the params exactly because it may bring certain type of
> errors in the logic, and you don't want to mitigate them.
> 
> If there will be a real need to adjust parameters on the fly - it will
> be another interesting topic (mostly interesting why user wants this).
> 

Makes sense. If the need arise, we will figure something out.
I have kept it simple for now.

         now = ktime_get();
         delta_ns = ktime_to_ns(ktime_sub(now, sm_core_ctx.prev_time));

         if (unlikely(delta_ns < NSEC_PER_MSEC)) {
                 pr_err_ratelimited("steal_monitor: work scheduled too soon delta_ns: %llu\n",
                                    delta_ns);
                 goto requeue_work;
         }


  reply	other threads:[~2026-07-09  4:38 UTC|newest]

Thread overview: 62+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-01 14:16 [PATCH v6 00/23] sched: Introduce cpu_preferred_mask and steal-driven vCPU backoff Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 01/23] sched/docs: Document cpu_preferred_mask and Preferred CPU concept Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 02/23] kconfig: Provide PREFERRED_CPU option Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 03/23] cpumask: Introduce cpu_preferred_mask Shrikanth Hegde
2026-07-01 15:35   ` Yury Norov
2026-07-01 16:40     ` Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 04/23] sysfs: Add preferred CPU file Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 05/23] sched/core: Try to use a preferred CPU in is_cpu_allowed Shrikanth Hegde
2026-07-01 16:09   ` Yury Norov
2026-07-01 16:49     ` Shrikanth Hegde
2026-07-02  6:30       ` Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 06/23] sched/fair: Load balance only among preferred CPUs Shrikanth Hegde
2026-07-01 16:19   ` Yury Norov
2026-07-01 16:41     ` Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 07/23] sched/fair: Pull the load on preferred CPU Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 08/23] sched/core: Keep tick on non-preferred CPUs until tasks are out Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 09/23] sched/core: Push current task from non preferred CPU Shrikanth Hegde
2026-07-01 16:50   ` Yury Norov
2026-07-01 17:03     ` Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 10/23] sched/debug: Add migration stats due to non preferred CPUs Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 11/23] virt/steal_monitor: Add documentation Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 12/23] virt: Introduce steal monitor driver Shrikanth Hegde
2026-07-03 18:20   ` Yury Norov
2026-07-03 18:27     ` Yury Norov
2026-07-06  8:24       ` Shrikanth Hegde
2026-07-06 18:49         ` Yury Norov
2026-07-07  6:47           ` Shrikanth Hegde
2026-07-06  8:09     ` Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 13/23] virt/steal_monitor: Restore to active on module disable Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 14/23] virt/steal_monitor: Define steal_monitor structure Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 15/23] virt/steal_monitor: Add control knobs for handling steal values Shrikanth Hegde
2026-07-02  7:16   ` Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 16/23] virt/steal_monitor: Compute work at regular intervals Shrikanth Hegde
2026-07-03 18:14   ` Yury Norov
2026-07-06  8:38     ` Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 17/23] virt/steal_monitor: Provide default method to get systemwide steal time Shrikanth Hegde
2026-07-03 19:16   ` Yury Norov
2026-07-06  8:47     ` Shrikanth Hegde
2026-07-06 19:04       ` Yury Norov
2026-07-07  6:50         ` Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 18/23] virt/steal_monitor: Provide default method to inc/dec preferred CPUs Shrikanth Hegde
2026-07-03 21:11   ` Yury Norov
2026-07-06  9:20     ` Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 19/23] virt/steal_monitor: Provide default method to get num of CPUs for steal ratio Shrikanth Hegde
2026-07-06 19:26   ` Yury Norov
2026-07-07  6:55     ` Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 20/23] virt/steal_monitor: Act on steal values at regular intervals Shrikanth Hegde
2026-07-02  7:18   ` Shrikanth Hegde
2026-07-06 20:33   ` Yury Norov
2026-07-07  7:16     ` Shrikanth Hegde
2026-07-08 10:33       ` Shrikanth Hegde
2026-07-08 16:13         ` Yury Norov
2026-07-08 16:55           ` Shrikanth Hegde
2026-07-08 19:59             ` Yury Norov
2026-07-09  4:38               ` Shrikanth Hegde [this message]
2026-07-01 14:16 ` [PATCH v6 21/23] virt/steal_monitor: Add direction control Shrikanth Hegde
2026-07-06 20:36   ` Yury Norov
2026-07-07  7:16     ` Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 22/23] virt/steal_monitor: Add design check of preferred subset of active Shrikanth Hegde
2026-07-01 14:16 ` [PATCH v6 23/23] virt/steal_monitor: Optimise decrease_preferred_cpus when all CPUs are housekeeping Shrikanth Hegde
2026-07-09  5:04 ` [PATCH v6 00/23] sched: Introduce cpu_preferred_mask and steal-driven vCPU backoff Yury Norov
2026-07-09  5:24   ` Shrikanth Hegde

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3962bf6a-23c7-42a3-bd72-76580c0a9297@linux.ibm.com \
    --to=sshegde@linux.ibm.com \
    --cc=arighi@nvidia.com \
    --cc=chleroy@kernel.org \
    --cc=christian.loehle@arm.com \
    --cc=corbet@lwn.net \
    --cc=dietmar.eggemann@arm.com \
    --cc=frederic@kernel.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=hdanton@sina.com \
    --cc=huschle@linux.ibm.com \
    --cc=iii@linux.ibm.com \
    --cc=juri.lelli@redhat.com \
    --cc=kernellwp@gmail.com \
    --cc=kprateek.nayak@amd.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maddy@linux.ibm.com \
    --cc=maz@kernel.org \
    --cc=mingo@kernel.org \
    --cc=pauld@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rafael@kernel.org \
    --cc=rdunlap@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=seanjc@google.com \
    --cc=srikar@linux.ibm.com \
    --cc=tglx@kernel.org \
    --cc=tj@kernel.org \
    --cc=tommaso.cucinotta@gmail.com \
    --cc=vincent.guittot@linaro.org \
    --cc=vineeth@bitbyteword.org \
    --cc=vschneid@redhat.com \
    --cc=ynorov@nvidia.com \
    --cc=yury.norov@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox