Re: [RFC PATCH] tick/sched: Ensure quiet_vmstat() is called when the idle tick was stopped too

[Marcelo Tosatti <mtosatti@redhat.com>]

On Sat, Feb 19, 2022 at 03:46:16PM +0000, Aaron Tomlin wrote:
> On Fri 2022-02-18 12:54 +0000, Aaron Tomlin wrote:
> > On Thu 2022-02-17 17:32 +0100, Frederic Weisbecker wrote:
> > > > If I understand correctly, in the context of nohz_full, since such work is
> > > > deferred, it will only be handled in a scenario when the periodic/or
> > > > scheduling-clock tick is enabled i.e. the timer was reprogrammed on exit
> > > > from idle.
> > > 
> > > Oh I see, it's a deferrable delayed work...
> > > Then I can see two other issues:
> > > 
> > > 1) Can an interrupt in idle modify the vmstat and thus trigger the need to
> > >    flush it? 

Yes. Page allocation and page freeing for example.

   6   3730  ../mm/page_alloc.c <<rmqueue>>
             __mod_zone_freepage_state(zone, -(1 << order),
   4   1096  ../mm/page_alloc.c <<__free_one_page>>
             __mod_zone_freepage_state(zone, -(1 << order),

> > >    I believe it's the case and then the problem goes beyond nohz_full
> > >    because if the idle interrupt fired while the tick is stopped and didn't set
> > >    TIF_RESCHED, we go back to sleep without calling quiet_vmstat().
> > 
> > Yes: e.g. a nohz_full CPU, in idle code, could indeed receive a reschedule
> > IPI; re-enable local IRQs and generic idle code sees the TIF_NEED_RESCHED
> > flag against the idle task. Additionally, the selected task could
> > indirectly released a few pages [to satisfy a low-memory condition] and
> > modify CPU-specific vmstat data i.e. vm_stat_diff[NR_FREE_PAGES].
> > 
> > 
> > > 2) What if we are running task A in kernel mode while the tick is stopped
> > >    (nohz_full). Task A modifies the vmstat and goes to userspace for a long
> > >    while.
> > > Your patch fixes case 1) but not case 2). The problem is that TIMER_DEFERRABLE
> > > should really be about dynticks-idle only and not dynticks-full. I've always
> > > been afraid about enforcing that rule though because that would break old
> > > noise-free setups. But perhaps I should...
> > 
> > If I understand correctly, I agree. For the latter case, nothing can be
> > done unfortunately since the scheduling-clock tick is stopped.
> 
> Hi Frederic,
> 
> As far vmstat_updateas I understand, in the context of nohz_full, options are indeed
> limited; albeit, if we can ensure CPU-specific vmstat data is folded on
> return to idle [when required] then this should be good enough.

I suppose the desired behaviour, with the deferred timer for vmstat_sync, is:

"Allow the per-CPU vmstats to be out of sync, but for a maximum of 
sysctl_stat_interval".

But Aaron, vmstat_shepherd should be ensuring that per-CPU vmstat_update
work are queued, if the per-CPU vmstat are out of sync.

And:

static void
trigger_dyntick_cpu(struct timer_base *base, struct timer_list *timer)
{
        if (!is_timers_nohz_active())
                return;

        /*
         * TODO: This wants some optimizing similar to the code below, but we
         * will do that when we switch from push to pull for deferrable timers.
         */
        if (timer->flags & TIMER_DEFERRABLE) {
                if (tick_nohz_full_cpu(base->cpu))
                        wake_up_nohz_cpu(base->cpu);
                return;
        }

 * @TIMER_DEFERRABLE: A deferrable timer will work normally when the
 * system is busy, but will not cause a CPU to come out of idle just
 * to service it; instead, the timer will be serviced when the CPU
 * eventually wakes up with a subsequent non-deferrable timer.

You'd want that vmstat_update to execute regardless of whether there are 
armed non-deferrable timers.

Should fix both 1 and 2 AFAICS.