proc/stat: idle goes backward

proc/stat: idle goes backward

[Oleg Nesterov]

Hello.

Out customer reports that "idle" field is not monotonic. So far this
is all I know. I do not know how to reproduce, etc.

But when I look at this code, this looks really possible even
ignoring drivers/cpuidle/ which plays with update_ts_time_stats().

So, get_cpu_idle_time_us(last_update_time => NULL) does:

	if (ts->idle_active && !nr_iowait_cpu(cpu)) {
		ktime_t delta = ktime_sub(now, ts->idle_entrytime);

		idle = ktime_add(ts->idle_sleeptime, delta);
	} else {
		idle = ts->idle_sleeptime;
	}


Suppose that ts->idle_active == T. By the time we calculate

	idle = ktime_add(ts->idle_sleeptime, delta);

this cpu can be already non-idle and ->idle_sleeptime can be already
updated by tick_nohz_stop_idle(), we return the wrong value.

If user-space reads /proc/stat again after that, "idle" can obviously
go back.

get_cpu_iowait_time_us() has the same problem.

Plus nr_iowait_cpu() can change in between even if cpu stays idle,
io_schedule() can return on another CPU.

Questions:

	- Any other reason why it can be non-monotonic?

	- Should we fix this or should we document that userspace
	  should handle this itself?

	  IOW, is this is bug or not?

Oleg.

Re: proc/stat: idle goes backward

[Frederic Weisbecker]

On Fri, Aug 16, 2013 at 04:47:04PM +0200, Oleg Nesterov wrote:
> Hello.
> 
> Out customer reports that "idle" field is not monotonic. So far this
> is all I know. I do not know how to reproduce, etc.
> 
> But when I look at this code, this looks really possible even
> ignoring drivers/cpuidle/ which plays with update_ts_time_stats().
> 
> So, get_cpu_idle_time_us(last_update_time => NULL) does:
> 
> 	if (ts->idle_active && !nr_iowait_cpu(cpu)) {
> 		ktime_t delta = ktime_sub(now, ts->idle_entrytime);
> 
> 		idle = ktime_add(ts->idle_sleeptime, delta);
> 	} else {
> 		idle = ts->idle_sleeptime;
> 	}
> 
> 
> Suppose that ts->idle_active == T. By the time we calculate
> 
> 	idle = ktime_add(ts->idle_sleeptime, delta);
> 
> this cpu can be already non-idle and ->idle_sleeptime can be already
> updated by tick_nohz_stop_idle(), we return the wrong value.
> 
> If user-space reads /proc/stat again after that, "idle" can obviously
> go back.
> 
> get_cpu_iowait_time_us() has the same problem.
> 
> Plus nr_iowait_cpu() can change in between even if cpu stays idle,
> io_schedule() can return on another CPU.
> 
> Questions:
> 
> 	- Any other reason why it can be non-monotonic?
> 
> 	- Should we fix this or should we document that userspace
> 	  should handle this itself?
> 
> 	  IOW, is this is bug or not?

I don't know if we want to fix it (I personally think we should because it is not the
first time I hear complains about this) but if we do, here is a possible fix:

https://lkml.org/lkml/2013/8/8/638

Thanks.

Re: proc/stat: idle goes backward

[Oleg Nesterov]

On 08/16, Frederic Weisbecker wrote:
>
> On Fri, Aug 16, 2013 at 04:47:04PM +0200, Oleg Nesterov wrote:
> >
> > Questions:
> >
> > 	- Any other reason why it can be non-monotonic?
> >
> > 	- Should we fix this or should we document that userspace
> > 	  should handle this itself?
> >
> > 	  IOW, is this is bug or not?
>
> I don't know if we want to fix it (I personally think we should because it is not the
> first time I hear complains about this) but if we do, here is a possible fix:
>
> https://lkml.org/lkml/2013/8/8/638

Thanks! it is not easy to read the patches on lkml.org and I do
not understand this code enough. But it seems that this should
address my concerns, including the "even ignoring drivers/cpuidle/".

Except, I am not sure that these changes handle the case when
nr_iowait() change in between.

Are you going to resend?

Oleg.

Re: proc/stat: idle goes backward

[Frederic Weisbecker]

On Fri, Aug 16, 2013 at 05:13:39PM +0200, Oleg Nesterov wrote:
> On 08/16, Frederic Weisbecker wrote:
> >
> > On Fri, Aug 16, 2013 at 04:47:04PM +0200, Oleg Nesterov wrote:
> > >
> > > Questions:
> > >
> > > 	- Any other reason why it can be non-monotonic?
> > >
> > > 	- Should we fix this or should we document that userspace
> > > 	  should handle this itself?
> > >
> > > 	  IOW, is this is bug or not?
> >
> > I don't know if we want to fix it (I personally think we should because it is not the
> > first time I hear complains about this) but if we do, here is a possible fix:
> >
> > https://lkml.org/lkml/2013/8/8/638
> 
> Thanks! it is not easy to read the patches on lkml.org and I do
> not understand this code enough. But it seems that this should
> address my concerns, including the "even ignoring drivers/cpuidle/".
> 
> Except, I am not sure that these changes handle the case when
> nr_iowait() change in between.

Yeah it handles iowait and idle sleeps.

> 
> Are you going to resend?

I'm more than happy that you want to review these patches.
I'm resending.

Thanks.

> 
> Oleg.
>