Re: RUNSTATE_runnable delta time for idle_domain accounted to HVM guest.

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Tue, Apr 29, 2014 at 10:16:39AM +0100, George Dunlap wrote:
> On 04/23/2014 10:28 PM, Konrad Rzeszutek Wilk wrote:
> >What we are observing is that if a domain is idle its steal
> >time* goes up. My first thought was - well that is the initial
> >domain taking the time - but after looking at the trace
> >did not see the initial domain to be scheduled at all.
> >
> >(*steal time: RUNSTATE_runnable + RUNSTATE_offline).
> 
> "Up" like how much?

6.7msec. Or ~1/4 of the timeslice
> 
> Steal time includes the time *being woken* up.  It takes time to be
> woken up; typically if it's being woken up from domain 0, for
> instance, the wake (which sets it to RUNSTATE_runnable) will happen
> on a different pcpu than the vcpu being woken is on, so there's the
> delay of the IPI, waking up, going through the scheduler, &c.

Right. In this case there are no IPIs. Just softirq handlers being
triggered (by some other VCPU it seems) and they run.. And the
time between the 'vcpu_wake' and the 'schedule' softirq are quite
long.
> 
> The more frequently a VM is already running, the lower probability
> that an interrupt will actually wake it up.

Right. But there are no interrupt here at all. It is just idling.
> 
> BTW, is there a reason you're using xentrace_format instead of xenalyze?

I did use xenanalyze and it told me that the vCPU is busy spending most
of its time in 'runnable' condition. The other vCPUs are doing other
work.
> 
> hg clone http://xenbits.xenproject.org/ext/xenalyze
> 
> Unlike xentrace_format, xenalyze can:
> 1. Report the trace records in the order they happened across all
> pcpus, so you can see interactions between pcpus

Right. Was thinking to look back at that, but for right now I just
want to understand why there is this long delay between 'vcpu_wake'
and 'schedule'. Added more tracing to help with this.

> 2. Do its own runstate analysis on a per-vcpu level, allowing you to
> see not only how much time was spent in the "runnable" state, but
> how much of it was due to being woken up vs being preempted.

Hm, that would be interesting, but I think it will tell me exactly
the same thing - very long time from switching from blocked to
runnable and then being scheduled.
> 3. Allow you to see statistics on how long the "waking up" process
> took (average, and 5th/50th/95th %ile)
> 4. Give you a framework to allow you to easily add your own analysis
> if you want.  For instance, you could add statistics for how often a
> vcpu was woken up due to a local timer event vs woken up by an event
> from dom0 on another cpu, &c.

I had issues with that. It seems to require some special record
in the trace that sometimes I don't have.
> 
>  -George
>