Re: vtime accounting

[Christoffer Dall <cdall@linaro.org>]

On Tue, Mar 14, 2017 at 08:32:02PM +0100, Radim Krčmář wrote:
> 2017-03-14 19:41+0100, Christoffer Dall:
> > On Tue, Mar 14, 2017 at 06:09:45PM +0100, Paolo Bonzini wrote:
> >> On 14/03/2017 17:58, Radim Krčmář wrote:
> >> >> I assume there's a good reason why we call guest_enter() and
> >> >> guest_exit() in the hot path on every KVM architecture?
> >> > I consider myself biased when it comes to jiffies, so no judgement. :)
> >> > 
> >> > From what I see, the mode switch is used only for statistics.
> >> 
> >> vtime is only for statistics, but guest_enter/exit are important because
> >> they enter an RCU extended quiescent state.  This means that (physical)
> >> CPUs running a guest are effectively "off" from the point of view of the
> >> RCU accounting machinery.  Not having to perform any RCU work is very
> >> good for jitter.
> 
> Ah, good point.
> 
> > So would it be worth considering factoring out vtime accounting from
> > guest_enter/exit, such that we could do the vtime accounting from vcpu
> > load/put and mark the RCU extended quiescent state in the run loop?
> 
> RCU is the reason why guest_exit() needs disabled interrupts, so if we
> split them, we could do rcu_virt_note_context_switch() before enabling
> interrupts, and guest_exit() right after.
> 

I'm not convinced that what you're saying is true ;)

I think we only fiddle with RCU during guest_enter, and further, a trace
of guest_exit reveals:

guest_exit_irqoff
 -> vtime_guest_exit
    -> __vtime_account_system
       -> get_vtime_delta
          -> account_other_time
             -> WARN_ON_ONCE(!irqs_disabled());

So I think we do need interrupts disabled when messing with vtime?

> > Disclaimer: I haven't completely convinced myself that vtime accounting
> > from load/put works as it should.  For example, when servicing a VM from
> > KVM, should we really be accounting this as kernel time, or as guest
> > time?  I think we do the former now, but if the latter is the right
> > thing, would changing the behavior constitute an ABI change to
> > userspace?
> 
> Not considering that option would be best. :)

If my statement above about needing interrupts disabled when dealing
with vtime, then considering this begins to sound interesting, also
given that the vtime thing is not entirely free and we're dealing with
the hot path of receiving IPIs here, for example.

Thanks,
-Christoffer