Re: vtime accounting

["Radim Krčmář" <rkrcmar@redhat.com>]

2017-03-14 21:01+0100, Christoffer Dall:
> On Tue, Mar 14, 2017 at 08:32:02PM +0100, Radim Krčmář wrote:
>> 2017-03-14 19:41+0100, Christoffer Dall:
>> > On Tue, Mar 14, 2017 at 06:09:45PM +0100, Paolo Bonzini wrote:
>> >> On 14/03/2017 17:58, Radim Krčmář wrote:
>> >> >> I assume there's a good reason why we call guest_enter() and
>> >> >> guest_exit() in the hot path on every KVM architecture?
>> >> > I consider myself biased when it comes to jiffies, so no judgement. :)
>> >> > 
>> >> > From what I see, the mode switch is used only for statistics.
>> >> 
>> >> vtime is only for statistics, but guest_enter/exit are important because
>> >> they enter an RCU extended quiescent state.  This means that (physical)
>> >> CPUs running a guest are effectively "off" from the point of view of the
>> >> RCU accounting machinery.  Not having to perform any RCU work is very
>> >> good for jitter.
>> 
>> Ah, good point.
>> 
>> > So would it be worth considering factoring out vtime accounting from
>> > guest_enter/exit, such that we could do the vtime accounting from vcpu
>> > load/put and mark the RCU extended quiescent state in the run loop?
>> 
>> RCU is the reason why guest_exit() needs disabled interrupts, so if we
>> split them, we could do rcu_virt_note_context_switch() before enabling
>> interrupts, and guest_exit() right after.
>> 
> 
> I'm not convinced that what you're saying is true ;)

I agree.

> I think we only fiddle with RCU during guest_enter, and further, a trace
> of guest_exit reveals:
> 
> guest_exit_irqoff
>  -> vtime_guest_exit
>     -> __vtime_account_system
>        -> get_vtime_delta
>           -> account_other_time
>              -> WARN_ON_ONCE(!irqs_disabled());
> 
> So I think we do need interrupts disabled when messing with vtime?

Seem like it.

>> > Disclaimer: I haven't completely convinced myself that vtime accounting
>> > from load/put works as it should.  For example, when servicing a VM from
>> > KVM, should we really be accounting this as kernel time, or as guest
>> > time?  I think we do the former now, but if the latter is the right
>> > thing, would changing the behavior constitute an ABI change to
>> > userspace?
>> 
>> Not considering that option would be best. :)
> 
> If my statement above about needing interrupts disabled when dealing
> with vtime, then considering this begins to sound interesting, also
> given that the vtime thing is not entirely free and we're dealing with
> the hot path of receiving IPIs here, for example.

I'm liking it less and less the more I read. :)
CONTEXT_USER vtime is coupled with context tracking and going out of
CONTEXT_KERNEL means that RCU cannot be used in between.  Using
CONTEXT_GUEST from load/put would change the meaning of contexts ...