Re: vtime accounting

Radim Krčmář
2017-03-14T21:52:34Z

2017-03-14 21:01+0100, Christoffer Dall:
> On Tue, Mar 14, 2017 at 08:32:02PM +0100, Radim Krčmář wrote:
>> 2017-03-14 19:41+0100, Christoffer Dall:
>> > On Tue, Mar 14, 2017 at 06:09:45PM +0100, Paolo Bonzini wrote:
>> >> On 14/03/2017 17:58, Radim Krčmář wrote:
>> >> >> I assume there's a good reason why we call guest_enter() and
>> >> >> guest_exit() in the hot path on every KVM architecture?
>> >> > I consider myself biased when it comes to jiffies, so no judgement. :)
>> >> >
>> >> > From what I see, the mode switch is used only for statistics.
>> >>
>> >> vtime is only for statistics, but guest_enter/exit are important because
>> >> they enter an RCU extended quiescent state. This means that (physical)
>> >> CPUs running a guest are effectively "off" from the point of view of the
>> >> RCU accounting machinery. Not having to perform any RCU work is very
>> >> good for jitter.
>>
>> Ah, good point.
>>
>> > So would it be worth considering factoring out vtime accounting from
>> > guest_enter/exit, such that we could do the vtime accounting from vcpu
>> > load/put and mark the RCU extended quiescent state in the run loop?
>>
>> RCU is the reason why guest_exit() needs disabled interrupts, so if we
>> split them, we could do rcu_virt_note_context_switch() before enabling
>> interrupts, and guest_exit() right after.
>>
>
> I'm not convinced that what you're saying is true ;)

I agree.

> I think we only fiddle with RCU during guest_enter, and further, a trace
> of guest_exit reveals:
>
> guest_exit_irqoff
> -> vtime_guest_exit
> -> __vtime_account_system
> -> get_vtime_delta
> -> account_other_time
> -> WARN_ON_ONCE(!irqs_disabled());
>
> So I think we do need interrupts disabled when messing with vtime?

Seem like it.

>> > Disclaimer: I haven't completely convinced myself that vtime accounting
>> > from load/put works as it should. For example, when servicing a VM from
>> > KVM, should we really be accounting this as kernel time, or as guest
>> > time? I think we do the former now, but if the latter is the right
>> > thing, would changing the behavior constitute an ABI change to
>> > userspace?
>>
>> Not considering that option would be best. :)
>
> If my statement above about needing interrupts disabled when dealing
> with vtime, then considering this begins to sound interesting, also
> given that the vtime thing is not entirely free and we're dealing with
> the hot path of receiving IPIs here, for example.

I'm liking it less and less the more I read. :)
CONTEXT_USER vtime is coupled with context tracking and going out of
CONTEXT_KERNEL means that RCU cannot be used in between. Using
CONTEXT_GUEST from load/put would change the meaning of contexts ...