Re: [Qemu-devel] [Qemu-ppc] [PATCH RFC v1 0/3] Enable MTTCG on PPC64

["Alex Bennée" <alex.bennee@linaro.org>]

Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:

> On Tue, 2017-04-11 at 14:28 +0200, Cédric Le Goater wrote:
>> I really don't know.
>>
>> Ben, now that we have mttcg activated by default on ppc, it takes
>> a while for the linux kernel to do the early setup. I think we are
>> in the code section where we spin loop the secondaries. Most of the
>> time is spent in skiboot under cpu_idle/relax.
>>
>> Any idea where that could come from ?
>>
>> > See c22edfebff29f63d793032e4fbd42a035bb73e27 for an example.
>>
>> Thanks for the hint.
>
> They are spinning, but they have smt_low instructions in the loop,
> maybe that causes us to do some kind of synchronization as we exit
> the emulation loop on these ? I added that to force relinguish time
> to other threads on the pre-MT TCG...

Yeah you need a tweak the approach when running with MTTCG as otherwise
you end up causing exits for one vCPUs loop to yield to vCPUs that are
already running in other threads.

> There isn't really such a "pause" instruction. At least not yet.... P9
> has a wait that is meant to wait for special AS_Notify cycles but will
> also wait for interrupts. We don't have an mwait at this point.

They are worth implementing. FWIW on ARM we only really handle WFI
(Wait-for-interrupt) which will cause the EXCP_HALT and that will put
the vCPU into a halted state which can be woken up next interrupt.

For the other cases YIELD and WFE (wait-for-event) we just treat them as
NOPs when MTTCG is enabled (test parallel_cpus). So they will busy-wait
spin around the guests wfe code but don't trigger expensive longjmps out
of the execution loop. This was all done in:

  c22edfebff target-arm: don't generate WFE/YIELD calls for MTTCG

One other thing I noticed while looking through the PPC stuff is I
couldn't see any handling of cpu_reset/powering on. There is a potential
race here which ThreadSanitizer will complain about if you start loading
up your about-to-be-powered-on-vCPU from another thread. The fix here is
to defer the setup with async work. See:

  062ba099e0 target-arm/powerctl: defer cpu reset work to CPU context

--
Alex Bennée