Re: [Qemu-devel] MTTCG External Halt

["Alex Bennée" <alex.bennee@linaro.org>]

Alistair Francis <alistair23@gmail.com> writes:

> Hey guys, I'm super stuck with an ugly MTTCG issue and was wondering
> if anyone had any ideas.
>
> In the Xilinx fork of QEMU (based on 2.11) we have a way for CPUs to
> halt other CPUs. This is used for example when the power control unit
> halts the ARM A53s. To do this we have internal GPIO signals that end
> up calling a function that basically does this:
>
> To halt:
>     cpu->halted = true;
>     cpu_interrupt(cpu, CPU_INTERRUPT_HALT);

Hmm I don't think you should be setting cpu->halted unless you know it
is safe to do so. As the other CPUs free-run during BQL this isn't
enough for a cross vCPU interaction. However you can schedule work to
run in the target vCPUs context safely.

That said isn't the cpu_interrupt enough to trigger the target vCPU to
halt?

>
> To un-halt
>     cpu->halted = false;
>     cpu_reset_interrupt(cpu, CPU_INTERRUPT_HALT);

Again if cross vCPU context this needs to be scheduled against the
target vCPU.

>
> We also have the standard ARM WFI (Wait For Interrupt) implementation
> in op_helper.c:
>     cs->halted = 1;
>     cs->exception_index = EXCP_HLT;
>     cpu_loop_exit(cs);
>
> Before MTTCG this used to work great, but now either we end up with
> the guest Linux complaining about CPU stalls or we hit:
> ERROR:/scratch/alistai/master-qemu/cpus.c:1516:qemu_tcg_cpu_thread_fn:
> assertion failed: (cpu->halted)
>
> If I remove the instances of manually setting cpu->halted then I don't
> see the asserts(), but the the WFI instruction doesn't work correctly.
> So it seems like setting the halted status externally from the CPU
> causes the issue.

  /* during start-up the vCPU is reset and the thread is
   * kicked several times. If we don't ensure we go back
   * to sleep in the halted state we won't cleanly
   * start-up when the vCPU is enabled.
   *
   * cpu->halted should ensure we sleep in wait_io_event
   */

I think what I'm trying to say is we should never be halted without
having gone via wait_io_event where we can sleep.


> I have tried setting it inside a lock, using atomic
> operations and running the setter async on the CPU, but nothing works.
>
> Any chance any one has some insight into a way to externally set a
> vCPU as halted/un-halted?

See the PSCI code which uses the async interface for exactly this.

>
> Thanks,
> Alistair


--
Alex Bennée