[Qemu-devel] TCG icount interaction with timer deadlines

[Qemu-devel] TCG icount interaction with timer deadlines

[Peter Maydell]

Does anybody understand how icount TCG is supposed to arrange to
respect timer deadlines?

https://bugs.launchpad.net/qemu/+bug/1754038 has a test case which
shows that we don't get this right.

At the moment what happens is:
 * when we're about to call tcg_cpu_exec(), we call
   prepare_icount_for_run(), which looks at the closest clock deadline
   to figure out how many insns to execute before dropping out, with
   a cap at INT32_MAX nanoseconds
 * however, if the guest reprograms the clock during the tcg_cpu_exec()
   run, we don't do anything to cause us to stop earlier
 * so we blithely continue on til that INT32_MAX cap, and then
   belatedly notice that we should have fired a timer interrupt

In the test case this manifests as the first timer interrupt being
very delayed, because the first tcg_cpu_exec() goes from "start of
program" to INT32_MAX nanoseconds. Later interrupts happen OK, because
the guest isn't reprogramming the timer interrupt, so the deadline
picked at the start of each tcg_cpu_exec run is correct.

What should we be doing to arrange to stop execution of the
tcg_cpu_exec() earlier when the deadline moves closer?

thanks
-- PMM

Re: [Qemu-devel] TCG icount interaction with timer deadlines

[Paolo Bonzini]

On 05/04/2018 18:01, Peter Maydell wrote:
>  * however, if the guest reprograms the clock during the tcg_cpu_exec()
>    run, we don't do anything to cause us to stop earlier

Anything that does this from the vCPU thread should be between
gen_icount_start and gen_icount_end.  (In fact, it's the entire reason
why cpu_io_recompile exists).

For completeness, if the I/O thread does it, it should (in icount mode
only) kick the running VCPU.

Am I missing something?

Paolo

Re: [Qemu-devel] TCG icount interaction with timer deadlines

[Peter Maydell]

On 5 April 2018 at 18:07, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 05/04/2018 18:01, Peter Maydell wrote:
>>  * however, if the guest reprograms the clock during the tcg_cpu_exec()
>>    run, we don't do anything to cause us to stop earlier
>
> Anything that does this from the vCPU thread should be between
> gen_icount_start and gen_icount_end.  (In fact, it's the entire reason
> why cpu_io_recompile exists).

Yes, and this does cause us to do a cpu_io_recompile, which
rebuilds the TB and does a longjmp. However:
 (1) that only takes us out to cpu_exec(), which will then
 just go ahead and execute the next TB, whereas the
 recalculation of deadlines happens at the next level out
 in tcg_cpu_exec()
 (2) the io_recompile happens *before* the guest writes to
 the timer register that reprograms the deadline, so even
 if we recomputed deadlines after this longjmp they wouldn't
 be correct

thanks
-- PMM

Re: [Qemu-devel] TCG icount interaction with timer deadlines

[Paolo Bonzini]

----- Original Message -----
> From: "Peter Maydell" <peter.maydell@linaro.org>
> To: "Paolo Bonzini" <pbonzini@redhat.com>
> Cc: "QEMU Developers" <qemu-devel@nongnu.org>, "Alex Bennée" <alex.bennee@linaro.org>, "Richard Henderson"
> <rth@twiddle.net>, "Emilio G. Cota" <cota@braap.org>, "Pavel Dovgalyuk" <dovgaluk@ispras.ru>
> Sent: Thursday, April 5, 2018 7:35:56 PM
> Subject: Re: TCG icount interaction with timer deadlines
> 
> On 5 April 2018 at 18:07, Paolo Bonzini <pbonzini@redhat.com> wrote:
> > On 05/04/2018 18:01, Peter Maydell wrote:
> >>  * however, if the guest reprograms the clock during the tcg_cpu_exec()
> >>    run, we don't do anything to cause us to stop earlier
> >
> > Anything that does this from the vCPU thread should be between
> > gen_icount_start and gen_icount_end.  (In fact, it's the entire reason
> > why cpu_io_recompile exists).
> 
> Yes, and this does cause us to do a cpu_io_recompile, which
> rebuilds the TB and does a longjmp. However:
>  (1) that only takes us out to cpu_exec(), which will then
>  just go ahead and execute the next TB, whereas the
>  recalculation of deadlines happens at the next level out
>  in tcg_cpu_exec()
>  (2) the io_recompile happens *before* the guest writes to
>  the timer register that reprograms the deadline, so even
>  if we recomputed deadlines after this longjmp they wouldn't
>  be correct

Right - that part would be handled here:

void qemu_timer_notify_cb(void *opaque, QEMUClockType type)
{
    if (!use_icount || type != QEMU_CLOCK_VIRTUAL) {
        qemu_notify_event();
        return;
    }

    if (!qemu_in_vcpu_thread() && first_cpu) {
        /* qemu_cpu_kick is not enough to kick a halted CPU out of
         * qemu_tcg_wait_io_event.  async_run_on_cpu, instead,
         * causes cpu_thread_is_idle to return false.  This way,
         * handle_icount_deadline can run.
         */
        async_run_on_cpu(first_cpu, do_nothing, RUN_ON_CPU_NULL);
    }
}

(called by timerlist_notify, called in turn by timerlist_rearm)
but that second "if" is too restrictive.  Maybe just removing
the first arm is enough.  All this was broken by MTTCG.

Paolo