Re: [Qemu-devel] QEMU ARM SMP: IPI delivery delayed until next main loop event // how to improve IPI latency?

["Alex Züpke" <alexander.zuepke@hs-rm.de>]

Am 15.06.2015 um 17:18 schrieb Peter Maydell:
> On 15 June 2015 at 16:07, Alex Züpke <alexander.zuepke@hs-rm.de> wrote:
>> Am 15.06.2015 um 17:04 schrieb Peter Maydell:
>>> On 12 June 2015 at 17:38, Alex Züpke <alexander.zuepke@hs-rm.de> wrote:
>>>> Hi,
>>>>
>>>> I'm benchmarking some IPI (== inter-processor-interrupt) synchronization stuff of my custom kernel on QEMU ARM (qemu-system-arm -M vexpress-a15 -smp 2) and ran into the following problem: pending IPIs are delayed until the QEMU main loop receives an event (for example the timer interrupt expires or I press a key on the console).
>>>>
>>>> The following timing diagram tries to show this:
>>>>
>>>>   CPU #0                       CPU #1
>>>>   ======                       ======
>>>>   ... other stuff ...          WFI (wait for interrupt, like x86 "HLT")
>>>>   send SGI in MPCore
>>>>   polls for completeness
>>>>                  <time passes ...>
>>>>   polls ...
>>>>                  <... and passes ...>
>>>>   still polls ...
>>>>                  <... and passes ...>
>>>>   still polls ...
>>>>                  <... and passes ...>
>>>>
>>>>
>>>>                  <timer interrupt expires>
>>>>                  <now QEMU switches to CPU #1>
>>>>                                receives IPI
>>>>                                signals completeness
>>>>                                WFI
>>>>                  <QEMU switches to CPU #0>
>>>>   polling done
>>>>   process timer interrupt
>>>>   ...
>>>
>>> Does your polling loop have a YIELD insn in it? We (and hardware)
>>> can use that as a hint that you're busy-looping and we should
>>> try doing something else. (QEMU doesn't implement that for A32/T32
>>> yet, but we should; we already do on A64.)
>>
>> Yes, I should be yielding here, but SEV isn't implemented.
>> Probably the notification should be done there as well.
> 
> YIELD isn't related to SEV -- it's just a generic "hey, I'm
> polling" hint. We NOP SEV, and make WFE be a "yield this CPU's
> timeslice" event", which is architecturally valid, and sufficient
> for this situation anyway.
> 
> -- PMM
> 

Ah, YIELD, thanks for the hint. I never used it because all CPUs I got my hand on were physical ones so far ...


So this is the way to go:

--- a/target-arm/translate.c
+++ b/target-arm/translate.c
@@ -4084,6 +4084,7 @@ static void gen_nop_hint(DisasContext *s, int val)
         gen_set_pc_im(s, s->pc);
         s->is_jmp = DISAS_WFI;
         break;
+    case 1: /* yield */
     case 2: /* wfe */
         gen_set_pc_im(s, s->pc);
         s->is_jmp = DISAS_WFE;


Thanks
Alex