Re: mips system emulation failure with virtio

["Philippe Mathieu-Daudé" <philmd@linaro.org>]

+rth/pm215/dhildenb

On 5/9/23 16:50, Richard Purdie wrote:
> On Tue, 2023-09-05 at 14:59 +0100, Alex Bennée wrote:
>> Richard Purdie <richard.purdie@linuxfoundation.org> writes:
>>
>>> With qemu 8.1.0 we see boot hangs fox x86-64 targets.
>>>
>>> These are fixed by 0d58c660689f6da1e3feff8a997014003d928b3b (softmmu:
>>> Use async_run_on_cpu in tcg_commit) but if I add that commit, mips and
>>> mips64 break, hanging at boot unable to find a rootfs.
>>
>> (Widen CC list)
>>
>>>
>>> We use virtio for network and disk and both of those change in the
>>> bootlog from messages like:
>>>
>>> [    1.726118] virtio-pci 0000:00:13.0: enabling device (0000 -> 0003)
>>> [    1.728864] virtio-pci 0000:00:14.0: enabling device (0000 -> 0003)
>>> [    1.729948] virtio-pci 0000:00:15.0: enabling device (0000 -> 0003)
>>> ...
>>> [    2.162148] virtio_blk virtio2: 1/0/0 default/read/poll queues
>>> [    2.168311] virtio_blk virtio2: [vda] 1184242 512-byte logical
>>>
>>> to:
>>>
>>> [    1.777051] virtio-pci 0000:00:13.0: enabling device (0000 -> 0003)
>>> [    1.779822] virtio-pci 0000:00:14.0: enabling device (0000 -> 0003)
>>> [    1.780926] virtio-pci 0000:00:15.0: enabling device (0000 -> 0003)
>>> ...
>>> [    1.894852] virtio_rng: probe of virtio1 failed with error -28
>>> ...
>>> [    2.063553] virtio_blk virtio2: 1/0/0 default/read/poll queues
>>> [    2.064260] virtio_blk: probe of virtio2 failed with error -28
>>> [    2.069080] virtio_net: probe of virtio0 failed with error -28
>>>
>>>
>>> i.e. the virtio drivers no longer work.
>>
>> Interesting, as you say this seems to be VirtIO specific as the baseline
>> tests (using IDE) work fine:
>>
>>    ➜  ./tests/venv/bin/avocado run ./tests/avocado/tuxrun_baselines.py:test_mips64
>>    JOB ID     : 71f3e3b7080164b78ef1c8c1bb6bc880932d8c9b
>>    JOB LOG    : /home/alex/avocado/job-results/job-2023-09-05T15.01-71f3e3b/job.log
>>     (1/2) ./tests/avocado/tuxrun_baselines.py:TuxRunBaselineTest.test_mips64: PASS (12.19 s)
>>     (2/2) ./tests/avocado/tuxrun_baselines.py:TuxRunBaselineTest.test_mips64el: PASS (11.78 s)
>>    RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>>    JOB TIME   : 24.79 s
>>
>>> I tested with current qemu master
>>> (17780edd81d27fcfdb7a802efc870a99788bd2fc) and mips is still broken
>>> there.
>>>
>>> Is this issue known about?
>>
>> Could you raise a bug at:
>>
>>    https://gitlab.com/qemu-project/qemu/-/issues
> 
> Done, https://gitlab.com/qemu-project/qemu/-/issues/1866
> 
>> I'm curious why MIPS VirtIO is affected but nothing else is...
> 
> Me too, it seems there is a real code issue somewhere in this...

This seems to fix the issue for me, but I'm not really sure what
I'm doing after various hours debugging, so sharing here before
I take some rest:

-- >8 --
diff --git a/softmmu/physmem.c b/softmmu/physmem.c
index 18277ddd67..ec31ebcb56 100644
--- a/softmmu/physmem.c
+++ b/softmmu/physmem.c
@@ -2517,7 +2517,7 @@ static void tcg_commit(MemoryListener *listener)
       * That said, the listener is also called during realize, before
       * all of the tcg machinery for run-on is initialized: thus halt_cond.
       */
-    if (cpu->halt_cond) {
+    if (cpu->halt_cond && !qemu_cpu_is_self(cpu)) {
          async_run_on_cpu(cpu, tcg_commit_cpu, RUN_ON_CPU_HOST_PTR(cpuas));
      } else {
          tcg_commit_cpu(cpu, RUN_ON_CPU_HOST_PTR(cpuas));
---

That said, the same logic moved generically to async_run_on_cpu()
also works ...:

-- >8 --
diff --git a/cpus-common.c b/cpus-common.c
index 45c745ecf6..b0539c4fb8 100644
--- a/cpus-common.c
+++ b/cpus-common.c
@@ -167,6 +167,9 @@ void do_run_on_cpu(CPUState *cpu, run_on_cpu_func 
func, run_on_cpu_data data,

  void async_run_on_cpu(CPUState *cpu, run_on_cpu_func func, 
run_on_cpu_data data)
  {
+    if (qemu_cpu_is_self(cpu)) {
+        return func(cpu, data);
+    }
      struct qemu_work_item *wi;

      wi = g_new0(struct qemu_work_item, 1);
---

Regards,

Phil.