[Qemu-devel] 2x-3x tcg performance regression on ppc64 (maybe others)

[Cleber Rosa <crosa@redhat.com>]

While working on the "boot_linux_console.py" tests, after a given
rebase, I noticed that the ppc64 specific test began timing out.  The
original timeout set to the test was 60 seconds, and it "always" had
room to spare when running either on my system, or on Travis CI.

Then, Alex mentioned that specific test timing out on his "slow"
system[1].  I did some further investigation and found out that commit
f7b78602f might have been responsible for a significant slowdown,
and maybe was also affecting his execution.

Getting straight to the point, this is the ppc64 "boot linux console"
test, running 10 times, on top my latest rebase (e8977901b, that
includes f7b78602f).

 ...
 (01/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(63.07 s)
 (02/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(64.00 s)
 (03/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(62.93 s)
 (04/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(64.01 s)
 (05/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(63.13 s)
 (06/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(64.65 s)
 (07/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(63.25 s)
 (08/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(66.77 s)
 (09/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(65.07 s)
 (10/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(63.55 s)
 ...

The exact branch I used for the results above can be seen at:
  - https://github.com/clebergnu/qemu/tree/regression_tcg_ppc64

Then, with f7b78602f reverted:

 (01/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(20.54 s)
 (02/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(21.06 s)
 (03/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(20.81 s)
 (04/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(21.00 s)
 (05/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(20.37 s)
 (06/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(20.40 s)
 (07/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(20.36 s)
 (08/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(20.39 s)
 (09/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(20.39 s)
 (10/10) boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: PASS
(20.38 s)

The exact branch I used for the results above can be seen at:
  - https://github.com/clebergnu/qemu/tree/regression_tcg_ppc64_revert

Even though there's a lot more noise involved, similar results can be
seen on Travis-CI:

  - https://travis-ci.org/clebergnu/qemu/builds/487607293#L3035
     (04/16)
/home/travis/build/clebergnu/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries:
 PASS (22.21 s)

  - https://travis-ci.org/clebergnu/qemu/builds/487606849#L3035
     (04/16)
/home/travis/build/clebergnu/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries:
 PASS (11.27 s)

The reasons for the extra noise include:
  1) Those tests download the kernel images
  2) They are run only once
  3) They run on shared infrastructure

Based on Travis-CI runs alone, it's hard to determine if there was any
other significant regression (or improvement?) for other targets.  I'm
posting this early, but if I come up with more relevant information,
follow up here.

Regards,
- Cleber.

---

[1] - https://www.mail-archive.com/qemu-devel@nongnu.org/msg589497.html