Re: [Qemu-devel] [PATCH 0/1] Change default CFLAGS to -O3

["Jin Guojie" <jinguojie@loongson.cn>]

Thanks all for your review comments.
To measure the actual performance difference between O2 and O3,
I did the following tests:

Environments:
* Hardware: AMD Athlon II X2 270 Processor, 3.4 GHz
* Host OS: Fedora25 X86_64
* qemu: 2.8.0
* gcc: 6.2.1
* Guest OS: linux-0.2.img
  (downloaded from http://wiki.qemu.org/Testing/System_Images)
* Benchmark: nbench within linux-0.2.img

Results:
  Each test has been run at least 3 times to minimize the deviation.
  Higher score is better.

|                            |   O2   |   O3   |
------------------------------------------------
| qemu-system-i386(stripped) | 8.1 MB | 8.9 MB | 
| nbench(NUMERIC SORT)       | 254.72 | 260.48 |
|                            | 259.2  | 259.52 |
|                            | 261.04,| 259.68 |

Conclusion:
  Compared with O2, O3 brings no obvious performance advantage.
  It's highly possible that the benchmark score is mainly determined
  by the quality of tcg, not the host compiler.

Related work:
(1) The same test is done on a Loongson 3A2000.
  It's a mips64el-compatible CPU, with 4 cores running at 1.5 GHz.
  The result and conclusion are almost the same with X86.
(2) Another scene of OS booting is taken into consideration.
  I compare the booting time of linux-0.2.img under O2 and O3.
  Once again, I cannot see an observable difference.

Based on these data I admit my patch has no benifit for performance.

Jin Guojie

 ------------------ Original ------------------
From:  "Peter Maydell";<peter.maydell@linaro.org>;
Send time: Monday, Jan 9, 2017 6:33 PM
To: "Jin Guojie"<jinguojie@loongson.cn>; 
Cc: "QEMU Developers"<qemu-devel@nongnu.org>; "Paolo Bonzini"<pbonzini@redhat.com>; 
Subject:  Re: [Qemu-devel] [PATCH 0/1] Change default CFLAGS to -O3



On 9 January 2017 at 10:01, Jin Guojie <jinguojie@loongson.cn> wrote:
> * in configure, the default optimization level is changed to O3.
> * Tested successfully on following machines:
>
>   | Host machine | Host OS  | qemu-system | Debian ISO  |
>   |-----------------------------------------------------|
>   |   x86 64     | Fedora25 |    i386     |    i386     |
>   | mips 64 le   | Fedora25 |    i386     |    i386     |
>
> Signed-off-by: Jin Guojie <jinguojie@loongson.cn>
>
> Jin Guojie (1):
>   Change default CFLAGS to -O3
>
>  configure | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

...so what are the measured performance improvements from
this change? As I understand it -O3 is where the "trade off
code size for speed" optimisations kick in, so we need
to benchmark to be sure that the increased icache usage
doesn't outweigh any speed gains. -O3 is not guaranteed
to be strictly better than -O2 (which is why for instance
Debian and Gentoo recommend -O2 as their default).

thanks
-- PMM