Re: [PATCH v6 00/25] tcg-plugins: add hooks for discontinuities

["Alex Bennée" <alex.bennee@linaro.org>]

"Julian Ganz" <neither@nut.email> writes:

> September 10, 2025 at 12:06 PM, "BALATON Zoltan" wrote:
>> On Tue, 9 Sep 2025, Julian Ganz wrote:
>> > I ran streamPPCpowerpcO3 on qemu with these patches:
>> > 
>> >  -------------------------------------------------------------
>> >  Function Best Rate MB/s Avg time Min time Max time
>> >  Copy: 2867.6 0.056828 0.055795 0.061792
>> >  Scale: 1057.5 0.153282 0.151305 0.158115
>> >  Add: 1308.8 0.187095 0.183380 0.193672
>> >  Triad: 1111.6 0.220863 0.215902 0.230440
>> >  -------------------------------------------------------------
>> > 
>> >  After doing a clean build, with the fans still audible:
>> > 
>> >  -------------------------------------------------------------
>> >  Function Best Rate MB/s Avg time Min time Max time
>> >  Copy: 2932.9 0.055131 0.054554 0.055667
>> >  Scale: 1067.9 0.151520 0.149832 0.155000
>> >  Add: 1324.9 0.184807 0.181150 0.191386
>> >  Triad: 1122.0 0.220080 0.213896 0.229302
>> >  -------------------------------------------------------------
>> > 
>> What was different between the above two runs? I guess maybe one is with plugins disabled but it's not clear from the description.
>
> The difference is nothing but a a clean rebuild of qemu. As you see
> there are fluctuations already. Plugins are enabled for both cases.
>
>> > On qemu (6a9fa5ef3230a7d51e0d953a59ee9ef10af705b8) without these
>> >  patches, but plugins enabled:
>> > 
>> >  -------------------------------------------------------------
>> >  Function Best Rate MB/s Avg time Min time Max time
>> >  Copy: 2972.1 0.054407 0.053834 0.054675
>> >  Scale: 1068.6 0.151503 0.149726 0.154594
>> >  Add: 1327.6 0.185160 0.180784 0.193181
>> >  Triad: 1127.2 0.219249 0.212915 0.229230
>> >  -------------------------------------------------------------
>> > 
>> >  And on qemu (6a9fa5ef3230a7d51e0d953a59ee9ef10af705b8) without these
>> >  patches, with plugins disabled:
>> > 
>> >  -------------------------------------------------------------
>> >  Function Best Rate MB/s Avg time Min time Max time
>> >  Copy: 2983.4 0.055141 0.053630 0.060013
>> >  Scale: 1058.3 0.152353 0.151186 0.155072
>> >  Add: 1323.9 0.184707 0.181279 0.188868
>> >  Triad: 1128.2 0.218674 0.212734 0.230314
>> >  -------------------------------------------------------------
>> > 
>> >  I fail to see any significant indication that these patches, or
>> >  plugins in general, would result in a degredation of performance.
>> > 
>> With worst case Copy test it seems to be about 3.5% (and about 1.7%
>> with plugins disabled?) and should be less than that normally so it
>> does not add much more overhead to plugins than there is already so
>> this should be acceptable. It may still be interesting to see if the
>> overhead with plugins disabled can be avoided with a similar way as
>> logging does it.
>
> The thing is: that's probably just usual fluctuations. As you have seen
> with the first two measurements the values fluctuate quite a bit between
> runs of the test on the very same qemu (assuming that a clean build did
> not incur any _other_ relevant change). For example, the best rate for
> scale shown with plugins enabled is one percent faster than with plugins
> disabled. Is this significant? Probably not. Or at least it doesn't make
> much sense.

I wouldn't spend too much time chasing this down. As you say this
fluctuation is well within the noise range.

I can recommend hyperfine as a runner:

  https://github.com/sharkdp/hyperfine

as it does some work on how many times you need to run a test before the
results are statistically relevant.

> I may do some more tests this week, with runtimes longer than a few
> seconds if I can find the motivation to set up everything I'd need to
> compile your benchmark. In the mean-time, you are welcome to make your
> own measurements if you want to. The patches are also availible at [1]
> if you don't want to apply them to your local tree yourself.

Balton,

I don't think worries about performance impact are justified and Julian
has certainly done enough due diligence here. If you can come up with a
repeatable test that shows a measurable impact then please do so.

>
> Regards,
> Julian
>
> [1]: https://github.com/patchew-project/qemu/tree/patchew/cover.1757018626.git.neither@nut.email

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro