Re: [PATCH v6 00/25] tcg-plugins: add hooks for discontinuities

[BALATON Zoltan <balaton@eik.bme.hu>]

[-- Attachment #1: Type: text/plain, Size: 4920 bytes --]

On Wed, 10 Sep 2025, Alex Bennée wrote:
> "Julian Ganz" <neither@nut.email> writes:
>> September 10, 2025 at 12:06 PM, "BALATON Zoltan" wrote:
>>> On Tue, 9 Sep 2025, Julian Ganz wrote:
>>>> I ran streamPPCpowerpcO3 on qemu with these patches:
>>>>
>>>>  -------------------------------------------------------------
>>>>  Function Best Rate MB/s Avg time Min time Max time
>>>>  Copy: 2867.6 0.056828 0.055795 0.061792
>>>>  Scale: 1057.5 0.153282 0.151305 0.158115
>>>>  Add: 1308.8 0.187095 0.183380 0.193672
>>>>  Triad: 1111.6 0.220863 0.215902 0.230440
>>>>  -------------------------------------------------------------
>>>>
>>>>  After doing a clean build, with the fans still audible:
>>>>
>>>>  -------------------------------------------------------------
>>>>  Function Best Rate MB/s Avg time Min time Max time
>>>>  Copy: 2932.9 0.055131 0.054554 0.055667
>>>>  Scale: 1067.9 0.151520 0.149832 0.155000
>>>>  Add: 1324.9 0.184807 0.181150 0.191386
>>>>  Triad: 1122.0 0.220080 0.213896 0.229302
>>>>  -------------------------------------------------------------
>>>>
>>> What was different between the above two runs? I guess maybe one is with plugins disabled but it's not clear from the description.
>>
>> The difference is nothing but a a clean rebuild of qemu. As you see
>> there are fluctuations already. Plugins are enabled for both cases.
>>
>>>> On qemu (6a9fa5ef3230a7d51e0d953a59ee9ef10af705b8) without these
>>>>  patches, but plugins enabled:
>>>>
>>>>  -------------------------------------------------------------
>>>>  Function Best Rate MB/s Avg time Min time Max time
>>>>  Copy: 2972.1 0.054407 0.053834 0.054675
>>>>  Scale: 1068.6 0.151503 0.149726 0.154594
>>>>  Add: 1327.6 0.185160 0.180784 0.193181
>>>>  Triad: 1127.2 0.219249 0.212915 0.229230
>>>>  -------------------------------------------------------------
>>>>
>>>>  And on qemu (6a9fa5ef3230a7d51e0d953a59ee9ef10af705b8) without these
>>>>  patches, with plugins disabled:
>>>>
>>>>  -------------------------------------------------------------
>>>>  Function Best Rate MB/s Avg time Min time Max time
>>>>  Copy: 2983.4 0.055141 0.053630 0.060013
>>>>  Scale: 1058.3 0.152353 0.151186 0.155072
>>>>  Add: 1323.9 0.184707 0.181279 0.188868
>>>>  Triad: 1128.2 0.218674 0.212734 0.230314
>>>>  -------------------------------------------------------------
>>>>
>>>>  I fail to see any significant indication that these patches, or
>>>>  plugins in general, would result in a degredation of performance.
>>>>
>>> With worst case Copy test it seems to be about 3.5% (and about 1.7%
>>> with plugins disabled?) and should be less than that normally so it
>>> does not add much more overhead to plugins than there is already so
>>> this should be acceptable. It may still be interesting to see if the
>>> overhead with plugins disabled can be avoided with a similar way as
>>> logging does it.
>>
>> The thing is: that's probably just usual fluctuations. As you have seen
>> with the first two measurements the values fluctuate quite a bit between
>> runs of the test on the very same qemu (assuming that a clean build did
>> not incur any _other_ relevant change). For example, the best rate for
>> scale shown with plugins enabled is one percent faster than with plugins
>> disabled. Is this significant? Probably not. Or at least it doesn't make
>> much sense.
>
> I wouldn't spend too much time chasing this down. As you say this
> fluctuation is well within the noise range.
>
> I can recommend hyperfine as a runner:
>
>  https://github.com/sharkdp/hyperfine
>
> as it does some work on how many times you need to run a test before the
> results are statistically relevant.
>
>> I may do some more tests this week, with runtimes longer than a few
>> seconds if I can find the motivation to set up everything I'd need to
>> compile your benchmark. In the mean-time, you are welcome to make your
>> own measurements if you want to. The patches are also availible at [1]
>> if you don't want to apply them to your local tree yourself.
>
> Balton,
>
> I don't think worries about performance impact are justified and Julian
> has certainly done enough due diligence here. If you can come up with a
> repeatable test that shows a measurable impact then please do so.

I agree this testing is enough to ensure there is no big impact. I just 
wanted to make sure there is some testing and not just adding stuff 
without worrying about performance. I'd like to keep QEMU quick and only 
add unavoidable overhead where possible but I don't demand to spend too 
much time on that. If Julian got interested and does more testing that may 
give some interesting results for possible optimisation but if no time for 
that this was enough to measure the impact for this series.

Regards,
BALATON Zoltan

>> Regards,
>> Julian
>>
>> [1]: https://github.com/patchew-project/qemu/tree/patchew/cover.1757018626.git.neither@nut.email
>
>