[REPORT] [GSoC - TCG Continuous Benchmarking] [#1] Measuring Basic Performance Metrics of QEMU

[REPORT] [GSoC - TCG Continuous Benchmarking] [#1] Measuring Basic Performance Metrics of QEMU

[Ahmed Karaman]

[-- Attachment #1: Type: text/plain, Size: 1268 bytes --]

Greetings,

Welcome to the TCG Continuous Benchmarking GSoC project!
This is the first report in a series that will be posted here on the
mailing list as part of the project.

During the project, multiple Linux profiling tools will be used.
These tools will help you profile your different QEMU execution
scenarios, and locate performance bottlenecks as well as performance
regression problems.

To provide you with a better understanding of these tools, this report
presents two of them - Perf and Callgrind.
It starts with measuring the basic performance metrics when running
a program using both tools, then it proceeds to calculating the top 25
most executed functions in QEMU.
To dissect the output of each of the two tools, a separate section
for comparing the output results is presented which focuses on how
these results apply to QEMU.
Finally, a stability test is devised that compares how the output
of the two tools differ with each subsequent invocation of QEMU.

For easier accessibility, all reports will be hosted on a dedicated
website:
https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/

Link to the first report:
https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/Measuring-Basic-Performance-Metrics-of-QEMU/

Best regards,
Ahmed Karaman

[-- Attachment #2: Type: text/html, Size: 1602 bytes --]

Re: [REPORT] [GSoC - TCG Continuous Benchmarking] [#1] Measuring Basic Performance Metrics of QEMU

[Aleksandar Markovic]

пон, 22. јун 2020. у 12:31 Ahmed Karaman
<ahmedkhaledkaraman@gmail.com> је написао/ла:
>
> Greetings,
>
> Welcome to the TCG Continuous Benchmarking GSoC project!
> This is the first report in a series that will be posted here on the
> mailing list as part of the project.
>
> During the project, multiple Linux profiling tools will be used.
> These tools will help you profile your different QEMU execution
> scenarios, and locate performance bottlenecks as well as performance
> regression problems.
>
> To provide you with a better understanding of these tools, this report
> presents two of them - Perf and Callgrind.
> It starts with measuring the basic performance metrics when running
> a program using both tools, then it proceeds to calculating the top 25
> most executed functions in QEMU.
> To dissect the output of each of the two tools, a separate section
> for comparing the output results is presented which focuses on how
> these results apply to QEMU.
> Finally, a stability test is devised that compares how the output
> of the two tools differ with each subsequent invocation of QEMU.
>
> For easier accessibility, all reports will be hosted on a dedicated
> website:
> https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/
>
> Link to the first report:
> https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/Measuring-Basic-Performance-Metrics-of-QEMU/
>

In introductory section on Callgrind, maybe you could add the general
form of command line that performs profiling using Callgrind,
something like this:

valgrind --tool=callgrind [callgrind options] prog [prog options]

Not a big deal, you can do some fine adjustments throughout the project.

Thanks,
Aleksandar

> Best regards,
> Ahmed Karaman

Re: [REPORT] [GSoC - TCG Continuous Benchmarking] [#1] Measuring Basic Performance Metrics of QEMU

[Ahmed Karaman]

On Thu, Jun 25, 2020 at 3:38 PM Aleksandar Markovic
<aleksandar.qemu.devel@gmail.com> wrote:
>
> In introductory section on Callgrind, maybe you could add the general
> form of command line that performs profiling using Callgrind,
> something like this:
>
> valgrind --tool=callgrind [callgrind options] prog [prog options]
>
> Not a big deal, you can do some fine adjustments throughout the project.
>
> Thanks,
> Aleksandar
>

Thanks for the tip, Mr. Aleksandar. I've added it to the report.

Regards,
Ahmed Karaman

Re: [REPORT] [GSoC - TCG Continuous Benchmarking] [#1] Measuring Basic Performance Metrics of QEMU

[Aleksandar Markovic]

пон, 22. јун 2020. у 12:31 Ahmed Karaman
<ahmedkhaledkaraman@gmail.com> је написао/ла:
>
> Greetings,
>
> Welcome to the TCG Continuous Benchmarking GSoC project!
> This is the first report in a series that will be posted here on the
> mailing list as part of the project.
>
> During the project, multiple Linux profiling tools will be used.
> These tools will help you profile your different QEMU execution
> scenarios, and locate performance bottlenecks as well as performance
> regression problems.
>
> To provide you with a better understanding of these tools, this report
> presents two of them - Perf and Callgrind.
> It starts with measuring the basic performance metrics when running
> a program using both tools, then it proceeds to calculating the top 25
> most executed functions in QEMU

Now, thinking longer about topN scripts, I think one really missing
thing is number of invocations (or calls, whatever term you prefer)
for any function in the list. This data must be possible to obtain
using callgrind_annotate (most likely by using --tree option). With
perf, i don't think this is possible, given that perf works based on
sampling.

You don't need to start working on it right now, or work on it at all
- this is more like a brainstorming suggestion from me. You can make
improvements and correction all the way towards the end of the
project, on Aug 31st.

At the end of the project, perhaps you can publish a "Master Project
Report" - a pdf that is basically a sum of all your reports produced
during the project. That would be a nice reading!

Regards,
Aleksandar

> To dissect the output of each of the two tools, a separate section
> for comparing the output results is presented which focuses on how
> these results apply to QEMU.
> Finally, a stability test is devised that compares how the output
> of the two tools differ with each subsequent invocation of QEMU.
>
> For easier accessibility, all reports will be hosted on a dedicated
> website:
> https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/
>
> Link to the first report:
> https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/Measuring-Basic-Performance-Metrics-of-QEMU/
>
> Best regards,
> Ahmed Karaman

Re: [REPORT] [GSoC - TCG Continuous Benchmarking] [#1] Measuring Basic Performance Metrics of QEMU

[Ahmed Karaman]

On Sun, Jun 28, 2020 at 7:20 PM Aleksandar Markovic
<aleksandar.qemu.devel@gmail.com> wrote:
>
> Now, thinking longer about topN scripts, I think one really missing
> thing is number of invocations (or calls, whatever term you prefer)
> for any function in the list. This data must be possible to obtain
> using callgrind_annotate (most likely by using --tree option). With
> perf, i don't think this is possible, given that perf works based on
> sampling.
>
> You don't need to start working on it right now, or work on it at all
> - this is more like a brainstorming suggestion from me. You can make
> improvements and correction all the way towards the end of the
> project, on Aug 31st.
>
> At the end of the project, perhaps you can publish a "Master Project
> Report" - a pdf that is basically a sum of all your reports produced
> during the project. That would be a nice reading!
>
> Regards,
> Aleksandar
>

Thanks Mr. Aleksandar for always sharring your thoughts and suggestions.
I will consider this for an updated version of the report.

Regards,
Ahmed Karaman