Re: [RFC] Adding Support for Firefox's Gecko Profile Format

[Anup Sharma <anupnewsmail@gmail.com>]

On Thu, Aug 10, 2023 at 01:31:09AM +0530, Anup Sharma wrote:
> On Thu, Aug 03, 2023 at 12:29:51AM +0530, Anup Sharma wrote:
> > On Wed, Jul 26, 2023 at 04:22:51PM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Wed, Jul 26, 2023 at 11:34:17PM +0530, Anup Sharma escreveu:
> > > > Dear all,
> > > > 
> > > > I want to share a recent development in our project that
> > > > I have been working on this week. I have successfully
> > > > implemented a new feature that allows users to directly
> > > > open the browser and visualize the output of perf.data
> > > > on Firefox Profiler.
> > > 
> > > Excellent news!
> > >  
> > > > To enhance user experience, I have made sure this option
> > > > is enabled by default. Additionally, I've also added a
> > > > handy command line argument, "--save-only," which allows
> > > > users to save the output file as needed. This enhancement
> > > > replaces the previous standard output method, providing a
> > > > much-improved experience.
> > > > 
> > > > Moreover, Everything is processed locally, and no data is
> > > > sent to any server.
> > > 
> > > That addresses the worry about sending possibly sensitive data over the
> > > network by default, great.
> > >  
> > > > During this implementation, I had the opportunity to delve
> > > > into several exciting topics, including threading, HTTP server,
> > > > and CORS mechanisms. The learning process has been both
> > > > challenging and rewarding.
> > > 
> > > Cool that you're so excited learning all this.
> > >  
> > > > Currently, I am in the process of performing some clean-up tasks.
> > > > Once that is complete, I will be sharing the next version of the
> > > > series, which includes this fantastic new feature, with all of
> > > > you very soon.
> > > > 
> > > > Thank you for your ongoing support and encouragement.
> > > 
> > > Keep it up!
> > 
> > This week, I'm working on documenting the gecko converter for our wiki.
> > I've dedicated a significant amount of time to think about the content
> > and how to approach it. So far, I've written sections covering the
> > introduction to Firefox Profiler and the formatting guidelines(not
> > published yet).
> > 
> > However, I've left the usage part blank because I'm uncertain about how
> > to enable gecko.py arguments when executing the "perf script gecko" command.
> > I've looked at examples like flamegraph-report and sctop-report, but I'm
> > still confused. I attempted to enable the script to take the gecko.py
> > command example "perf script report gecko --save-only=profiler.json,"
> > but this caused issues when running "perf script gecko -a sleep 3" again.
> > This has left me feeling rather puzzled, and I would appreciate discussing
> > this during our office hour, or any help here.
> > 
> > Due to this blocker, I haven't been able to send the updated patch
> > for test_gecko yet. However, I hope I'll find a solution soon and overcome
> > this obstacle.
> 
> Hello All,
> As discussed during last office hour, I was little occupied this week, so
> I haven't been able to make much progress on my TODO list. Nonetheless,
> I've resent a server patch and test script patch for your review. If I can
> get it reviewed it would be great to make further progress.
> 
> I'm also working on writing documentation for our wiki. I've already written a
> bit, and you can see it here: https://perf.wiki.kernel.org/index.php/Tutorial#Firefox_Profiler.
> If you have any thoughts or suggestions, I'd appreciate hearing them. I'll add
> more details once I've sorted out some issues I'm having with certain commands.
> 
> This week, I do not have a specific agenda. Hence, if we opt to skip this week's
> office hour, it wouldn't pose any inconvenience. Nevertheless, I am available if
> there are matters you wish to discuss.
> 
> Here's what I have in mind to work on next:
> 
> 1. Making sure that gecko script args work with commands, like
> (perf script gecko --save-only=profile.json) or (perf script gecko --kernel-color=green),which
> aren't working right now.

I have done this part and sent the patch for review.
https://lore.kernel.org/all/ZNf7W+EIrrCSHZN0@yoga/

> 2. Trying out "perf record -p firefox" or using it with a specific process ID to
> see if the current script supports these recording commands.

This is working with the existing script. I've tested it and documented it as well.
https://perf.wiki.kernel.org/index.php/Tutorial#Firefox_Profiler

> 3. Adding page fault and one more event in stackTable in gecko script.

I have few doubt regarding this will discuss in office hour.

> 4. Making the documentation better.

I am trying to make it better, but I'm not sure if it's good enough. Here's the link:
https://perf.wiki.kernel.org/index.php/Tutorial#Firefox_Profiler

I also need to understand on how to prepare the final report for GSoC and
planning to record a 5 min tutorial video on how to use the script.

Moreover, I need to make the script run as a root user.

This are few TODOs I have in mind. Please let me know if you have any suggestions.

> Thanks,
> Anup
> 
> > > - Arnaldo
> > > 
> > > > Thanks,
> > > > Anup
> > > > 
> > > > > > Regards,
> > > > > > Anup
> > > > > > > > 
> > > > > > > >>
> > > > > > > >> The second challenge revolved around the usage of event hooks provided with the perf script
> > > > > > > >> python event handlers. I found myself deliberating between two approaches. The first
> > > > > > > >> approach involved creating custom functions that would be called using the event
> > > > > > > >> handlers. These functions would then save the data in an organized format within
> > > > > > > >> globally created data structures. The alternative approach was to write the entire
> > > > > > > >> logic inside the event handlers themselves.
> > > > > > > >>
> > > > > > > >> Additionally, I contemplated whether it would be more suitable to handle the creation of
> > > > > > > >> a Gecko format for JSON and the profile format within the same script or to separate
> > > > > > > >> them into different scripts.
> > > > > > > >>
> > > > > > > >> I will discuss this points during tomorrow's office hour.
> > > > > > > >>
> > > > > > > >> However, I have gained a deeper understanding of the problem at hand and will use this
> > > > > > > >> knowledge to make more informed decisions and progress more effectively in the coming weeks.
> > > > > > > > 
> > > > > > > > Sounds like you did a good amount of research on how perf script
> > > > > > > > works.  We can talk about the details in the meeting.
> > > > > > > > 
> > > > > > > > Thanks,
> > > > > > > > Namhyung
> > > > > > > 
> > > 
> > > -- 
> > > 
> > > - Arnaldo