Re: Linux Foundation accepted for Google Summer-of-Code 2021

[Milian Wolff <milian.wolff@kdab.com>]

[-- Attachment #1: Type: text/plain, Size: 6122 bytes --]

On Sonntag, 14. März 2021 18:29:09 CET Ian Rogers wrote:
> Hi Milian,
> 
> Thanks for the email and hotspot is a very interesting project!
> 
> On Sat, Mar 13, 2021 at 8:25 AM Milian Wolff <milian.wolff@kdab.com> wrote:
> > On Mittwoch, 10. März 2021 17:32:25 CET Ian Rogers wrote:
> > > Hi,
> > > 
> > > I'm happy to share that the Linux Foundation has been accepted into
> > > the Google Summer-of-Code 2021 and we are looking for students
> > > interested in profiling, perf, etc. to apply to do projects.
> > > 
> > > The main details on the Linux foundation application are here:
> > > https://wiki.linuxfoundation.org/gsoc/google-summer-code-2021
> > > 
> > > Details on potential projects with perf here:
> > > https://wiki.linuxfoundation.org/gsoc/2021-gsoc-perf
> > > 
> > > Please help to share, especially to students.
> > > 
> > > Additional project ideas, mentors and help is also appreciated.
> > 
> > As the maintainer of hotspot [1], I'd be happy to discuss how we could
> > reuse the prior art in providing a GUI around `perf report` and more.
> > Considering that hotspot is already a pretty successfull project, I
> > wonder why we have to reinvent the wheel to provide a gtk-based GUI - is
> > it really just the framework choice - Qt vs. GTK? Or are there other
> > reasons?
> 
> There are a few issues:
>  - there's an observation that the gtk support in perf was bitrotting.
> Having a student project in this area would hopefully improve this
> situation.

Agreed.

>  - commands like perf report even in text mode are slow, single
> threaded, etc. Again, an area where a student project can help out.

Again, agreed. That is actually something that I've spent a considerable 
amount of time on. So far, I mostly concentrated on better caching - something 
which has had very significant effects on the performance of hotspot for some 
perf.data workloads. See e.g.: https://www.kdab.com/hotspot-version-1-3-0-released/

Threading wise, in hotspot, we have basically two threads - one in the 
background process perfparser which does unwinding and symbolication. Then 
another one in the hotspot GUI process, which does the data aggregation for 
the various views. So far, I shied away from trying to further parallelize the 
former part, as unwinding and symbolication are both heavily dependend on the 
order in which events are processed. I.e. we have to make sure to process mmap 
events and the like before we process a sampling event. Additionally, making 
caching efficient is much easier in a singlethreaded context, which was so far 
much more important to get right.

>  - wrt UI there is also pprof (https://github.com/google/pprof) which
> I think a web UI is interesting beyond just Qt, GTK, etc. Having the
> code be within the perf tool means that testing can be automated, it
> is easy for shared contributions, the kernel and user APIs aren't
> separated by projects..

Yes, nowadays there are tons of pretty good UI options out there - esp. in the 
web context. I have not used pprof, but heard a lot of praise for the Mozilla 
profiler too: https://profiler.firefox.com/

And then there's catapult which is integrated into chromium or Perfetto which 
is afaik used for Android? The list probably goes on and on ;-)

But in the end it would be extremely useful if perf could directly create a 
pre-processed, compressed and portable result file with some command. That 
could in turn use some easy-to-use binary format (CBOR, protobof, whatever) 
which could then be parsed by the GUI instead of it having to reinvent the 
wheel there.

> > Generally, I would love to see a proper library provided by perf that
> > could be used to do the `perf report` and/or `perf annotate` data
> > processing.
>
> There is a library interface developing:
> https://github.com/torvalds/linux/tree/master/tools/lib/perf

Ah, that's very interesting - that didn't exist back when we started 
perfparser/hotspot. 
 
> But it is mainly focussed on say 'perf record' than 'perf report'.
> Perhaps you could mentor a project to refactor the report APIs, which
> could be integrated with hotspot? If so, we can update the GSoC wiki.

While I'm honored by the invitation, I'm afraid that I don't have the capacity 
to fulfil this role. I've attended GSOC myself in the past, and even mentored 
a couple of times, so I know that doing a good job there as a mentor requires 
more time than I can spare currently :(

That said, I am very willing to be available for discussion as an interested 
party.

> It'd be nice to refactor things like event parsing into a library, in
> part to make unit and fuzz testing easier. The event sorting is a big
> source of perf's slowness and something that should be reasonable to
> parallelize, an issue here is a lack of a thread pool and other
> abstractions. These dependencies across projects can be something that
> is tricky.

In our case, sorting isn't the bottleneck at all. But it does make 
parallelization harder as I mentioned above. Unwinding and symbolication is 
usually what's much much slower compared to just sorting a couple of events 
in-memory.

Regarding missing abstractions: I'm a C++ developer myself, and having easy 
access to more abstractions like hash maps and more has helped tremendously in 
improving the performance of our perf.data parser too. So yes, I again totally 
agree with you in that regards.

> > As I said - I'm happy to discuss this topic in more detail with anyone
> > interested in this topic.
> 
> I'm interested! There is an IRC channel #perf see www.oftc.net, but
> I'd suggest the mailing list is better for discussion. I know that BPF
> is doing regular office hours. I could imagine that similar for perf
> would be useful, but I also know that maintainers are busy folks.

I've joined that channel now.

Cheers
-- 
Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer
KDAB (Deutschland) GmbH, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5272 bytes --]