Re: [PATCH] perf report: Add 'tgid' sort key

[Namhyung Kim <namhyung@kernel.org>]

On Wed, Feb 26, 2025 at 06:49:31PM -0300, Arnaldo Carvalho de Melo wrote:
> On Wed, Feb 26, 2025 at 06:40:29PM -0300, Arnaldo Carvalho de Melo wrote:
> > On Wed, Feb 26, 2025 at 06:38:50PM -0300, Arnaldo Carvalho de Melo wrote:
> > > On Wed, Feb 26, 2025 at 06:38:00PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > On Wed, Feb 26, 2025 at 11:34:13AM -0800, Namhyung Kim wrote:
> > > > > On Tue, Feb 25, 2025 at 08:48:56PM +0100, Arnaldo Carvalho de Melo wrote:
> > > > > > On Tue, Feb 25, 2025 at 08:25:35PM +0100, Arnaldo Carvalho de Melo wrote:
> > > > > > > On Tue, Feb 25, 2025 at 08:11:17PM +0100, Arnaldo Carvalho de Melo wrote:
> > > > > > > > On Tue, Feb 25, 2025 at 08:07:18PM +0100, Arnaldo Carvalho de Melo wrote:
> > > > > > > > > On Mon, Feb 24, 2025 at 11:51:35PM -0800, Namhyung Kim wrote:
> > > > > > > > > > On Mon, Feb 24, 2025 at 08:40:37PM -0800, Ian Rogers wrote:
> > > > > > > > > > > On Mon, Feb 24, 2025 at 6:51 PM Namhyung Kim <namhyung@kernel.org> wrote:
> > > > > > > > > > > > On Mon, Feb 24, 2025 at 10:18:37AM -0800, Ian Rogers wrote:
> > > > > > > > > > [SNIP]
> > > > > > > > > > > > > I thought the real-time processing had to use
> > > > > > > > > > > > > maps__fixup_overlap_and_insert (rather than maps__insert) as mmap
> > > > > > > > > > > > > events only give us VMA data and two mmaps may have been merged.
> > > > > > > > > > > > > Shouldn't doing this change be the simplest fix?
> > > > > > > > 
> > > > > > > > > > > > Make sense.  How about this?
> > > > > > > > 
> > > > > > > > > > > Lgtm, I have no way to test the issue. Why does maps__fixup_end need
> > > > > > > > > > > to get pushed later?
> > > > > > > > 
> > > > > > > > > > I just noticed it would add extra kernel maps after modules.  I think it
> > > > > > > > > > should fixup end address of the kernel maps after adding all maps first.
> > > > > > > > 
> > > > > > > > > > Arnaldo, can you please test this?
> > > > > > > >  
> > > > > > > > > Trying it now.
> > > > > > > > 
> > > > > > > > Now we have something different:
> > > > > > > > 
> > > > > > > > root@number:~# perf record sleep
> > > > > > > > sleep: missing operand
> > > > > > > > Try 'sleep --help' for more information.
> > > > > > > > [ perf record: Woken up 1 times to write data ]
> > > > > > > > perf: util/maps.c:80: check_invariants: Assertion `RC_CHK_EQUAL(map__kmap(map)->kmaps, maps)' failed.
> > > > > > > > Aborted (core dumped)
> > > > > > > > root@number:~#
> > > > > > > 
> > > > > > > __maps__insert() does:
> > > > > > > 
> > > > > > >         if (dso && dso__kernel(dso)) {
> > > > > > >                 struct kmap *kmap = map__kmap(new);
> > > > > > > 
> > > > > > >                 if (kmap)
> > > > > > >                         kmap->kmaps = maps;
> > > > > > >                 else
> > > > > > >                         pr_err("Internal error: kernel dso with non kernel map\n");
> > > > > > >         }
> > > > > > > 
> > > > > > > while maps__fixup_overlap_and_insert() doesn't.
> > > > > > > 
> > > > > > > It calls __maps__insert_sorted() that probably should do what
> > > > > > > __maps__insert() does?
> > > > > > 
> > > > > > Ok, so I did the following patch but this case fails:
> > > > > > 
> > > > > > @@ -910,6 +928,7 @@ static int __maps__fixup_overlap_and_insert(struct maps *maps, struct map *new)
> > > > > >                                  */
> > > > > >                                 map__put(maps_by_address[i]);
> > > > > >                                 maps_by_address[i] = map__get(new);
> > > > > > +                               map__set_kmap(new, maps);
> > > > > >                                 check_invariants(maps);
> > > > > >                                 return err;
> > > > > >                         }
> > > > > > 
> > > > > > With:
> > > > > > 
> > > > > > perf: util/maps.c:110: check_invariants: Assertion `refcount_read(map__refcnt(map)) > 1' failed.
> > > > > > 
> > > > > > As:
> > > > > > 
> > > > > > 106				/*
> > > > > > 107				 * Maps by name maps should be in maps_by_address, so
> > > > > > 108				 * the reference count should be higher.
> > > > > > 109				 */
> > > > > > 110				assert(refcount_read(map__refcnt(map)) > 1);
> > > > > > 
> > > > > > Since it is just replacing the map in the maps_by_address and not
> > > > > > touching on the maps_by_name? Thus the refcount is just 1:
> 
> > > > > Sounds like it.  Can you add this on top?
> 
> > > > Trying, but somehow its not applying cleanly, checking:
> 
> > > > ⬢ [acme@toolbox perf-tools]$ patch -p1 < ~/wb/1.patch 
> > > > patching file tools/perf/util/maps.c
> > > > Hunk #1 succeeded at 815 (offset 18 lines).
> > > > Hunk #2 succeeded at 826 (offset 18 lines).
> > > > Hunk #3 succeeded at 846 (offset 18 lines).
> > > > Hunk #4 succeeded at 893 (offset 18 lines).
> > > > Hunk #5 succeeded at 919 (offset 18 lines).
> > > > Hunk #6 FAILED at 923.
> > > > 1 out of 6 hunks FAILED -- saving rejects to file tools/perf/util/maps.c.rej
> > > > ⬢ [acme@toolbox perf-tools]$
> 
> > > > ⬢ [acme@toolbox perf-tools]$ git log --oneline -5
> > > > 4a9f5d76130b707f (HEAD -> perf-tools) wip: acme
> > > > d5ba0f5af35937c7 wip: namhyung
> > > > 42367eca7604e16e (perf-tools/tmp.perf-tools, perf-tools/perf-tools) tools: Remove redundant quiet setup
> > > > 293f324ce96d7001 tools: Unify top-level quiet infrastructure
> > > > 9fae5884bb0e3480 (tag: perf-tools-fixes-for-v6.14-2025-01-30) perf cpumap: Fix die and cluster IDs
> > > > ⬢ [acme@toolbox perf-tools]$
> 
> > > ⬢ [acme@toolbox perf-tools]$ cat tools/perf/util/maps.c.rej
> > > --- tools/perf/util/maps.c
> > > +++ tools/perf/util/maps.c
> > > @@ -923,6 +936,10 @@ static int __maps__fixup_overlap_and_insert(struct maps *maps, struct map *new)
> > >  				 */
> > >  				map__put(maps_by_address[i]);
> > >  				maps_by_address[i] = map__get(new);
> > > +				if (maps_by_name) {
> > > +					map__put(maps_by_name[ni]);
> > > +					maps_by_name[ni] = map__get(new);
> > > +				}
> > >  				check_invariants(maps);
> > >  				return err;
> > >  			}
> > > ⬢ [acme@toolbox perf-tools]$
> 
> > > Fixing this up by hand
>  
> > I see, I had tried this after sending that patch:
>  
> >      map__set_kmap(new, maps);
>  
> > Before check_invariants(), but that doesn't make sense, I should've
> > dropped that, doing it now.
> 
> Nope, it still triggers:
> 
>  930                 } else {
>  931                         struct map *next = NULL;
>  932 
>  933                         if (i + 1 < maps__nr_maps(maps))
>  934                                 next = maps_by_address[i + 1];
>  935 
>  936                         if (!next  || map__start(next) >= map__end(new)) {
>  937                                 /*
>  938                                  * Replace existing mapping and end knowing
>  939                                  * there aren't later overlapping or any
>  940                                  * mappings.
>  941                                  */
>  942                                 map__put(maps_by_address[i]);
>  943                                 maps_by_address[i] = map__get(new);
>  944                                 if (maps_by_name) {
>  945                                         map__put(maps_by_name[ni]);
>  946                                         maps_by_name[ni] = map__get(new);
>  947                                 }
>  948                                 check_invariants(maps);
>  949                                 return err;
>  950                         }
>  951                         __maps__remove(maps, pos);
>  952                         check_invariants(maps);
>  953                         /*
>  954                          * Maps are ordered but no need to increase `i` as the
>  955                          * later maps were moved down.
>  956                          */
>  957                 }
> 
> 
> perf: util/maps.c:80: check_invariants: Assertion `RC_CHK_EQUAL(map__kmap(map)->kmaps, maps)' failed.
> 
> Thread 1 "perf" received signal SIGABRT, Aborted.
> __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
> 44	      return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0;
> (gdb) bt
> #0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
> #1  0x00007ffff6ea80a3 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
> #2  0x00007ffff6e4ef1e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
> #3  0x00007ffff6e36902 in __GI_abort () at abort.c:79
> #4  0x00007ffff6e3681e in __assert_fail_base (fmt=0x7ffff6fc3bb8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x7bfd08 "RC_CHK_EQUAL(map__kmap(map)->kmaps, maps)", 
>     file=file@entry=0x7bfc53 "util/maps.c", line=line@entry=80, function=function@entry=0x7c0010 <__PRETTY_FUNCTION__.6> "check_invariants") at assert.c:96
> #5  0x00007ffff6e47047 in __assert_fail (assertion=0x7bfd08 "RC_CHK_EQUAL(map__kmap(map)->kmaps, maps)", file=0x7bfc53 "util/maps.c", line=80, 
>     function=0x7c0010 <__PRETTY_FUNCTION__.6> "check_invariants") at assert.c:105
> #6  0x0000000000633c74 in check_invariants (maps=0xf977c0) at util/maps.c:80
> #7  0x00000000006363a6 in __maps__fixup_overlap_and_insert (maps=0xf977c0, new=0xfc57e0) at util/maps.c:948
> #8  0x0000000000636460 in maps__fixup_overlap_and_insert (maps=0xf977c0, new=0xfc57e0) at util/maps.c:970
> #9  0x000000000062920a in machine__process_ksymbol_register (machine=0xf97158, event=0x7ffff7fbaba8, sample=0x7fffffffa860) at util/machine.c:715
> #10 0x00000000006294ca in machine__process_ksymbol (machine=0xf97158, event=0x7ffff7fbaba8, sample=0x7fffffffa860) at util/machine.c:779
> #11 0x00000000005ce2fd in perf_event__process_ksymbol (tool=0xec9ce0 <record>, event=0x7ffff7fbaba8, sample=0x7fffffffa860, machine=0xf97158) at util/event.c:296
> #12 0x000000000063b860 in machines__deliver_event (machines=0xf97158, evlist=0xf531f0, event=0x7ffff7fbaba8, sample=0x7fffffffa860, tool=0xec9ce0 <record>, file_offset=35752, 
>     file_path=0xf97850 "perf.data") at util/session.c:1334
> #13 0x000000000063ba45 in perf_session__deliver_event (session=0xf96f40, event=0x7ffff7fbaba8, tool=0xec9ce0 <record>, file_offset=35752, file_path=0xf97850 "perf.data")
>     at util/session.c:1367
> #14 0x000000000063c839 in perf_session__process_event (session=0xf96f40, event=0x7ffff7fbaba8, file_offset=35752, file_path=0xf97850 "perf.data") at util/session.c:1626
> #15 0x000000000063dfb9 in process_simple (session=0xf96f40, event=0x7ffff7fbaba8, file_offset=35752, file_path=0xf97850 "perf.data") at util/session.c:2203
> #16 0x000000000063dc70 in reader__read_event (rd=0x7fffffffafa0, session=0xf96f40, prog=0x7fffffffaf70) at util/session.c:2132
> #17 0x000000000063de6a in reader__process_events (rd=0x7fffffffafa0, session=0xf96f40, prog=0x7fffffffaf70) at util/session.c:2181
> #18 0x000000000063e107 in __perf_session__process_events (session=0xf96f40) at util/session.c:2226
> #19 0x000000000063eb04 in perf_session__process_events (session=0xf96f40) at util/session.c:2390
> #20 0x000000000042d98b in process_buildids (rec=0xec9ce0 <record>) at builtin-record.c:1475
> #21 0x000000000042e963 in record__finish_output (rec=0xec9ce0 <record>) at builtin-record.c:1798
> #22 0x0000000000431c46 in __cmd_record (rec=0xec9ce0 <record>, argc=2, argv=0x7fffffffde80) at builtin-record.c:2841
> #23 0x000000000043513f in cmd_record (argc=2, argv=0x7fffffffde80) at builtin-record.c:4260
> #24 0x00000000004bcf65 in run_builtin (p=0xeccd60 <commands+288>, argc=3, argv=0x7fffffffde80) at perf.c:351
> #25 0x00000000004bd20c in handle_internal_command (argc=3, argv=0x7fffffffde80) at perf.c:404
> #26 0x00000000004bd365 in run_argv (argcp=0x7fffffffdc6c, argv=0x7fffffffdc60) at perf.c:448
> #27 0x00000000004bd6ae in main (argc=3, argv=0x7fffffffde80) at perf.c:556
> (gdb)
> 
> humm, it seems that thing I removed may be needed after all...
> 
> Yeah, please check if adding this, on top of your latest (second) patch,
> makese sense:
> 
> diff --git a/tools/perf/util/maps.c b/tools/perf/util/maps.c
> index a710720e8bcfd305..776bbdaafdb32269 100644
> --- a/tools/perf/util/maps.c
> +++ b/tools/perf/util/maps.c
> @@ -945,6 +945,7 @@ static int __maps__fixup_overlap_and_insert(struct maps *maps, struct map *new)
>                                         map__put(maps_by_name[ni]);
>                                         maps_by_name[ni] = map__get(new);
>                                 }
> +                               map__set_kmap(new, maps);
>                                 check_invariants(maps);
>                                 return err;
>                         }
> ⬢ [acme@toolbox perf-tools]$
> 
> 
> With your two patches and my two patches finally I get:
> 
> root@number:~# perf record sleep 1
> [ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.035 MB perf.data (14 samples) ]
> root@number:~#
> 
> on a perf-tools/perf-tools build with DEBUG=1

Great!  Can you please send the final version as a formal patch?

Thanks,
Namhyung