[PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Donglin Peng]

From: Donglin Peng <pengdonglin@xiaomi.com>

Use BTF to print symbolic names for enum-type function arguments,
improving trace readability.

Before:
count_memcg_events(memcg=0xffff..., idx=20, count=0x1) {

After:
count_memcg_events(memcg=0xffff..., idx=20 [PGFAULT], count=0x1) {

Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: linux-trace-kernel@vger.kernel.org
Signed-off-by: Donglin Peng <pengdonglin@xiaomi.com>
---
 kernel/trace/trace_output.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/kernel/trace/trace_output.c b/kernel/trace/trace_output.c
index cc2d3306bb60..c395f768c3b8 100644
--- a/kernel/trace/trace_output.c
+++ b/kernel/trace/trace_output.c
@@ -695,12 +695,13 @@ void print_function_args(struct trace_seq *s, unsigned long *args,
 {
 	const struct btf_param *param;
 	const struct btf_type *t;
+	const struct btf_enum *enump;
 	const char *param_name;
 	char name[KSYM_NAME_LEN];
 	unsigned long arg;
 	struct btf *btf;
 	s32 tid, nr = 0;
-	int a, p, x;
+	int a, p, x, i;
 	u16 encode;
 
 	trace_seq_printf(s, "(");
@@ -754,6 +755,14 @@ void print_function_args(struct trace_seq *s, unsigned long *args,
 			break;
 		case BTF_KIND_ENUM:
 			trace_seq_printf(s, "%ld", arg);
+			for_each_enum(i, t, enump) {
+				if (arg == enump->val) {
+					trace_seq_printf(s, " [%s]",
+							 btf_name_by_offset(btf,
+							 enump->name_off));
+					break;
+				}
+			}
 			break;
 		default:
 			/* This does not handle complex arguments */
-- 
2.34.1

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Steven Rostedt]

On Mon,  2 Feb 2026 19:15:48 +0800
Donglin Peng <dolinux.peng@gmail.com> wrote:

> @@ -754,6 +755,14 @@ void print_function_args(struct trace_seq *s, unsigned long *args,
>  			break;
>  		case BTF_KIND_ENUM:
>  			trace_seq_printf(s, "%ld", arg);
> +			for_each_enum(i, t, enump) {
> +				if (arg == enump->val) {
> +					trace_seq_printf(s, " [%s]",
> +							 btf_name_by_offset(btf,
> +							 enump->name_off));
> +					break;
> +				}
> +			}
>  			break;

I have to ask; how big is that enum list?

Do we really want to do a linear search for every enum we come across?

What we could do is for the first time we hit an enum, create an array of
enums, sort them, and do a binary search from then on. This could be saved
in the tracing code itself (in kernel/trace/trace_btf.c), if BPF doesn't
care about it.

-- Steve

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Donglin Peng]

On Tue, Feb 3, 2026 at 12:12 AM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Mon,  2 Feb 2026 19:15:48 +0800
> Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> > @@ -754,6 +755,14 @@ void print_function_args(struct trace_seq *s, unsigned long *args,
> >                       break;
> >               case BTF_KIND_ENUM:
> >                       trace_seq_printf(s, "%ld", arg);
> > +                     for_each_enum(i, t, enump) {
> > +                             if (arg == enump->val) {
> > +                                     trace_seq_printf(s, " [%s]",
> > +                                                      btf_name_by_offset(btf,
> > +                                                      enump->name_off));
> > +                                     break;
> > +                             }
> > +                     }
> >                       break;
>
> I have to ask; how big is that enum list?

Based on local vmlinux analysis (6.18 kernel):
- 78% enums have vlen ≤ 10
- 95% enums have vlen ≤ 50
- Max observed vlen: 349 (extremely rare case)

$ bpftool btf dump file x86_64/vmlinux | grep "^\[" | grep ENUM | grep
-v "(anon)" | awk -F '[ =]' '{
    for(i=1; i<=NF; i++) {
        if ($i == "vlen") {
            count[$(i+1)]++
        }
    }
}
END {
    for (vlen in count) {
        print "vlen=" vlen ": " count[vlen] " times"
    }
}' | sort -k 2 -n
vlen=104: 1 times
vlen=114: 1 times
vlen=116: 1 times
vlen=129: 1 times
vlen=146: 1 times
vlen=167: 1 times
vlen=181: 1 times
vlen=183: 1 times
vlen=201: 1 times
vlen=213: 1 times
vlen=33: 1 times
vlen=349: 1 times
vlen=42: 1 times
vlen=45: 1 times
vlen=48: 1 times
vlen=50: 1 times
vlen=54: 1 times
vlen=55: 1 times
vlen=56: 1 times
vlen=59: 1 times
vlen=60: 1 times
vlen=61: 1 times
vlen=64: 1 times
vlen=68: 1 times
vlen=71: 1 times
vlen=74: 1 times
vlen=83: 1 times
vlen=85: 1 times
vlen=87: 1 times
vlen=122: 2 times
vlen=38: 2 times
vlen=39: 2 times
vlen=43: 2 times
vlen=44: 2 times
vlen=49: 2 times
vlen=52: 2 times
vlen=58: 2 times
vlen=73: 2 times
vlen=79: 2 times
vlen=35: 3 times
vlen=41: 3 times
vlen=46: 3 times
vlen=25: 4 times
vlen=27: 4 times
vlen=28: 4 times
vlen=29: 4 times
vlen=34: 4 times
vlen=37: 4 times
vlen=40: 4 times
vlen=19: 5 times
vlen=30: 5 times
vlen=20: 6 times
vlen=22: 6 times
vlen=24: 6 times
vlen=16: 8 times
vlen=23: 8 times
vlen=26: 8 times
vlen=31: 8 times
vlen=32: 8 times
vlen=17: 11 times
vlen=18: 12 times
vlen=21: 12 times
vlen=15: 13 times
vlen=14: 21 times
vlen=12: 25 times
vlen=13: 25 times
vlen=10: 34 times
vlen=11: 35 times
vlen=1: 58 times
vlen=8: 70 times
vlen=9: 74 times
vlen=7: 102 times
vlen=6: 108 times
vlen=5: 170 times
vlen=2: 193 times
vlen=4: 250 times
vlen=3: 333 times

>
> Do we really want to do a linear search for every enum we come across?

I think linear search is efficient for typical enum value lists (vlen < 10), I
also support using binary search where feasible.

>
> What we could do is for the first time we hit an enum, create an array of
> enums, sort them, and do a binary search from then on. This could be saved
> in the tracing code itself (in kernel/trace/trace_btf.c), if BPF doesn't
> care about it.

I think this approach adds unnecessary complexity. Instead, we could
leverage btf2btf during the resolve_btfids phase to proactively sort the
btf_enum entries of enum-type btf_type by their value, enabling efficient
binary search lookups.

>
> -- Steve

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Donglin Peng]

On Tue, Feb 3, 2026 at 10:17 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Tue, Feb 3, 2026 at 12:12 AM Steven Rostedt <rostedt@goodmis.org> wrote:
> >
> > On Mon,  2 Feb 2026 19:15:48 +0800
> > Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > > @@ -754,6 +755,14 @@ void print_function_args(struct trace_seq *s, unsigned long *args,
> > >                       break;
> > >               case BTF_KIND_ENUM:
> > >                       trace_seq_printf(s, "%ld", arg);
> > > +                     for_each_enum(i, t, enump) {
> > > +                             if (arg == enump->val) {
> > > +                                     trace_seq_printf(s, " [%s]",
> > > +                                                      btf_name_by_offset(btf,
> > > +                                                      enump->name_off));
> > > +                                     break;
> > > +                             }
> > > +                     }
> > >                       break;
> >
> > I have to ask; how big is that enum list?
>
> Based on local vmlinux analysis (6.18 kernel):
> - 78% enums have vlen ≤ 10
> - 95% enums have vlen ≤ 50
> - Max observed vlen: 349 (extremely rare case)
>
> $ bpftool btf dump file x86_64/vmlinux | grep "^\[" | grep ENUM | grep
> -v "(anon)" | awk -F '[ =]' '{
>     for(i=1; i<=NF; i++) {
>         if ($i == "vlen") {
>             count[$(i+1)]++
>         }
>     }
> }
> END {
>     for (vlen in count) {
>         print "vlen=" vlen ": " count[vlen] " times"
>     }
> }' | sort -k 2 -n
> vlen=104: 1 times
> vlen=114: 1 times
> vlen=116: 1 times
> vlen=129: 1 times
> vlen=146: 1 times
> vlen=167: 1 times
> vlen=181: 1 times
> vlen=183: 1 times
> vlen=201: 1 times
> vlen=213: 1 times
> vlen=33: 1 times
> vlen=349: 1 times
> vlen=42: 1 times
> vlen=45: 1 times
> vlen=48: 1 times
> vlen=50: 1 times
> vlen=54: 1 times
> vlen=55: 1 times
> vlen=56: 1 times
> vlen=59: 1 times
> vlen=60: 1 times
> vlen=61: 1 times
> vlen=64: 1 times
> vlen=68: 1 times
> vlen=71: 1 times
> vlen=74: 1 times
> vlen=83: 1 times
> vlen=85: 1 times
> vlen=87: 1 times
> vlen=122: 2 times
> vlen=38: 2 times
> vlen=39: 2 times
> vlen=43: 2 times
> vlen=44: 2 times
> vlen=49: 2 times
> vlen=52: 2 times
> vlen=58: 2 times
> vlen=73: 2 times
> vlen=79: 2 times
> vlen=35: 3 times
> vlen=41: 3 times
> vlen=46: 3 times
> vlen=25: 4 times
> vlen=27: 4 times
> vlen=28: 4 times
> vlen=29: 4 times
> vlen=34: 4 times
> vlen=37: 4 times
> vlen=40: 4 times
> vlen=19: 5 times
> vlen=30: 5 times
> vlen=20: 6 times
> vlen=22: 6 times
> vlen=24: 6 times
> vlen=16: 8 times
> vlen=23: 8 times
> vlen=26: 8 times
> vlen=31: 8 times
> vlen=32: 8 times
> vlen=17: 11 times
> vlen=18: 12 times
> vlen=21: 12 times
> vlen=15: 13 times
> vlen=14: 21 times
> vlen=12: 25 times
> vlen=13: 25 times
> vlen=10: 34 times
> vlen=11: 35 times
> vlen=1: 58 times
> vlen=8: 70 times
> vlen=9: 74 times
> vlen=7: 102 times
> vlen=6: 108 times
> vlen=5: 170 times
> vlen=2: 193 times
> vlen=4: 250 times
> vlen=3: 333 times
>
> >
> > Do we really want to do a linear search for every enum we come across?
>
> I think linear search is efficient for typical enum value lists (vlen < 10), I
> also support using binary search where feasible.
>
> >
> > What we could do is for the first time we hit an enum, create an array of
> > enums, sort them, and do a binary search from then on. This could be saved
> > in the tracing code itself (in kernel/trace/trace_btf.c), if BPF doesn't
> > care about it.
>
> I think this approach adds unnecessary complexity. Instead, we could
> leverage btf2btf during the resolve_btfids phase to proactively sort the
> btf_enum entries of enum-type btf_type by their value, enabling efficient
> binary search lookups.

Testing revealed that sorting within resolve_btfids introduces issues with
btf__dedup. Therefore, I plan to move the sorting logic directly into
btf__add_enum_value and btf__add_enum64_value in libbpf, which are
invoked by pahole. However, it means that we need a newer pahole
version.

>
> >
> > -- Steve

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Steven Rostedt]

On Tue, 3 Feb 2026 21:50:47 +0800
Donglin Peng <dolinux.peng@gmail.com> wrote:

> Testing revealed that sorting within resolve_btfids introduces issues with
> btf__dedup. Therefore, I plan to move the sorting logic directly into
> btf__add_enum_value and btf__add_enum64_value in libbpf, which are
> invoked by pahole. However, it means that we need a newer pahole
> version.

Sorting isn't a requirement just something I wanted to bring up. If it's
too complex and doesn't achieve much benefit then let's not do it.

My worry is because "cat trace" takes quite a long time just reading the
BTF arguments. I'm worried it will just get worse with enums as well.

I have trace-cmd reading BTF now (just haven't officially released it) and
doing an extract and reading the trace.dat file is much faster than reading
the trace file with arguments. I'll need to implement the enum logic too in
libtraceevent.

-- Steve

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Alexei Starovoitov]

On Tue, Feb 3, 2026 at 7:16 AM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Tue, 3 Feb 2026 21:50:47 +0800
> Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> > Testing revealed that sorting within resolve_btfids introduces issues with
> > btf__dedup. Therefore, I plan to move the sorting logic directly into
> > btf__add_enum_value and btf__add_enum64_value in libbpf, which are
> > invoked by pahole. However, it means that we need a newer pahole
> > version.
>
> Sorting isn't a requirement just something I wanted to bring up. If it's
> too complex and doesn't achieve much benefit then let's not do it.
>
> My worry is because "cat trace" takes quite a long time just reading the
> BTF arguments. I'm worried it will just get worse with enums as well.
>
> I have trace-cmd reading BTF now (just haven't officially released it) and
> doing an extract and reading the trace.dat file is much faster than reading
> the trace file with arguments. I'll need to implement the enum logic too in
> libtraceevent.

If you mean to do pretty printing of the trace in user space then +1 from me.

I don't like sorting enums either in resolve_btfid, pahole or kernel.
Sorted BTF by name was ok, since it doesn't change original semantics.
While sorting enums by value gets us to the grey zone where
the sequence of enum names in vmlinux.h becomes different than in dwarf.

Also id->name mapping in general is not precise.
There is no requirement for enums to be unique.
Just grabbing the first one:
ATA_PIO0 = 1,
ATA_PIO1 = 3,
ATA_PIO2 = 7,
ATA_UDMA0 = 1,
ATA_UDMA1 = 3,
ATA_UDMA2 = 7,
ATA_ID_CYLS = 1,
ATA_ID_HEADS = 3,
SCR_ERROR = 1,
SCR_CONTROL = 2,
SCR_ACTIVE = 3,

All these names are part of the same enum type.
Which one to print? First one?

another example:
enum {
        BPF_LOCAL_STORAGE_GET_F_CREATE = 1,
        BPF_SK_STORAGE_GET_F_CREATE = 1,
};

and another:
enum {
        BPF_SKB_TSTAMP_UNSPEC = 0,
        BPF_SKB_TSTAMP_DELIVERY_MONO = 1,
        BPF_SKB_CLOCK_REALTIME = 0,
        BPF_SKB_CLOCK_MONOTONIC = 1,
        BPF_SKB_CLOCK_TAI = 2,
};

I'd rather not print any and keep it integer only.

pw-bot: cr

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Donglin Peng]

On Wed, Feb 4, 2026 at 12:00 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Tue, Feb 3, 2026 at 7:16 AM Steven Rostedt <rostedt@goodmis.org> wrote:
> >
> > On Tue, 3 Feb 2026 21:50:47 +0800
> > Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > > Testing revealed that sorting within resolve_btfids introduces issues with
> > > btf__dedup. Therefore, I plan to move the sorting logic directly into
> > > btf__add_enum_value and btf__add_enum64_value in libbpf, which are
> > > invoked by pahole. However, it means that we need a newer pahole
> > > version.
> >
> > Sorting isn't a requirement just something I wanted to bring up. If it's
> > too complex and doesn't achieve much benefit then let's not do it.
> >
> > My worry is because "cat trace" takes quite a long time just reading the
> > BTF arguments. I'm worried it will just get worse with enums as well.
> >
> > I have trace-cmd reading BTF now (just haven't officially released it) and
> > doing an extract and reading the trace.dat file is much faster than reading
> > the trace file with arguments. I'll need to implement the enum logic too in
> > libtraceevent.
>
> If you mean to do pretty printing of the trace in user space then +1 from me.
>
> I don't like sorting enums either in resolve_btfid, pahole or kernel.
> Sorted BTF by name was ok, since it doesn't change original semantics.
> While sorting enums by value gets us to the grey zone where
> the sequence of enum names in vmlinux.h becomes different than in dwarf.

Thanks, I agreed.

>
> Also id->name mapping in general is not precise.
> There is no requirement for enums to be unique.
> Just grabbing the first one:
> ATA_PIO0 = 1,
> ATA_PIO1 = 3,
> ATA_PIO2 = 7,
> ATA_UDMA0 = 1,
> ATA_UDMA1 = 3,
> ATA_UDMA2 = 7,
> ATA_ID_CYLS = 1,
> ATA_ID_HEADS = 3,
> SCR_ERROR = 1,
> SCR_CONTROL = 2,
> SCR_ACTIVE = 3,
>
> All these names are part of the same enum type.
> Which one to print? First one?

I think these cases are not very common and printing the first
one would be helpful enough, and we can add documentation
notes in ftrace to guide users.

>
> another example:
> enum {
>         BPF_LOCAL_STORAGE_GET_F_CREATE = 1,
>         BPF_SK_STORAGE_GET_F_CREATE = 1,
> };
>
> and another:
> enum {
>         BPF_SKB_TSTAMP_UNSPEC = 0,
>         BPF_SKB_TSTAMP_DELIVERY_MONO = 1,
>         BPF_SKB_CLOCK_REALTIME = 0,
>         BPF_SKB_CLOCK_MONOTONIC = 1,
>         BPF_SKB_CLOCK_TAI = 2,
> };
>
> I'd rather not print any and keep it integer only.
>
> pw-bot: cr

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Donglin Peng]

On Wed, Feb 4, 2026 at 10:52 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Wed, Feb 4, 2026 at 12:00 AM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Tue, Feb 3, 2026 at 7:16 AM Steven Rostedt <rostedt@goodmis.org> wrote:
> > >
> > > On Tue, 3 Feb 2026 21:50:47 +0800
> > > Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > > Testing revealed that sorting within resolve_btfids introduces issues with
> > > > btf__dedup. Therefore, I plan to move the sorting logic directly into
> > > > btf__add_enum_value and btf__add_enum64_value in libbpf, which are
> > > > invoked by pahole. However, it means that we need a newer pahole
> > > > version.
> > >
> > > Sorting isn't a requirement just something I wanted to bring up. If it's
> > > too complex and doesn't achieve much benefit then let's not do it.
> > >
> > > My worry is because "cat trace" takes quite a long time just reading the
> > > BTF arguments. I'm worried it will just get worse with enums as well.
> > >
> > > I have trace-cmd reading BTF now (just haven't officially released it) and
> > > doing an extract and reading the trace.dat file is much faster than reading
> > > the trace file with arguments. I'll need to implement the enum logic too in
> > > libtraceevent.
> >
> > If you mean to do pretty printing of the trace in user space then +1 from me.
> >
> > I don't like sorting enums either in resolve_btfid, pahole or kernel.
> > Sorted BTF by name was ok, since it doesn't change original semantics.
> > While sorting enums by value gets us to the grey zone where
> > the sequence of enum names in vmlinux.h becomes different than in dwarf.
>
> Thanks, I agreed.
>
> >
> > Also id->name mapping in general is not precise.
> > There is no requirement for enums to be unique.
> > Just grabbing the first one:
> > ATA_PIO0 = 1,
> > ATA_PIO1 = 3,
> > ATA_PIO2 = 7,
> > ATA_UDMA0 = 1,
> > ATA_UDMA1 = 3,
> > ATA_UDMA2 = 7,
> > ATA_ID_CYLS = 1,
> > ATA_ID_HEADS = 3,
> > SCR_ERROR = 1,
> > SCR_CONTROL = 2,
> > SCR_ACTIVE = 3,
> >
> > All these names are part of the same enum type.
> > Which one to print? First one?

Another option is to print all matching entries, incurring increased
overhead and extended trace log length. However, I prefer printing
the first matching entry, though it might be inaccurate in rare cases.

>
> I think these cases are not very common and printing the first
> one would be helpful enough, and we can add documentation
> notes in ftrace to guide users.
>
> >
> > another example:
> > enum {
> >         BPF_LOCAL_STORAGE_GET_F_CREATE = 1,
> >         BPF_SK_STORAGE_GET_F_CREATE = 1,
> > };
> >
> > and another:
> > enum {
> >         BPF_SKB_TSTAMP_UNSPEC = 0,
> >         BPF_SKB_TSTAMP_DELIVERY_MONO = 1,
> >         BPF_SKB_CLOCK_REALTIME = 0,
> >         BPF_SKB_CLOCK_MONOTONIC = 1,
> >         BPF_SKB_CLOCK_TAI = 2,
> > };
> >
> > I'd rather not print any and keep it integer only.
> >
> > pw-bot: cr

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Alexei Starovoitov]

On Thu, Feb 5, 2026 at 1:21 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Wed, Feb 4, 2026 at 10:52 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > On Wed, Feb 4, 2026 at 12:00 AM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Tue, Feb 3, 2026 at 7:16 AM Steven Rostedt <rostedt@goodmis.org> wrote:
> > > >
> > > > On Tue, 3 Feb 2026 21:50:47 +0800
> > > > Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > >
> > > > > Testing revealed that sorting within resolve_btfids introduces issues with
> > > > > btf__dedup. Therefore, I plan to move the sorting logic directly into
> > > > > btf__add_enum_value and btf__add_enum64_value in libbpf, which are
> > > > > invoked by pahole. However, it means that we need a newer pahole
> > > > > version.
> > > >
> > > > Sorting isn't a requirement just something I wanted to bring up. If it's
> > > > too complex and doesn't achieve much benefit then let's not do it.
> > > >
> > > > My worry is because "cat trace" takes quite a long time just reading the
> > > > BTF arguments. I'm worried it will just get worse with enums as well.
> > > >
> > > > I have trace-cmd reading BTF now (just haven't officially released it) and
> > > > doing an extract and reading the trace.dat file is much faster than reading
> > > > the trace file with arguments. I'll need to implement the enum logic too in
> > > > libtraceevent.
> > >
> > > If you mean to do pretty printing of the trace in user space then +1 from me.
> > >
> > > I don't like sorting enums either in resolve_btfid, pahole or kernel.
> > > Sorted BTF by name was ok, since it doesn't change original semantics.
> > > While sorting enums by value gets us to the grey zone where
> > > the sequence of enum names in vmlinux.h becomes different than in dwarf.
> >
> > Thanks, I agreed.
> >
> > >
> > > Also id->name mapping in general is not precise.
> > > There is no requirement for enums to be unique.
> > > Just grabbing the first one:
> > > ATA_PIO0 = 1,
> > > ATA_PIO1 = 3,
> > > ATA_PIO2 = 7,
> > > ATA_UDMA0 = 1,
> > > ATA_UDMA1 = 3,
> > > ATA_UDMA2 = 7,
> > > ATA_ID_CYLS = 1,
> > > ATA_ID_HEADS = 3,
> > > SCR_ERROR = 1,
> > > SCR_CONTROL = 2,
> > > SCR_ACTIVE = 3,
> > >
> > > All these names are part of the same enum type.
> > > Which one to print? First one?
>
> Another option is to print all matching entries, incurring increased
> overhead and extended trace log length. However, I prefer printing
> the first matching entry, though it might be inaccurate in rare cases.

I disagree. It's not rare.
I wouldn't print anything. Let user space deal with it.

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Donglin Peng]

On Thu, Feb 5, 2026 at 11:56 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Thu, Feb 5, 2026 at 1:21 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > On Wed, Feb 4, 2026 at 10:52 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > On Wed, Feb 4, 2026 at 12:00 AM Alexei Starovoitov
> > > <alexei.starovoitov@gmail.com> wrote:
> > > >
> > > > On Tue, Feb 3, 2026 at 7:16 AM Steven Rostedt <rostedt@goodmis.org> wrote:
> > > > >
> > > > > On Tue, 3 Feb 2026 21:50:47 +0800
> > > > > Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > >
> > > > > > Testing revealed that sorting within resolve_btfids introduces issues with
> > > > > > btf__dedup. Therefore, I plan to move the sorting logic directly into
> > > > > > btf__add_enum_value and btf__add_enum64_value in libbpf, which are
> > > > > > invoked by pahole. However, it means that we need a newer pahole
> > > > > > version.
> > > > >
> > > > > Sorting isn't a requirement just something I wanted to bring up. If it's
> > > > > too complex and doesn't achieve much benefit then let's not do it.
> > > > >
> > > > > My worry is because "cat trace" takes quite a long time just reading the
> > > > > BTF arguments. I'm worried it will just get worse with enums as well.
> > > > >
> > > > > I have trace-cmd reading BTF now (just haven't officially released it) and
> > > > > doing an extract and reading the trace.dat file is much faster than reading
> > > > > the trace file with arguments. I'll need to implement the enum logic too in
> > > > > libtraceevent.
> > > >
> > > > If you mean to do pretty printing of the trace in user space then +1 from me.
> > > >
> > > > I don't like sorting enums either in resolve_btfid, pahole or kernel.
> > > > Sorted BTF by name was ok, since it doesn't change original semantics.
> > > > While sorting enums by value gets us to the grey zone where
> > > > the sequence of enum names in vmlinux.h becomes different than in dwarf.
> > >
> > > Thanks, I agreed.
> > >
> > > >
> > > > Also id->name mapping in general is not precise.
> > > > There is no requirement for enums to be unique.
> > > > Just grabbing the first one:
> > > > ATA_PIO0 = 1,
> > > > ATA_PIO1 = 3,
> > > > ATA_PIO2 = 7,
> > > > ATA_UDMA0 = 1,
> > > > ATA_UDMA1 = 3,
> > > > ATA_UDMA2 = 7,
> > > > ATA_ID_CYLS = 1,
> > > > ATA_ID_HEADS = 3,
> > > > SCR_ERROR = 1,
> > > > SCR_CONTROL = 2,
> > > > SCR_ACTIVE = 3,
> > > >
> > > > All these names are part of the same enum type.
> > > > Which one to print? First one?
> >
> > Another option is to print all matching entries, incurring increased
> > overhead and extended trace log length. However, I prefer printing
> > the first matching entry, though it might be inaccurate in rare cases.
>
> I disagree. It's not rare.
> I wouldn't print anything. Let user space deal with it.

Okay, I will implement this in libtraceevent first. By the way, would the first
patch [1] introducing the for_each_enumand for_each_enum64 helper
macros be acceptable?

[1] https://lore.kernel.org/lkml/20260202111548.3555306-2-dolinux.peng@gmail.com/

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Alexei Starovoitov]

On Thu, Feb 5, 2026 at 8:10 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Thu, Feb 5, 2026 at 11:56 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Thu, Feb 5, 2026 at 1:21 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > On Wed, Feb 4, 2026 at 10:52 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > >
> > > > On Wed, Feb 4, 2026 at 12:00 AM Alexei Starovoitov
> > > > <alexei.starovoitov@gmail.com> wrote:
> > > > >
> > > > > On Tue, Feb 3, 2026 at 7:16 AM Steven Rostedt <rostedt@goodmis.org> wrote:
> > > > > >
> > > > > > On Tue, 3 Feb 2026 21:50:47 +0800
> > > > > > Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > > >
> > > > > > > Testing revealed that sorting within resolve_btfids introduces issues with
> > > > > > > btf__dedup. Therefore, I plan to move the sorting logic directly into
> > > > > > > btf__add_enum_value and btf__add_enum64_value in libbpf, which are
> > > > > > > invoked by pahole. However, it means that we need a newer pahole
> > > > > > > version.
> > > > > >
> > > > > > Sorting isn't a requirement just something I wanted to bring up. If it's
> > > > > > too complex and doesn't achieve much benefit then let's not do it.
> > > > > >
> > > > > > My worry is because "cat trace" takes quite a long time just reading the
> > > > > > BTF arguments. I'm worried it will just get worse with enums as well.
> > > > > >
> > > > > > I have trace-cmd reading BTF now (just haven't officially released it) and
> > > > > > doing an extract and reading the trace.dat file is much faster than reading
> > > > > > the trace file with arguments. I'll need to implement the enum logic too in
> > > > > > libtraceevent.
> > > > >
> > > > > If you mean to do pretty printing of the trace in user space then +1 from me.
> > > > >
> > > > > I don't like sorting enums either in resolve_btfid, pahole or kernel.
> > > > > Sorted BTF by name was ok, since it doesn't change original semantics.
> > > > > While sorting enums by value gets us to the grey zone where
> > > > > the sequence of enum names in vmlinux.h becomes different than in dwarf.
> > > >
> > > > Thanks, I agreed.
> > > >
> > > > >
> > > > > Also id->name mapping in general is not precise.
> > > > > There is no requirement for enums to be unique.
> > > > > Just grabbing the first one:
> > > > > ATA_PIO0 = 1,
> > > > > ATA_PIO1 = 3,
> > > > > ATA_PIO2 = 7,
> > > > > ATA_UDMA0 = 1,
> > > > > ATA_UDMA1 = 3,
> > > > > ATA_UDMA2 = 7,
> > > > > ATA_ID_CYLS = 1,
> > > > > ATA_ID_HEADS = 3,
> > > > > SCR_ERROR = 1,
> > > > > SCR_CONTROL = 2,
> > > > > SCR_ACTIVE = 3,
> > > > >
> > > > > All these names are part of the same enum type.
> > > > > Which one to print? First one?
> > >
> > > Another option is to print all matching entries, incurring increased
> > > overhead and extended trace log length. However, I prefer printing
> > > the first matching entry, though it might be inaccurate in rare cases.
> >
> > I disagree. It's not rare.
> > I wouldn't print anything. Let user space deal with it.
>
> Okay, I will implement this in libtraceevent first. By the way, would the first
> patch [1] introducing the for_each_enumand for_each_enum64 helper
> macros be acceptable?
>
> [1] https://lore.kernel.org/lkml/20260202111548.3555306-2-dolinux.peng@gmail.com/

Just that patch alone? What's the point?
Refactor for what? Does it read better? No.

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Donglin Peng]

On Sat, Feb 7, 2026 at 12:05 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Thu, Feb 5, 2026 at 8:10 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > On Thu, Feb 5, 2026 at 11:56 PM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Thu, Feb 5, 2026 at 1:21 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > >
> > > > On Wed, Feb 4, 2026 at 10:52 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > >
> > > > > On Wed, Feb 4, 2026 at 12:00 AM Alexei Starovoitov
> > > > > <alexei.starovoitov@gmail.com> wrote:
> > > > > >
> > > > > > On Tue, Feb 3, 2026 at 7:16 AM Steven Rostedt <rostedt@goodmis.org> wrote:
> > > > > > >
> > > > > > > On Tue, 3 Feb 2026 21:50:47 +0800
> > > > > > > Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > > > >
> > > > > > > > Testing revealed that sorting within resolve_btfids introduces issues with
> > > > > > > > btf__dedup. Therefore, I plan to move the sorting logic directly into
> > > > > > > > btf__add_enum_value and btf__add_enum64_value in libbpf, which are
> > > > > > > > invoked by pahole. However, it means that we need a newer pahole
> > > > > > > > version.
> > > > > > >
> > > > > > > Sorting isn't a requirement just something I wanted to bring up. If it's
> > > > > > > too complex and doesn't achieve much benefit then let's not do it.
> > > > > > >
> > > > > > > My worry is because "cat trace" takes quite a long time just reading the
> > > > > > > BTF arguments. I'm worried it will just get worse with enums as well.
> > > > > > >
> > > > > > > I have trace-cmd reading BTF now (just haven't officially released it) and
> > > > > > > doing an extract and reading the trace.dat file is much faster than reading
> > > > > > > the trace file with arguments. I'll need to implement the enum logic too in
> > > > > > > libtraceevent.
> > > > > >
> > > > > > If you mean to do pretty printing of the trace in user space then +1 from me.
> > > > > >
> > > > > > I don't like sorting enums either in resolve_btfid, pahole or kernel.
> > > > > > Sorted BTF by name was ok, since it doesn't change original semantics.
> > > > > > While sorting enums by value gets us to the grey zone where
> > > > > > the sequence of enum names in vmlinux.h becomes different than in dwarf.
> > > > >
> > > > > Thanks, I agreed.
> > > > >
> > > > > >
> > > > > > Also id->name mapping in general is not precise.
> > > > > > There is no requirement for enums to be unique.
> > > > > > Just grabbing the first one:
> > > > > > ATA_PIO0 = 1,
> > > > > > ATA_PIO1 = 3,
> > > > > > ATA_PIO2 = 7,
> > > > > > ATA_UDMA0 = 1,
> > > > > > ATA_UDMA1 = 3,
> > > > > > ATA_UDMA2 = 7,
> > > > > > ATA_ID_CYLS = 1,
> > > > > > ATA_ID_HEADS = 3,
> > > > > > SCR_ERROR = 1,
> > > > > > SCR_CONTROL = 2,
> > > > > > SCR_ACTIVE = 3,
> > > > > >
> > > > > > All these names are part of the same enum type.
> > > > > > Which one to print? First one?
> > > >
> > > > Another option is to print all matching entries, incurring increased
> > > > overhead and extended trace log length. However, I prefer printing
> > > > the first matching entry, though it might be inaccurate in rare cases.
> > >
> > > I disagree. It's not rare.
> > > I wouldn't print anything. Let user space deal with it.
> >
> > Okay, I will implement this in libtraceevent first. By the way, would the first
> > patch [1] introducing the for_each_enumand for_each_enum64 helper
> > macros be acceptable?
> >
> > [1] https://lore.kernel.org/lkml/20260202111548.3555306-2-dolinux.peng@gmail.com/
>
> Just that patch alone? What's the point?
> Refactor for what? Does it read better? No.

Thanks, I see.

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Steven Rostedt]

On Wed, 4 Feb 2026 22:52:11 +0800
Donglin Peng <dolinux.peng@gmail.com> wrote:

> > > I have trace-cmd reading BTF now (just haven't officially released it) and
> > > doing an extract and reading the trace.dat file is much faster than reading
> > > the trace file with arguments. I'll need to implement the enum logic too in
> > > libtraceevent.  
> >
> > If you mean to do pretty printing of the trace in user space then +1 from me.
> >
> > I don't like sorting enums either in resolve_btfid, pahole or kernel.
> > Sorted BTF by name was ok, since it doesn't change original semantics.
> > While sorting enums by value gets us to the grey zone where
> > the sequence of enum names in vmlinux.h becomes different than in dwarf.  
> 
> Thanks, I agreed.

BTW, I just officially released trace-cmd v3.4 (where you can see whats
new in that release here[1]).

The biggest change is that it saves the BTF file in the trace.dat file
and parses it on the report (it requires libtraceevent v1.9):

 ~# trace-cmd record -p function_graph -O funcgraph-args -g do_sys_openat2
 [..]
 ~# trace-cmd report
       trace-cmd-50935 [002] ...1.  3490.518138: funcgraph_entry:                   |  do_sys_openat2(dfd=4294967196, filename=0x557bb9e3ee10, how=0xffff88815220fea8) {
       trace-cmd-50935 [002] ...1.  3490.518141: funcgraph_entry:                   |    getname_flags(filename=0x557bb9e3ee10, flags=0) {
       trace-cmd-50935 [002] ...1.  3490.518142: funcgraph_entry:                   |      getname_flags.part.0(filename=0x557bb9e3ee10, flags=0) {
       trace-cmd-50935 [002] ...1.  3490.518143: funcgraph_entry:                   |        kmem_cache_alloc_noprof(s=0xffff888106c7e000, gfpflags=0xcc0) {
       trace-cmd-50935 [002] ...1.  3490.518145: funcgraph_entry:                   |          stack_trace_save(store=0xffff88815220fac8, size=0x40, skipnr=0x0) {
       trace-cmd-50935 [002] ...1.  3490.518147: funcgraph_entry:                   |            arch_stack_walk(consume_entry=0xffffffff94d9dfe0, cookie=0xffff88815220fa58, task=0xffff88812d4c3580, regs=0x0) {
       trace-cmd-50935 [002] ...1.  3490.518148: funcgraph_entry:                   |              __unwind_start(state=0xffff88815220f988, task=0xffff88812d4c3580, regs=0x0, first_frame=0xffff88815220fa28) {
       trace-cmd-50935 [002] ...1.  3490.518149: funcgraph_entry:        1.518 us   |                get_stack_info(stack=0xffff88815220f938, task=0xffff88812d4c3580, info=0xffff88815220f988, visit_mask=0xffff88815220f9a8); (ret=0x0)
       trace-cmd-50935 [002] ...1.  3490.518152: funcgraph_entry:                   |                unwind_next_frame(state=0xffff88815220f988) {
       trace-cmd-50935 [002] ...1.  3490.518153: funcgraph_entry:        0.951 us   |                  __rcu_read_lock(); (ret=0xffff88812d4c3580)

-- Steve



[1] https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/tag/?h=trace-cmd-v3.4

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Donglin Peng]

On Fri, Feb 6, 2026 at 2:05 AM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Wed, 4 Feb 2026 22:52:11 +0800
> Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> > > > I have trace-cmd reading BTF now (just haven't officially released it) and
> > > > doing an extract and reading the trace.dat file is much faster than reading
> > > > the trace file with arguments. I'll need to implement the enum logic too in
> > > > libtraceevent.
> > >
> > > If you mean to do pretty printing of the trace in user space then +1 from me.
> > >
> > > I don't like sorting enums either in resolve_btfid, pahole or kernel.
> > > Sorted BTF by name was ok, since it doesn't change original semantics.
> > > While sorting enums by value gets us to the grey zone where
> > > the sequence of enum names in vmlinux.h becomes different than in dwarf.
> >
> > Thanks, I agreed.
>
> BTW, I just officially released trace-cmd v3.4 (where you can see whats
> new in that release here[1]).
>
> The biggest change is that it saves the BTF file in the trace.dat file
> and parses it on the report (it requires libtraceevent v1.9):
>
>  ~# trace-cmd record -p function_graph -O funcgraph-args -g do_sys_openat2
>  [..]
>  ~# trace-cmd report
>        trace-cmd-50935 [002] ...1.  3490.518138: funcgraph_entry:                   |  do_sys_openat2(dfd=4294967196, filename=0x557bb9e3ee10, how=0xffff88815220fea8) {
>        trace-cmd-50935 [002] ...1.  3490.518141: funcgraph_entry:                   |    getname_flags(filename=0x557bb9e3ee10, flags=0) {
>        trace-cmd-50935 [002] ...1.  3490.518142: funcgraph_entry:                   |      getname_flags.part.0(filename=0x557bb9e3ee10, flags=0) {
>        trace-cmd-50935 [002] ...1.  3490.518143: funcgraph_entry:                   |        kmem_cache_alloc_noprof(s=0xffff888106c7e000, gfpflags=0xcc0) {
>        trace-cmd-50935 [002] ...1.  3490.518145: funcgraph_entry:                   |          stack_trace_save(store=0xffff88815220fac8, size=0x40, skipnr=0x0) {
>        trace-cmd-50935 [002] ...1.  3490.518147: funcgraph_entry:                   |            arch_stack_walk(consume_entry=0xffffffff94d9dfe0, cookie=0xffff88815220fa58, task=0xffff88812d4c3580, regs=0x0) {
>        trace-cmd-50935 [002] ...1.  3490.518148: funcgraph_entry:                   |              __unwind_start(state=0xffff88815220f988, task=0xffff88812d4c3580, regs=0x0, first_frame=0xffff88815220fa28) {
>        trace-cmd-50935 [002] ...1.  3490.518149: funcgraph_entry:        1.518 us   |                get_stack_info(stack=0xffff88815220f938, task=0xffff88812d4c3580, info=0xffff88815220f988, visit_mask=0xffff88815220f9a8); (ret=0x0)
>        trace-cmd-50935 [002] ...1.  3490.518152: funcgraph_entry:                   |                unwind_next_frame(state=0xffff88815220f988) {
>        trace-cmd-50935 [002] ...1.  3490.518153: funcgraph_entry:        0.951 us   |                  __rcu_read_lock(); (ret=0xffff88812d4c3580)

Thanks, I will implement this feature in libtraceevent.

>
> -- Steve
>
>
>
> [1] https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/tag/?h=trace-cmd-v3.4

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Donglin Peng]

On Fri, Feb 6, 2026 at 2:05 AM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Wed, 4 Feb 2026 22:52:11 +0800
> Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> > > > I have trace-cmd reading BTF now (just haven't officially released it) and
> > > > doing an extract and reading the trace.dat file is much faster than reading
> > > > the trace file with arguments. I'll need to implement the enum logic too in
> > > > libtraceevent.
> > >
> > > If you mean to do pretty printing of the trace in user space then +1 from me.
> > >
> > > I don't like sorting enums either in resolve_btfid, pahole or kernel.
> > > Sorted BTF by name was ok, since it doesn't change original semantics.
> > > While sorting enums by value gets us to the grey zone where
> > > the sequence of enum names in vmlinux.h becomes different than in dwarf.
> >
> > Thanks, I agreed.
>
> BTW, I just officially released trace-cmd v3.4 (where you can see whats
> new in that release here[1]).
>
> The biggest change is that it saves the BTF file in the trace.dat file
> and parses it on the report (it requires libtraceevent v1.9):
>
>  ~# trace-cmd record -p function_graph -O funcgraph-args -g do_sys_openat2
>  [..]
>  ~# trace-cmd report
>        trace-cmd-50935 [002] ...1.  3490.518138: funcgraph_entry:                   |  do_sys_openat2(dfd=4294967196, filename=0x557bb9e3ee10, how=0xffff88815220fea8) {
>        trace-cmd-50935 [002] ...1.  3490.518141: funcgraph_entry:                   |    getname_flags(filename=0x557bb9e3ee10, flags=0) {
>        trace-cmd-50935 [002] ...1.  3490.518142: funcgraph_entry:                   |      getname_flags.part.0(filename=0x557bb9e3ee10, flags=0) {
>        trace-cmd-50935 [002] ...1.  3490.518143: funcgraph_entry:                   |        kmem_cache_alloc_noprof(s=0xffff888106c7e000, gfpflags=0xcc0) {
>        trace-cmd-50935 [002] ...1.  3490.518145: funcgraph_entry:                   |          stack_trace_save(store=0xffff88815220fac8, size=0x40, skipnr=0x0) {
>        trace-cmd-50935 [002] ...1.  3490.518147: funcgraph_entry:                   |            arch_stack_walk(consume_entry=0xffffffff94d9dfe0, cookie=0xffff88815220fa58, task=0xffff88812d4c3580, regs=0x0) {
>        trace-cmd-50935 [002] ...1.  3490.518148: funcgraph_entry:                   |              __unwind_start(state=0xffff88815220f988, task=0xffff88812d4c3580, regs=0x0, first_frame=0xffff88815220fa28) {
>        trace-cmd-50935 [002] ...1.  3490.518149: funcgraph_entry:        1.518 us   |                get_stack_info(stack=0xffff88815220f938, task=0xffff88812d4c3580, info=0xffff88815220f988, visit_mask=0xffff88815220f9a8); (ret=0x0)
>        trace-cmd-50935 [002] ...1.  3490.518152: funcgraph_entry:                   |                unwind_next_frame(state=0xffff88815220f988) {
>        trace-cmd-50935 [002] ...1.  3490.518153: funcgraph_entry:        0.951 us   |                  __rcu_read_lock(); (ret=0xffff88812d4c3580)

Hi Steve,

I noticed that trace-cmd 3.4 introduced a new feature:

- Add support for showing parent functions in function graph

  If the "funcgraph-retaddr" option is set during function graph tracing,
  the parent functions will now be displayed:

  preempt_count_add(val=65536); /* <-irq_enter_rcu */ (ret=0x10001)

However, it currently only prints the caller's entry point. I suggest
enhancing this to include the **offset within the caller** as well. This
would allow tools like `faddr2line` to locate the exact call site accurately.

I implemented a tool named FuncGraph[1] to convert function_graph
ftrace output into an interactive HTML file. It leverages fastfaddr2line
to map return addresses (e.g., __sys_bpf+0x51/0x500) to their
corresponding call sites, such as: __sys_bpfat kernel/bpf/syscall.c:6115.

What do you think?

Thanks,
Donglin

[1] https://github.com/pengdonglin137/FuncGraph

>
> -- Steve
>
>
>
> [1] https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/tag/?h=trace-cmd-v3.4

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Steven Rostedt]

On Sun, 8 Feb 2026 21:07:37 +0800
Donglin Peng <dolinux.peng@gmail.com> wrote:
> Hi Steve,
> 
> I noticed that trace-cmd 3.4 introduced a new feature:
> 
> - Add support for showing parent functions in function graph
> 
>   If the "funcgraph-retaddr" option is set during function graph tracing,
>   the parent functions will now be displayed:
> 
>   preempt_count_add(val=65536); /* <-irq_enter_rcu */ (ret=0x10001)
> 
> However, it currently only prints the caller's entry point. I suggest
> enhancing this to include the **offset within the caller** as well. This
> would allow tools like `faddr2line` to locate the exact call site accurately.

Yeah, it makes sense to always show the offset of where it was called.
We can add that for 3.4.1.

> 
> I implemented a tool named FuncGraph[1] to convert function_graph
> ftrace output into an interactive HTML file. It leverages fastfaddr2line
> to map return addresses (e.g., __sys_bpf+0x51/0x500) to their
> corresponding call sites, such as: __sys_bpfat kernel/bpf/syscall.c:6115.
> 
> What do you think?

Sounds cool. Is this the vibe coding thing you talked about on Linked-In?

-- Steve

> 
> Thanks,
> Donglin
> 
> [1] https://github.com/pengdonglin137/FuncGraph
> 
> >
> > -- Steve
> >
> >
> >
> > [1] https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/tag/?h=trace-cmd-v3.4  

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Donglin Peng]

On Sun, Feb 8, 2026 at 11:27 PM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Sun, 8 Feb 2026 21:07:37 +0800
> Donglin Peng <dolinux.peng@gmail.com> wrote:
> > Hi Steve,
> >
> > I noticed that trace-cmd 3.4 introduced a new feature:
> >
> > - Add support for showing parent functions in function graph
> >
> >   If the "funcgraph-retaddr" option is set during function graph tracing,
> >   the parent functions will now be displayed:
> >
> >   preempt_count_add(val=65536); /* <-irq_enter_rcu */ (ret=0x10001)
> >
> > However, it currently only prints the caller's entry point. I suggest
> > enhancing this to include the **offset within the caller** as well. This
> > would allow tools like `faddr2line` to locate the exact call site accurately.
>
> Yeah, it makes sense to always show the offset of where it was called.
> We can add that for 3.4.1.

Thanks, I can send a patch to implement it.

>
> >
> > I implemented a tool named FuncGraph[1] to convert function_graph
> > ftrace output into an interactive HTML file. It leverages fastfaddr2line
> > to map return addresses (e.g., __sys_bpf+0x51/0x500) to their
> > corresponding call sites, such as: __sys_bpfat kernel/bpf/syscall.c:6115.
> >
> > What do you think?
>
> Sounds cool. Is this the vibe coding thing you talked about on Linked-In?

Yes. The project repository includes a sample HTML file (`sample.html`)
demonstrating the functionality — feel free to try it out.

I hope future versions of `trace-cmd` or `Kernelshark` could incorporate
similar features to enhance trace analysis workflows.

>
> -- Steve
>
> >
> > Thanks,
> > Donglin
> >
> > [1] https://github.com/pengdonglin137/FuncGraph
> >
> > >
> > > -- Steve
> > >
> > >
> > >
> > > [1] https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/tag/?h=trace-cmd-v3.4
>

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Steven Rostedt]

On Sun, 8 Feb 2026 23:42:50 +0800
Donglin Peng <dolinux.peng@gmail.com> wrote:

> I hope future versions of `trace-cmd` or `Kernelshark` could incorporate
> similar features to enhance trace analysis workflows.

Hey, they are both open source projects. Patches welcomed ;-)

Note, trace-cmd is mostly a side project for me. I mostly work on it in
my spare time (or in the plane while I travel). Which explains why it
doesn't get the loving it deserves.

KernelShark is now maintained by Yordan Karadzhov, which I believe he's
in the same predicament as I am. He too is open for patches.

-- Steve

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Donglin Peng]

On Tue, Feb 3, 2026 at 11:16 PM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Tue, 3 Feb 2026 21:50:47 +0800
> Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> > Testing revealed that sorting within resolve_btfids introduces issues with
> > btf__dedup. Therefore, I plan to move the sorting logic directly into
> > btf__add_enum_value and btf__add_enum64_value in libbpf, which are
> > invoked by pahole. However, it means that we need a newer pahole
> > version.
>
> Sorting isn't a requirement just something I wanted to bring up. If it's
> too complex and doesn't achieve much benefit then let's not do it.

Thanks for clarifying. Analysis shows most enum-type btf_type have
small vlen (78% ≤ 10, 95% ≤50), so I think that  linear search could
be acceptable.

>
> My worry is because "cat trace" takes quite a long time just reading the
> BTF arguments. I'm worried it will just get worse with enums as well.

The delay stems from `btf_find_by_name_kind`’s linear search over
vmlinux BTF (10k+ types). This was resolved by adding binary search
to `btf_find_by_name_kind` [1].

Performance tests [2] confirm the improvement:
1. Original funcgraph-retval:
# time cat trace | wc -l
101024

real    0m0.682s
user    0m0.000s
sys     0m0.695s

2. Enhanced funcgraph-retval:
# time cat trace | wc -l
99326

real    0m12.886s
user    0m0.010s
sys     0m12.680s

3. Enhanced funcgraph-retval + optimizined btf_find_by_name_kind:
# time cat trace | wc -l
102922

real    0m0.794s
user    0m0.000s
sys     0m0.810s

Binary search reduces overhead to near-negligible levels (0.794s vs. 12.886s).

[1] https://lore.kernel.org/bpf/20260109130003.3313716-1-dolinux.peng@gmail.com/
[2] https://lore.kernel.org/lkml/20251209121349.525641-1-dolinux.peng@gmail.com/

>
> I have trace-cmd reading BTF now (just haven't officially released it) and
> doing an extract and reading the trace.dat file is much faster than reading
> the trace file with arguments. I'll need to implement the enum logic too in
> libtraceevent.
>
> -- Steve

Re: [PATCH 2/2] tracing: resolve enum names for function arguments via BTF

[Masami Hiramatsu]

On Mon,  2 Feb 2026 19:15:48 +0800
Donglin Peng <dolinux.peng@gmail.com> wrote:

> From: Donglin Peng <pengdonglin@xiaomi.com>
> 
> Use BTF to print symbolic names for enum-type function arguments,
> improving trace readability.
> 
> Before:
> count_memcg_events(memcg=0xffff..., idx=20, count=0x1) {
> 
> After:
> count_memcg_events(memcg=0xffff..., idx=20 [PGFAULT], count=0x1) {

Hmm, it is interesting idea. Maybe I can also introduce the same
feature to fetcharg for fprobe/kprobes as a new type.
Since the enum type should be solved in parsing the event definition,
for fprobe/kprobes the performance will not be a performance problem.

Thanks,

> 
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Masami Hiramatsu <mhiramat@kernel.org>
> Cc: linux-trace-kernel@vger.kernel.org
> Signed-off-by: Donglin Peng <pengdonglin@xiaomi.com>
> ---
>  kernel/trace/trace_output.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/trace/trace_output.c b/kernel/trace/trace_output.c
> index cc2d3306bb60..c395f768c3b8 100644
> --- a/kernel/trace/trace_output.c
> +++ b/kernel/trace/trace_output.c
> @@ -695,12 +695,13 @@ void print_function_args(struct trace_seq *s, unsigned long *args,
>  {
>  	const struct btf_param *param;
>  	const struct btf_type *t;
> +	const struct btf_enum *enump;
>  	const char *param_name;
>  	char name[KSYM_NAME_LEN];
>  	unsigned long arg;
>  	struct btf *btf;
>  	s32 tid, nr = 0;
> -	int a, p, x;
> +	int a, p, x, i;
>  	u16 encode;
>  
>  	trace_seq_printf(s, "(");
> @@ -754,6 +755,14 @@ void print_function_args(struct trace_seq *s, unsigned long *args,
>  			break;
>  		case BTF_KIND_ENUM:
>  			trace_seq_printf(s, "%ld", arg);
> +			for_each_enum(i, t, enump) {
> +				if (arg == enump->val) {
> +					trace_seq_printf(s, " [%s]",
> +							 btf_name_by_offset(btf,
> +							 enump->name_off));
> +					break;
> +				}
> +			}
>  			break;
>  		default:
>  			/* This does not handle complex arguments */
> -- 
> 2.34.1
> 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>