[PATCH bpf-next v7] selftests/bpf: add BPF program dump in veristat

[PATCH bpf-next v7] selftests/bpf: add BPF program dump in veristat

[Mykyta Yatsenko]

From: Mykyta Yatsenko <yatsenko@meta.com>

Add the ability to dump BPF program instructions directly from veristat.
Previously, inspecting a program required separate bpftool invocations:
one to load and another to dump it, which meant running multiple
commands.
During active development, it's common for developers to use veristat
for testing verification. Integrating instruction dumping into veristat
reduces the need to switch tools and simplifies the workflow.
By making this information more readily accessible, this change aims
to streamline the BPF development cycle and improve usability for
developers.
This implementation leverages bpftool, by running it directly via popen
to avoid any code duplication and keep veristat simple.

Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
---
 tools/testing/selftests/bpf/veristat.c | 55 +++++++++++++++++++++++++-
 1 file changed, 54 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/bpf/veristat.c b/tools/testing/selftests/bpf/veristat.c
index d532dd82a3a8..85ae7f6fee90 100644
--- a/tools/testing/selftests/bpf/veristat.c
+++ b/tools/testing/selftests/bpf/veristat.c
@@ -181,6 +181,12 @@ struct var_preset {
 	bool applied;
 };
 
+enum dump_mode {
+	DUMP_NONE = 0,
+	DUMP_XLATED = 1,
+	DUMP_JITED = 2,
+};
+
 static struct env {
 	char **filenames;
 	int filename_cnt;
@@ -227,6 +233,7 @@ static struct env {
 	char orig_cgroup[PATH_MAX];
 	char stat_cgroup[PATH_MAX];
 	int memory_peak_fd;
+	__u32 dump_mode;
 } env;
 
 static int libbpf_print_fn(enum libbpf_print_level level, const char *format, va_list args)
@@ -271,6 +278,7 @@ const char argp_program_doc[] =
 enum {
 	OPT_LOG_FIXED = 1000,
 	OPT_LOG_SIZE = 1001,
+	OPT_DUMP = 1002,
 };
 
 static const struct argp_option opts[] = {
@@ -295,6 +303,7 @@ static const struct argp_option opts[] = {
 	  "Force BPF verifier failure on register invariant violation (BPF_F_TEST_REG_INVARIANTS program flag)" },
 	{ "top-src-lines", 'S', "N", 0, "Emit N most frequent source code lines" },
 	{ "set-global-vars", 'G', "GLOBAL", 0, "Set global variables provided in the expression, for example \"var1 = 1\"" },
+	{ "dump", OPT_DUMP, "DUMP_MODE", OPTION_ARG_OPTIONAL, "Print BPF program dump (xlated, jited)" },
 	{},
 };
 
@@ -427,6 +436,16 @@ static error_t parse_arg(int key, char *arg, struct argp_state *state)
 			return err;
 		}
 		break;
+	case OPT_DUMP:
+		if (!arg || strcasecmp(arg, "xlated") == 0) {
+			env.dump_mode |= DUMP_XLATED;
+		} else if (strcasecmp(arg, "jited") == 0) {
+			env.dump_mode |= DUMP_JITED;
+		} else {
+			fprintf(stderr, "Unrecognized dump mode '%s'\n", arg);
+			return -EINVAL;
+		}
+		break;
 	default:
 		return ARGP_ERR_UNKNOWN;
 	}
@@ -1554,6 +1573,35 @@ static int parse_rvalue(const char *val, struct rvalue *rvalue)
 	return 0;
 }
 
+static void dump(__u32 prog_id, enum dump_mode mode, const char *file_name, const char *prog_name)
+{
+	char command[64], buf[4096];
+	FILE *fp;
+	int status;
+
+	status = system("which bpftool > /dev/null 2>&1");
+	if (status != 0) {
+		fprintf(stderr, "bpftool is not available, can't print program dump\n");
+		return;
+	}
+	snprintf(command, sizeof(command), "bpftool prog dump %s id %u",
+		 mode == DUMP_JITED ? "jited" : "xlated", prog_id);
+	fp = popen(command, "r");
+	if (!fp) {
+		fprintf(stderr, "bpftool failed with error: %d\n", errno);
+		return;
+	}
+
+	printf("%s/%s DUMP %s:\n", file_name, prog_name, mode == DUMP_JITED ? "JITED" : "XLATED");
+	while (fgets(buf, sizeof(buf), fp))
+		fputs(buf, stdout);
+
+	if (ferror(fp))
+		fprintf(stderr, "Failed to dump BPF prog with error: %d\n", errno);
+
+	pclose(fp);
+}
+
 static int process_prog(const char *filename, struct bpf_object *obj, struct bpf_program *prog)
 {
 	const char *base_filename = basename(strdupa(filename));
@@ -1630,8 +1678,13 @@ static int process_prog(const char *filename, struct bpf_object *obj, struct bpf
 
 	memset(&info, 0, info_len);
 	fd = bpf_program__fd(prog);
-	if (fd > 0 && bpf_prog_get_info_by_fd(fd, &info, &info_len) == 0)
+	if (fd > 0 && bpf_prog_get_info_by_fd(fd, &info, &info_len) == 0) {
 		stats->stats[JITED_SIZE] = info.jited_prog_len;
+		if (env.dump_mode & DUMP_JITED)
+			dump(info.id, DUMP_JITED, base_filename, prog_name);
+		if (env.dump_mode & DUMP_XLATED)
+			dump(info.id, DUMP_XLATED, base_filename, prog_name);
+	}
 
 	parse_verif_log(buf, buf_sz, stats);
 
-- 
2.51.0

Re: [PATCH bpf-next v7] selftests/bpf: add BPF program dump in veristat

[Eduard Zingerman]

On Fri, 2025-09-05 at 15:08 +0100, Mykyta Yatsenko wrote:
> From: Mykyta Yatsenko <yatsenko@meta.com>
> 
> Add the ability to dump BPF program instructions directly from veristat.
> Previously, inspecting a program required separate bpftool invocations:
> one to load and another to dump it, which meant running multiple
> commands.
> During active development, it's common for developers to use veristat
> for testing verification. Integrating instruction dumping into veristat
> reduces the need to switch tools and simplifies the workflow.
> By making this information more readily accessible, this change aims
> to streamline the BPF development cycle and improve usability for
> developers.
> This implementation leverages bpftool, by running it directly via popen
> to avoid any code duplication and keep veristat simple.
> 
> Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
> ---

Lgtm with a small nit.

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

> @@ -1554,6 +1573,35 @@ static int parse_rvalue(const char *val, struct rvalue *rvalue)
>  	return 0;
>  }
>  
> +static void dump(__u32 prog_id, enum dump_mode mode, const char *file_name, const char *prog_name)
> +{
> +	char command[64], buf[4096];
> +	FILE *fp;
> +	int status;
> +
> +	status = system("which bpftool > /dev/null 2>&1");

Fun fact: if you do a minimal Fedora install (dnf group install core)
          "which" is not installed by default o.O
          (not suggesting any changes).

> +	if (status != 0) {
> +		fprintf(stderr, "bpftool is not available, can't print program dump\n");
> +		return;
> +	}

[...]

> @@ -1630,8 +1678,13 @@ static int process_prog(const char *filename, struct bpf_object *obj, struct bpf
>  
>  	memset(&info, 0, info_len);
>  	fd = bpf_program__fd(prog);
> -	if (fd > 0 && bpf_prog_get_info_by_fd(fd, &info, &info_len) == 0)
> +	if (fd > 0 && bpf_prog_get_info_by_fd(fd, &info, &info_len) == 0) {
>  		stats->stats[JITED_SIZE] = info.jited_prog_len;
> +		if (env.dump_mode & DUMP_JITED)
> +			dump(info.id, DUMP_JITED, base_filename, prog_name);
> +		if (env.dump_mode & DUMP_XLATED)
> +			dump(info.id, DUMP_XLATED, base_filename, prog_name);

Nit: if you do `./veristat --dump=jited iters.bpf.o` there would be an empty line
     after dump for each program, but not for --dump=xlated.

> +	}
>  
>  	parse_verif_log(buf, buf_sz, stats);
>  

Re: [PATCH bpf-next v7] selftests/bpf: add BPF program dump in veristat

[Andrii Nakryiko]

On Fri, Sep 5, 2025 at 12:00 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Fri, 2025-09-05 at 15:08 +0100, Mykyta Yatsenko wrote:
> > From: Mykyta Yatsenko <yatsenko@meta.com>
> >
> > Add the ability to dump BPF program instructions directly from veristat.
> > Previously, inspecting a program required separate bpftool invocations:
> > one to load and another to dump it, which meant running multiple
> > commands.
> > During active development, it's common for developers to use veristat
> > for testing verification. Integrating instruction dumping into veristat
> > reduces the need to switch tools and simplifies the workflow.
> > By making this information more readily accessible, this change aims
> > to streamline the BPF development cycle and improve usability for
> > developers.
> > This implementation leverages bpftool, by running it directly via popen
> > to avoid any code duplication and keep veristat simple.
> >
> > Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
> > ---
>
> Lgtm with a small nit.
>
> Acked-by: Eduard Zingerman <eddyz87@gmail.com>
>
> > @@ -1554,6 +1573,35 @@ static int parse_rvalue(const char *val, struct rvalue *rvalue)
> >       return 0;
> >  }
> >
> > +static void dump(__u32 prog_id, enum dump_mode mode, const char *file_name, const char *prog_name)
> > +{
> > +     char command[64], buf[4096];
> > +     FILE *fp;
> > +     int status;
> > +
> > +     status = system("which bpftool > /dev/null 2>&1");
>
> Fun fact: if you do a minimal Fedora install (dnf group install core)
>           "which" is not installed by default o.O
>           (not suggesting any changes).

I switched to `command -v bpftool` for now, is there any gotcha with
that one as well?

>
> > +     if (status != 0) {
> > +             fprintf(stderr, "bpftool is not available, can't print program dump\n");
> > +             return;
> > +     }
>
> [...]
>
> > @@ -1630,8 +1678,13 @@ static int process_prog(const char *filename, struct bpf_object *obj, struct bpf
> >
> >       memset(&info, 0, info_len);
> >       fd = bpf_program__fd(prog);
> > -     if (fd > 0 && bpf_prog_get_info_by_fd(fd, &info, &info_len) == 0)
> > +     if (fd > 0 && bpf_prog_get_info_by_fd(fd, &info, &info_len) == 0) {
> >               stats->stats[JITED_SIZE] = info.jited_prog_len;
> > +             if (env.dump_mode & DUMP_JITED)
> > +                     dump(info.id, DUMP_JITED, base_filename, prog_name);
> > +             if (env.dump_mode & DUMP_XLATED)
> > +                     dump(info.id, DUMP_XLATED, base_filename, prog_name);
>
> Nit: if you do `./veristat --dump=jited iters.bpf.o` there would be an empty line
>      after dump for each program, but not for --dump=xlated.
>

Yeah, bpftool's output isn't consistent. I just added an extra empty
line, that makes dump a bit more clean (and I didn't mind two empty
lines, whatever).

I was also finding it hard to notice where the dump for a given
program starts, so I reformatted header a bit. Overall, applied the
following changes and pushed to bpf-next, thanks for a useful feature!

diff --git a/tools/testing/selftests/bpf/veristat.c
b/tools/testing/selftests/bpf/veristat.c
index 85ae7f6fee90..e962f133250c 100644
--- a/tools/testing/selftests/bpf/veristat.c
+++ b/tools/testing/selftests/bpf/veristat.c
@@ -1579,7 +1579,7 @@ static void dump(__u32 prog_id, enum dump_mode
mode, const char *file_name, cons
        FILE *fp;
        int status;

-       status = system("which bpftool > /dev/null 2>&1");
+       status = system("command -v bpftool > /dev/null 2>&1");
        if (status != 0) {
                fprintf(stderr, "bpftool is not available, can't print
program dump\n");
                return;
@@ -1592,9 +1592,10 @@ static void dump(__u32 prog_id, enum dump_mode
mode, const char *file_name, cons
                return;
        }

-       printf("%s/%s DUMP %s:\n", file_name, prog_name, mode ==
DUMP_JITED ? "JITED" : "XLATED");
+       printf("DUMP (%s) %s/%s:\n", mode == DUMP_JITED ? "JITED" :
"XLATED", file_name, prog_name);
        while (fgets(buf, sizeof(buf), fp))
                fputs(buf, stdout);
+       fprintf(stdout, "\n");

        if (ferror(fp))
                fprintf(stderr, "Failed to dump BPF prog with error:
%d\n", errno);


> > +     }
> >
> >       parse_verif_log(buf, buf_sz, stats);
> >

Re: [PATCH bpf-next v7] selftests/bpf: add BPF program dump in veristat

[Eduard Zingerman]

On Fri, 2025-09-05 at 12:14 -0700, Andrii Nakryiko wrote:
> On Fri, Sep 5, 2025 at 12:00 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > 
> > On Fri, 2025-09-05 at 15:08 +0100, Mykyta Yatsenko wrote:
> > > From: Mykyta Yatsenko <yatsenko@meta.com>
> > > 
> > > Add the ability to dump BPF program instructions directly from veristat.
> > > Previously, inspecting a program required separate bpftool invocations:
> > > one to load and another to dump it, which meant running multiple
> > > commands.
> > > During active development, it's common for developers to use veristat
> > > for testing verification. Integrating instruction dumping into veristat
> > > reduces the need to switch tools and simplifies the workflow.
> > > By making this information more readily accessible, this change aims
> > > to streamline the BPF development cycle and improve usability for
> > > developers.
> > > This implementation leverages bpftool, by running it directly via popen
> > > to avoid any code duplication and keep veristat simple.
> > > 
> > > Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
> > > ---
> > 
> > Lgtm with a small nit.
> > 
> > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> > 
> > > @@ -1554,6 +1573,35 @@ static int parse_rvalue(const char *val, struct rvalue *rvalue)
> > >       return 0;
> > >  }
> > > 
> > > +static void dump(__u32 prog_id, enum dump_mode mode, const char *file_name, const char *prog_name)
> > > +{
> > > +     char command[64], buf[4096];
> > > +     FILE *fp;
> > > +     int status;
> > > +
> > > +     status = system("which bpftool > /dev/null 2>&1");
> > 
> > Fun fact: if you do a minimal Fedora install (dnf group install core)
> >           "which" is not installed by default o.O
> >           (not suggesting any changes).
> 
> I switched to `command -v bpftool` for now, is there any gotcha with
> that one as well?

Should be fine, I guess:

  $ rpm -qf /usr/sbin/command
  bash-5.2.37-1.fc42.x86_64

> > 
> > > +     if (status != 0) {
> > > +             fprintf(stderr, "bpftool is not available, can't print program dump\n");
> > > +             return;
> > > +     }
> > 
> > [...]
> > 
> > > @@ -1630,8 +1678,13 @@ static int process_prog(const char *filename, struct bpf_object *obj, struct bpf
> > > 
> > >       memset(&info, 0, info_len);
> > >       fd = bpf_program__fd(prog);
> > > -     if (fd > 0 && bpf_prog_get_info_by_fd(fd, &info, &info_len) == 0)
> > > +     if (fd > 0 && bpf_prog_get_info_by_fd(fd, &info, &info_len) == 0) {
> > >               stats->stats[JITED_SIZE] = info.jited_prog_len;
> > > +             if (env.dump_mode & DUMP_JITED)
> > > +                     dump(info.id, DUMP_JITED, base_filename, prog_name);
> > > +             if (env.dump_mode & DUMP_XLATED)
> > > +                     dump(info.id, DUMP_XLATED, base_filename, prog_name);
> > 
> > Nit: if you do `./veristat --dump=jited iters.bpf.o` there would be an empty line
> >      after dump for each program, but not for --dump=xlated.
> > 
> 
> Yeah, bpftool's output isn't consistent. I just added an extra empty
> line, that makes dump a bit more clean (and I didn't mind two empty
> lines, whatever).

+1

> 
> I was also finding it hard to notice where the dump for a given
> program starts, so I reformatted header a bit. Overall, applied the
> following changes and pushed to bpf-next, thanks for a useful feature!

Yeap, nice little feature.
I was doing bogus __xlated("foo") before in tests,
just to see how assembly looks like.

Re: [PATCH bpf-next v7] selftests/bpf: add BPF program dump in veristat

[Andrii Nakryiko]

On Fri, Sep 5, 2025 at 12:19 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Fri, 2025-09-05 at 12:14 -0700, Andrii Nakryiko wrote:
> > On Fri, Sep 5, 2025 at 12:00 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > >
> > > On Fri, 2025-09-05 at 15:08 +0100, Mykyta Yatsenko wrote:
> > > > From: Mykyta Yatsenko <yatsenko@meta.com>
> > > >
> > > > Add the ability to dump BPF program instructions directly from veristat.
> > > > Previously, inspecting a program required separate bpftool invocations:
> > > > one to load and another to dump it, which meant running multiple
> > > > commands.
> > > > During active development, it's common for developers to use veristat
> > > > for testing verification. Integrating instruction dumping into veristat
> > > > reduces the need to switch tools and simplifies the workflow.
> > > > By making this information more readily accessible, this change aims
> > > > to streamline the BPF development cycle and improve usability for
> > > > developers.
> > > > This implementation leverages bpftool, by running it directly via popen
> > > > to avoid any code duplication and keep veristat simple.
> > > >
> > > > Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
> > > > ---
> > >
> > > Lgtm with a small nit.
> > >
> > > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> > >
> > > > @@ -1554,6 +1573,35 @@ static int parse_rvalue(const char *val, struct rvalue *rvalue)
> > > >       return 0;
> > > >  }
> > > >
> > > > +static void dump(__u32 prog_id, enum dump_mode mode, const char *file_name, const char *prog_name)
> > > > +{
> > > > +     char command[64], buf[4096];
> > > > +     FILE *fp;
> > > > +     int status;
> > > > +
> > > > +     status = system("which bpftool > /dev/null 2>&1");
> > >
> > > Fun fact: if you do a minimal Fedora install (dnf group install core)
> > >           "which" is not installed by default o.O
> > >           (not suggesting any changes).
> >
> > I switched to `command -v bpftool` for now, is there any gotcha with
> > that one as well?
>
> Should be fine, I guess:
>
>   $ rpm -qf /usr/sbin/command
>   bash-5.2.37-1.fc42.x86_64

command is actually a shell built-in ([0]). At least for Bourne shells, I think.

  [0] https://pubs.opengroup.org/onlinepubs/009695399/utilities/command.html

>
> > >
> > > > +     if (status != 0) {
> > > > +             fprintf(stderr, "bpftool is not available, can't print program dump\n");
> > > > +             return;
> > > > +     }
> > >
> > > [...]
> > >
> > > > @@ -1630,8 +1678,13 @@ static int process_prog(const char *filename, struct bpf_object *obj, struct bpf
> > > >
> > > >       memset(&info, 0, info_len);
> > > >       fd = bpf_program__fd(prog);
> > > > -     if (fd > 0 && bpf_prog_get_info_by_fd(fd, &info, &info_len) == 0)
> > > > +     if (fd > 0 && bpf_prog_get_info_by_fd(fd, &info, &info_len) == 0) {
> > > >               stats->stats[JITED_SIZE] = info.jited_prog_len;
> > > > +             if (env.dump_mode & DUMP_JITED)
> > > > +                     dump(info.id, DUMP_JITED, base_filename, prog_name);
> > > > +             if (env.dump_mode & DUMP_XLATED)
> > > > +                     dump(info.id, DUMP_XLATED, base_filename, prog_name);
> > >
> > > Nit: if you do `./veristat --dump=jited iters.bpf.o` there would be an empty line
> > >      after dump for each program, but not for --dump=xlated.
> > >
> >
> > Yeah, bpftool's output isn't consistent. I just added an extra empty
> > line, that makes dump a bit more clean (and I didn't mind two empty
> > lines, whatever).
>
> +1
>
> >
> > I was also finding it hard to notice where the dump for a given
> > program starts, so I reformatted header a bit. Overall, applied the
> > following changes and pushed to bpf-next, thanks for a useful feature!
>
> Yeap, nice little feature.
> I was doing bogus __xlated("foo") before in tests,
> just to see how assembly looks like.

Re: [PATCH bpf-next v7] selftests/bpf: add BPF program dump in veristat

[Eduard Zingerman]

On Fri, 2025-09-05 at 14:38 -0700, Andrii Nakryiko wrote:

[...]

> > > > Fun fact: if you do a minimal Fedora install (dnf group install core)
> > > >           "which" is not installed by default o.O
> > > >           (not suggesting any changes).
> > > 
> > > I switched to `command -v bpftool` for now, is there any gotcha with
> > > that one as well?
> > 
> > Should be fine, I guess:
> > 
> >   $ rpm -qf /usr/sbin/command
> >   bash-5.2.37-1.fc42.x86_64
> 
> command is actually a shell built-in ([0]). At least for Bourne shells, I think.
> 
>   [0] https://pubs.opengroup.org/onlinepubs/009695399/utilities/command.html
>

Yes, but looks like it's a separate binary, not a command:

  $ strace command -v ls 2>&1 | grep command
  execve("/usr/bin/command", ["command", "-v", "ls"], 0x7ffffeaef7b0 /* 65 vars */) = 0

(Not that it changes much).

[...]

Re: [PATCH bpf-next v7] selftests/bpf: add BPF program dump in veristat

[Andrii Nakryiko]

On Fri, Sep 5, 2025 at 2:43 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Fri, 2025-09-05 at 14:38 -0700, Andrii Nakryiko wrote:
>
> [...]
>
> > > > > Fun fact: if you do a minimal Fedora install (dnf group install core)
> > > > >           "which" is not installed by default o.O
> > > > >           (not suggesting any changes).
> > > >
> > > > I switched to `command -v bpftool` for now, is there any gotcha with
> > > > that one as well?
> > >
> > > Should be fine, I guess:
> > >
> > >   $ rpm -qf /usr/sbin/command
> > >   bash-5.2.37-1.fc42.x86_64
> >
> > command is actually a shell built-in ([0]). At least for Bourne shells, I think.
> >
> >   [0] https://pubs.opengroup.org/onlinepubs/009695399/utilities/command.html
> >
>
> Yes, but looks like it's a separate binary, not a command:
>
>   $ strace command -v ls 2>&1 | grep command
>   execve("/usr/bin/command", ["command", "-v", "ls"], 0x7ffffeaef7b0 /* 65 vars */) = 0
>
> (Not that it changes much).

You nerd sniped me here :) You get that execve("/usr/bin/command")
because strace forces the command to be resolved as binary. If you run
something like execsnoop in background and execute `command -v blah`
you won't see this execve. ¯\_(ツ)_/¯

>
> [...]

Re: [PATCH bpf-next v7] selftests/bpf: add BPF program dump in veristat

[Eduard Zingerman]

On Fri, 2025-09-05 at 14:51 -0700, Andrii Nakryiko wrote:

[...]

> > Yes, but looks like it's a separate binary, not a command:
> > 
> >   $ strace command -v ls 2>&1 | grep command
> >   execve("/usr/bin/command", ["command", "-v", "ls"], 0x7ffffeaef7b0 /* 65 vars */) = 0
> > 
> > (Not that it changes much).
> 
> You nerd sniped me here :) You get that execve("/usr/bin/command")
> because strace forces the command to be resolved as binary. If you run
> something like execsnoop in background and execute `command -v blah`
> you won't see this execve. ¯\_(ツ)_/¯

Interesting, so I should have done this:

  $ strace bash -c 'command -v ls' 2>&1 | grep command

So, it's like 'time' vs '/usr/bin/time' :)