[RFC bpf-next 0/2] libbpf Userspace Runtime-Defined Tracing (URDT)

[RFC bpf-next 0/2] libbpf Userspace Runtime-Defined Tracing (URDT)

[Alan Maguire]

Adding userspace tracepoints in other languages like python and
go is a very useful for observability.  libstapsdt [1]
and language bindings like python-stapsdt [2] that rely on it
use a clever scheme of emulating static (USDT) userspace tracepoints
at runtime.  This involves (as I understand it):

- fabricating a shared library
- annotating it with ELF notes that describe its tracepoints
- dlopen()ing it and calling the appropriate probe fire function
  to trigger probe firing.

bcc already supports this mechanism (the examples in [2] use
bcc to list/trigger the tracepoints), so it seems like it
would be a good candidate for adding support to libbpf.

However, before doing that, it's worth considering if there
are simpler ways to support runtime probe firing.  This
small series demonstrates a simple method based on USDT
probes added to libbpf itself.

The suggested solution comprises 3 parts

1. functions to fire dynamic probes are added to libbpf itself
   bpf_urdt__probeN(), where N is the number of probe arguemnts.
   A sample usage would be
	bpf_urdt__probe3("myprovider", "myprobe", 1, 2, 3);

   Under the hood these correspond to USDT probes with an
   additional argument for uniquely identifying the probe
   (a hash of provider/probe name).

2. we attach to the appropriate USDT probe for the specified
   number of arguments urdt/probe0 for none, urdt/probe1 for
   1, etc.  We utilize the high-order 32 bits of the attach
   cookie to store the hash of the provider/probe name.

3. when urdt/probeN fires, the BPF_URDT() macro (which
   is similar to BPF_USDT()) checks if the hash passed
   in (identifying provider/probe) matches the attach
   cookie high-order 32 bits; if not it must be a firing
   for a different dynamic probe and we exit early.

Auto-attach support is also added, for example the following
would add a dynamic probe for provider:myprobe:

SEC("udrt/libbpf.so:2:myprovider:myprobe")
int BPF_URDT(myprobe, int arg1, char *arg2)
{
 ...
}

(Note the "2" above specifies the number of arguments to
the probe, otherwise it is identical to USDT).

The above program can then be triggered by a call to

 BPF_URDT_PROBE2("myprovider", "myprobe", 1, "hi");

The useful thing about this is that by attaching to
libbpf.so (and firing probes using that library) we
can get system-wide dynamic probe firing.  It is also
easy to fire a dynamic probe - no setup is required.

More examples of auto and manual attach can be found in
the selftests (patch 2).

If this approach appears to be worth pursing, we could
also look at adding support to libstapsdt for it.

Alan Maguire (2):
  libbpf: add support for Userspace Runtime Dynamic Tracing (URDT)
  selftests/bpf: add tests for Userspace Runtime Defined Tracepoints
    (URDT)

 tools/lib/bpf/Build                           |   2 +-
 tools/lib/bpf/Makefile                        |   2 +-
 tools/lib/bpf/libbpf.c                        |  94 ++++++++++
 tools/lib/bpf/libbpf.h                        |  94 ++++++++++
 tools/lib/bpf/libbpf.map                      |  13 ++
 tools/lib/bpf/libbpf_internal.h               |   2 +
 tools/lib/bpf/urdt.bpf.h                      | 103 +++++++++++
 tools/lib/bpf/urdt.c                          | 145 +++++++++++++++
 tools/testing/selftests/bpf/Makefile          |   2 +-
 tools/testing/selftests/bpf/prog_tests/urdt.c | 173 ++++++++++++++++++
 tools/testing/selftests/bpf/progs/test_urdt.c | 100 ++++++++++
 .../selftests/bpf/progs/test_urdt_shared.c    |  59 ++++++
 12 files changed, 786 insertions(+), 3 deletions(-)
 create mode 100644 tools/lib/bpf/urdt.bpf.h
 create mode 100644 tools/lib/bpf/urdt.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/urdt.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_urdt.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_urdt_shared.c

-- 
2.39.3

[RFC bpf-next 1/2] libbpf: add support for Userspace Runtime Dynamic Tracing (URDT)

[Alan Maguire]

Userspace Static Defined Tracing (USDT) provides a means to add
tracepoints to userspace code; these are marked by a no-op and
described in a .notes.stapsdt section, and libbpf provides
mechanisms to support tracing USDT via uprobes and the
USDT manager.

However, in order to be able to add tracepoints to userspace
code in other languages like python or go, a library call
(that can then be wrapped in the runtime-specific glue code)
is required.  One example of how such runtime probes
are supported is libstapsdt [1]; it creates a shared
library and annotations on the fly in order to emulate USDT.
BCC supports this mechanism, and one option would be to
add similar support to libbpf.  However, it would involve
retrieving the ELF notes from the dynamically-created
library, which at first glance looked tricky.

The approach here is simpler; essentially we bootstrap from
USDT to supporting runtime probes.  USDT probes marking runtime
probe firing are added to libbpf itself, and these are
passed the probe arguments along with a hash of provider/probe
names.  For example, to fire a URDT probe, the user simply
calls:

  BPF_URDT_PROBE3("myprovider", "myprobe", arg1, arg2, arg3);

There is no need to declare such probes in advance.

Under the hood, an associated USDT probe then fires, and
this is the triggering mechanism for URDT.  All that is
left to do on the BPF side then is

1. ensure that the probe firing is the one we are tracing;
   this is done by matching the hash of provider/probe
   passed into the USDT probe with the cookie associated
   with the URDT attachment.  The high-order 32 bits are used
   for this; the rest is still available to the user to
   set as a per-attachment cookie value.  A mismatch
   here triggers early exit.
2. map from USDT -> URDT arguments. urdt.bpf.h provides
   means to do this; it is simply a matter of ignoring
   the last USDT hash argument.

BPF_URDT() is a macro similar to BPF_USDT() that handles
probe matching, argument assignment etc.

From the BPF side, a URDT BPF program will look like this:

SEC("urdt:/path:2:myprovider:myprobe")
int BPF_URDT(myprobe, int arg1, const char *arg2)
{
	...
}

Note that prior to provider probe, "2" is specified to
tell auto-attach that we will attach to the equivalent
2-argument URDT probe (which is a 3-argument USDT
probe under the hood).  The path should be the
binary that calls the firing functions itself (if libbpf
is statically linked like test_progs), or the path
to libbpf.so (if libbpf is dynamically linked),
since we want to attach to the USDT probe firings
in libbpf itself.  Future work could add this
URDT firing mechanism to libstapsdt also.

It appears to have a few advantages over the current
libstapsdt method:

- it supports global runtime probes.  As I understand
it, the on-the-fly library creation will be specific
to the process that triggers the probe in libstapsdt,
whereas with the above approach we can instrument
the libbpf shared library such that firings will
appear system-wide.
- it is a bit simpler to fire probes; no prior setup
is required

However, it may be preferable to add support for
the existing libstapsdt approach to libbpf. This RFC
is just intended to start that discussion.

[1] https://github.com/linux-usdt/libstapsdt

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/lib/bpf/Build             |   2 +-
 tools/lib/bpf/Makefile          |   2 +-
 tools/lib/bpf/libbpf.c          |  94 +++++++++++++++++++++
 tools/lib/bpf/libbpf.h          |  94 +++++++++++++++++++++
 tools/lib/bpf/libbpf.map        |  13 +++
 tools/lib/bpf/libbpf_internal.h |   2 +
 tools/lib/bpf/urdt.bpf.h        | 103 +++++++++++++++++++++++
 tools/lib/bpf/urdt.c            | 145 ++++++++++++++++++++++++++++++++
 8 files changed, 453 insertions(+), 2 deletions(-)
 create mode 100644 tools/lib/bpf/urdt.bpf.h
 create mode 100644 tools/lib/bpf/urdt.c

diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build
index b6619199a706..f196fce86089 100644
--- a/tools/lib/bpf/Build
+++ b/tools/lib/bpf/Build
@@ -1,4 +1,4 @@
 libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \
 	    netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \
 	    btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \
-	    usdt.o zip.o elf.o features.o
+	    usdt.o urdt.o zip.o elf.o features.o
diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
index 4be7144e4803..16aad308ab04 100644
--- a/tools/lib/bpf/Makefile
+++ b/tools/lib/bpf/Makefile
@@ -239,7 +239,7 @@ install_lib: all_cmd
 
 SRC_HDRS := bpf.h libbpf.h btf.h libbpf_common.h libbpf_legacy.h	     \
 	    bpf_helpers.h bpf_tracing.h bpf_endian.h bpf_core_read.h	     \
-	    skel_internal.h libbpf_version.h usdt.bpf.h
+	    skel_internal.h libbpf_version.h usdt.bpf.h urdt.bpf.h
 GEN_HDRS := $(BPF_GENERATED)
 
 INSTALL_PFX := $(DESTDIR)$(prefix)/include/bpf
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index fa7094ff3e66..0628167d1517 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -8975,6 +8975,7 @@ static int attach_kprobe(const struct bpf_program *prog, long cookie, struct bpf
 static int attach_uprobe(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_ksyscall(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_usdt(const struct bpf_program *prog, long cookie, struct bpf_link **link);
+static int attach_urdt(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_tp(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_raw_tp(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_trace(const struct bpf_program *prog, long cookie, struct bpf_link **link);
@@ -9003,6 +9004,7 @@ static const struct bpf_sec_def section_defs[] = {
 	SEC_DEF("kretsyscall+",		KPROBE, 0, SEC_NONE, attach_ksyscall),
 	SEC_DEF("usdt+",		KPROBE,	0, SEC_USDT, attach_usdt),
 	SEC_DEF("usdt.s+",		KPROBE,	0, SEC_USDT | SEC_SLEEPABLE, attach_usdt),
+	SEC_DEF("urdt+",		KPROBE, 0, SEC_USDT , attach_urdt),
 	SEC_DEF("tc/ingress",		SCHED_CLS, BPF_TCX_INGRESS, SEC_NONE), /* alias for tcx */
 	SEC_DEF("tc/egress",		SCHED_CLS, BPF_TCX_EGRESS, SEC_NONE),  /* alias for tcx */
 	SEC_DEF("tcx/ingress",		SCHED_CLS, BPF_TCX_INGRESS, SEC_NONE),
@@ -11884,6 +11886,98 @@ static int attach_usdt(const struct bpf_program *prog, long cookie, struct bpf_l
 	return err;
 }
 
+/* 2 less than USDT_MAX_ARG_CNT */
+#define URDT_MAX_ARG_CNT	11
+
+struct bpf_link *bpf_program__attach_urdt(const struct bpf_program *prog, pid_t pid,
+					  const char *binary_path,
+					  const char *urdt_provider, const char *urdt_name,
+					  const struct bpf_urdt_opts *opts)
+{
+	DECLARE_LIBBPF_OPTS(bpf_usdt_opts, usdt_opts);
+	char resolved_path[512];
+	struct bpf_link *link;
+	char probename[16];
+	unsigned short nargs = 0;
+	int err;
+
+	if (!OPTS_VALID(opts, bpf_urdt_opts))
+		return libbpf_err_ptr(-EINVAL);
+
+	if (bpf_program__fd(prog) < 0) {
+		pr_warn("prog '%s': can't attach BPF program w/o FD (did you load it?)\n",
+			prog->name);
+		return libbpf_err_ptr(-EINVAL);
+	}
+	if (!binary_path)
+		return libbpf_err_ptr(-EINVAL);
+
+	if (!strchr(binary_path, '/')) {
+		err = resolve_full_path(binary_path, resolved_path, sizeof(resolved_path));
+		if (err) {
+			pr_warn("prog '%s': failed to resolve full path for '%s': %d\n",
+				prog->name, binary_path, err);
+			return libbpf_err_ptr(err);
+		}
+		binary_path = resolved_path;
+	}
+
+	/* High-order 32 bits of cookie identify the provider/probe.
+	 * When the shared USDT probe fires, we use these bits to
+	 * compare to final USDT arg to identify probe firing.
+	 */
+	usdt_opts.usdt_cookie = ((long)urdt_probe_hash(urdt_provider, urdt_name)) << 32;
+
+	if (opts) {
+		/* low-order 32 bits can be specified by user */
+		usdt_opts.usdt_cookie |= opts->urdt_cookie;
+		nargs = opts->urdt_nargs;
+		if (nargs > URDT_MAX_ARG_CNT)
+			return libbpf_err_ptr(-EINVAL);
+	}
+	snprintf(probename, sizeof(probename), "probe%hu", nargs);
+
+	/* attach to USDT probe urdt:probeN */
+	link = bpf_program__attach_usdt(prog, pid, binary_path, "urdt", probename,
+					&usdt_opts);
+	err = libbpf_get_error(link);
+	if (err)
+		return libbpf_err_ptr(err);
+	return link;
+}
+
+static int attach_urdt(const struct bpf_program *prog, long cookie, struct bpf_link **link)
+{
+	char *path = NULL, *provider = NULL, *name = NULL;
+	const char *sec_name;
+	short nargs = 0;
+	int n, err;
+
+	sec_name = bpf_program__section_name(prog);
+	if (strcmp(sec_name, "urdt") == 0) {
+		/* no auto-attach for just SEC("urdt") */
+		*link = NULL;
+		return 0;
+	}
+	n = sscanf(sec_name, "urdt/%m[^:]:%hd:%m[^:]:%m[^:]", &path, &nargs, &provider, &name);
+	if (n != 4) {
+		pr_warn("invalid section '%s', expected SEC(\"urdt/<path>:<nargs>:<provider>:<name>\")\n",
+			sec_name);
+		err = -EINVAL;
+	} else {
+		DECLARE_LIBBPF_OPTS(bpf_urdt_opts, urdt_opts);
+
+		urdt_opts.urdt_nargs = nargs;
+		*link = bpf_program__attach_urdt(prog, -1 /* any process */, path,
+						 provider, name, &urdt_opts);
+		err = libbpf_get_error(*link);
+	}
+	free(path);
+	free(provider);
+	free(name);
+	return err;
+}
+
 static int determine_tracepoint_id(const char *tp_category,
 				   const char *tp_name)
 {
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index 5723cbbfcc41..eb2efe675d33 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -742,6 +742,100 @@ bpf_program__attach_usdt(const struct bpf_program *prog,
 			 const char *usdt_provider, const char *usdt_name,
 			 const struct bpf_usdt_opts *opts);
 
+struct bpf_urdt_opts {
+	/* size of this struct, for forward/backward compatibility */
+	size_t sz;
+	__u32 urdt_cookie;
+	unsigned short urdt_nargs;
+	size_t :0;
+};
+#define bpf_urdt_opts__last_field urdt_nargs
+
+/* API functions to dynamically fire URDT probes. */
+LIBBPF_API void bpf_urdt__probe0(const char *provider, const char *probe);
+LIBBPF_API void bpf_urdt__probe1(const char *provider, const char *probe, long arg1);
+LIBBPF_API void bpf_urdt__probe2(const char *provider, const char *probe, long arg1, long arg2);
+LIBBPF_API void bpf_urdt__probe3(const char *provider, const char *probe, long arg1, long arg2,
+		      long arg3);
+LIBBPF_API void bpf_urdt__probe4(const char *provider, const char *probe, long arg1, long arg2,
+		      long arg3, long arg4);
+LIBBPF_API void bpf_urdt__probe5(const char *provider, const char *probe, long arg1, long arg2,
+		      long arg3, long arg4, long arg5);
+LIBBPF_API void bpf_urdt__probe6(const char *provider, const char *probe, long arg1, long arg2,
+		      long arg3, long arg4, long arg5, long arg6);
+LIBBPF_API void bpf_urdt__probe7(const char *provider, const char *probe, long arg1, long arg2,
+		      long arg3, long arg4, long arg5, long arg6, long arg7);
+LIBBPF_API void bpf_urdt__probe8(const char *provider, const char *probe, long arg1, long arg2,
+		      long arg3, long arg4, long arg5, long arg6, long arg7, long arg8);
+LIBBPF_API void bpf_urdt__probe9(const char *provider, const char *probe, long arg1, long arg2,
+		      long arg3, long arg4, long arg5, long arg6, long arg7, long arg8,
+		      long arg9);
+LIBBPF_API void bpf_urdt__probe10(const char *provider, const char *probe, long arg1, long arg2,
+		      long arg3, long arg4, long arg5, long arg6, long arg7, long arg8,
+		      long arg9, long arg10);
+LIBBPF_API void bpf_urdt__probe11(const char *provider, const char *probe, long arg1, long arg2,
+		       long arg3, long arg4, long arg5, long arg6, long arg7, long arg8,
+		       long arg9, long arg10, long arg11);
+
+#define BPF_URDT_PROBE0(provider, probe)						\
+	bpf_urdt__probe0(provider, probe)
+#define BPF_URDT_PROBE1(provider, probe, arg1)						\
+	bpf_urdt__probe1(provider, probe, (long)arg1)
+#define BPF_URDT_PROBE2(provider, probe, arg1, arg2)					\
+	bpf_urdt__probe2(provider, probe, (long)arg1, (long)arg2)
+#define BPF_URDT_PROBE3(provider, probe, arg1, arg2, arg3)				\
+	bpf_urdt__probe3(provider, probe, (long)arg1, (long)arg2, (long)arg3)
+#define BPF_URDT_PROBE4(provider, probe, arg1, arg2, arg3, arg4)			\
+	bpf_urdt__probe4(provider, probe, (long)arg1, (long)arg2, (long)arg3, (long)arg4)
+#define BPF_URDT_PROBE5(provider, probe, arg1, arg2, arg3, arg4, arg5)			\
+	bpf_urdt__probe5(provider, probe, (long)arg1, (long)arg2, (long)arg3, (long)arg4, \
+			 (long)arg5)
+#define BPF_URDT_PROBE6(provider, probe, arg1, arg2, arg3, arg4, arg5, arg6)		\
+	bpf_urdt__probe6(provider, probe, (long)arg1, (long)arg2, (long)arg3, (long)arg4, \
+			 (long)arg5, (long)arg6)
+#define BPF_URDT_PROBE7(provider, probe, arg1, arg2, arg3, arg4, arg5, arg6, arg7)	\
+	bpf_urdt__probe7(provider, probe, (long)arg1, (long)arg2, (long)arg3, (long)arg4, \
+			 (long)arg5, (long)arg6, (long)arg7)
+#define BPF_URDT_PROBE8(provider, probe, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8) \
+	bpf_urdt__probe8(provider, probe, (long)arg1, (long)arg2, (long)arg3, (long)arg4, \
+			 (long)arg5, (long)arg6, (long)arg7, (long)arg8)
+#define BPF_URDT_PROBE9(provider, probe, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9) \
+	bpf_urdt__probe9(provider, probe, (long)arg1, (long)arg2, (long)arg3, (long)arg4, \
+			 (long)arg5, (long)arg6, (long)arg7, (long)arg8, (long)arg9)
+#define BPF_URDT_PROBE10(provider, probe, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, \
+			 arg10) \
+	bpf_urdt__probe10(provider, probe, (long)arg1, (long)arg2, (long)arg3, (long)arg4, \
+			 (long)arg5, (long)arg6, (long)arg7, (long)arg8, (long)arg9, \
+			 (long)arg10)
+#define BPF_URDT_PROBE11(provider, probe, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, \
+			 arg10, arg11) \
+	bpf_urdt__probe11(provider, probe, (long)arg1, (long)arg2, (long)arg3, (long)arg4, \
+			 (long)arg5, (long)arg6, (long)arg7, (long)arg8, (long)arg9, \
+			 (long)arg10, (long)arg11)
+
+
+
+/**
+ * @brief **bpf_program__attach_urdt()** is just like
+ * bpf_program__attach_usdt() except it covers URDT (User-space
+ * Runtime-Defined Tracepoint) attachment, instead of attaching to
+ * statically defined tracepoints.
+ *
+ * @param prog BPF program to attach
+ * @param pid Process ID to attach the uprobe to, 0 for self (own process),
+ * -1 for all processes
+ * @param urdt_provider USDT provider name
+ * @param urdt_name USDT probe name
+ * @param opts Options for altering program attachment
+ * @return Reference to the newly created BPF link; or NULL is returned on error,
+ * error code is stored in errno
+ */
+LIBBPF_API struct bpf_link *
+bpf_program__attach_urdt(const struct bpf_program *prog,
+			 pid_t pid, const char *binary_path,
+			 const char *urdt_provider, const char *urdt_name,
+			 const struct bpf_urdt_opts *opts);
+
 struct bpf_tracepoint_opts {
 	/* size of this struct, for forward/backward compatibility */
 	size_t sz;
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index d9e1f57534fa..1afcfbcc81a5 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -412,4 +412,17 @@ LIBBPF_1.3.0 {
 
 LIBBPF_1.4.0 {
 		bpf_token_create;
+		bpf_program__attach_urdt;
+		bpf_urdt__probe0;
+		bpf_urdt__probe1;
+		bpf_urdt__probe2;
+		bpf_urdt__probe3;
+		bpf_urdt__probe4;
+		bpf_urdt__probe5;
+		bpf_urdt__probe6;
+		bpf_urdt__probe7;
+		bpf_urdt__probe8;
+		bpf_urdt__probe9;
+		bpf_urdt__probe10;
+		bpf_urdt__probe11;
 } LIBBPF_1.3.0;
diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
index 930cc9616527..661967c0de2c 100644
--- a/tools/lib/bpf/libbpf_internal.h
+++ b/tools/lib/bpf/libbpf_internal.h
@@ -624,6 +624,8 @@ struct bpf_link * usdt_manager_attach_usdt(struct usdt_manager *man,
 					   const char *usdt_provider, const char *usdt_name,
 					   __u64 usdt_cookie);
 
+unsigned int urdt_probe_hash(const char *provider, const char *probe);
+
 static inline bool is_pow_of_2(size_t x)
 {
 	return x && (x & (x - 1)) == 0;
diff --git a/tools/lib/bpf/urdt.bpf.h b/tools/lib/bpf/urdt.bpf.h
new file mode 100644
index 000000000000..fb9bac89db10
--- /dev/null
+++ b/tools/lib/bpf/urdt.bpf.h
@@ -0,0 +1,103 @@
+/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
+/* Copyright (c) 2024, Oracle and/or its affiliates. */
+
+#ifndef __URDT_BPF_H__
+#define __URDT_BPF_H__
+
+#include "usdt.bpf.h"
+
+/* Return number of URDT arguments defined; these are 1 less then the USDT-defined
+ * number, as we have provider/probe hash after actual arguments.
+ */
+__weak __hidden
+int bpf_urdt_arg_cnt(struct pt_regs *ctx)
+{
+	int cnt = bpf_usdt_arg_cnt(ctx);
+
+	if (cnt < 0)
+		return cnt;
+	if (cnt < 1)
+		return -ENOENT;
+	return cnt - 1;
+}
+
+/* Fetch URDT argument #*arg_num* (zero-indexed) and put its value into *res.
+ * Returns 0 on success; negative error, otherwise.
+ * On error *res is guaranteed to be set to zero.
+ */
+__weak __hidden
+int bpf_urdt_arg(struct pt_regs *ctx, __u64 arg_num, long *res)
+{
+	if (arg_num >= bpf_urdt_arg_cnt(ctx))
+		return -ENOENT;
+	return bpf_usdt_arg(ctx, arg_num, res);
+}
+
+/* Retrieve user-specified cookie value provided during attach as
+ * bpf_urdt_opts.urdt_cookie.  This corresponds to the low 32 bits of
+ * the 64-bit USDT cookie; the higher-order bits are a hash identifying
+ * the provider/probe.
+ */
+__weak __hidden
+int bpf_urdt_cookie(struct pt_regs *ctx)
+{
+	long cookie = bpf_usdt_cookie(ctx);
+
+	return (int)cookie;
+}
+
+/* Return 0 if last USDT argument (provider/probe hash) matches high-order
+ * 32 bits of USDT cookie; this tells us the probe is for us in cases
+ * where the same USDT probe is shared among multiple URDT probes.
+ */
+static __always_inline int bpf_urdt_check_hash(struct pt_regs *ctx)
+{
+	int cnt = bpf_urdt_arg_cnt(ctx);
+	long h = 0, cookie = bpf_usdt_cookie(ctx);
+
+	if (cnt < 0)
+		return cnt;
+	if (bpf_usdt_arg(ctx, cnt, &h) || (int)h != (int)(cookie >> 32))
+		return -ENOENT;
+	return 0;
+}
+
+/* we rely on ___bpf_apply() and ___bpf_narg() macros already defined in bpf_tracing.h;
+ * urdt args start at arg 3 (args 0, 1 and 2 are provider, probe and hash respectively)
+ */
+#define ___bpf_urdt_args0() ctx
+#define ___bpf_urdt_args1(x) ___bpf_urdt_args0(), ({ long _x; bpf_urdt_arg(ctx, 0, &_x); (void *)_x; })
+#define ___bpf_urdt_args2(x, args...) ___bpf_urdt_args1(args), ({ long _x; bpf_urdt_arg(ctx, 1, &_x); (void *)_x; })
+#define ___bpf_urdt_args3(x, args...) ___bpf_urdt_args2(args), ({ long _x; bpf_urdt_arg(ctx, 2, &_x); (void *)_x; })
+#define ___bpf_urdt_args4(x, args...) ___bpf_urdt_args3(args), ({ long _x; bpf_urdt_arg(ctx, 3, &_x); (void *)_x; })
+#define ___bpf_urdt_args5(x, args...) ___bpf_urdt_args4(args), ({ long _x; bpf_urdt_arg(ctx, 4, &_x); (void *)_x; })
+#define ___bpf_urdt_args6(x, args...) ___bpf_urdt_args5(args), ({ long _x; bpf_urdt_arg(ctx, 5, &_x); (void *)_x; })
+#define ___bpf_urdt_args7(x, args...) ___bpf_urdt_args6(args), ({ long _x; bpf_urdt_arg(ctx, 6, &_x); (void *)_x; })
+#define ___bpf_urdt_args8(x, args...) ___bpf_urdt_args7(args), ({ long _x; bpf_urdt_arg(ctx, 7, &_x); (void *)_x; })
+#define ___bpf_urdt_args9(x, args...) ___bpf_urdt_args8(args), ({ long _x; bpf_urdt_arg(ctx, 8, &_x); (void *)_x; })
+#define ___bpf_urdt_args10(x, args...) ___bpf_urdt_args9(args), ({ long _x; bpf_urdt_arg(ctx, 9, &_x); (void *)_x; })
+#define ___bpf_urdt_args11(x, args...) ___bpf_urdt_args10(args), ({ long _x; bpf_urdt_arg(ctx, 10, &_x); (void *)_x; })
+#define ___bpf_urdt_args(args...) ___bpf_apply(___bpf_urdt_args, ___bpf_narg(args))(args)
+
+/*
+ * BPF_URDT serves the same purpose for URDT handlers as BPF_PROG for
+ * tp_btf/fentry/fexit BPF programs and BPF_KPROBE for kprobes.
+ * Original struct pt_regs * context is preserved as 'ctx' argument.
+ */
+#define BPF_URDT(name, args...)						    \
+name(struct pt_regs *ctx);						    \
+static __always_inline typeof(name(0))					    \
+____##name(struct pt_regs *ctx, ##args);				    \
+typeof(name(0)) name(struct pt_regs *ctx)				    \
+{									    \
+	if (bpf_urdt_check_hash(ctx))					    \
+		return 0;						    \
+	_Pragma("GCC diagnostic push")					    \
+	_Pragma("GCC diagnostic ignored \"-Wint-conversion\"")		    \
+	return ____##name(___bpf_usdt_args(args));			    \
+	_Pragma("GCC diagnostic pop")					    \
+}									    \
+static __always_inline typeof(name(0))					    \
+____##name(struct pt_regs *ctx, ##args)
+
+#endif /* __URDT_BPF_H__ */
diff --git a/tools/lib/bpf/urdt.c b/tools/lib/bpf/urdt.c
new file mode 100644
index 000000000000..0c2459bd9d71
--- /dev/null
+++ b/tools/lib/bpf/urdt.c
@@ -0,0 +1,145 @@
+// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+/* Copyright (c) 2024, Oracle and/or its affiliates. */
+
+#ifndef _GNU_SOURCE
+#define _GNU_SOURCE
+#endif
+#include <ctype.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <stdbool.h>
+#include <unistd.h>
+#include <linux/bpf.h>
+
+#include "libbpf.h"
+#include "libbpf_internal.h"
+
+/* sdt.h warns if __STDC_VERSION__ is not set. */
+#ifndef __STDC_VERSION__
+#define __STDC_VERSION__	199901L
+#endif
+
+#include "sdt.h"
+
+/*
+ * User-space Runtime-Defined Tracing - URDT.
+ */
+
+/*
+ * URDT allows a program to define runtime probes in a similar
+ * manner to the compile-time USDT.
+ *
+ * A probe can be fired by calling the BPF_URDT_PROBE[N]() function,
+ * where N is the number of arguments; for example
+ *
+ * BPF_URDT_PROBE2("myprovider", "myprobe", 1, "helloworld");
+ *
+ * This will trigger firing of the USDT probe urdt:probe2
+ * within libbpf itself.  Once this probe fires, a BPF program
+ * attached to it will fire.  URDT probes use the high-order
+ * 32 bits of the USDT cookie to identify the provider/probe
+ * by hashing the provider/probe name - see urdt.bpf.h for
+ * details.  If the upper 32 bits of the cookie match the
+ * hash passed into the probe, we know the probe firing is
+ * for us.
+ */
+static unsigned int hash_combine(unsigned int hash, const char *str)
+{
+	const char *s;
+
+	if (!str)
+		return hash;
+
+	for (s = str; *s != '\0'; s++)
+		hash = hash * 31 + *s;
+	return hash;
+}
+
+unsigned int urdt_probe_hash(const char *provider, const char *probe)
+{
+	unsigned int hash = 0;
+
+	hash = hash_combine(hash, provider);
+	return hash_combine(hash, probe);
+}
+
+void bpf_urdt__probe0(const char *provider, const char *probe)
+{
+	STAP_PROBE1(urdt, probe0, urdt_probe_hash(provider, probe));
+}
+
+void bpf_urdt__probe1(const char *provider, const char *probe, long arg1)
+{
+	STAP_PROBE2(urdt, probe1, arg1, urdt_probe_hash(provider, probe));
+}
+
+void bpf_urdt__probe2(const char *provider, const char *probe, long arg1, long arg2)
+{
+	STAP_PROBE3(urdt, probe2, arg1, arg2, urdt_probe_hash(provider, probe));
+}
+
+void bpf_urdt__probe3(const char *provider, const char *probe, long arg1, long arg2, long arg3)
+{
+	STAP_PROBE4(urdt, probe3, arg1, arg2, arg3, urdt_probe_hash(provider, probe));
+}
+
+void bpf_urdt__probe4(const char *provider, const char *probe, long arg1, long arg2, long arg3,
+		      long arg4)
+{
+	STAP_PROBE5(urdt, probe4, arg1, arg2, arg3, arg4, urdt_probe_hash(provider, probe));
+}
+
+void bpf_urdt__probe5(const char *provider, const char *probe, long arg1, long arg2, long arg3,
+		      long arg4, long arg5)
+{
+	STAP_PROBE6(urdt, probe5, arg1, arg2, arg3, arg4, arg5,
+		    urdt_probe_hash(provider, probe));
+}
+
+void bpf_urdt__probe6(const char *provider, const char *probe, long arg1, long arg2, long arg3,
+		      long arg4, long arg5, long arg6)
+{
+	STAP_PROBE7(urdt, probe6, arg1, arg2, arg3, arg4, arg5, arg6,
+		    urdt_probe_hash(provider, probe));
+}
+
+void bpf_urdt__probe7(const char *provider, const char *probe, long arg1, long arg2, long arg3,
+		      long arg4, long arg5, long arg6, long arg7)
+{
+	STAP_PROBE8(urdt, probe7, arg1, arg2, arg3, arg4, arg5, arg6, arg7,
+		    urdt_probe_hash(provider, probe));
+}
+
+void bpf_urdt__probe8(const char *provider, const char *probe, long arg1, long arg2, long arg3,
+		      long arg4, long arg5, long arg6, long arg7, long arg8)
+{
+	STAP_PROBE9(urdt, probe8, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8,
+		    urdt_probe_hash(provider, probe));
+}
+
+void bpf_urdt__probe9(const char *provider, const char *probe, long arg1, long arg2, long arg3,
+		      long arg4, long arg5, long arg6, long arg7, long arg8, long arg9)
+{
+	STAP_PROBE10(urdt, probe9, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9,
+		     urdt_probe_hash(provider, probe));
+}
+
+void bpf_urdt__probe10(const char *provider, const char *probe, long arg1, long arg2, long arg3,
+		       long arg4, long arg5, long arg6, long arg7, long arg8, long arg9,
+		       long arg10)
+{
+	STAP_PROBE11(urdt, probe10, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9,
+		     arg10, urdt_probe_hash(provider, probe));
+}
+
+void bpf_urdt__probe11(const char *provider, const char *probe, long arg1, long arg2, long arg3,
+		       long arg4, long arg5, long arg6, long arg7, long arg8, long arg9,
+		       long arg10, long arg11)
+{
+	STAP_PROBE12(urdt, probe11, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9,
+		     arg10, arg11, urdt_probe_hash(provider, probe));
+}
+
+
-- 
2.39.3

[RFC bpf-next 2/2] selftests/bpf: add tests for Userspace Runtime Defined Tracepoints (URDT)

[Alan Maguire]

Add tests that verify operation of URDT probes for both
the statically and dynamically-linked libbpf cases.

Ensure probe attach and auto-attach succeed, and argument counts,
cookies and argument values match expectations.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/testing/selftests/bpf/Makefile          |   2 +-
 tools/testing/selftests/bpf/prog_tests/urdt.c | 173 ++++++++++++++++++
 tools/testing/selftests/bpf/progs/test_urdt.c | 100 ++++++++++
 .../selftests/bpf/progs/test_urdt_shared.c    |  59 ++++++
 4 files changed, 333 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/urdt.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_urdt.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_urdt_shared.c

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index fd15017ed3b1..d21acb95a3e1 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -39,7 +39,7 @@ CFLAGS += -g $(OPT_FLAGS) -rdynamic					\
 	  -I$(CURDIR) -I$(INCLUDE_DIR) -I$(GENDIR) -I$(LIBDIR)		\
 	  -I$(TOOLSINCDIR) -I$(APIDIR) -I$(OUTPUT)
 LDFLAGS += $(SAN_LDFLAGS)
-LDLIBS += $(LIBELF_LIBS) -lz -lrt -lpthread
+LDLIBS += $(LIBELF_LIBS) -lz -lrt -lpthread -ldl
 
 ifneq ($(LLVM),)
 # Silence some warnings when compiled with clang
diff --git a/tools/testing/selftests/bpf/prog_tests/urdt.c b/tools/testing/selftests/bpf/prog_tests/urdt.c
new file mode 100644
index 000000000000..725d064d78f2
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/urdt.c
@@ -0,0 +1,173 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2024, Oracle and/or its affiliates. */
+
+#include <test_progs.h>
+
+#include <dlfcn.h>
+
+#include "../sdt.h"
+
+#include "test_urdt.skel.h"
+#include "test_urdt_shared.skel.h"
+
+static volatile __u64 bla = 0xFEDCBA9876543210ULL;
+
+static void subtest_basic_urdt(void)
+{
+	LIBBPF_OPTS(bpf_urdt_opts, opts);
+	struct test_urdt *skel;
+	struct test_urdt__bss *bss;
+	long x = 1;
+	int y = 42;
+	int err;
+	int i;
+
+	skel = test_urdt__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "skel_open"))
+		return;
+
+	bss = skel->bss;
+	bss->my_pid = getpid();
+
+	err = test_urdt__attach(skel);
+	if (!ASSERT_OK(err, "skel_attach"))
+		goto cleanup;
+
+	/* urdt0 won't be auto-attached */
+	opts.urdt_cookie = 0xcafedead;
+	opts.urdt_nargs = 0;
+	skel->links.urdt0 = bpf_program__attach_urdt(skel->progs.urdt0,
+						     0 /*self*/, "/proc/self/exe",
+						     "dyn", "urdt0", &opts);
+	if (!ASSERT_OK_PTR(skel->links.urdt0, "urdt0_link"))
+		goto cleanup;
+
+	BPF_URDT_PROBE0("dyn", "urdt0");
+
+	ASSERT_EQ(bss->urdt0_called, 1, "urdt0_called");
+
+	ASSERT_EQ(bss->urdt0_cookie, 0xcafedead, "urdt0_cookie");
+	ASSERT_EQ(bss->urdt0_arg_cnt, 0, "urdt0_arg_cnt");
+	ASSERT_EQ(bss->urdt0_arg_ret, -ENOENT, "urdt0_arg_ret");
+
+	BPF_URDT_PROBE3("dyn", "urdt3", x, y, &bla);
+
+	ASSERT_EQ(bss->urdt3_called, 1, "urdt3_called");
+	/* ensure the other 3-arg URDT probe does not trigger */
+	ASSERT_EQ(bss->urdt3alt_called, 0, "urdt3alt_notcalled");
+	/* auto-attached urdt3 gets default zero cookie value */
+	ASSERT_EQ(bss->urdt3_cookie, 0, "urdt3_cookie");
+	ASSERT_EQ(bss->urdt3_arg_cnt, 3, "urdt3_arg_cnt");
+
+	ASSERT_EQ(bss->urdt3_arg1, 1, "urdt3_arg1");
+	ASSERT_EQ(bss->urdt3_arg2, 42, "urdt3_arg2");
+	ASSERT_EQ((long)bss->urdt3_arg3, (long)&bla, "urdt3_arg3");
+
+	/* now call alternative 3-arg function, and make sure dyn/urdt3
+	 * does not trigger.
+	 */
+	BPF_URDT_PROBE3("dyn", "urdt3alt", y, &bla, x);
+
+	ASSERT_EQ(bss->urdt3alt_called, 1, "urdt3alt_called");
+	ASSERT_EQ(bss->urdt3_called, 1, "urdt3_notcalled");
+
+	ASSERT_EQ(bss->urdt3alt_arg1, 42, "urdt3alt_arg1");
+	ASSERT_EQ((long)bss->urdt3alt_arg2, (long)&bla, "urdt3_arg3");
+	ASSERT_EQ(bss->urdt3alt_arg3, 1, "urdt3alt_arg3");
+
+	BPF_URDT_PROBE11("dyn", "urdt11", 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11);
+
+	ASSERT_EQ(bss->urdt11_called, 1, "urdt11_called");
+	ASSERT_EQ(bss->urdt3_called, 1, "urdt3_called");
+	for (i = 0; i < 11; i++)
+		ASSERT_EQ(bss->urdt11_args[i], i + 1, "urdt11_arg");
+
+cleanup:
+	test_urdt__destroy(skel);
+}
+
+#define LIBBPF_SO_PATH	"./tools/build/libbpf/libbpf.so"
+
+/* verify shared object attach/firing works for libbpf.so */
+static void subtest_shared_urdt(void)
+{
+	LIBBPF_OPTS(bpf_urdt_opts, opts);
+	struct test_urdt_shared *skel;
+	void *dlh;
+	void (*probe0)(const char *provider, const char *probe);
+	void (*probe4)(const char *provider, const char *probe, long arg1, long arg2,
+		       long arg3, long arg4);
+	struct test_urdt_shared__bss *bss;
+	long x = 1;
+	int y = 42;
+	int z = 3;
+	int err;
+
+	skel = test_urdt_shared__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "skel_open"))
+		return;
+	bss = skel->bss;
+	bss->my_pid = getpid();
+
+	err = test_urdt_shared__attach(skel);
+	if (!ASSERT_OK(err, "skel_attach"))
+		goto cleanup;
+
+	/* urdt0 won't be auto-attached */
+	opts.urdt_cookie = 0xcafedead;
+	opts.urdt_nargs = 0;
+	skel->links.urdt0 = bpf_program__attach_urdt(skel->progs.urdt0,
+						     -1 /* all */,
+						     LIBBPF_SO_PATH,
+						     "dyn", "urdt0", &opts);
+	if (!ASSERT_OK_PTR(skel->links.urdt0, "urdt0_link"))
+		goto cleanup;
+
+	/* test_progs is statically linked with libbpf, so we need to dlopen/dlsym
+	 * probe firing functions in the shared object we have attached to in order
+	 * to trigger probe firing.  If a program is dynamically linked to libbpf
+	 * for probe firing, this won't be needed, but we want to make sure this
+	 * mode of operation works as it will likely be the common case.
+	 */
+	dlh = dlopen(LIBBPF_SO_PATH, RTLD_NOW);
+	if (!ASSERT_NEQ(dlh, NULL, "dlopen"))
+		goto cleanup;
+	probe0 = dlsym(dlh, "bpf_urdt__probe0");
+	if (!ASSERT_NEQ(probe0, NULL, "dlsym_probe0"))
+		goto cleanup;
+	probe4 = dlsym(dlh, "bpf_urdt__probe4");
+	if (!ASSERT_NEQ(probe4, NULL, "dlsym_probe4"))
+		goto cleanup;
+
+	probe0("dyn", "urdt0");
+
+	ASSERT_EQ(bss->urdt0_called, 1, "urdt0_called");
+
+	ASSERT_EQ(bss->urdt0_cookie, 0xcafedead, "urdt0_cookie");
+	ASSERT_EQ(bss->urdt0_arg_cnt, 0, "urdt0_arg_cnt");
+	ASSERT_EQ(bss->urdt0_arg_ret, -ENOENT, "urdt0_arg_ret");
+
+	probe4("dyn", "urdt4", (long)x, (long)y, (long)z, (long)&bla);
+
+	ASSERT_EQ(bss->urdt4_called, 1, "urdt4_called");
+	/* auto-attached urdt4 gets default zero cookie value */
+	ASSERT_EQ(bss->urdt4_cookie, 0, "urdt4_cookie");
+	ASSERT_EQ(bss->urdt4_arg_cnt, 4, "urdt4_arg_cnt");
+
+	ASSERT_EQ(bss->urdt4_arg1, 1, "urdt4_arg1");
+	ASSERT_EQ(bss->urdt4_arg2, 42, "urdt4_arg2");
+	ASSERT_EQ(bss->urdt4_arg3, 3, "urdt4_arg3");
+	ASSERT_EQ((long)bss->urdt4_arg4, (long)&bla, "urdt4_arg4");
+cleanup:
+	if (dlh)
+		dlclose(dlh);
+	test_urdt_shared__destroy(skel);
+}
+
+void test_urdt(void)
+{
+	if (test__start_subtest("basic"))
+		subtest_basic_urdt();
+	if (test__start_subtest("shared"))
+		subtest_shared_urdt();
+}
diff --git a/tools/testing/selftests/bpf/progs/test_urdt.c b/tools/testing/selftests/bpf/progs/test_urdt.c
new file mode 100644
index 000000000000..82d5fbdc5744
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/test_urdt.c
@@ -0,0 +1,100 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2024, Oracle and/or its affiliates. */
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/urdt.bpf.h>
+
+int my_pid;
+
+int urdt0_called;
+int urdt0_cookie;
+int urdt0_arg_cnt;
+int urdt0_arg_ret;
+
+SEC("urdt")
+int BPF_URDT(urdt0)
+{
+	long tmp;
+
+	if (my_pid != (bpf_get_current_pid_tgid() >> 32))
+		return 0;
+
+	__sync_fetch_and_add(&urdt0_called, 1);
+
+	urdt0_cookie = bpf_urdt_cookie(ctx);
+	urdt0_arg_cnt = bpf_urdt_arg_cnt(ctx);
+	/* should return -ENOENT for any arg_num */
+	urdt0_arg_ret = bpf_usdt_arg(ctx, bpf_get_prandom_u32(), &tmp);
+	return 0;
+}
+
+int urdt3_called;
+int urdt3_cookie;
+int urdt3_arg_cnt;
+long urdt3_arg1;
+int urdt3_arg2;
+__u64 *urdt3_arg3;
+
+SEC("urdt//proc/self/exe:3:dyn:urdt3")
+int BPF_URDT(urdt3, long x, int y, __u64 *bla)
+{
+	if (my_pid != (bpf_get_current_pid_tgid() >> 32))
+		return 0;
+
+	__sync_fetch_and_add(&urdt3_called, 1);
+
+	__sync_fetch_and_add(&urdt3_cookie, bpf_urdt_cookie(ctx));
+	__sync_fetch_and_add(&urdt3_arg_cnt, bpf_urdt_arg_cnt(ctx));
+
+	__sync_fetch_and_add(&urdt3_arg1, x);
+	__sync_fetch_and_add(&urdt3_arg2, y);
+	__sync_fetch_and_add(&urdt3_arg3, bla);
+
+	return 0;
+}
+
+int urdt3alt_called;
+int urdt3alt_cookie;
+int urdt3alt_arg1;
+__u64 *urdt3alt_arg2;
+long urdt3alt_arg3;
+
+SEC("urdt//proc/self/exe:3:dyn:urdt3alt")
+int BPF_URDT(urdt3alt, int y, __u64 *bla, long x)
+{
+	__sync_fetch_and_add(&urdt3alt_called, 1);
+
+	__sync_fetch_and_add(&urdt3alt_cookie, bpf_urdt_cookie(ctx));
+
+	__sync_fetch_and_add(&urdt3alt_arg1, y);
+	__sync_fetch_and_add(&urdt3alt_arg2, bla);
+	__sync_fetch_and_add(&urdt3alt_arg3, x);
+
+	return 0;
+}
+
+int urdt11_called;
+int urdt11_args[11];
+
+SEC("urdt//proc/self/exe:11:dyn:urdt11")
+int BPF_URDT(urdt11, int arg1, int arg2, int arg3, int arg4, int arg5,
+	     int arg6, int arg7, int arg8, int arg9, int arg10, int arg11)
+{
+	__sync_fetch_and_add(&urdt11_called, 1);
+	__sync_fetch_and_add(&urdt11_args[0], arg1);
+	__sync_fetch_and_add(&urdt11_args[1], arg2);
+	__sync_fetch_and_add(&urdt11_args[2], arg3);
+	__sync_fetch_and_add(&urdt11_args[3], arg4);
+	__sync_fetch_and_add(&urdt11_args[4], arg5);
+	__sync_fetch_and_add(&urdt11_args[5], arg6);
+	__sync_fetch_and_add(&urdt11_args[6], arg7);
+	__sync_fetch_and_add(&urdt11_args[7], arg8);
+	__sync_fetch_and_add(&urdt11_args[8], arg9);
+	__sync_fetch_and_add(&urdt11_args[9], arg10);
+	__sync_fetch_and_add(&urdt11_args[10], arg11);
+
+	return 0;
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/test_urdt_shared.c b/tools/testing/selftests/bpf/progs/test_urdt_shared.c
new file mode 100644
index 000000000000..2cdb181f73bd
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/test_urdt_shared.c
@@ -0,0 +1,59 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2024, Oracle and/or its affiliates. */
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/urdt.bpf.h>
+
+int my_pid;
+
+int urdt0_called;
+int urdt0_cookie;
+int urdt0_arg_cnt;
+int urdt0_arg_ret;
+
+SEC("urdt")
+int BPF_URDT(urdt0)
+{
+	long tmp;
+
+	if (my_pid != (bpf_get_current_pid_tgid() >> 32))
+		return 0;
+
+	__sync_fetch_and_add(&urdt0_called, 1);
+
+	urdt0_cookie = bpf_urdt_cookie(ctx);
+	urdt0_arg_cnt = bpf_urdt_arg_cnt(ctx);
+	/* should return -ENOENT for any arg_num */
+	urdt0_arg_ret = bpf_usdt_arg(ctx, bpf_get_prandom_u32(), &tmp);
+	return 0;
+}
+
+int urdt4_called;
+int urdt4_cookie;
+int urdt4_arg_cnt;
+long urdt4_arg1;
+int urdt4_arg2;
+int urdt4_arg3;
+__u64 *urdt4_arg4;
+
+SEC("urdt/./tools/build/libbpf/libbpf.so:4:dyn:urdt4")
+int BPF_URDT(urdt4, long x, int y, int z, __u64 *bla)
+{
+	if (my_pid != (bpf_get_current_pid_tgid() >> 32))
+		return 0;
+
+	__sync_fetch_and_add(&urdt4_called, 1);
+
+	__sync_fetch_and_add(&urdt4_cookie, bpf_urdt_cookie(ctx));
+	__sync_fetch_and_add(&urdt4_arg_cnt, bpf_urdt_arg_cnt(ctx));
+
+	__sync_fetch_and_add(&urdt4_arg1, x);
+	__sync_fetch_and_add(&urdt4_arg2, y);
+	__sync_fetch_and_add(&urdt4_arg3, z);
+	__sync_fetch_and_add(&urdt4_arg4, bla);
+
+	return 0;
+}
+
+char _license[] SEC("license") = "GPL";
-- 
2.39.3

Re: [RFC bpf-next 0/2] libbpf Userspace Runtime-Defined Tracing (URDT)

[Daniel Xu]

Hi Alan,

On Wed, Jan 31, 2024 at 04:20:01PM +0000, Alan Maguire wrote:
> Adding userspace tracepoints in other languages like python and
> go is a very useful for observability.  libstapsdt [1]
> and language bindings like python-stapsdt [2] that rely on it
> use a clever scheme of emulating static (USDT) userspace tracepoints
> at runtime.  This involves (as I understand it):
> 
> - fabricating a shared library
> - annotating it with ELF notes that describe its tracepoints
> - dlopen()ing it and calling the appropriate probe fire function
>   to trigger probe firing.
> 
> bcc already supports this mechanism (the examples in [2] use
> bcc to list/trigger the tracepoints), so it seems like it
> would be a good candidate for adding support to libbpf.
> 
> However, before doing that, it's worth considering if there
> are simpler ways to support runtime probe firing.  This
> small series demonstrates a simple method based on USDT
> probes added to libbpf itself.
> 
> The suggested solution comprises 3 parts
> 
> 1. functions to fire dynamic probes are added to libbpf itself
>    bpf_urdt__probeN(), where N is the number of probe arguemnts.
>    A sample usage would be
> 	bpf_urdt__probe3("myprovider", "myprobe", 1, 2, 3);
> 
>    Under the hood these correspond to USDT probes with an
>    additional argument for uniquely identifying the probe
>    (a hash of provider/probe name).
> 
> 2. we attach to the appropriate USDT probe for the specified
>    number of arguments urdt/probe0 for none, urdt/probe1 for
>    1, etc.  We utilize the high-order 32 bits of the attach
>    cookie to store the hash of the provider/probe name.
> 
> 3. when urdt/probeN fires, the BPF_URDT() macro (which
>    is similar to BPF_USDT()) checks if the hash passed
>    in (identifying provider/probe) matches the attach
>    cookie high-order 32 bits; if not it must be a firing
>    for a different dynamic probe and we exit early.
> 
> Auto-attach support is also added, for example the following
> would add a dynamic probe for provider:myprobe:
> 
> SEC("udrt/libbpf.so:2:myprovider:myprobe")
> int BPF_URDT(myprobe, int arg1, char *arg2)
> {
>  ...
> }
> 
> (Note the "2" above specifies the number of arguments to
> the probe, otherwise it is identical to USDT).
> 
> The above program can then be triggered by a call to
> 
>  BPF_URDT_PROBE2("myprovider", "myprobe", 1, "hi");
> 
> The useful thing about this is that by attaching to
> libbpf.so (and firing probes using that library) we
> can get system-wide dynamic probe firing.  It is also
> easy to fire a dynamic probe - no setup is required.
> 
> More examples of auto and manual attach can be found in
> the selftests (patch 2).
> 
> If this approach appears to be worth pursing, we could
> also look at adding support to libstapsdt for it.

This is quite interesting, thanks for the RFC. I hope to take a closer
look at it this week.

At a high level, it looks like you're basically defining a scheme for
well-known USDT probes, right? Since, not all languages enjoy linking
against C (looking at you golang...), perhaps it would make sense to
codify the scheme in a "spec". Probably just located in Documentation/
or something. That way there can be independent implementations.

This is something that would be nice to support in bpftrace as well. I'm
sure other tracers would probably find use as well.

Thanks,
Daniel

Re: [RFC bpf-next 0/2] libbpf Userspace Runtime-Defined Tracing (URDT)

[Alan Maguire]

On 31/01/2024 17:06, Daniel Xu wrote:
> Hi Alan,
> 
> On Wed, Jan 31, 2024 at 04:20:01PM +0000, Alan Maguire wrote:
>> Adding userspace tracepoints in other languages like python and
>> go is a very useful for observability.  libstapsdt [1]
>> and language bindings like python-stapsdt [2] that rely on it
>> use a clever scheme of emulating static (USDT) userspace tracepoints
>> at runtime.  This involves (as I understand it):
>>
>> - fabricating a shared library
>> - annotating it with ELF notes that describe its tracepoints
>> - dlopen()ing it and calling the appropriate probe fire function
>>   to trigger probe firing.
>>
>> bcc already supports this mechanism (the examples in [2] use
>> bcc to list/trigger the tracepoints), so it seems like it
>> would be a good candidate for adding support to libbpf.
>>
>> However, before doing that, it's worth considering if there
>> are simpler ways to support runtime probe firing.  This
>> small series demonstrates a simple method based on USDT
>> probes added to libbpf itself.
>>
>> The suggested solution comprises 3 parts
>>
>> 1. functions to fire dynamic probes are added to libbpf itself
>>    bpf_urdt__probeN(), where N is the number of probe arguemnts.
>>    A sample usage would be
>> 	bpf_urdt__probe3("myprovider", "myprobe", 1, 2, 3);
>>
>>    Under the hood these correspond to USDT probes with an
>>    additional argument for uniquely identifying the probe
>>    (a hash of provider/probe name).
>>
>> 2. we attach to the appropriate USDT probe for the specified
>>    number of arguments urdt/probe0 for none, urdt/probe1 for
>>    1, etc.  We utilize the high-order 32 bits of the attach
>>    cookie to store the hash of the provider/probe name.
>>
>> 3. when urdt/probeN fires, the BPF_URDT() macro (which
>>    is similar to BPF_USDT()) checks if the hash passed
>>    in (identifying provider/probe) matches the attach
>>    cookie high-order 32 bits; if not it must be a firing
>>    for a different dynamic probe and we exit early.
>>
>> Auto-attach support is also added, for example the following
>> would add a dynamic probe for provider:myprobe:
>>
>> SEC("udrt/libbpf.so:2:myprovider:myprobe")
>> int BPF_URDT(myprobe, int arg1, char *arg2)
>> {
>>  ...
>> }
>>
>> (Note the "2" above specifies the number of arguments to
>> the probe, otherwise it is identical to USDT).
>>
>> The above program can then be triggered by a call to
>>
>>  BPF_URDT_PROBE2("myprovider", "myprobe", 1, "hi");
>>
>> The useful thing about this is that by attaching to
>> libbpf.so (and firing probes using that library) we
>> can get system-wide dynamic probe firing.  It is also
>> easy to fire a dynamic probe - no setup is required.
>>
>> More examples of auto and manual attach can be found in
>> the selftests (patch 2).
>>
>> If this approach appears to be worth pursing, we could
>> also look at adding support to libstapsdt for it.
> 
> This is quite interesting, thanks for the RFC. I hope to take a closer
> look at it this week.
> 
> At a high level, it looks like you're basically defining a scheme for
> well-known USDT probes, right? Since, not all languages enjoy linking
> against C (looking at you golang...), perhaps it would make sense to
> codify the scheme in a "spec". Probably just located in Documentation/
> or something. That way there can be independent implementations.
> 

That's a great idea. If tracers or libraries like libstapsdt wanted
to define their own probe firings, we could add option fields to
the urdt attach function to specify the USDT provider/probe names
for attachment for runtime probe emulation. If another entity was
specifying USDT probes like this, it would also need to pass through a
hash value, so that triple could potentially be added to the urdt attach
options; doing that would give other entities like libstapsdt a free
hand to define their own trigger mechanisms, and not have to link to
libbpf. That said, the benefits of a common scheme are that libbpf does
not have to figure out which USDT probes to use in advance, so having
defaults that are documented makes sense. It's especially useful for
auto-attach by SEC() name.

> This is something that would be nice to support in bpftrace as well. I'm
> sure other tracers would probably find use as well.
> 

Yeah, I was aiming for as simple as possible to make it easy for tracers
to adopt. The aim is to try and facilitate wider support for runtime
probes, so I'm curious if other folks have run into issues in that area,
or have suggestions.

Thanks!

Alan

> Thanks,
> Daniel

Re: [RFC bpf-next 0/2] libbpf Userspace Runtime-Defined Tracing (URDT)

[Daniel Xu]

On Wed, Jan 31, 2024 at 05:38:37PM +0000, Alan Maguire wrote:
> On 31/01/2024 17:06, Daniel Xu wrote:

[..]

> 
> > This is something that would be nice to support in bpftrace as well. I'm
> > sure other tracers would probably find use as well.
> > 
> 
> Yeah, I was aiming for as simple as possible to make it easy for tracers
> to adopt. The aim is to try and facilitate wider support for runtime
> probes, so I'm curious if other folks have run into issues in that area,
> or have suggestions.

I think user_events is a thing: https://docs.kernel.org/trace/user_events.html

Might be good to take a look at that and see how URDT compares.

Re: [RFC bpf-next 0/2] libbpf Userspace Runtime-Defined Tracing (URDT)

[Alan Maguire]

On 31/01/2024 16:20, Alan Maguire wrote:
> Adding userspace tracepoints in other languages like python and
> go is a very useful for observability.  libstapsdt [1]
> and language bindings like python-stapsdt [2] that rely on it
> use a clever scheme of emulating static (USDT) userspace tracepoints
> at runtime.  This involves (as I understand it):
> 
> - fabricating a shared library
> - annotating it with ELF notes that describe its tracepoints
> - dlopen()ing it and calling the appropriate probe fire function
>   to trigger probe firing.
> 
> bcc already supports this mechanism (the examples in [2] use
> bcc to list/trigger the tracepoints), so it seems like it
> would be a good candidate for adding support to libbpf.
> 
> However, before doing that, it's worth considering if there
> are simpler ways to support runtime probe firing.  This
> small series demonstrates a simple method based on USDT
> probes added to libbpf itself.
>

[snip]

> The useful thing about this is that by attaching to
> libbpf.so (and firing probes using that library) we
> can get system-wide dynamic probe firing.  It is also
> easy to fire a dynamic probe - no setup is required.
> 
> More examples of auto and manual attach can be found in
> the selftests (patch 2).
> 
> If this approach appears to be worth pursing, we could
> also look at adding support to libstapsdt for it.
>

Proof-of-concept libstapsdt support has been built and tested;
consumers of libstapsdt continue to work the same way but
with URDT support, we can trace dynamic events system-wide.
See

https://github.com/linux-usdt/libstapsdt/compare/main...alan-maguire:libstapsdt:urdt


> Alan Maguire (2):
>   libbpf: add support for Userspace Runtime Dynamic Tracing (URDT)
>   selftests/bpf: add tests for Userspace Runtime Defined Tracepoints
>     (URDT)
> 
>  tools/lib/bpf/Build                           |   2 +-
>  tools/lib/bpf/Makefile                        |   2 +-
>  tools/lib/bpf/libbpf.c                        |  94 ++++++++++
>  tools/lib/bpf/libbpf.h                        |  94 ++++++++++
>  tools/lib/bpf/libbpf.map                      |  13 ++
>  tools/lib/bpf/libbpf_internal.h               |   2 +
>  tools/lib/bpf/urdt.bpf.h                      | 103 +++++++++++
>  tools/lib/bpf/urdt.c                          | 145 +++++++++++++++
>  tools/testing/selftests/bpf/Makefile          |   2 +-
>  tools/testing/selftests/bpf/prog_tests/urdt.c | 173 ++++++++++++++++++
>  tools/testing/selftests/bpf/progs/test_urdt.c | 100 ++++++++++
>  .../selftests/bpf/progs/test_urdt_shared.c    |  59 ++++++
>  12 files changed, 786 insertions(+), 3 deletions(-)
>  create mode 100644 tools/lib/bpf/urdt.bpf.h
>  create mode 100644 tools/lib/bpf/urdt.c
>  create mode 100644 tools/testing/selftests/bpf/prog_tests/urdt.c
>  create mode 100644 tools/testing/selftests/bpf/progs/test_urdt.c
>  create mode 100644 tools/testing/selftests/bpf/progs/test_urdt_shared.c
> 

Re: [RFC bpf-next 0/2] libbpf Userspace Runtime-Defined Tracing (URDT)

[Andrii Nakryiko]

On Wed, Jan 31, 2024 at 8:20 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> Adding userspace tracepoints in other languages like python and
> go is a very useful for observability.  libstapsdt [1]
> and language bindings like python-stapsdt [2] that rely on it
> use a clever scheme of emulating static (USDT) userspace tracepoints
> at runtime.  This involves (as I understand it):
>
> - fabricating a shared library
> - annotating it with ELF notes that describe its tracepoints
> - dlopen()ing it and calling the appropriate probe fire function
>   to trigger probe firing.
>
> bcc already supports this mechanism (the examples in [2] use
> bcc to list/trigger the tracepoints), so it seems like it
> would be a good candidate for adding support to libbpf.
>
> However, before doing that, it's worth considering if there
> are simpler ways to support runtime probe firing.  This
> small series demonstrates a simple method based on USDT
> probes added to libbpf itself.
>
> The suggested solution comprises 3 parts
>
> 1. functions to fire dynamic probes are added to libbpf itself
>    bpf_urdt__probeN(), where N is the number of probe arguemnts.
>    A sample usage would be
>         bpf_urdt__probe3("myprovider", "myprobe", 1, 2, 3);
>
>    Under the hood these correspond to USDT probes with an
>    additional argument for uniquely identifying the probe
>    (a hash of provider/probe name).
>
> 2. we attach to the appropriate USDT probe for the specified
>    number of arguments urdt/probe0 for none, urdt/probe1 for
>    1, etc.  We utilize the high-order 32 bits of the attach
>    cookie to store the hash of the provider/probe name.
>
> 3. when urdt/probeN fires, the BPF_URDT() macro (which
>    is similar to BPF_USDT()) checks if the hash passed
>    in (identifying provider/probe) matches the attach
>    cookie high-order 32 bits; if not it must be a firing
>    for a different dynamic probe and we exit early.

I'm sorry Alan, but I don't see this being added to libbpf. This is
nothing else than USDT with a bunch of extra conventions bolted on.
And those conventions might not work for many environments. It is
completely arbitrary that libbpf is a) assumed to be a dynamic library
and b) provides USDT hooks that will be triggered. Just because it
will be libbpf that will be used to trace those USDT hooks doesn't
mean that libbpf has to define those hooks. Just because libbpf can
trace USDTs it doesn't mean that libbpf should provide those
STAP_PROBEx() macros to define and trigger USDTs within some
application. Applications that define USDTs and applications that
attach to those USDTs are completely separate and independent. Same
here, there might be an overlap in some cases, but conceptually it's
two separate sides of the solution.

Overall, this is definitely a useful overall approach, to have a
single system-wide .so library that can be attached to trace some
USDTs, and we've explored this approach internally at Meta as well.
But I don't believe it should be part of libbpf. From libbpf's
standpoint it's just a standard USDT probe to attach to.


>
> Auto-attach support is also added, for example the following
> would add a dynamic probe for provider:myprobe:
>
> SEC("udrt/libbpf.so:2:myprovider:myprobe")
> int BPF_URDT(myprobe, int arg1, char *arg2)
> {
>  ...
> }
>
> (Note the "2" above specifies the number of arguments to
> the probe, otherwise it is identical to USDT).
>
> The above program can then be triggered by a call to
>
>  BPF_URDT_PROBE2("myprovider", "myprobe", 1, "hi");
>
> The useful thing about this is that by attaching to
> libbpf.so (and firing probes using that library) we
> can get system-wide dynamic probe firing.  It is also
> easy to fire a dynamic probe - no setup is required.
>
> More examples of auto and manual attach can be found in
> the selftests (patch 2).
>
> If this approach appears to be worth pursing, we could
> also look at adding support to libstapsdt for it.
>
> Alan Maguire (2):
>   libbpf: add support for Userspace Runtime Dynamic Tracing (URDT)
>   selftests/bpf: add tests for Userspace Runtime Defined Tracepoints
>     (URDT)
>
>  tools/lib/bpf/Build                           |   2 +-
>  tools/lib/bpf/Makefile                        |   2 +-
>  tools/lib/bpf/libbpf.c                        |  94 ++++++++++
>  tools/lib/bpf/libbpf.h                        |  94 ++++++++++
>  tools/lib/bpf/libbpf.map                      |  13 ++
>  tools/lib/bpf/libbpf_internal.h               |   2 +
>  tools/lib/bpf/urdt.bpf.h                      | 103 +++++++++++
>  tools/lib/bpf/urdt.c                          | 145 +++++++++++++++
>  tools/testing/selftests/bpf/Makefile          |   2 +-
>  tools/testing/selftests/bpf/prog_tests/urdt.c | 173 ++++++++++++++++++
>  tools/testing/selftests/bpf/progs/test_urdt.c | 100 ++++++++++
>  .../selftests/bpf/progs/test_urdt_shared.c    |  59 ++++++
>  12 files changed, 786 insertions(+), 3 deletions(-)
>  create mode 100644 tools/lib/bpf/urdt.bpf.h
>  create mode 100644 tools/lib/bpf/urdt.c
>  create mode 100644 tools/testing/selftests/bpf/prog_tests/urdt.c
>  create mode 100644 tools/testing/selftests/bpf/progs/test_urdt.c
>  create mode 100644 tools/testing/selftests/bpf/progs/test_urdt_shared.c
>
> --
> 2.39.3
>

Re: [RFC bpf-next 0/2] libbpf Userspace Runtime-Defined Tracing (URDT)

[Alan Maguire]

On 02/02/2024 21:39, Andrii Nakryiko wrote:
> On Wed, Jan 31, 2024 at 8:20 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>
>> Adding userspace tracepoints in other languages like python and
>> go is a very useful for observability.  libstapsdt [1]
>> and language bindings like python-stapsdt [2] that rely on it
>> use a clever scheme of emulating static (USDT) userspace tracepoints
>> at runtime.  This involves (as I understand it):
>>
>> - fabricating a shared library
>> - annotating it with ELF notes that describe its tracepoints
>> - dlopen()ing it and calling the appropriate probe fire function
>>   to trigger probe firing.
>>
>> bcc already supports this mechanism (the examples in [2] use
>> bcc to list/trigger the tracepoints), so it seems like it
>> would be a good candidate for adding support to libbpf.
>>
>> However, before doing that, it's worth considering if there
>> are simpler ways to support runtime probe firing.  This
>> small series demonstrates a simple method based on USDT
>> probes added to libbpf itself.
>>
>> The suggested solution comprises 3 parts
>>
>> 1. functions to fire dynamic probes are added to libbpf itself
>>    bpf_urdt__probeN(), where N is the number of probe arguemnts.
>>    A sample usage would be
>>         bpf_urdt__probe3("myprovider", "myprobe", 1, 2, 3);
>>
>>    Under the hood these correspond to USDT probes with an
>>    additional argument for uniquely identifying the probe
>>    (a hash of provider/probe name).
>>
>> 2. we attach to the appropriate USDT probe for the specified
>>    number of arguments urdt/probe0 for none, urdt/probe1 for
>>    1, etc.  We utilize the high-order 32 bits of the attach
>>    cookie to store the hash of the provider/probe name.
>>
>> 3. when urdt/probeN fires, the BPF_URDT() macro (which
>>    is similar to BPF_USDT()) checks if the hash passed
>>    in (identifying provider/probe) matches the attach
>>    cookie high-order 32 bits; if not it must be a firing
>>    for a different dynamic probe and we exit early.
> 
> I'm sorry Alan, but I don't see this being added to libbpf. This is
> nothing else than USDT with a bunch of extra conventions bolted on.
> And those conventions might not work for many environments. It is
> completely arbitrary that libbpf is a) assumed to be a dynamic library
> and b) provides USDT hooks that will be triggered. Just because it
> will be libbpf that will be used to trace those USDT hooks doesn't
> mean that libbpf has to define those hooks.

Right - that came up with the discussion with Daniel also. Adding
probes in libbpf was just a means of providing a method of last resort
for runtime probe firing, it's not strictly necessary.

> Just because libbpf can
> trace USDTs it doesn't mean that libbpf should provide those
> STAP_PROBEx() macros to define and trigger USDTs within some
> application. Applications that define USDTs and applications that
> attach to those USDTs are completely separate and independent. Same
> here, there might be an overlap in some cases, but conceptually it's
> two separate sides of the solution.
> 

I think the point is though that USDT got its start by establishing a
shared set of conventions between the to-be-traced side and the tracer,
and built upon existing uprobe support to make that work. Is there a
similar approach that we could apply for dynamic probes? libbpf isn't
necessarily the right vehicle for establishing those conventions and I'm
far from wedded to the specifics of this approach, but I do think it's a
question we should explore a bit.

> Overall, this is definitely a useful overall approach, to have a
> single system-wide .so library that can be attached to trace some
> USDTs, and we've explored this approach internally at Meta as well.
> But I don't believe it should be part of libbpf. From libbpf's
> standpoint it's just a standard USDT probe to attach to.
> 
>

For now we can pursue the approach of adding a static probe - triggered
when a dynamic probe firing is requested - to libstapsdt, and this would
give us libbpf support via USDT tracing of libstapsdt.

>>
>> Auto-attach support is also added, for example the following
>> would add a dynamic probe for provider:myprobe:
>>
>> SEC("udrt/libbpf.so:2:myprovider:myprobe")
>> int BPF_URDT(myprobe, int arg1, char *arg2)
>> {
>>  ...
>> }
>>
>> (Note the "2" above specifies the number of arguments to
>> the probe, otherwise it is identical to USDT).
>>
>> The above program can then be triggered by a call to
>>
>>  BPF_URDT_PROBE2("myprovider", "myprobe", 1, "hi");
>>
>> The useful thing about this is that by attaching to
>> libbpf.so (and firing probes using that library) we
>> can get system-wide dynamic probe firing.  It is also
>> easy to fire a dynamic probe - no setup is required.
>>
>> More examples of auto and manual attach can be found in
>> the selftests (patch 2).
>>
>> If this approach appears to be worth pursing, we could
>> also look at adding support to libstapsdt for it.
>>
>> Alan Maguire (2):
>>   libbpf: add support for Userspace Runtime Dynamic Tracing (URDT)
>>   selftests/bpf: add tests for Userspace Runtime Defined Tracepoints
>>     (URDT)
>>
>>  tools/lib/bpf/Build                           |   2 +-
>>  tools/lib/bpf/Makefile                        |   2 +-
>>  tools/lib/bpf/libbpf.c                        |  94 ++++++++++
>>  tools/lib/bpf/libbpf.h                        |  94 ++++++++++
>>  tools/lib/bpf/libbpf.map                      |  13 ++
>>  tools/lib/bpf/libbpf_internal.h               |   2 +
>>  tools/lib/bpf/urdt.bpf.h                      | 103 +++++++++++
>>  tools/lib/bpf/urdt.c                          | 145 +++++++++++++++
>>  tools/testing/selftests/bpf/Makefile          |   2 +-
>>  tools/testing/selftests/bpf/prog_tests/urdt.c | 173 ++++++++++++++++++
>>  tools/testing/selftests/bpf/progs/test_urdt.c | 100 ++++++++++
>>  .../selftests/bpf/progs/test_urdt_shared.c    |  59 ++++++
>>  12 files changed, 786 insertions(+), 3 deletions(-)
>>  create mode 100644 tools/lib/bpf/urdt.bpf.h
>>  create mode 100644 tools/lib/bpf/urdt.c
>>  create mode 100644 tools/testing/selftests/bpf/prog_tests/urdt.c
>>  create mode 100644 tools/testing/selftests/bpf/progs/test_urdt.c
>>  create mode 100644 tools/testing/selftests/bpf/progs/test_urdt_shared.c
>>
>> --
>> 2.39.3
>>
> 

Re: [RFC bpf-next 0/2] libbpf Userspace Runtime-Defined Tracing (URDT)

[Andrii Nakryiko]

On Tue, Feb 6, 2024 at 1:49 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> On 02/02/2024 21:39, Andrii Nakryiko wrote:
> > On Wed, Jan 31, 2024 at 8:20 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>
> >> Adding userspace tracepoints in other languages like python and
> >> go is a very useful for observability.  libstapsdt [1]
> >> and language bindings like python-stapsdt [2] that rely on it
> >> use a clever scheme of emulating static (USDT) userspace tracepoints
> >> at runtime.  This involves (as I understand it):
> >>
> >> - fabricating a shared library
> >> - annotating it with ELF notes that describe its tracepoints
> >> - dlopen()ing it and calling the appropriate probe fire function
> >>   to trigger probe firing.
> >>
> >> bcc already supports this mechanism (the examples in [2] use
> >> bcc to list/trigger the tracepoints), so it seems like it
> >> would be a good candidate for adding support to libbpf.
> >>
> >> However, before doing that, it's worth considering if there
> >> are simpler ways to support runtime probe firing.  This
> >> small series demonstrates a simple method based on USDT
> >> probes added to libbpf itself.
> >>
> >> The suggested solution comprises 3 parts
> >>
> >> 1. functions to fire dynamic probes are added to libbpf itself
> >>    bpf_urdt__probeN(), where N is the number of probe arguemnts.
> >>    A sample usage would be
> >>         bpf_urdt__probe3("myprovider", "myprobe", 1, 2, 3);
> >>
> >>    Under the hood these correspond to USDT probes with an
> >>    additional argument for uniquely identifying the probe
> >>    (a hash of provider/probe name).
> >>
> >> 2. we attach to the appropriate USDT probe for the specified
> >>    number of arguments urdt/probe0 for none, urdt/probe1 for
> >>    1, etc.  We utilize the high-order 32 bits of the attach
> >>    cookie to store the hash of the provider/probe name.
> >>
> >> 3. when urdt/probeN fires, the BPF_URDT() macro (which
> >>    is similar to BPF_USDT()) checks if the hash passed
> >>    in (identifying provider/probe) matches the attach
> >>    cookie high-order 32 bits; if not it must be a firing
> >>    for a different dynamic probe and we exit early.
> >
> > I'm sorry Alan, but I don't see this being added to libbpf. This is
> > nothing else than USDT with a bunch of extra conventions bolted on.
> > And those conventions might not work for many environments. It is
> > completely arbitrary that libbpf is a) assumed to be a dynamic library
> > and b) provides USDT hooks that will be triggered. Just because it
> > will be libbpf that will be used to trace those USDT hooks doesn't
> > mean that libbpf has to define those hooks.
>
> Right - that came up with the discussion with Daniel also. Adding
> probes in libbpf was just a means of providing a method of last resort
> for runtime probe firing, it's not strictly necessary.
>

Ok, glad we agree on not putting this into libbpf.

> > Just because libbpf can
> > trace USDTs it doesn't mean that libbpf should provide those
> > STAP_PROBEx() macros to define and trigger USDTs within some
> > application. Applications that define USDTs and applications that
> > attach to those USDTs are completely separate and independent. Same
> > here, there might be an overlap in some cases, but conceptually it's
> > two separate sides of the solution.
> >
>
> I think the point is though that USDT got its start by establishing a
> shared set of conventions between the to-be-traced side and the tracer,
> and built upon existing uprobe support to make that work. Is there a
> similar approach that we could apply for dynamic probes? libbpf isn't
> necessarily the right vehicle for establishing those conventions and I'm
> far from wedded to the specifics of this approach, but I do think it's a
> question we should explore a bit.

I'm a bit skeptical about "standardizing" this approach so early.
Let's see how and whether this gets used in practice widely enough,
before adding SEC("urdt") as a standard feature into libbpf.


>
> > Overall, this is definitely a useful overall approach, to have a
> > single system-wide .so library that can be attached to trace some
> > USDTs, and we've explored this approach internally at Meta as well.
> > But I don't believe it should be part of libbpf. From libbpf's
> > standpoint it's just a standard USDT probe to attach to.
> >
> >
>
> For now we can pursue the approach of adding a static probe - triggered
> when a dynamic probe firing is requested - to libstapsdt, and this would
> give us libbpf support via USDT tracing of libstapsdt.

Exactly. And I suspect big companies like Meta, Google and whatnot
might have their own small system-wide shared library, developed and
maintained internally, deployed using whatever deployment strategy
they see fit, with their own USDT name and convention about naming and
namespacing these dynamic USDTs, etc, etc.

As I said, the idea is sound and neat, I'm just not sure that assuming
libstapsdt (or whatever other predefined library) is going to work in
practice.

If anything, maybe making the kernel provide something like this as a
standard functionality is the right way to go. Something along the
VDSO lines, which doesn't trigger a context switch, but can be
centralized through the kernel? Have you considered such options?

>
> >>
> >> Auto-attach support is also added, for example the following
> >> would add a dynamic probe for provider:myprobe:
> >>
> >> SEC("udrt/libbpf.so:2:myprovider:myprobe")
> >> int BPF_URDT(myprobe, int arg1, char *arg2)
> >> {
> >>  ...
> >> }
> >>
> >> (Note the "2" above specifies the number of arguments to
> >> the probe, otherwise it is identical to USDT).
> >>
> >> The above program can then be triggered by a call to
> >>
> >>  BPF_URDT_PROBE2("myprovider", "myprobe", 1, "hi");
> >>
> >> The useful thing about this is that by attaching to
> >> libbpf.so (and firing probes using that library) we
> >> can get system-wide dynamic probe firing.  It is also
> >> easy to fire a dynamic probe - no setup is required.
> >>
> >> More examples of auto and manual attach can be found in
> >> the selftests (patch 2).
> >>
> >> If this approach appears to be worth pursing, we could
> >> also look at adding support to libstapsdt for it.
> >>
> >> Alan Maguire (2):
> >>   libbpf: add support for Userspace Runtime Dynamic Tracing (URDT)
> >>   selftests/bpf: add tests for Userspace Runtime Defined Tracepoints
> >>     (URDT)
> >>
> >>  tools/lib/bpf/Build                           |   2 +-
> >>  tools/lib/bpf/Makefile                        |   2 +-
> >>  tools/lib/bpf/libbpf.c                        |  94 ++++++++++
> >>  tools/lib/bpf/libbpf.h                        |  94 ++++++++++
> >>  tools/lib/bpf/libbpf.map                      |  13 ++
> >>  tools/lib/bpf/libbpf_internal.h               |   2 +
> >>  tools/lib/bpf/urdt.bpf.h                      | 103 +++++++++++
> >>  tools/lib/bpf/urdt.c                          | 145 +++++++++++++++
> >>  tools/testing/selftests/bpf/Makefile          |   2 +-
> >>  tools/testing/selftests/bpf/prog_tests/urdt.c | 173 ++++++++++++++++++
> >>  tools/testing/selftests/bpf/progs/test_urdt.c | 100 ++++++++++
> >>  .../selftests/bpf/progs/test_urdt_shared.c    |  59 ++++++
> >>  12 files changed, 786 insertions(+), 3 deletions(-)
> >>  create mode 100644 tools/lib/bpf/urdt.bpf.h
> >>  create mode 100644 tools/lib/bpf/urdt.c
> >>  create mode 100644 tools/testing/selftests/bpf/prog_tests/urdt.c
> >>  create mode 100644 tools/testing/selftests/bpf/progs/test_urdt.c
> >>  create mode 100644 tools/testing/selftests/bpf/progs/test_urdt_shared.c
> >>
> >> --
> >> 2.39.3
> >>
> >