[PATCH v2 0/4] ELF note-based USDT support

[PATCH v2 0/4] ELF note-based USDT support

[Alan Maguire]

This series adds support (patch 1) for ELF-note defined USDT
probes in binaries and libraries; patches 2-4 add tests.

Basic pid-specific USDT support is added; i.e. it is necessary
to specify the target pid in the provider name such as
"example1234" ; future work could add pid wildcarding.

ELF note defined probes are defined by including sys/sdt.h
from the systemtap-sdt-devel package, and are defined in
C programs via

DTRACE_PROBEn(provider, probe, [args...])

See the tests for concrete examples.

For python, go, etc, USDT probes can be added via libstapsdt [1]
and associated language-specific bindings.  This allows users
of those languages to add USDT probes too.  For example, the
following example program adds a "pythonapp" probe "firstProbe"
using the python-specific libstapsdt binding:

#!/usr/bin/python3
from time import sleep
import stapsdt
provider = stapsdt.Provider("pythonapp")
probe = provider.add_probe(
"firstProbe", stapsdt.ArgTypes.uint64, stapsdt.ArgTypes.int32)
provider.load()
while True:
    print("Firing probe...")
    if probe.fire("My little probe", 42):
        print("Probe fired!")
    sleep(1)

We can then trace this via dtrace:

# dtrace -n 'pythonapp503211:::* { printf("args %s, %d\n", copyinstr(arg0), arg1); }'

dtrace: description 'pythonapp503211:::* ' matched 1 probe
CPU     ID                    FUNCTION:NAME
  6 286628                      :firstProbe args My little probe, 42

[1] https://github.com/linux-usdt/libstapsdt

Alan Maguire (4):
  USDT: support ELF-note-defined probes
  selftests/usdt: add test for USDT note-defined probe firing, args
  selftests/usdt: add test for USDT notes in shared library
  selftests/usdt: add test covering different forms of USDT note args

 include/dtrace/pid.h                      |  29 ++
 libdtrace/dt_cg.c                         |  47 ++
 libdtrace/dt_cg.h                         |   1 +
 libdtrace/dt_pid.c                        | 466 ++++++++++++++++++++
 libdtrace/dt_prov_uprobe.c                |  19 +-
 test/unittest/usdt/sdt_notes.h            | 504 ++++++++++++++++++++++
 test/unittest/usdt/tst.usdt-notes-args.r  |   2 +
 test/unittest/usdt/tst.usdt-notes-args.sh |  51 +++
 test/unittest/usdt/tst.usdt-notes-lib.r   |  14 +
 test/unittest/usdt/tst.usdt-notes-lib.sh  | 145 +++++++
 test/unittest/usdt/tst.usdt-notes.r       |  14 +
 test/unittest/usdt/tst.usdt-notes.sh      | 121 ++++++
 12 files changed, 1406 insertions(+), 7 deletions(-)
 create mode 100644 test/unittest/usdt/sdt_notes.h
 create mode 100644 test/unittest/usdt/tst.usdt-notes-args.r
 create mode 100755 test/unittest/usdt/tst.usdt-notes-args.sh
 create mode 100644 test/unittest/usdt/tst.usdt-notes-lib.r
 create mode 100755 test/unittest/usdt/tst.usdt-notes-lib.sh
 create mode 100644 test/unittest/usdt/tst.usdt-notes.r
 create mode 100755 test/unittest/usdt/tst.usdt-notes.sh

-- 
2.43.5

[PATCH v2 1/4] USDT: support ELF-note-defined probes

[Alan Maguire]

As well as using dtrace -G to generate USDT probes, they can be added
via ELF notes describing the probe.

Read ELF notes from /proc/<pid>/exe and associated libraries,
 and parse them to retrieve uprobe address and argument-related info
to create the associated uprobe.

The painful part here is retrieving info from the string of USDT arguments
in the ELF note such that we can generate trampoline code to retrieve the
probe arguments.  Probe arguments can be either constants, register values
or dereferences from register values (plus offset).  Use bpf_probe_read[_user]
for the latter case.

Translating from the register names in the USDT argument string is
platform-specific, so we use arrays mapping the register names used
to the appropriate pt_regs field name, along with an offset (for the
aarch64 case where the regs[] array in user_pt_regs is used).

Wildcarded pid USDT probes are not yet supported; a specific
pid is required.

As well as supporting ELF-note defined probes in programs and
libraries, this patch supports dynamically-created probes that
are created via libstapsdt [1].  libstapsdt allows dynamic languages
like python to declare and fire probes by dynamically creating
a memfd-based shared library containing ELF notes for the probes.
With these changes we can also trace these probes.  This is very
useful since libstapsdt has python, NodeJS, go and luaJIT bindings.

[1] https://github.com/linux-usdt/libstapsdt

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 include/dtrace/pid.h       |  29 +++
 libdtrace/dt_cg.c          |  47 ++++
 libdtrace/dt_cg.h          |   1 +
 libdtrace/dt_pid.c         | 466 +++++++++++++++++++++++++++++++++++++
 libdtrace/dt_prov_uprobe.c |  19 +-
 5 files changed, 555 insertions(+), 7 deletions(-)

diff --git a/include/dtrace/pid.h b/include/dtrace/pid.h
index c53e6004..a8e26da4 100644
--- a/include/dtrace/pid.h
+++ b/include/dtrace/pid.h
@@ -26,6 +26,27 @@ typedef enum pid_probetype {
 	DTPPT_IS_ENABLED
 } pid_probetype_t;
 
+#define DT_USDT_MAX_ARGS     10
+
+enum dt_usdt_arg_type {
+	DT_USDT_ARG_NONE = 0,
+	DT_USDT_ARG_CONST,
+	DT_USDT_ARG_REG,
+	DT_USDT_ARG_REG_DEREF
+};
+
+struct dt_usdt_arg {
+	enum dt_usdt_arg_type ua_type;
+	int ua_val_sz;
+	int ua_val_off;
+	int64_t ua_const_val;
+	const char *ua_regs_name;	/* pt_regs/user_pt_regs */
+	const char *ua_regs_field;	/* x0/rsp etc */
+	int ua_regs_field_off;		/* used for array regs[] */
+};
+
+typedef struct dt_usdt_arg dt_usdt_arg_t;
+
 typedef struct pid_probespec {
 	pid_probetype_t pps_type;		/* probe type */
 	char *pps_prv;				/* provider (without pid) */
@@ -44,6 +65,14 @@ typedef struct pid_probespec {
 	size_t pps_xargvlen;			/* (high estimate of) length of array */
 	int8_t *pps_argmap;			/* mapped arg indexes */
 
+	int pps_nuargs;				/* number of arg specs in
+						 * pps_uargs
+						 */
+	dt_usdt_arg_t pps_uargs[DT_USDT_MAX_ARGS];
+						/* USDT ELF note-defined
+						 * provider arguments.
+						 */
+
 	/*
 	 * Fields below this point do not apply to underlying probes.
 	 */
diff --git a/libdtrace/dt_cg.c b/libdtrace/dt_cg.c
index e7e3a132..2c8f8210 100644
--- a/libdtrace/dt_cg.c
+++ b/libdtrace/dt_cg.c
@@ -651,6 +651,53 @@ dt_cg_tramp_copy_rval_from_regs(dt_pcb_t *pcb)
 		emit(dlp, BPF_STORE_IMM(BPF_DW, BPF_REG_7, DMST_ARG(i), 0));
 }
 
+void
+dt_cg_tramp_copy_args_from_usdt_spec(dt_pcb_t *pcb, const dt_usdt_arg_t *args)
+{
+	dtrace_hdl_t	*dtp = pcb->pcb_hdl;
+	dt_irlist_t	*dlp = &pcb->pcb_ir;
+	int		reg_val_off, i;
+
+	for (i = 0; i < DT_USDT_MAX_ARGS; i++) {
+		const dt_usdt_arg_t *arg = &args[i];
+		uint_t lbl_ok = dt_irlist_label(dlp);
+
+		switch (arg->ua_type) {
+		case DT_USDT_ARG_NONE:
+			return;
+		case DT_USDT_ARG_CONST:
+			emit(dlp, BPF_STORE_IMM(BPF_DW, BPF_REG_7, DMST_ARG(i),
+						arg->ua_const_val));
+			break;
+		case DT_USDT_ARG_REG:
+		case DT_USDT_ARG_REG_DEREF:
+			reg_val_off = dt_cg_ctf_offsetof(arg->ua_regs_name,
+							 arg->ua_regs_field, NULL, 0);
+			reg_val_off += arg->ua_regs_field_off;
+			emit(dlp,  BPF_LOAD(BPF_DW, BPF_REG_0, BPF_REG_8,
+					    reg_val_off));
+			/* do direct register value copy */
+			if (arg->ua_type == DT_USDT_ARG_REG) {
+				emit(dlp,  BPF_STORE(BPF_DW, BPF_REG_7, DMST_ARG(i),
+						     BPF_REG_0));
+				break;
+			}
+			/* otherwise call bpf_probe_read[_user] to get dereferenced value.
+			 */
+			emit(dlp,  BPF_MOV_REG(BPF_REG_1, BPF_REG_7));
+			emit(dlp,  BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, DMST_ARG(i)));
+			emit(dlp, BPF_MOV_IMM(BPF_REG_2, abs(arg->ua_val_sz)));
+			emit(dlp, BPF_MOV_REG(BPF_REG_3, BPF_REG_0));
+			emit(dlp, BPF_ALU64_IMM(BPF_ADD, BPF_REG_3, arg->ua_val_off));
+			emit(dlp,  BPF_CALL_HELPER(dtp->dt_bpfhelper[BPF_FUNC_probe_read_user]));
+			emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_0, 0, lbl_ok));
+			emit(dlp,  BPF_STORE_IMM(BPF_DW, BPF_REG_7, DMST_ARG(i), 0));
+			emitl(dlp, lbl_ok, BPF_NOP());
+			break;
+		}
+	}
+}
+
 static dt_node_t *
 dt_cg_tramp_var(const char *name)
 {
diff --git a/libdtrace/dt_cg.h b/libdtrace/dt_cg.h
index fb26c125..24257f0b 100644
--- a/libdtrace/dt_cg.h
+++ b/libdtrace/dt_cg.h
@@ -28,6 +28,7 @@ extern void dt_cg_tramp_copy_regs(dt_pcb_t *pcb);
 extern void dt_cg_tramp_copy_args_from_regs(dt_pcb_t *pcb, int called);
 extern void dt_cg_tramp_copy_pc_from_regs(dt_pcb_t *pcb);
 extern void dt_cg_tramp_copy_rval_from_regs(dt_pcb_t *pcb);
+extern void dt_cg_tramp_copy_args_from_usdt_spec(dt_pcb_t *pcb, const dt_usdt_arg_t *args);
 extern void dt_cg_tramp_decl_var(dt_pcb_t *pcb, dt_ident_t *idp);
 extern void dt_cg_tramp_get_var(dt_pcb_t *pcb, const char *name, int isstore,
 				int reg);
diff --git a/libdtrace/dt_pid.c b/libdtrace/dt_pid.c
index 4d53c023..cac52616 100644
--- a/libdtrace/dt_pid.c
+++ b/libdtrace/dt_pid.c
@@ -28,6 +28,8 @@
 #if defined(__amd64)
 #include <disasm.h>
 #endif
+#include <unistd.h>
+#include <linux/kernel.h>
 
 #include <port.h>
 #include <dof_parser.h>
@@ -780,6 +782,369 @@ validate_dof_record(const char *path, const dof_parsed_t *parsed,
 	return 1;
 }
 
+#define SEC_USDT_NOTE	".note.stapsdt"
+#define NAME_USDT_NOTE	"stapsdt"
+
+struct pt_regs_info {
+	const char ua_name[8];
+	const char name[8];
+	int off;
+} pt_regs_info[] = {
+
+#if defined(__aarch64__)
+	{ "sp", "sp",	0 },
+	{ "x0", "regs", 0 },
+	{ "x1", "regs", 1 * sizeof(unsigned long) },
+	{ "x2", "regs", 2 * sizeof(unsigned long) },
+	{ "x3", "regs", 3 * sizeof(unsigned long) },
+	{ "x4", "regs", 4 * sizeof(unsigned long) },
+	{ "x5", "regs", 5 * sizeof(unsigned long) },
+	{ "x6", "regs", 6 * sizeof(unsigned long) },
+	{ "x7", "regs", 7 * sizeof(unsigned long) },
+	{ "x8", "regs", 8 * sizeof(unsigned long) },
+	{ "x9", "regs", 9 * sizeof(unsigned long) },
+	{ "x10", "regs", 10 * sizeof(unsigned long) },
+	{ "x11", "regs", 11 * sizeof(unsigned long) },
+	{ "x12", "regs", 12 * sizeof(unsigned long) },
+	{ "x13", "regs", 13 * sizeof(unsigned long) },
+	{ "x14", "regs", 14 * sizeof(unsigned long) },
+	{ "x15", "regs", 15 * sizeof(unsigned long) },
+	{ "x16", "regs", 16 * sizeof(unsigned long) },
+	{ "x17", "regs", 17 * sizeof(unsigned long) },
+	{ "x18", "regs", 18 * sizeof(unsigned long) },
+	{ "x19", "regs", 19 * sizeof(unsigned long) },
+	{ "x20", "regs", 20 * sizeof(unsigned long) },
+	{ "x21", "regs", 21 * sizeof(unsigned long) },
+	{ "x222", "regs", 22 * sizeof(unsigned long) },
+	{ "x23", "regs", 23 * sizeof(unsigned long) },
+	{ "x24", "regs", 24 * sizeof(unsigned long) },
+	{ "x25", "regs", 25 * sizeof(unsigned long) },
+	{ "x26", "regs", 26 * sizeof(unsigned long) },
+	{ "x27", "regs", 27 * sizeof(unsigned long) },
+	{ "x28", "regs", 28 * sizeof(unsigned long) },
+	{ "x29", "regs", 29 * sizeof(unsigned long) },
+	{ "x30", "regs", 30 * sizeof(unsigned long) },
+	{ "x31", "regs", 31 * sizeof(unsigned long) }
+#else
+	{ "rip", "ip", 0 },
+	{ "eip", "ip", 0 },
+	{ "rax", "ax", 0 },
+	{ "eax", "ax", 0 },
+	{ "ax",  "ax", 0 },
+	{ "al",  "ax", 0 },
+	{ "rbx", "bx", 0 },
+	{ "ebx", "bx", 0 },
+	{ "bx",  "bx", 0 },
+	{ "bl",  "bx", 0 },
+	{ "rcx", "cx", 0 },
+	{ "ecx", "cx", 0 },
+	{ "cx",  "cx", 0 },
+	{ "cl",  "cx", 0 },
+	{ "rdx", "dx", 0 },
+	{ "edx", "dx", 0 },
+	{ "dx",  "dx", 0 },
+	{ "dl",  "dx", 0 },
+	{ "rsi", "si", 0 },
+	{ "esi", "si", 0 },
+	{ "si",  "si", 0 },
+	{ "sil", "si", 0 },
+	{ "rdi", "di", 0 },
+	{ "edi", "di", 0 },
+	{ "di",  "di", 0 },
+	{ "dil", "di", 0 },
+	{ "rbp", "bp", 0 },
+	{ "ebp", "bp", 0 },
+	{ "bp",  "bp", 0 },
+	{ "bpl", "bp", 0 },
+	{ "rsp", "sp", 0 },
+	{ "esp", "sp", 0 },
+	{ "sp",  "sp", 0 },
+	{ "spl", "sp", 0 }
+#endif
+};
+
+static void dt_usdt_note_print_arg(dt_usdt_arg_t *a)
+{
+	switch (a->ua_type) {
+	case DT_USDT_ARG_NONE:
+		return;
+	case DT_USDT_ARG_CONST:
+		dt_dprintf("CONST %ld\n", a->ua_const_val);
+		break;
+	case DT_USDT_ARG_REG_DEREF:
+		dt_dprintf("REG DEREF (%s.%s + %d) + %d\n",
+			   a->ua_regs_name, a->ua_regs_field,
+			   a->ua_regs_field_off, a->ua_val_off);
+		break;
+	case DT_USDT_ARG_REG:
+		dt_dprintf("REG VALUE (%s.%s + %d)\n",
+			   a->ua_regs_name, a->ua_regs_field,
+			   a->ua_regs_field_off);
+		break;
+	}
+}
+
+/* retrieve arguments; space-separated string of arguments of form:
+ * [-]numbytes@[optional_offset_from(]%regname[)]
+ *
+ * for example:
+ *
+ * -4@-4(%rbp) means memory dereference of 4 bytes, 4 bytes
+ * offset from %rbp value.
+ * 8@(%rax) means memory dereference of 8 bytes from
+ * rax register value (no offset)
+ * 8@%rax means 8 bytes from rax register value (no deref).
+ * 4@$32 means 4 byte constant value 32
+ */
+static int dt_usdt_note_parse_arg(char **argstr, struct dt_usdt_arg *a)
+{
+	char *arg = *argstr;
+	char reg[8] = {};
+	int len;
+
+	if (sscanf(arg,
+#if defined(__aarch64__)
+		   " %d @ \[ %[a-z0-9] , %d ] %n",
+		   &a->ua_val_sz, reg, &a->ua_val_off, &len)
+#else
+		   " %d @ %d ( %%%8[^)] ) %n",
+		   &a->ua_val_sz, &a->ua_val_off, reg, &len)
+#endif
+	    == 3) {
+		a->ua_type = DT_USDT_ARG_REG_DEREF;
+	} else if (sscanf(arg,
+#if defined(__aarch64__)
+			  " %d @ \[ %7[a-z0-9] ] %n", &a->ua_val_sz, reg, &len)
+#else
+			  " %d @ ( %%%7[^)] ) %n", &a->ua_val_sz, reg, &len)
+#endif
+	    == 2) {
+		a->ua_type = DT_USDT_ARG_REG_DEREF;
+	} else if (sscanf(arg,
+#if defined(__x86_64__)
+			  " %d @ $%ld %n", &a->ua_val_sz, &a->ua_const_val, &len)
+#else
+			  " %d @ %ld %n", &a->ua_val_sz, &a->ua_const_val, &len)
+#endif
+			  == 2) {
+		a->ua_type = DT_USDT_ARG_CONST;
+	} else if (sscanf(arg,
+#if defined(__aarch64__)
+			   " %d @ %7[a-z0-9] %n", &a->ua_val_sz, reg, &len)
+#else
+			  " %d @ %%%7s %n", &a->ua_val_sz, reg, &len)
+#endif
+	    == 2) {
+		    a->ua_type = DT_USDT_ARG_REG;
+	} else {
+		return -1;
+	}
+	if (strlen(reg) > 0) {
+		int i;
+
+#if defined(__aarch64__)
+		a->ua_regs_name = "struct user_pt_regs";
+#else
+		a->ua_regs_name = "struct pt_regs";
+#endif
+		for (i = 0; i < ARRAY_SIZE(pt_regs_info); i++) {
+			if (strcmp(pt_regs_info[i].ua_name, reg))
+				continue;
+			a->ua_regs_field = pt_regs_info[i].name;
+			a->ua_regs_field_off = pt_regs_info[i].off;
+		}
+	}
+	*argstr += len;
+	return 0;
+}
+
+static int dt_usdt_notes_parse(dtrace_hdl_t *dtp, dt_proc_t *dpr,
+			       dtrace_probedesc_t *pdp, dt_pcb_t *pcb,
+			       const dt_provider_t *pvp, char *path,
+			       unsigned long base_addr)
+{
+	Elf *elf;
+	Elf_Scn *scn = NULL;
+	GElf_Shdr shdr;
+	GElf_Nhdr nhdr;
+	size_t shstrndx, noff, doff, off, n;
+	Elf_Data *data;
+	GElf_Ehdr ehdr;
+	int i, ret = 0;
+	int fd = -1;
+	char *mod;
+
+	dt_dprintf("Scanning for USDT probes in ELF notes in '%s' (pid %i) matching %s:%s:%s\n",
+		   path, dpr->dpr_pid, pdp->mod, pdp->fun, pdp->prb);
+
+	fd = open(path, O_RDONLY);
+	if (fd < 0) {
+		dt_pid_error(dtp, pcb, dpr, D_PROC_USDT,
+			     "Cannot open %s: %s\n",
+			     path, strerror(errno));
+		return -1;
+	}
+	mod = strrchr(path, '/');
+	if (mod)
+		mod++;
+	else
+		mod = path;
+	elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);   // ELF_C_READ ?
+	assert(elf_kind(elf) == ELF_K_ELF);
+	elf_getshdrstrndx(elf, &shstrndx);
+
+	if (gelf_getehdr(elf, &ehdr)) {
+		switch (ehdr.e_type) {
+		case ET_EXEC:
+			/* binary does not require base addr adjustment */
+			base_addr = 0;
+			break;
+		case ET_DYN:
+			break;
+		default:
+			dt_dprintf("unexpected ELF hdr type 0x%x for '%s'\n",
+				   ehdr.e_type, path);
+			ret = -1;
+			goto out;
+		}
+	}
+
+	while (1) {
+		char *secname;
+
+		scn = elf_nextscn(elf, scn);
+		if (scn == NULL) {
+			/* no ELF notes found, not an error */
+			goto out;
+		}
+		assert(gelf_getshdr(scn, &shdr) != NULL);
+
+		secname = elf_strptr(elf, shstrndx, shdr.sh_name);
+		if (strcmp(secname, SEC_USDT_NOTE) == 0 &&
+		    shdr.sh_type == SHT_NOTE)
+			break;
+	}
+	/* No ELF notes, just bail. */
+	if (scn == NULL)
+		goto out;
+	data = elf_getdata(scn, 0);
+	for (off = 0;
+	     (off = gelf_getnote(data, off, &nhdr, &noff, &doff)) > 0;) {
+		pid_probespec_t psp = {0};
+		char *prv, *prb;
+		const char *fun;
+		char *dbuf = (char *)data->d_buf;
+		long *addrs = data->d_buf + doff; /* 3 addrs are loc/base/semaphore */
+		GElf_Sym sym;
+		const prmap_t *pmp;
+		int nargs = 0;
+
+		if (strncmp(dbuf + noff, NAME_USDT_NOTE, nhdr.n_namesz) != 0)
+			continue;
+		prv = dbuf + doff + (3*sizeof(long));
+		/* ensure prv/prb is null-terminated */
+		assert(strlen(prv) < nhdr.n_descsz);
+		prb = prv + strlen(prv) + 1;
+		assert(strlen(prb) < nhdr.n_descsz);
+		if (strncmp(pdp->prv, prv, strlen(prv)) != 0)
+			continue;
+		if (strcmp(pdp->prb, "*") != 0 && strcmp(pdp->prb, prb) != 0)
+			continue;
+		/* retrieve arguments; space-separated string of arguments
+		 * in form:
+		 * [-]numbytes@[optional_offset_from(]%regname[)]
+		 *
+		 * for example:
+		 *
+		 * -4@-4(%rbp) means memory dereference of 4 bytes, 4 bytes
+		 * offset from %rbp value.
+		 * 8@(%rax) means memory dereference of 8 bytes from
+		 * rax register value (no offset)
+		 * 8@%rax means 8 bytes from rax register value (no deref).
+		 * 4@$32 means 4 byte constant value 32
+		 */
+		if (prb + strlen(prb) + 1 < dbuf + doff + nhdr.n_descsz) {
+			char *argstr = prb + strlen(prb) + 1;
+
+			while (dt_usdt_note_parse_arg(&argstr,
+						&psp.pps_uargs[nargs]) == 0 &&
+			       nargs < DT_USDT_MAX_ARGS) {
+				dt_usdt_note_print_arg(&psp.pps_uargs[nargs]);
+				nargs++;
+			}
+			psp.pps_nuargs = nargs;
+
+		}
+		dt_dprintf("found ELF note for provider '%s', probe '%s' in %s, loc 0x%lx, base 0x%lx, nargs %d\n",
+			   prv, prb, path, addrs[0], addrs[1], nargs);
+		psp.pps_type = DTPPT_USDT;
+		psp.pps_prv = prv;
+		psp.pps_mod = mod;
+		psp.pps_prb = prb;
+		if (elf_getphdrnum(elf, &n))
+			continue;
+		for (i = 0; i < n; i++) {
+			GElf_Phdr phdr;
+
+			if (!gelf_getphdr(elf, i, &phdr))
+				break;
+
+			if (addrs[0] < phdr.p_vaddr ||
+			    addrs[0] > phdr.p_vaddr + phdr.p_memsz)
+				continue;
+			if (base_addr)
+				psp.pps_off = addrs[0];
+			else
+				psp.pps_off = addrs[0] - phdr.p_vaddr + phdr.p_offset;
+			break;
+		}
+		if (!psp.pps_off)
+			continue;
+		psp.pps_nameoff = 0;
+
+		pmp = Paddr_to_map(dpr->dpr_proc, base_addr + addrs[0]);
+		if (!pmp) {
+			dt_dprintf("%i: cannot determine 0x%lx's mapping\n",
+				   Pgetpid(dpr->dpr_proc), psp.pps_off);
+			continue;
+		}
+		psp.pps_fn = Pmap_mapfile_name(dpr->dpr_proc, pmp);
+		if (psp.pps_fn == NULL) {
+			dt_pid_error(dtp, pcb, dpr, D_PROC_USDT,
+				     "Cannot get name of mapping containing probe %s for pid %d\n",
+				     psp.pps_prb, dpr->dpr_pid);
+			ret = -1;
+			break;
+		}
+		if (dt_Plookup_by_addr(dtp, dpr->dpr_pid, base_addr + addrs[0],
+				       &fun, &sym) == 0)
+			psp.pps_fun = (char *)fun;
+		else
+			psp.pps_fun = "";
+		psp.pps_dev = pmp->pr_dev;
+		psp.pps_inum = pmp->pr_inum;
+		psp.pps_pid = dpr->dpr_pid;
+		psp.pps_nameoff = 0;
+
+		dt_dprintf("providing %s:%s:%s:%s for pid %d at addr 0x%lx\n", psp.pps_prv,
+			   psp.pps_mod, psp.pps_fun, psp.pps_prb, psp.pps_pid,
+			   base_addr + addrs[0]);
+		if (pvp->impl->provide_probe(dtp, &psp) < 0) {
+			dt_pid_error(dtp, pcb, dpr, D_PROC_USDT,
+				     "failed to instantiate probe %s for pid %d: %s",
+				     psp.pps_prb, psp.pps_pid,
+				     dtrace_errmsg(dtp, dtrace_errno(dtp)));
+			ret = -1;
+		}
+		free(psp.pps_fn);
+		if (ret == -1)
+			break;
+	}
+out:
+	elf_end(elf);
+	close(fd);
+	return ret;
+}
 
 /*
  * Create underlying probes relating to the probespec passed on input.
@@ -1202,6 +1567,105 @@ dt_pid_create_pid_probes(dtrace_probedesc_t *pdp, dtrace_hdl_t *dtp, dt_pcb_t *p
 	return err;
 }
 
+static int
+dt_pid_create_usdt_notes_probes(dtrace_probedesc_t *pdp, dtrace_hdl_t *dtp,
+				dt_pcb_t *pcb)
+{
+	const char *pidstr = &pdp->prv[strlen(pdp->prv)];
+	const dt_provider_t *pvp;
+	char path[PATH_MAX + 1];
+	dt_proc_t *dpr = NULL;
+	char line[1024];
+	FILE *fp = NULL;
+	pid_t pid = 0;
+	int err = 0;
+
+	/* only specific pids are support for ELF notes for now... */
+	while (isdigit(*(pidstr - 1)))
+		pidstr--;
+	if (strlen(pidstr) > 0)
+		pid = atoll(pidstr);
+	if (!Pexists(pid))
+		return 0;
+	pvp = dt_provider_lookup(dtp, "usdt");
+	assert(pvp != NULL);
+
+	if (dt_proc_grab_lock(dtp, pid, DTRACE_PROC_WAITING |
+			      DTRACE_PROC_SHORTLIVED) < 0) {
+		dt_pid_error(dtp, pcb, NULL, D_PROC_GRAB,
+			     "failed to grab process %d",
+			     (int)pid);
+		return 1;
+	}
+	dpr = dt_proc_lookup(dtp, pid);
+	assert(dpr != NULL);
+
+	snprintf(path, sizeof(path), "/proc/%d/maps", pid);
+	fp = fopen(path, "r");
+	if (!fp) {
+		dt_pid_error(dtp, pcb, NULL, D_PROC_GRAB,
+			     "no /proc/%d/maps found", (int)pid);
+		err = 1;
+		goto out;
+	}
+	while (fgets(line, sizeof(line) - 1, fp) != NULL) {
+		long addr_start, addr_end, file_offset;
+		long dev_major, dev_minor;
+		unsigned long inode;
+		char name[PATH_MAX + 1];
+		char perm[5];
+		int ret;
+
+		ret = sscanf(line,
+			     "%lx-%lx %4s %lx %lx:%lx %lu %[^\n]",
+			     &addr_start, &addr_end, perm, &file_offset,
+			     &dev_major, &dev_minor, &inode, name);
+		if (ret != 8 || !strchr(perm, 'x') || strchr(name, '[') != NULL)
+			continue;
+
+		/* libstapsdt uses an memfd-based library to dynamically create
+		 * stapsdt notes for dynamic languages like python; we need
+		 * the associated /proc/<pid>/fds/ fd to read these notes.
+		 */
+		if (strncmp(name, "/memfd:", strlen("/memfd:")) == 0) {
+			DIR *d;
+			struct dirent *dirent;
+			char *deleted;
+
+			deleted = strstr(name, " (deleted)");
+			*deleted = '\0';
+			snprintf(path, sizeof(path), "/proc/%d/fd", pid);
+			d = opendir(path);
+			if (d == NULL)
+				continue;
+			while ((dirent = readdir(d)) != NULL) {
+				struct stat s;
+
+				snprintf(path, sizeof(path), "/proc/%d/fd/%s",
+					 pid, dirent->d_name);
+				if (stat(path, &s) != 0 || s.st_ino != inode)
+					continue;
+				if (dt_usdt_notes_parse(dtp, dpr, pdp, pcb, pvp,
+							path, addr_start) != 0) {
+					err = 1;
+					goto out;
+				}
+			}
+		} else {
+			if (dt_usdt_notes_parse(dtp, dpr, pdp, pcb, pvp, name, addr_start) != 0) {
+				err = 1;
+				goto out;
+			}
+		}
+	}
+out:
+	if (fp)
+		fclose(fp);
+	dt_pid_fix_mod(NULL, pdp, dtp, dpr->dpr_pid);
+	dt_proc_release_unlock(dtp, pid);
+	return err;
+}
+
 int
 dt_pid_create_usdt_probes(dtrace_probedesc_t *pdp, dtrace_hdl_t *dtp, dt_pcb_t *pcb)
 {
@@ -1273,6 +1737,8 @@ dt_pid_create_usdt_probes(dtrace_probedesc_t *pdp, dtrace_hdl_t *dtp, dt_pcb_t *
 	free(globpat);
 	globfree(&globbuf);
 
+	err = dt_pid_create_usdt_notes_probes(pdp, dtp, pcb);
+
 	/* If no errors, report success. */
 	if (err == 0)
 		return 0;
diff --git a/libdtrace/dt_prov_uprobe.c b/libdtrace/dt_prov_uprobe.c
index 78d9aed6..08b60b44 100644
--- a/libdtrace/dt_prov_uprobe.c
+++ b/libdtrace/dt_prov_uprobe.c
@@ -51,6 +51,7 @@ static const char	prvname[] = "uprobe";
 #define PP_IS_ENABLED	0x4
 #define PP_IS_USDT	0x8
 #define PP_IS_MAPPED	0x10
+#define PP_IS_USDT_NOTE	0x20
 
 typedef struct dt_uprobe {
 	dev_t		dev;
@@ -58,6 +59,7 @@ typedef struct dt_uprobe {
 	char		*fn;
 	uint64_t	off;
 	int		flags;
+	dt_usdt_arg_t	uargs[DT_USDT_MAX_ARGS];
 	tp_probe_t	*tp;
 	int		argc;		   /* number of args */
 	dt_argdesc_t	*args;		   /* args array (points into argvbuf) */
@@ -651,7 +653,7 @@ static dt_probe_t *create_underlying(dtrace_hdl_t *dtp,
 	pd.prb = prb;
 
 	dt_dprintf("Providing underlying probe %s:%s:%s:%s @ %lx\n", psp->pps_prv,
-		   psp->pps_mod, psp->pps_fn, psp->pps_prb, psp->pps_off);
+		   psp->pps_mod, psp->pps_fun, psp->pps_prb, psp->pps_off);
 	uprp = dt_probe_lookup(dtp, &pd);
 	if (uprp == NULL) {
 		dt_provider_t	*pvp;
@@ -671,6 +673,7 @@ static dt_probe_t *create_underlying(dtrace_hdl_t *dtp,
 		upp->off = psp->pps_off;
 		upp->fn = strdup(psp->pps_fn);
 		upp->tp = dt_tp_alloc(dtp);
+		memcpy(&upp->uargs, psp->pps_uargs, sizeof(upp->uargs));
 		if (upp->tp == NULL)
 			goto fail;
 
@@ -702,11 +705,11 @@ static dt_probe_t *create_underlying(dtrace_hdl_t *dtp,
 	    break;
 	case DTPPT_USDT:
 	    upp->flags |= PP_IS_USDT;
+	    if (psp->pps_nuargs)
+		    upp->flags |= PP_IS_USDT_NOTE;
+	    break;
+	default:
 	    break;
-	default: ;
-	    /*
-	     * No flags needed for other types.
-	     */
 	}
 
         return uprp;
@@ -973,8 +976,10 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 	/* In some cases, we know there are no USDT probes. */  // FIXME: add more checks
 	if (upp->flags & PP_IS_RETURN)
 		goto out;
-
-	dt_cg_tramp_copy_args_from_regs(pcb, 0);
+	else if (upp->flags & PP_IS_USDT_NOTE)
+		dt_cg_tramp_copy_args_from_usdt_spec(pcb, upp->uargs);
+	else
+		dt_cg_tramp_copy_args_from_regs(pcb, 0);
 
 	/*
 	 * Apply arg mappings, if needed.
-- 
2.43.5

[PATCH v2 2/4] selftests/usdt: add test for USDT note-defined probe firing, args

[Alan Maguire]

Add test identical to the args tests to verify probe firing and
arg retrieval work for USDT notes-defined DTRACE_PROBEn() probes.
Need a copy of the sdt.h file which is public domain, so add it
in the test directory.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 test/unittest/usdt/sdt_notes.h       | 504 +++++++++++++++++++++++++++
 test/unittest/usdt/tst.usdt-notes.r  |  14 +
 test/unittest/usdt/tst.usdt-notes.sh | 121 +++++++
 3 files changed, 639 insertions(+)
 create mode 100644 test/unittest/usdt/sdt_notes.h
 create mode 100644 test/unittest/usdt/tst.usdt-notes.r
 create mode 100755 test/unittest/usdt/tst.usdt-notes.sh

diff --git a/test/unittest/usdt/sdt_notes.h b/test/unittest/usdt/sdt_notes.h
new file mode 100644
index 00000000..ec5423e2
--- /dev/null
+++ b/test/unittest/usdt/sdt_notes.h
@@ -0,0 +1,504 @@
+/* <sys/sdt.h> - Systemtap static probe definition macros.
+
+   This file is dedicated to the public domain, pursuant to CC0
+   (https://creativecommons.org/publicdomain/zero/1.0/)
+*/
+
+#ifndef _SYS_SDT_H
+#define _SYS_SDT_H    1
+
+/*
+  This file defines a family of macros
+
+       STAP_PROBEn(op1, ..., opn)
+
+  that emit a nop into the instruction stream, and some data into an auxiliary
+  note section.  The data in the note section describes the operands, in terms
+  of size and location.  Each location is encoded as assembler operand string.
+  Consumer tools such as gdb or systemtap insert breakpoints on top of
+  the nop, and decode the location operand-strings, like an assembler,
+  to find the values being passed.
+
+  The operand strings are selected by the compiler for each operand.
+  They are constrained by gcc inline-assembler codes.  The default is:
+
+  #define STAP_SDT_ARG_CONSTRAINT nor
+
+  This is a good default if the operands tend to be integral and
+  moderate in number (smaller than number of registers).  In other
+  cases, the compiler may report "'asm' requires impossible reload" or
+  similar.  In this case, consider simplifying the macro call (fewer
+  and simpler operands), reduce optimization, or override the default
+  constraints string via:
+
+  #define STAP_SDT_ARG_CONSTRAINT g
+  #include <sys/sdt.h>
+
+  See also:
+  https://sourceware.org/systemtap/wiki/UserSpaceProbeImplementation
+  https://gcc.gnu.org/onlinedocs/gcc/Constraints.html
+ */
+
+
+
+#ifdef __ASSEMBLER__
+# define _SDT_PROBE(provider, name, n, arglist)	\
+  _SDT_ASM_BODY(provider, name, _SDT_ASM_SUBSTR_1, (_SDT_DEPAREN_##n arglist)) \
+  _SDT_ASM_BASE
+# define _SDT_ASM_1(x)			x;
+# define _SDT_ASM_2(a, b)		a,b;
+# define _SDT_ASM_3(a, b, c)		a,b,c;
+# define _SDT_ASM_5(a, b, c, d, e)	a,b,c,d,e;
+# define _SDT_ASM_STRING_1(x)		.asciz #x;
+# define _SDT_ASM_SUBSTR_1(x)		.ascii #x;
+# define _SDT_DEPAREN_0()				/* empty */
+# define _SDT_DEPAREN_1(a)				a
+# define _SDT_DEPAREN_2(a,b)				a b
+# define _SDT_DEPAREN_3(a,b,c)				a b c
+# define _SDT_DEPAREN_4(a,b,c,d)			a b c d
+# define _SDT_DEPAREN_5(a,b,c,d,e)			a b c d e
+# define _SDT_DEPAREN_6(a,b,c,d,e,f)			a b c d e f
+# define _SDT_DEPAREN_7(a,b,c,d,e,f,g)			a b c d e f g
+# define _SDT_DEPAREN_8(a,b,c,d,e,f,g,h)		a b c d e f g h
+# define _SDT_DEPAREN_9(a,b,c,d,e,f,g,h,i)		a b c d e f g h i
+# define _SDT_DEPAREN_10(a,b,c,d,e,f,g,h,i,j)		a b c d e f g h i j
+# define _SDT_DEPAREN_11(a,b,c,d,e,f,g,h,i,j,k)		a b c d e f g h i j k
+# define _SDT_DEPAREN_12(a,b,c,d,e,f,g,h,i,j,k,l)	a b c d e f g h i j k l
+#else
+#if defined _SDT_HAS_SEMAPHORES
+#define _SDT_NOTE_SEMAPHORE_USE(provider, name) \
+  __asm__ __volatile__ ("" :: "m" (provider##_##name##_semaphore));
+#else
+#define _SDT_NOTE_SEMAPHORE_USE(provider, name)
+#endif
+
+# define _SDT_PROBE(provider, name, n, arglist) \
+  do {									    \
+    _SDT_NOTE_SEMAPHORE_USE(provider, name); \
+    __asm__ __volatile__ (_SDT_ASM_BODY(provider, name, _SDT_ASM_ARGS, (n)) \
+			  :: _SDT_ASM_OPERANDS_##n arglist);		    \
+    __asm__ __volatile__ (_SDT_ASM_BASE);				    \
+  } while (0)
+# define _SDT_S(x)			#x
+# define _SDT_ASM_1(x)			_SDT_S(x) "\n"
+# define _SDT_ASM_2(a, b)		_SDT_S(a) "," _SDT_S(b) "\n"
+# define _SDT_ASM_3(a, b, c)		_SDT_S(a) "," _SDT_S(b) "," \
+					_SDT_S(c) "\n"
+# define _SDT_ASM_5(a, b, c, d, e)	_SDT_S(a) "," _SDT_S(b) "," \
+					_SDT_S(c) "," _SDT_S(d) "," \
+					_SDT_S(e) "\n"
+# define _SDT_ASM_ARGS(n)		_SDT_ASM_TEMPLATE_##n
+# define _SDT_ASM_STRING_1(x)		_SDT_ASM_1(.asciz #x)
+# define _SDT_ASM_SUBSTR_1(x)		_SDT_ASM_1(.ascii #x)
+
+# define _SDT_ARGFMT(no)                _SDT_ASM_1(_SDT_SIGN %n[_SDT_S##no]) \
+                                        _SDT_ASM_1(_SDT_SIZE %n[_SDT_S##no]) \
+                                        _SDT_ASM_1(_SDT_TYPE %n[_SDT_S##no]) \
+                                        _SDT_ASM_SUBSTR(_SDT_ARGTMPL(_SDT_A##no))
+
+
+# ifndef STAP_SDT_ARG_CONSTRAINT
+# if defined __powerpc__
+# define STAP_SDT_ARG_CONSTRAINT        nZr
+# elif defined __arm__
+# define STAP_SDT_ARG_CONSTRAINT        g
+# else
+# define STAP_SDT_ARG_CONSTRAINT        nor
+# endif
+# endif
+
+# define _SDT_STRINGIFY(x)              #x
+# define _SDT_ARG_CONSTRAINT_STRING(x)  _SDT_STRINGIFY(x)
+/* _SDT_S encodes the size and type as 0xSSTT which is decoded by the assembler
+   macros _SDT_SIZE and _SDT_TYPE */
+# define _SDT_ARG(n, x)				    \
+  [_SDT_S##n] "n" ((_SDT_ARGSIGNED (x) ? (int)-1 : 1) * (-(((int) _SDT_ARGSIZE (x)) << 8) + (-(0x7f & __builtin_classify_type (x))))), \
+  [_SDT_A##n] _SDT_ARG_CONSTRAINT_STRING (STAP_SDT_ARG_CONSTRAINT) (_SDT_ARGVAL (x))
+#endif
+#define _SDT_ASM_STRING(x)		_SDT_ASM_STRING_1(x)
+#define _SDT_ASM_SUBSTR(x)		_SDT_ASM_SUBSTR_1(x)
+
+#define _SDT_ARGARRAY(x)	(__builtin_classify_type (x) == 14	\
+				 || __builtin_classify_type (x) == 5)
+
+#ifdef __cplusplus
+# define _SDT_ARGSIGNED(x)	(!_SDT_ARGARRAY (x) \
+				 && __sdt_type<__typeof (x)>::__sdt_signed)
+# define _SDT_ARGSIZE(x)	(_SDT_ARGARRAY (x) \
+				 ? sizeof (void *) : sizeof (x))
+# define _SDT_ARGVAL(x)		(x)
+
+# include <cstddef>
+
+template<typename __sdt_T>
+struct __sdt_type
+{
+  static const bool __sdt_signed = false;
+};
+  
+#define __SDT_ALWAYS_SIGNED(T) \
+template<> struct __sdt_type<T> { static const bool __sdt_signed = true; };
+#define __SDT_COND_SIGNED(T,CT)						\
+template<> struct __sdt_type<T> { static const bool __sdt_signed = ((CT)(-1) < 1); };
+__SDT_ALWAYS_SIGNED(signed char)
+__SDT_ALWAYS_SIGNED(short)
+__SDT_ALWAYS_SIGNED(int)
+__SDT_ALWAYS_SIGNED(long)
+__SDT_ALWAYS_SIGNED(long long)
+__SDT_ALWAYS_SIGNED(volatile signed char)
+__SDT_ALWAYS_SIGNED(volatile short)
+__SDT_ALWAYS_SIGNED(volatile int)
+__SDT_ALWAYS_SIGNED(volatile long)
+__SDT_ALWAYS_SIGNED(volatile long long)
+__SDT_ALWAYS_SIGNED(const signed char)
+__SDT_ALWAYS_SIGNED(const short)
+__SDT_ALWAYS_SIGNED(const int)
+__SDT_ALWAYS_SIGNED(const long)
+__SDT_ALWAYS_SIGNED(const long long)
+__SDT_ALWAYS_SIGNED(const volatile signed char)
+__SDT_ALWAYS_SIGNED(const volatile short)
+__SDT_ALWAYS_SIGNED(const volatile int)
+__SDT_ALWAYS_SIGNED(const volatile long)
+__SDT_ALWAYS_SIGNED(const volatile long long)
+__SDT_COND_SIGNED(char, char)
+__SDT_COND_SIGNED(wchar_t, wchar_t)
+__SDT_COND_SIGNED(volatile char, char)
+__SDT_COND_SIGNED(volatile wchar_t, wchar_t)
+__SDT_COND_SIGNED(const char, char)
+__SDT_COND_SIGNED(const wchar_t, wchar_t)
+__SDT_COND_SIGNED(const volatile char, char)
+__SDT_COND_SIGNED(const volatile wchar_t, wchar_t)
+#if defined (__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 4))
+/* __SDT_COND_SIGNED(char16_t) */
+/* __SDT_COND_SIGNED(char32_t) */
+#endif
+
+template<typename __sdt_E>
+struct __sdt_type<__sdt_E[]> : public __sdt_type<__sdt_E *> {};
+
+template<typename __sdt_E, size_t __sdt_N>
+struct __sdt_type<__sdt_E[__sdt_N]> : public __sdt_type<__sdt_E *> {};
+
+#elif !defined(__ASSEMBLER__)
+__extension__ extern unsigned long long __sdt_unsp;
+# define _SDT_ARGINTTYPE(x)						\
+  __typeof (__builtin_choose_expr (((__builtin_classify_type (x)	\
+				     + 3) & -4) == 4, (x), 0U))
+# define _SDT_ARGSIGNED(x)						\
+  (!__extension__							\
+   (__builtin_constant_p ((((unsigned long long)			\
+			    (_SDT_ARGINTTYPE (x)) __sdt_unsp)		\
+			   & ((unsigned long long)1 << (sizeof (unsigned long long)	\
+				       * __CHAR_BIT__ - 1))) == 0)	\
+    || (_SDT_ARGINTTYPE (x)) -1 > (_SDT_ARGINTTYPE (x)) 0))
+# define _SDT_ARGSIZE(x)	\
+  (_SDT_ARGARRAY (x) ? sizeof (void *) : sizeof (x))
+# define _SDT_ARGVAL(x)		(x)
+#endif
+
+#if defined __powerpc__ || defined __powerpc64__
+# define _SDT_ARGTMPL(id)	%I[id]%[id]
+#elif defined __i386__
+# define _SDT_ARGTMPL(id)	%k[id]  /* gcc.gnu.org/PR80115 sourceware.org/PR24541 */
+#else
+# define _SDT_ARGTMPL(id)	%[id]
+#endif
+
+/* NB: gdb PR24541 highlighted an unspecified corner of the sdt.h
+   operand note format.
+
+   The named register may be a longer or shorter (!) alias for the
+   storage where the value in question is found.  For example, on
+   i386, 64-bit value may be put in register pairs, and the register
+   name stored would identify just one of them.  Previously, gcc was
+   asked to emit the %w[id] (16-bit alias of some registers holding
+   operands), even when a wider 32-bit value was used.
+
+   Bottom line: the byte-width given before the @ sign governs.  If
+   there is a mismatch between that width and that of the named
+   register, then a sys/sdt.h note consumer may need to employ
+   architecture-specific heuristics to figure out where the compiler
+   has actually put the complete value.
+*/
+
+#ifdef __LP64__
+# define _SDT_ASM_ADDR	.8byte
+#else
+# define _SDT_ASM_ADDR	.4byte
+#endif
+
+/* The ia64 and s390 nop instructions take an argument. */
+#if defined(__ia64__) || defined(__s390__) || defined(__s390x__)
+#define _SDT_NOP	nop 0
+#else
+#define _SDT_NOP	nop
+#endif
+
+#define _SDT_NOTE_NAME	"stapsdt"
+#define _SDT_NOTE_TYPE	3
+
+# define _SDT_ASM_AUTOGROUP "?"
+
+#define _SDT_DEF_MACROS							     \
+	_SDT_ASM_1(.altmacro)						     \
+	_SDT_ASM_1(.macro _SDT_SIGN x)				     	     \
+	_SDT_ASM_3(.pushsection .note.stapsdt,"","note")		     \
+	_SDT_ASM_1(.iflt \\x)						     \
+	_SDT_ASM_1(.ascii "-")						     \
+	_SDT_ASM_1(.endif)						     \
+	_SDT_ASM_1(.popsection)						     \
+	_SDT_ASM_1(.endm)						     \
+	_SDT_ASM_1(.macro _SDT_SIZE_ x)					     \
+	_SDT_ASM_3(.pushsection .note.stapsdt,"","note")		     \
+	_SDT_ASM_1(.ascii "\x")						     \
+	_SDT_ASM_1(.popsection)						     \
+	_SDT_ASM_1(.endm)						     \
+	_SDT_ASM_1(.macro _SDT_SIZE x)					     \
+	_SDT_ASM_1(_SDT_SIZE_ %%((-(-\\x*((-\\x>0)-(-\\x<0))))>>8))	     \
+	_SDT_ASM_1(.endm)						     \
+	_SDT_ASM_1(.macro _SDT_TYPE_ x)				             \
+	_SDT_ASM_3(.pushsection .note.stapsdt,"","note")		     \
+	_SDT_ASM_2(.ifc 8,\\x)					     	     \
+	_SDT_ASM_1(.ascii "f")						     \
+	_SDT_ASM_1(.endif)						     \
+	_SDT_ASM_1(.ascii "@")						     \
+	_SDT_ASM_1(.popsection)						     \
+	_SDT_ASM_1(.endm)						     \
+	_SDT_ASM_1(.macro _SDT_TYPE x)				     	     \
+	_SDT_ASM_1(_SDT_TYPE_ %%((\\x)&(0xff)))			     \
+	_SDT_ASM_1(.endm)
+
+#define _SDT_UNDEF_MACROS						      \
+  _SDT_ASM_1(.purgem _SDT_SIGN)						      \
+  _SDT_ASM_1(.purgem _SDT_SIZE_)					      \
+  _SDT_ASM_1(.purgem _SDT_SIZE)						      \
+  _SDT_ASM_1(.purgem _SDT_TYPE_)					      \
+  _SDT_ASM_1(.purgem _SDT_TYPE)
+
+#define _SDT_ASM_BODY(provider, name, pack_args, args, ...)		      \
+  _SDT_DEF_MACROS							      \
+  _SDT_ASM_1(990:	_SDT_NOP)					      \
+  _SDT_ASM_3(		.pushsection .note.stapsdt,_SDT_ASM_AUTOGROUP,"note") \
+  _SDT_ASM_1(		.balign 4)					      \
+  _SDT_ASM_3(		.4byte 992f-991f, 994f-993f, _SDT_NOTE_TYPE)	      \
+  _SDT_ASM_1(991:	.asciz _SDT_NOTE_NAME)				      \
+  _SDT_ASM_1(992:	.balign 4)					      \
+  _SDT_ASM_1(993:	_SDT_ASM_ADDR 990b)				      \
+  _SDT_ASM_1(		_SDT_ASM_ADDR _.stapsdt.base)			      \
+  _SDT_SEMAPHORE(provider,name)						      \
+  _SDT_ASM_STRING(provider)						      \
+  _SDT_ASM_STRING(name)							      \
+  pack_args args							      \
+  _SDT_ASM_SUBSTR(\x00)							      \
+  _SDT_UNDEF_MACROS							      \
+  _SDT_ASM_1(994:	.balign 4)					      \
+  _SDT_ASM_1(		.popsection)
+
+#define _SDT_ASM_BASE							      \
+  _SDT_ASM_1(.ifndef _.stapsdt.base)					      \
+  _SDT_ASM_5(		.pushsection .stapsdt.base,"aG","progbits",	      \
+							.stapsdt.base,comdat) \
+  _SDT_ASM_1(		.weak _.stapsdt.base)				      \
+  _SDT_ASM_1(		.hidden _.stapsdt.base)				      \
+  _SDT_ASM_1(	_.stapsdt.base: .space 1)				      \
+  _SDT_ASM_2(		.size _.stapsdt.base, 1)			      \
+  _SDT_ASM_1(		.popsection)					      \
+  _SDT_ASM_1(.endif)
+
+#if defined _SDT_HAS_SEMAPHORES
+#define _SDT_SEMAPHORE(p,n) \
+	_SDT_ASM_1(		_SDT_ASM_ADDR p##_##n##_semaphore)
+#else
+#define _SDT_SEMAPHORE(p,n) _SDT_ASM_1(		_SDT_ASM_ADDR 0)
+#endif
+
+#define _SDT_ASM_BLANK _SDT_ASM_SUBSTR(\x20)
+#define _SDT_ASM_TEMPLATE_0		/* no arguments */
+#define _SDT_ASM_TEMPLATE_1		_SDT_ARGFMT(1)
+#define _SDT_ASM_TEMPLATE_2		_SDT_ASM_TEMPLATE_1 _SDT_ASM_BLANK _SDT_ARGFMT(2)
+#define _SDT_ASM_TEMPLATE_3		_SDT_ASM_TEMPLATE_2 _SDT_ASM_BLANK _SDT_ARGFMT(3)
+#define _SDT_ASM_TEMPLATE_4		_SDT_ASM_TEMPLATE_3 _SDT_ASM_BLANK _SDT_ARGFMT(4)
+#define _SDT_ASM_TEMPLATE_5		_SDT_ASM_TEMPLATE_4 _SDT_ASM_BLANK _SDT_ARGFMT(5)
+#define _SDT_ASM_TEMPLATE_6		_SDT_ASM_TEMPLATE_5 _SDT_ASM_BLANK _SDT_ARGFMT(6)
+#define _SDT_ASM_TEMPLATE_7		_SDT_ASM_TEMPLATE_6 _SDT_ASM_BLANK _SDT_ARGFMT(7)
+#define _SDT_ASM_TEMPLATE_8		_SDT_ASM_TEMPLATE_7 _SDT_ASM_BLANK _SDT_ARGFMT(8)
+#define _SDT_ASM_TEMPLATE_9		_SDT_ASM_TEMPLATE_8 _SDT_ASM_BLANK _SDT_ARGFMT(9)
+#define _SDT_ASM_TEMPLATE_10		_SDT_ASM_TEMPLATE_9 _SDT_ASM_BLANK _SDT_ARGFMT(10)
+#define _SDT_ASM_TEMPLATE_11		_SDT_ASM_TEMPLATE_10 _SDT_ASM_BLANK _SDT_ARGFMT(11)
+#define _SDT_ASM_TEMPLATE_12		_SDT_ASM_TEMPLATE_11 _SDT_ASM_BLANK _SDT_ARGFMT(12)
+#define _SDT_ASM_OPERANDS_0()		[__sdt_dummy] "g" (0)
+#define _SDT_ASM_OPERANDS_1(arg1)	_SDT_ARG(1, arg1)
+#define _SDT_ASM_OPERANDS_2(arg1, arg2) \
+  _SDT_ASM_OPERANDS_1(arg1), _SDT_ARG(2, arg2)
+#define _SDT_ASM_OPERANDS_3(arg1, arg2, arg3) \
+  _SDT_ASM_OPERANDS_2(arg1, arg2), _SDT_ARG(3, arg3)
+#define _SDT_ASM_OPERANDS_4(arg1, arg2, arg3, arg4) \
+  _SDT_ASM_OPERANDS_3(arg1, arg2, arg3), _SDT_ARG(4, arg4)
+#define _SDT_ASM_OPERANDS_5(arg1, arg2, arg3, arg4, arg5) \
+  _SDT_ASM_OPERANDS_4(arg1, arg2, arg3, arg4), _SDT_ARG(5, arg5)
+#define _SDT_ASM_OPERANDS_6(arg1, arg2, arg3, arg4, arg5, arg6) \
+  _SDT_ASM_OPERANDS_5(arg1, arg2, arg3, arg4, arg5), _SDT_ARG(6, arg6)
+#define _SDT_ASM_OPERANDS_7(arg1, arg2, arg3, arg4, arg5, arg6, arg7) \
+  _SDT_ASM_OPERANDS_6(arg1, arg2, arg3, arg4, arg5, arg6), _SDT_ARG(7, arg7)
+#define _SDT_ASM_OPERANDS_8(arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8) \
+  _SDT_ASM_OPERANDS_7(arg1, arg2, arg3, arg4, arg5, arg6, arg7), \
+    _SDT_ARG(8, arg8)
+#define _SDT_ASM_OPERANDS_9(arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9) \
+  _SDT_ASM_OPERANDS_8(arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8), \
+    _SDT_ARG(9, arg9)
+#define _SDT_ASM_OPERANDS_10(arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10) \
+  _SDT_ASM_OPERANDS_9(arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9), \
+    _SDT_ARG(10, arg10)
+#define _SDT_ASM_OPERANDS_11(arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10,arg11) \
+  _SDT_ASM_OPERANDS_10(arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10), \
+    _SDT_ARG(11, arg11)
+#define _SDT_ASM_OPERANDS_12(arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10,arg11,arg12) \
+  _SDT_ASM_OPERANDS_11(arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10, arg11), \
+    _SDT_ARG(12, arg12)
+
+/* These macros can be used in C, C++, or assembly code.
+   In assembly code the arguments should use normal assembly operand syntax.  */
+
+#define STAP_PROBE(provider, name) \
+  _SDT_PROBE(provider, name, 0, ())
+#define STAP_PROBE1(provider, name, arg1) \
+  _SDT_PROBE(provider, name, 1, (arg1))
+#define STAP_PROBE2(provider, name, arg1, arg2) \
+  _SDT_PROBE(provider, name, 2, (arg1, arg2))
+#define STAP_PROBE3(provider, name, arg1, arg2, arg3) \
+  _SDT_PROBE(provider, name, 3, (arg1, arg2, arg3))
+#define STAP_PROBE4(provider, name, arg1, arg2, arg3, arg4) \
+  _SDT_PROBE(provider, name, 4, (arg1, arg2, arg3, arg4))
+#define STAP_PROBE5(provider, name, arg1, arg2, arg3, arg4, arg5) \
+  _SDT_PROBE(provider, name, 5, (arg1, arg2, arg3, arg4, arg5))
+#define STAP_PROBE6(provider, name, arg1, arg2, arg3, arg4, arg5, arg6)	\
+  _SDT_PROBE(provider, name, 6, (arg1, arg2, arg3, arg4, arg5, arg6))
+#define STAP_PROBE7(provider, name, arg1, arg2, arg3, arg4, arg5, arg6, arg7) \
+  _SDT_PROBE(provider, name, 7, (arg1, arg2, arg3, arg4, arg5, arg6, arg7))
+#define STAP_PROBE8(provider,name,arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8) \
+  _SDT_PROBE(provider, name, 8, (arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8))
+#define STAP_PROBE9(provider,name,arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9)\
+  _SDT_PROBE(provider, name, 9, (arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9))
+#define STAP_PROBE10(provider,name,arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10) \
+  _SDT_PROBE(provider, name, 10, \
+	     (arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10))
+#define STAP_PROBE11(provider,name,arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10,arg11) \
+  _SDT_PROBE(provider, name, 11, \
+	     (arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10,arg11))
+#define STAP_PROBE12(provider,name,arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10,arg11,arg12) \
+  _SDT_PROBE(provider, name, 12, \
+	     (arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10,arg11,arg12))
+
+/* This STAP_PROBEV macro can be used in variadic scenarios, where the
+   number of probe arguments is not known until compile time.  Since
+   variadic macro support may vary with compiler options, you must
+   pre-#define SDT_USE_VARIADIC to enable this type of probe.
+
+   The trick to count __VA_ARGS__ was inspired by this post by
+   Laurent Deniau <laurent.deniau@cern.ch>:
+       http://groups.google.com/group/comp.std.c/msg/346fc464319b1ee5
+
+   Note that our _SDT_NARG is called with an extra 0 arg that's not
+   counted, so we don't have to worry about the behavior of macros
+   called without any arguments.  */
+
+#define _SDT_NARG(...) __SDT_NARG(__VA_ARGS__, 12,11,10,9,8,7,6,5,4,3,2,1,0)
+#define __SDT_NARG(_0,_1,_2,_3,_4,_5,_6,_7,_8,_9,_10,_11,_12, N, ...) N
+#ifdef SDT_USE_VARIADIC
+#define _SDT_PROBE_N(provider, name, N, ...) \
+  _SDT_PROBE(provider, name, N, (__VA_ARGS__))
+#define STAP_PROBEV(provider, name, ...) \
+  _SDT_PROBE_N(provider, name, _SDT_NARG(0, ##__VA_ARGS__), ##__VA_ARGS__)
+#endif
+
+/* These macros are for use in asm statements.  You must compile
+   with -std=gnu99 or -std=c99 to use the STAP_PROBE_ASM macro.
+
+   The STAP_PROBE_ASM macro generates a quoted string to be used in the
+   template portion of the asm statement, concatenated with strings that
+   contain the actual assembly code around the probe site.
+
+   For example:
+
+	asm ("before\n"
+	     STAP_PROBE_ASM(provider, fooprobe, %eax 4(%esi))
+	     "after");
+
+   emits the assembly code for "before\nafter", with a probe in between.
+   The probe arguments are the %eax register, and the value of the memory
+   word located 4 bytes past the address in the %esi register.  Note that
+   because this is a simple asm, not a GNU C extended asm statement, these
+   % characters do not need to be doubled to generate literal %reg names.
+
+   In a GNU C extended asm statement, the probe arguments can be specified
+   using the macro STAP_PROBE_ASM_TEMPLATE(n) for n arguments.  The paired
+   macro STAP_PROBE_ASM_OPERANDS gives the C values of these probe arguments,
+   and appears in the input operand list of the asm statement.  For example:
+
+	asm ("someinsn %0,%1\n" // %0 is output operand, %1 is input operand
+	     STAP_PROBE_ASM(provider, fooprobe, STAP_PROBE_ASM_TEMPLATE(3))
+	     "otherinsn %[namedarg]"
+	     : "r" (outvar)
+	     : "g" (some_value), [namedarg] "i" (1234),
+	       STAP_PROBE_ASM_OPERANDS(3, some_value, some_ptr->field, 1234));
+
+    This is just like writing:
+
+	STAP_PROBE3(provider, fooprobe, some_value, some_ptr->field, 1234));
+
+    but the probe site is right between "someinsn" and "otherinsn".
+
+    The probe arguments in STAP_PROBE_ASM can be given as assembly
+    operands instead, even inside a GNU C extended asm statement.
+    Note that these can use operand templates like %0 or %[name],
+    and likewise they must write %%reg for a literal operand of %reg.  */
+
+#define _SDT_ASM_BODY_1(p,n,...) _SDT_ASM_BODY(p,n,_SDT_ASM_SUBSTR,(__VA_ARGS__))
+#define _SDT_ASM_BODY_2(p,n,...) _SDT_ASM_BODY(p,n,/*_SDT_ASM_STRING */,__VA_ARGS__)
+#define _SDT_ASM_BODY_N2(p,n,no,...) _SDT_ASM_BODY_ ## no(p,n,__VA_ARGS__)
+#define _SDT_ASM_BODY_N1(p,n,no,...) _SDT_ASM_BODY_N2(p,n,no,__VA_ARGS__)
+#define _SDT_ASM_BODY_N(p,n,...) _SDT_ASM_BODY_N1(p,n,_SDT_NARG(0, __VA_ARGS__),__VA_ARGS__)
+
+#if __STDC_VERSION__ >= 199901L
+# define STAP_PROBE_ASM(provider, name, ...)		\
+  _SDT_ASM_BODY_N(provider, name, __VA_ARGS__)					\
+  _SDT_ASM_BASE
+# define STAP_PROBE_ASM_OPERANDS(n, ...) _SDT_ASM_OPERANDS_##n(__VA_ARGS__)
+#else
+# define STAP_PROBE_ASM(provider, name, args)	\
+  _SDT_ASM_BODY(provider, name, /* _SDT_ASM_STRING */, (args))	\
+  _SDT_ASM_BASE
+#endif
+#define STAP_PROBE_ASM_TEMPLATE(n) _SDT_ASM_TEMPLATE_##n,"use _SDT_ASM_TEMPLATE_"
+
+
+/* DTrace compatible macro names.  */
+#define DTRACE_PROBE(provider,probe)		\
+  STAP_PROBE(provider,probe)
+#define DTRACE_PROBE1(provider,probe,parm1)	\
+  STAP_PROBE1(provider,probe,parm1)
+#define DTRACE_PROBE2(provider,probe,parm1,parm2)	\
+  STAP_PROBE2(provider,probe,parm1,parm2)
+#define DTRACE_PROBE3(provider,probe,parm1,parm2,parm3) \
+  STAP_PROBE3(provider,probe,parm1,parm2,parm3)
+#define DTRACE_PROBE4(provider,probe,parm1,parm2,parm3,parm4)	\
+  STAP_PROBE4(provider,probe,parm1,parm2,parm3,parm4)
+#define DTRACE_PROBE5(provider,probe,parm1,parm2,parm3,parm4,parm5)	\
+  STAP_PROBE5(provider,probe,parm1,parm2,parm3,parm4,parm5)
+#define DTRACE_PROBE6(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6) \
+  STAP_PROBE6(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6)
+#define DTRACE_PROBE7(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6,parm7) \
+  STAP_PROBE7(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6,parm7)
+#define DTRACE_PROBE8(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6,parm7,parm8) \
+  STAP_PROBE8(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6,parm7,parm8)
+#define DTRACE_PROBE9(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6,parm7,parm8,parm9) \
+  STAP_PROBE9(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6,parm7,parm8,parm9)
+#define DTRACE_PROBE10(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6,parm7,parm8,parm9,parm10) \
+  STAP_PROBE10(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6,parm7,parm8,parm9,parm10)
+#define DTRACE_PROBE11(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6,parm7,parm8,parm9,parm10,parm11) \
+  STAP_PROBE11(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6,parm7,parm8,parm9,parm10,parm11)
+#define DTRACE_PROBE12(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6,parm7,parm8,parm9,parm10,parm11,parm12) \
+  STAP_PROBE12(provider,probe,parm1,parm2,parm3,parm4,parm5,parm6,parm7,parm8,parm9,parm10,parm11,parm12)
+
+
+#endif /* sys/sdt.h */
diff --git a/test/unittest/usdt/tst.usdt-notes.r b/test/unittest/usdt/tst.usdt-notes.r
new file mode 100644
index 00000000..db6d18cb
--- /dev/null
+++ b/test/unittest/usdt/tst.usdt-notes.r
@@ -0,0 +1,14 @@
+test:main:zero
+test:main:one:1
+test:main:two:2:3
+test:main:three:4:5:7
+test:main:four:7:8:9:10
+test:main:five:11:12:13:14:15
+test:main:six:16:17:18:19:20:21
+test:main:seven:22:23:24:25:26:27:28
+test:main:eight:29:30:31:32:33:34:35:36
+test:main:nine:37:38:39:40:41:42:43:44:45
+test:main:ten:46:47:48:49:50:51:52:53:54:55
+test:main:eleven:56:57:58:59:60:61:62:63:64:65
+test:main:twelve:67:68:69:70:71:72:73:74:75:76
+
diff --git a/test/unittest/usdt/tst.usdt-notes.sh b/test/unittest/usdt/tst.usdt-notes.sh
new file mode 100755
index 00000000..364ba8db
--- /dev/null
+++ b/test/unittest/usdt/tst.usdt-notes.sh
@@ -0,0 +1,121 @@
+#!/bin/bash
+#
+# Oracle Linux DTrace.
+# Copyright (c) 2024, Oracle and/or its affiliates. All rights reserved.
+# Licensed under the Universal Permissive License v 1.0 as shown at
+# http://oss.oracle.com/licenses/upl.
+
+# This test covers all USDT probes fired by the DTRACE_PROBEn macros.
+# Arguments values are checked only for first 10 arguments because
+# there is support for arg0 ... arg9 only at this moment.
+
+if [ $# != 1 ]; then
+	echo expected one argument: '<'dtrace-path'>'
+	exit 2
+fi
+
+dtrace=$1
+CC=/usr/bin/gcc
+CFLAGS="-I${PWD}/test/unittest/usdt"
+
+DIRNAME="$tmpdir/usdt-notes.$$.$RANDOM"
+mkdir -p $DIRNAME
+cd $DIRNAME
+
+cat > test.c <<EOF
+#include <sdt_notes.h>
+
+int
+main(int argc, char **argv)
+{
+	DTRACE_PROBE(test_prov, zero);
+	DTRACE_PROBE1(test_prov, one, argc);
+	DTRACE_PROBE2(test_prov, two, 2, 3);
+	DTRACE_PROBE3(test_prov, three, 4, 5, 7);
+	DTRACE_PROBE4(test_prov, four, 7, 8, 9, 10);
+	DTRACE_PROBE5(test_prov, five, 11, 12, 13, 14, 15);
+	DTRACE_PROBE6(test_prov, six, 16, 17, 18, 19, 20, 21);
+	DTRACE_PROBE7(test_prov, seven, 22, 23, 24, 25, 26, 27, 28);
+	DTRACE_PROBE8(test_prov, eight, 29, 30, 31, 32, 33, 34, 35, 36);
+	DTRACE_PROBE9(test_prov, nine, 37, 38, 39, 40, 41, 42, 43, 44, 45);
+	DTRACE_PROBE10(test_prov, ten, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55);
+	DTRACE_PROBE11(test_prov, eleven, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66);
+	DTRACE_PROBE12(test_prov, twelve, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78);
+}
+EOF
+
+${CC} ${CFLAGS} -o test test.c
+if [ $? -ne 0 ]; then
+	echo "failed to compile test.c" >& 2
+	exit 1
+fi
+
+$dtrace -c ./test -qs /dev/stdin <<EOF
+test_prov\$target:::zero
+{
+	printf("%s:%s:%s\n", probemod, probefunc, probename);
+}
+
+test_prov\$target:::one
+{
+	printf("%s:%s:%s:%li\n", probemod, probefunc, probename, arg0);
+}
+
+test_prov\$target:::two
+{
+	printf("%s:%s:%s:%li:%li\n", probemod, probefunc, probename, arg0, arg1);
+}
+
+test_prov\$target:::three
+{
+	printf("%s:%s:%s:%li:%li:%li\n", probemod, probefunc, probename, arg0, arg1,
+	       arg2);
+}
+
+test_prov\$target:::four
+{
+	printf("%s:%s:%s:%li:%li:%li:%li\n", probemod, probefunc, probename, arg0, arg1,
+	       arg2, arg3);
+}
+
+test_prov\$target:::five
+{
+	printf("%s:%s:%s:%li:%li:%li:%li:%li\n", probemod, probefunc, probename,
+	       arg0, arg1, arg2, arg3, arg4);
+}
+
+test_prov\$target:::six
+{
+	printf("%s:%s:%s:%li:%li:%li:%li:%li:%li\n", probemod, probefunc, probename,
+	       arg0, arg1, arg2, arg3, arg4, arg5);
+}
+
+test_prov\$target:::seven
+{
+	printf("%s:%s:%s:%li:%li:%li:%li:%li:%li:%li\n", probemod, probefunc, probename,
+	       arg0, arg1, arg2, arg3, arg4, arg5, arg6);
+}
+
+test_prov\$target:::eight
+{
+	printf("%s:%s:%s:%li:%li:%li:%li:%li:%li:%li:%li\n", probemod, probefunc, probename,
+	       arg0, arg1, arg2, arg3, arg4, arg5, arg6, arg7);
+}
+
+test_prov\$target:::nine
+{
+	printf("%s:%s:%s:%li:%li:%li:%li:%li:%li:%li:%li:%li\n", probemod, probefunc, probename,
+	       arg0, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8);
+}
+
+test_prov\$target:::ten,
+test_prov\$target:::eleven,
+test_prov\$target:::twelve
+{
+	printf("%s:%s:%s:%li:%li:%li:%li:%li:%li:%li:%li:%li:%li\n", probemod, probefunc, probename,
+	       arg0, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9);
+}
+EOF
+status=$?
+
+exit $status
-- 
2.43.5

[PATCH v2 3/4] selftests/usdt: add test for USDT notes in shared library

[Alan Maguire]

To ensure USDT notes are found/fire for shared libraries, create
a shared library and trace the USDT probes in it.  To ensure the
library is loaded when DTrace has started, call it early in the
program lifetime and sleep until DTrace starts; at that point
trace the probes with the pid and ensure they fire with expected args.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 test/unittest/usdt/tst.usdt-notes-lib.r  |  14 +++
 test/unittest/usdt/tst.usdt-notes-lib.sh | 145 +++++++++++++++++++++++
 2 files changed, 159 insertions(+)
 create mode 100644 test/unittest/usdt/tst.usdt-notes-lib.r
 create mode 100755 test/unittest/usdt/tst.usdt-notes-lib.sh

diff --git a/test/unittest/usdt/tst.usdt-notes-lib.r b/test/unittest/usdt/tst.usdt-notes-lib.r
new file mode 100644
index 00000000..cab7b6d5
--- /dev/null
+++ b/test/unittest/usdt/tst.usdt-notes-lib.r
@@ -0,0 +1,14 @@
+libusdttest.so:libfn:zero
+libusdttest.so:libfn:one:1
+libusdttest.so:libfn:two:2:3
+libusdttest.so:libfn:three:4:5:7
+libusdttest.so:libfn:four:7:8:9:10
+libusdttest.so:libfn:five:11:12:13:14:15
+libusdttest.so:libfn:six:16:17:18:19:20:21
+libusdttest.so:libfn:seven:22:23:24:25:26:27:28
+libusdttest.so:libfn:eight:29:30:31:32:33:34:35:36
+libusdttest.so:libfn:nine:37:38:39:40:41:42:43:44:45
+libusdttest.so:libfn:ten:46:47:48:49:50:51:52:53:54:55
+libusdttest.so:libfn:eleven:56:57:58:59:60:61:62:63:64:65
+libusdttest.so:libfn:twelve:67:68:69:70:71:72:73:74:75:76
+
diff --git a/test/unittest/usdt/tst.usdt-notes-lib.sh b/test/unittest/usdt/tst.usdt-notes-lib.sh
new file mode 100755
index 00000000..ed87e9f5
--- /dev/null
+++ b/test/unittest/usdt/tst.usdt-notes-lib.sh
@@ -0,0 +1,145 @@
+#!/bin/bash
+#
+# Oracle Linux DTrace.
+# Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+# Licensed under the Universal Permissive License v 1.0 as shown at
+# http://oss.oracle.com/licenses/upl.
+
+# This test covers all USDT probes fired by the DTRACE_PROBEn macros.
+# Arguments values are checked only for first 10 arguments because
+# there is support for arg0 ... arg9 only at this moment.
+
+if [ $# != 1 ]; then
+	echo expected one argument: '<'dtrace-path'>'
+	exit 2
+fi
+
+dtrace=$1
+CC=/usr/bin/gcc
+CFLAGS="-I${PWD}/test/unittest/usdt"
+
+DIRNAME="$tmpdir/usdt-notes.$$.$RANDOM"
+mkdir -p $DIRNAME
+cd $DIRNAME
+
+cat > libusdttest.c <<EOF
+#include <sdt_notes.h>
+
+void
+libfn(int argc, char **argv)
+{
+	if (argc == 0 && argv == 0)
+		return;
+	DTRACE_PROBE(test_prov, zero);
+	DTRACE_PROBE1(test_prov, one, argc);
+	DTRACE_PROBE2(test_prov, two, 2, 3);
+	DTRACE_PROBE3(test_prov, three, 4, 5, 7);
+	DTRACE_PROBE4(test_prov, four, 7, 8, 9, 10);
+	DTRACE_PROBE5(test_prov, five, 11, 12, 13, 14, 15);
+	DTRACE_PROBE6(test_prov, six, 16, 17, 18, 19, 20, 21);
+	DTRACE_PROBE7(test_prov, seven, 22, 23, 24, 25, 26, 27, 28);
+	DTRACE_PROBE8(test_prov, eight, 29, 30, 31, 32, 33, 34, 35, 36);
+	DTRACE_PROBE9(test_prov, nine, 37, 38, 39, 40, 41, 42, 43, 44, 45);
+	DTRACE_PROBE10(test_prov, ten, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55);
+	DTRACE_PROBE11(test_prov, eleven, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66);
+	DTRACE_PROBE12(test_prov, twelve, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78);
+}
+EOF
+
+cat > test.c <<EOF
+
+#include <unistd.h>
+
+extern void libfn(int argc, char **argv);
+
+int
+main(int argc, char **argv)
+{
+	libfn(0, 0);
+	sleep(10);
+	libfn(argc, argv);
+	return 0;
+}
+EOF
+
+${CC} ${CFLAGS} -c -fpic libusdttest.c
+${CC} -shared -o libusdttest.so libusdttest.o
+${CC} -L. ${CFLAGS} -o test test.c -lusdttest
+if [ $? -ne 0 ]; then
+	echo "failed to compile test.c" >& 2
+	exit 1
+fi
+
+export LD_LIBRARY_PATH=.:${LD_LIBRARY_PATH}
+LD_LIBRARY_PATH=.:${LD_LIBRARY_PATH} ./test &
+PID=$!
+sleep 5
+$dtrace -p $PID -qs /dev/stdin <<EOF
+test_prov\$target:::zero
+{
+	printf("%s:%s:%s\n", probemod, probefunc, probename);
+}
+
+test_prov\$target:::one
+{
+	printf("%s:%s:%s:%li\n", probemod, probefunc, probename, arg0);
+}
+
+test_prov\$target:::two
+{
+	printf("%s:%s:%s:%li:%li\n", probemod, probefunc, probename, arg0, arg1);
+}
+
+test_prov\$target:::three
+{
+	printf("%s:%s:%s:%li:%li:%li\n", probemod, probefunc, probename, arg0, arg1,
+	       arg2);
+}
+
+test_prov\$target:::four
+{
+	printf("%s:%s:%s:%li:%li:%li:%li\n", probemod, probefunc, probename, arg0, arg1,
+	       arg2, arg3);
+}
+
+test_prov\$target:::five
+{
+	printf("%s:%s:%s:%li:%li:%li:%li:%li\n", probemod, probefunc, probename,
+	       arg0, arg1, arg2, arg3, arg4);
+}
+
+test_prov\$target:::six
+{
+	printf("%s:%s:%s:%li:%li:%li:%li:%li:%li\n", probemod, probefunc, probename,
+	       arg0, arg1, arg2, arg3, arg4, arg5);
+}
+
+test_prov\$target:::seven
+{
+	printf("%s:%s:%s:%li:%li:%li:%li:%li:%li:%li\n", probemod, probefunc, probename,
+	       arg0, arg1, arg2, arg3, arg4, arg5, arg6);
+}
+
+test_prov\$target:::eight
+{
+	printf("%s:%s:%s:%li:%li:%li:%li:%li:%li:%li:%li\n", probemod, probefunc, probename,
+	       arg0, arg1, arg2, arg3, arg4, arg5, arg6, arg7);
+}
+
+test_prov\$target:::nine
+{
+	printf("%s:%s:%s:%li:%li:%li:%li:%li:%li:%li:%li:%li\n", probemod, probefunc, probename,
+	       arg0, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8);
+}
+
+test_prov\$target:::ten,
+test_prov\$target:::eleven,
+test_prov\$target:::twelve
+{
+	printf("%s:%s:%s:%li:%li:%li:%li:%li:%li:%li:%li:%li:%li\n", probemod, probefunc, probename,
+	       arg0, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9);
+}
+EOF
+status=$?
+
+exit $status
-- 
2.43.5

[PATCH v2 4/4] selftests/usdt: add test covering different forms of USDT note args

[Alan Maguire]

Add a test exercising various arg types supported by USDT notes;
register values, register + offset and constants.  The test generates
a binary with probes represented as follows on x86_64:

Displaying notes found in: .note.stapsdt
  Owner                 Data size       Description
  stapsdt              0x00000048       NT_STAPSDT (SystemTap probe descriptors)
    Provider: test_prov
    Name: args
    Location: 0x0000000000400557, Base: 0x00000000004005f8, Semaphore: 0x0000000000000000
    Arguments: -4@-4(%rbp) 8@%rax 8@%rdx -4@$18

Verify we get expected data for the probe arguments.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 test/unittest/usdt/tst.usdt-notes-args.r  |  2 +
 test/unittest/usdt/tst.usdt-notes-args.sh | 51 +++++++++++++++++++++++
 2 files changed, 53 insertions(+)
 create mode 100644 test/unittest/usdt/tst.usdt-notes-args.r
 create mode 100755 test/unittest/usdt/tst.usdt-notes-args.sh

diff --git a/test/unittest/usdt/tst.usdt-notes-args.r b/test/unittest/usdt/tst.usdt-notes-args.r
new file mode 100644
index 00000000..42bca19f
--- /dev/null
+++ b/test/unittest/usdt/tst.usdt-notes-args.r
@@ -0,0 +1,2 @@
+test:main:args:2:./test:val:18
+
diff --git a/test/unittest/usdt/tst.usdt-notes-args.sh b/test/unittest/usdt/tst.usdt-notes-args.sh
new file mode 100755
index 00000000..7c8ea37b
--- /dev/null
+++ b/test/unittest/usdt/tst.usdt-notes-args.sh
@@ -0,0 +1,51 @@
+#!/bin/bash
+#
+# Oracle Linux DTrace.
+# Copyright (c) 2024, Oracle and/or its affiliates. All rights reserved.
+# Licensed under the Universal Permissive License v 1.0 as shown at
+# http://oss.oracle.com/licenses/upl.
+
+# This test covers all USDT probes fired by the DTRACE_PROBEn macros.
+# Arguments values are checked only for first 10 arguments because
+# there is support for arg0 ... arg9 only at this moment.
+
+if [ $# != 1 ]; then
+	echo expected one argument: '<'dtrace-path'>'
+	exit 2
+fi
+
+dtrace=$1
+CC=/usr/bin/gcc
+CFLAGS="-I${PWD}/test/unittest/usdt"
+
+DIRNAME="$tmpdir/usdt-notes.$$.$RANDOM"
+mkdir -p $DIRNAME
+cd $DIRNAME
+
+cat > test.c <<EOF
+#include <sdt_notes.h>
+
+int
+main(int argc, char **argv)
+{
+	DTRACE_PROBE4(test_prov, args, argc, argv[0], argv[1] + 4, 18);
+}
+EOF
+
+${CC} ${CFLAGS} -o test test.c
+if [ $? -ne 0 ]; then
+	echo "failed to compile test.c" >& 2
+	exit 1
+fi
+
+$dtrace -c './test arg1val' -qs /dev/stdin <<EOF
+test_prov\$target:::args
+{
+	printf("%s:%s:%s:%li:%s:%s:%li\n", probemod, probefunc, probename,
+	       arg0, copyinstr(arg1), copyinstr(arg2), arg3);
+}
+
+EOF
+status=$?
+
+exit $status
-- 
2.43.5

Re: [PATCH v2 0/4] ELF note-based USDT support

[Kris Van Hees]

Thank you for rebasing your work on the newest tree.  That will certainly
help review them and move things forward.

I would definitely rework the commit message though, because

1. DTrace has a specific understanding of what USDT probes are and how they
   work and stap-based probes do not provide the same functionality.  One
   example of that you already point ay: they are not discoverable - i.e.
   they are not registered upon startup which is why you need to refer to
   them directly by provider name (with embedded pid).  I think you need to
   be very clear about the distinction.  Using STAPSDT or stapsdt might be
   a better choice than referring to USDT.

2. I think that the commit message fails to highlight that this support is
   to make it possible to trace programs (and shared librearies)that have been 
   built with stap style probes.  I don't think it is in the best interest of
   DTrace users to build their executables and shared libraries with stap
   style probes over DTrace USDT probes, especially given the significant
   advantage that DTrace USDT probes have (see 1. above).

3. While I can see the point of mentioning how to add stapsdt probes to
   code, it also is a source for confusion and thus is probably better left
   out.  Since this is a compability feature, surely those wanting to use it
   already have executables with such probes or know how to create them.
   By including it here, you also introduce the very unfortunate fact that
   stapsdt uses DTRACE_PROBE*() macros even though the probes have never really
   been DTrace compatible, and it is only with your proposed patch now that
   they could be used in DTrace.  The systemtap project shouldn't have
   piggy-backed on DTRACE_PROBE*() in the first place because it causes this
   type of confusion and complications, so I would very much prefer not to
   highlight that mess with this patch series.

The stap probe support is very significant because we unfortunately do have to
live with a world where there are multiple ways that such userspace probes
have been implemented.  And given that packages are released with probes and
people may want to trace them makes this addition certainly very worthwhile.
But I think it should be clear that this is for compatibilty/interoperability
purposes only.

	Kris

On Wed, Jan 29, 2025 at 02:55:18PM +0000, Alan Maguire wrote:
> This series adds support (patch 1) for ELF-note defined USDT
> probes in binaries and libraries; patches 2-4 add tests.
> 
> Basic pid-specific USDT support is added; i.e. it is necessary
> to specify the target pid in the provider name such as
> "example1234" ; future work could add pid wildcarding.
> 
> ELF note defined probes are defined by including sys/sdt.h
> from the systemtap-sdt-devel package, and are defined in
> C programs via
> 
> DTRACE_PROBEn(provider, probe, [args...])
> 
> See the tests for concrete examples.
> 
> For python, go, etc, USDT probes can be added via libstapsdt [1]
> and associated language-specific bindings.  This allows users
> of those languages to add USDT probes too.  For example, the
> following example program adds a "pythonapp" probe "firstProbe"
> using the python-specific libstapsdt binding:
> 
> #!/usr/bin/python3
> from time import sleep
> import stapsdt
> provider = stapsdt.Provider("pythonapp")
> probe = provider.add_probe(
> "firstProbe", stapsdt.ArgTypes.uint64, stapsdt.ArgTypes.int32)
> provider.load()
> while True:
>     print("Firing probe...")
>     if probe.fire("My little probe", 42):
>         print("Probe fired!")
>     sleep(1)
> 
> We can then trace this via dtrace:
> 
> # dtrace -n 'pythonapp503211:::* { printf("args %s, %d\n", copyinstr(arg0), arg1); }'
> 
> dtrace: description 'pythonapp503211:::* ' matched 1 probe
> CPU     ID                    FUNCTION:NAME
>   6 286628                      :firstProbe args My little probe, 42
> 
> [1] https://github.com/linux-usdt/libstapsdt
> 
> Alan Maguire (4):
>   USDT: support ELF-note-defined probes
>   selftests/usdt: add test for USDT note-defined probe firing, args
>   selftests/usdt: add test for USDT notes in shared library
>   selftests/usdt: add test covering different forms of USDT note args
> 
>  include/dtrace/pid.h                      |  29 ++
>  libdtrace/dt_cg.c                         |  47 ++
>  libdtrace/dt_cg.h                         |   1 +
>  libdtrace/dt_pid.c                        | 466 ++++++++++++++++++++
>  libdtrace/dt_prov_uprobe.c                |  19 +-
>  test/unittest/usdt/sdt_notes.h            | 504 ++++++++++++++++++++++
>  test/unittest/usdt/tst.usdt-notes-args.r  |   2 +
>  test/unittest/usdt/tst.usdt-notes-args.sh |  51 +++
>  test/unittest/usdt/tst.usdt-notes-lib.r   |  14 +
>  test/unittest/usdt/tst.usdt-notes-lib.sh  | 145 +++++++
>  test/unittest/usdt/tst.usdt-notes.r       |  14 +
>  test/unittest/usdt/tst.usdt-notes.sh      | 121 ++++++
>  12 files changed, 1406 insertions(+), 7 deletions(-)
>  create mode 100644 test/unittest/usdt/sdt_notes.h
>  create mode 100644 test/unittest/usdt/tst.usdt-notes-args.r
>  create mode 100755 test/unittest/usdt/tst.usdt-notes-args.sh
>  create mode 100644 test/unittest/usdt/tst.usdt-notes-lib.r
>  create mode 100755 test/unittest/usdt/tst.usdt-notes-lib.sh
>  create mode 100644 test/unittest/usdt/tst.usdt-notes.r
>  create mode 100755 test/unittest/usdt/tst.usdt-notes.sh
> 
> -- 
> 2.43.5
> 

Re: [PATCH v2 0/4] ELF note-based USDT support

[Alan Maguire]

On 29/01/2025 15:33, Kris Van Hees wrote:
> Thank you for rebasing your work on the newest tree.  That will certainly
> help review them and move things forward.
> 
> I would definitely rework the commit message though, because
> 
> 1. DTrace has a specific understanding of what USDT probes are and how they
>    work and stap-based probes do not provide the same functionality.  One
>    example of that you already point ay: they are not discoverable - i.e.
>    they are not registered upon startup which is why you need to refer to
>    them directly by provider name (with embedded pid).  I think you need to
>    be very clear about the distinction.  Using STAPSDT or stapsdt might be
>    a better choice than referring to USDT.
>

sure; I'll use stapsdt.

> 2. I think that the commit message fails to highlight that this support is
>    to make it possible to trace programs (and shared librearies)that have been 
>    built with stap style probes.  I don't think it is in the best interest of
>    DTrace users to build their executables and shared libraries with stap
>    style probes over DTrace USDT probes, especially given the significant
>    advantage that DTrace USDT probes have (see 1. above).
> > 3. While I can see the point of mentioning how to add stapsdt probes to
>    code, it also is a source for confusion and thus is probably better left
>    out.  Since this is a compability feature, surely those wanting to use it
>    already have executables with such probes or know how to create them.
>    By including it here, you also introduce the very unfortunate fact that
>    stapsdt uses DTRACE_PROBE*() macros even though the probes have never really
>    been DTrace compatible, and it is only with your proposed patch now that
>    they could be used in DTrace.  The systemtap project shouldn't have
>    piggy-backed on DTRACE_PROBE*() in the first place because it causes this
>    type of confusion and complications, so I would very much prefer not to
>    highlight that mess with this patch series.
> 
> The stap probe support is very significant because we unfortunately do have to
> live with a world where there are multiple ways that such userspace probes
> have been implemented.  And given that packages are released with probes and
> people may want to trace them makes this addition certainly very worthwhile.
> But I think it should be clear that this is for compatibilty/interoperability
> purposes only.
> 

Sounds good, I'll rework the patches accordingly.

Alan

> 	Kris
> 
> On Wed, Jan 29, 2025 at 02:55:18PM +0000, Alan Maguire wrote:
>> This series adds support (patch 1) for ELF-note defined USDT
>> probes in binaries and libraries; patches 2-4 add tests.
>>
>> Basic pid-specific USDT support is added; i.e. it is necessary
>> to specify the target pid in the provider name such as
>> "example1234" ; future work could add pid wildcarding.
>>
>> ELF note defined probes are defined by including sys/sdt.h
>> from the systemtap-sdt-devel package, and are defined in
>> C programs via
>>
>> DTRACE_PROBEn(provider, probe, [args...])
>>
>> See the tests for concrete examples.
>>
>> For python, go, etc, USDT probes can be added via libstapsdt [1]
>> and associated language-specific bindings.  This allows users
>> of those languages to add USDT probes too.  For example, the
>> following example program adds a "pythonapp" probe "firstProbe"
>> using the python-specific libstapsdt binding:
>>
>> #!/usr/bin/python3
>> from time import sleep
>> import stapsdt
>> provider = stapsdt.Provider("pythonapp")
>> probe = provider.add_probe(
>> "firstProbe", stapsdt.ArgTypes.uint64, stapsdt.ArgTypes.int32)
>> provider.load()
>> while True:
>>     print("Firing probe...")
>>     if probe.fire("My little probe", 42):
>>         print("Probe fired!")
>>     sleep(1)
>>
>> We can then trace this via dtrace:
>>
>> # dtrace -n 'pythonapp503211:::* { printf("args %s, %d\n", copyinstr(arg0), arg1); }'
>>
>> dtrace: description 'pythonapp503211:::* ' matched 1 probe
>> CPU     ID                    FUNCTION:NAME
>>   6 286628                      :firstProbe args My little probe, 42
>>
>> [1] https://github.com/linux-usdt/libstapsdt
>>
>> Alan Maguire (4):
>>   USDT: support ELF-note-defined probes
>>   selftests/usdt: add test for USDT note-defined probe firing, args
>>   selftests/usdt: add test for USDT notes in shared library
>>   selftests/usdt: add test covering different forms of USDT note args
>>
>>  include/dtrace/pid.h                      |  29 ++
>>  libdtrace/dt_cg.c                         |  47 ++
>>  libdtrace/dt_cg.h                         |   1 +
>>  libdtrace/dt_pid.c                        | 466 ++++++++++++++++++++
>>  libdtrace/dt_prov_uprobe.c                |  19 +-
>>  test/unittest/usdt/sdt_notes.h            | 504 ++++++++++++++++++++++
>>  test/unittest/usdt/tst.usdt-notes-args.r  |   2 +
>>  test/unittest/usdt/tst.usdt-notes-args.sh |  51 +++
>>  test/unittest/usdt/tst.usdt-notes-lib.r   |  14 +
>>  test/unittest/usdt/tst.usdt-notes-lib.sh  | 145 +++++++
>>  test/unittest/usdt/tst.usdt-notes.r       |  14 +
>>  test/unittest/usdt/tst.usdt-notes.sh      | 121 ++++++
>>  12 files changed, 1406 insertions(+), 7 deletions(-)
>>  create mode 100644 test/unittest/usdt/sdt_notes.h
>>  create mode 100644 test/unittest/usdt/tst.usdt-notes-args.r
>>  create mode 100755 test/unittest/usdt/tst.usdt-notes-args.sh
>>  create mode 100644 test/unittest/usdt/tst.usdt-notes-lib.r
>>  create mode 100755 test/unittest/usdt/tst.usdt-notes-lib.sh
>>  create mode 100644 test/unittest/usdt/tst.usdt-notes.r
>>  create mode 100755 test/unittest/usdt/tst.usdt-notes.sh
>>
>> -- 
>> 2.43.5
>>

Re: [DTrace-devel] [PATCH v2 0/4] ELF note-based USDT support

[Sam James]

Alan Maguire via DTrace-devel <dtrace-devel@oss.oracle.com> writes:

> On 29/01/2025 15:33, Kris Van Hees wrote:
>> Thank you for rebasing your work on the newest tree.  That will certainly
>> help review them and move things forward.
>> 
>> I would definitely rework the commit message though, because
>> 
>> 1. DTrace has a specific understanding of what USDT probes are and how they
>>    work and stap-based probes do not provide the same functionality.  One
>>    example of that you already point ay: they are not discoverable - i.e.
>>    they are not registered upon startup which is why you need to refer to
>>    them directly by provider name (with embedded pid).  I think you need to
>>    be very clear about the distinction.  Using STAPSDT or stapsdt might be
>>    a better choice than referring to USDT.
>>
>
> sure; I'll use stapsdt.
>
>> 2. I think that the commit message fails to highlight that this support is
>>    to make it possible to trace programs (and shared librearies)that have been 
>>    built with stap style probes.  I don't think it is in the best interest of
>>    DTrace users to build their executables and shared libraries with stap
>>    style probes over DTrace USDT probes, especially given the significant
>>    advantage that DTrace USDT probes have (see 1. above).
>> > 3. While I can see the point of mentioning how to add stapsdt probes to
>>    code, it also is a source for confusion and thus is probably better left
>>    out.  Since this is a compability feature, surely those wanting to use it
>>    already have executables with such probes or know how to create them.
>>    By including it here, you also introduce the very unfortunate fact that
>>    stapsdt uses DTRACE_PROBE*() macros even though the probes have never really
>>    been DTrace compatible, and it is only with your proposed patch now that
>>    they could be used in DTrace.  The systemtap project shouldn't have
>>    piggy-backed on DTRACE_PROBE*() in the first place because it causes this
>>    type of confusion and complications, so I would very much prefer not to
>>    highlight that mess with this patch series.
>> 
>> The stap probe support is very significant because we unfortunately do have to
>> live with a world where there are multiple ways that such userspace probes
>> have been implemented.  And given that packages are released with probes and
>> people may want to trace them makes this addition certainly very worthwhile.
>> But I think it should be clear that this is for compatibilty/interoperability
>> purposes only.
>> 
>
> Sounds good, I'll rework the patches accordingly.

Any update on this? I'd really love to start wiring up support for these
in packages. I guess it interacts heavily with the work Kris has been
doing on making LTO work with USDT though?

cheeers,
sam

>
> Alan
>
> [...]

Re: [DTrace-devel] [PATCH v2 0/4] ELF note-based USDT support

[Kris Van Hees]

On Sun, Jun 08, 2025 at 02:42:04AM +0100, Sam James wrote:
> Alan Maguire via DTrace-devel <dtrace-devel@oss.oracle.com> writes:
> 
> > On 29/01/2025 15:33, Kris Van Hees wrote:
> >> Thank you for rebasing your work on the newest tree.  That will certainly
> >> help review them and move things forward.
> >> 
> >> I would definitely rework the commit message though, because
> >> 
> >> 1. DTrace has a specific understanding of what USDT probes are and how they
> >>    work and stap-based probes do not provide the same functionality.  One
> >>    example of that you already point ay: they are not discoverable - i.e.
> >>    they are not registered upon startup which is why you need to refer to
> >>    them directly by provider name (with embedded pid).  I think you need to
> >>    be very clear about the distinction.  Using STAPSDT or stapsdt might be
> >>    a better choice than referring to USDT.
> >>
> >
> > sure; I'll use stapsdt.
> >
> >> 2. I think that the commit message fails to highlight that this support is
> >>    to make it possible to trace programs (and shared librearies)that have been 
> >>    built with stap style probes.  I don't think it is in the best interest of
> >>    DTrace users to build their executables and shared libraries with stap
> >>    style probes over DTrace USDT probes, especially given the significant
> >>    advantage that DTrace USDT probes have (see 1. above).
> >> > 3. While I can see the point of mentioning how to add stapsdt probes to
> >>    code, it also is a source for confusion and thus is probably better left
> >>    out.  Since this is a compability feature, surely those wanting to use it
> >>    already have executables with such probes or know how to create them.
> >>    By including it here, you also introduce the very unfortunate fact that
> >>    stapsdt uses DTRACE_PROBE*() macros even though the probes have never really
> >>    been DTrace compatible, and it is only with your proposed patch now that
> >>    they could be used in DTrace.  The systemtap project shouldn't have
> >>    piggy-backed on DTRACE_PROBE*() in the first place because it causes this
> >>    type of confusion and complications, so I would very much prefer not to
> >>    highlight that mess with this patch series.
> >> 
> >> The stap probe support is very significant because we unfortunately do have to
> >> live with a world where there are multiple ways that such userspace probes
> >> have been implemented.  And given that packages are released with probes and
> >> people may want to trace them makes this addition certainly very worthwhile.
> >> But I think it should be clear that this is for compatibilty/interoperability
> >> purposes only.
> >> 
> >
> > Sounds good, I'll rework the patches accordingly.
> 
> Any update on this? I'd really love to start wiring up support for these
> in packages. I guess it interacts heavily with the work Kris has been
> doing on making LTO work with USDT though?

Yes, the USDT rework I actually just completed is a pre-condition of getting
support for stapsdt probes in DTrace.  With the new USDT approach in place,
this series can be revisited and adjusted to make use of what I introduced.

	Kris