[PATCH 0/3] Fix DTrace on modern kernels and in containers

[PATCH 0/3] Fix DTrace on modern kernels and in containers

[Jacob Carlborg]

Hi, I've created three patches for the issues I've encountered when trying to
get DTrace to work in my environments. One environment is on my local computer
running macOS ARM64 using Docker through OrbStack. The other is our Kubernetes
cluster in the cloud. The first two issues are differences in the Linux kernel
compared to what DTrace expects. The third issue is container related.

The OrbStack environment is running version 7.0.11 of the Linux kernel. The
Kubernetes cluster is running 6.12.85.

I'll be honest, the analysis of the issues and the patches are all done by an
AI. I've verified everything manually myself.

Jacob Carlborg (3):
  libdtrace: resolve BTF type tags when importing BTF into CTF
  libdtrace: derive io provider dev_name from the device model
  libdtrace: match BEGIN/END PID in the tracer's own PID namespace

 libdtrace/dt_btf.c         | 20 ++++++++++++-
 libdtrace/dt_prov_dtrace.c | 58 ++++++++++++++++++++++++++++++++++----
 libdtrace/io.d.in          | 17 +++++++++--
 3 files changed, 86 insertions(+), 9 deletions(-)

--
2.50.1 (Apple Git-155)

[PATCH 1/3] libdtrace: resolve BTF type tags when importing BTF into CTF

[Jacob Carlborg]

Without this fix, running any DTrace commands results in:

```
dtrace: invalid probe specifier ::::
"/usr/lib64/dtrace/6.12.0/procfs.d", line 131: operator -> cannot be
applied to pointer to type "void"; must be applied to a struct or
union pointer
```

The issue occurs on kernels which have BTF with type tags. The fix is
to treat `BTF_KIND_TYPE_TAG` as a transparent modifier, resolving
straight to `type->type`.

Signed-off-by: Jacob Carlborg <doob@me.com>
---
 libdtrace/dt_btf.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/libdtrace/dt_btf.c b/libdtrace/dt_btf.c
index 1f793397..1414dd1e 100644
--- a/libdtrace/dt_btf.c
+++ b/libdtrace/dt_btf.c
@@ -672,10 +672,28 @@ dt_btf_add_to_ctf(dtrace_hdl_t *dtp, dt_btf_t *btf, ctf_dict_t *ctf,
 
 		return ctfid == CTF_ERR ? dt_ctf_error(dtp, ctf) : ctfid;
 	}
+	case BTF_KIND_TYPE_TAG: {
+		/*
+		 * A type tag (e.g. __rcu, __user, __percpu) is a transparent
+		 * annotation on the type it wraps.  CTF has no equivalent, so
+		 * we resolve straight through to the underlying type, just as
+		 * we do for const/volatile/restrict.  Treating it as ignored
+		 * (mapping it to void) would leave every tagged member -- such
+		 * as task_struct.real_parent (__rcu) -- resolving to "void",
+		 * which breaks translator compilation on kernels whose BTF
+		 * carries type tags.
+		 */
+		ctfid = dt_btf_add_to_ctf(dtp, btf, ctf, type->type);
+		if (ctfid == CTF_ERR)
+			return CTF_ERR;		/* errno already set */
+
+		btf->ctfids[type_id] = ctfid;
+
+		return ctfid;
+	}
 	case BTF_KIND_VAR:
 	case BTF_KIND_DATASEC:
 	case BTF_KIND_DECL_TAG:
-	case BTF_KIND_TYPE_TAG:
 	case BTF_KIND_ENUM64:
 	case BTF_KIND_FUNC:
 		return btf->ctfids[0];		/* Ignored for CTF */
-- 
2.50.1 (Apple Git-155)

[PATCH 2/3] libdtrace: derive io provider dev_name from the device model

[Jacob Carlborg]

Without this fix, running any DTrace commands, like `dtrace -l`,
results in:

```
dtrace: invalid probe specifier :::: "/usr/lib64/dtrace/6.12.0/io.d",
line 134: failed to resolve vmlinux`major_names: Unknown symbol name
```

The issue is that on newer kernels the `major_names` symbols is not
available in `kallsyms` anymore. For example, running `grep -w
major_names /proc/kallsyms` on a newer kernel shows no results.
Running the same command on an older kernel gives the following
result:

```
$ grep -w major_names /proc/kallsyms
0000000000000000 b major_names
```

The fix is to stop depending on the `major_names` symbol and instead
derive `dev_name` from the device model (the driver name), mirroring
the `struct buffer_head` translator.

Signed-off-by: Jacob Carlborg <doob@me.com>
---
 libdtrace/io.d.in | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/libdtrace/io.d.in b/libdtrace/io.d.in
index 5ea356ee..e96a5b23 100644
--- a/libdtrace/io.d.in
+++ b/libdtrace/io.d.in
@@ -124,16 +124,27 @@ translator devinfo_t < struct buffer_head *B > {
 		       );
 };
 
+define_for_kernel([[__part0_dev]], [[(m4_kver(5,11,0), [[part0->bd_device]])]], [[part0.__dev]])
 #pragma D binding "1.6.3" translator
 translator devinfo_t < struct bio *B > {
 	dev_major = B->__disk_chk == NULL ? 0 : getmajor(B->__bio_part_dev);
 	dev_minor = B->__disk_chk == NULL ? 0 : getminor(B->__bio_part_dev);
 	dev_instance = 0;
+	/*
+	 * Derive the driver name from the device model rather than the kernel's
+	 * `major_names` array: that symbol became static and is no longer
+	 * resolvable via kallsyms on newer kernels.  This mirrors how the
+	 * struct buffer_head devinfo translator above obtains dev_name.
+	 */
 	dev_name = B->__disk_chk == NULL
 		? "nfs"
-		: stringof(((struct blk_major_name **)vmlinux`major_names)[
-				getmajor(B->__bio_part_dev) % 255
-			   ]->name);
+		: B->__disk->__part0_dev.parent
+			? B->__disk->__part0_dev.parent->driver->name
+				? stringof(B->__disk->__part0_dev.parent->driver->name)
+				: "<none>"
+			: B->__disk->__part0_dev.driver->name
+				? stringof(B->__disk->__part0_dev.driver->name)
+				: "<none>";
 	dev_statname = B->__disk_chk == NULL ? "nfs" :
 	    (B->__bio_partno) == 0 ? stringof(B->__disk->disk_name) :
 	    strjoin(stringof(B->__disk->disk_name), lltostr(B->__bio_partno));
-- 
2.50.1 (Apple Git-155)

[PATCH 3/3] libdtrace: match BEGIN/END PID in the tracer's own PID namespace

[Jacob Carlborg]

Without this fix, running any probe hangs with no output,
like `dtrace -n 'BEGIN { trace("hi"); exit(0); }'`.

The issues is that the `BEGIN/END` probes are implemented as uprobes
on DTrace's own `BEGIN_probe()/END_probe()` functions in `dt_work.c`.
`bpf_get_current_pid_tgid()` reports the TGID in the initial PID
namespace, which does not match `getpid()` when DTrace runs inside a
PID namespace. Using the `--pid=host` flag when running OrbStack does
not help, because it shares the daemon's namespace, not the kernel
init namespace. OrbStack's own supervisor runs in a nested PID
namespace. On bare metal `--pid=host` does reach the init namespaces
and the bug wouldn't appear.

This fix is when DTrace is in a non-initial PID namespace and the
kernel is at least 5.7, use `bpf_get_ns_current_pid_tgid` instead of
`bpf_get_current_pid_tgid`. `bpf_get_ns_current_pid_tgid` is
namespace aware.

Signed-off-by: Jacob Carlborg <doob@me.com>
---
 libdtrace/dt_prov_dtrace.c | 58 ++++++++++++++++++++++++++++++++++----
 1 file changed, 53 insertions(+), 5 deletions(-)

diff --git a/libdtrace/dt_prov_dtrace.c b/libdtrace/dt_prov_dtrace.c
index 4b788507..179ee4d6 100644
--- a/libdtrace/dt_prov_dtrace.c
+++ b/libdtrace/dt_prov_dtrace.c
@@ -9,6 +9,7 @@
 #include <assert.h>
 #include <errno.h>
 #include <string.h>
+#include <sys/stat.h>
 
 #include <bpf_asm.h>
 
@@ -129,12 +130,59 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 	 * the trampoline to minimize the cost of pointless firings in other
 	 * tracers, even though this means preserving the context in %r1 around
 	 * the call.
+	 *
+	 * bpf_get_current_pid_tgid() reports the TGID in the initial PID
+	 * namespace, which does not match getpid() when DTrace runs inside a
+	 * PID namespace (e.g. in a container) -- so the BEGIN/END probes would
+	 * never recognise their own firing and tracing would never activate.
+	 * When we are in a non-initial PID namespace on a kernel that provides
+	 * it (5.7+), use bpf_get_ns_current_pid_tgid() with DTrace's own PID
+	 * namespace so the value is reported in the same namespace as getpid().
+	 * Otherwise fall back to bpf_get_current_pid_tgid(), leaving the
+	 * initial-namespace case (and kernels < 5.7) exactly as before.
 	 */
-	emit(dlp, BPF_MOV_REG(BPF_REG_6, BPF_REG_1));
-	emit(dlp, BPF_CALL_HELPER(BPF_FUNC_get_current_pid_tgid));
-	emit(dlp, BPF_ALU64_IMM(BPF_RSH, BPF_REG_0, 32));
-	emit(dlp, BPF_BRANCH_IMM(BPF_JNE, BPF_REG_0, getpid(), pcb->pcb_fastlbl));
-	emit(dlp, BPF_MOV_REG(BPF_REG_1, BPF_REG_6));
+	{
+		struct stat	st;
+		int		use_ns = 0;
+		uint64_t	dev = 0, ino = 0;
+
+		/*
+		 * The initial PID namespace has a fixed inode number
+		 * (PROC_PID_INIT_INO, 0xEFFFFFFC).  If we are in it, getpid()
+		 * already agrees with bpf_get_current_pid_tgid() and no
+		 * namespace lookup is needed.
+		 */
+		if (stat("/proc/self/ns/pid", &st) == 0 &&
+		    st.st_ino != 0xEFFFFFFCULL &&
+		    pcb->pcb_hdl->dt_kernver >= DT_VERSION_NUMBER(5, 7, 0)) {
+			use_ns = 1;
+			dev = st.st_dev;
+			ino = st.st_ino;
+		}
+
+		emit(dlp, BPF_MOV_REG(BPF_REG_6, BPF_REG_1));
+		if (use_ns) {
+			/*
+			 * bpf_get_ns_current_pid_tgid(dev, ino, &nsinfo, sz)
+			 * fills a struct bpf_pidns_info { u32 pid; u32 tgid; };
+			 * the tgid is at offset 4.
+			 */
+			dt_cg_xsetx(dlp, NULL, DT_LBL_NONE, BPF_REG_1, dev);
+			dt_cg_xsetx(dlp, NULL, DT_LBL_NONE, BPF_REG_2, ino);
+			emit(dlp, BPF_MOV_REG(BPF_REG_3, BPF_REG_FP));
+			emit(dlp, BPF_ALU64_IMM(BPF_ADD, BPF_REG_3, DT_TRAMP_SP_SLOT(0)));
+			emit(dlp, BPF_MOV_IMM(BPF_REG_4, 8));
+			emit(dlp, BPF_CALL_HELPER(BPF_FUNC_get_ns_current_pid_tgid));
+			emit(dlp, BPF_LOAD(BPF_W, BPF_REG_0, BPF_REG_FP,
+					   DT_TRAMP_SP_SLOT(0) + 4));
+		} else {
+			emit(dlp, BPF_CALL_HELPER(BPF_FUNC_get_current_pid_tgid));
+			emit(dlp, BPF_ALU64_IMM(BPF_RSH, BPF_REG_0, 32));
+		}
+		emit(dlp, BPF_BRANCH_IMM(BPF_JNE, BPF_REG_0, getpid(),
+					 pcb->pcb_fastlbl));
+		emit(dlp, BPF_MOV_REG(BPF_REG_1, BPF_REG_6));
+	}
 
 	dt_cg_tramp_prologue_act(pcb, act);
 
-- 
2.50.1 (Apple Git-155)

Re: [PATCH 0/3] Fix DTrace on modern kernels and in containers

[Kris Van Hees]

Thanks for reporting these issues and sending patches.  It ceetainly highlights
an area where our testing is lacking, i.e. containers and related namespace
issues.  That will need some attention as well.


On Mon, Jul 20, 2026 at 03:30:57PM +0200, Jacob Carlborg wrote:
> Hi, I've created three patches for the issues I've encountered when trying to
> get DTrace to work in my environments. One environment is on my local computer
> running macOS ARM64 using Docker through OrbStack. The other is our Kubernetes
> cluster
> 
> Hi, I've created three patches for the issues I've encountered when trying to
> get DTrace to work in my environments. One environment is on my local computer
> running macOS ARM64 using Docker through OrbStack. The other is our Kubernetes
> cluster in the cloud. The first two issues are differences in the Linux kernel
> compared to what DTrace expects. The third issue is container related.
> 
> The OrbStack environment is running version 7.0.11 of the Linux kernel. The
> Kubernetes cluster is running 6.12.85.
> 
> I'll be honest, the analysis of the issues and the patches are all done by an
> AI. I've verified everything manually myself.
> 
> Jacob Carlborg (3):
>   libdtrace: resolve BTF type tags when importing BTF into CTF
>   libdtrace: derive io provider dev_name from the device model
>   libdtrace: match BEGIN/END PID in the tracer's own PID namespace
> 
>  libdtrace/dt_btf.c         | 20 ++++++++++++-
>  libdtrace/dt_prov_dtrace.c | 58 ++++++++++++++++++++++++++++++++++----
>  libdtrace/io.d.in          | 17 +++++++++--
>  3 files changed, 86 insertions(+), 9 deletions(-)
> 
> --
> 2.50.1 (Apple Git-155)
> 
> 
>