[PATCH 01/14] Fix stack-skip counts for caller and stackdepth

[PATCH 01/14] Fix stack-skip counts for caller and stackdepth

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Apparently, when we call the BPF get_stack() helper function,
it generally knows how many frames to skip to get the real kernel
stack.  For fentry/fexit, however, this is apparently not the case,
and commit bc65cb44d
("cg: allow providers to specify a skip count for stack retrieval")
added the ability to skip frames for fentry/fexit probes.

When this "skip" is needed, however, it must must be even deeper
when we descend further frames, such as when we call dt_bpf_*()
precompiled functions.

Add this support for dt_bpf_caller() and dt_bpf_stackdepth().
That is, if there are stack-skip frames, skip yet one more frame
when inside a bpf/get_bvar.c function.

Note that we declare the skip count volatile.  The compiler might
optimize code that uses the STACK_SKIP value, but we will subsequently
perform relocations that adjust this value.

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 bpf/get_bvar.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/bpf/get_bvar.c b/bpf/get_bvar.c
index 9625e764e..99a6503d5 100644
--- a/bpf/get_bvar.c
+++ b/bpf/get_bvar.c
@@ -53,12 +53,17 @@ noinline uint64_t dt_bvar_args(const dt_dctx_t *dctx, uint32_t idx)
 
 noinline uint64_t dt_bvar_caller(const dt_dctx_t *dctx)
 {
-	uint64_t	buf[2] = { 0, };
+	uint64_t	buf[3] = { 0, };
+	volatile uint64_t
+			skip = (uint64_t)(&STACK_SKIP);
 
 	if (bpf_get_stack(dctx->ctx, buf, sizeof(buf),
-			  (uint64_t)(&STACK_SKIP) & BPF_F_SKIP_FIELD_MASK) < 0)
+			  skip & BPF_F_SKIP_FIELD_MASK) < 0)
 		return 0;
 
+	/* If we had to skip any frames, account for the dt_bvar_caller() frame. */
+	if (skip)
+		return buf[2];
 	return buf[1];
 }
 
@@ -203,9 +208,11 @@ noinline uint64_t dt_bvar_stackdepth(const dt_dctx_t *dctx)
 	uint32_t	bufsiz = (uint32_t) (uint64_t) (&STKSIZ);
 	char		*buf = dctx->mem + (uint64_t)(&STACK_OFF);
 	uint64_t	retv;
+	volatile uint64_t
+			skip = (uint64_t)(&STACK_SKIP);
 
 	retv = bpf_get_stack(dctx->ctx, buf, bufsiz,
-			     (uint64_t)(&STACK_SKIP) & BPF_F_SKIP_FIELD_MASK);
+			     skip & BPF_F_SKIP_FIELD_MASK);
 	if (retv < 0)
 		return error(dctx, DTRACEFLT_BADSTACK, 0 /* FIXME */);
 
@@ -217,7 +224,11 @@ noinline uint64_t dt_bvar_stackdepth(const dt_dctx_t *dctx)
 	 * If retv==bufsiz, presumably the stack is larger than what we
 	 * can retrieve.  But it's also possible that the buffer was exactly
 	 * large enough.  So, leave it to the user to interpret the result.
+	 *
+	 * If we had to skip any frames, account for the dt_bvar_stackdepth() frame.
 	 */
+	if (skip)
+		return retv / sizeof(uint64_t) - 1;
 	return retv / sizeof(uint64_t);
 }
 
-- 
2.43.5

[PATCH 02/14] Add stack-skip frame count for rawtp provider

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_prov_rawtp.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libdtrace/dt_prov_rawtp.c b/libdtrace/dt_prov_rawtp.c
index f1d3b6bc4..a36bb51f4 100644
--- a/libdtrace/dt_prov_rawtp.c
+++ b/libdtrace/dt_prov_rawtp.c
@@ -296,6 +296,7 @@ use_alt:
 dt_provimpl_t	dt_rawtp = {
 	.name		= prvname,
 	.prog_type	= BPF_PROG_TYPE_RAW_TRACEPOINT,
+	.stack_skip	= 4,
 	.populate	= &populate,
 	.load_prog	= &dt_bpf_prog_load,
 	.trampoline	= &trampoline,
-- 
2.43.5

[PATCH 03/14] Test: remove unnecessary "unstable" tag

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 test/unittest/sched/tst.stackdepth.d | 1 -
 1 file changed, 1 deletion(-)

diff --git a/test/unittest/sched/tst.stackdepth.d b/test/unittest/sched/tst.stackdepth.d
index 02a878bea..e6ffca01c 100644
--- a/test/unittest/sched/tst.stackdepth.d
+++ b/test/unittest/sched/tst.stackdepth.d
@@ -4,7 +4,6 @@
  * Licensed under the Universal Permissive License v 1.0 as shown at
  * http://oss.oracle.com/licenses/upl.
  */
-/* @@tags: unstable */
 
 #pragma D option switchrate=100hz
 
-- 
2.43.5

[PATCH 04/14] Test: caller and stackdepth tests for fbt provider

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

We will introduce a set of tests for the caller and stackdepth
built-in variables for a wide selection of providers.

For the caller test, we will essentially call
        stack(2);
        sym(caller);
and then compare the caller to the second stack frame using
the new script check_caller_to_stack2.awk.

For the stackdepth test, we will essentially call
        printf("%d\n", stackdepth);
        stack();
and then compare the stackdepth to the reported frames using
the new script check_stackdepth_to_stack.awk.

In this patch, introduce tests for the fbt provider, along with
the support scripts they need.  Subsequent patches will handle
other providers.

The old tst.caller2.d and tst.stackdepth2.d, which tested fbt,
become obsolete.  Remove them.

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 .../variables/bvar/check_caller_to_stack2.awk | 32 +++++++++++++++++
 ...pth2.r.p => check_stackdepth_to_stack.awk} | 16 ++++++---
 test/unittest/variables/bvar/tst.caller-fbt.d | 25 +++++++++++++
 test/unittest/variables/bvar/tst.caller-fbt.r |  1 +
 .../variables/bvar/tst.caller-fbt.r.p         |  1 +
 test/unittest/variables/bvar/tst.caller2.d    | 31 ----------------
 test/unittest/variables/bvar/tst.caller2.r    |  1 -
 test/unittest/variables/bvar/tst.caller2.r.p  | 11 ------
 .../variables/bvar/tst.stackdepth-fbt.d       | 27 ++++++++++++++
 ...tst.stackdepth2.r => tst.stackdepth-fbt.r} |  0
 .../variables/bvar/tst.stackdepth-fbt.r.p     |  1 +
 .../unittest/variables/bvar/tst.stackdepth2.d | 36 -------------------
 12 files changed, 99 insertions(+), 83 deletions(-)
 create mode 100755 test/unittest/variables/bvar/check_caller_to_stack2.awk
 rename test/unittest/variables/bvar/{tst.stackdepth2.r.p => check_stackdepth_to_stack.awk} (63%)
 create mode 100644 test/unittest/variables/bvar/tst.caller-fbt.d
 create mode 100644 test/unittest/variables/bvar/tst.caller-fbt.r
 create mode 120000 test/unittest/variables/bvar/tst.caller-fbt.r.p
 delete mode 100644 test/unittest/variables/bvar/tst.caller2.d
 delete mode 100644 test/unittest/variables/bvar/tst.caller2.r
 delete mode 100755 test/unittest/variables/bvar/tst.caller2.r.p
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-fbt.d
 rename test/unittest/variables/bvar/{tst.stackdepth2.r => tst.stackdepth-fbt.r} (100%)
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-fbt.r.p
 delete mode 100644 test/unittest/variables/bvar/tst.stackdepth2.d

diff --git a/test/unittest/variables/bvar/check_caller_to_stack2.awk b/test/unittest/variables/bvar/check_caller_to_stack2.awk
new file mode 100755
index 000000000..8499852a7
--- /dev/null
+++ b/test/unittest/variables/bvar/check_caller_to_stack2.awk
@@ -0,0 +1,32 @@
+#!/usr/bin/gawk -f
+
+# Check output of tst.caller-$provider.d tests of the form
+#
+# {
+#     stack(2);
+#     sym(caller);
+# }
+#
+# Confirm that the caller information matches the stack information.
+
+# Look for the first nonblank line.
+NF != 0 {
+	# It is the current frame.  Skip it.
+	if (getline != 1) { print "ERROR: missing expected output"; exit(1); }
+
+	# Now we have the caller frame.  Strip off the offset.
+	expect=$1
+	sub("+0x[0-9a-f]*$", "", expect);
+
+	# Finally, get the sym(caller) output.
+	if (getline != 1) { print "ERROR: missing expected output"; exit(1); }
+
+	# Compare.
+	if (expect == $1) {
+		print "success";
+		exit(0);
+	} else {
+		print "ERROR: expect", expect, "but got", $1;
+		exit(1);
+	}
+}
diff --git a/test/unittest/variables/bvar/tst.stackdepth2.r.p b/test/unittest/variables/bvar/check_stackdepth_to_stack.awk
similarity index 63%
rename from test/unittest/variables/bvar/tst.stackdepth2.r.p
rename to test/unittest/variables/bvar/check_stackdepth_to_stack.awk
index 9b071181f..fba1f4242 100755
--- a/test/unittest/variables/bvar/tst.stackdepth2.r.p
+++ b/test/unittest/variables/bvar/check_stackdepth_to_stack.awk
@@ -1,8 +1,16 @@
 #!/usr/bin/gawk -f
-# Oracle Linux DTrace.
-# Copyright (c) 2016, 2021, Oracle and/or its affiliates. All rights reserved.
-# Licensed under the Universal Permissive License v 1.0 as shown at
-# http://oss.oracle.com/licenses/upl.
+#
+# Check output of tst.stackdepth-$provider.d tests of the form
+#
+# {
+#     printf("DEPTH %d\n", stackdepth);
+#     printf("TRACE BEGIN\n");
+#     stack();
+#     printf("TRACE END\n");
+#     exit(0);
+# }
+#
+# Confirm that the stackdepth information matches the stack information.
 
 /^DEPTH/ {
 	depth = int($2);
diff --git a/test/unittest/variables/bvar/tst.caller-fbt.d b/test/unittest/variables/bvar/tst.caller-fbt.d
new file mode 100644
index 000000000..695104134
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-fbt.d
@@ -0,0 +1,25 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+/*
+ * @@trigger: periodic_output
+ */
+
+#pragma D option quiet
+
+fbt::ksys_write:entry
+/pid == $target/
+{
+	stack(2);
+	sym(caller);
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.caller-fbt.r b/test/unittest/variables/bvar/tst.caller-fbt.r
new file mode 100644
index 000000000..2e9ba477f
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-fbt.r
@@ -0,0 +1 @@
+success
diff --git a/test/unittest/variables/bvar/tst.caller-fbt.r.p b/test/unittest/variables/bvar/tst.caller-fbt.r.p
new file mode 120000
index 000000000..954ca96aa
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-fbt.r.p
@@ -0,0 +1 @@
+check_caller_to_stack2.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.caller2.d b/test/unittest/variables/bvar/tst.caller2.d
deleted file mode 100644
index ca6dbfd22..000000000
--- a/test/unittest/variables/bvar/tst.caller2.d
+++ /dev/null
@@ -1,31 +0,0 @@
-/*
- * Oracle Linux DTrace.
- * Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved.
- * Licensed under the Universal Permissive License v 1.0 as shown at
- * http://oss.oracle.com/licenses/upl.
- */
-
-/*
- * ASSERTION: The 'caller' should be consistent with stack().
- *
- * SECTION: Variables/Built-in Variables/caller
- */
-
-#pragma D option quiet
-#pragma D option destructive
-
-BEGIN
-{
-        system("echo write something > /dev/null");
-}
-
-fbt::ksys_write:entry
-{
-	stack(2);
-	sym(caller);
-	exit(0);
-}
-
-ERROR {
-	exit(1);
-}
diff --git a/test/unittest/variables/bvar/tst.caller2.r b/test/unittest/variables/bvar/tst.caller2.r
deleted file mode 100644
index 0cfbf0888..000000000
--- a/test/unittest/variables/bvar/tst.caller2.r
+++ /dev/null
@@ -1 +0,0 @@
-2
diff --git a/test/unittest/variables/bvar/tst.caller2.r.p b/test/unittest/variables/bvar/tst.caller2.r.p
deleted file mode 100755
index 1a26a4df2..000000000
--- a/test/unittest/variables/bvar/tst.caller2.r.p
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/bin/bash
-
-# sed:  remove +0x{ptr} offsets
-# awk:  look for the ksys_write frame, then write the next two lines
-# uniq:  count unique lines
-# awk:  report the counts
-
-sed 's/+0x.*$//' \
-| /usr/bin/gawk '/ksys_write/ {getline; print $1; getline; print $1; exit(0)}' \
-| uniq -c \
-| /usr/bin/gawk '{print $1}'
diff --git a/test/unittest/variables/bvar/tst.stackdepth-fbt.d b/test/unittest/variables/bvar/tst.stackdepth-fbt.d
new file mode 100644
index 000000000..10b05399e
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-fbt.d
@@ -0,0 +1,27 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+/*
+ * @@trigger: periodic_output
+ */
+
+#pragma D option quiet
+
+fbt::ksys_write:entry
+/pid == $target/
+{
+	printf("DEPTH %d\n", stackdepth);
+	printf("TRACE BEGIN\n");
+	stack(100);
+	printf("TRACE END\n");
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.stackdepth2.r b/test/unittest/variables/bvar/tst.stackdepth-fbt.r
similarity index 100%
rename from test/unittest/variables/bvar/tst.stackdepth2.r
rename to test/unittest/variables/bvar/tst.stackdepth-fbt.r
diff --git a/test/unittest/variables/bvar/tst.stackdepth-fbt.r.p b/test/unittest/variables/bvar/tst.stackdepth-fbt.r.p
new file mode 120000
index 000000000..e50f12822
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-fbt.r.p
@@ -0,0 +1 @@
+check_stackdepth_to_stack.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth2.d b/test/unittest/variables/bvar/tst.stackdepth2.d
deleted file mode 100644
index 2b1c0866a..000000000
--- a/test/unittest/variables/bvar/tst.stackdepth2.d
+++ /dev/null
@@ -1,36 +0,0 @@
-/*
- * Oracle Linux DTrace.
- * Copyright (c) 2006, 2021, Oracle and/or its affiliates. All rights reserved.
- * Licensed under the Universal Permissive License v 1.0 as shown at
- * http://oss.oracle.com/licenses/upl.
- */
-
-#pragma D option destructive
-#pragma D option quiet
-
-/*
- * ASSERTION:
- *   Test the stackdepth variable.  Verify its value against the stack trace.
- *
- * SECTION: Variables/Built-in Variables
- *
- */
-
-BEGIN
-{
-	system("echo write something > /dev/null");
-}
-
-fbt::ksys_write:entry
-{
-	printf("DEPTH %d\n", stackdepth);
-	printf("TRACE BEGIN\n");
-	stack();
-	printf("TRACE END\n");
-	exit(0);
-}
-
-ERROR
-{
-	exit(1);
-}
-- 
2.43.5

[PATCH 05/14] Test: caller and stackdepth tests for dtrace provider

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Also, a few old tests, which tested caller and stackdepth using the
dtrace provider, are now superfluous given the new, provider-named
tests.  The old tests were extremely lenient -- e.g., simply
checking that these built-in variables were not -1, even though
both variables are unsigned anyhow!

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 test/unittest/builtinvar/tst.caller.d         | 25 -------------------
 .../variables/bvar/tst.caller-dtrace.d        | 19 ++++++++++++++
 .../variables/bvar/tst.caller-dtrace.r        |  1 +
 test/unittest/variables/bvar/tst.caller.d     | 23 -----------------
 .../variables/bvar/tst.stackdepth-dtrace.d    | 22 ++++++++++++++++
 .../variables/bvar/tst.stackdepth-dtrace.r    |  5 ++++
 test/unittest/variables/bvar/tst.stackdepth.d | 23 -----------------
 7 files changed, 47 insertions(+), 71 deletions(-)
 delete mode 100644 test/unittest/builtinvar/tst.caller.d
 create mode 100644 test/unittest/variables/bvar/tst.caller-dtrace.d
 create mode 100644 test/unittest/variables/bvar/tst.caller-dtrace.r
 delete mode 100644 test/unittest/variables/bvar/tst.caller.d
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-dtrace.d
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-dtrace.r
 delete mode 100644 test/unittest/variables/bvar/tst.stackdepth.d

diff --git a/test/unittest/builtinvar/tst.caller.d b/test/unittest/builtinvar/tst.caller.d
deleted file mode 100644
index 19a4bab9e..000000000
--- a/test/unittest/builtinvar/tst.caller.d
+++ /dev/null
@@ -1,25 +0,0 @@
-/*
- * Oracle Linux DTrace.
- * Copyright (c) 2006, 2021, Oracle and/or its affiliates. All rights reserved.
- * Licensed under the Universal Permissive License v 1.0 as shown at
- * http://oss.oracle.com/licenses/upl.
- */
-
-/*
- * ASSERTION: print 'caller' and make sure it succeeds.
- *
- * SECTION: Variables/Built-in Variables
- */
-
-#pragma D option quiet
-
-BEGIN
-{
-	printf("The caller is %u\n", caller);
-	exit(0);
-}
-
-ERROR
-{
-	exit(1);
-}
diff --git a/test/unittest/variables/bvar/tst.caller-dtrace.d b/test/unittest/variables/bvar/tst.caller-dtrace.d
new file mode 100644
index 000000000..d69386930
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-dtrace.d
@@ -0,0 +1,19 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+#pragma D option quiet
+
+BEGIN {
+	trace(caller);
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.caller-dtrace.r b/test/unittest/variables/bvar/tst.caller-dtrace.r
new file mode 100644
index 000000000..573541ac9
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-dtrace.r
@@ -0,0 +1 @@
+0
diff --git a/test/unittest/variables/bvar/tst.caller.d b/test/unittest/variables/bvar/tst.caller.d
deleted file mode 100644
index 3d64fa98c..000000000
--- a/test/unittest/variables/bvar/tst.caller.d
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Oracle Linux DTrace.
- * Copyright (c) 2020, 2021, Oracle and/or its affiliates. All rights reserved.
- * Licensed under the Universal Permissive License v 1.0 as shown at
- * http://oss.oracle.com/licenses/upl.
- */
-
-/*
- * ASSERTION: The 'caller' variable can be accessed and is not -1.
- *
- * SECTION: Variables/Built-in Variables/caller
- */
-
-#pragma D option quiet
-
-BEGIN {
-	trace(caller);
-	exit(caller != -1 ? 0 : 1);
-}
-
-ERROR {
-	exit(1);
-}
diff --git a/test/unittest/variables/bvar/tst.stackdepth-dtrace.d b/test/unittest/variables/bvar/tst.stackdepth-dtrace.d
new file mode 100644
index 000000000..f0f828f23
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-dtrace.d
@@ -0,0 +1,22 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+#pragma D option quiet
+
+BEGIN {
+	printf("DEPTH %d\n", stackdepth);
+	printf("TRACE BEGIN\n");
+	stack();
+	printf("TRACE END\n");
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.stackdepth-dtrace.r b/test/unittest/variables/bvar/tst.stackdepth-dtrace.r
new file mode 100644
index 000000000..1afb1f057
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-dtrace.r
@@ -0,0 +1,5 @@
+DEPTH 0
+TRACE BEGIN
+
+TRACE END
+
diff --git a/test/unittest/variables/bvar/tst.stackdepth.d b/test/unittest/variables/bvar/tst.stackdepth.d
deleted file mode 100644
index ad728fd85..000000000
--- a/test/unittest/variables/bvar/tst.stackdepth.d
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Oracle Linux DTrace.
- * Copyright (c) 2020, 2021, Oracle and/or its affiliates. All rights reserved.
- * Licensed under the Universal Permissive License v 1.0 as shown at
- * http://oss.oracle.com/licenses/upl.
- */
-
-/*
- * ASSERTION: The 'stackdepth' variable can be accessed and is not -1.
- *
- * SECTION: Variables/Built-in Variables/stackdepth
- */
-
-#pragma D option quiet
-
-BEGIN {
-	trace(stackdepth);
-	exit(stackdepth != -1 ? 0 : 1);
-}
-
-ERROR {
-	exit(1);
-}
-- 
2.43.5

[PATCH 06/14] Test: caller and stackdepth tests for rawtp provider

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Also, add skip_rawtp_old.x, to skip rawtp testing on older kernels
for the reasons described in the file.

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 test/unittest/variables/bvar/skip_rawtp_old.x | 31 +++++++++++++++++++
 .../variables/bvar/tst.caller-rawtp.d         | 24 ++++++++++++++
 .../variables/bvar/tst.caller-rawtp.r         |  1 +
 .../variables/bvar/tst.caller-rawtp.r.p       |  1 +
 .../variables/bvar/tst.caller-rawtp.x         |  1 +
 .../variables/bvar/tst.stackdepth-rawtp.d     | 26 ++++++++++++++++
 .../variables/bvar/tst.stackdepth-rawtp.r     |  1 +
 .../variables/bvar/tst.stackdepth-rawtp.r.p   |  1 +
 .../variables/bvar/tst.stackdepth-rawtp.x     |  1 +
 9 files changed, 87 insertions(+)
 create mode 100755 test/unittest/variables/bvar/skip_rawtp_old.x
 create mode 100644 test/unittest/variables/bvar/tst.caller-rawtp.d
 create mode 100644 test/unittest/variables/bvar/tst.caller-rawtp.r
 create mode 120000 test/unittest/variables/bvar/tst.caller-rawtp.r.p
 create mode 120000 test/unittest/variables/bvar/tst.caller-rawtp.x
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-rawtp.d
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-rawtp.r
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-rawtp.r.p
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-rawtp.x

diff --git a/test/unittest/variables/bvar/skip_rawtp_old.x b/test/unittest/variables/bvar/skip_rawtp_old.x
new file mode 100755
index 000000000..bf7970832
--- /dev/null
+++ b/test/unittest/variables/bvar/skip_rawtp_old.x
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+#
+# In older kernels (e.g., UEK6), the BPF helper function had a bug with
+# skipping stack frames.  While that many stack frames were correctly
+# skipped on the leaf end of the stack, that many frames were also
+# incorrectly skipped at the root end of the output buffer.
+#
+# Only two DTrace providers ask for a nonzero skip count.  One is the
+# fentry/fexit implementation of fbt, but it was not used on such older
+# kernels.
+#
+# The other is rawtp.  Therefore, rawtp stack(), caller, and stackdepth
+# results on such older kernels can be incorrect.
+#
+# A few other providers also have some or most probes implemented in
+# terms of rawtp probes.  They could have similar problems, depending
+# on which probes are used.
+#
+
+read MAJOR MINOR <<< `uname -r | grep -Eo '^[0-9]+\.[0-9]+' | tr '.' ' '`
+
+if [ $MAJOR -gt 5 ]; then
+	exit 0
+fi
+if [ $MAJOR -eq 5 -a $MINOR -ge 10 ]; then
+	exit 0
+fi
+
+echo "rawtp caller, stack(), and stackdepth problematic in old kernels"
+exit 2
diff --git a/test/unittest/variables/bvar/tst.caller-rawtp.d b/test/unittest/variables/bvar/tst.caller-rawtp.d
new file mode 100644
index 000000000..9f8475071
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-rawtp.d
@@ -0,0 +1,24 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+/*
+ * @@trigger: periodic_output
+ */
+
+#pragma D option quiet
+
+rawtp:sched::
+{
+	stack(2);
+	sym(caller);
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.caller-rawtp.r b/test/unittest/variables/bvar/tst.caller-rawtp.r
new file mode 100644
index 000000000..2e9ba477f
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-rawtp.r
@@ -0,0 +1 @@
+success
diff --git a/test/unittest/variables/bvar/tst.caller-rawtp.r.p b/test/unittest/variables/bvar/tst.caller-rawtp.r.p
new file mode 120000
index 000000000..954ca96aa
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-rawtp.r.p
@@ -0,0 +1 @@
+check_caller_to_stack2.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.caller-rawtp.x b/test/unittest/variables/bvar/tst.caller-rawtp.x
new file mode 120000
index 000000000..37b3ef9f8
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-rawtp.x
@@ -0,0 +1 @@
+skip_rawtp_old.x
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth-rawtp.d b/test/unittest/variables/bvar/tst.stackdepth-rawtp.d
new file mode 100644
index 000000000..ddcc0a13f
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-rawtp.d
@@ -0,0 +1,26 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+/*
+ * @@trigger: periodic_output
+ */
+
+#pragma D option quiet
+
+rawtp:sched::
+{
+	printf("DEPTH %d\n", stackdepth);
+	printf("TRACE BEGIN\n");
+	stack(100);
+	printf("TRACE END\n");
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.stackdepth-rawtp.r b/test/unittest/variables/bvar/tst.stackdepth-rawtp.r
new file mode 100644
index 000000000..3bd29b8ed
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-rawtp.r
@@ -0,0 +1 @@
+Stack depth OK
diff --git a/test/unittest/variables/bvar/tst.stackdepth-rawtp.r.p b/test/unittest/variables/bvar/tst.stackdepth-rawtp.r.p
new file mode 120000
index 000000000..e50f12822
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-rawtp.r.p
@@ -0,0 +1 @@
+check_stackdepth_to_stack.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth-rawtp.x b/test/unittest/variables/bvar/tst.stackdepth-rawtp.x
new file mode 120000
index 000000000..37b3ef9f8
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-rawtp.x
@@ -0,0 +1 @@
+skip_rawtp_old.x
\ No newline at end of file
-- 
2.43.5

[PATCH 07/14] Test: caller and stackdepth tests for cpc provider

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 test/unittest/variables/bvar/tst.caller-cpc.d | 22 +++++++++++++++++
 test/unittest/variables/bvar/tst.caller-cpc.r |  1 +
 .../variables/bvar/tst.caller-cpc.r.p         |  1 +
 .../variables/bvar/tst.stackdepth-cpc.d       | 24 +++++++++++++++++++
 .../variables/bvar/tst.stackdepth-cpc.r       |  1 +
 .../variables/bvar/tst.stackdepth-cpc.r.p     |  1 +
 6 files changed, 50 insertions(+)
 create mode 100644 test/unittest/variables/bvar/tst.caller-cpc.d
 create mode 100644 test/unittest/variables/bvar/tst.caller-cpc.r
 create mode 120000 test/unittest/variables/bvar/tst.caller-cpc.r.p
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-cpc.d
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-cpc.r
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-cpc.r.p

diff --git a/test/unittest/variables/bvar/tst.caller-cpc.d b/test/unittest/variables/bvar/tst.caller-cpc.d
new file mode 100644
index 000000000..8932185ee
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-cpc.d
@@ -0,0 +1,22 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+#pragma D option quiet
+
+cpc:::cpu_clock-all-100000000
+/stackdepth > 1/
+{
+	stack(2);
+	sym(caller);
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.caller-cpc.r b/test/unittest/variables/bvar/tst.caller-cpc.r
new file mode 100644
index 000000000..2e9ba477f
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-cpc.r
@@ -0,0 +1 @@
+success
diff --git a/test/unittest/variables/bvar/tst.caller-cpc.r.p b/test/unittest/variables/bvar/tst.caller-cpc.r.p
new file mode 120000
index 000000000..954ca96aa
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-cpc.r.p
@@ -0,0 +1 @@
+check_caller_to_stack2.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth-cpc.d b/test/unittest/variables/bvar/tst.stackdepth-cpc.d
new file mode 100644
index 000000000..abefee0b0
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-cpc.d
@@ -0,0 +1,24 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+#pragma D option quiet
+
+cpc:::cpu_clock-all-100000000
+/stackdepth > 1/
+{
+	printf("DEPTH %d\n", stackdepth);
+	printf("TRACE BEGIN\n");
+	stack(100);
+	printf("TRACE END\n");
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.stackdepth-cpc.r b/test/unittest/variables/bvar/tst.stackdepth-cpc.r
new file mode 100644
index 000000000..3bd29b8ed
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-cpc.r
@@ -0,0 +1 @@
+Stack depth OK
diff --git a/test/unittest/variables/bvar/tst.stackdepth-cpc.r.p b/test/unittest/variables/bvar/tst.stackdepth-cpc.r.p
new file mode 120000
index 000000000..e50f12822
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-cpc.r.p
@@ -0,0 +1 @@
+check_stackdepth_to_stack.awk
\ No newline at end of file
-- 
2.43.5

[PATCH 08/14] Test: caller and stackdepth tests for ip provider

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 test/unittest/variables/bvar/perlping.pl      |  8 +++++++
 test/unittest/variables/bvar/tst.caller-ip.d  | 21 ++++++++++++++++
 test/unittest/variables/bvar/tst.caller-ip.r  |  1 +
 .../unittest/variables/bvar/tst.caller-ip.r.p |  1 +
 test/unittest/variables/bvar/tst.caller-ip.t  |  6 +++++
 .../variables/bvar/tst.stackdepth-ip.d        | 24 +++++++++++++++++++
 .../variables/bvar/tst.stackdepth-ip.r        |  1 +
 .../variables/bvar/tst.stackdepth-ip.r.p      |  1 +
 .../variables/bvar/tst.stackdepth-ip.t        |  6 +++++
 9 files changed, 69 insertions(+)
 create mode 100755 test/unittest/variables/bvar/perlping.pl
 create mode 100644 test/unittest/variables/bvar/tst.caller-ip.d
 create mode 100644 test/unittest/variables/bvar/tst.caller-ip.r
 create mode 120000 test/unittest/variables/bvar/tst.caller-ip.r.p
 create mode 100755 test/unittest/variables/bvar/tst.caller-ip.t
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-ip.d
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-ip.r
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-ip.r.p
 create mode 100755 test/unittest/variables/bvar/tst.stackdepth-ip.t

diff --git a/test/unittest/variables/bvar/perlping.pl b/test/unittest/variables/bvar/perlping.pl
new file mode 100755
index 000000000..97c9b09e9
--- /dev/null
+++ b/test/unittest/variables/bvar/perlping.pl
@@ -0,0 +1,8 @@
+#!/usr/bin/perl -w
+# Oracle Linux DTrace.
+# Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.
+# Licensed under the Universal Permissive License v 1.0 as shown at
+# http://oss.oracle.com/licenses/upl.
+use Net::Ping;
+my $p = Net::Ping->new(${ARGV[0]}, 5, 56);
+$p->ping(${ARGV[1]});
diff --git a/test/unittest/variables/bvar/tst.caller-ip.d b/test/unittest/variables/bvar/tst.caller-ip.d
new file mode 100644
index 000000000..83dbcec5e
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-ip.d
@@ -0,0 +1,21 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+#pragma D option quiet
+
+ip:::
+{
+	stack(2);
+	sym(caller);
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.caller-ip.r b/test/unittest/variables/bvar/tst.caller-ip.r
new file mode 100644
index 000000000..2e9ba477f
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-ip.r
@@ -0,0 +1 @@
+success
diff --git a/test/unittest/variables/bvar/tst.caller-ip.r.p b/test/unittest/variables/bvar/tst.caller-ip.r.p
new file mode 120000
index 000000000..954ca96aa
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-ip.r.p
@@ -0,0 +1 @@
+check_caller_to_stack2.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.caller-ip.t b/test/unittest/variables/bvar/tst.caller-ip.t
new file mode 100755
index 000000000..836d283da
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-ip.t
@@ -0,0 +1,6 @@
+#!/bin/sh
+
+testdir="$(dirname $_test)"
+local=127.0.0.1
+
+$testdir/perlping.pl icmp $local
diff --git a/test/unittest/variables/bvar/tst.stackdepth-ip.d b/test/unittest/variables/bvar/tst.stackdepth-ip.d
new file mode 100644
index 000000000..eef942b5b
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-ip.d
@@ -0,0 +1,24 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+#pragma D option quiet
+
+ip:::
+/stackdepth > 0/
+{
+	printf("DEPTH %d\n", stackdepth);
+	printf("TRACE BEGIN\n");
+	stack(100);
+	printf("TRACE END\n");
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.stackdepth-ip.r b/test/unittest/variables/bvar/tst.stackdepth-ip.r
new file mode 100644
index 000000000..3bd29b8ed
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-ip.r
@@ -0,0 +1 @@
+Stack depth OK
diff --git a/test/unittest/variables/bvar/tst.stackdepth-ip.r.p b/test/unittest/variables/bvar/tst.stackdepth-ip.r.p
new file mode 120000
index 000000000..e50f12822
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-ip.r.p
@@ -0,0 +1 @@
+check_stackdepth_to_stack.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth-ip.t b/test/unittest/variables/bvar/tst.stackdepth-ip.t
new file mode 100755
index 000000000..836d283da
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-ip.t
@@ -0,0 +1,6 @@
+#!/bin/sh
+
+testdir="$(dirname $_test)"
+local=127.0.0.1
+
+$testdir/perlping.pl icmp $local
-- 
2.43.5

[PATCH 09/14] Test: caller and stackdepth tests for profile provider

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 test/unittest/builtinvar/tst.caller1.d        | 29 -------------------
 .../variables/bvar/tst.caller-profile.d       | 22 ++++++++++++++
 .../variables/bvar/tst.caller-profile.r       |  1 +
 .../variables/bvar/tst.caller-profile.r.p     |  1 +
 .../variables/bvar/tst.stackdepth-profile.d   | 24 +++++++++++++++
 .../variables/bvar/tst.stackdepth-profile.r   |  1 +
 .../variables/bvar/tst.stackdepth-profile.r.p |  1 +
 7 files changed, 50 insertions(+), 29 deletions(-)
 delete mode 100644 test/unittest/builtinvar/tst.caller1.d
 create mode 100644 test/unittest/variables/bvar/tst.caller-profile.d
 create mode 100644 test/unittest/variables/bvar/tst.caller-profile.r
 create mode 120000 test/unittest/variables/bvar/tst.caller-profile.r.p
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-profile.d
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-profile.r
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-profile.r.p

diff --git a/test/unittest/builtinvar/tst.caller1.d b/test/unittest/builtinvar/tst.caller1.d
deleted file mode 100644
index ca0f098f3..000000000
--- a/test/unittest/builtinvar/tst.caller1.d
+++ /dev/null
@@ -1,29 +0,0 @@
-/*
- * Oracle Linux DTrace.
- * Copyright (c) 2006, 2021, Oracle and/or its affiliates. All rights reserved.
- * Licensed under the Universal Permissive License v 1.0 as shown at
- * http://oss.oracle.com/licenses/upl.
- */
-
-/*
- * ASSERTION: To print caller from profile and make sure it succeeds.
- *
- * SECTION: Variables/Built-in Variables
- */
-
-#pragma D option quiet
-
-BEGIN
-{
-}
-
-tick-10ms
-{
-	printf("The caller is %u\n", caller);
-	exit (0);
-}
-
-ERROR
-{
-	exit(1);
-}
diff --git a/test/unittest/variables/bvar/tst.caller-profile.d b/test/unittest/variables/bvar/tst.caller-profile.d
new file mode 100644
index 000000000..6e322bb70
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-profile.d
@@ -0,0 +1,22 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+#pragma D option quiet
+
+profile:::tick-50ms
+/stackdepth > 1/
+{
+	stack(2);
+	sym(caller);
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.caller-profile.r b/test/unittest/variables/bvar/tst.caller-profile.r
new file mode 100644
index 000000000..2e9ba477f
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-profile.r
@@ -0,0 +1 @@
+success
diff --git a/test/unittest/variables/bvar/tst.caller-profile.r.p b/test/unittest/variables/bvar/tst.caller-profile.r.p
new file mode 120000
index 000000000..954ca96aa
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-profile.r.p
@@ -0,0 +1 @@
+check_caller_to_stack2.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth-profile.d b/test/unittest/variables/bvar/tst.stackdepth-profile.d
new file mode 100644
index 000000000..8b42e9016
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-profile.d
@@ -0,0 +1,24 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+#pragma D option quiet
+
+profile:::tick-50ms
+/stackdepth > 1/
+{
+	printf("DEPTH %d\n", stackdepth);
+	printf("TRACE BEGIN\n");
+	stack(100);
+	printf("TRACE END\n");
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.stackdepth-profile.r b/test/unittest/variables/bvar/tst.stackdepth-profile.r
new file mode 100644
index 000000000..3bd29b8ed
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-profile.r
@@ -0,0 +1 @@
+Stack depth OK
diff --git a/test/unittest/variables/bvar/tst.stackdepth-profile.r.p b/test/unittest/variables/bvar/tst.stackdepth-profile.r.p
new file mode 120000
index 000000000..e50f12822
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-profile.r.p
@@ -0,0 +1 @@
+check_stackdepth_to_stack.awk
\ No newline at end of file
-- 
2.43.5

[PATCH 10/14] Test: caller and stackdepth tests for sched provider

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 .../variables/bvar/tst.caller-sched.d         | 31 +++++++++++++++++
 .../variables/bvar/tst.caller-sched.r         |  1 +
 .../variables/bvar/tst.caller-sched.r.p       |  1 +
 .../variables/bvar/tst.stackdepth-sched.d     | 33 +++++++++++++++++++
 .../variables/bvar/tst.stackdepth-sched.r     |  1 +
 .../variables/bvar/tst.stackdepth-sched.r.p   |  1 +
 6 files changed, 68 insertions(+)
 create mode 100644 test/unittest/variables/bvar/tst.caller-sched.d
 create mode 100644 test/unittest/variables/bvar/tst.caller-sched.r
 create mode 120000 test/unittest/variables/bvar/tst.caller-sched.r.p
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-sched.d
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-sched.r
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-sched.r.p

diff --git a/test/unittest/variables/bvar/tst.caller-sched.d b/test/unittest/variables/bvar/tst.caller-sched.d
new file mode 100644
index 000000000..087571531
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-sched.d
@@ -0,0 +1,31 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+/*
+ * @@trigger: periodic_output
+ */
+
+#pragma D option quiet
+
+
+/*
+ * Some probes are implemented in terms of rawtp probes.  Avoid them
+ * due to stack-skip issues on older kernels.
+ */
+sched:::dequeue,
+sched:::enqueue,
+sched:::tick
+{
+	stack(2);
+	sym(caller);
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.caller-sched.r b/test/unittest/variables/bvar/tst.caller-sched.r
new file mode 100644
index 000000000..2e9ba477f
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-sched.r
@@ -0,0 +1 @@
+success
diff --git a/test/unittest/variables/bvar/tst.caller-sched.r.p b/test/unittest/variables/bvar/tst.caller-sched.r.p
new file mode 120000
index 000000000..954ca96aa
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-sched.r.p
@@ -0,0 +1 @@
+check_caller_to_stack2.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth-sched.d b/test/unittest/variables/bvar/tst.stackdepth-sched.d
new file mode 100644
index 000000000..d3e9e18d5
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-sched.d
@@ -0,0 +1,33 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+/*
+ * @@trigger: periodic_output
+ */
+
+#pragma D option quiet
+
+
+/*
+ * Some probes are implemented in terms of rawtp probes.  Avoid them
+ * due to stack-skip issues on older kernels.
+ */
+sched:::dequeue,
+sched:::enqueue,
+sched:::tick
+{
+	printf("DEPTH %d\n", stackdepth);
+	printf("TRACE BEGIN\n");
+	stack(100);
+	printf("TRACE END\n");
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.stackdepth-sched.r b/test/unittest/variables/bvar/tst.stackdepth-sched.r
new file mode 100644
index 000000000..3bd29b8ed
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-sched.r
@@ -0,0 +1 @@
+Stack depth OK
diff --git a/test/unittest/variables/bvar/tst.stackdepth-sched.r.p b/test/unittest/variables/bvar/tst.stackdepth-sched.r.p
new file mode 120000
index 000000000..e50f12822
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-sched.r.p
@@ -0,0 +1 @@
+check_stackdepth_to_stack.awk
\ No newline at end of file
-- 
2.43.5

[PATCH 11/14] Test: caller and stackdepth tests for proc provider

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 .../unittest/variables/bvar/tst.caller-proc.d | 34 ++++++++++++++++++
 .../unittest/variables/bvar/tst.caller-proc.r |  1 +
 .../variables/bvar/tst.caller-proc.r.p        |  1 +
 .../variables/bvar/tst.stackdepth-proc.d      | 36 +++++++++++++++++++
 .../variables/bvar/tst.stackdepth-proc.r      |  1 +
 .../variables/bvar/tst.stackdepth-proc.r.p    |  1 +
 6 files changed, 74 insertions(+)
 create mode 100644 test/unittest/variables/bvar/tst.caller-proc.d
 create mode 100644 test/unittest/variables/bvar/tst.caller-proc.r
 create mode 120000 test/unittest/variables/bvar/tst.caller-proc.r.p
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-proc.d
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-proc.r
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-proc.r.p

diff --git a/test/unittest/variables/bvar/tst.caller-proc.d b/test/unittest/variables/bvar/tst.caller-proc.d
new file mode 100644
index 000000000..7c3557e2c
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-proc.d
@@ -0,0 +1,34 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+/*
+ * @@trigger: periodic_output
+ */
+
+#pragma D option quiet
+
+/*
+ * Some probes are implemented in terms of rawtp probes.  Avoid them
+ * due to stack-skip issues on older kernels.
+ */
+proc:::exec,
+proc:::exec-failure,
+proc:::exec-success,
+proc:::lwp-start,
+proc:::signal-clear,
+proc:::signal-send,
+proc:::start
+{
+	stack(2);
+	sym(caller);
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.caller-proc.r b/test/unittest/variables/bvar/tst.caller-proc.r
new file mode 100644
index 000000000..2e9ba477f
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-proc.r
@@ -0,0 +1 @@
+success
diff --git a/test/unittest/variables/bvar/tst.caller-proc.r.p b/test/unittest/variables/bvar/tst.caller-proc.r.p
new file mode 120000
index 000000000..954ca96aa
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-proc.r.p
@@ -0,0 +1 @@
+check_caller_to_stack2.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth-proc.d b/test/unittest/variables/bvar/tst.stackdepth-proc.d
new file mode 100644
index 000000000..75f720370
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-proc.d
@@ -0,0 +1,36 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+/*
+ * @@trigger: periodic_output
+ */
+
+#pragma D option quiet
+
+/*
+ * Some probes are implemented in terms of rawtp probes.  Avoid them
+ * due to stack-skip issues on older kernels.
+ */
+proc:::exec,
+proc:::exec-failure,
+proc:::exec-success,
+proc:::lwp-start,
+proc:::signal-clear,
+proc:::signal-send,
+proc:::start
+{
+	printf("DEPTH %d\n", stackdepth);
+	printf("TRACE BEGIN\n");
+	stack(100);
+	printf("TRACE END\n");
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.stackdepth-proc.r b/test/unittest/variables/bvar/tst.stackdepth-proc.r
new file mode 100644
index 000000000..3bd29b8ed
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-proc.r
@@ -0,0 +1 @@
+Stack depth OK
diff --git a/test/unittest/variables/bvar/tst.stackdepth-proc.r.p b/test/unittest/variables/bvar/tst.stackdepth-proc.r.p
new file mode 120000
index 000000000..e50f12822
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-proc.r.p
@@ -0,0 +1 @@
+check_stackdepth_to_stack.awk
\ No newline at end of file
-- 
2.43.5

[PATCH 12/14] Test: caller and stackdepth tests for rawfbt provider

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 .../variables/bvar/tst.caller-rawfbt.d        | 25 +++++++++++++++++
 .../variables/bvar/tst.caller-rawfbt.r        |  1 +
 .../variables/bvar/tst.caller-rawfbt.r.p      |  1 +
 .../variables/bvar/tst.stackdepth-rawfbt.d    | 27 +++++++++++++++++++
 .../variables/bvar/tst.stackdepth-rawfbt.r    |  1 +
 .../variables/bvar/tst.stackdepth-rawfbt.r.p  |  1 +
 6 files changed, 56 insertions(+)
 create mode 100644 test/unittest/variables/bvar/tst.caller-rawfbt.d
 create mode 100644 test/unittest/variables/bvar/tst.caller-rawfbt.r
 create mode 120000 test/unittest/variables/bvar/tst.caller-rawfbt.r.p
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-rawfbt.d
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-rawfbt.r
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-rawfbt.r.p

diff --git a/test/unittest/variables/bvar/tst.caller-rawfbt.d b/test/unittest/variables/bvar/tst.caller-rawfbt.d
new file mode 100644
index 000000000..08ab3eba1
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-rawfbt.d
@@ -0,0 +1,25 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+/*
+ * @@trigger: periodic_output
+ */
+
+#pragma D option quiet
+
+rawfbt::ksys_write:entry
+/pid == $target/
+{
+	stack(2);
+	sym(caller);
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.caller-rawfbt.r b/test/unittest/variables/bvar/tst.caller-rawfbt.r
new file mode 100644
index 000000000..2e9ba477f
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-rawfbt.r
@@ -0,0 +1 @@
+success
diff --git a/test/unittest/variables/bvar/tst.caller-rawfbt.r.p b/test/unittest/variables/bvar/tst.caller-rawfbt.r.p
new file mode 120000
index 000000000..954ca96aa
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-rawfbt.r.p
@@ -0,0 +1 @@
+check_caller_to_stack2.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth-rawfbt.d b/test/unittest/variables/bvar/tst.stackdepth-rawfbt.d
new file mode 100644
index 000000000..15a9991cb
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-rawfbt.d
@@ -0,0 +1,27 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+/*
+ * @@trigger: periodic_output
+ */
+
+#pragma D option quiet
+
+rawfbt::ksys_write:entry
+/pid == $target/
+{
+	printf("DEPTH %d\n", stackdepth);
+	printf("TRACE BEGIN\n");
+	stack(100);
+	printf("TRACE END\n");
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.stackdepth-rawfbt.r b/test/unittest/variables/bvar/tst.stackdepth-rawfbt.r
new file mode 100644
index 000000000..3bd29b8ed
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-rawfbt.r
@@ -0,0 +1 @@
+Stack depth OK
diff --git a/test/unittest/variables/bvar/tst.stackdepth-rawfbt.r.p b/test/unittest/variables/bvar/tst.stackdepth-rawfbt.r.p
new file mode 120000
index 000000000..e50f12822
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-rawfbt.r.p
@@ -0,0 +1 @@
+check_stackdepth_to_stack.awk
\ No newline at end of file
-- 
2.43.5

[PATCH 13/14] Test: caller and stackdepth tests for io provider

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 test/unittest/variables/bvar/tst.caller-io.r  |  1 +
 .../unittest/variables/bvar/tst.caller-io.r.p |  1 +
 test/unittest/variables/bvar/tst.caller-io.sh | 37 ++++++++++++++++++
 test/unittest/variables/bvar/tst.caller-io.x  |  1 +
 .../variables/bvar/tst.stackdepth-io.r        |  1 +
 .../variables/bvar/tst.stackdepth-io.r.p      |  1 +
 .../variables/bvar/tst.stackdepth-io.sh       | 39 +++++++++++++++++++
 .../variables/bvar/tst.stackdepth-io.x        |  1 +
 8 files changed, 82 insertions(+)
 create mode 100644 test/unittest/variables/bvar/tst.caller-io.r
 create mode 120000 test/unittest/variables/bvar/tst.caller-io.r.p
 create mode 100755 test/unittest/variables/bvar/tst.caller-io.sh
 create mode 120000 test/unittest/variables/bvar/tst.caller-io.x
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-io.r
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-io.r.p
 create mode 100755 test/unittest/variables/bvar/tst.stackdepth-io.sh
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-io.x

diff --git a/test/unittest/variables/bvar/tst.caller-io.r b/test/unittest/variables/bvar/tst.caller-io.r
new file mode 100644
index 000000000..2e9ba477f
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-io.r
@@ -0,0 +1 @@
+success
diff --git a/test/unittest/variables/bvar/tst.caller-io.r.p b/test/unittest/variables/bvar/tst.caller-io.r.p
new file mode 120000
index 000000000..954ca96aa
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-io.r.p
@@ -0,0 +1 @@
+check_caller_to_stack2.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.caller-io.sh b/test/unittest/variables/bvar/tst.caller-io.sh
new file mode 100755
index 000000000..876ea70e3
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-io.sh
@@ -0,0 +1,37 @@
+#!/bin/bash
+#
+# Oracle Linux DTrace.
+# Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+# Licensed under the Universal Permissive License v 1.0 as shown at
+# http://oss.oracle.com/licenses/upl.
+
+dtrace=$1
+nblocks=1024
+filesize=$((1024*$nblocks))
+fsoptions="defaults,atime,diratime,nosuid,nodev"
+iodir=$tmpdir/tst-caller-io.$$
+tempfile=`mktemp -u -p $iodir`
+
+trap "umount $iodir; rmdir $iodir; rm -f $iodir.img" QUIT EXIT
+
+# create loopback file system
+dd if=/dev/zero of=$iodir.img bs=1024 count=$((300*$nblocks)) status=none
+mkfs.xfs $iodir.img > /dev/null
+mkdir $iodir
+test/triggers/io-mount-local.sh $iodir xfs $fsoptions
+
+$dtrace $dt_flags -c "test/triggers/doio.sh $tempfile $filesize test/triggers/io-mount-local.sh $iodir xfs $fsoptions" -qn '
+io:::
+{
+	stack(2);
+	sym(caller);
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}'
+
+exit $?
diff --git a/test/unittest/variables/bvar/tst.caller-io.x b/test/unittest/variables/bvar/tst.caller-io.x
new file mode 120000
index 000000000..37b3ef9f8
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-io.x
@@ -0,0 +1 @@
+skip_rawtp_old.x
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth-io.r b/test/unittest/variables/bvar/tst.stackdepth-io.r
new file mode 100644
index 000000000..3bd29b8ed
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-io.r
@@ -0,0 +1 @@
+Stack depth OK
diff --git a/test/unittest/variables/bvar/tst.stackdepth-io.r.p b/test/unittest/variables/bvar/tst.stackdepth-io.r.p
new file mode 120000
index 000000000..e50f12822
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-io.r.p
@@ -0,0 +1 @@
+check_stackdepth_to_stack.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth-io.sh b/test/unittest/variables/bvar/tst.stackdepth-io.sh
new file mode 100755
index 000000000..3bbf38a47
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-io.sh
@@ -0,0 +1,39 @@
+#!/bin/bash
+#
+# Oracle Linux DTrace.
+# Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+# Licensed under the Universal Permissive License v 1.0 as shown at
+# http://oss.oracle.com/licenses/upl.
+
+dtrace=$1
+nblocks=1024
+filesize=$((1024*$nblocks))
+fsoptions="defaults,atime,diratime,nosuid,nodev"
+iodir=$tmpdir/tst-stackdepth-io.$$
+tempfile=`mktemp -u -p $iodir`
+
+trap "umount $iodir; rmdir $iodir; rm -f $iodir.img" QUIT EXIT
+
+# create loopback file system
+dd if=/dev/zero of=$iodir.img bs=1024 count=$((300*$nblocks)) status=none
+mkfs.xfs $iodir.img > /dev/null
+mkdir $iodir
+test/triggers/io-mount-local.sh $iodir xfs $fsoptions
+
+$dtrace $dt_flags -c "test/triggers/doio.sh $tempfile $filesize test/triggers/io-mount-local.sh $iodir xfs $fsoptions" -qn '
+io:::
+{
+	printf("DEPTH %d\n", stackdepth);
+	printf("TRACE BEGIN\n");
+	stack(100);
+	printf("TRACE END\n");
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}'
+
+exit $?
diff --git a/test/unittest/variables/bvar/tst.stackdepth-io.x b/test/unittest/variables/bvar/tst.stackdepth-io.x
new file mode 120000
index 000000000..37b3ef9f8
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-io.x
@@ -0,0 +1 @@
+skip_rawtp_old.x
\ No newline at end of file
-- 
2.43.5

[PATCH 14/14] Test: caller and stackdepth tests for lockstat provider

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 .../variables/bvar/skip_lockstat_5.10.x       | 13 ++++++++++
 .../variables/bvar/tst.caller-lockstat.d      | 22 +++++++++++++++++
 .../variables/bvar/tst.caller-lockstat.r      |  1 +
 .../variables/bvar/tst.caller-lockstat.r.p    |  1 +
 .../variables/bvar/tst.caller-lockstat.t      |  3 +++
 .../variables/bvar/tst.caller-lockstat.x      |  1 +
 .../variables/bvar/tst.stackdepth-lockstat.d  | 24 +++++++++++++++++++
 .../variables/bvar/tst.stackdepth-lockstat.r  |  1 +
 .../bvar/tst.stackdepth-lockstat.r.p          |  1 +
 .../variables/bvar/tst.stackdepth-lockstat.t  |  3 +++
 .../variables/bvar/tst.stackdepth-lockstat.x  |  1 +
 11 files changed, 71 insertions(+)
 create mode 100755 test/unittest/variables/bvar/skip_lockstat_5.10.x
 create mode 100644 test/unittest/variables/bvar/tst.caller-lockstat.d
 create mode 100644 test/unittest/variables/bvar/tst.caller-lockstat.r
 create mode 120000 test/unittest/variables/bvar/tst.caller-lockstat.r.p
 create mode 100755 test/unittest/variables/bvar/tst.caller-lockstat.t
 create mode 120000 test/unittest/variables/bvar/tst.caller-lockstat.x
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-lockstat.d
 create mode 100644 test/unittest/variables/bvar/tst.stackdepth-lockstat.r
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-lockstat.r.p
 create mode 100755 test/unittest/variables/bvar/tst.stackdepth-lockstat.t
 create mode 120000 test/unittest/variables/bvar/tst.stackdepth-lockstat.x

diff --git a/test/unittest/variables/bvar/skip_lockstat_5.10.x b/test/unittest/variables/bvar/skip_lockstat_5.10.x
new file mode 100755
index 000000000..146443fd2
--- /dev/null
+++ b/test/unittest/variables/bvar/skip_lockstat_5.10.x
@@ -0,0 +1,13 @@
+#!/bin/bash
+
+read MAJOR MINOR <<< `uname -r | grep -Eo '^[0-9]+\.[0-9]+' | tr '.' ' '`
+
+if [ $MAJOR -gt 5 ]; then
+	exit 0
+fi
+if [ $MAJOR -eq 5 -a $MINOR -ge 10 ]; then
+	exit 0
+fi
+
+echo "lockstat disabled prior to 5.10"
+exit 1
diff --git a/test/unittest/variables/bvar/tst.caller-lockstat.d b/test/unittest/variables/bvar/tst.caller-lockstat.d
new file mode 100644
index 000000000..95eb08578
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-lockstat.d
@@ -0,0 +1,22 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+#pragma D option quiet
+
+lockstat:::*acquire
+/pid == $target/
+{
+	stack(2);
+	sym(caller);
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.caller-lockstat.r b/test/unittest/variables/bvar/tst.caller-lockstat.r
new file mode 100644
index 000000000..2e9ba477f
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-lockstat.r
@@ -0,0 +1 @@
+success
diff --git a/test/unittest/variables/bvar/tst.caller-lockstat.r.p b/test/unittest/variables/bvar/tst.caller-lockstat.r.p
new file mode 120000
index 000000000..954ca96aa
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-lockstat.r.p
@@ -0,0 +1 @@
+check_caller_to_stack2.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.caller-lockstat.t b/test/unittest/variables/bvar/tst.caller-lockstat.t
new file mode 100755
index 000000000..fec0d2715
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-lockstat.t
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+sleep 10s
diff --git a/test/unittest/variables/bvar/tst.caller-lockstat.x b/test/unittest/variables/bvar/tst.caller-lockstat.x
new file mode 120000
index 000000000..539f14255
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.caller-lockstat.x
@@ -0,0 +1 @@
+skip_lockstat_5.10.x
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth-lockstat.d b/test/unittest/variables/bvar/tst.stackdepth-lockstat.d
new file mode 100644
index 000000000..46f937067
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-lockstat.d
@@ -0,0 +1,24 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+#pragma D option quiet
+
+lockstat:::*acquire
+/pid == $target/
+{
+	printf("DEPTH %d\n", stackdepth);
+	printf("TRACE BEGIN\n");
+	stack(100);
+	printf("TRACE END\n");
+	exit(0);
+}
+
+ERROR
+{
+	printf("error encountered\n");
+	exit(1);
+}
diff --git a/test/unittest/variables/bvar/tst.stackdepth-lockstat.r b/test/unittest/variables/bvar/tst.stackdepth-lockstat.r
new file mode 100644
index 000000000..3bd29b8ed
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-lockstat.r
@@ -0,0 +1 @@
+Stack depth OK
diff --git a/test/unittest/variables/bvar/tst.stackdepth-lockstat.r.p b/test/unittest/variables/bvar/tst.stackdepth-lockstat.r.p
new file mode 120000
index 000000000..e50f12822
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-lockstat.r.p
@@ -0,0 +1 @@
+check_stackdepth_to_stack.awk
\ No newline at end of file
diff --git a/test/unittest/variables/bvar/tst.stackdepth-lockstat.t b/test/unittest/variables/bvar/tst.stackdepth-lockstat.t
new file mode 100755
index 000000000..fec0d2715
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-lockstat.t
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+sleep 10s
diff --git a/test/unittest/variables/bvar/tst.stackdepth-lockstat.x b/test/unittest/variables/bvar/tst.stackdepth-lockstat.x
new file mode 120000
index 000000000..539f14255
--- /dev/null
+++ b/test/unittest/variables/bvar/tst.stackdepth-lockstat.x
@@ -0,0 +1 @@
+skip_lockstat_5.10.x
\ No newline at end of file
-- 
2.43.5

Re: [PATCH 01/14] Fix stack-skip counts for caller and stackdepth

[Nick Alcock]

On 22 May 2025, eugene loh told this:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Apparently, when we call the BPF get_stack() helper function,
> it generally knows how many frames to skip to get the real kernel
> stack.  For fentry/fexit, however, this is apparently not the case,
> and commit bc65cb44d
> ("cg: allow providers to specify a skip count for stack retrieval")
> added the ability to skip frames for fentry/fexit probes.
>
> When this "skip" is needed, however, it must must be even deeper
> when we descend further frames, such as when we call dt_bpf_*()
> precompiled functions.

Ack.

> Add this support for dt_bpf_caller() and dt_bpf_stackdepth().
> That is, if there are stack-skip frames, skip yet one more frame
> when inside a bpf/get_bvar.c function.

The general approach here seems sound, but this confuses me. Is the
assumption that if the skip is not needed, get_stack() will itself know
how many frames to skip, and will skip dt_dt_bvar_caller() for you?

(So a zero skip is actually a "the system knows", and a nonzero skip
causes the system to not skip anything, so you have to do all the
skipping yourself?

This behaviour is not documented in bpf_get_stack()'s documentation,
which absolutely does not mean it doesn't do it...)

> Note that we declare the skip count volatile.  The compiler might
> optimize code that uses the STACK_SKIP value, but we will subsequently
> perform relocations that adjust this value.

... why doesn't this apply to every other extern global variable in
get_bvar()? They're all similarly relocated...

> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> ---
>  bpf/get_bvar.c | 17 ++++++++++++++---
>  1 file changed, 14 insertions(+), 3 deletions(-)
>
> diff --git a/bpf/get_bvar.c b/bpf/get_bvar.c
> index 9625e764e..99a6503d5 100644
> --- a/bpf/get_bvar.c
> +++ b/bpf/get_bvar.c
> @@ -53,12 +53,17 @@ noinline uint64_t dt_bvar_args(const dt_dctx_t *dctx, uint32_t idx)
>  
>  noinline uint64_t dt_bvar_caller(const dt_dctx_t *dctx)
>  {
> -	uint64_t	buf[2] = { 0, };
> +	uint64_t	buf[3] = { 0, };
> +	volatile uint64_t
> +			skip = (uint64_t)(&STACK_SKIP);
>  
>  	if (bpf_get_stack(dctx->ctx, buf, sizeof(buf),
> -			  (uint64_t)(&STACK_SKIP) & BPF_F_SKIP_FIELD_MASK) < 0)
> +			  skip & BPF_F_SKIP_FIELD_MASK) < 0)
>  		return 0;
>  
> +	/* If we had to skip any frames, account for the dt_bvar_caller() frame. */
> +	if (skip)
> +		return buf[2];
>  	return buf[1];
>  }
>  
> @@ -203,9 +208,11 @@ noinline uint64_t dt_bvar_stackdepth(const dt_dctx_t *dctx)
>  	uint32_t	bufsiz = (uint32_t) (uint64_t) (&STKSIZ);
>  	char		*buf = dctx->mem + (uint64_t)(&STACK_OFF);
>  	uint64_t	retv;
> +	volatile uint64_t
> +			skip = (uint64_t)(&STACK_SKIP);
>  
>  	retv = bpf_get_stack(dctx->ctx, buf, bufsiz,
> -			     (uint64_t)(&STACK_SKIP) & BPF_F_SKIP_FIELD_MASK);
> +			     skip & BPF_F_SKIP_FIELD_MASK);
>  	if (retv < 0)
>  		return error(dctx, DTRACEFLT_BADSTACK, 0 /* FIXME */);
>  
> @@ -217,7 +224,11 @@ noinline uint64_t dt_bvar_stackdepth(const dt_dctx_t *dctx)
>  	 * If retv==bufsiz, presumably the stack is larger than what we
>  	 * can retrieve.  But it's also possible that the buffer was exactly
>  	 * large enough.  So, leave it to the user to interpret the result.
> +	 *
> +	 * If we had to skip any frames, account for the dt_bvar_stackdepth() frame.
>  	 */
> +	if (skip)
> +		return retv / sizeof(uint64_t) - 1;
>  	return retv / sizeof(uint64_t);
>  }

-- 
NULL && (void)

Re: [PATCH 02/14] Add stack-skip frame count for rawtp provider

[Nick Alcock]

On 22 May 2025, eugene loh outgrape:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

(looking back, 4 is the right figure. If it's wrong, the tests will tell
us, anyway -- presumably we hope that the underlying bug is fixed before
it changes in the future and requires us to make this a
per-kernel-release value...)

-- 
NULL && (void)

Re: [DTrace-devel] [PATCH 03/14] Test: remove unnecessary "unstable" tag

[Nick Alcock]

On 22 May 2025, eugene loh spake thusly:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

-- 
NULL && (void)

Re: [DTrace-devel] [PATCH 04/14] Test: caller and stackdepth tests for fbt provider

[Nick Alcock]

On 22 May 2025, eugene loh uttered the following:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> We will introduce a set of tests for the caller and stackdepth
> built-in variables for a wide selection of providers.
>
> For the caller test, we will essentially call
>         stack(2);
>         sym(caller);
> and then compare the caller to the second stack frame using
> the new script check_caller_to_stack2.awk.
>
> For the stackdepth test, we will essentially call
>         printf("%d\n", stackdepth);
>         stack();
> and then compare the stackdepth to the reported frames using
> the new script check_stackdepth_to_stack.awk.
>
> In this patch, introduce tests for the fbt provider, along with
> the support scripts they need.  Subsequent patches will handle
> other providers.
>
> The old tst.caller2.d and tst.stackdepth2.d, which tested fbt,
> become obsolete.  Remove them.

Nice! My only caveat (easily fixed) is that the old tests had header
comments saying what they were testing, while the new ones don't.
(Yes, the awk scripts do make it clearer, but still...)

> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

-- 
NULL && (void)

Re: [PATCH 05/14] Test: caller and stackdepth tests for dtrace provider

[Nick Alcock]

On 22 May 2025, eugene loh said:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Also, a few old tests, which tested caller and stackdepth using the
> dtrace provider, are now superfluous given the new, provider-named
> tests.  The old tests were extremely lenient -- e.g., simply
> checking that these built-in variables were not -1, even though
> both variables are unsigned anyhow!
>
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

(with the same comments regarding the old tests having a description of
what they were testing, and could the new ones do the same?)

-- 
NULL && (void)

Re: [PATCH 06/14] Test: caller and stackdepth tests for rawtp provider

[Nick Alcock]

On 22 May 2025, eugene loh uttered the following:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Also, add skip_rawtp_old.x, to skip rawtp testing on older kernels
> for the reasons described in the file.

> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

-- 
NULL && (void)

Re: [DTrace-devel] [PATCH 07/14] Test: caller and stackdepth tests for cpc provider

[Nick Alcock]

On 22 May 2025, eugene loh outgrape:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

> +cpc:::cpu_clock-all-100000000

1/10th of a second: seems legit. (But having it use nanoseconds makes
probe names hard to read! Yes, I know we have no choice here.)

-- 
NULL && (void)

Re: [DTrace-devel] [PATCH 08/14] Test: caller and stackdepth tests for ip provider

[Nick Alcock]

On 22 May 2025, eugene loh spake thusly:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

with the microscopic caveat below.

> diff --git a/test/unittest/variables/bvar/perlping.pl b/test/unittest/variables/bvar/perlping.pl
> new file mode 100755
> index 000000000..97c9b09e9
> --- /dev/null
> +++ b/test/unittest/variables/bvar/perlping.pl
> @@ -0,0 +1,8 @@
> +#!/usr/bin/perl -w
> +# Oracle Linux DTrace.
> +# Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.
> +# Licensed under the Universal Permissive License v 1.0 as shown at
> +# http://oss.oracle.com/licenses/upl.
> +use Net::Ping;
> +my $p = Net::Ping->new(${ARGV[0]}, 5, 56);
> +$p->ping(${ARGV[1]});

Is there some reason we can't just use $testdir/../../ip/perlping.pl and
avoid duplicating this?

-- 
NULL && (void)

Re: [DTrace-devel] [PATCH 09/14] Test: caller and stackdepth tests for profile provider

[Nick Alcock]

On 22 May 2025, eugene loh verbalised:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

(usual caveats regarding comments saying what these tests are doing...)

-- 
NULL && (void)

Re: [PATCH 10/14] Test: caller and stackdepth tests for sched provider

[Nick Alcock]

On 22 May 2025, eugene loh uttered the following:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

-- 
NULL && (void)

Re: [PATCH 11/14] Test: caller and stackdepth tests for proc provider

[Nick Alcock]

On 22 May 2025, eugene loh told this:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

-- 
NULL && (void)

Re: [DTrace-devel] [PATCH 12/14] Test: caller and stackdepth tests for rawfbt provider

[Nick Alcock]

On 22 May 2025, eugene loh spake thusly:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

I note the lack of any sort of skipping on older kernels. Do the
concerns about older kernels not apply here?

If not, and this was intentional...

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

-- 
NULL && (void)

Re: [DTrace-devel] [PATCH 13/14] Test: caller and stackdepth tests for io provider

[Nick Alcock]

On 22 May 2025, eugene loh spake thusly:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

> --- /dev/null
> +++ b/test/unittest/variables/bvar/tst.caller-io.sh
> @@ -0,0 +1,37 @@
> +#!/bin/bash
> +#
> +# Oracle Linux DTrace.
> +# Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
> +# Licensed under the Universal Permissive License v 1.0 as shown at
> +# http://oss.oracle.com/licenses/upl.
> +
> +dtrace=$1
> +nblocks=1024
> +filesize=$((1024*$nblocks))
> +fsoptions="defaults,atime,diratime,nosuid,nodev"
> +iodir=$tmpdir/tst-caller-io.$$
> +tempfile=`mktemp -u -p $iodir`
> +
> +trap "umount $iodir; rmdir $iodir; rm -f $iodir.img" QUIT EXIT
> +
> +# create loopback file system

We have quite a lot of these I/O-trigger filesystem things now (this is
our sixth and seventh). Maybe we should think about refactoring the
mount-and-mkfs-and-dd-and-umount bit out into shared code at some
point...

-- 
NULL && (void)

Re: [PATCH 14/14] Test: caller and stackdepth tests for lockstat provider

[Nick Alcock]

On 22 May 2025, eugene loh spake thusly:

> From: Eugene Loh <eugene.loh@oracle.com>
>
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

(I guess we can assume that as long as it has any trigger at all, it's
going to be firing lockstats like mad.)

-- 
NULL && (void)

Re: [PATCH 11/14] Test: caller and stackdepth tests for proc provider

[Eugene Loh]

Thanks.

I'm getting to these out of order, but I figured I'd mention that I'm 
going to turn the capitalized "Test:" into "test:" for this series of 
patches to conform to the style in related patches.

On 6/13/25 10:53, Nick Alcock wrote:
> On 22 May 2025, eugene loh told this:
>
>> From: Eugene Loh <eugene.loh@oracle.com>
>>
>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
>

Re: [PATCH 01/14] Fix stack-skip counts for caller and stackdepth

[Eugene Loh]

On 6/13/25 10:33, Nick Alcock wrote:

> On 22 May 2025, eugene loh told this:
>
>> From: Eugene Loh <eugene.loh@oracle.com>
>>
>> Apparently, when we call the BPF get_stack() helper function,
>> it generally knows how many frames to skip to get the real kernel
>> stack.  For fentry/fexit, however, this is apparently not the case,
>> and commit bc65cb44d
>> ("cg: allow providers to specify a skip count for stack retrieval")
>> added the ability to skip frames for fentry/fexit probes.
>>
>> When this "skip" is needed, however, it must must be even deeper
>> when we descend further frames, such as when we call dt_bpf_*()
>> precompiled functions.
> Ack.
>
>> Add this support for dt_bpf_caller() and dt_bpf_stackdepth().
>> That is, if there are stack-skip frames, skip yet one more frame
>> when inside a bpf/get_bvar.c function.
> The general approach here seems sound, but this confuses me. Is the
> assumption that if the skip is not needed, get_stack() will itself know
> how many frames to skip, and will skip dt_dt_bvar_caller() for you?

Yes.  I confirmed this with tests.

> (So a zero skip is actually a "the system knows", and a nonzero skip
> causes the system to not skip anything, so you have to do all the
> skipping yourself?

Yes.

> This behaviour is not documented in bpf_get_stack()'s documentation,
> which absolutely does not mean it doesn't do it...)

Yup:  I did not learn this from documentation!

>> Note that we declare the skip count volatile.  The compiler might
>> optimize code that uses the STACK_SKIP value, but we will subsequently
>> perform relocations that adjust this value.
> ... why doesn't this apply to every other extern global variable in
> get_bvar()? They're all similarly relocated...

Right.  There is potentially a broader problem.  But we simply do not 
have evidence of misbehavior in other cases.  Ruggedizing other cases 
could be the subject of a different patch.

The problem in this case is that the compiler seems to assume 
&symbol!=0, which is reasonable except that we violate that behavior for 
our relocation tricks.

Consider the C code:

     uint64_t dt_bvar_stackdepth(const dt_dctx_t *dctx)
     {
         uint32_t          bufsiz = (uint32_t) (uint64_t) (&STKSIZ);
         char              *buf = dctx->mem + (uint64_t)(&STACK_OFF);
         uint64_t          retv;
         volatile uint64_t skip = (uint64_t)(&STACK_SKIP);

         retv = bpf_get_stack(dctx->ctx,
                              buf,
                              bufsiz,
                              skip & BPF_F_SKIP_FIELD_MASK);

         if (skip)
                 return retv / sizeof(uint64_t) - 1;      // branch "A"
         return retv / sizeof(uint64_t);                  // branch "B"
     }

If you omit "volatile", the compiler assumes &STACK_SKIP!=0. The emitted 
code has:

     *)  no run-time "if (skip)" check

     *)  no code for branch "B"

     *)  only code for branch "A"

If you include "volatile", however, the compiler caches &STACK_SKIP on 
the BPF stack and later performs a run-time check on its value to 
correctly execute either branch "A" or branch "B".

Incidentally, I looked at this using the function like this:

     profile:::tick-1s,
     fbt:vmlinux:ksys_write:entry
     {
        trace(stackdepth);
     }

Notice that the profile probe has STACK_SKIP==0 while the fbt probe has 
STACK_SKIP==4.

I looked at the disassembly in three ways:

     *)  as emitted by the compiler:  objdump -d build/bpf--get_bvar.o
        (not too interesting but establishes a baseline;  e.g., that a 
branch is missing)

     *)  after relocation for tick-1s (-xdisasm=8 -S)

     *)  after relocation for fbt::ksys_write:entry (-xdisasm=8 -S)

The latter two confirm the relocations are correct.

>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
>> ---
>>   bpf/get_bvar.c | 17 ++++++++++++++---
>>   1 file changed, 14 insertions(+), 3 deletions(-)
>>
>> diff --git a/bpf/get_bvar.c b/bpf/get_bvar.c
>> index 9625e764e..99a6503d5 100644
>> --- a/bpf/get_bvar.c
>> +++ b/bpf/get_bvar.c
>> @@ -53,12 +53,17 @@ noinline uint64_t dt_bvar_args(const dt_dctx_t *dctx, uint32_t idx)
>>   
>>   noinline uint64_t dt_bvar_caller(const dt_dctx_t *dctx)
>>   {
>> -	uint64_t	buf[2] = { 0, };
>> +	uint64_t	buf[3] = { 0, };
>> +	volatile uint64_t
>> +			skip = (uint64_t)(&STACK_SKIP);
>>   
>>   	if (bpf_get_stack(dctx->ctx, buf, sizeof(buf),
>> -			  (uint64_t)(&STACK_SKIP) & BPF_F_SKIP_FIELD_MASK) < 0)
>> +			  skip & BPF_F_SKIP_FIELD_MASK) < 0)
>>   		return 0;
>>   
>> +	/* If we had to skip any frames, account for the dt_bvar_caller() frame. */
>> +	if (skip)
>> +		return buf[2];
>>   	return buf[1];
>>   }
>>   
>> @@ -203,9 +208,11 @@ noinline uint64_t dt_bvar_stackdepth(const dt_dctx_t *dctx)
>>   	uint32_t	bufsiz = (uint32_t) (uint64_t) (&STKSIZ);
>>   	char		*buf = dctx->mem + (uint64_t)(&STACK_OFF);
>>   	uint64_t	retv;
>> +	volatile uint64_t
>> +			skip = (uint64_t)(&STACK_SKIP);
>>   
>>   	retv = bpf_get_stack(dctx->ctx, buf, bufsiz,
>> -			     (uint64_t)(&STACK_SKIP) & BPF_F_SKIP_FIELD_MASK);
>> +			     skip & BPF_F_SKIP_FIELD_MASK);
>>   	if (retv < 0)
>>   		return error(dctx, DTRACEFLT_BADSTACK, 0 /* FIXME */);
>>   
>> @@ -217,7 +224,11 @@ noinline uint64_t dt_bvar_stackdepth(const dt_dctx_t *dctx)
>>   	 * If retv==bufsiz, presumably the stack is larger than what we
>>   	 * can retrieve.  But it's also possible that the buffer was exactly
>>   	 * large enough.  So, leave it to the user to interpret the result.
>> +	 *
>> +	 * If we had to skip any frames, account for the dt_bvar_stackdepth() frame.
>>   	 */
>> +	if (skip)
>> +		return retv / sizeof(uint64_t) - 1;
>>   	return retv / sizeof(uint64_t);
>>   }

Re: [DTrace-devel] [PATCH 04/14] Test: caller and stackdepth tests for fbt provider

[Eugene Loh]

On 6/13/25 10:40, Nick Alcock wrote:

> On 22 May 2025, eugene loh uttered the following:
>
>> From: Eugene Loh <eugene.loh@oracle.com>
>>
>> We will introduce a set of tests for the caller and stackdepth
>> built-in variables for a wide selection of providers.
>>
>> For the caller test, we will essentially call
>>          stack(2);
>>          sym(caller);
>> and then compare the caller to the second stack frame using
>> the new script check_caller_to_stack2.awk.
>>
>> For the stackdepth test, we will essentially call
>>          printf("%d\n", stackdepth);
>>          stack();
>> and then compare the stackdepth to the reported frames using
>> the new script check_stackdepth_to_stack.awk.
>>
>> In this patch, introduce tests for the fbt provider, along with
>> the support scripts they need.  Subsequent patches will handle
>> other providers.
>>
>> The old tst.caller2.d and tst.stackdepth2.d, which tested fbt,
>> become obsolete.  Remove them.
> Nice! My only caveat (easily fixed) is that the old tests had header
> comments saying what they were testing, while the new ones don't.
> (Yes, the awk scripts do make it clearer, but still...)

Got it.  I'll add a brief comment.

>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
>

Re: [DTrace-devel] [PATCH 08/14] Test: caller and stackdepth tests for ip provider

[Eugene Loh]

On 6/13/25 10:51, Nick Alcock wrote:

> On 22 May 2025, eugene loh spake thusly:
>
>> From: Eugene Loh <eugene.loh@oracle.com>
>>
>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
>
> with the microscopic caveat below.
>
>> diff --git a/test/unittest/variables/bvar/perlping.pl b/test/unittest/variables/bvar/perlping.pl
>> new file mode 100755
>> index 000000000..97c9b09e9
>> --- /dev/null
>> +++ b/test/unittest/variables/bvar/perlping.pl
>> @@ -0,0 +1,8 @@
>> +#!/usr/bin/perl -w
>> +# Oracle Linux DTrace.
>> +# Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.
>> +# Licensed under the Universal Permissive License v 1.0 as shown at
>> +# http://oss.oracle.com/licenses/upl.
>> +use Net::Ping;
>> +my $p = Net::Ping->new(${ARGV[0]}, 5, 56);
>> +$p->ping(${ARGV[1]});
> Is there some reason we can't just use $testdir/../../ip/perlping.pl and
> avoid duplicating this?


Thanks.  Okay, I did this.

Re: [DTrace-devel] [PATCH 12/14] Test: caller and stackdepth tests for rawfbt provider

[Eugene Loh]

On 6/13/25 10:54, Nick Alcock wrote:

> On 22 May 2025, eugene loh spake thusly:
>
>> From: Eugene Loh <eugene.loh@oracle.com>
>>
>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> I note the lack of any sort of skipping on older kernels. Do the
> concerns about older kernels not apply here?

You mean with skipping frames and the BPF helper function?  Right. Only 
two providers worry about that.  One is fentry, but we do not use it on 
those older kernels (fbt uses kprobes instead).  The other is rawtp.

Did I say "only two"?  Well some providers (like sched and proc) use 
rawtp to implement certain probes.  So, they kind of worry about this 
problem.

But this provider (rawfbt) seems to be fine.

> If not, and this was intentional...
>
> Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

Thanks.

Re: [PATCH 11/14] Test: caller and stackdepth tests for proc provider

[Nick Alcock]

On 15 Jun 2025, Eugene Loh stated:

> Thanks.
>
> I'm getting to these out of order, but I figured I'd mention that I'm going to turn the capitalized "Test:" into "test:" for this
> series of patches to conform to the style in related patches.

Makes sense! I totally failed to notice that.

Re: [PATCH 01/14] Fix stack-skip counts for caller and stackdepth

[Nick Alcock]

On 16 Jun 2025, Eugene Loh verbalised:

> On 6/13/25 10:33, Nick Alcock wrote:
>
>>> Note that we declare the skip count volatile.  The compiler might
>>> optimize code that uses the STACK_SKIP value, but we will subsequently
>>> perform relocations that adjust this value.
>> ... why doesn't this apply to every other extern global variable in
>> get_bvar()? They're all similarly relocated...
>
> Right.  There is potentially a broader problem.  But we simply do not have evidence of misbehavior in other cases.  Ruggedizing
> other cases could be the subject of a different patch.

Aha, OK. I was just wondering if there was some extra reason.

> The problem in this case is that the compiler seems to assume &symbol!=0, which is reasonable except that we violate that behavior
> for our relocation tricks.

I wonder where the code for that is... plenty of symbols have value
zero.

But, really...

> Consider the C code:
>
>     uint64_t dt_bvar_stackdepth(const dt_dctx_t *dctx)
>     {
>         uint32_t          bufsiz = (uint32_t) (uint64_t) (&STKSIZ);
>         char              *buf = dctx->mem + (uint64_t)(&STACK_OFF);

Hm...

extern uint64_t STACK_SKIP;

So we encode information about the stack size and skip value by encoding
it in the *address* of the variable? Is there some reason we don't use
its value? unlike the stack offset, we're *using* it as a value, not an
address...

>         uint64_t          retv;
>         volatile uint64_t skip = (uint64_t)(&STACK_SKIP);
>
>         retv = bpf_get_stack(dctx->ctx,
>                              buf,
>                              bufsiz,
>                              skip & BPF_F_SKIP_FIELD_MASK);
>
>         if (skip)
>                 return retv / sizeof(uint64_t) - 1;      // branch "A"
>         return retv / sizeof(uint64_t);                  // branch "B"
>     }
>
> If you omit "volatile", the compiler assumes &STACK_SKIP!=0. The emitted code has:

(which is a reasonable assumption if not freestanding, I'd say. Why
don't we compile BPF code with -ffreestanding? BPF is almost the
*definition* of a freestanding environment...)

>     *)  no run-time "if (skip)" check
>
>     *)  no code for branch "B"
>
>     *)  only code for branch "A"
>
> If you include "volatile", however, the compiler caches &STACK_SKIP on
> the BPF stack and later performs a run-time check on its value to
> correctly execute either branch "A" or branch "B".

This feels very mucvh like a workaround to me. Does compiling BPF with
-ffreestanding help?

I mean it fixes a bug, so I suppose it should go in if nothing else
works, but using volatile is almost always a desperate sticking plaster
and this feels like one of those occasions to me.

-- 
NULL && (void)

Re: [PATCH 01/14] Fix stack-skip counts for caller and stackdepth

[Kris Van Hees]

On Thu, Jun 19, 2025 at 02:03:56PM +0100, Nick Alcock wrote:
> On 16 Jun 2025, Eugene Loh verbalised:
> 
> > On 6/13/25 10:33, Nick Alcock wrote:
> >
> >>> Note that we declare the skip count volatile.  The compiler might
> >>> optimize code that uses the STACK_SKIP value, but we will subsequently
> >>> perform relocations that adjust this value.
> >> ... why doesn't this apply to every other extern global variable in
> >> get_bvar()? They're all similarly relocated...
> >
> > Right.  There is potentially a broader problem.  But we simply do not have evidence of misbehavior in other cases.  Ruggedizing
> > other cases could be the subject of a different patch.
> 
> Aha, OK. I was just wondering if there was some extra reason.
> 
> > The problem in this case is that the compiler seems to assume &symbol!=0, which is reasonable except that we violate that behavior
> > for our relocation tricks.
> 
> I wonder where the code for that is... plenty of symbols have value
> zero.
> 
> But, really...
> 
> > Consider the C code:
> >
> >     uint64_t dt_bvar_stackdepth(const dt_dctx_t *dctx)
> >     {
> >         uint32_t          bufsiz = (uint32_t) (uint64_t) (&STKSIZ);
> >         char              *buf = dctx->mem + (uint64_t)(&STACK_OFF);
> 
> Hm...
> 
> extern uint64_t STACK_SKIP;
> 
> So we encode information about the stack size and skip value by encoding
> it in the *address* of the variable? Is there some reason we don't use
> its value? unlike the stack offset, we're *using* it as a value, not an
> address...

The issue seems to be (and perhaps this is a cross compiler problem) that e.g.

extern uint64_t PC;

and then code accessing the value of PC (e.g. foo(PC) as a call argument) will
yield:

  50:   18 02 00 00 00 00 00 00         lddw %r2,0
  58:   00 00 00 00 00 00 00 00 
                        50: R_BPF_INSN_64       PC
  60:   79 22 00 00 00 00 00 00         ldxdw %r2,[%r2+0]

which shows that it is interpreting PC as an address to a symbol, because it
loads the address of the symbol and then dereferences it with offset 0.  So,
we cannot plug in the value during relocation because the only value we can
put there would be an address where the vlaue can be found.  To get around
this, we "use" Tthe address as the entity to store the value in, knowing that
we *never* will interpret it as an address for these specific externs.

> >         uint64_t          retv;
> >         volatile uint64_t skip = (uint64_t)(&STACK_SKIP);
> >
> >         retv = bpf_get_stack(dctx->ctx,
> >                              buf,
> >                              bufsiz,
> >                              skip & BPF_F_SKIP_FIELD_MASK);
> >
> >         if (skip)
> >                 return retv / sizeof(uint64_t) - 1;      // branch "A"
> >         return retv / sizeof(uint64_t);                  // branch "B"
> >     }
> >
> > If you omit "volatile", the compiler assumes &STACK_SKIP!=0. The emitted code has:
> 
> (which is a reasonable assumption if not freestanding, I'd say. Why
> don't we compile BPF code with -ffreestanding? BPF is almost the
> *definition* of a freestanding environment...)
> 
> >     *)  no run-time "if (skip)" check
> >
> >     *)  no code for branch "B"
> >
> >     *)  only code for branch "A"
> >
> > If you include "volatile", however, the compiler caches &STACK_SKIP on
> > the BPF stack and later performs a run-time check on its value to
> > correctly execute either branch "A" or branch "B".
> 
> This feels very mucvh like a workaround to me. Does compiling BPF with
> -ffreestanding help?
> 
> I mean it fixes a bug, so I suppose it should go in if nothing else
> works, but using volatile is almost always a desperate sticking plaster
> and this feels like one of those occasions to me.
> 
> -- 
> NULL && (void)

Re: [PATCH 01/14] Fix stack-skip counts for caller and stackdepth

[Kris Van Hees]

On Thu, Jun 19, 2025 at 12:20:22PM -0400, Kris Van Hees wrote:
> On Thu, Jun 19, 2025 at 02:03:56PM +0100, Nick Alcock wrote:
> > On 16 Jun 2025, Eugene Loh verbalised:
> > 
> > > On 6/13/25 10:33, Nick Alcock wrote:
> > >
> > >>> Note that we declare the skip count volatile.  The compiler might
> > >>> optimize code that uses the STACK_SKIP value, but we will subsequently
> > >>> perform relocations that adjust this value.
> > >> ... why doesn't this apply to every other extern global variable in
> > >> get_bvar()? They're all similarly relocated...
> > >
> > > Right.  There is potentially a broader problem.  But we simply do not have evidence of misbehavior in other cases.  Ruggedizing
> > > other cases could be the subject of a different patch.
> > 
> > Aha, OK. I was just wondering if there was some extra reason.
> > 
> > > The problem in this case is that the compiler seems to assume &symbol!=0, which is reasonable except that we violate that behavior
> > > for our relocation tricks.
> > 
> > I wonder where the code for that is... plenty of symbols have value
> > zero.
> > 
> > But, really...
> > 
> > > Consider the C code:
> > >
> > >     uint64_t dt_bvar_stackdepth(const dt_dctx_t *dctx)
> > >     {
> > >         uint32_t          bufsiz = (uint32_t) (uint64_t) (&STKSIZ);
> > >         char              *buf = dctx->mem + (uint64_t)(&STACK_OFF);
> > 
> > Hm...
> > 
> > extern uint64_t STACK_SKIP;
> > 
> > So we encode information about the stack size and skip value by encoding
> > it in the *address* of the variable? Is there some reason we don't use
> > its value? unlike the stack offset, we're *using* it as a value, not an
> > address...
> 
> The issue seems to be (and perhaps this is a cross compiler problem) that e.g.
> 
> extern uint64_t PC;
> 
> and then code accessing the value of PC (e.g. foo(PC) as a call argument) will
> yield:
> 
>   50:   18 02 00 00 00 00 00 00         lddw %r2,0
>   58:   00 00 00 00 00 00 00 00 
>                         50: R_BPF_INSN_64       PC
>   60:   79 22 00 00 00 00 00 00         ldxdw %r2,[%r2+0]
> 
> which shows that it is interpreting PC as an address to a symbol, because it
> loads the address of the symbol and then dereferences it with offset 0.  So,
> we cannot plug in the value during relocation because the only value we can
> put there would be an address where the vlaue can be found.  To get around
> this, we "use" Tthe address as the entity to store the value in, knowing that
> we *never* will interpret it as an address for these specific externs.

Not a compiler error - since PC is extern uint64_t PC it *is* a variable and
so it is present (and accessible) as an address in .data in the location where
it is actually defined.  Since we never define it, we don't have a .data (which
is fine because we only use this for constants known at link time) BUT the
compiler of course is free to assume that we *do* have an address to the
storage location for PC and thus that we get the value that way.  It does not
know that the value is constant.  So, the trick I use is needed to make this
work.

> > >         uint64_t          retv;
> > >         volatile uint64_t skip = (uint64_t)(&STACK_SKIP);
> > >
> > >         retv = bpf_get_stack(dctx->ctx,
> > >                              buf,
> > >                              bufsiz,
> > >                              skip & BPF_F_SKIP_FIELD_MASK);
> > >
> > >         if (skip)
> > >                 return retv / sizeof(uint64_t) - 1;      // branch "A"
> > >         return retv / sizeof(uint64_t);                  // branch "B"
> > >     }
> > >
> > > If you omit "volatile", the compiler assumes &STACK_SKIP!=0. The emitted code has:
> > 
> > (which is a reasonable assumption if not freestanding, I'd say. Why
> > don't we compile BPF code with -ffreestanding? BPF is almost the
> > *definition* of a freestanding environment...)
> > 
> > >     *)  no run-time "if (skip)" check
> > >
> > >     *)  no code for branch "B"
> > >
> > >     *)  only code for branch "A"
> > >
> > > If you include "volatile", however, the compiler caches &STACK_SKIP on
> > > the BPF stack and later performs a run-time check on its value to
> > > correctly execute either branch "A" or branch "B".
> > 
> > This feels very mucvh like a workaround to me. Does compiling BPF with
> > -ffreestanding help?
> > 
> > I mean it fixes a bug, so I suppose it should go in if nothing else
> > works, but using volatile is almost always a desperate sticking plaster
> > and this feels like one of those occasions to me.
> > 
> > -- 
> > NULL && (void)

Re: [PATCH 01/14] Fix stack-skip counts for caller and stackdepth

[Nick Alcock]

On 19 Jun 2025, Kris Van Hees uttered the following:

> On Thu, Jun 19, 2025 at 12:20:22PM -0400, Kris Van Hees wrote:
>> On Thu, Jun 19, 2025 at 02:03:56PM +0100, Nick Alcock wrote:
>> > On 16 Jun 2025, Eugene Loh verbalised:
>> > 
>> > > On 6/13/25 10:33, Nick Alcock wrote:
>> > >
>> > >>> Note that we declare the skip count volatile.  The compiler might
>> > >>> optimize code that uses the STACK_SKIP value, but we will subsequently
>> > >>> perform relocations that adjust this value.
>> > >> ... why doesn't this apply to every other extern global variable in
>> > >> get_bvar()? They're all similarly relocated...
>> > >
>> > > Right.  There is potentially a broader problem.  But we simply do not have evidence of misbehavior in other cases.  Ruggedizing
>> > > other cases could be the subject of a different patch.
>> > 
>> > Aha, OK. I was just wondering if there was some extra reason.
>> > 
>> > > The problem in this case is that the compiler seems to assume &symbol!=0, which is reasonable except that we violate that behavior
>> > > for our relocation tricks.
>> > 
>> > I wonder where the code for that is... plenty of symbols have value
>> > zero.
>> > 
>> > But, really...
>> > 
>> > > Consider the C code:
>> > >
>> > >     uint64_t dt_bvar_stackdepth(const dt_dctx_t *dctx)
>> > >     {
>> > >         uint32_t          bufsiz = (uint32_t) (uint64_t) (&STKSIZ);
>> > >         char              *buf = dctx->mem + (uint64_t)(&STACK_OFF);
>> > 
>> > Hm...
>> > 
>> > extern uint64_t STACK_SKIP;
>> > 
>> > So we encode information about the stack size and skip value by encoding
>> > it in the *address* of the variable? Is there some reason we don't use
>> > its value? unlike the stack offset, we're *using* it as a value, not an
>> > address...
>> 
>> The issue seems to be (and perhaps this is a cross compiler problem) that e.g.
>> 
>> extern uint64_t PC;
>> 
>> and then code accessing the value of PC (e.g. foo(PC) as a call argument) will
>> yield:
>> 
>>   50:   18 02 00 00 00 00 00 00         lddw %r2,0
>>   58:   00 00 00 00 00 00 00 00 
>>                         50: R_BPF_INSN_64       PC
>>   60:   79 22 00 00 00 00 00 00         ldxdw %r2,[%r2+0]
>> 
>> which shows that it is interpreting PC as an address to a symbol, because it
>> loads the address of the symbol and then dereferences it with offset 0.  So,
>> we cannot plug in the value during relocation because the only value we can
>> put there would be an address where the vlaue can be found.  To get around
>> this, we "use" Tthe address as the entity to store the value in, knowing that
>> we *never* will interpret it as an address for these specific externs.
>
> Not a compiler error - since PC is extern uint64_t PC it *is* a variable and
> so it is present (and accessible) as an address in .data in the location where
> it is actually defined.  Since we never define it, we don't have a .data (which
> is fine because we only use this for constants known at link time) BUT the
> compiler of course is free to assume that we *do* have an address to the
> storage location for PC and thus that we get the value that way.  It does not
> know that the value is constant.  So, the trick I use is needed to make this
> work.

I'm just not sure why it's assuming that its value cannot be NULL, given
that you're not dereferencing it anywhere. I guess it's because it's a
symbol, but that's only a guess: I haven't managed to find the code that
implements this (?)optimization.

Anyway, this is not an objection to this patch, and a patch changing
this approach to something that didn't require us to throw volatiles
around and hope the compiler doesn't optimize things wrong would be a
separate patch anyway: you have my

Revieed-by: Nick Alcock <nick.alcock@oracle.com>

Re: [DTrace-devel] [PATCH 04/14] Test: caller and stackdepth tests for fbt provider

[Kris Van Hees]

On Mon, Jun 16, 2025 at 03:43:50PM -0400, Eugene Loh wrote:
> On 6/13/25 10:40, Nick Alcock wrote:
> 
> > On 22 May 2025, eugene loh uttered the following:
> > 
> > > From: Eugene Loh <eugene.loh@oracle.com>
> > > 
> > > We will introduce a set of tests for the caller and stackdepth
> > > built-in variables for a wide selection of providers.
> > > 
> > > For the caller test, we will essentially call
> > >          stack(2);
> > >          sym(caller);
> > > and then compare the caller to the second stack frame using
> > > the new script check_caller_to_stack2.awk.
> > > 
> > > For the stackdepth test, we will essentially call
> > >          printf("%d\n", stackdepth);
> > >          stack();
> > > and then compare the stackdepth to the reported frames using
> > > the new script check_stackdepth_to_stack.awk.
> > > 
> > > In this patch, introduce tests for the fbt provider, along with
> > > the support scripts they need.  Subsequent patches will handle
> > > other providers.
> > > 
> > > The old tst.caller2.d and tst.stackdepth2.d, which tested fbt,
> > > become obsolete.  Remove them.
> > Nice! My only caveat (easily fixed) is that the old tests had header
> > comments saying what they were testing, while the new ones don't.
> > (Yes, the awk scripts do make it clearer, but still...)
> 
> Got it.  I'll add a brief comment.

Can you please post the updated v2 patch ASAP so I can merge the series?

> > > Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> > Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
> > 

Re: [DTrace-devel] [PATCH 08/14] Test: caller and stackdepth tests for ip provider

[Kris Van Hees]

On Mon, Jun 16, 2025 at 11:38:05PM -0400, Eugene Loh wrote:
> On 6/13/25 10:51, Nick Alcock wrote:
> 
> > On 22 May 2025, eugene loh spake thusly:
> > 
> > > From: Eugene Loh <eugene.loh@oracle.com>
> > > 
> > > Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> > Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
> > 
> > with the microscopic caveat below.
> > 
> > > diff --git a/test/unittest/variables/bvar/perlping.pl b/test/unittest/variables/bvar/perlping.pl
> > > new file mode 100755
> > > index 000000000..97c9b09e9
> > > --- /dev/null
> > > +++ b/test/unittest/variables/bvar/perlping.pl
> > > @@ -0,0 +1,8 @@
> > > +#!/usr/bin/perl -w
> > > +# Oracle Linux DTrace.
> > > +# Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.
> > > +# Licensed under the Universal Permissive License v 1.0 as shown at
> > > +# http://oss.oracle.com/licenses/upl.
> > > +use Net::Ping;
> > > +my $p = Net::Ping->new(${ARGV[0]}, 5, 56);
> > > +$p->ping(${ARGV[1]});
> > Is there some reason we can't just use $testdir/../../ip/perlping.pl and
> > avoid duplicating this?
> 
> 
> Thanks.  Okay, I did this.
> 

Can you please post the updated v2 ASAP so I can merge the series?

Re: [PATCH 11/14] Test: caller and stackdepth tests for proc provider

[Kris Van Hees]

On Sun, Jun 15, 2025 at 01:50:02PM -0400, Eugene Loh wrote:
> Thanks.
> 
> I'm getting to these out of order, but I figured I'd mention that I'm going
> to turn the capitalized "Test:" into "test:" for this series of patches to
> conform to the style in related patches.

In that case, please post v2 of the series ASAP because I cannot merge without
updated patches.

> 
> On 6/13/25 10:53, Nick Alcock wrote:
> > On 22 May 2025, eugene loh told this:
> > 
> > > From: Eugene Loh <eugene.loh@oracle.com>
> > > 
> > > Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> > Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
> >