[PATCH 20/38] Add a hook for a provider-specific "update" function

[PATCH 20/38] Add a hook for a provider-specific "update" function

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

For up-coming USDT-probe support, we need to update a BPF map
-- at least when the dtrace session starts but possibly also later
to support systemwide USDT tracing for processes that may start up
later.

One way to do this is with a USDT-specific update function.

For now, let's add a hook for providers to have provider-specific
update functions.  User space can either call

    for (i = 0; i < ARRAY_SIZE(dt_providers); i++) {
        if (dt_providers[i]->update)
            dt_providers[i]->update(...);
    }

any time it likes.  Or it can call dt_usdt.update(...).

This is for WIP.  A different approach can be adopted later instead.

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_provider.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libdtrace/dt_provider.h b/libdtrace/dt_provider.h
index b1b1b1b8..71b5a3c4 100644
--- a/libdtrace/dt_provider.h
+++ b/libdtrace/dt_provider.h
@@ -71,6 +71,8 @@ typedef struct dt_provimpl {
 			      void *datap);
 	void (*destroy)(dtrace_hdl_t *dtp,	/* free provider data */
 			      void *datap);
+	void (*update)(dtrace_hdl_t *dtp,	/* update provider-specific info */
+			      void *datap);
 } dt_provimpl_t;
 
 /* list dt_dtrace first */
-- 
2.18.4

[PATCH 21/38] Add some comments

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_prov_uprobe.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libdtrace/dt_prov_uprobe.c b/libdtrace/dt_prov_uprobe.c
index 5dbd75e3..5f0c56db 100644
--- a/libdtrace/dt_prov_uprobe.c
+++ b/libdtrace/dt_prov_uprobe.c
@@ -295,6 +295,7 @@ static int provide_probe(dtrace_hdl_t *dtp, const pid_probespec_t *psp,
 	/* Mark the provider as a PID-based provider. */
 	pvp->pv_flags |= DT_PROVIDER_PID;
 
+	/* Look up or create the underlying probe. */
 	uprp = create_underlying(dtp, psp);
 	if (uprp == NULL)
 		return -1;
@@ -302,6 +303,7 @@ static int provide_probe(dtrace_hdl_t *dtp, const pid_probespec_t *psp,
 	upp = uprp->prv_data;
 	upp->flags |= flags;
 
+	/* Look up the overlying probe. */
 	prp = dt_probe_lookup(dtp, &pd);
 	if (prp != NULL) {
 		/*
-- 
2.18.4

[PATCH 22/38] Fix aggs comment in dt_cg_tramp_prologue_act()

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_cg.c | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/libdtrace/dt_cg.c b/libdtrace/dt_cg.c
index 4fd2d359..2fb2d0d8 100644
--- a/libdtrace/dt_cg.c
+++ b/libdtrace/dt_cg.c
@@ -372,17 +372,6 @@ dt_cg_tramp_prologue_act(dt_pcb_t *pcb, dt_activity_t act)
 	 *	if (rc == 0)		// jeq %r0, 0, lbl_exit
 	 *		goto exit;
 	 *
-	 *	key = 0;		// stdw [%r9 + DCTX_AGG], 0
-	 *	rc = bpf_map_lookup_elem(rc, &key);
-	 *				// mov %r1, %r0
-	 *				// mov %r2, %r9
-	 *				// add %r2, DCTX_AGG
-	 *				// call bpf_map_lookup_elem
-	 *				//     (%r1 ... %r5 clobbered)
-	 *				//     (%r0 = aggs[cpuid] BPF map value)
-	 *	if (rc == 0)		// jeq %r0, 0, lbl_exit
-	 *		goto exit;
-	 *
 	 *	dctx.aggs = rc;		// stdw [%r9 + DCTX_AGG], %r0
 	 */
 	if (dtp->dt_maxaggdsize > 0) {
-- 
2.18.4

[PATCH 23/38] test: Clean up the specsize tests

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

The tests had actions like
    printf("%lld: Lots of data\n", x);
    printf("%lld: Has to be crammed into this buffer\n", x);
    printf("%lld: Until it overflows\n", x);
    printf("%lld: And causes flops\n", x);
suggesting that these strings were crowding the buffer, but these
strings are not passed from producer to consumer at all.

The tests also only tested one clause per speculation.  It would be
nice also to test multiple clauses per speculation.

There is much replicated code from one of the tests to the other, a
shortcoming that is amplified if we want to test more specsize values,
which is the case when we test multiple clauses per speculation.

Therefore, replace the multiple tests with a single test that checks
multiple clauses per speculation and more values of specsize.

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 .../speculation/tst.SpecSizeVariations.r      | 68 +++++++++++++++++
 .../speculation/tst.SpecSizeVariations.sh     | 74 +++++++++++++++++++
 .../speculation/tst.SpecSizeVariations4.d     | 66 -----------------
 .../speculation/tst.SpecSizeVariations4.r     |  5 --
 .../speculation/tst.SpecSizeVariations5.d     | 61 ---------------
 .../speculation/tst.SpecSizeVariations5.r     |  7 --
 6 files changed, 142 insertions(+), 139 deletions(-)
 create mode 100644 test/unittest/speculation/tst.SpecSizeVariations.r
 create mode 100755 test/unittest/speculation/tst.SpecSizeVariations.sh
 delete mode 100644 test/unittest/speculation/tst.SpecSizeVariations4.d
 delete mode 100644 test/unittest/speculation/tst.SpecSizeVariations4.r
 delete mode 100644 test/unittest/speculation/tst.SpecSizeVariations5.d
 delete mode 100644 test/unittest/speculation/tst.SpecSizeVariations5.r

diff --git a/test/unittest/speculation/tst.SpecSizeVariations.r b/test/unittest/speculation/tst.SpecSizeVariations.r
new file mode 100644
index 00000000..51f0596c
--- /dev/null
+++ b/test/unittest/speculation/tst.SpecSizeVariations.r
@@ -0,0 +1,68 @@
+Speculative buffer ID: 1
+counts: 0 1
+
+Speculative buffer ID: 1
+123456700
+123456701
+123456702
+123456703
+123456704
+123456705
+123456706
+counts: 1 1
+
+Speculative buffer ID: 1
+123456700
+123456701
+123456702
+123456703
+123456704
+123456705
+123456706
+counts: 1 1
+
+Speculative buffer ID: 1
+123456700
+123456701
+123456702
+123456703
+123456704
+123456705
+123456706
+counts: 2 1
+
+Speculative buffer ID: 1
+123456700
+123456701
+123456702
+123456703
+123456704
+123456705
+123456706
+counts: 2 1
+
+Speculative buffer ID: 1
+123456700
+123456701
+123456702
+123456703
+123456704
+123456705
+123456706
+123456800
+123456801
+123456802
+123456803
+123456804
+123456805
+123456806
+123456807
+123456808
+counts: 2 1
+
+-- @@stderr --
+dtrace: 2 speculative drops
+dtrace: 1 speculative drop
+dtrace: 1 speculative drop
+dtrace: 1 speculative drop
+dtrace: 1 speculative drop
diff --git a/test/unittest/speculation/tst.SpecSizeVariations.sh b/test/unittest/speculation/tst.SpecSizeVariations.sh
new file mode 100755
index 00000000..75e527d9
--- /dev/null
+++ b/test/unittest/speculation/tst.SpecSizeVariations.sh
@@ -0,0 +1,74 @@
+#!/bin/bash
+
+#
+# Oracle Linux DTrace.
+# Copyright (c) 2024, Oracle and/or its affiliates. All rights reserved.
+# Licensed under the Universal Permissive License v 1.0 as shown at
+# http://oss.oracle.com/licenses/upl.
+#
+
+dtrace=$1
+
+for x in 63 64 79 80 143 144; do
+	$dtrace $dt_flags -xspecsize=$x -qn '
+	BEGIN
+	{
+		x = 123456700ll;
+		self->nspeculate = 0;
+		self->ncommit = 0;
+		self->spec = speculation();
+		printf("Speculative buffer ID: %d\n", self->spec);
+	}
+
+	/* 16 + 7 * 8 = 72 bytes */
+	BEGIN
+	{
+		speculate(self->spec);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		self->nspeculate++;
+	}
+
+	BEGIN
+	{
+		x = 123456800ll;
+	}
+
+	/* 16 + 9 * 8 = 88 bytes */
+	BEGIN
+	{
+		speculate(self->spec);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		printf("%lld\n", x++);
+		self->nspeculate++;
+	}
+
+	BEGIN
+	{
+		commit(self->spec);
+		self->ncommit++;
+	}
+
+	BEGIN
+	{
+		printf("counts: %d %d\n", self->nspeculate, self->ncommit);
+		exit(0);
+	}
+
+	ERROR
+	{
+		exit(1);
+	}'
+done
diff --git a/test/unittest/speculation/tst.SpecSizeVariations4.d b/test/unittest/speculation/tst.SpecSizeVariations4.d
deleted file mode 100644
index 4221c89e..00000000
--- a/test/unittest/speculation/tst.SpecSizeVariations4.d
+++ /dev/null
@@ -1,66 +0,0 @@
-/*
- * Oracle Linux DTrace.
- * Copyright (c) 2006, 2023, Oracle and/or its affiliates. All rights reserved.
- * Licensed under the Universal Permissive License v 1.0 as shown at
- * http://oss.oracle.com/licenses/upl.
- */
-
-/*
- * ASSERTION:
- * Verify the behavior of speculations with changes in specsize.
- *
- * SECTION: Speculative Tracing/Options and Tuning;
- *	Options and Tunables/specsize
- *
- */
-
-#pragma D option quiet
-#pragma D option specsize=39
-
-long long x;
-
-BEGIN
-{
-	x = 123456789;
-	self->speculateFlag = 0;
-	self->commitFlag = 0;
-	self->spec = speculation();
-	printf("Speculative buffer ID: %d\n", self->spec);
-}
-
-BEGIN
-{
-	speculate(self->spec);
-	printf("%lld: Lots of data\n", x);
-	printf("%lld: Has to be crammed into this buffer\n", x);
-	printf("%lld: Until it overflows\n", x);
-	printf("%lld: And causes flops\n", x);
-	self->speculateFlag++;
-
-}
-
-BEGIN
-/1 <= self->speculateFlag/
-{
-	commit(self->spec);
-	self->commitFlag++;
-}
-
-BEGIN
-/1 <= self->commitFlag/
-{
-	printf("Statement was executed\n");
-	exit(1);
-}
-
-BEGIN
-/1 > self->commitFlag/
-{
-	printf("Statement wasn't executed\n");
-	exit(0);
-}
-
-ERROR
-{
-	exit(1);
-}
diff --git a/test/unittest/speculation/tst.SpecSizeVariations4.r b/test/unittest/speculation/tst.SpecSizeVariations4.r
deleted file mode 100644
index 7c4bb3b7..00000000
--- a/test/unittest/speculation/tst.SpecSizeVariations4.r
+++ /dev/null
@@ -1,5 +0,0 @@
-Speculative buffer ID: 1
-Statement wasn't executed
-
--- @@stderr --
-dtrace: 1 speculative drop
diff --git a/test/unittest/speculation/tst.SpecSizeVariations5.d b/test/unittest/speculation/tst.SpecSizeVariations5.d
deleted file mode 100644
index fb71dfed..00000000
--- a/test/unittest/speculation/tst.SpecSizeVariations5.d
+++ /dev/null
@@ -1,61 +0,0 @@
-/*
- * Oracle Linux DTrace.
- * Copyright (c) 2006, 2021, Oracle and/or its affiliates. All rights reserved.
- * Licensed under the Universal Permissive License v 1.0 as shown at
- * http://oss.oracle.com/licenses/upl.
- */
-
-/*
- * ASSERTION:
- * Verify the behavior of speculations with changes in specsize.
- *
- * SECTION: Speculative Tracing/Options and Tuning;
- *	Options and Tunables/specsize
- *
- */
-
-#pragma D option quiet
-#pragma D option specsize=40
-
-long long x;
-
-BEGIN
-{
-	x = 123456789;
-	self->speculateFlag = 0;
-	self->commitFlag = 0;
-	self->spec = speculation();
-	printf("Speculative buffer ID: %d\n", self->spec);
-}
-
-BEGIN
-{
-	speculate(self->spec);
-	printf("%lld: Lots of data\n", x);
-	printf("%lld: Has to be crammed into this buffer\n", x);
-	printf("%lld: Until it overflows\n", x);
-	printf("%lld: And causes flops\n", x);
-	self->speculateFlag++;
-
-}
-
-BEGIN
-/1 <= self->speculateFlag/
-{
-	commit(self->spec);
-	self->commitFlag++;
-}
-
-BEGIN
-/1 <= self->commitFlag/
-{
-	printf("Statement was executed\n");
-	exit(0);
-}
-
-BEGIN
-/1 > self->commitFlag/
-{
-	printf("Statement wasn't executed\n");
-	exit(1);
-}
diff --git a/test/unittest/speculation/tst.SpecSizeVariations5.r b/test/unittest/speculation/tst.SpecSizeVariations5.r
deleted file mode 100644
index d09013a2..00000000
--- a/test/unittest/speculation/tst.SpecSizeVariations5.r
+++ /dev/null
@@ -1,7 +0,0 @@
-Speculative buffer ID: 1
-123456789: Lots of data
-123456789: Has to be crammed into this buffer
-123456789: Until it overflows
-123456789: And causes flops
-Statement was executed
-
-- 
2.18.4

[PATCH 24/38] test: Make test independent of specific PC

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 test/unittest/error/tst.DTRACEFLT_UNKNOWN.d | 6 +++---
 test/unittest/error/tst.DTRACEFLT_UNKNOWN.r | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
index 001903ff..bfc77bf5 100644
--- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
+++ b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
@@ -1,6 +1,6 @@
 /*
  * Oracle Linux DTrace.
- * Copyright (c) 2006, 2020, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2006, 2024, Oracle and/or its affiliates. All rights reserved.
  * Licensed under the Universal Permissive License v 1.0 as shown at
  * http://oss.oracle.com/licenses/upl.
  */
@@ -19,8 +19,8 @@
 
 ERROR
 {
-	printf("The arguments are %u %u %u %u %u\n",
-		arg1, arg2, arg3, arg4, arg5);
+	printf("The arguments are %u %u PC %u %u\n",
+		arg1, arg2, arg4, arg5);
 	printf("The value of arg4 = %u\n", DTRACEFLT_UNKNOWN);
 	exit(0);
 }
diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
index b11f6c99..3e7caac4 100644
--- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
+++ b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
@@ -1,4 +1,4 @@
-The arguments are 2 2 4 1 64
+The arguments are 2 2 PC 1 64
 The value of arg4 = 0
 
 -- @@stderr --
-- 
2.18.4

[PATCH 25/38] test: Clean up tests still expecting obsolete "at DIF offset NN"

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Also, the numbering of EPIDs has changed.

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 test/demo/dtrace/error.d                                | 1 -
 test/demo/dtrace/error.r                                | 2 +-
 test/unittest/assocs/tst.invalidref.r                   | 4 ++--
 test/unittest/drops/drp.DTRACEDROP_DBLERROR.r           | 2 +-
 test/unittest/error/tst.DTRACEFLT_UNKNOWN.d             | 1 -
 test/unittest/error/tst.DTRACEFLT_UNKNOWN.r             | 4 ++--
 test/unittest/error/tst.DTRACEFLT_UNKNOWN.sparc64.r     | 5 -----
 test/unittest/pointers/err.BadAlign.d                   | 1 -
 test/unittest/pointers/err.BadAlign.r                   | 2 +-
 test/unittest/pointers/err.InvalidAddress2.d            | 1 -
 test/unittest/pointers/err.InvalidAddress2.r            | 2 +-
 test/unittest/pointers/err.InvalidAddress3.r            | 2 +-
 test/unittest/pointers/err.InvalidAddress4.d            | 1 -
 test/unittest/pointers/err.InvalidAddress4.r            | 2 +-
 test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.d | 2 +-
 test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.r | 4 ++--
 16 files changed, 13 insertions(+), 23 deletions(-)
 delete mode 100644 test/unittest/error/tst.DTRACEFLT_UNKNOWN.sparc64.r

diff --git a/test/demo/dtrace/error.d b/test/demo/dtrace/error.d
index 5700dd33..d55fb090 100644
--- a/test/demo/dtrace/error.d
+++ b/test/demo/dtrace/error.d
@@ -5,7 +5,6 @@
  * http://oss.oracle.com/licenses/upl.
  */
 
-/* @@xfail: dtv2 */
 /* @@trigger: none */
 
 BEGIN
diff --git a/test/demo/dtrace/error.r b/test/demo/dtrace/error.r
index d894776b..d3904f47 100644
--- a/test/demo/dtrace/error.r
+++ b/test/demo/dtrace/error.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/demo/dtrace/error.d' matched 2 probes
-dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at DIF offset 16 at BPF pc NNN
+dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/assocs/tst.invalidref.r b/test/unittest/assocs/tst.invalidref.r
index b050e436..20f131b4 100644
--- a/test/unittest/assocs/tst.invalidref.r
+++ b/test/unittest/assocs/tst.invalidref.r
@@ -1,4 +1,4 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID (ID: profile:::tick-1s): invalid address ({ptr}) in action #2 at DIF offset 64 at BPF pc NNN
-dtrace: error on enabled probe ID (ID: profile:::tick-1s): invalid address ({ptr}) in action #2 at DIF offset 64 at BPF pc NNN
+dtrace: error on enabled probe ID (ID: profile:::tick-1s): invalid address ({ptr}) in action #2 at BPF pc NNN
+dtrace: error on enabled probe ID (ID: profile:::tick-1s): invalid address ({ptr}) in action #2 at BPF pc NNN
diff --git a/test/unittest/drops/drp.DTRACEDROP_DBLERROR.r b/test/unittest/drops/drp.DTRACEDROP_DBLERROR.r
index 14654676..9fa54dd9 100644
--- a/test/unittest/drops/drp.DTRACEDROP_DBLERROR.r
+++ b/test/unittest/drops/drp.DTRACEDROP_DBLERROR.r
@@ -4,4 +4,4 @@
 -- @@stderr --
 dtrace: script 'test/unittest/drops/drp.DTRACEDROP_DBLERROR.d' matched 3 probes
 dtrace: [DTRACEDROP_DBLERROR] 1 error in ERROR probe enabling
-dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at DIF offset 16 at BPF pc NNN
+dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
index bfc77bf5..c74762ae 100644
--- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
+++ b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
@@ -4,7 +4,6 @@
  * Licensed under the Universal Permissive License v 1.0 as shown at
  * http://oss.oracle.com/licenses/upl.
  */
-/* @@xfail: dtv2 */
 
 /*
  * ASSERTION:
diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
index 3e7caac4..1e4fdd64 100644
--- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
+++ b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
@@ -1,5 +1,5 @@
-The arguments are 2 2 PC 1 64
+The arguments are 3 1 PC 1 64
 The value of arg4 = 0
 
 -- @@stderr --
-dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #2 at DIF offset 4 at BPF pc NNN
+dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.sparc64.r b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.sparc64.r
deleted file mode 100644
index 3944c138..00000000
--- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.sparc64.r
+++ /dev/null
@@ -1,5 +0,0 @@
-The arguments are 2 2 4 1 0
-The value of arg4 = 0
-
--- @@stderr --
-dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #2 at DIF offset 4
diff --git a/test/unittest/pointers/err.BadAlign.d b/test/unittest/pointers/err.BadAlign.d
index cd4138ae..e859dd75 100644
--- a/test/unittest/pointers/err.BadAlign.d
+++ b/test/unittest/pointers/err.BadAlign.d
@@ -4,7 +4,6 @@
  * Licensed under the Universal Permissive License v 1.0 as shown at
  * http://oss.oracle.com/licenses/upl.
  */
-/* @@xfail: dtv2 */
 
 /*
  * ASSERTION: This test reproduces the alignment error.
diff --git a/test/unittest/pointers/err.BadAlign.r b/test/unittest/pointers/err.BadAlign.r
index 4328aac4..187543b6 100644
--- a/test/unittest/pointers/err.BadAlign.r
+++ b/test/unittest/pointers/err.BadAlign.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #2 at DIF offset 4
+dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/pointers/err.InvalidAddress2.d b/test/unittest/pointers/err.InvalidAddress2.d
index 682ad650..b22f08fb 100644
--- a/test/unittest/pointers/err.InvalidAddress2.d
+++ b/test/unittest/pointers/err.InvalidAddress2.d
@@ -4,7 +4,6 @@
  * Licensed under the Universal Permissive License v 1.0 as shown at
  * http://oss.oracle.com/licenses/upl.
  */
-/* @@xfail: dtv2 */
 
 /*
  * ASSERTION: D pointers do not allow invalid pointer accesses.
diff --git a/test/unittest/pointers/err.InvalidAddress2.r b/test/unittest/pointers/err.InvalidAddress2.r
index d866eae1..187543b6 100644
--- a/test/unittest/pointers/err.InvalidAddress2.r
+++ b/test/unittest/pointers/err.InvalidAddress2.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #4 at DIF offset 8
+dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/pointers/err.InvalidAddress3.r b/test/unittest/pointers/err.InvalidAddress3.r
index 069ee1de..187543b6 100644
--- a/test/unittest/pointers/err.InvalidAddress3.r
+++ b/test/unittest/pointers/err.InvalidAddress3.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #2 at DIF offset 8
+dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/pointers/err.InvalidAddress4.d b/test/unittest/pointers/err.InvalidAddress4.d
index 1e2b4f62..586cddf9 100644
--- a/test/unittest/pointers/err.InvalidAddress4.d
+++ b/test/unittest/pointers/err.InvalidAddress4.d
@@ -4,7 +4,6 @@
  * Licensed under the Universal Permissive License v 1.0 as shown at
  * http://oss.oracle.com/licenses/upl.
  */
-/* @@xfail: dtv2 */
 
 /*
  * ASSERTION: Demonstrating valid memory access.
diff --git a/test/unittest/pointers/err.InvalidAddress4.r b/test/unittest/pointers/err.InvalidAddress4.r
index d866eae1..187543b6 100644
--- a/test/unittest/pointers/err.InvalidAddress4.r
+++ b/test/unittest/pointers/err.InvalidAddress4.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #4 at DIF offset 8
+dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.d b/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.d
index c23f9503..13032f77 100644
--- a/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.d
+++ b/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.d
@@ -4,7 +4,7 @@
  * Licensed under the Universal Permissive License v 1.0 as shown at
  * http://oss.oracle.com/licenses/upl.
  */
-/* @@xfail: dtv2 */
+/* @@xfail: dtv2 d_path */
 
 /*
  * ASSERTION:
diff --git a/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.r b/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.r
index 8c601a43..54ffb47f 100644
--- a/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.r
+++ b/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.r
@@ -1,6 +1,6 @@
-The arguments are 2 1 28 1 24
+The arguments are 3 1 28 1 24
 The value of arg4 should be 1
 The value of arg5 should be 24
 
 -- @@stderr --
-dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at DIF offset 28 at BPF pc NNN
+dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
-- 
2.18.4

[PATCH 26/38] test: Annotate xfail (chill not implemented yet)

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 test/unittest/speculation/tst.zerosize.d | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/unittest/speculation/tst.zerosize.d b/test/unittest/speculation/tst.zerosize.d
index 56c1fcea..996f1257 100644
--- a/test/unittest/speculation/tst.zerosize.d
+++ b/test/unittest/speculation/tst.zerosize.d
@@ -5,7 +5,7 @@
  * http://oss.oracle.com/licenses/upl.
  */
 
-/* @@xfail: dtv2 */
+/* @@xfail: dtv2, chill not implemented yet */
 
 #pragma D option destructive
 
-- 
2.18.4

[PATCH 27/38] test: Fix the speculative tests that checked bufsize

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Ever since speculations were reimplemented, these tests simply were
not testing what they claimed to test.  Among other things:

1)  There is a simple dependence on bufsize that is independent of
speculations.  So speculations have no special role in bufsize tests.

2)  The messages in the print statements were not taxing bufsize.
(A single default string would be enough to exhaust 60 bytes, but
the producer wasn't passing the consumer these strings at all.
The bufsize is being set by the implicit ERROR probe.)

3)  The various size ranges described in the comments do not exist
and were not being tested anyhow.

So remove these tests, and just rely on the other bufsize tests.

To be fair, there seemed to have been no other "negative bufsize"
test, but add a more straightforward such test and add it to where
the other bufsize tests are.

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 test/unittest/options/err.bufsize-negative.d  | 24 +++++++
 test/unittest/options/err.bufsize-negative.r  |  2 +
 .../speculation/err.BufSizeVariations1.d      | 67 -------------------
 .../speculation/err.BufSizeVariations2.d      | 67 -------------------
 .../speculation/err.NegativeBufSize.d         | 67 -------------------
 .../speculation/err.NegativeBufSize.r         |  2 -
 6 files changed, 26 insertions(+), 203 deletions(-)
 create mode 100644 test/unittest/options/err.bufsize-negative.d
 create mode 100644 test/unittest/options/err.bufsize-negative.r
 delete mode 100644 test/unittest/speculation/err.BufSizeVariations1.d
 delete mode 100644 test/unittest/speculation/err.BufSizeVariations2.d
 delete mode 100644 test/unittest/speculation/err.NegativeBufSize.d
 delete mode 100644 test/unittest/speculation/err.NegativeBufSize.r

diff --git a/test/unittest/options/err.bufsize-negative.d b/test/unittest/options/err.bufsize-negative.d
new file mode 100644
index 00000000..ac194dc8
--- /dev/null
+++ b/test/unittest/options/err.bufsize-negative.d
@@ -0,0 +1,24 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2024, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+/*
+ * ASSERTION: The -xbufsize option sets the trace buffer size.
+ *
+ * SECTION: Options and Tunables/Consumer Options
+ */
+
+/* @@runtest-opts: -xbufsize=-1 */
+
+BEGIN
+{
+	exit(0);
+}
+
+ERROR
+{
+	exit(1);
+}
diff --git a/test/unittest/options/err.bufsize-negative.r b/test/unittest/options/err.bufsize-negative.r
new file mode 100644
index 00000000..ea3089e9
--- /dev/null
+++ b/test/unittest/options/err.bufsize-negative.r
@@ -0,0 +1,2 @@
+-- @@stderr --
+dtrace: failed to set -x bufsize: Invalid value for specified option
diff --git a/test/unittest/speculation/err.BufSizeVariations1.d b/test/unittest/speculation/err.BufSizeVariations1.d
deleted file mode 100644
index c6e29b78..00000000
--- a/test/unittest/speculation/err.BufSizeVariations1.d
+++ /dev/null
@@ -1,67 +0,0 @@
-/*
- * Oracle Linux DTrace.
- * Copyright (c) 2006, Oracle and/or its affiliates. All rights reserved.
- * Licensed under the Universal Permissive License v 1.0 as shown at
- * http://oss.oracle.com/licenses/upl.
- */
-
-/*
- * ASSERTION:
- * Verify the behavior of variations in bufsize.
- *
- * SECTION: Speculative Tracing/Options and Tuning;
- *	Options and Tunables/bufsize
- *
- * NOTES: This test behaves differently depending on the values
- * assigned to bufsize.
- * 1. 0 > bufsize.
- * 2. 0 == bufsize.
- * 3. 0 < bufsize <= 7
- * 4. 8 <= bufsize <= 31
- * 5. 32 <= bufsize <= 47
- * 6. 48 <= bufsize <= 71
- * 7. 72 <= bufsize
- */
-
-#pragma D option quiet
-#pragma D option bufsize=41
-
-BEGIN
-{
-	self->speculateFlag = 0;
-	self->commitFlag = 0;
-	self->spec = speculation();
-	printf("Speculative buffer ID: %d\n", self->spec);
-}
-
-BEGIN
-{
-	speculate(self->spec);
-	printf("Lots of data\n");
-	printf("Has to be crammed into this buffer\n");
-	printf("Until it overflows\n");
-	printf("And causes flops\n");
-	self->speculateFlag++;
-
-}
-
-BEGIN
-/1 <= self->speculateFlag/
-{
-	commit(self->spec);
-	self->commitFlag++;
-}
-
-BEGIN
-/1 <= self->commitFlag/
-{
-	printf("Statement was executed\n");
-	exit(0);
-}
-
-BEGIN
-/1 > self->commitFlag/
-{
-	printf("Statement wasn't executed\n");
-	exit(1);
-}
diff --git a/test/unittest/speculation/err.BufSizeVariations2.d b/test/unittest/speculation/err.BufSizeVariations2.d
deleted file mode 100644
index 8094cd0b..00000000
--- a/test/unittest/speculation/err.BufSizeVariations2.d
+++ /dev/null
@@ -1,67 +0,0 @@
-/*
- * Oracle Linux DTrace.
- * Copyright (c) 2006, Oracle and/or its affiliates. All rights reserved.
- * Licensed under the Universal Permissive License v 1.0 as shown at
- * http://oss.oracle.com/licenses/upl.
- */
-
-/*
- * ASSERTION:
- * Verify the behavior of variations in bufsize.
- *
- * SECTION: Speculative Tracing/Options and Tuning;
- *	    Options and Tunables/bufsize
- *
- * NOTES: This test behaves differently depending on the values
- * assigned to bufsize.
- * 1. 0 > bufsize.
- * 2. 0 == bufsize.
- * 3. 0 < bufsize <= 7
- * 4. 8 <= bufsize <= 31
- * 5. 32 <= bufsize <= 47
- * 6. 48 <= bufsize <= 71
- * 7. 72 <= bufsize
- */
-
-#pragma D option quiet
-#pragma D option bufsize=4
-
-BEGIN
-{
-	self->speculateFlag = 0;
-	self->commitFlag = 0;
-	self->spec = speculation();
-	printf("Speculative buffer ID: %d\n", self->spec);
-}
-
-BEGIN
-{
-	speculate(self->spec);
-	printf("Lots of data\n");
-	printf("Has to be crammed into this buffer\n");
-	printf("Until it overflows\n");
-	printf("And causes flops\n");
-	self->speculateFlag++;
-
-}
-
-BEGIN
-/1 <= self->speculateFlag/
-{
-	commit(self->spec);
-	self->commitFlag++;
-}
-
-BEGIN
-/1 <= self->commitFlag/
-{
-	printf("Statement was executed\n");
-	exit(0);
-}
-
-BEGIN
-/1 > self->commitFlag/
-{
-	printf("Statement wasn't executed\n");
-	exit(1);
-}
diff --git a/test/unittest/speculation/err.NegativeBufSize.d b/test/unittest/speculation/err.NegativeBufSize.d
deleted file mode 100644
index a0154ccb..00000000
--- a/test/unittest/speculation/err.NegativeBufSize.d
+++ /dev/null
@@ -1,67 +0,0 @@
-/*
- * Oracle Linux DTrace.
- * Copyright (c) 2006, Oracle and/or its affiliates. All rights reserved.
- * Licensed under the Universal Permissive License v 1.0 as shown at
- * http://oss.oracle.com/licenses/upl.
- */
-
-/*
- * ASSERTION:
- * Verify the behavior of variations in bufsize.
- *
- * SECTION: Speculative Tracing/Options and Tuning;
- *	Options and Tunables/bufsize
- *
- * NOTES: This test behaves differently depending on the values
- * assigned to bufsize.
- * 1. 0 > bufsize.
- * 2. 0 == bufsize.
- * 3. 0 < bufsize <= 7
- * 4. 8 <= bufsize <= 31
- * 5. 32 <= bufsize <= 47
- * 6. 48 <= bufsize <= 71
- * 7. 72 <= bufsize
- */
-
-#pragma D option quiet
-#pragma D option bufsize=-72
-
-BEGIN
-{
-	self->speculateFlag = 0;
-	self->commitFlag = 0;
-	self->spec = speculation();
-	printf("Speculative buffer ID: %d\n", self->spec);
-}
-
-BEGIN
-{
-	speculate(self->spec);
-	printf("Lots of data\n");
-	printf("Has to be crammed into this buffer\n");
-	printf("Until it overflows\n");
-	printf("And causes flops\n");
-	self->speculateFlag++;
-
-}
-
-BEGIN
-/1 <= self->speculateFlag/
-{
-	commit(self->spec);
-	self->commitFlag++;
-}
-
-BEGIN
-/1 <= self->commitFlag/
-{
-	printf("Statement was executed\n");
-	exit(0);
-}
-
-BEGIN
-/1 > self->commitFlag/
-{
-	printf("Statement wasn't executed\n");
-	exit(1);
-}
diff --git a/test/unittest/speculation/err.NegativeBufSize.r b/test/unittest/speculation/err.NegativeBufSize.r
deleted file mode 100644
index 9d7be6c0..00000000
--- a/test/unittest/speculation/err.NegativeBufSize.r
+++ /dev/null
@@ -1,2 +0,0 @@
--- @@stderr --
-dtrace: failed to compile script test/unittest/speculation/err.NegativeBufSize.d: line 27: failed to set option 'bufsize' to '-72': Invalid value for specified option
-- 
2.18.4

[PATCH 28/38] Remove unused "next" arg from dt_flowindent()

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_consume.c | 24 +++---------------------
 1 file changed, 3 insertions(+), 21 deletions(-)

diff --git a/libdtrace/dt_consume.c b/libdtrace/dt_consume.c
index 5fb636fe..7dfec72f 100644
--- a/libdtrace/dt_consume.c
+++ b/libdtrace/dt_consume.c
@@ -432,11 +432,9 @@ static dt_htab_ops_t dt_spec_buf_htab_ops = {
 };
 
 static int
-dt_flowindent(dtrace_hdl_t *dtp, dtrace_probedata_t *data, dtrace_epid_t last,
-	      dtrace_epid_t next)
+dt_flowindent(dtrace_hdl_t *dtp, dtrace_probedata_t *data, dtrace_epid_t last)
 {
-	dtrace_probedesc_t	*pd = data->dtpda_pdesc, *npd;
-	dtrace_datadesc_t	*ndd;
+	dtrace_probedesc_t	*pd = data->dtpda_pdesc;
 	dtrace_flowkind_t	flow = DTRACEFLOW_NONE;
 	const char		*p = pd->prv;
 	const char		*n = pd->prb;
@@ -447,7 +445,6 @@ dt_flowindent(dtrace_hdl_t *dtp, dtrace_probedata_t *data, dtrace_epid_t last,
 	static const char	*ent = "entry", *ret = "return";
 	static int		entlen = 0, retlen = 0;
 	dtrace_epid_t		id = data->dtpda_epid;
-	int			rval;
 
 	if (entlen == 0) {
 		assert(retlen == 0);
@@ -485,21 +482,6 @@ dt_flowindent(dtrace_hdl_t *dtp, dtrace_probedata_t *data, dtrace_epid_t last,
 			flow = DTRACEFLOW_NONE;
 	}
 
-	/*
-	 * If we're going to unindent this, it's more difficult to see if
-	 * we don't actually want to unindent it -- we need to look at the
-	 * _next_ EPID.
-	 */
-	if (flow == DTRACEFLOW_RETURN && next != DTRACE_EPIDNONE &&
-	    next != id) {
-		rval = dt_epid_lookup(dtp, next, &ndd, &npd);
-		if (rval != 0)
-			return rval;
-
-		if (npd->id == pd->id)
-			flow = DTRACEFLOW_NONE;
-	}
-
 	if (flow == DTRACEFLOW_ENTRY || flow == DTRACEFLOW_RETURN)
 		data->dtpda_prefix = str;
 	else
@@ -2343,7 +2325,7 @@ dt_consume_one_probe(dtrace_hdl_t *dtp, FILE *fp, char *data, uint32_t size,
 
 	if (data_recording) {
 		if (flow)
-			dt_flowindent(dtp, pdat, *last, DTRACE_EPIDNONE);
+			dt_flowindent(dtp, pdat, *last);
 
 		rval = (*efunc)(pdat, arg);
 
-- 
2.18.4

[PATCH 29/38] Allow relocation of the ERROR PRID

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_bpf.h   | 1 +
 libdtrace/dt_cc.c    | 3 +++
 libdtrace/dt_dlibs.c | 1 +
 3 files changed, 5 insertions(+)

diff --git a/libdtrace/dt_bpf.h b/libdtrace/dt_bpf.h
index 5b2df264..58cf29cc 100644
--- a/libdtrace/dt_bpf.h
+++ b/libdtrace/dt_bpf.h
@@ -56,6 +56,7 @@ extern "C" {
 #define DT_CONST_ZERO_OFF		22
 #define DT_CONST_STACK_OFF		23
 #define DT_CONST_STACK_SKIP		24
+#define DT_CONST_ERROR_PRID		25
 
 #define DT_BPF_LOG_SIZE_DEFAULT	(UINT32_MAX >> 8)
 #define DT_BPF_LOG_SIZE_SMALL	4096
diff --git a/libdtrace/dt_cc.c b/libdtrace/dt_cc.c
index d1ee3843..3d9a9c79 100644
--- a/libdtrace/dt_cc.c
+++ b/libdtrace/dt_cc.c
@@ -1076,6 +1076,9 @@ dt_link_construct(dtrace_hdl_t *dtp, const dt_probe_t *prp, dtrace_difo_t *dp,
 				nrp->dofr_data = sizeof(uint64_t)
 				    * dtp->dt_options[DTRACEOPT_MAXFRAMES];
 				continue;
+			case DT_CONST_ERROR_PRID:
+				nrp->dofr_data = dtp->dt_error->desc->id;
+				continue;
 			case DT_CONST_BOOTTM:
 				if (boottime == 0 && get_boottime())
 					return -1;
diff --git a/libdtrace/dt_dlibs.c b/libdtrace/dt_dlibs.c
index bc883e11..1fb561ad 100644
--- a/libdtrace/dt_dlibs.c
+++ b/libdtrace/dt_dlibs.c
@@ -80,6 +80,7 @@ static const dt_ident_t		dt_bpf_symbols[] = {
 	DT_BPF_SYMBOL_ID(STBSZ, DT_IDENT_SCALAR, DT_CONST_STBSZ),
 	DT_BPF_SYMBOL_ID(STRSZ, DT_IDENT_SCALAR, DT_CONST_STRSZ),
 	DT_BPF_SYMBOL_ID(STKSIZ, DT_IDENT_SCALAR, DT_CONST_STKSIZ),
+	DT_BPF_SYMBOL_ID(ERROR_PRID, DT_IDENT_SCALAR, DT_CONST_ERROR_PRID),
 	DT_BPF_SYMBOL_ID(BOOTTM, DT_IDENT_SCALAR, DT_CONST_BOOTTM),
 	DT_BPF_SYMBOL_ID(NSPEC, DT_IDENT_SCALAR, DT_CONST_NSPEC),
 	DT_BPF_SYMBOL_ID(NCPUS, DT_IDENT_SCALAR, DT_CONST_NCPUS),
-- 
2.18.4

[PATCH 30/38] Allow relocation on BPF_OR instructions

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_as.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libdtrace/dt_as.c b/libdtrace/dt_as.c
index a634b855..4b397f51 100644
--- a/libdtrace/dt_as.c
+++ b/libdtrace/dt_as.c
@@ -280,6 +280,7 @@ dt_as(dt_pcb_t *pcb)
 		case BPF_ST | BPF_MEM | BPF_DW:		/* stdw */
 		case BPF_ALU64 | BPF_MOV | BPF_K:	/* mov */
 		case BPF_ALU64 | BPF_ADD | BPF_K:	/* add */
+		case BPF_ALU64 | BPF_OR  | BPF_K:	/* or */
 			if (idp->di_flags & DT_IDFLG_BPF)
 				brel++;
 			else
@@ -492,6 +493,7 @@ fail:
 			case BPF_ST | BPF_MEM | BPF_DW:		/* stdw */
 			case BPF_ALU64 | BPF_MOV | BPF_K:	/* mov */
 			case BPF_ALU64 | BPF_ADD | BPF_K:	/* add */
+			case BPF_ALU64 | BPF_OR  | BPF_K:	/* or */
 				rp->dofr_type = R_BPF_64_32;
 				break;
 			case BPF_LD | BPF_IMM | BPF_DW:		/* lddw */
-- 
2.18.4

[PATCH 31/38] Fix dt_pebs_init() call

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

The function had a few issues, mainly with its comments,
ranging from typos to incorrect descriptions of behavior.

The function has only one caller.  The enforcement of a
size minimum was problematic in a number of respects:
- it was missing 4 bytes
- it was enforcing the minimum before the size was
    increased by the caller
- it was checking dt_pebs_init()==-1,
    missing errors with other negative return values

So change the function to accept a minimum and change its
return values.

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_peb.c                          | 27 ++++++++++-----------
 libdtrace/dt_peb.h                          |  2 +-
 libdtrace/dt_work.c                         | 15 +++++++-----
 test/unittest/options/err.b-too-low.d       | 22 +++++++++--------
 test/unittest/options/err.bufsize-too-low.d | 22 +++++++++--------
 test/unittest/options/tst.b.d               | 22 +++++++++--------
 test/unittest/options/tst.bufsize.d         | 22 +++++++++--------
 7 files changed, 71 insertions(+), 61 deletions(-)

diff --git a/libdtrace/dt_peb.c b/libdtrace/dt_peb.c
index 5268f089..4983ba91 100644
--- a/libdtrace/dt_peb.c
+++ b/libdtrace/dt_peb.c
@@ -136,18 +136,14 @@ dt_pebs_exit(dtrace_hdl_t *dtp)
 }
 
 /*
- * Initialize the perf event buffers (one per online CPU).  Each buffer will
- * the given number of pages (i.e. the total size of each buffer will be
- * num_pages * getpagesize()).  The  allocated memory for each buffer is mmap'd
- * so the kernel can write to it, and its representative file descriptor is
- * recorded in the 'buffers' BPF map so that BPF code knows where to write
- * trace data for a specific CPU.
+ * Initialize the perf event buffers, one per online CPU.  Each buffer will
+ * have num_pages * getpagesize()).  The dt_peb_open() call mmaps the allocated
+ * memory so the kernel can write to it.  The file descriptor is recorded in
+ * the 'buffers' BPF map and added to the event polling file descriptor.
  *
- * An event polling file descriptor is created as well, and it is configured to
- * monitor all perf event buffers at once.  This file descriptor is returned
- * upon success..  Failure is indicated with a -1 return value.
+ * The return value indicates success (0) or failure (-1).
  */
-int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
+int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize, size_t bufsizemin)
 {
 	int		i;
 	int		mapfd;
@@ -166,12 +162,15 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
 		fprintf(stderr, "bufsize increased to %lu\n",
 			num_pages * getpagesize());
 
+	if (num_pages * getpagesize() < bufsizemin)
+		return dt_set_errno(dtp, EDT_BUFTOOSMALL);
+
 	/*
 	 * Determine the fd for the 'buffers' BPF map.
 	 */
 	idp = dt_dlib_get_map(dtp, "buffers");
 	if (idp == NULL || idp->di_id == DT_IDENT_UNDEF)
-		return -ENOENT;
+		return dt_set_errno(dtp, EDT_NOMEM);    // FIXME we used to do something more akin to ENOENT
 
 	mapfd = idp->di_id;
 
@@ -180,7 +179,7 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
 	 */
 	dtp->dt_pebset = dt_zalloc(dtp, sizeof(dt_pebset_t));
 	if (dtp->dt_pebset == NULL)
-		return -ENOMEM;
+		return dt_set_errno(dtp, EDT_NOMEM);
 
 	/*
 	 * Allocate the per-CPU perf event buffers.
@@ -189,7 +188,7 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
 			 sizeof(struct dt_peb));
 	if (pebs == NULL) {
 		dt_free(dtp, dtp->dt_pebset);
-		return -ENOMEM;
+		return dt_set_errno(dtp, EDT_NOMEM);
 	}
 
 	dtp->dt_pebset->pebs = pebs;
@@ -241,5 +240,5 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
 fail:
 	dt_pebs_exit(dtp);
 
-	return -1;
+	return dt_set_errno(dtp, EDT_NOMEM);  // FIXME need something better
 }
diff --git a/libdtrace/dt_peb.h b/libdtrace/dt_peb.h
index e0f408f2..9ad23252 100644
--- a/libdtrace/dt_peb.h
+++ b/libdtrace/dt_peb.h
@@ -43,7 +43,7 @@ typedef struct dt_pebset {
 } dt_pebset_t;
 
 extern void dt_pebs_exit(dtrace_hdl_t *);
-extern int dt_pebs_init(dtrace_hdl_t *, size_t);
+extern int dt_pebs_init(dtrace_hdl_t *, size_t, size_t);
 
 #ifdef	__cplusplus
 }
diff --git a/libdtrace/dt_work.c b/libdtrace/dt_work.c
index 69a86358..7a0eb1da 100644
--- a/libdtrace/dt_work.c
+++ b/libdtrace/dt_work.c
@@ -267,18 +267,21 @@ dtrace_go(dtrace_hdl_t *dtp, uint_t cflags)
 		return dt_set_errno(dtp, errno);
 
 	/*
-	 * We need enough space for the pref_event_header, a 32-bit size, a
+	 * We need enough space for the perf_event_header, a 32-bit size, a
 	 * 4-byte gap, and the largest trace data record we may be writing to
 	 * the buffer.  In other words, the buffer needs to be large enough to
 	 * hold at least one perf-encapsulated trace data record.
+	 *
+	 * While dt_pebs_init() rounds the requested size up, size==0 is a
+	 * special case.
 	 */
 	dtrace_getopt(dtp, "bufsize", &size);
-	if (size == 0 ||
-	    size < sizeof(struct perf_event_header) + sizeof(uint32_t) +
-		   dtp->dt_maxreclen)
+	if (size == 0)
 		return dt_set_errno(dtp, EDT_BUFTOOSMALL);
-	if (dt_pebs_init(dtp, size) == -1)
-		return dt_set_errno(dtp, EDT_NOMEM);
+	if (dt_pebs_init(dtp, size,
+			 sizeof(struct perf_event_header) + sizeof(uint32_t)
+			 + 4 + dtp->dt_maxreclen) == -1)
+		return -1;
 
 	/*
 	 * We must initialize the aggregation consumer handling before we
diff --git a/test/unittest/options/err.b-too-low.d b/test/unittest/options/err.b-too-low.d
index bb77e37c..f62155dd 100644
--- a/test/unittest/options/err.b-too-low.d
+++ b/test/unittest/options/err.b-too-low.d
@@ -1,6 +1,6 @@
 /*
  * Oracle Linux DTrace.
- * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
  * Licensed under the Universal Permissive License v 1.0 as shown at
  * http://oss.oracle.com/licenses/upl.
  */
@@ -12,19 +12,21 @@
  */
 
 /*
- * We use a buffer size of 59 because that should be just too small to hold the
- * trace records generated in this script:
- *	- perf_event_header (40 bytes)
- *	- size (4 bytes)
- *	- gap (4 bytes)
- *	- EPID (4 bytes)
- *	- tag (4 bytes)
- *	- exit value (4 bytes)
+ * We need over 4k bytes for the 4 string trace records generated in this script
+ * plus some meta data.
+ * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
+ * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
  */
-/* @@runtest-opts: -b59 */
+/* @@runtest-opts: -b4096 */
+
+#pragma D option strsize=1024
 
 BEGIN
 {
+	trace("abc");
+	trace("def");
+	trace("ghi");
+	trace("jkl");
 	exit(0);
 }
 
diff --git a/test/unittest/options/err.bufsize-too-low.d b/test/unittest/options/err.bufsize-too-low.d
index bbbdb5c5..25efdf72 100644
--- a/test/unittest/options/err.bufsize-too-low.d
+++ b/test/unittest/options/err.bufsize-too-low.d
@@ -1,6 +1,6 @@
 /*
  * Oracle Linux DTrace.
- * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
  * Licensed under the Universal Permissive License v 1.0 as shown at
  * http://oss.oracle.com/licenses/upl.
  */
@@ -12,19 +12,21 @@
  */
 
 /*
- * We use a buffer size of 59 because that should be just too small to hold the
- * trace records generated in this script:
- *	- perf_event_header (40 bytes)
- *	- size (4 bytes)
- *	- gap (4 bytes)
- *	- EPID (4 bytes)
- *	- tag (4 bytes)
- *	- exit value (4 bytes)
+ * We need over 4k bytes for the 4 string trace records generated in this script
+ * plus some meta data.
+ * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
+ * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
  */
-/* @@runtest-opts: -xbufsize=59 */
+/* @@runtest-opts: -xbufsize=4096 */
+
+#pragma D option strsize=1024
 
 BEGIN
 {
+	trace("abc");
+	trace("def");
+	trace("ghi");
+	trace("jkl");
 	exit(0);
 }
 
diff --git a/test/unittest/options/tst.b.d b/test/unittest/options/tst.b.d
index 57fa030d..3bf08edc 100644
--- a/test/unittest/options/tst.b.d
+++ b/test/unittest/options/tst.b.d
@@ -1,6 +1,6 @@
 /*
  * Oracle Linux DTrace.
- * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
  * Licensed under the Universal Permissive License v 1.0 as shown at
  * http://oss.oracle.com/licenses/upl.
  */
@@ -12,19 +12,21 @@
  */
 
 /*
- * We use a buffer size of 60 because that should be the exact size necessary
- * to hold the trace records generated in this script:
- *	- perf_event_header (40 bytes)
- *	- size (4 bytes)
- *	- gap (4 bytes)
- *	- EPID (4 bytes)
- *	- tag (4 bytes)
- *	- exit value (4 bytes)
+ * We need over 4k bytes for the 4 string trace records generated in this script
+ * plus some meta data.
+ * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
+ * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
  */
-/* @@runtest-opts: -b60 */
+/* @@runtest-opts: -b4097 */
+
+#pragma D option strsize=1024
 
 BEGIN
 {
+	trace("abc");
+	trace("def");
+	trace("ghi");
+	trace("jkl");
 	exit(0);
 }
 
diff --git a/test/unittest/options/tst.bufsize.d b/test/unittest/options/tst.bufsize.d
index 96b0f1b8..23af81aa 100644
--- a/test/unittest/options/tst.bufsize.d
+++ b/test/unittest/options/tst.bufsize.d
@@ -1,6 +1,6 @@
 /*
  * Oracle Linux DTrace.
- * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
  * Licensed under the Universal Permissive License v 1.0 as shown at
  * http://oss.oracle.com/licenses/upl.
  */
@@ -12,19 +12,21 @@
  */
 
 /*
- * We use a buffer size of 60 because that should be the exact size necessary
- * to hold the trace records generated in this script:
- *	- perf_event_header (40 bytes)
- *	- size (4 bytes)
- *	- gap (4 bytes)
- *	- EPID (4 bytes)
- *	- tag (4 bytes)
- *	- exit value (4 bytes)
+ * We need over 4k bytes for the 4 string trace records generated in this script
+ * plus some meta data.
+ * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
+ * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
  */
-/* @@runtest-opts: -xbufsize=60 */
+/* @@runtest-opts: -xbufsize=4097 */
+
+#pragma D option strsize=1024
 
 BEGIN
 {
+	trace("abc");
+	trace("def");
+	trace("ghi");
+	trace("jkl");
 	exit(0);
 }
 
-- 
2.18.4

[PATCH 32/38] Widen the EPID to include the PRID

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Each output record has a EPID associated with it, allowing the consumer
to get both a data description and a PRID (so it can report PRID and
probe function and name).  We want to support uprobes that trigger for
multiple PRIDs, however, basically breaking this scheme.

So, expand the EPID to 64 bits, half for the old-style EPID (based on
static information about the D programs) and half for the PRID (discovered
at run time).

The EPID seen by compile-time files like dt_cc.c or dt_map.c are still
the old-style EPIDs (even if now 64-bit) since they have no knowledge
of the run-time PRIDs.

Further, we do not need to expand:
- the mstate (since it already had the PRID anyhow)
- the output buffer (since it already had an unused 4-byte pad)

The combination of EPID and PRID is for the built-in variable epid as
well as what the consumer reads.

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 bpf/get_bvar.c                                |  6 ++-
 bpf/probe_error.c                             |  7 ++-
 include/dtrace/universal.h                    |  2 +-
 libdtrace/dt_bpf.c                            |  2 -
 libdtrace/dt_cg.c                             | 43 +++++++++++--------
 libdtrace/dt_consume.c                        | 28 ++++++++----
 libdtrace/dt_dctx.h                           | 12 ++++--
 libdtrace/dt_handle.c                         |  9 ++--
 libdtrace/dt_map.c                            |  3 ++
 libdtrace/dt_open.c                           |  2 +-
 test/demo/dtrace/error.r                      |  2 +-
 test/stress/buffering/tst.resize3-manual.r    |  2 +-
 test/stress/buffering/tst.resize3.r           |  2 +-
 test/unittest/actions/setopt/tst.badopt.r     | 14 +++---
 .../arrays/tst.declared-bounds.runtime_out.r  |  2 +-
 test/unittest/codegen/err.deref_0.r           |  2 +-
 test/unittest/codegen/err.deref_1.r           |  2 +-
 test/unittest/codegen/err.deref_i0.r          |  2 +-
 test/unittest/codegen/err.deref_i1.r          |  2 +-
 .../unittest/codegen/err.deref_string-assoc.r |  2 +-
 test/unittest/codegen/err.deref_string-gvar.r |  2 +-
 test/unittest/codegen/err.deref_string-lvar.r |  2 +-
 test/unittest/codegen/err.deref_string-tvar.r |  2 +-
 .../codegen/err.str_NULL_plus_offset-assoc.r  |  2 +-
 .../codegen/err.str_NULL_plus_offset-lvar.r   |  2 +-
 .../codegen/err.str_NULL_plus_offset-tvar.r   |  2 +-
 .../codegen/err.str_NULL_plus_offset.r        |  2 +-
 test/unittest/disasm/tst.vartab-bvar.r        |  2 +-
 .../tst.DTRACEFLT_BADADDR.null_ptr_field.r    |  2 +-
 test/unittest/error/tst.DTRACEFLT_BADADDR.r   |  4 +-
 .../error/tst.DTRACEFLT_DIVZERO.div.r         |  2 +-
 .../error/tst.DTRACEFLT_DIVZERO.mod.r         |  2 +-
 test/unittest/error/tst.DTRACEFLT_UNKNOWN.r   |  4 +-
 .../error/tst.clause_scope-begin-ended.r      |  2 +-
 test/unittest/error/tst.clause_scope-begin.r  |  2 +-
 .../error/tst.clause_scope-regular.r.p        | 12 +++++-
 test/unittest/error/tst.error.r               |  2 +-
 test/unittest/error/tst.errorend.r            |  2 +-
 .../alloca/err.alloca-bcopy-before-beyond.r   |  2 +-
 .../alloca/err.alloca-bcopy-before-bottom.r   |  2 +-
 .../alloca/err.alloca-bcopy-beyond-top.r      |  2 +-
 .../alloca/err.alloca-bcopy-crossing-bottom.r |  2 +-
 .../alloca/err.alloca-bcopy-crossing-top.r    |  2 +-
 .../alloca/err.alloca-crossing-clauses.r      |  2 +-
 .../alloca/err.alloca-load-before-bottom.r    |  2 +-
 .../funcs/alloca/err.alloca-load-beyond-top.r |  2 +-
 .../alloca/err.alloca-load-crossing-bottom.r  |  2 +-
 .../alloca/err.alloca-null-deref-lvalue.r     |  2 +-
 .../funcs/alloca/err.alloca-null-deref.r      |  2 +-
 .../err.alloca-scratch-exceeding-bcopy.r      |  2 +-
 .../alloca/err.alloca-store-before-bottom.r   |  2 +-
 .../alloca/err.alloca-store-beyond-top.r      |  2 +-
 .../alloca/err.alloca-store-crossing-bottom.r |  2 +-
 test/unittest/funcs/bcopy/err.badbcopy1.r     |  2 +-
 test/unittest/funcs/bcopy/err.badbcopy4.r     |  2 +-
 test/unittest/funcs/bcopy/err.badbcopy5.r     |  2 +-
 test/unittest/funcs/bcopy/err.badbcopy6.r     |  2 +-
 test/unittest/funcs/bcopy/err.badbcopy7.r     |  2 +-
 test/unittest/funcs/bcopy/err.badbcopy8.r     |  2 +-
 test/unittest/funcs/copyin/err.badaddr.r      |  2 +-
 test/unittest/funcs/copyin/err.null_arg1.r    |  2 +-
 test/unittest/funcs/copyinstr/err.badaddr.r   |  2 +-
 test/unittest/funcs/copyinstr/err.null_arg1.r |  2 +-
 test/unittest/funcs/copyinto/err.badaddr.r    |  2 +-
 test/unittest/funcs/copyinto/err.badsize.r    |  2 +-
 test/unittest/funcs/copyinto/err.null_arg1.r  |  2 +-
 test/unittest/funcs/err.badalloca.r.p         | 12 +++++-
 test/unittest/funcs/err.link_ntopbadaddr.r    |  2 +-
 test/unittest/funcs/err.link_ntopbadarg.r     |  2 +-
 .../inet_ntoa6/err.inet_ntoa6.arg1_null.r     |  2 +-
 .../err.inet_ntoa6.arg1_null_const.r          |  2 +-
 test/unittest/funcs/strlen/tst.null.r         |  2 +-
 test/unittest/funcs/strtok/tst.strtok_null.r  |  2 +-
 .../funcs/strtok/tst.strtok_nulldel.r         |  2 +-
 .../funcs/strtok/tst.strtok_nullstr.r         |  2 +-
 .../funcs/strtok/tst.strtok_nullstr2.r        |  2 +-
 .../funcs/substr/err.substr_null_arg1.r       |  2 +-
 test/unittest/pointers/err.AllocaOverrun.r    |  2 +-
 test/unittest/pointers/err.BadAlign.r         |  2 +-
 test/unittest/pointers/err.InvalidAddress2.r  |  2 +-
 test/unittest/pointers/err.InvalidAddress4.r  |  2 +-
 .../speculation/err.CommitWithInvalid.r       |  2 +-
 .../speculation/err.DiscardWithInvalid.r      |  2 +-
 .../speculation/tst.SpecSizeVariations.r      | 22 ----------
 .../speculation/tst.SpecSizeVariations.sh     |  2 +-
 test/unittest/speculation/tst.negcommit.r     |  2 +-
 86 files changed, 174 insertions(+), 148 deletions(-)

diff --git a/bpf/get_bvar.c b/bpf/get_bvar.c
index a0c04f3a..5673c94b 100644
--- a/bpf/get_bvar.c
+++ b/bpf/get_bvar.c
@@ -48,8 +48,10 @@ noinline uint64_t dt_get_bvar(const dt_dctx_t *dctx, uint32_t id, uint32_t idx)
 			mst->tstamp = bpf_ktime_get_ns();
 
 		return mst->tstamp;
-	case DIF_VAR_EPID:
-		return mst->epid;
+	case DIF_VAR_EPID: {
+		uint64_t val = mst->prid;
+		return (val << 32) | mst->epid;
+	}
 	case DIF_VAR_ID:
 		return mst->prid;
 	case DIF_VAR_ARG0: case DIF_VAR_ARG1: case DIF_VAR_ARG2:
diff --git a/bpf/probe_error.c b/bpf/probe_error.c
index c8ddcdfa..c8b21d7c 100644
--- a/bpf/probe_error.c
+++ b/bpf/probe_error.c
@@ -7,6 +7,8 @@
 #include <bpf/bpf_helpers.h>
 #include <dt_dctx.h>
 
+extern uint64_t ERROR_PRID;
+
 #ifndef noinline
 # define noinline	__attribute__((noinline))
 #endif
@@ -26,15 +28,18 @@ noinline void dt_probe_error(const dt_dctx_t *dctx, uint64_t pc, uint64_t fault,
 			     uint64_t illval)
 {
 	dt_mstate_t	*mst = dctx->mst;
+	int		oldprid = mst->prid;
 
 	mst->argv[0] = 0;
-	mst->argv[1] = mst->epid;
+	mst->argv[1] = (((uint64_t)mst->prid) << 32) | mst->epid;
 	mst->argv[2] = mst->clid;
 	mst->argv[3] = pc;
 	mst->argv[4] = fault;
 	mst->argv[5] = illval;
 
+	mst->prid = ((uint64_t)&ERROR_PRID);
 	dt_error(dctx);
+	mst->prid = oldprid;
 
 	mst->fault = fault;
 }
diff --git a/include/dtrace/universal.h b/include/dtrace/universal.h
index d6562489..655ea772 100644
--- a/include/dtrace/universal.h
+++ b/include/dtrace/universal.h
@@ -37,7 +37,7 @@ typedef uint16_t	dtrace_actkind_t;	/* action kind */
 
 typedef uint32_t	dtrace_aggid_t;		/* aggregation identifier */
 typedef uint32_t	dtrace_cacheid_t;	/* predicate cache identifier */
-typedef uint32_t	dtrace_epid_t;		/* enabled probe identifier */
+typedef uint64_t	dtrace_epid_t;		/* enabled probe identifier */
 typedef uint32_t	dtrace_optid_t;		/* option identifier */
 typedef uint32_t	dtrace_specid_t;	/* speculation identifier */
 
diff --git a/libdtrace/dt_bpf.c b/libdtrace/dt_bpf.c
index 71c6a446..428cb407 100644
--- a/libdtrace/dt_bpf.c
+++ b/libdtrace/dt_bpf.c
@@ -778,7 +778,6 @@ gmap_create_cpuinfo(dtrace_hdl_t *dtp)
  * The size of the memory region is the sum of:
  *	- size of the DTrace machine state, rounded up to the nearest
  *	  multiple of 8
- *	- 8 bytes padding for trace buffer alignment purposes
  *	- maximum trace buffer record size, rounded up to the nearest
  *	  multiple of 8
  *	- size of dctx->mem (see dt_dctx.h)
@@ -787,7 +786,6 @@ static int
 gmap_create_mem(dtrace_hdl_t *dtp)
 {
 	size_t	sz = roundup(sizeof(dt_mstate_t), 8) +
-		     8 +
 		     roundup(dtp->dt_maxreclen, 8) +
 		     DMEM_SIZE(dtp);
 
diff --git a/libdtrace/dt_cg.c b/libdtrace/dt_cg.c
index 2fb2d0d8..d27a8cb2 100644
--- a/libdtrace/dt_cg.c
+++ b/libdtrace/dt_cg.c
@@ -275,15 +275,9 @@ dt_cg_tramp_prologue_act(dt_pcb_t *pcb, dt_activity_t act)
 	 *	buf = rc + roundup(sizeof(dt_mstate_t), 8);
 	 *				// add %r0, roundup(
 	 *						sizeof(dt_mstate_t), 8)
-	 *	*((uint64_t *)&buf[0]) = 0;
-	 *				// stdw [%r0 + 0], 0
-	 *	buf += 8;		// add %r0, 8
-	 *				//     (%r0 = pointer to buffer space)
 	 *	dctx.buf = buf;		// stdw [%r9 + DCTX_BUF], %r0
 	 */
 	emit(dlp,  BPF_ALU64_IMM(BPF_ADD, BPF_REG_0, roundup(sizeof(dt_mstate_t), 8)));
-	emit(dlp,  BPF_STORE_IMM(BPF_DW, BPF_REG_0, 0, 0));
-	emit(dlp,  BPF_ALU64_IMM(BPF_ADD, BPF_REG_0, 8));
 	emit(dlp,  BPF_STORE(BPF_DW, BPF_REG_9, DCTX_BUF, BPF_REG_0));
 
 	/*
@@ -1094,8 +1088,6 @@ dt_cg_prologue(dt_pcb_t *pcb, dt_node_t *pred)
 	 *	dctx->mst->specsize = 0;// stdw [%r0 + DMST_SPECSIZE], 0
 	 *	dctx->mst->epid = EPID;	// stw [%r0 + DMST_EPID], EPID
 	 *	dctx->mst->clid = CLID;	// stw [%r0 + DMST_CLID], CLID
-	 *	*((uint32_t *)&buf[DBUF_EPID]) = EPID;
-	 *				// stw [%r9 + DBUF_EPID], EPID
 	 */
 	emit(dlp,  BPF_LOAD(BPF_DW, BPF_REG_0, BPF_REG_0, DCTX_MST));
 	emit(dlp,  BPF_STORE_IMM(BPF_DW, BPF_REG_0, DMST_FAULT, 0));
@@ -1103,7 +1095,13 @@ dt_cg_prologue(dt_pcb_t *pcb, dt_node_t *pred)
 	emit(dlp,  BPF_STORE_IMM(BPF_DW, BPF_REG_0, DMST_SPECSIZE, 0));
 	emite(dlp, BPF_STORE_IMM(BPF_W, BPF_REG_0, DMST_EPID, -1), epid);
 	emite(dlp, BPF_STORE_IMM(BPF_W, BPF_REG_0, DMST_CLID, -1), clid);
-	emite(dlp, BPF_STORE_IMM(BPF_W, BPF_REG_9, DBUF_EPID, -1), epid);
+
+	/*
+	 *	Zero out the leading 4 bytes of the buffer.
+	 *	*((uint32_t *)&buf[DBUF_PAD]) = 0;
+	 *				// stw [%r9 + DBUF_PAD], 0
+	 */
+	emit(dlp,  BPF_STORE_IMM(BPF_W, BPF_REG_9, DBUF_PAD, 0));
 
 	/*
 	 *	Set the speculation ID field to zero to indicate no active
@@ -1113,6 +1111,18 @@ dt_cg_prologue(dt_pcb_t *pcb, dt_node_t *pred)
 	 */
 	emit(dlp,  BPF_STORE_IMM(BPF_W, BPF_REG_9, DBUF_SPECID, 0));
 
+	/*
+	 *	*((uint64_t *)&buf[DBUF_EPID]) = (dctx->mst->prid << 32) | EPID;
+	 *				// ld %r1, [%r0 + DMST_PRID]
+	 *				// lsh %r1, 32
+	 *				// or %r1, epid
+	 *				// stdw [%r9 + DBUF_EPID], %r1
+	 */
+	emit (dlp, BPF_LOAD(BPF_W, BPF_REG_1, BPF_REG_0, DMST_PRID));
+	emit (dlp, BPF_ALU64_IMM(BPF_LSH, BPF_REG_1, 32));
+	emite(dlp, BPF_ALU64_IMM(BPF_OR, BPF_REG_1, -1), epid);
+	emit (dlp, BPF_STORE(BPF_DW, BPF_REG_9, DBUF_EPID, BPF_REG_1));
+
 	/*
 	 * If there is a predicate:
 	 *
@@ -1131,10 +1141,9 @@ dt_cg_prologue(dt_pcb_t *pcb, dt_node_t *pred)
 	TRACE_REGSET("Prologue: End  ");
 
 	/*
-	 * Account for 32-bit EPID (at offset 0) and 32-bit speculation ID (at
-	 * offset 4).
+	 * Set the offset for the beginning of trace data.
 	 */
-	pcb->pcb_bufoff += 2 * sizeof(uint32_t);
+	pcb->pcb_bufoff = DBUF_DATA;
 }
 
 /*
@@ -1169,15 +1178,15 @@ dt_cg_epilogue(dt_pcb_t *pcb)
 		/*
 		 *	rc = bpf_perf_event_output(dctx->ctx, &buffers,
 		 *				   BPF_F_CURRENT_CPU,
-		 *				   buf - 4, bufoff + 4);
+		 *				   buf + 4, bufoff - 4);
 		 *				// lddw %r1, [%fp + DT_STK_DCTX]
 		 *				// lddw %r1, [%r1 + DCTX_CTX]
 		 *				// lddw %r2, &buffers
 		 *				// lddw %r3, BPF_F_CURRENT_CPU
 		 *				// mov %r4, %r9
-		 *				// add %r4, -4
+		 *				// add %r4, 4
 		 *				// mov %r5, pcb->pcb_bufoff
-		 *				// add %r5, 4
+		 *				// add %r5, -4
 		 *				// call bpf_perf_event_output
 		 */
 		emit(dlp, BPF_LOAD(BPF_DW, BPF_REG_1, BPF_REG_FP, DT_STK_DCTX));
@@ -1185,9 +1194,9 @@ dt_cg_epilogue(dt_pcb_t *pcb)
 		dt_cg_xsetx(dlp, buffers, DT_LBL_NONE, BPF_REG_2, buffers->di_id);
 		dt_cg_xsetx(dlp, NULL, DT_LBL_NONE, BPF_REG_3, BPF_F_CURRENT_CPU);
 		emit(dlp, BPF_MOV_REG(BPF_REG_4, BPF_REG_9));
-		emit(dlp, BPF_ALU64_IMM(BPF_ADD, BPF_REG_4, -4));
+		emit(dlp, BPF_ALU64_IMM(BPF_ADD, BPF_REG_4, 4));
 		emit(dlp, BPF_MOV_IMM(BPF_REG_5, pcb->pcb_bufoff));
-		emit(dlp, BPF_ALU64_IMM(BPF_ADD, BPF_REG_5, 4));
+		emit(dlp, BPF_ALU64_IMM(BPF_ADD, BPF_REG_5, -4));
 		emit(dlp, BPF_CALL_HELPER(BPF_FUNC_perf_event_output));
 
 		/*
diff --git a/libdtrace/dt_consume.c b/libdtrace/dt_consume.c
index 7dfec72f..adbd2c7f 100644
--- a/libdtrace/dt_consume.c
+++ b/libdtrace/dt_consume.c
@@ -14,9 +14,11 @@
 #include <ctype.h>
 #include <alloca.h>
 #include <dt_impl.h>
+#include <dt_dctx.h>
 #include <dt_module.h>
 #include <dt_pcap.h>
 #include <dt_peb.h>
+#include <dt_probe.h>
 #include <dt_state.h>
 #include <dt_string.h>
 #include <libproc.h>
@@ -478,7 +480,7 @@ dt_flowindent(dtrace_hdl_t *dtp, dtrace_probedata_t *data, dtrace_epid_t last)
 	 */
 	if (flow == DTRACEFLOW_ENTRY) {
 		if (last != DTRACE_EPIDNONE && id != last &&
-		    pd->id == dtp->dt_pdesc[last]->id)
+		    pd->id == dtp->dt_probes[last >> 32]->desc->id)
 			flow = DTRACEFLOW_NONE;
 	}
 
@@ -2202,19 +2204,23 @@ dt_consume_one_probe(dtrace_hdl_t *dtp, FILE *fp, char *data, uint32_t size,
 		     int peekflags, dtrace_epid_t *last, int committing,
 		     void *arg)
 {
+	int			specid;
 	dtrace_epid_t		epid;
+	uint32_t		prid;
 	dtrace_datadesc_t	*epd;
 	dt_spec_buf_t		tmpl;
 	dt_spec_buf_t		*dtsb;
-	int			specid;
 	int			i;
 	int			rval;
 	dtrace_workstatus_t	ret;
 	int			commit_discard_seen, only_commit_discards;
 	int			data_recording = 1;
 
-	epid = ((uint32_t *)data)[0];
-	specid = ((uint32_t *)data)[1];
+	specid = *((uint32_t *)(data + DBUF_SPECID));
+	epid = *((uint64_t *)(data + DBUF_EPID));
+	prid = epid >> 32;
+	if (prid > dtp->dt_probe_id)
+		return dt_set_errno(dtp, EDT_BADEPID);
 
 	/*
 	 * Fill in the epid and address of the epid in the buffer.  We need to
@@ -2227,6 +2233,7 @@ dt_consume_one_probe(dtrace_hdl_t *dtp, FILE *fp, char *data, uint32_t size,
 					 &pdat->dtpda_pdesc);
 	if (rval != 0)
 		return dt_set_errno(dtp, EDT_BADEPID);
+	pdat->dtpda_pdesc = (dtrace_probedesc_t *)dtp->dt_probes[prid]->desc;
 
 	epd = pdat->dtpda_ddesc;
 	if (epd->dtdd_uarg != DT_ECB_DEFAULT) {
@@ -2661,9 +2668,8 @@ dt_consume_one(dtrace_hdl_t *dtp, FILE *fp, char *buf,
 		 * struct {
 		 *	struct perf_event_header	header;
 		 *	uint32_t			size;
-		 *	uint32_t			pad;
-		 *	uint32_t			epid;
 		 *	uint32_t			specid;
+		 *	dtrace_epid_t			epid;
 		 *	uint64_t			data[n];
 		 * }
 		 * and 'data' points to the 'size' member at this point.
@@ -2673,13 +2679,17 @@ dt_consume_one(dtrace_hdl_t *dtp, FILE *fp, char *buf,
 			return dt_set_errno(dtp, EDT_DSIZE);
 
 		size = *(uint32_t *)data;
-		data += sizeof(size);
 		ptr += sizeof(size) + size;
 		if (ptr != buf + hdr->size)
 			return dt_set_errno(dtp, EDT_DSIZE);
 
-		data += sizeof(uint32_t);		/* skip padding */
-		size -= sizeof(uint32_t);
+		/*
+		 * The "size" measures from specid to the end.  But our buffer
+		 * offsets are relative to &size itself, to preserve 8-byte
+		 * alignment.  So, we leave data pointing at size, and we increase
+		 * size by 4 bytes.
+		 */
+		size += 4;
 
 		return dt_consume_one_probe(dtp, fp, data, size, pdat, efunc,
 					    rfunc, flow, quiet, peekflags,
diff --git a/libdtrace/dt_dctx.h b/libdtrace/dt_dctx.h
index 1422ad24..6d38b0c2 100644
--- a/libdtrace/dt_dctx.h
+++ b/libdtrace/dt_dctx.h
@@ -82,16 +82,20 @@ typedef struct dt_dctx {
  * The dctx->buf pointer references a block of memory that contains:
  *
  *                       +----------------+
- *                  0 -> | EPID           |
+ *                  0 -> | pad            |
  *                       +----------------+
- *		    4 -> | Speculation ID |
+ *                  4 -> | Speculation ID |
  *                       +----------------+
- *                       | Trace Data     |
+ *                  8 -> | EPID           |
+ *                       +----------------+
+ *                 16 -> | Trace Data     |
  *                       |      ...       |
  *                       +----------------+
  */
-#define DBUF_EPID	0
+#define DBUF_PAD	0
 #define DBUF_SPECID	4
+#define DBUF_EPID	8
+#define DBUF_DATA	16
 
 /*
  * The dctx->mem pointer references a block of memory that contains:
diff --git a/libdtrace/dt_handle.c b/libdtrace/dt_handle.c
index 4c9b9413..b1ba5f9f 100644
--- a/libdtrace/dt_handle.c
+++ b/libdtrace/dt_handle.c
@@ -14,6 +14,7 @@
 #include <alloca.h>
 
 #include <dt_impl.h>
+#include <dt_probe.h>
 #include <dt_program.h>
 
 static const char _dt_errprog[] =
@@ -147,11 +148,11 @@ dt_handle_err(dtrace_hdl_t *dtp, dtrace_probedata_t *data)
 	 * This is an error.  We have the following items here:  EPID,
 	 * faulting action, BPF pc, fault code and faulting address.
 	 */
-	epid = (uint32_t)DT_REC(uint64_t, 0);
+	epid = DT_REC(uint64_t, 0);
 
 	if (dt_epid_lookup(dtp, epid, &errdd, &errpd) != 0)
 		return dt_set_errno(dtp, EDT_BADERROR);
-
+	errpd = (dtrace_probedesc_t *)dtp->dt_probes[epid>>32]->desc;
 	err.dteda_ddesc = errdd;
 	err.dteda_pdesc = errpd;
 	err.dteda_cpu = data->dtpda_cpu;
@@ -195,7 +196,7 @@ no_addr:
 		details[0] = 0;
 	}
 
-	snprintf(str, len, "error on enabled probe ID %u (ID %u: %s:%s:%s:%s): "
+	snprintf(str, len, "error on enabled probe ID %lu (ID %u: %s:%s:%s:%s): "
 			   "%s%s in %s%s",
 		 epid, errpd->id, errpd->prv, errpd->mod, errpd->fun,
 		 errpd->prb, dtrace_faultstr(dtp, err.dteda_fault), details,
@@ -256,7 +257,7 @@ dt_handle_liberr(dtrace_hdl_t *dtp, const dtrace_probedata_t *data,
 	str = alloca(len);
 
 	snprintf(str, len,
-		 "error on enabled probe ID %u (ID %u: %s:%s:%s:%s): %s",
+		 "error on enabled probe ID %lu (ID %u: %s:%s:%s:%s): %s",
 		 data->dtpda_epid, errpd->id, errpd->prv, errpd->mod,
 		 errpd->fun, errpd->prb, faultstr);
 
diff --git a/libdtrace/dt_map.c b/libdtrace/dt_map.c
index c685274d..fe6194a7 100644
--- a/libdtrace/dt_map.c
+++ b/libdtrace/dt_map.c
@@ -137,6 +137,9 @@ int
 dt_epid_lookup(dtrace_hdl_t *dtp, dtrace_epid_t epid, dtrace_datadesc_t **ddp,
 	       dtrace_probedesc_t **pdp)
 {
+	/* Remove the PRID portion of the EPID. */
+	epid &= 0xffffffff;
+
 	if (epid >= dtp->dt_maxprobe ||
 	    dtp->dt_ddesc[epid] == NULL || dtp->dt_pdesc[epid] == NULL)
 		return -1;
diff --git a/libdtrace/dt_open.c b/libdtrace/dt_open.c
index 5c922488..0550379f 100644
--- a/libdtrace/dt_open.c
+++ b/libdtrace/dt_open.c
@@ -169,7 +169,7 @@ static const dt_ident_t _dtrace_globals[] = {
 { "discard", DT_IDENT_ACTFUNC, 0, DT_ACT_DISCARD, DT_ATTR_STABCMN, DT_VERS_1_0,
 	&dt_idops_func, "void(int)" },
 { "epid", DT_IDENT_SCALAR, 0, DIF_VAR_EPID, DT_ATTR_STABCMN, DT_VERS_1_0,
-	&dt_idops_type, "uint_t" },
+	&dt_idops_type, "uint64_t" },
 { "errno", DT_IDENT_SCALAR, 0, DIF_VAR_ERRNO, DT_ATTR_STABCMN, DT_VERS_1_0,
 	&dt_idops_type, "int" },
 { "execname", DT_IDENT_SCALAR, 0, DIF_VAR_EXECNAME,
diff --git a/test/demo/dtrace/error.r b/test/demo/dtrace/error.r
index d3904f47..2504e793 100644
--- a/test/demo/dtrace/error.r
+++ b/test/demo/dtrace/error.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/demo/dtrace/error.d' matched 2 probes
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/stress/buffering/tst.resize3-manual.r b/test/stress/buffering/tst.resize3-manual.r
index 43b647c7..25819f15 100644
--- a/test/stress/buffering/tst.resize3-manual.r
+++ b/test/stress/buffering/tst.resize3-manual.r
@@ -1,5 +1,5 @@
                    FUNCTION:NAME
-                          :BEGIN           3
+                          :BEGIN            4294967299
                           :BEGIN 
 
 -- @@stderr --
diff --git a/test/stress/buffering/tst.resize3.r b/test/stress/buffering/tst.resize3.r
index 9c471158..5e6afe81 100644
--- a/test/stress/buffering/tst.resize3.r
+++ b/test/stress/buffering/tst.resize3.r
@@ -1,5 +1,5 @@
                    FUNCTION:NAME
-                          :BEGIN           3
+                          :BEGIN            4294967299
                           :BEGIN 
 
 -- @@stderr --
diff --git a/test/unittest/actions/setopt/tst.badopt.r b/test/unittest/actions/setopt/tst.badopt.r
index 29e39fd4..b1c6d148 100644
--- a/test/unittest/actions/setopt/tst.badopt.r
+++ b/test/unittest/actions/setopt/tst.badopt.r
@@ -1,16 +1,16 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): couldn't set option "Nixon" to "1": Invalid option name
+dtrace: error on enabled probe ID 4294967298 (ID 1: dtrace:::BEGIN): couldn't set option "Nixon" to "1": Invalid option name
 
-dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): couldn't set option "Harding" to "1": Invalid option name
+dtrace: error on enabled probe ID 4294967298 (ID 1: dtrace:::BEGIN): couldn't set option "Harding" to "1": Invalid option name
 
-dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): couldn't set option "Hoover" to "1": Invalid option name
+dtrace: error on enabled probe ID 4294967298 (ID 1: dtrace:::BEGIN): couldn't set option "Hoover" to "1": Invalid option name
 
-dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): couldn't set option "Bush" to "1": Invalid option name
+dtrace: error on enabled probe ID 4294967298 (ID 1: dtrace:::BEGIN): couldn't set option "Bush" to "1": Invalid option name
 
-dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): couldn't set option "quiet" to "um, no": Invalid value for specified option
+dtrace: error on enabled probe ID 4294967298 (ID 1: dtrace:::BEGIN): couldn't set option "quiet" to "um, no": Invalid value for specified option
 
-dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): couldn't set option "aggrate" to "0.5hz": Invalid value for specified option
+dtrace: error on enabled probe ID 4294967298 (ID 1: dtrace:::BEGIN): couldn't set option "aggrate" to "0.5hz": Invalid value for specified option
 
-dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): couldn't set option "bufsize" to "1m": Operation illegal when tracing is active
+dtrace: error on enabled probe ID 4294967298 (ID 1: dtrace:::BEGIN): couldn't set option "bufsize" to "1m": Operation illegal when tracing is active
 
diff --git a/test/unittest/arrays/tst.declared-bounds.runtime_out.r b/test/unittest/arrays/tst.declared-bounds.runtime_out.r
index 4917528d..fa6e3b73 100644
--- a/test/unittest/arrays/tst.declared-bounds.runtime_out.r
+++ b/test/unittest/arrays/tst.declared-bounds.runtime_out.r
@@ -1,3 +1,3 @@
 expected run-time error
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): index out of bounds (8) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): index out of bounds (8) in action #1 at BPF pc NNN
diff --git a/test/unittest/codegen/err.deref_0.r b/test/unittest/codegen/err.deref_0.r
index 07c1dc52..ef71d557 100644
--- a/test/unittest/codegen/err.deref_0.r
+++ b/test/unittest/codegen/err.deref_0.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address (0) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address (0) in action #1 at BPF pc NNN
diff --git a/test/unittest/codegen/err.deref_1.r b/test/unittest/codegen/err.deref_1.r
index a2ca8ac4..11bf0b20 100644
--- a/test/unittest/codegen/err.deref_1.r
+++ b/test/unittest/codegen/err.deref_1.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address (1) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address (1) in action #1 at BPF pc NNN
diff --git a/test/unittest/codegen/err.deref_i0.r b/test/unittest/codegen/err.deref_i0.r
index 07c1dc52..ef71d557 100644
--- a/test/unittest/codegen/err.deref_i0.r
+++ b/test/unittest/codegen/err.deref_i0.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address (0) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address (0) in action #1 at BPF pc NNN
diff --git a/test/unittest/codegen/err.deref_i1.r b/test/unittest/codegen/err.deref_i1.r
index a2ca8ac4..11bf0b20 100644
--- a/test/unittest/codegen/err.deref_i1.r
+++ b/test/unittest/codegen/err.deref_i1.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address (1) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address (1) in action #1 at BPF pc NNN
diff --git a/test/unittest/codegen/err.deref_string-assoc.r b/test/unittest/codegen/err.deref_string-assoc.r
index 08277992..b047a341 100644
--- a/test/unittest/codegen/err.deref_string-assoc.r
+++ b/test/unittest/codegen/err.deref_string-assoc.r
@@ -1,3 +1,3 @@
 66
 -- @@stderr --
-dtrace: error on enabled probe ID 4 (ID 1: dtrace:::BEGIN): invalid address (1) in action #2 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967300 (ID 1: dtrace:::BEGIN): invalid address (1) in action #2 at BPF pc NNN
diff --git a/test/unittest/codegen/err.deref_string-gvar.r b/test/unittest/codegen/err.deref_string-gvar.r
index 08277992..b047a341 100644
--- a/test/unittest/codegen/err.deref_string-gvar.r
+++ b/test/unittest/codegen/err.deref_string-gvar.r
@@ -1,3 +1,3 @@
 66
 -- @@stderr --
-dtrace: error on enabled probe ID 4 (ID 1: dtrace:::BEGIN): invalid address (1) in action #2 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967300 (ID 1: dtrace:::BEGIN): invalid address (1) in action #2 at BPF pc NNN
diff --git a/test/unittest/codegen/err.deref_string-lvar.r b/test/unittest/codegen/err.deref_string-lvar.r
index 08277992..b047a341 100644
--- a/test/unittest/codegen/err.deref_string-lvar.r
+++ b/test/unittest/codegen/err.deref_string-lvar.r
@@ -1,3 +1,3 @@
 66
 -- @@stderr --
-dtrace: error on enabled probe ID 4 (ID 1: dtrace:::BEGIN): invalid address (1) in action #2 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967300 (ID 1: dtrace:::BEGIN): invalid address (1) in action #2 at BPF pc NNN
diff --git a/test/unittest/codegen/err.deref_string-tvar.r b/test/unittest/codegen/err.deref_string-tvar.r
index 08277992..b047a341 100644
--- a/test/unittest/codegen/err.deref_string-tvar.r
+++ b/test/unittest/codegen/err.deref_string-tvar.r
@@ -1,3 +1,3 @@
 66
 -- @@stderr --
-dtrace: error on enabled probe ID 4 (ID 1: dtrace:::BEGIN): invalid address (1) in action #2 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967300 (ID 1: dtrace:::BEGIN): invalid address (1) in action #2 at BPF pc NNN
diff --git a/test/unittest/codegen/err.str_NULL_plus_offset-assoc.r b/test/unittest/codegen/err.str_NULL_plus_offset-assoc.r
index 187543b6..64a3861b 100644
--- a/test/unittest/codegen/err.str_NULL_plus_offset-assoc.r
+++ b/test/unittest/codegen/err.str_NULL_plus_offset-assoc.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/codegen/err.str_NULL_plus_offset-lvar.r b/test/unittest/codegen/err.str_NULL_plus_offset-lvar.r
index 187543b6..64a3861b 100644
--- a/test/unittest/codegen/err.str_NULL_plus_offset-lvar.r
+++ b/test/unittest/codegen/err.str_NULL_plus_offset-lvar.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/codegen/err.str_NULL_plus_offset-tvar.r b/test/unittest/codegen/err.str_NULL_plus_offset-tvar.r
index 187543b6..64a3861b 100644
--- a/test/unittest/codegen/err.str_NULL_plus_offset-tvar.r
+++ b/test/unittest/codegen/err.str_NULL_plus_offset-tvar.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/codegen/err.str_NULL_plus_offset.r b/test/unittest/codegen/err.str_NULL_plus_offset.r
index 187543b6..64a3861b 100644
--- a/test/unittest/codegen/err.str_NULL_plus_offset.r
+++ b/test/unittest/codegen/err.str_NULL_plus_offset.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/disasm/tst.vartab-bvar.r b/test/unittest/disasm/tst.vartab-bvar.r
index 06d7c52b..53e5f618 100644
--- a/test/unittest/disasm/tst.vartab-bvar.r
+++ b/test/unittest/disasm/tst.vartab-bvar.r
@@ -4,7 +4,7 @@ curthread               scl glb r    D type (pointer) (size 8)
 timestamp               scl glb r    D type (integer) (size 8)
 vtimestamp              scl glb r    D type (integer) (size 8)
 ipl                     scl glb r    D type (integer) (size 4)
-epid                    scl glb r    D type (integer) (size 4)
+epid                    scl glb r    D type (integer) (size 8)
 id                      scl glb r    D type (integer) (size 4)
 arg0                    scl glb r    D type (integer) (size 8)
 arg1                    scl glb r    D type (integer) (size 8)
diff --git a/test/unittest/error/tst.DTRACEFLT_BADADDR.null_ptr_field.r b/test/unittest/error/tst.DTRACEFLT_BADADDR.null_ptr_field.r
index 187543b6..64a3861b 100644
--- a/test/unittest/error/tst.DTRACEFLT_BADADDR.null_ptr_field.r
+++ b/test/unittest/error/tst.DTRACEFLT_BADADDR.null_ptr_field.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/error/tst.DTRACEFLT_BADADDR.r b/test/unittest/error/tst.DTRACEFLT_BADADDR.r
index b9f5f43c..5556955d 100644
--- a/test/unittest/error/tst.DTRACEFLT_BADADDR.r
+++ b/test/unittest/error/tst.DTRACEFLT_BADADDR.r
@@ -1,6 +1,6 @@
-The arguments are 3 1 1 0
+The arguments are 4294967299 1 1 0
 The value of arg4 should be 1
 The value of arg5 should be 0
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/error/tst.DTRACEFLT_DIVZERO.div.r b/test/unittest/error/tst.DTRACEFLT_DIVZERO.div.r
index e6d6afa2..80cd485b 100644
--- a/test/unittest/error/tst.DTRACEFLT_DIVZERO.div.r
+++ b/test/unittest/error/tst.DTRACEFLT_DIVZERO.div.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): divide-by-zero in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): divide-by-zero in action #1 at BPF pc NNN
diff --git a/test/unittest/error/tst.DTRACEFLT_DIVZERO.mod.r b/test/unittest/error/tst.DTRACEFLT_DIVZERO.mod.r
index e6d6afa2..80cd485b 100644
--- a/test/unittest/error/tst.DTRACEFLT_DIVZERO.mod.r
+++ b/test/unittest/error/tst.DTRACEFLT_DIVZERO.mod.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): divide-by-zero in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): divide-by-zero in action #1 at BPF pc NNN
diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
index 1e4fdd64..31752a9e 100644
--- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
+++ b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
@@ -1,5 +1,5 @@
-The arguments are 3 1 PC 1 64
+The arguments are 4294967299 1 PC 1 64
 The value of arg4 = 0
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/error/tst.clause_scope-begin-ended.r b/test/unittest/error/tst.clause_scope-begin-ended.r
index 8d57382e..7c88b227 100644
--- a/test/unittest/error/tst.clause_scope-begin-ended.r
+++ b/test/unittest/error/tst.clause_scope-begin-ended.r
@@ -2,4 +2,4 @@ Error fired
 Clause executed
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/error/tst.clause_scope-begin.r b/test/unittest/error/tst.clause_scope-begin.r
index 8d57382e..7c88b227 100644
--- a/test/unittest/error/tst.clause_scope-begin.r
+++ b/test/unittest/error/tst.clause_scope-begin.r
@@ -2,4 +2,4 @@ Error fired
 Clause executed
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/error/tst.clause_scope-regular.r.p b/test/unittest/error/tst.clause_scope-regular.r.p
index 7659601b..8117b8ab 100755
--- a/test/unittest/error/tst.clause_scope-regular.r.p
+++ b/test/unittest/error/tst.clause_scope-regular.r.p
@@ -1,3 +1,11 @@
-#!/bin/sed -f
+#!/usr/bin/awk -f
+
 # This report has a variable probe ID in it.
-s/ID [0-9][0-9]*: profile/ID nnn: profile/
+/^dtrace: error on enabled probe ID / {
+    $7 = and($7, 0xffffffff);    # mask out the high 32 bits (variable prid)
+}
+
+{
+    sub("ID [0-9][0-9]*: profile", "ID nnn: profile");
+    print;
+}
diff --git a/test/unittest/error/tst.error.r b/test/unittest/error/tst.error.r
index 0d29bcc8..82c6da2d 100644
--- a/test/unittest/error/tst.error.r
+++ b/test/unittest/error/tst.error.r
@@ -1,4 +1,4 @@
 Error fired
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/error/tst.errorend.r b/test/unittest/error/tst.errorend.r
index 73abf697..17cbee17 100644
--- a/test/unittest/error/tst.errorend.r
+++ b/test/unittest/error/tst.errorend.r
@@ -2,4 +2,4 @@ Error fired
 End fired after exit
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-bcopy-before-beyond.r b/test/unittest/funcs/alloca/err.alloca-bcopy-before-beyond.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/alloca/err.alloca-bcopy-before-beyond.r
+++ b/test/unittest/funcs/alloca/err.alloca-bcopy-before-beyond.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-bcopy-before-bottom.r b/test/unittest/funcs/alloca/err.alloca-bcopy-before-bottom.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/alloca/err.alloca-bcopy-before-bottom.r
+++ b/test/unittest/funcs/alloca/err.alloca-bcopy-before-bottom.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-bcopy-beyond-top.r b/test/unittest/funcs/alloca/err.alloca-bcopy-beyond-top.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/alloca/err.alloca-bcopy-beyond-top.r
+++ b/test/unittest/funcs/alloca/err.alloca-bcopy-beyond-top.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-bcopy-crossing-bottom.r b/test/unittest/funcs/alloca/err.alloca-bcopy-crossing-bottom.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/alloca/err.alloca-bcopy-crossing-bottom.r
+++ b/test/unittest/funcs/alloca/err.alloca-bcopy-crossing-bottom.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-bcopy-crossing-top.r b/test/unittest/funcs/alloca/err.alloca-bcopy-crossing-top.r
index 4257f567..9638fa5b 100644
--- a/test/unittest/funcs/alloca/err.alloca-bcopy-crossing-top.r
+++ b/test/unittest/funcs/alloca/err.alloca-bcopy-crossing-top.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid size ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid size ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-crossing-clauses.r b/test/unittest/funcs/alloca/err.alloca-crossing-clauses.r
index f5ff855d..35cf68b1 100644
--- a/test/unittest/funcs/alloca/err.alloca-crossing-clauses.r
+++ b/test/unittest/funcs/alloca/err.alloca-crossing-clauses.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 4 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #2 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967300 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #2 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-load-before-bottom.r b/test/unittest/funcs/alloca/err.alloca-load-before-bottom.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/alloca/err.alloca-load-before-bottom.r
+++ b/test/unittest/funcs/alloca/err.alloca-load-before-bottom.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-load-beyond-top.r b/test/unittest/funcs/alloca/err.alloca-load-beyond-top.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/alloca/err.alloca-load-beyond-top.r
+++ b/test/unittest/funcs/alloca/err.alloca-load-beyond-top.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-load-crossing-bottom.r b/test/unittest/funcs/alloca/err.alloca-load-crossing-bottom.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/alloca/err.alloca-load-crossing-bottom.r
+++ b/test/unittest/funcs/alloca/err.alloca-load-crossing-bottom.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-null-deref-lvalue.r b/test/unittest/funcs/alloca/err.alloca-null-deref-lvalue.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/alloca/err.alloca-null-deref-lvalue.r
+++ b/test/unittest/funcs/alloca/err.alloca-null-deref-lvalue.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-null-deref.r b/test/unittest/funcs/alloca/err.alloca-null-deref.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/alloca/err.alloca-null-deref.r
+++ b/test/unittest/funcs/alloca/err.alloca-null-deref.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-scratch-exceeding-bcopy.r b/test/unittest/funcs/alloca/err.alloca-scratch-exceeding-bcopy.r
index 4257f567..9638fa5b 100644
--- a/test/unittest/funcs/alloca/err.alloca-scratch-exceeding-bcopy.r
+++ b/test/unittest/funcs/alloca/err.alloca-scratch-exceeding-bcopy.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid size ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid size ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-store-before-bottom.r b/test/unittest/funcs/alloca/err.alloca-store-before-bottom.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/alloca/err.alloca-store-before-bottom.r
+++ b/test/unittest/funcs/alloca/err.alloca-store-before-bottom.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-store-beyond-top.r b/test/unittest/funcs/alloca/err.alloca-store-beyond-top.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/alloca/err.alloca-store-beyond-top.r
+++ b/test/unittest/funcs/alloca/err.alloca-store-beyond-top.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/alloca/err.alloca-store-crossing-bottom.r b/test/unittest/funcs/alloca/err.alloca-store-crossing-bottom.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/alloca/err.alloca-store-crossing-bottom.r
+++ b/test/unittest/funcs/alloca/err.alloca-store-crossing-bottom.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/bcopy/err.badbcopy1.r b/test/unittest/funcs/bcopy/err.badbcopy1.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/bcopy/err.badbcopy1.r
+++ b/test/unittest/funcs/bcopy/err.badbcopy1.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/bcopy/err.badbcopy4.r b/test/unittest/funcs/bcopy/err.badbcopy4.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/bcopy/err.badbcopy4.r
+++ b/test/unittest/funcs/bcopy/err.badbcopy4.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/bcopy/err.badbcopy5.r b/test/unittest/funcs/bcopy/err.badbcopy5.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/bcopy/err.badbcopy5.r
+++ b/test/unittest/funcs/bcopy/err.badbcopy5.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/bcopy/err.badbcopy6.r b/test/unittest/funcs/bcopy/err.badbcopy6.r
index 4257f567..9638fa5b 100644
--- a/test/unittest/funcs/bcopy/err.badbcopy6.r
+++ b/test/unittest/funcs/bcopy/err.badbcopy6.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid size ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid size ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/bcopy/err.badbcopy7.r b/test/unittest/funcs/bcopy/err.badbcopy7.r
index 4257f567..9638fa5b 100644
--- a/test/unittest/funcs/bcopy/err.badbcopy7.r
+++ b/test/unittest/funcs/bcopy/err.badbcopy7.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid size ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid size ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/bcopy/err.badbcopy8.r b/test/unittest/funcs/bcopy/err.badbcopy8.r
index 4257f567..9638fa5b 100644
--- a/test/unittest/funcs/bcopy/err.badbcopy8.r
+++ b/test/unittest/funcs/bcopy/err.badbcopy8.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid size ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid size ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/copyin/err.badaddr.r b/test/unittest/funcs/copyin/err.badaddr.r
index ba4a4695..dadaa45b 100644
--- a/test/unittest/funcs/copyin/err.badaddr.r
+++ b/test/unittest/funcs/copyin/err.badaddr.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/unittest/funcs/copyin/err.badaddr.d' matched 2 probes
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/copyin/err.null_arg1.r b/test/unittest/funcs/copyin/err.null_arg1.r
index a806d107..c0739a4d 100644
--- a/test/unittest/funcs/copyin/err.null_arg1.r
+++ b/test/unittest/funcs/copyin/err.null_arg1.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/unittest/funcs/copyin/err.null_arg1.d' matched 2 probes
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/copyinstr/err.badaddr.r b/test/unittest/funcs/copyinstr/err.badaddr.r
index 0f566d6e..76861803 100644
--- a/test/unittest/funcs/copyinstr/err.badaddr.r
+++ b/test/unittest/funcs/copyinstr/err.badaddr.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/unittest/funcs/copyinstr/err.badaddr.d' matched 2 probes
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/copyinstr/err.null_arg1.r b/test/unittest/funcs/copyinstr/err.null_arg1.r
index cdd7c22c..074882e0 100644
--- a/test/unittest/funcs/copyinstr/err.null_arg1.r
+++ b/test/unittest/funcs/copyinstr/err.null_arg1.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/unittest/funcs/copyinstr/err.null_arg1.d' matched 2 probes
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/copyinto/err.badaddr.r b/test/unittest/funcs/copyinto/err.badaddr.r
index 11861e1f..9c2e65c2 100644
--- a/test/unittest/funcs/copyinto/err.badaddr.r
+++ b/test/unittest/funcs/copyinto/err.badaddr.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/unittest/funcs/copyinto/err.badaddr.d' matched 2 probes
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/copyinto/err.badsize.r b/test/unittest/funcs/copyinto/err.badsize.r
index ec4b062e..7e99c307 100644
--- a/test/unittest/funcs/copyinto/err.badsize.r
+++ b/test/unittest/funcs/copyinto/err.badsize.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/unittest/funcs/copyinto/err.badsize.d' matched 2 probes
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid size ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid size ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/copyinto/err.null_arg1.r b/test/unittest/funcs/copyinto/err.null_arg1.r
index f568ee5c..7c1fa9e0 100644
--- a/test/unittest/funcs/copyinto/err.null_arg1.r
+++ b/test/unittest/funcs/copyinto/err.null_arg1.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/unittest/funcs/copyinto/err.null_arg1.d' matched 2 probes
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/err.badalloca.r.p b/test/unittest/funcs/err.badalloca.r.p
index d7a88a39..15f99035 100755
--- a/test/unittest/funcs/err.badalloca.r.p
+++ b/test/unittest/funcs/err.badalloca.r.p
@@ -1,3 +1,11 @@
-#!/bin/sed -f
+#!/usr/bin/awk -f
 
-s/(ID [0-9]*/(ID NNN/g
+# This report has a variable probe ID in it.
+/^dtrace: error on enabled probe ID / {
+    $7 = and($7, 0xffffffff);    # mask out the high 32 bits (variable prid)
+}
+
+{
+    sub("ID [0-9]*: profile", "ID NNN: profile");
+    print;
+}
diff --git a/test/unittest/funcs/err.link_ntopbadaddr.r b/test/unittest/funcs/err.link_ntopbadaddr.r
index b798b5e0..f0973529 100644
--- a/test/unittest/funcs/err.link_ntopbadaddr.r
+++ b/test/unittest/funcs/err.link_ntopbadaddr.r
@@ -1,3 +1,3 @@
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
 
diff --git a/test/unittest/funcs/err.link_ntopbadarg.r b/test/unittest/funcs/err.link_ntopbadarg.r
index e386a67c..3c677e9f 100644
--- a/test/unittest/funcs/err.link_ntopbadarg.r
+++ b/test/unittest/funcs/err.link_ntopbadarg.r
@@ -1,3 +1,3 @@
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): illegal operation in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): illegal operation in action #1 at BPF pc NNN
 
diff --git a/test/unittest/funcs/inet_ntoa6/err.inet_ntoa6.arg1_null.r b/test/unittest/funcs/inet_ntoa6/err.inet_ntoa6.arg1_null.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/inet_ntoa6/err.inet_ntoa6.arg1_null.r
+++ b/test/unittest/funcs/inet_ntoa6/err.inet_ntoa6.arg1_null.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/inet_ntoa6/err.inet_ntoa6.arg1_null_const.r b/test/unittest/funcs/inet_ntoa6/err.inet_ntoa6.arg1_null_const.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/inet_ntoa6/err.inet_ntoa6.arg1_null_const.r
+++ b/test/unittest/funcs/inet_ntoa6/err.inet_ntoa6.arg1_null_const.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/strlen/tst.null.r b/test/unittest/funcs/strlen/tst.null.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/strlen/tst.null.r
+++ b/test/unittest/funcs/strlen/tst.null.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/strtok/tst.strtok_null.r b/test/unittest/funcs/strtok/tst.strtok_null.r
index 03226aa1..4c51c5e9 100644
--- a/test/unittest/funcs/strtok/tst.strtok_null.r
+++ b/test/unittest/funcs/strtok/tst.strtok_null.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/unittest/funcs/strtok/tst.strtok_null.d' matched 2 probes
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/strtok/tst.strtok_nulldel.r b/test/unittest/funcs/strtok/tst.strtok_nulldel.r
index 70f8e4e2..2ab9938d 100644
--- a/test/unittest/funcs/strtok/tst.strtok_nulldel.r
+++ b/test/unittest/funcs/strtok/tst.strtok_nulldel.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/unittest/funcs/strtok/tst.strtok_nulldel.d' matched 2 probes
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/strtok/tst.strtok_nullstr.r b/test/unittest/funcs/strtok/tst.strtok_nullstr.r
index a57b2469..be5ffe14 100644
--- a/test/unittest/funcs/strtok/tst.strtok_nullstr.r
+++ b/test/unittest/funcs/strtok/tst.strtok_nullstr.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/unittest/funcs/strtok/tst.strtok_nullstr.d' matched 2 probes
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/funcs/strtok/tst.strtok_nullstr2.r b/test/unittest/funcs/strtok/tst.strtok_nullstr2.r
index d7df3aca..6c6fe35a 100644
--- a/test/unittest/funcs/strtok/tst.strtok_nullstr2.r
+++ b/test/unittest/funcs/strtok/tst.strtok_nullstr2.r
@@ -4,4 +4,4 @@
 
 -- @@stderr --
 dtrace: script 'test/unittest/funcs/strtok/tst.strtok_nullstr2.d' matched 4 probes
-dtrace: error on enabled probe ID 4 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #2 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967300 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #2 at BPF pc NNN
diff --git a/test/unittest/funcs/substr/err.substr_null_arg1.r b/test/unittest/funcs/substr/err.substr_null_arg1.r
index 187543b6..64a3861b 100644
--- a/test/unittest/funcs/substr/err.substr_null_arg1.r
+++ b/test/unittest/funcs/substr/err.substr_null_arg1.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/pointers/err.AllocaOverrun.r b/test/unittest/pointers/err.AllocaOverrun.r
index 187543b6..64a3861b 100644
--- a/test/unittest/pointers/err.AllocaOverrun.r
+++ b/test/unittest/pointers/err.AllocaOverrun.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/pointers/err.BadAlign.r b/test/unittest/pointers/err.BadAlign.r
index 187543b6..64a3861b 100644
--- a/test/unittest/pointers/err.BadAlign.r
+++ b/test/unittest/pointers/err.BadAlign.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/pointers/err.InvalidAddress2.r b/test/unittest/pointers/err.InvalidAddress2.r
index 187543b6..64a3861b 100644
--- a/test/unittest/pointers/err.InvalidAddress2.r
+++ b/test/unittest/pointers/err.InvalidAddress2.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/pointers/err.InvalidAddress4.r b/test/unittest/pointers/err.InvalidAddress4.r
index 187543b6..64a3861b 100644
--- a/test/unittest/pointers/err.InvalidAddress4.r
+++ b/test/unittest/pointers/err.InvalidAddress4.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967299 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
diff --git a/test/unittest/speculation/err.CommitWithInvalid.r b/test/unittest/speculation/err.CommitWithInvalid.r
index fc072417..e8a71c29 100644
--- a/test/unittest/speculation/err.CommitWithInvalid.r
+++ b/test/unittest/speculation/err.CommitWithInvalid.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 4 (ID 1: dtrace:::BEGIN): illegal operation in action #2 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967300 (ID 1: dtrace:::BEGIN): illegal operation in action #2 at BPF pc NNN
diff --git a/test/unittest/speculation/err.DiscardWithInvalid.r b/test/unittest/speculation/err.DiscardWithInvalid.r
index fc072417..e8a71c29 100644
--- a/test/unittest/speculation/err.DiscardWithInvalid.r
+++ b/test/unittest/speculation/err.DiscardWithInvalid.r
@@ -1,3 +1,3 @@
 
 -- @@stderr --
-dtrace: error on enabled probe ID 4 (ID 1: dtrace:::BEGIN): illegal operation in action #2 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967300 (ID 1: dtrace:::BEGIN): illegal operation in action #2 at BPF pc NNN
diff --git a/test/unittest/speculation/tst.SpecSizeVariations.r b/test/unittest/speculation/tst.SpecSizeVariations.r
index 51f0596c..2748b307 100644
--- a/test/unittest/speculation/tst.SpecSizeVariations.r
+++ b/test/unittest/speculation/tst.SpecSizeVariations.r
@@ -11,26 +11,6 @@ Speculative buffer ID: 1
 123456706
 counts: 1 1
 
-Speculative buffer ID: 1
-123456700
-123456701
-123456702
-123456703
-123456704
-123456705
-123456706
-counts: 1 1
-
-Speculative buffer ID: 1
-123456700
-123456701
-123456702
-123456703
-123456704
-123456705
-123456706
-counts: 2 1
-
 Speculative buffer ID: 1
 123456700
 123456701
@@ -64,5 +44,3 @@ counts: 2 1
 dtrace: 2 speculative drops
 dtrace: 1 speculative drop
 dtrace: 1 speculative drop
-dtrace: 1 speculative drop
-dtrace: 1 speculative drop
diff --git a/test/unittest/speculation/tst.SpecSizeVariations.sh b/test/unittest/speculation/tst.SpecSizeVariations.sh
index 75e527d9..79995b59 100755
--- a/test/unittest/speculation/tst.SpecSizeVariations.sh
+++ b/test/unittest/speculation/tst.SpecSizeVariations.sh
@@ -9,7 +9,7 @@
 
 dtrace=$1
 
-for x in 63 64 79 80 143 144; do
+for x in 71 72 159 160; do
 	$dtrace $dt_flags -xspecsize=$x -qn '
 	BEGIN
 	{
diff --git a/test/unittest/speculation/tst.negcommit.r b/test/unittest/speculation/tst.negcommit.r
index 69f246a0..29a3a538 100644
--- a/test/unittest/speculation/tst.negcommit.r
+++ b/test/unittest/speculation/tst.negcommit.r
@@ -3,4 +3,4 @@
 
 -- @@stderr --
 dtrace: script 'test/unittest/speculation/tst.negcommit.d' matched 2 probes
-dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): illegal operation in action #1 at BPF pc NNN
+dtrace: error on enabled probe ID 4294967298 (ID 1: dtrace:::BEGIN): illegal operation in action #1 at BPF pc NNN
-- 
2.18.4

[PATCH 33/38] Eliminate dt_pdesc

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_cc.c      |  8 +-------
 libdtrace/dt_consume.c |  3 +--
 libdtrace/dt_handle.c  |  2 +-
 libdtrace/dt_impl.h    |  7 ++-----
 libdtrace/dt_map.c     | 36 +++++++-----------------------------
 5 files changed, 12 insertions(+), 44 deletions(-)

diff --git a/libdtrace/dt_cc.c b/libdtrace/dt_cc.c
index 3d9a9c79..d09a9775 100644
--- a/libdtrace/dt_cc.c
+++ b/libdtrace/dt_cc.c
@@ -958,7 +958,6 @@ dt_link_construct(dtrace_hdl_t *dtp, const dt_probe_t *prp, dtrace_difo_t *dp,
 	uint_t			len = sdp->dtdo_brelen;
 	const dof_relodesc_t	*rp = sdp->dtdo_breltab;
 	dof_relodesc_t		*nrp = &dp->dtdo_breltab[rc];
-	dtrace_id_t		prid = prp->desc->id;
 	int			no_deps = 0;
 
 	if (idp != NULL) {
@@ -1199,11 +1198,6 @@ dt_link_construct(dtrace_hdl_t *dtp, const dt_probe_t *prp, dtrace_difo_t *dp,
 			case DT_CONST_STACK_SKIP:
 				nrp->dofr_data = prp->prov->impl->stack_skip;
 				continue;
-			default:
-				/* probe name -> value is probe id */
-				if (strchr(idp->di_name, ':') != NULL)
-					prid = rp->dofr_data;
-				continue;
 			}
 
 			continue;
@@ -1220,7 +1214,7 @@ dt_link_construct(dtrace_hdl_t *dtp, const dt_probe_t *prp, dtrace_difo_t *dp,
 			if (rdp == NULL)
 				return -1;
 			if (rdp->dtdo_ddesc != NULL) {
-				nepid = dt_epid_add(dtp, rdp->dtdo_ddesc, prid);
+				nepid = dt_epid_add(dtp, rdp->dtdo_ddesc);
 				clid++;
 			} else
 				nepid = 0;
diff --git a/libdtrace/dt_consume.c b/libdtrace/dt_consume.c
index adbd2c7f..8d96789e 100644
--- a/libdtrace/dt_consume.c
+++ b/libdtrace/dt_consume.c
@@ -2229,8 +2229,7 @@ dt_consume_one_probe(dtrace_hdl_t *dtp, FILE *fp, char *data, uint32_t size,
 	pdat->dtpda_epid = epid;
 	pdat->dtpda_data = data;
 
-	rval = dt_epid_lookup(dtp, epid, &pdat->dtpda_ddesc,
-					 &pdat->dtpda_pdesc);
+	rval = dt_epid_lookup(dtp, epid, &pdat->dtpda_ddesc);
 	if (rval != 0)
 		return dt_set_errno(dtp, EDT_BADEPID);
 	pdat->dtpda_pdesc = (dtrace_probedesc_t *)dtp->dt_probes[prid]->desc;
diff --git a/libdtrace/dt_handle.c b/libdtrace/dt_handle.c
index b1ba5f9f..79ecbac8 100644
--- a/libdtrace/dt_handle.c
+++ b/libdtrace/dt_handle.c
@@ -150,7 +150,7 @@ dt_handle_err(dtrace_hdl_t *dtp, dtrace_probedata_t *data)
 	 */
 	epid = DT_REC(uint64_t, 0);
 
-	if (dt_epid_lookup(dtp, epid, &errdd, &errpd) != 0)
+	if (dt_epid_lookup(dtp, epid, &errdd) != 0)
 		return dt_set_errno(dtp, EDT_BADERROR);
 	errpd = (dtrace_probedesc_t *)dtp->dt_probes[epid>>32]->desc;
 	err.dteda_ddesc = errdd;
diff --git a/libdtrace/dt_impl.h b/libdtrace/dt_impl.h
index 445cd602..f8799b86 100644
--- a/libdtrace/dt_impl.h
+++ b/libdtrace/dt_impl.h
@@ -371,7 +371,6 @@ struct dtrace_hdl {
 	dtrace_epid_t dt_nextepid; /* next enabled probe ID to assign */
 	size_t dt_maxprobe;	/* max enabled probe ID */
 	dtrace_datadesc_t **dt_ddesc; /* probe data descriptions */
-	dtrace_probedesc_t **dt_pdesc; /* probe descriptions for enabled prbs */
 	size_t dt_maxagg;	/* max aggregation ID */
 	dtrace_aggdesc_t **dt_adesc; /* aggregation descriptions */
 	dt_aggregate_t dt_aggregate; /* aggregate */
@@ -826,10 +825,8 @@ extern dtrace_datadesc_t *dt_datadesc_hold(dtrace_datadesc_t *ddp);
 extern void dt_datadesc_release(dtrace_hdl_t *, dtrace_datadesc_t *);
 extern dtrace_datadesc_t *dt_datadesc_create(dtrace_hdl_t *);
 extern int dt_datadesc_finalize(dtrace_hdl_t *, dtrace_datadesc_t *);
-extern dtrace_epid_t dt_epid_add(dtrace_hdl_t *, dtrace_datadesc_t *,
-				 dtrace_id_t);
-extern int dt_epid_lookup(dtrace_hdl_t *, dtrace_epid_t, dtrace_datadesc_t **,
-			  dtrace_probedesc_t **);
+extern dtrace_epid_t dt_epid_add(dtrace_hdl_t *, dtrace_datadesc_t *);
+extern int dt_epid_lookup(dtrace_hdl_t *, dtrace_epid_t, dtrace_datadesc_t **);
 extern void dt_epid_destroy(dtrace_hdl_t *);
 typedef void (*dt_cg_gap_f)(dt_pcb_t *, int);
 extern uint32_t dt_rec_add(dtrace_hdl_t *, dt_cg_gap_f, dtrace_actkind_t,
diff --git a/libdtrace/dt_map.c b/libdtrace/dt_map.c
index fe6194a7..513224ba 100644
--- a/libdtrace/dt_map.c
+++ b/libdtrace/dt_map.c
@@ -91,7 +91,7 @@ dt_datadesc_finalize(dtrace_hdl_t *dtp, dtrace_datadesc_t *ddp)
  * description.
  */
 dtrace_epid_t
-dt_epid_add(dtrace_hdl_t *dtp, dtrace_datadesc_t *ddp, dtrace_id_t prid)
+dt_epid_add(dtrace_hdl_t *dtp, dtrace_datadesc_t *ddp)
 {
 	dtrace_id_t	max = dtp->dt_maxprobe;
 	dtrace_epid_t	epid;
@@ -100,27 +100,19 @@ dt_epid_add(dtrace_hdl_t *dtp, dtrace_datadesc_t *ddp, dtrace_id_t prid)
 	if (epid >= max || dtp->dt_ddesc == NULL) {
 		dtrace_id_t		nmax = max ? (max << 1) : 2;
 		dtrace_datadesc_t	**nddesc;
-		dtrace_probedesc_t	**npdesc;
 
 		nddesc = dt_calloc(dtp, nmax, sizeof(void *));
-		npdesc = dt_calloc(dtp, nmax, sizeof(void *));
-		if (nddesc == NULL || npdesc == NULL) {
-			dt_free(dtp, nddesc);
-			dt_free(dtp, npdesc);
+		if (nddesc == NULL)
 			return dt_set_errno(dtp, EDT_NOMEM);
-		}
 
 		if (dtp->dt_ddesc != NULL) {
 			size_t	osize = max * sizeof(void *);
 
 			memcpy(nddesc, dtp->dt_ddesc, osize);
 			dt_free(dtp, dtp->dt_ddesc);
-			memcpy(npdesc, dtp->dt_pdesc, osize);
-			dt_free(dtp, dtp->dt_pdesc);
 		}
 
 		dtp->dt_ddesc = nddesc;
-		dtp->dt_pdesc = npdesc;
 		dtp->dt_maxprobe = nmax;
 	}
 
@@ -128,24 +120,20 @@ dt_epid_add(dtrace_hdl_t *dtp, dtrace_datadesc_t *ddp, dtrace_id_t prid)
 		return epid;
 
 	dtp->dt_ddesc[epid] = dt_datadesc_hold(ddp);
-	dtp->dt_pdesc[epid] = (dtrace_probedesc_t *)dtp->dt_probes[prid]->desc;
 
 	return epid;
 }
 
 int
-dt_epid_lookup(dtrace_hdl_t *dtp, dtrace_epid_t epid, dtrace_datadesc_t **ddp,
-	       dtrace_probedesc_t **pdp)
+dt_epid_lookup(dtrace_hdl_t *dtp, dtrace_epid_t epid, dtrace_datadesc_t **ddp)
 {
 	/* Remove the PRID portion of the EPID. */
 	epid &= 0xffffffff;
 
-	if (epid >= dtp->dt_maxprobe ||
-	    dtp->dt_ddesc[epid] == NULL || dtp->dt_pdesc[epid] == NULL)
+	if (epid >= dtp->dt_maxprobe || dtp->dt_ddesc[epid] == NULL)
 		return -1;
 
 	*ddp = dtp->dt_ddesc[epid];
-	*pdp = dtp->dt_pdesc[epid];
 
 	return 0;
 }
@@ -155,26 +143,16 @@ dt_epid_destroy(dtrace_hdl_t *dtp)
 {
 	size_t i;
 
-	assert((dtp->dt_pdesc != NULL && dtp->dt_ddesc != NULL &&
-	    dtp->dt_maxprobe > 0) || (dtp->dt_pdesc == NULL &&
-	    dtp->dt_ddesc == NULL && dtp->dt_maxprobe == 0));
-
-	if (dtp->dt_pdesc == NULL)
-		return;
+	assert((dtp->dt_ddesc != NULL && dtp->dt_maxprobe > 0) ||
+	       (dtp->dt_ddesc == NULL && dtp->dt_maxprobe == 0));
 
 	for (i = 0; i < dtp->dt_maxprobe; i++) {
-		if (dtp->dt_ddesc[i] == NULL) {
-			assert(dtp->dt_pdesc[i] == NULL);
+		if (dtp->dt_ddesc[i] == NULL)
 			continue;
-		}
 
 		dt_datadesc_release(dtp, dtp->dt_ddesc[i]);
-		assert(dtp->dt_pdesc[i] != NULL);
 	}
 
-	free(dtp->dt_pdesc);
-	dtp->dt_pdesc = NULL;
-
 	free(dtp->dt_ddesc);
 	dtp->dt_ddesc = NULL;
 	dtp->dt_nextepid = 0;
-- 
2.18.4

[PATCH 34/38] Create the BPF uprobes map

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

WIP, sizing is hardwired for now.
WIP, should add new pids and purge old ones (after the initial call).

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_bpf.c         | 44 +++++++++++++++++++++++++++
 libdtrace/dt_dlibs.c       |  1 +
 libdtrace/dt_impl.h        |  3 ++
 libdtrace/dt_prov_uprobe.c | 61 ++++++++++++++++++++++++++++++++++++++
 4 files changed, 109 insertions(+)

diff --git a/libdtrace/dt_bpf.c b/libdtrace/dt_bpf.c
index 428cb407..908c2823 100644
--- a/libdtrace/dt_bpf.c
+++ b/libdtrace/dt_bpf.c
@@ -944,6 +944,49 @@ gmap_create_probes(dtrace_hdl_t *dtp)
 	return 0;
 }
 
+/*
+ * Create the 'uprobes' BPF map.
+ *
+ * Uprobe information map.  This is a global hash map for use
+ * with USDT and pid probes).  It is indexed by (pid, probe ID),
+ * using the probe ID of the underlying probe.  The value is a
+ * probe ID for the overlying probe and a bit mask indicating
+ * which clauses to execute for this pid.
+ *
+ * WIP.  Just make up some sizes for now.
+ *
+ *     How big is a probe ID?  Sometimes, it's a dtrace_id_t.
+ *     And uts/common/sys/dtrace_types.h gives us "typedef uint32_t dtrace_id_t".
+ *     Meanwhile, libdtrace/dt_impl.h has "uint32_t dt_probe_id".
+ *     So either uint32_t or dtrace_id_t is fine.
+ *
+ *     How big should our bit mask be?  Start with 8*sizeof(long long) bits.
+ *
+ *     How many entries should we allow?  Start with 1000.
+ *
+ * WIP.  Note that two different overlying probes (differing by more
+ * than just pid) could map to the same underlying probe.  E.g., a
+ * USDT probe would have an underlying probe and pid probe with the
+ * right offset could map to the same uprobe.  This means that clauses
+ * called by the uprobe might have to be called more than once;  one
+ * would need to go through the clauses once for one overlying probe
+ * and then again for the other.  This scenario is not yet addressed.
+ * It is a problem even before this patch, but easier to fix.
+ */
+static int
+gmap_create_uprobes(dtrace_hdl_t *dtp)
+{
+	dtp->dt_uprobesmap_fd = create_gmap(dtp, "uprobes", BPF_MAP_TYPE_HASH,
+			 dtp->dt_uprobesmap_ksz, dtp->dt_uprobesmap_vsz, 1000);
+	if (dtp->dt_uprobesmap_fd == -1)
+		return -1;
+
+	/* Populate the newly created map.  FIXME:  this is probably not the right place for this. */
+	dt_uprobe.update(dtp, NULL);
+
+	return 0;
+}
+
 /*
  * Create the 'gvars' BPF map.
  *
@@ -1049,6 +1092,7 @@ dt_bpf_gmap_create(dtrace_hdl_t *dtp)
 	CREATE_MAP(scratchmem)
 	CREATE_MAP(strtab)
 	CREATE_MAP(probes)
+	CREATE_MAP(uprobes)
 	CREATE_MAP(gvars)
 	CREATE_MAP(lvars)
 	CREATE_MAP(dvars)
diff --git a/libdtrace/dt_dlibs.c b/libdtrace/dt_dlibs.c
index 1fb561ad..357396c8 100644
--- a/libdtrace/dt_dlibs.c
+++ b/libdtrace/dt_dlibs.c
@@ -66,6 +66,7 @@ static const dt_ident_t		dt_bpf_symbols[] = {
 	DT_BPF_SYMBOL(lvars, DT_IDENT_PTR),
 	DT_BPF_SYMBOL(mem, DT_IDENT_PTR),
 	DT_BPF_SYMBOL(probes, DT_IDENT_PTR),
+	DT_BPF_SYMBOL(uprobes, DT_IDENT_PTR),
 	DT_BPF_SYMBOL(scratchmem, DT_IDENT_PTR),
 	DT_BPF_SYMBOL(specs, DT_IDENT_PTR),
 	DT_BPF_SYMBOL(state, DT_IDENT_PTR),
diff --git a/libdtrace/dt_impl.h b/libdtrace/dt_impl.h
index f8799b86..1368d484 100644
--- a/libdtrace/dt_impl.h
+++ b/libdtrace/dt_impl.h
@@ -423,6 +423,9 @@ struct dtrace_hdl {
 	int dt_aggmap_fd;	/* file descriptor for the 'aggs' BPF map */
 	int dt_genmap_fd;	/* file descriptor for the 'agggen' BPF map */
 	int dt_cpumap_fd;	/* file descriptor for the 'cpuinfo' BPF map */
+	int dt_uprobesmap_fd;	/* file descriptor for the 'uprobes' BPF map */
+	int dt_uprobesmap_ksz;	/* 'uprobes' BPF map key size */
+	int dt_uprobesmap_vsz;	/* 'uprobes' BPF map value size */
 	dtrace_handle_err_f *dt_errhdlr; /* error handler, if any */
 	void *dt_errarg;	/* error handler argument */
 	dtrace_handle_drop_f *dt_drophdlr; /* drop handler, if any */
diff --git a/libdtrace/dt_prov_uprobe.c b/libdtrace/dt_prov_uprobe.c
index 5f0c56db..e99f02c3 100644
--- a/libdtrace/dt_prov_uprobe.c
+++ b/libdtrace/dt_prov_uprobe.c
@@ -45,6 +45,7 @@
 #include "dt_probe.h"
 #include "dt_pid.h"
 #include "dt_string.h"
+#include "port.h"
 
 /* Provider name for the underlying probes. */
 static const char	prvname[] = "uprobe";
@@ -69,6 +70,15 @@ typedef struct list_probe {
 	dt_probe_t	*probe;
 } list_probe_t;
 
+typedef struct uprobe_map_key {
+	pid_t		pid;
+	dtrace_id_t	uprid;
+} uprobe_map_key_t;
+typedef struct uprobe_map_val {
+	dtrace_id_t	prid;
+	long long	mask;
+} uprobe_map_val_t;
+
 static const dtrace_pattr_t	pattr = {
 { DTRACE_STABILITY_EVOLVING, DTRACE_STABILITY_EVOLVING, DTRACE_CLASS_ISA },
 { DTRACE_STABILITY_PRIVATE, DTRACE_STABILITY_PRIVATE, DTRACE_CLASS_UNKNOWN },
@@ -83,6 +93,9 @@ dt_provimpl_t	dt_usdt;
 
 static int populate(dtrace_hdl_t *dtp)
 {
+	dtp->dt_uprobesmap_ksz = sizeof(uprobe_map_key_t);
+	dtp->dt_uprobesmap_vsz = sizeof(uprobe_map_val_t);
+
 	if (dt_provider_create(dtp, dt_uprobe.name, &dt_uprobe, &pattr,
 			       NULL) == NULL ||
 	    dt_provider_create(dtp, dt_uprobe_is_enabled.name,
@@ -162,6 +175,53 @@ static int probe_add_clause(dtrace_hdl_t *dtp, dt_probe_t *prp, dt_ident_t *idp)
 	return 0;
 }
 
+/*
+ * Update the uprobe provider.
+ */
+static void update_uprobe(dtrace_hdl_t *dtp, void *datap)
+{
+	dt_probe_t	*prp;
+
+	for (prp = dt_list_next(&dtp->dt_enablings); prp != NULL;
+	     prp = dt_list_next(prp)) {
+		pid_t			pid;
+		const list_probe_t	*pup;
+
+		/* Make sure it is an overlying pid or USDT probe. */
+		if (prp->prov->impl != &dt_pid && prp->prov->impl != &dt_usdt)
+			continue;
+
+		/* FIXME passing in NULL pcb and dpr wreaks havoc on error reporting? */
+		pid = dt_pid_get_pid(prp->desc, dtp, NULL, NULL);
+
+		for (pup = prp->prv_data; pup != NULL; pup = dt_list_next(pup)) {
+			dt_probe_t		*uprp = pup->probe;
+			dt_probe_clause_t       *pcp;
+			long long		mask = 0, bit = 1;
+			uprobe_map_key_t	key;
+			uprobe_map_val_t	val;
+
+			for (pcp = dt_list_next(&uprp->clauses); pcp; pcp = dt_list_next(pcp)) {
+				int n = atoi(pcp->clause->di_name + strlen("dt_clause_"));
+
+				if (gmatch(prp->desc->prv, dtp->dt_retained[n]->prv))
+					mask |= bit;
+
+				bit <<= 1;
+			}
+
+			key.pid = pid;
+			key.uprid = uprp->desc->id;
+
+			val.prid = prp->desc->id;
+			val.mask = mask;
+
+			// FIXME Check return value, but how should errors be handled?
+			dt_bpf_map_update(dtp->dt_uprobesmap_fd, &key, &val);
+		}
+	}
+}
+
 /*
  * Look up or create an underlying (real) probe, corresponding directly to a
  * uprobe.  Since multiple pid and USDT probes may all map onto the same
@@ -820,6 +880,7 @@ dt_provimpl_t	dt_uprobe = {
 	.probe_info	= &probe_info,
 	.detach		= &detach,
 	.probe_destroy	= &probe_destroy_underlying,
+	.update		= &update_uprobe,
 };
 
 /*
-- 
2.18.4

[PATCH 35/38] Use uprobes map to call clauses conditionally

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Check that the is-enabled trampoline is sufficiently tested.

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_prov_uprobe.c | 147 +++++++++++++++++--------------------
 1 file changed, 68 insertions(+), 79 deletions(-)

diff --git a/libdtrace/dt_prov_uprobe.c b/libdtrace/dt_prov_uprobe.c
index e99f02c3..43c77fe4 100644
--- a/libdtrace/dt_prov_uprobe.c
+++ b/libdtrace/dt_prov_uprobe.c
@@ -517,8 +517,11 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 	dt_irlist_t		*dlp = &pcb->pcb_ir;
 	const dt_probe_t	*uprp = pcb->pcb_probe;
 	const dt_uprobe_t	*upp = uprp->prv_data;
-	const list_probe_t	*pop;
 	uint_t			lbl_exit = pcb->pcb_exitlbl;
+	dt_ident_t		*uprobes = dt_dlib_get_map(pcb->pcb_hdl, "uprobes");
+	dt_probe_clause_t       *pcp;
+
+	assert(uprobes != NULL);
 
 	dt_cg_tramp_prologue(pcb);
 
@@ -527,7 +530,6 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 	 *				//     (%r7 = dctx->mst)
 	 *				//     (%r8 = dctx->ctx)
 	 */
-
 	dt_cg_tramp_copy_regs(pcb);
 	if (upp->flags & PP_IS_RETURN)
 		dt_cg_tramp_copy_rval_from_regs(pcb);
@@ -542,47 +544,59 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 	emit(dlp,  BPF_ALU64_IMM(BPF_RSH, BPF_REG_0, 32));
 
 	/*
-	 * Generate a composite conditional clause:
-	 *
-	 *	if (pid == PID1) {
-	 *		dctx->mst->prid = PRID1;
-	 *		< any number of clause calls >
-	 *		goto exit;
-	 *	} else if (pid == PID2) {
-	 *		dctx->mst->prid = PRID2;
-	 *		< any number of clause calls >
-	 *		goto exit;
-	 *	} else if (pid == ...) {
-	 *		< ... >
-	 *	}
-	 *
-	 * It is valid and safe to use %r0 to hold the pid value because there
-	 * are no assignments to %r0 possible in between the conditional
-	 * statements.
+	 * Look up in the BPF uprobes map.  Space for the look-up key will be used
+	 * on the BPF stack at %r9-sizeof(uprobe_map_key_t).  The key comprises the
+	 * pid (in %r0) and the underlying-probe prid.
 	 */
-	for (pop = dt_list_next(&upp->probes); pop != NULL;
-	     pop = dt_list_next(pop)) {
-		const dt_probe_t	*prp = pop->probe;
-		uint_t			lbl_next = dt_irlist_label(dlp);
-		pid_t			pid;
-		dt_ident_t		*idp;
+	emit(dlp,  BPF_STORE(BPF_W, BPF_REG_9, (int)(-sizeof(uprobe_map_key_t)), BPF_REG_0));
+	emit(dlp,  BPF_STORE_IMM(BPF_W, BPF_REG_9, (int)(-sizeof(dtrace_id_t)) /* or -sizeof(uprobe_map_key_t) + sizeof(pid_t) */, uprp->desc->id));
+	dt_cg_xsetx(dlp, uprobes, DT_LBL_NONE, BPF_REG_1, uprobes->di_id);
+	emit(dlp,  BPF_MOV_REG(BPF_REG_2, BPF_REG_9));
+	emit(dlp,  BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, (int)(-sizeof(uprobe_map_key_t))));
+	emit(dlp,  BPF_CALL_HELPER(BPF_FUNC_map_lookup_elem));
+	emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_0, 0, lbl_exit));
+
+	/* Read the PRID from the table lookup and store to mst->prid. */
+	emit(dlp,  BPF_LOAD(BPF_W, BPF_REG_1, BPF_REG_0, 0));
+	emit(dlp,  BPF_STORE(BPF_W, BPF_REG_7, DMST_PRID, BPF_REG_1));
 
-		pid = dt_pid_get_pid(prp->desc, pcb->pcb_hdl, pcb, NULL);
-		assert(pid != -1);
+	/* Read the bit mask from the table lookup in %r6. */    // FIXME someday, extend this past 64 bits
+	emit(dlp,  BPF_LOAD(BPF_DW, BPF_REG_6, BPF_REG_0, offsetof(uprobe_map_val_t, mask)));
 
-		idp = dt_dlib_add_probe_var(pcb->pcb_hdl, prp);
-		assert(idp != NULL);
+	/*
+	 * Hold the bit mask in %r6 between clause calls.
+	 */
+	for (pcp = dt_list_next(&uprp->clauses); pcp; pcp = dt_list_next(pcp)) {
+		dt_ident_t	*idp = pcp->clause;
+		uint_t		lbl_next = dt_irlist_label(dlp);
+
+		/* If the lowest %r6 bit is 0, skip over this clause. */
+		emit(dlp,  BPF_MOV_REG(BPF_REG_1, BPF_REG_6));
+		emit(dlp,  BPF_ALU64_IMM(BPF_AND, BPF_REG_1, 1));
+		emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_1, 0, lbl_next));
 
 		/*
-		 * Check whether this pid-provider probe serves the current
-		 * process, and emit a sequence of clauses for it when it does.
+		 *      if (*dctx.act != act)   // ldw %r0, [%r9 + DCTX_ACT]
+		 *	      goto exit;      // ldw %r0, [%r0 + 0]
+		 *			      // jne %r0, act, lbl_exit
 		 */
-		emit(dlp,  BPF_BRANCH_IMM(BPF_JNE, BPF_REG_0, pid, lbl_next));
-		emite(dlp, BPF_STORE_IMM(BPF_W, BPF_REG_7, DMST_PRID, prp->desc->id), idp);
-		dt_cg_tramp_call_clauses(pcb, prp, DT_ACTIVITY_ACTIVE);
-		emit(dlp,  BPF_JUMP(lbl_exit));
+		emit(dlp,  BPF_LOAD(BPF_DW, BPF_REG_0, BPF_REG_9, DCTX_ACT));
+		emit(dlp,  BPF_LOAD(BPF_W, BPF_REG_0, BPF_REG_0, 0));
+		emit(dlp,  BPF_BRANCH_IMM(BPF_JNE, BPF_REG_0, DT_ACTIVITY_ACTIVE, lbl_exit));
+
+		/* dctx.mst->scratch_top = 8 */
+		emit(dlp,  BPF_STORE_IMM(BPF_W, BPF_REG_7, DMST_SCRATCH_TOP, 8));
+
+		/* Call clause. */
+		emit(dlp,  BPF_MOV_REG(BPF_REG_1, BPF_REG_9));
+		emite(dlp, BPF_CALL_FUNC(idp->di_id), idp);
+
+		/* Finished this clause. */
 		emitl(dlp, lbl_next,
 			   BPF_NOP());
+
+		/* Right-shift %r6. */
+		emit(dlp,  BPF_ALU64_IMM(BPF_RSH, BPF_REG_6, 1));
 	}
 
 	dt_cg_tramp_return(pcb);
@@ -630,10 +644,9 @@ static int trampoline_is_enabled(dt_pcb_t *pcb, uint_t exitlbl)
 {
 	dt_irlist_t		*dlp = &pcb->pcb_ir;
 	const dt_probe_t	*uprp = pcb->pcb_probe;
-	const dt_uprobe_t	*upp = uprp->prv_data;
-	const list_probe_t	*pop;
-	uint_t			lbl_assign = dt_irlist_label(dlp);
-	uint_t			lbl_exit = pcb->pcb_exitlbl;
+	dt_ident_t		*uprobes = dt_dlib_get_map(pcb->pcb_hdl, "uprobes");
+
+	assert(uprobes != NULL);
 
 	dt_cg_tramp_prologue(pcb);
 
@@ -642,8 +655,7 @@ static int trampoline_is_enabled(dt_pcb_t *pcb, uint_t exitlbl)
 	 *				//     (%r7 = dctx->mst)
 	 *				//     (%r8 = dctx->ctx)
 	 */
-
-	dt_cg_tramp_copy_regs(pcb);
+	dt_cg_tramp_copy_regs(pcb); // FIXME, wait, why are we doing this?
 
 	/*
 	 * Copy in the first function argument, a pointer value to which
@@ -660,46 +672,23 @@ static int trampoline_is_enabled(dt_pcb_t *pcb, uint_t exitlbl)
 	emit(dlp,  BPF_ALU64_IMM(BPF_RSH, BPF_REG_0, 32));
 
 	/*
-	 * Generate a composite conditional clause, as above, except that rather
-	 * than emitting call_clauses, we emit copyouts instead, using
-	 * copyout_val() above:
-	 *
-	 *	if (pid == PID1) {
-	 *		goto assign;
-	 *	} else if (pid == PID2) {
-	 *		goto assign;
-	 *	} else if (pid == ...) {
-	 *		goto assign;
-	 *	}
-	 *	goto exit;
-	 *	assign:
-	 *	    *arg0 = 1;
-	 *	goto exit;
-	 *
-	 * It is valid and safe to use %r0 to hold the pid value because there
-	 * are no assignments to %r0 possible in between the conditional
-	 * statements.
+	 * Look up in the BPF uprobes map.  Space for the look-up key will be used
+	 * on the BPF stack at %r9-sizeof(uprobe_map_key_t).  The key comprises the
+	 * pid (in %r0) and the underlying-probe prid.
 	 */
-	for (pop = dt_list_next(&upp->probes); pop != NULL;
-	     pop = dt_list_next(pop)) {
-		const dt_probe_t	*prp = pop->probe;
-		pid_t			pid;
-		dt_ident_t		*idp;
-
-		pid = dt_pid_get_pid(prp->desc, pcb->pcb_hdl, pcb, NULL);
-		assert(pid != -1);
+	emit(dlp,  BPF_STORE(BPF_W, BPF_REG_9, (int)(-sizeof(uprobe_map_key_t)), BPF_REG_0));
+	emit(dlp,  BPF_STORE_IMM(BPF_W, BPF_REG_9, (int)(-sizeof(dtrace_id_t)) /* or -sizeof(uprobe_map_key_t) + sizeof(pid_t) */, uprp->desc->id));
+	dt_cg_xsetx(dlp, uprobes, DT_LBL_NONE, BPF_REG_1, uprobes->di_id);
+	emit(dlp,  BPF_MOV_REG(BPF_REG_2, BPF_REG_9));
+	emit(dlp,  BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, (int)(-sizeof(uprobe_map_key_t))));
+	emit(dlp,  BPF_CALL_HELPER(BPF_FUNC_map_lookup_elem));
+	emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_0, 0, pcb->pcb_exitlbl));
 
-		idp = dt_dlib_add_probe_var(pcb->pcb_hdl, prp);
-		assert(idp != NULL);
-
-		/*
-		 * Check whether this pid-provider probe serves the current
-		 * process, and copy out a 1 into arg 0 if so.
-		 */
-		emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_0, pid, lbl_assign));
-	}
-	emit(dlp,  BPF_JUMP(lbl_exit));
-	copyout_val(pcb, lbl_assign, 1, 0);
+	/*
+	 * If we succeeded, then we use copyout_val() above to assign:
+	 *	    *arg0 = 1;
+	 */
+	copyout_val(pcb, DT_LBL_NONE, 1, 0);  // FIXME: This is the only copyout_val() call site... should we just inline it here?
 
 	dt_cg_tramp_return(pcb);
 
-- 
2.18.4

[PATCH 36/38] Inline copyout_val()

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Squash to "Use uprobes map to call clauses conditionally"?

Why do we call dt_cg_tramp_copy_regs()?

Why do we copy [%r8+PT_REGS_ARG0] to [%r7+DMST_ARG(0)]?  Can we not
use it in place?

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_prov_uprobe.c | 38 ++++++++------------------------------
 1 file changed, 8 insertions(+), 30 deletions(-)

diff --git a/libdtrace/dt_prov_uprobe.c b/libdtrace/dt_prov_uprobe.c
index 43c77fe4..aa605696 100644
--- a/libdtrace/dt_prov_uprobe.c
+++ b/libdtrace/dt_prov_uprobe.c
@@ -604,33 +604,6 @@ static int trampoline(dt_pcb_t *pcb, uint_t exitlbl)
 	return 0;
 }
 
-/*
- * Copy the given immediate value into the address given by the specified probe
- * argument.
- */
-static void
-copyout_val(dt_pcb_t *pcb, uint_t lbl, uint32_t val, int arg)
-{
-	dt_regset_t	*drp = pcb->pcb_regs;
-	dt_irlist_t	*dlp = &pcb->pcb_ir;
-
-	emitl(dlp, lbl, BPF_STORE_IMM(BPF_DW, BPF_REG_FP, DT_TRAMP_SP_SLOT(0),
-		val));
-
-	if (dt_regset_xalloc_args(drp) == -1)
-		longjmp(yypcb->pcb_jmpbuf, EDT_NOREG);
-	emit(dlp, BPF_LOAD(BPF_DW, BPF_REG_1, BPF_REG_7, DMST_ARG(arg)));
-	emit(dlp, BPF_MOV_REG(BPF_REG_2, BPF_REG_FP));
-	emit(dlp, BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, DT_TRAMP_SP_SLOT(0)));
-	emit(dlp, BPF_MOV_IMM(BPF_REG_3, sizeof(uint32_t)));
-	dt_regset_xalloc(drp, BPF_REG_0);
-	emit(dlp, BPF_CALL_HELPER(BPF_FUNC_probe_write_user));
-
-	/* XXX any point error-checking here? What can we possibly do? */
-	dt_regset_free(drp, BPF_REG_0);
-	dt_regset_free_args(drp);
-}
-
 /*
  * Generate a BPF trampoline for an is-enabled probe.  The is-enabled probe
  * prototype looks like:
@@ -661,7 +634,7 @@ static int trampoline_is_enabled(dt_pcb_t *pcb, uint_t exitlbl)
 	 * Copy in the first function argument, a pointer value to which
 	 * the is-enabled state of the probe will be written (necessarily
 	 * 1 if this probe is running at all).
-	 */
+	 */ // FIXME why are we doing this?  can't we use it in-place?
 	emit(dlp,  BPF_LOAD(BPF_DW, BPF_REG_0, BPF_REG_8, PT_REGS_ARG0));
 	emit(dlp,  BPF_STORE(BPF_DW, BPF_REG_7, DMST_ARG(0), BPF_REG_0));
 
@@ -685,10 +658,15 @@ static int trampoline_is_enabled(dt_pcb_t *pcb, uint_t exitlbl)
 	emit(dlp,  BPF_BRANCH_IMM(BPF_JEQ, BPF_REG_0, 0, pcb->pcb_exitlbl));
 
 	/*
-	 * If we succeeded, then we use copyout_val() above to assign:
+	 * If we succeeded, then we assign:
 	 *	    *arg0 = 1;
 	 */
-	copyout_val(pcb, DT_LBL_NONE, 1, 0);  // FIXME: This is the only copyout_val() call site... should we just inline it here?
+	emit(dlp, BPF_STORE_IMM(BPF_DW, BPF_REG_FP, DT_TRAMP_SP_SLOT(0), 1));
+	emit(dlp, BPF_LOAD(BPF_DW, BPF_REG_1, BPF_REG_7, DMST_ARG(0)));  // FIXME: Why don't we just copy from %r8+?
+	emit(dlp, BPF_MOV_REG(BPF_REG_2, BPF_REG_FP));
+	emit(dlp, BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, DT_TRAMP_SP_SLOT(0)));
+	emit(dlp, BPF_MOV_IMM(BPF_REG_3, sizeof(uint32_t)));
+	emit(dlp, BPF_CALL_HELPER(BPF_FUNC_probe_write_user));
 
 	dt_cg_tramp_return(pcb);
 
-- 
2.18.4

[PATCH 37/38] Fix some dctx->mst->specsize comments

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_cg.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libdtrace/dt_cg.c b/libdtrace/dt_cg.c
index d27a8cb2..1548d58f 100644
--- a/libdtrace/dt_cg.c
+++ b/libdtrace/dt_cg.c
@@ -1739,7 +1739,7 @@ ok:
 
 		/*
 		 * if (*((uint32_t *)&buf[DBUF_SPECID]) != 0) {
-		 *     if (dctx->dmst->specsize + off + size >
+		 *     if (dctx->mst->specsize + off + size >
 		 *	   dtp->dt_options[DTRACEOPT_SPECSIZE]) {
 		 *	   state[DT_STATE_SPEC_DROPS]++;
 		 *
@@ -2461,7 +2461,7 @@ dt_cg_act_setopt(dt_pcb_t *pcb, dt_node_t *dnp, dtrace_actkind_t kind)
  * back pending a commit() or discard() for the speculation with the given id.
  *
  * Updates the specid in the output buffer header, rather than emitting a new
- * record into it.  The dctx->dmst->specsize value is initialized with the size
+ * record into it.  The dctx->mst->specsize value is initialized with the size
  * of the data thus far recorded for this speculation.
  */
 static void
@@ -2491,7 +2491,7 @@ dt_cg_act_speculate(dt_pcb_t *pcb, dt_node_t *dnp, dtrace_actkind_t kind)
 	 *		goto exit;
 	 *	*((uint32_t *)&buf[DBUF_SPECID]) = specid;
 	 *				// mov [%r9 + DBUF_SPECID], %dn_reg
-	 *	dctx->dmst->specsize = spec->size;
+	 *	dctx->mst->specsize = spec->size;
 	 *	exit:			// nop
 	 */
 
-- 
2.18.4

[PATCH 38/38] Systemwide USDT WIP

[eugene.loh]

From: Eugene Loh <eugene.loh@oracle.com>

need to:
- review tests
- handle globs in dt_pid_create_probes_module()
- add process monitoring / inotify
- deal with FIXMEs

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 libdtrace/dt_cc.c                             |   3 +-
 libdtrace/dt_pid.c                            | 133 +++++++++++++++---
 .../dtrace-util/tst.ListProbesNameUSDT.sh     |   1 -
 .../dtrace-util/tst.ListProbesProviderUSDT.sh |   1 -
 test/unittest/usdt/tst.forker.sh              |   1 -
 5 files changed, 119 insertions(+), 20 deletions(-)

diff --git a/libdtrace/dt_cc.c b/libdtrace/dt_cc.c
index d09a9775..3f316775 100644
--- a/libdtrace/dt_cc.c
+++ b/libdtrace/dt_cc.c
@@ -278,7 +278,8 @@ dt_setcontext(dtrace_hdl_t *dtp, const dtrace_probedesc_t *pdp)
 	 * and tag -- we just have to longjmp() out of here.
 	 */
 	if (pdp->prv && pdp->prv[0] &&
-	    isdigit(pdp->prv[strlen(pdp->prv) - 1]) &&
+	    (isdigit(pdp->prv[strlen(pdp->prv) - 1]) ||
+	             pdp->prv[strlen(pdp->prv) - 1] == '*') &&
 	    ((pvp = dt_provider_lookup(dtp, pdp->prv)) == NULL ||
 	     pvp->pv_flags & DT_PROVIDER_PID) &&
 	    dt_pid_create_probes((dtrace_probedesc_t *)pdp, dtp, yypcb) != 0) {
diff --git a/libdtrace/dt_pid.c b/libdtrace/dt_pid.c
index 996543b1..079377ec 100644
--- a/libdtrace/dt_pid.c
+++ b/libdtrace/dt_pid.c
@@ -605,6 +605,9 @@ dt_pid_fix_mod(dt_pid_probe_t *pp, dtrace_probedesc_t *pdp, dtrace_hdl_t *dtp,
 	return pmp;
 }
 
+/*
+ * Create pid probes for the specified process.
+ */
 static int
 dt_pid_create_pid_probes(dtrace_probedesc_t *pdp, dtrace_hdl_t *dtp,
 			 dt_pcb_t *pcb, dt_proc_t *dpr)
@@ -702,6 +705,7 @@ dt_pid_create_pid_probes(dtrace_probedesc_t *pdp, dtrace_hdl_t *dtp,
 
 	return ret;
 }
+
 /*
  * Read a file into a buffer and return it.
  */
@@ -780,8 +784,8 @@ validate_dof_record(const char *path, const dof_parsed_t *parsed,
 /*
  * Create underlying probes relating to the probespec passed on input.
  *
- * If dpr is set, just set up probes relating to mappings found in that one
- * process.  (dpr must in this case be locked.)
+ * dpr must be set and locked.  Just set up probes relating to mappings found
+ * in this one process.
  *
  * Return 0 on success or -1 on error.  (Failure to create specific underlying
  * probes is not an error.)
@@ -795,9 +799,6 @@ dt_pid_create_usdt_probes(dtrace_hdl_t *dtp, dt_proc_t *dpr, dtrace_probedesc_t
 	char *probepath = NULL;
 	glob_t probeglob = {0};
 
-	/*
-	 * Systemwide probing: not yet implemented.
-	 */
 	assert(dpr != NULL && dpr->dpr_proc);
 	assert(MUTEX_HELD(&dpr->dpr_lock));
 
@@ -1094,22 +1095,99 @@ dt_pid_get_pid(const dtrace_probedesc_t *pdp, dtrace_hdl_t *dtp, dt_pcb_t *pcb,
 	return pid;
 }
 
+static const char *
+convert_pidglob(const char *s)
+{
+	/*
+	 * Convert a glob for "pid[1-9][0-9]*" into one for just the numerical
+	 * portion.  It will always be a subset of the input string.  A NULL
+	 * pointer is returned if there is no such string -- e.g., if the input
+	 * string is "foo1234".
+	 *
+	 * There is no need to check the entire string for legality.  E.g., if
+	 * the input string is "pid*q*", we can simply return "*q*".  Then, this
+	 * string will not match any "[1-9][0-9]*";  we do not need to intervene.
+	 */
+	const char	*p;
+	int		nchars = 0;	/* number of chars of "pid" we have seen so far */
+
+	for (p = s; ; p++) {
+		switch (*p) {
+		case 'p':
+			if (nchars > 0)
+				return NULL;
+			nchars = 1;
+			break;
+		case 'i':
+			if (nchars > 1)
+				return NULL;
+			if (nchars < 1 && p[-1] != '*')
+				return NULL;
+			nchars = 2;
+			break;
+		case 'd':
+			if (nchars > 2)
+				return NULL;
+			if (nchars < 2 && p[-1] != '*')
+				return NULL;
+			nchars = 3;
+			break;
+		case '*':
+			break;
+		case '\0':
+			if (p == s)
+				return NULL;
+			if (p[-1] != '*')
+				return p;
+			return p - 1;
+		default:
+			if (*p < '0' || *p > '9')
+				return NULL;
+			if (p == s)
+				return NULL;
+			if (p[-1] != '*')
+				return p;
+			return p - 1;
+		}
+	}
+}
+
 int
 dt_pid_create_probes(dtrace_probedesc_t *pdp, dtrace_hdl_t *dtp, dt_pcb_t *pcb)
 {
-	char provname[DTRACE_PROVNAMELEN];
 	dt_proc_t *dpr;
 	pid_t pid;
-	int err = 0;
+	int err = 0, i, nmatches = 0;
+	glob_t globbuf;
+	char *globpat;
 
 	assert(pcb != NULL);
 
-	if ((pid = dt_pid_get_pid(pdp, dtp, pcb, NULL)) == -1)
+	/* Exclude pid0 from being specifically requested. */
+	if (strcmp(pdp->prv, "pid0") == 0) {
+		dt_pid_error(dtp, pcb, NULL, D_PROC_BADPID,
+			     "pid0 does not contain a valid pid");
 		return -1;
+	}
+
+	/*
+	 * Try pid probes.
+	 */
+	asprintf(&globpat, "/proc/%s/map_files", convert_pidglob(pdp->prv));
+	nmatches = glob(globpat, 0, NULL, &globbuf) ? 0 : globbuf.gl_pathc;
+	for (i = 0; i < nmatches; i++) {
+		if (strncmp(globbuf.gl_pathv[i], "/proc/self", 10) == 0)
+			continue;
+
+		pid = strtol(globbuf.gl_pathv[i] + strlen("/proc/"), NULL, 10);
 
-	snprintf(provname, sizeof(provname), "pid%d", (int)pid);
+#if 1
+		// FIXME: omit this check once we have confidence in convert_pidglob()
+		char provname[DTRACE_PROVNAMELEN];
+		snprintf(provname, sizeof(provname), "pid%d", (int)pid);
+		assert(gmatch(provname, pdp->prv) != 0);
+#endif
 
-	if (gmatch(provname, pdp->prv) != 0) {
 		if (dt_proc_grab_lock(dtp, pid, DTRACE_PROC_WAITING) < 0) {
 			dt_pid_error(dtp, pcb, NULL, D_PROC_GRAB,
 			    "failed to grab process %d", (int)pid);
@@ -1119,14 +1197,36 @@ dt_pid_create_probes(dtrace_probedesc_t *pdp, dtrace_hdl_t *dtp, dt_pcb_t *pcb)
 		dpr = dt_proc_lookup(dtp, pid);
 		assert(dpr != NULL);
 
+		// FIXME How should err be handled?
 		err = dt_pid_create_pid_probes(pdp, dtp, pcb, dpr);
 		dt_proc_release_unlock(dtp, pid);
 	}
+	free(globpat);
+	globfree(&globbuf);
 
 	/*
-	 * If it's not strictly a pid provider, we might match a USDT provider.
+	 * Try USDT probes.
 	 */
-	if (strcmp(provname, pdp->prv) != 0) {
+	asprintf(&globpat, "%s/probes/*/%s", dtp->dt_dofstash_path, pdp->prv);
+	nmatches = glob(globpat, 0, NULL, &globbuf) ? 0 : globbuf.gl_pathc;
+	for (i = 0; i < nmatches; i++) {
+		char *s = globbuf.gl_pathv[i]
+			  + strlen(dtp->dt_dofstash_path)
+			  + strlen("/probes/");
+		dtrace_probedesc_t pdptmp;
+
+		pdptmp.prv = strchr(s, '/');
+		pdptmp.mod = pdp->mod[0] == '\0' ? "*" : pdp->mod;
+		pdptmp.fun = pdp->fun[0] == '\0' ? "*" : pdp->fun;
+		pdptmp.prb = pdp->prb[0] == '\0' ? "*" : pdp->prb;
+
+		pid = atoll(s);
+
+		// Check, since dtprobed takes a while to clean up dead processes.
+		if (!Pexists(pid))
+			continue;
+// FIXME should also check dof_version, though that doesn't do much yet
+
 		if (dt_proc_grab_lock(dtp, pid, DTRACE_PROC_WAITING |
 				      DTRACE_PROC_SHORTLIVED) < 0) {
 			dt_pid_error(dtp, pcb, NULL, D_PROC_GRAB,
@@ -1137,17 +1237,18 @@ dt_pid_create_probes(dtrace_probedesc_t *pdp, dtrace_hdl_t *dtp, dt_pcb_t *pcb)
 		dpr = dt_proc_lookup(dtp, pid);
 		assert(dpr != NULL);
 
-		err = dt_pid_create_usdt_probes(dtp, dpr, pdp, pcb);
+		// FIXME: How should err be handled?
+		err = dt_pid_create_usdt_probes(dtp, dpr, &pdptmp, pcb);
 
 		/*
 		 * Put the module name in its canonical form.
 		 */
-		dt_pid_fix_mod(NULL, pdp, dtp, dpr->dpr_pid);
+		dt_pid_fix_mod(NULL, &pdptmp, dtp, dpr->dpr_pid);
 
 		dt_proc_release_unlock(dtp, pid);
 	}
-
-	/* (USDT systemwide probing goes here.)  */
+	free(globpat);
+	globfree(&globbuf);
 
 	return err ? -1 : 0;
 }
diff --git a/test/unittest/dtrace-util/tst.ListProbesNameUSDT.sh b/test/unittest/dtrace-util/tst.ListProbesNameUSDT.sh
index c5dfc72b..9d8f06e9 100755
--- a/test/unittest/dtrace-util/tst.ListProbesNameUSDT.sh
+++ b/test/unittest/dtrace-util/tst.ListProbesNameUSDT.sh
@@ -5,7 +5,6 @@
 # Licensed under the Universal Permissive License v 1.0 as shown at
 # http://oss.oracle.com/licenses/upl.
 #
-# @@xfail: dtv2
 
 ##
 #
diff --git a/test/unittest/dtrace-util/tst.ListProbesProviderUSDT.sh b/test/unittest/dtrace-util/tst.ListProbesProviderUSDT.sh
index 644da2ac..6eae82ed 100755
--- a/test/unittest/dtrace-util/tst.ListProbesProviderUSDT.sh
+++ b/test/unittest/dtrace-util/tst.ListProbesProviderUSDT.sh
@@ -5,7 +5,6 @@
 # Licensed under the Universal Permissive License v 1.0 as shown at
 # http://oss.oracle.com/licenses/upl.
 #
-# @@xfail: dtv2
 
 ##
 #
diff --git a/test/unittest/usdt/tst.forker.sh b/test/unittest/usdt/tst.forker.sh
index 7cfa9eb5..92513eb5 100755
--- a/test/unittest/usdt/tst.forker.sh
+++ b/test/unittest/usdt/tst.forker.sh
@@ -7,7 +7,6 @@
 #
 #
 # @@timeout: 120
-# @@xfail: dtv2 USDT wildcard
 
 if [ $# != 1 ]; then
 	echo expected one argument: '<'dtrace-path'>'
-- 
2.18.4

Re: [PATCH 37/38] Fix some dctx->mst->specsize comments

[Kris Van Hees]

On Thu, Jun 27, 2024 at 01:39:03AM -0400, eugene.loh@oracle.com wrote:
> From: Eugene Loh <eugene.loh@oracle.com>
> 
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Kris Van Hees <kris.van.hees@oracle.com>

> ---
>  libdtrace/dt_cg.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/libdtrace/dt_cg.c b/libdtrace/dt_cg.c
> index d27a8cb2..1548d58f 100644
> --- a/libdtrace/dt_cg.c
> +++ b/libdtrace/dt_cg.c
> @@ -1739,7 +1739,7 @@ ok:
>  
>  		/*
>  		 * if (*((uint32_t *)&buf[DBUF_SPECID]) != 0) {
> -		 *     if (dctx->dmst->specsize + off + size >
> +		 *     if (dctx->mst->specsize + off + size >
>  		 *	   dtp->dt_options[DTRACEOPT_SPECSIZE]) {
>  		 *	   state[DT_STATE_SPEC_DROPS]++;
>  		 *
> @@ -2461,7 +2461,7 @@ dt_cg_act_setopt(dt_pcb_t *pcb, dt_node_t *dnp, dtrace_actkind_t kind)
>   * back pending a commit() or discard() for the speculation with the given id.
>   *
>   * Updates the specid in the output buffer header, rather than emitting a new
> - * record into it.  The dctx->dmst->specsize value is initialized with the size
> + * record into it.  The dctx->mst->specsize value is initialized with the size
>   * of the data thus far recorded for this speculation.
>   */
>  static void
> @@ -2491,7 +2491,7 @@ dt_cg_act_speculate(dt_pcb_t *pcb, dt_node_t *dnp, dtrace_actkind_t kind)
>  	 *		goto exit;
>  	 *	*((uint32_t *)&buf[DBUF_SPECID]) = specid;
>  	 *				// mov [%r9 + DBUF_SPECID], %dn_reg
> -	 *	dctx->dmst->specsize = spec->size;
> +	 *	dctx->mst->specsize = spec->size;
>  	 *	exit:			// nop
>  	 */
>  
> -- 
> 2.18.4
> 

Re: [PATCH 20/38] Add a hook for a provider-specific "update" function

[Kris Van Hees]

This patch surely should be part of another one that actually provides some
code to use the new callback.  The implementation of the callback in any
particular provider could remain in its own patch after that if you like,
but adding this without the code that actually calls it seems less useful.
If anything, the commit message itself makes it so generic sounding that one
is likely to have no idea what this is meant to be used for, where it would
be called from, why, and what the expected result ought to be.

So, combine with the code that uses it please.  Which I will review next :)

On Thu, Jun 27, 2024 at 01:38:46AM -0400, eugene.loh@oracle.com wrote:
> From: Eugene Loh <eugene.loh@oracle.com>
> 
> For up-coming USDT-probe support, we need to update a BPF map
> -- at least when the dtrace session starts but possibly also later
> to support systemwide USDT tracing for processes that may start up
> later.
> 
> One way to do this is with a USDT-specific update function.
> 
> For now, let's add a hook for providers to have provider-specific
> update functions.  User space can either call
> 
>     for (i = 0; i < ARRAY_SIZE(dt_providers); i++) {
>         if (dt_providers[i]->update)
>             dt_providers[i]->update(...);
>     }
> 
> any time it likes.  Or it can call dt_usdt.update(...).
> 
> This is for WIP.  A different approach can be adopted later instead.
> 
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> ---
>  libdtrace/dt_provider.h | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/libdtrace/dt_provider.h b/libdtrace/dt_provider.h
> index b1b1b1b8..71b5a3c4 100644
> --- a/libdtrace/dt_provider.h
> +++ b/libdtrace/dt_provider.h
> @@ -71,6 +71,8 @@ typedef struct dt_provimpl {
>  			      void *datap);
>  	void (*destroy)(dtrace_hdl_t *dtp,	/* free provider data */
>  			      void *datap);
> +	void (*update)(dtrace_hdl_t *dtp,	/* update provider-specific info */
> +			      void *datap);
>  } dt_provimpl_t;
>  
>  /* list dt_dtrace first */
> -- 
> 2.18.4
> 

Re: [PATCH 21/38] Add some comments

[Kris Van Hees]

On Thu, Jun 27, 2024 at 01:38:47AM -0400, eugene.loh@oracle.com wrote:
> From: Eugene Loh <eugene.loh@oracle.com>
> 
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Kris Van Hees <kris.van.hees@oracle.com>

... with tiny suggestion below

> ---
>  libdtrace/dt_prov_uprobe.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/libdtrace/dt_prov_uprobe.c b/libdtrace/dt_prov_uprobe.c
> index 5dbd75e3..5f0c56db 100644
> --- a/libdtrace/dt_prov_uprobe.c
> +++ b/libdtrace/dt_prov_uprobe.c
> @@ -295,6 +295,7 @@ static int provide_probe(dtrace_hdl_t *dtp, const pid_probespec_t *psp,
>  	/* Mark the provider as a PID-based provider. */
>  	pvp->pv_flags |= DT_PROVIDER_PID;
>  
> +	/* Look up or create the underlying probe. */

This seems to contradict the function name.  Perhaps rephrase this to say:
"Create and/or lookup the underlying probe."

>  	uprp = create_underlying(dtp, psp);
>  	if (uprp == NULL)
>  		return -1;
> @@ -302,6 +303,7 @@ static int provide_probe(dtrace_hdl_t *dtp, const pid_probespec_t *psp,
>  	upp = uprp->prv_data;
>  	upp->flags |= flags;
>  
> +	/* Look up the overlying probe. */
>  	prp = dt_probe_lookup(dtp, &pd);
>  	if (prp != NULL) {
>  		/*
> -- 
> 2.18.4
> 

Re: [PATCH 22/38] Fix aggs comment in dt_cg_tramp_prologue_act()

[Kris Van Hees]

On Thu, Jun 27, 2024 at 01:38:48AM -0400, eugene.loh@oracle.com wrote:
> From: Eugene Loh <eugene.loh@oracle.com>
> 
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Kris Van Hees <kris.van.hees@oracle.com>

... provided you mention in the commit msg what you are fixing.  Because
looking at the patch you are removing a big part of a comment block but
without any indication why, and the patch does not provide context to be
able to assess that.

> ---
>  libdtrace/dt_cg.c | 11 -----------
>  1 file changed, 11 deletions(-)
> 
> diff --git a/libdtrace/dt_cg.c b/libdtrace/dt_cg.c
> index 4fd2d359..2fb2d0d8 100644
> --- a/libdtrace/dt_cg.c
> +++ b/libdtrace/dt_cg.c
> @@ -372,17 +372,6 @@ dt_cg_tramp_prologue_act(dt_pcb_t *pcb, dt_activity_t act)
>  	 *	if (rc == 0)		// jeq %r0, 0, lbl_exit
>  	 *		goto exit;
>  	 *
> -	 *	key = 0;		// stdw [%r9 + DCTX_AGG], 0
> -	 *	rc = bpf_map_lookup_elem(rc, &key);
> -	 *				// mov %r1, %r0
> -	 *				// mov %r2, %r9
> -	 *				// add %r2, DCTX_AGG
> -	 *				// call bpf_map_lookup_elem
> -	 *				//     (%r1 ... %r5 clobbered)
> -	 *				//     (%r0 = aggs[cpuid] BPF map value)
> -	 *	if (rc == 0)		// jeq %r0, 0, lbl_exit
> -	 *		goto exit;
> -	 *
>  	 *	dctx.aggs = rc;		// stdw [%r9 + DCTX_AGG], %r0
>  	 */
>  	if (dtp->dt_maxaggdsize > 0) {
> -- 
> 2.18.4
> 

Re: [PATCH 24/38] test: Make test independent of specific PC

[Kris Van Hees]

On Thu, Jun 27, 2024 at 01:38:50AM -0400, eugene.loh@oracle.com wrote:
> From: Eugene Loh <eugene.loh@oracle.com>

I agree in principle that this test should not depend on a specific PC value,
but that leaves the issue that with this patch, I do not think there is any
real test to verify that the PC value is sane.  So, that probably should be
added.

> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> ---
>  test/unittest/error/tst.DTRACEFLT_UNKNOWN.d | 6 +++---
>  test/unittest/error/tst.DTRACEFLT_UNKNOWN.r | 2 +-
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
> index 001903ff..bfc77bf5 100644
> --- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
> +++ b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
> @@ -1,6 +1,6 @@
>  /*
>   * Oracle Linux DTrace.
> - * Copyright (c) 2006, 2020, Oracle and/or its affiliates. All rights reserved.
> + * Copyright (c) 2006, 2024, Oracle and/or its affiliates. All rights reserved.
>   * Licensed under the Universal Permissive License v 1.0 as shown at
>   * http://oss.oracle.com/licenses/upl.
>   */
> @@ -19,8 +19,8 @@
>  
>  ERROR
>  {
> -	printf("The arguments are %u %u %u %u %u\n",
> -		arg1, arg2, arg3, arg4, arg5);
> +	printf("The arguments are %u %u PC %u %u\n",
> +		arg1, arg2, arg4, arg5);
>  	printf("The value of arg4 = %u\n", DTRACEFLT_UNKNOWN);
>  	exit(0);
>  }
> diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
> index b11f6c99..3e7caac4 100644
> --- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
> +++ b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
> @@ -1,4 +1,4 @@
> -The arguments are 2 2 4 1 64
> +The arguments are 2 2 PC 1 64
>  The value of arg4 = 0
>  
>  -- @@stderr --
> -- 
> 2.18.4
> 

Re: [PATCH 25/38] test: Clean up tests still expecting obsolete "at DIF offset NN"

[Kris Van Hees]

On Thu, Jun 27, 2024 at 01:38:51AM -0400, eugene.loh@oracle.com wrote:
> From: Eugene Loh <eugene.loh@oracle.com>
> 
> Also, the numbering of EPIDs has changed.

Good catch on the DIF offset stuff that certainly should have been gone a long
time ago.

> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Kris Van Hees <kris.van.hees@oracle.com>

> ---
>  test/demo/dtrace/error.d                                | 1 -
>  test/demo/dtrace/error.r                                | 2 +-
>  test/unittest/assocs/tst.invalidref.r                   | 4 ++--
>  test/unittest/drops/drp.DTRACEDROP_DBLERROR.r           | 2 +-
>  test/unittest/error/tst.DTRACEFLT_UNKNOWN.d             | 1 -
>  test/unittest/error/tst.DTRACEFLT_UNKNOWN.r             | 4 ++--
>  test/unittest/error/tst.DTRACEFLT_UNKNOWN.sparc64.r     | 5 -----
>  test/unittest/pointers/err.BadAlign.d                   | 1 -
>  test/unittest/pointers/err.BadAlign.r                   | 2 +-
>  test/unittest/pointers/err.InvalidAddress2.d            | 1 -
>  test/unittest/pointers/err.InvalidAddress2.r            | 2 +-
>  test/unittest/pointers/err.InvalidAddress3.r            | 2 +-
>  test/unittest/pointers/err.InvalidAddress4.d            | 1 -
>  test/unittest/pointers/err.InvalidAddress4.r            | 2 +-
>  test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.d | 2 +-
>  test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.r | 4 ++--
>  16 files changed, 13 insertions(+), 23 deletions(-)
>  delete mode 100644 test/unittest/error/tst.DTRACEFLT_UNKNOWN.sparc64.r
> 
> diff --git a/test/demo/dtrace/error.d b/test/demo/dtrace/error.d
> index 5700dd33..d55fb090 100644
> --- a/test/demo/dtrace/error.d
> +++ b/test/demo/dtrace/error.d
> @@ -5,7 +5,6 @@
>   * http://oss.oracle.com/licenses/upl.
>   */
>  
> -/* @@xfail: dtv2 */
>  /* @@trigger: none */
>  
>  BEGIN
> diff --git a/test/demo/dtrace/error.r b/test/demo/dtrace/error.r
> index d894776b..d3904f47 100644
> --- a/test/demo/dtrace/error.r
> +++ b/test/demo/dtrace/error.r
> @@ -3,4 +3,4 @@
>  
>  -- @@stderr --
>  dtrace: script 'test/demo/dtrace/error.d' matched 2 probes
> -dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at DIF offset 16 at BPF pc NNN
> +dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
> diff --git a/test/unittest/assocs/tst.invalidref.r b/test/unittest/assocs/tst.invalidref.r
> index b050e436..20f131b4 100644
> --- a/test/unittest/assocs/tst.invalidref.r
> +++ b/test/unittest/assocs/tst.invalidref.r
> @@ -1,4 +1,4 @@
>  
>  -- @@stderr --
> -dtrace: error on enabled probe ID (ID: profile:::tick-1s): invalid address ({ptr}) in action #2 at DIF offset 64 at BPF pc NNN
> -dtrace: error on enabled probe ID (ID: profile:::tick-1s): invalid address ({ptr}) in action #2 at DIF offset 64 at BPF pc NNN
> +dtrace: error on enabled probe ID (ID: profile:::tick-1s): invalid address ({ptr}) in action #2 at BPF pc NNN
> +dtrace: error on enabled probe ID (ID: profile:::tick-1s): invalid address ({ptr}) in action #2 at BPF pc NNN
> diff --git a/test/unittest/drops/drp.DTRACEDROP_DBLERROR.r b/test/unittest/drops/drp.DTRACEDROP_DBLERROR.r
> index 14654676..9fa54dd9 100644
> --- a/test/unittest/drops/drp.DTRACEDROP_DBLERROR.r
> +++ b/test/unittest/drops/drp.DTRACEDROP_DBLERROR.r
> @@ -4,4 +4,4 @@
>  -- @@stderr --
>  dtrace: script 'test/unittest/drops/drp.DTRACEDROP_DBLERROR.d' matched 3 probes
>  dtrace: [DTRACEDROP_DBLERROR] 1 error in ERROR probe enabling
> -dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at DIF offset 16 at BPF pc NNN
> +dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
> diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
> index bfc77bf5..c74762ae 100644
> --- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
> +++ b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
> @@ -4,7 +4,6 @@
>   * Licensed under the Universal Permissive License v 1.0 as shown at
>   * http://oss.oracle.com/licenses/upl.
>   */
> -/* @@xfail: dtv2 */
>  
>  /*
>   * ASSERTION:
> diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
> index 3e7caac4..1e4fdd64 100644
> --- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
> +++ b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
> @@ -1,5 +1,5 @@
> -The arguments are 2 2 PC 1 64
> +The arguments are 3 1 PC 1 64
>  The value of arg4 = 0
>  
>  -- @@stderr --
> -dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #2 at DIF offset 4 at BPF pc NNN
> +dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
> diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.sparc64.r b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.sparc64.r
> deleted file mode 100644
> index 3944c138..00000000
> --- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.sparc64.r
> +++ /dev/null
> @@ -1,5 +0,0 @@
> -The arguments are 2 2 4 1 0
> -The value of arg4 = 0
> -
> --- @@stderr --
> -dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #2 at DIF offset 4
> diff --git a/test/unittest/pointers/err.BadAlign.d b/test/unittest/pointers/err.BadAlign.d
> index cd4138ae..e859dd75 100644
> --- a/test/unittest/pointers/err.BadAlign.d
> +++ b/test/unittest/pointers/err.BadAlign.d
> @@ -4,7 +4,6 @@
>   * Licensed under the Universal Permissive License v 1.0 as shown at
>   * http://oss.oracle.com/licenses/upl.
>   */
> -/* @@xfail: dtv2 */
>  
>  /*
>   * ASSERTION: This test reproduces the alignment error.
> diff --git a/test/unittest/pointers/err.BadAlign.r b/test/unittest/pointers/err.BadAlign.r
> index 4328aac4..187543b6 100644
> --- a/test/unittest/pointers/err.BadAlign.r
> +++ b/test/unittest/pointers/err.BadAlign.r
> @@ -1,3 +1,3 @@
>  
>  -- @@stderr --
> -dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #2 at DIF offset 4
> +dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
> diff --git a/test/unittest/pointers/err.InvalidAddress2.d b/test/unittest/pointers/err.InvalidAddress2.d
> index 682ad650..b22f08fb 100644
> --- a/test/unittest/pointers/err.InvalidAddress2.d
> +++ b/test/unittest/pointers/err.InvalidAddress2.d
> @@ -4,7 +4,6 @@
>   * Licensed under the Universal Permissive License v 1.0 as shown at
>   * http://oss.oracle.com/licenses/upl.
>   */
> -/* @@xfail: dtv2 */
>  
>  /*
>   * ASSERTION: D pointers do not allow invalid pointer accesses.
> diff --git a/test/unittest/pointers/err.InvalidAddress2.r b/test/unittest/pointers/err.InvalidAddress2.r
> index d866eae1..187543b6 100644
> --- a/test/unittest/pointers/err.InvalidAddress2.r
> +++ b/test/unittest/pointers/err.InvalidAddress2.r
> @@ -1,3 +1,3 @@
>  
>  -- @@stderr --
> -dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #4 at DIF offset 8
> +dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
> diff --git a/test/unittest/pointers/err.InvalidAddress3.r b/test/unittest/pointers/err.InvalidAddress3.r
> index 069ee1de..187543b6 100644
> --- a/test/unittest/pointers/err.InvalidAddress3.r
> +++ b/test/unittest/pointers/err.InvalidAddress3.r
> @@ -1,3 +1,3 @@
>  
>  -- @@stderr --
> -dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #2 at DIF offset 8
> +dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
> diff --git a/test/unittest/pointers/err.InvalidAddress4.d b/test/unittest/pointers/err.InvalidAddress4.d
> index 1e2b4f62..586cddf9 100644
> --- a/test/unittest/pointers/err.InvalidAddress4.d
> +++ b/test/unittest/pointers/err.InvalidAddress4.d
> @@ -4,7 +4,6 @@
>   * Licensed under the Universal Permissive License v 1.0 as shown at
>   * http://oss.oracle.com/licenses/upl.
>   */
> -/* @@xfail: dtv2 */
>  
>  /*
>   * ASSERTION: Demonstrating valid memory access.
> diff --git a/test/unittest/pointers/err.InvalidAddress4.r b/test/unittest/pointers/err.InvalidAddress4.r
> index d866eae1..187543b6 100644
> --- a/test/unittest/pointers/err.InvalidAddress4.r
> +++ b/test/unittest/pointers/err.InvalidAddress4.r
> @@ -1,3 +1,3 @@
>  
>  -- @@stderr --
> -dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #4 at DIF offset 8
> +dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
> diff --git a/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.d b/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.d
> index c23f9503..13032f77 100644
> --- a/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.d
> +++ b/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.d
> @@ -4,7 +4,7 @@
>   * Licensed under the Universal Permissive License v 1.0 as shown at
>   * http://oss.oracle.com/licenses/upl.
>   */
> -/* @@xfail: dtv2 */
> +/* @@xfail: dtv2 d_path */
>  
>  /*
>   * ASSERTION:
> diff --git a/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.r b/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.r
> index 8c601a43..54ffb47f 100644
> --- a/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.r
> +++ b/test/unittest/regression/tst.DTRACEFLT_BADADDR.d_path.r
> @@ -1,6 +1,6 @@
> -The arguments are 2 1 28 1 24
> +The arguments are 3 1 28 1 24
>  The value of arg4 should be 1
>  The value of arg5 should be 24
>  
>  -- @@stderr --
> -dtrace: error on enabled probe ID 2 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at DIF offset 28 at BPF pc NNN
> +dtrace: error on enabled probe ID 3 (ID 1: dtrace:::BEGIN): invalid address ({ptr}) in action #1 at BPF pc NNN
> -- 
> 2.18.4
> 

Re: [PATCH 26/38] test: Annotate xfail (chill not implemented yet)

[Kris Van Hees]

On Thu, Jun 27, 2024 at 01:38:52AM -0400, eugene.loh@oracle.com wrote:
> From: Eugene Loh <eugene.loh@oracle.com>

Before I can really assess whether the annotation is valid, I'd like to know
why chill) is even used here.  I.e. I don't quite understand the test :)
> 
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> ---
>  test/unittest/speculation/tst.zerosize.d | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/test/unittest/speculation/tst.zerosize.d b/test/unittest/speculation/tst.zerosize.d
> index 56c1fcea..996f1257 100644
> --- a/test/unittest/speculation/tst.zerosize.d
> +++ b/test/unittest/speculation/tst.zerosize.d
> @@ -5,7 +5,7 @@
>   * http://oss.oracle.com/licenses/upl.
>   */
>  
> -/* @@xfail: dtv2 */
> +/* @@xfail: dtv2, chill not implemented yet */
>  
>  #pragma D option destructive
>  
> -- 
> 2.18.4
> 

Re: [PATCH 30/38] Allow relocation on BPF_OR instructions

[Kris Van Hees]

On Thu, Jun 27, 2024 at 01:38:56AM -0400, eugene.loh@oracle.com wrote:
> From: Eugene Loh <eugene.loh@oracle.com>
> 
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Kris Van Hees <kris.van.hees@oracle.com>

> ---
>  libdtrace/dt_as.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/libdtrace/dt_as.c b/libdtrace/dt_as.c
> index a634b855..4b397f51 100644
> --- a/libdtrace/dt_as.c
> +++ b/libdtrace/dt_as.c
> @@ -280,6 +280,7 @@ dt_as(dt_pcb_t *pcb)
>  		case BPF_ST | BPF_MEM | BPF_DW:		/* stdw */
>  		case BPF_ALU64 | BPF_MOV | BPF_K:	/* mov */
>  		case BPF_ALU64 | BPF_ADD | BPF_K:	/* add */
> +		case BPF_ALU64 | BPF_OR  | BPF_K:	/* or */
>  			if (idp->di_flags & DT_IDFLG_BPF)
>  				brel++;
>  			else
> @@ -492,6 +493,7 @@ fail:
>  			case BPF_ST | BPF_MEM | BPF_DW:		/* stdw */
>  			case BPF_ALU64 | BPF_MOV | BPF_K:	/* mov */
>  			case BPF_ALU64 | BPF_ADD | BPF_K:	/* add */
> +			case BPF_ALU64 | BPF_OR  | BPF_K:	/* or */
>  				rp->dofr_type = R_BPF_64_32;
>  				break;
>  			case BPF_LD | BPF_IMM | BPF_DW:		/* lddw */
> -- 
> 2.18.4
> 

Re: [DTrace-devel] [PATCH 29/38] Allow relocation of the ERROR PRID

[Kris Van Hees]

Why is this patch needed?  As far as I can see, BEGIN, END, and ERROR are
always created as the first 3 probes (in that order), so they will have probe
IDs 1, 2, and 3.

On Thu, Jun 27, 2024 at 01:38:55AM -0400, eugene.loh--- via DTrace-devel wrote:
> From: Eugene Loh <eugene.loh@oracle.com>
> 
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> ---
>  libdtrace/dt_bpf.h   | 1 +
>  libdtrace/dt_cc.c    | 3 +++
>  libdtrace/dt_dlibs.c | 1 +
>  3 files changed, 5 insertions(+)
> 
> diff --git a/libdtrace/dt_bpf.h b/libdtrace/dt_bpf.h
> index 5b2df264..58cf29cc 100644
> --- a/libdtrace/dt_bpf.h
> +++ b/libdtrace/dt_bpf.h
> @@ -56,6 +56,7 @@ extern "C" {
>  #define DT_CONST_ZERO_OFF		22
>  #define DT_CONST_STACK_OFF		23
>  #define DT_CONST_STACK_SKIP		24
> +#define DT_CONST_ERROR_PRID		25
>  
>  #define DT_BPF_LOG_SIZE_DEFAULT	(UINT32_MAX >> 8)
>  #define DT_BPF_LOG_SIZE_SMALL	4096
> diff --git a/libdtrace/dt_cc.c b/libdtrace/dt_cc.c
> index d1ee3843..3d9a9c79 100644
> --- a/libdtrace/dt_cc.c
> +++ b/libdtrace/dt_cc.c
> @@ -1076,6 +1076,9 @@ dt_link_construct(dtrace_hdl_t *dtp, const dt_probe_t *prp, dtrace_difo_t *dp,
>  				nrp->dofr_data = sizeof(uint64_t)
>  				    * dtp->dt_options[DTRACEOPT_MAXFRAMES];
>  				continue;
> +			case DT_CONST_ERROR_PRID:
> +				nrp->dofr_data = dtp->dt_error->desc->id;
> +				continue;
>  			case DT_CONST_BOOTTM:
>  				if (boottime == 0 && get_boottime())
>  					return -1;
> diff --git a/libdtrace/dt_dlibs.c b/libdtrace/dt_dlibs.c
> index bc883e11..1fb561ad 100644
> --- a/libdtrace/dt_dlibs.c
> +++ b/libdtrace/dt_dlibs.c
> @@ -80,6 +80,7 @@ static const dt_ident_t		dt_bpf_symbols[] = {
>  	DT_BPF_SYMBOL_ID(STBSZ, DT_IDENT_SCALAR, DT_CONST_STBSZ),
>  	DT_BPF_SYMBOL_ID(STRSZ, DT_IDENT_SCALAR, DT_CONST_STRSZ),
>  	DT_BPF_SYMBOL_ID(STKSIZ, DT_IDENT_SCALAR, DT_CONST_STKSIZ),
> +	DT_BPF_SYMBOL_ID(ERROR_PRID, DT_IDENT_SCALAR, DT_CONST_ERROR_PRID),
>  	DT_BPF_SYMBOL_ID(BOOTTM, DT_IDENT_SCALAR, DT_CONST_BOOTTM),
>  	DT_BPF_SYMBOL_ID(NSPEC, DT_IDENT_SCALAR, DT_CONST_NSPEC),
>  	DT_BPF_SYMBOL_ID(NCPUS, DT_IDENT_SCALAR, DT_CONST_NCPUS),
> -- 
> 2.18.4
> 
> 
> _______________________________________________
> DTrace-devel mailing list
> DTrace-devel@oss.oracle.com
> https://oss.oracle.com/mailman/listinfo/dtrace-devel

Re: [PATCH 22/38] Fix aggs comment in dt_cg_tramp_prologue_act()

[Eugene Loh]

On 7/19/24 16:44, Kris Van Hees wrote:

> On Thu, Jun 27, 2024 at 01:38:48AM -0400, eugene.loh@oracle.com wrote:
>> From: Eugene Loh <eugene.loh@oracle.com>
>>
>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> Reviewed-by: Kris Van Hees <kris.van.hees@oracle.com>
>
> ... provided you mention in the commit msg what you are fixing.  Because
> looking at the patch you are removing a big part of a comment block but
> without any indication why, and the patch does not provide context to be
> able to assess that.

Fair enough.  I suppose I never really knew why, other than that the 
comment didn't match the code.  Okay, I did my homework and added
         Since 2dd92c02 ("Add support for aggregation keys") removed a bunch
         of code, the corresponding comments should also have been removed.
to the commit msg, along with your R_b.

>> diff --git a/libdtrace/dt_cg.c b/libdtrace/dt_cg.c
>> @@ -372,17 +372,6 @@ dt_cg_tramp_prologue_act(dt_pcb_t *pcb, dt_activity_t act)
>>   	 *	if (rc == 0)		// jeq %r0, 0, lbl_exit
>>   	 *		goto exit;
>>   	 *
>> -	 *	key = 0;		// stdw [%r9 + DCTX_AGG], 0
>> -	 *	rc = bpf_map_lookup_elem(rc, &key);
>> -	 *				// mov %r1, %r0
>> -	 *				// mov %r2, %r9
>> -	 *				// add %r2, DCTX_AGG
>> -	 *				// call bpf_map_lookup_elem
>> -	 *				//     (%r1 ... %r5 clobbered)
>> -	 *				//     (%r0 = aggs[cpuid] BPF map value)
>> -	 *	if (rc == 0)		// jeq %r0, 0, lbl_exit
>> -	 *		goto exit;
>> -	 *
>>   	 *	dctx.aggs = rc;		// stdw [%r9 + DCTX_AGG], %r0
>>   	 */
>>   	if (dtp->dt_maxaggdsize > 0) {

Re: [PATCH 26/38] test: Annotate xfail (chill not implemented yet)

[Eugene Loh]

On 7/19/24 17:12, Kris Van Hees wrote:

> On Thu, Jun 27, 2024 at 01:38:52AM -0400, eugene.loh@oracle.com wrote:
>> From: Eugene Loh <eugene.loh@oracle.com>
> Before I can really assess whether the annotation is valid,

Well, I'm pretty sure the annotation is right.

> I'd like to know
> why chill) is even used here.  I.e. I don't quite understand the test :)

Well, I'm all with you on that.  I'm pretty sure the chill() is pretty 
irrelevant and that the test is "valid" and passes without it.  I am of 
two minds:

1)  Remove the chill() and @@xfail and move on.

2)  Be grateful for idiosyncrasies in the test suite, since they often 
turn up bugs.  Weird tests turn up weird bugs.

The patch was going with #2;  that get my vote.  I'm okay with #1, 
however.  Feel free to decide.

>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
>> ---
>>   test/unittest/speculation/tst.zerosize.d | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/test/unittest/speculation/tst.zerosize.d b/test/unittest/speculation/tst.zerosize.d
>> index 56c1fcea..996f1257 100644
>> --- a/test/unittest/speculation/tst.zerosize.d
>> +++ b/test/unittest/speculation/tst.zerosize.d
>> @@ -5,7 +5,7 @@
>>    * http://oss.oracle.com/licenses/upl.
>>    */
>>   
>> -/* @@xfail: dtv2 */
>> +/* @@xfail: dtv2, chill not implemented yet */
>>   
>>   #pragma D option destructive
>>   
>> -- 
>> 2.18.4
>>

Re: [DTrace-devel] [PATCH 29/38] Allow relocation of the ERROR PRID

[Eugene Loh]

Yeah.  This patch was superseded by a v2 (Jun 27), which acknowledges 
your point.  It's "simpler" to say "3".  It is arguably more robust to 
set the value during relocation.  I figured I'd do it the "more robust" 
way and am happy to be talked into the simpler patch.

I'll wait for your comments on the v2 patch.  If there you say the extra 
effort is not worth it, I'll post a v3 with the simpler version.

On 7/19/24 17:41, Kris Van Hees wrote:
> Why is this patch needed?  As far as I can see, BEGIN, END, and ERROR are
> always created as the first 3 probes (in that order), so they will have probe
> IDs 1, 2, and 3.
>
> On Thu, Jun 27, 2024 at 01:38:55AM -0400, eugene.loh--- via DTrace-devel wrote:
>> From: Eugene Loh <eugene.loh@oracle.com>
>>
>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
>> ---
>>   libdtrace/dt_bpf.h   | 1 +
>>   libdtrace/dt_cc.c    | 3 +++
>>   libdtrace/dt_dlibs.c | 1 +
>>   3 files changed, 5 insertions(+)
>>
>> diff --git a/libdtrace/dt_bpf.h b/libdtrace/dt_bpf.h
>> index 5b2df264..58cf29cc 100644
>> --- a/libdtrace/dt_bpf.h
>> +++ b/libdtrace/dt_bpf.h
>> @@ -56,6 +56,7 @@ extern "C" {
>>   #define DT_CONST_ZERO_OFF		22
>>   #define DT_CONST_STACK_OFF		23
>>   #define DT_CONST_STACK_SKIP		24
>> +#define DT_CONST_ERROR_PRID		25
>>   
>>   #define DT_BPF_LOG_SIZE_DEFAULT	(UINT32_MAX >> 8)
>>   #define DT_BPF_LOG_SIZE_SMALL	4096
>> diff --git a/libdtrace/dt_cc.c b/libdtrace/dt_cc.c
>> index d1ee3843..3d9a9c79 100644
>> --- a/libdtrace/dt_cc.c
>> +++ b/libdtrace/dt_cc.c
>> @@ -1076,6 +1076,9 @@ dt_link_construct(dtrace_hdl_t *dtp, const dt_probe_t *prp, dtrace_difo_t *dp,
>>   				nrp->dofr_data = sizeof(uint64_t)
>>   				    * dtp->dt_options[DTRACEOPT_MAXFRAMES];
>>   				continue;
>> +			case DT_CONST_ERROR_PRID:
>> +				nrp->dofr_data = dtp->dt_error->desc->id;
>> +				continue;
>>   			case DT_CONST_BOOTTM:
>>   				if (boottime == 0 && get_boottime())
>>   					return -1;
>> diff --git a/libdtrace/dt_dlibs.c b/libdtrace/dt_dlibs.c
>> index bc883e11..1fb561ad 100644
>> --- a/libdtrace/dt_dlibs.c
>> +++ b/libdtrace/dt_dlibs.c
>> @@ -80,6 +80,7 @@ static const dt_ident_t		dt_bpf_symbols[] = {
>>   	DT_BPF_SYMBOL_ID(STBSZ, DT_IDENT_SCALAR, DT_CONST_STBSZ),
>>   	DT_BPF_SYMBOL_ID(STRSZ, DT_IDENT_SCALAR, DT_CONST_STRSZ),
>>   	DT_BPF_SYMBOL_ID(STKSIZ, DT_IDENT_SCALAR, DT_CONST_STKSIZ),
>> +	DT_BPF_SYMBOL_ID(ERROR_PRID, DT_IDENT_SCALAR, DT_CONST_ERROR_PRID),
>>   	DT_BPF_SYMBOL_ID(BOOTTM, DT_IDENT_SCALAR, DT_CONST_BOOTTM),
>>   	DT_BPF_SYMBOL_ID(NSPEC, DT_IDENT_SCALAR, DT_CONST_NSPEC),
>>   	DT_BPF_SYMBOL_ID(NCPUS, DT_IDENT_SCALAR, DT_CONST_NCPUS),
>> -- 
>> 2.18.4
>>
>>
>> _______________________________________________
>> DTrace-devel mailing list
>> DTrace-devel@oss.oracle.com
>> https://oss.oracle.com/mailman/listinfo/dtrace-devel

Re: [PATCH 20/38] Add a hook for a provider-specific "update" function

[Eugene Loh]

The plane has barely taken off the ground and already you expect me to 
have completed the aircraft design???  :^)  (I was hoping the magic 
letters WIP in the commit message would have absolved me of 
responsibility here.)

Would you be willing to proceed with reviews of the other patches and 
then we get back to this patch?  There is a clearly provider-specific 
motivation for this function, but the style is to make these functions 
as provider-agnostic as possible.  Maybe we can defer some of the 
questions this patch raises.  Like, how often will the function be 
called, what triggers that call, will the call be USDT-specific or will 
it check all providers, etc. For now, I only have USDT-specific answers, 
but we need to guess what other considerations we expect to have to 
accommodate.

On 7/19/24 16:31, Kris Van Hees wrote:
> This patch surely should be part of another one that actually provides some
> code to use the new callback.  The implementation of the callback in any
> particular provider could remain in its own patch after that if you like,
> but adding this without the code that actually calls it seems less useful.
> If anything, the commit message itself makes it so generic sounding that one
> is likely to have no idea what this is meant to be used for, where it would
> be called from, why, and what the expected result ought to be.
>
> So, combine with the code that uses it please.  Which I will review next :)
>
> On Thu, Jun 27, 2024 at 01:38:46AM -0400, eugene.loh@oracle.com wrote:
>> From: Eugene Loh <eugene.loh@oracle.com>
>>
>> For up-coming USDT-probe support, we need to update a BPF map
>> -- at least when the dtrace session starts but possibly also later
>> to support systemwide USDT tracing for processes that may start up
>> later.
>>
>> One way to do this is with a USDT-specific update function.
>>
>> For now, let's add a hook for providers to have provider-specific
>> update functions.  User space can either call
>>
>>      for (i = 0; i < ARRAY_SIZE(dt_providers); i++) {
>>          if (dt_providers[i]->update)
>>              dt_providers[i]->update(...);
>>      }
>>
>> any time it likes.  Or it can call dt_usdt.update(...).
>>
>> This is for WIP.  A different approach can be adopted later instead.
>>
>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
>> ---
>>   libdtrace/dt_provider.h | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/libdtrace/dt_provider.h b/libdtrace/dt_provider.h
>> index b1b1b1b8..71b5a3c4 100644
>> --- a/libdtrace/dt_provider.h
>> +++ b/libdtrace/dt_provider.h
>> @@ -71,6 +71,8 @@ typedef struct dt_provimpl {
>>   			      void *datap);
>>   	void (*destroy)(dtrace_hdl_t *dtp,	/* free provider data */
>>   			      void *datap);
>> +	void (*update)(dtrace_hdl_t *dtp,	/* update provider-specific info */
>> +			      void *datap);
>>   } dt_provimpl_t;
>>   
>>   /* list dt_dtrace first */
>> -- 
>> 2.18.4
>>

Re: [PATCH 24/38] test: Make test independent of specific PC

[Eugene Loh]

On 7/19/24 17:02, Kris Van Hees wrote:

> On Thu, Jun 27, 2024 at 01:38:50AM -0400, eugene.loh@oracle.com wrote:
>> From: Eugene Loh <eugene.loh@oracle.com>
> I agree in principle that this test should not depend on a specific PC value,
> but that leaves the issue that with this patch, I do not think there is any
> real test to verify that the PC value is sane.  So, that probably should be
> added.

Hmm, yeah, I guess so.  Okay, I posted v2 (with a tweaked title "test: 
Handle dtrace:::ERROR arg3 specially").  Also, I posted a v2 of patch 
25/38.  You already R-b'ed it, but it gets modified (in a very minor 
way) by the effects of this patch.  I'll assume your R-b still stands, 
but take a look if you're curious.

>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
>> ---
>>   test/unittest/error/tst.DTRACEFLT_UNKNOWN.d | 6 +++---
>>   test/unittest/error/tst.DTRACEFLT_UNKNOWN.r | 2 +-
>>   2 files changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
>> index 001903ff..bfc77bf5 100644
>> --- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
>> +++ b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.d
>> @@ -1,6 +1,6 @@
>>   /*
>>    * Oracle Linux DTrace.
>> - * Copyright (c) 2006, 2020, Oracle and/or its affiliates. All rights reserved.
>> + * Copyright (c) 2006, 2024, Oracle and/or its affiliates. All rights reserved.
>>    * Licensed under the Universal Permissive License v 1.0 as shown at
>>    * http://oss.oracle.com/licenses/upl.
>>    */
>> @@ -19,8 +19,8 @@
>>   
>>   ERROR
>>   {
>> -	printf("The arguments are %u %u %u %u %u\n",
>> -		arg1, arg2, arg3, arg4, arg5);
>> +	printf("The arguments are %u %u PC %u %u\n",
>> +		arg1, arg2, arg4, arg5);
>>   	printf("The value of arg4 = %u\n", DTRACEFLT_UNKNOWN);
>>   	exit(0);
>>   }
>> diff --git a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
>> index b11f6c99..3e7caac4 100644
>> --- a/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
>> +++ b/test/unittest/error/tst.DTRACEFLT_UNKNOWN.r
>> @@ -1,4 +1,4 @@
>> -The arguments are 2 2 4 1 64
>> +The arguments are 2 2 PC 1 64
>>   The value of arg4 = 0
>>   
>>   -- @@stderr --
>> -- 
>> 2.18.4
>>

Re: [PATCH 31/38] Fix dt_pebs_init() call

[Kris Van Hees]

This patch goes against the intent of the code, as far as I can see.  The
point of performing the size check prior to increasing the buffer size was
to enforce the limit on the size that was explicit set by the user.  Then,
if that size was sufficient, then DTrace is at liberty to increase that
size to a whole multiple of the page size.

But if I were to insist that a buffer can only be 24 bytes, and 24 bytes are
not enough to store a single trace record, DTrace should report an error
rather than increasing my bufsize setting on its own.

On Thu, Jun 27, 2024 at 01:38:57AM -0400, eugene.loh@oracle.com wrote:
> From: Eugene Loh <eugene.loh@oracle.com>
> 
> The function had a few issues, mainly with its comments,
> ranging from typos to incorrect descriptions of behavior.
> 
> The function has only one caller.  The enforcement of a
> size minimum was problematic in a number of respects:
> - it was missing 4 bytes
> - it was enforcing the minimum before the size was
>     increased by the caller
> - it was checking dt_pebs_init()==-1,
>     missing errors with other negative return values
> 
> So change the function to accept a minimum and change its
> return values.
> 
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> ---
>  libdtrace/dt_peb.c                          | 27 ++++++++++-----------
>  libdtrace/dt_peb.h                          |  2 +-
>  libdtrace/dt_work.c                         | 15 +++++++-----
>  test/unittest/options/err.b-too-low.d       | 22 +++++++++--------
>  test/unittest/options/err.bufsize-too-low.d | 22 +++++++++--------
>  test/unittest/options/tst.b.d               | 22 +++++++++--------
>  test/unittest/options/tst.bufsize.d         | 22 +++++++++--------
>  7 files changed, 71 insertions(+), 61 deletions(-)
> 
> diff --git a/libdtrace/dt_peb.c b/libdtrace/dt_peb.c
> index 5268f089..4983ba91 100644
> --- a/libdtrace/dt_peb.c
> +++ b/libdtrace/dt_peb.c
> @@ -136,18 +136,14 @@ dt_pebs_exit(dtrace_hdl_t *dtp)
>  }
>  
>  /*
> - * Initialize the perf event buffers (one per online CPU).  Each buffer will
> - * the given number of pages (i.e. the total size of each buffer will be
> - * num_pages * getpagesize()).  The  allocated memory for each buffer is mmap'd
> - * so the kernel can write to it, and its representative file descriptor is
> - * recorded in the 'buffers' BPF map so that BPF code knows where to write
> - * trace data for a specific CPU.
> + * Initialize the perf event buffers, one per online CPU.  Each buffer will
> + * have num_pages * getpagesize()).  The dt_peb_open() call mmaps the allocated
> + * memory so the kernel can write to it.  The file descriptor is recorded in
> + * the 'buffers' BPF map and added to the event polling file descriptor.
>   *
> - * An event polling file descriptor is created as well, and it is configured to
> - * monitor all perf event buffers at once.  This file descriptor is returned
> - * upon success..  Failure is indicated with a -1 return value.
> + * The return value indicates success (0) or failure (-1).
>   */
> -int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
> +int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize, size_t bufsizemin)
>  {
>  	int		i;
>  	int		mapfd;
> @@ -166,12 +162,15 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>  		fprintf(stderr, "bufsize increased to %lu\n",
>  			num_pages * getpagesize());
>  
> +	if (num_pages * getpagesize() < bufsizemin)
> +		return dt_set_errno(dtp, EDT_BUFTOOSMALL);
> +
>  	/*
>  	 * Determine the fd for the 'buffers' BPF map.
>  	 */
>  	idp = dt_dlib_get_map(dtp, "buffers");
>  	if (idp == NULL || idp->di_id == DT_IDENT_UNDEF)
> -		return -ENOENT;
> +		return dt_set_errno(dtp, EDT_NOMEM);    // FIXME we used to do something more akin to ENOENT
>  
>  	mapfd = idp->di_id;
>  
> @@ -180,7 +179,7 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>  	 */
>  	dtp->dt_pebset = dt_zalloc(dtp, sizeof(dt_pebset_t));
>  	if (dtp->dt_pebset == NULL)
> -		return -ENOMEM;
> +		return dt_set_errno(dtp, EDT_NOMEM);
>  
>  	/*
>  	 * Allocate the per-CPU perf event buffers.
> @@ -189,7 +188,7 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>  			 sizeof(struct dt_peb));
>  	if (pebs == NULL) {
>  		dt_free(dtp, dtp->dt_pebset);
> -		return -ENOMEM;
> +		return dt_set_errno(dtp, EDT_NOMEM);
>  	}
>  
>  	dtp->dt_pebset->pebs = pebs;
> @@ -241,5 +240,5 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>  fail:
>  	dt_pebs_exit(dtp);
>  
> -	return -1;
> +	return dt_set_errno(dtp, EDT_NOMEM);  // FIXME need something better
>  }
> diff --git a/libdtrace/dt_peb.h b/libdtrace/dt_peb.h
> index e0f408f2..9ad23252 100644
> --- a/libdtrace/dt_peb.h
> +++ b/libdtrace/dt_peb.h
> @@ -43,7 +43,7 @@ typedef struct dt_pebset {
>  } dt_pebset_t;
>  
>  extern void dt_pebs_exit(dtrace_hdl_t *);
> -extern int dt_pebs_init(dtrace_hdl_t *, size_t);
> +extern int dt_pebs_init(dtrace_hdl_t *, size_t, size_t);
>  
>  #ifdef	__cplusplus
>  }
> diff --git a/libdtrace/dt_work.c b/libdtrace/dt_work.c
> index 69a86358..7a0eb1da 100644
> --- a/libdtrace/dt_work.c
> +++ b/libdtrace/dt_work.c
> @@ -267,18 +267,21 @@ dtrace_go(dtrace_hdl_t *dtp, uint_t cflags)
>  		return dt_set_errno(dtp, errno);
>  
>  	/*
> -	 * We need enough space for the pref_event_header, a 32-bit size, a
> +	 * We need enough space for the perf_event_header, a 32-bit size, a
>  	 * 4-byte gap, and the largest trace data record we may be writing to
>  	 * the buffer.  In other words, the buffer needs to be large enough to
>  	 * hold at least one perf-encapsulated trace data record.
> +	 *
> +	 * While dt_pebs_init() rounds the requested size up, size==0 is a
> +	 * special case.
>  	 */
>  	dtrace_getopt(dtp, "bufsize", &size);
> -	if (size == 0 ||
> -	    size < sizeof(struct perf_event_header) + sizeof(uint32_t) +
> -		   dtp->dt_maxreclen)
> +	if (size == 0)
>  		return dt_set_errno(dtp, EDT_BUFTOOSMALL);
> -	if (dt_pebs_init(dtp, size) == -1)
> -		return dt_set_errno(dtp, EDT_NOMEM);
> +	if (dt_pebs_init(dtp, size,
> +			 sizeof(struct perf_event_header) + sizeof(uint32_t)
> +			 + 4 + dtp->dt_maxreclen) == -1)
> +		return -1;
>  
>  	/*
>  	 * We must initialize the aggregation consumer handling before we
> diff --git a/test/unittest/options/err.b-too-low.d b/test/unittest/options/err.b-too-low.d
> index bb77e37c..f62155dd 100644
> --- a/test/unittest/options/err.b-too-low.d
> +++ b/test/unittest/options/err.b-too-low.d
> @@ -1,6 +1,6 @@
>  /*
>   * Oracle Linux DTrace.
> - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
> + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
>   * Licensed under the Universal Permissive License v 1.0 as shown at
>   * http://oss.oracle.com/licenses/upl.
>   */
> @@ -12,19 +12,21 @@
>   */
>  
>  /*
> - * We use a buffer size of 59 because that should be just too small to hold the
> - * trace records generated in this script:
> - *	- perf_event_header (40 bytes)
> - *	- size (4 bytes)
> - *	- gap (4 bytes)
> - *	- EPID (4 bytes)
> - *	- tag (4 bytes)
> - *	- exit value (4 bytes)
> + * We need over 4k bytes for the 4 string trace records generated in this script
> + * plus some meta data.
> + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
> + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
>   */
> -/* @@runtest-opts: -b59 */
> +/* @@runtest-opts: -b4096 */
> +
> +#pragma D option strsize=1024
>  
>  BEGIN
>  {
> +	trace("abc");
> +	trace("def");
> +	trace("ghi");
> +	trace("jkl");
>  	exit(0);
>  }
>  
> diff --git a/test/unittest/options/err.bufsize-too-low.d b/test/unittest/options/err.bufsize-too-low.d
> index bbbdb5c5..25efdf72 100644
> --- a/test/unittest/options/err.bufsize-too-low.d
> +++ b/test/unittest/options/err.bufsize-too-low.d
> @@ -1,6 +1,6 @@
>  /*
>   * Oracle Linux DTrace.
> - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
> + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
>   * Licensed under the Universal Permissive License v 1.0 as shown at
>   * http://oss.oracle.com/licenses/upl.
>   */
> @@ -12,19 +12,21 @@
>   */
>  
>  /*
> - * We use a buffer size of 59 because that should be just too small to hold the
> - * trace records generated in this script:
> - *	- perf_event_header (40 bytes)
> - *	- size (4 bytes)
> - *	- gap (4 bytes)
> - *	- EPID (4 bytes)
> - *	- tag (4 bytes)
> - *	- exit value (4 bytes)
> + * We need over 4k bytes for the 4 string trace records generated in this script
> + * plus some meta data.
> + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
> + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
>   */
> -/* @@runtest-opts: -xbufsize=59 */
> +/* @@runtest-opts: -xbufsize=4096 */
> +
> +#pragma D option strsize=1024
>  
>  BEGIN
>  {
> +	trace("abc");
> +	trace("def");
> +	trace("ghi");
> +	trace("jkl");
>  	exit(0);
>  }
>  
> diff --git a/test/unittest/options/tst.b.d b/test/unittest/options/tst.b.d
> index 57fa030d..3bf08edc 100644
> --- a/test/unittest/options/tst.b.d
> +++ b/test/unittest/options/tst.b.d
> @@ -1,6 +1,6 @@
>  /*
>   * Oracle Linux DTrace.
> - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
> + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
>   * Licensed under the Universal Permissive License v 1.0 as shown at
>   * http://oss.oracle.com/licenses/upl.
>   */
> @@ -12,19 +12,21 @@
>   */
>  
>  /*
> - * We use a buffer size of 60 because that should be the exact size necessary
> - * to hold the trace records generated in this script:
> - *	- perf_event_header (40 bytes)
> - *	- size (4 bytes)
> - *	- gap (4 bytes)
> - *	- EPID (4 bytes)
> - *	- tag (4 bytes)
> - *	- exit value (4 bytes)
> + * We need over 4k bytes for the 4 string trace records generated in this script
> + * plus some meta data.
> + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
> + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
>   */
> -/* @@runtest-opts: -b60 */
> +/* @@runtest-opts: -b4097 */
> +
> +#pragma D option strsize=1024
>  
>  BEGIN
>  {
> +	trace("abc");
> +	trace("def");
> +	trace("ghi");
> +	trace("jkl");
>  	exit(0);
>  }
>  
> diff --git a/test/unittest/options/tst.bufsize.d b/test/unittest/options/tst.bufsize.d
> index 96b0f1b8..23af81aa 100644
> --- a/test/unittest/options/tst.bufsize.d
> +++ b/test/unittest/options/tst.bufsize.d
> @@ -1,6 +1,6 @@
>  /*
>   * Oracle Linux DTrace.
> - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
> + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
>   * Licensed under the Universal Permissive License v 1.0 as shown at
>   * http://oss.oracle.com/licenses/upl.
>   */
> @@ -12,19 +12,21 @@
>   */
>  
>  /*
> - * We use a buffer size of 60 because that should be the exact size necessary
> - * to hold the trace records generated in this script:
> - *	- perf_event_header (40 bytes)
> - *	- size (4 bytes)
> - *	- gap (4 bytes)
> - *	- EPID (4 bytes)
> - *	- tag (4 bytes)
> - *	- exit value (4 bytes)
> + * We need over 4k bytes for the 4 string trace records generated in this script
> + * plus some meta data.
> + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
> + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
>   */
> -/* @@runtest-opts: -xbufsize=60 */
> +/* @@runtest-opts: -xbufsize=4097 */
> +
> +#pragma D option strsize=1024
>  
>  BEGIN
>  {
> +	trace("abc");
> +	trace("def");
> +	trace("ghi");
> +	trace("jkl");
>  	exit(0);
>  }
>  
> -- 
> 2.18.4
> 

Re: [PATCH 31/38] Fix dt_pebs_init() call

[Eugene Loh]

On 8/26/24 10:30, Kris Van Hees wrote:

> This patch goes against the intent of the code, as far as I can see.  The
> point of performing the size check prior to increasing the buffer size was
> to enforce the limit on the size that was explicit set by the user.  Then,
> if that size was sufficient, then DTrace is at liberty to increase that
> size to a whole multiple of the page size.
>
> But if I were to insist that a buffer can only be 24 bytes, and 24 bytes are
> not enough to store a single trace record, DTrace should report an error
> rather than increasing my bufsize setting on its own.


The point is that DTrace *does* increase my bufsize setting on its own.  
Why should DTrace complain about my setting when it's going to ignore my 
setting anyhow?  What value does the pre-existing behavior offer to 
DTrace users?  Are you saying that DTrace should not resize?  That it 
should honor the user's bufsize?  No more resizing (to page size or 
whatever it wants)?

> On Thu, Jun 27, 2024 at 01:38:57AM -0400, eugene.loh@oracle.com wrote:
>> From: Eugene Loh <eugene.loh@oracle.com>
>>
>> The function had a few issues, mainly with its comments,
>> ranging from typos to incorrect descriptions of behavior.
>>
>> The function has only one caller.  The enforcement of a
>> size minimum was problematic in a number of respects:
>> - it was missing 4 bytes
>> - it was enforcing the minimum before the size was
>>      increased by the caller
>> - it was checking dt_pebs_init()==-1,
>>      missing errors with other negative return values
>>
>> So change the function to accept a minimum and change its
>> return values.
>>
>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
>> ---
>>   libdtrace/dt_peb.c                          | 27 ++++++++++-----------
>>   libdtrace/dt_peb.h                          |  2 +-
>>   libdtrace/dt_work.c                         | 15 +++++++-----
>>   test/unittest/options/err.b-too-low.d       | 22 +++++++++--------
>>   test/unittest/options/err.bufsize-too-low.d | 22 +++++++++--------
>>   test/unittest/options/tst.b.d               | 22 +++++++++--------
>>   test/unittest/options/tst.bufsize.d         | 22 +++++++++--------
>>   7 files changed, 71 insertions(+), 61 deletions(-)
>>
>> diff --git a/libdtrace/dt_peb.c b/libdtrace/dt_peb.c
>> index 5268f089..4983ba91 100644
>> --- a/libdtrace/dt_peb.c
>> +++ b/libdtrace/dt_peb.c
>> @@ -136,18 +136,14 @@ dt_pebs_exit(dtrace_hdl_t *dtp)
>>   }
>>   
>>   /*
>> - * Initialize the perf event buffers (one per online CPU).  Each buffer will
>> - * the given number of pages (i.e. the total size of each buffer will be
>> - * num_pages * getpagesize()).  The  allocated memory for each buffer is mmap'd
>> - * so the kernel can write to it, and its representative file descriptor is
>> - * recorded in the 'buffers' BPF map so that BPF code knows where to write
>> - * trace data for a specific CPU.
>> + * Initialize the perf event buffers, one per online CPU.  Each buffer will
>> + * have num_pages * getpagesize()).  The dt_peb_open() call mmaps the allocated
>> + * memory so the kernel can write to it.  The file descriptor is recorded in
>> + * the 'buffers' BPF map and added to the event polling file descriptor.
>>    *
>> - * An event polling file descriptor is created as well, and it is configured to
>> - * monitor all perf event buffers at once.  This file descriptor is returned
>> - * upon success..  Failure is indicated with a -1 return value.
>> + * The return value indicates success (0) or failure (-1).
>>    */
>> -int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>> +int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize, size_t bufsizemin)
>>   {
>>   	int		i;
>>   	int		mapfd;
>> @@ -166,12 +162,15 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>>   		fprintf(stderr, "bufsize increased to %lu\n",
>>   			num_pages * getpagesize());
>>   
>> +	if (num_pages * getpagesize() < bufsizemin)
>> +		return dt_set_errno(dtp, EDT_BUFTOOSMALL);
>> +
>>   	/*
>>   	 * Determine the fd for the 'buffers' BPF map.
>>   	 */
>>   	idp = dt_dlib_get_map(dtp, "buffers");
>>   	if (idp == NULL || idp->di_id == DT_IDENT_UNDEF)
>> -		return -ENOENT;
>> +		return dt_set_errno(dtp, EDT_NOMEM);    // FIXME we used to do something more akin to ENOENT
>>   
>>   	mapfd = idp->di_id;
>>   
>> @@ -180,7 +179,7 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>>   	 */
>>   	dtp->dt_pebset = dt_zalloc(dtp, sizeof(dt_pebset_t));
>>   	if (dtp->dt_pebset == NULL)
>> -		return -ENOMEM;
>> +		return dt_set_errno(dtp, EDT_NOMEM);
>>   
>>   	/*
>>   	 * Allocate the per-CPU perf event buffers.
>> @@ -189,7 +188,7 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>>   			 sizeof(struct dt_peb));
>>   	if (pebs == NULL) {
>>   		dt_free(dtp, dtp->dt_pebset);
>> -		return -ENOMEM;
>> +		return dt_set_errno(dtp, EDT_NOMEM);
>>   	}
>>   
>>   	dtp->dt_pebset->pebs = pebs;
>> @@ -241,5 +240,5 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>>   fail:
>>   	dt_pebs_exit(dtp);
>>   
>> -	return -1;
>> +	return dt_set_errno(dtp, EDT_NOMEM);  // FIXME need something better
>>   }
>> diff --git a/libdtrace/dt_peb.h b/libdtrace/dt_peb.h
>> index e0f408f2..9ad23252 100644
>> --- a/libdtrace/dt_peb.h
>> +++ b/libdtrace/dt_peb.h
>> @@ -43,7 +43,7 @@ typedef struct dt_pebset {
>>   } dt_pebset_t;
>>   
>>   extern void dt_pebs_exit(dtrace_hdl_t *);
>> -extern int dt_pebs_init(dtrace_hdl_t *, size_t);
>> +extern int dt_pebs_init(dtrace_hdl_t *, size_t, size_t);
>>   
>>   #ifdef	__cplusplus
>>   }
>> diff --git a/libdtrace/dt_work.c b/libdtrace/dt_work.c
>> index 69a86358..7a0eb1da 100644
>> --- a/libdtrace/dt_work.c
>> +++ b/libdtrace/dt_work.c
>> @@ -267,18 +267,21 @@ dtrace_go(dtrace_hdl_t *dtp, uint_t cflags)
>>   		return dt_set_errno(dtp, errno);
>>   
>>   	/*
>> -	 * We need enough space for the pref_event_header, a 32-bit size, a
>> +	 * We need enough space for the perf_event_header, a 32-bit size, a
>>   	 * 4-byte gap, and the largest trace data record we may be writing to
>>   	 * the buffer.  In other words, the buffer needs to be large enough to
>>   	 * hold at least one perf-encapsulated trace data record.
>> +	 *
>> +	 * While dt_pebs_init() rounds the requested size up, size==0 is a
>> +	 * special case.
>>   	 */
>>   	dtrace_getopt(dtp, "bufsize", &size);
>> -	if (size == 0 ||
>> -	    size < sizeof(struct perf_event_header) + sizeof(uint32_t) +
>> -		   dtp->dt_maxreclen)
>> +	if (size == 0)
>>   		return dt_set_errno(dtp, EDT_BUFTOOSMALL);
>> -	if (dt_pebs_init(dtp, size) == -1)
>> -		return dt_set_errno(dtp, EDT_NOMEM);
>> +	if (dt_pebs_init(dtp, size,
>> +			 sizeof(struct perf_event_header) + sizeof(uint32_t)
>> +			 + 4 + dtp->dt_maxreclen) == -1)
>> +		return -1;
>>   
>>   	/*
>>   	 * We must initialize the aggregation consumer handling before we
>> diff --git a/test/unittest/options/err.b-too-low.d b/test/unittest/options/err.b-too-low.d
>> index bb77e37c..f62155dd 100644
>> --- a/test/unittest/options/err.b-too-low.d
>> +++ b/test/unittest/options/err.b-too-low.d
>> @@ -1,6 +1,6 @@
>>   /*
>>    * Oracle Linux DTrace.
>> - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
>> + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
>>    * Licensed under the Universal Permissive License v 1.0 as shown at
>>    * http://oss.oracle.com/licenses/upl.
>>    */
>> @@ -12,19 +12,21 @@
>>    */
>>   
>>   /*
>> - * We use a buffer size of 59 because that should be just too small to hold the
>> - * trace records generated in this script:
>> - *	- perf_event_header (40 bytes)
>> - *	- size (4 bytes)
>> - *	- gap (4 bytes)
>> - *	- EPID (4 bytes)
>> - *	- tag (4 bytes)
>> - *	- exit value (4 bytes)
>> + * We need over 4k bytes for the 4 string trace records generated in this script
>> + * plus some meta data.
>> + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
>> + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
>>    */
>> -/* @@runtest-opts: -b59 */
>> +/* @@runtest-opts: -b4096 */
>> +
>> +#pragma D option strsize=1024
>>   
>>   BEGIN
>>   {
>> +	trace("abc");
>> +	trace("def");
>> +	trace("ghi");
>> +	trace("jkl");
>>   	exit(0);
>>   }
>>   
>> diff --git a/test/unittest/options/err.bufsize-too-low.d b/test/unittest/options/err.bufsize-too-low.d
>> index bbbdb5c5..25efdf72 100644
>> --- a/test/unittest/options/err.bufsize-too-low.d
>> +++ b/test/unittest/options/err.bufsize-too-low.d
>> @@ -1,6 +1,6 @@
>>   /*
>>    * Oracle Linux DTrace.
>> - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
>> + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
>>    * Licensed under the Universal Permissive License v 1.0 as shown at
>>    * http://oss.oracle.com/licenses/upl.
>>    */
>> @@ -12,19 +12,21 @@
>>    */
>>   
>>   /*
>> - * We use a buffer size of 59 because that should be just too small to hold the
>> - * trace records generated in this script:
>> - *	- perf_event_header (40 bytes)
>> - *	- size (4 bytes)
>> - *	- gap (4 bytes)
>> - *	- EPID (4 bytes)
>> - *	- tag (4 bytes)
>> - *	- exit value (4 bytes)
>> + * We need over 4k bytes for the 4 string trace records generated in this script
>> + * plus some meta data.
>> + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
>> + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
>>    */
>> -/* @@runtest-opts: -xbufsize=59 */
>> +/* @@runtest-opts: -xbufsize=4096 */
>> +
>> +#pragma D option strsize=1024
>>   
>>   BEGIN
>>   {
>> +	trace("abc");
>> +	trace("def");
>> +	trace("ghi");
>> +	trace("jkl");
>>   	exit(0);
>>   }
>>   
>> diff --git a/test/unittest/options/tst.b.d b/test/unittest/options/tst.b.d
>> index 57fa030d..3bf08edc 100644
>> --- a/test/unittest/options/tst.b.d
>> +++ b/test/unittest/options/tst.b.d
>> @@ -1,6 +1,6 @@
>>   /*
>>    * Oracle Linux DTrace.
>> - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
>> + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
>>    * Licensed under the Universal Permissive License v 1.0 as shown at
>>    * http://oss.oracle.com/licenses/upl.
>>    */
>> @@ -12,19 +12,21 @@
>>    */
>>   
>>   /*
>> - * We use a buffer size of 60 because that should be the exact size necessary
>> - * to hold the trace records generated in this script:
>> - *	- perf_event_header (40 bytes)
>> - *	- size (4 bytes)
>> - *	- gap (4 bytes)
>> - *	- EPID (4 bytes)
>> - *	- tag (4 bytes)
>> - *	- exit value (4 bytes)
>> + * We need over 4k bytes for the 4 string trace records generated in this script
>> + * plus some meta data.
>> + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
>> + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
>>    */
>> -/* @@runtest-opts: -b60 */
>> +/* @@runtest-opts: -b4097 */
>> +
>> +#pragma D option strsize=1024
>>   
>>   BEGIN
>>   {
>> +	trace("abc");
>> +	trace("def");
>> +	trace("ghi");
>> +	trace("jkl");
>>   	exit(0);
>>   }
>>   
>> diff --git a/test/unittest/options/tst.bufsize.d b/test/unittest/options/tst.bufsize.d
>> index 96b0f1b8..23af81aa 100644
>> --- a/test/unittest/options/tst.bufsize.d
>> +++ b/test/unittest/options/tst.bufsize.d
>> @@ -1,6 +1,6 @@
>>   /*
>>    * Oracle Linux DTrace.
>> - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
>> + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
>>    * Licensed under the Universal Permissive License v 1.0 as shown at
>>    * http://oss.oracle.com/licenses/upl.
>>    */
>> @@ -12,19 +12,21 @@
>>    */
>>   
>>   /*
>> - * We use a buffer size of 60 because that should be the exact size necessary
>> - * to hold the trace records generated in this script:
>> - *	- perf_event_header (40 bytes)
>> - *	- size (4 bytes)
>> - *	- gap (4 bytes)
>> - *	- EPID (4 bytes)
>> - *	- tag (4 bytes)
>> - *	- exit value (4 bytes)
>> + * We need over 4k bytes for the 4 string trace records generated in this script
>> + * plus some meta data.
>> + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
>> + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
>>    */
>> -/* @@runtest-opts: -xbufsize=60 */
>> +/* @@runtest-opts: -xbufsize=4097 */
>> +
>> +#pragma D option strsize=1024
>>   
>>   BEGIN
>>   {
>> +	trace("abc");
>> +	trace("def");
>> +	trace("ghi");
>> +	trace("jkl");
>>   	exit(0);
>>   }
>>   
>> -- 
>> 2.18.4
>>

Re: [PATCH 31/38] Fix dt_pebs_init() call

[Kris Van Hees]

On Mon, Aug 26, 2024 at 11:42:25AM -0400, Eugene Loh wrote:
> On 8/26/24 10:30, Kris Van Hees wrote:
> 
> > This patch goes against the intent of the code, as far as I can see.  The
> > point of performing the size check prior to increasing the buffer size was
> > to enforce the limit on the size that was explicit set by the user.  Then,
> > if that size was sufficient, then DTrace is at liberty to increase that
> > size to a whole multiple of the page size.
> > 
> > But if I were to insist that a buffer can only be 24 bytes, and 24 bytes are
> > not enough to store a single trace record, DTrace should report an error
> > rather than increasing my bufsize setting on its own.
> 
> 
> The point is that DTrace *does* increase my bufsize setting on its own.  Why
> should DTrace complain about my setting when it's going to ignore my setting
> anyhow?  What value does the pre-existing behavior offer to DTrace users? 
> Are you saying that DTrace should not resize?  That it should honor the
> user's bufsize?  No more resizing (to page size or whatever it wants)?

The main point of this code was to retain the bevhaviour that DTrace provided,
which is that if a user explicitly requests a bufsize that is less than the
minimum required for the given trace programs, an error is reported.  That is
a mater of being consistent (even if it has no practical application that I
can see).  Tests from the existing version failed because this was not being
done right.

As far as functionality, as long as the bufsize is large enough to hold the
largest trace record, DTrace can (and needs to) ensure that the buffer size
is a whole multiple of the page size because that is required by the perf
ring buffer imlementation.

So, DTrace only increases the buffer size to the required multiple of page size
if the user supplied bufsize is at least large enough to hold the largest trace
record (or if no bufsize was explicitly specified).

We could opt to change the behaviour of the bufsize opton to always round up to
a whole multiple of page size, but that is a deviation from the original
implementation and thus is something that would need to be documented, etc...

Aside from the computation of the minimum required size needing updating when
patches (e.g. the USDT changes) modify the layout and size of the trace records,
I do not think that the existing code is wrong.

I am certainly open to a feature change to make bufsize always be rounded up as
needed, but that is not a bug fix and I think that ought to be its own patch
(along with documentation changes etc).  And it might need some more discussion
because it brings up the question why we wouldn't just simply always increase
bufsize to be the smallest multiple of the page size to accomodate the largest
trace record.

> > On Thu, Jun 27, 2024 at 01:38:57AM -0400, eugene.loh@oracle.com wrote:
> > > From: Eugene Loh <eugene.loh@oracle.com>
> > > 
> > > The function had a few issues, mainly with its comments,
> > > ranging from typos to incorrect descriptions of behavior.
> > > 
> > > The function has only one caller.  The enforcement of a
> > > size minimum was problematic in a number of respects:
> > > - it was missing 4 bytes
> > > - it was enforcing the minimum before the size was
> > >      increased by the caller
> > > - it was checking dt_pebs_init()==-1,
> > >      missing errors with other negative return values
> > > 
> > > So change the function to accept a minimum and change its
> > > return values.
> > > 
> > > Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> > > ---
> > >   libdtrace/dt_peb.c                          | 27 ++++++++++-----------
> > >   libdtrace/dt_peb.h                          |  2 +-
> > >   libdtrace/dt_work.c                         | 15 +++++++-----
> > >   test/unittest/options/err.b-too-low.d       | 22 +++++++++--------
> > >   test/unittest/options/err.bufsize-too-low.d | 22 +++++++++--------
> > >   test/unittest/options/tst.b.d               | 22 +++++++++--------
> > >   test/unittest/options/tst.bufsize.d         | 22 +++++++++--------
> > >   7 files changed, 71 insertions(+), 61 deletions(-)
> > > 
> > > diff --git a/libdtrace/dt_peb.c b/libdtrace/dt_peb.c
> > > index 5268f089..4983ba91 100644
> > > --- a/libdtrace/dt_peb.c
> > > +++ b/libdtrace/dt_peb.c
> > > @@ -136,18 +136,14 @@ dt_pebs_exit(dtrace_hdl_t *dtp)
> > >   }
> > >   /*
> > > - * Initialize the perf event buffers (one per online CPU).  Each buffer will
> > > - * the given number of pages (i.e. the total size of each buffer will be
> > > - * num_pages * getpagesize()).  The  allocated memory for each buffer is mmap'd
> > > - * so the kernel can write to it, and its representative file descriptor is
> > > - * recorded in the 'buffers' BPF map so that BPF code knows where to write
> > > - * trace data for a specific CPU.
> > > + * Initialize the perf event buffers, one per online CPU.  Each buffer will
> > > + * have num_pages * getpagesize()).  The dt_peb_open() call mmaps the allocated
> > > + * memory so the kernel can write to it.  The file descriptor is recorded in
> > > + * the 'buffers' BPF map and added to the event polling file descriptor.
> > >    *
> > > - * An event polling file descriptor is created as well, and it is configured to
> > > - * monitor all perf event buffers at once.  This file descriptor is returned
> > > - * upon success..  Failure is indicated with a -1 return value.
> > > + * The return value indicates success (0) or failure (-1).
> > >    */
> > > -int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
> > > +int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize, size_t bufsizemin)
> > >   {
> > >   	int		i;
> > >   	int		mapfd;
> > > @@ -166,12 +162,15 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
> > >   		fprintf(stderr, "bufsize increased to %lu\n",
> > >   			num_pages * getpagesize());
> > > +	if (num_pages * getpagesize() < bufsizemin)
> > > +		return dt_set_errno(dtp, EDT_BUFTOOSMALL);
> > > +
> > >   	/*
> > >   	 * Determine the fd for the 'buffers' BPF map.
> > >   	 */
> > >   	idp = dt_dlib_get_map(dtp, "buffers");
> > >   	if (idp == NULL || idp->di_id == DT_IDENT_UNDEF)
> > > -		return -ENOENT;
> > > +		return dt_set_errno(dtp, EDT_NOMEM);    // FIXME we used to do something more akin to ENOENT
> > >   	mapfd = idp->di_id;
> > > @@ -180,7 +179,7 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
> > >   	 */
> > >   	dtp->dt_pebset = dt_zalloc(dtp, sizeof(dt_pebset_t));
> > >   	if (dtp->dt_pebset == NULL)
> > > -		return -ENOMEM;
> > > +		return dt_set_errno(dtp, EDT_NOMEM);
> > >   	/*
> > >   	 * Allocate the per-CPU perf event buffers.
> > > @@ -189,7 +188,7 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
> > >   			 sizeof(struct dt_peb));
> > >   	if (pebs == NULL) {
> > >   		dt_free(dtp, dtp->dt_pebset);
> > > -		return -ENOMEM;
> > > +		return dt_set_errno(dtp, EDT_NOMEM);
> > >   	}
> > >   	dtp->dt_pebset->pebs = pebs;
> > > @@ -241,5 +240,5 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
> > >   fail:
> > >   	dt_pebs_exit(dtp);
> > > -	return -1;
> > > +	return dt_set_errno(dtp, EDT_NOMEM);  // FIXME need something better
> > >   }
> > > diff --git a/libdtrace/dt_peb.h b/libdtrace/dt_peb.h
> > > index e0f408f2..9ad23252 100644
> > > --- a/libdtrace/dt_peb.h
> > > +++ b/libdtrace/dt_peb.h
> > > @@ -43,7 +43,7 @@ typedef struct dt_pebset {
> > >   } dt_pebset_t;
> > >   extern void dt_pebs_exit(dtrace_hdl_t *);
> > > -extern int dt_pebs_init(dtrace_hdl_t *, size_t);
> > > +extern int dt_pebs_init(dtrace_hdl_t *, size_t, size_t);
> > >   #ifdef	__cplusplus
> > >   }
> > > diff --git a/libdtrace/dt_work.c b/libdtrace/dt_work.c
> > > index 69a86358..7a0eb1da 100644
> > > --- a/libdtrace/dt_work.c
> > > +++ b/libdtrace/dt_work.c
> > > @@ -267,18 +267,21 @@ dtrace_go(dtrace_hdl_t *dtp, uint_t cflags)
> > >   		return dt_set_errno(dtp, errno);
> > >   	/*
> > > -	 * We need enough space for the pref_event_header, a 32-bit size, a
> > > +	 * We need enough space for the perf_event_header, a 32-bit size, a
> > >   	 * 4-byte gap, and the largest trace data record we may be writing to
> > >   	 * the buffer.  In other words, the buffer needs to be large enough to
> > >   	 * hold at least one perf-encapsulated trace data record.
> > > +	 *
> > > +	 * While dt_pebs_init() rounds the requested size up, size==0 is a
> > > +	 * special case.
> > >   	 */
> > >   	dtrace_getopt(dtp, "bufsize", &size);
> > > -	if (size == 0 ||
> > > -	    size < sizeof(struct perf_event_header) + sizeof(uint32_t) +
> > > -		   dtp->dt_maxreclen)
> > > +	if (size == 0)
> > >   		return dt_set_errno(dtp, EDT_BUFTOOSMALL);
> > > -	if (dt_pebs_init(dtp, size) == -1)
> > > -		return dt_set_errno(dtp, EDT_NOMEM);
> > > +	if (dt_pebs_init(dtp, size,
> > > +			 sizeof(struct perf_event_header) + sizeof(uint32_t)
> > > +			 + 4 + dtp->dt_maxreclen) == -1)
> > > +		return -1;
> > >   	/*
> > >   	 * We must initialize the aggregation consumer handling before we
> > > diff --git a/test/unittest/options/err.b-too-low.d b/test/unittest/options/err.b-too-low.d
> > > index bb77e37c..f62155dd 100644
> > > --- a/test/unittest/options/err.b-too-low.d
> > > +++ b/test/unittest/options/err.b-too-low.d
> > > @@ -1,6 +1,6 @@
> > >   /*
> > >    * Oracle Linux DTrace.
> > > - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
> > > + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
> > >    * Licensed under the Universal Permissive License v 1.0 as shown at
> > >    * http://oss.oracle.com/licenses/upl.
> > >    */
> > > @@ -12,19 +12,21 @@
> > >    */
> > >   /*
> > > - * We use a buffer size of 59 because that should be just too small to hold the
> > > - * trace records generated in this script:
> > > - *	- perf_event_header (40 bytes)
> > > - *	- size (4 bytes)
> > > - *	- gap (4 bytes)
> > > - *	- EPID (4 bytes)
> > > - *	- tag (4 bytes)
> > > - *	- exit value (4 bytes)
> > > + * We need over 4k bytes for the 4 string trace records generated in this script
> > > + * plus some meta data.
> > > + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
> > > + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
> > >    */
> > > -/* @@runtest-opts: -b59 */
> > > +/* @@runtest-opts: -b4096 */
> > > +
> > > +#pragma D option strsize=1024
> > >   BEGIN
> > >   {
> > > +	trace("abc");
> > > +	trace("def");
> > > +	trace("ghi");
> > > +	trace("jkl");
> > >   	exit(0);
> > >   }
> > > diff --git a/test/unittest/options/err.bufsize-too-low.d b/test/unittest/options/err.bufsize-too-low.d
> > > index bbbdb5c5..25efdf72 100644
> > > --- a/test/unittest/options/err.bufsize-too-low.d
> > > +++ b/test/unittest/options/err.bufsize-too-low.d
> > > @@ -1,6 +1,6 @@
> > >   /*
> > >    * Oracle Linux DTrace.
> > > - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
> > > + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
> > >    * Licensed under the Universal Permissive License v 1.0 as shown at
> > >    * http://oss.oracle.com/licenses/upl.
> > >    */
> > > @@ -12,19 +12,21 @@
> > >    */
> > >   /*
> > > - * We use a buffer size of 59 because that should be just too small to hold the
> > > - * trace records generated in this script:
> > > - *	- perf_event_header (40 bytes)
> > > - *	- size (4 bytes)
> > > - *	- gap (4 bytes)
> > > - *	- EPID (4 bytes)
> > > - *	- tag (4 bytes)
> > > - *	- exit value (4 bytes)
> > > + * We need over 4k bytes for the 4 string trace records generated in this script
> > > + * plus some meta data.
> > > + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
> > > + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
> > >    */
> > > -/* @@runtest-opts: -xbufsize=59 */
> > > +/* @@runtest-opts: -xbufsize=4096 */
> > > +
> > > +#pragma D option strsize=1024
> > >   BEGIN
> > >   {
> > > +	trace("abc");
> > > +	trace("def");
> > > +	trace("ghi");
> > > +	trace("jkl");
> > >   	exit(0);
> > >   }
> > > diff --git a/test/unittest/options/tst.b.d b/test/unittest/options/tst.b.d
> > > index 57fa030d..3bf08edc 100644
> > > --- a/test/unittest/options/tst.b.d
> > > +++ b/test/unittest/options/tst.b.d
> > > @@ -1,6 +1,6 @@
> > >   /*
> > >    * Oracle Linux DTrace.
> > > - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
> > > + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
> > >    * Licensed under the Universal Permissive License v 1.0 as shown at
> > >    * http://oss.oracle.com/licenses/upl.
> > >    */
> > > @@ -12,19 +12,21 @@
> > >    */
> > >   /*
> > > - * We use a buffer size of 60 because that should be the exact size necessary
> > > - * to hold the trace records generated in this script:
> > > - *	- perf_event_header (40 bytes)
> > > - *	- size (4 bytes)
> > > - *	- gap (4 bytes)
> > > - *	- EPID (4 bytes)
> > > - *	- tag (4 bytes)
> > > - *	- exit value (4 bytes)
> > > + * We need over 4k bytes for the 4 string trace records generated in this script
> > > + * plus some meta data.
> > > + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
> > > + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
> > >    */
> > > -/* @@runtest-opts: -b60 */
> > > +/* @@runtest-opts: -b4097 */
> > > +
> > > +#pragma D option strsize=1024
> > >   BEGIN
> > >   {
> > > +	trace("abc");
> > > +	trace("def");
> > > +	trace("ghi");
> > > +	trace("jkl");
> > >   	exit(0);
> > >   }
> > > diff --git a/test/unittest/options/tst.bufsize.d b/test/unittest/options/tst.bufsize.d
> > > index 96b0f1b8..23af81aa 100644
> > > --- a/test/unittest/options/tst.bufsize.d
> > > +++ b/test/unittest/options/tst.bufsize.d
> > > @@ -1,6 +1,6 @@
> > >   /*
> > >    * Oracle Linux DTrace.
> > > - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
> > > + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
> > >    * Licensed under the Universal Permissive License v 1.0 as shown at
> > >    * http://oss.oracle.com/licenses/upl.
> > >    */
> > > @@ -12,19 +12,21 @@
> > >    */
> > >   /*
> > > - * We use a buffer size of 60 because that should be the exact size necessary
> > > - * to hold the trace records generated in this script:
> > > - *	- perf_event_header (40 bytes)
> > > - *	- size (4 bytes)
> > > - *	- gap (4 bytes)
> > > - *	- EPID (4 bytes)
> > > - *	- tag (4 bytes)
> > > - *	- exit value (4 bytes)
> > > + * We need over 4k bytes for the 4 string trace records generated in this script
> > > + * plus some meta data.
> > > + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
> > > + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
> > >    */
> > > -/* @@runtest-opts: -xbufsize=60 */
> > > +/* @@runtest-opts: -xbufsize=4097 */
> > > +
> > > +#pragma D option strsize=1024
> > >   BEGIN
> > >   {
> > > +	trace("abc");
> > > +	trace("def");
> > > +	trace("ghi");
> > > +	trace("jkl");
> > >   	exit(0);
> > >   }
> > > -- 
> > > 2.18.4
> > > 

Re: [PATCH 28/38] Remove unused "next" arg from dt_flowindent()

[Kris Van Hees]

Lookinga at this - it might be that 'next' is not needed (ever) but I need to
look at this code again because I wrote it a very long time ago and I cannot
recall whether I reworked it to never need it or whether this is essentially
something that was left to be handled later (and forgotten).

On Thu, Jun 27, 2024 at 01:38:54AM -0400, eugene.loh@oracle.com wrote:
> From: Eugene Loh <eugene.loh@oracle.com>
> 
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> ---
>  libdtrace/dt_consume.c | 24 +++---------------------
>  1 file changed, 3 insertions(+), 21 deletions(-)
> 
> diff --git a/libdtrace/dt_consume.c b/libdtrace/dt_consume.c
> index 5fb636fe..7dfec72f 100644
> --- a/libdtrace/dt_consume.c
> +++ b/libdtrace/dt_consume.c
> @@ -432,11 +432,9 @@ static dt_htab_ops_t dt_spec_buf_htab_ops = {
>  };
>  
>  static int
> -dt_flowindent(dtrace_hdl_t *dtp, dtrace_probedata_t *data, dtrace_epid_t last,
> -	      dtrace_epid_t next)
> +dt_flowindent(dtrace_hdl_t *dtp, dtrace_probedata_t *data, dtrace_epid_t last)
>  {
> -	dtrace_probedesc_t	*pd = data->dtpda_pdesc, *npd;
> -	dtrace_datadesc_t	*ndd;
> +	dtrace_probedesc_t	*pd = data->dtpda_pdesc;
>  	dtrace_flowkind_t	flow = DTRACEFLOW_NONE;
>  	const char		*p = pd->prv;
>  	const char		*n = pd->prb;
> @@ -447,7 +445,6 @@ dt_flowindent(dtrace_hdl_t *dtp, dtrace_probedata_t *data, dtrace_epid_t last,
>  	static const char	*ent = "entry", *ret = "return";
>  	static int		entlen = 0, retlen = 0;
>  	dtrace_epid_t		id = data->dtpda_epid;
> -	int			rval;
>  
>  	if (entlen == 0) {
>  		assert(retlen == 0);
> @@ -485,21 +482,6 @@ dt_flowindent(dtrace_hdl_t *dtp, dtrace_probedata_t *data, dtrace_epid_t last,
>  			flow = DTRACEFLOW_NONE;
>  	}
>  
> -	/*
> -	 * If we're going to unindent this, it's more difficult to see if
> -	 * we don't actually want to unindent it -- we need to look at the
> -	 * _next_ EPID.
> -	 */
> -	if (flow == DTRACEFLOW_RETURN && next != DTRACE_EPIDNONE &&
> -	    next != id) {
> -		rval = dt_epid_lookup(dtp, next, &ndd, &npd);
> -		if (rval != 0)
> -			return rval;
> -
> -		if (npd->id == pd->id)
> -			flow = DTRACEFLOW_NONE;
> -	}
> -
>  	if (flow == DTRACEFLOW_ENTRY || flow == DTRACEFLOW_RETURN)
>  		data->dtpda_prefix = str;
>  	else
> @@ -2343,7 +2325,7 @@ dt_consume_one_probe(dtrace_hdl_t *dtp, FILE *fp, char *data, uint32_t size,
>  
>  	if (data_recording) {
>  		if (flow)
> -			dt_flowindent(dtp, pdat, *last, DTRACE_EPIDNONE);
> +			dt_flowindent(dtp, pdat, *last);
>  
>  		rval = (*efunc)(pdat, arg);
>  
> -- 
> 2.18.4
> 

Re: [PATCH 31/38] Fix dt_pebs_init() call

[Eugene Loh]

On 8/26/24 12:20, Kris Van Hees wrote:

> On Mon, Aug 26, 2024 at 11:42:25AM -0400, Eugene Loh wrote:
>> On 8/26/24 10:30, Kris Van Hees wrote:
>>
>>> This patch goes against the intent of the code, as far as I can see.  The
>>> point of performing the size check prior to increasing the buffer size was
>>> to enforce the limit on the size that was explicit set by the user.  Then,
>>> if that size was sufficient, then DTrace is at liberty to increase that
>>> size to a whole multiple of the page size.
>>>
>>> But if I were to insist that a buffer can only be 24 bytes, and 24 bytes are
>>> not enough to store a single trace record, DTrace should report an error
>>> rather than increasing my bufsize setting on its own.
>>
>> The point is that DTrace *does* increase my bufsize setting on its own.  Why
>> should DTrace complain about my setting when it's going to ignore my setting
>> anyhow?  What value does the pre-existing behavior offer to DTrace users?
>> Are you saying that DTrace should not resize?  That it should honor the
>> user's bufsize?  No more resizing (to page size or whatever it wants)?
> The main point of this code was to retain the bevhaviour that DTrace provided,

As far as adhering to legacy behavior... quite simply, we do not. We do 
not set the "principal buffer size" to bufsize.  Further, any legacy 
resizing behavior is, if I understand correctly, to go smaller than the 
user requested;  we do the opposite.  Looking to the legacy behavior for 
precedent is pointless.  The situation is that we are unlike legacy 
behavior in fundamental respects, opposite in another respect (automatic 
resizing), but consistent in some impractical and useless respect.  That 
consistency strikes me as small consolation.

There is no documentation to change;  the documentation is already 
wrong.  This patch was intended as a step to making the behavior 
something that was describable.  Clearly, trying to hold on to a legacy 
buffering model has produced a byzantine situation in the current port, 
which treats buffering very differently.

Perhaps we simply disagree on those points, and this patch is DOA. But 
the patch points out other problems as well.  E.g., the min computation 
wasn't even right and error checking was wrong.

> which is that if a user explicitly requests a bufsize that is less than the
> minimum required for the given trace programs, an error is reported.  That is
> a mater of being consistent (even if it has no practical application that I
> can see).  Tests from the existing version failed because this was not being
> done right.
>
> As far as functionality, as long as the bufsize is large enough to hold the
> largest trace record, DTrace can (and needs to) ensure that the buffer size
> is a whole multiple of the page size because that is required by the perf
> ring buffer imlementation.
>
> So, DTrace only increases the buffer size to the required multiple of page size
> if the user supplied bufsize is at least large enough to hold the largest trace
> record (or if no bufsize was explicitly specified).
>
> We could opt to change the behaviour of the bufsize opton to always round up to
> a whole multiple of page size, but that is a deviation from the original
> implementation and thus is something that would need to be documented, etc...
>
> Aside from the computation of the minimum required size needing updating when
> patches (e.g. the USDT changes) modify the layout and size of the trace records,
> I do not think that the existing code is wrong.
>
> I am certainly open to a feature change to make bufsize always be rounded up as
> needed, but that is not a bug fix and I think that ought to be its own patch
> (along with documentation changes etc).  And it might need some more discussion
> because it brings up the question why we wouldn't just simply always increase
> bufsize to be the smallest multiple of the page size to accomodate the largest
> trace record.
>
>>> On Thu, Jun 27, 2024 at 01:38:57AM -0400, eugene.loh@oracle.com wrote:
>>>> From: Eugene Loh <eugene.loh@oracle.com>
>>>>
>>>> The function had a few issues, mainly with its comments,
>>>> ranging from typos to incorrect descriptions of behavior.
>>>>
>>>> The function has only one caller.  The enforcement of a
>>>> size minimum was problematic in a number of respects:
>>>> - it was missing 4 bytes
>>>> - it was enforcing the minimum before the size was
>>>>       increased by the caller
>>>> - it was checking dt_pebs_init()==-1,
>>>>       missing errors with other negative return values
>>>>
>>>> So change the function to accept a minimum and change its
>>>> return values.
>>>>
>>>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
>>>> ---
>>>>    libdtrace/dt_peb.c                          | 27 ++++++++++-----------
>>>>    libdtrace/dt_peb.h                          |  2 +-
>>>>    libdtrace/dt_work.c                         | 15 +++++++-----
>>>>    test/unittest/options/err.b-too-low.d       | 22 +++++++++--------
>>>>    test/unittest/options/err.bufsize-too-low.d | 22 +++++++++--------
>>>>    test/unittest/options/tst.b.d               | 22 +++++++++--------
>>>>    test/unittest/options/tst.bufsize.d         | 22 +++++++++--------
>>>>    7 files changed, 71 insertions(+), 61 deletions(-)
>>>>
>>>> diff --git a/libdtrace/dt_peb.c b/libdtrace/dt_peb.c
>>>> index 5268f089..4983ba91 100644
>>>> --- a/libdtrace/dt_peb.c
>>>> +++ b/libdtrace/dt_peb.c
>>>> @@ -136,18 +136,14 @@ dt_pebs_exit(dtrace_hdl_t *dtp)
>>>>    }
>>>>    /*
>>>> - * Initialize the perf event buffers (one per online CPU).  Each buffer will
>>>> - * the given number of pages (i.e. the total size of each buffer will be
>>>> - * num_pages * getpagesize()).  The  allocated memory for each buffer is mmap'd
>>>> - * so the kernel can write to it, and its representative file descriptor is
>>>> - * recorded in the 'buffers' BPF map so that BPF code knows where to write
>>>> - * trace data for a specific CPU.
>>>> + * Initialize the perf event buffers, one per online CPU.  Each buffer will
>>>> + * have num_pages * getpagesize()).  The dt_peb_open() call mmaps the allocated
>>>> + * memory so the kernel can write to it.  The file descriptor is recorded in
>>>> + * the 'buffers' BPF map and added to the event polling file descriptor.
>>>>     *
>>>> - * An event polling file descriptor is created as well, and it is configured to
>>>> - * monitor all perf event buffers at once.  This file descriptor is returned
>>>> - * upon success..  Failure is indicated with a -1 return value.
>>>> + * The return value indicates success (0) or failure (-1).
>>>>     */
>>>> -int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>>>> +int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize, size_t bufsizemin)
>>>>    {
>>>>    	int		i;
>>>>    	int		mapfd;
>>>> @@ -166,12 +162,15 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>>>>    		fprintf(stderr, "bufsize increased to %lu\n",
>>>>    			num_pages * getpagesize());
>>>> +	if (num_pages * getpagesize() < bufsizemin)
>>>> +		return dt_set_errno(dtp, EDT_BUFTOOSMALL);
>>>> +
>>>>    	/*
>>>>    	 * Determine the fd for the 'buffers' BPF map.
>>>>    	 */
>>>>    	idp = dt_dlib_get_map(dtp, "buffers");
>>>>    	if (idp == NULL || idp->di_id == DT_IDENT_UNDEF)
>>>> -		return -ENOENT;
>>>> +		return dt_set_errno(dtp, EDT_NOMEM);    // FIXME we used to do something more akin to ENOENT
>>>>    	mapfd = idp->di_id;
>>>> @@ -180,7 +179,7 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>>>>    	 */
>>>>    	dtp->dt_pebset = dt_zalloc(dtp, sizeof(dt_pebset_t));
>>>>    	if (dtp->dt_pebset == NULL)
>>>> -		return -ENOMEM;
>>>> +		return dt_set_errno(dtp, EDT_NOMEM);
>>>>    	/*
>>>>    	 * Allocate the per-CPU perf event buffers.
>>>> @@ -189,7 +188,7 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>>>>    			 sizeof(struct dt_peb));
>>>>    	if (pebs == NULL) {
>>>>    		dt_free(dtp, dtp->dt_pebset);
>>>> -		return -ENOMEM;
>>>> +		return dt_set_errno(dtp, EDT_NOMEM);
>>>>    	}
>>>>    	dtp->dt_pebset->pebs = pebs;
>>>> @@ -241,5 +240,5 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
>>>>    fail:
>>>>    	dt_pebs_exit(dtp);
>>>> -	return -1;
>>>> +	return dt_set_errno(dtp, EDT_NOMEM);  // FIXME need something better
>>>>    }
>>>> diff --git a/libdtrace/dt_peb.h b/libdtrace/dt_peb.h
>>>> index e0f408f2..9ad23252 100644
>>>> --- a/libdtrace/dt_peb.h
>>>> +++ b/libdtrace/dt_peb.h
>>>> @@ -43,7 +43,7 @@ typedef struct dt_pebset {
>>>>    } dt_pebset_t;
>>>>    extern void dt_pebs_exit(dtrace_hdl_t *);
>>>> -extern int dt_pebs_init(dtrace_hdl_t *, size_t);
>>>> +extern int dt_pebs_init(dtrace_hdl_t *, size_t, size_t);
>>>>    #ifdef	__cplusplus
>>>>    }
>>>> diff --git a/libdtrace/dt_work.c b/libdtrace/dt_work.c
>>>> index 69a86358..7a0eb1da 100644
>>>> --- a/libdtrace/dt_work.c
>>>> +++ b/libdtrace/dt_work.c
>>>> @@ -267,18 +267,21 @@ dtrace_go(dtrace_hdl_t *dtp, uint_t cflags)
>>>>    		return dt_set_errno(dtp, errno);
>>>>    	/*
>>>> -	 * We need enough space for the pref_event_header, a 32-bit size, a
>>>> +	 * We need enough space for the perf_event_header, a 32-bit size, a
>>>>    	 * 4-byte gap, and the largest trace data record we may be writing to
>>>>    	 * the buffer.  In other words, the buffer needs to be large enough to
>>>>    	 * hold at least one perf-encapsulated trace data record.
>>>> +	 *
>>>> +	 * While dt_pebs_init() rounds the requested size up, size==0 is a
>>>> +	 * special case.
>>>>    	 */
>>>>    	dtrace_getopt(dtp, "bufsize", &size);
>>>> -	if (size == 0 ||
>>>> -	    size < sizeof(struct perf_event_header) + sizeof(uint32_t) +
>>>> -		   dtp->dt_maxreclen)
>>>> +	if (size == 0)
>>>>    		return dt_set_errno(dtp, EDT_BUFTOOSMALL);
>>>> -	if (dt_pebs_init(dtp, size) == -1)
>>>> -		return dt_set_errno(dtp, EDT_NOMEM);
>>>> +	if (dt_pebs_init(dtp, size,
>>>> +			 sizeof(struct perf_event_header) + sizeof(uint32_t)
>>>> +			 + 4 + dtp->dt_maxreclen) == -1)
>>>> +		return -1;
>>>>    	/*
>>>>    	 * We must initialize the aggregation consumer handling before we
>>>> diff --git a/test/unittest/options/err.b-too-low.d b/test/unittest/options/err.b-too-low.d
>>>> index bb77e37c..f62155dd 100644
>>>> --- a/test/unittest/options/err.b-too-low.d
>>>> +++ b/test/unittest/options/err.b-too-low.d
>>>> @@ -1,6 +1,6 @@
>>>>    /*
>>>>     * Oracle Linux DTrace.
>>>> - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
>>>> + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
>>>>     * Licensed under the Universal Permissive License v 1.0 as shown at
>>>>     * http://oss.oracle.com/licenses/upl.
>>>>     */
>>>> @@ -12,19 +12,21 @@
>>>>     */
>>>>    /*
>>>> - * We use a buffer size of 59 because that should be just too small to hold the
>>>> - * trace records generated in this script:
>>>> - *	- perf_event_header (40 bytes)
>>>> - *	- size (4 bytes)
>>>> - *	- gap (4 bytes)
>>>> - *	- EPID (4 bytes)
>>>> - *	- tag (4 bytes)
>>>> - *	- exit value (4 bytes)
>>>> + * We need over 4k bytes for the 4 string trace records generated in this script
>>>> + * plus some meta data.
>>>> + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
>>>> + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
>>>>     */
>>>> -/* @@runtest-opts: -b59 */
>>>> +/* @@runtest-opts: -b4096 */
>>>> +
>>>> +#pragma D option strsize=1024
>>>>    BEGIN
>>>>    {
>>>> +	trace("abc");
>>>> +	trace("def");
>>>> +	trace("ghi");
>>>> +	trace("jkl");
>>>>    	exit(0);
>>>>    }
>>>> diff --git a/test/unittest/options/err.bufsize-too-low.d b/test/unittest/options/err.bufsize-too-low.d
>>>> index bbbdb5c5..25efdf72 100644
>>>> --- a/test/unittest/options/err.bufsize-too-low.d
>>>> +++ b/test/unittest/options/err.bufsize-too-low.d
>>>> @@ -1,6 +1,6 @@
>>>>    /*
>>>>     * Oracle Linux DTrace.
>>>> - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
>>>> + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
>>>>     * Licensed under the Universal Permissive License v 1.0 as shown at
>>>>     * http://oss.oracle.com/licenses/upl.
>>>>     */
>>>> @@ -12,19 +12,21 @@
>>>>     */
>>>>    /*
>>>> - * We use a buffer size of 59 because that should be just too small to hold the
>>>> - * trace records generated in this script:
>>>> - *	- perf_event_header (40 bytes)
>>>> - *	- size (4 bytes)
>>>> - *	- gap (4 bytes)
>>>> - *	- EPID (4 bytes)
>>>> - *	- tag (4 bytes)
>>>> - *	- exit value (4 bytes)
>>>> + * We need over 4k bytes for the 4 string trace records generated in this script
>>>> + * plus some meta data.
>>>> + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
>>>> + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
>>>>     */
>>>> -/* @@runtest-opts: -xbufsize=59 */
>>>> +/* @@runtest-opts: -xbufsize=4096 */
>>>> +
>>>> +#pragma D option strsize=1024
>>>>    BEGIN
>>>>    {
>>>> +	trace("abc");
>>>> +	trace("def");
>>>> +	trace("ghi");
>>>> +	trace("jkl");
>>>>    	exit(0);
>>>>    }
>>>> diff --git a/test/unittest/options/tst.b.d b/test/unittest/options/tst.b.d
>>>> index 57fa030d..3bf08edc 100644
>>>> --- a/test/unittest/options/tst.b.d
>>>> +++ b/test/unittest/options/tst.b.d
>>>> @@ -1,6 +1,6 @@
>>>>    /*
>>>>     * Oracle Linux DTrace.
>>>> - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
>>>> + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
>>>>     * Licensed under the Universal Permissive License v 1.0 as shown at
>>>>     * http://oss.oracle.com/licenses/upl.
>>>>     */
>>>> @@ -12,19 +12,21 @@
>>>>     */
>>>>    /*
>>>> - * We use a buffer size of 60 because that should be the exact size necessary
>>>> - * to hold the trace records generated in this script:
>>>> - *	- perf_event_header (40 bytes)
>>>> - *	- size (4 bytes)
>>>> - *	- gap (4 bytes)
>>>> - *	- EPID (4 bytes)
>>>> - *	- tag (4 bytes)
>>>> - *	- exit value (4 bytes)
>>>> + * We need over 4k bytes for the 4 string trace records generated in this script
>>>> + * plus some meta data.
>>>> + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
>>>> + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
>>>>     */
>>>> -/* @@runtest-opts: -b60 */
>>>> +/* @@runtest-opts: -b4097 */
>>>> +
>>>> +#pragma D option strsize=1024
>>>>    BEGIN
>>>>    {
>>>> +	trace("abc");
>>>> +	trace("def");
>>>> +	trace("ghi");
>>>> +	trace("jkl");
>>>>    	exit(0);
>>>>    }
>>>> diff --git a/test/unittest/options/tst.bufsize.d b/test/unittest/options/tst.bufsize.d
>>>> index 96b0f1b8..23af81aa 100644
>>>> --- a/test/unittest/options/tst.bufsize.d
>>>> +++ b/test/unittest/options/tst.bufsize.d
>>>> @@ -1,6 +1,6 @@
>>>>    /*
>>>>     * Oracle Linux DTrace.
>>>> - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
>>>> + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
>>>>     * Licensed under the Universal Permissive License v 1.0 as shown at
>>>>     * http://oss.oracle.com/licenses/upl.
>>>>     */
>>>> @@ -12,19 +12,21 @@
>>>>     */
>>>>    /*
>>>> - * We use a buffer size of 60 because that should be the exact size necessary
>>>> - * to hold the trace records generated in this script:
>>>> - *	- perf_event_header (40 bytes)
>>>> - *	- size (4 bytes)
>>>> - *	- gap (4 bytes)
>>>> - *	- EPID (4 bytes)
>>>> - *	- tag (4 bytes)
>>>> - *	- exit value (4 bytes)
>>>> + * We need over 4k bytes for the 4 string trace records generated in this script
>>>> + * plus some meta data.
>>>> + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
>>>> + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
>>>>     */
>>>> -/* @@runtest-opts: -xbufsize=60 */
>>>> +/* @@runtest-opts: -xbufsize=4097 */
>>>> +
>>>> +#pragma D option strsize=1024
>>>>    BEGIN
>>>>    {
>>>> +	trace("abc");
>>>> +	trace("def");
>>>> +	trace("ghi");
>>>> +	trace("jkl");
>>>>    	exit(0);
>>>>    }
>>>> -- 
>>>> 2.18.4
>>>>

Re: [PATCH 31/38] Fix dt_pebs_init() call

[Kris Van Hees]

On Wed, Aug 28, 2024 at 04:57:43PM -0400, Eugene Loh wrote:
> On 8/26/24 12:20, Kris Van Hees wrote:
> 
> > On Mon, Aug 26, 2024 at 11:42:25AM -0400, Eugene Loh wrote:
> > > On 8/26/24 10:30, Kris Van Hees wrote:
> > > 
> > > > This patch goes against the intent of the code, as far as I can see.  The
> > > > point of performing the size check prior to increasing the buffer size was
> > > > to enforce the limit on the size that was explicit set by the user.  Then,
> > > > if that size was sufficient, then DTrace is at liberty to increase that
> > > > size to a whole multiple of the page size.
> > > > 
> > > > But if I were to insist that a buffer can only be 24 bytes, and 24 bytes are
> > > > not enough to store a single trace record, DTrace should report an error
> > > > rather than increasing my bufsize setting on its own.
> > > 
> > > The point is that DTrace *does* increase my bufsize setting on its own.  Why
> > > should DTrace complain about my setting when it's going to ignore my setting
> > > anyhow?  What value does the pre-existing behavior offer to DTrace users?
> > > Are you saying that DTrace should not resize?  That it should honor the
> > > user's bufsize?  No more resizing (to page size or whatever it wants)?
> > The main point of this code was to retain the bevhaviour that DTrace provided,
> 
> As far as adhering to legacy behavior... quite simply, we do not. We do not
> set the "principal buffer size" to bufsize.  Further, any legacy resizing
> behavior is, if I understand correctly, to go smaller than the user
> requested;  we do the opposite.  Looking to the legacy behavior for
> precedent is pointless.  The situation is that we are unlike legacy behavior
> in fundamental respects, opposite in another respect (automatic resizing),
> but consistent in some impractical and useless respect.  That consistency
> strikes me as small consolation.

You can look at this from different angles, yes.

> There is no documentation to change;  the documentation is already wrong. 
> This patch was intended as a step to making the behavior something that was
> describable.  Clearly, trying to hold on to a legacy buffering model has
> produced a byzantine situation in the current port, which treats buffering
> very differently.

Yes, but if you want to go that route we ought to go all the way, and not some
interim solution (see below).

> Perhaps we simply disagree on those points, and this patch is DOA. But the
> patch points out other problems as well.  E.g., the min computation wasn't
> even right and error checking was wrong.

Let's do this as two patches:
 - one that ensures that the computation is updated to the current
   implementation changes and is correct
 - one that provides for increasing the bufsize to be the smallest multiple of
   pagesize to fit the largest trace record needed for the tracing session

I don't think that checking the rounded up bufsize against the minimum size
makes any sense if you are going to increase the specified bufsize anyway.
Might as well increase it to fit the minimum size altogether.

> > which is that if a user explicitly requests a bufsize that is less than the
> > minimum required for the given trace programs, an error is reported.  That is
> > a mater of being consistent (even if it has no practical application that I
> > can see).  Tests from the existing version failed because this was not being
> > done right.
> > 
> > As far as functionality, as long as the bufsize is large enough to hold the
> > largest trace record, DTrace can (and needs to) ensure that the buffer size
> > is a whole multiple of the page size because that is required by the perf
> > ring buffer imlementation.
> > 
> > So, DTrace only increases the buffer size to the required multiple of page size
> > if the user supplied bufsize is at least large enough to hold the largest trace
> > record (or if no bufsize was explicitly specified).
> > 
> > We could opt to change the behaviour of the bufsize opton to always round up to
> > a whole multiple of page size, but that is a deviation from the original
> > implementation and thus is something that would need to be documented, etc...
> > 
> > Aside from the computation of the minimum required size needing updating when
> > patches (e.g. the USDT changes) modify the layout and size of the trace records,
> > I do not think that the existing code is wrong.
> > 
> > I am certainly open to a feature change to make bufsize always be rounded up as
> > needed, but that is not a bug fix and I think that ought to be its own patch
> > (along with documentation changes etc).  And it might need some more discussion
> > because it brings up the question why we wouldn't just simply always increase
> > bufsize to be the smallest multiple of the page size to accomodate the largest
> > trace record.
> > 
> > > > On Thu, Jun 27, 2024 at 01:38:57AM -0400, eugene.loh@oracle.com wrote:
> > > > > From: Eugene Loh <eugene.loh@oracle.com>
> > > > > 
> > > > > The function had a few issues, mainly with its comments,
> > > > > ranging from typos to incorrect descriptions of behavior.
> > > > > 
> > > > > The function has only one caller.  The enforcement of a
> > > > > size minimum was problematic in a number of respects:
> > > > > - it was missing 4 bytes
> > > > > - it was enforcing the minimum before the size was
> > > > >       increased by the caller
> > > > > - it was checking dt_pebs_init()==-1,
> > > > >       missing errors with other negative return values
> > > > > 
> > > > > So change the function to accept a minimum and change its
> > > > > return values.
> > > > > 
> > > > > Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> > > > > ---
> > > > >    libdtrace/dt_peb.c                          | 27 ++++++++++-----------
> > > > >    libdtrace/dt_peb.h                          |  2 +-
> > > > >    libdtrace/dt_work.c                         | 15 +++++++-----
> > > > >    test/unittest/options/err.b-too-low.d       | 22 +++++++++--------
> > > > >    test/unittest/options/err.bufsize-too-low.d | 22 +++++++++--------
> > > > >    test/unittest/options/tst.b.d               | 22 +++++++++--------
> > > > >    test/unittest/options/tst.bufsize.d         | 22 +++++++++--------
> > > > >    7 files changed, 71 insertions(+), 61 deletions(-)
> > > > > 
> > > > > diff --git a/libdtrace/dt_peb.c b/libdtrace/dt_peb.c
> > > > > index 5268f089..4983ba91 100644
> > > > > --- a/libdtrace/dt_peb.c
> > > > > +++ b/libdtrace/dt_peb.c
> > > > > @@ -136,18 +136,14 @@ dt_pebs_exit(dtrace_hdl_t *dtp)
> > > > >    }
> > > > >    /*
> > > > > - * Initialize the perf event buffers (one per online CPU).  Each buffer will
> > > > > - * the given number of pages (i.e. the total size of each buffer will be
> > > > > - * num_pages * getpagesize()).  The  allocated memory for each buffer is mmap'd
> > > > > - * so the kernel can write to it, and its representative file descriptor is
> > > > > - * recorded in the 'buffers' BPF map so that BPF code knows where to write
> > > > > - * trace data for a specific CPU.
> > > > > + * Initialize the perf event buffers, one per online CPU.  Each buffer will
> > > > > + * have num_pages * getpagesize()).  The dt_peb_open() call mmaps the allocated
> > > > > + * memory so the kernel can write to it.  The file descriptor is recorded in
> > > > > + * the 'buffers' BPF map and added to the event polling file descriptor.
> > > > >     *
> > > > > - * An event polling file descriptor is created as well, and it is configured to
> > > > > - * monitor all perf event buffers at once.  This file descriptor is returned
> > > > > - * upon success..  Failure is indicated with a -1 return value.
> > > > > + * The return value indicates success (0) or failure (-1).
> > > > >     */
> > > > > -int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
> > > > > +int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize, size_t bufsizemin)
> > > > >    {
> > > > >    	int		i;
> > > > >    	int		mapfd;
> > > > > @@ -166,12 +162,15 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
> > > > >    		fprintf(stderr, "bufsize increased to %lu\n",
> > > > >    			num_pages * getpagesize());
> > > > > +	if (num_pages * getpagesize() < bufsizemin)
> > > > > +		return dt_set_errno(dtp, EDT_BUFTOOSMALL);
> > > > > +
> > > > >    	/*
> > > > >    	 * Determine the fd for the 'buffers' BPF map.
> > > > >    	 */
> > > > >    	idp = dt_dlib_get_map(dtp, "buffers");
> > > > >    	if (idp == NULL || idp->di_id == DT_IDENT_UNDEF)
> > > > > -		return -ENOENT;
> > > > > +		return dt_set_errno(dtp, EDT_NOMEM);    // FIXME we used to do something more akin to ENOENT
> > > > >    	mapfd = idp->di_id;
> > > > > @@ -180,7 +179,7 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
> > > > >    	 */
> > > > >    	dtp->dt_pebset = dt_zalloc(dtp, sizeof(dt_pebset_t));
> > > > >    	if (dtp->dt_pebset == NULL)
> > > > > -		return -ENOMEM;
> > > > > +		return dt_set_errno(dtp, EDT_NOMEM);
> > > > >    	/*
> > > > >    	 * Allocate the per-CPU perf event buffers.
> > > > > @@ -189,7 +188,7 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
> > > > >    			 sizeof(struct dt_peb));
> > > > >    	if (pebs == NULL) {
> > > > >    		dt_free(dtp, dtp->dt_pebset);
> > > > > -		return -ENOMEM;
> > > > > +		return dt_set_errno(dtp, EDT_NOMEM);
> > > > >    	}
> > > > >    	dtp->dt_pebset->pebs = pebs;
> > > > > @@ -241,5 +240,5 @@ int dt_pebs_init(dtrace_hdl_t *dtp, size_t bufsize)
> > > > >    fail:
> > > > >    	dt_pebs_exit(dtp);
> > > > > -	return -1;
> > > > > +	return dt_set_errno(dtp, EDT_NOMEM);  // FIXME need something better
> > > > >    }
> > > > > diff --git a/libdtrace/dt_peb.h b/libdtrace/dt_peb.h
> > > > > index e0f408f2..9ad23252 100644
> > > > > --- a/libdtrace/dt_peb.h
> > > > > +++ b/libdtrace/dt_peb.h
> > > > > @@ -43,7 +43,7 @@ typedef struct dt_pebset {
> > > > >    } dt_pebset_t;
> > > > >    extern void dt_pebs_exit(dtrace_hdl_t *);
> > > > > -extern int dt_pebs_init(dtrace_hdl_t *, size_t);
> > > > > +extern int dt_pebs_init(dtrace_hdl_t *, size_t, size_t);
> > > > >    #ifdef	__cplusplus
> > > > >    }
> > > > > diff --git a/libdtrace/dt_work.c b/libdtrace/dt_work.c
> > > > > index 69a86358..7a0eb1da 100644
> > > > > --- a/libdtrace/dt_work.c
> > > > > +++ b/libdtrace/dt_work.c
> > > > > @@ -267,18 +267,21 @@ dtrace_go(dtrace_hdl_t *dtp, uint_t cflags)
> > > > >    		return dt_set_errno(dtp, errno);
> > > > >    	/*
> > > > > -	 * We need enough space for the pref_event_header, a 32-bit size, a
> > > > > +	 * We need enough space for the perf_event_header, a 32-bit size, a
> > > > >    	 * 4-byte gap, and the largest trace data record we may be writing to
> > > > >    	 * the buffer.  In other words, the buffer needs to be large enough to
> > > > >    	 * hold at least one perf-encapsulated trace data record.
> > > > > +	 *
> > > > > +	 * While dt_pebs_init() rounds the requested size up, size==0 is a
> > > > > +	 * special case.
> > > > >    	 */
> > > > >    	dtrace_getopt(dtp, "bufsize", &size);
> > > > > -	if (size == 0 ||
> > > > > -	    size < sizeof(struct perf_event_header) + sizeof(uint32_t) +
> > > > > -		   dtp->dt_maxreclen)
> > > > > +	if (size == 0)
> > > > >    		return dt_set_errno(dtp, EDT_BUFTOOSMALL);
> > > > > -	if (dt_pebs_init(dtp, size) == -1)
> > > > > -		return dt_set_errno(dtp, EDT_NOMEM);
> > > > > +	if (dt_pebs_init(dtp, size,
> > > > > +			 sizeof(struct perf_event_header) + sizeof(uint32_t)
> > > > > +			 + 4 + dtp->dt_maxreclen) == -1)
> > > > > +		return -1;
> > > > >    	/*
> > > > >    	 * We must initialize the aggregation consumer handling before we
> > > > > diff --git a/test/unittest/options/err.b-too-low.d b/test/unittest/options/err.b-too-low.d
> > > > > index bb77e37c..f62155dd 100644
> > > > > --- a/test/unittest/options/err.b-too-low.d
> > > > > +++ b/test/unittest/options/err.b-too-low.d
> > > > > @@ -1,6 +1,6 @@
> > > > >    /*
> > > > >     * Oracle Linux DTrace.
> > > > > - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
> > > > > + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
> > > > >     * Licensed under the Universal Permissive License v 1.0 as shown at
> > > > >     * http://oss.oracle.com/licenses/upl.
> > > > >     */
> > > > > @@ -12,19 +12,21 @@
> > > > >     */
> > > > >    /*
> > > > > - * We use a buffer size of 59 because that should be just too small to hold the
> > > > > - * trace records generated in this script:
> > > > > - *	- perf_event_header (40 bytes)
> > > > > - *	- size (4 bytes)
> > > > > - *	- gap (4 bytes)
> > > > > - *	- EPID (4 bytes)
> > > > > - *	- tag (4 bytes)
> > > > > - *	- exit value (4 bytes)
> > > > > + * We need over 4k bytes for the 4 string trace records generated in this script
> > > > > + * plus some meta data.
> > > > > + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
> > > > > + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
> > > > >     */
> > > > > -/* @@runtest-opts: -b59 */
> > > > > +/* @@runtest-opts: -b4096 */
> > > > > +
> > > > > +#pragma D option strsize=1024
> > > > >    BEGIN
> > > > >    {
> > > > > +	trace("abc");
> > > > > +	trace("def");
> > > > > +	trace("ghi");
> > > > > +	trace("jkl");
> > > > >    	exit(0);
> > > > >    }
> > > > > diff --git a/test/unittest/options/err.bufsize-too-low.d b/test/unittest/options/err.bufsize-too-low.d
> > > > > index bbbdb5c5..25efdf72 100644
> > > > > --- a/test/unittest/options/err.bufsize-too-low.d
> > > > > +++ b/test/unittest/options/err.bufsize-too-low.d
> > > > > @@ -1,6 +1,6 @@
> > > > >    /*
> > > > >     * Oracle Linux DTrace.
> > > > > - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
> > > > > + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
> > > > >     * Licensed under the Universal Permissive License v 1.0 as shown at
> > > > >     * http://oss.oracle.com/licenses/upl.
> > > > >     */
> > > > > @@ -12,19 +12,21 @@
> > > > >     */
> > > > >    /*
> > > > > - * We use a buffer size of 59 because that should be just too small to hold the
> > > > > - * trace records generated in this script:
> > > > > - *	- perf_event_header (40 bytes)
> > > > > - *	- size (4 bytes)
> > > > > - *	- gap (4 bytes)
> > > > > - *	- EPID (4 bytes)
> > > > > - *	- tag (4 bytes)
> > > > > - *	- exit value (4 bytes)
> > > > > + * We need over 4k bytes for the 4 string trace records generated in this script
> > > > > + * plus some meta data.
> > > > > + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
> > > > > + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
> > > > >     */
> > > > > -/* @@runtest-opts: -xbufsize=59 */
> > > > > +/* @@runtest-opts: -xbufsize=4096 */
> > > > > +
> > > > > +#pragma D option strsize=1024
> > > > >    BEGIN
> > > > >    {
> > > > > +	trace("abc");
> > > > > +	trace("def");
> > > > > +	trace("ghi");
> > > > > +	trace("jkl");
> > > > >    	exit(0);
> > > > >    }
> > > > > diff --git a/test/unittest/options/tst.b.d b/test/unittest/options/tst.b.d
> > > > > index 57fa030d..3bf08edc 100644
> > > > > --- a/test/unittest/options/tst.b.d
> > > > > +++ b/test/unittest/options/tst.b.d
> > > > > @@ -1,6 +1,6 @@
> > > > >    /*
> > > > >     * Oracle Linux DTrace.
> > > > > - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
> > > > > + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
> > > > >     * Licensed under the Universal Permissive License v 1.0 as shown at
> > > > >     * http://oss.oracle.com/licenses/upl.
> > > > >     */
> > > > > @@ -12,19 +12,21 @@
> > > > >     */
> > > > >    /*
> > > > > - * We use a buffer size of 60 because that should be the exact size necessary
> > > > > - * to hold the trace records generated in this script:
> > > > > - *	- perf_event_header (40 bytes)
> > > > > - *	- size (4 bytes)
> > > > > - *	- gap (4 bytes)
> > > > > - *	- EPID (4 bytes)
> > > > > - *	- tag (4 bytes)
> > > > > - *	- exit value (4 bytes)
> > > > > + * We need over 4k bytes for the 4 string trace records generated in this script
> > > > > + * plus some meta data.
> > > > > + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
> > > > > + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
> > > > >     */
> > > > > -/* @@runtest-opts: -b60 */
> > > > > +/* @@runtest-opts: -b4097 */
> > > > > +
> > > > > +#pragma D option strsize=1024
> > > > >    BEGIN
> > > > >    {
> > > > > +	trace("abc");
> > > > > +	trace("def");
> > > > > +	trace("ghi");
> > > > > +	trace("jkl");
> > > > >    	exit(0);
> > > > >    }
> > > > > diff --git a/test/unittest/options/tst.bufsize.d b/test/unittest/options/tst.bufsize.d
> > > > > index 96b0f1b8..23af81aa 100644
> > > > > --- a/test/unittest/options/tst.bufsize.d
> > > > > +++ b/test/unittest/options/tst.bufsize.d
> > > > > @@ -1,6 +1,6 @@
> > > > >    /*
> > > > >     * Oracle Linux DTrace.
> > > > > - * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
> > > > > + * Copyright (c) 2022, 2024, Oracle and/or its affiliates. All rights reserved.
> > > > >     * Licensed under the Universal Permissive License v 1.0 as shown at
> > > > >     * http://oss.oracle.com/licenses/upl.
> > > > >     */
> > > > > @@ -12,19 +12,21 @@
> > > > >     */
> > > > >    /*
> > > > > - * We use a buffer size of 60 because that should be the exact size necessary
> > > > > - * to hold the trace records generated in this script:
> > > > > - *	- perf_event_header (40 bytes)
> > > > > - *	- size (4 bytes)
> > > > > - *	- gap (4 bytes)
> > > > > - *	- EPID (4 bytes)
> > > > > - *	- tag (4 bytes)
> > > > > - *	- exit value (4 bytes)
> > > > > + * We need over 4k bytes for the 4 string trace records generated in this script
> > > > > + * plus some meta data.
> > > > > + * A bufsize of 1 to 4096 bytes is rounded up to 4096 and is insufficient.
> > > > > + * A bufsize of 4097 to 8192 bytes is rounded up to 8192 and is sufficient.
> > > > >     */
> > > > > -/* @@runtest-opts: -xbufsize=60 */
> > > > > +/* @@runtest-opts: -xbufsize=4097 */
> > > > > +
> > > > > +#pragma D option strsize=1024
> > > > >    BEGIN
> > > > >    {
> > > > > +	trace("abc");
> > > > > +	trace("def");
> > > > > +	trace("ghi");
> > > > > +	trace("jkl");
> > > > >    	exit(0);
> > > > >    }
> > > > > -- 
> > > > > 2.18.4
> > > > > 

Re: [PATCH 31/38] Fix dt_pebs_init() call

[Eugene Loh]

On 8/28/24 17:16, Kris Van Hees wrote:

> On Wed, Aug 28, 2024 at 04:57:43PM -0400, Eugene Loh wrote:
>> On 8/26/24 12:20, Kris Van Hees wrote:
>>> On Mon, Aug 26, 2024 at 11:42:25AM -0400, Eugene Loh wrote:
>>>> On 8/26/24 10:30, Kris Van Hees wrote:
>>>>> This patch goes against the intent of the code, as far as I can see.  The
>>>>> point of performing the size check prior to increasing the buffer size was
>>>>> to enforce the limit on the size that was explicit set by the user.  Then,
>>>>> if that size was sufficient, then DTrace is at liberty to increase that
>>>>> size to a whole multiple of the page size.
>>>>>
>>>>> But if I were to insist that a buffer can only be 24 bytes, and 24 bytes are
>>>>> not enough to store a single trace record, DTrace should report an error
>>>>> rather than increasing my bufsize setting on its own.
>>>> The point is that DTrace *does* increase my bufsize setting on its own.  Why
>>>> should DTrace complain about my setting when it's going to ignore my setting
>>>> anyhow?  What value does the pre-existing behavior offer to DTrace users?
>>>> Are you saying that DTrace should not resize?  That it should honor the
>>>> user's bufsize?  No more resizing (to page size or whatever it wants)?
>>> The main point of this code was to retain the bevhaviour that DTrace provided,
>> As far as adhering to legacy behavior... quite simply, we do not. We do not
>> set the "principal buffer size" to bufsize.  Further, any legacy resizing
>> behavior is, if I understand correctly, to go smaller than the user
>> requested;  we do the opposite.  Looking to the legacy behavior for
>> precedent is pointless.  The situation is that we are unlike legacy behavior
>> in fundamental respects, opposite in another respect (automatic resizing),
>> but consistent in some impractical and useless respect.  That consistency
>> strikes me as small consolation.
> You can look at this from different angles, yes.
>
>> There is no documentation to change;  the documentation is already wrong.
>> This patch was intended as a step to making the behavior something that was
>> describable.  Clearly, trying to hold on to a legacy buffering model has
>> produced a byzantine situation in the current port, which treats buffering
>> very differently.
> Yes, but if you want to go that route we ought to go all the way, and not some
> interim solution (see below).
>
>> Perhaps we simply disagree on those points, and this patch is DOA. But the
>> patch points out other problems as well.  E.g., the min computation wasn't
>> even right and error checking was wrong.
> Let's do this as two patches:
>   - one that ensures that the computation is updated to the current
>     implementation changes and is correct
>   - one that provides for increasing the bufsize to be the smallest multiple of
>     pagesize to fit the largest trace record needed for the tracing session
>
> I don't think that checking the rounded up bufsize against the minimum size
> makes any sense if you are going to increase the specified bufsize anyway.
> Might as well increase it to fit the minimum size altogether.

I'd like to understand this proposal.  Are you saying:

*)  Determine the min size (correctly).

*)  Round that up per perf_event requirements (2^n + 1 pages).

*)  Ignore the user's bufsize setting.

I'm fine with that (given that we do buffering so unlike the legacy 
implementation and will not be honoring the user's bufsize value 
anyhow), but I just wanted to be explicit about ignoring the user's bufsize.

>>> which is that if a user explicitly requests a bufsize that is less than the
>>> minimum required for the given trace programs, an error is reported.  That is
>>> a mater of being consistent (even if it has no practical application that I
>>> can see).  Tests from the existing version failed because this was not being
>>> done right.
>>>
>>> As far as functionality, as long as the bufsize is large enough to hold the
>>> largest trace record, DTrace can (and needs to) ensure that the buffer size
>>> is a whole multiple of the page size because that is required by the perf
>>> ring buffer imlementation.
>>>
>>> So, DTrace only increases the buffer size to the required multiple of page size
>>> if the user supplied bufsize is at least large enough to hold the largest trace
>>> record (or if no bufsize was explicitly specified).
>>>
>>> We could opt to change the behaviour of the bufsize opton to always round up to
>>> a whole multiple of page size, but that is a deviation from the original
>>> implementation and thus is something that would need to be documented, etc...
>>>
>>> Aside from the computation of the minimum required size needing updating when
>>> patches (e.g. the USDT changes) modify the layout and size of the trace records,
>>> I do not think that the existing code is wrong.
>>>
>>> I am certainly open to a feature change to make bufsize always be rounded up as
>>> needed, but that is not a bug fix and I think that ought to be its own patch
>>> (along with documentation changes etc).  And it might need some more discussion
>>> because it brings up the question why we wouldn't just simply always increase
>>> bufsize to be the smallest multiple of the page size to accomodate the largest
>>> trace record.
>>>
>>>>> On Thu, Jun 27, 2024 at 01:38:57AM -0400, eugene.loh@oracle.com wrote:
>>>>>> From: Eugene Loh <eugene.loh@oracle.com>
>>>>>>
>>>>>> The function had a few issues, mainly with its comments,
>>>>>> ranging from typos to incorrect descriptions of behavior.
>>>>>>
>>>>>> The function has only one caller.  The enforcement of a
>>>>>> size minimum was problematic in a number of respects:
>>>>>> - it was missing 4 bytes
>>>>>> - it was enforcing the minimum before the size was
>>>>>>        increased by the caller
>>>>>> - it was checking dt_pebs_init()==-1,
>>>>>>        missing errors with other negative return values
>>>>>>
>>>>>> So change the function to accept a minimum and change its
>>>>>> return values.

Re: [DTrace-devel] [PATCH 31/38] Fix dt_pebs_init() call

[Kris Van Hees]

On Thu, Aug 29, 2024 at 08:54:46PM -0400, Eugene Loh via DTrace-devel wrote:
> On 8/28/24 17:16, Kris Van Hees wrote:
> 
> > On Wed, Aug 28, 2024 at 04:57:43PM -0400, Eugene Loh wrote:
> > > On 8/26/24 12:20, Kris Van Hees wrote:
> > > > On Mon, Aug 26, 2024 at 11:42:25AM -0400, Eugene Loh wrote:
> > > > > On 8/26/24 10:30, Kris Van Hees wrote:
> > > > > > This patch goes against the intent of the code, as far as I can see.  The
> > > > > > point of performing the size check prior to increasing the buffer size was
> > > > > > to enforce the limit on the size that was explicit set by the user.  Then,
> > > > > > if that size was sufficient, then DTrace is at liberty to increase that
> > > > > > size to a whole multiple of the page size.
> > > > > > 
> > > > > > But if I were to insist that a buffer can only be 24 bytes, and 24 bytes are
> > > > > > not enough to store a single trace record, DTrace should report an error
> > > > > > rather than increasing my bufsize setting on its own.
> > > > > The point is that DTrace *does* increase my bufsize setting on its own.  Why
> > > > > should DTrace complain about my setting when it's going to ignore my setting
> > > > > anyhow?  What value does the pre-existing behavior offer to DTrace users?
> > > > > Are you saying that DTrace should not resize?  That it should honor the
> > > > > user's bufsize?  No more resizing (to page size or whatever it wants)?
> > > > The main point of this code was to retain the bevhaviour that DTrace provided,
> > > As far as adhering to legacy behavior... quite simply, we do not. We do not
> > > set the "principal buffer size" to bufsize.  Further, any legacy resizing
> > > behavior is, if I understand correctly, to go smaller than the user
> > > requested;  we do the opposite.  Looking to the legacy behavior for
> > > precedent is pointless.  The situation is that we are unlike legacy behavior
> > > in fundamental respects, opposite in another respect (automatic resizing),
> > > but consistent in some impractical and useless respect.  That consistency
> > > strikes me as small consolation.
> > You can look at this from different angles, yes.
> > 
> > > There is no documentation to change;  the documentation is already wrong.
> > > This patch was intended as a step to making the behavior something that was
> > > describable.  Clearly, trying to hold on to a legacy buffering model has
> > > produced a byzantine situation in the current port, which treats buffering
> > > very differently.
> > Yes, but if you want to go that route we ought to go all the way, and not some
> > interim solution (see below).
> > 
> > > Perhaps we simply disagree on those points, and this patch is DOA. But the
> > > patch points out other problems as well.  E.g., the min computation wasn't
> > > even right and error checking was wrong.
> > Let's do this as two patches:
> >   - one that ensures that the computation is updated to the current
> >     implementation changes and is correct
> >   - one that provides for increasing the bufsize to be the smallest multiple of
> >     pagesize to fit the largest trace record needed for the tracing session
> > 
> > I don't think that checking the rounded up bufsize against the minimum size
> > makes any sense if you are going to increase the specified bufsize anyway.
> > Might as well increase it to fit the minimum size altogether.
> 
> I'd like to understand this proposal.  Are you saying:
> 
> *)  Determine the min size (correctly).

Correct.

> *)  Round that up per perf_event requirements (2^n + 1 pages).

No, round up to nearest 2^n.  The extra page is allocated automatically by
dt_peb_open().

> *)  Ignore the user's bufsize setting.

Well, no.  The bufsize setting is not ignored.  We just change the semantics
a little (in view of the new implementation) to be a mechanism to allow the
user to tune the buffer size, but within certain limitations.

> I'm fine with that (given that we do buffering so unlike the legacy
> implementation and will not be honoring the user's bufsize value anyhow),
> but I just wanted to be explicit about ignoring the user's bufsize.

We do not ignore the bufsize as such.  But the extent to which the bugsize
can be tuned is limited by the implementation.  The bufsizw setting cannot be
less than the minimum requirement (and you prefer to not complain about that
and instead just increase the buffer size).  And its value must be a 2^n
multiple of the page size, so it will be rounded up to the nearest acceptable
value.  But that is therefore still controlled by the user's bufsize setting.

> > > > which is that if a user explicitly requests a bufsize that is less than the
> > > > minimum required for the given trace programs, an error is reported.  That is
> > > > a mater of being consistent (even if it has no practical application that I
> > > > can see).  Tests from the existing version failed because this was not being
> > > > done right.
> > > > 
> > > > As far as functionality, as long as the bufsize is large enough to hold the
> > > > largest trace record, DTrace can (and needs to) ensure that the buffer size
> > > > is a whole multiple of the page size because that is required by the perf
> > > > ring buffer imlementation.
> > > > 
> > > > So, DTrace only increases the buffer size to the required multiple of page size
> > > > if the user supplied bufsize is at least large enough to hold the largest trace
> > > > record (or if no bufsize was explicitly specified).
> > > > 
> > > > We could opt to change the behaviour of the bufsize opton to always round up to
> > > > a whole multiple of page size, but that is a deviation from the original
> > > > implementation and thus is something that would need to be documented, etc...
> > > > 
> > > > Aside from the computation of the minimum required size needing updating when
> > > > patches (e.g. the USDT changes) modify the layout and size of the trace records,
> > > > I do not think that the existing code is wrong.
> > > > 
> > > > I am certainly open to a feature change to make bufsize always be rounded up as
> > > > needed, but that is not a bug fix and I think that ought to be its own patch
> > > > (along with documentation changes etc).  And it might need some more discussion
> > > > because it brings up the question why we wouldn't just simply always increase
> > > > bufsize to be the smallest multiple of the page size to accomodate the largest
> > > > trace record.
> > > > 
> > > > > > On Thu, Jun 27, 2024 at 01:38:57AM -0400, eugene.loh@oracle.com wrote:
> > > > > > > From: Eugene Loh <eugene.loh@oracle.com>
> > > > > > > 
> > > > > > > The function had a few issues, mainly with its comments,
> > > > > > > ranging from typos to incorrect descriptions of behavior.
> > > > > > > 
> > > > > > > The function has only one caller.  The enforcement of a
> > > > > > > size minimum was problematic in a number of respects:
> > > > > > > - it was missing 4 bytes
> > > > > > > - it was enforcing the minimum before the size was
> > > > > > >        increased by the caller
> > > > > > > - it was checking dt_pebs_init()==-1,
> > > > > > >        missing errors with other negative return values
> > > > > > > 
> > > > > > > So change the function to accept a minimum and change its
> > > > > > > return values.
> 
> _______________________________________________
> DTrace-devel mailing list
> DTrace-devel@oss.oracle.com
> https://oss.oracle.com/mailman/listinfo/dtrace-devel

Re: [DTrace-devel] [PATCH 31/38] Fix dt_pebs_init() call

[Eugene Loh]

So do you have a proposal for what the new semantics for bufsize should 
be?  (And, why not simply ignore bufsize?)

On 8/29/24 22:26, Kris Van Hees wrote:
> On Thu, Aug 29, 2024 at 08:54:46PM -0400, Eugene Loh via DTrace-devel wrote:
>> On 8/28/24 17:16, Kris Van Hees wrote:
>>> On Wed, Aug 28, 2024 at 04:57:43PM -0400, Eugene Loh wrote:
>>>> On 8/26/24 12:20, Kris Van Hees wrote:
>>>>> On Mon, Aug 26, 2024 at 11:42:25AM -0400, Eugene Loh wrote:
>>>>>> On 8/26/24 10:30, Kris Van Hees wrote:
>>>>>>> This patch goes against the intent of the code, as far as I can see.  The
>>>>>>> point of performing the size check prior to increasing the buffer size was
>>>>>>> to enforce the limit on the size that was explicit set by the user.  Then,
>>>>>>> if that size was sufficient, then DTrace is at liberty to increase that
>>>>>>> size to a whole multiple of the page size.
>>>>>>>
>>>>>>> But if I were to insist that a buffer can only be 24 bytes, and 24 bytes are
>>>>>>> not enough to store a single trace record, DTrace should report an error
>>>>>>> rather than increasing my bufsize setting on its own.
>>>>>> The point is that DTrace *does* increase my bufsize setting on its own.  Why
>>>>>> should DTrace complain about my setting when it's going to ignore my setting
>>>>>> anyhow?  What value does the pre-existing behavior offer to DTrace users?
>>>>>> Are you saying that DTrace should not resize?  That it should honor the
>>>>>> user's bufsize?  No more resizing (to page size or whatever it wants)?
>>>>> The main point of this code was to retain the bevhaviour that DTrace provided,
>>>> As far as adhering to legacy behavior... quite simply, we do not. We do not
>>>> set the "principal buffer size" to bufsize.  Further, any legacy resizing
>>>> behavior is, if I understand correctly, to go smaller than the user
>>>> requested;  we do the opposite.  Looking to the legacy behavior for
>>>> precedent is pointless.  The situation is that we are unlike legacy behavior
>>>> in fundamental respects, opposite in another respect (automatic resizing),
>>>> but consistent in some impractical and useless respect.  That consistency
>>>> strikes me as small consolation.
>>> You can look at this from different angles, yes.
>>>
>>>> There is no documentation to change;  the documentation is already wrong.
>>>> This patch was intended as a step to making the behavior something that was
>>>> describable.  Clearly, trying to hold on to a legacy buffering model has
>>>> produced a byzantine situation in the current port, which treats buffering
>>>> very differently.
>>> Yes, but if you want to go that route we ought to go all the way, and not some
>>> interim solution (see below).
>>>
>>>> Perhaps we simply disagree on those points, and this patch is DOA. But the
>>>> patch points out other problems as well.  E.g., the min computation wasn't
>>>> even right and error checking was wrong.
>>> Let's do this as two patches:
>>>    - one that ensures that the computation is updated to the current
>>>      implementation changes and is correct
>>>    - one that provides for increasing the bufsize to be the smallest multiple of
>>>      pagesize to fit the largest trace record needed for the tracing session
>>>
>>> I don't think that checking the rounded up bufsize against the minimum size
>>> makes any sense if you are going to increase the specified bufsize anyway.
>>> Might as well increase it to fit the minimum size altogether.
>> I'd like to understand this proposal.  Are you saying:
>>
>> *)  Determine the min size (correctly).
> Correct.
>
>> *)  Round that up per perf_event requirements (2^n + 1 pages).
> No, round up to nearest 2^n.  The extra page is allocated automatically by
> dt_peb_open().
>
>> *)  Ignore the user's bufsize setting.
> Well, no.  The bufsize setting is not ignored.  We just change the semantics
> a little (in view of the new implementation) to be a mechanism to allow the
> user to tune the buffer size, but within certain limitations.
>
>> I'm fine with that (given that we do buffering so unlike the legacy
>> implementation and will not be honoring the user's bufsize value anyhow),
>> but I just wanted to be explicit about ignoring the user's bufsize.
> We do not ignore the bufsize as such.  But the extent to which the bugsize
> can be tuned is limited by the implementation.  The bufsizw setting cannot be
> less than the minimum requirement (and you prefer to not complain about that
> and instead just increase the buffer size).  And its value must be a 2^n
> multiple of the page size, so it will be rounded up to the nearest acceptable
> value.  But that is therefore still controlled by the user's bufsize setting.
>
>>>>> which is that if a user explicitly requests a bufsize that is less than the
>>>>> minimum required for the given trace programs, an error is reported.  That is
>>>>> a mater of being consistent (even if it has no practical application that I
>>>>> can see).  Tests from the existing version failed because this was not being
>>>>> done right.
>>>>>
>>>>> As far as functionality, as long as the bufsize is large enough to hold the
>>>>> largest trace record, DTrace can (and needs to) ensure that the buffer size
>>>>> is a whole multiple of the page size because that is required by the perf
>>>>> ring buffer imlementation.
>>>>>
>>>>> So, DTrace only increases the buffer size to the required multiple of page size
>>>>> if the user supplied bufsize is at least large enough to hold the largest trace
>>>>> record (or if no bufsize was explicitly specified).
>>>>>
>>>>> We could opt to change the behaviour of the bufsize opton to always round up to
>>>>> a whole multiple of page size, but that is a deviation from the original
>>>>> implementation and thus is something that would need to be documented, etc...
>>>>>
>>>>> Aside from the computation of the minimum required size needing updating when
>>>>> patches (e.g. the USDT changes) modify the layout and size of the trace records,
>>>>> I do not think that the existing code is wrong.
>>>>>
>>>>> I am certainly open to a feature change to make bufsize always be rounded up as
>>>>> needed, but that is not a bug fix and I think that ought to be its own patch
>>>>> (along with documentation changes etc).  And it might need some more discussion
>>>>> because it brings up the question why we wouldn't just simply always increase
>>>>> bufsize to be the smallest multiple of the page size to accomodate the largest
>>>>> trace record.
>>>>>
>>>>>>> On Thu, Jun 27, 2024 at 01:38:57AM -0400, eugene.loh@oracle.com wrote:
>>>>>>>> From: Eugene Loh <eugene.loh@oracle.com>
>>>>>>>>
>>>>>>>> The function had a few issues, mainly with its comments,
>>>>>>>> ranging from typos to incorrect descriptions of behavior.
>>>>>>>>
>>>>>>>> The function has only one caller.  The enforcement of a
>>>>>>>> size minimum was problematic in a number of respects:
>>>>>>>> - it was missing 4 bytes
>>>>>>>> - it was enforcing the minimum before the size was
>>>>>>>>         increased by the caller
>>>>>>>> - it was checking dt_pebs_init()==-1,
>>>>>>>>         missing errors with other negative return values
>>>>>>>>
>>>>>>>> So change the function to accept a minimum and change its
>>>>>>>> return values.
>> _______________________________________________
>> DTrace-devel mailing list
>> DTrace-devel@oss.oracle.com
>> https://oss.oracle.com/mailman/listinfo/dtrace-devel

Re: [DTrace-devel] [PATCH 31/38] Fix dt_pebs_init() call

[Kris Van Hees]

On Fri, Aug 30, 2024 at 01:42:38AM -0400, Eugene Loh wrote:
> So do you have a proposal for what the new semantics for bufsize should be? 

I believe the semantics of a user-specified bufsize should be as follows:

 - If the bufsize specified by the user is less than the minimum buffer size
   required to store the largest trace record, an error should be reported.
 - If the bufsize satisfies the minimum requirement, the implementation must
   ensure that it is round up (if needed) to satisfy the perf ring buffer
   size requirement (size needs to be 2^n * pagesize, for n = 0, 1, 2, ...)

Point 1 is clearly debatable but follows the principle that you cannot put
something bigger into something that is too small to hold it.

> (And, why not simply ignore bufsize?)

Because we want users to be able to tune the size of the tracing buffer, which
can affect performance in terms of how quickly trace data is generated versus
how quiickly it is consumed.  It interacts with things like switchrate.

> On 8/29/24 22:26, Kris Van Hees wrote:
> > On Thu, Aug 29, 2024 at 08:54:46PM -0400, Eugene Loh via DTrace-devel wrote:
> > > On 8/28/24 17:16, Kris Van Hees wrote:
> > > > On Wed, Aug 28, 2024 at 04:57:43PM -0400, Eugene Loh wrote:
> > > > > On 8/26/24 12:20, Kris Van Hees wrote:
> > > > > > On Mon, Aug 26, 2024 at 11:42:25AM -0400, Eugene Loh wrote:
> > > > > > > On 8/26/24 10:30, Kris Van Hees wrote:
> > > > > > > > This patch goes against the intent of the code, as far as I can see.  The
> > > > > > > > point of performing the size check prior to increasing the buffer size was
> > > > > > > > to enforce the limit on the size that was explicit set by the user.  Then,
> > > > > > > > if that size was sufficient, then DTrace is at liberty to increase that
> > > > > > > > size to a whole multiple of the page size.
> > > > > > > > 
> > > > > > > > But if I were to insist that a buffer can only be 24 bytes, and 24 bytes are
> > > > > > > > not enough to store a single trace record, DTrace should report an error
> > > > > > > > rather than increasing my bufsize setting on its own.
> > > > > > > The point is that DTrace *does* increase my bufsize setting on its own.  Why
> > > > > > > should DTrace complain about my setting when it's going to ignore my setting
> > > > > > > anyhow?  What value does the pre-existing behavior offer to DTrace users?
> > > > > > > Are you saying that DTrace should not resize?  That it should honor the
> > > > > > > user's bufsize?  No more resizing (to page size or whatever it wants)?
> > > > > > The main point of this code was to retain the bevhaviour that DTrace provided,
> > > > > As far as adhering to legacy behavior... quite simply, we do not. We do not
> > > > > set the "principal buffer size" to bufsize.  Further, any legacy resizing
> > > > > behavior is, if I understand correctly, to go smaller than the user
> > > > > requested;  we do the opposite.  Looking to the legacy behavior for
> > > > > precedent is pointless.  The situation is that we are unlike legacy behavior
> > > > > in fundamental respects, opposite in another respect (automatic resizing),
> > > > > but consistent in some impractical and useless respect.  That consistency
> > > > > strikes me as small consolation.
> > > > You can look at this from different angles, yes.
> > > > 
> > > > > There is no documentation to change;  the documentation is already wrong.
> > > > > This patch was intended as a step to making the behavior something that was
> > > > > describable.  Clearly, trying to hold on to a legacy buffering model has
> > > > > produced a byzantine situation in the current port, which treats buffering
> > > > > very differently.
> > > > Yes, but if you want to go that route we ought to go all the way, and not some
> > > > interim solution (see below).
> > > > 
> > > > > Perhaps we simply disagree on those points, and this patch is DOA. But the
> > > > > patch points out other problems as well.  E.g., the min computation wasn't
> > > > > even right and error checking was wrong.
> > > > Let's do this as two patches:
> > > >    - one that ensures that the computation is updated to the current
> > > >      implementation changes and is correct
> > > >    - one that provides for increasing the bufsize to be the smallest multiple of
> > > >      pagesize to fit the largest trace record needed for the tracing session
> > > > 
> > > > I don't think that checking the rounded up bufsize against the minimum size
> > > > makes any sense if you are going to increase the specified bufsize anyway.
> > > > Might as well increase it to fit the minimum size altogether.
> > > I'd like to understand this proposal.  Are you saying:
> > > 
> > > *)  Determine the min size (correctly).
> > Correct.
> > 
> > > *)  Round that up per perf_event requirements (2^n + 1 pages).
> > No, round up to nearest 2^n.  The extra page is allocated automatically by
> > dt_peb_open().
> > 
> > > *)  Ignore the user's bufsize setting.
> > Well, no.  The bufsize setting is not ignored.  We just change the semantics
> > a little (in view of the new implementation) to be a mechanism to allow the
> > user to tune the buffer size, but within certain limitations.
> > 
> > > I'm fine with that (given that we do buffering so unlike the legacy
> > > implementation and will not be honoring the user's bufsize value anyhow),
> > > but I just wanted to be explicit about ignoring the user's bufsize.
> > We do not ignore the bufsize as such.  But the extent to which the bugsize
> > can be tuned is limited by the implementation.  The bufsizw setting cannot be
> > less than the minimum requirement (and you prefer to not complain about that
> > and instead just increase the buffer size).  And its value must be a 2^n
> > multiple of the page size, so it will be rounded up to the nearest acceptable
> > value.  But that is therefore still controlled by the user's bufsize setting.
> > 
> > > > > > which is that if a user explicitly requests a bufsize that is less than the
> > > > > > minimum required for the given trace programs, an error is reported.  That is
> > > > > > a mater of being consistent (even if it has no practical application that I
> > > > > > can see).  Tests from the existing version failed because this was not being
> > > > > > done right.
> > > > > > 
> > > > > > As far as functionality, as long as the bufsize is large enough to hold the
> > > > > > largest trace record, DTrace can (and needs to) ensure that the buffer size
> > > > > > is a whole multiple of the page size because that is required by the perf
> > > > > > ring buffer imlementation.
> > > > > > 
> > > > > > So, DTrace only increases the buffer size to the required multiple of page size
> > > > > > if the user supplied bufsize is at least large enough to hold the largest trace
> > > > > > record (or if no bufsize was explicitly specified).
> > > > > > 
> > > > > > We could opt to change the behaviour of the bufsize opton to always round up to
> > > > > > a whole multiple of page size, but that is a deviation from the original
> > > > > > implementation and thus is something that would need to be documented, etc...
> > > > > > 
> > > > > > Aside from the computation of the minimum required size needing updating when
> > > > > > patches (e.g. the USDT changes) modify the layout and size of the trace records,
> > > > > > I do not think that the existing code is wrong.
> > > > > > 
> > > > > > I am certainly open to a feature change to make bufsize always be rounded up as
> > > > > > needed, but that is not a bug fix and I think that ought to be its own patch
> > > > > > (along with documentation changes etc).  And it might need some more discussion
> > > > > > because it brings up the question why we wouldn't just simply always increase
> > > > > > bufsize to be the smallest multiple of the page size to accomodate the largest
> > > > > > trace record.
> > > > > > 
> > > > > > > > On Thu, Jun 27, 2024 at 01:38:57AM -0400, eugene.loh@oracle.com wrote:
> > > > > > > > > From: Eugene Loh <eugene.loh@oracle.com>
> > > > > > > > > 
> > > > > > > > > The function had a few issues, mainly with its comments,
> > > > > > > > > ranging from typos to incorrect descriptions of behavior.
> > > > > > > > > 
> > > > > > > > > The function has only one caller.  The enforcement of a
> > > > > > > > > size minimum was problematic in a number of respects:
> > > > > > > > > - it was missing 4 bytes
> > > > > > > > > - it was enforcing the minimum before the size was
> > > > > > > > >         increased by the caller
> > > > > > > > > - it was checking dt_pebs_init()==-1,
> > > > > > > > >         missing errors with other negative return values
> > > > > > > > > 
> > > > > > > > > So change the function to accept a minimum and change its
> > > > > > > > > return values.
> > > _______________________________________________
> > > DTrace-devel mailing list
> > > DTrace-devel@oss.oracle.com
> > > https://oss.oracle.com/mailman/listinfo/dtrace-devel

Re: [DTrace-devel] [PATCH 31/38] Fix dt_pebs_init() call

[Eugene Loh]

On 8/30/24 12:53, Kris Van Hees wrote:

> On Fri, Aug 30, 2024 at 01:42:38AM -0400, Eugene Loh wrote:
>> So do you have a proposal for what the new semantics for bufsize should be?
> I believe the semantics of a user-specified bufsize should be as follows:
>
>   - If the bufsize specified by the user is less than the minimum buffer size
>     required to store the largest trace record, an error should be reported.
>   - If the bufsize satisfies the minimum requirement, the implementation must
>     ensure that it is round up (if needed) to satisfy the perf ring buffer
>     size requirement (size needs to be 2^n * pagesize, for n = 0, 1, 2, ...)
>
> Point 1 is clearly debatable but follows the principle that you cannot put
> something bigger into something that is too small to hold it.

Thanks.  It helps to see a specific proposal.  It just strikes me as odd 
that we would say a user specified too low when we're not going to use 
that size anyhow -- we're going to increase it.  Why should we complain 
about a request we're going to ignore?

What do you think about not automatically resizing?  Resizing in legacy 
I think was by default and rounded down.  You autosize up. How about we 
don't resize?  If they ask for less than min, we complain and tell them 
what the min size.  If they're not 2^n pages, we tell them so.  The 
default right now is 4m.  So if someone is playing with bufsize we might 
as well honor their request (and give them guidance if what they're 
trying to do doesn't work).

>> (And, why not simply ignore bufsize?)
> Because we want users to be able to tune the size of the tracing buffer, which
> can affect performance in terms of how quickly trace data is generated versus
> how quiickly it is consumed.  It interacts with things like switchrate.
>
>> On 8/29/24 22:26, Kris Van Hees wrote:
>>> On Thu, Aug 29, 2024 at 08:54:46PM -0400, Eugene Loh via DTrace-devel wrote:
>>>> On 8/28/24 17:16, Kris Van Hees wrote:
>>>>> On Wed, Aug 28, 2024 at 04:57:43PM -0400, Eugene Loh wrote:
>>>>>> On 8/26/24 12:20, Kris Van Hees wrote:
>>>>>>> On Mon, Aug 26, 2024 at 11:42:25AM -0400, Eugene Loh wrote:
>>>>>>>> On 8/26/24 10:30, Kris Van Hees wrote:
>>>>>>>>> This patch goes against the intent of the code, as far as I can see.  The
>>>>>>>>> point of performing the size check prior to increasing the buffer size was
>>>>>>>>> to enforce the limit on the size that was explicit set by the user.  Then,
>>>>>>>>> if that size was sufficient, then DTrace is at liberty to increase that
>>>>>>>>> size to a whole multiple of the page size.
>>>>>>>>>
>>>>>>>>> But if I were to insist that a buffer can only be 24 bytes, and 24 bytes are
>>>>>>>>> not enough to store a single trace record, DTrace should report an error
>>>>>>>>> rather than increasing my bufsize setting on its own.
>>>>>>>> The point is that DTrace *does* increase my bufsize setting on its own.  Why
>>>>>>>> should DTrace complain about my setting when it's going to ignore my setting
>>>>>>>> anyhow?  What value does the pre-existing behavior offer to DTrace users?
>>>>>>>> Are you saying that DTrace should not resize?  That it should honor the
>>>>>>>> user's bufsize?  No more resizing (to page size or whatever it wants)?
>>>>>>> The main point of this code was to retain the bevhaviour that DTrace provided,
>>>>>> As far as adhering to legacy behavior... quite simply, we do not. We do not
>>>>>> set the "principal buffer size" to bufsize.  Further, any legacy resizing
>>>>>> behavior is, if I understand correctly, to go smaller than the user
>>>>>> requested;  we do the opposite.  Looking to the legacy behavior for
>>>>>> precedent is pointless.  The situation is that we are unlike legacy behavior
>>>>>> in fundamental respects, opposite in another respect (automatic resizing),
>>>>>> but consistent in some impractical and useless respect.  That consistency
>>>>>> strikes me as small consolation.
>>>>> You can look at this from different angles, yes.
>>>>>
>>>>>> There is no documentation to change;  the documentation is already wrong.
>>>>>> This patch was intended as a step to making the behavior something that was
>>>>>> describable.  Clearly, trying to hold on to a legacy buffering model has
>>>>>> produced a byzantine situation in the current port, which treats buffering
>>>>>> very differently.
>>>>> Yes, but if you want to go that route we ought to go all the way, and not some
>>>>> interim solution (see below).
>>>>>
>>>>>> Perhaps we simply disagree on those points, and this patch is DOA. But the
>>>>>> patch points out other problems as well.  E.g., the min computation wasn't
>>>>>> even right and error checking was wrong.
>>>>> Let's do this as two patches:
>>>>>     - one that ensures that the computation is updated to the current
>>>>>       implementation changes and is correct
>>>>>     - one that provides for increasing the bufsize to be the smallest multiple of
>>>>>       pagesize to fit the largest trace record needed for the tracing session
>>>>>
>>>>> I don't think that checking the rounded up bufsize against the minimum size
>>>>> makes any sense if you are going to increase the specified bufsize anyway.
>>>>> Might as well increase it to fit the minimum size altogether.
>>>> I'd like to understand this proposal.  Are you saying:
>>>>
>>>> *)  Determine the min size (correctly).
>>> Correct.
>>>
>>>> *)  Round that up per perf_event requirements (2^n + 1 pages).
>>> No, round up to nearest 2^n.  The extra page is allocated automatically by
>>> dt_peb_open().
>>>
>>>> *)  Ignore the user's bufsize setting.
>>> Well, no.  The bufsize setting is not ignored.  We just change the semantics
>>> a little (in view of the new implementation) to be a mechanism to allow the
>>> user to tune the buffer size, but within certain limitations.
>>>
>>>> I'm fine with that (given that we do buffering so unlike the legacy
>>>> implementation and will not be honoring the user's bufsize value anyhow),
>>>> but I just wanted to be explicit about ignoring the user's bufsize.
>>> We do not ignore the bufsize as such.  But the extent to which the bugsize
>>> can be tuned is limited by the implementation.  The bufsizw setting cannot be
>>> less than the minimum requirement (and you prefer to not complain about that
>>> and instead just increase the buffer size).  And its value must be a 2^n
>>> multiple of the page size, so it will be rounded up to the nearest acceptable
>>> value.  But that is therefore still controlled by the user's bufsize setting.
>>>
>>>>>>> which is that if a user explicitly requests a bufsize that is less than the
>>>>>>> minimum required for the given trace programs, an error is reported.  That is
>>>>>>> a mater of being consistent (even if it has no practical application that I
>>>>>>> can see).  Tests from the existing version failed because this was not being
>>>>>>> done right.
>>>>>>>
>>>>>>> As far as functionality, as long as the bufsize is large enough to hold the
>>>>>>> largest trace record, DTrace can (and needs to) ensure that the buffer size
>>>>>>> is a whole multiple of the page size because that is required by the perf
>>>>>>> ring buffer imlementation.
>>>>>>>
>>>>>>> So, DTrace only increases the buffer size to the required multiple of page size
>>>>>>> if the user supplied bufsize is at least large enough to hold the largest trace
>>>>>>> record (or if no bufsize was explicitly specified).
>>>>>>>
>>>>>>> We could opt to change the behaviour of the bufsize opton to always round up to
>>>>>>> a whole multiple of page size, but that is a deviation from the original
>>>>>>> implementation and thus is something that would need to be documented, etc...
>>>>>>>
>>>>>>> Aside from the computation of the minimum required size needing updating when
>>>>>>> patches (e.g. the USDT changes) modify the layout and size of the trace records,
>>>>>>> I do not think that the existing code is wrong.
>>>>>>>
>>>>>>> I am certainly open to a feature change to make bufsize always be rounded up as
>>>>>>> needed, but that is not a bug fix and I think that ought to be its own patch
>>>>>>> (along with documentation changes etc).  And it might need some more discussion
>>>>>>> because it brings up the question why we wouldn't just simply always increase
>>>>>>> bufsize to be the smallest multiple of the page size to accomodate the largest
>>>>>>> trace record.
>>>>>>>
>>>>>>>>> On Thu, Jun 27, 2024 at 01:38:57AM -0400, eugene.loh@oracle.com wrote:
>>>>>>>>>> From: Eugene Loh <eugene.loh@oracle.com>
>>>>>>>>>>
>>>>>>>>>> The function had a few issues, mainly with its comments,
>>>>>>>>>> ranging from typos to incorrect descriptions of behavior.
>>>>>>>>>>
>>>>>>>>>> The function has only one caller.  The enforcement of a
>>>>>>>>>> size minimum was problematic in a number of respects:
>>>>>>>>>> - it was missing 4 bytes
>>>>>>>>>> - it was enforcing the minimum before the size was
>>>>>>>>>>          increased by the caller
>>>>>>>>>> - it was checking dt_pebs_init()==-1,
>>>>>>>>>>          missing errors with other negative return values
>>>>>>>>>>
>>>>>>>>>> So change the function to accept a minimum and change its
>>>>>>>>>> return values.
>>>> _______________________________________________
>>>> DTrace-devel mailing list
>>>> DTrace-devel@oss.oracle.com
>>>> https://oss.oracle.com/mailman/listinfo/dtrace-devel

Re: [DTrace-devel] [PATCH 31/38] Fix dt_pebs_init() call

[Kris Van Hees]

On Fri, Aug 30, 2024 at 03:06:02PM -0400, Eugene Loh wrote:
> On 8/30/24 12:53, Kris Van Hees wrote:
> 
> > On Fri, Aug 30, 2024 at 01:42:38AM -0400, Eugene Loh wrote:
> > > So do you have a proposal for what the new semantics for bufsize should be?
> > I believe the semantics of a user-specified bufsize should be as follows:
> > 
> >   - If the bufsize specified by the user is less than the minimum buffer size
> >     required to store the largest trace record, an error should be reported.
> >   - If the bufsize satisfies the minimum requirement, the implementation must
> >     ensure that it is round up (if needed) to satisfy the perf ring buffer
> >     size requirement (size needs to be 2^n * pagesize, for n = 0, 1, 2, ...)
> > 
> > Point 1 is clearly debatable but follows the principle that you cannot put
> > something bigger into something that is too small to hold it.
> 
> Thanks.  It helps to see a specific proposal.  It just strikes me as odd
> that we would say a user specified too low when we're not going to use that
> size anyhow -- we're going to increase it.  Why should we complain about a
> request we're going to ignore?

We do not ignore it.  If it is not big enough, we report and error and exit.
> 
> What do you think about not automatically resizing?  Resizing in legacy I
> think was by default and rounded down.  You autosize up. How about we don't
> resize?  If they ask for less than min, we complain and tell them what the
> min size.  If they're not 2^n pages, we tell them so.  The default right now
> is 4m.  So if someone is playing with bufsize we might as well honor their
> request (and give them guidance if what they're trying to do doesn't work).

Resizing in the original code was done if the allocation request could not be
satisfied by the kernel code.  The kernel code would then try a series of
smaller sizes until it found one that resulted in successful allocation.  We
currently do not do that and instead simply fail if allocations fail, because
we can - it is all done in userspace.  We could do auto-reduction to try a size
that works, but the user can do that themselves as easily.

The increasing of the size is done automatically because we do not want to bog
down the user with needing to know what select sizes are acceptable.  If they
provide a size and we can satisfy it with that size or the nearest acceptable
size, we can just do that.

Look at it in similarity to malloc()...  we don't know whether our malloc
request is satisfied by the exact size we supplied or by a larger block.  And
as a user, we do not really care.

> > > (And, why not simply ignore bufsize?)
> > Because we want users to be able to tune the size of the tracing buffer, which
> > can affect performance in terms of how quickly trace data is generated versus
> > how quiickly it is consumed.  It interacts with things like switchrate.
> > 
> > > On 8/29/24 22:26, Kris Van Hees wrote:
> > > > On Thu, Aug 29, 2024 at 08:54:46PM -0400, Eugene Loh via DTrace-devel wrote:
> > > > > On 8/28/24 17:16, Kris Van Hees wrote:
> > > > > > On Wed, Aug 28, 2024 at 04:57:43PM -0400, Eugene Loh wrote:
> > > > > > > On 8/26/24 12:20, Kris Van Hees wrote:
> > > > > > > > On Mon, Aug 26, 2024 at 11:42:25AM -0400, Eugene Loh wrote:
> > > > > > > > > On 8/26/24 10:30, Kris Van Hees wrote:
> > > > > > > > > > This patch goes against the intent of the code, as far as I can see.  The
> > > > > > > > > > point of performing the size check prior to increasing the buffer size was
> > > > > > > > > > to enforce the limit on the size that was explicit set by the user.  Then,
> > > > > > > > > > if that size was sufficient, then DTrace is at liberty to increase that
> > > > > > > > > > size to a whole multiple of the page size.
> > > > > > > > > > 
> > > > > > > > > > But if I were to insist that a buffer can only be 24 bytes, and 24 bytes are
> > > > > > > > > > not enough to store a single trace record, DTrace should report an error
> > > > > > > > > > rather than increasing my bufsize setting on its own.
> > > > > > > > > The point is that DTrace *does* increase my bufsize setting on its own.  Why
> > > > > > > > > should DTrace complain about my setting when it's going to ignore my setting
> > > > > > > > > anyhow?  What value does the pre-existing behavior offer to DTrace users?
> > > > > > > > > Are you saying that DTrace should not resize?  That it should honor the
> > > > > > > > > user's bufsize?  No more resizing (to page size or whatever it wants)?
> > > > > > > > The main point of this code was to retain the bevhaviour that DTrace provided,
> > > > > > > As far as adhering to legacy behavior... quite simply, we do not. We do not
> > > > > > > set the "principal buffer size" to bufsize.  Further, any legacy resizing
> > > > > > > behavior is, if I understand correctly, to go smaller than the user
> > > > > > > requested;  we do the opposite.  Looking to the legacy behavior for
> > > > > > > precedent is pointless.  The situation is that we are unlike legacy behavior
> > > > > > > in fundamental respects, opposite in another respect (automatic resizing),
> > > > > > > but consistent in some impractical and useless respect.  That consistency
> > > > > > > strikes me as small consolation.
> > > > > > You can look at this from different angles, yes.
> > > > > > 
> > > > > > > There is no documentation to change;  the documentation is already wrong.
> > > > > > > This patch was intended as a step to making the behavior something that was
> > > > > > > describable.  Clearly, trying to hold on to a legacy buffering model has
> > > > > > > produced a byzantine situation in the current port, which treats buffering
> > > > > > > very differently.
> > > > > > Yes, but if you want to go that route we ought to go all the way, and not some
> > > > > > interim solution (see below).
> > > > > > 
> > > > > > > Perhaps we simply disagree on those points, and this patch is DOA. But the
> > > > > > > patch points out other problems as well.  E.g., the min computation wasn't
> > > > > > > even right and error checking was wrong.
> > > > > > Let's do this as two patches:
> > > > > >     - one that ensures that the computation is updated to the current
> > > > > >       implementation changes and is correct
> > > > > >     - one that provides for increasing the bufsize to be the smallest multiple of
> > > > > >       pagesize to fit the largest trace record needed for the tracing session
> > > > > > 
> > > > > > I don't think that checking the rounded up bufsize against the minimum size
> > > > > > makes any sense if you are going to increase the specified bufsize anyway.
> > > > > > Might as well increase it to fit the minimum size altogether.
> > > > > I'd like to understand this proposal.  Are you saying:
> > > > > 
> > > > > *)  Determine the min size (correctly).
> > > > Correct.
> > > > 
> > > > > *)  Round that up per perf_event requirements (2^n + 1 pages).
> > > > No, round up to nearest 2^n.  The extra page is allocated automatically by
> > > > dt_peb_open().
> > > > 
> > > > > *)  Ignore the user's bufsize setting.
> > > > Well, no.  The bufsize setting is not ignored.  We just change the semantics
> > > > a little (in view of the new implementation) to be a mechanism to allow the
> > > > user to tune the buffer size, but within certain limitations.
> > > > 
> > > > > I'm fine with that (given that we do buffering so unlike the legacy
> > > > > implementation and will not be honoring the user's bufsize value anyhow),
> > > > > but I just wanted to be explicit about ignoring the user's bufsize.
> > > > We do not ignore the bufsize as such.  But the extent to which the bugsize
> > > > can be tuned is limited by the implementation.  The bufsizw setting cannot be
> > > > less than the minimum requirement (and you prefer to not complain about that
> > > > and instead just increase the buffer size).  And its value must be a 2^n
> > > > multiple of the page size, so it will be rounded up to the nearest acceptable
> > > > value.  But that is therefore still controlled by the user's bufsize setting.
> > > > 
> > > > > > > > which is that if a user explicitly requests a bufsize that is less than the
> > > > > > > > minimum required for the given trace programs, an error is reported.  That is
> > > > > > > > a mater of being consistent (even if it has no practical application that I
> > > > > > > > can see).  Tests from the existing version failed because this was not being
> > > > > > > > done right.
> > > > > > > > 
> > > > > > > > As far as functionality, as long as the bufsize is large enough to hold the
> > > > > > > > largest trace record, DTrace can (and needs to) ensure that the buffer size
> > > > > > > > is a whole multiple of the page size because that is required by the perf
> > > > > > > > ring buffer imlementation.
> > > > > > > > 
> > > > > > > > So, DTrace only increases the buffer size to the required multiple of page size
> > > > > > > > if the user supplied bufsize is at least large enough to hold the largest trace
> > > > > > > > record (or if no bufsize was explicitly specified).
> > > > > > > > 
> > > > > > > > We could opt to change the behaviour of the bufsize opton to always round up to
> > > > > > > > a whole multiple of page size, but that is a deviation from the original
> > > > > > > > implementation and thus is something that would need to be documented, etc...
> > > > > > > > 
> > > > > > > > Aside from the computation of the minimum required size needing updating when
> > > > > > > > patches (e.g. the USDT changes) modify the layout and size of the trace records,
> > > > > > > > I do not think that the existing code is wrong.
> > > > > > > > 
> > > > > > > > I am certainly open to a feature change to make bufsize always be rounded up as
> > > > > > > > needed, but that is not a bug fix and I think that ought to be its own patch
> > > > > > > > (along with documentation changes etc).  And it might need some more discussion
> > > > > > > > because it brings up the question why we wouldn't just simply always increase
> > > > > > > > bufsize to be the smallest multiple of the page size to accomodate the largest
> > > > > > > > trace record.
> > > > > > > > 
> > > > > > > > > > On Thu, Jun 27, 2024 at 01:38:57AM -0400, eugene.loh@oracle.com wrote:
> > > > > > > > > > > From: Eugene Loh <eugene.loh@oracle.com>
> > > > > > > > > > > 
> > > > > > > > > > > The function had a few issues, mainly with its comments,
> > > > > > > > > > > ranging from typos to incorrect descriptions of behavior.
> > > > > > > > > > > 
> > > > > > > > > > > The function has only one caller.  The enforcement of a
> > > > > > > > > > > size minimum was problematic in a number of respects:
> > > > > > > > > > > - it was missing 4 bytes
> > > > > > > > > > > - it was enforcing the minimum before the size was
> > > > > > > > > > >          increased by the caller
> > > > > > > > > > > - it was checking dt_pebs_init()==-1,
> > > > > > > > > > >          missing errors with other negative return values
> > > > > > > > > > > 
> > > > > > > > > > > So change the function to accept a minimum and change its
> > > > > > > > > > > return values.
> > > > > _______________________________________________
> > > > > DTrace-devel mailing list
> > > > > DTrace-devel@oss.oracle.com
> > > > > https://oss.oracle.com/mailman/listinfo/dtrace-devel

Re: [PATCH 30/38] Allow relocation on BPF_OR instructions

[Kris Van Hees]

On Fri, Jul 19, 2024 at 05:34:17PM -0400, Kris Van Hees wrote:
> On Thu, Jun 27, 2024 at 01:38:56AM -0400, eugene.loh@oracle.com wrote:
> > From: Eugene Loh <eugene.loh@oracle.com>
> > 
> > Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> 
> Reviewed-by: Kris Van Hees <kris.van.hees@oracle.com>

As far as I can see, there is nothing that uses this anymore.  I suggest we
ut it in cold storage until there is a need for it?  Alternatively, we could
turn this into a patch that adds support for all possible operations that
could require relocations like this, and introduce that to the tree instead
to support future uses.

> > ---
> >  libdtrace/dt_as.c | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/libdtrace/dt_as.c b/libdtrace/dt_as.c
> > index a634b855..4b397f51 100644
> > --- a/libdtrace/dt_as.c
> > +++ b/libdtrace/dt_as.c
> > @@ -280,6 +280,7 @@ dt_as(dt_pcb_t *pcb)
> >  		case BPF_ST | BPF_MEM | BPF_DW:		/* stdw */
> >  		case BPF_ALU64 | BPF_MOV | BPF_K:	/* mov */
> >  		case BPF_ALU64 | BPF_ADD | BPF_K:	/* add */
> > +		case BPF_ALU64 | BPF_OR  | BPF_K:	/* or */
> >  			if (idp->di_flags & DT_IDFLG_BPF)
> >  				brel++;
> >  			else
> > @@ -492,6 +493,7 @@ fail:
> >  			case BPF_ST | BPF_MEM | BPF_DW:		/* stdw */
> >  			case BPF_ALU64 | BPF_MOV | BPF_K:	/* mov */
> >  			case BPF_ALU64 | BPF_ADD | BPF_K:	/* add */
> > +			case BPF_ALU64 | BPF_OR  | BPF_K:	/* or */
> >  				rp->dofr_type = R_BPF_64_32;
> >  				break;
> >  			case BPF_LD | BPF_IMM | BPF_DW:		/* lddw */
> > -- 
> > 2.18.4
> > 

Re: [PATCH 30/38] Allow relocation on BPF_OR instructions

[Eugene Loh]

On 9/30/24 17:19, Kris Van Hees wrote:

> On Fri, Jul 19, 2024 at 05:34:17PM -0400, Kris Van Hees wrote:
>> On Thu, Jun 27, 2024 at 01:38:56AM -0400, eugene.loh@oracle.com wrote:
>>> From: Eugene Loh <eugene.loh@oracle.com>
>>>
>>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
>> Reviewed-by: Kris Van Hees <kris.van.hees@oracle.com>
> As far as I can see, there is nothing that uses this anymore.

Yes.  As best as I can remember, that's right.

> I suggest we
> ut it in cold storage until there is a need for it?

Sounds great.

> Alternatively, we could
> turn this into a patch that adds support for all possible operations that
> could require relocations like this, and introduce that to the tree instead
> to support future uses.

Meh.

>>> ---
>>>   libdtrace/dt_as.c | 2 ++
>>>   1 file changed, 2 insertions(+)
>>>
>>> diff --git a/libdtrace/dt_as.c b/libdtrace/dt_as.c
>>> index a634b855..4b397f51 100644
>>> --- a/libdtrace/dt_as.c
>>> +++ b/libdtrace/dt_as.c
>>> @@ -280,6 +280,7 @@ dt_as(dt_pcb_t *pcb)
>>>   		case BPF_ST | BPF_MEM | BPF_DW:		/* stdw */
>>>   		case BPF_ALU64 | BPF_MOV | BPF_K:	/* mov */
>>>   		case BPF_ALU64 | BPF_ADD | BPF_K:	/* add */
>>> +		case BPF_ALU64 | BPF_OR  | BPF_K:	/* or */
>>>   			if (idp->di_flags & DT_IDFLG_BPF)
>>>   				brel++;
>>>   			else
>>> @@ -492,6 +493,7 @@ fail:
>>>   			case BPF_ST | BPF_MEM | BPF_DW:		/* stdw */
>>>   			case BPF_ALU64 | BPF_MOV | BPF_K:	/* mov */
>>>   			case BPF_ALU64 | BPF_ADD | BPF_K:	/* add */
>>> +			case BPF_ALU64 | BPF_OR  | BPF_K:	/* or */
>>>   				rp->dofr_type = R_BPF_64_32;
>>>   				break;
>>>   			case BPF_LD | BPF_IMM | BPF_DW:		/* lddw */
>>> -- 
>>> 2.18.4
>>>

Re: [PATCH 26/38] test: Annotate xfail (chill not implemented yet)

[Kris Van Hees]

If memory serves me well, I think the plan was to change this test to not use
chill at all (and remove the xfail), and introduce a new test that exercises
the use of chill in the context of a clause that is a zero-size speculation.

On Fri, Jul 19, 2024 at 07:38:33PM -0400, Eugene Loh wrote:
> On 7/19/24 17:12, Kris Van Hees wrote:
> 
> > On Thu, Jun 27, 2024 at 01:38:52AM -0400, eugene.loh@oracle.com wrote:
> > > From: Eugene Loh <eugene.loh@oracle.com>
> > Before I can really assess whether the annotation is valid,
> 
> Well, I'm pretty sure the annotation is right.
> 
> > I'd like to know
> > why chill) is even used here.  I.e. I don't quite understand the test :)
> 
> Well, I'm all with you on that.  I'm pretty sure the chill() is pretty
> irrelevant and that the test is "valid" and passes without it.  I am of two
> minds:
> 
> 1)  Remove the chill() and @@xfail and move on.
> 
> 2)  Be grateful for idiosyncrasies in the test suite, since they often turn
> up bugs.  Weird tests turn up weird bugs.
> 
> The patch was going with #2;  that get my vote.  I'm okay with #1, however. 
> Feel free to decide.
> 
> > > Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
> > > ---
> > >   test/unittest/speculation/tst.zerosize.d | 2 +-
> > >   1 file changed, 1 insertion(+), 1 deletion(-)
> > > 
> > > diff --git a/test/unittest/speculation/tst.zerosize.d b/test/unittest/speculation/tst.zerosize.d
> > > index 56c1fcea..996f1257 100644
> > > --- a/test/unittest/speculation/tst.zerosize.d
> > > +++ b/test/unittest/speculation/tst.zerosize.d
> > > @@ -5,7 +5,7 @@
> > >    * http://oss.oracle.com/licenses/upl.
> > >    */
> > > -/* @@xfail: dtv2 */
> > > +/* @@xfail: dtv2, chill not implemented yet */
> > >   #pragma D option destructive
> > > -- 
> > > 2.18.4
> > > 

Re: [PATCH 26/38] test: Annotate xfail (chill not implemented yet)

[Eugene Loh]

On 10/29/24 11:05, Kris Van Hees wrote:

> If memory serves me well,

It does.

> I think the plan was to change this test to not use
> chill at all (and remove the xfail), and introduce a new test that exercises
> the use of chill in the context of a clause that is a zero-size speculation.

v2 will have the slightly different name
"test: Clean up zero-size speculation + chill()".

> On Fri, Jul 19, 2024 at 07:38:33PM -0400, Eugene Loh wrote:
>> On 7/19/24 17:12, Kris Van Hees wrote:
>>
>>> On Thu, Jun 27, 2024 at 01:38:52AM -0400, eugene.loh@oracle.com wrote:
>>>> From: Eugene Loh <eugene.loh@oracle.com>
>>> Before I can really assess whether the annotation is valid,
>> Well, I'm pretty sure the annotation is right.
>>
>>> I'd like to know
>>> why chill) is even used here.  I.e. I don't quite understand the test :)
>> Well, I'm all with you on that.  I'm pretty sure the chill() is pretty
>> irrelevant and that the test is "valid" and passes without it.  I am of two
>> minds:
>>
>> 1)  Remove the chill() and @@xfail and move on.
>>
>> 2)  Be grateful for idiosyncrasies in the test suite, since they often turn
>> up bugs.  Weird tests turn up weird bugs.
>>
>> The patch was going with #2;  that get my vote.  I'm okay with #1, however.
>> Feel free to decide.
>>
>>>> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
>>>> ---
>>>>    test/unittest/speculation/tst.zerosize.d | 2 +-
>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/test/unittest/speculation/tst.zerosize.d b/test/unittest/speculation/tst.zerosize.d
>>>> index 56c1fcea..996f1257 100644
>>>> --- a/test/unittest/speculation/tst.zerosize.d
>>>> +++ b/test/unittest/speculation/tst.zerosize.d
>>>> @@ -5,7 +5,7 @@
>>>>     * http://oss.oracle.com/licenses/upl.
>>>>     */
>>>> -/* @@xfail: dtv2 */
>>>> +/* @@xfail: dtv2, chill not implemented yet */
>>>>    #pragma D option destructive
>>>> -- 
>>>> 2.18.4
>>>>