Re: [PATCH] test: Have USDT "deferred" tests wait for dtrace to start

[Kris Van Hees <kris.van.hees@oracle.com>]

On Tue, Dec 10, 2024 at 06:02:02PM -0500, eugene.loh@oracle.com wrote:
> From: Eugene Loh <eugene.loh@oracle.com>
> 
> Systemwide USDT tracing includes having dtrace track USDT processes
> that start after it does.  Associated tests start dtrace first, wait
> a few seconds, and then start processes to be traced.
> 
> Waiting "a few seconds" can be insufficient, especially on slower
> systems or as more work is being performed during dtrace startup.
> 
> Change tests to wait for dtrace to start by polling on the output file.
> 
> Signed-off-by: Eugene Loh <eugene.loh@oracle.com>

Reviewed-by: Kris Van Hees <kris.van.hees@oracle.com>

... though hopefully in the very near future we will not need this anymore,
so we should revisit this once we have better (more timely) detection of USDT
probes.  Otherwise these tests might result in hiding issues with the timely
detection of the probes.

But for now, this is needed to ensure we have proper functional testing.

> ---
>  test/unittest/usdt/err.Z_no-w.sh      | 17 ++++++++++++++---
>  test/unittest/usdt/tst.defer-Z.sh     | 17 ++++++++++++++---
>  test/unittest/usdt/tst.defer.sh       | 17 ++++++++++++++---
>  test/unittest/usdt/tst.nusdtprobes.sh | 16 +++++++++++++---
>  4 files changed, 55 insertions(+), 12 deletions(-)
> 
> diff --git a/test/unittest/usdt/err.Z_no-w.sh b/test/unittest/usdt/err.Z_no-w.sh
> index 3833b4400..4f129341d 100755
> --- a/test/unittest/usdt/err.Z_no-w.sh
> +++ b/test/unittest/usdt/err.Z_no-w.sh
> @@ -34,9 +34,20 @@ testprov*:::foo
>  	raise(SIGUSR1);
>  }' &
>  dtpid=$!
> -sleep 4
> -if [[ ! -d /proc/$dtpid ]]; then
> -	echo ERROR dtrace died prematurely
> +
> +# Wait up to half of the timeout period for dtrace to start up.
> +
> +iter=$((timeout / 2))
> +while [ $iter -gt 0 ]; do
> +	sleep 1
> +	if [ -e dtrace.out ]; then
> +		break
> +	fi
> +	iter=$((iter - 1))
> +done
> +if [[ $iter -eq 0 ]]; then
> +	echo ERROR starting DTrace job
> +	cat dtrace.out
>  	exit 1
>  fi
>  
> diff --git a/test/unittest/usdt/tst.defer-Z.sh b/test/unittest/usdt/tst.defer-Z.sh
> index 52e92eb6b..ff2c5cbf1 100755
> --- a/test/unittest/usdt/tst.defer-Z.sh
> +++ b/test/unittest/usdt/tst.defer-Z.sh
> @@ -45,9 +45,20 @@ testprov*4:::bar
>  	@[pid, 3] = sum(pid % 100);
>  }' &
>  dtpid=$!
> -sleep 2
> -if [[ ! -d /proc/$dtpid ]]; then
> -	echo ERROR dtrace died
> +
> +# Wait up to half of the timeout period for dtrace to start up.
> +
> +iter=$((timeout / 2))
> +while [ $iter -gt 0 ]; do
> +	sleep 1
> +	if [ -e dtrace.out ]; then
> +		break
> +	fi
> +	iter=$((iter - 1))
> +done
> +if [[ $iter -eq 0 ]]; then
> +	echo ERROR starting DTrace job
> +	cat dtrace.out
>  	exit 1
>  fi
>  
> diff --git a/test/unittest/usdt/tst.defer.sh b/test/unittest/usdt/tst.defer.sh
> index 02ed1a767..073af12d5 100755
> --- a/test/unittest/usdt/tst.defer.sh
> +++ b/test/unittest/usdt/tst.defer.sh
> @@ -51,9 +51,20 @@ testprov*'$lastdigit':::bar
>  	@[pid, 3] = sum(pid % 100);
>  }' &
>  dtpid=$!
> -sleep 2
> -if [[ ! -d /proc/$dtpid ]]; then
> -	echo ERROR dtrace died
> +
> +# Wait up to half of the timeout period for dtrace to start up.
> +
> +iter=$((timeout / 2))
> +while [ $iter -gt 0 ]; do
> +	sleep 1
> +	if [ -e dtrace.out ]; then
> +		break
> +	fi
> +	iter=$((iter - 1))
> +done
> +if [[ $iter -eq 0 ]]; then
> +	echo ERROR starting DTrace job
> +	cat dtrace.out
>  	kill -USR1 ${pids[0]}
>  	wait       ${pids[0]}
>  	exit 1
> diff --git a/test/unittest/usdt/tst.nusdtprobes.sh b/test/unittest/usdt/tst.nusdtprobes.sh
> index d2d80fe8d..f275f921f 100755
> --- a/test/unittest/usdt/tst.nusdtprobes.sh
> +++ b/test/unittest/usdt/tst.nusdtprobes.sh
> @@ -105,9 +105,19 @@ for nusdt in "" "-xnusdtprobes=40" "-xnusdtprobes=39"; do
>  		@[probeprov, probemod, probefunc, probename] = count();
>  	}' &
>  	dtpid=$!
> -	sleep 2
> -	if [[ ! -d /proc/$dtpid ]]; then
> -		echo ERROR dtrace died
> +
> +	# Wait a little for dtrace to start up.
> +
> +	iter=$((timeout / 4))
> +	while [ $iter -gt 0 ]; do
> +		sleep 1
> +		if [ -e dtrace.out ]; then
> +			break
> +		fi
> +		iter=$((iter - 1))
> +	done
> +	if [[ $iter -eq 0 ]]; then
> +		echo ERROR starting DTrace job
>  		cat dtrace.out
>  		exit 1
>  	fi
> -- 
> 2.43.5
>