Re: [PATCH v1] perf tests: Harden branch stack sampling test

[Leo Yan <leo.yan@arm.com>]

Hi Namhyung,

On Thu, Mar 13, 2025 at 01:18:08PM -0700, Namhyung Kim wrote:

[...]

> > >  test_user_branches() {
> > >  	echo "Testing user branch stack sampling"
> > >  
> > > -	perf record -o $TMPDIR/perf.data --branch-filter any,save_type,u -- ${TESTPROG} > /dev/null 2>&1
> > > -	perf script -i $TMPDIR/perf.data --fields brstacksym | tr -s ' ' '\n' > $TMPDIR/perf.script
> > > +	perf record -o "$TMPDIR/perf.data" --branch-filter any,save_type,u -- ${TESTPROG} > "$TMPDIR/record.txt" 2>&1
> > > +	perf script -i "$TMPDIR/perf.data" --fields brstacksym > "$TMPDIR/perf.script"
> > >  
> > >  	# example of branch entries:
> > >  	# 	brstack_foo+0x14/brstack_bar+0x40/P/-/-/0/CALL
> > >  
> > > -	set -x
> > > -	grep -E -m1 "^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/IND_CALL/.*$"	$TMPDIR/perf.script
> > > -	grep -E -m1 "^brstack_foo\+[^ ]*/brstack_bar\+[^ ]*/CALL/.*$"	$TMPDIR/perf.script
> > > -	grep -E -m1 "^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/CALL/.*$"	$TMPDIR/perf.script
> > > -	grep -E -m1 "^brstack_bench\+[^ ]*/brstack_bar\+[^ ]*/CALL/.*$"	$TMPDIR/perf.script
> > > -	grep -E -m1 "^brstack_bar\+[^ ]*/brstack_foo\+[^ ]*/RET/.*$"		$TMPDIR/perf.script
> > > -	grep -E -m1 "^brstack_foo\+[^ ]*/brstack_bench\+[^ ]*/RET/.*$"	$TMPDIR/perf.script
> > > -	grep -E -m1 "^brstack_bench\+[^ ]*/brstack_bench\+[^ ]*/COND/.*$"	$TMPDIR/perf.script
> > > -	grep -E -m1 "^brstack\+[^ ]*/brstack\+[^ ]*/UNCOND/.*$"		$TMPDIR/perf.script
> > > -	set +x
> > > -
> > > +	expected=(
> > > +		"^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/IND_CALL/.*$"
> > > +		"^brstack_foo\+[^ ]*/brstack_bar\+[^ ]*/CALL/.*$"
> > > +		"^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/CALL/.*$"
> > > +		"^brstack_bench\+[^ ]*/brstack_bar\+[^ ]*/CALL/.*$"
> > > +		"^brstack_bar\+[^ ]*/brstack_foo\+[^ ]*/RET/.*$"
> > > +		"^brstack_foo\+[^ ]*/brstack_bench\+[^ ]*/RET/.*$"
> > > +		"^brstack_bench\+[^ ]*/brstack_bench\+[^ ]*/COND/.*$"
> > > +		"^brstack\+[^ ]*/brstack\+[^ ]*/UNCOND/.*$"
> > > +	)
> > > +	for x in "${expected[@]}"
> > > +	do
> > > +		if ! tr -s ' ' '\n' < "$TMPDIR/perf.script" | grep -E -m1 -q "$x"
> > > +		then
> > > +			echo "Branches missing $x"
> > > +			if [ "x$err" == "x0" ]
> > > +			then
> > > +				err=2
> > 
> > Here it sets "err=2", as a result, if any grep command fails, the script
> > exits while reporting to skip the test.  This seems incorrect to me.
> > 
> > My understanding is the regular expressions above are mandatory to be
> > matched, otherwise, it must be something is wrong.  We should not skip
> > the test in this case.
> > 
> > I can understand that 'perf record' cannot record all branch types, if
> > this is the case, maybe we can improve the recording quality rather
> > than reporting skip?  E.g.,
> > 
> >   cat <<EOF > "$TMPDIR/loop.sh"
> >   for run in {1..5}; do perf test -w brstack; done
> >   EOF
> > 
> >   perf record -o "$TMPDIR/perf.data" --branch-filter any,save_type,u
> >     -- sh $TMPDIR/loop.sh
> > 
> > If we run the test for 5 times, should this can allow us to ensure the
> > branch samples are recorded?
> 
> The brstack (and other workload programs) can take an argument to
> control its duration.  For brstack, it's the number of loop iteration
> and default is 999999.

Sorry I did not dig into the brstack workload program.

If the workload has run for a large number of loops, the question is:
why isn't the test capturing the expected branch stacks?

Thanks,
Leo