[PATCH 1/2] perf archive: Add new option '--all'

[PATCH 1/2] perf archive: Add new option '--all'

[vmolnaro]

From: Veronika Molnarova <vmolnaro@redhat.com>

Perf archive has limited functionality and people from Red Hat Global
Support Services sent a request for a new feature that would pack
perf.data file together with an archive with debug symbols created by
the command 'perf archive' as customers were being confused and often
would forget to send perf.data file with the debug symbols.

Perf archive now accepts an option '--all' that generates archive
'perf.all-hostname-date-time.tar.bz2' that holds file 'perf.data' and
a sub-tar 'perf.symbols.tar.bz2' with debug symbols. The functionality of
the command 'perf archive' was not changed.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
---
 tools/perf/perf-archive.sh | 40 ++++++++++++++++++++++++++++++--------
 1 file changed, 32 insertions(+), 8 deletions(-)
 mode change 100644 => 100755 tools/perf/perf-archive.sh

diff --git a/tools/perf/perf-archive.sh b/tools/perf/perf-archive.sh
old mode 100644
new mode 100755
index 133f0eddbcc4..a92042eae95a
--- a/tools/perf/perf-archive.sh
+++ b/tools/perf/perf-archive.sh
@@ -4,9 +4,19 @@
 # Arnaldo Carvalho de Melo <acme@redhat.com>
 
 PERF_DATA=perf.data
-if [ $# -ne 0 ] ; then
-	PERF_DATA=$1
-fi
+PERF_SYMBOLS=perf.symbols
+PERF_ALL=perf.all
+ALL=0
+
+while [ $# -gt 0 ] ; do
+	if [ $1 == "--all" ]; then
+		ALL=1
+		shift
+	else
+		PERF_DATA=$1
+		shift
+	fi
+done
 
 #
 # PERF_BUILDID_DIR environment variable set by perf
@@ -39,9 +49,23 @@ while read build_id ; do
 	echo ${filename#$PERF_BUILDID_LINKDIR} >> $MANIFEST
 done
 
-tar cjf $PERF_DATA.tar.bz2 -C $PERF_BUILDID_DIR -T $MANIFEST
-rm $MANIFEST $BUILDIDS || true
-echo -e "Now please run:\n"
-echo -e "$ tar xvf $PERF_DATA.tar.bz2 -C ~/.debug\n"
-echo "wherever you need to run 'perf report' on."
+if [ $ALL -eq 1 ]; then						# pack perf.data file together with tar containing debug symbols
+	HOSTNAME=$(hostname)
+	DATE=$(date '+%Y%m%d-%H%M%S')
+	tar cjf $PERF_SYMBOLS.tar.bz2 -C $PERF_BUILDID_DIR -T $MANIFEST
+	tar cjf	$PERF_ALL-$HOSTNAME-$DATE.tar.bz2 $PERF_DATA $PERF_SYMBOLS.tar.bz2
+	rm $PERF_SYMBOLS.tar.bz2 $MANIFEST $BUILDIDS || true
+
+	echo -e "Now please run:\n"
+	echo -e "$ tar xvf $PERF_ALL-$HOSTNAME-$DATE.tar.bz2 && tar xvf $PERF_SYMBOLS.tar.bz2 -C ~/.debug\n"
+	echo "wherever you need to run 'perf report' on."
+else										# pack only the debug symbols
+	tar cjf $PERF_DATA.tar.bz2 -C $PERF_BUILDID_DIR -T $MANIFEST
+	rm $MANIFEST $BUILDIDS || true
+
+	echo -e "Now please run:\n"
+	echo -e "$ tar xvf $PERF_DATA.tar.bz2 -C ~/.debug\n"
+	echo "wherever you need to run 'perf report' on."
+fi
+
 exit 0
-- 
2.43.0

[PATCH 2/2] perf archive: Add new option '--unpack'

[vmolnaro]

From: Veronika Molnarova <vmolnaro@redhat.com>

Archives generated by the command 'perf archive' have to be unpacked
manually. Following the addition of option '--all' now there also exist
a nested structure of tars, and after further discussion with Red Hat
Global Support Services, they found a feature correctly unpacking
archives of 'perf archive' convenient.

Option '--unpack' of 'perf archive' unpacks archives generated by the
command 'perf archive' as well as archives generated when used with
option '--all'. File 'perf.data' is placed in the current directory,
while debug symbols are unpacked in '~/.debug' directory. A tar filename
can be passed as an argument, and if not provided the command tries to
find a viable perf.tar file for unpacking.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
---
 tools/perf/perf-archive.sh | 66 +++++++++++++++++++++++++++++++++-----
 1 file changed, 58 insertions(+), 8 deletions(-)

diff --git a/tools/perf/perf-archive.sh b/tools/perf/perf-archive.sh
index a92042eae95a..f94795794b36 100755
--- a/tools/perf/perf-archive.sh
+++ b/tools/perf/perf-archive.sh
@@ -7,17 +7,72 @@ PERF_DATA=perf.data
 PERF_SYMBOLS=perf.symbols
 PERF_ALL=perf.all
 ALL=0
+UNPACK=0
 
 while [ $# -gt 0 ] ; do
 	if [ $1 == "--all" ]; then
 		ALL=1
 		shift
+	elif [ $1 == "--unpack" ]; then
+		UNPACK=1
+		shift
 	else
 		PERF_DATA=$1
+		UNPACK_TAR=$1
 		shift
 	fi
 done
 
+if [ $UNPACK -eq 1 ]; then
+	if [ ! -z "$UNPACK_TAR" ]; then					# tar given as an argument
+		if [ ! -e "$UNPACK_TAR" ]; then
+			echo "Provided file $UNPACK_TAR does not exist"
+			exit 1
+		fi
+		TARGET="$UNPACK_TAR"
+	else																# search for perf tar in the current directory
+		TARGET=`find . -regex "\./perf.*\.tar\.bz2"`
+		TARGET_NUM=`echo -n "$TARGET" | grep -c '^'`
+
+		if [ -z "$TARGET" -o $TARGET_NUM -gt 1 ]; then
+			echo -e "Error: $TARGET_NUM files found for unpacking:\n$TARGET"
+			echo "Provide the requested file as an argument"
+			exit 1
+		else
+			echo "Found target file for unpacking: $TARGET"
+		fi
+	fi
+
+	if [[ "$TARGET" =~ (\./)?$PERF_ALL.*.tar.bz2 ]]; then				# perf tar generated by --all option
+		TAR_CONTENTS=`tar tvf "$TARGET" | tr -s " " | cut -d " " -f 6`
+		VALID_TAR=`echo "$TAR_CONTENTS" | grep "$PERF_SYMBOLS.tar.bz2" | wc -l`		# check if it contains a sub-tar perf.symbols
+		if [ $VALID_TAR -ne 1 ]; then
+			echo "Error: $TARGET file is not valid (contains zero or multiple sub-tar files with debug symbols)"
+			exit 1
+		fi
+
+		INTERSECT=`comm -12 <(ls) <(echo "$TAR_CONTENTS") | tr "\n" " "`	# check for overwriting
+		if [ ! -z "$INTERSECT" ]; then										# prompt if file(s) already exist in the current directory
+			echo "File(s) ${INTERSECT::-1} already exist in the current directory."
+			while true; do
+				read -p 'Do you wish to overwrite them? ' yn
+				case $yn in
+					[Yy]* ) break;;
+					[Nn]* ) exit 1;;
+					* ) echo "Please answer yes or no.";;
+				esac
+			done
+		fi
+
+		# unzip the perf.data file in the current working directory	and debug symbols in ~/.debug directory
+		tar xvf $TARGET && tar xvf $PERF_SYMBOLS.tar.bz2 -C ~/.debug
+
+	else																# perf tar generated by perf archive (contains only debug symbols)
+		tar xvf $TARGET -C ~/.debug
+	fi
+	exit 0
+fi
+
 #
 # PERF_BUILDID_DIR environment variable set by perf
 # path to buildid directory, default to $HOME/.debug
@@ -55,17 +110,12 @@ if [ $ALL -eq 1 ]; then						# pack perf.data file together with tar containing
 	tar cjf $PERF_SYMBOLS.tar.bz2 -C $PERF_BUILDID_DIR -T $MANIFEST
 	tar cjf	$PERF_ALL-$HOSTNAME-$DATE.tar.bz2 $PERF_DATA $PERF_SYMBOLS.tar.bz2
 	rm $PERF_SYMBOLS.tar.bz2 $MANIFEST $BUILDIDS || true
-
-	echo -e "Now please run:\n"
-	echo -e "$ tar xvf $PERF_ALL-$HOSTNAME-$DATE.tar.bz2 && tar xvf $PERF_SYMBOLS.tar.bz2 -C ~/.debug\n"
-	echo "wherever you need to run 'perf report' on."
 else										# pack only the debug symbols
 	tar cjf $PERF_DATA.tar.bz2 -C $PERF_BUILDID_DIR -T $MANIFEST
 	rm $MANIFEST $BUILDIDS || true
-
-	echo -e "Now please run:\n"
-	echo -e "$ tar xvf $PERF_DATA.tar.bz2 -C ~/.debug\n"
-	echo "wherever you need to run 'perf report' on."
 fi
 
+echo -e "Now please run:\n"
+echo -e "$ perf archive --unpack\n"
+echo "or unpack the tar manually wherever you need to run 'perf report' on."
 exit 0
-- 
2.43.0

Re: [PATCH 1/2] perf archive: Add new option '--all'

[Arnaldo Carvalho de Melo]

Em Tue, Dec 12, 2023 at 05:59:08PM +0100, vmolnaro@redhat.com escreveu:
> From: Veronika Molnarova <vmolnaro@redhat.com>
> 
> Perf archive has limited functionality and people from Red Hat Global
> Support Services sent a request for a new feature that would pack
> perf.data file together with an archive with debug symbols created by
> the command 'perf archive' as customers were being confused and often
> would forget to send perf.data file with the debug symbols.
> 
> Perf archive now accepts an option '--all' that generates archive
> 'perf.all-hostname-date-time.tar.bz2' that holds file 'perf.data' and
> a sub-tar 'perf.symbols.tar.bz2' with debug symbols. The functionality of
> the command 'perf archive' was not changed.

Thanks for working on this, I'll do some testing and apply, probably
tomorrow.

- Arnaldo
 
> Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
> ---
>  tools/perf/perf-archive.sh | 40 ++++++++++++++++++++++++++++++--------
>  1 file changed, 32 insertions(+), 8 deletions(-)
>  mode change 100644 => 100755 tools/perf/perf-archive.sh
> 
> diff --git a/tools/perf/perf-archive.sh b/tools/perf/perf-archive.sh
> old mode 100644
> new mode 100755
> index 133f0eddbcc4..a92042eae95a
> --- a/tools/perf/perf-archive.sh
> +++ b/tools/perf/perf-archive.sh
> @@ -4,9 +4,19 @@
>  # Arnaldo Carvalho de Melo <acme@redhat.com>
>  
>  PERF_DATA=perf.data
> -if [ $# -ne 0 ] ; then
> -	PERF_DATA=$1
> -fi
> +PERF_SYMBOLS=perf.symbols
> +PERF_ALL=perf.all
> +ALL=0
> +
> +while [ $# -gt 0 ] ; do
> +	if [ $1 == "--all" ]; then
> +		ALL=1
> +		shift
> +	else
> +		PERF_DATA=$1
> +		shift
> +	fi
> +done
>  
>  #
>  # PERF_BUILDID_DIR environment variable set by perf
> @@ -39,9 +49,23 @@ while read build_id ; do
>  	echo ${filename#$PERF_BUILDID_LINKDIR} >> $MANIFEST
>  done
>  
> -tar cjf $PERF_DATA.tar.bz2 -C $PERF_BUILDID_DIR -T $MANIFEST
> -rm $MANIFEST $BUILDIDS || true
> -echo -e "Now please run:\n"
> -echo -e "$ tar xvf $PERF_DATA.tar.bz2 -C ~/.debug\n"
> -echo "wherever you need to run 'perf report' on."
> +if [ $ALL -eq 1 ]; then						# pack perf.data file together with tar containing debug symbols
> +	HOSTNAME=$(hostname)
> +	DATE=$(date '+%Y%m%d-%H%M%S')
> +	tar cjf $PERF_SYMBOLS.tar.bz2 -C $PERF_BUILDID_DIR -T $MANIFEST
> +	tar cjf	$PERF_ALL-$HOSTNAME-$DATE.tar.bz2 $PERF_DATA $PERF_SYMBOLS.tar.bz2
> +	rm $PERF_SYMBOLS.tar.bz2 $MANIFEST $BUILDIDS || true
> +
> +	echo -e "Now please run:\n"
> +	echo -e "$ tar xvf $PERF_ALL-$HOSTNAME-$DATE.tar.bz2 && tar xvf $PERF_SYMBOLS.tar.bz2 -C ~/.debug\n"
> +	echo "wherever you need to run 'perf report' on."
> +else										# pack only the debug symbols
> +	tar cjf $PERF_DATA.tar.bz2 -C $PERF_BUILDID_DIR -T $MANIFEST
> +	rm $MANIFEST $BUILDIDS || true
> +
> +	echo -e "Now please run:\n"
> +	echo -e "$ tar xvf $PERF_DATA.tar.bz2 -C ~/.debug\n"
> +	echo "wherever you need to run 'perf report' on."
> +fi
> +
>  exit 0
> -- 
> 2.43.0
> 

-- 

- Arnaldo

Re: [PATCH 1/2] perf archive: Add new option '--all'

[Arnaldo Carvalho de Melo]

Em Tue, Dec 12, 2023 at 03:06:47PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Tue, Dec 12, 2023 at 05:59:08PM +0100, vmolnaro@redhat.com escreveu:
> > From: Veronika Molnarova <vmolnaro@redhat.com>
> > 
> > Perf archive has limited functionality and people from Red Hat Global
> > Support Services sent a request for a new feature that would pack
> > perf.data file together with an archive with debug symbols created by
> > the command 'perf archive' as customers were being confused and often
> > would forget to send perf.data file with the debug symbols.
> > 
> > Perf archive now accepts an option '--all' that generates archive
> > 'perf.all-hostname-date-time.tar.bz2' that holds file 'perf.data' and
> > a sub-tar 'perf.symbols.tar.bz2' with debug symbols. The functionality of
> > the command 'perf archive' was not changed.
> 
> Thanks for working on this, I'll do some testing and apply, probably
> tomorrow.

I applied it, tested by doing a 'perf record' on an hybrid Intel Machine
(14700k) and then using --unpack on an hybrid ARM64 machine (rk3399-pc),
all working as expected:

https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/commit/?h=tmp.perf-tools-next&id=624dda101e03c3a3a155d51e37a7bb7607cb760b

Some suggestions for further improvements:

acme@roc-rk3399-pc:~$ perf archive --unpack
Found target file for unpacking: ./perf.all-number-20231219-104854.tar.bz2
perf.data
perf.symbols.tar.bz2
tar: /home/acme/.debug: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
acme@roc-rk3399-pc:~$

Check if the ~/.debug directory is present, if not, create it.

Also maybe use:

   Informative output
       --checkpoint[=N]
              Display progress messages every Nth record (default 10).

       --checkpoint-action=ACTION
              Run ACTION on each checkpoint.

with 'tar' to show some progress in decompressing the tarball.

- Arnaldo