[PATCH 0/4] tools: checkman.sh portability

[PATCH 0/4] tools: checkman.sh portability

[JWP]

So, Sami Kerola's checkmans.sh tool caught me stuttering in
hwclock.8 and I had to find out why checkmans did not work
on my system:

 1) Any test file with 'git' or 'lib' in its path fails.
    Including my entire repo under $PATH/github/util-linux
    and the ./lib/terminal-colors.d.5 man-page.

 2) It depends on the man-db package.

The changes seem to be an improvement as it detected 4 new
man-page problems (see separate patch).

I did alter the behavior slightly, previously failed pages
required checking them again manually. Now it shows the troff
parsing errors in progress. When grog fails there is nothing
more to check, and when lexgrog fails I left the old behavior
telling the caller to rerun the command, because I did not
bother to install man-db to see what lexgrog actually does.


The following changes since commit 2e5bf10e32b00c92a3f7fb8b38d793ea6fc1dff2:

  build-sys: add sfdisk.static (2015-01-13 16:59:03 +0100)

are available in the git repository at:

  git@github.com:jwpi/util-linux.git checkman

J William Piggott (4):
  tools: checkman.sh portability
  tools: checkman.sh portability lexgrog
  tools: checkmans.sh clean up 'let'
  tools: checkmans.sh add stats

 tools/checkmans.sh | 85 +++++++++++++++++++++++++++++++++---------------------
 1 file changed, 52 insertions(+), 33 deletions(-)

[PATCH 1/4] tools: checkman.sh portability

[JWP]

checkman.sh fails if 'git' or 'lib' are
anywhere in the tested file's path:
   Change to using relative paths.
   Improve the 'find' command call.

checkman.sh depends upon the man-db package:
   Remove all instants of the man command and
   use troff directly to improve portability.

Signed-off-by: J William Piggott <elseifthen@gmx.com>
---
 tools/checkmans.sh | 26 ++++++++++++--------------
 1 file changed, 12 insertions(+), 14 deletions(-)

diff --git a/tools/checkmans.sh b/tools/checkmans.sh
index bc3ba8f..6b80853 100755
--- a/tools/checkmans.sh
+++ b/tools/checkmans.sh
@@ -39,10 +39,6 @@ while getopts vVh OPTIONS; do
 	esac
 done
 
-ERROR_FILE=$(mktemp ${SCRIPT_INVOCATION_SHORT_NAME}.XXXXXXXXXX)
-# remove tmp file at exit
-trap "rm -f ${ERROR_FILE}" 0
-
 # Try to find missing manuals matching build targets with manual files.
 declare -A MAN_LIST BIN_LIST
 
@@ -68,9 +64,10 @@ remove_repeats()
 	set -u
 }
 
+cd $(git rev-parse --show-toplevel)
+
 for I in $(
-	find $(git rev-parse --show-toplevel) -name '*.[1-8]' |
-	egrep -v '(Documentation|.git|/.libs/|autom4te.cache)'
+	find -path './autom4te.cache' -prune -o -name '*[[:alpha:]].[1-8]' -print
 ); do
 	MAN_FILE=${I##*/}
 	MAN_LIST[${MAN_FILE%%.[0-9]}]=1
@@ -78,7 +75,7 @@ for I in $(
 		# Some manuals, such as x86_64, call inclusion and they
 		# should not be tested any further.
 		if ${VERBOSE}; then
-			printf "skipping: ${I##*util-linux/}: includes "
+			printf "skipping: ${I}: includes "
 			awk '{print $2}' ${I}
 		fi
 		continue
@@ -87,16 +84,17 @@ for I in $(
 	if ${VERBOSE}; then
 		echo "testing: ${I}"
 	fi
-	MANWIDTH=80 man --warnings=all ${I} >/dev/null 2>| ${ERROR_FILE}
-	if [ -s ${ERROR_FILE} ]; then
-		echo "error: run: man --warnings=all ${I##*util-linux/} >/dev/null" >&2
-		I_ERR=1
+	RET=1
+	cat ${I} | troff -mandoc -ww -z |& grep "<" && RET=$?
+	if [ $RET = 0 ]; then
+	echo "From: cat ${I} | troff -mandoc -ww -z"
+	echo "=================================================="
 	fi
 	if ! lexgrog ${I} >/dev/null; then
-		echo "error: run: lexgrog ${I##*util-linux/}" >&2
+		echo "error: run: lexgrog ${I}" >&2
 		I_ERR=1
 	fi
-	REPEATS=( $(MANWIDTH=2000 man -l ${I} |
+	REPEATS=( $(cat ${I} | troff -mandoc -ww -Tascii | grotty |
 		col -b |
 		sed  -e 's/\s\+/\n/g; /^$/d' |
 		awk 'BEGIN { p="" } { if (0 < length($0)) { if (p == $0) { print } } p = $0 }') )
@@ -109,7 +107,7 @@ for I in $(
 			let ITER=${ITER}-1 || true
 		done
 		if [ 0 -lt "${#REPEATS[@]}" ]; then
-			echo "warning: ${I##*util-linux/} has repeating words: ${REPEATS[@]}"
+			echo "warning: ${I} has repeating words: ${REPEATS[@]}"
 		fi
 	fi
 	# The 'let' may cause exit on error.

[PATCH 2/4] tools: checkman.sh portability lexgrog

[JWP]

checkman.sh depends upon the man-db package:
   Add using grog when lexgrog not installed.

   /dev/null some troff noise.

Signed-off-by: J William Piggott <elseifthen@gmx.com>
---
 tools/checkmans.sh | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)

diff --git a/tools/checkmans.sh b/tools/checkmans.sh
index 6b80853..513a750 100755
--- a/tools/checkmans.sh
+++ b/tools/checkmans.sh
@@ -90,11 +90,23 @@ for I in $(
 	echo "From: cat ${I} | troff -mandoc -ww -z"
 	echo "=================================================="
 	fi
-	if ! lexgrog ${I} >/dev/null; then
-		echo "error: run: lexgrog ${I}" >&2
-		I_ERR=1
+	GROG=1
+	if command -v lexgrog &> /dev/null; then
+		if ! lexgrog ${I} >/dev/null; then
+			echo "error: run: lexgrog ${I}"
+			echo "=================================================="
+			((++COUNT_ERRORS))
+		fi
+	elif command -v grog &> /dev/null; then
+		if ! grog ${I} | grep man >/dev/null; then
+			echo "error: grog ${I} is not a man file"
+			echo "=================================================="
+			((++COUNT_ERRORS))
+		fi
+	else
+	GROG=0
 	fi
-	REPEATS=( $(cat ${I} | troff -mandoc -ww -Tascii | grotty |
+	REPEATS=( $(cat ${I} | troff -mandoc -Tascii 2>/dev/null | grotty |
 		col -b |
 		sed  -e 's/\s\+/\n/g; /^$/d' |
 		awk 'BEGIN { p="" } { if (0 < length($0)) { if (p == $0) { print } } p = $0 }') )
@@ -147,8 +159,12 @@ for I in ${!BIN_LIST[@]}; do
 done
 set -u
 
+if [ ${GROG} = 0 ]; then
+echo "warning: neither grog nor lexgrog commands were found"
+fi
+
 if [ ${COUNT_ERRORS} -ne 0 ]; then
-	echo "error: ${SCRIPT_INVOCATION_SHORT_NAME}: ${COUNT_ERRORS} manuals failed" >&2
+	echo "error: ${SCRIPT_INVOCATION_SHORT_NAME}: ${COUNT_ERRORS} manuals failed"
 	exit 1
 fi
 

[PATCH 3/4] tools: checkmans.sh clean up 'let'

[JWP]

Clean up 'let' issues and loop iteration.

Signed-off-by: J William Piggott <elseifthen@gmx.com>
---
 tools/checkmans.sh | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/tools/checkmans.sh b/tools/checkmans.sh
index 513a750..bd2bc43 100755
--- a/tools/checkmans.sh
+++ b/tools/checkmans.sh
@@ -42,7 +42,8 @@ done
 # Try to find missing manuals matching build targets with manual files.
 declare -A MAN_LIST BIN_LIST
 
-COUNT_ERRORS=0
+declare -i ITER
+declare -i COUNT_ERRORS=0
 declare -a REPEATS
 declare -A KNOWN_REPEATS
 KNOWN_REPEATS[mount.8]='foo'
@@ -111,21 +112,14 @@ for I in $(
 		sed  -e 's/\s\+/\n/g; /^$/d' |
 		awk 'BEGIN { p="" } { if (0 < length($0)) { if (p == $0) { print } } p = $0 }') )
 	if [ 0 -lt "${#REPEATS[@]}" ]; then
-		ITER=${#REPEATS[@]}
-		while [ -1 -lt ${ITER} ]; do
+		ITER=${#REPEATS[@]}+1
+		while ((ITER--)); do
 			remove_repeats ${ITER}
-			# The 'let' may cause exit on error.
-			# When ITER == 0 -> let returns 1, bash bug?
-			let ITER=${ITER}-1 || true
 		done
 		if [ 0 -lt "${#REPEATS[@]}" ]; then
 			echo "warning: ${I} has repeating words: ${REPEATS[@]}"
 		fi
 	fi
-	# The 'let' may cause exit on error.
-	# COUNT_ERRORS=0; let COUNT_ERRORS=$COUNT_ERRORS+0; echo $?
-	# Is this a bash bug?
-	let COUNT_ERRORS=$COUNT_ERRORS+$I_ERR || true
 done
 
 # Create a list of build targets.

[PATCH 4/4] tools: checkmans.sh add stats

[JWP]

Add test results statistics.
Fix-up 'KNOWN_REPEATS'.
Version bump.

Signed-off-by: J William Piggott <elseifthen@gmx.com>
---
 tools/checkmans.sh | 33 ++++++++++++++++++++++-----------
 1 file changed, 22 insertions(+), 11 deletions(-)

diff --git a/tools/checkmans.sh b/tools/checkmans.sh
index bd2bc43..618f8ba 100755
--- a/tools/checkmans.sh
+++ b/tools/checkmans.sh
@@ -26,7 +26,7 @@ while getopts vVh OPTIONS; do
 			VERBOSE='true'
 			;;
 		V)
-			echo "util-linux: ${SCRIPT_INVOCATION_SHORT_NAME}: 2"
+			echo "util-linux: ${SCRIPT_INVOCATION_SHORT_NAME}: v2.1"
 			exit 0
 			;;
 		h)
@@ -43,11 +43,13 @@ done
 declare -A MAN_LIST BIN_LIST
 
 declare -i ITER
-declare -i COUNT_ERRORS=0
+declare -i COUNT_GROG=0
+declare -i COUNT_TROFF=0
+declare -i COUNT_REPEATS=0
 declare -a REPEATS
 declare -A KNOWN_REPEATS
-KNOWN_REPEATS[mount.8]='foo'
-KNOWN_REPEATS[sfdisk.8]="0 <c,h,s>"
+KNOWN_REPEATS[mount.8]='foo l2 l c'
+KNOWN_REPEATS[hexdump.1]='l'
 KNOWN_REPEATS[flock.1]='"$0"'
 KNOWN_REPEATS[switch_root.8]='$DIR'
 
@@ -81,7 +83,6 @@ for I in $(
 		fi
 		continue
 	fi
-	I_ERR=0
 	if ${VERBOSE}; then
 		echo "testing: ${I}"
 	fi
@@ -90,22 +91,23 @@ for I in $(
 	if [ $RET = 0 ]; then
 	echo "From: cat ${I} | troff -mandoc -ww -z"
 	echo "=================================================="
+	((++COUNT_TROFF))
 	fi
 	GROG=1
 	if command -v lexgrog &> /dev/null; then
 		if ! lexgrog ${I} >/dev/null; then
 			echo "error: run: lexgrog ${I}"
 			echo "=================================================="
-			((++COUNT_ERRORS))
+			((++COUNT_GROG))
 		fi
 	elif command -v grog &> /dev/null; then
 		if ! grog ${I} | grep man >/dev/null; then
 			echo "error: grog ${I} is not a man file"
 			echo "=================================================="
-			((++COUNT_ERRORS))
+			((++COUNT_GROG))
 		fi
 	else
-	GROG=0
+		GROG=0
 	fi
 	REPEATS=( $(cat ${I} | troff -mandoc -Tascii 2>/dev/null | grotty |
 		col -b |
@@ -118,6 +120,8 @@ for I in $(
 		done
 		if [ 0 -lt "${#REPEATS[@]}" ]; then
 			echo "warning: ${I} has repeating words: ${REPEATS[@]}"
+			echo "=================================================="
+			((++COUNT_REPEATS))
 		fi
 	fi
 done
@@ -148,17 +152,24 @@ done
 set +u
 for I in ${!BIN_LIST[@]}; do
 	if [ -v ${MAN_LIST[$I]} ]; then
-		echo "warning: ${I} does not have man page"
+		echo "warning: ${SCRIPT_INVOCATION_SHORT_NAME}: ${I} does not have man page"
 	fi
 done
 set -u
 
+echo "${SCRIPT_INVOCATION_SHORT_NAME}: ${#BIN_LIST[*]} build targets were found"
+echo "${SCRIPT_INVOCATION_SHORT_NAME}: ${#MAN_LIST[*]} files were tested"
+
 if [ ${GROG} = 0 ]; then
 echo "warning: neither grog nor lexgrog commands were found"
 fi
 
-if [ ${COUNT_ERRORS} -ne 0 ]; then
-	echo "error: ${SCRIPT_INVOCATION_SHORT_NAME}: ${COUNT_ERRORS} manuals failed"
+if [ ${COUNT_GROG} -ne 0 -o ${COUNT_TROFF} -ne 0 -o ${COUNT_REPEATS} -ne 0 ]; then
+	echo "error: ${SCRIPT_INVOCATION_SHORT_NAME}: ${COUNT_GROG} files failed (lex)grog man-page test"
+	echo "error: ${SCRIPT_INVOCATION_SHORT_NAME}: ${COUNT_TROFF} files failed troff parsing test"
+	echo "error: ${SCRIPT_INVOCATION_SHORT_NAME}: ${COUNT_REPEATS} files have repeating words"
+	ITER=${#MAN_LIST[*]}-${COUNT_GROG}-${COUNT_TROFF}-${COUNT_REPEATS}
+	echo "${SCRIPT_INVOCATION_SHORT_NAME}: ${ITER} man-pages approved"
 	exit 1
 fi
 

Re: [PATCH 0/4] tools: checkman.sh portability

[Karel Zak]

On Tue, Jan 13, 2015 at 01:46:58PM -0500, JWP wrote:
> J William Piggott (4):
>   tools: checkman.sh portability
>   tools: checkman.sh portability lexgrog
>   tools: checkmans.sh clean up 'let'
>   tools: checkmans.sh add stats

 Merged, thanks.
> 

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com