[PATCH i-g-t] scripts/doc-tests-cross-check: allow whitespace in keywords

[PATCH i-g-t] scripts/doc-tests-cross-check: allow whitespace in keywords

[Kamil Konieczny]

When main field name had whitespace, script incorrectly split it
into two separate words, for example when used with
"Mega feature" it treats it as two words: "Mega" and "feature".
This is not what we want it so fix it, also use "Mega feature"
in main call.

Also while at this, add additional verification step that
conflicting words are consistent and improve example of usage.

Cc: Katarzyna Piecielska <katarzyna.piecielska@intel.com>
Signed-off-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>
---
 scripts/doc-tests-cross-check.sh | 33 ++++++++++++++++++++++++--------
 1 file changed, 25 insertions(+), 8 deletions(-)

diff --git a/scripts/doc-tests-cross-check.sh b/scripts/doc-tests-cross-check.sh
index 174b14807..3f1dc7a83 100755
--- a/scripts/doc-tests-cross-check.sh
+++ b/scripts/doc-tests-cross-check.sh
@@ -34,10 +34,10 @@ grep_test ()
 	myout=$2
 
 	echo "running: $myout file: $test"
-	grep $myout: $test | sed -e "s/...$2: //" \
+	grep -E $myout: $test | sed -e "s/...$2: //" \
 		| tr '[:upper:]' '[:lower:]' | sed -e 's/, /\n/g' \
 		| sed -e 's/ /Y/g' >> b.$myout
-	grep -r $myout: ../tests/* >> n.$myout
+	grep -E -r $myout: ../tests/* >> n.$myout
 }
 
 
@@ -61,8 +61,12 @@ scan_dirs ()
 {
 	rm a.columns
 
-	for todo in $@; do
-		echo $todo >> a.columns
+        for todo in "$@"; do
+                echo "$todo" | sed -e 's/ /./g' >> a.columns
+	done
+
+	SCANWORDS=`cat a.columns`
+	for todo in $SCANWORDS; do
 		echo "Scanning name: $todo"
 		one_scan_dir $todo
 	done
@@ -89,7 +93,20 @@ check_test ()
 		grep -i -w "$test" w.$1 >> .tmp_check
 	done
 
-	sort -u < .tmp_check > e.$1
+	sort -u < .tmp_check > .tmp_ck2
+
+	# it is a match only when is a column in our original c.$1
+	# as a word could be shorter and grep will match it then
+	rm .tmp_ck3
+	CKWORDS=`cat .tmp_ck2`
+	for test in $CKWORDS; do
+		grep -E -i -w "$test" c.$1
+		if [ $? -eq 0 ]; then
+			echo $test >> .tmp_ck3
+		fi
+	done
+
+	sort -u < .tmp_ck3 > e.$1
 }
 
 # will use .tmpcols and create w.$1
@@ -134,7 +151,7 @@ echo "=============================================="
 
 # Mega feature / Category / Sub-category / Functionality
 # Mega-feature is not used now
-scan_dirs Category Sub-category Functionality
+scan_dirs Category Sub-category Functionality "Mega feature"
 
 echo "=============================================="
 echo "checking..."
@@ -153,8 +170,8 @@ wc -l e.[A-Za-z]*
 echo ""
 echo "=============================================="
 echo "example useage: take a word from e.* file and grep it in c*"
-echo "grep -i word c.*"
+echo "grep -E -i word c.*"
 echo "to see where it is used in tests, use:"
-echo "grep -i word n.*"
+echo "grep -E -i ':.*word' n.*"
 echo "=============================================="
 
-- 
2.42.0

RE: [PATCH i-g-t] scripts/doc-tests-cross-check: allow whitespace in keywords

[Piecielska, Katarzyna]

I used the tool and it is working as expected.
Reviewed-by: Katarzyna Piecielska <katarzyna.piecielska@intel.com>

Kasia

-----Original Message-----
From: Kamil Konieczny <kamil.konieczny@linux.intel.com> 
Sent: Thursday, April 4, 2024 7:42 PM
To: igt-dev@lists.freedesktop.org
Cc: Kamil Konieczny <kamil.konieczny@linux.intel.com>; Piecielska, Katarzyna <katarzyna.piecielska@intel.com>
Subject: [PATCH i-g-t] scripts/doc-tests-cross-check: allow whitespace in keywords

When main field name had whitespace, script incorrectly split it into two separate words, for example when used with "Mega feature" it treats it as two words: "Mega" and "feature".
This is not what we want it so fix it, also use "Mega feature"
in main call.

Also while at this, add additional verification step that conflicting words are consistent and improve example of usage.

Cc: Katarzyna Piecielska <katarzyna.piecielska@intel.com>
Signed-off-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>
---
 scripts/doc-tests-cross-check.sh | 33 ++++++++++++++++++++++++--------
 1 file changed, 25 insertions(+), 8 deletions(-)

diff --git a/scripts/doc-tests-cross-check.sh b/scripts/doc-tests-cross-check.sh
index 174b14807..3f1dc7a83 100755
--- a/scripts/doc-tests-cross-check.sh
+++ b/scripts/doc-tests-cross-check.sh
@@ -34,10 +34,10 @@ grep_test ()
 	myout=$2
 
 	echo "running: $myout file: $test"
-	grep $myout: $test | sed -e "s/...$2: //" \
+	grep -E $myout: $test | sed -e "s/...$2: //" \
 		| tr '[:upper:]' '[:lower:]' | sed -e 's/, /\n/g' \
 		| sed -e 's/ /Y/g' >> b.$myout
-	grep -r $myout: ../tests/* >> n.$myout
+	grep -E -r $myout: ../tests/* >> n.$myout
 }
 
 
@@ -61,8 +61,12 @@ scan_dirs ()
 {
 	rm a.columns
 
-	for todo in $@; do
-		echo $todo >> a.columns
+        for todo in "$@"; do
+                echo "$todo" | sed -e 's/ /./g' >> a.columns
+	done
+
+	SCANWORDS=`cat a.columns`
+	for todo in $SCANWORDS; do
 		echo "Scanning name: $todo"
 		one_scan_dir $todo
 	done
@@ -89,7 +93,20 @@ check_test ()
 		grep -i -w "$test" w.$1 >> .tmp_check
 	done
 
-	sort -u < .tmp_check > e.$1
+	sort -u < .tmp_check > .tmp_ck2
+
+	# it is a match only when is a column in our original c.$1
+	# as a word could be shorter and grep will match it then
+	rm .tmp_ck3
+	CKWORDS=`cat .tmp_ck2`
+	for test in $CKWORDS; do
+		grep -E -i -w "$test" c.$1
+		if [ $? -eq 0 ]; then
+			echo $test >> .tmp_ck3
+		fi
+	done
+
+	sort -u < .tmp_ck3 > e.$1
 }
 
 # will use .tmpcols and create w.$1
@@ -134,7 +151,7 @@ echo "=============================================="
 
 # Mega feature / Category / Sub-category / Functionality  # Mega-feature is not used now -scan_dirs Category Sub-category Functionality
+scan_dirs Category Sub-category Functionality "Mega feature"
 
 echo "=============================================="
 echo "checking..."
@@ -153,8 +170,8 @@ wc -l e.[A-Za-z]*
 echo ""
 echo "=============================================="
 echo "example useage: take a word from e.* file and grep it in c*"
-echo "grep -i word c.*"
+echo "grep -E -i word c.*"
 echo "to see where it is used in tests, use:"
-echo "grep -i word n.*"
+echo "grep -E -i ':.*word' n.*"
 echo "=============================================="
 
--
2.42.0