[igt-dev] [PATCH 0/6] Fix some issues at gcov parse tool and add a new table on reports

[igt-dev] [PATCH 0/6] Fix some issues at gcov parse tool and add a new table on reports

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Patches 1 to 4 fix some bugs at the script, some of them causing it to crash.

Patch 5 address a GCC optimization-related issue, where the function is
renamed at the object file, gaining a .irda.<number> extension.

Patch 6 adds a new functionality at the tool when producing an HTML output.
It basically calculates the interception of all code coverage sets, and shows
how many functions are common to all sets and with one are unique to an
specific tests. It helps to discover cases where tests or test sets are not needed
to be taken into account when producing a report.

Mauro Carvalho Chehab (6):
  scripts/code_cov_parse_info: avoid warning messages on parsing
  scripts/code_cov_parse_info: fix --only-drm filtering rules
  scripts/code_cov_parse_info: fix brf record handling
  scripts/code_cov_parse_info: Fix indentation at the output html file
  scripts/code_cov_parse_info: take gcc IPA optimizations into account
  scripts/code_cov_parse_info: print common coverage and extra per
    function

 scripts/code_cov_parse_info | 111 ++++++++++++++++++++++++++++++++----
 1 file changed, 99 insertions(+), 12 deletions(-)

-- 
2.37.2

[igt-dev] [PATCH 1/6] scripts/code_cov_parse_info: avoid warning messages on parsing

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Some fields (FNH, BRH, LH) produce warning messages when they're found
at the info file.

Fix it.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---

To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover.
See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1663836123.git.mchehab@kernel.org/

 scripts/code_cov_parse_info | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/scripts/code_cov_parse_info b/scripts/code_cov_parse_info
index 09f9aba5760b..78b7e2dada5e 100755
--- a/scripts/code_cov_parse_info
+++ b/scripts/code_cov_parse_info
@@ -195,7 +195,7 @@ sub parse_info_data($)
 		# FNH:<number of function hit>
 		if (m/^FNH:(-?\d+)/) {
 			my $hits = $1;
-			if ($record{$source}{$func}{fnh} < $hits) {
+			if (!defined($record{$source}{$func}{fnh}) || $record{$source}{$func}{fnh} < $hits) {
 				$record{$source}{$func}{fnh} = $hits;
 			}
 			next;
@@ -234,7 +234,7 @@ sub parse_info_data($)
 		# BRH:<number of branches hit>
 		if (m/^BRH:(-?\d+)/) {
 			my $hits = $1;
-			if ($record{$source}{$func}{brh} < $hits) {
+			if (!defined($record{$source}{$func}{brh}) || $record{$source}{$func}{brh} < $hits) {
 				$record{$source}{$func}{brh} = $hits;
 			}
 			next;
@@ -268,7 +268,7 @@ sub parse_info_data($)
 		# LH:<number of lines with a non-zero execution count>
 		if (m/^LH:(-?\d+)/) {
 			my $hits = $1;
-			if ($record{$source}{$func}{lh} < $hits) {
+			if (!defined($record{$source}{$func}{lh}) || $record{$source}{$func}{lh} < $hits) {
 				$record{$source}{$func}{lh} = $hits;
 			}
 			next;
-- 
2.37.2

[igt-dev] [PATCH 2/6] scripts/code_cov_parse_info: fix --only-drm filtering rules

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

The --only-drm filtering rules were filtering too much, ending to
exclude functions on files with "h" on their names. Fix it.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---

To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover.
See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1663836123.git.mchehab@kernel.org/

 scripts/code_cov_parse_info | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/scripts/code_cov_parse_info b/scripts/code_cov_parse_info
index 78b7e2dada5e..037a07e26b1d 100755
--- a/scripts/code_cov_parse_info
+++ b/scripts/code_cov_parse_info
@@ -857,8 +857,8 @@ if ($only_i915) {
 
 if ($only_drm) {
 	# Please keep in sync with the documentation
-	push @src_exclude_regexes, "trace.*\.h";
-	push @src_exclude_regexes, "drm.*\.h";
+	push @src_exclude_regexes, "trace.*\.h\$";
+	push @src_exclude_regexes, "^/drm/";
 }
 
 $str = open_filter_file($func_filters, \@func_regexes, \@func_exclude_regexes);
@@ -1051,7 +1051,9 @@ E. g. it will exclude *.h files that match the following regular expressions:
 
 And *.h files that don't match:
 
-	- drm
+	- /drm/
+
+(e. g. macros and other code outside DRM subsystem)
 
 =item B<--only-i915> or B<--only_i915>
 
-- 
2.37.2

[igt-dev] [PATCH 3/6] scripts/code_cov_parse_info: fix brf record handling

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

Don't override $record{source} due to BRF, or the script will crash.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---

To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover.
See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1663836123.git.mchehab@kernel.org/

 scripts/code_cov_parse_info | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/code_cov_parse_info b/scripts/code_cov_parse_info
index 037a07e26b1d..ef26192f2456 100755
--- a/scripts/code_cov_parse_info
+++ b/scripts/code_cov_parse_info
@@ -228,7 +228,7 @@ sub parse_info_data($)
 
 		# BRF:<number of branches found>
 		if (m/^BRF:(-?\d+)/) {
-			$record{$source}{brf} = $1;
+			$record{$source}{$func}{brf} = $1;
 			next;
 		}
 		# BRH:<number of branches hit>
-- 
2.37.2

[igt-dev] [PATCH 4/6] scripts/code_cov_parse_info: Fix indentation at the output html file

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

The indentation is not relevant for the output, but it makes easier
when debugging issues at the produced file. So, fix it.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---

To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover.
See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1663836123.git.mchehab@kernel.org/

 scripts/code_cov_parse_info | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/scripts/code_cov_parse_info b/scripts/code_cov_parse_info
index ef26192f2456..e8bac730e88b 100755
--- a/scripts/code_cov_parse_info
+++ b/scripts/code_cov_parse_info
@@ -650,7 +650,7 @@ sub generate_report($)
 	} else {
 		print OUT "    <td>N. A.</td>\n";
 	}
-	print OUT "  <td>" . $stats{"func_count"} . "</td>";
+	print OUT "    <td>" . $stats{"func_count"} . "</td>\n";
 	print OUT "  </tr><tr>\n";
 
 	print OUT "    <td><b>Branches</b></td>\n";
@@ -671,7 +671,7 @@ sub generate_report($)
 	} else {
 		print OUT "    <td>N. A.</td>\n";
 	}
-	print OUT "  <td>" . $stats{"branch_count"} . "</td>";
+	print OUT "    <td>" . $stats{"branch_count"} . "</td>\n";
 	print OUT "  </tr><tr>\n";
 
 	print OUT "    <td><b>Lines</b></td>\n";
@@ -693,7 +693,7 @@ sub generate_report($)
 	} else {
 		print OUT "    <td>N. A.</td>\n";
 	}
-	print OUT "  <td>" . $stats{"line_count"} . "</td>";
+	print OUT "    <td>" . $stats{"line_count"} . "</td>\n";
 
 	# If there are more than one tests per file, report them
 	my $total = scalar(keys %test_names);
-- 
2.37.2

Re: [igt-dev] [PATCH 4/6] scripts/code_cov_parse_info: Fix indentation at the output html file

[Kamil Konieczny]

Hi Mauro,

On 2022-09-22 at 10:46:16 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
> 
> The indentation is not relevant for the output, but it makes easier
> when debugging issues at the produced file. So, fix it.
--------------------------------------------- ^^^^^^^^^^
Please drop this from commit message,
rest looks good so

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>

> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
> ---
> 
> To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover.
> See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1663836123.git.mchehab@kernel.org/
> 
>  scripts/code_cov_parse_info | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/scripts/code_cov_parse_info b/scripts/code_cov_parse_info
> index ef26192f2456..e8bac730e88b 100755
> --- a/scripts/code_cov_parse_info
> +++ b/scripts/code_cov_parse_info
> @@ -650,7 +650,7 @@ sub generate_report($)
>  	} else {
>  		print OUT "    <td>N. A.</td>\n";
>  	}
> -	print OUT "  <td>" . $stats{"func_count"} . "</td>";
> +	print OUT "    <td>" . $stats{"func_count"} . "</td>\n";
>  	print OUT "  </tr><tr>\n";
>  
>  	print OUT "    <td><b>Branches</b></td>\n";
> @@ -671,7 +671,7 @@ sub generate_report($)
>  	} else {
>  		print OUT "    <td>N. A.</td>\n";
>  	}
> -	print OUT "  <td>" . $stats{"branch_count"} . "</td>";
> +	print OUT "    <td>" . $stats{"branch_count"} . "</td>\n";
>  	print OUT "  </tr><tr>\n";
>  
>  	print OUT "    <td><b>Lines</b></td>\n";
> @@ -693,7 +693,7 @@ sub generate_report($)
>  	} else {
>  		print OUT "    <td>N. A.</td>\n";
>  	}
> -	print OUT "  <td>" . $stats{"line_count"} . "</td>";
> +	print OUT "    <td>" . $stats{"line_count"} . "</td>\n";
>  
>  	# If there are more than one tests per file, report them
>  	my $total = scalar(keys %test_names);
> -- 
> 2.37.2
> 

[igt-dev] [PATCH 5/6] scripts/code_cov_parse_info: take gcc IPA optimizations into account

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

When -O2/-O3 is used with GCC, IPA will produce function variants
at the object code. So, for a function or macro foo(), it may
produce several actual functions at the object code, like:

	foo.isra.3
	foo.isra.4

For the purposes of code coverage, all of those are the same
function, so use a regex with "\w+", which will exclude the
.isra.<number>, considering all variants as the same function.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---

To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover.
See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1663836123.git.mchehab@kernel.org/

 scripts/code_cov_parse_info | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/scripts/code_cov_parse_info b/scripts/code_cov_parse_info
index e8bac730e88b..7f3e1f9ef116 100755
--- a/scripts/code_cov_parse_info
+++ b/scripts/code_cov_parse_info
@@ -136,7 +136,9 @@ sub parse_info_data($)
 		# Function coverage
 
 		# FN:<line number of function start>,<function name>
-		if (m/^FN:(-?\d+),(.*)/) {
+		# Note: /w+ intentionally removes IPA gcc-optimization names, as
+		# this is more related to branch coverage
+		if (m/^FN:(-?\d+),(\w+)/) {
 			my $ln = $1;
 
 			$func = $2;
@@ -156,7 +158,9 @@ sub parse_info_data($)
 
 		# Parse functions that were actually used
 		# FNDA:<execution count>,<function name>
-		if (m/^FNDA:(-?\d+),(.*)/) {
+		# Note: /w+ intentionally removes IPA gcc-optimization names, as
+		# this is more related to branch coverage
+		if (m/^FNDA:(-?\d+),(\w+)/) {
 			my $count = $1;
 
 			# Negative gcov results are possible, as reported at:
-- 
2.37.2

[igt-dev] [PATCH 6/6] scripts/code_cov_parse_info: print common coverage and extra per function

[Mauro Carvalho Chehab]

From: Mauro Carvalho Chehab <mchehab@kernel.org>

While analyzing the data, it is nice to have a quick summary about
how much each set of the analyzed data contributes to the total.

So, generate a function summary report with the interception of
all the sets ("common function coverage"), plus the difference
that an individual test group contributes.

This is useful to provide a quick feedback on several situations:

- When comparing FULL with BAT:
	- how many functions are added on FULL?
	- are there functions that are only on BAT?

- When comparing different machines:
	- How many functions each individual machine has over
	  the common group?

- When comparing a group of tests:
	- How many functions an individual test adds for the
	  coverage that weren't covered at the common group yet?

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---

To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover.
See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1663836123.git.mchehab@kernel.org/

 scripts/code_cov_parse_info | 81 +++++++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)

diff --git a/scripts/code_cov_parse_info b/scripts/code_cov_parse_info
index 7f3e1f9ef116..72a1e8c2b0ad 100755
--- a/scripts/code_cov_parse_info
+++ b/scripts/code_cov_parse_info
@@ -574,6 +574,8 @@ sub generate_report($)
 	%used_source = ();
 	%files = ();
 	%test_names = ();
+	my @all_func_keys;
+	my %info_files_with_func;
 
 	foreach my $f (@info_files) {
 		foreach my $source (keys(%{$report{$f}{"all_line"}})) {
@@ -586,6 +588,14 @@ sub generate_report($)
 			foreach my $file (keys(%{$report{$f}{"all_func"}{$func}})) {
 				$all_func{$func}{$file}->{ln} = $report{$f}{"all_func"}{$func}{$file}->{ln};
 				$used_func{$func}->{$file} = 1 if ($report{$f}{"used_func"}{$func}->{$file});
+				if ($report{$f}{"used_func"}{$func}->{$file}) {
+					$used_func{$func}->{$file} = 1;
+					if (!$info_files_with_func{"$file $func"}) {
+						$info_files_with_func{"$file $func"} = 1;
+					} else {
+						$info_files_with_func{"$file $func"}++;
+					}
+				}
 			}
 		}
 		foreach my $source (keys(%{$report{$f}{"all_branch"}})) {
@@ -603,6 +613,25 @@ sub generate_report($)
 	}
 	gen_stats();
 
+	# Gen code coverage set comparision counters
+	my $common_func_count = 0;
+	foreach my $k (keys %info_files_with_func) {
+		$common_func_count++ if ($info_files_with_func{$k} == scalar (@info_files));
+	}
+	foreach my $f (@info_files) {
+		$report{$f}{"more_func"} = 0;
+		$report{$f}{"uniq_func"} = 0;
+		foreach my $func (keys(%{$report{$f}{"all_func"}})) {
+			foreach my $file (keys(%{$report{$f}{"all_func"}{$func}})) {
+				next if (!$report{$f}{"used_func"}{$func}->{$file});
+				next if ($info_files_with_func{"$file $func"} == scalar (@info_files));
+
+				$report{$f}{"more_func"}++;
+				$report{$f}{"uniq_func"}++ if ($info_files_with_func{"$file $func"} == 1);
+			}
+		}
+	}
+
 	# Colors for the html output
 
 	my $red    = "style=\"background-color:#ffb3b3\"";
@@ -720,6 +749,58 @@ sub generate_report($)
 	}
 	print OUT "  </tr>\n</table><p/>\n\n";
 
+	# Print function diff
+	print OUT "  <h2>Differences on function coverage</h2>\n";
+	print OUT "<table width=\"100%\" border=1 cellspacing=0 cellpadding=0>\n  <tr>\n";
+	print OUT "    <th></th>\n";
+	print OUT "    <th>Common to all</th>\n";
+	foreach my $f (@info_files) {
+		print OUT "    <th>$f</th>\n";
+	}
+	print OUT "    <th>TOTAL</th>\n";
+
+	print OUT "  </tr><tr>\n";
+	print OUT "    <td>#Functions per category</td>\n";
+
+	printf OUT "    <td>%d</td>\n", $common_func_count;
+
+	foreach my $f (@info_files) {
+		my %st = %{$report{$f}{"stats"}};
+		if ($st{"func_count"}) {
+			printf OUT "    <td>%d</td>\n", $st{"func_used"};
+
+		}
+	}
+	print OUT "    <td>" . $stats{"func_count"} . "</td>\n";
+
+	print OUT "  </tr><tr>\n";
+	print OUT "    <td>#functions not in common</td>\n";
+	print OUT "    <td></td>\n";
+	foreach my $f (@info_files) {
+		my %st = %{$report{$f}{"stats"}};
+		if ($st{"func_count"}) {
+			printf OUT "    <td>%d</td>\n", $report{$f}{"more_func"};
+
+		}
+	}
+	print OUT "    <td></td>\n";
+
+	print OUT "  </tr><tr>\n";
+	print OUT "    <td>Unique functions</td>\n";
+	print OUT "    <td></td>\n";
+	foreach my $f (@info_files) {
+		my %st = %{$report{$f}{"stats"}};
+		if ($st{"func_count"}) {
+			printf OUT "    <td>%d</td>\n", $report{$f}{"uniq_func"};
+
+		}
+	}
+	print OUT "    <td></td>\n";
+
+	print OUT "  </tr>\n</table><p/>\n\n";
+
+	# Print the filters applied when generating the report
+
 	if ($filter_str ne "") {
 		printf OUT "<p>Filters: %s.</p>\n", $filter_str;
 	} else {
-- 
2.37.2