linux-perf-users.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: vmolnaro@redhat.com
To: linux-perf-users@vger.kernel.org, acme@redhat.com, namhyung@kernel.org
Cc: acme@kernel.org, mpetlan@redhat.com, irogers@google.com,
	atrajeev@linux.vnet.ibm.com, masami.hiramatsu.pt@hitachi.com
Subject: [PATCH v2 08/11] perf testsuite: Add common output checking helper
Date: Tue,  2 Jul 2024 13:08:47 +0200	[thread overview]
Message-ID: <20240702110849.31904-10-vmolnaro@redhat.com> (raw)
In-Reply-To: <20240702110849.31904-1-vmolnaro@redhat.com>

From: Veronika Molnarova <vmolnaro@redhat.com>

As a form of validation, it is a common practice to check the outputs
of commands whether they contain expected patterns or match a certain
regular expression.

This output checking helper is designed to allow checking stderr output
of perf commands for unexpected messages, while ignoring messages that
are known to be harmless, e.g.:
  "Lowering default frequency rate to \d+\."
  "\d+ out of order events recorded."
  etc.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../shell/common/check_errors_whitelisted.pl  | 51 +++++++++++++++++++
 1 file changed, 51 insertions(+)
 create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl

diff --git a/tools/perf/tests/shell/common/check_errors_whitelisted.pl b/tools/perf/tests/shell/common/check_errors_whitelisted.pl
new file mode 100755
index 0000000000..c57d355dd7
--- /dev/null
+++ b/tools/perf/tests/shell/common/check_errors_whitelisted.pl
@@ -0,0 +1,51 @@
+#!/usr/bin/perl
+# SPDX-License-Identifier: GPL-2.0
+
+$whitelist_file = shift;
+
+if (defined $whitelist_file)
+{
+	open (INFILE, $whitelist_file) or die "Checker error: Unable to open the whitelist file: $whitelist_file\n";
+	@regexps = <INFILE>;
+	close INFILE or die "Checker error: Unable to close the whitelist file: $whitelist_file\n";
+}
+else
+{
+	@regexps = ();
+}
+
+$max_printed_lines = 20;
+$max_printed_lines = $ENV{TESTLOG_ERR_MSG_MAX_LINES} if (defined $ENV{TESTLOG_ERR_MSG_MAX_LINES});
+
+$quiet = 1;
+$quiet = 0 if (defined $ENV{TESTLOG_VERBOSITY} && $ENV{TESTLOG_VERBOSITY} ge 2);
+
+$passed = 1;
+$lines_printed = 0;
+
+while (<STDIN>)
+{
+	s/\n//;
+
+	$line_matched = 0;
+	for $r (@regexps)
+	{
+		chomp $r;
+		if (/$r/)
+		{
+			$line_matched = 1;
+			last;
+		}
+	}
+
+	unless ($line_matched)
+	{
+		if ($lines_printed++ < $max_printed_lines)
+		{
+			print "Line did not match any pattern: \"$_\"\n" unless $quiet;
+		}
+		$passed = 0;
+	}
+}
+
+exit ($passed == 0);
-- 
2.43.0


  parent reply	other threads:[~2024-07-02 11:14 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
2024-06-24 13:07 ` [PATCH 01/11] perf tests shell: Skip base_* dirs in test script search Michael Petlan
2024-06-24 13:08 ` [PATCH 02/11] perf testsuite: Merge settings files for shell tests Michael Petlan
2024-06-24 13:08 ` [PATCH 03/11] perf testsuite: Fix shellcheck warnings Michael Petlan
2024-06-24 13:08 ` [PATCH 04/11] perf testsuite probe: Add test for blacklisted kprobes handling Michael Petlan
2024-06-24 13:08 ` [PATCH 05/11] perf testsuite probe: Add test for basic perf-probe options Michael Petlan
2024-06-24 13:08 ` [PATCH 06/11] perf testsuite probe: Add test for invalid options Michael Petlan
2024-06-24 13:08 ` [PATCH 07/11] perf testsuite probe: Add test for line semantics Michael Petlan
2024-06-24 13:08 ` [PATCH 08/11] perf testsuite: Add common output checking helper Michael Petlan
2024-06-24 13:08 ` [PATCH 09/11] perf testsuite report: Add test for perf-report basic functionality Michael Petlan
2024-06-24 13:08 ` [PATCH 10/11] perf testsuite report: Add test case for perf report Michael Petlan
2024-06-24 13:08 ` [PATCH 11/11] perf testsuite: Install perf-report tests Michael Petlan
2024-06-27 22:29 ` [PATCH 00/11] perftool-testsuite 2nd batch Namhyung Kim
2024-07-02 11:08   ` [PATCH v2 " vmolnaro
2024-07-02 11:08     ` [PATCH v2 01/11] perf tests shell: Skip base_* dirs in test script search vmolnaro
2024-07-02 11:08     ` [PATCH v2 02/11] perf testsuite: Merge settings files for shell tests vmolnaro
2024-07-02 11:08     ` [PATCH v2 03/11] perf testsuite: Fix shellcheck warnings vmolnaro
2024-07-02 11:08     ` [PATCH v2 04/11] perf testsuite probe: Add test for blacklisted kprobes handling vmolnaro
2024-07-02 11:08     ` [PATCH v2 05/11] perf testsuite probe: Add test for basic perf-probe options vmolnaro
2024-07-02 11:08     ` [PATCH v2 06/11] perf testsuite probe: Add test for invalid options vmolnaro
2024-07-02 11:08     ` [PATCH v2 07/11] perf testsuite probe: Add test for line semantics vmolnaro
2024-07-02 11:08     ` vmolnaro [this message]
2024-07-02 11:08     ` [PATCH v2 09/11] perf testsuite report: Add test for perf-report basic functionality vmolnaro
2024-07-02 11:08     ` [PATCH v2 10/11] perf testsuite report: Add test case for perf report vmolnaro
2024-07-02 11:08     ` [PATCH v2 11/11] perf testsuite: Install perf-report tests vmolnaro
2024-08-28 14:10     ` [PATCH v2 00/11] perftool-testsuite 2nd batch Veronika Molnarova
2024-08-28 20:39       ` Arnaldo Carvalho de Melo
2024-08-29 11:29         ` [PATCH v2 01/11] perf tests shell: Skip base_* dirs in test script search vmolnaro
2024-08-29 11:29         ` [PATCH v2 11/11] perf testsuite: Install perf-report tests vmolnaro
2024-08-30 15:27         ` [PATCH v2 00/11] perftool-testsuite 2nd batch Arnaldo Carvalho de Melo
2024-09-02 14:05           ` Veronika Molnarova
2024-09-02 14:31             ` Arnaldo Carvalho de Melo
2024-09-02 14:41               ` Arnaldo Carvalho de Melo
2024-09-02 14:42                 ` Arnaldo Carvalho de Melo
2024-09-02 15:10                 ` Veronika Molnarova
2024-09-02 18:46                   ` Arnaldo Carvalho de Melo
2024-09-02 21:00                     ` Arnaldo Carvalho de Melo
2024-09-05 15:11                     ` Masami Hiramatsu
2024-09-05 19:04                       ` Arnaldo Carvalho de Melo
2024-09-05 19:15                         ` Arnaldo Carvalho de Melo
2024-09-12 13:07                           ` Arnaldo Carvalho de Melo
2024-09-19 13:14                             ` Veronika Molnarova
2024-09-26 22:33                               ` Namhyung Kim

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240702110849.31904-10-vmolnaro@redhat.com \
    --to=vmolnaro@redhat.com \
    --cc=acme@kernel.org \
    --cc=acme@redhat.com \
    --cc=atrajeev@linux.vnet.ibm.com \
    --cc=irogers@google.com \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=masami.hiramatsu.pt@hitachi.com \
    --cc=mpetlan@redhat.com \
    --cc=namhyung@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).