[PATCH 0/4] [GIT PULL] ktest: Updates for 3.18

[PATCH 0/4] [GIT PULL] ktest: Updates for 3.18

[Steven Rostedt]

Linus,

A fix and a clean up to ktest, as well as two small features.

 A way to allow users to skip a manual bisect.
 Allowing cherry picked patches to be tested.

The cherry pick worked for a test I needed, but stressing it may
not have all the desired effects. It doesn't cause any regressions
so I kept it in.

Please pull the latest ktest-v3.18 tree, which can be found at:


  git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest.git
ktest-v3.18

Tag SHA1: 48c20121c5971c194410fa229d5797811a25a9e8
Head SHA1: d832d743385dd5e735660951aa9d7d36a6a4176a


Chris J Arges (1):
      ktest: add ability to skip during BISECT_MANUAL

Steven Rostedt (Red Hat) (3):
      ktest: Add PATCHCHECK_CHERRY
      ktest: Fix check for new kernel success on rebooting to good kernel
      ktest: Don't bother with bisect good or bad on replay

----
 tools/testing/ktest/ktest.pl    | 61 +++++++++++++++++++++++++++++++----------
 tools/testing/ktest/sample.conf | 10 +++++++
 2 files changed, 57 insertions(+), 14 deletions(-)

[PATCH 1/4] ktest: Add PATCHCHECK_CHERRY

[Steven Rostedt]

[-- Attachment #1: 0001-ktest-Add-PATCHCHECK_CHERRY.patch --]
[-- Type: text/plain, Size: 3234 bytes --]

From: "Steven Rostedt (Red Hat)" <rostedt@goodmis.org>

Add a way to run a patchcheck test on the commits that are in one branch
but not in another. This uses git cherry to find a list of commits to
test each one with.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    | 35 +++++++++++++++++++++++++++++------
 tools/testing/ktest/sample.conf | 10 ++++++++++
 2 files changed, 39 insertions(+), 6 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 55ab700f6ba5..3b7a180d9c0d 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -194,6 +194,7 @@ my $config_bisect_check;
 
 my $patchcheck_type;
 my $patchcheck_start;
+my $patchcheck_cherry;
 my $patchcheck_end;
 
 # set when a test is something other that just building or install
@@ -320,6 +321,7 @@ my %option_map = (
 
     "PATCHCHECK_TYPE"		=> \$patchcheck_type,
     "PATCHCHECK_START"		=> \$patchcheck_start,
+    "PATCHCHECK_CHERRY"		=> \$patchcheck_cherry,
     "PATCHCHECK_END"		=> \$patchcheck_end,
 );
 
@@ -3181,9 +3183,16 @@ sub patchcheck {
 
     my $start = $patchcheck_start;
 
+    my $cherry = $patchcheck_cherry;
+    if (!defined($cherry)) {
+	$cherry = 0;
+    }
+
     my $end = "HEAD";
     if (defined($patchcheck_end)) {
 	$end = $patchcheck_end;
+    } elsif ($cherry) {
+	die "PATCHCHECK_END must be defined with PATCHCHECK_CHERRY\n";
     }
 
     # Get the true sha1's since we can use things like HEAD~3
@@ -3197,24 +3206,38 @@ sub patchcheck {
 	$type = "boot";
     }
 
-    open (IN, "git log --pretty=oneline $end|") or
-	dodie "could not get git list";
+    if ($cherry) {
+	open (IN, "git cherry -v $start $end|") or
+	    dodie "could not get git list";
+    } else {
+	open (IN, "git log --pretty=oneline $end|") or
+	    dodie "could not get git list";
+    }
 
     my @list;
 
     while (<IN>) {
 	chomp;
+	# git cherry adds a '+' we want to remove
+	s/^\+ //;
 	$list[$#list+1] = $_;
 	last if (/^$start/);
     }
     close(IN);
 
-    if ($list[$#list] !~ /^$start/) {
-	fail "SHA1 $start not found";
+    if (!$cherry) {
+	if ($list[$#list] !~ /^$start/) {
+	    fail "SHA1 $start not found";
+	}
+
+	# go backwards in the list
+	@list = reverse @list;
     }
 
-    # go backwards in the list
-    @list = reverse @list;
+    doprint("Going to test the following commits:\n");
+    foreach my $l (@list) {
+	doprint "$l\n";
+    }
 
     my $save_clean = $noclean;
     my %ignored_warnings;
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index 911e45ad657a..6c58cd8bbbae 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -906,6 +906,16 @@
 #
 #  PATCHCHECK_END is the last patch to check (default HEAD)
 #
+#  PATCHCHECK_CHERRY if set to non zero, then git cherry will be
+#      performed against PATCHCHECK_START and PATCHCHECK_END. That is
+#
+#      git cherry ${PATCHCHECK_START} ${PATCHCHECK_END}
+#
+#      Then the changes found will be tested.
+#
+#      Note, PATCHCHECK_CHERRY requires PATCHCHECK_END to be defined.
+#      (default 0)
+#
 #  PATCHCHECK_TYPE is required and is the type of test to run:
 #      build, boot, test.
 #
-- 
2.0.1

[PATCH 2/4] ktest: add ability to skip during BISECT_MANUAL

[Steven Rostedt]

[-- Attachment #1: 0002-ktest-add-ability-to-skip-during-BISECT_MANUAL.patch --]
[-- Type: text/plain, Size: 1221 bytes --]

From: Chris J Arges <chris.j.arges@canonical.com>

When doing a manual bisect, a build can fail or a test can be inconclusive.
In these cases it would be helpful to be able to skip the test entirely.

Link: http://lkml.kernel.org/r/1409164021-2136-1-git-send-email-chris.j.arges@canonical.com

Reviewed-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Signed-off-by: Chris J Arges <chris.j.arges@canonical.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 3b7a180d9c0d..085452fa045b 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -2338,15 +2338,17 @@ sub success {
 
 sub answer_bisect {
     for (;;) {
-	doprint "Pass or fail? [p/f]";
+	doprint "Pass, fail, or skip? [p/f/s]";
 	my $ans = <STDIN>;
 	chomp $ans;
 	if ($ans eq "p" || $ans eq "P") {
 	    return 1;
 	} elsif ($ans eq "f" || $ans eq "F") {
 	    return 0;
+	} elsif ($ans eq "s" || $ans eq "S") {
+	    return -1;
 	} else {
-	    print "Please answer 'P' or 'F'\n";
+	    print "Please answer 'p', 'f', or 's'\n";
 	}
     }
 }
-- 
2.0.1

[PATCH 3/4] ktest: Fix check for new kernel success on rebooting to good kernel

[Steven Rostedt]

[-- Attachment #1: 0003-ktest-Fix-check-for-new-kernel-success-on-rebooting-.patch --]
[-- Type: text/plain, Size: 1073 bytes --]

From: "Steven Rostedt (Red Hat)" <rostedt@goodmis.org>

The reboot function when rebooting back to a good kernel has a check
to make sure that a new kernel was indeed booted. But that check
uses a timeout value, which when calling the monitor will still
return success if the timeout is hit (no bug was found). It should
return an error to let the reboot code know that a new kernel was
not reached. Only the reboot code checks the return value of the
monitor.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 085452fa045b..c518b0fb6d01 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -1450,6 +1450,12 @@ sub wait_for_monitor {
 	}
     }
     print "** Monitor flushed **\n";
+
+    # if stop is defined but wasn't hit, return error
+    # used by reboot (which wants to see a reboot)
+    if (defined($stop) && !$booted) {
+	$bug = 1;
+    }
     return $bug;
 }
 
-- 
2.0.1

[PATCH 4/4] ktest: Dont bother with bisect good or bad on replay

[Steven Rostedt]

[-- Attachment #1: 0004-ktest-Don-t-bother-with-bisect-good-or-bad-on-replay.patch --]
[-- Type: text/plain, Size: 1338 bytes --]

From: "Steven Rostedt (Red Hat)" <rostedt@goodmis.org>

If git bisect reply is being used in the bisect tests, don't bother
doing the git bisect good or git bisect bad calls. The git bisect
reply will override them anyway, and that's called immediately
after the other two. Going the git bisect (good|bad) is just a
waste of time.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index c518b0fb6d01..bf1398180785 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -2736,15 +2736,17 @@ sub bisect {
     run_command "git bisect start$start_files" or
 	dodie "could not start bisect";
 
-    run_command "git bisect good $good" or
-	dodie "could not set bisect good to $good";
-
-    run_git_bisect "git bisect bad $bad" or
-	dodie "could not set bisect bad to $bad";
-
     if (defined($replay)) {
 	run_command "git bisect replay $replay" or
 	    dodie "failed to run replay";
+    } else {
+
+	run_command "git bisect good $good" or
+	    dodie "could not set bisect good to $good";
+
+	run_git_bisect "git bisect bad $bad" or
+	    dodie "could not set bisect bad to $bad";
+
     }
 
     if (defined($start)) {
-- 
2.0.1