[PATCH 00/21] [GIT PULL] ktest: lots of nice new features

[PATCH 00/21] [GIT PULL] ktest: lots of nice new features

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 1823 bytes --]

Linus,

This is the udates I have done since the last merge window. Some included fixes
I needed to present my tutorial at LinuxCon. My tutorial was based on the feature
set added by this patch pull.

Please pull the following patch set:

HEAD SHA1: 815e2bd7d609da9c7615ea28a3990064a394312f

-- Steve



The following patches are in:

  git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest.git

    branch: for-linus


Andrew Jones (3):
      ktest: Create outputdir if it does not exist
      ktest: Only need to save .config when doing mrproper
      ktest: Include monitor in reboot code

Steven Rostedt (18):
      ktest: Add TEST_TYPE install option
      ktest: Fail when grub menu not found
      ktest: Add NO_INSTALL option to not install for a test
      ktest: Add option REBOOT_SUCCESS_LINE to stop waiting after a reboot
      ktest: Do not reboot on config or build issues
      ktest: Add IF and ELSE to config sections
      ktest: Let IF keyword take comparisons
      ktest: Add INCLUDE keyword to include other config files
      ktest: Consolidate TEST_TYPE and DEFAULT code
      ktest: Add OVERRIDE keyword to DEFAULTS section
      ktest: Add DEFINED keyword for IF statements
      ktest: Sort make_min_config configs by dependecies
      ktest: Fix parsing of config section lines
      ktest: Add processing of complex conditionals
      ktest: Do not opencode reboot in grub setting
      ktest: Add another monitor flush before installing kernel
      ktest: Add variable ${PWD}
      ktest: Evaluate variables entered on the command line

----
 tools/testing/ktest/ktest.pl    |  515 +++++++++++++++++++++++++++++++--------
 tools/testing/ktest/sample.conf |  146 +++++++++++-
 2 files changed, 553 insertions(+), 108 deletions(-)

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 01/21] ktest: Add TEST_TYPE install option

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 1861 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

In testing one of my boxes, I found that I only wanted to build and
install the kernel. I wanted to manually reboot the box and test it.
Adding a TEST_TYPE option "install" allows this to happen.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |    7 +++++++
 tools/testing/ktest/sample.conf |    7 ++++---
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 8d02ccb..e087cb4 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -2929,6 +2929,13 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
 	build $build_type or next;
     }
 
+    if ($test_type eq "install") {
+	get_version;
+	install;
+	success $i;
+	next;
+    }
+
     if ($test_type ne "build") {
 	my $failed = 0;
 	start_monitor_and_boot or $failed = 1;
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index b8bcd14..eadca3e 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -253,9 +253,10 @@
 
 # The default test type (default test)
 # The test types may be:
-#   build - only build the kernel, do nothing else
-#   boot - build and boot the kernel
-#   test - build, boot and if TEST is set, run the test script
+#   build   - only build the kernel, do nothing else
+#   install - build and install, but do nothing else (does not reboot)
+#   boot    - build, install, and boot the kernel
+#   test    - build, boot and if TEST is set, run the test script
 #          (If TEST is not set, it defaults back to boot)
 #   bisect - Perform a bisect on the kernel (see BISECT_TYPE below)
 #   patchcheck - Do a test on a series of commits in git (see PATCHCHECK below)
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 02/21] ktest: Create outputdir if it does not exist

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 929 bytes --]

From: Andrew Jones <drjones@redhat.com>

Signed-off-by: Andrew Jones <drjones@redhat.com>
Link: http://lkml.kernel.org/r/1313155932-20092-2-git-send-email-drjones@redhat.com
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index e087cb4..253e9a2 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -2850,9 +2850,11 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
 
     chdir $builddir || die "can't change directory to $builddir";
 
-    if (!-d $tmpdir) {
-	mkpath($tmpdir) or
-	    die "can't create $tmpdir";
+    foreach my $dir ($tmpdir, $outputdir) {
+	if (!-d $dir) {
+	    mkpath($dir) or
+		die "can't create $dir";
+	}
     }
 
     $ENV{"SSH_USER"} = $ssh_user;
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 03/21] ktest: Only need to save .config when doing mrproper

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 1317 bytes --]

From: Andrew Jones <drjones@redhat.com>

Only save the .config file if we're doing mrproper

Signed-off-by: Andrew Jones <drjones@redhat.com>
Link: http://lkml.kernel.org/r/1313155932-20092-3-git-send-email-drjones@redhat.com
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   14 +++++++-------
 1 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 253e9a2..20b0e73 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -1272,15 +1272,15 @@ sub build {
 	# allow for empty configs
 	run_command "touch $output_config";
 
-	run_command "mv $output_config $outputdir/config_temp" or
-	    dodie "moving .config";
+	if (!$noclean) {
+	    run_command "mv $output_config $outputdir/config_temp" or
+		dodie "moving .config";
 
-	if (!$noclean && !run_command "$make mrproper") {
-	    dodie "make mrproper";
-	}
+	    run_command "$make mrproper" or dodie "make mrproper";
 
-	run_command "mv $outputdir/config_temp $output_config" or
-	    dodie "moving config_temp";
+	    run_command "mv $outputdir/config_temp $output_config" or
+		dodie "moving config_temp";
+	}
 
     } elsif (!$noclean) {
 	unlink "$output_config";
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 04/21] ktest: Include monitor in reboot code

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 2674 bytes --]

From: Andrew Jones <drjones@redhat.com>

Several places that call reboot do the same thing with respect to the
monitor. By adding this code into the reboot code, redundant code is
removed and it paves the way for the the reset time patch.

Signed-off-by: Andrew Jones <drjones@redhat.com>
Link: http://lkml.kernel.org/r/1313155932-20092-4-git-send-email-drjones@redhat.com
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   36 ++++++++++++++++--------------------
 1 files changed, 16 insertions(+), 20 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 20b0e73..debc689 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -603,8 +603,13 @@ sub doprint {
 }
 
 sub run_command;
+sub start_monitor;
+sub end_monitor;
+sub wait_for_monitor;
 
 sub reboot {
+    my ($time) = @_;
+
     # try to reboot normally
     if (run_command $reboot) {
 	if (defined($powercycle_after_reboot)) {
@@ -615,6 +620,12 @@ sub reboot {
 	# nope? power cycle it.
 	run_command "$power_cycle";
     }
+
+    if (defined($time)) {
+	start_monitor;
+	wait_for_monitor $time;
+	end_monitor;
+    }
 }
 
 sub do_not_reboot {
@@ -719,10 +730,7 @@ sub fail {
 	# no need to reboot for just building.
 	if (!do_not_reboot) {
 	    doprint "REBOOTING\n";
-	    reboot;
-	    start_monitor;
-	    wait_for_monitor $sleep_time;
-	    end_monitor;
+	    reboot $sleep_time;
 	}
 
 	my $name = "";
@@ -1356,10 +1364,7 @@ sub success {
 
     if ($i != $opt{"NUM_TESTS"} && !do_not_reboot) {
 	doprint "Reboot and wait $sleep_time seconds\n";
-	reboot;
-	start_monitor;
-	wait_for_monitor $sleep_time;
-	end_monitor;
+	reboot $sleep_time;
     }
 }
 
@@ -1500,10 +1505,7 @@ sub run_git_bisect {
 
 sub bisect_reboot {
     doprint "Reboot and sleep $bisect_sleep_time seconds\n";
-    reboot;
-    start_monitor;
-    wait_for_monitor $bisect_sleep_time;
-    end_monitor;
+    reboot $bisect_sleep_time;
 }
 
 # returns 1 on success, 0 on failure, -1 on skip
@@ -2066,10 +2068,7 @@ sub config_bisect {
 
 sub patchcheck_reboot {
     doprint "Reboot and sleep $patchcheck_sleep_time seconds\n";
-    reboot;
-    start_monitor;
-    wait_for_monitor $patchcheck_sleep_time;
-    end_monitor;
+    reboot $patchcheck_sleep_time;
 }
 
 sub patchcheck {
@@ -2659,10 +2658,7 @@ sub make_min_config {
 	}
 
 	doprint "Reboot and wait $sleep_time seconds\n";
-	reboot;
-	start_monitor;
-	wait_for_monitor $sleep_time;
-	end_monitor;
+	reboot $sleep_time;
     }
 
     success $i;
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 05/21] ktest: Fail when grub menu not found

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 1100 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Currently if the grub menu that is supplied is not found, it will
just boot into the last grub menu in menu.lst. Fail instead of
confusing the user why their kernel is not booting.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index debc689..ca6ff99 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -862,9 +862,12 @@ sub get_grub_index {
     open(IN, "$ssh_grub |")
 	or die "unable to get menu.lst";
 
+    my $found = 0;
+
     while (<IN>) {
 	if (/^\s*title\s+$grub_menu\s*$/) {
 	    $grub_number++;
+	    $found = 1;
 	    last;
 	} elsif (/^\s*title\s/) {
 	    $grub_number++;
@@ -873,7 +876,7 @@ sub get_grub_index {
     close(IN);
 
     die "Could not find '$grub_menu' in /boot/grub/menu on $machine"
-	if ($grub_number < 0);
+	if (!$found);
     doprint "$grub_number\n";
 }
 
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 06/21] ktest: Add NO_INSTALL option to not install for a test

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 3079 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

There's cases where running the same kernel over and over again
is useful, and being able to not install the same kernel can
save time between tests.

Add a NO_INSTALL option that tells ktest.pl to not install the
new kernel.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   10 +++++++++-
 tools/testing/ktest/sample.conf |    7 +++++++
 2 files changed, 16 insertions(+), 1 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index ca6ff99..74fb2ac 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -42,6 +42,7 @@ $default{"BISECT_MANUAL"}	= 0;
 $default{"BISECT_SKIP"}		= 1;
 $default{"SUCCESS_LINE"}	= "login:";
 $default{"DETECT_TRIPLE_FAULT"} = 1;
+$default{"NO_INSTALL"}		= 0;
 $default{"BOOTED_TIMEOUT"}	= 1;
 $default{"DIE_ON_FAILURE"}	= 1;
 $default{"SSH_EXEC"}		= "ssh \$SSH_USER\@\$MACHINE \$SSH_COMMAND";
@@ -84,6 +85,7 @@ my $grub_number;
 my $target;
 my $make;
 my $post_install;
+my $no_install;
 my $noclean;
 my $minconfig;
 my $start_minconfig;
@@ -1094,6 +1096,8 @@ sub do_post_install {
 
 sub install {
 
+    return if ($no_install);
+
     run_scp "$outputdir/$build_target", "$target_image" or
 	dodie "failed to copy image";
 
@@ -2810,6 +2814,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     $reboot_type = set_test_option("REBOOT_TYPE", $i);
     $grub_menu = set_test_option("GRUB_MENU", $i);
     $post_install = set_test_option("POST_INSTALL", $i);
+    $no_install = set_test_option("NO_INSTALL", $i);
     $reboot_script = set_test_option("REBOOT_SCRIPT", $i);
     $reboot_on_error = set_test_option("REBOOT_ON_ERROR", $i);
     $poweroff_on_error = set_test_option("POWEROFF_ON_ERROR", $i);
@@ -2890,8 +2895,11 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
 	$run_type = "ERROR";
     }
 
+    my $installme = "";
+    $installme = " no_install" if ($no_install);
+
     doprint "\n\n";
-    doprint "RUNNING TEST $i of $opt{NUM_TESTS} with option $test_type $run_type\n\n";
+    doprint "RUNNING TEST $i of $opt{NUM_TESTS} with option $test_type $run_type$installme\n\n";
 
     unlink $dmesg;
     unlink $buildlog;
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index eadca3e..b3e0dc1 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -294,6 +294,13 @@
 # or on some systems:
 #POST_INSTALL = ssh user@target /sbin/dracut -f /boot/initramfs-test.img $KERNEL_VERSION
 
+# If for some reason you just want to boot the kernel and you do not
+# want the test to install anything new. For example, you may just want
+# to boot test the same kernel over and over and do not want to go through
+# the hassle of installing anything, you can set this option to 1
+# (default 0)
+#NO_INSTALL = 1
+
 # If there is a script that you require to run before the build is done
 # you can specify it with PRE_BUILD.
 #
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 07/21] ktest: Add option REBOOT_SUCCESS_LINE to stop waiting after a reboot

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 3805 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

When ktest.pl reboots, it will usuall wait SLEEP_TIME seconds of idle
console before starting the next test. By setting the
REBOOT_SUCCESS_LINE, ktest will not wait SLEEP_TIME when it detects the
line while rebooting to a new kernel.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   33 +++++++++++++++++++++++++++------
 tools/testing/ktest/sample.conf |   10 ++++++++++
 2 files changed, 37 insertions(+), 6 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 74fb2ac..51ddaa5 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -117,6 +117,7 @@ my $timeout;
 my $booted_timeout;
 my $detect_triplefault;
 my $console;
+my $reboot_success_line;
 my $success_line;
 my $stop_after_success;
 my $stop_after_failure;
@@ -612,6 +613,13 @@ sub wait_for_monitor;
 sub reboot {
     my ($time) = @_;
 
+    if (defined($time)) {
+	start_monitor;
+	# flush out current monitor
+	# May contain the reboot success line
+	wait_for_monitor 1;
+    }
+
     # try to reboot normally
     if (run_command $reboot) {
 	if (defined($powercycle_after_reboot)) {
@@ -624,8 +632,7 @@ sub reboot {
     }
 
     if (defined($time)) {
-	start_monitor;
-	wait_for_monitor $time;
+	wait_for_monitor($time, $reboot_success_line);
 	end_monitor;
     }
 }
@@ -706,16 +713,29 @@ sub end_monitor {
 }
 
 sub wait_for_monitor {
-    my ($time) = @_;
+    my ($time, $stop) = @_;
+    my $full_line = "";
     my $line;
+    my $booted = 0;
 
     doprint "** Wait for monitor to settle down **\n";
 
     # read the monitor and wait for the system to calm down
-    do {
+    while (!$booted) {
 	$line = wait_for_input($monitor_fp, $time);
-	print "$line" if (defined($line));
-    } while (defined($line));
+	last if (!defined($line));
+	print "$line";
+	$full_line .= $line;
+
+	if (defined($stop) && $full_line =~ /$stop/) {
+	    doprint "wait for monitor detected $stop\n";
+	    $booted = 1;
+	}
+
+	if ($line =~ /\n/) {
+	    $full_line = "";
+	}
+    }
     print "** Monitor flushed **\n";
 }
 
@@ -2836,6 +2856,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     $console = set_test_option("CONSOLE", $i);
     $detect_triplefault = set_test_option("DETECT_TRIPLE_FAULT", $i);
     $success_line = set_test_option("SUCCESS_LINE", $i);
+    $reboot_success_line = set_test_option("REBOOT_SUCCESS_LINE", $i);
     $stop_after_success = set_test_option("STOP_AFTER_SUCCESS", $i);
     $stop_after_failure = set_test_option("STOP_AFTER_FAILURE", $i);
     $stop_test_after = set_test_option("STOP_TEST_AFTER", $i);
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index b3e0dc1..67bc4e0 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -423,6 +423,14 @@
 # (default "login:")
 #SUCCESS_LINE = login:
 
+# To speed up between reboots, defining a line that the
+# default kernel produces that represents that the default
+# kernel has successfully booted and can be used to pass
+# a new test kernel to it. Otherwise ktest.pl will wait till
+# SLEEP_TIME to continue.
+# (default undefined)
+#REBOOT_SUCCESS_LINE = login:
+
 # In case the console constantly fills the screen, having
 # a specified time to stop the test after success is recommended.
 # (in seconds)
@@ -488,6 +496,8 @@
 # another test. If a reboot to the reliable kernel happens,
 # we wait SLEEP_TIME for the console to stop producing output
 # before starting the next test.
+#
+# You can speed up reboot times even more by setting REBOOT_SUCCESS_LINE.
 # (default 60)
 #SLEEP_TIME = 60
 
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 08/21] ktest: Do not reboot on config or build issues

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 2287 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Even if REBOOT_ON_ERROR is set, it becomes annoying that the target
machine is rebooted when a config option is incorrect or a build
fails. There's no reason to reboot the target for host only issues.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   22 ++++++++++++++++++++--
 1 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 51ddaa5..057676a 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -133,6 +133,9 @@ my %config_help;
 my %variable;
 my %force_config;
 
+# do not force reboots on config problems
+my $no_reboot = 1;
+
 $config_help{"MACHINE"} = << "EOF"
  The machine hostname that you will test.
 EOF
@@ -640,7 +643,7 @@ sub reboot {
 sub do_not_reboot {
     my $i = $iteration;
 
-    return $test_type eq "build" ||
+    return $test_type eq "build" || $no_reboot ||
 	($test_type eq "patchcheck" && $opt{"PATCHCHECK_TYPE[$i]"} eq "build") ||
 	($test_type eq "bisect" && $opt{"BISECT_TYPE[$i]"} eq "build");
 }
@@ -1285,6 +1288,10 @@ sub build {
 
     unlink $buildlog;
 
+    # Failed builds should not reboot the target
+    my $save_no_reboot = $no_reboot;
+    $no_reboot = 1;
+
     if (defined($pre_build)) {
 	my $ret = run_command $pre_build;
 	if (!$ret && defined($pre_build_die) &&
@@ -1353,10 +1360,15 @@ sub build {
 
     if (!$build_ret) {
 	# bisect may need this to pass
-	return 0 if ($in_bisect);
+	if ($in_bisect) {
+	    $no_reboot = $save_no_reboot;
+	    return 0;
+	}
 	fail "failed build" and return 0;
     }
 
+    $no_reboot = $save_no_reboot;
+
     return 1;
 }
 
@@ -2806,6 +2818,9 @@ sub set_test_option {
 # First we need to do is the builds
 for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
 
+    # Do not reboot on failing test options
+    $no_reboot = 1;
+
     $iteration = $i;
 
     my $makecmd = set_test_option("MAKE_CMD", $i);
@@ -2941,6 +2956,9 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
 	    die "failed to checkout $checkout";
     }
 
+    $no_reboot = 0;
+
+
     if ($test_type eq "bisect") {
 	bisect $i;
 	next;
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 09/21] ktest: Add IF and ELSE to config sections

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 4536 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Add IF keyword to sections within the config. Also added an ELSE
keyword that allows different config options to be set for a given
section.

For example:

TYPE := 1
STATUS := 0

DEFAULTS IF ${TYPE}
[...]
ELSE IF ${STATUS}
[...]
ELSE
[...]

The above will process the first section as $TYPE is true. If it
was false, it would process the last section as $STATUS is false.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   64 ++++++++++++++++++++++++++++++++++++--
 tools/testing/ktest/sample.conf |   38 +++++++++++++++++++++++
 2 files changed, 98 insertions(+), 4 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 057676a..c76e18f 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -361,6 +361,21 @@ sub set_variable {
     }
 }
 
+sub process_if {
+    my ($name, $value) = @_;
+
+    my $val = process_variables($value);
+
+    if ($val =~ /^\s*0\s*$/) {
+	return 0;
+    } elsif ($val =~ /^\s*\d+\s*$/) {
+	return 1;
+    }
+
+    die ("$name: $.: Undefined variable $val in if statement\n");
+    return 1;
+}
+
 sub read_config {
     my ($config) = @_;
 
@@ -376,6 +391,8 @@ sub read_config {
     my $skip = 0;
     my $rest;
     my $test_case = 0;
+    my $if = 0;
+    my $if_set = 0;
 
     while (<IN>) {
 
@@ -397,7 +414,7 @@ sub read_config {
 	    $default = 0;
 	    $repeat = 1;
 
-	    if ($rest =~ /\s+SKIP(.*)/) {
+	    if ($rest =~ /\s+SKIP\b(.*)/) {
 		$rest = $1;
 		$skip = 1;
 	    } else {
@@ -411,9 +428,16 @@ sub read_config {
 		$repeat_tests{"$test_num"} = $repeat;
 	    }
 
-	    if ($rest =~ /\s+SKIP(.*)/) {
-		$rest = $1;
-		$skip = 1;
+	    if ($rest =~ /\sIF\s+(\S*)(.*)/) {
+		$rest = $2;
+		if (process_if($name, $1)) {
+		    $if_set = 1;
+		} else {
+		    $skip = 1;
+		}
+		$if = 1;
+	    } else {
+		$if = 0;
 	    }
 
 	    if ($rest !~ /^\s*$/) {
@@ -437,10 +461,42 @@ sub read_config {
 		$skip = 0;
 	    }
 
+	    if ($rest =~ /\sIF\s+(\S*)(.*)/) {
+		$if = 1;
+		$rest = $2;
+		if (process_if($name, $1)) {
+		    $if_set = 1;
+		} else {
+		    $skip = 1;
+		}
+	    } else {
+		$if = 0;
+	    }
+
 	    if ($rest !~ /^\s*$/) {
 		die "$name: $.: Gargbage found after DEFAULTS\n$_";
 	    }
 
+	} elsif (/^\s*ELSE(.*)$/) {
+	    if (!$if) {
+		die "$name: $.: ELSE found with out matching IF section\n$_";
+	    }
+	    $rest = $1;
+	    if ($if_set) {
+		$skip = 1;
+	    } else {
+		$skip = 0;
+
+		if ($rest =~ /\sIF\s+(\S*)(.*)/) {
+		    # May be a ELSE IF section.
+		    if (!process_if($name, $1)) {
+			$skip = 1;
+		    }
+		} else {
+		    $if = 0;
+		}
+	    }
+
 	} elsif (/^\s*([A-Z_\[\]\d]+)\s*=\s*(.*?)\s*$/) {
 
 	    next if ($skip);
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index 67bc4e0..6a0a0ba 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -72,6 +72,44 @@
 # the same option name under the same test or as default
 # ktest will fail to execute, and no tests will run.
 #
+# Both TEST_START and DEFAULTS sections can also have the IF keyword
+# The value after the IF must evaluate into a 0 or non 0 positive
+# integer, and can use the config variables (explained below).
+#
+# DEFAULTS IF ${IS_X86_32}
+#
+# The above will process the DEFAULTS section if the config
+# variable IS_X86_32 evaluates to a non zero positive integer
+# otherwise if it evaluates to zero, it will act the same
+# as if the SKIP keyword was used.
+#
+# The ELSE keyword can be used directly after a section with
+# a IF statement.
+#
+# TEST_START IF ${RUN_NET_TESTS}
+# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-network
+#
+# ELSE
+#
+# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-normal
+#
+#
+# The ELSE keyword can also contain an IF statement to allow multiple
+# if then else sections. But all the sections must be either
+# DEFAULT or TEST_START, they can not be a mixture.
+#
+# TEST_START IF ${RUN_NET_TESTS}
+# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-network
+#
+# ELSE IF ${RUN_DISK_TESTS}
+# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-tests
+#
+# ELSE IF ${RUN_CPU_TESTS}
+# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-cpu
+#
+# ELSE
+# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-network
+#
 
 #### Config variables ####
 #
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 10/21] ktest: Let IF keyword take comparisons

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 4034 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Allow ==, !=, <=, >=, <, and > to be used in IF statements
to compare if a section should be processed or not.

For example:

BITS := 32

DEFAULTS IF ${BITS} == 32
MIN_CONFIG = ${CONFIG_DIR}/config-32
ELSE
MIN_CONFIG = ${CONFIG_DIR}/config-64

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   54 ++++++++++++++++++++++++++++++++++----
 tools/testing/ktest/sample.conf |   14 ++++++++++
 2 files changed, 62 insertions(+), 6 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index c76e18f..ed20d68 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -361,11 +361,47 @@ sub set_variable {
     }
 }
 
+sub process_compare {
+    my ($lval, $cmp, $rval) = @_;
+
+    # remove whitespace
+
+    $lval =~ s/^\s*//;
+    $lval =~ s/\s*$//;
+
+    $rval =~ s/^\s*//;
+    $rval =~ s/\s*$//;
+
+    if ($cmp eq "==") {
+	return $lval eq $rval;
+    } elsif ($cmp eq "!=") {
+	return $lval ne $rval;
+    }
+
+    my $statement = "$lval $cmp $rval";
+    my $ret = eval $statement;
+
+    # $@ stores error of eval
+    if ($@) {
+	return -1;
+    }
+
+    return $ret;
+}
+
 sub process_if {
     my ($name, $value) = @_;
 
     my $val = process_variables($value);
 
+    if ($val =~ /(.*)(==|\!=|>=|<=|>|<)(.*)/) {
+	my $ret = process_compare($1, $2, $3);
+	if ($ret < 0) {
+	    die "$name: $.: Unable to process comparison\n";
+	}
+	return $ret;
+    }
+
     if ($val =~ /^\s*0\s*$/) {
 	return 0;
     } elsif ($val =~ /^\s*\d+\s*$/) {
@@ -428,8 +464,8 @@ sub read_config {
 		$repeat_tests{"$test_num"} = $repeat;
 	    }
 
-	    if ($rest =~ /\sIF\s+(\S*)(.*)/) {
-		$rest = $2;
+	    if ($rest =~ /\sIF\s+(.*)/) {
+		$rest = "";
 		if (process_if($name, $1)) {
 		    $if_set = 1;
 		} else {
@@ -461,14 +497,14 @@ sub read_config {
 		$skip = 0;
 	    }
 
-	    if ($rest =~ /\sIF\s+(\S*)(.*)/) {
+	    if ($rest =~ /\sIF\s+(.*)/) {
 		$if = 1;
-		$rest = $2;
 		if (process_if($name, $1)) {
 		    $if_set = 1;
 		} else {
 		    $skip = 1;
 		}
+		$rest = "";
 	    } else {
 		$if = 0;
 	    }
@@ -477,26 +513,32 @@ sub read_config {
 		die "$name: $.: Gargbage found after DEFAULTS\n$_";
 	    }
 
-	} elsif (/^\s*ELSE(.*)$/) {
+	} elsif (/^\s*ELSE\b(.*)$/) {
 	    if (!$if) {
 		die "$name: $.: ELSE found with out matching IF section\n$_";
 	    }
 	    $rest = $1;
 	    if ($if_set) {
 		$skip = 1;
+		$rest = "";
 	    } else {
 		$skip = 0;
 
-		if ($rest =~ /\sIF\s+(\S*)(.*)/) {
+		if ($rest =~ /\sIF\s+(.*)/) {
 		    # May be a ELSE IF section.
 		    if (!process_if($name, $1)) {
 			$skip = 1;
 		    }
+		    $rest = "";
 		} else {
 		    $if = 0;
 		}
 	    }
 
+	    if ($rest !~ /^\s*$/) {
+		die "$name: $.: Gargbage found after DEFAULTS\n$_";
+	    }
+
 	} elsif (/^\s*([A-Z_\[\]\d]+)\s*=\s*(.*?)\s*$/) {
 
 	    next if ($skip);
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index 6a0a0ba..4e8fb91 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -72,6 +72,8 @@
 # the same option name under the same test or as default
 # ktest will fail to execute, and no tests will run.
 #
+#
+#
 # Both TEST_START and DEFAULTS sections can also have the IF keyword
 # The value after the IF must evaluate into a 0 or non 0 positive
 # integer, and can use the config variables (explained below).
@@ -110,6 +112,18 @@
 # ELSE
 # BUILD_TYPE = useconfig:${CONFIG_DIR}/config-network
 #
+# The if statement may also have comparisons that will and for
+# == and !=, strings may be used for both sides.
+#
+# BOX_TYPE := x86_32
+#
+# DEFAULTS IF ${BOX_TYPE} == x86_32
+# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-32
+# ELSE
+# BUILD_TYPE = useconfig:${CONFIG_DIR}/config-64
+#
+
+
 
 #### Config variables ####
 #
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 11/21] ktest: Add INCLUDE keyword to include other config files

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 4231 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Have the reading of the config file allow reading of other config
files using the INCLUDE keyword. This allows multiple config files
to share config options.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   55 ++++++++++++++++++++++++++++++++++-----
 tools/testing/ktest/sample.conf |   32 +++++++++++++++++++++-
 2 files changed, 78 insertions(+), 9 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index ed20d68..62de47d 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -412,15 +412,16 @@ sub process_if {
     return 1;
 }
 
-sub read_config {
-    my ($config) = @_;
+sub __read_config {
+    my ($config, $current_test_num) = @_;
 
-    open(IN, $config) || die "can't read file $config";
+    my $in;
+    open($in, $config) || die "can't read file $config";
 
     my $name = $config;
     $name =~ s,.*/(.*),$1,;
 
-    my $test_num = 0;
+    my $test_num = $$current_test_num;
     my $default = 1;
     my $repeat = 1;
     my $num_tests_set = 0;
@@ -430,7 +431,7 @@ sub read_config {
     my $if = 0;
     my $if_set = 0;
 
-    while (<IN>) {
+    while (<$in>) {
 
 	# ignore blank lines and comments
 	next if (/^\s*$/ || /\s*\#/);
@@ -539,6 +540,33 @@ sub read_config {
 		die "$name: $.: Gargbage found after DEFAULTS\n$_";
 	    }
 
+	} elsif (/^\s*INCLUDE\s+(\S+)/) {
+
+	    next if ($skip);
+
+	    if (!$default) {
+		die "$name: $.: INCLUDE can only be done in default sections\n$_";
+	    }
+
+	    my $file = process_variables($1);
+
+	    if ($file !~ m,^/,) {
+		# check the path of the config file first
+		if ($config =~ m,(.*)/,) {
+		    if (-f "$1/$file") {
+			$file = "$1/$file";
+		    }
+		}
+	    }
+		
+	    if ( ! -r $file ) {
+		die "$name: $.: Can't read file $file\n$_";
+	    }
+
+	    if (__read_config($file, \$test_num)) {
+		$test_case = 1;
+	    }
+
 	} elsif (/^\s*([A-Z_\[\]\d]+)\s*=\s*(.*?)\s*$/) {
 
 	    next if ($skip);
@@ -594,13 +622,26 @@ sub read_config {
 	}
     }
 
-    close(IN);
-
     if ($test_num) {
 	$test_num += $repeat - 1;
 	$opt{"NUM_TESTS"} = $test_num;
     }
 
+    close($in);
+
+    $$current_test_num = $test_num;
+
+    return $test_case;
+}
+
+sub read_config {
+    my ($config) = @_;
+
+    my $test_case;
+    my $test_num = 0;
+
+    $test_case = __read_config $config, \$test_num;
+
     # make sure we have all mandatory configs
     get_ktest_configs;
 
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index 4e8fb91..ae2a93c 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -122,8 +122,36 @@
 # ELSE
 # BUILD_TYPE = useconfig:${CONFIG_DIR}/config-64
 #
-
-
+#
+# INCLUDE file
+#
+# The INCLUDE keyword may be used in DEFAULT sections. This will
+# read another config file and process that file as well. The included
+# file can include other files, add new test cases or default
+# statements. Config variables will be passed to these files and changes
+# to config variables will be seen by top level config files. Including
+# a file is processed just like the contents of the file was cut and pasted
+# into the top level file, except, that include files that end with
+# TEST_START sections will have that section ended at the end of
+# the include file. That is, an included file is included followed
+# by another DEFAULT keyword.
+#
+# Unlike other files referenced in this config, the file path does not need
+# to be absolute. If the file does not start with '/', then the directory
+# that the current config file was located in is used. If no config by the
+# given name is found there, then the current directory is searched.
+#
+# INCLUDE myfile
+# DEFAULT
+#
+# is the same as:
+#
+# INCLUDE myfile
+#
+# Note, if the include file does not contain a full path, the file is
+# searched first by the location of the original include file, and then
+# by the location that ktest.pl was executed in.
+#
 
 #### Config variables ####
 #
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 12/21] ktest: Consolidate TEST_TYPE and DEFAULT code

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 2497 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

The code that handles parsing the TEST_TYPE and DEFAULT code share
a lot of common functionality. Combine the two and add a if statement
that does what is different between them.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   61 ++++++++++++++---------------------------
 1 files changed, 21 insertions(+), 40 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 62de47d..b4f32e7 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -436,20 +436,29 @@ sub __read_config {
 	# ignore blank lines and comments
 	next if (/^\s*$/ || /\s*\#/);
 
-	if (/^\s*TEST_START(.*)/) {
+	if (/^\s*(TEST_START|DEFAULTS)\b(.*)/) {
 
-	    $rest = $1;
+	    my $type = $1;
+	    $rest = $2;
 
-	    if ($num_tests_set) {
-		die "$name: $.: Can not specify both NUM_TESTS and TEST_START\n";
-	    }
+	    my $old_test_num;
+	    my $old_repeat;
+
+	    if ($type eq "TEST_START") {
 
-	    my $old_test_num = $test_num;
-	    my $old_repeat = $repeat;
+		if ($num_tests_set) {
+		    die "$name: $.: Can not specify both NUM_TESTS and TEST_START\n";
+		}
 
-	    $test_num += $repeat;
-	    $default = 0;
-	    $repeat = 1;
+		$old_test_num = $test_num;
+		$old_repeat = $repeat;
+
+		$test_num += $repeat;
+		$default = 0;
+		$repeat = 1;
+	    } else {
+		$default = 1;
+	    }
 
 	    if ($rest =~ /\s+SKIP\b(.*)/) {
 		$rest = $1;
@@ -478,42 +487,14 @@ sub __read_config {
 	    }
 
 	    if ($rest !~ /^\s*$/) {
-		die "$name: $.: Gargbage found after TEST_START\n$_";
+		die "$name: $.: Gargbage found after $type\n$_";
 	    }
 
-	    if ($skip) {
+	    if ($skip && $type eq "TEST_START") {
 		$test_num = $old_test_num;
 		$repeat = $old_repeat;
 	    }
 
-	} elsif (/^\s*DEFAULTS(.*)$/) {
-	    $default = 1;
-
-	    $rest = $1;
-
-	    if ($rest =~ /\s+SKIP(.*)/) {
-		$rest = $1;
-		$skip = 1;
-	    } else {
-		$skip = 0;
-	    }
-
-	    if ($rest =~ /\sIF\s+(.*)/) {
-		$if = 1;
-		if (process_if($name, $1)) {
-		    $if_set = 1;
-		} else {
-		    $skip = 1;
-		}
-		$rest = "";
-	    } else {
-		$if = 0;
-	    }
-
-	    if ($rest !~ /^\s*$/) {
-		die "$name: $.: Gargbage found after DEFAULTS\n$_";
-	    }
-
 	} elsif (/^\s*ELSE\b(.*)$/) {
 	    if (!$if) {
 		die "$name: $.: ELSE found with out matching IF section\n$_";
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 13/21] ktest: Add OVERRIDE keyword to DEFAULTS section

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 3982 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

The OVERRIDE keyword will allow options defined in the given
DEFAULTS section to override options defined in previous DEFAULT
sections.

Normally, options will error if they were previous defined.
The OVERRIDE keyword allows options that have been previously
defined to be changed in the given section.

Note, the same option can not be defined in the same DEFAULT section
even if that section is marked as OVERRIDE.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   37 +++++++++++++++++++++++++++++--------
 tools/testing/ktest/sample.conf |   15 +++++++++++++++
 2 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index b4f32e7..7bce412 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -337,10 +337,17 @@ sub process_variables {
 }
 
 sub set_value {
-    my ($lvalue, $rvalue) = @_;
+    my ($lvalue, $rvalue, $override, $overrides, $name) = @_;
 
     if (defined($opt{$lvalue})) {
-	die "Error: Option $lvalue defined more than once!\n";
+	if (!$override || defined(${$overrides}{$lvalue})) {
+	    my $extra = "";
+	    if ($override) {
+		$extra = "In the same override section!\n";
+	    }
+	    die "$name: $.: Option $lvalue defined more than once!\n$extra";
+	}
+	${$overrides}{$lvalue} = $rvalue;
     }
     if ($rvalue =~ /^\s*$/) {
 	delete $opt{$lvalue};
@@ -430,6 +437,9 @@ sub __read_config {
     my $test_case = 0;
     my $if = 0;
     my $if_set = 0;
+    my $override = 0;
+
+    my %overrides;
 
     while (<$in>) {
 
@@ -443,6 +453,7 @@ sub __read_config {
 
 	    my $old_test_num;
 	    my $old_repeat;
+	    $override = 0;
 
 	    if ($type eq "TEST_START") {
 
@@ -468,10 +479,20 @@ sub __read_config {
 		$skip = 0;
 	    }
 
-	    if ($rest =~ /\s+ITERATE\s+(\d+)(.*)$/) {
-		$repeat = $1;
-		$rest = $2;
-		$repeat_tests{"$test_num"} = $repeat;
+	    if (!$skip) {
+		if ($type eq "TEST_START") {
+		    if ($rest =~ /\s+ITERATE\s+(\d+)(.*)$/) {
+			$repeat = $1;
+			$rest = $2;
+			$repeat_tests{"$test_num"} = $repeat;
+		    }
+		} elsif ($rest =~ /\sOVERRIDE\b(.*)/) {
+		    # DEFAULT only
+		    $rest = $1;
+		    $override = 1;
+		    # Clear previous overrides
+		    %overrides = ();
+		}
 	    }
 
 	    if ($rest =~ /\sIF\s+(.*)/) {
@@ -573,10 +594,10 @@ sub __read_config {
 	    }
 
 	    if ($default || $lvalue =~ /\[\d+\]$/) {
-		set_value($lvalue, $rvalue);
+		set_value($lvalue, $rvalue, $override, \%overrides, $name);
 	    } else {
 		my $val = "$lvalue\[$test_num\]";
-		set_value($val, $rvalue);
+		set_value($val, $rvalue, $override, \%overrides, $name);
 
 		if ($repeat > 1) {
 		    $repeats{$val} = $repeat;
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index ae2a93c..0fd3ca3 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -72,6 +72,21 @@
 # the same option name under the same test or as default
 # ktest will fail to execute, and no tests will run.
 #
+# DEFAULTS OVERRIDE
+#
+# Options defined in the DEFAULTS section can not be duplicated
+# even if they are defined in two different DEFAULT sections.
+# This is done to catch mistakes where an option is added but
+# the previous option was forgotten about and not commented.
+#
+# The OVERRIDE keyword can be added to a section to allow this
+# section to override other DEFAULT sections values that have
+# been defined previously. It will only override options that
+# have been defined before its use. Options defined later
+# in a non override section will still error. The same option
+# can not be defined in the same section even if that section
+# is marked OVERRIDE.
+#
 #
 #
 # Both TEST_START and DEFAULTS sections can also have the IF keyword
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 14/21] ktest: Add DEFINED keyword for IF statements

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 2107 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Have IF statements process if a config variable or option has been
defined or not. Can use NOT DEFINED in the case for telling if
a variable or option has not been defined.

DEFAULTS IF NOT DEFINED SSH_USER
SSH_USER = root

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   17 ++++++++++++++++-
 tools/testing/ktest/sample.conf |   17 +++++++++++++++++
 2 files changed, 33 insertions(+), 1 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 7bce412..76a5964 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -396,6 +396,13 @@ sub process_compare {
     return $ret;
 }
 
+sub value_defined {
+    my ($val) = @_;
+
+    return defined($variable{$2}) ||
+	defined($opt{$2});
+}
+
 sub process_if {
     my ($name, $value) = @_;
 
@@ -409,13 +416,21 @@ sub process_if {
 	return $ret;
     }
 
+    if ($val =~ /^\s*(NOT\s*)?DEFINED\s+(\S+)\s*$/) {
+	if (defined $1) {
+	    return !value_defined($2);
+	} else {
+	    return value_defined($2);
+	}
+    }
+
     if ($val =~ /^\s*0\s*$/) {
 	return 0;
     } elsif ($val =~ /^\s*\d+\s*$/) {
 	return 1;
     }
 
-    die ("$name: $.: Undefined variable $val in if statement\n");
+    die ("$name: $.: Undefined content $val in if statement\n");
     return 1;
 }
 
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index 0fd3ca3..7b49f07 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -137,6 +137,23 @@
 # ELSE
 # BUILD_TYPE = useconfig:${CONFIG_DIR}/config-64
 #
+# The DEFINED keyword can be used by the IF statements too.
+# It returns true if the given config variable or option has been defined
+# or false otherwise.
+#
+# 
+# DEFAULTS IF DEFINED USE_CC
+# CC := ${USE_CC}
+# ELSE
+# CC := gcc
+#
+#
+# As well as NOT DEFINED.
+#
+# DEFAULTS IF NOT DEFINED MAKE_CMD
+# MAKE_CMD := make ARCH=x86
+#
+#
 #
 # INCLUDE file
 #
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 15/21] ktest: Sort make_min_config configs by dependecies

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 4141 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

The make_min_config test will turn off one config at a time and check
if the config boots or not, and if it does, it will remove that config
plus any config that depended on that config.

ktest already looks if a config has a dependency and will try the
dependency config first. But by sorting the configs and trying the
config with the most configs dependent on it, we can shrink the
minconfig faster.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   71 +++++++++++++++++++++++++++++-------------
 1 files changed, 49 insertions(+), 22 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 76a5964..2a9d042 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -2372,12 +2372,31 @@ sub patchcheck {
 }
 
 my %depends;
+my %depcount;
 my $iflevel = 0;
 my @ifdeps;
 
 # prevent recursion
 my %read_kconfigs;
 
+sub add_dep {
+    # $config depends on $dep
+    my ($config, $dep) = @_;
+
+    if (defined($depends{$config})) {
+	$depends{$config} .= " " . $dep;
+    } else {
+	$depends{$config} = $dep;
+    }
+
+    # record the number of configs depending on $dep
+    if (defined $depcount{$dep}) {
+	$depcount{$dep}++;
+    } else {
+	$depcount{$dep} = 1;
+    } 
+}
+
 # taken from streamline_config.pl
 sub read_kconfig {
     my ($kconfig) = @_;
@@ -2424,30 +2443,19 @@ sub read_kconfig {
 	    $config = $2;
 
 	    for (my $i = 0; $i < $iflevel; $i++) {
-		if ($i) {
-		    $depends{$config} .= " " . $ifdeps[$i];
-		} else {
-		    $depends{$config} = $ifdeps[$i];
-		}
-		$state = "DEP";
+		add_dep $config, $ifdeps[$i];
 	    }
 
 	# collect the depends for the config
 	} elsif ($state eq "NEW" && /^\s*depends\s+on\s+(.*)$/) {
 
-	    if (defined($depends{$1})) {
-		$depends{$config} .= " " . $1;
-	    } else {
-		$depends{$config} = $1;
-	    }
+	    add_dep $config, $1;
 
 	# Get the configs that select this config
-	} elsif ($state ne "NONE" && /^\s*select\s+(\S+)/) {
-	    if (defined($depends{$1})) {
-		$depends{$1} .= " " . $config;
-	    } else {
-		$depends{$1} = $config;
-	    }
+	} elsif ($state eq "NEW" && /^\s*select\s+(\S+)/) {
+
+	    # selected by depends on config
+	    add_dep $1, $config;
 
 	# Check for if statements
 	} elsif (/^if\s+(.*\S)\s*$/) {
@@ -2559,11 +2567,18 @@ sub make_new_config {
     close OUT;
 }
 
+sub chomp_config {
+    my ($config) = @_;
+
+    $config =~ s/CONFIG_//;
+
+    return $config;
+}
+
 sub get_depends {
     my ($dep) = @_;
 
-    my $kconfig = $dep;
-    $kconfig =~ s/CONFIG_//;
+    my $kconfig = chomp_config $dep;
 
     $dep = $depends{"$kconfig"};
 
@@ -2613,8 +2628,7 @@ sub test_this_config {
 	return undef;
     }
 
-    my $kconfig = $config;
-    $kconfig =~ s/CONFIG_//;
+    my $kconfig = chomp_config $config;
 
     # Test dependencies first
     if (defined($depends{"$kconfig"})) {
@@ -2704,6 +2718,14 @@ sub make_min_config {
 
     my @config_keys = keys %min_configs;
 
+    # All configs need a depcount
+    foreach my $config (@config_keys) {
+	my $kconfig = chomp_config $config;
+	if (!defined $depcount{$kconfig}) {
+		$depcount{$kconfig} = 0;
+	}
+    }
+
     # Remove anything that was set by the make allnoconfig
     # we shouldn't need them as they get set for us anyway.
     foreach my $config (@config_keys) {
@@ -2742,8 +2764,13 @@ sub make_min_config {
 	# Now disable each config one by one and do a make oldconfig
 	# till we find a config that changes our list.
 
-	# Put configs that did not modify the config at the end.
 	my @test_configs = keys %min_configs;
+
+	# Sort keys by who is most dependent on
+	@test_configs = sort  { $depcount{chomp_config($b)} <=> $depcount{chomp_config($a)} }
+			  @test_configs ;
+
+	# Put configs that did not modify the config at the end.
 	my $reset = 1;
 	for (my $i = 0; $i < $#test_configs; $i++) {
 	    if (!defined($nochange_config{$test_configs[0]})) {
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 16/21] ktest: Fix parsing of config section lines

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 2759 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

The order for some of the keywords on a section line
(TEST_START or DEFAULTS) does not really matter. Simply need
to remove the keyword from the line as we process it and
evaluate the next keyword in the line. By removing the keywords
as we find them, we do not need to keep track of where on the
line they were found.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   49 ++++++++++++++++++++++++++---------------
 1 files changed, 31 insertions(+), 18 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 2a9d042..d292c2d 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -449,6 +449,7 @@ sub __read_config {
     my $num_tests_set = 0;
     my $skip = 0;
     my $rest;
+    my $line;
     my $test_case = 0;
     my $if = 0;
     my $if_set = 0;
@@ -465,6 +466,7 @@ sub __read_config {
 
 	    my $type = $1;
 	    $rest = $2;
+	    $line = $2;
 
 	    my $old_test_num;
 	    my $old_repeat;
@@ -486,32 +488,28 @@ sub __read_config {
 		$default = 1;
 	    }
 
-	    if ($rest =~ /\s+SKIP\b(.*)/) {
-		$rest = $1;
+	    # If SKIP is anywhere in the line, the command will be skipped
+	    if ($rest =~ s/\s+SKIP\b//) {
 		$skip = 1;
 	    } else {
 		$test_case = 1;
 		$skip = 0;
 	    }
 
-	    if (!$skip) {
-		if ($type eq "TEST_START") {
-		    if ($rest =~ /\s+ITERATE\s+(\d+)(.*)$/) {
-			$repeat = $1;
-			$rest = $2;
-			$repeat_tests{"$test_num"} = $repeat;
-		    }
-		} elsif ($rest =~ /\sOVERRIDE\b(.*)/) {
-		    # DEFAULT only
-		    $rest = $1;
-		    $override = 1;
-		    # Clear previous overrides
-		    %overrides = ();
+	    if ($rest =~ s/\sELSE\b//) {
+		if (!$if) {
+		    die "$name: $.: ELSE found with out matching IF section\n$_";
+		}
+		$if = 0;
+
+		if ($if_set) {
+		    $skip = 1;
+		} else {
+		    $skip = 0;
 		}
 	    }
 
-	    if ($rest =~ /\sIF\s+(.*)/) {
-		$rest = "";
+	    if ($rest =~ s/\sIF\s+(.*)//) {
 		if (process_if($name, $1)) {
 		    $if_set = 1;
 		} else {
@@ -520,9 +518,24 @@ sub __read_config {
 		$if = 1;
 	    } else {
 		$if = 0;
+		$if_set = 0;
 	    }
 
-	    if ($rest !~ /^\s*$/) {
+	    if (!$skip) {
+		if ($type eq "TEST_START") {
+		    if ($rest =~ s/\s+ITERATE\s+(\d+)//) {
+			$repeat = $1;
+			$repeat_tests{"$test_num"} = $repeat;
+		    }
+		} elsif ($rest =~ s/\sOVERRIDE\b//) {
+		    # DEFAULT only
+		    $override = 1;
+		    # Clear previous overrides
+		    %overrides = ();
+		}
+	    }
+
+	    if (!$skip && $rest !~ /^\s*$/) {
 		die "$name: $.: Gargbage found after $type\n$_";
 	    }
 
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 17/21] ktest: Add processing of complex conditionals

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 3286 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

The IF statements for DEFAULTS and TEST_START sections now handle
complex statements (&&,||)

Example:

  TEST_START IF (DEFINED ALL_TESTS || ${MYTEST} == boottest) && ${MACHINE} == gandalf

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   53 +++++++++++++++++++++++++++++++++++---
 tools/testing/ktest/sample.conf |   10 +++++++
 2 files changed, 58 insertions(+), 5 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index d292c2d..1bda07f 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -304,7 +304,7 @@ sub get_ktest_configs {
 }
 
 sub process_variables {
-    my ($value) = @_;
+    my ($value, $remove_undef) = @_;
     my $retval = "";
 
     # We want to check for '\', and it is just easier
@@ -322,6 +322,10 @@ sub process_variables {
 	$retval = "$retval$begin";
 	if (defined($variable{$var})) {
 	    $retval = "$retval$variable{$var}";
+	} elsif (defined($remove_undef) && $remove_undef) {
+	    # for if statements, any variable that is not defined,
+	    # we simple convert to 0
+	    $retval = "${retval}0";
 	} else {
 	    # put back the origin piece.
 	    $retval = "$retval\$\{$var\}";
@@ -403,10 +407,40 @@ sub value_defined {
 	defined($opt{$2});
 }
 
-sub process_if {
-    my ($name, $value) = @_;
+my $d = 0;
+sub process_expression {
+    my ($name, $val) = @_;
+
+    my $c = $d++;
+
+    while ($val =~ s/\(([^\(]*?)\)/\&\&\&\&VAL\&\&\&\&/) {
+	my $express = $1;
+
+	if (process_expression($name, $express)) {
+	    $val =~ s/\&\&\&\&VAL\&\&\&\&/ 1 /;
+	} else {
+	    $val =~ s/\&\&\&\&VAL\&\&\&\&/ 0 /;
+	}
+    }
+
+    $d--;
+    my $OR = "\\|\\|";
+    my $AND = "\\&\\&";
 
-    my $val = process_variables($value);
+    while ($val =~ s/^(.*?)($OR|$AND)//) {
+	my $express = $1;
+	my $op = $2;
+
+	if (process_expression($name, $express)) {
+	    if ($op eq "||") {
+		return 1;
+	    }
+	} else {
+	    if ($op eq "&&") {
+		return 0;
+	    }
+	}
+    }
 
     if ($val =~ /(.*)(==|\!=|>=|<=|>|<)(.*)/) {
 	my $ret = process_compare($1, $2, $3);
@@ -431,7 +465,16 @@ sub process_if {
     }
 
     die ("$name: $.: Undefined content $val in if statement\n");
-    return 1;
+}
+
+sub process_if {
+    my ($name, $value) = @_;
+
+    # Convert variables and replace undefined ones with 0
+    my $val = process_variables($value, 1);
+    my $ret = process_expression $name, $val;
+
+    return $ret;
 }
 
 sub __read_config {
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index 7b49f07..dbedfa1 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -154,6 +154,16 @@
 # MAKE_CMD := make ARCH=x86
 #
 #
+# And/or ops (&&,||) may also be used to make complex conditionals.
+#
+# TEST_START IF (DEFINED ALL_TESTS || ${MYTEST} == boottest) && ${MACHINE} == gandalf
+#
+# Notice the use of paranthesis. Without any paranthesis the above would be
+# processed the same as:
+#
+# TEST_START IF DEFINED ALL_TESTS || (${MYTEST} == boottest && ${MACHINE} == gandalf)
+#
+#
 #
 # INCLUDE file
 #
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 18/21] ktest: Do not opencode reboot in grub setting

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 1033 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

When setting the next kernel to boot to with grub, do not opencode
the reboot operation.  The normal reboot operation can be modified by
config options (namely POWERCYCLE_AFTER_REBOOT). This needs to affect
all reboots. Remove the opencoded reboot to make sure that any changes
to the reboot code also affect all reboots.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 1bda07f..d60c496 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -1150,7 +1150,8 @@ sub wait_for_input
 
 sub reboot_to {
     if ($reboot_type eq "grub") {
-	run_ssh "'(echo \"savedefault --default=$grub_number --once\" | grub --batch && reboot)'";
+	run_ssh "'(echo \"savedefault --default=$grub_number --once\" | grub --batch)'";
+	reboot;
 	return;
     }
 
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 19/21] ktest: Add another monitor flush before installing kernel

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 1053 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

On some tests that do multiple boots (patchcheck, bisect, etc), the build
of the next kernel to run may finish before the stable kernel has finished
booting. Then the install of the new kernel will fail when it tries to connect
as the machine has not finished the boot process.

Do one more monitor flush to make sure the machine is up and running before
trying to connect to it again.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index d60c496..652446d 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -1391,6 +1391,11 @@ sub get_version {
 }
 
 sub start_monitor_and_boot {
+    # Make sure the stable kernel has finished booting
+    start_monitor;
+    wait_for_monitor 5;
+    end_monitor;
+
     get_grub_index;
     get_version;
     install;
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 20/21] ktest: Add variable ${PWD}

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 752 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Adding the variable ${PWD} that equals `pwd` makes the config files
much simpler.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 652446d..9d9ee32 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -136,6 +136,9 @@ my %force_config;
 # do not force reboots on config problems
 my $no_reboot = 1;
 
+# default variables that can be used
+chomp ($variable{"PWD"} = `pwd`);
+
 $config_help{"MACHINE"} = << "EOF"
  The machine hostname that you will test.
 EOF
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 21/21] ktest: Evaluate variables entered on the command line

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 1495 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

When ktest.pl is called without any arguments, or if the config
file does not exist, ktest.pl will ask the user for some information.
Some of these questions are code paths. Allowing the user to type
${PWD} for the current directory greatly simplifies these entries.

Add variable processing to the entered values.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   10 ++++++----
 1 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 9d9ee32..30e2bef 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -250,6 +250,7 @@ sub read_yn {
 
 sub get_ktest_config {
     my ($config) = @_;
+    my $ans;
 
     return if (defined($opt{$config}));
 
@@ -263,16 +264,17 @@ sub get_ktest_config {
 	if (defined($default{$config})) {
 	    print "\[$default{$config}\] ";
 	}
-	$entered_configs{$config} = <STDIN>;
-	$entered_configs{$config} =~ s/^\s*(.*\S)\s*$/$1/;
-	if ($entered_configs{$config} =~ /^\s*$/) {
+	$ans = <STDIN>;
+	$ans =~ s/^\s*(.*\S)\s*$/$1/;
+	if ($ans =~ /^\s*$/) {
 	    if ($default{$config}) {
-		$entered_configs{$config} = $default{$config};
+		$ans = $default{$config};
 	    } else {
 		print "Your answer can not be blank\n";
 		next;
 	    }
 	}
+	$entered_configs{$config} = process_variables($ans);
 	last;
     }
 }
-- 
1.7.6.3



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 00/21] [GIT PULL] ktest: lots of nice new features

[Steven Rostedt]

On Fri, 2011-10-28 at 07:15 -0400, Steven Rostedt wrote:
> Linus,
> 
> This is the udates I have done since the last merge window. Some included fixes
> I needed to present my tutorial at LinuxCon. My tutorial was based on the feature
> set added by this patch pull.
> 
> Please pull the following patch set:
> 
> HEAD SHA1: 815e2bd7d609da9c7615ea28a3990064a394312f


Hi Linus,

I noticed the merge window closed and this pull request didn't make it.
Was there an issue with it, or did it just get lost in the noise?

Thanks,

-- Steve



> 
> 
> The following patches are in:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest.git
> 
>     branch: for-linus
> 
> 
> Andrew Jones (3):
>       ktest: Create outputdir if it does not exist
>       ktest: Only need to save .config when doing mrproper
>       ktest: Include monitor in reboot code
> 
> Steven Rostedt (18):
>       ktest: Add TEST_TYPE install option
>       ktest: Fail when grub menu not found
>       ktest: Add NO_INSTALL option to not install for a test
>       ktest: Add option REBOOT_SUCCESS_LINE to stop waiting after a reboot
>       ktest: Do not reboot on config or build issues
>       ktest: Add IF and ELSE to config sections
>       ktest: Let IF keyword take comparisons
>       ktest: Add INCLUDE keyword to include other config files
>       ktest: Consolidate TEST_TYPE and DEFAULT code
>       ktest: Add OVERRIDE keyword to DEFAULTS section
>       ktest: Add DEFINED keyword for IF statements
>       ktest: Sort make_min_config configs by dependecies
>       ktest: Fix parsing of config section lines
>       ktest: Add processing of complex conditionals
>       ktest: Do not opencode reboot in grub setting
>       ktest: Add another monitor flush before installing kernel
>       ktest: Add variable ${PWD}
>       ktest: Evaluate variables entered on the command line
> 
> ----
>  tools/testing/ktest/ktest.pl    |  515 +++++++++++++++++++++++++++++++--------
>  tools/testing/ktest/sample.conf |  146 +++++++++++-
>  2 files changed, 553 insertions(+), 108 deletions(-)