[PATCH 00/19] [GIT PULL] ktest: new features and fixes

[PATCH 00/19] [GIT PULL] ktest: new features and fixes

[Steven Rostedt]

Linus,

Please pull the following patches.

The following patches are in:

  git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-ktest.git

    branch: for-linus


Steven Rostedt (19):
      ktest: Notify reason to break out of monitoring boot
      ktest: Add detection of triple faults
      ktest: Add CONFIG_BISECT_GOOD option
      ktest: Add TEST_NAME option
      ktest: Implement our own force min config
      ktest: Have wait on stdio honor bug timeout
      ktest: Have LOG_FILE evaluate options as well
      ktest: Allow initrd processing without modules defined
      ktest: Add POST/PRE_BUILD options
      ktest: Have the testing tmp dir include machine name
      ktest: Fix tar extracting of modules to target
      ktest: Add IGNORE_WARNINGS to ignore warnings in some patches
      ktest: Add helper function to avoid duplicate code
      ktest: Require one TEST_START in config file
      ktest: Add test type make_min_config
      ktest: Use Kconfig dependencies to shorten time to make min_config
      ktest: Add prompt to use OUTPUT_MIN_CONFIG
      ktest: Keep fonud configs separate from default configs
      ktest: Fix bug when ADD_CONFIG is set but MIN_CONFIG is not

----
 tools/testing/ktest/ktest.pl    |  970 +++++++++++++++++++++++++++++++++------
 tools/testing/ktest/sample.conf |  142 ++++++-
 2 files changed, 963 insertions(+), 149 deletions(-)

[PATCH 01/19] ktest: Notify reason to break out of monitoring boot

[Steven Rostedt]

[-- Attachment #1: 0001-ktest-Notify-reason-to-break-out-of-monitoring-boot.patch --]
[-- Type: text/plain, Size: 1151 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Different timeouts can cause the ktest monitor to break out of the
loop. It becomes annoying that one does not know the reason why
it exited the monitor loop. Display the cause of the reason why
the loop was exited.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   12 ++++++++++--
 1 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index cef28e6..b96d381 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -841,12 +841,20 @@ sub monitor {
 
 	if ($booted) {
 	    $line = wait_for_input($monitor_fp, $booted_timeout);
+	    if (!defined($line)) {
+		my $s = $booted_timeout == 1 ? "" : "s";
+		doprint "Successful boot found: break after $booted_timeout second$s\n";
+		last;
+	    }
 	} else {
 	    $line = wait_for_input($monitor_fp);
+	    if (!defined($line)) {
+		my $s = $timeout == 1 ? "" : "s";
+		doprint "Timed out after $timeout second$s\n";
+		last;
+	    }
 	}
 
-	last if (!defined($line));
-
 	doprint $line;
 	print DMESG $line;
 
-- 
1.7.5.4

[PATCH 02/19] ktest: Add detection of triple faults

[Steven Rostedt]

[-- Attachment #1: 0002-ktest-Add-detection-of-triple-faults.patch --]
[-- Type: text/plain, Size: 4185 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

When a triple fault happens in a test, no call trace nor panic
is displayed. Instead, the system reboots to the good kernel.
Since the good kernel may display a boot prompt that matches the
success string, ktest may think that the test succeeded, when it
did not.

Detecting triple faults is tricky because it is hard to generalize
what a reboot looks like. The best that we can come up with for now
is to examine the Linux banner. If we detect that the Linux banner
matches the test we want to test, then look to see if we hit another
Linux banner with a different kernel is booted. This can be assumed
to be a triple fault.

We can't just check for two Linux banners because things like
early printk may cause the Linux banner to be displayed twice. Checking
for different kernel versions should be the safe bet.

If this for some reason detects a false triple boot. A new ktest
config option is also created:

 DETECT_TRIPLE_FAULT

This can be set to 0 to disable this checking.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   20 ++++++++++++++++++++
 tools/testing/ktest/sample.conf |   10 ++++++++++
 2 files changed, 30 insertions(+), 0 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index b96d381..a8e1826 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -41,6 +41,7 @@ $default{"CLEAR_LOG"}		= 0;
 $default{"BISECT_MANUAL"}	= 0;
 $default{"BISECT_SKIP"}		= 1;
 $default{"SUCCESS_LINE"}	= "login:";
+$default{"DETECT_TRIPLE_FAULT"} = 1;
 $default{"BOOTED_TIMEOUT"}	= 1;
 $default{"DIE_ON_FAILURE"}	= 1;
 $default{"SSH_EXEC"}		= "ssh \$SSH_USER\@\$MACHINE \$SSH_COMMAND";
@@ -101,6 +102,7 @@ my $patchcheck_sleep_time;
 my $store_failures;
 my $timeout;
 my $booted_timeout;
+my $detect_triplefault;
 my $console;
 my $success_line;
 my $stop_after_success;
@@ -836,6 +838,7 @@ sub monitor {
     my $failure_start;
     my $monitor_start = time;
     my $done = 0;
+    my $version_found = 0;
 
     while (!$done) {
 
@@ -904,6 +907,22 @@ sub monitor {
 	    $bug = 1;
 	}
 
+	# Detect triple faults by testing the banner
+	if ($full_line =~ /\bLinux version (\S+).*\n/) {
+	    if ($1 eq $version) {
+		$version_found = 1;
+	    } elsif ($version_found && $detect_triplefault) {
+		# We already booted into the kernel we are testing,
+		# but now we booted into another kernel?
+		# Consider this a triple fault.
+		doprint "Aleady booted in Linux kernel $version, but now\n";
+		doprint "we booted into Linux kernel $1.\n";
+		doprint "Assuming that this is a triple fault.\n";
+		doprint "To disable this: set DETECT_TRIPLE_FAULT to 0\n";
+		last;
+	    }
+	}
+
 	if ($line =~ /\n/) {
 	    $full_line = "";
 	}
@@ -2159,6 +2178,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     $timeout = set_test_option("TIMEOUT", $i);
     $booted_timeout = set_test_option("BOOTED_TIMEOUT", $i);
     $console = set_test_option("CONSOLE", $i);
+    $detect_triplefault = set_test_option("DETECT_TRIPLE_FAULT", $i);
     $success_line = set_test_option("SUCCESS_LINE", $i);
     $stop_after_success = set_test_option("STOP_AFTER_SUCCESS", $i);
     $stop_after_failure = set_test_option("STOP_AFTER_FAILURE", $i);
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index 48cbcc80..c2c072e 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -518,6 +518,16 @@
 # The variables SSH_USER and MACHINE are defined.
 #REBOOT = ssh $SSH_USER@$MACHINE reboot
 
+# The way triple faults are detected is by testing the kernel
+# banner. If the kernel banner for the kernel we are testing is
+# found, and then later a kernel banner for another kernel version
+# is found, it is considered that we encountered a triple fault,
+# and there is no panic or callback, but simply a reboot.
+# To disable this (because it did a false positive) set the following
+# to 0.
+# (default 1)
+#DETECT_TRIPLE_FAULT = 0
+
 #### Per test run options ####
 # The following options are only allowed in TEST_START sections.
 # They are ignored in the DEFAULTS sections.
-- 
1.7.5.4

[PATCH 03/19] ktest: Add CONFIG_BISECT_GOOD option

[Steven Rostedt]

[-- Attachment #1: 0003-ktest-Add-CONFIG_BISECT_GOOD-option.patch --]
[-- Type: text/plain, Size: 3277 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Currently the config_bisect compares the min config with the
CONFIG_BISECT config. There may be another config that we know
is good that we want to ignore configs on. By passing in this
config it will ignore the options that are set in the good config.

Note: This only ignores the config, it does not (yet) handle
options that are different between the two configs. If the good
config has "SLAB" set and the bad config has "SLUB" it will not
find the bug if the bug had to do with changing these two options.

This is something that I intend to implement in the future.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |    6 ++++++
 tools/testing/ktest/sample.conf |   19 ++++++++++++-------
 2 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index a8e1826..dbc02de 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -88,6 +88,7 @@ my $bisect_bad = "";
 my $reverse_bisect;
 my $bisect_manual;
 my $bisect_skip;
+my $config_bisect_good;
 my $in_patchcheck = 0;
 my $run_test;
 my $redirect;
@@ -1745,6 +1746,10 @@ sub config_bisect {
 
     my $tmpconfig = "$tmpdir/use_config";
 
+    if (defined($config_bisect_good)) {
+	process_config_ignore $config_bisect_good;
+    }
+
     # Make the file with the bad config and the min config
     if (defined($minconfig)) {
 	# read the min config for things to ignore
@@ -2174,6 +2179,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     $patchcheck_sleep_time = set_test_option("PATCHCHECK_SLEEP_TIME", $i);
     $bisect_manual = set_test_option("BISECT_MANUAL", $i);
     $bisect_skip = set_test_option("BISECT_SKIP", $i);
+    $config_bisect_good = set_test_option("CONFIG_BISECT_GOOD", $i);
     $store_failures = set_test_option("STORE_FAILURES", $i);
     $timeout = set_test_option("TIMEOUT", $i);
     $booted_timeout = set_test_option("BOOTED_TIMEOUT", $i);
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index c2c072e..be531c2 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -749,13 +749,18 @@
 #	boot	- bad builds but fails to boot
 #	test	- bad boots but fails a test
 #
-#   CONFIG_BISECT is the config that failed to boot
-#
-#   If BISECT_MANUAL is set, it will pause between iterations.
-#   This is useful to use just ktest.pl just for the config bisect.
-#   If you set it to build, it will run the bisect and you can
-#   control what happens in between iterations. It will ask you if
-#   the test succeeded or not and continue the config bisect.
+#  CONFIG_BISECT is the config that failed to boot
+#
+#  If BISECT_MANUAL is set, it will pause between iterations.
+#  This is useful to use just ktest.pl just for the config bisect.
+#  If you set it to build, it will run the bisect and you can
+#  control what happens in between iterations. It will ask you if
+#  the test succeeded or not and continue the config bisect.
+#
+# CONFIG_BISECT_GOOD (optional)
+#  If you have a good config to start with, then you
+#  can specify it with CONFIG_BISECT_GOOD. Otherwise
+#  the MIN_CONFIG is the base.
 #
 # Example:
 #   TEST_START
-- 
1.7.5.4

[PATCH 04/19] ktest: Add TEST_NAME option

[Steven Rostedt]

[-- Attachment #1: 0004-ktest-Add-TEST_NAME-option.patch --]
[-- Type: text/plain, Size: 2882 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Searching through several tests, it gets confusing which test result
is for which test. By adding the TEST_NAME option, the user can tell
which test result belongs to which test.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   18 ++++++++++++++++--
 tools/testing/ktest/sample.conf |    6 ++++++
 2 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index dbc02de..579569f 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -101,6 +101,7 @@ my $sleep_time;
 my $bisect_sleep_time;
 my $patchcheck_sleep_time;
 my $store_failures;
+my $test_name;
 my $timeout;
 my $booted_timeout;
 my $detect_triplefault;
@@ -620,9 +621,15 @@ sub fail {
 	    end_monitor;
 	}
 
+	my $name = "";
+
+	if (defined($test_name)) {
+	    $name = " ($test_name)";
+	}
+
 	doprint "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n";
 	doprint "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n";
-	doprint "KTEST RESULT: TEST $i Failed: ", @_, "\n";
+	doprint "KTEST RESULT: TEST $i$name Failed: ", @_, "\n";
 	doprint "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n";
 	doprint "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n";
 
@@ -1130,9 +1137,15 @@ sub success {
 
     $successes++;
 
+    my $name = "";
+
+    if (defined($test_name)) {
+	$name = " ($test_name)";
+    }
+
     doprint "\n\n*******************************************\n";
     doprint     "*******************************************\n";
-    doprint     "KTEST RESULT: TEST $i SUCCESS!!!!         **\n";
+    doprint     "KTEST RESULT: TEST $i$name SUCCESS!!!!         **\n";
     doprint     "*******************************************\n";
     doprint     "*******************************************\n";
 
@@ -2181,6 +2194,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     $bisect_skip = set_test_option("BISECT_SKIP", $i);
     $config_bisect_good = set_test_option("CONFIG_BISECT_GOOD", $i);
     $store_failures = set_test_option("STORE_FAILURES", $i);
+    $test_name = set_test_option("TEST_NAME", $i);
     $timeout = set_test_option("TIMEOUT", $i);
     $booted_timeout = set_test_option("BOOTED_TIMEOUT", $i);
     $console = set_test_option("CONSOLE", $i);
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index be531c2..0e5f764 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -545,6 +545,12 @@
 #  all preceding tests until a new CHECKOUT is set.
 #
 #
+# TEST_NAME = name
+#
+#  If you want the test to have a name that is displayed in
+#  the test result banner at the end of the test, then use this
+#  option. This is useful to search for the RESULT keyword and
+#  not have to translate a test number to a test in the config.
 #
 # For TEST_TYPE = patchcheck
 #
-- 
1.7.5.4

[PATCH 05/19] ktest: Implement our own force min config

[Steven Rostedt]

[-- Attachment #1: 0005-ktest-Implement-our-own-force-min-config.patch --]
[-- Type: text/plain, Size: 4056 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Using the build KCONFIG_ALLCONFIG environment variable to force
the min config may not always work properly. Since ktest is
written in perl, it is trivial to read and replace the current
config with the configs specified by the min config.

Now the min config (and add configs) are read by perl and before
a make is done, these configs in the .config file are replaced
by the version in the min config.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   74 ++++++++++++++++++++++++++++++++++-------
 1 files changed, 61 insertions(+), 13 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 579569f..aa442a9 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -119,6 +119,7 @@ my $successes = 0;
 my %entered_configs;
 my %config_help;
 my %variable;
+my %force_config;
 
 $config_help{"MACHINE"} = << "EOF"
  The machine hostname that you will test.
@@ -1044,21 +1045,69 @@ sub check_buildlog {
     return 1;
 }
 
+sub apply_min_config {
+    my $outconfig = "$output_config.new";
+
+    # Read the config file and remove anything that
+    # is in the force_config hash (from minconfig and others)
+    # then add the force config back.
+
+    doprint "Applying minimum configurations into $output_config.new\n";
+
+    open (OUT, ">$outconfig") or
+	dodie "Can't create $outconfig";
+
+    if (-f $output_config) {
+	open (IN, $output_config) or
+	    dodie "Failed to open $output_config";
+	while (<IN>) {
+	    if (/^(# )?(CONFIG_[^\s=]*)/) {
+		next if (defined($force_config{$2}));
+	    }
+	    print OUT;
+	}
+	close IN;
+    }
+    foreach my $config (keys %force_config) {
+	print OUT "$force_config{$config}\n";
+    }
+    close OUT;
+
+    run_command "mv $outconfig $output_config";
+}
+
 sub make_oldconfig {
-    my ($defconfig) = @_;
 
-    if (!run_command "$defconfig $make oldnoconfig") {
+    apply_min_config;
+
+    if (!run_command "$make oldnoconfig") {
 	# Perhaps oldnoconfig doesn't exist in this version of the kernel
 	# try a yes '' | oldconfig
 	doprint "oldnoconfig failed, trying yes '' | make oldconfig\n";
-	run_command "yes '' | $defconfig $make oldconfig" or
+	run_command "yes '' | $make oldconfig" or
 	    dodie "failed make config oldconfig";
     }
 }
 
+# read a config file and use this to force new configs.
+sub load_force_config {
+    my ($config) = @_;
+
+    open(IN, $config) or
+	dodie "failed to read $config";
+    while (<IN>) {
+	chomp;
+	if (/^(CONFIG[^\s=]*)(\s*=.*)/) {
+	    $force_config{$1} = $_;
+	} elsif (/^# (CONFIG_\S*) is not set/) {
+	    $force_config{$1} = $_;
+	}
+    }
+    close IN;
+}
+
 sub build {
     my ($type) = @_;
-    my $defconfig = "";
 
     unlink $buildlog;
 
@@ -1098,15 +1147,15 @@ sub build {
     close(OUT);
 
     if (defined($minconfig)) {
-	$defconfig = "KCONFIG_ALLCONFIG=$minconfig";
+	load_force_config($minconfig);
     }
 
-    if ($type eq "oldnoconfig") {
-	make_oldconfig $defconfig;
-    } else {
-	run_command "$defconfig $make $type" or
+    if ($type ne "oldnoconfig") {
+	run_command "$make $type" or
 	    dodie "failed make config";
     }
+    # Run old config regardless, to enforce min configurations
+    make_oldconfig;
 
     $redirect = "$buildlog";
     if (!run_command "$make $build_options") {
@@ -1587,7 +1636,7 @@ sub create_config {
     close(OUT);
 
 #    exit;
-    make_oldconfig "";
+    make_oldconfig;
 }
 
 sub compare_configs {
@@ -1778,9 +1827,8 @@ sub config_bisect {
 	    dodie "failed to append $addconfig";
     }
 
-    my $defconfig = "";
     if (-f $tmpconfig) {
-	$defconfig = "KCONFIG_ALLCONFIG=$tmpconfig";
+	load_force_config($tmpconfig);
 	process_config_ignore $tmpconfig;
     }
 
@@ -1801,7 +1849,7 @@ sub config_bisect {
     close(IN);
 
     # Now run oldconfig with the minconfig (and addconfigs)
-    make_oldconfig $defconfig;
+    make_oldconfig;
 
     # check to see what we lost (or gained)
     open (IN, $output_config)
-- 
1.7.5.4

[PATCH 06/19] ktest: Have wait on stdio honor bug timeout

[Steven Rostedt]

[-- Attachment #1: 0006-ktest-Have-wait-on-stdio-honor-bug-timeout.patch --]
[-- Type: text/plain, Size: 1729 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

After a bug is found, the STOP_AFTER_FAILURE timeout is used to
determine how much output should be printed before breaking out
of the monitor loop. This is to get things like call traces and
enough infromation about the bug to help determine what caused it.

The STOP_AFTER_FAILURE is usually much shorter than the TIMEOUT
that is used to determine when to quit after no more stdio is given.

But since the stdio read uses a wait on I/O, the STOP_AFTER_FAILURE is
only checked after we get something from I/O. But if the I/O does
not return any more data, we wait the TIMEOUT period instead, even
though we already triggered a bug report.

The wait on I/O should honor the STOP_AFTER_FAILURE time if a bug has
been found.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   11 ++++++++++-
 1 files changed, 10 insertions(+), 1 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index aa442a9..1e1fe83 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -851,7 +851,16 @@ sub monitor {
 
     while (!$done) {
 
-	if ($booted) {
+	if ($bug && defined($stop_after_failure) &&
+	    $stop_after_failure >= 0) {
+	    my $time = $stop_after_failure - (time - $failure_start);
+	    $line = wait_for_input($monitor_fp, $time);
+	    if (!defined($line)) {
+		doprint "bug timed out after $booted_timeout seconds\n";
+		doprint "Test forced to stop after $stop_after_failure seconds after failure\n";
+		last;
+	    }
+	} elsif ($booted) {
 	    $line = wait_for_input($monitor_fp, $booted_timeout);
 	    if (!defined($line)) {
 		my $s = $booted_timeout == 1 ? "" : "s";
-- 
1.7.5.4

[PATCH 07/19] ktest: Have LOG_FILE evaluate options as well

[Steven Rostedt]

[-- Attachment #1: 0007-ktest-Have-LOG_FILE-evaluate-options-as-well.patch --]
[-- Type: text/plain, Size: 3859 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

The LOG_FILE variable needs to evaluate the $ options as well.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |  126 ++++++++++++++++++++++-------------------
 1 files changed, 68 insertions(+), 58 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 1e1fe83..83dcfaf 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -478,6 +478,69 @@ sub read_config {
     }
 }
 
+sub __eval_option {
+    my ($option, $i) = @_;
+
+    # Add space to evaluate the character before $
+    $option = " $option";
+    my $retval = "";
+
+    while ($option =~ /(.*?[^\\])\$\{(.*?)\}(.*)/) {
+	my $start = $1;
+	my $var = $2;
+	my $end = $3;
+
+	# Append beginning of line
+	$retval = "$retval$start";
+
+	# If the iteration option OPT[$i] exists, then use that.
+	# otherwise see if the default OPT (without [$i]) exists.
+
+	my $o = "$var\[$i\]";
+
+	if (defined($opt{$o})) {
+	    $o = $opt{$o};
+	    $retval = "$retval$o";
+	} elsif (defined($opt{$var})) {
+	    $o = $opt{$var};
+	    $retval = "$retval$o";
+	} else {
+	    $retval = "$retval\$\{$var\}";
+	}
+
+	$option = $end;
+    }
+
+    $retval = "$retval$option";
+
+    $retval =~ s/^ //;
+
+    return $retval;
+}
+
+sub eval_option {
+    my ($option, $i) = @_;
+
+    my $prev = "";
+
+    # Since an option can evaluate to another option,
+    # keep iterating until we do not evaluate any more
+    # options.
+    my $r = 0;
+    while ($prev ne $option) {
+	# Check for recursive evaluations.
+	# 100 deep should be more than enough.
+	if ($r++ > 100) {
+	    die "Over 100 evaluations accurred with $option\n" .
+		"Check for recursive variables\n";
+	}
+	$prev = $option;
+	$option = __eval_option($option, $i);
+    }
+
+    return $option;
+}
+
 sub _logit {
     if (defined($opt{"LOG_FILE"})) {
 	open(OUT, ">> $opt{LOG_FILE}") or die "Can't write to $opt{LOG_FILE}";
@@ -2079,6 +2142,10 @@ EOF
 }
 read_config $ktest_config;
 
+if (defined($opt{"LOG_FILE"})) {
+    $opt{"LOG_FILE"} = eval_option($opt{"LOG_FILE"}, -1);
+}
+
 # Append any configs entered in manually to the config file.
 my @new_configs = keys %entered_configs;
 if ($#new_configs >= 0) {
@@ -2147,70 +2214,13 @@ sub __set_test_option {
     return undef;
 }
 
-sub eval_option {
-    my ($option, $i) = @_;
-
-    # Add space to evaluate the character before $
-    $option = " $option";
-    my $retval = "";
-
-    while ($option =~ /(.*?[^\\])\$\{(.*?)\}(.*)/) {
-	my $start = $1;
-	my $var = $2;
-	my $end = $3;
-
-	# Append beginning of line
-	$retval = "$retval$start";
-
-	# If the iteration option OPT[$i] exists, then use that.
-	# otherwise see if the default OPT (without [$i]) exists.
-
-	my $o = "$var\[$i\]";
-
-	if (defined($opt{$o})) {
-	    $o = $opt{$o};
-	    $retval = "$retval$o";
-	} elsif (defined($opt{$var})) {
-	    $o = $opt{$var};
-	    $retval = "$retval$o";
-	} else {
-	    $retval = "$retval\$\{$var\}";
-	}
-
-	$option = $end;
-    }
-
-    $retval = "$retval$option";
-
-    $retval =~ s/^ //;
-
-    return $retval;
-}
-
 sub set_test_option {
     my ($name, $i) = @_;
 
     my $option = __set_test_option($name, $i);
     return $option if (!defined($option));
 
-    my $prev = "";
-
-    # Since an option can evaluate to another option,
-    # keep iterating until we do not evaluate any more
-    # options.
-    my $r = 0;
-    while ($prev ne $option) {
-	# Check for recursive evaluations.
-	# 100 deep should be more than enough.
-	if ($r++ > 100) {
-	    die "Over 100 evaluations accurred with $name\n" .
-		"Check for recursive variables\n";
-	}
-	$prev = $option;
-	$option = eval_option($option, $i);
-    }
-
-    return $option;
+    return eval_option($option, $i);
 }
 
 # First we need to do is the builds
-- 
1.7.5.4

[PATCH 08/19] ktest: Allow initrd processing without modules defined

[Steven Rostedt]

[-- Attachment #1: 0008-ktest-Allow-initrd-processing-without-modules-define.patch --]
[-- Type: text/plain, Size: 1546 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

When a config is set with CONFIG_MODULES=n, it does not mean that the
kernel does not need an initrd to boot. For systems that depend on LVM
and such, an initrd must run first.

If POST_INSTALL is defined, then run the post install regardless if
modules are needed or not.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   18 ++++++++++++------
 1 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 83dcfaf..fb46e12 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -1031,6 +1031,16 @@ sub monitor {
     return 1;
 }
 
+sub do_post_install {
+
+    return if (!defined($post_install));
+
+    my $cp_post_install = $post_install;
+    $cp_post_install =~ s/\$KERNEL_VERSION/$version/g;
+    run_command "$cp_post_install" or
+	dodie "Failed to run post install";
+}
+
 sub install {
 
     run_scp "$outputdir/$build_target", "$target_image" or
@@ -1050,6 +1060,7 @@ sub install {
     close(IN);
 
     if (!$install_mods) {
+	do_post_install;
 	doprint "No modules needed\n";
 	return;
     }
@@ -1077,12 +1088,7 @@ sub install {
 
     run_ssh "rm -f /tmp/$modtar";
 
-    return if (!defined($post_install));
-
-    my $cp_post_install = $post_install;
-    $cp_post_install =~ s/\$KERNEL_VERSION/$version/g;
-    run_command "$cp_post_install" or
-	dodie "Failed to run post install";
+    do_post_install;
 }
 
 sub check_buildlog {
-- 
1.7.5.4

[PATCH 09/19] ktest: Add POST/PRE_BUILD options

[Steven Rostedt]

[-- Attachment #1: 0009-ktest-Add-POST-PRE_BUILD-options.patch --]
[-- Type: text/plain, Size: 4052 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

There are some cases that a patch may be needed to apply to the kernel
in patchcheck or bisect tests. Adding a PRE_BUILD option to apply the
patch and POST_BUILD to remove it, allows for this to be done easily.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   31 ++++++++++++++++++++++++++++---
 tools/testing/ktest/sample.conf |   32 ++++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+), 3 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index fb46e12..d0e1de6 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -63,6 +63,10 @@ my $output_config;
 my $test_type;
 my $build_type;
 my $build_options;
+my $pre_build;
+my $post_build;
+my $pre_build_die;
+my $post_build_die;
 my $reboot_type;
 my $reboot_script;
 my $power_cycle;
@@ -1189,6 +1193,14 @@ sub build {
 
     unlink $buildlog;
 
+    if (defined($pre_build)) {
+	my $ret = run_command $pre_build;
+	if (!$ret && defined($pre_build_die) &&
+	    $pre_build_die) {
+	    dodie "failed to pre_build\n";
+	}
+    }
+
     if ($type =~ /^useconfig:(.*)/) {
 	run_command "cp $1 $output_config" or
 	    dodie "could not copy $1 to .config";
@@ -1236,13 +1248,22 @@ sub build {
     make_oldconfig;
 
     $redirect = "$buildlog";
-    if (!run_command "$make $build_options") {
-	undef $redirect;
+    my $build_ret = run_command "$make $build_options";
+    undef $redirect;
+
+    if (defined($post_build)) {
+	my $ret = run_command $post_build;
+	if (!$ret && defined($post_build_die) &&
+	    $post_build_die) {
+	    dodie "failed to post_build\n";
+	}
+    }
+
+    if (!$build_ret) {
 	# bisect may need this to pass
 	return 0 if ($in_bisect);
 	fail "failed build" and return 0;
     }
-    undef $redirect;
 
     return 1;
 }
@@ -2244,6 +2265,10 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     $test_type = set_test_option("TEST_TYPE", $i);
     $build_type = set_test_option("BUILD_TYPE", $i);
     $build_options = set_test_option("BUILD_OPTIONS", $i);
+    $pre_build = set_test_option("PRE_BUILD", $i);
+    $post_build = set_test_option("POST_BUILD", $i);
+    $pre_build_die = set_test_option("PRE_BUILD_DIE", $i);
+    $post_build_die = set_test_option("POST_BUILD_DIE", $i);
     $power_cycle = set_test_option("POWER_CYCLE", $i);
     $reboot = set_test_option("REBOOT", $i);
     $noclean = set_test_option("BUILD_NOCLEAN", $i);
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index 0e5f764..1092e47 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -293,6 +293,38 @@
 # or on some systems:
 #POST_INSTALL = ssh user@target /sbin/dracut -f /boot/initramfs-test.img $KERNEL_VERSION
 
+# If there is a script that you require to run before the build is done
+# you can specify it with PRE_BUILD.
+#
+# One example may be if you must add a temporary patch to the build to
+# fix a unrelated bug to perform a patchcheck test. This will apply the
+# patch before each build that is made. Use the POST_BUILD to do a git reset --hard
+# to remove the patch.
+#
+# (default undef)
+#PRE_BUILD = cd ${BUILD_DIR} && patch -p1 < /tmp/temp.patch
+
+# To specify if the test should fail if the PRE_BUILD fails,
+# PRE_BUILD_DIE needs to be set to 1. Otherwise the PRE_BUILD
+# result is ignored.
+# (default 0)
+# PRE_BUILD_DIE = 1
+
+# If there is a script that should run after the build is done
+# you can specify it with POST_BUILD.
+#
+# As the example in PRE_BUILD, POST_BUILD can be used to reset modifications
+# made by the PRE_BUILD.
+#
+# (default undef)
+#POST_BUILD = cd ${BUILD_DIR} && git reset --hard
+
+# To specify if the test should fail if the POST_BUILD fails,
+# POST_BUILD_DIE needs to be set to 1. Otherwise the POST_BUILD
+# result is ignored.
+# (default 0)
+#POST_BUILD_DIE = 1
+
 # Way to reboot the box to the test kernel.
 # Only valid options so far are "grub" and "script"
 # (default grub)
-- 
1.7.5.4

[PATCH 10/19] ktest: Have the testing tmp dir include machine name

[Steven Rostedt]

[-- Attachment #1: 0010-ktest-Have-the-testing-tmp-dir-include-machine-name.patch --]
[-- Type: text/plain, Size: 1513 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

As multiple tests may be executed by the same server, have the test
machine name add uniqueness to the value of the temp directory.
Otherwise the temp directories may overwrite each other's tests.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |    2 +-
 tools/testing/ktest/sample.conf |    4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index d0e1de6..24286ce 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -27,7 +27,7 @@ $default{"TEST_TYPE"}		= "test";
 $default{"BUILD_TYPE"}		= "randconfig";
 $default{"MAKE_CMD"}		= "make";
 $default{"TIMEOUT"}		= 120;
-$default{"TMP_DIR"}		= "/tmp/ktest";
+$default{"TMP_DIR"}		= "/tmp/ktest/\${MACHINE}";
 $default{"SLEEP_TIME"}		= 60;	# sleep time between tests
 $default{"BUILD_NOCLEAN"}	= 0;
 $default{"REBOOT_ON_ERROR"}	= 0;
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index 1092e47..e2d8d83 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -392,8 +392,8 @@
 #ADD_CONFIG = /home/test/config-broken
 
 # The location on the host where to write temp files
-# (default /tmp/ktest)
-#TMP_DIR = /tmp/ktest
+# (default /tmp/ktest/${MACHINE})
+#TMP_DIR = /tmp/ktest/${MACHINE}
 
 # Optional log file to write the status (recommended)
 #  Note, this is a DEFAULT section only option.
-- 
1.7.5.4

[PATCH 11/19] ktest: Fix tar extracting of modules to target

[Steven Rostedt]

[-- Attachment #1: 0011-ktest-Fix-tar-extracting-of-modules-to-target.patch --]
[-- Type: text/plain, Size: 805 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

The tar command to create the module directory is cjf, but the
extraction only had xf. This works on most versions of tar, but some
versions of tar require xjf for extraction as well.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 24286ce..5b35fa0 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -1087,7 +1087,7 @@ sub install {
 
     unlink "$tmpdir/$modtar";
 
-    run_ssh "'(cd / && tar xf /tmp/$modtar)'" or
+    run_ssh "'(cd / && tar xjf /tmp/$modtar)'" or
 	dodie "failed to tar modules";
 
     run_ssh "rm -f /tmp/$modtar";
-- 
1.7.5.4

[PATCH 12/19] ktest: Add IGNORE_WARNINGS to ignore warnings in some patches

[Steven Rostedt]

[-- Attachment #1: 0012-ktest-Add-IGNORE_WARNINGS-to-ignore-warnings-in-some.patch --]
[-- Type: text/plain, Size: 3079 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Doing a patchcheck test, there may be warnings that gcc produces which
may be OK, and the test should not fail on that commit. By adding a
IGNORE_WARNINGS option to list a space delimited SHA1s that are ignored
lets the user avoid having the test fail on certain commits.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   14 +++++++++++++-
 tools/testing/ktest/sample.conf |    8 +++++++-
 2 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 5b35fa0..5924f14 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -104,6 +104,7 @@ my $monitor_cnt = 0;
 my $sleep_time;
 my $bisect_sleep_time;
 my $patchcheck_sleep_time;
+my $ignore_warnings;
 my $store_failures;
 my $test_name;
 my $timeout;
@@ -2074,6 +2075,13 @@ sub patchcheck {
     @list = reverse @list;
 
     my $save_clean = $noclean;
+    my %ignored_warnings;
+
+    if (defined($ignore_warnings)) {
+	foreach my $sha1 (split /\s+/, $ignore_warnings) {
+	    $ignored_warnings{$sha1} = 1;
+	}
+    }
 
     $in_patchcheck = 1;
     foreach my $item (@list) {
@@ -2100,7 +2108,10 @@ sub patchcheck {
 	    build "oldconfig" or return 0;
 	}
 
-	check_buildlog $sha1 or return 0;
+
+	if (!defined($ignored_warnings{$sha1})) {
+	    check_buildlog $sha1 or return 0;
+	}
 
 	next if ($type eq "build");
 
@@ -2288,6 +2299,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     $sleep_time = set_test_option("SLEEP_TIME", $i);
     $bisect_sleep_time = set_test_option("BISECT_SLEEP_TIME", $i);
     $patchcheck_sleep_time = set_test_option("PATCHCHECK_SLEEP_TIME", $i);
+    $ignore_warnings = set_test_option("IGNORE_WARNINGS", $i);
     $bisect_manual = set_test_option("BISECT_MANUAL", $i);
     $bisect_skip = set_test_option("BISECT_SKIP", $i);
     $config_bisect_good = set_test_option("CONFIG_BISECT_GOOD", $i);
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index e2d8d83..82c966c 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -604,7 +604,12 @@
 #      build, boot, test.
 #
 #   Note, the build test will look for warnings, if a warning occurred
-#     in a file that a commit touches, the build will fail.
+#     in a file that a commit touches, the build will fail, unless
+#     IGNORE_WARNINGS is set for the given commit's sha1
+#
+#   IGNORE_WARNINGS can be used to disable the failure of patchcheck
+#     on a particuler commit (SHA1). You can add more than one commit
+#     by adding a list of SHA1s that are space delimited.
 #
 #   If BUILD_NOCLEAN is set, then make mrproper will not be run on
 #   any of the builds, just like all other TEST_TYPE tests. But
@@ -619,6 +624,7 @@
 #   PATCHCHECK_TYPE = boot
 #   PATCHCHECK_START = 747e94ae3d1b4c9bf5380e569f614eb9040b79e7
 #   PATCHCHECK_END = HEAD~2
+#   IGNORE_WARNINGS = 42f9c6b69b54946ffc0515f57d01dc7f5c0e4712 0c17ca2c7187f431d8ffc79e81addc730f33d128
 #
 #
 #
-- 
1.7.5.4

[PATCH 13/19] ktest: Add helper function to avoid duplicate code

[Steven Rostedt]

[-- Attachment #1: 0013-ktest-Add-helper-function-to-avoid-duplicate-code.patch --]
[-- Type: text/plain, Size: 2350 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Several places had the following code:

    get_grub_index;
    get_version;
    install;

    start_monitor;
    return monitor;

Creating a function "start_monitor_and_boot()" replaces these mulitple
uses with a single call.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   46 ++++++++++++++++++-----------------------
 1 files changed, 20 insertions(+), 26 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 5924f14..099ceee 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -1096,6 +1096,23 @@ sub install {
     do_post_install;
 }
 
+sub get_version {
+    # get the release name
+    doprint "$make kernelrelease ... ";
+    $version = `$make kernelrelease | tail -1`;
+    chomp($version);
+    doprint "$version\n";
+}
+
+sub start_monitor_and_boot {
+    get_grub_index;
+    get_version;
+    install;
+
+    start_monitor;
+    return monitor;
+}
+
 sub check_buildlog {
     my ($patch) = @_;
 
@@ -1307,14 +1324,6 @@ sub success {
     }
 }
 
-sub get_version {
-    # get the release name
-    doprint "$make kernelrelease ... ";
-    $version = `$make kernelrelease | tail -1`;
-    chomp($version);
-    doprint "$version\n";
-}
-
 sub answer_bisect {
     for (;;) {
 	doprint "Pass or fail? [p/f]";
@@ -1479,12 +1488,7 @@ sub run_bisect_test {
 	dodie "Failed on build" if $failed;
 
 	# Now boot the box
-	get_grub_index;
-	get_version;
-	install;
-
-	start_monitor;
-	monitor or $failed = 1;
+	start_monitor_and_boot or $failed = 1;
 
 	if ($type ne "boot") {
 	    if ($failed && $bisect_skip) {
@@ -2115,14 +2119,9 @@ sub patchcheck {
 
 	next if ($type eq "build");
 
-	get_grub_index;
-	get_version;
-	install;
-
 	my $failed = 0;
 
-	start_monitor;
-	monitor or $failed = 1;
+	start_monitor_and_boot or $failed = 1;
 
 	if (!$failed && $type ne "boot"){
 	    do_run_test or $failed = 1;
@@ -2393,13 +2392,8 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     }
 
     if ($test_type ne "build") {
-	get_grub_index;
-	get_version;
-	install;
-
 	my $failed = 0;
-	start_monitor;
-	monitor or $failed = 1;;
+	start_monitor_and_boot or $failed = 1;
 
 	if (!$failed && $test_type ne "boot" && defined($run_test)) {
 	    do_run_test or $failed = 1;
-- 
1.7.5.4

[PATCH 14/19] ktest: Require one TEST_START in config file

[Steven Rostedt]

[-- Attachment #1: 0014-ktest-Require-one-TEST_START-in-config-file.patch --]
[-- Type: text/plain, Size: 1495 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

There has been too many times that I put in one too many SKIP
TEST_STARTs and start the test with the default randconfig by accident
that I added this to have ktest ask the user for which test they want to
run if no TEST_START is specified.

Now if I accidently start the test with all TEST_STARTs skipped, ktest
asks what test do I want to run, and I now have a chance to kill it
before it does a make mrproper on my build directory.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 099ceee..6166f3a 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -345,6 +345,7 @@ sub read_config {
     my $num_tests_set = 0;
     my $skip = 0;
     my $rest;
+    my $test_case = 0;
 
     while (<IN>) {
 
@@ -370,6 +371,7 @@ sub read_config {
 		$rest = $1;
 		$skip = 1;
 	    } else {
+		$test_case = 1;
 		$skip = 0;
 	    }
 
@@ -474,6 +476,15 @@ sub read_config {
     # make sure we have all mandatory configs
     get_ktest_configs;
 
+    # was a test specified?
+    if (!$test_case) {
+	print "No test case specified.\n";
+	print "What test case would you like to run?\n";
+	my $ans = <STDIN>;
+	chomp $ans;
+	$default{"TEST_TYPE"} = $ans;
+    }
+
     # set any defaults
 
     foreach my $default (keys %default) {
-- 
1.7.5.4

[PATCH 15/19] ktest: Add test type make_min_config

[Steven Rostedt]

[-- Attachment #1: 0015-ktest-Add-test-type-make_min_config.patch --]
[-- Type: text/plain, Size: 12474 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

After doing a make localyesconfig, your kernel configuration may
not be the most useful minimum configuration. Having a true minimum
config that you can use against other configs is very useful if
someone else has a config that breaks on your code. By only forcing
those configurations that are truly required to boot your machine
will give you less of a chance that one of your set configurations
will make the bug go away. This will give you a better chance to
be able to reproduce the reported bug matching the broken config.

Note, this does take some time, and may require you to run the
test over night, or perhaps over the weekend. But it also allows
you to interrupt it, and gives you the current minimum config
that was found till that time.

Note, this test automatically assumes a BUILD_TYPE of oldconfig
and its test type acts like boot.

TODO: add a test version that makes the config do more than just
      boot, like having network access.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |  255 ++++++++++++++++++++++++++++++++++++++-
 tools/testing/ktest/sample.conf |   50 ++++++++
 2 files changed, 301 insertions(+), 4 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 6166f3a..5323c6f 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -86,6 +86,9 @@ my $make;
 my $post_install;
 my $noclean;
 my $minconfig;
+my $start_minconfig;
+my $output_minconfig;
+my $ignore_config;
 my $addconfig;
 my $in_bisect = 0;
 my $bisect_bad = "";
@@ -1189,7 +1192,11 @@ sub apply_min_config {
 
 sub make_oldconfig {
 
-    apply_min_config;
+    my @force_list = keys %force_config;
+
+    if ($#force_list >= 0) {
+	apply_min_config;
+    }
 
     if (!run_command "$make oldnoconfig") {
 	# Perhaps oldnoconfig doesn't exist in this version of the kernel
@@ -1678,21 +1685,27 @@ my %null_config;
 
 my %dependency;
 
-sub process_config_ignore {
-    my ($config) = @_;
+sub assign_configs {
+    my ($hash, $config) = @_;
 
     open (IN, $config)
 	or dodie "Failed to read $config";
 
     while (<IN>) {
 	if (/^((CONFIG\S*)=.*)/) {
-	    $config_ignore{$2} = $1;
+	    ${$hash}{$2} = $1;
 	}
     }
 
     close(IN);
 }
 
+sub process_config_ignore {
+    my ($config) = @_;
+
+    assign_configs \%config_ignore, $config;
+}
+
 sub read_current_config {
     my ($config_ref) = @_;
 
@@ -2149,6 +2162,226 @@ sub patchcheck {
     return 1;
 }
 
+sub read_config_list {
+    my ($config) = @_;
+
+    open (IN, $config)
+	or dodie "Failed to read $config";
+
+    while (<IN>) {
+	if (/^((CONFIG\S*)=.*)/) {
+	    if (!defined($config_ignore{$2})) {
+		$config_list{$2} = $1;
+	    }
+	}
+    }
+
+    close(IN);
+}
+
+sub read_output_config {
+    my ($config) = @_;
+
+    assign_configs \%config_ignore, $config;
+}
+
+sub make_new_config {
+    my @configs = @_;
+
+    open (OUT, ">$output_config")
+	or dodie "Failed to write $output_config";
+
+    foreach my $config (@configs) {
+	print OUT "$config\n";
+    }
+    close OUT;
+}
+
+sub make_min_config {
+    my ($i) = @_;
+
+    if (!defined($output_minconfig)) {
+	fail "OUTPUT_MIN_CONFIG not defined" and return;
+    }
+    if (!defined($start_minconfig)) {
+	fail "START_MIN_CONFIG or MIN_CONFIG not defined" and return;
+    }
+
+    # First things first. We build an allnoconfig to find
+    # out what the defaults are that we can't touch.
+    # Some are selections, but we really can't handle selections.
+
+    my $save_minconfig = $minconfig;
+    undef $minconfig;
+
+    run_command "$make allnoconfig" or return 0;
+
+    process_config_ignore $output_config;
+    my %keep_configs;
+
+    if (defined($ignore_config)) {
+	# make sure the file exists
+	`touch $ignore_config`;
+	assign_configs \%keep_configs, $ignore_config;
+    }
+
+    doprint "Load initial configs from $start_minconfig\n";
+
+    # Look at the current min configs, and save off all the
+    # ones that were set via the allnoconfig
+    my %min_configs;
+    assign_configs \%min_configs, $start_minconfig;
+
+    my @config_keys = keys %min_configs;
+
+    # Remove anything that was set by the make allnoconfig
+    # we shouldn't need them as they get set for us anyway.
+    foreach my $config (@config_keys) {
+	# Remove anything in the ignore_config
+	if (defined($keep_configs{$config})) {
+	    my $file = $ignore_config;
+	    $file =~ s,.*/(.*?)$,$1,;
+	    doprint "$config set by $file ... ignored\n";
+	    delete $min_configs{$config};
+	    next;
+	}
+	# But make sure the settings are the same. If a min config
+	# sets a selection, we do not want to get rid of it if
+	# it is not the same as what we have. Just move it into
+	# the keep configs.
+	if (defined($config_ignore{$config})) {
+	    if ($config_ignore{$config} ne $min_configs{$config}) {
+		doprint "$config is in allnoconfig as '$config_ignore{$config}'";
+		doprint " but it is '$min_configs{$config}' in minconfig .. keeping\n";
+		$keep_configs{$config} = $min_configs{$config};
+	    } else {
+		doprint "$config set by allnoconfig ... ignored\n";
+	    }
+	    delete $min_configs{$config};
+	}
+    }
+
+    my %nochange_config;
+
+    my $done = 0;
+
+    while (!$done) {
+
+	my $config;
+	my $found;
+
+	# Now disable each config one by one and do a make oldconfig
+	# till we find a config that changes our list.
+
+	# Put configs that did not modify the config at the end.
+	my @test_configs = keys %min_configs;
+	my $reset = 1;
+	for (my $i = 0; $i < $#test_configs; $i++) {
+	    if (!defined($nochange_config{$test_configs[0]})) {
+		$reset = 0;
+		last;
+	    }
+	    # This config didn't change the .config last time.
+	    # Place it at the end
+	    my $config = shift @test_configs;
+	    push @test_configs, $config;
+	}
+
+	# if every test config has failed to modify the .config file
+	# in the past, then reset and start over.
+	if ($reset) {
+	    undef %nochange_config;
+	}
+
+	foreach my $config (@test_configs) {
+
+	    # Remove this config from the list of configs
+	    # do a make oldnoconfig and then read the resulting
+	    # .config to make sure it is missing the config that
+	    # we had before
+	    my %configs = %min_configs;
+	    delete $configs{$config};
+	    make_new_config ((values %configs), (values %keep_configs));
+	    make_oldconfig;
+	    undef %configs;
+	    assign_configs \%configs, $output_config;
+
+	    if (!defined($configs{$config})) {
+		$found = $config;
+		last;
+	    }
+
+	    doprint "disabling config $config did not change .config\n";
+
+	    # oh well, try another config
+	    $nochange_config{$config} = 1;
+	}
+
+	if (!defined($found)) {
+	    doprint "No more configs found that we can disable\n";
+	    $done = 1;
+	    last;
+	}
+
+	$config = $found;
+
+	doprint "Test with $config disabled\n";
+
+	# set in_bisect to keep build and monitor from dieing
+	$in_bisect = 1;
+
+	my $failed = 0;
+	build "oldconfig";
+	start_monitor_and_boot or $failed = 1;
+	end_monitor;
+
+	$in_bisect = 0;
+
+	if ($failed) {
+	    doprint "$config is needed to boot the box... keeping\n";
+	    # this config is needed, add it to the ignore list.
+	    $keep_configs{$config} = $min_configs{$config};
+	    delete $min_configs{$config};
+	} else {
+	    # We booted without this config, remove it from the minconfigs.
+	    doprint "$config is not needed, disabling\n";
+
+	    delete $min_configs{$config};
+
+	    # Also disable anything that is not enabled in this config
+	    my %configs;
+	    assign_configs \%configs, $output_config;
+	    my @config_keys = keys %min_configs;
+	    foreach my $config (@config_keys) {
+		if (!defined($configs{$config})) {
+		    doprint "$config is not set, disabling\n";
+		    delete $min_configs{$config};
+		}
+	    }
+
+	    # Save off all the current mandidory configs
+	    open (OUT, ">$output_minconfig")
+		or die "Can't write to $output_minconfig";
+	    foreach my $config (keys %keep_configs) {
+		print OUT "$keep_configs{$config}\n";
+	    }
+	    foreach my $config (keys %min_configs) {
+		print OUT "$min_configs{$config}\n";
+	    }
+	    close OUT;
+	}
+
+	doprint "Reboot and wait $sleep_time seconds\n";
+	reboot;
+	start_monitor;
+	wait_for_monitor $sleep_time;
+	end_monitor;
+    }
+
+    success $i;
+    return 1;
+}
+
 $#ARGV < 1 or die "ktest.pl version: $VERSION\n   usage: ktest.pl config-file\n";
 
 if ($#ARGV == 0) {
@@ -2294,6 +2527,9 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     $reboot = set_test_option("REBOOT", $i);
     $noclean = set_test_option("BUILD_NOCLEAN", $i);
     $minconfig = set_test_option("MIN_CONFIG", $i);
+    $output_minconfig = set_test_option("OUTPUT_MIN_CONFIG", $i);
+    $start_minconfig = set_test_option("START_MIN_CONFIG", $i);
+    $ignore_config = set_test_option("IGNORE_CONFIG", $i);
     $run_test = set_test_option("TEST", $i);
     $addconfig = set_test_option("ADD_CONFIG", $i);
     $reboot_type = set_test_option("REBOOT_TYPE", $i);
@@ -2329,6 +2565,10 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     $target_image = set_test_option("TARGET_IMAGE", $i);
     $localversion = set_test_option("LOCALVERSION", $i);
 
+    if (!defined($start_minconfig)) {
+	$start_minconfig = $minconfig;
+    }
+
     chdir $builddir || die "can't change directory to $builddir";
 
     if (!-d $tmpdir) {
@@ -2361,6 +2601,10 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
 	$run_type = $opt{"CONFIG_BISECT_TYPE[$i]"};
     }
 
+    if ($test_type eq "make_min_config") {
+	$run_type = "";
+    }
+
     # mistake in config file?
     if (!defined($run_type)) {
 	$run_type = "ERROR";
@@ -2396,6 +2640,9 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     } elsif ($test_type eq "patchcheck") {
 	patchcheck $i;
 	next;
+    } elsif ($test_type eq "make_min_config") {
+	make_min_config $i;
+	next;
     }
 
     if ($build_type ne "nobuild") {
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index 82c966c..a83846d 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -814,3 +814,53 @@
 #   MIN_CONFIG = /home/test/config-min
 #   BISECT_MANUAL = 1
 #
+#
+#
+# For TEST_TYPE = make_min_config
+#
+#  After doing a make localyesconfig, your kernel configuration may
+#  not be the most useful minimum configuration. Having a true minimum
+#  config that you can use against other configs is very useful if
+#  someone else has a config that breaks on your code. By only forcing
+#  those configurations that are truly required to boot your machine
+#  will give you less of a chance that one of your set configurations
+#  will make the bug go away. This will give you a better chance to
+#  be able to reproduce the reported bug matching the broken config.
+#
+#  Note, this does take some time, and may require you to run the
+#  test over night, or perhaps over the weekend. But it also allows
+#  you to interrupt it, and gives you the current minimum config
+#  that was found till that time.
+#
+#  Note, this test automatically assumes a BUILD_TYPE of oldconfig
+#  and its test type acts like boot.
+#  TODO: add a test version that makes the config do more than just
+#   boot, like having network access.
+#
+#  OUTPUT_MIN_CONFIG is the path and filename of the file that will
+#   be created from the MIN_CONFIG. If you interrupt the test, set
+#   this file as your new min config, and use it to continue the test.
+#   This file does not need to exist on start of test.
+#   This file is not created until a config is found that can be removed.
+#   (required field)
+#
+#  START_MIN_CONFIG is the config to use to start the test with.
+#   you can set this as the same OUTPUT_MIN_CONFIG, but if you do
+#   the OUTPUT_MIN_CONFIG file must exist.
+#   (default MIN_CONFIG)
+#
+#  IGNORE_CONFIG is used to specify a config file that has configs that
+#   you already know must be set. Configs are written here that have
+#   been tested and proved to be required. It is best to define this
+#   file if you intend on interrupting the test and running it where
+#   it left off. New configs that it finds will be written to this file
+#   and will not be tested again in later runs.
+#   (optional)
+#
+# Example:
+#
+#  TEST_TYPE = make_min_config
+#  OUTPUT_MIN_CONFIG = /path/to/config-new-min
+#  START_MIN_CONFIG = /path/to/config-min
+#  IGNORE_CONFIG = /path/to/config-tested
+#
-- 
1.7.5.4

[PATCH 16/19] ktest: Use Kconfig dependencies to shorten time to make min_config

[Steven Rostedt]

[-- Attachment #1: 0016-ktest-Use-Kconfig-dependencies-to-shorten-time-to-ma.patch --]
[-- Type: text/plain, Size: 10259 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

To save time, the test does not just grab any option and test
it. The Kconfig files are examined to determine the dependencies
of the configs. If a config is chosen that depends on another
config, that config will be checked first. By checking the
parents first, we can eliminate whole groups of configs that
may have been enabled.

For example, if a USB device config is chosen and depends on
CONFIG_USB, the CONFIG_USB will be tested before the device.
If CONFIG_USB is found not to be needed, it, as well as all
configs that depend on it, will be disabled and removed from
the current min_config.

Note, the code from streamline_config (make localmodconfig)
was copied and used to find the dependencies in the Kconfig file.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |  283 ++++++++++++++++++++++++++++++++++++---
 tools/testing/ktest/sample.conf |   12 ++
 2 files changed, 273 insertions(+), 22 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 5323c6f..a9f2e10 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -2162,6 +2162,159 @@ sub patchcheck {
     return 1;
 }
 
+my %depends;
+my $iflevel = 0;
+my @ifdeps;
+
+# prevent recursion
+my %read_kconfigs;
+
+# taken from streamline_config.pl
+sub read_kconfig {
+    my ($kconfig) = @_;
+
+    my $state = "NONE";
+    my $config;
+    my @kconfigs;
+
+    my $cont = 0;
+    my $line;
+
+
+    if (! -f $kconfig) {
+	doprint "file $kconfig does not exist, skipping\n";
+	return;
+    }
+
+    open(KIN, "$kconfig")
+	or die "Can't open $kconfig";
+    while (<KIN>) {
+	chomp;
+
+	# Make sure that lines ending with \ continue
+	if ($cont) {
+	    $_ = $line . " " . $_;
+	}
+
+	if (s/\\$//) {
+	    $cont = 1;
+	    $line = $_;
+	    next;
+	}
+
+	$cont = 0;
+
+	# collect any Kconfig sources
+	if (/^source\s*"(.*)"/) {
+	    $kconfigs[$#kconfigs+1] = $1;
+	}
+
+	# configs found
+	if (/^\s*(menu)?config\s+(\S+)\s*$/) {
+	    $state = "NEW";
+	    $config = $2;
+
+	    for (my $i = 0; $i < $iflevel; $i++) {
+		if ($i) {
+		    $depends{$config} .= " " . $ifdeps[$i];
+		} else {
+		    $depends{$config} = $ifdeps[$i];
+		}
+		$state = "DEP";
+	    }
+
+	# collect the depends for the config
+	} elsif ($state eq "NEW" && /^\s*depends\s+on\s+(.*)$/) {
+
+	    if (defined($depends{$1})) {
+		$depends{$config} .= " " . $1;
+	    } else {
+		$depends{$config} = $1;
+	    }
+
+	# Get the configs that select this config
+	} elsif ($state ne "NONE" && /^\s*select\s+(\S+)/) {
+	    if (defined($depends{$1})) {
+		$depends{$1} .= " " . $config;
+	    } else {
+		$depends{$1} = $config;
+	    }
+
+	# Check for if statements
+	} elsif (/^if\s+(.*\S)\s*$/) {
+	    my $deps = $1;
+	    # remove beginning and ending non text
+	    $deps =~ s/^[^a-zA-Z0-9_]*//;
+	    $deps =~ s/[^a-zA-Z0-9_]*$//;
+
+	    my @deps = split /[^a-zA-Z0-9_]+/, $deps;
+
+	    $ifdeps[$iflevel++] = join ':', @deps;
+
+	} elsif (/^endif/) {
+
+	    $iflevel-- if ($iflevel);
+
+	# stop on "help"
+	} elsif (/^\s*help\s*$/) {
+	    $state = "NONE";
+	}
+    }
+    close(KIN);
+
+    # read in any configs that were found.
+    foreach $kconfig (@kconfigs) {
+	if (!defined($read_kconfigs{$kconfig})) {
+	    $read_kconfigs{$kconfig} = 1;
+	    read_kconfig("$builddir/$kconfig");
+	}
+    }
+}
+
+sub read_depends {
+    # find out which arch this is by the kconfig file
+    open (IN, $output_config)
+	or dodie "Failed to read $output_config";
+    my $arch;
+    while (<IN>) {
+	if (m,Linux/(\S+)\s+\S+\s+Kernel Configuration,) {
+	    $arch = $1;
+	    last;
+	}
+    }
+    close IN;
+
+    if (!defined($arch)) {
+	doprint "Could not find arch from config file\n";
+	doprint "no dependencies used\n";
+	return;
+    }
+
+    # arch is really the subarch, we need to know
+    # what directory to look at.
+    if ($arch eq "i386" || $arch eq "x86_64") {
+	$arch = "x86";
+    } elsif ($arch =~ /^tile/) {
+	$arch = "tile";
+    }
+
+    my $kconfig = "$builddir/arch/$arch/Kconfig";
+
+    if (! -f $kconfig && $arch =~ /\d$/) {
+	my $orig = $arch;
+ 	# some subarchs have numbers, truncate them
+	$arch =~ s/\d*$//;
+	$kconfig = "$builddir/arch/$arch/Kconfig";
+	if (! -f $kconfig) {
+	    doprint "No idea what arch dir $orig is for\n";
+	    doprint "no dependencies used\n";
+	    return;
+	}
+    }
+
+    read_kconfig($kconfig);
+}
+
 sub read_config_list {
     my ($config) = @_;
 
@@ -2197,6 +2350,95 @@ sub make_new_config {
     close OUT;
 }
 
+sub get_depends {
+    my ($dep) = @_;
+
+    my $kconfig = $dep;
+    $kconfig =~ s/CONFIG_//;
+
+    $dep = $depends{"$kconfig"};
+
+    # the dep string we have saves the dependencies as they
+    # were found, including expressions like ! && ||. We
+    # want to split this out into just an array of configs.
+
+    my $valid = "A-Za-z_0-9";
+
+    my @configs;
+
+    while ($dep =~ /[$valid]/) {
+
+	if ($dep =~ /^[^$valid]*([$valid]+)/) {
+	    my $conf = "CONFIG_" . $1;
+
+	    $configs[$#configs + 1] = $conf;
+
+	    $dep =~ s/^[^$valid]*[$valid]+//;
+	} else {
+	    die "this should never happen";
+	}
+    }
+
+    return @configs;
+}
+
+my %min_configs;
+my %keep_configs;
+my %processed_configs;
+my %nochange_config;
+
+sub test_this_config {
+    my ($config) = @_;
+
+    my $found;
+
+    # if we already processed this config, skip it
+    if (defined($processed_configs{$config})) {
+	return undef;
+    }
+    $processed_configs{$config} = 1;
+
+    # if this config failed during this round, skip it
+    if (defined($nochange_config{$config})) {
+	return undef;
+    }
+
+    my $kconfig = $config;
+    $kconfig =~ s/CONFIG_//;
+
+    # Test dependencies first
+    if (defined($depends{"$kconfig"})) {
+	my @parents = get_depends $config;
+	foreach my $parent (@parents) {
+	    # if the parent is in the min config, check it first
+	    next if (!defined($min_configs{$parent}));
+	    $found = test_this_config($parent);
+	    if (defined($found)) {
+		return $found;
+	    }
+	}
+    }
+
+    # Remove this config from the list of configs
+    # do a make oldnoconfig and then read the resulting
+    # .config to make sure it is missing the config that
+    # we had before
+    my %configs = %min_configs;
+    delete $configs{$config};
+    make_new_config ((values %configs), (values %keep_configs));
+    make_oldconfig;
+    undef %configs;
+    assign_configs \%configs, $output_config;
+
+    return $config if (!defined($configs{$config}));
+
+    doprint "disabling config $config did not change .config\n";
+
+    $nochange_config{$config} = 1;
+
+    return undef;
+}
+
 sub make_min_config {
     my ($i) = @_;
 
@@ -2216,8 +2458,12 @@ sub make_min_config {
 
     run_command "$make allnoconfig" or return 0;
 
+    read_depends;
+
     process_config_ignore $output_config;
-    my %keep_configs;
+
+    undef %keep_configs;
+    undef %min_configs;
 
     if (defined($ignore_config)) {
 	# make sure the file exists
@@ -2229,7 +2475,6 @@ sub make_min_config {
 
     # Look at the current min configs, and save off all the
     # ones that were set via the allnoconfig
-    my %min_configs;
     assign_configs \%min_configs, $start_minconfig;
 
     my @config_keys = keys %min_configs;
@@ -2261,9 +2506,8 @@ sub make_min_config {
 	}
     }
 
-    my %nochange_config;
-
     my $done = 0;
+    my $take_two = 0;
 
     while (!$done) {
 
@@ -2293,35 +2537,30 @@ sub make_min_config {
 	    undef %nochange_config;
 	}
 
-	foreach my $config (@test_configs) {
+	undef %processed_configs;
 
-	    # Remove this config from the list of configs
-	    # do a make oldnoconfig and then read the resulting
-	    # .config to make sure it is missing the config that
-	    # we had before
-	    my %configs = %min_configs;
-	    delete $configs{$config};
-	    make_new_config ((values %configs), (values %keep_configs));
-	    make_oldconfig;
-	    undef %configs;
-	    assign_configs \%configs, $output_config;
+	foreach my $config (@test_configs) {
 
-	    if (!defined($configs{$config})) {
-		$found = $config;
-		last;
-	    }
+	    $found = test_this_config $config;
 
-	    doprint "disabling config $config did not change .config\n";
+	    last if (defined($found));
 
 	    # oh well, try another config
-	    $nochange_config{$config} = 1;
 	}
 
 	if (!defined($found)) {
+	    # we could have failed due to the nochange_config hash
+	    # reset and try again
+	    if (!$take_two) {
+		undef %nochange_config;
+		$take_two = 1;
+		next;
+	    }
 	    doprint "No more configs found that we can disable\n";
 	    $done = 1;
 	    last;
 	}
+	$take_two = 0;
 
 	$config = $found;
 
@@ -2338,7 +2577,7 @@ sub make_min_config {
 	$in_bisect = 0;
 
 	if ($failed) {
-	    doprint "$config is needed to boot the box... keeping\n";
+	    doprint "$min_configs{$config} is needed to boot the box... keeping\n";
 	    # this config is needed, add it to the ignore list.
 	    $keep_configs{$config} = $min_configs{$config};
 	    delete $min_configs{$config};
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index a83846d..d096a0c 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -837,6 +837,18 @@
 #  TODO: add a test version that makes the config do more than just
 #   boot, like having network access.
 #
+#  To save time, the test does not just grab any option and test
+#  it. The Kconfig files are examined to determine the dependencies
+#  of the configs. If a config is chosen that depends on another
+#  config, that config will be checked first. By checking the
+#  parents first, we can eliminate whole groups of configs that
+#  may have been enabled.
+#
+#  For example, if a USB device config is chosen and depends on CONFIG_USB,
+#  the CONFIG_USB will be tested before the device. If CONFIG_USB is
+#  found not to be needed, it, as well as all configs that depend on
+#  it, will be disabled and removed from the current min_config.
+#
 #  OUTPUT_MIN_CONFIG is the path and filename of the file that will
 #   be created from the MIN_CONFIG. If you interrupt the test, set
 #   this file as your new min config, and use it to continue the test.
-- 
1.7.5.4

[PATCH 17/19] ktest: Add prompt to use OUTPUT_MIN_CONFIG

[Steven Rostedt]

[-- Attachment #1: 0017-ktest-Add-prompt-to-use-OUTPUT_MIN_CONFIG.patch --]
[-- Type: text/plain, Size: 5314 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

If the defined OUTPUT_MIN_CONFIG in the make_min_config test exists,
then give a prompt to ask the user if they want to use that config
instead, as it is very often the case, especially when the test has been
interrupted. The OUTPUT_MIN_CONFIG is usually the config that one wants
to use to continue the test where they left off.

But if START_MIN_CONFIG is defined (thus the MIN_CONFIG is not the
default), then do not prompt, as it will be annoying if the user has
this as one of many tests, and the test pauses waiting for input, while
the user is sleeping.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    |   70 +++++++++++++++++++++++++++++++--------
 tools/testing/ktest/sample.conf |    3 ++
 2 files changed, 59 insertions(+), 14 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index a9f2e10..cf45f58 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -87,6 +87,7 @@ my $post_install;
 my $noclean;
 my $minconfig;
 my $start_minconfig;
+my $start_minconfig_defined;
 my $output_minconfig;
 my $ignore_config;
 my $addconfig;
@@ -217,6 +218,26 @@ $config_help{"REBOOT_SCRIPT"} = << "EOF"
 EOF
     ;
 
+sub read_yn {
+    my ($prompt) = @_;
+
+    my $ans;
+
+    for (;;) {
+	print "$prompt [Y/n] ";
+	$ans = <STDIN>;
+	chomp $ans;
+	if ($ans =~ /^\s*$/) {
+	    $ans = "y";
+	}
+	last if ($ans =~ /^y$/i || $ans =~ /^n$/i);
+	print "Please answer either 'y' or 'n'.\n";
+    }
+    if ($ans !~ /^y$/i) {
+	return 0;
+    }
+    return 1;
+}
 
 sub get_ktest_config {
     my ($config) = @_;
@@ -2445,10 +2466,23 @@ sub make_min_config {
     if (!defined($output_minconfig)) {
 	fail "OUTPUT_MIN_CONFIG not defined" and return;
     }
+
+    # If output_minconfig exists, and the start_minconfig
+    # came from min_config, than ask if we should use
+    # that instead.
+    if (-f $output_minconfig && !$start_minconfig_defined) {
+	print "$output_minconfig exists\n";
+	if (read_yn " Use it as minconfig?") {
+	    $start_minconfig = $output_minconfig;
+	}
+    }
+
     if (!defined($start_minconfig)) {
 	fail "START_MIN_CONFIG or MIN_CONFIG not defined" and return;
     }
 
+    my $temp_config = "$tmpdir/temp_config";
+
     # First things first. We build an allnoconfig to find
     # out what the defaults are that we can't touch.
     # Some are selections, but we really can't handle selections.
@@ -2581,6 +2615,19 @@ sub make_min_config {
 	    # this config is needed, add it to the ignore list.
 	    $keep_configs{$config} = $min_configs{$config};
 	    delete $min_configs{$config};
+
+	    # update new ignore configs
+	    if (defined($ignore_config)) {
+		open (OUT, ">$temp_config")
+		    or die "Can't write to $temp_config";
+		foreach my $config (keys %keep_configs) {
+		    print OUT "$keep_configs{$config}\n";
+		}
+		close OUT;
+		run_command "mv $temp_config $ignore_config" or
+		    dodie "failed to copy update to $ignore_config";
+	    }
+
 	} else {
 	    # We booted without this config, remove it from the minconfigs.
 	    doprint "$config is not needed, disabling\n";
@@ -2599,8 +2646,8 @@ sub make_min_config {
 	    }
 
 	    # Save off all the current mandidory configs
-	    open (OUT, ">$output_minconfig")
-		or die "Can't write to $output_minconfig";
+	    open (OUT, ">$temp_config")
+		or die "Can't write to $temp_config";
 	    foreach my $config (keys %keep_configs) {
 		print OUT "$keep_configs{$config}\n";
 	    }
@@ -2608,6 +2655,9 @@ sub make_min_config {
 		print OUT "$min_configs{$config}\n";
 	    }
 	    close OUT;
+
+	    run_command "mv $temp_config $output_minconfig" or
+		dodie "failed to copy update to $output_minconfig";
 	}
 
 	doprint "Reboot and wait $sleep_time seconds\n";
@@ -2627,18 +2677,7 @@ if ($#ARGV == 0) {
     $ktest_config = $ARGV[0];
     if (! -f $ktest_config) {
 	print "$ktest_config does not exist.\n";
-	my $ans;
-        for (;;) {
-	    print "Create it? [Y/n] ";
-	    $ans = <STDIN>;
-	    chomp $ans;
-	    if ($ans =~ /^\s*$/) {
-		$ans = "y";
-	    }
-	    last if ($ans =~ /^y$/i || $ans =~ /^n$/i);
-	    print "Please answer either 'y' or 'n'.\n";
-	}
-	if ($ans !~ /^y$/i) {
+	if (!read_yn "Create it?") {
 	    exit 0;
 	}
     }
@@ -2804,7 +2843,10 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     $target_image = set_test_option("TARGET_IMAGE", $i);
     $localversion = set_test_option("LOCALVERSION", $i);
 
+    $start_minconfig_defined = 1;
+
     if (!defined($start_minconfig)) {
+	$start_minconfig_defined = 0;
 	$start_minconfig = $minconfig;
     }
 
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index d096a0c..b8bcd14 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -854,6 +854,9 @@
 #   this file as your new min config, and use it to continue the test.
 #   This file does not need to exist on start of test.
 #   This file is not created until a config is found that can be removed.
+#   If this file exists, you will be prompted if you want to use it
+#   as the min_config (overriding MIN_CONFIG) if START_MIN_CONFIG
+#   is not defined.
 #   (required field)
 #
 #  START_MIN_CONFIG is the config to use to start the test with.
-- 
1.7.5.4

[PATCH 18/19] ktest: Keep fonud configs separate from default configs

[Steven Rostedt]

[-- Attachment #1: 0018-ktest-Keep-fonud-configs-separate-from-default-confi.patch --]
[-- Type: text/plain, Size: 2440 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

The IGNORE_CONFIG file holds the configs that we don't want to change
(with their proper settings). But on start up, the make noconfig is
executed, and the configs that are on are also put into the ignore
config category. But these are configs that were forced on by the
kconfig scripts and not something that we found must be enabled to boot
our machine. By keeping the configs that are forced on by default,
separate from the configs we found that are required to boot the box, we
can get a much more interesting IGNORE_CONFIG. In fact, the
IGNORE_CONFIG can usually end up being the must have configs to boot,
and only have 6 or 7 configs set.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   12 ++++++++----
 1 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index cf45f58..e826704 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -2405,6 +2405,7 @@ sub get_depends {
 
 my %min_configs;
 my %keep_configs;
+my %save_configs;
 my %processed_configs;
 my %nochange_config;
 
@@ -2496,15 +2497,17 @@ sub make_min_config {
 
     process_config_ignore $output_config;
 
-    undef %keep_configs;
+    undef %save_configs;
     undef %min_configs;
 
     if (defined($ignore_config)) {
 	# make sure the file exists
 	`touch $ignore_config`;
-	assign_configs \%keep_configs, $ignore_config;
+	assign_configs \%save_configs, $ignore_config;
     }
 
+    %keep_configs = %save_configs;
+
     doprint "Load initial configs from $start_minconfig\n";
 
     # Look at the current min configs, and save off all the
@@ -2614,14 +2617,15 @@ sub make_min_config {
 	    doprint "$min_configs{$config} is needed to boot the box... keeping\n";
 	    # this config is needed, add it to the ignore list.
 	    $keep_configs{$config} = $min_configs{$config};
+	    $save_configs{$config} = $min_configs{$config};
 	    delete $min_configs{$config};
 
 	    # update new ignore configs
 	    if (defined($ignore_config)) {
 		open (OUT, ">$temp_config")
 		    or die "Can't write to $temp_config";
-		foreach my $config (keys %keep_configs) {
-		    print OUT "$keep_configs{$config}\n";
+		foreach my $config (keys %save_configs) {
+		    print OUT "$save_configs{$config}\n";
 		}
 		close OUT;
 		run_command "mv $temp_config $ignore_config" or
-- 
1.7.5.4

[PATCH 19/19] ktest: Fix bug when ADD_CONFIG is set but MIN_CONFIG is not

[Steven Rostedt]

[-- Attachment #1: 0019-ktest-Fix-bug-when-ADD_CONFIG-is-set-but-MIN_CONFIG-.patch --]
[-- Type: text/plain, Size: 2237 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

The MIN_CONFIG is a single config that is considered to have all the
configs that are required to boot the box.

ADD_CONFIG is a list of configs that we add that may contain configs
known to be broken (set off) or just configs that we want every box to
have and this can include shared configs.

If a config has no MIN_CONFIG defined, but has multiple files defined
for the ADD_CONFIG, the test will die, because the MIN_CONFIG will
default to ADD_CONFIG. The problem is the code to open MIN_CONFIG
expects a string of one file, not multiple, and the open will fail.

Since the real minconfig that is used is a concatination of MIN_CONFIG
and ADD_CONFIG files, we change the code to open that instead of
whatever MIN_CONFIG defaults to.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl |   19 +++++++------------
 1 files changed, 7 insertions(+), 12 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index e826704..8d02ccb 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -1970,12 +1970,6 @@ sub config_bisect {
 	unlink $tmpconfig;
     }
 
-    # Add other configs
-    if (defined($addconfig)) {
-	run_command "cat $addconfig >> $tmpconfig" or
-	    dodie "failed to append $addconfig";
-    }
-
     if (-f $tmpconfig) {
 	load_force_config($tmpconfig);
 	process_config_ignore $tmpconfig;
@@ -1997,7 +1991,7 @@ sub config_bisect {
     }
     close(IN);
 
-    # Now run oldconfig with the minconfig (and addconfigs)
+    # Now run oldconfig with the minconfig
     make_oldconfig;
 
     # check to see what we lost (or gained)
@@ -2901,11 +2895,12 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     unlink $dmesg;
     unlink $buildlog;
 
-    if (!defined($minconfig)) {
-	$minconfig = $addconfig;
-
-    } elsif (defined($addconfig)) {
-	run_command "cat $addconfig $minconfig > $tmpdir/add_config" or
+    if (defined($addconfig)) {
+	my $min = $minconfig;
+	if (!defined($minconfig)) {
+	    $min = "";
+	}
+	run_command "cat $addconfig $min > $tmpdir/add_config" or
 	    dodie "Failed to create temp config";
 	$minconfig = "$tmpdir/add_config";
     }
-- 
1.7.5.4