[PATCH 0/5] ktest.pl: Enhancements for 6.17

[PATCH 0/5] ktest.pl: Enhancements for 6.17

[Steven Rostedt]

Some ktest.pl improvements:

- Add new -D option that allows to override variables and options

  For example:

    ./ktest.pl -DPATCH_START:=HEAD~1 -DOUTPUT_DIR=/work/build/urgent config

  The above sets the variable "PATCH_START" to HEAD~1 and the OUTPUT_DIR
  option to "/work/build/urgent".

  This is useful because currently the only way to make a slight change to a
  config file is by modifying that config file. For one time changes, this
  can be annoying. Having a way to do a one time override from the command
  line simplifies the workflow.

  Temp variables (PATCH_START) will override every temp variable in the
  config file,  whereas options will act like a normal OVERRIDE option and
  will only affect the session they define.

     -DBUILD_OUTPUT=/work/git/linux.git

  Replaces the default BUILD_OUTPUT option.

     '-DBUILD_OUTPUT[2]=/work/git/linux.git'

  Only replaces the BUILD_OUTPUT variable for test #2.

- If an option contains itself, just drop it instead of going into an
  infinite loop and failing to parse (it doesn't crash, it detects the
  recursion after 100 iterations anyway).

  Some configs may define a variable with the same name as the option:

     ADD_CONFIG := $(ADD_CONFIG)

  But if the option doesn't exist, it the above will fail to parse. In these
  cases, just ignore evaluating the option inside the definition of another
  option if it has the same name.

- Display the BUILD_DIR and OUTPUT_DIR options at the start of every test

  It is useful to know which kernel source and what destination a test is
  using when it starts, in case a mistake is made. This makes it easier to
  abort the test if the wrong source or destination is being used instead of
  waiting until the test completes.

Steven Rostedt (5):
      ktest.pl: Add -D option to override options
      ktest.pl: Allow command option -D to override temp variables
      ktest.pl: Have -D option work without a space
      ktest.pl: Prevent recursion of default variable options
      ktest.pl: Always display BUILD_DIR and OUTPUT_DIR at the start of tests

----
 tools/testing/ktest/ktest.pl | 83 ++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 80 insertions(+), 3 deletions(-)

[PATCH 1/5] ktest.pl: Add -D option to override options

[Steven Rostedt]

From: Steven Rostedt <rostedt@goodmis.org>

Add -D option that lets the user override options in the config.

For instance, if the config has: BUILD_NOCLEAN=1 which prevents mrproper
from being called before builds, and the user wants to call it once. The
user can run:

  ktest -D BUILD_NOCLEAN=0 config

And the default "BUILD_NOCLEAN" options will be disabled.

If the user wants to change the second test to do a build and not boot,
the user can run:

  ktest -D 'TEST_TYPE[2]=build' config

Where the '[#]' is for the test to assign the variable for. In the above
example, it will happen on test 2.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 45 +++++++++++++++++++++++++++++++++++-
 1 file changed, 44 insertions(+), 1 deletion(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index a5f7fdd0c1fb..8fcc09893986 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -21,6 +21,7 @@ my %opt;
 my %repeat_tests;
 my %repeats;
 my %evals;
+my @command_vars;
 
 #default opts
 my %default = (
@@ -1286,6 +1287,19 @@ sub read_config {
 
     $test_case = __read_config $config, \$test_num;
 
+    foreach my $val (@command_vars) {
+	chomp $val;
+	my %command_overrides;
+	if ($val =~ m/^\s*([A-Z_\[\]\d]+)\s*=\s*(.*?)\s*$/) {
+	    my $lvalue = $1;
+	    my $rvalue = $2;
+
+	    set_value($lvalue, $rvalue, 1, \%command_overrides, "COMMAND LINE");
+	} else {
+	    die "Invalid option definition '$val'\n";
+	}
+    }
+
     # make sure we have all mandatory configs
     get_mandatory_configs;
 
@@ -4242,8 +4256,37 @@ sub cancel_test {
     die "\nCaught Sig Int, test interrupted: $!\n"
 }
 
-$#ARGV < 1 or die "ktest.pl version: $VERSION\n   usage: ktest.pl [config-file]\n";
+sub die_usage {
+    die << "EOF"
+ktest.pl version: $VERSION
+   usage: ktest.pl [options] [config-file]
+    [options]:
+       -D value: Where value can act as an option override.
+                -D BUILD_NOCLEAN=1
+                    Sets global BUILD_NOCLEAN to 1
+                -D TEST_TYPE[2]=build
+                    Sets TEST_TYPE of test 2 to "build"
+
+EOF
+;
+}
+
+while ( $#ARGV >= 0 ) {
+    if ( $ARGV[0] eq "-D" ) {
+	shift;
+	die_usage if ($#ARGV < 1);
+	my $val = shift;
+
+	$command_vars[$#command_vars + 1] = $val;
+
+    } elsif ( $ARGV[0] eq "-h" ) {
+	die_usage;
+    } else {
+	last;
+    }
+}
 
+$#ARGV < 1 or die_usage;
 if ($#ARGV == 0) {
     $ktest_config = $ARGV[0];
     if (! -f $ktest_config) {
-- 
2.47.2

[PATCH 2/5] ktest.pl: Allow command option -D to override temp variables

[Steven Rostedt]

From: Steven Rostedt <rostedt@goodmis.org>

Currently -D only updates the persistent options that are defined with
"=". Allow it to also override all temp variables that are defined with
":=".

 ktest.pl -D 'USE_TEMP_DIR:=1' -D 'TEST_TYPE[2]=build' config

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 8fcc09893986..c441934f1def 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -22,6 +22,7 @@ my %repeat_tests;
 my %repeats;
 my %evals;
 my @command_vars;
+my %command_tmp_vars;
 
 #default opts
 my %default = (
@@ -901,14 +902,22 @@ sub set_eval {
 }
 
 sub set_variable {
-    my ($lvalue, $rvalue) = @_;
+    my ($lvalue, $rvalue, $command) = @_;
 
+    # Command line variables override all others
+    if (defined($command_tmp_vars{$lvalue})) {
+	return;
+    }
     if ($rvalue =~ /^\s*$/) {
 	delete $variable{$lvalue};
     } else {
 	$rvalue = process_variables($rvalue);
 	$variable{$lvalue} = $rvalue;
     }
+
+    if (defined($command)) {
+	$command_tmp_vars{$lvalue} = 1;
+    }
 }
 
 sub process_compare {
@@ -4267,6 +4276,11 @@ ktest.pl version: $VERSION
                 -D TEST_TYPE[2]=build
                     Sets TEST_TYPE of test 2 to "build"
 
+	        It can also override all temp variables.
+                 -D USE_TEMP_DIR:=1
+                    Will override all variables that use
+                    "USE_TEMP_DIR="
+
 EOF
 ;
 }
@@ -4277,7 +4291,11 @@ while ( $#ARGV >= 0 ) {
 	die_usage if ($#ARGV < 1);
 	my $val = shift;
 
-	$command_vars[$#command_vars + 1] = $val;
+	if ($val =~ m/(.*?):=(.*)$/) {
+	    set_variable($1, $2, 1);
+	} else {
+	    $command_vars[$#command_vars + 1] = $val;
+	}
 
     } elsif ( $ARGV[0] eq "-h" ) {
 	die_usage;
-- 
2.47.2

[PATCH 3/5] ktest.pl: Have -D option work without a space

[Steven Rostedt]

From: Steven Rostedt <rostedt@goodmis.org>

Allow -DBUILD_TYPE=boot work the same as -D BUILD_TYPE=boot just like
normal single character option does in most applications.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index c441934f1def..075c386af5e5 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -4297,6 +4297,15 @@ while ( $#ARGV >= 0 ) {
 	    $command_vars[$#command_vars + 1] = $val;
 	}
 
+    } elsif ( $ARGV[0] =~ m/^-D(.*)/) {
+	my $val = $1;
+	shift;
+
+	if ($val =~ m/(.*?):=(.*)$/) {
+	    set_variable($1, $2, 1);
+	} else {
+	    $command_vars[$#command_vars + 1] = $val;
+	}
     } elsif ( $ARGV[0] eq "-h" ) {
 	die_usage;
     } else {
-- 
2.47.2

[PATCH 4/5] ktest.pl: Prevent recursion of default variable options

[Steven Rostedt]

From: Steven Rostedt <rostedt@goodmis.org>

If a default variable contains itself, do not recurse on it.

For example:

  ADD_CONFIG := ${CONFIG_DIR}/temp_config
  DEFAULTS
  ADD_CONFIG = ${CONFIG_DIR}/default_config ${ADD_CONFIG}

The above works because the temp variable ADD_CONFIG (is a temp because it
is created with ":=") is already defined, it will be substituted in the
variable option. But if it gets commented out:

  # ADD_CONFIG := ${CONFIG_DIR}/temp_config
  DEFAULTS
  ADD_CONFIG = ${CONFIG_DIR}/default_config ${ADD_CONFIG}

Then the above will go into a recursive loop where ${ADD_CONFIG} will
get replaced with the current definition of ADD_CONFIG which contains the
${ADD_CONFIG} and that will also try to get converted. ktest.pl will error
after 100 attempts of recursion and fail.

When replacing a variable with the default variable, if the default
variable contains itself, do not replace it.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 075c386af5e5..b2971430d7e4 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -1394,7 +1394,10 @@ sub __eval_option {
 	# If a variable contains itself, use the default var
 	if (($var eq $name) && defined($opt{$var})) {
 	    $o = $opt{$var};
-	    $retval = "$retval$o";
+	    # Only append if the default doesn't contain itself
+	    if ($o !~ m/\$\{$var\}/) {
+		$retval = "$retval$o";
+	    }
 	} elsif (defined($opt{$o})) {
 	    $o = $opt{$o};
 	    $retval = "$retval$o";
-- 
2.47.2

[PATCH 5/5] ktest.pl: Always display BUILD_DIR and OUTPUT_DIR at the start of tests

[Steven Rostedt]

From: Steven Rostedt <rostedt@goodmis.org>

As ktest.pl can run in various different directories, to make sure the
test is running in the proper directory with the proper source and proper
destination directory, display the content of BUILD_DIR and OUTPUT_DIR at
the start of every test.

This can be helpful for the test runner to stop the test if a test is
running in the wrong location instead of finding out after the test has
completed.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index b2971430d7e4..c460838dfd6f 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -4539,6 +4539,10 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
 
     doprint "RUNNING TEST $i of $opt{NUM_TESTS}$name with option $test_type $run_type$installme\n\n";
 
+    # Always show which build directory and output directory is being used
+    doprint "BUILD_DIR=$builddir\n";
+    doprint "OUTPUT_DIR=$outputdir\n\n";
+
     if (defined($pre_test)) {
 	my $ret = run_command $pre_test;
 	if (!$ret && defined($pre_test_die) &&
-- 
2.47.2