[PATCH 1/2] ktest: add 2nd parameter of run_command() to set the redirect target file

[PATCH 1/2] ktest: add 2nd parameter of run_command() to set the redirect target file

[Satoru Takeuchi]

From: Satoru Takeuchi <satoru.takeuchi@gmail.com>

If we'd like to set the redirect target file of run_command(),
we should define $redirect before calling this function and should undef it
after calling this function. Since it's user-unfriendly, add 2nd parameter of
run_command() for this purpose.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 4063156..f731ef6 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -149,7 +149,6 @@ my $bisect_ret_abort;
 my $bisect_ret_default;
 my $in_patchcheck = 0;
 my $run_test;
-my $redirect;
 my $buildlog;
 my $testlog;
 my $dmesg;
@@ -1529,7 +1528,7 @@ sub fail {
 }
 
 sub run_command {
-    my ($command) = @_;
+    my ($command, $redirect) = @_;
     my $dolog = 0;
     my $dord = 0;
     my $pid;
@@ -2265,9 +2264,7 @@ sub build {
     # Run old config regardless, to enforce min configurations
     make_oldconfig;
 
-    $redirect = "$buildlog";
-    my $build_ret = run_command "$make $build_options";
-    undef $redirect;
+    my $build_ret = run_command "$make $build_options", $buildlog;
 
     if (defined($post_build)) {
 	# Because a post build may change the kernel version
@@ -2360,9 +2357,7 @@ sub child_run_test {
     $poweroff_on_error = 0;
     $die_on_failure = 1;
 
-    $redirect = "$testlog";
-    run_command $run_test or $failed = 1;
-    undef $redirect;
+    run_command $run_test, $testlog or $failed = 1;
 
     exit $failed;
 }
-- 
1.8.5.3

[PATCH 2/2] ktest: run make mrproper not only for OUTPUT_DIR but also for BUILD_DIR to avoid build failure

[Satoru Takeuchi]

From: Satoru Takeuchi <satoru.takeuchi@gmail.com>

Build failure hapens with the following log if BUILD_DIR is dirty.

===============================================================================
...
RUNNING TEST 1 of 1 with option build olddefconfig

make O=/home/sat/ktest/output mrproper ...   CLEAN   arch/x86/tools
  CLEAN   scripts/basic
  CLEAN   scripts/kconfig
  CLEAN   include/config include/generated arch/x86/include/generated
  CLEAN   .config.old include/generated/uapi/linux/version.h
SUCCESS
...
make O=/home/sat/ktest/output -j8 ...   GEN     /home/sat/ktest/output/Makefile
...
  Using /home/sat/src/test-linux-stable/linux-stable as source for kernel
  /home/sat/src/test-linux-stable/linux-stable is not clean, please run 'make mrproper'
  in the '/home/sat/src/test-linux-stable/linux-stable' directory.
make[1]: *** [prepare3] Error 1
make[1]: *** Waiting for unfinished jobs....
  HOSTLD  arch/x86/tools/relocs
make[1]: *** wait: No child processes.  Stop.
make: *** [sub-make] Error 2
FAILED!
CRITICAL FAILURE... failed build
===============================================================================

To avoid this problem, it's necessary to run make mrproper not only
for OUTPUT_DIR, but also for BUILD_DIR.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>

---
 tools/testing/ktest/ktest.pl | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index f731ef6..1169b2a 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -120,6 +120,7 @@ my $syslinux;
 my $syslinux_path;
 my $syslinux_label;
 my $target;
+my $makecmd;
 my $make;
 my $pre_install;
 my $post_install;
@@ -2198,6 +2199,11 @@ sub load_force_config {
     close IN;
 }
 
+sub mrproper {
+    run_command "$makecmd mrproper" or dodie "make mrproper for $builddir";
+    run_command "$make mrproper" or dodie "make mrproper for $outputdir";
+}
+
 sub build {
     my ($type) = @_;
 
@@ -2235,17 +2241,13 @@ sub build {
 	if (!$noclean) {
 	    run_command "mv $output_config $outputdir/config_temp" or
 		dodie "moving .config";
-
-	    run_command "$make mrproper" or dodie "make mrproper";
-
+	    mrproper;
 	    run_command "mv $outputdir/config_temp $output_config" or
 		dodie "moving config_temp";
 	}
-
     } elsif (!$noclean) {
 	unlink "$output_config";
-	run_command "$make mrproper" or
-	    dodie "make mrproper";
+	mrproper;
     }
 
     # add something to distinguish this build
@@ -3996,7 +3998,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
 
     undef %force_config;
 
-    my $makecmd = set_test_option("MAKE_CMD", $i);
+    $makecmd = set_test_option("MAKE_CMD", $i);
 
     $outputdir = set_test_option("OUTPUT_DIR", $i);
     $builddir = set_test_option("BUILD_DIR", $i);
-- 
1.8.5.3

Re: [PATCH 2/2] ktest: run make mrproper not only for OUTPUT_DIR but also for BUILD_DIR to avoid build failure

[Steven Rostedt]

On Sun, 02 Mar 2014 21:23:36 +0900
Satoru Takeuchi <satoru.takeuchi@gmail.com> wrote:

> From: Satoru Takeuchi <satoru.takeuchi@gmail.com>
> 
> Build failure hapens with the following log if BUILD_DIR is dirty.

If you want to do make mrproper in the BUILD_DIR, add the command to
PRE_BUILD or something. I have TAGS in my BUILD_DIRs and this will
remove them for me. I can't accept this change.

I've noticed that there's a bug in the Linux make system that causes
the O=.. builds to fail because somehow the source directory got dirty.
We need to figure out what that bug is.

Thanks,

-- Steve


> 
> ===============================================================================
> ...
> RUNNING TEST 1 of 1 with option build olddefconfig
> 
> make O=/home/sat/ktest/output mrproper ...   CLEAN   arch/x86/tools
>   CLEAN   scripts/basic
>   CLEAN   scripts/kconfig
>   CLEAN   include/config include/generated arch/x86/include/generated
>   CLEAN   .config.old include/generated/uapi/linux/version.h
> SUCCESS
> ...
> make O=/home/sat/ktest/output -j8 ...   GEN     /home/sat/ktest/output/Makefile
> ...
>   Using /home/sat/src/test-linux-stable/linux-stable as source for kernel
>   /home/sat/src/test-linux-stable/linux-stable is not clean, please run 'make mrproper'
>   in the '/home/sat/src/test-linux-stable/linux-stable' directory.
> make[1]: *** [prepare3] Error 1
> make[1]: *** Waiting for unfinished jobs....
>   HOSTLD  arch/x86/tools/relocs
> make[1]: *** wait: No child processes.  Stop.
> make: *** [sub-make] Error 2
> FAILED!
> CRITICAL FAILURE... failed build
> ===============================================================================
> 
> To avoid this problem, it's necessary to run make mrproper not only
> for OUTPUT_DIR, but also for BUILD_DIR.
> 
> Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> 
> ---
>  tools/testing/ktest/ktest.pl | 16 +++++++++-------
>  1 file changed, 9 insertions(+), 7 deletions(-)
> 
> diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
> index f731ef6..1169b2a 100755
> --- a/tools/testing/ktest/ktest.pl
> +++ b/tools/testing/ktest/ktest.pl
> @@ -120,6 +120,7 @@ my $syslinux;
>  my $syslinux_path;
>  my $syslinux_label;
>  my $target;
> +my $makecmd;
>  my $make;
>  my $pre_install;
>  my $post_install;
> @@ -2198,6 +2199,11 @@ sub load_force_config {
>      close IN;
>  }
>  
> +sub mrproper {
> +    run_command "$makecmd mrproper" or dodie "make mrproper for $builddir";
> +    run_command "$make mrproper" or dodie "make mrproper for $outputdir";
> +}
> +
>  sub build {
>      my ($type) = @_;
>  
> @@ -2235,17 +2241,13 @@ sub build {
>  	if (!$noclean) {
>  	    run_command "mv $output_config $outputdir/config_temp" or
>  		dodie "moving .config";
> -
> -	    run_command "$make mrproper" or dodie "make mrproper";
> -
> +	    mrproper;
>  	    run_command "mv $outputdir/config_temp $output_config" or
>  		dodie "moving config_temp";
>  	}
> -
>      } elsif (!$noclean) {
>  	unlink "$output_config";
> -	run_command "$make mrproper" or
> -	    dodie "make mrproper";
> +	mrproper;
>      }
>  
>      # add something to distinguish this build
> @@ -3996,7 +3998,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
>  
>      undef %force_config;
>  
> -    my $makecmd = set_test_option("MAKE_CMD", $i);
> +    $makecmd = set_test_option("MAKE_CMD", $i);
>  
>      $outputdir = set_test_option("OUTPUT_DIR", $i);
>      $builddir = set_test_option("BUILD_DIR", $i);

Re: [PATCH 2/2] ktest: run make mrproper not only for OUTPUT_DIR but also for BUILD_DIR to avoid build failure

[Satoru Takeuchi]

At Sun, 2 Mar 2014 08:29:16 -0500,
Steven Rostedt wrote:
> 
> On Sun, 02 Mar 2014 21:23:36 +0900
> Satoru Takeuchi <satoru.takeuchi@gmail.com> wrote:
> 
> > From: Satoru Takeuchi <satoru.takeuchi@gmail.com>
> > 
> > Build failure hapens with the following log if BUILD_DIR is dirty.
> 
> If you want to do make mrproper in the BUILD_DIR, add the command to
> PRE_BUILD or something. I have TAGS in my BUILD_DIRs and this will
> remove them for me. I can't accept this change.
> 
> I've noticed that there's a bug in the Linux make system that causes
> the O=.. builds to fail because somehow the source directory got dirty.
> We need to figure out what that bug is.

I see. So what I should fix is kbuild system.

Thanks,
Satoru

> 
> Thanks,
> 
> -- Steve
> 
> 
> > 
> > ===============================================================================
> > ...
> > RUNNING TEST 1 of 1 with option build olddefconfig
> > 
> > make O=/home/sat/ktest/output mrproper ...   CLEAN   arch/x86/tools
> >   CLEAN   scripts/basic
> >   CLEAN   scripts/kconfig
> >   CLEAN   include/config include/generated arch/x86/include/generated
> >   CLEAN   .config.old include/generated/uapi/linux/version.h
> > SUCCESS
> > ...
> > make O=/home/sat/ktest/output -j8 ...   GEN     /home/sat/ktest/output/Makefile
> > ...
> >   Using /home/sat/src/test-linux-stable/linux-stable as source for kernel
> >   /home/sat/src/test-linux-stable/linux-stable is not clean, please run 'make mrproper'
> >   in the '/home/sat/src/test-linux-stable/linux-stable' directory.
> > make[1]: *** [prepare3] Error 1
> > make[1]: *** Waiting for unfinished jobs....
> >   HOSTLD  arch/x86/tools/relocs
> > make[1]: *** wait: No child processes.  Stop.
> > make: *** [sub-make] Error 2
> > FAILED!
> > CRITICAL FAILURE... failed build
> > ===============================================================================
> > 
> > To avoid this problem, it's necessary to run make mrproper not only
> > for OUTPUT_DIR, but also for BUILD_DIR.
> > 
> > Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
> > Cc: Steven Rostedt <rostedt@goodmis.org>
> > 
> > ---
> >  tools/testing/ktest/ktest.pl | 16 +++++++++-------
> >  1 file changed, 9 insertions(+), 7 deletions(-)
> > 
> > diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
> > index f731ef6..1169b2a 100755
> > --- a/tools/testing/ktest/ktest.pl
> > +++ b/tools/testing/ktest/ktest.pl
> > @@ -120,6 +120,7 @@ my $syslinux;
> >  my $syslinux_path;
> >  my $syslinux_label;
> >  my $target;
> > +my $makecmd;
> >  my $make;
> >  my $pre_install;
> >  my $post_install;
> > @@ -2198,6 +2199,11 @@ sub load_force_config {
> >      close IN;
> >  }
> >  
> > +sub mrproper {
> > +    run_command "$makecmd mrproper" or dodie "make mrproper for $builddir";
> > +    run_command "$make mrproper" or dodie "make mrproper for $outputdir";
> > +}
> > +
> >  sub build {
> >      my ($type) = @_;
> >  
> > @@ -2235,17 +2241,13 @@ sub build {
> >  	if (!$noclean) {
> >  	    run_command "mv $output_config $outputdir/config_temp" or
> >  		dodie "moving .config";
> > -
> > -	    run_command "$make mrproper" or dodie "make mrproper";
> > -
> > +	    mrproper;
> >  	    run_command "mv $outputdir/config_temp $output_config" or
> >  		dodie "moving config_temp";
> >  	}
> > -
> >      } elsif (!$noclean) {
> >  	unlink "$output_config";
> > -	run_command "$make mrproper" or
> > -	    dodie "make mrproper";
> > +	mrproper;
> >      }
> >  
> >      # add something to distinguish this build
> > @@ -3996,7 +3998,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
> >  
> >      undef %force_config;
> >  
> > -    my $makecmd = set_test_option("MAKE_CMD", $i);
> > +    $makecmd = set_test_option("MAKE_CMD", $i);
> >  
> >      $outputdir = set_test_option("OUTPUT_DIR", $i);
> >      $builddir = set_test_option("BUILD_DIR", $i);
> 

Re: [PATCH 2/2] ktest: run make mrproper not only for OUTPUT_DIR but also for BUILD_DIR to avoid build failure

[Steven Rostedt]

On Sun, 02 Mar 2014 23:17:42 +0900
Satoru Takeuchi <satoru.takeuchi@gmail.com> wrote:

> At Sun, 2 Mar 2014 08:29:16 -0500,
> Steven Rostedt wrote:
> > 
> > On Sun, 02 Mar 2014 21:23:36 +0900
> > Satoru Takeuchi <satoru.takeuchi@gmail.com> wrote:
> > 
> > > From: Satoru Takeuchi <satoru.takeuchi@gmail.com>
> > > 
> > > Build failure hapens with the following log if BUILD_DIR is dirty.
> > 
> > If you want to do make mrproper in the BUILD_DIR, add the command to
> > PRE_BUILD or something. I have TAGS in my BUILD_DIRs and this will
> > remove them for me. I can't accept this change.
> > 
> > I've noticed that there's a bug in the Linux make system that causes
> > the O=.. builds to fail because somehow the source directory got dirty.
> > We need to figure out what that bug is.
> 
> I see. So what I should fix is kbuild system.
> 

That would be great :-)

-- Steve

Re: [PATCH 1/2] ktest: add 2nd parameter of run_command() to set the redirect target file

[Satoru Takeuchi]

Hi Steven,

At Sun, 02 Mar 2014 21:20:31 +0900,
Satoru Takeuchi wrote:
> 
> From: Satoru Takeuchi <satoru.takeuchi@gmail.com>
> 
> If we'd like to set the redirect target file of run_command(),
> we should define $redirect before calling this function and should undef it
> after calling this function. Since it's user-unfriendly, add 2nd parameter of
> run_command() for this purpose.
> 
> Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>

Could you tell whether this patch will be applied? Although you told me that you
won't apply the following patch, I didn't hear about this patch.

> [PATCH 2/2] ktest: run make mrproper not only for OUTPUT_DIR but also for BUILD_DIR to avoid build failure

I make this question since I'll send next patch set this weekend and want to
know whether I can make patch on the top of this patch.

Thanks,
Satoru

> ---
>  tools/testing/ktest/ktest.pl | 11 +++--------
>  1 file changed, 3 insertions(+), 8 deletions(-)
> 
> diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
> index 4063156..f731ef6 100755
> --- a/tools/testing/ktest/ktest.pl
> +++ b/tools/testing/ktest/ktest.pl
> @@ -149,7 +149,6 @@ my $bisect_ret_abort;
>  my $bisect_ret_default;
>  my $in_patchcheck = 0;
>  my $run_test;
> -my $redirect;
>  my $buildlog;
>  my $testlog;
>  my $dmesg;
> @@ -1529,7 +1528,7 @@ sub fail {
>  }
>  
>  sub run_command {
> -    my ($command) = @_;
> +    my ($command, $redirect) = @_;
>      my $dolog = 0;
>      my $dord = 0;
>      my $pid;
> @@ -2265,9 +2264,7 @@ sub build {
>      # Run old config regardless, to enforce min configurations
>      make_oldconfig;
>  
> -    $redirect = "$buildlog";
> -    my $build_ret = run_command "$make $build_options";
> -    undef $redirect;
> +    my $build_ret = run_command "$make $build_options", $buildlog;
>  
>      if (defined($post_build)) {
>  	# Because a post build may change the kernel version
> @@ -2360,9 +2357,7 @@ sub child_run_test {
>      $poweroff_on_error = 0;
>      $die_on_failure = 1;
>  
> -    $redirect = "$testlog";
> -    run_command $run_test or $failed = 1;
> -    undef $redirect;
> +    run_command $run_test, $testlog or $failed = 1;
>  
>      exit $failed;
>  }
> -- 
> 1.8.5.3
> 

Re: [PATCH 1/2] ktest: add 2nd parameter of run_command() to set the redirect target file

[Steven Rostedt]

On Sat, 08 Mar 2014 09:12:31 +0900
Satoru Takeuchi <satoru.takeuchi@gmail.com> wrote:

> Hi Steven,
> 
> At Sun, 02 Mar 2014 21:20:31 +0900,
> Satoru Takeuchi wrote:
> > 
> > From: Satoru Takeuchi <satoru.takeuchi@gmail.com>
> > 
> > If we'd like to set the redirect target file of run_command(),
> > we should define $redirect before calling this function and should undef it
> > after calling this function. Since it's user-unfriendly, add 2nd parameter of
> > run_command() for this purpose.
> > 
> > Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
> > Cc: Steven Rostedt <rostedt@goodmis.org>
> 
> Could you tell whether this patch will be applied? Although you told me that you
> won't apply the following patch, I didn't hear about this patch.

Ah, I was thinking this one was needed for the next patch, so I didn't
apply it.

> 
> > [PATCH 2/2] ktest: run make mrproper not only for OUTPUT_DIR but also for BUILD_DIR to avoid build failure
> 
> I make this question since I'll send next patch set this weekend and want to
> know whether I can make patch on the top of this patch.

You can add this one again. I haven't really reviewed it, but if other
patches depend on it, include it as well. Include it anyway if you want
me to review it.

Thanks,

-- Steve

Re: [PATCH 1/2] ktest: add 2nd parameter of run_command() to set the redirect target file

[Satoru Takeuchi]

At Fri, 7 Mar 2014 19:43:28 -0500,
Steven Rostedt wrote:
> 
> On Sat, 08 Mar 2014 09:12:31 +0900
> Satoru Takeuchi <satoru.takeuchi@gmail.com> wrote:
> 
> > Hi Steven,
> > 
> > At Sun, 02 Mar 2014 21:20:31 +0900,
> > Satoru Takeuchi wrote:
> > > 
> > > From: Satoru Takeuchi <satoru.takeuchi@gmail.com>
> > > 
> > > If we'd like to set the redirect target file of run_command(),
> > > we should define $redirect before calling this function and should undef it
> > > after calling this function. Since it's user-unfriendly, add 2nd parameter of
> > > run_command() for this purpose.
> > > 
> > > Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
> > > Cc: Steven Rostedt <rostedt@goodmis.org>
> > 
> > Could you tell whether this patch will be applied? Although you told me that you
> > won't apply the following patch, I didn't hear about this patch.
> 
> Ah, I was thinking this one was needed for the next patch, so I didn't
> apply it.

The two patches don't depend on earch other.

> 
> > 
> > > [PATCH 2/2] ktest: run make mrproper not only for OUTPUT_DIR but also for BUILD_DIR to avoid build failure
> > 
> > I make this question since I'll send next patch set this weekend and want to
> > know whether I can make patch on the top of this patch.
> 
> You can add this one again. I haven't really reviewed it, but if other
> patches depend on it, include it as well. Include it anyway if you want
> me to review it.

OK, I'll resend patch 1/2 at the next post of my patchset.

Thanks,
Satoru

> 
> Thanks,
> 
> -- Steve
> 
>