[PATCH 0/2] [GIT PULL] localmodconfig: Fix missing depends of config files included in if statements

[PATCH 0/2] [GIT PULL] localmodconfig: Fix missing depends of config files included in if statements

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 762 bytes --]


Linus,

A bug was recently found (today) in the make localmodconfig where it would miss
dependencies of config files that are included in other config files inside
an if statement.

Also added a debug print that helped in solving this bug.

Please pull the latest ktest-v3.10 tree, which can be found at:

  git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest.git
ktest-v3.10

Head SHA1: 7714fe5ac0cf752eae7a20ef23408aa26271a3eb


Steven Rostedt (2):
      localmodconfig: Add debug prints for dependencies of module configs
      localmodconfig: Process source kconfig files as they are found

----
 scripts/kconfig/streamline_config.pl |   26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

[PATCH 1/2] localmodconfig: Add debug prints for dependencies of module configs

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 1211 bytes --]

From: Steven Rostedt <rostedt@goodmis.org>

When a config for a module is added to the list to save in the final
config file, add a print to show what dependencies are used. This is
useful to debug when a config is disabled by the make oldconfig after
localmodconfig is finished.

This print only appears if the environment variable LOCALMODCONFIG_DEBUG
is defined.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |    9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 3368939..343a568 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -396,6 +396,15 @@ foreach my $module (keys(%modules)) {
 	foreach my $conf (@arr) {
 	    $configs{$conf} = $module;
 	    dprint "$conf added by direct ($module)\n";
+	    if ($debugprint) {
+		my $c=$conf;
+		$c =~ s/^CONFIG_//;
+		if (defined($depends{$c})) {
+		    dprint " deps = $depends{$c}\n";
+		} else {
+		    dprint " no deps\n";
+		}
+	    }
 	}
     } else {
 	# Most likely, someone has a custom (binary?) module loaded.
-- 
1.7.10.4



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

[PATCH 2/2] localmodconfig: Process source kconfig files as they are found

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 2793 bytes --]

From: Steven Rostedt <rostedt@goodmis.org>

A bug was reported that caused localmodconfig to not keep all the
dependencies of ATH9K. This was caused by the kconfig file:

In drivers/net/wireless/ath/Kconfig:

---
if ATH_CARDS

config ATH_DEBUG
        bool "Atheros wireless debugging"
        ---help---
          Say Y, if you want to debug atheros wireless drivers.
          Right now only ath9k makes use of this.

source "drivers/net/wireless/ath/ath5k/Kconfig"
source "drivers/net/wireless/ath/ath9k/Kconfig"
source "drivers/net/wireless/ath/carl9170/Kconfig"
source "drivers/net/wireless/ath/ath6kl/Kconfig"
source "drivers/net/wireless/ath/ar5523/Kconfig"
source "drivers/net/wireless/ath/wil6210/Kconfig"

endif
---

The current way kconfig works, it processes new source files after the
first file is completed. It creates an array of new source config files
and when the one file is finished, it continues with the next file.

Unfortunately, this means that it loses the fact that the source file is
within an "if" statement, and this means that each of these source file's
configs will not have the proper dependencies set.

As ATH9K requires ATH_CARDS set, the localmodconfig did not see that
dependency, and did not enable ATH_CARDS. When the oldconfig was run, it
forced ATH9K to be disabled.

Link: http://lkml.kernel.org/r/alpine.DEB.2.02.1304291022320.9234@oneiric

Cc: stable@vger.kernel.org # 3.8+
Reported-by: Robert P. J. Day <rpjday@crashcourse.ca>
Tested-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |   17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 343a568..4606cdf 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -156,7 +156,6 @@ sub read_kconfig {
 
     my $state = "NONE";
     my $config;
-    my @kconfigs;
 
     my $cont = 0;
     my $line;
@@ -190,7 +189,13 @@ sub read_kconfig {
 
 	# collect any Kconfig sources
 	if (/^source\s*"(.*)"/) {
-	    $kconfigs[$#kconfigs+1] = $1;
+	    my $kconfig = $1;
+	    # prevent reading twice.
+	    if (!defined($read_kconfigs{$kconfig})) {
+		$read_kconfigs{$kconfig} = 1;
+		read_kconfig($kconfig);
+	    }
+	    next;
 	}
 
 	# configs found
@@ -250,14 +255,6 @@ sub read_kconfig {
 	}
     }
     close($kinfile);
-
-    # read in any configs that were found.
-    foreach my $kconfig (@kconfigs) {
-	if (!defined($read_kconfigs{$kconfig})) {
-	    $read_kconfigs{$kconfig} = 1;
-	    read_kconfig($kconfig);
-	}
-    }
 }
 
 if ($kconfig) {
-- 
1.7.10.4



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

[PATCH 2/2] localmodconfig: Process source kconfig files as they are found

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 2793 bytes --]

From: Steven Rostedt <rostedt@goodmis.org>

A bug was reported that caused localmodconfig to not keep all the
dependencies of ATH9K. This was caused by the kconfig file:

In drivers/net/wireless/ath/Kconfig:

---
if ATH_CARDS

config ATH_DEBUG
        bool "Atheros wireless debugging"
        ---help---
          Say Y, if you want to debug atheros wireless drivers.
          Right now only ath9k makes use of this.

source "drivers/net/wireless/ath/ath5k/Kconfig"
source "drivers/net/wireless/ath/ath9k/Kconfig"
source "drivers/net/wireless/ath/carl9170/Kconfig"
source "drivers/net/wireless/ath/ath6kl/Kconfig"
source "drivers/net/wireless/ath/ar5523/Kconfig"
source "drivers/net/wireless/ath/wil6210/Kconfig"

endif
---

The current way kconfig works, it processes new source files after the
first file is completed. It creates an array of new source config files
and when the one file is finished, it continues with the next file.

Unfortunately, this means that it loses the fact that the source file is
within an "if" statement, and this means that each of these source file's
configs will not have the proper dependencies set.

As ATH9K requires ATH_CARDS set, the localmodconfig did not see that
dependency, and did not enable ATH_CARDS. When the oldconfig was run, it
forced ATH9K to be disabled.

Link: http://lkml.kernel.org/r/alpine.DEB.2.02.1304291022320.9234@oneiric

Cc: stable@vger.kernel.org # 3.8+
Reported-by: Robert P. J. Day <rpjday@crashcourse.ca>
Tested-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |   17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 343a568..4606cdf 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -156,7 +156,6 @@ sub read_kconfig {
 
     my $state = "NONE";
     my $config;
-    my @kconfigs;
 
     my $cont = 0;
     my $line;
@@ -190,7 +189,13 @@ sub read_kconfig {
 
 	# collect any Kconfig sources
 	if (/^source\s*"(.*)"/) {
-	    $kconfigs[$#kconfigs+1] = $1;
+	    my $kconfig = $1;
+	    # prevent reading twice.
+	    if (!defined($read_kconfigs{$kconfig})) {
+		$read_kconfigs{$kconfig} = 1;
+		read_kconfig($kconfig);
+	    }
+	    next;
 	}
 
 	# configs found
@@ -250,14 +255,6 @@ sub read_kconfig {
 	}
     }
     close($kinfile);
-
-    # read in any configs that were found.
-    foreach my $kconfig (@kconfigs) {
-	if (!defined($read_kconfigs{$kconfig})) {
-	    $read_kconfigs{$kconfig} = 1;
-	    read_kconfig($kconfig);
-	}
-    }
 }
 
 if ($kconfig) {
-- 
1.7.10.4



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [PATCH 0/2] [GIT PULL] localmodconfig: Fix missing depends of config files included in if statements

[Linus Torvalds]

On Mon, Apr 29, 2013 at 12:51 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
>
>
> Please pull the latest ktest-v3.10 tree, which can be found at:
>
>   git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest.git
> ktest-v3.10

Hmm. This is your ktest tree (which I already pulled the ktest changes
from), not your localmodconfig tree. But then all the explanations are
about your localmodconfig changes.

I assume you *meant* to ask me to pull your linux-kconfig.git tree?
That has a "ktest-v3.10" tag too, but I think that's a mistake too,
and you *meant* to make a tag called "localconfig" like you have done
in the past.

There's just too much confusion here for me to touch anything at all,
so please fix things up.

              Linus

Re: [PATCH 0/2] [GIT PULL] localmodconfig: Fix missing depends of config files included in if statements

[Linus Torvalds]

On Mon, Apr 29, 2013 at 2:01 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> There's just too much confusion here for me to touch anything at all,
> so please fix things up.

Oh, and you might as well check that I resolved the conflicts in the
trace pull correctly while you're at it. They looked pretty obvious,
and it compiles for me, but ..

              Linus

Re: [PATCH 0/2] [GIT PULL] localmodconfig: Fix missing depends of config files included in if statements

[Steven Rostedt]

On Mon, 2013-04-29 at 14:42 -0700, Linus Torvalds wrote:
> On Mon, Apr 29, 2013 at 2:01 PM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > There's just too much confusion here for me to touch anything at all,
> > so please fix things up.
> 
> Oh, and you might as well check that I resolved the conflicts in the
> trace pull correctly while you're at it. They looked pretty obvious,
> and it compiles for me, but ..

Almost.

I'll also run my full ftrace test suite on your latest kernel, and see
if it finds anything else.

Please apply:

---
tracing: Fix small merge bug

During the 3.10 merge, a conflict happened and the resolution was
almost, but not quite, correct. An if statement was reversed.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>

diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 581630a..ae6fa2d 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -904,7 +904,7 @@ update_max_tr_single(struct trace_array *tr, struct task_struct *tsk, int cpu)
 		return;
 
 	WARN_ON_ONCE(!irqs_disabled());
-	if (tr->allocated_snapshot) {
+	if (!tr->allocated_snapshot) {
 		/* Only the nop tracer should hit this when disabling */
 		WARN_ON_ONCE(tr->current_trace != &nop_trace);
 		return;

Re: [PATCH 0/2] [GIT PULL] localmodconfig: Fix missing depends of config files included in if statements

[Steven Rostedt]

On Mon, 2013-04-29 at 20:08 -0400, Steven Rostedt wrote:
> On Mon, 2013-04-29 at 14:42 -0700, Linus Torvalds wrote:
> > On Mon, Apr 29, 2013 at 2:01 PM, Linus Torvalds
> > <torvalds@linux-foundation.org> wrote:
> > >
> > > There's just too much confusion here for me to touch anything at all,
> > > so please fix things up.
> > 
> > Oh, and you might as well check that I resolved the conflicts in the
> > trace pull correctly while you're at it. They looked pretty obvious,
> > and it compiles for me, but ..
> 
> Almost.
> 

BTW, what's the preferred method for this. I already posted a lot of
work to linux-next when I found bugs that required going into your tree.
I backported the fixes knowing that it will cause conflicts when you
merge.

IIRC, you stated that you don't mind doing conflict resolutions
yourself, so I did not try to fix it a head of time, as the conflicts
were rather minor.

Should I have merged your tree and done the conflict resolutions myself,
or was it OK to do what I did, and let you do the conflict resolution
and send you any fixes that needed to be done afterward?

-- Steve

Re: [PATCH 0/2] [GIT PULL] localmodconfig: Fix missing depends of config files included in if statements

[Linus Torvalds]

On Mon, Apr 29, 2013 at 5:17 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
>
> BTW, what's the preferred method for this. I already posted a lot of
> work to linux-next when I found bugs that required going into your tree.
> I backported the fixes knowing that it will cause conflicts when you
> merge.
>
> IIRC, you stated that you don't mind doing conflict resolutions
> yourself, so I did not try to fix it a head of time, as the conflicts
> were rather minor.
>
> Should I have merged your tree and done the conflict resolutions myself,
> or was it OK to do what I did, and let you do the conflict resolution
> and send you any fixes that needed to be done afterward?

You did the right thing. In general, if you know there will be
conflicts, it's nice if you mention then in the pull request, but for
simple stuff like this it's really not a big deal. The fact that I
screwed up and then missed a "!" when editing it all is embarrassing,
but it wasn't because the conflict was really *complicated*, it was
just stupid editing error.

If the conflicts are really complex, at some point I really enjoy
getting a "here's a pre-merged branch if you prefer it", and if people
send that, I still tend to do the merge myself, but then I often just
compare against the pre-merged one afterwards to verify. But that's
actually extra work, so I'd suggest doing that only for things that
really warrant it. It's generally a bad thing if it happens, because
it means that we had some bad workflow and people stepped on each
others toes (or we had unlucky backports etc that ended up being in
the same area as much more work).

                   Linus

Re: [PATCH 0/2] [GIT PULL] localmodconfig: Fix missing depends of config files included in if statements

[Steven Rostedt]

On Mon, 2013-04-29 at 20:08 -0400, Steven Rostedt wrote:

> I'll also run my full ftrace test suite on your latest kernel, and see
> if it finds anything else.
> 

With the fix I sent, everything passed.

Thanks,

-- Steve

Re: [PATCH 0/2] [GIT PULL] localmodconfig: Fix missing depends of config files included in if statements

[Steven Rostedt]

On Mon, 2013-04-29 at 14:01 -0700, Linus Torvalds wrote:
> On Mon, Apr 29, 2013 at 12:51 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
> >
> >
> > Please pull the latest ktest-v3.10 tree, which can be found at:
> >
> >   git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest.git
> > ktest-v3.10
> 
> Hmm. This is your ktest tree (which I already pulled the ktest changes
> from), not your localmodconfig tree. But then all the explanations are
> about your localmodconfig changes.
> 
> I assume you *meant* to ask me to pull your linux-kconfig.git tree?

Face palm!

> That has a "ktest-v3.10" tag too, but I think that's a mistake too,
> and you *meant* to make a tag called "localconfig" like you have done
> in the past.
> 
> There's just too much confusion here for me to touch anything at all,
> so please fix things up.

Yeah, sorry. I have no idea what I was thinking. I was doing my double
check of ktest changes and must have still had that on my mind when I
switched over to the localmodconfig stuff.

I'll clean it up.

-- Steve

[PATCH 0/2] [GIT PULL take 2 ] localmodconfig: Fix missing depends of config files included in if statements

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 952 bytes --]


Linus,

[
  Sorry for the previous confusion. My scripts to make my patch sets
  key off of the tag name. Thus by using the ktest tag, my scripts
  put in the linux-ktest.git tree.
]

A bug was recently found in the make localmodconfig where it would miss
dependencies of config files are include in other config files inside
an if statement.

Also added a debug print that helped in solving this bug.


Please pull the latest localmodconfig-v3.10 tree, which can be found at:

  git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-kconfig.git
localmodconfig-v3.10

Head SHA1: 0d0f18b32c5382431163391f224195dea251cae8


Steven Rostedt (2):
      localmodconfig: Add debug prints for dependencies of module configs
      localmodconfig: Process source kconfig files as they are found

----
 scripts/kconfig/streamline_config.pl |   26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

[PATCH 1/2] localmodconfig: Add debug prints for dependencies of module configs

[Steven Rostedt]

[-- Attachment #1: Type: text/plain, Size: 1211 bytes --]

From: Steven Rostedt <rostedt@goodmis.org>

When a config for a module is added to the list to save in the final
config file, add a print to show what dependencies are used. This is
useful to debug when a config is disabled by the make oldconfig after
localmodconfig is finished.

This print only appears if the environment variable LOCALMODCONFIG_DEBUG
is defined.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |    9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 3368939..343a568 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -396,6 +396,15 @@ foreach my $module (keys(%modules)) {
 	foreach my $conf (@arr) {
 	    $configs{$conf} = $module;
 	    dprint "$conf added by direct ($module)\n";
+	    if ($debugprint) {
+		my $c=$conf;
+		$c =~ s/^CONFIG_//;
+		if (defined($depends{$c})) {
+		    dprint " deps = $depends{$c}\n";
+		} else {
+		    dprint " no deps\n";
+		}
+	    }
 	}
     } else {
 	# Most likely, someone has a custom (binary?) module loaded.
-- 
1.7.10.4



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]