[PATCH 0/3] MAINTAINERS: Add and support "K:" keyword

[PATCH 0/3] MAINTAINERS: Add and support "K:" keyword

[Joe Perches]

K: is a regex that's used to match content in a patch and
add entries from a MAINTAINERS section

Joe Perches (3):
  scripts/get_maintainer.pl: Add patch/file search for keywords
  MAINTAINERS: Document new "K:" entry type
  MAINTAINERS: Add OPEN FIRMWARE DEVICE TREE section

 MAINTAINERS               |   76 ++++++++++++++++++++++++++------------------
 scripts/get_maintainer.pl |   37 ++++++++++++++++++++-
 2 files changed, 80 insertions(+), 33 deletions(-)

[PATCH 1/3] scripts/get_maintainer.pl: Add patch/file search for keywords

[Joe Perches]

Based on an idea from Wolfram Sang <w.sang@pengutronix.de>

Add search for MAINTAINERS line "K:" regex pattern match in a patch or file
Matches are added after file pattern matches
Add --keywords command line switch (default 1, on)
Change version to 0.21

Signed-off-by: Joe Perches <joe@perches.com>
---
 scripts/get_maintainer.pl |   37 +++++++++++++++++++++++++++++++++++--
 1 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/scripts/get_maintainer.pl b/scripts/get_maintainer.pl
index cdb44b6..102b766 100755
--- a/scripts/get_maintainer.pl
+++ b/scripts/get_maintainer.pl
@@ -13,7 +13,7 @@
 use strict;
 
 my $P = $0;
-my $V = '0.20';
+my $V = '0.21';
 
 use Getopt::Long qw(:config no_auto_abbrev);
 
@@ -37,6 +37,7 @@ my $scm = 0;
 my $web = 0;
 my $subsystem = 0;
 my $status = 0;
+my $keywords = 1;
 my $from_filename = 0;
 my $pattern_depth = 0;
 my $version = 0;
@@ -84,6 +85,7 @@ if (!GetOptions(
 		'scm!' => \$scm,
 		'web!' => \$web,
 		'pattern-depth=i' => \$pattern_depth,
+		'k|keywords!' => \$keywords,
 		'f|file' => \$from_filename,
 		'v|version' => \$version,
 		'h|help' => \$help,
@@ -132,6 +134,8 @@ if (!top_of_kernel_tree($lk_path)) {
 ## Read MAINTAINERS for type/value pairs
 
 my @typevalue = ();
+my %keyword_hash;
+
 open(MAINT, "<${lk_path}MAINTAINERS") || die "$P: Can't open MAINTAINERS\n";
 while (<MAINT>) {
     my $line = $_;
@@ -149,6 +153,8 @@ while (<MAINT>) {
 	    if ((-d $value)) {
 		$value =~ s@([^/])$@$1/@;
 	    }
+	} elsif ($type eq "K") {
+	    $keyword_hash{@typevalue} = $value;
 	}
 	push(@typevalue, "$type:$value");
     } elsif (!/^(\s)*$/) {
@@ -188,6 +194,7 @@ if ($email_remove_duplicates) {
 
 my @files = ();
 my @range = ();
+my @keyword_tvi = ();
 
 foreach my $file (@ARGV) {
     ##if $file is a directory and it lacks a trailing slash, add one
@@ -198,11 +205,24 @@ foreach my $file (@ARGV) {
     }
     if ($from_filename) {
 	push(@files, $file);
+	if (-f $file && $keywords) {
+	    open(FILE, "<$file") or die "$P: Can't open ${file}\n";
+	    while (<FILE>) {
+		my $patch_line = $_;
+		foreach my $line (keys %keyword_hash) {
+		    if ($patch_line =~ m/^.*$keyword_hash{$line}/x) {
+			push(@keyword_tvi, $line);
+		    }
+		}
+	    }
+	    close(FILE);
+	}
     } else {
 	my $file_cnt = @files;
 	my $lastfile;
 	open(PATCH, "<$file") or die "$P: Can't open ${file}\n";
 	while (<PATCH>) {
+	    my $patch_line = $_;
 	    if (m/^\+\+\+\s+(\S+)/) {
 		my $filename = $1;
 		$filename =~ s@^[^/]*/@@;
@@ -213,6 +233,12 @@ foreach my $file (@ARGV) {
 		if ($email_git_blame) {
 		    push(@range, "$lastfile:$1:$2");
 		}
+	    } elsif ($keywords) {
+		foreach my $line (keys %keyword_hash) {
+		    if ($patch_line =~ m/^[+-].*$keyword_hash{$line}/x) {
+			push(@keyword_tvi, $line);
+		    }
+		}
 	    }
 	}
 	close(PATCH);
@@ -286,6 +312,13 @@ foreach my $file (@files) {
     }
 }
 
+if ($keywords) {
+    @keyword_tvi = sort_and_uniq(@keyword_tvi);
+    foreach my $line (@keyword_tvi) {
+	add_categories($line);
+    }
+}
+
 if ($email) {
     foreach my $chief (@penguin_chief) {
 	if ($chief =~ m/^(.*):(.*)/) {
@@ -384,6 +417,7 @@ Output type options:
 
 Other options:
   --pattern-depth => Number of pattern directory traversals (default: 0 (all))
+  --keywords => scan patch for keywords (default: 1 (on))
   --version => show version
   --help => show this help information
 
@@ -486,7 +520,6 @@ sub format_email {
 }
 
 sub find_starting_index {
-
     my ($index) = @_;
 
     while ($index > 0) {
-- 
1.6.5.rc3.dirty

Re: [PATCH 1/3] scripts/get_maintainer.pl: Add patch/file search for keywords

[Wolfram Sang]

[-- Attachment #1: Type: text/plain, Size: 4740 bytes --]

Hi Joe,

On Fri, Oct 09, 2009 at 11:05:02PM -0700, Joe Perches wrote:
> Based on an idea from Wolfram Sang <w.sang@pengutronix.de>
> 
> Add search for MAINTAINERS line "K:" regex pattern match in a patch or file
> Matches are added after file pattern matches
> Add --keywords command line switch (default 1, on)
> Change version to 0.21
> 
> Signed-off-by: Joe Perches <joe@perches.com>

I tested the patches and they work. Some comments below:

> ---
>  scripts/get_maintainer.pl |   37 +++++++++++++++++++++++++++++++++++--
>  1 files changed, 35 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/get_maintainer.pl b/scripts/get_maintainer.pl
> index cdb44b6..102b766 100755
> --- a/scripts/get_maintainer.pl
> +++ b/scripts/get_maintainer.pl
> @@ -13,7 +13,7 @@
>  use strict;
>  
>  my $P = $0;
> -my $V = '0.20';
> +my $V = '0.21';
>  
>  use Getopt::Long qw(:config no_auto_abbrev);
>  
> @@ -37,6 +37,7 @@ my $scm = 0;
>  my $web = 0;
>  my $subsystem = 0;
>  my $status = 0;
> +my $keywords = 1;
>  my $from_filename = 0;
>  my $pattern_depth = 0;
>  my $version = 0;
> @@ -84,6 +85,7 @@ if (!GetOptions(
>  		'scm!' => \$scm,
>  		'web!' => \$web,
>  		'pattern-depth=i' => \$pattern_depth,
> +		'k|keywords!' => \$keywords,
>  		'f|file' => \$from_filename,
>  		'v|version' => \$version,
>  		'h|help' => \$help,
> @@ -132,6 +134,8 @@ if (!top_of_kernel_tree($lk_path)) {
>  ## Read MAINTAINERS for type/value pairs
>  
>  my @typevalue = ();
> +my %keyword_hash;
> +
>  open(MAINT, "<${lk_path}MAINTAINERS") || die "$P: Can't open MAINTAINERS\n";
>  while (<MAINT>) {
>      my $line = $_;
> @@ -149,6 +153,8 @@ while (<MAINT>) {
>  	    if ((-d $value)) {
>  		$value =~ s@([^/])$@$1/@;
>  	    }
> +	} elsif ($type eq "K") {
> +	    $keyword_hash{@typevalue} = $value;

It matches the way how the current implementation works, but I guess somewhen
this @typevalue-array should be converted into a hash like

	$types{$type} = $value;

or similar unless I missed something.

>  	}
>  	push(@typevalue, "$type:$value");
>      } elsif (!/^(\s)*$/) {
> @@ -188,6 +194,7 @@ if ($email_remove_duplicates) {
>  
>  my @files = ();
>  my @range = ();
> +my @keyword_tvi = ();
>  
>  foreach my $file (@ARGV) {
>      ##if $file is a directory and it lacks a trailing slash, add one
> @@ -198,11 +205,24 @@ foreach my $file (@ARGV) {
>      }
>      if ($from_filename) {
>  	push(@files, $file);
> +	if (-f $file && $keywords) {
> +	    open(FILE, "<$file") or die "$P: Can't open ${file}\n";
> +	    while (<FILE>) {

Maybe the file should be read completely to save the loop.

> +		my $patch_line = $_;
> +		foreach my $line (keys %keyword_hash) {

This variable should be named $key instead of $line IMHO.

> +		    if ($patch_line =~ m/^.*$keyword_hash{$line}/x) {
> +			push(@keyword_tvi, $line);
> +		    }
> +		}
> +	    }
> +	    close(FILE);
> +	}
>      } else {
>  	my $file_cnt = @files;
>  	my $lastfile;
>  	open(PATCH, "<$file") or die "$P: Can't open ${file}\n";
>  	while (<PATCH>) {
> +	    my $patch_line = $_;
>  	    if (m/^\+\+\+\s+(\S+)/) {
>  		my $filename = $1;
>  		$filename =~ s@^[^/]*/@@;
> @@ -213,6 +233,12 @@ foreach my $file (@ARGV) {
>  		if ($email_git_blame) {
>  		    push(@range, "$lastfile:$1:$2");
>  		}
> +	    } elsif ($keywords) {
> +		foreach my $line (keys %keyword_hash) {

ditto

> +		    if ($patch_line =~ m/^[+-].*$keyword_hash{$line}/x) {
> +			push(@keyword_tvi, $line);
> +		    }
> +		}
>  	    }
>  	}
>  	close(PATCH);
> @@ -286,6 +312,13 @@ foreach my $file (@files) {
>      }
>  }
>  
> +if ($keywords) {
> +    @keyword_tvi = sort_and_uniq(@keyword_tvi);
> +    foreach my $line (@keyword_tvi) {
> +	add_categories($line);
> +    }
> +}
> +

A bit unrelated: I noticed that lkml gets always added due to "F: *". Maybe
this list should be skipped if there has been another mailing list found?

>  if ($email) {
>      foreach my $chief (@penguin_chief) {
>  	if ($chief =~ m/^(.*):(.*)/) {
> @@ -384,6 +417,7 @@ Output type options:
>  
>  Other options:
>    --pattern-depth => Number of pattern directory traversals (default: 0 (all))
> +  --keywords => scan patch for keywords (default: 1 (on))
>    --version => show version
>    --help => show this help information
>  
> @@ -486,7 +520,6 @@ sub format_email {
>  }
>  
>  sub find_starting_index {
> -
>      my ($index) = @_;
>  
>      while ($index > 0) {
> -- 
> 1.6.5.rc3.dirty
> 

-- 
Pengutronix e.K.                           | Wolfram Sang                |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [PATCH 1/3] scripts/get_maintainer.pl: Add patch/file search for keywords

[Joe Perches]

On Mon, 2009-10-19 at 11:41 +0200, Wolfram Sang wrote:
> > @@ -149,6 +153,8 @@ while (<MAINT>) {
> >  	    if ((-d $value)) {
> >  		$value =~ s@([^/])$@$1/@;
> >  	    }
> > +	} elsif ($type eq "K") {
> > +	    $keyword_hash{@typevalue} = $value;
> 
> It matches the way how the current implementation works, but I guess somewhen
> this @typevalue-array should be converted into a hash like
>	$types{$type} = $value;
> or similar unless I missed something.

I think you missed something.

What this is doing is saving the typevalue index of the
section where the keyword is specified.  That index is
used on the type match to add maintainers.

> Maybe the file should be read completely to save the loop.

I see if that matters.

> A bit unrelated: I noticed that lkml gets always added due to "F: *". Maybe
> this list should be skipped if there has been another mailing list found?

I think that get_maintainers is generally used by people
that aren't already sure who should get an email about 
a patch or file.  I think lkml should be cc'd in almost
every case when someone is not already sure where to
send something.

cheers, Joe

[PATCH 2/3] MAINTAINERS: Document new "K:" entry type

[Joe Perches]

K: is for keyword.  Syntax is perl extended regex.

Reorganized header documentation and indent the section
entry descriptions so that the first K: would not be
considered a regex to match by get_maintainer.pl

Signed-off-by: Joe Perches <joe@perches.com>
---
 MAINTAINERS |   70 ++++++++++++++++++++++++++++++++--------------------------
 1 files changed, 39 insertions(+), 31 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index e1da925..4407de2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -65,43 +65,51 @@ trivial patch so apply some common sense.
 
 8.	Happy hacking.
 
-		-----------------------------------
-
-Maintainers List (try to look for most precise areas first)
+Descriptions of section entries:
+
+	P: Person (obsolete)
+	M: Mail patches to: FullName <address@domain>
+	L: Mailing list that is relevant to this area
+	W: Web-page with status/info
+	T: SCM tree type and location.  Type is one of: git, hg, quilt, stgit.
+	S: Status, one of the following:
+	   Supported:	Someone is actually paid to look after this.
+	   Maintained:	Someone actually looks after it.
+	   Odd Fixes:	It has a maintainer but they don't have time to do
+			much other than throw the odd patch in. See below..
+	   Orphan:	No current maintainer [but maybe you could take the
+			role as you write your new code].
+	   Obsolete:	Old code. Something tagged obsolete generally means
+			it has been replaced by a better system and you
+			should be using that.
+	F: Files and directories with wildcard patterns.
+	   A trailing slash includes all files and subdirectory files.
+	   F:	drivers/net/	all files in and below drivers/net
+	   F:	drivers/net/*	all files in drivers/net, but not below
+	   F:	*/net/*		all files in "any top level directory"/net
+	   One pattern per line.  Multiple F: lines acceptable.
+	X: Files and directories that are NOT maintained, same rules as F:
+	   Files exclusions are tested before file matches.
+	   Can be useful for excluding a specific subdirectory, for instance:
+	   F:	net/
+	   X:	net/ipv6/
+	   matches all files in and below net excluding net/ipv6/
+	K: Keyword perl extended regex pattern to match content in a
+	   patch or file.  For instance:
+	   K: of_get_profile
+	      matches patches or files that contain "of_get_profile"
+	   K: \b(printk|pr_(info|err))\b
+	      matches patches or files that contain one or more of the words
+	      printk, pr_info or pr_err
+	   One regex pattern per line.  Multiple K: lines acceptable.
 
 Note: For the hard of thinking, this list is meant to remain in alphabetical
 order. If you could add yourselves to it in alphabetical order that would be
 so much easier [Ed]
 
-P: Person (obsolete)
-M: Mail patches to: FullName <address@domain>
-L: Mailing list that is relevant to this area
-W: Web-page with status/info
-T: SCM tree type and location.  Type is one of: git, hg, quilt, stgit.
-S: Status, one of the following:
-
-	Supported:	Someone is actually paid to look after this.
-	Maintained:	Someone actually looks after it.
-	Odd Fixes:	It has a maintainer but they don't have time to do
-			much other than throw the odd patch in. See below..
-	Orphan:		No current maintainer [but maybe you could take the
-			role as you write your new code].
-	Obsolete:	Old code. Something tagged obsolete generally means
-			it has been replaced by a better system and you
-			should be using that.
+Maintainers List (try to look for most precise areas first)
 
-F: Files and directories with wildcard patterns.
-   A trailing slash includes all files and subdirectory files.
-	F:	drivers/net/	all files in and below drivers/net
-	F:	drivers/net/*	all files in drivers/net, but not below
-	F:	*/net/*		all files in "any top level directory"/net
-   One pattern per line.  Multiple F: lines acceptable.
-X: Files and directories that are NOT maintained, same rules as F:
-   Files exclusions are tested before file matches.
-   Can be useful for excluding a specific subdirectory, for instance:
-	F:	net/
-	X:	net/ipv6/
-   matches all files in and below net excluding net/ipv6/
+		-----------------------------------
 
 3C505 NETWORK DRIVER
 M:	Philip Blundell <philb@gnu.org>
-- 
1.6.5.rc3.dirty

Re: [PATCH 2/3] MAINTAINERS: Document new "K:" entry type

[Jiri Kosina]

On Fri, 9 Oct 2009, Joe Perches wrote:

> diff --git a/MAINTAINERS b/MAINTAINERS
> index e1da925..4407de2 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -65,43 +65,51 @@ trivial patch so apply some common sense.
>  
>  8.	Happy hacking.
>  
> -		-----------------------------------
> -
> -Maintainers List (try to look for most precise areas first)
> +Descriptions of section entries:
[ ... ]
> +	K: Keyword perl extended regex pattern to match content in a
> +	   patch or file.  For instance:
> +	   K: of_get_profile
> +	      matches patches or files that contain "of_get_profile"
> +	   K: \b(printk|pr_(info|err))\b
> +	      matches patches or files that contain one or more of the words
> +	      printk, pr_info or pr_err
> +	   One regex pattern per line.  Multiple K: lines acceptable.

What expressions do you expect people will be using here? (the example 
with printk() doesn't really demonstrate usefulness of this).

-- 
Jiri Kosina
SUSE Labs, Novell Inc.

Re: [PATCH 2/3] MAINTAINERS: Document new "K:" entry type

[Joe Perches]

On Mon, 2009-10-12 at 12:14 +0200, Jiri Kosina wrote:
> On Fri, 9 Oct 2009, Joe Perches wrote:
> > +	K: Keyword perl extended regex pattern to match content in a
> > +	   patch or file.  For instance:
> > +	   K: of_get_profile
> > +	      matches patches or files that contain "of_get_profile"
> > +	   K: \b(printk|pr_(info|err))\b
> > +	      matches patches or files that contain one or more of the words
> > +	      printk, pr_info or pr_err
> What expressions do you expect people will be using here?

Keywords or keyword initiators specific to a facility
that's used kernel-wide.

> (the example with printk() doesn't really demonstrate usefulness of this).

A useful, simple example would be cheerfully substituted.
maybe:
	K: \b(AUDIT|audit)_

[PATCH 3/3] MAINTAINERS: Add OPEN FIRMWARE DEVICE TREE section

[Joe Perches]

Added L: mailing list, W: wiki and K: of_get_property keyword

Signed-off-by: Joe Perches <joe@perches.com>
---
 MAINTAINERS |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 4407de2..a88ea6b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3884,6 +3884,12 @@ S:	Maintained
 F:	Documentation/i2c/busses/i2c-ocores
 F:	drivers/i2c/busses/i2c-ocores.c
 
+OPEN FIRMWARE DEVICE TREE
+L:	devicetree-discuss@lists.ozlabs.org
+W:	http://fdt.secretlab.ca/Main_Page
+S:	Odd Fixes
+K:	of_get_property
+
 OPROFILE
 M:	Robert Richter <robert.richter@amd.com>
 L:	oprofile-list@lists.sf.net
-- 
1.6.5.rc3.dirty

Re: [PATCH 3/3] MAINTAINERS: Add OPEN FIRMWARE DEVICE TREE section

[Grant Likely]

On Sat, Oct 10, 2009 at 12:05 AM, Joe Perches <joe@perches.com> wrote:
> Added L: mailing list, W: wiki and K: of_get_property keyword
>
> Signed-off-by: Joe Perches <joe@perches.com>

NACK.  But only because I've also got a conflicting patch that adds
the same entry (without the W or K tags).  I'll merge this with that
and post right now.  :-)

g.

> ---
>  MAINTAINERS |    6 ++++++
>  1 files changed, 6 insertions(+), 0 deletions(-)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 4407de2..a88ea6b 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -3884,6 +3884,12 @@ S:       Maintained
>  F:     Documentation/i2c/busses/i2c-ocores
>  F:     drivers/i2c/busses/i2c-ocores.c
>
> +OPEN FIRMWARE DEVICE TREE
> +L:     devicetree-discuss@lists.ozlabs.org
> +W:     http://fdt.secretlab.ca/Main_Page
> +S:     Odd Fixes
> +K:     of_get_property
-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.