* [PATCH v2 02/16] ABI: Fix KernelVersion tags
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 03/16] scripts: add an script to parse the ABI files Mauro Carvalho Chehab
` (14 subsequent siblings)
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Kees Cook, Anton Vorontsov, Colin Cross,
Tony Luck, Andrew Donnellan, Rafael J. Wysocki, Rajat Jain,
Bjorn Helgaas
It is "KernelVersion:" and not "Kernel Version:".
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/ABI/testing/pstore | 2 +-
.../sysfs-bus-event_source-devices-format | 2 +-
.../ABI/testing/sysfs-bus-i2c-devices-hm6352 | 6 ++---
.../testing/sysfs-bus-pci-devices-aer_stats | 12 +++++-----
.../ABI/testing/sysfs-bus-pci-devices-cciss | 22 +++++++++----------
.../testing/sysfs-bus-usb-devices-usbsevseg | 10 ++++-----
.../ABI/testing/sysfs-driver-altera-cvp | 2 +-
Documentation/ABI/testing/sysfs-driver-ppi | 2 +-
Documentation/ABI/testing/sysfs-driver-st | 2 +-
Documentation/ABI/testing/sysfs-driver-wacom | 2 +-
10 files changed, 31 insertions(+), 31 deletions(-)
diff --git a/Documentation/ABI/testing/pstore b/Documentation/ABI/testing/pstore
index 8d6e48f4e8ef..d45209abdb1b 100644
--- a/Documentation/ABI/testing/pstore
+++ b/Documentation/ABI/testing/pstore
@@ -1,6 +1,6 @@
What: /sys/fs/pstore/... (or /dev/pstore/...)
Date: March 2011
-Kernel Version: 2.6.39
+KernelVersion: 2.6.39
Contact: tony.luck@intel.com
Description: Generic interface to platform dependent persistent storage.
diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-format b/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
index b6f8748e0200..5bb793ec926c 100644
--- a/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
+++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
@@ -1,6 +1,6 @@
What: /sys/bus/event_source/devices/<dev>/format
Date: January 2012
-Kernel Version: 3.3
+KernelVersion: 3.3
Contact: Jiri Olsa <jolsa@redhat.com>
Description:
Attribute group to describe the magic bits that go into
diff --git a/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 b/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
index 29bd447e50a0..4a251b7f11e4 100644
--- a/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
+++ b/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
@@ -1,20 +1,20 @@
What: /sys/bus/i2c/devices/.../heading0_input
Date: April 2010
-Kernel Version: 2.6.36?
+KernelVersion: 2.6.36?
Contact: alan.cox@intel.com
Description: Reports the current heading from the compass as a floating
point value in degrees.
What: /sys/bus/i2c/devices/.../power_state
Date: April 2010
-Kernel Version: 2.6.36?
+KernelVersion: 2.6.36?
Contact: alan.cox@intel.com
Description: Sets the power state of the device. 0 sets the device into
sleep mode, 1 wakes it up.
What: /sys/bus/i2c/devices/.../calibration
Date: April 2010
-Kernel Version: 2.6.36?
+KernelVersion: 2.6.36?
Contact: alan.cox@intel.com
Description: Sets the calibration on or off (1 = on, 0 = off). See the
chip data sheet.
diff --git a/Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats b/Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats
index ff229d71961c..3c9a8c4a25eb 100644
--- a/Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats
+++ b/Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats
@@ -11,7 +11,7 @@ saw any problems).
What: /sys/bus/pci/devices/<dev>/aer_dev_correctable
Date: July 2018
-Kernel Version: 4.19.0
+KernelVersion: 4.19.0
Contact: linux-pci@vger.kernel.org, rajatja@google.com
Description: List of correctable errors seen and reported by this
PCI device using ERR_COR. Note that since multiple errors may
@@ -33,7 +33,7 @@ TOTAL_ERR_COR 2
What: /sys/bus/pci/devices/<dev>/aer_dev_fatal
Date: July 2018
-Kernel Version: 4.19.0
+KernelVersion: 4.19.0
Contact: linux-pci@vger.kernel.org, rajatja@google.com
Description: List of uncorrectable fatal errors seen and reported by this
PCI device using ERR_FATAL. Note that since multiple errors may
@@ -64,7 +64,7 @@ TOTAL_ERR_FATAL 0
What: /sys/bus/pci/devices/<dev>/aer_dev_nonfatal
Date: July 2018
-Kernel Version: 4.19.0
+KernelVersion: 4.19.0
Contact: linux-pci@vger.kernel.org, rajatja@google.com
Description: List of uncorrectable nonfatal errors seen and reported by this
PCI device using ERR_NONFATAL. Note that since multiple errors
@@ -105,18 +105,18 @@ messages on the PCI hierarchy originating at that root port.
What: /sys/bus/pci/devices/<dev>/aer_stats/aer_rootport_total_err_cor
Date: July 2018
-Kernel Version: 4.19.0
+KernelVersion: 4.19.0
Contact: linux-pci@vger.kernel.org, rajatja@google.com
Description: Total number of ERR_COR messages reported to rootport.
What: /sys/bus/pci/devices/<dev>/aer_stats/aer_rootport_total_err_fatal
Date: July 2018
-Kernel Version: 4.19.0
+KernelVersion: 4.19.0
Contact: linux-pci@vger.kernel.org, rajatja@google.com
Description: Total number of ERR_FATAL messages reported to rootport.
What: /sys/bus/pci/devices/<dev>/aer_stats/aer_rootport_total_err_nonfatal
Date: July 2018
-Kernel Version: 4.19.0
+KernelVersion: 4.19.0
Contact: linux-pci@vger.kernel.org, rajatja@google.com
Description: Total number of ERR_NONFATAL messages reported to rootport.
diff --git a/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss b/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
index eb449169c30b..92a94e1068c2 100644
--- a/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
+++ b/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
@@ -1,68 +1,68 @@
What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/model
Date: March 2009
-Kernel Version: 2.6.30
+KernelVersion: 2.6.30
Contact: iss_storagedev@hp.com
Description: Displays the SCSI INQUIRY page 0 model for logical drive
Y of controller X.
What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/rev
Date: March 2009
-Kernel Version: 2.6.30
+KernelVersion: 2.6.30
Contact: iss_storagedev@hp.com
Description: Displays the SCSI INQUIRY page 0 revision for logical
drive Y of controller X.
What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/unique_id
Date: March 2009
-Kernel Version: 2.6.30
+KernelVersion: 2.6.30
Contact: iss_storagedev@hp.com
Description: Displays the SCSI INQUIRY page 83 serial number for logical
drive Y of controller X.
What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/vendor
Date: March 2009
-Kernel Version: 2.6.30
+KernelVersion: 2.6.30
Contact: iss_storagedev@hp.com
Description: Displays the SCSI INQUIRY page 0 vendor for logical drive
Y of controller X.
What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/block:cciss!cXdY
Date: March 2009
-Kernel Version: 2.6.30
+KernelVersion: 2.6.30
Contact: iss_storagedev@hp.com
Description: A symbolic link to /sys/block/cciss!cXdY
What: /sys/bus/pci/devices/<dev>/ccissX/rescan
Date: August 2009
-Kernel Version: 2.6.31
+KernelVersion: 2.6.31
Contact: iss_storagedev@hp.com
Description: Kicks of a rescan of the controller to discover logical
drive topology changes.
What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/lunid
Date: August 2009
-Kernel Version: 2.6.31
+KernelVersion: 2.6.31
Contact: iss_storagedev@hp.com
Description: Displays the 8-byte LUN ID used to address logical
drive Y of controller X.
What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/raid_level
Date: August 2009
-Kernel Version: 2.6.31
+KernelVersion: 2.6.31
Contact: iss_storagedev@hp.com
Description: Displays the RAID level of logical drive Y of
controller X.
What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/usage_count
Date: August 2009
-Kernel Version: 2.6.31
+KernelVersion: 2.6.31
Contact: iss_storagedev@hp.com
Description: Displays the usage count (number of opens) of logical drive Y
of controller X.
What: /sys/bus/pci/devices/<dev>/ccissX/resettable
Date: February 2011
-Kernel Version: 2.6.38
+KernelVersion: 2.6.38
Contact: iss_storagedev@hp.com
Description: Value of 1 indicates the controller can honor the reset_devices
kernel parameter. Value of 0 indicates reset_devices cannot be
@@ -73,7 +73,7 @@ Description: Value of 1 indicates the controller can honor the reset_devices
What: /sys/bus/pci/devices/<dev>/ccissX/transport_mode
Date: July 2011
-Kernel Version: 3.0
+KernelVersion: 3.0
Contact: iss_storagedev@hp.com
Description: Value of "simple" indicates that the controller has been placed
in "simple mode". Value of "performant" indicates that the
diff --git a/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg b/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
index f6199b314196..9ade80f81f96 100644
--- a/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
+++ b/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
@@ -1,6 +1,6 @@
What: /sys/bus/usb/.../powered
Date: August 2008
-Kernel Version: 2.6.26
+KernelVersion: 2.6.26
Contact: Harrison Metzger <harrisonmetz@gmail.com>
Description: Controls whether the device's display will powered.
A value of 0 is off and a non-zero value is on.
@@ -8,7 +8,7 @@ Description: Controls whether the device's display will powered.
What: /sys/bus/usb/.../mode_msb
What: /sys/bus/usb/.../mode_lsb
Date: August 2008
-Kernel Version: 2.6.26
+KernelVersion: 2.6.26
Contact: Harrison Metzger <harrisonmetz@gmail.com>
Description: Controls the devices display mode.
For a 6 character display the values are
@@ -18,7 +18,7 @@ Description: Controls the devices display mode.
What: /sys/bus/usb/.../textmode
Date: August 2008
-Kernel Version: 2.6.26
+KernelVersion: 2.6.26
Contact: Harrison Metzger <harrisonmetz@gmail.com>
Description: Controls the way the device interprets its text buffer.
raw: each character controls its segment manually
@@ -27,13 +27,13 @@ Description: Controls the way the device interprets its text buffer.
What: /sys/bus/usb/.../text
Date: August 2008
-Kernel Version: 2.6.26
+KernelVersion: 2.6.26
Contact: Harrison Metzger <harrisonmetz@gmail.com>
Description: The text (or data) for the device to display
What: /sys/bus/usb/.../decimals
Date: August 2008
-Kernel Version: 2.6.26
+KernelVersion: 2.6.26
Contact: Harrison Metzger <harrisonmetz@gmail.com>
Description: Controls the decimal places on the device.
To set the nth decimal place, give this field
diff --git a/Documentation/ABI/testing/sysfs-driver-altera-cvp b/Documentation/ABI/testing/sysfs-driver-altera-cvp
index 8cde64a71edb..fbd8078fd7ad 100644
--- a/Documentation/ABI/testing/sysfs-driver-altera-cvp
+++ b/Documentation/ABI/testing/sysfs-driver-altera-cvp
@@ -1,6 +1,6 @@
What: /sys/bus/pci/drivers/altera-cvp/chkcfg
Date: May 2017
-Kernel Version: 4.13
+KernelVersion: 4.13
Contact: Anatolij Gustschin <agust@denx.de>
Description:
Contains either 1 or 0 and controls if configuration
diff --git a/Documentation/ABI/testing/sysfs-driver-ppi b/Documentation/ABI/testing/sysfs-driver-ppi
index 9921ef285899..1a56fc507689 100644
--- a/Documentation/ABI/testing/sysfs-driver-ppi
+++ b/Documentation/ABI/testing/sysfs-driver-ppi
@@ -1,6 +1,6 @@
What: /sys/class/tpm/tpmX/ppi/
Date: August 2012
-Kernel Version: 3.6
+KernelVersion: 3.6
Contact: xiaoyan.zhang@intel.com
Description:
This folder includes the attributes related with PPI (Physical
diff --git a/Documentation/ABI/testing/sysfs-driver-st b/Documentation/ABI/testing/sysfs-driver-st
index ba5d77008a85..88cab66fd77f 100644
--- a/Documentation/ABI/testing/sysfs-driver-st
+++ b/Documentation/ABI/testing/sysfs-driver-st
@@ -1,6 +1,6 @@
What: /sys/bus/scsi/drivers/st/debug_flag
Date: October 2015
-Kernel Version: ?.?
+KernelVersion: ?.?
Contact: shane.seymour@hpe.com
Description:
This file allows you to turn debug output from the st driver
diff --git a/Documentation/ABI/testing/sysfs-driver-wacom b/Documentation/ABI/testing/sysfs-driver-wacom
index 2aa5503ee200..afc48fc163b5 100644
--- a/Documentation/ABI/testing/sysfs-driver-wacom
+++ b/Documentation/ABI/testing/sysfs-driver-wacom
@@ -1,6 +1,6 @@
What: /sys/bus/hid/devices/<bus>:<vid>:<pid>.<n>/speed
Date: April 2010
-Kernel Version: 2.6.35
+KernelVersion: 2.6.35
Contact: linux-bluetooth@vger.kernel.org
Description:
The /sys/bus/hid/devices/<bus>:<vid>:<pid>.<n>/speed file
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 03/16] scripts: add an script to parse the ABI files
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 02/16] ABI: Fix KernelVersion tags Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-17 12:57 ` Greg Kroah-Hartman
2019-06-14 17:52 ` [PATCH v2 04/16] scripts/get_abi.pl: parse files with text at beginning Mauro Carvalho Chehab
` (13 subsequent siblings)
15 siblings, 1 reply; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
Add a script to parse the Documentation/ABI files and produce
an output with all entries inside an ABI (sub)directory.
Right now, it outputs its contents on ReST format. It shouldn't
be hard to make it produce other kind of outputs, since the ABI
file parser is implemented in separate than the output generator.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 212 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 212 insertions(+)
create mode 100755 scripts/get_abi.pl
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
new file mode 100755
index 000000000000..f7c9944a833c
--- /dev/null
+++ b/scripts/get_abi.pl
@@ -0,0 +1,212 @@
+#!/usr/bin/perl
+
+use strict;
+use Pod::Usage;
+use Getopt::Long;
+use File::Find;
+use Fcntl ':mode';
+
+my $help;
+my $man;
+my $debug;
+
+GetOptions(
+ "debug|d+" => \$debug,
+ 'help|?' => \$help,
+ man => \$man
+) or pod2usage(2);
+
+pod2usage(1) if $help;
+pod2usage(-exitstatus => 0, -verbose => 2) if $man;
+
+pod2usage(2) if (scalar @ARGV != 1);
+
+my ($prefix) = @ARGV;
+
+require Data::Dumper if ($debug);
+
+my %data;
+
+#
+# Displays an error message, printing file name and line
+#
+sub parse_error($$$$) {
+ my ($file, $ln, $msg, $data) = @_;
+
+ print STDERR "file $file#$ln: $msg at\n\t$data";
+}
+
+#
+# Parse an ABI file, storing its contents at %data
+#
+sub parse_abi {
+ my $file = $File::Find::name;
+
+ my $mode = (stat($file))[2];
+ return if ($mode & S_IFDIR);
+ return if ($file =~ m,/README,);
+
+ my $name = $file;
+ $name =~ s,.*/,,;
+
+ my $type = $file;
+ $type =~ s,.*/(.*)/.*,$1,;
+
+ my $what;
+ my $new_what;
+ my $tag;
+ my $ln;
+
+ print STDERR "Opening $file\n" if ($debug > 1);
+ open IN, $file;
+ while(<IN>) {
+ $ln++;
+ if (m/^(\S+):\s*(.*)/i) {
+ my $new_tag = lc($1);
+ my $content = $2;
+
+ if (!($new_tag =~ m/(what|date|kernelversion|contact|description|users)/)) {
+ if ($tag eq "description") {
+ $data{$what}->{$tag} .= "\n$content";;
+ $data{$what}->{$tag} =~ s/\n+$//;
+ next;
+ } else {
+ parse_error($file, $ln, "tag '$tag' is invalid", $_);
+ }
+ }
+
+ if ($new_tag =~ m/what/) {
+ if ($tag =~ m/what/) {
+ $what .= ", " . $content;
+ } else {
+ $what = $content;
+ $new_what = 1;
+ }
+ $tag = $new_tag;
+ next;
+ }
+
+ $tag = $new_tag;
+
+ if ($new_what) {
+ $new_what = 0;
+
+ $data{$what}->{type} = $type;
+ $data{$what}->{file} = $name;
+ print STDERR "\twhat: $what\n" if ($debug > 1);
+ }
+
+ if (!$what) {
+ parse_error($file, $ln, "'What:' should come first:", $_);
+ next;
+ }
+ $data{$what}->{$tag} = $content;
+ next;
+ }
+
+ # Silently ignore any headers before the database
+ next if (!$tag);
+
+ if (m/^\s*(.*)/) {
+ $data{$what}->{$tag} .= "\n$1";
+ $data{$what}->{$tag} =~ s/\n+$//;
+ next;
+ }
+
+ # Everything else is error
+ parse_error($file, $ln, "Unexpected line:", $_);
+ }
+ close IN;
+}
+
+# Outputs the output on ReST format
+sub output_rest {
+ foreach my $what (sort keys %data) {
+ my $type = $data{$what}->{type};
+ my $file = $data{$what}->{file};
+
+ my $w = $what;
+ $w =~ s/([\(\)\_\-\*\=\^\~\\])/\\$1/g;
+
+ print "$w\n\n";
+ print "- defined on file $file (type: $type)\n\n::\n\n";
+
+ my $desc = $data{$what}->{description};
+ $desc =~ s/^\s+//;
+
+ # Remove title markups from the description, as they won't work
+ $desc =~ s/\n[\-\*\=\^\~]+\n/\n/g;
+
+ # put everything inside a code block
+ $desc =~ s/\n/\n /g;
+
+
+ if (!($desc =~ /^\s*$/)) {
+ print " $desc\n\n";
+ } else {
+ print " DESCRIPTION MISSING\n\n";
+ }
+ }
+}
+
+#
+# Parses all ABI files located at $prefix dir
+#
+find({wanted =>\&parse_abi, no_chdir => 1}, $prefix);
+
+print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug);
+
+#
+# Outputs an ReST file with the ABI contents
+#
+output_rest
+
+
+__END__
+
+=head1 NAME
+
+abi_book.pl - parse the Linux ABI files and produce a ReST book.
+
+=head1 SYNOPSIS
+
+B<abi_book.pl> [--debug] <ABI_DIR>]
+
+=head1 OPTIONS
+
+=over 8
+
+=item B<--debug>
+
+Put the script in verbose mode, useful for debugging. Can be called multiple
+times, to increase verbosity.
+
+=item B<--help>
+
+Prints a brief help message and exits.
+
+=item B<--man>
+
+Prints the manual page and exits.
+
+=back
+
+=head1 DESCRIPTION
+
+Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI)
+and produce a ReST book containing the Linux ABI.
+
+=head1 BUGS
+
+Report bugs to Mauro Carvalho Chehab <mchehab@s-opensource.com>
+
+=head1 COPYRIGHT
+
+Copyright (c) 2016 by Mauro Carvalho Chehab <mchehab@s-opensource.com>.
+
+License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
+
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law.
+
+=cut
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH v2 03/16] scripts: add an script to parse the ABI files
2019-06-14 17:52 ` [PATCH v2 03/16] scripts: add an script to parse the ABI files Mauro Carvalho Chehab
@ 2019-06-17 12:57 ` Greg Kroah-Hartman
0 siblings, 0 replies; 19+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-17 12:57 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
On Fri, Jun 14, 2019 at 02:52:17PM -0300, Mauro Carvalho Chehab wrote:
> Add a script to parse the Documentation/ABI files and produce
> an output with all entries inside an ABI (sub)directory.
>
> Right now, it outputs its contents on ReST format. It shouldn't
> be hard to make it produce other kind of outputs, since the ABI
> file parser is implemented in separate than the output generator.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> ---
> scripts/get_abi.pl | 212 +++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 212 insertions(+)
> create mode 100755 scripts/get_abi.pl
>
> diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
> new file mode 100755
> index 000000000000..f7c9944a833c
> --- /dev/null
> +++ b/scripts/get_abi.pl
> @@ -0,0 +1,212 @@
> +#!/usr/bin/perl
> +
Ok, I was going to apply this, but there is no SPDX line on the script.
Can you resend this series with that on it, so that I can apply the
patches of the series that adds the script to the kernel tree?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v2 04/16] scripts/get_abi.pl: parse files with text at beginning
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 02/16] ABI: Fix KernelVersion tags Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 03/16] scripts: add an script to parse the ABI files Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 05/16] scripts/get_abi.pl: avoid use literal blocks when not needed Mauro Carvalho Chehab
` (12 subsequent siblings)
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
It sounds usefult o parse files with has some text at the
beginning. Add support for it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 59 ++++++++++++++++++++++++++++++++++++++++++----
1 file changed, 54 insertions(+), 5 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index f7c9944a833c..ef9b6e108973 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -55,7 +55,10 @@ sub parse_abi {
my $what;
my $new_what;
my $tag;
+ my $label;
my $ln;
+ my $has_file;
+ my $xrefs;
print STDERR "Opening $file\n" if ($debug > 1);
open IN, $file;
@@ -67,7 +70,7 @@ sub parse_abi {
if (!($new_tag =~ m/(what|date|kernelversion|contact|description|users)/)) {
if ($tag eq "description") {
- $data{$what}->{$tag} .= "\n$content";;
+ $data{$what}->{$tag} .= "\n$content";
$data{$what}->{$tag} =~ s/\n+$//;
next;
} else {
@@ -83,6 +86,25 @@ sub parse_abi {
$new_what = 1;
}
$tag = $new_tag;
+
+ if ($has_file) {
+ $label = "abi_" . $content . " ";
+ $label =~ tr/A-Z/a-z/;
+
+ # Convert special chars to "_"
+ $label =~s/[\x00-\x2f]+/_/g;
+ $label =~s/[\x3a-\x40]+/_/g;
+ $label =~s/[\x7b-\xff]+/_/g;
+ $label =~ s,_+,_,g;
+ $label =~ s,_$,,;
+
+ $data{$what}->{label} .= $label;
+
+ # Escape special chars from content
+ $content =~s/([\x00-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])/\\$1/g;
+
+ $xrefs .= "- :ref:`$content <$label>`\n\n";
+ }
next;
}
@@ -104,8 +126,18 @@ sub parse_abi {
next;
}
- # Silently ignore any headers before the database
- next if (!$tag);
+ # Store any contents before the database
+ if (!$tag) {
+ next if (/^\n/);
+
+ my $my_what = "File $name";
+ $data{$my_what}->{what} = "File $name";
+ $data{$my_what}->{type} = "File";
+ $data{$my_what}->{file} = $name;
+ $data{$my_what}->{description} .= $_;
+ $has_file = 1;
+ next;
+ }
if (m/^\s*(.*)/) {
$data{$what}->{$tag} .= "\n$1";
@@ -117,6 +149,11 @@ sub parse_abi {
parse_error($file, $ln, "Unexpected line:", $_);
}
close IN;
+
+ if ($has_file) {
+ my $my_what = "File $name";
+ $data{$my_what}->{xrefs} = $xrefs;
+ }
}
# Outputs the output on ReST format
@@ -128,8 +165,17 @@ sub output_rest {
my $w = $what;
$w =~ s/([\(\)\_\-\*\=\^\~\\])/\\$1/g;
+ if ($data{$what}->{label}) {
+ my @labels = split(/\s/, $data{$what}->{label});
+ foreach my $label (@labels) {
+ printf ".. _%s:\n\n", $label;
+ }
+ }
+
print "$w\n\n";
- print "- defined on file $file (type: $type)\n\n::\n\n";
+
+ print "- defined on file $file (type: $type)\n\n" if ($type ne "File");
+ print "::\n\n";
my $desc = $data{$what}->{description};
$desc =~ s/^\s+//;
@@ -144,8 +190,11 @@ sub output_rest {
if (!($desc =~ /^\s*$/)) {
print " $desc\n\n";
} else {
- print " DESCRIPTION MISSING\n\n";
+ print " DESCRIPTION MISSING for $what\n\n";
}
+
+ printf "Has the following ABI:\n\n%s", $data{$what}->{xrefs} if ($data{$what}->{xrefs});
+
}
}
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 05/16] scripts/get_abi.pl: avoid use literal blocks when not needed
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (2 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 04/16] scripts/get_abi.pl: parse files with text at beginning Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 06/16] scripts/get_abi.pl: split label naming from xref logic Mauro Carvalho Chehab
` (11 subsequent siblings)
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
The usage of literal blocks make the document very complex,
causing the browser to take a long time to load.
On most ABI descriptions, they're a plain text, and don't
require a literal block.
So, add a logic there with identifies when a literal block
is needed.
As, on literal blocks, we need to respect the original
document space, the most complex part of this patch is
to preserve the original spacing where needed.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 104 +++++++++++++++++++++++++++++++++------------
1 file changed, 78 insertions(+), 26 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index ef9b6e108973..0ede9593c639 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -59,29 +59,34 @@ sub parse_abi {
my $ln;
my $has_file;
my $xrefs;
+ my $space;
print STDERR "Opening $file\n" if ($debug > 1);
open IN, $file;
while(<IN>) {
$ln++;
- if (m/^(\S+):\s*(.*)/i) {
+ if (m/^(\S+)(:\s*)(.*)/i) {
my $new_tag = lc($1);
- my $content = $2;
+ my $sep = $2;
+ my $content = $3;
if (!($new_tag =~ m/(what|date|kernelversion|contact|description|users)/)) {
if ($tag eq "description") {
- $data{$what}->{$tag} .= "\n$content";
- $data{$what}->{$tag} =~ s/\n+$//;
- next;
+ # New "tag" is actually part of
+ # description. Don't consider it a tag
+ $new_tag = "";
} else {
parse_error($file, $ln, "tag '$tag' is invalid", $_);
}
}
if ($new_tag =~ m/what/) {
+ $space = "";
if ($tag =~ m/what/) {
$what .= ", " . $content;
} else {
+ parse_error($file, $ln, "What '$what' doesn't have a description", "") if ($what && !$data{$what}->{description});
+
$what = $content;
$new_what = 1;
}
@@ -108,25 +113,38 @@ sub parse_abi {
next;
}
- $tag = $new_tag;
+ if ($new_tag) {
+ $tag = $new_tag;
- if ($new_what) {
- $new_what = 0;
+ if ($new_what) {
+ $new_what = 0;
- $data{$what}->{type} = $type;
- $data{$what}->{file} = $name;
- print STDERR "\twhat: $what\n" if ($debug > 1);
- }
+ $data{$what}->{type} = $type;
+ $data{$what}->{file} = $name;
+ print STDERR "\twhat: $what\n" if ($debug > 1);
+ }
- if (!$what) {
- parse_error($file, $ln, "'What:' should come first:", $_);
+ if (!$what) {
+ parse_error($file, $ln, "'What:' should come first:", $_);
+ next;
+ }
+ if ($tag eq "description") {
+ next if ($content =~ m/^\s*$/);
+ if ($content =~ m/^(\s*)(.*)/) {
+ my $new_content = $2;
+ $space = $new_tag . $sep . $1;
+ while ($space =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
+ $space =~ s/./ /g;
+ $data{$what}->{$tag} .= "$new_content\n";
+ }
+ } else {
+ $data{$what}->{$tag} = $content;
+ }
next;
}
- $data{$what}->{$tag} = $content;
- next;
}
- # Store any contents before the database
+ # Store any contents before tags at the database
if (!$tag) {
next if (/^\n/);
@@ -139,6 +157,32 @@ sub parse_abi {
next;
}
+ if ($tag eq "description") {
+ if (!$data{$what}->{description}) {
+ next if (m/^\s*\n/);
+ if (m/^(\s*)(.*)/) {
+ $space = $1;
+ while ($space =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
+ $data{$what}->{$tag} .= "$2\n";
+ }
+ } else {
+ my $content = $_;
+ if (m/^\s*\n/) {
+ $data{$what}->{$tag} .= $content;
+ next;
+ }
+
+ while ($content =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
+ $space = "" if (!($content =~ s/^($space)//));
+
+ # Compress spaces with tabs
+ $content =~ s<^ {8}> <\t>;
+ $content =~ s<^ {1,7}\t> <\t>;
+ $content =~ s< {1,7}\t> <\t>;
+ $data{$what}->{$tag} .= $content;
+ }
+ next;
+ }
if (m/^\s*(.*)/) {
$data{$what}->{$tag} .= "\n$1";
$data{$what}->{$tag} =~ s/\n+$//;
@@ -165,6 +209,9 @@ sub output_rest {
my $w = $what;
$w =~ s/([\(\)\_\-\*\=\^\~\\])/\\$1/g;
+ my $bar = $w;
+ $bar =~ s/./-/g;
+
if ($data{$what}->{label}) {
my @labels = split(/\s/, $data{$what}->{label});
foreach my $label (@labels) {
@@ -172,10 +219,9 @@ sub output_rest {
}
}
- print "$w\n\n";
+ print "$w\n$bar\n\n";
print "- defined on file $file (type: $type)\n\n" if ($type ne "File");
- print "::\n\n";
my $desc = $data{$what}->{description};
$desc =~ s/^\s+//;
@@ -183,18 +229,24 @@ sub output_rest {
# Remove title markups from the description, as they won't work
$desc =~ s/\n[\-\*\=\^\~]+\n/\n/g;
- # put everything inside a code block
- $desc =~ s/\n/\n /g;
-
-
if (!($desc =~ /^\s*$/)) {
- print " $desc\n\n";
+ if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
+ # put everything inside a code block
+ $desc =~ s/\n/\n /g;
+
+ print "::\n\n";
+ print " $desc\n\n";
+ } else {
+ # Escape any special chars from description
+ $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
+
+ print "$desc\n\n";
+ }
} else {
- print " DESCRIPTION MISSING for $what\n\n";
+ print "DESCRIPTION MISSING for $what\n\n";
}
printf "Has the following ABI:\n\n%s", $data{$what}->{xrefs} if ($data{$what}->{xrefs});
-
}
}
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 06/16] scripts/get_abi.pl: split label naming from xref logic
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (3 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 05/16] scripts/get_abi.pl: avoid use literal blocks when not needed Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 07/16] scripts/get_abi.pl: add support for searching for ABI symbols Mauro Carvalho Chehab
` (10 subsequent siblings)
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
Instead of using a ReST compilant label while parsing,
move the label to ReST output. That makes the parsing logic
more generic, allowing it to provide other types of output.
As a side effect, now all files used to generate the output
will be output. We can later add command line arguments to
filter.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 94 ++++++++++++++++++++++++++--------------------
1 file changed, 53 insertions(+), 41 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index 0ede9593c639..f84d321a72bb 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -49,17 +49,23 @@ sub parse_abi {
my $name = $file;
$name =~ s,.*/,,;
+ my $nametag = "File $name";
+ $data{$nametag}->{what} = "File $name";
+ $data{$nametag}->{type} = "File";
+ $data{$nametag}->{file} = $name;
+ $data{$nametag}->{is_file} = 1;
+
my $type = $file;
$type =~ s,.*/(.*)/.*,$1,;
my $what;
my $new_what;
my $tag;
- my $label;
my $ln;
- my $has_file;
my $xrefs;
my $space;
+ my @labels;
+ my $label;
print STDERR "Opening $file\n" if ($debug > 1);
open IN, $file;
@@ -88,28 +94,13 @@ sub parse_abi {
parse_error($file, $ln, "What '$what' doesn't have a description", "") if ($what && !$data{$what}->{description});
$what = $content;
+ $label = $content;
$new_what = 1;
}
+ push @labels, [($content, $label)];
$tag = $new_tag;
- if ($has_file) {
- $label = "abi_" . $content . " ";
- $label =~ tr/A-Z/a-z/;
-
- # Convert special chars to "_"
- $label =~s/[\x00-\x2f]+/_/g;
- $label =~s/[\x3a-\x40]+/_/g;
- $label =~s/[\x7b-\xff]+/_/g;
- $label =~ s,_+,_,g;
- $label =~ s,_$,,;
-
- $data{$what}->{label} .= $label;
-
- # Escape special chars from content
- $content =~s/([\x00-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])/\\$1/g;
-
- $xrefs .= "- :ref:`$content <$label>`\n\n";
- }
+ push @{$data{$nametag}->{xrefs}}, [($content, $label)] if ($data{$nametag}->{what});
next;
}
@@ -117,6 +108,9 @@ sub parse_abi {
$tag = $new_tag;
if ($new_what) {
+ @{$data{$what}->{label}} = @labels if ($data{$nametag}->{what});
+ @labels = ();
+ $label = "";
$new_what = 0;
$data{$what}->{type} = $type;
@@ -145,15 +139,8 @@ sub parse_abi {
}
# Store any contents before tags at the database
- if (!$tag) {
- next if (/^\n/);
-
- my $my_what = "File $name";
- $data{$my_what}->{what} = "File $name";
- $data{$my_what}->{type} = "File";
- $data{$my_what}->{file} = $name;
- $data{$my_what}->{description} .= $_;
- $has_file = 1;
+ if (!$tag && $data{$nametag}->{what}) {
+ $data{$nametag}->{description} .= $_;
next;
}
@@ -192,12 +179,8 @@ sub parse_abi {
# Everything else is error
parse_error($file, $ln, "Unexpected line:", $_);
}
+ $data{$nametag}->{description} =~ s/^\n+//;
close IN;
-
- if ($has_file) {
- my $my_what = "File $name";
- $data{$my_what}->{xrefs} = $xrefs;
- }
}
# Outputs the output on ReST format
@@ -212,11 +195,22 @@ sub output_rest {
my $bar = $w;
$bar =~ s/./-/g;
- if ($data{$what}->{label}) {
- my @labels = split(/\s/, $data{$what}->{label});
- foreach my $label (@labels) {
- printf ".. _%s:\n\n", $label;
- }
+ foreach my $p (@{$data{$what}->{label}}) {
+ my ($content, $label) = @{$p};
+ $label = "abi_" . $label . " ";
+ $label =~ tr/A-Z/a-z/;
+
+ # Convert special chars to "_"
+ $label =~s/([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xff])/_/g;
+ $label =~ s,_+,_,g;
+ $label =~ s,_$,,;
+
+ $data{$what}->{label} .= $label;
+
+ printf ".. _%s:\n\n", $label;
+
+ # only one label is enough
+ last;
}
print "$w\n$bar\n\n";
@@ -243,10 +237,28 @@ sub output_rest {
print "$desc\n\n";
}
} else {
- print "DESCRIPTION MISSING for $what\n\n";
+ print "DESCRIPTION MISSING for $what\n\n" if (!$data{$what}->{is_file});
}
- printf "Has the following ABI:\n\n%s", $data{$what}->{xrefs} if ($data{$what}->{xrefs});
+ if ($data{$what}->{xrefs}) {
+ printf "Has the following ABI:\n\n";
+
+ foreach my $p(@{$data{$what}->{xrefs}}) {
+ my ($content, $label) = @{$p};
+ $label = "abi_" . $label . " ";
+ $label =~ tr/A-Z/a-z/;
+
+ # Convert special chars to "_"
+ $label =~s/([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xff])/_/g;
+ $label =~ s,_+,_,g;
+ $label =~ s,_$,,;
+
+ # Escape special chars from content
+ $content =~s/([\x00-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])/\\$1/g;
+
+ print "- :ref:`$content <$label>`\n\n";
+ }
+ }
}
}
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 07/16] scripts/get_abi.pl: add support for searching for ABI symbols
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (4 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 06/16] scripts/get_abi.pl: split label naming from xref logic Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 08/16] scripts/get_abi.pl: represent what in tables Mauro Carvalho Chehab
` (9 subsequent siblings)
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
Change its syntax to allow switching between ReST output mode
and a new search mode, with allows to seek for ABI symbols
using regex.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 112 +++++++++++++++++++++++++++++++++++++++++----
1 file changed, 103 insertions(+), 9 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index f84d321a72bb..ecf6b91df7c4 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -9,9 +9,11 @@ use Fcntl ':mode';
my $help;
my $man;
my $debug;
+my $prefix="Documentation/ABI";
GetOptions(
"debug|d+" => \$debug,
+ "dir=s" => \$prefix,
'help|?' => \$help,
man => \$man
) or pod2usage(2);
@@ -19,9 +21,12 @@ GetOptions(
pod2usage(1) if $help;
pod2usage(-exitstatus => 0, -verbose => 2) if $man;
-pod2usage(2) if (scalar @ARGV != 1);
+pod2usage(2) if (scalar @ARGV < 1 || @ARGV > 2);
-my ($prefix) = @ARGV;
+my ($cmd, $arg) = @ARGV;
+
+pod2usage(2) if ($cmd ne "search" && $cmd ne "rest");
+pod2usage(2) if ($cmd eq "search" && !$arg);
require Data::Dumper if ($debug);
@@ -53,6 +58,7 @@ sub parse_abi {
$data{$nametag}->{what} = "File $name";
$data{$nametag}->{type} = "File";
$data{$nametag}->{file} = $name;
+ $data{$nametag}->{filepath} = $file;
$data{$nametag}->{is_file} = 1;
my $type = $file;
@@ -115,6 +121,7 @@ sub parse_abi {
$data{$what}->{type} = $type;
$data{$what}->{file} = $name;
+ $data{$what}->{filepath} = $file;
print STDERR "\twhat: $what\n" if ($debug > 1);
}
@@ -183,7 +190,9 @@ sub parse_abi {
close IN;
}
-# Outputs the output on ReST format
+#
+# Outputs the book on ReST format
+#
sub output_rest {
foreach my $what (sort keys %data) {
my $type = $data{$what}->{type};
@@ -262,6 +271,45 @@ sub output_rest {
}
}
+#
+# Searches for ABI symbols
+#
+sub search_symbols {
+ foreach my $what (sort keys %data) {
+ next if (!($what =~ m/($arg)/));
+
+ my $type = $data{$what}->{type};
+ next if ($type eq "File");
+
+ my $file = $data{$what}->{filepath};
+
+ my $bar = $what;
+ $bar =~ s/./-/g;
+
+ print "\n$what\n$bar\n\n";
+
+ my $kernelversion = $data{$what}->{kernelversion};
+ my $contact = $data{$what}->{contact};
+ my $users = $data{$what}->{users};
+ my $date = $data{$what}->{date};
+ my $desc = $data{$what}->{description};
+ $kernelversion =~ s/^\s+//;
+ $contact =~ s/^\s+//;
+ $users =~ s/^\s+//;
+ $users =~ s/\n//g;
+ $date =~ s/^\s+//;
+ $desc =~ s/^\s+//;
+
+ printf "Kernel version:\t\t%s\n", $kernelversion if ($kernelversion);
+ printf "Date:\t\t\t%s\n", $date if ($date);
+ printf "Contact:\t\t%s\n", $contact if ($contact);
+ printf "Users:\t\t\t%s\n", $users if ($users);
+ print "Defined on file:\t$file\n\n";
+ print "Description:\n\n$desc";
+ }
+}
+
+
#
# Parses all ABI files located at $prefix dir
#
@@ -270,9 +318,13 @@ find({wanted =>\&parse_abi, no_chdir => 1}, $prefix);
print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug);
#
-# Outputs an ReST file with the ABI contents
+# Handles the command
#
-output_rest
+if ($cmd eq "rest") {
+ output_rest;
+} else {
+ search_symbols;
+}
__END__
@@ -283,12 +335,27 @@ abi_book.pl - parse the Linux ABI files and produce a ReST book.
=head1 SYNOPSIS
-B<abi_book.pl> [--debug] <ABI_DIR>]
+B<abi_book.pl> [--debug] <COMAND> [<ARGUMENT>]
+
+Where <COMMAND> can be:
+
+=over 8
+
+B<search> [SEARCH_REGEX] - search for [SEARCH_REGEX] inside ABI
+
+B<rest> - output the ABI in ReST markup language
+
+=back
=head1 OPTIONS
=over 8
+=item B<--dir>
+
+Changes the location of the ABI search. By default, it uses
+the Documentation/ABI directory.
+
=item B<--debug>
Put the script in verbose mode, useful for debugging. Can be called multiple
@@ -306,8 +373,35 @@ Prints the manual page and exits.
=head1 DESCRIPTION
-Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI)
-and produce a ReST book containing the Linux ABI.
+Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI),
+allowing to search for ABI symbols or to produce a ReST book containing
+the Linux ABI documentation.
+
+=head1 EXAMPLES
+
+Search for all stable symbols with the word "usb":
+
+=over 8
+
+$ scripts/get_abi.pl search usb --dir Documentation/ABI/stable
+
+=back
+
+Search for all symbols that match the regex expression "usb.*cap":
+
+=over 8
+
+$ scripts/get_abi.pl search usb.*cap
+
+=back
+
+Output all obsoleted symbols in ReST format
+
+=over 8
+
+$ scripts/get_abi.pl rest --dir Documentation/ABI/obsolete
+
+=back
=head1 BUGS
@@ -315,7 +409,7 @@ Report bugs to Mauro Carvalho Chehab <mchehab@s-opensource.com>
=head1 COPYRIGHT
-Copyright (c) 2016 by Mauro Carvalho Chehab <mchehab@s-opensource.com>.
+Copyright (c) 2016-2017 by Mauro Carvalho Chehab <mchehab@s-opensource.com>.
License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 08/16] scripts/get_abi.pl: represent what in tables
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (5 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 07/16] scripts/get_abi.pl: add support for searching for ABI symbols Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 09/16] scripts/get_abi.pl: fix parse issues with some files Mauro Carvalho Chehab
` (8 subsequent siblings)
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
Several entries at the ABI have multiple What: with the same
description.
Instead of showing those symbols as sections, let's show them
as tables. That makes easier to read on the final output,
and avoid too much recursion at Sphinx parsing.
We need to put file references at the end, as we don't want
non-file tables to be mangled with other entries.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 41 ++++++++++++++++++++++++++++++++++++-----
1 file changed, 36 insertions(+), 5 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index ecf6b91df7c4..7d454e359d25 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -193,16 +193,19 @@ sub parse_abi {
#
# Outputs the book on ReST format
#
+
sub output_rest {
- foreach my $what (sort keys %data) {
+ foreach my $what (sort {
+ ($data{$a}->{type} eq "File") cmp ($data{$b}->{type} eq "File") ||
+ $a cmp $b
+ } keys %data) {
my $type = $data{$what}->{type};
my $file = $data{$what}->{file};
+ my $filepath = $data{$what}->{filepath};
my $w = $what;
$w =~ s/([\(\)\_\-\*\=\^\~\\])/\\$1/g;
- my $bar = $w;
- $bar =~ s/./-/g;
foreach my $p (@{$data{$what}->{label}}) {
my ($content, $label) = @{$p};
@@ -222,9 +225,37 @@ sub output_rest {
last;
}
- print "$w\n$bar\n\n";
- print "- defined on file $file (type: $type)\n\n" if ($type ne "File");
+ $filepath =~ s,.*/(.*/.*),\1,;;
+ $filepath =~ s,[/\-],_,g;;
+ my $fileref = "abi_file_".$filepath;
+
+ if ($type eq "File") {
+ my $bar = $w;
+ $bar =~ s/./-/g;
+
+ print ".. _$fileref:\n\n";
+ print "$w\n$bar\n\n";
+ } else {
+ my @names = split /\s*,\s*/,$w;
+
+ my $len = 0;
+
+ foreach my $name (@names) {
+ $len = length($name) if (length($name) > $len);
+ }
+
+ print "What:\n\n";
+
+ print "+-" . "-" x $len . "-+\n";
+ foreach my $name (@names) {
+ printf "| %s", $name . " " x ($len - length($name)) . " |\n";
+ print "+-" . "-" x $len . "-+\n";
+ }
+ print "\n";
+ }
+
+ print "Defined on file :ref:`$file <$fileref>`\n\n" if ($type ne "File");
my $desc = $data{$what}->{description};
$desc =~ s/^\s+//;
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 09/16] scripts/get_abi.pl: fix parse issues with some files
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (6 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 08/16] scripts/get_abi.pl: represent what in tables Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 10/16] scripts/get_abi.pl: avoid creating duplicate names Mauro Carvalho Chehab
` (7 subsequent siblings)
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
A few files are failing to parse:
Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats
Documentation/ABI/testing/sysfs-class-pktcdvd
Documentation/ABI/testing/sysfs-bus-nfit
On all three files, the problem is that there is a ":" character
at the initial file description.
Improve the parse in order to handle those special cases.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index 7d454e359d25..116f0c33c16d 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -87,7 +87,7 @@ sub parse_abi {
# New "tag" is actually part of
# description. Don't consider it a tag
$new_tag = "";
- } else {
+ } elsif ($tag ne "") {
parse_error($file, $ln, "tag '$tag' is invalid", $_);
}
}
@@ -110,7 +110,7 @@ sub parse_abi {
next;
}
- if ($new_tag) {
+ if ($tag ne "" && $new_tag) {
$tag = $new_tag;
if ($new_what) {
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 10/16] scripts/get_abi.pl: avoid creating duplicate names
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (7 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 09/16] scripts/get_abi.pl: fix parse issues with some files Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 11/16] scripts/get_abi.pl: add a handler for invalid "where" tag Mauro Carvalho Chehab
` (6 subsequent siblings)
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
The file the Documentation/ABI/testing/sysfs-class-power has
voltage_min, voltage_max and voltage_now symbols duplicated.
They are defined first for "General Properties" and then for
"USB Properties".
This cause those warnings:
get_abi.pl rest --dir $srctree/Documentation/ABI/testing:26933: WARNING: Duplicate explicit target name: "abi_sys_class_power_supply_supply_name_voltage_max".
get_abi.pl rest --dir $srctree/Documentation/ABI/testing:26968: WARNING: Duplicate explicit target name: "abi_sys_class_power_supply_supply_name_voltage_min".
get_abi.pl rest --dir $srctree/Documentation/ABI/testing:27008: WARNING: Duplicate explicit target name: "abi_sys_class_power_supply_supply_name_voltage_now".
And, as the references are not valid, it will also generate
warnings about links to undefined references.
Fix it by storing labels into a hash table and, when a duplicated
one is found, appending random characters at the end.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index 116f0c33c16d..329ace635ac2 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -194,6 +194,8 @@ sub parse_abi {
# Outputs the book on ReST format
#
+my %labels;
+
sub output_rest {
foreach my $what (sort {
($data{$a}->{type} eq "File") cmp ($data{$b}->{type} eq "File") ||
@@ -217,6 +219,13 @@ sub output_rest {
$label =~ s,_+,_,g;
$label =~ s,_$,,;
+ # Avoid duplicated labels
+ while (defined($labels{$label})) {
+ my @chars = ("A".."Z", "a".."z");
+ $label .= $chars[rand @chars];
+ }
+ $labels{$label} = 1;
+
$data{$what}->{label} .= $label;
printf ".. _%s:\n\n", $label;
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 11/16] scripts/get_abi.pl: add a handler for invalid "where" tag
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (8 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 10/16] scripts/get_abi.pl: avoid creating duplicate names Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 12/16] scripts/get_abi.pl: add a validate command Mauro Carvalho Chehab
` (5 subsequent siblings)
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
The ABI README file doesn't provide any meaning for a Where:
tag. Yet, a few ABI symbols use it. So, make the parser
handle it, emitting a warning.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index 329ace635ac2..c5038a0a7313 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -92,6 +92,12 @@ sub parse_abi {
}
}
+ # Invalid, but it is a common mistake
+ if ($new_tag eq "where") {
+ parse_error($file, $ln, "tag 'Where' is invalid. Should be 'What:' instead", $_);
+ $new_tag = "what";
+ }
+
if ($new_tag =~ m/what/) {
$space = "";
if ($tag =~ m/what/) {
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 12/16] scripts/get_abi.pl: add a validate command
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (9 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 11/16] scripts/get_abi.pl: add a handler for invalid "where" tag Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 13/16] doc-rst: add ABI documentation to the admin-guide book Mauro Carvalho Chehab
` (4 subsequent siblings)
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
Sometimes, we just want the parser to retrieve all symbols from
ABI, in order to check for parsing errors. So, add a new
"validate" command.
While here, update the man/help pages.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 16 +++++++++-------
1 file changed, 9 insertions(+), 7 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index c5038a0a7313..774e9b809ead 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -25,7 +25,7 @@ pod2usage(2) if (scalar @ARGV < 1 || @ARGV > 2);
my ($cmd, $arg) = @ARGV;
-pod2usage(2) if ($cmd ne "search" && $cmd ne "rest");
+pod2usage(2) if ($cmd ne "search" && $cmd ne "rest" && $cmd ne "validate");
pod2usage(2) if ($cmd eq "search" && !$arg);
require Data::Dumper if ($debug);
@@ -82,7 +82,7 @@ sub parse_abi {
my $sep = $2;
my $content = $3;
- if (!($new_tag =~ m/(what|date|kernelversion|contact|description|users)/)) {
+ if (!($new_tag =~ m/(what|where|date|kernelversion|contact|description|users)/)) {
if ($tag eq "description") {
# New "tag" is actually part of
# description. Don't consider it a tag
@@ -368,7 +368,7 @@ print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug);
#
if ($cmd eq "rest") {
output_rest;
-} else {
+} elsif ($cmd eq "search") {
search_symbols;
}
@@ -381,7 +381,7 @@ abi_book.pl - parse the Linux ABI files and produce a ReST book.
=head1 SYNOPSIS
-B<abi_book.pl> [--debug] <COMAND> [<ARGUMENT>]
+B<abi_book.pl> [--debug] [--man] [--help] [--dir=<dir>] <COMAND> [<ARGUMENT>]
Where <COMMAND> can be:
@@ -389,7 +389,9 @@ Where <COMMAND> can be:
B<search> [SEARCH_REGEX] - search for [SEARCH_REGEX] inside ABI
-B<rest> - output the ABI in ReST markup language
+B<rest> - output the ABI in ReST markup language
+
+B<validate> - validate the ABI contents
=back
@@ -451,11 +453,11 @@ $ scripts/get_abi.pl rest --dir Documentation/ABI/obsolete
=head1 BUGS
-Report bugs to Mauro Carvalho Chehab <mchehab@s-opensource.com>
+Report bugs to Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
=head1 COPYRIGHT
-Copyright (c) 2016-2017 by Mauro Carvalho Chehab <mchehab@s-opensource.com>.
+Copyright (c) 2016-2019 by Mauro Carvalho Chehab <mchehab+samsung@kernel.org>.
License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 13/16] doc-rst: add ABI documentation to the admin-guide book
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (10 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 12/16] scripts/get_abi.pl: add a validate command Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 14/16] docs: sphinx/kernel_abi.py: fix UTF-8 support Mauro Carvalho Chehab
` (3 subsequent siblings)
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Thomas Gleixner, Theodore Ts'o,
Alexey Budankov, Changbin Du, Darrick J. Wong
As we don't want a generic Sphinx extension to execute commands,
change the one proposed to Markus to call the abi_book.pl
script.
Use a script to parse the Documentation/ABI directory and output
it at the admin-guide.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/admin-guide/abi-obsolete.rst | 10 ++
Documentation/admin-guide/abi-removed.rst | 4 +
Documentation/admin-guide/abi-stable.rst | 13 ++
Documentation/admin-guide/abi-testing.rst | 19 +++
Documentation/admin-guide/abi.rst | 11 ++
Documentation/admin-guide/index.rst | 1 +
Documentation/conf.py | 2 +-
Documentation/sphinx/kernel_abi.py | 155 +++++++++++++++++++++
8 files changed, 214 insertions(+), 1 deletion(-)
create mode 100644 Documentation/admin-guide/abi-obsolete.rst
create mode 100644 Documentation/admin-guide/abi-removed.rst
create mode 100644 Documentation/admin-guide/abi-stable.rst
create mode 100644 Documentation/admin-guide/abi-testing.rst
create mode 100644 Documentation/admin-guide/abi.rst
create mode 100644 Documentation/sphinx/kernel_abi.py
diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst
new file mode 100644
index 000000000000..cda9168445a5
--- /dev/null
+++ b/Documentation/admin-guide/abi-obsolete.rst
@@ -0,0 +1,10 @@
+ABI obsolete symbols
+====================
+
+Documents interfaces that are still remaining in the kernel, but are
+marked to be removed at some later point in time.
+
+The description of the interface will document the reason why it is
+obsolete and when it can be expected to be removed.
+
+.. kernel-abi:: $srctree/Documentation/ABI/obsolete
diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst
new file mode 100644
index 000000000000..497978fc9632
--- /dev/null
+++ b/Documentation/admin-guide/abi-removed.rst
@@ -0,0 +1,4 @@
+ABI removed symbols
+===================
+
+.. kernel-abi:: $srctree/Documentation/ABI/removed
diff --git a/Documentation/admin-guide/abi-stable.rst b/Documentation/admin-guide/abi-stable.rst
new file mode 100644
index 000000000000..7495d7a35048
--- /dev/null
+++ b/Documentation/admin-guide/abi-stable.rst
@@ -0,0 +1,13 @@
+ABI stable symbols
+==================
+
+Documents the interfaces that the developer has defined to be stable.
+
+Userspace programs are free to use these interfaces with no
+restrictions, and backward compatibility for them will be guaranteed
+for at least 2 years.
+
+Most interfaces (like syscalls) are expected to never change and always
+be available.
+
+.. kernel-abi:: $srctree/Documentation/ABI/stable
diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst
new file mode 100644
index 000000000000..5c886fc50b9e
--- /dev/null
+++ b/Documentation/admin-guide/abi-testing.rst
@@ -0,0 +1,19 @@
+ABI testing symbols
+===================
+
+Documents interfaces that are felt to be stable,
+as the main development of this interface has been completed.
+
+The interface can be changed to add new features, but the
+current interface will not break by doing this, unless grave
+errors or security problems are found in them.
+
+Userspace programs can start to rely on these interfaces, but they must
+be aware of changes that can occur before these interfaces move to
+be marked stable.
+
+Programs that use these interfaces are strongly encouraged to add their
+name to the description of these interfaces, so that the kernel
+developers can easily notify them if any changes occur.
+
+.. kernel-abi:: $srctree/Documentation/ABI/testing
diff --git a/Documentation/admin-guide/abi.rst b/Documentation/admin-guide/abi.rst
new file mode 100644
index 000000000000..3b9645c77469
--- /dev/null
+++ b/Documentation/admin-guide/abi.rst
@@ -0,0 +1,11 @@
+=====================
+Linux ABI description
+=====================
+
+.. toctree::
+ :maxdepth: 1
+
+ abi-stable
+ abi-testing
+ abi-obsolete
+ abi-removed
diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guide/index.rst
index 8001917ee012..20c3020fd73c 100644
--- a/Documentation/admin-guide/index.rst
+++ b/Documentation/admin-guide/index.rst
@@ -16,6 +16,7 @@ etc.
README
kernel-parameters
devices
+ abi
This section describes CPU vulnerabilities and their mitigations.
diff --git a/Documentation/conf.py b/Documentation/conf.py
index 7ace3f8852bd..598256fb5c98 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -34,7 +34,7 @@ needs_sphinx = '1.3'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig']
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig', 'kernel_abi']
# The name of the math extension changed on Sphinx 1.4
if (major == 1 and minor > 3) or (major > 1):
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
new file mode 100644
index 000000000000..32ce90775d96
--- /dev/null
+++ b/Documentation/sphinx/kernel_abi.py
@@ -0,0 +1,155 @@
+# -*- coding: utf-8; mode: python -*-
+u"""
+ kernel-abi
+ ~~~~~~~~~~
+
+ Implementation of the ``kernel-abi`` reST-directive.
+
+ :copyright: Copyright (C) 2016 Markus Heiser
+ :copyright: Copyright (C) 2016 Mauro Carvalho Chehab
+ :license: GPL Version 2, June 1991 see Linux/COPYING for details.
+
+ The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
+ scripts/get_abi.pl script to parse the Kernel ABI files.
+
+ Overview of directive's argument and options.
+
+ .. code-block:: rst
+
+ .. kernel-abi:: <ABI directory location>
+ :debug:
+
+ The argument ``<ABI directory location>`` is required. It contains the
+ location of the ABI files to be parsed.
+
+ ``debug``
+ Inserts a code-block with the *raw* reST. Sometimes it is helpful to see
+ what reST is generated.
+
+"""
+
+import sys
+import os
+from os import path
+import subprocess
+
+from sphinx.ext.autodoc import AutodocReporter
+
+from docutils import nodes
+from docutils.parsers.rst import Directive, directives
+from docutils.statemachine import ViewList
+from docutils.utils.error_reporting import ErrorString
+
+
+__version__ = '1.0'
+
+# We can't assume that six is installed
+PY3 = sys.version_info[0] == 3
+PY2 = sys.version_info[0] == 2
+if PY3:
+ # pylint: disable=C0103, W0622
+ unicode = str
+ basestring = str
+
+def setup(app):
+
+ app.add_directive("kernel-abi", KernelCmd)
+ return dict(
+ version = __version__
+ , parallel_read_safe = True
+ , parallel_write_safe = True
+ )
+
+class KernelCmd(Directive):
+
+ u"""KernelABI (``kernel-abi``) directive"""
+
+ required_arguments = 1
+ optional_arguments = 0
+ has_content = False
+ final_argument_whitespace = True
+
+ option_spec = {
+ "debug" : directives.flag
+ }
+
+ def warn(self, message, **replace):
+ replace["fname"] = self.state.document.current_source
+ replace["line_no"] = replace.get("line_no", self.lineno)
+ message = ("%(fname)s:%(line_no)s: [kernel-abi WARN] : " + message) % replace
+ self.state.document.settings.env.app.warn(message, prefix="")
+
+ def run(self):
+
+ doc = self.state.document
+ if not doc.settings.file_insertion_enabled:
+ raise self.warning("docutils: file insertion disabled")
+
+ env = doc.settings.env
+ cwd = path.dirname(doc.current_source)
+ cmd = "get_abi.pl rest --dir "
+ cmd += self.arguments[0]
+
+ srctree = path.abspath(os.environ["srctree"])
+
+ fname = cmd
+
+ # extend PATH with $(srctree)/scripts
+ path_env = os.pathsep.join([
+ srctree + os.sep + "scripts",
+ os.environ["PATH"]
+ ])
+ shell_env = os.environ.copy()
+ shell_env["PATH"] = path_env
+ shell_env["srctree"] = srctree
+
+ lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
+ nodeList = self.nestedParse(lines, fname)
+ return nodeList
+
+ def runCmd(self, cmd, **kwargs):
+ u"""Run command ``cmd`` and return it's stdout as unicode."""
+
+ try:
+ proc = subprocess.Popen(
+ cmd
+ , stdout = subprocess.PIPE
+ , stderr = subprocess.PIPE
+ , universal_newlines = True
+ , **kwargs
+ )
+ out, err = proc.communicate()
+ if err:
+ self.warn(err)
+ if proc.returncode != 0:
+ raise self.severe(
+ u"command '%s' failed with return code %d"
+ % (cmd, proc.returncode)
+ )
+ except OSError as exc:
+ raise self.severe(u"problems with '%s' directive: %s."
+ % (self.name, ErrorString(exc)))
+ return unicode(out)
+
+ def nestedParse(self, lines, fname):
+ content = ViewList()
+ node = nodes.section()
+
+ if "debug" in self.options:
+ code_block = "\n\n.. code-block:: rst\n :linenos:\n"
+ for l in lines.split("\n"):
+ code_block += "\n " + l
+ lines = code_block + "\n\n"
+
+ for c, l in enumerate(lines.split("\n")):
+ content.append(l, fname, c)
+
+ buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
+ self.state.memo.title_styles = []
+ self.state.memo.section_level = 0
+ self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
+ try:
+ self.state.nested_parse(content, 0, node, match_titles=1)
+ finally:
+ self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
+ return node.children
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 14/16] docs: sphinx/kernel_abi.py: fix UTF-8 support
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (11 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 13/16] doc-rst: add ABI documentation to the admin-guide book Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 15/16] sphinx/kernel_abi.py: make it compatible with Sphinx 1.7+ Mauro Carvalho Chehab
` (2 subsequent siblings)
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
The parser breaks with UTF-8 characters with Sphinx 1.4.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/sphinx/kernel_abi.py | 20 +++++++-------------
1 file changed, 7 insertions(+), 13 deletions(-)
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 32ce90775d96..0f3e51e67e8d 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -1,4 +1,5 @@
-# -*- coding: utf-8; mode: python -*-
+# coding=utf-8
+#
u"""
kernel-abi
~~~~~~~~~~
@@ -28,6 +29,7 @@ u"""
"""
+import codecs
import sys
import os
from os import path
@@ -43,14 +45,6 @@ from docutils.utils.error_reporting import ErrorString
__version__ = '1.0'
-# We can't assume that six is installed
-PY3 = sys.version_info[0] == 3
-PY2 = sys.version_info[0] == 2
-if PY3:
- # pylint: disable=C0103, W0622
- unicode = str
- basestring = str
-
def setup(app):
app.add_directive("kernel-abi", KernelCmd)
@@ -115,12 +109,12 @@ class KernelCmd(Directive):
cmd
, stdout = subprocess.PIPE
, stderr = subprocess.PIPE
- , universal_newlines = True
, **kwargs
)
out, err = proc.communicate()
- if err:
- self.warn(err)
+
+ out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
+
if proc.returncode != 0:
raise self.severe(
u"command '%s' failed with return code %d"
@@ -129,7 +123,7 @@ class KernelCmd(Directive):
except OSError as exc:
raise self.severe(u"problems with '%s' directive: %s."
% (self.name, ErrorString(exc)))
- return unicode(out)
+ return out
def nestedParse(self, lines, fname):
content = ViewList()
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 15/16] sphinx/kernel_abi.py: make it compatible with Sphinx 1.7+
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (12 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 14/16] docs: sphinx/kernel_abi.py: fix UTF-8 support Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 17:52 ` [PATCH v2 16/16] docs: Kconfig/Makefile: add a check for broken ABI files Mauro Carvalho Chehab
2019-06-14 18:07 ` [PATCH v2 00/16] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
The same way kerneldoc.py needed changes to work with newer
Sphinx, this script needs the same changes.
While here, reorganize the include order to match kerneldoc.py.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/sphinx/kernel_abi.py | 39 +++++++++++++++++++++---------
1 file changed, 27 insertions(+), 12 deletions(-)
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 0f3e51e67e8d..2d5d582207f7 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -30,18 +30,27 @@ u"""
"""
import codecs
-import sys
import os
-from os import path
import subprocess
+import sys
-from sphinx.ext.autodoc import AutodocReporter
+from os import path
-from docutils import nodes
-from docutils.parsers.rst import Directive, directives
+from docutils import nodes, statemachine
from docutils.statemachine import ViewList
+from docutils.parsers.rst import directives, Directive
from docutils.utils.error_reporting import ErrorString
+#
+# AutodocReporter is only good up to Sphinx 1.7
+#
+import sphinx
+
+Use_SSI = sphinx.__version__[:3] >= '1.7'
+if Use_SSI:
+ from sphinx.util.docutils import switch_source_input
+else:
+ from sphinx.ext.autodoc import AutodocReporter
__version__ = '1.0'
@@ -139,11 +148,17 @@ class KernelCmd(Directive):
content.append(l, fname, c)
buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
- self.state.memo.title_styles = []
- self.state.memo.section_level = 0
- self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
- try:
- self.state.nested_parse(content, 0, node, match_titles=1)
- finally:
- self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
+
+ if Use_SSI:
+ with switch_source_input(self.state, content):
+ self.state.nested_parse(content, 0, node, match_titles=1)
+ else:
+ self.state.memo.title_styles = []
+ self.state.memo.section_level = 0
+ self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
+ try:
+ self.state.nested_parse(content, 0, node, match_titles=1)
+ finally:
+ self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
+
return node.children
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 16/16] docs: Kconfig/Makefile: add a check for broken ABI files
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (13 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 15/16] sphinx/kernel_abi.py: make it compatible with Sphinx 1.7+ Mauro Carvalho Chehab
@ 2019-06-14 17:52 ` Mauro Carvalho Chehab
2019-06-14 18:07 ` [PATCH v2 00/16] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
15 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 17:52 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Andrew Morton, Masahiro Yamada, Kees Cook,
Petr Mladek, Andy Shevchenko, Matthew Wilcox, Joe Lawrence,
Mikulas Patocka, Tetsuo Handa, Sri Krishna chowdary, Changbin Du
The files under Documentation/ABI should follow the syntax
as defined at Documentation/ABI/README.
Allow checking if they're following the syntax by running
the ABI parser script on COMPILE_TEST.
With that, when there's a problem with a file under
Documentation/ABI, it would produce a warning like:
Warning: file ./Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats#14:
What '/sys/bus/pci/devices/<dev>/aer_stats/aer_rootport_total_err_cor' doesn't have a description
Warning: file ./Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats#21:
What '/sys/bus/pci/devices/<dev>/aer_stats/aer_rootport_total_err_fatal' doesn't have a description
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/Kconfig | 11 +++++++++++
Documentation/Makefile | 5 +++++
lib/Kconfig.debug | 2 ++
scripts/get_abi.pl | 14 +++++++++++---
4 files changed, 29 insertions(+), 3 deletions(-)
create mode 100644 Documentation/Kconfig
diff --git a/Documentation/Kconfig b/Documentation/Kconfig
new file mode 100644
index 000000000000..a8b0701c1422
--- /dev/null
+++ b/Documentation/Kconfig
@@ -0,0 +1,11 @@
+config WARN_ABI_ERRORS
+ bool "Warn if there are errors at ABI files"
+ depends on COMPILE_TEST
+ help
+ The files under Documentation/ABI should follow what's
+ described at Documentation/ABI/README. Yet, as they're manually
+ written, it would be possible that some of those files would
+ have errors that would break them for being parsed by
+ scripts/get_abi.pl. Add a check to verify them.
+
+ If unsure, select 'N'.
diff --git a/Documentation/Makefile b/Documentation/Makefile
index e889e7cb8511..c6480ed22884 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -4,6 +4,11 @@
subdir-y := devicetree/bindings/
+# Check for broken ABI files
+ifeq ($(CONFIG_WARN_ABI_ERRORS),y)
+$(shell $(srctree)/scripts/get_abi.pl validate --dir $(srctree)/Documentation/ABI)
+endif
+
# You can set these variables from the command line.
SPHINXBUILD = sphinx-build
SPHINXOPTS =
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index cbdfae379896..b1b7e141ca99 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -2110,4 +2110,6 @@ config IO_STRICT_DEVMEM
source "arch/$(SRCARCH)/Kconfig.debug"
+source "Documentation/Kconfig"
+
endmenu # Kernel hacking
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index 774e9b809ead..25248c012eb3 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -38,7 +38,15 @@ my %data;
sub parse_error($$$$) {
my ($file, $ln, $msg, $data) = @_;
- print STDERR "file $file#$ln: $msg at\n\t$data";
+ $data =~ s/\s+$/\n/;
+
+ print STDERR "Warning: file $file#$ln:\n\t$msg";
+
+ if ($data ne "") {
+ print STDERR ". Line\n\t\t$data";
+ } else {
+ print STDERR "\n";
+ }
}
#
@@ -94,7 +102,7 @@ sub parse_abi {
# Invalid, but it is a common mistake
if ($new_tag eq "where") {
- parse_error($file, $ln, "tag 'Where' is invalid. Should be 'What:' instead", $_);
+ parse_error($file, $ln, "tag 'Where' is invalid. Should be 'What:' instead", "");
$new_tag = "what";
}
@@ -190,7 +198,7 @@ sub parse_abi {
}
# Everything else is error
- parse_error($file, $ln, "Unexpected line:", $_);
+ parse_error($file, $ln, "Unexpected content", $_);
}
$data{$nametag}->{description} =~ s/^\n+//;
close IN;
--
2.21.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH v2 00/16] Add support to generate ABI documentation at admin-guide
2019-06-14 17:52 [PATCH v2 01/16] ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag Mauro Carvalho Chehab
` (14 preceding siblings ...)
2019-06-14 17:52 ` [PATCH v2 16/16] docs: Kconfig/Makefile: add a check for broken ABI files Mauro Carvalho Chehab
@ 2019-06-14 18:07 ` Mauro Carvalho Chehab
2019-06-15 6:15 ` Greg Kroah-Hartman
15 siblings, 1 reply; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 18:07 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet, Rajat Jain,
Bjorn Helgaas
Greg,
In time: forgot to add a 00/16 patch... someone come to my desk while I was
sending this... sorry for that.
That's the second version of the doc ABI updated logic.
Changes from version 1:
- I updated my e-mail on older patches;
- Two new ABI fix patches;
- Improved the parser to detect a few more issues I discovered on
some ABI files.
- There's now a "validate" command at get_abi.pl. It simply runs
the parser without any output, except for warnings;
- It now runs "get_abi.pl validate" during the build (if enabled).
Please notice that the last patch will conflict with another patch
merged via docs-next tree, with creates a Documentation/Kconfig and
adds a check at Documentation/Makefile for broken documentation
links. The conflict is trivial to solve, through: just add the contents
on both tress.
This series is based on your driver-core git tree.
-
On this series:
- The first two patches contain two extra ABI fixes;
- The next 10 patches contain the ABI parsing script. I don't
mind if you prefer folding them on a single patch, but IMO,
preserving the history may help tracking bugs, if any;
- The 13th patch adds the new script to the documentation build
system, together with a new python Sphinx extension with calls it;
- The 14th patch fixes the python script when running with newer
Sphinx versions (1.7 and upper);
- The 15th patch fixes an UTF-8 troubles;
- The final patch adds a Kconfig var that will check for ABI
file problems, if COMPILE_TEST and WARN_ABI_ERRORS are
enabled.
Mauro Carvalho Chehab (16):
ABI: sysfs-bus-pci-devices-aer_stats uses an invalid tag
ABI: Fix KernelVersion tags
scripts: add an script to parse the ABI files
scripts/get_abi.pl: parse files with text at beginning
scripts/get_abi.pl: avoid use literal blocks when not needed
scripts/get_abi.pl: split label naming from xref logic
scripts/get_abi.pl: add support for searching for ABI symbols
scripts/get_abi.pl: represent what in tables
scripts/get_abi.pl: fix parse issues with some files
scripts/get_abi.pl: avoid creating duplicate names
scripts/get_abi.pl: add a handler for invalid "where" tag
scripts/get_abi.pl: add a validate command
doc-rst: add ABI documentation to the admin-guide book
docs: sphinx/kernel_abi.py: fix UTF-8 support
sphinx/kernel_abi.py: make it compatible with Sphinx 1.7+
docs: Kconfig/Makefile: add a check for broken ABI files
Documentation/ABI/testing/pstore | 2 +-
.../sysfs-bus-event_source-devices-format | 2 +-
.../ABI/testing/sysfs-bus-i2c-devices-hm6352 | 6 +-
.../testing/sysfs-bus-pci-devices-aer_stats | 24 +-
.../ABI/testing/sysfs-bus-pci-devices-cciss | 22 +-
.../testing/sysfs-bus-usb-devices-usbsevseg | 10 +-
.../ABI/testing/sysfs-driver-altera-cvp | 2 +-
Documentation/ABI/testing/sysfs-driver-ppi | 2 +-
Documentation/ABI/testing/sysfs-driver-st | 2 +-
Documentation/ABI/testing/sysfs-driver-wacom | 2 +-
Documentation/Kconfig | 11 +
Documentation/Makefile | 5 +
Documentation/admin-guide/abi-obsolete.rst | 10 +
Documentation/admin-guide/abi-removed.rst | 4 +
Documentation/admin-guide/abi-stable.rst | 13 +
Documentation/admin-guide/abi-testing.rst | 19 +
Documentation/admin-guide/abi.rst | 11 +
Documentation/admin-guide/index.rst | 1 +
Documentation/conf.py | 2 +-
Documentation/sphinx/kernel_abi.py | 164 ++++++
lib/Kconfig.debug | 2 +
scripts/get_abi.pl | 475 ++++++++++++++++++
22 files changed, 753 insertions(+), 38 deletions(-)
create mode 100644 Documentation/Kconfig
create mode 100644 Documentation/admin-guide/abi-obsolete.rst
create mode 100644 Documentation/admin-guide/abi-removed.rst
create mode 100644 Documentation/admin-guide/abi-stable.rst
create mode 100644 Documentation/admin-guide/abi-testing.rst
create mode 100644 Documentation/admin-guide/abi.rst
create mode 100644 Documentation/sphinx/kernel_abi.py
create mode 100755 scripts/get_abi.pl
--
2.21.0
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH v2 00/16] Add support to generate ABI documentation at admin-guide
2019-06-14 18:07 ` [PATCH v2 00/16] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
@ 2019-06-15 6:15 ` Greg Kroah-Hartman
0 siblings, 0 replies; 19+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-15 6:15 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Rajat Jain, Bjorn Helgaas
On Fri, Jun 14, 2019 at 03:07:36PM -0300, Mauro Carvalho Chehab wrote:
> Greg,
>
> In time: forgot to add a 00/16 patch... someone come to my desk while I was
> sending this... sorry for that.
>
> That's the second version of the doc ABI updated logic.
>
> Changes from version 1:
>
> - I updated my e-mail on older patches;
>
> - Two new ABI fix patches;
Both abi fix patches now queued up, thanks.
greg k-h
^ permalink raw reply [flat|nested] 19+ messages in thread