From mboxrd@z Thu Jan 1 00:00:00 1970 From: Julia Lawall Date: Wed, 05 Oct 2016 20:25:35 +0000 Subject: Re: kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) Message-Id: MIME-Version: 1 Content-Type: multipart/mixed; boundary="8323329-1882111399-1475699148=:3130" List-Id: References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> <87h98quc1w.fsf@intel.com> In-Reply-To: <87h98quc1w.fsf@intel.com> To: Jani Nikula Cc: Daniel Vetter , Julia Lawall , Linux PM list , kernel-janitors@vger.kernel.org, Linux Kernel Mailing List , dri-devel , linux-clk@vger.kernel.org, "linaro-mm-sig@lists.linaro.org" , linux-mtd@lists.infradead.org, "linux-tegra@vger.kernel.org" , drbd-dev@lists.linbit.com, linux-metag@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , "linux-media@vger.kernel.org" , linux-doc@vger.kernel.org This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --8323329-1882111399-1475699148=:3130 Content-Type: TEXT/PLAIN; charset="utf-8" Content-Transfer-Encoding: 8bit On Wed, 5 Oct 2016, Jani Nikula wrote: > On Wed, 05 Oct 2016, Daniel Vetter wrote: > > Jani Nikula has a patch with a scrip to make the one kernel-doc parser > > into a lint/checker pass over the entire kernel. I think that'd would > > be more robust instead of trying to approximate the real kerneldoc > > parser. Otoh that parser is a horror show of a perl/regex driven state > > machine ;-) > > > > Jani, can you pls digg out these patches? Can't find them right now ... > > Expanding the massive Cc: with linux-doc list... > > Here goes. It's a quick hack from months ago, but still seems to > somewhat work. At least for the kernel-doc parts. The reStructuredText > lint part isn't all that great, and doesn't have mapping to line numbers > like the Sphinx kernel-doc extension does. Anyway I'm happy how this > integrates with kernel build CHECK and C=1/C=2. > > I guess Julia's goal is to automate the *fixing* of some of the error > classes from kernel-doc. Not sure how well this could be made to > integrate with any of that. No, my work doesn't fix anything. Coccinelle can't actually process comments. I just correlated the parsed comment with the function header. julia > > BR, > Jani. > > > From 1244efa0f63a7b13795e8c37f81733a3c8bfc56a Mon Sep 17 00:00:00 2001 > From: Jani Nikula > Date: Tue, 31 May 2016 18:11:33 +0300 > Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and rst > correctness > Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo > Cc: Jani Nikula > > Simple kernel-doc and reStructuredText lint tool that can be used > independently and as a kernel build CHECK tool to validate kernel-doc > comments. > > Independent usage: > $ kernel-doc-rst-lint FILE > > Kernel CHECK usage: > $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) > > Depends on docutils and the rst-lint package > https://pypi.python.org/pypi/restructuredtext_lint > > Signed-off-by: Jani Nikula > --- > scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 106 insertions(+) > create mode 100755 scripts/kernel-doc-rst-lint > > diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint > new file mode 100755 > index 000000000000..7e0157679f83 > --- /dev/null > +++ b/scripts/kernel-doc-rst-lint > @@ -0,0 +1,106 @@ > +#!/usr/bin/env python > +# coding=utf-8 > +# > +# Copyright © 2016 Intel Corporation > +# > +# Permission is hereby granted, free of charge, to any person obtaining a > +# copy of this software and associated documentation files (the "Software"), > +# to deal in the Software without restriction, including without limitation > +# the rights to use, copy, modify, merge, publish, distribute, sublicense, > +# and/or sell copies of the Software, and to permit persons to whom the > +# Software is furnished to do so, subject to the following conditions: > +# > +# The above copyright notice and this permission notice (including the next > +# paragraph) shall be included in all copies or substantial portions of the > +# Software. > +# > +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL > +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS > +# IN THE SOFTWARE. > +# > +# Authors: > +# Jani Nikula > +# > +# Simple kernel-doc and reStructuredText lint tool that can be used > +# independently and as a kernel build CHECK tool to validate kernel-doc > +# comments. > +# > +# Independent usage: > +# $ kernel-doc-rst-lint FILE > +# > +# Kernel CHECK usage: > +# $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) > +# > +# Depends on docutils and the rst-lint package > +# https://pypi.python.org/pypi/restructuredtext_lint > +# > + > +import os > +import subprocess > +import sys > + > +from docutils.parsers.rst import directives > +from docutils.parsers.rst import Directive > +from docutils.parsers.rst import roles > +from docutils import nodes, statemachine > +import restructuredtext_lint > + > +class DummyDirective(Directive): > + required_argument = 1 > + optional_arguments = 0 > + option_spec = { } > + has_content = True > + > + def run(self): > + return [] > + > +# Fake the Sphinx C Domain directives and roles > +directives.register_directive('c:function', DummyDirective) > +directives.register_directive('c:type', DummyDirective) > +roles.register_generic_role('c:func', nodes.emphasis) > +roles.register_generic_role('c:type', nodes.emphasis) > + > +# We accept but ignore parameters to be compatible with how the kernel build > +# invokes CHECK. > +if len(sys.argv) < 2: > + sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FILE\n'); > + sys.exit(1) > + > +infile = sys.argv[len(sys.argv) - 1] > +cmd = ['scripts/kernel-doc', '-rst', infile] > + > +try: > + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) > + out, err = p.communicate() > + > + # python2 needs conversion to unicode. > + # python3 with universal_newlines=True returns strings. > + if sys.version_info.major < 3: > + out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8') > + > + # kernel-doc errors > + sys.stderr.write(err) > + if p.returncode != 0: > + sys.exit(p.returncode) > + > + # restructured text errors > + lines = statemachine.string2lines(out, 8, convert_whitespace=True) > + lint_errors = restructuredtext_lint.lint(out, infile) > + for error in lint_errors: > + # Ignore INFO > + if error.level <= 1: > + continue > + > + print(error.source + ': ' + error.type + ': ' + error.full_message) > + if error.line is not None: > + print('Context:') > + print('\t' + lines[error.line - 1]) > + print('\t' + lines[error.line]) > + > +except Exception as e: > + sys.stderr.write(str(e) + '\n') > + sys.exit(1) > -- > 2.1.4 > > > -- > Jani Nikula, Intel Open Source Technology Center > -- > To unsubscribe from this list: send the line "unsubscribe kernel-janitors" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > --8323329-1882111399-1475699148=:3130-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:34452 "EHLO mail2-relais-roc.national.inria.fr" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752784AbcJEUZw (ORCPT ); Wed, 5 Oct 2016 16:25:52 -0400 Date: Wed, 5 Oct 2016 22:25:35 +0200 (CEST) From: Julia Lawall To: Jani Nikula cc: Daniel Vetter , Julia Lawall , Linux PM list , kernel-janitors@vger.kernel.org, Linux Kernel Mailing List , dri-devel , linux-clk@vger.kernel.org, "linaro-mm-sig@lists.linaro.org" , linux-mtd@lists.infradead.org, "linux-tegra@vger.kernel.org" , drbd-dev@lists.linbit.com, linux-metag@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , "linux-media@vger.kernel.org" , linux-doc@vger.kernel.org Subject: Re: kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) In-Reply-To: <87h98quc1w.fsf@intel.com> Message-ID: References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> <87h98quc1w.fsf@intel.com> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="8323329-1882111399-1475699148=:3130" Sender: linux-clk-owner@vger.kernel.org List-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --8323329-1882111399-1475699148=:3130 Content-Type: TEXT/PLAIN; charset=utf-8 Content-Transfer-Encoding: 8BIT On Wed, 5 Oct 2016, Jani Nikula wrote: > On Wed, 05 Oct 2016, Daniel Vetter wrote: > > Jani Nikula has a patch with a scrip to make the one kernel-doc parser > > into a lint/checker pass over the entire kernel. I think that'd would > > be more robust instead of trying to approximate the real kerneldoc > > parser. Otoh that parser is a horror show of a perl/regex driven state > > machine ;-) > > > > Jani, can you pls digg out these patches? Can't find them right now ... > > Expanding the massive Cc: with linux-doc list... > > Here goes. It's a quick hack from months ago, but still seems to > somewhat work. At least for the kernel-doc parts. The reStructuredText > lint part isn't all that great, and doesn't have mapping to line numbers > like the Sphinx kernel-doc extension does. Anyway I'm happy how this > integrates with kernel build CHECK and C=1/C=2. > > I guess Julia's goal is to automate the *fixing* of some of the error > classes from kernel-doc. Not sure how well this could be made to > integrate with any of that. No, my work doesn't fix anything. Coccinelle can't actually process comments. I just correlated the parsed comment with the function header. julia > > BR, > Jani. > > > From 1244efa0f63a7b13795e8c37f81733a3c8bfc56a Mon Sep 17 00:00:00 2001 > From: Jani Nikula > Date: Tue, 31 May 2016 18:11:33 +0300 > Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and rst > correctness > Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo > Cc: Jani Nikula > > Simple kernel-doc and reStructuredText lint tool that can be used > independently and as a kernel build CHECK tool to validate kernel-doc > comments. > > Independent usage: > $ kernel-doc-rst-lint FILE > > Kernel CHECK usage: > $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) > > Depends on docutils and the rst-lint package > https://pypi.python.org/pypi/restructuredtext_lint > > Signed-off-by: Jani Nikula > --- > scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 106 insertions(+) > create mode 100755 scripts/kernel-doc-rst-lint > > diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint > new file mode 100755 > index 000000000000..7e0157679f83 > --- /dev/null > +++ b/scripts/kernel-doc-rst-lint > @@ -0,0 +1,106 @@ > +#!/usr/bin/env python > +# coding=utf-8 > +# > +# Copyright © 2016 Intel Corporation > +# > +# Permission is hereby granted, free of charge, to any person obtaining a > +# copy of this software and associated documentation files (the "Software"), > +# to deal in the Software without restriction, including without limitation > +# the rights to use, copy, modify, merge, publish, distribute, sublicense, > +# and/or sell copies of the Software, and to permit persons to whom the > +# Software is furnished to do so, subject to the following conditions: > +# > +# The above copyright notice and this permission notice (including the next > +# paragraph) shall be included in all copies or substantial portions of the > +# Software. > +# > +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL > +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS > +# IN THE SOFTWARE. > +# > +# Authors: > +# Jani Nikula > +# > +# Simple kernel-doc and reStructuredText lint tool that can be used > +# independently and as a kernel build CHECK tool to validate kernel-doc > +# comments. > +# > +# Independent usage: > +# $ kernel-doc-rst-lint FILE > +# > +# Kernel CHECK usage: > +# $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) > +# > +# Depends on docutils and the rst-lint package > +# https://pypi.python.org/pypi/restructuredtext_lint > +# > + > +import os > +import subprocess > +import sys > + > +from docutils.parsers.rst import directives > +from docutils.parsers.rst import Directive > +from docutils.parsers.rst import roles > +from docutils import nodes, statemachine > +import restructuredtext_lint > + > +class DummyDirective(Directive): > + required_argument = 1 > + optional_arguments = 0 > + option_spec = { } > + has_content = True > + > + def run(self): > + return [] > + > +# Fake the Sphinx C Domain directives and roles > +directives.register_directive('c:function', DummyDirective) > +directives.register_directive('c:type', DummyDirective) > +roles.register_generic_role('c:func', nodes.emphasis) > +roles.register_generic_role('c:type', nodes.emphasis) > + > +# We accept but ignore parameters to be compatible with how the kernel build > +# invokes CHECK. > +if len(sys.argv) < 2: > + sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FILE\n'); > + sys.exit(1) > + > +infile = sys.argv[len(sys.argv) - 1] > +cmd = ['scripts/kernel-doc', '-rst', infile] > + > +try: > + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) > + out, err = p.communicate() > + > + # python2 needs conversion to unicode. > + # python3 with universal_newlines=True returns strings. > + if sys.version_info.major < 3: > + out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8') > + > + # kernel-doc errors > + sys.stderr.write(err) > + if p.returncode != 0: > + sys.exit(p.returncode) > + > + # restructured text errors > + lines = statemachine.string2lines(out, 8, convert_whitespace=True) > + lint_errors = restructuredtext_lint.lint(out, infile) > + for error in lint_errors: > + # Ignore INFO > + if error.level <= 1: > + continue > + > + print(error.source + ': ' + error.type + ': ' + error.full_message) > + if error.line is not None: > + print('Context:') > + print('\t' + lines[error.line - 1]) > + print('\t' + lines[error.line]) > + > +except Exception as e: > + sys.stderr.write(str(e) + '\n') > + sys.exit(1) > -- > 2.1.4 > > > -- > Jani Nikula, Intel Open Source Technology Center > -- > To unsubscribe from this list: send the line "unsubscribe kernel-janitors" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > --8323329-1882111399-1475699148=:3130-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from zimbra13.linbit.com (zimbra.linbit.com [212.69.161.123]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mail09.linbit.com (LINBIT Mail Daemon) with ESMTPS id 1BD301056477 for ; Fri, 7 Oct 2016 14:07:43 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by zimbra13.linbit.com (Postfix) with ESMTP id E2EBB46E978 for ; Fri, 7 Oct 2016 14:07:42 +0200 (CEST) Received: from zimbra13.linbit.com ([127.0.0.1]) by localhost (zimbra13.linbit.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id R8CHiT7gdWj2 for ; Fri, 7 Oct 2016 14:07:42 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by zimbra13.linbit.com (Postfix) with ESMTP id BF2DB46E972 for ; Fri, 7 Oct 2016 14:07:42 +0200 (CEST) Received: from zimbra13.linbit.com ([127.0.0.1]) by localhost (zimbra13.linbit.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Jc-3qJQBq2Km for ; Fri, 7 Oct 2016 14:07:42 +0200 (CEST) Received: from soda.linbit (tuerlsteher.linbit.com [86.59.100.100]) by zimbra13.linbit.com (Postfix) with ESMTPS id 9822746E97B for ; Fri, 7 Oct 2016 14:07:42 +0200 (CEST) Resent-Message-ID: <20161007120742.GU3302@soda.linbit> Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail09.linbit.com (LINBIT Mail Daemon) with ESMTPS id 022071056332 for ; Wed, 5 Oct 2016 22:25:50 +0200 (CEST) Date: Wed, 5 Oct 2016 22:25:35 +0200 (CEST) From: Julia Lawall To: Jani Nikula In-Reply-To: <87h98quc1w.fsf@intel.com> Message-ID: References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> <87h98quc1w.fsf@intel.com> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="8323329-1882111399-1475699148=:3130" Cc: Linux PM list , linux-doc@vger.kernel.org, kernel-janitors@vger.kernel.org, Linux Kernel Mailing List , dri-devel , linux-metag@vger.kernel.org, "linaro-mm-sig@lists.linaro.org" , Julia Lawall , linux-mtd@lists.infradead.org, Daniel Vetter , "linux-tegra@vger.kernel.org" , "linux-media@vger.kernel.org" , linux-clk@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , drbd-dev@lists.linbit.com Subject: Re: [Drbd-dev] kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) List-Id: "*Coordination* of development, patches, contributions -- *Questions* \(even to developers\) go to drbd-user, please." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --8323329-1882111399-1475699148=:3130 Content-Type: TEXT/PLAIN; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Wed, 5 Oct 2016, Jani Nikula wrote: > On Wed, 05 Oct 2016, Daniel Vetter wrote: > > Jani Nikula has a patch with a scrip to make the one kernel-doc parse= r > > into a lint/checker pass over the entire kernel. I think that'd would > > be more robust instead of trying to approximate the real kerneldoc > > parser. Otoh that parser is a horror show of a perl/regex driven stat= e > > machine ;-) > > > > Jani, can you pls digg out these patches? Can't find them right now .= .. > > Expanding the massive Cc: with linux-doc list... > > Here goes. It's a quick hack from months ago, but still seems to > somewhat work. At least for the kernel-doc parts. The reStructuredText > lint part isn't all that great, and doesn't have mapping to line number= s > like the Sphinx kernel-doc extension does. Anyway I'm happy how this > integrates with kernel build CHECK and C=3D1/C=3D2. > > I guess Julia's goal is to automate the *fixing* of some of the error > classes from kernel-doc. Not sure how well this could be made to > integrate with any of that. No, my work doesn't fix anything. Coccinelle can't actually process comments. I just correlated the parsed comment with the function header. julia > > BR, > Jani. > > > From 1244efa0f63a7b13795e8c37f81733a3c8bfc56a Mon Sep 17 00:00:00 2001 > From: Jani Nikula > Date: Tue, 31 May 2016 18:11:33 +0300 > Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and = rst > correctness > Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160= Espoo > Cc: Jani Nikula > > Simple kernel-doc and reStructuredText lint tool that can be used > independently and as a kernel build CHECK tool to validate kernel-doc > comments. > > Independent usage: > $ kernel-doc-rst-lint FILE > > Kernel CHECK usage: > $ make CHECK=3Dscripts/kernel-doc-rst-lint C=3D1 # (or C=3D2) > > Depends on docutils and the rst-lint package > https://pypi.python.org/pypi/restructuredtext_lint > > Signed-off-by: Jani Nikula > --- > scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++= ++++++++ > 1 file changed, 106 insertions(+) > create mode 100755 scripts/kernel-doc-rst-lint > > diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint > new file mode 100755 > index 000000000000..7e0157679f83 > --- /dev/null > +++ b/scripts/kernel-doc-rst-lint > @@ -0,0 +1,106 @@ > +#!/usr/bin/env python > +# coding=3Dutf-8 > +# > +# Copyright =C2=A9 2016 Intel Corporation > +# > +# Permission is hereby granted, free of charge, to any person obtainin= g a > +# copy of this software and associated documentation files (the "Softw= are"), > +# to deal in the Software without restriction, including without limit= ation > +# the rights to use, copy, modify, merge, publish, distribute, sublice= nse, > +# and/or sell copies of the Software, and to permit persons to whom th= e > +# Software is furnished to do so, subject to the following conditions: > +# > +# The above copyright notice and this permission notice (including the= next > +# paragraph) shall be included in all copies or substantial portions o= f the > +# Software. > +# > +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPR= ESS OR > +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL= ITY, > +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT S= HALL > +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR= OTHER > +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARIS= ING > +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER = DEALINGS > +# IN THE SOFTWARE. > +# > +# Authors: > +# Jani Nikula > +# > +# Simple kernel-doc and reStructuredText lint tool that can be used > +# independently and as a kernel build CHECK tool to validate kernel-do= c > +# comments. > +# > +# Independent usage: > +# $ kernel-doc-rst-lint FILE > +# > +# Kernel CHECK usage: > +# $ make CHECK=3Dscripts/kernel-doc-rst-lint C=3D1 # (or C=3D2) > +# > +# Depends on docutils and the rst-lint package > +# https://pypi.python.org/pypi/restructuredtext_lint > +# > + > +import os > +import subprocess > +import sys > + > +from docutils.parsers.rst import directives > +from docutils.parsers.rst import Directive > +from docutils.parsers.rst import roles > +from docutils import nodes, statemachine > +import restructuredtext_lint > + > +class DummyDirective(Directive): > + required_argument =3D 1 > + optional_arguments =3D 0 > + option_spec =3D { } > + has_content =3D True > + > + def run(self): > + return [] > + > +# Fake the Sphinx C Domain directives and roles > +directives.register_directive('c:function', DummyDirective) > +directives.register_directive('c:type', DummyDirective) > +roles.register_generic_role('c:func', nodes.emphasis) > +roles.register_generic_role('c:type', nodes.emphasis) > + > +# We accept but ignore parameters to be compatible with how the kernel= build > +# invokes CHECK. > +if len(sys.argv) < 2: > + sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FIL= E\n'); > + sys.exit(1) > + > +infile =3D sys.argv[len(sys.argv) - 1] > +cmd =3D ['scripts/kernel-doc', '-rst', infile] > + > +try: > + p =3D subprocess.Popen(cmd, stdout=3Dsubprocess.PIPE, stderr=3Dsub= process.PIPE, universal_newlines=3DTrue) > + out, err =3D p.communicate() > + > + # python2 needs conversion to unicode. > + # python3 with universal_newlines=3DTrue returns strings. > + if sys.version_info.major < 3: > + out, err =3D unicode(out, 'utf-8'), unicode(err, 'utf-8') > + > + # kernel-doc errors > + sys.stderr.write(err) > + if p.returncode !=3D 0: > + sys.exit(p.returncode) > + > + # restructured text errors > + lines =3D statemachine.string2lines(out, 8, convert_whitespace=3DT= rue) > + lint_errors =3D restructuredtext_lint.lint(out, infile) > + for error in lint_errors: > + # Ignore INFO > + if error.level <=3D 1: > + continue > + > + print(error.source + ': ' + error.type + ': ' + error.full_mes= sage) > + if error.line is not None: > + print('Context:') > + print('\t' + lines[error.line - 1]) > + print('\t' + lines[error.line]) > + > +except Exception as e: > + sys.stderr.write(str(e) + '\n') > + sys.exit(1) > -- > 2.1.4 > > > -- > Jani Nikula, Intel Open Source Technology Center > -- > To unsubscribe from this list: send the line "unsubscribe kernel-janito= rs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > --8323329-1882111399-1475699148=:3130-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: julia.lawall@lip6.fr (Julia Lawall) Date: Wed, 5 Oct 2016 22:25:35 +0200 (CEST) Subject: kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) In-Reply-To: <87h98quc1w.fsf@intel.com> References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> <87h98quc1w.fsf@intel.com> Message-ID: To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Wed, 5 Oct 2016, Jani Nikula wrote: > On Wed, 05 Oct 2016, Daniel Vetter wrote: > > Jani Nikula has a patch with a scrip to make the one kernel-doc parser > > into a lint/checker pass over the entire kernel. I think that'd would > > be more robust instead of trying to approximate the real kerneldoc > > parser. Otoh that parser is a horror show of a perl/regex driven state > > machine ;-) > > > > Jani, can you pls digg out these patches? Can't find them right now ... > > Expanding the massive Cc: with linux-doc list... > > Here goes. It's a quick hack from months ago, but still seems to > somewhat work. At least for the kernel-doc parts. The reStructuredText > lint part isn't all that great, and doesn't have mapping to line numbers > like the Sphinx kernel-doc extension does. Anyway I'm happy how this > integrates with kernel build CHECK and C=1/C=2. > > I guess Julia's goal is to automate the *fixing* of some of the error > classes from kernel-doc. Not sure how well this could be made to > integrate with any of that. No, my work doesn't fix anything. Coccinelle can't actually process comments. I just correlated the parsed comment with the function header. julia > > BR, > Jani. > > > From 1244efa0f63a7b13795e8c37f81733a3c8bfc56a Mon Sep 17 00:00:00 2001 > From: Jani Nikula > Date: Tue, 31 May 2016 18:11:33 +0300 > Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and rst > correctness > Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo > Cc: Jani Nikula > > Simple kernel-doc and reStructuredText lint tool that can be used > independently and as a kernel build CHECK tool to validate kernel-doc > comments. > > Independent usage: > $ kernel-doc-rst-lint FILE > > Kernel CHECK usage: > $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) > > Depends on docutils and the rst-lint package > https://pypi.python.org/pypi/restructuredtext_lint > > Signed-off-by: Jani Nikula > --- > scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 106 insertions(+) > create mode 100755 scripts/kernel-doc-rst-lint > > diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint > new file mode 100755 > index 000000000000..7e0157679f83 > --- /dev/null > +++ b/scripts/kernel-doc-rst-lint > @@ -0,0 +1,106 @@ > +#!/usr/bin/env python > +# coding=utf-8 > +# > +# Copyright ? 2016 Intel Corporation > +# > +# Permission is hereby granted, free of charge, to any person obtaining a > +# copy of this software and associated documentation files (the "Software"), > +# to deal in the Software without restriction, including without limitation > +# the rights to use, copy, modify, merge, publish, distribute, sublicense, > +# and/or sell copies of the Software, and to permit persons to whom the > +# Software is furnished to do so, subject to the following conditions: > +# > +# The above copyright notice and this permission notice (including the next > +# paragraph) shall be included in all copies or substantial portions of the > +# Software. > +# > +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL > +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS > +# IN THE SOFTWARE. > +# > +# Authors: > +# Jani Nikula > +# > +# Simple kernel-doc and reStructuredText lint tool that can be used > +# independently and as a kernel build CHECK tool to validate kernel-doc > +# comments. > +# > +# Independent usage: > +# $ kernel-doc-rst-lint FILE > +# > +# Kernel CHECK usage: > +# $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) > +# > +# Depends on docutils and the rst-lint package > +# https://pypi.python.org/pypi/restructuredtext_lint > +# > + > +import os > +import subprocess > +import sys > + > +from docutils.parsers.rst import directives > +from docutils.parsers.rst import Directive > +from docutils.parsers.rst import roles > +from docutils import nodes, statemachine > +import restructuredtext_lint > + > +class DummyDirective(Directive): > + required_argument = 1 > + optional_arguments = 0 > + option_spec = { } > + has_content = True > + > + def run(self): > + return [] > + > +# Fake the Sphinx C Domain directives and roles > +directives.register_directive('c:function', DummyDirective) > +directives.register_directive('c:type', DummyDirective) > +roles.register_generic_role('c:func', nodes.emphasis) > +roles.register_generic_role('c:type', nodes.emphasis) > + > +# We accept but ignore parameters to be compatible with how the kernel build > +# invokes CHECK. > +if len(sys.argv) < 2: > + sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FILE\n'); > + sys.exit(1) > + > +infile = sys.argv[len(sys.argv) - 1] > +cmd = ['scripts/kernel-doc', '-rst', infile] > + > +try: > + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) > + out, err = p.communicate() > + > + # python2 needs conversion to unicode. > + # python3 with universal_newlines=True returns strings. > + if sys.version_info.major < 3: > + out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8') > + > + # kernel-doc errors > + sys.stderr.write(err) > + if p.returncode != 0: > + sys.exit(p.returncode) > + > + # restructured text errors > + lines = statemachine.string2lines(out, 8, convert_whitespace=True) > + lint_errors = restructuredtext_lint.lint(out, infile) > + for error in lint_errors: > + # Ignore INFO > + if error.level <= 1: > + continue > + > + print(error.source + ': ' + error.type + ': ' + error.full_message) > + if error.line is not None: > + print('Context:') > + print('\t' + lines[error.line - 1]) > + print('\t' + lines[error.line]) > + > +except Exception as e: > + sys.stderr.write(str(e) + '\n') > + sys.exit(1) > -- > 2.1.4 > > > -- > Jani Nikula, Intel Open Source Technology Center > -- > To unsubscribe from this list: send the line "unsubscribe kernel-janitors" in > the body of a message to majordomo at vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >