From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jani Nikula Date: Wed, 05 Oct 2016 14:04:43 +0000 Subject: kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) Message-Id: <87h98quc1w.fsf@intel.com> List-Id: References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable To: Daniel Vetter , Julia Lawall Cc: Linux PM list , kernel-janitors@vger.kernel.org, Linux Kernel Mailing List , dri-devel , linux-clk@vger.kernel.org, "linaro-mm-sig@lists.linaro.org" , linux-mtd@lists.infradead.org, "linux-tegra@vger.kernel.org" , drbd-dev@lists.linbit.com, linux-metag@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , "linux-media@vger.kernel.org" , linux-doc@vger.kernel.org On Wed, 05 Oct 2016, Daniel Vetter wrote: > Jani Nikula has a patch with a scrip to make the one kernel-doc parser > into a lint/checker pass over the entire kernel. I think that'd would > be more robust instead of trying to approximate the real kerneldoc > parser. Otoh that parser is a horror show of a perl/regex driven state > machine ;-) > > Jani, can you pls digg out these patches? Can't find them right now ... Expanding the massive Cc: with linux-doc list... Here goes. It's a quick hack from months ago, but still seems to somewhat work. At least for the kernel-doc parts. The reStructuredText lint part isn't all that great, and doesn't have mapping to line numbers like the Sphinx kernel-doc extension does. Anyway I'm happy how this integrates with kernel build CHECK and C=3D1/C=3D2. I guess Julia's goal is to automate the *fixing* of some of the error classes from kernel-doc. Not sure how well this could be made to integrate with any of that. BR, Jani. >From 1244efa0f63a7b13795e8c37f81733a3c8bfc56a Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Tue, 31 May 2016 18:11:33 +0300 Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and rst correctness Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Esp= oo Cc: Jani Nikula Simple kernel-doc and reStructuredText lint tool that can be used independently and as a kernel build CHECK tool to validate kernel-doc comments. Independent usage: $ kernel-doc-rst-lint FILE Kernel CHECK usage: $ make CHECK=3Dscripts/kernel-doc-rst-lint C=3D1 # (or C=3D2) Depends on docutils and the rst-lint package https://pypi.python.org/pypi/restructuredtext_lint Signed-off-by: Jani Nikula --- scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 106 insertions(+) create mode 100755 scripts/kernel-doc-rst-lint diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint new file mode 100755 index 000000000000..7e0157679f83 --- /dev/null +++ b/scripts/kernel-doc-rst-lint @@ -0,0 +1,106 @@ +#!/usr/bin/env python +# coding=3Dutf-8 +# +# Copyright =C2=A9 2016 Intel Corporation +# +# Permission is hereby granted, free of charge, to any person obtaining a +# copy of this software and associated documentation files (the "Software"= ), +# to deal in the Software without restriction, including without limitation +# the rights to use, copy, modify, merge, publish, distribute, sublicense, +# and/or sell copies of the Software, and to permit persons to whom the +# Software is furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice (including the next +# paragraph) shall be included in all copies or substantial portions of the +# Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS = OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTH= ER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEAL= INGS +# IN THE SOFTWARE. +# +# Authors: +# Jani Nikula +# +# Simple kernel-doc and reStructuredText lint tool that can be used +# independently and as a kernel build CHECK tool to validate kernel-doc +# comments. +# +# Independent usage: +# $ kernel-doc-rst-lint FILE +# +# Kernel CHECK usage: +# $ make CHECK=3Dscripts/kernel-doc-rst-lint C=3D1 # (or C=3D2) +# +# Depends on docutils and the rst-lint package +# https://pypi.python.org/pypi/restructuredtext_lint +# + +import os +import subprocess +import sys + +from docutils.parsers.rst import directives +from docutils.parsers.rst import Directive +from docutils.parsers.rst import roles +from docutils import nodes, statemachine +import restructuredtext_lint + +class DummyDirective(Directive): + required_argument =3D 1 + optional_arguments =3D 0 + option_spec =3D { } + has_content =3D True + + def run(self): + return [] + +# Fake the Sphinx C Domain directives and roles +directives.register_directive('c:function', DummyDirective) +directives.register_directive('c:type', DummyDirective) +roles.register_generic_role('c:func', nodes.emphasis) +roles.register_generic_role('c:type', nodes.emphasis) + +# We accept but ignore parameters to be compatible with how the kernel bui= ld +# invokes CHECK. +if len(sys.argv) < 2: + sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FILE\n'= ); + sys.exit(1) + +infile =3D sys.argv[len(sys.argv) - 1] +cmd =3D ['scripts/kernel-doc', '-rst', infile] + +try: + p =3D subprocess.Popen(cmd, stdout=3Dsubprocess.PIPE, stderr=3Dsubproc= ess.PIPE, universal_newlines=3DTrue) + out, err =3D p.communicate() + + # python2 needs conversion to unicode. + # python3 with universal_newlines=3DTrue returns strings. + if sys.version_info.major < 3: + out, err =3D unicode(out, 'utf-8'), unicode(err, 'utf-8') + + # kernel-doc errors + sys.stderr.write(err) + if p.returncode !=3D 0: + sys.exit(p.returncode) + + # restructured text errors + lines =3D statemachine.string2lines(out, 8, convert_whitespace=3DTrue) + lint_errors =3D restructuredtext_lint.lint(out, infile) + for error in lint_errors: + # Ignore INFO + if error.level <=3D 1: + continue + + print(error.source + ': ' + error.type + ': ' + error.full_message) + if error.line is not None: + print('Context:') + print('\t' + lines[error.line - 1]) + print('\t' + lines[error.line]) + +except Exception as e: + sys.stderr.write(str(e) + '\n') + sys.exit(1) --=20 2.1.4 --=20 Jani Nikula, Intel Open Source Technology Center -- To unsubscribe from this list: send the line "unsubscribe kernel-janitors" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Jani Nikula To: Daniel Vetter , Julia Lawall Subject: kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) In-Reply-To: References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> Date: Wed, 05 Oct 2016 17:04:43 +0300 Message-ID: <87h98quc1w.fsf@intel.com> MIME-Version: 1.0 List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Linux PM list , linux-doc@vger.kernel.org, kernel-janitors@vger.kernel.org, Linux Kernel Mailing List , dri-devel , linux-metag@vger.kernel.org, "linaro-mm-sig@lists.linaro.org" , linux-mtd@lists.infradead.org, "linux-tegra@vger.kernel.org" , "linux-media@vger.kernel.org" , linux-clk@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , drbd-dev@lists.linbit.com Content-Type: text/plain; charset="utf-8" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+mturquette=baylibre.com@lists.infradead.org List-ID: T24gV2VkLCAwNSBPY3QgMjAxNiwgRGFuaWVsIFZldHRlciA8ZGFuaWVsQGZmd2xsLmNoPiB3cm90 ZToKPiBKYW5pIE5pa3VsYSBoYXMgYSBwYXRjaCB3aXRoIGEgc2NyaXAgdG8gbWFrZSB0aGUgb25l IGtlcm5lbC1kb2MgcGFyc2VyCj4gaW50byBhIGxpbnQvY2hlY2tlciBwYXNzIG92ZXIgdGhlIGVu dGlyZSBrZXJuZWwuIEkgdGhpbmsgdGhhdCdkIHdvdWxkCj4gYmUgbW9yZSByb2J1c3QgaW5zdGVh ZCBvZiB0cnlpbmcgdG8gYXBwcm94aW1hdGUgdGhlIHJlYWwga2VybmVsZG9jCj4gcGFyc2VyLiBP dG9oIHRoYXQgcGFyc2VyIGlzIGEgaG9ycm9yIHNob3cgb2YgYSBwZXJsL3JlZ2V4IGRyaXZlbiBz dGF0ZQo+IG1hY2hpbmUgOy0pCj4KPiBKYW5pLCBjYW4geW91IHBscyBkaWdnIG91dCB0aGVzZSBw YXRjaGVzPyBDYW4ndCBmaW5kIHRoZW0gcmlnaHQgbm93IC4uLgoKRXhwYW5kaW5nIHRoZSBtYXNz aXZlIENjOiB3aXRoIGxpbnV4LWRvYyBsaXN0Li4uCgpIZXJlIGdvZXMuIEl0J3MgYSBxdWljayBo YWNrIGZyb20gbW9udGhzIGFnbywgYnV0IHN0aWxsIHNlZW1zIHRvCnNvbWV3aGF0IHdvcmsuIEF0 IGxlYXN0IGZvciB0aGUga2VybmVsLWRvYyBwYXJ0cy4gVGhlIHJlU3RydWN0dXJlZFRleHQKbGlu dCBwYXJ0IGlzbid0IGFsbCB0aGF0IGdyZWF0LCBhbmQgZG9lc24ndCBoYXZlIG1hcHBpbmcgdG8g bGluZSBudW1iZXJzCmxpa2UgdGhlIFNwaGlueCBrZXJuZWwtZG9jIGV4dGVuc2lvbiBkb2VzLiBB bnl3YXkgSSdtIGhhcHB5IGhvdyB0aGlzCmludGVncmF0ZXMgd2l0aCBrZXJuZWwgYnVpbGQgQ0hF Q0sgYW5kIEM9MS9DPTIuCgpJIGd1ZXNzIEp1bGlhJ3MgZ29hbCBpcyB0byBhdXRvbWF0ZSB0aGUg KmZpeGluZyogb2Ygc29tZSBvZiB0aGUgZXJyb3IKY2xhc3NlcyBmcm9tIGtlcm5lbC1kb2MuIE5v dCBzdXJlIGhvdyB3ZWxsIHRoaXMgY291bGQgYmUgbWFkZSB0bwppbnRlZ3JhdGUgd2l0aCBhbnkg b2YgdGhhdC4KCkJSLApKYW5pLgoKCkZyb20gMTI0NGVmYTBmNjNhN2IxMzc5NWU4YzM3ZjgxNzMz YTNjOGJmYzU2YSBNb24gU2VwIDE3IDAwOjAwOjAwIDIwMDEKRnJvbTogSmFuaSBOaWt1bGEgPGph bmkubmlrdWxhQGludGVsLmNvbT4KRGF0ZTogVHVlLCAzMSBNYXkgMjAxNiAxODoxMTozMyArMDMw MApTdWJqZWN0OiBbUEFUQ0hdIGtlcm5lbC1kb2MtcnN0LWxpbnQ6IGFkZCB0b29sIHRvIGNoZWNr IGtlcm5lbC1kb2MgYW5kIHJzdAogY29ycmVjdG5lc3MKT3JnYW5pemF0aW9uOiBJbnRlbCBGaW5s YW5kIE95IC0gQklDIDAzNTc2MDYtNCAtIFdlc3RlbmRpbmthdHUgNywgMDIxNjAgRXNwb28KQ2M6 IEphbmkgTmlrdWxhIDxqYW5pLm5pa3VsYUBpbnRlbC5jb20+CgpTaW1wbGUga2VybmVsLWRvYyBh bmQgcmVTdHJ1Y3R1cmVkVGV4dCBsaW50IHRvb2wgdGhhdCBjYW4gYmUgdXNlZAppbmRlcGVuZGVu dGx5IGFuZCBhcyBhIGtlcm5lbCBidWlsZCBDSEVDSyB0b29sIHRvIHZhbGlkYXRlIGtlcm5lbC1k b2MKY29tbWVudHMuCgpJbmRlcGVuZGVudCB1c2FnZToKJCBrZXJuZWwtZG9jLXJzdC1saW50IEZJ TEUKCktlcm5lbCBDSEVDSyB1c2FnZToKJCBtYWtlIENIRUNLPXNjcmlwdHMva2VybmVsLWRvYy1y c3QtbGludCBDPTEJCSMgKG9yIEM9MikKCkRlcGVuZHMgb24gZG9jdXRpbHMgYW5kIHRoZSByc3Qt bGludCBwYWNrYWdlCmh0dHBzOi8vcHlwaS5weXRob24ub3JnL3B5cGkvcmVzdHJ1Y3R1cmVkdGV4 dF9saW50CgpTaWduZWQtb2ZmLWJ5OiBKYW5pIE5pa3VsYSA8amFuaS5uaWt1bGFAaW50ZWwuY29t PgotLS0KIHNjcmlwdHMva2VybmVsLWRvYy1yc3QtbGludCB8IDEwNiArKysrKysrKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrKwogMSBmaWxlIGNoYW5nZWQsIDEwNiBpbnNlcnRp b25zKCspCiBjcmVhdGUgbW9kZSAxMDA3NTUgc2NyaXB0cy9rZXJuZWwtZG9jLXJzdC1saW50Cgpk aWZmIC0tZ2l0IGEvc2NyaXB0cy9rZXJuZWwtZG9jLXJzdC1saW50IGIvc2NyaXB0cy9rZXJuZWwt ZG9jLXJzdC1saW50Cm5ldyBmaWxlIG1vZGUgMTAwNzU1CmluZGV4IDAwMDAwMDAwMDAwMC4uN2Uw MTU3Njc5ZjgzCi0tLSAvZGV2L251bGwKKysrIGIvc2NyaXB0cy9rZXJuZWwtZG9jLXJzdC1saW50 CkBAIC0wLDAgKzEsMTA2IEBACisjIS91c3IvYmluL2VudiBweXRob24KKyMgY29kaW5nPXV0Zi04 CisjCisjIENvcHlyaWdodCDCqSAyMDE2IEludGVsIENvcnBvcmF0aW9uCisjCisjIFBlcm1pc3Np b24gaXMgaGVyZWJ5IGdyYW50ZWQsIGZyZWUgb2YgY2hhcmdlLCB0byBhbnkgcGVyc29uIG9idGFp bmluZyBhCisjIGNvcHkgb2YgdGhpcyBzb2Z0d2FyZSBhbmQgYXNzb2NpYXRlZCBkb2N1bWVudGF0 aW9uIGZpbGVzICh0aGUgIlNvZnR3YXJlIiksCisjIHRvIGRlYWwgaW4gdGhlIFNvZnR3YXJlIHdp dGhvdXQgcmVzdHJpY3Rpb24sIGluY2x1ZGluZyB3aXRob3V0IGxpbWl0YXRpb24KKyMgdGhlIHJp Z2h0cyB0byB1c2UsIGNvcHksIG1vZGlmeSwgbWVyZ2UsIHB1Ymxpc2gsIGRpc3RyaWJ1dGUsIHN1 YmxpY2Vuc2UsCisjIGFuZC9vciBzZWxsIGNvcGllcyBvZiB0aGUgU29mdHdhcmUsIGFuZCB0byBw ZXJtaXQgcGVyc29ucyB0byB3aG9tIHRoZQorIyBTb2Z0d2FyZSBpcyBmdXJuaXNoZWQgdG8gZG8g c28sIHN1YmplY3QgdG8gdGhlIGZvbGxvd2luZyBjb25kaXRpb25zOgorIworIyBUaGUgYWJvdmUg Y29weXJpZ2h0IG5vdGljZSBhbmQgdGhpcyBwZXJtaXNzaW9uIG5vdGljZSAoaW5jbHVkaW5nIHRo ZSBuZXh0CisjIHBhcmFncmFwaCkgc2hhbGwgYmUgaW5jbHVkZWQgaW4gYWxsIGNvcGllcyBvciBz dWJzdGFudGlhbCBwb3J0aW9ucyBvZiB0aGUKKyMgU29mdHdhcmUuCisjCisjIFRIRSBTT0ZUV0FS RSBJUyBQUk9WSURFRCAiQVMgSVMiLCBXSVRIT1VUIFdBUlJBTlRZIE9GIEFOWSBLSU5ELCBFWFBS RVNTIE9SCisjIElNUExJRUQsIElOQ0xVRElORyBCVVQgTk9UIExJTUlURUQgVE8gVEhFIFdBUlJB TlRJRVMgT0YgTUVSQ0hBTlRBQklMSVRZLAorIyBGSVRORVNTIEZPUiBBIFBBUlRJQ1VMQVIgUFVS UE9TRSBBTkQgTk9OSU5GUklOR0VNRU5ULiAgSU4gTk8gRVZFTlQgU0hBTEwKKyMgVEhFIEFVVEhP UlMgT1IgQ09QWVJJR0hUIEhPTERFUlMgQkUgTElBQkxFIEZPUiBBTlkgQ0xBSU0sIERBTUFHRVMg T1IgT1RIRVIKKyMgTElBQklMSVRZLCBXSEVUSEVSIElOIEFOIEFDVElPTiBPRiBDT05UUkFDVCwg VE9SVCBPUiBPVEhFUldJU0UsIEFSSVNJTkcKKyMgRlJPTSwgT1VUIE9GIE9SIElOIENPTk5FQ1RJ T04gV0lUSCBUSEUgU09GVFdBUkUgT1IgVEhFIFVTRSBPUiBPVEhFUiBERUFMSU5HUworIyBJTiBU SEUgU09GVFdBUkUuCisjCisjIEF1dGhvcnM6CisjICAgIEphbmkgTmlrdWxhIDxqYW5pLm5pa3Vs YUBpbnRlbC5jb20+CisjCisjIFNpbXBsZSBrZXJuZWwtZG9jIGFuZCByZVN0cnVjdHVyZWRUZXh0 IGxpbnQgdG9vbCB0aGF0IGNhbiBiZSB1c2VkCisjIGluZGVwZW5kZW50bHkgYW5kIGFzIGEga2Vy bmVsIGJ1aWxkIENIRUNLIHRvb2wgdG8gdmFsaWRhdGUga2VybmVsLWRvYworIyBjb21tZW50cy4K KyMKKyMgSW5kZXBlbmRlbnQgdXNhZ2U6CisjICQga2VybmVsLWRvYy1yc3QtbGludCBGSUxFCisj CisjIEtlcm5lbCBDSEVDSyB1c2FnZToKKyMgJCBtYWtlIENIRUNLPXNjcmlwdHMva2VybmVsLWRv Yy1yc3QtbGludCBDPTEJCSMgKG9yIEM9MikKKyMKKyMgRGVwZW5kcyBvbiBkb2N1dGlscyBhbmQg dGhlIHJzdC1saW50IHBhY2thZ2UKKyMgaHR0cHM6Ly9weXBpLnB5dGhvbi5vcmcvcHlwaS9yZXN0 cnVjdHVyZWR0ZXh0X2xpbnQKKyMKKworaW1wb3J0IG9zCitpbXBvcnQgc3VicHJvY2VzcworaW1w b3J0IHN5cworCitmcm9tIGRvY3V0aWxzLnBhcnNlcnMucnN0IGltcG9ydCBkaXJlY3RpdmVzCitm cm9tIGRvY3V0aWxzLnBhcnNlcnMucnN0IGltcG9ydCBEaXJlY3RpdmUKK2Zyb20gZG9jdXRpbHMu cGFyc2Vycy5yc3QgaW1wb3J0IHJvbGVzCitmcm9tIGRvY3V0aWxzIGltcG9ydCBub2Rlcywgc3Rh dGVtYWNoaW5lCitpbXBvcnQgcmVzdHJ1Y3R1cmVkdGV4dF9saW50CisKK2NsYXNzIER1bW15RGly ZWN0aXZlKERpcmVjdGl2ZSk6CisgICAgcmVxdWlyZWRfYXJndW1lbnQgPSAxCisgICAgb3B0aW9u YWxfYXJndW1lbnRzID0gMAorICAgIG9wdGlvbl9zcGVjID0geyB9CisgICAgaGFzX2NvbnRlbnQg PSBUcnVlCisKKyAgICBkZWYgcnVuKHNlbGYpOgorICAgICAgICByZXR1cm4gW10KKworIyBGYWtl IHRoZSBTcGhpbnggQyBEb21haW4gZGlyZWN0aXZlcyBhbmQgcm9sZXMKK2RpcmVjdGl2ZXMucmVn aXN0ZXJfZGlyZWN0aXZlKCdjOmZ1bmN0aW9uJywgRHVtbXlEaXJlY3RpdmUpCitkaXJlY3RpdmVz LnJlZ2lzdGVyX2RpcmVjdGl2ZSgnYzp0eXBlJywgRHVtbXlEaXJlY3RpdmUpCityb2xlcy5yZWdp c3Rlcl9nZW5lcmljX3JvbGUoJ2M6ZnVuYycsIG5vZGVzLmVtcGhhc2lzKQorcm9sZXMucmVnaXN0 ZXJfZ2VuZXJpY19yb2xlKCdjOnR5cGUnLCBub2Rlcy5lbXBoYXNpcykKKworIyBXZSBhY2NlcHQg YnV0IGlnbm9yZSBwYXJhbWV0ZXJzIHRvIGJlIGNvbXBhdGlibGUgd2l0aCBob3cgdGhlIGtlcm5l bCBidWlsZAorIyBpbnZva2VzIENIRUNLLgoraWYgbGVuKHN5cy5hcmd2KSA8IDI6CisgICAgc3lz LnN0ZGVyci53cml0ZSgndXNhZ2U6IGtlcm5lbC1kb2MtcnN0LWxpbnQgW0lHTk9SRUQgT1BUSU9O U10gRklMRVxuJyk7CisgICAgc3lzLmV4aXQoMSkKKworaW5maWxlID0gc3lzLmFyZ3ZbbGVuKHN5 cy5hcmd2KSAtIDFdCitjbWQgPSBbJ3NjcmlwdHMva2VybmVsLWRvYycsICctcnN0JywgaW5maWxl XQorCit0cnk6CisgICAgcCA9IHN1YnByb2Nlc3MuUG9wZW4oY21kLCBzdGRvdXQ9c3VicHJvY2Vz cy5QSVBFLCBzdGRlcnI9c3VicHJvY2Vzcy5QSVBFLCB1bml2ZXJzYWxfbmV3bGluZXM9VHJ1ZSkK KyAgICBvdXQsIGVyciA9IHAuY29tbXVuaWNhdGUoKQorCisgICAgIyBweXRob24yIG5lZWRzIGNv bnZlcnNpb24gdG8gdW5pY29kZS4KKyAgICAjIHB5dGhvbjMgd2l0aCB1bml2ZXJzYWxfbmV3bGlu ZXM9VHJ1ZSByZXR1cm5zIHN0cmluZ3MuCisgICAgaWYgc3lzLnZlcnNpb25faW5mby5tYWpvciA8 IDM6CisgICAgICAgIG91dCwgZXJyID0gdW5pY29kZShvdXQsICd1dGYtOCcpLCB1bmljb2RlKGVy ciwgJ3V0Zi04JykKKworICAgICMga2VybmVsLWRvYyBlcnJvcnMKKyAgICBzeXMuc3RkZXJyLndy aXRlKGVycikKKyAgICBpZiBwLnJldHVybmNvZGUgIT0gMDoKKyAgICAgICAgc3lzLmV4aXQocC5y ZXR1cm5jb2RlKQorCisgICAgIyByZXN0cnVjdHVyZWQgdGV4dCBlcnJvcnMKKyAgICBsaW5lcyA9 IHN0YXRlbWFjaGluZS5zdHJpbmcybGluZXMob3V0LCA4LCBjb252ZXJ0X3doaXRlc3BhY2U9VHJ1 ZSkKKyAgICBsaW50X2Vycm9ycyA9IHJlc3RydWN0dXJlZHRleHRfbGludC5saW50KG91dCwgaW5m aWxlKQorICAgIGZvciBlcnJvciBpbiBsaW50X2Vycm9yczoKKyAgICAgICAgIyBJZ25vcmUgSU5G TworICAgICAgICBpZiBlcnJvci5sZXZlbCA8PSAxOgorICAgICAgICAgICAgY29udGludWUKKwor ICAgICAgICBwcmludChlcnJvci5zb3VyY2UgKyAnOiAnICsgZXJyb3IudHlwZSArICc6ICcgKyBl cnJvci5mdWxsX21lc3NhZ2UpCisgICAgICAgIGlmIGVycm9yLmxpbmUgaXMgbm90IE5vbmU6Cisg ICAgICAgICAgICBwcmludCgnQ29udGV4dDonKQorICAgICAgICAgICAgcHJpbnQoJ1x0JyArIGxp bmVzW2Vycm9yLmxpbmUgLSAxXSkKKyAgICAgICAgICAgIHByaW50KCdcdCcgKyBsaW5lc1tlcnJv ci5saW5lXSkKKworZXhjZXB0IEV4Y2VwdGlvbiBhcyBlOgorICAgIHN5cy5zdGRlcnIud3JpdGUo c3RyKGUpICsgJ1xuJykKKyAgICBzeXMuZXhpdCgxKQotLSAKMi4xLjQKCgotLSAKSmFuaSBOaWt1 bGEsIEludGVsIE9wZW4gU291cmNlIFRlY2hub2xvZ3kgQ2VudGVyCgpfX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcg bGlzdApsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmlu ZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jani Nikula Subject: kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) Date: Wed, 05 Oct 2016 17:04:43 +0300 Message-ID: <87h98quc1w.fsf@intel.com> References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: Sender: linux-doc-owner@vger.kernel.org List-ID: Content-Type: text/plain; charset="iso-8859-1" To: Daniel Vetter , Julia Lawall Cc: Linux PM list , kernel-janitors@vger.kernel.org, Linux Kernel Mailing List , dri-devel , linux-clk@vger.kernel.org, "linaro-mm-sig@lists.linaro.org" , linux-mtd@lists.infradead.org, "linux-tegra@vger.kernel.org" , drbd-dev@lists.linbit.com, linux-metag@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , "linux-media@vger.kernel.org" , linux-doc@vger.kernel.org On Wed, 05 Oct 2016, Daniel Vetter wrote: > Jani Nikula has a patch with a scrip to make the one kernel-doc parser > into a lint/checker pass over the entire kernel. I think that'd would > be more robust instead of trying to approximate the real kerneldoc > parser. Otoh that parser is a horror show of a perl/regex driven state > machine ;-) > > Jani, can you pls digg out these patches? Can't find them right now ... Expanding the massive Cc: with linux-doc list... Here goes. It's a quick hack from months ago, but still seems to somewhat work. At least for the kernel-doc parts. The reStructuredText lint part isn't all that great, and doesn't have mapping to line numbers like the Sphinx kernel-doc extension does. Anyway I'm happy how this integrates with kernel build CHECK and C=3D1/C=3D2. I guess Julia's goal is to automate the *fixing* of some of the error classes from kernel-doc. Not sure how well this could be made to integrate with any of that. BR, Jani. >From 1244efa0f63a7b13795e8c37f81733a3c8bfc56a Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Tue, 31 May 2016 18:11:33 +0300 Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and rst correctness Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Esp= oo Cc: Jani Nikula Simple kernel-doc and reStructuredText lint tool that can be used independently and as a kernel build CHECK tool to validate kernel-doc comments. Independent usage: $ kernel-doc-rst-lint FILE Kernel CHECK usage: $ make CHECK=3Dscripts/kernel-doc-rst-lint C=3D1 # (or C=3D2) Depends on docutils and the rst-lint package https://pypi.python.org/pypi/restructuredtext_lint Signed-off-by: Jani Nikula --- scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 106 insertions(+) create mode 100755 scripts/kernel-doc-rst-lint diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint new file mode 100755 index 000000000000..7e0157679f83 --- /dev/null +++ b/scripts/kernel-doc-rst-lint @@ -0,0 +1,106 @@ +#!/usr/bin/env python +# coding=3Dutf-8 +# +# Copyright =C2=A9 2016 Intel Corporation +# +# Permission is hereby granted, free of charge, to any person obtaining a +# copy of this software and associated documentation files (the "Software"= ), +# to deal in the Software without restriction, including without limitation +# the rights to use, copy, modify, merge, publish, distribute, sublicense, +# and/or sell copies of the Software, and to permit persons to whom the +# Software is furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice (including the next +# paragraph) shall be included in all copies or substantial portions of the +# Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS = OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTH= ER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEAL= INGS +# IN THE SOFTWARE. +# +# Authors: +# Jani Nikula +# +# Simple kernel-doc and reStructuredText lint tool that can be used +# independently and as a kernel build CHECK tool to validate kernel-doc +# comments. +# +# Independent usage: +# $ kernel-doc-rst-lint FILE +# +# Kernel CHECK usage: +# $ make CHECK=3Dscripts/kernel-doc-rst-lint C=3D1 # (or C=3D2) +# +# Depends on docutils and the rst-lint package +# https://pypi.python.org/pypi/restructuredtext_lint +# + +import os +import subprocess +import sys + +from docutils.parsers.rst import directives +from docutils.parsers.rst import Directive +from docutils.parsers.rst import roles +from docutils import nodes, statemachine +import restructuredtext_lint + +class DummyDirective(Directive): + required_argument =3D 1 + optional_arguments =3D 0 + option_spec =3D { } + has_content =3D True + + def run(self): + return [] + +# Fake the Sphinx C Domain directives and roles +directives.register_directive('c:function', DummyDirective) +directives.register_directive('c:type', DummyDirective) +roles.register_generic_role('c:func', nodes.emphasis) +roles.register_generic_role('c:type', nodes.emphasis) + +# We accept but ignore parameters to be compatible with how the kernel bui= ld +# invokes CHECK. +if len(sys.argv) < 2: + sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FILE\n'= ); + sys.exit(1) + +infile =3D sys.argv[len(sys.argv) - 1] +cmd =3D ['scripts/kernel-doc', '-rst', infile] + +try: + p =3D subprocess.Popen(cmd, stdout=3Dsubprocess.PIPE, stderr=3Dsubproc= ess.PIPE, universal_newlines=3DTrue) + out, err =3D p.communicate() + + # python2 needs conversion to unicode. + # python3 with universal_newlines=3DTrue returns strings. + if sys.version_info.major < 3: + out, err =3D unicode(out, 'utf-8'), unicode(err, 'utf-8') + + # kernel-doc errors + sys.stderr.write(err) + if p.returncode !=3D 0: + sys.exit(p.returncode) + + # restructured text errors + lines =3D statemachine.string2lines(out, 8, convert_whitespace=3DTrue) + lint_errors =3D restructuredtext_lint.lint(out, infile) + for error in lint_errors: + # Ignore INFO + if error.level <=3D 1: + continue + + print(error.source + ': ' + error.type + ': ' + error.full_message) + if error.line is not None: + print('Context:') + print('\t' + lines[error.line - 1]) + print('\t' + lines[error.line]) + +except Exception as e: + sys.stderr.write(str(e) + '\n') + sys.exit(1) --=20 2.1.4 --=20 Jani Nikula, Intel Open Source Technology Center From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jani Nikula To: Daniel Vetter , Julia Lawall Cc: Linux PM list , kernel-janitors@vger.kernel.org, Linux Kernel Mailing List , dri-devel , linux-clk@vger.kernel.org, "linaro-mm-sig\@lists.linaro.org" , linux-mtd@lists.infradead.org, "linux-tegra\@vger.kernel.org" , drbd-dev@lists.linbit.com, linux-metag@vger.kernel.org, "linux-arm-kernel\@lists.infradead.org" , "linux-media\@vger.kernel.org" , linux-doc@vger.kernel.org Subject: kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) In-Reply-To: References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> Date: Wed, 05 Oct 2016 17:04:43 +0300 Message-ID: <87h98quc1w.fsf@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Linux MTD discussion mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Wed, 05 Oct 2016, Daniel Vetter wrote: > Jani Nikula has a patch with a scrip to make the one kernel-doc parser > into a lint/checker pass over the entire kernel. I think that'd would > be more robust instead of trying to approximate the real kerneldoc > parser. Otoh that parser is a horror show of a perl/regex driven state > machine ;-) > > Jani, can you pls digg out these patches? Can't find them right now ... Expanding the massive Cc: with linux-doc list... Here goes. It's a quick hack from months ago, but still seems to somewhat work. At least for the kernel-doc parts. The reStructuredText lint part isn't all that great, and doesn't have mapping to line numbers like the Sphinx kernel-doc extension does. Anyway I'm happy how this integrates with kernel build CHECK and C=3D1/C=3D2. I guess Julia's goal is to automate the *fixing* of some of the error classes from kernel-doc. Not sure how well this could be made to integrate with any of that. BR, Jani. >>From 1244efa0f63a7b13795e8c37f81733a3c8bfc56a Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Tue, 31 May 2016 18:11:33 +0300 Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and rst correctness Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Esp= oo Cc: Jani Nikula Simple kernel-doc and reStructuredText lint tool that can be used independently and as a kernel build CHECK tool to validate kernel-doc comments. Independent usage: $ kernel-doc-rst-lint FILE Kernel CHECK usage: $ make CHECK=3Dscripts/kernel-doc-rst-lint C=3D1 # (or C=3D2) Depends on docutils and the rst-lint package https://pypi.python.org/pypi/restructuredtext_lint Signed-off-by: Jani Nikula --- scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 106 insertions(+) create mode 100755 scripts/kernel-doc-rst-lint diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint new file mode 100755 index 000000000000..7e0157679f83 --- /dev/null +++ b/scripts/kernel-doc-rst-lint @@ -0,0 +1,106 @@ +#!/usr/bin/env python +# coding=3Dutf-8 +# +# Copyright =C2=A9 2016 Intel Corporation +# +# Permission is hereby granted, free of charge, to any person obtaining a +# copy of this software and associated documentation files (the "Software"= ), +# to deal in the Software without restriction, including without limitation +# the rights to use, copy, modify, merge, publish, distribute, sublicense, +# and/or sell copies of the Software, and to permit persons to whom the +# Software is furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice (including the next +# paragraph) shall be included in all copies or substantial portions of the +# Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS = OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTH= ER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEAL= INGS +# IN THE SOFTWARE. +# +# Authors: +# Jani Nikula +# +# Simple kernel-doc and reStructuredText lint tool that can be used +# independently and as a kernel build CHECK tool to validate kernel-doc +# comments. +# +# Independent usage: +# $ kernel-doc-rst-lint FILE +# +# Kernel CHECK usage: +# $ make CHECK=3Dscripts/kernel-doc-rst-lint C=3D1 # (or C=3D2) +# +# Depends on docutils and the rst-lint package +# https://pypi.python.org/pypi/restructuredtext_lint +# + +import os +import subprocess +import sys + +from docutils.parsers.rst import directives +from docutils.parsers.rst import Directive +from docutils.parsers.rst import roles +from docutils import nodes, statemachine +import restructuredtext_lint + +class DummyDirective(Directive): + required_argument =3D 1 + optional_arguments =3D 0 + option_spec =3D { } + has_content =3D True + + def run(self): + return [] + +# Fake the Sphinx C Domain directives and roles +directives.register_directive('c:function', DummyDirective) +directives.register_directive('c:type', DummyDirective) +roles.register_generic_role('c:func', nodes.emphasis) +roles.register_generic_role('c:type', nodes.emphasis) + +# We accept but ignore parameters to be compatible with how the kernel bui= ld +# invokes CHECK. +if len(sys.argv) < 2: + sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FILE\n'= ); + sys.exit(1) + +infile =3D sys.argv[len(sys.argv) - 1] +cmd =3D ['scripts/kernel-doc', '-rst', infile] + +try: + p =3D subprocess.Popen(cmd, stdout=3Dsubprocess.PIPE, stderr=3Dsubproc= ess.PIPE, universal_newlines=3DTrue) + out, err =3D p.communicate() + + # python2 needs conversion to unicode. + # python3 with universal_newlines=3DTrue returns strings. + if sys.version_info.major < 3: + out, err =3D unicode(out, 'utf-8'), unicode(err, 'utf-8') + + # kernel-doc errors + sys.stderr.write(err) + if p.returncode !=3D 0: + sys.exit(p.returncode) + + # restructured text errors + lines =3D statemachine.string2lines(out, 8, convert_whitespace=3DTrue) + lint_errors =3D restructuredtext_lint.lint(out, infile) + for error in lint_errors: + # Ignore INFO + if error.level <=3D 1: + continue + + print(error.source + ': ' + error.type + ': ' + error.full_message) + if error.line is not None: + print('Context:') + print('\t' + lines[error.line - 1]) + print('\t' + lines[error.line]) + +except Exception as e: + sys.stderr.write(str(e) + '\n') + sys.exit(1) --=20 2.1.4 --=20 Jani Nikula, Intel Open Source Technology Center From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jani Nikula Subject: kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) Date: Wed, 05 Oct 2016 17:04:43 +0300 Message-ID: <87h98quc1w.fsf@intel.com> References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8BIT Return-path: In-Reply-To: Sender: linux-doc-owner@vger.kernel.org To: Daniel Vetter , Julia Lawall Cc: Linux PM list , kernel-janitors@vger.kernel.org, Linux Kernel Mailing List , dri-devel , linux-clk@vger.kernel.org, "linaro-mm-sig@lists.linaro.org" , linux-mtd@lists.infradead.org, "linux-tegra@vger.kernel.org" , drbd-dev@lists.linbit.com, linux-metag@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , "linux-media@vger.kernel.org" , linux-doc@vger.kernel.org List-Id: linux-pm@vger.kernel.org On Wed, 05 Oct 2016, Daniel Vetter wrote: > Jani Nikula has a patch with a scrip to make the one kernel-doc parser > into a lint/checker pass over the entire kernel. I think that'd would > be more robust instead of trying to approximate the real kerneldoc > parser. Otoh that parser is a horror show of a perl/regex driven state > machine ;-) > > Jani, can you pls digg out these patches? Can't find them right now ... Expanding the massive Cc: with linux-doc list... Here goes. It's a quick hack from months ago, but still seems to somewhat work. At least for the kernel-doc parts. The reStructuredText lint part isn't all that great, and doesn't have mapping to line numbers like the Sphinx kernel-doc extension does. Anyway I'm happy how this integrates with kernel build CHECK and C=1/C=2. I guess Julia's goal is to automate the *fixing* of some of the error classes from kernel-doc. Not sure how well this could be made to integrate with any of that. BR, Jani. >>From 1244efa0f63a7b13795e8c37f81733a3c8bfc56a Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Tue, 31 May 2016 18:11:33 +0300 Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and rst correctness Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo Cc: Jani Nikula Simple kernel-doc and reStructuredText lint tool that can be used independently and as a kernel build CHECK tool to validate kernel-doc comments. Independent usage: $ kernel-doc-rst-lint FILE Kernel CHECK usage: $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) Depends on docutils and the rst-lint package https://pypi.python.org/pypi/restructuredtext_lint Signed-off-by: Jani Nikula --- scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100755 scripts/kernel-doc-rst-lint diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint new file mode 100755 index 000000000000..7e0157679f83 --- /dev/null +++ b/scripts/kernel-doc-rst-lint @@ -0,0 +1,106 @@ +#!/usr/bin/env python +# coding=utf-8 +# +# Copyright © 2016 Intel Corporation +# +# Permission is hereby granted, free of charge, to any person obtaining a +# copy of this software and associated documentation files (the "Software"), +# to deal in the Software without restriction, including without limitation +# the rights to use, copy, modify, merge, publish, distribute, sublicense, +# and/or sell copies of the Software, and to permit persons to whom the +# Software is furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice (including the next +# paragraph) shall be included in all copies or substantial portions of the +# Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +# IN THE SOFTWARE. +# +# Authors: +# Jani Nikula +# +# Simple kernel-doc and reStructuredText lint tool that can be used +# independently and as a kernel build CHECK tool to validate kernel-doc +# comments. +# +# Independent usage: +# $ kernel-doc-rst-lint FILE +# +# Kernel CHECK usage: +# $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) +# +# Depends on docutils and the rst-lint package +# https://pypi.python.org/pypi/restructuredtext_lint +# + +import os +import subprocess +import sys + +from docutils.parsers.rst import directives +from docutils.parsers.rst import Directive +from docutils.parsers.rst import roles +from docutils import nodes, statemachine +import restructuredtext_lint + +class DummyDirective(Directive): + required_argument = 1 + optional_arguments = 0 + option_spec = { } + has_content = True + + def run(self): + return [] + +# Fake the Sphinx C Domain directives and roles +directives.register_directive('c:function', DummyDirective) +directives.register_directive('c:type', DummyDirective) +roles.register_generic_role('c:func', nodes.emphasis) +roles.register_generic_role('c:type', nodes.emphasis) + +# We accept but ignore parameters to be compatible with how the kernel build +# invokes CHECK. +if len(sys.argv) < 2: + sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FILE\n'); + sys.exit(1) + +infile = sys.argv[len(sys.argv) - 1] +cmd = ['scripts/kernel-doc', '-rst', infile] + +try: + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) + out, err = p.communicate() + + # python2 needs conversion to unicode. + # python3 with universal_newlines=True returns strings. + if sys.version_info.major < 3: + out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8') + + # kernel-doc errors + sys.stderr.write(err) + if p.returncode != 0: + sys.exit(p.returncode) + + # restructured text errors + lines = statemachine.string2lines(out, 8, convert_whitespace=True) + lint_errors = restructuredtext_lint.lint(out, infile) + for error in lint_errors: + # Ignore INFO + if error.level <= 1: + continue + + print(error.source + ': ' + error.type + ': ' + error.full_message) + if error.line is not None: + print('Context:') + print('\t' + lines[error.line - 1]) + print('\t' + lines[error.line]) + +except Exception as e: + sys.stderr.write(str(e) + '\n') + sys.exit(1) -- 2.1.4 -- Jani Nikula, Intel Open Source Technology Center From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jani Nikula Subject: kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) Date: Wed, 05 Oct 2016 17:04:43 +0300 Message-ID: <87h98quc1w.fsf@intel.com> References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8BIT Return-path: In-Reply-To: Sender: linux-doc-owner@vger.kernel.org To: Daniel Vetter , Julia Lawall Cc: Linux PM list , kernel-janitors@vger.kernel.org, Linux Kernel Mailing List , dri-devel , linux-clk@vger.kernel.org, "linaro-mm-sig@lists.linaro.org" , linux-mtd@lists.infradead.org, "linux-tegra@vger.kernel.org" , drbd-dev@lists.linbit.com, linux-metag@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , "linux-media@vger.kernel.org" , linux-doc@vger.kernel.org List-Id: linux-tegra@vger.kernel.org On Wed, 05 Oct 2016, Daniel Vetter wrote: > Jani Nikula has a patch with a scrip to make the one kernel-doc parser > into a lint/checker pass over the entire kernel. I think that'd would > be more robust instead of trying to approximate the real kerneldoc > parser. Otoh that parser is a horror show of a perl/regex driven state > machine ;-) > > Jani, can you pls digg out these patches? Can't find them right now ... Expanding the massive Cc: with linux-doc list... Here goes. It's a quick hack from months ago, but still seems to somewhat work. At least for the kernel-doc parts. The reStructuredText lint part isn't all that great, and doesn't have mapping to line numbers like the Sphinx kernel-doc extension does. Anyway I'm happy how this integrates with kernel build CHECK and C=1/C=2. I guess Julia's goal is to automate the *fixing* of some of the error classes from kernel-doc. Not sure how well this could be made to integrate with any of that. BR, Jani. >From 1244efa0f63a7b13795e8c37f81733a3c8bfc56a Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Tue, 31 May 2016 18:11:33 +0300 Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and rst correctness Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo Cc: Jani Nikula Simple kernel-doc and reStructuredText lint tool that can be used independently and as a kernel build CHECK tool to validate kernel-doc comments. Independent usage: $ kernel-doc-rst-lint FILE Kernel CHECK usage: $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) Depends on docutils and the rst-lint package https://pypi.python.org/pypi/restructuredtext_lint Signed-off-by: Jani Nikula --- scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100755 scripts/kernel-doc-rst-lint diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint new file mode 100755 index 000000000000..7e0157679f83 --- /dev/null +++ b/scripts/kernel-doc-rst-lint @@ -0,0 +1,106 @@ +#!/usr/bin/env python +# coding=utf-8 +# +# Copyright © 2016 Intel Corporation +# +# Permission is hereby granted, free of charge, to any person obtaining a +# copy of this software and associated documentation files (the "Software"), +# to deal in the Software without restriction, including without limitation +# the rights to use, copy, modify, merge, publish, distribute, sublicense, +# and/or sell copies of the Software, and to permit persons to whom the +# Software is furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice (including the next +# paragraph) shall be included in all copies or substantial portions of the +# Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +# IN THE SOFTWARE. +# +# Authors: +# Jani Nikula +# +# Simple kernel-doc and reStructuredText lint tool that can be used +# independently and as a kernel build CHECK tool to validate kernel-doc +# comments. +# +# Independent usage: +# $ kernel-doc-rst-lint FILE +# +# Kernel CHECK usage: +# $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) +# +# Depends on docutils and the rst-lint package +# https://pypi.python.org/pypi/restructuredtext_lint +# + +import os +import subprocess +import sys + +from docutils.parsers.rst import directives +from docutils.parsers.rst import Directive +from docutils.parsers.rst import roles +from docutils import nodes, statemachine +import restructuredtext_lint + +class DummyDirective(Directive): + required_argument = 1 + optional_arguments = 0 + option_spec = { } + has_content = True + + def run(self): + return [] + +# Fake the Sphinx C Domain directives and roles +directives.register_directive('c:function', DummyDirective) +directives.register_directive('c:type', DummyDirective) +roles.register_generic_role('c:func', nodes.emphasis) +roles.register_generic_role('c:type', nodes.emphasis) + +# We accept but ignore parameters to be compatible with how the kernel build +# invokes CHECK. +if len(sys.argv) < 2: + sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FILE\n'); + sys.exit(1) + +infile = sys.argv[len(sys.argv) - 1] +cmd = ['scripts/kernel-doc', '-rst', infile] + +try: + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) + out, err = p.communicate() + + # python2 needs conversion to unicode. + # python3 with universal_newlines=True returns strings. + if sys.version_info.major < 3: + out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8') + + # kernel-doc errors + sys.stderr.write(err) + if p.returncode != 0: + sys.exit(p.returncode) + + # restructured text errors + lines = statemachine.string2lines(out, 8, convert_whitespace=True) + lint_errors = restructuredtext_lint.lint(out, infile) + for error in lint_errors: + # Ignore INFO + if error.level <= 1: + continue + + print(error.source + ': ' + error.type + ': ' + error.full_message) + if error.line is not None: + print('Context:') + print('\t' + lines[error.line - 1]) + print('\t' + lines[error.line]) + +except Exception as e: + sys.stderr.write(str(e) + '\n') + sys.exit(1) -- 2.1.4 -- Jani Nikula, Intel Open Source Technology Center From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from zimbra13.linbit.com (zimbra.linbit.com [212.69.161.123]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mail09.linbit.com (LINBIT Mail Daemon) with ESMTPS id 49A691056469 for ; Fri, 7 Oct 2016 14:07:37 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by zimbra13.linbit.com (Postfix) with ESMTP id EE03A46E972 for ; Fri, 7 Oct 2016 14:07:36 +0200 (CEST) Received: from zimbra13.linbit.com ([127.0.0.1]) by localhost (zimbra13.linbit.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id upSFbjoxS9R5 for ; Fri, 7 Oct 2016 14:07:36 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by zimbra13.linbit.com (Postfix) with ESMTP id D19F546E97B for ; Fri, 7 Oct 2016 14:07:36 +0200 (CEST) Received: from zimbra13.linbit.com ([127.0.0.1]) by localhost (zimbra13.linbit.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id vXkLTkt5k1X1 for ; Fri, 7 Oct 2016 14:07:36 +0200 (CEST) Received: from soda.linbit (tuerlsteher.linbit.com [86.59.100.100]) by zimbra13.linbit.com (Postfix) with ESMTPS id A734246E972 for ; Fri, 7 Oct 2016 14:07:36 +0200 (CEST) Resent-Message-ID: <20161007120736.GR3302@soda.linbit> Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail09.linbit.com (LINBIT Mail Daemon) with ESMTPS id 66F541056332 for ; Wed, 5 Oct 2016 16:14:53 +0200 (CEST) From: Jani Nikula To: Daniel Vetter , Julia Lawall In-Reply-To: References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> Date: Wed, 05 Oct 2016 17:04:43 +0300 Message-ID: <87h98quc1w.fsf@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Linux PM list , linux-doc@vger.kernel.org, kernel-janitors@vger.kernel.org, Linux Kernel Mailing List , dri-devel , linux-metag@vger.kernel.org, "linaro-mm-sig@lists.linaro.org" , linux-mtd@lists.infradead.org, "linux-tegra@vger.kernel.org" , "linux-media@vger.kernel.org" , linux-clk@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , drbd-dev@lists.linbit.com Subject: [Drbd-dev] kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) List-Id: "*Coordination* of development, patches, contributions -- *Questions* \(even to developers\) go to drbd-user, please." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Wed, 05 Oct 2016, Daniel Vetter wrote: > Jani Nikula has a patch with a scrip to make the one kernel-doc parser > into a lint/checker pass over the entire kernel. I think that'd would > be more robust instead of trying to approximate the real kerneldoc > parser. Otoh that parser is a horror show of a perl/regex driven state > machine ;-) > > Jani, can you pls digg out these patches? Can't find them right now ... Expanding the massive Cc: with linux-doc list... Here goes. It's a quick hack from months ago, but still seems to somewhat work. At least for the kernel-doc parts. The reStructuredText lint part isn't all that great, and doesn't have mapping to line numbers like the Sphinx kernel-doc extension does. Anyway I'm happy how this integrates with kernel build CHECK and C=3D1/C=3D2. I guess Julia's goal is to automate the *fixing* of some of the error classes from kernel-doc. Not sure how well this could be made to integrate with any of that. BR, Jani. >From 1244efa0f63a7b13795e8c37f81733a3c8bfc56a Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Tue, 31 May 2016 18:11:33 +0300 Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and rst correctness Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Esp= oo Cc: Jani Nikula Simple kernel-doc and reStructuredText lint tool that can be used independently and as a kernel build CHECK tool to validate kernel-doc comments. Independent usage: $ kernel-doc-rst-lint FILE Kernel CHECK usage: $ make CHECK=3Dscripts/kernel-doc-rst-lint C=3D1 # (or C=3D2) Depends on docutils and the rst-lint package https://pypi.python.org/pypi/restructuredtext_lint Signed-off-by: Jani Nikula --- scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 106 insertions(+) create mode 100755 scripts/kernel-doc-rst-lint diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint new file mode 100755 index 000000000000..7e0157679f83 --- /dev/null +++ b/scripts/kernel-doc-rst-lint @@ -0,0 +1,106 @@ +#!/usr/bin/env python +# coding=3Dutf-8 +# +# Copyright =C2=A9 2016 Intel Corporation +# +# Permission is hereby granted, free of charge, to any person obtaining a +# copy of this software and associated documentation files (the "Software"= ), +# to deal in the Software without restriction, including without limitation +# the rights to use, copy, modify, merge, publish, distribute, sublicense, +# and/or sell copies of the Software, and to permit persons to whom the +# Software is furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice (including the next +# paragraph) shall be included in all copies or substantial portions of the +# Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS = OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTH= ER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEAL= INGS +# IN THE SOFTWARE. +# +# Authors: +# Jani Nikula +# +# Simple kernel-doc and reStructuredText lint tool that can be used +# independently and as a kernel build CHECK tool to validate kernel-doc +# comments. +# +# Independent usage: +# $ kernel-doc-rst-lint FILE +# +# Kernel CHECK usage: +# $ make CHECK=3Dscripts/kernel-doc-rst-lint C=3D1 # (or C=3D2) +# +# Depends on docutils and the rst-lint package +# https://pypi.python.org/pypi/restructuredtext_lint +# + +import os +import subprocess +import sys + +from docutils.parsers.rst import directives +from docutils.parsers.rst import Directive +from docutils.parsers.rst import roles +from docutils import nodes, statemachine +import restructuredtext_lint + +class DummyDirective(Directive): + required_argument =3D 1 + optional_arguments =3D 0 + option_spec =3D { } + has_content =3D True + + def run(self): + return [] + +# Fake the Sphinx C Domain directives and roles +directives.register_directive('c:function', DummyDirective) +directives.register_directive('c:type', DummyDirective) +roles.register_generic_role('c:func', nodes.emphasis) +roles.register_generic_role('c:type', nodes.emphasis) + +# We accept but ignore parameters to be compatible with how the kernel bui= ld +# invokes CHECK. +if len(sys.argv) < 2: + sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FILE\n'= ); + sys.exit(1) + +infile =3D sys.argv[len(sys.argv) - 1] +cmd =3D ['scripts/kernel-doc', '-rst', infile] + +try: + p =3D subprocess.Popen(cmd, stdout=3Dsubprocess.PIPE, stderr=3Dsubproc= ess.PIPE, universal_newlines=3DTrue) + out, err =3D p.communicate() + + # python2 needs conversion to unicode. + # python3 with universal_newlines=3DTrue returns strings. + if sys.version_info.major < 3: + out, err =3D unicode(out, 'utf-8'), unicode(err, 'utf-8') + + # kernel-doc errors + sys.stderr.write(err) + if p.returncode !=3D 0: + sys.exit(p.returncode) + + # restructured text errors + lines =3D statemachine.string2lines(out, 8, convert_whitespace=3DTrue) + lint_errors =3D restructuredtext_lint.lint(out, infile) + for error in lint_errors: + # Ignore INFO + if error.level <=3D 1: + continue + + print(error.source + ': ' + error.type + ': ' + error.full_message) + if error.line is not None: + print('Context:') + print('\t' + lines[error.line - 1]) + print('\t' + lines[error.line]) + +except Exception as e: + sys.stderr.write(str(e) + '\n') + sys.exit(1) --=20 2.1.4 --=20 Jani Nikula, Intel Open Source Technology Center From mboxrd@z Thu Jan 1 00:00:00 1970 From: jani.nikula@linux.intel.com (Jani Nikula) Date: Wed, 05 Oct 2016 17:04:43 +0300 Subject: kernel-doc-rst-lint (was: Re: [PATCH 00/15] improve function-level documentation) In-Reply-To: References: <1475351192-27079-1-git-send-email-Julia.Lawall@lip6.fr> Message-ID: <87h98quc1w.fsf@intel.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Wed, 05 Oct 2016, Daniel Vetter wrote: > Jani Nikula has a patch with a scrip to make the one kernel-doc parser > into a lint/checker pass over the entire kernel. I think that'd would > be more robust instead of trying to approximate the real kerneldoc > parser. Otoh that parser is a horror show of a perl/regex driven state > machine ;-) > > Jani, can you pls digg out these patches? Can't find them right now ... Expanding the massive Cc: with linux-doc list... Here goes. It's a quick hack from months ago, but still seems to somewhat work. At least for the kernel-doc parts. The reStructuredText lint part isn't all that great, and doesn't have mapping to line numbers like the Sphinx kernel-doc extension does. Anyway I'm happy how this integrates with kernel build CHECK and C=1/C=2. I guess Julia's goal is to automate the *fixing* of some of the error classes from kernel-doc. Not sure how well this could be made to integrate with any of that. BR, Jani. >>From 1244efa0f63a7b13795e8c37f81733a3c8bfc56a Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Tue, 31 May 2016 18:11:33 +0300 Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and rst correctness Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo Cc: Jani Nikula Simple kernel-doc and reStructuredText lint tool that can be used independently and as a kernel build CHECK tool to validate kernel-doc comments. Independent usage: $ kernel-doc-rst-lint FILE Kernel CHECK usage: $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) Depends on docutils and the rst-lint package https://pypi.python.org/pypi/restructuredtext_lint Signed-off-by: Jani Nikula --- scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100755 scripts/kernel-doc-rst-lint diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint new file mode 100755 index 000000000000..7e0157679f83 --- /dev/null +++ b/scripts/kernel-doc-rst-lint @@ -0,0 +1,106 @@ +#!/usr/bin/env python +# coding=utf-8 +# +# Copyright ? 2016 Intel Corporation +# +# Permission is hereby granted, free of charge, to any person obtaining a +# copy of this software and associated documentation files (the "Software"), +# to deal in the Software without restriction, including without limitation +# the rights to use, copy, modify, merge, publish, distribute, sublicense, +# and/or sell copies of the Software, and to permit persons to whom the +# Software is furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice (including the next +# paragraph) shall be included in all copies or substantial portions of the +# Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +# IN THE SOFTWARE. +# +# Authors: +# Jani Nikula +# +# Simple kernel-doc and reStructuredText lint tool that can be used +# independently and as a kernel build CHECK tool to validate kernel-doc +# comments. +# +# Independent usage: +# $ kernel-doc-rst-lint FILE +# +# Kernel CHECK usage: +# $ make CHECK=scripts/kernel-doc-rst-lint C=1 # (or C=2) +# +# Depends on docutils and the rst-lint package +# https://pypi.python.org/pypi/restructuredtext_lint +# + +import os +import subprocess +import sys + +from docutils.parsers.rst import directives +from docutils.parsers.rst import Directive +from docutils.parsers.rst import roles +from docutils import nodes, statemachine +import restructuredtext_lint + +class DummyDirective(Directive): + required_argument = 1 + optional_arguments = 0 + option_spec = { } + has_content = True + + def run(self): + return [] + +# Fake the Sphinx C Domain directives and roles +directives.register_directive('c:function', DummyDirective) +directives.register_directive('c:type', DummyDirective) +roles.register_generic_role('c:func', nodes.emphasis) +roles.register_generic_role('c:type', nodes.emphasis) + +# We accept but ignore parameters to be compatible with how the kernel build +# invokes CHECK. +if len(sys.argv) < 2: + sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FILE\n'); + sys.exit(1) + +infile = sys.argv[len(sys.argv) - 1] +cmd = ['scripts/kernel-doc', '-rst', infile] + +try: + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) + out, err = p.communicate() + + # python2 needs conversion to unicode. + # python3 with universal_newlines=True returns strings. + if sys.version_info.major < 3: + out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8') + + # kernel-doc errors + sys.stderr.write(err) + if p.returncode != 0: + sys.exit(p.returncode) + + # restructured text errors + lines = statemachine.string2lines(out, 8, convert_whitespace=True) + lint_errors = restructuredtext_lint.lint(out, infile) + for error in lint_errors: + # Ignore INFO + if error.level <= 1: + continue + + print(error.source + ': ' + error.type + ': ' + error.full_message) + if error.line is not None: + print('Context:') + print('\t' + lines[error.line - 1]) + print('\t' + lines[error.line]) + +except Exception as e: + sys.stderr.write(str(e) + '\n') + sys.exit(1) -- 2.1.4 -- Jani Nikula, Intel Open Source Technology Center