Re: [RFC PATCH v1 3/6] kernel-doc: add kerneldoc-lint command

[Jani Nikula <jani.nikula@intel.com>]

On Wed, 25 Jan 2017, Daniel Vetter <daniel@ffwll.ch> wrote:
> On Tue, Jan 24, 2017 at 08:52:41PM +0100, Markus Heiser wrote:
>> this patch adds a command to lint kernel-doc comments.::
>> 
>>   scripts/kerneldoc-lint --help
>> 
>> The lint check include (only) kernel-doc rules described at [1]. It
>> does not check against reST (sphinx-doc) markup used in the kernel-doc
>> comments.  Since reST markups could include depencies to the build-
>> context (e.g. open/closed refs) only a sphinx-doc build can check the
>> reST markup in the context of the document it builds.
>> 
>> With 'kerneldoc-lint' command you can check a single file or a whole
>> folder, e.g:
>> 
>>   scripts/kerneldoc-lint include/drm
>>   ...
>>   scripts/kerneldoc-lint include/media/media-device.h
>> 
>> The lint-implementation is a part of the parser module (kernel_doc.py).
>> The comandline implementation consist only of a argument parser ('opts')
>> which calls the kernel-doc parser with a 'NullTranslator'.::
>> 
>>    parser = kerneldoc.Parser(opts, kerneldoc.NullTranslator())
>> 
>> Latter is also a small example of how-to implement kernel-doc
>> applications with the kernel-doc parser architecture.
>> 
>> [1] https://www.kernel.org/doc/html/latest/doc-guide/kernel-doc.html#writing-kernel-doc-comments
>> 
>> Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
>
> Didn't we have a patch from Jani to gives us a make target doing this? I
> think that'd be even neater ...

Yes, see below. It's simplistic and it has an external dependency, but
it got the job done. And it does not depend on Sphinx; it's just a
kernel-doc and rst lint, not Sphinx lint. Whether that's a good or a bad
thing is debatable.

Anyway, I do think the approach of making 'make CHECK=the-tool C=1' work
is what we should aim at. Markus' patch could probably be made to do
that by accepting the same arguments that are passed to compilers.

BR,
Jani.


>From 96e780dea5fe0cafcb500d7e16a16f85416dea6d Mon Sep 17 00:00:00 2001
From: Jani Nikula <jani.nikula@intel.com>
Date: Tue, 31 May 2016 18:11:33 +0300
Subject: [PATCH] kernel-doc-rst-lint: add tool to check kernel-doc and rst
 correctness
Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo
Cc: Jani Nikula <jani.nikula@intel.com>

Simple kernel-doc and reStructuredText lint tool that can be used
independently and as a kernel build CHECK tool to validate kernel-doc
comments.

Independent usage:
$ kernel-doc-rst-lint FILE

Kernel CHECK usage:
$ make CHECK=scripts/kernel-doc-rst-lint C=1		# (or C=2)

Depends on docutils and the rst-lint package
https://pypi.python.org/pypi/restructuredtext_lint

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc-rst-lint | 106 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 106 insertions(+)
 create mode 100755 scripts/kernel-doc-rst-lint

diff --git a/scripts/kernel-doc-rst-lint b/scripts/kernel-doc-rst-lint
new file mode 100755
index 000000000000..7e0157679f83
--- /dev/null
+++ b/scripts/kernel-doc-rst-lint
@@ -0,0 +1,106 @@
+#!/usr/bin/env python
+# coding=utf-8
+#
+# Copyright © 2016 Intel Corporation
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+#
+# Authors:
+#    Jani Nikula <jani.nikula@intel.com>
+#
+# Simple kernel-doc and reStructuredText lint tool that can be used
+# independently and as a kernel build CHECK tool to validate kernel-doc
+# comments.
+#
+# Independent usage:
+# $ kernel-doc-rst-lint FILE
+#
+# Kernel CHECK usage:
+# $ make CHECK=scripts/kernel-doc-rst-lint C=1		# (or C=2)
+#
+# Depends on docutils and the rst-lint package
+# https://pypi.python.org/pypi/restructuredtext_lint
+#
+
+import os
+import subprocess
+import sys
+
+from docutils.parsers.rst import directives
+from docutils.parsers.rst import Directive
+from docutils.parsers.rst import roles
+from docutils import nodes, statemachine
+import restructuredtext_lint
+
+class DummyDirective(Directive):
+    required_argument = 1
+    optional_arguments = 0
+    option_spec = { }
+    has_content = True
+
+    def run(self):
+        return []
+
+# Fake the Sphinx C Domain directives and roles
+directives.register_directive('c:function', DummyDirective)
+directives.register_directive('c:type', DummyDirective)
+roles.register_generic_role('c:func', nodes.emphasis)
+roles.register_generic_role('c:type', nodes.emphasis)
+
+# We accept but ignore parameters to be compatible with how the kernel build
+# invokes CHECK.
+if len(sys.argv) < 2:
+    sys.stderr.write('usage: kernel-doc-rst-lint [IGNORED OPTIONS] FILE\n');
+    sys.exit(1)
+
+infile = sys.argv[len(sys.argv) - 1]
+cmd = ['scripts/kernel-doc', '-rst', infile]
+
+try:
+    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
+    out, err = p.communicate()
+
+    # python2 needs conversion to unicode.
+    # python3 with universal_newlines=True returns strings.
+    if sys.version_info.major < 3:
+        out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8')
+
+    # kernel-doc errors
+    sys.stderr.write(err)
+    if p.returncode != 0:
+        sys.exit(p.returncode)
+
+    # restructured text errors
+    lines = statemachine.string2lines(out, 8, convert_whitespace=True)
+    lint_errors = restructuredtext_lint.lint(out, infile)
+    for error in lint_errors:
+        # Ignore INFO
+        if error.level <= 1:
+            continue
+
+        print(error.source + ': ' + error.type + ': ' + error.full_message)
+        if error.line is not None:
+            print('Context:')
+            print('\t' + lines[error.line - 1])
+            print('\t' + lines[error.line])
+
+except Exception as e:
+    sys.stderr.write(str(e) + '\n')
+    sys.exit(1)
-- 
2.1.4





-- 
Jani Nikula, Intel Open Source Technology Center