* [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-02-28 17:13 [PATCH 0/6] more kernel-doc patches Daniel Vetter
@ 2017-02-28 17:13 ` Daniel Vetter
2017-03-01 15:56 ` Gabriel Krisman Bertazi
` (2 more replies)
0 siblings, 3 replies; 56+ messages in thread
From: Daniel Vetter @ 2017-02-28 17:13 UTC (permalink / raw)
To: linux-doc
Cc: DRI Development, Markus Heiser, Jonathan Corbet, Jani Nikula,
Mauro Carvalho Chehab, Daniel Vetter
From: Markus Heiser <markus.heiser@darmarit.de>
This patch brings scalable figure, image handling and a concept to
embed *render* markups:
* DOT (http://www.graphviz.org)
* SVG
For image handling use the 'image' replacement::
.. kernel-image:: svg_image.svg
:alt: simple SVG image
For figure handling use the 'figure' replacement::
.. kernel-figure:: svg_image.svg
:alt: simple SVG image
SVG image example
Embed *render* markups (or languages) like Graphviz's **DOT** is
provided by the *render* directive.::
.. kernel-render:: DOT
:alt: foobar digraph
:caption: Embedded **DOT** (Graphviz) code.
digraph foo {
"bar" -> "baz";
}
The *render* directive is a concept to integrate *render* markups and
languages, yet supported markups:
* DOT: render embedded Graphviz's **DOT**
* SVG: render embedded Scalable Vector Graphics (**SVG**)
v2: s/DOC/DOT/ in a few places (by Daniel).
v3: Simplify stuff a bit (by Daniel):
- Remove path detection and setup/check code for that. In
Documentation/media/Makefile we already simply use these tools,
better to have one consolidated check if we want/need one. Also
remove the convertsvg support, we require ImageMagick's convert
already in the doc build, no need for a 2nd fallback.
- Use sphinx for depency tracking, remove hand-rolled version.
- Forward stderr from dot and convert, otherwise debugging issues with
the diagrams is impossible.
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> (v1)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/conf.py | 2 +-
Documentation/doc-guide/hello.dot | 3 +
Documentation/doc-guide/sphinx.rst | 90 ++++++-
Documentation/doc-guide/svg_image.svg | 10 +
Documentation/process/changes.rst | 7 +-
Documentation/sphinx/kfigure.py | 442 ++++++++++++++++++++++++++++++++++
6 files changed, 548 insertions(+), 6 deletions(-)
create mode 100644 Documentation/doc-guide/hello.dot
create mode 100644 Documentation/doc-guide/svg_image.svg
create mode 100644 Documentation/sphinx/kfigure.py
diff --git a/Documentation/conf.py b/Documentation/conf.py
index f6823cf01275..e3f537ce2935 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -34,7 +34,7 @@ from load_config import loadConfig
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain']
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure']
# The name of the math extension changed on Sphinx 1.4
if major == 1 and minor > 3:
diff --git a/Documentation/doc-guide/hello.dot b/Documentation/doc-guide/hello.dot
new file mode 100644
index 000000000000..504621dfc595
--- /dev/null
+++ b/Documentation/doc-guide/hello.dot
@@ -0,0 +1,3 @@
+graph G {
+ Hello -- World
+}
diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index 532d65b70500..b902744ce7dd 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -34,8 +34,10 @@ format-specific subdirectories under ``Documentation/output``.
To generate documentation, Sphinx (``sphinx-build``) must obviously be
installed. For prettier HTML output, the Read the Docs Sphinx theme
-(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is also
-needed. All of these are widely available and packaged in distributions.
+(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need
+``XeLaTeX`` and CairoSVG (http://cairosvg.org) or alternatively ``convert(1)``
+from ImageMagick (https://www.imagemagick.org). All of these are widely
+available and packaged in distributions.
To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make
variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose
@@ -232,3 +234,87 @@ Rendered as:
* .. _`last row`:
- column 3
+
+
+Figures & Images
+================
+
+If you want to add an image, you should use the ``kernel-figure`` and
+``kernel-image`` directives. E.g. to insert a figure with a scalable
+image format use SVG::
+
+ .. kernel-figure:: svg_image.svg
+ :alt: simple SVG image
+
+ SVG image example
+
+.. kernel-figure:: svg_image.svg
+ :alt: simple SVG image
+
+ SVG image example
+
+The kernel figure (and image) directive support **DOT** formated files, see
+
+* DOT: http://graphviz.org/pdf/dotguide.pdf
+* Graphviz: http://www.graphviz.org/content/dot-language
+
+A simple example::
+
+ .. kernel-figure:: hello.dot
+ :alt: hello world
+
+ DOT's hello world example
+
+.. kernel-figure:: hello.dot
+ :alt: hello world
+
+ DOT's hello world example
+
+Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the
+``kernel-render`` directives.::
+
+ .. kernel-render:: DOT
+ :alt: foobar digraph
+ :caption: Embedded **DOT** (Graphviz) code.
+
+ digraph foo {
+ "bar" -> "baz";
+ }
+
+How this will be rendered depends on the installed tools. If Graphviz is
+installed, you will see an vector image. If not the raw markup is inserted as
+*literal-block*.
+
+.. kernel-render:: DOT
+ :alt: foobar digraph
+ :caption: Embedded **DOT** (Graphviz) code.
+
+ digraph foo {
+ "bar" -> "baz";
+ }
+
+The *render* directive has all the options known from the *figure* directive,
+plus option ``caption``. If ``caption`` has a value, a *figure* node is
+inserted. If not, a *image* node is inserted.
+
+Embedded **SVG**::
+
+ .. kernel-render:: SVG
+ :caption: Embedded **SVG** markup.
+ :alt: so-nw-arrow
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <svg xmlns="http://www.w3.org/2000/svg" version="1.1" ...>
+ ...
+ </svg>
+
+.. kernel-render:: SVG
+ :caption: Embedded **SVG** markup.
+ :alt: so-nw-arrow
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <svg xmlns="http://www.w3.org/2000/svg"
+ version="1.1" baseProfile="full" width="70px" height="40px" viewBox="0 0 700 400">
+ <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
+ <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
+ </svg>
diff --git a/Documentation/doc-guide/svg_image.svg b/Documentation/doc-guide/svg_image.svg
new file mode 100644
index 000000000000..5405f85b8137
--- /dev/null
+++ b/Documentation/doc-guide/svg_image.svg
@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- originate: https://commons.wikimedia.org/wiki/File:Variable_Resistor.svg -->
+<svg xmlns="http://www.w3.org/2000/svg"
+ version="1.1" baseProfile="full"
+ width="70px" height="40px" viewBox="0 0 700 400">
+ <line x1="0" y1="200" x2="700" y2="200" stroke="black" stroke-width="20px"/>
+ <rect x="100" y="100" width="500" height="200" fill="white" stroke="black" stroke-width="20px"/>
+ <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
+ <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
+</svg>
diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index 56ce66114665..e4f25038ef65 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -318,9 +318,10 @@ PDF outputs, it is recommended to use version 1.4.6.
.. note::
Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX``
- version 3.14159265. Depending on the distribution, you may also need
- to install a series of ``texlive`` packages that provide the minimal
- set of functionalities required for ``XeLaTex`` to work.
+ version 3.14159265. Depending on the distribution, you may also need to
+ install a series of ``texlive`` packages that provide the minimal set of
+ functionalities required for ``XeLaTex`` to work. For PDF output you'll also
+ need ``convert(1)`` from ImageMagick (https://www.imagemagick.org).
Other tools
-----------
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
new file mode 100644
index 000000000000..8f749d389dae
--- /dev/null
+++ b/Documentation/sphinx/kfigure.py
@@ -0,0 +1,442 @@
+# -*- coding: utf-8; mode: python -*-
+# pylint: disable=C0103
+u"""
+ scalable figure and image handling
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ Sphinx extension which implements scalable image handling.
+
+ :copyright: Copyright (C) 2016 Markus Heiser
+ :license: GPL Version 2, June 1991 see Linux/COPYING for details.
+
+ The build for image formats depence on image's source format and output's
+ destination format. This extension implement methods to simplify image
+ handling from the author's POV. Directives like ``kernel-figure`` implement
+ methods *to* always get the best output-format even if some tools are not
+ installed.For more details take a look at ``convert_image(...)`` which is
+ the core of all conversions.
+
+ * ``.. kernel-image``: for image handling / ``.. image::`` replacement
+
+ * ``.. kernel-figure``: for figure handling / ``.. figure::`` replacement
+
+ * ``.. kernel-render``: for render markup / a concept to embed *render*
+ markups (or languages). Supported markups (see ``RENDER_MARKUP_EXT``)
+
+ + ``DOT``: render embedded Graphviz's **DOC**
+ + ``SVG``: render embedded Scalable Vector Graphics (**SVG**)
+ + ... *developable*
+
+ Used tools:
+
+ * ``dot(1)``: Graphviz (http://www.graphviz.org). If Graphviz is not
+ available, the DOT language is inserted as literal-block.
+
+ * SVG to PDF: To generate PDF, you need at least one of this tools:
+
+ - ``convert(1)``: ImageMagick (https://www.imagemagick.org)
+
+ List of customizations:
+
+ * generate PDF from SVG / used by PDF (LaTeX) builder
+
+ * generate SVG (html-builder) and PDF (latex-builder) from DOT files.
+ DOT: see http://www.graphviz.org/content/dot-language
+
+ """
+
+import os
+from os import path
+import subprocess
+from hashlib import sha1
+import sys
+
+from docutils import nodes
+from docutils.statemachine import ViewList
+from docutils.parsers.rst import directives
+from docutils.parsers.rst.directives import images
+
+from sphinx.directives import patches
+
+__version__ = '1.0'
+
+# simple helper
+# -------------
+
+def mkdir(folder, mode=0o775):
+ if not path.isdir(folder):
+ os.makedirs(folder, mode)
+
+# def debug_handle(self, node): # pylint: disable=W0613
+# from linuxdoc.kernel_doc import CONSOLE
+# CONSOLE()
+
+def pass_handle(self, node): # pylint: disable=W0613
+ pass
+
+# setup conversion tools and sphinx extension
+# -------------------------------------------
+
+def setup(app):
+ # image handling
+ app.add_directive("kernel-image", KernelImage)
+ app.add_node(kernel_image,
+ html = (visit_kernel_image, pass_handle),
+ latex = (visit_kernel_image, pass_handle),
+ texinfo = (visit_kernel_image, pass_handle),
+ text = (visit_kernel_image, pass_handle),
+ man = (visit_kernel_image, pass_handle), )
+
+ # figure handling
+ app.add_directive("kernel-figure", KernelFigure)
+ app.add_node(kernel_figure,
+ html = (visit_kernel_figure, pass_handle),
+ latex = (visit_kernel_figure, pass_handle),
+ texinfo = (visit_kernel_figure, pass_handle),
+ text = (visit_kernel_figure, pass_handle),
+ man = (visit_kernel_figure, pass_handle), )
+
+ # render handling
+ app.add_directive('kernel-render', KernelRender)
+ app.add_node(kernel_render,
+ html = (visit_kernel_render, pass_handle),
+ latex = (visit_kernel_render, pass_handle),
+ texinfo = (visit_kernel_render, pass_handle),
+ text = (visit_kernel_render, pass_handle),
+ man = (visit_kernel_render, pass_handle), )
+
+ return dict(
+ version = __version__,
+ parallel_read_safe = True,
+ parallel_write_safe = True
+ )
+
+# integrate conversion tools
+# --------------------------
+
+RENDER_MARKUP_EXT = {
+ # The '.ext' must be handled by convert_image(..) function's *in_ext* input.
+ # <name> : <.ext>
+ 'DOT' : '.dot'
+ , 'SVG' : '.svg'
+}
+
+def convert_image(img_node, translator): # pylint: disable=R0912
+ """Convert an image node for the builder.
+
+ Different builder prefer different image formats, e.g. *latex* builder
+ prefer PDF while *html* builder prefer SVG format for images.
+
+ This function handles outputs image formats in depence of source the format
+ of the image and the translator's output format. This also means to
+ manipulate/update the *image* dictionary of the builder (``builder.images``)
+
+ """
+ fname, in_ext = path.splitext(path.basename(img_node['uri']))
+ src_fname = path.join(translator.builder.srcdir, img_node['uri'])
+ src_folder = path.dirname(img_node['uri'])
+ out_dir = translator.builder.outdir
+ dst_fname = None
+
+ # in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages
+ verbose = translator.builder.app.verbose
+ warn = translator.builder.warn
+
+ verbose('assert best format for: ' + img_node['uri'])
+
+ if in_ext == '.dot':
+
+ # ----------
+ # handle DOT
+ # ----------
+
+ dst_fname = path.join(out_dir, fname + '.pdf')
+
+ if translator.builder.format == 'html':
+ dst_fname = path.join(out_dir, src_folder, fname + '.svg')
+ else:
+ # all other builder formats will include DOT as raw
+ with open(src_fname, "r") as dot:
+ data = dot.read()
+ node = nodes.literal_block(data, data)
+ img_node.replace_self(node)
+
+
+ elif in_ext == '.svg':
+
+ # ----------
+ # handle SVG
+ # ----------
+
+ if translator.builder.format == 'latex':
+ dst_fname = path.join(out_dir, fname + '.pdf')
+
+ if dst_fname:
+ name = dst_fname[len(out_dir) + 1:]
+ # the builder needs not to copy one more time, so pop it if exists.
+ translator.builder.images.pop(img_node['uri'], None)
+ img_node['uri'] = dst_fname
+ img_node['candidates'] = {'*': dst_fname}
+
+ mkdir(path.dirname(dst_fname))
+
+ if in_ext == '.dot':
+ verbose('convert DOT to: {out}/' + name)
+ dot2format(src_fname, dst_fname)
+
+ elif in_ext == '.svg':
+ verbose('convert SVG to: {out}/' + name)
+ svg2pdf(src_fname, dst_fname)
+
+def dot2format(dot_fname, out_fname):
+ """Converts DOT file to ``out_fname`` using ``dot(1)``.
+
+ * ``dot_fname`` pathname of the input DOT file, including extension ``.dot``
+ * ``out_fname`` pathname of the output file, including format extension
+
+ The *format extension* depends on the ``dot`` command (see ``man dot``
+ option ``-Txxx``). Normally you will use one of the following extensions:
+
+ - ``.ps`` for PostScript,
+ - ``.svg`` or ``svgz`` for Structured Vector Graphics,
+ - ``.fig`` for XFIG graphics and
+ - ``.png`` or ``gif`` for common bitmap graphics.
+
+ """
+ out_format = path.splitext(out_fname)[1][1:]
+ cmd = ['dot', '-T%s' % out_format, dot_fname]
+ exit_code = 42
+ with open(out_fname, "w") as out:
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
+ sys.stderr.write(err)
+
+ exit_code = p.returncode
+ out.flush()
+ return bool(exit_code == 0)
+
+def svg2pdf(svg_fname, pdf_fname):
+ """Converts SVG to PDF with CairoSVG or ``convert(1)`` command.
+
+ Uses ``convert(1)`` from ImageMagick (https://www.imagemagick.org) for
+ conversion. Returns ``True`` on success and ``False`` if an error occurred
+ (e.g. none of the conversion tool is available).
+
+ * ``svg_fname`` pathname of the input SVG file with extension (``.svg``)
+ * ``pdf_name`` pathname of the output PDF file with extension (``.pdf``)
+
+ """
+ cmd = [convert_cmd, svg_fname, pdf_fname]
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
+ sys.stderr.write(err)
+
+ exit_code = p.returncode
+ return bool(exit_code == 0)
+
+
+# image handling
+# ---------------------
+
+def visit_kernel_image(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_image`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0]
+ convert_image(img_node, self)
+
+class kernel_image(nodes.General, nodes.Element):
+ """Node for ``kernel-image`` directive."""
+ pass
+
+class KernelImage(images.Image):
+ u"""KernelImage directive
+
+ Earns everything from ``.. image::`` directive, except *remote URI* and
+ *glob* pattern. The KernelImage wraps a image node into a
+ kernel_image node. See ``visit_kernel_image``.
+ """
+
+ def run(self):
+ env = self.state.document.settings.env
+
+ uri = self.arguments[0]
+ if uri.endswith('.*') or uri.find('://') != -1:
+ raise self.severe(
+ 'Error in "%s: %s": glob pattern and remote images are not allowed'
+ % (self.name, uri))
+
+ # Tell sphinx of the dependency
+ env.note_dependency(os.path.abspath(uri))
+
+ result = images.Image.run(self)
+ if len(result) == 2 or isinstance(result[0], nodes.system_message):
+ return result
+ (image_node,) = result
+ # wrap image node into a kernel_image node / see visitors
+ node = kernel_image('', image_node)
+ return [node]
+
+# figure handling
+# ---------------------
+
+def visit_kernel_figure(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_figure`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0][0]
+ convert_image(img_node, self)
+
+class kernel_figure(nodes.General, nodes.Element):
+ """Node for ``kernel-figure`` directive."""
+
+class KernelFigure(patches.Figure):
+ u"""KernelImage directive
+
+ Earns everything from ``.. figure::`` directive, except *remote URI* and
+ *glob* pattern. The KernelFigure wraps a figure node into a kernel_figure
+ node. See ``visit_kernel_figure``.
+ """
+
+ def run(self):
+ env = self.state.document.settings.env
+
+ uri = self.arguments[0]
+ if uri.endswith('.*') or uri.find('://') != -1:
+ raise self.severe(
+ 'Error in "%s: %s":'
+ ' glob pattern and remote images are not allowed'
+ % (self.name, uri))
+
+ # Tell sphinx of the dependency
+ env.note_dependency(os.path.abspath(uri))
+
+ result = patches.Figure.run(self)
+ if len(result) == 2 or isinstance(result[0], nodes.system_message):
+ return result
+ (figure_node,) = result
+ # wrap figure node into a kernel_figure node / see visitors
+ node = kernel_figure('', figure_node)
+ return [node]
+
+
+# render handling
+# ---------------------
+
+def visit_kernel_render(self, node):
+ """Visitor of the ``kernel_render`` Node.
+
+ If rendering tools available, save the markup of the ``literal_block`` child
+ node into a file and replace the ``literal_block`` node with a new created
+ ``image`` node, pointing to the saved markup file. Afterwards, handle the
+ image child-node with the ``convert_image(...)``.
+ """
+
+ verbose = self.builder.app.verbose
+ warn = self.builder.warn
+ srclang = node.get('srclang')
+
+ verbose('visit kernel-render node lang: "%s"' % (srclang))
+
+ tmp_ext = RENDER_MARKUP_EXT.get(srclang, None)
+ if tmp_ext is None:
+ warn('kernel-render: "%s" unknow / include raw.' % (srclang))
+ return
+
+ literal_block = node[0]
+ code = literal_block.astext()
+
+ if tmp_ext:
+ hashobj = code.encode('utf-8') # str(node.attributes)
+ fname = '%s-%s' % (srclang, sha1(hashobj).hexdigest())
+ tmp_fname = path.join(
+ self.builder.outdir, self.builder.imagedir, fname + tmp_ext)
+
+ if not path.isfile(tmp_fname):
+ mkdir(path.dirname(tmp_fname))
+ with open(tmp_fname, "w") as out:
+ out.write(code)
+
+ image_node = nodes.image(node.rawsource, **node.attributes)
+ image_node['uri'] = tmp_fname
+
+ literal_block.replace_self(image_node)
+ convert_image(image_node, self)
+
+
+class kernel_render(nodes.General, nodes.Inline, nodes.Element):
+ """Node for ``kernel-render`` directive."""
+ pass
+
+class KernelRender(patches.Figure):
+ u"""KernelRender directive
+
+ Render content by external tool. Has all the options known from the
+ *figure* directive, plus option ``caption``. If ``caption`` has a
+ value, a figure node with the *caption* is inserted. If not, a image node is
+ inserted.
+
+ The KernelRender directive wraps the text of the directive into a
+ literal_block node and wraps it into a kernel_render node. See
+ ``visit_kernel_render``.
+ """
+ has_content = True
+ required_arguments = 1
+ optional_arguments = 0
+ final_argument_whitespace = False
+
+ # earn options from 'figure'
+ option_spec = patches.Figure.option_spec.copy()
+ option_spec['caption'] = directives.unchanged
+
+ def run(self):
+ return [self.build_node()]
+
+ def build_node(self):
+
+ srclang = self.arguments[0].strip()
+ if srclang not in RENDER_MARKUP_EXT.keys():
+ return [self.state_machine.reporter.warning(
+ 'Unknow source language "%s", use one of: %s.' % (
+ srclang, ",".join(RENDER_MARKUP_EXT.keys())),
+ line=self.lineno)]
+
+ code = '\n'.join(self.content)
+ if not code.strip():
+ return [self.state_machine.reporter.warning(
+ 'Ignoring "%s" directive without content.' % (
+ self.name),
+ line=self.lineno)]
+
+ node = kernel_render()
+ node['alt'] = self.options.get('alt','')
+ node['srclang'] = srclang
+ literal_node = nodes.literal_block(code, code)
+ node += literal_node
+
+ caption = self.options.get('caption')
+ if caption:
+ # parse cation's content
+ parsed = nodes.Element()
+ self.state.nested_parse(
+ ViewList([caption], source=''), self.content_offset, parsed)
+ caption_node = nodes.caption(
+ parsed[0].rawsource, '', *parsed[0].children)
+ caption_node.source = parsed[0].source
+ caption_node.line = parsed[0].line
+
+ figure_node = nodes.figure('', node)
+ for k,v in self.options.items():
+ figure_node[k] = v
+ figure_node += caption_node
+
+ node = figure_node
+
+ return node
+
--
2.11.0
^ permalink raw reply related [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-02-28 17:13 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
@ 2017-03-01 15:56 ` Gabriel Krisman Bertazi
2017-03-02 7:12 ` Daniel Vetter
2017-03-02 12:26 ` Laurent Pinchart
2017-03-06 10:12 ` Daniel Vetter
2 siblings, 1 reply; 56+ messages in thread
From: Gabriel Krisman Bertazi @ 2017-03-01 15:56 UTC (permalink / raw)
To: Daniel Vetter
Cc: Markus Heiser, Jonathan Corbet, linux-doc, DRI Development,
Mauro Carvalho Chehab, Daniel Vetter
Daniel Vetter <daniel.vetter@ffwll.ch> writes:
Hi Daniel,
> + if dst_fname:
> + name = dst_fname[len(out_dir) + 1:]
> + # the builder needs not to copy one more time, so pop it if exists.
> + translator.builder.images.pop(img_node['uri'], None)
> + img_node['uri'] = dst_fname
> + img_node['candidates'] = {'*': dst_fname}
> +
> + mkdir(path.dirname(dst_fname))
> +
> + if in_ext == '.dot':
> + verbose('convert DOT to: {out}/' + name)
> + dot2format(src_fname, dst_fname)
> +
> + elif in_ext == '.svg':
> + verbose('convert SVG to: {out}/' + name)
> + svg2pdf(src_fname, dst_fname)
Small nit, but, shouldn't you add dst_fname to img_node only if
dot2format or svg2pdf was successful?
Thanks,
--
Gabriel Krisman Bertazi
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-01 15:56 ` Gabriel Krisman Bertazi
@ 2017-03-02 7:12 ` Daniel Vetter
2017-03-02 8:23 ` Markus Heiser
0 siblings, 1 reply; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 7:12 UTC (permalink / raw)
To: Gabriel Krisman Bertazi
Cc: Markus Heiser, Jonathan Corbet, Daniel Vetter, linux-doc,
DRI Development, Mauro Carvalho Chehab, Daniel Vetter
On Wed, Mar 01, 2017 at 12:56:36PM -0300, Gabriel Krisman Bertazi wrote:
> Daniel Vetter <daniel.vetter@ffwll.ch> writes:
>
> Hi Daniel,
>
> > + if dst_fname:
> > + name = dst_fname[len(out_dir) + 1:]
> > + # the builder needs not to copy one more time, so pop it if exists.
> > + translator.builder.images.pop(img_node['uri'], None)
> > + img_node['uri'] = dst_fname
> > + img_node['candidates'] = {'*': dst_fname}
> > +
> > + mkdir(path.dirname(dst_fname))
> > +
> > + if in_ext == '.dot':
> > + verbose('convert DOT to: {out}/' + name)
> > + dot2format(src_fname, dst_fname)
> > +
> > + elif in_ext == '.svg':
> > + verbose('convert SVG to: {out}/' + name)
> > + svg2pdf(src_fname, dst_fname)
>
> Small nit, but, shouldn't you add dst_fname to img_node only if
> dot2format or svg2pdf was successful?
I kinda assumed they're always successful, since the media documentation
build already requires them.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 7:12 ` Daniel Vetter
@ 2017-03-02 8:23 ` Markus Heiser
0 siblings, 0 replies; 56+ messages in thread
From: Markus Heiser @ 2017-03-02 8:23 UTC (permalink / raw)
To: Daniel Vetter
Cc: Jonathan Corbet, Daniel Vetter, linux-doc, DRI Development,
Mauro Carvalho Chehab, Daniel Vetter, Gabriel Krisman Bertazi
Am 02.03.2017 um 08:12 schrieb Daniel Vetter <daniel@ffwll.ch>:
> On Wed, Mar 01, 2017 at 12:56:36PM -0300, Gabriel Krisman Bertazi wrote:
>> Daniel Vetter <daniel.vetter@ffwll.ch> writes:
>>
>> Hi Daniel,
>>
>>> + if dst_fname:
>>> + name = dst_fname[len(out_dir) + 1:]
>>> + # the builder needs not to copy one more time, so pop it if exists.
>>> + translator.builder.images.pop(img_node['uri'], None)
>>> + img_node['uri'] = dst_fname
>>> + img_node['candidates'] = {'*': dst_fname}
>>> +
>>> + mkdir(path.dirname(dst_fname))
>>> +
>>> + if in_ext == '.dot':
>>> + verbose('convert DOT to: {out}/' + name)
>>> + dot2format(src_fname, dst_fname)
>>> +
>>> + elif in_ext == '.svg':
>>> + verbose('convert SVG to: {out}/' + name)
>>> + svg2pdf(src_fname, dst_fname)
>>
>> Small nit, but, shouldn't you add dst_fname to img_node only if
>> dot2format or svg2pdf was successful?
>
> I kinda assumed they're always successful, since the media documentation
> build already requires them.
SHORT:
At this time (Phase 1: Reading), the internal doctree is build and
with these two lines:
>>> + img_node['uri'] = dst_fname
>>> + img_node['candidates'] = {'*': dst_fname}
the image is added as wildcard "*" to the doctree.
Later (Phase 4: Writing), when the output is generated from
doctree, the (pdf-/html-) builder will search for the best
existing "*" candidate, e.g. "*.svg" for html and "*.pdf"
for LaTeX.
MORE VERBOSE:
To see/understand the whole play we have to know that Sphinx/docutils
has Build Phases:
* http://www.sphinx-doc.org/en/stable/extdev/tutorial.html#build-phases
"""
Phase 1: Reading
In Phase 1, all source files (and on subsequent builds, those that are new or changed) are read and parsed. This is the phase where directives and roles are encountered by docutils, and the corresponding code is executed. The output of this phase is a doctree for each source file"""
...
Phase 4: Writing
This phase converts the resolved doctrees to the desired output format, such as HTML or LaTeX. This happens via a so-called docutils writer that visits the individual nodes of each doctree and produces some output in the process.
"""
The visitor for handling kernel_image, kernel_render and kernel_figure
directives are analog, so lets look at the simple one::
+def visit_kernel_image(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_image`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0]
+ convert_image(img_node, self)
this is the visitor, for building the internal doctree. The visitor
calls convert_image() with img_node as first argument.
The convert_image() first evals a new dst_fname for image
types which are covered by:
+ dst_fname = None
...
+ if in_ext == '.dot':
+ dst_fname = path.join(out_dir, fname + '.pdf')
....
+
+ elif in_ext == '.svg':
+ if translator.builder.format == 'latex':
+ dst_fname = path.join(out_dir, fname + '.pdf')
Consider that dst_fname is always in the out_dir. So, if this image type
is covered by convert_image(), we have to remove the old image from
builder's image-stack (the stack is outside of the node-tree).
+ if dst_fname:
+ name = dst_fname[len(out_dir) + 1:]
+ # the builder needs not to copy one more time, so pop it if exists.
+ translator.builder.images.pop(img_node['uri'], None)
+ img_node['uri'] = dst_fname
+ img_node['candidates'] = {'*': dst_fname}
So you might already guess, this image handling method is bonded
only to Phase 1 and we do not need to touch any builder (Phase 4).
The central point is the last line:
+ img_node['candidates'] = {'*': dst_fname}
The other important thing is, that generated files (dst_fname) are
always in the out_dir. This is needed, since kernel-build should not
pollute the src_folder.
@Daniel: Thanks a lot for your:
v2: s/DOC/DOT/ in a few places (by Daniel).
v3: Simplify stuff a bit (by Daniel):
I haven't had time to test and I can't promise that I will
find the time for. So, for the first, take my Acked-by.
--Markus--
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-02-28 17:13 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
2017-03-01 15:56 ` Gabriel Krisman Bertazi
@ 2017-03-02 12:26 ` Laurent Pinchart
2017-03-02 13:54 ` Daniel Vetter
2017-03-06 10:12 ` Daniel Vetter
2 siblings, 1 reply; 56+ messages in thread
From: Laurent Pinchart @ 2017-03-02 12:26 UTC (permalink / raw)
To: dri-devel
Cc: Markus Heiser, Jonathan Corbet, Daniel Vetter, linux-doc,
Mauro Carvalho Chehab, Daniel Vetter
Hi Daniel,
Thank you for the patch.
With this applied, I get
make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
SPHINX htmldocs --> file:///home/laurent/src/iob/renesas/linux64/Documentation/output
PARSE include/uapi/linux/videodev2.h
Running Sphinx v1.3.1
Extension error:
Could not import extension kfigure (exception: cannot import name patches)
make[2]: *** [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70: htmldocs] Error 1
make[1]: *** [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64'
make: *** [Makefile:152: sub-make] Error 2
sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump the
minimum required version I think a notice is needed.
On Tuesday 28 Feb 2017 18:13:15 Daniel Vetter wrote:
> From: Markus Heiser <markus.heiser@darmarit.de>
>
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
>
> * DOT (http://www.graphviz.org)
> * SVG
>
> For image handling use the 'image' replacement::
>
> .. kernel-image:: svg_image.svg
>
> :alt: simple SVG image
>
> For figure handling use the 'figure' replacement::
>
> .. kernel-figure:: svg_image.svg
>
> :alt: simple SVG image
>
> SVG image example
>
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
>
> .. kernel-render:: DOT
>
> :alt: foobar digraph
> :caption: Embedded **DOT** (Graphviz) code.
>
> digraph foo {
> "bar" -> "baz";
> }
>
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
>
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
>
> v2: s/DOC/DOT/ in a few places (by Daniel).
>
> v3: Simplify stuff a bit (by Daniel):
>
> - Remove path detection and setup/check code for that. In
> Documentation/media/Makefile we already simply use these tools,
> better to have one consolidated check if we want/need one. Also
> remove the convertsvg support, we require ImageMagick's convert
> already in the doc build, no need for a 2nd fallback.
>
> - Use sphinx for depency tracking, remove hand-rolled version.
>
> - Forward stderr from dot and convert, otherwise debugging issues with
> the diagrams is impossible.
>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: linux-doc@vger.kernel.org
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> (v1)
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> ---
> Documentation/conf.py | 2 +-
> Documentation/doc-guide/hello.dot | 3 +
> Documentation/doc-guide/sphinx.rst | 90 ++++++-
> Documentation/doc-guide/svg_image.svg | 10 +
> Documentation/process/changes.rst | 7 +-
> Documentation/sphinx/kfigure.py | 442 ++++++++++++++++++++++++++++++
> 6 files changed, 548 insertions(+), 6 deletions(-)
> create mode 100644 Documentation/doc-guide/hello.dot
> create mode 100644 Documentation/doc-guide/svg_image.svg
> create mode 100644 Documentation/sphinx/kfigure.py
--
Regards,
Laurent Pinchart
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 12:26 ` Laurent Pinchart
@ 2017-03-02 13:54 ` Daniel Vetter
2017-03-02 14:14 ` Laurent Pinchart
0 siblings, 1 reply; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 13:54 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Markus Heiser, Jonathan Corbet, Linux Doc Mailing List, dri-devel,
Mauro Carvalho Chehab, Daniel Vetter
On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
> Hi Daniel,
>
> Thank you for the patch.
>
> With this applied, I get
>
> make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
> SPHINX htmldocs --> file:///home/laurent/src/iob/renesas/linux64/Documentation/output
> PARSE include/uapi/linux/videodev2.h
> Running Sphinx v1.3.1
>
> Extension error:
> Could not import extension kfigure (exception: cannot import name patches)
> make[2]: *** [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70: htmldocs] Error 1
> make[1]: *** [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
> make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64'
> make: *** [Makefile:152: sub-make] Error 2
>
> sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump the
> minimum required version I think a notice is needed.
Ugh. But this also goes completely over my head, no idea whether we
must require sphinx 1.4 (it was released Mar 28, 2016), or whether
there's some way to work around this ... Halp?
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 13:54 ` Daniel Vetter
@ 2017-03-02 14:14 ` Laurent Pinchart
2017-03-02 14:58 ` Markus Heiser
0 siblings, 1 reply; 56+ messages in thread
From: Laurent Pinchart @ 2017-03-02 14:14 UTC (permalink / raw)
To: Daniel Vetter
Cc: Markus Heiser, Jonathan Corbet, Linux Doc Mailing List, dri-devel,
Mauro Carvalho Chehab, Daniel Vetter
Hi Daniel,
On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:
> On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:
> > Hi Daniel,
> >
> > Thank you for the patch.
> >
> > With this applied, I get
> >
> > make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
> >
> > SPHINX htmldocs -->
> > file:///home/laurent/src/iob/renesas/linux64/Documentation/output PARSE
> > include/uapi/linux/videodev2.h
> >
> > Running Sphinx v1.3.1
> >
> > Extension error:
> > Could not import extension kfigure (exception: cannot import name patches)
> > make[2]: ***
> > [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
> > htmldocs] Error 1 make[1]: ***
> > [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
> > make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64' make:
> > *** [Makefile:152: sub-make] Error 2
> >
> > sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump
> > the minimum required version I think a notice is needed.
>
> Ugh. But this also goes completely over my head, no idea whether we
> must require sphinx 1.4 (it was released Mar 28, 2016), or whether
> there's some way to work around this ... Halp?
I'm not a Sphinx expert so I don't know, but what I can tell is that copying
the patches.py from Sphinx 1.4 to Documentation/sphinx/ and modifying
kfigure.py to import it from there fixes the build. There's thus no extra
depencency on Sphinx 1.4 (or newer).
I'm not sure we want to set a precedent by copying part of the Sphinx source
code to the kernel tree (or inlining the single small function that the module
provides), and I'll let someone more knowledgeable than me decide how to
proceed.
--
Regards,
Laurent Pinchart
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 14:14 ` Laurent Pinchart
@ 2017-03-02 14:58 ` Markus Heiser
2017-03-02 15:11 ` Daniel Vetter
0 siblings, 1 reply; 56+ messages in thread
From: Markus Heiser @ 2017-03-02 14:58 UTC (permalink / raw)
To: Laurent Pinchart, Daniel Vetter
Cc: Mauro Carvalho Chehab, Daniel Vetter, Jonathan Corbet, dri-devel,
Linux Doc Mailing List
Hi Daniel, Laurent
Am 02.03.2017 um 15:14 schrieb Laurent Pinchart <laurent.pinchart@ideasonboard.com>:
> Hi Daniel,
>
> On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:
>> On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:
>>> Hi Daniel,
>>>
>>> Thank you for the patch.
>>>
>>> With this applied, I get
>>>
>>> make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
>>>
>>> SPHINX htmldocs -->
>>> file:///home/laurent/src/iob/renesas/linux64/Documentation/output PARSE
>>> include/uapi/linux/videodev2.h
>>>
>>> Running Sphinx v1.3.1
>>>
>>> Extension error:
>>> Could not import extension kfigure (exception: cannot import name patches)
>>> make[2]: ***
>>> [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
>>> htmldocs] Error 1 make[1]: ***
>>> [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
>>> make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64' make:
>>> *** [Makefile:152: sub-make] Error 2
>>>
>>> sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump
>>> the minimum required version I think a notice is needed.
>>
>> Ugh. But this also goes completely over my head, no idea whether we
>> must require sphinx 1.4 (it was released Mar 28, 2016), or whether
>> there's some way to work around this ... Halp?
>
> I'm not a Sphinx expert so I don't know, but what I can tell is that copying
> the patches.py from Sphinx 1.4 to Documentation/sphinx/ and modifying
> kfigure.py to import it from there fixes the build. There's thus no extra
> depencency on Sphinx 1.4 (or newer).
>
> I'm not sure we want to set a precedent by copying part of the Sphinx source
> code to the kernel tree (or inlining the single small function that the module
> provides), and I'll let someone more knowledgeable than me decide how to
> proceed.
Aargh ... we need virtualenv! For interim something like the following
might help. In file Documentation/sphinx/kfigure.py edit the imports
...
from docutils.parsers.rst.directives import images
try:
from sphinx.directives.patches import Figure
except ImportError:
Figure = images.Figure
...
And fix the class definition, so it use 'Figure' and no
longer 'patch.Figure'::
...
-class KernelFigure(patches.Figure):
+class KernelFigure(Figure):
...
Sorry that I have not yet the time to send you a decent and tested
patch. Do you like to test my suggestion? / thanks!
-- Markus --
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 14:58 ` Markus Heiser
@ 2017-03-02 15:11 ` Daniel Vetter
2017-03-02 15:17 ` Laurent Pinchart
` (2 more replies)
0 siblings, 3 replies; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 15:11 UTC (permalink / raw)
To: Markus Heiser
Cc: Jonathan Corbet, Daniel Vetter, Linux Doc Mailing List, dri-devel,
Mauro Carvalho Chehab, Laurent Pinchart, Daniel Vetter
On Thu, Mar 02, 2017 at 03:58:36PM +0100, Markus Heiser wrote:
> Hi Daniel, Laurent
>
> Am 02.03.2017 um 15:14 schrieb Laurent Pinchart <laurent.pinchart@ideasonboard.com>:
>
> > Hi Daniel,
> >
> > On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:
> >> On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:
> >>> Hi Daniel,
> >>>
> >>> Thank you for the patch.
> >>>
> >>> With this applied, I get
> >>>
> >>> make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
> >>>
> >>> SPHINX htmldocs -->
> >>> file:///home/laurent/src/iob/renesas/linux64/Documentation/output PARSE
> >>> include/uapi/linux/videodev2.h
> >>>
> >>> Running Sphinx v1.3.1
> >>>
> >>> Extension error:
> >>> Could not import extension kfigure (exception: cannot import name patches)
> >>> make[2]: ***
> >>> [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
> >>> htmldocs] Error 1 make[1]: ***
> >>> [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
> >>> make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64' make:
> >>> *** [Makefile:152: sub-make] Error 2
> >>>
> >>> sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump
> >>> the minimum required version I think a notice is needed.
> >>
> >> Ugh. But this also goes completely over my head, no idea whether we
> >> must require sphinx 1.4 (it was released Mar 28, 2016), or whether
> >> there's some way to work around this ... Halp?
> >
> > I'm not a Sphinx expert so I don't know, but what I can tell is that copying
> > the patches.py from Sphinx 1.4 to Documentation/sphinx/ and modifying
> > kfigure.py to import it from there fixes the build. There's thus no extra
> > depencency on Sphinx 1.4 (or newer).
> >
> > I'm not sure we want to set a precedent by copying part of the Sphinx source
> > code to the kernel tree (or inlining the single small function that the module
> > provides), and I'll let someone more knowledgeable than me decide how to
> > proceed.
>
>
> Aargh ... we need virtualenv! For interim something like the following
> might help. In file Documentation/sphinx/kfigure.py edit the imports
>
> ...
> from docutils.parsers.rst.directives import images
> try:
> from sphinx.directives.patches import Figure
> except ImportError:
> Figure = images.Figure
> ...
>
> And fix the class definition, so it use 'Figure' and no
> longer 'patch.Figure'::
>
> ...
> -class KernelFigure(patches.Figure):
> +class KernelFigure(Figure):
> ...
>
> Sorry that I have not yet the time to send you a decent and tested
> patch. Do you like to test my suggestion? / thanks!
I'll give it a shot at implementing it, but I can't (easily at least) test
on sphinx 1.3.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* [PATCH 1/6] doc: Explain light-handed markup preference a bit better
@ 2017-03-02 15:16 Daniel Vetter
2017-03-02 15:16 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
` (5 more replies)
0 siblings, 6 replies; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 15:16 UTC (permalink / raw)
To: linux-doc
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Peter Zijlstra,
Daniel Vetter, Christoph Hellwig, DRI Development, Daniel Vetter
We already had a super-short blurb, but worth extending it I think:
We're still pretty far away from anything like a consensus, but
there's clearly a lot of people who prefer an as-light as possible
approach to converting existing .txt files to .rst. Make sure this is
properly taken into account and clear.
Motivated by discussions with Peter and Christoph and others.
v2:
- Mention that existing headings should be kept when converting
existing .txt files (Mauro).
- Explain that we prefer :: for quoting code, it's easier on the
eyes (Mauro).
- Explain that blindly converting outdated docs is harmful. Motived
by comments Peter did in our discussion.
v3: Make the explanations around fixed-width quoting more concise
(Jani).
v4:
- Rebase onto docs-4.10.
- Go with the more terse recommendation from Jani, defer to the much
more detailed conversion guide Mauro is working on for details.
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/doc-guide/sphinx.rst | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index 96fe7ccb2c67..532d65b70500 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -73,7 +73,16 @@ Specific guidelines for the kernel documentation
Here are some specific guidelines for the kernel documentation:
-* Please don't go overboard with reStructuredText markup. Keep it simple.
+* Please don't go overboard with reStructuredText markup. Keep it
+ simple. For the most part the documentation should be plain text with
+ just enough consistency in formatting that it can be converted to
+ other formats.
+
+* Please keep the formatting changes minimal when converting existing
+ documentation to reStructuredText.
+
+* Also update the content, not just the formatting, when converting
+ documentation.
* Please stick to this order of heading adornments:
@@ -103,6 +112,12 @@ Here are some specific guidelines for the kernel documentation:
the order as encountered."), having the higher levels the same overall makes
it easier to follow the documents.
+* For inserting fixed width text blocks (for code examples, use case
+ examples, etc.), use ``::`` for anything that doesn't really benefit
+ from syntax highlighting, especially short snippets. Use
+ ``.. code-block:: <language>`` for longer code blocks that benefit
+ from highlighting.
+
the C domain
------------
--
2.11.0
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:16 [PATCH 1/6] doc: Explain light-handed markup preference a bit better Daniel Vetter
@ 2017-03-02 15:16 ` Daniel Vetter
2017-03-02 15:20 ` Laurent Pinchart
` (2 more replies)
2017-03-02 15:16 ` [PATCH 3/6] drm/doc: Add KMS overview graphs Daniel Vetter
` (4 subsequent siblings)
5 siblings, 3 replies; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 15:16 UTC (permalink / raw)
To: linux-doc
Cc: Markus Heiser, Jonathan Corbet, DRI Development,
Mauro Carvalho Chehab, Laurent Pinchart, Daniel Vetter
From: Markus Heiser <markus.heiser@darmarit.de>
This patch brings scalable figure, image handling and a concept to
embed *render* markups:
* DOT (http://www.graphviz.org)
* SVG
For image handling use the 'image' replacement::
.. kernel-image:: svg_image.svg
:alt: simple SVG image
For figure handling use the 'figure' replacement::
.. kernel-figure:: svg_image.svg
:alt: simple SVG image
SVG image example
Embed *render* markups (or languages) like Graphviz's **DOT** is
provided by the *render* directive.::
.. kernel-render:: DOT
:alt: foobar digraph
:caption: Embedded **DOT** (Graphviz) code.
digraph foo {
"bar" -> "baz";
}
The *render* directive is a concept to integrate *render* markups and
languages, yet supported markups:
* DOT: render embedded Graphviz's **DOT**
* SVG: render embedded Scalable Vector Graphics (**SVG**)
v2: s/DOC/DOT/ in a few places (by Daniel).
v3: Simplify stuff a bit (by Daniel):
- Remove path detection and setup/check code for that. In
Documentation/media/Makefile we already simply use these tools,
better to have one consolidated check if we want/need one. Also
remove the convertsvg support, we require ImageMagick's convert
already in the doc build, no need for a 2nd fallback.
- Use sphinx for depency tracking, remove hand-rolled version.
- Forward stderr from dot and convert, otherwise debugging issues with
the diagrams is impossible.
v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
Implement Markus suggestion for backwards compatability with earlier
releases. Laurent reported this, running sphinx 1.3. Solution entirely
untested.
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Cc: Markus Heiser <markus.heiser@darmarit.de>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> (v1)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/conf.py | 2 +-
Documentation/doc-guide/hello.dot | 3 +
Documentation/doc-guide/sphinx.rst | 90 ++++++-
Documentation/doc-guide/svg_image.svg | 10 +
Documentation/process/changes.rst | 7 +-
Documentation/sphinx/kfigure.py | 444 ++++++++++++++++++++++++++++++++++
6 files changed, 550 insertions(+), 6 deletions(-)
create mode 100644 Documentation/doc-guide/hello.dot
create mode 100644 Documentation/doc-guide/svg_image.svg
create mode 100644 Documentation/sphinx/kfigure.py
diff --git a/Documentation/conf.py b/Documentation/conf.py
index f6823cf01275..e3f537ce2935 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -34,7 +34,7 @@ from load_config import loadConfig
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain']
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure']
# The name of the math extension changed on Sphinx 1.4
if major == 1 and minor > 3:
diff --git a/Documentation/doc-guide/hello.dot b/Documentation/doc-guide/hello.dot
new file mode 100644
index 000000000000..504621dfc595
--- /dev/null
+++ b/Documentation/doc-guide/hello.dot
@@ -0,0 +1,3 @@
+graph G {
+ Hello -- World
+}
diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index 532d65b70500..b902744ce7dd 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -34,8 +34,10 @@ format-specific subdirectories under ``Documentation/output``.
To generate documentation, Sphinx (``sphinx-build``) must obviously be
installed. For prettier HTML output, the Read the Docs Sphinx theme
-(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is also
-needed. All of these are widely available and packaged in distributions.
+(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need
+``XeLaTeX`` and CairoSVG (http://cairosvg.org) or alternatively ``convert(1)``
+from ImageMagick (https://www.imagemagick.org). All of these are widely
+available and packaged in distributions.
To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make
variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose
@@ -232,3 +234,87 @@ Rendered as:
* .. _`last row`:
- column 3
+
+
+Figures & Images
+================
+
+If you want to add an image, you should use the ``kernel-figure`` and
+``kernel-image`` directives. E.g. to insert a figure with a scalable
+image format use SVG::
+
+ .. kernel-figure:: svg_image.svg
+ :alt: simple SVG image
+
+ SVG image example
+
+.. kernel-figure:: svg_image.svg
+ :alt: simple SVG image
+
+ SVG image example
+
+The kernel figure (and image) directive support **DOT** formated files, see
+
+* DOT: http://graphviz.org/pdf/dotguide.pdf
+* Graphviz: http://www.graphviz.org/content/dot-language
+
+A simple example::
+
+ .. kernel-figure:: hello.dot
+ :alt: hello world
+
+ DOT's hello world example
+
+.. kernel-figure:: hello.dot
+ :alt: hello world
+
+ DOT's hello world example
+
+Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the
+``kernel-render`` directives.::
+
+ .. kernel-render:: DOT
+ :alt: foobar digraph
+ :caption: Embedded **DOT** (Graphviz) code.
+
+ digraph foo {
+ "bar" -> "baz";
+ }
+
+How this will be rendered depends on the installed tools. If Graphviz is
+installed, you will see an vector image. If not the raw markup is inserted as
+*literal-block*.
+
+.. kernel-render:: DOT
+ :alt: foobar digraph
+ :caption: Embedded **DOT** (Graphviz) code.
+
+ digraph foo {
+ "bar" -> "baz";
+ }
+
+The *render* directive has all the options known from the *figure* directive,
+plus option ``caption``. If ``caption`` has a value, a *figure* node is
+inserted. If not, a *image* node is inserted.
+
+Embedded **SVG**::
+
+ .. kernel-render:: SVG
+ :caption: Embedded **SVG** markup.
+ :alt: so-nw-arrow
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <svg xmlns="http://www.w3.org/2000/svg" version="1.1" ...>
+ ...
+ </svg>
+
+.. kernel-render:: SVG
+ :caption: Embedded **SVG** markup.
+ :alt: so-nw-arrow
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <svg xmlns="http://www.w3.org/2000/svg"
+ version="1.1" baseProfile="full" width="70px" height="40px" viewBox="0 0 700 400">
+ <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
+ <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
+ </svg>
diff --git a/Documentation/doc-guide/svg_image.svg b/Documentation/doc-guide/svg_image.svg
new file mode 100644
index 000000000000..5405f85b8137
--- /dev/null
+++ b/Documentation/doc-guide/svg_image.svg
@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- originate: https://commons.wikimedia.org/wiki/File:Variable_Resistor.svg -->
+<svg xmlns="http://www.w3.org/2000/svg"
+ version="1.1" baseProfile="full"
+ width="70px" height="40px" viewBox="0 0 700 400">
+ <line x1="0" y1="200" x2="700" y2="200" stroke="black" stroke-width="20px"/>
+ <rect x="100" y="100" width="500" height="200" fill="white" stroke="black" stroke-width="20px"/>
+ <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
+ <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
+</svg>
diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index 56ce66114665..e4f25038ef65 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -318,9 +318,10 @@ PDF outputs, it is recommended to use version 1.4.6.
.. note::
Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX``
- version 3.14159265. Depending on the distribution, you may also need
- to install a series of ``texlive`` packages that provide the minimal
- set of functionalities required for ``XeLaTex`` to work.
+ version 3.14159265. Depending on the distribution, you may also need to
+ install a series of ``texlive`` packages that provide the minimal set of
+ functionalities required for ``XeLaTex`` to work. For PDF output you'll also
+ need ``convert(1)`` from ImageMagick (https://www.imagemagick.org).
Other tools
-----------
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
new file mode 100644
index 000000000000..cbb23e089186
--- /dev/null
+++ b/Documentation/sphinx/kfigure.py
@@ -0,0 +1,444 @@
+# -*- coding: utf-8; mode: python -*-
+# pylint: disable=C0103
+u"""
+ scalable figure and image handling
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ Sphinx extension which implements scalable image handling.
+
+ :copyright: Copyright (C) 2016 Markus Heiser
+ :license: GPL Version 2, June 1991 see Linux/COPYING for details.
+
+ The build for image formats depence on image's source format and output's
+ destination format. This extension implement methods to simplify image
+ handling from the author's POV. Directives like ``kernel-figure`` implement
+ methods *to* always get the best output-format even if some tools are not
+ installed.For more details take a look at ``convert_image(...)`` which is
+ the core of all conversions.
+
+ * ``.. kernel-image``: for image handling / ``.. image::`` replacement
+
+ * ``.. kernel-figure``: for figure handling / ``.. figure::`` replacement
+
+ * ``.. kernel-render``: for render markup / a concept to embed *render*
+ markups (or languages). Supported markups (see ``RENDER_MARKUP_EXT``)
+
+ + ``DOT``: render embedded Graphviz's **DOC**
+ + ``SVG``: render embedded Scalable Vector Graphics (**SVG**)
+ + ... *developable*
+
+ Used tools:
+
+ * ``dot(1)``: Graphviz (http://www.graphviz.org). If Graphviz is not
+ available, the DOT language is inserted as literal-block.
+
+ * SVG to PDF: To generate PDF, you need at least one of this tools:
+
+ - ``convert(1)``: ImageMagick (https://www.imagemagick.org)
+
+ List of customizations:
+
+ * generate PDF from SVG / used by PDF (LaTeX) builder
+
+ * generate SVG (html-builder) and PDF (latex-builder) from DOT files.
+ DOT: see http://www.graphviz.org/content/dot-language
+
+ """
+
+import os
+from os import path
+import subprocess
+from hashlib import sha1
+import sys
+
+from docutils import nodes
+from docutils.statemachine import ViewList
+from docutils.parsers.rst import directives
+from docutils.parsers.rst.directives import images
+try:
+ from sphinx.directives.patches import Figure
+except ImportError:
+ Figure = images.Figure
+
+__version__ = '1.0'
+
+# simple helper
+# -------------
+
+def mkdir(folder, mode=0o775):
+ if not path.isdir(folder):
+ os.makedirs(folder, mode)
+
+# def debug_handle(self, node): # pylint: disable=W0613
+# from linuxdoc.kernel_doc import CONSOLE
+# CONSOLE()
+
+def pass_handle(self, node): # pylint: disable=W0613
+ pass
+
+# setup conversion tools and sphinx extension
+# -------------------------------------------
+
+def setup(app):
+ # image handling
+ app.add_directive("kernel-image", KernelImage)
+ app.add_node(kernel_image,
+ html = (visit_kernel_image, pass_handle),
+ latex = (visit_kernel_image, pass_handle),
+ texinfo = (visit_kernel_image, pass_handle),
+ text = (visit_kernel_image, pass_handle),
+ man = (visit_kernel_image, pass_handle), )
+
+ # figure handling
+ app.add_directive("kernel-figure", KernelFigure)
+ app.add_node(kernel_figure,
+ html = (visit_kernel_figure, pass_handle),
+ latex = (visit_kernel_figure, pass_handle),
+ texinfo = (visit_kernel_figure, pass_handle),
+ text = (visit_kernel_figure, pass_handle),
+ man = (visit_kernel_figure, pass_handle), )
+
+ # render handling
+ app.add_directive('kernel-render', KernelRender)
+ app.add_node(kernel_render,
+ html = (visit_kernel_render, pass_handle),
+ latex = (visit_kernel_render, pass_handle),
+ texinfo = (visit_kernel_render, pass_handle),
+ text = (visit_kernel_render, pass_handle),
+ man = (visit_kernel_render, pass_handle), )
+
+ return dict(
+ version = __version__,
+ parallel_read_safe = True,
+ parallel_write_safe = True
+ )
+
+# integrate conversion tools
+# --------------------------
+
+RENDER_MARKUP_EXT = {
+ # The '.ext' must be handled by convert_image(..) function's *in_ext* input.
+ # <name> : <.ext>
+ 'DOT' : '.dot'
+ , 'SVG' : '.svg'
+}
+
+def convert_image(img_node, translator): # pylint: disable=R0912
+ """Convert an image node for the builder.
+
+ Different builder prefer different image formats, e.g. *latex* builder
+ prefer PDF while *html* builder prefer SVG format for images.
+
+ This function handles outputs image formats in depence of source the format
+ of the image and the translator's output format. This also means to
+ manipulate/update the *image* dictionary of the builder (``builder.images``)
+
+ """
+ fname, in_ext = path.splitext(path.basename(img_node['uri']))
+ src_fname = path.join(translator.builder.srcdir, img_node['uri'])
+ src_folder = path.dirname(img_node['uri'])
+ out_dir = translator.builder.outdir
+ dst_fname = None
+
+ # in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages
+ verbose = translator.builder.app.verbose
+ warn = translator.builder.warn
+
+ verbose('assert best format for: ' + img_node['uri'])
+
+ if in_ext == '.dot':
+
+ # ----------
+ # handle DOT
+ # ----------
+
+ dst_fname = path.join(out_dir, fname + '.pdf')
+
+ if translator.builder.format == 'html':
+ dst_fname = path.join(out_dir, src_folder, fname + '.svg')
+ else:
+ # all other builder formats will include DOT as raw
+ with open(src_fname, "r") as dot:
+ data = dot.read()
+ node = nodes.literal_block(data, data)
+ img_node.replace_self(node)
+
+
+ elif in_ext == '.svg':
+
+ # ----------
+ # handle SVG
+ # ----------
+
+ if translator.builder.format == 'latex':
+ dst_fname = path.join(out_dir, fname + '.pdf')
+
+ if dst_fname:
+ name = dst_fname[len(out_dir) + 1:]
+ # the builder needs not to copy one more time, so pop it if exists.
+ translator.builder.images.pop(img_node['uri'], None)
+ img_node['uri'] = dst_fname
+ img_node['candidates'] = {'*': dst_fname}
+
+ mkdir(path.dirname(dst_fname))
+
+ if in_ext == '.dot':
+ verbose('convert DOT to: {out}/' + name)
+ dot2format(src_fname, dst_fname)
+
+ elif in_ext == '.svg':
+ verbose('convert SVG to: {out}/' + name)
+ svg2pdf(src_fname, dst_fname)
+
+def dot2format(dot_fname, out_fname):
+ """Converts DOT file to ``out_fname`` using ``dot(1)``.
+
+ * ``dot_fname`` pathname of the input DOT file, including extension ``.dot``
+ * ``out_fname`` pathname of the output file, including format extension
+
+ The *format extension* depends on the ``dot`` command (see ``man dot``
+ option ``-Txxx``). Normally you will use one of the following extensions:
+
+ - ``.ps`` for PostScript,
+ - ``.svg`` or ``svgz`` for Structured Vector Graphics,
+ - ``.fig`` for XFIG graphics and
+ - ``.png`` or ``gif`` for common bitmap graphics.
+
+ """
+ out_format = path.splitext(out_fname)[1][1:]
+ cmd = ['dot', '-T%s' % out_format, dot_fname]
+ exit_code = 42
+ with open(out_fname, "w") as out:
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
+ sys.stderr.write(err)
+
+ exit_code = p.returncode
+ out.flush()
+ return bool(exit_code == 0)
+
+def svg2pdf(svg_fname, pdf_fname):
+ """Converts SVG to PDF with CairoSVG or ``convert(1)`` command.
+
+ Uses ``convert(1)`` from ImageMagick (https://www.imagemagick.org) for
+ conversion. Returns ``True`` on success and ``False`` if an error occurred
+ (e.g. none of the conversion tool is available).
+
+ * ``svg_fname`` pathname of the input SVG file with extension (``.svg``)
+ * ``pdf_name`` pathname of the output PDF file with extension (``.pdf``)
+
+ """
+ cmd = [convert_cmd, svg_fname, pdf_fname]
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
+ sys.stderr.write(err)
+
+ exit_code = p.returncode
+ return bool(exit_code == 0)
+
+
+# image handling
+# ---------------------
+
+def visit_kernel_image(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_image`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0]
+ convert_image(img_node, self)
+
+class kernel_image(nodes.General, nodes.Element):
+ """Node for ``kernel-image`` directive."""
+ pass
+
+class KernelImage(images.Image):
+ u"""KernelImage directive
+
+ Earns everything from ``.. image::`` directive, except *remote URI* and
+ *glob* pattern. The KernelImage wraps a image node into a
+ kernel_image node. See ``visit_kernel_image``.
+ """
+
+ def run(self):
+ env = self.state.document.settings.env
+
+ uri = self.arguments[0]
+ if uri.endswith('.*') or uri.find('://') != -1:
+ raise self.severe(
+ 'Error in "%s: %s": glob pattern and remote images are not allowed'
+ % (self.name, uri))
+
+ # Tell sphinx of the dependency
+ env.note_dependency(os.path.abspath(uri))
+
+ result = images.Image.run(self)
+ if len(result) == 2 or isinstance(result[0], nodes.system_message):
+ return result
+ (image_node,) = result
+ # wrap image node into a kernel_image node / see visitors
+ node = kernel_image('', image_node)
+ return [node]
+
+# figure handling
+# ---------------------
+
+def visit_kernel_figure(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_figure`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0][0]
+ convert_image(img_node, self)
+
+class kernel_figure(nodes.General, nodes.Element):
+ """Node for ``kernel-figure`` directive."""
+
+class KernelFigure(Figure):
+ u"""KernelImage directive
+
+ Earns everything from ``.. figure::`` directive, except *remote URI* and
+ *glob* pattern. The KernelFigure wraps a figure node into a kernel_figure
+ node. See ``visit_kernel_figure``.
+ """
+
+ def run(self):
+ env = self.state.document.settings.env
+
+ uri = self.arguments[0]
+ if uri.endswith('.*') or uri.find('://') != -1:
+ raise self.severe(
+ 'Error in "%s: %s":'
+ ' glob pattern and remote images are not allowed'
+ % (self.name, uri))
+
+ # Tell sphinx of the dependency
+ env.note_dependency(os.path.abspath(uri))
+
+ result = Figure.run(self)
+ if len(result) == 2 or isinstance(result[0], nodes.system_message):
+ return result
+ (figure_node,) = result
+ # wrap figure node into a kernel_figure node / see visitors
+ node = kernel_figure('', figure_node)
+ return [node]
+
+
+# render handling
+# ---------------------
+
+def visit_kernel_render(self, node):
+ """Visitor of the ``kernel_render`` Node.
+
+ If rendering tools available, save the markup of the ``literal_block`` child
+ node into a file and replace the ``literal_block`` node with a new created
+ ``image`` node, pointing to the saved markup file. Afterwards, handle the
+ image child-node with the ``convert_image(...)``.
+ """
+
+ verbose = self.builder.app.verbose
+ warn = self.builder.warn
+ srclang = node.get('srclang')
+
+ verbose('visit kernel-render node lang: "%s"' % (srclang))
+
+ tmp_ext = RENDER_MARKUP_EXT.get(srclang, None)
+ if tmp_ext is None:
+ warn('kernel-render: "%s" unknow / include raw.' % (srclang))
+ return
+
+ literal_block = node[0]
+ code = literal_block.astext()
+
+ if tmp_ext:
+ hashobj = code.encode('utf-8') # str(node.attributes)
+ fname = '%s-%s' % (srclang, sha1(hashobj).hexdigest())
+ tmp_fname = path.join(
+ self.builder.outdir, self.builder.imagedir, fname + tmp_ext)
+
+ if not path.isfile(tmp_fname):
+ mkdir(path.dirname(tmp_fname))
+ with open(tmp_fname, "w") as out:
+ out.write(code)
+
+ image_node = nodes.image(node.rawsource, **node.attributes)
+ image_node['uri'] = tmp_fname
+
+ literal_block.replace_self(image_node)
+ convert_image(image_node, self)
+
+
+class kernel_render(nodes.General, nodes.Inline, nodes.Element):
+ """Node for ``kernel-render`` directive."""
+ pass
+
+class KernelRender(Figure):
+ u"""KernelRender directive
+
+ Render content by external tool. Has all the options known from the
+ *figure* directive, plus option ``caption``. If ``caption`` has a
+ value, a figure node with the *caption* is inserted. If not, a image node is
+ inserted.
+
+ The KernelRender directive wraps the text of the directive into a
+ literal_block node and wraps it into a kernel_render node. See
+ ``visit_kernel_render``.
+ """
+ has_content = True
+ required_arguments = 1
+ optional_arguments = 0
+ final_argument_whitespace = False
+
+ # earn options from 'figure'
+ option_spec = Figure.option_spec.copy()
+ option_spec['caption'] = directives.unchanged
+
+ def run(self):
+ return [self.build_node()]
+
+ def build_node(self):
+
+ srclang = self.arguments[0].strip()
+ if srclang not in RENDER_MARKUP_EXT.keys():
+ return [self.state_machine.reporter.warning(
+ 'Unknow source language "%s", use one of: %s.' % (
+ srclang, ",".join(RENDER_MARKUP_EXT.keys())),
+ line=self.lineno)]
+
+ code = '\n'.join(self.content)
+ if not code.strip():
+ return [self.state_machine.reporter.warning(
+ 'Ignoring "%s" directive without content.' % (
+ self.name),
+ line=self.lineno)]
+
+ node = kernel_render()
+ node['alt'] = self.options.get('alt','')
+ node['srclang'] = srclang
+ literal_node = nodes.literal_block(code, code)
+ node += literal_node
+
+ caption = self.options.get('caption')
+ if caption:
+ # parse cation's content
+ parsed = nodes.Element()
+ self.state.nested_parse(
+ ViewList([caption], source=''), self.content_offset, parsed)
+ caption_node = nodes.caption(
+ parsed[0].rawsource, '', *parsed[0].children)
+ caption_node.source = parsed[0].source
+ caption_node.line = parsed[0].line
+
+ figure_node = nodes.figure('', node)
+ for k,v in self.options.items():
+ figure_node[k] = v
+ figure_node += caption_node
+
+ node = figure_node
+
+ return node
+
--
2.11.0
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PATCH 3/6] drm/doc: Add KMS overview graphs
2017-03-02 15:16 [PATCH 1/6] doc: Explain light-handed markup preference a bit better Daniel Vetter
2017-03-02 15:16 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
@ 2017-03-02 15:16 ` Daniel Vetter
2017-03-02 15:16 ` [PATCH 4/6] drm/doc: Consistent kerneldoc include order Daniel Vetter
` (3 subsequent siblings)
5 siblings, 0 replies; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 15:16 UTC (permalink / raw)
To: linux-doc; +Cc: Daniel Vetter, Laurent Pinchart, DRI Development, Daniel Vetter
Oh, the shiny and pretties!
v2: Review from Laurent.
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/gpu/drm-kms-helpers.rst | 4 +
Documentation/gpu/drm-kms.rst | 134 +++++++++++++++++++++++++++++++++-
2 files changed, 137 insertions(+), 1 deletion(-)
diff --git a/Documentation/gpu/drm-kms-helpers.rst b/Documentation/gpu/drm-kms-helpers.rst
index 03040aa14fe8..012b6ff3ec3f 100644
--- a/Documentation/gpu/drm-kms-helpers.rst
+++ b/Documentation/gpu/drm-kms-helpers.rst
@@ -114,6 +114,8 @@ Framebuffer CMA Helper Functions Reference
.. kernel-doc:: drivers/gpu/drm/drm_fb_cma_helper.c
:export:
+.. _drm_bridges:
+
Bridges
=======
@@ -139,6 +141,8 @@ Bridge Helper Reference
.. kernel-doc:: drivers/gpu/drm/drm_bridge.c
:export:
+.. _drm_panel_helper:
+
Panel Helper Reference
======================
diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index 4d4068855ec4..bcf475f86e83 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -15,7 +15,139 @@ be setup by initializing the following fields.
- struct drm_mode_config_funcs \*funcs;
Mode setting functions.
-Mode Configuration
+Overview
+========
+
+.. kernel-render:: DOT
+ :alt: KMS Display Pipeline
+ :caption: KMS Display Pipeline Overview
+
+ digraph "KMS" {
+ node [shape=box]
+
+ subgraph cluster_static {
+ style=dashed
+ label="Static Objects"
+
+ node [bgcolor=grey style=filled]
+ "drm_plane A" -> "drm_crtc"
+ "drm_plane B" -> "drm_crtc"
+ "drm_crtc" -> "drm_encoder A"
+ "drm_crtc" -> "drm_encoder B"
+ }
+
+ subgraph cluster_user_created {
+ style=dashed
+ label="Userspace-Created"
+
+ node [shape=oval]
+ "drm_framebuffer 1" -> "drm_plane A"
+ "drm_framebuffer 2" -> "drm_plane B"
+ }
+
+ subgraph cluster_connector {
+ style=dashed
+ label="Hotpluggable"
+
+ "drm_encoder A" -> "drm_connector A"
+ "drm_encoder B" -> "drm_connector B"
+ }
+ }
+
+The basic object structure KMS presents to userspace is fairly simple.
+Framebuffers (represented by :c:type:`struct drm_framebuffer <drm_framebuffer>`,
+see `Frame Buffer Abstraction`_) feed into planes. One or more (or even no)
+planes feed their pixel data into a CRTC (represented by :c:type:`struct
+drm_crtc <drm_crtc>`, see `CRTC Abstraction`_) for blending. The precise
+blending step is explained in more detail in `Plane Composition Properties`_ and
+related chapters.
+
+For the output routing the first step is encoders (represented by
+:c:type:`struct drm_encoder <drm_encoder>`, see `Encoder Abstraction`_). Those
+are really just internal artifacts of the helper libraries used to implement KMS
+drivers. Besides that they make it unecessarily more complicated for userspace
+to figure out which connections between a CRTC and a connector are possible, and
+what kind of cloning is supported, they serve no purpose in the userspace API.
+Unfortunately encoders have been exposed to userspace, hence can't remove them
+at this point. Futhermore the exposed restrictions are often wrongly set by
+drivers, and in many cases not powerful enough to express the real restrictions.
+A CRTC can be connected to multiple encoders, and for an active CRTC there must
+be at least one encoder.
+
+The final, and real, endpoint in the display chain is the connector (represented
+by :c:type:`struct drm_connector <drm_connector>`, see `Connector
+Abstraction`_). Connectors can have different possible encoders, but the kernel
+driver selects which encoder to use for each connector. The use case is DVI,
+which could switch between an analog and a digital encoder. Encoders can also
+drive multiple different connectors. There is exactly one active connector for
+every active encoder.
+
+Internally the output pipeline is a bit more complex and matches today's
+hardware more closely:
+
+.. kernel-render:: DOT
+ :alt: KMS Output Pipeline
+ :caption: KMS Output Pipeline
+
+ digraph "Output Pipeline" {
+ node [shape=box]
+
+ subgraph {
+ "drm_crtc" [bgcolor=grey style=filled]
+ }
+
+ subgraph cluster_internal {
+ style=dashed
+ label="Internal Pipeline"
+ {
+ node [bgcolor=grey style=filled]
+ "drm_encoder A";
+ "drm_encoder B";
+ "drm_encoder C";
+ }
+
+ {
+ node [bgcolor=grey style=filled]
+ "drm_encoder B" -> "drm_bridge B"
+ "drm_encoder C" -> "drm_bridge C1"
+ "drm_bridge C1" -> "drm_bridge C2";
+ }
+ }
+
+ "drm_crtc" -> "drm_encoder A"
+ "drm_crtc" -> "drm_encoder B"
+ "drm_crtc" -> "drm_encoder C"
+
+
+ subgraph cluster_output {
+ style=dashed
+ label="Outputs"
+
+ "drm_encoder A" -> "drm_connector A";
+ "drm_bridge B" -> "drm_connector B";
+ "drm_bridge C2" -> "drm_connector C";
+
+ "drm_panel"
+ }
+ }
+
+Internally two additional helper objects come into play. First, to be able to
+share code for encoders (sometimes on the same SoC, sometimes off-chip) one or
+more :ref:`drm_bridges` (represented by :c:type:`struct drm_bridge
+<drm_bridge>`) can be linked to an encoder. This link is static and cannot be
+changed, which means the cross-bar (if there is any) needs to be mapped between
+the CRTC and any encoders. Often for drivers with bridges there's no code left
+at the encoder level. Atomic drivers can leave out all the encoder callbacks to
+essentially only leave a dummy routing object behind, which is needed for
+backwards compatibility since encoders are exposed to userspace.
+
+The second object is for panels, represented by :c:type:`struct drm_panel
+<drm_panel>`, see :ref:`drm_panel_helper`. Panels do not have a fixed binding
+point, but are generally linked to the driver private structure that embeds
+:c:type:`struct drm_connector <drm_connector>`.
+
+Note that currently the bridge chaining and interactions with connectors and
+panels are still in-flux and not really fully sorted out yet.
KMS Core Structures and Functions
=================================
--
2.11.0
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PATCH 4/6] drm/doc: Consistent kerneldoc include order
2017-03-02 15:16 [PATCH 1/6] doc: Explain light-handed markup preference a bit better Daniel Vetter
2017-03-02 15:16 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
2017-03-02 15:16 ` [PATCH 3/6] drm/doc: Add KMS overview graphs Daniel Vetter
@ 2017-03-02 15:16 ` Daniel Vetter
2017-03-02 15:16 ` [PATCH 5/6] drm/doc: diagram for mode objects and properties Daniel Vetter
` (2 subsequent siblings)
5 siblings, 0 replies; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 15:16 UTC (permalink / raw)
To: linux-doc; +Cc: Daniel Vetter, DRI Development, Daniel Vetter
First overview text (if there is any), then headers (since generally
you want to start out with the data structures), then all the other
stuff with functions.
Most of this is pre-shpinx, since with the old docbook only the
overview stuff was pulled in directly. Everything else was put in a
per-section index, so include order didn't really matter.
Acked-by: Eric Anholt <eric@anholt.net>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/gpu/drm-internals.rst | 6 +++---
Documentation/gpu/drm-kms-helpers.rst | 22 +++++++++++-----------
Documentation/gpu/drm-kms.rst | 24 ++++++++++++------------
Documentation/gpu/drm-mm.rst | 24 ++++++++++++------------
4 files changed, 38 insertions(+), 38 deletions(-)
diff --git a/Documentation/gpu/drm-internals.rst b/Documentation/gpu/drm-internals.rst
index e35920db1f4c..29d6bf7bb1ac 100644
--- a/Documentation/gpu/drm-internals.rst
+++ b/Documentation/gpu/drm-internals.rst
@@ -140,12 +140,12 @@ Device Instance and Driver Handling
.. kernel-doc:: drivers/gpu/drm/drm_drv.c
:doc: driver instance overview
-.. kernel-doc:: drivers/gpu/drm/drm_drv.c
- :export:
-
.. kernel-doc:: include/drm/drm_drv.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_drv.c
+ :export:
+
Driver Load
-----------
diff --git a/Documentation/gpu/drm-kms-helpers.rst b/Documentation/gpu/drm-kms-helpers.rst
index 012b6ff3ec3f..050ebe81d256 100644
--- a/Documentation/gpu/drm-kms-helpers.rst
+++ b/Documentation/gpu/drm-kms-helpers.rst
@@ -37,10 +37,10 @@ Modeset Helper Reference for Common Vtables
===========================================
.. kernel-doc:: include/drm/drm_modeset_helper_vtables.h
- :internal:
+ :doc: overview
.. kernel-doc:: include/drm/drm_modeset_helper_vtables.h
- :doc: overview
+ :internal:
Atomic Modeset Helper Functions Reference
=========================================
@@ -84,27 +84,27 @@ Legacy CRTC/Modeset Helper Functions Reference
Simple KMS Helper Reference
===========================
+.. kernel-doc:: drivers/gpu/drm/drm_simple_kms_helper.c
+ :doc: overview
+
.. kernel-doc:: include/drm/drm_simple_kms_helper.h
:internal:
.. kernel-doc:: drivers/gpu/drm/drm_simple_kms_helper.c
:export:
-.. kernel-doc:: drivers/gpu/drm/drm_simple_kms_helper.c
- :doc: overview
-
fbdev Helper Functions Reference
================================
.. kernel-doc:: drivers/gpu/drm/drm_fb_helper.c
:doc: fbdev helpers
-.. kernel-doc:: drivers/gpu/drm/drm_fb_helper.c
- :export:
-
.. kernel-doc:: include/drm/drm_fb_helper.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_fb_helper.c
+ :export:
+
Framebuffer CMA Helper Functions Reference
==========================================
@@ -146,15 +146,15 @@ Bridge Helper Reference
Panel Helper Reference
======================
+.. kernel-doc:: drivers/gpu/drm/drm_panel.c
+ :doc: drm panel
+
.. kernel-doc:: include/drm/drm_panel.h
:internal:
.. kernel-doc:: drivers/gpu/drm/drm_panel.c
:export:
-.. kernel-doc:: drivers/gpu/drm/drm_panel.c
- :doc: drm panel
-
Display Port Helper Functions Reference
=======================================
diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index bcf475f86e83..b11c537c73f4 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -152,12 +152,12 @@ panels are still in-flux and not really fully sorted out yet.
KMS Core Structures and Functions
=================================
-.. kernel-doc:: drivers/gpu/drm/drm_mode_config.c
- :export:
-
.. kernel-doc:: include/drm/drm_mode_config.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_mode_config.c
+ :export:
+
Modeset Base Object Abstraction
===============================
@@ -170,12 +170,12 @@ Modeset Base Object Abstraction
Atomic Mode Setting Function Reference
======================================
-.. kernel-doc:: drivers/gpu/drm/drm_atomic.c
- :export:
-
.. kernel-doc:: include/drm/drm_atomic.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_atomic.c
+ :export:
+
CRTC Abstraction
================
@@ -200,12 +200,12 @@ Frame Buffer Abstraction
Frame Buffer Functions Reference
--------------------------------
-.. kernel-doc:: drivers/gpu/drm/drm_framebuffer.c
- :export:
-
.. kernel-doc:: include/drm/drm_framebuffer.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_framebuffer.c
+ :export:
+
DRM Format Handling
===================
@@ -508,8 +508,8 @@ operation handler.
Vertical Blanking and Interrupt Handling Functions Reference
------------------------------------------------------------
-.. kernel-doc:: drivers/gpu/drm/drm_irq.c
- :export:
-
.. kernel-doc:: include/drm/drm_irq.h
:internal:
+
+.. kernel-doc:: drivers/gpu/drm/drm_irq.c
+ :export:
diff --git a/Documentation/gpu/drm-mm.rst b/Documentation/gpu/drm-mm.rst
index fd35998acefc..d47cbe3a9ef3 100644
--- a/Documentation/gpu/drm-mm.rst
+++ b/Documentation/gpu/drm-mm.rst
@@ -365,36 +365,36 @@ from the client in libdrm.
GEM Function Reference
----------------------
-.. kernel-doc:: drivers/gpu/drm/drm_gem.c
- :export:
-
.. kernel-doc:: include/drm/drm_gem.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_gem.c
+ :export:
+
GEM CMA Helper Functions Reference
----------------------------------
.. kernel-doc:: drivers/gpu/drm/drm_gem_cma_helper.c
:doc: cma helpers
-.. kernel-doc:: drivers/gpu/drm/drm_gem_cma_helper.c
- :export:
-
.. kernel-doc:: include/drm/drm_gem_cma_helper.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_gem_cma_helper.c
+ :export:
+
VMA Offset Manager
==================
.. kernel-doc:: drivers/gpu/drm/drm_vma_manager.c
:doc: vma offset manager
-.. kernel-doc:: drivers/gpu/drm/drm_vma_manager.c
- :export:
-
.. kernel-doc:: include/drm/drm_vma_manager.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_vma_manager.c
+ :export:
+
PRIME Buffer Sharing
====================
@@ -470,12 +470,12 @@ LRU Scan/Eviction Support
DRM MM Range Allocator Function References
------------------------------------------
-.. kernel-doc:: drivers/gpu/drm/drm_mm.c
- :export:
-
.. kernel-doc:: include/drm/drm_mm.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_mm.c
+ :export:
+
DRM Cache Handling
==================
--
2.11.0
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PATCH 5/6] drm/doc: diagram for mode objects and properties
2017-03-02 15:16 [PATCH 1/6] doc: Explain light-handed markup preference a bit better Daniel Vetter
` (2 preceding siblings ...)
2017-03-02 15:16 ` [PATCH 4/6] drm/doc: Consistent kerneldoc include order Daniel Vetter
@ 2017-03-02 15:16 ` Daniel Vetter
2017-03-02 15:16 ` [PATCH 6/6] drm/doc: atomic overview, with graph Daniel Vetter
2017-03-07 16:40 ` [PATCH 1/6] doc: Explain light-handed markup preference a bit better Daniel Vetter
5 siblings, 0 replies; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 15:16 UTC (permalink / raw)
To: linux-doc; +Cc: Manasi Navare, Daniel Vetter, Laurent Pinchart, DRI Development
Resulted in confusion a few times in the past.
v2: Spelling fix (Eric).
Cc: Eric Anholt <eric@anholt.net>
Acked-by: Eric Anholt <eric@anholt.net>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Manasi Navare <manasi.d.navare@intel.com>
Reviewed-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
---
Documentation/gpu/drm-kms.rst | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index b11c537c73f4..1c42448c7aae 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -161,6 +161,28 @@ KMS Core Structures and Functions
Modeset Base Object Abstraction
===============================
+.. kernel-render:: DOT
+ :alt: Mode Objects and Properties
+ :caption: Mode Objects and Properties
+
+ digraph {
+ node [shape=box]
+
+ "drm_property A" -> "drm_mode_object A"
+ "drm_property A" -> "drm_mode_object B"
+ "drm_property B" -> "drm_mode_object A"
+ }
+
+The base structure for all KMS objects is :c:type:`struct drm_mode_object
+<drm_mode_object>`. One of the base services it provides is tracking properties,
+which are especially important for the atomic IOCTL (see `Atomic Mode
+Setting`_). The somewhat surprising part here is that properties are not
+directly instantiated on each object, but free-standing mode objects themselves,
+represented by :c:type:`struct drm_property <drm_property>`, which only specify
+the type and value range of a property. Any given property can be attached
+multiple times to different objects using :c:func:`drm_object_attach_property()
+<drm_object_attach_property>`.
+
.. kernel-doc:: include/drm/drm_mode_object.h
:internal:
--
2.11.0
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PATCH 6/6] drm/doc: atomic overview, with graph
2017-03-02 15:16 [PATCH 1/6] doc: Explain light-handed markup preference a bit better Daniel Vetter
` (3 preceding siblings ...)
2017-03-02 15:16 ` [PATCH 5/6] drm/doc: diagram for mode objects and properties Daniel Vetter
@ 2017-03-02 15:16 ` Daniel Vetter
2017-03-02 15:24 ` Gabriel Krisman Bertazi
2017-03-07 16:40 ` [PATCH 1/6] doc: Explain light-handed markup preference a bit better Daniel Vetter
5 siblings, 1 reply; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 15:16 UTC (permalink / raw)
To: linux-doc
Cc: Daniel Vetter, DRI Development, Laurent Pinchart, Daniel Vetter,
Gabriel Krisman Bertazi
I want to split up a few more things and document some details better
(like how exactly to subclass drm_atomic_state). And maybe also split
up the helpers a bit per-topic, but this should be a ok-ish start for
better atomic overview.
v2: Spelling and clarifications (Eric).
v3: Implement suggestion from Gabriel to fix the graph.
v4: Review from Laurent.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Acked-by: Eric Anholt <eric@anholt.net>
Cc: Eric Anholt <eric@anholt.net>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/gpu/drm-kms-helpers.rst | 2 +
Documentation/gpu/drm-kms.rst | 84 ++++++++++++++++++++++++++++++++++-
2 files changed, 85 insertions(+), 1 deletion(-)
diff --git a/Documentation/gpu/drm-kms-helpers.rst b/Documentation/gpu/drm-kms-helpers.rst
index 050ebe81d256..ac53c0b893f6 100644
--- a/Documentation/gpu/drm-kms-helpers.rst
+++ b/Documentation/gpu/drm-kms-helpers.rst
@@ -42,6 +42,8 @@ Modeset Helper Reference for Common Vtables
.. kernel-doc:: include/drm/drm_modeset_helper_vtables.h
:internal:
+.. _drm_atomic_helper:
+
Atomic Modeset Helper Functions Reference
=========================================
diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index 1c42448c7aae..bfecd21a8cdf 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -189,8 +189,90 @@ multiple times to different objects using :c:func:`drm_object_attach_property()
.. kernel-doc:: drivers/gpu/drm/drm_mode_object.c
:export:
+Atomic Mode Setting
+===================
+
+
+.. kernel-render:: DOT
+ :alt: Mode Objects and Properties
+ :caption: Mode Objects and Properties
+
+ digraph {
+ node [shape=box]
+
+ subgraph cluster_state {
+ style=dashed
+ label="Free-standing state"
+
+ "drm_atomic_state" -> "duplicated drm_plane_state A"
+ "drm_atomic_state" -> "duplicated drm_plane_state B"
+ "drm_atomic_state" -> "duplicated drm_crtc_state"
+ "drm_atomic_state" -> "duplicated drm_connector_state"
+ "drm_atomic_state" -> "duplicated driver private state"
+ }
+
+ subgraph cluster_current {
+ style=dashed
+ label="Current state"
+
+ "drm_device" -> "drm_plane A"
+ "drm_device" -> "drm_plane B"
+ "drm_device" -> "drm_crtc"
+ "drm_device" -> "drm_connector"
+ "drm_device" -> "driver private object"
+
+ "drm_plane A" -> "drm_plane_state A"
+ "drm_plane B" -> "drm_plane_state B"
+ "drm_crtc" -> "drm_crtc_state"
+ "drm_connector" -> "drm_connector_state"
+ "driver private object" -> "driver private state"
+ }
+
+ "drm_atomic_state" -> "drm_device" [label="atomic_commit"]
+ "duplicated drm_plane_state A" -> "drm_device"[style=invis]
+ }
+
+Atomic provides transactional modeset (including planes) updates, but a
+bit differently from the usual transactional approach of try-commit and
+rollback:
+
+- Firstly, no hardware changes are allowed when the commit would fail. This
+ allows us to implement the DRM_MODE_ATOMIC_TEST_ONLY mode, which allows
+ userspace to explore whether certain configurations would work or not.
+
+- This would still allow setting and rollback of just the software state,
+ simplifying conversion of existing drivers. But auditing drivers for
+ correctness of the atomic_check code becomes really hard with that: Rolling
+ back changes in data structures all over the place is hard to get right.
+
+- Lastly, for backwards compatibility and to support all use-cases, atomic
+ updates need to be incremental and be able to execute in parallel. Hardware
+ doesn't always allow it, but where possible plane updates on different CRTCs
+ should not interfere, and not get stalled due to output routing changing on
+ different CRTCs.
+
+Taken all together there's two consequences for the atomic design:
+
+- The overall state is split up into per-object state structures:
+ :c:type:`struct drm_plane_state <drm_plane_state>` for planes, :c:type:`struct
+ drm_crtc_state <drm_crtc_state>` for CRTCs and :c:type:`struct
+ drm_connector_state <drm_connector_state>` for connectors. These are the only
+ objects with userspace-visible and settable state. For internal state drivers
+ can subclass these structures through embeddeding, or add entirely new state
+ structures for their globally shared hardware functions.
+
+- An atomic update is assembled and validated as an entirely free-standing pile
+ of structures within the :c:type:`drm_atomic_state <drm_atomic_state>`
+ container. Again drivers can subclass that container for their own state
+ structure tracking needs. Only when a state is committed is it applied to the
+ driver and modeset objects. This way rolling back an update boils down to
+ releasing memory and unreferencing objects like framebuffers.
+
+Read on in this chapter, and also in :ref:`drm_atomic_helper` for more detailed
+coverage of specific topics.
+
Atomic Mode Setting Function Reference
-======================================
+--------------------------------------
.. kernel-doc:: include/drm/drm_atomic.h
:internal:
--
2.11.0
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:11 ` Daniel Vetter
@ 2017-03-02 15:17 ` Laurent Pinchart
2017-03-02 15:21 ` Markus Heiser
2017-03-02 15:22 ` Jonathan Corbet
2 siblings, 0 replies; 56+ messages in thread
From: Laurent Pinchart @ 2017-03-02 15:17 UTC (permalink / raw)
To: Daniel Vetter
Cc: Markus Heiser, Jonathan Corbet, Daniel Vetter,
Linux Doc Mailing List, dri-devel, Mauro Carvalho Chehab,
Daniel Vetter
Hi Daniel and Markus,
On Thursday 02 Mar 2017 16:11:08 Daniel Vetter wrote:
> On Thu, Mar 02, 2017 at 03:58:36PM +0100, Markus Heiser wrote:
> > Am 02.03.2017 um 15:14 schrieb Laurent Pinchart:
> > > On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:
> > >> On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:
> > >>> Hi Daniel,
> > >>>
> > >>> Thank you for the patch.
> > >>>
> > >>> With this applied, I get
> > >>>
> > >>> make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
> > >>>
> > >>> SPHINX htmldocs -->
> > >>> file:///home/laurent/src/iob/renesas/linux64/Documentation/output
> > >>> PARSE
> > >>>
> > >>> include/uapi/linux/videodev2.h
> > >>>
> > >>> Running Sphinx v1.3.1
> > >>>
> > >>> Extension error:
> > >>> Could not import extension kfigure (exception: cannot import name
> > >>> patches)
> > >>> make[2]: ***
> > >>> [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
> > >>> htmldocs] Error 1 make[1]: ***
> > >>> [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
> > >>> make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64'
> > >>> make:
> > >>> *** [Makefile:152: sub-make] Error 2
> > >>>
> > >>> sphinx.directive.patches got introduced in Sphinx 1.4. If you want to
> > >>> bump
> > >>> the minimum required version I think a notice is needed.
> > >>
> > >> Ugh. But this also goes completely over my head, no idea whether we
> > >> must require sphinx 1.4 (it was released Mar 28, 2016), or whether
> > >> there's some way to work around this ... Halp?
> > >
> > > I'm not a Sphinx expert so I don't know, but what I can tell is that
> > > copying the patches.py from Sphinx 1.4 to Documentation/sphinx/ and
> > > modifying kfigure.py to import it from there fixes the build. There's
> > > thus no extra depencency on Sphinx 1.4 (or newer).
> > >
> > > I'm not sure we want to set a precedent by copying part of the Sphinx
> > > source code to the kernel tree (or inlining the single small function
> > > that the module provides), and I'll let someone more knowledgeable than
> > > me decide how to proceed.
> >
> > Aargh ... we need virtualenv! For interim something like the following
> > might help. In file Documentation/sphinx/kfigure.py edit the imports
> >
> > ...
> > from docutils.parsers.rst.directives import images
> >
> > try:
> > from sphinx.directives.patches import Figure
> >
> > except ImportError:
> > Figure = images.Figure
> >
> > ...
> >
> > And fix the class definition, so it use 'Figure' and no
> > longer 'patch.Figure'::
> >
> > ...
> > -class KernelFigure(patches.Figure):
> > +class KernelFigure(Figure):
> > ...
> >
> > Sorry that I have not yet the time to send you a decent and tested
> > patch. Do you like to test my suggestion? / thanks!
>
> I'll give it a shot at implementing it, but I can't (easily at least) test
> on sphinx 1.3.
I can, and it works.
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Thank you for the quick fix Markus.
--
Regards,
Laurent Pinchart
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:16 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
@ 2017-03-02 15:20 ` Laurent Pinchart
2017-03-02 15:40 ` [PATCH] " Daniel Vetter
2017-03-02 22:15 ` [PATCH 2/6] " Markus Heiser
2 siblings, 0 replies; 56+ messages in thread
From: Laurent Pinchart @ 2017-03-02 15:20 UTC (permalink / raw)
To: Daniel Vetter
Cc: Markus Heiser, Jonathan Corbet, linux-doc, Mauro Carvalho Chehab,
DRI Development, Daniel Vetter
Hi Daniel,
Thank you for the patch.
On Thursday 02 Mar 2017 16:16:34 Daniel Vetter wrote:
> From: Markus Heiser <markus.heiser@darmarit.de>
>
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
>
> * DOT (http://www.graphviz.org)
> * SVG
>
> For image handling use the 'image' replacement::
>
> .. kernel-image:: svg_image.svg
>
> :alt: simple SVG image
>
> For figure handling use the 'figure' replacement::
>
> .. kernel-figure:: svg_image.svg
>
> :alt: simple SVG image
>
> SVG image example
>
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
>
> .. kernel-render:: DOT
>
> :alt: foobar digraph
> :caption: Embedded **DOT** (Graphviz) code.
>
> digraph foo {
> "bar" -> "baz";
> }
>
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
>
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
>
> v2: s/DOC/DOT/ in a few places (by Daniel).
>
> v3: Simplify stuff a bit (by Daniel):
>
> - Remove path detection and setup/check code for that. In
> Documentation/media/Makefile we already simply use these tools,
> better to have one consolidated check if we want/need one. Also
> remove the convertsvg support, we require ImageMagick's convert
> already in the doc build, no need for a 2nd fallback.
>
> - Use sphinx for depency tracking, remove hand-rolled version.
>
> - Forward stderr from dot and convert, otherwise debugging issues with
> the diagrams is impossible.
>
> v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
However, you might want to explicitly check the Sphinx version, as done in
Documentation/conf.py:
import sphinx
# Get Sphinx version
major, minor, patch = map(int, sphinx.__version__.split("."))
if major == 1 and minor > 3:
...
This will make it clearer what version we depend on, and easier to remove dead
code when we'll bump the minimum version requirements.
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: linux-doc@vger.kernel.org
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> Cc: Markus Heiser <markus.heiser@darmarit.de>
> Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> (v1)
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> ---
> Documentation/conf.py | 2 +-
> Documentation/doc-guide/hello.dot | 3 +
> Documentation/doc-guide/sphinx.rst | 90 ++++++-
> Documentation/doc-guide/svg_image.svg | 10 +
> Documentation/process/changes.rst | 7 +-
> Documentation/sphinx/kfigure.py | 444 ++++++++++++++++++++++++++++++
> 6 files changed, 550 insertions(+), 6 deletions(-)
> create mode 100644 Documentation/doc-guide/hello.dot
> create mode 100644 Documentation/doc-guide/svg_image.svg
> create mode 100644 Documentation/sphinx/kfigure.py
--
Regards,
Laurent Pinchart
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:11 ` Daniel Vetter
2017-03-02 15:17 ` Laurent Pinchart
@ 2017-03-02 15:21 ` Markus Heiser
2017-03-02 15:45 ` Mauro Carvalho Chehab
2017-03-02 15:22 ` Jonathan Corbet
2 siblings, 1 reply; 56+ messages in thread
From: Markus Heiser @ 2017-03-02 15:21 UTC (permalink / raw)
To: Daniel Vetter
Cc: Jonathan Corbet, Daniel Vetter, Linux Doc Mailing List, dri-devel,
Mauro Carvalho Chehab, Laurent Pinchart, Daniel Vetter
Am 02.03.2017 um 16:11 schrieb Daniel Vetter <daniel@ffwll.ch>:
> On Thu, Mar 02, 2017 at 03:58:36PM +0100, Markus Heiser wrote:
>> Hi Daniel, Laurent
>>
>> Am 02.03.2017 um 15:14 schrieb Laurent Pinchart <laurent.pinchart@ideasonboard.com>:
>>
>>> Hi Daniel,
>>>
>>> On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:
>>>> On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:
>>>>> Hi Daniel,
>>>>>
>>>>> Thank you for the patch.
>>>>>
>>>>> With this applied, I get
>>>>>
>>>>> make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
>>>>>
>>>>> SPHINX htmldocs -->
>>>>> file:///home/laurent/src/iob/renesas/linux64/Documentation/output PARSE
>>>>> include/uapi/linux/videodev2.h
>>>>>
>>>>> Running Sphinx v1.3.1
>>>>>
>>>>> Extension error:
>>>>> Could not import extension kfigure (exception: cannot import name patches)
>>>>> make[2]: ***
>>>>> [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
>>>>> htmldocs] Error 1 make[1]: ***
>>>>> [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
>>>>> make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64' make:
>>>>> *** [Makefile:152: sub-make] Error 2
>>>>>
>>>>> sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump
>>>>> the minimum required version I think a notice is needed.
>>>>
>>>> Ugh. But this also goes completely over my head, no idea whether we
>>>> must require sphinx 1.4 (it was released Mar 28, 2016), or whether
>>>> there's some way to work around this ... Halp?
>>>
>>> I'm not a Sphinx expert so I don't know, but what I can tell is that copying
>>> the patches.py from Sphinx 1.4 to Documentation/sphinx/ and modifying
>>> kfigure.py to import it from there fixes the build. There's thus no extra
>>> depencency on Sphinx 1.4 (or newer).
>>>
>>> I'm not sure we want to set a precedent by copying part of the Sphinx source
>>> code to the kernel tree (or inlining the single small function that the module
>>> provides), and I'll let someone more knowledgeable than me decide how to
>>> proceed.
>>
>>
>> Aargh ... we need virtualenv! For interim something like the following
>> might help. In file Documentation/sphinx/kfigure.py edit the imports
>>
>> ...
>> from docutils.parsers.rst.directives import images
>> try:
>> from sphinx.directives.patches import Figure
>> except ImportError:
>> Figure = images.Figure
>> ...
>>
>> And fix the class definition, so it use 'Figure' and no
>> longer 'patch.Figure'::
>>
>> ...
>> -class KernelFigure(patches.Figure):
>> +class KernelFigure(Figure):
>> ...
>>
>> Sorry that I have not yet the time to send you a decent and tested
>> patch. Do you like to test my suggestion? / thanks!
>
> I'll give it a shot at implementing it, but I can't (easily at least) test
> on sphinx 1.3.
You can test it with virtualenv https://virtualenv.pypa.io/en/stable/userguide/
In short:
$ cd kernel-src
$ virtualenv myenv
$ source myenv/bin/activate
$ pip install 'Sphinx==1.3.1'
$ make ....
-- Markus --
> -Daniel
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:11 ` Daniel Vetter
2017-03-02 15:17 ` Laurent Pinchart
2017-03-02 15:21 ` Markus Heiser
@ 2017-03-02 15:22 ` Jonathan Corbet
2017-03-02 15:53 ` Daniel Vetter
2 siblings, 1 reply; 56+ messages in thread
From: Jonathan Corbet @ 2017-03-02 15:22 UTC (permalink / raw)
To: Daniel Vetter
Cc: Markus Heiser, Linux Doc Mailing List, Daniel Vetter, dri-devel,
Mauro Carvalho Chehab, Laurent Pinchart, Daniel Vetter
On Thu, 2 Mar 2017 16:11:08 +0100
Daniel Vetter <daniel@ffwll.ch> wrote:
> I'll give it a shot at implementing it, but I can't (easily at least) test
> on sphinx 1.3.
Virtualenv makes that kind of thing pretty easy to do. Maybe we should
add a script to create and populate a suitable venv for this kind of
thing.
Meanwhile, I've been seeing only parts 1 and 2 of what's clearly a bigger
series. Do you want me to carry just these two?
jon
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 6/6] drm/doc: atomic overview, with graph
2017-03-02 15:16 ` [PATCH 6/6] drm/doc: atomic overview, with graph Daniel Vetter
@ 2017-03-02 15:24 ` Gabriel Krisman Bertazi
2017-03-14 14:14 ` Daniel Vetter
0 siblings, 1 reply; 56+ messages in thread
From: Gabriel Krisman Bertazi @ 2017-03-02 15:24 UTC (permalink / raw)
To: Daniel Vetter; +Cc: linux-doc, DRI Development, Laurent Pinchart, Daniel Vetter
Daniel Vetter <daniel.vetter@ffwll.ch> writes:
> I want to split up a few more things and document some details better
> (like how exactly to subclass drm_atomic_state). And maybe also split
> up the helpers a bit per-topic, but this should be a ok-ish start for
> better atomic overview.
Reviewed-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
--
Gabriel Krisman Bertazi
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:16 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
2017-03-02 15:20 ` Laurent Pinchart
@ 2017-03-02 15:40 ` Daniel Vetter
2017-03-02 18:06 ` Mauro Carvalho Chehab
2017-03-02 21:29 ` Mauro Carvalho Chehab
2017-03-02 22:15 ` [PATCH 2/6] " Markus Heiser
2 siblings, 2 replies; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 15:40 UTC (permalink / raw)
To: linux-doc
Cc: Markus Heiser, Jonathan Corbet, DRI Development,
Mauro Carvalho Chehab, Laurent Pinchart, Daniel Vetter
From: Markus Heiser <markus.heiser@darmarit.de>
This patch brings scalable figure, image handling and a concept to
embed *render* markups:
* DOT (http://www.graphviz.org)
* SVG
For image handling use the 'image' replacement::
.. kernel-image:: svg_image.svg
:alt: simple SVG image
For figure handling use the 'figure' replacement::
.. kernel-figure:: svg_image.svg
:alt: simple SVG image
SVG image example
Embed *render* markups (or languages) like Graphviz's **DOT** is
provided by the *render* directive.::
.. kernel-render:: DOT
:alt: foobar digraph
:caption: Embedded **DOT** (Graphviz) code.
digraph foo {
"bar" -> "baz";
}
The *render* directive is a concept to integrate *render* markups and
languages, yet supported markups:
* DOT: render embedded Graphviz's **DOT**
* SVG: render embedded Scalable Vector Graphics (**SVG**)
v2: s/DOC/DOT/ in a few places (by Daniel).
v3: Simplify stuff a bit (by Daniel):
- Remove path detection and setup/check code for that. In
Documentation/media/Makefile we already simply use these tools,
better to have one consolidated check if we want/need one. Also
remove the convertsvg support, we require ImageMagick's convert
already in the doc build, no need for a 2nd fallback.
- Use sphinx for depency tracking, remove hand-rolled version.
- Forward stderr from dot and convert, otherwise debugging issues with
the diagrams is impossible.
v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
Implement Markus suggestion for backwards compatability with earlier
releases. Laurent reported this, running sphinx 1.3. Solution entirely
untested.
v5: Use an explicit version check (suggested by Laurent).
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Cc: Markus Heiser <markus.heiser@darmarit.de>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Markus Heiser <markus.heiser@darmarit.de>
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> (v1)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/conf.py | 2 +-
Documentation/doc-guide/hello.dot | 3 +
Documentation/doc-guide/sphinx.rst | 90 ++++++-
Documentation/doc-guide/svg_image.svg | 10 +
Documentation/process/changes.rst | 7 +-
Documentation/sphinx/kfigure.py | 450 ++++++++++++++++++++++++++++++++++
6 files changed, 556 insertions(+), 6 deletions(-)
create mode 100644 Documentation/doc-guide/hello.dot
create mode 100644 Documentation/doc-guide/svg_image.svg
create mode 100644 Documentation/sphinx/kfigure.py
diff --git a/Documentation/conf.py b/Documentation/conf.py
index f6823cf01275..e3f537ce2935 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -34,7 +34,7 @@ from load_config import loadConfig
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain']
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure']
# The name of the math extension changed on Sphinx 1.4
if major == 1 and minor > 3:
diff --git a/Documentation/doc-guide/hello.dot b/Documentation/doc-guide/hello.dot
new file mode 100644
index 000000000000..504621dfc595
--- /dev/null
+++ b/Documentation/doc-guide/hello.dot
@@ -0,0 +1,3 @@
+graph G {
+ Hello -- World
+}
diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index 532d65b70500..b902744ce7dd 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -34,8 +34,10 @@ format-specific subdirectories under ``Documentation/output``.
To generate documentation, Sphinx (``sphinx-build``) must obviously be
installed. For prettier HTML output, the Read the Docs Sphinx theme
-(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is also
-needed. All of these are widely available and packaged in distributions.
+(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need
+``XeLaTeX`` and CairoSVG (http://cairosvg.org) or alternatively ``convert(1)``
+from ImageMagick (https://www.imagemagick.org). All of these are widely
+available and packaged in distributions.
To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make
variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose
@@ -232,3 +234,87 @@ Rendered as:
* .. _`last row`:
- column 3
+
+
+Figures & Images
+================
+
+If you want to add an image, you should use the ``kernel-figure`` and
+``kernel-image`` directives. E.g. to insert a figure with a scalable
+image format use SVG::
+
+ .. kernel-figure:: svg_image.svg
+ :alt: simple SVG image
+
+ SVG image example
+
+.. kernel-figure:: svg_image.svg
+ :alt: simple SVG image
+
+ SVG image example
+
+The kernel figure (and image) directive support **DOT** formated files, see
+
+* DOT: http://graphviz.org/pdf/dotguide.pdf
+* Graphviz: http://www.graphviz.org/content/dot-language
+
+A simple example::
+
+ .. kernel-figure:: hello.dot
+ :alt: hello world
+
+ DOT's hello world example
+
+.. kernel-figure:: hello.dot
+ :alt: hello world
+
+ DOT's hello world example
+
+Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the
+``kernel-render`` directives.::
+
+ .. kernel-render:: DOT
+ :alt: foobar digraph
+ :caption: Embedded **DOT** (Graphviz) code.
+
+ digraph foo {
+ "bar" -> "baz";
+ }
+
+How this will be rendered depends on the installed tools. If Graphviz is
+installed, you will see an vector image. If not the raw markup is inserted as
+*literal-block*.
+
+.. kernel-render:: DOT
+ :alt: foobar digraph
+ :caption: Embedded **DOT** (Graphviz) code.
+
+ digraph foo {
+ "bar" -> "baz";
+ }
+
+The *render* directive has all the options known from the *figure* directive,
+plus option ``caption``. If ``caption`` has a value, a *figure* node is
+inserted. If not, a *image* node is inserted.
+
+Embedded **SVG**::
+
+ .. kernel-render:: SVG
+ :caption: Embedded **SVG** markup.
+ :alt: so-nw-arrow
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <svg xmlns="http://www.w3.org/2000/svg" version="1.1" ...>
+ ...
+ </svg>
+
+.. kernel-render:: SVG
+ :caption: Embedded **SVG** markup.
+ :alt: so-nw-arrow
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <svg xmlns="http://www.w3.org/2000/svg"
+ version="1.1" baseProfile="full" width="70px" height="40px" viewBox="0 0 700 400">
+ <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
+ <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
+ </svg>
diff --git a/Documentation/doc-guide/svg_image.svg b/Documentation/doc-guide/svg_image.svg
new file mode 100644
index 000000000000..5405f85b8137
--- /dev/null
+++ b/Documentation/doc-guide/svg_image.svg
@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- originate: https://commons.wikimedia.org/wiki/File:Variable_Resistor.svg -->
+<svg xmlns="http://www.w3.org/2000/svg"
+ version="1.1" baseProfile="full"
+ width="70px" height="40px" viewBox="0 0 700 400">
+ <line x1="0" y1="200" x2="700" y2="200" stroke="black" stroke-width="20px"/>
+ <rect x="100" y="100" width="500" height="200" fill="white" stroke="black" stroke-width="20px"/>
+ <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
+ <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
+</svg>
diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index 56ce66114665..e4f25038ef65 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -318,9 +318,10 @@ PDF outputs, it is recommended to use version 1.4.6.
.. note::
Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX``
- version 3.14159265. Depending on the distribution, you may also need
- to install a series of ``texlive`` packages that provide the minimal
- set of functionalities required for ``XeLaTex`` to work.
+ version 3.14159265. Depending on the distribution, you may also need to
+ install a series of ``texlive`` packages that provide the minimal set of
+ functionalities required for ``XeLaTex`` to work. For PDF output you'll also
+ need ``convert(1)`` from ImageMagick (https://www.imagemagick.org).
Other tools
-----------
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
new file mode 100644
index 000000000000..32eab0f4cfba
--- /dev/null
+++ b/Documentation/sphinx/kfigure.py
@@ -0,0 +1,450 @@
+# -*- coding: utf-8; mode: python -*-
+# pylint: disable=C0103
+u"""
+ scalable figure and image handling
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ Sphinx extension which implements scalable image handling.
+
+ :copyright: Copyright (C) 2016 Markus Heiser
+ :license: GPL Version 2, June 1991 see Linux/COPYING for details.
+
+ The build for image formats depence on image's source format and output's
+ destination format. This extension implement methods to simplify image
+ handling from the author's POV. Directives like ``kernel-figure`` implement
+ methods *to* always get the best output-format even if some tools are not
+ installed.For more details take a look at ``convert_image(...)`` which is
+ the core of all conversions.
+
+ * ``.. kernel-image``: for image handling / ``.. image::`` replacement
+
+ * ``.. kernel-figure``: for figure handling / ``.. figure::`` replacement
+
+ * ``.. kernel-render``: for render markup / a concept to embed *render*
+ markups (or languages). Supported markups (see ``RENDER_MARKUP_EXT``)
+
+ + ``DOT``: render embedded Graphviz's **DOC**
+ + ``SVG``: render embedded Scalable Vector Graphics (**SVG**)
+ + ... *developable*
+
+ Used tools:
+
+ * ``dot(1)``: Graphviz (http://www.graphviz.org). If Graphviz is not
+ available, the DOT language is inserted as literal-block.
+
+ * SVG to PDF: To generate PDF, you need at least one of this tools:
+
+ - ``convert(1)``: ImageMagick (https://www.imagemagick.org)
+
+ List of customizations:
+
+ * generate PDF from SVG / used by PDF (LaTeX) builder
+
+ * generate SVG (html-builder) and PDF (latex-builder) from DOT files.
+ DOT: see http://www.graphviz.org/content/dot-language
+
+ """
+
+import os
+from os import path
+import subprocess
+from hashlib import sha1
+import sys
+
+from docutils import nodes
+from docutils.statemachine import ViewList
+from docutils.parsers.rst import directives
+from docutils.parsers.rst.directives import images
+
+import sphinx
+
+# Get Sphinx version
+major, minor, patch = map(int, sphinx.__version__.split("."))
+if major == 1 and minor > 3:
+ # patches.Figure only landed in Sphinx 1.4
+ from sphinx.directives.patches import Figure
+else:
+ Figure = images.Figure
+
+__version__ = '1.0'
+
+# simple helper
+# -------------
+
+def mkdir(folder, mode=0o775):
+ if not path.isdir(folder):
+ os.makedirs(folder, mode)
+
+# def debug_handle(self, node): # pylint: disable=W0613
+# from linuxdoc.kernel_doc import CONSOLE
+# CONSOLE()
+
+def pass_handle(self, node): # pylint: disable=W0613
+ pass
+
+# setup conversion tools and sphinx extension
+# -------------------------------------------
+
+def setup(app):
+ # image handling
+ app.add_directive("kernel-image", KernelImage)
+ app.add_node(kernel_image,
+ html = (visit_kernel_image, pass_handle),
+ latex = (visit_kernel_image, pass_handle),
+ texinfo = (visit_kernel_image, pass_handle),
+ text = (visit_kernel_image, pass_handle),
+ man = (visit_kernel_image, pass_handle), )
+
+ # figure handling
+ app.add_directive("kernel-figure", KernelFigure)
+ app.add_node(kernel_figure,
+ html = (visit_kernel_figure, pass_handle),
+ latex = (visit_kernel_figure, pass_handle),
+ texinfo = (visit_kernel_figure, pass_handle),
+ text = (visit_kernel_figure, pass_handle),
+ man = (visit_kernel_figure, pass_handle), )
+
+ # render handling
+ app.add_directive('kernel-render', KernelRender)
+ app.add_node(kernel_render,
+ html = (visit_kernel_render, pass_handle),
+ latex = (visit_kernel_render, pass_handle),
+ texinfo = (visit_kernel_render, pass_handle),
+ text = (visit_kernel_render, pass_handle),
+ man = (visit_kernel_render, pass_handle), )
+
+ return dict(
+ version = __version__,
+ parallel_read_safe = True,
+ parallel_write_safe = True
+ )
+
+# integrate conversion tools
+# --------------------------
+
+RENDER_MARKUP_EXT = {
+ # The '.ext' must be handled by convert_image(..) function's *in_ext* input.
+ # <name> : <.ext>
+ 'DOT' : '.dot'
+ , 'SVG' : '.svg'
+}
+
+def convert_image(img_node, translator): # pylint: disable=R0912
+ """Convert an image node for the builder.
+
+ Different builder prefer different image formats, e.g. *latex* builder
+ prefer PDF while *html* builder prefer SVG format for images.
+
+ This function handles outputs image formats in depence of source the format
+ of the image and the translator's output format. This also means to
+ manipulate/update the *image* dictionary of the builder (``builder.images``)
+
+ """
+ fname, in_ext = path.splitext(path.basename(img_node['uri']))
+ src_fname = path.join(translator.builder.srcdir, img_node['uri'])
+ src_folder = path.dirname(img_node['uri'])
+ out_dir = translator.builder.outdir
+ dst_fname = None
+
+ # in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages
+ verbose = translator.builder.app.verbose
+ warn = translator.builder.warn
+
+ verbose('assert best format for: ' + img_node['uri'])
+
+ if in_ext == '.dot':
+
+ # ----------
+ # handle DOT
+ # ----------
+
+ dst_fname = path.join(out_dir, fname + '.pdf')
+
+ if translator.builder.format == 'html':
+ dst_fname = path.join(out_dir, src_folder, fname + '.svg')
+ else:
+ # all other builder formats will include DOT as raw
+ with open(src_fname, "r") as dot:
+ data = dot.read()
+ node = nodes.literal_block(data, data)
+ img_node.replace_self(node)
+
+
+ elif in_ext == '.svg':
+
+ # ----------
+ # handle SVG
+ # ----------
+
+ if translator.builder.format == 'latex':
+ dst_fname = path.join(out_dir, fname + '.pdf')
+
+ if dst_fname:
+ name = dst_fname[len(out_dir) + 1:]
+ # the builder needs not to copy one more time, so pop it if exists.
+ translator.builder.images.pop(img_node['uri'], None)
+ img_node['uri'] = dst_fname
+ img_node['candidates'] = {'*': dst_fname}
+
+ mkdir(path.dirname(dst_fname))
+
+ if in_ext == '.dot':
+ verbose('convert DOT to: {out}/' + name)
+ dot2format(src_fname, dst_fname)
+
+ elif in_ext == '.svg':
+ verbose('convert SVG to: {out}/' + name)
+ svg2pdf(src_fname, dst_fname)
+
+def dot2format(dot_fname, out_fname):
+ """Converts DOT file to ``out_fname`` using ``dot(1)``.
+
+ * ``dot_fname`` pathname of the input DOT file, including extension ``.dot``
+ * ``out_fname`` pathname of the output file, including format extension
+
+ The *format extension* depends on the ``dot`` command (see ``man dot``
+ option ``-Txxx``). Normally you will use one of the following extensions:
+
+ - ``.ps`` for PostScript,
+ - ``.svg`` or ``svgz`` for Structured Vector Graphics,
+ - ``.fig`` for XFIG graphics and
+ - ``.png`` or ``gif`` for common bitmap graphics.
+
+ """
+ out_format = path.splitext(out_fname)[1][1:]
+ cmd = ['dot', '-T%s' % out_format, dot_fname]
+ exit_code = 42
+ with open(out_fname, "w") as out:
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
+ sys.stderr.write(err)
+
+ exit_code = p.returncode
+ out.flush()
+ return bool(exit_code == 0)
+
+def svg2pdf(svg_fname, pdf_fname):
+ """Converts SVG to PDF with CairoSVG or ``convert(1)`` command.
+
+ Uses ``convert(1)`` from ImageMagick (https://www.imagemagick.org) for
+ conversion. Returns ``True`` on success and ``False`` if an error occurred
+ (e.g. none of the conversion tool is available).
+
+ * ``svg_fname`` pathname of the input SVG file with extension (``.svg``)
+ * ``pdf_name`` pathname of the output PDF file with extension (``.pdf``)
+
+ """
+ cmd = [convert_cmd, svg_fname, pdf_fname]
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
+ sys.stderr.write(err)
+
+ exit_code = p.returncode
+ return bool(exit_code == 0)
+
+
+# image handling
+# ---------------------
+
+def visit_kernel_image(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_image`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0]
+ convert_image(img_node, self)
+
+class kernel_image(nodes.General, nodes.Element):
+ """Node for ``kernel-image`` directive."""
+ pass
+
+class KernelImage(images.Image):
+ u"""KernelImage directive
+
+ Earns everything from ``.. image::`` directive, except *remote URI* and
+ *glob* pattern. The KernelImage wraps a image node into a
+ kernel_image node. See ``visit_kernel_image``.
+ """
+
+ def run(self):
+ env = self.state.document.settings.env
+
+ uri = self.arguments[0]
+ if uri.endswith('.*') or uri.find('://') != -1:
+ raise self.severe(
+ 'Error in "%s: %s": glob pattern and remote images are not allowed'
+ % (self.name, uri))
+
+ # Tell sphinx of the dependency
+ env.note_dependency(os.path.abspath(uri))
+
+ result = images.Image.run(self)
+ if len(result) == 2 or isinstance(result[0], nodes.system_message):
+ return result
+ (image_node,) = result
+ # wrap image node into a kernel_image node / see visitors
+ node = kernel_image('', image_node)
+ return [node]
+
+# figure handling
+# ---------------------
+
+def visit_kernel_figure(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_figure`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0][0]
+ convert_image(img_node, self)
+
+class kernel_figure(nodes.General, nodes.Element):
+ """Node for ``kernel-figure`` directive."""
+
+class KernelFigure(Figure):
+ u"""KernelImage directive
+
+ Earns everything from ``.. figure::`` directive, except *remote URI* and
+ *glob* pattern. The KernelFigure wraps a figure node into a kernel_figure
+ node. See ``visit_kernel_figure``.
+ """
+
+ def run(self):
+ env = self.state.document.settings.env
+
+ uri = self.arguments[0]
+ if uri.endswith('.*') or uri.find('://') != -1:
+ raise self.severe(
+ 'Error in "%s: %s":'
+ ' glob pattern and remote images are not allowed'
+ % (self.name, uri))
+
+ # Tell sphinx of the dependency
+ env.note_dependency(os.path.abspath(uri))
+
+ result = Figure.run(self)
+ if len(result) == 2 or isinstance(result[0], nodes.system_message):
+ return result
+ (figure_node,) = result
+ # wrap figure node into a kernel_figure node / see visitors
+ node = kernel_figure('', figure_node)
+ return [node]
+
+
+# render handling
+# ---------------------
+
+def visit_kernel_render(self, node):
+ """Visitor of the ``kernel_render`` Node.
+
+ If rendering tools available, save the markup of the ``literal_block`` child
+ node into a file and replace the ``literal_block`` node with a new created
+ ``image`` node, pointing to the saved markup file. Afterwards, handle the
+ image child-node with the ``convert_image(...)``.
+ """
+
+ verbose = self.builder.app.verbose
+ warn = self.builder.warn
+ srclang = node.get('srclang')
+
+ verbose('visit kernel-render node lang: "%s"' % (srclang))
+
+ tmp_ext = RENDER_MARKUP_EXT.get(srclang, None)
+ if tmp_ext is None:
+ warn('kernel-render: "%s" unknow / include raw.' % (srclang))
+ return
+
+ literal_block = node[0]
+ code = literal_block.astext()
+
+ if tmp_ext:
+ hashobj = code.encode('utf-8') # str(node.attributes)
+ fname = '%s-%s' % (srclang, sha1(hashobj).hexdigest())
+ tmp_fname = path.join(
+ self.builder.outdir, self.builder.imagedir, fname + tmp_ext)
+
+ if not path.isfile(tmp_fname):
+ mkdir(path.dirname(tmp_fname))
+ with open(tmp_fname, "w") as out:
+ out.write(code)
+
+ image_node = nodes.image(node.rawsource, **node.attributes)
+ image_node['uri'] = tmp_fname
+
+ literal_block.replace_self(image_node)
+ convert_image(image_node, self)
+
+
+class kernel_render(nodes.General, nodes.Inline, nodes.Element):
+ """Node for ``kernel-render`` directive."""
+ pass
+
+class KernelRender(Figure):
+ u"""KernelRender directive
+
+ Render content by external tool. Has all the options known from the
+ *figure* directive, plus option ``caption``. If ``caption`` has a
+ value, a figure node with the *caption* is inserted. If not, a image node is
+ inserted.
+
+ The KernelRender directive wraps the text of the directive into a
+ literal_block node and wraps it into a kernel_render node. See
+ ``visit_kernel_render``.
+ """
+ has_content = True
+ required_arguments = 1
+ optional_arguments = 0
+ final_argument_whitespace = False
+
+ # earn options from 'figure'
+ option_spec = Figure.option_spec.copy()
+ option_spec['caption'] = directives.unchanged
+
+ def run(self):
+ return [self.build_node()]
+
+ def build_node(self):
+
+ srclang = self.arguments[0].strip()
+ if srclang not in RENDER_MARKUP_EXT.keys():
+ return [self.state_machine.reporter.warning(
+ 'Unknow source language "%s", use one of: %s.' % (
+ srclang, ",".join(RENDER_MARKUP_EXT.keys())),
+ line=self.lineno)]
+
+ code = '\n'.join(self.content)
+ if not code.strip():
+ return [self.state_machine.reporter.warning(
+ 'Ignoring "%s" directive without content.' % (
+ self.name),
+ line=self.lineno)]
+
+ node = kernel_render()
+ node['alt'] = self.options.get('alt','')
+ node['srclang'] = srclang
+ literal_node = nodes.literal_block(code, code)
+ node += literal_node
+
+ caption = self.options.get('caption')
+ if caption:
+ # parse cation's content
+ parsed = nodes.Element()
+ self.state.nested_parse(
+ ViewList([caption], source=''), self.content_offset, parsed)
+ caption_node = nodes.caption(
+ parsed[0].rawsource, '', *parsed[0].children)
+ caption_node.source = parsed[0].source
+ caption_node.line = parsed[0].line
+
+ figure_node = nodes.figure('', node)
+ for k,v in self.options.items():
+ figure_node[k] = v
+ figure_node += caption_node
+
+ node = figure_node
+
+ return node
+
--
2.11.0
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:21 ` Markus Heiser
@ 2017-03-02 15:45 ` Mauro Carvalho Chehab
2017-03-02 15:49 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 15:45 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, Laurent Pinchart, Daniel Vetter, dri-devel,
Linux Doc Mailing List, Jonathan Corbet, Daniel Vetter
Em Thu, 2 Mar 2017 16:21:33 +0100
Markus Heiser <markus.heiser@darmarit.de> escreveu:
> Am 02.03.2017 um 16:11 schrieb Daniel Vetter <daniel@ffwll.ch>:
>
> > On Thu, Mar 02, 2017 at 03:58:36PM +0100, Markus Heiser wrote:
> >> Hi Daniel, Laurent
> >>
> >> Am 02.03.2017 um 15:14 schrieb Laurent Pinchart <laurent.pinchart@ideasonboard.com>:
> >>
> >>> Hi Daniel,
> >>>
> >>> On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:
> >>>> On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:
> >>>>> Hi Daniel,
> >>>>>
> >>>>> Thank you for the patch.
> >>>>>
> >>>>> With this applied, I get
> >>>>>
> >>>>> make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
> >>>>>
> >>>>> SPHINX htmldocs -->
> >>>>> file:///home/laurent/src/iob/renesas/linux64/Documentation/output PARSE
> >>>>> include/uapi/linux/videodev2.h
> >>>>>
> >>>>> Running Sphinx v1.3.1
> >>>>>
> >>>>> Extension error:
> >>>>> Could not import extension kfigure (exception: cannot import name patches)
> >>>>> make[2]: ***
> >>>>> [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
> >>>>> htmldocs] Error 1 make[1]: ***
> >>>>> [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
> >>>>> make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64' make:
> >>>>> *** [Makefile:152: sub-make] Error 2
> >>>>>
> >>>>> sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump
> >>>>> the minimum required version I think a notice is needed.
> >>>>
> >>>> Ugh. But this also goes completely over my head, no idea whether we
> >>>> must require sphinx 1.4 (it was released Mar 28, 2016), or whether
> >>>> there's some way to work around this ... Halp?
> >>>
> >>> I'm not a Sphinx expert so I don't know, but what I can tell is that copying
> >>> the patches.py from Sphinx 1.4 to Documentation/sphinx/ and modifying
> >>> kfigure.py to import it from there fixes the build. There's thus no extra
> >>> depencency on Sphinx 1.4 (or newer).
> >>>
> >>> I'm not sure we want to set a precedent by copying part of the Sphinx source
> >>> code to the kernel tree (or inlining the single small function that the module
> >>> provides), and I'll let someone more knowledgeable than me decide how to
> >>> proceed.
> >>
> >>
> >> Aargh ... we need virtualenv! For interim something like the following
> >> might help. In file Documentation/sphinx/kfigure.py edit the imports
> >>
> >> ...
> >> from docutils.parsers.rst.directives import images
> >> try:
> >> from sphinx.directives.patches import Figure
> >> except ImportError:
> >> Figure = images.Figure
> >> ...
> >>
> >> And fix the class definition, so it use 'Figure' and no
> >> longer 'patch.Figure'::
> >>
> >> ...
> >> -class KernelFigure(patches.Figure):
> >> +class KernelFigure(Figure):
> >> ...
> >>
> >> Sorry that I have not yet the time to send you a decent and tested
> >> patch. Do you like to test my suggestion? / thanks!
> >
> > I'll give it a shot at implementing it, but I can't (easily at least) test
> > on sphinx 1.3.
>
> You can test it with virtualenv https://virtualenv.pypa.io/en/stable/userguide/
>
> In short:
>
> $ cd kernel-src
> $ virtualenv myenv
> $ source myenv/bin/activate
> $ pip install 'Sphinx==1.3.1'
> $ make ....
Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
virtualenv is broken:
writing output... [ 16%] media/intro
Exception occurred:
File "/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
assert not self.context, 'len(context) = %s' % len(self.context)
AssertionError: len(context) = 1
The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
make[1]: *** [htmldocs] Error 1
Makefile:1450: recipe for target 'htmldocs' failed
make: *** [htmldocs] Error 2
Perhaps it is time for us to move minimal requirements to 1.4?
Thanks,
Mauro
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:45 ` Mauro Carvalho Chehab
@ 2017-03-02 15:49 ` Mauro Carvalho Chehab
2017-03-02 16:13 ` Markus Heiser
0 siblings, 1 reply; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 15:49 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, Laurent Pinchart, Daniel Vetter, dri-devel,
Linux Doc Mailing List, Jonathan Corbet, Daniel Vetter
Em Thu, 2 Mar 2017 12:45:32 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
> Em Thu, 2 Mar 2017 16:21:33 +0100
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
>
> > Am 02.03.2017 um 16:11 schrieb Daniel Vetter <daniel@ffwll.ch>:
> >
> > > On Thu, Mar 02, 2017 at 03:58:36PM +0100, Markus Heiser wrote:
> > >> Hi Daniel, Laurent
> > >>
> > >> Am 02.03.2017 um 15:14 schrieb Laurent Pinchart <laurent.pinchart@ideasonboard.com>:
> > >>
> > >>> Hi Daniel,
> > >>>
> > >>> On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:
> > >>>> On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:
> > >>>>> Hi Daniel,
> > >>>>>
> > >>>>> Thank you for the patch.
> > >>>>>
> > >>>>> With this applied, I get
> > >>>>>
> > >>>>> make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
> > >>>>>
> > >>>>> SPHINX htmldocs -->
> > >>>>> file:///home/laurent/src/iob/renesas/linux64/Documentation/output PARSE
> > >>>>> include/uapi/linux/videodev2.h
> > >>>>>
> > >>>>> Running Sphinx v1.3.1
> > >>>>>
> > >>>>> Extension error:
> > >>>>> Could not import extension kfigure (exception: cannot import name patches)
> > >>>>> make[2]: ***
> > >>>>> [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
> > >>>>> htmldocs] Error 1 make[1]: ***
> > >>>>> [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
> > >>>>> make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64' make:
> > >>>>> *** [Makefile:152: sub-make] Error 2
> > >>>>>
> > >>>>> sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump
> > >>>>> the minimum required version I think a notice is needed.
> > >>>>
> > >>>> Ugh. But this also goes completely over my head, no idea whether we
> > >>>> must require sphinx 1.4 (it was released Mar 28, 2016), or whether
> > >>>> there's some way to work around this ... Halp?
> > >>>
> > >>> I'm not a Sphinx expert so I don't know, but what I can tell is that copying
> > >>> the patches.py from Sphinx 1.4 to Documentation/sphinx/ and modifying
> > >>> kfigure.py to import it from there fixes the build. There's thus no extra
> > >>> depencency on Sphinx 1.4 (or newer).
> > >>>
> > >>> I'm not sure we want to set a precedent by copying part of the Sphinx source
> > >>> code to the kernel tree (or inlining the single small function that the module
> > >>> provides), and I'll let someone more knowledgeable than me decide how to
> > >>> proceed.
> > >>
> > >>
> > >> Aargh ... we need virtualenv! For interim something like the following
> > >> might help. In file Documentation/sphinx/kfigure.py edit the imports
> > >>
> > >> ...
> > >> from docutils.parsers.rst.directives import images
> > >> try:
> > >> from sphinx.directives.patches import Figure
> > >> except ImportError:
> > >> Figure = images.Figure
> > >> ...
> > >>
> > >> And fix the class definition, so it use 'Figure' and no
> > >> longer 'patch.Figure'::
> > >>
> > >> ...
> > >> -class KernelFigure(patches.Figure):
> > >> +class KernelFigure(Figure):
> > >> ...
> > >>
> > >> Sorry that I have not yet the time to send you a decent and tested
> > >> patch. Do you like to test my suggestion? / thanks!
> > >
> > > I'll give it a shot at implementing it, but I can't (easily at least) test
> > > on sphinx 1.3.
> >
> > You can test it with virtualenv https://virtualenv.pypa.io/en/stable/userguide/
> >
> > In short:
> >
> > $ cd kernel-src
> > $ virtualenv myenv
> > $ source myenv/bin/activate
> > $ pip install 'Sphinx==1.3.1'
> > $ make ....
>
> Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
> virtualenv is broken:
>
> writing output... [ 16%] media/intro
> Exception occurred:
> File "/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
> assert not self.context, 'len(context) = %s' % len(self.context)
> AssertionError: len(context) = 1
> The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you want to report the issue to the developers.
> Please also report this if it was a user error, so that a better error message can be provided next time.
> A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> make[1]: *** [htmldocs] Error 1
> Makefile:1450: recipe for target 'htmldocs' failed
> make: *** [htmldocs] Error 2
>
> Perhaps it is time for us to move minimal requirements to 1.4?
Hmm... the same happened with 1.4.9 inside virtualenv. It built fine
with 1.5.2 (for htmldocs).
Thanks,
Mauro
-
This is the error log with 1.4:
# Sphinx version: 1.4.9
# Python version: 2.7.13 (CPython)
# Docutils version: 0.13.1 release
# Jinja2 version: 2.9.5
# Last messages:
# writing output... [ 18%] media/dvb-drivers/faq
# writing output... [ 18%] media/dvb-drivers/index
# writing output... [ 18%] media/dvb-drivers/intro
# writing output... [ 18%] media/dvb-drivers/lmedm04
# writing output... [ 18%] media/dvb-drivers/opera-firmware
# writing output... [ 18%] media/dvb-drivers/technisat
# writing output... [ 19%] media/dvb-drivers/ttusb-dec
# writing output... [ 19%] media/dvb-drivers/udev
# writing output... [ 19%] media/index
# writing output... [ 19%] media/intro
# Loaded extensions:
# kernel_include (1.0) from /devel/v4l/patchwork/Documentation/sphinx/kernel_include.pyc
# rstFlatTable (1.0) from /devel/v4l/patchwork/Documentation/sphinx/rstFlatTable.pyc
# cdomain (1.0) from /devel/v4l/patchwork/Documentation/sphinx/cdomain.pyc
# kerneldoc (1.0) from /devel/v4l/patchwork/Documentation/sphinx/kerneldoc.pyc
# alabaster (0.7.10) from /devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/alabaster/__init__.pyc
# sphinx.ext.imgmath (1.4.9) from /devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/sphinx/ext/imgmath.pyc
Traceback (most recent call last):
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/sphinx/cmdline.py", line 244, in main
app.build(opts.force_all, filenames)
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/sphinx/application.py", line 297, in build
self.builder.build_update()
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 251, in build_update
'out of date' % len(to_build))
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 322, in build
self.write(docnames, list(updated_docnames), method)
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 360, in write
self._write_serial(sorted(docnames), warnings)
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 368, in _write_serial
self.write_doc(docname, doctree)
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/sphinx/builders/html.py", line 448, in write_doc
self.docwriter.write(doctree, destination)
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/__init__.py", line 80, in write
self.translate()
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/sphinx/writers/html.py", line 47, in translate
self.document.walkabout(visitor)
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py", line 187, in walkabout
visitor.dispatch_departure(self)
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py", line 1895, in dispatch_departure
return method(node)
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
assert not self.context, 'len(context) = %s' % len(self.context)
AssertionError: len(context) = 1
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:22 ` Jonathan Corbet
@ 2017-03-02 15:53 ` Daniel Vetter
2017-03-02 16:01 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 15:53 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Markus Heiser, Linux Doc Mailing List, dri-devel,
Mauro Carvalho Chehab, Laurent Pinchart, Daniel Vetter
On Thu, Mar 2, 2017 at 4:22 PM, Jonathan Corbet <corbet@lwn.net> wrote:
> On Thu, 2 Mar 2017 16:11:08 +0100
> Daniel Vetter <daniel@ffwll.ch> wrote:
>
>> I'll give it a shot at implementing it, but I can't (easily at least) test
>> on sphinx 1.3.
>
> Virtualenv makes that kind of thing pretty easy to do. Maybe we should
> add a script to create and populate a suitable venv for this kind of
> thing.
Laurent quickly checked v5 of this patch, looks all good now.
> Meanwhile, I've been seeing only parts 1 and 2 of what's clearly a bigger
> series. Do you want me to carry just these two?
I submitted the entire series to both linux-doc and dri-devel. But
yeah, probably best if you just pull in the first 2 and send me a
topic pull for drm so I can apply the drm stuff on top in drm-misc.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:53 ` Daniel Vetter
@ 2017-03-02 16:01 ` Mauro Carvalho Chehab
2017-03-02 19:49 ` Daniel Vetter
0 siblings, 1 reply; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 16:01 UTC (permalink / raw)
To: Daniel Vetter
Cc: Jonathan Corbet, Markus Heiser, Laurent Pinchart, dri-devel,
Linux Doc Mailing List, Daniel Vetter
Em Thu, 2 Mar 2017 16:53:04 +0100
Daniel Vetter <daniel@ffwll.ch> escreveu:
> On Thu, Mar 2, 2017 at 4:22 PM, Jonathan Corbet <corbet@lwn.net> wrote:
> > On Thu, 2 Mar 2017 16:11:08 +0100
> > Daniel Vetter <daniel@ffwll.ch> wrote:
> >
> >> I'll give it a shot at implementing it, but I can't (easily at least) test
> >> on sphinx 1.3.
> >
> > Virtualenv makes that kind of thing pretty easy to do. Maybe we should
> > add a script to create and populate a suitable venv for this kind of
> > thing.
>
> Laurent quickly checked v5 of this patch, looks all good now.
>
> > Meanwhile, I've been seeing only parts 1 and 2 of what's clearly a bigger
> > series. Do you want me to carry just these two?
>
> I submitted the entire series to both linux-doc and dri-devel.
Sure you sent to linux-doc? I'm not seeing it there.
I'm not seeing it at the mirrors neither:
https://marc.info/?l=linux-doc&r=1&b=201703&w=2
https://www.spinics.net/lists/linux-doc/maillist.html
Regards,
Mauro
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:49 ` Mauro Carvalho Chehab
@ 2017-03-02 16:13 ` Markus Heiser
2017-03-02 16:29 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 56+ messages in thread
From: Markus Heiser @ 2017-03-02 16:13 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Jonathan Corbet, Daniel Vetter, Linux Doc Mailing List, dri-devel,
Laurent Pinchart, Daniel Vetter
> Am 02.03.2017 um 16:49 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
>
>>> You can test it with virtualenv https://virtualenv.pypa.io/en/stable/userguide/
>>>
>>> In short:
>>>
>>> $ cd kernel-src
>>> $ virtualenv myenv
>>> $ source myenv/bin/activate
>>> $ pip install 'Sphinx==1.3.1'
>>> $ make ....
>>
>> Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
>> virtualenv is broken:
>>
>> writing output... [ 16%] media/intro
>> Exception occurred:
>> File "/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
>> assert not self.context, 'len(context) = %s' % len(self.context)
>> AssertionError: len(context) = 1
>> The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you want to report the issue to the developers.
>> Please also report this if it was a user error, so that a better error message can be provided next time.
>> A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
>> make[1]: *** [htmldocs] Error 1
>> Makefile:1450: recipe for target 'htmldocs' failed
>> make: *** [htmldocs] Error 2
>>
>> Perhaps it is time for us to move minimal requirements to 1.4?
>
> Hmm... the same happened with 1.4.9 inside virtualenv. It built fine
> with 1.5.2 (for htmldocs).
>
> Thanks,
> Mauro
>
> -
>
> This is the error log with 1.4:
>
> # Sphinx version: 1.4.9
....
> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py", line 187, in walkabout
> visitor.dispatch_departure(self)
> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py", line 1895, in dispatch_departure
> return method(node)
> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
> assert not self.context, 'len(context) = %s' % len(self.context)
> AssertionError: len(context) = 1
I guess this is a error from newer docutils, so we have to fix docutils version also.
Sphinx itself specifies "docutils>=0.11"
https://github.com/sphinx-doc/sphinx/blob/1.3.1/setup.py#L52
So I guess it uses a up to date docutils or the ducutils which are already installed system wide.
— Markus —
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 16:13 ` Markus Heiser
@ 2017-03-02 16:29 ` Mauro Carvalho Chehab
2017-03-02 18:16 ` Markus Heiser
0 siblings, 1 reply; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 16:29 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, Laurent Pinchart, Daniel Vetter, dri-devel,
Linux Doc Mailing List, Jonathan Corbet, Daniel Vetter
Em Thu, 2 Mar 2017 17:13:25 +0100
Markus Heiser <markus.heiser@darmarit.de> escreveu:
> > Am 02.03.2017 um 16:49 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> >
> >>> You can test it with virtualenv https://virtualenv.pypa.io/en/stable/userguide/
> >>>
> >>> In short:
> >>>
> >>> $ cd kernel-src
> >>> $ virtualenv myenv
> >>> $ source myenv/bin/activate
> >>> $ pip install 'Sphinx==1.3.1'
> >>> $ make ....
> >>
> >> Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
> >> virtualenv is broken:
> >>
> >> writing output... [ 16%] media/intro
> >> Exception occurred:
> >> File "/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
> >> assert not self.context, 'len(context) = %s' % len(self.context)
> >> AssertionError: len(context) = 1
> >> The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you want to report the issue to the developers.
> >> Please also report this if it was a user error, so that a better error message can be provided next time.
> >> A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> >> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> >> make[1]: *** [htmldocs] Error 1
> >> Makefile:1450: recipe for target 'htmldocs' failed
> >> make: *** [htmldocs] Error 2
> >>
> >> Perhaps it is time for us to move minimal requirements to 1.4?
> >
> > Hmm... the same happened with 1.4.9 inside virtualenv. It built fine
> > with 1.5.2 (for htmldocs).
> >
> > Thanks,
> > Mauro
> >
> > -
> >
> > This is the error log with 1.4:
> >
> > # Sphinx version: 1.4.9
> ....
> > File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py", line 187, in walkabout
> > visitor.dispatch_departure(self)
> > File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py", line 1895, in dispatch_departure
> > return method(node)
> > File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
> > assert not self.context, 'len(context) = %s' % len(self.context)
> > AssertionError: len(context) = 1
>
> I guess this is a error from newer docutils, so we have to fix docutils version also.
>
> Sphinx itself specifies "docutils>=0.11"
>
> https://github.com/sphinx-doc/sphinx/blob/1.3.1/setup.py#L52
>
> So I guess it uses a up to date docutils or the ducutils which are already installed system wide.
The system-wide docutils is this one:
python2-docutils-0.13.1-3.fc25.noarch
python3-docutils-0.13.1-3.fc25.noarch
Btw, I tested also with virtualenv-3/pip3 and the same issue happens there.
Manually installing version 0.11 makes it to work again.
Considering that Sphinx require a specific docutils package for it to
work, perhaps it is time for us to consider to use the virtenv
enchantments at make docs targets :-p
Thanks,
Mauro
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:40 ` [PATCH] " Daniel Vetter
@ 2017-03-02 18:06 ` Mauro Carvalho Chehab
2017-03-02 19:06 ` Markus Heiser
2017-03-02 21:29 ` Mauro Carvalho Chehab
1 sibling, 1 reply; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 18:06 UTC (permalink / raw)
To: Daniel Vetter
Cc: linux-doc, DRI Development, Markus Heiser, Jonathan Corbet,
Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 16:40:02 +0100
Daniel Vetter <daniel.vetter@ffwll.ch> escreveu:
> From: Markus Heiser <markus.heiser@darmarit.de>
>
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
>
> * DOT (http://www.graphviz.org)
> * SVG
>
> For image handling use the 'image' replacement::
>
> .. kernel-image:: svg_image.svg
> :alt: simple SVG image
>
> For figure handling use the 'figure' replacement::
>
> .. kernel-figure:: svg_image.svg
> :alt: simple SVG image
>
> SVG image example
>
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
>
> .. kernel-render:: DOT
> :alt: foobar digraph
> :caption: Embedded **DOT** (Graphviz) code.
>
> digraph foo {
> "bar" -> "baz";
> }
>
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
>
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
>
> v2: s/DOC/DOT/ in a few places (by Daniel).
>
> v3: Simplify stuff a bit (by Daniel):
>
> - Remove path detection and setup/check code for that. In
> Documentation/media/Makefile we already simply use these tools,
> better to have one consolidated check if we want/need one. Also
> remove the convertsvg support, we require ImageMagick's convert
> already in the doc build, no need for a 2nd fallback.
>
> - Use sphinx for depency tracking, remove hand-rolled version.
>
> - Forward stderr from dot and convert, otherwise debugging issues with
> the diagrams is impossible.
>
> v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
>
> v5: Use an explicit version check (suggested by Laurent).
Tested here with the enclosed patch. It crashed:
Exception occurred:
File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
sys.stderr.write(err)
TypeError: write() argument must be str, not bytes
The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
make[1]: *** [htmldocs] Error 1
Makefile:1450: recipe for target 'htmldocs' failed
make: *** [htmldocs] Error 2
Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
Full trace:
Traceback (most recent call last):
File "/home/mchehab/.local/lib/python3.5/site-packages/sphinx/util/parallel.py", line 73, in _process
ret = func(arg)
File "/home/mchehab/.local/lib/python3.5/site-packages/sphinx/builders/__init__.py", line 380, in write_process
self.write_doc(docname, doctree)
File "/home/mchehab/.local/lib/python3.5/site-packages/sphinx/builders/html.py", line 448, in write_doc
self.docwriter.write(doctree, destination)
File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/writers/__init__.py", line 80, in write
self.translate()
File "/home/mchehab/.local/lib/python3.5/site-packages/sphinx/writers/html.py", line 47, in translate
self.document.walkabout(visitor)
File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", line 174, in walkabout
if child.walkabout(visitor):
File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", line 174, in walkabout
if child.walkabout(visitor):
File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", line 174, in walkabout
if child.walkabout(visitor):
File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", line 166, in walkabout
visitor.dispatch_visit(self)
File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", line 1882, in dispatch_visit
return method(node)
File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 302, in visit_kernel_figure
convert_image(img_node, self)
File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 193, in convert_image
dot2format(src_fname, dst_fname)
File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
sys.stderr.write(err)
TypeError: write() argument must be str, not bytes
Thanks,
Mauro
--
[PATCH] docs-rst: Don't use explicit Makefile rules to build SVG and
DOT files
Now that we have an extension to handle images, use it.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
diff --git a/Documentation/media/Makefile b/Documentation/media/Makefile
index 32663602ff25..5bd52ea1eff0 100644
--- a/Documentation/media/Makefile
+++ b/Documentation/media/Makefile
@@ -1,51 +1,6 @@
-# Rules to convert DOT and SVG to Sphinx images
-
-SRC_DIR=$(srctree)/Documentation/media
-
-DOTS = \
- uapi/v4l/pipeline.dot \
-
-IMAGES = \
- typical_media_device.svg \
- uapi/dvb/dvbstb.svg \
- uapi/v4l/bayer.svg \
- uapi/v4l/constraints.svg \
- uapi/v4l/crop.svg \
- uapi/v4l/fieldseq_bt.svg \
- uapi/v4l/fieldseq_tb.svg \
- uapi/v4l/nv12mt.svg \
- uapi/v4l/nv12mt_example.svg \
- uapi/v4l/pipeline.svg \
- uapi/v4l/selection.svg \
- uapi/v4l/subdev-image-processing-full.svg \
- uapi/v4l/subdev-image-processing-scaling-multi-source.svg \
- uapi/v4l/subdev-image-processing-crop.svg \
- uapi/v4l/vbi_525.svg \
- uapi/v4l/vbi_625.svg \
- uapi/v4l/vbi_hsync.svg \
-
-DOTTGT := $(patsubst %.dot,%.svg,$(DOTS))
-IMGDOT := $(patsubst %,$(SRC_DIR)/%,$(DOTTGT))
-
-IMGTGT := $(patsubst %.svg,%.pdf,$(IMAGES))
-IMGPDF := $(patsubst %,$(SRC_DIR)/%,$(IMGTGT))
-
-cmd = $(echo-cmd) $(cmd_$(1))
-
-quiet_cmd_genpdf = GENPDF $2
- cmd_genpdf = convert $2 $3
-
-quiet_cmd_gendot = DOT $2
- cmd_gendot = dot -Tsvg $2 > $3
-
-%.pdf: %.svg
- @$(call cmd,genpdf,$<,$@)
-
-%.svg: %.dot
- @$(call cmd,gendot,$<,$@)
-
# Rules to convert a .h file to inline RST documentation
+SRC_DIR=$(srctree)/Documentation/media
PARSER = $(srctree)/Documentation/sphinx/parse-headers.pl
UAPI = $(srctree)/include/uapi/linux
KAPI = $(srctree)/include/linux
diff --git a/Documentation/media/intro.rst b/Documentation/media/intro.rst
index 8f7490c9a8ef..5562fba1d51d 100644
--- a/Documentation/media/intro.rst
+++ b/Documentation/media/intro.rst
@@ -13,8 +13,8 @@ A typical media device hardware is shown at :ref:`typical_media_device`.
.. _typical_media_device:
-.. figure:: typical_media_device.*
- :alt: typical_media_device.pdf / typical_media_device.svg
+.. kernel-figure:: typical_media_device.svg
+ :alt: typical_media_device.svg
:align: center
Typical Media Device
diff --git a/Documentation/media/uapi/dvb/intro.rst b/Documentation/media/uapi/dvb/intro.rst
index 2ed5c23102b4..0e76067a5b58 100644
--- a/Documentation/media/uapi/dvb/intro.rst
+++ b/Documentation/media/uapi/dvb/intro.rst
@@ -55,8 +55,8 @@ Overview
.. _stb_components:
-.. figure:: dvbstb.*
- :alt: dvbstb.pdf / dvbstb.svg
+.. kernel-figure:: dvbstb.svg
+ :alt: dvbstb.svg
:align: center
Components of a DVB card/STB
diff --git a/Documentation/media/uapi/v4l/crop.rst b/Documentation/media/uapi/v4l/crop.rst
index be58894c9c89..adea29b14135 100644
--- a/Documentation/media/uapi/v4l/crop.rst
+++ b/Documentation/media/uapi/v4l/crop.rst
@@ -53,8 +53,8 @@ Cropping Structures
.. _crop-scale:
-.. figure:: crop.*
- :alt: crop.pdf / crop.svg
+.. kernel-figure:: crop.svg
+ :alt: crop.svg
:align: center
Image Cropping, Insertion and Scaling
diff --git a/Documentation/media/uapi/v4l/dev-raw-vbi.rst b/Documentation/media/uapi/v4l/dev-raw-vbi.rst
index baf5f2483927..f761d3a93783 100644
--- a/Documentation/media/uapi/v4l/dev-raw-vbi.rst
+++ b/Documentation/media/uapi/v4l/dev-raw-vbi.rst
@@ -221,8 +221,8 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
.. _vbi-hsync:
-.. figure:: vbi_hsync.*
- :alt: vbi_hsync.pdf / vbi_hsync.svg
+.. kernel-figure:: vbi_hsync.svg
+ :alt: vbi_hsync.svg
:align: center
**Figure 4.1. Line synchronization**
@@ -230,8 +230,8 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
.. _vbi-525:
-.. figure:: vbi_525.*
- :alt: vbi_525.pdf / vbi_525.svg
+.. kernel-figure:: vbi_525.svg
+ :alt: vbi_525.svg
:align: center
**Figure 4.2. ITU-R 525 line numbering (M/NTSC and M/PAL)**
@@ -240,8 +240,8 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
.. _vbi-625:
-.. figure:: vbi_625.*
- :alt: vbi_625.pdf / vbi_625.svg
+.. kernel-figure:: vbi_625.svg
+ :alt: vbi_625.svg
:align: center
**Figure 4.3. ITU-R 625 line numbering**
diff --git a/Documentation/media/uapi/v4l/dev-subdev.rst b/Documentation/media/uapi/v4l/dev-subdev.rst
index cd2870180208..0a54e4d95013 100644
--- a/Documentation/media/uapi/v4l/dev-subdev.rst
+++ b/Documentation/media/uapi/v4l/dev-subdev.rst
@@ -99,8 +99,8 @@ the video sensor and the host image processing hardware.
.. _pipeline-scaling:
-.. figure:: pipeline.*
- :alt: pipeline.pdf / pipeline.svg
+.. kernel-figure:: pipeline.dot
+ :alt: pipeline.dot
:align: center
Image Format Negotiation on Pipelines
@@ -404,8 +404,8 @@ selection will refer to the sink pad format dimensions instead.
.. _subdev-image-processing-crop:
-.. figure:: subdev-image-processing-crop.*
- :alt: subdev-image-processing-crop.pdf / subdev-image-processing-crop.svg
+.. kernel-figure:: subdev-image-processing-crop.svg
+ :alt: subdev-image-processing-crop.svg
:align: center
**Figure 4.5. Image processing in subdevs: simple crop example**
@@ -421,8 +421,8 @@ pad.
.. _subdev-image-processing-scaling-multi-source:
-.. figure:: subdev-image-processing-scaling-multi-source.*
- :alt: subdev-image-processing-scaling-multi-source.pdf / subdev-image-processing-scaling-multi-source.svg
+.. kernel-figure:: subdev-image-processing-scaling-multi-source.svg
+ :alt: subdev-image-processing-scaling-multi-source.svg
:align: center
**Figure 4.6. Image processing in subdevs: scaling with multiple sources**
@@ -437,8 +437,8 @@ an area at location specified by the source crop rectangle from it.
.. _subdev-image-processing-full:
-.. figure:: subdev-image-processing-full.*
- :alt: subdev-image-processing-full.pdf / subdev-image-processing-full.svg
+.. kernel-figure:: subdev-image-processing-full.svg
+ :alt: subdev-image-processing-full.svg
:align: center
**Figure 4.7. Image processing in subdevs: scaling and composition with multiple sinks and sources**
diff --git a/Documentation/media/uapi/v4l/field-order.rst b/Documentation/media/uapi/v4l/field-order.rst
index e05fb1041363..7a1a0ac8e37a 100644
--- a/Documentation/media/uapi/v4l/field-order.rst
+++ b/Documentation/media/uapi/v4l/field-order.rst
@@ -141,8 +141,8 @@ enum v4l2_field
Field Order, Top Field First Transmitted
========================================
-.. figure:: fieldseq_tb.*
- :alt: fieldseq_tb.pdf / fieldseq_tb.svg
+.. kernel-figure:: fieldseq_tb.svg
+ :alt: fieldseq_tb.svg
:align: center
@@ -151,7 +151,7 @@ Field Order, Top Field First Transmitted
Field Order, Bottom Field First Transmitted
===========================================
-.. figure:: fieldseq_bt.*
- :alt: fieldseq_bt.pdf / fieldseq_bt.svg
+.. kernel-figure:: fieldseq_bt.svg
+ :alt: fieldseq_bt.svg
:align: center
diff --git a/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst b/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst
index 32d0c8743460..a4d64edc6200 100644
--- a/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst
@@ -33,8 +33,8 @@ Layout of macroblocks in memory is presented in the following figure.
.. _nv12mt:
-.. figure:: nv12mt.*
- :alt: nv12mt.pdf / nv12mt.svg
+.. kernel-figure:: nv12mt.svg
+ :alt: nv12mt.svg
:align: center
V4L2_PIX_FMT_NV12MT macroblock Z shape memory layout
@@ -50,8 +50,8 @@ interleaved. Height of the buffer is aligned to 32.
.. _nv12mt_ex:
-.. figure:: nv12mt_example.*
- :alt: nv12mt_example.pdf / nv12mt_example.svg
+.. kernel-figure:: nv12mt_example.svg
+ :alt: nv12mt_example.svg
:align: center
Example V4L2_PIX_FMT_NV12MT memory layout of macroblocks
diff --git a/Documentation/media/uapi/v4l/selection-api-003.rst b/Documentation/media/uapi/v4l/selection-api-003.rst
index 21686f93c38f..04e9384e1be8 100644
--- a/Documentation/media/uapi/v4l/selection-api-003.rst
+++ b/Documentation/media/uapi/v4l/selection-api-003.rst
@@ -7,8 +7,8 @@ Selection targets
.. _sel-targets-capture:
-.. figure:: selection.*
- :alt: selection.pdf / selection.svg
+.. kernel-figure:: selection.svg
+ :alt: selection.svg
:align: center
Cropping and composing targets
diff --git a/Documentation/media/uapi/v4l/subdev-formats.rst b/Documentation/media/uapi/v4l/subdev-formats.rst
index d6152c907b8b..4fbf2e29e653 100644
--- a/Documentation/media/uapi/v4l/subdev-formats.rst
+++ b/Documentation/media/uapi/v4l/subdev-formats.rst
@@ -1514,8 +1514,8 @@ be named ``MEDIA_BUS_FMT_SRGGB10_2X8_PADHI_LE``.
.. _bayer-patterns:
-.. figure:: bayer.*
- :alt: bayer.pdf / bayer.svg
+.. figure:: bayer.svg
+ :alt: bayer.svg
:align: center
**Figure 4.8 Bayer Patterns**
^ permalink raw reply related [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 16:29 ` Mauro Carvalho Chehab
@ 2017-03-02 18:16 ` Markus Heiser
2017-03-02 18:20 ` Jonathan Corbet
2017-03-02 19:09 ` Mauro Carvalho Chehab
0 siblings, 2 replies; 56+ messages in thread
From: Markus Heiser @ 2017-03-02 18:16 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Jonathan Corbet, Daniel Vetter, Linux Doc Mailing List, dri-devel,
Laurent Pinchart, Daniel Vetter
> Am 02.03.2017 um 17:29 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
>
> Em Thu, 2 Mar 2017 17:13:25 +0100
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
>
>>> Am 02.03.2017 um 16:49 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
>>>
>>>>> You can test it with virtualenv https://virtualenv.pypa.io/en/stable/userguide/
>>>>>
>>>>> In short:
>>>>>
>>>>> $ cd kernel-src
>>>>> $ virtualenv myenv
>>>>> $ source myenv/bin/activate
>>>>> $ pip install 'Sphinx==1.3.1'
>>>>> $ make ....
>>>>
>>>> Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
>>>> virtualenv is broken:
>>>>
>>>> writing output... [ 16%] media/intro
>>>> Exception occurred:
>>>> File "/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
>>>> assert not self.context, 'len(context) = %s' % len(self.context)
>>>> AssertionError: len(context) = 1
>>>> The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you want to report the issue to the developers.
>>>> Please also report this if it was a user error, so that a better error message can be provided next time.
>>>> A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>>> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
>>>> make[1]: *** [htmldocs] Error 1
>>>> Makefile:1450: recipe for target 'htmldocs' failed
>>>> make: *** [htmldocs] Error 2
>>>>
>>>> Perhaps it is time for us to move minimal requirements to 1.4?
>>>
>>> Hmm... the same happened with 1.4.9 inside virtualenv. It built fine
>>> with 1.5.2 (for htmldocs).
>>>
>>> Thanks,
>>> Mauro
>>>
>>> -
>>>
>>> This is the error log with 1.4:
>>>
>>> # Sphinx version: 1.4.9
>> ....
>>> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py", line 187, in walkabout
>>> visitor.dispatch_departure(self)
>>> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py", line 1895, in dispatch_departure
>>> return method(node)
>>> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
>>> assert not self.context, 'len(context) = %s' % len(self.context)
>>> AssertionError: len(context) = 1
>>
>> I guess this is a error from newer docutils, so we have to fix docutils version also.
>>
>> Sphinx itself specifies "docutils>=0.11"
>>
>> https://github.com/sphinx-doc/sphinx/blob/1.3.1/setup.py#L52
>>
>> So I guess it uses a up to date docutils or the ducutils which are already installed system wide.
>
> The system-wide docutils is this one:
>
> python2-docutils-0.13.1-3.fc25.noarch
> python3-docutils-0.13.1-3.fc25.noarch
Sorry my mistake, in the traceback we see
File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
assert not self.context, 'len(context) = %s' % len(self.context)
... which means: system wide installation does not matter.
> Btw, I tested also with virtualenv-3/pip3 and the same issue happens there.
>
> Manually installing version 0.11 makes it to work again.
Yes, its only about "docutils>=0.11" in Sphinx 1.3 dependencies.
In Sphinx 1.5 the error:
https://github.com/sphinx-doc/sphinx/issues/3212
is fixed:
https://github.com/tk0miya/sphinx/commit/73663f63672f22304810ce6bb9787490ad250127
But this will never be fixed downwards.
All this is about semantic versioning. If you want to promise your
builds, you have to name which versions of dependencies you support.
I guess this is nothing new for kernel developers ;)
The problem is, that PEP440 defines not only ONE version scheme
"""Some hard to read version identifiers are permitted by
this scheme in order to better accommodate the wide range
of versioning practices across existing public and private
Python projects."""
In practice, the python projects use slightly different schemes
which not follow one rule like <main>.<compatible feature>.<patch>
From history packaging in Python is the hell, it becomes better, but
the problem with slightly different version schemes still exist.
How can this info help us? Now we know, that we have to stick Sphinx
and docutils by patch-levels we are willing to support. Or with
your words ....
> Considering that Sphinx require a specific docutils package for it to
> work, perhaps it is time for us to consider to use the virtenv
> enchantments at make docs targets :-p
This is very easy, if we use a requiremts.txt file where we
stick the versions and run the sphinx in this build in a
virtualenv which is build up by this requirements.txt.
https://pip.pypa.io/en/stable/user_guide/#requirements-files
To summarize, I recommend a Makefile.sphinx cmd which does
something like:
virtualenv output/myenv
source output/myenv/bin/activate
pip install -r requirements.txt
sphinx-build ...
I guess this is something we should discuss with Jon, he
is also familiar with it virtualenv.
-- Markus --
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 18:16 ` Markus Heiser
@ 2017-03-02 18:20 ` Jonathan Corbet
2017-03-02 18:47 ` Markus Heiser
2017-03-02 19:09 ` Mauro Carvalho Chehab
1 sibling, 1 reply; 56+ messages in thread
From: Jonathan Corbet @ 2017-03-02 18:20 UTC (permalink / raw)
To: Markus Heiser
Cc: Linux Doc Mailing List, Daniel Vetter, dri-devel,
Mauro Carvalho Chehab, Laurent Pinchart, Daniel Vetter
On Thu, 2 Mar 2017 19:16:47 +0100
Markus Heiser <markus.heiser@darmarit.de> wrote:
> This is very easy, if we use a requiremts.txt file where we
> stick the versions and run the sphinx in this build in a
> virtualenv which is build up by this requirements.txt.
>
> https://pip.pypa.io/en/stable/user_guide/#requirements-files
>
> To summarize, I recommend a Makefile.sphinx cmd which does
> something like:
>
> virtualenv output/myenv
> source output/myenv/bin/activate
> pip install -r requirements.txt
> sphinx-build ...
>
> I guess this is something we should discuss with Jon, he
> is also familiar with it virtualenv.
That would perhaps make the build more reliable, but it would also make
the build dependent on net access to PyPI, and that's not an idea I like a
whole lot. We should be able to do a build without going out on the
network.
I'm kind of pressed for time, but will try to ponder on this more
shortly...
jon
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 18:20 ` Jonathan Corbet
@ 2017-03-02 18:47 ` Markus Heiser
0 siblings, 0 replies; 56+ messages in thread
From: Markus Heiser @ 2017-03-02 18:47 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Linux Doc Mailing List, Daniel Vetter, dri-devel,
Mauro Carvalho Chehab, Laurent Pinchart, Daniel Vetter
> Am 02.03.2017 um 19:20 schrieb Jonathan Corbet <corbet@lwn.net>:
>
> On Thu, 2 Mar 2017 19:16:47 +0100
> Markus Heiser <markus.heiser@darmarit.de> wrote:
>
>> This is very easy, if we use a requiremts.txt file where we
>> stick the versions and run the sphinx in this build in a
>> virtualenv which is build up by this requirements.txt.
>>
>> https://pip.pypa.io/en/stable/user_guide/#requirements-files
>>
>> To summarize, I recommend a Makefile.sphinx cmd which does
>> something like:
>>
>> virtualenv output/myenv
>> source output/myenv/bin/activate
>> pip install -r requirements.txt
>> sphinx-build ...
>>
>> I guess this is something we should discuss with Jon, he
>> is also familiar with it virtualenv.
>
> That would perhaps make the build more reliable, but it would also make
> the build dependent on net access to PyPI, and that's not an idea I like a
> whole lot. We should be able to do a build without going out on the
> network.
Right, there are PROS and CONS. Another point is where to place
the virtualenv. In my example above it is placed in output/ with
I’am not really happy.
> I'm kind of pressed for time, but will try to ponder on this more
> shortly…
My recommendation are mostly from a python developer POV, which is
sometimes diametral to what kernel development needs. You now both
POV in deep so I’am hopeful that you find the right conceptual
answers for those open questions ;)
-- Markus --
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 18:06 ` Mauro Carvalho Chehab
@ 2017-03-02 19:06 ` Markus Heiser
2017-03-02 19:34 ` Mauro Carvalho Chehab
2017-03-02 19:52 ` Daniel Vetter
0 siblings, 2 replies; 56+ messages in thread
From: Markus Heiser @ 2017-03-02 19:06 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Jonathan Corbet, Daniel Vetter, DRI Development,
Linux Doc Mailing List, Laurent Pinchart, Daniel Vetter
Hi Mauro,
> Tested here with the enclosed patch.
great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
> It crashed:
> Exception occurred:
> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> sys.stderr.write(err)
> TypeError: write() argument must be str, not bytes
> The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> Please also report this if it was a user error, so that a better error message can be provided next time.
> A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> make[1]: *** [htmldocs] Error 1
> Makefile:1450: recipe for target 'htmldocs' failed
> make: *** [htmldocs] Error 2
>
> Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
I guess that the dot command writes something to stderr. This is captured
by the extension and printed to stderr ...
+def dot2format(dot_fname, out_fname):
...
+ exit_code = 42
+ with open(out_fname, "w") as out:
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
+ sys.stderr.write(err)
+
+ exit_code = p.returncode
+ out.flush()
+ return bool(exit_code == 0)
> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> sys.stderr.write(err)
> TypeError: write() argument must be str, not bytes
Do we need this stderr output? For a first test, uncomment the
"sys.stderr.write(err)“ in line 222. Or, if we really need the
stderr, try:
- sys.stderr.write(err)
+ sys.stderr.write(str(err))
I this fixes, there is another "sys.stderr.write(err)" in
func svg2pdf(..) which should also fixed ….
+def svg2pdf(svg_fname, pdf_fname):
...
+ cmd = [convert_cmd, svg_fname, pdf_fname]
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
- sys.stderr.write(err)
+ sys.stderr.write(str(err))
+
+ exit_code = p.returncode
+ return bool(exit_code == 0)
Thanks!
-- Markus --
>
>
> Thanks,
> Mauro
>
> --
>
> [PATCH] docs-rst: Don't use explicit Makefile rules to build SVG and
> DOT files
>
> Now that we have an extension to handle images, use it.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
>
> diff --git a/Documentation/media/Makefile b/Documentation/media/Makefile
> index 32663602ff25..5bd52ea1eff0 100644
> --- a/Documentation/media/Makefile
> +++ b/Documentation/media/Makefile
> @@ -1,51 +1,6 @@
> -# Rules to convert DOT and SVG to Sphinx images
> -
> -SRC_DIR=$(srctree)/Documentation/media
> -
> -DOTS = \
> - uapi/v4l/pipeline.dot \
> -
> -IMAGES = \
> - typical_media_device.svg \
> - uapi/dvb/dvbstb.svg \
> - uapi/v4l/bayer.svg \
> - uapi/v4l/constraints.svg \
> - uapi/v4l/crop.svg \
> - uapi/v4l/fieldseq_bt.svg \
> - uapi/v4l/fieldseq_tb.svg \
> - uapi/v4l/nv12mt.svg \
> - uapi/v4l/nv12mt_example.svg \
> - uapi/v4l/pipeline.svg \
> - uapi/v4l/selection.svg \
> - uapi/v4l/subdev-image-processing-full.svg \
> - uapi/v4l/subdev-image-processing-scaling-multi-source.svg \
> - uapi/v4l/subdev-image-processing-crop.svg \
> - uapi/v4l/vbi_525.svg \
> - uapi/v4l/vbi_625.svg \
> - uapi/v4l/vbi_hsync.svg \
> -
> -DOTTGT := $(patsubst %.dot,%.svg,$(DOTS))
> -IMGDOT := $(patsubst %,$(SRC_DIR)/%,$(DOTTGT))
> -
> -IMGTGT := $(patsubst %.svg,%.pdf,$(IMAGES))
> -IMGPDF := $(patsubst %,$(SRC_DIR)/%,$(IMGTGT))
> -
> -cmd = $(echo-cmd) $(cmd_$(1))
> -
> -quiet_cmd_genpdf = GENPDF $2
> - cmd_genpdf = convert $2 $3
> -
> -quiet_cmd_gendot = DOT $2
> - cmd_gendot = dot -Tsvg $2 > $3
> -
> -%.pdf: %.svg
> - @$(call cmd,genpdf,$<,$@)
> -
> -%.svg: %.dot
> - @$(call cmd,gendot,$<,$@)
> -
> # Rules to convert a .h file to inline RST documentation
>
> +SRC_DIR=$(srctree)/Documentation/media
> PARSER = $(srctree)/Documentation/sphinx/parse-headers.pl
> UAPI = $(srctree)/include/uapi/linux
> KAPI = $(srctree)/include/linux
> diff --git a/Documentation/media/intro.rst b/Documentation/media/intro.rst
> index 8f7490c9a8ef..5562fba1d51d 100644
> --- a/Documentation/media/intro.rst
> +++ b/Documentation/media/intro.rst
> @@ -13,8 +13,8 @@ A typical media device hardware is shown at :ref:`typical_media_device`.
>
> .. _typical_media_device:
>
> -.. figure:: typical_media_device.*
> - :alt: typical_media_device.pdf / typical_media_device.svg
> +.. kernel-figure:: typical_media_device.svg
> + :alt: typical_media_device.svg
> :align: center
>
> Typical Media Device
> diff --git a/Documentation/media/uapi/dvb/intro.rst b/Documentation/media/uapi/dvb/intro.rst
> index 2ed5c23102b4..0e76067a5b58 100644
> --- a/Documentation/media/uapi/dvb/intro.rst
> +++ b/Documentation/media/uapi/dvb/intro.rst
> @@ -55,8 +55,8 @@ Overview
>
> .. _stb_components:
>
> -.. figure:: dvbstb.*
> - :alt: dvbstb.pdf / dvbstb.svg
> +.. kernel-figure:: dvbstb.svg
> + :alt: dvbstb.svg
> :align: center
>
> Components of a DVB card/STB
> diff --git a/Documentation/media/uapi/v4l/crop.rst b/Documentation/media/uapi/v4l/crop.rst
> index be58894c9c89..adea29b14135 100644
> --- a/Documentation/media/uapi/v4l/crop.rst
> +++ b/Documentation/media/uapi/v4l/crop.rst
> @@ -53,8 +53,8 @@ Cropping Structures
>
> .. _crop-scale:
>
> -.. figure:: crop.*
> - :alt: crop.pdf / crop.svg
> +.. kernel-figure:: crop.svg
> + :alt: crop.svg
> :align: center
>
> Image Cropping, Insertion and Scaling
> diff --git a/Documentation/media/uapi/v4l/dev-raw-vbi.rst b/Documentation/media/uapi/v4l/dev-raw-vbi.rst
> index baf5f2483927..f761d3a93783 100644
> --- a/Documentation/media/uapi/v4l/dev-raw-vbi.rst
> +++ b/Documentation/media/uapi/v4l/dev-raw-vbi.rst
> @@ -221,8 +221,8 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
>
> .. _vbi-hsync:
>
> -.. figure:: vbi_hsync.*
> - :alt: vbi_hsync.pdf / vbi_hsync.svg
> +.. kernel-figure:: vbi_hsync.svg
> + :alt: vbi_hsync.svg
> :align: center
>
> **Figure 4.1. Line synchronization**
> @@ -230,8 +230,8 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
>
> .. _vbi-525:
>
> -.. figure:: vbi_525.*
> - :alt: vbi_525.pdf / vbi_525.svg
> +.. kernel-figure:: vbi_525.svg
> + :alt: vbi_525.svg
> :align: center
>
> **Figure 4.2. ITU-R 525 line numbering (M/NTSC and M/PAL)**
> @@ -240,8 +240,8 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
>
> .. _vbi-625:
>
> -.. figure:: vbi_625.*
> - :alt: vbi_625.pdf / vbi_625.svg
> +.. kernel-figure:: vbi_625.svg
> + :alt: vbi_625.svg
> :align: center
>
> **Figure 4.3. ITU-R 625 line numbering**
> diff --git a/Documentation/media/uapi/v4l/dev-subdev.rst b/Documentation/media/uapi/v4l/dev-subdev.rst
> index cd2870180208..0a54e4d95013 100644
> --- a/Documentation/media/uapi/v4l/dev-subdev.rst
> +++ b/Documentation/media/uapi/v4l/dev-subdev.rst
> @@ -99,8 +99,8 @@ the video sensor and the host image processing hardware.
>
> .. _pipeline-scaling:
>
> -.. figure:: pipeline.*
> - :alt: pipeline.pdf / pipeline.svg
> +.. kernel-figure:: pipeline.dot
> + :alt: pipeline.dot
> :align: center
>
> Image Format Negotiation on Pipelines
> @@ -404,8 +404,8 @@ selection will refer to the sink pad format dimensions instead.
>
> .. _subdev-image-processing-crop:
>
> -.. figure:: subdev-image-processing-crop.*
> - :alt: subdev-image-processing-crop.pdf / subdev-image-processing-crop.svg
> +.. kernel-figure:: subdev-image-processing-crop.svg
> + :alt: subdev-image-processing-crop.svg
> :align: center
>
> **Figure 4.5. Image processing in subdevs: simple crop example**
> @@ -421,8 +421,8 @@ pad.
>
> .. _subdev-image-processing-scaling-multi-source:
>
> -.. figure:: subdev-image-processing-scaling-multi-source.*
> - :alt: subdev-image-processing-scaling-multi-source.pdf / subdev-image-processing-scaling-multi-source.svg
> +.. kernel-figure:: subdev-image-processing-scaling-multi-source.svg
> + :alt: subdev-image-processing-scaling-multi-source.svg
> :align: center
>
> **Figure 4.6. Image processing in subdevs: scaling with multiple sources**
> @@ -437,8 +437,8 @@ an area at location specified by the source crop rectangle from it.
>
> .. _subdev-image-processing-full:
>
> -.. figure:: subdev-image-processing-full.*
> - :alt: subdev-image-processing-full.pdf / subdev-image-processing-full.svg
> +.. kernel-figure:: subdev-image-processing-full.svg
> + :alt: subdev-image-processing-full.svg
> :align: center
>
> **Figure 4.7. Image processing in subdevs: scaling and composition with multiple sinks and sources**
> diff --git a/Documentation/media/uapi/v4l/field-order.rst b/Documentation/media/uapi/v4l/field-order.rst
> index e05fb1041363..7a1a0ac8e37a 100644
> --- a/Documentation/media/uapi/v4l/field-order.rst
> +++ b/Documentation/media/uapi/v4l/field-order.rst
> @@ -141,8 +141,8 @@ enum v4l2_field
> Field Order, Top Field First Transmitted
> ========================================
>
> -.. figure:: fieldseq_tb.*
> - :alt: fieldseq_tb.pdf / fieldseq_tb.svg
> +.. kernel-figure:: fieldseq_tb.svg
> + :alt: fieldseq_tb.svg
> :align: center
>
>
> @@ -151,7 +151,7 @@ Field Order, Top Field First Transmitted
> Field Order, Bottom Field First Transmitted
> ===========================================
>
> -.. figure:: fieldseq_bt.*
> - :alt: fieldseq_bt.pdf / fieldseq_bt.svg
> +.. kernel-figure:: fieldseq_bt.svg
> + :alt: fieldseq_bt.svg
> :align: center
>
> diff --git a/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst b/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst
> index 32d0c8743460..a4d64edc6200 100644
> --- a/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst
> +++ b/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst
> @@ -33,8 +33,8 @@ Layout of macroblocks in memory is presented in the following figure.
>
> .. _nv12mt:
>
> -.. figure:: nv12mt.*
> - :alt: nv12mt.pdf / nv12mt.svg
> +.. kernel-figure:: nv12mt.svg
> + :alt: nv12mt.svg
> :align: center
>
> V4L2_PIX_FMT_NV12MT macroblock Z shape memory layout
> @@ -50,8 +50,8 @@ interleaved. Height of the buffer is aligned to 32.
>
> .. _nv12mt_ex:
>
> -.. figure:: nv12mt_example.*
> - :alt: nv12mt_example.pdf / nv12mt_example.svg
> +.. kernel-figure:: nv12mt_example.svg
> + :alt: nv12mt_example.svg
> :align: center
>
> Example V4L2_PIX_FMT_NV12MT memory layout of macroblocks
> diff --git a/Documentation/media/uapi/v4l/selection-api-003.rst b/Documentation/media/uapi/v4l/selection-api-003.rst
> index 21686f93c38f..04e9384e1be8 100644
> --- a/Documentation/media/uapi/v4l/selection-api-003.rst
> +++ b/Documentation/media/uapi/v4l/selection-api-003.rst
> @@ -7,8 +7,8 @@ Selection targets
>
> .. _sel-targets-capture:
>
> -.. figure:: selection.*
> - :alt: selection.pdf / selection.svg
> +.. kernel-figure:: selection.svg
> + :alt: selection.svg
> :align: center
>
> Cropping and composing targets
> diff --git a/Documentation/media/uapi/v4l/subdev-formats.rst b/Documentation/media/uapi/v4l/subdev-formats.rst
> index d6152c907b8b..4fbf2e29e653 100644
> --- a/Documentation/media/uapi/v4l/subdev-formats.rst
> +++ b/Documentation/media/uapi/v4l/subdev-formats.rst
> @@ -1514,8 +1514,8 @@ be named ``MEDIA_BUS_FMT_SRGGB10_2X8_PADHI_LE``.
>
> .. _bayer-patterns:
>
> -.. figure:: bayer.*
> - :alt: bayer.pdf / bayer.svg
> +.. figure:: bayer.svg
> + :alt: bayer.svg
> :align: center
>
> **Figure 4.8 Bayer Patterns**
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 18:16 ` Markus Heiser
2017-03-02 18:20 ` Jonathan Corbet
@ 2017-03-02 19:09 ` Mauro Carvalho Chehab
2017-03-02 20:16 ` Markus Heiser
1 sibling, 1 reply; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 19:09 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, Laurent Pinchart, Daniel Vetter, dri-devel,
Linux Doc Mailing List, Jonathan Corbet, Daniel Vetter
Em Thu, 2 Mar 2017 19:16:47 +0100
Markus Heiser <markus.heiser@darmarit.de> escreveu:
> > Am 02.03.2017 um 17:29 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> >
> > Em Thu, 2 Mar 2017 17:13:25 +0100
> > Markus Heiser <markus.heiser@darmarit.de> escreveu:
> >
> >>> Am 02.03.2017 um 16:49 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> >>>
> >>>>> You can test it with virtualenv https://virtualenv.pypa.io/en/stable/userguide/
> >>>>>
> >>>>> In short:
> >>>>>
> >>>>> $ cd kernel-src
> >>>>> $ virtualenv myenv
> >>>>> $ source myenv/bin/activate
> >>>>> $ pip install 'Sphinx==1.3.1'
> >>>>> $ make ....
> >>>>
> >>>> Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
> >>>> virtualenv is broken:
> >>>>
> >>>> writing output... [ 16%] media/intro
> >>>> Exception occurred:
> >>>> File "/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
> >>>> assert not self.context, 'len(context) = %s' % len(self.context)
> >>>> AssertionError: len(context) = 1
> >>>> The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you want to report the issue to the developers.
> >>>> Please also report this if it was a user error, so that a better error message can be provided next time.
> >>>> A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> >>>> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> >>>> make[1]: *** [htmldocs] Error 1
> >>>> Makefile:1450: recipe for target 'htmldocs' failed
> >>>> make: *** [htmldocs] Error 2
> >>>>
> >>>> Perhaps it is time for us to move minimal requirements to 1.4?
> >>>
> >>> Hmm... the same happened with 1.4.9 inside virtualenv. It built fine
> >>> with 1.5.2 (for htmldocs).
> >>>
> >>> Thanks,
> >>> Mauro
> >>>
> >>> -
> >>>
> >>> This is the error log with 1.4:
> >>>
> >>> # Sphinx version: 1.4.9
> >> ....
> >>> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py", line 187, in walkabout
> >>> visitor.dispatch_departure(self)
> >>> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py", line 1895, in dispatch_departure
> >>> return method(node)
> >>> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
> >>> assert not self.context, 'len(context) = %s' % len(self.context)
> >>> AssertionError: len(context) = 1
> >>
> >> I guess this is a error from newer docutils, so we have to fix docutils version also.
> >>
> >> Sphinx itself specifies "docutils>=0.11"
> >>
> >> https://github.com/sphinx-doc/sphinx/blob/1.3.1/setup.py#L52
> >>
> >> So I guess it uses a up to date docutils or the ducutils which are already installed system wide.
> >
> > The system-wide docutils is this one:
> >
> > python2-docutils-0.13.1-3.fc25.noarch
> > python3-docutils-0.13.1-3.fc25.noarch
>
> Sorry my mistake, in the traceback we see
>
> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
> assert not self.context, 'len(context) = %s' % len(self.context)
>
> ... which means: system wide installation does not matter.
>
> > Btw, I tested also with virtualenv-3/pip3 and the same issue happens there.
> >
> > Manually installing version 0.11 makes it to work again.
>
> Yes, its only about "docutils>=0.11" in Sphinx 1.3 dependencies.
> In Sphinx 1.5 the error:
>
> https://github.com/sphinx-doc/sphinx/issues/3212
>
> is fixed:
>
> https://github.com/tk0miya/sphinx/commit/73663f63672f22304810ce6bb9787490ad250127
>
> But this will never be fixed downwards.
Crap. This kind of patch should be backported to Sphinx 1.3/1.4,
or Python's PIP repository should be fixed to require docutils
version to be either 0.11 or 0.12, if one installs version 1.3
or 1.4 of Sphinx.
The way it is, PIP repository is broken!
> All this is about semantic versioning. If you want to promise your
> builds, you have to name which versions of dependencies you support.
> I guess this is nothing new for kernel developers ;)
No. At the Kernel, we do everything possible to not break APIs.
If this were not the case, you wouldn't be able to run Sphinx 1.5
with legacy Kernel versions ;-)
> The problem is, that PEP440 defines not only ONE version scheme
>
> """Some hard to read version identifiers are permitted by
> this scheme in order to better accommodate the wide range
> of versioning practices across existing public and private
> Python projects."""
>
> In practice, the python projects use slightly different schemes
> which not follow one rule like <main>.<compatible feature>.<patch>
> From history packaging in Python is the hell, it becomes better, but
> the problem with slightly different version schemes still exist.
IMHO, the way Python and python libraries break compatibility is crazy.
Good packaging sense says that, if APIs can break on every single new
release (with seems to be the case of docutils), then a package
depending on such bad-developed library should require the exact
version(s) it is known to work.
When we're discussing about the docs toolchain, I mentioned that I
was afraid that the Python development model would cause this sort
of issues. Unfortunately, it seems that my concerns were pertinent :-(
> How can this info help us? Now we know, that we have to stick Sphinx
> and docutils by patch-levels we are willing to support. Or with
> your words ....
>
> > Considering that Sphinx require a specific docutils package for it to
> > work, perhaps it is time for us to consider to use the virtenv
> > enchantments at make docs targets :-p
>
>
> This is very easy, if we use a requiremts.txt file where we
> stick the versions and run the sphinx in this build in a
> virtualenv which is build up by this requirements.txt.
>
> https://pip.pypa.io/en/stable/user_guide/#requirements-files
>
> To summarize, I recommend a Makefile.sphinx cmd which does
> something like:
>
> virtualenv output/myenv
> source output/myenv/bin/activate
> pip install -r requirements.txt
> sphinx-build ...
>
> I guess this is something we should discuss with Jon, he
> is also familiar with it virtualenv.
Yeah, making kernel build to depend on network can be a problem.
Maybe one way would be to have a sort of "prepare" script that
would be network-dependent, and will install whatever needed to
build the docs. If called, make *docs will use the virtualenv.
Otherwise, it would print a warning message saying that the doc
build is not reliable, but would try to use whatever installed
on the machine.
Thanks,
Mauro
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 19:06 ` Markus Heiser
@ 2017-03-02 19:34 ` Mauro Carvalho Chehab
2017-03-02 19:56 ` Mauro Carvalho Chehab
2017-03-02 21:16 ` Markus Heiser
2017-03-02 19:52 ` Daniel Vetter
1 sibling, 2 replies; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 19:34 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, Linux Doc Mailing List, DRI Development,
Jonathan Corbet, Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 20:06:39 +0100
Markus Heiser <markus.heiser@darmarit.de> escreveu:
> Hi Mauro,
>
> > Tested here with the enclosed patch.
>
> great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
>
> > It crashed:
> > Exception occurred:
> > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > sys.stderr.write(err)
> > TypeError: write() argument must be str, not bytes
> > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> > Please also report this if it was a user error, so that a better error message can be provided next time.
> > A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > make[1]: *** [htmldocs] Error 1
> > Makefile:1450: recipe for target 'htmldocs' failed
> > make: *** [htmldocs] Error 2
> >
> > Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
>
> I guess that the dot command writes something to stderr. This is captured
> by the extension and printed to stderr ...
>
> +def dot2format(dot_fname, out_fname):
> ...
> + exit_code = 42
> + with open(out_fname, "w") as out:
> + p = subprocess.Popen(
> + cmd, stdout = out, stderr = subprocess.PIPE )
> + nil, err = p.communicate()
> +
> + sys.stderr.write(err)
> +
> + exit_code = p.returncode
> + out.flush()
> + return bool(exit_code == 0)
>
> > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > sys.stderr.write(err)
> > TypeError: write() argument must be str, not bytes
>
> Do we need this stderr output? For a first test, uncomment the
> "sys.stderr.write(err)“ in line 222. Or, if we really need the
> stderr, try:
>
> - sys.stderr.write(err)
> + sys.stderr.write(str(err))
Yes, this fixed. I actually did:
- sys.stderr.write(err)
+ sys.stderr.write(str(err))
+ sys.stderr.write("\n")
It is now printing:
b''
I added the \n print to avoid it to be mixed with the "writing output"
prints.
No idea how to make sense from it - but clearly, the error report
logic require some care ;-)
Thanks,
Mauro
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 16:01 ` Mauro Carvalho Chehab
@ 2017-03-02 19:49 ` Daniel Vetter
0 siblings, 0 replies; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 19:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Markus Heiser, Linux Doc Mailing List, Jonathan Corbet, dri-devel,
Laurent Pinchart, Daniel Vetter
On Thu, Mar 02, 2017 at 01:01:36PM -0300, Mauro Carvalho Chehab wrote:
> Em Thu, 2 Mar 2017 16:53:04 +0100
> Daniel Vetter <daniel@ffwll.ch> escreveu:
>
> > On Thu, Mar 2, 2017 at 4:22 PM, Jonathan Corbet <corbet@lwn.net> wrote:
> > > On Thu, 2 Mar 2017 16:11:08 +0100
> > > Daniel Vetter <daniel@ffwll.ch> wrote:
> > >
> > >> I'll give it a shot at implementing it, but I can't (easily at least) test
> > >> on sphinx 1.3.
> > >
> > > Virtualenv makes that kind of thing pretty easy to do. Maybe we should
> > > add a script to create and populate a suitable venv for this kind of
> > > thing.
> >
> > Laurent quickly checked v5 of this patch, looks all good now.
> >
> > > Meanwhile, I've been seeing only parts 1 and 2 of what's clearly a bigger
> > > series. Do you want me to carry just these two?
> >
> > I submitted the entire series to both linux-doc and dri-devel.
>
> Sure you sent to linux-doc? I'm not seeing it there.
>
> I'm not seeing it at the mirrors neither:
> https://marc.info/?l=linux-doc&r=1&b=201703&w=2
> https://www.spinics.net/lists/linux-doc/maillist.html
First one I spotted:
https://marc.info/?l=linux-doc&m=148848308304746&w=2
So maybe just mail server hiccup that's recovering now?
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 19:06 ` Markus Heiser
2017-03-02 19:34 ` Mauro Carvalho Chehab
@ 2017-03-02 19:52 ` Daniel Vetter
2017-03-02 20:04 ` Mauro Carvalho Chehab
1 sibling, 1 reply; 56+ messages in thread
From: Daniel Vetter @ 2017-03-02 19:52 UTC (permalink / raw)
To: Markus Heiser
Cc: Jonathan Corbet, Daniel Vetter, Linux Doc Mailing List,
Mauro Carvalho Chehab, DRI Development, Daniel Vetter,
Laurent Pinchart
On Thu, Mar 02, 2017 at 08:06:39PM +0100, Markus Heiser wrote:
> Hi Mauro,
>
> > Tested here with the enclosed patch.
>
> great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
>
> > It crashed:
> > Exception occurred:
> > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > sys.stderr.write(err)
> > TypeError: write() argument must be str, not bytes
> > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> > Please also report this if it was a user error, so that a better error message can be provided next time.
> > A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > make[1]: *** [htmldocs] Error 1
> > Makefile:1450: recipe for target 'htmldocs' failed
> > make: *** [htmldocs] Error 2
> >
> > Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
>
> I guess that the dot command writes something to stderr. This is captured
> by the extension and printed to stderr ...
>
> +def dot2format(dot_fname, out_fname):
> ...
> + exit_code = 42
> + with open(out_fname, "w") as out:
> + p = subprocess.Popen(
> + cmd, stdout = out, stderr = subprocess.PIPE )
> + nil, err = p.communicate()
> +
> + sys.stderr.write(err)
> +
> + exit_code = p.returncode
> + out.flush()
> + return bool(exit_code == 0)
>
> > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > sys.stderr.write(err)
> > TypeError: write() argument must be str, not bytes
>
> Do we need this stderr output? For a first test, uncomment the
> "sys.stderr.write(err)“ in line 222. Or, if we really need the
> stderr, try:
>
> - sys.stderr.write(err)
> + sys.stderr.write(str(err))
>
> I this fixes, there is another "sys.stderr.write(err)" in
> func svg2pdf(..) which should also fixed ….
>
> +def svg2pdf(svg_fname, pdf_fname):
> ...
> + cmd = [convert_cmd, svg_fname, pdf_fname]
> + p = subprocess.Popen(
> + cmd, stdout = out, stderr = subprocess.PIPE )
> + nil, err = p.communicate()
> +
> - sys.stderr.write(err)
> + sys.stderr.write(str(err))
> +
> + exit_code = p.returncode
> + return bool(exit_code == 0)
Yes, I very much want stderr to be forward. Without that you don't see
error output from dot or convert, and that makes it impossible to debug
anything. If I want a direct forwarding of the bytes, how should I do this
in python? Capturing stderr and then re-dumping it is kinda silly ...
Note that I copied this pattern from the kernel-doc extension, seems to
have worked there.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 19:34 ` Mauro Carvalho Chehab
@ 2017-03-02 19:56 ` Mauro Carvalho Chehab
2017-03-02 21:16 ` Markus Heiser
1 sibling, 0 replies; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 19:56 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, Linux Doc Mailing List, DRI Development,
Jonathan Corbet, Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 16:34:22 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
> Em Thu, 2 Mar 2017 20:06:39 +0100
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
>
> > Hi Mauro,
> >
> > > Tested here with the enclosed patch.
> >
> > great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
> >
> > > It crashed:
> > > Exception occurred:
> > > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > sys.stderr.write(err)
> > > TypeError: write() argument must be str, not bytes
> > > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> > > Please also report this if it was a user error, so that a better error message can be provided next time.
> > > A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> > > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > > make[1]: *** [htmldocs] Error 1
> > > Makefile:1450: recipe for target 'htmldocs' failed
> > > make: *** [htmldocs] Error 2
> > >
> > > Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
> >
> > I guess that the dot command writes something to stderr. This is captured
> > by the extension and printed to stderr ...
> >
> > +def dot2format(dot_fname, out_fname):
> > ...
> > + exit_code = 42
> > + with open(out_fname, "w") as out:
> > + p = subprocess.Popen(
> > + cmd, stdout = out, stderr = subprocess.PIPE )
> > + nil, err = p.communicate()
> > +
> > + sys.stderr.write(err)
> > +
> > + exit_code = p.returncode
> > + out.flush()
> > + return bool(exit_code == 0)
> >
> > > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > sys.stderr.write(err)
> > > TypeError: write() argument must be str, not bytes
> >
> > Do we need this stderr output? For a first test, uncomment the
> > "sys.stderr.write(err)“ in line 222. Or, if we really need the
> > stderr, try:
> >
> > - sys.stderr.write(err)
> > + sys.stderr.write(str(err))
>
> Yes, this fixed. I actually did:
>
> - sys.stderr.write(err)
> + sys.stderr.write(str(err))
> + sys.stderr.write("\n")
>
> It is now printing:
> b''
>
> I added the \n print to avoid it to be mixed with the "writing output"
> prints.
>
> No idea how to make sense from it - but clearly, the error report
> logic require some care ;-)
I'm not a Python programmer, but googling about the right syntax for
p.communicate(), I suspect that the fix should be similar to this code
snippet below.
Without the if, this code:
sys.stderr.write("Error:" + repr(p.communicate()[0]) + "\n")
prints:
Error:None
So, I guess the code below is ok.
Markus, please check.
Daniel, Feel free to merge it with the original patch if OK.
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 32eab0f4cfba..b154c5f17752 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -217,11 +217,12 @@ def dot2format(dot_fname, out_fname):
with open(out_fname, "w") as out:
p = subprocess.Popen(
cmd, stdout = out, stderr = subprocess.PIPE )
- nil, err = p.communicate()
-
- sys.stderr.write(err)
exit_code = p.returncode
+
+ if exit_code == 0:
+ sys.stderr.write("Error:" + repr(p.communicate()[0]) + "\n")
+
out.flush()
return bool(exit_code == 0)
@@ -239,11 +240,12 @@ def svg2pdf(svg_fname, pdf_fname):
cmd = [convert_cmd, svg_fname, pdf_fname]
p = subprocess.Popen(
cmd, stdout = out, stderr = subprocess.PIPE )
- nil, err = p.communicate()
-
- sys.stderr.write(err)
exit_code = p.returncode
+
+ if exit_code == 0:
+ sys.stderr.write("Error:" + repr(p.communicate()[0]) + "\n")
+
return bool(exit_code == 0)
^ permalink raw reply related [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 19:52 ` Daniel Vetter
@ 2017-03-02 20:04 ` Mauro Carvalho Chehab
2017-03-02 20:54 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 20:04 UTC (permalink / raw)
To: Daniel Vetter
Cc: Markus Heiser, Daniel Vetter, Linux Doc Mailing List,
DRI Development, Jonathan Corbet, Jani Nikula, Laurent Pinchart,
Daniel Vetter
Em Thu, 2 Mar 2017 20:52:59 +0100
Daniel Vetter <daniel@ffwll.ch> escreveu:
> On Thu, Mar 02, 2017 at 08:06:39PM +0100, Markus Heiser wrote:
> > Hi Mauro,
> >
> > > Tested here with the enclosed patch.
> >
> > great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
> >
> > > It crashed:
> > > Exception occurred:
> > > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > sys.stderr.write(err)
> > > TypeError: write() argument must be str, not bytes
> > > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> > > Please also report this if it was a user error, so that a better error message can be provided next time.
> > > A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> > > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > > make[1]: *** [htmldocs] Error 1
> > > Makefile:1450: recipe for target 'htmldocs' failed
> > > make: *** [htmldocs] Error 2
> > >
> > > Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
> >
> > I guess that the dot command writes something to stderr. This is captured
> > by the extension and printed to stderr ...
> >
> > +def dot2format(dot_fname, out_fname):
> > ...
> > + exit_code = 42
> > + with open(out_fname, "w") as out:
> > + p = subprocess.Popen(
> > + cmd, stdout = out, stderr = subprocess.PIPE )
> > + nil, err = p.communicate()
> > +
> > + sys.stderr.write(err)
> > +
> > + exit_code = p.returncode
> > + out.flush()
> > + return bool(exit_code == 0)
> >
> > > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > sys.stderr.write(err)
> > > TypeError: write() argument must be str, not bytes
> >
> > Do we need this stderr output? For a first test, uncomment the
> > "sys.stderr.write(err)“ in line 222. Or, if we really need the
> > stderr, try:
> >
> > - sys.stderr.write(err)
> > + sys.stderr.write(str(err))
> >
> > I this fixes, there is another "sys.stderr.write(err)" in
> > func svg2pdf(..) which should also fixed ….
> >
> > +def svg2pdf(svg_fname, pdf_fname):
> > ...
> > + cmd = [convert_cmd, svg_fname, pdf_fname]
> > + p = subprocess.Popen(
> > + cmd, stdout = out, stderr = subprocess.PIPE )
> > + nil, err = p.communicate()
> > +
> > - sys.stderr.write(err)
> > + sys.stderr.write(str(err))
> > +
> > + exit_code = p.returncode
> > + return bool(exit_code == 0)
>
> Yes, I very much want stderr to be forward.
Yes, error report is required.
> Without that you don't see
> error output from dot or convert, and that makes it impossible to debug
> anything. If I want a direct forwarding of the bytes, how should I do this
> in python? Capturing stderr and then re-dumping it is kinda silly ...
Markus or some other Python programmer may help us with that.
> Note that I copied this pattern from the kernel-doc extension, seems to
> have worked there.
Maybe it is broken there too then, or this is another python API
that changed over time. Here, I'm testing with:
python3-3.5.2-4.fc25.x86_64
>From here:
https://docs.python.org/2/library/subprocess.html
communicate returns a tuple.
I used repr(p.communicate()[0], on the code snippet I sent,
as I copied from an example that I found at:
https://stackoverflow.com/questions/33295284/python-subprocess-popen-write-to-stderr
Thanks,
Mauro
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 19:09 ` Mauro Carvalho Chehab
@ 2017-03-02 20:16 ` Markus Heiser
2017-03-02 20:28 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 56+ messages in thread
From: Markus Heiser @ 2017-03-02 20:16 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Jonathan Corbet, Daniel Vetter, Linux Doc Mailing List, dri-devel,
Laurent Pinchart, Daniel Vetter
> Am 02.03.2017 um 20:09 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
...
>>
>> Yes, its only about "docutils>=0.11" in Sphinx 1.3 dependencies.
>> In Sphinx 1.5 the error:
>>
>> https://github.com/sphinx-doc/sphinx/issues/3212
>>
>> is fixed:
>>
>> https://github.com/tk0miya/sphinx/commit/73663f63672f22304810ce6bb9787490ad250127
>>
>> But this will never be fixed downwards.
>
> Crap. This kind of patch should be backported to Sphinx 1.3/1.4,
> or Python's PIP repository should be fixed to require docutils
> version to be either 0.11 or 0.12, if one installs version 1.3
> or 1.4 of Sphinx.
>
> The way it is, PIP repository is broken!
I leaved a comment at sphinx-doc project:
https://github.com/sphinx-doc/sphinx/issues/3212#issuecomment-283756374
>> All this is about semantic versioning. If you want to promise your
>> builds, you have to name which versions of dependencies you support.
>> I guess this is nothing new for kernel developers ;)
>
> No. At the Kernel, we do everything possible to not break APIs.
Sorry I was not precise. I was talking about third tools dependencies
and that this is well handled by the kernel ;)
> If this were not the case, you wouldn't be able to run Sphinx 1.5
> with legacy Kernel versions ;-)
This is what I mean ;)
>
>> The problem is, that PEP440 defines not only ONE version scheme
>>
>> """Some hard to read version identifiers are permitted by
>> this scheme in order to better accommodate the wide range
>> of versioning practices across existing public and private
>> Python projects."""
>>
>> In practice, the python projects use slightly different schemes
>> which not follow one rule like <main>.<compatible feature>.<patch>
>> From history packaging in Python is the hell, it becomes better, but
>> the problem with slightly different version schemes still exist.
>
> IMHO, the way Python and python libraries break compatibility is crazy.
>
> Good packaging sense says that, if APIs can break on every single new
> release (with seems to be the case of docutils), then a package
> depending on such bad-developed library should require the exact
> version(s) it is known to work.
I can only repeat myself, the main problem is, that PEP440 allows
multiple version schemes. Instead of one scheme
<main>.<compatible feature>.<patch>
Every project use a slightly different scheme and others
do not care about any scheme.
> When we're discussing about the docs toolchain, I mentioned that I
> was afraid that the Python development model would cause this sort
> of issues. Unfortunately, it seems that my concerns were pertinent :-(
Not really ;) .. you are tend to mix at least three parts
1. Python
2. Python packaging
3. Sphinx developers who do not stick there depencies
But you are right, when you say that in all parts some confusion
prevail. E.g.
1. Python 2 to 3 movement has been done reckless
--> in the meantime we have https://pythonhosted.org/six/
2. Python packaging is a mess (setup-tools, distutils, pip, easy_install ..)
--> in the meantime we have PyPA who brings us more structure (https://www.pypa.io/en/latest)
3. Developer using Python
--> we all have a learn curve and making errors all days long.
But this should not stop us from continue :)
>> I guess this is something we should discuss with Jon, he
>> is also familiar with it virtualenv.
>
> Yeah, making kernel build to depend on network can be a problem.
>
> Maybe one way would be to have a sort of "prepare" script that
> would be network-dependent, and will install whatever needed to
> build the docs. If called, make *docs will use the virtualenv.
> Otherwise, it would print a warning message saying that the doc
> build is not reliable, but would try to use whatever installed
> on the machine.
Could be a workaround. May Jon has continuative ideas, nothing
we have to solve today ... give Jon some time.
-- Markus --
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 20:16 ` Markus Heiser
@ 2017-03-02 20:28 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 20:28 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, Laurent Pinchart, Daniel Vetter, dri-devel,
Linux Doc Mailing List, Jonathan Corbet, Daniel Vetter
Em Thu, 2 Mar 2017 21:16:05 +0100
Markus Heiser <markus.heiser@darmarit.de> escreveu:
> > Am 02.03.2017 um 20:09 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> I leaved a comment at sphinx-doc project:
>
> https://github.com/sphinx-doc/sphinx/issues/3212#issuecomment-283756374
Thanks!
> > Maybe one way would be to have a sort of "prepare" script that
> > would be network-dependent, and will install whatever needed to
> > build the docs. If called, make *docs will use the virtualenv.
> > Otherwise, it would print a warning message saying that the doc
> > build is not reliable, but would try to use whatever installed
> > on the machine.
>
> Could be a workaround. May Jon has continuative ideas, nothing
> we have to solve today ... give Jon some time.
Sure.
Thanks,
Mauro
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 20:04 ` Mauro Carvalho Chehab
@ 2017-03-02 20:54 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 20:54 UTC (permalink / raw)
To: Daniel Vetter
Cc: Markus Heiser, Daniel Vetter, Linux Doc Mailing List,
DRI Development, Jonathan Corbet, Jani Nikula, Laurent Pinchart,
Daniel Vetter
Em Thu, 2 Mar 2017 17:04:01 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
> Em Thu, 2 Mar 2017 20:52:59 +0100
> Daniel Vetter <daniel@ffwll.ch> escreveu:
>
> > On Thu, Mar 02, 2017 at 08:06:39PM +0100, Markus Heiser wrote:
> > > Hi Mauro,
> > >
> > > > Tested here with the enclosed patch.
> > >
> > > great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
> > >
> > > > It crashed:
> > > > Exception occurred:
> > > > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > > sys.stderr.write(err)
> > > > TypeError: write() argument must be str, not bytes
> > > > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> > > > Please also report this if it was a user error, so that a better error message can be provided next time.
> > > > A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> > > > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > > > make[1]: *** [htmldocs] Error 1
> > > > Makefile:1450: recipe for target 'htmldocs' failed
> > > > make: *** [htmldocs] Error 2
> > > >
> > > > Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
> > >
> > > I guess that the dot command writes something to stderr. This is captured
> > > by the extension and printed to stderr ...
> > >
> > > +def dot2format(dot_fname, out_fname):
> > > ...
> > > + exit_code = 42
> > > + with open(out_fname, "w") as out:
> > > + p = subprocess.Popen(
> > > + cmd, stdout = out, stderr = subprocess.PIPE )
> > > + nil, err = p.communicate()
> > > +
> > > + sys.stderr.write(err)
> > > +
> > > + exit_code = p.returncode
> > > + out.flush()
> > > + return bool(exit_code == 0)
> > >
> > > > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > > sys.stderr.write(err)
> > > > TypeError: write() argument must be str, not bytes
> > >
> > > Do we need this stderr output? For a first test, uncomment the
> > > "sys.stderr.write(err)“ in line 222. Or, if we really need the
> > > stderr, try:
> > >
> > > - sys.stderr.write(err)
> > > + sys.stderr.write(str(err))
> > >
> > > I this fixes, there is another "sys.stderr.write(err)" in
> > > func svg2pdf(..) which should also fixed ….
> > >
> > > +def svg2pdf(svg_fname, pdf_fname):
> > > ...
> > > + cmd = [convert_cmd, svg_fname, pdf_fname]
> > > + p = subprocess.Popen(
> > > + cmd, stdout = out, stderr = subprocess.PIPE )
> > > + nil, err = p.communicate()
> > > +
> > > - sys.stderr.write(err)
> > > + sys.stderr.write(str(err))
> > > +
> > > + exit_code = p.returncode
> > > + return bool(exit_code == 0)
> >
> > Yes, I very much want stderr to be forward.
>
> Yes, error report is required.
>
> > Without that you don't see
> > error output from dot or convert, and that makes it impossible to debug
> > anything. If I want a direct forwarding of the bytes, how should I do this
> > in python? Capturing stderr and then re-dumping it is kinda silly ...
>
> Markus or some other Python programmer may help us with that.
>
> > Note that I copied this pattern from the kernel-doc extension, seems to
> > have worked there.
>
> Maybe it is broken there too then, or this is another python API
> that changed over time. Here, I'm testing with:
> python3-3.5.2-4.fc25.x86_64
>
> From here:
> https://docs.python.org/2/library/subprocess.html
>
> communicate returns a tuple.
>
> I used repr(p.communicate()[0], on the code snippet I sent,
> as I copied from an example that I found at:
> https://stackoverflow.com/questions/33295284/python-subprocess-popen-write-to-stderr
>
> Thanks,
> Mauro
I suspect that the actual fixup would be something like:
commit ddf93ae81af10bb43caa651b9acd355f1d74cebe
Author: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Date: Thu Mar 2 17:11:47 2017 -0300
kfigure.py fixups
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 32eab0f4cfba..19389cb34d6f 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -217,11 +217,13 @@ def dot2format(dot_fname, out_fname):
with open(out_fname, "w") as out:
p = subprocess.Popen(
cmd, stdout = out, stderr = subprocess.PIPE )
- nil, err = p.communicate()
-
- sys.stderr.write(err)
+ err = p.communicate()
exit_code = p.returncode
+
+ if exit_code == 0:
+ sys.stderr.write("Error:" + repr(err[0]) + "\n")
+
out.flush()
return bool(exit_code == 0)
@@ -239,11 +241,13 @@ def svg2pdf(svg_fname, pdf_fname):
cmd = [convert_cmd, svg_fname, pdf_fname]
p = subprocess.Popen(
cmd, stdout = out, stderr = subprocess.PIPE )
- nil, err = p.communicate()
-
- sys.stderr.write(err)
+ err = p.communicate()
exit_code = p.returncode
+
+ if exit_code == 0:
+ sys.stderr.write("Error:" + repr(err[0]) + "\n")
+
return bool(exit_code == 0)
Thanks,
Mauro
^ permalink raw reply related [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 19:34 ` Mauro Carvalho Chehab
2017-03-02 19:56 ` Mauro Carvalho Chehab
@ 2017-03-02 21:16 ` Markus Heiser
2017-03-02 21:22 ` Mauro Carvalho Chehab
1 sibling, 1 reply; 56+ messages in thread
From: Markus Heiser @ 2017-03-02 21:16 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Jonathan Corbet, Daniel Vetter, DRI Development,
Linux Doc Mailing List, Laurent Pinchart, Daniel Vetter
> Am 02.03.2017 um 20:34 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
>
> Em Thu, 2 Mar 2017 20:06:39 +0100
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
>
>> Hi Mauro,
>>
>>> Tested here with the enclosed patch.
>>
>> great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
>>
>>> It crashed:
>>> Exception occurred:
>>> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
>>> sys.stderr.write(err)
>>> TypeError: write() argument must be str, not bytes
>>> The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
>>> Please also report this if it was a user error, so that a better error message can be provided next time.
>>> A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
>>> make[1]: *** [htmldocs] Error 1
>>> Makefile:1450: recipe for target 'htmldocs' failed
>>> make: *** [htmldocs] Error 2
>>>
>>> Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
>>
>> I guess that the dot command writes something to stderr. This is captured
>> by the extension and printed to stderr ...
>>
>> +def dot2format(dot_fname, out_fname):
>> ...
>> + exit_code = 42
>> + with open(out_fname, "w") as out:
>> + p = subprocess.Popen(
>> + cmd, stdout = out, stderr = subprocess.PIPE )
>> + nil, err = p.communicate()
>> +
>> + sys.stderr.write(err)
>> +
>> + exit_code = p.returncode
>> + out.flush()
>> + return bool(exit_code == 0)
>>
>>> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
>>> sys.stderr.write(err)
>>> TypeError: write() argument must be str, not bytes
>>
>> Do we need this stderr output? For a first test, uncomment the
>> "sys.stderr.write(err)“ in line 222. Or, if we really need the
>> stderr, try:
>>
>> - sys.stderr.write(err)
>> + sys.stderr.write(str(err))
>
> Yes, this fixed. I actually did:
>
> - sys.stderr.write(err)
> + sys.stderr.write(str(err))
> + sys.stderr.write("\n")
>
> It is now printing:
> b''
>
> I added the \n print to avoid it to be mixed with the "writing output"
> prints.
> No idea how to make sense from it - but clearly, the error report
> logic require some care ;-)
Aargh, I’am a idiot ... I guess 'sys.stderr.write(err)‘ is a artefact
of my development, simply drop it and the subprocess.PIPE of stderr
also.
+ with open(out_fname, "w") as out:
+ p = subprocess.Popen(
- cmd, stdout = out, stderr = subprocess.PIPE )
+ cmd, stdout = out)
+ nil, err = p.communicate()
-
- sys.stderr.write(err)
-
+ exit_code = p.returncode
+ out.flush()
+ return bool(exit_code == 0)
I can’t test it ATM, but without redirect stderr, the stderr
of the parent process is inherited.
https://docs.python.org/3.6/library/subprocess.html#popen-constructor
The Popen.communicate() always returns a tuple (stdout_data, stderr_data)
with above the tuple is always (None, None).
https://docs.python.org/3.6/library/subprocess.html#subprocess.Popen.communicate
"""to get anything other than None in the result tuple,
you need to give stdout=PIPE and/or stderr=PIPE too."""
Sorry, that I made all this mistakes, but „here“ I have only mail
and web, no dev-env and I miss my emacs ;)
If the suggestion above does not work, I have to investigate
more time next weekend.
-- Markus --
>
> Thanks,
> Mauro
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 21:16 ` Markus Heiser
@ 2017-03-02 21:22 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 21:22 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, Linux Doc Mailing List, DRI Development,
Jonathan Corbet, Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 22:16:49 +0100
Markus Heiser <markus.heiser@darmarit.de> escreveu:
> > Am 02.03.2017 um 20:34 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> >
> > Em Thu, 2 Mar 2017 20:06:39 +0100
> > Markus Heiser <markus.heiser@darmarit.de> escreveu:
> >
> >> Hi Mauro,
> >>
> >>> Tested here with the enclosed patch.
> >>
> >> great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
> >>
> >>> It crashed:
> >>> Exception occurred:
> >>> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> >>> sys.stderr.write(err)
> >>> TypeError: write() argument must be str, not bytes
> >>> The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> >>> Please also report this if it was a user error, so that a better error message can be provided next time.
> >>> A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> >>> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> >>> make[1]: *** [htmldocs] Error 1
> >>> Makefile:1450: recipe for target 'htmldocs' failed
> >>> make: *** [htmldocs] Error 2
> >>>
> >>> Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
> >>
> >> I guess that the dot command writes something to stderr. This is captured
> >> by the extension and printed to stderr ...
> >>
> >> +def dot2format(dot_fname, out_fname):
> >> ...
> >> + exit_code = 42
> >> + with open(out_fname, "w") as out:
> >> + p = subprocess.Popen(
> >> + cmd, stdout = out, stderr = subprocess.PIPE )
> >> + nil, err = p.communicate()
> >> +
> >> + sys.stderr.write(err)
> >> +
> >> + exit_code = p.returncode
> >> + out.flush()
> >> + return bool(exit_code == 0)
> >>
> >>> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> >>> sys.stderr.write(err)
> >>> TypeError: write() argument must be str, not bytes
> >>
> >> Do we need this stderr output? For a first test, uncomment the
> >> "sys.stderr.write(err)“ in line 222. Or, if we really need the
> >> stderr, try:
> >>
> >> - sys.stderr.write(err)
> >> + sys.stderr.write(str(err))
> >
> > Yes, this fixed. I actually did:
> >
> > - sys.stderr.write(err)
> > + sys.stderr.write(str(err))
> > + sys.stderr.write("\n")
> >
> > It is now printing:
> > b''
> >
> > I added the \n print to avoid it to be mixed with the "writing output"
> > prints.
> > No idea how to make sense from it - but clearly, the error report
> > logic require some care ;-)
>
>
> Aargh, I’am a idiot ... I guess 'sys.stderr.write(err)‘ is a artefact
> of my development, simply drop it and the subprocess.PIPE of stderr
> also.
>
> + with open(out_fname, "w") as out:
> + p = subprocess.Popen(
> - cmd, stdout = out, stderr = subprocess.PIPE )
> + cmd, stdout = out)
> + nil, err = p.communicate()
> -
> - sys.stderr.write(err)
> -
> + exit_code = p.returncode
> + out.flush()
> + return bool(exit_code == 0)
>
> I can’t test it ATM, but without redirect stderr, the stderr
> of the parent process is inherited.
>
> https://docs.python.org/3.6/library/subprocess.html#popen-constructor
>
> The Popen.communicate() always returns a tuple (stdout_data, stderr_data)
> with above the tuple is always (None, None).
>
> https://docs.python.org/3.6/library/subprocess.html#subprocess.Popen.communicate
>
> """to get anything other than None in the result tuple,
> you need to give stdout=PIPE and/or stderr=PIPE too."""
>
> Sorry, that I made all this mistakes, but „here“ I have only mail
> and web, no dev-env and I miss my emacs ;)
>
> If the suggestion above does not work, I have to investigate
> more time next weekend.
Hmm... I would be more verbose on output the error code, printing from
where the error came, e. g. printing a message like:
Error #0 when calling dot for '/devel/v4l/patchwork/Documentation/media/uapi/v4l/pipeline.dot': None
As on this patch:
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 32eab0f4cfba..a366a89f4f98 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -217,11 +217,13 @@ def dot2format(dot_fname, out_fname):
with open(out_fname, "w") as out:
p = subprocess.Popen(
cmd, stdout = out, stderr = subprocess.PIPE )
- nil, err = p.communicate()
-
- sys.stderr.write(err)
+ err = p.communicate()
exit_code = p.returncode
+
+ if exit_code != 0:
+ sys.stderr.write("Error #%d when calling dot for '%s': %s\n" % (exit_code, dot_fname, repr(err[0])))
+
out.flush()
return bool(exit_code == 0)
@@ -239,11 +241,13 @@ def svg2pdf(svg_fname, pdf_fname):
cmd = [convert_cmd, svg_fname, pdf_fname]
p = subprocess.Popen(
cmd, stdout = out, stderr = subprocess.PIPE )
- nil, err = p.communicate()
-
- sys.stderr.write(err)
+ err = p.communicate()
exit_code = p.returncode
+
+ if exit_code != 0:
+ sys.stderr.write("Error #%d when calling convert for '%s': %s\n" % (exit_code, dot_fname, repr(err[0])))
+
return bool(exit_code == 0)
^ permalink raw reply related [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:40 ` [PATCH] " Daniel Vetter
2017-03-02 18:06 ` Mauro Carvalho Chehab
@ 2017-03-02 21:29 ` Mauro Carvalho Chehab
2017-03-02 21:36 ` Mauro Carvalho Chehab
1 sibling, 1 reply; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 21:29 UTC (permalink / raw)
To: Daniel Vetter
Cc: linux-doc, DRI Development, Markus Heiser, Jonathan Corbet,
Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 16:40:02 +0100
Daniel Vetter <daniel.vetter@ffwll.ch> escreveu:
> From: Markus Heiser <markus.heiser@darmarit.de>
>
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
>
> * DOT (http://www.graphviz.org)
> * SVG
>
> For image handling use the 'image' replacement::
>
> .. kernel-image:: svg_image.svg
> :alt: simple SVG image
>
> For figure handling use the 'figure' replacement::
>
> .. kernel-figure:: svg_image.svg
> :alt: simple SVG image
>
> SVG image example
>
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
>
> .. kernel-render:: DOT
> :alt: foobar digraph
> :caption: Embedded **DOT** (Graphviz) code.
>
> digraph foo {
> "bar" -> "baz";
> }
>
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
>
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
>
> v2: s/DOC/DOT/ in a few places (by Daniel).
>
> v3: Simplify stuff a bit (by Daniel):
>
> - Remove path detection and setup/check code for that. In
> Documentation/media/Makefile we already simply use these tools,
> better to have one consolidated check if we want/need one. Also
> remove the convertsvg support, we require ImageMagick's convert
> already in the doc build, no need for a 2nd fallback.
>
> - Use sphinx for depency tracking, remove hand-rolled version.
>
> - Forward stderr from dot and convert, otherwise debugging issues with
> the diagrams is impossible.
>
> v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
>
> v5: Use an explicit version check (suggested by Laurent).
Found another issue on the patch. The HTML output is pointing to the
wrong place: instead of using a relative patch, it is keeping
an absolute one.
This is what it produced from Documentation/media/uapi/v4l/dev-subdev.rst:
<div class="figure align-center" id="id2">
<img alt="pipeline.dot" src="/d00/kernel/Documentation/output/media/uapi/v4l/pipeline.svg" /><p class="caption"><span class="caption-text">Image Format Negotiation on Pipelines</span></p>
<div class="legend">
High quality and high speed pipeline configuration</div>
</div>
There, the "src=" is pointing to the full patch, with doesn't work, as
my html server uses a different patch to find the file. It should,
instead, use a patch relative to the place where the html file is
stored, e. g. in this case, either:
./pipeline.svg
or just:
pipeline.svg
Regards,
Mauro
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 21:29 ` Mauro Carvalho Chehab
@ 2017-03-02 21:36 ` Mauro Carvalho Chehab
2017-03-02 21:47 ` Mauro Carvalho Chehab
2017-03-03 8:54 ` Daniel Vetter
0 siblings, 2 replies; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 21:36 UTC (permalink / raw)
To: Daniel Vetter
Cc: linux-doc, DRI Development, Markus Heiser, Jonathan Corbet,
Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 18:29:39 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
> Em Thu, 2 Mar 2017 16:40:02 +0100
> Daniel Vetter <daniel.vetter@ffwll.ch> escreveu:
>
> > From: Markus Heiser <markus.heiser@darmarit.de>
> >
> > This patch brings scalable figure, image handling and a concept to
> > embed *render* markups:
> >
> > * DOT (http://www.graphviz.org)
> > * SVG
> >
> > For image handling use the 'image' replacement::
> >
> > .. kernel-image:: svg_image.svg
> > :alt: simple SVG image
> >
> > For figure handling use the 'figure' replacement::
> >
> > .. kernel-figure:: svg_image.svg
> > :alt: simple SVG image
> >
> > SVG image example
> >
> > Embed *render* markups (or languages) like Graphviz's **DOT** is
> > provided by the *render* directive.::
> >
> > .. kernel-render:: DOT
> > :alt: foobar digraph
> > :caption: Embedded **DOT** (Graphviz) code.
> >
> > digraph foo {
> > "bar" -> "baz";
> > }
> >
> > The *render* directive is a concept to integrate *render* markups and
> > languages, yet supported markups:
> >
> > * DOT: render embedded Graphviz's **DOT**
> > * SVG: render embedded Scalable Vector Graphics (**SVG**)
> >
> > v2: s/DOC/DOT/ in a few places (by Daniel).
> >
> > v3: Simplify stuff a bit (by Daniel):
> >
> > - Remove path detection and setup/check code for that. In
> > Documentation/media/Makefile we already simply use these tools,
> > better to have one consolidated check if we want/need one. Also
> > remove the convertsvg support, we require ImageMagick's convert
> > already in the doc build, no need for a 2nd fallback.
> >
> > - Use sphinx for depency tracking, remove hand-rolled version.
> >
> > - Forward stderr from dot and convert, otherwise debugging issues with
> > the diagrams is impossible.
> >
> > v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> > Implement Markus suggestion for backwards compatability with earlier
> > releases. Laurent reported this, running sphinx 1.3. Solution entirely
> > untested.
> >
> > v5: Use an explicit version check (suggested by Laurent).
>
> Found another issue on the patch. The HTML output is pointing to the
> wrong place: instead of using a relative patch, it is keeping
> an absolute one.
>
> This is what it produced from Documentation/media/uapi/v4l/dev-subdev.rst:
>
> <div class="figure align-center" id="id2">
> <img alt="pipeline.dot" src="/d00/kernel/Documentation/output/media/uapi/v4l/pipeline.svg" /><p class="caption"><span class="caption-text">Image Format Negotiation on Pipelines</span></p>
> <div class="legend">
> High quality and high speed pipeline configuration</div>
> </div>
>
> There, the "src=" is pointing to the full patch, with doesn't work, as
> my html server uses a different patch to find the file. It should,
> instead, use a patch relative to the place where the html file is
> stored, e. g. in this case, either:
> ./pipeline.svg
> or just:
> pipeline.svg
Btw, PDF conversion is also not working:
File "/d00/kernel/Documentation/sphinx/kfigure.py", line 241, in svg2pdf
cmd = [convert_cmd, svg_fname, pdf_fname]
NameError: name 'convert_cmd' is not defined
And including SVG files for HTML output also seems to be problematic.
I'll post the RFCv2 patch that I'm using to test it.
Regards,
Mauro
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 21:36 ` Mauro Carvalho Chehab
@ 2017-03-02 21:47 ` Mauro Carvalho Chehab
2017-03-02 21:54 ` Markus Heiser
2017-03-03 8:54 ` Daniel Vetter
1 sibling, 1 reply; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 21:47 UTC (permalink / raw)
To: Daniel Vetter, Daniel Vetter
Cc: linux-doc, DRI Development, Markus Heiser, Jonathan Corbet,
Jani Nikula, Laurent Pinchart
Em Thu, 2 Mar 2017 18:36:31 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
> > Found another issue on the patch. The HTML output is pointing to the
> > wrong place: instead of using a relative patch, it is keeping
> > an absolute one.
> >
> > This is what it produced from Documentation/media/uapi/v4l/dev-subdev.rst:
> >
> > <div class="figure align-center" id="id2">
> > <img alt="pipeline.dot" src="/d00/kernel/Documentation/output/media/uapi/v4l/pipeline.svg" /><p class="caption"><span class="caption-text">Image Format Negotiation on Pipelines</span></p>
> > <div class="legend">
> > High quality and high speed pipeline configuration</div>
> > </div>
> >
> > There, the "src=" is pointing to the full patch, with doesn't work, as
> > my html server uses a different patch to find the file. It should,
> > instead, use a patch relative to the place where the html file is
> > stored, e. g. in this case, either:
> > ./pipeline.svg
> > or just:
> > pipeline.svg
>
> Btw, PDF conversion is also not working:
>
>
> File "/d00/kernel/Documentation/sphinx/kfigure.py", line 241, in svg2pdf
> cmd = [convert_cmd, svg_fname, pdf_fname]
>
> NameError: name 'convert_cmd' is not defined
>
> And including SVG files for HTML output also seems to be problematic.
Forgot to mention, but I'm using here Sphinx 1.4.9, installed via
pip3 (So, python3).
>
> I'll post the RFCv2 patch that I'm using to test it.
Patch posted. Hopefully, it will help you to see the problems I'm
facing on my tests.
Thanks,
Mauro
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 21:47 ` Mauro Carvalho Chehab
@ 2017-03-02 21:54 ` Markus Heiser
0 siblings, 0 replies; 56+ messages in thread
From: Markus Heiser @ 2017-03-02 21:54 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Daniel Vetter
Cc: Jonathan Corbet, DRI Development, Linux Doc Mailing List,
Laurent Pinchart, Daniel Vetter
>> Btw, PDF conversion is also not working:
>>
>>
>> File "/d00/kernel/Documentation/sphinx/kfigure.py", line 241, in svg2pdf
>> cmd = [convert_cmd, svg_fname, pdf_fname]
>>
>> NameError: name 'convert_cmd' is not defined
>>
>> And including SVG files for HTML output also seems to be problematic.
>
> Forgot to mention, but I'm using here Sphinx 1.4.9, installed via
> pip3 (So, python3).
It seems, that Daniel drops some lines from my first RFC, I miss this lines:
+# setup conversion tools and sphinx extension
+# -------------------------------------------
+
+# Graphviz's dot(1) support
+dot_cmd = which('dot')
+
+# ImageMagick' convert(1) support
+convert_cmd = which('convert')
@Daniel: can you take a look at this / Thanks
-- Markus --
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:16 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
2017-03-02 15:20 ` Laurent Pinchart
2017-03-02 15:40 ` [PATCH] " Daniel Vetter
@ 2017-03-02 22:15 ` Markus Heiser
2017-03-02 23:19 ` Mauro Carvalho Chehab
2 siblings, 1 reply; 56+ messages in thread
From: Markus Heiser @ 2017-03-02 22:15 UTC (permalink / raw)
To: Daniel Vetter
Cc: Jonathan Corbet, DRI Development, linux-doc,
Mauro Carvalho Chehab, Laurent Pinchart, Daniel Vetter
Hi Daniel, hi Mauro,
I have to say, that I lost control over our thread ;)
> v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
I have to come back to a more systematic work, this means;
I have to test this v3 and consider what Mauro and you
already posted.
For this, give me some time. I will send a v4 patch this weekend.
Thanks for all for comments and hints.
-- Markus --
> Am 02.03.2017 um 16:16 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>:
>
> From: Markus Heiser <markus.heiser@darmarit.de>
>
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
>
> * DOT (http://www.graphviz.org)
> * SVG
>
> For image handling use the 'image' replacement::
>
> .. kernel-image:: svg_image.svg
> :alt: simple SVG image
>
> For figure handling use the 'figure' replacement::
>
> .. kernel-figure:: svg_image.svg
> :alt: simple SVG image
>
> SVG image example
>
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
>
> .. kernel-render:: DOT
> :alt: foobar digraph
> :caption: Embedded **DOT** (Graphviz) code.
>
> digraph foo {
> "bar" -> "baz";
> }
>
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
>
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
>
> v2: s/DOC/DOT/ in a few places (by Daniel).
>
> v3: Simplify stuff a bit (by Daniel):
>
> - Remove path detection and setup/check code for that. In
> Documentation/media/Makefile we already simply use these tools,
> better to have one consolidated check if we want/need one. Also
> remove the convertsvg support, we require ImageMagick's convert
> already in the doc build, no need for a 2nd fallback.
>
> - Use sphinx for depency tracking, remove hand-rolled version.
>
> - Forward stderr from dot and convert, otherwise debugging issues with
> the diagrams is impossible.
>
> v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: linux-doc@vger.kernel.org
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> Cc: Markus Heiser <markus.heiser@darmarit.de>
> Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> (v1)
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> ---
> Documentation/conf.py | 2 +-
> Documentation/doc-guide/hello.dot | 3 +
> Documentation/doc-guide/sphinx.rst | 90 ++++++-
> Documentation/doc-guide/svg_image.svg | 10 +
> Documentation/process/changes.rst | 7 +-
> Documentation/sphinx/kfigure.py | 444 ++++++++++++++++++++++++++++++++++
> 6 files changed, 550 insertions(+), 6 deletions(-)
> create mode 100644 Documentation/doc-guide/hello.dot
> create mode 100644 Documentation/doc-guide/svg_image.svg
> create mode 100644 Documentation/sphinx/kfigure.py
>
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index f6823cf01275..e3f537ce2935 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -34,7 +34,7 @@ from load_config import loadConfig
> # Add any Sphinx extension module names here, as strings. They can be
> # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
> # ones.
> -extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain']
> +extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure']
>
> # The name of the math extension changed on Sphinx 1.4
> if major == 1 and minor > 3:
> diff --git a/Documentation/doc-guide/hello.dot b/Documentation/doc-guide/hello.dot
> new file mode 100644
> index 000000000000..504621dfc595
> --- /dev/null
> +++ b/Documentation/doc-guide/hello.dot
> @@ -0,0 +1,3 @@
> +graph G {
> + Hello -- World
> +}
> diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
> index 532d65b70500..b902744ce7dd 100644
> --- a/Documentation/doc-guide/sphinx.rst
> +++ b/Documentation/doc-guide/sphinx.rst
> @@ -34,8 +34,10 @@ format-specific subdirectories under ``Documentation/output``.
>
> To generate documentation, Sphinx (``sphinx-build``) must obviously be
> installed. For prettier HTML output, the Read the Docs Sphinx theme
> -(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is also
> -needed. All of these are widely available and packaged in distributions.
> +(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need
> +``XeLaTeX`` and CairoSVG (http://cairosvg.org) or alternatively ``convert(1)``
> +from ImageMagick (https://www.imagemagick.org). All of these are widely
> +available and packaged in distributions.
>
> To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make
> variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose
> @@ -232,3 +234,87 @@ Rendered as:
> * .. _`last row`:
>
> - column 3
> +
> +
> +Figures & Images
> +================
> +
> +If you want to add an image, you should use the ``kernel-figure`` and
> +``kernel-image`` directives. E.g. to insert a figure with a scalable
> +image format use SVG::
> +
> + .. kernel-figure:: svg_image.svg
> + :alt: simple SVG image
> +
> + SVG image example
> +
> +.. kernel-figure:: svg_image.svg
> + :alt: simple SVG image
> +
> + SVG image example
> +
> +The kernel figure (and image) directive support **DOT** formated files, see
> +
> +* DOT: http://graphviz.org/pdf/dotguide.pdf
> +* Graphviz: http://www.graphviz.org/content/dot-language
> +
> +A simple example::
> +
> + .. kernel-figure:: hello.dot
> + :alt: hello world
> +
> + DOT's hello world example
> +
> +.. kernel-figure:: hello.dot
> + :alt: hello world
> +
> + DOT's hello world example
> +
> +Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the
> +``kernel-render`` directives.::
> +
> + .. kernel-render:: DOT
> + :alt: foobar digraph
> + :caption: Embedded **DOT** (Graphviz) code.
> +
> + digraph foo {
> + "bar" -> "baz";
> + }
> +
> +How this will be rendered depends on the installed tools. If Graphviz is
> +installed, you will see an vector image. If not the raw markup is inserted as
> +*literal-block*.
> +
> +.. kernel-render:: DOT
> + :alt: foobar digraph
> + :caption: Embedded **DOT** (Graphviz) code.
> +
> + digraph foo {
> + "bar" -> "baz";
> + }
> +
> +The *render* directive has all the options known from the *figure* directive,
> +plus option ``caption``. If ``caption`` has a value, a *figure* node is
> +inserted. If not, a *image* node is inserted.
> +
> +Embedded **SVG**::
> +
> + .. kernel-render:: SVG
> + :caption: Embedded **SVG** markup.
> + :alt: so-nw-arrow
> +
> + <?xml version="1.0" encoding="UTF-8"?>
> + <svg xmlns="http://www.w3.org/2000/svg" version="1.1" ...>
> + ...
> + </svg>
> +
> +.. kernel-render:: SVG
> + :caption: Embedded **SVG** markup.
> + :alt: so-nw-arrow
> +
> + <?xml version="1.0" encoding="UTF-8"?>
> + <svg xmlns="http://www.w3.org/2000/svg"
> + version="1.1" baseProfile="full" width="70px" height="40px" viewBox="0 0 700 400">
> + <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
> + <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
> + </svg>
> diff --git a/Documentation/doc-guide/svg_image.svg b/Documentation/doc-guide/svg_image.svg
> new file mode 100644
> index 000000000000..5405f85b8137
> --- /dev/null
> +++ b/Documentation/doc-guide/svg_image.svg
> @@ -0,0 +1,10 @@
> +<?xml version="1.0" encoding="UTF-8"?>
> +<!-- originate: https://commons.wikimedia.org/wiki/File:Variable_Resistor.svg -->
> +<svg xmlns="http://www.w3.org/2000/svg"
> + version="1.1" baseProfile="full"
> + width="70px" height="40px" viewBox="0 0 700 400">
> + <line x1="0" y1="200" x2="700" y2="200" stroke="black" stroke-width="20px"/>
> + <rect x="100" y="100" width="500" height="200" fill="white" stroke="black" stroke-width="20px"/>
> + <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
> + <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
> +</svg>
> diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
> index 56ce66114665..e4f25038ef65 100644
> --- a/Documentation/process/changes.rst
> +++ b/Documentation/process/changes.rst
> @@ -318,9 +318,10 @@ PDF outputs, it is recommended to use version 1.4.6.
> .. note::
>
> Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX``
> - version 3.14159265. Depending on the distribution, you may also need
> - to install a series of ``texlive`` packages that provide the minimal
> - set of functionalities required for ``XeLaTex`` to work.
> + version 3.14159265. Depending on the distribution, you may also need to
> + install a series of ``texlive`` packages that provide the minimal set of
> + functionalities required for ``XeLaTex`` to work. For PDF output you'll also
> + need ``convert(1)`` from ImageMagick (https://www.imagemagick.org).
>
> Other tools
> -----------
> diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
> new file mode 100644
> index 000000000000..cbb23e089186
> --- /dev/null
> +++ b/Documentation/sphinx/kfigure.py
> @@ -0,0 +1,444 @@
> +# -*- coding: utf-8; mode: python -*-
> +# pylint: disable=C0103
> +u"""
> + scalable figure and image handling
> + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> + Sphinx extension which implements scalable image handling.
> +
> + :copyright: Copyright (C) 2016 Markus Heiser
> + :license: GPL Version 2, June 1991 see Linux/COPYING for details.
> +
> + The build for image formats depence on image's source format and output's
> + destination format. This extension implement methods to simplify image
> + handling from the author's POV. Directives like ``kernel-figure`` implement
> + methods *to* always get the best output-format even if some tools are not
> + installed.For more details take a look at ``convert_image(...)`` which is
> + the core of all conversions.
> +
> + * ``.. kernel-image``: for image handling / ``.. image::`` replacement
> +
> + * ``.. kernel-figure``: for figure handling / ``.. figure::`` replacement
> +
> + * ``.. kernel-render``: for render markup / a concept to embed *render*
> + markups (or languages). Supported markups (see ``RENDER_MARKUP_EXT``)
> +
> + + ``DOT``: render embedded Graphviz's **DOC**
> + + ``SVG``: render embedded Scalable Vector Graphics (**SVG**)
> + + ... *developable*
> +
> + Used tools:
> +
> + * ``dot(1)``: Graphviz (http://www.graphviz.org). If Graphviz is not
> + available, the DOT language is inserted as literal-block.
> +
> + * SVG to PDF: To generate PDF, you need at least one of this tools:
> +
> + - ``convert(1)``: ImageMagick (https://www.imagemagick.org)
> +
> + List of customizations:
> +
> + * generate PDF from SVG / used by PDF (LaTeX) builder
> +
> + * generate SVG (html-builder) and PDF (latex-builder) from DOT files.
> + DOT: see http://www.graphviz.org/content/dot-language
> +
> + """
> +
> +import os
> +from os import path
> +import subprocess
> +from hashlib import sha1
> +import sys
> +
> +from docutils import nodes
> +from docutils.statemachine import ViewList
> +from docutils.parsers.rst import directives
> +from docutils.parsers.rst.directives import images
> +try:
> + from sphinx.directives.patches import Figure
> +except ImportError:
> + Figure = images.Figure
> +
> +__version__ = '1.0'
> +
> +# simple helper
> +# -------------
> +
> +def mkdir(folder, mode=0o775):
> + if not path.isdir(folder):
> + os.makedirs(folder, mode)
> +
> +# def debug_handle(self, node): # pylint: disable=W0613
> +# from linuxdoc.kernel_doc import CONSOLE
> +# CONSOLE()
> +
> +def pass_handle(self, node): # pylint: disable=W0613
> + pass
> +
> +# setup conversion tools and sphinx extension
> +# -------------------------------------------
> +
> +def setup(app):
> + # image handling
> + app.add_directive("kernel-image", KernelImage)
> + app.add_node(kernel_image,
> + html = (visit_kernel_image, pass_handle),
> + latex = (visit_kernel_image, pass_handle),
> + texinfo = (visit_kernel_image, pass_handle),
> + text = (visit_kernel_image, pass_handle),
> + man = (visit_kernel_image, pass_handle), )
> +
> + # figure handling
> + app.add_directive("kernel-figure", KernelFigure)
> + app.add_node(kernel_figure,
> + html = (visit_kernel_figure, pass_handle),
> + latex = (visit_kernel_figure, pass_handle),
> + texinfo = (visit_kernel_figure, pass_handle),
> + text = (visit_kernel_figure, pass_handle),
> + man = (visit_kernel_figure, pass_handle), )
> +
> + # render handling
> + app.add_directive('kernel-render', KernelRender)
> + app.add_node(kernel_render,
> + html = (visit_kernel_render, pass_handle),
> + latex = (visit_kernel_render, pass_handle),
> + texinfo = (visit_kernel_render, pass_handle),
> + text = (visit_kernel_render, pass_handle),
> + man = (visit_kernel_render, pass_handle), )
> +
> + return dict(
> + version = __version__,
> + parallel_read_safe = True,
> + parallel_write_safe = True
> + )
> +
> +# integrate conversion tools
> +# --------------------------
> +
> +RENDER_MARKUP_EXT = {
> + # The '.ext' must be handled by convert_image(..) function's *in_ext* input.
> + # <name> : <.ext>
> + 'DOT' : '.dot'
> + , 'SVG' : '.svg'
> +}
> +
> +def convert_image(img_node, translator): # pylint: disable=R0912
> + """Convert an image node for the builder.
> +
> + Different builder prefer different image formats, e.g. *latex* builder
> + prefer PDF while *html* builder prefer SVG format for images.
> +
> + This function handles outputs image formats in depence of source the format
> + of the image and the translator's output format. This also means to
> + manipulate/update the *image* dictionary of the builder (``builder.images``)
> +
> + """
> + fname, in_ext = path.splitext(path.basename(img_node['uri']))
> + src_fname = path.join(translator.builder.srcdir, img_node['uri'])
> + src_folder = path.dirname(img_node['uri'])
> + out_dir = translator.builder.outdir
> + dst_fname = None
> +
> + # in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages
> + verbose = translator.builder.app.verbose
> + warn = translator.builder.warn
> +
> + verbose('assert best format for: ' + img_node['uri'])
> +
> + if in_ext == '.dot':
> +
> + # ----------
> + # handle DOT
> + # ----------
> +
> + dst_fname = path.join(out_dir, fname + '.pdf')
> +
> + if translator.builder.format == 'html':
> + dst_fname = path.join(out_dir, src_folder, fname + '.svg')
> + else:
> + # all other builder formats will include DOT as raw
> + with open(src_fname, "r") as dot:
> + data = dot.read()
> + node = nodes.literal_block(data, data)
> + img_node.replace_self(node)
> +
> +
> + elif in_ext == '.svg':
> +
> + # ----------
> + # handle SVG
> + # ----------
> +
> + if translator.builder.format == 'latex':
> + dst_fname = path.join(out_dir, fname + '.pdf')
> +
> + if dst_fname:
> + name = dst_fname[len(out_dir) + 1:]
> + # the builder needs not to copy one more time, so pop it if exists.
> + translator.builder.images.pop(img_node['uri'], None)
> + img_node['uri'] = dst_fname
> + img_node['candidates'] = {'*': dst_fname}
> +
> + mkdir(path.dirname(dst_fname))
> +
> + if in_ext == '.dot':
> + verbose('convert DOT to: {out}/' + name)
> + dot2format(src_fname, dst_fname)
> +
> + elif in_ext == '.svg':
> + verbose('convert SVG to: {out}/' + name)
> + svg2pdf(src_fname, dst_fname)
> +
> +def dot2format(dot_fname, out_fname):
> + """Converts DOT file to ``out_fname`` using ``dot(1)``.
> +
> + * ``dot_fname`` pathname of the input DOT file, including extension ``.dot``
> + * ``out_fname`` pathname of the output file, including format extension
> +
> + The *format extension* depends on the ``dot`` command (see ``man dot``
> + option ``-Txxx``). Normally you will use one of the following extensions:
> +
> + - ``.ps`` for PostScript,
> + - ``.svg`` or ``svgz`` for Structured Vector Graphics,
> + - ``.fig`` for XFIG graphics and
> + - ``.png`` or ``gif`` for common bitmap graphics.
> +
> + """
> + out_format = path.splitext(out_fname)[1][1:]
> + cmd = ['dot', '-T%s' % out_format, dot_fname]
> + exit_code = 42
> + with open(out_fname, "w") as out:
> + p = subprocess.Popen(
> + cmd, stdout = out, stderr = subprocess.PIPE )
> + nil, err = p.communicate()
> +
> + sys.stderr.write(err)
> +
> + exit_code = p.returncode
> + out.flush()
> + return bool(exit_code == 0)
> +
> +def svg2pdf(svg_fname, pdf_fname):
> + """Converts SVG to PDF with CairoSVG or ``convert(1)`` command.
> +
> + Uses ``convert(1)`` from ImageMagick (https://www.imagemagick.org) for
> + conversion. Returns ``True`` on success and ``False`` if an error occurred
> + (e.g. none of the conversion tool is available).
> +
> + * ``svg_fname`` pathname of the input SVG file with extension (``.svg``)
> + * ``pdf_name`` pathname of the output PDF file with extension (``.pdf``)
> +
> + """
> + cmd = [convert_cmd, svg_fname, pdf_fname]
> + p = subprocess.Popen(
> + cmd, stdout = out, stderr = subprocess.PIPE )
> + nil, err = p.communicate()
> +
> + sys.stderr.write(err)
> +
> + exit_code = p.returncode
> + return bool(exit_code == 0)
> +
> +
> +# image handling
> +# ---------------------
> +
> +def visit_kernel_image(self, node): # pylint: disable=W0613
> + """Visitor of the ``kernel_image`` Node.
> +
> + Handles the ``image`` child-node with the ``convert_image(...)``.
> + """
> + img_node = node[0]
> + convert_image(img_node, self)
> +
> +class kernel_image(nodes.General, nodes.Element):
> + """Node for ``kernel-image`` directive."""
> + pass
> +
> +class KernelImage(images.Image):
> + u"""KernelImage directive
> +
> + Earns everything from ``.. image::`` directive, except *remote URI* and
> + *glob* pattern. The KernelImage wraps a image node into a
> + kernel_image node. See ``visit_kernel_image``.
> + """
> +
> + def run(self):
> + env = self.state.document.settings.env
> +
> + uri = self.arguments[0]
> + if uri.endswith('.*') or uri.find('://') != -1:
> + raise self.severe(
> + 'Error in "%s: %s": glob pattern and remote images are not allowed'
> + % (self.name, uri))
> +
> + # Tell sphinx of the dependency
> + env.note_dependency(os.path.abspath(uri))
> +
> + result = images.Image.run(self)
> + if len(result) == 2 or isinstance(result[0], nodes.system_message):
> + return result
> + (image_node,) = result
> + # wrap image node into a kernel_image node / see visitors
> + node = kernel_image('', image_node)
> + return [node]
> +
> +# figure handling
> +# ---------------------
> +
> +def visit_kernel_figure(self, node): # pylint: disable=W0613
> + """Visitor of the ``kernel_figure`` Node.
> +
> + Handles the ``image`` child-node with the ``convert_image(...)``.
> + """
> + img_node = node[0][0]
> + convert_image(img_node, self)
> +
> +class kernel_figure(nodes.General, nodes.Element):
> + """Node for ``kernel-figure`` directive."""
> +
> +class KernelFigure(Figure):
> + u"""KernelImage directive
> +
> + Earns everything from ``.. figure::`` directive, except *remote URI* and
> + *glob* pattern. The KernelFigure wraps a figure node into a kernel_figure
> + node. See ``visit_kernel_figure``.
> + """
> +
> + def run(self):
> + env = self.state.document.settings.env
> +
> + uri = self.arguments[0]
> + if uri.endswith('.*') or uri.find('://') != -1:
> + raise self.severe(
> + 'Error in "%s: %s":'
> + ' glob pattern and remote images are not allowed'
> + % (self.name, uri))
> +
> + # Tell sphinx of the dependency
> + env.note_dependency(os.path.abspath(uri))
> +
> + result = Figure.run(self)
> + if len(result) == 2 or isinstance(result[0], nodes.system_message):
> + return result
> + (figure_node,) = result
> + # wrap figure node into a kernel_figure node / see visitors
> + node = kernel_figure('', figure_node)
> + return [node]
> +
> +
> +# render handling
> +# ---------------------
> +
> +def visit_kernel_render(self, node):
> + """Visitor of the ``kernel_render`` Node.
> +
> + If rendering tools available, save the markup of the ``literal_block`` child
> + node into a file and replace the ``literal_block`` node with a new created
> + ``image`` node, pointing to the saved markup file. Afterwards, handle the
> + image child-node with the ``convert_image(...)``.
> + """
> +
> + verbose = self.builder.app.verbose
> + warn = self.builder.warn
> + srclang = node.get('srclang')
> +
> + verbose('visit kernel-render node lang: "%s"' % (srclang))
> +
> + tmp_ext = RENDER_MARKUP_EXT.get(srclang, None)
> + if tmp_ext is None:
> + warn('kernel-render: "%s" unknow / include raw.' % (srclang))
> + return
> +
> + literal_block = node[0]
> + code = literal_block.astext()
> +
> + if tmp_ext:
> + hashobj = code.encode('utf-8') # str(node.attributes)
> + fname = '%s-%s' % (srclang, sha1(hashobj).hexdigest())
> + tmp_fname = path.join(
> + self.builder.outdir, self.builder.imagedir, fname + tmp_ext)
> +
> + if not path.isfile(tmp_fname):
> + mkdir(path.dirname(tmp_fname))
> + with open(tmp_fname, "w") as out:
> + out.write(code)
> +
> + image_node = nodes.image(node.rawsource, **node.attributes)
> + image_node['uri'] = tmp_fname
> +
> + literal_block.replace_self(image_node)
> + convert_image(image_node, self)
> +
> +
> +class kernel_render(nodes.General, nodes.Inline, nodes.Element):
> + """Node for ``kernel-render`` directive."""
> + pass
> +
> +class KernelRender(Figure):
> + u"""KernelRender directive
> +
> + Render content by external tool. Has all the options known from the
> + *figure* directive, plus option ``caption``. If ``caption`` has a
> + value, a figure node with the *caption* is inserted. If not, a image node is
> + inserted.
> +
> + The KernelRender directive wraps the text of the directive into a
> + literal_block node and wraps it into a kernel_render node. See
> + ``visit_kernel_render``.
> + """
> + has_content = True
> + required_arguments = 1
> + optional_arguments = 0
> + final_argument_whitespace = False
> +
> + # earn options from 'figure'
> + option_spec = Figure.option_spec.copy()
> + option_spec['caption'] = directives.unchanged
> +
> + def run(self):
> + return [self.build_node()]
> +
> + def build_node(self):
> +
> + srclang = self.arguments[0].strip()
> + if srclang not in RENDER_MARKUP_EXT.keys():
> + return [self.state_machine.reporter.warning(
> + 'Unknow source language "%s", use one of: %s.' % (
> + srclang, ",".join(RENDER_MARKUP_EXT.keys())),
> + line=self.lineno)]
> +
> + code = '\n'.join(self.content)
> + if not code.strip():
> + return [self.state_machine.reporter.warning(
> + 'Ignoring "%s" directive without content.' % (
> + self.name),
> + line=self.lineno)]
> +
> + node = kernel_render()
> + node['alt'] = self.options.get('alt','')
> + node['srclang'] = srclang
> + literal_node = nodes.literal_block(code, code)
> + node += literal_node
> +
> + caption = self.options.get('caption')
> + if caption:
> + # parse cation's content
> + parsed = nodes.Element()
> + self.state.nested_parse(
> + ViewList([caption], source=''), self.content_offset, parsed)
> + caption_node = nodes.caption(
> + parsed[0].rawsource, '', *parsed[0].children)
> + caption_node.source = parsed[0].source
> + caption_node.line = parsed[0].line
> +
> + figure_node = nodes.figure('', node)
> + for k,v in self.options.items():
> + figure_node[k] = v
> + figure_node += caption_node
> +
> + node = figure_node
> +
> + return node
> +
> --
> 2.11.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-doc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 22:15 ` [PATCH 2/6] " Markus Heiser
@ 2017-03-02 23:19 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 56+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 23:19 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, linux-doc, DRI Development, Jonathan Corbet,
Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 23:15:49 +0100
Markus Heiser <markus.heiser@darmarit.de> escreveu:
> Hi Daniel, hi Mauro,
>
> I have to say, that I lost control over our thread ;)
>
> > v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> > Implement Markus suggestion for backwards compatability with earlier
> > releases. Laurent reported this, running sphinx 1.3. Solution entirely
> > untested.
>
> I have to come back to a more systematic work, this means;
> I have to test this v3 and consider what Mauro and you
> already posted.
Yes, you're lost :-) The last patch Daniel Sent was v5. From his last patch:
> v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
>
> v5: Use an explicit version check (suggested by Laurent).
https://marc.info/?l=linux-doc&m=148847115500505&w=2
Thanks,
Mauro
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 21:36 ` Mauro Carvalho Chehab
2017-03-02 21:47 ` Mauro Carvalho Chehab
@ 2017-03-03 8:54 ` Daniel Vetter
1 sibling, 0 replies; 56+ messages in thread
From: Daniel Vetter @ 2017-03-03 8:54 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Markus Heiser, Jonathan Corbet, DRI Development,
Linux Doc Mailing List, Laurent Pinchart, Daniel Vetter
On Thu, Mar 2, 2017 at 10:36 PM, Mauro Carvalho Chehab
<mchehab@s-opensource.com> wrote:
> Em Thu, 2 Mar 2017 18:29:39 -0300
> Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
>
>> Em Thu, 2 Mar 2017 16:40:02 +0100
>> Daniel Vetter <daniel.vetter@ffwll.ch> escreveu:
>>
>> > From: Markus Heiser <markus.heiser@darmarit.de>
>> >
>> > This patch brings scalable figure, image handling and a concept to
>> > embed *render* markups:
>> >
>> > * DOT (http://www.graphviz.org)
>> > * SVG
>> >
>> > For image handling use the 'image' replacement::
>> >
>> > .. kernel-image:: svg_image.svg
>> > :alt: simple SVG image
>> >
>> > For figure handling use the 'figure' replacement::
>> >
>> > .. kernel-figure:: svg_image.svg
>> > :alt: simple SVG image
>> >
>> > SVG image example
>> >
>> > Embed *render* markups (or languages) like Graphviz's **DOT** is
>> > provided by the *render* directive.::
>> >
>> > .. kernel-render:: DOT
>> > :alt: foobar digraph
>> > :caption: Embedded **DOT** (Graphviz) code.
>> >
>> > digraph foo {
>> > "bar" -> "baz";
>> > }
>> >
>> > The *render* directive is a concept to integrate *render* markups and
>> > languages, yet supported markups:
>> >
>> > * DOT: render embedded Graphviz's **DOT**
>> > * SVG: render embedded Scalable Vector Graphics (**SVG**)
>> >
>> > v2: s/DOC/DOT/ in a few places (by Daniel).
>> >
>> > v3: Simplify stuff a bit (by Daniel):
>> >
>> > - Remove path detection and setup/check code for that. In
>> > Documentation/media/Makefile we already simply use these tools,
>> > better to have one consolidated check if we want/need one. Also
>> > remove the convertsvg support, we require ImageMagick's convert
>> > already in the doc build, no need for a 2nd fallback.
>> >
>> > - Use sphinx for depency tracking, remove hand-rolled version.
>> >
>> > - Forward stderr from dot and convert, otherwise debugging issues with
>> > the diagrams is impossible.
>> >
>> > v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
>> > Implement Markus suggestion for backwards compatability with earlier
>> > releases. Laurent reported this, running sphinx 1.3. Solution entirely
>> > untested.
>> >
>> > v5: Use an explicit version check (suggested by Laurent).
>>
>> Found another issue on the patch. The HTML output is pointing to the
>> wrong place: instead of using a relative patch, it is keeping
>> an absolute one.
>>
>> This is what it produced from Documentation/media/uapi/v4l/dev-subdev.rst:
>>
>> <div class="figure align-center" id="id2">
>> <img alt="pipeline.dot" src="/d00/kernel/Documentation/output/media/uapi/v4l/pipeline.svg" /><p class="caption"><span class="caption-text">Image Format Negotiation on Pipelines</span></p>
>> <div class="legend">
>> High quality and high speed pipeline configuration</div>
>> </div>
>>
>> There, the "src=" is pointing to the full patch, with doesn't work, as
>> my html server uses a different patch to find the file. It should,
>> instead, use a patch relative to the place where the html file is
>> stored, e. g. in this case, either:
>> ./pipeline.svg
>> or just:
>> pipeline.svg
>
> Btw, PDF conversion is also not working:
>
>
> File "/d00/kernel/Documentation/sphinx/kfigure.py", line 241, in svg2pdf
> cmd = [convert_cmd, svg_fname, pdf_fname]
>
> NameError: name 'convert_cmd' is not defined
Oops, should be fixed in the next version. But my tex here is broken,
so I can't test :(
> And including SVG files for HTML output also seems to be problematic.
Yeah, it's all using absolute paths. I'm fixing it.
> I'll post the RFCv2 patch that I'm using to test it.
I'll also fix the stderr string conversion thing and then resend the
new version.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-02-28 17:13 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
2017-03-01 15:56 ` Gabriel Krisman Bertazi
2017-03-02 12:26 ` Laurent Pinchart
@ 2017-03-06 10:12 ` Daniel Vetter
2017-03-06 13:13 ` Markus Heiser
2 siblings, 1 reply; 56+ messages in thread
From: Daniel Vetter @ 2017-03-06 10:12 UTC (permalink / raw)
To: linux-doc
Cc: Markus Heiser, Jonathan Corbet, Mauro Carvalho Chehab,
DRI Development, Daniel Vetter
On Tue, Feb 28, 2017 at 06:13:15PM +0100, Daniel Vetter wrote:
> From: Markus Heiser <markus.heiser@darmarit.de>
>
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
>
> * DOT (http://www.graphviz.org)
> * SVG
>
> For image handling use the 'image' replacement::
>
> .. kernel-image:: svg_image.svg
> :alt: simple SVG image
>
> For figure handling use the 'figure' replacement::
>
> .. kernel-figure:: svg_image.svg
> :alt: simple SVG image
>
> SVG image example
>
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
>
> .. kernel-render:: DOT
> :alt: foobar digraph
> :caption: Embedded **DOT** (Graphviz) code.
>
> digraph foo {
> "bar" -> "baz";
> }
>
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
>
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
>
> v2: s/DOC/DOT/ in a few places (by Daniel).
>
> v3: Simplify stuff a bit (by Daniel):
>
> - Remove path detection and setup/check code for that. In
> Documentation/media/Makefile we already simply use these tools,
> better to have one consolidated check if we want/need one. Also
> remove the convertsvg support, we require ImageMagick's convert
> already in the doc build, no need for a 2nd fallback.
>
> - Use sphinx for depency tracking, remove hand-rolled version.
>
> - Forward stderr from dot and convert, otherwise debugging issues with
> the diagrams is impossible.
>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: linux-doc@vger.kernel.org
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> (v1)
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Just to avoid confusion: Markus&I chatted a bit in private, and he
volunteered to take over, and fix the few issues that need to be fixed in
his original patch:
- drop the convertsvg python fallback, since we need convert anyway
- relative paths
- stderr fwd
- and the small polish I noticed in docs
I think with my blind hacking I botched the job too much :-)
Anyway, still hoping we could land this for 4.12, I really want the
drm-side doc improvements this provides ...
Thanks, Daniel
> ---
> Documentation/conf.py | 2 +-
> Documentation/doc-guide/hello.dot | 3 +
> Documentation/doc-guide/sphinx.rst | 90 ++++++-
> Documentation/doc-guide/svg_image.svg | 10 +
> Documentation/process/changes.rst | 7 +-
> Documentation/sphinx/kfigure.py | 442 ++++++++++++++++++++++++++++++++++
> 6 files changed, 548 insertions(+), 6 deletions(-)
> create mode 100644 Documentation/doc-guide/hello.dot
> create mode 100644 Documentation/doc-guide/svg_image.svg
> create mode 100644 Documentation/sphinx/kfigure.py
>
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index f6823cf01275..e3f537ce2935 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -34,7 +34,7 @@ from load_config import loadConfig
> # Add any Sphinx extension module names here, as strings. They can be
> # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
> # ones.
> -extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain']
> +extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure']
>
> # The name of the math extension changed on Sphinx 1.4
> if major == 1 and minor > 3:
> diff --git a/Documentation/doc-guide/hello.dot b/Documentation/doc-guide/hello.dot
> new file mode 100644
> index 000000000000..504621dfc595
> --- /dev/null
> +++ b/Documentation/doc-guide/hello.dot
> @@ -0,0 +1,3 @@
> +graph G {
> + Hello -- World
> +}
> diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
> index 532d65b70500..b902744ce7dd 100644
> --- a/Documentation/doc-guide/sphinx.rst
> +++ b/Documentation/doc-guide/sphinx.rst
> @@ -34,8 +34,10 @@ format-specific subdirectories under ``Documentation/output``.
>
> To generate documentation, Sphinx (``sphinx-build``) must obviously be
> installed. For prettier HTML output, the Read the Docs Sphinx theme
> -(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is also
> -needed. All of these are widely available and packaged in distributions.
> +(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need
> +``XeLaTeX`` and CairoSVG (http://cairosvg.org) or alternatively ``convert(1)``
> +from ImageMagick (https://www.imagemagick.org). All of these are widely
> +available and packaged in distributions.
>
> To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make
> variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose
> @@ -232,3 +234,87 @@ Rendered as:
> * .. _`last row`:
>
> - column 3
> +
> +
> +Figures & Images
> +================
> +
> +If you want to add an image, you should use the ``kernel-figure`` and
> +``kernel-image`` directives. E.g. to insert a figure with a scalable
> +image format use SVG::
> +
> + .. kernel-figure:: svg_image.svg
> + :alt: simple SVG image
> +
> + SVG image example
> +
> +.. kernel-figure:: svg_image.svg
> + :alt: simple SVG image
> +
> + SVG image example
> +
> +The kernel figure (and image) directive support **DOT** formated files, see
> +
> +* DOT: http://graphviz.org/pdf/dotguide.pdf
> +* Graphviz: http://www.graphviz.org/content/dot-language
> +
> +A simple example::
> +
> + .. kernel-figure:: hello.dot
> + :alt: hello world
> +
> + DOT's hello world example
> +
> +.. kernel-figure:: hello.dot
> + :alt: hello world
> +
> + DOT's hello world example
> +
> +Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the
> +``kernel-render`` directives.::
> +
> + .. kernel-render:: DOT
> + :alt: foobar digraph
> + :caption: Embedded **DOT** (Graphviz) code.
> +
> + digraph foo {
> + "bar" -> "baz";
> + }
> +
> +How this will be rendered depends on the installed tools. If Graphviz is
> +installed, you will see an vector image. If not the raw markup is inserted as
> +*literal-block*.
> +
> +.. kernel-render:: DOT
> + :alt: foobar digraph
> + :caption: Embedded **DOT** (Graphviz) code.
> +
> + digraph foo {
> + "bar" -> "baz";
> + }
> +
> +The *render* directive has all the options known from the *figure* directive,
> +plus option ``caption``. If ``caption`` has a value, a *figure* node is
> +inserted. If not, a *image* node is inserted.
> +
> +Embedded **SVG**::
> +
> + .. kernel-render:: SVG
> + :caption: Embedded **SVG** markup.
> + :alt: so-nw-arrow
> +
> + <?xml version="1.0" encoding="UTF-8"?>
> + <svg xmlns="http://www.w3.org/2000/svg" version="1.1" ...>
> + ...
> + </svg>
> +
> +.. kernel-render:: SVG
> + :caption: Embedded **SVG** markup.
> + :alt: so-nw-arrow
> +
> + <?xml version="1.0" encoding="UTF-8"?>
> + <svg xmlns="http://www.w3.org/2000/svg"
> + version="1.1" baseProfile="full" width="70px" height="40px" viewBox="0 0 700 400">
> + <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
> + <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
> + </svg>
> diff --git a/Documentation/doc-guide/svg_image.svg b/Documentation/doc-guide/svg_image.svg
> new file mode 100644
> index 000000000000..5405f85b8137
> --- /dev/null
> +++ b/Documentation/doc-guide/svg_image.svg
> @@ -0,0 +1,10 @@
> +<?xml version="1.0" encoding="UTF-8"?>
> +<!-- originate: https://commons.wikimedia.org/wiki/File:Variable_Resistor.svg -->
> +<svg xmlns="http://www.w3.org/2000/svg"
> + version="1.1" baseProfile="full"
> + width="70px" height="40px" viewBox="0 0 700 400">
> + <line x1="0" y1="200" x2="700" y2="200" stroke="black" stroke-width="20px"/>
> + <rect x="100" y="100" width="500" height="200" fill="white" stroke="black" stroke-width="20px"/>
> + <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
> + <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
> +</svg>
> diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
> index 56ce66114665..e4f25038ef65 100644
> --- a/Documentation/process/changes.rst
> +++ b/Documentation/process/changes.rst
> @@ -318,9 +318,10 @@ PDF outputs, it is recommended to use version 1.4.6.
> .. note::
>
> Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX``
> - version 3.14159265. Depending on the distribution, you may also need
> - to install a series of ``texlive`` packages that provide the minimal
> - set of functionalities required for ``XeLaTex`` to work.
> + version 3.14159265. Depending on the distribution, you may also need to
> + install a series of ``texlive`` packages that provide the minimal set of
> + functionalities required for ``XeLaTex`` to work. For PDF output you'll also
> + need ``convert(1)`` from ImageMagick (https://www.imagemagick.org).
>
> Other tools
> -----------
> diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
> new file mode 100644
> index 000000000000..8f749d389dae
> --- /dev/null
> +++ b/Documentation/sphinx/kfigure.py
> @@ -0,0 +1,442 @@
> +# -*- coding: utf-8; mode: python -*-
> +# pylint: disable=C0103
> +u"""
> + scalable figure and image handling
> + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> + Sphinx extension which implements scalable image handling.
> +
> + :copyright: Copyright (C) 2016 Markus Heiser
> + :license: GPL Version 2, June 1991 see Linux/COPYING for details.
> +
> + The build for image formats depence on image's source format and output's
> + destination format. This extension implement methods to simplify image
> + handling from the author's POV. Directives like ``kernel-figure`` implement
> + methods *to* always get the best output-format even if some tools are not
> + installed.For more details take a look at ``convert_image(...)`` which is
> + the core of all conversions.
> +
> + * ``.. kernel-image``: for image handling / ``.. image::`` replacement
> +
> + * ``.. kernel-figure``: for figure handling / ``.. figure::`` replacement
> +
> + * ``.. kernel-render``: for render markup / a concept to embed *render*
> + markups (or languages). Supported markups (see ``RENDER_MARKUP_EXT``)
> +
> + + ``DOT``: render embedded Graphviz's **DOC**
> + + ``SVG``: render embedded Scalable Vector Graphics (**SVG**)
> + + ... *developable*
> +
> + Used tools:
> +
> + * ``dot(1)``: Graphviz (http://www.graphviz.org). If Graphviz is not
> + available, the DOT language is inserted as literal-block.
> +
> + * SVG to PDF: To generate PDF, you need at least one of this tools:
> +
> + - ``convert(1)``: ImageMagick (https://www.imagemagick.org)
> +
> + List of customizations:
> +
> + * generate PDF from SVG / used by PDF (LaTeX) builder
> +
> + * generate SVG (html-builder) and PDF (latex-builder) from DOT files.
> + DOT: see http://www.graphviz.org/content/dot-language
> +
> + """
> +
> +import os
> +from os import path
> +import subprocess
> +from hashlib import sha1
> +import sys
> +
> +from docutils import nodes
> +from docutils.statemachine import ViewList
> +from docutils.parsers.rst import directives
> +from docutils.parsers.rst.directives import images
> +
> +from sphinx.directives import patches
> +
> +__version__ = '1.0'
> +
> +# simple helper
> +# -------------
> +
> +def mkdir(folder, mode=0o775):
> + if not path.isdir(folder):
> + os.makedirs(folder, mode)
> +
> +# def debug_handle(self, node): # pylint: disable=W0613
> +# from linuxdoc.kernel_doc import CONSOLE
> +# CONSOLE()
> +
> +def pass_handle(self, node): # pylint: disable=W0613
> + pass
> +
> +# setup conversion tools and sphinx extension
> +# -------------------------------------------
> +
> +def setup(app):
> + # image handling
> + app.add_directive("kernel-image", KernelImage)
> + app.add_node(kernel_image,
> + html = (visit_kernel_image, pass_handle),
> + latex = (visit_kernel_image, pass_handle),
> + texinfo = (visit_kernel_image, pass_handle),
> + text = (visit_kernel_image, pass_handle),
> + man = (visit_kernel_image, pass_handle), )
> +
> + # figure handling
> + app.add_directive("kernel-figure", KernelFigure)
> + app.add_node(kernel_figure,
> + html = (visit_kernel_figure, pass_handle),
> + latex = (visit_kernel_figure, pass_handle),
> + texinfo = (visit_kernel_figure, pass_handle),
> + text = (visit_kernel_figure, pass_handle),
> + man = (visit_kernel_figure, pass_handle), )
> +
> + # render handling
> + app.add_directive('kernel-render', KernelRender)
> + app.add_node(kernel_render,
> + html = (visit_kernel_render, pass_handle),
> + latex = (visit_kernel_render, pass_handle),
> + texinfo = (visit_kernel_render, pass_handle),
> + text = (visit_kernel_render, pass_handle),
> + man = (visit_kernel_render, pass_handle), )
> +
> + return dict(
> + version = __version__,
> + parallel_read_safe = True,
> + parallel_write_safe = True
> + )
> +
> +# integrate conversion tools
> +# --------------------------
> +
> +RENDER_MARKUP_EXT = {
> + # The '.ext' must be handled by convert_image(..) function's *in_ext* input.
> + # <name> : <.ext>
> + 'DOT' : '.dot'
> + , 'SVG' : '.svg'
> +}
> +
> +def convert_image(img_node, translator): # pylint: disable=R0912
> + """Convert an image node for the builder.
> +
> + Different builder prefer different image formats, e.g. *latex* builder
> + prefer PDF while *html* builder prefer SVG format for images.
> +
> + This function handles outputs image formats in depence of source the format
> + of the image and the translator's output format. This also means to
> + manipulate/update the *image* dictionary of the builder (``builder.images``)
> +
> + """
> + fname, in_ext = path.splitext(path.basename(img_node['uri']))
> + src_fname = path.join(translator.builder.srcdir, img_node['uri'])
> + src_folder = path.dirname(img_node['uri'])
> + out_dir = translator.builder.outdir
> + dst_fname = None
> +
> + # in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages
> + verbose = translator.builder.app.verbose
> + warn = translator.builder.warn
> +
> + verbose('assert best format for: ' + img_node['uri'])
> +
> + if in_ext == '.dot':
> +
> + # ----------
> + # handle DOT
> + # ----------
> +
> + dst_fname = path.join(out_dir, fname + '.pdf')
> +
> + if translator.builder.format == 'html':
> + dst_fname = path.join(out_dir, src_folder, fname + '.svg')
> + else:
> + # all other builder formats will include DOT as raw
> + with open(src_fname, "r") as dot:
> + data = dot.read()
> + node = nodes.literal_block(data, data)
> + img_node.replace_self(node)
> +
> +
> + elif in_ext == '.svg':
> +
> + # ----------
> + # handle SVG
> + # ----------
> +
> + if translator.builder.format == 'latex':
> + dst_fname = path.join(out_dir, fname + '.pdf')
> +
> + if dst_fname:
> + name = dst_fname[len(out_dir) + 1:]
> + # the builder needs not to copy one more time, so pop it if exists.
> + translator.builder.images.pop(img_node['uri'], None)
> + img_node['uri'] = dst_fname
> + img_node['candidates'] = {'*': dst_fname}
> +
> + mkdir(path.dirname(dst_fname))
> +
> + if in_ext == '.dot':
> + verbose('convert DOT to: {out}/' + name)
> + dot2format(src_fname, dst_fname)
> +
> + elif in_ext == '.svg':
> + verbose('convert SVG to: {out}/' + name)
> + svg2pdf(src_fname, dst_fname)
> +
> +def dot2format(dot_fname, out_fname):
> + """Converts DOT file to ``out_fname`` using ``dot(1)``.
> +
> + * ``dot_fname`` pathname of the input DOT file, including extension ``.dot``
> + * ``out_fname`` pathname of the output file, including format extension
> +
> + The *format extension* depends on the ``dot`` command (see ``man dot``
> + option ``-Txxx``). Normally you will use one of the following extensions:
> +
> + - ``.ps`` for PostScript,
> + - ``.svg`` or ``svgz`` for Structured Vector Graphics,
> + - ``.fig`` for XFIG graphics and
> + - ``.png`` or ``gif`` for common bitmap graphics.
> +
> + """
> + out_format = path.splitext(out_fname)[1][1:]
> + cmd = ['dot', '-T%s' % out_format, dot_fname]
> + exit_code = 42
> + with open(out_fname, "w") as out:
> + p = subprocess.Popen(
> + cmd, stdout = out, stderr = subprocess.PIPE )
> + nil, err = p.communicate()
> +
> + sys.stderr.write(err)
> +
> + exit_code = p.returncode
> + out.flush()
> + return bool(exit_code == 0)
> +
> +def svg2pdf(svg_fname, pdf_fname):
> + """Converts SVG to PDF with CairoSVG or ``convert(1)`` command.
> +
> + Uses ``convert(1)`` from ImageMagick (https://www.imagemagick.org) for
> + conversion. Returns ``True`` on success and ``False`` if an error occurred
> + (e.g. none of the conversion tool is available).
> +
> + * ``svg_fname`` pathname of the input SVG file with extension (``.svg``)
> + * ``pdf_name`` pathname of the output PDF file with extension (``.pdf``)
> +
> + """
> + cmd = [convert_cmd, svg_fname, pdf_fname]
> + p = subprocess.Popen(
> + cmd, stdout = out, stderr = subprocess.PIPE )
> + nil, err = p.communicate()
> +
> + sys.stderr.write(err)
> +
> + exit_code = p.returncode
> + return bool(exit_code == 0)
> +
> +
> +# image handling
> +# ---------------------
> +
> +def visit_kernel_image(self, node): # pylint: disable=W0613
> + """Visitor of the ``kernel_image`` Node.
> +
> + Handles the ``image`` child-node with the ``convert_image(...)``.
> + """
> + img_node = node[0]
> + convert_image(img_node, self)
> +
> +class kernel_image(nodes.General, nodes.Element):
> + """Node for ``kernel-image`` directive."""
> + pass
> +
> +class KernelImage(images.Image):
> + u"""KernelImage directive
> +
> + Earns everything from ``.. image::`` directive, except *remote URI* and
> + *glob* pattern. The KernelImage wraps a image node into a
> + kernel_image node. See ``visit_kernel_image``.
> + """
> +
> + def run(self):
> + env = self.state.document.settings.env
> +
> + uri = self.arguments[0]
> + if uri.endswith('.*') or uri.find('://') != -1:
> + raise self.severe(
> + 'Error in "%s: %s": glob pattern and remote images are not allowed'
> + % (self.name, uri))
> +
> + # Tell sphinx of the dependency
> + env.note_dependency(os.path.abspath(uri))
> +
> + result = images.Image.run(self)
> + if len(result) == 2 or isinstance(result[0], nodes.system_message):
> + return result
> + (image_node,) = result
> + # wrap image node into a kernel_image node / see visitors
> + node = kernel_image('', image_node)
> + return [node]
> +
> +# figure handling
> +# ---------------------
> +
> +def visit_kernel_figure(self, node): # pylint: disable=W0613
> + """Visitor of the ``kernel_figure`` Node.
> +
> + Handles the ``image`` child-node with the ``convert_image(...)``.
> + """
> + img_node = node[0][0]
> + convert_image(img_node, self)
> +
> +class kernel_figure(nodes.General, nodes.Element):
> + """Node for ``kernel-figure`` directive."""
> +
> +class KernelFigure(patches.Figure):
> + u"""KernelImage directive
> +
> + Earns everything from ``.. figure::`` directive, except *remote URI* and
> + *glob* pattern. The KernelFigure wraps a figure node into a kernel_figure
> + node. See ``visit_kernel_figure``.
> + """
> +
> + def run(self):
> + env = self.state.document.settings.env
> +
> + uri = self.arguments[0]
> + if uri.endswith('.*') or uri.find('://') != -1:
> + raise self.severe(
> + 'Error in "%s: %s":'
> + ' glob pattern and remote images are not allowed'
> + % (self.name, uri))
> +
> + # Tell sphinx of the dependency
> + env.note_dependency(os.path.abspath(uri))
> +
> + result = patches.Figure.run(self)
> + if len(result) == 2 or isinstance(result[0], nodes.system_message):
> + return result
> + (figure_node,) = result
> + # wrap figure node into a kernel_figure node / see visitors
> + node = kernel_figure('', figure_node)
> + return [node]
> +
> +
> +# render handling
> +# ---------------------
> +
> +def visit_kernel_render(self, node):
> + """Visitor of the ``kernel_render`` Node.
> +
> + If rendering tools available, save the markup of the ``literal_block`` child
> + node into a file and replace the ``literal_block`` node with a new created
> + ``image`` node, pointing to the saved markup file. Afterwards, handle the
> + image child-node with the ``convert_image(...)``.
> + """
> +
> + verbose = self.builder.app.verbose
> + warn = self.builder.warn
> + srclang = node.get('srclang')
> +
> + verbose('visit kernel-render node lang: "%s"' % (srclang))
> +
> + tmp_ext = RENDER_MARKUP_EXT.get(srclang, None)
> + if tmp_ext is None:
> + warn('kernel-render: "%s" unknow / include raw.' % (srclang))
> + return
> +
> + literal_block = node[0]
> + code = literal_block.astext()
> +
> + if tmp_ext:
> + hashobj = code.encode('utf-8') # str(node.attributes)
> + fname = '%s-%s' % (srclang, sha1(hashobj).hexdigest())
> + tmp_fname = path.join(
> + self.builder.outdir, self.builder.imagedir, fname + tmp_ext)
> +
> + if not path.isfile(tmp_fname):
> + mkdir(path.dirname(tmp_fname))
> + with open(tmp_fname, "w") as out:
> + out.write(code)
> +
> + image_node = nodes.image(node.rawsource, **node.attributes)
> + image_node['uri'] = tmp_fname
> +
> + literal_block.replace_self(image_node)
> + convert_image(image_node, self)
> +
> +
> +class kernel_render(nodes.General, nodes.Inline, nodes.Element):
> + """Node for ``kernel-render`` directive."""
> + pass
> +
> +class KernelRender(patches.Figure):
> + u"""KernelRender directive
> +
> + Render content by external tool. Has all the options known from the
> + *figure* directive, plus option ``caption``. If ``caption`` has a
> + value, a figure node with the *caption* is inserted. If not, a image node is
> + inserted.
> +
> + The KernelRender directive wraps the text of the directive into a
> + literal_block node and wraps it into a kernel_render node. See
> + ``visit_kernel_render``.
> + """
> + has_content = True
> + required_arguments = 1
> + optional_arguments = 0
> + final_argument_whitespace = False
> +
> + # earn options from 'figure'
> + option_spec = patches.Figure.option_spec.copy()
> + option_spec['caption'] = directives.unchanged
> +
> + def run(self):
> + return [self.build_node()]
> +
> + def build_node(self):
> +
> + srclang = self.arguments[0].strip()
> + if srclang not in RENDER_MARKUP_EXT.keys():
> + return [self.state_machine.reporter.warning(
> + 'Unknow source language "%s", use one of: %s.' % (
> + srclang, ",".join(RENDER_MARKUP_EXT.keys())),
> + line=self.lineno)]
> +
> + code = '\n'.join(self.content)
> + if not code.strip():
> + return [self.state_machine.reporter.warning(
> + 'Ignoring "%s" directive without content.' % (
> + self.name),
> + line=self.lineno)]
> +
> + node = kernel_render()
> + node['alt'] = self.options.get('alt','')
> + node['srclang'] = srclang
> + literal_node = nodes.literal_block(code, code)
> + node += literal_node
> +
> + caption = self.options.get('caption')
> + if caption:
> + # parse cation's content
> + parsed = nodes.Element()
> + self.state.nested_parse(
> + ViewList([caption], source=''), self.content_offset, parsed)
> + caption_node = nodes.caption(
> + parsed[0].rawsource, '', *parsed[0].children)
> + caption_node.source = parsed[0].source
> + caption_node.line = parsed[0].line
> +
> + figure_node = nodes.figure('', node)
> + for k,v in self.options.items():
> + figure_node[k] = v
> + figure_node += caption_node
> +
> + node = figure_node
> +
> + return node
> +
> --
> 2.11.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-06 10:12 ` Daniel Vetter
@ 2017-03-06 13:13 ` Markus Heiser
2017-03-06 15:03 ` Daniel Vetter
0 siblings, 1 reply; 56+ messages in thread
From: Markus Heiser @ 2017-03-06 13:13 UTC (permalink / raw)
To: Daniel Vetter
Cc: Jonathan Corbet, linux-doc, Mauro Carvalho Chehab,
DRI Development, Daniel Vetter
Am 06.03.2017 um 11:12 schrieb Daniel Vetter <daniel@ffwll.ch>:
> Just to avoid confusion: Markus&I chatted a bit in private, and he
> volunteered to take over, and fix the few issues that need to be fixed in
> his original patch:
> - drop the convertsvg python fallback, since we need convert anyway
> - relative paths
> - stderr fwd
> - and the small polish I noticed in docs
>
> I think with my blind hacking I botched the job too much :-)
>
> Anyway, still hoping we could land this for 4.12, I really want the
> drm-side doc improvements this provides ...
>
> Thanks, Daniel
FYI: the patch is available:
https://www.mail-archive.com/linux-doc@vger.kernel.org/msg09881.html
-- Markus --
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-06 13:13 ` Markus Heiser
@ 2017-03-06 15:03 ` Daniel Vetter
0 siblings, 0 replies; 56+ messages in thread
From: Daniel Vetter @ 2017-03-06 15:03 UTC (permalink / raw)
To: Markus Heiser
Cc: Jonathan Corbet, linux-doc, Mauro Carvalho Chehab,
DRI Development, Daniel Vetter
On Mon, Mar 06, 2017 at 02:13:00PM +0100, Markus Heiser wrote:
>
> Am 06.03.2017 um 11:12 schrieb Daniel Vetter <daniel@ffwll.ch>:
>
> > Just to avoid confusion: Markus&I chatted a bit in private, and he
> > volunteered to take over, and fix the few issues that need to be fixed in
> > his original patch:
> > - drop the convertsvg python fallback, since we need convert anyway
> > - relative paths
> > - stderr fwd
> > - and the small polish I noticed in docs
> >
> > I think with my blind hacking I botched the job too much :-)
> >
> > Anyway, still hoping we could land this for 4.12, I really want the
> > drm-side doc improvements this provides ...
> >
> > Thanks, Daniel
>
> FYI: the patch is available:
>
> https://www.mail-archive.com/linux-doc@vger.kernel.org/msg09881.html
Ah cool, I'll get right around to testing it.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 1/6] doc: Explain light-handed markup preference a bit better
2017-03-02 15:16 [PATCH 1/6] doc: Explain light-handed markup preference a bit better Daniel Vetter
` (4 preceding siblings ...)
2017-03-02 15:16 ` [PATCH 6/6] drm/doc: atomic overview, with graph Daniel Vetter
@ 2017-03-07 16:40 ` Daniel Vetter
2017-03-07 16:55 ` Jonathan Corbet
5 siblings, 1 reply; 56+ messages in thread
From: Daniel Vetter @ 2017-03-07 16:40 UTC (permalink / raw)
To: linux-doc
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Peter Zijlstra,
Daniel Vetter, Christoph Hellwig, DRI Development, Daniel Vetter
On Thu, Mar 02, 2017 at 04:16:33PM +0100, Daniel Vetter wrote:
> We already had a super-short blurb, but worth extending it I think:
> We're still pretty far away from anything like a consensus, but
> there's clearly a lot of people who prefer an as-light as possible
> approach to converting existing .txt files to .rst. Make sure this is
> properly taken into account and clear.
>
> Motivated by discussions with Peter and Christoph and others.
>
> v2:
> - Mention that existing headings should be kept when converting
> existing .txt files (Mauro).
> - Explain that we prefer :: for quoting code, it's easier on the
> eyes (Mauro).
> - Explain that blindly converting outdated docs is harmful. Motived
> by comments Peter did in our discussion.
>
> v3: Make the explanations around fixed-width quoting more concise
> (Jani).
>
> v4:
> - Rebase onto docs-4.10.
> - Go with the more terse recommendation from Jani, defer to the much
> more detailed conversion guide Mauro is working on for details.
>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: linux-doc@vger.kernel.org
> Cc: Christoph Hellwig <hch@infradead.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Jon, can you pls pick this one up, or want me to resend stand-alone?
Thanks, Daniel
> ---
> Documentation/doc-guide/sphinx.rst | 17 ++++++++++++++++-
> 1 file changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
> index 96fe7ccb2c67..532d65b70500 100644
> --- a/Documentation/doc-guide/sphinx.rst
> +++ b/Documentation/doc-guide/sphinx.rst
> @@ -73,7 +73,16 @@ Specific guidelines for the kernel documentation
>
> Here are some specific guidelines for the kernel documentation:
>
> -* Please don't go overboard with reStructuredText markup. Keep it simple.
> +* Please don't go overboard with reStructuredText markup. Keep it
> + simple. For the most part the documentation should be plain text with
> + just enough consistency in formatting that it can be converted to
> + other formats.
> +
> +* Please keep the formatting changes minimal when converting existing
> + documentation to reStructuredText.
> +
> +* Also update the content, not just the formatting, when converting
> + documentation.
>
> * Please stick to this order of heading adornments:
>
> @@ -103,6 +112,12 @@ Here are some specific guidelines for the kernel documentation:
> the order as encountered."), having the higher levels the same overall makes
> it easier to follow the documents.
>
> +* For inserting fixed width text blocks (for code examples, use case
> + examples, etc.), use ``::`` for anything that doesn't really benefit
> + from syntax highlighting, especially short snippets. Use
> + ``.. code-block:: <language>`` for longer code blocks that benefit
> + from highlighting.
> +
>
> the C domain
> ------------
> --
> 2.11.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 1/6] doc: Explain light-handed markup preference a bit better
2017-03-07 16:40 ` [PATCH 1/6] doc: Explain light-handed markup preference a bit better Daniel Vetter
@ 2017-03-07 16:55 ` Jonathan Corbet
0 siblings, 0 replies; 56+ messages in thread
From: Jonathan Corbet @ 2017-03-07 16:55 UTC (permalink / raw)
To: Daniel Vetter
Cc: Mauro Carvalho Chehab, linux-doc, Peter Zijlstra, Daniel Vetter,
Christoph Hellwig, DRI Development, Daniel Vetter
On Tue, 7 Mar 2017 17:40:35 +0100
Daniel Vetter <daniel@ffwll.ch> wrote:
> Jon, can you pls pick this one up, or want me to resend stand-alone?
I got it; I should get both applied before too long.
jon
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH 6/6] drm/doc: atomic overview, with graph
2017-03-02 15:24 ` Gabriel Krisman Bertazi
@ 2017-03-14 14:14 ` Daniel Vetter
0 siblings, 0 replies; 56+ messages in thread
From: Daniel Vetter @ 2017-03-14 14:14 UTC (permalink / raw)
To: Gabriel Krisman Bertazi
Cc: linux-doc, Daniel Vetter, DRI Development, Laurent Pinchart,
Daniel Vetter
On Thu, Mar 02, 2017 at 12:24:25PM -0300, Gabriel Krisman Bertazi wrote:
> Daniel Vetter <daniel.vetter@ffwll.ch> writes:
>
> > I want to split up a few more things and document some details better
> > (like how exactly to subclass drm_atomic_state). And maybe also split
> > up the helpers a bit per-topic, but this should be a ok-ish start for
> > better atomic overview.
>
> Reviewed-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Jon Corbet just sent me the topic pull, and I applied patches 3-6 in this
series to drm-misc-next now.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 56+ messages in thread
end of thread, other threads:[~2017-03-14 14:14 UTC | newest]
Thread overview: 56+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-03-02 15:16 [PATCH 1/6] doc: Explain light-handed markup preference a bit better Daniel Vetter
2017-03-02 15:16 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
2017-03-02 15:20 ` Laurent Pinchart
2017-03-02 15:40 ` [PATCH] " Daniel Vetter
2017-03-02 18:06 ` Mauro Carvalho Chehab
2017-03-02 19:06 ` Markus Heiser
2017-03-02 19:34 ` Mauro Carvalho Chehab
2017-03-02 19:56 ` Mauro Carvalho Chehab
2017-03-02 21:16 ` Markus Heiser
2017-03-02 21:22 ` Mauro Carvalho Chehab
2017-03-02 19:52 ` Daniel Vetter
2017-03-02 20:04 ` Mauro Carvalho Chehab
2017-03-02 20:54 ` Mauro Carvalho Chehab
2017-03-02 21:29 ` Mauro Carvalho Chehab
2017-03-02 21:36 ` Mauro Carvalho Chehab
2017-03-02 21:47 ` Mauro Carvalho Chehab
2017-03-02 21:54 ` Markus Heiser
2017-03-03 8:54 ` Daniel Vetter
2017-03-02 22:15 ` [PATCH 2/6] " Markus Heiser
2017-03-02 23:19 ` Mauro Carvalho Chehab
2017-03-02 15:16 ` [PATCH 3/6] drm/doc: Add KMS overview graphs Daniel Vetter
2017-03-02 15:16 ` [PATCH 4/6] drm/doc: Consistent kerneldoc include order Daniel Vetter
2017-03-02 15:16 ` [PATCH 5/6] drm/doc: diagram for mode objects and properties Daniel Vetter
2017-03-02 15:16 ` [PATCH 6/6] drm/doc: atomic overview, with graph Daniel Vetter
2017-03-02 15:24 ` Gabriel Krisman Bertazi
2017-03-14 14:14 ` Daniel Vetter
2017-03-07 16:40 ` [PATCH 1/6] doc: Explain light-handed markup preference a bit better Daniel Vetter
2017-03-07 16:55 ` Jonathan Corbet
-- strict thread matches above, loose matches on Subject: below --
2017-02-28 17:13 [PATCH 0/6] more kernel-doc patches Daniel Vetter
2017-02-28 17:13 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
2017-03-01 15:56 ` Gabriel Krisman Bertazi
2017-03-02 7:12 ` Daniel Vetter
2017-03-02 8:23 ` Markus Heiser
2017-03-02 12:26 ` Laurent Pinchart
2017-03-02 13:54 ` Daniel Vetter
2017-03-02 14:14 ` Laurent Pinchart
2017-03-02 14:58 ` Markus Heiser
2017-03-02 15:11 ` Daniel Vetter
2017-03-02 15:17 ` Laurent Pinchart
2017-03-02 15:21 ` Markus Heiser
2017-03-02 15:45 ` Mauro Carvalho Chehab
2017-03-02 15:49 ` Mauro Carvalho Chehab
2017-03-02 16:13 ` Markus Heiser
2017-03-02 16:29 ` Mauro Carvalho Chehab
2017-03-02 18:16 ` Markus Heiser
2017-03-02 18:20 ` Jonathan Corbet
2017-03-02 18:47 ` Markus Heiser
2017-03-02 19:09 ` Mauro Carvalho Chehab
2017-03-02 20:16 ` Markus Heiser
2017-03-02 20:28 ` Mauro Carvalho Chehab
2017-03-02 15:22 ` Jonathan Corbet
2017-03-02 15:53 ` Daniel Vetter
2017-03-02 16:01 ` Mauro Carvalho Chehab
2017-03-02 19:49 ` Daniel Vetter
2017-03-06 10:12 ` Daniel Vetter
2017-03-06 13:13 ` Markus Heiser
2017-03-06 15:03 ` Daniel Vetter
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).