Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab <mchehab@s-opensource.com>]

Em Thu, 2 Mar 2017 17:04:01 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:

> Em Thu, 2 Mar 2017 20:52:59 +0100
> Daniel Vetter <daniel@ffwll.ch> escreveu:
> 
> > On Thu, Mar 02, 2017 at 08:06:39PM +0100, Markus Heiser wrote:  
> > > Hi Mauro,
> > >     
> > > > Tested here with the enclosed patch.    
> > > 
> > > great, big step forward making /media/Makefile smaller ...  thanks a lot!!!!
> > >     
> > > > It crashed:
> > > > Exception occurred:
> > > >  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > >    sys.stderr.write(err)
> > > > TypeError: write() argument must be str, not bytes
> > > > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> > > > Please also report this if it was a user error, so that a better error message can be provided next time.
> > > > A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> > > > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > > > make[1]: *** [htmldocs] Error 1
> > > > Makefile:1450: recipe for target 'htmldocs' failed
> > > > make: *** [htmldocs] Error 2
> > > > 
> > > > Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.    
> > > 
> > > I guess that the dot command writes something to stderr. This is captured 
> > > by the extension and printed to stderr ...
> > > 
> > > +def dot2format(dot_fname, out_fname):
> > > ...
> > > +    exit_code = 42
> > > +    with open(out_fname, "w") as out:
> > > +        p = subprocess.Popen(
> > > +            cmd, stdout = out, stderr = subprocess.PIPE )
> > > +        nil, err = p.communicate()
> > > +
> > > +        sys.stderr.write(err)
> > > +
> > > +        exit_code = p.returncode
> > > +        out.flush()
> > > +    return bool(exit_code == 0)
> > >     
> > > >  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > >    sys.stderr.write(err)
> > > > TypeError: write() argument must be str, not bytes    
> > > 
> > > Do we need this stderr output? For a first test, uncomment the 
> > > "sys.stderr.write(err)“ in line 222. Or, if we really need the
> > > stderr, try:
> > > 
> > > -        sys.stderr.write(err)
> > > +        sys.stderr.write(str(err))
> > > 
> > > I this fixes, there is another "sys.stderr.write(err)" in 
> > > func svg2pdf(..) which should also fixed ….
> > >  
> > > +def svg2pdf(svg_fname, pdf_fname):
> > > ...
> > > +    cmd = [convert_cmd, svg_fname, pdf_fname]
> > > +    p = subprocess.Popen(
> > > +        cmd, stdout = out, stderr = subprocess.PIPE )
> > > +    nil, err = p.communicate()
> > > +
> > > -        sys.stderr.write(err)
> > > +        sys.stderr.write(str(err))
> > > +
> > > +    exit_code = p.returncode
> > > +    return bool(exit_code == 0)    
> > 
> > Yes, I very much want stderr to be forward.  
> 
> Yes, error report is required.
> 
> > Without that you don't see
> > error output from dot or convert, and that makes it impossible to debug
> > anything. If I want a direct forwarding of the bytes, how should I do this
> > in python? Capturing stderr and then re-dumping it is kinda silly ...  
> 
> Markus or some other Python programmer may help us with that.
> 
> > Note that I copied this pattern from the kernel-doc extension, seems to
> > have worked there.  
> 
> Maybe it is broken there too then, or this is another python API
> that changed over time. Here, I'm testing with:
> 	python3-3.5.2-4.fc25.x86_64
> 
> From here:
> 	https://docs.python.org/2/library/subprocess.html
> 
> communicate returns a tuple.
> 
> I used repr(p.communicate()[0], on the code snippet I sent, 
> as I copied from an example that I found at:
> 	https://stackoverflow.com/questions/33295284/python-subprocess-popen-write-to-stderr
> 
> Thanks,
> Mauro

I suspect that the actual fixup would be something like:


commit ddf93ae81af10bb43caa651b9acd355f1d74cebe
Author: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Date:   Thu Mar 2 17:11:47 2017 -0300

    kfigure.py fixups
    
    Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 32eab0f4cfba..19389cb34d6f 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -217,11 +217,13 @@ def dot2format(dot_fname, out_fname):
     with open(out_fname, "w") as out:
         p = subprocess.Popen(
             cmd, stdout = out, stderr = subprocess.PIPE )
-        nil, err = p.communicate()
-
-        sys.stderr.write(err)
+        err = p.communicate()
 
         exit_code = p.returncode
+
+        if exit_code == 0:
+            sys.stderr.write("Error:" + repr(err[0]) + "\n")
+
         out.flush()
     return bool(exit_code == 0)
 
@@ -239,11 +241,13 @@ def svg2pdf(svg_fname, pdf_fname):
     cmd = [convert_cmd, svg_fname, pdf_fname]
     p = subprocess.Popen(
         cmd, stdout = out, stderr = subprocess.PIPE )
-    nil, err = p.communicate()
-
-    sys.stderr.write(err)
+    err = p.communicate()
 
     exit_code = p.returncode
+
+    if exit_code == 0:
+        sys.stderr.write("Error:" + repr(err[0]) + "\n")
+
     return bool(exit_code == 0)
 
 



Thanks,
Mauro