Re: [PATCH 0/2] docs: Programmatically render MAINTAINERS into ReST

[Jonathan Corbet <corbet@lwn.net>]

On Tue, 1 Oct 2019 12:09:30 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:

> >   Sphinx parallel build error:
> >   UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128)
> > 
> > For extra fun, the build process simply hangs, requiring a ^C to blow it
> > away.  You've managed to get new behavior out of Sphinx that I've not seen
> > before, congratulations :)
> > 
> > This almost certainly has to do with the fact that I'm (intentionally)
> > running the Python2 Sphinx build here.  Something's not doing unicode that
> > should be.
> > 
> > I would suggest that we might just want to repair this before merging this
> > feature.  Either that, or we bite the bullet and deprecate the use of
> > Python 2 entirely - something that's probably not too far into our future
> > regardless.  Anybody have thoughts on that matter?  
> 
> I'm almost sure we got this already. If I'm not mistaken, the solution
> is to add the encoding line after shebang.

As mentioned before, that's not it.  The problem is that we're feeding
UTF8 into Sphinx as an ordinary str, then sphinx is trying to decode it
as ASCII. The attached hack makes things work.

Kees, I can either just keep this fix (breaking bisectability of the docs
build :) or you can send a new version - up to you.

Thanks,

jon

From c32bfbb07a7840662ba3b367c61b7f2946028b27 Mon Sep 17 00:00:00 2001
From: Jonathan Corbet <corbet@lwn.net>
Date: Tue, 1 Oct 2019 11:26:20 -0600
Subject: [PATCH] docs: make maintainers_include work with Python 2

The MAINTAINERS file contains UTF-8 data, so Python 2 code has to be
explicit about moving it into the Unicode realm.  Explicitly decode all
data from the file as soon as we read it.

This hack can go away once we deprecate Python 2 support.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 Documentation/sphinx/maintainers_include.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Documentation/sphinx/maintainers_include.py b/Documentation/sphinx/maintainers_include.py
index de62dd3dabd5..705ba9fd5336 100755
--- a/Documentation/sphinx/maintainers_include.py
+++ b/Documentation/sphinx/maintainers_include.py
@@ -19,6 +19,7 @@ u"""
 
 import re
 import os.path
+import sys
 
 from docutils import statemachine
 from docutils.utils.error_reporting import ErrorString
@@ -60,6 +61,8 @@ class MaintainersInclude(Include):
         field_content = ""
 
         for line in open(path):
+            if sys.version_info.major == 2:
+                line = unicode(line, 'utf-8')
             # Have we reached the end of the preformatted Descriptions text?
             if descriptions and line.startswith('Maintainers'):
                 descriptions = False
-- 
2.21.0