[Qemu-devel] QMP Example Formatting in ReST, Sphinx, and Pygments (was: Re: [Bug] Docs build fails at interop.rst)

[John Snow <jsnow@redhat.com>]

On 5/20/19 12:37 PM, John Snow wrote:
> 
> 
> On 5/20/19 7:30 AM, Aarushi Mehta wrote:
>> https://paste.fedoraproject.org/paste/kOPx4jhtUli---TmxSLrlw
>> running python3-sphinx-2.0.1-1.fc31.noarch on Fedora release 31
>> (Rawhide)
>>
>> uname - a
>> Linux iouring 5.1.0-0.rc6.git3.1.fc31.x86_64 #1 SMP Thu Apr 25 14:25:32
>> UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
>>
>> Reverting commmit 90edef80a0852cf8a3d2668898ee40e8970e431
>> allows for the build to occur
>>
>> Regards
>> Aarushi Mehta
>>
>>
> 
> Ah, dang. The blocks aren't strictly conforming json, but the version I
> tested this under didn't seem to care. Your version is much newer. (I
> was using 1.7 as provided by Fedora 29.)
> 
> For now, try reverting 9e5b6cb87db66dfb606604fe6cf40e5ddf1ef0e7 instead,
> which should at least turn off the "warnings as errors" option, but I
> don't think that reverting -n will turn off this warning.
> 
> I'll try to get ahold of this newer version and see if I can't fix it
> more appropriately.
> 
> --js
> 

...Sigh, okay.

So, I am still not actually sure what changed from pygments 2.2 and
sphinx 1.7 to pygments 2.4 and sphinx 2.0.1, but it appears as if Sphinx
by default always tries to do add a filter to the pygments lexer that
raises an error on highlighting failure, instead of the default behavior
which is to just highlight those errors in the output. There is no
option to Sphinx that I am aware of to retain this lexing behavior.
(Effectively, it's strict or nothing.)

This approach, apparently, is broken in Sphinx 1.7/Pygments 2.2, so the
build works with our malformed json.

There are a few options:

1. Update conf.py to ignore these warnings (and all future lexing
errors), and settle for the fact that there will be no QMP highlighting
wherever we use the directionality indicators ('->', '<-').

2. Update bitmaps.rst to remove the directionality indicators.

3. Update bitmaps.rst to format the QMP blocks as raw text instead of JSON.

4. Update bitmaps.rst to remove the "json" specification from the code
block. This will cause sphinx to "guess" the formatting, and the
pygments guesser will decide it's Python3.

This will parse well enough, but will mis-highlight 'true' and 'false'
which are not python keywords. This approach may break in the future if
the Python3 lexer is upgraded to be stricter (because '->' and '<-' are
still invalid), and leaves us at the mercy of both the guesser and the
lexer.

I'm not actually sure what I dislike the least; I think I dislike #1 the
most. #4 gets us most of what we want but is perhaps porcelain.

I suspect if we attempt to move more of our documentation to ReST and
Sphinx that we will need to answer for ourselves how we intend to
document QMP code flow examples.

--js