Re: [PATCH v4 2/2] meson: Deserialize the man pages and html builds

[Fabiano Rosas <farosas@suse.de>]

Peter Maydell <peter.maydell@linaro.org> writes:

> On Wed, 3 May 2023 at 21:39, Fabiano Rosas <farosas@suse.de> wrote:
>>
>> For the documentation builds (man pages & manual), we let Sphinx
>> decide when to rebuild and use a depfile to know when to trigger the
>> make target.
>>
>> We currently use a trick of having the man pages custom_target take as
>> input the html pages custom_target object, which causes both targets
>> to be executed if one of the dependencies has changed. However, having
>> this at the custom_target level means that the two builds are
>> effectively serialized.
>>
>> We can eliminate the dependency between the targets by adding a second
>> depfile for the man pages build, allowing them to be parallelized by
>> ninja while keeping sphinx in charge of deciding when to rebuild.
>>
>> Since they can now run in parallel, separate the Sphinx cache
>> directory of the two builds. We need this not only for data
>> consistency but also because Sphinx writes builder-dependent
>> environment information to the cache directory (see notes under
>> smartquotes_excludes in sphinx docs [1]).
>
> The sphinx-build manpage disagrees about that last part.
> https://www.sphinx-doc.org/en/master/man/sphinx-build.html
> says about -d:
> "with this option you can select a different cache directory
>  (the doctrees can be shared between all builders)"
>

The issue I had is that sphinx by default uses smart quotes for html
builders, but not for man builders. But whichever builder runs first
gets to set the smartquotes option and that sticks for the next
builder. That causes our man pages to come up with fancy curly quotes
instead of ' which is probably not an issue, but I didn't want to
produce different output from what we already have today.

I ended up conflating the cache directory (-d) with the environment
(-E), so it is possible that we can reuse the cache but not the
environment (where I assume the smartquotes option is stored). Well, I
better go read the sphinx code and figure that out.

> If we don't share the cache directory, presumably Sphinx
> now ends up parsing all the input files twice, once per
> builder, rather than being able to share them?
>

Yes, but having it run in parallel from the ninja level is still
faster. Of course, if we could reuse the cache, this could potentially
be even faster. I'll try to determine if it is really safe to do so.