[Qemu-devel] Sphinx and docs/index.rst: dead code?

[Qemu-devel] Sphinx and docs/index.rst: dead code?

[John Snow]

Does anything actually use this file? It doesn't appear to be used for
generating the HTML manuals.

It looks like we might use it for latex, man and texinfo output from
sphinx judging by docs/conf.py, but I don't think we actually use sphinx
to generate such output, so I think this is all dead code.

Am I mistaken?

--js

Re: [Qemu-devel] Sphinx and docs/index.rst: dead code?

[Peter Maydell]

On Thu, 25 Jul 2019 at 00:22, John Snow <jsnow@redhat.com> wrote:
>
> Does anything actually use this file? It doesn't appear to be used for
> generating the HTML manuals.

It's there for if you want to do a "build all the manuals into
a single document" -- see the comments at the top of docs/conf.py.
Basically this is there because for QEMU's own purposes we want
to build several separate manuals (devel, interop, etc) so we
can install for the user only the ones that are user-facing
(ie "not devel"). But it seemed to me at the time to be worth
also supporting the "build a full thing" for the benefit of
sites like readthedocs. Currently the only way to use it is
if you hand-run an appropriate sphinx-build command.
(The fact that we will need to do some running of other
commands to autogenerate .rst input from .hx files might mean
that it's not really possible to support this sort of "third party
site" docs setup in the end; but for the moment we retain the
option to do this.)

Doing a top-level build will also complain about a couple
random .rst files in docs/ not being in the toc tree --
we need to fix this at some point anyway by putting that
documentation into its proper place in the manual set.

> It looks like we might use it for latex, man and texinfo output from
> sphinx judging by docs/conf.py, but I don't think we actually use sphinx
> to generate such output, so I think this is all dead code.

We will generate the manpage output in the same way we do
HTML, by invoking sphinx-build on the different manual
directories -- nothing in-tree does this today but this
patch:
https://patchew.org/QEMU/20190618104718.25433-1-peter.maydell@linaro.org/
does the conversion of the qemu-ga docs/manpage and should
be ready to go in once 4.1 is out the door.

thanks
-- PMM

Re: [Qemu-devel] Sphinx and docs/index.rst: dead code?

[John Snow]

On 7/25/19 5:08 AM, Peter Maydell wrote:
> On Thu, 25 Jul 2019 at 00:22, John Snow <jsnow@redhat.com> wrote:
>>
>> Does anything actually use this file? It doesn't appear to be used for
>> generating the HTML manuals.
> 
> It's there for if you want to do a "build all the manuals into
> a single document" -- see the comments at the top of docs/conf.py.
> Basically this is there because for QEMU's own purposes we want
> to build several separate manuals (devel, interop, etc) so we
> can install for the user only the ones that are user-facing
> (ie "not devel"). But it seemed to me at the time to be worth
> also supporting the "build a full thing" for the benefit of
> sites like readthedocs. Currently the only way to use it is
> if you hand-run an appropriate sphinx-build command.

Yup; I think a single point of entry would be nice -- I think we need to
start hosting our sphinx documentation because it's confusing that we
have both the traditional manual (hosted by Stefan Weil) and this newer
one that isn't available anywhere.

The interop manual in particular is crucial to get hosted.

> (The fact that we will need to do some running of other
> commands to autogenerate .rst input from .hx files might mean
> that it's not really possible to support this sort of "third party
> site" docs setup in the end; but for the moment we retain the
> option to do this.)
> 

OK, sure -- just wasn't sure if the feature was coming or going. I'll
consider it "coming" then.

We could perhaps formalize this as follows:

- index.rst, which is an "absolutely everything included" single point
of entry manual for developers and contributors,

- user.rst, which could be a single point of entry for end users, to be
bundled in distro packaging.

Then we can build either a single-target dev manual, a single-target
user manual, or any of the individual component manuals. I'm not sure
which should be the default, though.

> Doing a top-level build will also complain about a couple
> random .rst files in docs/ not being in the toc tree --
> we need to fix this at some point anyway by putting that
> documentation into its proper place in the manual set.
> 
>> It looks like we might use it for latex, man and texinfo output from
>> sphinx judging by docs/conf.py, but I don't think we actually use sphinx
>> to generate such output, so I think this is all dead code.
> 
> We will generate the manpage output in the same way we do
> HTML, by invoking sphinx-build on the different manual
> directories -- nothing in-tree does this today but this
> patch:
> https://patchew.org/QEMU/20190618104718.25433-1-peter.maydell@linaro.org/
> does the conversion of the qemu-ga docs/manpage and should
> be ready to go in once 4.1 is out the door.
> 
> thanks
> -- PMM
> 

Nice!

Thanks for the info.

--js

Re: [Qemu-devel] Sphinx and docs/index.rst: dead code?

[Peter Maydell]

On Thu, 25 Jul 2019 at 17:34, John Snow <jsnow@redhat.com> wrote:
> Yup; I think a single point of entry would be nice -- I think we need to
> start hosting our sphinx documentation because it's confusing that we
> have both the traditional manual (hosted by Stefan Weil) and this newer
> one that isn't available anywhere.
>
> The interop manual in particular is crucial to get hosted.

Yes, this would be a good thing.

> We could perhaps formalize this as follows:
>
> - index.rst, which is an "absolutely everything included" single point
> of entry manual for developers and contributors,
>
> - user.rst, which could be a single point of entry for end users, to be
> bundled in distro packaging.

This means you'll end up building 90% of our documentation twice,
which is something I was trying to avoid with the current setup.

It occurs to me that we don't necessarily need the 'top level'
page to be generated by Sphinx -- we could just ship an index.html
which has helpful links to the individual manuals.

(https://wiki.qemu.org/Features/Documentation is the current
plan and lists the various manuals we'll end up with. 'user'
in that plan means the documentation for the user-mode emulation.)

thanks
-- PMM

Re: [Qemu-devel] Sphinx and docs/index.rst: dead code?

[John Snow]

On 7/25/19 12:42 PM, Peter Maydell wrote:
> On Thu, 25 Jul 2019 at 17:34, John Snow <jsnow@redhat.com> wrote:
>> Yup; I think a single point of entry would be nice -- I think we need to
>> start hosting our sphinx documentation because it's confusing that we
>> have both the traditional manual (hosted by Stefan Weil) and this newer
>> one that isn't available anywhere.
>>
>> The interop manual in particular is crucial to get hosted.
> 
> Yes, this would be a good thing.
> 
>> We could perhaps formalize this as follows:
>>
>> - index.rst, which is an "absolutely everything included" single point
>> of entry manual for developers and contributors,
>>
>> - user.rst, which could be a single point of entry for end users, to be
>> bundled in distro packaging.
> 
> This means you'll end up building 90% of our documentation twice,
> which is something I was trying to avoid with the current setup.
> 

Why? Wouldn't it suffice to build just one of the top-level docs just once?

(I guess if you later decided to build the other top-level doc later it
would duplicate the work, but is that the usual case?)

> It occurs to me that we don't necessarily need the 'top level'
> page to be generated by Sphinx -- we could just ship an index.html
> which has helpful links to the individual manuals.
> 

True; we can leave it as a manual process and check the build artifact
into the repo if we want. [We likely ought to leave the source in the
tree in that case though, if we want to update the theming and other stuff.]

> (https://wiki.qemu.org/Features/Documentation is the current
> plan and lists the various manuals we'll end up with. 'user'
> in that plan means the documentation for the user-mode emulation.)
> 

Ah, whoops -- sorry for the namespace collision. I'll call it
'user-manual' or something instead if I talk about this in the future.

Thanks for the wiki page.

> thanks
> -- PMM
> 

Re: [Qemu-devel] Sphinx and docs/index.rst: dead code?

[Peter Maydell]

On Thu, 25 Jul 2019 at 18:26, John Snow <jsnow@redhat.com> wrote:
> On 7/25/19 12:42 PM, Peter Maydell wrote:
> > This means you'll end up building 90% of our documentation twice,
> > which is something I was trying to avoid with the current setup.
> >
>
> Why? Wouldn't it suffice to build just one of the top-level docs just once?
>
> (I guess if you later decided to build the other top-level doc later it
> would duplicate the work, but is that the usual case?)

"make" needs to build the including-for-developers set so
you can refer to it in tree. But what we want for "make install"
is the only-for-users set. We don't want to build stuff during
"make install"; so "make" needs to have already built that set too.

The underlying problem here is that Sphinx doesn't really like
the idea of building documentation but only installing a subset.
So you get to pick which downsides you dislike least when
choosing the workaround. The current setup isn't necessarily
ideal but it seemed good-enough and to more or less match
the five-manuals approach Paolo originally outlined. We can
certainly change it if there's a less-awkward way to do things.

thanks
-- PMM