[PATCH 0/4] Rewrite the top-level index.rst

[PATCH 0/4] Rewrite the top-level index.rst

[Jonathan Corbet]

The top-level index.rst file is the entry point for the kernel's
documentation, especially for readers of the HTML output.  It is currently
a mess containing everything we thought to throw in there.  Firefox says it
would require 26 pages of paper to print it.  That is not a user-friendly
introduction.

This series aims to improve our documentation entry point with a focus on
rewriting index.rst.  The result is, IMO, simpler and more approachable.
For anybody who wants to see the rendered results without building the
docs, have a look at:

  https://static.lwn.net/kerneldoc/

Those pages are rendered with the "Alabaster" theme, which pays attention
to the html_sidebar directive.  I am not proposing a switch to that theme
(I just picked it at random), but I do think we should reconsider the
default theme at some point.

This is only a beginning; I think this kind of organizational effort has to
be pushed down into the lower layers of the docs tree itself.  But one has
to start somewhere.


Jonathan Corbet (4):
  docs: promote the title of process/index.html
  docs: Rewrite the front page
  docs: reconfigure the HTML left column
  docs: remove some index.rst cruft

 Documentation/conf.py            |   3 +-
 Documentation/index.rst          | 157 +++++++++++--------------------
 Documentation/process/index.rst  |   1 +
 Documentation/subsystem-apis.rst |  56 +++++++++++
 4 files changed, 116 insertions(+), 101 deletions(-)
 create mode 100644 Documentation/subsystem-apis.rst

-- 
2.37.2

[PATCH 1/4] docs: promote the title of process/index.html

[Jonathan Corbet]

...otherwise Sphinx won't cooperate when trying to list it explicitly in
the top-level index.rst file

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 Documentation/process/index.rst | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Documentation/process/index.rst b/Documentation/process/index.rst
index 2ba2a1582bbe..d4b6217472b0 100644
--- a/Documentation/process/index.rst
+++ b/Documentation/process/index.rst
@@ -5,6 +5,7 @@
 
 .. _process_index:
 
+=============================================
 Working with the kernel development community
 =============================================
 
-- 
2.37.2

[PATCH 2/4] docs: Rewrite the front page

[Jonathan Corbet]

The front page is the entry point to the documentation, especially for
people who read it online.  It's a big mess of everything we could think to
toss into it.  Rewrite the page with an eye toward simplicity and making it
easy for readers to get going toward what they really want to find.

This is only a beginning, but it makes our docs more approachable than
before.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 Documentation/index.rst          | 151 ++++++++++++-------------------
 Documentation/subsystem-apis.rst |  56 ++++++++++++
 2 files changed, 113 insertions(+), 94 deletions(-)
 create mode 100644 Documentation/subsystem-apis.rst

diff --git a/Documentation/index.rst b/Documentation/index.rst
index 4737c18c97ff..428cfd8e369b 100644
--- a/Documentation/index.rst
+++ b/Documentation/index.rst
@@ -18,14 +18,52 @@ documents into a coherent whole.  Please note that improvements to the
 documentation are welcome; join the linux-doc list at vger.kernel.org if
 you want to help out.
 
-Licensing documentation
------------------------
+Working with the development community
+--------------------------------------
 
-The following describes the license of the Linux kernel source code
-(GPLv2), how to properly mark the license of individual files in the source
-tree, as well as links to the full license text.
+The essential guides for interacting with the kernel's development
+community and getting your work upstream.
+
+.. toctree::
+   :maxdepth: 1
+
+   process/development-process
+   process/submitting-patches
+   Code of conduct <process/code-of-conduct>
+   maintainer/index
+   All development-process docs <process/index>
+
+
+Internal API manuals
+--------------------
+
+Manuals for use by developers working to interface with the rest of the
+kernel.
+
+.. toctree::
+   :maxdepth: 1
+
+   core-api/index
+   driver-api/index
+   subsystem-apis
+   Locking in the kernel <locking/index>
+
+Development tools and processes
+-------------------------------
+
+Various other manuals with useful information for all kernel developers.
+
+.. toctree::
+   :maxdepth: 1
+
+   process/license-rules
+   doc-guide/index
+   dev-tools/index
+   kernel-hacking/index
+   trace/index
+   fault-injection/index
+   livepatch/index
 
-* :ref:`kernel_licensing`
 
 User-oriented documentation
 ---------------------------
@@ -34,21 +72,10 @@ The following manuals are written for *users* of the kernel — those who are
 trying to get it to work optimally on a given system.
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
 
    admin-guide/index
-   kbuild/index
-
-Firmware-related documentation
-------------------------------
-The following holds information on the kernel's expectations regarding the
-platform firmwares.
-
-.. toctree::
-   :maxdepth: 2
-
-   firmware-guide/index
-   devicetree/index
+   The kernel build system <kbuild/index>
 
 Application-developer documentation
 -----------------------------------
@@ -57,92 +84,28 @@ The user-space API manual gathers together documents describing aspects of
 the kernel interface as seen by application developers.
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
 
    userspace-api/index
 
 
-Introduction to kernel development
-----------------------------------
-
-These manuals contain overall information about how to develop the kernel.
-The kernel community is quite large, with thousands of developers
-contributing over the course of a year.  As with any large community,
-knowing how things are done will make the process of getting your changes
-merged much easier.
+Firmware-related documentation
+------------------------------
+The following holds information on the kernel's expectations regarding the
+platform firmwares.
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
 
-   process/index
-   dev-tools/index
-   doc-guide/index
-   kernel-hacking/index
-   trace/index
-   maintainer/index
-   fault-injection/index
-   livepatch/index
-
-
-Kernel API documentation
-------------------------
-
-These books get into the details of how specific kernel subsystems work
-from the point of view of a kernel developer.  Much of the information here
-is taken directly from the kernel source, with supplemental material added
-as needed (or at least as we managed to add it — probably *not* all that is
-needed).
-
-.. toctree::
-   :maxdepth: 2
+   firmware-guide/index
+   devicetree/index
 
-   driver-api/index
-   core-api/index
-   locking/index
-   accounting/index
-   block/index
-   cdrom/index
-   cpu-freq/index
-   fb/index
-   fpga/index
-   hid/index
-   i2c/index
-   iio/index
-   isdn/index
-   infiniband/index
-   leds/index
-   netlabel/index
-   networking/index
-   pcmcia/index
-   power/index
-   target/index
-   timers/index
-   spi/index
-   w1/index
-   watchdog/index
-   virt/index
-   input/index
-   hwmon/index
-   gpu/index
-   security/index
-   sound/index
-   crypto/index
-   filesystems/index
-   mm/index
-   bpf/index
-   usb/index
-   PCI/index
-   scsi/index
-   misc-devices/index
-   scheduler/index
-   mhi/index
-   peci/index
 
 Architecture-agnostic documentation
 -----------------------------------
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
 
    asm-annotations
 
@@ -150,7 +113,7 @@ Architecture-specific documentation
 -----------------------------------
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
 
    arch
 
@@ -163,7 +126,7 @@ of the documentation body, or may require some adjustments and/or conversion
 to ReStructured Text format, or are simply too old.
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
 
    tools/index
    staging/index
diff --git a/Documentation/subsystem-apis.rst b/Documentation/subsystem-apis.rst
new file mode 100644
index 000000000000..f5d8bcc023c8
--- /dev/null
+++ b/Documentation/subsystem-apis.rst
@@ -0,0 +1,56 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==================================
+Kernel subsystem API documentation
+==================================
+
+These books get into the details of how specific kernel subsystems work
+from the point of view of a kernel developer.  Much of the information here
+is taken directly from the kernel source, with supplemental material added
+as needed (or at least as we managed to add it — probably *not* all that is
+needed).
+
+.. toctree::
+   :maxdepth: 1
+
+   driver-api/index
+   core-api/index
+   locking/index
+   accounting/index
+   block/index
+   cdrom/index
+   cpu-freq/index
+   fb/index
+   fpga/index
+   hid/index
+   i2c/index
+   iio/index
+   isdn/index
+   infiniband/index
+   leds/index
+   netlabel/index
+   networking/index
+   pcmcia/index
+   power/index
+   target/index
+   timers/index
+   spi/index
+   w1/index
+   watchdog/index
+   virt/index
+   input/index
+   hwmon/index
+   gpu/index
+   security/index
+   sound/index
+   crypto/index
+   filesystems/index
+   mm/index
+   bpf/index
+   usb/index
+   PCI/index
+   scsi/index
+   misc-devices/index
+   scheduler/index
+   mhi/index
+   peci/index
-- 
2.37.2

Re: [PATCH 2/4] docs: Rewrite the front page

[Jani Nikula]

On Thu, 01 Sep 2022, Jonathan Corbet <corbet@lwn.net> wrote:
> The front page is the entry point to the documentation, especially for
> people who read it online.  It's a big mess of everything we could think to
> toss into it.  Rewrite the page with an eye toward simplicity and making it
> easy for readers to get going toward what they really want to find.
>
> This is only a beginning, but it makes our docs more approachable than
> before.

\o/

> diff --git a/Documentation/subsystem-apis.rst b/Documentation/subsystem-apis.rst
> new file mode 100644
> index 000000000000..f5d8bcc023c8
> --- /dev/null
> +++ b/Documentation/subsystem-apis.rst
> @@ -0,0 +1,56 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +==================================
> +Kernel subsystem API documentation
> +==================================

How about making this "Kernel subsystem documentation"?

For example the GPU documentation is so much more than just API
documentation. And the paragraph below is more generic too, "how
specific subsystems work".

BR,
Jani.

> +These books get into the details of how specific kernel subsystems work
> +from the point of view of a kernel developer.  Much of the information here
> +is taken directly from the kernel source, with supplemental material added
> +as needed (or at least as we managed to add it — probably *not* all that is
> +needed).

-- 
Jani Nikula, Intel Open Source Graphics Center

[PATCH 3/4] docs: reconfigure the HTML left column

[Jonathan Corbet]

Use the html_sidebars directive to get a more useful set of links in the
left column.

Unfortunately, this is a no-op with the default RTD theme, but others
observe it.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 Documentation/conf.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/conf.py b/Documentation/conf.py
index 78dd6d1e7b88..22c9d4df1967 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -370,7 +370,8 @@ html_static_path = ['sphinx-static']
 html_use_smartypants = False
 
 # Custom sidebar templates, maps document names to template names.
-#html_sidebars = {}
+# Note that the RTD theme ignores this
+html_sidebars = { '**': ['searchbox.html', 'localtoc.html', 'sourcelink.html']}
 
 # Additional templates that should be rendered to pages, maps page names to
 # template names.
-- 
2.37.2

Re: [PATCH 3/4] docs: reconfigure the HTML left column

[Jani Nikula]

On Thu, 01 Sep 2022, Jonathan Corbet <corbet@lwn.net> wrote:
> Use the html_sidebars directive to get a more useful set of links in the
> left column.
>
> Unfortunately, this is a no-op with the default RTD theme, but others
> observe it.

I regret picking the RTD theme way back when, on more of a whim than
anything else, and claiming we could change it later on. :(

BR,
Jani.

>
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
> ---
>  Documentation/conf.py | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index 78dd6d1e7b88..22c9d4df1967 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -370,7 +370,8 @@ html_static_path = ['sphinx-static']
>  html_use_smartypants = False
>  
>  # Custom sidebar templates, maps document names to template names.
> -#html_sidebars = {}
> +# Note that the RTD theme ignores this
> +html_sidebars = { '**': ['searchbox.html', 'localtoc.html', 'sourcelink.html']}
>  
>  # Additional templates that should be rendered to pages, maps page names to
>  # template names.

-- 
Jani Nikula, Intel Open Source Graphics Center

[PATCH 4/4] docs: remove some index.rst cruft

[Jonathan Corbet]

There is some useless boilerplate text that was added by sphinx when this
file was first created; take it out.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 Documentation/index.rst | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/Documentation/index.rst b/Documentation/index.rst
index 428cfd8e369b..1e983e3fa6f8 100644
--- a/Documentation/index.rst
+++ b/Documentation/index.rst
@@ -1,11 +1,5 @@
 .. SPDX-License-Identifier: GPL-2.0
 
-
-.. The Linux Kernel documentation master file, created by
-   sphinx-quickstart on Fri Feb 12 13:51:46 2016.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
-
 .. _linux_doc:
 
 The Linux Kernel documentation
-- 
2.37.2

Re: [PATCH 0/4] Rewrite the top-level index.rst

[Thorsten Leemhuis]

On 02.09.22 01:16, Jonathan Corbet wrote:
> The top-level index.rst file is the entry point for the kernel's
> documentation, especially for readers of the HTML output.  It is currently
> a mess containing everything we thought to throw in there.  Firefox says it
> would require 26 pages of paper to print it.  That is not a user-friendly
> introduction.
<
> This series aims to improve our documentation entry point with a focus on
> rewriting index.rst.  The result is, IMO, simpler and more approachable.
> For anybody who wants to see the rendered results without building the
> docs, have a look at:
> 
>   https://static.lwn.net/kerneldoc/
> [...]

Great initiative. But looking at the rendered result made me wonder:
what overall structure for the docs are you aiming for in the end? I'm
sure you have a picture in your head, but I failed to grasp it, as for
me a few things looked a little odd:

* we do all of this for the users, so shouldn't the section aimed at
users be at the top? And list more things they will look for?

* What is so important about "Architecture-agnostic documentation" and
"Architecture-specific documentation" (both with just one entry) that
they have to be listed here? Same for "Firmware-related documentation".
And is the User-oriented section really the right place for the kbuild
stuff, as from a quite look it seems most of those aim at developers and
not at users?

* Quite a few things I'd had expected on that front page aren't listed
there. Sure, everybody has different expectations on what's important,
but I for example hat expected "command-line parameters" or "Reporting
issues" (here I'm obviously biased) to be somewhere on that page.

This made me think: should that main index page maybe just have these
three sections (apart from Translations) ?

* User-oriented documentation
* Application-developer documentation
* Other documentation on the Linux kernel and its development

I'd say that makes it quite clear where readers need to go from there,
even if the name of the third section is a bit vague (but in contrast it
becomes clear I'd say).

Each section could list its five to ten most important documents before
linking to a separate index file with more. And that index file for will
need subcategories, too, otherwise it will become large, too.

And sure, quite a few documents will be hard to categorize currently.
Making things fit properly might take a decade or two (unless somebody
hires a few people to bring order into this). But it would set a clear
direction. It also would tell doc writers what tone and detail level to
use when writing their texts, as that depends on the audience which
becomes clearer this way.

Ciao, Thorsten

P.S.: /me wonders if Jonathan posted this patch-set as a bait and will
force everyone replying to come to his LPC/kernel summit session "What
kernel documentation could be"
/me despite this replied, as he had planned to go anyway

Re: [PATCH 0/4] Rewrite the top-level index.rst

[Jonathan Corbet]

Thorsten Leemhuis <linux@leemhuis.info> writes:

> Great initiative. But looking at the rendered result made me wonder:
> what overall structure for the docs are you aiming for in the end? I'm
> sure you have a picture in your head, but I failed to grasp it, as for
> me a few things looked a little odd:

Thanks for taking a look!

> * we do all of this for the users, so shouldn't the section aimed at
> users be at the top? And list more things they will look for?

My thinking is that the main consumers of the kernel docs is kernel
developers (I can't prove this, I just know it :), and that I see a lot
of referrals to the process documentation.  So I started with that.  I'm
not wedded to that organization if something else seems better.

> * What is so important about "Architecture-agnostic documentation" and
> "Architecture-specific documentation" (both with just one entry) that
> they have to be listed here? Same for "Firmware-related documentation".

I kind of ran out of energy after moving a lot of stuff from the front
page and wasn't sure what to do with them.  There's definitely room for
improvement. 

> And is the User-oriented section really the right place for the kbuild
> stuff, as from a quite look it seems most of those aim at developers and
> not at users?

I guess I saw building and installation as a *use* of the kernel.  This
one does sort of cross the lines and could certainly go somewhere else.
I was mostly trying to avoid a bunch of subsections with a single entry. 

> * Quite a few things I'd had expected on that front page aren't listed
> there. Sure, everybody has different expectations on what's important,
> but I for example hat expected "command-line parameters" or "Reporting
> issues" (here I'm obviously biased) to be somewhere on that page.

I'm happy to change the mix; "reporting issues" probably does belong
there, at least.  As long as we don't get back to the current state
where *everything* is on the front page.

> This made me think: should that main index page maybe just have these
> three sections (apart from Translations) ?
>
> * User-oriented documentation
> * Application-developer documentation
> * Other documentation on the Linux kernel and its development

That relegates an awful lot of our important stuff to "other"; as said
above, I think that the main consumers of the documentation are kernel
developers, and the documentation organization should reflect that.

> I'd say that makes it quite clear where readers need to go from there,
> even if the name of the third section is a bit vague (but in contrast it
> becomes clear I'd say).
>
> Each section could list its five to ten most important documents before
> linking to a separate index file with more. And that index file for will
> need subcategories, too, otherwise it will become large, too.
>
> And sure, quite a few documents will be hard to categorize currently.
> Making things fit properly might take a decade or two (unless somebody
> hires a few people to bring order into this). But it would set a clear
> direction. It also would tell doc writers what tone and detail level to
> use when writing their texts, as that depends on the audience which
> becomes clearer this way.

The creation of a bit more structure is certainly one of the goals here.
After several years I'm not having to argue quite so much about grouping
documentation for the intended readers, so it seems like time to stir
things up again :)

> Ciao, Thorsten
>
> P.S.: /me wonders if Jonathan posted this patch-set as a bait and will
> force everyone replying to come to his LPC/kernel summit session "What
> kernel documentation could be"
> /me despite this replied, as he had planned to go anyway

An awful lot of kernel work gets done on conference-presentation
deadline schedules...  The session would be a good time to talk about
what we think our overall structure should be.

I will try to do another pass on this before then, but there's no
guarantees.  If nothing else, it'll clean up the bottom-of-page
messiness where I got lazy the first time.

Thanks,

jon

Re: [PATCH 0/4] Rewrite the top-level index.rst

[Kees Cook]

On Sat, Sep 03, 2022 at 12:03:17PM +0200, Thorsten Leemhuis wrote:
> On 02.09.22 01:16, Jonathan Corbet wrote:
> > The top-level index.rst file is the entry point for the kernel's
> > documentation, especially for readers of the HTML output.  It is currently
> > a mess containing everything we thought to throw in there.  Firefox says it
> > would require 26 pages of paper to print it.  That is not a user-friendly
> > introduction.
> <
> > This series aims to improve our documentation entry point with a focus on
> > rewriting index.rst.  The result is, IMO, simpler and more approachable.
> > For anybody who wants to see the rendered results without building the
> > docs, have a look at:
> > 
> >   https://static.lwn.net/kerneldoc/
> > [...]

I like it -- FWIW, I am able to find stuff much more easily with
this. I am traditionally looking most for internal API details, how
to test exposed userspace interfaces, and process docs (so I can send
reference links to contributors when I'm doing reviews). These map to
"how do I do it?", "how do I test it?", and "where can I aim people for
common process details?"

(So I'd expect to see
https://static.lwn.net/kerneldoc/dev-tools/testing-overview.html
linked under "Development tools and processes")

> Great initiative. But looking at the rendered result made me wonder:
> what overall structure for the docs are you aiming for in the end? I'm
> sure you have a picture in your head, but I failed to grasp it, as for
> me a few things looked a little odd:
> 
> * we do all of this for the users, so shouldn't the section aimed at
> users be at the top? And list more things they will look for?

I'd agree with Jon: I expect the primary consumer to be new and existing
contributors. (Where "new" may just mean "new to this area of the code"
too.)

Under "Other documentation" on the front page that can move:
"Atomic Types", "Atomic bitops" can be moved to Core API docs under
"Data structures". I think "Memory Barriers" can go to "Concurrency
primitives".

-Kees

-- 
Kees Cook