Re: [PATCH v2 00/10] qapi: enforce section ordering

[Markus Armbruster <armbru@redhat.com>]

John Snow <jsnow@redhat.com> writes:

> Hiya, this series is meant to accomplish mostly one thing: Enforce a
> stricter ordering of sections in QAPI documentation blocks.
>
> The reason to do this is mostly for the sake of the inliner: if QAPI
> documentation blocks have some known, canonical order, it is easier to
> merge two documentation blocks together for the purposes of showing all
> arguments for commands/etc in a simple, flat list without needing to
> follow hyperlink breadcrumbs.
>
> Another reason to do this is to simplify where we insert autogenerated
> documentation. If the order is enforced, then inserting "Not Documented"
> stubs for members and generated "Returns:" statements can have a much
> simpler algorithm that will always match how manually written
> documentation is presented, in the same order.
>
> This is still pretty RFC quality, the tests have not been implemented
> and the implementation of changes in the parser are still pretty
> fuzzy. The main point of this series at this point in time is to review
> the QAPI source changes and decide if the strategy employed in fixing
> the section ordering is the direction we ultimately want to go in.
>
> V2:
>  - Add quite a few FIXME stubs for tests
>  - Much more carefully delineate QAPI source changes into ones required
>    to prevent visible changes, and ones that explictly create visible
>    changes
>  - Various commit message / comment changes
>  - Fix heuristic for griping about Intro/Details "ambiguity" to also
>    ignore generated "Returns" sections, which was missing before and
>    missed quite a few cases that did impact rendered output
>
> To verify rendering changes (or lack thereof), I used this strategy:
>
> (1) For a reference output before a change, I ran a build:
>     > V=1 DEBUG=1 make -j13;
>
> (2) Then I created some reference output for the intermediate rST
>     debugging output files (fish syntax):
>     > for i in *.ir; sed -E 's|\.json:[0-9]{4}|.json:nnnn|g' $i > $i.ref; end
>
> (3) Then after applying a patch, to check for any differences, I re-ran
>     the build as in (1) and then:
>     > for i in *.ir; sed -E 's|\.json:[0-9]{4}|.json:nnnn|g' $i > $i.new; end
>     > for i in *.ir; meld $i.ref $i.new; end
>
> An observation: Most of the time, the Intro section is only one
> paragraph anyway. We might be able to save on some explicit "Details:"
> syntax if we just formalize the idea that the intro can only ever be at
> most one paragraph. I don't know if we want to do that (Do we want to
> keep the ability to run long in the "intro"?) - but it would cut down on
> quite a lot of markup that this series adds.

The series adds 59 "Details:" markers to ~1150 doc comments, i.e. about
one in twenty doc comments needs one.

I can see several ways to skin this cat:

1. Intro ends when something else begins

   Intro can be any number of paragraphs.

   When intro is followed by details, we need a way to mark the
   boundary.  That's your "Details:" marker.  Relatively rare.

   Drawback: since you need "Details:" only rarely, it is easy to
   forget.  Details are then mistaken for intro or the other way round.
   The inliner will lose details / fail to lose intro.

   This is what your series gives us.

2. Intro ends after the first paragraph if we have one.

   Intro can be empty or one paragraph.

   When empty intro is followed by details, we need a way to mark the
   boundary, such as "Details:".  This is quite rare.

   Drawback: "at most one paragraph" is not obvious and easy to forget.
   Extra intro paragraphs end up in details then, where the inliner will
   copy them.

   Drawback: since you need "Details:" almost never, it is easy to
   forget.  The first paragraph of Details is mistaken for intro then.
   The inliner will lose it.

3. Make the end of intro syntactically obvious always

3a. Require the "Details:" line

   Intro can be any number of paragraphs.

   Drawback: the syntax is rather clumsy.  ~270 doc comments have
   details, and every one of them needs a "Details:" line.

   Drawback: you can still forget the "Details:" line.  If you do, the
   entire details will be mistaken for intro, and the inliner will lose
   them.

3b. Require a "Details:" tag

   Like 3a, but with slightly less clumsy syntax.  Instead of

      # @Frob:
      #
      # Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
      # eiusmod tempor incididunt ut labore et dolore magna aliqua.
      #
      # Details:
      #
      # Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
      # nisi ut aliquip ex ea commodo consequat.
      #
      # Duis aute irure dolor in reprehenderit in voluptate velit esse
      # cillum dolore eu fugiat nulla pariatur.

   we get

      # @Frob:
      #
      # Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
      # eiusmod tempor incididunt ut labore et dolore magna aliqua.
      #
      # Details: Ut enim ad minim veniam, quis nostrud exercitation ullamco
      #     laboris nisi ut aliquip ex ea commodo consequat.
      #
      #     Duis aute irure dolor in reprehenderit in voluptate velit esse
      #     cillum dolore eu fugiat nulla pariatur.

   Drawback: the syntax is still clumsy.  ~270 doc comments have
   details, and every one of them needs a "Details:" tag and be
   reindented.

   Drawback: if you forget to tag details, they get mistaken for intro,
   and the inliner will lose them.  Less likely than with 3a?

3c. Indent intro like descriptions and tagged sections.

   Like so:

      # @Frob:
      #     Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
      #     eiusmod tempor incididunt ut labore et dolore magna aliqua.
      #
      # Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
      # nisi ut aliquip ex ea commodo consequat.
      #
      # Duis aute irure dolor in reprehenderit in voluptate velit esse
      # cillum dolore eu fugiat nulla pariatur.

   Drawback: all the intros get reindented.

   Drawback: if you format intro the old way, it gets mistaken for
   details, and the inliner will fail to lose it.  Likely to happen
   initially, but hopefully becomes unlikely once people got used to it.

   Anything else?

3d. Both 3b and 3c

   Intros and details written the old way get rejected.

   Drawback: all intros and all details need to be reindented.

   Anything else?

Thoughts?