KMS documentation for userspace

KMS documentation for userspace

[Simon Ser]

Hi all,

With Wayland compositors and DRM leases, the number of DRM userspace
programs has increased (and will probably continue to increase).
However it's been pretty hard to find good DRM docs for userspace.

The resources currently available are:

- The kernel docs, although only the high level overview is useful to
  userspace.
- User-contributed tutorials, like [1] and [2].
- To help userspace understand hardware capabilities and limitations,
  I've recently started a DRM database dump project [3].

(Anything else I've missed?)

We still lack docs that describe how to properly use the KMS API and
the KMS properties. There is a deprecated list at [4], and another list
mixed with kernel structs at [5]. This is not great for userspace
developers.

So I wanted to know if there's interest from the kernel community to
maintain documentation for userspace? Or should userspace maintain its
own docs of the kernel API?

I don't know what would be the best place for these docs. If they live
in kernel doc comments, then references to kernel functions and
guidelines for kernel devs will appear (just like in the current docs).
If they live outside of the kernel source code or outside of the kernel
tree, they might get outdated.

Thoughts?

Thanks,

[1]: https://github.com/ascent12/drm_doc
[2]: https://gitlab.freedesktop.org/daniels/kms-quads
[3]: https://drmdb.emersion.fr/
[4]: https://01.org/linuxgraphics/gfx-docs/drm/gpu/drm-kms.html#standard-connector-properties
[5]: https://01.org/linuxgraphics/gfx-docs/drm/gpu/drm-kms.html#existing-kms-properties

--
Simon Ser
https://emersion.fr


_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: KMS documentation for userspace

[Daniel Vetter]

On Wed, Jun 19, 2019 at 9:36 PM Simon Ser <contact@emersion.fr> wrote:
>
> Hi all,
>
> With Wayland compositors and DRM leases, the number of DRM userspace
> programs has increased (and will probably continue to increase).
> However it's been pretty hard to find good DRM docs for userspace.
>
> The resources currently available are:
>
> - The kernel docs, although only the high level overview is useful to
>   userspace.
> - User-contributed tutorials, like [1] and [2].
> - To help userspace understand hardware capabilities and limitations,
>   I've recently started a DRM database dump project [3].
>
> (Anything else I've missed?)
>
> We still lack docs that describe how to properly use the KMS API and
> the KMS properties. There is a deprecated list at [4], and another list
> mixed with kernel structs at [5]. This is not great for userspace
> developers.
>
> So I wanted to know if there's interest from the kernel community to
> maintain documentation for userspace? Or should userspace maintain its
> own docs of the kernel API?
>
> I don't know what would be the best place for these docs. If they live
> in kernel doc comments, then references to kernel functions and
> guidelines for kernel devs will appear (just like in the current docs).
> If they live outside of the kernel source code or outside of the kernel
> tree, they might get outdated.
>
> Thoughts?

tldr; Yes, I just didn't get around to it yet.

The rough plan is to actually document ioctls and properties and all
that stuff in drm-uapi.rst, and then cross-link that with the
driver-side documentation. Current status:
- ioctl docs: doesn't exist
- property docs: spotty, needs to be extracted from
https://01.org/linuxgraphics/gfx-docs/drm/gpu/drm-kms.html#kms-properties
without going crazy (I think you got your links for references 4 and 5
wrong, but it's all the same chapter anyway)
- figure out whether we can use the media uapi sphinx stuff to catch
when new ioctls/structs/props have been added without corresponding
documentation
- figure out what to do with the libdrm manpages. Some stuff we really
want to also document there. But geez nroff. Maybe we need to stuff
libdrm into the kernel, dunno.

I'd say 5-10 years of chugging along until this is fixed. The
in-kernel docs took about as long:

commit 2d2ef822758e3f5da59c40a392d0c6d89394d4b4
Author: Jesse Barnes <jbarnes@virtuousgeek.org>
Date:   Mon Oct 26 13:06:31 2009 -0700

    drm: add initial DRM developer documentation

If you are exceendingly bored and want to get started right away,
document everything in include/uapi/drm/drm*.h as a start :-)

Cheers, Daniel

> Thanks,
>
> [1]: https://github.com/ascent12/drm_doc
> [2]: https://gitlab.freedesktop.org/daniels/kms-quads
> [3]: https://drmdb.emersion.fr/
> [4]: https://01.org/linuxgraphics/gfx-docs/drm/gpu/drm-kms.html#standard-connector-properties
> [5]: https://01.org/linuxgraphics/gfx-docs/drm/gpu/drm-kms.html#existing-kms-properties
>
> --
> Simon Ser
> https://emersion.fr
>
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel



-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: KMS documentation for userspace

[Jani Nikula]

On Wed, 19 Jun 2019, Daniel Vetter <daniel@ffwll.ch> wrote:
> - figure out what to do with the libdrm manpages. Some stuff we really
> want to also document there. But geez nroff. Maybe we need to stuff
> libdrm into the kernel, dunno.

How would people feel about converting libdrm man pages to rst, and
using rst2man in the build? We did that for igt man pages [1]. Looking
at the diff, I think it's plain to see how that could lower the bar for
contributing. And perhaps it would be easier to refactor and move
documentation around too.

I forget, I probably used pandoc or something to do the bulk of the igt
conversion, and added some manual polish on top. I'm not volunteering
for the libdrm man page conversion though. ;)

BR,
Jani.


[1] https://gitlab.freedesktop.org/drm/igt-gpu-tools/commit/cc7387f17ce5750fc640355d94ed60b6936c2d7b

-- 
Jani Nikula, Intel Open Source Graphics Center
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: KMS documentation for userspace

[Simon Ser]

On Wednesday, June 19, 2019 10:53 PM, Daniel Vetter <daniel@ffwll.ch> wrote:
> tldr; Yes, I just didn't get around to it yet.
>
> The rough plan is to actually document ioctls and properties and all
> that stuff in drm-uapi.rst, and then cross-link that with the
> driver-side documentation.

I'm confused regarding drm-uapi.rst's role. Is it a document for kernel driver
writers to expand the uAPI, or is it a document for userspace?

It's currently filled with references to internal kernel symbols
(drm_master_get, drm_ioctl_desc, drm_ioctl_permit and so on). Some chapters seem
dedicated to kernel devs (e.g. "Testing and validation").

Is it really the right place for userspace devs to learn how to use KMS?
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: KMS documentation for userspace

[Daniel Vetter]

On Sat, Jun 22, 2019 at 11:42 AM Simon Ser <contact@emersion.fr> wrote:
> On Wednesday, June 19, 2019 10:53 PM, Daniel Vetter <daniel@ffwll.ch> wrote:
> > tldr; Yes, I just didn't get around to it yet.
> >
> > The rough plan is to actually document ioctls and properties and all
> > that stuff in drm-uapi.rst, and then cross-link that with the
> > driver-side documentation.
>
> I'm confused regarding drm-uapi.rst's role. Is it a document for kernel driver
> writers to expand the uAPI, or is it a document for userspace?
>
> It's currently filled with references to internal kernel symbols
> (drm_master_get, drm_ioctl_desc, drm_ioctl_permit and so on). Some chapters seem
> dedicated to kernel devs (e.g. "Testing and validation").
>
> Is it really the right place for userspace devs to learn how to use KMS?

There's more to drm than kms, but yeah I think currently that's the
best starting point we have for documenting the uapi. We might also
need to separate some of the more kernel-internal bits into other
chapters, e.g. ioctl and master stuff is currently there because those
are fairly important concept from a drm uapi pov. But maybe we should
pull out the implementation details into some other place.

Given that 0 of our ioctls and ioctl structures are currently
documented, I'm not really worried about those issues just yet :-)
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: KMS documentation for userspace

[Daniel Vetter]

On Wed, Jun 19, 2019 at 10:16 PM Jani Nikula
<jani.nikula@linux.intel.com> wrote:
>
> On Wed, 19 Jun 2019, Daniel Vetter <daniel@ffwll.ch> wrote:
> > - figure out what to do with the libdrm manpages. Some stuff we really
> > want to also document there. But geez nroff. Maybe we need to stuff
> > libdrm into the kernel, dunno.
>
> How would people feel about converting libdrm man pages to rst, and
> using rst2man in the build? We did that for igt man pages [1]. Looking
> at the diff, I think it's plain to see how that could lower the bar for
> contributing. And perhaps it would be easier to refactor and move
> documentation around too.

I think that'd be very nice.

> I forget, I probably used pandoc or something to do the bulk of the igt
> conversion, and added some manual polish on top. I'm not volunteering
> for the libdrm man page conversion though. ;)

Hm, maybe dig out the old tools you used and point Simon at them?
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: KMS documentation for userspace

[Jani Nikula]

On Sat, 22 Jun 2019, Daniel Vetter <daniel@ffwll.ch> wrote:
> On Wed, Jun 19, 2019 at 10:16 PM Jani Nikula
> <jani.nikula@linux.intel.com> wrote:
>>
>> On Wed, 19 Jun 2019, Daniel Vetter <daniel@ffwll.ch> wrote:
>> > - figure out what to do with the libdrm manpages. Some stuff we really
>> > want to also document there. But geez nroff. Maybe we need to stuff
>> > libdrm into the kernel, dunno.
>>
>> How would people feel about converting libdrm man pages to rst, and
>> using rst2man in the build? We did that for igt man pages [1]. Looking
>> at the diff, I think it's plain to see how that could lower the bar for
>> contributing. And perhaps it would be easier to refactor and move
>> documentation around too.
>
> I think that'd be very nice.
>
>> I forget, I probably used pandoc or something to do the bulk of the igt
>> conversion, and added some manual polish on top. I'm not volunteering
>> for the libdrm man page conversion though. ;)
>
> Hm, maybe dig out the old tools you used and point Simon at them?

I could swear I did it with pandoc only, but the pandoc I have on my
distro right now does not support man input. Upstream documentation
claims it does, so I don't know what gives.

If that fails, first man2html and then pandoc from html to rst works to
an extent.

Regardless, I think the hardest part is deciding what style to use in
the rst files (all the headings and synopsis and references etc.) and
then tediously do the manual cleanup after the conversion. It's a
one-time effort, so there are limits to the ROI on polishing the
toolchain.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Graphics Center
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: KMS documentation for userspace

[Simon Ser]

On Thursday, June 27, 2019 4:59 PM, Jani Nikula <jani.nikula@linux.intel.com> wrote:

> On Sat, 22 Jun 2019, Daniel Vetter daniel@ffwll.ch wrote:
>
> > On Wed, Jun 19, 2019 at 10:16 PM Jani Nikula
> > jani.nikula@linux.intel.com wrote:
> >
> > > On Wed, 19 Jun 2019, Daniel Vetter daniel@ffwll.ch wrote:
> > >
> > > > -   figure out what to do with the libdrm manpages. Some stuff we really
> > > >     want to also document there. But geez nroff. Maybe we need to stuff
> > > >     libdrm into the kernel, dunno.
> > > >
> > >
> > > How would people feel about converting libdrm man pages to rst, and
> > > using rst2man in the build? We did that for igt man pages [1]. Looking
> > > at the diff, I think it's plain to see how that could lower the bar for
> > > contributing. And perhaps it would be easier to refactor and move
> > > documentation around too.
> >
> > I think that'd be very nice.
> >
> > > I forget, I probably used pandoc or something to do the bulk of the igt
> > > conversion, and added some manual polish on top. I'm not volunteering
> > > for the libdrm man page conversion though. ;)
> >
> > Hm, maybe dig out the old tools you used and point Simon at them?
>
> I could swear I did it with pandoc only, but the pandoc I have on my
> distro right now does not support man input. Upstream documentation
> claims it does, so I don't know what gives.
>
> If that fails, first man2html and then pandoc from html to rst works to
> an extent.
>
> Regardless, I think the hardest part is deciding what style to use in
> the rst files (all the headings and synopsis and references etc.) and
> then tediously do the manual cleanup after the conversion. It's a
> one-time effort, so there are limits to the ROI on polishing the
> toolchain.

After all these months, I've started converting the man pages to
rst [1]. Better late than never!

I've used this command to convert xml to rst:

    pandoc -s -f docbook -t rst -o man/drm.rst man/drm.xml

This mostly works but needs some rework to fixup references and
headings. You can see the final result in the MR.

[1]: https://gitlab.freedesktop.org/mesa/drm/-/merge_requests/72
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel