Re: The GNU C Library Manual - Authoritative or not?

Re: The GNU C Library Manual - Authoritative or not?

[Michael Kerrisk]

[CC += linux-man@]
[Context: https://lwn.net/ml/libc-alpha/875300cf-92ca-c115-c42d-19dda5de5a4a@redhat.com/]

On Fri, May 22, 2020 at 3:07 PM Carlos O'Donell via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> On 5/21/20 8:22 PM, Martin Sebor wrote:
> > On 5/21/20 4:11 PM, Carlos O'Donell wrote:
> >> On 5/21/20 1:46 PM, Martin Sebor wrote:

[...]

> > It would be a considerable
> > effort to bring the manual up to par with the Linux man pages.  I see
> > little point in investing it into duplicating what already exists
> > elsewhere and what according to documentation.html many of you are
> > already contributing to.  My suggestion instead is to declare
> > the Linux man pages the reference and treat the manual as a user
> > guide.

So, as an aside, I think the Linux man-pages are rather better than
the glibc manual in many areas, but there some areas where where the
reverse is true. And lately I see patches to the glibc manual from
Florian, and sometimes I realize: "oh that's not covered in Linux
man-pages". So, notwithstanding what the glibc developers may thing
of the idea, from my perspective the notion of declaring man-pages as
the [authoritative] reference would not be without problems.

> I appreciate your suggestion.
>
> I agree it would be a waste of time to duplicate what already exists.
>
> The glibc manual should be a guide and it should contain complete and
> authoritative information about the APIs it implements. This does
> not include version-by-version changes, bugs, raw syscalls, etc.
>
> I think having the linux man pages as a non-authoritative reference
> is very good, and I contribute to the linux man pages.
>
> I think having WG14 work on ISO C is also good, so I contribute where
> I can to SC22 in Canada for that reason.
>
> That makes 4 documents covering the same APIs!
>
> * ISO C (the standard)
> * POSIX (the extended standard)
> * glibc manual (the authoritative manual for glibc's implementation)
> * Linux man pages (detailed syscall and API reference documentation)
>
> FWIW they all serve different purposes.
>
> The linux man pages are *great* they even document version to version
> differences and if you're interested in targeting specific versions of
> glibc you can use them accurately to write code that does exactly
> what you would expect. That's awesome! Our manual does not have that as
> a goal.
>
> >> This is not a duplication of effort IMO. The manual and the linux man
> >> pages solve different needs. The manual is task-oriented, covering
> >> sections of the standard APIs and how they could and should be used
> >> together,

And yes, in places this is where the glbc manual does really excel.

> >> while the the linux man pages are API references only
> >> (in isolation to the larger set of APIs).

Broadly true, although in places, Linux man-pages tries also to draw
bigger pictures (e.g., various overview pages in Section 7).

[...]

> > A report of the warning above is what prompted my question.  I had
> > checked the C and in some cases also POSIX standard as well as
> > the Glibc manual when adding the attribute.  C doesn't have this
> > extension, Glibc doesn't document it, and I missed it in POSIX (or
> > more likely didn't think to look there in this case).  What I was
> > looking for with the question is an acknowledgment of what I had
> > suspected, namely that the Linux pages can be trusted to accurately
> > document this and other Glibc extensions.
>
> The linux man pages can *absolutely* be trusted to document every
> change glibc makes, but that doesn't make them authoritative, it makes
> them detailed. As a developer I love the detail in `man mbstowcs`.

I commented on this already. I wish the above were true, but Linux
man-pages does not manage to track all of the changes.

[...]

> > But from a simple usability point of view, it's unhelpful to tell
> > people to consider the union of the Glibc manual and all external
> > documentation (or some subset of it).  Not just because it's
> > impractical to read everything, but also because not everything
> > is correct or up to date.  How are they/we supposed to resolve conflicts?
>
> We resolve conflicts by writing things in the manual to cover such cases,
> and we do so tactically to resolve problems as they arise.
>
> Over the years the project has had a significant lack of engagement with
> writing good documentation,

Yes, the manual seems to have started very well, but then the wheels
came off for many years. (Surprisingly, during that time I would
occasionally get a piece of very helpful input for man-pages from
Ulrich Drepper!)

> and I can understand that. It's hard to
> write clear and unambiguous English sentences to describe an interface
> and how it dovetails into the rest of the APIs.

Yes, best to leave that task to the Germans. (Sorry; I could not
resist the hat tip to Florian.)

> Such writing is not as
> much fun as writing code.

Yes, I never really understood that one. It's only by explaining (my)
code very clearly at least to myself, but probably to others, that I
can feel like it is/I have written good code. Writing good human
language expression is just as much an interesting challenge as
writing good programming language expression.

> We really need to engage with technical writers
> and involve a broader set of industry skills in our projects.

I want to add a note of caution here. It's great to have technical
writers (and like good developers companies should be paying them),
but they can't do the job on their own. A lot of developer input is
still required.

Thanks,

Michael

Re: The GNU C Library Manual - Authoritative or not?

[J William Piggott]

On Mon, 25 May 2020, Michael Kerrisk via Libc-alpha wrote:

... >8
>
>> We really need to engage with technical writers
>> and involve a broader set of industry skills in our projects.
>
> I want to add a note of caution here. It's great to have technical
> writers (and like good developers companies should be paying them),
> but they can't do the job on their own. A lot of developer input is
> still required.

Another caution, many HR departments hire 'technical writers' that in
reality are copy editors, who's knowledge base is grammar/writing/language.
In my experience, they tend to make things a lot worse. Wordsmiths like
to use words, lots of words. They want to create novels. The complete
opposite of what technical writing should be.

>
> Thanks,
>
> Michael
>

Re: The GNU C Library Manual - Authoritative or not?

[Carlos O'Donell]

On 5/25/20 11:51 AM, J William Piggott wrote:
> 
> 
> On Mon, 25 May 2020, Michael Kerrisk via Libc-alpha wrote:
> 
> ... >8
>>
>>> We really need to engage with technical writers
>>> and involve a broader set of industry skills in our projects.
>>
>> I want to add a note of caution here. It's great to have technical
>> writers (and like good developers companies should be paying them),
>> but they can't do the job on their own. A lot of developer input is
>> still required.
> 
> Another caution, many HR departments hire 'technical writers' that in
> reality are copy editors, who's knowledge base is grammar/writing/language.
> In my experience, they tend to make things a lot worse. Wordsmiths like
> to use words, lots of words. They want to create novels. The complete
> opposite of what technical writing should be.

Agreed. We don't need copy editors. We need true technical writers that
understand C and C++. Sadly, I've rarely met such people, and I agree with
Michael, that I also find describing my code to be an illuminating part of
design. Even though others don't share my interests, the linux man pages as
a project shows there are enough of those people that the project can thrive
and keep all the man pages well enough updated that they are more useful for
API reference than the glibc manual.

-- 
Cheers,
Carlos.

Re: The GNU C Library Manual - Authoritative or not?

[Michael Kerrisk]

[CC += linux-man@]
[Context: https://lwn.net/ml/libc-alpha/875300cf-92ca-c115-c42d-19dda5de5a4a@redhat.com/]

Hello Florian, Carlos, et al.

On Fri, May 22, 2020 at 3:22 PM Florian Weimer via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> * Carlos O'Donell via Libc-alpha:
>
> > "What is the authoritative source for public glibc APIs?"
> > https://sourceware.org/glibc/wiki/FAQ#What_is_the_authoritative_source_for_public_glibc_APIs.3F
>
> Current text:
>
> | The GNU C Library manual is the authoritative place for such
> | information that is related to the implementation of functions in
> | glibc.
> |
> | The Linux Man Pages are non-authoritative, but they are incredibly
> | useful, easy to use, and often the first source of such information.
> |
> | The Linux Man Pages is generally authoritative on kernel syscalls, and
> | we have worked hard in cases like futex to ensure the documentation is
> | clear enough for all C libraries.
> |
> | We should all work together to keep both the manual (glibc manual) and
> | the shorter form API references (linux man pages) up to date with the
> | most accurate information we have.
> |
> | Where you find issues with the manual or the linux man pages please
> | reach out to discuss the issue.
>
> > "What other sources of documentation about glibc are available?"
> > https://sourceware.org/glibc/wiki/FAQ#What_other_sources_of_documentation_about_glibc_are_available.3F
>
> | The glibc manual is part of glibc, it is also available online.
> |
> | The Linux man-pages project documents the Linux kernel and the C
> | library interfaces.
> |
> | The Open Group maintains the POSIX Standard which is the authoritative
> | reference for the POSIX APIs.
> |
> | The ISO JTC1 SC22 WG14 maintains the C Standard which is the
> | authoritative reference for the ISO C APIs.
> |
> | The official home page of glibc is at http://www.gnu.org/software/libc.
> |
> | The glibc wiki is at http://sourceware.org/glibc/wiki/HomePage.
> |
> | For bugs, the glibc project uses the sourceware bugzilla with
> |component 'glibc'.
>
> I don't think this is very helpful.  It paints a simple picture which is
> turns out to be rather misleading when inspected closely.  For example,
> POSIX often claims that ISO C takes precedence, but then proceeds to
> specify conflicting requirements with ISO C.  What does that even mean?
> After fall, it's not possible to have multiple authoritative sources for
> the behavior of a single function.
>
> Practically speaking, I see the following problems.
>
> The GNU C Library manual is not often consulted by developers.  I don't
> know why.  One reason may be that it is not readily installable from
> package repositories on Debian or Ubuntu (at least not in current
> versions from the main archive).  But our experience at Red Hat suggests
> that our developers do not read the manual, either, although we do ship
> it.  I base this on the paucity of bug reports against the manual
> compared to what we receive for the man-pages package (which are often
> misfiled initially against glibc).  In my opinion, a manual that is not
> actually used by the people who benefit from the information in it has
> at best a dubious claim to authority.
>
> Reading our manual requires considerable skill.  You need to know the
> history of the project, the lingering Linux vs Hurd conflict, the late
> arrival of threading support in UNIX-like environments, the tension
> between the POSIX and C standards, the lack of maintenance of both, and
> so on.  Without such knowledge, it is often not possible to reach the
> right conclusions.  Even senior developers can easily get confused.
> (For a recent example, consider Kees Cook's misinterpretation of the
> O_EXEC documentation in the manual.)  Part of the problem here is that
> we do not have a team that combs through the manual from time to time
> and keeps it up to date, as our understanding of the documented matter
> evolves.
>
> When it comes to Linux interfaces, any claim about authority of the
> manual is very misleading.  It does not matter if the system call is
> described by POSIX.  For example, if Linux developers change the signal
> that waitpid reports after a failed execve, and our manual documents
> something else, then the manual is now wrong.  And not the kernel.  If
> the manual were authoritative, it would be the other way round.  (The
> man-pages project is not authoritative in that sense, either—it did
> document the SIGKILL signal and had to be updated.)

Thanks. I find a lot of wisdom in what you say and do not disagree
with any of it. I just want to add a few thoughts and observations.

"The GNU C Library manual is not often consulted by developers"

Each year I come into contact with quite a large number of developers
(some few hundred each year) in many locations in my day job (or, at
least what used to be my day job until COVID-19 landed), and I think
*very* few of them are aware of the existence of the glibc manual.
Most are aware of manual pages. (However, they mostly aren't aware
that there is an project entity called "Linux man-pages"; rather, they
just know that they get a pile of manual pages on their systems, and
many of them consult those pages.)

And then there is the "info" thing. As a complete document (i.e.,
PDF), the glibc manual is quite a handsome document with a lot of good
information, but not the thing one wants to reach for when facing a
specific API problem. What is one then left with? "info". I think in
all of the years that I have been around Linux, I cannot recall
meeting anyone who had a kind word to say about this format/interface.
People generally don't understand how to navigate in "info", and I
think the whole idea of hyperlinking in a textual UI is one that
doesn't work well from a usability perspective. https://xkcd.com/912/
is funny for a reason.

"I base this on the paucity of bug reports against the manual compared
to what we receive for the man-pages package"

I want to add some further perspective here. Linux man-pages roughly
follows the release frequency of the kernel [1], thus a new release
every 10 weeks or so. The next release will feature changes resulting
from input from more than *70* people (email bug reporters, patch
submitters, reviewers, people who sent me random email that gave me
ideas, bugzilla reporters).

I put that high number of contributors [2] down to many factors:

* As you (Florian) observe, I think it's true that (many) more people know
about manual pages (than the glibc manual/"info").

* I try to make it easy for people to know how to report bugs.
Notably, since 2007, each manual page in the released set has a
COLOPHON [3] that has some minimal information about the origin of the
page *and how to report bugs*.

* That there is someone who actually responds to the documentation bug
reports. And here I paint myself in a good and bad light. When I am
very active, I do notice more bug reports start coming in. When I am
less active (e.g. in the last couple of years), there is a noticeable
fall in bug reports and contributions. (These observations are
subjective/anecdotal, but I think there is a real trend underneath,
since I've been  through this cycle a few times..)

* I try to make it easy for people to contribute. There's a website
with a fair amount of information about how people should send patches
[4], and I get a surprising number of random patches that actually
follow the guidelines [5]. By contrast, even among those who are aware
of the glibc manual, I estimate that few are aware of how to
contribute to it.

And on that last point, I circle back to an issue that I've banged on
about before. The CLA. It's just a huge barrier to contribution (and,
I remain convinced, A Bad Thing [TM], even if its motivation is well
intentioned [6]). Just in the last day or two, there's someone doing
what seems natural to so many in this (FOSS) world:

https://lwn.net/ml/libc-alpha/20200523191809.19663-1-aurelien.aptel%40gmail.com/

I presume this patch submitter has no idea of the existence of the
CLA. Once that person learns of it, will they bother doing the
paperwork, or will they just never bother submitting another patch? I
know which way I would bet my money.

> Many of these issues are beyond our control.  Some of the issues which
> are in our area would need a tremendous amount of work to address.

Yes. Writing good documentation is a lot of work. I know for sure that
man-pages could be a full-time (end even enjoyable!) job for me (or
someone else)--there is that much work that *could* be done--but
something else must pay the bills.

Cheers,

Michael

[1] This is a somewhat arbitrary decision that I made a few years ago,
simply to avoid the "big minor version numbers" problem, which was
roughly the problem that Linus Torvalds was also trying to avoid with
the kernel, although he came to that decision a few years earlier than
me.

[2] The high contribution rate is something of a local maxima in the
last couple of years. See my comments in the other thread about
contribution by others in relation to my own activity elsewhere.
(Turns out that lockdown has some positive effects not just for the
environment, but also for man-pages.)

[3] Example: https://man7.org/linux/man-pages/man7/user_namespaces.7.html#COLOPHON

[4] https://www.kernel.org/doc/man-pages/patches.html

[5] I don't want to paint too rosy a picture here. I still write the
majority of the commits that go into manual pages, and I remain the
reviewer of last resort for the majority of patches, which clearly
does not scale well. And far too many things fall on the floor when my
time is limited. (@Carlos, you probably have too rosy a picture of my
efficiency, because I am usually fairly responsive to input from you
and Florian. But that's because I know from past experience that I can
be fairly confident that what I receive from you will be of a quality
that is typically effortless for me to process.)

[6] https://lwn.net/Articles/529522/

Re: The GNU C Library Manual - Authoritative or not?

[Florian Weimer]

* Michael Kerrisk:

> Each year I come into contact with quite a large number of developers
> (some few hundred each year) in many locations in my day job (or, at
> least what used to be my day job until COVID-19 landed), and I think
> *very* few of them are aware of the existence of the glibc manual.
> Most are aware of manual pages. (However, they mostly aren't aware
> that there is an project entity called "Linux man-pages"; rather, they
> just know that they get a pile of manual pages on their systems, and
> many of them consult those pages.)

Thanks for sharing your perspective.

> And then there is the "info" thing. As a complete document (i.e.,
> PDF), the glibc manual is quite a handsome document with a lot of good
> information, but not the thing one wants to reach for when facing a
> specific API problem. What is one then left with? "info". I think in
> all of the years that I have been around Linux, I cannot recall
> meeting anyone who had a kind word to say about this format/interface.
> People generally don't understand how to navigate in "info", and I
> think the whole idea of hyperlinking in a textual UI is one that
> doesn't work well from a usability perspective. https://xkcd.com/912/
> is funny for a reason.

Even for Emacs users, I suspect that many more are aware of “M-x man RET
RET” than those that are aware of “C-h S”, which jumps right to the
function documentation in the glibc manual.  I have not figured out how
this actually works in practice.  I suspect it uses the Texinfo function
index.  Unfortunately, quite a bit of useful information in the Texinfo
sources is not visible in rendered versions.

One could try to get something similar to “C-h S” into Visual Studio
Code and other IDEs.  But I'm not convinced this is a good use of
resources.  Even if I can remember the Emacs command, I usually need the
manual pages because I'm more interested in the system call
documentation.

> And on that last point, I circle back to an issue that I've banged on
> about before. The CLA. It's just a huge barrier to contribution (and,
> I remain convinced, A Bad Thing [TM], even if its motivation is well
> intentioned [6]). Just in the last day or two, there's someone doing
> what seems natural to so many in this (FOSS) world:
>
> https://lwn.net/ml/libc-alpha/20200523191809.19663-1-aurelien.aptel%40gmail.com/
>
> I presume this patch submitter has no idea of the existence of the
> CLA. Once that person learns of it, will they bother doing the
> paperwork, or will they just never bother submitting another patch? I
> know which way I would bet my money.

I still haven't given up hope entirely for relicensing the manual under
a license that is compatible with Debian's requirements, and also making
it easier to move code and documentation between the manual and the
implementation itself.  The current copyright assignment procedure means
that there is no legal or technical obstacle to relicensing, one has
simply to convince the single copyright owner.

Thanks,
Florian

Re: The GNU C Library Manual - Authoritative or not?

[J William Piggott]

On Mon, 25 May 2020, Michael Kerrisk via Libc-alpha wrote:

  ... >8

> * I try to make it easy for people to contribute.

Yes, the barrier to entry is pretty high; especially for a simple manual
fix. I speak from experience, I had a list of corrects to make; it was
relatively easy for the Linux man-pages. I believe, after getting two
accepted for The Manual I gave up.

Perhaps a separate mailing list dedicated to The Manual accepting
patches with relaxed rules?

As to discovery, that is, The Manual being unknown. For years my go to
tool for information was apropos(1). Of course you cannot discover
info(1) pages that way. A script could convert The Manual into a man
page. I'd be huge and probably ugly, but people could find it. Actually,
I already use The Manual in a similar way. I cat and format it into a
monolithic text file. I use the pager's search to find what I need. I am
used to the search patterns that, for example, find x-refs, nodes, etc.
It works for me (better then info(1) does).

In the beginning I found the fragmentation of Linux docs frustrating.
Not just info and man pages, but also html, pdf, text, howtos, kernel
docs, etc. I'm used to it now, and I'm thankful that we have as much
information as we do. There seems to be a lot of negative response to
The Manual; I'd like to say that it is a very useful body of work for
me. Michael's project is too. So a big thank you to all that put time
and effort into documentation!