critic on documentation of the network stack

critic on documentation of the network stack

[Hannes Frederic Sowa]

Hello!

After net-next is closed I wanted to put the following link here:

  <http://linux.slashdot.org/comments.pl?sid=4356053&cid=45184693>

I don't want to start a flamefest or come too close to someone but I
fear some of the critic is reasonable.  Maybe we can do better (I have
to admit, I also hate writing documentation, e.g. have not yet send the
IP_PMTUDISC_INTERFACE man-page patches).

I try to start with some constructive discussion:

There are some great features in the network stack that some people miss
because of lack documentation. One possible solution is documentation
directly in the kernel, but mostly this is just written as a reference
and the real wonderful stuff is only achieved by putting lots of those
features correclty together.

Maybe this is the second or third time this was proposed but I'll try
again: Would it make sense to just start slow and setup a wiki where we
just throw in the various snippets we use for testing while developing
patches, maybe with a bit of background information? This may well attract
interested people outside of netdev@ which could start helping cleaning
up the wiki or add more useful documentation on their own. We could
check from time to time what could be fed back into Documentation/? The
reason why I would definitely help to improve the wiki is because I
am sure I can learn from other setups and testing methodologies, too,
and definitely still have not yet seen everything what is possible with
the linux network stack.

Thanks && Hopes for a constructive dicussion,

  Hannes

Re: critic on documentation of the network stack

[Richard Weinberger]

On Fri, Jan 24, 2014 at 4:23 AM, Hannes Frederic Sowa
<hannes@stressinduktion.org> wrote:
> Hello!
>
> After net-next is closed I wanted to put the following link here:
>
>   <http://linux.slashdot.org/comments.pl?sid=4356053&cid=45184693>
>
> I don't want to start a flamefest or come too close to someone but I
> fear some of the critic is reasonable.  Maybe we can do better (I have
> to admit, I also hate writing documentation, e.g. have not yet send the
> IP_PMTUDISC_INTERFACE man-page patches).
>
> I try to start with some constructive discussion:
>
> There are some great features in the network stack that some people miss
> because of lack documentation. One possible solution is documentation
> directly in the kernel, but mostly this is just written as a reference
> and the real wonderful stuff is only achieved by putting lots of those
> features correclty together.
>
> Maybe this is the second or third time this was proposed but I'll try
> again: Would it make sense to just start slow and setup a wiki where we
> just throw in the various snippets we use for testing while developing
> patches, maybe with a bit of background information? This may well attract
> interested people outside of netdev@ which could start helping cleaning
> up the wiki or add more useful documentation on their own. We could
> check from time to time what could be fed back into Documentation/? The
> reason why I would definitely help to improve the wiki is because I
> am sure I can learn from other setups and testing methodologies, too,
> and definitely still have not yet seen everything what is possible with
> the linux network stack.

I fully agree with you. Kernel features without userspace docs suck.
I'm not sure whether a wiki is a good idea but at least the manpages
be kept in sync.

The tcp socket option TCP_CONGESTION is such a case.

Kernel implemenation was added by:
commit 5f8ef48d240963093451bcf83df89f1a1364f51d
Author: Stephen Hemminger <shemminger@osdl.org>
Date:   Thu Jun 23 20:37:36 2005 -0700

    [TCP]: Allow choosing TCP congestion control via sockopt.

    Allow using setsockopt to set TCP congestion control to use on a per
    socket basis.

    Signed-off-by: Stephen Hemminger <shemminger@osdl.org>
    Signed-off-by: David S. Miller <davem@davemloft.net>

Linux manpages tell about the feature since:
commit d6d58656220f3ee24990e88dd9f37967a46fb290
Author: Michael Kerrisk <mtk.manpages@gmail.com>
Date:   Fri Nov 21 12:29:37 2008 -0500

    tcp.7: Document /proc file tcp_allowed_congestion_control (new in
Linux 2.4.20)

    Text taken from Documentation/networking/ip-sysctl.txt

    Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>

and finally:
commit bf561a0fbcfed101aea2d523fe5cd50e90273786
Author: Michael Kerrisk <mtk.manpages@gmail.com>
Date:   Thu Jan 23 05:11:10 2014 +0100

    tcp.7: Document TCP_CONGESTION

    Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>


A hardcore net/ hacker may know all nice features but joey random
network programer
just reads the manpages...

-- 
Thanks,
//richard

Re: critic on documentation of the network stack

[Stephen Hemminger]

On Fri, 24 Jan 2014 04:23:24 +0100
Hannes Frederic Sowa <hannes@stressinduktion.org> wrote:

> Hello!
> 
> After net-next is closed I wanted to put the following link here:
> 
>   <http://linux.slashdot.org/comments.pl?sid=4356053&cid=45184693>

The problem I have is more that there are more incorrect sources of documentation
and differing opinions on the Internet. Maybe the problem is users, maybe the
problem is lack of SEO, or developers not being paid to write documentation, or
old web sites not being updated. For example, this commenter obviously never
found http://www.lartc.org/

Re: critic on documentation of the network stack

[Florian Fainelli]

Le 24/01/2014 15:58, Stephen Hemminger a écrit :
> On Fri, 24 Jan 2014 04:23:24 +0100
> Hannes Frederic Sowa <hannes@stressinduktion.org> wrote:
>
>> Hello!
>>
>> After net-next is closed I wanted to put the following link here:
>>
>>    <http://linux.slashdot.org/comments.pl?sid=4356053&cid=45184693>
>
> The problem I have is more that there are more incorrect sources of documentation
> and differing opinions on the Internet. Maybe the problem is users, maybe the
> problem is lack of SEO, or developers not being paid to write documentation, or
> old web sites not being updated. For example, this commenter obviously never
> found http://www.lartc.org/

(which unfortunately is rather outdated)

I do not buy the fact that some developers do not provide documentation 
of the features they are adding potentially on purpose, truth is 
probably much simpler, you worked on X, you have now moved on and work on Y.

If nobody pays enough attention to what gets added through netdev, 
iproute2, ethtool, man-pages and enforces the need for documentation, 
then comes the current status quo where not all features are documented, 
until some benevolant person realizes this needs fixing. Considering the 
high volume of the list, this is all understandable.

There could probably be some programmatical ways to enforce such 
documentation by only allowing patches coming with, say kernel-doc 
content along the code, and have man-pages and other projects scan for 
new kernel-doc entries they have no reference for.
--
Florian

Re: critic on documentation of the network stack

[Richard Weinberger]

On Sun, Jan 26, 2014 at 4:18 AM, Florian Fainelli <f.fainelli@gmail.com> wrote:
> Le 24/01/2014 15:58, Stephen Hemminger a écrit :
>
>> On Fri, 24 Jan 2014 04:23:24 +0100
>> Hannes Frederic Sowa <hannes@stressinduktion.org> wrote:
>>
>>> Hello!
>>>
>>> After net-next is closed I wanted to put the following link here:
>>>
>>>    <http://linux.slashdot.org/comments.pl?sid=4356053&cid=45184693>
>>
>>
>> The problem I have is more that there are more incorrect sources of
>> documentation
>> and differing opinions on the Internet. Maybe the problem is users, maybe
>> the
>> problem is lack of SEO, or developers not being paid to write
>> documentation, or
>> old web sites not being updated. For example, this commenter obviously
>> never
>> found http://www.lartc.org/
>
>
> (which unfortunately is rather outdated)

s/outdated/dead

> I do not buy the fact that some developers do not provide documentation of
> the features they are adding potentially on purpose, truth is probably much
> simpler, you worked on X, you have now moved on and work on Y.

IMHO one cause of the problem is that many nice features get
implemented by contractors.
The customer is only interested in the kernel interface and has it's
own (proprietary) userland.
Don't get me wrong, this is perfectly fine.
But it has the side effect that a) the customer is not interested in
man-pages or other open docs/tools
(he does not want to reveal his magic sauce too early) and
b) the contractor is therefore not paid to provide docs.

> If nobody pays enough attention to what gets added through netdev, iproute2,
> ethtool, man-pages and enforces the need for documentation, then comes the
> current status quo where not all features are documented, until some
> benevolant person realizes this needs fixing. Considering the high volume of
> the list, this is all understandable.
>
> There could probably be some programmatical ways to enforce such
> documentation by only allowing patches coming with, say kernel-doc content
> along the code, and have man-pages and other projects scan for new
> kernel-doc entries they have no reference for.
> --
> Florian
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Thanks,
//richard

Re: critic on documentation of the network stack

[Arkadiusz Miskiewicz]

On Friday 24 of January 2014, Hannes Frederic Sowa wrote:
> Hello!
> 
> After net-next is closed I wanted to put the following link here:
> 
>   <http://linux.slashdot.org/comments.pl?sid=4356053&cid=45184693>
> 
> I don't want to start a flamefest or come too close to someone but I
> fear some of the critic is reasonable.  Maybe we can do better (I have
> to admit, I also hate writing documentation, e.g. have not yet send the
> IP_PMTUDISC_INTERFACE man-page patches).
> 
> I try to start with some constructive discussion:
> 
> There are some great features in the network stack that some people miss
> because of lack documentation. One possible solution is documentation
> directly in the kernel, but mostly this is just written as a reference
> and the real wonderful stuff is only achieved by putting lots of those
> features correclty together.
> 
> Maybe this is the second or third time this was proposed but I'll try
> again: Would it make sense to just start slow and setup a wiki

Some other project never merge patches if there is no documentation.

Maintainers could do the same, so eg. kernel patch is merged only if there is 
man-pages patch send, too (or other kind of documentation).

-- 
Arkadiusz Miśkiewicz, arekm / maven.pl

Re: critic on documentation of the network stack

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 1721 bytes --]

On Fri, 2014-01-24 at 04:23 +0100, Hannes Frederic Sowa wrote:
> Hello!
> 
> After net-next is closed I wanted to put the following link here:
> 
>   <http://linux.slashdot.org/comments.pl?sid=4356053&cid=45184693>
> 
> I don't want to start a flamefest or come too close to someone but I
> fear some of the critic is reasonable.  Maybe we can do better (I have
> to admit, I also hate writing documentation, e.g. have not yet send the
> IP_PMTUDISC_INTERFACE man-page patches).
> 
> I try to start with some constructive discussion:
> 
> There are some great features in the network stack that some people miss
> because of lack documentation. One possible solution is documentation
> directly in the kernel, but mostly this is just written as a reference
> and the real wonderful stuff is only achieved by putting lots of those
> features correclty together.

I think the reference documentation is also severely lacking.  I have
occasionally filled in documentation of the ethtool API (as kernel-doc
comments) after having to effectively reverse-engineer it from the
implementations.

> Maybe this is the second or third time this was proposed but I'll try
> again: Would it make sense to just start slow and setup a wiki where we
> just throw in the various snippets we use for testing while developing
> patches, maybe with a bit of background information?
[...]

There seems to be an appropriate wiki available:
http://www.linuxfoundation.org/collaborate/workgroups/networking

But developers will need to get into the habit of using it (maybe with
pressure from David Miller).

Ben.

-- 
Ben Hutchings
If the facts do not conform to your theory, they must be disposed of.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]