[conntrack-tools] conntrack.8: Document --stats counters

[conntrack-tools] conntrack.8: Document --stats counters

[Phil Sutter]

Provide at least a brief description of each counter's meaning based on
code-analysis in kernel's nf_conntrack_core.c.

Signed-off-by: Phil Sutter <phil@nwl.cc>
---
While most values are pretty obvious, I am not entirely sure I got the
insert_failed and drop reasons right.
---
 conntrack.8 | 37 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)

diff --git a/conntrack.8 b/conntrack.8
index 2bfd80e5d6aa4..b562e16839a32 100644
--- a/conntrack.8
+++ b/conntrack.8
@@ -108,7 +108,42 @@ Flush the whole given table
 Show the table counter.
 .TP
 .BI "-S, --stats "
-Show the in-kernel connection tracking system statistics.
+Show the in-kernel connection tracking system statistics. The returned values
+for each CPU are:
+.RS
+.TP
+.B found
+Number of successful conntrack table lookups
+.TP
+.B invalid
+Number of invalid packets encountered
+.TP
+.B insert
+Number of entries inserted into conntrack table
+.TP
+.B insert_failed
+Number of failed inserts (clash resolution failure, conntrack extensions in
+inconsistent state, entry in dying state, oversized hash bucket encountered)
+.TP
+.B drop
+Number of packets dropped (clash resolution failure, removed conntrack helper,
+stress, TCP connection aborted)
+.TP
+.B early_drop
+Number of packets dropped up front due to full table
+.TP
+.B error
+Number of invalid ICMP/ICMPv6 packets received
+.TP
+.B search_restart
+Number of times a table lookup had to be restarted due to table reorg
+.TP
+.B clash_resolve
+Number of entry insert clashes resolved
+.TP
+.B chaintoolong
+Number of oversized hash bucket encounters
+.RE
 .TP
 .BI "-R, --load-file "
 Load entries from a given file. To read from stdin, "\-" should be specified.
-- 
2.54.0

Re: [conntrack-tools] conntrack.8: Document --stats counters

[Florian Westphal]

Phil Sutter <phil@nwl.cc> wrote:
> Provide at least a brief description of each counter's meaning based on
> code-analysis in kernel's nf_conntrack_core.c.
> 
> Signed-off-by: Phil Sutter <phil@nwl.cc>
> ---
> While most values are pretty obvious, I am not entirely sure I got the
> insert_failed and drop reasons right.

Thanks for working on this.   I think the hardest part is to explain
how to interpret this, i.e. which counters may indicate issues and which
ones do not.

> ---
>  conntrack.8 | 37 ++++++++++++++++++++++++++++++++++++-
>  1 file changed, 36 insertions(+), 1 deletion(-)
> 
> diff --git a/conntrack.8 b/conntrack.8
> index 2bfd80e5d6aa4..b562e16839a32 100644
> --- a/conntrack.8
> +++ b/conntrack.8
> @@ -108,7 +108,42 @@ Flush the whole given table
>  Show the table counter.
>  .TP
>  .BI "-S, --stats "
> -Show the in-kernel connection tracking system statistics.
> +Show the in-kernel connection tracking system statistics. The returned values
> +for each CPU are:
> +.RS
> +.TP
> +.B found
> +Number of successful conntrack table lookups

We don't count those anymore, its too expensive.
This is only incremented when a new connection cannot reuse
the tuple and has to create a new nat mapping.

See nf_conntrack_tuple_taken().

> +.B insert
> +Number of entries inserted into conntrack table

IIRC its only incremented for insertions from netlink path.

> +.B insert_failed
> +Number of failed inserts (clash resolution failure, conntrack extensions in
> +inconsistent state, entry in dying state, oversized hash bucket encountered)

oversized hash buckets should not be mentioned here, they have their own
counter.  Should probably not increment both counters in kernel when it
happens.

Maybe: "Number of NEW connections dropped because of clashes with existing entry."?

> +.B drop
> +Number of packets dropped (clash resolution failure, removed conntrack helper,
> +stress, TCP connection aborted)

Yes, I think we need to fix this in the kernel and not increment two
counters for the same reason.

Drop should mostly mean "incomplete packet / out of memory".

> +.B early_drop
> +Number of packets dropped up front due to full table

AFAICS its number of connections dropped because of a full table.

> +.B search_restart
> +Number of times a table lookup had to be restarted due to table reorg

Not sure about this one.  This is an implementation detail (SLAB_TYPESAFE_BY_RCU).
This can happen at any time when entry is removed from table and
other CPU is re-instantiating a new conection using the just-unlinked
entry.

> +.TP
> +.B clash_resolve
> +Number of entry insert clashes resolved

Maybe mention that this is not a cause for alarm and mostly
expected with DNS these days?

> +.TP
> +.B chaintoolong
> +Number of oversized hash bucket encounters

Maybe mention that this happens only on insert and results in conntrack
to drop the packet.