From: Paolo Abeni <pabeni@redhat.com>
To: Eric Dumazet <eric.dumazet@gmail.com>,
Marc Haber <mh+netdev@zugschlus.de>
Cc: netdev@vger.kernel.org
Subject: Re: After a while of system running no incoming UDP any more?
Date: Fri, 28 Jul 2017 10:15:02 +0200 [thread overview]
Message-ID: <1501229702.2597.3.camel@redhat.com> (raw)
In-Reply-To: <1501229105.12695.23.camel@edumazet-glaptop3.roam.corp.google.com>
Hi,
On Fri, 2017-07-28 at 01:05 -0700, Eric Dumazet wrote:
> On Fri, 2017-07-28 at 08:26 +0200, Marc Haber wrote:
> > On Mon, Jul 24, 2017 at 04:19:10PM +0200, Paolo Abeni wrote:
> > > Once that a system enter the buggy status, do the packets reach the
> > > relevant socket's queue?
> > >
> > > ss -u
> > > nstat |grep -e Udp -e Ip
> > >
> > > will help checking that.
> >
> > I now have the issue on one machine, a Xen guest acting as authoritative
> > nameserver for my domains. Here are the outputs during normal use, with
> > artificial queries coming in on eth0:
> >
> > [9/1075]mh@impetus:~ $ ss -u
> > Recv-Q Send-Q Local Address:Port Peer Address:Port
> > 0 0 127.0.0.1:56547 127.0.0.1:domain
> > 0 0 216.231.132.60:27667 198.41.0.4:domain
> > 0 0 216.231.132.60:44121 8.8.8.8:domain
> > 0 0 216.231.132.60:29814 198.41.0.4:domain
> > [10/1076]mh@impetus:~ $ ss -u
> > Recv-Q Send-Q Local Address:Port Peer Address:Port
> > [11/1076]mh@impetus:~ $ ss -u
> > Recv-Q Send-Q Local Address:Port Peer Address:Port
> > [12/1076]mh@impetus:~ $ ss -u
> > Recv-Q Send-Q Local Address:Port Peer Address:Port
> > [13/1076]mh@impetus:~ $ ss -u
> > Recv-Q Send-Q Local Address:Port Peer Address:Port
> > [14/1076]mh@impetus:~ $ nstat | grep -e Udp -e Ip
> > IpInReceives 400688 0.0
> > IpInAddrErrors 18567 0.0
> > IpInUnknownProtos 3 0.0
> > IpInDelivers 330634 0.0
> > IpOutRequests 283637 0.0
> > UdpInDatagrams 145860 0.0
> > UdpNoPorts 1313 0.0
> > UdpInErrors 9356 0.0
> > UdpOutDatagrams 153093 0.0
> > UdpIgnoredMulti 34148 0.0
> > Ip6InReceives 161178 0.0
> > Ip6InNoRoutes 8 0.0
> > Ip6InDelivers 73841 0.0
> > Ip6OutRequests 77575 0.0
> > Ip6InMcastPkts 87332 0.0
> > Ip6OutMcastPkts 109 0.0
> > Ip6InOctets 21880674 0.0
> > Ip6OutOctets 9633059 0.0
> > Ip6InMcastOctets 9371483 0.0
> > Ip6OutMcastOctets 6636 0.0
> > Ip6InNoECTPkts 161202 0.0
> > Ip6InECT1Pkts 15 0.0
> > Ip6InECT0Pkts 11 0.0
> > Ip6InCEPkts 4 0.0
> > Udp6InDatagrams 11725 0.0
> > Udp6NoPorts 2 0.0
> > Udp6InErrors 1989 0.0
> > Udp6OutDatagrams 14483 0.0
> > IpExtInBcastPkts 34148 0.0
> > IpExtInOctets 47462716 0.0
> > IpExtOutOctets 31262696 0.0
> > IpExtInBcastOctets 7476059 0.0
> > IpExtInNoECTPkts 400178 0.0
> > IpExtInECT1Pkts 22 0.0
> > IpExtInECT0Pkts 481 0.0
> > IpExtInCEPkts 14 0.0
> > [15/1077]mh@impetus:~ $ nstat | grep -e Udp -e Ip
> > IpInReceives 25 0.0
> > IpInDelivers 25 0.0
> > IpOutRequests 16 0.0
> > UdpInDatagrams 1 0.0
> > UdpInErrors 24 0.0
> > UdpOutDatagrams 16 0.0
> > Ip6InReceives 15 0.0
> > Ip6InDelivers 14 0.0
> > Ip6OutRequests 12 0.0
> > Ip6InMcastPkts 1 0.0
> > Ip6InOctets 1219 0.0
> > Ip6OutOctets 4384 0.0
> > Ip6InMcastOctets 131 0.0
> > Ip6InNoECTPkts 15 0.0
> > IpExtInOctets 11779 0.0
> > IpExtOutOctets 1023 0.0
> > IpExtInNoECTPkts 25 0.0
> > [16/1077]mh@impetus:~ $ nstat | grep -e Udp -e Ip
> > IpInReceives 24 0.0
> > IpInDelivers 24 0.0
> > IpOutRequests 18 0.0
> > UdpInErrors 22 0.0
> > UdpOutDatagrams 16 0.0
> > Ip6InReceives 15 0.0
> > Ip6InDelivers 12 0.0
> > Ip6OutRequests 10 0.0
> > Ip6InMcastPkts 3 0.0
> > Ip6InOctets 1160 0.0
> > Ip6OutOctets 2456 0.0
> > Ip6InMcastOctets 216 0.0
> > Ip6InNoECTPkts 15 0.0
> > IpExtInOctets 8612 0.0
> > IpExtOutOctets 1127 0.0
> > IpExtInNoECTPkts 24 0.0
> > [17/1077]mh@impetus:~ $ nstat | grep -e Udp -e Ip
> > IpInReceives 5 0.0
> > IpInDelivers 4 0.0
> > IpOutRequests 3 0.0
> > UdpNoPorts 1 0.0
> > UdpInErrors 2 0.0
> > UdpOutDatagrams 1 0.0
> > Ip6InReceives 12 0.0
> > Ip6InDelivers 12 0.0
> > Ip6OutRequests 10 0.0
> > Ip6InOctets 944 0.0
> > Ip6OutOctets 2364 0.0
> > Ip6InNoECTPkts 12 0.0
> > IpExtInOctets 429 0.0
> > IpExtOutOctets 226 0.0
> > IpExtInNoECTPkts 5 0.0
> > [18/1077]mh@impetus:~ $
> >
> > And here, hopefully a bit more helpful:
> >
> > [19/1078]mh@impetus:~ $ ss -u ; nstat | grep -e Udp -e Ip ; dig +time=2 @8.8.8.8 zugschlus.de mx ; ss -u ; nstat | grep -e Udp -e Ip
> > Recv-Q Send-Q Local Address:Port Peer Address:Port
> > 0 0 216.231.132.60:27333 198.41.0.4:domain
> > 0 0 216.231.132.60:38101 198.41.0.4:domain
> > 0 0 216.231.132.60:15836 198.41.0.4:domain
> > 0 0 216.231.132.60:50655 8.8.8.8:domain
> > 0 0 216.231.132.60:41953 198.41.0.4:domain
> > 0 0 216.231.132.60:6888 198.41.0.4:domain
> > 0 0 216.231.132.60:51441 198.41.0.4:domain
> > 0 0 216.231.132.60:42503 198.41.0.4:domain
> > 0 0 216.231.132.60:12575 198.41.0.4:domain
> > 0 0 216.231.132.60:13857 198.41.0.4:domain
> > 0 0 216.231.132.60:16419 192.36.148.17:domain
> > 0 0 216.231.132.60:39227 198.41.0.4:domain
> > 0 0 127.0.0.1:54608 127.0.0.1:domain
> > 0 0 216.231.132.60:20818 198.41.0.4:domain
> > 0 0 216.231.132.60:56662 198.41.0.4:domain
> > 0 0 216.231.132.60:48259 192.36.148.17:domain
> > 0 0 216.231.132.60:37803 198.41.0.4:domain
> > IpInReceives 59 0.0
> > IpInAddrErrors 1 0.0
> > IpInDelivers 56 0.0
> > IpOutRequests 50 0.0
> > UdpInDatagrams 1 0.0
> > UdpInErrors 50 0.0
> > UdpOutDatagrams 47 0.0
> > UdpIgnoredMulti 1 0.0
> > Ip6InReceives 75 0.0
> > Ip6InDelivers 73 0.0
> > Ip6OutRequests 64 0.0
> > Ip6InMcastPkts 2 0.0
> > Ip6InOctets 7837 0.0
> > Ip6OutOctets 11876 0.0
> > Ip6InMcastOctets 279 0.0
> > Ip6InNoECTPkts 75 0.0
> > Udp6InErrors 3 0.0
> > IpExtInBcastPkts 1 0.0
> > IpExtInOctets 18447 0.0
> > IpExtOutOctets 3478 0.0
> > IpExtInBcastOctets 183 0.0
> > IpExtInNoECTPkts 59 0.0
> >
> > ; <<>> DiG 9.10.3-P4-Debian <<>> +time=2 @8.8.8.8 zugschlus.de mx
> > ; (1 server found)
> > ;; global options: +cmd
> > ;; connection timed out; no servers could be reached
> > Recv-Q Send-Q Local Address:Port Peer Address:Port
> > 0 0 216.231.132.60:7879 202.12.27.33:domain
> > 0 0 216.231.132.60:32711 202.12.27.33:domain
> > 0 0 216.231.132.60:54238 202.12.27.33:domain
> > 0 0 216.231.132.60:30948 192.228.79.201:domain
> > 0 0 216.231.132.60:4106 202.12.27.33:domain
> > 0 0 216.231.132.60:6667 202.12.27.33:domain
> > 0 0 216.231.132.60:2090 192.228.79.201:domain
> > 0 0 216.231.132.60:60459 192.228.79.201:domain
> > 0 0 216.231.132.60:16427 202.12.27.33:domain
> > 0 0 216.231.132.60:9019 202.12.27.33:domain
> > 0 0 216.231.132.60:2113 202.12.27.33:domain
> > 0 0 216.231.132.60:34907 202.12.27.33:domain
> > 0 0 216.231.132.60:34654 202.12.27.33:domain
> > 0 0 216.231.132.60:47725 202.12.27.33:domain
> > 0 0 216.231.132.60:35774 202.12.27.33:domain
> > IpInReceives 38 0.0
> > IpInDelivers 38 0.0
> > IpOutRequests 38 0.0
> > UdpInDatagrams 2 0.0
> > UdpInErrors 34 0.0
> > UdpOutDatagrams 36 0.0
> > Ip6InReceives 14 0.0
> > Ip6InDelivers 13 0.0
> > Ip6OutRequests 13 0.0
> > Ip6InMcastPkts 1 0.0
> > Ip6InOctets 1046 0.0
> > Ip6OutOctets 6277 0.0
> > Ip6InMcastOctets 133 0.0
> > Ip6InNoECTPkts 13 0.0
> > Ip6InECT0Pkts 1 0.0
> > Udp6InDatagrams 1 0.0
> > Udp6OutDatagrams 1 0.0
> > IpExtInOctets 15963 0.0
> > IpExtOutOctets 2397 0.0
> > IpExtInNoECTPkts 37 0.0
> > IpExtInECT0Pkts 1 0.0
> > [20/1079]mh@impetus:~ $
> >
> > I am afraid I cannot keep this state for much longer than a few
> > additional hours as this is an authoritative name server...
> >
> > Greetings
> > Marc
> >
>
> To confirm the refcount issue, you might send us :
>
> cat /proc/net/udp6
>
> Normally, the refcount column should have a value of 2
I think that the leaked sockets are still unhashed on close() (via
udp_lib_close() -> sk_common_release() ->sk_proto->unhash()), so they
should not be listed there ?!? (Rush answer, subject to lack of coffee
issues)
Thanks,
Paolo
next prev parent reply other threads:[~2017-07-28 8:15 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-24 12:09 After a while of system running no incoming UDP any more? Marc Haber
2017-07-24 14:19 ` Paolo Abeni
2017-07-25 11:57 ` Marc Haber
2017-07-25 12:17 ` Paolo Abeni
2017-07-26 8:10 ` Marc Haber
2017-07-26 8:33 ` Paolo Abeni
2017-07-28 6:26 ` Marc Haber
2017-07-28 8:05 ` Eric Dumazet
2017-07-28 8:15 ` Paolo Abeni [this message]
2017-07-28 8:41 ` Eric Dumazet
2017-07-28 8:07 ` Paolo Abeni
2017-07-28 12:14 ` Marc Haber
2017-08-11 14:34 ` Marc Haber
2017-08-11 20:07 ` Marc Haber
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1501229702.2597.3.camel@redhat.com \
--to=pabeni@redhat.com \
--cc=eric.dumazet@gmail.com \
--cc=mh+netdev@zugschlus.de \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.