Re: Multicast routing stops functioning after 4G multicast packets recived.

["Bob Falken" <NetFestivalHaveFun@gmx.com>]

OK, so at the exact time that the incoming interface for multicast packet count reaches 2^32, 
the /proc/net/ip_mr_cache stops updating. 
after a while, one by one the multicast groups in ip_mr_cache disapperes, and after 227sec all of them are gone. 


perf script net_dropmonitor:
-----------
# ========
# captured on: Sat Dec 21 23:27:37 2013
# ========
#
Starting trace (Ctrl-C to dump results)
Warning:
Processed 788648 events and lost 118 chunks!
 
Check IO/CPU overload!
 
Gathering kallsyms data
35200/35200
                 LOCATION                    OFFSET                     COUNT
                   _stext      18446744071578845580                         6
                   _stext      18446744071578843536                    785790
                   _stext      18446744071578843530                         1
 
 
 
-------------
 
netstat -s:
Ip:
    622406 total packets received
    2 with invalid addresses
    0 forwarded
    0 incoming packets discarded
    599574 incoming packets delivered
    520762 requests sent out
    8 dropped because of missing route
Icmp:
    19361 ICMP messages received
    0 input ICMP message failed.
    ICMP input histogram:
        echo requests: 8415
        echo replies: 10946
    19361 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        echo request: 10946
        echo replies: 8415
IcmpMsg:
        InType0: 10946
        InType8: 8415
        OutType0: 8415
        OutType8: 10946
Tcp:
    15 active connections openings
    15 passive connection openings
    0 failed connection attempts
    0 connection resets received
    29 connections established
    477938 segments received
    482321 segments send out
    4 segments retransmited
    0 bad segments received.
    0 resets sent
Udp:
    586 packets received
    0 packets to unknown port received.
    0 packet receive errors
    649 packets sent
UdpLite:
TcpExt:
    15862 delayed acks sent
    Quick ack mode was activated 1 times
    1 packets directly queued to recvmsg prequeue.
    390374 packet headers predicted
    1767 acknowledgments not containing data payload received
    58169 predicted acknowledgments
    4 congestion windows recovered without slow start after partial ack
    4 other TCP timeouts
    1 DSACKs sent for old packets
    4 DSACKs received
    TCPSackShiftFallback: 3
IpExt:
    InNoRoutes: 1
    InMcastPkts: 40015
    OutMcastPkts: 18427
    InBcastPkts: 80035
    InOctets: 1116615859
    OutOctets: 33742922
    InMcastOctets: 1046924948
    OutMcastOctets: 734556
    InBcastOctets: 7255577
  
 
--------------------- 
----- Original Message -----
From: Hannes Frederic Sowa
Sent: 12/19/13 06:32 PM
To: Eric Dumazet
Subject: Re: Multicast routing stops functioning after 4G multicast packets recived.
 On Thu, Dec 19, 2013 at 09:24:18AM -0800, Eric Dumazet wrote:
> On Thu, 2013-12-19 at 17:28 +0100, Bob Falken wrote:
> > The only reason why i give information about 2.6.36.4 is that its the
> > only latest kernel that was functioning properly.
> > i.e kernel >= 2.6.37 is not woking. so its a bisecting of the kernel
> > versions to help a coder see when/where the isse was implemented in
> > the kernel.
> > 
> > I do not need a backport patch for an old kernel, I generally only
> > need the issue looked into and get fixed so that I dont have to use an
> > old kernel. :)
> > 
> > I have no issue reproducing the issue on the recent kernels. however i
> > have not tried the GIT kernel.
> > 
> > I restarted the server just a moment ago. i will install and run
> > dropwatch and provide feedback in about 17hours. 
> 
> You said that "cat /proc/net/ip_mr_cache" gives nothing at all after
> 2^32 packets ?
> 
> Thats a bit scary ... maybe a 32bit refcnt overflow, because of some
> imbalance...

That's my thought, too. :/

The ipmr.c rcu conversion happend in 2.6.37.