Re: ICMP echo reply fails

[Eric Dumazet <eric.dumazet@gmail.com>]

Le vendredi 26 mars 2010 à 23:06 +0100, Eric Dumazet a écrit :
> Le vendredi 26 mars 2010 à 16:48 -0500, Andy Fleming a écrit :
> > For various reasons, we have been running a stress test on one of our
> > boards.  The test consists of initiating 2-3 flood pings from a
> > Windows box running Cygwin, plus one additional ping we use as a
> > "heartbeat".  The ping flood is overwhelming our board (we're dropping
> > packets at a prodigious rate), but the board continues to respond for
> > a while.  In addition, we are running a script on the  board which
> > alternates bringing up and bringing down the interface every ten
> > seconds.  After a highly variable amount of time, the board stops
> > replying to the pings.  We suspected a driver issue, however, on
> > closer inspection, we are still able to send and receive packets (I
> > can ping *from* the board to the PC, and I can *telnet* from the PC to
> > the board).  We tried pinging the board from another PC, and it also
> > failed.  Essentially, ICMP echo requests are being ignored (A glance
> > at memory indicates that packets are arriving, but no packets are
> > being enqueued to the ethernet controller).  We still have a lot more
> > debugging to do, but I was wondering if anyone had ever seen something
> > like this, or might be quicker to realize the obvious mistake we're
> > making.
> > 
> > Thanks,
> > Andy Fleming
> 
> 
> kernel version ?
> 
> NIC driver ?
> 
> Are ICMP echo request received ? (grep Icmp /proc/net/snmp)
> 

vi +1166 net/ipv4/icmp.c

        /* Enough space for 2 64K ICMP packets, including
         * sk_buff struct overhead.
         */
        sk->sk_sndbuf =
                (2 * ((64 * 1024) + sizeof(struct sk_buff)));


If many ICMP replies are lost/leaked by your driver when doing up/down
things, ICMP socket can consume all its sndbuf reserve and no more icmp
replies can be sent (a reboot is needed)

You could try changing sk->sk_sndbuf to 0x7FFFFFFF  to see if the icmp
replies survive longer to your tests. If this is the case, then find the
leaks in your driver (tx path, maybe you forgot to free skbs in some
reset cases ?)



We should add a SNMP counter for failed ip_append() calls in
icmp_push_reply()...