sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo?

sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo?

[Michael Tokarev]

On our gateway machine, wich is running 2.6 kernel, I'm seeing quite
several messages like this:

kernel: 192.168.4.2 sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo
last message repeated 3 times
kernel: 192.168.4.2 sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo
last message repeated 3 times

kernel: 81.13.90.174 sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo
gate last message repeated 10 times

kernel: 81.13.90.174 sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo
last message repeated 9 times

All the IP addresses mentioned are local to this box.  It's always
like this, type11 code0 to 0.0.0.0 on lo.

The box is a gateway, which have "external" interface, several "LAN"
ifaces, DMZ, dialin modem pool and accepts vtund connections.  And
it has iptables-based firewall and NAT enabled.  Here's a lis of
network-related modules:

ppp_deflate             4736  0
zlib_deflate           21528  1 ppp_deflate
zlib_inflate           16640  1 ppp_deflate
ppp_async               8576  0
crc_ccitt               1920  1 ppp_async
ppp_generic            19860  2 ppp_deflate,ppp_async
slhc                    6144  1 ppp_generic
tun                     8320  13
crc32                   3968  1 tun
ipt_ECN                 2944  163
iptable_mangle          2304  1
ipt_REJECT              4096  1
ipt_LOG                 6272  3
ipt_state               1664  3
ipt_comment             1536  22
iptable_filter          2432  1
iptable_nat            19420  1
ipt_NOTRACK             1920  3
iptable_raw             1792  1
ip_tables              18304  10 ipt_ECN,iptable_mangle,ipt_REJECT,ipt_LOG,ipt_state,ipt_comment,iptable_filter,iptable_nat,ipt_NOTRACK,iptable_raw
md5                     3840  1
ipv6                  218688  12
ip_conntrack           37560  3 ipt_state,iptable_nat,ipt_NOTRACK
aes_i586               38656  1
airo                   63520  0
3c509                  11476  0

I've seen several posts about problems *like* this, but
they're all not from the same box (usually it's some problem
with *another* device on the network which is sending bogus
packets, or with hubs/switches or cables).  In this case,
it looks like it is this same box who's generating those
bad packets.

So far, I wasn't able to relate this message with any particular
network activity.  It happens "randomly".

What it can be?

Thanks.

/mjt

Re: sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo?

[Richard B. Johnson]

On Wed, 6 Jul 2005, Michael Tokarev wrote:

> On our gateway machine, wich is running 2.6 kernel, I'm seeing quite
> several messages like this:
>
> kernel: 192.168.4.2 sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo
> last message repeated 3 times
> kernel: 192.168.4.2 sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo
> last message repeated 3 times
>
> kernel: 81.13.90.174 sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo
> gate last message repeated 10 times
>
> kernel: 81.13.90.174 sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo
> last message repeated 9 times
>
> All the IP addresses mentioned are local to this box.  It's always
> like this, type11 code0 to 0.0.0.0 on lo.
>
> The box is a gateway, which have "external" interface, several "LAN"
> ifaces, DMZ, dialin modem pool and accepts vtund connections.  And
> it has iptables-based firewall and NAT enabled.  Here's a lis of
> network-related modules:
>
> ppp_deflate             4736  0
> zlib_deflate           21528  1 ppp_deflate
> zlib_inflate           16640  1 ppp_deflate
> ppp_async               8576  0
> crc_ccitt               1920  1 ppp_async
> ppp_generic            19860  2 ppp_deflate,ppp_async
> slhc                    6144  1 ppp_generic
> tun                     8320  13
> crc32                   3968  1 tun
> ipt_ECN                 2944  163
> iptable_mangle          2304  1
> ipt_REJECT              4096  1
> ipt_LOG                 6272  3
> ipt_state               1664  3
> ipt_comment             1536  22
> iptable_filter          2432  1
> iptable_nat            19420  1
> ipt_NOTRACK             1920  3
> iptable_raw             1792  1
> ip_tables              18304  10 ipt_ECN,iptable_mangle,ipt_REJECT,ipt_LOG,ipt_state,ipt_comment,iptable_filter,iptable_nat,ipt_NOTRACK,iptable_raw
> md5                     3840  1
> ipv6                  218688  12
> ip_conntrack           37560  3 ipt_state,iptable_nat,ipt_NOTRACK
> aes_i586               38656  1
> airo                   63520  0
> 3c509                  11476  0
>
> I've seen several posts about problems *like* this, but
> they're all not from the same box (usually it's some problem
> with *another* device on the network which is sending bogus
> packets, or with hubs/switches or cables).  In this case,
> it looks like it is this same box who's generating those
> bad packets.
>
> So far, I wasn't able to relate this message with any particular
> network activity.  It happens "randomly".
>
> What it can be?
>
> Thanks.
>

Are you sure `lo` is configured properly, i.e.

ifconfig lo 127.0.0.1 netmask 255.0.0.0
route add -net 127.0.0.0 netmask 255.0.0.0 dev lo

There should be no LAN routes going through lo.
It looks as though there are:
> kernel: 192.168.4.2 sent an invalid ICMP type 11, code 0 error
> to a broadcast: 0.0.0.0 on lo
                              |________ 192.168.4.2 pinged lo

Only the 127.0.0.0 network should be routed through the loop-back
device.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.12 on an i686 machine (5537.79 BogoMips).
  Notice : All mail here is now cached for review by Dictator Bush.
                  98.36% of all statistics are fiction.

Re: sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo?

[Denis Vlasenko]

On Wednesday 06 July 2005 20:27, Richard B. Johnson wrote:
> Only the 127.0.0.0 network should be routed through the loop-back
> device.

This is the normal dose of disinformation embedded into
otherwise useful reply from Richard. Sometimes I wonder
whether he does this for fun for for some other dark
and obscure reason.

Anyway, lo is perfetly happy with packets other than 127.x.x.x
going thru it.

# ip a
1: if: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:04:23:15:ba:76 brd ff:ff:ff:ff:ff:ff
    inet 172.17.13.22/16 brd 172.17.255.255 scope global if
2: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo

# ip r
172.17.0.0/16 dev if  proto kernel  scope link  src 172.17.13.22
default via 172.17.0.1 dev if

# ping -c4 172.17.13.22
PING 172.17.13.22 (172.17.13.22) 56(84) bytes of data.
64 bytes from 172.17.13.22: icmp_seq=1 ttl=64 time=0.155 ms
64 bytes from 172.17.13.22: icmp_seq=2 ttl=64 time=0.113 ms
64 bytes from 172.17.13.22: icmp_seq=3 ttl=64 time=0.122 ms
64 bytes from 172.17.13.22: icmp_seq=4 ttl=64 time=0.121 ms
--- 172.17.13.22 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3011ms
rtt min/avg/max/mdev = 0.113/0.127/0.155/0.021 ms

On the other console:

# tcpdump -nlilo
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type EN10MB (Ethernet), capture size 68 bytes
11:20:37.669996 IP 172.17.13.22 > 172.17.13.22: icmp 64: echo request seq 1
11:20:37.670093 IP 172.17.13.22 > 172.17.13.22: icmp 64: echo reply seq 1
11:20:38.671066 IP 172.17.13.22 > 172.17.13.22: icmp 64: echo request seq 2
11:20:38.671125 IP 172.17.13.22 > 172.17.13.22: icmp 64: echo reply seq 2
11:20:39.672327 IP 172.17.13.22 > 172.17.13.22: icmp 64: echo request seq 3
11:20:39.672389 IP 172.17.13.22 > 172.17.13.22: icmp 64: echo reply seq 3
11:20:40.681857 IP 172.17.13.22 > 172.17.13.22: icmp 64: echo request seq 4
11:20:40.681922 IP 172.17.13.22 > 172.17.13.22: icmp 64: echo reply seq 4
--
vda

Re: sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo?

[Michael Tokarev]

Richard B. Johnson wrote:
> On Wed, 6 Jul 2005, Michael Tokarev wrote:
> 
>> kernel: 192.168.4.2 sent an invalid ICMP type 11, code 0 error to a
>> broadcast: 0.0.0.0 on lo
[]
>> All the IP addresses mentioned are local to this box.


[]
> Are you sure `lo` is configured properly, i.e.

Yes.  More, rp_filter is activated on all interfaces.

> ifconfig lo 127.0.0.1 netmask 255.0.0.0
> route add -net 127.0.0.0 netmask 255.0.0.0 dev lo
> 
> There should be no LAN routes going through lo.

There's no.

> It looks as though there are:
> 
>> kernel: 192.168.4.2 sent an invalid ICMP type 11, code 0 error
>> to a broadcast: 0.0.0.0 on lo
> 
>                              |________ 192.168.4.2 pinged lo
> 
> Only the 127.0.0.0 network should be routed through the loop-back
> device.

Again: All the IP addresses mentioned are local to this box.

If you ping an IP address on your eth0, the traffic will "go"
over loopback.  You can verify it using tcpdump:

1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:a0:d2:1d:a7:63 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.1/24 brd 192.168.1.255 scope global eth0

# tcpdump -npi lo proto ICMP
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes
15:55:13.679234 IP 192.168.1.1 > 192.168.1.1: icmp 64: echo request seq 1
15:55:13.679269 IP 192.168.1.1 > 192.168.1.1: icmp 64: echo reply seq 1

$ ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.060 ms
--- 192.168.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.060/0.060/0.060/0.000 ms

/mjt

Re: sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo?

[Richard B. Johnson]

On Thu, 7 Jul 2005, Michael Tokarev wrote:

> Richard B. Johnson wrote:
>> On Wed, 6 Jul 2005, Michael Tokarev wrote:
>>
>>> kernel: 192.168.4.2 sent an invalid ICMP type 11, code 0 error to a
>>> broadcast: 0.0.0.0 on lo
> []
>>> All the IP addresses mentioned are local to this box.
>
> []
>> Are you sure `lo` is configured properly, i.e.
>
> Yes.  More, rp_filter is activated on all interfaces.
>
>> ifconfig lo 127.0.0.1 netmask 255.0.0.0
>> route add -net 127.0.0.0 netmask 255.0.0.0 dev lo
>>
>> There should be no LAN routes going through lo.
>
> There's no.
>
>> It looks as though there are:
>>
>>> kernel: 192.168.4.2 sent an invalid ICMP type 11, code 0 error
>>> to a broadcast: 0.0.0.0 on lo
>>
>>                              |________ 192.168.4.2 pinged lo
>>
>> Only the 127.0.0.0 network should be routed through the loop-back
>> device.
>
> Again: All the IP addresses mentioned are local to this box.
>
> If you ping an IP address on your eth0, the traffic will "go"
> over loopback.  You can verify it using tcpdump:
>

If you ping an IP address on your computer, the traffic will go
through lo. However, I think that the IP address shown is
the result of an instrumentation error because it is impossible
to put, for instance your 192.168.1.1, through a 127.0.0.0 network,
the ONLY route through lo. This shows that 'local' traffic bypasses
the lo route filtering altogether. You can verify this by
deleting the lo route altogether, you can still ping the local
addresses.

Somebody else mentioned that lo was 'perfectly happy' to
carry whatever. The fact that something bogus appears on
lo can be a sign of a misconfiguration error, just as
the reserved 127.0.0.0 network must never appear on ethernet.

In the case of 0.0.0.0 (a possible broadcast), there is
no "local" address that could cause a bypass via lo. Instead,
any such traffic should have been on the ethernet wire. This
shows the possible configuration error that I mentioned.


> 1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
>    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
                     ^^^^^^^^^^^^^^^^
>    inet 127.0.0.1/8 scope host lo

This looks as though there is no netmask set. My configuration
shows:

lo        Link encap:Local Loopback
           inet addr:127.0.0.1  Mask:255.0.0.0
           inet6 addr: ::1/128 Scope:Host
           UP LOOPBACK RUNNING  MTU:16436  Metric:1

This is a possible configuration error.

> 2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
>    link/ether 00:a0:d2:1d:a7:63 brd ff:ff:ff:ff:ff:ff
>    inet 192.168.1.1/24 brd 192.168.1.255 scope global eth0
>
> # tcpdump -npi lo proto ICMP
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes
> 15:55:13.679234 IP 192.168.1.1 > 192.168.1.1: icmp 64: echo request seq 1
> 15:55:13.679269 IP 192.168.1.1 > 192.168.1.1: icmp 64: echo reply seq 1
>
[SNIPPED rest]

>
> /mjt
>

Cheers,
Dick Johnson
Penguin : Linux version 2.6.12 on an i686 machine (5537.79 BogoMips).
  Notice : All mail here is now cached for review by Dictator Bush.
                  98.36% of all statistics are fiction.

Re: sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo?

[Michael Tokarev]

Richard B. Johnson wrote:
[]
> If you ping an IP address on your computer, the traffic will go
> through lo. However, I think that the IP address shown is
> the result of an instrumentation error because it is impossible
> to put, for instance your 192.168.1.1, through a 127.0.0.0 network,
> the ONLY route through lo. This shows that 'local' traffic bypasses
> the lo route filtering altogether. You can verify this by
> deleting the lo route altogether, you can still ping the local
> addresses.

Hmm.  I can't parse this. ;)
Well, I can bring loopback down, but in this case I can't even
ping my local IP addresses anymore, ping reports 'Invalid argument'
error (there's only one route left after deleting lo on my test
machine - 192.168.1.0/24).

> Somebody else mentioned that lo was 'perfectly happy' to
> carry whatever. The fact that something bogus appears on
> lo can be a sign of a misconfiguration error, just as
> the reserved 127.0.0.0 network must never appear on ethernet.

I'm not denying it may be some misconfiguration.  The problem
is that I don't see what/where it is.  All the stuff looks pretty
normal.

> In the case of 0.0.0.0 (a possible broadcast), there is
> no "local" address that could cause a bypass via lo. Instead,
> any such traffic should have been on the ethernet wire. This
> shows the possible configuration error that I mentioned.

Again, I don't understand what do you mean.  The message
in $subj says that something *on this box* sent that bogus
ICMP packet, with source address on this same box.  It may
be a reply to bogus packet sent with src=0.0.0.0 somehow,
or maybe not.  Maybe the host is trying to reply to a packet
sent to one of its local IPs -- eg, imagine I send packet
with src=0.0.0.0 to another host to non-listening port
(rp_filter should be activated in that case I think, but
IF that packet comes from the interface where the default
route goes, rp_filter may not trigger) - and kernel is trying
to send an ICMP reply... back to 0.0.0.0...

Even that does not explain everything still.  My 'default route'
interface - the one which goes to my ISP - I doubt it's possible
from the ISP side to send a packet destined to 192.168.4.2 so
that the packet will come to us - unless their routing is hosed.

The problem is, I can't see what is causing this misconfiguration
or whatever.  I wasn't able to capture such a packet so far either --
it never happened while tcpdump was running.

 Note the local IP address mentioned is different, I've
seen 3 so far, all 3 are local on this box and are on 3
different (ethernet) interfaces (but the ICMP always comes
from lo).

> 
>> 1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
>>    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 
>                     ^^^^^^^^^^^^^^^^
> 
>>    inet 127.0.0.1/8 scope host lo
> 
> 
> This looks as though there is no netmask set. My configuration
> shows:

The netmask is perfect - it's 127.255.255.255.  The thing you
quoted is *ethernet* address, not IP address - and for loopback,
it's ok to have that as all-zeros.

> lo        Link encap:Local Loopback
>           inet addr:127.0.0.1  Mask:255.0.0.0
>           inet6 addr: ::1/128 Scope:Host
>           UP LOOPBACK RUNNING  MTU:16436  Metric:1
> 
> This is a possible configuration error.

Here's how ifconfig shows my interface:

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1

The above output is from `ip addr'.

/mjt

Re: sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo?

[Denis Vlasenko]

On Thursday 07 July 2005 15:34, Richard B. Johnson wrote:
> >> Only the 127.0.0.0 network should be routed through the loop-back
> >> device.
> >
> > Again: All the IP addresses mentioned are local to this box.
> >
> > If you ping an IP address on your eth0, the traffic will "go"
> > over loopback.  You can verify it using tcpdump:
> >
> 
> If you ping an IP address on your computer, the traffic will go
> through lo. However, I think that the IP address shown is
> the result of an instrumentation error because it is impossible
> to put, for instance your 192.168.1.1, through a 127.0.0.0 network,
> the ONLY route through lo. This shows that 'local' traffic bypasses
> the lo route filtering altogether. You can verify this by
> deleting the lo route altogether, you can still ping the local
> addresses.
> 
> Somebody else mentioned that lo was 'perfectly happy' to
> carry whatever. The fact that something bogus appears on
> lo can be a sign of a misconfiguration error, just as
> the reserved 127.0.0.0 network must never appear on ethernet.

Care to tcpdump your own lo?

> In the case of 0.0.0.0 (a possible broadcast), there is
> no "local" address that could cause a bypass via lo. Instead,
> any such traffic should have been on the ethernet wire. This
> shows the possible configuration error that I mentioned.
> 
> 
> > 1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
> >    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>                      ^^^^^^^^^^^^^^^^
> >    inet 127.0.0.1/8 scope host lo
> 
> This looks as though there is no netmask set. My configuration
> shows:

BS. 00:00:00:00:00:00's above aren't netmasks.

> lo        Link encap:Local Loopback
>            inet addr:127.0.0.1  Mask:255.0.0.0
>            inet6 addr: ::1/128 Scope:Host
>            UP LOOPBACK RUNNING  MTU:16436  Metric:1
> 
> This is a possible configuration error.

Yours is ifconfig output, whereas
"link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00"
line above was from ip, not ifconfig.
--
vda

Re: sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo?

[Denis Vlasenko]

> The problem is, I can't see what is causing this misconfiguration
> or whatever.  I wasn't able to capture such a packet so far either --
> it never happened while tcpdump was running.

You may try to add printk("bad boy is: %s\n", current()->comm)),
or dump_stack(), or something like that in icmp path of IP stack.
(I am currently tracking an intermittent network problem
on my home box in similar way).

>  Note the local IP address mentioned is different, I've
> seen 3 so far, all 3 are local on this box and are on 3
> different (ethernet) interfaces (but the ICMP always comes
> from lo).

BTW what tcpdump actually shows?
--
vda

Re: sent an invalid ICMP type 11, code 0 error to a broadcast: 0.0.0.0 on lo?

[Richard B. Johnson]

On Fri, 8 Jul 2005, Denis Vlasenko wrote:

>> The problem is, I can't see what is causing this misconfiguration
>> or whatever.  I wasn't able to capture such a packet so far either --
>> it never happened while tcpdump was running.
>
> You may try to add printk("bad boy is: %s\n", current()->comm)),
> or dump_stack(), or something like that in icmp path of IP stack.
> (I am currently tracking an intermittent network problem
> on my home box in similar way).
>
>>  Note the local IP address mentioned is different, I've
>> seen 3 so far, all 3 are local on this box and are on 3
>> different (ethernet) interfaces (but the ICMP always comes
>> from lo).
>
> BTW what tcpdump actually shows?
> --
> vda
>

Please read line 1148 of /usr/src/linux-`uname -r`/net/ipv4/icmp.c.
The error message is in response to a bogus frame. The NETMASK
must match at both ends (a configuration error). If the mask matches,
then you can turn off the messages with 'sysctl', i.e., ....
`echo "1" >/proc/sys/net/icmp_ignore_bogus_error_responses`.
Otherwise, find the interface netmask that's broken and fix it.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.12 on an i686 machine (5537.79 BogoMips).
  Notice : All mail here is now cached for review by Dictator Bush.
                  98.36% of all statistics are fiction.