Re: Trouble getting a stable miredo relay

Re: Trouble getting a stable miredo relay

[Rémi Denis-Courmont]

	Hello,

Le jeudi 12 mars 2009 16:06:13 Wouter de Jong, vous avez écrit :
> I'm trying to get a miredo relay up & running.
> While it should be so easy, I can't get it stable...

> After a very short period (< 5-10 minutes) of announcing 2001::/32
> via BGP to the rest of the world,
> I see a lot of packetloss towards hosts in our network that I reach
> through our relay from teredo clients.
> Sometimes the addresses are not reachable at all (not even the
> relay-address).

This is symptomatic of an overflow in the route/neighbor caches of the kernel. 
Please try to increase the size (a lot) manually via sysctl. I don't know 
anything sane userland/TUNTAP can do about this.

> From native IPv6 hosts from outside -> native IPv6 hosts inside our
> network and vice versa, there is no packetloss at all.

> I tried various hardware (SuperMicro Dual Xeon, Dell 860 Dual-Core
> machine, HP DL360, etc)... all ending up with the same result. Tried
> various distro's (CentOS 5.2, Fedora 10, Ubuntu server 8.10,
> Debian 4.0r7). Distro specific packages, self-compile (against Judy,
> without Judy).... no change.

> I tried separating the IPv4 traffic from the IPv6 traffic (both on
> different nic's), tried tweaking sysctl parameters .... all to no
> avail. 

> At no point the relay took more than 10Mbit/s.

Miredo holds a paradoxical situation whereby it (in userland) can easily 
handle millions of peers, but the kernel starts failing at 1024 of them.

See also 
http://kerneltrap.org/mailarchive/linux-netdev/2008/8/31/3146914/thread and 
http://kerneltrap.org/mailarchive/linux-netdev/2008/8/31/3147634/thread

You don't need much bandwidth to hit 1024 peers. You just to export your route 
to sufficiently many people. I don't know why there is a neighbors cache for 
no-ARP interfaces in the first place. I guess nobody ever bothered to 
conditionnaly disable the code.

> When I restart miredo, it usually works again for a minute (but not
> always).  With tcpdump i do see a lot of teredo traffic when it
> appears to give unreachable messages for hosts.

Best regards,

-- 
Rémi Denis-Courmont
http://www.remlab.net/