Re: [PATCH net-next] IPv6: enable TCP to use an anycast address

[Hannes Frederic Sowa <hannes@stressinduktion.org>]

Hi!

On Thu, Jan 23, 2014 at 01:11:52AM +0400, Alexey Kuznetsov wrote:
> On Wed, Jan 22, 2014 at 6:32 PM, François-Xavier Le Bail
> <fx.lebail@yahoo.com> wrote:
> >> Actually, I was alerted by reset processing in your patch, it cannot be right.
> >
> > Perhaps, I missed something.
> 
> Perhaps, I missed something. According to my old knowledge, tcp cannot
> be used with anycasts
> by many reasons and I have no idea what could change to the moment.
> 
> F.e. what do you do when a segment arrives on listening socket w/o
> connection associated?
> If you sent reset, you could reset someone's valid connection (I mean
> this, saying "it cannot be right").
> If you do not, you violate protocol, those resets are required to
> clean up stale states.

When using anycast addresses with routing protocols, like e.g. anycast
bgp one cannot make the distinction in the kernel at all. Operators
currently depend on the BGP being stable enough so that the anycast
destination towards one server remains stable during the whole connection.

In case a flap happens, of course, packets can end up at a different server
and will get a RST. I guess this currently works well enough for anycast
network operators. Most DNS server use this model currently and DNS depends on
UDP and TCP being reachable for name resolution.

But this change does affect anycast IPv6 only in one subnet. If an anycast
address is allocated in the kernel, neighbour discovery logic tries as
hard to stick to one particular neighbour in the current subnet by always
sending neighbour advertisments with non-override flag. So in case we
need to reprobe the hardware address for the anycast neighbour, we will
get multiple answers and will pick that one which does not override the
current address. In case the anycast server really disappears from the
network, we could end up in a situation where the TCP packet gets send
to the wrong server which could result in a RST. IMHO this is acceptable,
but I may be wrong here. What is your opinion on that?

> >> Do not you think this must not be enabled for common use? At least
> >> some separate sysctl disabled by default.
> >
> > We could indeed use a sysctl, disabled by default.
> >
> > But my goal was to enable anycast address usage transparently, if possible.
> 
> IMHO it is logically impossible. That's why my first question was
> about a document which
> allowed anycasts with TCP. (BTW I found your answer unconvincing, at least.)
> Traditional anycasting used to be stateless and it was impossible to
> use with TCP
> because unsynchronized TCP connections would be unavoidable.
> AFAIK only suggestion from ancient RFC1546 could cure this problem,
> but it is very tricky and implies client knows that destination is anycast.
> 
> Actually, I do not even understand how the idea of use of anycast with TCP
> emerged. The only situation I can imagine: the router is expected to
> do full scale
> connection tracking and to bind connections to anycast destinations to
> correct nodes.
> (And the same moment you understand that the notion of anycast is
> completely lost
> in this picture: the same thing is done by load balancers in IPv4 for
> unicast addresses
> for ages) I still have no idea how it is going to work when both
> client and servers bound
> to anycasts are on the same link.
> 
> So, it looks like you need this sysctl to announce: "someone cares
> about proper routing
> on this network and tcp should work". It looks obvious this mode
> cannot be enabled by default.

IMHO it would be ok if a tcp socket binds to an anycast address, because the
administrator or software can check either for the pre-defined ones (the
subnet router anycast address) or the application allocated an anycast via
setsockopt specifically (the address could also be requested by another
process and the tcp socket could bind to it by accident, that might be a
problem. If we want to protect from this scenario we must add a new sysctl
knob.).

We don't use anycast addresses for normal source address selection, so if the
application does not specifically set the anycast address it will always use a
unicast one.

Greetings,

  Hannes