IPv6 connect() from site-local to global IPv6 address.

IPv6 connect() from site-local to global IPv6 address.

[David Woodhouse]

I've been using and testing IPv6 on a private network.

Machines have RFC1918 IPv4 addresses, with connectivity by NAT to the
outside world. They also have site-local IPv6 addresses. There is
approximately zero chance of getting corporate approval to have external
IPv6 connectivity.

Since updating the kernel to 2.6.16, I've got problems with external
connectivity to hosts which have both IPv4 and (Global) IPv6 addresses
in DNS. Glibc used to return the IPv4 address in the A record first, and
all was well. But now it returns the IPv6 address in the AAAA record
first, and I can't communicate with that. So I get a three-minute
timeout whenever I try to connect to anything in the outside which has
both A and AAAA records.

One of the things which glibc's implementation of RFC3484 address
selection (http://people.redhat.com/drepper/linux-rfc3484.html) does is
to perform a dummy connect() of a SOCK_DGRAM socket to each of the
potential addresses. On older kernels this used to fail when we
attempted to connect to a global IPv6 address and we didn't have a
global IPv6 address of our own...

socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "2001:8b0:10b:1:20d:93ff:fe7a:3f2c", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EADDRNOTAVAIL (Cannot assign requested address)
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("81.187.2.168")}, 16) = 0
getsockname(3, {sa_family=AF_INET, sin_port=htons(33450), sin_addr=inet_addr("172.16.18.126")}, [16]) = 0
Trying 81.187.2.168...

On the newer kernel, the connect() succeeds:

socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "2001:8b0:10b:1:20d:93ff:fe7a:3f2c", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = 0
getsockname(3, {sa_family=AF_INET6, sin6_port=htons(32772), inet_pton(AF_INET6, "::172.16.18.67", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("81.187.2.168")}, 16) = 0
getsockname(3, {sa_family=AF_INET, sin_port=htons(32772), sin_addr=inet_addr("172.16.18.67")}, [16]) = 0
Trying 2001:8b0:10b:1:20d:93ff:fe7a:3f2c...

Is this change in behaviour intentional? Is it useful?

How can we get sane behaviour from glibc again? What we had before was
ideal -- if we have an IPv6 default route _and_ we have a Global IPv6
address of our own, then return the Global IPv6 address in the AAAA
record first. Else return the IPv4 address in the A record.

-- 
dwmw2

Re: IPv6 connect() from site-local to global IPv6 address.

[YOSHIFUJI Hideaki / 吉藤英明]

In article <1146862832.2766.54.camel@pmac.infradead.org> (at Fri, 05 May 2006 22:00:32 +0100), David Woodhouse <dwmw2@infradead.org> says:

> Since updating the kernel to 2.6.16, I've got problems with external
> connectivity to hosts which have both IPv4 and (Global) IPv6 addresses
> in DNS. Glibc used to return the IPv4 address in the A record first, and
> all was well. But now it returns the IPv6 address in the AAAA record
> first, and I can't communicate with that. So I get a three-minute
> timeout whenever I try to connect to anything in the outside which has
> both A and AAAA records.
> 
> One of the things which glibc's implementation of RFC3484 address
> selection (http://people.redhat.com/drepper/linux-rfc3484.html) does is
> to perform a dummy connect() of a SOCK_DGRAM socket to each of the
> potential addresses. On older kernels this used to fail when we
> attempted to connect to a global IPv6 address and we didn't have a
> global IPv6 address of our own...
:
> socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
> connect(3, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "2001:8b0:10b:1:20d:93ff:fe7a:3f2c", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = 0
> getsockname(3, {sa_family=AF_INET6, sin6_port=htons(32772), inet_pton(AF_INET6, "::172.16.18.67", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
> socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
> connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("81.187.2.168")}, 16) = 0
> getsockname(3, {sa_family=AF_INET, sin_port=htons(32772), sin_addr=inet_addr("172.16.18.67")}, [16]) = 0
> Trying 2001:8b0:10b:1:20d:93ff:fe7a:3f2c...
> 
> Is this change in behaviour intentional? Is it useful?
> 
> How can we get sane behaviour from glibc again? What we had before was
> ideal -- if we have an IPv6 default route _and_ we have a Global IPv6
> address of our own, then return the Global IPv6 address in the AAAA
> record first. Else return the IPv4 address in the A record.

You have compatible address.
Do you really use the tunnel? How did you configure it?

--yoshfuji

Re: IPv6 connect() from site-local to global IPv6 address.

[David Woodhouse]

On Sat, 2006-05-06 at 09:19 +0900, YOSHIFUJI Hideaki wrote:
> You have compatible address.
> Do you really use the tunnel? How did you configure it? 

Sorry, I should have shown a strace from a different machine. Try this
one from an autoconfigured machine...

socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "2001:8b0:10b:1:20d:93ff:fe7a:3f2c", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = 0
getsockname(3, {sa_family=AF_INET6, sin6_port=htons(32837), inet_pton(AF_INET6, "fec0::1:202:b3ff:fe03:45c1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("81.187.2.168")}, 16) = 0
getsockname(3, {sa_family=AF_INET, sin_port=htons(32837), sin_addr=inet_addr("172.16.18.64")}, [16]) = 0
Trying 2001:8b0:10b:1:20d:93ff:fe7a:3f2c...

There is a default route, because I believe that's the only thing that
radvd can do. I cannot advertise a route to _only_ fec0::/16, can I?


(The machine I showed you before was a router between networks --
the ::172.16.18.67 address was on its tunnel interface, and was added
automatically when I created the tunnel and brought it up.) 

-- 
dwmw2

Re: IPv6 connect() from site-local to global IPv6 address.

[YOSHIFUJI Hideaki / 吉藤英明]

In article <1146876802.2503.32.camel@shinybook.infradead.org> (at Sat, 06 May 2006 01:53:21 +0100), David Woodhouse <dwmw2@infradead.org> says:

> There is a default route, because I believe that's the only thing that
> radvd can do. I cannot advertise a route to _only_ fec0::/16, can I?

Yes, you can, via Route Information option.
(>= 2.6.17-rc1 support this.)

Anyway, it is valid to use (obsolete) site-local source address
for global destination address.
The problem seems that router does NOT send ICMPv6 destination
unreachable to the sender. I don't know why, but it SHOULD.

--yoshfuji

Re: IPv6 connect() from site-local to global IPv6 address.

[David Woodhouse]

On Sat, 2006-05-06 at 11:39 +0900, YOSHIFUJI Hideaki wrote:
> In article <1146876802.2503.32.camel@shinybook.infradead.org> (at Sat, 06 May 2006 01:53:21 +0100), David Woodhouse <dwmw2@infradead.org> says:
> 
> > There is a default route, because I believe that's the only thing that
> > radvd can do. I cannot advertise a route to _only_ fec0::/16, can I?
> 
> Yes, you can, via Route Information option.
> (>= 2.6.17-rc1 support this.)

Hm. Has radvd also been updated to send it, or did I miss it when I
looked for such an option?

Either way, this isn't particularly useful to me yet, since no deployed
systems support it.

> Anyway, it is valid to use (obsolete) site-local source address
> for global destination address.
> The problem seems that router does NOT send ICMPv6 destination
> unreachable to the sender. I don't know why, but it SHOULD. 

I'll pursue that question later. It wouldn't be _sufficient_ since there
are (buggy) programs, including Evolution, which will not fall back to
the second and subsequent addresses from getaddrinfo() -- they'll just
give up when the first attempt to connect fails. So we really do need
the IPv4 address to be listed _first_ in the results, as it used to be.

Glibc _used_ to do what we want -- I always attributed it to Rule 2 of
the destination address selection, without looking hard at the
implementation details.

Let's forget about any details of my current implementation and I'll ask
a _simple_ question...

I have machines on an internal company network, which is all RFC1918
IPv4 addresses and has connectivity by NAT to the outside world. These
machines are mostly Linux, of various versions. 

I wish to deploy IPv6 internally so that we can develop and test IPv6
support. There is _no_ chance of getting proper IPv6 connectivity to the
outside world through the corporate firewall. I'd like IPv6 to be usable
_internally_ though, without breaking connectivity to the outside world
over IPv4.

How should I do this?


In the past, I've done it with site-local addresses. A machine on each
Ethernet subnet runs radvd, and machines pick up a site-local address
(and default route) automatically. The machines running radvd also have
IPv6-over-IPv4 tunnels for routing between subnets. 

Glibc's getaddrinfo() has in the past given me optimal behaviour. If
connecting to a remote machine which has a site-local IPv6 address, it's
favoured that site-local address. If connecting to a remote machine
which has a _Global_ IPv6 address, it's given the IPv4 address first
instead.

On new kernels, however, glibc has started to return IPv6 addresses even
for external machines which can't be reached. Hence my mail about
$SUBJECT. If I'm doing this the wrong way, what _should_ I be doing?

Uli's response has been to switch glibc so that it _always_ favours IPv4
over IPv6, which AIUI basically means that IPv6 will never get _any_
usage on Linux machines unless in an IPv6-only environment rather than
dual-stack. I.e. Unless I publish _only_ an AAAA record for my server
instead of both A and AAAA records, I won't get any IPv6 traffic. I
think it would be best to avoid that situation.

-- 
dwmw2

Re: IPv6 connect() from site-local to global IPv6 address.

[Rick Jones]

>>Anyway, it is valid to use (obsolete) site-local source address
>>for global destination address.
>>The problem seems that router does NOT send ICMPv6 destination
>>unreachable to the sender. I don't know why, but it SHOULD. 
> 
> 
> I'll pursue that question later. It wouldn't be _sufficient_ since there
> are (buggy) programs, including Evolution, which will not fall back to
> the second and subsequent addresses from getaddrinfo() -- they'll just
> give up when the first attempt to connect fails. So we really do need
> the IPv4 address to be listed _first_ in the results, as it used to be.

Or get the applications fixed no?  Kludging around application bugs 
sounds a bit like the "Fram Oil Filter" commercial where the mechanic is 
grinning while he says "You can pay me now, or you can pay be later." As 
in pay for the slightly more expensive oil filter now, or engine repair 
later.

> I wish to deploy IPv6 internally so that we can develop and test IPv6
> support. There is _no_ chance of getting proper IPv6 connectivity to the
> outside world through the corporate firewall. I'd like IPv6 to be usable
> _internally_ though, without breaking connectivity to the outside world
> over IPv4.
> 
> How should I do this?

Other than fixing the applications that only take the first response 
(isn't that a generic application bug going back nearly decades now? 
amazing how things stay the same isn't it) Can you run a caching-only 
name server at the edge that filters-out the IPv6 responses so your 
systems never see Global IPV6 responses?

rick jones

Re: IPv6 connect() from site-local to global IPv6 address.

[David Woodhouse]

On Mon, 2006-05-08 at 09:44 -0700, Rick Jones wrote:
> Or get the applications fixed no?  Kludging around application bugs 
> sounds a bit like the "Fram Oil Filter" commercial where the mechanic is 
> grinning while he says "You can pay me now, or you can pay be later." As 
> in pay for the slightly more expensive oil filter now, or engine repair 
> later.

Well, obviously. That's _why_ I want to deploy IPv6 and get it tested.
But I used to be able to do this without actually breaking the network,
and without being told to _stop_ running radvd because it breaks things.

> Other than fixing the applications that only take the first response 
> (isn't that a generic application bug going back nearly decades now? 
> amazing how things stay the same isn't it) Can you run a caching-only 
> name server at the edge that filters-out the IPv6 responses so your 
> systems never see Global IPV6 responses?

I don't think that kind of answer is going to be sufficient to persuade
Uli to switch back from favouring IPv4 over IPv6. That's done the trick,
admittedly -- by ensuring that we get _no_ testing of IPv6 unless we run
with IPv6-only networking :)

-- 
dwmw2

Re: IPv6 connect() from site-local to global IPv6 address.

[Kazunori Miyazawa]

Hi,

I lost some mails on the list because of my network trouble.
This might not be a correct thread to reply. Sorry.

Anyway, I traced the probelem.

My test environment likes:

The host in my network could reach to the global network via NAT-T on IPv4.
But it could not reach to the global network on IPv6 because router did not have
the default route.
The radvd on the router advertised a global scope prefix for the hosts.
The hosts accordingly have a global scoped address.
DNS server returned both AAAA and A of the target host(server).

The router and the host is Ubuntu 5.10 with the kernel 2.6.16.9.

I tested with these steps:

1. I did "dig" to check that the host could got AAAA address.
2. I did ping6 to check that the host received dest unreach
    with no route from the router.
3. I run "evalution" and connected to my server which have AAAA and A.

It falled back to IPv4 and it could connect the server via IPv4 network.
It also falled back by "Time exceeded". My evolution version is 2.4.1.
Of course, when I added IPv6 default route, it connected via IPv6 network.

I think they works well as far as ICMPv6 error messages can be received.
I did not test in the environment in which ICMPv6 error messages are filtered.

Best regards,

David Woodhouse wrote:
> On Mon, 2006-05-08 at 09:44 -0700, Rick Jones wrote:
> 
>>Or get the applications fixed no?  Kludging around application bugs 
>>sounds a bit like the "Fram Oil Filter" commercial where the mechanic is 
>>grinning while he says "You can pay me now, or you can pay be later." As 
>>in pay for the slightly more expensive oil filter now, or engine repair 
>>later.
> 
> 
> Well, obviously. That's _why_ I want to deploy IPv6 and get it tested.
> But I used to be able to do this without actually breaking the network,
> and without being told to _stop_ running radvd because it breaks things.
> 
> 
>>Other than fixing the applications that only take the first response 
>>(isn't that a generic application bug going back nearly decades now? 
>>amazing how things stay the same isn't it) Can you run a caching-only 
>>name server at the edge that filters-out the IPv6 responses so your 
>>systems never see Global IPV6 responses?
> 
> 
> I don't think that kind of answer is going to be sufficient to persuade
> Uli to switch back from favouring IPv4 over IPv6. That's done the trick,
> admittedly -- by ensuring that we get _no_ testing of IPv6 unless we run
> with IPv6-only networking :)
> 

--
Kazunori Miyazawa