specifying scopid's for link-local IPv6 addrs

specifying scopid's for link-local IPv6 addrs

[Rick Jones]

Folks -

People running netperf have reported that they have trouble with IPv6 under 
Linux.  Specifically, wereas the use of link-local IPv6 addresses "just works" 
in netperf under a number of "other OSes" they do not under Linux.  I'm 
ass-u-me-ing 2.6 here, but not sure exactly which ones - I've seen it on a 
2.6.18-based RHEL5.

Some poking about and conversation has suggested that one has to set a 
sin6_scope_id in the sockaddr_in6.  This needs to be an index of one of the 
interfaces in the system, which I presume means walking some additional structures.

Is this a requirement which might be expected to remain in the future, or is it 
something which might just go away?  That will have an effect on netperf future 
development.

thanks,

rick jones

Re: specifying scopid's for link-local IPv6 addrs

[Bill Fink]

On Mon, 23 Jul 2007, Rick Jones wrote:

> Folks -
> 
> People running netperf have reported that they have trouble with IPv6 under 
> Linux.  Specifically, wereas the use of link-local IPv6 addresses "just works" 
> in netperf under a number of "other OSes" they do not under Linux.  I'm 
> ass-u-me-ing 2.6 here, but not sure exactly which ones - I've seen it on a 
> 2.6.18-based RHEL5.
> 
> Some poking about and conversation has suggested that one has to set a 
> sin6_scope_id in the sockaddr_in6.  This needs to be an index of one of the 
> interfaces in the system, which I presume means walking some additional structures.
> 
> Is this a requirement which might be expected to remain in the future, or is it 
> something which might just go away?  That will have an effect on netperf future 
> development.
> 
> thanks,
> 
> rick jones

Rick,

I don't see any way around this.  For example, on one of my test
systems, I have the following link local routes:

chance% netstat -A inet6 -rn | grep fe80::/64
fe80::/64                                   ::                                      U     256    0        0 eth0
fe80::/64                                   ::                                      U     256    0        0 eth2
fe80::/64                                   ::                                      U     256    0        0 eth3
fe80::/64                                   ::                                      U     256    0        0 eth4
fe80::/64                                   ::                                      U     256    0        0 eth5
fe80::/64                                   ::                                      U     256    0        0 eth6

So if I want to run a link local test to fe80::202:b3ff:fed4:cd1,
the system has no way to choose which is the correct interface to
use for the test, and will give an error if the interface isn't
specified.  Here's an example of this with nuttcp:

chance% nuttcp -P5100 fe80::202:b3ff:fed4:cd1
nuttcp-t: Info: attempting to switch to deprecated "classic" mode
nuttcp-t: Info: will use less reliable transmitter side statistics
nuttcp-t: v5.5.5: Error: connect: Invalid argument
errno=22

You must explicitly specify the desired interface.  For example,
on my test system, the correct interface is eth6 which is interface 8
(lo eth0 eth1 eth2 ... eth5 eth6).  Here is an example nuttcp test
specifying interface 8:

chance% nuttcp -P5100 fe80::202:b3ff:fed4:cd1%8
 1178.5809 MB /  10.02 sec =  986.2728 Mbps 12 %TX 15 %RX

nuttcp uses getaddrinfo() which parses the "%<ifindex>" field,
and then copies the sin6_scope_id from the res structure to the
server's sockaddr_in6 structure before initiating the connect().

						-Bill

Re: specifying scopid's for link-local IPv6 addrs

[Rick Jones]

> Rick,
> 
> I don't see any way around this.  For example, on one of my test
> systems, I have the following link local routes:
> 
> chance% netstat -A inet6 -rn | grep fe80::/64
> fe80::/64                                   ::                                      U     256    0        0 eth0
> fe80::/64                                   ::                                      U     256    0        0 eth2
> fe80::/64                                   ::                                      U     256    0        0 eth3
> fe80::/64                                   ::                                      U     256    0        0 eth4
> fe80::/64                                   ::                                      U     256    0        0 eth5
> fe80::/64                                   ::                                      U     256    0        0 eth6
> 
> So if I want to run a link local test to fe80::202:b3ff:fed4:cd1,
> the system has no way to choose which is the correct interface to
> use for the test, and will give an error if the interface isn't
> specified. 

Yeah, I was wondering about that.  I'm not sure if the attempts on "those other 
OSes" happened to involve multiple interfaces or not.  Even so, it "feels" 
unpleasant for an application to deal with and I wonder if there is a way for a 
stack to deal with it on the application's behalf.  I guess that might involve 
some sort of layer violation between neightbor discovery and routing (typing 
while I think about things I know little about...)

Is there RFC chapter and verse I might read about routing with multiple 
link-local's on a system?

> You must explicitly specify the desired interface.  For example,
> on my test system, the correct interface is eth6 which is interface 8
> (lo eth0 eth1 eth2 ... eth5 eth6).  Here is an example nuttcp test
> specifying interface 8:
> 
> chance% nuttcp -P5100 fe80::202:b3ff:fed4:cd1%8
>  1178.5809 MB /  10.02 sec =  986.2728 Mbps 12 %TX 15 %RX
> 
> nuttcp uses getaddrinfo() which parses the "%<ifindex>" field,
> and then copies the sin6_scope_id from the res structure to the
> server's sockaddr_in6 structure before initiating the connect().

OK, I'll give that a quick try with netperf:

[root@hpcpc106 ~]# netperf -H 192.168.2.107 -c -C -i 30,3 -- -s 1M -S 1M -m 64K 
-H fe80::207:43ff:fe05:9d%2
TCP STREAM TEST from ::0 (::) port 0 AF_INET6 to fe80::207:43ff:fe05:9d%2 
(fe80::207:43ff:fe05:9d) port 0 AF_INET6 : +/-2.5% @ 99% conf.

Cool - it establishes the data connection just fine.


To further demonstrate my ignorance :)  is that %n suffix something one might 
expect in most/all getaddrinfo()'s or is that unique to the one in Linux?

rick jones

Re: specifying scopid's for link-local IPv6 addrs

[Rick Jones]

>> You must explicitly specify the desired interface.  For example,
>> on my test system, the correct interface is eth6 which is interface 8
>> (lo eth0 eth1 eth2 ... eth5 eth6).  Here is an example nuttcp test
>> specifying interface 8:
>>
>> chance% nuttcp -P5100 fe80::202:b3ff:fed4:cd1%8
>>  1178.5809 MB /  10.02 sec =  986.2728 Mbps 12 %TX 15 %RX
>>
>> nuttcp uses getaddrinfo() which parses the "%<ifindex>" field,
>> and then copies the sin6_scope_id from the res structure to the
>> server's sockaddr_in6 structure before initiating the connect().
> 
> 
> OK, I'll give that a quick try with netperf:
> 
> [root@hpcpc106 ~]# netperf -H 192.168.2.107 -c -C -i 30,3 -- -s 1M -S 1M 
> -m 64K -H fe80::207:43ff:fe05:9d%2
> TCP STREAM TEST from ::0 (::) port 0 AF_INET6 to 
> fe80::207:43ff:fe05:9d%2 (fe80::207:43ff:fe05:9d) port 0 AF_INET6 : 
> +/-2.5% @ 99% conf.
> 
> Cool - it establishes the data connection just fine.

Well, I spoke too soon - while it got me past my EINVAL, the connection 
establishement timed-out.  Either I picked the wrong value for n, or I may yet 
need to make some tweaks to netperf.

rick jones

Re: specifying scopid's for link-local IPv6 addrs

[Sridhar Samudrala]

On Tue, 2007-07-24 at 10:13 -0700, Rick Jones wrote:
> > Rick,
> > 
> > I don't see any way around this.  For example, on one of my test
> > systems, I have the following link local routes:
> > 
> > chance% netstat -A inet6 -rn | grep fe80::/64
> > fe80::/64                                   ::                                      U     256    0        0 eth0
> > fe80::/64                                   ::                                      U     256    0        0 eth2
> > fe80::/64                                   ::                                      U     256    0        0 eth3
> > fe80::/64                                   ::                                      U     256    0        0 eth4
> > fe80::/64                                   ::                                      U     256    0        0 eth5
> > fe80::/64                                   ::                                      U     256    0        0 eth6
> > 
> > So if I want to run a link local test to fe80::202:b3ff:fed4:cd1,
> > the system has no way to choose which is the correct interface to
> > use for the test, and will give an error if the interface isn't
> > specified. 
> 
> Yeah, I was wondering about that.  I'm not sure if the attempts on "those other 
> OSes" happened to involve multiple interfaces or not.  Even so, it "feels" 
> unpleasant for an application to deal with and I wonder if there is a way for a 
> stack to deal with it on the application's behalf.  I guess that might involve 
> some sort of layer violation between neightbor discovery and routing (typing 
> while I think about things I know little about...)
> 
> Is there RFC chapter and verse I might read about routing with multiple 
> link-local's on a system?
> 
> > You must explicitly specify the desired interface.  For example,
> > on my test system, the correct interface is eth6 which is interface 8
> > (lo eth0 eth1 eth2 ... eth5 eth6).  Here is an example nuttcp test
> > specifying interface 8:
> > 
> > chance% nuttcp -P5100 fe80::202:b3ff:fed4:cd1%8
> >  1178.5809 MB /  10.02 sec =  986.2728 Mbps 12 %TX 15 %RX
> > 
> > nuttcp uses getaddrinfo() which parses the "%<ifindex>" field,
> > and then copies the sin6_scope_id from the res structure to the
> > server's sockaddr_in6 structure before initiating the connect().
> 
> OK, I'll give that a quick try with netperf:
> 
> [root@hpcpc106 ~]# netperf -H 192.168.2.107 -c -C -i 30,3 -- -s 1M -S 1M -m 64K 
> -H fe80::207:43ff:fe05:9d%2

We can even specify the interface name instead of the interface index
   <link-local>%ethX

getaddrinfo() uses if_nametoindex() internally to get the index.

Thanks
Sridhar
> TCP STREAM TEST from ::0 (::) port 0 AF_INET6 to fe80::207:43ff:fe05:9d%2 
> (fe80::207:43ff:fe05:9d) port 0 AF_INET6 : +/-2.5% @ 99% conf.
> 
> Cool - it establishes the data connection just fine.
> 
> 
> To further demonstrate my ignorance :)  is that %n suffix something one might 
> expect in most/all getaddrinfo()'s or is that unique to the one in Linux?
> 
> rick jones
> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: specifying scopid's for link-local IPv6 addrs

[Bill Fink]

On Tue, 24 Jul 2007, Sridhar Samudrala wrote:

> On Tue, 2007-07-24 at 10:13 -0700, Rick Jones wrote:
> > > Rick,
> > > 
> > > I don't see any way around this.  For example, on one of my test
> > > systems, I have the following link local routes:
> > > 
> > > chance% netstat -A inet6 -rn | grep fe80::/64
> > > fe80::/64                                   ::                                      U     256    0        0 eth0
> > > fe80::/64                                   ::                                      U     256    0        0 eth2
> > > fe80::/64                                   ::                                      U     256    0        0 eth3
> > > fe80::/64                                   ::                                      U     256    0        0 eth4
> > > fe80::/64                                   ::                                      U     256    0        0 eth5
> > > fe80::/64                                   ::                                      U     256    0        0 eth6
> > > 
> > > So if I want to run a link local test to fe80::202:b3ff:fed4:cd1,
> > > the system has no way to choose which is the correct interface to
> > > use for the test, and will give an error if the interface isn't
> > > specified. 
> > 
> > Yeah, I was wondering about that.  I'm not sure if the attempts on "those other 
> > OSes" happened to involve multiple interfaces or not.  Even so, it "feels" 
> > unpleasant for an application to deal with and I wonder if there is a way for a 
> > stack to deal with it on the application's behalf.  I guess that might involve 
> > some sort of layer violation between neightbor discovery and routing (typing 
> > while I think about things I know little about...)
> > 
> > Is there RFC chapter and verse I might read about routing with multiple 
> > link-local's on a system?
> > 
> > > You must explicitly specify the desired interface.  For example,
> > > on my test system, the correct interface is eth6 which is interface 8
> > > (lo eth0 eth1 eth2 ... eth5 eth6).  Here is an example nuttcp test
> > > specifying interface 8:
> > > 
> > > chance% nuttcp -P5100 fe80::202:b3ff:fed4:cd1%8
> > >  1178.5809 MB /  10.02 sec =  986.2728 Mbps 12 %TX 15 %RX
> > > 
> > > nuttcp uses getaddrinfo() which parses the "%<ifindex>" field,
> > > and then copies the sin6_scope_id from the res structure to the
> > > server's sockaddr_in6 structure before initiating the connect().
> > 
> > OK, I'll give that a quick try with netperf:
> > 
> > [root@hpcpc106 ~]# netperf -H 192.168.2.107 -c -C -i 30,3 -- -s 1M -S 1M -m 64K 
> > -H fe80::207:43ff:fe05:9d%2
> 
> We can even specify the interface name instead of the interface index
>    <link-local>%ethX
> 
> getaddrinfo() uses if_nametoindex() internally to get the index.
> 
> Thanks
> Sridhar

Cool!  That's much easier and works great.  :-)

chance% nuttcp -P5100 fe80::202:b3ff:fed4:cd1%eth6
 1178.5468 MB /  10.02 sec =  986.3239 Mbps 13 %TX 15 %RX

Still learn something new every day.  Now if I just could remember
it all when I needed it later.  :-)

						-Thanks

						-Bill

Re: specifying scopid's for link-local IPv6 addrs

[Vlad Yasevich]

Rick Jones wrote:
>> Rick,
>>
>> I don't see any way around this.  For example, on one of my test
>> systems, I have the following link local routes:
>>
>> chance% netstat -A inet6 -rn | grep fe80::/64
>> fe80::/64                                  
>> ::                                      U     256    0        0 eth0
>> fe80::/64                                  
>> ::                                      U     256    0        0 eth2
>> fe80::/64                                  
>> ::                                      U     256    0        0 eth3
>> fe80::/64                                  
>> ::                                      U     256    0        0 eth4
>> fe80::/64                                  
>> ::                                      U     256    0        0 eth5
>> fe80::/64                                  
>> ::                                      U     256    0        0 eth6
>>
>> So if I want to run a link local test to fe80::202:b3ff:fed4:cd1,
>> the system has no way to choose which is the correct interface to
>> use for the test, and will give an error if the interface isn't
>> specified. 
> 
> Yeah, I was wondering about that.  I'm not sure if the attempts on
> "those other OSes" happened to involve multiple interfaces or not.

Yes, there have been attempts.   BSD has a concept of default interface.
The default interface is used when the user did not specify the interface/
scope_id.

Other OSs do different things.  For example, Tru64 (to pick on a dead system)
would try to find the right interface base on the preferences you could
set up.

But, in the end the whole thing is really utterly broken since no-one has
truly implemented scoped address architecture for link-local addresses.
The concept of the link is so closely tied to the 'interface' that I don't
know anyone who has separated the two.


>  Even
> so, it "feels" unpleasant for an application to deal with and I wonder
> if there is a way for a stack to deal with it on the application's
> behalf.  I guess that might involve some sort of layer violation between
> neightbor discovery and routing (typing while I think about things I
> know little about...)
> 
> Is there RFC chapter and verse I might read about routing with multiple
> link-local's on a system?

See RFC 4007 for the concepts.  

> 
>> You must explicitly specify the desired interface.  For example,
>> on my test system, the correct interface is eth6 which is interface 8
>> (lo eth0 eth1 eth2 ... eth5 eth6).  Here is an example nuttcp test
>> specifying interface 8:
>>
>> chance% nuttcp -P5100 fe80::202:b3ff:fed4:cd1%8
>>  1178.5809 MB /  10.02 sec =  986.2728 Mbps 12 %TX 15 %RX
>>
>> nuttcp uses getaddrinfo() which parses the "%<ifindex>" field,
>> and then copies the sin6_scope_id from the res structure to the
>> server's sockaddr_in6 structure before initiating the connect().
> 
> OK, I'll give that a quick try with netperf:
> 
> [root@hpcpc106 ~]# netperf -H 192.168.2.107 -c -C -i 30,3 -- -s 1M -S 1M
> -m 64K -H fe80::207:43ff:fe05:9d%2
> TCP STREAM TEST from ::0 (::) port 0 AF_INET6 to
> fe80::207:43ff:fe05:9d%2 (fe80::207:43ff:fe05:9d) port 0 AF_INET6 :
> +/-2.5% @ 99% conf.
> 
> Cool - it establishes the data connection just fine.
> 
> 
> To further demonstrate my ignorance :)  is that %n suffix something one
> might expect in most/all getaddrinfo()'s or is that unique to the one in
> Linux?

This is becoming more generic.  I believe HP-UX supports it (if the don't
you should kick them :).

-vlad