Re: [ewg] rping is not resolving ipv6 addresses

Re: [ewg] rping is not resolving ipv6 addresses

[Or Gerlitz]

David J. Wilder <dwilder-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org> wrote:
> I added an option to rping to specify a source address and supply it to

patch?

> rdma_resolve_addr(), but now it is failing rdma_resolve_route().
> $ ./rping -d  -c -v -a fe80::202:c903:1:1925 -i fe80::202:c903:1:28ed
> cma_event type RDMA_CM_EVENT_ADDR_RESOLVED cma_id 0x100213d0 (parent)
> cma_event type RDMA_CM_EVENT_ROUTE_ERROR cma_id 0x100213d0 (parent)
> cma event RDMA_CM_EVENT_ROUTE_ERROR, error -22

what does the neighbour info (ip neigh show | grep 1925)  shows after
running rping?
can you do ipoib ping and ping6 to the fe80::202:c903:1:1925 host?

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ewg] rping is not resolving ipv6 addresses

[David J. Wilder]

On Fri, 2009-10-09 at 22:53 +0200, Or Gerlitz wrote:
> David J. Wilder <dwilder-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org> wrote:
> > I added an option to rping to specify a source address and supply it to
> 
> patch?
> 
> > rdma_resolve_addr(), but now it is failing rdma_resolve_route().
> > $ ./rping -d  -c -v -a fe80::202:c903:1:1925 -i fe80::202:c903:1:28ed
> > cma_event type RDMA_CM_EVENT_ADDR_RESOLVED cma_id 0x100213d0 (parent)
> > cma_event type RDMA_CM_EVENT_ROUTE_ERROR cma_id 0x100213d0 (parent)
> > cma event RDMA_CM_EVENT_ROUTE_ERROR, error -22
> 
> what does the neighbour info (ip neigh show | grep 1925)  shows after
> running rping?

If I run rping without my rping change to add the source address to
rdma_resolve_address(),  ip neigh show gives:
 fe80::202:c903:1:1925 dev eth1  FAILED

Notice that interface is incorrect, it should be ib0. tcpdump showed the
neighbor-discovery sent out the eth0 interface.

Running with my rping change to specify the local-link address of my ib0
interface "ip neigh show" never shows any entry for
fe80::202:c903:1:1925


> can you do ipoib ping and ping6 to the fe80::202:c903:1:1925 host?
> 
> Or.

Yes ping6 will work but I must specify the interface to use:
ping6 fe80::202:c903:1:1925%ib0

Here is the patch to rping to allow the source address to be specified.
I am not sure this is the correct approach to fix this.

--- rping.c.orig	2009-10-08 15:22:29.000000000 -0500
+++ rping.c	2009-10-08 15:29:59.000000000 -0500
@@ -144,6 +144,7 @@ struct rping_cb {
 	sem_t sem;
 
 	struct sockaddr_storage sin;
+	struct sockaddr_storage src;
 	uint16_t port;			/* dst port in NBO */
 	int verbose;			/* verbose logging */
 	int count;			/* ping count */
@@ -1003,7 +1004,7 @@ static int rping_bind_client(struct rpin
 
 	DEBUG_LOG("rping_bind_client: about to call rdma_resolve_addr\n");
 
-	ret = rdma_resolve_addr(cb->cm_id, NULL, (struct sockaddr *) &cb->sin, 2000);
+	ret = rdma_resolve_addr(cb->cm_id, &cb->src, (struct sockaddr *) &cb->sin, 2000);
 	if (ret) {
 		fprintf(stderr, "rdma_resolve_addr error %d\n", ret);
 		return ret;
@@ -1131,11 +1132,14 @@ int main(int argc, char *argv[])
 	sem_init(&cb->sem, 0, 0);
 
 	opterr = 0;
-	while ((op=getopt(argc, argv, "a:Pp:C:S:t:scvVd")) != -1) {
+	while ((op=getopt(argc, argv, "a:i:Pp:C:S:t:scvVd")) != -1) {
 		switch (op) {
 		case 'a':
 			ret = get_addr(optarg, (struct sockaddr *) &cb->sin);
 			break;
+		case 'i':
+			ret = get_addr(optarg, (struct sockaddr *) &cb->src);
+			break;
 		case 'P':
 			persistent_server = 1;
 			break;


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ewg] rping is not resolving ipv6 addresses

[Or Gerlitz]

David J. Wilder <dwilder-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org> wrote:

> If I run rping without my rping change to add the source address to
> rdma_resolve_address(),  ip neigh show gives:  fe80::202:c903:1:1925 dev eth1  FAILED
> Notice that interface is incorrect, it should be ib0. tcpdump showed the
> neighbor-discovery sent out the eth0 interface.

yes, this is as of what Roland explained.


> Running with my rping change to specify the local-link address of my ib0
> interface "ip neigh show" never shows any entry for  fe80::202:c903:1:1925

mmm... weird, run your rping with tcp dump in another screen and see
if ND takes place

> ping6 will work but I must specify the interface to use: ping6 fe80::202:c903:1:1925%ib0

after the tcpdump experiment, run ping6 and immediatly following that
or in parallel on another window run rping
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ewg] rping is not resolving ipv6 addresses

[Jason Gunthorpe]

On Fri, Oct 09, 2009 at 02:15:01PM -0700, David J. Wilder wrote:

> Yes ping6 will work but I must specify the interface to use:
> ping6 fe80::202:c903:1:1925%ib0
> 
> Here is the patch to rping to allow the source address to be specified.
> I am not sure this is the correct approach to fix this.

It is not, IPv6 link local addresses must be scoped. rping is
parsing the address with getaddrinfo, that does correctly set the
sin6_scope_id value in the sockaddr.

rdma_resolve_addr (and likely the underlying kernel component) must
respect this when dealing with link local scope addresses.

rmda_resolve_addr should fail if a link local IPv6 address is provided
without scope, like the normal IP stack:

connect(3, {sa_family=AF_INET6, sin6_port=htons(1025), inet_pton(AF_INET6, "fe80::202:c903:1:1925", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EINVAL (Invalid argument)

So this is two bugs in the rdma cm, rping is correct as is..

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [ewg] rping is not resolving ipv6 addresses

[Sean Hefty]

>rdma_resolve_addr (and likely the underlying kernel component) must
>respect this when dealing with link local scope addresses.
>
>rmda_resolve_addr should fail if a link local IPv6 address is provided
>without scope, like the normal IP stack:

The rdma cm was never fully coded or tested for ipv6 support.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: rping is not resolving ipv6 addresses

[Or Gerlitz]

Sean Hefty wrote:
> The rdma cm was never fully coded or tested for ipv6 support.

Sean, even if not fully coded/tested, some work has been done, e.g commits 38617c64 "RDMA/addr: Add support for translating IPv6 addresses" and 1f5175ad "RDMA/cma: Add IPv6 support". I suggest we'll try to see what does it take to make this better or even fully support ipv6.

Jason, can you restate what are the two problems you saw from David's reports? the 1st was related to scope in link-local addresses, and what's the 2nd?

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: rping is not resolving ipv6 addresses

[Jason Gunthorpe]

On Sun, Oct 11, 2009 at 09:29:08AM +0200, Or Gerlitz wrote:
> Sean Hefty wrote:
> > The rdma cm was never fully coded or tested for ipv6 support.
 
> Sean, even if not fully coded/tested, some work has been done, e.g
> commits 38617c64 "RDMA/addr: Add support for translating IPv6
> addresses" and 1f5175ad "RDMA/cma: Add IPv6 support". I suggest
> we'll try to see what does it take to make this better or even fully
> support ipv6.

Or is right, these are kernel bugs, if nobody is able to work on them
then I think we all have to accept that these parts of the OFA stack
are unfinished and unmaintained - and see if that can be delt with.

> Jason, can you restate what are the two problems you saw from
> David's reports? the 1st was related to scope in link-local
> addresses, and what's the 2nd?

The kernel must not accept a link local address with 0 scope id

When using link local addresses the kernel must bind to the interface
index in the scope id.

The scope is ignored for other addresses scopes.

The rules will be more complex if a source and dest are specified, in
that case the scopes must match.

This is the same basic problem as the other issues regarding ND and
route resolution, the RDMA CM does not follow the same procedures as
TCP does, and the variations produce bugs.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ewg] rping is not resolving ipv6 addresses

[David J. Wilder]

On Fri, 2009-10-09 at 16:46 -0600, Jason Gunthorpe wrote:
> On Fri, Oct 09, 2009 at 02:15:01PM -0700, David J. Wilder wrote:
> 
> > Yes ping6 will work but I must specify the interface to use:
> > ping6 fe80::202:c903:1:1925%ib0
> > 
> > Here is the patch to rping to allow the source address to be specified.
> > I am not sure this is the correct approach to fix this.
> 
> It is not, IPv6 link local addresses must be scoped. rping is
> parsing the address with getaddrinfo, that does correctly set the
> sin6_scope_id value in the sockaddr.

ping6 is scoping the address (setting sin6_scope_id) by hand. So rping
should do the same thing.

> 
> rdma_resolve_addr (and likely the underlying kernel component) must
> respect this when dealing with link local scope addresses.
> 
> rmda_resolve_addr should fail if a link local IPv6 address is provided
> without scope, like the normal IP stack:
> 
> connect(3, {sa_family=AF_INET6, sin6_port=htons(1025), inet_pton(AF_INET6, "fe80::202:c903:1:1925", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EINVAL (Invalid argument)
> 
> So this is two bugs in the rdma cm, rping is correct as is..

It may be correct, but not functional with link-local address.

> 
> Jason

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ewg] rping is not resolving ipv6 addresses

[Jason Gunthorpe]

On Mon, Oct 12, 2009 at 10:52:59AM -0700, David J. Wilder wrote:

> > It is not, IPv6 link local addresses must be scoped. rping is
> > parsing the address with getaddrinfo, that does correctly set the
> > sin6_scope_id value in the sockaddr.
> 
> ping6 is scoping the address (setting sin6_scope_id) by hand. So rping
> should do the same thing.

getaddrinfo already sets sin6_scope_id. Newish iputils ping6 also uses
getaddrinfo and doesn't touch the sin6_scope_id except in some
specialized cases.

> > So this is two bugs in the rdma cm, rping is correct as is..
> 
> It may be correct, but not functional with link-local address.

Like I said, the RDMA CM kernel component has the bugs, not rping..

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html