Multiple connections from the same source address.

Multiple connections from the same source address.

[David Laight]

Can anyone think how to create multiple outgoing connections
from the same subset of local IP addresses and the same port?

We bind() to the first IP address and port, then use
SCTP_SOCKOPT_BINDX_ADD to add the second IP address.
But even with IP_REUASADDR set you can't repeat that on
a second socket.

It is possible to bind a second socket by reversing the
order of the IP addresses (this might be deemed to be a bug!)

If you bind to IN_ADDR_ANY then a second socket can be bound
to the same port once the outgoing connection has been made
(and the actual local address assigned).
This works for TCP, but for SCTP you almost always need to
constrain the local addresses advertised on the connection.

Ideas?

I've not tried binding to IN_ADDR_ANY and using bindx to
add all the addresses.

	David

Re: Multiple connections from the same source address.

[Michael Tuexen]

> On 8. Sep 2017, at 11:03, David Laight <David.Laight@ACULAB.COM> wrote:
> 
> Can anyone think how to create multiple outgoing connections
> from the same subset of local IP addresses and the same port?
> 
> We bind() to the first IP address and port, then use
> SCTP_SOCKOPT_BINDX_ADD to add the second IP address.
> But even with IP_REUASADDR set you can't repeat that on
> a second socket.
> 
> It is possible to bind a second socket by reversing the
> order of the IP addresses (this might be deemed to be a bug!)
> 
> If you bind to IN_ADDR_ANY then a second socket can be bound
> to the same port once the outgoing connection has been made
> (and the actual local address assigned).
> This works for TCP, but for SCTP you almost always need to
> constrain the local addresses advertised on the connection.
> 
> Ideas?
Why not use a 1-to-many style socket?

Best regards
Michael
> 
> I've not tried binding to IN_ADDR_ANY and using bindx to
> add all the addresses.
> 
> 	David
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Multiple connections from the same source address.

[David Laight]

From: Michael Tuexen
> Sent: 08 September 2017 10:30
> > On 8. Sep 2017, at 11:03, David Laight <David.Laight@ACULAB.COM> wrote:
> >
> > Can anyone think how to create multiple outgoing connections
> > from the same subset of local IP addresses and the same port?
> >
> > We bind() to the first IP address and port, then use
> > SCTP_SOCKOPT_BINDX_ADD to add the second IP address.
> > But even with IP_REUASADDR set you can't repeat that on
> > a second socket.
> >
> > It is possible to bind a second socket by reversing the
> > order of the IP addresses (this might be deemed to be a bug!)
> >
> > If you bind to IN_ADDR_ANY then a second socket can be bound
> > to the same port once the outgoing connection has been made
> > (and the actual local address assigned).
> > This works for TCP, but for SCTP you almost always need to
> > constrain the local addresses advertised on the connection.
> >
> > Ideas?
> Why not use a 1-to-many style socket?

Because we don't know until we try to make a connection
that the local addresses match an existing one.
And it would complicate the code horribly.

In principle it could be a completely separate application.
(Although in our case it is a kernel M3UA driver.)

	David

Re: Multiple connections from the same source address.

[Marcelo Ricardo Leitner]

On Fri, Sep 08, 2017 at 09:03:31AM +0000, David Laight wrote:
> Can anyone think how to create multiple outgoing connections
> from the same subset of local IP addresses and the same port?
> 
> We bind() to the first IP address and port, then use
> SCTP_SOCKOPT_BINDX_ADD to add the second IP address.
> But even with IP_REUASADDR set you can't repeat that on
> a second socket.

When did you set IP_REUSEADDR? Checking the code, seems you have to set
it before any binding, because then the port entry will be created and
such flag will be initialized with that state.

> 
> It is possible to bind a second socket by reversing the
> order of the IP addresses (this might be deemed to be a bug!)

Interesting. It does seem like a bug. Probably related to the above
behavior.

> 
> If you bind to IN_ADDR_ANY then a second socket can be bound
> to the same port once the outgoing connection has been made
> (and the actual local address assigned).
> This works for TCP, but for SCTP you almost always need to
> constrain the local addresses advertised on the connection.
> 
> Ideas?
> 
> I've not tried binding to IN_ADDR_ANY and using bindx to
> add all the addresses.
> 
> 	David
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

RE: Multiple connections from the same source address.

[David Laight]

From: Marcelo Ricardo Leitner
> Sent: 08 September 2017 15:50
> On Fri, Sep 08, 2017 at 09:03:31AM +0000, David Laight wrote:
> > Can anyone think how to create multiple outgoing connections
> > from the same subset of local IP addresses and the same port?
> >
> > We bind() to the first IP address and port, then use
> > SCTP_SOCKOPT_BINDX_ADD to add the second IP address.
> > But even with IP_REUASADDR set you can't repeat that on
> > a second socket.
> 
> When did you set IP_REUSEADDR? Checking the code, seems you have to set
> it before any binding, because then the port entry will be created and
> such flag will be initialized with that state.

Indeed, otherwise it has no effect.

The point is that (IIRC) IP_REUSEADDR only lets you bind the same
port number to 2 sockets - the IP addresses have to be different.
Code that does this for TCP (like the old ftp data connection)
only worked because the address it was binding was IN_ADDR_ANY
and the existing socket had a local IP address.
It always had to retry because two sockets couldn't be bound to
the port and IN_ADDR_ANY either.

> > It is possible to bind a second socket by reversing the
> > order of the IP addresses (this might be deemed to be a bug!)
> 
> Interesting. It does seem like a bug. Probably related to the above
> behavior.

More likely the duplicate check is in some generic code that
doesn't look at the extra addresses.

	David

Re: Multiple connections from the same source address.

['Marcelo Ricardo Leitner']

On Fri, Sep 08, 2017 at 03:01:22PM +0000, David Laight wrote:
> From: Marcelo Ricardo Leitner
> > Sent: 08 September 2017 15:50
> > On Fri, Sep 08, 2017 at 09:03:31AM +0000, David Laight wrote:
> > > Can anyone think how to create multiple outgoing connections
> > > from the same subset of local IP addresses and the same port?
> > >
> > > We bind() to the first IP address and port, then use
> > > SCTP_SOCKOPT_BINDX_ADD to add the second IP address.
> > > But even with IP_REUASADDR set you can't repeat that on
> > > a second socket.
> > 
> > When did you set IP_REUSEADDR? Checking the code, seems you have to set
> > it before any binding, because then the port entry will be created and
> > such flag will be initialized with that state.
> 
> Indeed, otherwise it has no effect.
> 

For both sockets, right? I'll debug this if yes.

> The point is that (IIRC) IP_REUSEADDR only lets you bind the same
> port number to 2 sockets - the IP addresses have to be different.

I'm not seeing where this could be happening. I see this:
                if (pp->fastreuse && sk->sk_reuse &&
                        sk->sk_state != SCTP_SS_LISTENING)
                        goto success;
Meaning that if the port allows reusing and so does the new socket, and
it's not in listening state, it would be allowed.

> Code that does this for TCP (like the old ftp data connection)
> only worked because the address it was binding was IN_ADDR_ANY
> and the existing socket had a local IP address.
> It always had to retry because two sockets couldn't be bound to
> the port and IN_ADDR_ANY either.
> 
> > > It is possible to bind a second socket by reversing the
> > > order of the IP addresses (this might be deemed to be a bug!)
> > 
> > Interesting. It does seem like a bug. Probably related to the above
> > behavior.
> 
> More likely the duplicate check is in some generic code that
> doesn't look at the extra addresses.

Can be.

  Marcelo

Re: Multiple connections from the same source address.

[Michael Tuexen]

> On 8. Sep 2017, at 11:37, David Laight <David.Laight@ACULAB.COM> wrote:
> 
> From: Michael Tuexen
>> Sent: 08 September 2017 10:30
>>> On 8. Sep 2017, at 11:03, David Laight <David.Laight@ACULAB.COM> wrote:
>>> 
>>> Can anyone think how to create multiple outgoing connections
>>> from the same subset of local IP addresses and the same port?
>>> 
>>> We bind() to the first IP address and port, then use
>>> SCTP_SOCKOPT_BINDX_ADD to add the second IP address.
>>> But even with IP_REUASADDR set you can't repeat that on
>>> a second socket.
>>> 
>>> It is possible to bind a second socket by reversing the
>>> order of the IP addresses (this might be deemed to be a bug!)
>>> 
>>> If you bind to IN_ADDR_ANY then a second socket can be bound
>>> to the same port once the outgoing connection has been made
>>> (and the actual local address assigned).
>>> This works for TCP, but for SCTP you almost always need to
>>> constrain the local addresses advertised on the connection.
>>> 
>>> Ideas?
>> Why not use a 1-to-many style socket?
> 
> Because we don't know until we try to make a connection
> that the local addresses match an existing one.
> And it would complicate the code horribly.
OK, I see. I just double checked. If Linux follows RFC 6458,
it should be possible to do
fd = socket(AF_INET, SOCK_STREAM, IPPROTO_SCTP);
setsockopt(fd, IPPROTO_SCTP, SCTP_REUSE_PORT, &on, sizeof(int));
and then call bind().

I can run this kind of program on FreeBSD multiple times in
parallel, each one talking to a different end-point.
I'm not sure if Linux supports SCTP_REUSE_PORT.

Best regards
Michael
> 
> In principle it could be a completely separate application.
> (Although in our case it is a kernel M3UA driver.)
> 
> 	David
> 

RE: Multiple connections from the same source address.

[David Laight]

From: 'Marcelo Ricardo Leitner'
> Sent: 08 September 2017 16:11
> On Fri, Sep 08, 2017 at 03:01:22PM +0000, David Laight wrote:
> > From: Marcelo Ricardo Leitner
> > > Sent: 08 September 2017 15:50
> > > On Fri, Sep 08, 2017 at 09:03:31AM +0000, David Laight wrote:
> > > > Can anyone think how to create multiple outgoing connections
> > > > from the same subset of local IP addresses and the same port?
> > > >
> > > > We bind() to the first IP address and port, then use
> > > > SCTP_SOCKOPT_BINDX_ADD to add the second IP address.
> > > > But even with IP_REUASADDR set you can't repeat that on
> > > > a second socket.
> > >
> > > When did you set IP_REUSEADDR? Checking the code, seems you have to set
> > > it before any binding, because then the port entry will be created and
> > > such flag will be initialized with that state.
> >
> > Indeed, otherwise it has no effect.
> >
> 
> For both sockets, right? I'll debug this if yes.
> 
> > The point is that (IIRC) IP_REUSEADDR only lets you bind the same
> > port number to 2 sockets - the IP addresses have to be different.
> 
> I'm not seeing where this could be happening. I see this:
>                 if (pp->fastreuse && sk->sk_reuse &&
>                         sk->sk_state != SCTP_SS_LISTENING)
>                         goto success;
> Meaning that if the port allows reusing and so does the new socket, and
> it's not in listening state, it would be allowed.

I wonder if the behaviour has changed since I last tested it.
(Nothing obvious since 2008)

	David

RE: Multiple connections from the same source address.

[David Laight]

From: Michael Tuexen
> Sent: 08 September 2017 16:29
..
> OK, I see. I just double checked. If Linux follows RFC 6458,
> it should be possible to do
> fd = socket(AF_INET, SOCK_STREAM, IPPROTO_SCTP);
> setsockopt(fd, IPPROTO_SCTP, SCTP_REUSE_PORT, &on, sizeof(int));
> and then call bind().
> 
> I can run this kind of program on FreeBSD multiple times in
> parallel, each one talking to a different end-point.
> I'm not sure if Linux supports SCTP_REUSE_PORT.

Are you binding to a local address as well as a port?

	David

Re: Multiple connections from the same source address.

[Michael Tuexen]

> On 8. Sep 2017, at 17:42, David Laight <David.Laight@ACULAB.COM> wrote:
> 
> From: Michael Tuexen
>> Sent: 08 September 2017 16:29
> ..
>> OK, I see. I just double checked. If Linux follows RFC 6458,
>> it should be possible to do
>> fd = socket(AF_INET, SOCK_STREAM, IPPROTO_SCTP);
>> setsockopt(fd, IPPROTO_SCTP, SCTP_REUSE_PORT, &on, sizeof(int));
>> and then call bind().
>> 
>> I can run this kind of program on FreeBSD multiple times in
>> parallel, each one talking to a different end-point.
>> I'm not sure if Linux supports SCTP_REUSE_PORT.
> 
> Are you binding to a local address as well as a port?
Yes. In the test program I wrote, I was binding to a public
IPv4 address. But again: This was a userland program on FreeBSD.
Just to make sure that something you want to do should work, if
the SCTP socket API is implemented as specified in RFC 6458...
I have not tested if Linux supports the SCTP_REUSE_PORT socket option.

Best regards
Michael
> 
> 	David
>