[Cluster-devel] [PATCH] [DLM] Bind connections from known local address when using TCP

[Cluster-devel] [PATCH] [DLM] Bind connections from known local address when using TCP

[Lon Hohberger]

A common problem occurs when multiple IP addresses within the same
subnet are assigned to the same NIC.  If we make a connection attempt to
another address on the same subnet as one of those addresses, the
connection attempt will not necessarily be routed from the address we
want.

In the case of the DLM, the other nodes will quickly drop the connection
attempt, causing problems.

This patch makes the DLM bind to the local address it acquired from the
cluster manager when using TCP prior to making a connection, obviating
the need for administrators to "fix" their systems or use clever routing
tricks.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: dlm-lowcomms-tcp-src-upstream.patch
Type: text/x-patch
Size: 1099 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/cluster-devel/attachments/20071025/cc427c3d/attachment.bin>

[Cluster-devel] [PATCH] [DLM] Bind connections from known local address when using TCP

[Lon Hohberger]

On Thu, 2007-10-25 at 18:49 -0400, Lon Hohberger wrote:
> A common problem occurs when multiple IP addresses within the same
> subnet are assigned to the same NIC.  If we make a connection attempt to
> another address on the same subnet as one of those addresses, the
> connection attempt will not necessarily be routed from the address we
> want.
> 
> In the case of the DLM, the other nodes will quickly drop the connection
> attempt, causing problems.
> 
> This patch makes the DLM bind to the local address it acquired from the
> cluster manager when using TCP prior to making a connection, obviating
> the need for administrators to "fix" their systems or use clever routing
> tricks.
> 

[Cluster-devel] [PATCH] [DLM] Bind connections from known local address when using TCP

[Lon Hohberger]

Whoops; sorry for the spam there.

Patch with fixed comments.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: dlm-lowcomms-tcp-src-upstream.patch
Type: text/x-patch
Size: 1064 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/cluster-devel/attachments/20071025/eb7e9203/attachment.bin>

[Cluster-devel] Re: [PATCH] [DLM] Bind connections from known local address when using TCP

[Steven Whitehouse]

Hi,

On Thu, 2007-10-25 at 18:49 -0400, Lon Hohberger wrote:
> A common problem occurs when multiple IP addresses within the same
> subnet are assigned to the same NIC.  If we make a connection attempt to
> another address on the same subnet as one of those addresses, the
> connection attempt will not necessarily be routed from the address we
> want.
> 
> In the case of the DLM, the other nodes will quickly drop the connection
> attempt, causing problems.
> 
> This patch makes the DLM bind to the local address it acquired from the
> cluster manager when using TCP prior to making a connection, obviating
> the need for administrators to "fix" their systems or use clever routing
> tricks.
> 

I'm happy to apply the patch, but I wonder why not just set the prefsrc
for the route? That will fix it for all applications not just the DLM.
Also if there are multiple NICs in the system with the same IP address
set assigned to them, then this solution may fail whereas the prefsrc
solution will continue to work as that allows specification of a device
too (amoung other things),

Steve.

[Cluster-devel] Re: [PATCH] [DLM] Bind connections from known local address when using TCP

[Patrick Caulfield]

Steven Whitehouse wrote:
> Hi,
> 
> On Thu, 2007-10-25 at 18:49 -0400, Lon Hohberger wrote:
>> A common problem occurs when multiple IP addresses within the same
>> subnet are assigned to the same NIC.  If we make a connection attempt to
>> another address on the same subnet as one of those addresses, the
>> connection attempt will not necessarily be routed from the address we
>> want.
>>
>> In the case of the DLM, the other nodes will quickly drop the connection
>> attempt, causing problems.
>>
>> This patch makes the DLM bind to the local address it acquired from the
>> cluster manager when using TCP prior to making a connection, obviating
>> the need for administrators to "fix" their systems or use clever routing
>> tricks.
>>
> 
> I'm happy to apply the patch, but I wonder why not just set the prefsrc
> for the route? That will fix it for all applications not just the DLM.
> Also if there are multiple NICs in the system with the same IP address
> set assigned to them, then this solution may fail whereas the prefsrc
> solution will continue to work as that allows specification of a device
> too (amoung other things),

I think it's appropriate to fix this in the DLM code. DLM doesn't work without
it in this situation and there night (possibly!?) be applications that don't
want prefsrc setting.

-- 
Patrick

[Cluster-devel] Re: [PATCH] [DLM] Bind connections from known local address when using TCP

[Steven Whitehouse]

Hi,

On Tue, 2007-10-30 at 14:02 +0000, Patrick Caulfield wrote:
> Steven Whitehouse wrote:
> > Hi,
> > 
> > On Thu, 2007-10-25 at 18:49 -0400, Lon Hohberger wrote:
> >> A common problem occurs when multiple IP addresses within the same
> >> subnet are assigned to the same NIC.  If we make a connection attempt to
> >> another address on the same subnet as one of those addresses, the
> >> connection attempt will not necessarily be routed from the address we
> >> want.
> >>
> >> In the case of the DLM, the other nodes will quickly drop the connection
> >> attempt, causing problems.
> >>
> >> This patch makes the DLM bind to the local address it acquired from the
> >> cluster manager when using TCP prior to making a connection, obviating
> >> the need for administrators to "fix" their systems or use clever routing
> >> tricks.
> >>
> > 
> > I'm happy to apply the patch, but I wonder why not just set the prefsrc
> > for the route? That will fix it for all applications not just the DLM.
> > Also if there are multiple NICs in the system with the same IP address
> > set assigned to them, then this solution may fail whereas the prefsrc
> > solution will continue to work as that allows specification of a device
> > too (amoung other things),
> 
> I think it's appropriate to fix this in the DLM code. DLM doesn't work without
> it in this situation and there night (possibly!?) be applications that don't
> want prefsrc setting.
> 
Ok, well I'll put the patch in, but it still sounds very odd to me. If
they really need to do different things for different applications then
setting the fwmark and doing route by fwmark would be the sensible
solution,

Steve.

[Cluster-devel] Re: [PATCH] [DLM] Bind connections from known local address when using TCP

[Patrick Caulfield]

Steven Whitehouse wrote:
> Hi,
> 
> On Tue, 2007-10-30 at 14:02 +0000, Patrick Caulfield wrote:
>> Steven Whitehouse wrote:
>>> Hi,
>>>
>>> On Thu, 2007-10-25 at 18:49 -0400, Lon Hohberger wrote:
>>>> A common problem occurs when multiple IP addresses within the same
>>>> subnet are assigned to the same NIC.  If we make a connection attempt to
>>>> another address on the same subnet as one of those addresses, the
>>>> connection attempt will not necessarily be routed from the address we
>>>> want.
>>>>
>>>> In the case of the DLM, the other nodes will quickly drop the connection
>>>> attempt, causing problems.
>>>>
>>>> This patch makes the DLM bind to the local address it acquired from the
>>>> cluster manager when using TCP prior to making a connection, obviating
>>>> the need for administrators to "fix" their systems or use clever routing
>>>> tricks.
>>>>
>>> I'm happy to apply the patch, but I wonder why not just set the prefsrc
>>> for the route? That will fix it for all applications not just the DLM.
>>> Also if there are multiple NICs in the system with the same IP address
>>> set assigned to them, then this solution may fail whereas the prefsrc
>>> solution will continue to work as that allows specification of a device
>>> too (amoung other things),
>> I think it's appropriate to fix this in the DLM code. DLM doesn't work without
>> it in this situation and there night (possibly!?) be applications that don't
>> want prefsrc setting.
>>
> Ok, well I'll put the patch in, but it still sounds very odd to me. If
> they really need to do different things for different applications then
> setting the fwmark and doing route by fwmark would be the sensible
> solution,

I think the point is that the DLM explicitly checks the source address of
incoming connections. The purpose of this is to ensure (to some extent at
least!) that it is talking to a known cluster node. If we didn't do this check
(and I'm reluctant to take it out) then we could also do without this bind code.

All we're doing here really, is matching the two halves of the connection code
up properly.

Patrick

[Cluster-devel] [PATCH] dlm: bind connections from known local address when using TCP

[David Teigland]

From: Lon Hohberger <lhh@redhat.com>

A common problem occurs when multiple IP addresses within the same
subnet are assigned to the same NIC.  If we make a connection attempt to
another address on the same subnet as one of those addresses, the
connection attempt will not necessarily be routed from the address we
want.

In the case of the DLM, the other nodes will quickly drop the connection
attempt, causing problems.

This patch makes the DLM bind to the local address it acquired from the
cluster manager when using TCP prior to making a connection, obviating
the need for administrators to "fix" their systems or use clever routing
tricks.

Signed-off-by: Lon Hohberger <lhh@redhat.com>
Signed-off-by: Patrick Caulfield <pcaulfie@redhat.com>
Signed-off-by: David Teigland <teigland@redhat.com>
---
 fs/dlm/lowcomms.c |   13 ++++++++++++-
 1 files changed, 12 insertions(+), 1 deletions(-)

diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c
index e9923ca..0bea802 100644
--- a/fs/dlm/lowcomms.c
+++ b/fs/dlm/lowcomms.c
@@ -864,7 +864,7 @@ static void sctp_init_assoc(struct connection *con)
 static void tcp_connect_to_sock(struct connection *con)
 {
 	int result = -EHOSTUNREACH;
-	struct sockaddr_storage saddr;
+	struct sockaddr_storage saddr, src_addr;
 	int addr_len;
 	struct socket *sock;
 
@@ -898,6 +898,17 @@ static void tcp_connect_to_sock(struct connection *con)
 	con->connect_action = tcp_connect_to_sock;
 	add_sock(sock, con);
 
+	/* Bind to our cluster-known address connecting to avoid
+	   routing problems */
+	memcpy(&src_addr, dlm_local_addr[0], sizeof(src_addr));
+	make_sockaddr(&src_addr, 0, &addr_len);
+	result = sock->ops->bind(sock, (struct sockaddr *) &src_addr,
+				 addr_len);
+	if (result < 0) {
+		printk("dlm: could not bind for connect: %d\n", result);
+		/* This *may* not indicate a critical error */
+	}
+
 	make_sockaddr(&saddr, dlm_config.ci_tcp_port, &addr_len);
 
 	log_print("connecting to %d", con->nodeid);
-- 
1.5.3.3