[Cluster-devel] [PATCH] dlm: fix lowcomms_connect_node for sctp

[Cluster-devel] [PATCH] dlm: fix lowcomms_connect_node for sctp

[David Teigland]

The recently added dlm_lowcomms_connect_node() from
391fbdc5d527149578490db2f1619951d91f3561 does not work
when using SCTP instead of TCP.  The sctp connection code
has nothing to do without data to send.  Check for no data
in the sctp connection code and do nothing instead of
triggering a BUG.  Also have connect_node() do nothing
when the protocol is sctp.

Signed-off-by: David Teigland <teigland@redhat.com>
---
 fs/dlm/lowcomms.c |   13 ++++++++++---
 1 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c
index 240cef1..a3350e4 100644
--- a/fs/dlm/lowcomms.c
+++ b/fs/dlm/lowcomms.c
@@ -316,6 +316,10 @@ int dlm_lowcomms_connect_node(int nodeid)
 {
 	struct connection *con;
 
+	/* with sctp there's no connecting without sending */
+	if (dlm_config.ci_protocol != 0)
+		return 0;
+
 	if (nodeid == dlm_our_nodeid())
 		return 0;
 
@@ -855,11 +859,14 @@ static void sctp_init_assoc(struct connection *con)
 	outmessage.msg_flags = MSG_EOR;
 
 	spin_lock(&con->writequeue_lock);
-	e = list_entry(con->writequeue.next, struct writequeue_entry,
-		       list);
 
-	BUG_ON((struct list_head *) e == &con->writequeue);
+	if (list_empty(&con->writequeue)) {
+		spin_unlock(&con->writequeue_lock);
+		log_print("writequeue empty for nodeid %d", con->nodeid);
+		return;
+	}
 
+	e = list_first_entry(&con->writequeue, struct writequeue_entry, list);
 	len = e->len;
 	offset = e->offset;
 	spin_unlock(&con->writequeue_lock);
-- 
1.5.5.6

[Cluster-devel] [PATCH] dlm: fix lowcomms_connect_node for sctp

[David Teigland]

On Fri, Sep 18, 2009 at 02:39:06PM -0500, David Teigland wrote:
> The recently added dlm_lowcomms_connect_node() from
> 391fbdc5d527149578490db2f1619951d91f3561 does not work
> when using SCTP instead of TCP.  The sctp connection code
> has nothing to do without data to send.  Check for no data
> in the sctp connection code and do nothing instead of
> triggering a BUG.  Also have connect_node() do nothing
> when the protocol is sctp.

With this patch I can use sctp (single interface), but I'm getting these
alloc_fd errors (which seem harmless enough) have you seen them before?

Also, gfs is *really* slow using sctp!

dlm: Using SCTP for communications
SCTP: Hash tables configured (established 58254 bind 58254)
dlm: foo: joining the lockspace group...
dlm: got new/restarted association 1 nodeid 4
dlm: foo: group event done 0 0
dlm: foo: recover 1
dlm: foo: add member 5
dlm: foo: add member 4
dlm: foo: add member 2
dlm: Initiating association with node 2
alloc_fd: slot 0 not NULL!
dlm: got new/restarted association 2 nodeid 2
dlm: foo: join complete
GFS2: fsid=bull:foo.2: Joined cluster. Now mounting FS...
dlm: foo: total members 3 error 0
dlm: foo: dlm_recover_directory
dlm: foo: dlm_recover_directory 13 entries

[Cluster-devel] [PATCH] dlm: fix lowcomms_connect_node for sctp

[Christine Caulfield]

On 18/09/09 22:37, David Teigland wrote:
> On Fri, Sep 18, 2009 at 02:39:06PM -0500, David Teigland wrote:
>> The recently added dlm_lowcomms_connect_node() from
>> 391fbdc5d527149578490db2f1619951d91f3561 does not work
>> when using SCTP instead of TCP.  The sctp connection code
>> has nothing to do without data to send.  Check for no data
>> in the sctp connection code and do nothing instead of
>> triggering a BUG.  Also have connect_node() do nothing
>> when the protocol is sctp.

That patch looks sane to me

> With this patch I can use sctp (single interface), but I'm getting these
> alloc_fd errors (which seem harmless enough) have you seen them before?

Yes I have seen them before. At the time I put them down to either a bug 
in SCTP or an artifact of the way it works, because lowcomms isn't 
explicitly allocating any new FDs here - they are hidden behind the sctp 
layer and passed back to use later on in process_sctp_notification().

> Also, gfs is *really* slow using sctp!


I haven't done any benchmarks for ages. The last time some were done I 
was told that SCTP was actually faster than TCP! ... though that was 
with the 'old' DLM. I put this down to the fact that SCTP is optimised 
for streaming because of its origins.

If it's now slowed down a lot it's probably worth investigating further 
- I really wouldn't expect the single-home configurations to differ by 
very much.

Chrissie.