cluster-devel.redhat.com archive mirror
 help / color / mirror / Atom feed
From: Alexander Aring <aahringo@redhat.com>
To: cluster-devel.redhat.com
Subject: [Cluster-devel] [PATCHv2 dlm-next 0/3] fs: dlm: receive handling changes
Date: Mon, 27 Jul 2020 09:13:35 -0400	[thread overview]
Message-ID: <20200727131338.9111-1-aahringo@redhat.com> (raw)

Hi,

this patch series will do some changes according to the receive handling
in dlm. My goal is to survive tcpkill [0] which is my main testcase while
doing some heavy load on dlm. The code is still in a situation where dlm
can end in a deadlock state while e.g. using gfs2. For generating a lot
of dlm traffic I use gfs2 and running make_panic [1].

I mainly just run make_panic on two nodes and let run:

tcpkill -9 -i $IFACE port 21064

on both nodes. The first two patches changes the behaviour how to react
on invalid dlm messages (which does not occur on my test case). The third
patch increase in my test scenario the stability of dlm connection and
running tcpkill. The last patch will ensure that the other end agrees
to close the socket if we want to do a clean tcp termination.

FUTURE WORK:

I still get deadlocks, as PATCH 2/3 shows that I think the TCP stack
will drop some packets when receiving a tcp reset (and the application
layer think it's successful send). This is one of my future work to
solve this behaviour. I also think about to extend the test case to e.g.
umount/mount gfs2 while my testcase is running and doing randomly
drops/delaying/reordering of dlm tcp by using the introduced mark
functionality and traffic control.

- Alex

[0] https://salsa.debian.org/pkg-security-team/dsniff
[1] https://fedorapeople.org/cgit/teigland/public_git/dct-stuff.git/tree/fs/make_panic

changes since v2:
 - fixup patch 2/4 with 1/4, otherwise we have some broken state
 - correct cbuf_eat() to "eat" full received buffer size in case of
   receiption of invalid dlm message.

Alexander Aring (3):
  fs: dlm: don't close socket on invalid message
  fs: dlm: change handling of reconnects
  fs: dlm: implement tcp graceful shutdown

 fs/dlm/lowcomms.c | 112 +++++++++++++++++++++++++++++++++++-----------
 1 file changed, 87 insertions(+), 25 deletions(-)

-- 
2.26.2



             reply	other threads:[~2020-07-27 13:13 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-27 13:13 Alexander Aring [this message]
2020-07-27 13:13 ` [Cluster-devel] [PATCHv2 dlm-next 1/3] fs: dlm: don't close socket on invalid message Alexander Aring
2020-07-27 13:13 ` [Cluster-devel] [PATCHv2 dlm-next 2/3] fs: dlm: change handling of reconnects Alexander Aring
2020-07-27 13:13 ` [Cluster-devel] [PATCHv2 dlm-next 3/3] fs: dlm: implement tcp graceful shutdown Alexander Aring

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200727131338.9111-1-aahringo@redhat.com \
    --to=aahringo@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).