linux-scsi.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Bob Pearson <rpearsonhpe@gmail.com>
To: Bart Van Assche <bvanassche@acm.org>,
	Shinichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Cc: "linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>
Subject: Re: [bug report] blktests srp/002 hang
Date: Thu, 24 Aug 2023 11:24:16 -0500	[thread overview]
Message-ID: <c1c55bff-ed73-06c0-abea-ad80c70465d9@gmail.com> (raw)
In-Reply-To: <2668f6c9-df53-b3c5-3452-d411d11057e1@acm.org>

On 8/23/23 14:46, Bart Van Assche wrote:
> On 8/23/23 09:19, Bob Pearson wrote:
>> I have also seen the same hangs in siw. Not as frequently but the same symptoms.
>> About every month or so I take another run at trying to find and fix this bug but
>> I have not succeeded yet. I haven't seen anything that looks like bad behavior from
>> the rxe side but that doesn't prove anything. I also saw these hangs on my system
>> before the WQ patch went in if my memory serves. Out main application for this
>> driver at HPE is Lustre which is a little different than SRP but uses the same
>> general approach with fast MRs. Currently we are finding the driver to be quite stable
>> even under very heavy stress.
>>
>> I would be happy to collaborate with someone (you?) who knows the SRP side well to resolve
>> this hang. I think that is the quickest way to fix this. I have no idea what SRP is waiting for.
> 
> Hi Bob,
> 
> I cannot reproduce these issues. All SRP tests work reliably on my test setup on
> top of the v6.5-rc7 kernel, whether I use the siw driver or whether I use the
> rdma_rxe driver. Additionally, I do not see any SRP abort messages.

Thank you for this. This is good news.
> 
> # uname -a
> Linux opensuse-vm 6.5.0-rc7 #28 SMP PREEMPT_DYNAMIC Wed Aug 23 10:42:35 PDT 2023 x86_64 x86_64 x86_64 GNU/Linux
> # journalctl --since=today | grep 'SRP abort' | wc
>       0       0       0
> 
> Since I installed openSUSE Tumbleweed in the VM in which I run kernel tests: if
> you are using a Linux distro that is based on Debian it may include a buggy
> version of multipathd. Last time I ran the SRP tests in a Debian VM I had to
> build multipathd from source - the SRP tests did not work with the Debian version
> of multipathd. The shell script that I use to build and install multipathd is as
> follows (must be run in the multipath-tools source directory):

I run on Ubuntu which is Debian based. So perhaps that is the root of the problems
I have been seeing.

I'll try to follow your lead here.

Bob
> 
> #!/bin/bash
> 
> scriptdir="$(dirname "$0")"
> 
> if type -p zypper >/dev/null 2>&1; then
>     rpms=(device-mapper-devel libaio-devel libjson-c-devel librados-devel
>       liburcu-devel readline-devel systemd-devel)
>     for p in "${rpms[@]}"; do
>     sudo zypper install -y "$p"
>     done
> elif type -p apt-get >/dev/null 2>&1; then
>     export LIB=/lib
>     sudo apt-get install -y libaio-dev libdevmapper-dev libjson-c-dev librados-dev \
>         libreadline-dev libsystemd-dev liburcu-dev
> fi
> 
> git clean -f
> make -s "$@"
> sudo make -s "$@" install
> 
> Bart.


  reply	other threads:[~2023-08-24 16:25 UTC|newest]

Thread overview: 86+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-08-21  6:46 [bug report] blktests srp/002 hang Shinichiro Kawasaki
2023-08-22  1:46 ` Bob Pearson
2023-08-22 10:18   ` Shinichiro Kawasaki
2023-08-22 15:20     ` Bart Van Assche
2023-08-23 16:19       ` Bob Pearson
2023-08-23 19:46         ` Bart Van Assche
2023-08-24 16:24           ` Bob Pearson [this message]
2023-08-24  8:55         ` Bernard Metzler
2023-08-24 15:35         ` Bernard Metzler
2023-08-24 16:05           ` Bart Van Assche
2023-08-24 16:27             ` Bob Pearson
2023-08-25  1:11       ` Shinichiro Kawasaki
2023-08-25  1:36         ` Bob Pearson
2023-08-25 10:16           ` Shinichiro Kawasaki
2023-08-25 13:49           ` Bart Van Assche
2023-08-25 13:52         ` Bart Van Assche
2023-09-13 17:36           ` Bob Pearson
2023-09-13 23:38             ` Zhu Yanjun
2023-09-16  5:59               ` Zhu Yanjun
2023-09-19  4:14                 ` Shinichiro Kawasaki
2023-09-19  8:07                   ` Zhu Yanjun
2023-09-19 16:30                     ` Pearson, Robert B
2023-09-19 18:11                     ` Bob Pearson
2023-09-20  4:22                       ` Zhu Yanjun
2023-09-20 16:24                         ` Bob Pearson
2023-09-20 16:36                           ` Bart Van Assche
2023-09-20 17:18                             ` Bob Pearson
2023-09-20 17:22                               ` Bart Van Assche
2023-09-20 17:29                                 ` Bob Pearson
2023-09-21  5:46                                   ` Zhu Yanjun
2023-09-21 10:06                                   ` Zhu Yanjun
2023-09-21 14:23                                   ` Rain River
2023-09-21 14:39                                     ` Bob Pearson
2023-09-21 15:08                                       ` Zhu Yanjun
2023-09-21 15:10                                       ` Zhu Yanjun
2023-09-22 18:14                                         ` Bob Pearson
2023-09-22 22:06                                           ` Bart Van Assche
2023-09-24  1:17                                           ` Rain River
2023-09-25  4:47                                             ` Daisuke Matsuda (Fujitsu)
2023-09-25 14:31                                               ` Zhu Yanjun
2023-09-26  1:09                                                 ` Daisuke Matsuda (Fujitsu)
2023-09-26  6:09                                                   ` Zhu Yanjun
2023-09-25 15:00                                               ` Bart Van Assche
2023-09-25 15:25                                                 ` Bob Pearson
2023-09-25 15:52                                                 ` Jason Gunthorpe
2023-09-25 15:54                                                   ` Bob Pearson
2023-09-25 19:57                                                 ` Bob Pearson
2023-09-25 20:33                                                   ` Bart Van Assche
2023-09-25 20:40                                                     ` Bob Pearson
2023-09-26 15:36                                                   ` Rain River
2023-09-26  1:17                                                 ` Daisuke Matsuda (Fujitsu)
2023-10-17 17:09                                                   ` Bob Pearson
2023-10-17 17:13                                                     ` Bart Van Assche
2023-10-17 17:15                                                       ` Bob Pearson
2023-10-17 17:19                                                       ` Bob Pearson
2023-10-17 17:34                                                         ` Bart Van Assche
2023-10-17 17:58                                                     ` Jason Gunthorpe
2023-10-17 18:44                                                       ` Bob Pearson
2023-10-17 18:51                                                         ` Jason Gunthorpe
2023-10-17 19:55                                                           ` Bob Pearson
2023-10-17 20:06                                                             ` Bart Van Assche
2023-10-17 20:13                                                               ` Bob Pearson
2023-10-17 21:14                                                               ` Bob Pearson
2023-10-17 21:18                                                                 ` Bart Van Assche
2023-10-17 21:23                                                                   ` Bob Pearson
2023-10-17 21:30                                                                     ` Bart Van Assche
2023-10-17 21:39                                                                       ` Bob Pearson
2023-10-17 22:42                                                                         ` Bart Van Assche
2023-10-18 18:29                                                                           ` Bob Pearson
2023-10-18 19:17                                                                             ` Jason Gunthorpe
2023-10-18 19:48                                                                               ` Bart Van Assche
2023-10-18 20:03                                                                                 ` Bob Pearson
2023-10-18 20:04                                                                                 ` Bob Pearson
2023-10-18 20:14                                                                                 ` Bob Pearson
     [not found]                                                                                 ` <fb5f6da5-5017-440d-9cb5-38796554366c@gmail.com>
2023-10-18 20:49                                                                                   ` Bart Van Assche
2023-10-18 21:17                                                                                     ` Pearson, Robert B
2023-10-18 21:27                                                                                       ` Bart Van Assche
2023-10-18 21:52                                                                                         ` Bob Pearson
2023-10-19 19:17                                                                                           ` Bart Van Assche
2023-10-20 17:12                                                                                             ` Bob Pearson
2023-10-20 17:41                                                                                               ` Bart Van Assche
2023-10-18 19:38                                                                             ` Bart Van Assche
2023-10-17 19:18                                                       ` Bart Van Assche
2023-10-18  8:16                                                     ` Zhu Yanjun
2023-09-22 11:06 ` Linux regression tracking #adding (Thorsten Leemhuis)
2023-10-13 12:51   ` Linux regression tracking #update (Thorsten Leemhuis)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=c1c55bff-ed73-06c0-abea-ad80c70465d9@gmail.com \
    --to=rpearsonhpe@gmail.com \
    --cc=bvanassche@acm.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=shinichiro.kawasaki@wdc.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).