From: Jason Gunthorpe <jgg@nvidia.com>
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Sagi Grimberg <sagi@grimberg.me>, Max Gurtovoy <maxg@nvidia.com>,
<linux-rdma@vger.kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Doug Ledford <dledford@redhat.com>
Subject: Re: [PATCH] IB/iser: Remove in_interrupt() usage.
Date: Fri, 27 Nov 2020 10:31:38 -0400 [thread overview]
Message-ID: <20201127143138.GG552508@nvidia.com> (raw)
In-Reply-To: <20201127141432.z5hqxosugi6uu6i7@linutronix.de>
On Fri, Nov 27, 2020 at 03:14:32PM +0100, Sebastian Andrzej Siewior wrote:
> On 2020-11-27 09:03:14 [-0400], Jason Gunthorpe wrote:
> > I was able to get the internal bug report that caused the
> > 7414dde0a6c3a commit.
> >
> > The issue here is that the state_mutex is protecting
> >
> > This:
> >
> > if (unlikely(iser_conn->state != ISER_CONN_UP)) {
> >
> > Which indicates that this:
> >
> > dma_addr = ib_dma_map_single(device->ib_device, (void *)tx_desc,
> >
> > Won't crash because iser_con->ib_con is invalid. The notes say that
> > the iSCSI stack is in some state where data traffic won't flow but
> > management traffic is still possible. I suppose this is some fast path
> > so it was "optimized" to eliminate the lock for data traffic.
> >
> > A call chain of interest for the lock at least is:
> >
> > Nov 3 12:24:37 rsws10 BUG: unable to handle kernel
> > Nov 3 12:24:37 NULL pointer dereference
> > Nov 3 12:24:37 rsws10 Pid: 5245, comm: scsi_eh_5 Tainted: GF O 3.8.13-16.2.1.el6uek.x86_64 #1 IBM System x3550 M3 -[7944KEG]-/90Y4784
> > [..]
> > Nov 3 12:24:37 rsws10 [<ffffffffa069d628>] iscsi_iser_task_init+0x28/0x70 [ib_iser]
> > Nov 3 12:24:37 rsws10 [<ffffffffa0610029>] iscsi_prep_mgmt_task+0x129/0x150 [libiscsi]
> > Nov 3 12:24:37 rsws10 [<ffffffffa061354c>] __iscsi_conn_send_pdu+0x23c/0x310 [libiscsi]
> > Nov 3 12:24:37 rsws10 [<ffffffffa0614277>] iscsi_exec_task_mgmt_fn+0x37/0x290 [libiscsi]
> > Nov 3 12:24:37 rsws10 [<ffffffffa061497b>] iscsi_eh_device_reset+0x1bb/0x2d0 [libiscsi]
>
> preemptible until here and this function has:
>
> | mutex_lock(&session->eh_mutex);
> | spin_lock_bh(&session->frwd_lock);
>
> I don't see the lock dropped between here and iscsi_iser_task_init().
Hmm, nor do I
This whole thing does look broken.
So.. it looks like the "fix" in 7414dde0a6c3a was adding the:
+ if (unlikely(iser_conn->state != ISER_CONN_UP)) {
Without any locking. Which is a pretty typical mistake :\
> Sure, I would do that but as noted above, it the `frwd_lock' is acquired
> so you can't acquire the mutex here.
Ok, well, I'm thinking this patch is OK as is. Lets wait for Max and Sagi
Jason
next prev parent reply other threads:[~2020-11-27 14:31 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-26 20:27 [PATCH] IB/iser: Remove in_interrupt() usage Sebastian Andrzej Siewior
2020-11-26 20:53 ` Jason Gunthorpe
2020-11-27 12:34 ` Sebastian Andrzej Siewior
2020-11-27 13:03 ` Jason Gunthorpe
2020-11-27 14:14 ` Sebastian Andrzej Siewior
2020-11-27 14:31 ` Jason Gunthorpe [this message]
2020-12-03 13:56 ` Sebastian Andrzej Siewior
2020-12-03 19:30 ` Sagi Grimberg
2020-12-04 17:42 ` [PATCH v2] " Sebastian Andrzej Siewior
2020-12-07 20:32 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201127143138.GG552508@nvidia.com \
--to=jgg@nvidia.com \
--cc=bigeasy@linutronix.de \
--cc=dledford@redhat.com \
--cc=linux-rdma@vger.kernel.org \
--cc=maxg@nvidia.com \
--cc=sagi@grimberg.me \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).