public inbox for linux-rdma@vger.kernel.org
 help / color / mirror / Atom feed
From: "Steve Wise" <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
To: 'Sean Hefty' <sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org,
	'Sagi Grimberg' <sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>,
	'Christoph Hellwig' <hch-jcswGhMUV9g@public.gmane.org>
Subject: RE: Flush warning
Date: Wed, 9 Aug 2017 11:21:38 -0500	[thread overview]
Message-ID: <00ff01d3112b$94142350$bc3c69f0$@opengridcomputing.com> (raw)
In-Reply-To: <003401d30f91$c7e2a3f0$57a7ebd0$@opengridcomputing.com>

> 
> > > The WARNING seems to be due to nvmet_rdma_queue_connect() calling
> > > flush_scheduled_work() while in the upcall from the RDMA_CM.  It I running
> on
> > > the iw_cm event workqueue, which is created with WQ_MEM_RECLAIM set.
> I'm
> > not
> > > sure what this WARNING is telling me.  Does the iw_cm workqueue NOT need
> > > WQ_MEM_RECLAIM set?  Or is there some other issue with the nvmet/rdma
> > code doing
> > > work flushing in the iw_cm workq context?
> >
> > This flush is designed to prevent nvmet-rdma from having too much
> > inflight resources in case of a high pace of controller teardown and
> > establishment (like you trigger in your test).
> >
> > queue teardowns are run on system_wq, does iw_cm needs memory
> > reclamation protection?
> 
> I don't know.  I read the workqueue doc on WQ_MEM_RECLAIM, but I don't know
> how
> to tell if iw_cm needs this or not.  Can you give me an example of a workqueue
> that _does_ need WQ_MEM_RECLAIM?  I _think_ it means your workqueue is
> required
> to run something that would get triggered by the oom OS code, but I don't know
> if that would include rdma CMs or not...

Many of the workqueues in infiniband/core use WQ_MEM_RECLAIM: cma, iwcm, mad,
multicast, sa_query, and ucma. 

Hey Sean, do you have any insight into whether the CMA modules really need
WQ_MEM_RECLAIM for their workqueues?

Does anyone else know?

Thanks!

Steve.


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

  reply	other threads:[~2017-08-09 16:21 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-08-03 18:32 Flush warning Steve Wise
2017-08-07  1:06 ` Sagi Grimberg
     [not found]   ` <9bc142de-b8ba-acb6-5ea1-2ccdbb578655-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-08-07 15:28     ` Steve Wise
2017-08-09 16:21       ` Steve Wise [this message]
2017-08-09 16:27         ` Jason Gunthorpe
     [not found]           ` <20170809162749.GA4069-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2017-08-09 16:38             ` Steve Wise
2017-08-13  6:46               ` Leon Romanovsky
     [not found]                 ` <20170813064651.GR24282-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2017-08-13  9:14                   ` Sagi Grimberg
     [not found]                     ` <90ada4f5-d6a1-2b93-5164-c593955c20cf-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-08-13 10:33                       ` Leon Romanovsky
     [not found]                         ` <20170813103359.GW24282-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2017-08-14 12:13                           ` Hal Rosenstock
     [not found]                             ` <3d9cca62-4612-53dc-776c-3aeb2df58365-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2017-08-14 17:01                               ` Hefty, Sean
     [not found]                                 ` <1828884A29C6694DAF28B7E6B8A82373AB17595A-P5GAC/sN6hkd3b2yrw5b5LfspsVTdybXVpNB7YpNyf8@public.gmane.org>
2017-08-14 17:21                                   ` Steve Wise
2017-08-15  9:09                                     ` Sagi Grimberg
     [not found]                                       ` <64134765-9a10-d014-3ac4-b0f747d6c670-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-08-15 13:42                                         ` Steve Wise

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='00ff01d3112b$94142350$bc3c69f0$@opengridcomputing.com' \
    --to=swise-7bpotxp6k4+p2yhjcf5u+vpxobypeauw@public.gmane.org \
    --cc=hch-jcswGhMUV9g@public.gmane.org \
    --cc=linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org \
    --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org \
    --cc=sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox