From: sagi@lightbits.io (Sagi Grimberg)
Subject: nvme-fabrics: crash at nvme connect-all
Date: Thu, 9 Jun 2016 16:48:20 +0300 [thread overview]
Message-ID: <575973A4.9080001@lightbits.io> (raw)
In-Reply-To: <005701d1c253$f9590550$ec0b0ff0$@opengridcomputing.com>
>>> Steve, did you see this before? I'm wandering if we need some sort
>>> of logic handling with resource limitation in iWARP (global mrs pool...)
>>
>> Haven't seen this. Does 'cat /sys/kernel/debug/iw_cxgb4/blah/stats' show
>> anything interesting? Where/why is it crashing?
>>
>
> So this is the failure:
>
> [ 703.239462] rdma_rw_init_mrs: failed to allocated 128 MRs
> [ 703.239498] failed to init MR pool ret= -12
> [ 703.239541] nvmet_rdma: failed to create_qp ret= -12
> [ 703.239582] nvmet_rdma: nvmet_rdma_alloc_queue: creating RDMA queue failed
> (-12).
>
> Not sure why it would fail. I would think my setup would be allocating more
> given I have 16 cores on the host and target. The debugfs "stats" file I
> mentioned above should show us something if we're running out of adapter
> resources for MR or PBL records.
Note that Marta ran both the host and the target on the same machine.
So, 8 (cores) x 128 (queue entries) x 2 (host and target) gives 2048
MRs...
What is the T5 limitation?
next prev parent reply other threads:[~2016-06-09 13:48 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-09 9:18 nvme-fabrics: crash at nvme connect-all Marta Rybczynska
2016-06-09 9:29 ` Sagi Grimberg
2016-06-09 10:07 ` Marta Rybczynska
2016-06-09 11:09 ` Sagi Grimberg
2016-06-09 12:12 ` Marta Rybczynska
2016-06-09 12:30 ` Sagi Grimberg
2016-06-09 13:27 ` Steve Wise
2016-06-09 13:36 ` Steve Wise
2016-06-09 13:48 ` Sagi Grimberg [this message]
2016-06-09 14:09 ` Steve Wise
2016-06-09 14:22 ` Steve Wise
2016-06-09 14:29 ` Steve Wise
2016-06-09 15:04 ` Marta Rybczynska
2016-06-09 15:40 ` Steve Wise
2016-06-09 15:48 ` Steve Wise
2016-06-10 9:03 ` Marta Rybczynska
2016-06-10 13:40 ` Steve Wise
2016-06-10 13:42 ` Marta Rybczynska
2016-06-10 13:49 ` Steve Wise
2016-06-09 13:25 ` Christoph Hellwig
2016-06-09 13:24 ` Christoph Hellwig
2016-06-09 15:37 ` Marta Rybczynska
2016-06-09 20:25 ` Steve Wise
2016-06-09 20:35 ` Ming Lin
2016-06-09 21:06 ` Steve Wise
2016-06-09 22:26 ` Ming Lin
2016-06-09 22:40 ` Steve Wise
[not found] ` <055801d1c29f$e164c000$a42e4000$@opengridcomputing.com>
2016-06-10 15:11 ` Steve Wise
2016-06-10 16:22 ` Steve Wise
2016-06-10 18:43 ` Ming Lin
2016-06-10 19:17 ` Steve Wise
2016-06-10 20:00 ` Ming Lin
2016-06-10 20:15 ` Steve Wise
2016-06-10 20:18 ` Ming Lin
2016-06-10 21:14 ` Steve Wise
2016-06-10 21:20 ` Ming Lin
2016-06-10 21:25 ` Steve Wise
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=575973A4.9080001@lightbits.io \
--to=sagi@lightbits.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox