All of lore.kernel.org
 help / color / mirror / Atom feed
From: Bart Van Assche <Bart.VanAssche@sandisk.com>
To: "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
	"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
	"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
	"target-devel@vger.kernel.org" <target-devel@vger.kernel.org>,
	"sagi@grimberg.me" <sagi@grimberg.me>
Subject: Re: [PATCH rfc 06/10] IB/cq: Don't force IB_POLL_DIRECT poll context for ib_process_cq_direct
Date: Tue, 14 Mar 2017 21:32:12 +0000	[thread overview]
Message-ID: <1489527119.2676.15.camel@sandisk.com> (raw)
In-Reply-To: <f4ade02b-9d17-20ae-f910-47c718f2a5bd@grimberg.me>

On Mon, 2017-03-13 at 10:24 +0200, Sagi Grimberg wrote:
> > Before this patch
> > the completions from each CQ were processed sequentially. That's a big
> > change so I think this should be mentioned clearly in the header above
> > ib_process_cq_direct().
>=20
> Note that I now see that the cq lock is not sufficient for mutual
> exclusion here because we're referencing cq->wc array outside of it.
>=20
> There are three options I see here:
> 1. we'll need to allocate a different wc array for polling mode,
> perhaps a smaller one?
> 2. Export __ib_process_cq (in some form) with an option to pass in a wc
> array.
> 3. Simply not support non-selective polling but it seems like a shame...
>=20
> Any thoughts?

I doubt it is possible to come up with an algorithm that recognizes whether
or not two different ib_process_cq() calls are serialized. So the
ib_process_cq() caller will have to provide that information. How about add=
ing
an ib_wc array argument to ib_process_cq() and modifying __ib_process_cq()
such that it uses that array if specified and cq->wc otherwise?

Bart.=

WARNING: multiple messages have this Message-ID (diff)
From: Bart.VanAssche@sandisk.com (Bart Van Assche)
Subject: [PATCH rfc 06/10] IB/cq: Don't force IB_POLL_DIRECT poll context for ib_process_cq_direct
Date: Tue, 14 Mar 2017 21:32:12 +0000	[thread overview]
Message-ID: <1489527119.2676.15.camel@sandisk.com> (raw)
In-Reply-To: <f4ade02b-9d17-20ae-f910-47c718f2a5bd@grimberg.me>

On Mon, 2017-03-13@10:24 +0200, Sagi Grimberg wrote:
> > Before this patch
> > the completions from each CQ were processed sequentially. That's a big
> > change so I think this should be mentioned clearly in the header above
> > ib_process_cq_direct().
> 
> Note that I now see that the cq lock is not sufficient for mutual
> exclusion here because we're referencing cq->wc array outside of it.
> 
> There are three options I see here:
> 1. we'll need to allocate a different wc array for polling mode,
> perhaps a smaller one?
> 2. Export __ib_process_cq (in some form) with an option to pass in a wc
> array.
> 3. Simply not support non-selective polling but it seems like a shame...
> 
> Any thoughts?

I doubt it is possible to come up with an algorithm that recognizes whether
or not two different ib_process_cq() calls are serialized. So the
ib_process_cq() caller will have to provide that information. How about adding
an ib_wc array argument to ib_process_cq() and modifying __ib_process_cq()
such that it uses that array if specified and cq->wc otherwise?

Bart.

WARNING: multiple messages have this Message-ID (diff)
From: Bart Van Assche <Bart.VanAssche@sandisk.com>
To: "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
	"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
	"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
	"target-devel@vger.kernel.org" <target-devel@vger.kernel.org>,
	"sagi@grimberg.me" <sagi@grimberg.me>
Subject: Re: [PATCH rfc 06/10] IB/cq: Don't force IB_POLL_DIRECT poll context for ib_process_cq_direct
Date: Tue, 14 Mar 2017 21:32:12 +0000	[thread overview]
Message-ID: <1489527119.2676.15.camel@sandisk.com> (raw)
In-Reply-To: <f4ade02b-9d17-20ae-f910-47c718f2a5bd@grimberg.me>

On Mon, 2017-03-13 at 10:24 +0200, Sagi Grimberg wrote:
> > Before this patch
> > the completions from each CQ were processed sequentially. That's a big
> > change so I think this should be mentioned clearly in the header above
> > ib_process_cq_direct().
> 
> Note that I now see that the cq lock is not sufficient for mutual
> exclusion here because we're referencing cq->wc array outside of it.
> 
> There are three options I see here:
> 1. we'll need to allocate a different wc array for polling mode,
> perhaps a smaller one?
> 2. Export __ib_process_cq (in some form) with an option to pass in a wc
> array.
> 3. Simply not support non-selective polling but it seems like a shame...
> 
> Any thoughts?

I doubt it is possible to come up with an algorithm that recognizes whether
or not two different ib_process_cq() calls are serialized. So the
ib_process_cq() caller will have to provide that information. How about adding
an ib_wc array argument to ib_process_cq() and modifying __ib_process_cq()
such that it uses that array if specified and cq->wc otherwise?

Bart.

  reply	other threads:[~2017-03-14 21:34 UTC|newest]

Thread overview: 85+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-03-09 13:16 [PATCH rfc 00/10] non selective polling block interface Sagi Grimberg
2017-03-09 13:16 ` Sagi Grimberg
2017-03-09 13:16 ` Sagi Grimberg
2017-03-09 13:16 ` [PATCH rfc 01/10] nvme-pci: Split __nvme_process_cq to poll and handle Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:57   ` Johannes Thumshirn
2017-03-09 13:57     ` Johannes Thumshirn
2017-03-09 13:57     ` Johannes Thumshirn
2017-03-22 19:07   ` Christoph Hellwig
2017-03-22 19:07     ` Christoph Hellwig
2017-03-22 19:07     ` Christoph Hellwig
2017-03-09 13:16 ` [PATCH rfc 02/10] nvme-pci: Add budget to __nvme_process_cq Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:46   ` Johannes Thumshirn
2017-03-09 13:46     ` Johannes Thumshirn
2017-03-09 13:46     ` Johannes Thumshirn
2017-03-22 19:08   ` Christoph Hellwig
2017-03-22 19:08     ` Christoph Hellwig
2017-03-09 13:16 ` [PATCH rfc 03/10] nvme-pci: open-code polling logic in nvme_poll Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:56   ` Johannes Thumshirn
2017-03-09 13:56     ` Johannes Thumshirn
2017-03-09 13:56     ` Johannes Thumshirn
2017-03-22 19:09   ` Christoph Hellwig
2017-03-22 19:09     ` Christoph Hellwig
2017-03-22 19:09     ` Christoph Hellwig
2017-03-09 13:16 ` [PATCH rfc 04/10] block: Add a non-selective polling interface Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:44   ` Johannes Thumshirn
2017-03-09 13:44     ` Johannes Thumshirn
2017-03-09 13:44     ` Johannes Thumshirn
2017-03-10  3:04     ` Damien Le Moal
2017-03-10  3:04       ` Damien Le Moal
2017-03-13  8:26       ` Sagi Grimberg
2017-03-13  8:26         ` Sagi Grimberg
2017-03-09 16:25   ` Bart Van Assche
2017-03-09 16:25     ` Bart Van Assche
2017-03-09 16:25     ` Bart Van Assche
2017-03-13  8:15     ` Sagi Grimberg
2017-03-13  8:15       ` Sagi Grimberg
2017-03-14 21:21       ` Bart Van Assche
2017-03-14 21:21         ` Bart Van Assche
2017-03-14 21:21         ` Bart Van Assche
2017-03-09 13:16 ` [PATCH rfc 05/10] nvme-pci: Support blk_poll_batch Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:16 ` [PATCH rfc 06/10] IB/cq: Don't force IB_POLL_DIRECT poll context for ib_process_cq_direct Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 16:30   ` Bart Van Assche
2017-03-09 16:30     ` Bart Van Assche
2017-03-09 16:30     ` Bart Van Assche
2017-03-13  8:24     ` Sagi Grimberg
2017-03-13  8:24       ` Sagi Grimberg
2017-03-14 21:32       ` Bart Van Assche [this message]
2017-03-14 21:32         ` Bart Van Assche
2017-03-14 21:32         ` Bart Van Assche
2017-03-09 13:16 ` [PATCH rfc 07/10] nvme-rdma: Don't rearm the CQ when polling directly Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:52   ` Johannes Thumshirn
2017-03-09 13:52     ` Johannes Thumshirn
2017-03-09 13:52     ` Johannes Thumshirn
2017-03-09 13:16 ` [PATCH rfc 08/10] nvme-rdma: Support blk_poll_batch Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:16 ` [PATCH rfc 09/10] nvmet: Use non-selective polling Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:54   ` Johannes Thumshirn
2017-03-09 13:54     ` Johannes Thumshirn
2017-03-09 13:54     ` Johannes Thumshirn
2017-03-09 13:16 ` [PATCH rfc 10/10] target: " Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-09 13:16   ` Sagi Grimberg
2017-03-18 23:58   ` Nicholas A. Bellinger
2017-03-18 23:58     ` Nicholas A. Bellinger
2017-03-18 23:58     ` Nicholas A. Bellinger
2017-03-21 11:35     ` Sagi Grimberg
2017-03-21 11:35       ` Sagi Grimberg
2017-03-21 11:35       ` Sagi Grimberg

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1489527119.2676.15.camel@sandisk.com \
    --to=bart.vanassche@sandisk.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=sagi@grimberg.me \
    --cc=target-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.