From: Tyrel Datwyler <tyreld@linux.ibm.com>
To: Hannes Reinecke <hare@suse.de>, james.bottomley@hansenpartnership.com
Cc: brking@linux.ibm.com, linuxppc-dev@lists.ozlabs.org,
linux-scsi@vger.kernel.org, martin.petersen@oracle.com,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH 00/13] ibmvfc: initial MQ development
Date: Wed, 2 Dec 2020 09:19:12 -0800 [thread overview]
Message-ID: <1d5ec685-7160-52da-417b-23a53bcfc47e@linux.ibm.com> (raw)
In-Reply-To: <90e9a8ac-d2b9-bb64-7c7d-607adaea0f26@suse.de>
On 12/2/20 4:03 AM, Hannes Reinecke wrote:
> On 11/26/20 2:48 AM, Tyrel Datwyler wrote:
>> Recent updates in pHyp Firmware and VIOS releases provide new infrastructure
>> towards enabling Subordinate Command Response Queues (Sub-CRQs) such that each
>> Sub-CRQ is a channel backed by an actual hardware queue in the FC stack on the
>> partner VIOS. Sub-CRQs are registered with the firmware via hypercalls and then
>> negotiated with the VIOS via new Management Datagrams (MADs) for channel setup.
>>
>> This initial implementation adds the necessary Sub-CRQ framework and implements
>> the new MADs for negotiating and assigning a set of Sub-CRQs to associated VIOS
>> HW backed channels. The event pool and locking still leverages the legacy single
>> queue implementation, and as such lock contention is problematic when increasing
>> the number of queues. However, this initial work demonstrates a 1.2x factor
>> increase in IOPs when configured with two HW queues despite lock contention.
>>
> Why do you still hold the hold lock during submission?
Proof of concept.
> An initial check on the submission code path didn't reveal anything obvious, so
> it _should_ be possible to drop the host lock there.
Its used to protect the event pool and the event free/sent lists. This could
probably have its own lock instead of the host lock.
> Or at least move it into the submission function itself to avoid lock
> contention. Hmm?
I have a followup patch to do that, but I didn't see any change in performance.
I've got another patch I'm finishing that provides dedicated event pools for
each subqueue such that they will no longer have any dependency on the host lock.
-Tyrel
>
> Cheers,
>
> Hannes
prev parent reply other threads:[~2020-12-02 17:21 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-26 1:48 [PATCH 00/13] ibmvfc: initial MQ development Tyrel Datwyler
2020-11-26 1:48 ` [PATCH 01/13] ibmvfc: add vhost fields and defaults for MQ enablement Tyrel Datwyler
2020-11-27 17:45 ` Brian King
2020-11-30 17:22 ` Tyrel Datwyler
2020-11-26 1:48 ` [PATCH 02/13] ibmvfc: define hcall wrapper for registering a Sub-CRQ Tyrel Datwyler
2020-11-27 17:45 ` Brian King
2020-11-26 1:48 ` [PATCH 03/13] ibmvfc: add Subordinate CRQ definitions Tyrel Datwyler
2020-11-27 17:46 ` Brian King
2020-11-26 1:48 ` [PATCH 04/13] ibmvfc: add alloc/dealloc routines for SCSI Sub-CRQ Channels Tyrel Datwyler
2020-11-27 17:46 ` Brian King
2020-11-30 17:26 ` Tyrel Datwyler
2020-11-26 1:48 ` [PATCH 05/13] ibmvfc: add Sub-CRQ IRQ enable/disable routine Tyrel Datwyler
2020-11-27 17:47 ` Brian King
2020-11-26 1:48 ` [PATCH 06/13] ibmvfc: add handlers to drain and complete Sub-CRQ responses Tyrel Datwyler
2020-11-27 17:47 ` Brian King
2020-11-30 17:27 ` Tyrel Datwyler
2020-11-26 1:48 ` [PATCH 07/13] ibmvfc: define Sub-CRQ interrupt handler routine Tyrel Datwyler
2020-11-27 17:48 ` Brian King
2020-11-26 1:48 ` [PATCH 08/13] ibmvfc: map/request irq and register Sub-CRQ interrupt handler Tyrel Datwyler
2020-11-27 17:48 ` Brian King
2020-11-26 1:48 ` [PATCH 09/13] ibmvfc: implement channel enquiry and setup commands Tyrel Datwyler
2020-11-27 17:49 ` Brian King
2020-11-30 17:29 ` Tyrel Datwyler
2020-11-26 1:48 ` [PATCH 10/13] ibmvfc: advertise client support for using hardware channels Tyrel Datwyler
2020-11-27 17:49 ` Brian King
2020-11-26 1:48 ` [PATCH 11/13] ibmvfc: set and track hw queue in ibmvfc_event struct Tyrel Datwyler
2020-11-27 17:50 ` Brian King
2020-11-26 1:48 ` [PATCH 12/13] ibmvfc: send commands down HW Sub-CRQ when channelized Tyrel Datwyler
2020-11-27 17:50 ` Brian King
2020-11-26 1:48 ` [PATCH 13/13] ibmvfc: register Sub-CRQ handles with VIOS during channel setup Tyrel Datwyler
2020-11-27 17:50 ` Brian King
2020-12-02 12:03 ` [PATCH 00/13] ibmvfc: initial MQ development Hannes Reinecke
2020-12-02 17:19 ` Tyrel Datwyler [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1d5ec685-7160-52da-417b-23a53bcfc47e@linux.ibm.com \
--to=tyreld@linux.ibm.com \
--cc=brking@linux.ibm.com \
--cc=hare@suse.de \
--cc=james.bottomley@hansenpartnership.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=martin.petersen@oracle.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).