From: Sagi Grimberg <sagig@dev.mellanox.co.il>
To: Chuck Lever <chuck.lever@oracle.com>
Cc: linux-rdma@vger.kernel.org,
Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH v2 06/16] xprtrdma: Use workqueue to process RPC/RDMA replies
Date: Wed, 7 Oct 2015 18:17:26 +0300 [thread overview]
Message-ID: <56153786.2000801@dev.mellanox.co.il> (raw)
In-Reply-To: <4983C0DB-A6E3-4A64-B822-8D45D9847C31@oracle.com>
On 10/7/2015 5:48 PM, Chuck Lever wrote:
>
>> On Oct 7, 2015, at 10:39 AM, Sagi Grimberg <sagig@dev.mellanox.co.il> wrote:
>>
>> On 10/6/2015 5:59 PM, Chuck Lever wrote:
>>> The reply tasklet is fast, but it's single threaded. After reply
>>> traffic saturates a single CPU, there's no more reply processing
>>> capacity.
>>>
>>> Replace the tasklet with a workqueue to spread reply handling across
>>> all CPUs. This also moves RPC/RDMA reply handling out of the soft
>>> IRQ context and into a context that allows sleeps.
>>
>> Hi Chuck,
>>
>> I'm probably missing something here, but do you ever schedule in
>> the workqueue context? Don't you need to explicitly schedule after
>> a jiffie or so the code works also in a non fully preemptable kernel?
>
> Each RPC reply gets its own work request. This is unlike
> the tasklet, which continues to run as long as there are
> items on xprtrdma’s global tasklet queue.
OK I understand now.
So this and the rest of the series looks good to me.
Reviewed-by: Sagi Grimberg <sagig@mellanox.com>
WARNING: multiple messages have this Message-ID (diff)
From: Sagi Grimberg <sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
To: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Linux NFS Mailing List
<linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Subject: Re: [PATCH v2 06/16] xprtrdma: Use workqueue to process RPC/RDMA replies
Date: Wed, 7 Oct 2015 18:17:26 +0300 [thread overview]
Message-ID: <56153786.2000801@dev.mellanox.co.il> (raw)
In-Reply-To: <4983C0DB-A6E3-4A64-B822-8D45D9847C31-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
On 10/7/2015 5:48 PM, Chuck Lever wrote:
>
>> On Oct 7, 2015, at 10:39 AM, Sagi Grimberg <sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
>>
>> On 10/6/2015 5:59 PM, Chuck Lever wrote:
>>> The reply tasklet is fast, but it's single threaded. After reply
>>> traffic saturates a single CPU, there's no more reply processing
>>> capacity.
>>>
>>> Replace the tasklet with a workqueue to spread reply handling across
>>> all CPUs. This also moves RPC/RDMA reply handling out of the soft
>>> IRQ context and into a context that allows sleeps.
>>
>> Hi Chuck,
>>
>> I'm probably missing something here, but do you ever schedule in
>> the workqueue context? Don't you need to explicitly schedule after
>> a jiffie or so the code works also in a non fully preemptable kernel?
>
> Each RPC reply gets its own work request. This is unlike
> the tasklet, which continues to run as long as there are
> items on xprtrdma’s global tasklet queue.
OK I understand now.
So this and the rest of the series looks good to me.
Reviewed-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2015-10-07 15:17 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-06 14:58 [PATCH v2 00/16] NFS/RDMA patches for merging into v4.4 Chuck Lever
2015-10-06 14:58 ` Chuck Lever
2015-10-06 14:58 ` [PATCH v2 01/16] xprtrdma: Enable swap-on-NFS/RDMA Chuck Lever
2015-10-06 14:58 ` Chuck Lever
2015-10-06 14:58 ` [PATCH v2 02/16] xprtrdma: Re-arm after missed events Chuck Lever
2015-10-06 14:58 ` Chuck Lever
2015-10-06 18:17 ` Devesh Sharma
2015-10-06 18:17 ` Devesh Sharma
2015-10-06 14:58 ` [PATCH v2 03/16] xprtrdma: Prevent loss of completion signals Chuck Lever
2015-10-06 14:58 ` Chuck Lever
2015-10-06 18:15 ` Devesh Sharma
2015-10-06 18:15 ` Devesh Sharma
2015-10-06 14:59 ` [PATCH v2 04/16] xprtrdma: Refactor reply handler error handling Chuck Lever
2015-10-06 14:59 ` Chuck Lever
2015-10-06 18:21 ` Devesh Sharma
2015-10-06 18:21 ` Devesh Sharma
2015-10-06 14:59 ` [PATCH v2 05/16] xprtrdma: Replace send and receive arrays Chuck Lever
2015-10-06 14:59 ` Chuck Lever
2015-10-06 18:26 ` Devesh Sharma
2015-10-06 18:26 ` Devesh Sharma
2015-10-06 14:59 ` [PATCH v2 06/16] xprtrdma: Use workqueue to process RPC/RDMA replies Chuck Lever
2015-10-06 14:59 ` Chuck Lever
2015-10-06 18:30 ` Devesh Sharma
2015-10-06 18:30 ` Devesh Sharma
2015-10-07 14:39 ` Sagi Grimberg
2015-10-07 14:39 ` Sagi Grimberg
2015-10-07 14:48 ` Chuck Lever
2015-10-07 14:48 ` Chuck Lever
2015-10-07 15:17 ` Sagi Grimberg [this message]
2015-10-07 15:17 ` Sagi Grimberg
2015-10-06 14:59 ` [PATCH v2 07/16] xprtrdma: Remove reply tasklet Chuck Lever
2015-10-06 14:59 ` Chuck Lever
2015-10-06 14:59 ` [PATCH v2 08/16] xprtrdma: Saving IRQs no longer needed for rb_lock Chuck Lever
2015-10-06 14:59 ` Chuck Lever
2015-10-06 14:59 ` [PATCH v2 09/16] SUNRPC: Abstract backchannel operations Chuck Lever
2015-10-06 14:59 ` Chuck Lever
2015-10-06 14:59 ` [PATCH v2 10/16] xprtrdma: Pre-allocate backward rpc_rqst and send/receive buffers Chuck Lever
2015-10-06 14:59 ` Chuck Lever
2015-10-06 15:00 ` [PATCH v2 11/16] xprtrdma: Pre-allocate Work Requests for backchannel Chuck Lever
2015-10-06 15:00 ` Chuck Lever
2015-10-06 15:00 ` [PATCH v2 12/16] xprtrdma: Add support for sending backward direction RPC replies Chuck Lever
2015-10-06 15:00 ` Chuck Lever
2015-10-06 15:00 ` [PATCH v2 13/16] xprtrdma: Handle incoming backward direction RPC calls Chuck Lever
2015-10-06 15:00 ` Chuck Lever
2015-10-06 15:00 ` [PATCH v2 14/16] svcrdma: Add backward direction service for RPC/RDMA transport Chuck Lever
2015-10-06 15:00 ` Chuck Lever
2015-10-06 15:00 ` [PATCH v2 15/16] SUNRPC: Remove the TCP-only restriction in bc_svc_process() Chuck Lever
2015-10-06 15:00 ` Chuck Lever
2015-10-06 15:00 ` [PATCH v2 16/16] NFS: Enable client side NFSv4.1 backchannel to use other transports Chuck Lever
2015-10-06 15:00 ` Chuck Lever
2015-10-07 9:14 ` Leon Romanovsky
2015-10-07 9:14 ` Leon Romanovsky
2015-10-07 14:13 ` Chuck Lever
2015-10-07 14:13 ` Chuck Lever
2015-10-14 13:32 ` [PATCH v2 00/16] NFS/RDMA patches for merging into v4.4 Devesh Sharma
2015-10-14 13:32 ` Devesh Sharma
2015-10-14 15:34 ` Chuck Lever
2015-10-14 15:34 ` Chuck Lever
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=56153786.2000801@dev.mellanox.co.il \
--to=sagig@dev.mellanox.co.il \
--cc=chuck.lever@oracle.com \
--cc=linux-nfs@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.