From: Boaz Harrosh <boaz@plexistor.com>
To: Trond Myklebust <trond.myklebust@primarydata.com>,
"Matt W. Benjamin" <matt@linuxbox.com>
Cc: Christoph Hellwig <hch@lst.de>,
Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
"Adam C. Emerson" <aemerson@linuxbox.com>
Subject: Re: [PATCH] pnfs: Kick a pnfs_layoutcommit_inode on recall
Date: Tue, 26 Aug 2014 19:56:58 +0300 [thread overview]
Message-ID: <53FCBC5A.4060304@plexistor.com> (raw)
In-Reply-To: <CAHQdGtT-B7NTP8T7ETgTJD1+84k_KW_5aOfrYjLpA04hoR3YsA@mail.gmail.com>
On 08/26/2014 06:36 PM, Trond Myklebust wrote:
> On Tue, Aug 26, 2014 at 11:24 AM, Matt W. Benjamin <matt@linuxbox.com> wrote:
>> IIUC, the problem is the forechannel slot count, since the call you want to make synchronously is on the forechannel?
Matt no top post on a Linux mailing list ;-)
> Yep. layoutcommit will be sent on the fore channel, which is why it
> can deadlock with the initial layoutget (or whatever operation that
> triggered the layout recall).
Trond you said below:
> The above can deadlock if there are no session slots available to send
> the layoutcommit, in which case the recall won't complete, and the
> layoutget won't get a reply (which would free up the slot).
Why would the layoutget not-get-a-reply ?
This is how it goes with Both ganesha server and knfsd last I tested.
[1]
The LAYOUT_GET cause LAYOUT_RECALL case: (including the lo_commit)
client Server comments
~~~~~~ ~~~~~~ ~~~~~~~~
LAYOUT_GET ==>
<== LAYOUT_GET_REPLAY(ERR_RECALL_CONFLICT)
<--------- fore-channel is free
<== RECALL
LAYOUT_COMMIT ==>
<== LAYOUT_COMMIT_REPLAY
<--------- fore-channel is free
RECALL_REPLY(NO_MATCHING) =>
<--------- back-channel is free
Note that in this case the server is to send the RECALL only after
the error reply to LAYOUT_GET, specifically it is not aloud to get stuck
inside LAYOUT_GET and wait for the RECALL. (mandated by STD)
[2]
The LAYOUT_GET sent all the while a RECALL is on the wire:
client Server comments
~~~~~~ ~~~~~~ ~~~~~~~~
<== RECALL
LAYOUT_GET ==>
<== LAYOUT_GET_REPLAY(ERR_RECALL_CONFLICT)
<--------- fore-channel is free
LAYOUT_COMMIT ==>
LAYOUT_COMMIT_REPLAY
<--------- fore-channel is free
RECALL_REPLY(NO_MATCHING) =>
<--------- back-channel is free
[3]
Or the worst case that lo_commit needs to wait for the channel Similar
to [2] above:
client Server comments
~~~~~~ ~~~~~~ ~~~~~~~~
<== RECALL
LAYOUT_GET ==>
initiate_lo_commit ==> slot is taken needs to wait
<== LAYOUT_GET_REPLAY(ERR_RECALL_CONFLICT)
<--------- fore-channel is free
LAYOUT_COMMIT ==> slot is now free lo_commit goes through
<== LAYOUT_COMMIT_REPLAY
<--------- fore-channel is free
RECALL_REPLY(NO_MATCHING) =>
<--------- back-channel is free
So the most important is that the server must not get stuck in lo_get
and since there is a slot for each channel the lo_commit can be sent
from within the recall.
What am I missing?
Thanks
Boaz
next prev parent reply other threads:[~2014-08-26 16:57 UTC|newest]
Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <pnfs block layout driver fixes V2>
2014-08-21 16:09 ` Christoph Hellwig
2014-08-21 16:09 ` [PATCH 01/19] nfs: cap request size to fit a kmalloced page array Christoph Hellwig
2014-08-21 16:09 ` [PATCH 02/19] pnfs: do not pass uninitialized lsegs to ->free_lseg Christoph Hellwig
2014-08-21 16:09 ` [PATCH 03/19] pnfs: force a layout commit when encountering busy segments during recall Christoph Hellwig
2014-08-24 17:49 ` Boaz Harrosh
2014-08-24 19:18 ` Christoph Hellwig
2014-08-26 14:10 ` [PATCH] pnfs: Kick a pnfs_layoutcommit_inode on recall Boaz Harrosh
2014-08-26 14:26 ` Trond Myklebust
2014-08-26 14:37 ` Boaz Harrosh
2014-08-26 14:52 ` Boaz Harrosh
2014-08-26 14:55 ` Trond Myklebust
2014-08-26 15:02 ` Boaz Harrosh
2014-08-26 15:24 ` Matt W. Benjamin
2014-08-26 15:36 ` Trond Myklebust
2014-08-26 16:56 ` Boaz Harrosh [this message]
2014-08-26 16:59 ` Trond Myklebust
2014-08-26 17:06 ` Boaz Harrosh
2014-08-26 17:54 ` Trond Myklebust
2014-08-26 18:19 ` Boaz Harrosh
2014-08-26 18:34 ` Boaz Harrosh
2014-08-26 18:41 ` Trond Myklebust
2014-08-26 19:46 ` Trond Myklebust
2014-08-27 8:50 ` Boaz Harrosh
2014-08-27 8:22 ` Boaz Harrosh
2014-09-09 0:37 ` [PATCH 03/19] pnfs: force a layout commit when encountering busy segments during recall Trond Myklebust
2014-09-09 5:49 ` Christoph Hellwig
2014-09-09 14:38 ` Trond Myklebust
2014-08-21 16:09 ` [PATCH 04/19] pnfs: don't check sequence on new stateids in layoutget Christoph Hellwig
2014-08-21 16:09 ` [PATCH 05/19] pnfs: retry after a bad stateid error from layoutget Christoph Hellwig
2014-08-21 16:09 ` [PATCH 06/19] pnfs: avoid using stale stateids after layoutreturn Christoph Hellwig
2014-08-21 16:09 ` [PATCH 07/19] pnfs: add flag to force read-modify-write in ->write_begin Christoph Hellwig
2014-09-09 3:50 ` Trond Myklebust
2014-09-09 5:53 ` Christoph Hellwig
2014-09-09 14:41 ` Trond Myklebust
2014-08-21 16:09 ` [PATCH 08/19] pnfs: add return_range method Christoph Hellwig
2014-08-25 13:50 ` Anna Schumaker
2014-08-25 14:09 ` Christoph Hellwig
2014-08-25 14:17 ` Anna Schumaker
2014-08-25 14:20 ` Christoph Hellwig
2014-09-09 3:57 ` Trond Myklebust
2014-08-21 16:09 ` [PATCH 09/19] pnfs: allow splicing pre-encoded pages into the layoutcommit args Christoph Hellwig
2014-08-21 16:09 ` [PATCH 10/19] pnfs/blocklayout: reject pnfs blocksize larger than page size Christoph Hellwig
2014-08-21 16:09 ` [PATCH 11/19] pnfs/blocklayout: improve GETDEVICEINFO error reporting Christoph Hellwig
2014-08-21 16:09 ` [PATCH 12/19] pnfs/blocklayout: plug block queues Christoph Hellwig
2014-08-21 16:09 ` [PATCH 13/19] pnfs/blocklayout: correctly decrement extent length Christoph Hellwig
2015-02-09 6:01 ` NeilBrown
2015-02-09 18:24 ` Christoph Hellwig
2014-08-21 16:09 ` [PATCH 14/19] pnfs/blocklayout: remove read-modify-write handling in bl_write_pagelist Christoph Hellwig
2014-09-09 4:43 ` Trond Myklebust
2014-08-21 16:09 ` [PATCH 15/19] pnfs/blocklayout: don't set pages uptodate Christoph Hellwig
2014-09-09 4:48 ` Trond Myklebust
2014-08-21 16:09 ` [PATCH 16/19] pnfs/blocklayout: rewrite extent tracking Christoph Hellwig
2014-08-25 14:36 ` Anna Schumaker
2014-08-25 14:43 ` Christoph Hellwig
2014-08-26 9:06 ` Boaz Harrosh
2014-09-09 4:50 ` Trond Myklebust
2014-08-21 16:09 ` [PATCH 17/19] pnfs/blocklayout: implement the return_range method Christoph Hellwig
2014-09-09 4:03 ` Trond Myklebust
2014-08-21 16:09 ` [PATCH 18/19] pnfs/blocklayout: return layouts on setattr Christoph Hellwig
2014-09-09 4:09 ` Trond Myklebust
2014-08-21 16:09 ` [PATCH 19/19] pnfs/blocklayout: allocate separate pages for the layoutcommit payload Christoph Hellwig
2014-09-09 4:52 ` Trond Myklebust
2014-08-21 16:13 ` pnfs block layout driver fixes V2 Christoph Hellwig
2014-09-09 4:12 ` Trond Myklebust
2014-09-09 5:54 ` Christoph Hellwig
2014-09-09 14:40 ` Trond Myklebust
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=53FCBC5A.4060304@plexistor.com \
--to=boaz@plexistor.com \
--cc=aemerson@linuxbox.com \
--cc=hch@lst.de \
--cc=linux-nfs@vger.kernel.org \
--cc=matt@linuxbox.com \
--cc=trond.myklebust@primarydata.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).