From mboxrd@z Thu Jan 1 00:00:00 1970 From: jianchao.w.wang@oracle.com (jianchao.wang) Date: Tue, 16 Jan 2018 15:52:16 +0800 Subject: [Suspected-Phishing]Re: [PATCH V3 1/2] nvme: split resetting state into reset_prepate and resetting In-Reply-To: <2d198b6a-47f4-8d2b-024d-76161f4b0f90@oracle.com> References: <1515647268-1717-1-git-send-email-jianchao.w.wang@oracle.com> <1515647268-1717-2-git-send-email-jianchao.w.wang@oracle.com> <1c001532-234f-bc56-7fb4-bcd08142842e@mellanox.com> <2d198b6a-47f4-8d2b-024d-76161f4b0f90@oracle.com> Message-ID: <26bce6af-b1d4-ea20-775e-193a0a6f9dd4@oracle.com> On 01/16/2018 01:57 PM, jianchao.wang wrote: > Hi Max > > Thanks for your kindly comment. > > On 01/15/2018 09:36 PM, Max Gurtovoy wrote: >>>>> ????? case NVME_CTRL_RECONNECTING: >>>>> ????????? switch (old_state) { >>>>> ????????? case NVME_CTRL_LIVE: >>>>> ????????? case NVME_CTRL_RESETTING: >>>>> +??????? case NVME_CTRL_RESET_PREPARE: >> >> I forget to add that we shouldn't move from RESET_PREPARE to RECONNECTING (with my suggestion to rdma.c). >> Also need to consider adding another check in nvmf_check_init_req (/drivers/nvme/host/fabrics.h) for the new state. > > After Sagi's nvme-rdma: fix concurrent reset and reconnect, the rdma ctrl state is changed to RECONNECTING state > after some clearing and shutdown work, then some initializing procedure, no matter reset work path or error recovery path. > The fc reset work also does the same thing. > So if we define the range that RESET_PREPARE includes scheduling gap and disable and clear work, RESETTING includes initializing > procedure, RECONNECTING is very similar with RESETTING. > > Maybe we could do like this; > In nvme fc/rdma > - set state to RESET_PREPARE, queue reset_work/err_work > - clear/shutdown works, set state to RECONNECTING > - initialization, set state to LIVE > > In nvme pci > - set state to RESET_PREPARE, queue reset_work > - clear/shutdown works, set state to RESETTING > - initialization, set state to LIVE Hi Christoph, Keith, Sagi Can you please comment on this ? Thanks in advance. Jianchao From jianchao.w.wang at oracle.com Tue Jan 16 20:54:35 2018 From: jianchao.w.wang@oracle.com (Jianchao Wang) Date: Wed, 17 Jan 2018 12:54:35 +0800 Subject: [PATCH V4 0/2] nvme-pci: fix the timeout case when reset is ongoing Message-ID: <1516164877-2170-1-git-send-email-jianchao.w.wang@oracle.com> Hello NVME_CTRL_RESETTING used to indicate the range of nvme initializing strictly in fd634f41(nvme: merge probe_work and reset_work), but it is not now. The NVME_CTRL_RESETTING is set before queue the reset_work, there could be a big gap before the reset work handles the outstanding requests. So when the NVME_CTRL_RESETTING is set, nvme_timeout will not only meet the admin requests from the initializing procedure, but also the IO and admin requests from previous work before nvme_dev_disable is invoked. To fix this, based on Christoph's suggestion, splits the NVME_CTRL_RESETTING into NVME_CTRL_RESET_PREPARE and NVME_CTRL_RESETTING. At the same time, after Sagi introduced d5bf4b7 (nvme-rdma: fix concurrent reset and reconnect), both nvme-rdma/fc use NVME_CTRL_RECONNECTING to mark the setup and reconnect procedure. The RESETTING state has been narrowed. So we use this new state NVME_CTRL_RESET_PREPARE to mark the reset_work or error recovery work, scheduling gap and disable procedure. After that: - For nvme-pci, nvmet-loop, set state to RESETTING, start initialization. - For nvme-rdma, nvme-fc, set state to RECONNECTING, start initialization or reconnect. Finally, we could use NVME_CTRL_RESET_PREPARE to distinguish the different requests and handle them separately. More details, please refer to the comment of the 2nd patch. V4: - rebase patches on Jens' for-next - let RESETTING equal to RECONNECTING in terms of work procedure - change the 1st patch's name and comment - other misc changes V3: - fix wrong reference in loop.c - other misc changes V2: - split NVME_CTRL_RESETTING into NVME_CTRL_RESET_PREPARE and NVME_CTRL_RESETTING. Introduce new patch based on this. - distinguish the requests based on the new state in nvme_timeout - change comments of patch Jianchao Wang (2) 0001-nvme-add-NVME_CTRL_RESET_PREPARE-state.patch 0002-nvme-pci-fix-the-timeout-case-when-reset-is-ongoing.patch drivers/nvme/host/core.c | 18 +++++++++++++--- drivers/nvme/host/fc.c | 4 ++-- drivers/nvme/host/nvme.h | 8 +++++++ drivers/nvme/host/pci.c | 54 +++++++++++++++++++++++++++++++++++----------- drivers/nvme/host/rdma.c | 2 +- drivers/nvme/target/loop.c | 5 +++++ 6 files changed, 72 insertions(+), 19 deletions(-) Thanks Jianchao