Re: [PATCH 1/9] nvme-apple: fix controller shutdown in apple_nvme_disable

Hector Martin
2023-01-11T04:10:20Z

On 11/01/2023 02.47, Janne Grunau wrote:
> On 2022-11-30 00:41:45 +0900, Hector Martin wrote:
>> On 29/11/2022 22.22, Christoph Hellwig wrote:
>>> nvme_shutdown_ctrl already shuts the controller down, there is no
>>> need to also call nvme_disable_ctrl for the shutdown case.
>>>
>>> Signed-off-by: Christoph Hellwig <hch@lst.de>
>>> ---
>>> drivers/nvme/host/apple.c | 3 ++-
>>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/nvme/host/apple.c b/drivers/nvme/host/apple.c
>>> index 94ef797e8b4a5f..56d9e9be945b76 100644
>>> --- a/drivers/nvme/host/apple.c
>>> +++ b/drivers/nvme/host/apple.c
>>> @@ -831,7 +831,8 @@ static void apple_nvme_disable(struct apple_nvme *anv, bool shutdown)
>>>
>>> if (shutdown)
>>> nvme_shutdown_ctrl(&anv->ctrl);
>>> - nvme_disable_ctrl(&anv->ctrl);
>>> + else
>>> + nvme_disable_ctrl(&anv->ctrl);
>>> }
>>>
>>> WRITE_ONCE(anv->ioq.enabled, false);
>>
>> Reviewed-by: Hector Martin <marcan@marcan.st>
>>
>> I looked at some of our other implementations and we always seem to do
>> both, but this makes sense. If it breaks something we'll notice and
>> shout loudly when it makes it into an -rc at the latest :)
>
> This breaks resume for the apple nvme controller. The immediate problem
> is that apple_nvme_reset_work() uses NVME_CC_ENABLE to decide whether to
> call apple_nvme_disable(). Only nvme_disable_ctrl(ctrl, false) clears
> NVME_CC_ENABLE.
>
> Even after extending the check to consider NVME_CC_SHN_NORMAL resume is
> still broken. The host specific reset to re-enable the controller works
> but IO is blocked. The controller logs following messages into its
> syslog:
>
> | RTKit: syslog message: cmd.c:728: Oldest tag 12 is 19 seconds VERY OLD with cmdQe:2
> | RTKit: syslog message: cmd.c:732: Oldest tag is about to crash!!
> | RTKit: syslog message: cmd.c:720: Oldest high water mark moved up to 21,tag 12, cmdQ CQ_PRI0W
>
> This looks looks like the controller reset after/during shutdown is required
> on this controller.

It's actually required by the NVMe spec. See the SHST description in
Figure 47 (NVMe Base Spec 2.0c). I'll send a patch.

- Hector