Re: [PATCH v2 1/2] nvme-apple: Reset controller during shutdown

Re: [PATCH v2 1/2] nvme-apple: Reset controller during shutdown

[Christoph Hellwig]

On Tue, Jan 17, 2023 at 07:25:00PM +0100, Janne Grunau wrote:
> +		/*
> +		 * Always reset the NVMe controller on shutdown. The reset is
> +		 * required to shutdown the co-processor cleanly.
> +		 */

Hmm.  This comment doesn't seem to match the discussion we had last
week.  Which would be:

		/*
		 * NVMe requires a reset before setting up a controller to
		 * ensure it is in a clean state.  For NVMe PCIe this is
		 * done in the setup path to be able to deal with controllers
		 * in any kind of state.  For for Apple devices, the firmware
		 * will not be available at that time and the reset will
		 * time out.  Thus reset after shutting the NVMe controller
		 * down and before shutting the firmware down.
		 */

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/2] nvme-apple: Fix suspend-resume regression

[Christoph Hellwig]

I'll queue this up in nvme-6.2 as an urgent fix.  But I'd love to hear
clarification on what hsould be in the comment based on the discussion
last week.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] nvme-apple: Reset controller during shutdown

[Christoph Hellwig]

Folks, can you chime in if this comment makes sense?  I'd really
like to send the patches off to Jens before rc5.

On Wed, Jan 18, 2023 at 06:24:50AM +0100, Christoph Hellwig wrote:
> On Tue, Jan 17, 2023 at 07:25:00PM +0100, Janne Grunau wrote:
> > +		/*
> > +		 * Always reset the NVMe controller on shutdown. The reset is
> > +		 * required to shutdown the co-processor cleanly.
> > +		 */
> 
> Hmm.  This comment doesn't seem to match the discussion we had last
> week.  Which would be:
> 
> 		/*
> 		 * NVMe requires a reset before setting up a controller to
> 		 * ensure it is in a clean state.  For NVMe PCIe this is
> 		 * done in the setup path to be able to deal with controllers
> 		 * in any kind of state.  For for Apple devices, the firmware
> 		 * will not be available at that time and the reset will
> 		 * time out.  Thus reset after shutting the NVMe controller
> 		 * down and before shutting the firmware down.
> 		 */
---end quoted text---

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] nvme-apple: Reset controller during shutdown

[Janne Grunau]

Hej,

On 2023-01-18 06:24:50 +0100, Christoph Hellwig wrote:
> On Tue, Jan 17, 2023 at 07:25:00PM +0100, Janne Grunau wrote:
> > +		/*
> > +		 * Always reset the NVMe controller on shutdown. The reset is
> > +		 * required to shutdown the co-processor cleanly.
> > +		 */
> 
> Hmm.  This comment doesn't seem to match the discussion we had last
> week.  Which would be:
> 
> 		/*
> 		 * NVMe requires a reset before setting up a controller to
> 		 * ensure it is in a clean state.  For NVMe PCIe this is
> 		 * done in the setup path to be able to deal with controllers
> 		 * in any kind of state.  For for Apple devices, the firmware
> 		 * will not be available at that time and the reset will
> 		 * time out.  Thus reset after shutting the NVMe controller
> 		 * down and before shutting the firmware down.
> 		 */

yes, it differs from the discussion last week. I tried to issue the 
reset later in the setup path after the firmware was brought back up.  
That fixes the hang but the device is still not useable. So it appears 
we need to reset the controller before the firmware is shutdown.

Janne

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] nvme-apple: Reset controller during shutdown

[Hector Martin "marcan"]

(Replying from mobile, please excuse formatting)

I'm actually not sure exactly how this works any more. The previous series I sent (which had slightly different logic) worked for me on a t8103 Mac Mini in smoke tests and I'd assumed fixed the issue, but it turned out to fail (in a different way) on other machines/circumstances. This one seems to work everywhere, but I can't explain exactly why. Maybe we do in fact need to issue an NVMe disable before shutting down the firmware to reliably come up properly on firmware restart.

Maybe something like this?

/*
 * Always disable the NVMe controller after shutdown.
 * We need to do this to bring it back up later anyway,
 * and we can't do it while the firmware is not running
 * (e.g. in the resume reset path before RTKit is
 * initialized), so for Apple controllers it makes sense to
 * unconditionally do it here. Additionally, this sequence
 * of events is reliable, while others (like disabling after
 * bringing back the firmware on resume) seem to run
 * into trouble under some circumstances.
 *
 * Both U-Boot and m1n1 also use this convention
 * (i.e. an ANS NVMe controller is handed off with
 * firmware shut down, in an NVMe disabled state,
 * after a clean shutdown).
 */

On 2023年1月19日 15:14:52 JST, Christoph Hellwig <hch@lst.de> wrote:
>Folks, can you chime in if this comment makes sense?  I'd really
>like to send the patches off to Jens before rc5.
>
>On Wed, Jan 18, 2023 at 06:24:50AM +0100, Christoph Hellwig wrote:
>> On Tue, Jan 17, 2023 at 07:25:00PM +0100, Janne Grunau wrote:
>> > +		/*
>> > +		 * Always reset the NVMe controller on shutdown. The reset is
>> > +		 * required to shutdown the co-processor cleanly.
>> > +		 */
>> 
>> Hmm.  This comment doesn't seem to match the discussion we had last
>> week.  Which would be:
>> 
>> 		/*
>> 		 * NVMe requires a reset before setting up a controller to
>> 		 * ensure it is in a clean state.  For NVMe PCIe this is
>> 		 * done in the setup path to be able to deal with controllers
>> 		 * in any kind of state.  For for Apple devices, the firmware
>> 		 * will not be available at that time and the reset will
>> 		 * time out.  Thus reset after shutting the NVMe controller
>> 		 * down and before shutting the firmware down.
>> 		 */
>---end quoted text---
>

-- 
Hector Martin "marcan" (marcan@marcan.st)
Public key: https://mrcn.st/pub

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] nvme-apple: Reset controller during shutdown

[Christoph Hellwig]

Thanks, this looks good.  Updated commit here:

http://git.infradead.org/nvme.git/commitdiff/c06ba7b892a50b48522ad441a40053f483dfee9e

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] nvme-apple: Reset controller during shutdown

[Janne Grunau]

On 2023-01-19 09:08:39 +0100, Christoph Hellwig wrote:
> Thanks, this looks good.  Updated commit here:
> 
> http://git.infradead.org/nvme.git/commitdiff/c06ba7b892a50b48522ad441a40053f483dfee9e

looks good to me as well.

thanks

Janne

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel