Re: [RFC PATCH 0/1] DOE usage with pcie/portdrv

[Lukas Wunner <lukas@wunner.de>]

On Wed, May 11, 2022 at 12:43:34PM -0700, Dan Williams wrote:
> On Wed, May 11, 2022 at 12:20 PM Lukas Wunner <lukas@wunner.de> wrote:
> > But the reset argument still stands:  That same section says that all
> > IDE streams transition to Insecure and all keys are invalidated upon
> > reset.
> 
> Right, this isn't the only problem with reset vs ongoing CXL operations...
> 
> https://lore.kernel.org/linux-cxl/164740402242.3912056.8303625392871313860.stgit@dwillia2-desk3.amr.corp.intel.com/

The above-linked cover letter refers to AER.

I believe with AER, the kernel is notified of an error via an interrupt
and asynchronously attempts recovery through a reset.
Obviously, an eternity may pass until the kernel gets around to do that
and whether accesses performed between the initial error and the reset
succeed is sort of undefined.  So it's kind of a "best effort" error
recovery.

With the advent of DPC, the situation has improved considerably as the
hardware (not the kernel) automatically disables the link upon occurrence
of the initial error.  Any subsequent accesses will fail and the kernel
does not perform a reset itself (the hardware already did that) but merely
attempts to bring the link back up.  That has made error recovery pretty
solid and NVMe drives now seamlessly recover from errors without the need
to unbind/rebind the driver.  Data centers heavily depend on that feature.

Perhaps if CXL.mem used DPC, it would be able to recover more reliably?

Circling back to the SPDM/IDE topic, while NVMe is now capable of
reliably recovering from errors, it does expect the kernel to handle
recovery within a few seconds.  I'm not sure we can continue to
guarantee that if the kernel depends on user space to perform
re-authentication with SPDM after reset.  That's another headache
that we could avoid with in-kernel SPDM authentication.

Thanks,

Lukas