Re: Question: NVMe AER completion delivery during controller removal

[Mohamed Khalfella <mkhalfella@purestorage.com>]

On Wed 2026-11-04 12:50:42 +0800, zhang san wrote:
> Hello,
> 
> We are seeking clarification about asynchronous event request (AER)
> completion behavior during NVMe controller removal, before treating an
> observed test failure as a production-reachable driver bug.
> 
> Our source analysis suggests a possible lifetime race between the AEN
> completion path, which can requeue nvme_async_event_work, and the
> controller removal path. We observed KASAN use-after-free in an
> emulated environment on Linux v7.4-rc1 (6628e7d4c6b2).


My reading of the code suggests this can happen. nvme_stop_ctrl()
flushes async_event_work, but the admin queue can still deliver an AER
completion afterwards, either from the admin queue interrupt handler or
from the final admin CQ poll in nvme_dev_disable():

nvme_remove() ->
  nvme_dev_disable() ->
    nvme_poll_irqdisable(&dev->queues[0]) ->
      nvme_poll_cq() ->
        nvme_handle_cqe() ->
          nvme_complete_async_event() ->
	    queue_work(nvme_wq, &ctrl->async_event_work);


Nothing flushes async_event_work after that, so the work can run after
nvme_uninit_ctrl() drops the last reference and the nvme_dev is freed.

> 
> However, our test induced an ERROR AEN through an out-of-specification
> MMIO doorbell write while the device was being removed. This
> deliberately synthetic trigger does not establish that a compliant
> controller would generate the same completion.

--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3909,20 +3909,21 @@ static void nvme_remove(struct pci_dev *pdev)

        if (!pci_device_is_present(pdev)) {
                nvme_change_ctrl_state(&dev->ctrl, NVME_CTRL_DEAD);
                nvme_dev_disable(dev, true);
        }

        flush_work(&dev->ctrl.reset_work);
        nvme_stop_ctrl(&dev->ctrl);
        nvme_remove_namespaces(&dev->ctrl);
        nvme_dev_disable(dev, true);
+       flush_work(&dev->ctrl.async_event_work);
        nvme_free_host_mem(dev);
        nvme_dev_remove_admin(dev);
        nvme_dbbuf_dma_free(dev);
        nvme_free_queues(dev, 0);
        mempool_destroy(dev->dmavec_mempool);
        nvme_release_descriptor_pools(dev);
        nvme_dev_unmap(dev);
        nvme_uninit_ctrl(&dev->ctrl);
 }

Since you are able to reproduce this issue reliabely, how about you try
the diff above and see if it fixes the issue.

> 
> Our main question is: can a specification-compliant controller deliver
> a completion for a previously submitted AER during the interval in
> which the host is disabling the controller and tearing down the admin
> queue, or must such completions be quiesced before disable completes?
> We would also welcome guidance on the exact point after which the host
> may safely assume that no further AER completions can be delivered.

If the above diff fixes the issue for you, then I think this is a driver
bug.