[PATCH] nvme: do not reset controllers in NVME_CTRL_NEW state

[PATCH] nvme: do not reset controllers in NVME_CTRL_NEW state

[Maurizio Lombardi]

During NVMe controller creation, the controller is exposed to sysfs via
nvme_add_ctrl() while it is still in the NVME_CTRL_NEW state.
This creates a narrow race window where a userspace process can write to
the reset_controller sysfs node before the initialization thread
transitions the state to NVME_CTRL_CONNECTING.

If a reset is triggered during this window, the state machine allows the
transition from NVME_CTRL_NEW to NVME_CTRL_RESETTING, and the reset work
is queued. However, the original creation thread continues its execution,
subsequently moving the state to NVME_CTRL_CONNECTING and finally to
NVME_CTRL_LIVE.

When the delayed reset work finally executes, it attempts to tear down
the controller and transition the state to NVME_CTRL_CONNECTING. Because
the state is now NVME_CTRL_LIVE, this transition fails, triggering a
WARN_ON in the reset work.

Fix this by removing NVME_CTRL_NEW from the allowed prior states for
the NVME_CTRL_RESETTING transition.

Fixes: 8bfc3b4c6f9d ("nvmet: switch loopback target state to connecting when resetting")
Reported-by: syzbot+4a1d521d19d6321f5aad@syzkaller.appspotmail.com
Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>
---
 drivers/nvme/host/core.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index beea23d04a70..7ea8dd27d08d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -598,7 +598,6 @@ bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl,
 		break;
 	case NVME_CTRL_RESETTING:
 		switch (old_state) {
-		case NVME_CTRL_NEW:
 		case NVME_CTRL_LIVE:
 			changed = true;
 			atomic_long_inc(&ctrl->nr_reset);
-- 
2.55.0

Re: [PATCH] nvme: do not reset controllers in NVME_CTRL_NEW state

[Daniel Wagner]

On Tue, Sep 15, 2026 at 11:58:27AM +0200, Maurizio Lombardi wrote:
> During NVMe controller creation, the controller is exposed to sysfs via
> nvme_add_ctrl() while it is still in the NVME_CTRL_NEW state.
> This creates a narrow race window where a userspace process can write to
> the reset_controller sysfs node before the initialization thread
> transitions the state to NVME_CTRL_CONNECTING.
> 
> If a reset is triggered during this window, the state machine allows the
> transition from NVME_CTRL_NEW to NVME_CTRL_RESETTING, and the reset work
> is queued. However, the original creation thread continues its execution,
> subsequently moving the state to NVME_CTRL_CONNECTING and finally to
> NVME_CTRL_LIVE.
> 
> When the delayed reset work finally executes, it attempts to tear down
> the controller and transition the state to NVME_CTRL_CONNECTING. Because
> the state is now NVME_CTRL_LIVE, this transition fails, triggering a
> WARN_ON in the reset work.
> 
> Fix this by removing NVME_CTRL_NEW from the allowed prior states for
> the NVME_CTRL_RESETTING transition.
> 
> Fixes: 8bfc3b4c6f9d ("nvmet: switch loopback target state to connecting when resetting")
> Reported-by: syzbot+4a1d521d19d6321f5aad@syzkaller.appspotmail.com
> Signed-off-by: Maurizio Lombardi <mlombard@redhat.com or why it was
> there it there in the first place

Looks reasonable. It might be worth explaining why we don't need to
handle the NEW -> RESETTING state transition or why it was there in the
first place.

Reviewed-by: Daniel Wagner <dwagner@suse.de>

Re: [PATCH] nvme: do not reset controllers in NVME_CTRL_NEW state

[Maurizio Lombardi]

On Tue Sep 15, 2026 at 3:11 PM CEST, Daniel Wagner wrote:
> On Tue, Sep 15, 2026 at 11:58:27AM +0200, Maurizio Lombardi wrote:
>> During NVMe controller creation, the controller is exposed to sysfs via
>> nvme_add_ctrl() while it is still in the NVME_CTRL_NEW state.
>> This creates a narrow race window where a userspace process can write to
>> the reset_controller sysfs node before the initialization thread
>> transitions the state to NVME_CTRL_CONNECTING.
>> 
>> If a reset is triggered during this window, the state machine allows the
>> transition from NVME_CTRL_NEW to NVME_CTRL_RESETTING, and the reset work
>> is queued. However, the original creation thread continues its execution,
>> subsequently moving the state to NVME_CTRL_CONNECTING and finally to
>> NVME_CTRL_LIVE.
>> 
>> When the delayed reset work finally executes, it attempts to tear down
>> the controller and transition the state to NVME_CTRL_CONNECTING. Because
>> the state is now NVME_CTRL_LIVE, this transition fails, triggering a
>> WARN_ON in the reset work.
>> 
>> Fix this by removing NVME_CTRL_NEW from the allowed prior states for
>> the NVME_CTRL_RESETTING transition.
>> 
>> Fixes: 8bfc3b4c6f9d ("nvmet: switch loopback target state to connecting when resetting")
>> Reported-by: syzbot+4a1d521d19d6321f5aad@syzkaller.appspotmail.com
>> Signed-off-by: Maurizio Lombardi <mlombard@redhat.com or why it was
>> there it there in the first place
>
> Looks reasonable. It might be worth explaining why we don't need to
> handle the NEW -> RESETTING state transition or why it was there in the
> first place.

Maybe the last paragraph could be expanded in the following way:

"
 Fix this by removing NVME_CTRL_NEW from the allowed prior states for
 the NVME_CTRL_RESETTING transition. It is safe to drop this transition
 entirely because a controller in the NVME_CTRL_NEW state is merely
 allocated and not yet initialized, a reset operation at this stage is
 meaningless.
"

For what concernes the "why it was there in the first place" question,
I think that the reason is that in ancient times a CONNECTING state
didn't exist. The controller initialization was done via a reset;
so the transition was NEW -> RESETTING -> LIVE.

Maurizio

Re: [PATCH] nvme: do not reset controllers in NVME_CTRL_NEW state

[Daniel Wagner]

On Tue, Sep 15, 2026 at 03:49:36PM +0200, Maurizio Lombardi wrote:
> "
>  Fix this by removing NVME_CTRL_NEW from the allowed prior states for
>  the NVME_CTRL_RESETTING transition. It is safe to drop this transition
>  entirely because a controller in the NVME_CTRL_NEW state is merely
>  allocated and not yet initialized, a reset operation at this stage is
>  meaningless.
> "
> 
> For what concernes the "why it was there in the first place" question,
> I think that the reason is that in ancient times a CONNECTING state
> didn't exist. The controller initialization was done via a reset;
> so the transition was NEW -> RESETTING -> LIVE.

This rings a bell. With the introduction of the fabrics drivers, the
CONNECTING state was introduced and the PCI driver didn't use the
CONNECTING state for while. But due to various reasons, the PCI
transport follows the same state transitions as the fabrics ones now.

I am just a bit careful with the state machine, even obvious correct
changes can introduce subtle regressions.

Re: [PATCH] nvme: do not reset controllers in NVME_CTRL_NEW state

[Keith Busch]

On Tue, Sep 15, 2026 at 04:49:31PM +0200, Daniel Wagner wrote:
> On Tue, Sep 15, 2026 at 03:49:36PM +0200, Maurizio Lombardi wrote:
> > "
> >  Fix this by removing NVME_CTRL_NEW from the allowed prior states for
> >  the NVME_CTRL_RESETTING transition. It is safe to drop this transition
> >  entirely because a controller in the NVME_CTRL_NEW state is merely
> >  allocated and not yet initialized, a reset operation at this stage is
> >  meaningless.
> > "
> > 
> > For what concernes the "why it was there in the first place" question,
> > I think that the reason is that in ancient times a CONNECTING state
> > didn't exist. The controller initialization was done via a reset;
> > so the transition was NEW -> RESETTING -> LIVE.
> 
> This rings a bell. With the introduction of the fabrics drivers, the
> CONNECTING state was introduced and the PCI driver didn't use the
> CONNECTING state for while. But due to various reasons, the PCI
> transport follows the same state transitions as the fabrics ones now.

Yep, nvme pci used to always kick most of the initialization to the
reset_work, so that transport wanted the NEW->RESETTING transition.
With asynchronous probe support added, we do the initial setup without
reset_work.

Re: [PATCH] nvme: do not reset controllers in NVME_CTRL_NEW state

[Keith Busch]

On Tue, Sep 15, 2026 at 11:58:27AM +0200, Maurizio Lombardi wrote:
> During NVMe controller creation, the controller is exposed to sysfs via
> nvme_add_ctrl() while it is still in the NVME_CTRL_NEW state.
> This creates a narrow race window where a userspace process can write to
> the reset_controller sysfs node before the initialization thread
> transitions the state to NVME_CTRL_CONNECTING.
> 
> If a reset is triggered during this window, the state machine allows the
> transition from NVME_CTRL_NEW to NVME_CTRL_RESETTING, and the reset work
> is queued. However, the original creation thread continues its execution,
> subsequently moving the state to NVME_CTRL_CONNECTING and finally to
> NVME_CTRL_LIVE.
> 
> When the delayed reset work finally executes, it attempts to tear down
> the controller and transition the state to NVME_CTRL_CONNECTING. Because
> the state is now NVME_CTRL_LIVE, this transition fails, triggering a
> WARN_ON in the reset work.
> 
> Fix this by removing NVME_CTRL_NEW from the allowed prior states for
> the NVME_CTRL_RESETTING transition.

Thanks, applied to nvme-7.3.

Re: [PATCH] nvme: do not reset controllers in NVME_CTRL_NEW state

[Yureka Lilian]

On 9/15/26 11:58, Maurizio Lombardi wrote:
> During NVMe controller creation, the controller is exposed to sysfs via
> nvme_add_ctrl() while it is still in the NVME_CTRL_NEW state.
> This creates a narrow race window where a userspace process can write to
> the reset_controller sysfs node before the initialization thread
> transitions the state to NVME_CTRL_CONNECTING.
>
> If a reset is triggered during this window, the state machine allows the
> transition from NVME_CTRL_NEW to NVME_CTRL_RESETTING, and the reset work
> is queued. However, the original creation thread continues its execution,
> subsequently moving the state to NVME_CTRL_CONNECTING and finally to
> NVME_CTRL_LIVE.
>
> When the delayed reset work finally executes, it attempts to tear down
> the controller and transition the state to NVME_CTRL_CONNECTING. Because
> the state is now NVME_CTRL_LIVE, this transition fails, triggering a
> WARN_ON in the reset work.
>
> Fix this by removing NVME_CTRL_NEW from the allowed prior states for
> the NVME_CTRL_RESETTING transition.
>
> Fixes: 8bfc3b4c6f9d ("nvmet: switch loopback target state to connecting when resetting")
> Reported-by: syzbot+4a1d521d19d6321f5aad@syzkaller.appspotmail.com
> Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>
> ---
>   drivers/nvme/host/core.c | 1 -
>   1 file changed, 1 deletion(-)
>
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index beea23d04a70..7ea8dd27d08d 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -598,7 +598,6 @@ bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl,
>   		break;
>   	case NVME_CTRL_RESETTING:
>   		switch (old_state) {
> -		case NVME_CTRL_NEW:
>   		case NVME_CTRL_LIVE:
>   			changed = true;
>   			atomic_long_inc(&ctrl->nr_reset);

Hi!

I was testing something with linux-next 2026-10-02 and ran into an issue 
with the
nvme-apple driver where /dev/nvme0 exists but none of the namespaces appear,
and the dmesg is awfully quiet about the nvme device. I bisected it down to
this patch, which as far as I can tell leads to the nvme-apple being 
stuck in
NVME_CTRL_NEW state. Without being familiar with the matter, this looks like
the nvme-apple driver was using this transition as part of its 
initialization
procedure.  If one of the NVMe people who know how these transitions are
supposed to work could have a look, that would be much appreciated, as
otherwise this will break nvme-apple in 7.3-rc6.

Thanks,

- Yureka

Re: [PATCH] nvme: do not reset controllers in NVME_CTRL_NEW state

[Maurizio Lombardi]

On Sat Oct 3, 2026 at 1:55 PM CEST, Yureka Lilian wrote:
> On 9/15/26 11:58, Maurizio Lombardi wrote:
>> During NVMe controller creation, the controller is exposed to sysfs via
>> nvme_add_ctrl() while it is still in the NVME_CTRL_NEW state.
>> This creates a narrow race window where a userspace process can write to
>> the reset_controller sysfs node before the initialization thread
>> transitions the state to NVME_CTRL_CONNECTING.
>>
>> If a reset is triggered during this window, the state machine allows the
>> transition from NVME_CTRL_NEW to NVME_CTRL_RESETTING, and the reset work
>> is queued. However, the original creation thread continues its execution,
>> subsequently moving the state to NVME_CTRL_CONNECTING and finally to
>> NVME_CTRL_LIVE.
>>
>> When the delayed reset work finally executes, it attempts to tear down
>> the controller and transition the state to NVME_CTRL_CONNECTING. Because
>> the state is now NVME_CTRL_LIVE, this transition fails, triggering a
>> WARN_ON in the reset work.
>>
>> Fix this by removing NVME_CTRL_NEW from the allowed prior states for
>> the NVME_CTRL_RESETTING transition.
>>
>> Fixes: 8bfc3b4c6f9d ("nvmet: switch loopback target state to connecting when resetting")
>> Reported-by: syzbot+4a1d521d19d6321f5aad@syzkaller.appspotmail.com
>> Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>
>> ---
>>   drivers/nvme/host/core.c | 1 -
>>   1 file changed, 1 deletion(-)
>>
>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>> index beea23d04a70..7ea8dd27d08d 100644
>> --- a/drivers/nvme/host/core.c
>> +++ b/drivers/nvme/host/core.c
>> @@ -598,7 +598,6 @@ bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl,
>>   		break;
>>   	case NVME_CTRL_RESETTING:
>>   		switch (old_state) {
>> -		case NVME_CTRL_NEW:
>>   		case NVME_CTRL_LIVE:
>>   			changed = true;
>>   			atomic_long_inc(&ctrl->nr_reset);
>
> Hi!
>
> I was testing something with linux-next 2026-10-02 and ran into an issue 
> with the
> nvme-apple driver where /dev/nvme0 exists but none of the namespaces appear,
> and the dmesg is awfully quiet about the nvme device. I bisected it down to
> this patch, which as far as I can tell leads to the nvme-apple being 
> stuck in
> NVME_CTRL_NEW state. Without being familiar with the matter, this looks like
> the nvme-apple driver was using this transition as part of its 
> initialization
> procedure.  If one of the NVMe people who know how these transitions are
> supposed to work could have a look, that would be much appreciated, as
> otherwise this will break nvme-apple in 7.3-rc6.
>
> Thanks,

Thanks for reporting this,
it seems that nvme-apple is the only driver that still initializes the
controller by performing a reset directly from a NEW state.

This should be easy to fix by moving the state to CONNECTING in
apple_nvme_probe, just before nvme_reset_ctrl() is called.

Would you be able to test the patch if I prepare it?

Maurizio

Re: [PATCH] nvme: do not reset controllers in NVME_CTRL_NEW state

[Maurizio Lombardi]

On Sat Oct 3, 2026 at 2:33 PM CEST, Maurizio Lombardi wrote:
> Thanks for reporting this,
> it seems that nvme-apple is the only driver that still initializes the
> controller by performing a reset directly from a NEW state.
>
> This should be easy to fix by moving the state to CONNECTING in
> apple_nvme_probe, just before nvme_reset_ctrl() is called.

Ops, this won't work, as we can enter the resetting state only when the
controller is live.

Keith, I think we have no other option but revert this patch to unbreak
apple-nvme until I find a proper fix.

Maurizio

Re: [PATCH] nvme: do not reset controllers in NVME_CTRL_NEW state

[Yureka Lilian]

On 10/3/26 15:09, Maurizio Lombardi wrote:
> On Sat Oct 3, 2026 at 2:33 PM CEST, Maurizio Lombardi wrote:
>> Thanks for reporting this,
>> it seems that nvme-apple is the only driver that still initializes the
>> controller by performing a reset directly from a NEW state.
>>
>> This should be easy to fix by moving the state to CONNECTING in
>> apple_nvme_probe, just before nvme_reset_ctrl() is called.
> Ops, this won't work, as we can enter the resetting state only when the
> controller is live.
>
> Keith, I think we have no other option but revert this patch to unbreak
> apple-nvme until I find a proper fix.
>
> Maurizio


When you eventually propose a new variant of this, I can certainly test 
it with nvme-apple.


- Yureka

Re: [PATCH] nvme: do not reset controllers in NVME_CTRL_NEW state

[Keith Busch]

On Sat, Oct 03, 2026 at 04:22:09PM +0200, Yureka Lilian wrote:
> On 10/3/26 15:09, Maurizio Lombardi wrote:
> > On Sat Oct 3, 2026 at 2:33 PM CEST, Maurizio Lombardi wrote:
> > > Thanks for reporting this,
> > > it seems that nvme-apple is the only driver that still initializes the
> > > controller by performing a reset directly from a NEW state.
> > > 
> > > This should be easy to fix by moving the state to CONNECTING in
> > > apple_nvme_probe, just before nvme_reset_ctrl() is called.
> > Ops, this won't work, as we can enter the resetting state only when the
> > controller is live.
> > 
> > Keith, I think we have no other option but revert this patch to unbreak
> > apple-nvme until I find a proper fix.
> > 
> > Maurizio
> 
> 
> When you eventually propose a new variant of this, I can certainly test it
> with nvme-apple.

Thanks for the notice. Patch reverted committed to nvme-7.3 and will
send a pull request shortly.