The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Paolo Abeni <pabeni@redhat.com>
To: Slawomir Stepien <sst@poczta.fm>
Cc: syzbot <syzbot@kernel.org>,
	syzkaller-bugs@googlegroups.com,
	Andrew Lunn <andrew+netdev@lunn.ch>,
	"David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@google.com>,
	Jakub Kicinski <kuba@kernel.org>,
	netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
	syzbot@lists.linux.dev
Subject: Re: [PATCH] netdevsim: fix deadlock in nsim_bus_dev_max_vfs_write()
Date: Wed, 5 Aug 2026 11:05:45 +0200	[thread overview]
Message-ID: <14b8701a-44a6-40dc-b46a-46096ea5bca2@redhat.com> (raw)
In-Reply-To: <anL6nNIJDqWOynzw@nr200>

On 8/5/26 10:55 AM, Slawomir Stepien wrote:
> On sie 04, 2026 16:33, Paolo Abeni wrote:
>> On 7/31/26 11:55 AM, syzbot wrote:
>>> From: Slawomir Stepien <sst@poczta.fm>
>>>
>>> There is an ABBA deadlock between the devlink instance lock and the debugfs
>>> active file reference mechanism.
>>>
>>> When a device is being removed (e.g., via nsim_drv_remove()), the driver
>>> acquires the devlink instance lock and then calls
>>> debugfs_remove_recursive(). This function blocks waiting for all active
>>> file operations on the debugfs files to complete.
>>>
>>> Concurrently, if a user writes to the "max_vfs" debugfs file, the VFS layer
>>> acquires an active reference to the file. The write handler,
>>> nsim_bus_dev_max_vfs_write(), then attempts to acquire the devlink instance
>>> lock, which is already held by the removal task.
>>>
>>> This creates a circular dependency resulting in a deadlock:
>>>
>>> INFO: task blocked for more than 143 seconds.
>>> Call Trace:
>>>  wait_for_completion+0x2ca/0x5e0 kernel/sched/completion.c:153
>>>  __debugfs_file_removed fs/debugfs/inode.c:751 [inline]
>>>  remove_one+0x2df/0x3b0 fs/debugfs/inode.c:758
>>>  __simple_recursive_removal+0x215/0x520 fs/libfs.c:623
>>>  debugfs_remove+0x5b/0x70 fs/debugfs/inode.c:781
>>>  nsim_dev_debugfs_exit drivers/net/netdevsim/dev.c:372 [inline]
>>>  nsim_drv_remove+0xc0/0x170 drivers/net/netdevsim/dev.c:1803
>>>
>>> INFO: task blocked for more than 143 seconds.
>>> Call Trace:
>>>  __mutex_lock+0x7bf/0x1550 kernel/locking/mutex.c:821
>>>  nsim_bus_dev_max_vfs_write+0x229/0x3d0 drivers/net/netdevsim/dev.c:276
>>>  full_proxy_write+0x127/0x1f0 fs/debugfs/file.c:388
>>>  vfs_write+0x296/0xba0 fs/read_write.c:685
>>>
>>> To fix this, use devl_trylock() in nsim_bus_dev_max_vfs_write() instead of
>>> devl_lock(). If the lock cannot be acquired, return -EBUSY. This aborts the
>>> write operation, releases the debugfs active file reference, and allows the
>>> pending debugfs_remove_recursive() to proceed.
>>>
>>> Assisted-by: Gemini:gemini-3.5-flash Gemini:gemini-3.1-pro-preview syzbot
>>> Reported-by: syzbot+3147c5de186107ffc7a1@syzkaller.appspotmail.com
>>> Closes: https://syzkaller.appspot.com/bug?extid=3147c5de186107ffc7a1
>>> Link: https://syzkaller.appspot.com/ai_job?id=fcdda182-bebf-49ab-ada0-d4d2e814ebf7
>>> Signed-off-by: Slawomir Stepien <sst@poczta.fm>
>>>
>>> ---
>>> diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c
>>> index aed9ad5f1..421cd7327 100644
>>> --- a/drivers/net/netdevsim/dev.c
>>> +++ b/drivers/net/netdevsim/dev.c
>>> @@ -273,7 +273,11 @@ static ssize_t nsim_bus_dev_max_vfs_write(struct file *file,
>>>  		return -ENOMEM;
>>>  
>>>  	nsim_dev = file->private_data;
>>> -	devl_lock(priv_to_devlink(nsim_dev));
>>> +	if (!devl_trylock(priv_to_devlink(nsim_dev))) {
>>> +		ret = -EBUSY;
>>
>> I think this should return restart_syscall(). This kind of schema is
>> calling for trouble, but for netdevsim should be okish.
> 
> Thanks Paolo for your comment!
> 
> Can you explain a bit more? What is your reasoning here? Is it, that we expect that we could have
> the lock in e.g. 1ns, so it is worth restarting without bothering userspace[1]?
> 
> I see a lot of:
> 
> if (!rtnl_trylock())
> 	return restart_syscall();
> 
> Why this pattern is so popular? Is it for the same reason as above?
> 
> [1] https://kernel-internals.org/syscalls/restart-block/
If you don't restart, the write will randomly fail when the lock is
contended, as reported by sashiko gemini:

https://sashiko.dev/#/patchset/b7bf56ea-7522-4163-acd5-aaa69ad03b3a%40mail.kernel.org

the user-experience will be terrible at best.

Also, I missed this other report initially:

https://netdev-ai.bots.linux.dev/sashiko/#/patchset/b7bf56ea-7522-4163-acd5-aaa69ad03b3a%40mail.kernel.org

it looks like the above is not a complete fix.

/P


  reply	other threads:[~2026-08-05  9:05 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-31  9:55 [PATCH] netdevsim: fix deadlock in nsim_bus_dev_max_vfs_write() syzbot
2026-08-04 14:33 ` Paolo Abeni
2026-08-05  8:55   ` Slawomir Stepien
2026-08-05  9:05     ` Paolo Abeni [this message]
2026-08-05 23:08 ` Jakub Kicinski
2026-08-07  7:31   ` Slawomir Stepien

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=14b8701a-44a6-40dc-b46a-46096ea5bca2@redhat.com \
    --to=pabeni@redhat.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=sst@poczta.fm \
    --cc=syzbot@kernel.org \
    --cc=syzbot@lists.linux.dev \
    --cc=syzkaller-bugs@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox