Linux kernel -stable discussions
 help / color / mirror / Atom feed
* [PATCH] devpts: create pty nodes under the parent lock
@ 2026-11-17 15:27 Christian Brauner
  2026-11-17 16:46 ` Alexander Mikhalitsyn
  0 siblings, 1 reply; 2+ messages in thread
From: Christian Brauner @ 2026-11-17 15:27 UTC (permalink / raw)
  To: linux-fsdevel
  Cc: Alexander Viro, Jan Kara, linux-kernel, Cheng Yongkang, stable,
	Christian Brauner (Amutable)

devpts_pty_new() creates /dev/pts/N with d_alloc_name() and
d_make_persistent() without holding the inode lock of the devpts root.
A lookup of /dev/pts/N that misses the dcache holds that lock shared,
gets an in-lookup dentry from d_alloc_parallel() and calls
simple_lookup(), which hashes it as a negative dentry. Nothing
serializes the two, so the pty can be created and hashed after the
lookup missed it, and simple_lookup() then hashes its negative dentry
for the same name right in front of it:

  lookup                           devpts_pty_new()
  inode_lock_shared(root)
  d_alloc_parallel(root, "N")
    miss
                                   d_alloc_name(root, "N")
                                   d_make_persistent()
  simple_lookup()
    d_splice_alias(NULL)
  inode_unlock_shared(root)

Both dentries are now hashed under the same name and the negative one
is found first. So stat() or open() of a freshly allocated pty fails
with ENOENT until the negative dentry goes away. Cheng measured about
26k bogus ENOENTs in 6.5 million lookups while ptys were being created
in a loop.

The parent lock used to be held here but was dropped in 8ead9dd54716
("devpts: more pty driver interface cleanups"). b4a76faf9d96 ("convert
devpts") then switched mknod_ptmx() over to simple_start_creating() but
left devpts_pty_new() alone.

Do what mknod_ptmx() does and create the pty node with
simple_start_creating() and simple_done_creating(). That takes the
parent lock exclusive, so creation waits for lookups in flight, and it
reuses the negative dentry a finished lookup might have left behind
instead of hashing a second dentry next to it.

devpts_pty_kill() doesn't need the same treatment. It only unhashes the
dentry and a lookup racing with it either still finds the pty or misses
and correctly reports ENOENT. The index of a pty is only freed in
->remove() after the master was closed and devpts_pty_kill() ran, so
devpts_pty_new() never finds a positive dentry for the name it creates.

Note that ptmx_open() calls devpts_pty_new() with the tty locked. We
never take the tty lock under the inode lock of the devpts root:
lookup_open() drops the parent lock before do_open() calls ->open() and
devpts has no ->atomic_open().

Reported-by: Cheng Yongkang <teel4res@gmail.com>
Closes: https://lore.kernel.org/20261115163620.225761-1-teel4res@gmail.com
Fixes: 8ead9dd54716 ("devpts: more pty driver interface cleanups")
Cc: stable@vger.kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
 fs/devpts/inode.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/fs/devpts/inode.c b/fs/devpts/inode.c
index bd1e8eb26edb7..65f6aee74fa45 100644
--- a/fs/devpts/inode.c
+++ b/fs/devpts/inode.c
@@ -521,15 +521,15 @@ struct dentry *devpts_pty_new(struct pts_fs_info *fsi, int index, void *priv)
 
 	sprintf(s, "%d", index);
 
-	dentry = d_alloc_name(root, s);
-	if (!dentry) {
+	dentry = simple_start_creating(root, s);
+	if (IS_ERR(dentry)) {
 		iput(inode);
-		return ERR_PTR(-ENOMEM);
+		return dentry;
 	}
 	dentry->d_fsdata = priv;
 	d_make_persistent(dentry, inode);
 	fsnotify_create(d_inode(root), dentry);
-	dput(dentry);
+	simple_done_creating(dentry);
 	return dentry; // borrowed
 }
 

---
base-commit: 666539eecdf4db623f5e3c3e55cf9f2dfad24aa4
change-id: 20261117-work-devpts-pty_new_locking-9b3d01e643c3


^ permalink raw reply related	[flat|nested] 2+ messages in thread

* Re: [PATCH] devpts: create pty nodes under the parent lock
  2026-11-17 15:27 [PATCH] devpts: create pty nodes under the parent lock Christian Brauner
@ 2026-11-17 16:46 ` Alexander Mikhalitsyn
  0 siblings, 0 replies; 2+ messages in thread
From: Alexander Mikhalitsyn @ 2026-11-17 16:46 UTC (permalink / raw)
  To: Christian Brauner, linux-fsdevel
  Cc: Alexander Viro, Jan Kara, linux-kernel, Cheng Yongkang, stable

On Tue, 2026-11-17 at 16:27 +0100, Christian Brauner wrote:
> devpts_pty_new() creates /dev/pts/N with d_alloc_name() and
> d_make_persistent() without holding the inode lock of the devpts
> root.
> A lookup of /dev/pts/N that misses the dcache holds that lock shared,
> gets an in-lookup dentry from d_alloc_parallel() and calls
> simple_lookup(), which hashes it as a negative dentry. Nothing
> serializes the two, so the pty can be created and hashed after the
> lookup missed it, and simple_lookup() then hashes its negative dentry
> for the same name right in front of it:
> 
>   lookup                           devpts_pty_new()
>   inode_lock_shared(root)
>   d_alloc_parallel(root, "N")
>     miss
>                                    d_alloc_name(root, "N")
>                                    d_make_persistent()
>   simple_lookup()
>     d_splice_alias(NULL)
>   inode_unlock_shared(root)
> 
> Both dentries are now hashed under the same name and the negative one
> is found first. So stat() or open() of a freshly allocated pty fails
> with ENOENT until the negative dentry goes away. Cheng measured about
> 26k bogus ENOENTs in 6.5 million lookups while ptys were being
> created
> in a loop.
> 
> The parent lock used to be held here but was dropped in 8ead9dd54716
> ("devpts: more pty driver interface cleanups"). b4a76faf9d96
> ("convert
> devpts") then switched mknod_ptmx() over to simple_start_creating()
> but
> left devpts_pty_new() alone.
> 
> Do what mknod_ptmx() does and create the pty node with
> simple_start_creating() and simple_done_creating(). That takes the
> parent lock exclusive, so creation waits for lookups in flight, and
> it
> reuses the negative dentry a finished lookup might have left behind
> instead of hashing a second dentry next to it.
> 
> devpts_pty_kill() doesn't need the same treatment. It only unhashes
> the
> dentry and a lookup racing with it either still finds the pty or
> misses
> and correctly reports ENOENT. The index of a pty is only freed in
> ->remove() after the master was closed and devpts_pty_kill() ran, so
> devpts_pty_new() never finds a positive dentry for the name it
> creates.
> 
> Note that ptmx_open() calls devpts_pty_new() with the tty locked. We
> never take the tty lock under the inode lock of the devpts root:
> lookup_open() drops the parent lock before do_open() calls ->open()
> and
> devpts has no ->atomic_open().
> 
> Reported-by: Cheng Yongkang <teel4res@gmail.com>
> Closes:
> https://lore.kernel.org/20261115163620.225761-1-teel4res@gmail.com
> Fixes: 8ead9dd54716 ("devpts: more pty driver interface cleanups")
> Cc: stable@vger.kernel.org

Reviewed-by: Alexander Mikhalitsyn <alex@amutable.com>

> Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
> ---
>  fs/devpts/inode.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/devpts/inode.c b/fs/devpts/inode.c
> index bd1e8eb26edb7..65f6aee74fa45 100644
> --- a/fs/devpts/inode.c
> +++ b/fs/devpts/inode.c
> @@ -521,15 +521,15 @@ struct dentry *devpts_pty_new(struct
> pts_fs_info *fsi, int index, void *priv)
>  
>  	sprintf(s, "%d", index);
>  
> -	dentry = d_alloc_name(root, s);
> -	if (!dentry) {
> +	dentry = simple_start_creating(root, s);
> +	if (IS_ERR(dentry)) {
>  		iput(inode);
> -		return ERR_PTR(-ENOMEM);
> +		return dentry;
>  	}
>  	dentry->d_fsdata = priv;
>  	d_make_persistent(dentry, inode);
>  	fsnotify_create(d_inode(root), dentry);
> -	dput(dentry);
> +	simple_done_creating(dentry);
>  	return dentry; // borrowed
>  }
>  
> 
> ---
> base-commit: 666539eecdf4db623f5e3c3e55cf9f2dfad24aa4
> change-id: 20261117-work-devpts-pty_new_locking-9b3d01e643c3

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2026-11-17 16:46 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-17 15:27 [PATCH] devpts: create pty nodes under the parent lock Christian Brauner
2026-11-17 16:46 ` Alexander Mikhalitsyn

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox