Linux filesystem development
 help / color / mirror / Atom feed
* [PATCH RFC v4 00/18] coredump, files: exit files on request
@ 2026-09-10 15:47 Christian Brauner
  2026-09-10 15:47 ` [PATCH RFC v4 01/18] fs: don't open-code file_close_fd() in close_fd() Christian Brauner
                   ` (18 more replies)
  0 siblings, 19 replies; 28+ messages in thread
From: Christian Brauner @ 2026-09-10 15:47 UTC (permalink / raw)
  To: NeilBrown, Oleg Nesterov, linux-fsdevel
  Cc: Jann Horn, Alexander Viro, Jan Kara, Xin Zhao, Mateusz Guzik,
	Jeff Layton, Jens Axboe, Christian Brauner (Amutable), NeilBrown

We've had quite a few proposal for exiting files before generating the
coredump (see [1]-[8]). The various proposed solution were quite
unacceptable. So I took some time to look what a _remotely_ acceptable
version of this could look like. Here it is.

The gist is that we allow the coredump socket to request the
thread-group shed the various fdtables closed synchronously before
generating the coredump.

The intricate task is that this requires us to add a primitive to close
fdtables synchronously. Coredump is not enough to justify this but
performance numbers for this look pretty convincing. For tasks exiting
with a large file descriptor table this means getting rid of _a lot_ of
cmpxchg()s and hammering on task->pi_lock. The cost of this was
mentioned in various threads over the years. I found at least recent
comments by Oleg and Neil.

The obvious problematic case are kernel threads. fput() punts their
final __fput() to a workqueue. kthreads never return to userspace to run
task work and some of them may need to finish a umount and so cannot
->release() inline. No path we change is taken by a kthread. They don't
own a descriptor table and kthreadd is created with CLONE_FILES and
every kthread inherits it. So all of them share init_files and init_task
pins that forever. Their exit_files() never drops the last reference.

kernel_execve() refuses PF_KTHREAD outright. Usermodehelpers are spawned
without the kthread flag. close_range() is a syscall. The one indirect
way is a failing copy_process() calling exit_files() on a
child from a kthread parent. A dup_fd() in copy_process() cannot hold
the last reference to any of its files while the source table is alive.

Worker threads such as io_uring workers are user threads without
PF_KTHREAD that share the fdtable. One of them can be the last to put
the fdtable at exit. Today fput() hands their final __fput()s to task
work. The worker runs that itself in exit_task_work() a few lines later.
Now the same thread does the same work in exit_files() instead. vhost
workers are created without a descriptor table at all, so they never
even put an fdtable. Neither kind ever execs or calls close_range().

Now the performance numbers. I measured this on a 64 vCPU KVM guest (AMD
EPYC 9754) on top of vfs-7.4.coredump with and without the series and
the same x86_64_defconfig for both. The benchmark opens N files in a
forked worker, moves them to random positions in the descriptor table so
that the table order has nothing to do with the allocation order, and
times the teardown:

- process exit
- execve() with N close-on-exec descriptors
- close_range() with and without CLOSE_RANGE_UNSHARE

Here are the medians over two boots which agree within 2%. With
/dev/null as the file and 4096, 65536 and 1M descriptors:

                         4096 fds        65536 fds        1M fds
  ----------------------------------------------------------------------
  exit                   1045 -> 895 us  18.9 -> 14.9 ms  429 -> 306 ms
                          -14.4%         -21.2%           -28.7%
  ----------------------------------------------------------------------
  execve, close-on-exec  1393 -> 1190 us 21.4 -> 16.9 ms  471 -> 325 ms
                          -14.6%         -21.0%           -31.1%
  ----------------------------------------------------------------------
  close_range()          1179 -> 988 us  21.9 -> 17.4 ms  484 -> 337 ms
                          -16.1%         -20.6%           -30.3%
  ----------------------------------------------------------------------
  close_range(UNSHARE)   1118 -> 958 us  22.1 -> 17.8 ms  487 -> 333 ms
                          -14.3%         -19.8%           -31.7%

So that's a 16-21% win or 40-150 ns per closed file. This sheds
init_task_work() and task_work_add() per file with its cmpxchg() and the
TIF_NOTIFY_RESUME test-and-set, the list walk with the indirect call
in the path out, and then a second pass over every struct file that
isn't cache-hot anymore by then.

The performance boost holds from 256 to 1M descriptors and is largest
at 1M. Note the thread count doesn't matter since one thread does the
close anyway.

So with eight processes tearing down 65536 descriptors each at the same
time the win is 16-21% (34.9 -> 29.5 ms for exit, 34.4 -> 27.0 ms
for execve). The old code walks the table once to queue the task work
and then walks all 16 MB of struct file a second time from
task_work_run().

So once eight CPUs compete for the cache that second pass gets
very expensive. With 32 or 64 processes at once both kernels are bound
by pushing millions of files through slab and RCU. So then the
difference shrinks to 4-8%. Still though...

Files whose own release dominates the cleanup gain a little less. 13%
for eventfds, 11% for pipes, 6% for AF_UNIX sockets and 7% for distinct
tmpfs inodes. For them a release costs about 2 us per descriptor anyway.

Using function profiling a 65536 descriptor exit spends 4.6 ms in
exit_files() and 11.1 ms in task_work_run() before and now 13.7 ms in
exit_files() and nothing in task_work_run() after.

close(2) is untouched and measures the same. will-it-scale (open1,
open3, dup1, eventfd1, unix1, pipe1, signal1, processes and threads)
is within 2% either way. dup1_threads at 16 tasks and getppid1 at 64
tasks a few percent better which I'd put down to layout.

So, back to coredumps. With COREDUMP_CLOSE_FILES coredumps switch to an
empty file descriptor table before generating the coredump. That
requires a bit of synchronization but I think I got the basics down.

The coredumping thread allocates a new empty file descriptor table. It
then wakes all threads in the thread-group and tells them to switch to
the empty file descriptor table and get rid of the old one. They report
back once they're done.

So that handles most cases where locks would be held for an unreasonable
time until the coredump is generated but since files can be shared
between completely unrelated processes that's not a guarantee and we
can't give one. But it solves the reported issue without resorting to
even grosser hacks.

Link: https://lore.kernel.org/20260618030700.2511668-1-jackzxcui1989@163.com [1]
Link: https://lore.kernel.org/20260618150301.3226517-1-jackzxcui1989@163.com [2]
Link: https://lore.kernel.org/20260619122419.3954581-1-jackzxcui1989@163.com [3]
Link: https://lore.kernel.org/20260624145552.70143-1-jackzxcui1989@163.com [4]
Link: https://lore.kernel.org/20260630075604.52533-1-jackzxcui1989@163.com [5]
Link: https://lore.kernel.org/20260804001703.1340667-1-jackzxcui1989@163.com [6]
Link: https://lore.kernel.org/20260807040124.1706927-1-jackzxcui1989@163.com [7]
Link: https://lore.kernel.org/20260808052732.2589657-1-jackzxcui1989@163.com [8]

Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
Changes in v4:
- Fix performance number variations. It said variable within 4%. It's
  actually just 2%.
- Make various simplifications and add wait_var_event_state().
- Link to v3: https://patch.msgid.link/20260909-work-coredump-unlock-self-v3-0-04f907687563@kernel.org

Changes in v3:
- Drop the llist. This brings yet another performance boost.
- Link to v2: https://patch.msgid.link/20260902-work-coredump-unlock-self-v2-0-1bece368cbb1@kernel.org

Changes in v2:
- Revamp.
- Link to v1: https://patch.msgid.link/20260824-work-coredump-unlock-self-v1-0-48fd1cca7eda@kernel.org

---
Christian Brauner (18):
      fs: don't open-code file_close_fd() in close_fd()
      fs: add switch_files_struct()
      fs: move unshare_fd() to fs/file.c
      fs: remove unshare_files()
      fs: add filp_close_sync()
      fs: make close_files() synchronous
      fs: make close_range() synchronous
      fs: rename do_close_on_exec() to close_cloexec_files()
      fs: make close_cloexec_files() synchronous
      coredump: drop core_state->dumper
      sched: add wait_var_event_state()
      coredump: replace the startup completion with a thread count
      coredump: factor out coredump_wait_inactive()
      fs: add alloc_files_struct()
      coredump: add COREDUMP_CLOSE_FILES
      coredump: cancel io_uring requests before closing files
      tools: sync coredump.h header
      selftests/coredump: test COREDUMP_CLOSE_FILES

 fs/binfmt_elf.c                                    |   2 +-
 fs/binfmt_elf_fdpic.c                              |   2 +-
 fs/coredump.c                                      |  91 +++-
 fs/exec.c                                          |  13 +-
 fs/file.c                                          | 107 ++--
 fs/internal.h                                      |   1 +
 fs/open.c                                          |  17 +-
 include/linux/fdtable.h                            |   6 +-
 include/linux/sched/signal.h                       |   8 +-
 include/linux/wait_bit.h                           |  26 +
 include/uapi/linux/coredump.h                      |   8 +
 kernel/exit.c                                      |  22 +-
 kernel/fork.c                                      |  48 +-
 tools/include/uapi/linux/coredump.h                |   8 +
 tools/testing/selftests/coredump/Makefile          |   4 +-
 .../selftests/coredump/coredump_close_files_test.c | 592 +++++++++++++++++++++
 .../coredump/coredump_socket_protocol_test.c       |   6 +
 .../selftests/coredump/coredump_test_helpers.c     |   3 +-
 18 files changed, 840 insertions(+), 124 deletions(-)
---
base-commit: a5625efa7a0e77c26e857de147284a21d53fa3ad
change-id: 20260824-work-coredump-unlock-self-63a898912870


^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2026-09-26 11:53 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-10 15:47 [PATCH RFC v4 00/18] coredump, files: exit files on request Christian Brauner
2026-09-10 15:47 ` [PATCH RFC v4 01/18] fs: don't open-code file_close_fd() in close_fd() Christian Brauner
2026-09-10 15:48 ` [PATCH RFC v4 02/18] fs: add switch_files_struct() Christian Brauner
2026-09-10 15:48 ` [PATCH RFC v4 03/18] fs: move unshare_fd() to fs/file.c Christian Brauner
2026-09-10 15:48 ` [PATCH RFC v4 04/18] fs: remove unshare_files() Christian Brauner
2026-09-10 15:48 ` [PATCH RFC v4 05/18] fs: add filp_close_sync() Christian Brauner
2026-09-10 15:48 ` [PATCH RFC v4 06/18] fs: make close_files() synchronous Christian Brauner
2026-09-10 15:48 ` [PATCH RFC v4 07/18] fs: make close_range() synchronous Christian Brauner
2026-09-10 15:48 ` [PATCH RFC v4 08/18] fs: rename do_close_on_exec() to close_cloexec_files() Christian Brauner
2026-09-10 15:48 ` [PATCH RFC v4 09/18] fs: make close_cloexec_files() synchronous Christian Brauner
2026-09-24 12:07   ` Oleg Nesterov
2026-09-10 15:48 ` [PATCH RFC v4 10/18] coredump: drop core_state->dumper Christian Brauner
2026-09-23 15:27   ` Oleg Nesterov
2026-09-10 15:48 ` [PATCH RFC v4 11/18] sched: add wait_var_event_state() Christian Brauner
2026-09-23 15:28   ` Oleg Nesterov
2026-09-10 15:48 ` [PATCH RFC v4 12/18] coredump: replace the startup completion with a thread count Christian Brauner
2026-09-23 15:28   ` Oleg Nesterov
2026-09-10 15:48 ` [PATCH RFC v4 13/18] coredump: factor out coredump_wait_inactive() Christian Brauner
2026-09-23 15:29   ` Oleg Nesterov
2026-09-10 15:48 ` [PATCH RFC v4 14/18] fs: add alloc_files_struct() Christian Brauner
2026-09-10 15:48 ` [PATCH RFC v4 15/18] coredump: add COREDUMP_CLOSE_FILES Christian Brauner
2026-09-24 14:47   ` Oleg Nesterov
2026-09-25 16:01     ` Christian Brauner
2026-09-26 11:52       ` Oleg Nesterov
2026-09-10 15:48 ` [PATCH RFC v4 16/18] coredump: cancel io_uring requests before closing files Christian Brauner
2026-09-10 15:48 ` [PATCH RFC v4 17/18] tools: sync coredump.h header Christian Brauner
2026-09-10 15:48 ` [PATCH RFC v4 18/18] selftests/coredump: test COREDUMP_CLOSE_FILES Christian Brauner
2026-09-10 23:48 ` [PATCH RFC v4 00/18] coredump, files: exit files on request NeilBrown

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox