linux-hyperv.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH RFC POC 00/50] file: handle files on syscall exit
@ 2026-09-15 11:30 Christian Brauner
  2026-09-15 11:30 ` [PATCH RFC POC 01/50] file: install " Christian Brauner
                   ` (51 more replies)
  0 siblings, 52 replies; 108+ messages in thread
From: Christian Brauner @ 2026-09-15 11:30 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexander Viro, Jann Horn, Jan Kara, Ingo Molnar, Peter Zijlstra,
	linux-fsdevel, linux-kernel, linux-mm, Oleg Nesterov, linux-alpha,
	linux-snps-arc, linux-arm-kernel, linux-csky, linux-hexagon,
	linux-m68k, linux-mips, linux-openrisc, linux-parisc, linux-sh,
	sparclinux, linux-um, Jens Axboe, io-uring, netdev, linuxppc-dev,
	linux-gpio, linux-arm-msm, dri-devel, bpf, David Airlie,
	virtualization, kvm, kexec, linux-hyperv,
	Christian Brauner (Amutable)

Hey,

The idea in this series is old-ish and really never let go of me and so
I wanted to at least dump it onto the list once even if it's just for
illustrative purposes.

I still had parts of an implementation laying around that I started
montsh ago alongside FD_PREPARE()/FD_ADD(). I sat down and finished it.

We've spoken about this idea a few times over the years that we could
reserve fds and files on a task and then install or a clean them up on
syscall success or error and get rid of most of the complicated cleanup
dance that we have in a lot of code. In particular drm. It last came up
during the Rust file descriptor reservation discussion quite some time
ago.

TL;DR, this lets arch code handle fd install and cleanup. fd_prepare()
allocates a descriptor like get_unused_fd_flags() does and records it in
a slot on the task. fd_stage() attaches the file to that slot and
returns the number.

When the syscall returns success the exit path installs every staged
file. When it returns an error it drops the descriptors and the files.

So a caller reserves, hands the number to userspace whenever it wants,
creates the file, stages it and returns errors without unwinding
anything.

fd_prepare() returns the slot itself, as a const pointer. The
preexisting fd_prepare_fd() and fd_prepare_file() give access to the fd
and file.

get_unused_fd_flags() and fd_install() don't change. A descriptor is
only reserved where a caller asks for it. And open(), dup() and all
other syscalls that maximize speed simply use FD_ADD().

The task keeps two slots inline. For SCM_RIGHTS and multi-descriptor
ioctls a spill array is added. It stick with the task.

Reservations belong to the thread and the syscall that made them. A
child of fork() starts without any. A thread can't unshare its fdtagble
with outstanding reservations. Kernel threads never return to userspace
so nothing would commit. Anything left at exit is a bug and gets warned
about and dropped.

Christian

Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
---
Christian Brauner (50):
      file: install files on syscall exit
      entry: commit fds on syscall exit
      alpha: commit fds on syscall exit
      ARC: commit fds on syscall exit
      ARM: commit fds on syscall exit
      arm64: commit fds on syscall exit
      csky: commit fds on syscall exit
      hexagon: commit fds on syscall exit
      m68k: commit fds on syscall exit
      microblaze: commit fds on syscall exit
      MIPS: commit fds on syscall exit
      nios2: commit fds on syscall exit
      openrisc: commit fds on syscall exit
      parisc: commit fds on syscall exit
      sh: commit fds on syscall exit
      sparc: commit fds on syscall exit
      um: commit fds on syscall exit
      xtensa: commit fds on syscall exit
      file: require the syscall exit hook from every architecture
      file: warn when the descriptor table is unshared with slots
      io_uring: commit fds per request
      net: install SCM_RIGHTS descriptors when recvmsg() returns
      file: open-code receive_fd()'s immediate install
      file: make FD_ADD() a standalone immediate install
      file: reimplement FD_PREPARE() on the deferred fd_prepare() path
      dma-buf: stop unwinding sync file descriptors by hand
      drm/amdkfd: stop collecting CRIU dma-buf descriptors for a final install
      drm/msm: install the out-fence descriptor when the ioctl returns
      drm/virtio: install the out-fence descriptor when the ioctl returns
      drm/vmwgfx: install the out-fence descriptor when the ioctl returns
      vfio: install the migration data descriptor when the ioctl returns
      liveupdate: install the session descriptors when the ioctl returns
      io_uring/zcrx: install the exported descriptor when the request returns
      sctp: install the peeloff descriptor when the syscall returns
      ALSA: compress: install the task descriptors when the ioctl returns
      nitro_enclaves: install the enclave descriptor when the ioctl returns
      tpm: vtpm_proxy: install the server descriptor when the ioctl returns
      perf: stop putting the event descriptor back on failure
      seccomp: stop putting the listener descriptor back on failure
      KVM: stop putting descriptors back on failure
      KVM: guest_memfd: stop putting the descriptor back on failure
      drm: stop unwinding descriptors by hand
      drm/amdgpu: stop unwinding the fence descriptor by hand
      drm/etnaviv: install the out-fence descriptor when the ioctl returns
      accel/habanalabs: stop putting the dma-buf descriptor back on failure
      xen/gntdev-dmabuf: stop putting the descriptor back on failure
      iio: buffer: install the buffer descriptor when the ioctl returns
      misc: fastrpc: install the dma-buf descriptor when the ioctl returns
      iommufd: stop putting descriptors back on failure
      Drivers: hv: mshv: stop putting descriptors back on failure

 arch/alpha/include/asm/thread_info.h         |   2 +
 arch/alpha/kernel/entry.S                    |  24 ++-
 arch/alpha/kernel/ptrace.c                   |   3 +
 arch/arc/include/asm/thread_info.h           |   2 +
 arch/arc/kernel/entry.S                      |   5 +
 arch/arc/kernel/ptrace.c                     |   4 +
 arch/arm/include/asm/thread_info.h           |   2 +
 arch/arm/kernel/entry-common.S               |   2 +
 arch/arm/kernel/ptrace.c                     |   4 +
 arch/arm64/include/asm/thread_info.h         |   4 +-
 arch/arm64/kernel/ptrace.c                   |   4 +
 arch/csky/include/asm/thread_info.h          |   2 +
 arch/csky/kernel/entry.S                     |   9 +
 arch/csky/kernel/ptrace.c                    |   4 +
 arch/hexagon/include/asm/thread_info.h       |   2 +
 arch/hexagon/kernel/traps.c                  |   4 +
 arch/m68k/68000/entry.S                      |  10 ++
 arch/m68k/coldfire/entry.S                   |  12 ++
 arch/m68k/include/asm/thread_info.h          |   2 +
 arch/m68k/kernel/entry.S                     |   2 +
 arch/m68k/kernel/ptrace.c                    |   3 +
 arch/microblaze/include/asm/thread_info.h    |   5 +-
 arch/microblaze/kernel/ptrace.c              |   4 +
 arch/mips/include/asm/thread_info.h          |   5 +-
 arch/mips/kernel/ptrace.c                    |   4 +
 arch/nios2/include/asm/thread_info.h         |   2 +
 arch/nios2/kernel/entry.S                    |   5 +
 arch/nios2/kernel/ptrace.c                   |   3 +
 arch/openrisc/include/asm/thread_info.h      |   5 +-
 arch/openrisc/kernel/entry.S                 |   5 +-
 arch/openrisc/kernel/ptrace.c                |   4 +
 arch/parisc/include/asm/thread_info.h        |   2 +
 arch/parisc/kernel/entry.S                   |  12 ++
 arch/parisc/kernel/ptrace.c                  |   4 +
 arch/powerpc/platforms/cell/spufs/inode.c    |  12 +-
 arch/sh/include/asm/thread_info.h            |  10 +-
 arch/sh/kernel/entry-common.S                |   8 +-
 arch/sh/kernel/ptrace_32.c                   |   4 +
 arch/sparc/include/asm/thread_info_32.h      |   2 +
 arch/sparc/include/asm/thread_info_64.h      |   9 +-
 arch/sparc/kernel/entry.S                    |   2 +-
 arch/sparc/kernel/ptrace_32.c                |   4 +
 arch/sparc/kernel/ptrace_64.c                |   4 +
 arch/sparc/kernel/syscalls.S                 |   4 +-
 arch/um/include/asm/thread_info.h            |   2 +
 arch/um/kernel/ptrace.c                      |   4 +
 arch/xtensa/include/asm/thread_info.h        |   2 +
 arch/xtensa/kernel/entry.S                   |   5 +
 arch/xtensa/kernel/ptrace.c                  |   4 +
 drivers/accel/habanalabs/common/memory.c     |  13 +-
 drivers/char/tpm/tpm_vtpm_proxy.c            |  30 +---
 drivers/dma-buf/dma-buf.c                    |  33 ++--
 drivers/dma-buf/sw_sync.c                    |  40 ++---
 drivers/dma-buf/sync_file.c                  |  52 ++----
 drivers/gpio/gpiolib-cdev.c                  |  18 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c       |  16 +-
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c     |  67 ++------
 drivers/gpu/drm/drm_lease.c                  |  20 +--
 drivers/gpu/drm/drm_prime.c                  |  13 +-
 drivers/gpu/drm/drm_syncobj.c                |  44 ++---
 drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c |  14 +-
 drivers/gpu/drm/msm/msm_gem_submit.c         |  22 +--
 drivers/gpu/drm/msm/msm_gem_vma.c            |  22 +--
 drivers/gpu/drm/msm/msm_perfcntr.c           |   6 +-
 drivers/gpu/drm/virtio/virtgpu_submit.c      |  32 ++--
 drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c      |  29 ++--
 drivers/hv/mshv_root_main.c                  |  27 +--
 drivers/hv/mshv_vtl_main.c                   |  20 +--
 drivers/iio/industrialio-buffer.c            |  26 +--
 drivers/iommu/iommufd/eventq.c               |  39 ++---
 drivers/media/mc/mc-request.c                |   6 +-
 drivers/misc/fastrpc.c                       |  19 +--
 drivers/misc/ntsync.c                        |   6 +-
 drivers/vfio/vfio_main.c                     |  25 +--
 drivers/virt/nitro_enclaves/ne_misc_dev.c    |  33 ++--
 drivers/xen/gntdev-dmabuf.c                  |  14 +-
 fs/eventfd.c                                 |   6 +-
 fs/eventpoll.c                               |   6 +-
 fs/exec.c                                    |   3 +
 fs/file.c                                    | 245 +++++++++++++++++++++++++--
 fs/namespace.c                               |  12 +-
 fs/nsfs.c                                    |   6 +-
 fs/xfs/xfs_handle.c                          |   6 +-
 include/linux/entry-common.h                 |   8 +-
 include/linux/file.h                         | 160 +++++------------
 include/linux/sched.h                        |  21 +++
 include/linux/thread_info.h                  |   2 +
 io_uring/io_uring.c                          |  27 +++
 io_uring/mock_file.c                         |   5 +-
 io_uring/zcrx.c                              |  25 +--
 kernel/bpf/bpf_iter.c                        |   6 +-
 kernel/bpf/token.c                           |   6 +-
 kernel/events/core.c                         |  31 ++--
 kernel/exit.c                                |   1 +
 kernel/fork.c                                |   3 +
 kernel/liveupdate/luo_core.c                 |  48 ++----
 kernel/liveupdate/luo_session.c              |  24 +--
 kernel/seccomp.c                             |  13 +-
 mm/userfaultfd.c                             |   6 +-
 net/core/scm.c                               |  13 +-
 net/handshake/netlink.c                      |  20 ++-
 net/kcm/kcmsock.c                            |   5 +-
 net/sctp/socket.c                            |  78 ++++-----
 sound/core/compress_offload.c                |  24 ++-
 virt/kvm/guest_memfd.c                       |  20 +--
 virt/kvm/kvm_main.c                          |  65 +++----
 106 files changed, 962 insertions(+), 861 deletions(-)
---
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
change-id: 20260915-work-fd-reserve-unify-folded-b89d3154b3d0


^ permalink raw reply	[flat|nested] 108+ messages in thread

end of thread, other threads:[~2026-09-16  9:50 UTC | newest]

Thread overview: 108+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-15 11:30 [PATCH RFC POC 00/50] file: handle files on syscall exit Christian Brauner
2026-09-15 11:30 ` [PATCH RFC POC 01/50] file: install " Christian Brauner
2026-09-15 11:55   ` sashiko-bot
2026-09-15 11:30 ` [PATCH RFC POC 02/50] entry: commit fds " Christian Brauner
2026-09-15 11:52   ` sashiko-bot
2026-09-15 11:30 ` [PATCH RFC POC 03/50] alpha: " Christian Brauner
2026-09-15 11:50   ` sashiko-bot
2026-09-15 11:30 ` [PATCH RFC POC 04/50] ARC: " Christian Brauner
2026-09-15 11:48   ` sashiko-bot
2026-09-15 11:30 ` [PATCH RFC POC 05/50] ARM: " Christian Brauner
2026-09-15 11:59   ` sashiko-bot
2026-09-15 11:30 ` [PATCH RFC POC 06/50] arm64: " Christian Brauner
2026-09-15 11:48   ` sashiko-bot
2026-09-15 11:30 ` [PATCH RFC POC 07/50] csky: " Christian Brauner
2026-09-15 11:47   ` sashiko-bot
2026-09-15 11:30 ` [PATCH RFC POC 08/50] hexagon: " Christian Brauner
2026-09-15 11:49   ` sashiko-bot
2026-09-15 11:30 ` [PATCH RFC POC 09/50] m68k: " Christian Brauner
2026-09-15 11:50   ` sashiko-bot
2026-09-15 11:30 ` [PATCH RFC POC 10/50] microblaze: " Christian Brauner
2026-09-15 11:48   ` sashiko-bot
2026-09-15 11:30 ` [PATCH RFC POC 11/50] MIPS: " Christian Brauner
2026-09-15 11:45   ` sashiko-bot
2026-09-15 11:30 ` [PATCH RFC POC 12/50] nios2: " Christian Brauner
2026-09-15 11:54   ` sashiko-bot
2026-09-15 11:30 ` [PATCH RFC POC 13/50] openrisc: " Christian Brauner
2026-09-15 12:02   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 14/50] parisc: " Christian Brauner
2026-09-15 11:59   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 15/50] sh: " Christian Brauner
2026-09-15 11:55   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 16/50] sparc: " Christian Brauner
2026-09-15 11:57   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 17/50] um: " Christian Brauner
2026-09-15 11:55   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 18/50] xtensa: " Christian Brauner
2026-09-15 12:00   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 19/50] file: require the syscall exit hook from every architecture Christian Brauner
2026-09-15 11:54   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 20/50] file: warn when the descriptor table is unshared with slots Christian Brauner
2026-09-15 12:02   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 21/50] io_uring: commit fds per request Christian Brauner
2026-09-15 12:11   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 22/50] net: install SCM_RIGHTS descriptors when recvmsg() returns Christian Brauner
2026-09-15 12:03   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 23/50] file: open-code receive_fd()'s immediate install Christian Brauner
2026-09-15 12:00   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 24/50] file: make FD_ADD() a standalone " Christian Brauner
2026-09-15 12:01   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 25/50] file: reimplement FD_PREPARE() on the deferred fd_prepare() path Christian Brauner
2026-09-15 12:05   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 26/50] dma-buf: stop unwinding sync file descriptors by hand Christian Brauner
2026-09-15 12:05   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 27/50] drm/amdkfd: stop collecting CRIU dma-buf descriptors for a final install Christian Brauner
2026-09-15 12:04   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 28/50] drm/msm: install the out-fence descriptor when the ioctl returns Christian Brauner
2026-09-15 12:04   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 29/50] drm/virtio: " Christian Brauner
2026-09-15 12:11   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 30/50] drm/vmwgfx: " Christian Brauner
2026-09-15 12:09   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 31/50] vfio: install the migration data " Christian Brauner
2026-09-15 12:07   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 32/50] liveupdate: install the session descriptors " Christian Brauner
2026-09-15 12:09   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 33/50] io_uring/zcrx: install the exported descriptor when the request returns Christian Brauner
2026-09-15 12:09   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 34/50] sctp: install the peeloff descriptor when the syscall returns Christian Brauner
2026-09-15 12:14   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 35/50] ALSA: compress: install the task descriptors when the ioctl returns Christian Brauner
2026-09-15 12:13   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 36/50] nitro_enclaves: install the enclave descriptor " Christian Brauner
2026-09-15 12:13   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 37/50] tpm: vtpm_proxy: install the server " Christian Brauner
2026-09-15 12:14   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 38/50] perf: stop putting the event descriptor back on failure Christian Brauner
2026-09-15 12:12   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 39/50] seccomp: stop putting the listener " Christian Brauner
2026-09-15 12:14   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 40/50] KVM: stop putting descriptors " Christian Brauner
2026-09-15 12:15   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 41/50] KVM: guest_memfd: stop putting the descriptor " Christian Brauner
2026-09-15 12:16   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 42/50] drm: stop unwinding descriptors by hand Christian Brauner
2026-09-15 12:17   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 43/50] drm/amdgpu: stop unwinding the fence descriptor " Christian Brauner
2026-09-15 12:14   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 44/50] drm/etnaviv: install the out-fence descriptor when the ioctl returns Christian Brauner
2026-09-15 12:15   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 45/50] accel/habanalabs: stop putting the dma-buf descriptor back on failure Christian Brauner
2026-09-15 12:18   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 46/50] xen/gntdev-dmabuf: stop putting the " Christian Brauner
2026-09-15 12:21   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 47/50] iio: buffer: install the buffer descriptor when the ioctl returns Christian Brauner
2026-09-15 12:18   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 48/50] misc: fastrpc: install the dma-buf " Christian Brauner
2026-09-15 12:20   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 49/50] iommufd: stop putting descriptors back on failure Christian Brauner
2026-09-15 12:21   ` sashiko-bot
2026-09-15 11:31 ` [PATCH RFC POC 50/50] Drivers: hv: mshv: " Christian Brauner
2026-09-15 12:19   ` sashiko-bot
2026-09-15 16:02 ` [PATCH RFC POC 00/50] file: handle files on syscall exit Linus Torvalds
2026-09-15 22:21   ` Rob Clark
2026-09-15 22:54     ` Linus Torvalds
2026-09-16  7:09   ` Christian Brauner
2026-09-15 17:51 ` Jann Horn
2026-09-15 19:08   ` Linus Torvalds
2026-09-16  9:50     ` David Laight

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).