[PATCH v3 0/3] fuse: atomic open

[Jimmy Zuber <jamz@amazon.com>]

Add FUSE_OPEN_ATOMIC, allowing the kernel to skip LOOKUP to improve
performance. A server implementing FUSE_OPEN_ATOMIC will look up
the name, create it if O_CREAT is supplied, and open it in one request.

The reply says whether the file was created, as Miklos suggested in [1].
The overall pattern follows his sketch in [2]. For a cached
name, the kernel compares the returned nodeid with the cached one and
looks the name up again on a mismatch.

A server without FUSE_OPEN_ATOMIC replies -ENOSYS and keeps today's
behaviour.

This series applies on top of "fuse: fix use-after-free in
fuse_create_open() error path" [3].

Tested with a new selftest under UML and KVM.

Changes in v3 (from the Sashiko review [4]):
- Patch 1: strip FOPEN_CREATED from the open flags, which broke
  passthrough. Check name_max before FUSE_OPEN_ATOMIC. Reject a reply
  whose inode is not a regular file or is bad, and skip page cache
  invalidation for DAX, as fuse_open() does. With default_permissions,
  check the attributes in the reply before opening an existing file.
- Patch 2: only an open that skipped the LOOKUP checks the name, so
  reopens through /proc/self/fd, open_by_handle_at() and other
  vfs_open() callers open by nodeid. Keep the LOOKUP for directory
  opens, and with default_permissions while the cached attributes have
  expired. Check the open again against the attributes in the reply.
  Set the open flags only after the reply checks, so a mismatched
  passthrough reply is released without an inode. Read the parent and
  name under rename_lock, and send RELEASE before FORGET.
- Patch 3: test the above. Skip without /dev/fuse, and leave the server
  state mapped at teardown.

Changes in v2:
- Drop the d_drop() before d_splice_alias() for an entry-only reply, and
  use d_splice_alias() for FUSE_CREATE and FUSE_OPEN_ATOMIC replies.
  FUSE_TMPFILE keeps d_instantiate(), as its dentry stays unhashed
  (NeilBrown).
- Describe what the server does and when the kernel checks permission
  in each case (NeilBrown).
- Check in the selftest that an O_TMPFILE file has no name in its
  directory.
- Rebase onto fuse for-next, which has f1ac607f786f ("VFS: enhance
  d_splice_alias() to handle hashed dentries"). Accept a fuse_open_out
  without backing_id_64, as FUSE_CREATE and FUSE_OPEN now do, and test
  it.

v2: https://lore.kernel.org/fuse-devel/20261026151828.562870-1-jamz@amazon.com/
v1: https://lore.kernel.org/fuse-devel/20261020141647.3271341-1-jamz@amazon.com/

[1] https://lore.kernel.org/all/CAJfpeguEK4JsFo60=yHkpLL6XkDGS7cQ5GOhiFL379KsYk0TsQ@mail.gmail.com/
[2] https://lore.kernel.org/all/CAJfpegvXtXY=Pbxv+dMGFR8mvWN0DUwhSo6NwaVexk6Y6sao+w@mail.gmail.com/
[3] https://lore.kernel.org/fuse-devel/20261026180649.1265825-1-jamz@amazon.com/
[4] https://sashiko.dev/#/patchset/20261026151828.562870-1-jamz@amazon.com

Jimmy Zuber (3):
  fuse: add FUSE_OPEN_ATOMIC
  fuse: open cached files with FUSE_OPEN_ATOMIC
  selftests/fuse: add test for FUSE_OPEN_ATOMIC

 fs/fuse/dir.c                                 | 456 +++++++++++--
 fs/fuse/file.c                                |   4 +-
 fs/fuse/fuse_i.h                              |   7 +
 include/uapi/linux/fuse.h                     |   8 +-
 .../selftests/filesystems/fuse/.gitignore     |   1 +
 .../selftests/filesystems/fuse/Makefile       |   3 +-
 .../filesystems/fuse/atomic_open_test.c       | 624 ++++++++++++++++++
 7 files changed, 1043 insertions(+), 60 deletions(-)
 create mode 100644 tools/testing/selftests/filesystems/fuse/atomic_open_test.c


base-commit: 9b1da7b4a4e197072e630c3b85d4100a399752ff
prerequisite-patch-id: 95a6559bc4490d7d82bc80ba961ac52e80429b34
-- 
2.50.1