* Re: [BUG] ext4: BUG_ON in ext4_write_inline_data (fs/ext4/inline.c:240)
From: Theodore Tso @ 2026-04-29 4:40 UTC (permalink / raw)
To: Demi Marie Obenour
Cc: Zw Tang, Andreas Dilger, libaokun, jack, ojaswin, linux-ext4,
linux-kernel, yi.zhang, syzkaller-bugs
In-Reply-To: <5981069b-8c54-4b5b-a808-5ebdc8cd7265@gmail.com>
On Tue, Apr 28, 2026 at 04:50:14PM -0400, Demi Marie Obenour wrote:
> This is an example of a BadUSB attack, which has been known since
> at least 2014. USB sticks *do* have their own microcontrollers to
> run their firmware. At least in the past this firmware has been
> programmable and not been digitally signed. This means that the USB
> stick *can* be reprogrammed to perform a TOCTOU attack on a filesystem,
> or indeed to implement a completely different kind of USB device.
Honestly, if that's what you are worried about, then best solution is
put epoxy in every single USB port. I've since financial institutions
that have done precisely this, and both Android and Chrome OS supports
enterprise security policies which does the equivalent in software.
> Protecting against replay attacks requires a Merkle tree. The only
> Linux filesystems that I know have one are ZFS, bcachefs, and BTRFS.
> The first two are out of tree and the third is not shipped in RHEL
> at least.
FYI, there was a patch for btrfs, but it was never landed. It's
unclear how many people would be willing to pay the performance tax of
using hmac-sha256 for every single data and metadata block write.
> Of course, you are free to choose which (if any) of these attacks you
> care about. One can that USB sticks should be mounted in userspace,
> UEFI secure boot with Microsoft keys is irrelevant as long as
> administrator -> kernel isn't a security boundary on Windows, and that
> confidential computing only makes sense for stateless workloads (which
> can use dm-verity) until there is a way to trust storage devices.
> But it's always best to be aware that an attack vector exists,
> whether or not one chooses to address it.
Sure, but a drive-by comment on a patch review advocating that we slow
down ext4 to protect a single instance where the attacker has
read/write access to a mounted block device, when the file system
doesn't have generalized protections against that whole class of
attacks.... isn't particularly helpful.
By the way, I'm not aware of *any* company that has been interested in
funding work to protect against this class of attacks. Given that
most file system developers prefer food with their meals, and have
enough *other* unfunded mandates from our user community, it doesn't
seem likely that we're going to see much forward progress towards your
desires/interests.
Cheers,
- Ted
^ permalink raw reply
* Re: [BUG] ext4: BUG_ON in ext4_write_inline_data (fs/ext4/inline.c:240)
From: Demi Marie Obenour @ 2026-04-29 5:32 UTC (permalink / raw)
To: Theodore Tso
Cc: Zw Tang, Andreas Dilger, libaokun, jack, ojaswin, linux-ext4,
linux-kernel, yi.zhang, syzkaller-bugs, qubes-users,
Spectrum OS Discussion
In-Reply-To: <20260429044030.GB16497@macsyma-wired.lan>
[-- Attachment #1.1.1: Type: text/plain, Size: 4642 bytes --]
On 4/29/26 00:40, Theodore Tso wrote:
> On Tue, Apr 28, 2026 at 04:50:14PM -0400, Demi Marie Obenour wrote:
>> This is an example of a BadUSB attack, which has been known since
>> at least 2014. USB sticks *do* have their own microcontrollers to
>> run their firmware. At least in the past this firmware has been
>> programmable and not been digitally signed. This means that the USB
>> stick *can* be reprogrammed to perform a TOCTOU attack on a filesystem,
>> or indeed to implement a completely different kind of USB device.
>
> Honestly, if that's what you are worried about, then best solution is
> put epoxy in every single USB port. I've since financial institutions
> that have done precisely this, and both Android and Chrome OS supports
> enterprise security policies which does the equivalent in software.
That works well in enterprises using laptops, tablets, or mobile
devices. Enterprises can require that all devices have built-in,
non-USB input devices or touchscreens. Furthermore, corporate
environments use network-based backup and file sharing. So there
is hardly a need for USB except for security tokens, smart card
readers, OS installation media, and miscellaneous hardware devices
that generally will not be hotplugged. Only the third is a block
device, it is trusted, and it can be a USB stick with a physical
write-protect switch and signed firmware.
>> Protecting against replay attacks requires a Merkle tree. The only
>> Linux filesystems that I know have one are ZFS, bcachefs, and BTRFS.
>> The first two are out of tree and the third is not shipped in RHEL
>> at least.
>
> FYI, there was a patch for btrfs, but it was never landed. It's
> unclear how many people would be willing to pay the performance tax of
> using hmac-sha256 for every single data and metadata block write.
I thought BTRFS already had one. In any case corrupting an encrypted
disk will cause a checksum failure with fairly high probability,
as a 128-bit region is completely scrambled.
>> Of course, you are free to choose which (if any) of these attacks you
>> care about. One can that USB sticks should be mounted in userspace,
>> UEFI secure boot with Microsoft keys is irrelevant as long as
>> administrator -> kernel isn't a security boundary on Windows, and that
>> confidential computing only makes sense for stateless workloads (which
>> can use dm-verity) until there is a way to trust storage devices.
>> But it's always best to be aware that an attack vector exists,
>> whether or not one chooses to address it.
>
> Sure, but a drive-by comment on a patch review advocating that we slow
> down ext4 to protect a single instance where the attacker has
> read/write access to a mounted block device, when the file system
> doesn't have generalized protections against that whole class of
> attacks.... isn't particularly helpful.
My main goal is to point out that the attack vector does exist.
If nothing else, hopefully this will persuade distro maintainers
to switch to using libguestfs + FUSE as the default way to mount
USB drives. That isolates the driver in a VM.
Qubes OS users can already mount the device in a disposable VM,
and presumably many of them already do that. Again, this provides
strong isolation and severely limits the impact of an exploit.
I do wonder if this could be used against confidential computing
workloads. That said, work there would more likely be put into
allowing them to attest their storage.
> By the way, I'm not aware of *any* company that has been interested in
> funding work to protect against this class of attacks. Given that
> most file system developers prefer food with their meals, and have
> enough *other* unfunded mandates from our user community, it doesn't
> seem likely that we're going to see much forward progress towards your
> desires/interests.
I'm not surprised at all.
The people who would benefit the most from this work are consumers
who are running desktop Linux on general purpose computers they own.
I have spent most of my career providing secure solutions for these
people. I worked on Qubes OS for several years, and I now work
on Spectrum.
Unfortunately, this group has very little budget and so very little
market power. Therefore, work that benefits them is perpetually
and massively underfunded. Grants do exist, and crowdfunding might
also be an option. However, unless one is very passionate about the
client space, it is hard to resist much better-paying jobs in the
server world.
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* Re: [PATCH v3] generic/790: test post-EOF gap zeroing persistence
From: Brian Foster @ 2026-04-29 11:24 UTC (permalink / raw)
To: Zhang Yi
Cc: fstests, zlang, linux-ext4, linux-fsdevel, jack, yi.zhang,
yizhang089, yangerkun
In-Reply-To: <20260428085750.1072612-1-yi.zhang@huaweicloud.com>
On Tue, Apr 28, 2026 at 04:57:50PM +0800, Zhang Yi wrote:
> From: Zhang Yi <yi.zhang@huawei.com>
>
> Test that extending a file past a non-block-aligned EOF correctly
> zero-fills the gap [old_EOF, block_boundary), and that this zeroing
> persists through a filesystem shutdown+remount cycle.
>
> Stale data beyond EOF can persist on disk when append write data blocks
> are flushed before the on-disk file size update, or when concurrent
> append writeback and mmap writes persist non-zero data past EOF.
> Subsequent post-EOF operations (append write, fallocate, truncate up)
> must zero-fill and persist the gap to prevent exposing stale data.
>
> The test pollutes the file's last physical block (via FIEMAP + raw
> device write) with a sentinel pattern beyond i_size, then performs each
> extend operation and verifies the gap is zeroed both in memory and on
> disk.
>
> Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
> ---
Reviewed-by: Brian Foster <bfoster@redhat.com>
> v2->v3:
> - Add error check for the raw device pwrite, a failed pwrite would
> silently leave the test continuing with an unpolluted block,
> producing false-positive passes.
> - Add sync_range -a to wait until the extending I/O completes and to
> ensure file size update is persisted before shutdown, preventing
> unexpected file size errors.
> v1->v2:
> - Add _require_no_realtime to prevent testing on XFS realtime devices,
> where file data may reside on $SCRATCH_RTDEV.
> - Add _exclude_fs btrfs since FIEMAP returns logical addresses, not
> physical device offsets, writing to these offsets on $SCRATCH_DEV
> would corrupt the filesystem in multi-device setups. Besides, since
> btrfs doesn't support shutdown right now, we can support it later.
> - Add -v flag to od in _check_gap_zero() to prevent line folding of
> identical consecutive lines.
> - Add expected_new_sz parameter to _test_eof_zeroing(), verify file
> size was not rolled back after shutdown+remount cycle, and also drop
> the unnecessary file size check before the shutdown as well.
> - Clarify the comment regarding when stale data beyond EOF can persist.
>
> tests/generic/790 | 168 ++++++++++++++++++++++++++++++++++++++++++
> tests/generic/790.out | 4 +
> 2 files changed, 172 insertions(+)
> create mode 100755 tests/generic/790
> create mode 100644 tests/generic/790.out
>
> diff --git a/tests/generic/790 b/tests/generic/790
> new file mode 100755
> index 00000000..6daf3793
> --- /dev/null
> +++ b/tests/generic/790
> @@ -0,0 +1,168 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2026 Huawei. All Rights Reserved.
> +#
> +# FS QA Test No. 790
> +#
> +# Test that extending a file past a non-block-aligned EOF correctly zero-fills
> +# the gap [old_EOF, block_boundary), and that this zeroing persists through a
> +# filesystem shutdown+remount cycle.
> +#
> +# Stale data beyond EOF can persist on disk when:
> +# 1) append write data blocks are flushed before the on-disk file size update,
> +# and the system crashes in this window.
> +# 2) concurrent append writeback and mmap writes persist non-zero data past EOF.
> +#
> +# Subsequent post-EOF operations (append write, fallocate, truncate up) must
> +# zero-fill and persist the gap to prevent exposing stale data.
> +#
> +# The test pollutes the file's last physical block (via FIEMAP + raw device
> +# write) with a sentinel pattern beyond i_size, then performs each extend
> +# operation and verifies the gap is zeroed both in memory and on disk.
> +#
> +. ./common/preamble
> +_begin_fstest auto quick rw shutdown
> +
> +. ./common/filter
> +
> +_require_scratch
> +_require_block_device $SCRATCH_DEV
> +_require_no_realtime
> +_require_scratch_shutdown
> +_require_metadata_journaling $SCRATCH_DEV
> +
> +# FIEMAP on Btrfs returns logical addresses within the filesystem's address
> +# space, not physical device offsets. Writing to these offsets on $SCRATCH_DEV
> +# would corrupt the filesystem in multi-device setups.
> +_exclude_fs btrfs
> +
> +_require_xfs_io_command "fiemap"
> +_require_xfs_io_command "falloc"
> +_require_xfs_io_command "pwrite"
> +_require_xfs_io_command "truncate"
> +_require_xfs_io_command "sync_range"
> +
> +# Check that gap region [offset, offset+nbytes) is entirely zero
> +_check_gap_zero()
> +{
> + local file="$1"
> + local offset="$2"
> + local nbytes="$3"
> + local label="$4"
> + local data
> + local stripped
> +
> + data=$(od -A n -t x1 -v -j $offset -N $nbytes "$file" 2>/dev/null)
> +
> + # Remove whitespace and check if any byte is non-zero
> + stripped=$(printf '%s' "$data" | tr -d ' \n\t')
> + if [ -n "$stripped" ] && ! echo "$stripped" | grep -qE "^0+$"; then
> + echo "FAIL: non-zero data in gap [$offset,$((offset + nbytes))) $label"
> + _hexdump -N $((offset + nbytes)) "$file"
> + return 1
> + fi
> + return 0
> +}
> +
> +# Get the physical block offset (in bytes) of the file's first block on device
> +_get_phys_offset()
> +{
> + local file="$1"
> + local fiemap_output
> + local phys_blk
> +
> + fiemap_output=$($XFS_IO_PROG -r -c "fiemap -v" "$file" 2>/dev/null)
> + phys_blk=$(echo "$fiemap_output" | _filter_xfs_io_fiemap | head -1 | awk '{print $3}')
> + if [ -z "$phys_blk" ]; then
> + echo ""
> + return
> + fi
> + # Convert 512-byte blocks to bytes
> + echo $((phys_blk * 512))
> +}
> +
> +_test_eof_zeroing()
> +{
> + local test_name="$1"
> + local extend_cmd="$2"
> + local expected_new_sz="$3"
> + local file=$SCRATCH_MNT/testfile_${test_name}
> +
> + echo "$test_name" | tee -a $seqres.full
> +
> + # Compute non-block-aligned EOF offset
> + local gap_bytes=16
> + local eof_offset=$((blksz - gap_bytes))
> +
> + # Step 1: Write one full block to ensure the filesystem allocates a
> + # physical block for the file instead of using inline data.
> + $XFS_IO_PROG -f -c "pwrite -S 0x5a 0 $blksz" -c fsync \
> + "$file" >> $seqres.full 2>&1
> +
> + # Step 2: Get physical block offset on device via FIEMAP
> + local phys_offset
> + phys_offset=$(_get_phys_offset "$file")
> + if [ -z "$phys_offset" ]; then
> + _fail "$test_name: failed to get physical block offset via fiemap"
> + fi
> +
> + # Step 3: Truncate file to non-block-aligned size and fsync.
> + # The on-disk region [eof_offset, blksz) may or may not be
> + # zeroed by the filesystem at this point.
> + $XFS_IO_PROG -c "truncate $eof_offset" -c fsync \
> + "$file" >> $seqres.full 2>&1
> +
> + # Step 4: Unmount and restore the physical block to all-0x5a on disk.
> + # This bypasses the kernel's pagecache EOF-zeroing to ensure
> + # the stale pattern is present on disk. Then remount.
> + _scratch_unmount
> + $XFS_IO_PROG -d -c "pwrite -S 0x5a $phys_offset $blksz" \
> + $SCRATCH_DEV >> $seqres.full 2>&1
> + if [ $? -ne 0 ]; then
> + _fail "$test_name: failed to inject stale data on disk"
> + fi
> + _scratch_mount >> $seqres.full 2>&1
> +
> + # Step 5: Execute the extend operation.
> + $XFS_IO_PROG -c "$extend_cmd" "$file" >> $seqres.full 2>&1
> +
> + # Step 6: Verify gap [eof_offset, blksz) is zeroed BEFORE shutdown
> + _check_gap_zero "$file" $eof_offset $gap_bytes "before shutdown" || return 1
> +
> + # Step 7: Sync the extended range and shutdown the filesystem with
> + # journal flush. This persists the file size extending, and
> + # the filesystem should persist the zeroed data in the gap
> + # range as well.
> + if [ "$extend_cmd" != "${extend_cmd#pwrite}" ]; then
> + $XFS_IO_PROG -c "sync_range -w $blksz $blksz" \
> + -c "sync_range -a $blksz $blksz" \
> + "$file" >> $seqres.full 2>&1
> + fi
> + _scratch_shutdown -f
> +
> + # Step 8: Remount and verify gap is still zeroed
> + _scratch_cycle_mount
> +
> + # Verify file size was not rolled back after shutdown+remount
> + local sz
> + sz=$(stat -c %s "$file")
> + if [ "$sz" -ne "$expected_new_sz" ]; then
> + _fail "$test_name: file size rolled back after shutdown+remount: $sz != $expected_new_sz"
> + fi
> +
> + _check_gap_zero "$file" $eof_offset $gap_bytes "after shutdown+remount" || return 1
> +}
> +
> +_scratch_mkfs >> $seqres.full 2>&1
> +_scratch_mount
> +
> +blksz=$(_get_block_size $SCRATCH_MNT)
> +
> +# Test three variants of EOF-extending operations
> +_test_eof_zeroing "append_write" "pwrite -S 0x42 $blksz $blksz" $((blksz * 2))
> +_test_eof_zeroing "truncate_up" "truncate $((blksz * 2))" $((blksz * 2))
> +_test_eof_zeroing "fallocate" "falloc $blksz $blksz" $((blksz * 2))
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/790.out b/tests/generic/790.out
> new file mode 100644
> index 00000000..e5e2cc09
> --- /dev/null
> +++ b/tests/generic/790.out
> @@ -0,0 +1,4 @@
> +QA output created by 790
> +append_write
> +truncate_up
> +fallocate
> --
> 2.52.0
>
^ permalink raw reply
* [PATCHBLIZZARD v8] fuse/libfuse/e2fsprogs: faster file IO for containerized ext4 servers
From: Darrick J. Wong @ 2026-04-29 14:11 UTC (permalink / raw)
To: linux-fsdevel, linux-ext4, fuse-devel
Cc: Miklos Szeredi, Bernd Schubert, Joanne Koong, Theodore Ts'o,
Neal Gompa, Amir Goldstein, Christian Brauner
In-Reply-To: <20260223224617.GA2390314@frogsfrogsfrogs>
Hi everyone,
This is the eighth public draft of a prototype to connect the Linux
fuse driver to fs-iomap for regular file IO operations to and from files
whose contents persist to locally attached storage devices. With this
release, I show that it's possible to build a fuse server for a real
filesystem (ext4) that runs entirely in userspace yet maintains most of
its performance.
This effort is now separate from the one to run fuse servers in a
constrained environment via systemd. Putting fuse servers in a
container gets you all the blast radii reduction advantages and provides
a pathway to removing less popular filesystem drivers to reduce
maintenance work in the kernel; now we want trade relaxation of that
isolation for better performance.
The fuse command plumbing is very simple -- the ->iomap_begin,
->iomap_end, and iomap ->ioend calls within iomap are turned into
upcalls to the fuse server via a trio of new fuse commands. Pagecache
writeback is now a directio write. The fuse server can upsert mappings
into the kernel for cached access (== zero upcalls for rereads and pure
overwrites!) and the iomap cache revalidation code works.
At this stage I still get about 95% of the kernel ext4 driver's
streaming directio performance on streaming IO, and 110% of its
streaming buffered IO performance. Random buffered IO is about 85% as
fast as the kernel. Random direct IO is about 80% as fast as the
kernel; see the cover letter for the fuse2fs iomap changes for more
details. Unwritten extent conversions on random direct writes are
especially painful for fuse+iomap (~90% more overhead) due to upcall
overhead. And that's with (now dynamic) debugging turned on!
This series has been rebased to 7.1-rc1 since the seventh RFC, but it
has not otherwise changed much. Most changes happened in userspace:
1. I've written some example fuse-iomap servers, so I now have a vehicle
for testing that out of place writes works (they do) and that inline
data works.
2. Ted has started merging the very large quantity of fuse2fs
improvements into e2fsprogs.
3. I reordered the systemd service container patchset towards master
because the maintainer indicated that he wanted to merge it.
There are some questions remaining:
a. I would like to continue the discussion about how the design review
of this code should be structured, and how might I go about creating
new userspace filesystem servers -- lightweight new ones based off
the existing userspace tools? Or by merging lklfuse?
b. fuse2fs doesn't support the ext4 journal. Urk.
c. I've dropped the fstests and BPF parts of the patchbomb because v7
was just way too long. I'm also not including some extra
enhancements to fuse4fs, also for brevity.
I would like to get the main parts of this submission reviewed for 7.2
now that this has been collecting comments and tweaks in non-rfc status
for 5.5 months.
Kernel:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=fuse-service-container
libfuse:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/libfuse.git/log/?h=fuse-iomap-cache
e2fsprogs:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/e2fsprogs.git/log/?h=fuse4fs-memory-reclaim
fstests:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=fuse2fs
--Darrick
Unreviewed patches:
[PATCHSET v8 1/8] fuse: general bug fixes
[PATCH 3/4] fuse: update file mode when updating acls
[PATCH 4/4] fuse: propagate default and file acls on creation
[PATCHSET v8 2/8] iomap: cleanups ahead of adding fuse support
[PATCH 2/2] iomap: allow NULL swap info bdev when activating swapfile
[PATCHSET v8 3/8] fuse: cleanups ahead of adding fuse support
[PATCH 1/2] fuse: move the passthrough-specific code back to
[PATCHSET v8 4/8] fuse: allow servers to use iomap for better file IO
[PATCH 01/33] fuse: implement the basic iomap mechanisms
[PATCH 02/33] fuse_trace: implement the basic iomap mechanisms
[PATCH 03/33] fuse: make debugging configurable at runtime
[PATCH 04/33] fuse: adapt FUSE_DEV_IOC_BACKING_{OPEN,CLOSE} to add
[PATCH 05/33] fuse_trace: adapt FUSE_DEV_IOC_BACKING_{OPEN,CLOSE} to
[PATCH 06/33] fuse: enable SYNCFS and ensure we flush everything
[PATCH 07/33] fuse: clean up per-file type inode initialization
[PATCH 08/33] fuse: create a per-inode flag for setting exclusive
[PATCH 10/33] fuse_trace: create a per-inode flag for toggling iomap
[PATCH 11/33] fuse: isolate the other regular file IO paths from
[PATCH 12/33] fuse: implement basic iomap reporting such as FIEMAP
[PATCH 13/33] fuse_trace: implement basic iomap reporting such as
[PATCH 14/33] fuse: implement direct IO with iomap
[PATCH 15/33] fuse_trace: implement direct IO with iomap
[PATCH 16/33] fuse: implement buffered IO with iomap
[PATCH 17/33] fuse_trace: implement buffered IO with iomap
[PATCH 18/33] fuse: use an unrestricted backing device with iomap
[PATCH 20/33] fuse: advertise support for iomap
[PATCH 21/33] fuse: query filesystem geometry when using iomap
[PATCH 22/33] fuse_trace: query filesystem geometry when using iomap
[PATCH 23/33] fuse: implement fadvise for iomap files
[PATCH 24/33] fuse: invalidate ranges of block devices being used for
[PATCH 25/33] fuse_trace: invalidate ranges of block devices being
[PATCH 26/33] fuse: implement inline data file IO via iomap
[PATCH 27/33] fuse_trace: implement inline data file IO via iomap
[PATCH 28/33] fuse: allow more statx fields
[PATCH 29/33] fuse: support atomic writes with iomap
[PATCH 30/33] fuse_trace: support atomic writes with iomap
[PATCH 31/33] fuse: disable direct fs reclaim for any fuse server
[PATCH 32/33] fuse: enable swapfile activation on iomap
[PATCH 33/33] fuse: implement freeze and shutdowns for iomap
[PATCHSET v8 5/8] fuse: allow servers to specify root node id
[PATCH 1/3] fuse: make the root nodeid dynamic
[PATCH 2/3] fuse_trace: make the root nodeid dynamic
[PATCH 3/3] fuse: allow setting of root nodeid
[PATCHSET v8 6/8] fuse: handle timestamps and ACLs correctly when
[PATCH 1/9] fuse: enable caching of timestamps
[PATCH 2/9] fuse: force a ctime update after a fileattr_set call when
[PATCH 3/9] fuse: allow local filesystems to set some VFS iflags
[PATCH 4/9] fuse_trace: allow local filesystems to set some VFS
[PATCH 5/9] fuse: cache atime when in iomap mode
[PATCH 6/9] fuse: let the kernel handle KILL_SUID/KILL_SGID for iomap
[PATCH 7/9] fuse_trace: let the kernel handle KILL_SUID/KILL_SGID for
[PATCH 8/9] fuse: update ctime when updating acls on an iomap inode
[PATCH 9/9] fuse: always cache ACLs when using iomap
[PATCHSET v8 7/8] fuse: cache iomap mappings for even better file IO
[PATCH 01/12] fuse: cache iomaps
[PATCH 02/12] fuse_trace: cache iomaps
[PATCH 03/12] fuse: use the iomap cache for iomap_begin
[PATCH 04/12] fuse_trace: use the iomap cache for iomap_begin
[PATCH 05/12] fuse: invalidate iomap cache after file updates
[PATCH 06/12] fuse_trace: invalidate iomap cache after file updates
[PATCH 07/12] fuse: enable iomap cache management
[PATCH 08/12] fuse_trace: enable iomap cache management
[PATCH 09/12] fuse: overlay iomap inode info in struct fuse_inode
[PATCH 10/12] fuse: constrain iomap mapping cache size
[PATCH 11/12] fuse_trace: constrain iomap mapping cache size
[PATCH 12/12] fuse: enable iomap
[PATCHSET v8 8/8] fuse: run fuse servers as a contained service
[PATCH 1/2] fuse: allow privileged mount helpers to pre-approve iomap
[PATCH 2/2] fuse: set iomap backing device block size
[PATCHSET v8 1/6] libfuse: allow servers to use iomap for better file
[PATCH 01/25] libfuse: bump kernel and library ABI versions
[PATCH 02/25] libfuse: wait in do_destroy until all open files are
[PATCH 03/25] libfuse: add kernel gates for FUSE_IOMAP
[PATCH 04/25] libfuse: add fuse commands for iomap_begin and end
[PATCH 05/25] libfuse: add upper level iomap commands
[PATCH 06/25] libfuse: add a lowlevel notification to add a new
[PATCH 07/25] libfuse: add upper-level iomap add device function
[PATCH 08/25] libfuse: add iomap ioend low level handler
[PATCH 09/25] libfuse: add upper level iomap ioend commands
[PATCH 10/25] libfuse: add a reply function to send FUSE_ATTR_* to
[PATCH 11/25] libfuse: connect high level fuse library to
[PATCH 12/25] libfuse: support enabling exclusive mode for files
[PATCH 13/25] libfuse: support direct I/O through iomap
[PATCH 14/25] libfuse: don't allow hardlinking of iomap files in the
[PATCH 15/25] libfuse: allow discovery of the kernel's iomap
[PATCH 16/25] libfuse: add lower level iomap_config implementation
[PATCH 17/25] libfuse: add upper level iomap_config implementation
[PATCH 18/25] libfuse: add low level code to invalidate iomap block
[PATCH 19/25] libfuse: add upper-level API to invalidate parts of an
[PATCH 20/25] libfuse: add atomic write support
[PATCH 21/25] libfuse: allow disabling of fs memory reclaim and write
[PATCH 22/25] libfuse: create a helper to transform an open regular
[PATCH 23/25] libfuse: add swapfile support for iomap files
[PATCH 24/25] libfuse: add lower-level filesystem freeze, thaw,
[PATCH 25/25] libfuse: add upper-level filesystem freeze, thaw,
[PATCHSET v8 2/6] libfuse: allow servers to specify root node id
[PATCH 1/1] libfuse: allow root_nodeid mount option
[PATCHSET v8 3/6] libfuse: implement syncfs
[PATCH 1/2] libfuse: add strictatime/lazytime mount options
[PATCH 2/2] libfuse: set sync, immutable,
[PATCHSET v8 4/6] libfuse: add some service helper commands for iomap
[PATCH 1/3] mount_service: delegate iomap privilege from
[PATCH 2/3] libfuse: enable setting iomap block device block size
[PATCH 3/3] mount_service: create loop devices for regular files
[PATCHSET v8 5/6] fuse: add sample iomap fuse servers
[PATCH 1/7] example/iomap_ll: create a simple iomap server
[PATCH 2/7] example/iomap_ll: track block state
[PATCH 3/7] example/iomap_ll: implement atomic writes
[PATCH 4/7] example/iomap_inline_ll: create a simple server to test
[PATCH 5/7] example/iomap_ow_ll: create a simple iomap out of place
[PATCH 6/7] example/iomap_ow_ll: implement atomic writes
[PATCH 7/7] example/iomap_service_ll: create a sample systemd service
[PATCHSET v8 6/6] libfuse: cache iomap mappings for even better file
[PATCH 1/9] libfuse: enable iomap cache management for lowlevel fuse
[PATCH 2/9] libfuse: add upper-level iomap cache management
[PATCH 3/9] libfuse: allow constraining of iomap mapping cache size
[PATCH 4/9] libfuse: add upper-level iomap mapping cache constraint
[PATCH 5/9] libfuse: enable iomap
[PATCH 6/9] example/iomap_ll: cache mappings for later
[PATCH 7/9] example/iomap_inline_ll: cache iomappings in the kernel
[PATCH 8/9] example/iomap_ow_ll: cache iomappings in the kernel
[PATCH 9/9] example/iomap_service_ll: cache iomappings in the kernel
[PATCHSET v8 1/6] libext2fs: refactoring for fuse2fs iomap support
[PATCH 1/5] libext2fs: invalidate cached blocks when freeing them
[PATCH 2/5] libext2fs: only flush affected blocks in unix_write_byte
[PATCH 3/5] libext2fs: allow unix_write_byte when the write would be
[PATCH 4/5] libext2fs: allow clients to ask to write full superblocks
[PATCH 5/5] libext2fs: allow callers to disallow I/O to file data
[PATCHSET v8 2/6] fuse2fs: use fuse iomap data paths for better file
[PATCH 01/19] fuse2fs: implement bare minimum iomap for file mapping
[PATCH 02/19] fuse2fs: add iomap= mount option
[PATCH 03/19] fuse2fs: implement iomap configuration
[PATCH 04/19] fuse2fs: register block devices for use with iomap
[PATCH 05/19] fuse2fs: implement directio file reads
[PATCH 06/19] fuse2fs: add extent dump function for debugging
[PATCH 07/19] fuse2fs: implement direct write support
[PATCH 08/19] fuse2fs: turn on iomap for pagecache IO
[PATCH 09/19] fuse2fs: don't zero bytes in punch hole
[PATCH 10/19] fuse2fs: don't do file data block IO when iomap is
[PATCH 11/19] fuse2fs: try to create loop device when ext4 device is
[PATCH 12/19] fuse2fs: enable file IO to inline data files
[PATCH 13/19] fuse2fs: set iomap-related inode flags
[PATCH 14/19] fuse2fs: configure block device block size
[PATCH 15/19] fuse4fs: separate invalidation
[PATCH 16/19] fuse2fs: implement statx
[PATCH 17/19] fuse2fs: enable atomic writes
[PATCH 18/19] fuse4fs: disable fs reclaim and write throttling
[PATCH 19/19] fuse2fs: implement freeze and shutdown requests
[PATCHSET v8 3/6] fuse4fs: adapt iomap for fuse services
[PATCH 1/3] fuse4fs: configure iomap when running as a service
[PATCH 2/3] fuse4fs: set iomap backing device blocksize
[PATCH 3/3] fuse4fs: ask for loop devices when opening via
[PATCHSET v8 4/6] fuse4fs: specify the root node id
[PATCH 1/1] fuse4fs: don't use inode number translation when possible
[PATCHSET v8 5/6] fuse2fs: handle timestamps and ACLs correctly when
[PATCH 01/10] fuse2fs: add strictatime/lazytime mount options
[PATCH 02/10] fuse2fs: skip permission checking on utimens when iomap
[PATCH 03/10] fuse2fs: let the kernel tell us about acl/mode updates
[PATCH 04/10] fuse2fs: better debugging for file mode updates
[PATCH 05/10] fuse2fs: debug timestamp updates
[PATCH 06/10] fuse2fs: use coarse timestamps for iomap mode
[PATCH 07/10] fuse2fs: add tracing for retrieving timestamps
[PATCH 08/10] fuse2fs: enable syncfs
[PATCH 09/10] fuse2fs: set sync, immutable,
[PATCH 10/10] fuse4fs: increase attribute timeout in iomap mode
[PATCHSET v8 6/6] fuse2fs: cache iomap mappings for even better file
[PATCH 1/4] fuse2fs: enable caching of iomaps
[PATCH 2/4] fuse2fs: constrain iomap mapping cache size
[PATCH 3/4] fuse4fs: upsert first file mapping to kernel on open
[PATCH 4/4] fuse2fs: enable iomap
^ permalink raw reply
* [PATCHBLIZZARD v8] fuse/libfuse/e2fsprogs: faster file IO for containerized ext4 servers
From: Darrick J. Wong @ 2026-04-29 14:12 UTC (permalink / raw)
To: linux-fsdevel, linux-ext4, fuse-devel
Cc: Miklos Szeredi, Bernd Schubert, Joanne Koong, Theodore Ts'o,
Neal Gompa, Amir Goldstein, Christian Brauner
[let's send this as a separate thread]
Hi everyone,
This is the eighth public draft of a prototype to connect the Linux
fuse driver to fs-iomap for regular file IO operations to and from files
whose contents persist to locally attached storage devices. With this
release, I show that it's possible to build a fuse server for a real
filesystem (ext4) that runs entirely in userspace yet maintains most of
its performance.
This effort is now separate from the one to run fuse servers in a
constrained environment via systemd. Putting fuse servers in a
container gets you all the blast radii reduction advantages and provides
a pathway to removing less popular filesystem drivers to reduce
maintenance work in the kernel; now we want trade relaxation of that
isolation for better performance.
The fuse command plumbing is very simple -- the ->iomap_begin,
->iomap_end, and iomap ->ioend calls within iomap are turned into
upcalls to the fuse server via a trio of new fuse commands. Pagecache
writeback is now a directio write. The fuse server can upsert mappings
into the kernel for cached access (== zero upcalls for rereads and pure
overwrites!) and the iomap cache revalidation code works.
At this stage I still get about 95% of the kernel ext4 driver's
streaming directio performance on streaming IO, and 110% of its
streaming buffered IO performance. Random buffered IO is about 85% as
fast as the kernel. Random direct IO is about 80% as fast as the
kernel; see the cover letter for the fuse2fs iomap changes for more
details. Unwritten extent conversions on random direct writes are
especially painful for fuse+iomap (~90% more overhead) due to upcall
overhead. And that's with (now dynamic) debugging turned on!
This series has been rebased to 7.1-rc1 since the seventh RFC, but it
has not otherwise changed much. Most changes happened in userspace
this time:
1. I've written some example fuse-iomap servers, so I now have a vehicle
for testing that out of place writes works (they do) and that inline
data works.
2. Ted has started merging the very large quantity of fuse2fs
improvements into e2fsprogs.
3. I reordered the systemd service container patchset towards master
because the maintainer indicated that he wanted to merge it.
There are some questions remaining:
a. I would like to continue the discussion about how the design review
of this code should be structured, and how might I go about creating
new userspace filesystem servers -- lightweight new ones based off
the existing userspace tools? Or by merging lklfuse?
b. fuse2fs doesn't support the ext4 journal. Urk.
c. I've dropped the fstests and BPF parts of the patchbomb because v7
was just way too long. I'm also not including some extra
enhancements to fuse4fs, also for brevity.
I would like to get the main parts of this submission reviewed for 7.2
now that this has been collecting comments and tweaks in non-rfc status
for 5.5 months.
Kernel:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=fuse-service-container
libfuse:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/libfuse.git/log/?h=fuse-iomap-cache
e2fsprogs:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/e2fsprogs.git/log/?h=fuse4fs-memory-reclaim
fstests:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=fuse2fs
--Darrick
Unreviewed patches:
[PATCHSET v8 1/8] fuse: general bug fixes
[PATCH 3/4] fuse: update file mode when updating acls
[PATCH 4/4] fuse: propagate default and file acls on creation
[PATCHSET v8 2/8] iomap: cleanups ahead of adding fuse support
[PATCH 2/2] iomap: allow NULL swap info bdev when activating swapfile
[PATCHSET v8 3/8] fuse: cleanups ahead of adding fuse support
[PATCH 1/2] fuse: move the passthrough-specific code back to
[PATCHSET v8 4/8] fuse: allow servers to use iomap for better file IO
[PATCH 01/33] fuse: implement the basic iomap mechanisms
[PATCH 02/33] fuse_trace: implement the basic iomap mechanisms
[PATCH 03/33] fuse: make debugging configurable at runtime
[PATCH 04/33] fuse: adapt FUSE_DEV_IOC_BACKING_{OPEN,CLOSE} to add
[PATCH 05/33] fuse_trace: adapt FUSE_DEV_IOC_BACKING_{OPEN,CLOSE} to
[PATCH 06/33] fuse: enable SYNCFS and ensure we flush everything
[PATCH 07/33] fuse: clean up per-file type inode initialization
[PATCH 08/33] fuse: create a per-inode flag for setting exclusive
[PATCH 10/33] fuse_trace: create a per-inode flag for toggling iomap
[PATCH 11/33] fuse: isolate the other regular file IO paths from
[PATCH 12/33] fuse: implement basic iomap reporting such as FIEMAP
[PATCH 13/33] fuse_trace: implement basic iomap reporting such as
[PATCH 14/33] fuse: implement direct IO with iomap
[PATCH 15/33] fuse_trace: implement direct IO with iomap
[PATCH 16/33] fuse: implement buffered IO with iomap
[PATCH 17/33] fuse_trace: implement buffered IO with iomap
[PATCH 18/33] fuse: use an unrestricted backing device with iomap
[PATCH 20/33] fuse: advertise support for iomap
[PATCH 21/33] fuse: query filesystem geometry when using iomap
[PATCH 22/33] fuse_trace: query filesystem geometry when using iomap
[PATCH 23/33] fuse: implement fadvise for iomap files
[PATCH 24/33] fuse: invalidate ranges of block devices being used for
[PATCH 25/33] fuse_trace: invalidate ranges of block devices being
[PATCH 26/33] fuse: implement inline data file IO via iomap
[PATCH 27/33] fuse_trace: implement inline data file IO via iomap
[PATCH 28/33] fuse: allow more statx fields
[PATCH 29/33] fuse: support atomic writes with iomap
[PATCH 30/33] fuse_trace: support atomic writes with iomap
[PATCH 31/33] fuse: disable direct fs reclaim for any fuse server
[PATCH 32/33] fuse: enable swapfile activation on iomap
[PATCH 33/33] fuse: implement freeze and shutdowns for iomap
[PATCHSET v8 5/8] fuse: allow servers to specify root node id
[PATCH 1/3] fuse: make the root nodeid dynamic
[PATCH 2/3] fuse_trace: make the root nodeid dynamic
[PATCH 3/3] fuse: allow setting of root nodeid
[PATCHSET v8 6/8] fuse: handle timestamps and ACLs correctly when
[PATCH 1/9] fuse: enable caching of timestamps
[PATCH 2/9] fuse: force a ctime update after a fileattr_set call when
[PATCH 3/9] fuse: allow local filesystems to set some VFS iflags
[PATCH 4/9] fuse_trace: allow local filesystems to set some VFS
[PATCH 5/9] fuse: cache atime when in iomap mode
[PATCH 6/9] fuse: let the kernel handle KILL_SUID/KILL_SGID for iomap
[PATCH 7/9] fuse_trace: let the kernel handle KILL_SUID/KILL_SGID for
[PATCH 8/9] fuse: update ctime when updating acls on an iomap inode
[PATCH 9/9] fuse: always cache ACLs when using iomap
[PATCHSET v8 7/8] fuse: cache iomap mappings for even better file IO
[PATCH 01/12] fuse: cache iomaps
[PATCH 02/12] fuse_trace: cache iomaps
[PATCH 03/12] fuse: use the iomap cache for iomap_begin
[PATCH 04/12] fuse_trace: use the iomap cache for iomap_begin
[PATCH 05/12] fuse: invalidate iomap cache after file updates
[PATCH 06/12] fuse_trace: invalidate iomap cache after file updates
[PATCH 07/12] fuse: enable iomap cache management
[PATCH 08/12] fuse_trace: enable iomap cache management
[PATCH 09/12] fuse: overlay iomap inode info in struct fuse_inode
[PATCH 10/12] fuse: constrain iomap mapping cache size
[PATCH 11/12] fuse_trace: constrain iomap mapping cache size
[PATCH 12/12] fuse: enable iomap
[PATCHSET v8 8/8] fuse: run fuse servers as a contained service
[PATCH 1/2] fuse: allow privileged mount helpers to pre-approve iomap
[PATCH 2/2] fuse: set iomap backing device block size
[PATCHSET v8 1/6] libfuse: allow servers to use iomap for better file
[PATCH 01/25] libfuse: bump kernel and library ABI versions
[PATCH 02/25] libfuse: wait in do_destroy until all open files are
[PATCH 03/25] libfuse: add kernel gates for FUSE_IOMAP
[PATCH 04/25] libfuse: add fuse commands for iomap_begin and end
[PATCH 05/25] libfuse: add upper level iomap commands
[PATCH 06/25] libfuse: add a lowlevel notification to add a new
[PATCH 07/25] libfuse: add upper-level iomap add device function
[PATCH 08/25] libfuse: add iomap ioend low level handler
[PATCH 09/25] libfuse: add upper level iomap ioend commands
[PATCH 10/25] libfuse: add a reply function to send FUSE_ATTR_* to
[PATCH 11/25] libfuse: connect high level fuse library to
[PATCH 12/25] libfuse: support enabling exclusive mode for files
[PATCH 13/25] libfuse: support direct I/O through iomap
[PATCH 14/25] libfuse: don't allow hardlinking of iomap files in the
[PATCH 15/25] libfuse: allow discovery of the kernel's iomap
[PATCH 16/25] libfuse: add lower level iomap_config implementation
[PATCH 17/25] libfuse: add upper level iomap_config implementation
[PATCH 18/25] libfuse: add low level code to invalidate iomap block
[PATCH 19/25] libfuse: add upper-level API to invalidate parts of an
[PATCH 20/25] libfuse: add atomic write support
[PATCH 21/25] libfuse: allow disabling of fs memory reclaim and write
[PATCH 22/25] libfuse: create a helper to transform an open regular
[PATCH 23/25] libfuse: add swapfile support for iomap files
[PATCH 24/25] libfuse: add lower-level filesystem freeze, thaw,
[PATCH 25/25] libfuse: add upper-level filesystem freeze, thaw,
[PATCHSET v8 2/6] libfuse: allow servers to specify root node id
[PATCH 1/1] libfuse: allow root_nodeid mount option
[PATCHSET v8 3/6] libfuse: implement syncfs
[PATCH 1/2] libfuse: add strictatime/lazytime mount options
[PATCH 2/2] libfuse: set sync, immutable,
[PATCHSET v8 4/6] libfuse: add some service helper commands for iomap
[PATCH 1/3] mount_service: delegate iomap privilege from
[PATCH 2/3] libfuse: enable setting iomap block device block size
[PATCH 3/3] mount_service: create loop devices for regular files
[PATCHSET v8 5/6] fuse: add sample iomap fuse servers
[PATCH 1/7] example/iomap_ll: create a simple iomap server
[PATCH 2/7] example/iomap_ll: track block state
[PATCH 3/7] example/iomap_ll: implement atomic writes
[PATCH 4/7] example/iomap_inline_ll: create a simple server to test
[PATCH 5/7] example/iomap_ow_ll: create a simple iomap out of place
[PATCH 6/7] example/iomap_ow_ll: implement atomic writes
[PATCH 7/7] example/iomap_service_ll: create a sample systemd service
[PATCHSET v8 6/6] libfuse: cache iomap mappings for even better file
[PATCH 1/9] libfuse: enable iomap cache management for lowlevel fuse
[PATCH 2/9] libfuse: add upper-level iomap cache management
[PATCH 3/9] libfuse: allow constraining of iomap mapping cache size
[PATCH 4/9] libfuse: add upper-level iomap mapping cache constraint
[PATCH 5/9] libfuse: enable iomap
[PATCH 6/9] example/iomap_ll: cache mappings for later
[PATCH 7/9] example/iomap_inline_ll: cache iomappings in the kernel
[PATCH 8/9] example/iomap_ow_ll: cache iomappings in the kernel
[PATCH 9/9] example/iomap_service_ll: cache iomappings in the kernel
[PATCHSET v8 1/6] libext2fs: refactoring for fuse2fs iomap support
[PATCH 1/5] libext2fs: invalidate cached blocks when freeing them
[PATCH 2/5] libext2fs: only flush affected blocks in unix_write_byte
[PATCH 3/5] libext2fs: allow unix_write_byte when the write would be
[PATCH 4/5] libext2fs: allow clients to ask to write full superblocks
[PATCH 5/5] libext2fs: allow callers to disallow I/O to file data
[PATCHSET v8 2/6] fuse2fs: use fuse iomap data paths for better file
[PATCH 01/19] fuse2fs: implement bare minimum iomap for file mapping
[PATCH 02/19] fuse2fs: add iomap= mount option
[PATCH 03/19] fuse2fs: implement iomap configuration
[PATCH 04/19] fuse2fs: register block devices for use with iomap
[PATCH 05/19] fuse2fs: implement directio file reads
[PATCH 06/19] fuse2fs: add extent dump function for debugging
[PATCH 07/19] fuse2fs: implement direct write support
[PATCH 08/19] fuse2fs: turn on iomap for pagecache IO
[PATCH 09/19] fuse2fs: don't zero bytes in punch hole
[PATCH 10/19] fuse2fs: don't do file data block IO when iomap is
[PATCH 11/19] fuse2fs: try to create loop device when ext4 device is
[PATCH 12/19] fuse2fs: enable file IO to inline data files
[PATCH 13/19] fuse2fs: set iomap-related inode flags
[PATCH 14/19] fuse2fs: configure block device block size
[PATCH 15/19] fuse4fs: separate invalidation
[PATCH 16/19] fuse2fs: implement statx
[PATCH 17/19] fuse2fs: enable atomic writes
[PATCH 18/19] fuse4fs: disable fs reclaim and write throttling
[PATCH 19/19] fuse2fs: implement freeze and shutdown requests
[PATCHSET v8 3/6] fuse4fs: adapt iomap for fuse services
[PATCH 1/3] fuse4fs: configure iomap when running as a service
[PATCH 2/3] fuse4fs: set iomap backing device blocksize
[PATCH 3/3] fuse4fs: ask for loop devices when opening via
[PATCHSET v8 4/6] fuse4fs: specify the root node id
[PATCH 1/1] fuse4fs: don't use inode number translation when possible
[PATCHSET v8 5/6] fuse2fs: handle timestamps and ACLs correctly when
[PATCH 01/10] fuse2fs: add strictatime/lazytime mount options
[PATCH 02/10] fuse2fs: skip permission checking on utimens when iomap
[PATCH 03/10] fuse2fs: let the kernel tell us about acl/mode updates
[PATCH 04/10] fuse2fs: better debugging for file mode updates
[PATCH 05/10] fuse2fs: debug timestamp updates
[PATCH 06/10] fuse2fs: use coarse timestamps for iomap mode
[PATCH 07/10] fuse2fs: add tracing for retrieving timestamps
[PATCH 08/10] fuse2fs: enable syncfs
[PATCH 09/10] fuse2fs: set sync, immutable,
[PATCH 10/10] fuse4fs: increase attribute timeout in iomap mode
[PATCHSET v8 6/6] fuse2fs: cache iomap mappings for even better file
[PATCH 1/4] fuse2fs: enable caching of iomaps
[PATCH 2/4] fuse2fs: constrain iomap mapping cache size
[PATCH 3/4] fuse4fs: upsert first file mapping to kernel on open
[PATCH 4/4] fuse2fs: enable iomap
^ permalink raw reply
* [PATCHSET v8 1/6] libext2fs: refactoring for fuse2fs iomap support
From: Darrick J. Wong @ 2026-04-29 14:20 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <20260429141253.GQ7739@frogsfrogsfrogs>
Hi all,
In preparation for connecting fuse, iomap, and fuse2fs for a much more
performant file IO path, make some changes to the Unix IO manager in
libext2fs so that we can have better IO. First we start by making
filesystem flushes a lot more efficient by eliding fsyncs when they're
not necessary, and allowing library clients to turn off the racy code
that writes the superblock byte by byte but exposes stale checksums.
XXX: The second part of this series adds IO tagging so that we could tag
IOs by inode number to distinguish file data blocks in cache from
everything else. This is temporary scaffolding whilst we're in the
middle adding directio and later buffered writes. Once we can use the
pagecache for all file IO activity I think we could drop the back half
of this series.
If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.
Comments and questions are, as always, welcome.
e2fsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/e2fsprogs.git/log/?h=libext2fs-iomap-prep
---
Commits in this patchset:
* libext2fs: invalidate cached blocks when freeing them
* libext2fs: only flush affected blocks in unix_write_byte
* libext2fs: allow unix_write_byte when the write would be aligned
* libext2fs: allow clients to ask to write full superblocks
* libext2fs: allow callers to disallow I/O to file data blocks
---
lib/ext2fs/ext2_io.h | 8 ++++++-
lib/ext2fs/ext2fs.h | 4 +++
debian/libext2fs2t64.symbols | 1 +
lib/ext2fs/alloc_stats.c | 6 +++++
lib/ext2fs/closefs.c | 7 ++++++
lib/ext2fs/fileio.c | 12 +++++++++-
lib/ext2fs/io_manager.c | 9 +++++++
lib/ext2fs/unix_io.c | 51 ++++++++++++++++++++++++++++++++++++++++--
8 files changed, 93 insertions(+), 5 deletions(-)
^ permalink raw reply
* [PATCHSET v8 2/6] fuse2fs: use fuse iomap data paths for better file I/O performance
From: Darrick J. Wong @ 2026-04-29 14:20 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <20260429141253.GQ7739@frogsfrogsfrogs>
Hi all,
Switch fuse2fs to use the new iomap file data IO paths instead of
pushing it very slowly through the /dev/fuse connection. For local
filesystems, all we have to do is respond to requests for file to device
mappings; the rest of the IO hot path stays within the kernel. This
means that we can get rid of all file data block processing within
fuse2fs.
Because we're not pinning dirty pages through a potentially slow network
connection, we don't need the heavy BDI throttling for which most fuse
servers have become infamous. Yes, mapping lookups for writeback can
stall, but mappings are small as compared to data and this situation
exists for all kernel filesystems as well.
The performance of this new data path is quite stunning: on a warm
system, streaming reads and writes through the pagecache go from
60-90MB/s to 2-2.5GB/s. Direct IO reads and writes improve from the
same baseline to 2.5-8GB/s. FIEMAP and SEEK_DATA/SEEK_HOLE now work
too. The kernel ext4 driver can manage about 1.6GB/s for pagecache IO
and about 2.6-8.5GB/s, which means that fuse2fs is about as fast as the
kernel for streaming file IO.
Random 4k buffered IO is not so good: plain fuse2fs pokes along at
25-50MB/s, whereas fuse2fs with iomap manages 90-1300MB/s. The kernel
can do 900-1300MB/s. Random directio is worse: plain fuse2fs does
20-30MB/s, fuse-iomap does about 30-35MB/s, and the kernel does
40-55MB/s. I suspect that metadata heavy workloads do not perform well
on fuse2fs because libext2fs wasn't designed for that and it doesn't
even have a journal to absorb all the fsync writes. We also probably
need iomap caching really badly.
These performance numbers are slanted: my machine is 12 years old, and
fuse2fs is VERY poorly optimized for performance. It contains a single
Big Filesystem Lock which nukes multi-threaded scalability. There's no
inode cache nor is there a proper buffer cache, which means that fuse2fs
reads metadata in from disk and checksums it on EVERY ACCESS. Sad!
Despite these gaps, this RFC demonstrates that it's feasible to run the
metadata parsing parts of a filesystem in userspace while not
sacrificing much performance. We now have a vehicle to move the
filesystems out of the kernel, where they can be containerized so that
malicious filesystems can be contained, somewhat.
iomap mode also calls FUSE_DESTROY before unmounting the filesystem, so
for capable systems, fuse2fs doesn't need to run in fuseblk mode
anymore.
However, there are some major warts remaining:
1. The iomap cookie validation is not present, which can lead to subtle
races between pagecache zeroing and writeback on filesystems that
support unwritten and delalloc mappings.
2. Mappings ought to be cached in the kernel for more speed.
3. iomap doesn't support things like fscrypt or fsverity, and I haven't
yet figured out how inline data is supposed to work.
4. I would like to be able to turn on fuse+iomap on a per-inode basis,
which currently isn't possible because the kernel fuse driver will iget
inodes prior to calling FUSE_GETATTR to discover the properties of the
inode it just read.
5. ext4 doesn't support out of place writes so I don't know if that
actually works correctly.
6. iomap is an inode-based service, not a file-based service. This
means that we /must/ push ext2's inode numbers into the kernel via
FUSE_GETATTR so that it can report those same numbers back out through
the FUSE_IOMAP_* calls. However, the fuse kernel uses a separate nodeid
to index its incore inode, so we have to pass those too so that
notifications work properly.
If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.
Comments and questions are, as always, welcome.
e2fsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/e2fsprogs.git/log/?h=fuse2fs-iomap-fileio
---
Commits in this patchset:
* fuse2fs: implement bare minimum iomap for file mapping reporting
* fuse2fs: add iomap= mount option
* fuse2fs: implement iomap configuration
* fuse2fs: register block devices for use with iomap
* fuse2fs: implement directio file reads
* fuse2fs: add extent dump function for debugging
* fuse2fs: implement direct write support
* fuse2fs: turn on iomap for pagecache IO
* fuse2fs: don't zero bytes in punch hole
* fuse2fs: don't do file data block IO when iomap is enabled
* fuse2fs: try to create loop device when ext4 device is a regular file
* fuse2fs: enable file IO to inline data files
* fuse2fs: set iomap-related inode flags
* fuse2fs: configure block device block size
* fuse4fs: separate invalidation
* fuse2fs: implement statx
* fuse2fs: enable atomic writes
* fuse4fs: disable fs reclaim and write throttling
* fuse2fs: implement freeze and shutdown requests
---
configure | 90 ++
configure.ac | 54 +
fuse4fs/fuse4fs.1.in | 6
fuse4fs/fuse4fs.c | 1934 +++++++++++++++++++++++++++++++++++++++++++++++++-
lib/config.h.in | 6
misc/fuse2fs.1.in | 6
misc/fuse2fs.c | 1947 ++++++++++++++++++++++++++++++++++++++++++++++++++
7 files changed, 4016 insertions(+), 27 deletions(-)
^ permalink raw reply
* [PATCHSET v8 3/6] fuse4fs: adapt iomap for fuse services
From: Darrick J. Wong @ 2026-04-29 14:20 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <20260429141253.GQ7739@frogsfrogsfrogs>
Hi all,
This series adapts the iomap code to work in systemd service mode.
If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.
Comments and questions are, as always, welcome.
e2fsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/e2fsprogs.git/log/?h=fuse4fs-iomap-service
---
Commits in this patchset:
* fuse4fs: configure iomap when running as a service
* fuse4fs: set iomap backing device blocksize
* fuse4fs: ask for loop devices when opening via fuservicemount
---
fuse4fs/fuse4fs.c | 94 +++++++++++++++++++++++++++++++++++++++++++++++------
1 file changed, 83 insertions(+), 11 deletions(-)
^ permalink raw reply
* [PATCHSET v8 4/6] fuse4fs: specify the root node id
From: Darrick J. Wong @ 2026-04-29 14:21 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <20260429141253.GQ7739@frogsfrogsfrogs>
Hi all,
This series adapts fuse4fs to have a 1:1 mapping of ext2_ino_t to fuse_ino_t
for slightly better performance and less confusing code interpretation.
If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.
Comments and questions are, as always, welcome.
e2fsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/e2fsprogs.git/log/?h=fuse2fs-root-nodeid
---
Commits in this patchset:
* fuse4fs: don't use inode number translation when possible
---
fuse4fs/fuse4fs.c | 30 ++++++++++++++++++++++++------
1 file changed, 24 insertions(+), 6 deletions(-)
^ permalink raw reply
* [PATCHSET v8 5/6] fuse2fs: handle timestamps and ACLs correctly when iomap is enabled
From: Darrick J. Wong @ 2026-04-29 14:21 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <20260429141253.GQ7739@frogsfrogsfrogs>
Hi all,
When iomap is enabled for a fuse file, we try to keep as much of the
file IO path in the kernel as we possibly can. That means no calling
out to the fuse server in the IO path when we can avoid it. However,
the existing FUSE architecture defers all file attributes to the fuse
server -- [cm]time updates, ACL metadata management, set[ug]id removal,
and permissions checking thereof, etc.
We'd really rather do all these attribute updates in the kernel, and
only push them to the fuse server when it's actually necessary (e.g.
fsync). Furthermore, the POSIX ACL code has the weird behavior that if
the access ACL can be represented entirely by i_mode bits, it will
change the mode and delete the ACL, which fuse servers generally don't
seem to implement.
IOWs, we want consistent and correct (as defined by fstests) behavior
of file attributes in iomap mode. Let's make the kernel manage all that
and push the results to userspace as needed. This improves performance
even further, since it's sort of like writeback_cache mode but more
aggressive.
If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.
Comments and questions are, as always, welcome.
e2fsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/e2fsprogs.git/log/?h=fuse2fs-iomap-attrs
---
Commits in this patchset:
* fuse2fs: add strictatime/lazytime mount options
* fuse2fs: skip permission checking on utimens when iomap is enabled
* fuse2fs: let the kernel tell us about acl/mode updates
* fuse2fs: better debugging for file mode updates
* fuse2fs: debug timestamp updates
* fuse2fs: use coarse timestamps for iomap mode
* fuse2fs: add tracing for retrieving timestamps
* fuse2fs: enable syncfs
* fuse2fs: set sync, immutable, and append at file load time
* fuse4fs: increase attribute timeout in iomap mode
---
fuse4fs/fuse4fs.1.in | 6 +
fuse4fs/fuse4fs.c | 226 ++++++++++++++++++++++++++++++----------
misc/fuse2fs.1.in | 6 +
misc/fuse2fs.c | 282 +++++++++++++++++++++++++++++++++++++-------------
4 files changed, 389 insertions(+), 131 deletions(-)
^ permalink raw reply
* [PATCHSET v8 6/6] fuse2fs: cache iomap mappings for even better file IO performance
From: Darrick J. Wong @ 2026-04-29 14:21 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <20260429141253.GQ7739@frogsfrogsfrogs>
Hi all,
This series improves the performance (and correctness for some
filesystems) by adding the ability to cache iomap mappings in the
kernel. For filesystems that can change mapping states during pagecache
writeback (e.g. unwritten extent conversion) this is absolutely
necessary to deal with races with writes to the pagecache because
writeback does not take i_rwsem. For everyone else, it simply
eliminates roundtrips to userspace.
If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.
Comments and questions are, as always, welcome.
e2fsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/e2fsprogs.git/log/?h=fuse2fs-iomap-cache
---
Commits in this patchset:
* fuse2fs: enable caching of iomaps
* fuse2fs: constrain iomap mapping cache size
* fuse4fs: upsert first file mapping to kernel on open
* fuse2fs: enable iomap
---
fuse4fs/fuse4fs.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++-----
misc/fuse2fs.c | 38 ++++++++++++++++++++++-----
2 files changed, 101 insertions(+), 13 deletions(-)
^ permalink raw reply
* [PATCH 1/5] libext2fs: invalidate cached blocks when freeing them
From: Darrick J. Wong @ 2026-04-29 14:51 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214224.4107228.16300103064218258692.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
When we're freeing blocks, we should tell the IO manager to drop them
from any cache it might be maintaining to improve performance.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
lib/ext2fs/ext2_io.h | 8 +++++++-
debian/libext2fs2t64.symbols | 1 +
lib/ext2fs/alloc_stats.c | 6 ++++++
lib/ext2fs/io_manager.c | 9 +++++++++
lib/ext2fs/unix_io.c | 35 +++++++++++++++++++++++++++++++++++
5 files changed, 58 insertions(+), 1 deletion(-)
diff --git a/lib/ext2fs/ext2_io.h b/lib/ext2fs/ext2_io.h
index c880ea2524f248..0148492caf63b6 100644
--- a/lib/ext2fs/ext2_io.h
+++ b/lib/ext2fs/ext2_io.h
@@ -104,7 +104,10 @@ struct struct_io_manager {
unsigned long long count);
errcode_t (*flock)(io_channel channel, unsigned int flock_flags);
errcode_t (*get_fd)(io_channel channel, int *fd);
- long reserved[12];
+ errcode_t (*invalidate_blocks)(io_channel channel,
+ unsigned long long block,
+ unsigned long long count);
+ long reserved[11];
};
#define IO_FLAG_RW 0x0001
@@ -157,6 +160,9 @@ extern errcode_t io_channel_cache_readahead(io_channel io,
extern errcode_t io_channel_flock(io_channel io, unsigned int flock_flags);
extern errcode_t io_channel_funlock(io_channel io);
extern errcode_t io_channel_get_fd(io_channel io, int *fd);
+extern errcode_t io_channel_invalidate_blocks(io_channel io,
+ unsigned long long block,
+ unsigned long long count);
#ifdef _WIN32
/* windows_io.c */
diff --git a/debian/libext2fs2t64.symbols b/debian/libext2fs2t64.symbols
index 555fbbb0c98878..b19a362967f00e 100644
--- a/debian/libext2fs2t64.symbols
+++ b/debian/libext2fs2t64.symbols
@@ -702,6 +702,7 @@ libext2fs.so.2 libext2fs2t64 #MINVER#
io_channel_flock@Base 1.47.99
io_channel_funlock@Base 1.47.99
io_channel_get_fd@Base 1.47.99
+ io_channel_invalidate_blocks@Base 1.47.99
io_channel_read_blk64@Base 1.41.1
io_channel_set_options@Base 1.37
io_channel_write_blk64@Base 1.41.1
diff --git a/lib/ext2fs/alloc_stats.c b/lib/ext2fs/alloc_stats.c
index 95a6438f252e0f..68bbe6807a8ed3 100644
--- a/lib/ext2fs/alloc_stats.c
+++ b/lib/ext2fs/alloc_stats.c
@@ -82,6 +82,9 @@ void ext2fs_block_alloc_stats2(ext2_filsys fs, blk64_t blk, int inuse)
-inuse * (blk64_t) EXT2FS_CLUSTER_RATIO(fs));
ext2fs_mark_super_dirty(fs);
ext2fs_mark_bb_dirty(fs);
+ if (inuse < 0)
+ io_channel_invalidate_blocks(fs->io, blk,
+ EXT2FS_CLUSTER_RATIO(fs));
if (fs->block_alloc_stats)
(fs->block_alloc_stats)(fs, (blk64_t) blk, inuse);
}
@@ -144,11 +147,14 @@ void ext2fs_block_alloc_stats_range(ext2_filsys fs, blk64_t blk,
ext2fs_bg_flags_clear(fs, group, EXT2_BG_BLOCK_UNINIT);
ext2fs_group_desc_csum_set(fs, group);
ext2fs_free_blocks_count_add(fs->super, -inuse * (blk64_t) n);
+
blk += n;
num -= n;
}
ext2fs_mark_super_dirty(fs);
ext2fs_mark_bb_dirty(fs);
+ if (inuse < 0)
+ io_channel_invalidate_blocks(fs->io, orig_blk, orig_num);
if (fs->block_alloc_stats_range)
(fs->block_alloc_stats_range)(fs, orig_blk, orig_num, inuse);
}
diff --git a/lib/ext2fs/io_manager.c b/lib/ext2fs/io_manager.c
index dff3d73552827f..a92dba7b9dc880 100644
--- a/lib/ext2fs/io_manager.c
+++ b/lib/ext2fs/io_manager.c
@@ -174,3 +174,12 @@ errcode_t io_channel_get_fd(io_channel io, int *fd)
return io->manager->get_fd(io, fd);
}
+
+errcode_t io_channel_invalidate_blocks(io_channel io, unsigned long long block,
+ unsigned long long count)
+{
+ if (!io->manager->invalidate_blocks)
+ return EXT2_ET_OP_NOT_SUPPORTED;
+
+ return io->manager->invalidate_blocks(io, block, count);
+}
diff --git a/lib/ext2fs/unix_io.c b/lib/ext2fs/unix_io.c
index 567bbd9493f7f1..54bd4b5597ea9e 100644
--- a/lib/ext2fs/unix_io.c
+++ b/lib/ext2fs/unix_io.c
@@ -672,6 +672,25 @@ static errcode_t reuse_cache(io_channel channel,
#define FLUSH_INVALIDATE 0x01
#define FLUSH_NOLOCK 0x02
+/* Remove blocks from the cache. Dirty contents are discarded. */
+static void invalidate_cached_blocks(io_channel channel,
+ struct unix_private_data *data,
+ unsigned long long block,
+ unsigned long long count)
+{
+ struct unix_cache *cache;
+ int i;
+
+ mutex_lock(data, CACHE_MTX);
+ for (i = 0, cache = data->cache; i < data->cache_size; i++, cache++) {
+ if (!cache->in_use || cache->block < block ||
+ cache->block >= block + count)
+ continue;
+ cache->in_use = 0;
+ }
+ mutex_unlock(data, CACHE_MTX);
+}
+
/*
* Flush all of the blocks in the cache
*/
@@ -1832,6 +1851,20 @@ static errcode_t unix_get_fd(io_channel channel, int *fd)
return 0;
}
+static errcode_t unix_invalidate_blocks(io_channel channel,
+ unsigned long long block,
+ unsigned long long count)
+{
+ struct unix_private_data *data;
+
+ EXT2_CHECK_MAGIC(channel, EXT2_ET_MAGIC_IO_CHANNEL);
+ data = (struct unix_private_data *) channel->private_data;
+ EXT2_CHECK_MAGIC(data, EXT2_ET_MAGIC_UNIX_IO_CHANNEL);
+
+ invalidate_cached_blocks(channel, data, block, count);
+ return 0;
+}
+
#if __GNUC_PREREQ (4, 6)
#pragma GCC diagnostic pop
#endif
@@ -1855,6 +1888,7 @@ static struct struct_io_manager struct_unix_manager = {
.zeroout = unix_zeroout,
.flock = unix_flock,
.get_fd = unix_get_fd,
+ .invalidate_blocks = unix_invalidate_blocks,
};
io_manager unix_io_manager = &struct_unix_manager;
@@ -1878,6 +1912,7 @@ static struct struct_io_manager struct_unixfd_manager = {
.zeroout = unix_zeroout,
.flock = unix_flock,
.get_fd = unix_get_fd,
+ .invalidate_blocks = unix_invalidate_blocks,
};
io_manager unixfd_io_manager = &struct_unixfd_manager;
^ permalink raw reply related
* [PATCH 2/5] libext2fs: only flush affected blocks in unix_write_byte
From: Darrick J. Wong @ 2026-04-29 14:51 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214224.4107228.16300103064218258692.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
There's no need to invalidate the entire cache when writing a range of
bytes to the device. The only ones we need to invalidate are the ones
that we're writing separately.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
lib/ext2fs/unix_io.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/lib/ext2fs/unix_io.c b/lib/ext2fs/unix_io.c
index 54bd4b5597ea9e..35c42c35f735a3 100644
--- a/lib/ext2fs/unix_io.c
+++ b/lib/ext2fs/unix_io.c
@@ -1588,6 +1588,7 @@ static errcode_t unix_write_byte(io_channel channel, unsigned long offset,
{
struct unix_private_data *data;
errcode_t retval = 0;
+ unsigned long long bno, nbno;
ssize_t actual;
EXT2_CHECK_MAGIC(channel, EXT2_ET_MAGIC_IO_CHANNEL);
@@ -1603,10 +1604,17 @@ static errcode_t unix_write_byte(io_channel channel, unsigned long offset,
#ifndef NO_IO_CACHE
/*
- * Flush out the cache completely
+ * Flush all the dirty blocks, then invalidate the blocks we're about
+ * to write.
*/
- if ((retval = flush_cached_blocks(channel, data, FLUSH_INVALIDATE)))
+ retval = flush_cached_blocks(channel, data, 0);
+ if (retval)
return retval;
+
+ bno = offset / channel->block_size;
+ nbno = (offset + size + channel->block_size - 1) / channel->block_size;
+
+ invalidate_cached_blocks(channel, data, bno, nbno - bno);
#endif
if (lseek(data->dev, offset + data->offset, SEEK_SET) < 0)
^ permalink raw reply related
* [PATCH 3/5] libext2fs: allow unix_write_byte when the write would be aligned
From: Darrick J. Wong @ 2026-04-29 14:52 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214224.4107228.16300103064218258692.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
If someone calls write_byte on an IO channel with an alignment
requirement and the range to be written is aligned correctly, go ahead
and do the write. This will be needed later when we try to speed up
superblock writes.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
lib/ext2fs/unix_io.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/lib/ext2fs/unix_io.c b/lib/ext2fs/unix_io.c
index 35c42c35f735a3..ea8ee56b7d5163 100644
--- a/lib/ext2fs/unix_io.c
+++ b/lib/ext2fs/unix_io.c
@@ -1599,7 +1599,9 @@ static errcode_t unix_write_byte(io_channel channel, unsigned long offset,
#ifdef ALIGN_DEBUG
printf("unix_write_byte: O_DIRECT fallback\n");
#endif
- return EXT2_ET_UNIMPLEMENTED;
+ if (!IS_ALIGNED(data->offset + offset, channel->align) ||
+ !IS_ALIGNED(data->offset + offset + size, channel->align))
+ return EXT2_ET_UNIMPLEMENTED;
}
#ifndef NO_IO_CACHE
^ permalink raw reply related
* [PATCH 4/5] libext2fs: allow clients to ask to write full superblocks
From: Darrick J. Wong @ 2026-04-29 14:52 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214224.4107228.16300103064218258692.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
write_primary_superblock currently does this weird dance where it will
try to write only the dirty bytes of the primary superblock to disk. In
theory, this is done so that tune2fs can incrementally update superblock
bytes when the filesystem is mounted; ext2 was famous for allowing using
this dance to set new fs parameters and have them take effect in real
time.
The ability to do this safely was obliterated back in 2001 when ext3 was
introduced with journalling, because tune2fs has no way to know if the
journal has already logged an updated primary superblock but not yet
written it to disk, which means that they can race to write, and changes
can be lost.
This (non-)safety was further obliterated back in 2012 when I added
checksums to all the metadata blocks in ext4 because anyone else with
the block device open can see the primary superblock in an intermediate
state where the checksum does not match the superblock contents.
At this point in 2025 it's kind of stupid for fuse2fs to be doing this
because you can't have the kernel and fuse2fs mount the same filesystem
at the same time. It also makes fuse2fs op_fsync slow because libext2fs
performs a bunch of small writes and introduce extra fsyncs.
So, add a new flag to ask for full superblock writes, which fuse2fs will
use later.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
lib/ext2fs/ext2fs.h | 1 +
lib/ext2fs/closefs.c | 7 +++++++
2 files changed, 8 insertions(+)
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 02c3cbcea92482..8fad4c4011dd5a 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -220,6 +220,7 @@ typedef struct ext2_file *ext2_file_t;
#define EXT2_FLAG_IBITMAP_TAIL_PROBLEM 0x2000000
#define EXT2_FLAG_THREADS 0x4000000
#define EXT2_FLAG_IGNORE_SWAP_DIRENT 0x8000000
+#define EXT2_FLAG_WRITE_FULL_SUPER 0x10000000
/*
* Internal flags for use by the ext2fs library only
diff --git a/lib/ext2fs/closefs.c b/lib/ext2fs/closefs.c
index 8e5bec03a050de..9a67db76e7b326 100644
--- a/lib/ext2fs/closefs.c
+++ b/lib/ext2fs/closefs.c
@@ -196,6 +196,13 @@ static errcode_t write_primary_superblock(ext2_filsys fs,
int check_idx, write_idx, size;
errcode_t retval;
+ if (fs->flags & EXT2_FLAG_WRITE_FULL_SUPER) {
+ retval = io_channel_write_byte(fs->io, SUPERBLOCK_OFFSET,
+ SUPERBLOCK_SIZE, super);
+ if (!retval)
+ return 0;
+ }
+
if (!fs->io->manager->write_byte || !fs->orig_super) {
fallback:
io_channel_set_blksize(fs->io, SUPERBLOCK_OFFSET);
^ permalink raw reply related
* [PATCH 5/5] libext2fs: allow callers to disallow I/O to file data blocks
From: Darrick J. Wong @ 2026-04-29 14:52 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214224.4107228.16300103064218258692.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
Add a flag to ext2_file_t to disallow read and write I/O to file data
blocks. This supports fuse2fs iomap support, which will keep all the
file data I/O inside the kerne.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
lib/ext2fs/ext2fs.h | 3 +++
lib/ext2fs/fileio.c | 12 +++++++++++-
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 8fad4c4011dd5a..23b0695a32d150 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -178,6 +178,9 @@ typedef struct ext2_struct_dblist *ext2_dblist;
#define EXT2_FILE_WRITE 0x0001
#define EXT2_FILE_CREATE 0x0002
+/* no file I/O to disk blocks, only to inline data */
+#define EXT2_FILE_NOBLOCKIO 0x0004
+
#define EXT2_FILE_MASK 0x00FF
#define EXT2_FILE_BUF_DIRTY 0x4000
diff --git a/lib/ext2fs/fileio.c b/lib/ext2fs/fileio.c
index 3a36e9e7fff43b..95ee45ec7371ae 100644
--- a/lib/ext2fs/fileio.c
+++ b/lib/ext2fs/fileio.c
@@ -314,6 +314,11 @@ errcode_t ext2fs_file_read(ext2_file_t file, void *buf,
if (file->inode.i_flags & EXT4_INLINE_DATA_FL)
return ext2fs_file_read_inline_data(file, buf, wanted, got);
+ if (file->flags & EXT2_FILE_NOBLOCKIO) {
+ retval = EXT2_ET_OP_NOT_SUPPORTED;
+ goto fail;
+ }
+
while ((file->pos < EXT2_I_SIZE(&file->inode)) && (wanted > 0)) {
retval = sync_buffer_position(file);
if (retval)
@@ -441,6 +446,11 @@ errcode_t ext2fs_file_write(ext2_file_t file, const void *buf,
retval = 0;
}
+ if (file->flags & EXT2_FILE_NOBLOCKIO) {
+ retval = EXT2_ET_OP_NOT_SUPPORTED;
+ goto fail;
+ }
+
while (nbytes > 0) {
retval = sync_buffer_position(file);
if (retval)
@@ -609,7 +619,7 @@ static errcode_t ext2fs_file_zero_past_offset(ext2_file_t file,
int ret_flags;
errcode_t retval;
- if (off == 0)
+ if (off == 0 || (file->flags & EXT2_FILE_NOBLOCKIO))
return 0;
retval = sync_buffer_position(file);
^ permalink raw reply related
* [PATCH 01/19] fuse2fs: implement bare minimum iomap for file mapping reporting
From: Darrick J. Wong @ 2026-04-29 14:52 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214459.4107473.9520724883867588970.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
Add enough of an iomap implementation that we can do FIEMAP and
SEEK_DATA and SEEK_HOLE.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
configure | 49 +++++
configure.ac | 31 +++
fuse4fs/fuse4fs.c | 540 +++++++++++++++++++++++++++++++++++++++++++++++++++++
lib/config.h.in | 3
misc/fuse2fs.c | 540 +++++++++++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 1161 insertions(+), 2 deletions(-)
diff --git a/configure b/configure
index 80aad505da550c..344c7af2ee48f8 100755
--- a/configure
+++ b/configure
@@ -14608,6 +14608,7 @@ printf "%s\n" "yes" >&6; }
fi
+have_fuse_iomap=
if test -n "$FUSE_LIB"
then
FUSE_USE_VERSION=319
@@ -14634,12 +14635,60 @@ esac
fi
done
+
+ { printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for iomap_begin in libfuse" >&5
+printf %s "checking for iomap_begin in libfuse... " >&6; }
+ cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+ #define _GNU_SOURCE
+ #define _FILE_OFFSET_BITS 64
+ #define FUSE_USE_VERSION 399
+ #include <fuse.h>
+
+int
+main (void)
+{
+
+ struct fuse_operations fs_ops = {
+ .iomap_begin = NULL,
+ .iomap_end = NULL,
+ };
+ struct fuse_file_iomap narf = { };
+
+ ;
+ return 0;
+}
+
+_ACEOF
+if ac_fn_c_try_link "$LINENO"
+then :
+ have_fuse_iomap=yes
+ { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+printf "%s\n" "yes" >&6; }
+else case e in #(
+ e) { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: no" >&5
+printf "%s\n" "no" >&6; } ;;
+esac
+fi
+rm -f core conftest.err conftest.$ac_objext conftest.beam \
+ conftest$ac_exeext conftest.$ac_ext
+ if test "$have_fuse_iomap" = yes
+ then
+ FUSE_USE_VERSION=399
+ fi
fi
if test -n "$FUSE_USE_VERSION"
then
printf "%s\n" "#define FUSE_USE_VERSION $FUSE_USE_VERSION" >>confdefs.h
+fi
+if test -n "$have_fuse_iomap"
+then
+
+printf "%s\n" "#define HAVE_FUSE_IOMAP 1" >>confdefs.h
+
fi
have_fuse_lowlevel=
diff --git a/configure.ac b/configure.ac
index 63a5cd697a6dde..8d85e9966877ea 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1385,6 +1385,7 @@ dnl
dnl Set FUSE_USE_VERSION, which is how fuse servers build against a particular
dnl libfuse ABI. Currently we link against the libfuse 3.19 ABI (hence 319)
dnl
+have_fuse_iomap=
if test -n "$FUSE_LIB"
then
FUSE_USE_VERSION=319
@@ -1394,12 +1395,42 @@ then
[AC_MSG_FAILURE([Cannot build against fuse3 headers])],
[#define _FILE_OFFSET_BITS 64
#define FUSE_USE_VERSION 319])
+
+ dnl
+ dnl Check if the fuse library supports iomap, which requires a higher
+ dnl FUSE_USE_VERSION ABI version (3.99)
+ dnl
+ AC_MSG_CHECKING(for iomap_begin in libfuse)
+ AC_LINK_IFELSE(
+ [ AC_LANG_PROGRAM([[
+ #define _GNU_SOURCE
+ #define _FILE_OFFSET_BITS 64
+ #define FUSE_USE_VERSION 399
+ #include <fuse.h>
+ ]], [[
+ struct fuse_operations fs_ops = {
+ .iomap_begin = NULL,
+ .iomap_end = NULL,
+ };
+ struct fuse_file_iomap narf = { };
+ ]])
+ ], have_fuse_iomap=yes
+ AC_MSG_RESULT(yes),
+ AC_MSG_RESULT(no))
+ if test "$have_fuse_iomap" = yes
+ then
+ FUSE_USE_VERSION=399
+ fi
fi
if test -n "$FUSE_USE_VERSION"
then
AC_DEFINE_UNQUOTED(FUSE_USE_VERSION, $FUSE_USE_VERSION,
[Define to the version of FUSE to use])
fi
+if test -n "$have_fuse_iomap"
+then
+ AC_DEFINE(HAVE_FUSE_IOMAP, 1, [Define to 1 if fuse supports iomap])
+fi
dnl
dnl Check if the FUSE lowlevel library is supported
diff --git a/fuse4fs/fuse4fs.c b/fuse4fs/fuse4fs.c
index dc5a0ede9f5072..a159024f778ba2 100644
--- a/fuse4fs/fuse4fs.c
+++ b/fuse4fs/fuse4fs.c
@@ -155,6 +155,9 @@ static inline uint64_t round_down(uint64_t b, unsigned int align)
return b - m;
}
+#define max(a, b) ((a) > (b) ? (a) : (b))
+#define min(a, b) ((a) < (b) ? (a) : (b))
+
#define dbg_printf(fuse4fs, format, ...) \
while ((fuse4fs)->debug) { \
printf("FUSE4FS (%s): tid=%llu " format, (fuse4fs)->shortdev, get_thread_id(), ##__VA_ARGS__); \
@@ -233,6 +236,14 @@ enum fuse4fs_opstate {
F4OP_SHUTDOWN,
};
+#ifdef HAVE_FUSE_IOMAP
+enum fuse4fs_iomap_state {
+ IOMAP_DISABLED,
+ IOMAP_UNKNOWN,
+ IOMAP_ENABLED,
+};
+#endif
+
/* Main program context */
#define FUSE4FS_MAGIC (0xEF53DEADUL)
struct fuse4fs {
@@ -260,6 +271,9 @@ struct fuse4fs {
int logfd;
int blocklog;
int oom_score_adj;
+#ifdef HAVE_FUSE_IOMAP
+ enum fuse4fs_iomap_state iomap_state;
+#endif
unsigned int blockmask;
unsigned long offset;
unsigned int next_generation;
@@ -882,6 +896,15 @@ fuse4fs_set_handle(struct fuse_file_info *fp, struct fuse4fs_file_handle *fh)
fp->keep_cache = 1;
}
+#ifdef HAVE_FUSE_IOMAP
+static inline int fuse4fs_iomap_enabled(const struct fuse4fs *ff)
+{
+ return ff->iomap_state >= IOMAP_ENABLED;
+}
+#else
+# define fuse4fs_iomap_enabled(...) (0)
+#endif
+
static void get_now(struct timespec *now)
{
#ifdef CLOCK_REALTIME
@@ -1514,7 +1537,7 @@ static errcode_t fuse4fs_open(struct fuse4fs *ff)
char options[128];
double deadline;
int flags = EXT2_FLAG_64BITS | EXT2_FLAG_THREADS | EXT2_FLAG_RW |
- EXT2_FLAG_EXCLUSIVE;
+ EXT2_FLAG_EXCLUSIVE | EXT2_FLAG_WRITE_FULL_SUPER;
errcode_t err;
if (ff->lockfile) {
@@ -1808,6 +1831,15 @@ static void op_destroy(void *userdata)
(stats->cache_hits + stats->cache_misses));
}
+ /*
+ * If we're mounting in iomap mode, we need to unmount in op_destroy so
+ * that the block device will be released before umount(2) returns.
+ */
+ if (ff->iomap_state == IOMAP_ENABLED) {
+ fuse4fs_mmp_cancel(ff);
+ fuse4fs_unmount(ff);
+ }
+
fuse4fs_finish(ff, 0);
}
@@ -1948,6 +1980,44 @@ static inline int fuse_set_feature_flag(struct fuse_conn_info *conn,
}
#endif
+#ifdef HAVE_FUSE_IOMAP
+static inline bool fuse4fs_wants_iomap(struct fuse4fs *ff)
+{
+ if (ff->iomap_state == IOMAP_DISABLED)
+ return false;
+
+ /* iomap only works with block devices */
+ if (!(ff->fs->io->flags & CHANNEL_FLAGS_BLOCK_DEVICE))
+ return false;
+
+ /*
+ * iomap addrs must be aligned to the bdev lba size; we use fs
+ * blocksize as a proxy here
+ */
+ if (ff->offset % ff->fs->blocksize > 0)
+ return false;
+
+ return true;
+}
+
+static void fuse4fs_iomap_enable(struct fuse_conn_info *conn,
+ struct fuse4fs *ff)
+{
+ /* Don't let anyone touch iomap until the end of the patchset. */
+ ff->iomap_state = IOMAP_DISABLED;
+ return;
+
+ if (fuse4fs_wants_iomap(ff) &&
+ fuse_set_feature_flag(conn, FUSE_CAP_IOMAP))
+ ff->iomap_state = IOMAP_ENABLED;
+
+ if (ff->iomap_state == IOMAP_UNKNOWN)
+ ff->iomap_state = IOMAP_DISABLED;
+}
+#else
+# define fuse4fs_iomap_enable(...) ((void)0)
+#endif
+
static void op_init(void *userdata, struct fuse_conn_info *conn)
{
struct fuse4fs *ff = userdata;
@@ -1970,6 +2040,7 @@ static void op_init(void *userdata, struct fuse_conn_info *conn)
#ifdef FUSE_CAP_NO_EXPORT_SUPPORT
fuse_set_feature_flag(conn, FUSE_CAP_NO_EXPORT_SUPPORT);
#endif
+ fuse4fs_iomap_enable(conn, ff);
conn->time_gran = 1;
if (ff->opstate == F4OP_WRITABLE)
@@ -5917,6 +5988,466 @@ static void op_fallocate(fuse_req_t req, fuse_ino_t fino EXT2FS_ATTR((unused)),
}
#endif /* SUPPORT_FALLOCATE */
+#ifdef HAVE_FUSE_IOMAP
+static void fuse4fs_iomap_hole(struct fuse4fs *ff, struct fuse_file_iomap *iomap,
+ off_t pos, uint64_t count)
+{
+ iomap->dev = FUSE_IOMAP_DEV_NULL;
+ iomap->addr = FUSE_IOMAP_NULL_ADDR;
+ iomap->offset = pos;
+ iomap->length = count;
+ iomap->type = FUSE_IOMAP_TYPE_HOLE;
+}
+
+static void fuse4fs_iomap_hole_to_eof(struct fuse4fs *ff,
+ struct fuse_file_iomap *iomap, off_t pos,
+ off_t count,
+ const struct ext2_inode_large *inode)
+{
+ ext2_filsys fs = ff->fs;
+ uint64_t isize = EXT2_I_SIZE(inode);
+
+ /*
+ * We have to be careful about handling a hole to the right of the
+ * entire mapping tree. First, the mapping must start and end on a
+ * block boundary because they must be aligned to at least an LBA for
+ * the block layer; and to the fsblock for smoother operation.
+ *
+ * As for the length -- we could return a mapping all the way to
+ * i_size, but i_size could be less than pos/count if we're zeroing the
+ * EOF block in anticipation of a truncate operation. Similarly, we
+ * don't want to end the mapping at pos+count because we know there's
+ * nothing mapped beyond here.
+ */
+ uint64_t startoff = round_down(pos, fs->blocksize);
+ uint64_t eofoff = round_up(max(pos + count, isize), fs->blocksize);
+
+ dbg_printf(ff,
+ "pos=0x%llx count=0x%llx isize=0x%llx startoff=0x%llx eofoff=0x%llx\n",
+ (unsigned long long)pos,
+ (unsigned long long)count,
+ (unsigned long long)isize,
+ (unsigned long long)startoff,
+ (unsigned long long)eofoff);
+
+ fuse4fs_iomap_hole(ff, iomap, startoff, eofoff - startoff);
+}
+
+#define DEBUG_IOMAP
+#ifdef DEBUG_IOMAP
+# define __DUMP_EXTENT(ff, func, tag, startoff, err, extent) \
+ do { \
+ dbg_printf((ff), \
+ "%s: %s startoff 0x%llx err %ld lblk 0x%llx pblk 0x%llx len 0x%x flags 0x%x\n", \
+ (func), (tag), (startoff), (err), (extent)->e_lblk, \
+ (extent)->e_pblk, (extent)->e_len, \
+ (extent)->e_flags & EXT2_EXTENT_FLAGS_UNINIT); \
+ } while(0)
+# define DUMP_EXTENT(ff, tag, startoff, err, extent) \
+ __DUMP_EXTENT((ff), __func__, (tag), (startoff), (err), (extent))
+
+# define __DUMP_INFO(ff, func, tag, startoff, err, info) \
+ do { \
+ dbg_printf((ff), \
+ "%s: %s startoff 0x%llx err %ld entry %d/%d/%d level %d/%d\n", \
+ (func), (tag), (startoff), (err), \
+ (info)->curr_entry, (info)->num_entries, \
+ (info)->max_entries, (info)->curr_level, \
+ (info)->max_depth); \
+ } while(0)
+# define DUMP_INFO(ff, tag, startoff, err, info) \
+ __DUMP_INFO((ff), __func__, (tag), (startoff), (err), (info))
+#else
+# define __DUMP_EXTENT(...) ((void)0)
+# define DUMP_EXTENT(...) ((void)0)
+# define DUMP_INFO(...) ((void)0)
+#endif
+
+static inline errcode_t __fuse4fs_get_mapping_at(struct fuse4fs *ff,
+ ext2_extent_handle_t handle,
+ blk64_t startoff,
+ struct ext2fs_extent *bmap,
+ const char *func)
+{
+ errcode_t err;
+
+ /*
+ * Find the file mapping at startoff. We don't check the return value
+ * of _goto because _get will error out if _goto failed. There's a
+ * subtlety to the outcome of _goto when startoff falls in a sparse
+ * hole however:
+ *
+ * Most of the time, _goto points the cursor at the mapping whose lblk
+ * is just to the left of startoff. The mapping may or may not overlap
+ * startoff; this is ok. In other words, the tree lookup behaves as if
+ * we asked it to use a less than or equals comparison.
+ *
+ * However, if startoff is to the left of the first mapping in the
+ * extent tree, _goto points the cursor at that first mapping because
+ * it doesn't know how to deal with this situation. In this case,
+ * the tree lookup behaves as if we asked it to use a greater than
+ * or equals comparison.
+ *
+ * Note: If _get() returns 'no current node', that means that there
+ * aren't any mappings at all.
+ */
+ ext2fs_extent_goto(handle, startoff);
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT, bmap);
+ __DUMP_EXTENT(ff, func, "lookup", startoff, err, bmap);
+ if (err == EXT2_ET_NO_CURRENT_NODE)
+ err = EXT2_ET_EXTENT_NOT_FOUND;
+ return err;
+}
+
+static inline errcode_t __fuse4fs_get_next_mapping(struct fuse4fs *ff,
+ ext2_extent_handle_t handle,
+ blk64_t startoff,
+ struct ext2fs_extent *bmap,
+ const char *func)
+{
+ struct ext2fs_extent newex;
+ struct ext2_extent_info info;
+ errcode_t err;
+
+ /*
+ * The extent tree code has this (probably broken) behavior that if
+ * more than two of the highest levels of the cursor point at the
+ * rightmost edge of an extent tree block, a _NEXT_LEAF movement fails
+ * to move the cursor position of any of the lower levels. IOWs, if
+ * leaf level N is at the right edge, it will only advance level N-1
+ * to the right. If N-1 was at the right edge, the cursor resets to
+ * record 0 of that level and goes down to the wrong leaf.
+ *
+ * Work around this by walking up (towards root level 0) the extent
+ * tree until we find a level where we're not already at the rightmost
+ * edge. The _NEXT_LEAF movement will walk down the tree to find the
+ * leaves.
+ */
+ err = ext2fs_extent_get_info(handle, &info);
+ DUMP_INFO(ff, "UP?", startoff, err, &info);
+ if (err)
+ return err;
+
+ while (info.curr_entry == info.num_entries && info.curr_level > 0) {
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_UP, &newex);
+ DUMP_EXTENT(ff, "UP", startoff, err, &newex);
+ if (err)
+ return err;
+ err = ext2fs_extent_get_info(handle, &info);
+ DUMP_INFO(ff, "UP", startoff, err, &info);
+ if (err)
+ return err;
+ }
+
+ /*
+ * If we're at the root and there are no more entries, there's nothing
+ * else to be found.
+ */
+ if (info.curr_level == 0 && info.curr_entry == info.num_entries)
+ return EXT2_ET_EXTENT_NOT_FOUND;
+
+ /* Otherwise grab this next leaf and return it. */
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT_LEAF, &newex);
+ DUMP_EXTENT(ff, "NEXT", startoff, err, &newex);
+ if (err)
+ return err;
+
+ *bmap = newex;
+ return 0;
+}
+
+#define fuse4fs_get_mapping_at(ff, handle, startoff, bmap) \
+ __fuse4fs_get_mapping_at((ff), (handle), (startoff), (bmap), __func__)
+#define fuse4fs_get_next_mapping(ff, handle, startoff, bmap) \
+ __fuse4fs_get_next_mapping((ff), (handle), (startoff), (bmap), __func__)
+
+static errcode_t fuse4fs_iomap_begin_extent(struct fuse4fs *ff, uint64_t ino,
+ struct ext2_inode_large *inode,
+ off_t pos, uint64_t count,
+ uint32_t opflags,
+ struct fuse_file_iomap *iomap)
+{
+ ext2_extent_handle_t handle;
+ struct ext2fs_extent extent = { };
+ ext2_filsys fs = ff->fs;
+ const blk64_t startoff = FUSE4FS_B_TO_FSBT(ff, pos);
+ errcode_t err;
+ int ret = 0;
+
+ err = ext2fs_extent_open2(fs, ino, EXT2_INODE(inode), &handle);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = fuse4fs_get_mapping_at(ff, handle, startoff, &extent);
+ if (err == EXT2_ET_EXTENT_NOT_FOUND) {
+ /* No mappings at all; the whole range is a hole. */
+ fuse4fs_iomap_hole_to_eof(ff, iomap, pos, count, inode);
+ goto out_handle;
+ }
+ if (err) {
+ ret = translate_error(fs, ino, err);
+ goto out_handle;
+ }
+
+ if (startoff < extent.e_lblk) {
+ /*
+ * Mapping starts to the right of the current position.
+ * Synthesize a hole going to that next extent.
+ */
+ fuse4fs_iomap_hole(ff, iomap, FUSE4FS_FSB_TO_B(ff, startoff),
+ FUSE4FS_FSB_TO_B(ff, extent.e_lblk - startoff));
+ goto out_handle;
+ }
+
+ if (startoff >= extent.e_lblk + extent.e_len) {
+ /*
+ * Mapping ends to the left of the current position. Try to
+ * find the next mapping. If there is no next mapping, the
+ * whole range is in a hole.
+ */
+ err = fuse4fs_get_next_mapping(ff, handle, startoff, &extent);
+ if (err == EXT2_ET_EXTENT_NOT_FOUND) {
+ fuse4fs_iomap_hole_to_eof(ff, iomap, pos, count, inode);
+ goto out_handle;
+ }
+
+ /*
+ * If the new mapping starts to the right of startoff, there's
+ * a hole from startoff to the start of the new mapping.
+ */
+ if (startoff < extent.e_lblk) {
+ fuse4fs_iomap_hole(ff, iomap,
+ FUSE4FS_FSB_TO_B(ff, startoff),
+ FUSE4FS_FSB_TO_B(ff, extent.e_lblk - startoff));
+ goto out_handle;
+ }
+
+ /*
+ * The new mapping starts at startoff. Something weird
+ * happened in the extent tree lookup, but we found a valid
+ * mapping so we'll run with it.
+ */
+ }
+
+ /* Mapping overlaps startoff, report this. */
+ iomap->dev = FUSE_IOMAP_DEV_NULL;
+ iomap->addr = FUSE4FS_FSB_TO_B(ff, extent.e_pblk) + ff->offset;
+ iomap->offset = FUSE4FS_FSB_TO_B(ff, extent.e_lblk);
+ iomap->length = FUSE4FS_FSB_TO_B(ff, extent.e_len);
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT)
+ iomap->type = FUSE_IOMAP_TYPE_UNWRITTEN;
+ else
+ iomap->type = FUSE_IOMAP_TYPE_MAPPED;
+
+out_handle:
+ ext2fs_extent_free(handle);
+ return ret;
+}
+
+static int fuse4fs_iomap_begin_indirect(struct fuse4fs *ff, uint64_t ino,
+ struct ext2_inode_large *inode,
+ off_t pos, uint64_t count,
+ uint32_t opflags,
+ struct fuse_file_iomap *iomap)
+{
+ ext2_filsys fs = ff->fs;
+ blk64_t startoff = FUSE4FS_B_TO_FSBT(ff, pos);
+ uint64_t isize = EXT2_I_SIZE(inode);
+ uint64_t real_count = min(count, 131072);
+ const blk64_t endoff = FUSE4FS_B_TO_FSB(ff, pos + real_count);
+ blk64_t startblock;
+ errcode_t err;
+
+ err = ext2fs_bmap2(fs, ino, EXT2_INODE(inode), NULL, 0, startoff, NULL,
+ &startblock);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ iomap->dev = FUSE_IOMAP_DEV_NULL;
+ iomap->offset = FUSE4FS_FSB_TO_B(ff, startoff);
+ iomap->flags |= FUSE_IOMAP_F_MERGED;
+ if (startblock) {
+ iomap->addr = FUSE4FS_FSB_TO_B(ff, startblock) + ff->offset;
+ iomap->type = FUSE_IOMAP_TYPE_MAPPED;
+ } else {
+ iomap->addr = FUSE_IOMAP_NULL_ADDR;
+ iomap->type = FUSE_IOMAP_TYPE_HOLE;
+ }
+ iomap->length = fs->blocksize;
+
+ /* See how long the mapping goes for. */
+ for (startoff++; startoff < endoff; startoff++) {
+ blk64_t prev_startblock = startblock;
+
+ err = ext2fs_bmap2(fs, ino, EXT2_INODE(inode), NULL, 0,
+ startoff, NULL, &startblock);
+ if (err)
+ break;
+
+ if (iomap->type == FUSE_IOMAP_TYPE_MAPPED) {
+ if (startblock == prev_startblock + 1)
+ iomap->length += fs->blocksize;
+ else
+ break;
+ } else {
+ if (startblock == 0)
+ iomap->length += fs->blocksize;
+ else
+ break;
+ }
+ }
+
+ /*
+ * If this is a hole that goes beyond EOF, report this as a hole to the
+ * end of the range queried so that FIEMAP doesn't go mad.
+ */
+ if (iomap->type == FUSE_IOMAP_TYPE_HOLE &&
+ iomap->offset + iomap->length >= isize)
+ fuse4fs_iomap_hole_to_eof(ff, iomap, pos, count, inode);
+
+ return 0;
+}
+
+static int fuse4fs_iomap_begin_inline(struct fuse4fs *ff, ext2_ino_t ino,
+ struct ext2_inode_large *inode, off_t pos,
+ uint64_t count, struct fuse_file_iomap *iomap)
+{
+ uint64_t one_fsb = FUSE4FS_FSB_TO_B(ff, 1);
+
+ if (pos >= one_fsb) {
+ fuse4fs_iomap_hole_to_eof(ff, iomap, pos, count, inode);
+ } else {
+ /* ext4 only supports inline data files up to 1 fsb */
+ iomap->dev = FUSE_IOMAP_DEV_NULL;
+ iomap->addr = FUSE_IOMAP_NULL_ADDR;
+ iomap->offset = 0;
+ iomap->length = one_fsb;
+ iomap->type = FUSE_IOMAP_TYPE_INLINE;
+ }
+
+ return 0;
+}
+
+static int fuse4fs_iomap_begin_report(struct fuse4fs *ff, ext2_ino_t ino,
+ struct ext2_inode_large *inode,
+ off_t pos, uint64_t count,
+ uint32_t opflags,
+ struct fuse_file_iomap *read)
+{
+ if (inode->i_flags & EXT4_INLINE_DATA_FL)
+ return fuse4fs_iomap_begin_inline(ff, ino, inode, pos, count,
+ read);
+
+ if (inode->i_flags & EXT4_EXTENTS_FL)
+ return fuse4fs_iomap_begin_extent(ff, ino, inode, pos, count,
+ opflags, read);
+
+ return fuse4fs_iomap_begin_indirect(ff, ino, inode, pos, count,
+ opflags, read);
+}
+
+static int fuse4fs_iomap_begin_read(struct fuse4fs *ff, ext2_ino_t ino,
+ struct ext2_inode_large *inode, off_t pos,
+ uint64_t count, uint32_t opflags,
+ struct fuse_file_iomap *read)
+{
+ return -ENOSYS;
+}
+
+static int fuse4fs_iomap_begin_write(struct fuse4fs *ff, ext2_ino_t ino,
+ struct ext2_inode_large *inode, off_t pos,
+ uint64_t count, uint32_t opflags,
+ struct fuse_file_iomap *read)
+{
+ return -ENOSYS;
+}
+
+static void op_iomap_begin(fuse_req_t req, fuse_ino_t fino, uint64_t dontcare,
+ off_t pos, uint64_t count, uint32_t opflags)
+{
+ struct fuse4fs *ff = fuse4fs_get(req);
+ struct ext2_inode_large inode;
+ struct fuse_file_iomap read = { };
+ ext2_filsys fs;
+ ext2_ino_t ino;
+ errcode_t err;
+ int ret = 0;
+
+ FUSE4FS_CHECK_CONTEXT(req);
+ FUSE4FS_CONVERT_FINO(req, &ino, fino);
+
+ dbg_printf(ff, "%s: ino=%d pos=0x%llx count=0x%llx opflags=0x%x\n",
+ __func__, ino,
+ (unsigned long long)pos,
+ (unsigned long long)count,
+ opflags);
+
+ fs = fuse4fs_start(ff);
+ err = fuse4fs_read_inode(fs, ino, &inode);
+ if (err) {
+ ret = translate_error(fs, ino, err);
+ goto out_unlock;
+ }
+
+ if (opflags & FUSE_IOMAP_OP_REPORT)
+ ret = fuse4fs_iomap_begin_report(ff, ino, &inode, pos, count,
+ opflags, &read);
+ else if (fuse_iomap_is_write(opflags))
+ ret = fuse4fs_iomap_begin_write(ff, ino, &inode, pos, count,
+ opflags, &read);
+ else
+ ret = fuse4fs_iomap_begin_read(ff, ino, &inode, pos, count,
+ opflags, &read);
+ if (ret)
+ goto out_unlock;
+
+ dbg_printf(ff,
+ "%s: ino=%d pos=0x%llx -> addr=0x%llx offset=0x%llx length=0x%llx type=%u flags=0x%x\n",
+ __func__, ino,
+ (unsigned long long)pos,
+ (unsigned long long)read.addr,
+ (unsigned long long)read.offset,
+ (unsigned long long)read.length,
+ read.type,
+ read.flags);
+
+ /* Not filling even the first byte will make the kernel unhappy. */
+ if (read.offset > pos || read.offset + read.length <= pos) {
+ ret = translate_error(fs, ino, EXT2_ET_INODE_CORRUPTED);
+ goto out_unlock;
+ }
+
+out_unlock:
+ fuse4fs_finish(ff, ret);
+ if (ret)
+ fuse_reply_err(req, -ret);
+ else
+ fuse_reply_iomap_begin(req, &read, NULL);
+}
+
+static void op_iomap_end(fuse_req_t req, fuse_ino_t fino, uint64_t dontcare,
+ off_t pos, uint64_t count, uint32_t opflags,
+ ssize_t written, const struct fuse_file_iomap *iomap)
+{
+ struct fuse4fs *ff = fuse4fs_get(req);
+ ext2_ino_t ino;
+
+ FUSE4FS_CHECK_CONTEXT(req);
+ FUSE4FS_CONVERT_FINO(req, &ino, fino);
+
+ dbg_printf(ff,
+ "%s: ino=%d pos=0x%llx count=0x%llx opflags=0x%x written=0x%zx mapflags=0x%x\n",
+ __func__, ino,
+ (unsigned long long)pos,
+ (unsigned long long)count,
+ opflags,
+ written,
+ iomap->flags);
+
+ fuse_reply_err(req, 0);
+}
+#endif /* HAVE_FUSE_IOMAP */
+
static struct fuse_lowlevel_ops fs_ops = {
.lookup = op_lookup,
.setattr = op_setattr,
@@ -5960,6 +6491,10 @@ static struct fuse_lowlevel_ops fs_ops = {
#ifdef SUPPORT_FALLOCATE
.fallocate = op_fallocate,
#endif
+#ifdef HAVE_FUSE_IOMAP
+ .iomap_begin = op_iomap_begin,
+ .iomap_end = op_iomap_end,
+#endif /* HAVE_FUSE_IOMAP */
};
static int get_random_bytes(void *p, size_t sz)
@@ -6413,6 +6948,9 @@ int main(int argc, char *argv[])
.opstate = F4OP_WRITABLE,
#ifdef HAVE_FUSE4FS_SERVICE
.bdev_fd = -1,
+#endif
+#ifdef HAVE_FUSE_IOMAP
+ .iomap_state = IOMAP_UNKNOWN,
#endif
};
errcode_t err;
diff --git a/lib/config.h.in b/lib/config.h.in
index 2c25632188e4f3..58338cc926590e 100644
--- a/lib/config.h.in
+++ b/lib/config.h.in
@@ -148,6 +148,9 @@
/* Define to 1 if you have the <fuse.h> header file. */
#undef HAVE_FUSE_H
+/* Define to 1 if fuse supports iomap */
+#undef HAVE_FUSE_IOMAP
+
/* Define to 1 if fuse supports lowlevel API */
#undef HAVE_FUSE_LOWLEVEL
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index 0f4781bc49f18f..63c9b59e54fb04 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -138,6 +138,9 @@ static inline uint64_t round_down(uint64_t b, unsigned int align)
return b - m;
}
+#define max(a, b) ((a) > (b) ? (a) : (b))
+#define min(a, b) ((a) < (b) ? (a) : (b))
+
#define dbg_printf(fuse2fs, format, ...) \
while ((fuse2fs)->debug) { \
printf("FUSE2FS (%s): tid=%llu " format, (fuse2fs)->shortdev, get_thread_id(), ##__VA_ARGS__); \
@@ -215,6 +218,14 @@ enum fuse2fs_opstate {
F2OP_SHUTDOWN,
};
+#ifdef HAVE_FUSE_IOMAP
+enum fuse2fs_iomap_state {
+ IOMAP_DISABLED,
+ IOMAP_UNKNOWN,
+ IOMAP_ENABLED,
+};
+#endif
+
/* Main program context */
#define FUSE2FS_MAGIC (0xEF53DEADUL)
struct fuse2fs {
@@ -242,6 +253,9 @@ struct fuse2fs {
int logfd;
int blocklog;
int oom_score_adj;
+#ifdef HAVE_FUSE_IOMAP
+ enum fuse2fs_iomap_state iomap_state;
+#endif
unsigned int blockmask;
unsigned long offset;
unsigned int next_generation;
@@ -693,6 +707,15 @@ fuse2fs_set_handle(struct fuse_file_info *fp, struct fuse2fs_file_handle *fh)
fp->fh = (uintptr_t)fh;
}
+#ifdef HAVE_FUSE_IOMAP
+static inline int fuse2fs_iomap_enabled(const struct fuse2fs *ff)
+{
+ return ff->iomap_state >= IOMAP_ENABLED;
+}
+#else
+# define fuse2fs_iomap_enabled(...) (0)
+#endif
+
static void get_now(struct timespec *now)
{
#ifdef CLOCK_REALTIME
@@ -1122,7 +1145,7 @@ static errcode_t fuse2fs_open(struct fuse2fs *ff)
char options[128];
double deadline;
int flags = EXT2_FLAG_64BITS | EXT2_FLAG_THREADS | EXT2_FLAG_RW |
- EXT2_FLAG_EXCLUSIVE;
+ EXT2_FLAG_EXCLUSIVE | EXT2_FLAG_WRITE_FULL_SUPER;
errcode_t err;
if (ff->lockfile) {
@@ -1409,6 +1432,15 @@ static void op_destroy(void *p EXT2FS_ATTR((unused)))
(stats->cache_hits + stats->cache_misses));
}
+ /*
+ * If we're mounting in iomap mode, we need to unmount in op_destroy so
+ * that the block device will be released before umount(2) returns.
+ */
+ if (ff->iomap_state == IOMAP_ENABLED) {
+ fuse2fs_mmp_cancel(ff);
+ fuse2fs_unmount(ff);
+ }
+
fuse2fs_finish(ff, 0);
}
@@ -1545,6 +1577,44 @@ static inline int fuse_set_feature_flag(struct fuse_conn_info *conn,
}
#endif
+#ifdef HAVE_FUSE_IOMAP
+static inline bool fuse2fs_wants_iomap(struct fuse2fs *ff)
+{
+ if (ff->iomap_state == IOMAP_DISABLED)
+ return false;
+
+ /* iomap only works with block devices */
+ if (!(ff->fs->io->flags & CHANNEL_FLAGS_BLOCK_DEVICE))
+ return false;
+
+ /*
+ * iomap addrs must be aligned to the bdev lba size; we use fs
+ * blocksize as a proxy here
+ */
+ if (ff->offset % ff->fs->blocksize > 0)
+ return false;
+
+ return true;
+}
+
+static void fuse2fs_iomap_enable(struct fuse_conn_info *conn,
+ struct fuse2fs *ff)
+{
+ /* Don't let anyone touch iomap until the end of the patchset. */
+ ff->iomap_state = IOMAP_DISABLED;
+ return;
+
+ if (fuse2fs_wants_iomap(ff) &&
+ fuse_set_feature_flag(conn, FUSE_CAP_IOMAP))
+ ff->iomap_state = IOMAP_ENABLED;
+
+ if (ff->iomap_state == IOMAP_UNKNOWN)
+ ff->iomap_state = IOMAP_DISABLED;
+}
+#else
+# define fuse2fs_iomap_enable(...) ((void)0)
+#endif
+
static void *op_init(struct fuse_conn_info *conn,
struct fuse_config *cfg EXT2FS_ATTR((unused)))
{
@@ -1578,6 +1648,8 @@ static void *op_init(struct fuse_conn_info *conn,
#ifdef FUSE_CAP_NO_EXPORT_SUPPORT
fuse_set_feature_flag(conn, FUSE_CAP_NO_EXPORT_SUPPORT);
#endif
+ fuse2fs_iomap_enable(conn, ff);
+
conn->time_gran = 1;
cfg->use_ino = 1;
if (ff->debug)
@@ -5150,6 +5222,465 @@ static int op_fallocate(const char *path EXT2FS_ATTR((unused)), int mode,
}
#endif /* SUPPORT_FALLOCATE */
+#ifdef HAVE_FUSE_IOMAP
+static void fuse2fs_iomap_hole(struct fuse2fs *ff, struct fuse_file_iomap *iomap,
+ off_t pos, uint64_t count)
+{
+ iomap->dev = FUSE_IOMAP_DEV_NULL;
+ iomap->addr = FUSE_IOMAP_NULL_ADDR;
+ iomap->offset = pos;
+ iomap->length = count;
+ iomap->type = FUSE_IOMAP_TYPE_HOLE;
+}
+
+static void fuse2fs_iomap_hole_to_eof(struct fuse2fs *ff,
+ struct fuse_file_iomap *iomap, off_t pos,
+ off_t count,
+ const struct ext2_inode_large *inode)
+{
+ ext2_filsys fs = ff->fs;
+ uint64_t isize = EXT2_I_SIZE(inode);
+
+ /*
+ * We have to be careful about handling a hole to the right of the
+ * entire mapping tree. First, the mapping must start and end on a
+ * block boundary because they must be aligned to at least an LBA for
+ * the block layer; and to the fsblock for smoother operation.
+ *
+ * As for the length -- we could return a mapping all the way to
+ * i_size, but i_size could be less than pos/count if we're zeroing the
+ * EOF block in anticipation of a truncate operation. Similarly, we
+ * don't want to end the mapping at pos+count because we know there's
+ * nothing mapped beyond here.
+ */
+ uint64_t startoff = round_down(pos, fs->blocksize);
+ uint64_t eofoff = round_up(max(pos + count, isize), fs->blocksize);
+
+ dbg_printf(ff,
+ "pos=0x%llx count=0x%llx isize=0x%llx startoff=0x%llx eofoff=0x%llx\n",
+ (unsigned long long)pos,
+ (unsigned long long)count,
+ (unsigned long long)isize,
+ (unsigned long long)startoff,
+ (unsigned long long)eofoff);
+
+ fuse2fs_iomap_hole(ff, iomap, startoff, eofoff - startoff);
+}
+
+#define DEBUG_IOMAP
+#ifdef DEBUG_IOMAP
+# define __DUMP_EXTENT(ff, func, tag, startoff, err, extent) \
+ do { \
+ dbg_printf((ff), \
+ "%s: %s startoff 0x%llx err %ld lblk 0x%llx pblk 0x%llx len 0x%x flags 0x%x\n", \
+ (func), (tag), (startoff), (err), (extent)->e_lblk, \
+ (extent)->e_pblk, (extent)->e_len, \
+ (extent)->e_flags & EXT2_EXTENT_FLAGS_UNINIT); \
+ } while(0)
+# define DUMP_EXTENT(ff, tag, startoff, err, extent) \
+ __DUMP_EXTENT((ff), __func__, (tag), (startoff), (err), (extent))
+
+# define __DUMP_INFO(ff, func, tag, startoff, err, info) \
+ do { \
+ dbg_printf((ff), \
+ "%s: %s startoff 0x%llx err %ld entry %d/%d/%d level %d/%d\n", \
+ (func), (tag), (startoff), (err), \
+ (info)->curr_entry, (info)->num_entries, \
+ (info)->max_entries, (info)->curr_level, \
+ (info)->max_depth); \
+ } while(0)
+# define DUMP_INFO(ff, tag, startoff, err, info) \
+ __DUMP_INFO((ff), __func__, (tag), (startoff), (err), (info))
+#else
+# define __DUMP_EXTENT(...) ((void)0)
+# define DUMP_EXTENT(...) ((void)0)
+# define DUMP_INFO(...) ((void)0)
+#endif
+
+static inline errcode_t __fuse2fs_get_mapping_at(struct fuse2fs *ff,
+ ext2_extent_handle_t handle,
+ blk64_t startoff,
+ struct ext2fs_extent *bmap,
+ const char *func)
+{
+ errcode_t err;
+
+ /*
+ * Find the file mapping at startoff. We don't check the return value
+ * of _goto because _get will error out if _goto failed. There's a
+ * subtlety to the outcome of _goto when startoff falls in a sparse
+ * hole however:
+ *
+ * Most of the time, _goto points the cursor at the mapping whose lblk
+ * is just to the left of startoff. The mapping may or may not overlap
+ * startoff; this is ok. In other words, the tree lookup behaves as if
+ * we asked it to use a less than or equals comparison.
+ *
+ * However, if startoff is to the left of the first mapping in the
+ * extent tree, _goto points the cursor at that first mapping because
+ * it doesn't know how to deal with this situation. In this case,
+ * the tree lookup behaves as if we asked it to use a greater than
+ * or equals comparison.
+ *
+ * Note: If _get() returns 'no current node', that means that there
+ * aren't any mappings at all.
+ */
+ ext2fs_extent_goto(handle, startoff);
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT, bmap);
+ __DUMP_EXTENT(ff, func, "lookup", startoff, err, bmap);
+ if (err == EXT2_ET_NO_CURRENT_NODE)
+ err = EXT2_ET_EXTENT_NOT_FOUND;
+ return err;
+}
+
+static inline errcode_t __fuse2fs_get_next_mapping(struct fuse2fs *ff,
+ ext2_extent_handle_t handle,
+ blk64_t startoff,
+ struct ext2fs_extent *bmap,
+ const char *func)
+{
+ struct ext2fs_extent newex;
+ struct ext2_extent_info info;
+ errcode_t err;
+
+ /*
+ * The extent tree code has this (probably broken) behavior that if
+ * more than two of the highest levels of the cursor point at the
+ * rightmost edge of an extent tree block, a _NEXT_LEAF movement fails
+ * to move the cursor position of any of the lower levels. IOWs, if
+ * leaf level N is at the right edge, it will only advance level N-1
+ * to the right. If N-1 was at the right edge, the cursor resets to
+ * record 0 of that level and goes down to the wrong leaf.
+ *
+ * Work around this by walking up (towards root level 0) the extent
+ * tree until we find a level where we're not already at the rightmost
+ * edge. The _NEXT_LEAF movement will walk down the tree to find the
+ * leaves.
+ */
+ err = ext2fs_extent_get_info(handle, &info);
+ DUMP_INFO(ff, "UP?", startoff, err, &info);
+ if (err)
+ return err;
+
+ while (info.curr_entry == info.num_entries && info.curr_level > 0) {
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_UP, &newex);
+ DUMP_EXTENT(ff, "UP", startoff, err, &newex);
+ if (err)
+ return err;
+ err = ext2fs_extent_get_info(handle, &info);
+ DUMP_INFO(ff, "UP", startoff, err, &info);
+ if (err)
+ return err;
+ }
+
+ /*
+ * If we're at the root and there are no more entries, there's nothing
+ * else to be found.
+ */
+ if (info.curr_level == 0 && info.curr_entry == info.num_entries)
+ return EXT2_ET_EXTENT_NOT_FOUND;
+
+ /* Otherwise grab this next leaf and return it. */
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT_LEAF, &newex);
+ DUMP_EXTENT(ff, "NEXT", startoff, err, &newex);
+ if (err)
+ return err;
+
+ *bmap = newex;
+ return 0;
+}
+
+#define fuse2fs_get_mapping_at(ff, handle, startoff, bmap) \
+ __fuse2fs_get_mapping_at((ff), (handle), (startoff), (bmap), __func__)
+#define fuse2fs_get_next_mapping(ff, handle, startoff, bmap) \
+ __fuse2fs_get_next_mapping((ff), (handle), (startoff), (bmap), __func__)
+
+static errcode_t fuse2fs_iomap_begin_extent(struct fuse2fs *ff, uint64_t ino,
+ struct ext2_inode_large *inode,
+ off_t pos, uint64_t count,
+ uint32_t opflags,
+ struct fuse_file_iomap *iomap)
+{
+ ext2_extent_handle_t handle;
+ struct ext2fs_extent extent = { };
+ ext2_filsys fs = ff->fs;
+ const blk64_t startoff = FUSE2FS_B_TO_FSBT(ff, pos);
+ errcode_t err;
+ int ret = 0;
+
+ err = ext2fs_extent_open2(fs, ino, EXT2_INODE(inode), &handle);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = fuse2fs_get_mapping_at(ff, handle, startoff, &extent);
+ if (err == EXT2_ET_EXTENT_NOT_FOUND) {
+ /* No mappings at all; the whole range is a hole. */
+ fuse2fs_iomap_hole_to_eof(ff, iomap, pos, count, inode);
+ goto out_handle;
+ }
+ if (err) {
+ ret = translate_error(fs, ino, err);
+ goto out_handle;
+ }
+
+ if (startoff < extent.e_lblk) {
+ /*
+ * Mapping starts to the right of the current position.
+ * Synthesize a hole going to that next extent.
+ */
+ fuse2fs_iomap_hole(ff, iomap, FUSE2FS_FSB_TO_B(ff, startoff),
+ FUSE2FS_FSB_TO_B(ff, extent.e_lblk - startoff));
+ goto out_handle;
+ }
+
+ if (startoff >= extent.e_lblk + extent.e_len) {
+ /*
+ * Mapping ends to the left of the current position. Try to
+ * find the next mapping. If there is no next mapping, the
+ * whole range is in a hole.
+ */
+ err = fuse2fs_get_next_mapping(ff, handle, startoff, &extent);
+ if (err == EXT2_ET_EXTENT_NOT_FOUND) {
+ fuse2fs_iomap_hole_to_eof(ff, iomap, pos, count, inode);
+ goto out_handle;
+ }
+
+ /*
+ * If the new mapping starts to the right of startoff, there's
+ * a hole from startoff to the start of the new mapping.
+ */
+ if (startoff < extent.e_lblk) {
+ fuse2fs_iomap_hole(ff, iomap,
+ FUSE2FS_FSB_TO_B(ff, startoff),
+ FUSE2FS_FSB_TO_B(ff, extent.e_lblk - startoff));
+ goto out_handle;
+ }
+
+ /*
+ * The new mapping starts at startoff. Something weird
+ * happened in the extent tree lookup, but we found a valid
+ * mapping so we'll run with it.
+ */
+ }
+
+ /* Mapping overlaps startoff, report this. */
+ iomap->dev = FUSE_IOMAP_DEV_NULL;
+ iomap->addr = FUSE2FS_FSB_TO_B(ff, extent.e_pblk) + ff->offset;
+ iomap->offset = FUSE2FS_FSB_TO_B(ff, extent.e_lblk);
+ iomap->length = FUSE2FS_FSB_TO_B(ff, extent.e_len);
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT)
+ iomap->type = FUSE_IOMAP_TYPE_UNWRITTEN;
+ else
+ iomap->type = FUSE_IOMAP_TYPE_MAPPED;
+
+out_handle:
+ ext2fs_extent_free(handle);
+ return ret;
+}
+
+static int fuse2fs_iomap_begin_indirect(struct fuse2fs *ff, uint64_t ino,
+ struct ext2_inode_large *inode,
+ off_t pos, uint64_t count,
+ uint32_t opflags,
+ struct fuse_file_iomap *iomap)
+{
+ ext2_filsys fs = ff->fs;
+ blk64_t startoff = FUSE2FS_B_TO_FSBT(ff, pos);
+ uint64_t isize = EXT2_I_SIZE(inode);
+ uint64_t real_count = min(count, 131072);
+ const blk64_t endoff = FUSE2FS_B_TO_FSB(ff, pos + real_count);
+ blk64_t startblock;
+ errcode_t err;
+
+ err = ext2fs_bmap2(fs, ino, EXT2_INODE(inode), NULL, 0, startoff, NULL,
+ &startblock);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ iomap->dev = FUSE_IOMAP_DEV_NULL;
+ iomap->offset = FUSE2FS_FSB_TO_B(ff, startoff);
+ iomap->flags |= FUSE_IOMAP_F_MERGED;
+ if (startblock) {
+ iomap->addr = FUSE2FS_FSB_TO_B(ff, startblock) + ff->offset;
+ iomap->type = FUSE_IOMAP_TYPE_MAPPED;
+ } else {
+ iomap->addr = FUSE_IOMAP_NULL_ADDR;
+ iomap->type = FUSE_IOMAP_TYPE_HOLE;
+ }
+ iomap->length = fs->blocksize;
+
+ /* See how long the mapping goes for. */
+ for (startoff++; startoff < endoff; startoff++) {
+ blk64_t prev_startblock = startblock;
+
+ err = ext2fs_bmap2(fs, ino, EXT2_INODE(inode), NULL, 0,
+ startoff, NULL, &startblock);
+ if (err)
+ break;
+
+ if (iomap->type == FUSE_IOMAP_TYPE_MAPPED) {
+ if (startblock == prev_startblock + 1)
+ iomap->length += fs->blocksize;
+ else
+ break;
+ } else {
+ if (startblock == 0)
+ iomap->length += fs->blocksize;
+ else
+ break;
+ }
+ }
+
+ /*
+ * If this is a hole that goes beyond EOF, report this as a hole to the
+ * end of the range queried so that FIEMAP doesn't go mad.
+ */
+ if (iomap->type == FUSE_IOMAP_TYPE_HOLE &&
+ iomap->offset + iomap->length >= isize)
+ fuse2fs_iomap_hole_to_eof(ff, iomap, pos, count, inode);
+
+ return 0;
+}
+
+static int fuse2fs_iomap_begin_inline(struct fuse2fs *ff, ext2_ino_t ino,
+ struct ext2_inode_large *inode, off_t pos,
+ uint64_t count, struct fuse_file_iomap *iomap)
+{
+ uint64_t one_fsb = FUSE2FS_FSB_TO_B(ff, 1);
+
+ if (pos >= one_fsb) {
+ fuse2fs_iomap_hole_to_eof(ff, iomap, pos, count, inode);
+ } else {
+ /* ext4 only supports inline data files up to 1 fsb */
+ iomap->dev = FUSE_IOMAP_DEV_NULL;
+ iomap->addr = FUSE_IOMAP_NULL_ADDR;
+ iomap->offset = 0;
+ iomap->length = one_fsb;
+ iomap->type = FUSE_IOMAP_TYPE_INLINE;
+ }
+
+ return 0;
+}
+
+static int fuse2fs_iomap_begin_report(struct fuse2fs *ff, ext2_ino_t ino,
+ struct ext2_inode_large *inode,
+ off_t pos, uint64_t count,
+ uint32_t opflags,
+ struct fuse_file_iomap *read)
+{
+ if (inode->i_flags & EXT4_INLINE_DATA_FL)
+ return fuse2fs_iomap_begin_inline(ff, ino, inode, pos, count,
+ read);
+
+ if (inode->i_flags & EXT4_EXTENTS_FL)
+ return fuse2fs_iomap_begin_extent(ff, ino, inode, pos, count,
+ opflags, read);
+
+ return fuse2fs_iomap_begin_indirect(ff, ino, inode, pos, count,
+ opflags, read);
+}
+
+static int fuse2fs_iomap_begin_read(struct fuse2fs *ff, ext2_ino_t ino,
+ struct ext2_inode_large *inode, off_t pos,
+ uint64_t count, uint32_t opflags,
+ struct fuse_file_iomap *read)
+{
+ return -ENOSYS;
+}
+
+static int fuse2fs_iomap_begin_write(struct fuse2fs *ff, ext2_ino_t ino,
+ struct ext2_inode_large *inode, off_t pos,
+ uint64_t count, uint32_t opflags,
+ struct fuse_file_iomap *read)
+{
+ return -ENOSYS;
+}
+
+static int op_iomap_begin(const char *path, uint64_t nodeid, uint64_t attr_ino,
+ off_t pos, uint64_t count, uint32_t opflags,
+ struct fuse_file_iomap *read,
+ struct fuse_file_iomap *write)
+{
+ struct fuse2fs *ff = fuse2fs_get();
+ struct ext2_inode_large inode;
+ ext2_filsys fs;
+ errcode_t err;
+ int ret = 0;
+
+ FUSE2FS_CHECK_CONTEXT(ff);
+
+ dbg_printf(ff,
+ "%s: path=%s nodeid=%llu attr_ino=%llu pos=0x%llx count=0x%llx opflags=0x%x\n",
+ __func__, path,
+ (unsigned long long)nodeid,
+ (unsigned long long)attr_ino,
+ (unsigned long long)pos,
+ (unsigned long long)count,
+ opflags);
+
+ fs = fuse2fs_start(ff);
+ err = fuse2fs_read_inode(fs, attr_ino, &inode);
+ if (err) {
+ ret = translate_error(fs, attr_ino, err);
+ goto out_unlock;
+ }
+
+ if (opflags & FUSE_IOMAP_OP_REPORT)
+ ret = fuse2fs_iomap_begin_report(ff, attr_ino, &inode, pos,
+ count, opflags, read);
+ else if (fuse_iomap_is_write(opflags))
+ ret = fuse2fs_iomap_begin_write(ff, attr_ino, &inode, pos,
+ count, opflags, read);
+ else
+ ret = fuse2fs_iomap_begin_read(ff, attr_ino, &inode, pos,
+ count, opflags, read);
+ if (ret)
+ goto out_unlock;
+
+ dbg_printf(ff, "%s: nodeid=%llu attr_ino=%llu pos=0x%llx -> addr=0x%llx offset=0x%llx length=0x%llx type=%u\n",
+ __func__,
+ (unsigned long long)nodeid,
+ (unsigned long long)attr_ino,
+ (unsigned long long)pos,
+ (unsigned long long)read->addr,
+ (unsigned long long)read->offset,
+ (unsigned long long)read->length,
+ read->type);
+
+ /* Not filling even the first byte will make the kernel unhappy. */
+ if (read->offset > pos || read->offset + read->length <= pos) {
+ ret = translate_error(fs, attr_ino, EXT2_ET_INODE_CORRUPTED);
+ goto out_unlock;
+ }
+
+out_unlock:
+ fuse2fs_finish(ff, ret);
+ return ret;
+}
+
+static int op_iomap_end(const char *path, uint64_t nodeid, uint64_t attr_ino,
+ off_t pos, uint64_t count, uint32_t opflags,
+ ssize_t written, const struct fuse_file_iomap *iomap)
+{
+ struct fuse2fs *ff = fuse2fs_get();
+
+ FUSE2FS_CHECK_CONTEXT(ff);
+
+ dbg_printf(ff,
+ "%s: path=%s nodeid=%llu attr_ino=%llu pos=0x%llx count=0x%llx opflags=0x%x written=0x%zx mapflags=0x%x\n",
+ __func__, path,
+ (unsigned long long)nodeid,
+ (unsigned long long)attr_ino,
+ (unsigned long long)pos,
+ (unsigned long long)count,
+ opflags,
+ written,
+ iomap->flags);
+
+ return 0;
+}
+#endif /* HAVE_FUSE_IOMAP */
+
static struct fuse_operations fs_ops = {
.init = op_init,
.destroy = op_destroy,
@@ -5191,6 +5722,10 @@ static struct fuse_operations fs_ops = {
#ifdef SUPPORT_FALLOCATE
.fallocate = op_fallocate,
#endif
+#ifdef HAVE_FUSE_IOMAP
+ .iomap_begin = op_iomap_begin,
+ .iomap_end = op_iomap_end,
+#endif /* HAVE_FUSE_IOMAP */
};
static int get_random_bytes(void *p, size_t sz)
@@ -5477,6 +6012,9 @@ int main(int argc, char *argv[])
.bfl = (pthread_mutex_t)PTHREAD_MUTEX_INITIALIZER,
.oom_score_adj = -500,
.opstate = F2OP_WRITABLE,
+#ifdef HAVE_FUSE_IOMAP
+ .iomap_state = IOMAP_UNKNOWN,
+#endif
};
errcode_t err;
FILE *orig_stderr = stderr;
^ permalink raw reply related
* [PATCH 02/19] fuse2fs: add iomap= mount option
From: Darrick J. Wong @ 2026-04-29 14:53 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214459.4107473.9520724883867588970.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
Add a mount option to control iomap usage so that we can test before and
after scenarios.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
fuse4fs/fuse4fs.1.in | 6 ++++++
fuse4fs/fuse4fs.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
misc/fuse2fs.1.in | 6 ++++++
misc/fuse2fs.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 104 insertions(+)
diff --git a/fuse4fs/fuse4fs.1.in b/fuse4fs/fuse4fs.1.in
index 8bef5f48802385..8855867d27101d 100644
--- a/fuse4fs/fuse4fs.1.in
+++ b/fuse4fs/fuse4fs.1.in
@@ -75,6 +75,12 @@ .SS "fuse4fs options:"
\fB-o\fR fuse4fs_debug
enable fuse4fs debugging
.TP
+\fB-o\fR iomap=
+If set to \fI1\fR, requires iomap to be enabled.
+If set to \fI0\fR, forbids use of iomap.
+If set to \fIdefault\fR (or not set), enables iomap if present.
+This substantially improves the performance of the fuse4fs server.
+.TP
\fB-o\fR kernel
Behave more like the kernel ext4 driver in the following ways:
Allows processes owned by other users to access the filesystem.
diff --git a/fuse4fs/fuse4fs.c b/fuse4fs/fuse4fs.c
index a159024f778ba2..df2bda7cc22bf2 100644
--- a/fuse4fs/fuse4fs.c
+++ b/fuse4fs/fuse4fs.c
@@ -236,6 +236,12 @@ enum fuse4fs_opstate {
F4OP_SHUTDOWN,
};
+enum fuse4fs_feature_toggle {
+ FT_DISABLE,
+ FT_ENABLE,
+ FT_DEFAULT,
+};
+
#ifdef HAVE_FUSE_IOMAP
enum fuse4fs_iomap_state {
IOMAP_DISABLED,
@@ -272,6 +278,7 @@ struct fuse4fs {
int blocklog;
int oom_score_adj;
#ifdef HAVE_FUSE_IOMAP
+ enum fuse4fs_feature_toggle iomap_want;
enum fuse4fs_iomap_state iomap_state;
#endif
unsigned int blockmask;
@@ -2013,6 +2020,12 @@ static void fuse4fs_iomap_enable(struct fuse_conn_info *conn,
if (ff->iomap_state == IOMAP_UNKNOWN)
ff->iomap_state = IOMAP_DISABLED;
+
+ if (!fuse4fs_iomap_enabled(ff)) {
+ if (ff->iomap_want == FT_ENABLE)
+ err_printf(ff, "%s\n", _("Could not enable iomap."));
+ return;
+ }
}
#else
# define fuse4fs_iomap_enable(...) ((void)0)
@@ -6522,6 +6535,9 @@ enum {
FUSE4FS_CACHE_SIZE,
FUSE4FS_DIRSYNC,
FUSE4FS_ERRORS_BEHAVIOR,
+#ifdef HAVE_FUSE_IOMAP
+ FUSE4FS_IOMAP,
+#endif
};
#define FUSE4FS_OPT(t, p, v) { t, offsetof(struct fuse4fs, p), v }
@@ -6553,6 +6569,10 @@ static struct fuse_opt fuse4fs_opts[] = {
FUSE_OPT_KEY("cache_size=%s", FUSE4FS_CACHE_SIZE),
FUSE_OPT_KEY("dirsync", FUSE4FS_DIRSYNC),
FUSE_OPT_KEY("errors=%s", FUSE4FS_ERRORS_BEHAVIOR),
+#ifdef HAVE_FUSE_IOMAP
+ FUSE_OPT_KEY("iomap=%s", FUSE4FS_IOMAP),
+ FUSE_OPT_KEY("iomap", FUSE4FS_IOMAP),
+#endif
FUSE_OPT_KEY("-V", FUSE4FS_VERSION),
FUSE_OPT_KEY("--version", FUSE4FS_VERSION),
@@ -6604,6 +6624,23 @@ static int fuse4fs_opt_proc(void *data, const char *arg,
/* do not pass through to libfuse */
return 0;
+#ifdef HAVE_FUSE_IOMAP
+ case FUSE4FS_IOMAP:
+ if (strcmp(arg, "iomap") == 0 || strcmp(arg + 6, "1") == 0)
+ ff->iomap_want = FT_ENABLE;
+ else if (strcmp(arg + 6, "0") == 0)
+ ff->iomap_want = FT_DISABLE;
+ else if (strcmp(arg + 6, "default") == 0)
+ ff->iomap_want = FT_DEFAULT;
+ else {
+ fprintf(stderr, "%s: %s\n", arg,
+ _("unknown iomap= behavior."));
+ return -1;
+ }
+
+ /* do not pass through to libfuse */
+ return 0;
+#endif
case FUSE4FS_IGNORED:
return 0;
case FUSE4FS_HELP:
@@ -6631,6 +6668,9 @@ static int fuse4fs_opt_proc(void *data, const char *arg,
" -o cache_size=N[KMG] use a disk cache of this size\n"
" -o errors= behavior when an error is encountered:\n"
" continue|remount-ro|panic\n"
+#ifdef HAVE_FUSE_IOMAP
+ " -o iomap= 0 to disable iomap, 1 to enable iomap\n"
+#endif
"\n",
outargs->argv[0]);
if (key == FUSE4FS_HELPFULL) {
@@ -6950,6 +6990,7 @@ int main(int argc, char *argv[])
.bdev_fd = -1,
#endif
#ifdef HAVE_FUSE_IOMAP
+ .iomap_want = FT_DEFAULT,
.iomap_state = IOMAP_UNKNOWN,
#endif
};
@@ -6983,6 +7024,11 @@ int main(int argc, char *argv[])
if (fuse4fs_is_service(&fctx))
fuse4fs_service_set_proc_cmdline(&fctx, argc, argv, &args);
+#ifdef HAVE_FUSE_IOMAP
+ if (fctx.iomap_want == FT_DISABLE)
+ fctx.iomap_state = IOMAP_DISABLED;
+#endif
+
/* /dev/sda -> sda for reporting */
fctx.shortdev = strrchr(fctx.device, '/');
if (fctx.shortdev)
diff --git a/misc/fuse2fs.1.in b/misc/fuse2fs.1.in
index 6acfa092851292..2b55fa0e723966 100644
--- a/misc/fuse2fs.1.in
+++ b/misc/fuse2fs.1.in
@@ -75,6 +75,12 @@ .SS "fuse2fs options:"
\fB-o\fR fuse2fs_debug
enable fuse2fs debugging
.TP
+\fB-o\fR iomap=
+If set to \fI1\fR, requires iomap to be enabled.
+If set to \fI0\fR, forbids use of iomap.
+If set to \fIdefault\fR (or not set), enables iomap if present.
+This substantially improves the performance of the fuse2fs server.
+.TP
\fB-o\fR kernel
Behave more like the kernel ext4 driver in the following ways:
Allows processes owned by other users to access the filesystem.
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index 63c9b59e54fb04..15ebe6b39f1288 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -218,6 +218,12 @@ enum fuse2fs_opstate {
F2OP_SHUTDOWN,
};
+enum fuse2fs_feature_toggle {
+ FT_DISABLE,
+ FT_ENABLE,
+ FT_DEFAULT,
+};
+
#ifdef HAVE_FUSE_IOMAP
enum fuse2fs_iomap_state {
IOMAP_DISABLED,
@@ -254,6 +260,7 @@ struct fuse2fs {
int blocklog;
int oom_score_adj;
#ifdef HAVE_FUSE_IOMAP
+ enum fuse2fs_feature_toggle iomap_want;
enum fuse2fs_iomap_state iomap_state;
#endif
unsigned int blockmask;
@@ -1610,6 +1617,12 @@ static void fuse2fs_iomap_enable(struct fuse_conn_info *conn,
if (ff->iomap_state == IOMAP_UNKNOWN)
ff->iomap_state = IOMAP_DISABLED;
+
+ if (!fuse2fs_iomap_enabled(ff)) {
+ if (ff->iomap_want == FT_ENABLE)
+ err_printf(ff, "%s\n", _("Could not enable iomap."));
+ return;
+ }
}
#else
# define fuse2fs_iomap_enable(...) ((void)0)
@@ -5753,6 +5766,9 @@ enum {
FUSE2FS_CACHE_SIZE,
FUSE2FS_DIRSYNC,
FUSE2FS_ERRORS_BEHAVIOR,
+#ifdef HAVE_FUSE_IOMAP
+ FUSE2FS_IOMAP,
+#endif
};
#define FUSE2FS_OPT(t, p, v) { t, offsetof(struct fuse2fs, p), v }
@@ -5784,6 +5800,10 @@ static struct fuse_opt fuse2fs_opts[] = {
FUSE_OPT_KEY("cache_size=%s", FUSE2FS_CACHE_SIZE),
FUSE_OPT_KEY("dirsync", FUSE2FS_DIRSYNC),
FUSE_OPT_KEY("errors=%s", FUSE2FS_ERRORS_BEHAVIOR),
+#ifdef HAVE_FUSE_IOMAP
+ FUSE_OPT_KEY("iomap=%s", FUSE2FS_IOMAP),
+ FUSE_OPT_KEY("iomap", FUSE2FS_IOMAP),
+#endif
FUSE_OPT_KEY("-V", FUSE2FS_VERSION),
FUSE_OPT_KEY("--version", FUSE2FS_VERSION),
@@ -5835,6 +5855,23 @@ static int fuse2fs_opt_proc(void *data, const char *arg,
/* do not pass through to libfuse */
return 0;
+#ifdef HAVE_FUSE_IOMAP
+ case FUSE2FS_IOMAP:
+ if (strcmp(arg, "iomap") == 0 || strcmp(arg + 6, "1") == 0)
+ ff->iomap_want = FT_ENABLE;
+ else if (strcmp(arg + 6, "0") == 0)
+ ff->iomap_want = FT_DISABLE;
+ else if (strcmp(arg + 6, "default") == 0)
+ ff->iomap_want = FT_DEFAULT;
+ else {
+ fprintf(stderr, "%s: %s\n", arg,
+ _("unknown iomap= behavior."));
+ return -1;
+ }
+
+ /* do not pass through to libfuse */
+ return 0;
+#endif
case FUSE2FS_IGNORED:
return 0;
case FUSE2FS_HELP:
@@ -5862,6 +5899,9 @@ static int fuse2fs_opt_proc(void *data, const char *arg,
" -o cache_size=N[KMG] use a disk cache of this size\n"
" -o errors= behavior when an error is encountered:\n"
" continue|remount-ro|panic\n"
+#ifdef HAVE_FUSE_IOMAP
+ " -o iomap= 0 to disable iomap, 1 to enable iomap\n"
+#endif
"\n",
outargs->argv[0]);
if (key == FUSE2FS_HELPFULL) {
@@ -6013,6 +6053,7 @@ int main(int argc, char *argv[])
.oom_score_adj = -500,
.opstate = F2OP_WRITABLE,
#ifdef HAVE_FUSE_IOMAP
+ .iomap_want = FT_DEFAULT,
.iomap_state = IOMAP_UNKNOWN,
#endif
};
@@ -6029,6 +6070,11 @@ int main(int argc, char *argv[])
exit(1);
}
+#ifdef HAVE_FUSE_IOMAP
+ if (fctx.iomap_want == FT_DISABLE)
+ fctx.iomap_state = IOMAP_DISABLED;
+#endif
+
/* /dev/sda -> sda for reporting */
fctx.shortdev = strrchr(fctx.device, '/');
if (fctx.shortdev)
^ permalink raw reply related
* [PATCH 03/19] fuse2fs: implement iomap configuration
From: Darrick J. Wong @ 2026-04-29 14:53 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214459.4107473.9520724883867588970.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
Upload the filesystem geometry to the kernel when asked.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
fuse4fs/fuse4fs.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++++++--
misc/fuse2fs.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++++++--
2 files changed, 188 insertions(+), 6 deletions(-)
diff --git a/fuse4fs/fuse4fs.c b/fuse4fs/fuse4fs.c
index df2bda7cc22bf2..feb46bdfbac39b 100644
--- a/fuse4fs/fuse4fs.c
+++ b/fuse4fs/fuse4fs.c
@@ -208,6 +208,10 @@ static inline uint64_t round_down(uint64_t b, unsigned int align)
# define FL_ZERO_RANGE_FLAG (0)
#endif
+#ifndef NSEC_PER_SEC
+# define NSEC_PER_SEC (1000000000L)
+#endif
+
errcode_t ext2fs_check_ext3_journal(ext2_filsys fs);
errcode_t ext2fs_run_ext3_journal(ext2_filsys *fs);
@@ -995,9 +999,9 @@ static int update_atime(ext2_filsys fs, ext2_ino_t ino)
EXT4_INODE_GET_XTIME(i_mtime, &mtime, pinode);
get_now(&now);
- datime = atime.tv_sec + ((double)atime.tv_nsec / 1000000000);
- dmtime = mtime.tv_sec + ((double)mtime.tv_nsec / 1000000000);
- dnow = now.tv_sec + ((double)now.tv_nsec / 1000000000);
+ datime = atime.tv_sec + ((double)atime.tv_nsec / NSEC_PER_SEC);
+ dmtime = mtime.tv_sec + ((double)mtime.tv_nsec / NSEC_PER_SEC);
+ dnow = now.tv_sec + ((double)now.tv_nsec / NSEC_PER_SEC);
/*
* If atime is newer than mtime and atime hasn't been updated in thirty
@@ -6459,6 +6463,93 @@ static void op_iomap_end(fuse_req_t req, fuse_ino_t fino, uint64_t dontcare,
fuse_reply_err(req, 0);
}
+
+/*
+ * Maximal extent format file size.
+ * Resulting logical blkno at s_maxbytes must fit in our on-disk
+ * extent format containers, within a sector_t, and within i_blocks
+ * in the vfs. ext4 inode has 48 bits of i_block in fsblock units,
+ * so that won't be a limiting factor.
+ *
+ * However there is other limiting factor. We do store extents in the form
+ * of starting block and length, hence the resulting length of the extent
+ * covering maximum file size must fit into on-disk format containers as
+ * well. Given that length is always by 1 unit bigger than max unit (because
+ * we count 0 as well) we have to lower the s_maxbytes by one fs block.
+ *
+ * Note, this does *not* consider any metadata overhead for vfs i_blocks.
+ */
+static off_t fuse4fs_max_size(struct fuse4fs *ff, off_t upper_limit)
+{
+ off_t res;
+
+ if (!ext2fs_has_feature_huge_file(ff->fs->super)) {
+ upper_limit = (1LL << 32) - 1;
+
+ /* total blocks in file system block size */
+ upper_limit >>= (ff->blocklog - 9);
+ upper_limit <<= ff->blocklog;
+ }
+
+ /*
+ * 32-bit extent-start container, ee_block. We lower the maxbytes
+ * by one fs block, so ee_len can cover the extent of maximum file
+ * size
+ */
+ res = (1LL << 32) - 1;
+ res <<= ff->blocklog;
+
+ /* Sanity check against vm- & vfs- imposed limits */
+ if (res > upper_limit)
+ res = upper_limit;
+
+ return res;
+}
+
+static void op_iomap_config(fuse_req_t req,
+ const struct fuse_iomap_config_params *p,
+ size_t psize)
+{
+ struct fuse_iomap_config cfg = { };
+ struct fuse4fs *ff = fuse4fs_get(req);
+ ext2_filsys fs;
+
+ FUSE4FS_CHECK_CONTEXT(req);
+
+ dbg_printf(ff, "%s: flags=0x%llx maxbytes=0x%llx\n", __func__,
+ (unsigned long long)p->flags,
+ (unsigned long long)p->maxbytes);
+ fs = fuse4fs_start(ff);
+
+ cfg.flags |= FUSE_IOMAP_CONFIG_UUID;
+ memcpy(cfg.s_uuid, fs->super->s_uuid, sizeof(cfg.s_uuid));
+ cfg.s_uuid_len = sizeof(fs->super->s_uuid);
+
+ cfg.flags |= FUSE_IOMAP_CONFIG_BLOCKSIZE;
+ cfg.s_blocksize = FUSE4FS_FSB_TO_B(ff, 1);
+
+ /*
+ * If there inode is large enough to house i_[acm]time_extra then we
+ * can turn on nanosecond timestamps; i_crtime was the next field added
+ * after i_atime_extra.
+ */
+ cfg.flags |= FUSE_IOMAP_CONFIG_TIME;
+ if (fs->super->s_inode_size >=
+ offsetof(struct ext2_inode_large, i_crtime)) {
+ cfg.s_time_gran = 1;
+ cfg.s_time_max = EXT4_EXTRA_TIMESTAMP_MAX;
+ } else {
+ cfg.s_time_gran = NSEC_PER_SEC;
+ cfg.s_time_max = EXT4_NON_EXTRA_TIMESTAMP_MAX;
+ }
+ cfg.s_time_min = EXT4_TIMESTAMP_MIN;
+
+ cfg.flags |= FUSE_IOMAP_CONFIG_MAXBYTES;
+ cfg.s_maxbytes = fuse4fs_max_size(ff, p->maxbytes);
+
+ fuse4fs_finish(ff, 0);
+ fuse_reply_iomap_config(req, &cfg);
+}
#endif /* HAVE_FUSE_IOMAP */
static struct fuse_lowlevel_ops fs_ops = {
@@ -6507,6 +6598,7 @@ static struct fuse_lowlevel_ops fs_ops = {
#ifdef HAVE_FUSE_IOMAP
.iomap_begin = op_iomap_begin,
.iomap_end = op_iomap_end,
+ .iomap_config = op_iomap_config,
#endif /* HAVE_FUSE_IOMAP */
};
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index 15ebe6b39f1288..7df4e127e5981a 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -191,6 +191,10 @@ static inline uint64_t round_down(uint64_t b, unsigned int align)
# define FL_ZERO_RANGE_FLAG (0)
#endif
+#ifndef NSEC_PER_SEC
+# define NSEC_PER_SEC (1000000000L)
+#endif
+
errcode_t ext2fs_check_ext3_journal(ext2_filsys fs);
errcode_t ext2fs_run_ext3_journal(ext2_filsys *fs);
@@ -806,9 +810,9 @@ static int update_atime(ext2_filsys fs, ext2_ino_t ino)
EXT4_INODE_GET_XTIME(i_mtime, &mtime, pinode);
get_now(&now);
- datime = atime.tv_sec + ((double)atime.tv_nsec / 1000000000);
- dmtime = mtime.tv_sec + ((double)mtime.tv_nsec / 1000000000);
- dnow = now.tv_sec + ((double)now.tv_nsec / 1000000000);
+ datime = atime.tv_sec + ((double)atime.tv_nsec / NSEC_PER_SEC);
+ dmtime = mtime.tv_sec + ((double)mtime.tv_nsec / NSEC_PER_SEC);
+ dnow = now.tv_sec + ((double)now.tv_nsec / NSEC_PER_SEC);
/*
* If atime is newer than mtime and atime hasn't been updated in thirty
@@ -5692,6 +5696,91 @@ static int op_iomap_end(const char *path, uint64_t nodeid, uint64_t attr_ino,
return 0;
}
+
+/*
+ * Maximal extent format file size.
+ * Resulting logical blkno at s_maxbytes must fit in our on-disk
+ * extent format containers, within a sector_t, and within i_blocks
+ * in the vfs. ext4 inode has 48 bits of i_block in fsblock units,
+ * so that won't be a limiting factor.
+ *
+ * However there is other limiting factor. We do store extents in the form
+ * of starting block and length, hence the resulting length of the extent
+ * covering maximum file size must fit into on-disk format containers as
+ * well. Given that length is always by 1 unit bigger than max unit (because
+ * we count 0 as well) we have to lower the s_maxbytes by one fs block.
+ *
+ * Note, this does *not* consider any metadata overhead for vfs i_blocks.
+ */
+static off_t fuse2fs_max_size(struct fuse2fs *ff, off_t upper_limit)
+{
+ off_t res;
+
+ if (!ext2fs_has_feature_huge_file(ff->fs->super)) {
+ upper_limit = (1LL << 32) - 1;
+
+ /* total blocks in file system block size */
+ upper_limit >>= (ff->blocklog - 9);
+ upper_limit <<= ff->blocklog;
+ }
+
+ /*
+ * 32-bit extent-start container, ee_block. We lower the maxbytes
+ * by one fs block, so ee_len can cover the extent of maximum file
+ * size
+ */
+ res = (1LL << 32) - 1;
+ res <<= ff->blocklog;
+
+ /* Sanity check against vm- & vfs- imposed limits */
+ if (res > upper_limit)
+ res = upper_limit;
+
+ return res;
+}
+
+static int op_iomap_config(const struct fuse_iomap_config_params *p,
+ size_t psize, struct fuse_iomap_config *cfg)
+{
+ struct fuse2fs *ff = fuse2fs_get();
+ ext2_filsys fs;
+
+ FUSE2FS_CHECK_CONTEXT(ff);
+
+ dbg_printf(ff, "%s: flags=0x%llx maxbytes=0x%llx\n", __func__,
+ (unsigned long long)p->flags,
+ (unsigned long long)p->maxbytes);
+ fs = fuse2fs_start(ff);
+
+ cfg->flags |= FUSE_IOMAP_CONFIG_UUID;
+ memcpy(cfg->s_uuid, fs->super->s_uuid, sizeof(cfg->s_uuid));
+ cfg->s_uuid_len = sizeof(fs->super->s_uuid);
+
+ cfg->flags |= FUSE_IOMAP_CONFIG_BLOCKSIZE;
+ cfg->s_blocksize = FUSE2FS_FSB_TO_B(ff, 1);
+
+ /*
+ * If there inode is large enough to house i_[acm]time_extra then we
+ * can turn on nanosecond timestamps; i_crtime was the next field added
+ * after i_atime_extra.
+ */
+ cfg->flags |= FUSE_IOMAP_CONFIG_TIME;
+ if (fs->super->s_inode_size >=
+ offsetof(struct ext2_inode_large, i_crtime)) {
+ cfg->s_time_gran = 1;
+ cfg->s_time_max = EXT4_EXTRA_TIMESTAMP_MAX;
+ } else {
+ cfg->s_time_gran = NSEC_PER_SEC;
+ cfg->s_time_max = EXT4_NON_EXTRA_TIMESTAMP_MAX;
+ }
+ cfg->s_time_min = EXT4_TIMESTAMP_MIN;
+
+ cfg->flags |= FUSE_IOMAP_CONFIG_MAXBYTES;
+ cfg->s_maxbytes = fuse2fs_max_size(ff, p->maxbytes);
+
+ fuse2fs_finish(ff, 0);
+ return 0;
+}
#endif /* HAVE_FUSE_IOMAP */
static struct fuse_operations fs_ops = {
@@ -5738,6 +5827,7 @@ static struct fuse_operations fs_ops = {
#ifdef HAVE_FUSE_IOMAP
.iomap_begin = op_iomap_begin,
.iomap_end = op_iomap_end,
+ .iomap_config = op_iomap_config,
#endif /* HAVE_FUSE_IOMAP */
};
^ permalink raw reply related
* [PATCH 04/19] fuse2fs: register block devices for use with iomap
From: Darrick J. Wong @ 2026-04-29 14:53 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214459.4107473.9520724883867588970.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
Register the ext4 block device with the kernel for use with iomap.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
fuse4fs/fuse4fs.c | 44 ++++++++++++++++++++++++++++++++++++++++----
misc/fuse2fs.c | 42 ++++++++++++++++++++++++++++++++++++++----
2 files changed, 78 insertions(+), 8 deletions(-)
diff --git a/fuse4fs/fuse4fs.c b/fuse4fs/fuse4fs.c
index feb46bdfbac39b..3e9852f585302d 100644
--- a/fuse4fs/fuse4fs.c
+++ b/fuse4fs/fuse4fs.c
@@ -284,6 +284,7 @@ struct fuse4fs {
#ifdef HAVE_FUSE_IOMAP
enum fuse4fs_feature_toggle iomap_want;
enum fuse4fs_iomap_state iomap_state;
+ uint32_t iomap_dev;
#endif
unsigned int blockmask;
unsigned long offset;
@@ -6247,7 +6248,7 @@ static errcode_t fuse4fs_iomap_begin_extent(struct fuse4fs *ff, uint64_t ino,
}
/* Mapping overlaps startoff, report this. */
- iomap->dev = FUSE_IOMAP_DEV_NULL;
+ iomap->dev = ff->iomap_dev;
iomap->addr = FUSE4FS_FSB_TO_B(ff, extent.e_pblk) + ff->offset;
iomap->offset = FUSE4FS_FSB_TO_B(ff, extent.e_lblk);
iomap->length = FUSE4FS_FSB_TO_B(ff, extent.e_len);
@@ -6280,13 +6281,14 @@ static int fuse4fs_iomap_begin_indirect(struct fuse4fs *ff, uint64_t ino,
if (err)
return translate_error(fs, ino, err);
- iomap->dev = FUSE_IOMAP_DEV_NULL;
iomap->offset = FUSE4FS_FSB_TO_B(ff, startoff);
iomap->flags |= FUSE_IOMAP_F_MERGED;
if (startblock) {
+ iomap->dev = ff->iomap_dev;
iomap->addr = FUSE4FS_FSB_TO_B(ff, startblock) + ff->offset;
iomap->type = FUSE_IOMAP_TYPE_MAPPED;
} else {
+ iomap->dev = FUSE_IOMAP_DEV_NULL;
iomap->addr = FUSE_IOMAP_NULL_ADDR;
iomap->type = FUSE_IOMAP_TYPE_HOLE;
}
@@ -6506,6 +6508,30 @@ static off_t fuse4fs_max_size(struct fuse4fs *ff, off_t upper_limit)
return res;
}
+static int fuse4fs_iomap_config_devices(struct fuse4fs *ff)
+{
+ errcode_t err;
+ int fd;
+ int ret;
+
+ err = io_channel_get_fd(ff->fs->io, &fd);
+ if (err)
+ return translate_error(ff->fs, 0, err);
+
+ ret = fuse_lowlevel_iomap_device_add(ff->fuse, fd, 0);
+ if (ret < 0) {
+ dbg_printf(ff, "%s: cannot register iomap dev fd=%d, err=%d\n",
+ __func__, fd, -ret);
+ return translate_error(ff->fs, 0, -ret);
+ }
+
+ dbg_printf(ff, "%s: registered iomap dev fd=%d iomap_dev=%u\n",
+ __func__, fd, ff->iomap_dev);
+
+ ff->iomap_dev = ret;
+ return 0;
+}
+
static void op_iomap_config(fuse_req_t req,
const struct fuse_iomap_config_params *p,
size_t psize)
@@ -6513,6 +6539,7 @@ static void op_iomap_config(fuse_req_t req,
struct fuse_iomap_config cfg = { };
struct fuse4fs *ff = fuse4fs_get(req);
ext2_filsys fs;
+ int ret = 0;
FUSE4FS_CHECK_CONTEXT(req);
@@ -6547,8 +6574,16 @@ static void op_iomap_config(fuse_req_t req,
cfg.flags |= FUSE_IOMAP_CONFIG_MAXBYTES;
cfg.s_maxbytes = fuse4fs_max_size(ff, p->maxbytes);
- fuse4fs_finish(ff, 0);
- fuse_reply_iomap_config(req, &cfg);
+ ret = fuse4fs_iomap_config_devices(ff);
+ if (ret)
+ goto out_unlock;
+
+out_unlock:
+ fuse4fs_finish(ff, ret);
+ if (ret)
+ fuse_reply_err(req, -ret);
+ else
+ fuse_reply_iomap_config(req, &cfg);
}
#endif /* HAVE_FUSE_IOMAP */
@@ -7084,6 +7119,7 @@ int main(int argc, char *argv[])
#ifdef HAVE_FUSE_IOMAP
.iomap_want = FT_DEFAULT,
.iomap_state = IOMAP_UNKNOWN,
+ .iomap_dev = FUSE_IOMAP_DEV_NULL,
#endif
};
errcode_t err;
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index 7df4e127e5981a..c24ae461dad2ad 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -40,6 +40,7 @@
# define _FILE_OFFSET_BITS 64
#endif /* _FILE_OFFSET_BITS */
#include <fuse.h>
+#include <fuse_lowlevel.h>
#ifdef __SET_FOB_FOR_FUSE
# undef _FILE_OFFSET_BITS
#endif /* __SET_FOB_FOR_FUSE */
@@ -266,6 +267,7 @@ struct fuse2fs {
#ifdef HAVE_FUSE_IOMAP
enum fuse2fs_feature_toggle iomap_want;
enum fuse2fs_iomap_state iomap_state;
+ uint32_t iomap_dev;
#endif
unsigned int blockmask;
unsigned long offset;
@@ -5481,7 +5483,7 @@ static errcode_t fuse2fs_iomap_begin_extent(struct fuse2fs *ff, uint64_t ino,
}
/* Mapping overlaps startoff, report this. */
- iomap->dev = FUSE_IOMAP_DEV_NULL;
+ iomap->dev = ff->iomap_dev;
iomap->addr = FUSE2FS_FSB_TO_B(ff, extent.e_pblk) + ff->offset;
iomap->offset = FUSE2FS_FSB_TO_B(ff, extent.e_lblk);
iomap->length = FUSE2FS_FSB_TO_B(ff, extent.e_len);
@@ -5514,13 +5516,14 @@ static int fuse2fs_iomap_begin_indirect(struct fuse2fs *ff, uint64_t ino,
if (err)
return translate_error(fs, ino, err);
- iomap->dev = FUSE_IOMAP_DEV_NULL;
iomap->offset = FUSE2FS_FSB_TO_B(ff, startoff);
iomap->flags |= FUSE_IOMAP_F_MERGED;
if (startblock) {
+ iomap->dev = ff->iomap_dev;
iomap->addr = FUSE2FS_FSB_TO_B(ff, startblock) + ff->offset;
iomap->type = FUSE_IOMAP_TYPE_MAPPED;
} else {
+ iomap->dev = FUSE_IOMAP_DEV_NULL;
iomap->addr = FUSE_IOMAP_NULL_ADDR;
iomap->type = FUSE_IOMAP_TYPE_HOLE;
}
@@ -5739,11 +5742,36 @@ static off_t fuse2fs_max_size(struct fuse2fs *ff, off_t upper_limit)
return res;
}
+static int fuse2fs_iomap_config_devices(struct fuse2fs *ff)
+{
+ errcode_t err;
+ int fd;
+ int ret;
+
+ err = io_channel_get_fd(ff->fs->io, &fd);
+ if (err)
+ return translate_error(ff->fs, 0, err);
+
+ ret = fuse_fs_iomap_device_add(fd, 0);
+ if (ret < 0) {
+ dbg_printf(ff, "%s: cannot register iomap dev fd=%d, err=%d\n",
+ __func__, fd, -ret);
+ return translate_error(ff->fs, 0, -ret);
+ }
+
+ dbg_printf(ff, "%s: registered iomap dev fd=%d iomap_dev=%u\n",
+ __func__, fd, ff->iomap_dev);
+
+ ff->iomap_dev = ret;
+ return 0;
+}
+
static int op_iomap_config(const struct fuse_iomap_config_params *p,
size_t psize, struct fuse_iomap_config *cfg)
{
struct fuse2fs *ff = fuse2fs_get();
ext2_filsys fs;
+ int ret = 0;
FUSE2FS_CHECK_CONTEXT(ff);
@@ -5778,8 +5806,13 @@ static int op_iomap_config(const struct fuse_iomap_config_params *p,
cfg->flags |= FUSE_IOMAP_CONFIG_MAXBYTES;
cfg->s_maxbytes = fuse2fs_max_size(ff, p->maxbytes);
- fuse2fs_finish(ff, 0);
- return 0;
+ ret = fuse2fs_iomap_config_devices(ff);
+ if (ret)
+ goto out_unlock;
+
+out_unlock:
+ fuse2fs_finish(ff, ret);
+ return ret;
}
#endif /* HAVE_FUSE_IOMAP */
@@ -6145,6 +6178,7 @@ int main(int argc, char *argv[])
#ifdef HAVE_FUSE_IOMAP
.iomap_want = FT_DEFAULT,
.iomap_state = IOMAP_UNKNOWN,
+ .iomap_dev = FUSE_IOMAP_DEV_NULL,
#endif
};
errcode_t err;
^ permalink raw reply related
* [PATCH 05/19] fuse2fs: implement directio file reads
From: Darrick J. Wong @ 2026-04-29 14:53 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214459.4107473.9520724883867588970.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
Implement file reads via iomap. Currently only directio is supported.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
fuse4fs/fuse4fs.c | 14 +++++++++++++-
misc/fuse2fs.c | 14 +++++++++++++-
2 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/fuse4fs/fuse4fs.c b/fuse4fs/fuse4fs.c
index 3e9852f585302d..a1d931aed8f393 100644
--- a/fuse4fs/fuse4fs.c
+++ b/fuse4fs/fuse4fs.c
@@ -6370,7 +6370,19 @@ static int fuse4fs_iomap_begin_read(struct fuse4fs *ff, ext2_ino_t ino,
uint64_t count, uint32_t opflags,
struct fuse_file_iomap *read)
{
- return -ENOSYS;
+ if (!(opflags & FUSE_IOMAP_OP_DIRECT))
+ return -ENOSYS;
+
+ /* fall back to slow path for inline data reads */
+ if (inode->i_flags & EXT4_INLINE_DATA_FL)
+ return -ENOSYS;
+
+ if (inode->i_flags & EXT4_EXTENTS_FL)
+ return fuse4fs_iomap_begin_extent(ff, ino, inode, pos, count,
+ opflags, read);
+
+ return fuse4fs_iomap_begin_indirect(ff, ino, inode, pos, count,
+ opflags, read);
}
static int fuse4fs_iomap_begin_write(struct fuse4fs *ff, ext2_ino_t ino,
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index c24ae461dad2ad..739867fa41dd91 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -5605,7 +5605,19 @@ static int fuse2fs_iomap_begin_read(struct fuse2fs *ff, ext2_ino_t ino,
uint64_t count, uint32_t opflags,
struct fuse_file_iomap *read)
{
- return -ENOSYS;
+ if (!(opflags & FUSE_IOMAP_OP_DIRECT))
+ return -ENOSYS;
+
+ /* fall back to slow path for inline data reads */
+ if (inode->i_flags & EXT4_INLINE_DATA_FL)
+ return -ENOSYS;
+
+ if (inode->i_flags & EXT4_EXTENTS_FL)
+ return fuse2fs_iomap_begin_extent(ff, ino, inode, pos, count,
+ opflags, read);
+
+ return fuse2fs_iomap_begin_indirect(ff, ino, inode, pos, count,
+ opflags, read);
}
static int fuse2fs_iomap_begin_write(struct fuse2fs *ff, ext2_ino_t ino,
^ permalink raw reply related
* [PATCH 06/19] fuse2fs: add extent dump function for debugging
From: Darrick J. Wong @ 2026-04-29 14:54 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214459.4107473.9520724883867588970.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
Add a function to dump an inode's extent map for debugging purposes.
This helped debug a problem with generic/299 failing on 1k fsblock
filesystems:
--- a/tests/generic/299.out 2025-07-15 14:45:15.030113607 -0700
+++ b/tests/generic/299.out.bad 2025-07-16 19:33:50.889344998 -0700
@@ -3,3 +3,4 @@ QA output created by 299
Run fio with random aio-dio pattern
Start fallocate/truncate loop
+fio: io_u error on file /opt/direct_aio.0.0: Input/output error: write offset=2602827776, buflen=131072
(The cause of this was misuse of the libext2fs extent code)
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
fuse4fs/fuse4fs.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++
misc/fuse2fs.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 140 insertions(+)
diff --git a/fuse4fs/fuse4fs.c b/fuse4fs/fuse4fs.c
index a1d931aed8f393..1489be2104f2b2 100644
--- a/fuse4fs/fuse4fs.c
+++ b/fuse4fs/fuse4fs.c
@@ -917,6 +917,74 @@ static inline int fuse4fs_iomap_enabled(const struct fuse4fs *ff)
# define fuse4fs_iomap_enabled(...) (0)
#endif
+static inline void fuse4fs_dump_extents(struct fuse4fs *ff, ext2_ino_t ino,
+ struct ext2_inode_large *inode,
+ const char *why)
+{
+ ext2_filsys fs = ff->fs;
+ unsigned int nr = 0;
+ blk64_t blockcount = 0;
+ struct ext2_inode_large xinode;
+ struct ext2fs_extent extent;
+ ext2_extent_handle_t extents;
+ int op = EXT2_EXTENT_ROOT;
+ errcode_t retval;
+
+ if (!inode) {
+ inode = &xinode;
+
+ retval = fuse4fs_read_inode(fs, ino, inode);
+ if (retval) {
+ com_err(__func__, retval, _("reading ino %u"), ino);
+ return;
+ }
+ }
+
+ if (!(inode->i_flags & EXT4_EXTENTS_FL))
+ return;
+
+ printf("%s: %s ino=%u isize %llu iblocks %llu\n", __func__, why, ino,
+ EXT2_I_SIZE(inode),
+ (ext2fs_get_stat_i_blocks(fs, EXT2_INODE(inode)) * 512) /
+ fs->blocksize);
+ fflush(stdout);
+
+ retval = ext2fs_extent_open(fs, ino, &extents);
+ if (retval) {
+ com_err(__func__, retval, _("opening extents of ino \"%u\""),
+ ino);
+ return;
+ }
+
+ while ((retval = ext2fs_extent_get(extents, op, &extent)) == 0) {
+ op = EXT2_EXTENT_NEXT;
+
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_SECOND_VISIT)
+ continue;
+
+ printf("[%u]: %s ino=%u lblk 0x%llx pblk 0x%llx len 0x%x flags 0x%x\n",
+ nr++, why, ino, extent.e_lblk, extent.e_pblk,
+ extent.e_len, extent.e_flags);
+ fflush(stdout);
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_LEAF)
+ blockcount += extent.e_len;
+ else
+ blockcount++;
+ }
+ if (retval == EXT2_ET_EXTENT_NO_NEXT)
+ retval = 0;
+ if (retval) {
+ com_err(__func__, retval, ("getting extents of ino %u"),
+ ino);
+ }
+ if (inode->i_file_acl)
+ blockcount++;
+ printf("%s: %s sum(e_len) %llu\n", __func__, why, blockcount);
+ fflush(stdout);
+
+ ext2fs_extent_free(extents);
+}
+
static void get_now(struct timespec *now)
{
#ifdef CLOCK_REALTIME
@@ -6444,6 +6512,8 @@ static void op_iomap_begin(fuse_req_t req, fuse_ino_t fino, uint64_t dontcare,
/* Not filling even the first byte will make the kernel unhappy. */
if (read.offset > pos || read.offset + read.length <= pos) {
+ if (ff->debug)
+ fuse4fs_dump_extents(ff, ino, &inode, "BAD DATA");
ret = translate_error(fs, ino, EXT2_ET_INODE_CORRUPTED);
goto out_unlock;
}
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index 739867fa41dd91..4b37bcde63d0f2 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -729,6 +729,74 @@ static inline int fuse2fs_iomap_enabled(const struct fuse2fs *ff)
# define fuse2fs_iomap_enabled(...) (0)
#endif
+static inline void fuse2fs_dump_extents(struct fuse2fs *ff, ext2_ino_t ino,
+ struct ext2_inode_large *inode,
+ const char *why)
+{
+ ext2_filsys fs = ff->fs;
+ unsigned int nr = 0;
+ blk64_t blockcount = 0;
+ struct ext2_inode_large xinode;
+ struct ext2fs_extent extent;
+ ext2_extent_handle_t extents;
+ int op = EXT2_EXTENT_ROOT;
+ errcode_t retval;
+
+ if (!inode) {
+ inode = &xinode;
+
+ retval = fuse2fs_read_inode(fs, ino, inode);
+ if (retval) {
+ com_err(__func__, retval, _("reading ino %u"), ino);
+ return;
+ }
+ }
+
+ if (!(inode->i_flags & EXT4_EXTENTS_FL))
+ return;
+
+ printf("%s: %s ino=%u isize %llu iblocks %llu\n", __func__, why, ino,
+ EXT2_I_SIZE(inode),
+ (ext2fs_get_stat_i_blocks(fs, EXT2_INODE(inode)) * 512) /
+ fs->blocksize);
+ fflush(stdout);
+
+ retval = ext2fs_extent_open(fs, ino, &extents);
+ if (retval) {
+ com_err(__func__, retval, _("opening extents of ino \"%u\""),
+ ino);
+ return;
+ }
+
+ while ((retval = ext2fs_extent_get(extents, op, &extent)) == 0) {
+ op = EXT2_EXTENT_NEXT;
+
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_SECOND_VISIT)
+ continue;
+
+ printf("[%u]: %s ino=%u lblk 0x%llx pblk 0x%llx len 0x%x flags 0x%x\n",
+ nr++, why, ino, extent.e_lblk, extent.e_pblk,
+ extent.e_len, extent.e_flags);
+ fflush(stdout);
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_LEAF)
+ blockcount += extent.e_len;
+ else
+ blockcount++;
+ }
+ if (retval == EXT2_ET_EXTENT_NO_NEXT)
+ retval = 0;
+ if (retval) {
+ com_err(__func__, retval, ("getting extents of ino %u"),
+ ino);
+ }
+ if (inode->i_file_acl)
+ blockcount++;
+ printf("%s: %s sum(e_len) %llu\n", __func__, why, blockcount);
+ fflush(stdout);
+
+ ext2fs_extent_free(extents);
+}
+
static void get_now(struct timespec *now)
{
#ifdef CLOCK_REALTIME
@@ -5681,6 +5749,8 @@ static int op_iomap_begin(const char *path, uint64_t nodeid, uint64_t attr_ino,
/* Not filling even the first byte will make the kernel unhappy. */
if (read->offset > pos || read->offset + read->length <= pos) {
+ if (ff->debug)
+ fuse2fs_dump_extents(ff, attr_ino, &inode, "BAD DATA");
ret = translate_error(fs, attr_ino, EXT2_ET_INODE_CORRUPTED);
goto out_unlock;
}
^ permalink raw reply related
* [PATCH 07/19] fuse2fs: implement direct write support
From: Darrick J. Wong @ 2026-04-29 14:54 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214459.4107473.9520724883867588970.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
Wire up an iomap_begin method that can allocate into holes so that we
can do directio writes.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
fuse4fs/fuse4fs.c | 477 +++++++++++++++++++++++++++++++++++++++++++++++++++++
misc/fuse2fs.c | 471 ++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 942 insertions(+), 6 deletions(-)
diff --git a/fuse4fs/fuse4fs.c b/fuse4fs/fuse4fs.c
index 1489be2104f2b2..8b508de5b8cb65 100644
--- a/fuse4fs/fuse4fs.c
+++ b/fuse4fs/fuse4fs.c
@@ -6453,12 +6453,106 @@ static int fuse4fs_iomap_begin_read(struct fuse4fs *ff, ext2_ino_t ino,
opflags, read);
}
+static int fuse4fs_iomap_write_allocate(struct fuse4fs *ff, ext2_ino_t ino,
+ struct ext2_inode_large *inode,
+ off_t pos, uint64_t count,
+ uint32_t opflags,
+ struct fuse_file_iomap *read,
+ bool *dirty)
+{
+ ext2_filsys fs = ff->fs;
+ blk64_t startoff = FUSE4FS_B_TO_FSBT(ff, pos);
+ blk64_t stopoff = FUSE4FS_B_TO_FSB(ff, pos + count);
+ blk64_t old_iblocks;
+ errcode_t err;
+ int ret;
+
+ dbg_printf(ff,
+ "%s: ino=%d startoff 0x%llx blockcount 0x%llx\n",
+ __func__, ino, startoff, stopoff - startoff);
+
+ if (!fuse4fs_can_allocate(ff, stopoff - startoff))
+ return -ENOSPC;
+
+ old_iblocks = ext2fs_get_stat_i_blocks(fs, EXT2_INODE(inode));
+ err = ext2fs_fallocate(fs, EXT2_FALLOCATE_FORCE_UNINIT, ino,
+ EXT2_INODE(inode), ~0ULL, startoff,
+ stopoff - startoff);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /*
+ * New allocations for file data blocks on indirect mapped files are
+ * zeroed through the IO manager so we have to flush it to disk.
+ */
+ if (!(inode->i_flags & EXT4_EXTENTS_FL) &&
+ old_iblocks != ext2fs_get_stat_i_blocks(fs, EXT2_INODE(inode))) {
+ err = io_channel_flush(fs->io);
+ if (err)
+ return translate_error(fs, ino, err);
+ }
+
+ /* pick up the newly allocated mapping */
+ ret = fuse4fs_iomap_begin_read(ff, ino, inode, pos, count, opflags,
+ read);
+ if (ret)
+ return ret;
+
+ read->flags |= FUSE_IOMAP_F_DIRTY;
+ *dirty = true;
+ return 0;
+}
+
+static off_t fuse4fs_max_file_size(const struct fuse4fs *ff,
+ const struct ext2_inode_large *inode)
+{
+ ext2_filsys fs = ff->fs;
+ blk64_t addr_per_block, max_map_block;
+
+ if (inode->i_flags & EXT4_EXTENTS_FL) {
+ max_map_block = (1ULL << 32) - 1;
+ } else {
+ addr_per_block = fs->blocksize >> 2;
+ max_map_block = addr_per_block;
+ max_map_block += addr_per_block * addr_per_block;
+ max_map_block += addr_per_block * addr_per_block * addr_per_block;
+ max_map_block += 12;
+ }
+
+ return FUSE4FS_FSB_TO_B(ff, max_map_block) + (fs->blocksize - 1);
+}
+
static int fuse4fs_iomap_begin_write(struct fuse4fs *ff, ext2_ino_t ino,
struct ext2_inode_large *inode, off_t pos,
uint64_t count, uint32_t opflags,
- struct fuse_file_iomap *read)
+ struct fuse_file_iomap *read,
+ bool *dirty)
{
- return -ENOSYS;
+ off_t max_size = fuse4fs_max_file_size(ff, inode);
+ int ret;
+
+ if (!(opflags & FUSE_IOMAP_OP_DIRECT))
+ return -ENOSYS;
+
+ if (pos >= max_size)
+ return -EFBIG;
+
+ if (pos >= max_size - count)
+ count = max_size - pos;
+
+ ret = fuse4fs_iomap_begin_read(ff, ino, inode, pos, count, opflags,
+ read);
+ if (ret)
+ return ret;
+
+ if (fuse_iomap_need_write_allocate(opflags, read)) {
+ ret = fuse4fs_iomap_write_allocate(ff, ino, inode, pos, count,
+ opflags, read, dirty);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
}
static void op_iomap_begin(fuse_req_t req, fuse_ino_t fino, uint64_t dontcare,
@@ -6470,6 +6564,7 @@ static void op_iomap_begin(fuse_req_t req, fuse_ino_t fino, uint64_t dontcare,
ext2_filsys fs;
ext2_ino_t ino;
errcode_t err;
+ bool dirty = false;
int ret = 0;
FUSE4FS_CHECK_CONTEXT(req);
@@ -6493,7 +6588,7 @@ static void op_iomap_begin(fuse_req_t req, fuse_ino_t fino, uint64_t dontcare,
opflags, &read);
else if (fuse_iomap_is_write(opflags))
ret = fuse4fs_iomap_begin_write(ff, ino, &inode, pos, count,
- opflags, &read);
+ opflags, &read, &dirty);
else
ret = fuse4fs_iomap_begin_read(ff, ino, &inode, pos, count,
opflags, &read);
@@ -6518,6 +6613,14 @@ static void op_iomap_begin(fuse_req_t req, fuse_ino_t fino, uint64_t dontcare,
goto out_unlock;
}
+ if (dirty) {
+ err = fuse4fs_write_inode(fs, ino, &inode);
+ if (err) {
+ ret = translate_error(fs, ino, err);
+ goto out_unlock;
+ }
+ }
+
out_unlock:
fuse4fs_finish(ff, ret);
if (ret)
@@ -6667,6 +6770,373 @@ static void op_iomap_config(fuse_req_t req,
else
fuse_reply_iomap_config(req, &cfg);
}
+
+static inline bool fuse4fs_can_merge_mappings(const struct ext2fs_extent *left,
+ const struct ext2fs_extent *right)
+{
+ uint64_t max_len = (left->e_flags & EXT2_EXTENT_FLAGS_UNINIT) ?
+ EXT_UNINIT_MAX_LEN : EXT_INIT_MAX_LEN;
+
+ return left->e_lblk + left->e_len == right->e_lblk &&
+ left->e_pblk + left->e_len == right->e_pblk &&
+ (left->e_flags & EXT2_EXTENT_FLAGS_UNINIT) ==
+ (right->e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (uint64_t)left->e_len + right->e_len <= max_len;
+}
+
+static int fuse4fs_try_merge_mappings(struct fuse4fs *ff, ext2_ino_t ino,
+ ext2_extent_handle_t handle,
+ blk64_t startoff)
+{
+ ext2_filsys fs = ff->fs;
+ struct ext2fs_extent left, right;
+ errcode_t err;
+
+ /* Look up the mappings before startoff */
+ err = fuse4fs_get_mapping_at(ff, handle, startoff - 1, &left);
+ if (err == EXT2_ET_EXTENT_NOT_FOUND)
+ return 0;
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /* Look up the mapping at startoff */
+ err = fuse4fs_get_mapping_at(ff, handle, startoff, &right);
+ if (err == EXT2_ET_EXTENT_NOT_FOUND)
+ return 0;
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /* Can we combine them? */
+ if (!fuse4fs_can_merge_mappings(&left, &right))
+ return 0;
+
+ /*
+ * Delete the mapping after startoff because libext2fs cannot handle
+ * overlapping mappings.
+ */
+ err = ext2fs_extent_delete(handle, 0);
+ DUMP_EXTENT(ff, "remover", startoff, err, &right);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixremover", startoff, err, &right);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /* Move back and lengthen the mapping before startoff */
+ err = ext2fs_extent_goto(handle, left.e_lblk);
+ DUMP_EXTENT(ff, "movel", startoff - 1, err, &left);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ left.e_len += right.e_len;
+ err = ext2fs_extent_replace(handle, 0, &left);
+ DUMP_EXTENT(ff, "replacel", startoff - 1, err, &left);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixreplacel", startoff - 1, err, &left);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ return 0;
+}
+
+static int fuse4fs_convert_unwritten_mapping(struct fuse4fs *ff,
+ ext2_ino_t ino,
+ struct ext2_inode_large *inode,
+ ext2_extent_handle_t handle,
+ blk64_t *cursor, blk64_t stopoff)
+{
+ ext2_filsys fs = ff->fs;
+ struct ext2fs_extent extent;
+ blk64_t startoff = *cursor;
+ errcode_t err;
+
+ /*
+ * Find the mapping at startoff. Note that we can find holes because
+ * the mapping data can change due to racing writes.
+ */
+ err = fuse4fs_get_mapping_at(ff, handle, startoff, &extent);
+ if (err == EXT2_ET_EXTENT_NOT_FOUND) {
+ /*
+ * If we didn't find any mappings at all then the file is
+ * completely sparse. There's nothing to convert.
+ */
+ *cursor = stopoff;
+ return 0;
+ }
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /*
+ * The mapping is completely to the left of the range that we want.
+ * Let's see what's in the next extent, if there is one.
+ */
+ if (startoff >= extent.e_lblk + extent.e_len) {
+ /*
+ * Mapping ends to the left of the current position. Try to
+ * find the next mapping. If there is no next mapping, then
+ * we're done.
+ */
+ err = fuse4fs_get_next_mapping(ff, handle, startoff, &extent);
+ if (err == EXT2_ET_EXTENT_NOT_FOUND) {
+ *cursor = stopoff;
+ return 0;
+ }
+ if (err)
+ return translate_error(fs, ino, err);
+ }
+
+ /*
+ * The mapping is completely to the right of the range that we want,
+ * so we're done.
+ */
+ if (extent.e_lblk >= stopoff) {
+ *cursor = stopoff;
+ return 0;
+ }
+
+ /*
+ * At this point, we have a mapping that overlaps (startoff, stopoff].
+ * If the mapping is already written, move on to the next one.
+ */
+ if (!(extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT))
+ goto next;
+
+ if (startoff > extent.e_lblk) {
+ struct ext2fs_extent newex = extent;
+
+ /*
+ * Unwritten mapping starts before startoff. Shorten
+ * the previous mapping...
+ */
+ newex.e_len = startoff - extent.e_lblk;
+ err = ext2fs_extent_replace(handle, 0, &newex);
+ DUMP_EXTENT(ff, "shortenp", startoff, err, &newex);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixshortenp", startoff, err, &newex);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /* ...and create new written mapping at startoff. */
+ extent.e_len -= newex.e_len;
+ extent.e_lblk += newex.e_len;
+ extent.e_pblk += newex.e_len;
+ extent.e_flags = newex.e_flags & ~EXT2_EXTENT_FLAGS_UNINIT;
+
+ err = ext2fs_extent_insert(handle,
+ EXT2_EXTENT_INSERT_AFTER,
+ &extent);
+ DUMP_EXTENT(ff, "insertx", startoff, err, &extent);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixinsertx", startoff, err, &extent);
+ if (err)
+ return translate_error(fs, ino, err);
+ }
+
+ if (extent.e_lblk + extent.e_len > stopoff) {
+ struct ext2fs_extent newex = extent;
+
+ /*
+ * Unwritten mapping ends after stopoff. Shorten the current
+ * mapping...
+ */
+ extent.e_len = stopoff - extent.e_lblk;
+ extent.e_flags &= ~EXT2_EXTENT_FLAGS_UNINIT;
+
+ err = ext2fs_extent_replace(handle, 0, &extent);
+ DUMP_EXTENT(ff, "shortenn", startoff, err, &extent);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixshortenn", startoff, err, &extent);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /* ..and create a new unwritten mapping at stopoff. */
+ newex.e_pblk += extent.e_len;
+ newex.e_lblk += extent.e_len;
+ newex.e_len -= extent.e_len;
+ newex.e_flags |= EXT2_EXTENT_FLAGS_UNINIT;
+
+ err = ext2fs_extent_insert(handle,
+ EXT2_EXTENT_INSERT_AFTER,
+ &newex);
+ DUMP_EXTENT(ff, "insertn", startoff, err, &newex);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixinsertn", startoff, err, &newex);
+ if (err)
+ return translate_error(fs, ino, err);
+ }
+
+ /* Still unwritten? Update the state. */
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT) {
+ extent.e_flags &= ~EXT2_EXTENT_FLAGS_UNINIT;
+
+ err = ext2fs_extent_replace(handle, 0, &extent);
+ DUMP_EXTENT(ff, "replacex", startoff, err, &extent);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixreplacex", startoff, err, &extent);
+ if (err)
+ return translate_error(fs, ino, err);
+ }
+
+next:
+ /* Try to merge with the previous extent */
+ if (startoff > 0) {
+ err = fuse4fs_try_merge_mappings(ff, ino, handle, startoff);
+ if (err)
+ return translate_error(fs, ino, err);
+ }
+
+ *cursor = extent.e_lblk + extent.e_len;
+ return 0;
+}
+
+static int fuse4fs_convert_unwritten_mappings(struct fuse4fs *ff,
+ ext2_ino_t ino,
+ struct ext2_inode_large *inode,
+ off_t pos, size_t written)
+{
+ ext2_extent_handle_t handle;
+ ext2_filsys fs = ff->fs;
+ blk64_t startoff = FUSE4FS_B_TO_FSBT(ff, pos);
+ const blk64_t stopoff = FUSE4FS_B_TO_FSB(ff, pos + written);
+ errcode_t err;
+ int ret;
+
+ err = ext2fs_extent_open2(fs, ino, EXT2_INODE(inode), &handle);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /* Walk every mapping in the range, converting them. */
+ while (startoff < stopoff) {
+ blk64_t old_startoff = startoff;
+
+ ret = fuse4fs_convert_unwritten_mapping(ff, ino, inode, handle,
+ &startoff, stopoff);
+ if (ret)
+ goto out_handle;
+ if (startoff <= old_startoff) {
+ /* Do not go backwards. */
+ ret = translate_error(fs, ino, EXT2_ET_INODE_CORRUPTED);
+ goto out_handle;
+ }
+ }
+
+ /* Try to merge the right edge */
+ ret = fuse4fs_try_merge_mappings(ff, ino, handle, stopoff);
+out_handle:
+ ext2fs_extent_free(handle);
+ return ret;
+}
+
+static void op_iomap_ioend(fuse_req_t req, fuse_ino_t fino, uint64_t dontcare,
+ off_t pos, size_t written, uint32_t ioendflags,
+ int error, uint32_t dev, uint64_t new_addr)
+{
+ struct fuse4fs *ff = fuse4fs_get(req);
+ struct ext2_inode_large inode;
+ ext2_filsys fs;
+ ext2_ino_t ino;
+ ext2_off64_t isize;
+ errcode_t err;
+ bool dirty = false;
+ off_t newsize = -1;
+ int ret = 0;
+
+ FUSE4FS_CHECK_CONTEXT(req);
+ FUSE4FS_CONVERT_FINO(req, &ino, fino);
+
+ dbg_printf(ff,
+ "%s: ino=%d pos=0x%llx written=0x%zx ioendflags=0x%x error=%d dev=%u new_addr=0x%llx\n",
+ __func__, ino,
+ (unsigned long long)pos,
+ written,
+ ioendflags,
+ error,
+ dev,
+ (unsigned long long)new_addr);
+
+ if (error) {
+ fuse_reply_err(req, -error);
+ return;
+ }
+
+ fs = fuse4fs_start(ff);
+
+ /* should never see these ioend types */
+ if (ioendflags & FUSE_IOMAP_IOEND_SHARED) {
+ ret = translate_error(fs, ino, EXT2_ET_FILESYSTEM_CORRUPTED);
+ goto out_unlock;
+ }
+
+ err = fuse4fs_read_inode(fs, ino, &inode);
+ if (err) {
+ ret = translate_error(fs, ino, err);
+ goto out_unlock;
+ }
+
+ if (ioendflags & FUSE_IOMAP_IOEND_UNWRITTEN) {
+ /* unwritten extents are only supported on extents files */
+ if (!(inode.i_flags & EXT4_EXTENTS_FL)) {
+ ret = translate_error(fs, ino,
+ EXT2_ET_FILESYSTEM_CORRUPTED);
+ goto out_unlock;
+ }
+
+ ret = fuse4fs_convert_unwritten_mappings(ff, ino, &inode,
+ pos, written);
+ if (ret)
+ goto out_unlock;
+
+ dirty = true;
+ }
+
+ isize = EXT2_I_SIZE(&inode);
+ if (pos + written > isize) {
+ err = ext2fs_inode_size_set(fs, EXT2_INODE(&inode),
+ pos + written);
+ if (err) {
+ ret = translate_error(fs, ino, err);
+ goto out_unlock;
+ }
+
+ dirty = true;
+ }
+
+ if (dirty) {
+ err = fuse4fs_write_inode(fs, ino, &inode);
+ if (err) {
+ ret = translate_error(fs, ino, err);
+ goto out_unlock;
+ }
+ }
+
+ newsize = EXT2_I_SIZE(&inode);
+out_unlock:
+ fuse4fs_finish(ff, ret);
+ if (ret)
+ fuse_reply_err(req, -ret);
+ else
+ fuse_reply_iomap_ioend(req, newsize);
+}
#endif /* HAVE_FUSE_IOMAP */
static struct fuse_lowlevel_ops fs_ops = {
@@ -6716,6 +7186,7 @@ static struct fuse_lowlevel_ops fs_ops = {
.iomap_begin = op_iomap_begin,
.iomap_end = op_iomap_end,
.iomap_config = op_iomap_config,
+ .iomap_ioend = op_iomap_ioend,
#endif /* HAVE_FUSE_IOMAP */
};
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index 4b37bcde63d0f2..67a5bc4c5cc986 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -5688,12 +5688,103 @@ static int fuse2fs_iomap_begin_read(struct fuse2fs *ff, ext2_ino_t ino,
opflags, read);
}
+static int fuse2fs_iomap_write_allocate(struct fuse2fs *ff, ext2_ino_t ino,
+ struct ext2_inode_large *inode, off_t pos,
+ uint64_t count, uint32_t opflags,
+ struct fuse_file_iomap *read, bool *dirty)
+{
+ ext2_filsys fs = ff->fs;
+ blk64_t startoff = FUSE2FS_B_TO_FSBT(ff, pos);
+ blk64_t stopoff = FUSE2FS_B_TO_FSB(ff, pos + count);
+ blk64_t old_iblocks;
+ errcode_t err;
+ int ret;
+
+ dbg_printf(ff, "%s: write_alloc ino=%u startoff 0x%llx blockcount 0x%llx\n",
+ __func__, ino, startoff, stopoff - startoff);
+
+ if (!fs_can_allocate(ff, stopoff - startoff))
+ return -ENOSPC;
+
+ old_iblocks = ext2fs_get_stat_i_blocks(fs, EXT2_INODE(inode));
+ err = ext2fs_fallocate(fs, EXT2_FALLOCATE_FORCE_UNINIT, ino,
+ EXT2_INODE(inode), ~0ULL, startoff,
+ stopoff - startoff);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /*
+ * New allocations for file data blocks on indirect mapped files are
+ * zeroed through the IO manager so we have to flush it to disk.
+ */
+ if (!(inode->i_flags & EXT4_EXTENTS_FL) &&
+ old_iblocks != ext2fs_get_stat_i_blocks(fs, EXT2_INODE(inode))) {
+ err = io_channel_flush(fs->io);
+ if (err)
+ return translate_error(fs, ino, err);
+ }
+
+ /* pick up the newly allocated mapping */
+ ret = fuse2fs_iomap_begin_read(ff, ino, inode, pos, count, opflags,
+ read);
+ if (ret)
+ return ret;
+
+ read->flags |= FUSE_IOMAP_F_DIRTY;
+ *dirty = true;
+ return 0;
+}
+
+static off_t fuse2fs_max_file_size(const struct fuse2fs *ff,
+ const struct ext2_inode_large *inode)
+{
+ ext2_filsys fs = ff->fs;
+ blk64_t addr_per_block, max_map_block;
+
+ if (inode->i_flags & EXT4_EXTENTS_FL) {
+ max_map_block = (1ULL << 32) - 1;
+ } else {
+ addr_per_block = fs->blocksize >> 2;
+ max_map_block = addr_per_block;
+ max_map_block += addr_per_block * addr_per_block;
+ max_map_block += addr_per_block * addr_per_block * addr_per_block;
+ max_map_block += 12;
+ }
+
+ return FUSE2FS_FSB_TO_B(ff, max_map_block) + (fs->blocksize - 1);
+}
+
static int fuse2fs_iomap_begin_write(struct fuse2fs *ff, ext2_ino_t ino,
struct ext2_inode_large *inode, off_t pos,
uint64_t count, uint32_t opflags,
- struct fuse_file_iomap *read)
+ struct fuse_file_iomap *read,
+ bool *dirty)
{
- return -ENOSYS;
+ off_t max_size = fuse2fs_max_file_size(ff, inode);
+ int ret;
+
+ if (!(opflags & FUSE_IOMAP_OP_DIRECT))
+ return -ENOSYS;
+
+ if (pos >= max_size)
+ return -EFBIG;
+
+ if (pos >= max_size - count)
+ count = max_size - pos;
+
+ ret = fuse2fs_iomap_begin_read(ff, ino, inode, pos, count, opflags,
+ read);
+ if (ret)
+ return ret;
+
+ if (fuse_iomap_need_write_allocate(opflags, read)) {
+ ret = fuse2fs_iomap_write_allocate(ff, ino, inode, pos, count,
+ opflags, read, dirty);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
}
static int op_iomap_begin(const char *path, uint64_t nodeid, uint64_t attr_ino,
@@ -5705,6 +5796,7 @@ static int op_iomap_begin(const char *path, uint64_t nodeid, uint64_t attr_ino,
struct ext2_inode_large inode;
ext2_filsys fs;
errcode_t err;
+ bool dirty = false;
int ret = 0;
FUSE2FS_CHECK_CONTEXT(ff);
@@ -5730,7 +5822,7 @@ static int op_iomap_begin(const char *path, uint64_t nodeid, uint64_t attr_ino,
count, opflags, read);
else if (fuse_iomap_is_write(opflags))
ret = fuse2fs_iomap_begin_write(ff, attr_ino, &inode, pos,
- count, opflags, read);
+ count, opflags, read, &dirty);
else
ret = fuse2fs_iomap_begin_read(ff, attr_ino, &inode, pos,
count, opflags, read);
@@ -5755,6 +5847,14 @@ static int op_iomap_begin(const char *path, uint64_t nodeid, uint64_t attr_ino,
goto out_unlock;
}
+ if (dirty) {
+ err = fuse2fs_write_inode(fs, attr_ino, &inode);
+ if (err) {
+ ret = translate_error(fs, attr_ino, err);
+ goto out_unlock;
+ }
+ }
+
out_unlock:
fuse2fs_finish(ff, ret);
return ret;
@@ -5896,6 +5996,370 @@ static int op_iomap_config(const struct fuse_iomap_config_params *p,
fuse2fs_finish(ff, ret);
return ret;
}
+
+static inline bool fuse2fs_can_merge_mappings(const struct ext2fs_extent *left,
+ const struct ext2fs_extent *right)
+{
+ uint64_t max_len = (left->e_flags & EXT2_EXTENT_FLAGS_UNINIT) ?
+ EXT_UNINIT_MAX_LEN : EXT_INIT_MAX_LEN;
+
+ return left->e_lblk + left->e_len == right->e_lblk &&
+ left->e_pblk + left->e_len == right->e_pblk &&
+ (left->e_flags & EXT2_EXTENT_FLAGS_UNINIT) ==
+ (right->e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (uint64_t)left->e_len + right->e_len <= max_len;
+}
+
+static int fuse2fs_try_merge_mappings(struct fuse2fs *ff, ext2_ino_t ino,
+ ext2_extent_handle_t handle,
+ blk64_t startoff)
+{
+ ext2_filsys fs = ff->fs;
+ struct ext2fs_extent left, right;
+ errcode_t err;
+
+ /* Look up the mappings before startoff */
+ err = fuse2fs_get_mapping_at(ff, handle, startoff - 1, &left);
+ if (err == EXT2_ET_EXTENT_NOT_FOUND)
+ return 0;
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /* Look up the mapping at startoff */
+ err = fuse2fs_get_mapping_at(ff, handle, startoff, &right);
+ if (err == EXT2_ET_EXTENT_NOT_FOUND)
+ return 0;
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /* Can we combine them? */
+ if (!fuse2fs_can_merge_mappings(&left, &right))
+ return 0;
+
+ /*
+ * Delete the mapping after startoff because libext2fs cannot handle
+ * overlapping mappings.
+ */
+ err = ext2fs_extent_delete(handle, 0);
+ DUMP_EXTENT(ff, "remover", startoff, err, &right);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixremover", startoff, err, &right);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /* Move back and lengthen the mapping before startoff */
+ err = ext2fs_extent_goto(handle, left.e_lblk);
+ DUMP_EXTENT(ff, "movel", startoff - 1, err, &left);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ left.e_len += right.e_len;
+ err = ext2fs_extent_replace(handle, 0, &left);
+ DUMP_EXTENT(ff, "replacel", startoff - 1, err, &left);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixreplacel", startoff - 1, err, &left);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ return 0;
+}
+
+static int fuse2fs_convert_unwritten_mapping(struct fuse2fs *ff,
+ ext2_ino_t ino,
+ struct ext2_inode_large *inode,
+ ext2_extent_handle_t handle,
+ blk64_t *cursor, blk64_t stopoff)
+{
+ ext2_filsys fs = ff->fs;
+ struct ext2fs_extent extent;
+ blk64_t startoff = *cursor;
+ errcode_t err;
+
+ /*
+ * Find the mapping at startoff. Note that we can find holes because
+ * the mapping data can change due to racing writes.
+ */
+ err = fuse2fs_get_mapping_at(ff, handle, startoff, &extent);
+ if (err == EXT2_ET_EXTENT_NOT_FOUND) {
+ /*
+ * If we didn't find any mappings at all then the file is
+ * completely sparse. There's nothing to convert.
+ */
+ *cursor = stopoff;
+ return 0;
+ }
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /*
+ * The mapping is completely to the left of the range that we want.
+ * Let's see what's in the next extent, if there is one.
+ */
+ if (startoff >= extent.e_lblk + extent.e_len) {
+ /*
+ * Mapping ends to the left of the current position. Try to
+ * find the next mapping. If there is no next mapping, then
+ * we're done.
+ */
+ err = fuse2fs_get_next_mapping(ff, handle, startoff, &extent);
+ if (err == EXT2_ET_EXTENT_NOT_FOUND) {
+ *cursor = stopoff;
+ return 0;
+ }
+ if (err)
+ return translate_error(fs, ino, err);
+ }
+
+ /*
+ * The mapping is completely to the right of the range that we want,
+ * so we're done.
+ */
+ if (extent.e_lblk >= stopoff) {
+ *cursor = stopoff;
+ return 0;
+ }
+
+ /*
+ * At this point, we have a mapping that overlaps (startoff, stopoff].
+ * If the mapping is already written, move on to the next one.
+ */
+ if (!(extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT))
+ goto next;
+
+ if (startoff > extent.e_lblk) {
+ struct ext2fs_extent newex = extent;
+
+ /*
+ * Unwritten mapping starts before startoff. Shorten
+ * the previous mapping...
+ */
+ newex.e_len = startoff - extent.e_lblk;
+ err = ext2fs_extent_replace(handle, 0, &newex);
+ DUMP_EXTENT(ff, "shortenp", startoff, err, &newex);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixshortenp", startoff, err, &newex);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /* ...and create new written mapping at startoff. */
+ extent.e_len -= newex.e_len;
+ extent.e_lblk += newex.e_len;
+ extent.e_pblk += newex.e_len;
+ extent.e_flags = newex.e_flags & ~EXT2_EXTENT_FLAGS_UNINIT;
+
+ err = ext2fs_extent_insert(handle,
+ EXT2_EXTENT_INSERT_AFTER,
+ &extent);
+ DUMP_EXTENT(ff, "insertx", startoff, err, &extent);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixinsertx", startoff, err, &extent);
+ if (err)
+ return translate_error(fs, ino, err);
+ }
+
+ if (extent.e_lblk + extent.e_len > stopoff) {
+ struct ext2fs_extent newex = extent;
+
+ /*
+ * Unwritten mapping ends after stopoff. Shorten the current
+ * mapping...
+ */
+ extent.e_len = stopoff - extent.e_lblk;
+ extent.e_flags &= ~EXT2_EXTENT_FLAGS_UNINIT;
+
+ err = ext2fs_extent_replace(handle, 0, &extent);
+ DUMP_EXTENT(ff, "shortenn", startoff, err, &extent);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixshortenn", startoff, err, &extent);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /* ..and create a new unwritten mapping at stopoff. */
+ newex.e_pblk += extent.e_len;
+ newex.e_lblk += extent.e_len;
+ newex.e_len -= extent.e_len;
+ newex.e_flags |= EXT2_EXTENT_FLAGS_UNINIT;
+
+ err = ext2fs_extent_insert(handle,
+ EXT2_EXTENT_INSERT_AFTER,
+ &newex);
+ DUMP_EXTENT(ff, "insertn", startoff, err, &newex);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixinsertn", startoff, err, &newex);
+ if (err)
+ return translate_error(fs, ino, err);
+ }
+
+ /* Still unwritten? Update the state. */
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT) {
+ extent.e_flags &= ~EXT2_EXTENT_FLAGS_UNINIT;
+
+ err = ext2fs_extent_replace(handle, 0, &extent);
+ DUMP_EXTENT(ff, "replacex", startoff, err, &extent);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = ext2fs_extent_fix_parents(handle);
+ DUMP_EXTENT(ff, "fixreplacex", startoff, err, &extent);
+ if (err)
+ return translate_error(fs, ino, err);
+ }
+
+next:
+ /* Try to merge with the previous extent */
+ if (startoff > 0) {
+ err = fuse2fs_try_merge_mappings(ff, ino, handle, startoff);
+ if (err)
+ return translate_error(fs, ino, err);
+ }
+
+ *cursor = extent.e_lblk + extent.e_len;
+ return 0;
+}
+
+static int fuse2fs_convert_unwritten_mappings(struct fuse2fs *ff,
+ ext2_ino_t ino,
+ struct ext2_inode_large *inode,
+ off_t pos, size_t written)
+{
+ ext2_extent_handle_t handle;
+ ext2_filsys fs = ff->fs;
+ blk64_t startoff = FUSE2FS_B_TO_FSBT(ff, pos);
+ const blk64_t stopoff = FUSE2FS_B_TO_FSB(ff, pos + written);
+ errcode_t err;
+ int ret;
+
+ err = ext2fs_extent_open2(fs, ino, EXT2_INODE(inode), &handle);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ /* Walk every mapping in the range, converting them. */
+ while (startoff < stopoff) {
+ blk64_t old_startoff = startoff;
+
+ ret = fuse2fs_convert_unwritten_mapping(ff, ino, inode, handle,
+ &startoff, stopoff);
+ if (ret)
+ goto out_handle;
+ if (startoff <= old_startoff) {
+ /* Do not go backwards. */
+ ret = translate_error(fs, ino, EXT2_ET_INODE_CORRUPTED);
+ goto out_handle;
+ }
+ }
+
+ /* Try to merge the right edge */
+ ret = fuse2fs_try_merge_mappings(ff, ino, handle, stopoff);
+out_handle:
+ ext2fs_extent_free(handle);
+ return ret;
+}
+
+static int op_iomap_ioend(const char *path, uint64_t nodeid, uint64_t attr_ino,
+ off_t pos, size_t written, uint32_t ioendflags,
+ int error, uint32_t dev, uint64_t new_addr,
+ off_t *newsize)
+{
+ struct fuse2fs *ff = fuse2fs_get();
+ struct ext2_inode_large inode;
+ ext2_filsys fs;
+ errcode_t err;
+ ext2_off64_t isize;
+ bool dirty = false;
+ int ret = 0;
+
+ FUSE2FS_CHECK_CONTEXT(ff);
+
+ dbg_printf(ff,
+ "%s: path=%s nodeid=%llu attr_ino=%llu pos=0x%llx written=0x%zx ioendflags=0x%x error=%d dev=%u new_addr=%llu\n",
+ __func__, path,
+ (unsigned long long)nodeid,
+ (unsigned long long)attr_ino,
+ (unsigned long long)pos,
+ written,
+ ioendflags,
+ error,
+ dev,
+ (unsigned long long)new_addr);
+
+ fs = fuse2fs_start(ff);
+ if (error) {
+ ret = error;
+ goto out_unlock;
+ }
+
+ /* should never see these ioend types */
+ if (ioendflags & FUSE_IOMAP_IOEND_SHARED) {
+ ret = translate_error(fs, attr_ino,
+ EXT2_ET_FILESYSTEM_CORRUPTED);
+ goto out_unlock;
+ }
+
+ err = fuse2fs_read_inode(fs, attr_ino, &inode);
+ if (err) {
+ ret = translate_error(fs, attr_ino, err);
+ goto out_unlock;
+ }
+
+ if (ioendflags & FUSE_IOMAP_IOEND_UNWRITTEN) {
+ /* unwritten extents are only supported on extents files */
+ if (!(inode.i_flags & EXT4_EXTENTS_FL)) {
+ ret = translate_error(fs, attr_ino,
+ EXT2_ET_FILESYSTEM_CORRUPTED);
+ goto out_unlock;
+ }
+
+ ret = fuse2fs_convert_unwritten_mappings(ff, attr_ino, &inode,
+ pos, written);
+ if (ret)
+ goto out_unlock;
+
+ dirty = true;
+ }
+
+ isize = EXT2_I_SIZE(&inode);
+ if (pos + written > isize) {
+ err = ext2fs_inode_size_set(fs, EXT2_INODE(&inode),
+ pos + written);
+ if (err) {
+ ret = translate_error(fs, attr_ino, err);
+ goto out_unlock;
+ }
+
+ dirty = true;
+ }
+
+ if (dirty) {
+ err = fuse2fs_write_inode(fs, attr_ino, &inode);
+ if (err) {
+ ret = translate_error(fs, attr_ino, err);
+ goto out_unlock;
+ }
+ }
+
+ *newsize = EXT2_I_SIZE(&inode);
+out_unlock:
+ fuse2fs_finish(ff, ret);
+ return ret;
+}
#endif /* HAVE_FUSE_IOMAP */
static struct fuse_operations fs_ops = {
@@ -5943,6 +6407,7 @@ static struct fuse_operations fs_ops = {
.iomap_begin = op_iomap_begin,
.iomap_end = op_iomap_end,
.iomap_config = op_iomap_config,
+ .iomap_ioend = op_iomap_ioend,
#endif /* HAVE_FUSE_IOMAP */
};
^ permalink raw reply related
* [PATCH 08/19] fuse2fs: turn on iomap for pagecache IO
From: Darrick J. Wong @ 2026-04-29 14:54 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214459.4107473.9520724883867588970.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
Turn on iomap for pagecache IO to regular files.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
fuse4fs/fuse4fs.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++------
misc/fuse2fs.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++------
2 files changed, 108 insertions(+), 14 deletions(-)
diff --git a/fuse4fs/fuse4fs.c b/fuse4fs/fuse4fs.c
index 8b508de5b8cb65..fa82fda99ff687 100644
--- a/fuse4fs/fuse4fs.c
+++ b/fuse4fs/fuse4fs.c
@@ -6438,9 +6438,6 @@ static int fuse4fs_iomap_begin_read(struct fuse4fs *ff, ext2_ino_t ino,
uint64_t count, uint32_t opflags,
struct fuse_file_iomap *read)
{
- if (!(opflags & FUSE_IOMAP_OP_DIRECT))
- return -ENOSYS;
-
/* fall back to slow path for inline data reads */
if (inode->i_flags & EXT4_INLINE_DATA_FL)
return -ENOSYS;
@@ -6531,9 +6528,6 @@ static int fuse4fs_iomap_begin_write(struct fuse4fs *ff, ext2_ino_t ino,
off_t max_size = fuse4fs_max_file_size(ff, inode);
int ret;
- if (!(opflags & FUSE_IOMAP_OP_DIRECT))
- return -ENOSYS;
-
if (pos >= max_size)
return -EFBIG;
@@ -6629,12 +6623,51 @@ static void op_iomap_begin(fuse_req_t req, fuse_ino_t fino, uint64_t dontcare,
fuse_reply_iomap_begin(req, &read, NULL);
}
+static int fuse4fs_iomap_append_setsize(struct fuse4fs *ff, ext2_ino_t ino,
+ loff_t newsize)
+{
+ ext2_filsys fs = ff->fs;
+ struct ext2_inode_large inode;
+ ext2_off64_t isize;
+ errcode_t err;
+
+ dbg_printf(ff, "%s: ino=%u newsize=%llu\n", __func__, ino,
+ (unsigned long long)newsize);
+
+ err = fuse4fs_read_inode(fs, ino, &inode);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ isize = EXT2_I_SIZE(&inode);
+ if (newsize <= isize)
+ return 0;
+
+ dbg_printf(ff, "%s: ino=%u oldsize=%llu newsize=%llu\n", __func__, ino,
+ (unsigned long long)isize,
+ (unsigned long long)newsize);
+
+ /*
+ * XXX cheesily update the ondisk size even though we only want to do
+ * the incore size until writeback happens
+ */
+ err = ext2fs_inode_size_set(fs, EXT2_INODE(&inode), newsize);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = fuse4fs_write_inode(fs, ino, &inode);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ return 0;
+}
+
static void op_iomap_end(fuse_req_t req, fuse_ino_t fino, uint64_t dontcare,
off_t pos, uint64_t count, uint32_t opflags,
ssize_t written, const struct fuse_file_iomap *iomap)
{
struct fuse4fs *ff = fuse4fs_get(req);
ext2_ino_t ino;
+ int ret = 0;
FUSE4FS_CHECK_CONTEXT(req);
FUSE4FS_CONVERT_FINO(req, &ino, fino);
@@ -6648,7 +6681,21 @@ static void op_iomap_end(fuse_req_t req, fuse_ino_t fino, uint64_t dontcare,
written,
iomap->flags);
- fuse_reply_err(req, 0);
+ fuse4fs_start(ff);
+
+ /* XXX is this really necessary? */
+ if ((opflags & FUSE_IOMAP_OP_WRITE) &&
+ !(opflags & FUSE_IOMAP_OP_DIRECT) &&
+ (iomap->flags & FUSE_IOMAP_F_SIZE_CHANGED) &&
+ written > 0) {
+ ret = fuse4fs_iomap_append_setsize(ff, ino, pos + written);
+ if (ret)
+ goto out_unlock;
+ }
+
+out_unlock:
+ fuse4fs_finish(ff, ret);
+ fuse_reply_err(req, -ret);
}
/*
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index 67a5bc4c5cc986..679406323df86f 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -5673,9 +5673,6 @@ static int fuse2fs_iomap_begin_read(struct fuse2fs *ff, ext2_ino_t ino,
uint64_t count, uint32_t opflags,
struct fuse_file_iomap *read)
{
- if (!(opflags & FUSE_IOMAP_OP_DIRECT))
- return -ENOSYS;
-
/* fall back to slow path for inline data reads */
if (inode->i_flags & EXT4_INLINE_DATA_FL)
return -ENOSYS;
@@ -5763,9 +5760,6 @@ static int fuse2fs_iomap_begin_write(struct fuse2fs *ff, ext2_ino_t ino,
off_t max_size = fuse2fs_max_file_size(ff, inode);
int ret;
- if (!(opflags & FUSE_IOMAP_OP_DIRECT))
- return -ENOSYS;
-
if (pos >= max_size)
return -EFBIG;
@@ -5860,11 +5854,50 @@ static int op_iomap_begin(const char *path, uint64_t nodeid, uint64_t attr_ino,
return ret;
}
+static int fuse2fs_iomap_append_setsize(struct fuse2fs *ff, ext2_ino_t ino,
+ loff_t newsize)
+{
+ ext2_filsys fs = ff->fs;
+ struct ext2_inode_large inode;
+ ext2_off64_t isize;
+ errcode_t err;
+
+ dbg_printf(ff, "%s: ino=%u newsize=%llu\n", __func__, ino,
+ (unsigned long long)newsize);
+
+ err = fuse2fs_read_inode(fs, ino, &inode);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ isize = EXT2_I_SIZE(&inode);
+ if (newsize <= isize)
+ return 0;
+
+ dbg_printf(ff, "%s: ino=%u oldsize=%llu newsize=%llu\n", __func__, ino,
+ (unsigned long long)isize,
+ (unsigned long long)newsize);
+
+ /*
+ * XXX cheesily update the ondisk size even though we only want to do
+ * the incore size until writeback happens
+ */
+ err = ext2fs_inode_size_set(fs, EXT2_INODE(&inode), newsize);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ err = fuse2fs_write_inode(fs, ino, &inode);
+ if (err)
+ return translate_error(fs, ino, err);
+
+ return 0;
+}
+
static int op_iomap_end(const char *path, uint64_t nodeid, uint64_t attr_ino,
off_t pos, uint64_t count, uint32_t opflags,
ssize_t written, const struct fuse_file_iomap *iomap)
{
struct fuse2fs *ff = fuse2fs_get();
+ int ret = 0;
FUSE2FS_CHECK_CONTEXT(ff);
@@ -5879,7 +5912,21 @@ static int op_iomap_end(const char *path, uint64_t nodeid, uint64_t attr_ino,
written,
iomap->flags);
- return 0;
+ fuse2fs_start(ff);
+
+ /* XXX is this really necessary? */
+ if ((opflags & FUSE_IOMAP_OP_WRITE) &&
+ !(opflags & FUSE_IOMAP_OP_DIRECT) &&
+ (iomap->flags & FUSE_IOMAP_F_SIZE_CHANGED) &&
+ written > 0) {
+ ret = fuse2fs_iomap_append_setsize(ff, attr_ino, pos + written);
+ if (ret)
+ goto out_unlock;
+ }
+
+out_unlock:
+ fuse2fs_finish(ff, ret);
+ return ret;
}
/*
^ permalink raw reply related
* [PATCH 09/19] fuse2fs: don't zero bytes in punch hole
From: Darrick J. Wong @ 2026-04-29 14:54 UTC (permalink / raw)
To: tytso
Cc: bernd, miklos, linux-ext4, neal, linux-fsdevel, fuse-devel,
joannelkoong
In-Reply-To: <177747214459.4107473.9520724883867588970.stgit@frogsfrogsfrogs>
From: Darrick J. Wong <djwong@kernel.org>
When iomap is in use for the pagecache, it will take care of zeroing the
unaligned parts of punched out regions so we don't have to do it
ourselves.
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
fuse4fs/fuse4fs.c | 8 ++++++++
misc/fuse2fs.c | 9 +++++++++
2 files changed, 17 insertions(+)
diff --git a/fuse4fs/fuse4fs.c b/fuse4fs/fuse4fs.c
index fa82fda99ff687..40713b0d0d5e37 100644
--- a/fuse4fs/fuse4fs.c
+++ b/fuse4fs/fuse4fs.c
@@ -5868,6 +5868,10 @@ static errcode_t fuse4fs_zero_middle(struct fuse4fs *ff, ext2_ino_t ino,
int retflags;
errcode_t err;
+ /* the kernel does this for us in iomap mode */
+ if (fuse4fs_iomap_enabled(ff))
+ return 0;
+
if (!*buf) {
err = ext2fs_get_mem(fs->blocksize, buf);
if (err)
@@ -5904,6 +5908,10 @@ static errcode_t fuse4fs_zero_edge(struct fuse4fs *ff, ext2_ino_t ino,
off_t residue;
errcode_t err;
+ /* the kernel does this for us in iomap mode */
+ if (fuse4fs_iomap_enabled(ff))
+ return 0;
+
residue = FUSE4FS_OFF_IN_FSB(ff, offset);
if (residue == 0)
return 0;
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index 679406323df86f..a37851cdf30785 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -727,6 +727,7 @@ static inline int fuse2fs_iomap_enabled(const struct fuse2fs *ff)
}
#else
# define fuse2fs_iomap_enabled(...) (0)
+# define fuse2fs_iomap_enabled(...) (0)
#endif
static inline void fuse2fs_dump_extents(struct fuse2fs *ff, ext2_ino_t ino,
@@ -5104,6 +5105,10 @@ static errcode_t clean_block_middle(struct fuse2fs *ff, ext2_ino_t ino,
int retflags;
errcode_t err;
+ /* the kernel does this for us in iomap mode */
+ if (fuse2fs_iomap_enabled(ff))
+ return 0;
+
if (!*buf) {
err = ext2fs_get_mem(fs->blocksize, buf);
if (err)
@@ -5140,6 +5145,10 @@ static errcode_t clean_block_edge(struct fuse2fs *ff, ext2_ino_t ino,
off_t residue;
errcode_t err;
+ /* the kernel does this for us in iomap mode */
+ if (fuse2fs_iomap_enabled(ff))
+ return 0;
+
residue = FUSE2FS_OFF_IN_FSB(ff, offset);
if (residue == 0)
return 0;
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox