From: "Darrick J. Wong" <djwong@kernel.org>
To: tytso@mit.edu
Cc: John@groves.net, linux-ext4@vger.kernel.org, miklos@szeredi.hu,
joannelkoong@gmail.com, bernd@bsbernd.com,
linux-fsdevel@vger.kernel.org
Subject: [PATCHSET RFC[RAP] 2/3] libext2fs: refactoring for fuse2fs iomap support
Date: Wed, 21 May 2025 17:02:09 -0700 [thread overview]
Message-ID: <174787198025.1484572.10345977324531146086.stgit@frogsfrogsfrogs> (raw)
In-Reply-To: <20250521235837.GB9688@frogsfrogsfrogs>
Hi all,
In preparation for connecting fuse, iomap, and fuse2fs for a much more
performant file IO path, make some changes to the Unix IO manager in
libext2fs so that we can have better IO. First we start by making
filesystem flushes a lot more efficient by eliding fsyncs when they're
not necessary, and allowing library clients to turn off the racy code
that writes the superblock byte by byte but exposes stale checksums.
XXX: The second part of this series adds IO tagging so that we could tag
IOs by inode number to distinguish file data blocks in cache from
everything else. This is temporary scaffolding whilst we're in the
middle adding directio and later buffered writes. Once we can use the
pagecache for all file IO activity I think we could drop the back half
of this series.
If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.
Comments and questions are, as always, welcome.
e2fsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/e2fsprogs.git/log/?h=libext2fs-iomap-prep
---
Commits in this patchset:
* libext2fs: always fsync the device when flushing the cache
* libext2fs: always fsync the device when closing the unix IO manager
* libext2fs: only fsync the unix fd if we wrote to the device
* libext2fs: invalidate cached blocks when freeing them
* libext2fs: add tagged block IO for better caching
* libext2fs: add tagged block IO caching to the unix IO manager
* libext2fs: only flush affected blocks in unix_write_byte
* libext2fs: allow unix_write_byte when the write would be aligned
* libext2fs: allow clients to ask to write full superblocks
* libext2fs: allow callers to disallow I/O to file data blocks
---
lib/ext2fs/ext2_io.h | 29 ++++
lib/ext2fs/ext2fs.h | 4 +
debian/libext2fs2t64.symbols | 5 +
lib/ext2fs/alloc_stats.c | 7 +
lib/ext2fs/closefs.c | 7 +
lib/ext2fs/fileio.c | 26 +++-
lib/ext2fs/io_manager.c | 56 ++++++++
lib/ext2fs/unix_io.c | 281 +++++++++++++++++++++++++++++++++++-------
8 files changed, 362 insertions(+), 53 deletions(-)
next prev parent reply other threads:[~2025-05-22 0:02 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-05-21 23:58 [RFC[RAP]] fuse: use fs-iomap for better performance so we can containerize ext4 Darrick J. Wong
2025-05-22 0:01 ` [PATCHSET 1/3] fuse2fs: upgrade to libfuse 3.17 Darrick J. Wong
2025-05-22 0:07 ` [PATCH 1/3] fuse2fs: bump library version Darrick J. Wong
2025-05-22 0:07 ` [PATCH 2/3] fuse2fs: wrap the fuse_set_feature_flag helper for older libfuse Darrick J. Wong
2025-05-22 0:08 ` [PATCH 3/3] fuse2fs: disable nfs exports Darrick J. Wong
2025-05-22 0:02 ` Darrick J. Wong [this message]
2025-05-22 0:08 ` [PATCH 01/10] libext2fs: always fsync the device when flushing the cache Darrick J. Wong
2025-05-22 0:08 ` [PATCH 02/10] libext2fs: always fsync the device when closing the unix IO manager Darrick J. Wong
2025-05-22 0:09 ` [PATCH 03/10] libext2fs: only fsync the unix fd if we wrote to the device Darrick J. Wong
2025-05-22 0:09 ` [PATCH 04/10] libext2fs: invalidate cached blocks when freeing them Darrick J. Wong
2025-05-22 0:09 ` [PATCH 05/10] libext2fs: add tagged block IO for better caching Darrick J. Wong
2025-05-22 0:09 ` [PATCH 06/10] libext2fs: add tagged block IO caching to the unix IO manager Darrick J. Wong
2025-05-22 0:10 ` [PATCH 07/10] libext2fs: only flush affected blocks in unix_write_byte Darrick J. Wong
2025-05-22 0:10 ` [PATCH 08/10] libext2fs: allow unix_write_byte when the write would be aligned Darrick J. Wong
2025-05-22 0:10 ` [PATCH 09/10] libext2fs: allow clients to ask to write full superblocks Darrick J. Wong
2025-05-22 0:10 ` [PATCH 10/10] libext2fs: allow callers to disallow I/O to file data blocks Darrick J. Wong
2025-05-22 0:02 ` [PATCHSET RFC[RAP] 3/3] fuse2fs: use fuse iomap data paths for better file I/O performance Darrick J. Wong
2025-05-22 0:11 ` [PATCH 01/16] fuse2fs: implement bare minimum iomap for file mapping reporting Darrick J. Wong
2025-05-22 0:11 ` [PATCH 02/16] fuse2fs: register block devices for use with iomap Darrick J. Wong
2025-05-22 0:11 ` [PATCH 03/16] fuse2fs: always use directio disk reads with fuse2fs Darrick J. Wong
2025-05-22 0:11 ` [PATCH 04/16] fuse2fs: implement directio file reads Darrick J. Wong
2025-05-22 0:12 ` [PATCH 05/16] fuse2fs: use tagged block IO for zeroing sub-block regions Darrick J. Wong
2025-05-22 0:12 ` [PATCH 06/16] fuse2fs: only flush the cache for the file under directio read Darrick J. Wong
2025-05-22 0:12 ` [PATCH 07/16] fuse2fs: add extent dump function for debugging Darrick J. Wong
2025-05-22 0:12 ` [PATCH 08/16] fuse2fs: implement direct write support Darrick J. Wong
2025-05-22 0:13 ` [PATCH 09/16] fuse2fs: turn on iomap for pagecache IO Darrick J. Wong
2025-05-22 0:13 ` [PATCH 10/16] fuse2fs: flush and invalidate the buffer cache on trim Darrick J. Wong
2025-05-22 0:13 ` [PATCH 11/16] fuse2fs: improve tracing for fallocate Darrick J. Wong
2025-05-22 0:13 ` [PATCH 12/16] fuse2fs: don't zero bytes in punch hole Darrick J. Wong
2025-05-22 0:14 ` [PATCH 13/16] fuse2fs: don't do file data block IO when iomap is enabled Darrick J. Wong
2025-05-22 0:14 ` [PATCH 14/16] fuse2fs: disable most io channel flush/invalidate in iomap pagecache mode Darrick J. Wong
2025-05-22 0:14 ` [PATCH 15/16] fuse2fs: re-enable the block device pagecache for metadata IO Darrick J. Wong
2025-05-22 0:15 ` [PATCH 16/16] fuse2fs: avoid fuseblk mode if fuse-iomap support is likely Darrick J. Wong
2025-05-22 16:24 ` [RFC[RAP]] fuse: use fs-iomap for better performance so we can containerize ext4 Amir Goldstein
2025-05-29 16:45 ` Darrick J. Wong
2025-05-29 19:41 ` Amir Goldstein
2025-06-09 22:31 ` Darrick J. Wong
2025-06-10 10:59 ` Amir Goldstein
2025-06-10 19:00 ` Darrick J. Wong
2025-06-10 19:51 ` Amir Goldstein
2025-06-11 6:00 ` Darrick J. Wong
2025-06-11 8:54 ` Amir Goldstein
2025-06-12 5:54 ` Miklos Szeredi
2025-06-13 17:44 ` Darrick J. Wong
2025-06-11 11:56 ` Theodore Ts'o
2025-06-12 3:20 ` Darrick J. Wong
2025-06-12 6:10 ` Amir Goldstein
2025-06-20 8:58 ` Allison Karlitskaya
2025-06-20 11:50 ` Bernd Schubert
2025-07-01 6:02 ` Darrick J. Wong
2025-07-01 5:58 ` Darrick J. Wong
2025-07-12 10:57 ` Amir Goldstein
2025-06-13 17:37 ` [RFC[RAP] V2] " Darrick J. Wong
2025-06-23 13:16 ` Miklos Szeredi
2025-07-01 6:05 ` Darrick J. Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=174787198025.1484572.10345977324531146086.stgit@frogsfrogsfrogs \
--to=djwong@kernel.org \
--cc=John@groves.net \
--cc=bernd@bsbernd.com \
--cc=joannelkoong@gmail.com \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=miklos@szeredi.hu \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox