* ovl: sync_file_range() does nothing on overlayfs files
@ 2026-11-18 4:00 Ding Feng
2026-11-18 10:16 ` Christian Brauner
0 siblings, 1 reply; 5+ messages in thread
From: Ding Feng @ 2026-11-18 4:00 UTC (permalink / raw)
To: linux-unionfs, linux-fsdevel
Cc: Amir Goldstein, Miklos Szeredi, Christian Brauner,
Pavel Tikhomirov
Hi,
sync_file_range(2) acts on file->f_mapping. For an overlayfs file that
is the overlay inode's mapping, which holds no pages: the data sits in
the real file's page cache. So on an overlayfs file the call writes and
waits on nothing and returns 0.
On a Docker overlayfs root (6.10.14): write 64 MiB to a file, then call
sync_file_range(fd, 0, 0, SYNC_FILE_RANGE_WAIT_BEFORE |
SYNC_FILE_RANGE_WRITE |
SYNC_FILE_RANGE_WAIT_AFTER);
Dirty in /proc/meminfo stays 64 MiB higher until fsync(). The same
program on ext4 sees Dirty drop by 64 MiB after the call. Mainline
fs/sync.c still takes file->f_mapping.
Amir posted a fix in 2018 [1]. Miklos preferred to leave it [2], and
Dave pointed out that the call used WB_SYNC_NONE and so guaranteed
nothing. Since commit c553ea4fdf27 ("fs/sync.c: sync_file_range(2) may
use WB_SYNC_ALL writeback"), the full write-and-wait form does
data-integrity writeback on other filesystems, so on overlayfs it now
silently skips what it does elsewhere.
cachestat() has the same cause, and Pavel's series in June [3] fixes it
by forwarding to the real file, as ovl_fadvise() does. Christian
suggested a helper in fs/backing-file.c for that instead of a new file
operation. Could sync_file_range() use the same helper? I can send a
patch along those lines if that direction is acceptable.
I found this while checking sync_file_range() against its man page
with an AI-assisted verification tool, and reproduced it as above.
[1] https://lore.kernel.org/linux-fsdevel/1535300717-26686-7-git-send-email-amir73il@gmail.com/
[2] https://lore.kernel.org/linux-fsdevel/CAJfpegv50HELSZuwt9gGjgSyB+4aPbiTncc-yBNqUVgTLNHaXw@mail.gmail.com/
[3] https://lore.kernel.org/linux-fsdevel/?q=%22ovl%3A+make+cachestat%28%29+work%22
Thanks,
Ding Feng
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: ovl: sync_file_range() does nothing on overlayfs files 2026-11-18 4:00 ovl: sync_file_range() does nothing on overlayfs files Ding Feng @ 2026-11-18 10:16 ` Christian Brauner 2026-11-18 10:37 ` Amir Goldstein 0 siblings, 1 reply; 5+ messages in thread From: Christian Brauner @ 2026-11-18 10:16 UTC (permalink / raw) To: Ding Feng, Miklos Szeredi, Amir Goldstein Cc: Josef Bacik, Pavel Tikhomirov, linux-unionfs, linux-fsdevel On Wed, Nov 18, 2026 at 12:00:56PM +0800, Ding Feng wrote: > cachestat() has the same cause, and Pavel's series in June [3] fixes it > by forwarding to the real file, as ovl_fadvise() does. Christian > suggested a helper in fs/backing-file.c for that instead of a new file > operation. Could sync_file_range() use the same helper? I can send a > patch along those lines if that direction is acceptable. This is really an overlayfs call, I think. A helper in fs/backing-file.c that gets the file whose mapping actually has the pages might be fine. For overlayfs copy-up changes the file so the helper has to go ask overlayfs, I think. But really, Amir is better equipped to decide whether this makes sense. ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: ovl: sync_file_range() does nothing on overlayfs files 2026-11-18 10:16 ` Christian Brauner @ 2026-11-18 10:37 ` Amir Goldstein 2026-11-18 11:32 ` Ding Feng 0 siblings, 1 reply; 5+ messages in thread From: Amir Goldstein @ 2026-11-18 10:37 UTC (permalink / raw) To: Christian Brauner Cc: Ding Feng, Miklos Szeredi, Josef Bacik, Pavel Tikhomirov, linux-unionfs, linux-fsdevel [-- Attachment #1: Type: text/plain, Size: 883 bytes --] On Wed, Nov 18, 2026 at 11:16 AM Christian Brauner <brauner@kernel.org> wrote: > > On Wed, Nov 18, 2026 at 12:00:56PM +0800, Ding Feng wrote: > > cachestat() has the same cause, and Pavel's series in June [3] fixes it > > by forwarding to the real file, as ovl_fadvise() does. Christian > > suggested a helper in fs/backing-file.c for that instead of a new file > > operation. Could sync_file_range() use the same helper? I can send a > > patch along those lines if that direction is acceptable. > > This is really an overlayfs call, I think. A helper in fs/backing-file.c > that gets the file whose mapping actually has the pages might be fine. > For overlayfs copy-up changes the file so the helper has to go ask > overlayfs, I think. But really, Amir is better equipped to decide > whether this makes sense. Like this? Completely untested. Thanks, Amir. [-- Attachment #2: 0001-fs-add-f_real-file-operation.patch --] [-- Type: text/x-patch, Size: 6461 bytes --] From: Amir Goldstein <amir73il@gmail.com> Subject: [PATCH RFC POC] fs: add ->f_real() file operation sync_file_range(2) and cachestat(2) operate on file->f_mapping. For a file on a stacking filesystem, such as overlayfs, the data and its page cache are in the real (backing) file, so on a stacked file sync_file_range() writes back nothing and cachestat() reports nothing. Add a ->f_real() file operation, modelled after ->d_real(), and a file_real() helper in backing-file.h that returns the backing file holding the data of a stacked file, or the file itself. file_real() is a no-op without CONFIG_FS_STACK; otherwise its slow path in fs/backing-file.c calls ->f_real() and verifies that it returned a backing file. The real file is owned by the stacked file, so it is valid for as long as the caller holds the stacked file. file_real() is only meant for inspecting and writing back the page cache of the data. Operations that need the credentials or permission checks of the stacking filesystem need a dedicated file operation, like ->fadvise(). Use file_real() in sync_file_range() (also used by io_uring) and in cachestat(), and implement ->f_real() for overlayfs. Signed-off-by: Amir Goldstein <amir73il@gmail.com> --- fs/backing-file.c | 12 ++++++++++++ fs/overlayfs/file.c | 11 +++++++++++ fs/sync.c | 8 ++++++++ include/linux/backing-file.h | 26 ++++++++++++++++++++++++++ include/linux/fs.h | 1 + mm/filemap.c | 9 ++++++++- 6 files changed, 66 insertions(+), 1 deletion(-) --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1960,6 +1960,7 @@ struct file_operations { struct file *file_out, loff_t pos_out, loff_t len, unsigned int remap_flags); int (*fadvise)(struct file *, loff_t, loff_t, int); + struct file *(*f_real)(const struct file *); int (*uring_cmd)(struct io_uring_cmd *ioucmd, unsigned int issue_flags); int (*uring_cmd_iopoll)(struct io_uring_cmd *, struct io_comp_batch *, unsigned int poll_flags); --- a/include/linux/backing-file.h +++ b/include/linux/backing-file.h @@ -40,5 +40,31 @@ ssize_t backing_file_splice_write(struct struct backing_file_ctx *ctx); int backing_file_mmap(struct file *file, struct vm_area_struct *vma, struct backing_file_ctx *ctx); +struct file *__file_real(struct file *file); + +/** + * file_real - Return the real file hosting the data + * @file: the file to query + * + * If @file is on a stacking filesystem (e.g. overlayfs) whose data is + * accessed through a backing file, return that backing file. + * Otherwise return @file itself. + * + * The returned file is owned by @file and is valid for as long as the + * caller holds a reference to @file. + * + * The real file is only meant to be used for inspecting and writing + * back the page cache of the data (e.g. sync_file_range(2), cachestat(2)). + * It must not be used for operations that require the credentials or the + * permission checks of the stacking filesystem; those need a dedicated + * file operation (e.g. ->fadvise()). + */ +static inline struct file *file_real(struct file *file) +{ + if (!IS_ENABLED(CONFIG_FS_STACK) || likely(!file->f_op->f_real)) + return file; + + return __file_real(file); +} #endif /* _LINUX_BACKING_FILE_H */ --- a/fs/backing-file.c +++ b/fs/backing-file.c @@ -102,6 +102,18 @@ struct file *backing_tmpfile_open(const } EXPORT_SYMBOL(backing_tmpfile_open); +/* Slow path of file_real() for a file with ->f_real() */ +struct file *__file_real(struct file *file) +{ + struct file *real_file = file->f_op->f_real(file); + + if (!IS_ERR(real_file) && real_file != file && + WARN_ON_ONCE(!(real_file->f_mode & FMODE_BACKING))) + return ERR_PTR(-EIO); + + return real_file; +} + struct backing_aio { struct kiocb iocb; refcount_t ref; --- a/fs/sync.c +++ b/fs/sync.c @@ -7,6 +7,7 @@ #include <linux/kernel.h> #include <linux/file.h> #include <linux/fs.h> +#include <linux/backing-file.h> #include <linux/slab.h> #include <linux/export.h> #include <linux/namei.h> @@ -269,6 +270,13 @@ int sync_file_range(struct file *file, l if (!S_ISREG(i_mode) && !S_ISBLK(i_mode) && !S_ISDIR(i_mode)) goto out; + /* The page cache of a stacked file is in its real file */ + file = file_real(file); + if (IS_ERR(file)) { + ret = PTR_ERR(file); + goto out; + } + mapping = file->f_mapping; ret = 0; if (flags & SYNC_FILE_RANGE_WAIT_BEFORE) { --- a/mm/filemap.c +++ b/mm/filemap.c @@ -31,6 +31,7 @@ #include <linux/hash.h> #include <linux/writeback.h> #include <linux/backing-dev.h> +#include <linux/backing-file.h> #include <linux/folio_batch.h> #include <linux/security.h> #include <linux/cpuset.h> @@ -4782,6 +4783,7 @@ SYSCALL_DEFINE4(cachestat, unsigned int, struct cachestat __user *, cstat, unsigned int, flags) { CLASS(fd, f)(fd); + struct file *real_file; struct address_space *mapping; struct cachestat_range csr; struct cachestat cs; @@ -4804,11 +4806,16 @@ SYSCALL_DEFINE4(cachestat, unsigned int, if (flags != 0) return -EINVAL; + /* The page cache of a stacked file is in its real file */ + real_file = file_real(fd_file(f)); + if (IS_ERR(real_file)) + return PTR_ERR(real_file); + first_index = csr.off >> PAGE_SHIFT; last_index = csr.len == 0 ? ULONG_MAX : (csr.off + csr.len - 1) >> PAGE_SHIFT; memset(&cs, 0, sizeof(struct cachestat)); - mapping = fd_file(f)->f_mapping; + mapping = real_file->f_mapping; filemap_cachestat(mapping, first_index, last_index, &cs); if (copy_to_user(cstat, &cs, sizeof(struct cachestat))) --- a/fs/overlayfs/file.c +++ b/fs/overlayfs/file.c @@ -195,6 +195,16 @@ static struct file *ovl_real_file(const return ovl_real_file_path(file, &realpath); } +/* + * The real file that holds the page cache of the data: the upper file after + * copy up, otherwise the lower data file. After copy up of a file that was + * opened from lower, this opens the upper file, just like a write would. + */ +static struct file *ovl_f_real(const struct file *file) +{ + return ovl_real_file(file); +} + static int ovl_open(struct inode *inode, struct file *file) { struct dentry *dentry = file_dentry(file); @@ -656,6 +666,7 @@ const struct file_operations ovl_file_op .mmap = ovl_mmap, .fallocate = ovl_fallocate, .fadvise = ovl_fadvise, + .f_real = ovl_f_real, .flush = ovl_flush, .splice_read = ovl_splice_read, .splice_write = ovl_splice_write, ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: ovl: sync_file_range() does nothing on overlayfs files 2026-11-18 10:37 ` Amir Goldstein @ 2026-11-18 11:32 ` Ding Feng 2026-11-18 12:18 ` Amir Goldstein 0 siblings, 1 reply; 5+ messages in thread From: Ding Feng @ 2026-11-18 11:32 UTC (permalink / raw) To: amir73il Cc: Christian Brauner, Miklos Szeredi, Josef Bacik, Pavel Tikhomirov, linux-unionfs, linux-fsdevel, Ding Feng Amir Goldstein wrote: > Like this? > > Completely untested. Thanks, Amir. I tested it on v7.4-rc3 (20eef29b7c84), where it applies with an 8-line offset in mm/filemap.c. Setup: arm64 defconfig in QEMU; overlayfs with lower and upper on one ext4 disk; periodic writeback off (dirty_writeback_centisecs=0). Each case writes 64 MiB (16384 pages) through the overlay and calls sync_file_range(fd, 0, 0, flags). Dirty pages are read with cachestat() on the upper file's ext4 path, before and after the call. before patch with patch ext4 file, flags=7 16384 -> 0 16384 -> 0 overlay new file, flags=7 16384 -> 16384 16384 -> 0 overlay new file, flags=2 16384 -> 16384 16384 -> 0 (4096 under writeback) read-only fd opened before copy-up, written via another fd, flags=7 16384 -> 16384 16384 -> 0 cachestat() on the overlay fd reported cache=0 dirty=0 before the patch and cache=16384 dirty=16384 with it, in all three overlay cases. All calls returned 0, and the kernel log showed no warnings. Tested-by: Ding Feng <dingf3ng@gmail.com> Two questions: 1. FUSE passthrough also keeps data in a backing file (fs/fuse/passthrough.c) and already forwards ->fadvise(). Does it need ->f_real() too? I have not tested that. 2. In the copy-up case, sync_file_range() on a read-only fd can now open the upper file and return that open's error. The man page lists none of those codes. I assume that is acceptable, as a write would see the same error, but I wanted to mention it. Thanks, Ding Feng ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: ovl: sync_file_range() does nothing on overlayfs files 2026-11-18 11:32 ` Ding Feng @ 2026-11-18 12:18 ` Amir Goldstein 0 siblings, 0 replies; 5+ messages in thread From: Amir Goldstein @ 2026-11-18 12:18 UTC (permalink / raw) To: Ding Feng Cc: Christian Brauner, Miklos Szeredi, Josef Bacik, Pavel Tikhomirov, linux-unionfs, linux-fsdevel On Wed, Nov 18, 2026 at 12:32 PM Ding Feng <dingf3ng@gmail.com> wrote: > > Amir Goldstein wrote: > > Like this? > > > > Completely untested. > > Thanks, Amir. I tested it on v7.4-rc3 (20eef29b7c84), where it applies > with an 8-line offset in mm/filemap.c. > > Setup: arm64 defconfig in QEMU; overlayfs with lower and upper on one > ext4 disk; periodic writeback off (dirty_writeback_centisecs=0). Each > case writes 64 MiB (16384 pages) through the overlay and calls > sync_file_range(fd, 0, 0, flags). Dirty pages are read with cachestat() > on the upper file's ext4 path, before and after the call. > > before patch with patch > ext4 file, flags=7 16384 -> 0 16384 -> 0 > overlay new file, flags=7 16384 -> 16384 16384 -> 0 > overlay new file, flags=2 16384 -> 16384 16384 -> 0 (4096 under writeback) > read-only fd opened before > copy-up, written via another > fd, flags=7 16384 -> 16384 16384 -> 0 > > cachestat() on the overlay fd reported cache=0 dirty=0 before the patch > and cache=16384 dirty=16384 with it, in all three overlay cases. All > calls returned 0, and the kernel log showed no warnings. > > Tested-by: Ding Feng <dingf3ng@gmail.com> > > Two questions: > > 1. FUSE passthrough also keeps data in a backing file > (fs/fuse/passthrough.c) and already forwards ->fadvise(). Does it need > ->f_real() too? I have not tested that. Yes it does, although FUSE does not have d_real(), meaning that nobody probably tested uprobes with FUSE passthrough I don't know if IMA/EVM/IPE are applicable to FUSE. There is no obligation to implement FUSE f_real() from the start if sync_range doesnt work now it can continue not to work. > > 2. In the copy-up case, sync_file_range() on a read-only fd can now > open the upper file and return that open's error. The man page lists > none of those codes. I assume that is acceptable, as a write would see > the same error, but I wanted to mention it. The man page never covers all possible errors especially general errors like EIO, EACCESS, EPERM should always be anticipated. I am sure this is documented somewhere but I don't know where. Thanks, Amir. ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-11-18 12:18 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-11-18 4:00 ovl: sync_file_range() does nothing on overlayfs files Ding Feng 2026-11-18 10:16 ` Christian Brauner 2026-11-18 10:37 ` Amir Goldstein 2026-11-18 11:32 ` Ding Feng 2026-11-18 12:18 ` Amir Goldstein
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).