[PATCH v3 00/12] fuse: support large folios

[PATCH v3 00/12] fuse: support large folios

[Joanne Koong]

This patchset adds support for folios larger than one page size in FUSE.

This patchset is rebased on top of the (unmerged) patchset that removes temp
folios in writeback [1]. This patchset was tested by running it through fstests
on passthrough_hp.

Please note that writes are still effectively one page size. Larger writes can
be enabled by setting the order on the fgp flag passed in to __filemap_get_folio()
but benchmarks show this significantly degrades performance. More investigation
needs to be done into this. As such, buffered writes will be optimized in a
future patchset.

Benchmarks show roughly a ~45% improvement in read throughput.

Benchmark setup:

-- Set up server --
 ./libfuse/build/example/passthrough_hp --bypass-rw=1 ~/libfuse
~/mounts/fuse/ --nopassthrough
(using libfuse patched with https://github.com/libfuse/libfuse/pull/807)

-- Run fio --
 fio --name=read --ioengine=sync --rw=read --bs=1M --size=1G
--numjobs=2 --ramp_time=30 --group_reporting=1
--directory=mounts/fuse/

Machine 1:
    No large folios:     ~4400 MiB/s
    Large folios:        ~7100 MiB/s

Machine 2:
    No large folios:     ~3700 MiB/s
    Large folios:        ~6400 MiB/s


[1] https://lore.kernel.org/linux-fsdevel/20241122232359.429647-1-joannelkoong@gmail.com/

Changelog:
v2: https://lore.kernel.org/linux-fsdevel/20241125220537.3663725-1-joannelkoong@gmail.com/
v2 -> v3:
* Fix direct io parsing to check each extracted page instead of assuming all
  pages in a large folio will be used (Matthew)

v1: https://lore.kernel.org/linux-fsdevel/20241109001258.2216604-1-joannelkoong@gmail.com/
v1 -> v2:
* Change naming from "non-writeback write" to "writethrough write"
* Fix deadlock for writethrough writes by calling fault_in_iov_iter_readable()
* first
  before __filemap_get_folio() (Josef)
* For readahead, retain original folio_size() for descs.length (Josef)
* Use folio_zero_range() api in fuse_copy_folio() (Josef)
* Add Josef's reviewed-bys

Joanne Koong (12):
  fuse: support copying large folios
  fuse: support large folios for retrieves
  fuse: refactor fuse_fill_write_pages()
  fuse: support large folios for writethrough writes
  fuse: support large folios for folio reads
  fuse: support large folios for symlinks
  fuse: support large folios for stores
  fuse: support large folios for queued writes
  fuse: support large folios for readahead
  fuse: optimize direct io large folios processing
  fuse: support large folios for writeback
  fuse: enable large folios

 fs/fuse/dev.c  | 128 ++++++++++++++++++++++---------------------
 fs/fuse/dir.c  |   8 +--
 fs/fuse/file.c | 144 +++++++++++++++++++++++++++++++++----------------
 3 files changed, 166 insertions(+), 114 deletions(-)

-- 
2.43.5

[PATCH v3 01/12] fuse: support copying large folios

[Joanne Koong]

Currently, all folios associated with fuse are one page size. As part of
the work to enable large folios, this commit adds support for copying
to/from folios larger than one page size.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
---
 fs/fuse/dev.c | 86 ++++++++++++++++++++++-----------------------------
 1 file changed, 37 insertions(+), 49 deletions(-)

diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c
index 27ccae63495d..0a3dfb66c7cd 100644
--- a/fs/fuse/dev.c
+++ b/fs/fuse/dev.c
@@ -703,7 +703,7 @@ struct fuse_copy_state {
 	struct page *pg;
 	unsigned len;
 	unsigned offset;
-	unsigned move_pages:1;
+	unsigned move_folios:1;
 };
 
 static void fuse_copy_init(struct fuse_copy_state *cs, int write,
@@ -836,10 +836,10 @@ static int fuse_check_folio(struct folio *folio)
 	return 0;
 }
 
-static int fuse_try_move_page(struct fuse_copy_state *cs, struct page **pagep)
+static int fuse_try_move_folio(struct fuse_copy_state *cs, struct folio **foliop)
 {
 	int err;
-	struct folio *oldfolio = page_folio(*pagep);
+	struct folio *oldfolio = *foliop;
 	struct folio *newfolio;
 	struct pipe_buffer *buf = cs->pipebufs;
 
@@ -860,7 +860,7 @@ static int fuse_try_move_page(struct fuse_copy_state *cs, struct page **pagep)
 	cs->pipebufs++;
 	cs->nr_segs--;
 
-	if (cs->len != PAGE_SIZE)
+	if (cs->len != folio_size(oldfolio))
 		goto out_fallback;
 
 	if (!pipe_buf_try_steal(cs->pipe, buf))
@@ -906,7 +906,7 @@ static int fuse_try_move_page(struct fuse_copy_state *cs, struct page **pagep)
 	if (test_bit(FR_ABORTED, &cs->req->flags))
 		err = -ENOENT;
 	else
-		*pagep = &newfolio->page;
+		*foliop = newfolio;
 	spin_unlock(&cs->req->waitq.lock);
 
 	if (err) {
@@ -939,8 +939,8 @@ static int fuse_try_move_page(struct fuse_copy_state *cs, struct page **pagep)
 	goto out_put_old;
 }
 
-static int fuse_ref_page(struct fuse_copy_state *cs, struct page *page,
-			 unsigned offset, unsigned count)
+static int fuse_ref_folio(struct fuse_copy_state *cs, struct folio *folio,
+			  unsigned offset, unsigned count)
 {
 	struct pipe_buffer *buf;
 	int err;
@@ -948,17 +948,17 @@ static int fuse_ref_page(struct fuse_copy_state *cs, struct page *page,
 	if (cs->nr_segs >= cs->pipe->max_usage)
 		return -EIO;
 
-	get_page(page);
+	folio_get(folio);
 	err = unlock_request(cs->req);
 	if (err) {
-		put_page(page);
+		folio_put(folio);
 		return err;
 	}
 
 	fuse_copy_finish(cs);
 
 	buf = cs->pipebufs;
-	buf->page = page;
+	buf->page = &folio->page;
 	buf->offset = offset;
 	buf->len = count;
 
@@ -970,20 +970,21 @@ static int fuse_ref_page(struct fuse_copy_state *cs, struct page *page,
 }
 
 /*
- * Copy a page in the request to/from the userspace buffer.  Must be
+ * Copy a folio in the request to/from the userspace buffer.  Must be
  * done atomically
  */
-static int fuse_copy_page(struct fuse_copy_state *cs, struct page **pagep,
-			  unsigned offset, unsigned count, int zeroing)
+static int fuse_copy_folio(struct fuse_copy_state *cs, struct folio **foliop,
+			   unsigned offset, unsigned count, int zeroing)
 {
 	int err;
-	struct page *page = *pagep;
+	struct folio *folio = *foliop;
+	size_t size = folio_size(folio);
 
-	if (page && zeroing && count < PAGE_SIZE)
-		clear_highpage(page);
+	if (folio && zeroing && count < size)
+		folio_zero_range(folio, 0, size);
 
 	while (count) {
-		if (cs->write && cs->pipebufs && page) {
+		if (cs->write && cs->pipebufs && folio) {
 			/*
 			 * Can't control lifetime of pipe buffers, so always
 			 * copy user pages.
@@ -993,12 +994,12 @@ static int fuse_copy_page(struct fuse_copy_state *cs, struct page **pagep,
 				if (err)
 					return err;
 			} else {
-				return fuse_ref_page(cs, page, offset, count);
+				return fuse_ref_folio(cs, folio, offset, count);
 			}
 		} else if (!cs->len) {
-			if (cs->move_pages && page &&
-			    offset == 0 && count == PAGE_SIZE) {
-				err = fuse_try_move_page(cs, pagep);
+			if (cs->move_folios && folio &&
+			    offset == 0 && count == folio_size(folio)) {
+				err = fuse_try_move_folio(cs, foliop);
 				if (err <= 0)
 					return err;
 			} else {
@@ -1007,22 +1008,22 @@ static int fuse_copy_page(struct fuse_copy_state *cs, struct page **pagep,
 					return err;
 			}
 		}
-		if (page) {
-			void *mapaddr = kmap_local_page(page);
-			void *buf = mapaddr + offset;
+		if (folio) {
+			void *mapaddr = kmap_local_folio(folio, offset);
+			void *buf = mapaddr;
 			offset += fuse_copy_do(cs, &buf, &count);
 			kunmap_local(mapaddr);
 		} else
 			offset += fuse_copy_do(cs, NULL, &count);
 	}
-	if (page && !cs->write)
-		flush_dcache_page(page);
+	if (folio && !cs->write)
+		flush_dcache_folio(folio);
 	return 0;
 }
 
-/* Copy pages in the request to/from userspace buffer */
-static int fuse_copy_pages(struct fuse_copy_state *cs, unsigned nbytes,
-			   int zeroing)
+/* Copy folios in the request to/from userspace buffer */
+static int fuse_copy_folios(struct fuse_copy_state *cs, unsigned nbytes,
+			    int zeroing)
 {
 	unsigned i;
 	struct fuse_req *req = cs->req;
@@ -1032,23 +1033,12 @@ static int fuse_copy_pages(struct fuse_copy_state *cs, unsigned nbytes,
 		int err;
 		unsigned int offset = ap->descs[i].offset;
 		unsigned int count = min(nbytes, ap->descs[i].length);
-		struct page *orig, *pagep;
-
-		orig = pagep = &ap->folios[i]->page;
 
-		err = fuse_copy_page(cs, &pagep, offset, count, zeroing);
+		err = fuse_copy_folio(cs, &ap->folios[i], offset, count, zeroing);
 		if (err)
 			return err;
 
 		nbytes -= count;
-
-		/*
-		 *  fuse_copy_page may have moved a page from a pipe instead of
-		 *  copying into our given page, so update the folios if it was
-		 *  replaced.
-		 */
-		if (pagep != orig)
-			ap->folios[i] = page_folio(pagep);
 	}
 	return 0;
 }
@@ -1078,7 +1068,7 @@ static int fuse_copy_args(struct fuse_copy_state *cs, unsigned numargs,
 	for (i = 0; !err && i < numargs; i++)  {
 		struct fuse_arg *arg = &args[i];
 		if (i == numargs - 1 && argpages)
-			err = fuse_copy_pages(cs, arg->size, zeroing);
+			err = fuse_copy_folios(cs, arg->size, zeroing);
 		else
 			err = fuse_copy_one(cs, arg->value, arg->size);
 	}
@@ -1665,7 +1655,6 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size,
 	num = outarg.size;
 	while (num) {
 		struct folio *folio;
-		struct page *page;
 		unsigned int this_num;
 
 		folio = filemap_grab_folio(mapping, index);
@@ -1673,9 +1662,8 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size,
 		if (IS_ERR(folio))
 			goto out_iput;
 
-		page = &folio->page;
 		this_num = min_t(unsigned, num, folio_size(folio) - offset);
-		err = fuse_copy_page(cs, &page, offset, this_num, 0);
+		err = fuse_copy_folio(cs, &folio, offset, this_num, 0);
 		if (!folio_test_uptodate(folio) && !err && offset == 0 &&
 		    (this_num == folio_size(folio) || file_size == end)) {
 			folio_zero_segment(folio, this_num, folio_size(folio));
@@ -1902,8 +1890,8 @@ static int fuse_notify_resend(struct fuse_conn *fc)
 static int fuse_notify(struct fuse_conn *fc, enum fuse_notify_code code,
 		       unsigned int size, struct fuse_copy_state *cs)
 {
-	/* Don't try to move pages (yet) */
-	cs->move_pages = 0;
+	/* Don't try to move folios (yet) */
+	cs->move_folios = 0;
 
 	switch (code) {
 	case FUSE_NOTIFY_POLL:
@@ -2044,7 +2032,7 @@ static ssize_t fuse_dev_do_write(struct fuse_dev *fud,
 	spin_unlock(&fpq->lock);
 	cs->req = req;
 	if (!req->args->page_replace)
-		cs->move_pages = 0;
+		cs->move_folios = 0;
 
 	if (oh.error)
 		err = nbytes != sizeof(oh) ? -EINVAL : 0;
@@ -2163,7 +2151,7 @@ static ssize_t fuse_dev_splice_write(struct pipe_inode_info *pipe,
 	cs.pipe = pipe;
 
 	if (flags & SPLICE_F_MOVE)
-		cs.move_pages = 1;
+		cs.move_folios = 1;
 
 	ret = fuse_dev_do_write(fud, &cs, len);
 
-- 
2.43.5

[PATCH v3 02/12] fuse: support large folios for retrieves

[Joanne Koong]

Add support for folios larger than one page size for retrieves.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/fuse/dev.c | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c
index 0a3dfb66c7cd..2a2a5e66412f 100644
--- a/fs/fuse/dev.c
+++ b/fs/fuse/dev.c
@@ -1716,7 +1716,7 @@ static int fuse_retrieve(struct fuse_mount *fm, struct inode *inode,
 	unsigned int num;
 	unsigned int offset;
 	size_t total_len = 0;
-	unsigned int num_pages, cur_pages = 0;
+	unsigned int num_pages;
 	struct fuse_conn *fc = fm->fc;
 	struct fuse_retrieve_args *ra;
 	size_t args_size = sizeof(*ra);
@@ -1734,6 +1734,7 @@ static int fuse_retrieve(struct fuse_mount *fm, struct inode *inode,
 
 	num_pages = (num + offset + PAGE_SIZE - 1) >> PAGE_SHIFT;
 	num_pages = min(num_pages, fc->max_pages);
+	num = min(num, num_pages << PAGE_SHIFT);
 
 	args_size += num_pages * (sizeof(ap->folios[0]) + sizeof(ap->descs[0]));
 
@@ -1754,25 +1755,29 @@ static int fuse_retrieve(struct fuse_mount *fm, struct inode *inode,
 
 	index = outarg->offset >> PAGE_SHIFT;
 
-	while (num && cur_pages < num_pages) {
+	while (num) {
 		struct folio *folio;
-		unsigned int this_num;
+		unsigned int folio_offset;
+		unsigned int nr_bytes;
+		unsigned int nr_pages;
 
 		folio = filemap_get_folio(mapping, index);
 		if (IS_ERR(folio))
 			break;
 
-		this_num = min_t(unsigned, num, PAGE_SIZE - offset);
+		folio_offset = ((index - folio->index) << PAGE_SHIFT) + offset;
+		nr_bytes = min(folio_size(folio) - folio_offset, num);
+		nr_pages = (offset + nr_bytes + PAGE_SIZE - 1) >> PAGE_SHIFT;
+
 		ap->folios[ap->num_folios] = folio;
-		ap->descs[ap->num_folios].offset = offset;
-		ap->descs[ap->num_folios].length = this_num;
+		ap->descs[ap->num_folios].offset = folio_offset;
+		ap->descs[ap->num_folios].length = nr_bytes;
 		ap->num_folios++;
-		cur_pages++;
 
 		offset = 0;
-		num -= this_num;
-		total_len += this_num;
-		index++;
+		num -= nr_bytes;
+		total_len += nr_bytes;
+		index += nr_pages;
 	}
 	ra->inarg.offset = outarg->offset;
 	ra->inarg.size = total_len;
-- 
2.43.5

[PATCH v3 03/12] fuse: refactor fuse_fill_write_pages()

[Joanne Koong]

Refactor the logic in fuse_fill_write_pages() for copying out write
data. This will make the future change for supporting large folios for
writes easier. No functional changes.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/fuse/file.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index 49cb9e84bd2e..c041bb328203 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -1138,21 +1138,21 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
 	struct fuse_args_pages *ap = &ia->ap;
 	struct fuse_conn *fc = get_fuse_conn(mapping->host);
 	unsigned offset = pos & (PAGE_SIZE - 1);
-	unsigned int nr_pages = 0;
 	size_t count = 0;
+	unsigned int num;
 	int err;
 
+	num = min(iov_iter_count(ii), fc->max_write);
+	num = min(num, max_pages << PAGE_SHIFT);
+
 	ap->args.in_pages = true;
 	ap->descs[0].offset = offset;
 
-	do {
+	while (num) {
 		size_t tmp;
 		struct folio *folio;
 		pgoff_t index = pos >> PAGE_SHIFT;
-		size_t bytes = min_t(size_t, PAGE_SIZE - offset,
-				     iov_iter_count(ii));
-
-		bytes = min_t(size_t, bytes, fc->max_write - count);
+		unsigned int bytes = min(PAGE_SIZE - offset, num);
 
  again:
 		err = -EFAULT;
@@ -1182,10 +1182,10 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
 		ap->folios[ap->num_folios] = folio;
 		ap->descs[ap->num_folios].length = tmp;
 		ap->num_folios++;
-		nr_pages++;
 
 		count += tmp;
 		pos += tmp;
+		num -= tmp;
 		offset += tmp;
 		if (offset == PAGE_SIZE)
 			offset = 0;
@@ -1202,8 +1202,9 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
 		}
 		if (!fc->big_writes)
 			break;
-	} while (iov_iter_count(ii) && count < fc->max_write &&
-		 nr_pages < max_pages && offset == 0);
+		if (offset != 0)
+			break;
+	}
 
 	return count > 0 ? count : err;
 }
-- 
2.43.5

[PATCH v3 04/12] fuse: support large folios for writethrough writes

[Joanne Koong]

Add support for folios larger than one page size for writethrough
writes.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
---
 fs/fuse/file.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index c041bb328203..84e39426862a 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -1135,6 +1135,7 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
 				     struct iov_iter *ii, loff_t pos,
 				     unsigned int max_pages)
 {
+	size_t max_folio_size = mapping_max_folio_size(mapping);
 	struct fuse_args_pages *ap = &ia->ap;
 	struct fuse_conn *fc = get_fuse_conn(mapping->host);
 	unsigned offset = pos & (PAGE_SIZE - 1);
@@ -1146,17 +1147,17 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
 	num = min(num, max_pages << PAGE_SHIFT);
 
 	ap->args.in_pages = true;
-	ap->descs[0].offset = offset;
 
 	while (num) {
 		size_t tmp;
 		struct folio *folio;
 		pgoff_t index = pos >> PAGE_SHIFT;
-		unsigned int bytes = min(PAGE_SIZE - offset, num);
+		unsigned int bytes;
+		unsigned int folio_offset;
 
  again:
 		err = -EFAULT;
-		if (fault_in_iov_iter_readable(ii, bytes))
+		if (fault_in_iov_iter_readable(ii, max_folio_size) == max_folio_size)
 			break;
 
 		folio = __filemap_get_folio(mapping, index, FGP_WRITEBEGIN,
@@ -1169,7 +1170,10 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
 		if (mapping_writably_mapped(mapping))
 			flush_dcache_folio(folio);
 
-		tmp = copy_folio_from_iter_atomic(folio, offset, bytes, ii);
+		folio_offset = ((index - folio->index) << PAGE_SHIFT) + offset;
+		bytes = min(folio_size(folio) - folio_offset, num);
+
+		tmp = copy_folio_from_iter_atomic(folio, folio_offset, bytes, ii);
 		flush_dcache_folio(folio);
 
 		if (!tmp) {
@@ -1180,6 +1184,7 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
 
 		err = 0;
 		ap->folios[ap->num_folios] = folio;
+		ap->descs[ap->num_folios].offset = folio_offset;
 		ap->descs[ap->num_folios].length = tmp;
 		ap->num_folios++;
 
@@ -1187,11 +1192,11 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
 		pos += tmp;
 		num -= tmp;
 		offset += tmp;
-		if (offset == PAGE_SIZE)
+		if (offset == folio_size(folio))
 			offset = 0;
 
-		/* If we copied full page, mark it uptodate */
-		if (tmp == PAGE_SIZE)
+		/* If we copied full folio, mark it uptodate */
+		if (tmp == folio_size(folio))
 			folio_mark_uptodate(folio);
 
 		if (folio_test_uptodate(folio)) {
-- 
2.43.5

[PATCH v3 05/12] fuse: support large folios for folio reads

[Joanne Koong]

Add support for folios larger than one page size for folio reads into
the page cache.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/fuse/file.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index 84e39426862a..2f704f522b00 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -797,7 +797,7 @@ static int fuse_do_readfolio(struct file *file, struct folio *folio)
 	struct inode *inode = folio->mapping->host;
 	struct fuse_mount *fm = get_fuse_mount(inode);
 	loff_t pos = folio_pos(folio);
-	struct fuse_folio_desc desc = { .length = PAGE_SIZE };
+	struct fuse_folio_desc desc = { .length = folio_size(folio) };
 	struct fuse_io_args ia = {
 		.ap.args.page_zeroing = true,
 		.ap.args.out_pages = true,
-- 
2.43.5

[PATCH v3 06/12] fuse: support large folios for symlinks

[Joanne Koong]

Support large folios for symlinks and change the name from
fuse_getlink_page() to fuse_getlink_folio().

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/fuse/dir.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/fs/fuse/dir.c b/fs/fuse/dir.c
index 494ac372ace0..65f31bd3a8bb 100644
--- a/fs/fuse/dir.c
+++ b/fs/fuse/dir.c
@@ -1586,10 +1586,10 @@ static int fuse_permission(struct mnt_idmap *idmap,
 	return err;
 }
 
-static int fuse_readlink_page(struct inode *inode, struct folio *folio)
+static int fuse_readlink_folio(struct inode *inode, struct folio *folio)
 {
 	struct fuse_mount *fm = get_fuse_mount(inode);
-	struct fuse_folio_desc desc = { .length = PAGE_SIZE - 1 };
+	struct fuse_folio_desc desc = { .length = folio_size(folio) - 1 };
 	struct fuse_args_pages ap = {
 		.num_folios = 1,
 		.folios = &folio,
@@ -1644,7 +1644,7 @@ static const char *fuse_get_link(struct dentry *dentry, struct inode *inode,
 	if (!folio)
 		goto out_err;
 
-	err = fuse_readlink_page(inode, folio);
+	err = fuse_readlink_folio(inode, folio);
 	if (err) {
 		folio_put(folio);
 		goto out_err;
@@ -2232,7 +2232,7 @@ void fuse_init_dir(struct inode *inode)
 
 static int fuse_symlink_read_folio(struct file *null, struct folio *folio)
 {
-	int err = fuse_readlink_page(folio->mapping->host, folio);
+	int err = fuse_readlink_folio(folio->mapping->host, folio);
 
 	if (!err)
 		folio_mark_uptodate(folio);
-- 
2.43.5

[PATCH v3 07/12] fuse: support large folios for stores

[Joanne Koong]

Add support for folios larger than one page size for stores.
Also change variable naming from "this_num" to "nr_bytes".

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/fuse/dev.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c
index 2a2a5e66412f..791688750caf 100644
--- a/fs/fuse/dev.c
+++ b/fs/fuse/dev.c
@@ -1655,18 +1655,23 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size,
 	num = outarg.size;
 	while (num) {
 		struct folio *folio;
-		unsigned int this_num;
+		unsigned int folio_offset;
+		unsigned int nr_bytes;
+		unsigned int nr_pages;
 
 		folio = filemap_grab_folio(mapping, index);
 		err = PTR_ERR(folio);
 		if (IS_ERR(folio))
 			goto out_iput;
 
-		this_num = min_t(unsigned, num, folio_size(folio) - offset);
-		err = fuse_copy_folio(cs, &folio, offset, this_num, 0);
+		folio_offset = ((index - folio->index) << PAGE_SHIFT) + offset;
+		nr_bytes = min_t(unsigned, num, folio_size(folio) - folio_offset);
+		nr_pages = (offset + nr_bytes + PAGE_SIZE - 1) >> PAGE_SHIFT;
+
+		err = fuse_copy_folio(cs, &folio, folio_offset, nr_bytes, 0);
 		if (!folio_test_uptodate(folio) && !err && offset == 0 &&
-		    (this_num == folio_size(folio) || file_size == end)) {
-			folio_zero_segment(folio, this_num, folio_size(folio));
+		    (nr_bytes == folio_size(folio) || file_size == end)) {
+			folio_zero_segment(folio, nr_bytes, folio_size(folio));
 			folio_mark_uptodate(folio);
 		}
 		folio_unlock(folio);
@@ -1675,9 +1680,9 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size,
 		if (err)
 			goto out_iput;
 
-		num -= this_num;
+		num -= nr_bytes;
 		offset = 0;
-		index++;
+		index += nr_pages;
 	}
 
 	err = 0;
-- 
2.43.5

[PATCH v3 08/12] fuse: support large folios for queued writes

[Joanne Koong]

Add support for folios larger than one page size for queued writes.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/fuse/file.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index 2f704f522b00..94e304a63f9d 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -1798,11 +1798,14 @@ __releases(fi->lock)
 __acquires(fi->lock)
 {
 	struct fuse_inode *fi = get_fuse_inode(wpa->inode);
+	struct fuse_args_pages *ap = &wpa->ia.ap;
 	struct fuse_write_in *inarg = &wpa->ia.write.in;
-	struct fuse_args *args = &wpa->ia.ap.args;
-	/* Currently, all folios in FUSE are one page */
-	__u64 data_size = wpa->ia.ap.num_folios * PAGE_SIZE;
-	int err;
+	struct fuse_args *args = &ap->args;
+	__u64 data_size = 0;
+	int err, i;
+
+	for (i = 0; i < ap->num_folios; i++)
+		data_size += ap->descs[i].length;
 
 	fi->writectr++;
 	if (inarg->offset + data_size <= size) {
-- 
2.43.5

[PATCH v3 09/12] fuse: support large folios for readahead

[Joanne Koong]

Add support for folios larger than one page size for readahead.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
---
 fs/fuse/file.c | 26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index 94e304a63f9d..971624557810 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -885,14 +885,13 @@ static void fuse_readpages_end(struct fuse_mount *fm, struct fuse_args *args,
 	fuse_io_free(ia);
 }
 
-static void fuse_send_readpages(struct fuse_io_args *ia, struct file *file)
+static void fuse_send_readpages(struct fuse_io_args *ia, struct file *file,
+				unsigned int count)
 {
 	struct fuse_file *ff = file->private_data;
 	struct fuse_mount *fm = ff->fm;
 	struct fuse_args_pages *ap = &ia->ap;
 	loff_t pos = folio_pos(ap->folios[0]);
-	/* Currently, all folios in FUSE are one page */
-	size_t count = ap->num_folios << PAGE_SHIFT;
 	ssize_t res;
 	int err;
 
@@ -929,6 +928,7 @@ static void fuse_readahead(struct readahead_control *rac)
 	unsigned int max_pages, nr_pages;
 	loff_t first = readahead_pos(rac);
 	loff_t last = first + readahead_length(rac) - 1;
+	struct folio *folio = NULL;
 
 	if (fuse_is_bad(inode))
 		return;
@@ -952,8 +952,8 @@ static void fuse_readahead(struct readahead_control *rac)
 	while (nr_pages) {
 		struct fuse_io_args *ia;
 		struct fuse_args_pages *ap;
-		struct folio *folio;
 		unsigned cur_pages = min(max_pages, nr_pages);
+		unsigned int pages = 0;
 
 		if (fc->num_background >= fc->congestion_threshold &&
 		    rac->ra->async_size >= readahead_count(rac))
@@ -968,14 +968,24 @@ static void fuse_readahead(struct readahead_control *rac)
 			return;
 		ap = &ia->ap;
 
-		while (ap->num_folios < cur_pages) {
-			folio = readahead_folio(rac);
+		while (pages < cur_pages) {
+			unsigned int folio_pages;
+
+			if (!folio)
+				folio = readahead_folio(rac);
+
+			folio_pages = folio_nr_pages(folio);
+			if (folio_pages > cur_pages - pages)
+				break;
+
 			ap->folios[ap->num_folios] = folio;
 			ap->descs[ap->num_folios].length = folio_size(folio);
 			ap->num_folios++;
+			pages += folio_pages;
+			folio = NULL;
 		}
-		fuse_send_readpages(ia, rac->file);
-		nr_pages -= cur_pages;
+		fuse_send_readpages(ia, rac->file, pages << PAGE_SHIFT);
+		nr_pages -= pages;
 	}
 }
 
-- 
2.43.5

[PATCH v3 10/12] fuse: optimize direct io large folios processing

[Joanne Koong]

Optimize processing folios larger than one page size for the direct io
case. If contiguous pages are part of the same folio, collate the
processing instead of processing each page in the folio separately.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
---
 fs/fuse/file.c | 52 +++++++++++++++++++++++++++++++++++++-------------
 1 file changed, 39 insertions(+), 13 deletions(-)

diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index 971624557810..bbc862c1b3fa 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -1484,7 +1484,8 @@ static int fuse_get_user_pages(struct fuse_args_pages *ap, struct iov_iter *ii,
 	}
 
 	while (nbytes < *nbytesp && nr_pages < max_pages) {
-		unsigned nfolios, i;
+		struct folio *prev_folio = NULL;
+		unsigned npages, i;
 		size_t start;
 
 		ret = iov_iter_extract_pages(ii, &pages,
@@ -1496,23 +1497,48 @@ static int fuse_get_user_pages(struct fuse_args_pages *ap, struct iov_iter *ii,
 
 		nbytes += ret;
 
-		nfolios = DIV_ROUND_UP(ret + start, PAGE_SIZE);
+		npages = DIV_ROUND_UP(ret + start, PAGE_SIZE);
 
-		for (i = 0; i < nfolios; i++) {
+		/*
+		 * We must check each extracted page. We can't assume every page
+		 * in a large folio is used. For example, userspace may mmap() a
+		 * file PROT_WRITE, MAP_PRIVATE, and then store to the middle of
+		 * a large folio, in which case the extracted pages could be
+		 *
+		 * folio A page 0
+		 * folio A page 1
+		 * folio B page 0
+		 * folio A page 3
+		 *
+		 * where folio A belongs to the file and folio B is an anonymous
+		 * COW page.
+		 */
+		for (i = 0; i < npages && ret; i++) {
 			struct folio *folio = page_folio(pages[i]);
-			unsigned int offset = start +
-				(folio_page_idx(folio, pages[i]) << PAGE_SHIFT);
-			unsigned int len = min_t(unsigned int, ret, PAGE_SIZE - start);
+			unsigned int offset;
+			unsigned int len;
+
+			WARN_ON(!folio);
+
+			len = min_t(unsigned int, ret, PAGE_SIZE - start);
+
+			if (folio == prev_folio && pages[i] != pages[i - 1]) {
+				WARN_ON(ap->folios[ap->num_folios - 1] != folio);
+				ap->descs[ap->num_folios - 1].length += len;
+				WARN_ON(ap->descs[ap->num_folios - 1].length > folio_size(folio));
+			} else {
+				offset = start + (folio_page_idx(folio, pages[i]) << PAGE_SHIFT);
+				ap->descs[ap->num_folios].offset = offset;
+				ap->descs[ap->num_folios].length = len;
+				ap->folios[ap->num_folios] = folio;
+				start = 0;
+				ap->num_folios++;
+				prev_folio = folio;
+			}
 
-			ap->descs[ap->num_folios].offset = offset;
-			ap->descs[ap->num_folios].length = len;
-			ap->folios[ap->num_folios] = folio;
-			start = 0;
 			ret -= len;
-			ap->num_folios++;
 		}
-
-		nr_pages += nfolios;
+		nr_pages += npages;
 	}
 	kfree(pages);
 
-- 
2.43.5

[PATCH v3 11/12] fuse: support large folios for writeback

[Joanne Koong]

Add support for folios larger than one page size for writeback.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/fuse/file.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index bbc862c1b3fa..6a7141e73606 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -2014,7 +2014,7 @@ static void fuse_writepage_args_page_fill(struct fuse_writepage_args *wpa, struc
 	folio_get(folio);
 	ap->folios[folio_index] = folio;
 	ap->descs[folio_index].offset = 0;
-	ap->descs[folio_index].length = PAGE_SIZE;
+	ap->descs[folio_index].length = folio_size(folio);
 
 	inc_wb_stat(&inode_to_bdi(inode)->wb, WB_WRITEBACK);
 	node_stat_add_folio(folio, NR_WRITEBACK);
@@ -2089,6 +2089,7 @@ struct fuse_fill_wb_data {
 	struct fuse_file *ff;
 	struct inode *inode;
 	unsigned int max_folios;
+	unsigned int nr_pages;
 };
 
 static bool fuse_pages_realloc(struct fuse_fill_wb_data *data)
@@ -2136,15 +2137,15 @@ static bool fuse_writepage_need_send(struct fuse_conn *fc, struct folio *folio,
 	WARN_ON(!ap->num_folios);
 
 	/* Reached max pages */
-	if (ap->num_folios == fc->max_pages)
+	if (data->nr_pages + folio_nr_pages(folio) > fc->max_pages)
 		return true;
 
 	/* Reached max write bytes */
-	if ((ap->num_folios + 1) * PAGE_SIZE > fc->max_write)
+	if ((data->nr_pages * PAGE_SIZE) + folio_size(folio) > fc->max_write)
 		return true;
 
 	/* Discontinuity */
-	if (ap->folios[ap->num_folios - 1]->index + 1 != folio_index(folio))
+	if (folio_next_index(ap->folios[ap->num_folios - 1]) != folio_index(folio))
 		return true;
 
 	/* Need to grow the pages array?  If so, did the expansion fail? */
@@ -2175,6 +2176,7 @@ static int fuse_writepages_fill(struct folio *folio,
 	if (wpa && fuse_writepage_need_send(fc, folio, ap, data)) {
 		fuse_writepages_send(data);
 		data->wpa = NULL;
+		data->nr_pages = 0;
 	}
 
 	if (data->wpa == NULL) {
@@ -2189,6 +2191,7 @@ static int fuse_writepages_fill(struct folio *folio,
 	folio_start_writeback(folio);
 
 	fuse_writepage_args_page_fill(wpa, folio, ap->num_folios);
+	data->nr_pages += folio_nr_pages(folio);
 
 	err = 0;
 	ap->num_folios++;
@@ -2219,6 +2222,7 @@ static int fuse_writepages(struct address_space *mapping,
 	data.inode = inode;
 	data.wpa = NULL;
 	data.ff = NULL;
+	data.nr_pages = 0;
 
 	err = write_cache_pages(mapping, wbc, fuse_writepages_fill, &data);
 	if (data.wpa) {
-- 
2.43.5

[PATCH v3 12/12] fuse: enable large folios

[Joanne Koong]

Enable folios larger than one page size.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
---
 fs/fuse/file.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index 6a7141e73606..e313ded276a9 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -3187,12 +3187,17 @@ void fuse_init_file_inode(struct inode *inode, unsigned int flags)
 {
 	struct fuse_inode *fi = get_fuse_inode(inode);
 	struct fuse_conn *fc = get_fuse_conn(inode);
+	unsigned int max_pages, max_order;
 
 	inode->i_fop = &fuse_file_operations;
 	inode->i_data.a_ops = &fuse_file_aops;
 	if (fc->writeback_cache)
 		mapping_set_writeback_may_block(&inode->i_data);
 
+	max_pages = min(fc->max_write >> PAGE_SHIFT, fc->max_pages);
+	max_order = ilog2(max_pages);
+	mapping_set_folio_order_range(inode->i_mapping, 0, max_order);
+
 	INIT_LIST_HEAD(&fi->write_files);
 	INIT_LIST_HEAD(&fi->queued_writes);
 	fi->writectr = 0;
-- 
2.43.5

Re: [PATCH v3 04/12] fuse: support large folios for writethrough writes

[Jeff Layton]

On Fri, 2024-12-13 at 14:18 -0800, Joanne Koong wrote:
> Add support for folios larger than one page size for writethrough
> writes.
> 
> Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
> ---
>  fs/fuse/file.c | 19 ++++++++++++-------
>  1 file changed, 12 insertions(+), 7 deletions(-)
> 
> diff --git a/fs/fuse/file.c b/fs/fuse/file.c
> index c041bb328203..84e39426862a 100644
> --- a/fs/fuse/file.c
> +++ b/fs/fuse/file.c
> @@ -1135,6 +1135,7 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
>  				     struct iov_iter *ii, loff_t pos,
>  				     unsigned int max_pages)
>  {
> +	size_t max_folio_size = mapping_max_folio_size(mapping);
>  	struct fuse_args_pages *ap = &ia->ap;
>  	struct fuse_conn *fc = get_fuse_conn(mapping->host);
>  	unsigned offset = pos & (PAGE_SIZE - 1);
> @@ -1146,17 +1147,17 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
>  	num = min(num, max_pages << PAGE_SHIFT);
>  
>  	ap->args.in_pages = true;
> -	ap->descs[0].offset = offset;
>  
>  	while (num) {
>  		size_t tmp;
>  		struct folio *folio;
>  		pgoff_t index = pos >> PAGE_SHIFT;
> -		unsigned int bytes = min(PAGE_SIZE - offset, num);
> +		unsigned int bytes;
> +		unsigned int folio_offset;
>  
>   again:
>  		err = -EFAULT;
> -		if (fault_in_iov_iter_readable(ii, bytes))
> +		if (fault_in_iov_iter_readable(ii, max_folio_size) == max_folio_size)
>  			break;
>  
>  		folio = __filemap_get_folio(mapping, index, FGP_WRITEBEGIN,
> @@ -1169,7 +1170,10 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
>  		if (mapping_writably_mapped(mapping))
>  			flush_dcache_folio(folio);
>  
> -		tmp = copy_folio_from_iter_atomic(folio, offset, bytes, ii);
> +		folio_offset = ((index - folio->index) << PAGE_SHIFT) + offset;
> +		bytes = min(folio_size(folio) - folio_offset, num);
> +
> +		tmp = copy_folio_from_iter_atomic(folio, folio_offset, bytes, ii);

Just to save someone else going down the same rabbit hole:

copy_folio_from_iter_atomic() is defined as:

static inline size_t copy_folio_from_iter_atomic(struct folio *folio,
                size_t offset, size_t bytes, struct iov_iter *i)
{
        return copy_page_from_iter_atomic(&folio->page, offset, bytes, i);
}

...which _looks_ sort of like it's not fully baked yet and can't handle
a large folio, but it turns out that copy_page_from_iter_atomic() can
handle compound pages, so I think this is actually OK. Whew!


>  		flush_dcache_folio(folio);
>  
>  		if (!tmp) {
> @@ -1180,6 +1184,7 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
>  
>  		err = 0;
>  		ap->folios[ap->num_folios] = folio;
> +		ap->descs[ap->num_folios].offset = folio_offset;
>  		ap->descs[ap->num_folios].length = tmp;
>  		ap->num_folios++;
>  
> @@ -1187,11 +1192,11 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
>  		pos += tmp;
>  		num -= tmp;
>  		offset += tmp;
> -		if (offset == PAGE_SIZE)
> +		if (offset == folio_size(folio))
>  			offset = 0;
>  
> -		/* If we copied full page, mark it uptodate */
> -		if (tmp == PAGE_SIZE)
> +		/* If we copied full folio, mark it uptodate */
> +		if (tmp == folio_size(folio))
>  			folio_mark_uptodate(folio);
>  
>  		if (folio_test_uptodate(folio)) {

-- 
Jeff Layton <jlayton@kernel.org>

Re: [PATCH v3 00/12] fuse: support large folios

[Jeff Layton]

On Fri, 2024-12-13 at 14:18 -0800, Joanne Koong wrote:
> This patchset adds support for folios larger than one page size in FUSE.
> 
> This patchset is rebased on top of the (unmerged) patchset that removes temp
> folios in writeback [1]. This patchset was tested by running it through fstests
> on passthrough_hp.
> 
> Please note that writes are still effectively one page size. Larger writes can
> be enabled by setting the order on the fgp flag passed in to __filemap_get_folio()
> but benchmarks show this significantly degrades performance. More investigation
> needs to be done into this. As such, buffered writes will be optimized in a
> future patchset.
> 
> Benchmarks show roughly a ~45% improvement in read throughput.
> 
> Benchmark setup:
> 
> -- Set up server --
>  ./libfuse/build/example/passthrough_hp --bypass-rw=1 ~/libfuse
> ~/mounts/fuse/ --nopassthrough
> (using libfuse patched with https://github.com/libfuse/libfuse/pull/807)
> 
> -- Run fio --
>  fio --name=read --ioengine=sync --rw=read --bs=1M --size=1G
> --numjobs=2 --ramp_time=30 --group_reporting=1
> --directory=mounts/fuse/
> 
> Machine 1:
>     No large folios:     ~4400 MiB/s
>     Large folios:        ~7100 MiB/s
> 
> Machine 2:
>     No large folios:     ~3700 MiB/s
>     Large folios:        ~6400 MiB/s
> 
> 
> [1] https://lore.kernel.org/linux-fsdevel/20241122232359.429647-1-joannelkoong@gmail.com/
> 
> Changelog:
> v2: https://lore.kernel.org/linux-fsdevel/20241125220537.3663725-1-joannelkoong@gmail.com/
> v2 -> v3:
> * Fix direct io parsing to check each extracted page instead of assuming all
>   pages in a large folio will be used (Matthew)
> 
> v1: https://lore.kernel.org/linux-fsdevel/20241109001258.2216604-1-joannelkoong@gmail.com/
> v1 -> v2:
> * Change naming from "non-writeback write" to "writethrough write"
> * Fix deadlock for writethrough writes by calling fault_in_iov_iter_readable()
> * first
>   before __filemap_get_folio() (Josef)
> * For readahead, retain original folio_size() for descs.length (Josef)
> * Use folio_zero_range() api in fuse_copy_folio() (Josef)
> * Add Josef's reviewed-bys
> 
> Joanne Koong (12):
>   fuse: support copying large folios
>   fuse: support large folios for retrieves
>   fuse: refactor fuse_fill_write_pages()
>   fuse: support large folios for writethrough writes
>   fuse: support large folios for folio reads
>   fuse: support large folios for symlinks
>   fuse: support large folios for stores
>   fuse: support large folios for queued writes
>   fuse: support large folios for readahead
>   fuse: optimize direct io large folios processing
>   fuse: support large folios for writeback
>   fuse: enable large folios
> 
>  fs/fuse/dev.c  | 128 ++++++++++++++++++++++---------------------
>  fs/fuse/dir.c  |   8 +--
>  fs/fuse/file.c | 144 +++++++++++++++++++++++++++++++++----------------
>  3 files changed, 166 insertions(+), 114 deletions(-)
> 


Nice work, Joanne!

Reviewed-by: Jeff Layton <jlayton@kernel.org>

Re: [PATCH v3 04/12] fuse: support large folios for writethrough writes

[Matthew Wilcox]

On Thu, Dec 19, 2024 at 01:08:15PM -0500, Jeff Layton wrote:
> > +		tmp = copy_folio_from_iter_atomic(folio, folio_offset, bytes, ii);
> 
> Just to save someone else going down the same rabbit hole:
> 
> copy_folio_from_iter_atomic() is defined as:
> 
> static inline size_t copy_folio_from_iter_atomic(struct folio *folio,
>                 size_t offset, size_t bytes, struct iov_iter *i)
> {
>         return copy_page_from_iter_atomic(&folio->page, offset, bytes, i);
> }
> 
> ...which _looks_ sort of like it's not fully baked yet and can't handle
> a large folio, but it turns out that copy_page_from_iter_atomic() can
> handle compound pages, so I think this is actually OK. Whew!

Yes, this is fine.  I'd love to clean this up further.  If someone wants
the kudos of doing that, ntfs_compress_write() is your challenge.  It's
the only remaining caller of copy_page_from_iter_atomic() and
once it's converted to use folios, we can push folios deeper into
the iov_iter code.

Re: [PATCH v3 00/12] fuse: support large folios

[Joanne Koong]

On Fri, Dec 13, 2024 at 2:23 PM Joanne Koong <joannelkoong@gmail.com> wrote:
>
> This patchset adds support for folios larger than one page size in FUSE.
>
> This patchset is rebased on top of the (unmerged) patchset that removes temp
> folios in writeback [1]. This patchset was tested by running it through fstests
> on passthrough_hp.
>
> Please note that writes are still effectively one page size. Larger writes can
> be enabled by setting the order on the fgp flag passed in to __filemap_get_folio()
> but benchmarks show this significantly degrades performance. More investigation
> needs to be done into this. As such, buffered writes will be optimized in a
> future patchset.
>
> Benchmarks show roughly a ~45% improvement in read throughput.
>
> Benchmark setup:
>
> -- Set up server --
>  ./libfuse/build/example/passthrough_hp --bypass-rw=1 ~/libfuse
> ~/mounts/fuse/ --nopassthrough
> (using libfuse patched with https://github.com/libfuse/libfuse/pull/807)
>
> -- Run fio --
>  fio --name=read --ioengine=sync --rw=read --bs=1M --size=1G
> --numjobs=2 --ramp_time=30 --group_reporting=1
> --directory=mounts/fuse/
>
> Machine 1:
>     No large folios:     ~4400 MiB/s
>     Large folios:        ~7100 MiB/s
>
> Machine 2:
>     No large folios:     ~3700 MiB/s
>     Large folios:        ~6400 MiB/s
>
>
> [1] https://lore.kernel.org/linux-fsdevel/20241122232359.429647-1-joannelkoong@gmail.com/
>

A couple of updates on this:
* I'm going to remove the writeback patch (patch 11/12) in this series
and resubmit, and leave large folios writeback to be done as a
separate future patchset. Getting writeback to work with large folios
has a dependency on [1], which unfortunately does not look like it'll
be resolved anytime soon. If we cannot remove tmp pages, then we'll
likely need to use a different data structure than the rb tree to
account for large folios w/ tmp pages. I believe we can still enable
large folios overall even without large folios writeback, as even with
the inode->i_mapping set to a large folio order range, writeback will
still only operate on 4k folios until fgf_set_order() is explicitly
set in fuse_write_begin() for the __filemap_get_folio() call.

* There's a discussion here [2] about perf degradation for writeback
writes on large folios due to writeback throttling when balancing
dirty pages. This is due to fuse enabling bdi strictlimit. More
experimentation will be needed to figure out what a good folio order
is, and whether it's possible to do something like remove the
strictlimit for privileged servers.

* Writeback on FUSE will need support for more granular dirty
tracking, so that we don't have to write back the entire large folio
if only a few pages in it are dirtied. I'm planning to take a look at
iomap and netfs and see if maybe FUSE can hook into that for it.


Thanks,
Joanne

[1] https://lore.kernel.org/linux-fsdevel/20241122232359.429647-1-joannelkoong@gmail.com/
[2] https://lore.kernel.org/linux-fsdevel/CAJnrk1a38pv3OgFZRfdTiDMXuPWuBgN8KY47XfOsYHj=N2wxAg@mail.gmail.com/

> Changelog:
> v2: https://lore.kernel.org/linux-fsdevel/20241125220537.3663725-1-joannelkoong@gmail.com/
> v2 -> v3:
> * Fix direct io parsing to check each extracted page instead of assuming all
>   pages in a large folio will be used (Matthew)
>
> v1: https://lore.kernel.org/linux-fsdevel/20241109001258.2216604-1-joannelkoong@gmail.com/
> v1 -> v2:
> * Change naming from "non-writeback write" to "writethrough write"
> * Fix deadlock for writethrough writes by calling fault_in_iov_iter_readable()
> * first
>   before __filemap_get_folio() (Josef)
> * For readahead, retain original folio_size() for descs.length (Josef)
> * Use folio_zero_range() api in fuse_copy_folio() (Josef)
> * Add Josef's reviewed-bys
>
> Joanne Koong (12):
>   fuse: support copying large folios
>   fuse: support large folios for retrieves
>   fuse: refactor fuse_fill_write_pages()
>   fuse: support large folios for writethrough writes
>   fuse: support large folios for folio reads
>   fuse: support large folios for symlinks
>   fuse: support large folios for stores
>   fuse: support large folios for queued writes
>   fuse: support large folios for readahead
>   fuse: optimize direct io large folios processing
>   fuse: support large folios for writeback
>   fuse: enable large folios
>
>  fs/fuse/dev.c  | 128 ++++++++++++++++++++++---------------------
>  fs/fuse/dir.c  |   8 +--
>  fs/fuse/file.c | 144 +++++++++++++++++++++++++++++++++----------------
>  3 files changed, 166 insertions(+), 114 deletions(-)
>
> --
> 2.43.5
>

Re: [PATCH v3 00/12] fuse: support large folios

[Jingbo Xu]

On 1/23/25 7:23 AM, Joanne Koong wrote:
> On Fri, Dec 13, 2024 at 2:23 PM Joanne Koong <joannelkoong@gmail.com> wrote:
>>
>> This patchset adds support for folios larger than one page size in FUSE.
>>
>> This patchset is rebased on top of the (unmerged) patchset that removes temp
>> folios in writeback [1]. This patchset was tested by running it through fstests
>> on passthrough_hp.
>>
>> Please note that writes are still effectively one page size. Larger writes can
>> be enabled by setting the order on the fgp flag passed in to __filemap_get_folio()
>> but benchmarks show this significantly degrades performance. More investigation
>> needs to be done into this. As such, buffered writes will be optimized in a
>> future patchset.
>>
>> Benchmarks show roughly a ~45% improvement in read throughput.
>>
>> Benchmark setup:
>>
>> -- Set up server --
>>  ./libfuse/build/example/passthrough_hp --bypass-rw=1 ~/libfuse
>> ~/mounts/fuse/ --nopassthrough
>> (using libfuse patched with https://github.com/libfuse/libfuse/pull/807)
>>
>> -- Run fio --
>>  fio --name=read --ioengine=sync --rw=read --bs=1M --size=1G
>> --numjobs=2 --ramp_time=30 --group_reporting=1
>> --directory=mounts/fuse/
>>
>> Machine 1:
>>     No large folios:     ~4400 MiB/s
>>     Large folios:        ~7100 MiB/s
>>
>> Machine 2:
>>     No large folios:     ~3700 MiB/s
>>     Large folios:        ~6400 MiB/s
>>
>>
>> [1] https://lore.kernel.org/linux-fsdevel/20241122232359.429647-1-joannelkoong@gmail.com/
>>
> 
> A couple of updates on this:
> * I'm going to remove the writeback patch (patch 11/12) in this series
> and resubmit, and leave large folios writeback to be done as a
> separate future patchset. Getting writeback to work with large folios
> has a dependency on [1], which unfortunately does not look like it'll
> be resolved anytime soon. If we cannot remove tmp pages, then we'll
> likely need to use a different data structure than the rb tree to
> account for large folios w/ tmp pages. I believe we can still enable
> large folios overall even without large folios writeback, as even with
> the inode->i_mapping set to a large folio order range, writeback will
> still only operate on 4k folios until fgf_set_order() is explicitly
> set in fuse_write_begin() for the __filemap_get_folio() call.
> 
> * There's a discussion here [2] about perf degradation for writeback
> writes on large folios due to writeback throttling when balancing
> dirty pages. This is due to fuse enabling bdi strictlimit. More
> experimentation will be needed to figure out what a good folio order
> is, and whether it's possible to do something like remove the
> strictlimit for privileged servers.

FYI the sysadmin can already disable strictlimit for FUSE through
/sys/class/bdi/<bdi>/strict_limit knob[*].

[*] https://lore.kernel.org/all/20221119005215.3052436-1-shr@devkernel.io/

-- 
Thanks,
Jingbo

Re: [PATCH v3 00/12] fuse: support large folios

[Joanne Koong]

On Wed, Jan 22, 2025 at 5:24 PM Jingbo Xu <jefflexu@linux.alibaba.com> wrote:
>
>
>
> On 1/23/25 7:23 AM, Joanne Koong wrote:
> > On Fri, Dec 13, 2024 at 2:23 PM Joanne Koong <joannelkoong@gmail.com> wrote:
> >>
> >> This patchset adds support for folios larger than one page size in FUSE.
> >>
> >> This patchset is rebased on top of the (unmerged) patchset that removes temp
> >> folios in writeback [1]. This patchset was tested by running it through fstests
> >> on passthrough_hp.
> >>
> >> Please note that writes are still effectively one page size. Larger writes can
> >> be enabled by setting the order on the fgp flag passed in to __filemap_get_folio()
> >> but benchmarks show this significantly degrades performance. More investigation
> >> needs to be done into this. As such, buffered writes will be optimized in a
> >> future patchset.
> >>
> >> Benchmarks show roughly a ~45% improvement in read throughput.
> >>
> >> Benchmark setup:
> >>
> >> -- Set up server --
> >>  ./libfuse/build/example/passthrough_hp --bypass-rw=1 ~/libfuse
> >> ~/mounts/fuse/ --nopassthrough
> >> (using libfuse patched with https://github.com/libfuse/libfuse/pull/807)
> >>
> >> -- Run fio --
> >>  fio --name=read --ioengine=sync --rw=read --bs=1M --size=1G
> >> --numjobs=2 --ramp_time=30 --group_reporting=1
> >> --directory=mounts/fuse/
> >>
> >> Machine 1:
> >>     No large folios:     ~4400 MiB/s
> >>     Large folios:        ~7100 MiB/s
> >>
> >> Machine 2:
> >>     No large folios:     ~3700 MiB/s
> >>     Large folios:        ~6400 MiB/s
> >>
> >>
> >> [1] https://lore.kernel.org/linux-fsdevel/20241122232359.429647-1-joannelkoong@gmail.com/
> >>
> >
> > A couple of updates on this:
> > * I'm going to remove the writeback patch (patch 11/12) in this series
> > and resubmit, and leave large folios writeback to be done as a
> > separate future patchset. Getting writeback to work with large folios
> > has a dependency on [1], which unfortunately does not look like it'll
> > be resolved anytime soon. If we cannot remove tmp pages, then we'll
> > likely need to use a different data structure than the rb tree to
> > account for large folios w/ tmp pages. I believe we can still enable
> > large folios overall even without large folios writeback, as even with
> > the inode->i_mapping set to a large folio order range, writeback will
> > still only operate on 4k folios until fgf_set_order() is explicitly
> > set in fuse_write_begin() for the __filemap_get_folio() call.
> >
> > * There's a discussion here [2] about perf degradation for writeback
> > writes on large folios due to writeback throttling when balancing
> > dirty pages. This is due to fuse enabling bdi strictlimit. More
> > experimentation will be needed to figure out what a good folio order
> > is, and whether it's possible to do something like remove the
> > strictlimit for privileged servers.
>
> FYI the sysadmin can already disable strictlimit for FUSE through
> /sys/class/bdi/<bdi>/strict_limit knob[*].
>
> [*] https://lore.kernel.org/all/20221119005215.3052436-1-shr@devkernel.io/

Oh cool, thanks for pointing this out! AFAICT, this means the sysadmin
would have to do this individually for every fuse server that gets
run. I wonder if we should do something like a) have fuse only enforce
the strictlimit for unprivileged servers or b) add a fuse sysctl that
sysadmins can set more easily for removing strictlimit for any server
that gets run instead of having to do it individually

Thanks,
Joanne

>
> --
> Thanks,
> Jingbo

Re: [PATCH v3 00/12] fuse: support large folios

[Matthew Wilcox]

On Wed, Jan 22, 2025 at 03:23:08PM -0800, Joanne Koong wrote:
> * I'm going to remove the writeback patch (patch 11/12) in this series
> and resubmit, and leave large folios writeback to be done as a
> separate future patchset. Getting writeback to work with large folios
> has a dependency on [1], which unfortunately does not look like it'll
> be resolved anytime soon. If we cannot remove tmp pages, then we'll
> likely need to use a different data structure than the rb tree to
> account for large folios w/ tmp pages. I believe we can still enable
> large folios overall even without large folios writeback, as even with
> the inode->i_mapping set to a large folio order range, writeback will
> still only operate on 4k folios until fgf_set_order() is explicitly
> set in fuse_write_begin() for the __filemap_get_folio() call.

Maybe you already understand this and just expressed yourself badly,
but what you've said isn't true.

The fgf_set_order() call is about creating large folios during write().
If instead you do a large read() (or do consecutive read() calls which
get turned into large readaheads), you'll get large clean folios.
If you then dirty those folios, we won't split them.  Writeback will
still see large folios in this case.

It depends on your workload how common a scenario this is.

Re: [PATCH v3 00/12] fuse: support large folios

[Joanne Koong]

On Thu, Jan 23, 2025 at 10:24 AM Matthew Wilcox <willy@infradead.org> wrote:
>
> On Wed, Jan 22, 2025 at 03:23:08PM -0800, Joanne Koong wrote:
> > * I'm going to remove the writeback patch (patch 11/12) in this series
> > and resubmit, and leave large folios writeback to be done as a
> > separate future patchset. Getting writeback to work with large folios
> > has a dependency on [1], which unfortunately does not look like it'll
> > be resolved anytime soon. If we cannot remove tmp pages, then we'll
> > likely need to use a different data structure than the rb tree to
> > account for large folios w/ tmp pages. I believe we can still enable
> > large folios overall even without large folios writeback, as even with
> > the inode->i_mapping set to a large folio order range, writeback will
> > still only operate on 4k folios until fgf_set_order() is explicitly
> > set in fuse_write_begin() for the __filemap_get_folio() call.
>
> Maybe you already understand this and just expressed yourself badly,
> but what you've said isn't true.
>
> The fgf_set_order() call is about creating large folios during write().
> If instead you do a large read() (or do consecutive read() calls which
> get turned into large readaheads), you'll get large clean folios.
> If you then dirty those folios, we won't split them.  Writeback will
> still see large folios in this case.
>
> It depends on your workload how common a scenario this is.

Hi Matthew,

Thanks for the correction. I realized my misassumption after I sent
the email, so for v4 [1] I ended up dropping both patch 11 (writeback)
and 12 (turning large folios on).


[1] https://lore.kernel.org/linux-fsdevel/20250123012448.2479372-1-joannelkoong@gmail.com/

Thanks,
Joanne