From: Jeffle Xu <jefflexu@linux.alibaba.com>
To: dhowells@redhat.com, linux-cachefs@redhat.com, xiang@kernel.org,
chao@kernel.org, linux-erofs@lists.ozlabs.org
Cc: gregkh@linuxfoundation.org, willy@infradead.org,
linux-kernel@vger.kernel.org, joseph.qi@linux.alibaba.com,
linux-fsdevel@vger.kernel.org, gerry@linux.alibaba.com,
torvalds@linux-foundation.org
Subject: [PATCH v3 02/22] fscache: add a method to support on-demand read semantics
Date: Wed, 9 Feb 2022 14:00:48 +0800 [thread overview]
Message-ID: <20220209060108.43051-3-jefflexu@linux.alibaba.com> (raw)
In-Reply-To: <20220209060108.43051-1-jefflexu@linux.alibaba.com>
Add .ondemand_read() callback to netfs_cache_ops to implement on-demand
read.
The precondition for implementing on-demand read semantics is that,
all blob files have been placed under corresponding directory with
correct file size (sparse files) on the first beginning. When upper fs
starts to access the blob file, it will "cache miss" (hit the hole).
Then .ondemand_read() callback can be called to notify backend to
prepare the data.
The implementation of .ondemand_read() callback can be backend specific.
The following patch will introduce the implementation for cachefiles,
which will notify user daemon the requested file range to read. The
.ondemand_read() callback will get blocked until the user daemon has
prepared the corresponding data.
Then once .ondemand_read() callback returns with 0, it is guaranteed
that the requested data has been ready. In this case, users can retry to
read from the backing file.
Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com>
---
Documentation/filesystems/netfs_library.rst | 18 +++++++++++++++
include/linux/fscache.h | 25 +++++++++++++++++++++
include/linux/netfs.h | 4 ++++
3 files changed, 47 insertions(+)
diff --git a/Documentation/filesystems/netfs_library.rst b/Documentation/filesystems/netfs_library.rst
index 4f373a8ec47b..e544d6688100 100644
--- a/Documentation/filesystems/netfs_library.rst
+++ b/Documentation/filesystems/netfs_library.rst
@@ -466,6 +466,8 @@ operation table looks like the following::
int (*query_occupancy)(struct netfs_cache_resources *cres,
loff_t start, size_t len, size_t granularity,
loff_t *_data_start, size_t *_data_len);
+ int (*ondemand_read)(struct netfs_cache_resources *cres,
+ loff_t start_pos, size_t len);
};
With a termination handler function pointer::
@@ -552,6 +554,22 @@ The methods defined in the table are:
It returns 0 if some data was found, -ENODATA if there was no usable data
within the region or -ENOBUFS if there is no caching on this file.
+ * ``ondemand_read()``
+
+ [Optional] Called to make cache prepare for the data. It shall be called only
+ when on-demand read semantics is required. It will be called when a cache
+ miss is encountered. The function will make the backend somehow prepare for
+ the data in the region specified by @start_pos/@len of the cache file. It may
+ get blocked until the backend has prepared the data in the cache file
+ successfully, or error encountered.
+
+ Once it returns with 0, it is guaranteed that the requested data has been
+ ready in the cache file. In this case, users can retry to read from the cache
+ file.
+
+ It returns 0 if data has been ready in the cache file, or other error code
+ from the cache, such as -ENOMEM.
+
Note that these methods are passed a pointer to the cache resource structure,
not the read request structure as they could be used in other situations where
there isn't a read request structure as well, such as writing dirty data to the
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index d2430da8aa67..efcd5d5c6726 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -514,6 +514,31 @@ int fscache_read(struct netfs_cache_resources *cres,
term_func, term_func_priv);
}
+/**
+ * fscache_ondemand_read - Make cache prepare for the data.
+ * @cres: The cache resources to use
+ * @start_pos: The beginning file offset in the cache file
+ * @len: The length of the file offset range in the cache file
+ *
+ * This shall only be called when a cache miss is encountered. It will make
+ * the backend somehow prepare for the data in the file offset range specified
+ * by @start_pos/@len of the cache file. It may get blocked until the backend
+ * has prepared the data in the cache file successfully, or error encountered.
+ *
+ * Returns:
+ * * 0 - Success (Data is ready in the cache file)
+ * * Other error code from the cache, such as -ENOMEM.
+ */
+static inline
+int fscache_ondemand_read(struct netfs_cache_resources *cres,
+ loff_t start_pos, size_t len)
+{
+ const struct netfs_cache_ops *ops = fscache_operation_valid(cres);
+ if (ops->ondemand_read)
+ return ops->ondemand_read(cres, start_pos, len);
+ return -EOPNOTSUPP;
+}
+
/**
* fscache_begin_write_operation - Begin a write operation for the netfs lib
* @cres: The cache resources for the write being performed
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 614f22213e21..81fe707ad38d 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -251,6 +251,10 @@ struct netfs_cache_ops {
int (*query_occupancy)(struct netfs_cache_resources *cres,
loff_t start, size_t len, size_t granularity,
loff_t *_data_start, size_t *_data_len);
+
+ /* Make cache prepare for the data */
+ int (*ondemand_read)(struct netfs_cache_resources *cres,
+ loff_t start_pos, size_t len);
};
struct readahead_control;
--
2.27.0
WARNING: multiple messages have this Message-ID (diff)
From: Jeffle Xu <jefflexu@linux.alibaba.com>
To: dhowells@redhat.com, linux-cachefs@redhat.com, xiang@kernel.org,
chao@kernel.org, linux-erofs@lists.ozlabs.org
Cc: torvalds@linux-foundation.org, gregkh@linuxfoundation.org,
willy@infradead.org, linux-fsdevel@vger.kernel.org,
joseph.qi@linux.alibaba.com, bo.liu@linux.alibaba.com,
tao.peng@linux.alibaba.com, gerry@linux.alibaba.com,
eguan@linux.alibaba.com, linux-kernel@vger.kernel.org
Subject: [PATCH v3 02/22] fscache: add a method to support on-demand read semantics
Date: Wed, 9 Feb 2022 14:00:48 +0800 [thread overview]
Message-ID: <20220209060108.43051-3-jefflexu@linux.alibaba.com> (raw)
In-Reply-To: <20220209060108.43051-1-jefflexu@linux.alibaba.com>
Add .ondemand_read() callback to netfs_cache_ops to implement on-demand
read.
The precondition for implementing on-demand read semantics is that,
all blob files have been placed under corresponding directory with
correct file size (sparse files) on the first beginning. When upper fs
starts to access the blob file, it will "cache miss" (hit the hole).
Then .ondemand_read() callback can be called to notify backend to
prepare the data.
The implementation of .ondemand_read() callback can be backend specific.
The following patch will introduce the implementation for cachefiles,
which will notify user daemon the requested file range to read. The
.ondemand_read() callback will get blocked until the user daemon has
prepared the corresponding data.
Then once .ondemand_read() callback returns with 0, it is guaranteed
that the requested data has been ready. In this case, users can retry to
read from the backing file.
Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com>
---
Documentation/filesystems/netfs_library.rst | 18 +++++++++++++++
include/linux/fscache.h | 25 +++++++++++++++++++++
include/linux/netfs.h | 4 ++++
3 files changed, 47 insertions(+)
diff --git a/Documentation/filesystems/netfs_library.rst b/Documentation/filesystems/netfs_library.rst
index 4f373a8ec47b..e544d6688100 100644
--- a/Documentation/filesystems/netfs_library.rst
+++ b/Documentation/filesystems/netfs_library.rst
@@ -466,6 +466,8 @@ operation table looks like the following::
int (*query_occupancy)(struct netfs_cache_resources *cres,
loff_t start, size_t len, size_t granularity,
loff_t *_data_start, size_t *_data_len);
+ int (*ondemand_read)(struct netfs_cache_resources *cres,
+ loff_t start_pos, size_t len);
};
With a termination handler function pointer::
@@ -552,6 +554,22 @@ The methods defined in the table are:
It returns 0 if some data was found, -ENODATA if there was no usable data
within the region or -ENOBUFS if there is no caching on this file.
+ * ``ondemand_read()``
+
+ [Optional] Called to make cache prepare for the data. It shall be called only
+ when on-demand read semantics is required. It will be called when a cache
+ miss is encountered. The function will make the backend somehow prepare for
+ the data in the region specified by @start_pos/@len of the cache file. It may
+ get blocked until the backend has prepared the data in the cache file
+ successfully, or error encountered.
+
+ Once it returns with 0, it is guaranteed that the requested data has been
+ ready in the cache file. In this case, users can retry to read from the cache
+ file.
+
+ It returns 0 if data has been ready in the cache file, or other error code
+ from the cache, such as -ENOMEM.
+
Note that these methods are passed a pointer to the cache resource structure,
not the read request structure as they could be used in other situations where
there isn't a read request structure as well, such as writing dirty data to the
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index d2430da8aa67..efcd5d5c6726 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -514,6 +514,31 @@ int fscache_read(struct netfs_cache_resources *cres,
term_func, term_func_priv);
}
+/**
+ * fscache_ondemand_read - Make cache prepare for the data.
+ * @cres: The cache resources to use
+ * @start_pos: The beginning file offset in the cache file
+ * @len: The length of the file offset range in the cache file
+ *
+ * This shall only be called when a cache miss is encountered. It will make
+ * the backend somehow prepare for the data in the file offset range specified
+ * by @start_pos/@len of the cache file. It may get blocked until the backend
+ * has prepared the data in the cache file successfully, or error encountered.
+ *
+ * Returns:
+ * * 0 - Success (Data is ready in the cache file)
+ * * Other error code from the cache, such as -ENOMEM.
+ */
+static inline
+int fscache_ondemand_read(struct netfs_cache_resources *cres,
+ loff_t start_pos, size_t len)
+{
+ const struct netfs_cache_ops *ops = fscache_operation_valid(cres);
+ if (ops->ondemand_read)
+ return ops->ondemand_read(cres, start_pos, len);
+ return -EOPNOTSUPP;
+}
+
/**
* fscache_begin_write_operation - Begin a write operation for the netfs lib
* @cres: The cache resources for the write being performed
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 614f22213e21..81fe707ad38d 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -251,6 +251,10 @@ struct netfs_cache_ops {
int (*query_occupancy)(struct netfs_cache_resources *cres,
loff_t start, size_t len, size_t granularity,
loff_t *_data_start, size_t *_data_len);
+
+ /* Make cache prepare for the data */
+ int (*ondemand_read)(struct netfs_cache_resources *cres,
+ loff_t start_pos, size_t len);
};
struct readahead_control;
--
2.27.0
next prev parent reply other threads:[~2022-02-09 6:01 UTC|newest]
Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-09 6:00 [PATCH v3 00/22] fscache,erofs: fscache-based demand-read semantics Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-09 6:00 ` [PATCH v3 01/22] fscache: export fscache_end_operation() Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-17 7:44 ` Liu Bo
2022-02-17 7:44 ` Liu Bo
2022-02-09 6:00 ` Jeffle Xu [this message]
2022-02-09 6:00 ` [PATCH v3 02/22] fscache: add a method to support on-demand read semantics Jeffle Xu
2022-02-09 6:00 ` [PATCH v3 03/22] cachefiles: extract generic function for daemon methods Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-17 8:17 ` Liu Bo
2022-02-17 8:17 ` Liu Bo
2022-02-09 6:00 ` [PATCH v3 04/22] cachefiles: detect backing file size in on-demand read mode Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-09 6:00 ` [PATCH v3 05/22] cachefiles: introduce new devnode for " Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-15 9:03 ` JeffleXu
2022-02-15 9:03 ` JeffleXu
2022-02-15 10:37 ` Greg KH
2022-02-15 10:37 ` Greg KH
2022-02-16 8:17 ` JeffleXu
2022-02-16 8:17 ` JeffleXu
2022-02-15 11:13 ` [PATCH v4 05/23] " Jeffle Xu
2022-02-15 11:13 ` Jeffle Xu
2022-02-16 10:48 ` Greg KH
2022-02-16 10:48 ` Greg KH
2022-02-16 12:49 ` JeffleXu
2022-02-16 12:49 ` JeffleXu
2022-02-16 17:48 ` Greg KH
2022-02-16 17:48 ` Greg KH
2022-02-17 1:49 ` JeffleXu
2022-02-17 1:49 ` JeffleXu
2022-02-09 6:00 ` [PATCH v3 06/22] erofs: use meta buffers for erofs_read_superblock() Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-09 7:52 ` Gao Xiang
2022-02-09 7:52 ` Gao Xiang
2022-02-09 6:00 ` [PATCH v3 07/22] erofs: export erofs_map_blocks() Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-09 6:00 ` [PATCH v3 08/22] erofs: add mode checking helper Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-09 6:00 ` [PATCH v3 09/22] erofs: register global fscache volume Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-09 6:00 ` [PATCH v3 10/22] erofs: add cookie context helper functions Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-09 6:00 ` [PATCH v3 11/22] erofs: add anonymous inode managing page cache of blob file Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-09 6:00 ` [PATCH v3 12/22] erofs: add erofs_fscache_read_page() helper Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-09 6:00 ` [PATCH v3 13/22] erofs: register cookie context for bootstrap blob Jeffle Xu
2022-02-09 6:00 ` Jeffle Xu
2022-02-09 6:01 ` [PATCH v3 14/22] erofs: implement fscache-based metadata read Jeffle Xu
2022-02-09 6:01 ` Jeffle Xu
2022-02-09 6:01 ` [PATCH v3 15/22] erofs: implement fscache-based data read for non-inline layout Jeffle Xu
2022-02-09 6:01 ` Jeffle Xu
2022-02-09 6:01 ` [PATCH v3 16/22] erofs: implement fscache-based data read for inline layout Jeffle Xu
2022-02-09 6:01 ` Jeffle Xu
2022-02-09 6:01 ` [PATCH v3 17/22] erofs: register cookie context for data blobs Jeffle Xu
2022-02-09 6:01 ` Jeffle Xu
2022-02-09 6:01 ` [PATCH v3 18/22] erofs: implement fscache-based data read " Jeffle Xu
2022-02-09 6:01 ` Jeffle Xu
2022-02-09 6:01 ` [PATCH v3 19/22] erofs: implement fscache-based data readahead for hole Jeffle Xu
2022-02-09 6:01 ` Jeffle Xu
2022-02-09 6:01 ` [PATCH v3 20/22] erofs: implement fscache-based data readahead for non-inline layout Jeffle Xu
2022-02-09 6:01 ` Jeffle Xu
2022-02-09 6:01 ` [PATCH v3 21/22] erofs: implement fscache-based data readahead for inline layout Jeffle Xu
2022-02-09 6:01 ` Jeffle Xu
2022-02-09 6:01 ` [PATCH v3 22/22] erofs: add 'uuid' mount option Jeffle Xu
2022-02-09 6:01 ` Jeffle Xu
2022-02-10 5:58 ` [Linux-cachefs] [PATCH v3 00/22] fscache, erofs: fscache-based demand-read semantics Gao Xiang
2022-02-10 5:58 ` Gao Xiang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220209060108.43051-3-jefflexu@linux.alibaba.com \
--to=jefflexu@linux.alibaba.com \
--cc=chao@kernel.org \
--cc=dhowells@redhat.com \
--cc=gerry@linux.alibaba.com \
--cc=gregkh@linuxfoundation.org \
--cc=joseph.qi@linux.alibaba.com \
--cc=linux-cachefs@redhat.com \
--cc=linux-erofs@lists.ozlabs.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=willy@infradead.org \
--cc=xiang@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.