All of lore.kernel.org
 help / color / mirror / Atom feed
From: JeffleXu <jefflexu@linux.alibaba.com>
To: dhowells@redhat.com, linux-cachefs@redhat.com, xiang@kernel.org,
	chao@kernel.org, linux-erofs@lists.ozlabs.org
Cc: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org,
	willy@infradead.org, joseph.qi@linux.alibaba.com,
	linux-fsdevel@vger.kernel.org, gerry@linux.alibaba.com,
	torvalds@linux-foundation.org
Subject: Re: [PATCH v3 05/22] cachefiles: introduce new devnode for on-demand read mode
Date: Tue, 15 Feb 2022 17:03:16 +0800	[thread overview]
Message-ID: <bd9cb3bb-e29c-d4b3-e9bf-915b9771b553@linux.alibaba.com> (raw)
In-Reply-To: <20220209060108.43051-6-jefflexu@linux.alibaba.com>

Hi David,

FYI I've updated this patch on [1].

[1]
https://github.com/lostjeffle/linux/commit/589dd838dc539aee291d1032406653a8f6269e6f.

This new version mainly adds cachefiles_ondemand_flush_reqs(), which
drains the pending read requests when cachefilesd is going to exit.

On 2/9/22 2:00 PM, Jeffle Xu wrote:
> This patch introduces a new devnode 'cachefiles_ondemand' to support the
> newly introduced on-demand read mode.
> 
> The precondition for on-demand reading semantics is that, all blob files
> have been placed under corresponding directory with correct file size
> (sparse files) on the first beginning. When upper fs starts to access
> the blob file, it will "cache miss" (hit the hole) and then turn to user
> daemon for preparing the data.
> 
> The interaction between kernel and user daemon is described as below.
> 1. Once cache miss, .ondemand_read() callback of corresponding fscache
>    backend is called to prepare the data. As for cachefiles, it just
>    packages related metadata (file range to read, etc.) into a pending
>    read request, and then the process triggering cache miss will fall
>    asleep until the corresponding data gets fetched later.
> 2. User daemon needs to poll on the devnode ('cachefiles_ondemand'),
>    waiting for pending read request.
> 3. Once there's pending read request, user daemon will be notified and
>    shall read the devnode ('cachefiles_ondemand') to fetch one pending
>    read request to process.
> 4. For the fetched read request, user daemon need to somehow prepare the
>    data (e.g. download from remote through network) and then write the
>    fetched data into the backing file to fill the hole.
> 5. After that, user daemon need to notify cachefiles backend by writing a
>    'done' command to devnode ('cachefiles_ondemand'). It will also
>    awake the previous asleep process triggering cache miss.
> 6. By the time the process gets awaken, the data has been ready in the
>    backing file. Then process can re-initiate a read request from the
>    backing file.
> 
> Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com>
> ---


-- 
Thanks,
Jeffle

WARNING: multiple messages have this Message-ID (diff)
From: JeffleXu <jefflexu@linux.alibaba.com>
To: dhowells@redhat.com, linux-cachefs@redhat.com, xiang@kernel.org,
	chao@kernel.org, linux-erofs@lists.ozlabs.org
Cc: gregkh@linuxfoundation.org, willy@infradead.org,
	linux-kernel@vger.kernel.org, joseph.qi@linux.alibaba.com,
	linux-fsdevel@vger.kernel.org, gerry@linux.alibaba.com,
	torvalds@linux-foundation.org
Subject: Re: [PATCH v3 05/22] cachefiles: introduce new devnode for on-demand read mode
Date: Tue, 15 Feb 2022 17:03:16 +0800	[thread overview]
Message-ID: <bd9cb3bb-e29c-d4b3-e9bf-915b9771b553@linux.alibaba.com> (raw)
In-Reply-To: <20220209060108.43051-6-jefflexu@linux.alibaba.com>

Hi David,

FYI I've updated this patch on [1].

[1]
https://github.com/lostjeffle/linux/commit/589dd838dc539aee291d1032406653a8f6269e6f.

This new version mainly adds cachefiles_ondemand_flush_reqs(), which
drains the pending read requests when cachefilesd is going to exit.

On 2/9/22 2:00 PM, Jeffle Xu wrote:
> This patch introduces a new devnode 'cachefiles_ondemand' to support the
> newly introduced on-demand read mode.
> 
> The precondition for on-demand reading semantics is that, all blob files
> have been placed under corresponding directory with correct file size
> (sparse files) on the first beginning. When upper fs starts to access
> the blob file, it will "cache miss" (hit the hole) and then turn to user
> daemon for preparing the data.
> 
> The interaction between kernel and user daemon is described as below.
> 1. Once cache miss, .ondemand_read() callback of corresponding fscache
>    backend is called to prepare the data. As for cachefiles, it just
>    packages related metadata (file range to read, etc.) into a pending
>    read request, and then the process triggering cache miss will fall
>    asleep until the corresponding data gets fetched later.
> 2. User daemon needs to poll on the devnode ('cachefiles_ondemand'),
>    waiting for pending read request.
> 3. Once there's pending read request, user daemon will be notified and
>    shall read the devnode ('cachefiles_ondemand') to fetch one pending
>    read request to process.
> 4. For the fetched read request, user daemon need to somehow prepare the
>    data (e.g. download from remote through network) and then write the
>    fetched data into the backing file to fill the hole.
> 5. After that, user daemon need to notify cachefiles backend by writing a
>    'done' command to devnode ('cachefiles_ondemand'). It will also
>    awake the previous asleep process triggering cache miss.
> 6. By the time the process gets awaken, the data has been ready in the
>    backing file. Then process can re-initiate a read request from the
>    backing file.
> 
> Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com>
> ---


-- 
Thanks,
Jeffle

  reply	other threads:[~2022-02-15  9:03 UTC|newest]

Thread overview: 70+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-02-09  6:00 [PATCH v3 00/22] fscache,erofs: fscache-based demand-read semantics Jeffle Xu
2022-02-09  6:00 ` Jeffle Xu
2022-02-09  6:00 ` [PATCH v3 01/22] fscache: export fscache_end_operation() Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-17  7:44   ` Liu Bo
2022-02-17  7:44     ` Liu Bo
2022-02-09  6:00 ` [PATCH v3 02/22] fscache: add a method to support on-demand read semantics Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-09  6:00 ` [PATCH v3 03/22] cachefiles: extract generic function for daemon methods Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-17  8:17   ` Liu Bo
2022-02-17  8:17     ` Liu Bo
2022-02-09  6:00 ` [PATCH v3 04/22] cachefiles: detect backing file size in on-demand read mode Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-09  6:00 ` [PATCH v3 05/22] cachefiles: introduce new devnode for " Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-15  9:03   ` JeffleXu [this message]
2022-02-15  9:03     ` JeffleXu
2022-02-15 10:37     ` Greg KH
2022-02-15 10:37       ` Greg KH
2022-02-16  8:17       ` JeffleXu
2022-02-16  8:17         ` JeffleXu
2022-02-15 11:13     ` [PATCH v4 05/23] " Jeffle Xu
2022-02-15 11:13       ` Jeffle Xu
2022-02-16 10:48       ` Greg KH
2022-02-16 10:48         ` Greg KH
2022-02-16 12:49         ` JeffleXu
2022-02-16 12:49           ` JeffleXu
2022-02-16 17:48           ` Greg KH
2022-02-16 17:48             ` Greg KH
2022-02-17  1:49             ` JeffleXu
2022-02-17  1:49               ` JeffleXu
2022-02-09  6:00 ` [PATCH v3 06/22] erofs: use meta buffers for erofs_read_superblock() Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-09  7:52   ` Gao Xiang
2022-02-09  7:52     ` Gao Xiang
2022-02-09  6:00 ` [PATCH v3 07/22] erofs: export erofs_map_blocks() Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-09  6:00 ` [PATCH v3 08/22] erofs: add mode checking helper Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-09  6:00 ` [PATCH v3 09/22] erofs: register global fscache volume Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-09  6:00 ` [PATCH v3 10/22] erofs: add cookie context helper functions Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-09  6:00 ` [PATCH v3 11/22] erofs: add anonymous inode managing page cache of blob file Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-09  6:00 ` [PATCH v3 12/22] erofs: add erofs_fscache_read_page() helper Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-09  6:00 ` [PATCH v3 13/22] erofs: register cookie context for bootstrap blob Jeffle Xu
2022-02-09  6:00   ` Jeffle Xu
2022-02-09  6:01 ` [PATCH v3 14/22] erofs: implement fscache-based metadata read Jeffle Xu
2022-02-09  6:01   ` Jeffle Xu
2022-02-09  6:01 ` [PATCH v3 15/22] erofs: implement fscache-based data read for non-inline layout Jeffle Xu
2022-02-09  6:01   ` Jeffle Xu
2022-02-09  6:01 ` [PATCH v3 16/22] erofs: implement fscache-based data read for inline layout Jeffle Xu
2022-02-09  6:01   ` Jeffle Xu
2022-02-09  6:01 ` [PATCH v3 17/22] erofs: register cookie context for data blobs Jeffle Xu
2022-02-09  6:01   ` Jeffle Xu
2022-02-09  6:01 ` [PATCH v3 18/22] erofs: implement fscache-based data read " Jeffle Xu
2022-02-09  6:01   ` Jeffle Xu
2022-02-09  6:01 ` [PATCH v3 19/22] erofs: implement fscache-based data readahead for hole Jeffle Xu
2022-02-09  6:01   ` Jeffle Xu
2022-02-09  6:01 ` [PATCH v3 20/22] erofs: implement fscache-based data readahead for non-inline layout Jeffle Xu
2022-02-09  6:01   ` Jeffle Xu
2022-02-09  6:01 ` [PATCH v3 21/22] erofs: implement fscache-based data readahead for inline layout Jeffle Xu
2022-02-09  6:01   ` Jeffle Xu
2022-02-09  6:01 ` [PATCH v3 22/22] erofs: add 'uuid' mount option Jeffle Xu
2022-02-09  6:01   ` Jeffle Xu
2022-02-10  5:58 ` [Linux-cachefs] [PATCH v3 00/22] fscache, erofs: fscache-based demand-read semantics Gao Xiang
2022-02-10  5:58   ` Gao Xiang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=bd9cb3bb-e29c-d4b3-e9bf-915b9771b553@linux.alibaba.com \
    --to=jefflexu@linux.alibaba.com \
    --cc=chao@kernel.org \
    --cc=dhowells@redhat.com \
    --cc=gerry@linux.alibaba.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=joseph.qi@linux.alibaba.com \
    --cc=linux-cachefs@redhat.com \
    --cc=linux-erofs@lists.ozlabs.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=willy@infradead.org \
    --cc=xiang@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.