* [PATCH 4.19.y 1/2] erofs: remove the occupied parameter from z_erofs_pagevec_enqueue() [not found] <1636983460149191@kroah.com> @ 2021-11-16 2:41 ` Gao Xiang 2021-11-16 2:41 ` [PATCH 4.19.y 2/2] erofs: fix unsafe pagevec reuse of hooked pclusters Gao Xiang 2021-11-19 14:34 ` Patch "erofs: remove the occupied parameter from z_erofs_pagevec_enqueue()" " gregkh 0 siblings, 2 replies; 4+ messages in thread From: Gao Xiang @ 2021-11-16 2:41 UTC (permalink / raw) To: Greg Kroah-Hartman, stable; +Cc: Gao Xiang, Yue Hu, linux-erofs From: Yue Hu <huyue2@yulong.com> commit 7dea3de7d384f4c8156e8bd93112ba6db1eb276c upstream. No any behavior to variable occupied in z_erofs_attach_page() which is only caller to z_erofs_pagevec_enqueue(). Link: https://lore.kernel.org/r/20210419102623.2015-1-zbestahu@gmail.com Signed-off-by: Yue Hu <huyue2@yulong.com> Reviewed-by: Gao Xiang <xiang@kernel.org> Signed-off-by: Gao Xiang <xiang@kernel.org> [ Gao Xiang: handle 4.19 codebase conflicts manually. ] Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> --- Gao Xiang: Same to 5.4.y and 5.10.y, apply this trivial cleanup as well. drivers/staging/erofs/unzip_pagevec.h | 5 +---- drivers/staging/erofs/unzip_vle.c | 4 +--- 2 files changed, 2 insertions(+), 7 deletions(-) diff --git a/drivers/staging/erofs/unzip_pagevec.h b/drivers/staging/erofs/unzip_pagevec.h index 23856ba2742d..64724dd1e04e 100644 --- a/drivers/staging/erofs/unzip_pagevec.h +++ b/drivers/staging/erofs/unzip_pagevec.h @@ -117,10 +117,8 @@ static inline void z_erofs_pagevec_ctor_init(struct z_erofs_pagevec_ctor *ctor, static inline bool z_erofs_pagevec_ctor_enqueue(struct z_erofs_pagevec_ctor *ctor, struct page *page, - enum z_erofs_page_type type, - bool *occupied) + enum z_erofs_page_type type) { - *occupied = false; if (unlikely(ctor->next == NULL && type)) if (ctor->index + 1 == ctor->nr) return false; @@ -135,7 +133,6 @@ z_erofs_pagevec_ctor_enqueue(struct z_erofs_pagevec_ctor *ctor, /* should remind that collector->next never equal to 1, 2 */ if (type == (uintptr_t)ctor->next) { ctor->next = page; - *occupied = true; } ctor->pages[ctor->index++] = diff --git a/drivers/staging/erofs/unzip_vle.c b/drivers/staging/erofs/unzip_vle.c index 0f1558c6747e..48c21a4d5dc8 100644 --- a/drivers/staging/erofs/unzip_vle.c +++ b/drivers/staging/erofs/unzip_vle.c @@ -234,7 +234,6 @@ static int z_erofs_vle_work_add_page( enum z_erofs_page_type type) { int ret; - bool occupied; /* give priority for the compressed data storage */ if (builder->role >= Z_EROFS_VLE_WORK_PRIMARY && @@ -242,8 +241,7 @@ static int z_erofs_vle_work_add_page( try_to_reuse_as_compressed_page(builder, page)) return 0; - ret = z_erofs_pagevec_ctor_enqueue(&builder->vector, - page, type, &occupied); + ret = z_erofs_pagevec_ctor_enqueue(&builder->vector, page, type); builder->work->vcnt += (unsigned)ret; return ret ? 0 : -EAGAIN; -- 2.24.4 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH 4.19.y 2/2] erofs: fix unsafe pagevec reuse of hooked pclusters 2021-11-16 2:41 ` [PATCH 4.19.y 1/2] erofs: remove the occupied parameter from z_erofs_pagevec_enqueue() Gao Xiang @ 2021-11-16 2:41 ` Gao Xiang 2021-11-19 14:34 ` Patch "erofs: fix unsafe pagevec reuse of hooked pclusters" has been added to the 4.19-stable tree gregkh 2021-11-19 14:34 ` Patch "erofs: remove the occupied parameter from z_erofs_pagevec_enqueue()" " gregkh 1 sibling, 1 reply; 4+ messages in thread From: Gao Xiang @ 2021-11-16 2:41 UTC (permalink / raw) To: Greg Kroah-Hartman, stable; +Cc: Gao Xiang, linux-erofs commit 86432a6dca9bed79111990851df5756d3eb5f57c upstream. There are pclusters in runtime marked with Z_EROFS_PCLUSTER_TAIL before actual I/O submission. Thus, the decompression chain can be extended if the following pcluster chain hooks such tail pcluster. As the related comment mentioned, if some page is made of a hooked pcluster and another followed pcluster, it can be reused for in-place I/O (since I/O should be submitted anyway): _______________________________________________________________ | tail (partial) page | head (partial) page | |_____PRIMARY_HOOKED___|____________PRIMARY_FOLLOWED____________| However, it's by no means safe to reuse as pagevec since if such PRIMARY_HOOKED pclusters finally move into bypass chain without I/O submission. It's somewhat hard to reproduce with LZ4 and I just found it (general protection fault) by ro_fsstressing a LZMA image for long time. I'm going to actively clean up related code together with multi-page folio adaption in the next few months. Let's address it directly for easier backporting for now. Call trace for reference: z_erofs_decompress_pcluster+0x10a/0x8a0 [erofs] z_erofs_decompress_queue.isra.36+0x3c/0x60 [erofs] z_erofs_runqueue+0x5f3/0x840 [erofs] z_erofs_readahead+0x1e8/0x320 [erofs] read_pages+0x91/0x270 page_cache_ra_unbounded+0x18b/0x240 filemap_get_pages+0x10a/0x5f0 filemap_read+0xa9/0x330 new_sync_read+0x11b/0x1a0 vfs_read+0xf1/0x190 Link: https://lore.kernel.org/r/20211103182006.4040-1-xiang@kernel.org Fixes: 3883a79abd02 ("staging: erofs: introduce VLE decompression support") Cc: <stable@vger.kernel.org> # 4.19+ Reviewed-by: Chao Yu <chao@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> --- drivers/staging/erofs/unzip_pagevec.h | 13 ++++++++++--- drivers/staging/erofs/unzip_vle.c | 17 +++++++++-------- 2 files changed, 19 insertions(+), 11 deletions(-) diff --git a/drivers/staging/erofs/unzip_pagevec.h b/drivers/staging/erofs/unzip_pagevec.h index 64724dd1e04e..efbf541e11bb 100644 --- a/drivers/staging/erofs/unzip_pagevec.h +++ b/drivers/staging/erofs/unzip_pagevec.h @@ -117,11 +117,18 @@ static inline void z_erofs_pagevec_ctor_init(struct z_erofs_pagevec_ctor *ctor, static inline bool z_erofs_pagevec_ctor_enqueue(struct z_erofs_pagevec_ctor *ctor, struct page *page, - enum z_erofs_page_type type) + enum z_erofs_page_type type, + bool pvec_safereuse) { - if (unlikely(ctor->next == NULL && type)) - if (ctor->index + 1 == ctor->nr) + if (!ctor->next) { + /* some pages cannot be reused as pvec safely without I/O */ + if (type == Z_EROFS_PAGE_TYPE_EXCLUSIVE && !pvec_safereuse) + type = Z_EROFS_VLE_PAGE_TYPE_TAIL_SHARED; + + if (type != Z_EROFS_PAGE_TYPE_EXCLUSIVE && + ctor->index + 1 == ctor->nr) return false; + } if (unlikely(ctor->index >= ctor->nr)) z_erofs_pagevec_ctor_pagedown(ctor, false); diff --git a/drivers/staging/erofs/unzip_vle.c b/drivers/staging/erofs/unzip_vle.c index 48c21a4d5dc8..83e4d9384bd2 100644 --- a/drivers/staging/erofs/unzip_vle.c +++ b/drivers/staging/erofs/unzip_vle.c @@ -228,10 +228,10 @@ static inline bool try_to_reuse_as_compressed_page( } /* callers must be with work->lock held */ -static int z_erofs_vle_work_add_page( - struct z_erofs_vle_work_builder *builder, - struct page *page, - enum z_erofs_page_type type) +static int z_erofs_vle_work_add_page(struct z_erofs_vle_work_builder *builder, + struct page *page, + enum z_erofs_page_type type, + bool pvec_safereuse) { int ret; @@ -241,9 +241,9 @@ static int z_erofs_vle_work_add_page( try_to_reuse_as_compressed_page(builder, page)) return 0; - ret = z_erofs_pagevec_ctor_enqueue(&builder->vector, page, type); + ret = z_erofs_pagevec_ctor_enqueue(&builder->vector, page, type, + pvec_safereuse); builder->work->vcnt += (unsigned)ret; - return ret ? 0 : -EAGAIN; } @@ -688,14 +688,15 @@ static int z_erofs_do_read_page(struct z_erofs_vle_frontend *fe, tight &= builder_is_followed(builder); retry: - err = z_erofs_vle_work_add_page(builder, page, page_type); + err = z_erofs_vle_work_add_page(builder, page, page_type, + builder_is_followed(builder)); /* should allocate an additional staging page for pagevec */ if (err == -EAGAIN) { struct page *const newpage = __stagingpage_alloc(page_pool, GFP_NOFS); err = z_erofs_vle_work_add_page(builder, - newpage, Z_EROFS_PAGE_TYPE_EXCLUSIVE); + newpage, Z_EROFS_PAGE_TYPE_EXCLUSIVE, true); if (likely(!err)) goto retry; } -- 2.24.4 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Patch "erofs: fix unsafe pagevec reuse of hooked pclusters" has been added to the 4.19-stable tree 2021-11-16 2:41 ` [PATCH 4.19.y 2/2] erofs: fix unsafe pagevec reuse of hooked pclusters Gao Xiang @ 2021-11-19 14:34 ` gregkh 0 siblings, 0 replies; 4+ messages in thread From: gregkh @ 2021-11-19 14:34 UTC (permalink / raw) To: chao, gregkh, hsiangkao, linux-erofs; +Cc: stable-commits This is a note to let you know that I've just added the patch titled erofs: fix unsafe pagevec reuse of hooked pclusters to the 4.19-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary The filename of the patch is: erofs-fix-unsafe-pagevec-reuse-of-hooked-pclusters.patch and it can be found in the queue-4.19 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable@vger.kernel.org> know about it. From foo@baz Fri Nov 19 03:26:51 PM CET 2021 From: Gao Xiang <hsiangkao@linux.alibaba.com> Date: Tue, 16 Nov 2021 10:41:53 +0800 Subject: erofs: fix unsafe pagevec reuse of hooked pclusters To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, stable@vger.kernel.org Cc: linux-erofs@lists.ozlabs.org, Gao Xiang <hsiangkao@linux.alibaba.com>, Chao Yu <chao@kernel.org> Message-ID: <20211116024153.245131-2-hsiangkao@linux.alibaba.com> From: Gao Xiang <hsiangkao@linux.alibaba.com> commit 86432a6dca9bed79111990851df5756d3eb5f57c upstream. There are pclusters in runtime marked with Z_EROFS_PCLUSTER_TAIL before actual I/O submission. Thus, the decompression chain can be extended if the following pcluster chain hooks such tail pcluster. As the related comment mentioned, if some page is made of a hooked pcluster and another followed pcluster, it can be reused for in-place I/O (since I/O should be submitted anyway): _______________________________________________________________ | tail (partial) page | head (partial) page | |_____PRIMARY_HOOKED___|____________PRIMARY_FOLLOWED____________| However, it's by no means safe to reuse as pagevec since if such PRIMARY_HOOKED pclusters finally move into bypass chain without I/O submission. It's somewhat hard to reproduce with LZ4 and I just found it (general protection fault) by ro_fsstressing a LZMA image for long time. I'm going to actively clean up related code together with multi-page folio adaption in the next few months. Let's address it directly for easier backporting for now. Call trace for reference: z_erofs_decompress_pcluster+0x10a/0x8a0 [erofs] z_erofs_decompress_queue.isra.36+0x3c/0x60 [erofs] z_erofs_runqueue+0x5f3/0x840 [erofs] z_erofs_readahead+0x1e8/0x320 [erofs] read_pages+0x91/0x270 page_cache_ra_unbounded+0x18b/0x240 filemap_get_pages+0x10a/0x5f0 filemap_read+0xa9/0x330 new_sync_read+0x11b/0x1a0 vfs_read+0xf1/0x190 Link: https://lore.kernel.org/r/20211103182006.4040-1-xiang@kernel.org Fixes: 3883a79abd02 ("staging: erofs: introduce VLE decompression support") Cc: <stable@vger.kernel.org> # 4.19+ Reviewed-by: Chao Yu <chao@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> --- drivers/staging/erofs/unzip_pagevec.h | 13 ++++++++++--- drivers/staging/erofs/unzip_vle.c | 17 +++++++++-------- 2 files changed, 19 insertions(+), 11 deletions(-) --- a/drivers/staging/erofs/unzip_pagevec.h +++ b/drivers/staging/erofs/unzip_pagevec.h @@ -117,11 +117,18 @@ static inline void z_erofs_pagevec_ctor_ static inline bool z_erofs_pagevec_ctor_enqueue(struct z_erofs_pagevec_ctor *ctor, struct page *page, - enum z_erofs_page_type type) + enum z_erofs_page_type type, + bool pvec_safereuse) { - if (unlikely(ctor->next == NULL && type)) - if (ctor->index + 1 == ctor->nr) + if (!ctor->next) { + /* some pages cannot be reused as pvec safely without I/O */ + if (type == Z_EROFS_PAGE_TYPE_EXCLUSIVE && !pvec_safereuse) + type = Z_EROFS_VLE_PAGE_TYPE_TAIL_SHARED; + + if (type != Z_EROFS_PAGE_TYPE_EXCLUSIVE && + ctor->index + 1 == ctor->nr) return false; + } if (unlikely(ctor->index >= ctor->nr)) z_erofs_pagevec_ctor_pagedown(ctor, false); --- a/drivers/staging/erofs/unzip_vle.c +++ b/drivers/staging/erofs/unzip_vle.c @@ -228,10 +228,10 @@ static inline bool try_to_reuse_as_compr } /* callers must be with work->lock held */ -static int z_erofs_vle_work_add_page( - struct z_erofs_vle_work_builder *builder, - struct page *page, - enum z_erofs_page_type type) +static int z_erofs_vle_work_add_page(struct z_erofs_vle_work_builder *builder, + struct page *page, + enum z_erofs_page_type type, + bool pvec_safereuse) { int ret; @@ -241,9 +241,9 @@ static int z_erofs_vle_work_add_page( try_to_reuse_as_compressed_page(builder, page)) return 0; - ret = z_erofs_pagevec_ctor_enqueue(&builder->vector, page, type); + ret = z_erofs_pagevec_ctor_enqueue(&builder->vector, page, type, + pvec_safereuse); builder->work->vcnt += (unsigned)ret; - return ret ? 0 : -EAGAIN; } @@ -688,14 +688,15 @@ hitted: tight &= builder_is_followed(builder); retry: - err = z_erofs_vle_work_add_page(builder, page, page_type); + err = z_erofs_vle_work_add_page(builder, page, page_type, + builder_is_followed(builder)); /* should allocate an additional staging page for pagevec */ if (err == -EAGAIN) { struct page *const newpage = __stagingpage_alloc(page_pool, GFP_NOFS); err = z_erofs_vle_work_add_page(builder, - newpage, Z_EROFS_PAGE_TYPE_EXCLUSIVE); + newpage, Z_EROFS_PAGE_TYPE_EXCLUSIVE, true); if (likely(!err)) goto retry; } Patches currently in stable-queue which might be from hsiangkao@linux.alibaba.com are queue-4.19/lib-xz-validate-the-value-before-assigning-it-to-an-.patch queue-4.19/erofs-fix-unsafe-pagevec-reuse-of-hooked-pclusters.patch queue-4.19/lib-xz-avoid-overlapping-memcpy-with-invalid-input-w.patch queue-4.19/erofs-remove-the-occupied-parameter-from-z_erofs_pagevec_enqueue.patch ^ permalink raw reply [flat|nested] 4+ messages in thread
* Patch "erofs: remove the occupied parameter from z_erofs_pagevec_enqueue()" has been added to the 4.19-stable tree 2021-11-16 2:41 ` [PATCH 4.19.y 1/2] erofs: remove the occupied parameter from z_erofs_pagevec_enqueue() Gao Xiang 2021-11-16 2:41 ` [PATCH 4.19.y 2/2] erofs: fix unsafe pagevec reuse of hooked pclusters Gao Xiang @ 2021-11-19 14:34 ` gregkh 1 sibling, 0 replies; 4+ messages in thread From: gregkh @ 2021-11-19 14:34 UTC (permalink / raw) To: gregkh, hsiangkao, huyue2, linux-erofs, xiang; +Cc: stable-commits This is a note to let you know that I've just added the patch titled erofs: remove the occupied parameter from z_erofs_pagevec_enqueue() to the 4.19-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary The filename of the patch is: erofs-remove-the-occupied-parameter-from-z_erofs_pagevec_enqueue.patch and it can be found in the queue-4.19 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable@vger.kernel.org> know about it. From foo@baz Fri Nov 19 03:26:51 PM CET 2021 From: Gao Xiang <hsiangkao@linux.alibaba.com> Date: Tue, 16 Nov 2021 10:41:52 +0800 Subject: erofs: remove the occupied parameter from z_erofs_pagevec_enqueue() To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, stable@vger.kernel.org Cc: linux-erofs@lists.ozlabs.org, Yue Hu <huyue2@yulong.com>, Gao Xiang <xiang@kernel.org>, Gao Xiang <hsiangkao@linux.alibaba.com> Message-ID: <20211116024153.245131-1-hsiangkao@linux.alibaba.com> From: Yue Hu <huyue2@yulong.com> commit 7dea3de7d384f4c8156e8bd93112ba6db1eb276c upstream. No any behavior to variable occupied in z_erofs_attach_page() which is only caller to z_erofs_pagevec_enqueue(). Link: https://lore.kernel.org/r/20210419102623.2015-1-zbestahu@gmail.com Signed-off-by: Yue Hu <huyue2@yulong.com> Reviewed-by: Gao Xiang <xiang@kernel.org> Signed-off-by: Gao Xiang <xiang@kernel.org> [ Gao Xiang: handle 4.19 codebase conflicts manually. ] Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> --- drivers/staging/erofs/unzip_pagevec.h | 5 +---- drivers/staging/erofs/unzip_vle.c | 4 +--- 2 files changed, 2 insertions(+), 7 deletions(-) --- a/drivers/staging/erofs/unzip_pagevec.h +++ b/drivers/staging/erofs/unzip_pagevec.h @@ -117,10 +117,8 @@ static inline void z_erofs_pagevec_ctor_ static inline bool z_erofs_pagevec_ctor_enqueue(struct z_erofs_pagevec_ctor *ctor, struct page *page, - enum z_erofs_page_type type, - bool *occupied) + enum z_erofs_page_type type) { - *occupied = false; if (unlikely(ctor->next == NULL && type)) if (ctor->index + 1 == ctor->nr) return false; @@ -135,7 +133,6 @@ z_erofs_pagevec_ctor_enqueue(struct z_er /* should remind that collector->next never equal to 1, 2 */ if (type == (uintptr_t)ctor->next) { ctor->next = page; - *occupied = true; } ctor->pages[ctor->index++] = --- a/drivers/staging/erofs/unzip_vle.c +++ b/drivers/staging/erofs/unzip_vle.c @@ -234,7 +234,6 @@ static int z_erofs_vle_work_add_page( enum z_erofs_page_type type) { int ret; - bool occupied; /* give priority for the compressed data storage */ if (builder->role >= Z_EROFS_VLE_WORK_PRIMARY && @@ -242,8 +241,7 @@ static int z_erofs_vle_work_add_page( try_to_reuse_as_compressed_page(builder, page)) return 0; - ret = z_erofs_pagevec_ctor_enqueue(&builder->vector, - page, type, &occupied); + ret = z_erofs_pagevec_ctor_enqueue(&builder->vector, page, type); builder->work->vcnt += (unsigned)ret; return ret ? 0 : -EAGAIN; Patches currently in stable-queue which might be from hsiangkao@linux.alibaba.com are queue-4.19/lib-xz-validate-the-value-before-assigning-it-to-an-.patch queue-4.19/erofs-fix-unsafe-pagevec-reuse-of-hooked-pclusters.patch queue-4.19/lib-xz-avoid-overlapping-memcpy-with-invalid-input-w.patch queue-4.19/erofs-remove-the-occupied-parameter-from-z_erofs_pagevec_enqueue.patch ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-11-19 14:34 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <1636983460149191@kroah.com>
2021-11-16 2:41 ` [PATCH 4.19.y 1/2] erofs: remove the occupied parameter from z_erofs_pagevec_enqueue() Gao Xiang
2021-11-16 2:41 ` [PATCH 4.19.y 2/2] erofs: fix unsafe pagevec reuse of hooked pclusters Gao Xiang
2021-11-19 14:34 ` Patch "erofs: fix unsafe pagevec reuse of hooked pclusters" has been added to the 4.19-stable tree gregkh
2021-11-19 14:34 ` Patch "erofs: remove the occupied parameter from z_erofs_pagevec_enqueue()" " gregkh
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox