All of lore.kernel.org
 help / color / mirror / Atom feed
From: Gao Xiang <hsiangkao@linux.alibaba.com>
To: Juhyung Park <qkrwngud825@gmail.com>
Cc: Gao Xiang <xiang@kernel.org>,
	linux-erofs@lists.ozlabs.org,
	linux-f2fs-devel@lists.sourceforge.net,
	linux-crypto@vger.kernel.org,
	Yann Collet <yann.collet.73@gmail.com>
Subject: Re: Weird EROFS data corruption
Date: Mon, 4 Dec 2023 11:28:02 +0800	[thread overview]
Message-ID: <275f025d-e2f1-eaff-6af1-e909d370cee0@linux.alibaba.com> (raw)
In-Reply-To: <CAD14+f1u6gnHLhGSoQxL9wLq9vDYse+Ac8zxep-O2E8hHreT2w@mail.gmail.com>



On 2023/12/4 01:32, Juhyung Park wrote:
> Hi Gao,

...

>>>
>>>>
>>>> What is the difference between these two machines? just different CPU or
>>>> they have some other difference like different compliers?
>>>
>>> I fully and exclusively control both devices, and the setup is almost the same.
>>> Same Ubuntu version, kernel/compiler version.
>>>
>>> But as I said, on my laptop, the issue happens on kernels that someone
>>> else (Canonical) built, so I don't think it matters.
>>
>> The only thing I could say is that the kernel side has optimized
>> inplace decompression compared to fuse so that it will reuse the
>> same buffer for decompression but with a safe margin (according to
>> the current lz4 decompression implementation).  It shouldn't behave
>> different just due to different CPUs.  Let me find more clues
>> later, also maybe we should introduce a way for users to turn off
>> this if needed.
> 
> Cool :)
> 
> I'm comfortable changing and building my own custom kernel for this
> specific laptop. Feel free to ask me to try out some patches.

Thanks, I need to narrow down this issue:

-  First, could you apply the following diff to test if it's still
    reproducable?

diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
index 021be5feb1bc..40a306628e1a 100644
--- a/fs/erofs/decompressor.c
+++ b/fs/erofs/decompressor.c
@@ -131,7 +131,7 @@ static void *z_erofs_lz4_handle_overlap(struct z_erofs_lz4_decompress_ctx *ctx,

  	if (rq->inplace_io) {
  		omargin = PAGE_ALIGN(ctx->oend) - ctx->oend;
-		if (rq->partial_decoding || !may_inplace ||
+		if (1 || rq->partial_decoding || !may_inplace ||
  		    omargin < LZ4_DECOMPRESS_INPLACE_MARGIN(rq->inputsize))
  			goto docopy;

- Could you share the full message about the output of `lscpu`?

Thanks,
Gao Xiang

WARNING: multiple messages have this Message-ID (diff)
From: Gao Xiang <hsiangkao@linux.alibaba.com>
To: Juhyung Park <qkrwngud825@gmail.com>
Cc: Yann Collet <yann.collet.73@gmail.com>,
	linux-erofs@lists.ozlabs.org, linux-crypto@vger.kernel.org,
	linux-f2fs-devel@lists.sourceforge.net
Subject: Re: Weird EROFS data corruption
Date: Mon, 4 Dec 2023 11:28:02 +0800	[thread overview]
Message-ID: <275f025d-e2f1-eaff-6af1-e909d370cee0@linux.alibaba.com> (raw)
In-Reply-To: <CAD14+f1u6gnHLhGSoQxL9wLq9vDYse+Ac8zxep-O2E8hHreT2w@mail.gmail.com>



On 2023/12/4 01:32, Juhyung Park wrote:
> Hi Gao,

...

>>>
>>>>
>>>> What is the difference between these two machines? just different CPU or
>>>> they have some other difference like different compliers?
>>>
>>> I fully and exclusively control both devices, and the setup is almost the same.
>>> Same Ubuntu version, kernel/compiler version.
>>>
>>> But as I said, on my laptop, the issue happens on kernels that someone
>>> else (Canonical) built, so I don't think it matters.
>>
>> The only thing I could say is that the kernel side has optimized
>> inplace decompression compared to fuse so that it will reuse the
>> same buffer for decompression but with a safe margin (according to
>> the current lz4 decompression implementation).  It shouldn't behave
>> different just due to different CPUs.  Let me find more clues
>> later, also maybe we should introduce a way for users to turn off
>> this if needed.
> 
> Cool :)
> 
> I'm comfortable changing and building my own custom kernel for this
> specific laptop. Feel free to ask me to try out some patches.

Thanks, I need to narrow down this issue:

-  First, could you apply the following diff to test if it's still
    reproducable?

diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
index 021be5feb1bc..40a306628e1a 100644
--- a/fs/erofs/decompressor.c
+++ b/fs/erofs/decompressor.c
@@ -131,7 +131,7 @@ static void *z_erofs_lz4_handle_overlap(struct z_erofs_lz4_decompress_ctx *ctx,

  	if (rq->inplace_io) {
  		omargin = PAGE_ALIGN(ctx->oend) - ctx->oend;
-		if (rq->partial_decoding || !may_inplace ||
+		if (1 || rq->partial_decoding || !may_inplace ||
  		    omargin < LZ4_DECOMPRESS_INPLACE_MARGIN(rq->inputsize))
  			goto docopy;

- Could you share the full message about the output of `lscpu`?

Thanks,
Gao Xiang

WARNING: multiple messages have this Message-ID (diff)
From: Gao Xiang <hsiangkao@linux.alibaba.com>
To: Juhyung Park <qkrwngud825@gmail.com>
Cc: Yann Collet <yann.collet.73@gmail.com>,
	Gao Xiang <xiang@kernel.org>,
	linux-erofs@lists.ozlabs.org, linux-crypto@vger.kernel.org,
	linux-f2fs-devel@lists.sourceforge.net
Subject: Re: [f2fs-dev] Weird EROFS data corruption
Date: Mon, 4 Dec 2023 11:28:02 +0800	[thread overview]
Message-ID: <275f025d-e2f1-eaff-6af1-e909d370cee0@linux.alibaba.com> (raw)
In-Reply-To: <CAD14+f1u6gnHLhGSoQxL9wLq9vDYse+Ac8zxep-O2E8hHreT2w@mail.gmail.com>



On 2023/12/4 01:32, Juhyung Park wrote:
> Hi Gao,

...

>>>
>>>>
>>>> What is the difference between these two machines? just different CPU or
>>>> they have some other difference like different compliers?
>>>
>>> I fully and exclusively control both devices, and the setup is almost the same.
>>> Same Ubuntu version, kernel/compiler version.
>>>
>>> But as I said, on my laptop, the issue happens on kernels that someone
>>> else (Canonical) built, so I don't think it matters.
>>
>> The only thing I could say is that the kernel side has optimized
>> inplace decompression compared to fuse so that it will reuse the
>> same buffer for decompression but with a safe margin (according to
>> the current lz4 decompression implementation).  It shouldn't behave
>> different just due to different CPUs.  Let me find more clues
>> later, also maybe we should introduce a way for users to turn off
>> this if needed.
> 
> Cool :)
> 
> I'm comfortable changing and building my own custom kernel for this
> specific laptop. Feel free to ask me to try out some patches.

Thanks, I need to narrow down this issue:

-  First, could you apply the following diff to test if it's still
    reproducable?

diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
index 021be5feb1bc..40a306628e1a 100644
--- a/fs/erofs/decompressor.c
+++ b/fs/erofs/decompressor.c
@@ -131,7 +131,7 @@ static void *z_erofs_lz4_handle_overlap(struct z_erofs_lz4_decompress_ctx *ctx,

  	if (rq->inplace_io) {
  		omargin = PAGE_ALIGN(ctx->oend) - ctx->oend;
-		if (rq->partial_decoding || !may_inplace ||
+		if (1 || rq->partial_decoding || !may_inplace ||
  		    omargin < LZ4_DECOMPRESS_INPLACE_MARGIN(rq->inputsize))
  			goto docopy;

- Could you share the full message about the output of `lscpu`?

Thanks,
Gao Xiang


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

  reply	other threads:[~2023-12-04  3:28 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-12-03 16:22 Weird EROFS data corruption Juhyung Park
2023-12-03 16:22 ` [f2fs-dev] " Juhyung Park
2023-12-03 16:22 ` Juhyung Park
2023-12-03 16:52 ` Gao Xiang
2023-12-03 16:52   ` [f2fs-dev] " Gao Xiang
2023-12-03 16:52   ` Gao Xiang
2023-12-03 17:01   ` Juhyung Park
2023-12-03 17:01     ` [f2fs-dev] " Juhyung Park
2023-12-03 17:01     ` Juhyung Park
2023-12-03 17:21     ` Gao Xiang
2023-12-03 17:21       ` [f2fs-dev] " Gao Xiang
2023-12-03 17:21       ` Gao Xiang
2023-12-03 17:32       ` Juhyung Park
2023-12-03 17:32         ` [f2fs-dev] " Juhyung Park
2023-12-03 17:32         ` Juhyung Park
2023-12-04  3:28         ` Gao Xiang [this message]
2023-12-04  3:28           ` [f2fs-dev] " Gao Xiang
2023-12-04  3:28           ` Gao Xiang
2023-12-04  3:41           ` Juhyung Park
2023-12-04  3:41             ` [f2fs-dev] " Juhyung Park
2023-12-04  3:41             ` Juhyung Park
2023-12-05  7:32             ` Gao Xiang
2023-12-05  7:32               ` [f2fs-dev] " Gao Xiang
2023-12-05  7:32               ` Gao Xiang
2023-12-05 14:23               ` Juhyung Park
2023-12-05 14:23                 ` [f2fs-dev] " Juhyung Park
2023-12-05 14:23                 ` Juhyung Park
2023-12-05 14:34                 ` Gao Xiang
2023-12-05 14:34                   ` [f2fs-dev] " Gao Xiang
2023-12-05 14:34                   ` Gao Xiang
2023-12-05 14:43                   ` Juhyung Park
2023-12-05 14:43                     ` [f2fs-dev] " Juhyung Park
2023-12-05 14:43                     ` Juhyung Park
2023-12-06  3:11                     ` Gao Xiang
2023-12-06  3:11                       ` [f2fs-dev] " Gao Xiang
2023-12-06  3:11                       ` Gao Xiang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=275f025d-e2f1-eaff-6af1-e909d370cee0@linux.alibaba.com \
    --to=hsiangkao@linux.alibaba.com \
    --cc=linux-crypto@vger.kernel.org \
    --cc=linux-erofs@lists.ozlabs.org \
    --cc=linux-f2fs-devel@lists.sourceforge.net \
    --cc=qkrwngud825@gmail.com \
    --cc=xiang@kernel.org \
    --cc=yann.collet.73@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.