[regression 6.1.y] f2fs: invalid zstd compress level: 6

[regression 6.1.y] f2fs: invalid zstd compress level: 6

[Salvatore Bonaccorso]

Hi Jaegeuk Kim, Chao Yu,

In Debian the following regression was reported after a Dhya updated
to 6.1.76:

On Wed, Feb 07, 2024 at 10:43:47PM -0500, Dhya wrote:
> Package: src:linux
> Version: 6.1.76-1
> Severity: critical
> Justification: breaks the whole system
> 
> Dear Maintainer,
> 
> After upgrade to linux-image-6.1.0-18-amd64 6.1.76-1 F2FS filesystem
> fails to mount rw.  Message in the boot journal:
> 
>   kernel: F2FS-fs (nvme0n1p6): invalid zstd compress level: 6
> 
> There was recently an f2fs patch to the 6.1 kernel tree which might be
> related: https://www.spinics.net/lists/stable-commits/msg329957.html
> 
> Was able to recover the system by doing:
> 
> sudo mount -o remount,rw,relatime,lazytime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,compress_algorithm=lz4,compress_log_size=2,compress_mode=fs,atgc,discard_unit=block,memory=normal /dev/nvme0n1p6 /
> 
> under the running bad 6.1.0-18-amd64 kernel, then editing
> /etc/default/grub:
> 
>   GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-17-amd64"
> 
> and running 'update-grub' and rebooting to boot the 6.1.0-17-amd64
> kernel.

The issue is easily reproducible by:

# dd if=/dev/zero of=test.img count=100 bs=1M
# mkfs.f2fs -f -O compression,extra_attr ./test.img
# mount -t f2fs -o compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime ./test.img /mnt

resulting in

[   60.789982] F2FS-fs (loop0): invalid zstd compress level: 6

A bugzilla report has been submitted in
https://bugzilla.kernel.org/show_bug.cgi?id=218471

#regzbot introduced: v6.1.69..v6.1.76
#regzbot link: https://bugs.debian.org/1063422
#regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=218471

Regards,
Salvatore

Re: [regression 6.1.y] f2fs: invalid zstd compress level: 6

[Jaegeuk Kim]

Hi,

Let me check this soon.

Thanks,

On 02/08, Salvatore Bonaccorso wrote:
> Hi Jaegeuk Kim, Chao Yu,
> 
> In Debian the following regression was reported after a Dhya updated
> to 6.1.76:
> 
> On Wed, Feb 07, 2024 at 10:43:47PM -0500, Dhya wrote:
> > Package: src:linux
> > Version: 6.1.76-1
> > Severity: critical
> > Justification: breaks the whole system
> > 
> > Dear Maintainer,
> > 
> > After upgrade to linux-image-6.1.0-18-amd64 6.1.76-1 F2FS filesystem
> > fails to mount rw.  Message in the boot journal:
> > 
> >   kernel: F2FS-fs (nvme0n1p6): invalid zstd compress level: 6
> > 
> > There was recently an f2fs patch to the 6.1 kernel tree which might be
> > related: https://www.spinics.net/lists/stable-commits/msg329957.html
> > 
> > Was able to recover the system by doing:
> > 
> > sudo mount -o remount,rw,relatime,lazytime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,compress_algorithm=lz4,compress_log_size=2,compress_mode=fs,atgc,discard_unit=block,memory=normal /dev/nvme0n1p6 /
> > 
> > under the running bad 6.1.0-18-amd64 kernel, then editing
> > /etc/default/grub:
> > 
> >   GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-17-amd64"
> > 
> > and running 'update-grub' and rebooting to boot the 6.1.0-17-amd64
> > kernel.
> 
> The issue is easily reproducible by:
> 
> # dd if=/dev/zero of=test.img count=100 bs=1M
> # mkfs.f2fs -f -O compression,extra_attr ./test.img
> # mount -t f2fs -o compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime ./test.img /mnt
> 
> resulting in
> 
> [   60.789982] F2FS-fs (loop0): invalid zstd compress level: 6
> 
> A bugzilla report has been submitted in
> https://bugzilla.kernel.org/show_bug.cgi?id=218471
> 
> #regzbot introduced: v6.1.69..v6.1.76
> #regzbot link: https://bugs.debian.org/1063422
> #regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=218471
> 
> Regards,
> Salvatore

Re: [regression 6.1.y] f2fs: invalid zstd compress level: 6

[Chao Yu]

On 2024/2/9 4:19, Salvatore Bonaccorso wrote:
> Hi Jaegeuk Kim, Chao Yu,
> 
> In Debian the following regression was reported after a Dhya updated
> to 6.1.76:
> 
> On Wed, Feb 07, 2024 at 10:43:47PM -0500, Dhya wrote:
>> Package: src:linux
>> Version: 6.1.76-1
>> Severity: critical
>> Justification: breaks the whole system
>>
>> Dear Maintainer,
>>
>> After upgrade to linux-image-6.1.0-18-amd64 6.1.76-1 F2FS filesystem
>> fails to mount rw.  Message in the boot journal:
>>
>>    kernel: F2FS-fs (nvme0n1p6): invalid zstd compress level: 6
>>
>> There was recently an f2fs patch to the 6.1 kernel tree which might be
>> related: https://www.spinics.net/lists/stable-commits/msg329957.html
>>
>> Was able to recover the system by doing:
>>
>> sudo mount -o remount,rw,relatime,lazytime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,compress_algorithm=lz4,compress_log_size=2,compress_mode=fs,atgc,discard_unit=block,memory=normal /dev/nvme0n1p6 /
>>
>> under the running bad 6.1.0-18-amd64 kernel, then editing
>> /etc/default/grub:
>>
>>    GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-17-amd64"
>>
>> and running 'update-grub' and rebooting to boot the 6.1.0-17-amd64
>> kernel.
> 
> The issue is easily reproducible by:
> 
> # dd if=/dev/zero of=test.img count=100 bs=1M
> # mkfs.f2fs -f -O compression,extra_attr ./test.img
> # mount -t f2fs -o compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime ./test.img /mnt
> 
> resulting in
> 
> [   60.789982] F2FS-fs (loop0): invalid zstd compress level: 6

Hi Salvatore,

Can you please try below fixes:

[PATCH 6.1] f2fs: add helper to check compression level
https://lore.kernel.org/linux-f2fs-devel/20240212160530.1017205-1-chao@kernel.org

[PATCH] f2fs: compress: fix to check zstd compress level correctly in mount option
https://lore.kernel.org/linux-f2fs-devel/20240212160818.1020903-1-chao@kernel.org

Thanks,

> 
> A bugzilla report has been submitted in
> https://bugzilla.kernel.org/show_bug.cgi?id=218471
> 
> #regzbot introduced: v6.1.69..v6.1.76
> #regzbot link: https://bugs.debian.org/1063422
> #regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=218471
> 
> Regards,
> Salvatore

Re: [regression 6.1.y] f2fs: invalid zstd compress level: 6

[Salvatore Bonaccorso]

Hi,

On Mon, Feb 19, 2024 at 10:35:13AM +0800, Chao Yu wrote:
> On 2024/2/9 4:19, Salvatore Bonaccorso wrote:
> > Hi Jaegeuk Kim, Chao Yu,
> > 
> > In Debian the following regression was reported after a Dhya updated
> > to 6.1.76:
> > 
> > On Wed, Feb 07, 2024 at 10:43:47PM -0500, Dhya wrote:
> > > Package: src:linux
> > > Version: 6.1.76-1
> > > Severity: critical
> > > Justification: breaks the whole system
> > > 
> > > Dear Maintainer,
> > > 
> > > After upgrade to linux-image-6.1.0-18-amd64 6.1.76-1 F2FS filesystem
> > > fails to mount rw.  Message in the boot journal:
> > > 
> > >    kernel: F2FS-fs (nvme0n1p6): invalid zstd compress level: 6
> > > 
> > > There was recently an f2fs patch to the 6.1 kernel tree which might be
> > > related: https://www.spinics.net/lists/stable-commits/msg329957.html
> > > 
> > > Was able to recover the system by doing:
> > > 
> > > sudo mount -o remount,rw,relatime,lazytime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,compress_algorithm=lz4,compress_log_size=2,compress_mode=fs,atgc,discard_unit=block,memory=normal /dev/nvme0n1p6 /
> > > 
> > > under the running bad 6.1.0-18-amd64 kernel, then editing
> > > /etc/default/grub:
> > > 
> > >    GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-17-amd64"
> > > 
> > > and running 'update-grub' and rebooting to boot the 6.1.0-17-amd64
> > > kernel.
> > 
> > The issue is easily reproducible by:
> > 
> > # dd if=/dev/zero of=test.img count=100 bs=1M
> > # mkfs.f2fs -f -O compression,extra_attr ./test.img
> > # mount -t f2fs -o compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime ./test.img /mnt
> > 
> > resulting in
> > 
> > [   60.789982] F2FS-fs (loop0): invalid zstd compress level: 6
> 
> Hi Salvatore,
> 
> Can you please try below fixes:
> 
> [PATCH 6.1] f2fs: add helper to check compression level
> https://lore.kernel.org/linux-f2fs-devel/20240212160530.1017205-1-chao@kernel.org

Confirmed that this fixes the reported issue as it was reported to us
in Debian in https://bugs.debian.org/1063422 . Thanks a lot!
(note just tested with the first commit as it landed in 6.1.78 to
confirm the immediate regression).

#regzbot fixed-by: cf3d57ad6ff8b566deba3544b9ad3384781fb604

Regards,
Salvatore

Re: [regression 6.1.y] f2fs: invalid zstd compress level: 6

[Chao Yu]

On 2024/2/20 3:54, Salvatore Bonaccorso wrote:
> Hi,
> 
> On Mon, Feb 19, 2024 at 10:35:13AM +0800, Chao Yu wrote:
>> On 2024/2/9 4:19, Salvatore Bonaccorso wrote:
>>> Hi Jaegeuk Kim, Chao Yu,
>>>
>>> In Debian the following regression was reported after a Dhya updated
>>> to 6.1.76:
>>>
>>> On Wed, Feb 07, 2024 at 10:43:47PM -0500, Dhya wrote:
>>>> Package: src:linux
>>>> Version: 6.1.76-1
>>>> Severity: critical
>>>> Justification: breaks the whole system
>>>>
>>>> Dear Maintainer,
>>>>
>>>> After upgrade to linux-image-6.1.0-18-amd64 6.1.76-1 F2FS filesystem
>>>> fails to mount rw.  Message in the boot journal:
>>>>
>>>>     kernel: F2FS-fs (nvme0n1p6): invalid zstd compress level: 6
>>>>
>>>> There was recently an f2fs patch to the 6.1 kernel tree which might be
>>>> related: https://www.spinics.net/lists/stable-commits/msg329957.html
>>>>
>>>> Was able to recover the system by doing:
>>>>
>>>> sudo mount -o remount,rw,relatime,lazytime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,compress_algorithm=lz4,compress_log_size=2,compress_mode=fs,atgc,discard_unit=block,memory=normal /dev/nvme0n1p6 /
>>>>
>>>> under the running bad 6.1.0-18-amd64 kernel, then editing
>>>> /etc/default/grub:
>>>>
>>>>     GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-17-amd64"
>>>>
>>>> and running 'update-grub' and rebooting to boot the 6.1.0-17-amd64
>>>> kernel.
>>>
>>> The issue is easily reproducible by:
>>>
>>> # dd if=/dev/zero of=test.img count=100 bs=1M
>>> # mkfs.f2fs -f -O compression,extra_attr ./test.img
>>> # mount -t f2fs -o compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime ./test.img /mnt
>>>
>>> resulting in
>>>
>>> [   60.789982] F2FS-fs (loop0): invalid zstd compress level: 6
>>
>> Hi Salvatore,
>>
>> Can you please try below fixes:
>>
>> [PATCH 6.1] f2fs: add helper to check compression level
>> https://lore.kernel.org/linux-f2fs-devel/20240212160530.1017205-1-chao@kernel.org
> 
> Confirmed that this fixes the reported issue as it was reported to us
> in Debian in https://bugs.debian.org/1063422 . Thanks a lot!
> (note just tested with the first commit as it landed in 6.1.78 to
> confirm the immediate regression).
> 
> #regzbot fixed-by: cf3d57ad6ff8b566deba3544b9ad3384781fb604

Hi,

Thank you for confirmation.

Thanks,

> 
> Regards,
> Salvatore