Re: I/O blocked after booting

["Massimo B." <massimo.b@gmx.net>]

On Fri, 2024-03-29 at 06:53 +1030, Qu Wenruo wrote:
> I mean you should not do any fstrim/discard to see if everything works
> fine first.
> 
> This is to make sure the problem is really from the trim/discard part.

I'm still facing this issue, sometimes after boot and sometimes also after
longer uptimes...

Meanwhile I switched from mount option discard=async to nodiscard.
I also disabled my weekly cronjob doing fstrim -av.

btrfsmaintenance currently looks like this:

# grep PERIOD /etc/default/btrfsmaintenance 
BTRFS_DEFRAG_PERIOD="none"
BTRFS_BALANCE_PERIOD="none"
BTRFS_SCRUB_PERIOD="monthly"
BTRFS_TRIM_PERIOD="none"

So there should be no trim or discard happen anymore in the background.
I'm going to also disable the SCRUB now, though I doubt it's related to that.

But I still run into blocked IO sometimes. It's weird, some commands and apps I
still can run, some others are blocked, maybe due to cached IO. I'm able to
continue doing builds and system updates by the package manager (Gentoo, emerge)
in the background but at the same time it fails to start or stop some
applications. I can't even shutdown the window manager or the system properly.
All data, root and home are on the same btrfs on luks on NVMe or SSD, just
different subvolumes.

In that situation (kernel 6.6.13, rebooting into 6.6.30) there is nothing to see
in dmesg. Looking at the syslog, the last line always is from last night
midnight before the hard reboot via SysRq is done:

Jun 14 00:00:00 [fcron] Job 'run-parts /etc/cron.daily' started for user systab (pid 29389)
Jun 14 07:51:49 [kernel] Linux version 6.6.30-gentoo (root@gentoo-m) (gcc (Gentoo 13.2.1_p20240210 p14) 13.2.1 20240210, GNU ld (Gentoo 2.42 p3) 2.42.0) #1 SMP PREEMPT_DYNAMIC Fri Jun 14 07:43:15 CEST 2024

In cron.daily there is nothing special:
# ls /etc/cron.daily/
1metalog_flush*  disabled/  logrotate*  makewhatis*  man-db*  mlocate*  rkhunter*  systemd-tmpfiles-clean*

1metalog_flush only does...
kill -USR1 $metalog_pid	# Set metalog to sync...
kill -USR2 $metalog_pid	# Set metalog to async...
... in order to flush the metalog cached logs.

Again after the shutdown doesn't do anything I start using the SysRq series R E
I S U B and again after the E (SIGTERM to all processes) I see some errors about
trim popping on the virtual console followed by a call trace:


BTRFS warning (device dm-0): failed to trim 361 block group(s), last error -512
BTRFS warning (device dm-0): failed to trim 1 block device(s), last error -512
----------------------------[ cut here ]----------------------------------
WARNING: CPU: 0:PID: 4783 at 0xffffffffffffffa025fed3
Modules linked in: af_alg md5 ....
Hardware name:...
RIP:...
Code:...
...
Call Trace:
<TASK>
...
</TASK>
---[ end trace 000000000000 ]---
acpid: exiting
BTRFS info (device dm-2): last umount of filesystem 1d5.....
INIT: Switching to runlevel: 6
...


Could it be that something else could still trigger a trim in the background?
If trim is really the cause of failure, then I wonder that I have that issue on
all my machines with different hardwares, NVMe but also SATA SSDs. Just the
Gentoo Linux installation is almost identical.

Best regards,
Massimo