* BUG: kernel stack overflow when mounting with data_flush @ 2019-03-27 13:59 Hagbard Celine 2019-03-30 3:25 ` Chao Yu 0 siblings, 1 reply; 16+ messages in thread From: Hagbard Celine @ 2019-03-27 13:59 UTC (permalink / raw) To: linux-f2fs-devel Hi, this is a long standing bug that I've hit before on older kernels, but I was not able to get the syslog saved because of the nature of the bug. This time I had booted form a pen-drive, and was able to save the log to it's efi-partition. What i did to trigger it was to create a partition and format it f2fs, then mount it with options: "rw,relatime,lazytime,background_gc=on,disable_ext_identify,discard,heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,data_flush,extent_cache,mode=adaptive,active_logs=6,whint_mode=fs-based,alloc_mode=default,fsync_mode=strict". Then I unpacked a big .tar.xz to the partition (I used a gentoo-stage3-tarball as I was in process of installing Gentoo). Same options just without data_flush gives no problems. Syslog from crash follows: Mar 20 20:20:34 usbgentoo syslog-ng[3644]: syslog-ng starting up; version='3.17.2' Mar 20 20:20:34 usbgentoo /usr/sbin/gpm[3674]: *** info [daemon/startup.c(136)]: Mar 20 20:20:34 usbgentoo /usr/sbin/gpm[3674]: Started gpm successfully. Entered daemon mode. Mar 20 20:20:34 usbgentoo kernel: ip (3771) used greatest stack depth: 12312 bytes left Mar 20 20:20:34 usbgentoo dhcpcd[3840]: eth0: waiting for carrier Mar 20 20:20:37 usbgentoo kernel: igb 0000:03:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX Mar 20 20:20:37 usbgentoo kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Mar 20 20:20:37 usbgentoo dhcpcd[3840]: eth0: carrier acquired Mar 20 20:20:38 usbgentoo dhcpcd[3840]: DUID 00:01:00:01:24:24:e8:94:d0:50:99:3b:c9:21 Mar 20 20:20:38 usbgentoo dhcpcd[3840]: eth0: IAID 99:3b:c9:21 Mar 20 20:20:38 usbgentoo dhcpcd[3840]: eth0: adding address fe80::3c99:2bbf:63bb:8354 Mar 20 20:20:38 usbgentoo dhcpcd[3840]: eth0: rebinding lease of 192.168.1.22 Mar 20 20:20:38 usbgentoo dhcpcd[3840]: eth0: probing address 192.168.1.22/24 Mar 20 20:20:39 usbgentoo dhcpcd[3840]: eth0: soliciting an IPv6 router Mar 20 20:20:43 usbgentoo dhcpcd[3840]: eth0: leased 192.168.1.22 for 86400 seconds Mar 20 20:20:43 usbgentoo dhcpcd[3840]: eth0: adding route to 192.168.1.0/24 Mar 20 20:20:43 usbgentoo dhcpcd[3840]: eth0: adding default route via 192.168.1.1 Mar 20 20:20:43 usbgentoo dhcpcd[3840]: forked to background, child pid 3883 Mar 20 20:20:48 usbgentoo login[4003]: pam_unix(login:auth): authentication failure; logname=LOGIN uid=0 euid=0 tty=/dev/tty1 ruser= rhost= user=root Mar 20 20:20:51 usbgentoo dhcpcd[3883]: eth0: no IPv6 Routers available Mar 20 20:21:06 usbgentoo login[4003]: pam_unix(login:session): session opened for user root by LOGIN(uid=0) Mar 20 20:21:06 usbgentoo login[4012]: ROOT LOGIN on '/dev/tty1' Mar 20 20:22:30 usbgentoo kernel: EXT4-fs (sdb4): recovery complete Mar 20 20:22:30 usbgentoo kernel: EXT4-fs (sdb4): mounted filesystem with ordered data mode. Opts: (null) Mar 20 20:22:39 usbgentoo kernel: EXT4-fs (sdb5): recovery complete Mar 20 20:22:39 usbgentoo kernel: EXT4-fs (sdb5): mounted filesystem with ordered data mode. Opts: (null) Mar 20 20:22:46 usbgentoo login[4003]: pam_unix(login:session): session closed for user root Mar 20 20:22:51 usbgentoo login[4025]: pam_unix(login:session): session opened for user root by LOGIN(uid=0) Mar 20 20:22:51 usbgentoo login[4027]: ROOT LOGIN on '/dev/tty1' Mar 20 20:23:34 usbgentoo login[4004]: pam_unix(login:session): session opened for user root by LOGIN(uid=0) Mar 20 20:23:34 usbgentoo login[4034]: ROOT LOGIN on '/dev/tty2' Mar 20 20:30:21 usbgentoo kernel: F2FS-fs (nvme0n1p5): Found nat_bits in checkpoint Mar 20 20:30:22 usbgentoo kernel: F2FS-fs (nvme0n1p5): Mounted with checkpoint version = b9e8e7 Mar 20 20:30:35 usbgentoo login[4025]: pam_unix(login:session): session closed for user root Mar 20 20:30:42 usbgentoo login[4061]: pam_unix(login:session): session opened for user root by LOGIN(uid=0) Mar 20 20:30:42 usbgentoo login[4063]: ROOT LOGIN on '/dev/tty1' Mar 20 20:40:31 usbgentoo kernel: Adding 23984124k swap on /dev/nvme0n1p6. Priority:-2 extents:1 across:23984124k SSDsc Mar 20 20:54:01 usbgentoo kernel: FAT-fs (nvme0n1p4): Volume was not properly unmounted. Some data may be corrupt. Please run fsck. Mar 20 21:05:23 usbgentoo kernel: kworker/dying (1588) used greatest stack depth: 12064 bytes left Mar 20 21:06:40 usbgentoo kernel: BUG: stack guard page was hit at 00000000a4b0733c (stack is 0000000056016422..0000000096e7463f) Mar 20 21:06:40 usbgentoo kernel: kernel stack overflow (double-fault): 0000 [#1] SMP PTI Mar 20 21:06:40 usbgentoo kernel: CPU: 7 PID: 1606 Comm: kworker/u16:15 Not tainted 5.0.3-gentoo #6 Mar 20 21:06:40 usbgentoo kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./C226 WS, BIOS P3.40 06/25/2018 Mar 20 21:06:40 usbgentoo kernel: Workqueue: writeback wb_workfn (flush-259:0) Mar 20 21:06:40 usbgentoo kernel: RIP: 0010:f2fs_inode_chksum_verify+0x14/0xc0 Mar 20 21:06:40 usbgentoo kernel: Code: 00 00 04 0f 45 f0 e9 3b 85 e8 ff 90 66 2e 0f 1f 84 00 00 00 00 00 48 8b 47 48 a8 40 0f 85 9d 00 00 00 41 55 41 54 55 48 89 fd <53> 48 89 f3 e8 33 fd ff ff ba 01 00 00 00 84 c0 75 09 5b 89 d0 5d Mar 20 21:06:40 usbgentoo kernel: RSP: 0018:ffffb3ff4b0f4000 EFLAGS: 00010246 Mar 20 21:06:40 usbgentoo kernel: RAX: 0000000000000001 RBX: ffffd57b5afeaa00 RCX: 0000000000000000 Mar 20 21:06:40 usbgentoo kernel: RDX: 0000000000000000 RSI: ffffd57b5afeaa00 RDI: ffff9341f77f2000 Mar 20 21:06:40 usbgentoo kernel: RBP: ffff9341f77f2000 R08: ffffb3ff4b0f4110 R09: ffff9340bfaa9377 Mar 20 21:06:40 usbgentoo kernel: R10: ffffb3ff4b0f41a0 R11: 00000000fffffff3 R12: 000000000005c712 Mar 20 21:06:40 usbgentoo kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000000 Mar 20 21:06:40 usbgentoo kernel: FS: 0000000000000000(0000) GS:ffff9341fefc0000(0000) knlGS:0000000000000000 Mar 20 21:06:40 usbgentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Mar 20 21:06:40 usbgentoo kernel: CR2: ffffb3ff4b0f3ff8 CR3: 000000006320e006 CR4: 00000000003606e0 Mar 20 21:06:40 usbgentoo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Mar 20 21:06:40 usbgentoo kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Mar 20 21:06:40 usbgentoo kernel: Call Trace: Mar 20 21:06:40 usbgentoo kernel: read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: ? xas_load+0x8/0x50 Mar 20 21:06:40 usbgentoo kernel: __get_node_page+0x73/0x2a0 Mar 20 21:06:40 usbgentoo kernel: f2fs_get_dnode_of_data+0x34e/0x580 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_inline_data+0x5e/0x2a0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x421/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: ? find_get_entries_tag+0x91/0x260 Mar 20 21:06:40 usbgentoo kernel: ? get_partial_node.isra.86.part.87+0x2d/0x200 Mar 20 21:06:40 usbgentoo kernel: ? mempool_alloc+0x62/0x190 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: ? wait_woken+0x80/0x80 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: ? __switch_to_asm+0x40/0x70 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: ? raid6_avx5122_gen_syndrome+0x151/0x160 Mar 20 21:06:40 usbgentoo kernel: ? sbitmap_get+0x71/0x160 Mar 20 21:06:40 usbgentoo kernel: ? __sbitmap_queue_get+0x24/0x90 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 Mar 20 21:06:40 usbgentoo kernel: ? __update_load_avg_cfs_rq+0x1b1/0x220 Mar 20 21:06:40 usbgentoo kernel: ? enqueue_task_fair+0x441/0xfd0 Mar 20 21:06:40 usbgentoo kernel: ? guard_bio_eod+0x27/0xf0 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? try_to_wake_up+0x52/0x550 Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 Mar 20 21:06:40 usbgentoo kernel: ? nr_blockdev_pages+0xc/0x60 Mar 20 21:06:40 usbgentoo kernel: ? si_meminfo+0x3a/0x60 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_available_free_memory+0x29/0x240 Mar 20 21:06:40 usbgentoo kernel: ? complete+0x36/0x50 Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 Mar 20 21:06:40 usbgentoo kernel: ? __set_page_dirty_nobuffers+0xcb/0x100 Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_set+0xc/0x40 Mar 20 21:06:40 usbgentoo kernel: f2fs_write_inode+0x79/0x1c0 Mar 20 21:06:40 usbgentoo kernel: __writeback_single_inode+0x2a1/0x340 Mar 20 21:06:40 usbgentoo kernel: ? soft_cursor+0x1b4/0x220 Mar 20 21:06:40 usbgentoo kernel: writeback_sb_inodes+0x1d5/0x3e0 Mar 20 21:06:40 usbgentoo kernel: __writeback_inodes_wb+0x58/0xa0 Mar 20 21:06:40 usbgentoo kernel: wb_writeback+0x250/0x2e0 Mar 20 21:06:40 usbgentoo kernel: ? 0xffffffff8c000000 Mar 20 21:06:40 usbgentoo kernel: ? cpumask_next+0x16/0x20 Mar 20 21:06:40 usbgentoo kernel: wb_workfn+0x2f6/0x3b0 Mar 20 21:06:40 usbgentoo kernel: ? __switch_to_asm+0x40/0x70 Mar 20 21:06:40 usbgentoo kernel: process_one_work+0x1f5/0x3f0 Mar 20 21:06:40 usbgentoo kernel: worker_thread+0x28/0x3c0 Mar 20 21:06:40 usbgentoo kernel: ? rescuer_thread+0x330/0x330 Mar 20 21:06:40 usbgentoo kernel: kthread+0x10e/0x130 Mar 20 21:06:40 usbgentoo kernel: ? kthread_create_on_node+0x60/0x60 Mar 20 21:06:40 usbgentoo kernel: ret_from_fork+0x35/0x40 Mar 20 21:06:40 usbgentoo kernel: Modules linked in: iTCO_wdt efivarfs Mar 20 21:06:40 usbgentoo kernel: ---[ end trace 7d32682090ec7829 ]--- Mar 20 21:06:40 usbgentoo kernel: RIP: 0010:f2fs_inode_chksum_verify+0x14/0xc0 Mar 20 21:06:40 usbgentoo kernel: Code: 00 00 04 0f 45 f0 e9 3b 85 e8 ff 90 66 2e 0f 1f 84 00 00 00 00 00 48 8b 47 48 a8 40 0f 85 9d 00 00 00 41 55 41 54 55 48 89 fd <53> 48 89 f3 e8 33 fd ff ff ba 01 00 00 00 84 c0 75 09 5b 89 d0 5d Mar 20 21:06:40 usbgentoo kernel: RSP: 0018:ffffb3ff4b0f4000 EFLAGS: 00010246 Mar 20 21:06:40 usbgentoo kernel: RAX: 0000000000000001 RBX: ffffd57b5afeaa00 RCX: 0000000000000000 Mar 20 21:06:40 usbgentoo kernel: RDX: 0000000000000000 RSI: ffffd57b5afeaa00 RDI: ffff9341f77f2000 Mar 20 21:06:40 usbgentoo kernel: RBP: ffff9341f77f2000 R08: ffffb3ff4b0f4110 R09: ffff9340bfaa9377 Mar 20 21:06:40 usbgentoo kernel: R10: ffffb3ff4b0f41a0 R11: 00000000fffffff3 R12: 000000000005c712 Mar 20 21:06:40 usbgentoo kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000000 Mar 20 21:06:40 usbgentoo kernel: FS: 0000000000000000(0000) GS:ffff9341fefc0000(0000) knlGS:0000000000000000 Mar 20 21:06:40 usbgentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Mar 20 21:06:40 usbgentoo kernel: CR2: ffffb3ff4b0f3ff8 CR3: 000000006320e006 CR4: 00000000003606e0 Mar 20 21:06:40 usbgentoo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Mar 20 21:06:40 usbgentoo kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Mar 20 21:06:40 usbgentoo kernel: kworker/u16:15 (1606) used greatest stack depth: 8 bytes left Mar 20 21:06:40 usbgentoo kernel: WARNING: CPU: 7 PID: 0 at kernel/rcu/tree.c:600 rcu_idle_enter+0xe6/0xf0 Mar 20 21:06:40 usbgentoo kernel: Modules linked in: iTCO_wdt efivarfs Mar 20 21:06:40 usbgentoo kernel: CPU: 7 PID: 0 Comm: swapper/7 Tainted: G D 5.0.3-gentoo #6 Mar 20 21:06:40 usbgentoo kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./C226 WS, BIOS P3.40 06/25/2018 Mar 20 21:06:40 usbgentoo kernel: RIP: 0010:rcu_idle_enter+0xe6/0xf0 Mar 20 21:06:40 usbgentoo kernel: Code: 27 48 8b 45 00 48 8b 7d 08 48 83 c5 18 31 c9 45 89 e8 ba 01 00 00 00 4c 89 e6 e8 45 40 d5 00 48 8b 45 00 48 85 c0 75 dd eb 86 <0f> 0b e9 42 ff ff ff 0f 1f 00 41 57 41 56 41 55 41 54 55 53 48 c7 Mar 20 21:06:40 usbgentoo kernel: RSP: 0018:ffffb3ff431dfec0 EFLAGS: 00010002 Mar 20 21:06:40 usbgentoo kernel: RAX: ffff9341fefe14c0 RBX: 00000000000214c0 RCX: 0000000000000000 Mar 20 21:06:40 usbgentoo kernel: RDX: 4000000000000000 RSI: 0000000000000348 RDI: ffff9341fefe8120 Mar 20 21:06:40 usbgentoo kernel: RBP: ffffffff8d592920 R08: 000000000002bd5d R09: 0000000000039437 Mar 20 21:06:40 usbgentoo kernel: R10: ffff9341fefdf9e4 R11: ffff9341fefdf9c4 R12: ffff9341fefe8120 Mar 20 21:06:40 usbgentoo kernel: R13: ffffffff8d4a8a20 R14: 0000000000000004 R15: 0000000000000000 Mar 20 21:06:40 usbgentoo kernel: FS: 0000000000000000(0000) GS:ffff9341fefc0000(0000) knlGS:0000000000000000 Mar 20 21:06:40 usbgentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Mar 20 21:06:40 usbgentoo kernel: CR2: ffffb3ff4b0f3ff8 CR3: 000000006320e006 CR4: 00000000003606e0 Mar 20 21:06:40 usbgentoo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Mar 20 21:06:40 usbgentoo kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Mar 20 21:06:40 usbgentoo kernel: Call Trace: Mar 20 21:06:40 usbgentoo kernel: do_idle+0x193/0x1e0 Mar 20 21:06:40 usbgentoo kernel: cpu_startup_entry+0x14/0x20 Mar 20 21:06:40 usbgentoo kernel: start_secondary+0x183/0x1b0 Mar 20 21:06:40 usbgentoo kernel: secondary_startup_64+0xa4/0xb0 Mar 20 21:06:40 usbgentoo kernel: ---[ end trace 7d32682090ec782a ]--- ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-03-27 13:59 BUG: kernel stack overflow when mounting with data_flush Hagbard Celine @ 2019-03-30 3:25 ` Chao Yu 2019-03-30 7:29 ` Chao Yu 0 siblings, 1 reply; 16+ messages in thread From: Chao Yu @ 2019-03-30 3:25 UTC (permalink / raw) To: Hagbard Celine, linux-f2fs-devel Hi Hagbard, Sorry for the delay. On 2019/3/27 21:59, Hagbard Celine wrote: > Hi, this is a long standing bug that I've hit before on older kernels, > but I was not able to get the syslog saved because of the nature of > the bug. This time I had booted form a pen-drive, and was able to save > the log to it's efi-partition. Now which version of kernel do you use? and do you remember what is your kernel version when this bug occured at first time? > What i did to trigger it was to create a partition and format it f2fs, > then mount it with options: > "rw,relatime,lazytime,background_gc=on,disable_ext_identify,discard,heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,data_flush,extent_cache,mode=adaptive,active_logs=6,whint_mode=fs-based,alloc_mode=default,fsync_mode=strict". > Then I unpacked a big .tar.xz to the partition (I used a > gentoo-stage3-tarball as I was in process of installing Gentoo). > > Same options just without data_flush gives no problems. > > Syslog from crash follows: > Mar 20 20:20:34 usbgentoo syslog-ng[3644]: syslog-ng starting up; > version='3.17.2' > Mar 20 20:20:34 usbgentoo /usr/sbin/gpm[3674]: *** info > [daemon/startup.c(136)]: > Mar 20 20:20:34 usbgentoo /usr/sbin/gpm[3674]: Started gpm > successfully. Entered daemon mode. > Mar 20 20:20:34 usbgentoo kernel: ip (3771) used greatest stack depth: > 12312 bytes left > Mar 20 20:20:34 usbgentoo dhcpcd[3840]: eth0: waiting for carrier > Mar 20 20:20:37 usbgentoo kernel: igb 0000:03:00.0 eth0: igb: eth0 NIC > Link is Up 1000 Mbps Full Duplex, Flow Control: RX > Mar 20 20:20:37 usbgentoo kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: > link becomes ready > Mar 20 20:20:37 usbgentoo dhcpcd[3840]: eth0: carrier acquired > Mar 20 20:20:38 usbgentoo dhcpcd[3840]: DUID > 00:01:00:01:24:24:e8:94:d0:50:99:3b:c9:21 > Mar 20 20:20:38 usbgentoo dhcpcd[3840]: eth0: IAID 99:3b:c9:21 > Mar 20 20:20:38 usbgentoo dhcpcd[3840]: eth0: adding address > fe80::3c99:2bbf:63bb:8354 > Mar 20 20:20:38 usbgentoo dhcpcd[3840]: eth0: rebinding lease of 192.168.1.22 > Mar 20 20:20:38 usbgentoo dhcpcd[3840]: eth0: probing address 192.168.1.22/24 > Mar 20 20:20:39 usbgentoo dhcpcd[3840]: eth0: soliciting an IPv6 router > Mar 20 20:20:43 usbgentoo dhcpcd[3840]: eth0: leased 192.168.1.22 for > 86400 seconds > Mar 20 20:20:43 usbgentoo dhcpcd[3840]: eth0: adding route to 192.168.1.0/24 > Mar 20 20:20:43 usbgentoo dhcpcd[3840]: eth0: adding default route via > 192.168.1.1 > Mar 20 20:20:43 usbgentoo dhcpcd[3840]: forked to background, child pid 3883 > Mar 20 20:20:48 usbgentoo login[4003]: pam_unix(login:auth): > authentication failure; logname=LOGIN uid=0 euid=0 tty=/dev/tty1 > ruser= rhost= user=root > Mar 20 20:20:51 usbgentoo dhcpcd[3883]: eth0: no IPv6 Routers available > Mar 20 20:21:06 usbgentoo login[4003]: pam_unix(login:session): > session opened for user root by LOGIN(uid=0) > Mar 20 20:21:06 usbgentoo login[4012]: ROOT LOGIN on '/dev/tty1' > Mar 20 20:22:30 usbgentoo kernel: EXT4-fs (sdb4): recovery complete > Mar 20 20:22:30 usbgentoo kernel: EXT4-fs (sdb4): mounted filesystem > with ordered data mode. Opts: (null) > Mar 20 20:22:39 usbgentoo kernel: EXT4-fs (sdb5): recovery complete > Mar 20 20:22:39 usbgentoo kernel: EXT4-fs (sdb5): mounted filesystem > with ordered data mode. Opts: (null) > Mar 20 20:22:46 usbgentoo login[4003]: pam_unix(login:session): > session closed for user root > Mar 20 20:22:51 usbgentoo login[4025]: pam_unix(login:session): > session opened for user root by LOGIN(uid=0) > Mar 20 20:22:51 usbgentoo login[4027]: ROOT LOGIN on '/dev/tty1' > Mar 20 20:23:34 usbgentoo login[4004]: pam_unix(login:session): > session opened for user root by LOGIN(uid=0) > Mar 20 20:23:34 usbgentoo login[4034]: ROOT LOGIN on '/dev/tty2' > Mar 20 20:30:21 usbgentoo kernel: F2FS-fs (nvme0n1p5): Found nat_bits > in checkpoint > Mar 20 20:30:22 usbgentoo kernel: F2FS-fs (nvme0n1p5): Mounted with > checkpoint version = b9e8e7 > Mar 20 20:30:35 usbgentoo login[4025]: pam_unix(login:session): > session closed for user root > Mar 20 20:30:42 usbgentoo login[4061]: pam_unix(login:session): > session opened for user root by LOGIN(uid=0) > Mar 20 20:30:42 usbgentoo login[4063]: ROOT LOGIN on '/dev/tty1' > Mar 20 20:40:31 usbgentoo kernel: Adding 23984124k swap on > /dev/nvme0n1p6. Priority:-2 extents:1 across:23984124k SSDsc > Mar 20 20:54:01 usbgentoo kernel: FAT-fs (nvme0n1p4): Volume was not > properly unmounted. Some data may be corrupt. Please run fsck. It looks that there are any other problems in your environment, nvme0n1p4 volume is corrupted. > Mar 20 21:05:23 usbgentoo kernel: kworker/dying (1588) used greatest > stack depth: 12064 bytes left > Mar 20 21:06:40 usbgentoo kernel: BUG: stack guard page was hit at > 00000000a4b0733c (stack is 0000000056016422..0000000096e7463f) > Mar 20 21:06:40 usbgentoo kernel: kernel stack overflow And your kernel stack is overflow.... Thanks, > (double-fault): 0000 [#1] SMP PTI > Mar 20 21:06:40 usbgentoo kernel: CPU: 7 PID: 1606 Comm: > kworker/u16:15 Not tainted 5.0.3-gentoo #6 > Mar 20 21:06:40 usbgentoo kernel: Hardware name: To Be Filled By > O.E.M. To Be Filled By O.E.M./C226 WS, BIOS P3.40 06/25/2018 > Mar 20 21:06:40 usbgentoo kernel: Workqueue: writeback wb_workfn (flush-259:0) > Mar 20 21:06:40 usbgentoo kernel: RIP: 0010:f2fs_inode_chksum_verify+0x14/0xc0 > Mar 20 21:06:40 usbgentoo kernel: Code: 00 00 04 0f 45 f0 e9 3b 85 e8 > ff 90 66 2e 0f 1f 84 00 00 00 00 00 48 8b 47 48 a8 40 0f 85 9d 00 00 > 00 41 55 41 54 55 48 89 fd <53> 48 89 f3 e8 33 fd ff ff ba 01 00 00 00 > 84 c0 75 09 5b 89 d0 5d > Mar 20 21:06:40 usbgentoo kernel: RSP: 0018:ffffb3ff4b0f4000 EFLAGS: 00010246 > Mar 20 21:06:40 usbgentoo kernel: RAX: 0000000000000001 RBX: > ffffd57b5afeaa00 RCX: 0000000000000000 > Mar 20 21:06:40 usbgentoo kernel: RDX: 0000000000000000 RSI: > ffffd57b5afeaa00 RDI: ffff9341f77f2000 > Mar 20 21:06:40 usbgentoo kernel: RBP: ffff9341f77f2000 R08: > ffffb3ff4b0f4110 R09: ffff9340bfaa9377 > Mar 20 21:06:40 usbgentoo kernel: R10: ffffb3ff4b0f41a0 R11: > 00000000fffffff3 R12: 000000000005c712 > Mar 20 21:06:40 usbgentoo kernel: R13: 0000000000000000 R14: > 0000000000000001 R15: 0000000000000000 > Mar 20 21:06:40 usbgentoo kernel: FS: 0000000000000000(0000) > GS:ffff9341fefc0000(0000) knlGS:0000000000000000 > Mar 20 21:06:40 usbgentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: > 0000000080050033 > Mar 20 21:06:40 usbgentoo kernel: CR2: ffffb3ff4b0f3ff8 CR3: > 000000006320e006 CR4: 00000000003606e0 > Mar 20 21:06:40 usbgentoo kernel: DR0: 0000000000000000 DR1: > 0000000000000000 DR2: 0000000000000000 > Mar 20 21:06:40 usbgentoo kernel: DR3: 0000000000000000 DR6: > 00000000fffe0ff0 DR7: 0000000000000400 > Mar 20 21:06:40 usbgentoo kernel: Call Trace: > Mar 20 21:06:40 usbgentoo kernel: read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: ? xas_load+0x8/0x50 > Mar 20 21:06:40 usbgentoo kernel: __get_node_page+0x73/0x2a0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_get_dnode_of_data+0x34e/0x580 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_inline_data+0x5e/0x2a0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x421/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: ? find_get_entries_tag+0x91/0x260 > Mar 20 21:06:40 usbgentoo kernel: ? get_partial_node.isra.86.part.87+0x2d/0x200 > Mar 20 21:06:40 usbgentoo kernel: ? mempool_alloc+0x62/0x190 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: ? wait_woken+0x80/0x80 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: ? __switch_to_asm+0x40/0x70 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: ? raid6_avx5122_gen_syndrome+0x151/0x160 > Mar 20 21:06:40 usbgentoo kernel: ? sbitmap_get+0x71/0x160 > Mar 20 21:06:40 usbgentoo kernel: ? __sbitmap_queue_get+0x24/0x90 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 > Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 > Mar 20 21:06:40 usbgentoo kernel: ? __update_load_avg_cfs_rq+0x1b1/0x220 > Mar 20 21:06:40 usbgentoo kernel: ? enqueue_task_fair+0x441/0xfd0 > Mar 20 21:06:40 usbgentoo kernel: ? guard_bio_eod+0x27/0xf0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? try_to_wake_up+0x52/0x550 > Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 > Mar 20 21:06:40 usbgentoo kernel: ? nr_blockdev_pages+0xc/0x60 > Mar 20 21:06:40 usbgentoo kernel: ? si_meminfo+0x3a/0x60 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_available_free_memory+0x29/0x240 > Mar 20 21:06:40 usbgentoo kernel: ? complete+0x36/0x50 > Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 > Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 > Mar 20 21:06:40 usbgentoo kernel: ? __set_page_dirty_nobuffers+0xcb/0x100 > Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 > Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_set+0xc/0x40 > Mar 20 21:06:40 usbgentoo kernel: f2fs_write_inode+0x79/0x1c0 > Mar 20 21:06:40 usbgentoo kernel: __writeback_single_inode+0x2a1/0x340 > Mar 20 21:06:40 usbgentoo kernel: ? soft_cursor+0x1b4/0x220 > Mar 20 21:06:40 usbgentoo kernel: writeback_sb_inodes+0x1d5/0x3e0 > Mar 20 21:06:40 usbgentoo kernel: __writeback_inodes_wb+0x58/0xa0 > Mar 20 21:06:40 usbgentoo kernel: wb_writeback+0x250/0x2e0 > Mar 20 21:06:40 usbgentoo kernel: ? 0xffffffff8c000000 > Mar 20 21:06:40 usbgentoo kernel: ? cpumask_next+0x16/0x20 > Mar 20 21:06:40 usbgentoo kernel: wb_workfn+0x2f6/0x3b0 > Mar 20 21:06:40 usbgentoo kernel: ? __switch_to_asm+0x40/0x70 > Mar 20 21:06:40 usbgentoo kernel: process_one_work+0x1f5/0x3f0 > Mar 20 21:06:40 usbgentoo kernel: worker_thread+0x28/0x3c0 > Mar 20 21:06:40 usbgentoo kernel: ? rescuer_thread+0x330/0x330 > Mar 20 21:06:40 usbgentoo kernel: kthread+0x10e/0x130 > Mar 20 21:06:40 usbgentoo kernel: ? kthread_create_on_node+0x60/0x60 > Mar 20 21:06:40 usbgentoo kernel: ret_from_fork+0x35/0x40 > Mar 20 21:06:40 usbgentoo kernel: Modules linked in: iTCO_wdt efivarfs > Mar 20 21:06:40 usbgentoo kernel: ---[ end trace 7d32682090ec7829 ]--- > Mar 20 21:06:40 usbgentoo kernel: RIP: 0010:f2fs_inode_chksum_verify+0x14/0xc0 > Mar 20 21:06:40 usbgentoo kernel: Code: 00 00 04 0f 45 f0 e9 3b 85 e8 > ff 90 66 2e 0f 1f 84 00 00 00 00 00 48 8b 47 48 a8 40 0f 85 9d 00 00 > 00 41 55 41 54 55 48 89 fd <53> 48 89 f3 e8 33 fd ff ff ba 01 00 00 00 > 84 c0 75 09 5b 89 d0 5d > Mar 20 21:06:40 usbgentoo kernel: RSP: 0018:ffffb3ff4b0f4000 EFLAGS: 00010246 > Mar 20 21:06:40 usbgentoo kernel: RAX: 0000000000000001 RBX: > ffffd57b5afeaa00 RCX: 0000000000000000 > Mar 20 21:06:40 usbgentoo kernel: RDX: 0000000000000000 RSI: > ffffd57b5afeaa00 RDI: ffff9341f77f2000 > Mar 20 21:06:40 usbgentoo kernel: RBP: ffff9341f77f2000 R08: > ffffb3ff4b0f4110 R09: ffff9340bfaa9377 > Mar 20 21:06:40 usbgentoo kernel: R10: ffffb3ff4b0f41a0 R11: > 00000000fffffff3 R12: 000000000005c712 > Mar 20 21:06:40 usbgentoo kernel: R13: 0000000000000000 R14: > 0000000000000001 R15: 0000000000000000 > Mar 20 21:06:40 usbgentoo kernel: FS: 0000000000000000(0000) > GS:ffff9341fefc0000(0000) knlGS:0000000000000000 > Mar 20 21:06:40 usbgentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: > 0000000080050033 > Mar 20 21:06:40 usbgentoo kernel: CR2: ffffb3ff4b0f3ff8 CR3: > 000000006320e006 CR4: 00000000003606e0 > Mar 20 21:06:40 usbgentoo kernel: DR0: 0000000000000000 DR1: > 0000000000000000 DR2: 0000000000000000 > Mar 20 21:06:40 usbgentoo kernel: DR3: 0000000000000000 DR6: > 00000000fffe0ff0 DR7: 0000000000000400 > Mar 20 21:06:40 usbgentoo kernel: kworker/u16:15 (1606) used greatest > stack depth: 8 bytes left > Mar 20 21:06:40 usbgentoo kernel: WARNING: CPU: 7 PID: 0 at > kernel/rcu/tree.c:600 rcu_idle_enter+0xe6/0xf0 > Mar 20 21:06:40 usbgentoo kernel: Modules linked in: iTCO_wdt efivarfs > Mar 20 21:06:40 usbgentoo kernel: CPU: 7 PID: 0 Comm: swapper/7 > Tainted: G D 5.0.3-gentoo #6 > Mar 20 21:06:40 usbgentoo kernel: Hardware name: To Be Filled By > O.E.M. To Be Filled By O.E.M./C226 WS, BIOS P3.40 06/25/2018 > Mar 20 21:06:40 usbgentoo kernel: RIP: 0010:rcu_idle_enter+0xe6/0xf0 > Mar 20 21:06:40 usbgentoo kernel: Code: 27 48 8b 45 00 48 8b 7d 08 48 > 83 c5 18 31 c9 45 89 e8 ba 01 00 00 00 4c 89 e6 e8 45 40 d5 00 48 8b > 45 00 48 85 c0 75 dd eb 86 <0f> 0b e9 42 ff ff ff 0f 1f 00 41 57 41 56 > 41 55 41 54 55 53 48 c7 > Mar 20 21:06:40 usbgentoo kernel: RSP: 0018:ffffb3ff431dfec0 EFLAGS: 00010002 > Mar 20 21:06:40 usbgentoo kernel: RAX: ffff9341fefe14c0 RBX: > 00000000000214c0 RCX: 0000000000000000 > Mar 20 21:06:40 usbgentoo kernel: RDX: 4000000000000000 RSI: > 0000000000000348 RDI: ffff9341fefe8120 > Mar 20 21:06:40 usbgentoo kernel: RBP: ffffffff8d592920 R08: > 000000000002bd5d R09: 0000000000039437 > Mar 20 21:06:40 usbgentoo kernel: R10: ffff9341fefdf9e4 R11: > ffff9341fefdf9c4 R12: ffff9341fefe8120 > Mar 20 21:06:40 usbgentoo kernel: R13: ffffffff8d4a8a20 R14: > 0000000000000004 R15: 0000000000000000 > Mar 20 21:06:40 usbgentoo kernel: FS: 0000000000000000(0000) > GS:ffff9341fefc0000(0000) knlGS:0000000000000000 > Mar 20 21:06:40 usbgentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: > 0000000080050033 > Mar 20 21:06:40 usbgentoo kernel: CR2: ffffb3ff4b0f3ff8 CR3: > 000000006320e006 CR4: 00000000003606e0 > Mar 20 21:06:40 usbgentoo kernel: DR0: 0000000000000000 DR1: > 0000000000000000 DR2: 0000000000000000 > Mar 20 21:06:40 usbgentoo kernel: DR3: 0000000000000000 DR6: > 00000000fffe0ff0 DR7: 0000000000000400 > Mar 20 21:06:40 usbgentoo kernel: Call Trace: > Mar 20 21:06:40 usbgentoo kernel: do_idle+0x193/0x1e0 > Mar 20 21:06:40 usbgentoo kernel: cpu_startup_entry+0x14/0x20 > Mar 20 21:06:40 usbgentoo kernel: start_secondary+0x183/0x1b0 > Mar 20 21:06:40 usbgentoo kernel: secondary_startup_64+0xa4/0xb0 > Mar 20 21:06:40 usbgentoo kernel: ---[ end trace 7d32682090ec782a ]--- > > > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel > . > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-03-30 3:25 ` Chao Yu @ 2019-03-30 7:29 ` Chao Yu 2019-03-30 18:54 ` Hagbard Celine 0 siblings, 1 reply; 16+ messages in thread From: Chao Yu @ 2019-03-30 7:29 UTC (permalink / raw) To: Hagbard Celine, linux-f2fs-devel Oh, sorry, it's quite possible that bug is caused by data_flush, could remove that mount option first? Thanks, On 2019/3/30 11:25, Chao Yu wrote: > Hi Hagbard, > > Sorry for the delay. > > On 2019/3/27 21:59, Hagbard Celine wrote: >> Hi, this is a long standing bug that I've hit before on older kernels, >> but I was not able to get the syslog saved because of the nature of >> the bug. This time I had booted form a pen-drive, and was able to save >> the log to it's efi-partition. > > Now which version of kernel do you use? and do you remember what is your > kernel version when this bug occured at first time? > >> What i did to trigger it was to create a partition and format it f2fs, >> then mount it with options: >> "rw,relatime,lazytime,background_gc=on,disable_ext_identify,discard,heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,data_flush,extent_cache,mode=adaptive,active_logs=6,whint_mode=fs-based,alloc_mode=default,fsync_mode=strict". >> Then I unpacked a big .tar.xz to the partition (I used a >> gentoo-stage3-tarball as I was in process of installing Gentoo). >> >> Same options just without data_flush gives no problems. >> >> Syslog from crash follows: >> Mar 20 20:20:34 usbgentoo syslog-ng[3644]: syslog-ng starting up; >> version='3.17.2' >> Mar 20 20:20:34 usbgentoo /usr/sbin/gpm[3674]: *** info >> [daemon/startup.c(136)]: >> Mar 20 20:20:34 usbgentoo /usr/sbin/gpm[3674]: Started gpm >> successfully. Entered daemon mode. >> Mar 20 20:20:34 usbgentoo kernel: ip (3771) used greatest stack depth: >> 12312 bytes left >> Mar 20 20:20:34 usbgentoo dhcpcd[3840]: eth0: waiting for carrier >> Mar 20 20:20:37 usbgentoo kernel: igb 0000:03:00.0 eth0: igb: eth0 NIC >> Link is Up 1000 Mbps Full Duplex, Flow Control: RX >> Mar 20 20:20:37 usbgentoo kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: >> link becomes ready >> Mar 20 20:20:37 usbgentoo dhcpcd[3840]: eth0: carrier acquired >> Mar 20 20:20:38 usbgentoo dhcpcd[3840]: DUID >> 00:01:00:01:24:24:e8:94:d0:50:99:3b:c9:21 >> Mar 20 20:20:38 usbgentoo dhcpcd[3840]: eth0: IAID 99:3b:c9:21 >> Mar 20 20:20:38 usbgentoo dhcpcd[3840]: eth0: adding address >> fe80::3c99:2bbf:63bb:8354 >> Mar 20 20:20:38 usbgentoo dhcpcd[3840]: eth0: rebinding lease of 192.168.1.22 >> Mar 20 20:20:38 usbgentoo dhcpcd[3840]: eth0: probing address 192.168.1.22/24 >> Mar 20 20:20:39 usbgentoo dhcpcd[3840]: eth0: soliciting an IPv6 router >> Mar 20 20:20:43 usbgentoo dhcpcd[3840]: eth0: leased 192.168.1.22 for >> 86400 seconds >> Mar 20 20:20:43 usbgentoo dhcpcd[3840]: eth0: adding route to 192.168.1.0/24 >> Mar 20 20:20:43 usbgentoo dhcpcd[3840]: eth0: adding default route via >> 192.168.1.1 >> Mar 20 20:20:43 usbgentoo dhcpcd[3840]: forked to background, child pid 3883 >> Mar 20 20:20:48 usbgentoo login[4003]: pam_unix(login:auth): >> authentication failure; logname=LOGIN uid=0 euid=0 tty=/dev/tty1 >> ruser= rhost= user=root >> Mar 20 20:20:51 usbgentoo dhcpcd[3883]: eth0: no IPv6 Routers available >> Mar 20 20:21:06 usbgentoo login[4003]: pam_unix(login:session): >> session opened for user root by LOGIN(uid=0) >> Mar 20 20:21:06 usbgentoo login[4012]: ROOT LOGIN on '/dev/tty1' >> Mar 20 20:22:30 usbgentoo kernel: EXT4-fs (sdb4): recovery complete >> Mar 20 20:22:30 usbgentoo kernel: EXT4-fs (sdb4): mounted filesystem >> with ordered data mode. Opts: (null) >> Mar 20 20:22:39 usbgentoo kernel: EXT4-fs (sdb5): recovery complete >> Mar 20 20:22:39 usbgentoo kernel: EXT4-fs (sdb5): mounted filesystem >> with ordered data mode. Opts: (null) >> Mar 20 20:22:46 usbgentoo login[4003]: pam_unix(login:session): >> session closed for user root >> Mar 20 20:22:51 usbgentoo login[4025]: pam_unix(login:session): >> session opened for user root by LOGIN(uid=0) >> Mar 20 20:22:51 usbgentoo login[4027]: ROOT LOGIN on '/dev/tty1' >> Mar 20 20:23:34 usbgentoo login[4004]: pam_unix(login:session): >> session opened for user root by LOGIN(uid=0) >> Mar 20 20:23:34 usbgentoo login[4034]: ROOT LOGIN on '/dev/tty2' >> Mar 20 20:30:21 usbgentoo kernel: F2FS-fs (nvme0n1p5): Found nat_bits >> in checkpoint >> Mar 20 20:30:22 usbgentoo kernel: F2FS-fs (nvme0n1p5): Mounted with >> checkpoint version = b9e8e7 >> Mar 20 20:30:35 usbgentoo login[4025]: pam_unix(login:session): >> session closed for user root >> Mar 20 20:30:42 usbgentoo login[4061]: pam_unix(login:session): >> session opened for user root by LOGIN(uid=0) >> Mar 20 20:30:42 usbgentoo login[4063]: ROOT LOGIN on '/dev/tty1' >> Mar 20 20:40:31 usbgentoo kernel: Adding 23984124k swap on >> /dev/nvme0n1p6. Priority:-2 extents:1 across:23984124k SSDsc >> Mar 20 20:54:01 usbgentoo kernel: FAT-fs (nvme0n1p4): Volume was not >> properly unmounted. Some data may be corrupt. Please run fsck. > > It looks that there are any other problems in your environment, nvme0n1p4 > volume is corrupted. > >> Mar 20 21:05:23 usbgentoo kernel: kworker/dying (1588) used greatest >> stack depth: 12064 bytes left >> Mar 20 21:06:40 usbgentoo kernel: BUG: stack guard page was hit at >> 00000000a4b0733c (stack is 0000000056016422..0000000096e7463f) >> Mar 20 21:06:40 usbgentoo kernel: kernel stack overflow > > And your kernel stack is overflow.... > > Thanks, > >> (double-fault): 0000 [#1] SMP PTI >> Mar 20 21:06:40 usbgentoo kernel: CPU: 7 PID: 1606 Comm: >> kworker/u16:15 Not tainted 5.0.3-gentoo #6 >> Mar 20 21:06:40 usbgentoo kernel: Hardware name: To Be Filled By >> O.E.M. To Be Filled By O.E.M./C226 WS, BIOS P3.40 06/25/2018 >> Mar 20 21:06:40 usbgentoo kernel: Workqueue: writeback wb_workfn (flush-259:0) >> Mar 20 21:06:40 usbgentoo kernel: RIP: 0010:f2fs_inode_chksum_verify+0x14/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: Code: 00 00 04 0f 45 f0 e9 3b 85 e8 >> ff 90 66 2e 0f 1f 84 00 00 00 00 00 48 8b 47 48 a8 40 0f 85 9d 00 00 >> 00 41 55 41 54 55 48 89 fd <53> 48 89 f3 e8 33 fd ff ff ba 01 00 00 00 >> 84 c0 75 09 5b 89 d0 5d >> Mar 20 21:06:40 usbgentoo kernel: RSP: 0018:ffffb3ff4b0f4000 EFLAGS: 00010246 >> Mar 20 21:06:40 usbgentoo kernel: RAX: 0000000000000001 RBX: >> ffffd57b5afeaa00 RCX: 0000000000000000 >> Mar 20 21:06:40 usbgentoo kernel: RDX: 0000000000000000 RSI: >> ffffd57b5afeaa00 RDI: ffff9341f77f2000 >> Mar 20 21:06:40 usbgentoo kernel: RBP: ffff9341f77f2000 R08: >> ffffb3ff4b0f4110 R09: ffff9340bfaa9377 >> Mar 20 21:06:40 usbgentoo kernel: R10: ffffb3ff4b0f41a0 R11: >> 00000000fffffff3 R12: 000000000005c712 >> Mar 20 21:06:40 usbgentoo kernel: R13: 0000000000000000 R14: >> 0000000000000001 R15: 0000000000000000 >> Mar 20 21:06:40 usbgentoo kernel: FS: 0000000000000000(0000) >> GS:ffff9341fefc0000(0000) knlGS:0000000000000000 >> Mar 20 21:06:40 usbgentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: >> 0000000080050033 >> Mar 20 21:06:40 usbgentoo kernel: CR2: ffffb3ff4b0f3ff8 CR3: >> 000000006320e006 CR4: 00000000003606e0 >> Mar 20 21:06:40 usbgentoo kernel: DR0: 0000000000000000 DR1: >> 0000000000000000 DR2: 0000000000000000 >> Mar 20 21:06:40 usbgentoo kernel: DR3: 0000000000000000 DR6: >> 00000000fffe0ff0 DR7: 0000000000000400 >> Mar 20 21:06:40 usbgentoo kernel: Call Trace: >> Mar 20 21:06:40 usbgentoo kernel: read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: ? xas_load+0x8/0x50 >> Mar 20 21:06:40 usbgentoo kernel: __get_node_page+0x73/0x2a0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_get_dnode_of_data+0x34e/0x580 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_inline_data+0x5e/0x2a0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x421/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: ? find_get_entries_tag+0x91/0x260 >> Mar 20 21:06:40 usbgentoo kernel: ? get_partial_node.isra.86.part.87+0x2d/0x200 >> Mar 20 21:06:40 usbgentoo kernel: ? mempool_alloc+0x62/0x190 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: ? wait_woken+0x80/0x80 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: ? __switch_to_asm+0x40/0x70 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: ? raid6_avx5122_gen_syndrome+0x151/0x160 >> Mar 20 21:06:40 usbgentoo kernel: ? sbitmap_get+0x71/0x160 >> Mar 20 21:06:40 usbgentoo kernel: ? __sbitmap_queue_get+0x24/0x90 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_verify+0x1d/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: ? read_node_page+0x71/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_dirtied+0x21/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: __write_data_page+0x4fb/0x690 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_cache_pages+0x1cf/0x460 >> Mar 20 21:06:40 usbgentoo kernel: ? __update_load_avg_cfs_rq+0x1b1/0x220 >> Mar 20 21:06:40 usbgentoo kernel: ? enqueue_task_fair+0x441/0xfd0 >> Mar 20 21:06:40 usbgentoo kernel: ? guard_bio_eod+0x27/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_data_pages+0x2b3/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? try_to_wake_up+0x52/0x550 >> Mar 20 21:06:40 usbgentoo kernel: do_writepages+0x3c/0xd0 >> Mar 20 21:06:40 usbgentoo kernel: ? nr_blockdev_pages+0xc/0x60 >> Mar 20 21:06:40 usbgentoo kernel: ? si_meminfo+0x3a/0x60 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_available_free_memory+0x29/0x240 >> Mar 20 21:06:40 usbgentoo kernel: ? complete+0x36/0x50 >> Mar 20 21:06:40 usbgentoo kernel: __filemap_fdatawrite_range+0x7c/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_sync_dirty_inodes+0xf2/0x200 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs_bg+0x2a3/0x2c0 >> Mar 20 21:06:40 usbgentoo kernel: ? __set_page_dirty_nobuffers+0xcb/0x100 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_balance_fs+0xd6/0x2b0 >> Mar 20 21:06:40 usbgentoo kernel: ? f2fs_inode_chksum_set+0xc/0x40 >> Mar 20 21:06:40 usbgentoo kernel: f2fs_write_inode+0x79/0x1c0 >> Mar 20 21:06:40 usbgentoo kernel: __writeback_single_inode+0x2a1/0x340 >> Mar 20 21:06:40 usbgentoo kernel: ? soft_cursor+0x1b4/0x220 >> Mar 20 21:06:40 usbgentoo kernel: writeback_sb_inodes+0x1d5/0x3e0 >> Mar 20 21:06:40 usbgentoo kernel: __writeback_inodes_wb+0x58/0xa0 >> Mar 20 21:06:40 usbgentoo kernel: wb_writeback+0x250/0x2e0 >> Mar 20 21:06:40 usbgentoo kernel: ? 0xffffffff8c000000 >> Mar 20 21:06:40 usbgentoo kernel: ? cpumask_next+0x16/0x20 >> Mar 20 21:06:40 usbgentoo kernel: wb_workfn+0x2f6/0x3b0 >> Mar 20 21:06:40 usbgentoo kernel: ? __switch_to_asm+0x40/0x70 >> Mar 20 21:06:40 usbgentoo kernel: process_one_work+0x1f5/0x3f0 >> Mar 20 21:06:40 usbgentoo kernel: worker_thread+0x28/0x3c0 >> Mar 20 21:06:40 usbgentoo kernel: ? rescuer_thread+0x330/0x330 >> Mar 20 21:06:40 usbgentoo kernel: kthread+0x10e/0x130 >> Mar 20 21:06:40 usbgentoo kernel: ? kthread_create_on_node+0x60/0x60 >> Mar 20 21:06:40 usbgentoo kernel: ret_from_fork+0x35/0x40 >> Mar 20 21:06:40 usbgentoo kernel: Modules linked in: iTCO_wdt efivarfs >> Mar 20 21:06:40 usbgentoo kernel: ---[ end trace 7d32682090ec7829 ]--- >> Mar 20 21:06:40 usbgentoo kernel: RIP: 0010:f2fs_inode_chksum_verify+0x14/0xc0 >> Mar 20 21:06:40 usbgentoo kernel: Code: 00 00 04 0f 45 f0 e9 3b 85 e8 >> ff 90 66 2e 0f 1f 84 00 00 00 00 00 48 8b 47 48 a8 40 0f 85 9d 00 00 >> 00 41 55 41 54 55 48 89 fd <53> 48 89 f3 e8 33 fd ff ff ba 01 00 00 00 >> 84 c0 75 09 5b 89 d0 5d >> Mar 20 21:06:40 usbgentoo kernel: RSP: 0018:ffffb3ff4b0f4000 EFLAGS: 00010246 >> Mar 20 21:06:40 usbgentoo kernel: RAX: 0000000000000001 RBX: >> ffffd57b5afeaa00 RCX: 0000000000000000 >> Mar 20 21:06:40 usbgentoo kernel: RDX: 0000000000000000 RSI: >> ffffd57b5afeaa00 RDI: ffff9341f77f2000 >> Mar 20 21:06:40 usbgentoo kernel: RBP: ffff9341f77f2000 R08: >> ffffb3ff4b0f4110 R09: ffff9340bfaa9377 >> Mar 20 21:06:40 usbgentoo kernel: R10: ffffb3ff4b0f41a0 R11: >> 00000000fffffff3 R12: 000000000005c712 >> Mar 20 21:06:40 usbgentoo kernel: R13: 0000000000000000 R14: >> 0000000000000001 R15: 0000000000000000 >> Mar 20 21:06:40 usbgentoo kernel: FS: 0000000000000000(0000) >> GS:ffff9341fefc0000(0000) knlGS:0000000000000000 >> Mar 20 21:06:40 usbgentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: >> 0000000080050033 >> Mar 20 21:06:40 usbgentoo kernel: CR2: ffffb3ff4b0f3ff8 CR3: >> 000000006320e006 CR4: 00000000003606e0 >> Mar 20 21:06:40 usbgentoo kernel: DR0: 0000000000000000 DR1: >> 0000000000000000 DR2: 0000000000000000 >> Mar 20 21:06:40 usbgentoo kernel: DR3: 0000000000000000 DR6: >> 00000000fffe0ff0 DR7: 0000000000000400 >> Mar 20 21:06:40 usbgentoo kernel: kworker/u16:15 (1606) used greatest >> stack depth: 8 bytes left >> Mar 20 21:06:40 usbgentoo kernel: WARNING: CPU: 7 PID: 0 at >> kernel/rcu/tree.c:600 rcu_idle_enter+0xe6/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: Modules linked in: iTCO_wdt efivarfs >> Mar 20 21:06:40 usbgentoo kernel: CPU: 7 PID: 0 Comm: swapper/7 >> Tainted: G D 5.0.3-gentoo #6 >> Mar 20 21:06:40 usbgentoo kernel: Hardware name: To Be Filled By >> O.E.M. To Be Filled By O.E.M./C226 WS, BIOS P3.40 06/25/2018 >> Mar 20 21:06:40 usbgentoo kernel: RIP: 0010:rcu_idle_enter+0xe6/0xf0 >> Mar 20 21:06:40 usbgentoo kernel: Code: 27 48 8b 45 00 48 8b 7d 08 48 >> 83 c5 18 31 c9 45 89 e8 ba 01 00 00 00 4c 89 e6 e8 45 40 d5 00 48 8b >> 45 00 48 85 c0 75 dd eb 86 <0f> 0b e9 42 ff ff ff 0f 1f 00 41 57 41 56 >> 41 55 41 54 55 53 48 c7 >> Mar 20 21:06:40 usbgentoo kernel: RSP: 0018:ffffb3ff431dfec0 EFLAGS: 00010002 >> Mar 20 21:06:40 usbgentoo kernel: RAX: ffff9341fefe14c0 RBX: >> 00000000000214c0 RCX: 0000000000000000 >> Mar 20 21:06:40 usbgentoo kernel: RDX: 4000000000000000 RSI: >> 0000000000000348 RDI: ffff9341fefe8120 >> Mar 20 21:06:40 usbgentoo kernel: RBP: ffffffff8d592920 R08: >> 000000000002bd5d R09: 0000000000039437 >> Mar 20 21:06:40 usbgentoo kernel: R10: ffff9341fefdf9e4 R11: >> ffff9341fefdf9c4 R12: ffff9341fefe8120 >> Mar 20 21:06:40 usbgentoo kernel: R13: ffffffff8d4a8a20 R14: >> 0000000000000004 R15: 0000000000000000 >> Mar 20 21:06:40 usbgentoo kernel: FS: 0000000000000000(0000) >> GS:ffff9341fefc0000(0000) knlGS:0000000000000000 >> Mar 20 21:06:40 usbgentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: >> 0000000080050033 >> Mar 20 21:06:40 usbgentoo kernel: CR2: ffffb3ff4b0f3ff8 CR3: >> 000000006320e006 CR4: 00000000003606e0 >> Mar 20 21:06:40 usbgentoo kernel: DR0: 0000000000000000 DR1: >> 0000000000000000 DR2: 0000000000000000 >> Mar 20 21:06:40 usbgentoo kernel: DR3: 0000000000000000 DR6: >> 00000000fffe0ff0 DR7: 0000000000000400 >> Mar 20 21:06:40 usbgentoo kernel: Call Trace: >> Mar 20 21:06:40 usbgentoo kernel: do_idle+0x193/0x1e0 >> Mar 20 21:06:40 usbgentoo kernel: cpu_startup_entry+0x14/0x20 >> Mar 20 21:06:40 usbgentoo kernel: start_secondary+0x183/0x1b0 >> Mar 20 21:06:40 usbgentoo kernel: secondary_startup_64+0xa4/0xb0 >> Mar 20 21:06:40 usbgentoo kernel: ---[ end trace 7d32682090ec782a ]--- >> >> >> _______________________________________________ >> Linux-f2fs-devel mailing list >> Linux-f2fs-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel >> . >> > > > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel > . > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-03-30 7:29 ` Chao Yu @ 2019-03-30 18:54 ` Hagbard Celine 2019-04-01 8:05 ` Chao Yu 0 siblings, 1 reply; 16+ messages in thread From: Hagbard Celine @ 2019-03-30 18:54 UTC (permalink / raw) To: linux-f2fs-devel, yuchao0 First, yes it is caused by data_flush, this is what I am trying to report. Without that option there is no "stack guard page was hit" and no "kernel stack overflow" and kernel is stable. This time I was using kernel 5.0.3, as can be seen in the log in my first mail. I do not remember exactly what kernel version I tried the first time a saw this bug, but I believe the mount option data_flush was just added when I tried it the first time. The option has always lead to crash here. 2019-03-30 8:29 GMT+01:00, Chao Yu <yuchao0@huawei.com>: > Oh, sorry, it's quite possible that bug is caused by data_flush, could > remove that mount option first? > > Thanks, > > On 2019/3/30 11:25, Chao Yu wrote: >> Hi Hagbard, >> >> Sorry for the delay. >> >> On 2019/3/27 21:59, Hagbard Celine wrote: >>> Hi, this is a long standing bug that I've hit before on older kernels, >>> but I was not able to get the syslog saved because of the nature of >>> the bug. This time I had booted form a pen-drive, and was able to save >>> the log to it's efi-partition. >> >> Now which version of kernel do you use? and do you remember what is your >> kernel version when this bug occured at first time? >> ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-03-30 18:54 ` Hagbard Celine @ 2019-04-01 8:05 ` Chao Yu 2019-04-02 12:41 ` Hagbard Celine 0 siblings, 1 reply; 16+ messages in thread From: Chao Yu @ 2019-04-01 8:05 UTC (permalink / raw) To: Hagbard Celine, linux-f2fs-devel On 2019/3/31 2:54, Hagbard Celine wrote: > First, yes it is caused by data_flush, this is what I am trying to > report. Without that option there is no "stack guard page was hit" and > no "kernel stack overflow" and kernel is stable. > This time I was using kernel 5.0.3, as can be seen in the log in my first mail. > I do not remember exactly what kernel version I tried the first time a > saw this bug, but I believe the mount option data_flush was just added > when I tried it the first time. The option has always lead to crash > here. Sorry, out of mind at that time, data_flush key words slip out of my eye... Could you please try below patch? >From 65edbf14a198d0b50765e10340255e2071f7ae75 Mon Sep 17 00:00:00 2001 From: Chao Yu <yuchao0@huawei.com> Date: Mon, 1 Apr 2019 15:59:16 +0800 Subject: [PATCH] f2fs: fix potential recursive call when enabling data_flush Signed-off-by: Chao Yu <yuchao0@huawei.com> --- fs/f2fs/checkpoint.c | 6 ++---- fs/f2fs/data.c | 3 ++- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c index a98e1b02279e..935ebdb9cf47 100644 --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -1009,13 +1009,11 @@ int f2fs_sync_dirty_inodes(struct f2fs_sb_info *sbi, enum inode_type type) if (inode) { unsigned long cur_ino = inode->i_ino; - if (is_dir) - F2FS_I(inode)->cp_task = current; + F2FS_I(inode)->cp_task = current; filemap_fdatawrite(inode->i_mapping); - if (is_dir) - F2FS_I(inode)->cp_task = NULL; + F2FS_I(inode)->cp_task = NULL; iput(inode); /* We need to give cpu to another writers. */ diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index d87dfa5aa112..9d3c11e09a03 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -2038,7 +2038,8 @@ static int __write_data_page(struct page *page, bool *submitted, } unlock_page(page); - if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode)) + if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) && + !F2FS_I(inode)->cp_task) f2fs_balance_fs(sbi, need_balance_fs); if (unlikely(f2fs_cp_error(sbi))) { -- 2.18.0.rc1 > > 2019-03-30 8:29 GMT+01:00, Chao Yu <yuchao0@huawei.com>: >> Oh, sorry, it's quite possible that bug is caused by data_flush, could >> remove that mount option first? >> >> Thanks, >> >> On 2019/3/30 11:25, Chao Yu wrote: >>> Hi Hagbard, >>> >>> Sorry for the delay. >>> >>> On 2019/3/27 21:59, Hagbard Celine wrote: >>>> Hi, this is a long standing bug that I've hit before on older kernels, >>>> but I was not able to get the syslog saved because of the nature of >>>> the bug. This time I had booted form a pen-drive, and was able to save >>>> the log to it's efi-partition. >>> >>> Now which version of kernel do you use? and do you remember what is your >>> kernel version when this bug occured at first time? >>> > . > ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-04-01 8:05 ` Chao Yu @ 2019-04-02 12:41 ` Hagbard Celine 2019-04-02 13:31 ` Chao Yu 0 siblings, 1 reply; 16+ messages in thread From: Hagbard Celine @ 2019-04-02 12:41 UTC (permalink / raw) To: Chao Yu, linux-f2fs-devel That seems to have fixed it. No more errors in syslog after extracting my stage3 tarball. Also ran a couple of kernel compiles on a partition mounted with data_flush and system seems stable. 2019-04-01 10:05 GMT+02:00, Chao Yu <yuchao0@huawei.com>: > On 2019/3/31 2:54, Hagbard Celine wrote: >> First, yes it is caused by data_flush, this is what I am trying to >> report. Without that option there is no "stack guard page was hit" and >> no "kernel stack overflow" and kernel is stable. >> This time I was using kernel 5.0.3, as can be seen in the log in my first >> mail. >> I do not remember exactly what kernel version I tried the first time a >> saw this bug, but I believe the mount option data_flush was just added >> when I tried it the first time. The option has always lead to crash >> here. > > Sorry, out of mind at that time, data_flush key words slip out of my eye... > > Could you please try below patch? > > From 65edbf14a198d0b50765e10340255e2071f7ae75 Mon Sep 17 00:00:00 2001 > From: Chao Yu <yuchao0@huawei.com> > Date: Mon, 1 Apr 2019 15:59:16 +0800 > Subject: [PATCH] f2fs: fix potential recursive call when enabling > data_flush > > Signed-off-by: Chao Yu <yuchao0@huawei.com> > --- > fs/f2fs/checkpoint.c | 6 ++---- > fs/f2fs/data.c | 3 ++- > 2 files changed, 4 insertions(+), 5 deletions(-) > > diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c > index a98e1b02279e..935ebdb9cf47 100644 > --- a/fs/f2fs/checkpoint.c > +++ b/fs/f2fs/checkpoint.c > @@ -1009,13 +1009,11 @@ int f2fs_sync_dirty_inodes(struct f2fs_sb_info *sbi, > enum inode_type type) > if (inode) { > unsigned long cur_ino = inode->i_ino; > > - if (is_dir) > - F2FS_I(inode)->cp_task = current; > + F2FS_I(inode)->cp_task = current; > > filemap_fdatawrite(inode->i_mapping); > > - if (is_dir) > - F2FS_I(inode)->cp_task = NULL; > + F2FS_I(inode)->cp_task = NULL; > > iput(inode); > /* We need to give cpu to another writers. */ > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c > index d87dfa5aa112..9d3c11e09a03 100644 > --- a/fs/f2fs/data.c > +++ b/fs/f2fs/data.c > @@ -2038,7 +2038,8 @@ static int __write_data_page(struct page *page, bool > *submitted, > } > > unlock_page(page); > - if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode)) > + if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) && > + !F2FS_I(inode)->cp_task) > f2fs_balance_fs(sbi, need_balance_fs); > > if (unlikely(f2fs_cp_error(sbi))) { > -- > 2.18.0.rc1 > > > >> >> 2019-03-30 8:29 GMT+01:00, Chao Yu <yuchao0@huawei.com>: >>> Oh, sorry, it's quite possible that bug is caused by data_flush, could >>> remove that mount option first? >>> >>> Thanks, >>> >>> On 2019/3/30 11:25, Chao Yu wrote: >>>> Hi Hagbard, >>>> >>>> Sorry for the delay. >>>> >>>> On 2019/3/27 21:59, Hagbard Celine wrote: >>>>> Hi, this is a long standing bug that I've hit before on older kernels, >>>>> but I was not able to get the syslog saved because of the nature of >>>>> the bug. This time I had booted form a pen-drive, and was able to save >>>>> the log to it's efi-partition. >>>> >>>> Now which version of kernel do you use? and do you remember what is >>>> your >>>> kernel version when this bug occured at first time? >>>> >> . >> > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-04-02 12:41 ` Hagbard Celine @ 2019-04-02 13:31 ` Chao Yu 2019-05-14 18:13 ` Hagbard Celine 0 siblings, 1 reply; 16+ messages in thread From: Chao Yu @ 2019-04-02 13:31 UTC (permalink / raw) To: Hagbard Celine, Chao Yu, linux-f2fs-devel On 2019-4-2 20:41, Hagbard Celine wrote: > That seems to have fixed it. No more errors in syslog after extracting > my stage3 tarball. Also ran a couple of kernel compiles on a partition > mounted with data_flush and system seems stable. Thanks a lot for your quick test. :) Thanks, > > 2019-04-01 10:05 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >> On 2019/3/31 2:54, Hagbard Celine wrote: >>> First, yes it is caused by data_flush, this is what I am trying to >>> report. Without that option there is no "stack guard page was hit" and >>> no "kernel stack overflow" and kernel is stable. >>> This time I was using kernel 5.0.3, as can be seen in the log in my first >>> mail. >>> I do not remember exactly what kernel version I tried the first time a >>> saw this bug, but I believe the mount option data_flush was just added >>> when I tried it the first time. The option has always lead to crash >>> here. >> >> Sorry, out of mind at that time, data_flush key words slip out of my eye... >> >> Could you please try below patch? >> >> From 65edbf14a198d0b50765e10340255e2071f7ae75 Mon Sep 17 00:00:00 2001 >> From: Chao Yu <yuchao0@huawei.com> >> Date: Mon, 1 Apr 2019 15:59:16 +0800 >> Subject: [PATCH] f2fs: fix potential recursive call when enabling >> data_flush >> >> Signed-off-by: Chao Yu <yuchao0@huawei.com> >> --- >> fs/f2fs/checkpoint.c | 6 ++---- >> fs/f2fs/data.c | 3 ++- >> 2 files changed, 4 insertions(+), 5 deletions(-) >> >> diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c >> index a98e1b02279e..935ebdb9cf47 100644 >> --- a/fs/f2fs/checkpoint.c >> +++ b/fs/f2fs/checkpoint.c >> @@ -1009,13 +1009,11 @@ int f2fs_sync_dirty_inodes(struct f2fs_sb_info *sbi, >> enum inode_type type) >> if (inode) { >> unsigned long cur_ino = inode->i_ino; >> >> - if (is_dir) >> - F2FS_I(inode)->cp_task = current; >> + F2FS_I(inode)->cp_task = current; >> >> filemap_fdatawrite(inode->i_mapping); >> >> - if (is_dir) >> - F2FS_I(inode)->cp_task = NULL; >> + F2FS_I(inode)->cp_task = NULL; >> >> iput(inode); >> /* We need to give cpu to another writers. */ >> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c >> index d87dfa5aa112..9d3c11e09a03 100644 >> --- a/fs/f2fs/data.c >> +++ b/fs/f2fs/data.c >> @@ -2038,7 +2038,8 @@ static int __write_data_page(struct page *page, bool >> *submitted, >> } >> >> unlock_page(page); >> - if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode)) >> + if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) && >> + !F2FS_I(inode)->cp_task) >> f2fs_balance_fs(sbi, need_balance_fs); >> >> if (unlikely(f2fs_cp_error(sbi))) { >> -- >> 2.18.0.rc1 >> >> >> >>> >>> 2019-03-30 8:29 GMT+01:00, Chao Yu <yuchao0@huawei.com>: >>>> Oh, sorry, it's quite possible that bug is caused by data_flush, could >>>> remove that mount option first? >>>> >>>> Thanks, >>>> >>>> On 2019/3/30 11:25, Chao Yu wrote: >>>>> Hi Hagbard, >>>>> >>>>> Sorry for the delay. >>>>> >>>>> On 2019/3/27 21:59, Hagbard Celine wrote: >>>>>> Hi, this is a long standing bug that I've hit before on older kernels, >>>>>> but I was not able to get the syslog saved because of the nature of >>>>>> the bug. This time I had booted form a pen-drive, and was able to save >>>>>> the log to it's efi-partition. >>>>> >>>>> Now which version of kernel do you use? and do you remember what is >>>>> your >>>>> kernel version when this bug occured at first time? >>>>> >>> . >>> >> > > > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-04-02 13:31 ` Chao Yu @ 2019-05-14 18:13 ` Hagbard Celine 2019-05-15 2:25 ` Chao Yu 0 siblings, 1 reply; 16+ messages in thread From: Hagbard Celine @ 2019-05-14 18:13 UTC (permalink / raw) To: Chao Yu; +Cc: linux-f2fs-devel 2019-04-02 15:31 GMT+02:00, Chao Yu <chao@kernel.org>: > On 2019-4-2 20:41, Hagbard Celine wrote: >> That seems to have fixed it. No more errors in syslog after extracting >> my stage3 tarball. Also ran a couple of kernel compiles on a partition >> mounted with data_flush and system seems stable. > > Thanks a lot for your quick test. :) My test might have been a little too quick, or I found another data_flush bug that behaves similar. While trying to find a faster method to trigger the "watchdog: BUG: soft lockup.. after heavy disk access" issue I reported in another mail; I got again "stack guard page was hit...", "kernel stack overflow (double-fault)..." which appear only when mounted with data_flush. What I did to trigger this time was I made two scripts: --BEGIN first script #!/bin/bash mkfs.f2fs -a 1 -f -i -l NVME_Exherbo-ts2 -o 7 /dev/nvme0n1p7 mount -o "rw,relatime,lazytime,background_gc=on,disable_ext_identify,discard,heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,data_flush,mode=adaptive,active_logs=6,whint_mode=fs-based,alloc_mode=default,fsync_mode=strict" /dev/nvme0n1p7 /mnt/exherbo-2tst/ mount -o "rw,relatime,lazytime,background_gc=on,disable_ext_identify,discard,heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,data_flush,mode=adaptive,active_logs=6,whint_mode=fs-based,alloc_mode=default,fsync_mode=strict" /dev/nvme0n1p8 /mnt/exherbo mkdir /mnt/exherbo-2tst/a mkdir /mnt/exherbo-2tst/b mkdir /mnt/exherbo-2tst/c mkdir /mnt/exherbo-2tst/d mkdir /mnt/exherbo-2tst/e mkdir /mnt/exherbo-2tst/f mkdir /mnt/exherbo-2tst/g mkdir /mnt/exherbo-2tst/h cd /mnt/exherbo cp -a . /mnt/exherbo-2tst/a cp -a . /mnt/exherbo-2tst/b cp -a . /mnt/exherbo-2tst/c cp -a . /mnt/exherbo-2tst/d cp -a . /mnt/exherbo-2tst/e cp -a . /mnt/exherbo-2tst/f cp -a . /mnt/exherbo-2tst/g cp -a . /mnt/exherbo-2tst/h cd df -h umount /mnt/exherbo umount /mnt/exherbo-2tst/ --END first script --BEGIN second script #!/bin/bash mount -o "rw,relatime,lazytime,background_gc=on,disable_ext_identify,discard,heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,data_flush,mode=adaptive,active_logs=6,whint_mode=fs-based,alloc_mode=default,fsync_mode=strict" /dev/nvme0n1p7 /mnt/exherbo-2tst/ mount -o "rw,relatime,lazytime,background_gc=on,disable_ext_identify,discard,heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,data_flush,mode=adaptive,active_logs=6,whint_mode=fs-based,alloc_mode=default,fsync_mode=strict" /dev/nvme0n1p8 /mnt/exherbo cd /mnt/exherbo-2tst/ rm -r * cd mkdir /mnt/exherbo-2tst/a mkdir /mnt/exherbo-2tst/b mkdir /mnt/exherbo-2tst/c mkdir /mnt/exherbo-2tst/d mkdir /mnt/exherbo-2tst/e mkdir /mnt/exherbo-2tst/f mkdir /mnt/exherbo-2tst/g mkdir /mnt/exherbo-2tst/h cd /mnt/exherbo cp -a . /mnt/exherbo-2tst/a cp -a . /mnt/exherbo-2tst/b cp -a . /mnt/exherbo-2tst/c cp -a . /mnt/exherbo-2tst/d cp -a . /mnt/exherbo-2tst/e cp -a . /mnt/exherbo-2tst/f cp -a . /mnt/exherbo-2tst/g cp -a . /mnt/exherbo-2tst/h cd df -h umount /mnt/exherbo umount /mnt/exherbo-2tst/ --END second script I ran these in order, with /dev/nvme0n1p8(sorce partition) formatted with same options as used on /dev/nvme0n1p7 in script and containing a exherbo-install of 17GB according to "df -h". When running the second script the bug triggers during copying every time. If I remove data_flush from mount options in scripts, bug does not trigger. Both partitions used are 128GB in size. This was on kernel-5.0.15 with "[PATCH] f2fs: fix potential recursive call when enabling data_flush" by Chao Yu Syslog follows: <6>[ 1020.669305] EXT4-fs (nvme0n1p2): mounted filesystem with ordered data mode. Opts: discard <5>[ 1400.426449] F2FS-fs (nvme0n1p7): Found nat_bits in checkpoint <5>[ 1400.487987] F2FS-fs (nvme0n1p7): Mounted with checkpoint version = 7f73ca21 <5>[ 1400.528024] F2FS-fs (nvme0n1p8): Mounted with checkpoint version = 7a6b5e4a <5>[ 1678.585243] F2FS-fs (nvme0n1p7): Found nat_bits in checkpoint <5>[ 1678.629891] F2FS-fs (nvme0n1p7): Mounted with checkpoint version = 7f73cba1 <5>[ 1678.664250] F2FS-fs (nvme0n1p8): Mounted with checkpoint version = 7a6b5e4e <0>[ 1810.859985] BUG: stack guard page was hit at 00000000973394e8 (stack is 000000005c69b096..000000005a84ab36) <4>[ 1810.864326] kernel stack overflow (double-fault): 0000 [#1] PREEMPT SMP PTI <4>[ 1810.868562] CPU: 0 PID: 2328 Comm: cp Not tainted 5.0.15-gentoo #2 <4>[ 1810.872779] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./C226 WS, BIOS P3.40 06/25/2018 <4>[ 1810.877036] RIP: 0010:__alloc_pages_nodemask+0x0/0x230 <4>[ 1810.881238] Code: 83 3c 24 08 0f 84 f7 fa ff ff 8b 4c 24 44 85 c9 0f 85 eb fa ff ff c7 44 24 38 00 00 00 00 e9 df f4 ff ff e8 b2 1c ee ff 66 90 <e8> fb 64 9f 00 41 56 41 55 41 54 55 53 48 83 ec 30 65 48 8b 04 25 <4>[ 1810.885794] RSP: 0018:ffffa8b7c2bb4000 EFLAGS: 00010006 <4>[ 1810.890226] RAX: 0000000000000000 RBX: ffff98307ec1b840 RCX: 0000000000000000 <4>[ 1810.894584] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000415200 <4>[ 1810.898789] RBP: 0000000000411200 R08: ffff98307d654840 R09: 0000000000000002 <4>[ 1810.902961] R10: 0000000000024638 R11: ffffffffffffffb8 R12: 0000000000411200 <4>[ 1810.907099] R13: 0000000000000000 R14: ffffc8b7bfc06a90 R15: 000000000000000c <4>[ 1810.911222] FS: 000078b3bca47740(0000) GS:ffff98309ec00000(0000) knlGS:0000000000000000 <4>[ 1810.915417] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4>[ 1810.919645] CR2: ffffa8b7c2bb3ff8 CR3: 00000003c4d2e002 CR4: 00000000003606f0 <4>[ 1810.923967] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 <4>[ 1810.928339] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 <4>[ 1810.932722] Call Trace: <4>[ 1810.937105] cache_grow_begin.constprop.72+0x77/0x460 <4>[ 1810.941576] kmem_cache_alloc+0x1ae/0x6b0 <4>[ 1810.946065] ? finish_wait+0x80/0x80 <4>[ 1810.950549] mempool_alloc+0x65/0x190 <4>[ 1810.955044] ? finish_wait+0x80/0x80 <4>[ 1810.959549] ? mempool_alloc+0x65/0x190 <4>[ 1810.964087] bvec_alloc+0x81/0xd0 <4>[ 1810.968634] bio_alloc_bioset+0x1a7/0x210 <4>[ 1810.973233] f2fs_submit_page_write+0x1bb/0x5e0 <4>[ 1810.977889] do_write_page+0x99/0x180 <4>[ 1810.982581] f2fs_outplace_write_data+0x55/0xe0 <4>[ 1810.987334] f2fs_do_write_data_page+0x588/0x940 <4>[ 1810.992080] __write_data_page+0x4f2/0x6f0 <4>[ 1810.996845] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.001616] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.006369] ? preempt_count_add+0x79/0xb0 <4>[ 1811.011132] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.015812] ? preempt_count_add+0x79/0xb0 <4>[ 1811.020401] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.024987] do_writepages+0x41/0xd0 <4>[ 1811.029546] ? preempt_count_add+0x79/0xb0 <4>[ 1811.034102] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.038650] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.043203] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.047772] ? si_meminfo+0x3f/0x60 <4>[ 1811.052339] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.056963] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.061614] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.066257] ? up_read+0x5/0x20 <4>[ 1811.070881] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1811.075546] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.080102] __write_data_page+0x5b7/0x6f0 <4>[ 1811.084566] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.089043] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.093494] ? preempt_count_add+0x79/0xb0 <4>[ 1811.097951] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.102384] ? preempt_count_add+0x79/0xb0 <4>[ 1811.106783] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.111157] do_writepages+0x41/0xd0 <4>[ 1811.115515] ? preempt_count_add+0x79/0xb0 <4>[ 1811.119896] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.124261] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.128607] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.132962] ? si_meminfo+0x3f/0x60 <4>[ 1811.137303] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.141674] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.146034] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.150416] ? up_read+0x5/0x20 <4>[ 1811.154764] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1811.159046] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.163235] __write_data_page+0x5b7/0x6f0 <4>[ 1811.167386] ? _raw_spin_unlock_irqrestore+0x34/0x40 <4>[ 1811.171485] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.175509] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.179333] ? __attach_extent_node+0x87/0xb0 <4>[ 1811.182968] ? preempt_count_add+0x79/0xb0 <4>[ 1811.186559] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.190152] do_writepages+0x41/0xd0 <4>[ 1811.193731] ? preempt_count_add+0x79/0xb0 <4>[ 1811.197315] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.200878] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.204378] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.207829] ? si_meminfo+0x3f/0x60 <4>[ 1811.211220] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.214633] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.218067] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.221447] ? up_read+0x5/0x20 <4>[ 1811.224791] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1811.228162] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.231527] __write_data_page+0x5b7/0x6f0 <4>[ 1811.234913] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.238324] ? preempt_count_add+0x79/0xb0 <4>[ 1811.241741] ? _raw_spin_lock+0x13/0x30 <4>[ 1811.245157] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.248589] ? preempt_count_add+0x79/0xb0 <4>[ 1811.252013] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.255405] ? preempt_count_add+0x79/0xb0 <4>[ 1811.258768] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.262129] do_writepages+0x41/0xd0 <4>[ 1811.265471] ? preempt_count_add+0x79/0xb0 <4>[ 1811.268815] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.272134] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.275438] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.278766] ? si_meminfo+0x3f/0x60 <4>[ 1811.282108] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.285487] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.288873] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.292252] ? up_read+0x5/0x20 <4>[ 1811.295636] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1811.299072] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.302508] __write_data_page+0x5b7/0x6f0 <4>[ 1811.305952] ? _raw_spin_unlock_irqrestore+0x34/0x40 <4>[ 1811.309413] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.312879] ? preempt_count_add+0x79/0xb0 <4>[ 1811.316344] ? _raw_spin_lock+0x13/0x30 <4>[ 1811.319797] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.323255] ? preempt_count_add+0x79/0xb0 <4>[ 1811.326702] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.330119] ? preempt_count_add+0x79/0xb0 <4>[ 1811.333506] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.336896] do_writepages+0x41/0xd0 <4>[ 1811.340268] ? preempt_count_add+0x79/0xb0 <4>[ 1811.343675] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.347079] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.350475] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.353873] ? si_meminfo+0x3f/0x60 <4>[ 1811.357256] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.360693] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.364144] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.367589] ? up_read+0x5/0x20 <4>[ 1811.371021] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1811.374486] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.377938] __write_data_page+0x5b7/0x6f0 <4>[ 1811.381395] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.384869] ? preempt_count_add+0x79/0xb0 <4>[ 1811.388334] ? _raw_spin_lock+0x13/0x30 <4>[ 1811.391776] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.395231] ? preempt_count_add+0x79/0xb0 <4>[ 1811.398672] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.402069] ? preempt_count_add+0x79/0xb0 <4>[ 1811.405454] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.408833] do_writepages+0x41/0xd0 <4>[ 1811.412198] ? preempt_count_add+0x79/0xb0 <4>[ 1811.415578] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.418931] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.422276] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.425640] ? si_meminfo+0x3f/0x60 <4>[ 1811.429006] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.432407] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.435807] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.439193] ? up_read+0x5/0x20 <4>[ 1811.442554] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1811.445958] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.449358] __write_data_page+0x5b7/0x6f0 <4>[ 1811.452749] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.456141] ? preempt_count_add+0x79/0xb0 <4>[ 1811.459533] ? _raw_spin_lock+0x13/0x30 <4>[ 1811.462907] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.466289] ? preempt_count_add+0x79/0xb0 <4>[ 1811.469672] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.473025] ? preempt_count_add+0x79/0xb0 <4>[ 1811.476353] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.479694] do_writepages+0x41/0xd0 <4>[ 1811.483052] ? preempt_count_add+0x79/0xb0 <4>[ 1811.486426] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.489773] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.493110] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.496473] ? si_meminfo+0x3f/0x60 <4>[ 1811.499831] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.503226] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.506629] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.510011] ? up_read+0x5/0x20 <4>[ 1811.513377] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1811.516779] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.520171] __write_data_page+0x5b7/0x6f0 <4>[ 1811.523563] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.526954] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.530336] ? preempt_count_add+0x79/0xb0 <4>[ 1811.533697] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.537022] ? preempt_count_add+0x79/0xb0 <4>[ 1811.540316] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.543603] do_writepages+0x41/0xd0 <4>[ 1811.546878] ? preempt_count_add+0x79/0xb0 <4>[ 1811.550159] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.553435] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.556728] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.560021] ? si_meminfo+0x3f/0x60 <4>[ 1811.563311] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.566624] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.569966] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.573316] ? up_read+0x5/0x20 <4>[ 1811.576650] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1811.580023] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.583389] __write_data_page+0x5b7/0x6f0 <4>[ 1811.586749] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.590126] ? __schedule+0x2cd/0x8b0 <4>[ 1811.593495] ? preempt_count_add+0x79/0xb0 <4>[ 1811.596853] ? _raw_spin_lock+0x13/0x30 <4>[ 1811.600197] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.603551] ? __attach_extent_node+0x87/0xb0 <4>[ 1811.606899] ? preempt_count_add+0x79/0xb0 <4>[ 1811.610240] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.613574] do_writepages+0x41/0xd0 <4>[ 1811.616895] ? preempt_count_add+0x79/0xb0 <4>[ 1811.620221] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.623548] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.626880] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.630222] ? si_meminfo+0x3f/0x60 <4>[ 1811.633552] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.636912] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.640296] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.643686] ? up_read+0x5/0x20 <4>[ 1811.647064] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1811.650478] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.653886] __write_data_page+0x5b7/0x6f0 <4>[ 1811.657290] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.660710] ? preempt_count_add+0x79/0xb0 <4>[ 1811.664130] ? _raw_spin_lock+0x13/0x30 <4>[ 1811.667526] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.670931] ? __switch_to_asm+0x34/0x70 <4>[ 1811.674335] ? _raw_spin_unlock_irq+0xd/0x30 <4>[ 1811.677731] do_writepages+0x41/0xd0 <4>[ 1811.681109] ? preempt_count_add+0x79/0xb0 <4>[ 1811.684503] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.687863] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.691207] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.694577] ? si_meminfo+0x3f/0x60 <4>[ 1811.697957] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.701365] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.704786] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.708199] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 1811.711663] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.715138] __write_data_page+0x5b7/0x6f0 <4>[ 1811.718619] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.722119] ? preempt_count_add+0x79/0xb0 <4>[ 1811.725608] ? _raw_spin_lock+0x13/0x30 <4>[ 1811.729078] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.732563] ? __switch_to_asm+0x34/0x70 <4>[ 1811.736039] ? _raw_spin_unlock_irq+0xd/0x30 <4>[ 1811.739508] do_writepages+0x41/0xd0 <4>[ 1811.742961] ? preempt_count_add+0x79/0xb0 <4>[ 1811.746431] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.749859] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.753284] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.756704] ? si_meminfo+0x3f/0x60 <4>[ 1811.760139] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.763626] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.767119] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.770607] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 1811.774124] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.777668] __write_data_page+0x5b7/0x6f0 <4>[ 1811.781223] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.784798] ? preempt_count_add+0x79/0xb0 <4>[ 1811.788373] ? _raw_spin_lock+0x13/0x30 <4>[ 1811.791941] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.795510] ? __switch_to_asm+0x34/0x70 <4>[ 1811.799076] ? _raw_spin_unlock_irq+0xd/0x30 <4>[ 1811.802645] do_writepages+0x41/0xd0 <4>[ 1811.806196] ? preempt_count_add+0x79/0xb0 <4>[ 1811.809742] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.813261] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.816762] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.820255] ? si_meminfo+0x3f/0x60 <4>[ 1811.823738] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.827253] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.830765] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.834273] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 1811.837804] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.841337] __write_data_page+0x5b7/0x6f0 <4>[ 1811.844883] ? _raw_spin_unlock_irqrestore+0x34/0x40 <4>[ 1811.848458] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.852063] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.855671] ? __switch_to_asm+0x34/0x70 <4>[ 1811.859276] ? _raw_spin_unlock_irq+0xd/0x30 <4>[ 1811.862886] do_writepages+0x41/0xd0 <4>[ 1811.866477] ? preempt_count_add+0x79/0xb0 <4>[ 1811.870074] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.873644] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.877202] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.880747] ? si_meminfo+0x3f/0x60 <4>[ 1811.884267] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.887815] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.891358] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.894890] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 1811.898458] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.902017] __write_data_page+0x5b7/0x6f0 <4>[ 1811.905579] ? _raw_spin_unlock_irqrestore+0x34/0x40 <4>[ 1811.909176] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.912776] ? perf_trace_sched_kthread_stop_ret+0xbf/0xe0 <4>[ 1811.916412] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.920079] ? __attach_extent_node+0x87/0xb0 <4>[ 1811.923763] ? preempt_count_add+0x79/0xb0 <4>[ 1811.927444] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.931125] do_writepages+0x41/0xd0 <4>[ 1811.934787] ? preempt_count_add+0x79/0xb0 <4>[ 1811.938449] ? preempt_count_sub+0x5/0xa0 <4>[ 1811.942087] ? _raw_spin_unlock+0x12/0x30 <4>[ 1811.945697] ? nr_blockdev_pages+0x59/0x70 <4>[ 1811.949288] ? si_meminfo+0x3f/0x60 <4>[ 1811.952865] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1811.956466] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1811.960062] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1811.963656] ? up_read+0x5/0x20 <4>[ 1811.967232] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1811.970843] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1811.974452] __write_data_page+0x5b7/0x6f0 <4>[ 1811.978066] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1811.981690] ? __x64_sys_sched_get_priority_min+0x1/0x40 <4>[ 1811.985372] ? preempt_count_add+0x79/0xb0 <4>[ 1811.989063] ? _raw_spin_lock+0x13/0x30 <4>[ 1811.992749] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1811.996451] ? preempt_count_sub+0x5/0xa0 <4>[ 1812.000141] ? _raw_spin_unlock+0x12/0x30 <4>[ 1812.003815] ? f2fs_inode_dirtied+0x64/0xf0 <4>[ 1812.007493] ? preempt_count_add+0x79/0xb0 <4>[ 1812.011155] ? preempt_count_sub+0x5/0xa0 <4>[ 1812.014775] do_writepages+0x41/0xd0 <4>[ 1812.018381] ? preempt_count_add+0x79/0xb0 <4>[ 1812.021992] ? preempt_count_sub+0x5/0xa0 <4>[ 1812.025571] ? _raw_spin_unlock+0x12/0x30 <4>[ 1812.029142] ? nr_blockdev_pages+0x59/0x70 <4>[ 1812.032708] ? si_meminfo+0x3f/0x60 <4>[ 1812.036258] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1812.039838] f2fs_sync_dirty_inodes+0xb1/0x1f0 <4>[ 1812.043411] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1812.046984] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1812.050552] f2fs_create+0x131/0x3e0 <4>[ 1812.054116] ? f2fs_lookup+0x20e/0x360 <4>[ 1812.057682] path_openat+0x11a8/0x1590 <4>[ 1812.061246] do_filp_open+0x93/0x100 <4>[ 1812.064794] ? preempt_count_sub+0x5/0xa0 <4>[ 1812.068344] do_sys_open+0x186/0x220 <4>[ 1812.071879] do_syscall_64+0x69/0x399 <4>[ 1812.075387] ? trace_hardirqs_off_thunk+0x1a/0x1c <4>[ 1812.078908] entry_SYSCALL_64_after_hwframe+0x49/0xbe <4>[ 1812.082445] RIP: 0033:0x78b3bcb37112 <4>[ 1812.085962] Code: 25 00 00 41 00 3d 00 00 41 00 74 4c 48 8d 05 d5 42 0d 00 8b 00 85 c0 75 6d 89 f2 b8 01 01 00 00 48 89 fe bf 9c ff ff ff 0f 05 <48> 3d 00 f0 ff ff 0f 87 a2 00 00 00 48 8b 4c 24 28 64 48 33 0c 25 <4>[ 1812.089905] RSP: 002b:00007ffe484d4290 EFLAGS: 00000246 ORIG_RAX: 0000000000000101 <4>[ 1812.093850] RAX: ffffffffffffffda RBX: 00007ffe484d49e0 RCX: 000078b3bcb37112 <4>[ 1812.097834] RDX: 00000000000000c1 RSI: 00005d8c1b59f690 RDI: 00000000ffffff9c <4>[ 1812.101846] RBP: 00007ffe484d46d0 R08: 00007ffe484d49e0 R09: 00007ffe484d49e0 <4>[ 1812.105875] R10: 0000000000000180 R11: 0000000000000246 R12: 00000000000001a4 <4>[ 1812.109922] R13: 0000000000000003 R14: 0000000000008000 R15: 0000000000000000 <4>[ 1812.113972] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc nls_cp437 uas usb_storage pl2303 sd_mod x86_pkg_temp_thermal kvm_intel iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich firewire_ohci igb firewire_core crc_itu_t ahci libahci dca processor_thermal_device intel_soc_dts_iosf int340x_thermal_zone pcc_cpufreq efivarfs <4>[ 1812.123203] ---[ end trace 58008f46c7a6df03 ]--- > > Thanks, > >> >> 2019-04-01 10:05 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >>> On 2019/3/31 2:54, Hagbard Celine wrote: >>>> First, yes it is caused by data_flush, this is what I am trying to >>>> report. Without that option there is no "stack guard page was hit" and >>>> no "kernel stack overflow" and kernel is stable. >>>> This time I was using kernel 5.0.3, as can be seen in the log in my >>>> first >>>> mail. >>>> I do not remember exactly what kernel version I tried the first time a >>>> saw this bug, but I believe the mount option data_flush was just added >>>> when I tried it the first time. The option has always lead to crash >>>> here. >>> >>> Sorry, out of mind at that time, data_flush key words slip out of my >>> eye... >>> >>> Could you please try below patch? >>> >>> From 65edbf14a198d0b50765e10340255e2071f7ae75 Mon Sep 17 00:00:00 2001 >>> From: Chao Yu <yuchao0@huawei.com> >>> Date: Mon, 1 Apr 2019 15:59:16 +0800 >>> Subject: [PATCH] f2fs: fix potential recursive call when enabling >>> data_flush >>> >>> Signed-off-by: Chao Yu <yuchao0@huawei.com> >>> --- >>> fs/f2fs/checkpoint.c | 6 ++---- >>> fs/f2fs/data.c | 3 ++- >>> 2 files changed, 4 insertions(+), 5 deletions(-) >>> >>> diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c >>> index a98e1b02279e..935ebdb9cf47 100644 >>> --- a/fs/f2fs/checkpoint.c >>> +++ b/fs/f2fs/checkpoint.c >>> @@ -1009,13 +1009,11 @@ int f2fs_sync_dirty_inodes(struct f2fs_sb_info >>> *sbi, >>> enum inode_type type) >>> if (inode) { >>> unsigned long cur_ino = inode->i_ino; >>> >>> - if (is_dir) >>> - F2FS_I(inode)->cp_task = current; >>> + F2FS_I(inode)->cp_task = current; >>> >>> filemap_fdatawrite(inode->i_mapping); >>> >>> - if (is_dir) >>> - F2FS_I(inode)->cp_task = NULL; >>> + F2FS_I(inode)->cp_task = NULL; >>> >>> iput(inode); >>> /* We need to give cpu to another writers. */ >>> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c >>> index d87dfa5aa112..9d3c11e09a03 100644 >>> --- a/fs/f2fs/data.c >>> +++ b/fs/f2fs/data.c >>> @@ -2038,7 +2038,8 @@ static int __write_data_page(struct page *page, >>> bool >>> *submitted, >>> } >>> >>> unlock_page(page); >>> - if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode)) >>> + if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) && >>> + !F2FS_I(inode)->cp_task) >>> f2fs_balance_fs(sbi, need_balance_fs); >>> >>> if (unlikely(f2fs_cp_error(sbi))) { >>> -- >>> 2.18.0.rc1 >>> >>> >>> >>>> >>>> 2019-03-30 8:29 GMT+01:00, Chao Yu <yuchao0@huawei.com>: >>>>> Oh, sorry, it's quite possible that bug is caused by data_flush, could >>>>> remove that mount option first? >>>>> >>>>> Thanks, >>>>> >>>>> On 2019/3/30 11:25, Chao Yu wrote: >>>>>> Hi Hagbard, >>>>>> >>>>>> Sorry for the delay. >>>>>> >>>>>> On 2019/3/27 21:59, Hagbard Celine wrote: >>>>>>> Hi, this is a long standing bug that I've hit before on older >>>>>>> kernels, >>>>>>> but I was not able to get the syslog saved because of the nature of >>>>>>> the bug. This time I had booted form a pen-drive, and was able to >>>>>>> save >>>>>>> the log to it's efi-partition. >>>>>> >>>>>> Now which version of kernel do you use? and do you remember what is >>>>>> your >>>>>> kernel version when this bug occured at first time? >>>>>> >>>> . >>>> >>> >> >> >> _______________________________________________ >> Linux-f2fs-devel mailing list >> Linux-f2fs-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel >> > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-05-14 18:13 ` Hagbard Celine @ 2019-05-15 2:25 ` Chao Yu 2019-05-15 8:03 ` Hagbard Celine 0 siblings, 1 reply; 16+ messages in thread From: Chao Yu @ 2019-05-15 2:25 UTC (permalink / raw) To: Hagbard Celine, Chao Yu; +Cc: linux-f2fs-devel On 2019/5/15 2:13, Hagbard Celine wrote: > 2019-04-02 15:31 GMT+02:00, Chao Yu <chao@kernel.org>: >> On 2019-4-2 20:41, Hagbard Celine wrote: >>> That seems to have fixed it. No more errors in syslog after extracting >>> my stage3 tarball. Also ran a couple of kernel compiles on a partition >>> mounted with data_flush and system seems stable. >> >> Thanks a lot for your quick test. :) > > My test might have been a little too quick, or I found another > data_flush bug that behaves similar. oops... >>>> >>>> - if (is_dir) >>>> - F2FS_I(inode)->cp_task = current; >>>> + F2FS_I(inode)->cp_task = current; If you're sure that this patch was applying before you test, I guess we need an extra barrier here to avoid out-of-order execution. smp_mb() >>>> >>>> filemap_fdatawrite(inode->i_mapping); >>>> >>>> - if (is_dir) >>>> - F2FS_I(inode)->cp_task = NULL; >>>> + F2FS_I(inode)->cp_task = NULL; Thanks, ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-05-15 2:25 ` Chao Yu @ 2019-05-15 8:03 ` Hagbard Celine 2019-05-15 8:13 ` Chao Yu 0 siblings, 1 reply; 16+ messages in thread From: Hagbard Celine @ 2019-05-15 8:03 UTC (permalink / raw) To: Chao Yu; +Cc: linux-f2fs-devel 2019-05-15 4:25 GMT+02:00, Chao Yu <yuchao0@huawei.com>: > On 2019/5/15 2:13, Hagbard Celine wrote: >> 2019-04-02 15:31 GMT+02:00, Chao Yu <chao@kernel.org>: >>> On 2019-4-2 20:41, Hagbard Celine wrote: >>>> That seems to have fixed it. No more errors in syslog after extracting >>>> my stage3 tarball. Also ran a couple of kernel compiles on a partition >>>> mounted with data_flush and system seems stable. >>> >>> Thanks a lot for your quick test. :) >> >> My test might have been a little too quick, or I found another >> data_flush bug that behaves similar. > > oops... > >>>>> >>>>> - if (is_dir) >>>>> - F2FS_I(inode)->cp_task = current; >>>>> + F2FS_I(inode)->cp_task = current; > > If you're sure that this patch was applying before you test, I guess we need > an > extra barrier here to avoid out-of-order execution. > > smp_mb() > >>>>> >>>>> filemap_fdatawrite(inode->i_mapping); >>>>> >>>>> - if (is_dir) >>>>> - F2FS_I(inode)->cp_task = NULL; >>>>> + F2FS_I(inode)->cp_task = NULL; > > Thanks, > If I did this correctly; it did not get rid of the stack overflow. Here is what I did: Added smb_mb() in checkpoint.c so the relevant part looks like this: unsigned long cur_ino = inode->i_ino; F2FS_I(inode)->cp_task = current; smp_mb(); filemap_fdatawrite(inode->i_mapping); F2FS_I(inode)->cp_task = NULL; iput(inode); Compiled, rebooted and ran my test-scripts again. And got this during copy-phase in second script: <5>[ 1215.731077] F2FS-fs (nvme0n1p7): Found nat_bits in checkpoint <5>[ 1215.812730] F2FS-fs (nvme0n1p7): Mounted with checkpoint version = 6319b5f3 <5>[ 1215.856781] F2FS-fs (nvme0n1p8): Mounted with checkpoint version = 7a6b5e6d <5>[ 1587.552859] F2FS-fs (nvme0n1p7): Found nat_bits in checkpoint <5>[ 1587.597483] F2FS-fs (nvme0n1p7): Mounted with checkpoint version = 6319b776 <5>[ 1587.630029] F2FS-fs (nvme0n1p8): Mounted with checkpoint version = 7a6b5e71 <0>[ 1720.608369] BUG: stack guard page was hit at 0000000033d16c42 (stack is 00000000ed3eabe7..00000000ffbe4ff0) <4>[ 1720.612537] kernel stack overflow (double-fault): 0000 [#1] PREEMPT SMP PTI <4>[ 1720.616750] CPU: 3 PID: 1982 Comm: kworker/u16:0 Not tainted 5.0.15-gentoo-f2fsbarr #3 <4>[ 1720.621057] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./C226 WS, BIOS P3.40 06/25/2018 <4>[ 1720.625465] Workqueue: writeback wb_workfn (flush-259:0) <4>[ 1720.629881] RIP: 0010:sched_clock_cpu+0x9/0xd0 <4>[ 1720.634283] Code: 08 e8 2b 9b f0 ff 48 89 03 48 03 05 a1 2e 62 01 48 2b 43 08 5b 48 89 05 8d 2e 62 01 c3 0f 1f 40 00 41 54 55 53 0f 1f 44 00 00 <e8> 02 9b f0 ff 48 03 05 7b 2e 62 01 48 89 c2 5b 48 89 d0 5d 41 5c <4>[ 1720.639109] RSP: 0018:ffffa661c0364000 EFLAGS: 00010046 <4>[ 1720.643893] RAX: 0000000000000003 RBX: ffff91cf5ecd54c0 RCX: ffffa661c03640f8 <4>[ 1720.648739] RDX: 0000000000000000 RSI: 0000000000000003 RDI: 0000000000000003 <4>[ 1720.653589] RBP: ffffffffb16485c0 R08: 0000000000000004 R09: 0000000000020e00 <4>[ 1720.658441] R10: ffffffffb16485c0 R11: 00000000000000cb R12: 0000000000000000 <4>[ 1720.663255] R13: ffffa661c03640f8 R14: 0000000000000046 R15: ffff91cf3c8a01c0 <4>[ 1720.668069] FS: 0000000000000000(0000) GS:ffff91cf5ecc0000(0000) knlGS:0000000000000000 <4>[ 1720.672971] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4>[ 1720.677885] CR2: ffffa661c0363ff8 CR3: 000000069bc0c003 CR4: 00000000003606e0 <4>[ 1720.682859] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 <4>[ 1720.687839] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 <4>[ 1720.692821] Call Trace: <4>[ 1720.697807] record_times+0x16/0xb0 <4>[ 1720.702795] psi_task_change+0xe9/0x210 <4>[ 1720.707795] activate_task+0xac/0x120 <4>[ 1720.712772] ttwu_do_activate+0x43/0x80 <4>[ 1720.717768] try_to_wake_up+0x1ef/0x510 <4>[ 1720.722547] __queue_work+0xf2/0x3f0 <4>[ 1720.727110] mod_delayed_work_on+0x59/0xa0 <4>[ 1720.731725] kblockd_mod_delayed_work_on+0x17/0x20 <4>[ 1720.736403] blk_mq_run_hw_queue+0x88/0xe0 <4>[ 1720.741094] blk_mq_flush_plug_list+0x19e/0x300 <4>[ 1720.745810] blk_flush_plug_list+0xd7/0x100 <4>[ 1720.750534] io_schedule_prepare+0x3c/0x40 <4>[ 1720.755171] io_schedule+0xb/0x40 <4>[ 1720.759697] __lock_page+0x13c/0x240 <4>[ 1720.764214] ? file_check_and_advance_wb_err+0xe0/0xe0 <4>[ 1720.768762] f2fs_write_cache_pages+0x253/0x450 <4>[ 1720.773297] ? kmem_cache_alloc+0x81/0x6b0 <4>[ 1720.777858] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 1720.782426] ? finish_wait+0x80/0x80 <4>[ 1720.787000] ? mempool_alloc+0x65/0x190 <4>[ 1720.791586] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1720.796213] ? preempt_count_add+0x79/0xb0 <4>[ 1720.800864] ? preempt_count_sub+0x5/0xa0 <4>[ 1720.805490] ? preempt_count_add+0x79/0xb0 <4>[ 1720.810088] ? _raw_spin_unlock+0x12/0x30 <4>[ 1720.814680] do_writepages+0x41/0xd0 <4>[ 1720.819149] ? preempt_count_add+0x79/0xb0 <4>[ 1720.823515] ? preempt_count_sub+0x5/0xa0 <4>[ 1720.827842] ? _raw_spin_unlock+0x12/0x30 <4>[ 1720.832150] ? nr_blockdev_pages+0x59/0x70 <4>[ 1720.836467] ? si_meminfo+0x3f/0x60 <4>[ 1720.840774] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1720.845097] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1720.849405] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1720.853705] ? up_read+0x5/0x20 <4>[ 1720.857997] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1720.862337] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1720.866668] __write_data_page+0x5b7/0x6f0 <4>[ 1720.871012] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1720.875370] ? account_entity_enqueue+0x4e/0x80 <4>[ 1720.879745] ? __switch_to_asm+0x40/0x70 <4>[ 1720.884094] ? __switch_to_asm+0x34/0x70 <4>[ 1720.888437] ? __switch_to_asm+0x40/0x70 <4>[ 1720.892727] ? __switch_to_asm+0x34/0x70 <4>[ 1720.896967] ? __switch_to_asm+0x40/0x70 <4>[ 1720.901195] ? preempt_count_add+0x79/0xb0 <4>[ 1720.905468] ? _raw_spin_lock+0x13/0x30 <4>[ 1720.909772] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1720.913993] ? __switch_to_asm+0x34/0x70 <4>[ 1720.918122] ? _raw_spin_unlock_irq+0xd/0x30 <4>[ 1720.922195] do_writepages+0x41/0xd0 <4>[ 1720.926166] ? preempt_count_add+0x79/0xb0 <4>[ 1720.930068] ? preempt_count_sub+0x5/0xa0 <4>[ 1720.933926] ? _raw_spin_unlock+0x12/0x30 <4>[ 1720.937584] ? nr_blockdev_pages+0x59/0x70 <4>[ 1720.941055] ? si_meminfo+0x3f/0x60 <4>[ 1720.944467] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1720.947901] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1720.951335] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1720.954790] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 1720.958268] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1720.961678] __write_data_page+0x5b7/0x6f0 <4>[ 1720.965050] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1720.968391] ? preempt_count_add+0x79/0xb0 <4>[ 1720.971723] ? _raw_spin_lock+0x13/0x30 <4>[ 1720.975074] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1720.978392] ? __switch_to_asm+0x40/0x70 <4>[ 1720.981712] ? __schedule+0x2cd/0x8b0 <4>[ 1720.985020] do_writepages+0x41/0xd0 <4>[ 1720.988323] ? preempt_schedule_common+0xd/0x80 <4>[ 1720.991652] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1720.995001] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1720.998353] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.001699] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 1721.005068] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1721.008422] __write_data_page+0x5b7/0x6f0 <4>[ 1721.011786] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1721.015164] ? preempt_count_add+0x79/0xb0 <4>[ 1721.018547] ? _raw_spin_lock+0x13/0x30 <4>[ 1721.021935] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1721.025351] ? preempt_count_add+0x79/0xb0 <4>[ 1721.028747] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.032100] ? preempt_count_add+0x79/0xb0 <4>[ 1721.035423] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.038743] do_writepages+0x41/0xd0 <4>[ 1721.042049] ? preempt_count_add+0x79/0xb0 <4>[ 1721.045372] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.048660] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.051932] ? nr_blockdev_pages+0x59/0x70 <4>[ 1721.055227] ? si_meminfo+0x3f/0x60 <4>[ 1721.058534] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1721.061874] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1721.065222] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.068562] ? up_read+0x5/0x20 <4>[ 1721.071909] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1721.075306] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1721.078700] __write_data_page+0x5b7/0x6f0 <4>[ 1721.082101] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1721.085507] ? retint_kernel+0x2d/0x2d <4>[ 1721.088900] ? preempt_count_add+0x79/0xb0 <4>[ 1721.092298] ? _raw_spin_lock+0x13/0x30 <4>[ 1721.095689] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1721.099085] do_writepages+0x41/0xd0 <4>[ 1721.102458] ? preempt_count_add+0x79/0xb0 <4>[ 1721.105844] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.109194] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.112525] ? nr_blockdev_pages+0x59/0x70 <4>[ 1721.115852] ? si_meminfo+0x3f/0x60 <4>[ 1721.119202] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1721.122617] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1721.126073] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.129561] ? up_read+0x5/0x20 <4>[ 1721.133033] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1721.136547] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1721.140054] __write_data_page+0x5b7/0x6f0 <4>[ 1721.143562] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1721.147088] ? __switch_to_asm+0x40/0x70 <4>[ 1721.150605] ? __switch_to_asm+0x34/0x70 <4>[ 1721.154085] ? __switch_to_asm+0x40/0x70 <4>[ 1721.157552] ? __switch_to_asm+0x34/0x70 <4>[ 1721.160992] ? __switch_to_asm+0x40/0x70 <4>[ 1721.164398] ? preempt_count_add+0x79/0xb0 <4>[ 1721.167806] ? _raw_spin_lock+0x13/0x30 <4>[ 1721.171197] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1721.174597] ? __switch_to_asm+0x34/0x70 <4>[ 1721.177989] ? _raw_spin_unlock_irq+0xd/0x30 <4>[ 1721.181369] do_writepages+0x41/0xd0 <4>[ 1721.184722] ? preempt_count_add+0x79/0xb0 <4>[ 1721.188086] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.191415] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.194726] ? nr_blockdev_pages+0x59/0x70 <4>[ 1721.198039] ? si_meminfo+0x3f/0x60 <4>[ 1721.201335] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1721.204659] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1721.207995] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.211338] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 1721.214704] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1721.218058] __write_data_page+0x5b7/0x6f0 <4>[ 1721.221412] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1721.224782] ? preempt_count_add+0x79/0xb0 <4>[ 1721.228130] ? _raw_spin_lock+0x13/0x30 <4>[ 1721.231453] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1721.234791] ? preempt_count_add+0x79/0xb0 <4>[ 1721.238118] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.241411] ? preempt_count_add+0x79/0xb0 <4>[ 1721.244684] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.247946] do_writepages+0x41/0xd0 <4>[ 1721.251192] ? preempt_count_add+0x79/0xb0 <4>[ 1721.254475] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.257756] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.261029] ? nr_blockdev_pages+0x59/0x70 <4>[ 1721.264308] ? si_meminfo+0x3f/0x60 <4>[ 1721.267574] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1721.270894] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1721.274227] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.277556] ? up_read+0x5/0x20 <4>[ 1721.280871] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1721.284217] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1721.287553] __write_data_page+0x5b7/0x6f0 <4>[ 1721.290896] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1721.294255] ? preempt_count_add+0x79/0xb0 <4>[ 1721.297602] ? _raw_spin_lock+0x13/0x30 <4>[ 1721.300930] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1721.304276] ? __attach_extent_node+0x87/0xb0 <4>[ 1721.307612] ? preempt_count_add+0x79/0xb0 <4>[ 1721.310939] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.314263] do_writepages+0x41/0xd0 <4>[ 1721.317577] ? preempt_count_add+0x79/0xb0 <4>[ 1721.320898] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.324188] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.327461] ? nr_blockdev_pages+0x59/0x70 <4>[ 1721.330757] ? si_meminfo+0x3f/0x60 <4>[ 1721.334056] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1721.337385] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1721.340718] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.344039] ? up_read+0x5/0x20 <4>[ 1721.347338] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1721.350674] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1721.354005] __write_data_page+0x5b7/0x6f0 <4>[ 1721.357331] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1721.360661] ? preempt_count_add+0x79/0xb0 <4>[ 1721.363987] ? _raw_spin_lock+0x13/0x30 <4>[ 1721.367298] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1721.370617] ? preempt_count_add+0x79/0xb0 <4>[ 1721.373972] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.377339] ? preempt_count_add+0x79/0xb0 <4>[ 1721.380720] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.384134] do_writepages+0x41/0xd0 <4>[ 1721.387537] ? preempt_count_add+0x79/0xb0 <4>[ 1721.390953] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.394340] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.397713] ? nr_blockdev_pages+0x59/0x70 <4>[ 1721.401088] ? si_meminfo+0x3f/0x60 <4>[ 1721.404451] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1721.407844] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1721.411262] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.414687] ? up_read+0x5/0x20 <4>[ 1721.418099] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1721.421551] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1721.424989] __write_data_page+0x5b7/0x6f0 <4>[ 1721.428427] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1721.431880] ? __x64_sys_sched_get_priority_min+0x3/0x40 <4>[ 1721.435348] ? preempt_count_add+0x79/0xb0 <4>[ 1721.438799] ? _raw_spin_lock+0x13/0x30 <4>[ 1721.442241] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1721.445688] ? __attach_extent_node+0x87/0xb0 <4>[ 1721.449130] ? preempt_count_add+0x79/0xb0 <4>[ 1721.452572] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.456005] do_writepages+0x41/0xd0 <4>[ 1721.459421] ? preempt_count_add+0x79/0xb0 <4>[ 1721.462873] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.466323] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.469766] ? nr_blockdev_pages+0x59/0x70 <4>[ 1721.473212] ? si_meminfo+0x3f/0x60 <4>[ 1721.476644] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1721.480131] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1721.483633] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.487129] ? up_read+0x5/0x20 <4>[ 1721.490612] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1721.494125] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1721.497629] __write_data_page+0x5b7/0x6f0 <4>[ 1721.501138] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1721.504662] ? preempt_count_add+0x79/0xb0 <4>[ 1721.508175] ? _raw_spin_lock+0x13/0x30 <4>[ 1721.511669] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1721.515177] ? __switch_to_asm+0x34/0x70 <4>[ 1721.518671] ? _raw_spin_unlock_irq+0xd/0x30 <4>[ 1721.522161] do_writepages+0x41/0xd0 <4>[ 1721.525643] ? preempt_count_add+0x79/0xb0 <4>[ 1721.529129] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.532587] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.536032] ? nr_blockdev_pages+0x59/0x70 <4>[ 1721.539482] ? si_meminfo+0x3f/0x60 <4>[ 1721.542921] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1721.546389] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1721.549883] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.553385] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 1721.556918] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1721.560449] __write_data_page+0x5b7/0x6f0 <4>[ 1721.563979] ? _raw_spin_unlock_irqrestore+0x34/0x40 <4>[ 1721.567530] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1721.571089] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1721.574636] ? __switch_to_asm+0x34/0x70 <4>[ 1721.578169] ? _raw_spin_unlock_irq+0xd/0x30 <4>[ 1721.581694] do_writepages+0x41/0xd0 <4>[ 1721.585209] ? preempt_count_add+0x79/0xb0 <4>[ 1721.588727] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.592211] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.595687] ? nr_blockdev_pages+0x59/0x70 <4>[ 1721.599160] ? si_meminfo+0x3f/0x60 <4>[ 1721.602615] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1721.606128] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1721.609666] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.613204] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 1721.616777] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1721.620346] __write_data_page+0x5b7/0x6f0 <4>[ 1721.623946] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1721.627569] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1721.631186] ? __switch_to_asm+0x34/0x70 <4>[ 1721.634797] ? _raw_spin_unlock_irq+0xd/0x30 <4>[ 1721.638404] do_writepages+0x41/0xd0 <4>[ 1721.641994] ? preempt_count_add+0x79/0xb0 <4>[ 1721.645593] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.649164] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.652711] ? nr_blockdev_pages+0x59/0x70 <4>[ 1721.656247] ? si_meminfo+0x3f/0x60 <4>[ 1721.659770] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1721.663313] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1721.666842] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.670347] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 1721.673883] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1721.677413] __write_data_page+0x5b7/0x6f0 <4>[ 1721.680944] ? _raw_spin_unlock_irqrestore+0x34/0x40 <4>[ 1721.684505] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1721.688076] ? __x64_sys_sched_get_priority_min+0xf/0x40 <4>[ 1721.691684] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1721.695294] ? preempt_count_add+0x79/0xb0 <4>[ 1721.698924] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.702560] do_writepages+0x41/0xd0 <4>[ 1721.706188] ? preempt_count_add+0x79/0xb0 <4>[ 1721.709828] ? preempt_count_sub+0x5/0xa0 <4>[ 1721.713431] ? _raw_spin_unlock+0x12/0x30 <4>[ 1721.717016] ? nr_blockdev_pages+0x59/0x70 <4>[ 1721.720602] ? si_meminfo+0x3f/0x60 <4>[ 1721.724175] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1721.727766] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1721.731347] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.734925] ? up_read+0x5/0x20 <4>[ 1721.738476] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 1721.742057] f2fs_balance_fs+0xe5/0x2c0 <4>[ 1721.745639] __write_data_page+0x5b7/0x6f0 <4>[ 1721.749222] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 1721.752816] ? perf_trace_sched_kthread_stop_ret+0xb3/0xe0 <4>[ 1721.756455] f2fs_write_data_pages+0x2ed/0x320 <4>[ 1721.760099] ? __switch_to_asm+0x40/0x70 <4>[ 1721.763738] ? __schedule+0x2cd/0x8b0 <4>[ 1721.767387] do_writepages+0x41/0xd0 <4>[ 1721.771034] ? preempt_schedule_common+0xd/0x80 <4>[ 1721.774708] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 1721.778397] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 1721.782078] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 1721.785749] ? retint_kernel+0x2d/0x2d <4>[ 1721.789417] f2fs_write_node_pages+0x51/0x220 <4>[ 1721.793085] ? trace_hardirqs_on_caller+0x32/0xf0 <4>[ 1721.796752] do_writepages+0x41/0xd0 <4>[ 1721.800409] ? retint_kernel+0x2d/0x2d <4>[ 1721.804053] __writeback_single_inode+0x3d/0x3d0 <4>[ 1721.807708] writeback_sb_inodes+0x1e8/0x410 <4>[ 1721.811369] __writeback_inodes_wb+0x5d/0xb0 <4>[ 1721.815028] wb_writeback+0x28f/0x340 <4>[ 1721.818685] wb_workfn+0x2d3/0x420 <4>[ 1721.822323] process_one_work+0x1a1/0x3d0 <4>[ 1721.825969] ? process_one_work+0x3d0/0x3d0 <4>[ 1721.829644] worker_thread+0x30/0x380 <4>[ 1721.833319] ? process_one_work+0x3d0/0x3d0 <4>[ 1721.837003] kthread+0x116/0x130 <4>[ 1721.840676] ? kthread_create_worker_on_cpu+0x70/0x70 <4>[ 1721.844386] ret_from_fork+0x3a/0x50 <4>[ 1721.848079] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc nls_cp437 sd_mod uas usb_storage pl2303 x86_pkg_temp_thermal kvm_intel iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich firewire_ohci firewire_core igb crc_itu_t ahci libahci dca processor_thermal_device intel_soc_dts_iosf int340x_thermal_zone pcc_cpufreq efivarfs <4>[ 1721.856652] ---[ end trace 64133457c53b53fc ]--- ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-05-15 8:03 ` Hagbard Celine @ 2019-05-15 8:13 ` Chao Yu 2019-05-15 16:50 ` Hagbard Celine 0 siblings, 1 reply; 16+ messages in thread From: Chao Yu @ 2019-05-15 8:13 UTC (permalink / raw) To: Hagbard Celine; +Cc: linux-f2fs-devel On 2019/5/15 16:03, Hagbard Celine wrote: > 2019-05-15 4:25 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >> On 2019/5/15 2:13, Hagbard Celine wrote: >>> 2019-04-02 15:31 GMT+02:00, Chao Yu <chao@kernel.org>: >>>> On 2019-4-2 20:41, Hagbard Celine wrote: >>>>> That seems to have fixed it. No more errors in syslog after extracting >>>>> my stage3 tarball. Also ran a couple of kernel compiles on a partition >>>>> mounted with data_flush and system seems stable. >>>> >>>> Thanks a lot for your quick test. :) >>> >>> My test might have been a little too quick, or I found another >>> data_flush bug that behaves similar. >> >> oops... >> >>>>>> >>>>>> - if (is_dir) >>>>>> - F2FS_I(inode)->cp_task = current; >>>>>> + F2FS_I(inode)->cp_task = current; >> >> If you're sure that this patch was applying before you test, I guess we need >> an >> extra barrier here to avoid out-of-order execution. >> >> smp_mb() >> >>>>>> >>>>>> filemap_fdatawrite(inode->i_mapping); >>>>>> >>>>>> - if (is_dir) >>>>>> - F2FS_I(inode)->cp_task = NULL; >>>>>> + F2FS_I(inode)->cp_task = NULL; >> >> Thanks, >> > If I did this correctly; it did not get rid of the stack overflow. > Here is what I did: > > Added smb_mb() in checkpoint.c so the relevant part looks like this: > > unsigned long cur_ino = inode->i_ino; > > F2FS_I(inode)->cp_task = current; > > smp_mb(); > > filemap_fdatawrite(inode->i_mapping); > > F2FS_I(inode)->cp_task = NULL; > > iput(inode); > > > Compiled, rebooted and ran my test-scripts again. And got this during > copy-phase in second script: It looks very easy to reproduce this bug, could you add log to track F2FS_I(inode)->cp_task's value: --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -1074,6 +1074,11 @@ int f2fs_sync_dirty_inodes(struct f2fs_sb_info *sbi, enum inode_type type) F2FS_I(inode)->cp_task = current; + smp_mb(); + + printk("sync_dirty_inodes: inode:%lu, cp_task:%p", + inode->i_ino, F2FS_I(inode)->cp_task); + filemap_fdatawrite(inode->i_mapping); F2FS_I(inode)->cp_task = NULL; diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index 88988241a0e8..84fbbd337c37 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -2392,6 +2392,10 @@ int f2fs_write_single_data_page(struct page *page, int *submitted, if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) && !F2FS_I(inode)->cp_task) { f2fs_submit_ipu_bio(sbi, bio, page); + + printk("write_data_page: inode:%lu, cp_task:%p", + inode->i_ino, F2FS_I(inode)->cp_task); + f2fs_balance_fs(sbi, need_balance_fs); } Thanks, > > <5>[ 1215.731077] F2FS-fs (nvme0n1p7): Found nat_bits in checkpoint > <5>[ 1215.812730] F2FS-fs (nvme0n1p7): Mounted with checkpoint version > = 6319b5f3 > <5>[ 1215.856781] F2FS-fs (nvme0n1p8): Mounted with checkpoint version > = 7a6b5e6d > <5>[ 1587.552859] F2FS-fs (nvme0n1p7): Found nat_bits in checkpoint > <5>[ 1587.597483] F2FS-fs (nvme0n1p7): Mounted with checkpoint version > = 6319b776 > <5>[ 1587.630029] F2FS-fs (nvme0n1p8): Mounted with checkpoint version > = 7a6b5e71 > <0>[ 1720.608369] BUG: stack guard page was hit at 0000000033d16c42 > (stack is 00000000ed3eabe7..00000000ffbe4ff0) > <4>[ 1720.612537] kernel stack overflow (double-fault): 0000 [#1] > PREEMPT SMP PTI > <4>[ 1720.616750] CPU: 3 PID: 1982 Comm: kworker/u16:0 Not tainted > 5.0.15-gentoo-f2fsbarr #3 > <4>[ 1720.621057] Hardware name: To Be Filled By O.E.M. To Be Filled > By O.E.M./C226 WS, BIOS P3.40 06/25/2018 > <4>[ 1720.625465] Workqueue: writeback wb_workfn (flush-259:0) > <4>[ 1720.629881] RIP: 0010:sched_clock_cpu+0x9/0xd0 > <4>[ 1720.634283] Code: 08 e8 2b 9b f0 ff 48 89 03 48 03 05 a1 2e 62 > 01 48 2b 43 08 5b 48 89 05 8d 2e 62 01 c3 0f 1f 40 00 41 54 55 53 0f > 1f 44 00 00 <e8> 02 9b f0 ff 48 03 05 7b 2e 62 01 48 89 c2 5b 48 89 d0 > 5d 41 5c > <4>[ 1720.639109] RSP: 0018:ffffa661c0364000 EFLAGS: 00010046 > <4>[ 1720.643893] RAX: 0000000000000003 RBX: ffff91cf5ecd54c0 RCX: > ffffa661c03640f8 > <4>[ 1720.648739] RDX: 0000000000000000 RSI: 0000000000000003 RDI: > 0000000000000003 > <4>[ 1720.653589] RBP: ffffffffb16485c0 R08: 0000000000000004 R09: > 0000000000020e00 > <4>[ 1720.658441] R10: ffffffffb16485c0 R11: 00000000000000cb R12: > 0000000000000000 > <4>[ 1720.663255] R13: ffffa661c03640f8 R14: 0000000000000046 R15: > ffff91cf3c8a01c0 > <4>[ 1720.668069] FS: 0000000000000000(0000) > GS:ffff91cf5ecc0000(0000) knlGS:0000000000000000 > <4>[ 1720.672971] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > <4>[ 1720.677885] CR2: ffffa661c0363ff8 CR3: 000000069bc0c003 CR4: > 00000000003606e0 > <4>[ 1720.682859] DR0: 0000000000000000 DR1: 0000000000000000 DR2: > 0000000000000000 > <4>[ 1720.687839] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: > 0000000000000400 > <4>[ 1720.692821] Call Trace: > <4>[ 1720.697807] record_times+0x16/0xb0 > <4>[ 1720.702795] psi_task_change+0xe9/0x210 > <4>[ 1720.707795] activate_task+0xac/0x120 > <4>[ 1720.712772] ttwu_do_activate+0x43/0x80 > <4>[ 1720.717768] try_to_wake_up+0x1ef/0x510 > <4>[ 1720.722547] __queue_work+0xf2/0x3f0 > <4>[ 1720.727110] mod_delayed_work_on+0x59/0xa0 > <4>[ 1720.731725] kblockd_mod_delayed_work_on+0x17/0x20 > <4>[ 1720.736403] blk_mq_run_hw_queue+0x88/0xe0 > <4>[ 1720.741094] blk_mq_flush_plug_list+0x19e/0x300 > <4>[ 1720.745810] blk_flush_plug_list+0xd7/0x100 > <4>[ 1720.750534] io_schedule_prepare+0x3c/0x40 > <4>[ 1720.755171] io_schedule+0xb/0x40 > <4>[ 1720.759697] __lock_page+0x13c/0x240 > <4>[ 1720.764214] ? file_check_and_advance_wb_err+0xe0/0xe0 > <4>[ 1720.768762] f2fs_write_cache_pages+0x253/0x450 > <4>[ 1720.773297] ? kmem_cache_alloc+0x81/0x6b0 > <4>[ 1720.777858] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 1720.782426] ? finish_wait+0x80/0x80 > <4>[ 1720.787000] ? mempool_alloc+0x65/0x190 > <4>[ 1720.791586] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1720.796213] ? preempt_count_add+0x79/0xb0 > <4>[ 1720.800864] ? preempt_count_sub+0x5/0xa0 > <4>[ 1720.805490] ? preempt_count_add+0x79/0xb0 > <4>[ 1720.810088] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1720.814680] do_writepages+0x41/0xd0 > <4>[ 1720.819149] ? preempt_count_add+0x79/0xb0 > <4>[ 1720.823515] ? preempt_count_sub+0x5/0xa0 > <4>[ 1720.827842] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1720.832150] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1720.836467] ? si_meminfo+0x3f/0x60 > <4>[ 1720.840774] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1720.845097] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1720.849405] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1720.853705] ? up_read+0x5/0x20 > <4>[ 1720.857997] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 1720.862337] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1720.866668] __write_data_page+0x5b7/0x6f0 > <4>[ 1720.871012] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1720.875370] ? account_entity_enqueue+0x4e/0x80 > <4>[ 1720.879745] ? __switch_to_asm+0x40/0x70 > <4>[ 1720.884094] ? __switch_to_asm+0x34/0x70 > <4>[ 1720.888437] ? __switch_to_asm+0x40/0x70 > <4>[ 1720.892727] ? __switch_to_asm+0x34/0x70 > <4>[ 1720.896967] ? __switch_to_asm+0x40/0x70 > <4>[ 1720.901195] ? preempt_count_add+0x79/0xb0 > <4>[ 1720.905468] ? _raw_spin_lock+0x13/0x30 > <4>[ 1720.909772] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1720.913993] ? __switch_to_asm+0x34/0x70 > <4>[ 1720.918122] ? _raw_spin_unlock_irq+0xd/0x30 > <4>[ 1720.922195] do_writepages+0x41/0xd0 > <4>[ 1720.926166] ? preempt_count_add+0x79/0xb0 > <4>[ 1720.930068] ? preempt_count_sub+0x5/0xa0 > <4>[ 1720.933926] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1720.937584] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1720.941055] ? si_meminfo+0x3f/0x60 > <4>[ 1720.944467] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1720.947901] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1720.951335] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1720.954790] ? _raw_spin_unlock_irqrestore+0x3d/0x40 > <4>[ 1720.958268] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1720.961678] __write_data_page+0x5b7/0x6f0 > <4>[ 1720.965050] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1720.968391] ? preempt_count_add+0x79/0xb0 > <4>[ 1720.971723] ? _raw_spin_lock+0x13/0x30 > <4>[ 1720.975074] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1720.978392] ? __switch_to_asm+0x40/0x70 > <4>[ 1720.981712] ? __schedule+0x2cd/0x8b0 > <4>[ 1720.985020] do_writepages+0x41/0xd0 > <4>[ 1720.988323] ? preempt_schedule_common+0xd/0x80 > <4>[ 1720.991652] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1720.995001] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1720.998353] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.001699] ? _raw_spin_unlock_irqrestore+0x3d/0x40 > <4>[ 1721.005068] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1721.008422] __write_data_page+0x5b7/0x6f0 > <4>[ 1721.011786] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1721.015164] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.018547] ? _raw_spin_lock+0x13/0x30 > <4>[ 1721.021935] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1721.025351] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.028747] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.032100] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.035423] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.038743] do_writepages+0x41/0xd0 > <4>[ 1721.042049] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.045372] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.048660] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.051932] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1721.055227] ? si_meminfo+0x3f/0x60 > <4>[ 1721.058534] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1721.061874] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1721.065222] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.068562] ? up_read+0x5/0x20 > <4>[ 1721.071909] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 1721.075306] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1721.078700] __write_data_page+0x5b7/0x6f0 > <4>[ 1721.082101] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1721.085507] ? retint_kernel+0x2d/0x2d > <4>[ 1721.088900] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.092298] ? _raw_spin_lock+0x13/0x30 > <4>[ 1721.095689] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1721.099085] do_writepages+0x41/0xd0 > <4>[ 1721.102458] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.105844] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.109194] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.112525] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1721.115852] ? si_meminfo+0x3f/0x60 > <4>[ 1721.119202] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1721.122617] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1721.126073] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.129561] ? up_read+0x5/0x20 > <4>[ 1721.133033] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 1721.136547] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1721.140054] __write_data_page+0x5b7/0x6f0 > <4>[ 1721.143562] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1721.147088] ? __switch_to_asm+0x40/0x70 > <4>[ 1721.150605] ? __switch_to_asm+0x34/0x70 > <4>[ 1721.154085] ? __switch_to_asm+0x40/0x70 > <4>[ 1721.157552] ? __switch_to_asm+0x34/0x70 > <4>[ 1721.160992] ? __switch_to_asm+0x40/0x70 > <4>[ 1721.164398] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.167806] ? _raw_spin_lock+0x13/0x30 > <4>[ 1721.171197] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1721.174597] ? __switch_to_asm+0x34/0x70 > <4>[ 1721.177989] ? _raw_spin_unlock_irq+0xd/0x30 > <4>[ 1721.181369] do_writepages+0x41/0xd0 > <4>[ 1721.184722] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.188086] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.191415] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.194726] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1721.198039] ? si_meminfo+0x3f/0x60 > <4>[ 1721.201335] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1721.204659] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1721.207995] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.211338] ? _raw_spin_unlock_irqrestore+0x3d/0x40 > <4>[ 1721.214704] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1721.218058] __write_data_page+0x5b7/0x6f0 > <4>[ 1721.221412] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1721.224782] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.228130] ? _raw_spin_lock+0x13/0x30 > <4>[ 1721.231453] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1721.234791] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.238118] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.241411] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.244684] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.247946] do_writepages+0x41/0xd0 > <4>[ 1721.251192] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.254475] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.257756] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.261029] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1721.264308] ? si_meminfo+0x3f/0x60 > <4>[ 1721.267574] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1721.270894] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1721.274227] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.277556] ? up_read+0x5/0x20 > <4>[ 1721.280871] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 1721.284217] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1721.287553] __write_data_page+0x5b7/0x6f0 > <4>[ 1721.290896] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1721.294255] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.297602] ? _raw_spin_lock+0x13/0x30 > <4>[ 1721.300930] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1721.304276] ? __attach_extent_node+0x87/0xb0 > <4>[ 1721.307612] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.310939] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.314263] do_writepages+0x41/0xd0 > <4>[ 1721.317577] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.320898] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.324188] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.327461] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1721.330757] ? si_meminfo+0x3f/0x60 > <4>[ 1721.334056] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1721.337385] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1721.340718] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.344039] ? up_read+0x5/0x20 > <4>[ 1721.347338] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 1721.350674] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1721.354005] __write_data_page+0x5b7/0x6f0 > <4>[ 1721.357331] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1721.360661] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.363987] ? _raw_spin_lock+0x13/0x30 > <4>[ 1721.367298] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1721.370617] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.373972] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.377339] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.380720] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.384134] do_writepages+0x41/0xd0 > <4>[ 1721.387537] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.390953] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.394340] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.397713] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1721.401088] ? si_meminfo+0x3f/0x60 > <4>[ 1721.404451] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1721.407844] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1721.411262] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.414687] ? up_read+0x5/0x20 > <4>[ 1721.418099] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 1721.421551] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1721.424989] __write_data_page+0x5b7/0x6f0 > <4>[ 1721.428427] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1721.431880] ? __x64_sys_sched_get_priority_min+0x3/0x40 > <4>[ 1721.435348] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.438799] ? _raw_spin_lock+0x13/0x30 > <4>[ 1721.442241] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1721.445688] ? __attach_extent_node+0x87/0xb0 > <4>[ 1721.449130] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.452572] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.456005] do_writepages+0x41/0xd0 > <4>[ 1721.459421] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.462873] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.466323] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.469766] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1721.473212] ? si_meminfo+0x3f/0x60 > <4>[ 1721.476644] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1721.480131] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1721.483633] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.487129] ? up_read+0x5/0x20 > <4>[ 1721.490612] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 1721.494125] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1721.497629] __write_data_page+0x5b7/0x6f0 > <4>[ 1721.501138] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1721.504662] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.508175] ? _raw_spin_lock+0x13/0x30 > <4>[ 1721.511669] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1721.515177] ? __switch_to_asm+0x34/0x70 > <4>[ 1721.518671] ? _raw_spin_unlock_irq+0xd/0x30 > <4>[ 1721.522161] do_writepages+0x41/0xd0 > <4>[ 1721.525643] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.529129] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.532587] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.536032] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1721.539482] ? si_meminfo+0x3f/0x60 > <4>[ 1721.542921] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1721.546389] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1721.549883] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.553385] ? _raw_spin_unlock_irqrestore+0x3d/0x40 > <4>[ 1721.556918] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1721.560449] __write_data_page+0x5b7/0x6f0 > <4>[ 1721.563979] ? _raw_spin_unlock_irqrestore+0x34/0x40 > <4>[ 1721.567530] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1721.571089] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1721.574636] ? __switch_to_asm+0x34/0x70 > <4>[ 1721.578169] ? _raw_spin_unlock_irq+0xd/0x30 > <4>[ 1721.581694] do_writepages+0x41/0xd0 > <4>[ 1721.585209] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.588727] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.592211] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.595687] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1721.599160] ? si_meminfo+0x3f/0x60 > <4>[ 1721.602615] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1721.606128] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1721.609666] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.613204] ? _raw_spin_unlock_irqrestore+0x3d/0x40 > <4>[ 1721.616777] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1721.620346] __write_data_page+0x5b7/0x6f0 > <4>[ 1721.623946] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1721.627569] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1721.631186] ? __switch_to_asm+0x34/0x70 > <4>[ 1721.634797] ? _raw_spin_unlock_irq+0xd/0x30 > <4>[ 1721.638404] do_writepages+0x41/0xd0 > <4>[ 1721.641994] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.645593] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.649164] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.652711] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1721.656247] ? si_meminfo+0x3f/0x60 > <4>[ 1721.659770] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1721.663313] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1721.666842] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.670347] ? _raw_spin_unlock_irqrestore+0x3d/0x40 > <4>[ 1721.673883] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1721.677413] __write_data_page+0x5b7/0x6f0 > <4>[ 1721.680944] ? _raw_spin_unlock_irqrestore+0x34/0x40 > <4>[ 1721.684505] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1721.688076] ? __x64_sys_sched_get_priority_min+0xf/0x40 > <4>[ 1721.691684] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1721.695294] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.698924] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.702560] do_writepages+0x41/0xd0 > <4>[ 1721.706188] ? preempt_count_add+0x79/0xb0 > <4>[ 1721.709828] ? preempt_count_sub+0x5/0xa0 > <4>[ 1721.713431] ? _raw_spin_unlock+0x12/0x30 > <4>[ 1721.717016] ? nr_blockdev_pages+0x59/0x70 > <4>[ 1721.720602] ? si_meminfo+0x3f/0x60 > <4>[ 1721.724175] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1721.727766] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1721.731347] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.734925] ? up_read+0x5/0x20 > <4>[ 1721.738476] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 1721.742057] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 1721.745639] __write_data_page+0x5b7/0x6f0 > <4>[ 1721.749222] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 1721.752816] ? perf_trace_sched_kthread_stop_ret+0xb3/0xe0 > <4>[ 1721.756455] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 1721.760099] ? __switch_to_asm+0x40/0x70 > <4>[ 1721.763738] ? __schedule+0x2cd/0x8b0 > <4>[ 1721.767387] do_writepages+0x41/0xd0 > <4>[ 1721.771034] ? preempt_schedule_common+0xd/0x80 > <4>[ 1721.774708] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 1721.778397] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 1721.782078] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 1721.785749] ? retint_kernel+0x2d/0x2d > <4>[ 1721.789417] f2fs_write_node_pages+0x51/0x220 > <4>[ 1721.793085] ? trace_hardirqs_on_caller+0x32/0xf0 > <4>[ 1721.796752] do_writepages+0x41/0xd0 > <4>[ 1721.800409] ? retint_kernel+0x2d/0x2d > <4>[ 1721.804053] __writeback_single_inode+0x3d/0x3d0 > <4>[ 1721.807708] writeback_sb_inodes+0x1e8/0x410 > <4>[ 1721.811369] __writeback_inodes_wb+0x5d/0xb0 > <4>[ 1721.815028] wb_writeback+0x28f/0x340 > <4>[ 1721.818685] wb_workfn+0x2d3/0x420 > <4>[ 1721.822323] process_one_work+0x1a1/0x3d0 > <4>[ 1721.825969] ? process_one_work+0x3d0/0x3d0 > <4>[ 1721.829644] worker_thread+0x30/0x380 > <4>[ 1721.833319] ? process_one_work+0x3d0/0x3d0 > <4>[ 1721.837003] kthread+0x116/0x130 > <4>[ 1721.840676] ? kthread_create_worker_on_cpu+0x70/0x70 > <4>[ 1721.844386] ret_from_fork+0x3a/0x50 > <4>[ 1721.848079] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc > nls_cp437 sd_mod uas usb_storage pl2303 x86_pkg_temp_thermal kvm_intel > iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich > firewire_ohci firewire_core igb crc_itu_t ahci libahci dca > processor_thermal_device intel_soc_dts_iosf int340x_thermal_zone > pcc_cpufreq efivarfs > <4>[ 1721.856652] ---[ end trace 64133457c53b53fc ]--- > . > ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-05-15 8:13 ` Chao Yu @ 2019-05-15 16:50 ` Hagbard Celine 2019-05-15 17:01 ` Hagbard Celine 0 siblings, 1 reply; 16+ messages in thread From: Hagbard Celine @ 2019-05-15 16:50 UTC (permalink / raw) To: Chao Yu; +Cc: linux-f2fs-devel 2019-05-15 10:13 GMT+02:00, Chao Yu <yuchao0@huawei.com>: > On 2019/5/15 16:03, Hagbard Celine wrote: >> 2019-05-15 4:25 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >>> On 2019/5/15 2:13, Hagbard Celine wrote: >>>> 2019-04-02 15:31 GMT+02:00, Chao Yu <chao@kernel.org>: >>>>> On 2019-4-2 20:41, Hagbard Celine wrote: >>>>>> That seems to have fixed it. No more errors in syslog after >>>>>> extracting >>>>>> my stage3 tarball. Also ran a couple of kernel compiles on a >>>>>> partition >>>>>> mounted with data_flush and system seems stable. >>>>> >>>>> Thanks a lot for your quick test. :) >>>> >>>> My test might have been a little too quick, or I found another >>>> data_flush bug that behaves similar. >>> >>> oops... >>> >>>>>>> >>>>>>> - if (is_dir) >>>>>>> - F2FS_I(inode)->cp_task = current; >>>>>>> + F2FS_I(inode)->cp_task = current; >>> >>> If you're sure that this patch was applying before you test, I guess we >>> need >>> an >>> extra barrier here to avoid out-of-order execution. >>> >>> smp_mb() >>> >>>>>>> >>>>>>> filemap_fdatawrite(inode->i_mapping); >>>>>>> >>>>>>> - if (is_dir) >>>>>>> - F2FS_I(inode)->cp_task = NULL; >>>>>>> + F2FS_I(inode)->cp_task = NULL; >>> >>> Thanks, >>> >> If I did this correctly; it did not get rid of the stack overflow. >> Here is what I did: >> >> Added smb_mb() in checkpoint.c so the relevant part looks like this: >> >> unsigned long cur_ino = inode->i_ino; >> >> F2FS_I(inode)->cp_task = current; >> >> smp_mb(); >> >> filemap_fdatawrite(inode->i_mapping); >> >> F2FS_I(inode)->cp_task = NULL; >> >> iput(inode); >> >> >> Compiled, rebooted and ran my test-scripts again. And got this during >> copy-phase in second script: > > It looks very easy to reproduce this bug, could you add log to track > F2FS_I(inode)->cp_task's value: That wasn't so easy, with all the logging from those prink the copy process would hang where it would oops without the printk's. I was able to reproduse the bug with one of the two printk's at a time only, and I had to disable syslog-ng and fcron for it not to hang even then. Followig is log from two runs, one with each of the printk's, hope it helps. --BEGIN log one <4>[ 593.806592] write_data_page: inode:710085, cp_task: (null) <4>[ 593.806688] write_data_page: inode:710110, cp_task: (null) <4>[ 593.808558] write_data_page: inode:710321, cp_task: (null) <4>[ 593.808575] write_data_page: inode:710325, cp_task: (null) <4>[ 593.808590] write_data_page: inode:710326, cp_task: (null) <4>[ 593.808606] write_data_page: inode:710332, cp_task: (null) <4>[ 593.966185] write_data_page: inode:721775, cp_task: (null) <4>[ 593.966203] write_data_page: inode:721776, cp_task: (null) <4>[ 593.966219] write_data_page: inode:721777, cp_task: (null) <4>[ 593.966235] write_data_page: inode:721778, cp_task: (null) <4>[ 593.966250] write_data_page: inode:721779, cp_task: (null) <4>[ 593.966266] write_data_page: inode:721780, cp_task: (null) <4>[ 593.966281] write_data_page: inode:721781, cp_task: (null) <4>[ 593.966296] write_data_page: inode:721782, cp_task: (null) <4>[ 593.966311] write_data_page: inode:721783, cp_task: (null) <4>[ 593.966327] write_data_page: inode:721784, cp_task: (null) <4>[ 593.966343] write_data_page: inode:721785, cp_task: (null) <4>[ 593.966359] write_data_page: inode:721786, cp_task: (null) <4>[ 593.966374] write_data_page: inode:721787, cp_task: (null) <4>[ 594.340072] write_data_page: inode:746183, cp_task: (null) <0>[ 594.923096] BUG: stack guard page was hit at 000000006e7354a5 (stack is 000000006445beb4..00000000988529ca) <0>[ 594.923108] BUG: stack guard page was hit at 00000000d2c9ec98 (stack is 00000000b417d4d3..000000001b88c4fe) <4>[ 594.926975] kernel stack overflow (double-fault): 0000 [#1] PREEMPT SMP PTI <4>[ 594.934772] CPU: 7 PID: 2158 Comm: cp Not tainted 5.0.15-gentoo-f2fslog_data_page #5 <4>[ 594.938757] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./C226 WS, BIOS P3.40 06/25/2018 <4>[ 594.942880] RIP: 0010:tag_pages_for_writeback+0x0/0x180 <4>[ 594.947036] Code: 30 48 83 ff 07 0f 86 eb fe ff ff 49 8b 56 40 49 03 56 38 48 d1 ea 49 89 d0 e9 6a fe ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 90 <e8> 2b 41 9f 00 41 54 48 83 c7 08 55 48 89 d5 53 48 83 ec 38 65 48 <4>[ 594.951623] RSP: 0018:ffffb3f0028dc000 EFLAGS: 00010246 <4>[ 594.956187] RAX: 0000000000000001 RBX: ffff9ea2d6bd7080 RCX: ffffb3f0028dc060 <4>[ 594.960816] RDX: 0007ffffffffffff RSI: 0000000000000000 RDI: ffff9ea2d6bd71e0 <4>[ 594.965375] RBP: ffffb3f0028dc1f0 R08: ffff9ea43eee0100 R09: 0000000000000000 <4>[ 594.969873] R10: ffffffff97e485c0 R11: 0000000000000001 R12: ffff9ea2d6bd71e0 <4>[ 594.974384] R13: 0000000000000009 R14: 0000000000000000 R15: ffff9ea43bfce000 <4>[ 594.978892] FS: 000073c62042f740(0000) GS:ffff9ea45edc0000(0000) knlGS:0000000000000000 <4>[ 594.983467] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4>[ 594.988055] CR2: ffffb3f0028dbff8 CR3: 0000000073aa8005 CR4: 00000000003606e0 <4>[ 594.992689] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 <4>[ 594.997310] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 <4>[ 595.001860] Call Trace: <4>[ 595.006323] f2fs_write_cache_pages+0x102/0x450 <4>[ 595.010829] ? kmem_cache_alloc+0x81/0x6b0 <4>[ 595.015219] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 595.019483] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.023749] ? __enqueue_entity+0x5/0x60 <4>[ 595.027995] ? enqueue_task_fair+0x4b1/0xb20 <4>[ 595.032232] do_writepages+0x41/0xd0 <4>[ 595.036481] ? preempt_count_add+0x79/0xb0 <4>[ 595.040754] ? preempt_count_sub+0x5/0xa0 <4>[ 595.045017] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.049304] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.053594] ? si_meminfo+0x3f/0x60 <4>[ 595.057892] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.062218] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.066549] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.070880] ? preempt_count_sub+0x5/0xa0 <4>[ 595.075077] ? _raw_spin_unlock_irqrestore+0x22/0x40 <4>[ 595.079177] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.083266] __write_data_page+0x1c8/0x6e0 <4>[ 595.087352] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.091460] ? preempt_count_add+0x79/0xb0 <4>[ 595.095565] ? _raw_spin_lock+0x13/0x30 <4>[ 595.099650] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.103769] ? __enqueue_entity+0x5/0x60 <4>[ 595.107922] ? enqueue_task_fair+0x4b1/0xb20 <4>[ 595.112117] do_writepages+0x41/0xd0 <4>[ 595.116347] ? preempt_count_add+0x79/0xb0 <4>[ 595.120630] ? preempt_count_sub+0x5/0xa0 <4>[ 595.124931] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.129268] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.133636] ? si_meminfo+0x3f/0x60 <4>[ 595.138039] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.142520] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.146992] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.151467] ? preempt_count_sub+0x5/0xa0 <4>[ 595.155947] ? _raw_spin_unlock_irqrestore+0x22/0x40 <4>[ 595.160466] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.164964] __write_data_page+0x1c8/0x6e0 <4>[ 595.169478] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.174019] ? preempt_count_add+0x79/0xb0 <4>[ 595.178545] ? _raw_spin_lock+0x13/0x30 <4>[ 595.183050] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.187613] ? __enqueue_entity+0x5/0x60 <4>[ 595.192150] ? enqueue_task_fair+0x4b1/0xb20 <4>[ 595.196678] do_writepages+0x41/0xd0 <4>[ 595.201226] ? preempt_count_add+0x79/0xb0 <4>[ 595.205819] ? preempt_count_sub+0x5/0xa0 <4>[ 595.210430] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.214933] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.219454] ? si_meminfo+0x3f/0x60 <4>[ 595.223862] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.228245] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.232543] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.236761] ? preempt_count_sub+0x5/0xa0 <4>[ 595.240792] ? _raw_spin_unlock_irqrestore+0x22/0x40 <4>[ 595.244662] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.248474] __write_data_page+0x1c8/0x6e0 <4>[ 595.252281] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.256076] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.259827] ? __enqueue_entity+0x5/0x60 <4>[ 595.263581] ? enqueue_task_fair+0x4b1/0xb20 <4>[ 595.267280] do_writepages+0x41/0xd0 <4>[ 595.270917] ? preempt_count_add+0x79/0xb0 <4>[ 595.274529] ? preempt_count_sub+0x5/0xa0 <4>[ 595.278101] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.281681] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.285226] ? si_meminfo+0x3f/0x60 <4>[ 595.288729] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.292263] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.295800] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.299328] ? preempt_count_sub+0x5/0xa0 <4>[ 595.302854] ? _raw_spin_unlock_irqrestore+0x22/0x40 <4>[ 595.306405] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.309936] __write_data_page+0x1c8/0x6e0 <4>[ 595.313471] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.317052] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.320637] ? __enqueue_entity+0x5/0x60 <4>[ 595.324215] ? enqueue_task_fair+0x4b1/0xb20 <4>[ 595.327802] do_writepages+0x41/0xd0 <4>[ 595.331372] ? preempt_count_add+0x79/0xb0 <4>[ 595.334951] ? preempt_count_sub+0x5/0xa0 <4>[ 595.338497] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.342030] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.345555] ? si_meminfo+0x3f/0x60 <4>[ 595.349054] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.352579] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.356099] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.359604] ? preempt_count_sub+0x5/0xa0 <4>[ 595.363110] ? _raw_spin_unlock_irqrestore+0x22/0x40 <4>[ 595.366639] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.370156] __write_data_page+0x1c8/0x6e0 <4>[ 595.373681] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.377218] ? __schedule+0x2cd/0x8b0 <4>[ 595.380747] ? __lock_page+0x155/0x240 <4>[ 595.384292] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.387868] ? __enqueue_entity+0x5/0x60 <4>[ 595.391445] ? enqueue_task_fair+0x4b1/0xb20 <4>[ 595.395030] do_writepages+0x41/0xd0 <4>[ 595.398597] ? preempt_count_add+0x79/0xb0 <4>[ 595.402169] ? preempt_count_sub+0x5/0xa0 <4>[ 595.405716] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.409250] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.412777] ? si_meminfo+0x3f/0x60 <4>[ 595.416275] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.419810] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.423335] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.426850] ? preempt_count_sub+0x5/0xa0 <4>[ 595.430370] ? _raw_spin_unlock_irqrestore+0x22/0x40 <4>[ 595.433904] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.437432] __write_data_page+0x1c8/0x6e0 <4>[ 595.440969] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.444511] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.448045] ? preempt_count_add+0x79/0xb0 <4>[ 595.451600] ? preempt_count_sub+0x5/0xa0 <4>[ 595.455136] ? preempt_count_add+0x79/0xb0 <4>[ 595.458661] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.462177] do_writepages+0x41/0xd0 <4>[ 595.465672] ? preempt_count_add+0x79/0xb0 <4>[ 595.469169] ? preempt_count_sub+0x5/0xa0 <4>[ 595.472643] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.476106] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.479555] ? si_meminfo+0x3f/0x60 <4>[ 595.482984] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.486441] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.489880] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.493322] ? up_read+0x5/0x20 <4>[ 595.496743] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 595.500205] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.503660] __write_data_page+0x1c8/0x6e0 <4>[ 595.507118] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.510615] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.514121] ? __enqueue_entity+0x5/0x60 <4>[ 595.517622] ? enqueue_task_fair+0x4b1/0xb20 <4>[ 595.521142] do_writepages+0x41/0xd0 <4>[ 595.524630] ? preempt_count_add+0x79/0xb0 <4>[ 595.528135] ? preempt_count_sub+0x5/0xa0 <4>[ 595.531601] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.535056] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.538498] ? si_meminfo+0x3f/0x60 <4>[ 595.541912] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.545355] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.548792] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.552235] ? preempt_count_sub+0x5/0xa0 <4>[ 595.555665] ? _raw_spin_unlock_irqrestore+0x22/0x40 <4>[ 595.559120] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.562568] __write_data_page+0x1c8/0x6e0 <4>[ 595.566033] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.569527] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.573022] ? __enqueue_entity+0x5/0x60 <4>[ 595.576517] ? enqueue_task_fair+0x4b1/0xb20 <4>[ 595.580021] do_writepages+0x41/0xd0 <4>[ 595.583507] ? preempt_count_add+0x79/0xb0 <4>[ 595.586997] ? preempt_count_sub+0x5/0xa0 <4>[ 595.590462] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.593923] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.597377] ? si_meminfo+0x3f/0x60 <4>[ 595.600796] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.604249] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.607691] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.611124] ? preempt_count_sub+0x5/0xa0 <4>[ 595.614569] ? _raw_spin_unlock_irqrestore+0x22/0x40 <4>[ 595.618024] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.621477] __write_data_page+0x1c8/0x6e0 <4>[ 595.624933] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.628396] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.631852] ? __enqueue_entity+0x5/0x60 <4>[ 595.635334] ? enqueue_task_fair+0x4b1/0xb20 <4>[ 595.638828] do_writepages+0x41/0xd0 <4>[ 595.642314] ? preempt_count_add+0x79/0xb0 <4>[ 595.645816] ? preempt_count_sub+0x5/0xa0 <4>[ 595.649282] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.652726] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.656177] ? si_meminfo+0x3f/0x60 <4>[ 595.659610] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.663057] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.666491] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.669919] ? preempt_count_sub+0x5/0xa0 <4>[ 595.673342] ? _raw_spin_unlock_irqrestore+0x22/0x40 <4>[ 595.676791] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.680243] __write_data_page+0x1c8/0x6e0 <4>[ 595.683722] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.687244] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.690766] ? __enqueue_entity+0x5/0x60 <4>[ 595.694278] ? enqueue_task_fair+0x4b1/0xb20 <4>[ 595.697809] do_writepages+0x41/0xd0 <4>[ 595.701342] ? preempt_count_add+0x79/0xb0 <4>[ 595.704903] ? preempt_count_sub+0x5/0xa0 <4>[ 595.708451] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.711975] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.715488] ? si_meminfo+0x3f/0x60 <4>[ 595.718983] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.722508] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.726031] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.729539] ? preempt_count_sub+0x5/0xa0 <4>[ 595.733044] ? _raw_spin_unlock_irqrestore+0x22/0x40 <4>[ 595.736581] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.740086] __write_data_page+0x1c8/0x6e0 <4>[ 595.743605] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.747130] ? _raw_spin_lock+0x13/0x30 <4>[ 595.750654] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.754190] ? __enqueue_entity+0x5/0x60 <4>[ 595.757724] ? enqueue_task_fair+0x4b1/0xb20 <4>[ 595.761288] do_writepages+0x41/0xd0 <4>[ 595.764855] ? preempt_count_add+0x79/0xb0 <4>[ 595.768435] ? preempt_count_sub+0x5/0xa0 <4>[ 595.772001] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.775544] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.779091] ? si_meminfo+0x3f/0x60 <4>[ 595.782617] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.786164] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.789709] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.793237] ? preempt_count_sub+0x5/0xa0 <4>[ 595.796755] ? _raw_spin_unlock_irqrestore+0x22/0x40 <4>[ 595.800292] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.803843] __write_data_page+0x1c8/0x6e0 <4>[ 595.807384] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.810944] ? __enqueue_entity+0x5/0x60 <4>[ 595.814491] ? preempt_count_add+0x79/0xb0 <4>[ 595.818039] ? _raw_spin_lock+0x13/0x30 <4>[ 595.821621] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.825212] ? preempt_count_sub+0x5/0xa0 <4>[ 595.828808] ? _raw_spin_unlock_irqrestore+0x22/0x40 <4>[ 595.832427] ? try_to_wake_up+0x52/0x510 <4>[ 595.836032] do_writepages+0x41/0xd0 <4>[ 595.839623] ? preempt_count_add+0x79/0xb0 <4>[ 595.843222] ? preempt_count_sub+0x5/0xa0 <4>[ 595.846785] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.850328] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.853856] ? si_meminfo+0x3f/0x60 <4>[ 595.857374] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.860909] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.864465] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.867996] ? up_read+0x5/0x20 <4>[ 595.871514] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 595.875066] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.878612] __write_data_page+0x1c8/0x6e0 <4>[ 595.882190] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.885795] ? preempt_count_add+0x79/0xb0 <4>[ 595.889400] ? _raw_spin_lock+0x13/0x30 <4>[ 595.893018] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.896624] ? mempool_alloc+0x65/0x190 <4>[ 595.900219] ? find_get_entry+0xdc/0x150 <4>[ 595.903828] do_writepages+0x41/0xd0 <4>[ 595.907416] ? preempt_count_add+0x79/0xb0 <4>[ 595.911003] ? preempt_count_sub+0x5/0xa0 <4>[ 595.914549] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.918081] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.921610] ? si_meminfo+0x3f/0x60 <4>[ 595.925130] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.928670] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.932210] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.935742] ? unlock_page+0x5/0x30 <4>[ 595.939260] f2fs_balance_fs+0xe5/0x2c0 <4>[ 595.942818] __write_data_page+0x1c8/0x6e0 <4>[ 595.946381] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 595.949960] ? find_busiest_group+0x40/0x580 <4>[ 595.953544] ? preempt_count_add+0x79/0xb0 <4>[ 595.957121] ? _raw_spin_lock+0x13/0x30 <4>[ 595.960683] f2fs_write_data_pages+0x2ed/0x320 <4>[ 595.964267] ? find_get_entry+0xdc/0x150 <4>[ 595.967849] do_writepages+0x41/0xd0 <4>[ 595.971404] ? preempt_count_add+0x79/0xb0 <4>[ 595.974962] ? preempt_count_sub+0x5/0xa0 <4>[ 595.978489] ? _raw_spin_unlock+0x12/0x30 <4>[ 595.982005] ? nr_blockdev_pages+0x59/0x70 <4>[ 595.985505] ? si_meminfo+0x3f/0x60 <4>[ 595.988990] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 595.992488] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 595.995995] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 595.999497] ? unlock_page+0x5/0x30 <4>[ 596.002978] f2fs_balance_fs+0xe5/0x2c0 <4>[ 596.006497] __write_data_page+0x1c8/0x6e0 <4>[ 596.010031] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 596.013590] ? stat_show+0xd01/0x1280 <4>[ 596.017136] ? up_read+0x5/0x20 <4>[ 596.020661] ? f2fs_getxattr+0x19e/0x3e0 <4>[ 596.024194] ? preempt_count_add+0x79/0xb0 <4>[ 596.027720] ? _raw_spin_lock+0x13/0x30 <4>[ 596.031237] f2fs_write_data_pages+0x2ed/0x320 <4>[ 596.034759] ? preempt_count_add+0x79/0xb0 <4>[ 596.038254] ? preempt_count_sub+0x5/0xa0 <4>[ 596.041726] ? _raw_spin_unlock+0x12/0x30 <4>[ 596.045174] ? f2fs_inode_dirtied+0x64/0xf0 <4>[ 596.048619] ? preempt_count_add+0x79/0xb0 <4>[ 596.052066] ? preempt_count_sub+0x5/0xa0 <4>[ 596.055503] do_writepages+0x41/0xd0 <4>[ 596.058916] ? preempt_count_add+0x79/0xb0 <4>[ 596.062334] ? preempt_count_sub+0x5/0xa0 <4>[ 596.065718] ? _raw_spin_unlock+0x12/0x30 <4>[ 596.069076] ? nr_blockdev_pages+0x59/0x70 <4>[ 596.072438] ? si_meminfo+0x3f/0x60 <4>[ 596.075787] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 596.079190] f2fs_sync_dirty_inodes+0xb7/0x1f0 <4>[ 596.082605] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 596.086024] f2fs_balance_fs+0xe5/0x2c0 <4>[ 596.089415] f2fs_create+0x131/0x3e0 <4>[ 596.092781] ? f2fs_lookup+0x20e/0x360 <4>[ 596.096141] path_openat+0x11a8/0x1590 <4>[ 596.099493] do_filp_open+0x93/0x100 <4>[ 596.102836] ? preempt_count_sub+0x5/0xa0 <4>[ 596.106176] do_sys_open+0x186/0x220 <4>[ 596.109496] do_syscall_64+0x69/0x399 <4>[ 596.112799] ? trace_hardirqs_off_thunk+0x1a/0x1c <4>[ 596.116121] entry_SYSCALL_64_after_hwframe+0x49/0xbe <4>[ 596.119459] RIP: 0033:0x73c62051f112 <4>[ 596.122789] Code: 25 00 00 41 00 3d 00 00 41 00 74 4c 48 8d 05 d5 42 0d 00 8b 00 85 c0 75 6d 89 f2 b8 01 01 00 00 48 89 fe bf 9c ff ff ff 0f 05 <48> 3d 00 f0 ff ff 0f 87 a2 00 00 00 48 8b 4c 24 28 64 48 33 0c 25 <4>[ 596.126545] RSP: 002b:00007ffe665bf1a0 EFLAGS: 00000246 ORIG_RAX: 0000000000000101 <4>[ 596.130299] RAX: ffffffffffffffda RBX: 00007ffe665bf8f0 RCX: 000073c62051f112 <4>[ 596.134110] RDX: 00000000000000c1 RSI: 00005670d5042690 RDI: 00000000ffffff9c <4>[ 596.137944] RBP: 00007ffe665bf5e0 R08: 00007ffe665bf8f0 R09: 00007ffe665bf8f0 <4>[ 596.141784] R10: 0000000000000180 R11: 0000000000000246 R12: 00000000000001a4 <4>[ 596.145664] R13: 0000000000000003 R14: 0000000000008000 R15: 0000000000000000 <4>[ 596.149536] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc nls_cp437 uas usb_storage pl2303 sd_mod x86_pkg_temp_thermal kvm_intel iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich firewire_ohci firewire_core igb ahci crc_itu_t processor_thermal_device libahci dca intel_soc_dts_iosf int340x_thermal_zone pcc_cpufreq efivarfs <4>[ 596.158385] ---[ end trace d508e01f5312dbde ]--- --END log one --BEGIN log two <4>[ 462.870628] sync_dirty_inodes: inode:624255, cp_task:000000006480c3a0 <4>[ 462.870633] sync_dirty_inodes: inode:624255, cp_task:000000002fea927f <4>[ 462.870634] sync_dirty_inodes: inode:624257, cp_task:000000006480c3a0 <4>[ 462.870638] sync_dirty_inodes: inode:624257, cp_task:000000002fea927f <4>[ 462.870660] sync_dirty_inodes: inode:623968, cp_task:000000002fea927f <4>[ 462.870665] sync_dirty_inodes: inode:623970, cp_task:000000002fea927f <4>[ 462.870679] sync_dirty_inodes: inode:623974, cp_task:000000002fea927f <4>[ 462.870681] sync_dirty_inodes: inode:623974, cp_task:000000006480c3a0 <4>[ 462.870691] sync_dirty_inodes: inode:623975, cp_task:000000002fea927f <4>[ 462.870696] sync_dirty_inodes: inode:623976, cp_task:000000002fea927f <4>[ 462.870697] sync_dirty_inodes: inode:623976, cp_task:000000006480c3a0 <4>[ 462.870711] sync_dirty_inodes: inode:623978, cp_task:000000006480c3a0 <4>[ 462.870715] sync_dirty_inodes: inode:623981, cp_task:000000006480c3a0 <4>[ 462.870717] sync_dirty_inodes: inode:623981, cp_task:000000002fea927f <4>[ 462.870719] sync_dirty_inodes: inode:623982, cp_task:000000006480c3a0 <4>[ 462.870721] sync_dirty_inodes: inode:623982, cp_task:000000002fea927f <4>[ 462.870754] sync_dirty_inodes: inode:623986, cp_task:000000002fea927f <4>[ 462.870755] sync_dirty_inodes: inode:623986, cp_task:000000006480c3a0 <4>[ 462.870774] sync_dirty_inodes: inode:623987, cp_task:000000006480c3a0 <4>[ 462.870775] sync_dirty_inodes: inode:623987, cp_task:000000002fea927f <4>[ 462.870783] sync_dirty_inodes: inode:623988, cp_task:000000002fea927f <4>[ 462.870788] sync_dirty_inodes: inode:623991, cp_task:000000002fea927f <4>[ 462.870794] sync_dirty_inodes: inode:623993, cp_task:000000002fea927f <4>[ 462.870795] sync_dirty_inodes: inode:623993, cp_task:000000006480c3a0 <4>[ 462.870801] sync_dirty_inodes: inode:623995, cp_task:000000006480c3a0 <4>[ 462.870808] sync_dirty_inodes: inode:623995, cp_task:000000002fea927f <4>[ 462.870810] sync_dirty_inodes: inode:623995, cp_task:000000002fea927f <4>[ 462.870812] sync_dirty_inodes: inode:623998, cp_task:000000002fea927f <4>[ 462.870813] sync_dirty_inodes: inode:623998, cp_task:000000006480c3a0 <4>[ 462.870848] sync_dirty_inodes: inode:624000, cp_task:000000002fea927f <4>[ 462.870852] sync_dirty_inodes: inode:624000, cp_task:000000006480c3a0 <4>[ 462.870853] sync_dirty_inodes: inode:624003, cp_task:000000002fea927f <4>[ 462.870856] sync_dirty_inodes: inode:624003, cp_task:000000006480c3a0 <4>[ 462.870858] sync_dirty_inodes: inode:624004, cp_task:000000002fea927f <4>[ 462.870860] sync_dirty_inodes: inode:624004, cp_task:000000006480c3a0 <4>[ 462.870862] sync_dirty_inodes: inode:624007, cp_task:000000002fea927f <4>[ 462.870863] sync_dirty_inodes: inode:624007, cp_task:000000006480c3a0 <4>[ 462.870875] sync_dirty_inodes: inode:624009, cp_task:000000006480c3a0 <4>[ 462.870902] sync_dirty_inodes: inode:624009, cp_task:000000002fea927f <4>[ 462.870905] sync_dirty_inodes: inode:624010, cp_task:000000002fea927f <4>[ 462.870908] sync_dirty_inodes: inode:624010, cp_task:000000006480c3a0 <4>[ 462.870913] sync_dirty_inodes: inode:624013, cp_task:000000002fea927f <4>[ 462.870917] sync_dirty_inodes: inode:624014, cp_task:000000002fea927f <4>[ 462.870921] sync_dirty_inodes: inode:624014, cp_task:000000006480c3a0 <4>[ 462.870947] sync_dirty_inodes: inode:624016, cp_task:000000006480c3a0 <4>[ 462.870951] sync_dirty_inodes: inode:624016, cp_task:000000002fea927f <4>[ 462.870953] sync_dirty_inodes: inode:624016, cp_task:000000002fea927f <4>[ 462.870955] sync_dirty_inodes: inode:624016, cp_task:000000002fea927f <4>[ 462.870956] sync_dirty_inodes: inode:624017, cp_task:000000002fea927f <4>[ 462.870958] sync_dirty_inodes: inode:624017, cp_task:000000006480c3a0 <4>[ 462.870961] sync_dirty_inodes: inode:624021, cp_task:000000002fea927f <4>[ 462.870962] sync_dirty_inodes: inode:624021, cp_task:000000006480c3a0 <4>[ 462.870968] sync_dirty_inodes: inode:624022, cp_task:000000006480c3a0 <4>[ 462.870970] sync_dirty_inodes: inode:624022, cp_task:000000002fea927f <4>[ 462.870975] sync_dirty_inodes: inode:624024, cp_task:000000006480c3a0 <4>[ 462.870976] sync_dirty_inodes: inode:624024, cp_task:000000002fea927f <4>[ 462.870989] sync_dirty_inodes: inode:624026, cp_task:000000002fea927f <4>[ 462.870990] sync_dirty_inodes: inode:624026, cp_task:000000006480c3a0 <0>[ 462.870997] BUG: stack guard page was hit at 0000000056e52d0a (stack is 00000000f6235c3f..00000000f20b69a0) <4>[ 462.876404] kernel stack overflow (double-fault): 0000 [#1] PREEMPT SMP PTI <4>[ 462.881794] CPU: 6 PID: 875 Comm: kworker/u16:9 Not tainted 5.0.15-gentoo-f2fslog_sync_dirty_inodes #5 <4>[ 462.887185] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./C226 WS, BIOS P3.40 06/25/2018 <4>[ 462.892563] Workqueue: writeback wb_workfn (flush-259:0) <4>[ 462.897877] RIP: 0010:tracer_hardirqs_off+0x4/0xf0 <4>[ 462.903078] Code: ee 48 0f 44 dd 4c 89 ff 48 89 da e8 d6 f9 ff ff 49 c7 45 20 00 00 00 00 f0 41 ff 4d 00 e9 67 ff ff ff 0f 1f 40 00 41 57 41 56 <41> 55 41 54 55 53 48 83 ec 08 65 44 8b 25 0a 9f 06 4b f6 05 53 12 <4>[ 462.908571] RSP: 0018:ffffaa364054c000 EFLAGS: 00010046 <4>[ 462.913921] RAX: 0000000000000000 RBX: ffff962b5b81ca80 RCX: ffffffffb6b66b50 <4>[ 462.919235] RDX: ffffffffb6b66b40 RSI: 0000000000000000 RDI: ffffffffb5059c74 <4>[ 462.924523] RBP: 0000000000411200 R08: 0000000000000002 R09: 0000000000000001 <4>[ 462.929772] R10: 0000000001cb731b R11: 0000000001cb731c R12: 0000000000411200 <4>[ 462.934889] R13: 0000000000000246 R14: ffffaa364054c0c8 R15: ffffffffb4f29520 <4>[ 462.939888] FS: 0000000000000000(0000) GS:ffff962b5ed80000(0000) knlGS:0000000000000000 <4>[ 462.944841] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4>[ 462.949681] CR2: ffffaa364054bff8 CR3: 000000072380c002 CR4: 00000000003606e0 <4>[ 462.954463] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 <4>[ 462.959191] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 <4>[ 462.963797] Call Trace: <4>[ 462.968280] ? finish_wait+0x80/0x80 <4>[ 462.972785] trace_hardirqs_off+0x2b/0xe0 <4>[ 462.977174] kmem_cache_alloc+0x44/0x6b0 <4>[ 462.981443] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 462.985703] ? finish_wait+0x80/0x80 <4>[ 462.989939] mempool_alloc+0x65/0x190 <4>[ 462.994143] ? preempt_count_add+0x79/0xb0 <4>[ 462.998379] bio_alloc_bioset+0x140/0x210 <4>[ 463.002628] f2fs_submit_page_write+0x1bb/0x5e0 <4>[ 463.006912] do_write_page+0x99/0x180 <4>[ 463.011201] f2fs_outplace_write_data+0x55/0xe0 <4>[ 463.015538] f2fs_do_write_data_page+0x588/0x940 <4>[ 463.019897] __write_data_page+0x4f2/0x6f0 <4>[ 463.024251] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.028616] ? pointer+0x110/0x3b0 <4>[ 463.032972] ? log_store+0xf5/0x260 <4>[ 463.037183] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.041277] ? __down_trylock_console_sem.isra.14+0x42/0x50 <4>[ 463.045404] ? vprintk_emit+0x28f/0x2a0 <4>[ 463.049517] do_writepages+0x41/0xd0 <4>[ 463.053634] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.057778] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.061911] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.066055] ? up_read+0x5/0x20 <4>[ 463.070224] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 463.074464] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.078742] __write_data_page+0x5b7/0x6f0 <4>[ 463.083072] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.087459] ? pointer+0x110/0x3b0 <4>[ 463.091880] ? log_store+0xf5/0x260 <4>[ 463.096333] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.100853] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.105422] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.109943] do_writepages+0x41/0xd0 <4>[ 463.114451] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.119000] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.123565] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.128116] ? up_read+0x5/0x20 <4>[ 463.132657] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 463.137238] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.141817] __write_data_page+0x5b7/0x6f0 <4>[ 463.146390] ? _raw_spin_unlock_irqrestore+0x34/0x40 <4>[ 463.151025] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.155642] ? pointer+0x110/0x3b0 <4>[ 463.160240] ? log_store+0xf5/0x260 <4>[ 463.164859] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.169533] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.174254] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.178838] do_writepages+0x41/0xd0 <4>[ 463.183323] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.187757] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.192105] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.196367] ? up_read+0x5/0x20 <4>[ 463.200592] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 463.204670] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.208563] __write_data_page+0x5b7/0x6f0 <4>[ 463.212425] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.216279] ? pointer+0x110/0x3b0 <4>[ 463.220107] ? log_store+0xf5/0x260 <4>[ 463.223927] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.227747] ? __down_trylock_console_sem.isra.14+0x42/0x50 <4>[ 463.231564] ? vprintk_emit+0x28f/0x2a0 <4>[ 463.235333] do_writepages+0x41/0xd0 <4>[ 463.239049] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.242783] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.246543] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.250250] ? up_read+0x5/0x20 <4>[ 463.253949] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 463.257687] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.261419] __write_data_page+0x5b7/0x6f0 <4>[ 463.265159] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.268916] ? pointer+0x110/0x3b0 <4>[ 463.272675] ? log_store+0xf5/0x260 <4>[ 463.276420] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.280192] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.283952] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.287664] do_writepages+0x41/0xd0 <4>[ 463.291360] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.295087] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.298810] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.302519] ? up_read+0x5/0x20 <4>[ 463.306207] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 463.309944] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.313670] __write_data_page+0x5b7/0x6f0 <4>[ 463.317379] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.321080] ? pointer+0x110/0x3b0 <4>[ 463.324773] ? log_store+0xf5/0x260 <4>[ 463.328444] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.332132] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.335820] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.339467] do_writepages+0x41/0xd0 <4>[ 463.343094] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.346740] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.350392] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.354029] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 463.357687] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.361353] __write_data_page+0x5b7/0x6f0 <4>[ 463.365020] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.368695] ? pointer+0x110/0x3b0 <4>[ 463.372399] ? log_store+0xf5/0x260 <4>[ 463.376091] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.379796] ? __down_trylock_console_sem.isra.14+0x42/0x50 <4>[ 463.383534] ? vprintk_emit+0x28f/0x2a0 <4>[ 463.387267] do_writepages+0x41/0xd0 <4>[ 463.390989] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.394729] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.398460] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.402181] ? up_read+0x5/0x20 <4>[ 463.405875] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 463.409590] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.413295] __write_data_page+0x5b7/0x6f0 <4>[ 463.417019] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.420752] ? pointer+0x110/0x3b0 <4>[ 463.424466] ? log_store+0xf5/0x260 <4>[ 463.428207] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.431971] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.435729] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.439437] do_writepages+0x41/0xd0 <4>[ 463.443137] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.446863] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.450581] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.454281] ? up_read+0x5/0x20 <4>[ 463.457962] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 463.461671] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.465368] __write_data_page+0x5b7/0x6f0 <4>[ 463.469084] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.472807] ? pointer+0x110/0x3b0 <4>[ 463.476499] ? log_store+0xf5/0x260 <4>[ 463.480154] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.483842] ? __down_trylock_console_sem.isra.14+0x42/0x50 <4>[ 463.487562] ? vprintk_emit+0x28f/0x2a0 <4>[ 463.491266] do_writepages+0x41/0xd0 <4>[ 463.494963] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.498684] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.502397] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.506098] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 463.509824] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.513542] __write_data_page+0x5b7/0x6f0 <4>[ 463.517257] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.521001] ? pointer+0x110/0x3b0 <4>[ 463.524731] ? log_store+0xf5/0x260 <4>[ 463.528437] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.532208] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.535979] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.539701] do_writepages+0x41/0xd0 <4>[ 463.543413] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.547150] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.550888] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.554611] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 463.558355] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.562091] __write_data_page+0x5b7/0x6f0 <4>[ 463.565824] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.569569] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.573317] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.577052] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.580741] do_writepages+0x41/0xd0 <4>[ 463.584464] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.588216] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.591965] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.595696] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 463.599454] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.603195] __write_data_page+0x5b7/0x6f0 <4>[ 463.606931] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.610679] ? pointer+0x110/0x3b0 <4>[ 463.614406] ? log_store+0xf5/0x260 <4>[ 463.618115] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.621860] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.625598] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.629284] do_writepages+0x41/0xd0 <4>[ 463.632926] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.636607] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.640281] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.643949] ? up_read+0x5/0x20 <4>[ 463.647596] ? f2fs_do_write_data_page+0x2cb/0x940 <4>[ 463.651274] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.654945] __write_data_page+0x5b7/0x6f0 <4>[ 463.658605] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.662281] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.665936] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.669582] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.673203] do_writepages+0x41/0xd0 <4>[ 463.676809] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.680434] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.684108] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.687775] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 463.691461] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.695135] __write_data_page+0x5b7/0x6f0 <4>[ 463.698813] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.702513] ? pointer+0x110/0x3b0 <4>[ 463.706187] ? log_store+0xf5/0x260 <4>[ 463.709837] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.713508] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.717171] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.720786] do_writepages+0x41/0xd0 <4>[ 463.724405] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.728046] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.731666] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.735258] ? _raw_spin_unlock_irqrestore+0x3d/0x40 <4>[ 463.738885] f2fs_balance_fs+0xe5/0x2c0 <4>[ 463.742501] __write_data_page+0x5b7/0x6f0 <4>[ 463.746115] f2fs_write_cache_pages+0x1e0/0x450 <4>[ 463.749744] ? pointer+0x110/0x3b0 <4>[ 463.753345] ? log_store+0xf5/0x260 <4>[ 463.756920] f2fs_write_data_pages+0x2ed/0x320 <4>[ 463.760517] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.764105] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.767643] do_writepages+0x41/0xd0 <4>[ 463.771191] __filemap_fdatawrite_range+0x81/0xb0 <4>[ 463.774760] f2fs_sync_dirty_inodes+0x1dd/0x200 <4>[ 463.778328] f2fs_balance_fs_bg+0x2a7/0x2c0 <4>[ 463.781928] ? trace_hardirqs_on+0x2c/0xe0 <4>[ 463.785524] ? retint_kernel+0x2d/0x2d <4>[ 463.789100] f2fs_write_node_pages+0x51/0x220 <4>[ 463.792687] ? trace_hardirqs_on_caller+0x32/0xf0 <4>[ 463.796285] do_writepages+0x41/0xd0 <4>[ 463.799860] ? retint_kernel+0x2d/0x2d <4>[ 463.803417] __writeback_single_inode+0x3d/0x3d0 <4>[ 463.806999] writeback_sb_inodes+0x1e8/0x410 <4>[ 463.810573] __writeback_inodes_wb+0x5d/0xb0 <4>[ 463.814158] wb_writeback+0x28f/0x340 <4>[ 463.817730] wb_workfn+0x2d3/0x420 <4>[ 463.821328] process_one_work+0x1a1/0x3d0 <4>[ 463.824935] worker_thread+0x30/0x380 <4>[ 463.828520] ? process_one_work+0x3d0/0x3d0 <4>[ 463.832105] kthread+0x116/0x130 <4>[ 463.835675] ? kthread_create_worker_on_cpu+0x70/0x70 <4>[ 463.839278] ret_from_fork+0x3a/0x50 <4>[ 463.842855] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc nls_cp437 uas usb_storage pl2303 sd_mod x86_pkg_temp_thermal kvm_intel iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich firewire_ohci firewire_core igb crc_itu_t dca ahci libahci processor_thermal_device intel_soc_dts_iosf int340x_thermal_zone pcc_cpufreq efivarfs <4>[ 463.851196] ---[ end trace b6d0a43e90218d45 ]--- --END log two > > --- a/fs/f2fs/checkpoint.c > +++ b/fs/f2fs/checkpoint.c > @@ -1074,6 +1074,11 @@ int f2fs_sync_dirty_inodes(struct f2fs_sb_info *sbi, > enum > inode_type type) > > F2FS_I(inode)->cp_task = current; > > + smp_mb(); > + > + printk("sync_dirty_inodes: inode:%lu, cp_task:%p", > + inode->i_ino, F2FS_I(inode)->cp_task); > + > filemap_fdatawrite(inode->i_mapping); > > F2FS_I(inode)->cp_task = NULL; > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c > index 88988241a0e8..84fbbd337c37 100644 > --- a/fs/f2fs/data.c > +++ b/fs/f2fs/data.c > @@ -2392,6 +2392,10 @@ int f2fs_write_single_data_page(struct page *page, > int > *submitted, > if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) && > !F2FS_I(inode)->cp_task) { > f2fs_submit_ipu_bio(sbi, bio, page); > + > + printk("write_data_page: inode:%lu, cp_task:%p", > + inode->i_ino, F2FS_I(inode)->cp_task); > + > f2fs_balance_fs(sbi, need_balance_fs); > } > > Thanks, ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-05-15 16:50 ` Hagbard Celine @ 2019-05-15 17:01 ` Hagbard Celine 2019-05-20 9:37 ` Chao Yu 0 siblings, 1 reply; 16+ messages in thread From: Hagbard Celine @ 2019-05-15 17:01 UTC (permalink / raw) To: Chao Yu; +Cc: linux-f2fs-devel 2019-05-15 18:50 GMT+02:00, Hagbard Celine <hagbardcelin@gmail.com>: > 2019-05-15 10:13 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >> On 2019/5/15 16:03, Hagbard Celine wrote: >>> 2019-05-15 4:25 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >>>> On 2019/5/15 2:13, Hagbard Celine wrote: >>>>> 2019-04-02 15:31 GMT+02:00, Chao Yu <chao@kernel.org>: >>>>>> On 2019-4-2 20:41, Hagbard Celine wrote: >>>>>>> That seems to have fixed it. No more errors in syslog after >>>>>>> extracting >>>>>>> my stage3 tarball. Also ran a couple of kernel compiles on a >>>>>>> partition >>>>>>> mounted with data_flush and system seems stable. >>>>>> >>>>>> Thanks a lot for your quick test. :) >>>>> >>>>> My test might have been a little too quick, or I found another >>>>> data_flush bug that behaves similar. >>>> >>>> oops... >>>> >>>>>>>> >>>>>>>> - if (is_dir) >>>>>>>> - F2FS_I(inode)->cp_task = current; >>>>>>>> + F2FS_I(inode)->cp_task = current; >>>> >>>> If you're sure that this patch was applying before you test, I guess we >>>> need >>>> an >>>> extra barrier here to avoid out-of-order execution. >>>> >>>> smp_mb() >>>> >>>>>>>> >>>>>>>> filemap_fdatawrite(inode->i_mapping); >>>>>>>> >>>>>>>> - if (is_dir) >>>>>>>> - F2FS_I(inode)->cp_task = NULL; >>>>>>>> + F2FS_I(inode)->cp_task = NULL; >>>> >>>> Thanks, >>>> >>> If I did this correctly; it did not get rid of the stack overflow. >>> Here is what I did: >>> >>> Added smb_mb() in checkpoint.c so the relevant part looks like this: >>> >>> unsigned long cur_ino = inode->i_ino; >>> >>> F2FS_I(inode)->cp_task = current; >>> >>> smp_mb(); >>> >>> filemap_fdatawrite(inode->i_mapping); >>> >>> F2FS_I(inode)->cp_task = NULL; >>> >>> iput(inode); >>> >>> >>> Compiled, rebooted and ran my test-scripts again. And got this during >>> copy-phase in second script: >> >> It looks very easy to reproduce this bug, could you add log to track >> F2FS_I(inode)->cp_task's value: > That wasn't so easy, with all the logging from those prink the copy > process would hang where it would oops without the printk's. Forgot in last mail that I actually had log from hang with both printk enabled also: [ 194.681126] sync_dirty_inodes: inode:590309, cp_task:0000000013327ef9 [ 194.682258] sync_dirty_inodes: inode:590301, cp_task:0000000013327ef9 [ 194.682665] sync_dirty_inodes: inode:590311, cp_task:0000000013327ef9 [ 194.682703] sync_dirty_inodes: inode:590312, cp_task:0000000013327ef9 [ 194.682791] sync_dirty_inodes: inode:590313, cp_task:0000000013327ef9 [ 194.683566] sync_dirty_inodes: inode:590314, cp_task:0000000013327ef9 [ 194.683669] sync_dirty_inodes: inode:590315, cp_task:0000000013327ef9 [ 194.684829] sync_dirty_inodes: inode:590316, cp_task:0000000013327ef9 [ 194.712860] sync_dirty_inodes: inode:590317, cp_task:0000000013327ef9 [ 194.712908] sync_dirty_inodes: inode:590310, cp_task:0000000013327ef9 [ 194.713094] sync_dirty_inodes: inode:590319, cp_task:0000000013327ef9 [ 194.713348] sync_dirty_inodes: inode:590320, cp_task:0000000013327ef9 [ 194.713384] sync_dirty_inodes: inode:590321, cp_task:0000000013327ef9 [ 194.714634] sync_dirty_inodes: inode:590322, cp_task:0000000013327ef9 [ 194.715349] sync_dirty_inodes: inode:590323, cp_task:0000000013327ef9 [ 194.715381] sync_dirty_inodes: inode:590324, cp_task:0000000013327ef9 [ 194.718592] sync_dirty_inodes: inode:590326, cp_task:0000000013327ef9 [ 194.719217] sync_dirty_inodes: inode:590327, cp_task:0000000013327ef9 [ 194.719354] sync_dirty_inodes: inode:590328, cp_task:0000000013327ef9 [ 194.719903] sync_dirty_inodes: inode:590329, cp_task:0000000013327ef9 [ 194.720859] sync_dirty_inodes: inode:590521, cp_task:0000000013327ef9 [ 194.720868] sync_dirty_inodes: inode:590300, cp_task:0000000013327ef9 [ 194.720985] sync_dirty_inodes: inode:590523, cp_task:0000000013327ef9 [ 194.738075] sync_dirty_inodes: inode:591528, cp_task:0000000013327ef9 [ 194.738168] sync_dirty_inodes: inode:591529, cp_task:0000000013327ef9 [ 194.738190] sync_dirty_inodes: inode:591533, cp_task:0000000013327ef9 [ 194.738263] sync_dirty_inodes: inode:591537, cp_task:0000000013327ef9 [ 194.738275] sync_dirty_inodes: inode:591538, cp_task:0000000013327ef9 [ 194.738307] sync_dirty_inodes: inode:591540, cp_task:0000000013327ef9 [ 194.738312] sync_dirty_inodes: inode:591542, cp_task:0000000013327ef9 [ 194.759368] sync_dirty_inodes: inode:593095, cp_task:0000000013327ef9 [ 194.759587] sync_dirty_inodes: inode:593351, cp_task:0000000013327ef9 [ 194.759797] sync_dirty_inodes: inode:593352, cp_task:0000000013327ef9 [ 194.760555] sync_dirty_inodes: inode:593353, cp_task:0000000013327ef9 [ 194.760562] sync_dirty_inodes: inode:593354, cp_task:0000000013327ef9 [ 194.761168] sync_dirty_inodes: inode:593355, cp_task:0000000013327ef9 [ 194.761207] sync_dirty_inodes: inode:593356, cp_task:0000000013327ef9 [ 194.761234] sync_dirty_inodes: inode:593357, cp_task:0000000013327ef9 [ 194.761251] sync_dirty_inodes: inode:593358, cp_task:0000000013327ef9 [ 194.761285] sync_dirty_inodes: inode:593360, cp_task:0000000013327ef9 [ 194.761290] sync_dirty_inodes: inode:593361, cp_task:0000000013327ef9 [ 194.763583] sync_dirty_inodes: inode:593363, cp_task:0000000013327ef9 [ 194.763845] sync_dirty_inodes: inode:593362, cp_task:0000000013327ef9 [ 194.763891] sync_dirty_inodes: inode:593367, cp_task:0000000013327ef9 [ 194.763992] sync_dirty_inodes: inode:593368, cp_task:0000000013327ef9 [ 194.764073] sync_dirty_inodes: inode:593369, cp_task:0000000013327ef9 [ 194.764090] sync_dirty_inodes: inode:593370, cp_task:0000000013327ef9 [ 194.764165] sync_dirty_inodes: inode:593371, cp_task:0000000013327ef9 [ 194.764185] sync_dirty_inodes: inode:593372, cp_task:0000000013327ef9 [ 194.764214] sync_dirty_inodes: inode:593365, cp_task:0000000013327ef9 [ 194.764323] sync_dirty_inodes: inode:593374, cp_task:0000000013327ef9 [ 194.764419] sync_dirty_inodes: inode:593373, cp_task:0000000013327ef9 [ 194.764512] sync_dirty_inodes: inode:593375, cp_task:0000000013327ef9 [ 194.764533] sync_dirty_inodes: inode:593376, cp_task:0000000013327ef9 [ 194.764608] sync_dirty_inodes: inode:593377, cp_task:0000000013327ef9 [ 194.764857] sync_dirty_inodes: inode:593378, cp_task:0000000013327ef9 [ 194.764884] sync_dirty_inodes: inode:593379, cp_task:0000000013327ef9 [ 194.765164] sync_dirty_inodes: inode:593380, cp_task:0000000013327ef9 [ 194.765210] sync_dirty_inodes: inode:593381, cp_task:0000000013327ef9 [ 194.765286] sync_dirty_inodes: inode:593382, cp_task:0000000013327ef9 [ 194.765513] sync_dirty_inodes: inode:593383, cp_task:0000000013327ef9 [ 194.765671] sync_dirty_inodes: inode:593384, cp_task:0000000013327ef9 [ 194.765777] sync_dirty_inodes: inode:593385, cp_task:0000000013327ef9 [ 194.766079] sync_dirty_inodes: inode:593386, cp_task:0000000013327ef9 [ 194.766568] sync_dirty_inodes: inode:593387, cp_task:0000000013327ef9 [ 194.766628] sync_dirty_inodes: inode:593388, cp_task:0000000013327ef9 [ 194.766742] sync_dirty_inodes: inode:593389, cp_task:0000000013327ef9 [ 194.766750] sync_dirty_inodes: inode:593390, cp_task:0000000013327ef9 [ 194.766788] sync_dirty_inodes: inode:593391, cp_task:0000000013327ef9 [ 194.766806] sync_dirty_inodes: inode:593392, cp_task:0000000013327ef9 [ 194.766856] sync_dirty_inodes: inode:593364, cp_task:0000000013327ef9 [ 194.767016] sync_dirty_inodes: inode:593394, cp_task:0000000013327ef9 [ 194.767033] sync_dirty_inodes: inode:593395, cp_task:0000000013327ef9 [ 194.768454] sync_dirty_inodes: inode:593393, cp_task:0000000013327ef9 [ 194.768651] sync_dirty_inodes: inode:593397, cp_task:0000000013327ef9 [ 194.768665] sync_dirty_inodes: inode:593398, cp_task:0000000013327ef9 [ 194.768678] sync_dirty_inodes: inode:593399, cp_task:0000000013327ef9 [ 194.768705] sync_dirty_inodes: inode:593400, cp_task:0000000013327ef9 [ 194.768869] sync_dirty_inodes: inode:593401, cp_task:0000000013327ef9 [ 194.768945] sync_dirty_inodes: inode:593402, cp_task:0000000013327ef9 [ 194.769081] sync_dirty_inodes: inode:593403, cp_task:0000000013327ef9 [ 194.769106] sync_dirty_inodes: inode:593404, cp_task:0000000013327ef9 [ 194.769198] sync_dirty_inodes: inode:593396, cp_task:0000000013327ef9 [ 194.769297] sync_dirty_inodes: inode:593405, cp_task:0000000013327ef9 [ 194.769390] sync_dirty_inodes: inode:593407, cp_task:0000000013327ef9 [ 194.769670] sync_dirty_inodes: inode:593408, cp_task:0000000013327ef9 [ 194.769685] sync_dirty_inodes: inode:593409, cp_task:0000000013327ef9 [ 194.769699] sync_dirty_inodes: inode:593410, cp_task:0000000013327ef9 [ 194.769716] sync_dirty_inodes: inode:593411, cp_task:0000000013327ef9 [ 194.769738] sync_dirty_inodes: inode:593412, cp_task:0000000013327ef9 [ 194.770032] sync_dirty_inodes: inode:593413, cp_task:0000000013327ef9 [ 194.770074] sync_dirty_inodes: inode:593414, cp_task:0000000013327ef9 [ 194.770157] sync_dirty_inodes: inode:593415, cp_task:0000000013327ef9 [ 194.770185] sync_dirty_inodes: inode:593416, cp_task:0000000013327ef9 [ 194.778086] sync_dirty_inodes: inode:593406, cp_task:0000000013327ef9 [ 194.778239] sync_dirty_inodes: inode:593417, cp_task:0000000013327ef9 [ 194.778322] sync_dirty_inodes: inode:593418, cp_task:0000000013327ef9 [ 194.778484] sync_dirty_inodes: inode:593419, cp_task:0000000013327ef9 [ 194.778562] sync_dirty_inodes: inode:593420, cp_task:0000000013327ef9 [ 194.778621] sync_dirty_inodes: inode:593421, cp_task:0000000013327ef9 [ 194.778635] sync_dirty_inodes: inode:593422, cp_task:0000000013327ef9 [ 194.778661] sync_dirty_inodes: inode:593423, cp_task:0000000013327ef9 [ 194.779162] sync_dirty_inodes: inode:593424, cp_task:0000000013327ef9 [ 194.779227] sync_dirty_inodes: inode:593425, cp_task:0000000013327ef9 [ 194.781339] sync_dirty_inodes: inode:593427, cp_task:0000000013327ef9 [ 194.841045] write_data_page: inode:764980, cp_task: (null) [ 194.861488] sync_dirty_inodes: inode:593948, cp_task:0000000013327ef9 [ 194.941948] sync_dirty_inodes: inode:593948, cp_task:0000000045fd0ed6 [ 194.958797] sync_dirty_inodes: inode:593948, cp_task:0000000013327ef9 [ 360.534693] usb 3-3: USB disconnect, device number 2 [ 369.937827] INFO: task kworker/u16:5:344 blocked for more than 120 seconds. [ 369.937829] Not tainted 5.0.15-gentoo-f2fslog #4 [ 369.937830] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 369.937831] kworker/u16:5 D 0 344 2 0x80000000 [ 369.937836] Workqueue: writeback wb_workfn (flush-259:0) [ 369.937838] Call Trace: [ 369.937843] ? __schedule+0x2c5/0x8b0 [ 369.937846] schedule+0x32/0x80 [ 369.937848] schedule_preempt_disabled+0x14/0x20 [ 369.937850] __mutex_lock.isra.8+0x2ba/0x4d0 [ 369.937853] ? log_store+0xf5/0x260 [ 369.937857] f2fs_write_data_pages+0x133/0x320 [ 369.937860] ? trace_hardirqs_on+0x2c/0xe0 [ 369.937863] do_writepages+0x41/0xd0 [ 369.937866] __filemap_fdatawrite_range+0x81/0xb0 [ 369.937869] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 369.937871] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 369.937874] ? up_read+0x5/0x20 [ 369.937876] ? f2fs_do_write_data_page+0x2cb/0x940 [ 369.937877] f2fs_balance_fs+0xe5/0x2c0 [ 369.937880] __write_data_page+0x1c8/0x6e0 [ 369.937884] f2fs_write_cache_pages+0x1e0/0x450 [ 369.937889] f2fs_write_data_pages+0x14b/0x320 [ 369.937891] ? trace_hardirqs_on+0x2c/0xe0 [ 369.937894] do_writepages+0x41/0xd0 [ 369.937896] __filemap_fdatawrite_range+0x81/0xb0 [ 369.937898] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 369.937900] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 369.937902] f2fs_write_node_pages+0x51/0x220 [ 369.937905] do_writepages+0x41/0xd0 [ 369.937908] __writeback_single_inode+0x3d/0x3d0 [ 369.937910] writeback_sb_inodes+0x1e8/0x410 [ 369.937913] __writeback_inodes_wb+0x5d/0xb0 [ 369.937915] wb_writeback+0x28f/0x340 [ 369.937918] ? cpumask_next+0x16/0x20 [ 369.937920] wb_workfn+0x33e/0x420 [ 369.937923] process_one_work+0x1a1/0x3d0 [ 369.937925] worker_thread+0x30/0x380 [ 369.937927] ? process_one_work+0x3d0/0x3d0 [ 369.937929] kthread+0x116/0x130 [ 369.937931] ? kthread_create_worker_on_cpu+0x70/0x70 [ 369.937933] ret_from_fork+0x3a/0x50 [ 369.937946] INFO: task cp:2049 blocked for more than 120 seconds. [ 369.937946] Not tainted 5.0.15-gentoo-f2fslog #4 [ 369.937947] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 369.937948] cp D 0 2049 2029 0x00000000 [ 369.937949] Call Trace: [ 369.937952] ? __schedule+0x2c5/0x8b0 [ 369.937954] schedule+0x32/0x80 [ 369.937956] schedule_preempt_disabled+0x14/0x20 [ 369.937958] __mutex_lock.isra.8+0x2ba/0x4d0 [ 369.937960] ? log_store+0xf5/0x260 [ 369.937963] f2fs_write_data_pages+0x133/0x320 [ 369.937965] ? trace_hardirqs_on+0x2c/0xe0 [ 369.937968] do_writepages+0x41/0xd0 [ 369.937971] __filemap_fdatawrite_range+0x81/0xb0 [ 369.937973] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 369.937975] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 369.937977] f2fs_balance_fs+0xe5/0x2c0 [ 369.937980] f2fs_mkdir+0x124/0x160 [ 369.937982] vfs_mkdir+0x102/0x1b0 [ 369.937985] do_mkdirat+0x7d/0xf0 [ 369.937987] do_syscall_64+0x69/0x399 [ 369.937989] ? trace_hardirqs_off_thunk+0x1a/0x1c [ 369.937992] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 369.937994] RIP: 0033:0x717f2eaa603b [ 369.937998] Code: Bad RIP value. [ 369.937999] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: 0000000000000053 [ 369.938001] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: 0000717f2eaa603b [ 369.938002] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: 00005b54b1a17ff0 [ 369.938003] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: 00007ffd8a71cf80 [ 369.938004] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: 00000000000001ed [ 369.938005] R13: 0000000000000000 R14: 0000000000004000 R15: 0000000000000000 [ 492.817796] INFO: task kworker/u16:5:344 blocked for more than 120 seconds. [ 492.817797] Not tainted 5.0.15-gentoo-f2fslog #4 [ 492.817797] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.817798] kworker/u16:5 D 0 344 2 0x80000000 [ 492.817810] Workqueue: writeback wb_workfn (flush-259:0) [ 492.817811] Call Trace: [ 492.817813] ? __schedule+0x2c5/0x8b0 [ 492.817815] schedule+0x32/0x80 [ 492.817816] schedule_preempt_disabled+0x14/0x20 [ 492.817817] __mutex_lock.isra.8+0x2ba/0x4d0 [ 492.817819] ? log_store+0xf5/0x260 [ 492.817821] f2fs_write_data_pages+0x133/0x320 [ 492.817823] ? trace_hardirqs_on+0x2c/0xe0 [ 492.817825] do_writepages+0x41/0xd0 [ 492.817826] __filemap_fdatawrite_range+0x81/0xb0 [ 492.817828] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 492.817829] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 492.817831] ? up_read+0x5/0x20 [ 492.817832] ? f2fs_do_write_data_page+0x2cb/0x940 [ 492.817833] f2fs_balance_fs+0xe5/0x2c0 [ 492.817835] __write_data_page+0x1c8/0x6e0 [ 492.817837] f2fs_write_cache_pages+0x1e0/0x450 [ 492.817840] f2fs_write_data_pages+0x14b/0x320 [ 492.817842] ? trace_hardirqs_on+0x2c/0xe0 [ 492.817843] do_writepages+0x41/0xd0 [ 492.817845] __filemap_fdatawrite_range+0x81/0xb0 [ 492.817846] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 492.817847] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 492.817849] f2fs_write_node_pages+0x51/0x220 [ 492.817851] do_writepages+0x41/0xd0 [ 492.817853] __writeback_single_inode+0x3d/0x3d0 [ 492.817854] writeback_sb_inodes+0x1e8/0x410 [ 492.817856] __writeback_inodes_wb+0x5d/0xb0 [ 492.817857] wb_writeback+0x28f/0x340 [ 492.817859] ? cpumask_next+0x16/0x20 [ 492.817860] wb_workfn+0x33e/0x420 [ 492.817862] process_one_work+0x1a1/0x3d0 [ 492.817863] worker_thread+0x30/0x380 [ 492.817865] ? process_one_work+0x3d0/0x3d0 [ 492.817865] kthread+0x116/0x130 [ 492.817866] ? kthread_create_worker_on_cpu+0x70/0x70 [ 492.817868] ret_from_fork+0x3a/0x50 [ 492.817873] INFO: task cp:2049 blocked for more than 120 seconds. [ 492.817873] Not tainted 5.0.15-gentoo-f2fslog #4 [ 492.817874] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.817874] cp D 0 2049 2029 0x00000000 [ 492.817875] Call Trace: [ 492.817877] ? __schedule+0x2c5/0x8b0 [ 492.817878] schedule+0x32/0x80 [ 492.817879] schedule_preempt_disabled+0x14/0x20 [ 492.817880] __mutex_lock.isra.8+0x2ba/0x4d0 [ 492.817882] ? log_store+0xf5/0x260 [ 492.817884] f2fs_write_data_pages+0x133/0x320 [ 492.817886] ? trace_hardirqs_on+0x2c/0xe0 [ 492.817887] do_writepages+0x41/0xd0 [ 492.817889] __filemap_fdatawrite_range+0x81/0xb0 [ 492.817890] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 492.817892] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 492.817893] f2fs_balance_fs+0xe5/0x2c0 [ 492.817894] f2fs_mkdir+0x124/0x160 [ 492.817896] vfs_mkdir+0x102/0x1b0 [ 492.817897] do_mkdirat+0x7d/0xf0 [ 492.817899] do_syscall_64+0x69/0x399 [ 492.817900] ? trace_hardirqs_off_thunk+0x1a/0x1c [ 492.817902] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 492.817903] RIP: 0033:0x717f2eaa603b [ 492.817905] Code: Bad RIP value. [ 492.817906] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: 0000000000000053 [ 492.817907] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: 0000717f2eaa603b [ 492.817908] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: 00005b54b1a17ff0 [ 492.817908] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: 00007ffd8a71cf80 [ 492.817909] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: 00000000000001ed [ 492.817910] R13: 0000000000000000 R14: 0000000000004000 R15: 0000000000000000 [ 615.697824] INFO: task kworker/u16:5:344 blocked for more than 120 seconds. [ 615.697825] Not tainted 5.0.15-gentoo-f2fslog #4 [ 615.697826] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 615.697827] kworker/u16:5 D 0 344 2 0x80000000 [ 615.697831] Workqueue: writeback wb_workfn (flush-259:0) [ 615.697832] Call Trace: [ 615.697836] ? __schedule+0x2c5/0x8b0 [ 615.697839] schedule+0x32/0x80 [ 615.697841] schedule_preempt_disabled+0x14/0x20 [ 615.697842] __mutex_lock.isra.8+0x2ba/0x4d0 [ 615.697845] ? log_store+0xf5/0x260 [ 615.697848] f2fs_write_data_pages+0x133/0x320 [ 615.697851] ? trace_hardirqs_on+0x2c/0xe0 [ 615.697854] do_writepages+0x41/0xd0 [ 615.697857] __filemap_fdatawrite_range+0x81/0xb0 [ 615.697859] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 615.697861] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 615.697863] ? up_read+0x5/0x20 [ 615.697865] ? f2fs_do_write_data_page+0x2cb/0x940 [ 615.697867] f2fs_balance_fs+0xe5/0x2c0 [ 615.697869] __write_data_page+0x1c8/0x6e0 [ 615.697873] f2fs_write_cache_pages+0x1e0/0x450 [ 615.697878] f2fs_write_data_pages+0x14b/0x320 [ 615.697880] ? trace_hardirqs_on+0x2c/0xe0 [ 615.697883] do_writepages+0x41/0xd0 [ 615.697885] __filemap_fdatawrite_range+0x81/0xb0 [ 615.697887] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 615.697889] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 615.697891] f2fs_write_node_pages+0x51/0x220 [ 615.697894] do_writepages+0x41/0xd0 [ 615.697897] __writeback_single_inode+0x3d/0x3d0 [ 615.697899] writeback_sb_inodes+0x1e8/0x410 [ 615.697902] __writeback_inodes_wb+0x5d/0xb0 [ 615.697904] wb_writeback+0x28f/0x340 [ 615.697906] ? cpumask_next+0x16/0x20 [ 615.697908] wb_workfn+0x33e/0x420 [ 615.697911] process_one_work+0x1a1/0x3d0 [ 615.697913] worker_thread+0x30/0x380 [ 615.697915] ? process_one_work+0x3d0/0x3d0 [ 615.697916] kthread+0x116/0x130 [ 615.697918] ? kthread_create_worker_on_cpu+0x70/0x70 [ 615.697921] ret_from_fork+0x3a/0x50 [ 615.697927] INFO: task cp:2049 blocked for more than 120 seconds. [ 615.697928] Not tainted 5.0.15-gentoo-f2fslog #4 [ 615.697929] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 615.697929] cp D 0 2049 2029 0x00000000 [ 615.697931] Call Trace: [ 615.697933] ? __schedule+0x2c5/0x8b0 [ 615.697935] schedule+0x32/0x80 [ 615.697937] schedule_preempt_disabled+0x14/0x20 [ 615.697939] __mutex_lock.isra.8+0x2ba/0x4d0 [ 615.697942] ? log_store+0xf5/0x260 [ 615.697944] f2fs_write_data_pages+0x133/0x320 [ 615.697947] ? trace_hardirqs_on+0x2c/0xe0 [ 615.697950] do_writepages+0x41/0xd0 [ 615.697952] __filemap_fdatawrite_range+0x81/0xb0 [ 615.697954] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 615.697956] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 615.697958] f2fs_balance_fs+0xe5/0x2c0 [ 615.697961] f2fs_mkdir+0x124/0x160 [ 615.697963] vfs_mkdir+0x102/0x1b0 [ 615.697965] do_mkdirat+0x7d/0xf0 [ 615.697968] do_syscall_64+0x69/0x399 [ 615.697969] ? trace_hardirqs_off_thunk+0x1a/0x1c [ 615.697972] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 615.697974] RIP: 0033:0x717f2eaa603b [ 615.697978] Code: Bad RIP value. [ 615.697979] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: 0000000000000053 [ 615.697980] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: 0000717f2eaa603b [ 615.697981] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: 00005b54b1a17ff0 [ 615.697982] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: 00007ffd8a71cf80 [ 615.697983] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: 00000000000001ed [ 615.697984] R13: 0000000000000000 R14: 0000000000004000 R15: 0000000000000000 [ 738.577796] INFO: task kworker/u16:5:344 blocked for more than 120 seconds. [ 738.577798] Not tainted 5.0.15-gentoo-f2fslog #4 [ 738.577798] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 738.577799] kworker/u16:5 D 0 344 2 0x80000000 [ 738.577812] Workqueue: writeback wb_workfn (flush-259:0) [ 738.577812] Call Trace: [ 738.577815] ? __schedule+0x2c5/0x8b0 [ 738.577817] schedule+0x32/0x80 [ 738.577818] schedule_preempt_disabled+0x14/0x20 [ 738.577819] __mutex_lock.isra.8+0x2ba/0x4d0 [ 738.577821] ? log_store+0xf5/0x260 [ 738.577823] f2fs_write_data_pages+0x133/0x320 [ 738.577825] ? trace_hardirqs_on+0x2c/0xe0 [ 738.577827] do_writepages+0x41/0xd0 [ 738.577829] __filemap_fdatawrite_range+0x81/0xb0 [ 738.577830] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 738.577832] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 738.577833] ? up_read+0x5/0x20 [ 738.577834] ? f2fs_do_write_data_page+0x2cb/0x940 [ 738.577835] f2fs_balance_fs+0xe5/0x2c0 [ 738.577837] __write_data_page+0x1c8/0x6e0 [ 738.577839] f2fs_write_cache_pages+0x1e0/0x450 [ 738.577842] f2fs_write_data_pages+0x14b/0x320 [ 738.577844] ? trace_hardirqs_on+0x2c/0xe0 [ 738.577846] do_writepages+0x41/0xd0 [ 738.577847] __filemap_fdatawrite_range+0x81/0xb0 [ 738.577849] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 738.577850] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 738.577851] f2fs_write_node_pages+0x51/0x220 [ 738.577853] do_writepages+0x41/0xd0 [ 738.577855] __writeback_single_inode+0x3d/0x3d0 [ 738.577856] writeback_sb_inodes+0x1e8/0x410 [ 738.577858] __writeback_inodes_wb+0x5d/0xb0 [ 738.577859] wb_writeback+0x28f/0x340 [ 738.577861] ? cpumask_next+0x16/0x20 [ 738.577862] wb_workfn+0x33e/0x420 [ 738.577864] process_one_work+0x1a1/0x3d0 [ 738.577865] worker_thread+0x30/0x380 [ 738.577867] ? process_one_work+0x3d0/0x3d0 [ 738.577867] kthread+0x116/0x130 [ 738.577868] ? kthread_create_worker_on_cpu+0x70/0x70 [ 738.577870] ret_from_fork+0x3a/0x50 [ 738.577875] INFO: task cp:2049 blocked for more than 120 seconds. [ 738.577875] Not tainted 5.0.15-gentoo-f2fslog #4 [ 738.577875] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 738.577876] cp D 0 2049 2029 0x00000000 [ 738.577877] Call Trace: [ 738.577878] ? __schedule+0x2c5/0x8b0 [ 738.577880] schedule+0x32/0x80 [ 738.577881] schedule_preempt_disabled+0x14/0x20 [ 738.577882] __mutex_lock.isra.8+0x2ba/0x4d0 [ 738.577884] ? log_store+0xf5/0x260 [ 738.577886] f2fs_write_data_pages+0x133/0x320 [ 738.577887] ? trace_hardirqs_on+0x2c/0xe0 [ 738.577889] do_writepages+0x41/0xd0 [ 738.577891] __filemap_fdatawrite_range+0x81/0xb0 [ 738.577892] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 738.577893] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 738.577895] f2fs_balance_fs+0xe5/0x2c0 [ 738.577896] f2fs_mkdir+0x124/0x160 [ 738.577898] vfs_mkdir+0x102/0x1b0 [ 738.577899] do_mkdirat+0x7d/0xf0 [ 738.577901] do_syscall_64+0x69/0x399 [ 738.577902] ? trace_hardirqs_off_thunk+0x1a/0x1c [ 738.577904] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 738.577905] RIP: 0033:0x717f2eaa603b [ 738.577907] Code: Bad RIP value. [ 738.577908] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: 0000000000000053 [ 738.577909] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: 0000717f2eaa603b [ 738.577909] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: 00005b54b1a17ff0 [ 738.577910] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: 00007ffd8a71cf80 [ 738.577910] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: 00000000000001ed [ 738.577911] R13: 0000000000000000 R14: 0000000000004000 R15: 0000000000000000 [ 861.457820] INFO: task kworker/u16:5:344 blocked for more than 120 seconds. [ 861.457823] Not tainted 5.0.15-gentoo-f2fslog #4 [ 861.457823] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 861.457825] kworker/u16:5 D 0 344 2 0x80000000 [ 861.457830] Workqueue: writeback wb_workfn (flush-259:0) [ 861.457832] Call Trace: [ 861.457837] ? __schedule+0x2c5/0x8b0 [ 861.457841] schedule+0x32/0x80 [ 861.457843] schedule_preempt_disabled+0x14/0x20 [ 861.457846] __mutex_lock.isra.8+0x2ba/0x4d0 [ 861.457850] ? log_store+0xf5/0x260 [ 861.457854] f2fs_write_data_pages+0x133/0x320 [ 861.457858] ? trace_hardirqs_on+0x2c/0xe0 [ 861.457862] do_writepages+0x41/0xd0 [ 861.457865] __filemap_fdatawrite_range+0x81/0xb0 [ 861.457869] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 861.457872] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 861.457874] ? up_read+0x5/0x20 [ 861.457877] ? f2fs_do_write_data_page+0x2cb/0x940 [ 861.457879] f2fs_balance_fs+0xe5/0x2c0 [ 861.457883] __write_data_page+0x1c8/0x6e0 [ 861.457888] f2fs_write_cache_pages+0x1e0/0x450 [ 861.457894] f2fs_write_data_pages+0x14b/0x320 [ 861.457898] ? trace_hardirqs_on+0x2c/0xe0 [ 861.457901] do_writepages+0x41/0xd0 [ 861.457904] __filemap_fdatawrite_range+0x81/0xb0 [ 861.457908] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 861.457910] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 861.457913] f2fs_write_node_pages+0x51/0x220 [ 861.457917] do_writepages+0x41/0xd0 [ 861.457921] __writeback_single_inode+0x3d/0x3d0 [ 861.457924] writeback_sb_inodes+0x1e8/0x410 [ 861.457928] __writeback_inodes_wb+0x5d/0xb0 [ 861.457930] wb_writeback+0x28f/0x340 [ 861.457934] ? cpumask_next+0x16/0x20 [ 861.457936] wb_workfn+0x33e/0x420 [ 861.457940] process_one_work+0x1a1/0x3d0 [ 861.457943] worker_thread+0x30/0x380 [ 861.457946] ? process_one_work+0x3d0/0x3d0 [ 861.457948] kthread+0x116/0x130 [ 861.457950] ? kthread_create_worker_on_cpu+0x70/0x70 [ 861.457953] ret_from_fork+0x3a/0x50 [ 861.457962] INFO: task cp:2049 blocked for more than 120 seconds. [ 861.457963] Not tainted 5.0.15-gentoo-f2fslog #4 [ 861.457964] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 861.457965] cp D 0 2049 2029 0x00000000 [ 861.457967] Call Trace: [ 861.457970] ? __schedule+0x2c5/0x8b0 [ 861.457974] schedule+0x32/0x80 [ 861.457976] schedule_preempt_disabled+0x14/0x20 [ 861.457978] __mutex_lock.isra.8+0x2ba/0x4d0 [ 861.457982] ? log_store+0xf5/0x260 [ 861.457986] f2fs_write_data_pages+0x133/0x320 [ 861.457990] ? trace_hardirqs_on+0x2c/0xe0 [ 861.457993] do_writepages+0x41/0xd0 [ 861.457996] __filemap_fdatawrite_range+0x81/0xb0 [ 861.458000] f2fs_sync_dirty_inodes+0x1dd/0x200 [ 861.458002] f2fs_balance_fs_bg+0x2a7/0x2c0 [ 861.458005] f2fs_balance_fs+0xe5/0x2c0 [ 861.458009] f2fs_mkdir+0x124/0x160 [ 861.458012] vfs_mkdir+0x102/0x1b0 [ 861.458015] do_mkdirat+0x7d/0xf0 [ 861.458018] do_syscall_64+0x69/0x399 [ 861.458020] ? trace_hardirqs_off_thunk+0x1a/0x1c [ 861.458025] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 861.458027] RIP: 0033:0x717f2eaa603b [ 861.458032] Code: Bad RIP value. [ 861.458033] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: 0000000000000053 [ 861.458035] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: 0000717f2eaa603b [ 861.458036] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: 00005b54b1a17ff0 [ 861.458038] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: 00007ffd8a71cf80 [ 861.458039] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: 00000000000001ed [ 861.458040] R13: 0000000000000000 R14: 0000000000004000 R15: 0000000000000000 > I was able to reproduse the bug with one of the two printk's at a time > only, and I had to disable syslog-ng and fcron for it not to hang even > then. > > Followig is log from two runs, one with each of the printk's, hope it > helps. > > --BEGIN log one > <4>[ 593.806592] write_data_page: inode:710085, cp_task: (null) > <4>[ 593.806688] write_data_page: inode:710110, cp_task: (null) > <4>[ 593.808558] write_data_page: inode:710321, cp_task: (null) > <4>[ 593.808575] write_data_page: inode:710325, cp_task: (null) > <4>[ 593.808590] write_data_page: inode:710326, cp_task: (null) > <4>[ 593.808606] write_data_page: inode:710332, cp_task: (null) > <4>[ 593.966185] write_data_page: inode:721775, cp_task: (null) > <4>[ 593.966203] write_data_page: inode:721776, cp_task: (null) > <4>[ 593.966219] write_data_page: inode:721777, cp_task: (null) > <4>[ 593.966235] write_data_page: inode:721778, cp_task: (null) > <4>[ 593.966250] write_data_page: inode:721779, cp_task: (null) > <4>[ 593.966266] write_data_page: inode:721780, cp_task: (null) > <4>[ 593.966281] write_data_page: inode:721781, cp_task: (null) > <4>[ 593.966296] write_data_page: inode:721782, cp_task: (null) > <4>[ 593.966311] write_data_page: inode:721783, cp_task: (null) > <4>[ 593.966327] write_data_page: inode:721784, cp_task: (null) > <4>[ 593.966343] write_data_page: inode:721785, cp_task: (null) > <4>[ 593.966359] write_data_page: inode:721786, cp_task: (null) > <4>[ 593.966374] write_data_page: inode:721787, cp_task: (null) > <4>[ 594.340072] write_data_page: inode:746183, cp_task: (null) > <0>[ 594.923096] BUG: stack guard page was hit at 000000006e7354a5 > (stack is 000000006445beb4..00000000988529ca) > <0>[ 594.923108] BUG: stack guard page was hit at 00000000d2c9ec98 > (stack is 00000000b417d4d3..000000001b88c4fe) > <4>[ 594.926975] kernel stack overflow (double-fault): 0000 [#1] > PREEMPT SMP PTI > <4>[ 594.934772] CPU: 7 PID: 2158 Comm: cp Not tainted > 5.0.15-gentoo-f2fslog_data_page #5 > <4>[ 594.938757] Hardware name: To Be Filled By O.E.M. To Be Filled > By O.E.M./C226 WS, BIOS P3.40 06/25/2018 > <4>[ 594.942880] RIP: 0010:tag_pages_for_writeback+0x0/0x180 > <4>[ 594.947036] Code: 30 48 83 ff 07 0f 86 eb fe ff ff 49 8b 56 40 > 49 03 56 38 48 d1 ea 49 89 d0 e9 6a fe ff ff 66 66 2e 0f 1f 84 00 00 > 00 00 00 90 <e8> 2b 41 9f 00 41 54 48 83 c7 08 55 48 89 d5 53 48 83 ec > 38 65 48 > <4>[ 594.951623] RSP: 0018:ffffb3f0028dc000 EFLAGS: 00010246 > <4>[ 594.956187] RAX: 0000000000000001 RBX: ffff9ea2d6bd7080 RCX: > ffffb3f0028dc060 > <4>[ 594.960816] RDX: 0007ffffffffffff RSI: 0000000000000000 RDI: > ffff9ea2d6bd71e0 > <4>[ 594.965375] RBP: ffffb3f0028dc1f0 R08: ffff9ea43eee0100 R09: > 0000000000000000 > <4>[ 594.969873] R10: ffffffff97e485c0 R11: 0000000000000001 R12: > ffff9ea2d6bd71e0 > <4>[ 594.974384] R13: 0000000000000009 R14: 0000000000000000 R15: > ffff9ea43bfce000 > <4>[ 594.978892] FS: 000073c62042f740(0000) > GS:ffff9ea45edc0000(0000) knlGS:0000000000000000 > <4>[ 594.983467] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > <4>[ 594.988055] CR2: ffffb3f0028dbff8 CR3: 0000000073aa8005 CR4: > 00000000003606e0 > <4>[ 594.992689] DR0: 0000000000000000 DR1: 0000000000000000 DR2: > 0000000000000000 > <4>[ 594.997310] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: > 0000000000000400 > <4>[ 595.001860] Call Trace: > <4>[ 595.006323] f2fs_write_cache_pages+0x102/0x450 > <4>[ 595.010829] ? kmem_cache_alloc+0x81/0x6b0 > <4>[ 595.015219] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 595.019483] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.023749] ? __enqueue_entity+0x5/0x60 > <4>[ 595.027995] ? enqueue_task_fair+0x4b1/0xb20 > <4>[ 595.032232] do_writepages+0x41/0xd0 > <4>[ 595.036481] ? preempt_count_add+0x79/0xb0 > <4>[ 595.040754] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.045017] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.049304] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.053594] ? si_meminfo+0x3f/0x60 > <4>[ 595.057892] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.062218] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.066549] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.070880] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.075077] ? _raw_spin_unlock_irqrestore+0x22/0x40 > <4>[ 595.079177] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.083266] __write_data_page+0x1c8/0x6e0 > <4>[ 595.087352] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.091460] ? preempt_count_add+0x79/0xb0 > <4>[ 595.095565] ? _raw_spin_lock+0x13/0x30 > <4>[ 595.099650] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.103769] ? __enqueue_entity+0x5/0x60 > <4>[ 595.107922] ? enqueue_task_fair+0x4b1/0xb20 > <4>[ 595.112117] do_writepages+0x41/0xd0 > <4>[ 595.116347] ? preempt_count_add+0x79/0xb0 > <4>[ 595.120630] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.124931] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.129268] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.133636] ? si_meminfo+0x3f/0x60 > <4>[ 595.138039] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.142520] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.146992] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.151467] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.155947] ? _raw_spin_unlock_irqrestore+0x22/0x40 > <4>[ 595.160466] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.164964] __write_data_page+0x1c8/0x6e0 > <4>[ 595.169478] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.174019] ? preempt_count_add+0x79/0xb0 > <4>[ 595.178545] ? _raw_spin_lock+0x13/0x30 > <4>[ 595.183050] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.187613] ? __enqueue_entity+0x5/0x60 > <4>[ 595.192150] ? enqueue_task_fair+0x4b1/0xb20 > <4>[ 595.196678] do_writepages+0x41/0xd0 > <4>[ 595.201226] ? preempt_count_add+0x79/0xb0 > <4>[ 595.205819] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.210430] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.214933] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.219454] ? si_meminfo+0x3f/0x60 > <4>[ 595.223862] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.228245] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.232543] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.236761] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.240792] ? _raw_spin_unlock_irqrestore+0x22/0x40 > <4>[ 595.244662] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.248474] __write_data_page+0x1c8/0x6e0 > <4>[ 595.252281] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.256076] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.259827] ? __enqueue_entity+0x5/0x60 > <4>[ 595.263581] ? enqueue_task_fair+0x4b1/0xb20 > <4>[ 595.267280] do_writepages+0x41/0xd0 > <4>[ 595.270917] ? preempt_count_add+0x79/0xb0 > <4>[ 595.274529] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.278101] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.281681] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.285226] ? si_meminfo+0x3f/0x60 > <4>[ 595.288729] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.292263] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.295800] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.299328] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.302854] ? _raw_spin_unlock_irqrestore+0x22/0x40 > <4>[ 595.306405] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.309936] __write_data_page+0x1c8/0x6e0 > <4>[ 595.313471] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.317052] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.320637] ? __enqueue_entity+0x5/0x60 > <4>[ 595.324215] ? enqueue_task_fair+0x4b1/0xb20 > <4>[ 595.327802] do_writepages+0x41/0xd0 > <4>[ 595.331372] ? preempt_count_add+0x79/0xb0 > <4>[ 595.334951] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.338497] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.342030] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.345555] ? si_meminfo+0x3f/0x60 > <4>[ 595.349054] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.352579] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.356099] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.359604] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.363110] ? _raw_spin_unlock_irqrestore+0x22/0x40 > <4>[ 595.366639] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.370156] __write_data_page+0x1c8/0x6e0 > <4>[ 595.373681] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.377218] ? __schedule+0x2cd/0x8b0 > <4>[ 595.380747] ? __lock_page+0x155/0x240 > <4>[ 595.384292] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.387868] ? __enqueue_entity+0x5/0x60 > <4>[ 595.391445] ? enqueue_task_fair+0x4b1/0xb20 > <4>[ 595.395030] do_writepages+0x41/0xd0 > <4>[ 595.398597] ? preempt_count_add+0x79/0xb0 > <4>[ 595.402169] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.405716] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.409250] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.412777] ? si_meminfo+0x3f/0x60 > <4>[ 595.416275] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.419810] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.423335] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.426850] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.430370] ? _raw_spin_unlock_irqrestore+0x22/0x40 > <4>[ 595.433904] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.437432] __write_data_page+0x1c8/0x6e0 > <4>[ 595.440969] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.444511] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.448045] ? preempt_count_add+0x79/0xb0 > <4>[ 595.451600] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.455136] ? preempt_count_add+0x79/0xb0 > <4>[ 595.458661] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.462177] do_writepages+0x41/0xd0 > <4>[ 595.465672] ? preempt_count_add+0x79/0xb0 > <4>[ 595.469169] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.472643] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.476106] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.479555] ? si_meminfo+0x3f/0x60 > <4>[ 595.482984] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.486441] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.489880] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.493322] ? up_read+0x5/0x20 > <4>[ 595.496743] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 595.500205] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.503660] __write_data_page+0x1c8/0x6e0 > <4>[ 595.507118] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.510615] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.514121] ? __enqueue_entity+0x5/0x60 > <4>[ 595.517622] ? enqueue_task_fair+0x4b1/0xb20 > <4>[ 595.521142] do_writepages+0x41/0xd0 > <4>[ 595.524630] ? preempt_count_add+0x79/0xb0 > <4>[ 595.528135] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.531601] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.535056] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.538498] ? si_meminfo+0x3f/0x60 > <4>[ 595.541912] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.545355] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.548792] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.552235] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.555665] ? _raw_spin_unlock_irqrestore+0x22/0x40 > <4>[ 595.559120] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.562568] __write_data_page+0x1c8/0x6e0 > <4>[ 595.566033] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.569527] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.573022] ? __enqueue_entity+0x5/0x60 > <4>[ 595.576517] ? enqueue_task_fair+0x4b1/0xb20 > <4>[ 595.580021] do_writepages+0x41/0xd0 > <4>[ 595.583507] ? preempt_count_add+0x79/0xb0 > <4>[ 595.586997] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.590462] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.593923] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.597377] ? si_meminfo+0x3f/0x60 > <4>[ 595.600796] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.604249] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.607691] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.611124] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.614569] ? _raw_spin_unlock_irqrestore+0x22/0x40 > <4>[ 595.618024] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.621477] __write_data_page+0x1c8/0x6e0 > <4>[ 595.624933] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.628396] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.631852] ? __enqueue_entity+0x5/0x60 > <4>[ 595.635334] ? enqueue_task_fair+0x4b1/0xb20 > <4>[ 595.638828] do_writepages+0x41/0xd0 > <4>[ 595.642314] ? preempt_count_add+0x79/0xb0 > <4>[ 595.645816] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.649282] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.652726] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.656177] ? si_meminfo+0x3f/0x60 > <4>[ 595.659610] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.663057] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.666491] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.669919] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.673342] ? _raw_spin_unlock_irqrestore+0x22/0x40 > <4>[ 595.676791] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.680243] __write_data_page+0x1c8/0x6e0 > <4>[ 595.683722] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.687244] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.690766] ? __enqueue_entity+0x5/0x60 > <4>[ 595.694278] ? enqueue_task_fair+0x4b1/0xb20 > <4>[ 595.697809] do_writepages+0x41/0xd0 > <4>[ 595.701342] ? preempt_count_add+0x79/0xb0 > <4>[ 595.704903] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.708451] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.711975] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.715488] ? si_meminfo+0x3f/0x60 > <4>[ 595.718983] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.722508] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.726031] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.729539] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.733044] ? _raw_spin_unlock_irqrestore+0x22/0x40 > <4>[ 595.736581] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.740086] __write_data_page+0x1c8/0x6e0 > <4>[ 595.743605] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.747130] ? _raw_spin_lock+0x13/0x30 > <4>[ 595.750654] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.754190] ? __enqueue_entity+0x5/0x60 > <4>[ 595.757724] ? enqueue_task_fair+0x4b1/0xb20 > <4>[ 595.761288] do_writepages+0x41/0xd0 > <4>[ 595.764855] ? preempt_count_add+0x79/0xb0 > <4>[ 595.768435] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.772001] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.775544] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.779091] ? si_meminfo+0x3f/0x60 > <4>[ 595.782617] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.786164] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.789709] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.793237] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.796755] ? _raw_spin_unlock_irqrestore+0x22/0x40 > <4>[ 595.800292] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.803843] __write_data_page+0x1c8/0x6e0 > <4>[ 595.807384] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.810944] ? __enqueue_entity+0x5/0x60 > <4>[ 595.814491] ? preempt_count_add+0x79/0xb0 > <4>[ 595.818039] ? _raw_spin_lock+0x13/0x30 > <4>[ 595.821621] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.825212] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.828808] ? _raw_spin_unlock_irqrestore+0x22/0x40 > <4>[ 595.832427] ? try_to_wake_up+0x52/0x510 > <4>[ 595.836032] do_writepages+0x41/0xd0 > <4>[ 595.839623] ? preempt_count_add+0x79/0xb0 > <4>[ 595.843222] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.846785] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.850328] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.853856] ? si_meminfo+0x3f/0x60 > <4>[ 595.857374] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.860909] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.864465] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.867996] ? up_read+0x5/0x20 > <4>[ 595.871514] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 595.875066] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.878612] __write_data_page+0x1c8/0x6e0 > <4>[ 595.882190] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.885795] ? preempt_count_add+0x79/0xb0 > <4>[ 595.889400] ? _raw_spin_lock+0x13/0x30 > <4>[ 595.893018] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.896624] ? mempool_alloc+0x65/0x190 > <4>[ 595.900219] ? find_get_entry+0xdc/0x150 > <4>[ 595.903828] do_writepages+0x41/0xd0 > <4>[ 595.907416] ? preempt_count_add+0x79/0xb0 > <4>[ 595.911003] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.914549] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.918081] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.921610] ? si_meminfo+0x3f/0x60 > <4>[ 595.925130] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.928670] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.932210] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.935742] ? unlock_page+0x5/0x30 > <4>[ 595.939260] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 595.942818] __write_data_page+0x1c8/0x6e0 > <4>[ 595.946381] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 595.949960] ? find_busiest_group+0x40/0x580 > <4>[ 595.953544] ? preempt_count_add+0x79/0xb0 > <4>[ 595.957121] ? _raw_spin_lock+0x13/0x30 > <4>[ 595.960683] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 595.964267] ? find_get_entry+0xdc/0x150 > <4>[ 595.967849] do_writepages+0x41/0xd0 > <4>[ 595.971404] ? preempt_count_add+0x79/0xb0 > <4>[ 595.974962] ? preempt_count_sub+0x5/0xa0 > <4>[ 595.978489] ? _raw_spin_unlock+0x12/0x30 > <4>[ 595.982005] ? nr_blockdev_pages+0x59/0x70 > <4>[ 595.985505] ? si_meminfo+0x3f/0x60 > <4>[ 595.988990] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 595.992488] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 595.995995] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 595.999497] ? unlock_page+0x5/0x30 > <4>[ 596.002978] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 596.006497] __write_data_page+0x1c8/0x6e0 > <4>[ 596.010031] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 596.013590] ? stat_show+0xd01/0x1280 > <4>[ 596.017136] ? up_read+0x5/0x20 > <4>[ 596.020661] ? f2fs_getxattr+0x19e/0x3e0 > <4>[ 596.024194] ? preempt_count_add+0x79/0xb0 > <4>[ 596.027720] ? _raw_spin_lock+0x13/0x30 > <4>[ 596.031237] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 596.034759] ? preempt_count_add+0x79/0xb0 > <4>[ 596.038254] ? preempt_count_sub+0x5/0xa0 > <4>[ 596.041726] ? _raw_spin_unlock+0x12/0x30 > <4>[ 596.045174] ? f2fs_inode_dirtied+0x64/0xf0 > <4>[ 596.048619] ? preempt_count_add+0x79/0xb0 > <4>[ 596.052066] ? preempt_count_sub+0x5/0xa0 > <4>[ 596.055503] do_writepages+0x41/0xd0 > <4>[ 596.058916] ? preempt_count_add+0x79/0xb0 > <4>[ 596.062334] ? preempt_count_sub+0x5/0xa0 > <4>[ 596.065718] ? _raw_spin_unlock+0x12/0x30 > <4>[ 596.069076] ? nr_blockdev_pages+0x59/0x70 > <4>[ 596.072438] ? si_meminfo+0x3f/0x60 > <4>[ 596.075787] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 596.079190] f2fs_sync_dirty_inodes+0xb7/0x1f0 > <4>[ 596.082605] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 596.086024] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 596.089415] f2fs_create+0x131/0x3e0 > <4>[ 596.092781] ? f2fs_lookup+0x20e/0x360 > <4>[ 596.096141] path_openat+0x11a8/0x1590 > <4>[ 596.099493] do_filp_open+0x93/0x100 > <4>[ 596.102836] ? preempt_count_sub+0x5/0xa0 > <4>[ 596.106176] do_sys_open+0x186/0x220 > <4>[ 596.109496] do_syscall_64+0x69/0x399 > <4>[ 596.112799] ? trace_hardirqs_off_thunk+0x1a/0x1c > <4>[ 596.116121] entry_SYSCALL_64_after_hwframe+0x49/0xbe > <4>[ 596.119459] RIP: 0033:0x73c62051f112 > <4>[ 596.122789] Code: 25 00 00 41 00 3d 00 00 41 00 74 4c 48 8d 05 > d5 42 0d 00 8b 00 85 c0 75 6d 89 f2 b8 01 01 00 00 48 89 fe bf 9c ff > ff ff 0f 05 <48> 3d 00 f0 ff ff 0f 87 a2 00 00 00 48 8b 4c 24 28 64 48 > 33 0c 25 > <4>[ 596.126545] RSP: 002b:00007ffe665bf1a0 EFLAGS: 00000246 > ORIG_RAX: 0000000000000101 > <4>[ 596.130299] RAX: ffffffffffffffda RBX: 00007ffe665bf8f0 RCX: > 000073c62051f112 > <4>[ 596.134110] RDX: 00000000000000c1 RSI: 00005670d5042690 RDI: > 00000000ffffff9c > <4>[ 596.137944] RBP: 00007ffe665bf5e0 R08: 00007ffe665bf8f0 R09: > 00007ffe665bf8f0 > <4>[ 596.141784] R10: 0000000000000180 R11: 0000000000000246 R12: > 00000000000001a4 > <4>[ 596.145664] R13: 0000000000000003 R14: 0000000000008000 R15: > 0000000000000000 > <4>[ 596.149536] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc > nls_cp437 uas usb_storage pl2303 sd_mod x86_pkg_temp_thermal kvm_intel > iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich > firewire_ohci firewire_core igb ahci crc_itu_t > processor_thermal_device libahci dca intel_soc_dts_iosf > int340x_thermal_zone pcc_cpufreq efivarfs > <4>[ 596.158385] ---[ end trace d508e01f5312dbde ]--- > --END log one > > --BEGIN log two > <4>[ 462.870628] sync_dirty_inodes: inode:624255, cp_task:000000006480c3a0 > <4>[ 462.870633] sync_dirty_inodes: inode:624255, cp_task:000000002fea927f > <4>[ 462.870634] sync_dirty_inodes: inode:624257, cp_task:000000006480c3a0 > <4>[ 462.870638] sync_dirty_inodes: inode:624257, cp_task:000000002fea927f > <4>[ 462.870660] sync_dirty_inodes: inode:623968, cp_task:000000002fea927f > <4>[ 462.870665] sync_dirty_inodes: inode:623970, cp_task:000000002fea927f > <4>[ 462.870679] sync_dirty_inodes: inode:623974, cp_task:000000002fea927f > <4>[ 462.870681] sync_dirty_inodes: inode:623974, cp_task:000000006480c3a0 > <4>[ 462.870691] sync_dirty_inodes: inode:623975, cp_task:000000002fea927f > <4>[ 462.870696] sync_dirty_inodes: inode:623976, cp_task:000000002fea927f > <4>[ 462.870697] sync_dirty_inodes: inode:623976, cp_task:000000006480c3a0 > <4>[ 462.870711] sync_dirty_inodes: inode:623978, cp_task:000000006480c3a0 > <4>[ 462.870715] sync_dirty_inodes: inode:623981, cp_task:000000006480c3a0 > <4>[ 462.870717] sync_dirty_inodes: inode:623981, cp_task:000000002fea927f > <4>[ 462.870719] sync_dirty_inodes: inode:623982, cp_task:000000006480c3a0 > <4>[ 462.870721] sync_dirty_inodes: inode:623982, cp_task:000000002fea927f > <4>[ 462.870754] sync_dirty_inodes: inode:623986, cp_task:000000002fea927f > <4>[ 462.870755] sync_dirty_inodes: inode:623986, cp_task:000000006480c3a0 > <4>[ 462.870774] sync_dirty_inodes: inode:623987, cp_task:000000006480c3a0 > <4>[ 462.870775] sync_dirty_inodes: inode:623987, cp_task:000000002fea927f > <4>[ 462.870783] sync_dirty_inodes: inode:623988, cp_task:000000002fea927f > <4>[ 462.870788] sync_dirty_inodes: inode:623991, cp_task:000000002fea927f > <4>[ 462.870794] sync_dirty_inodes: inode:623993, cp_task:000000002fea927f > <4>[ 462.870795] sync_dirty_inodes: inode:623993, cp_task:000000006480c3a0 > <4>[ 462.870801] sync_dirty_inodes: inode:623995, cp_task:000000006480c3a0 > <4>[ 462.870808] sync_dirty_inodes: inode:623995, cp_task:000000002fea927f > <4>[ 462.870810] sync_dirty_inodes: inode:623995, cp_task:000000002fea927f > <4>[ 462.870812] sync_dirty_inodes: inode:623998, cp_task:000000002fea927f > <4>[ 462.870813] sync_dirty_inodes: inode:623998, cp_task:000000006480c3a0 > <4>[ 462.870848] sync_dirty_inodes: inode:624000, cp_task:000000002fea927f > <4>[ 462.870852] sync_dirty_inodes: inode:624000, cp_task:000000006480c3a0 > <4>[ 462.870853] sync_dirty_inodes: inode:624003, cp_task:000000002fea927f > <4>[ 462.870856] sync_dirty_inodes: inode:624003, cp_task:000000006480c3a0 > <4>[ 462.870858] sync_dirty_inodes: inode:624004, cp_task:000000002fea927f > <4>[ 462.870860] sync_dirty_inodes: inode:624004, cp_task:000000006480c3a0 > <4>[ 462.870862] sync_dirty_inodes: inode:624007, cp_task:000000002fea927f > <4>[ 462.870863] sync_dirty_inodes: inode:624007, cp_task:000000006480c3a0 > <4>[ 462.870875] sync_dirty_inodes: inode:624009, cp_task:000000006480c3a0 > <4>[ 462.870902] sync_dirty_inodes: inode:624009, cp_task:000000002fea927f > <4>[ 462.870905] sync_dirty_inodes: inode:624010, cp_task:000000002fea927f > <4>[ 462.870908] sync_dirty_inodes: inode:624010, cp_task:000000006480c3a0 > <4>[ 462.870913] sync_dirty_inodes: inode:624013, cp_task:000000002fea927f > <4>[ 462.870917] sync_dirty_inodes: inode:624014, cp_task:000000002fea927f > <4>[ 462.870921] sync_dirty_inodes: inode:624014, cp_task:000000006480c3a0 > <4>[ 462.870947] sync_dirty_inodes: inode:624016, cp_task:000000006480c3a0 > <4>[ 462.870951] sync_dirty_inodes: inode:624016, cp_task:000000002fea927f > <4>[ 462.870953] sync_dirty_inodes: inode:624016, cp_task:000000002fea927f > <4>[ 462.870955] sync_dirty_inodes: inode:624016, cp_task:000000002fea927f > <4>[ 462.870956] sync_dirty_inodes: inode:624017, cp_task:000000002fea927f > <4>[ 462.870958] sync_dirty_inodes: inode:624017, cp_task:000000006480c3a0 > <4>[ 462.870961] sync_dirty_inodes: inode:624021, cp_task:000000002fea927f > <4>[ 462.870962] sync_dirty_inodes: inode:624021, cp_task:000000006480c3a0 > <4>[ 462.870968] sync_dirty_inodes: inode:624022, cp_task:000000006480c3a0 > <4>[ 462.870970] sync_dirty_inodes: inode:624022, cp_task:000000002fea927f > <4>[ 462.870975] sync_dirty_inodes: inode:624024, cp_task:000000006480c3a0 > <4>[ 462.870976] sync_dirty_inodes: inode:624024, cp_task:000000002fea927f > <4>[ 462.870989] sync_dirty_inodes: inode:624026, cp_task:000000002fea927f > <4>[ 462.870990] sync_dirty_inodes: inode:624026, cp_task:000000006480c3a0 > <0>[ 462.870997] BUG: stack guard page was hit at 0000000056e52d0a > (stack is 00000000f6235c3f..00000000f20b69a0) > <4>[ 462.876404] kernel stack overflow (double-fault): 0000 [#1] > PREEMPT SMP PTI > <4>[ 462.881794] CPU: 6 PID: 875 Comm: kworker/u16:9 Not tainted > 5.0.15-gentoo-f2fslog_sync_dirty_inodes #5 > <4>[ 462.887185] Hardware name: To Be Filled By O.E.M. To Be Filled > By O.E.M./C226 WS, BIOS P3.40 06/25/2018 > <4>[ 462.892563] Workqueue: writeback wb_workfn (flush-259:0) > <4>[ 462.897877] RIP: 0010:tracer_hardirqs_off+0x4/0xf0 > <4>[ 462.903078] Code: ee 48 0f 44 dd 4c 89 ff 48 89 da e8 d6 f9 ff > ff 49 c7 45 20 00 00 00 00 f0 41 ff 4d 00 e9 67 ff ff ff 0f 1f 40 00 > 41 57 41 56 <41> 55 41 54 55 53 48 83 ec 08 65 44 8b 25 0a 9f 06 4b f6 > 05 53 12 > <4>[ 462.908571] RSP: 0018:ffffaa364054c000 EFLAGS: 00010046 > <4>[ 462.913921] RAX: 0000000000000000 RBX: ffff962b5b81ca80 RCX: > ffffffffb6b66b50 > <4>[ 462.919235] RDX: ffffffffb6b66b40 RSI: 0000000000000000 RDI: > ffffffffb5059c74 > <4>[ 462.924523] RBP: 0000000000411200 R08: 0000000000000002 R09: > 0000000000000001 > <4>[ 462.929772] R10: 0000000001cb731b R11: 0000000001cb731c R12: > 0000000000411200 > <4>[ 462.934889] R13: 0000000000000246 R14: ffffaa364054c0c8 R15: > ffffffffb4f29520 > <4>[ 462.939888] FS: 0000000000000000(0000) > GS:ffff962b5ed80000(0000) knlGS:0000000000000000 > <4>[ 462.944841] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > <4>[ 462.949681] CR2: ffffaa364054bff8 CR3: 000000072380c002 CR4: > 00000000003606e0 > <4>[ 462.954463] DR0: 0000000000000000 DR1: 0000000000000000 DR2: > 0000000000000000 > <4>[ 462.959191] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: > 0000000000000400 > <4>[ 462.963797] Call Trace: > <4>[ 462.968280] ? finish_wait+0x80/0x80 > <4>[ 462.972785] trace_hardirqs_off+0x2b/0xe0 > <4>[ 462.977174] kmem_cache_alloc+0x44/0x6b0 > <4>[ 462.981443] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 462.985703] ? finish_wait+0x80/0x80 > <4>[ 462.989939] mempool_alloc+0x65/0x190 > <4>[ 462.994143] ? preempt_count_add+0x79/0xb0 > <4>[ 462.998379] bio_alloc_bioset+0x140/0x210 > <4>[ 463.002628] f2fs_submit_page_write+0x1bb/0x5e0 > <4>[ 463.006912] do_write_page+0x99/0x180 > <4>[ 463.011201] f2fs_outplace_write_data+0x55/0xe0 > <4>[ 463.015538] f2fs_do_write_data_page+0x588/0x940 > <4>[ 463.019897] __write_data_page+0x4f2/0x6f0 > <4>[ 463.024251] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.028616] ? pointer+0x110/0x3b0 > <4>[ 463.032972] ? log_store+0xf5/0x260 > <4>[ 463.037183] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.041277] ? __down_trylock_console_sem.isra.14+0x42/0x50 > <4>[ 463.045404] ? vprintk_emit+0x28f/0x2a0 > <4>[ 463.049517] do_writepages+0x41/0xd0 > <4>[ 463.053634] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.057778] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.061911] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.066055] ? up_read+0x5/0x20 > <4>[ 463.070224] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 463.074464] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.078742] __write_data_page+0x5b7/0x6f0 > <4>[ 463.083072] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.087459] ? pointer+0x110/0x3b0 > <4>[ 463.091880] ? log_store+0xf5/0x260 > <4>[ 463.096333] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.100853] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.105422] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.109943] do_writepages+0x41/0xd0 > <4>[ 463.114451] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.119000] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.123565] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.128116] ? up_read+0x5/0x20 > <4>[ 463.132657] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 463.137238] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.141817] __write_data_page+0x5b7/0x6f0 > <4>[ 463.146390] ? _raw_spin_unlock_irqrestore+0x34/0x40 > <4>[ 463.151025] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.155642] ? pointer+0x110/0x3b0 > <4>[ 463.160240] ? log_store+0xf5/0x260 > <4>[ 463.164859] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.169533] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.174254] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.178838] do_writepages+0x41/0xd0 > <4>[ 463.183323] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.187757] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.192105] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.196367] ? up_read+0x5/0x20 > <4>[ 463.200592] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 463.204670] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.208563] __write_data_page+0x5b7/0x6f0 > <4>[ 463.212425] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.216279] ? pointer+0x110/0x3b0 > <4>[ 463.220107] ? log_store+0xf5/0x260 > <4>[ 463.223927] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.227747] ? __down_trylock_console_sem.isra.14+0x42/0x50 > <4>[ 463.231564] ? vprintk_emit+0x28f/0x2a0 > <4>[ 463.235333] do_writepages+0x41/0xd0 > <4>[ 463.239049] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.242783] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.246543] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.250250] ? up_read+0x5/0x20 > <4>[ 463.253949] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 463.257687] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.261419] __write_data_page+0x5b7/0x6f0 > <4>[ 463.265159] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.268916] ? pointer+0x110/0x3b0 > <4>[ 463.272675] ? log_store+0xf5/0x260 > <4>[ 463.276420] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.280192] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.283952] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.287664] do_writepages+0x41/0xd0 > <4>[ 463.291360] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.295087] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.298810] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.302519] ? up_read+0x5/0x20 > <4>[ 463.306207] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 463.309944] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.313670] __write_data_page+0x5b7/0x6f0 > <4>[ 463.317379] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.321080] ? pointer+0x110/0x3b0 > <4>[ 463.324773] ? log_store+0xf5/0x260 > <4>[ 463.328444] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.332132] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.335820] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.339467] do_writepages+0x41/0xd0 > <4>[ 463.343094] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.346740] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.350392] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.354029] ? _raw_spin_unlock_irqrestore+0x3d/0x40 > <4>[ 463.357687] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.361353] __write_data_page+0x5b7/0x6f0 > <4>[ 463.365020] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.368695] ? pointer+0x110/0x3b0 > <4>[ 463.372399] ? log_store+0xf5/0x260 > <4>[ 463.376091] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.379796] ? __down_trylock_console_sem.isra.14+0x42/0x50 > <4>[ 463.383534] ? vprintk_emit+0x28f/0x2a0 > <4>[ 463.387267] do_writepages+0x41/0xd0 > <4>[ 463.390989] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.394729] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.398460] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.402181] ? up_read+0x5/0x20 > <4>[ 463.405875] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 463.409590] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.413295] __write_data_page+0x5b7/0x6f0 > <4>[ 463.417019] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.420752] ? pointer+0x110/0x3b0 > <4>[ 463.424466] ? log_store+0xf5/0x260 > <4>[ 463.428207] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.431971] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.435729] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.439437] do_writepages+0x41/0xd0 > <4>[ 463.443137] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.446863] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.450581] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.454281] ? up_read+0x5/0x20 > <4>[ 463.457962] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 463.461671] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.465368] __write_data_page+0x5b7/0x6f0 > <4>[ 463.469084] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.472807] ? pointer+0x110/0x3b0 > <4>[ 463.476499] ? log_store+0xf5/0x260 > <4>[ 463.480154] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.483842] ? __down_trylock_console_sem.isra.14+0x42/0x50 > <4>[ 463.487562] ? vprintk_emit+0x28f/0x2a0 > <4>[ 463.491266] do_writepages+0x41/0xd0 > <4>[ 463.494963] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.498684] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.502397] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.506098] ? _raw_spin_unlock_irqrestore+0x3d/0x40 > <4>[ 463.509824] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.513542] __write_data_page+0x5b7/0x6f0 > <4>[ 463.517257] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.521001] ? pointer+0x110/0x3b0 > <4>[ 463.524731] ? log_store+0xf5/0x260 > <4>[ 463.528437] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.532208] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.535979] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.539701] do_writepages+0x41/0xd0 > <4>[ 463.543413] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.547150] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.550888] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.554611] ? _raw_spin_unlock_irqrestore+0x3d/0x40 > <4>[ 463.558355] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.562091] __write_data_page+0x5b7/0x6f0 > <4>[ 463.565824] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.569569] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.573317] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.577052] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.580741] do_writepages+0x41/0xd0 > <4>[ 463.584464] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.588216] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.591965] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.595696] ? _raw_spin_unlock_irqrestore+0x3d/0x40 > <4>[ 463.599454] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.603195] __write_data_page+0x5b7/0x6f0 > <4>[ 463.606931] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.610679] ? pointer+0x110/0x3b0 > <4>[ 463.614406] ? log_store+0xf5/0x260 > <4>[ 463.618115] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.621860] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.625598] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.629284] do_writepages+0x41/0xd0 > <4>[ 463.632926] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.636607] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.640281] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.643949] ? up_read+0x5/0x20 > <4>[ 463.647596] ? f2fs_do_write_data_page+0x2cb/0x940 > <4>[ 463.651274] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.654945] __write_data_page+0x5b7/0x6f0 > <4>[ 463.658605] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.662281] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.665936] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.669582] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.673203] do_writepages+0x41/0xd0 > <4>[ 463.676809] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.680434] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.684108] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.687775] ? _raw_spin_unlock_irqrestore+0x3d/0x40 > <4>[ 463.691461] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.695135] __write_data_page+0x5b7/0x6f0 > <4>[ 463.698813] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.702513] ? pointer+0x110/0x3b0 > <4>[ 463.706187] ? log_store+0xf5/0x260 > <4>[ 463.709837] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.713508] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.717171] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.720786] do_writepages+0x41/0xd0 > <4>[ 463.724405] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.728046] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.731666] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.735258] ? _raw_spin_unlock_irqrestore+0x3d/0x40 > <4>[ 463.738885] f2fs_balance_fs+0xe5/0x2c0 > <4>[ 463.742501] __write_data_page+0x5b7/0x6f0 > <4>[ 463.746115] f2fs_write_cache_pages+0x1e0/0x450 > <4>[ 463.749744] ? pointer+0x110/0x3b0 > <4>[ 463.753345] ? log_store+0xf5/0x260 > <4>[ 463.756920] f2fs_write_data_pages+0x2ed/0x320 > <4>[ 463.760517] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.764105] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.767643] do_writepages+0x41/0xd0 > <4>[ 463.771191] __filemap_fdatawrite_range+0x81/0xb0 > <4>[ 463.774760] f2fs_sync_dirty_inodes+0x1dd/0x200 > <4>[ 463.778328] f2fs_balance_fs_bg+0x2a7/0x2c0 > <4>[ 463.781928] ? trace_hardirqs_on+0x2c/0xe0 > <4>[ 463.785524] ? retint_kernel+0x2d/0x2d > <4>[ 463.789100] f2fs_write_node_pages+0x51/0x220 > <4>[ 463.792687] ? trace_hardirqs_on_caller+0x32/0xf0 > <4>[ 463.796285] do_writepages+0x41/0xd0 > <4>[ 463.799860] ? retint_kernel+0x2d/0x2d > <4>[ 463.803417] __writeback_single_inode+0x3d/0x3d0 > <4>[ 463.806999] writeback_sb_inodes+0x1e8/0x410 > <4>[ 463.810573] __writeback_inodes_wb+0x5d/0xb0 > <4>[ 463.814158] wb_writeback+0x28f/0x340 > <4>[ 463.817730] wb_workfn+0x2d3/0x420 > <4>[ 463.821328] process_one_work+0x1a1/0x3d0 > <4>[ 463.824935] worker_thread+0x30/0x380 > <4>[ 463.828520] ? process_one_work+0x3d0/0x3d0 > <4>[ 463.832105] kthread+0x116/0x130 > <4>[ 463.835675] ? kthread_create_worker_on_cpu+0x70/0x70 > <4>[ 463.839278] ret_from_fork+0x3a/0x50 > <4>[ 463.842855] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc > nls_cp437 uas usb_storage pl2303 sd_mod x86_pkg_temp_thermal kvm_intel > iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich > firewire_ohci firewire_core igb crc_itu_t dca ahci libahci > processor_thermal_device intel_soc_dts_iosf int340x_thermal_zone > pcc_cpufreq efivarfs > <4>[ 463.851196] ---[ end trace b6d0a43e90218d45 ]--- > --END log two > >> >> --- a/fs/f2fs/checkpoint.c >> +++ b/fs/f2fs/checkpoint.c >> @@ -1074,6 +1074,11 @@ int f2fs_sync_dirty_inodes(struct f2fs_sb_info >> *sbi, >> enum >> inode_type type) >> >> F2FS_I(inode)->cp_task = current; >> >> + smp_mb(); >> + >> + printk("sync_dirty_inodes: inode:%lu, cp_task:%p", >> + inode->i_ino, F2FS_I(inode)->cp_task); >> + >> filemap_fdatawrite(inode->i_mapping); >> >> F2FS_I(inode)->cp_task = NULL; >> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c >> index 88988241a0e8..84fbbd337c37 100644 >> --- a/fs/f2fs/data.c >> +++ b/fs/f2fs/data.c >> @@ -2392,6 +2392,10 @@ int f2fs_write_single_data_page(struct page *page, >> int >> *submitted, >> if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) && >> !F2FS_I(inode)->cp_task) { >> f2fs_submit_ipu_bio(sbi, bio, page); >> + >> + printk("write_data_page: inode:%lu, cp_task:%p", >> + inode->i_ino, F2FS_I(inode)->cp_task); >> + >> f2fs_balance_fs(sbi, need_balance_fs); >> } >> >> Thanks, > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-05-15 17:01 ` Hagbard Celine @ 2019-05-20 9:37 ` Chao Yu 2019-05-20 12:58 ` Hagbard Celine 0 siblings, 1 reply; 16+ messages in thread From: Chao Yu @ 2019-05-20 9:37 UTC (permalink / raw) To: Hagbard Celine; +Cc: linux-f2fs-devel On 2019/5/16 1:01, Hagbard Celine wrote: > 2019-05-15 18:50 GMT+02:00, Hagbard Celine <hagbardcelin@gmail.com>: >> 2019-05-15 10:13 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >>> On 2019/5/15 16:03, Hagbard Celine wrote: >>>> 2019-05-15 4:25 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >>>>> On 2019/5/15 2:13, Hagbard Celine wrote: >>>>>> 2019-04-02 15:31 GMT+02:00, Chao Yu <chao@kernel.org>: >>>>>>> On 2019-4-2 20:41, Hagbard Celine wrote: >>>>>>>> That seems to have fixed it. No more errors in syslog after >>>>>>>> extracting >>>>>>>> my stage3 tarball. Also ran a couple of kernel compiles on a >>>>>>>> partition >>>>>>>> mounted with data_flush and system seems stable. >>>>>>> >>>>>>> Thanks a lot for your quick test. :) >>>>>> >>>>>> My test might have been a little too quick, or I found another >>>>>> data_flush bug that behaves similar. >>>>> >>>>> oops... >>>>> >>>>>>>>> >>>>>>>>> - if (is_dir) >>>>>>>>> - F2FS_I(inode)->cp_task = current; >>>>>>>>> + F2FS_I(inode)->cp_task = current; >>>>> >>>>> If you're sure that this patch was applying before you test, I guess we >>>>> need >>>>> an >>>>> extra barrier here to avoid out-of-order execution. >>>>> >>>>> smp_mb() >>>>> >>>>>>>>> >>>>>>>>> filemap_fdatawrite(inode->i_mapping); >>>>>>>>> >>>>>>>>> - if (is_dir) >>>>>>>>> - F2FS_I(inode)->cp_task = NULL; >>>>>>>>> + F2FS_I(inode)->cp_task = NULL; >>>>> >>>>> Thanks, >>>>> >>>> If I did this correctly; it did not get rid of the stack overflow. >>>> Here is what I did: >>>> >>>> Added smb_mb() in checkpoint.c so the relevant part looks like this: >>>> >>>> unsigned long cur_ino = inode->i_ino; >>>> >>>> F2FS_I(inode)->cp_task = current; >>>> >>>> smp_mb(); >>>> >>>> filemap_fdatawrite(inode->i_mapping); >>>> >>>> F2FS_I(inode)->cp_task = NULL; >>>> >>>> iput(inode); >>>> >>>> >>>> Compiled, rebooted and ran my test-scripts again. And got this during >>>> copy-phase in second script: >>> >>> It looks very easy to reproduce this bug, could you add log to track >>> F2FS_I(inode)->cp_task's value: >> That wasn't so easy, with all the logging from those prink the copy >> process would hang where it would oops without the printk's. > > Forgot in last mail that I actually had log from hang with both printk > enabled also: Sorry for the delay. I found another two issues related to data_flush, could you try below fixing patch? [PATCH] f2fs: fix to avoid deadloop if data_flush is on Thanks, > > [ 194.681126] sync_dirty_inodes: inode:590309, cp_task:0000000013327ef9 > [ 194.682258] sync_dirty_inodes: inode:590301, cp_task:0000000013327ef9 > [ 194.682665] sync_dirty_inodes: inode:590311, cp_task:0000000013327ef9 > [ 194.682703] sync_dirty_inodes: inode:590312, cp_task:0000000013327ef9 > [ 194.682791] sync_dirty_inodes: inode:590313, cp_task:0000000013327ef9 > [ 194.683566] sync_dirty_inodes: inode:590314, cp_task:0000000013327ef9 > [ 194.683669] sync_dirty_inodes: inode:590315, cp_task:0000000013327ef9 > [ 194.684829] sync_dirty_inodes: inode:590316, cp_task:0000000013327ef9 > [ 194.712860] sync_dirty_inodes: inode:590317, cp_task:0000000013327ef9 > [ 194.712908] sync_dirty_inodes: inode:590310, cp_task:0000000013327ef9 > [ 194.713094] sync_dirty_inodes: inode:590319, cp_task:0000000013327ef9 > [ 194.713348] sync_dirty_inodes: inode:590320, cp_task:0000000013327ef9 > [ 194.713384] sync_dirty_inodes: inode:590321, cp_task:0000000013327ef9 > [ 194.714634] sync_dirty_inodes: inode:590322, cp_task:0000000013327ef9 > [ 194.715349] sync_dirty_inodes: inode:590323, cp_task:0000000013327ef9 > [ 194.715381] sync_dirty_inodes: inode:590324, cp_task:0000000013327ef9 > [ 194.718592] sync_dirty_inodes: inode:590326, cp_task:0000000013327ef9 > [ 194.719217] sync_dirty_inodes: inode:590327, cp_task:0000000013327ef9 > [ 194.719354] sync_dirty_inodes: inode:590328, cp_task:0000000013327ef9 > [ 194.719903] sync_dirty_inodes: inode:590329, cp_task:0000000013327ef9 > [ 194.720859] sync_dirty_inodes: inode:590521, cp_task:0000000013327ef9 > [ 194.720868] sync_dirty_inodes: inode:590300, cp_task:0000000013327ef9 > [ 194.720985] sync_dirty_inodes: inode:590523, cp_task:0000000013327ef9 > [ 194.738075] sync_dirty_inodes: inode:591528, cp_task:0000000013327ef9 > [ 194.738168] sync_dirty_inodes: inode:591529, cp_task:0000000013327ef9 > [ 194.738190] sync_dirty_inodes: inode:591533, cp_task:0000000013327ef9 > [ 194.738263] sync_dirty_inodes: inode:591537, cp_task:0000000013327ef9 > [ 194.738275] sync_dirty_inodes: inode:591538, cp_task:0000000013327ef9 > [ 194.738307] sync_dirty_inodes: inode:591540, cp_task:0000000013327ef9 > [ 194.738312] sync_dirty_inodes: inode:591542, cp_task:0000000013327ef9 > [ 194.759368] sync_dirty_inodes: inode:593095, cp_task:0000000013327ef9 > [ 194.759587] sync_dirty_inodes: inode:593351, cp_task:0000000013327ef9 > [ 194.759797] sync_dirty_inodes: inode:593352, cp_task:0000000013327ef9 > [ 194.760555] sync_dirty_inodes: inode:593353, cp_task:0000000013327ef9 > [ 194.760562] sync_dirty_inodes: inode:593354, cp_task:0000000013327ef9 > [ 194.761168] sync_dirty_inodes: inode:593355, cp_task:0000000013327ef9 > [ 194.761207] sync_dirty_inodes: inode:593356, cp_task:0000000013327ef9 > [ 194.761234] sync_dirty_inodes: inode:593357, cp_task:0000000013327ef9 > [ 194.761251] sync_dirty_inodes: inode:593358, cp_task:0000000013327ef9 > [ 194.761285] sync_dirty_inodes: inode:593360, cp_task:0000000013327ef9 > [ 194.761290] sync_dirty_inodes: inode:593361, cp_task:0000000013327ef9 > [ 194.763583] sync_dirty_inodes: inode:593363, cp_task:0000000013327ef9 > [ 194.763845] sync_dirty_inodes: inode:593362, cp_task:0000000013327ef9 > [ 194.763891] sync_dirty_inodes: inode:593367, cp_task:0000000013327ef9 > [ 194.763992] sync_dirty_inodes: inode:593368, cp_task:0000000013327ef9 > [ 194.764073] sync_dirty_inodes: inode:593369, cp_task:0000000013327ef9 > [ 194.764090] sync_dirty_inodes: inode:593370, cp_task:0000000013327ef9 > [ 194.764165] sync_dirty_inodes: inode:593371, cp_task:0000000013327ef9 > [ 194.764185] sync_dirty_inodes: inode:593372, cp_task:0000000013327ef9 > [ 194.764214] sync_dirty_inodes: inode:593365, cp_task:0000000013327ef9 > [ 194.764323] sync_dirty_inodes: inode:593374, cp_task:0000000013327ef9 > [ 194.764419] sync_dirty_inodes: inode:593373, cp_task:0000000013327ef9 > [ 194.764512] sync_dirty_inodes: inode:593375, cp_task:0000000013327ef9 > [ 194.764533] sync_dirty_inodes: inode:593376, cp_task:0000000013327ef9 > [ 194.764608] sync_dirty_inodes: inode:593377, cp_task:0000000013327ef9 > [ 194.764857] sync_dirty_inodes: inode:593378, cp_task:0000000013327ef9 > [ 194.764884] sync_dirty_inodes: inode:593379, cp_task:0000000013327ef9 > [ 194.765164] sync_dirty_inodes: inode:593380, cp_task:0000000013327ef9 > [ 194.765210] sync_dirty_inodes: inode:593381, cp_task:0000000013327ef9 > [ 194.765286] sync_dirty_inodes: inode:593382, cp_task:0000000013327ef9 > [ 194.765513] sync_dirty_inodes: inode:593383, cp_task:0000000013327ef9 > [ 194.765671] sync_dirty_inodes: inode:593384, cp_task:0000000013327ef9 > [ 194.765777] sync_dirty_inodes: inode:593385, cp_task:0000000013327ef9 > [ 194.766079] sync_dirty_inodes: inode:593386, cp_task:0000000013327ef9 > [ 194.766568] sync_dirty_inodes: inode:593387, cp_task:0000000013327ef9 > [ 194.766628] sync_dirty_inodes: inode:593388, cp_task:0000000013327ef9 > [ 194.766742] sync_dirty_inodes: inode:593389, cp_task:0000000013327ef9 > [ 194.766750] sync_dirty_inodes: inode:593390, cp_task:0000000013327ef9 > [ 194.766788] sync_dirty_inodes: inode:593391, cp_task:0000000013327ef9 > [ 194.766806] sync_dirty_inodes: inode:593392, cp_task:0000000013327ef9 > [ 194.766856] sync_dirty_inodes: inode:593364, cp_task:0000000013327ef9 > [ 194.767016] sync_dirty_inodes: inode:593394, cp_task:0000000013327ef9 > [ 194.767033] sync_dirty_inodes: inode:593395, cp_task:0000000013327ef9 > [ 194.768454] sync_dirty_inodes: inode:593393, cp_task:0000000013327ef9 > [ 194.768651] sync_dirty_inodes: inode:593397, cp_task:0000000013327ef9 > [ 194.768665] sync_dirty_inodes: inode:593398, cp_task:0000000013327ef9 > [ 194.768678] sync_dirty_inodes: inode:593399, cp_task:0000000013327ef9 > [ 194.768705] sync_dirty_inodes: inode:593400, cp_task:0000000013327ef9 > [ 194.768869] sync_dirty_inodes: inode:593401, cp_task:0000000013327ef9 > [ 194.768945] sync_dirty_inodes: inode:593402, cp_task:0000000013327ef9 > [ 194.769081] sync_dirty_inodes: inode:593403, cp_task:0000000013327ef9 > [ 194.769106] sync_dirty_inodes: inode:593404, cp_task:0000000013327ef9 > [ 194.769198] sync_dirty_inodes: inode:593396, cp_task:0000000013327ef9 > [ 194.769297] sync_dirty_inodes: inode:593405, cp_task:0000000013327ef9 > [ 194.769390] sync_dirty_inodes: inode:593407, cp_task:0000000013327ef9 > [ 194.769670] sync_dirty_inodes: inode:593408, cp_task:0000000013327ef9 > [ 194.769685] sync_dirty_inodes: inode:593409, cp_task:0000000013327ef9 > [ 194.769699] sync_dirty_inodes: inode:593410, cp_task:0000000013327ef9 > [ 194.769716] sync_dirty_inodes: inode:593411, cp_task:0000000013327ef9 > [ 194.769738] sync_dirty_inodes: inode:593412, cp_task:0000000013327ef9 > [ 194.770032] sync_dirty_inodes: inode:593413, cp_task:0000000013327ef9 > [ 194.770074] sync_dirty_inodes: inode:593414, cp_task:0000000013327ef9 > [ 194.770157] sync_dirty_inodes: inode:593415, cp_task:0000000013327ef9 > [ 194.770185] sync_dirty_inodes: inode:593416, cp_task:0000000013327ef9 > [ 194.778086] sync_dirty_inodes: inode:593406, cp_task:0000000013327ef9 > [ 194.778239] sync_dirty_inodes: inode:593417, cp_task:0000000013327ef9 > [ 194.778322] sync_dirty_inodes: inode:593418, cp_task:0000000013327ef9 > [ 194.778484] sync_dirty_inodes: inode:593419, cp_task:0000000013327ef9 > [ 194.778562] sync_dirty_inodes: inode:593420, cp_task:0000000013327ef9 > [ 194.778621] sync_dirty_inodes: inode:593421, cp_task:0000000013327ef9 > [ 194.778635] sync_dirty_inodes: inode:593422, cp_task:0000000013327ef9 > [ 194.778661] sync_dirty_inodes: inode:593423, cp_task:0000000013327ef9 > [ 194.779162] sync_dirty_inodes: inode:593424, cp_task:0000000013327ef9 > [ 194.779227] sync_dirty_inodes: inode:593425, cp_task:0000000013327ef9 > [ 194.781339] sync_dirty_inodes: inode:593427, cp_task:0000000013327ef9 > [ 194.841045] write_data_page: inode:764980, cp_task: (null) > [ 194.861488] sync_dirty_inodes: inode:593948, cp_task:0000000013327ef9 > [ 194.941948] sync_dirty_inodes: inode:593948, cp_task:0000000045fd0ed6 > [ 194.958797] sync_dirty_inodes: inode:593948, cp_task:0000000013327ef9 > [ 360.534693] usb 3-3: USB disconnect, device number 2 > [ 369.937827] INFO: task kworker/u16:5:344 blocked for more than 120 seconds. > [ 369.937829] Not tainted 5.0.15-gentoo-f2fslog #4 > [ 369.937830] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 369.937831] kworker/u16:5 D 0 344 2 0x80000000 > [ 369.937836] Workqueue: writeback wb_workfn (flush-259:0) > [ 369.937838] Call Trace: > [ 369.937843] ? __schedule+0x2c5/0x8b0 > [ 369.937846] schedule+0x32/0x80 > [ 369.937848] schedule_preempt_disabled+0x14/0x20 > [ 369.937850] __mutex_lock.isra.8+0x2ba/0x4d0 > [ 369.937853] ? log_store+0xf5/0x260 > [ 369.937857] f2fs_write_data_pages+0x133/0x320 > [ 369.937860] ? trace_hardirqs_on+0x2c/0xe0 > [ 369.937863] do_writepages+0x41/0xd0 > [ 369.937866] __filemap_fdatawrite_range+0x81/0xb0 > [ 369.937869] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 369.937871] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 369.937874] ? up_read+0x5/0x20 > [ 369.937876] ? f2fs_do_write_data_page+0x2cb/0x940 > [ 369.937877] f2fs_balance_fs+0xe5/0x2c0 > [ 369.937880] __write_data_page+0x1c8/0x6e0 > [ 369.937884] f2fs_write_cache_pages+0x1e0/0x450 > [ 369.937889] f2fs_write_data_pages+0x14b/0x320 > [ 369.937891] ? trace_hardirqs_on+0x2c/0xe0 > [ 369.937894] do_writepages+0x41/0xd0 > [ 369.937896] __filemap_fdatawrite_range+0x81/0xb0 > [ 369.937898] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 369.937900] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 369.937902] f2fs_write_node_pages+0x51/0x220 > [ 369.937905] do_writepages+0x41/0xd0 > [ 369.937908] __writeback_single_inode+0x3d/0x3d0 > [ 369.937910] writeback_sb_inodes+0x1e8/0x410 > [ 369.937913] __writeback_inodes_wb+0x5d/0xb0 > [ 369.937915] wb_writeback+0x28f/0x340 > [ 369.937918] ? cpumask_next+0x16/0x20 > [ 369.937920] wb_workfn+0x33e/0x420 > [ 369.937923] process_one_work+0x1a1/0x3d0 > [ 369.937925] worker_thread+0x30/0x380 > [ 369.937927] ? process_one_work+0x3d0/0x3d0 > [ 369.937929] kthread+0x116/0x130 > [ 369.937931] ? kthread_create_worker_on_cpu+0x70/0x70 > [ 369.937933] ret_from_fork+0x3a/0x50 > [ 369.937946] INFO: task cp:2049 blocked for more than 120 seconds. > [ 369.937946] Not tainted 5.0.15-gentoo-f2fslog #4 > [ 369.937947] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 369.937948] cp D 0 2049 2029 0x00000000 > [ 369.937949] Call Trace: > [ 369.937952] ? __schedule+0x2c5/0x8b0 > [ 369.937954] schedule+0x32/0x80 > [ 369.937956] schedule_preempt_disabled+0x14/0x20 > [ 369.937958] __mutex_lock.isra.8+0x2ba/0x4d0 > [ 369.937960] ? log_store+0xf5/0x260 > [ 369.937963] f2fs_write_data_pages+0x133/0x320 > [ 369.937965] ? trace_hardirqs_on+0x2c/0xe0 > [ 369.937968] do_writepages+0x41/0xd0 > [ 369.937971] __filemap_fdatawrite_range+0x81/0xb0 > [ 369.937973] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 369.937975] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 369.937977] f2fs_balance_fs+0xe5/0x2c0 > [ 369.937980] f2fs_mkdir+0x124/0x160 > [ 369.937982] vfs_mkdir+0x102/0x1b0 > [ 369.937985] do_mkdirat+0x7d/0xf0 > [ 369.937987] do_syscall_64+0x69/0x399 > [ 369.937989] ? trace_hardirqs_off_thunk+0x1a/0x1c > [ 369.937992] entry_SYSCALL_64_after_hwframe+0x49/0xbe > [ 369.937994] RIP: 0033:0x717f2eaa603b > [ 369.937998] Code: Bad RIP value. > [ 369.937999] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: > 0000000000000053 > [ 369.938001] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: 0000717f2eaa603b > [ 369.938002] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: 00005b54b1a17ff0 > [ 369.938003] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: 00007ffd8a71cf80 > [ 369.938004] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: 00000000000001ed > [ 369.938005] R13: 0000000000000000 R14: 0000000000004000 R15: 0000000000000000 > [ 492.817796] INFO: task kworker/u16:5:344 blocked for more than 120 seconds. > [ 492.817797] Not tainted 5.0.15-gentoo-f2fslog #4 > [ 492.817797] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 492.817798] kworker/u16:5 D 0 344 2 0x80000000 > [ 492.817810] Workqueue: writeback wb_workfn (flush-259:0) > [ 492.817811] Call Trace: > [ 492.817813] ? __schedule+0x2c5/0x8b0 > [ 492.817815] schedule+0x32/0x80 > [ 492.817816] schedule_preempt_disabled+0x14/0x20 > [ 492.817817] __mutex_lock.isra.8+0x2ba/0x4d0 > [ 492.817819] ? log_store+0xf5/0x260 > [ 492.817821] f2fs_write_data_pages+0x133/0x320 > [ 492.817823] ? trace_hardirqs_on+0x2c/0xe0 > [ 492.817825] do_writepages+0x41/0xd0 > [ 492.817826] __filemap_fdatawrite_range+0x81/0xb0 > [ 492.817828] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 492.817829] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 492.817831] ? up_read+0x5/0x20 > [ 492.817832] ? f2fs_do_write_data_page+0x2cb/0x940 > [ 492.817833] f2fs_balance_fs+0xe5/0x2c0 > [ 492.817835] __write_data_page+0x1c8/0x6e0 > [ 492.817837] f2fs_write_cache_pages+0x1e0/0x450 > [ 492.817840] f2fs_write_data_pages+0x14b/0x320 > [ 492.817842] ? trace_hardirqs_on+0x2c/0xe0 > [ 492.817843] do_writepages+0x41/0xd0 > [ 492.817845] __filemap_fdatawrite_range+0x81/0xb0 > [ 492.817846] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 492.817847] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 492.817849] f2fs_write_node_pages+0x51/0x220 > [ 492.817851] do_writepages+0x41/0xd0 > [ 492.817853] __writeback_single_inode+0x3d/0x3d0 > [ 492.817854] writeback_sb_inodes+0x1e8/0x410 > [ 492.817856] __writeback_inodes_wb+0x5d/0xb0 > [ 492.817857] wb_writeback+0x28f/0x340 > [ 492.817859] ? cpumask_next+0x16/0x20 > [ 492.817860] wb_workfn+0x33e/0x420 > [ 492.817862] process_one_work+0x1a1/0x3d0 > [ 492.817863] worker_thread+0x30/0x380 > [ 492.817865] ? process_one_work+0x3d0/0x3d0 > [ 492.817865] kthread+0x116/0x130 > [ 492.817866] ? kthread_create_worker_on_cpu+0x70/0x70 > [ 492.817868] ret_from_fork+0x3a/0x50 > [ 492.817873] INFO: task cp:2049 blocked for more than 120 seconds. > [ 492.817873] Not tainted 5.0.15-gentoo-f2fslog #4 > [ 492.817874] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 492.817874] cp D 0 2049 2029 0x00000000 > [ 492.817875] Call Trace: > [ 492.817877] ? __schedule+0x2c5/0x8b0 > [ 492.817878] schedule+0x32/0x80 > [ 492.817879] schedule_preempt_disabled+0x14/0x20 > [ 492.817880] __mutex_lock.isra.8+0x2ba/0x4d0 > [ 492.817882] ? log_store+0xf5/0x260 > [ 492.817884] f2fs_write_data_pages+0x133/0x320 > [ 492.817886] ? trace_hardirqs_on+0x2c/0xe0 > [ 492.817887] do_writepages+0x41/0xd0 > [ 492.817889] __filemap_fdatawrite_range+0x81/0xb0 > [ 492.817890] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 492.817892] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 492.817893] f2fs_balance_fs+0xe5/0x2c0 > [ 492.817894] f2fs_mkdir+0x124/0x160 > [ 492.817896] vfs_mkdir+0x102/0x1b0 > [ 492.817897] do_mkdirat+0x7d/0xf0 > [ 492.817899] do_syscall_64+0x69/0x399 > [ 492.817900] ? trace_hardirqs_off_thunk+0x1a/0x1c > [ 492.817902] entry_SYSCALL_64_after_hwframe+0x49/0xbe > [ 492.817903] RIP: 0033:0x717f2eaa603b > [ 492.817905] Code: Bad RIP value. > [ 492.817906] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: > 0000000000000053 > [ 492.817907] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: 0000717f2eaa603b > [ 492.817908] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: 00005b54b1a17ff0 > [ 492.817908] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: 00007ffd8a71cf80 > [ 492.817909] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: 00000000000001ed > [ 492.817910] R13: 0000000000000000 R14: 0000000000004000 R15: 0000000000000000 > [ 615.697824] INFO: task kworker/u16:5:344 blocked for more than 120 seconds. > [ 615.697825] Not tainted 5.0.15-gentoo-f2fslog #4 > [ 615.697826] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 615.697827] kworker/u16:5 D 0 344 2 0x80000000 > [ 615.697831] Workqueue: writeback wb_workfn (flush-259:0) > [ 615.697832] Call Trace: > [ 615.697836] ? __schedule+0x2c5/0x8b0 > [ 615.697839] schedule+0x32/0x80 > [ 615.697841] schedule_preempt_disabled+0x14/0x20 > [ 615.697842] __mutex_lock.isra.8+0x2ba/0x4d0 > [ 615.697845] ? log_store+0xf5/0x260 > [ 615.697848] f2fs_write_data_pages+0x133/0x320 > [ 615.697851] ? trace_hardirqs_on+0x2c/0xe0 > [ 615.697854] do_writepages+0x41/0xd0 > [ 615.697857] __filemap_fdatawrite_range+0x81/0xb0 > [ 615.697859] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 615.697861] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 615.697863] ? up_read+0x5/0x20 > [ 615.697865] ? f2fs_do_write_data_page+0x2cb/0x940 > [ 615.697867] f2fs_balance_fs+0xe5/0x2c0 > [ 615.697869] __write_data_page+0x1c8/0x6e0 > [ 615.697873] f2fs_write_cache_pages+0x1e0/0x450 > [ 615.697878] f2fs_write_data_pages+0x14b/0x320 > [ 615.697880] ? trace_hardirqs_on+0x2c/0xe0 > [ 615.697883] do_writepages+0x41/0xd0 > [ 615.697885] __filemap_fdatawrite_range+0x81/0xb0 > [ 615.697887] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 615.697889] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 615.697891] f2fs_write_node_pages+0x51/0x220 > [ 615.697894] do_writepages+0x41/0xd0 > [ 615.697897] __writeback_single_inode+0x3d/0x3d0 > [ 615.697899] writeback_sb_inodes+0x1e8/0x410 > [ 615.697902] __writeback_inodes_wb+0x5d/0xb0 > [ 615.697904] wb_writeback+0x28f/0x340 > [ 615.697906] ? cpumask_next+0x16/0x20 > [ 615.697908] wb_workfn+0x33e/0x420 > [ 615.697911] process_one_work+0x1a1/0x3d0 > [ 615.697913] worker_thread+0x30/0x380 > [ 615.697915] ? process_one_work+0x3d0/0x3d0 > [ 615.697916] kthread+0x116/0x130 > [ 615.697918] ? kthread_create_worker_on_cpu+0x70/0x70 > [ 615.697921] ret_from_fork+0x3a/0x50 > [ 615.697927] INFO: task cp:2049 blocked for more than 120 seconds. > [ 615.697928] Not tainted 5.0.15-gentoo-f2fslog #4 > [ 615.697929] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 615.697929] cp D 0 2049 2029 0x00000000 > [ 615.697931] Call Trace: > [ 615.697933] ? __schedule+0x2c5/0x8b0 > [ 615.697935] schedule+0x32/0x80 > [ 615.697937] schedule_preempt_disabled+0x14/0x20 > [ 615.697939] __mutex_lock.isra.8+0x2ba/0x4d0 > [ 615.697942] ? log_store+0xf5/0x260 > [ 615.697944] f2fs_write_data_pages+0x133/0x320 > [ 615.697947] ? trace_hardirqs_on+0x2c/0xe0 > [ 615.697950] do_writepages+0x41/0xd0 > [ 615.697952] __filemap_fdatawrite_range+0x81/0xb0 > [ 615.697954] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 615.697956] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 615.697958] f2fs_balance_fs+0xe5/0x2c0 > [ 615.697961] f2fs_mkdir+0x124/0x160 > [ 615.697963] vfs_mkdir+0x102/0x1b0 > [ 615.697965] do_mkdirat+0x7d/0xf0 > [ 615.697968] do_syscall_64+0x69/0x399 > [ 615.697969] ? trace_hardirqs_off_thunk+0x1a/0x1c > [ 615.697972] entry_SYSCALL_64_after_hwframe+0x49/0xbe > [ 615.697974] RIP: 0033:0x717f2eaa603b > [ 615.697978] Code: Bad RIP value. > [ 615.697979] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: > 0000000000000053 > [ 615.697980] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: 0000717f2eaa603b > [ 615.697981] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: 00005b54b1a17ff0 > [ 615.697982] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: 00007ffd8a71cf80 > [ 615.697983] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: 00000000000001ed > [ 615.697984] R13: 0000000000000000 R14: 0000000000004000 R15: 0000000000000000 > [ 738.577796] INFO: task kworker/u16:5:344 blocked for more than 120 seconds. > [ 738.577798] Not tainted 5.0.15-gentoo-f2fslog #4 > [ 738.577798] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 738.577799] kworker/u16:5 D 0 344 2 0x80000000 > [ 738.577812] Workqueue: writeback wb_workfn (flush-259:0) > [ 738.577812] Call Trace: > [ 738.577815] ? __schedule+0x2c5/0x8b0 > [ 738.577817] schedule+0x32/0x80 > [ 738.577818] schedule_preempt_disabled+0x14/0x20 > [ 738.577819] __mutex_lock.isra.8+0x2ba/0x4d0 > [ 738.577821] ? log_store+0xf5/0x260 > [ 738.577823] f2fs_write_data_pages+0x133/0x320 > [ 738.577825] ? trace_hardirqs_on+0x2c/0xe0 > [ 738.577827] do_writepages+0x41/0xd0 > [ 738.577829] __filemap_fdatawrite_range+0x81/0xb0 > [ 738.577830] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 738.577832] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 738.577833] ? up_read+0x5/0x20 > [ 738.577834] ? f2fs_do_write_data_page+0x2cb/0x940 > [ 738.577835] f2fs_balance_fs+0xe5/0x2c0 > [ 738.577837] __write_data_page+0x1c8/0x6e0 > [ 738.577839] f2fs_write_cache_pages+0x1e0/0x450 > [ 738.577842] f2fs_write_data_pages+0x14b/0x320 > [ 738.577844] ? trace_hardirqs_on+0x2c/0xe0 > [ 738.577846] do_writepages+0x41/0xd0 > [ 738.577847] __filemap_fdatawrite_range+0x81/0xb0 > [ 738.577849] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 738.577850] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 738.577851] f2fs_write_node_pages+0x51/0x220 > [ 738.577853] do_writepages+0x41/0xd0 > [ 738.577855] __writeback_single_inode+0x3d/0x3d0 > [ 738.577856] writeback_sb_inodes+0x1e8/0x410 > [ 738.577858] __writeback_inodes_wb+0x5d/0xb0 > [ 738.577859] wb_writeback+0x28f/0x340 > [ 738.577861] ? cpumask_next+0x16/0x20 > [ 738.577862] wb_workfn+0x33e/0x420 > [ 738.577864] process_one_work+0x1a1/0x3d0 > [ 738.577865] worker_thread+0x30/0x380 > [ 738.577867] ? process_one_work+0x3d0/0x3d0 > [ 738.577867] kthread+0x116/0x130 > [ 738.577868] ? kthread_create_worker_on_cpu+0x70/0x70 > [ 738.577870] ret_from_fork+0x3a/0x50 > [ 738.577875] INFO: task cp:2049 blocked for more than 120 seconds. > [ 738.577875] Not tainted 5.0.15-gentoo-f2fslog #4 > [ 738.577875] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 738.577876] cp D 0 2049 2029 0x00000000 > [ 738.577877] Call Trace: > [ 738.577878] ? __schedule+0x2c5/0x8b0 > [ 738.577880] schedule+0x32/0x80 > [ 738.577881] schedule_preempt_disabled+0x14/0x20 > [ 738.577882] __mutex_lock.isra.8+0x2ba/0x4d0 > [ 738.577884] ? log_store+0xf5/0x260 > [ 738.577886] f2fs_write_data_pages+0x133/0x320 > [ 738.577887] ? trace_hardirqs_on+0x2c/0xe0 > [ 738.577889] do_writepages+0x41/0xd0 > [ 738.577891] __filemap_fdatawrite_range+0x81/0xb0 > [ 738.577892] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 738.577893] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 738.577895] f2fs_balance_fs+0xe5/0x2c0 > [ 738.577896] f2fs_mkdir+0x124/0x160 > [ 738.577898] vfs_mkdir+0x102/0x1b0 > [ 738.577899] do_mkdirat+0x7d/0xf0 > [ 738.577901] do_syscall_64+0x69/0x399 > [ 738.577902] ? trace_hardirqs_off_thunk+0x1a/0x1c > [ 738.577904] entry_SYSCALL_64_after_hwframe+0x49/0xbe > [ 738.577905] RIP: 0033:0x717f2eaa603b > [ 738.577907] Code: Bad RIP value. > [ 738.577908] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: > 0000000000000053 > [ 738.577909] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: 0000717f2eaa603b > [ 738.577909] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: 00005b54b1a17ff0 > [ 738.577910] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: 00007ffd8a71cf80 > [ 738.577910] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: 00000000000001ed > [ 738.577911] R13: 0000000000000000 R14: 0000000000004000 R15: 0000000000000000 > [ 861.457820] INFO: task kworker/u16:5:344 blocked for more than 120 seconds. > [ 861.457823] Not tainted 5.0.15-gentoo-f2fslog #4 > [ 861.457823] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 861.457825] kworker/u16:5 D 0 344 2 0x80000000 > [ 861.457830] Workqueue: writeback wb_workfn (flush-259:0) > [ 861.457832] Call Trace: > [ 861.457837] ? __schedule+0x2c5/0x8b0 > [ 861.457841] schedule+0x32/0x80 > [ 861.457843] schedule_preempt_disabled+0x14/0x20 > [ 861.457846] __mutex_lock.isra.8+0x2ba/0x4d0 > [ 861.457850] ? log_store+0xf5/0x260 > [ 861.457854] f2fs_write_data_pages+0x133/0x320 > [ 861.457858] ? trace_hardirqs_on+0x2c/0xe0 > [ 861.457862] do_writepages+0x41/0xd0 > [ 861.457865] __filemap_fdatawrite_range+0x81/0xb0 > [ 861.457869] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 861.457872] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 861.457874] ? up_read+0x5/0x20 > [ 861.457877] ? f2fs_do_write_data_page+0x2cb/0x940 > [ 861.457879] f2fs_balance_fs+0xe5/0x2c0 > [ 861.457883] __write_data_page+0x1c8/0x6e0 > [ 861.457888] f2fs_write_cache_pages+0x1e0/0x450 > [ 861.457894] f2fs_write_data_pages+0x14b/0x320 > [ 861.457898] ? trace_hardirqs_on+0x2c/0xe0 > [ 861.457901] do_writepages+0x41/0xd0 > [ 861.457904] __filemap_fdatawrite_range+0x81/0xb0 > [ 861.457908] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 861.457910] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 861.457913] f2fs_write_node_pages+0x51/0x220 > [ 861.457917] do_writepages+0x41/0xd0 > [ 861.457921] __writeback_single_inode+0x3d/0x3d0 > [ 861.457924] writeback_sb_inodes+0x1e8/0x410 > [ 861.457928] __writeback_inodes_wb+0x5d/0xb0 > [ 861.457930] wb_writeback+0x28f/0x340 > [ 861.457934] ? cpumask_next+0x16/0x20 > [ 861.457936] wb_workfn+0x33e/0x420 > [ 861.457940] process_one_work+0x1a1/0x3d0 > [ 861.457943] worker_thread+0x30/0x380 > [ 861.457946] ? process_one_work+0x3d0/0x3d0 > [ 861.457948] kthread+0x116/0x130 > [ 861.457950] ? kthread_create_worker_on_cpu+0x70/0x70 > [ 861.457953] ret_from_fork+0x3a/0x50 > [ 861.457962] INFO: task cp:2049 blocked for more than 120 seconds. > [ 861.457963] Not tainted 5.0.15-gentoo-f2fslog #4 > [ 861.457964] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [ 861.457965] cp D 0 2049 2029 0x00000000 > [ 861.457967] Call Trace: > [ 861.457970] ? __schedule+0x2c5/0x8b0 > [ 861.457974] schedule+0x32/0x80 > [ 861.457976] schedule_preempt_disabled+0x14/0x20 > [ 861.457978] __mutex_lock.isra.8+0x2ba/0x4d0 > [ 861.457982] ? log_store+0xf5/0x260 > [ 861.457986] f2fs_write_data_pages+0x133/0x320 > [ 861.457990] ? trace_hardirqs_on+0x2c/0xe0 > [ 861.457993] do_writepages+0x41/0xd0 > [ 861.457996] __filemap_fdatawrite_range+0x81/0xb0 > [ 861.458000] f2fs_sync_dirty_inodes+0x1dd/0x200 > [ 861.458002] f2fs_balance_fs_bg+0x2a7/0x2c0 > [ 861.458005] f2fs_balance_fs+0xe5/0x2c0 > [ 861.458009] f2fs_mkdir+0x124/0x160 > [ 861.458012] vfs_mkdir+0x102/0x1b0 > [ 861.458015] do_mkdirat+0x7d/0xf0 > [ 861.458018] do_syscall_64+0x69/0x399 > [ 861.458020] ? trace_hardirqs_off_thunk+0x1a/0x1c > [ 861.458025] entry_SYSCALL_64_after_hwframe+0x49/0xbe > [ 861.458027] RIP: 0033:0x717f2eaa603b > [ 861.458032] Code: Bad RIP value. > [ 861.458033] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: > 0000000000000053 > [ 861.458035] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: 0000717f2eaa603b > [ 861.458036] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: 00005b54b1a17ff0 > [ 861.458038] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: 00007ffd8a71cf80 > [ 861.458039] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: 00000000000001ed > [ 861.458040] R13: 0000000000000000 R14: 0000000000004000 R15: 0000000000000000 > > >> I was able to reproduse the bug with one of the two printk's at a time >> only, and I had to disable syslog-ng and fcron for it not to hang even >> then. >> >> Followig is log from two runs, one with each of the printk's, hope it >> helps. >> >> --BEGIN log one >> <4>[ 593.806592] write_data_page: inode:710085, cp_task: (null) >> <4>[ 593.806688] write_data_page: inode:710110, cp_task: (null) >> <4>[ 593.808558] write_data_page: inode:710321, cp_task: (null) >> <4>[ 593.808575] write_data_page: inode:710325, cp_task: (null) >> <4>[ 593.808590] write_data_page: inode:710326, cp_task: (null) >> <4>[ 593.808606] write_data_page: inode:710332, cp_task: (null) >> <4>[ 593.966185] write_data_page: inode:721775, cp_task: (null) >> <4>[ 593.966203] write_data_page: inode:721776, cp_task: (null) >> <4>[ 593.966219] write_data_page: inode:721777, cp_task: (null) >> <4>[ 593.966235] write_data_page: inode:721778, cp_task: (null) >> <4>[ 593.966250] write_data_page: inode:721779, cp_task: (null) >> <4>[ 593.966266] write_data_page: inode:721780, cp_task: (null) >> <4>[ 593.966281] write_data_page: inode:721781, cp_task: (null) >> <4>[ 593.966296] write_data_page: inode:721782, cp_task: (null) >> <4>[ 593.966311] write_data_page: inode:721783, cp_task: (null) >> <4>[ 593.966327] write_data_page: inode:721784, cp_task: (null) >> <4>[ 593.966343] write_data_page: inode:721785, cp_task: (null) >> <4>[ 593.966359] write_data_page: inode:721786, cp_task: (null) >> <4>[ 593.966374] write_data_page: inode:721787, cp_task: (null) >> <4>[ 594.340072] write_data_page: inode:746183, cp_task: (null) >> <0>[ 594.923096] BUG: stack guard page was hit at 000000006e7354a5 >> (stack is 000000006445beb4..00000000988529ca) >> <0>[ 594.923108] BUG: stack guard page was hit at 00000000d2c9ec98 >> (stack is 00000000b417d4d3..000000001b88c4fe) >> <4>[ 594.926975] kernel stack overflow (double-fault): 0000 [#1] >> PREEMPT SMP PTI >> <4>[ 594.934772] CPU: 7 PID: 2158 Comm: cp Not tainted >> 5.0.15-gentoo-f2fslog_data_page #5 >> <4>[ 594.938757] Hardware name: To Be Filled By O.E.M. To Be Filled >> By O.E.M./C226 WS, BIOS P3.40 06/25/2018 >> <4>[ 594.942880] RIP: 0010:tag_pages_for_writeback+0x0/0x180 >> <4>[ 594.947036] Code: 30 48 83 ff 07 0f 86 eb fe ff ff 49 8b 56 40 >> 49 03 56 38 48 d1 ea 49 89 d0 e9 6a fe ff ff 66 66 2e 0f 1f 84 00 00 >> 00 00 00 90 <e8> 2b 41 9f 00 41 54 48 83 c7 08 55 48 89 d5 53 48 83 ec >> 38 65 48 >> <4>[ 594.951623] RSP: 0018:ffffb3f0028dc000 EFLAGS: 00010246 >> <4>[ 594.956187] RAX: 0000000000000001 RBX: ffff9ea2d6bd7080 RCX: >> ffffb3f0028dc060 >> <4>[ 594.960816] RDX: 0007ffffffffffff RSI: 0000000000000000 RDI: >> ffff9ea2d6bd71e0 >> <4>[ 594.965375] RBP: ffffb3f0028dc1f0 R08: ffff9ea43eee0100 R09: >> 0000000000000000 >> <4>[ 594.969873] R10: ffffffff97e485c0 R11: 0000000000000001 R12: >> ffff9ea2d6bd71e0 >> <4>[ 594.974384] R13: 0000000000000009 R14: 0000000000000000 R15: >> ffff9ea43bfce000 >> <4>[ 594.978892] FS: 000073c62042f740(0000) >> GS:ffff9ea45edc0000(0000) knlGS:0000000000000000 >> <4>[ 594.983467] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >> <4>[ 594.988055] CR2: ffffb3f0028dbff8 CR3: 0000000073aa8005 CR4: >> 00000000003606e0 >> <4>[ 594.992689] DR0: 0000000000000000 DR1: 0000000000000000 DR2: >> 0000000000000000 >> <4>[ 594.997310] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: >> 0000000000000400 >> <4>[ 595.001860] Call Trace: >> <4>[ 595.006323] f2fs_write_cache_pages+0x102/0x450 >> <4>[ 595.010829] ? kmem_cache_alloc+0x81/0x6b0 >> <4>[ 595.015219] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 595.019483] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.023749] ? __enqueue_entity+0x5/0x60 >> <4>[ 595.027995] ? enqueue_task_fair+0x4b1/0xb20 >> <4>[ 595.032232] do_writepages+0x41/0xd0 >> <4>[ 595.036481] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.040754] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.045017] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.049304] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.053594] ? si_meminfo+0x3f/0x60 >> <4>[ 595.057892] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.062218] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.066549] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.070880] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.075077] ? _raw_spin_unlock_irqrestore+0x22/0x40 >> <4>[ 595.079177] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.083266] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.087352] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.091460] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.095565] ? _raw_spin_lock+0x13/0x30 >> <4>[ 595.099650] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.103769] ? __enqueue_entity+0x5/0x60 >> <4>[ 595.107922] ? enqueue_task_fair+0x4b1/0xb20 >> <4>[ 595.112117] do_writepages+0x41/0xd0 >> <4>[ 595.116347] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.120630] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.124931] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.129268] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.133636] ? si_meminfo+0x3f/0x60 >> <4>[ 595.138039] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.142520] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.146992] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.151467] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.155947] ? _raw_spin_unlock_irqrestore+0x22/0x40 >> <4>[ 595.160466] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.164964] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.169478] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.174019] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.178545] ? _raw_spin_lock+0x13/0x30 >> <4>[ 595.183050] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.187613] ? __enqueue_entity+0x5/0x60 >> <4>[ 595.192150] ? enqueue_task_fair+0x4b1/0xb20 >> <4>[ 595.196678] do_writepages+0x41/0xd0 >> <4>[ 595.201226] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.205819] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.210430] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.214933] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.219454] ? si_meminfo+0x3f/0x60 >> <4>[ 595.223862] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.228245] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.232543] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.236761] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.240792] ? _raw_spin_unlock_irqrestore+0x22/0x40 >> <4>[ 595.244662] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.248474] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.252281] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.256076] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.259827] ? __enqueue_entity+0x5/0x60 >> <4>[ 595.263581] ? enqueue_task_fair+0x4b1/0xb20 >> <4>[ 595.267280] do_writepages+0x41/0xd0 >> <4>[ 595.270917] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.274529] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.278101] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.281681] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.285226] ? si_meminfo+0x3f/0x60 >> <4>[ 595.288729] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.292263] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.295800] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.299328] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.302854] ? _raw_spin_unlock_irqrestore+0x22/0x40 >> <4>[ 595.306405] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.309936] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.313471] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.317052] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.320637] ? __enqueue_entity+0x5/0x60 >> <4>[ 595.324215] ? enqueue_task_fair+0x4b1/0xb20 >> <4>[ 595.327802] do_writepages+0x41/0xd0 >> <4>[ 595.331372] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.334951] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.338497] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.342030] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.345555] ? si_meminfo+0x3f/0x60 >> <4>[ 595.349054] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.352579] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.356099] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.359604] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.363110] ? _raw_spin_unlock_irqrestore+0x22/0x40 >> <4>[ 595.366639] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.370156] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.373681] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.377218] ? __schedule+0x2cd/0x8b0 >> <4>[ 595.380747] ? __lock_page+0x155/0x240 >> <4>[ 595.384292] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.387868] ? __enqueue_entity+0x5/0x60 >> <4>[ 595.391445] ? enqueue_task_fair+0x4b1/0xb20 >> <4>[ 595.395030] do_writepages+0x41/0xd0 >> <4>[ 595.398597] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.402169] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.405716] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.409250] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.412777] ? si_meminfo+0x3f/0x60 >> <4>[ 595.416275] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.419810] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.423335] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.426850] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.430370] ? _raw_spin_unlock_irqrestore+0x22/0x40 >> <4>[ 595.433904] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.437432] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.440969] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.444511] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.448045] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.451600] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.455136] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.458661] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.462177] do_writepages+0x41/0xd0 >> <4>[ 595.465672] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.469169] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.472643] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.476106] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.479555] ? si_meminfo+0x3f/0x60 >> <4>[ 595.482984] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.486441] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.489880] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.493322] ? up_read+0x5/0x20 >> <4>[ 595.496743] ? f2fs_do_write_data_page+0x2cb/0x940 >> <4>[ 595.500205] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.503660] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.507118] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.510615] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.514121] ? __enqueue_entity+0x5/0x60 >> <4>[ 595.517622] ? enqueue_task_fair+0x4b1/0xb20 >> <4>[ 595.521142] do_writepages+0x41/0xd0 >> <4>[ 595.524630] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.528135] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.531601] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.535056] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.538498] ? si_meminfo+0x3f/0x60 >> <4>[ 595.541912] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.545355] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.548792] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.552235] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.555665] ? _raw_spin_unlock_irqrestore+0x22/0x40 >> <4>[ 595.559120] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.562568] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.566033] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.569527] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.573022] ? __enqueue_entity+0x5/0x60 >> <4>[ 595.576517] ? enqueue_task_fair+0x4b1/0xb20 >> <4>[ 595.580021] do_writepages+0x41/0xd0 >> <4>[ 595.583507] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.586997] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.590462] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.593923] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.597377] ? si_meminfo+0x3f/0x60 >> <4>[ 595.600796] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.604249] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.607691] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.611124] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.614569] ? _raw_spin_unlock_irqrestore+0x22/0x40 >> <4>[ 595.618024] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.621477] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.624933] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.628396] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.631852] ? __enqueue_entity+0x5/0x60 >> <4>[ 595.635334] ? enqueue_task_fair+0x4b1/0xb20 >> <4>[ 595.638828] do_writepages+0x41/0xd0 >> <4>[ 595.642314] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.645816] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.649282] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.652726] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.656177] ? si_meminfo+0x3f/0x60 >> <4>[ 595.659610] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.663057] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.666491] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.669919] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.673342] ? _raw_spin_unlock_irqrestore+0x22/0x40 >> <4>[ 595.676791] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.680243] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.683722] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.687244] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.690766] ? __enqueue_entity+0x5/0x60 >> <4>[ 595.694278] ? enqueue_task_fair+0x4b1/0xb20 >> <4>[ 595.697809] do_writepages+0x41/0xd0 >> <4>[ 595.701342] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.704903] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.708451] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.711975] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.715488] ? si_meminfo+0x3f/0x60 >> <4>[ 595.718983] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.722508] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.726031] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.729539] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.733044] ? _raw_spin_unlock_irqrestore+0x22/0x40 >> <4>[ 595.736581] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.740086] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.743605] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.747130] ? _raw_spin_lock+0x13/0x30 >> <4>[ 595.750654] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.754190] ? __enqueue_entity+0x5/0x60 >> <4>[ 595.757724] ? enqueue_task_fair+0x4b1/0xb20 >> <4>[ 595.761288] do_writepages+0x41/0xd0 >> <4>[ 595.764855] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.768435] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.772001] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.775544] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.779091] ? si_meminfo+0x3f/0x60 >> <4>[ 595.782617] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.786164] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.789709] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.793237] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.796755] ? _raw_spin_unlock_irqrestore+0x22/0x40 >> <4>[ 595.800292] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.803843] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.807384] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.810944] ? __enqueue_entity+0x5/0x60 >> <4>[ 595.814491] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.818039] ? _raw_spin_lock+0x13/0x30 >> <4>[ 595.821621] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.825212] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.828808] ? _raw_spin_unlock_irqrestore+0x22/0x40 >> <4>[ 595.832427] ? try_to_wake_up+0x52/0x510 >> <4>[ 595.836032] do_writepages+0x41/0xd0 >> <4>[ 595.839623] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.843222] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.846785] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.850328] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.853856] ? si_meminfo+0x3f/0x60 >> <4>[ 595.857374] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.860909] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.864465] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.867996] ? up_read+0x5/0x20 >> <4>[ 595.871514] ? f2fs_do_write_data_page+0x2cb/0x940 >> <4>[ 595.875066] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.878612] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.882190] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.885795] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.889400] ? _raw_spin_lock+0x13/0x30 >> <4>[ 595.893018] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.896624] ? mempool_alloc+0x65/0x190 >> <4>[ 595.900219] ? find_get_entry+0xdc/0x150 >> <4>[ 595.903828] do_writepages+0x41/0xd0 >> <4>[ 595.907416] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.911003] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.914549] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.918081] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.921610] ? si_meminfo+0x3f/0x60 >> <4>[ 595.925130] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.928670] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.932210] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.935742] ? unlock_page+0x5/0x30 >> <4>[ 595.939260] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 595.942818] __write_data_page+0x1c8/0x6e0 >> <4>[ 595.946381] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 595.949960] ? find_busiest_group+0x40/0x580 >> <4>[ 595.953544] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.957121] ? _raw_spin_lock+0x13/0x30 >> <4>[ 595.960683] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 595.964267] ? find_get_entry+0xdc/0x150 >> <4>[ 595.967849] do_writepages+0x41/0xd0 >> <4>[ 595.971404] ? preempt_count_add+0x79/0xb0 >> <4>[ 595.974962] ? preempt_count_sub+0x5/0xa0 >> <4>[ 595.978489] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 595.982005] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 595.985505] ? si_meminfo+0x3f/0x60 >> <4>[ 595.988990] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 595.992488] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 595.995995] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 595.999497] ? unlock_page+0x5/0x30 >> <4>[ 596.002978] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 596.006497] __write_data_page+0x1c8/0x6e0 >> <4>[ 596.010031] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 596.013590] ? stat_show+0xd01/0x1280 >> <4>[ 596.017136] ? up_read+0x5/0x20 >> <4>[ 596.020661] ? f2fs_getxattr+0x19e/0x3e0 >> <4>[ 596.024194] ? preempt_count_add+0x79/0xb0 >> <4>[ 596.027720] ? _raw_spin_lock+0x13/0x30 >> <4>[ 596.031237] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 596.034759] ? preempt_count_add+0x79/0xb0 >> <4>[ 596.038254] ? preempt_count_sub+0x5/0xa0 >> <4>[ 596.041726] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 596.045174] ? f2fs_inode_dirtied+0x64/0xf0 >> <4>[ 596.048619] ? preempt_count_add+0x79/0xb0 >> <4>[ 596.052066] ? preempt_count_sub+0x5/0xa0 >> <4>[ 596.055503] do_writepages+0x41/0xd0 >> <4>[ 596.058916] ? preempt_count_add+0x79/0xb0 >> <4>[ 596.062334] ? preempt_count_sub+0x5/0xa0 >> <4>[ 596.065718] ? _raw_spin_unlock+0x12/0x30 >> <4>[ 596.069076] ? nr_blockdev_pages+0x59/0x70 >> <4>[ 596.072438] ? si_meminfo+0x3f/0x60 >> <4>[ 596.075787] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 596.079190] f2fs_sync_dirty_inodes+0xb7/0x1f0 >> <4>[ 596.082605] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 596.086024] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 596.089415] f2fs_create+0x131/0x3e0 >> <4>[ 596.092781] ? f2fs_lookup+0x20e/0x360 >> <4>[ 596.096141] path_openat+0x11a8/0x1590 >> <4>[ 596.099493] do_filp_open+0x93/0x100 >> <4>[ 596.102836] ? preempt_count_sub+0x5/0xa0 >> <4>[ 596.106176] do_sys_open+0x186/0x220 >> <4>[ 596.109496] do_syscall_64+0x69/0x399 >> <4>[ 596.112799] ? trace_hardirqs_off_thunk+0x1a/0x1c >> <4>[ 596.116121] entry_SYSCALL_64_after_hwframe+0x49/0xbe >> <4>[ 596.119459] RIP: 0033:0x73c62051f112 >> <4>[ 596.122789] Code: 25 00 00 41 00 3d 00 00 41 00 74 4c 48 8d 05 >> d5 42 0d 00 8b 00 85 c0 75 6d 89 f2 b8 01 01 00 00 48 89 fe bf 9c ff >> ff ff 0f 05 <48> 3d 00 f0 ff ff 0f 87 a2 00 00 00 48 8b 4c 24 28 64 48 >> 33 0c 25 >> <4>[ 596.126545] RSP: 002b:00007ffe665bf1a0 EFLAGS: 00000246 >> ORIG_RAX: 0000000000000101 >> <4>[ 596.130299] RAX: ffffffffffffffda RBX: 00007ffe665bf8f0 RCX: >> 000073c62051f112 >> <4>[ 596.134110] RDX: 00000000000000c1 RSI: 00005670d5042690 RDI: >> 00000000ffffff9c >> <4>[ 596.137944] RBP: 00007ffe665bf5e0 R08: 00007ffe665bf8f0 R09: >> 00007ffe665bf8f0 >> <4>[ 596.141784] R10: 0000000000000180 R11: 0000000000000246 R12: >> 00000000000001a4 >> <4>[ 596.145664] R13: 0000000000000003 R14: 0000000000008000 R15: >> 0000000000000000 >> <4>[ 596.149536] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc >> nls_cp437 uas usb_storage pl2303 sd_mod x86_pkg_temp_thermal kvm_intel >> iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich >> firewire_ohci firewire_core igb ahci crc_itu_t >> processor_thermal_device libahci dca intel_soc_dts_iosf >> int340x_thermal_zone pcc_cpufreq efivarfs >> <4>[ 596.158385] ---[ end trace d508e01f5312dbde ]--- >> --END log one >> >> --BEGIN log two >> <4>[ 462.870628] sync_dirty_inodes: inode:624255, cp_task:000000006480c3a0 >> <4>[ 462.870633] sync_dirty_inodes: inode:624255, cp_task:000000002fea927f >> <4>[ 462.870634] sync_dirty_inodes: inode:624257, cp_task:000000006480c3a0 >> <4>[ 462.870638] sync_dirty_inodes: inode:624257, cp_task:000000002fea927f >> <4>[ 462.870660] sync_dirty_inodes: inode:623968, cp_task:000000002fea927f >> <4>[ 462.870665] sync_dirty_inodes: inode:623970, cp_task:000000002fea927f >> <4>[ 462.870679] sync_dirty_inodes: inode:623974, cp_task:000000002fea927f >> <4>[ 462.870681] sync_dirty_inodes: inode:623974, cp_task:000000006480c3a0 >> <4>[ 462.870691] sync_dirty_inodes: inode:623975, cp_task:000000002fea927f >> <4>[ 462.870696] sync_dirty_inodes: inode:623976, cp_task:000000002fea927f >> <4>[ 462.870697] sync_dirty_inodes: inode:623976, cp_task:000000006480c3a0 >> <4>[ 462.870711] sync_dirty_inodes: inode:623978, cp_task:000000006480c3a0 >> <4>[ 462.870715] sync_dirty_inodes: inode:623981, cp_task:000000006480c3a0 >> <4>[ 462.870717] sync_dirty_inodes: inode:623981, cp_task:000000002fea927f >> <4>[ 462.870719] sync_dirty_inodes: inode:623982, cp_task:000000006480c3a0 >> <4>[ 462.870721] sync_dirty_inodes: inode:623982, cp_task:000000002fea927f >> <4>[ 462.870754] sync_dirty_inodes: inode:623986, cp_task:000000002fea927f >> <4>[ 462.870755] sync_dirty_inodes: inode:623986, cp_task:000000006480c3a0 >> <4>[ 462.870774] sync_dirty_inodes: inode:623987, cp_task:000000006480c3a0 >> <4>[ 462.870775] sync_dirty_inodes: inode:623987, cp_task:000000002fea927f >> <4>[ 462.870783] sync_dirty_inodes: inode:623988, cp_task:000000002fea927f >> <4>[ 462.870788] sync_dirty_inodes: inode:623991, cp_task:000000002fea927f >> <4>[ 462.870794] sync_dirty_inodes: inode:623993, cp_task:000000002fea927f >> <4>[ 462.870795] sync_dirty_inodes: inode:623993, cp_task:000000006480c3a0 >> <4>[ 462.870801] sync_dirty_inodes: inode:623995, cp_task:000000006480c3a0 >> <4>[ 462.870808] sync_dirty_inodes: inode:623995, cp_task:000000002fea927f >> <4>[ 462.870810] sync_dirty_inodes: inode:623995, cp_task:000000002fea927f >> <4>[ 462.870812] sync_dirty_inodes: inode:623998, cp_task:000000002fea927f >> <4>[ 462.870813] sync_dirty_inodes: inode:623998, cp_task:000000006480c3a0 >> <4>[ 462.870848] sync_dirty_inodes: inode:624000, cp_task:000000002fea927f >> <4>[ 462.870852] sync_dirty_inodes: inode:624000, cp_task:000000006480c3a0 >> <4>[ 462.870853] sync_dirty_inodes: inode:624003, cp_task:000000002fea927f >> <4>[ 462.870856] sync_dirty_inodes: inode:624003, cp_task:000000006480c3a0 >> <4>[ 462.870858] sync_dirty_inodes: inode:624004, cp_task:000000002fea927f >> <4>[ 462.870860] sync_dirty_inodes: inode:624004, cp_task:000000006480c3a0 >> <4>[ 462.870862] sync_dirty_inodes: inode:624007, cp_task:000000002fea927f >> <4>[ 462.870863] sync_dirty_inodes: inode:624007, cp_task:000000006480c3a0 >> <4>[ 462.870875] sync_dirty_inodes: inode:624009, cp_task:000000006480c3a0 >> <4>[ 462.870902] sync_dirty_inodes: inode:624009, cp_task:000000002fea927f >> <4>[ 462.870905] sync_dirty_inodes: inode:624010, cp_task:000000002fea927f >> <4>[ 462.870908] sync_dirty_inodes: inode:624010, cp_task:000000006480c3a0 >> <4>[ 462.870913] sync_dirty_inodes: inode:624013, cp_task:000000002fea927f >> <4>[ 462.870917] sync_dirty_inodes: inode:624014, cp_task:000000002fea927f >> <4>[ 462.870921] sync_dirty_inodes: inode:624014, cp_task:000000006480c3a0 >> <4>[ 462.870947] sync_dirty_inodes: inode:624016, cp_task:000000006480c3a0 >> <4>[ 462.870951] sync_dirty_inodes: inode:624016, cp_task:000000002fea927f >> <4>[ 462.870953] sync_dirty_inodes: inode:624016, cp_task:000000002fea927f >> <4>[ 462.870955] sync_dirty_inodes: inode:624016, cp_task:000000002fea927f >> <4>[ 462.870956] sync_dirty_inodes: inode:624017, cp_task:000000002fea927f >> <4>[ 462.870958] sync_dirty_inodes: inode:624017, cp_task:000000006480c3a0 >> <4>[ 462.870961] sync_dirty_inodes: inode:624021, cp_task:000000002fea927f >> <4>[ 462.870962] sync_dirty_inodes: inode:624021, cp_task:000000006480c3a0 >> <4>[ 462.870968] sync_dirty_inodes: inode:624022, cp_task:000000006480c3a0 >> <4>[ 462.870970] sync_dirty_inodes: inode:624022, cp_task:000000002fea927f >> <4>[ 462.870975] sync_dirty_inodes: inode:624024, cp_task:000000006480c3a0 >> <4>[ 462.870976] sync_dirty_inodes: inode:624024, cp_task:000000002fea927f >> <4>[ 462.870989] sync_dirty_inodes: inode:624026, cp_task:000000002fea927f >> <4>[ 462.870990] sync_dirty_inodes: inode:624026, cp_task:000000006480c3a0 >> <0>[ 462.870997] BUG: stack guard page was hit at 0000000056e52d0a >> (stack is 00000000f6235c3f..00000000f20b69a0) >> <4>[ 462.876404] kernel stack overflow (double-fault): 0000 [#1] >> PREEMPT SMP PTI >> <4>[ 462.881794] CPU: 6 PID: 875 Comm: kworker/u16:9 Not tainted >> 5.0.15-gentoo-f2fslog_sync_dirty_inodes #5 >> <4>[ 462.887185] Hardware name: To Be Filled By O.E.M. To Be Filled >> By O.E.M./C226 WS, BIOS P3.40 06/25/2018 >> <4>[ 462.892563] Workqueue: writeback wb_workfn (flush-259:0) >> <4>[ 462.897877] RIP: 0010:tracer_hardirqs_off+0x4/0xf0 >> <4>[ 462.903078] Code: ee 48 0f 44 dd 4c 89 ff 48 89 da e8 d6 f9 ff >> ff 49 c7 45 20 00 00 00 00 f0 41 ff 4d 00 e9 67 ff ff ff 0f 1f 40 00 >> 41 57 41 56 <41> 55 41 54 55 53 48 83 ec 08 65 44 8b 25 0a 9f 06 4b f6 >> 05 53 12 >> <4>[ 462.908571] RSP: 0018:ffffaa364054c000 EFLAGS: 00010046 >> <4>[ 462.913921] RAX: 0000000000000000 RBX: ffff962b5b81ca80 RCX: >> ffffffffb6b66b50 >> <4>[ 462.919235] RDX: ffffffffb6b66b40 RSI: 0000000000000000 RDI: >> ffffffffb5059c74 >> <4>[ 462.924523] RBP: 0000000000411200 R08: 0000000000000002 R09: >> 0000000000000001 >> <4>[ 462.929772] R10: 0000000001cb731b R11: 0000000001cb731c R12: >> 0000000000411200 >> <4>[ 462.934889] R13: 0000000000000246 R14: ffffaa364054c0c8 R15: >> ffffffffb4f29520 >> <4>[ 462.939888] FS: 0000000000000000(0000) >> GS:ffff962b5ed80000(0000) knlGS:0000000000000000 >> <4>[ 462.944841] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >> <4>[ 462.949681] CR2: ffffaa364054bff8 CR3: 000000072380c002 CR4: >> 00000000003606e0 >> <4>[ 462.954463] DR0: 0000000000000000 DR1: 0000000000000000 DR2: >> 0000000000000000 >> <4>[ 462.959191] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: >> 0000000000000400 >> <4>[ 462.963797] Call Trace: >> <4>[ 462.968280] ? finish_wait+0x80/0x80 >> <4>[ 462.972785] trace_hardirqs_off+0x2b/0xe0 >> <4>[ 462.977174] kmem_cache_alloc+0x44/0x6b0 >> <4>[ 462.981443] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 462.985703] ? finish_wait+0x80/0x80 >> <4>[ 462.989939] mempool_alloc+0x65/0x190 >> <4>[ 462.994143] ? preempt_count_add+0x79/0xb0 >> <4>[ 462.998379] bio_alloc_bioset+0x140/0x210 >> <4>[ 463.002628] f2fs_submit_page_write+0x1bb/0x5e0 >> <4>[ 463.006912] do_write_page+0x99/0x180 >> <4>[ 463.011201] f2fs_outplace_write_data+0x55/0xe0 >> <4>[ 463.015538] f2fs_do_write_data_page+0x588/0x940 >> <4>[ 463.019897] __write_data_page+0x4f2/0x6f0 >> <4>[ 463.024251] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.028616] ? pointer+0x110/0x3b0 >> <4>[ 463.032972] ? log_store+0xf5/0x260 >> <4>[ 463.037183] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.041277] ? __down_trylock_console_sem.isra.14+0x42/0x50 >> <4>[ 463.045404] ? vprintk_emit+0x28f/0x2a0 >> <4>[ 463.049517] do_writepages+0x41/0xd0 >> <4>[ 463.053634] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.057778] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.061911] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.066055] ? up_read+0x5/0x20 >> <4>[ 463.070224] ? f2fs_do_write_data_page+0x2cb/0x940 >> <4>[ 463.074464] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.078742] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.083072] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.087459] ? pointer+0x110/0x3b0 >> <4>[ 463.091880] ? log_store+0xf5/0x260 >> <4>[ 463.096333] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.100853] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.105422] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.109943] do_writepages+0x41/0xd0 >> <4>[ 463.114451] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.119000] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.123565] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.128116] ? up_read+0x5/0x20 >> <4>[ 463.132657] ? f2fs_do_write_data_page+0x2cb/0x940 >> <4>[ 463.137238] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.141817] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.146390] ? _raw_spin_unlock_irqrestore+0x34/0x40 >> <4>[ 463.151025] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.155642] ? pointer+0x110/0x3b0 >> <4>[ 463.160240] ? log_store+0xf5/0x260 >> <4>[ 463.164859] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.169533] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.174254] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.178838] do_writepages+0x41/0xd0 >> <4>[ 463.183323] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.187757] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.192105] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.196367] ? up_read+0x5/0x20 >> <4>[ 463.200592] ? f2fs_do_write_data_page+0x2cb/0x940 >> <4>[ 463.204670] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.208563] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.212425] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.216279] ? pointer+0x110/0x3b0 >> <4>[ 463.220107] ? log_store+0xf5/0x260 >> <4>[ 463.223927] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.227747] ? __down_trylock_console_sem.isra.14+0x42/0x50 >> <4>[ 463.231564] ? vprintk_emit+0x28f/0x2a0 >> <4>[ 463.235333] do_writepages+0x41/0xd0 >> <4>[ 463.239049] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.242783] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.246543] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.250250] ? up_read+0x5/0x20 >> <4>[ 463.253949] ? f2fs_do_write_data_page+0x2cb/0x940 >> <4>[ 463.257687] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.261419] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.265159] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.268916] ? pointer+0x110/0x3b0 >> <4>[ 463.272675] ? log_store+0xf5/0x260 >> <4>[ 463.276420] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.280192] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.283952] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.287664] do_writepages+0x41/0xd0 >> <4>[ 463.291360] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.295087] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.298810] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.302519] ? up_read+0x5/0x20 >> <4>[ 463.306207] ? f2fs_do_write_data_page+0x2cb/0x940 >> <4>[ 463.309944] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.313670] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.317379] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.321080] ? pointer+0x110/0x3b0 >> <4>[ 463.324773] ? log_store+0xf5/0x260 >> <4>[ 463.328444] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.332132] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.335820] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.339467] do_writepages+0x41/0xd0 >> <4>[ 463.343094] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.346740] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.350392] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.354029] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >> <4>[ 463.357687] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.361353] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.365020] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.368695] ? pointer+0x110/0x3b0 >> <4>[ 463.372399] ? log_store+0xf5/0x260 >> <4>[ 463.376091] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.379796] ? __down_trylock_console_sem.isra.14+0x42/0x50 >> <4>[ 463.383534] ? vprintk_emit+0x28f/0x2a0 >> <4>[ 463.387267] do_writepages+0x41/0xd0 >> <4>[ 463.390989] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.394729] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.398460] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.402181] ? up_read+0x5/0x20 >> <4>[ 463.405875] ? f2fs_do_write_data_page+0x2cb/0x940 >> <4>[ 463.409590] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.413295] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.417019] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.420752] ? pointer+0x110/0x3b0 >> <4>[ 463.424466] ? log_store+0xf5/0x260 >> <4>[ 463.428207] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.431971] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.435729] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.439437] do_writepages+0x41/0xd0 >> <4>[ 463.443137] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.446863] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.450581] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.454281] ? up_read+0x5/0x20 >> <4>[ 463.457962] ? f2fs_do_write_data_page+0x2cb/0x940 >> <4>[ 463.461671] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.465368] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.469084] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.472807] ? pointer+0x110/0x3b0 >> <4>[ 463.476499] ? log_store+0xf5/0x260 >> <4>[ 463.480154] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.483842] ? __down_trylock_console_sem.isra.14+0x42/0x50 >> <4>[ 463.487562] ? vprintk_emit+0x28f/0x2a0 >> <4>[ 463.491266] do_writepages+0x41/0xd0 >> <4>[ 463.494963] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.498684] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.502397] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.506098] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >> <4>[ 463.509824] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.513542] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.517257] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.521001] ? pointer+0x110/0x3b0 >> <4>[ 463.524731] ? log_store+0xf5/0x260 >> <4>[ 463.528437] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.532208] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.535979] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.539701] do_writepages+0x41/0xd0 >> <4>[ 463.543413] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.547150] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.550888] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.554611] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >> <4>[ 463.558355] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.562091] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.565824] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.569569] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.573317] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.577052] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.580741] do_writepages+0x41/0xd0 >> <4>[ 463.584464] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.588216] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.591965] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.595696] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >> <4>[ 463.599454] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.603195] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.606931] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.610679] ? pointer+0x110/0x3b0 >> <4>[ 463.614406] ? log_store+0xf5/0x260 >> <4>[ 463.618115] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.621860] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.625598] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.629284] do_writepages+0x41/0xd0 >> <4>[ 463.632926] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.636607] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.640281] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.643949] ? up_read+0x5/0x20 >> <4>[ 463.647596] ? f2fs_do_write_data_page+0x2cb/0x940 >> <4>[ 463.651274] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.654945] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.658605] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.662281] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.665936] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.669582] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.673203] do_writepages+0x41/0xd0 >> <4>[ 463.676809] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.680434] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.684108] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.687775] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >> <4>[ 463.691461] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.695135] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.698813] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.702513] ? pointer+0x110/0x3b0 >> <4>[ 463.706187] ? log_store+0xf5/0x260 >> <4>[ 463.709837] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.713508] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.717171] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.720786] do_writepages+0x41/0xd0 >> <4>[ 463.724405] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.728046] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.731666] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.735258] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >> <4>[ 463.738885] f2fs_balance_fs+0xe5/0x2c0 >> <4>[ 463.742501] __write_data_page+0x5b7/0x6f0 >> <4>[ 463.746115] f2fs_write_cache_pages+0x1e0/0x450 >> <4>[ 463.749744] ? pointer+0x110/0x3b0 >> <4>[ 463.753345] ? log_store+0xf5/0x260 >> <4>[ 463.756920] f2fs_write_data_pages+0x2ed/0x320 >> <4>[ 463.760517] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.764105] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.767643] do_writepages+0x41/0xd0 >> <4>[ 463.771191] __filemap_fdatawrite_range+0x81/0xb0 >> <4>[ 463.774760] f2fs_sync_dirty_inodes+0x1dd/0x200 >> <4>[ 463.778328] f2fs_balance_fs_bg+0x2a7/0x2c0 >> <4>[ 463.781928] ? trace_hardirqs_on+0x2c/0xe0 >> <4>[ 463.785524] ? retint_kernel+0x2d/0x2d >> <4>[ 463.789100] f2fs_write_node_pages+0x51/0x220 >> <4>[ 463.792687] ? trace_hardirqs_on_caller+0x32/0xf0 >> <4>[ 463.796285] do_writepages+0x41/0xd0 >> <4>[ 463.799860] ? retint_kernel+0x2d/0x2d >> <4>[ 463.803417] __writeback_single_inode+0x3d/0x3d0 >> <4>[ 463.806999] writeback_sb_inodes+0x1e8/0x410 >> <4>[ 463.810573] __writeback_inodes_wb+0x5d/0xb0 >> <4>[ 463.814158] wb_writeback+0x28f/0x340 >> <4>[ 463.817730] wb_workfn+0x2d3/0x420 >> <4>[ 463.821328] process_one_work+0x1a1/0x3d0 >> <4>[ 463.824935] worker_thread+0x30/0x380 >> <4>[ 463.828520] ? process_one_work+0x3d0/0x3d0 >> <4>[ 463.832105] kthread+0x116/0x130 >> <4>[ 463.835675] ? kthread_create_worker_on_cpu+0x70/0x70 >> <4>[ 463.839278] ret_from_fork+0x3a/0x50 >> <4>[ 463.842855] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc >> nls_cp437 uas usb_storage pl2303 sd_mod x86_pkg_temp_thermal kvm_intel >> iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich >> firewire_ohci firewire_core igb crc_itu_t dca ahci libahci >> processor_thermal_device intel_soc_dts_iosf int340x_thermal_zone >> pcc_cpufreq efivarfs >> <4>[ 463.851196] ---[ end trace b6d0a43e90218d45 ]--- >> --END log two >> >>> >>> --- a/fs/f2fs/checkpoint.c >>> +++ b/fs/f2fs/checkpoint.c >>> @@ -1074,6 +1074,11 @@ int f2fs_sync_dirty_inodes(struct f2fs_sb_info >>> *sbi, >>> enum >>> inode_type type) >>> >>> F2FS_I(inode)->cp_task = current; >>> >>> + smp_mb(); >>> + >>> + printk("sync_dirty_inodes: inode:%lu, cp_task:%p", >>> + inode->i_ino, F2FS_I(inode)->cp_task); >>> + >>> filemap_fdatawrite(inode->i_mapping); >>> >>> F2FS_I(inode)->cp_task = NULL; >>> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c >>> index 88988241a0e8..84fbbd337c37 100644 >>> --- a/fs/f2fs/data.c >>> +++ b/fs/f2fs/data.c >>> @@ -2392,6 +2392,10 @@ int f2fs_write_single_data_page(struct page *page, >>> int >>> *submitted, >>> if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) && >>> !F2FS_I(inode)->cp_task) { >>> f2fs_submit_ipu_bio(sbi, bio, page); >>> + >>> + printk("write_data_page: inode:%lu, cp_task:%p", >>> + inode->i_ino, F2FS_I(inode)->cp_task); >>> + >>> f2fs_balance_fs(sbi, need_balance_fs); >>> } >>> >>> Thanks, >> > . > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-05-20 9:37 ` Chao Yu @ 2019-05-20 12:58 ` Hagbard Celine 2019-05-20 15:51 ` Chao Yu 0 siblings, 1 reply; 16+ messages in thread From: Hagbard Celine @ 2019-05-20 12:58 UTC (permalink / raw) To: Chao Yu; +Cc: linux-f2fs-devel 2019-05-20 11:37 GMT+02:00, Chao Yu <yuchao0@huawei.com>: > On 2019/5/16 1:01, Hagbard Celine wrote: >> 2019-05-15 18:50 GMT+02:00, Hagbard Celine <hagbardcelin@gmail.com>: >>> 2019-05-15 10:13 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >>>> On 2019/5/15 16:03, Hagbard Celine wrote: >>>>> 2019-05-15 4:25 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >>>>>> On 2019/5/15 2:13, Hagbard Celine wrote: >>>>>>> 2019-04-02 15:31 GMT+02:00, Chao Yu <chao@kernel.org>: >>>>>>>> On 2019-4-2 20:41, Hagbard Celine wrote: >>>>>>>>> That seems to have fixed it. No more errors in syslog after >>>>>>>>> extracting >>>>>>>>> my stage3 tarball. Also ran a couple of kernel compiles on a >>>>>>>>> partition >>>>>>>>> mounted with data_flush and system seems stable. >>>>>>>> >>>>>>>> Thanks a lot for your quick test. :) >>>>>>> >>>>>>> My test might have been a little too quick, or I found another >>>>>>> data_flush bug that behaves similar. >>>>>> >>>>>> oops... >>>>>> >>>>>>>>>> >>>>>>>>>> - if (is_dir) >>>>>>>>>> - F2FS_I(inode)->cp_task = current; >>>>>>>>>> + F2FS_I(inode)->cp_task = current; >>>>>> >>>>>> If you're sure that this patch was applying before you test, I guess >>>>>> we >>>>>> need >>>>>> an >>>>>> extra barrier here to avoid out-of-order execution. >>>>>> >>>>>> smp_mb() >>>>>> >>>>>>>>>> >>>>>>>>>> filemap_fdatawrite(inode->i_mapping); >>>>>>>>>> >>>>>>>>>> - if (is_dir) >>>>>>>>>> - F2FS_I(inode)->cp_task = NULL; >>>>>>>>>> + F2FS_I(inode)->cp_task = NULL; >>>>>> >>>>>> Thanks, >>>>>> >>>>> If I did this correctly; it did not get rid of the stack overflow. >>>>> Here is what I did: >>>>> >>>>> Added smb_mb() in checkpoint.c so the relevant part looks like this: >>>>> >>>>> unsigned long cur_ino = inode->i_ino; >>>>> >>>>> F2FS_I(inode)->cp_task = current; >>>>> >>>>> smp_mb(); >>>>> >>>>> filemap_fdatawrite(inode->i_mapping); >>>>> >>>>> F2FS_I(inode)->cp_task = NULL; >>>>> >>>>> iput(inode); >>>>> >>>>> >>>>> Compiled, rebooted and ran my test-scripts again. And got this during >>>>> copy-phase in second script: >>>> >>>> It looks very easy to reproduce this bug, could you add log to track >>>> F2FS_I(inode)->cp_task's value: >>> That wasn't so easy, with all the logging from those prink the copy >>> process would hang where it would oops without the printk's. >> >> Forgot in last mail that I actually had log from hang with both printk >> enabled also: > > Sorry for the delay. > > I found another two issues related to data_flush, could you try below fixing > patch? > > [PATCH] f2fs: fix to avoid deadloop if data_flush is on I ran several runs of my test scripts on with this new patch on top of kernel 5.0.15 with "[PATCH] f2fs: fix potential recursive call when enabling data_flush" and the extra smp_mb() in checkpoint.c. When that worked I did the same with this new patch on top of kernel 5.0.15 with "[PATCH] f2fs: fix potential recursive call when enabling data_flush" and _without_ the extra smp_mb() in checkpoint.c. In both cases I get no oops or hang. > > Thanks, > >> >> [ 194.681126] sync_dirty_inodes: inode:590309, cp_task:0000000013327ef9 >> [ 194.682258] sync_dirty_inodes: inode:590301, cp_task:0000000013327ef9 >> [ 194.682665] sync_dirty_inodes: inode:590311, cp_task:0000000013327ef9 >> [ 194.682703] sync_dirty_inodes: inode:590312, cp_task:0000000013327ef9 >> [ 194.682791] sync_dirty_inodes: inode:590313, cp_task:0000000013327ef9 >> [ 194.683566] sync_dirty_inodes: inode:590314, cp_task:0000000013327ef9 >> [ 194.683669] sync_dirty_inodes: inode:590315, cp_task:0000000013327ef9 >> [ 194.684829] sync_dirty_inodes: inode:590316, cp_task:0000000013327ef9 >> [ 194.712860] sync_dirty_inodes: inode:590317, cp_task:0000000013327ef9 >> [ 194.712908] sync_dirty_inodes: inode:590310, cp_task:0000000013327ef9 >> [ 194.713094] sync_dirty_inodes: inode:590319, cp_task:0000000013327ef9 >> [ 194.713348] sync_dirty_inodes: inode:590320, cp_task:0000000013327ef9 >> [ 194.713384] sync_dirty_inodes: inode:590321, cp_task:0000000013327ef9 >> [ 194.714634] sync_dirty_inodes: inode:590322, cp_task:0000000013327ef9 >> [ 194.715349] sync_dirty_inodes: inode:590323, cp_task:0000000013327ef9 >> [ 194.715381] sync_dirty_inodes: inode:590324, cp_task:0000000013327ef9 >> [ 194.718592] sync_dirty_inodes: inode:590326, cp_task:0000000013327ef9 >> [ 194.719217] sync_dirty_inodes: inode:590327, cp_task:0000000013327ef9 >> [ 194.719354] sync_dirty_inodes: inode:590328, cp_task:0000000013327ef9 >> [ 194.719903] sync_dirty_inodes: inode:590329, cp_task:0000000013327ef9 >> [ 194.720859] sync_dirty_inodes: inode:590521, cp_task:0000000013327ef9 >> [ 194.720868] sync_dirty_inodes: inode:590300, cp_task:0000000013327ef9 >> [ 194.720985] sync_dirty_inodes: inode:590523, cp_task:0000000013327ef9 >> [ 194.738075] sync_dirty_inodes: inode:591528, cp_task:0000000013327ef9 >> [ 194.738168] sync_dirty_inodes: inode:591529, cp_task:0000000013327ef9 >> [ 194.738190] sync_dirty_inodes: inode:591533, cp_task:0000000013327ef9 >> [ 194.738263] sync_dirty_inodes: inode:591537, cp_task:0000000013327ef9 >> [ 194.738275] sync_dirty_inodes: inode:591538, cp_task:0000000013327ef9 >> [ 194.738307] sync_dirty_inodes: inode:591540, cp_task:0000000013327ef9 >> [ 194.738312] sync_dirty_inodes: inode:591542, cp_task:0000000013327ef9 >> [ 194.759368] sync_dirty_inodes: inode:593095, cp_task:0000000013327ef9 >> [ 194.759587] sync_dirty_inodes: inode:593351, cp_task:0000000013327ef9 >> [ 194.759797] sync_dirty_inodes: inode:593352, cp_task:0000000013327ef9 >> [ 194.760555] sync_dirty_inodes: inode:593353, cp_task:0000000013327ef9 >> [ 194.760562] sync_dirty_inodes: inode:593354, cp_task:0000000013327ef9 >> [ 194.761168] sync_dirty_inodes: inode:593355, cp_task:0000000013327ef9 >> [ 194.761207] sync_dirty_inodes: inode:593356, cp_task:0000000013327ef9 >> [ 194.761234] sync_dirty_inodes: inode:593357, cp_task:0000000013327ef9 >> [ 194.761251] sync_dirty_inodes: inode:593358, cp_task:0000000013327ef9 >> [ 194.761285] sync_dirty_inodes: inode:593360, cp_task:0000000013327ef9 >> [ 194.761290] sync_dirty_inodes: inode:593361, cp_task:0000000013327ef9 >> [ 194.763583] sync_dirty_inodes: inode:593363, cp_task:0000000013327ef9 >> [ 194.763845] sync_dirty_inodes: inode:593362, cp_task:0000000013327ef9 >> [ 194.763891] sync_dirty_inodes: inode:593367, cp_task:0000000013327ef9 >> [ 194.763992] sync_dirty_inodes: inode:593368, cp_task:0000000013327ef9 >> [ 194.764073] sync_dirty_inodes: inode:593369, cp_task:0000000013327ef9 >> [ 194.764090] sync_dirty_inodes: inode:593370, cp_task:0000000013327ef9 >> [ 194.764165] sync_dirty_inodes: inode:593371, cp_task:0000000013327ef9 >> [ 194.764185] sync_dirty_inodes: inode:593372, cp_task:0000000013327ef9 >> [ 194.764214] sync_dirty_inodes: inode:593365, cp_task:0000000013327ef9 >> [ 194.764323] sync_dirty_inodes: inode:593374, cp_task:0000000013327ef9 >> [ 194.764419] sync_dirty_inodes: inode:593373, cp_task:0000000013327ef9 >> [ 194.764512] sync_dirty_inodes: inode:593375, cp_task:0000000013327ef9 >> [ 194.764533] sync_dirty_inodes: inode:593376, cp_task:0000000013327ef9 >> [ 194.764608] sync_dirty_inodes: inode:593377, cp_task:0000000013327ef9 >> [ 194.764857] sync_dirty_inodes: inode:593378, cp_task:0000000013327ef9 >> [ 194.764884] sync_dirty_inodes: inode:593379, cp_task:0000000013327ef9 >> [ 194.765164] sync_dirty_inodes: inode:593380, cp_task:0000000013327ef9 >> [ 194.765210] sync_dirty_inodes: inode:593381, cp_task:0000000013327ef9 >> [ 194.765286] sync_dirty_inodes: inode:593382, cp_task:0000000013327ef9 >> [ 194.765513] sync_dirty_inodes: inode:593383, cp_task:0000000013327ef9 >> [ 194.765671] sync_dirty_inodes: inode:593384, cp_task:0000000013327ef9 >> [ 194.765777] sync_dirty_inodes: inode:593385, cp_task:0000000013327ef9 >> [ 194.766079] sync_dirty_inodes: inode:593386, cp_task:0000000013327ef9 >> [ 194.766568] sync_dirty_inodes: inode:593387, cp_task:0000000013327ef9 >> [ 194.766628] sync_dirty_inodes: inode:593388, cp_task:0000000013327ef9 >> [ 194.766742] sync_dirty_inodes: inode:593389, cp_task:0000000013327ef9 >> [ 194.766750] sync_dirty_inodes: inode:593390, cp_task:0000000013327ef9 >> [ 194.766788] sync_dirty_inodes: inode:593391, cp_task:0000000013327ef9 >> [ 194.766806] sync_dirty_inodes: inode:593392, cp_task:0000000013327ef9 >> [ 194.766856] sync_dirty_inodes: inode:593364, cp_task:0000000013327ef9 >> [ 194.767016] sync_dirty_inodes: inode:593394, cp_task:0000000013327ef9 >> [ 194.767033] sync_dirty_inodes: inode:593395, cp_task:0000000013327ef9 >> [ 194.768454] sync_dirty_inodes: inode:593393, cp_task:0000000013327ef9 >> [ 194.768651] sync_dirty_inodes: inode:593397, cp_task:0000000013327ef9 >> [ 194.768665] sync_dirty_inodes: inode:593398, cp_task:0000000013327ef9 >> [ 194.768678] sync_dirty_inodes: inode:593399, cp_task:0000000013327ef9 >> [ 194.768705] sync_dirty_inodes: inode:593400, cp_task:0000000013327ef9 >> [ 194.768869] sync_dirty_inodes: inode:593401, cp_task:0000000013327ef9 >> [ 194.768945] sync_dirty_inodes: inode:593402, cp_task:0000000013327ef9 >> [ 194.769081] sync_dirty_inodes: inode:593403, cp_task:0000000013327ef9 >> [ 194.769106] sync_dirty_inodes: inode:593404, cp_task:0000000013327ef9 >> [ 194.769198] sync_dirty_inodes: inode:593396, cp_task:0000000013327ef9 >> [ 194.769297] sync_dirty_inodes: inode:593405, cp_task:0000000013327ef9 >> [ 194.769390] sync_dirty_inodes: inode:593407, cp_task:0000000013327ef9 >> [ 194.769670] sync_dirty_inodes: inode:593408, cp_task:0000000013327ef9 >> [ 194.769685] sync_dirty_inodes: inode:593409, cp_task:0000000013327ef9 >> [ 194.769699] sync_dirty_inodes: inode:593410, cp_task:0000000013327ef9 >> [ 194.769716] sync_dirty_inodes: inode:593411, cp_task:0000000013327ef9 >> [ 194.769738] sync_dirty_inodes: inode:593412, cp_task:0000000013327ef9 >> [ 194.770032] sync_dirty_inodes: inode:593413, cp_task:0000000013327ef9 >> [ 194.770074] sync_dirty_inodes: inode:593414, cp_task:0000000013327ef9 >> [ 194.770157] sync_dirty_inodes: inode:593415, cp_task:0000000013327ef9 >> [ 194.770185] sync_dirty_inodes: inode:593416, cp_task:0000000013327ef9 >> [ 194.778086] sync_dirty_inodes: inode:593406, cp_task:0000000013327ef9 >> [ 194.778239] sync_dirty_inodes: inode:593417, cp_task:0000000013327ef9 >> [ 194.778322] sync_dirty_inodes: inode:593418, cp_task:0000000013327ef9 >> [ 194.778484] sync_dirty_inodes: inode:593419, cp_task:0000000013327ef9 >> [ 194.778562] sync_dirty_inodes: inode:593420, cp_task:0000000013327ef9 >> [ 194.778621] sync_dirty_inodes: inode:593421, cp_task:0000000013327ef9 >> [ 194.778635] sync_dirty_inodes: inode:593422, cp_task:0000000013327ef9 >> [ 194.778661] sync_dirty_inodes: inode:593423, cp_task:0000000013327ef9 >> [ 194.779162] sync_dirty_inodes: inode:593424, cp_task:0000000013327ef9 >> [ 194.779227] sync_dirty_inodes: inode:593425, cp_task:0000000013327ef9 >> [ 194.781339] sync_dirty_inodes: inode:593427, cp_task:0000000013327ef9 >> [ 194.841045] write_data_page: inode:764980, cp_task: (null) >> [ 194.861488] sync_dirty_inodes: inode:593948, cp_task:0000000013327ef9 >> [ 194.941948] sync_dirty_inodes: inode:593948, cp_task:0000000045fd0ed6 >> [ 194.958797] sync_dirty_inodes: inode:593948, cp_task:0000000013327ef9 >> [ 360.534693] usb 3-3: USB disconnect, device number 2 >> [ 369.937827] INFO: task kworker/u16:5:344 blocked for more than 120 >> seconds. >> [ 369.937829] Not tainted 5.0.15-gentoo-f2fslog #4 >> [ 369.937830] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >> disables this message. >> [ 369.937831] kworker/u16:5 D 0 344 2 0x80000000 >> [ 369.937836] Workqueue: writeback wb_workfn (flush-259:0) >> [ 369.937838] Call Trace: >> [ 369.937843] ? __schedule+0x2c5/0x8b0 >> [ 369.937846] schedule+0x32/0x80 >> [ 369.937848] schedule_preempt_disabled+0x14/0x20 >> [ 369.937850] __mutex_lock.isra.8+0x2ba/0x4d0 >> [ 369.937853] ? log_store+0xf5/0x260 >> [ 369.937857] f2fs_write_data_pages+0x133/0x320 >> [ 369.937860] ? trace_hardirqs_on+0x2c/0xe0 >> [ 369.937863] do_writepages+0x41/0xd0 >> [ 369.937866] __filemap_fdatawrite_range+0x81/0xb0 >> [ 369.937869] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 369.937871] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 369.937874] ? up_read+0x5/0x20 >> [ 369.937876] ? f2fs_do_write_data_page+0x2cb/0x940 >> [ 369.937877] f2fs_balance_fs+0xe5/0x2c0 >> [ 369.937880] __write_data_page+0x1c8/0x6e0 >> [ 369.937884] f2fs_write_cache_pages+0x1e0/0x450 >> [ 369.937889] f2fs_write_data_pages+0x14b/0x320 >> [ 369.937891] ? trace_hardirqs_on+0x2c/0xe0 >> [ 369.937894] do_writepages+0x41/0xd0 >> [ 369.937896] __filemap_fdatawrite_range+0x81/0xb0 >> [ 369.937898] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 369.937900] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 369.937902] f2fs_write_node_pages+0x51/0x220 >> [ 369.937905] do_writepages+0x41/0xd0 >> [ 369.937908] __writeback_single_inode+0x3d/0x3d0 >> [ 369.937910] writeback_sb_inodes+0x1e8/0x410 >> [ 369.937913] __writeback_inodes_wb+0x5d/0xb0 >> [ 369.937915] wb_writeback+0x28f/0x340 >> [ 369.937918] ? cpumask_next+0x16/0x20 >> [ 369.937920] wb_workfn+0x33e/0x420 >> [ 369.937923] process_one_work+0x1a1/0x3d0 >> [ 369.937925] worker_thread+0x30/0x380 >> [ 369.937927] ? process_one_work+0x3d0/0x3d0 >> [ 369.937929] kthread+0x116/0x130 >> [ 369.937931] ? kthread_create_worker_on_cpu+0x70/0x70 >> [ 369.937933] ret_from_fork+0x3a/0x50 >> [ 369.937946] INFO: task cp:2049 blocked for more than 120 seconds. >> [ 369.937946] Not tainted 5.0.15-gentoo-f2fslog #4 >> [ 369.937947] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >> disables this message. >> [ 369.937948] cp D 0 2049 2029 0x00000000 >> [ 369.937949] Call Trace: >> [ 369.937952] ? __schedule+0x2c5/0x8b0 >> [ 369.937954] schedule+0x32/0x80 >> [ 369.937956] schedule_preempt_disabled+0x14/0x20 >> [ 369.937958] __mutex_lock.isra.8+0x2ba/0x4d0 >> [ 369.937960] ? log_store+0xf5/0x260 >> [ 369.937963] f2fs_write_data_pages+0x133/0x320 >> [ 369.937965] ? trace_hardirqs_on+0x2c/0xe0 >> [ 369.937968] do_writepages+0x41/0xd0 >> [ 369.937971] __filemap_fdatawrite_range+0x81/0xb0 >> [ 369.937973] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 369.937975] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 369.937977] f2fs_balance_fs+0xe5/0x2c0 >> [ 369.937980] f2fs_mkdir+0x124/0x160 >> [ 369.937982] vfs_mkdir+0x102/0x1b0 >> [ 369.937985] do_mkdirat+0x7d/0xf0 >> [ 369.937987] do_syscall_64+0x69/0x399 >> [ 369.937989] ? trace_hardirqs_off_thunk+0x1a/0x1c >> [ 369.937992] entry_SYSCALL_64_after_hwframe+0x49/0xbe >> [ 369.937994] RIP: 0033:0x717f2eaa603b >> [ 369.937998] Code: Bad RIP value. >> [ 369.937999] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: >> 0000000000000053 >> [ 369.938001] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: >> 0000717f2eaa603b >> [ 369.938002] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: >> 00005b54b1a17ff0 >> [ 369.938003] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: >> 00007ffd8a71cf80 >> [ 369.938004] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: >> 00000000000001ed >> [ 369.938005] R13: 0000000000000000 R14: 0000000000004000 R15: >> 0000000000000000 >> [ 492.817796] INFO: task kworker/u16:5:344 blocked for more than 120 >> seconds. >> [ 492.817797] Not tainted 5.0.15-gentoo-f2fslog #4 >> [ 492.817797] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >> disables this message. >> [ 492.817798] kworker/u16:5 D 0 344 2 0x80000000 >> [ 492.817810] Workqueue: writeback wb_workfn (flush-259:0) >> [ 492.817811] Call Trace: >> [ 492.817813] ? __schedule+0x2c5/0x8b0 >> [ 492.817815] schedule+0x32/0x80 >> [ 492.817816] schedule_preempt_disabled+0x14/0x20 >> [ 492.817817] __mutex_lock.isra.8+0x2ba/0x4d0 >> [ 492.817819] ? log_store+0xf5/0x260 >> [ 492.817821] f2fs_write_data_pages+0x133/0x320 >> [ 492.817823] ? trace_hardirqs_on+0x2c/0xe0 >> [ 492.817825] do_writepages+0x41/0xd0 >> [ 492.817826] __filemap_fdatawrite_range+0x81/0xb0 >> [ 492.817828] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 492.817829] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 492.817831] ? up_read+0x5/0x20 >> [ 492.817832] ? f2fs_do_write_data_page+0x2cb/0x940 >> [ 492.817833] f2fs_balance_fs+0xe5/0x2c0 >> [ 492.817835] __write_data_page+0x1c8/0x6e0 >> [ 492.817837] f2fs_write_cache_pages+0x1e0/0x450 >> [ 492.817840] f2fs_write_data_pages+0x14b/0x320 >> [ 492.817842] ? trace_hardirqs_on+0x2c/0xe0 >> [ 492.817843] do_writepages+0x41/0xd0 >> [ 492.817845] __filemap_fdatawrite_range+0x81/0xb0 >> [ 492.817846] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 492.817847] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 492.817849] f2fs_write_node_pages+0x51/0x220 >> [ 492.817851] do_writepages+0x41/0xd0 >> [ 492.817853] __writeback_single_inode+0x3d/0x3d0 >> [ 492.817854] writeback_sb_inodes+0x1e8/0x410 >> [ 492.817856] __writeback_inodes_wb+0x5d/0xb0 >> [ 492.817857] wb_writeback+0x28f/0x340 >> [ 492.817859] ? cpumask_next+0x16/0x20 >> [ 492.817860] wb_workfn+0x33e/0x420 >> [ 492.817862] process_one_work+0x1a1/0x3d0 >> [ 492.817863] worker_thread+0x30/0x380 >> [ 492.817865] ? process_one_work+0x3d0/0x3d0 >> [ 492.817865] kthread+0x116/0x130 >> [ 492.817866] ? kthread_create_worker_on_cpu+0x70/0x70 >> [ 492.817868] ret_from_fork+0x3a/0x50 >> [ 492.817873] INFO: task cp:2049 blocked for more than 120 seconds. >> [ 492.817873] Not tainted 5.0.15-gentoo-f2fslog #4 >> [ 492.817874] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >> disables this message. >> [ 492.817874] cp D 0 2049 2029 0x00000000 >> [ 492.817875] Call Trace: >> [ 492.817877] ? __schedule+0x2c5/0x8b0 >> [ 492.817878] schedule+0x32/0x80 >> [ 492.817879] schedule_preempt_disabled+0x14/0x20 >> [ 492.817880] __mutex_lock.isra.8+0x2ba/0x4d0 >> [ 492.817882] ? log_store+0xf5/0x260 >> [ 492.817884] f2fs_write_data_pages+0x133/0x320 >> [ 492.817886] ? trace_hardirqs_on+0x2c/0xe0 >> [ 492.817887] do_writepages+0x41/0xd0 >> [ 492.817889] __filemap_fdatawrite_range+0x81/0xb0 >> [ 492.817890] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 492.817892] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 492.817893] f2fs_balance_fs+0xe5/0x2c0 >> [ 492.817894] f2fs_mkdir+0x124/0x160 >> [ 492.817896] vfs_mkdir+0x102/0x1b0 >> [ 492.817897] do_mkdirat+0x7d/0xf0 >> [ 492.817899] do_syscall_64+0x69/0x399 >> [ 492.817900] ? trace_hardirqs_off_thunk+0x1a/0x1c >> [ 492.817902] entry_SYSCALL_64_after_hwframe+0x49/0xbe >> [ 492.817903] RIP: 0033:0x717f2eaa603b >> [ 492.817905] Code: Bad RIP value. >> [ 492.817906] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: >> 0000000000000053 >> [ 492.817907] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: >> 0000717f2eaa603b >> [ 492.817908] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: >> 00005b54b1a17ff0 >> [ 492.817908] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: >> 00007ffd8a71cf80 >> [ 492.817909] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: >> 00000000000001ed >> [ 492.817910] R13: 0000000000000000 R14: 0000000000004000 R15: >> 0000000000000000 >> [ 615.697824] INFO: task kworker/u16:5:344 blocked for more than 120 >> seconds. >> [ 615.697825] Not tainted 5.0.15-gentoo-f2fslog #4 >> [ 615.697826] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >> disables this message. >> [ 615.697827] kworker/u16:5 D 0 344 2 0x80000000 >> [ 615.697831] Workqueue: writeback wb_workfn (flush-259:0) >> [ 615.697832] Call Trace: >> [ 615.697836] ? __schedule+0x2c5/0x8b0 >> [ 615.697839] schedule+0x32/0x80 >> [ 615.697841] schedule_preempt_disabled+0x14/0x20 >> [ 615.697842] __mutex_lock.isra.8+0x2ba/0x4d0 >> [ 615.697845] ? log_store+0xf5/0x260 >> [ 615.697848] f2fs_write_data_pages+0x133/0x320 >> [ 615.697851] ? trace_hardirqs_on+0x2c/0xe0 >> [ 615.697854] do_writepages+0x41/0xd0 >> [ 615.697857] __filemap_fdatawrite_range+0x81/0xb0 >> [ 615.697859] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 615.697861] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 615.697863] ? up_read+0x5/0x20 >> [ 615.697865] ? f2fs_do_write_data_page+0x2cb/0x940 >> [ 615.697867] f2fs_balance_fs+0xe5/0x2c0 >> [ 615.697869] __write_data_page+0x1c8/0x6e0 >> [ 615.697873] f2fs_write_cache_pages+0x1e0/0x450 >> [ 615.697878] f2fs_write_data_pages+0x14b/0x320 >> [ 615.697880] ? trace_hardirqs_on+0x2c/0xe0 >> [ 615.697883] do_writepages+0x41/0xd0 >> [ 615.697885] __filemap_fdatawrite_range+0x81/0xb0 >> [ 615.697887] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 615.697889] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 615.697891] f2fs_write_node_pages+0x51/0x220 >> [ 615.697894] do_writepages+0x41/0xd0 >> [ 615.697897] __writeback_single_inode+0x3d/0x3d0 >> [ 615.697899] writeback_sb_inodes+0x1e8/0x410 >> [ 615.697902] __writeback_inodes_wb+0x5d/0xb0 >> [ 615.697904] wb_writeback+0x28f/0x340 >> [ 615.697906] ? cpumask_next+0x16/0x20 >> [ 615.697908] wb_workfn+0x33e/0x420 >> [ 615.697911] process_one_work+0x1a1/0x3d0 >> [ 615.697913] worker_thread+0x30/0x380 >> [ 615.697915] ? process_one_work+0x3d0/0x3d0 >> [ 615.697916] kthread+0x116/0x130 >> [ 615.697918] ? kthread_create_worker_on_cpu+0x70/0x70 >> [ 615.697921] ret_from_fork+0x3a/0x50 >> [ 615.697927] INFO: task cp:2049 blocked for more than 120 seconds. >> [ 615.697928] Not tainted 5.0.15-gentoo-f2fslog #4 >> [ 615.697929] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >> disables this message. >> [ 615.697929] cp D 0 2049 2029 0x00000000 >> [ 615.697931] Call Trace: >> [ 615.697933] ? __schedule+0x2c5/0x8b0 >> [ 615.697935] schedule+0x32/0x80 >> [ 615.697937] schedule_preempt_disabled+0x14/0x20 >> [ 615.697939] __mutex_lock.isra.8+0x2ba/0x4d0 >> [ 615.697942] ? log_store+0xf5/0x260 >> [ 615.697944] f2fs_write_data_pages+0x133/0x320 >> [ 615.697947] ? trace_hardirqs_on+0x2c/0xe0 >> [ 615.697950] do_writepages+0x41/0xd0 >> [ 615.697952] __filemap_fdatawrite_range+0x81/0xb0 >> [ 615.697954] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 615.697956] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 615.697958] f2fs_balance_fs+0xe5/0x2c0 >> [ 615.697961] f2fs_mkdir+0x124/0x160 >> [ 615.697963] vfs_mkdir+0x102/0x1b0 >> [ 615.697965] do_mkdirat+0x7d/0xf0 >> [ 615.697968] do_syscall_64+0x69/0x399 >> [ 615.697969] ? trace_hardirqs_off_thunk+0x1a/0x1c >> [ 615.697972] entry_SYSCALL_64_after_hwframe+0x49/0xbe >> [ 615.697974] RIP: 0033:0x717f2eaa603b >> [ 615.697978] Code: Bad RIP value. >> [ 615.697979] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: >> 0000000000000053 >> [ 615.697980] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: >> 0000717f2eaa603b >> [ 615.697981] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: >> 00005b54b1a17ff0 >> [ 615.697982] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: >> 00007ffd8a71cf80 >> [ 615.697983] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: >> 00000000000001ed >> [ 615.697984] R13: 0000000000000000 R14: 0000000000004000 R15: >> 0000000000000000 >> [ 738.577796] INFO: task kworker/u16:5:344 blocked for more than 120 >> seconds. >> [ 738.577798] Not tainted 5.0.15-gentoo-f2fslog #4 >> [ 738.577798] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >> disables this message. >> [ 738.577799] kworker/u16:5 D 0 344 2 0x80000000 >> [ 738.577812] Workqueue: writeback wb_workfn (flush-259:0) >> [ 738.577812] Call Trace: >> [ 738.577815] ? __schedule+0x2c5/0x8b0 >> [ 738.577817] schedule+0x32/0x80 >> [ 738.577818] schedule_preempt_disabled+0x14/0x20 >> [ 738.577819] __mutex_lock.isra.8+0x2ba/0x4d0 >> [ 738.577821] ? log_store+0xf5/0x260 >> [ 738.577823] f2fs_write_data_pages+0x133/0x320 >> [ 738.577825] ? trace_hardirqs_on+0x2c/0xe0 >> [ 738.577827] do_writepages+0x41/0xd0 >> [ 738.577829] __filemap_fdatawrite_range+0x81/0xb0 >> [ 738.577830] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 738.577832] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 738.577833] ? up_read+0x5/0x20 >> [ 738.577834] ? f2fs_do_write_data_page+0x2cb/0x940 >> [ 738.577835] f2fs_balance_fs+0xe5/0x2c0 >> [ 738.577837] __write_data_page+0x1c8/0x6e0 >> [ 738.577839] f2fs_write_cache_pages+0x1e0/0x450 >> [ 738.577842] f2fs_write_data_pages+0x14b/0x320 >> [ 738.577844] ? trace_hardirqs_on+0x2c/0xe0 >> [ 738.577846] do_writepages+0x41/0xd0 >> [ 738.577847] __filemap_fdatawrite_range+0x81/0xb0 >> [ 738.577849] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 738.577850] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 738.577851] f2fs_write_node_pages+0x51/0x220 >> [ 738.577853] do_writepages+0x41/0xd0 >> [ 738.577855] __writeback_single_inode+0x3d/0x3d0 >> [ 738.577856] writeback_sb_inodes+0x1e8/0x410 >> [ 738.577858] __writeback_inodes_wb+0x5d/0xb0 >> [ 738.577859] wb_writeback+0x28f/0x340 >> [ 738.577861] ? cpumask_next+0x16/0x20 >> [ 738.577862] wb_workfn+0x33e/0x420 >> [ 738.577864] process_one_work+0x1a1/0x3d0 >> [ 738.577865] worker_thread+0x30/0x380 >> [ 738.577867] ? process_one_work+0x3d0/0x3d0 >> [ 738.577867] kthread+0x116/0x130 >> [ 738.577868] ? kthread_create_worker_on_cpu+0x70/0x70 >> [ 738.577870] ret_from_fork+0x3a/0x50 >> [ 738.577875] INFO: task cp:2049 blocked for more than 120 seconds. >> [ 738.577875] Not tainted 5.0.15-gentoo-f2fslog #4 >> [ 738.577875] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >> disables this message. >> [ 738.577876] cp D 0 2049 2029 0x00000000 >> [ 738.577877] Call Trace: >> [ 738.577878] ? __schedule+0x2c5/0x8b0 >> [ 738.577880] schedule+0x32/0x80 >> [ 738.577881] schedule_preempt_disabled+0x14/0x20 >> [ 738.577882] __mutex_lock.isra.8+0x2ba/0x4d0 >> [ 738.577884] ? log_store+0xf5/0x260 >> [ 738.577886] f2fs_write_data_pages+0x133/0x320 >> [ 738.577887] ? trace_hardirqs_on+0x2c/0xe0 >> [ 738.577889] do_writepages+0x41/0xd0 >> [ 738.577891] __filemap_fdatawrite_range+0x81/0xb0 >> [ 738.577892] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 738.577893] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 738.577895] f2fs_balance_fs+0xe5/0x2c0 >> [ 738.577896] f2fs_mkdir+0x124/0x160 >> [ 738.577898] vfs_mkdir+0x102/0x1b0 >> [ 738.577899] do_mkdirat+0x7d/0xf0 >> [ 738.577901] do_syscall_64+0x69/0x399 >> [ 738.577902] ? trace_hardirqs_off_thunk+0x1a/0x1c >> [ 738.577904] entry_SYSCALL_64_after_hwframe+0x49/0xbe >> [ 738.577905] RIP: 0033:0x717f2eaa603b >> [ 738.577907] Code: Bad RIP value. >> [ 738.577908] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: >> 0000000000000053 >> [ 738.577909] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: >> 0000717f2eaa603b >> [ 738.577909] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: >> 00005b54b1a17ff0 >> [ 738.577910] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: >> 00007ffd8a71cf80 >> [ 738.577910] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: >> 00000000000001ed >> [ 738.577911] R13: 0000000000000000 R14: 0000000000004000 R15: >> 0000000000000000 >> [ 861.457820] INFO: task kworker/u16:5:344 blocked for more than 120 >> seconds. >> [ 861.457823] Not tainted 5.0.15-gentoo-f2fslog #4 >> [ 861.457823] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >> disables this message. >> [ 861.457825] kworker/u16:5 D 0 344 2 0x80000000 >> [ 861.457830] Workqueue: writeback wb_workfn (flush-259:0) >> [ 861.457832] Call Trace: >> [ 861.457837] ? __schedule+0x2c5/0x8b0 >> [ 861.457841] schedule+0x32/0x80 >> [ 861.457843] schedule_preempt_disabled+0x14/0x20 >> [ 861.457846] __mutex_lock.isra.8+0x2ba/0x4d0 >> [ 861.457850] ? log_store+0xf5/0x260 >> [ 861.457854] f2fs_write_data_pages+0x133/0x320 >> [ 861.457858] ? trace_hardirqs_on+0x2c/0xe0 >> [ 861.457862] do_writepages+0x41/0xd0 >> [ 861.457865] __filemap_fdatawrite_range+0x81/0xb0 >> [ 861.457869] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 861.457872] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 861.457874] ? up_read+0x5/0x20 >> [ 861.457877] ? f2fs_do_write_data_page+0x2cb/0x940 >> [ 861.457879] f2fs_balance_fs+0xe5/0x2c0 >> [ 861.457883] __write_data_page+0x1c8/0x6e0 >> [ 861.457888] f2fs_write_cache_pages+0x1e0/0x450 >> [ 861.457894] f2fs_write_data_pages+0x14b/0x320 >> [ 861.457898] ? trace_hardirqs_on+0x2c/0xe0 >> [ 861.457901] do_writepages+0x41/0xd0 >> [ 861.457904] __filemap_fdatawrite_range+0x81/0xb0 >> [ 861.457908] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 861.457910] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 861.457913] f2fs_write_node_pages+0x51/0x220 >> [ 861.457917] do_writepages+0x41/0xd0 >> [ 861.457921] __writeback_single_inode+0x3d/0x3d0 >> [ 861.457924] writeback_sb_inodes+0x1e8/0x410 >> [ 861.457928] __writeback_inodes_wb+0x5d/0xb0 >> [ 861.457930] wb_writeback+0x28f/0x340 >> [ 861.457934] ? cpumask_next+0x16/0x20 >> [ 861.457936] wb_workfn+0x33e/0x420 >> [ 861.457940] process_one_work+0x1a1/0x3d0 >> [ 861.457943] worker_thread+0x30/0x380 >> [ 861.457946] ? process_one_work+0x3d0/0x3d0 >> [ 861.457948] kthread+0x116/0x130 >> [ 861.457950] ? kthread_create_worker_on_cpu+0x70/0x70 >> [ 861.457953] ret_from_fork+0x3a/0x50 >> [ 861.457962] INFO: task cp:2049 blocked for more than 120 seconds. >> [ 861.457963] Not tainted 5.0.15-gentoo-f2fslog #4 >> [ 861.457964] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >> disables this message. >> [ 861.457965] cp D 0 2049 2029 0x00000000 >> [ 861.457967] Call Trace: >> [ 861.457970] ? __schedule+0x2c5/0x8b0 >> [ 861.457974] schedule+0x32/0x80 >> [ 861.457976] schedule_preempt_disabled+0x14/0x20 >> [ 861.457978] __mutex_lock.isra.8+0x2ba/0x4d0 >> [ 861.457982] ? log_store+0xf5/0x260 >> [ 861.457986] f2fs_write_data_pages+0x133/0x320 >> [ 861.457990] ? trace_hardirqs_on+0x2c/0xe0 >> [ 861.457993] do_writepages+0x41/0xd0 >> [ 861.457996] __filemap_fdatawrite_range+0x81/0xb0 >> [ 861.458000] f2fs_sync_dirty_inodes+0x1dd/0x200 >> [ 861.458002] f2fs_balance_fs_bg+0x2a7/0x2c0 >> [ 861.458005] f2fs_balance_fs+0xe5/0x2c0 >> [ 861.458009] f2fs_mkdir+0x124/0x160 >> [ 861.458012] vfs_mkdir+0x102/0x1b0 >> [ 861.458015] do_mkdirat+0x7d/0xf0 >> [ 861.458018] do_syscall_64+0x69/0x399 >> [ 861.458020] ? trace_hardirqs_off_thunk+0x1a/0x1c >> [ 861.458025] entry_SYSCALL_64_after_hwframe+0x49/0xbe >> [ 861.458027] RIP: 0033:0x717f2eaa603b >> [ 861.458032] Code: Bad RIP value. >> [ 861.458033] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: >> 0000000000000053 >> [ 861.458035] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: >> 0000717f2eaa603b >> [ 861.458036] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: >> 00005b54b1a17ff0 >> [ 861.458038] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: >> 00007ffd8a71cf80 >> [ 861.458039] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: >> 00000000000001ed >> [ 861.458040] R13: 0000000000000000 R14: 0000000000004000 R15: >> 0000000000000000 >> >> >>> I was able to reproduse the bug with one of the two printk's at a time >>> only, and I had to disable syslog-ng and fcron for it not to hang even >>> then. >>> >>> Followig is log from two runs, one with each of the printk's, hope it >>> helps. >>> >>> --BEGIN log one >>> <4>[ 593.806592] write_data_page: inode:710085, cp_task: >>> (null) >>> <4>[ 593.806688] write_data_page: inode:710110, cp_task: >>> (null) >>> <4>[ 593.808558] write_data_page: inode:710321, cp_task: >>> (null) >>> <4>[ 593.808575] write_data_page: inode:710325, cp_task: >>> (null) >>> <4>[ 593.808590] write_data_page: inode:710326, cp_task: >>> (null) >>> <4>[ 593.808606] write_data_page: inode:710332, cp_task: >>> (null) >>> <4>[ 593.966185] write_data_page: inode:721775, cp_task: >>> (null) >>> <4>[ 593.966203] write_data_page: inode:721776, cp_task: >>> (null) >>> <4>[ 593.966219] write_data_page: inode:721777, cp_task: >>> (null) >>> <4>[ 593.966235] write_data_page: inode:721778, cp_task: >>> (null) >>> <4>[ 593.966250] write_data_page: inode:721779, cp_task: >>> (null) >>> <4>[ 593.966266] write_data_page: inode:721780, cp_task: >>> (null) >>> <4>[ 593.966281] write_data_page: inode:721781, cp_task: >>> (null) >>> <4>[ 593.966296] write_data_page: inode:721782, cp_task: >>> (null) >>> <4>[ 593.966311] write_data_page: inode:721783, cp_task: >>> (null) >>> <4>[ 593.966327] write_data_page: inode:721784, cp_task: >>> (null) >>> <4>[ 593.966343] write_data_page: inode:721785, cp_task: >>> (null) >>> <4>[ 593.966359] write_data_page: inode:721786, cp_task: >>> (null) >>> <4>[ 593.966374] write_data_page: inode:721787, cp_task: >>> (null) >>> <4>[ 594.340072] write_data_page: inode:746183, cp_task: >>> (null) >>> <0>[ 594.923096] BUG: stack guard page was hit at 000000006e7354a5 >>> (stack is 000000006445beb4..00000000988529ca) >>> <0>[ 594.923108] BUG: stack guard page was hit at 00000000d2c9ec98 >>> (stack is 00000000b417d4d3..000000001b88c4fe) >>> <4>[ 594.926975] kernel stack overflow (double-fault): 0000 [#1] >>> PREEMPT SMP PTI >>> <4>[ 594.934772] CPU: 7 PID: 2158 Comm: cp Not tainted >>> 5.0.15-gentoo-f2fslog_data_page #5 >>> <4>[ 594.938757] Hardware name: To Be Filled By O.E.M. To Be Filled >>> By O.E.M./C226 WS, BIOS P3.40 06/25/2018 >>> <4>[ 594.942880] RIP: 0010:tag_pages_for_writeback+0x0/0x180 >>> <4>[ 594.947036] Code: 30 48 83 ff 07 0f 86 eb fe ff ff 49 8b 56 40 >>> 49 03 56 38 48 d1 ea 49 89 d0 e9 6a fe ff ff 66 66 2e 0f 1f 84 00 00 >>> 00 00 00 90 <e8> 2b 41 9f 00 41 54 48 83 c7 08 55 48 89 d5 53 48 83 ec >>> 38 65 48 >>> <4>[ 594.951623] RSP: 0018:ffffb3f0028dc000 EFLAGS: 00010246 >>> <4>[ 594.956187] RAX: 0000000000000001 RBX: ffff9ea2d6bd7080 RCX: >>> ffffb3f0028dc060 >>> <4>[ 594.960816] RDX: 0007ffffffffffff RSI: 0000000000000000 RDI: >>> ffff9ea2d6bd71e0 >>> <4>[ 594.965375] RBP: ffffb3f0028dc1f0 R08: ffff9ea43eee0100 R09: >>> 0000000000000000 >>> <4>[ 594.969873] R10: ffffffff97e485c0 R11: 0000000000000001 R12: >>> ffff9ea2d6bd71e0 >>> <4>[ 594.974384] R13: 0000000000000009 R14: 0000000000000000 R15: >>> ffff9ea43bfce000 >>> <4>[ 594.978892] FS: 000073c62042f740(0000) >>> GS:ffff9ea45edc0000(0000) knlGS:0000000000000000 >>> <4>[ 594.983467] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >>> <4>[ 594.988055] CR2: ffffb3f0028dbff8 CR3: 0000000073aa8005 CR4: >>> 00000000003606e0 >>> <4>[ 594.992689] DR0: 0000000000000000 DR1: 0000000000000000 DR2: >>> 0000000000000000 >>> <4>[ 594.997310] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: >>> 0000000000000400 >>> <4>[ 595.001860] Call Trace: >>> <4>[ 595.006323] f2fs_write_cache_pages+0x102/0x450 >>> <4>[ 595.010829] ? kmem_cache_alloc+0x81/0x6b0 >>> <4>[ 595.015219] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 595.019483] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.023749] ? __enqueue_entity+0x5/0x60 >>> <4>[ 595.027995] ? enqueue_task_fair+0x4b1/0xb20 >>> <4>[ 595.032232] do_writepages+0x41/0xd0 >>> <4>[ 595.036481] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.040754] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.045017] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.049304] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.053594] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.057892] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.062218] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.066549] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.070880] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.075077] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>> <4>[ 595.079177] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.083266] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.087352] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.091460] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.095565] ? _raw_spin_lock+0x13/0x30 >>> <4>[ 595.099650] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.103769] ? __enqueue_entity+0x5/0x60 >>> <4>[ 595.107922] ? enqueue_task_fair+0x4b1/0xb20 >>> <4>[ 595.112117] do_writepages+0x41/0xd0 >>> <4>[ 595.116347] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.120630] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.124931] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.129268] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.133636] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.138039] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.142520] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.146992] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.151467] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.155947] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>> <4>[ 595.160466] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.164964] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.169478] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.174019] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.178545] ? _raw_spin_lock+0x13/0x30 >>> <4>[ 595.183050] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.187613] ? __enqueue_entity+0x5/0x60 >>> <4>[ 595.192150] ? enqueue_task_fair+0x4b1/0xb20 >>> <4>[ 595.196678] do_writepages+0x41/0xd0 >>> <4>[ 595.201226] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.205819] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.210430] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.214933] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.219454] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.223862] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.228245] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.232543] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.236761] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.240792] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>> <4>[ 595.244662] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.248474] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.252281] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.256076] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.259827] ? __enqueue_entity+0x5/0x60 >>> <4>[ 595.263581] ? enqueue_task_fair+0x4b1/0xb20 >>> <4>[ 595.267280] do_writepages+0x41/0xd0 >>> <4>[ 595.270917] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.274529] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.278101] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.281681] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.285226] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.288729] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.292263] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.295800] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.299328] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.302854] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>> <4>[ 595.306405] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.309936] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.313471] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.317052] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.320637] ? __enqueue_entity+0x5/0x60 >>> <4>[ 595.324215] ? enqueue_task_fair+0x4b1/0xb20 >>> <4>[ 595.327802] do_writepages+0x41/0xd0 >>> <4>[ 595.331372] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.334951] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.338497] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.342030] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.345555] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.349054] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.352579] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.356099] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.359604] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.363110] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>> <4>[ 595.366639] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.370156] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.373681] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.377218] ? __schedule+0x2cd/0x8b0 >>> <4>[ 595.380747] ? __lock_page+0x155/0x240 >>> <4>[ 595.384292] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.387868] ? __enqueue_entity+0x5/0x60 >>> <4>[ 595.391445] ? enqueue_task_fair+0x4b1/0xb20 >>> <4>[ 595.395030] do_writepages+0x41/0xd0 >>> <4>[ 595.398597] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.402169] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.405716] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.409250] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.412777] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.416275] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.419810] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.423335] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.426850] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.430370] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>> <4>[ 595.433904] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.437432] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.440969] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.444511] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.448045] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.451600] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.455136] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.458661] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.462177] do_writepages+0x41/0xd0 >>> <4>[ 595.465672] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.469169] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.472643] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.476106] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.479555] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.482984] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.486441] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.489880] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.493322] ? up_read+0x5/0x20 >>> <4>[ 595.496743] ? f2fs_do_write_data_page+0x2cb/0x940 >>> <4>[ 595.500205] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.503660] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.507118] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.510615] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.514121] ? __enqueue_entity+0x5/0x60 >>> <4>[ 595.517622] ? enqueue_task_fair+0x4b1/0xb20 >>> <4>[ 595.521142] do_writepages+0x41/0xd0 >>> <4>[ 595.524630] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.528135] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.531601] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.535056] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.538498] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.541912] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.545355] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.548792] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.552235] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.555665] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>> <4>[ 595.559120] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.562568] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.566033] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.569527] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.573022] ? __enqueue_entity+0x5/0x60 >>> <4>[ 595.576517] ? enqueue_task_fair+0x4b1/0xb20 >>> <4>[ 595.580021] do_writepages+0x41/0xd0 >>> <4>[ 595.583507] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.586997] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.590462] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.593923] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.597377] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.600796] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.604249] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.607691] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.611124] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.614569] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>> <4>[ 595.618024] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.621477] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.624933] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.628396] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.631852] ? __enqueue_entity+0x5/0x60 >>> <4>[ 595.635334] ? enqueue_task_fair+0x4b1/0xb20 >>> <4>[ 595.638828] do_writepages+0x41/0xd0 >>> <4>[ 595.642314] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.645816] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.649282] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.652726] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.656177] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.659610] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.663057] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.666491] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.669919] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.673342] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>> <4>[ 595.676791] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.680243] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.683722] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.687244] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.690766] ? __enqueue_entity+0x5/0x60 >>> <4>[ 595.694278] ? enqueue_task_fair+0x4b1/0xb20 >>> <4>[ 595.697809] do_writepages+0x41/0xd0 >>> <4>[ 595.701342] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.704903] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.708451] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.711975] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.715488] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.718983] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.722508] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.726031] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.729539] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.733044] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>> <4>[ 595.736581] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.740086] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.743605] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.747130] ? _raw_spin_lock+0x13/0x30 >>> <4>[ 595.750654] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.754190] ? __enqueue_entity+0x5/0x60 >>> <4>[ 595.757724] ? enqueue_task_fair+0x4b1/0xb20 >>> <4>[ 595.761288] do_writepages+0x41/0xd0 >>> <4>[ 595.764855] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.768435] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.772001] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.775544] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.779091] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.782617] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.786164] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.789709] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.793237] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.796755] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>> <4>[ 595.800292] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.803843] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.807384] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.810944] ? __enqueue_entity+0x5/0x60 >>> <4>[ 595.814491] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.818039] ? _raw_spin_lock+0x13/0x30 >>> <4>[ 595.821621] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.825212] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.828808] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>> <4>[ 595.832427] ? try_to_wake_up+0x52/0x510 >>> <4>[ 595.836032] do_writepages+0x41/0xd0 >>> <4>[ 595.839623] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.843222] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.846785] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.850328] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.853856] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.857374] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.860909] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.864465] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.867996] ? up_read+0x5/0x20 >>> <4>[ 595.871514] ? f2fs_do_write_data_page+0x2cb/0x940 >>> <4>[ 595.875066] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.878612] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.882190] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.885795] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.889400] ? _raw_spin_lock+0x13/0x30 >>> <4>[ 595.893018] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.896624] ? mempool_alloc+0x65/0x190 >>> <4>[ 595.900219] ? find_get_entry+0xdc/0x150 >>> <4>[ 595.903828] do_writepages+0x41/0xd0 >>> <4>[ 595.907416] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.911003] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.914549] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.918081] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.921610] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.925130] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.928670] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.932210] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.935742] ? unlock_page+0x5/0x30 >>> <4>[ 595.939260] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 595.942818] __write_data_page+0x1c8/0x6e0 >>> <4>[ 595.946381] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 595.949960] ? find_busiest_group+0x40/0x580 >>> <4>[ 595.953544] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.957121] ? _raw_spin_lock+0x13/0x30 >>> <4>[ 595.960683] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 595.964267] ? find_get_entry+0xdc/0x150 >>> <4>[ 595.967849] do_writepages+0x41/0xd0 >>> <4>[ 595.971404] ? preempt_count_add+0x79/0xb0 >>> <4>[ 595.974962] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 595.978489] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 595.982005] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 595.985505] ? si_meminfo+0x3f/0x60 >>> <4>[ 595.988990] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 595.992488] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 595.995995] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 595.999497] ? unlock_page+0x5/0x30 >>> <4>[ 596.002978] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 596.006497] __write_data_page+0x1c8/0x6e0 >>> <4>[ 596.010031] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 596.013590] ? stat_show+0xd01/0x1280 >>> <4>[ 596.017136] ? up_read+0x5/0x20 >>> <4>[ 596.020661] ? f2fs_getxattr+0x19e/0x3e0 >>> <4>[ 596.024194] ? preempt_count_add+0x79/0xb0 >>> <4>[ 596.027720] ? _raw_spin_lock+0x13/0x30 >>> <4>[ 596.031237] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 596.034759] ? preempt_count_add+0x79/0xb0 >>> <4>[ 596.038254] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 596.041726] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 596.045174] ? f2fs_inode_dirtied+0x64/0xf0 >>> <4>[ 596.048619] ? preempt_count_add+0x79/0xb0 >>> <4>[ 596.052066] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 596.055503] do_writepages+0x41/0xd0 >>> <4>[ 596.058916] ? preempt_count_add+0x79/0xb0 >>> <4>[ 596.062334] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 596.065718] ? _raw_spin_unlock+0x12/0x30 >>> <4>[ 596.069076] ? nr_blockdev_pages+0x59/0x70 >>> <4>[ 596.072438] ? si_meminfo+0x3f/0x60 >>> <4>[ 596.075787] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 596.079190] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>> <4>[ 596.082605] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 596.086024] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 596.089415] f2fs_create+0x131/0x3e0 >>> <4>[ 596.092781] ? f2fs_lookup+0x20e/0x360 >>> <4>[ 596.096141] path_openat+0x11a8/0x1590 >>> <4>[ 596.099493] do_filp_open+0x93/0x100 >>> <4>[ 596.102836] ? preempt_count_sub+0x5/0xa0 >>> <4>[ 596.106176] do_sys_open+0x186/0x220 >>> <4>[ 596.109496] do_syscall_64+0x69/0x399 >>> <4>[ 596.112799] ? trace_hardirqs_off_thunk+0x1a/0x1c >>> <4>[ 596.116121] entry_SYSCALL_64_after_hwframe+0x49/0xbe >>> <4>[ 596.119459] RIP: 0033:0x73c62051f112 >>> <4>[ 596.122789] Code: 25 00 00 41 00 3d 00 00 41 00 74 4c 48 8d 05 >>> d5 42 0d 00 8b 00 85 c0 75 6d 89 f2 b8 01 01 00 00 48 89 fe bf 9c ff >>> ff ff 0f 05 <48> 3d 00 f0 ff ff 0f 87 a2 00 00 00 48 8b 4c 24 28 64 48 >>> 33 0c 25 >>> <4>[ 596.126545] RSP: 002b:00007ffe665bf1a0 EFLAGS: 00000246 >>> ORIG_RAX: 0000000000000101 >>> <4>[ 596.130299] RAX: ffffffffffffffda RBX: 00007ffe665bf8f0 RCX: >>> 000073c62051f112 >>> <4>[ 596.134110] RDX: 00000000000000c1 RSI: 00005670d5042690 RDI: >>> 00000000ffffff9c >>> <4>[ 596.137944] RBP: 00007ffe665bf5e0 R08: 00007ffe665bf8f0 R09: >>> 00007ffe665bf8f0 >>> <4>[ 596.141784] R10: 0000000000000180 R11: 0000000000000246 R12: >>> 00000000000001a4 >>> <4>[ 596.145664] R13: 0000000000000003 R14: 0000000000008000 R15: >>> 0000000000000000 >>> <4>[ 596.149536] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc >>> nls_cp437 uas usb_storage pl2303 sd_mod x86_pkg_temp_thermal kvm_intel >>> iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich >>> firewire_ohci firewire_core igb ahci crc_itu_t >>> processor_thermal_device libahci dca intel_soc_dts_iosf >>> int340x_thermal_zone pcc_cpufreq efivarfs >>> <4>[ 596.158385] ---[ end trace d508e01f5312dbde ]--- >>> --END log one >>> >>> --BEGIN log two >>> <4>[ 462.870628] sync_dirty_inodes: inode:624255, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870633] sync_dirty_inodes: inode:624255, >>> cp_task:000000002fea927f >>> <4>[ 462.870634] sync_dirty_inodes: inode:624257, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870638] sync_dirty_inodes: inode:624257, >>> cp_task:000000002fea927f >>> <4>[ 462.870660] sync_dirty_inodes: inode:623968, >>> cp_task:000000002fea927f >>> <4>[ 462.870665] sync_dirty_inodes: inode:623970, >>> cp_task:000000002fea927f >>> <4>[ 462.870679] sync_dirty_inodes: inode:623974, >>> cp_task:000000002fea927f >>> <4>[ 462.870681] sync_dirty_inodes: inode:623974, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870691] sync_dirty_inodes: inode:623975, >>> cp_task:000000002fea927f >>> <4>[ 462.870696] sync_dirty_inodes: inode:623976, >>> cp_task:000000002fea927f >>> <4>[ 462.870697] sync_dirty_inodes: inode:623976, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870711] sync_dirty_inodes: inode:623978, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870715] sync_dirty_inodes: inode:623981, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870717] sync_dirty_inodes: inode:623981, >>> cp_task:000000002fea927f >>> <4>[ 462.870719] sync_dirty_inodes: inode:623982, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870721] sync_dirty_inodes: inode:623982, >>> cp_task:000000002fea927f >>> <4>[ 462.870754] sync_dirty_inodes: inode:623986, >>> cp_task:000000002fea927f >>> <4>[ 462.870755] sync_dirty_inodes: inode:623986, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870774] sync_dirty_inodes: inode:623987, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870775] sync_dirty_inodes: inode:623987, >>> cp_task:000000002fea927f >>> <4>[ 462.870783] sync_dirty_inodes: inode:623988, >>> cp_task:000000002fea927f >>> <4>[ 462.870788] sync_dirty_inodes: inode:623991, >>> cp_task:000000002fea927f >>> <4>[ 462.870794] sync_dirty_inodes: inode:623993, >>> cp_task:000000002fea927f >>> <4>[ 462.870795] sync_dirty_inodes: inode:623993, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870801] sync_dirty_inodes: inode:623995, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870808] sync_dirty_inodes: inode:623995, >>> cp_task:000000002fea927f >>> <4>[ 462.870810] sync_dirty_inodes: inode:623995, >>> cp_task:000000002fea927f >>> <4>[ 462.870812] sync_dirty_inodes: inode:623998, >>> cp_task:000000002fea927f >>> <4>[ 462.870813] sync_dirty_inodes: inode:623998, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870848] sync_dirty_inodes: inode:624000, >>> cp_task:000000002fea927f >>> <4>[ 462.870852] sync_dirty_inodes: inode:624000, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870853] sync_dirty_inodes: inode:624003, >>> cp_task:000000002fea927f >>> <4>[ 462.870856] sync_dirty_inodes: inode:624003, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870858] sync_dirty_inodes: inode:624004, >>> cp_task:000000002fea927f >>> <4>[ 462.870860] sync_dirty_inodes: inode:624004, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870862] sync_dirty_inodes: inode:624007, >>> cp_task:000000002fea927f >>> <4>[ 462.870863] sync_dirty_inodes: inode:624007, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870875] sync_dirty_inodes: inode:624009, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870902] sync_dirty_inodes: inode:624009, >>> cp_task:000000002fea927f >>> <4>[ 462.870905] sync_dirty_inodes: inode:624010, >>> cp_task:000000002fea927f >>> <4>[ 462.870908] sync_dirty_inodes: inode:624010, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870913] sync_dirty_inodes: inode:624013, >>> cp_task:000000002fea927f >>> <4>[ 462.870917] sync_dirty_inodes: inode:624014, >>> cp_task:000000002fea927f >>> <4>[ 462.870921] sync_dirty_inodes: inode:624014, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870947] sync_dirty_inodes: inode:624016, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870951] sync_dirty_inodes: inode:624016, >>> cp_task:000000002fea927f >>> <4>[ 462.870953] sync_dirty_inodes: inode:624016, >>> cp_task:000000002fea927f >>> <4>[ 462.870955] sync_dirty_inodes: inode:624016, >>> cp_task:000000002fea927f >>> <4>[ 462.870956] sync_dirty_inodes: inode:624017, >>> cp_task:000000002fea927f >>> <4>[ 462.870958] sync_dirty_inodes: inode:624017, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870961] sync_dirty_inodes: inode:624021, >>> cp_task:000000002fea927f >>> <4>[ 462.870962] sync_dirty_inodes: inode:624021, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870968] sync_dirty_inodes: inode:624022, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870970] sync_dirty_inodes: inode:624022, >>> cp_task:000000002fea927f >>> <4>[ 462.870975] sync_dirty_inodes: inode:624024, >>> cp_task:000000006480c3a0 >>> <4>[ 462.870976] sync_dirty_inodes: inode:624024, >>> cp_task:000000002fea927f >>> <4>[ 462.870989] sync_dirty_inodes: inode:624026, >>> cp_task:000000002fea927f >>> <4>[ 462.870990] sync_dirty_inodes: inode:624026, >>> cp_task:000000006480c3a0 >>> <0>[ 462.870997] BUG: stack guard page was hit at 0000000056e52d0a >>> (stack is 00000000f6235c3f..00000000f20b69a0) >>> <4>[ 462.876404] kernel stack overflow (double-fault): 0000 [#1] >>> PREEMPT SMP PTI >>> <4>[ 462.881794] CPU: 6 PID: 875 Comm: kworker/u16:9 Not tainted >>> 5.0.15-gentoo-f2fslog_sync_dirty_inodes #5 >>> <4>[ 462.887185] Hardware name: To Be Filled By O.E.M. To Be Filled >>> By O.E.M./C226 WS, BIOS P3.40 06/25/2018 >>> <4>[ 462.892563] Workqueue: writeback wb_workfn (flush-259:0) >>> <4>[ 462.897877] RIP: 0010:tracer_hardirqs_off+0x4/0xf0 >>> <4>[ 462.903078] Code: ee 48 0f 44 dd 4c 89 ff 48 89 da e8 d6 f9 ff >>> ff 49 c7 45 20 00 00 00 00 f0 41 ff 4d 00 e9 67 ff ff ff 0f 1f 40 00 >>> 41 57 41 56 <41> 55 41 54 55 53 48 83 ec 08 65 44 8b 25 0a 9f 06 4b f6 >>> 05 53 12 >>> <4>[ 462.908571] RSP: 0018:ffffaa364054c000 EFLAGS: 00010046 >>> <4>[ 462.913921] RAX: 0000000000000000 RBX: ffff962b5b81ca80 RCX: >>> ffffffffb6b66b50 >>> <4>[ 462.919235] RDX: ffffffffb6b66b40 RSI: 0000000000000000 RDI: >>> ffffffffb5059c74 >>> <4>[ 462.924523] RBP: 0000000000411200 R08: 0000000000000002 R09: >>> 0000000000000001 >>> <4>[ 462.929772] R10: 0000000001cb731b R11: 0000000001cb731c R12: >>> 0000000000411200 >>> <4>[ 462.934889] R13: 0000000000000246 R14: ffffaa364054c0c8 R15: >>> ffffffffb4f29520 >>> <4>[ 462.939888] FS: 0000000000000000(0000) >>> GS:ffff962b5ed80000(0000) knlGS:0000000000000000 >>> <4>[ 462.944841] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >>> <4>[ 462.949681] CR2: ffffaa364054bff8 CR3: 000000072380c002 CR4: >>> 00000000003606e0 >>> <4>[ 462.954463] DR0: 0000000000000000 DR1: 0000000000000000 DR2: >>> 0000000000000000 >>> <4>[ 462.959191] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: >>> 0000000000000400 >>> <4>[ 462.963797] Call Trace: >>> <4>[ 462.968280] ? finish_wait+0x80/0x80 >>> <4>[ 462.972785] trace_hardirqs_off+0x2b/0xe0 >>> <4>[ 462.977174] kmem_cache_alloc+0x44/0x6b0 >>> <4>[ 462.981443] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 462.985703] ? finish_wait+0x80/0x80 >>> <4>[ 462.989939] mempool_alloc+0x65/0x190 >>> <4>[ 462.994143] ? preempt_count_add+0x79/0xb0 >>> <4>[ 462.998379] bio_alloc_bioset+0x140/0x210 >>> <4>[ 463.002628] f2fs_submit_page_write+0x1bb/0x5e0 >>> <4>[ 463.006912] do_write_page+0x99/0x180 >>> <4>[ 463.011201] f2fs_outplace_write_data+0x55/0xe0 >>> <4>[ 463.015538] f2fs_do_write_data_page+0x588/0x940 >>> <4>[ 463.019897] __write_data_page+0x4f2/0x6f0 >>> <4>[ 463.024251] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.028616] ? pointer+0x110/0x3b0 >>> <4>[ 463.032972] ? log_store+0xf5/0x260 >>> <4>[ 463.037183] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.041277] ? __down_trylock_console_sem.isra.14+0x42/0x50 >>> <4>[ 463.045404] ? vprintk_emit+0x28f/0x2a0 >>> <4>[ 463.049517] do_writepages+0x41/0xd0 >>> <4>[ 463.053634] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.057778] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.061911] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.066055] ? up_read+0x5/0x20 >>> <4>[ 463.070224] ? f2fs_do_write_data_page+0x2cb/0x940 >>> <4>[ 463.074464] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.078742] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.083072] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.087459] ? pointer+0x110/0x3b0 >>> <4>[ 463.091880] ? log_store+0xf5/0x260 >>> <4>[ 463.096333] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.100853] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.105422] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.109943] do_writepages+0x41/0xd0 >>> <4>[ 463.114451] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.119000] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.123565] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.128116] ? up_read+0x5/0x20 >>> <4>[ 463.132657] ? f2fs_do_write_data_page+0x2cb/0x940 >>> <4>[ 463.137238] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.141817] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.146390] ? _raw_spin_unlock_irqrestore+0x34/0x40 >>> <4>[ 463.151025] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.155642] ? pointer+0x110/0x3b0 >>> <4>[ 463.160240] ? log_store+0xf5/0x260 >>> <4>[ 463.164859] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.169533] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.174254] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.178838] do_writepages+0x41/0xd0 >>> <4>[ 463.183323] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.187757] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.192105] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.196367] ? up_read+0x5/0x20 >>> <4>[ 463.200592] ? f2fs_do_write_data_page+0x2cb/0x940 >>> <4>[ 463.204670] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.208563] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.212425] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.216279] ? pointer+0x110/0x3b0 >>> <4>[ 463.220107] ? log_store+0xf5/0x260 >>> <4>[ 463.223927] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.227747] ? __down_trylock_console_sem.isra.14+0x42/0x50 >>> <4>[ 463.231564] ? vprintk_emit+0x28f/0x2a0 >>> <4>[ 463.235333] do_writepages+0x41/0xd0 >>> <4>[ 463.239049] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.242783] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.246543] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.250250] ? up_read+0x5/0x20 >>> <4>[ 463.253949] ? f2fs_do_write_data_page+0x2cb/0x940 >>> <4>[ 463.257687] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.261419] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.265159] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.268916] ? pointer+0x110/0x3b0 >>> <4>[ 463.272675] ? log_store+0xf5/0x260 >>> <4>[ 463.276420] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.280192] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.283952] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.287664] do_writepages+0x41/0xd0 >>> <4>[ 463.291360] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.295087] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.298810] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.302519] ? up_read+0x5/0x20 >>> <4>[ 463.306207] ? f2fs_do_write_data_page+0x2cb/0x940 >>> <4>[ 463.309944] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.313670] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.317379] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.321080] ? pointer+0x110/0x3b0 >>> <4>[ 463.324773] ? log_store+0xf5/0x260 >>> <4>[ 463.328444] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.332132] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.335820] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.339467] do_writepages+0x41/0xd0 >>> <4>[ 463.343094] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.346740] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.350392] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.354029] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >>> <4>[ 463.357687] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.361353] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.365020] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.368695] ? pointer+0x110/0x3b0 >>> <4>[ 463.372399] ? log_store+0xf5/0x260 >>> <4>[ 463.376091] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.379796] ? __down_trylock_console_sem.isra.14+0x42/0x50 >>> <4>[ 463.383534] ? vprintk_emit+0x28f/0x2a0 >>> <4>[ 463.387267] do_writepages+0x41/0xd0 >>> <4>[ 463.390989] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.394729] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.398460] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.402181] ? up_read+0x5/0x20 >>> <4>[ 463.405875] ? f2fs_do_write_data_page+0x2cb/0x940 >>> <4>[ 463.409590] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.413295] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.417019] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.420752] ? pointer+0x110/0x3b0 >>> <4>[ 463.424466] ? log_store+0xf5/0x260 >>> <4>[ 463.428207] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.431971] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.435729] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.439437] do_writepages+0x41/0xd0 >>> <4>[ 463.443137] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.446863] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.450581] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.454281] ? up_read+0x5/0x20 >>> <4>[ 463.457962] ? f2fs_do_write_data_page+0x2cb/0x940 >>> <4>[ 463.461671] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.465368] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.469084] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.472807] ? pointer+0x110/0x3b0 >>> <4>[ 463.476499] ? log_store+0xf5/0x260 >>> <4>[ 463.480154] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.483842] ? __down_trylock_console_sem.isra.14+0x42/0x50 >>> <4>[ 463.487562] ? vprintk_emit+0x28f/0x2a0 >>> <4>[ 463.491266] do_writepages+0x41/0xd0 >>> <4>[ 463.494963] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.498684] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.502397] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.506098] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >>> <4>[ 463.509824] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.513542] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.517257] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.521001] ? pointer+0x110/0x3b0 >>> <4>[ 463.524731] ? log_store+0xf5/0x260 >>> <4>[ 463.528437] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.532208] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.535979] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.539701] do_writepages+0x41/0xd0 >>> <4>[ 463.543413] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.547150] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.550888] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.554611] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >>> <4>[ 463.558355] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.562091] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.565824] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.569569] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.573317] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.577052] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.580741] do_writepages+0x41/0xd0 >>> <4>[ 463.584464] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.588216] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.591965] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.595696] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >>> <4>[ 463.599454] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.603195] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.606931] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.610679] ? pointer+0x110/0x3b0 >>> <4>[ 463.614406] ? log_store+0xf5/0x260 >>> <4>[ 463.618115] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.621860] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.625598] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.629284] do_writepages+0x41/0xd0 >>> <4>[ 463.632926] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.636607] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.640281] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.643949] ? up_read+0x5/0x20 >>> <4>[ 463.647596] ? f2fs_do_write_data_page+0x2cb/0x940 >>> <4>[ 463.651274] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.654945] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.658605] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.662281] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.665936] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.669582] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.673203] do_writepages+0x41/0xd0 >>> <4>[ 463.676809] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.680434] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.684108] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.687775] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >>> <4>[ 463.691461] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.695135] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.698813] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.702513] ? pointer+0x110/0x3b0 >>> <4>[ 463.706187] ? log_store+0xf5/0x260 >>> <4>[ 463.709837] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.713508] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.717171] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.720786] do_writepages+0x41/0xd0 >>> <4>[ 463.724405] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.728046] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.731666] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.735258] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >>> <4>[ 463.738885] f2fs_balance_fs+0xe5/0x2c0 >>> <4>[ 463.742501] __write_data_page+0x5b7/0x6f0 >>> <4>[ 463.746115] f2fs_write_cache_pages+0x1e0/0x450 >>> <4>[ 463.749744] ? pointer+0x110/0x3b0 >>> <4>[ 463.753345] ? log_store+0xf5/0x260 >>> <4>[ 463.756920] f2fs_write_data_pages+0x2ed/0x320 >>> <4>[ 463.760517] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.764105] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.767643] do_writepages+0x41/0xd0 >>> <4>[ 463.771191] __filemap_fdatawrite_range+0x81/0xb0 >>> <4>[ 463.774760] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> <4>[ 463.778328] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> <4>[ 463.781928] ? trace_hardirqs_on+0x2c/0xe0 >>> <4>[ 463.785524] ? retint_kernel+0x2d/0x2d >>> <4>[ 463.789100] f2fs_write_node_pages+0x51/0x220 >>> <4>[ 463.792687] ? trace_hardirqs_on_caller+0x32/0xf0 >>> <4>[ 463.796285] do_writepages+0x41/0xd0 >>> <4>[ 463.799860] ? retint_kernel+0x2d/0x2d >>> <4>[ 463.803417] __writeback_single_inode+0x3d/0x3d0 >>> <4>[ 463.806999] writeback_sb_inodes+0x1e8/0x410 >>> <4>[ 463.810573] __writeback_inodes_wb+0x5d/0xb0 >>> <4>[ 463.814158] wb_writeback+0x28f/0x340 >>> <4>[ 463.817730] wb_workfn+0x2d3/0x420 >>> <4>[ 463.821328] process_one_work+0x1a1/0x3d0 >>> <4>[ 463.824935] worker_thread+0x30/0x380 >>> <4>[ 463.828520] ? process_one_work+0x3d0/0x3d0 >>> <4>[ 463.832105] kthread+0x116/0x130 >>> <4>[ 463.835675] ? kthread_create_worker_on_cpu+0x70/0x70 >>> <4>[ 463.839278] ret_from_fork+0x3a/0x50 >>> <4>[ 463.842855] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc >>> nls_cp437 uas usb_storage pl2303 sd_mod x86_pkg_temp_thermal kvm_intel >>> iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich >>> firewire_ohci firewire_core igb crc_itu_t dca ahci libahci >>> processor_thermal_device intel_soc_dts_iosf int340x_thermal_zone >>> pcc_cpufreq efivarfs >>> <4>[ 463.851196] ---[ end trace b6d0a43e90218d45 ]--- >>> --END log two >>> >>>> >>>> --- a/fs/f2fs/checkpoint.c >>>> +++ b/fs/f2fs/checkpoint.c >>>> @@ -1074,6 +1074,11 @@ int f2fs_sync_dirty_inodes(struct f2fs_sb_info >>>> *sbi, >>>> enum >>>> inode_type type) >>>> >>>> F2FS_I(inode)->cp_task = current; >>>> >>>> + smp_mb(); >>>> + >>>> + printk("sync_dirty_inodes: inode:%lu, cp_task:%p", >>>> + inode->i_ino, F2FS_I(inode)->cp_task); >>>> + >>>> filemap_fdatawrite(inode->i_mapping); >>>> >>>> F2FS_I(inode)->cp_task = NULL; >>>> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c >>>> index 88988241a0e8..84fbbd337c37 100644 >>>> --- a/fs/f2fs/data.c >>>> +++ b/fs/f2fs/data.c >>>> @@ -2392,6 +2392,10 @@ int f2fs_write_single_data_page(struct page >>>> *page, >>>> int >>>> *submitted, >>>> if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) && >>>> !F2FS_I(inode)->cp_task) { >>>> f2fs_submit_ipu_bio(sbi, bio, page); >>>> + >>>> + printk("write_data_page: inode:%lu, cp_task:%p", >>>> + inode->i_ino, F2FS_I(inode)->cp_task); >>>> + >>>> f2fs_balance_fs(sbi, need_balance_fs); >>>> } >>>> >>>> Thanks, >>> >> . >> > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: BUG: kernel stack overflow when mounting with data_flush 2019-05-20 12:58 ` Hagbard Celine @ 2019-05-20 15:51 ` Chao Yu 0 siblings, 0 replies; 16+ messages in thread From: Chao Yu @ 2019-05-20 15:51 UTC (permalink / raw) To: Hagbard Celine, Chao Yu; +Cc: linux-f2fs-devel On 2019-5-20 20:58, Hagbard Celine wrote: > 2019-05-20 11:37 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >> On 2019/5/16 1:01, Hagbard Celine wrote: >>> 2019-05-15 18:50 GMT+02:00, Hagbard Celine <hagbardcelin@gmail.com>: >>>> 2019-05-15 10:13 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >>>>> On 2019/5/15 16:03, Hagbard Celine wrote: >>>>>> 2019-05-15 4:25 GMT+02:00, Chao Yu <yuchao0@huawei.com>: >>>>>>> On 2019/5/15 2:13, Hagbard Celine wrote: >>>>>>>> 2019-04-02 15:31 GMT+02:00, Chao Yu <chao@kernel.org>: >>>>>>>>> On 2019-4-2 20:41, Hagbard Celine wrote: >>>>>>>>>> That seems to have fixed it. No more errors in syslog after >>>>>>>>>> extracting >>>>>>>>>> my stage3 tarball. Also ran a couple of kernel compiles on a >>>>>>>>>> partition >>>>>>>>>> mounted with data_flush and system seems stable. >>>>>>>>> >>>>>>>>> Thanks a lot for your quick test. :) >>>>>>>> >>>>>>>> My test might have been a little too quick, or I found another >>>>>>>> data_flush bug that behaves similar. >>>>>>> >>>>>>> oops... >>>>>>> >>>>>>>>>>> >>>>>>>>>>> - if (is_dir) >>>>>>>>>>> - F2FS_I(inode)->cp_task = current; >>>>>>>>>>> + F2FS_I(inode)->cp_task = current; >>>>>>> >>>>>>> If you're sure that this patch was applying before you test, I guess >>>>>>> we >>>>>>> need >>>>>>> an >>>>>>> extra barrier here to avoid out-of-order execution. >>>>>>> >>>>>>> smp_mb() >>>>>>> >>>>>>>>>>> >>>>>>>>>>> filemap_fdatawrite(inode->i_mapping); >>>>>>>>>>> >>>>>>>>>>> - if (is_dir) >>>>>>>>>>> - F2FS_I(inode)->cp_task = NULL; >>>>>>>>>>> + F2FS_I(inode)->cp_task = NULL; >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>> If I did this correctly; it did not get rid of the stack overflow. >>>>>> Here is what I did: >>>>>> >>>>>> Added smb_mb() in checkpoint.c so the relevant part looks like this: >>>>>> >>>>>> unsigned long cur_ino = inode->i_ino; >>>>>> >>>>>> F2FS_I(inode)->cp_task = current; >>>>>> >>>>>> smp_mb(); >>>>>> >>>>>> filemap_fdatawrite(inode->i_mapping); >>>>>> >>>>>> F2FS_I(inode)->cp_task = NULL; >>>>>> >>>>>> iput(inode); >>>>>> >>>>>> >>>>>> Compiled, rebooted and ran my test-scripts again. And got this during >>>>>> copy-phase in second script: >>>>> >>>>> It looks very easy to reproduce this bug, could you add log to track >>>>> F2FS_I(inode)->cp_task's value: >>>> That wasn't so easy, with all the logging from those prink the copy >>>> process would hang where it would oops without the printk's. >>> >>> Forgot in last mail that I actually had log from hang with both printk >>> enabled also: >> >> Sorry for the delay. >> >> I found another two issues related to data_flush, could you try below fixing >> patch? >> >> [PATCH] f2fs: fix to avoid deadloop if data_flush is on > > I ran several runs of my test scripts on with this new patch on top of > kernel 5.0.15 with "[PATCH] f2fs: fix potential recursive call when > enabling data_flush" and the extra smp_mb() in checkpoint.c. > When that worked I did the same with this new patch on top of kernel > 5.0.15 with "[PATCH] f2fs: fix potential recursive call when enabling > data_flush" and _without_ the extra smp_mb() in checkpoint.c. > > In both cases I get no oops or hang. Cool, but we need more time to check whether this patch has fixed all deadloop cases, let me know if data_flush hangs kernel later. Thanks, > >> >> Thanks, >> >>> >>> [ 194.681126] sync_dirty_inodes: inode:590309, cp_task:0000000013327ef9 >>> [ 194.682258] sync_dirty_inodes: inode:590301, cp_task:0000000013327ef9 >>> [ 194.682665] sync_dirty_inodes: inode:590311, cp_task:0000000013327ef9 >>> [ 194.682703] sync_dirty_inodes: inode:590312, cp_task:0000000013327ef9 >>> [ 194.682791] sync_dirty_inodes: inode:590313, cp_task:0000000013327ef9 >>> [ 194.683566] sync_dirty_inodes: inode:590314, cp_task:0000000013327ef9 >>> [ 194.683669] sync_dirty_inodes: inode:590315, cp_task:0000000013327ef9 >>> [ 194.684829] sync_dirty_inodes: inode:590316, cp_task:0000000013327ef9 >>> [ 194.712860] sync_dirty_inodes: inode:590317, cp_task:0000000013327ef9 >>> [ 194.712908] sync_dirty_inodes: inode:590310, cp_task:0000000013327ef9 >>> [ 194.713094] sync_dirty_inodes: inode:590319, cp_task:0000000013327ef9 >>> [ 194.713348] sync_dirty_inodes: inode:590320, cp_task:0000000013327ef9 >>> [ 194.713384] sync_dirty_inodes: inode:590321, cp_task:0000000013327ef9 >>> [ 194.714634] sync_dirty_inodes: inode:590322, cp_task:0000000013327ef9 >>> [ 194.715349] sync_dirty_inodes: inode:590323, cp_task:0000000013327ef9 >>> [ 194.715381] sync_dirty_inodes: inode:590324, cp_task:0000000013327ef9 >>> [ 194.718592] sync_dirty_inodes: inode:590326, cp_task:0000000013327ef9 >>> [ 194.719217] sync_dirty_inodes: inode:590327, cp_task:0000000013327ef9 >>> [ 194.719354] sync_dirty_inodes: inode:590328, cp_task:0000000013327ef9 >>> [ 194.719903] sync_dirty_inodes: inode:590329, cp_task:0000000013327ef9 >>> [ 194.720859] sync_dirty_inodes: inode:590521, cp_task:0000000013327ef9 >>> [ 194.720868] sync_dirty_inodes: inode:590300, cp_task:0000000013327ef9 >>> [ 194.720985] sync_dirty_inodes: inode:590523, cp_task:0000000013327ef9 >>> [ 194.738075] sync_dirty_inodes: inode:591528, cp_task:0000000013327ef9 >>> [ 194.738168] sync_dirty_inodes: inode:591529, cp_task:0000000013327ef9 >>> [ 194.738190] sync_dirty_inodes: inode:591533, cp_task:0000000013327ef9 >>> [ 194.738263] sync_dirty_inodes: inode:591537, cp_task:0000000013327ef9 >>> [ 194.738275] sync_dirty_inodes: inode:591538, cp_task:0000000013327ef9 >>> [ 194.738307] sync_dirty_inodes: inode:591540, cp_task:0000000013327ef9 >>> [ 194.738312] sync_dirty_inodes: inode:591542, cp_task:0000000013327ef9 >>> [ 194.759368] sync_dirty_inodes: inode:593095, cp_task:0000000013327ef9 >>> [ 194.759587] sync_dirty_inodes: inode:593351, cp_task:0000000013327ef9 >>> [ 194.759797] sync_dirty_inodes: inode:593352, cp_task:0000000013327ef9 >>> [ 194.760555] sync_dirty_inodes: inode:593353, cp_task:0000000013327ef9 >>> [ 194.760562] sync_dirty_inodes: inode:593354, cp_task:0000000013327ef9 >>> [ 194.761168] sync_dirty_inodes: inode:593355, cp_task:0000000013327ef9 >>> [ 194.761207] sync_dirty_inodes: inode:593356, cp_task:0000000013327ef9 >>> [ 194.761234] sync_dirty_inodes: inode:593357, cp_task:0000000013327ef9 >>> [ 194.761251] sync_dirty_inodes: inode:593358, cp_task:0000000013327ef9 >>> [ 194.761285] sync_dirty_inodes: inode:593360, cp_task:0000000013327ef9 >>> [ 194.761290] sync_dirty_inodes: inode:593361, cp_task:0000000013327ef9 >>> [ 194.763583] sync_dirty_inodes: inode:593363, cp_task:0000000013327ef9 >>> [ 194.763845] sync_dirty_inodes: inode:593362, cp_task:0000000013327ef9 >>> [ 194.763891] sync_dirty_inodes: inode:593367, cp_task:0000000013327ef9 >>> [ 194.763992] sync_dirty_inodes: inode:593368, cp_task:0000000013327ef9 >>> [ 194.764073] sync_dirty_inodes: inode:593369, cp_task:0000000013327ef9 >>> [ 194.764090] sync_dirty_inodes: inode:593370, cp_task:0000000013327ef9 >>> [ 194.764165] sync_dirty_inodes: inode:593371, cp_task:0000000013327ef9 >>> [ 194.764185] sync_dirty_inodes: inode:593372, cp_task:0000000013327ef9 >>> [ 194.764214] sync_dirty_inodes: inode:593365, cp_task:0000000013327ef9 >>> [ 194.764323] sync_dirty_inodes: inode:593374, cp_task:0000000013327ef9 >>> [ 194.764419] sync_dirty_inodes: inode:593373, cp_task:0000000013327ef9 >>> [ 194.764512] sync_dirty_inodes: inode:593375, cp_task:0000000013327ef9 >>> [ 194.764533] sync_dirty_inodes: inode:593376, cp_task:0000000013327ef9 >>> [ 194.764608] sync_dirty_inodes: inode:593377, cp_task:0000000013327ef9 >>> [ 194.764857] sync_dirty_inodes: inode:593378, cp_task:0000000013327ef9 >>> [ 194.764884] sync_dirty_inodes: inode:593379, cp_task:0000000013327ef9 >>> [ 194.765164] sync_dirty_inodes: inode:593380, cp_task:0000000013327ef9 >>> [ 194.765210] sync_dirty_inodes: inode:593381, cp_task:0000000013327ef9 >>> [ 194.765286] sync_dirty_inodes: inode:593382, cp_task:0000000013327ef9 >>> [ 194.765513] sync_dirty_inodes: inode:593383, cp_task:0000000013327ef9 >>> [ 194.765671] sync_dirty_inodes: inode:593384, cp_task:0000000013327ef9 >>> [ 194.765777] sync_dirty_inodes: inode:593385, cp_task:0000000013327ef9 >>> [ 194.766079] sync_dirty_inodes: inode:593386, cp_task:0000000013327ef9 >>> [ 194.766568] sync_dirty_inodes: inode:593387, cp_task:0000000013327ef9 >>> [ 194.766628] sync_dirty_inodes: inode:593388, cp_task:0000000013327ef9 >>> [ 194.766742] sync_dirty_inodes: inode:593389, cp_task:0000000013327ef9 >>> [ 194.766750] sync_dirty_inodes: inode:593390, cp_task:0000000013327ef9 >>> [ 194.766788] sync_dirty_inodes: inode:593391, cp_task:0000000013327ef9 >>> [ 194.766806] sync_dirty_inodes: inode:593392, cp_task:0000000013327ef9 >>> [ 194.766856] sync_dirty_inodes: inode:593364, cp_task:0000000013327ef9 >>> [ 194.767016] sync_dirty_inodes: inode:593394, cp_task:0000000013327ef9 >>> [ 194.767033] sync_dirty_inodes: inode:593395, cp_task:0000000013327ef9 >>> [ 194.768454] sync_dirty_inodes: inode:593393, cp_task:0000000013327ef9 >>> [ 194.768651] sync_dirty_inodes: inode:593397, cp_task:0000000013327ef9 >>> [ 194.768665] sync_dirty_inodes: inode:593398, cp_task:0000000013327ef9 >>> [ 194.768678] sync_dirty_inodes: inode:593399, cp_task:0000000013327ef9 >>> [ 194.768705] sync_dirty_inodes: inode:593400, cp_task:0000000013327ef9 >>> [ 194.768869] sync_dirty_inodes: inode:593401, cp_task:0000000013327ef9 >>> [ 194.768945] sync_dirty_inodes: inode:593402, cp_task:0000000013327ef9 >>> [ 194.769081] sync_dirty_inodes: inode:593403, cp_task:0000000013327ef9 >>> [ 194.769106] sync_dirty_inodes: inode:593404, cp_task:0000000013327ef9 >>> [ 194.769198] sync_dirty_inodes: inode:593396, cp_task:0000000013327ef9 >>> [ 194.769297] sync_dirty_inodes: inode:593405, cp_task:0000000013327ef9 >>> [ 194.769390] sync_dirty_inodes: inode:593407, cp_task:0000000013327ef9 >>> [ 194.769670] sync_dirty_inodes: inode:593408, cp_task:0000000013327ef9 >>> [ 194.769685] sync_dirty_inodes: inode:593409, cp_task:0000000013327ef9 >>> [ 194.769699] sync_dirty_inodes: inode:593410, cp_task:0000000013327ef9 >>> [ 194.769716] sync_dirty_inodes: inode:593411, cp_task:0000000013327ef9 >>> [ 194.769738] sync_dirty_inodes: inode:593412, cp_task:0000000013327ef9 >>> [ 194.770032] sync_dirty_inodes: inode:593413, cp_task:0000000013327ef9 >>> [ 194.770074] sync_dirty_inodes: inode:593414, cp_task:0000000013327ef9 >>> [ 194.770157] sync_dirty_inodes: inode:593415, cp_task:0000000013327ef9 >>> [ 194.770185] sync_dirty_inodes: inode:593416, cp_task:0000000013327ef9 >>> [ 194.778086] sync_dirty_inodes: inode:593406, cp_task:0000000013327ef9 >>> [ 194.778239] sync_dirty_inodes: inode:593417, cp_task:0000000013327ef9 >>> [ 194.778322] sync_dirty_inodes: inode:593418, cp_task:0000000013327ef9 >>> [ 194.778484] sync_dirty_inodes: inode:593419, cp_task:0000000013327ef9 >>> [ 194.778562] sync_dirty_inodes: inode:593420, cp_task:0000000013327ef9 >>> [ 194.778621] sync_dirty_inodes: inode:593421, cp_task:0000000013327ef9 >>> [ 194.778635] sync_dirty_inodes: inode:593422, cp_task:0000000013327ef9 >>> [ 194.778661] sync_dirty_inodes: inode:593423, cp_task:0000000013327ef9 >>> [ 194.779162] sync_dirty_inodes: inode:593424, cp_task:0000000013327ef9 >>> [ 194.779227] sync_dirty_inodes: inode:593425, cp_task:0000000013327ef9 >>> [ 194.781339] sync_dirty_inodes: inode:593427, cp_task:0000000013327ef9 >>> [ 194.841045] write_data_page: inode:764980, cp_task: (null) >>> [ 194.861488] sync_dirty_inodes: inode:593948, cp_task:0000000013327ef9 >>> [ 194.941948] sync_dirty_inodes: inode:593948, cp_task:0000000045fd0ed6 >>> [ 194.958797] sync_dirty_inodes: inode:593948, cp_task:0000000013327ef9 >>> [ 360.534693] usb 3-3: USB disconnect, device number 2 >>> [ 369.937827] INFO: task kworker/u16:5:344 blocked for more than 120 >>> seconds. >>> [ 369.937829] Not tainted 5.0.15-gentoo-f2fslog #4 >>> [ 369.937830] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >>> disables this message. >>> [ 369.937831] kworker/u16:5 D 0 344 2 0x80000000 >>> [ 369.937836] Workqueue: writeback wb_workfn (flush-259:0) >>> [ 369.937838] Call Trace: >>> [ 369.937843] ? __schedule+0x2c5/0x8b0 >>> [ 369.937846] schedule+0x32/0x80 >>> [ 369.937848] schedule_preempt_disabled+0x14/0x20 >>> [ 369.937850] __mutex_lock.isra.8+0x2ba/0x4d0 >>> [ 369.937853] ? log_store+0xf5/0x260 >>> [ 369.937857] f2fs_write_data_pages+0x133/0x320 >>> [ 369.937860] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 369.937863] do_writepages+0x41/0xd0 >>> [ 369.937866] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 369.937869] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 369.937871] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 369.937874] ? up_read+0x5/0x20 >>> [ 369.937876] ? f2fs_do_write_data_page+0x2cb/0x940 >>> [ 369.937877] f2fs_balance_fs+0xe5/0x2c0 >>> [ 369.937880] __write_data_page+0x1c8/0x6e0 >>> [ 369.937884] f2fs_write_cache_pages+0x1e0/0x450 >>> [ 369.937889] f2fs_write_data_pages+0x14b/0x320 >>> [ 369.937891] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 369.937894] do_writepages+0x41/0xd0 >>> [ 369.937896] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 369.937898] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 369.937900] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 369.937902] f2fs_write_node_pages+0x51/0x220 >>> [ 369.937905] do_writepages+0x41/0xd0 >>> [ 369.937908] __writeback_single_inode+0x3d/0x3d0 >>> [ 369.937910] writeback_sb_inodes+0x1e8/0x410 >>> [ 369.937913] __writeback_inodes_wb+0x5d/0xb0 >>> [ 369.937915] wb_writeback+0x28f/0x340 >>> [ 369.937918] ? cpumask_next+0x16/0x20 >>> [ 369.937920] wb_workfn+0x33e/0x420 >>> [ 369.937923] process_one_work+0x1a1/0x3d0 >>> [ 369.937925] worker_thread+0x30/0x380 >>> [ 369.937927] ? process_one_work+0x3d0/0x3d0 >>> [ 369.937929] kthread+0x116/0x130 >>> [ 369.937931] ? kthread_create_worker_on_cpu+0x70/0x70 >>> [ 369.937933] ret_from_fork+0x3a/0x50 >>> [ 369.937946] INFO: task cp:2049 blocked for more than 120 seconds. >>> [ 369.937946] Not tainted 5.0.15-gentoo-f2fslog #4 >>> [ 369.937947] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >>> disables this message. >>> [ 369.937948] cp D 0 2049 2029 0x00000000 >>> [ 369.937949] Call Trace: >>> [ 369.937952] ? __schedule+0x2c5/0x8b0 >>> [ 369.937954] schedule+0x32/0x80 >>> [ 369.937956] schedule_preempt_disabled+0x14/0x20 >>> [ 369.937958] __mutex_lock.isra.8+0x2ba/0x4d0 >>> [ 369.937960] ? log_store+0xf5/0x260 >>> [ 369.937963] f2fs_write_data_pages+0x133/0x320 >>> [ 369.937965] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 369.937968] do_writepages+0x41/0xd0 >>> [ 369.937971] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 369.937973] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 369.937975] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 369.937977] f2fs_balance_fs+0xe5/0x2c0 >>> [ 369.937980] f2fs_mkdir+0x124/0x160 >>> [ 369.937982] vfs_mkdir+0x102/0x1b0 >>> [ 369.937985] do_mkdirat+0x7d/0xf0 >>> [ 369.937987] do_syscall_64+0x69/0x399 >>> [ 369.937989] ? trace_hardirqs_off_thunk+0x1a/0x1c >>> [ 369.937992] entry_SYSCALL_64_after_hwframe+0x49/0xbe >>> [ 369.937994] RIP: 0033:0x717f2eaa603b >>> [ 369.937998] Code: Bad RIP value. >>> [ 369.937999] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: >>> 0000000000000053 >>> [ 369.938001] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: >>> 0000717f2eaa603b >>> [ 369.938002] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: >>> 00005b54b1a17ff0 >>> [ 369.938003] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: >>> 00007ffd8a71cf80 >>> [ 369.938004] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: >>> 00000000000001ed >>> [ 369.938005] R13: 0000000000000000 R14: 0000000000004000 R15: >>> 0000000000000000 >>> [ 492.817796] INFO: task kworker/u16:5:344 blocked for more than 120 >>> seconds. >>> [ 492.817797] Not tainted 5.0.15-gentoo-f2fslog #4 >>> [ 492.817797] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >>> disables this message. >>> [ 492.817798] kworker/u16:5 D 0 344 2 0x80000000 >>> [ 492.817810] Workqueue: writeback wb_workfn (flush-259:0) >>> [ 492.817811] Call Trace: >>> [ 492.817813] ? __schedule+0x2c5/0x8b0 >>> [ 492.817815] schedule+0x32/0x80 >>> [ 492.817816] schedule_preempt_disabled+0x14/0x20 >>> [ 492.817817] __mutex_lock.isra.8+0x2ba/0x4d0 >>> [ 492.817819] ? log_store+0xf5/0x260 >>> [ 492.817821] f2fs_write_data_pages+0x133/0x320 >>> [ 492.817823] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 492.817825] do_writepages+0x41/0xd0 >>> [ 492.817826] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 492.817828] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 492.817829] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 492.817831] ? up_read+0x5/0x20 >>> [ 492.817832] ? f2fs_do_write_data_page+0x2cb/0x940 >>> [ 492.817833] f2fs_balance_fs+0xe5/0x2c0 >>> [ 492.817835] __write_data_page+0x1c8/0x6e0 >>> [ 492.817837] f2fs_write_cache_pages+0x1e0/0x450 >>> [ 492.817840] f2fs_write_data_pages+0x14b/0x320 >>> [ 492.817842] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 492.817843] do_writepages+0x41/0xd0 >>> [ 492.817845] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 492.817846] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 492.817847] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 492.817849] f2fs_write_node_pages+0x51/0x220 >>> [ 492.817851] do_writepages+0x41/0xd0 >>> [ 492.817853] __writeback_single_inode+0x3d/0x3d0 >>> [ 492.817854] writeback_sb_inodes+0x1e8/0x410 >>> [ 492.817856] __writeback_inodes_wb+0x5d/0xb0 >>> [ 492.817857] wb_writeback+0x28f/0x340 >>> [ 492.817859] ? cpumask_next+0x16/0x20 >>> [ 492.817860] wb_workfn+0x33e/0x420 >>> [ 492.817862] process_one_work+0x1a1/0x3d0 >>> [ 492.817863] worker_thread+0x30/0x380 >>> [ 492.817865] ? process_one_work+0x3d0/0x3d0 >>> [ 492.817865] kthread+0x116/0x130 >>> [ 492.817866] ? kthread_create_worker_on_cpu+0x70/0x70 >>> [ 492.817868] ret_from_fork+0x3a/0x50 >>> [ 492.817873] INFO: task cp:2049 blocked for more than 120 seconds. >>> [ 492.817873] Not tainted 5.0.15-gentoo-f2fslog #4 >>> [ 492.817874] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >>> disables this message. >>> [ 492.817874] cp D 0 2049 2029 0x00000000 >>> [ 492.817875] Call Trace: >>> [ 492.817877] ? __schedule+0x2c5/0x8b0 >>> [ 492.817878] schedule+0x32/0x80 >>> [ 492.817879] schedule_preempt_disabled+0x14/0x20 >>> [ 492.817880] __mutex_lock.isra.8+0x2ba/0x4d0 >>> [ 492.817882] ? log_store+0xf5/0x260 >>> [ 492.817884] f2fs_write_data_pages+0x133/0x320 >>> [ 492.817886] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 492.817887] do_writepages+0x41/0xd0 >>> [ 492.817889] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 492.817890] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 492.817892] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 492.817893] f2fs_balance_fs+0xe5/0x2c0 >>> [ 492.817894] f2fs_mkdir+0x124/0x160 >>> [ 492.817896] vfs_mkdir+0x102/0x1b0 >>> [ 492.817897] do_mkdirat+0x7d/0xf0 >>> [ 492.817899] do_syscall_64+0x69/0x399 >>> [ 492.817900] ? trace_hardirqs_off_thunk+0x1a/0x1c >>> [ 492.817902] entry_SYSCALL_64_after_hwframe+0x49/0xbe >>> [ 492.817903] RIP: 0033:0x717f2eaa603b >>> [ 492.817905] Code: Bad RIP value. >>> [ 492.817906] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: >>> 0000000000000053 >>> [ 492.817907] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: >>> 0000717f2eaa603b >>> [ 492.817908] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: >>> 00005b54b1a17ff0 >>> [ 492.817908] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: >>> 00007ffd8a71cf80 >>> [ 492.817909] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: >>> 00000000000001ed >>> [ 492.817910] R13: 0000000000000000 R14: 0000000000004000 R15: >>> 0000000000000000 >>> [ 615.697824] INFO: task kworker/u16:5:344 blocked for more than 120 >>> seconds. >>> [ 615.697825] Not tainted 5.0.15-gentoo-f2fslog #4 >>> [ 615.697826] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >>> disables this message. >>> [ 615.697827] kworker/u16:5 D 0 344 2 0x80000000 >>> [ 615.697831] Workqueue: writeback wb_workfn (flush-259:0) >>> [ 615.697832] Call Trace: >>> [ 615.697836] ? __schedule+0x2c5/0x8b0 >>> [ 615.697839] schedule+0x32/0x80 >>> [ 615.697841] schedule_preempt_disabled+0x14/0x20 >>> [ 615.697842] __mutex_lock.isra.8+0x2ba/0x4d0 >>> [ 615.697845] ? log_store+0xf5/0x260 >>> [ 615.697848] f2fs_write_data_pages+0x133/0x320 >>> [ 615.697851] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 615.697854] do_writepages+0x41/0xd0 >>> [ 615.697857] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 615.697859] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 615.697861] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 615.697863] ? up_read+0x5/0x20 >>> [ 615.697865] ? f2fs_do_write_data_page+0x2cb/0x940 >>> [ 615.697867] f2fs_balance_fs+0xe5/0x2c0 >>> [ 615.697869] __write_data_page+0x1c8/0x6e0 >>> [ 615.697873] f2fs_write_cache_pages+0x1e0/0x450 >>> [ 615.697878] f2fs_write_data_pages+0x14b/0x320 >>> [ 615.697880] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 615.697883] do_writepages+0x41/0xd0 >>> [ 615.697885] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 615.697887] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 615.697889] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 615.697891] f2fs_write_node_pages+0x51/0x220 >>> [ 615.697894] do_writepages+0x41/0xd0 >>> [ 615.697897] __writeback_single_inode+0x3d/0x3d0 >>> [ 615.697899] writeback_sb_inodes+0x1e8/0x410 >>> [ 615.697902] __writeback_inodes_wb+0x5d/0xb0 >>> [ 615.697904] wb_writeback+0x28f/0x340 >>> [ 615.697906] ? cpumask_next+0x16/0x20 >>> [ 615.697908] wb_workfn+0x33e/0x420 >>> [ 615.697911] process_one_work+0x1a1/0x3d0 >>> [ 615.697913] worker_thread+0x30/0x380 >>> [ 615.697915] ? process_one_work+0x3d0/0x3d0 >>> [ 615.697916] kthread+0x116/0x130 >>> [ 615.697918] ? kthread_create_worker_on_cpu+0x70/0x70 >>> [ 615.697921] ret_from_fork+0x3a/0x50 >>> [ 615.697927] INFO: task cp:2049 blocked for more than 120 seconds. >>> [ 615.697928] Not tainted 5.0.15-gentoo-f2fslog #4 >>> [ 615.697929] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >>> disables this message. >>> [ 615.697929] cp D 0 2049 2029 0x00000000 >>> [ 615.697931] Call Trace: >>> [ 615.697933] ? __schedule+0x2c5/0x8b0 >>> [ 615.697935] schedule+0x32/0x80 >>> [ 615.697937] schedule_preempt_disabled+0x14/0x20 >>> [ 615.697939] __mutex_lock.isra.8+0x2ba/0x4d0 >>> [ 615.697942] ? log_store+0xf5/0x260 >>> [ 615.697944] f2fs_write_data_pages+0x133/0x320 >>> [ 615.697947] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 615.697950] do_writepages+0x41/0xd0 >>> [ 615.697952] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 615.697954] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 615.697956] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 615.697958] f2fs_balance_fs+0xe5/0x2c0 >>> [ 615.697961] f2fs_mkdir+0x124/0x160 >>> [ 615.697963] vfs_mkdir+0x102/0x1b0 >>> [ 615.697965] do_mkdirat+0x7d/0xf0 >>> [ 615.697968] do_syscall_64+0x69/0x399 >>> [ 615.697969] ? trace_hardirqs_off_thunk+0x1a/0x1c >>> [ 615.697972] entry_SYSCALL_64_after_hwframe+0x49/0xbe >>> [ 615.697974] RIP: 0033:0x717f2eaa603b >>> [ 615.697978] Code: Bad RIP value. >>> [ 615.697979] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: >>> 0000000000000053 >>> [ 615.697980] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: >>> 0000717f2eaa603b >>> [ 615.697981] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: >>> 00005b54b1a17ff0 >>> [ 615.697982] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: >>> 00007ffd8a71cf80 >>> [ 615.697983] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: >>> 00000000000001ed >>> [ 615.697984] R13: 0000000000000000 R14: 0000000000004000 R15: >>> 0000000000000000 >>> [ 738.577796] INFO: task kworker/u16:5:344 blocked for more than 120 >>> seconds. >>> [ 738.577798] Not tainted 5.0.15-gentoo-f2fslog #4 >>> [ 738.577798] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >>> disables this message. >>> [ 738.577799] kworker/u16:5 D 0 344 2 0x80000000 >>> [ 738.577812] Workqueue: writeback wb_workfn (flush-259:0) >>> [ 738.577812] Call Trace: >>> [ 738.577815] ? __schedule+0x2c5/0x8b0 >>> [ 738.577817] schedule+0x32/0x80 >>> [ 738.577818] schedule_preempt_disabled+0x14/0x20 >>> [ 738.577819] __mutex_lock.isra.8+0x2ba/0x4d0 >>> [ 738.577821] ? log_store+0xf5/0x260 >>> [ 738.577823] f2fs_write_data_pages+0x133/0x320 >>> [ 738.577825] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 738.577827] do_writepages+0x41/0xd0 >>> [ 738.577829] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 738.577830] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 738.577832] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 738.577833] ? up_read+0x5/0x20 >>> [ 738.577834] ? f2fs_do_write_data_page+0x2cb/0x940 >>> [ 738.577835] f2fs_balance_fs+0xe5/0x2c0 >>> [ 738.577837] __write_data_page+0x1c8/0x6e0 >>> [ 738.577839] f2fs_write_cache_pages+0x1e0/0x450 >>> [ 738.577842] f2fs_write_data_pages+0x14b/0x320 >>> [ 738.577844] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 738.577846] do_writepages+0x41/0xd0 >>> [ 738.577847] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 738.577849] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 738.577850] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 738.577851] f2fs_write_node_pages+0x51/0x220 >>> [ 738.577853] do_writepages+0x41/0xd0 >>> [ 738.577855] __writeback_single_inode+0x3d/0x3d0 >>> [ 738.577856] writeback_sb_inodes+0x1e8/0x410 >>> [ 738.577858] __writeback_inodes_wb+0x5d/0xb0 >>> [ 738.577859] wb_writeback+0x28f/0x340 >>> [ 738.577861] ? cpumask_next+0x16/0x20 >>> [ 738.577862] wb_workfn+0x33e/0x420 >>> [ 738.577864] process_one_work+0x1a1/0x3d0 >>> [ 738.577865] worker_thread+0x30/0x380 >>> [ 738.577867] ? process_one_work+0x3d0/0x3d0 >>> [ 738.577867] kthread+0x116/0x130 >>> [ 738.577868] ? kthread_create_worker_on_cpu+0x70/0x70 >>> [ 738.577870] ret_from_fork+0x3a/0x50 >>> [ 738.577875] INFO: task cp:2049 blocked for more than 120 seconds. >>> [ 738.577875] Not tainted 5.0.15-gentoo-f2fslog #4 >>> [ 738.577875] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >>> disables this message. >>> [ 738.577876] cp D 0 2049 2029 0x00000000 >>> [ 738.577877] Call Trace: >>> [ 738.577878] ? __schedule+0x2c5/0x8b0 >>> [ 738.577880] schedule+0x32/0x80 >>> [ 738.577881] schedule_preempt_disabled+0x14/0x20 >>> [ 738.577882] __mutex_lock.isra.8+0x2ba/0x4d0 >>> [ 738.577884] ? log_store+0xf5/0x260 >>> [ 738.577886] f2fs_write_data_pages+0x133/0x320 >>> [ 738.577887] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 738.577889] do_writepages+0x41/0xd0 >>> [ 738.577891] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 738.577892] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 738.577893] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 738.577895] f2fs_balance_fs+0xe5/0x2c0 >>> [ 738.577896] f2fs_mkdir+0x124/0x160 >>> [ 738.577898] vfs_mkdir+0x102/0x1b0 >>> [ 738.577899] do_mkdirat+0x7d/0xf0 >>> [ 738.577901] do_syscall_64+0x69/0x399 >>> [ 738.577902] ? trace_hardirqs_off_thunk+0x1a/0x1c >>> [ 738.577904] entry_SYSCALL_64_after_hwframe+0x49/0xbe >>> [ 738.577905] RIP: 0033:0x717f2eaa603b >>> [ 738.577907] Code: Bad RIP value. >>> [ 738.577908] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: >>> 0000000000000053 >>> [ 738.577909] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: >>> 0000717f2eaa603b >>> [ 738.577909] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: >>> 00005b54b1a17ff0 >>> [ 738.577910] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: >>> 00007ffd8a71cf80 >>> [ 738.577910] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: >>> 00000000000001ed >>> [ 738.577911] R13: 0000000000000000 R14: 0000000000004000 R15: >>> 0000000000000000 >>> [ 861.457820] INFO: task kworker/u16:5:344 blocked for more than 120 >>> seconds. >>> [ 861.457823] Not tainted 5.0.15-gentoo-f2fslog #4 >>> [ 861.457823] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >>> disables this message. >>> [ 861.457825] kworker/u16:5 D 0 344 2 0x80000000 >>> [ 861.457830] Workqueue: writeback wb_workfn (flush-259:0) >>> [ 861.457832] Call Trace: >>> [ 861.457837] ? __schedule+0x2c5/0x8b0 >>> [ 861.457841] schedule+0x32/0x80 >>> [ 861.457843] schedule_preempt_disabled+0x14/0x20 >>> [ 861.457846] __mutex_lock.isra.8+0x2ba/0x4d0 >>> [ 861.457850] ? log_store+0xf5/0x260 >>> [ 861.457854] f2fs_write_data_pages+0x133/0x320 >>> [ 861.457858] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 861.457862] do_writepages+0x41/0xd0 >>> [ 861.457865] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 861.457869] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 861.457872] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 861.457874] ? up_read+0x5/0x20 >>> [ 861.457877] ? f2fs_do_write_data_page+0x2cb/0x940 >>> [ 861.457879] f2fs_balance_fs+0xe5/0x2c0 >>> [ 861.457883] __write_data_page+0x1c8/0x6e0 >>> [ 861.457888] f2fs_write_cache_pages+0x1e0/0x450 >>> [ 861.457894] f2fs_write_data_pages+0x14b/0x320 >>> [ 861.457898] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 861.457901] do_writepages+0x41/0xd0 >>> [ 861.457904] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 861.457908] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 861.457910] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 861.457913] f2fs_write_node_pages+0x51/0x220 >>> [ 861.457917] do_writepages+0x41/0xd0 >>> [ 861.457921] __writeback_single_inode+0x3d/0x3d0 >>> [ 861.457924] writeback_sb_inodes+0x1e8/0x410 >>> [ 861.457928] __writeback_inodes_wb+0x5d/0xb0 >>> [ 861.457930] wb_writeback+0x28f/0x340 >>> [ 861.457934] ? cpumask_next+0x16/0x20 >>> [ 861.457936] wb_workfn+0x33e/0x420 >>> [ 861.457940] process_one_work+0x1a1/0x3d0 >>> [ 861.457943] worker_thread+0x30/0x380 >>> [ 861.457946] ? process_one_work+0x3d0/0x3d0 >>> [ 861.457948] kthread+0x116/0x130 >>> [ 861.457950] ? kthread_create_worker_on_cpu+0x70/0x70 >>> [ 861.457953] ret_from_fork+0x3a/0x50 >>> [ 861.457962] INFO: task cp:2049 blocked for more than 120 seconds. >>> [ 861.457963] Not tainted 5.0.15-gentoo-f2fslog #4 >>> [ 861.457964] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >>> disables this message. >>> [ 861.457965] cp D 0 2049 2029 0x00000000 >>> [ 861.457967] Call Trace: >>> [ 861.457970] ? __schedule+0x2c5/0x8b0 >>> [ 861.457974] schedule+0x32/0x80 >>> [ 861.457976] schedule_preempt_disabled+0x14/0x20 >>> [ 861.457978] __mutex_lock.isra.8+0x2ba/0x4d0 >>> [ 861.457982] ? log_store+0xf5/0x260 >>> [ 861.457986] f2fs_write_data_pages+0x133/0x320 >>> [ 861.457990] ? trace_hardirqs_on+0x2c/0xe0 >>> [ 861.457993] do_writepages+0x41/0xd0 >>> [ 861.457996] __filemap_fdatawrite_range+0x81/0xb0 >>> [ 861.458000] f2fs_sync_dirty_inodes+0x1dd/0x200 >>> [ 861.458002] f2fs_balance_fs_bg+0x2a7/0x2c0 >>> [ 861.458005] f2fs_balance_fs+0xe5/0x2c0 >>> [ 861.458009] f2fs_mkdir+0x124/0x160 >>> [ 861.458012] vfs_mkdir+0x102/0x1b0 >>> [ 861.458015] do_mkdirat+0x7d/0xf0 >>> [ 861.458018] do_syscall_64+0x69/0x399 >>> [ 861.458020] ? trace_hardirqs_off_thunk+0x1a/0x1c >>> [ 861.458025] entry_SYSCALL_64_after_hwframe+0x49/0xbe >>> [ 861.458027] RIP: 0033:0x717f2eaa603b >>> [ 861.458032] Code: Bad RIP value. >>> [ 861.458033] RSP: 002b:00007ffd8a71c8c8 EFLAGS: 00000202 ORIG_RAX: >>> 0000000000000053 >>> [ 861.458035] RAX: ffffffffffffffda RBX: 00007ffd8a71cf80 RCX: >>> 0000717f2eaa603b >>> [ 861.458036] RDX: 000000000001ae3c RSI: 00000000000001c0 RDI: >>> 00005b54b1a17ff0 >>> [ 861.458038] RBP: 00007ffd8a71cc70 R08: 00007ffd8a71cf80 R09: >>> 00007ffd8a71cf80 >>> [ 861.458039] R10: 00007ffd8a71ca00 R11: 0000000000000202 R12: >>> 00000000000001ed >>> [ 861.458040] R13: 0000000000000000 R14: 0000000000004000 R15: >>> 0000000000000000 >>> >>> >>>> I was able to reproduse the bug with one of the two printk's at a time >>>> only, and I had to disable syslog-ng and fcron for it not to hang even >>>> then. >>>> >>>> Followig is log from two runs, one with each of the printk's, hope it >>>> helps. >>>> >>>> --BEGIN log one >>>> <4>[ 593.806592] write_data_page: inode:710085, cp_task: >>>> (null) >>>> <4>[ 593.806688] write_data_page: inode:710110, cp_task: >>>> (null) >>>> <4>[ 593.808558] write_data_page: inode:710321, cp_task: >>>> (null) >>>> <4>[ 593.808575] write_data_page: inode:710325, cp_task: >>>> (null) >>>> <4>[ 593.808590] write_data_page: inode:710326, cp_task: >>>> (null) >>>> <4>[ 593.808606] write_data_page: inode:710332, cp_task: >>>> (null) >>>> <4>[ 593.966185] write_data_page: inode:721775, cp_task: >>>> (null) >>>> <4>[ 593.966203] write_data_page: inode:721776, cp_task: >>>> (null) >>>> <4>[ 593.966219] write_data_page: inode:721777, cp_task: >>>> (null) >>>> <4>[ 593.966235] write_data_page: inode:721778, cp_task: >>>> (null) >>>> <4>[ 593.966250] write_data_page: inode:721779, cp_task: >>>> (null) >>>> <4>[ 593.966266] write_data_page: inode:721780, cp_task: >>>> (null) >>>> <4>[ 593.966281] write_data_page: inode:721781, cp_task: >>>> (null) >>>> <4>[ 593.966296] write_data_page: inode:721782, cp_task: >>>> (null) >>>> <4>[ 593.966311] write_data_page: inode:721783, cp_task: >>>> (null) >>>> <4>[ 593.966327] write_data_page: inode:721784, cp_task: >>>> (null) >>>> <4>[ 593.966343] write_data_page: inode:721785, cp_task: >>>> (null) >>>> <4>[ 593.966359] write_data_page: inode:721786, cp_task: >>>> (null) >>>> <4>[ 593.966374] write_data_page: inode:721787, cp_task: >>>> (null) >>>> <4>[ 594.340072] write_data_page: inode:746183, cp_task: >>>> (null) >>>> <0>[ 594.923096] BUG: stack guard page was hit at 000000006e7354a5 >>>> (stack is 000000006445beb4..00000000988529ca) >>>> <0>[ 594.923108] BUG: stack guard page was hit at 00000000d2c9ec98 >>>> (stack is 00000000b417d4d3..000000001b88c4fe) >>>> <4>[ 594.926975] kernel stack overflow (double-fault): 0000 [#1] >>>> PREEMPT SMP PTI >>>> <4>[ 594.934772] CPU: 7 PID: 2158 Comm: cp Not tainted >>>> 5.0.15-gentoo-f2fslog_data_page #5 >>>> <4>[ 594.938757] Hardware name: To Be Filled By O.E.M. To Be Filled >>>> By O.E.M./C226 WS, BIOS P3.40 06/25/2018 >>>> <4>[ 594.942880] RIP: 0010:tag_pages_for_writeback+0x0/0x180 >>>> <4>[ 594.947036] Code: 30 48 83 ff 07 0f 86 eb fe ff ff 49 8b 56 40 >>>> 49 03 56 38 48 d1 ea 49 89 d0 e9 6a fe ff ff 66 66 2e 0f 1f 84 00 00 >>>> 00 00 00 90 <e8> 2b 41 9f 00 41 54 48 83 c7 08 55 48 89 d5 53 48 83 ec >>>> 38 65 48 >>>> <4>[ 594.951623] RSP: 0018:ffffb3f0028dc000 EFLAGS: 00010246 >>>> <4>[ 594.956187] RAX: 0000000000000001 RBX: ffff9ea2d6bd7080 RCX: >>>> ffffb3f0028dc060 >>>> <4>[ 594.960816] RDX: 0007ffffffffffff RSI: 0000000000000000 RDI: >>>> ffff9ea2d6bd71e0 >>>> <4>[ 594.965375] RBP: ffffb3f0028dc1f0 R08: ffff9ea43eee0100 R09: >>>> 0000000000000000 >>>> <4>[ 594.969873] R10: ffffffff97e485c0 R11: 0000000000000001 R12: >>>> ffff9ea2d6bd71e0 >>>> <4>[ 594.974384] R13: 0000000000000009 R14: 0000000000000000 R15: >>>> ffff9ea43bfce000 >>>> <4>[ 594.978892] FS: 000073c62042f740(0000) >>>> GS:ffff9ea45edc0000(0000) knlGS:0000000000000000 >>>> <4>[ 594.983467] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >>>> <4>[ 594.988055] CR2: ffffb3f0028dbff8 CR3: 0000000073aa8005 CR4: >>>> 00000000003606e0 >>>> <4>[ 594.992689] DR0: 0000000000000000 DR1: 0000000000000000 DR2: >>>> 0000000000000000 >>>> <4>[ 594.997310] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: >>>> 0000000000000400 >>>> <4>[ 595.001860] Call Trace: >>>> <4>[ 595.006323] f2fs_write_cache_pages+0x102/0x450 >>>> <4>[ 595.010829] ? kmem_cache_alloc+0x81/0x6b0 >>>> <4>[ 595.015219] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 595.019483] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.023749] ? __enqueue_entity+0x5/0x60 >>>> <4>[ 595.027995] ? enqueue_task_fair+0x4b1/0xb20 >>>> <4>[ 595.032232] do_writepages+0x41/0xd0 >>>> <4>[ 595.036481] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.040754] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.045017] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.049304] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.053594] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.057892] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.062218] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.066549] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.070880] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.075077] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>>> <4>[ 595.079177] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.083266] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.087352] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.091460] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.095565] ? _raw_spin_lock+0x13/0x30 >>>> <4>[ 595.099650] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.103769] ? __enqueue_entity+0x5/0x60 >>>> <4>[ 595.107922] ? enqueue_task_fair+0x4b1/0xb20 >>>> <4>[ 595.112117] do_writepages+0x41/0xd0 >>>> <4>[ 595.116347] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.120630] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.124931] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.129268] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.133636] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.138039] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.142520] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.146992] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.151467] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.155947] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>>> <4>[ 595.160466] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.164964] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.169478] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.174019] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.178545] ? _raw_spin_lock+0x13/0x30 >>>> <4>[ 595.183050] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.187613] ? __enqueue_entity+0x5/0x60 >>>> <4>[ 595.192150] ? enqueue_task_fair+0x4b1/0xb20 >>>> <4>[ 595.196678] do_writepages+0x41/0xd0 >>>> <4>[ 595.201226] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.205819] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.210430] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.214933] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.219454] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.223862] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.228245] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.232543] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.236761] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.240792] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>>> <4>[ 595.244662] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.248474] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.252281] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.256076] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.259827] ? __enqueue_entity+0x5/0x60 >>>> <4>[ 595.263581] ? enqueue_task_fair+0x4b1/0xb20 >>>> <4>[ 595.267280] do_writepages+0x41/0xd0 >>>> <4>[ 595.270917] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.274529] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.278101] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.281681] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.285226] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.288729] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.292263] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.295800] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.299328] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.302854] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>>> <4>[ 595.306405] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.309936] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.313471] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.317052] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.320637] ? __enqueue_entity+0x5/0x60 >>>> <4>[ 595.324215] ? enqueue_task_fair+0x4b1/0xb20 >>>> <4>[ 595.327802] do_writepages+0x41/0xd0 >>>> <4>[ 595.331372] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.334951] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.338497] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.342030] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.345555] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.349054] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.352579] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.356099] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.359604] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.363110] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>>> <4>[ 595.366639] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.370156] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.373681] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.377218] ? __schedule+0x2cd/0x8b0 >>>> <4>[ 595.380747] ? __lock_page+0x155/0x240 >>>> <4>[ 595.384292] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.387868] ? __enqueue_entity+0x5/0x60 >>>> <4>[ 595.391445] ? enqueue_task_fair+0x4b1/0xb20 >>>> <4>[ 595.395030] do_writepages+0x41/0xd0 >>>> <4>[ 595.398597] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.402169] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.405716] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.409250] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.412777] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.416275] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.419810] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.423335] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.426850] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.430370] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>>> <4>[ 595.433904] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.437432] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.440969] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.444511] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.448045] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.451600] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.455136] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.458661] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.462177] do_writepages+0x41/0xd0 >>>> <4>[ 595.465672] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.469169] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.472643] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.476106] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.479555] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.482984] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.486441] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.489880] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.493322] ? up_read+0x5/0x20 >>>> <4>[ 595.496743] ? f2fs_do_write_data_page+0x2cb/0x940 >>>> <4>[ 595.500205] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.503660] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.507118] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.510615] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.514121] ? __enqueue_entity+0x5/0x60 >>>> <4>[ 595.517622] ? enqueue_task_fair+0x4b1/0xb20 >>>> <4>[ 595.521142] do_writepages+0x41/0xd0 >>>> <4>[ 595.524630] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.528135] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.531601] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.535056] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.538498] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.541912] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.545355] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.548792] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.552235] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.555665] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>>> <4>[ 595.559120] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.562568] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.566033] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.569527] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.573022] ? __enqueue_entity+0x5/0x60 >>>> <4>[ 595.576517] ? enqueue_task_fair+0x4b1/0xb20 >>>> <4>[ 595.580021] do_writepages+0x41/0xd0 >>>> <4>[ 595.583507] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.586997] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.590462] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.593923] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.597377] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.600796] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.604249] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.607691] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.611124] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.614569] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>>> <4>[ 595.618024] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.621477] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.624933] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.628396] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.631852] ? __enqueue_entity+0x5/0x60 >>>> <4>[ 595.635334] ? enqueue_task_fair+0x4b1/0xb20 >>>> <4>[ 595.638828] do_writepages+0x41/0xd0 >>>> <4>[ 595.642314] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.645816] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.649282] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.652726] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.656177] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.659610] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.663057] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.666491] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.669919] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.673342] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>>> <4>[ 595.676791] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.680243] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.683722] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.687244] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.690766] ? __enqueue_entity+0x5/0x60 >>>> <4>[ 595.694278] ? enqueue_task_fair+0x4b1/0xb20 >>>> <4>[ 595.697809] do_writepages+0x41/0xd0 >>>> <4>[ 595.701342] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.704903] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.708451] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.711975] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.715488] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.718983] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.722508] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.726031] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.729539] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.733044] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>>> <4>[ 595.736581] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.740086] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.743605] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.747130] ? _raw_spin_lock+0x13/0x30 >>>> <4>[ 595.750654] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.754190] ? __enqueue_entity+0x5/0x60 >>>> <4>[ 595.757724] ? enqueue_task_fair+0x4b1/0xb20 >>>> <4>[ 595.761288] do_writepages+0x41/0xd0 >>>> <4>[ 595.764855] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.768435] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.772001] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.775544] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.779091] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.782617] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.786164] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.789709] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.793237] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.796755] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>>> <4>[ 595.800292] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.803843] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.807384] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.810944] ? __enqueue_entity+0x5/0x60 >>>> <4>[ 595.814491] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.818039] ? _raw_spin_lock+0x13/0x30 >>>> <4>[ 595.821621] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.825212] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.828808] ? _raw_spin_unlock_irqrestore+0x22/0x40 >>>> <4>[ 595.832427] ? try_to_wake_up+0x52/0x510 >>>> <4>[ 595.836032] do_writepages+0x41/0xd0 >>>> <4>[ 595.839623] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.843222] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.846785] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.850328] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.853856] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.857374] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.860909] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.864465] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.867996] ? up_read+0x5/0x20 >>>> <4>[ 595.871514] ? f2fs_do_write_data_page+0x2cb/0x940 >>>> <4>[ 595.875066] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.878612] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.882190] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.885795] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.889400] ? _raw_spin_lock+0x13/0x30 >>>> <4>[ 595.893018] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.896624] ? mempool_alloc+0x65/0x190 >>>> <4>[ 595.900219] ? find_get_entry+0xdc/0x150 >>>> <4>[ 595.903828] do_writepages+0x41/0xd0 >>>> <4>[ 595.907416] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.911003] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.914549] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.918081] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.921610] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.925130] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.928670] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.932210] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.935742] ? unlock_page+0x5/0x30 >>>> <4>[ 595.939260] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 595.942818] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 595.946381] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 595.949960] ? find_busiest_group+0x40/0x580 >>>> <4>[ 595.953544] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.957121] ? _raw_spin_lock+0x13/0x30 >>>> <4>[ 595.960683] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 595.964267] ? find_get_entry+0xdc/0x150 >>>> <4>[ 595.967849] do_writepages+0x41/0xd0 >>>> <4>[ 595.971404] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 595.974962] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 595.978489] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 595.982005] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 595.985505] ? si_meminfo+0x3f/0x60 >>>> <4>[ 595.988990] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 595.992488] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 595.995995] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 595.999497] ? unlock_page+0x5/0x30 >>>> <4>[ 596.002978] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 596.006497] __write_data_page+0x1c8/0x6e0 >>>> <4>[ 596.010031] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 596.013590] ? stat_show+0xd01/0x1280 >>>> <4>[ 596.017136] ? up_read+0x5/0x20 >>>> <4>[ 596.020661] ? f2fs_getxattr+0x19e/0x3e0 >>>> <4>[ 596.024194] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 596.027720] ? _raw_spin_lock+0x13/0x30 >>>> <4>[ 596.031237] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 596.034759] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 596.038254] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 596.041726] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 596.045174] ? f2fs_inode_dirtied+0x64/0xf0 >>>> <4>[ 596.048619] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 596.052066] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 596.055503] do_writepages+0x41/0xd0 >>>> <4>[ 596.058916] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 596.062334] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 596.065718] ? _raw_spin_unlock+0x12/0x30 >>>> <4>[ 596.069076] ? nr_blockdev_pages+0x59/0x70 >>>> <4>[ 596.072438] ? si_meminfo+0x3f/0x60 >>>> <4>[ 596.075787] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 596.079190] f2fs_sync_dirty_inodes+0xb7/0x1f0 >>>> <4>[ 596.082605] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 596.086024] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 596.089415] f2fs_create+0x131/0x3e0 >>>> <4>[ 596.092781] ? f2fs_lookup+0x20e/0x360 >>>> <4>[ 596.096141] path_openat+0x11a8/0x1590 >>>> <4>[ 596.099493] do_filp_open+0x93/0x100 >>>> <4>[ 596.102836] ? preempt_count_sub+0x5/0xa0 >>>> <4>[ 596.106176] do_sys_open+0x186/0x220 >>>> <4>[ 596.109496] do_syscall_64+0x69/0x399 >>>> <4>[ 596.112799] ? trace_hardirqs_off_thunk+0x1a/0x1c >>>> <4>[ 596.116121] entry_SYSCALL_64_after_hwframe+0x49/0xbe >>>> <4>[ 596.119459] RIP: 0033:0x73c62051f112 >>>> <4>[ 596.122789] Code: 25 00 00 41 00 3d 00 00 41 00 74 4c 48 8d 05 >>>> d5 42 0d 00 8b 00 85 c0 75 6d 89 f2 b8 01 01 00 00 48 89 fe bf 9c ff >>>> ff ff 0f 05 <48> 3d 00 f0 ff ff 0f 87 a2 00 00 00 48 8b 4c 24 28 64 48 >>>> 33 0c 25 >>>> <4>[ 596.126545] RSP: 002b:00007ffe665bf1a0 EFLAGS: 00000246 >>>> ORIG_RAX: 0000000000000101 >>>> <4>[ 596.130299] RAX: ffffffffffffffda RBX: 00007ffe665bf8f0 RCX: >>>> 000073c62051f112 >>>> <4>[ 596.134110] RDX: 00000000000000c1 RSI: 00005670d5042690 RDI: >>>> 00000000ffffff9c >>>> <4>[ 596.137944] RBP: 00007ffe665bf5e0 R08: 00007ffe665bf8f0 R09: >>>> 00007ffe665bf8f0 >>>> <4>[ 596.141784] R10: 0000000000000180 R11: 0000000000000246 R12: >>>> 00000000000001a4 >>>> <4>[ 596.145664] R13: 0000000000000003 R14: 0000000000008000 R15: >>>> 0000000000000000 >>>> <4>[ 596.149536] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc >>>> nls_cp437 uas usb_storage pl2303 sd_mod x86_pkg_temp_thermal kvm_intel >>>> iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich >>>> firewire_ohci firewire_core igb ahci crc_itu_t >>>> processor_thermal_device libahci dca intel_soc_dts_iosf >>>> int340x_thermal_zone pcc_cpufreq efivarfs >>>> <4>[ 596.158385] ---[ end trace d508e01f5312dbde ]--- >>>> --END log one >>>> >>>> --BEGIN log two >>>> <4>[ 462.870628] sync_dirty_inodes: inode:624255, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870633] sync_dirty_inodes: inode:624255, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870634] sync_dirty_inodes: inode:624257, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870638] sync_dirty_inodes: inode:624257, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870660] sync_dirty_inodes: inode:623968, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870665] sync_dirty_inodes: inode:623970, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870679] sync_dirty_inodes: inode:623974, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870681] sync_dirty_inodes: inode:623974, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870691] sync_dirty_inodes: inode:623975, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870696] sync_dirty_inodes: inode:623976, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870697] sync_dirty_inodes: inode:623976, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870711] sync_dirty_inodes: inode:623978, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870715] sync_dirty_inodes: inode:623981, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870717] sync_dirty_inodes: inode:623981, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870719] sync_dirty_inodes: inode:623982, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870721] sync_dirty_inodes: inode:623982, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870754] sync_dirty_inodes: inode:623986, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870755] sync_dirty_inodes: inode:623986, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870774] sync_dirty_inodes: inode:623987, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870775] sync_dirty_inodes: inode:623987, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870783] sync_dirty_inodes: inode:623988, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870788] sync_dirty_inodes: inode:623991, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870794] sync_dirty_inodes: inode:623993, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870795] sync_dirty_inodes: inode:623993, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870801] sync_dirty_inodes: inode:623995, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870808] sync_dirty_inodes: inode:623995, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870810] sync_dirty_inodes: inode:623995, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870812] sync_dirty_inodes: inode:623998, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870813] sync_dirty_inodes: inode:623998, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870848] sync_dirty_inodes: inode:624000, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870852] sync_dirty_inodes: inode:624000, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870853] sync_dirty_inodes: inode:624003, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870856] sync_dirty_inodes: inode:624003, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870858] sync_dirty_inodes: inode:624004, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870860] sync_dirty_inodes: inode:624004, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870862] sync_dirty_inodes: inode:624007, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870863] sync_dirty_inodes: inode:624007, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870875] sync_dirty_inodes: inode:624009, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870902] sync_dirty_inodes: inode:624009, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870905] sync_dirty_inodes: inode:624010, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870908] sync_dirty_inodes: inode:624010, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870913] sync_dirty_inodes: inode:624013, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870917] sync_dirty_inodes: inode:624014, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870921] sync_dirty_inodes: inode:624014, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870947] sync_dirty_inodes: inode:624016, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870951] sync_dirty_inodes: inode:624016, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870953] sync_dirty_inodes: inode:624016, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870955] sync_dirty_inodes: inode:624016, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870956] sync_dirty_inodes: inode:624017, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870958] sync_dirty_inodes: inode:624017, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870961] sync_dirty_inodes: inode:624021, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870962] sync_dirty_inodes: inode:624021, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870968] sync_dirty_inodes: inode:624022, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870970] sync_dirty_inodes: inode:624022, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870975] sync_dirty_inodes: inode:624024, >>>> cp_task:000000006480c3a0 >>>> <4>[ 462.870976] sync_dirty_inodes: inode:624024, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870989] sync_dirty_inodes: inode:624026, >>>> cp_task:000000002fea927f >>>> <4>[ 462.870990] sync_dirty_inodes: inode:624026, >>>> cp_task:000000006480c3a0 >>>> <0>[ 462.870997] BUG: stack guard page was hit at 0000000056e52d0a >>>> (stack is 00000000f6235c3f..00000000f20b69a0) >>>> <4>[ 462.876404] kernel stack overflow (double-fault): 0000 [#1] >>>> PREEMPT SMP PTI >>>> <4>[ 462.881794] CPU: 6 PID: 875 Comm: kworker/u16:9 Not tainted >>>> 5.0.15-gentoo-f2fslog_sync_dirty_inodes #5 >>>> <4>[ 462.887185] Hardware name: To Be Filled By O.E.M. To Be Filled >>>> By O.E.M./C226 WS, BIOS P3.40 06/25/2018 >>>> <4>[ 462.892563] Workqueue: writeback wb_workfn (flush-259:0) >>>> <4>[ 462.897877] RIP: 0010:tracer_hardirqs_off+0x4/0xf0 >>>> <4>[ 462.903078] Code: ee 48 0f 44 dd 4c 89 ff 48 89 da e8 d6 f9 ff >>>> ff 49 c7 45 20 00 00 00 00 f0 41 ff 4d 00 e9 67 ff ff ff 0f 1f 40 00 >>>> 41 57 41 56 <41> 55 41 54 55 53 48 83 ec 08 65 44 8b 25 0a 9f 06 4b f6 >>>> 05 53 12 >>>> <4>[ 462.908571] RSP: 0018:ffffaa364054c000 EFLAGS: 00010046 >>>> <4>[ 462.913921] RAX: 0000000000000000 RBX: ffff962b5b81ca80 RCX: >>>> ffffffffb6b66b50 >>>> <4>[ 462.919235] RDX: ffffffffb6b66b40 RSI: 0000000000000000 RDI: >>>> ffffffffb5059c74 >>>> <4>[ 462.924523] RBP: 0000000000411200 R08: 0000000000000002 R09: >>>> 0000000000000001 >>>> <4>[ 462.929772] R10: 0000000001cb731b R11: 0000000001cb731c R12: >>>> 0000000000411200 >>>> <4>[ 462.934889] R13: 0000000000000246 R14: ffffaa364054c0c8 R15: >>>> ffffffffb4f29520 >>>> <4>[ 462.939888] FS: 0000000000000000(0000) >>>> GS:ffff962b5ed80000(0000) knlGS:0000000000000000 >>>> <4>[ 462.944841] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >>>> <4>[ 462.949681] CR2: ffffaa364054bff8 CR3: 000000072380c002 CR4: >>>> 00000000003606e0 >>>> <4>[ 462.954463] DR0: 0000000000000000 DR1: 0000000000000000 DR2: >>>> 0000000000000000 >>>> <4>[ 462.959191] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: >>>> 0000000000000400 >>>> <4>[ 462.963797] Call Trace: >>>> <4>[ 462.968280] ? finish_wait+0x80/0x80 >>>> <4>[ 462.972785] trace_hardirqs_off+0x2b/0xe0 >>>> <4>[ 462.977174] kmem_cache_alloc+0x44/0x6b0 >>>> <4>[ 462.981443] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 462.985703] ? finish_wait+0x80/0x80 >>>> <4>[ 462.989939] mempool_alloc+0x65/0x190 >>>> <4>[ 462.994143] ? preempt_count_add+0x79/0xb0 >>>> <4>[ 462.998379] bio_alloc_bioset+0x140/0x210 >>>> <4>[ 463.002628] f2fs_submit_page_write+0x1bb/0x5e0 >>>> <4>[ 463.006912] do_write_page+0x99/0x180 >>>> <4>[ 463.011201] f2fs_outplace_write_data+0x55/0xe0 >>>> <4>[ 463.015538] f2fs_do_write_data_page+0x588/0x940 >>>> <4>[ 463.019897] __write_data_page+0x4f2/0x6f0 >>>> <4>[ 463.024251] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.028616] ? pointer+0x110/0x3b0 >>>> <4>[ 463.032972] ? log_store+0xf5/0x260 >>>> <4>[ 463.037183] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.041277] ? __down_trylock_console_sem.isra.14+0x42/0x50 >>>> <4>[ 463.045404] ? vprintk_emit+0x28f/0x2a0 >>>> <4>[ 463.049517] do_writepages+0x41/0xd0 >>>> <4>[ 463.053634] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.057778] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.061911] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.066055] ? up_read+0x5/0x20 >>>> <4>[ 463.070224] ? f2fs_do_write_data_page+0x2cb/0x940 >>>> <4>[ 463.074464] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.078742] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.083072] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.087459] ? pointer+0x110/0x3b0 >>>> <4>[ 463.091880] ? log_store+0xf5/0x260 >>>> <4>[ 463.096333] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.100853] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.105422] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.109943] do_writepages+0x41/0xd0 >>>> <4>[ 463.114451] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.119000] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.123565] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.128116] ? up_read+0x5/0x20 >>>> <4>[ 463.132657] ? f2fs_do_write_data_page+0x2cb/0x940 >>>> <4>[ 463.137238] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.141817] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.146390] ? _raw_spin_unlock_irqrestore+0x34/0x40 >>>> <4>[ 463.151025] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.155642] ? pointer+0x110/0x3b0 >>>> <4>[ 463.160240] ? log_store+0xf5/0x260 >>>> <4>[ 463.164859] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.169533] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.174254] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.178838] do_writepages+0x41/0xd0 >>>> <4>[ 463.183323] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.187757] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.192105] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.196367] ? up_read+0x5/0x20 >>>> <4>[ 463.200592] ? f2fs_do_write_data_page+0x2cb/0x940 >>>> <4>[ 463.204670] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.208563] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.212425] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.216279] ? pointer+0x110/0x3b0 >>>> <4>[ 463.220107] ? log_store+0xf5/0x260 >>>> <4>[ 463.223927] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.227747] ? __down_trylock_console_sem.isra.14+0x42/0x50 >>>> <4>[ 463.231564] ? vprintk_emit+0x28f/0x2a0 >>>> <4>[ 463.235333] do_writepages+0x41/0xd0 >>>> <4>[ 463.239049] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.242783] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.246543] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.250250] ? up_read+0x5/0x20 >>>> <4>[ 463.253949] ? f2fs_do_write_data_page+0x2cb/0x940 >>>> <4>[ 463.257687] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.261419] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.265159] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.268916] ? pointer+0x110/0x3b0 >>>> <4>[ 463.272675] ? log_store+0xf5/0x260 >>>> <4>[ 463.276420] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.280192] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.283952] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.287664] do_writepages+0x41/0xd0 >>>> <4>[ 463.291360] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.295087] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.298810] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.302519] ? up_read+0x5/0x20 >>>> <4>[ 463.306207] ? f2fs_do_write_data_page+0x2cb/0x940 >>>> <4>[ 463.309944] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.313670] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.317379] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.321080] ? pointer+0x110/0x3b0 >>>> <4>[ 463.324773] ? log_store+0xf5/0x260 >>>> <4>[ 463.328444] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.332132] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.335820] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.339467] do_writepages+0x41/0xd0 >>>> <4>[ 463.343094] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.346740] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.350392] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.354029] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >>>> <4>[ 463.357687] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.361353] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.365020] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.368695] ? pointer+0x110/0x3b0 >>>> <4>[ 463.372399] ? log_store+0xf5/0x260 >>>> <4>[ 463.376091] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.379796] ? __down_trylock_console_sem.isra.14+0x42/0x50 >>>> <4>[ 463.383534] ? vprintk_emit+0x28f/0x2a0 >>>> <4>[ 463.387267] do_writepages+0x41/0xd0 >>>> <4>[ 463.390989] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.394729] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.398460] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.402181] ? up_read+0x5/0x20 >>>> <4>[ 463.405875] ? f2fs_do_write_data_page+0x2cb/0x940 >>>> <4>[ 463.409590] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.413295] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.417019] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.420752] ? pointer+0x110/0x3b0 >>>> <4>[ 463.424466] ? log_store+0xf5/0x260 >>>> <4>[ 463.428207] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.431971] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.435729] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.439437] do_writepages+0x41/0xd0 >>>> <4>[ 463.443137] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.446863] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.450581] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.454281] ? up_read+0x5/0x20 >>>> <4>[ 463.457962] ? f2fs_do_write_data_page+0x2cb/0x940 >>>> <4>[ 463.461671] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.465368] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.469084] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.472807] ? pointer+0x110/0x3b0 >>>> <4>[ 463.476499] ? log_store+0xf5/0x260 >>>> <4>[ 463.480154] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.483842] ? __down_trylock_console_sem.isra.14+0x42/0x50 >>>> <4>[ 463.487562] ? vprintk_emit+0x28f/0x2a0 >>>> <4>[ 463.491266] do_writepages+0x41/0xd0 >>>> <4>[ 463.494963] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.498684] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.502397] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.506098] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >>>> <4>[ 463.509824] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.513542] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.517257] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.521001] ? pointer+0x110/0x3b0 >>>> <4>[ 463.524731] ? log_store+0xf5/0x260 >>>> <4>[ 463.528437] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.532208] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.535979] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.539701] do_writepages+0x41/0xd0 >>>> <4>[ 463.543413] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.547150] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.550888] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.554611] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >>>> <4>[ 463.558355] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.562091] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.565824] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.569569] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.573317] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.577052] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.580741] do_writepages+0x41/0xd0 >>>> <4>[ 463.584464] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.588216] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.591965] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.595696] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >>>> <4>[ 463.599454] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.603195] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.606931] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.610679] ? pointer+0x110/0x3b0 >>>> <4>[ 463.614406] ? log_store+0xf5/0x260 >>>> <4>[ 463.618115] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.621860] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.625598] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.629284] do_writepages+0x41/0xd0 >>>> <4>[ 463.632926] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.636607] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.640281] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.643949] ? up_read+0x5/0x20 >>>> <4>[ 463.647596] ? f2fs_do_write_data_page+0x2cb/0x940 >>>> <4>[ 463.651274] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.654945] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.658605] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.662281] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.665936] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.669582] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.673203] do_writepages+0x41/0xd0 >>>> <4>[ 463.676809] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.680434] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.684108] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.687775] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >>>> <4>[ 463.691461] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.695135] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.698813] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.702513] ? pointer+0x110/0x3b0 >>>> <4>[ 463.706187] ? log_store+0xf5/0x260 >>>> <4>[ 463.709837] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.713508] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.717171] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.720786] do_writepages+0x41/0xd0 >>>> <4>[ 463.724405] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.728046] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.731666] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.735258] ? _raw_spin_unlock_irqrestore+0x3d/0x40 >>>> <4>[ 463.738885] f2fs_balance_fs+0xe5/0x2c0 >>>> <4>[ 463.742501] __write_data_page+0x5b7/0x6f0 >>>> <4>[ 463.746115] f2fs_write_cache_pages+0x1e0/0x450 >>>> <4>[ 463.749744] ? pointer+0x110/0x3b0 >>>> <4>[ 463.753345] ? log_store+0xf5/0x260 >>>> <4>[ 463.756920] f2fs_write_data_pages+0x2ed/0x320 >>>> <4>[ 463.760517] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.764105] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.767643] do_writepages+0x41/0xd0 >>>> <4>[ 463.771191] __filemap_fdatawrite_range+0x81/0xb0 >>>> <4>[ 463.774760] f2fs_sync_dirty_inodes+0x1dd/0x200 >>>> <4>[ 463.778328] f2fs_balance_fs_bg+0x2a7/0x2c0 >>>> <4>[ 463.781928] ? trace_hardirqs_on+0x2c/0xe0 >>>> <4>[ 463.785524] ? retint_kernel+0x2d/0x2d >>>> <4>[ 463.789100] f2fs_write_node_pages+0x51/0x220 >>>> <4>[ 463.792687] ? trace_hardirqs_on_caller+0x32/0xf0 >>>> <4>[ 463.796285] do_writepages+0x41/0xd0 >>>> <4>[ 463.799860] ? retint_kernel+0x2d/0x2d >>>> <4>[ 463.803417] __writeback_single_inode+0x3d/0x3d0 >>>> <4>[ 463.806999] writeback_sb_inodes+0x1e8/0x410 >>>> <4>[ 463.810573] __writeback_inodes_wb+0x5d/0xb0 >>>> <4>[ 463.814158] wb_writeback+0x28f/0x340 >>>> <4>[ 463.817730] wb_workfn+0x2d3/0x420 >>>> <4>[ 463.821328] process_one_work+0x1a1/0x3d0 >>>> <4>[ 463.824935] worker_thread+0x30/0x380 >>>> <4>[ 463.828520] ? process_one_work+0x3d0/0x3d0 >>>> <4>[ 463.832105] kthread+0x116/0x130 >>>> <4>[ 463.835675] ? kthread_create_worker_on_cpu+0x70/0x70 >>>> <4>[ 463.839278] ret_from_fork+0x3a/0x50 >>>> <4>[ 463.842855] Modules linked in: ipv6 crc_ccitt 8021q garp stp llc >>>> nls_cp437 uas usb_storage pl2303 sd_mod x86_pkg_temp_thermal kvm_intel >>>> iTCO_wdt kvm irqbypass ghash_clmulni_intel serio_raw i2c_i801 lpc_ich >>>> firewire_ohci firewire_core igb crc_itu_t dca ahci libahci >>>> processor_thermal_device intel_soc_dts_iosf int340x_thermal_zone >>>> pcc_cpufreq efivarfs >>>> <4>[ 463.851196] ---[ end trace b6d0a43e90218d45 ]--- >>>> --END log two >>>> >>>>> >>>>> --- a/fs/f2fs/checkpoint.c >>>>> +++ b/fs/f2fs/checkpoint.c >>>>> @@ -1074,6 +1074,11 @@ int f2fs_sync_dirty_inodes(struct f2fs_sb_info >>>>> *sbi, >>>>> enum >>>>> inode_type type) >>>>> >>>>> F2FS_I(inode)->cp_task = current; >>>>> >>>>> + smp_mb(); >>>>> + >>>>> + printk("sync_dirty_inodes: inode:%lu, cp_task:%p", >>>>> + inode->i_ino, F2FS_I(inode)->cp_task); >>>>> + >>>>> filemap_fdatawrite(inode->i_mapping); >>>>> >>>>> F2FS_I(inode)->cp_task = NULL; >>>>> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c >>>>> index 88988241a0e8..84fbbd337c37 100644 >>>>> --- a/fs/f2fs/data.c >>>>> +++ b/fs/f2fs/data.c >>>>> @@ -2392,6 +2392,10 @@ int f2fs_write_single_data_page(struct page >>>>> *page, >>>>> int >>>>> *submitted, >>>>> if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) && >>>>> !F2FS_I(inode)->cp_task) { >>>>> f2fs_submit_ipu_bio(sbi, bio, page); >>>>> + >>>>> + printk("write_data_page: inode:%lu, cp_task:%p", >>>>> + inode->i_ino, F2FS_I(inode)->cp_task); >>>>> + >>>>> f2fs_balance_fs(sbi, need_balance_fs); >>>>> } >>>>> >>>>> Thanks, >>>> >>> . >>> >> ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2019-05-20 15:52 UTC | newest] Thread overview: 16+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-03-27 13:59 BUG: kernel stack overflow when mounting with data_flush Hagbard Celine 2019-03-30 3:25 ` Chao Yu 2019-03-30 7:29 ` Chao Yu 2019-03-30 18:54 ` Hagbard Celine 2019-04-01 8:05 ` Chao Yu 2019-04-02 12:41 ` Hagbard Celine 2019-04-02 13:31 ` Chao Yu 2019-05-14 18:13 ` Hagbard Celine 2019-05-15 2:25 ` Chao Yu 2019-05-15 8:03 ` Hagbard Celine 2019-05-15 8:13 ` Chao Yu 2019-05-15 16:50 ` Hagbard Celine 2019-05-15 17:01 ` Hagbard Celine 2019-05-20 9:37 ` Chao Yu 2019-05-20 12:58 ` Hagbard Celine 2019-05-20 15:51 ` Chao Yu
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).