From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77D45352C39 for ; Sat, 2 May 2026 12:42:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777725728; cv=none; b=F+7vD7yaVz3ValUQtPBdABKz0Jn3puZF0GpdEm2mwpQlRjrRoQ14gTBlYAb+JqUSz5b3hcG16CbtZdKCIg6n24isq8nW0MsD29V+U6dvFsY5vuRPkhDbgwYNb2OfLD1Ffde+qpuu7wKzTFqO4uGlZZFJUR69prTCZ2TioICP9vQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777725728; c=relaxed/simple; bh=ig9xNJcBofog0t8+4HrBm2cVba3DiZW60+qd2gplkqE=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=oCESda8VbNEQgWzETaC54JU/4mQkw+o7QWm1ve6X9dm1Eenygebd0EtVeUTPp4SsYsmj0AFt2GwWVICdr92cgYMhDwFSydtqWwpdUCCLGXcZ2YCFhg+6y5r+WjBdKvE53njZWSKaOAgqATer3FPDEMwXZvX7+NZJZvyoPUfyAAA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WZHuFmha; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WZHuFmha" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-35fbca04006so1341923a91.1 for ; Sat, 02 May 2026 05:42:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1777725727; x=1778330527; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=0TYlNOGKPAoKWtSOwNLPArH6N1dYOc6JBwGgHZq51pU=; b=WZHuFmha2YNjtkxyGM85llZrLKlY48pQGnVQRVGYp8cWGCg9RmfQ4nPz/4LAGH2k4Z 1Ic4GL4QTRdosjiC1rKWEfT7G2ZGfchBurRcevl0G0cgqJOuOEUmbhMmg6qe/6xOTBVd LjB/WMwG1LxkbG7g2bVnurNHLPUBB8//7ypzWJw6gS0aNxb4XXyJvREygOnU005Vqkar L+Q29cc5x0yql1oJIQnpnSUuOYtBHJCMgJ4AUFt7Ytj8SV+6t0SnRgocdMcZ4XreB8CC up4nk0t2la1ZX9CA7rPH1WwWtbyutQlvoF9UObrby8xIikRAoK7ho9vYRLtiY5jg4JMi g3qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777725727; x=1778330527; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=0TYlNOGKPAoKWtSOwNLPArH6N1dYOc6JBwGgHZq51pU=; b=HargvyyvYq/0vxs+vXSkSvDpGj0F+tAp+r+gAq11Xo8GdEm8JioDdYKx7XN0tQr+8L 2fFs2KL4yY+UtiblK+qabZBi2Q+B1frzDyOb14Uobf/DUBYD1pkv6/woZJ7xYeD2SZtw dDbgFQPLnqOe/nyDZiXbTCqOE3wxrAHGNj61mnbW540H1uwJgRh00QV/WwBFYLFZju42 YbFOJl3GQObnEY+MqX7vt8ua5U/jqSRQdliEbGDQx7PVohl49E5WYZduBk+H2+O1/oDP OGWn0GUKQoc4I/r+HoN/LODsHh3hIWuVIsty3yup4rpwZ2B9RcgKZ07AGGEnXiZuFFcS 320A== X-Forwarded-Encrypted: i=1; AFNElJ9CgPWPzUSH2pLQzNGfXQI6r8ubWroBhGfnUYzbn1weg6+X/epBvsZ/NLSvWc5wt1nWaMJu940z84BKpHQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yy7VVXZjRCBEL0KuuJVX8CyoTUaFECb1NDgWlkuoPgj4loG7Hvi TW2DyEZIfGTcC40Mo6HS6hM4AycPGJgjCn5Uv0TMB22NQjg3Lk+cHsdoul9BXyQl X-Gm-Gg: AeBDiesRtGVoLK/Q6M6oT7RKjo9Tivr2mK3HTzvOcKSSPulNiyTeu9EyqUdDL3fabEh PTsiFz+87I8s5Rd7njK0vYvVy2p0RBaA47ALfgE5EZbC10MC8iAPZeJdukGVR542gree0PZSbWO Mx/cwBQEAgryl/qcybSYEaT0OBBWOz6rEXxvH0Tp4pCxtIRb2LWKAI2aoQIOGKzweqsg1C6urXT WLJuB7SOjeWzxGgraF53ggTW8ef54IlPGMIh7zFUuqJuxTFO9rgRA1BaIrOtLu7Qh5piecz3Yly 5omuoB0ktBJcXu0TSrEW3Dv4d1h3R+4Y8OA7hAPANgh2th0fQoierByohksPsqEPpeJXrEkv3NI ONrpvUD+QQavOtymsy2ERUzmgciEZI9G3qz+Tm74/n114wamWRdQ0XLN7cg+gAXbInoD+SuHzjG tYFEZnCOgyEtkNA+1M5tqIUUT3Or/ZPQWQtTkSIxs2wEPbhgnKLgA/RYitC6Mctmrnlsi9JASQm XOl31+oHJ6g4xTG X-Received: by 2002:a17:90b:2b45:b0:35f:b69d:7292 with SMTP id 98e67ed59e1d1-3650ce36fe3mr2803961a91.15.1777725726567; Sat, 02 May 2026 05:42:06 -0700 (PDT) Received: from lixiang-ThinkCentre-M755e-N000.company.local ([210.184.73.204]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-364ebec73aasm5394930a91.2.2026.05.02.05.42.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 02 May 2026 05:42:06 -0700 (PDT) From: ruipengqi To: jaegeuk@kernel.org Cc: chao@kernel.org, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Ruipeng Qi Subject: [PATCH v3] f2fs: fix potential deadlock in f2fs_balance_fs() Date: Sat, 2 May 2026 20:41:57 +0800 Message-Id: <20260502124157.3406780-1-ruipengqi3@gmail.com> X-Mailer: git-send-email 2.25.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Ruipeng Qi When the f2fs filesystem space is nearly exhausted, we encounter deadlock issues as below: INFO: task A:1890 blocked for more than 120 seconds. Tainted: G O 6.12.41-g3fe07ddf05ab #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. task:A state:D stack:0 pid:1890 tgid:1626 ppid:1153 flags:0x00000204 Call trace: __switch_to+0xf4/0x158 __schedule+0x27c/0x908 schedule+0x3c/0x118 io_schedule+0x44/0x68 folio_wait_bit_common+0x174/0x370 folio_wait_bit+0x20/0x38 folio_wait_writeback+0x54/0xc8 truncate_inode_partial_folio+0x70/0x1e0 truncate_inode_pages_range+0x1b0/0x450 truncate_pagecache+0x54/0x88 f2fs_file_write_iter+0x3e8/0xb80 do_iter_readv_writev+0xf0/0x1e0 vfs_writev+0x138/0x2c8 do_writev+0x88/0x130 __arm64_sys_writev+0x28/0x40 invoke_syscall+0x50/0x120 el0_svc_common.constprop.0+0xc8/0xf0 do_el0_svc+0x24/0x38 el0_svc+0x30/0xf8 el0t_64_sync_handler+0x120/0x130 el0t_64_sync+0x190/0x198 INFO: task kworker/u8:11:2680853 blocked for more than 120 seconds. Tainted: G O 6.12.41-g3fe07ddf05ab #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. task:kworker/u8:11 state:D stack:0 pid:2680853 tgid:2680853 ppid:2 flags:0x00000208 Workqueue: writeback wb_workfn (flush-254:0) Call trace: __switch_to+0xf4/0x158 __schedule+0x27c/0x908 schedule+0x3c/0x118 io_schedule+0x44/0x68 folio_wait_bit_common+0x174/0x370 __filemap_get_folio+0x214/0x348 pagecache_get_page+0x20/0x70 f2fs_get_read_data_page+0x150/0x3e8 f2fs_get_lock_data_page+0x2c/0x160 move_data_page+0x50/0x478 do_garbage_collect+0xd38/0x1528 f2fs_gc+0x240/0x7e0 f2fs_balance_fs+0x1a0/0x208 f2fs_write_single_data_page+0x6e4/0x730 f2fs_write_cache_pages+0x378/0x9b0 f2fs_write_data_pages+0x2e4/0x388 do_writepages+0x8c/0x2c8 __writeback_single_inode+0x4c/0x498 writeback_sb_inodes+0x234/0x4a8 __writeback_inodes_wb+0x58/0x118 wb_writeback+0x2f8/0x3c0 wb_workfn+0x2c4/0x508 process_one_work+0x180/0x408 worker_thread+0x258/0x368 kthread+0x118/0x128 ret_from_fork+0x10/0x200 INFO: task kworker/u8:8:2641297 blocked for more than 120 seconds. Tainted: G O 6.12.41-g3fe07ddf05ab #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. task:kworker/u8:8 state:D stack:0 pid:2641297 tgid:2641297 ppid:2 flags:0x00000208 Workqueue: writeback wb_workfn (flush-254:0) Call trace: __switch_to+0xf4/0x158 __schedule+0x27c/0x908 rt_mutex_schedule+0x30/0x60 __rt_mutex_slowlock_locked.constprop.0+0x460/0x8a8 rwbase_write_lock+0x24c/0x378 down_write+0x1c/0x30 f2fs_balance_fs+0x184/0x208 f2fs_write_inode+0xf4/0x328 __writeback_single_inode+0x370/0x498 writeback_sb_inodes+0x234/0x4a8 __writeback_inodes_wb+0x58/0x118 wb_writeback+0x2f8/0x3c0 wb_workfn+0x2c4/0x508 process_one_work+0x180/0x408 worker_thread+0x258/0x368 kthread+0x118/0x128 ret_from_fork+0x10/0x20 INFO: task B:1902 blocked for more than 120 seconds. Tainted: G O 6.12.41-g3fe07ddf05ab #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. task:B state:D stack:0 pid:1902 tgid:1626 ppid:1153 flags:0x0000020c Call trace: __switch_to+0xf4/0x158 __schedule+0x27c/0x908 rt_mutex_schedule+0x30/0x60 __rt_mutex_slowlock_locked.constprop.0+0x460/0x8a8 rwbase_write_lock+0x24c/0x378 down_write+0x1c/0x30 f2fs_balance_fs+0x184/0x208 f2fs_map_blocks+0x94c/0x1110 f2fs_file_write_iter+0x228/0xb80 do_iter_readv_writev+0xf0/0x1e0 vfs_writev+0x138/0x2c8 do_writev+0x88/0x130 __arm64_sys_writev+0x28/0x40 invoke_syscall+0x50/0x120 el0_svc_common.constprop.0+0xc8/0xf0 do_el0_svc+0x24/0x38 el0_svc+0x30/0xf8 el0t_64_sync_handler+0x120/0x130 el0t_64_sync+0x190/0x198 INFO: task sync:2769849 blocked for more than 120 seconds. Tainted: G O 6.12.41-g3fe07ddf05ab #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. task:sync state:D stack:0 pid:2769849 tgid:2769849 ppid:736 flags:0x0000020c Call trace: __switch_to+0xf4/0x158 __schedule+0x27c/0x908 schedule+0x3c/0x118 wb_wait_for_completion+0xb0/0xe8 sync_inodes_sb+0xc8/0x2b0 sync_inodes_one_sb+0x24/0x38 iterate_supers+0xa8/0x138 ksys_sync+0x54/0xc8 __arm64_sys_sync+0x18/0x30 invoke_syscall+0x50/0x120 el0_svc_common.constprop.0+0xc8/0xf0 do_el0_svc+0x24/0x38 el0_svc+0x30/0xf8 el0t_64_sync_handler+0x120/0x130 el0t_64_sync+0x190/0x198 The root cause is a potential deadlock between the following tasks: kworker/u8:11 Thread A - f2fs_write_single_data_page - f2fs_do_write_data_page - folio_start_writeback(X) - f2fs_outplace_write_data - bio_add_folio(X) - folio_unlock(X) - truncate_inode_pages_range - __filemap_get_folio(X, FGP_LOCK) - truncate_inode_partial_folio(X) - folio_wait_writeback(X) - f2fs_balance_fs - f2fs_gc - do_garbage_collect - move_data_page - f2fs_get_lock_data_page - __filemap_get_folio(X, FGP_LOCK) Both threads try to access folio X. Thread A holds the lock but waits for writeback, while kworker waits for the lock. This causes a deadlock. Other threads also enter D state, waiting for locks such as gc_lock and writepages. OPU/IPU DATA folio are all affected by this issue. To avoid such potential deadlocks, always commit these cached folios before triggering f2fs_gc() in f2fs_balance_fs(). v2: - Commit cached OPU/IPU folios, not just OPU folios as in v1. v3: - Fixed minor grammatical issues - Add comment on lockless list_empty() to explain why it is safe without holding bio_list_lock Suggested-by: Chao Signed-off-by: Ruipeng Qi --- fs/f2fs/data.c | 29 +++++++++++++++++++++++++++++ fs/f2fs/f2fs.h | 1 + fs/f2fs/segment.c | 8 ++++++++ 3 files changed, 38 insertions(+) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index 338df7a2aea6..98e3863b9b54 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -939,6 +939,35 @@ void f2fs_submit_merged_ipu_write(struct f2fs_sb_info *sbi, } } +void f2fs_submit_all_merged_ipu_writes(struct f2fs_sb_info *sbi) +{ + struct bio_entry *be, *tmp; + struct f2fs_bio_info *io; + enum temp_type temp; + + for (temp = HOT; temp < NR_TEMP_TYPE; temp++) { + LIST_HEAD(list); + + io = sbi->write_io[DATA] + temp; + + /* A lockless list_empty() check is safe here: any bios from + * other kworkers that we miss will be submitted by those + * kworkers accordingly. + */ + if (list_empty(&io->bio_list)) + continue; + + f2fs_down_write(&io->bio_list_lock); + list_splice_init(&io->bio_list, &list); + f2fs_up_write(&io->bio_list_lock); + + list_for_each_entry_safe(be, tmp, &list, list) { + f2fs_submit_write_bio(sbi, be->bio, DATA); + del_bio_entry(be); + } + } +} + int f2fs_merge_page_bio(struct f2fs_io_info *fio) { struct bio *bio = *fio->bio; diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index bb34e864d0ef..e9038ab1b2bd 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -4148,6 +4148,7 @@ void f2fs_submit_merged_write_folio(struct f2fs_sb_info *sbi, struct folio *folio, enum page_type type); void f2fs_submit_merged_ipu_write(struct f2fs_sb_info *sbi, struct bio **bio, struct folio *folio); +void f2fs_submit_all_merged_ipu_writes(struct f2fs_sb_info *sbi); void f2fs_flush_merged_writes(struct f2fs_sb_info *sbi); int f2fs_submit_page_bio(struct f2fs_io_info *fio); int f2fs_merge_page_bio(struct f2fs_io_info *fio); diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index 6a97fe76712b..b57d36f66681 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -462,6 +462,14 @@ void f2fs_balance_fs(struct f2fs_sb_info *sbi, bool need) .should_migrate_blocks = false, .err_gc_skipped = false, .nr_free_secs = 1 }; + + /* + * Submit all cached OPU/IPU DATA bios before triggering + * foreground GC to avoid potential deadlocks. + */ + f2fs_submit_merged_write(sbi, DATA); + f2fs_submit_all_merged_ipu_writes(sbi); + f2fs_down_write_trace(&sbi->gc_lock, &gc_control.lc); stat_inc_gc_call_count(sbi, FOREGROUND); f2fs_gc(sbi, &gc_control); -- 2.25.1