From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 09BEC14A4CC for ; Sun, 26 Jul 2026 03:53:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785037993; cv=none; b=CTvJ/6VrLLb6Ozp7WnK1phY6CNalukuthQZCrG7ur0Ri9YTwI/HnmJ+90+9nse19wpcL/2jAHCUS+phKGO91eSt6eXpTYlf2iU3RSgDV4zqVQdPWAVU52yD9jtbgLorUynKBEdGmWwoPg7ucVej5W3rzBatdDDvf5FTmZgN1xiM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785037993; c=relaxed/simple; bh=up8u1A8KDhweYWnJ8xn/9Nuj5CHC3RaeggSKJ2CUJLM=; h=Date:To:From:Subject:Message-Id; b=uLwsBBWGxaf7kJhr1AuB4wVYYtpRKNhj8bJN1rpJkNj2Ue5JqLePzjtrjy6urIP+8jhtwaU+aGTyFbTUDKBb93uljx0+5JRUWGaPZOf5GMX7jcPVyC5RwW49wD9XDURugTg1bcc7Na+y4kZlSH7l6y9x1u9gNFxo+qtqu5ZBuXw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=um9wxzmG; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="um9wxzmG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 869E71F000E9; Sun, 26 Jul 2026 03:53:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=korg; t=1785037991; bh=0O8/d8gjwQ9WDIAUBRR7bhFNaeQJ8t9mgVRqMaMafMY=; h=Date:To:From:Subject; b=um9wxzmGn1L++Xj/UnnI4uoON29G/6gw228rQd9fJDiTxqgsj+wVT0856W8NglKZz aHWBI6i8vIqJ86BRZ2uRc+Lg2zaL2VQkoxUX6D/Vw/GaqhMIUu9hXFRNKyYOQ1OQG2 C0v2gO/Eu+SFZ1hoh45xz77HYDB23fCUv2P+VA8s= Date: Sat, 25 Jul 2026 20:53:11 -0700 To: mm-commits@vger.kernel.org,piaojun@huawei.com,mark@fasheh.com,junxiao.bi@oracle.com,joseph.qi@linux.alibaba.com,jlbec@evilplan.org,heming.zhao@suse.com,gechangwei@live.cn,zzzccc427@gmail.com,akpm@linux-foundation.org From: Andrew Morton Subject: + ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown.patch added to mm-nonmm-unstable branch Message-Id: <20260726035311.869E71F000E9@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The patch titled Subject: ocfs2: synchronize heartbeat callbacks with o2net teardown has been added to the -mm mm-nonmm-unstable branch. Its filename is ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown.patch This patch will later appear in the mm-nonmm-unstable branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via various branches at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there most days ------------------------------------------------------ From: Cen Zhang Subject: ocfs2: synchronize heartbeat callbacks with o2net teardown Date: Wed, 24 Jun 2026 17:53:09 +0800 Patch series "ocfs2: harden heartbeat teardown races". This series fixes two OCFS2 heartbeat/o2net teardown races found by KASAN. This patch (of 2): Heartbeat callbacks stay registered while configfs local-node teardown enters o2net_stop_listening(). A node-down event can still run through o2net_disconnect_node() and o2net_set_nn_state() while teardown is destroying o2net_wq, so the later queue/flush operations can hit a dead workqueue. KASAN has caught this as a slab-use-after-free in __queue_work() with the call chain: KASAN slab-use-after-free in __queue_work+0x56/0xa90 Read of size 4 Call trace: dump_stack_lvl+0x66/0xa0 print_report+0xce/0x630 __queue_work+0x56/0xa90 srso_alias_return_thunk+0x5/0xfbef5 __virt_addr_valid+0x19f/0x330 kasan_report+0xe0/0x110 __queue_delayed_work+0x58/0x1e0 queue_delayed_work_on+0xb4/0xc0 o2net_set_nn_state+0x467/0x840 o2net_disconnect_node+0x7b/0xe0 o2net_hb_node_down_cb+0x54/0x60 o2hb_run_event_list+0x236/0x2d0 o2hb_check_slot+0xad4/0xbc0 lock_release+0xc8/0x290 o2hb_check_slot+0x9ea/0xbc0 trace_hardirqs_on+0x18/0x130 o2hb_do_disk_heartbeat+0x646/0xb30 (fs/ocfs2/cluster/heartbeat.c:1079) __lock_acquire+0x466/0x2260 lockdep_hardirqs_on_prepare+0xea/0x1a0 ktime_get_with_offset+0xe9/0x230 o2hb_thread+0x14e/0x770 kthread+0x1ad/0x1f0 ret_from_fork+0x3c9/0x540 __switch_to+0x2e9/0x730 ret_from_fork_asm+0x1a/0x30 Allocated by task stack: kasan_save_stack+0x33/0x60 kasan_save_track+0x14/0x30 __kasan_kmalloc+0xaa/0xb0 __kmalloc_noprof+0x292/0x760 __alloc_workqueue+0x736/0xc60 alloc_workqueue_noprof+0xb1/0x110 o2net_start_listening+0xe5/0x430 o2nm_node_local_store+0x184/0x310 configfs_write_iter+0x18a/0x210 vfs_write+0x469/0x810 ksys_write+0xd2/0x170 do_syscall_64+0x115/0x6a0 (arch/x86/entry/syscall_64.c:87) entry_SYSCALL_64_after_hwframe+0x77/0x7f Freed by task stack: kasan_save_stack+0x33/0x60 kasan_save_track+0x14/0x30 kasan_save_free_info+0x3b/0x60 __kasan_slab_free+0x5f/0x80 kfree+0x313/0x590 rcu_core+0x4f4/0x1320 handle_softirqs+0x156/0x660 queue_delayed_work_on o2net_set_nn_state o2net_disconnect_node o2net_hb_node_down_cb o2hb_run_event_list Keep heartbeat callbacks registered so quorum state still tracks node state, but stop them from driving o2net reconnect/disconnect work once local teardown starts. Mark the transport offline before destroying o2net_wq, wait for any in-flight heartbeat callback to finish, and delay bring-up replay until the new local node is published through o2nm_this_node(). The replay also has to stay serialized with heartbeat callback delivery. Otherwise a live-node snapshot can be copied, a real hb_down callback can install -ENOTCONN for a peer, and the stale replay can call o2net_hb_node_up() for that same peer and queue reconnect work even though heartbeat is already down. The buggy scenario involves two paths, with each column showing the order within that path: local-node teardown: heartbeat node-down callback: 1. configfs local-off enters 1. o2hb_run_event_list() invokes o2net_stop_listening(). o2net_hb_node_down_cb(). 2. teardown heads for 2. the callback reaches destroy_workqueue(o2net_wq). o2net_disconnect_node() and o2net_set_nn_state(). 3. teardown destroys and NULLs 3. the callback flushes or queues o2net_wq. work through o2net_wq. Link: https://lore.kernel.org/20260624095310.763763-1-zzzccc427@gmail.com Link: https://lore.kernel.org/20260624095310.763763-2-zzzccc427@gmail.com Fixes: 98211489d414 ("[PATCH] OCFS2: The Second Oracle Cluster Filesystem") Signed-off-by: Cen Zhang Assisted-by: Codex:gpt-5.5 Reviewed-by: Joseph Qi Cc: Changwei Ge Cc: Heming Zhao Cc: Joel Becker Cc: Jun Piao Cc: Junxiao Bi Cc: Mark Fasheh Signed-off-by: Andrew Morton --- fs/ocfs2/cluster/heartbeat.c | 43 ++++++++++++---- fs/ocfs2/cluster/heartbeat.h | 5 + fs/ocfs2/cluster/nodemanager.c | 4 + fs/ocfs2/cluster/tcp.c | 83 ++++++++++++++++++++++++------- fs/ocfs2/cluster/tcp.h | 1 5 files changed, 109 insertions(+), 27 deletions(-) --- a/fs/ocfs2/cluster/heartbeat.c~ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown +++ a/fs/ocfs2/cluster/heartbeat.c @@ -1492,13 +1492,38 @@ void o2hb_init(void) o2hb_debug_init(); } -/* if we're already in a callback then we're already serialized by the sem */ -static void o2hb_fill_node_map_from_callback(unsigned long *map, - unsigned int bits) +static void __o2hb_fill_node_map(unsigned long *map, unsigned int bits) { bitmap_copy(map, o2hb_live_node_bitmap, bits); } +void o2hb_callback_read_lock(void) +{ + down_read(&o2hb_callback_sem); +} + +void o2hb_callback_read_unlock(void) +{ + up_read(&o2hb_callback_sem); +} + +void o2hb_synchronize_callbacks(void) +{ + down_write(&o2hb_callback_sem); + up_write(&o2hb_callback_sem); +} + +/* + * Callers must already hold o2hb_callback_sem for read or write so the copy + * stays serialized with callback delivery. + */ +void o2hb_fill_node_map_locked(unsigned long *map, unsigned int bits) +{ + spin_lock(&o2hb_live_lock); + __o2hb_fill_node_map(map, bits); + spin_unlock(&o2hb_live_lock); +} + /* * get a map of all nodes that are heartbeating in any regions */ @@ -1506,11 +1531,9 @@ void o2hb_fill_node_map(unsigned long *m { /* callers want to serialize this map and callbacks so that they * can trust that they don't miss nodes coming to the party */ - down_read(&o2hb_callback_sem); - spin_lock(&o2hb_live_lock); - o2hb_fill_node_map_from_callback(map, bits); - spin_unlock(&o2hb_live_lock); - up_read(&o2hb_callback_sem); + o2hb_callback_read_lock(); + o2hb_fill_node_map_locked(map, bits); + o2hb_callback_read_unlock(); } EXPORT_SYMBOL_GPL(o2hb_fill_node_map); @@ -2609,7 +2632,7 @@ int o2hb_check_node_heartbeating_no_sem( unsigned long testing_map[BITS_TO_LONGS(O2NM_MAX_NODES)]; spin_lock(&o2hb_live_lock); - o2hb_fill_node_map_from_callback(testing_map, O2NM_MAX_NODES); + __o2hb_fill_node_map(testing_map, O2NM_MAX_NODES); spin_unlock(&o2hb_live_lock); if (!test_bit(node_num, testing_map)) { mlog(ML_HEARTBEAT, @@ -2626,7 +2649,7 @@ int o2hb_check_node_heartbeating_from_ca { unsigned long testing_map[BITS_TO_LONGS(O2NM_MAX_NODES)]; - o2hb_fill_node_map_from_callback(testing_map, O2NM_MAX_NODES); + o2hb_fill_node_map_locked(testing_map, O2NM_MAX_NODES); if (!test_bit(node_num, testing_map)) { mlog(ML_HEARTBEAT, "node (%u) does not have heartbeating enabled.\n", --- a/fs/ocfs2/cluster/heartbeat.h~ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown +++ a/fs/ocfs2/cluster/heartbeat.h @@ -58,6 +58,11 @@ int o2hb_register_callback(const char *r struct o2hb_callback_func *hc); void o2hb_unregister_callback(const char *region_uuid, struct o2hb_callback_func *hc); +void o2hb_callback_read_lock(void); +void o2hb_callback_read_unlock(void); +void o2hb_synchronize_callbacks(void); +void o2hb_fill_node_map_locked(unsigned long *map, + unsigned int bits); void o2hb_fill_node_map(unsigned long *map, unsigned int bits); void o2hb_exit(void); --- a/fs/ocfs2/cluster/nodemanager.c~ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown +++ a/fs/ocfs2/cluster/nodemanager.c @@ -326,6 +326,7 @@ static ssize_t o2nm_node_local_store(str struct o2nm_node *node = to_o2nm_node(item); struct o2nm_cluster *cluster; unsigned long tmp; + bool starting = false; char *p = (char *)page; ssize_t ret; @@ -362,6 +363,7 @@ static ssize_t o2nm_node_local_store(str ret = o2net_start_listening(node); if (ret) goto out; + starting = true; } if (!tmp && cluster->cl_has_local && @@ -375,6 +377,8 @@ static ssize_t o2nm_node_local_store(str if (node->nd_local) { cluster->cl_has_local = tmp; cluster->cl_local_node = node->nd_num; + if (starting) + o2net_complete_start_listening(node); } ret = count; --- a/fs/ocfs2/cluster/tcp.c~ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown +++ a/fs/ocfs2/cluster/tcp.c @@ -105,6 +105,8 @@ static struct socket *o2net_listen_sock; * destroying the work queue. */ static struct workqueue_struct *o2net_wq; +/* Heartbeat callbacks stay registered across local-node off/on. */ +static bool o2net_listening; static struct work_struct o2net_listen_work; static struct o2hb_callback_func o2net_hb_up, o2net_hb_down; @@ -1692,6 +1694,19 @@ static void o2net_still_up(struct work_s /* ------------------------------------------------------------ */ +static void o2net_hb_node_up(struct o2net_node *nn) +{ + /* ensure an immediate connect attempt */ + nn->nn_last_connect_attempt = jiffies - + (msecs_to_jiffies(o2net_reconnect_delay()) + 1); + + spin_lock(&nn->nn_lock); + atomic_set(&nn->nn_timeout, 0); + if (nn->nn_persistent_error) + o2net_set_nn_state(nn, NULL, 0, 0); + spin_unlock(&nn->nn_lock); +} + void o2net_disconnect_node(struct o2nm_node *node) { struct o2net_node *nn = o2net_nn_from_num(node->nd_num); @@ -1713,41 +1728,39 @@ void o2net_disconnect_node(struct o2nm_n static void o2net_hb_node_down_cb(struct o2nm_node *node, int node_num, void *data) { + u8 this_node; + o2quo_hb_down(node_num); if (!node) - return; + goto out; - if (node_num != o2nm_this_node()) + this_node = o2nm_this_node(); + if (!READ_ONCE(o2net_listening) || this_node == O2NM_MAX_NODES) + goto out; + + if (node_num != this_node) o2net_disconnect_node(node); +out: BUG_ON(atomic_read(&o2net_connected_peers) < 0); } static void o2net_hb_node_up_cb(struct o2nm_node *node, int node_num, void *data) { - struct o2net_node *nn = o2net_nn_from_num(node_num); + u8 this_node; o2quo_hb_up(node_num); BUG_ON(!node); - /* ensure an immediate connect attempt */ - nn->nn_last_connect_attempt = jiffies - - (msecs_to_jiffies(o2net_reconnect_delay()) + 1); + this_node = o2nm_this_node(); + if (!READ_ONCE(o2net_listening) || this_node == O2NM_MAX_NODES) + return; - if (node_num != o2nm_this_node()) { - /* believe it or not, accept and node heartbeating testing - * can succeed for this node before we got here.. so - * only use set_nn_state to clear the persistent error - * if that hasn't already happened */ - spin_lock(&nn->nn_lock); - atomic_set(&nn->nn_timeout, 0); - if (nn->nn_persistent_error) - o2net_set_nn_state(nn, NULL, 0, 0); - spin_unlock(&nn->nn_lock); - } + if (node_num != this_node) + o2net_hb_node_up(o2net_nn_from_num(node_num)); } void o2net_unregister_hb_callbacks(void) @@ -1756,6 +1769,37 @@ void o2net_unregister_hb_callbacks(void) o2hb_unregister_callback(NULL, &o2net_hb_down); } +/* + * Delay heartbeat-driven network work until the local node is fully published + * through o2nm_this_node(), then replay the nodes that are already live while + * callback delivery stays blocked. + */ +void o2net_complete_start_listening(struct o2nm_node *node) +{ + unsigned long live_nodes[BITS_TO_LONGS(O2NM_MAX_NODES)]; + unsigned long node_num; + u8 local_node; + + local_node = o2nm_this_node(); + if (WARN_ON_ONCE(local_node == O2NM_MAX_NODES)) + return; + if (WARN_ON_ONCE(local_node != node->nd_num)) + return; + if (WARN_ON_ONCE(!o2net_wq)) + return; + + o2hb_callback_read_lock(); + WRITE_ONCE(o2net_listening, true); + o2hb_fill_node_map_locked(live_nodes, O2NM_MAX_NODES); + for_each_set_bit(node_num, live_nodes, O2NM_MAX_NODES) { + if (node_num == local_node) + continue; + + o2net_hb_node_up(o2net_nn_from_num(node_num)); + } + o2hb_callback_read_unlock(); +} + int o2net_register_hb_callbacks(void) { int ret; @@ -2034,6 +2078,8 @@ int o2net_start_listening(struct o2nm_no { int ret = 0; + if (WARN_ON_ONCE(READ_ONCE(o2net_listening))) + return -EBUSY; BUG_ON(o2net_wq != NULL); BUG_ON(o2net_listen_sock != NULL); @@ -2065,6 +2111,9 @@ void o2net_stop_listening(struct o2nm_no BUG_ON(o2net_wq == NULL); BUG_ON(o2net_listen_sock == NULL); + WRITE_ONCE(o2net_listening, false); + o2hb_synchronize_callbacks(); + /* stop the listening socket from generating work */ write_lock_bh(&sock->sk->sk_callback_lock); sock->sk->sk_data_ready = sock->sk->sk_user_data; --- a/fs/ocfs2/cluster/tcp.h~ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown +++ a/fs/ocfs2/cluster/tcp.h @@ -96,6 +96,7 @@ struct o2nm_node; int o2net_register_hb_callbacks(void); void o2net_unregister_hb_callbacks(void); int o2net_start_listening(struct o2nm_node *node); +void o2net_complete_start_listening(struct o2nm_node *node); void o2net_stop_listening(struct o2nm_node *node); void o2net_disconnect_node(struct o2nm_node *node); int o2net_num_connected_peers(void); _ Patches currently in -mm which might be from zzzccc427@gmail.com are ocfs2-cluster-keep-heartbeat-local-node-stable.patch ocfs2-validate-inline-xattrs-during-inode-block-validation.patch ocfs2-validate-external-xattr-entries-when-reading-metadata.patch ocfs2-synchronize-heartbeat-callbacks-with-o2net-teardown.patch ocfs2-o2hb-quiesce-negotiate-handlers-and-timeout-work.patch