From: Stanislav Fomichev <stfomichev@gmail.com>
To: Ilya Maximets <i.maximets@ovn.org>
Cc: Eric Dumazet <edumazet@google.com>,
"David S . Miller" <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
netdev@vger.kernel.org, eric.dumazet@gmail.com,
Dan Streetman <dan.streetman@canonical.com>,
Steffen Klassert <steffen.klassert@secunet.com>
Subject: Re: [PATCH net] net: defer final 'struct net' free in netns dismantle
Date: Tue, 3 Dec 2024 13:04:38 -0800 [thread overview]
Message-ID: <Z09yZqv82dQn-1zI@mini-arch> (raw)
In-Reply-To: <53482ace-71a3-4fa3-a7d3-592311fc3c1b@ovn.org>
On 12/03, Ilya Maximets wrote:
> On 12/3/24 17:50, Eric Dumazet wrote:
> > Ilya reported a slab-use-after-free in dst_destroy [1]
> >
> > Issue is in xfrm6_net_init() and xfrm4_net_init() :
> >
> > They copy xfrm[46]_dst_ops_template into net->xfrm.xfrm[46]_dst_ops.
> >
> > But net structure might be freed before all the dst callbacks are
> > called. So when dst_destroy() calls later :
> >
> > if (dst->ops->destroy)
> > dst->ops->destroy(dst);
> >
> > dst->ops points to the old net->xfrm.xfrm[46]_dst_ops, which has been freed.
> >
> > See a relevant issue fixed in :
> >
> > ac888d58869b ("net: do not delay dst_entries_add() in dst_release()")
> >
> > A fix is to queue the 'struct net' to be freed after one
> > another cleanup_net() round (and existing rcu_barrier())
> >
> > [1]
>
> <snip>
>
> Hi, Eric. Thanks for the patch!
>
> Though I tried to test it by applying directly on top of v6.12 tag, but I got
> the following UAF shortly after booting the kernel. Seems like podman service
> was initializing something and creating namespaces for that.
>
> I can try applying the change on top of net tree, if that helps.
>
> Best regards, Ilya Maximets.
>
> The log:
>
> Dec 3 13:12:09 systemd-logind[1240]: New session 3 of user root.
> Dec 3 13:12:09 systemd[1]: Started Session 3 of User root.
> Dec 3 13:12:39 systemd[1]: systemd-hostnamed.service: Deactivated successfully.
> Dec 3 13:12:40 kernel: ==================================================================
> Dec 3 13:12:40 kernel: BUG: KASAN: slab-use-after-free in cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:655)
> Dec 3 13:12:40 kernel: Read of size 8 at addr ffff888166941bf8 by task kworker/u160:1/13
> Dec 3 13:12:40 kernel:
> Dec 3 13:12:40 kernel: CPU: 34 UID: 0 PID: 13 Comm: kworker/u160:1 Not tainted 6.12.0+ #69
> Dec 3 13:12:40 kernel: Hardware name: Red Hat KVM/RHEL, BIOS 1.16.1-1.el9 04/01/2014
> Dec 3 13:12:40 kernel: Workqueue: netns cleanup_net
> Dec 3 13:12:40 kernel: Call Trace:
> Dec 3 13:12:40 kernel: <TASK>
> Dec 3 13:12:40 kernel: dump_stack_lvl (lib/dump_stack.c:124)
> Dec 3 13:12:40 kernel: print_address_description.constprop.0 (mm/kasan/report.c:378)
> Dec 3 13:12:40 kernel: ? cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:655)
> Dec 3 13:12:40 kernel: print_report (mm/kasan/report.c:489)
> Dec 3 13:12:40 kernel: ? cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:655)
> Dec 3 13:12:40 kernel: ? kasan_addr_to_slab (mm/kasan/common.c:37)
> Dec 3 13:12:40 kernel: kasan_report (mm/kasan/report.c:603)
> Dec 3 13:12:40 kernel: ? cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:655)
> Dec 3 13:12:40 kernel: cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:655)
> Dec 3 13:12:40 kernel: ? __pfx_lock_acquire (kernel/locking/lockdep.c:5793)
> Dec 3 13:12:40 kernel: ? __pfx_cleanup_net (net/core/net_namespace.c:586)
> Dec 3 13:12:40 kernel: ? lock_is_held_type (kernel/locking/lockdep.c:5566 kernel/locking/lockdep.c:5897)
> Dec 3 13:12:40 kernel: process_one_work (kernel/workqueue.c:3229)
> Dec 3 13:12:40 kernel: ? __pfx_lock_acquire (kernel/locking/lockdep.c:5793)
> Dec 3 13:12:40 kernel: ? __pfx_process_one_work (kernel/workqueue.c:3131)
> Dec 3 13:12:40 kernel: ? assign_work (kernel/workqueue.c:1200)
> Dec 3 13:12:40 kernel: ? lock_is_held_type (kernel/locking/lockdep.c:5566 kernel/locking/lockdep.c:5897)
> Dec 3 13:12:40 kernel: worker_thread (kernel/workqueue.c:3304 kernel/workqueue.c:3391)
> Dec 3 13:12:40 kernel: ? __kthread_parkme (./arch/x86/include/asm/bitops.h:206 ./arch/x86/include/asm/bitops.h:238 ./include/asm-generic/bitops/instrumented-non-atomic.h:142 kernel/kthread.c:280)
> Dec 3 13:12:40 kernel: ? __pfx_worker_thread (kernel/workqueue.c:3337)
> Dec 3 13:12:40 kernel: kthread (kernel/kthread.c:389)
> Dec 3 13:12:40 kernel: ? __pfx_kthread (kernel/kthread.c:342)
> Dec 3 13:12:40 kernel: ret_from_fork (arch/x86/kernel/process.c:147)
> Dec 3 13:12:40 kernel: ? __pfx_kthread (kernel/kthread.c:342)
> Dec 3 13:12:40 kernel: ret_from_fork_asm (arch/x86/entry/entry_64.S:257)
> Dec 3 13:12:40 kernel: </TASK>
> Dec 3 13:12:40 kernel:
> Dec 3 13:12:40 kernel: Allocated by task 1250:
> Dec 3 13:12:40 kernel: kasan_save_stack (mm/kasan/common.c:48)
> Dec 3 13:12:40 kernel: kasan_save_track (./arch/x86/include/asm/current.h:49 mm/kasan/common.c:60 mm/kasan/common.c:69)
> Dec 3 13:12:40 kernel: __kasan_slab_alloc (mm/kasan/common.c:319 mm/kasan/common.c:345)
> Dec 3 13:12:40 kernel: kmem_cache_alloc_noprof (mm/slub.c:4085 mm/slub.c:4134 mm/slub.c:4141)
> Dec 3 13:12:40 kernel: copy_net_ns (net/core/net_namespace.c:421 net/core/net_namespace.c:496)
> Dec 3 13:12:40 kernel: create_new_namespaces (kernel/nsproxy.c:110)
> Dec 3 13:12:40 kernel: unshare_nsproxy_namespaces (kernel/nsproxy.c:228 (discriminator 4))
> Dec 3 13:12:40 kernel: ksys_unshare (kernel/fork.c:3313)
> Dec 3 13:12:40 kernel: __x64_sys_unshare (kernel/fork.c:3382)
> Dec 3 13:12:40 kernel: do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
> Dec 3 13:12:40 kernel: entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130)
> Dec 3 13:12:40 kernel:
> Dec 3 13:12:40 kernel: Freed by task 13:
> Dec 3 13:12:40 kernel: kasan_save_stack (mm/kasan/common.c:48)
> Dec 3 13:12:40 kernel: kasan_save_track (./arch/x86/include/asm/current.h:49 mm/kasan/common.c:60 mm/kasan/common.c:69)
> Dec 3 13:12:40 kernel: kasan_save_free_info (mm/kasan/generic.c:582)
> Dec 3 13:12:40 kernel: __kasan_slab_free (mm/kasan/common.c:271)
> Dec 3 13:12:40 kernel: kmem_cache_free (mm/slub.c:4579 mm/slub.c:4681)
> Dec 3 13:12:40 kernel: cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:655)
> Dec 3 13:12:40 kernel: process_one_work (kernel/workqueue.c:3229)
> Dec 3 13:12:40 kernel: worker_thread (kernel/workqueue.c:3304 kernel/workqueue.c:3391)
> Dec 3 13:12:40 kernel: kthread (kernel/kthread.c:389)
> Dec 3 13:12:40 kernel: ret_from_fork (arch/x86/kernel/process.c:147)
> Dec 3 13:12:40 kernel: ret_from_fork_asm (arch/x86/entry/entry_64.S:257)
> Dec 3 13:12:40 kernel:
> Dec 3 13:12:40 kernel: The buggy address belongs to the object at ffff888166941b40#012 which belongs to the cache net_namespace of size 6720
> Dec 3 13:12:40 kernel: The buggy address is located 184 bytes inside of#012 freed 6720-byte region [ffff888166941b40, ffff888166943580)
> Dec 3 13:12:40 kernel:
> Dec 3 13:12:40 kernel: The buggy address belongs to the physical page:
> Dec 3 13:12:40 kernel: page: refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x166940
> Dec 3 13:12:40 kernel: head: order:3 mapcount:0 entire_mapcount:0 nr_pages_mapped:0 pincount:0
> Dec 3 13:12:40 kernel: memcg:ffff8881229685c1
> Dec 3 13:12:40 kernel: flags: 0x17ffffc0000040(head|node=0|zone=2|lastcpupid=0x1fffff)
> Dec 3 13:12:40 kernel: page_type: f5(slab)
> Dec 3 13:12:40 kernel: raw: 0017ffffc0000040 ffff888100053980 dead000000000122 0000000000000000
> Dec 3 13:12:40 kernel: raw: 0000000000000000 0000000080040004 00000001f5000000 ffff8881229685c1
> Dec 3 13:12:40 kernel: head: 0017ffffc0000040 ffff888100053980 dead000000000122 0000000000000000
> Dec 3 13:12:40 kernel: head: 0000000000000000 0000000080040004 00000001f5000000 ffff8881229685c1
> Dec 3 13:12:40 kernel: head: 0017ffffc0000003 ffffea00059a5001 ffffffffffffffff 0000000000000000
> Dec 3 13:12:40 kernel: head: 0000000000000008 0000000000000000 00000000ffffffff 0000000000000000
> Dec 3 13:12:40 kernel: page dumped because: kasan: bad access detected
> Dec 3 13:12:40 kernel:
> Dec 3 13:12:40 kernel: Memory state around the buggy address:
> Dec 3 13:12:40 kernel: ffff888166941a80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> Dec 3 13:12:40 kernel: ffff888166941b00: fc fc fc fc fc fc fc fc fa fb fb fb fb fb fb fb
> Dec 3 13:12:40 kernel: >ffff888166941b80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> Dec 3 13:12:40 kernel: ^
> Dec 3 13:12:40 kernel: ffff888166941c00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> Dec 3 13:12:40 kernel: ffff888166941c80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> Dec 3 13:12:40 kernel: ==================================================================
> Dec 3 13:12:40 kernel: Disabling lock debugging due to kernel taint
> Dec 3 13:14:14 systemd[1]: var-lib-containers-storage-overlay-compat1591001862-merged.mount: Deactivated successfully.
> Dec 3 13:14:14 kernel: evm: overlay not supported
> Dec 3 13:14:14 systemd[1]: var-lib-containers-storage-overlay-metacopyx2dcheck2012509683-merged.mount: Deactivated successfully.
> Dec 3 13:14:14 podman[5241]: 2024-12-03 13:14:14.444912997 -0500 EST m=+0.123882461 system refresh
> Dec 3 13:14:15 systemd[1]: var-lib-containers-storage-overlay.mount: Deactivated successfully.
Let's also kick it out from the NIPA queue:
---
pw-bot: cr
prev parent reply other threads:[~2024-12-03 21:04 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-12-03 16:50 [PATCH net] net: defer final 'struct net' free in netns dismantle Eric Dumazet
2024-12-03 18:23 ` Ilya Maximets
2024-12-03 18:33 ` Eric Dumazet
2024-12-03 22:01 ` Ilya Maximets
2024-12-03 21:04 ` Stanislav Fomichev [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Z09yZqv82dQn-1zI@mini-arch \
--to=stfomichev@gmail.com \
--cc=dan.streetman@canonical.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=eric.dumazet@gmail.com \
--cc=i.maximets@ovn.org \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=steffen.klassert@secunet.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).