From: Stanislav Fomichev <stfomichev@gmail.com>
To: Ilya Maximets <i.maximets@ovn.org>
Cc: Eric Dumazet <edumazet@google.com>,
"David S . Miller" <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
netdev@vger.kernel.org, eric.dumazet@gmail.com,
Dan Streetman <dan.streetman@canonical.com>,
Steffen Klassert <steffen.klassert@secunet.com>
Subject: Re: [PATCH net] net: defer final 'struct net' free in netns dismantle
Date: Tue, 3 Dec 2024 13:04:38 -0800 [thread overview]
Message-ID: <Z09yZqv82dQn-1zI@mini-arch> (raw)
In-Reply-To: <53482ace-71a3-4fa3-a7d3-592311fc3c1b@ovn.org>
On 12/03, Ilya Maximets wrote:
> On 12/3/24 17:50, Eric Dumazet wrote:
> > Ilya reported a slab-use-after-free in dst_destroy [1]
> >
> > Issue is in xfrm6_net_init() and xfrm4_net_init() :
> >
> > They copy xfrm[46]_dst_ops_template into net->xfrm.xfrm[46]_dst_ops.
> >
> > But net structure might be freed before all the dst callbacks are
> > called. So when dst_destroy() calls later :
> >
> > if (dst->ops->destroy)
> > dst->ops->destroy(dst);
> >
> > dst->ops points to the old net->xfrm.xfrm[46]_dst_ops, which has been freed.
> >
> > See a relevant issue fixed in :
> >
> > ac888d58869b ("net: do not delay dst_entries_add() in dst_release()")
> >
> > A fix is to queue the 'struct net' to be freed after one
> > another cleanup_net() round (and existing rcu_barrier())
> >
> > [1]
>
> <snip>
>
> Hi, Eric. Thanks for the patch!
>
> Though I tried to test it by applying directly on top of v6.12 tag, but I got
> the following UAF shortly after booting the kernel. Seems like podman service
> was initializing something and creating namespaces for that.
>
> I can try applying the change on top of net tree, if that helps.
>
> Best regards, Ilya Maximets.
>
> The log:
>
> Dec 3 13:12:09 systemd-logind[1240]: New session 3 of user root.
> Dec 3 13:12:09 systemd[1]: Started Session 3 of User root.
> Dec 3 13:12:39 systemd[1]: systemd-hostnamed.service: Deactivated successfully.
> Dec 3 13:12:40 kernel: ==================================================================
> Dec 3 13:12:40 kernel: BUG: KASAN: slab-use-after-free in cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:655)
> Dec 3 13:12:40 kernel: Read of size 8 at addr ffff888166941bf8 by task kworker/u160:1/13
> Dec 3 13:12:40 kernel:
> Dec 3 13:12:40 kernel: CPU: 34 UID: 0 PID: 13 Comm: kworker/u160:1 Not tainted 6.12.0+ #69
> Dec 3 13:12:40 kernel: Hardware name: Red Hat KVM/RHEL, BIOS 1.16.1-1.el9 04/01/2014
> Dec 3 13:12:40 kernel: Workqueue: netns cleanup_net
> Dec 3 13:12:40 kernel: Call Trace:
> Dec 3 13:12:40 kernel: <TASK>
> Dec 3 13:12:40 kernel: dump_stack_lvl (lib/dump_stack.c:124)
> Dec 3 13:12:40 kernel: print_address_description.constprop.0 (mm/kasan/report.c:378)
> Dec 3 13:12:40 kernel: ? cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:655)
> Dec 3 13:12:40 kernel: print_report (mm/kasan/report.c:489)
> Dec 3 13:12:40 kernel: ? cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:655)
> Dec 3 13:12:40 kernel: ? kasan_addr_to_slab (mm/kasan/common.c:37)
> Dec 3 13:12:40 kernel: kasan_report (mm/kasan/report.c:603)
> Dec 3 13:12:40 kernel: ? cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:655)
> Dec 3 13:12:40 kernel: cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:655)
> Dec 3 13:12:40 kernel: ? __pfx_lock_acquire (kernel/locking/lockdep.c:5793)
> Dec 3 13:12:40 kernel: ? __pfx_cleanup_net (net/core/net_namespace.c:586)
> Dec 3 13:12:40 kernel: ? lock_is_held_type (kernel/locking/lockdep.c:5566 kernel/locking/lockdep.c:5897)
> Dec 3 13:12:40 kernel: process_one_work (kernel/workqueue.c:3229)
> Dec 3 13:12:40 kernel: ? __pfx_lock_acquire (kernel/locking/lockdep.c:5793)
> Dec 3 13:12:40 kernel: ? __pfx_process_one_work (kernel/workqueue.c:3131)
> Dec 3 13:12:40 kernel: ? assign_work (kernel/workqueue.c:1200)
> Dec 3 13:12:40 kernel: ? lock_is_held_type (kernel/locking/lockdep.c:5566 kernel/locking/lockdep.c:5897)
> Dec 3 13:12:40 kernel: worker_thread (kernel/workqueue.c:3304 kernel/workqueue.c:3391)
> Dec 3 13:12:40 kernel: ? __kthread_parkme (./arch/x86/include/asm/bitops.h:206 ./arch/x86/include/asm/bitops.h:238 ./include/asm-generic/bitops/instrumented-non-atomic.h:142 kernel/kthread.c:280)
> Dec 3 13:12:40 kernel: ? __pfx_worker_thread (kernel/workqueue.c:3337)
> Dec 3 13:12:40 kernel: kthread (kernel/kthread.c:389)
> Dec 3 13:12:40 kernel: ? __pfx_kthread (kernel/kthread.c:342)
> Dec 3 13:12:40 kernel: ret_from_fork (arch/x86/kernel/process.c:147)
> Dec 3 13:12:40 kernel: ? __pfx_kthread (kernel/kthread.c:342)
> Dec 3 13:12:40 kernel: ret_from_fork_asm (arch/x86/entry/entry_64.S:257)
> Dec 3 13:12:40 kernel: </TASK>
> Dec 3 13:12:40 kernel:
> Dec 3 13:12:40 kernel: Allocated by task 1250:
> Dec 3 13:12:40 kernel: kasan_save_stack (mm/kasan/common.c:48)
> Dec 3 13:12:40 kernel: kasan_save_track (./arch/x86/include/asm/current.h:49 mm/kasan/common.c:60 mm/kasan/common.c:69)
> Dec 3 13:12:40 kernel: __kasan_slab_alloc (mm/kasan/common.c:319 mm/kasan/common.c:345)
> Dec 3 13:12:40 kernel: kmem_cache_alloc_noprof (mm/slub.c:4085 mm/slub.c:4134 mm/slub.c:4141)
> Dec 3 13:12:40 kernel: copy_net_ns (net/core/net_namespace.c:421 net/core/net_namespace.c:496)
> Dec 3 13:12:40 kernel: create_new_namespaces (kernel/nsproxy.c:110)
> Dec 3 13:12:40 kernel: unshare_nsproxy_namespaces (kernel/nsproxy.c:228 (discriminator 4))
> Dec 3 13:12:40 kernel: ksys_unshare (kernel/fork.c:3313)
> Dec 3 13:12:40 kernel: __x64_sys_unshare (kernel/fork.c:3382)
> Dec 3 13:12:40 kernel: do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
> Dec 3 13:12:40 kernel: entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130)
> Dec 3 13:12:40 kernel:
> Dec 3 13:12:40 kernel: Freed by task 13:
> Dec 3 13:12:40 kernel: kasan_save_stack (mm/kasan/common.c:48)
> Dec 3 13:12:40 kernel: kasan_save_track (./arch/x86/include/asm/current.h:49 mm/kasan/common.c:60 mm/kasan/common.c:69)
> Dec 3 13:12:40 kernel: kasan_save_free_info (mm/kasan/generic.c:582)
> Dec 3 13:12:40 kernel: __kasan_slab_free (mm/kasan/common.c:271)
> Dec 3 13:12:40 kernel: kmem_cache_free (mm/slub.c:4579 mm/slub.c:4681)
> Dec 3 13:12:40 kernel: cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:655)
> Dec 3 13:12:40 kernel: process_one_work (kernel/workqueue.c:3229)
> Dec 3 13:12:40 kernel: worker_thread (kernel/workqueue.c:3304 kernel/workqueue.c:3391)
> Dec 3 13:12:40 kernel: kthread (kernel/kthread.c:389)
> Dec 3 13:12:40 kernel: ret_from_fork (arch/x86/kernel/process.c:147)
> Dec 3 13:12:40 kernel: ret_from_fork_asm (arch/x86/entry/entry_64.S:257)
> Dec 3 13:12:40 kernel:
> Dec 3 13:12:40 kernel: The buggy address belongs to the object at ffff888166941b40#012 which belongs to the cache net_namespace of size 6720
> Dec 3 13:12:40 kernel: The buggy address is located 184 bytes inside of#012 freed 6720-byte region [ffff888166941b40, ffff888166943580)
> Dec 3 13:12:40 kernel:
> Dec 3 13:12:40 kernel: The buggy address belongs to the physical page:
> Dec 3 13:12:40 kernel: page: refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x166940
> Dec 3 13:12:40 kernel: head: order:3 mapcount:0 entire_mapcount:0 nr_pages_mapped:0 pincount:0
> Dec 3 13:12:40 kernel: memcg:ffff8881229685c1
> Dec 3 13:12:40 kernel: flags: 0x17ffffc0000040(head|node=0|zone=2|lastcpupid=0x1fffff)
> Dec 3 13:12:40 kernel: page_type: f5(slab)
> Dec 3 13:12:40 kernel: raw: 0017ffffc0000040 ffff888100053980 dead000000000122 0000000000000000
> Dec 3 13:12:40 kernel: raw: 0000000000000000 0000000080040004 00000001f5000000 ffff8881229685c1
> Dec 3 13:12:40 kernel: head: 0017ffffc0000040 ffff888100053980 dead000000000122 0000000000000000
> Dec 3 13:12:40 kernel: head: 0000000000000000 0000000080040004 00000001f5000000 ffff8881229685c1
> Dec 3 13:12:40 kernel: head: 0017ffffc0000003 ffffea00059a5001 ffffffffffffffff 0000000000000000
> Dec 3 13:12:40 kernel: head: 0000000000000008 0000000000000000 00000000ffffffff 0000000000000000
> Dec 3 13:12:40 kernel: page dumped because: kasan: bad access detected
> Dec 3 13:12:40 kernel:
> Dec 3 13:12:40 kernel: Memory state around the buggy address:
> Dec 3 13:12:40 kernel: ffff888166941a80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> Dec 3 13:12:40 kernel: ffff888166941b00: fc fc fc fc fc fc fc fc fa fb fb fb fb fb fb fb
> Dec 3 13:12:40 kernel: >ffff888166941b80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> Dec 3 13:12:40 kernel: ^
> Dec 3 13:12:40 kernel: ffff888166941c00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> Dec 3 13:12:40 kernel: ffff888166941c80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> Dec 3 13:12:40 kernel: ==================================================================
> Dec 3 13:12:40 kernel: Disabling lock debugging due to kernel taint
> Dec 3 13:14:14 systemd[1]: var-lib-containers-storage-overlay-compat1591001862-merged.mount: Deactivated successfully.
> Dec 3 13:14:14 kernel: evm: overlay not supported
> Dec 3 13:14:14 systemd[1]: var-lib-containers-storage-overlay-metacopyx2dcheck2012509683-merged.mount: Deactivated successfully.
> Dec 3 13:14:14 podman[5241]: 2024-12-03 13:14:14.444912997 -0500 EST m=+0.123882461 system refresh
> Dec 3 13:14:15 systemd[1]: var-lib-containers-storage-overlay.mount: Deactivated successfully.
Let's also kick it out from the NIPA queue:
---
pw-bot: cr
prev parent reply other threads:[~2024-12-03 21:04 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-12-03 16:50 [PATCH net] net: defer final 'struct net' free in netns dismantle Eric Dumazet
2024-12-03 18:23 ` Ilya Maximets
2024-12-03 18:33 ` Eric Dumazet
2024-12-03 22:01 ` Ilya Maximets
2024-12-03 21:04 ` Stanislav Fomichev [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Z09yZqv82dQn-1zI@mini-arch \
--to=stfomichev@gmail.com \
--cc=dan.streetman@canonical.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=eric.dumazet@gmail.com \
--cc=i.maximets@ovn.org \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=steffen.klassert@secunet.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.