Netdev List
 help / color / mirror / Atom feed
From: Shuangpeng <shuangpeng.kernel@gmail.com>
To: saeed bishara <saeed.bishara.os@gmail.com>
Cc: netdev@vger.kernel.org, Jakub Kicinski <kuba@kernel.org>,
	Andrew Lunn <andrew+netdev@lunn.ch>,
	"David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@google.com>,
	Paolo Abeni <pabeni@redhat.com>, Simon Horman <horms@kernel.org>,
	linux-kernel@vger.kernel.org
Subject: Re: [BUG] netdevsim: KASAN slab-use-after-free in ref_tracker_free
Date: Fri, 26 Jun 2026 13:41:13 -0400	[thread overview]
Message-ID: <2037CBE9-D72E-4E81-9B18-62AF3735306D@gmail.com> (raw)
In-Reply-To: <CAHfVqdXqZkrp5Dm0a8BJU0Tbc_T09A=e02PenwxrZFDMQrEmyQ@mail.gmail.com>



> On Jun 16, 2026, at 05:34, saeed bishara <saeed.bishara.os@gmail.com> wrote:
> 
> I tried gemini, here its analysis and a fix suggestion:
> 
> This is a brilliant, subtle bug. Let's break this down with a rigorous
> audit of the Linux network device refcounting architecture to see
> exactly why this Use-After-Free is occurring.
> The root cause is a classic "Reverse-Teardown Fallacy" colliding with
> the kernel's dual-refcount lifecycle for network devices (pcpu_refcnt
> vs. kobject refcount).
> 
> The Dual-Lifecycle Trap
> In modern kernel networking, a struct net_device is kept alive by two
> distinct mechanisms that must be carefully orchestrated:
> dev->pcpu_refcnt (Operational Lifecycle): Managed by dev_hold() and
> dev_put(). This tracks active operational references. The
> unregister_netdevice sequence blocks in netdev_run_todo() waiting for
> this to hit zero before it invokes free_netdev().
> dev->dev.kobj (Memory Lifecycle): The underlying memory structure
> manager. free_netdev() drops a reference to this. When the kobject
> refcount hits 0, device_release() is synchronously triggered, and the
> kmalloc-cg-8k slab is actually freed.
> 
> The Exact Race Sequence
> Look at the teardown order in the error path of sysfs_rtnl_lock():
> 
> unbreak:
>    sysfs_unbreak_active_protection(kn);  // LINE 121
>    dev_put(ndev);                        // LINE 122
> 
> Normally, developers write error paths in the exact reverse order of
> initialization. However, in this specific case, adhering to that
> convention is fatal.
> The Setup: sysfs_rtnl_lock starts by calling dev_hold(ndev),
> incrementing the pcpu_refcnt. It then calls
> sysfs_break_active_protection(), which explicitly calls kobject_get(),
> elevating the kobj memory refcount.
> The Unregister Race: In parallel, another CPU initiates device
> teardown (e.g., echo 1 > del_device on netdevsim). The sysfs entries
> are deleted, and device_del() drops its primary kobj reference. The
> teardown thread hits netdev_run_todo() and blocks, waiting for
> sysfs_rtnl_lock to release its dev_hold.
> The Lock Fails: sysfs_rtnl_lock fails to get the rtnl lock and jumps to unbreak.
> The Fatal Drop (Line 121): sysfs_unbreak_active_protection(kn)
> executes. It calls kobject_put(). Because the unregistration path
> already dropped the standard kobject references, this is the absolute
> last kobject reference. device_release() is synchronously invoked, and
> the net_device memory is instantly freed to the slab.
> The KASAN Trigger (Line 122): dev_put(ndev) executes. Under the hood,
> it attempts to call netdev_tracker_free(&ndev->ref_tracker, ...). It
> reads from the ndev structure that was just destroyed microseconds
> earlier. KASAN screams.
> 
> To fix this, the teardown convention must be deliberately violated to
> respect the underlying memory dependencies.
> 

Thanks for the analysis and patch suggestion.

I tested the patch together with the local delay/debug patch that was
used to trigger the original report.  I rebuilt the kernel, booted a
fresh VM, and reran the same netdevsim reproducer.

With the patched kernel, the original KASAN UAF was not reproduced in
this run:

  BUG: KASAN: slab-use-after-free in ref_tracker_free

However, the same rx-0 create_dir failure path was still reached, and
the run still produced a ref tracker warning:

  kobject_add_internal create_dir failed for rx-0 ...
  WARNING: lib/ref_tracker.c:295 at ref_tracker_free
  sysfs_rtnl_lock
  phys_port_name_show


> diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c
> index a1b2c3d4e5f6..7f8e9d0c1b2a 100644
> --- a/net/core/net-sysfs.c
> +++ b/net/core/net-sysfs.c
> @@ -118,8 +118,8 @@ static int sysfs_rtnl_lock(struct kobject *kobj,
> struct attribute *attr,
>  return 0;
> 
> unbreak:
> - sysfs_unbreak_active_protection(kn);
>  dev_put(ndev);
> + sysfs_unbreak_active_protection(kn);
>  return ret;
> }
> 
> 
> On Mon, Jun 15, 2026 at 4:18 AM Shuangpeng Bai
> <shuangpeng.kernel@gmail.com> wrote:
>> 
>> Hi netdev maintainers,
>> 
>> I hit the following KASAN report while testing an upstream kernel.
>> 
>> The issue was reproduced with netdevsim. I have not confirmed whether this is
>> specific to netdevsim or whether other net devices can trigger a similar issue.
>> 
>> The KASAN report shows a slab-use-after-free in ref_tracker_free(), reached from
>> sysfs_rtnl_lock() while reading phys_port_name.
>> 
>> I reproduced this on commit: e8c2f9fdadee7cbc75134dc463c1e0d856d6e5c7 (May 25 2026)
>> 
>> To help trigger the bug more reliably, we applied a minimal diagnostic patch
>> that only adds delays and print statements.
>> 
>> The reproducer and .config files are here.
>> https://gist.github.com/shuangpengbai/b49765d646ec4610917015371aa1c3ca
>> 
>> I'm happy to test debug patches or provide additional information.
>> 
>> Reported-by: Shuangpeng Bai <shuangpeng.kernel@gmail.com>
>> 
>> [ 3145.449971][T17497] BUG: KASAN: slab-use-after-free in ref_tracker_free (lib/ref_tracker.c:295)
>> [ 3145.452089][T17497] Read of size 1 at addr ffff888107678598 by task cat/17497
>> [ 3145.454439][T17497]
>> [ 3145.454977][T17497] Tainted: [W]=WARN
>> [ 3145.454980][T17497] Hardware name: QEMU Ubuntu 24.04 PC v2 (i440FX + PIIX, arch_caps fix, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
>> [ 3145.454985][T17497] Call Trace:
>> [ 3145.454991][T17497]  <TASK>
>> [ 3145.454994][T17497]  dump_stack_lvl (lib/dump_stack.c:94 lib/dump_stack.c:120)
>> [ 3145.455002][T17497]  print_report (mm/kasan/report.c:378 mm/kasan/report.c:482)
>> [ 3145.455028][T17497]  kasan_report (mm/kasan/report.c:595)
>> [ 3145.455046][T17497]  ref_tracker_free (lib/ref_tracker.c:295)
>> [ 3145.455083][T17497]  sysfs_rtnl_lock (include/linux/netdevice.h:4491 include/linux/netdevice.h:4508 include/linux/netdevice.h:4534 net/core/net-sysfs.c:122)
>> [ 3145.455091][T17497]  phys_port_name_show (net/core/net-sysfs.c:665)
>> [ 3145.455118][T17497]  dev_attr_show (drivers/base/core.c:2421)
>> [ 3145.455128][T17497]  sysfs_kf_seq_show (fs/sysfs/file.c:65)
>> [ 3145.455135][T17497]  seq_read_iter (fs/seq_file.c:231)
>> [ 3145.455144][T17497]  vfs_read (fs/read_write.c:493 fs/read_write.c:574)
>> [ 3145.455169][T17497]  ksys_read (fs/read_write.c:717)
>> [ 3145.455181][T17497]  do_syscall_64 (arch/x86/entry/syscall_64.c:63 arch/x86/entry/syscall_64.c:94)
>> [ 3145.455188][T17497]  entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
>> [ 3145.455193][T17497] RIP: 0033:0x7fcf098c43ce
>> [ 3145.455200][T17497] Code: c0 e9 b6 fe ff ff 50 48 8d 3d 6e 08 0b 00 e8 69 01 02 00 66 0f 1f 84 00 00 00 00 00 64 8b 04 25 18 00 00 00 85 c0 75 14 0f 05 <48> 3d 00 f0 ff ff 77 5a c3 66 0f 1f 84 00 00 00 00 00 48 83 ec 28
>> [ 3145.455204][T17497] RSP: 002b:00007ffd05e76b98 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
>> [ 3145.455211][T17497] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007fcf098c43ce
>> [ 3145.455214][T17497] RDX: 0000000000020000 RSI: 00007fcf095e4000 RDI: 0000000000000003
>> [ 3145.455217][T17497] RBP: 00007fcf095e4000 R08: 00007fcf095e3010 R09: 0000000000000000
>> [ 3145.455219][T17497] R10: fffffffffffffbc5 R11: 0000000000000246 R12: 0000000000000000
>> [ 3145.455222][T17497] R13: 0000000000000003 R14: 0000000000020000 R15: 0000000000020000
>> [ 3145.455227][T17497]  </TASK>
>> [ 3145.455229][T17497]
>> [ 3145.479014][T17497] Freed by task 17497 on cpu 0 at 3145.447575s:
>> [ 3145.479559][T17497]  kasan_save_track (mm/kasan/common.c:57 mm/kasan/common.c:78)
>> [ 3145.479963][T17497]  kasan_save_free_info (mm/kasan/generic.c:584)
>> [ 3145.480411][T17497]  __kasan_slab_free (mm/kasan/common.c:253 mm/kasan/common.c:285)
>> [ 3145.480813][T17497]  kfree (include/linux/kasan.h:235 mm/slub.c:2689 mm/slub.c:6251 mm/slub.c:6566)
>> [ 3145.481148][T17497]  device_release (drivers/base/core.c:2542)
>> [ 3145.481567][T17497]  kobject_put (lib/kobject.c:689 lib/kobject.c:720 include/linux/kref.h:65 lib/kobject.c:737)
>> [ 3145.481951][T17497]  sysfs_rtnl_lock (net/core/net-sysfs.c:121)
>> [ 3145.482351][T17497]  phys_port_name_show (net/core/net-sysfs.c:665)
>> [ 3145.482782][T17497]  dev_attr_show (drivers/base/core.c:2421)
>> [ 3145.483154][T17497]  sysfs_kf_seq_show (fs/sysfs/file.c:65)
>> [ 3145.483586][T17497]  seq_read_iter (fs/seq_file.c:231)
>> [ 3145.483975][T17497]  vfs_read (fs/read_write.c:493 fs/read_write.c:574)
>> [ 3145.484334][T17497]  ksys_read (fs/read_write.c:717)
>> [ 3145.484701][T17497]  do_syscall_64 (arch/x86/entry/syscall_64.c:63 arch/x86/entry/syscall_64.c:94)
>> [ 3145.485092][T17497]  entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
>> [ 3145.485592][T17497]
>> [ 3145.485794][T17497] The buggy address belongs to the object at ffff888107678000
>> [ 3145.485794][T17497]  which belongs to the cache kmalloc-cg-8k of size 8192
>> [ 3145.486991][T17497] The buggy address is located 1432 bytes inside of
>> [ 3145.486991][T17497]  freed 8192-byte region [ffff888107678000, ffff88810767a000)
>> [ 3145.488159][T17497]
>> [ 3145.488367][T17497] The buggy address belongs to the physical page:
>> 
>> 
>> Best,
>> Shuangpeng
>> 


      reply	other threads:[~2026-06-26 17:41 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-15  1:16 [BUG] netdevsim: KASAN slab-use-after-free in ref_tracker_free Shuangpeng Bai
2026-06-16  9:34 ` saeed bishara
2026-06-26 17:41   ` Shuangpeng [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2037CBE9-D72E-4E81-9B18-62AF3735306D@gmail.com \
    --to=shuangpeng.kernel@gmail.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=horms@kernel.org \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=saeed.bishara.os@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox