[PATCH] static_call: Fix a wild-memory-access bug in static_call_del

All of lore.kernel.org
 help / color / mirror / Atom feed

* [PATCH] static_call: Fix a wild-memory-access bug in static_call_del_module()
@ 2023-09-15  8:21 Jinjie Ruan
  2023-09-15  9:01 ` Peter Zijlstra
  2024-09-02 12:01 ` Jinjie Ruan
  0 siblings, 2 replies; 29+ messages in thread
From: Jinjie Ruan @ 2023-09-15  8:21 UTC (permalink / raw)
  To: tglx, linux-kernel, Peter Zijlstra (Intel), Christophe Leroy,
	Josh Poimboeuf
  Cc: ruanjinjie

Inject fault while probing btrfs.ko, if the first kzalloc() fails
in __static_call_init(), key->mods will no be initialized. And then
in static_call_del_module() the site_mod->mod will cause
wild-memory-access as below:

So assign key->mods to NULL in __static_call_init() if it fails
to fix the issue. And if kzalloc fails, it will just return in init
func, so it should break if it the key->mods is NULL in exit func.

 general protection fault, probably for non-canonical address 0xeb800159c89f94a0: 0000 [#1] PREEMPT SMP KASAN
 KASAN: maybe wild-memory-access in range [0x5c002ace44fca500-0x5c002ace44fca507]
 CPU: 2 PID: 1843 Comm: modprobe Tainted: G        W        N 6.6.0-rc1+ #60
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
 RIP: 0010:static_call_del_module+0x113/0x280
 Code: 3c 20 00 0f 85 ef 00 00 00 49 8b 6e 08 48 85 ed 75 0d eb 75 48 85 db 74 70 49 89 ef 48 89 dd 48 8d 7d 08 48 89 f8 48 c1 e8 03 <42> 80 3c 20 00 75 78 48 89 e8 4c 8b 6d 08 48 c1 e8 03 42 80 3c 20
 RSP: 0018:ffff888101d3f860 EFLAGS: 00010206
 RAX: 0b800559c89f94a0 RBX: 5c002ace44fca4f8 RCX: ffffffffa0210f00
 RDX: ffffffffa0210ed4 RSI: ffffffffa0210edc RDI: 5c002ace44fca500
 RBP: 5c002ace44fca4f8 R08: 0000000000000000 R09: ffffed10233e4eea
 R10: ffffed10233e4ee9 R11: ffff888119f2774b R12: dffffc0000000000
 R13: 80002ace3cfca4f8 R14: ffffffff85196de0 R15: ffffffff84ee9f99
 FS:  00007f4ff6faa540(0000) GS:ffff888119f00000(0000) knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 CR2: 00007ffc3d1f19e8 CR3: 0000000109fa6001 CR4: 0000000000170ee0
 DR0: ffffffff8faefce8 DR1: ffffffff8faefce9 DR2: ffffffff8faefcea
 DR3: ffffffff8faefceb DR6: 00000000ffff0ff0 DR7: 0000000000000600
 Call Trace:
  <TASK>
  ? __die_body+0x1b/0x60
  ? die_addr+0x43/0x70
  ? exc_general_protection+0x121/0x210
  ? asm_exc_general_protection+0x22/0x30
  ? static_call_del_module+0x113/0x280
  ? __SCT__tp_func_ipi_exit+0x8/0x8
  static_call_module_notify+0x27f/0x390
  ? rcu_segcblist_inc_len+0x17/0x20
  notifier_call_chain+0xbf/0x280
  notifier_call_chain_robust+0x7f/0xe0
  ? notifier_call_chain+0x280/0x280
  ? kasan_quarantine_put+0x46/0x160
  blocking_notifier_call_chain_robust+0x5b/0x80
  load_module+0x4d1d/0x69f0
  ? module_frob_arch_sections+0x20/0x20
  ? update_cfs_group+0x10c/0x2a0
  ? __wake_up_common+0x10b/0x5d0
  ? kernel_read_file+0x3ca/0x510
  ? __x64_sys_fsconfig+0x650/0x650
  ? __schedule+0xa0b/0x2a60
  ? init_module_from_file+0xd2/0x130
  init_module_from_file+0xd2/0x130
  ? __ia32_sys_init_module+0xa0/0xa0
  ? _raw_spin_lock_irqsave+0xe0/0xe0
  ? ptrace_stop+0x487/0x790
  idempotent_init_module+0x32d/0x6a0
  ? init_module_from_file+0x130/0x130
  ? __fget_light+0x57/0x500
  __x64_sys_finit_module+0xbb/0x130
  do_syscall_64+0x35/0x80
  entry_SYSCALL_64_after_hwframe+0x46/0xb0
 RIP: 0033:0x7f4ff691b839
 Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48
 RSP: 002b:00007ffc07b09718 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
 RAX: ffffffffffffffda RBX: 000055978f13e070 RCX: 00007f4ff691b839
 RDX: 0000000000000000 RSI: 000055978da1bc2e RDI: 0000000000000003
 RBP: 000055978da1bc2e R08: 0000000000000000 R09: 000055978f13ddb0
 R10: 0000000000000003 R11: 0000000000000246 R12: 0000000000000000
 R13: 000055978f13e020 R14: 0000000000040000 R15: 000055978f13ddb0
  </TASK>
 Modules linked in: tifm_core(+)
 Dumping ftrace buffer:
    (ftrace buffer empty)
 ---[ end trace 0000000000000000 ]---
 RIP: 0010:static_call_del_module+0x113/0x280
 Code: 3c 20 00 0f 85 ef 00 00 00 49 8b 6e 08 48 85 ed 75 0d eb 75 48 85 db 74 70 49 89 ef 48 89 dd 48 8d 7d 08 48 89 f8 48 c1 e8 03 <42> 80 3c 20 00 75 78 48 89 e8 4c 8b 6d 08 48 c1 e8 03 42 80 3c 20
 RSP: 0018:ffff888101d3f860 EFLAGS: 00010206
 RAX: 0b800559c89f94a0 RBX: 5c002ace44fca4f8 RCX: ffffffffa0210f00
 RDX: ffffffffa0210ed4 RSI: ffffffffa0210edc RDI: 5c002ace44fca500
 RBP: 5c002ace44fca4f8 R08: 0000000000000000 R09: ffffed10233e4eea
 R10: ffffed10233e4ee9 R11: ffff888119f2774b R12: dffffc0000000000
 R13: 80002ace3cfca4f8 R14: ffffffff85196de0 R15: ffffffff84ee9f99
 FS:  00007f4ff6faa540(0000) GS:ffff888119f00000(0000) knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 CR2: 00007ffc3d1f19e8 CR3: 0000000109fa6001 CR4: 0000000000170ee0
 DR0: ffffffff8faefce8 DR1: ffffffff8faefce9 DR2: ffffffff8faefcea
 DR3: ffffffff8faefceb DR6: 00000000ffff0ff0 DR7: 0000000000000600
 Kernel panic - not syncing: Fatal exception
 Dumping ftrace buffer:
    (ftrace buffer empty)
 Kernel Offset: disabled
 Rebooting in 1 seconds..

Fixes: 8fd4ddda2f49 ("static_call: Don't make __static_call_return0 static")
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
---
 kernel/static_call_inline.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kernel/static_call_inline.c b/kernel/static_call_inline.c
index 639397b5491c..e7aa70d33530 100644
--- a/kernel/static_call_inline.c
+++ b/kernel/static_call_inline.c
@@ -256,8 +256,10 @@ static int __static_call_init(struct module *mod,
 			}
 
 			site_mod = kzalloc(sizeof(*site_mod), GFP_KERNEL);
-			if (!site_mod)
+			if (!site_mod) {
+				key->mods = NULL;
 				return -ENOMEM;
+			}
 
 			/*
 			 * When the key has a direct sites pointer, extract
@@ -422,7 +424,7 @@ static void static_call_del_module(struct module *mod)
 			;
 
 		if (!site_mod)
-			continue;
+			break;
 
 		*prev = site_mod->next;
 		kfree(site_mod);
-- 
2.34.1


^ permalink raw reply related	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Fix a wild-memory-access bug in static_call_del_module()
  2023-09-15  8:21 [PATCH] static_call: Fix a wild-memory-access bug in static_call_del_module() Jinjie Ruan
@ 2023-09-15  9:01 ` Peter Zijlstra
  2024-09-02 12:01 ` Jinjie Ruan
  1 sibling, 0 replies; 29+ messages in thread
From: Peter Zijlstra @ 2023-09-15  9:01 UTC (permalink / raw)
  To: Jinjie Ruan; +Cc: tglx, linux-kernel, Christophe Leroy, Josh Poimboeuf

On Fri, Sep 15, 2023 at 04:21:25PM +0800, Jinjie Ruan wrote:
> Inject fault while probing btrfs.ko, if the first kzalloc() fails
> in __static_call_init(), key->mods will no be initialized. And then
> in static_call_del_module() the site_mod->mod will cause
> wild-memory-access as below:
> 
> So assign key->mods to NULL in __static_call_init() if it fails
> to fix the issue. And if kzalloc fails, it will just return in init
> func, so it should break if it the key->mods is NULL in exit func.
> 

I don't think we need that full splat.

> 
> Fixes: 8fd4ddda2f49 ("static_call: Don't make __static_call_return0 static")

And that looks wrong, that just moved code around.

> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
> ---
>  kernel/static_call_inline.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/static_call_inline.c b/kernel/static_call_inline.c
> index 639397b5491c..e7aa70d33530 100644
> --- a/kernel/static_call_inline.c
> +++ b/kernel/static_call_inline.c
> @@ -256,8 +256,10 @@ static int __static_call_init(struct module *mod,
>  			}
>  
>  			site_mod = kzalloc(sizeof(*site_mod), GFP_KERNEL);
> -			if (!site_mod)
> +			if (!site_mod) {
> +				key->mods = NULL;
>  				return -ENOMEM;
> +			}
>  
>  			/*
>  			 * When the key has a direct sites pointer, extract
> @@ -422,7 +424,7 @@ static void static_call_del_module(struct module *mod)
>  			;
>  
>  		if (!site_mod)
> -			continue;
> +			break;
>  
>  		*prev = site_mod->next;
>  		kfree(site_mod);

The actual patch looks okay.

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Fix a wild-memory-access bug in static_call_del_module()
  2023-09-15  8:21 [PATCH] static_call: Fix a wild-memory-access bug in static_call_del_module() Jinjie Ruan
  2023-09-15  9:01 ` Peter Zijlstra
@ 2024-09-02 12:01 ` Jinjie Ruan
  2024-09-02 12:06   ` Christophe Leroy
  2024-09-03 20:01   ` Thomas Gleixner
  1 sibling, 2 replies; 29+ messages in thread
From: Jinjie Ruan @ 2024-09-02 12:01 UTC (permalink / raw)
  To: tglx, linux-kernel, Peter Zijlstra (Intel), Christophe Leroy,
	Josh Poimboeuf

Hi, it seems that this bug still exit.

On 2023/9/15 16:21, Jinjie Ruan wrote:
> Inject fault while probing btrfs.ko, if the first kzalloc() fails
> in __static_call_init(), key->mods will no be initialized. And then
> in static_call_del_module() the site_mod->mod will cause
> wild-memory-access as below:
> 
> So assign key->mods to NULL in __static_call_init() if it fails
> to fix the issue. And if kzalloc fails, it will just return in init
> func, so it should break if it the key->mods is NULL in exit func.
> 
>  general protection fault, probably for non-canonical address 0xeb800159c89f94a0: 0000 [#1] PREEMPT SMP KASAN
>  KASAN: maybe wild-memory-access in range [0x5c002ace44fca500-0x5c002ace44fca507]
>  CPU: 2 PID: 1843 Comm: modprobe Tainted: G        W        N 6.6.0-rc1+ #60
>  Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
>  RIP: 0010:static_call_del_module+0x113/0x280
>  Code: 3c 20 00 0f 85 ef 00 00 00 49 8b 6e 08 48 85 ed 75 0d eb 75 48 85 db 74 70 49 89 ef 48 89 dd 48 8d 7d 08 48 89 f8 48 c1 e8 03 <42> 80 3c 20 00 75 78 48 89 e8 4c 8b 6d 08 48 c1 e8 03 42 80 3c 20
>  RSP: 0018:ffff888101d3f860 EFLAGS: 00010206
>  RAX: 0b800559c89f94a0 RBX: 5c002ace44fca4f8 RCX: ffffffffa0210f00
>  RDX: ffffffffa0210ed4 RSI: ffffffffa0210edc RDI: 5c002ace44fca500
>  RBP: 5c002ace44fca4f8 R08: 0000000000000000 R09: ffffed10233e4eea
>  R10: ffffed10233e4ee9 R11: ffff888119f2774b R12: dffffc0000000000
>  R13: 80002ace3cfca4f8 R14: ffffffff85196de0 R15: ffffffff84ee9f99
>  FS:  00007f4ff6faa540(0000) GS:ffff888119f00000(0000) knlGS:0000000000000000
>  CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>  CR2: 00007ffc3d1f19e8 CR3: 0000000109fa6001 CR4: 0000000000170ee0
>  DR0: ffffffff8faefce8 DR1: ffffffff8faefce9 DR2: ffffffff8faefcea
>  DR3: ffffffff8faefceb DR6: 00000000ffff0ff0 DR7: 0000000000000600
>  Call Trace:
>   <TASK>
>   ? __die_body+0x1b/0x60
>   ? die_addr+0x43/0x70
>   ? exc_general_protection+0x121/0x210
>   ? asm_exc_general_protection+0x22/0x30
>   ? static_call_del_module+0x113/0x280
>   ? __SCT__tp_func_ipi_exit+0x8/0x8
>   static_call_module_notify+0x27f/0x390
>   ? rcu_segcblist_inc_len+0x17/0x20
>   notifier_call_chain+0xbf/0x280
>   notifier_call_chain_robust+0x7f/0xe0
>   ? notifier_call_chain+0x280/0x280
>   ? kasan_quarantine_put+0x46/0x160
>   blocking_notifier_call_chain_robust+0x5b/0x80
>   load_module+0x4d1d/0x69f0
>   ? module_frob_arch_sections+0x20/0x20
>   ? update_cfs_group+0x10c/0x2a0
>   ? __wake_up_common+0x10b/0x5d0
>   ? kernel_read_file+0x3ca/0x510
>   ? __x64_sys_fsconfig+0x650/0x650
>   ? __schedule+0xa0b/0x2a60
>   ? init_module_from_file+0xd2/0x130
>   init_module_from_file+0xd2/0x130
>   ? __ia32_sys_init_module+0xa0/0xa0
>   ? _raw_spin_lock_irqsave+0xe0/0xe0
>   ? ptrace_stop+0x487/0x790
>   idempotent_init_module+0x32d/0x6a0
>   ? init_module_from_file+0x130/0x130
>   ? __fget_light+0x57/0x500
>   __x64_sys_finit_module+0xbb/0x130
>   do_syscall_64+0x35/0x80
>   entry_SYSCALL_64_after_hwframe+0x46/0xb0
>  RIP: 0033:0x7f4ff691b839
>  Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48
>  RSP: 002b:00007ffc07b09718 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
>  RAX: ffffffffffffffda RBX: 000055978f13e070 RCX: 00007f4ff691b839
>  RDX: 0000000000000000 RSI: 000055978da1bc2e RDI: 0000000000000003
>  RBP: 000055978da1bc2e R08: 0000000000000000 R09: 000055978f13ddb0
>  R10: 0000000000000003 R11: 0000000000000246 R12: 0000000000000000
>  R13: 000055978f13e020 R14: 0000000000040000 R15: 000055978f13ddb0
>   </TASK>
>  Modules linked in: tifm_core(+)
>  Dumping ftrace buffer:
>     (ftrace buffer empty)
>  ---[ end trace 0000000000000000 ]---
>  RIP: 0010:static_call_del_module+0x113/0x280
>  Code: 3c 20 00 0f 85 ef 00 00 00 49 8b 6e 08 48 85 ed 75 0d eb 75 48 85 db 74 70 49 89 ef 48 89 dd 48 8d 7d 08 48 89 f8 48 c1 e8 03 <42> 80 3c 20 00 75 78 48 89 e8 4c 8b 6d 08 48 c1 e8 03 42 80 3c 20
>  RSP: 0018:ffff888101d3f860 EFLAGS: 00010206
>  RAX: 0b800559c89f94a0 RBX: 5c002ace44fca4f8 RCX: ffffffffa0210f00
>  RDX: ffffffffa0210ed4 RSI: ffffffffa0210edc RDI: 5c002ace44fca500
>  RBP: 5c002ace44fca4f8 R08: 0000000000000000 R09: ffffed10233e4eea
>  R10: ffffed10233e4ee9 R11: ffff888119f2774b R12: dffffc0000000000
>  R13: 80002ace3cfca4f8 R14: ffffffff85196de0 R15: ffffffff84ee9f99
>  FS:  00007f4ff6faa540(0000) GS:ffff888119f00000(0000) knlGS:0000000000000000
>  CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>  CR2: 00007ffc3d1f19e8 CR3: 0000000109fa6001 CR4: 0000000000170ee0
>  DR0: ffffffff8faefce8 DR1: ffffffff8faefce9 DR2: ffffffff8faefcea
>  DR3: ffffffff8faefceb DR6: 00000000ffff0ff0 DR7: 0000000000000600
>  Kernel panic - not syncing: Fatal exception
>  Dumping ftrace buffer:
>     (ftrace buffer empty)
>  Kernel Offset: disabled
>  Rebooting in 1 seconds..
> 
> Fixes: 8fd4ddda2f49 ("static_call: Don't make __static_call_return0 static")
> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
> ---
>  kernel/static_call_inline.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/static_call_inline.c b/kernel/static_call_inline.c
> index 639397b5491c..e7aa70d33530 100644
> --- a/kernel/static_call_inline.c
> +++ b/kernel/static_call_inline.c
> @@ -256,8 +256,10 @@ static int __static_call_init(struct module *mod,
>  			}
>  
>  			site_mod = kzalloc(sizeof(*site_mod), GFP_KERNEL);
> -			if (!site_mod)
> +			if (!site_mod) {
> +				key->mods = NULL;
>  				return -ENOMEM;
> +			}
>  
>  			/*
>  			 * When the key has a direct sites pointer, extract
> @@ -422,7 +424,7 @@ static void static_call_del_module(struct module *mod)
>  			;
>  
>  		if (!site_mod)
> -			continue;
> +			break;
>  
>  		*prev = site_mod->next;
>  		kfree(site_mod);

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Fix a wild-memory-access bug in static_call_del_module()
  2024-09-02 12:01 ` Jinjie Ruan
@ 2024-09-02 12:06   ` Christophe Leroy
  2024-09-02 12:07     ` Jinjie Ruan
  2024-09-03 20:01   ` Thomas Gleixner
  1 sibling, 1 reply; 29+ messages in thread
From: Christophe Leroy @ 2024-09-02 12:06 UTC (permalink / raw)
  To: Jinjie Ruan, tglx, linux-kernel, Peter Zijlstra (Intel),
	Josh Poimboeuf

Hi,

Le 02/09/2024 à 14:01, Jinjie Ruan a écrit :
> Hi, it seems that this bug still exit.

Can you provide more details ?

In particular, what makes you think it is the exact same bug ?

Christophe


> 
> On 2023/9/15 16:21, Jinjie Ruan wrote:
>> Inject fault while probing btrfs.ko, if the first kzalloc() fails
>> in __static_call_init(), key->mods will no be initialized. And then
>> in static_call_del_module() the site_mod->mod will cause
>> wild-memory-access as below:
>>
>> So assign key->mods to NULL in __static_call_init() if it fails
>> to fix the issue. And if kzalloc fails, it will just return in init
>> func, so it should break if it the key->mods is NULL in exit func.
>>
>>   general protection fault, probably for non-canonical address 0xeb800159c89f94a0: 0000 [#1] PREEMPT SMP KASAN
>>   KASAN: maybe wild-memory-access in range [0x5c002ace44fca500-0x5c002ace44fca507]
>>   CPU: 2 PID: 1843 Comm: modprobe Tainted: G        W        N 6.6.0-rc1+ #60
>>   Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
>>   RIP: 0010:static_call_del_module+0x113/0x280
>>   Code: 3c 20 00 0f 85 ef 00 00 00 49 8b 6e 08 48 85 ed 75 0d eb 75 48 85 db 74 70 49 89 ef 48 89 dd 48 8d 7d 08 48 89 f8 48 c1 e8 03 <42> 80 3c 20 00 75 78 48 89 e8 4c 8b 6d 08 48 c1 e8 03 42 80 3c 20
>>   RSP: 0018:ffff888101d3f860 EFLAGS: 00010206
>>   RAX: 0b800559c89f94a0 RBX: 5c002ace44fca4f8 RCX: ffffffffa0210f00
>>   RDX: ffffffffa0210ed4 RSI: ffffffffa0210edc RDI: 5c002ace44fca500
>>   RBP: 5c002ace44fca4f8 R08: 0000000000000000 R09: ffffed10233e4eea
>>   R10: ffffed10233e4ee9 R11: ffff888119f2774b R12: dffffc0000000000
>>   R13: 80002ace3cfca4f8 R14: ffffffff85196de0 R15: ffffffff84ee9f99
>>   FS:  00007f4ff6faa540(0000) GS:ffff888119f00000(0000) knlGS:0000000000000000
>>   CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>   CR2: 00007ffc3d1f19e8 CR3: 0000000109fa6001 CR4: 0000000000170ee0
>>   DR0: ffffffff8faefce8 DR1: ffffffff8faefce9 DR2: ffffffff8faefcea
>>   DR3: ffffffff8faefceb DR6: 00000000ffff0ff0 DR7: 0000000000000600
>>   Call Trace:
>>    <TASK>
>>    ? __die_body+0x1b/0x60
>>    ? die_addr+0x43/0x70
>>    ? exc_general_protection+0x121/0x210
>>    ? asm_exc_general_protection+0x22/0x30
>>    ? static_call_del_module+0x113/0x280
>>    ? __SCT__tp_func_ipi_exit+0x8/0x8
>>    static_call_module_notify+0x27f/0x390
>>    ? rcu_segcblist_inc_len+0x17/0x20
>>    notifier_call_chain+0xbf/0x280
>>    notifier_call_chain_robust+0x7f/0xe0
>>    ? notifier_call_chain+0x280/0x280
>>    ? kasan_quarantine_put+0x46/0x160
>>    blocking_notifier_call_chain_robust+0x5b/0x80
>>    load_module+0x4d1d/0x69f0
>>    ? module_frob_arch_sections+0x20/0x20
>>    ? update_cfs_group+0x10c/0x2a0
>>    ? __wake_up_common+0x10b/0x5d0
>>    ? kernel_read_file+0x3ca/0x510
>>    ? __x64_sys_fsconfig+0x650/0x650
>>    ? __schedule+0xa0b/0x2a60
>>    ? init_module_from_file+0xd2/0x130
>>    init_module_from_file+0xd2/0x130
>>    ? __ia32_sys_init_module+0xa0/0xa0
>>    ? _raw_spin_lock_irqsave+0xe0/0xe0
>>    ? ptrace_stop+0x487/0x790
>>    idempotent_init_module+0x32d/0x6a0
>>    ? init_module_from_file+0x130/0x130
>>    ? __fget_light+0x57/0x500
>>    __x64_sys_finit_module+0xbb/0x130
>>    do_syscall_64+0x35/0x80
>>    entry_SYSCALL_64_after_hwframe+0x46/0xb0
>>   RIP: 0033:0x7f4ff691b839
>>   Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48
>>   RSP: 002b:00007ffc07b09718 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
>>   RAX: ffffffffffffffda RBX: 000055978f13e070 RCX: 00007f4ff691b839
>>   RDX: 0000000000000000 RSI: 000055978da1bc2e RDI: 0000000000000003
>>   RBP: 000055978da1bc2e R08: 0000000000000000 R09: 000055978f13ddb0
>>   R10: 0000000000000003 R11: 0000000000000246 R12: 0000000000000000
>>   R13: 000055978f13e020 R14: 0000000000040000 R15: 000055978f13ddb0
>>    </TASK>
>>   Modules linked in: tifm_core(+)
>>   Dumping ftrace buffer:
>>      (ftrace buffer empty)
>>   ---[ end trace 0000000000000000 ]---
>>   RIP: 0010:static_call_del_module+0x113/0x280
>>   Code: 3c 20 00 0f 85 ef 00 00 00 49 8b 6e 08 48 85 ed 75 0d eb 75 48 85 db 74 70 49 89 ef 48 89 dd 48 8d 7d 08 48 89 f8 48 c1 e8 03 <42> 80 3c 20 00 75 78 48 89 e8 4c 8b 6d 08 48 c1 e8 03 42 80 3c 20
>>   RSP: 0018:ffff888101d3f860 EFLAGS: 00010206
>>   RAX: 0b800559c89f94a0 RBX: 5c002ace44fca4f8 RCX: ffffffffa0210f00
>>   RDX: ffffffffa0210ed4 RSI: ffffffffa0210edc RDI: 5c002ace44fca500
>>   RBP: 5c002ace44fca4f8 R08: 0000000000000000 R09: ffffed10233e4eea
>>   R10: ffffed10233e4ee9 R11: ffff888119f2774b R12: dffffc0000000000
>>   R13: 80002ace3cfca4f8 R14: ffffffff85196de0 R15: ffffffff84ee9f99
>>   FS:  00007f4ff6faa540(0000) GS:ffff888119f00000(0000) knlGS:0000000000000000
>>   CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>   CR2: 00007ffc3d1f19e8 CR3: 0000000109fa6001 CR4: 0000000000170ee0
>>   DR0: ffffffff8faefce8 DR1: ffffffff8faefce9 DR2: ffffffff8faefcea
>>   DR3: ffffffff8faefceb DR6: 00000000ffff0ff0 DR7: 0000000000000600
>>   Kernel panic - not syncing: Fatal exception
>>   Dumping ftrace buffer:
>>      (ftrace buffer empty)
>>   Kernel Offset: disabled
>>   Rebooting in 1 seconds..
>>
>> Fixes: 8fd4ddda2f49 ("static_call: Don't make __static_call_return0 static")
>> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
>> ---
>>   kernel/static_call_inline.c | 6 ++++--
>>   1 file changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/kernel/static_call_inline.c b/kernel/static_call_inline.c
>> index 639397b5491c..e7aa70d33530 100644
>> --- a/kernel/static_call_inline.c
>> +++ b/kernel/static_call_inline.c
>> @@ -256,8 +256,10 @@ static int __static_call_init(struct module *mod,
>>   			}
>>   
>>   			site_mod = kzalloc(sizeof(*site_mod), GFP_KERNEL);
>> -			if (!site_mod)
>> +			if (!site_mod) {
>> +				key->mods = NULL;
>>   				return -ENOMEM;
>> +			}
>>   
>>   			/*
>>   			 * When the key has a direct sites pointer, extract
>> @@ -422,7 +424,7 @@ static void static_call_del_module(struct module *mod)
>>   			;
>>   
>>   		if (!site_mod)
>> -			continue;
>> +			break;
>>   
>>   		*prev = site_mod->next;
>>   		kfree(site_mod);

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Fix a wild-memory-access bug in static_call_del_module()
  2024-09-02 12:06   ` Christophe Leroy
@ 2024-09-02 12:07     ` Jinjie Ruan
  2024-09-02 12:14       ` Christophe Leroy
  0 siblings, 1 reply; 29+ messages in thread
From: Jinjie Ruan @ 2024-09-02 12:07 UTC (permalink / raw)
  To: Christophe Leroy, tglx, linux-kernel, Peter Zijlstra (Intel),
	Josh Poimboeuf



On 2024/9/2 20:06, Christophe Leroy wrote:
> Hi,
> 
> Le 02/09/2024 à 14:01, Jinjie Ruan a écrit :
>> Hi, it seems that this bug still exit.
> 
> Can you provide more details ?
> 
> In particular, what makes you think it is the exact same bug ?

I apply this patch, and the problem not occurs.

> 
> Christophe
> 
> 
>>
>> On 2023/9/15 16:21, Jinjie Ruan wrote:
>>> Inject fault while probing btrfs.ko, if the first kzalloc() fails
>>> in __static_call_init(), key->mods will no be initialized. And then
>>> in static_call_del_module() the site_mod->mod will cause
>>> wild-memory-access as below:
>>>
>>> So assign key->mods to NULL in __static_call_init() if it fails
>>> to fix the issue. And if kzalloc fails, it will just return in init
>>> func, so it should break if it the key->mods is NULL in exit func.
>>>
>>>   general protection fault, probably for non-canonical address
>>> 0xeb800159c89f94a0: 0000 [#1] PREEMPT SMP KASAN
>>>   KASAN: maybe wild-memory-access in range
>>> [0x5c002ace44fca500-0x5c002ace44fca507]
>>>   CPU: 2 PID: 1843 Comm: modprobe Tainted: G        W        N
>>> 6.6.0-rc1+ #60
>>>   Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
>>> 1.13.0-1ubuntu1.1 04/01/2014
>>>   RIP: 0010:static_call_del_module+0x113/0x280
>>>   Code: 3c 20 00 0f 85 ef 00 00 00 49 8b 6e 08 48 85 ed 75 0d eb 75
>>> 48 85 db 74 70 49 89 ef 48 89 dd 48 8d 7d 08 48 89 f8 48 c1 e8 03
>>> <42> 80 3c 20 00 75 78 48 89 e8 4c 8b 6d 08 48 c1 e8 03 42 80 3c 20
>>>   RSP: 0018:ffff888101d3f860 EFLAGS: 00010206
>>>   RAX: 0b800559c89f94a0 RBX: 5c002ace44fca4f8 RCX: ffffffffa0210f00
>>>   RDX: ffffffffa0210ed4 RSI: ffffffffa0210edc RDI: 5c002ace44fca500
>>>   RBP: 5c002ace44fca4f8 R08: 0000000000000000 R09: ffffed10233e4eea
>>>   R10: ffffed10233e4ee9 R11: ffff888119f2774b R12: dffffc0000000000
>>>   R13: 80002ace3cfca4f8 R14: ffffffff85196de0 R15: ffffffff84ee9f99
>>>   FS:  00007f4ff6faa540(0000) GS:ffff888119f00000(0000)
>>> knlGS:0000000000000000
>>>   CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>   CR2: 00007ffc3d1f19e8 CR3: 0000000109fa6001 CR4: 0000000000170ee0
>>>   DR0: ffffffff8faefce8 DR1: ffffffff8faefce9 DR2: ffffffff8faefcea
>>>   DR3: ffffffff8faefceb DR6: 00000000ffff0ff0 DR7: 0000000000000600
>>>   Call Trace:
>>>    <TASK>
>>>    ? __die_body+0x1b/0x60
>>>    ? die_addr+0x43/0x70
>>>    ? exc_general_protection+0x121/0x210
>>>    ? asm_exc_general_protection+0x22/0x30
>>>    ? static_call_del_module+0x113/0x280
>>>    ? __SCT__tp_func_ipi_exit+0x8/0x8
>>>    static_call_module_notify+0x27f/0x390
>>>    ? rcu_segcblist_inc_len+0x17/0x20
>>>    notifier_call_chain+0xbf/0x280
>>>    notifier_call_chain_robust+0x7f/0xe0
>>>    ? notifier_call_chain+0x280/0x280
>>>    ? kasan_quarantine_put+0x46/0x160
>>>    blocking_notifier_call_chain_robust+0x5b/0x80
>>>    load_module+0x4d1d/0x69f0
>>>    ? module_frob_arch_sections+0x20/0x20
>>>    ? update_cfs_group+0x10c/0x2a0
>>>    ? __wake_up_common+0x10b/0x5d0
>>>    ? kernel_read_file+0x3ca/0x510
>>>    ? __x64_sys_fsconfig+0x650/0x650
>>>    ? __schedule+0xa0b/0x2a60
>>>    ? init_module_from_file+0xd2/0x130
>>>    init_module_from_file+0xd2/0x130
>>>    ? __ia32_sys_init_module+0xa0/0xa0
>>>    ? _raw_spin_lock_irqsave+0xe0/0xe0
>>>    ? ptrace_stop+0x487/0x790
>>>    idempotent_init_module+0x32d/0x6a0
>>>    ? init_module_from_file+0x130/0x130
>>>    ? __fget_light+0x57/0x500
>>>    __x64_sys_finit_module+0xbb/0x130
>>>    do_syscall_64+0x35/0x80
>>>    entry_SYSCALL_64_after_hwframe+0x46/0xb0
>>>   RIP: 0033:0x7f4ff691b839
>>>   Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8
>>> 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05
>>> <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48
>>>   RSP: 002b:00007ffc07b09718 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
>>>   RAX: ffffffffffffffda RBX: 000055978f13e070 RCX: 00007f4ff691b839
>>>   RDX: 0000000000000000 RSI: 000055978da1bc2e RDI: 0000000000000003
>>>   RBP: 000055978da1bc2e R08: 0000000000000000 R09: 000055978f13ddb0
>>>   R10: 0000000000000003 R11: 0000000000000246 R12: 0000000000000000
>>>   R13: 000055978f13e020 R14: 0000000000040000 R15: 000055978f13ddb0
>>>    </TASK>
>>>   Modules linked in: tifm_core(+)
>>>   Dumping ftrace buffer:
>>>      (ftrace buffer empty)
>>>   ---[ end trace 0000000000000000 ]---
>>>   RIP: 0010:static_call_del_module+0x113/0x280
>>>   Code: 3c 20 00 0f 85 ef 00 00 00 49 8b 6e 08 48 85 ed 75 0d eb 75
>>> 48 85 db 74 70 49 89 ef 48 89 dd 48 8d 7d 08 48 89 f8 48 c1 e8 03
>>> <42> 80 3c 20 00 75 78 48 89 e8 4c 8b 6d 08 48 c1 e8 03 42 80 3c 20
>>>   RSP: 0018:ffff888101d3f860 EFLAGS: 00010206
>>>   RAX: 0b800559c89f94a0 RBX: 5c002ace44fca4f8 RCX: ffffffffa0210f00
>>>   RDX: ffffffffa0210ed4 RSI: ffffffffa0210edc RDI: 5c002ace44fca500
>>>   RBP: 5c002ace44fca4f8 R08: 0000000000000000 R09: ffffed10233e4eea
>>>   R10: ffffed10233e4ee9 R11: ffff888119f2774b R12: dffffc0000000000
>>>   R13: 80002ace3cfca4f8 R14: ffffffff85196de0 R15: ffffffff84ee9f99
>>>   FS:  00007f4ff6faa540(0000) GS:ffff888119f00000(0000)
>>> knlGS:0000000000000000
>>>   CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>   CR2: 00007ffc3d1f19e8 CR3: 0000000109fa6001 CR4: 0000000000170ee0
>>>   DR0: ffffffff8faefce8 DR1: ffffffff8faefce9 DR2: ffffffff8faefcea
>>>   DR3: ffffffff8faefceb DR6: 00000000ffff0ff0 DR7: 0000000000000600
>>>   Kernel panic - not syncing: Fatal exception
>>>   Dumping ftrace buffer:
>>>      (ftrace buffer empty)
>>>   Kernel Offset: disabled
>>>   Rebooting in 1 seconds..
>>>
>>> Fixes: 8fd4ddda2f49 ("static_call: Don't make __static_call_return0
>>> static")
>>> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
>>> ---
>>>   kernel/static_call_inline.c | 6 ++++--
>>>   1 file changed, 4 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/kernel/static_call_inline.c b/kernel/static_call_inline.c
>>> index 639397b5491c..e7aa70d33530 100644
>>> --- a/kernel/static_call_inline.c
>>> +++ b/kernel/static_call_inline.c
>>> @@ -256,8 +256,10 @@ static int __static_call_init(struct module *mod,
>>>               }
>>>                 site_mod = kzalloc(sizeof(*site_mod), GFP_KERNEL);
>>> -            if (!site_mod)
>>> +            if (!site_mod) {
>>> +                key->mods = NULL;
>>>                   return -ENOMEM;
>>> +            }
>>>                 /*
>>>                * When the key has a direct sites pointer, extract
>>> @@ -422,7 +424,7 @@ static void static_call_del_module(struct module
>>> *mod)
>>>               ;
>>>             if (!site_mod)
>>> -            continue;
>>> +            break;
>>>             *prev = site_mod->next;
>>>           kfree(site_mod);
> 

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Fix a wild-memory-access bug in static_call_del_module()
  2024-09-02 12:07     ` Jinjie Ruan
@ 2024-09-02 12:14       ` Christophe Leroy
  0 siblings, 0 replies; 29+ messages in thread
From: Christophe Leroy @ 2024-09-02 12:14 UTC (permalink / raw)
  To: Jinjie Ruan, tglx, linux-kernel, Peter Zijlstra (Intel),
	Josh Poimboeuf



Le 02/09/2024 à 14:07, Jinjie Ruan a écrit :
> 
> 
> On 2024/9/2 20:06, Christophe Leroy wrote:
>> Hi,
>>
>> Le 02/09/2024 à 14:01, Jinjie Ruan a écrit :
>>> Hi, it seems that this bug still exit.
>>
>> Can you provide more details ?
>>
>> In particular, what makes you think it is the exact same bug ?
> 
> I apply this patch, and the problem not occurs.

Ah, you mean the patch hasn't been applied yet ?

I thought you were telling that despite the patch the problem was still 
there.

Christophe

> 
>>
>> Christophe
>>
>>
>>>
>>> On 2023/9/15 16:21, Jinjie Ruan wrote:
>>>> Inject fault while probing btrfs.ko, if the first kzalloc() fails
>>>> in __static_call_init(), key->mods will no be initialized. And then
>>>> in static_call_del_module() the site_mod->mod will cause
>>>> wild-memory-access as below:
>>>>
>>>> So assign key->mods to NULL in __static_call_init() if it fails
>>>> to fix the issue. And if kzalloc fails, it will just return in init
>>>> func, so it should break if it the key->mods is NULL in exit func.
>>>>
>>>>    general protection fault, probably for non-canonical address
>>>> 0xeb800159c89f94a0: 0000 [#1] PREEMPT SMP KASAN
>>>>    KASAN: maybe wild-memory-access in range
>>>> [0x5c002ace44fca500-0x5c002ace44fca507]
>>>>    CPU: 2 PID: 1843 Comm: modprobe Tainted: G        W        N
>>>> 6.6.0-rc1+ #60
>>>>    Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
>>>> 1.13.0-1ubuntu1.1 04/01/2014
>>>>    RIP: 0010:static_call_del_module+0x113/0x280
>>>>    Code: 3c 20 00 0f 85 ef 00 00 00 49 8b 6e 08 48 85 ed 75 0d eb 75
>>>> 48 85 db 74 70 49 89 ef 48 89 dd 48 8d 7d 08 48 89 f8 48 c1 e8 03
>>>> <42> 80 3c 20 00 75 78 48 89 e8 4c 8b 6d 08 48 c1 e8 03 42 80 3c 20
>>>>    RSP: 0018:ffff888101d3f860 EFLAGS: 00010206
>>>>    RAX: 0b800559c89f94a0 RBX: 5c002ace44fca4f8 RCX: ffffffffa0210f00
>>>>    RDX: ffffffffa0210ed4 RSI: ffffffffa0210edc RDI: 5c002ace44fca500
>>>>    RBP: 5c002ace44fca4f8 R08: 0000000000000000 R09: ffffed10233e4eea
>>>>    R10: ffffed10233e4ee9 R11: ffff888119f2774b R12: dffffc0000000000
>>>>    R13: 80002ace3cfca4f8 R14: ffffffff85196de0 R15: ffffffff84ee9f99
>>>>    FS:  00007f4ff6faa540(0000) GS:ffff888119f00000(0000)
>>>> knlGS:0000000000000000
>>>>    CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>>    CR2: 00007ffc3d1f19e8 CR3: 0000000109fa6001 CR4: 0000000000170ee0
>>>>    DR0: ffffffff8faefce8 DR1: ffffffff8faefce9 DR2: ffffffff8faefcea
>>>>    DR3: ffffffff8faefceb DR6: 00000000ffff0ff0 DR7: 0000000000000600
>>>>    Call Trace:
>>>>     <TASK>
>>>>     ? __die_body+0x1b/0x60
>>>>     ? die_addr+0x43/0x70
>>>>     ? exc_general_protection+0x121/0x210
>>>>     ? asm_exc_general_protection+0x22/0x30
>>>>     ? static_call_del_module+0x113/0x280
>>>>     ? __SCT__tp_func_ipi_exit+0x8/0x8
>>>>     static_call_module_notify+0x27f/0x390
>>>>     ? rcu_segcblist_inc_len+0x17/0x20
>>>>     notifier_call_chain+0xbf/0x280
>>>>     notifier_call_chain_robust+0x7f/0xe0
>>>>     ? notifier_call_chain+0x280/0x280
>>>>     ? kasan_quarantine_put+0x46/0x160
>>>>     blocking_notifier_call_chain_robust+0x5b/0x80
>>>>     load_module+0x4d1d/0x69f0
>>>>     ? module_frob_arch_sections+0x20/0x20
>>>>     ? update_cfs_group+0x10c/0x2a0
>>>>     ? __wake_up_common+0x10b/0x5d0
>>>>     ? kernel_read_file+0x3ca/0x510
>>>>     ? __x64_sys_fsconfig+0x650/0x650
>>>>     ? __schedule+0xa0b/0x2a60
>>>>     ? init_module_from_file+0xd2/0x130
>>>>     init_module_from_file+0xd2/0x130
>>>>     ? __ia32_sys_init_module+0xa0/0xa0
>>>>     ? _raw_spin_lock_irqsave+0xe0/0xe0
>>>>     ? ptrace_stop+0x487/0x790
>>>>     idempotent_init_module+0x32d/0x6a0
>>>>     ? init_module_from_file+0x130/0x130
>>>>     ? __fget_light+0x57/0x500
>>>>     __x64_sys_finit_module+0xbb/0x130
>>>>     do_syscall_64+0x35/0x80
>>>>     entry_SYSCALL_64_after_hwframe+0x46/0xb0
>>>>    RIP: 0033:0x7f4ff691b839
>>>>    Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8
>>>> 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05
>>>> <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48
>>>>    RSP: 002b:00007ffc07b09718 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
>>>>    RAX: ffffffffffffffda RBX: 000055978f13e070 RCX: 00007f4ff691b839
>>>>    RDX: 0000000000000000 RSI: 000055978da1bc2e RDI: 0000000000000003
>>>>    RBP: 000055978da1bc2e R08: 0000000000000000 R09: 000055978f13ddb0
>>>>    R10: 0000000000000003 R11: 0000000000000246 R12: 0000000000000000
>>>>    R13: 000055978f13e020 R14: 0000000000040000 R15: 000055978f13ddb0
>>>>     </TASK>
>>>>    Modules linked in: tifm_core(+)
>>>>    Dumping ftrace buffer:
>>>>       (ftrace buffer empty)
>>>>    ---[ end trace 0000000000000000 ]---
>>>>    RIP: 0010:static_call_del_module+0x113/0x280
>>>>    Code: 3c 20 00 0f 85 ef 00 00 00 49 8b 6e 08 48 85 ed 75 0d eb 75
>>>> 48 85 db 74 70 49 89 ef 48 89 dd 48 8d 7d 08 48 89 f8 48 c1 e8 03
>>>> <42> 80 3c 20 00 75 78 48 89 e8 4c 8b 6d 08 48 c1 e8 03 42 80 3c 20
>>>>    RSP: 0018:ffff888101d3f860 EFLAGS: 00010206
>>>>    RAX: 0b800559c89f94a0 RBX: 5c002ace44fca4f8 RCX: ffffffffa0210f00
>>>>    RDX: ffffffffa0210ed4 RSI: ffffffffa0210edc RDI: 5c002ace44fca500
>>>>    RBP: 5c002ace44fca4f8 R08: 0000000000000000 R09: ffffed10233e4eea
>>>>    R10: ffffed10233e4ee9 R11: ffff888119f2774b R12: dffffc0000000000
>>>>    R13: 80002ace3cfca4f8 R14: ffffffff85196de0 R15: ffffffff84ee9f99
>>>>    FS:  00007f4ff6faa540(0000) GS:ffff888119f00000(0000)
>>>> knlGS:0000000000000000
>>>>    CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>>    CR2: 00007ffc3d1f19e8 CR3: 0000000109fa6001 CR4: 0000000000170ee0
>>>>    DR0: ffffffff8faefce8 DR1: ffffffff8faefce9 DR2: ffffffff8faefcea
>>>>    DR3: ffffffff8faefceb DR6: 00000000ffff0ff0 DR7: 0000000000000600
>>>>    Kernel panic - not syncing: Fatal exception
>>>>    Dumping ftrace buffer:
>>>>       (ftrace buffer empty)
>>>>    Kernel Offset: disabled
>>>>    Rebooting in 1 seconds..
>>>>
>>>> Fixes: 8fd4ddda2f49 ("static_call: Don't make __static_call_return0
>>>> static")
>>>> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
>>>> ---
>>>>    kernel/static_call_inline.c | 6 ++++--
>>>>    1 file changed, 4 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/kernel/static_call_inline.c b/kernel/static_call_inline.c
>>>> index 639397b5491c..e7aa70d33530 100644
>>>> --- a/kernel/static_call_inline.c
>>>> +++ b/kernel/static_call_inline.c
>>>> @@ -256,8 +256,10 @@ static int __static_call_init(struct module *mod,
>>>>                }
>>>>                  site_mod = kzalloc(sizeof(*site_mod), GFP_KERNEL);
>>>> -            if (!site_mod)
>>>> +            if (!site_mod) {
>>>> +                key->mods = NULL;
>>>>                    return -ENOMEM;
>>>> +            }
>>>>                  /*
>>>>                 * When the key has a direct sites pointer, extract
>>>> @@ -422,7 +424,7 @@ static void static_call_del_module(struct module
>>>> *mod)
>>>>                ;
>>>>              if (!site_mod)
>>>> -            continue;
>>>> +            break;
>>>>              *prev = site_mod->next;
>>>>            kfree(site_mod);
>>

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Fix a wild-memory-access bug in static_call_del_module()
  2024-09-02 12:01 ` Jinjie Ruan
  2024-09-02 12:06   ` Christophe Leroy
@ 2024-09-03 20:01   ` Thomas Gleixner
  2024-09-03 22:58     ` [PATCH] static_call: Handle module init failure correctly " Thomas Gleixner
  1 sibling, 1 reply; 29+ messages in thread
From: Thomas Gleixner @ 2024-09-03 20:01 UTC (permalink / raw)
  To: Jinjie Ruan, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf

On Mon, Sep 02 2024 at 20:01, Jinjie Ruan wrote:
> On 2023/9/15 16:21, Jinjie Ruan wrote:
>> 
>> diff --git a/kernel/static_call_inline.c b/kernel/static_call_inline.c
>> index 639397b5491c..e7aa70d33530 100644
>> --- a/kernel/static_call_inline.c
>> +++ b/kernel/static_call_inline.c
>> @@ -256,8 +256,10 @@ static int __static_call_init(struct module *mod,
>>  			}
>>  
>>  			site_mod = kzalloc(sizeof(*site_mod), GFP_KERNEL);
>> -			if (!site_mod)
>> +			if (!site_mod) {
>> +				key->mods = NULL;
>>  				return -ENOMEM;

You fail to explain what setting key->mods to NULL prevents and why the
same problem does not happen when the kzalloc() five lines further down
fails.

Actually if you read the comment below carefully then you might find the
reason why the current code explodes and you might figure out why you
can't set the pointer to NULL.

>> +			}
>>  
>>  			/*
>>  			 * When the key has a direct sites pointer, extract
>> @@ -422,7 +424,7 @@ static void static_call_del_module(struct module *mod)
>>  			;
>>  
>>  		if (!site_mod)
>> -			continue;
>> +			break;

Now why does this "fix" things?

The unmodified kernel crashes dereferencing key->mods. That means there
is garbage in key->mods. But that garbage is not garbage:

struct static_call_key {
	void *func;
	union {
		/* bit 0: 0 = mods, 1 = sites */
		unsigned long type;
		struct static_call_mod *mods;
		struct static_call_site *sites;
	};
};

key->mods is actually key->sites, which holds the static call sites of
the key. It has bit 0 set.

As that code does not expect anything else than a valid static_call_mod
pointer it dereferences it unconditionally and crashes. See?

So why can't you set the pointer to NULL?

If that key is builtin and has actual builtin usage sites, then you
destroy the key. If the memory allocation fail was temporary then a
subsequent update of that static call will find no call site and the
kernel happily invokes the previous function.

Now that the root cause is properly analyzed, the proper fix is
obvious. See uncomplied below.

It does not need to do anything vs. the other kzalloc() fail because
that is harmless. The key has key->mods set, but key->mods->mod and
key->mods->next are both NULL in that case. So the inner loop breaks out
and continues to the next site which has the uninitialized one. If all
keys were already converted to mods _before_ this fail then the loop
simply iterates further, but will nowhere find a site_mod->mod == mod.

Thanks,

        tglx
---
--- a/kernel/static_call_inline.c
+++ b/kernel/static_call_inline.c
@@ -411,6 +411,17 @@ static void static_call_del_module(struc

 	for (site = start; site < stop; site++) {
 		key = static_call_key(site);
+
+		/*
+		 * If the key was not updated due to a memory allocation
+		 * failure in __static_call_init() then treating key::sites
+		 * as key::mods in the code below would cause random memory
+		 * access and #GP. In that case all subsequent sites have
+		 * not been touched either, so stop iterating.
+		 */
+		if (static_call_key_sites(key))
+			break;
+
 		if (key == prev_key)
 			continue;

^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-03 20:01   ` Thomas Gleixner
@ 2024-09-03 22:58     ` Thomas Gleixner
  2024-09-04  3:32       ` Jinjie Ruan
  0 siblings, 1 reply; 29+ messages in thread
From: Thomas Gleixner @ 2024-09-03 22:58 UTC (permalink / raw)
  To: Jinjie Ruan, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf

Module insertion invokes static_call_add_module() to initialize the static
calls in a module. static_call_add_module() invokes __static_call_init(),
which allocates a struct static_call_mod to either encapsulate the built-in
static call sites of the associated key into it so further modules can be
added or to append the module to the module chain.

If that allocation fails the function returns with an error code and the
module core invokes static_call_del_module() to clean up eventually added
static_call_mod entries.

This works correctly, when all keys used by the module were converted over
to a module chain before the failure. If not then static_call_del_module()
causes a #GP as it blindly assumes that key::mods points to a valid struct
static_call_mod.

The problem is that key::mods is not a individual struct member of struct
static_call_key, it's part of a union to save space:

        union {
                /* bit 0: 0 = mods, 1 = sites */
                unsigned long type;
                struct static_call_mod *mods;
                struct static_call_site *sites;
	};

key::sites is a pointer to the list of built-in usage sites of the static
call. The type of the pointer is differentiated by bit 0. A mods pointer
has the bit clear, the sites pointer has the bit set.

As static_call_del_module() blindly assumes that the pointer is a valid
static_call_mod type, it fails to check for this failure case and
dereferences the pointer to the list of built-in call sites, which is
obviously bogus.

Cure it by checking whether the key has a sites or a mods pointer. 

If it is a sites pointer then the key is not to be touched. As the sites
are walked in the same order as in __static_call_init() the site walk
can be terminated because all subsequent sites have not been touched by
the init code due to the error exit.

If it was converted before the allocation fail, then the inner loop which
searches for a module match will find nothing.

A fail in the second allocation in __static_call_init() is harmless and
does not require special treatment. The first allocation succeeded and
converted the key to a module chain. That first entry has mod::mod == NULL
and mod::next == NULL, so the inner loop of static_call_del_module() will
neither find a module match nor a module chain. The next site in the walk
was either already converted, but can't match the module, or it will exit
the outer loop because it has a static_call_site pointer and not a
static_call_mod pointer.

Fixes: 9183c3f9ed71 ("static_call: Add inline static call infrastructure")
Reported-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: stable@vger.kernel.org
Closes: https://lore.kernel.org/all/20230915082126.4187913-1-ruanjinjie@huawei.com
---
 kernel/static_call_inline.c |   11 +++++++++++
 1 file changed, 11 insertions(+)

--- a/kernel/static_call_inline.c
+++ b/kernel/static_call_inline.c
@@ -411,6 +411,17 @@ static void static_call_del_module(struc

 	for (site = start; site < stop; site++) {
 		key = static_call_key(site);
+
+		/*
+		 * If the key was not updated due to a memory allocation
+		 * failure in __static_call_init() then treating key::sites
+		 * as key::mods in the code below would cause random memory
+		 * access and #GP. In that case all subsequent sites have
+		 * not been touched either, so stop iterating.
+		 */
+		if (static_call_key_sites(key))
+			break;
+
 		if (key == prev_key)
 			continue;

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-03 22:58     ` [PATCH] static_call: Handle module init failure correctly " Thomas Gleixner
@ 2024-09-04  3:32       ` Jinjie Ruan
  2024-09-04  7:08         ` Thomas Gleixner
  0 siblings, 1 reply; 29+ messages in thread
From: Jinjie Ruan @ 2024-09-04  3:32 UTC (permalink / raw)
  To: Thomas Gleixner, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf



On 2024/9/4 6:58, Thomas Gleixner wrote:
> Module insertion invokes static_call_add_module() to initialize the static
> calls in a module. static_call_add_module() invokes __static_call_init(),
> which allocates a struct static_call_mod to either encapsulate the built-in
> static call sites of the associated key into it so further modules can be
> added or to append the module to the module chain.
> 
> If that allocation fails the function returns with an error code and the
> module core invokes static_call_del_module() to clean up eventually added
> static_call_mod entries.
> 
> This works correctly, when all keys used by the module were converted over
> to a module chain before the failure. If not then static_call_del_module()
> causes a #GP as it blindly assumes that key::mods points to a valid struct
> static_call_mod.
> 
> The problem is that key::mods is not a individual struct member of struct
> static_call_key, it's part of a union to save space:
> 
>         union {
>                 /* bit 0: 0 = mods, 1 = sites */
>                 unsigned long type;
>                 struct static_call_mod *mods;
>                 struct static_call_site *sites;
> 	};
> 
> key::sites is a pointer to the list of built-in usage sites of the static
> call. The type of the pointer is differentiated by bit 0. A mods pointer
> has the bit clear, the sites pointer has the bit set.
> 
> As static_call_del_module() blindly assumes that the pointer is a valid
> static_call_mod type, it fails to check for this failure case and
> dereferences the pointer to the list of built-in call sites, which is
> obviously bogus.
> 
> Cure it by checking whether the key has a sites or a mods pointer. 
> 
> If it is a sites pointer then the key is not to be touched. As the sites
> are walked in the same order as in __static_call_init() the site walk
> can be terminated because all subsequent sites have not been touched by
> the init code due to the error exit.
> 
> If it was converted before the allocation fail, then the inner loop which
> searches for a module match will find nothing.
> 
> A fail in the second allocation in __static_call_init() is harmless and
> does not require special treatment. The first allocation succeeded and
> converted the key to a module chain. That first entry has mod::mod == NULL
> and mod::next == NULL, so the inner loop of static_call_del_module() will
> neither find a module match nor a module chain. The next site in the walk
> was either already converted, but can't match the module, or it will exit
> the outer loop because it has a static_call_site pointer and not a
> static_call_mod pointer.
> 
> Fixes: 9183c3f9ed71 ("static_call: Add inline static call infrastructure")
> Reported-by: Jinjie Ruan <ruanjinjie@huawei.com>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: stable@vger.kernel.org
> Closes: https://lore.kernel.org/all/20230915082126.4187913-1-ruanjinjie@huawei.com
> ---
>  kernel/static_call_inline.c |   11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> --- a/kernel/static_call_inline.c
> +++ b/kernel/static_call_inline.c
> @@ -411,6 +411,17 @@ static void static_call_del_module(struc
>  
>  	for (site = start; site < stop; site++) {
>  		key = static_call_key(site);
> +
> +		/*
> +		 * If the key was not updated due to a memory allocation
> +		 * failure in __static_call_init() then treating key::sites
> +		 * as key::mods in the code below would cause random memory
> +		 * access and #GP. In that case all subsequent sites have
> +		 * not been touched either, so stop iterating.
> +		 */
> +		if (static_call_key_sites(key))
> +			break;
> +

Hi, Thomas,

This patch seems not solve the issue, with this patch, the below problem
also occurs when inject fault when modprobe amdgpu:

[  700.274277] name failslab, interval 1, probability 0, space 0, times 0
[  700.275147] CPU: 6 UID: 0 PID: 9846 Comm: modprobe Tainted: G
W          6.11.0-rc6+ #48
[  700.275662] Tainted: [W]=WARN
[  700.275856] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.15.0-1 04/01/2014
[  700.276332] Call Trace:
[  700.276479]  <TASK>
[  700.276614]  dump_stack_lvl+0xba/0xe0
[  700.276846]  should_fail_ex+0x4b1/0x5b0
[  700.277083]  should_failslab+0xc2/0x120
[  700.277315]  __kmalloc_cache_noprof+0x6b/0x350
[  700.277590]  ? __static_call_init.part.0+0x158/0x5d0
[  700.277893]  __static_call_init.part.0+0x158/0x5d0
[  700.278185]  ? amdgpu_gmc_fw_reg_write_reg_wait+0x3ad/0x430 [amdgpu]
[  700.279871]  static_call_module_notify+0x9b/0x3d0
[  700.280158]  notifier_call_chain+0xad/0x2f0
[  700.280407]  blocking_notifier_call_chain_robust+0xc8/0x150
[  700.280739]  ? __pfx_blocking_notifier_call_chain_robust+0x10/0x10
[  700.281104]  load_module+0x5026/0x63e0
[  700.281323]  ? generic_file_read_iter+0x269/0x3a0
[  700.281629]  ? __pfx_load_module+0x10/0x10
[  700.281896]  ? selinux_file_permission+0x118/0x4b0
[  700.282176]  ? security_kernel_post_read_file+0x8d/0xc0
[  700.282484]  ? kernel_read_file+0x3e7/0x660
[  700.282743]  ? __pfx_kernel_read_file+0x10/0x10
[  700.283009]  ? __pfx___lock_acquire+0x10/0x10
[  700.283272]  ? lock_acquire+0x19d/0x530
[  700.283500]  ? init_module_from_file+0xe6/0x150
[  700.283756]  init_module_from_file+0xe6/0x150
[  700.284011]  ? __pfx_init_module_from_file+0x10/0x10
[  700.284308]  idempotent_init_module+0x37c/0x690
[  700.284569]  ? __pfx_idempotent_init_module+0x10/0x10
[  700.284860]  ? __startup_64+0x190/0x330
[  700.285082]  ? security_capable+0x8d/0xc0
[  700.285316]  __x64_sys_finit_module+0xcc/0x130
[  700.285569]  do_syscall_64+0xc1/0x1d0
[  700.285783]  entry_SYSCALL_64_after_hwframe+0x77/0x7f
[  700.286070] RIP: 0033:0x7efe8bfb6839
[  700.286274] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00
48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f
05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48
[  700.287266] RSP: 002b:00007ffca29f00b8 EFLAGS: 00000246 ORIG_RAX:
0000000000000139
[  700.287675] RAX: ffffffffffffffda RBX: 0000561a219bbca0 RCX:
00007efe8bfb6839
[  700.288064] RDX: 0000000000000000 RSI: 0000561a1dc1bc2e RDI:
0000000000000003
[  700.288447] RBP: 0000561a1dc1bc2e R08: 0000000000000000 R09:
0000561a219bbca0
[  700.288833] R10: 0000000000000003 R11: 0000000000000246 R12:
0000000000000000
[  700.289219] R13: 0000561a219bbe90 R14: 0000000000040000 R15:
0000561a219bbca0
[  700.289613]  </TASK>
[  700.289766] ------------[ cut here ]------------
[  700.290023] Failed to allocate memory for static calls
[  700.290048] WARNING: CPU: 6 PID: 9846 at
kernel/static_call_inline.c:456 static_call_module_notify+0x2d5/0x3d0
[  700.290899] Modules linked in: amdgpu(+) amdxcp drm_ttm_helper
drm_suballoc_helper [last unloaded: amdgpu]
[  700.291431] CPU: 6 UID: 0 PID: 9846 Comm: modprobe Tainted: G
W          6.11.0-rc6+ #48
[  700.291929] Tainted: [W]=WARN
[  700.292106] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.15.0-1 04/01/2014
[  700.292558] RIP: 0010:static_call_module_notify+0x2d5/0x3d0
[  700.292888] Code: 07 48 c7 c7 c0 94 ac 9d 41 bc 17 80 00 00 49 8d 34
07 e8 4e 2d d5 ff e8 d9 56 f1 ff 90 48 c7 c7 20 95 ac 9d e8 cc 3f bd ff
90 <0f> 0b 90 90 48 8b 7c 24 08 e8 6d ef ff ff 48 c7 c7 40 e6 a9 9e e8
[  700.293906] RSP: 0018:ff11000105437868 EFLAGS: 00010282
[  700.294207] RAX: 0000000000000000 RBX: 00000000fffffff4 RCX:
ffffffff9a16a61d
[  700.294595] RDX: ff1100010dad8000 RSI: 0000000000000004 RDI:
ff1100011ad28a08
[  700.295003] RBP: 00000000dcc77697 R08: 0000000000000001 R09:
ffe21c00235a5141
[  700.295388] R10: ff1100011ad28a0b R11: 0000000000000000 R12:
000000000000800d
[  700.295793] R13: dffffc0000000000 R14: ffffffff9e5f1808 R15:
ffffffffc1c9e325
[  700.296183] FS:  00007efe8c4a5540(0000) GS:ff1100011ad00000(0000)
knlGS:0000000000000000
[  700.296630] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  700.296966] CR2: 000055ec0729e838 CR3: 0000000114d16003 CR4:
0000000000771ef0
[  700.297366] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  700.297771] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[  700.298162] PKRU: 55555554
[  700.298314] Call Trace:
[  700.298461]  <TASK>
[  700.298589]  ? show_regs+0x73/0x80
[  700.298816]  ? __warn+0xe5/0x330
[  700.299010]  ? static_call_module_notify+0x2d5/0x3d0
[  700.299290]  ? report_bug+0x2a3/0x380
[  700.299501]  ? static_call_module_notify+0x2d5/0x3d0
[  700.299794]  ? handle_bug+0x5e/0xc0
[  700.299999]  ? exc_invalid_op+0x25/0x60
[  700.300222]  ? asm_exc_invalid_op+0x1a/0x20
[  700.300466]  ? __warn_printk+0x15d/0x200
[  700.300707]  ? static_call_module_notify+0x2d5/0x3d0
[  700.300991]  ? static_call_module_notify+0x2d4/0x3d0
[  700.301270]  notifier_call_chain+0xad/0x2f0
[  700.301521]  blocking_notifier_call_chain_robust+0xc8/0x150
[  700.301848]  ? __pfx_blocking_notifier_call_chain_robust+0x10/0x10
[  700.302219]  load_module+0x5026/0x63e0
[  700.302437]  ? generic_file_read_iter+0x269/0x3a0
[  700.302747]  ? __pfx_load_module+0x10/0x10
[  700.302989]  ? selinux_file_permission+0x118/0x4b0
[  700.303268]  ? security_kernel_post_read_file+0x8d/0xc0
[  700.303559]  ? kernel_read_file+0x3e7/0x660
[  700.303812]  ? __pfx_kernel_read_file+0x10/0x10
[  700.304066]  ? __pfx___lock_acquire+0x10/0x10
[  700.304316]  ? lock_acquire+0x19d/0x530
[  700.304543]  ? init_module_from_file+0xe6/0x150
[  700.304823]  init_module_from_file+0xe6/0x150
[  700.305077]  ? __pfx_init_module_from_file+0x10/0x10
[  700.305370]  idempotent_init_module+0x37c/0x690
[  700.305631]  ? __pfx_idempotent_init_module+0x10/0x10
[  700.305936]  ? __startup_64+0x190/0x330
[  700.306163]  ? security_capable+0x8d/0xc0
[  700.306395]  __x64_sys_finit_module+0xcc/0x130
[  700.306654]  do_syscall_64+0xc1/0x1d0
[  700.306891]  entry_SYSCALL_64_after_hwframe+0x77/0x7f
[  700.307180] RIP: 0033:0x7efe8bfb6839
[  700.307387] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00
48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f
05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48
[  700.308395] RSP: 002b:00007ffca29f00b8 EFLAGS: 00000246 ORIG_RAX:
0000000000000139
[  700.308828] RAX: ffffffffffffffda RBX: 0000561a219bbca0 RCX:
00007efe8bfb6839
[  700.309221] RDX: 0000000000000000 RSI: 0000561a1dc1bc2e RDI:
0000000000000003
[  700.309619] RBP: 0000561a1dc1bc2e R08: 0000000000000000 R09:
0000561a219bbca0
[  700.310034] R10: 0000000000000003 R11: 0000000000000246 R12:
0000000000000000
[  700.310434] R13: 0000561a219bbe90 R14: 0000000000040000 R15:
0000561a219bbca0
[  700.310867]  </TASK>
[  700.311002] irq event stamp: 65841
[  700.311200] hardirqs last  enabled at (65851): [<ffffffff9a2ef5b1>]
console_unlock+0x1b1/0x1c0
[  700.311678] hardirqs last disabled at (65860): [<ffffffff9a2ef596>]
console_unlock+0x196/0x1c0
[  700.312175] softirqs last  enabled at (65826): [<ffffffff9a184c32>]
handle_softirqs+0x512/0x770
[  700.312658] softirqs last disabled at (65805): [<ffffffff9a185824>]
irq_exit_rcu+0x94/0xc0
[  700.313130] ---[ end trace 0000000000000000 ]---
[  700.313418] Oops: general protection fault, probably for
non-canonical address 0xdffffc0000000001: 0000 [#1] PREEMPT SMP KASAN NOPTI
[  700.314079] KASAN: null-ptr-deref in range
[0x0000000000000008-0x000000000000000f]
[  700.314505] CPU: 6 UID: 0 PID: 9846 Comm: modprobe Tainted: G
W          6.11.0-rc6+ #48
[  700.314994] Tainted: [W]=WARN
[  700.315169] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.15.0-1 04/01/2014
[  700.315627] RIP: 0010:static_call_del_module+0x145/0x210
[  700.315935] Code: 66 f1 ff 48 85 db 75 12 eb 5c e8 26 66 f1 ff 4d 85
e4 74 52 49 89 de 4c 89 e3 e8 16 66 f1 ff 48 8d 7b 08 48 89 f8 48 c1 e8
03 <80> 3c 28 00 75 6e 48 89 d8 48 8b 53 08 48 c1 e8 03 80 3c 28 00 75
[  700.317483] RSP: 0018:ff11000105437818 EFLAGS: 00010202
[  700.317928] RAX: 0000000000000001 RBX: 0000000000000001 RCX:
ffffffff9a5955aa
[  700.318377] RDX: ff1100010dad8000 RSI: 0000000000000004 RDI:
0000000000000009
[  700.318845] RBP: dffffc0000000000 R08: 0000000000000001 R09:
ffe21c00235a5141
[  700.319363] R10: ff1100011ad28a0b R11: 0000000000000000 R12:
ffffffffc1ad3cc0
[  700.319810] R13: ffffffffc1c9e1d1 R14: ffffffffc1ad3cc8 R15:
ffffffffc1c9e340
[  700.320279] FS:  00007efe8c4a5540(0000) GS:ff1100011ad00000(0000)
knlGS:0000000000000000
[  700.320800] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  700.321189] CR2: 000055ec0729e838 CR3: 0000000114d16003 CR4:
0000000000771ef0
[  700.321638] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  700.322097] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[  700.322918] PKRU: 55555554
[  700.323389] Call Trace:
[  700.323796]  <TASK>
[  700.324168]  ? show_regs+0x73/0x80
[  700.324780]  ? die_addr+0x46/0xc0
[  700.325358]  ? exc_general_protection+0x161/0x2a0
[  700.326130]  ? asm_exc_general_protection+0x26/0x30
[  700.326888]  ? static_call_del_module+0x13a/0x210
[  700.327584]  ? static_call_del_module+0x145/0x210
[  700.328237]  ? static_call_del_module+0x13a/0x210
[  700.328910]  static_call_module_notify+0x2e3/0x3d0
[  700.329590]  notifier_call_chain+0xad/0x2f0
[  700.330179]  blocking_notifier_call_chain_robust+0xc8/0x150
[  700.330776]  ? __pfx_blocking_notifier_call_chain_robust+0x10/0x10
[  700.331443]  load_module+0x5026/0x63e0
[  700.331871]  ? generic_file_read_iter+0x269/0x3a0
[  700.332516]  ? __pfx_load_module+0x10/0x10
[  700.333060]  ? selinux_file_permission+0x118/0x4b0
[  700.333723]  ? security_kernel_post_read_file+0x8d/0xc0
[  700.334470]  ? kernel_read_file+0x3e7/0x660
[  700.335041]  ? __pfx_kernel_read_file+0x10/0x10
[  700.335442]  ? __pfx___lock_acquire+0x10/0x10
[  700.335992]  ? lock_acquire+0x19d/0x530
[  700.336959]  ? init_module_from_file+0xe6/0x150
[  700.338077]  init_module_from_file+0xe6/0x150
[  700.339172]  ? __pfx_init_module_from_file+0x10/0x10
[  700.340397]  idempotent_init_module+0x37c/0x690
[  700.341521]  ? __pfx_idempotent_init_module+0x10/0x10
[  700.342735]  ? __startup_64+0x190/0x330
[  700.343524]  ? security_capable+0x8d/0xc0
[  700.344082]  __x64_sys_finit_module+0xcc/0x130
[  700.345035]  do_syscall_64+0xc1/0x1d0
[  700.345813]  entry_SYSCALL_64_after_hwframe+0x77/0x7f
[  700.346864] RIP: 0033:0x7efe8bfb6839
[  700.347581] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00
48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f
05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48
[  700.351338] RSP: 002b:00007ffca29f00b8 EFLAGS: 00000246 ORIG_RAX:
0000000000000139
[  700.352525] RAX: ffffffffffffffda RBX: 0000561a219bbca0 RCX:
00007efe8bfb6839
[  700.353354] RDX: 0000000000000000 RSI: 0000561a1dc1bc2e RDI:
0000000000000003
[  700.354195] RBP: 0000561a1dc1bc2e R08: 0000000000000000 R09:
0000561a219bbca0
[  700.355019] R10: 0000000000000003 R11: 0000000000000246 R12:
0000000000000000
[  700.355702] R13: 0000561a219bbe90 R14: 0000000000040000 R15:
0000561a219bbca0
[  700.356453]  </TASK>
[  700.356720] Modules linked in: amdgpu(+) amdxcp drm_ttm_helper
drm_suballoc_helper [last unloaded: amdgpu]
[  700.357844] Dumping ftrace buffer:
[  700.358243]    (ftrace buffer empty)
[  700.358721] ---[ end trace 0000000000000000 ]---
[  700.359241] RIP: 0010:static_call_del_module+0x145/0x210
[  700.359835] Code: 66 f1 ff 48 85 db 75 12 eb 5c e8 26 66 f1 ff 4d 85
e4 74 52 49 89 de 4c 89 e3 e8 16 66 f1 ff 48 8d 7b 08 48 89 f8 48 c1 e8
03 <80> 3c 28 00 75 6e 48 89 d8 48 8b 53 08 48 c1 e8 03 80 3c 28 00 75
[  700.361913] RSP: 0018:ff11000105437818 EFLAGS: 00010202
[  700.362603] RAX: 0000000000000001 RBX: 0000000000000001 RCX:
ffffffff9a5955aa
[  700.363374] RDX: ff1100010dad8000 RSI: 0000000000000004 RDI:
0000000000000009
[  700.364114] RBP: dffffc0000000000 R08: 0000000000000001 R09:
ffe21c00235a5141
[  700.365014] R10: ff1100011ad28a0b R11: 0000000000000000 R12:
ffffffffc1ad3cc0
[  700.365784] R13: ffffffffc1c9e1d1 R14: ffffffffc1ad3cc8 R15:
ffffffffc1c9e340
[  700.366458] FS:  00007efe8c4a5540(0000) GS:ff1100011ad00000(0000)
knlGS:0000000000000000
[  700.367325] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  700.368062] CR2: 000055ec0729e838 CR3: 0000000114d16003 CR4:
0000000000771ef0
[  700.368851] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  700.370439] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[  700.371814] PKRU: 55555554
[  700.372292] Kernel panic - not syncing: Fatal exception
[  700.373772] Dumping ftrace buffer:
[  700.374235]    (ftrace buffer empty)
[  700.374707] Kernel Offset: 0x19000000 from 0xffffffff81000000
(relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[  700.376056] Rebooting in 1 seconds..


>  		if (key == prev_key)
>  			continue;
>  

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-04  3:32       ` Jinjie Ruan
@ 2024-09-04  7:08         ` Thomas Gleixner
  2024-09-04  8:00           ` Thomas Gleixner
  2024-09-04  8:03           ` [PATCH] static_call: Handle module init failure correctly in static_call_del_module() Jinjie Ruan
  0 siblings, 2 replies; 29+ messages in thread
From: Thomas Gleixner @ 2024-09-04  7:08 UTC (permalink / raw)
  To: Jinjie Ruan, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf

On Wed, Sep 04 2024 at 11:32, Jinjie Ruan wrote:
> On 2024/9/4 6:58, Thomas Gleixner wrote:
>> +		/*
>> +		 * If the key was not updated due to a memory allocation
>> +		 * failure in __static_call_init() then treating key::sites
>> +		 * as key::mods in the code below would cause random memory
>> +		 * access and #GP. In that case all subsequent sites have
>> +		 * not been touched either, so stop iterating.
>> +		 */
>> +		if (static_call_key_sites(key))
>> +			break;
>> +
>
> Hi, Thomas,
>
> This patch seems not solve the issue, with this patch, the below problem
> also occurs when inject fault when modprobe amdgpu:

That's a different problem.

 Oops: general protection fault, probably for
 non-canonical address 0xdffffc0000000001: 0000 [#1] PREEMPT SMP KASAN NOPTI

It's dereferencing a NULL pointer at 0x1. That's odd because bit 0 is
set, which looks like a sites pointer. So static_call_key_sites() should
return true, but obviously does not. So how does that happen?

It can't be a built-in key, so it's a module local one with key::sites
== 0x1. So static_call_key_sites() sees bit 0 set, and then returns
key::sites & ~0x01, which is obviously NULL. So the condition is false
and the code below uses key::mods == 0x1....

So the check must be:

	if (!static_call_key_has_mods(key))
        	break;

I missed the module local case completely in my analysis. Can you please
modify the condition and retest?

Thanks,

        tglx






^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-04  7:08         ` Thomas Gleixner
@ 2024-09-04  8:00           ` Thomas Gleixner
  2024-09-04  8:17             ` Peter Zijlstra
  2024-09-04  8:03           ` [PATCH] static_call: Handle module init failure correctly in static_call_del_module() Jinjie Ruan
  1 sibling, 1 reply; 29+ messages in thread
From: Thomas Gleixner @ 2024-09-04  8:00 UTC (permalink / raw)
  To: Jinjie Ruan, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf

On Wed, Sep 04 2024 at 09:08, Thomas Gleixner wrote:
> On Wed, Sep 04 2024 at 11:32, Jinjie Ruan wrote:
> So the check must be:
>
> 	if (!static_call_key_has_mods(key))
>         	break;
>
> I missed the module local case completely in my analysis. Can you please
> modify the condition and retest?

That said. This code is pointlessly noisy for the failure case.

Allocation fails are not a reason to warn about. -ENOMEM is propagated
all the way to the caller, so it's sufficient to emit a pr_warn().

Peter?

Thanks,

        tglx
---
--- a/kernel/static_call_inline.c
+++ b/kernel/static_call_inline.c
@@ -453,7 +453,7 @@ static int static_call_module_notify(str
 	case MODULE_STATE_COMING:
 		ret = static_call_add_module(mod);
 		if (ret) {
-			WARN(1, "Failed to allocate memory for static calls");
+			pr_warn("Failed to allocate memory for static calls\n");
 			static_call_del_module(mod);
 		}
 		break;

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-04  8:00           ` Thomas Gleixner
@ 2024-09-04  8:17             ` Peter Zijlstra
  2024-09-04  9:08               ` [PATCH] static_call: Replace pointless WARN_ON() in static_call_module_notify() Thomas Gleixner
  0 siblings, 1 reply; 29+ messages in thread
From: Peter Zijlstra @ 2024-09-04  8:17 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Jinjie Ruan, linux-kernel, Christophe Leroy, Josh Poimboeuf

On Wed, Sep 04, 2024 at 10:00:52AM +0200, Thomas Gleixner wrote:
> On Wed, Sep 04 2024 at 09:08, Thomas Gleixner wrote:
> > On Wed, Sep 04 2024 at 11:32, Jinjie Ruan wrote:
> > So the check must be:
> >
> > 	if (!static_call_key_has_mods(key))
> >         	break;
> >
> > I missed the module local case completely in my analysis. Can you please
> > modify the condition and retest?
> 
> That said. This code is pointlessly noisy for the failure case.
> 
> Allocation fails are not a reason to warn about. -ENOMEM is propagated
> all the way to the caller, so it's sufficient to emit a pr_warn().
> 
> Peter?

Yeah, I think that should do.

> Thanks,
> 
>         tglx
> ---
> --- a/kernel/static_call_inline.c
> +++ b/kernel/static_call_inline.c
> @@ -453,7 +453,7 @@ static int static_call_module_notify(str
>  	case MODULE_STATE_COMING:
>  		ret = static_call_add_module(mod);
>  		if (ret) {
> -			WARN(1, "Failed to allocate memory for static calls");
> +			pr_warn("Failed to allocate memory for static calls\n");
>  			static_call_del_module(mod);
>  		}
>  		break;

^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PATCH] static_call: Replace pointless WARN_ON() in static_call_module_notify()
  2024-09-04  8:17             ` Peter Zijlstra
@ 2024-09-04  9:08               ` Thomas Gleixner
  2024-09-06 14:41                 ` [tip: locking/urgent] " tip-bot2 for Thomas Gleixner
  0 siblings, 1 reply; 29+ messages in thread
From: Thomas Gleixner @ 2024-09-04  9:08 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Jinjie Ruan, linux-kernel, Christophe Leroy, Josh Poimboeuf

static_call_module_notify() triggers a WARN_ON(), when memory allocation
fails in __static_call_add_module().

That's not really justified, because the failure case must be correctly
handled by the well known call chain and the error code is passed
through to the initiating userspace application.

A memory allocation fail is not a fatal problem, but the WARN_ON() takes
the machine out when panic_on_warn is set.

Replace it with a pr_warn().

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/static_call_inline.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/kernel/static_call_inline.c
+++ b/kernel/static_call_inline.c
@@ -453,7 +453,7 @@ static int static_call_module_notify(str
 	case MODULE_STATE_COMING:
 		ret = static_call_add_module(mod);
 		if (ret) {
-			WARN(1, "Failed to allocate memory for static calls");
+			pr_warn("Failed to allocate memory for static calls\n");
 			static_call_del_module(mod);
 		}
 		break;

^ permalink raw reply	[flat|nested] 29+ messages in thread

* [tip: locking/urgent] static_call: Replace pointless WARN_ON() in static_call_module_notify()
  2024-09-04  9:08               ` [PATCH] static_call: Replace pointless WARN_ON() in static_call_module_notify() Thomas Gleixner
@ 2024-09-06 14:41                 ` tip-bot2 for Thomas Gleixner
  0 siblings, 0 replies; 29+ messages in thread
From: tip-bot2 for Thomas Gleixner @ 2024-09-06 14:41 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: Thomas Gleixner, Peter Zijlstra (Intel), x86, linux-kernel

The following commit has been merged into the locking/urgent branch of tip:

Commit-ID:     fe513c2ef0a172a58f158e2e70465c4317f0a9a2
Gitweb:        https://git.kernel.org/tip/fe513c2ef0a172a58f158e2e70465c4317f0a9a2
Author:        Thomas Gleixner <tglx@linutronix.de>
AuthorDate:    Wed, 04 Sep 2024 11:08:28 +02:00
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Fri, 06 Sep 2024 16:29:22 +02:00

static_call: Replace pointless WARN_ON() in static_call_module_notify()

static_call_module_notify() triggers a WARN_ON(), when memory allocation
fails in __static_call_add_module().

That's not really justified, because the failure case must be correctly
handled by the well known call chain and the error code is passed
through to the initiating userspace application.

A memory allocation fail is not a fatal problem, but the WARN_ON() takes
the machine out when panic_on_warn is set.

Replace it with a pr_warn().

Fixes: 9183c3f9ed71 ("static_call: Add inline static call infrastructure")
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/8734mf7pmb.ffs@tglx
---
 kernel/static_call_inline.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/static_call_inline.c b/kernel/static_call_inline.c
index 7bb0962..5259cda 100644
--- a/kernel/static_call_inline.c
+++ b/kernel/static_call_inline.c
@@ -453,7 +453,7 @@ static int static_call_module_notify(struct notifier_block *nb,
 	case MODULE_STATE_COMING:
 		ret = static_call_add_module(mod);
 		if (ret) {
-			WARN(1, "Failed to allocate memory for static calls");
+			pr_warn("Failed to allocate memory for static calls\n");
 			static_call_del_module(mod);
 		}
 		break;

^ permalink raw reply related	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-04  7:08         ` Thomas Gleixner
  2024-09-04  8:00           ` Thomas Gleixner
@ 2024-09-04  8:03           ` Jinjie Ruan
  2024-09-04  8:51             ` Thomas Gleixner
  2024-09-04  9:09             ` [PATCH v2] " Thomas Gleixner
  1 sibling, 2 replies; 29+ messages in thread
From: Jinjie Ruan @ 2024-09-04  8:03 UTC (permalink / raw)
  To: Thomas Gleixner, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf



On 2024/9/4 15:08, Thomas Gleixner wrote:
> On Wed, Sep 04 2024 at 11:32, Jinjie Ruan wrote:
>> On 2024/9/4 6:58, Thomas Gleixner wrote:
>>> +		/*
>>> +		 * If the key was not updated due to a memory allocation
>>> +		 * failure in __static_call_init() then treating key::sites
>>> +		 * as key::mods in the code below would cause random memory
>>> +		 * access and #GP. In that case all subsequent sites have
>>> +		 * not been touched either, so stop iterating.
>>> +		 */
>>> +		if (static_call_key_sites(key))
>>> +			break;
>>> +
>>
>> Hi, Thomas,
>>
>> This patch seems not solve the issue, with this patch, the below problem
>> also occurs when inject fault when modprobe amdgpu:
> 
> That's a different problem.
> 
>  Oops: general protection fault, probably for
>  non-canonical address 0xdffffc0000000001: 0000 [#1] PREEMPT SMP KASAN NOPTI
> 
> It's dereferencing a NULL pointer at 0x1. That's odd because bit 0 is
> set, which looks like a sites pointer. So static_call_key_sites() should
> return true, but obviously does not. So how does that happen?
> 
> It can't be a built-in key, so it's a module local one with key::sites
> == 0x1. So static_call_key_sites() sees bit 0 set, and then returns
> key::sites & ~0x01, which is obviously NULL. So the condition is false
> and the code below uses key::mods == 0x1....
> 
> So the check must be:
> 
> 	if (!static_call_key_has_mods(key))
>         	break;

Hi, Thomas,

with this patch, the issue not occurs again，

but there are some memory leak here same to the following problem:

Link:
https://lore.kernel.org/all/20221112114602.1268989-4-liushixin2@huawei.com/

unreferenced object 0xff11000104078480 (size 32):
  comm "modprobe", pid 16978, jiffies 4295928553
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 07 04 01 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc 86f0f2f0):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000432b600 (size 32):
  comm "modprobe", pid 16994, jiffies 4295929985
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 32 04 00 00 11 ff 00 00 00 00 00 00 00 00  y.2.............
  backtrace (crc 32209446):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000115b24ac0 (size 32):
  comm "modprobe", pid 17010, jiffies 4295931422
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 b2 15 01 00 11 ff 02 00 00 00 01 00 00 00  y...............
  backtrace (crc b9fe288c):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100010196bc40 (size 32):
  comm "modprobe", pid 17032, jiffies 4295933559
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 96 01 01 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc e311731b):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000009b4ee40 (size 32):
  comm "modprobe", pid 17038, jiffies 4295934300
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 b4 09 00 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc ebd506c):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000a3fe300 (size 32):
  comm "modprobe", pid 17044, jiffies 4295935011
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 3f 0a 00 00 11 ff 00 00 00 fc ff df 41 57  y.?...........AW
  backtrace (crc b6459b1e):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000007ef52c0 (size 32):
  comm "modprobe", pid 17050, jiffies 4295935726
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 ef 07 00 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc 9c68acac):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000798b980 (size 32):
  comm "modprobe", pid 17066, jiffies 4295937149
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 98 07 00 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc 13cab529):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000006d79100 (size 32):
  comm "modprobe", pid 17072, jiffies 4295937871
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 d7 06 00 00 11 ff 34 0a 0e 00 00 00 00 00  y.......4.......
  backtrace (crc 468c4c12):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000eb6d300 (size 32):
  comm "modprobe", pid 17088, jiffies 4295939305
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 b6 0e 00 00 11 ff 44 24 08 89 0c 24 e8 9d  y.......D$...$..
  backtrace (crc 39abd416):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000001029680 (size 32):
  comm "modprobe", pid 17114, jiffies 4295941403
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 02 01 00 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc 84d963e1):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000101919f80 (size 32):
  comm "modprobe", pid 17130, jiffies 4295942829
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 91 01 01 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc 86dcd9db):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000108634800 (size 32):
  comm "modprobe", pid 17136, jiffies 4295943552
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 63 08 01 00 11 ff 00 00 00 00 00 00 00 00  y.c.............
  backtrace (crc 893b4b39):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110000051d5640 (size 32):
  comm "modprobe", pid 17142, jiffies 4295944268
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 1d 05 00 00 11 ff 00 00 00 00 00 fc ff df  y...............
  backtrace (crc 4dd0396e):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110001013def40 (size 32):
  comm "modprobe", pid 17148, jiffies 4295944996
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 3d 01 01 00 11 ff 40 2b 0d 00 00 00 d4 ff  y.=.....@+......
  backtrace (crc 72d18c0f):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000102f95280 (size 32):
  comm "modprobe", pid 17154, jiffies 4295945705
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 f9 02 01 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc 8492240a):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000a957fc0 (size 32):
  comm "modprobe", pid 17170, jiffies 4295947128
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 95 0a 00 00 11 ff 65 63 6b 00 00 00 00 00  y.......eck.....
  backtrace (crc a37aa232):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000ec1d300 (size 32):
  comm "modprobe", pid 17186, jiffies 4295948570
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 c1 0e 00 00 11 ff 69 6e 69 73 68 00 00 00  y.......inish...
  backtrace (crc 66341c55):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000108066ac0 (size 32):
  comm "modprobe", pid 17212, jiffies 4295950745
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 06 08 01 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc 8145aa7d):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110000125e2980 (size 32):
  comm "modprobe", pid 17218, jiffies 4295951462
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 5e 12 00 00 11 ff 00 00 00 00 00 00 00 00  y.^.............
  backtrace (crc 80eaf345):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000002460380 (size 32):
  comm "modprobe", pid 17264, jiffies 4295954895
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 00 00 00 00 00 00 00 ed 1e 00 00 00 d4 ff  y...............
  backtrace (crc aaffd3fe):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000009b4e180 (size 32):
  comm "modprobe", pid 17290, jiffies 4295957017
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 b4 09 00 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc ebd506c):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110000040cf140 (size 32):
  comm "modprobe", pid 17306, jiffies 4295958400
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 0c 04 00 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc 526d8572):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000006d77f80 (size 32):
  comm "modprobe", pid 17322, jiffies 4295959804
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 d7 06 00 00 11 ff f4 11 0e 00 00 00 00 00  y...............
  backtrace (crc 59f3c311):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110001013d3180 (size 32):
  comm "modprobe", pid 17398, jiffies 4295965375
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 3d 01 01 00 11 ff 69 6e 69 73 68 00 00 00  y.=.....inish...
  backtrace (crc 99084885):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000f1826c0 (size 32):
  comm "modprobe", pid 17414, jiffies 4295966816
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 18 0f 00 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc caba89a6):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000100d2a240 (size 32):
  comm "modprobe", pid 17450, jiffies 4295969601
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 d2 00 01 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc 6692d274):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000524ea40 (size 32):
  comm "modprobe", pid 17466, jiffies 4295971059
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 24 05 00 00 11 ff 2b 00 00 00 00 00 00 00  y.$.....+.......
  backtrace (crc d5da4419):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000102f957c0 (size 32):
  comm "modprobe", pid 17482, jiffies 4295972468
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 f9 02 01 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc 8492240a):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110001072e3d00 (size 32):
  comm "modprobe", pid 17526, jiffies 4295976765
  hex dump (first 32 bytes):
    61 6d 64 67 70 75 5f 73 79 6e 63 5f 65 6e 74 72  amdgpu_sync_entr
    79 00 2e 07 01 00 11 ff 00 00 00 00 00 00 00 00  y...............
  backtrace (crc 580819a6):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<00000000c53a6d7f>] 0xffffffffc01503a2
    [<00000000c68a3256>] 0xffffffffc0058026
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000a3fdb60 (size 16):
  comm "modprobe", pid 17578, jiffies 4295981003
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000005e51280 (size 16):
  comm "modprobe", pid 17594, jiffies 4295982450
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110000071a1b60 (size 16):
  comm "modprobe", pid 17609, jiffies 4295983945
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100010f4a2ae0 (size 16):
  comm "modprobe", pid 17646, jiffies 4295986890
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000f20bd80 (size 16):
  comm "modprobe", pid 17662, jiffies 4295988248
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000100d26a00 (size 16):
  comm "modprobe", pid 17718, jiffies 4295992419
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100010152d240 (size 16):
  comm "modprobe", pid 17734, jiffies 4295993810
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 00 00  amdgpu_fence....
  backtrace (crc 9f81b1cd):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110001013ce0a0 (size 16):
  comm "modprobe", pid 17750, jiffies 4295995280
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000005e51dc0 (size 16):
  comm "modprobe", pid 17756, jiffies 4295996002
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000008b1bf80 (size 16):
  comm "modprobe", pid 17782, jiffies 4295998115
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000114116820 (size 16):
  comm "modprobe", pid 17798, jiffies 4295999501
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100011101d5e0 (size 16):
  comm "modprobe", pid 17825, jiffies 4296001629
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000009718320 (size 16):
  comm "modprobe", pid 17831, jiffies 4296002346
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100011515d660 (size 16):
  comm "modprobe", pid 17847, jiffies 4296003774
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000114a85080 (size 16):
  comm "modprobe", pid 17854, jiffies 4296004484
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000eb139a0 (size 16):
  comm "modprobe", pid 17860, jiffies 4296005214
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000009718b60 (size 16):
  comm "modprobe", pid 17876, jiffies 4296006637
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000005b81fc0 (size 16):
  comm "modprobe", pid 17892, jiffies 4296008053
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110000071a1b40 (size 16):
  comm "modprobe", pid 17898, jiffies 4296008778
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000f20bb00 (size 16):
  comm "modprobe", pid 17904, jiffies 4296009509
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000df76200 (size 16):
  comm "modprobe", pid 17930, jiffies 4296011592
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110000064b1ba0 (size 16):
  comm "modprobe", pid 17965, jiffies 4296014396
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110001011cbe80 (size 16):
  comm "modprobe", pid 17971, jiffies 4296015146
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000f073bc0 (size 16):
  comm "modprobe", pid 17987, jiffies 4296016566
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000107214fe0 (size 16):
  comm "modprobe", pid 18003, jiffies 4296017979
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000df769c0 (size 16):
  comm "modprobe", pid 18019, jiffies 4296019396
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100010f4a28a0 (size 16):
  comm "modprobe", pid 18035, jiffies 4296020834
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100000383b980 (size 16):
  comm "modprobe", pid 18041, jiffies 4296021552
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110000023b4e60 (size 16):
  comm "modprobe", pid 18047, jiffies 4296022306
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff11000008b1b940 (size 16):
  comm "modprobe", pid 18053, jiffies 4296023061
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff1100010b8d64c0 (size 16):
  comm "modprobe", pid 18059, jiffies 4296023783
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110001151a1ac0 (size 16):
  comm "modprobe", pid 18075, jiffies 4296025098
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150
unreferenced object 0xff110000050fcea0 (size 16):
  comm "modprobe", pid 18101, jiffies 4296027230
  hex dump (first 16 bytes):
    61 6d 64 67 70 75 5f 66 65 6e 63 65 00 00 11 ff  amdgpu_fence....
  backtrace (crc e15a7d50):
    [<000000001b48c137>] __kmalloc_node_track_caller_noprof+0x33c/0x410
    [<000000003ab002c7>] kstrdup+0x3f/0x80
    [<00000000479d3195>] kstrdup_const+0x39/0x60
    [<000000005f96320e>] kvasprintf_const+0xf9/0x190
    [<00000000725db2e2>] kobject_set_name_vargs+0x5a/0x150
    [<00000000745aee9e>] kobject_init_and_add+0xcd/0x170
    [<00000000c0d3244b>] sysfs_slab_add+0x172/0x210
    [<00000000a6c0a88d>] __kmem_cache_create+0x215/0x5c0
    [<0000000081d00250>] kmem_cache_create_usercopy+0x213/0x340
    [<000000006d9fd37f>] kmem_cache_create+0x11/0x20
    [<000000009827d27d>] 0xffffffffc00c8052
    [<00000000d74769dd>] 0xffffffffc0058038
    [<00000000b58b0e32>] do_one_initcall+0xdc/0x550
    [<0000000040a69501>] do_init_module+0x241/0x630
    [<00000000bca76796>] load_module+0x52be/0x63e0
    [<000000006d613fc0>] init_module_from_file+0xe6/0x150


> 
> I missed the module local case completely in my analysis. Can you please
> modify the condition and retest?
> 
> Thanks,
> 
>         tglx
> 
> 
> 
> 
> 

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-04  8:03           ` [PATCH] static_call: Handle module init failure correctly in static_call_del_module() Jinjie Ruan
@ 2024-09-04  8:51             ` Thomas Gleixner
  2024-09-04  9:50               ` Jinjie Ruan
  2024-09-05  3:34               ` Jinjie Ruan
  2024-09-04  9:09             ` [PATCH v2] " Thomas Gleixner
  1 sibling, 2 replies; 29+ messages in thread
From: Thomas Gleixner @ 2024-09-04  8:51 UTC (permalink / raw)
  To: Jinjie Ruan, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf

On Wed, Sep 04 2024 at 16:03, Jinjie Ruan wrote:
> On 2024/9/4 15:08, Thomas Gleixner wrote:
>> So the check must be:
>> 
>> 	if (!static_call_key_has_mods(key))
>>         	break;
>
> Hi, Thomas,
>
> with this patch, the issue not occurs again，
>
> but there are some memory leak here same to the following problem:

That has absolutely nothing to do with static calls and the memory
allocation failure case there.

The module passed all preparatory steps, otherwise it would not be able
to create a kmem_cache from the module init() function:

     kmem_cache_create+0x11/0x20
     do_one_initcall+0xdc/0x550
     do_init_module+0x241/0x630

amdgpu_init()

	r = amdgpu_sync_init();
	if (r)
		goto error_sync;

	r = amdgpu_fence_slab_init();
	if (r)
		goto error_fence;

        <SNIP>
        
	return pci_register_driver(&amdgpu_kms_pci_driver);

error_fence:
	amdgpu_sync_fini();
error_sync:
        return r;

Can you spot the problem?

Thanks,

        tglx

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-04  8:51             ` Thomas Gleixner
@ 2024-09-04  9:50               ` Jinjie Ruan
  2024-09-05  3:34               ` Jinjie Ruan
  1 sibling, 0 replies; 29+ messages in thread
From: Jinjie Ruan @ 2024-09-04  9:50 UTC (permalink / raw)
  To: Thomas Gleixner, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf



On 2024/9/4 16:51, Thomas Gleixner wrote:
> On Wed, Sep 04 2024 at 16:03, Jinjie Ruan wrote:
>> On 2024/9/4 15:08, Thomas Gleixner wrote:
>>> So the check must be:
>>>
>>> 	if (!static_call_key_has_mods(key))
>>>         	break;
>>
>> Hi, Thomas,
>>
>> with this patch, the issue not occurs again，
>>
>> but there are some memory leak here same to the following problem:
> 
> That has absolutely nothing to do with static calls and the memory
> allocation failure case there.
> 
> The module passed all preparatory steps, otherwise it would not be able
> to create a kmem_cache from the module init() function:
> 
>      kmem_cache_create+0x11/0x20
>      do_one_initcall+0xdc/0x550
>      do_init_module+0x241/0x630
> 
> amdgpu_init()
> 
> 	r = amdgpu_sync_init();
> 	if (r)
> 		goto error_sync;
> 
> 	r = amdgpu_fence_slab_init();
> 	if (r)
> 		goto error_fence;
> 
>         <SNIP>
>         
> 	return pci_register_driver(&amdgpu_kms_pci_driver);
> 
> error_fence:
> 	amdgpu_sync_fini();
> error_sync:
>         return r;
> 
> Can you spot the problem?

I see, let me test it.

> 
> Thanks,
> 
>         tglx

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-04  8:51             ` Thomas Gleixner
  2024-09-04  9:50               ` Jinjie Ruan
@ 2024-09-05  3:34               ` Jinjie Ruan
  2024-09-05  9:44                 ` Thomas Gleixner
  1 sibling, 1 reply; 29+ messages in thread
From: Jinjie Ruan @ 2024-09-05  3:34 UTC (permalink / raw)
  To: Thomas Gleixner, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf



On 2024/9/4 16:51, Thomas Gleixner wrote:
> On Wed, Sep 04 2024 at 16:03, Jinjie Ruan wrote:
>> On 2024/9/4 15:08, Thomas Gleixner wrote:
>>> So the check must be:
>>>
>>> 	if (!static_call_key_has_mods(key))
>>>         	break;
>>
>> Hi, Thomas,
>>
>> with this patch, the issue not occurs again，
>>
>> but there are some memory leak here same to the following problem:
> 
> That has absolutely nothing to do with static calls and the memory
> allocation failure case there.
> 
> The module passed all preparatory steps, otherwise it would not be able
> to create a kmem_cache from the module init() function:
> 
>      kmem_cache_create+0x11/0x20
>      do_one_initcall+0xdc/0x550
>      do_init_module+0x241/0x630
> 
> amdgpu_init()
> 
> 	r = amdgpu_sync_init();
> 	if (r)
> 		goto error_sync;
> 
> 	r = amdgpu_fence_slab_init();
> 	if (r)
> 		goto error_fence;
> 
>         <SNIP>
>         
> 	return pci_register_driver(&amdgpu_kms_pci_driver);
> 
> error_fence:
> 	amdgpu_sync_fini();
> error_sync:
>         return r;
> 
> Can you spot the problem?

Hi, Thomas,

It seems that it is not the memory leak source,

I test with the following patch, the memory leak also occurs,

but with the Link patch, the problem solved and the memory leak missed.

 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 094498a0964b..3589e4768bd6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -3032,11 +3032,11 @@ static int __init amdgpu_init(void)

        r = amdgpu_sync_init();
        if (r)
-               goto error_sync;
+               return r;

        r = amdgpu_fence_slab_init();
        if (r)
-               goto error_fence;
+               goto error_sync;

        DRM_INFO("amdgpu kernel modesetting enabled.\n");
        amdgpu_register_atpx_handler();
@@ -3046,12 +3046,18 @@ static int __init amdgpu_init(void)
        amdgpu_amdkfd_init();

        /* let modprobe override vga console setting */
-       return pci_register_driver(&amdgpu_kms_pci_driver);
+       r = pci_register_driver(&amdgpu_kms_pci_driver);
+       if (r)
+               goto error_fence;
+
+       return 0;

 error_fence:
-       amdgpu_sync_fini();
+       amdgpu_fence_slab_fini();

 error_sync:
+       amdgpu_sync_fini();
+
        return r;
 }


> 
> Thanks,
> 
>         tglx

^ permalink raw reply related	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-05  3:34               ` Jinjie Ruan
@ 2024-09-05  9:44                 ` Thomas Gleixner
  2024-09-06 23:24                   ` Luis Chamberlain
  0 siblings, 1 reply; 29+ messages in thread
From: Thomas Gleixner @ 2024-09-05  9:44 UTC (permalink / raw)
  To: Jinjie Ruan, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf, mcgrof, Liu Shixin

On Thu, Sep 05 2024 at 11:34, Jinjie Ruan wrote:
> On 2024/9/4 16:51, Thomas Gleixner wrote:
>> Can you spot the problem?
>
> It seems that it is not the memory leak source,

That's possible, but the code in there is definitely incorrect when
pci_register_driver() fails. Simply because module->exit() is not
invoked when module->init() fails.

> but with the Link patch, the problem solved and the memory leak missed.

Sure, but the change log of this patch is not really helpful at all and
you sent me a gazillion of leak dumps, but failed to explain what the
actual failure scenario is.

So without a coherent description how to reproduce this issue there is
not much I can do than looking at the code. The amdgpu init() function
was an obvious candidate, but there seems to be some other way to cause
that problem.

Now you at least provided the information that the missing cleanup in
the init() function is not the problem. So the obvious place to look is
in the module core code whether there is a failure path _after_
module->init() returned success.

do_init_module()
        ret = do_one_initcall(mod->init);
        ...
	ret = module_enable_rodata_ro(mod, true);
	if (ret)
		goto fail_mutex_unlock;

and that error path does _not_ invoke module->exit(), which is obviously
not correct. Luis?

I assume that's not the problem when looking at the change log of that:

 https://lore.kernel.org/all/20221112114602.1268989-4-liushixin2@huawei.com/

> Following the rules stated in the comment for kobject_init_and_add():
>  If this function returns an error, kobject_put() must be called to
>  properly clean up the memory associated with the object.
> 
> kobject_put() is more appropriate for error handling after kobject_init().

Why? The rules are exactly the same, no?

> And we can use this function to solve this problem.

Which function and which problem?

> For the cache created early, the related sysfs_slab_add() is called in
> slab_sysfs_init(). Skip free these kmem_cache since they are important
> for system. Keep them working without sysfs.

Let me try to decode this. I assume that sysfs_slab_add() fails after
kobject_init_and_add() or kobject_init_and_add) itself fails.

So the problem is that there are two ways how this can be invoked:

   1) from slab_sysfs_init() for the kmem_cache instances which have
      been allocated during early boot before sysfs was available.

   2) from kmem_cache_create() after early boot

So what Liu tries to avoid is to invoke kobject_put(), because
kobject_put() would not only free the name (which is what the leak
complains about), but also invoke the release callback, which would 
try to destroy the kmem_cache itself.

That's a problem for the mm people to solve, but that does not make my
observations about the amdgpu init() function and the error path in
do_init_module() moot :)

Thanks,

        tglx

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-05  9:44                 ` Thomas Gleixner
@ 2024-09-06 23:24                   ` Luis Chamberlain
  2024-09-19  9:53                     ` Luis Chamberlain
  0 siblings, 1 reply; 29+ messages in thread
From: Luis Chamberlain @ 2024-09-06 23:24 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Jinjie Ruan, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf, Liu Shixin

On Thu, Sep 05, 2024 at 11:44:00AM +0200, Thomas Gleixner wrote:
> Now you at least provided the information that the missing cleanup in
> the init() function is not the problem. So the obvious place to look is
> in the module core code whether there is a failure path _after_
> module->init() returned success.
> 
> do_init_module()
>         ret = do_one_initcall(mod->init);
>         ...
> 	ret = module_enable_rodata_ro(mod, true);
> 	if (ret)
> 		goto fail_mutex_unlock;
> 
> and that error path does _not_ invoke module->exit(), which is obviously
> not correct. Luis?

You're spot on this needs fixing.

  Luis

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-06 23:24                   ` Luis Chamberlain
@ 2024-09-19  9:53                     ` Luis Chamberlain
  2024-09-24  7:22                       ` Mike Rapoport
  0 siblings, 1 reply; 29+ messages in thread
From: Luis Chamberlain @ 2024-09-19  9:53 UTC (permalink / raw)
  To: Christophe Leroy, Thomas Gleixner, Mike Rapoport
  Cc: Jinjie Ruan, linux-kernel, Peter Zijlstra (Intel), Josh Poimboeuf,
	Liu Shixin

On Fri, Sep 06, 2024 at 04:24:56PM -0700, Luis Chamberlain wrote:
> On Thu, Sep 05, 2024 at 11:44:00AM +0200, Thomas Gleixner wrote:
> > Now you at least provided the information that the missing cleanup in
> > the init() function is not the problem. So the obvious place to look is
> > in the module core code whether there is a failure path _after_
> > module->init() returned success.
> > 
> > do_init_module()
> >         ret = do_one_initcall(mod->init);
> >         ...
> > 	ret = module_enable_rodata_ro(mod, true);
> > 	if (ret)
> > 		goto fail_mutex_unlock;
> > 
> > and that error path does _not_ invoke module->exit(), which is obviously
> > not correct. Luis?
> 
> You're spot on this needs fixing.

Christophe, this is a regression caused by the second hunk of your commit
d1909c0221739 ("module: Don't ignore errors from set_memory_XX()") on v6.9.
Sadly there are a few issues with trying to get to call mod->exit():

- We should try try_stop_module()  and that can fail
- source_list may not be empty and that would block removal
- mod->exit may not exist

I'm wondering if instead we should try to do the module_enable_rodata_ro()
before the init, but that requires a bit more careful evaluation...

  Luis

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-19  9:53                     ` Luis Chamberlain
@ 2024-09-24  7:22                       ` Mike Rapoport
  2024-11-08  8:12                         ` Christophe Leroy
  0 siblings, 1 reply; 29+ messages in thread
From: Mike Rapoport @ 2024-09-24  7:22 UTC (permalink / raw)
  To: Luis Chamberlain
  Cc: Christophe Leroy, Thomas Gleixner, Jinjie Ruan, linux-kernel,
	Peter Zijlstra (Intel), Josh Poimboeuf, Liu Shixin

On Thu, Sep 19, 2024 at 02:53:34AM -0700, Luis Chamberlain wrote:
> On Fri, Sep 06, 2024 at 04:24:56PM -0700, Luis Chamberlain wrote:
> > On Thu, Sep 05, 2024 at 11:44:00AM +0200, Thomas Gleixner wrote:
> > > Now you at least provided the information that the missing cleanup in
> > > the init() function is not the problem. So the obvious place to look is
> > > in the module core code whether there is a failure path _after_
> > > module->init() returned success.
> > > 
> > > do_init_module()
> > >         ret = do_one_initcall(mod->init);
> > >         ...
> > > 	ret = module_enable_rodata_ro(mod, true);
> > > 	if (ret)
> > > 		goto fail_mutex_unlock;
> > > 
> > > and that error path does _not_ invoke module->exit(), which is obviously
> > > not correct. Luis?
> > 
> > You're spot on this needs fixing.
> 
> Christophe, this is a regression caused by the second hunk of your commit
> d1909c0221739 ("module: Don't ignore errors from set_memory_XX()") on v6.9.
> Sadly there are a few issues with trying to get to call mod->exit():
> 
> - We should try try_stop_module()  and that can fail
> - source_list may not be empty and that would block removal
> - mod->exit may not exist
> 
> I'm wondering if instead we should try to do the module_enable_rodata_ro()
> before the init, but that requires a bit more careful evaluation...

There is ro_after_init section, we can't really make it RO before ->init()
 
>   Luis

-- 
Sincerely yours,
Mike.

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-24  7:22                       ` Mike Rapoport
@ 2024-11-08  8:12                         ` Christophe Leroy
  2024-11-08 15:49                           ` Luis Chamberlain
  0 siblings, 1 reply; 29+ messages in thread
From: Christophe Leroy @ 2024-11-08  8:12 UTC (permalink / raw)
  To: Mike Rapoport, Luis Chamberlain
  Cc: Thomas Gleixner, Jinjie Ruan, linux-kernel,
	Peter Zijlstra (Intel), Josh Poimboeuf, Liu Shixin

Hi Luis,

Le 24/09/2024 à 09:22, Mike Rapoport a écrit :
> On Thu, Sep 19, 2024 at 02:53:34AM -0700, Luis Chamberlain wrote:
>> On Fri, Sep 06, 2024 at 04:24:56PM -0700, Luis Chamberlain wrote:
>>> On Thu, Sep 05, 2024 at 11:44:00AM +0200, Thomas Gleixner wrote:
>>>> Now you at least provided the information that the missing cleanup in
>>>> the init() function is not the problem. So the obvious place to look is
>>>> in the module core code whether there is a failure path _after_
>>>> module->init() returned success.
>>>>
>>>> do_init_module()
>>>>          ret = do_one_initcall(mod->init);
>>>>          ...
>>>> 	ret = module_enable_rodata_ro(mod, true);
>>>> 	if (ret)
>>>> 		goto fail_mutex_unlock;
>>>>
>>>> and that error path does _not_ invoke module->exit(), which is obviously
>>>> not correct. Luis?
>>>
>>> You're spot on this needs fixing.
>>
>> Christophe, this is a regression caused by the second hunk of your commit
>> d1909c0221739 ("module: Don't ignore errors from set_memory_XX()") on v6.9.
>> Sadly there are a few issues with trying to get to call mod->exit():
>>
>> - We should try try_stop_module()  and that can fail
>> - source_list may not be empty and that would block removal
>> - mod->exit may not exist
>>
>> I'm wondering if instead we should try to do the module_enable_rodata_ro()
>> before the init, but that requires a bit more careful evaluation...
> 
> There is ro_after_init section, we can't really make it RO before ->init()
>   

Surprisingly I never received Luis's email allthough I got this answer 
from Mike that I overlooked.

So coming back here from 
https://lore.kernel.org/all/ZyQhbHxDTRXTJgIx@bombadil.infradead.org/

As far as I understand, indeed once init is called it is too late to 
fail, right ? Especially when the module has no exit() path or when 
CONFIG_MODULE_UNLOAD is not built in.

So the only thing we can do then is a big fat warning telling 
set_memory_ro() on ro_after_init memory has failed ?

Maybe we should try and change it to RO then back to RW before calling 
init, to be on a safer side hopping that if change to RO works once it 
will work twice ?

Christophe

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-11-08  8:12                         ` Christophe Leroy
@ 2024-11-08 15:49                           ` Luis Chamberlain
  2024-11-08 16:09                             ` Daniel Gomez
  0 siblings, 1 reply; 29+ messages in thread
From: Luis Chamberlain @ 2024-11-08 15:49 UTC (permalink / raw)
  To: Christophe Leroy, Daniel Gomez, Petr Pavlu, Sami Tolvanen,
	linux-modules
  Cc: Mike Rapoport, Thomas Gleixner, Jinjie Ruan, linux-kernel,
	Peter Zijlstra (Intel), Josh Poimboeuf, Liu Shixin

+ Other new module maintainers

On Fri, Nov 08, 2024 at 09:12:03AM +0100, Christophe Leroy wrote:
> Hi Luis,
> 
> Le 24/09/2024 à 09:22, Mike Rapoport a écrit :
> > On Thu, Sep 19, 2024 at 02:53:34AM -0700, Luis Chamberlain wrote:
> > > On Fri, Sep 06, 2024 at 04:24:56PM -0700, Luis Chamberlain wrote:
> > > > On Thu, Sep 05, 2024 at 11:44:00AM +0200, Thomas Gleixner wrote:
> > > > > Now you at least provided the information that the missing cleanup in
> > > > > the init() function is not the problem. So the obvious place to look is
> > > > > in the module core code whether there is a failure path _after_
> > > > > module->init() returned success.
> > > > > 
> > > > > do_init_module()
> > > > >          ret = do_one_initcall(mod->init);
> > > > >          ...
> > > > > 	ret = module_enable_rodata_ro(mod, true);
> > > > > 	if (ret)
> > > > > 		goto fail_mutex_unlock;
> > > > > 
> > > > > and that error path does _not_ invoke module->exit(), which is obviously
> > > > > not correct. Luis?
> > > > 
> > > > You're spot on this needs fixing.
> > > 
> > > Christophe, this is a regression caused by the second hunk of your commit
> > > d1909c0221739 ("module: Don't ignore errors from set_memory_XX()") on v6.9.
> > > Sadly there are a few issues with trying to get to call mod->exit():
> > > 
> > > - We should try try_stop_module()  and that can fail
> > > - source_list may not be empty and that would block removal
> > > - mod->exit may not exist
> > > 
> > > I'm wondering if instead we should try to do the module_enable_rodata_ro()
> > > before the init, but that requires a bit more careful evaluation...
> > 
> > There is ro_after_init section, we can't really make it RO before ->init()
> 
> Surprisingly I never received Luis's email

So odd..

> allthough I got this answer from Mike that I overlooked.
> 
> So coming back here from
> https://lore.kernel.org/all/ZyQhbHxDTRXTJgIx@bombadil.infradead.org/
> 
> As far as I understand, indeed once init is called it is too late to fail,

Partly yes, party no. Party yes in that its a can of worms we have not
had to deal with before, and also I worry about deadlocks, and the code
to address this seems complex. right ?


> Especially when the module has no exit() path or when
> CONFIG_MODULE_UNLOAD is not built in.

That's exactly the other extreme case I fear for.

> So the only thing we can do then is a big fat warning telling
> set_memory_ro() on ro_after_init memory has failed ?

I suspect this is more sensible to do.

> Maybe we should try and change it to RO then back to RW before calling init,
> to be on a safer side hopping that if change to RO works once it will work
> twice ?

That's another approach wich could work, if we proove that this does
work, it's a nice best effort and I think less or a mess to the codebase
then special-casing the error handling of trying to deal with the
driver's exit.

Daniel Gomez has been looking at this, so his feedback here would be
valuable.

  Luis

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-11-08 15:49                           ` Luis Chamberlain
@ 2024-11-08 16:09                             ` Daniel Gomez
  2024-11-08 16:22                               ` Daniel Gomez
  0 siblings, 1 reply; 29+ messages in thread
From: Daniel Gomez @ 2024-11-08 16:09 UTC (permalink / raw)
  To: Luis Chamberlain, Christophe Leroy, Petr Pavlu, Sami Tolvanen,
	linux-modules
  Cc: Mike Rapoport, Thomas Gleixner, Jinjie Ruan, linux-kernel,
	Peter Zijlstra (Intel), Josh Poimboeuf, Liu Shixin

On Fri Nov 8, 2024 at 4:49 PM CET, Luis Chamberlain wrote:
> + Other new module maintainers
>
> On Fri, Nov 08, 2024 at 09:12:03AM +0100, Christophe Leroy wrote:
>> Hi Luis,
>> 
>> Le 24/09/2024 à 09:22, Mike Rapoport a écrit :
>> > On Thu, Sep 19, 2024 at 02:53:34AM -0700, Luis Chamberlain wrote:
>> > > On Fri, Sep 06, 2024 at 04:24:56PM -0700, Luis Chamberlain wrote:
>> > > > On Thu, Sep 05, 2024 at 11:44:00AM +0200, Thomas Gleixner wrote:
>> > > > > Now you at least provided the information that the missing cleanup in
>> > > > > the init() function is not the problem. So the obvious place to look is
>> > > > > in the module core code whether there is a failure path _after_
>> > > > > module->init() returned success.
>> > > > > 
>> > > > > do_init_module()
>> > > > >          ret = do_one_initcall(mod->init);
>> > > > >          ...
>> > > > > 	ret = module_enable_rodata_ro(mod, true);
>> > > > > 	if (ret)
>> > > > > 		goto fail_mutex_unlock;
>> > > > > 
>> > > > > and that error path does _not_ invoke module->exit(), which is obviously
>> > > > > not correct. Luis?
>> > > > 
>> > > > You're spot on this needs fixing.
>> > > 
>> > > Christophe, this is a regression caused by the second hunk of your commit
>> > > d1909c0221739 ("module: Don't ignore errors from set_memory_XX()") on v6.9.
>> > > Sadly there are a few issues with trying to get to call mod->exit():
>> > > 
>> > > - We should try try_stop_module()  and that can fail
>> > > - source_list may not be empty and that would block removal
>> > > - mod->exit may not exist
>> > > 
>> > > I'm wondering if instead we should try to do the module_enable_rodata_ro()
>> > > before the init, but that requires a bit more careful evaluation...
>> > 
>> > There is ro_after_init section, we can't really make it RO before ->init()
>> 
>> Surprisingly I never received Luis's email
>
> So odd..
>
>> allthough I got this answer from Mike that I overlooked.
>> 
>> So coming back here from
>> https://lore.kernel.org/all/ZyQhbHxDTRXTJgIx@bombadil.infradead.org/
>> 
>> As far as I understand, indeed once init is called it is too late to fail,
>
> Partly yes, party no. Party yes in that its a can of worms we have not
> had to deal with before, and also I worry about deadlocks, and the code
> to address this seems complex. right ?

I have a RFC ready with this, I'll send this now so we can discuss on
with a proposal.

>
>
>> Especially when the module has no exit() path or when
>> CONFIG_MODULE_UNLOAD is not built in.
>
> That's exactly the other extreme case I fear for.
>
>> So the only thing we can do then is a big fat warning telling
>> set_memory_ro() on ro_after_init memory has failed ?
>
> I suspect this is more sensible to do.

I came to the same conclusion while trying to fix this path. + I added
an alternative for discussion.

>
>> Maybe we should try and change it to RO then back to RW before calling init,
>> to be on a safer side hopping that if change to RO works once it will work
>> twice ?
>
> That's another approach wich could work, if we proove that this does
> work, it's a nice best effort and I think less or a mess to the codebase
> then special-casing the error handling of trying to deal with the
> driver's exit.
>
> Daniel Gomez has been looking at this, so his feedback here would be
> valuable.

What if we detect ro_after_init first, and block any module
initialization depending on this ro_after_init to actually start loading
it? That way we can stop and unload the module successfully.

>
>   Luis


^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH] static_call: Handle module init failure correctly in static_call_del_module()
  2024-11-08 16:09                             ` Daniel Gomez
@ 2024-11-08 16:22                               ` Daniel Gomez
  0 siblings, 0 replies; 29+ messages in thread
From: Daniel Gomez @ 2024-11-08 16:22 UTC (permalink / raw)
  To: Luis Chamberlain, Christophe Leroy, Petr Pavlu, Sami Tolvanen,
	linux-modules
  Cc: Mike Rapoport, Thomas Gleixner, Jinjie Ruan, linux-kernel,
	Peter Zijlstra (Intel), Josh Poimboeuf, Liu Shixin

On Fri Nov 8, 2024 at 5:09 PM CET, Daniel Gomez wrote:
> On Fri Nov 8, 2024 at 4:49 PM CET, Luis Chamberlain wrote:
>> + Other new module maintainers
>>
>> On Fri, Nov 08, 2024 at 09:12:03AM +0100, Christophe Leroy wrote:
>>> Hi Luis,
>>> 
>>> Le 24/09/2024 à 09:22, Mike Rapoport a écrit :
>>> > On Thu, Sep 19, 2024 at 02:53:34AM -0700, Luis Chamberlain wrote:
>>> > > On Fri, Sep 06, 2024 at 04:24:56PM -0700, Luis Chamberlain wrote:
>>> > > > On Thu, Sep 05, 2024 at 11:44:00AM +0200, Thomas Gleixner wrote:
>>> > > > > Now you at least provided the information that the missing cleanup in
>>> > > > > the init() function is not the problem. So the obvious place to look is
>>> > > > > in the module core code whether there is a failure path _after_
>>> > > > > module->init() returned success.
>>> > > > > 
>>> > > > > do_init_module()
>>> > > > >          ret = do_one_initcall(mod->init);
>>> > > > >          ...
>>> > > > > 	ret = module_enable_rodata_ro(mod, true);
>>> > > > > 	if (ret)
>>> > > > > 		goto fail_mutex_unlock;
>>> > > > > 
>>> > > > > and that error path does _not_ invoke module->exit(), which is obviously
>>> > > > > not correct. Luis?
>>> > > > 
>>> > > > You're spot on this needs fixing.
>>> > > 
>>> > > Christophe, this is a regression caused by the second hunk of your commit
>>> > > d1909c0221739 ("module: Don't ignore errors from set_memory_XX()") on v6.9.
>>> > > Sadly there are a few issues with trying to get to call mod->exit():
>>> > > 
>>> > > - We should try try_stop_module()  and that can fail
>>> > > - source_list may not be empty and that would block removal
>>> > > - mod->exit may not exist
>>> > > 
>>> > > I'm wondering if instead we should try to do the module_enable_rodata_ro()
>>> > > before the init, but that requires a bit more careful evaluation...
>>> > 
>>> > There is ro_after_init section, we can't really make it RO before ->init()
>>> 
>>> Surprisingly I never received Luis's email
>>
>> So odd..
>>
>>> allthough I got this answer from Mike that I overlooked.
>>> 
>>> So coming back here from
>>> https://lore.kernel.org/all/ZyQhbHxDTRXTJgIx@bombadil.infradead.org/
>>> 
>>> As far as I understand, indeed once init is called it is too late to fail,
>>
>> Partly yes, party no. Party yes in that its a can of worms we have not
>> had to deal with before, and also I worry about deadlocks, and the code
>> to address this seems complex. right ?
>
> I have a RFC ready with this, I'll send this now so we can discuss on
> with a proposal.
>
>>
>>
>>> Especially when the module has no exit() path or when
>>> CONFIG_MODULE_UNLOAD is not built in.
>>
>> That's exactly the other extreme case I fear for.
>>
>>> So the only thing we can do then is a big fat warning telling
>>> set_memory_ro() on ro_after_init memory has failed ?
>>
>> I suspect this is more sensible to do.
>
> I came to the same conclusion while trying to fix this path. + I added
> an alternative for discussion.
>
>>
>>> Maybe we should try and change it to RO then back to RW before calling init,
>>> to be on a safer side hopping that if change to RO works once it will work
>>> twice ?
>>
>> That's another approach wich could work, if we proove that this does
>> work, it's a nice best effort and I think less or a mess to the codebase
>> then special-casing the error handling of trying to deal with the
>> driver's exit.
>>
>> Daniel Gomez has been looking at this, so his feedback here would be
>> valuable.
>
> What if we detect ro_after_init first, and block any module
> initialization depending on this ro_after_init to actually start loading
> it? That way we can stop and unload the module successfully.

In case I'm missing someone, I've just sent the RFC here:

https://lore.kernel.org/linux-modules/20241108-modules-ro_after_init-v3-0-6dd041b588a5@samsung.com/T/#t

Please ignore "v3" prefix. That was a mistake. Not sure why b4 added
that.

>
>>
>>   Luis


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PATCH v2] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-04  8:03           ` [PATCH] static_call: Handle module init failure correctly in static_call_del_module() Jinjie Ruan
  2024-09-04  8:51             ` Thomas Gleixner
@ 2024-09-04  9:09             ` Thomas Gleixner
  2024-09-04  9:19               ` Jinjie Ruan
  2024-09-06 14:41               ` [tip: locking/urgent] " tip-bot2 for Thomas Gleixner
  1 sibling, 2 replies; 29+ messages in thread
From: Thomas Gleixner @ 2024-09-04  9:09 UTC (permalink / raw)
  To: Jinjie Ruan, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf

Module insertion invokes static_call_add_module() to initialize the static
calls in a module. static_call_add_module() invokes __static_call_init(),
which allocates a struct static_call_mod to either encapsulate the built-in
static call sites of the associated key into it so further modules can be
added or to append the module to the module chain.

If that allocation fails the function returns with an error code and the
module core invokes static_call_del_module() to clean up eventually added
static_call_mod entries.

This works correctly, when all keys used by the module were converted over
to a module chain before the failure. If not then static_call_del_module()
causes a #GP as it blindly assumes that key::mods points to a valid struct
static_call_mod.

The problem is that key::mods is not a individual struct member of struct
static_call_key, it's part of a union to save space:

        union {
                /* bit 0: 0 = mods, 1 = sites */
                unsigned long type;
                struct static_call_mod *mods;
                struct static_call_site *sites;
	};

key::sites is a pointer to the list of built-in usage sites of the static
call. The type of the pointer is differentiated by bit 0. A mods pointer
has the bit clear, the sites pointer has the bit set.

As static_call_del_module() blidly assumes that the pointer is a valid
static_call_mod type, it fails to check for this failure case and
dereferences the pointer to the list of built-in call sites, which is
obviously bogus.

Cure it by checking whether the key has a sites or a mods pointer. 

If it's a sites pointer then the key is not to be touched. As the sites are
walked in the same order as in __static_call_init() the site walk can be
terminated because all subsequent sites have not been touched by the init
code due to the error exit.

If it was converted before the allocation fail, then the inner loop which
searches for a module match will find nothing.

A fail in the second allocation in __static_call_init() is harmless and
does not require special treatment. The first allocation succeeded and
converted the key to a module chain. That first entry has mod::mod == NULL
and mod::next == NULL, so the inner loop of static_call_del_module() will
neither find a module match nor a module chain. The next site in the walk
was either already converted, but can't match the module, or it will exit
the outer loop because it has a static_call_site pointer and not a
static_call_mod pointer.

Fixes: 9183c3f9ed71 ("static_call: Add inline static call infrastructure")
Reported-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Closes: https://lore.kernel.org/all/20230915082126.4187913-1-ruanjinjie@huawei.com
---
V2: Use static_call_key_has_mods() instead
---
 kernel/static_call_inline.c |   11 +++++++++++
 1 file changed, 11 insertions(+)

--- a/kernel/static_call_inline.c
+++ b/kernel/static_call_inline.c
@@ -411,6 +411,17 @@ static void static_call_del_module(struc

 	for (site = start; site < stop; site++) {
 		key = static_call_key(site);
+
+		/*
+		 * If the key was not updated due to a memory allocation
+		 * failure in __static_call_init() then treating key::sites
+		 * as key::mods in the code below would cause random memory
+		 * access and #GP. In that case all subsequent sites have
+		 * not been touched either, so stop iterating.
+		 */
+		if (!static_call_key_has_mods(key))
+			break;
+
 		if (key == prev_key)
 			continue;

^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PATCH v2] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-04  9:09             ` [PATCH v2] " Thomas Gleixner
@ 2024-09-04  9:19               ` Jinjie Ruan
  2024-09-06 14:41               ` [tip: locking/urgent] " tip-bot2 for Thomas Gleixner
  1 sibling, 0 replies; 29+ messages in thread
From: Jinjie Ruan @ 2024-09-04  9:19 UTC (permalink / raw)
  To: Thomas Gleixner, linux-kernel, Peter Zijlstra (Intel),
	Christophe Leroy, Josh Poimboeuf



On 2024/9/4 17:09, Thomas Gleixner wrote:
> Module insertion invokes static_call_add_module() to initialize the static
> calls in a module. static_call_add_module() invokes __static_call_init(),
> which allocates a struct static_call_mod to either encapsulate the built-in
> static call sites of the associated key into it so further modules can be
> added or to append the module to the module chain.
> 
> If that allocation fails the function returns with an error code and the
> module core invokes static_call_del_module() to clean up eventually added
> static_call_mod entries.
> 
> This works correctly, when all keys used by the module were converted over
> to a module chain before the failure. If not then static_call_del_module()
> causes a #GP as it blindly assumes that key::mods points to a valid struct
> static_call_mod.
> 
> The problem is that key::mods is not a individual struct member of struct
> static_call_key, it's part of a union to save space:
> 
>         union {
>                 /* bit 0: 0 = mods, 1 = sites */
>                 unsigned long type;
>                 struct static_call_mod *mods;
>                 struct static_call_site *sites;
> 	};
> 
> key::sites is a pointer to the list of built-in usage sites of the static
> call. The type of the pointer is differentiated by bit 0. A mods pointer
> has the bit clear, the sites pointer has the bit set.
> 
> As static_call_del_module() blidly assumes that the pointer is a valid
> static_call_mod type, it fails to check for this failure case and
> dereferences the pointer to the list of built-in call sites, which is
> obviously bogus.
> 
> Cure it by checking whether the key has a sites or a mods pointer. 
> 
> If it's a sites pointer then the key is not to be touched. As the sites are
> walked in the same order as in __static_call_init() the site walk can be
> terminated because all subsequent sites have not been touched by the init
> code due to the error exit.
> 
> If it was converted before the allocation fail, then the inner loop which
> searches for a module match will find nothing.
> 
> A fail in the second allocation in __static_call_init() is harmless and
> does not require special treatment. The first allocation succeeded and
> converted the key to a module chain. That first entry has mod::mod == NULL
> and mod::next == NULL, so the inner loop of static_call_del_module() will
> neither find a module match nor a module chain. The next site in the walk
> was either already converted, but can't match the module, or it will exit
> the outer loop because it has a static_call_site pointer and not a
> static_call_mod pointer.
> 
> Fixes: 9183c3f9ed71 ("static_call: Add inline static call infrastructure")
> Reported-by: Jinjie Ruan <ruanjinjie@huawei.com>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Closes: https://lore.kernel.org/all/20230915082126.4187913-1-ruanjinjie@huawei.com
> ---
> V2: Use static_call_key_has_mods() instead
> ---
>  kernel/static_call_inline.c |   11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> --- a/kernel/static_call_inline.c
> +++ b/kernel/static_call_inline.c
> @@ -411,6 +411,17 @@ static void static_call_del_module(struc
>  
>  	for (site = start; site < stop; site++) {
>  		key = static_call_key(site);
> +
> +		/*
> +		 * If the key was not updated due to a memory allocation
> +		 * failure in __static_call_init() then treating key::sites
> +		 * as key::mods in the code below would cause random memory
> +		 * access and #GP. In that case all subsequent sites have
> +		 * not been touched either, so stop iterating.
> +		 */
> +		if (!static_call_key_has_mods(key))
> +			break;
> +

Tested-by: Jinjie Ruan <ruanjinjie@huawei.com>

>  		if (key == prev_key)
>  			continue;
>  

^ permalink raw reply	[flat|nested] 29+ messages in thread

* [tip: locking/urgent] static_call: Handle module init failure correctly in static_call_del_module()
  2024-09-04  9:09             ` [PATCH v2] " Thomas Gleixner
  2024-09-04  9:19               ` Jinjie Ruan
@ 2024-09-06 14:41               ` tip-bot2 for Thomas Gleixner
  1 sibling, 0 replies; 29+ messages in thread
From: tip-bot2 for Thomas Gleixner @ 2024-09-06 14:41 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: Jinjie Ruan, Thomas Gleixner, Peter Zijlstra (Intel), x86,
	linux-kernel

The following commit has been merged into the locking/urgent branch of tip:

Commit-ID:     4b30051c4864234ec57290c3d142db7c88f10d8a
Gitweb:        https://git.kernel.org/tip/4b30051c4864234ec57290c3d142db7c88f10d8a
Author:        Thomas Gleixner <tglx@linutronix.de>
AuthorDate:    Wed, 04 Sep 2024 11:09:07 +02:00
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Fri, 06 Sep 2024 16:29:21 +02:00

static_call: Handle module init failure correctly in static_call_del_module()

Module insertion invokes static_call_add_module() to initialize the static
calls in a module. static_call_add_module() invokes __static_call_init(),
which allocates a struct static_call_mod to either encapsulate the built-in
static call sites of the associated key into it so further modules can be
added or to append the module to the module chain.

If that allocation fails the function returns with an error code and the
module core invokes static_call_del_module() to clean up eventually added
static_call_mod entries.

This works correctly, when all keys used by the module were converted over
to a module chain before the failure. If not then static_call_del_module()
causes a #GP as it blindly assumes that key::mods points to a valid struct
static_call_mod.

The problem is that key::mods is not a individual struct member of struct
static_call_key, it's part of a union to save space:

        union {
                /* bit 0: 0 = mods, 1 = sites */
                unsigned long type;
                struct static_call_mod *mods;
                struct static_call_site *sites;
	};

key::sites is a pointer to the list of built-in usage sites of the static
call. The type of the pointer is differentiated by bit 0. A mods pointer
has the bit clear, the sites pointer has the bit set.

As static_call_del_module() blidly assumes that the pointer is a valid
static_call_mod type, it fails to check for this failure case and
dereferences the pointer to the list of built-in call sites, which is
obviously bogus.

Cure it by checking whether the key has a sites or a mods pointer.

If it's a sites pointer then the key is not to be touched. As the sites are
walked in the same order as in __static_call_init() the site walk can be
terminated because all subsequent sites have not been touched by the init
code due to the error exit.

If it was converted before the allocation fail, then the inner loop which
searches for a module match will find nothing.

A fail in the second allocation in __static_call_init() is harmless and
does not require special treatment. The first allocation succeeded and
converted the key to a module chain. That first entry has mod::mod == NULL
and mod::next == NULL, so the inner loop of static_call_del_module() will
neither find a module match nor a module chain. The next site in the walk
was either already converted, but can't match the module, or it will exit
the outer loop because it has a static_call_site pointer and not a
static_call_mod pointer.

Fixes: 9183c3f9ed71 ("static_call: Add inline static call infrastructure")
Closes: https://lore.kernel.org/all/20230915082126.4187913-1-ruanjinjie@huawei.com
Reported-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Jinjie Ruan <ruanjinjie@huawei.com>
Link: https://lore.kernel.org/r/87zfon6b0s.ffs@tglx
---
 kernel/static_call_inline.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/kernel/static_call_inline.c b/kernel/static_call_inline.c
index 639397b..7bb0962 100644
--- a/kernel/static_call_inline.c
+++ b/kernel/static_call_inline.c
@@ -411,6 +411,17 @@ static void static_call_del_module(struct module *mod)

 	for (site = start; site < stop; site++) {
 		key = static_call_key(site);
+
+		/*
+		 * If the key was not updated due to a memory allocation
+		 * failure in __static_call_init() then treating key::sites
+		 * as key::mods in the code below would cause random memory
+		 * access and #GP. In that case all subsequent sites have
+		 * not been touched either, so stop iterating.
+		 */
+		if (!static_call_key_has_mods(key))
+			break;
+
 		if (key == prev_key)
 			continue;

^ permalink raw reply related	[flat|nested] 29+ messages in thread

end of thread, other threads:[~2024-11-08 16:22 UTC | newest]

Thread overview: 29+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-09-15  8:21 [PATCH] static_call: Fix a wild-memory-access bug in static_call_del_module() Jinjie Ruan
2023-09-15  9:01 ` Peter Zijlstra
2024-09-02 12:01 ` Jinjie Ruan
2024-09-02 12:06   ` Christophe Leroy
2024-09-02 12:07     ` Jinjie Ruan
2024-09-02 12:14       ` Christophe Leroy
2024-09-03 20:01   ` Thomas Gleixner
2024-09-03 22:58     ` [PATCH] static_call: Handle module init failure correctly " Thomas Gleixner
2024-09-04  3:32       ` Jinjie Ruan
2024-09-04  7:08         ` Thomas Gleixner
2024-09-04  8:00           ` Thomas Gleixner
2024-09-04  8:17             ` Peter Zijlstra
2024-09-04  9:08               ` [PATCH] static_call: Replace pointless WARN_ON() in static_call_module_notify() Thomas Gleixner
2024-09-06 14:41                 ` [tip: locking/urgent] " tip-bot2 for Thomas Gleixner
2024-09-04  8:03           ` [PATCH] static_call: Handle module init failure correctly in static_call_del_module() Jinjie Ruan
2024-09-04  8:51             ` Thomas Gleixner
2024-09-04  9:50               ` Jinjie Ruan
2024-09-05  3:34               ` Jinjie Ruan
2024-09-05  9:44                 ` Thomas Gleixner
2024-09-06 23:24                   ` Luis Chamberlain
2024-09-19  9:53                     ` Luis Chamberlain
2024-09-24  7:22                       ` Mike Rapoport
2024-11-08  8:12                         ` Christophe Leroy
2024-11-08 15:49                           ` Luis Chamberlain
2024-11-08 16:09                             ` Daniel Gomez
2024-11-08 16:22                               ` Daniel Gomez
2024-09-04  9:09             ` [PATCH v2] " Thomas Gleixner
2024-09-04  9:19               ` Jinjie Ruan
2024-09-06 14:41               ` [tip: locking/urgent] " tip-bot2 for Thomas Gleixner

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.