Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6

Mirsad Goran Todorovac
2023-02-08T19:55:45Z

[-- Attachment #1: Type: text/plain, Size: 6736 bytes --]

Hi all,

On 31. 01. 2023. 10:36, Mirsad Goran Todorovac wrote:
> Hi all,
>
> I came across this memory leak apparently in the GPIO device driver.
> It is still present in 6.2-rc6 release candidate kernel (just ran kselftest).
>
> This is a vanilla Torvalds tree kernel with MGLRU and KMEMLEAK (obviously)
> enabled.
>
> If you think this bug is significant, I can attempt the bug bisect in the
> environment that triggered it (Lenovo LENOVO_MT_10TX_BU_Lenovo_FM_V530S-07ICB)
> with BIOS M22KT49A from 11/10/2022 and AlmaLinux 8.7.
>
> Here is the /sys/kernel/debug/kmemleak output:
>
> unreferenced object 0xffff9e67ad71f160 (size 32):
>   comm "gpio-sim.sh", pid 208926, jiffies 4372229685 (age 2101.564s)
>   hex dump (first 32 bytes):
>     67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30  gpio-sim.0-node0
>     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>   backtrace:
>     [<0000000098bf3d1b>] slab_post_alloc_hook+0x91/0x320
>     [<00000000da3205c5>] __kmem_cache_alloc_node+0x1bf/0x2b0
>     [<00000000aa51a58a>] __kmalloc_node_track_caller+0x55/0x140
>     [<00000000bd682ecc>] kvasprintf+0x6b/0xd0
>     [<00000000a3431d55>] kasprintf+0x4e/0x70
>     [<00000000f52d2629>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>     [<00000000673fc6df>] configfs_write_iter+0xcc/0x130
>     [<000000001d5d0829>] vfs_write+0x2b4/0x3d0
>     [<00000000d2336251>] ksys_write+0x61/0xe0
>     [<00000000f7015bb1>] __x64_sys_write+0x1a/0x20
>     [<000000008ac743d2>] do_syscall_64+0x58/0x80
>     [<000000004d7b7d50>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
> [root@pc-mtodorov marvin]#

The new development on the bug is that it probably requires some superuser privileges
or some capability to be exploited, for it requires access to configfs,
but it was reproduced on different hardware as well.

The minimum reproducing script is attached, with its output log.

From the testing "age" of the unreferenced object I assume that it is allocated earlier
in a part of script I am unable to locate or specify, but orphaned at the end of the script.

root@/home/user/kernel_bugs/gpio-sim# time ./gpio-reproduce-min.sh
2.14. Lines can be hogged
Scanning stage 2.14.7 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.7 clean.
Rescanning stage 2.14.7 ... done.
Sleeping 60 seconds ... done.
unreferenced object 0xffff9593b9d16bc0 (size 32):
comm "gpio-reproduce-", pid 7594, jiffies 4295865460 (age 136.184s)
hex dump (first 32 bytes):
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
[<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
[<00000000c4efe87f>] kvasprintf+0x6b/0xd0
[<000000000c0f91cd>] kasprintf+0x4e/0x70
[<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
[<0000000052ce6759>] configfs_write_iter+0xcc/0x130
[<0000000006087fd2>] vfs_write+0x2b4/0x3d0
[<000000008a17e041>] ksys_write+0x61/0xe0
[<000000007bded8ea>] __x64_sys_write+0x1a/0x20
[<00000000e1220148>] do_syscall_64+0x58/0x80
[<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd

real 2m16.398s
user 0m0.023s
sys 0m16.360s
root@/home/user/kernel_bugs/gpio-sim#

The effect is cummulative:

root@marvin-IdeaPad-3-15ITL6:/home/user/kernel_bugs/gpio-sim# cat /sys/kernel/debug/kmemleak
unreferenced object 0xffff9593b9d16bc0 (size 32):
comm "gpio-reproduce-", pid 7594, jiffies 4295865460 (age 520.296s)
hex dump (first 32 bytes):
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
[<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
[<00000000c4efe87f>] kvasprintf+0x6b/0xd0
[<000000000c0f91cd>] kasprintf+0x4e/0x70
[<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
[<0000000052ce6759>] configfs_write_iter+0xcc/0x130
[<0000000006087fd2>] vfs_write+0x2b4/0x3d0
[<000000008a17e041>] ksys_write+0x61/0xe0
[<000000007bded8ea>] __x64_sys_write+0x1a/0x20
[<00000000e1220148>] do_syscall_64+0x58/0x80
[<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
unreferenced object 0xffff95938918fb40 (size 32):
comm "gpio-reproduce-", pid 7675, jiffies 4295954327 (age 164.832s)
hex dump (first 32 bytes):
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
[<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
[<00000000c4efe87f>] kvasprintf+0x6b/0xd0
[<000000000c0f91cd>] kasprintf+0x4e/0x70
[<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
[<0000000052ce6759>] configfs_write_iter+0xcc/0x130
[<0000000006087fd2>] vfs_write+0x2b4/0x3d0
[<000000008a17e041>] ksys_write+0x61/0xe0
[<000000007bded8ea>] __x64_sys_write+0x1a/0x20
[<00000000e1220148>] do_syscall_64+0x58/0x80
[<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
unreferenced object 0xffff9594a3cf1820 (size 32):
comm "gpio-reproduce-", pid 7721, jiffies 4295976853 (age 74.728s)
hex dump (first 32 bytes):
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
[<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
[<00000000c4efe87f>] kvasprintf+0x6b/0xd0
[<000000000c0f91cd>] kasprintf+0x4e/0x70
[<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
[<0000000052ce6759>] configfs_write_iter+0xcc/0x130
[<0000000006087fd2>] vfs_write+0x2b4/0x3d0
[<000000008a17e041>] ksys_write+0x61/0xe0
[<000000007bded8ea>] __x64_sys_write+0x1a/0x20
[<00000000e1220148>] do_syscall_64+0x58/0x80
[<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
root@marvin-IdeaPad-3-15ITL6:/home/user/kernel_bugs/gpio-sim#

With sufficient privileges (or possibly even without them), an exploit can be devised
to automate allocation of orphaned objects, at a rate of a couple per second (PoC attached).

Regards,

--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu

System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
The European Union

[-- Attachment #2: gpio-trace.log --]
[-- Type: text/x-log, Size: 2418 bytes --]

2.14. Lines can be hogged
Scanning stage 2.14.0 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.0 clean.
Rescanning stage 2.14.0 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.0 re-check clean.
Re-rescanning stage 2.14.0 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.0 re-check clean.
Scanning stage 2.14.1 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.1 clean.
Rescanning stage 2.14.1 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.1 re-check clean.
Re-rescanning stage 2.14.1 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.1 re-check clean.
Scanning stage 2.14.2 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.2 clean.
Rescanning stage 2.14.2 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.2 re-check clean.
Re-rescanning stage 2.14.2 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.2 re-check clean.
Scanning stage 2.14.3 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.3 clean.
Rescanning stage 2.14.3 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.3 re-check clean.
Re-rescanning stage 2.14.3 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.3 re-check clean.
Scanning stage 2.14.4 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.4 clean.
Rescanning stage 2.14.4 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.4 re-check clean.
Re-rescanning stage 2.14.4 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.4 re-check clean.
Scanning stage 2.14.5 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.5 clean.
Rescanning stage 2.14.5 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.5 re-check clean.
Re-rescanning stage 2.14.5 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.5 re-check clean.
Scanning stage 2.14.6 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.6 clean.
Rescanning stage 2.14.6 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.6 re-check clean.
Re-rescanning stage 2.14.6 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.6 re-check clean.
Scanning stage 2.14.7 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.7 clean.
Rescanning stage 2.14.7 ... done.
Sleeping 60 seconds ... done.
comm "gpio-sim-reprod", pid 74359, jiffies 4317617873 (age 531.940s)
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
Re-rescanning stage 2.14.7 ... done.
Sleeping 60 seconds ... done.
comm "gpio-sim-reprod", pid 74359, jiffies 4317617873 (age 598.480s)
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0

[-- Attachment #3: gpio-reproduce-min.sh --]
[-- Type: application/x-shellscript, Size: 5067 bytes --]

[-- Attachment #4: gpio-min-iterate.sh --]
[-- Type: application/x-shellscript, Size: 5166 bytes --]