[PATCH libgpiod] gpiosim: fix data race that corrupts heap

[PATCH libgpiod] gpiosim: fix data race that corrupts heap

[Erik Schilling]

Hit this while seeing some heap corruptions when running cargo test on
the Rust bindings.

Took a bit to track down since I first used address sanitizers, but with
the thread sanitizer it becomes obvious immediately (output simplified):

    ==================
    WARNING: ThreadSanitizer: data race (pid=288119)
      Write of size 8 at 0x0000018f1e78 by thread T6:
        #0 id_free /libgpiod/tests/gpiosim/gpiosim.c:141:17
        #1 dev_release /libgpiod/tests/gpiosim/gpiosim.c:600:2
        #2 refcount_dec /libgpiod/tests/gpiosim/gpiosim.c:176:3
        #3 gpiosim_dev_unref /libgpiod/tests/gpiosim/gpiosim.c:671:2
        #4 bank_release /libgpiod/tests/gpiosim/gpiosim.c:873:2
        #5 refcount_dec /libgpiod/tests/gpiosim/gpiosim.c:176:3
        #6 gpiosim_bank_unref /libgpiod/tests/gpiosim/gpiosim.c:941:2
        [...]

      Previous write of size 8 at 0x0000018f1e78 by thread T1:
        #0 id_free /libgpiod/tests/gpiosim/gpiosim.c:141:17
        #1 bank_release /libgpiod/tests/gpiosim/gpiosim.c:878:2
        #2 refcount_dec /libgpiod/tests/gpiosim/gpiosim.c:176:3
        #3 gpiosim_bank_unref /libgpiod/tests/gpiosim/gpiosim.c:941:2
        [...]

      Location is global 'id_del_ctx' of size 16 at 0x0000018f1e70

      Thread T6 'chip::verify::f' (tid=288126, running) created by main thread at:
        #7 test::run_tests::hd53a07a011bd771f /.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:407:21
        [...]

      Thread T1 'chip::open::gpi' (tid=288121, finished) created by main thread at:
        #7 test::run_tests::hd53a07a011bd771f /.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:407:21
        [...]

    SUMMARY: ThreadSanitizer: data race /libgpiod/tests/gpiosim/gpiosim.c:141:17 in id_free
    ==================

This eventually can either lead to leaks or double free's that corrupt
the heap and lead to crashes.

The issue got introduced when a previously local variable that did not
require protection was turned into a global variable.

Fixes: 5e111df2fca56d57193a1825e45e78dd8b76c0f1
Signed-off-by: Erik Schilling <erik.schilling@linaro.org>
---
To: Linux-GPIO <linux-gpio@vger.kernel.org>
Cc: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
---
 tests/gpiosim/gpiosim.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/gpiosim/gpiosim.c b/tests/gpiosim/gpiosim.c
index b49a61a..fca6b7f 100644
--- a/tests/gpiosim/gpiosim.c
+++ b/tests/gpiosim/gpiosim.c
@@ -137,11 +137,11 @@ static int id_alloc(void)
 
 static void id_free(int id)
 {
+	pthread_mutex_lock(&id_lock);
+
 	id_del_ctx.id = id;
 	id_del_ctx.idp = NULL;
 
-	pthread_mutex_lock(&id_lock);
-
 	twalk(id_root, id_del);
 	if (id_del_ctx.idp) {
 		tdelete(id_del_ctx.idp, &id_root, id_compare);

---
base-commit: d04639ddd11ed7d02c630e693bf07d97f53e17d3
change-id: 20230626-datarace-e62e9bcfa3ee

Best regards,
-- 
Erik Schilling <erik.schilling@linaro.org>

Re: [PATCH libgpiod] gpiosim: fix data race that corrupts heap

[Bartosz Golaszewski]

On Mon, Jun 26, 2023 at 3:15 PM Erik Schilling
<erik.schilling@linaro.org> wrote:
>
> Hit this while seeing some heap corruptions when running cargo test on
> the Rust bindings.
>
> Took a bit to track down since I first used address sanitizers, but with
> the thread sanitizer it becomes obvious immediately (output simplified):
>
>     ==================
>     WARNING: ThreadSanitizer: data race (pid=288119)
>       Write of size 8 at 0x0000018f1e78 by thread T6:
>         #0 id_free /libgpiod/tests/gpiosim/gpiosim.c:141:17
>         #1 dev_release /libgpiod/tests/gpiosim/gpiosim.c:600:2
>         #2 refcount_dec /libgpiod/tests/gpiosim/gpiosim.c:176:3
>         #3 gpiosim_dev_unref /libgpiod/tests/gpiosim/gpiosim.c:671:2
>         #4 bank_release /libgpiod/tests/gpiosim/gpiosim.c:873:2
>         #5 refcount_dec /libgpiod/tests/gpiosim/gpiosim.c:176:3
>         #6 gpiosim_bank_unref /libgpiod/tests/gpiosim/gpiosim.c:941:2
>         [...]
>
>       Previous write of size 8 at 0x0000018f1e78 by thread T1:
>         #0 id_free /libgpiod/tests/gpiosim/gpiosim.c:141:17
>         #1 bank_release /libgpiod/tests/gpiosim/gpiosim.c:878:2
>         #2 refcount_dec /libgpiod/tests/gpiosim/gpiosim.c:176:3
>         #3 gpiosim_bank_unref /libgpiod/tests/gpiosim/gpiosim.c:941:2
>         [...]
>
>       Location is global 'id_del_ctx' of size 16 at 0x0000018f1e70
>
>       Thread T6 'chip::verify::f' (tid=288126, running) created by main thread at:
>         #7 test::run_tests::hd53a07a011bd771f /.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:407:21
>         [...]
>
>       Thread T1 'chip::open::gpi' (tid=288121, finished) created by main thread at:
>         #7 test::run_tests::hd53a07a011bd771f /.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:407:21
>         [...]
>
>     SUMMARY: ThreadSanitizer: data race /libgpiod/tests/gpiosim/gpiosim.c:141:17 in id_free
>     ==================
>
> This eventually can either lead to leaks or double free's that corrupt
> the heap and lead to crashes.
>
> The issue got introduced when a previously local variable that did not
> require protection was turned into a global variable.
>
> Fixes: 5e111df2fca56d57193a1825e45e78dd8b76c0f1
> Signed-off-by: Erik Schilling <erik.schilling@linaro.org>
> ---
> To: Linux-GPIO <linux-gpio@vger.kernel.org>
> Cc: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
> ---
>  tests/gpiosim/gpiosim.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/tests/gpiosim/gpiosim.c b/tests/gpiosim/gpiosim.c
> index b49a61a..fca6b7f 100644
> --- a/tests/gpiosim/gpiosim.c
> +++ b/tests/gpiosim/gpiosim.c
> @@ -137,11 +137,11 @@ static int id_alloc(void)
>
>  static void id_free(int id)
>  {
> +       pthread_mutex_lock(&id_lock);
> +
>         id_del_ctx.id = id;
>         id_del_ctx.idp = NULL;
>
> -       pthread_mutex_lock(&id_lock);
> -
>         twalk(id_root, id_del);
>         if (id_del_ctx.idp) {
>                 tdelete(id_del_ctx.idp, &id_root, id_compare);
>
> ---
> base-commit: d04639ddd11ed7d02c630e693bf07d97f53e17d3
> change-id: 20230626-datarace-e62e9bcfa3ee
>
> Best regards,
> --
> Erik Schilling <erik.schilling@linaro.org>
>

Nice catch, thanks! I applied it and just changed the Fixes tag to
conform to the kernel style.

I actually saw some issues when running rust tests some time ago but
couldn't reproduce reliably, I suspect this was it.

Bart

Re: [PATCH libgpiod] gpiosim: fix data race that corrupts heap

[Erik Schilling]

On Mon Jun 26, 2023 at 4:36 PM CEST, Bartosz Golaszewski wrote:
> Nice catch, thanks!
> I applied it and just changed the Fixes tag to
> conform to the kernel style.

Thanks! I updated my gitconfig with the correct pretty format.

> I actually saw some issues when running rust tests some time ago but
> couldn't reproduce reliably, I suspect this was it.

Yeah, it was a bit nasty since this mixed up the double-free backtraces
really bad. I was starting at the ref counters and Rust abstractions
for quite a while and could not make sense of it. And once one added
too many debug prints the chance of reproducing it got reduced pretty
severly on my machine.

- Erik