[PATCH] batman-adv: Broken sync while rescheduling delayed work

[PATCH] batman-adv: Broken sync while rescheduling delayed work

[Vladislav Efanov]

Syzkaller got a lot of crashes like:
KASAN: use-after-free Write in *_timers*

All of these crashes point to the same memory area:

The buggy address belongs to the object at ffff88801f870000
 which belongs to the cache kmalloc-8k of size 8192
The buggy address is located 5320 bytes inside of
 8192-byte region [ffff88801f870000, ffff88801f872000)

This area belongs to :
        batadv_priv->batadv_priv_dat->delayed_work->timer_list

The reason for these issues is the lack of synchronization. Delayed
work (batadv_dat_purge) schedules new timer/work while the device
is being deleted. As the result new timer/delayed work is set after
cancel_delayed_work_sync() was called. So after the device is freed
the timer list contains pointer to already freed memory.

Found by Linux Verification Center (linuxtesting.org) with syzkaller.

Fixes: 2f1dfbe18507 ("batman-adv: Distributed ARP Table - implement local storage")
Signed-off-by: Vladislav Efanov <VEfanov@ispras.ru>
---
 net/batman-adv/distributed-arp-table.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/batman-adv/distributed-arp-table.c b/net/batman-adv/distributed-arp-table.c
index 6968e55eb971..28a939d56090 100644
--- a/net/batman-adv/distributed-arp-table.c
+++ b/net/batman-adv/distributed-arp-table.c
@@ -101,7 +101,6 @@ static void batadv_dat_purge(struct work_struct *work);
  */
 static void batadv_dat_start_timer(struct batadv_priv *bat_priv)
 {
-	INIT_DELAYED_WORK(&bat_priv->dat.work, batadv_dat_purge);
 	queue_delayed_work(batadv_event_workqueue, &bat_priv->dat.work,
 			   msecs_to_jiffies(10000));
 }
@@ -819,6 +818,7 @@ int batadv_dat_init(struct batadv_priv *bat_priv)
 	if (!bat_priv->dat.hash)
 		return -ENOMEM;
 
+	INIT_DELAYED_WORK(&bat_priv->dat.work, batadv_dat_purge);
 	batadv_dat_start_timer(bat_priv);
 
 	batadv_tvlv_handler_register(bat_priv, batadv_dat_tvlv_ogm_handler_v1,
-- 
2.34.1

Re: [PATCH] batman-adv: Broken sync while rescheduling delayed work

[Sven Eckelmann]

[-- Attachment #1: Type: text/plain, Size: 877 bytes --]

On Friday, 26 May 2023 18:16:32 CEST Vladislav Efanov wrote:
> The reason for these issues is the lack of synchronization. Delayed
> work (batadv_dat_purge) schedules new timer/work while the device
> is being deleted. As the result new timer/delayed work is set after
> cancel_delayed_work_sync() was called. So after the device is freed
> the timer list contains pointer to already freed memory.

You are most likely right but could you please point out what in the worker is 
checked by the workqueue code that prevents it from being scheduled again?  
(and which seems to be overwritten as your patch seems to suggest)

I think __cancel_work_timer marked the work as canceling but 
batadv_dat_start_timer reinitialized the worked (thus removing this important 
state). Would be nice if you could either correct me or confirm what I think to 
remember.

Kind regards,
	Sven

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] batman-adv: Broken sync while rescheduling delayed work

[Vlad Efanov]

Sven,


cancel_delayed_work_sync() and queue_delayed_work()

use WORK_STRUCT_PENDING_BIT in work->data to synchronize.

INIT_DELAYED_WORK() clears this bit.


The situation is :  __cancel_work_timer() sets WORK_STRUCT_PENDING_BIT

but INIT_DELAYED_WORK() in batadv_dat_start_timer() clears it

and queue_delayed_work() schedules new work.


Best regards,

Vlad.

On 26.05.2023 19:49, Sven Eckelmann wrote:
> On Friday, 26 May 2023 18:16:32 CEST Vladislav Efanov wrote:
>> The reason for these issues is the lack of synchronization. Delayed
>> work (batadv_dat_purge) schedules new timer/work while the device
>> is being deleted. As the result new timer/delayed work is set after
>> cancel_delayed_work_sync() was called. So after the device is freed
>> the timer list contains pointer to already freed memory.
> You are most likely right but could you please point out what in the worker is
> checked by the workqueue code that prevents it from being scheduled again?
> (and which seems to be overwritten as your patch seems to suggest)
>
> I think __cancel_work_timer marked the work as canceling but
> batadv_dat_start_timer reinitialized the worked (thus removing this important
> state). Would be nice if you could either correct me or confirm what I think to
> remember.
>
> Kind regards,
> 	Sven

Re: [PATCH] batman-adv: Broken sync while rescheduling delayed work

[Sven Eckelmann]

[-- Attachment #1: Type: text/plain, Size: 1147 bytes --]

On Friday, 26 May 2023 18:16:32 CEST Vladislav Efanov wrote:
> Syzkaller got a lot of crashes like:
> KASAN: use-after-free Write in *_timers*
> 
> All of these crashes point to the same memory area:
> 
> The buggy address belongs to the object at ffff88801f870000
>  which belongs to the cache kmalloc-8k of size 8192
> The buggy address is located 5320 bytes inside of
>  8192-byte region [ffff88801f870000, ffff88801f872000)
> 
> This area belongs to :
>         batadv_priv->batadv_priv_dat->delayed_work->timer_list
> 
> The reason for these issues is the lack of synchronization. Delayed
> work (batadv_dat_purge) schedules new timer/work while the device
> is being deleted. As the result new timer/delayed work is set after
> cancel_delayed_work_sync() was called. So after the device is freed
> the timer list contains pointer to already freed memory.
> 
> Found by Linux Verification Center (linuxtesting.org) with syzkaller.
> 
> Fixes: 2f1dfbe18507 ("batman-adv: Distributed ARP Table - implement local storage")
> Signed-off-by: Vladislav Efanov <VEfanov@ispras.ru>
> ---


Acked-by: Sven Eckelmann <sven@narfation.org>

Thanks,
	Sven

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]