[PATCH 1/3] net: sched: fix use-after-free in taprio_change()

[PATCH 1/3] net: sched: fix use-after-free in taprio_change()

[Dmitry Antipov]

In 'taprio_change()', 'admin' pointer may become dangling due to sched
switch / removal caused by 'advance_sched()', and critical section
protected by 'q->current_entry_lock' is too small to prevent from such
a scenario (which causes use-after-free detected by KASAN). Fix this
by prefer 'rcu_replace_pointer()' over 'rcu_assign_pointer()' to update
'admin' immediately before an attempt to schedule freeing.

Fixes: a3d43c0d56f1 ("taprio: Add support adding an admin schedule")
Reported-by: syzbot+b65e0af58423fc8a73aa@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=b65e0af58423fc8a73aa
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
---
 net/sched/sch_taprio.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index cc2df9f8c14a..59fad74d5ff9 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -1963,7 +1963,8 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
 
 		taprio_start_sched(sch, start, new_admin);
 
-		rcu_assign_pointer(q->admin_sched, new_admin);
+		admin = rcu_replace_pointer(q->admin_sched, new_admin,
+					    lockdep_rtnl_is_held());
 		if (admin)
 			call_rcu(&admin->rcu, taprio_free_sched_cb);
 
-- 
2.45.2

[PATCH 2/3] net: sched: consistently use rcu_replace_pointer() in taprio_change()

[Dmitry Antipov]

According to Vinicius (and carefully looking through the whole thing
once again), txtime branch of 'taprio_change()' is not going to race
against 'advance_sched()'. But using 'rcu_replace_pointer()' in the
former may be a good idea as well.

Suggested-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
---
 net/sched/sch_taprio.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index 59fad74d5ff9..9f4e004cdb8b 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -1952,7 +1952,9 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
 			goto unlock;
 		}
 
-		rcu_assign_pointer(q->admin_sched, new_admin);
+		/* Not going to race against advance_sched(), but still */
+		admin = rcu_replace_pointer(q->admin_sched, new_admin,
+					    lockdep_rtnl_is_held());
 		if (admin)
 			call_rcu(&admin->rcu, taprio_free_sched_cb);
 	} else {
-- 
2.45.2

[PATCH 3/3] net: sched: use RCU read-side critical section in taprio_dump()

[Dmitry Antipov]

Not sure why this is not reproducible on x86, but I occasionally see
the following crash on arm64 (note that original issue was found at
https://syzkaller.appspot.com/bug?extid=b65e0af58423fc8a73aa on arm64):

[ 1601.079132][T15862] BUG: KASAN: slab-use-after-free in taprio_dump+0xa0c/0xbb0
[ 1601.082101][T15862] Read of size 4 at addr ffff0000d4bb88f8 by task repro/15862
[ 1601.085149][T15862]
[ 1601.093445][T15862] CPU: 0 UID: 0 PID: 15862 Comm: repro Not tainted 6.11.0-rc1-00293-gdefaf1a2113a-dirty #2
[ 1601.100771][T15862] Hardware name: QEMU QEMU Virtual Machine, BIOS edk2-20240524-5.fc40 05/24/2024
[ 1601.106651][T15862] Call trace:
[ 1601.107395][T15862]  dump_backtrace+0x20c/0x220
[ 1601.108397][T15862]  show_stack+0x2c/0x40
[ 1601.109220][T15862]  dump_stack_lvl+0xf8/0x174
[ 1601.110041][T15862]  print_report+0x170/0x4d8
[ 1601.110848][T15862]  kasan_report+0xb8/0x1d4
[ 1601.111991][T15862]  __asan_report_load4_noabort+0x20/0x2c
[ 1601.112880][T15862]  taprio_dump+0xa0c/0xbb0
[ 1601.113725][T15862]  tc_fill_qdisc+0x540/0x1020
[ 1601.114586][T15862]  qdisc_notify.isra.0+0x330/0x3a0
[ 1601.115506][T15862]  tc_modify_qdisc+0x7b8/0x1838
[ 1601.116378][T15862]  rtnetlink_rcv_msg+0x3c8/0xc20
[ 1601.117320][T15862]  netlink_rcv_skb+0x1f8/0x3d4
[ 1601.118164][T15862]  rtnetlink_rcv+0x28/0x40
[ 1601.119037][T15862]  netlink_unicast+0x51c/0x790
[ 1601.119874][T15862]  netlink_sendmsg+0x79c/0xc20
[ 1601.120706][T15862]  __sock_sendmsg+0xe0/0x1a0
[ 1601.121802][T15862]  ____sys_sendmsg+0x6c0/0x840
[ 1601.122722][T15862]  ___sys_sendmsg+0x1ac/0x1f0
[ 1601.123653][T15862]  __sys_sendmsg+0x110/0x1d0
[ 1601.124459][T15862]  __arm64_sys_sendmsg+0x74/0xb0
[ 1601.125316][T15862]  invoke_syscall+0x88/0x2e0
[ 1601.126155][T15862]  el0_svc_common.constprop.0+0xe4/0x2a0
[ 1601.127051][T15862]  do_el0_svc+0x44/0x60
[ 1601.127837][T15862]  el0_svc+0x50/0x184
[ 1601.128639][T15862]  el0t_64_sync_handler+0x120/0x12c
[ 1601.129505][T15862]  el0t_64_sync+0x190/0x194
[ 1601.130591][T15862]
[ 1601.131361][T15862] Allocated by task 15857:
[ 1601.132224][T15862]  kasan_save_stack+0x3c/0x70
[ 1601.133193][T15862]  kasan_save_track+0x20/0x3c
[ 1601.134102][T15862]  kasan_save_alloc_info+0x40/0x60
[ 1601.134955][T15862]  __kasan_kmalloc+0xd4/0xe0
[ 1601.135965][T15862]  __kmalloc_cache_noprof+0x194/0x334
[ 1601.136874][T15862]  taprio_change+0x45c/0x2fe0
[ 1601.137859][T15862]  tc_modify_qdisc+0x6a8/0x1838
[ 1601.138838][T15862]  rtnetlink_rcv_msg+0x3c8/0xc20
[ 1601.139799][T15862]  netlink_rcv_skb+0x1f8/0x3d4
[ 1601.140664][T15862]  rtnetlink_rcv+0x28/0x40
[ 1601.141725][T15862]  netlink_unicast+0x51c/0x790
[ 1601.142662][T15862]  netlink_sendmsg+0x79c/0xc20
[ 1601.143523][T15862]  __sock_sendmsg+0xe0/0x1a0
[ 1601.144445][T15862]  ____sys_sendmsg+0x6c0/0x840
[ 1601.145467][T15862]  ___sys_sendmsg+0x1ac/0x1f0
[ 1601.146410][T15862]  __sys_sendmsg+0x110/0x1d0
[ 1601.147293][T15862]  __arm64_sys_sendmsg+0x74/0xb0
[ 1601.148116][T15862]  invoke_syscall+0x88/0x2e0
[ 1601.148912][T15862]  el0_svc_common.constprop.0+0xe4/0x2a0
[ 1601.149754][T15862]  do_el0_svc+0x44/0x60
[ 1601.150532][T15862]  el0_svc+0x50/0x184
[ 1601.151438][T15862]  el0t_64_sync_handler+0x120/0x12c
[ 1601.152311][T15862]  el0t_64_sync+0x190/0x194
[ 1601.153208][T15862]
[ 1601.153751][T15862] Freed by task 6192:
[ 1601.154491][T15862]  kasan_save_stack+0x3c/0x70
[ 1601.155491][T15862]  kasan_save_track+0x20/0x3c
[ 1601.156521][T15862]  kasan_save_free_info+0x4c/0x80
[ 1601.157357][T15862]  poison_slab_object+0x110/0x160
[ 1601.158300][T15862]  __kasan_slab_free+0x3c/0x74
[ 1601.159265][T15862]  kfree+0x134/0x3c0
[ 1601.160068][T15862]  taprio_free_sched_cb+0x18c/0x220
[ 1601.161046][T15862]  rcu_core+0x920/0x1b7c
[ 1601.161906][T15862]  rcu_core_si+0x10/0x1c
[ 1601.162693][T15862]  handle_softirqs+0x2e8/0xd64
[ 1601.163518][T15862]  __do_softirq+0x14/0x20

Fix this by adding RCU read-side critical section to 'taprio_dump()'.

Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
---
 net/sched/sch_taprio.c | 37 +++++++++++++++++++++----------------
 1 file changed, 21 insertions(+), 16 deletions(-)

diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index 9f4e004cdb8b..f31feca381c4 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -2374,9 +2374,6 @@ static int taprio_dump(struct Qdisc *sch, struct sk_buff *skb)
 	struct tc_mqprio_qopt opt = { 0 };
 	struct nlattr *nest, *sched_nest;
 
-	oper = rtnl_dereference(q->oper_sched);
-	admin = rtnl_dereference(q->admin_sched);
-
 	mqprio_qopt_reconstruct(dev, &opt);
 
 	nest = nla_nest_start_noflag(skb, TCA_OPTIONS);
@@ -2397,29 +2394,37 @@ static int taprio_dump(struct Qdisc *sch, struct sk_buff *skb)
 	    nla_put_u32(skb, TCA_TAPRIO_ATTR_TXTIME_DELAY, q->txtime_delay))
 		goto options_error;
 
+	rcu_read_lock();
+
+	oper = rtnl_dereference(q->oper_sched);
+	admin = rtnl_dereference(q->admin_sched);
+
 	if (oper && taprio_dump_tc_entries(skb, q, oper))
-		goto options_error;
+		goto unlock;
 
 	if (oper && dump_schedule(skb, oper))
-		goto options_error;
+		goto unlock;
 
-	if (!admin)
-		goto done;
+	if (admin) {
+		sched_nest =
+			nla_nest_start_noflag(skb, TCA_TAPRIO_ATTR_ADMIN_SCHED);
+		if (!sched_nest)
+			goto unlock;
 
-	sched_nest = nla_nest_start_noflag(skb, TCA_TAPRIO_ATTR_ADMIN_SCHED);
-	if (!sched_nest)
-		goto options_error;
+		if (dump_schedule(skb, admin)) {
+			nla_nest_cancel(skb, sched_nest);
+			goto unlock;
+		}
 
-	if (dump_schedule(skb, admin))
-		goto admin_error;
+		nla_nest_end(skb, sched_nest);
+	}
 
-	nla_nest_end(skb, sched_nest);
+	rcu_read_unlock();
 
-done:
 	return nla_nest_end(skb, nest);
 
-admin_error:
-	nla_nest_cancel(skb, sched_nest);
+unlock:
+	rcu_read_unlock();
 
 options_error:
 	nla_nest_cancel(skb, nest);
-- 
2.45.2

Re: [PATCH 3/3] net: sched: use RCU read-side critical section in taprio_dump()

[Vinicius Costa Gomes]

Dmitry Antipov <dmantipov@yandex.ru> writes:



> Not sure why this is not reproducible on x86, but I occasionally see
> the following crash on arm64 (note that original issue was found at
> https://syzkaller.appspot.com/bug?extid=b65e0af58423fc8a73aa on arm64):
>
> [ 1601.079132][T15862] BUG: KASAN: slab-use-after-free in taprio_dump+0xa0c/0xbb0
> [ 1601.082101][T15862] Read of size 4 at addr ffff0000d4bb88f8 by task repro/15862
> [ 1601.085149][T15862]
> [ 1601.093445][T15862] CPU: 0 UID: 0 PID: 15862 Comm: repro Not tainted 6.11.0-rc1-00293-gdefaf1a2113a-dirty #2
> [ 1601.100771][T15862] Hardware name: QEMU QEMU Virtual Machine, BIOS edk2-20240524-5.fc40 05/24/2024
> [ 1601.106651][T15862] Call trace:
> [ 1601.107395][T15862]  dump_backtrace+0x20c/0x220
> [ 1601.108397][T15862]  show_stack+0x2c/0x40
> [ 1601.109220][T15862]  dump_stack_lvl+0xf8/0x174
> [ 1601.110041][T15862]  print_report+0x170/0x4d8
> [ 1601.110848][T15862]  kasan_report+0xb8/0x1d4
> [ 1601.111991][T15862]  __asan_report_load4_noabort+0x20/0x2c
> [ 1601.112880][T15862]  taprio_dump+0xa0c/0xbb0
> [ 1601.113725][T15862]  tc_fill_qdisc+0x540/0x1020
> [ 1601.114586][T15862]  qdisc_notify.isra.0+0x330/0x3a0
> [ 1601.115506][T15862]  tc_modify_qdisc+0x7b8/0x1838
> [ 1601.116378][T15862]  rtnetlink_rcv_msg+0x3c8/0xc20
> [ 1601.117320][T15862]  netlink_rcv_skb+0x1f8/0x3d4
> [ 1601.118164][T15862]  rtnetlink_rcv+0x28/0x40
> [ 1601.119037][T15862]  netlink_unicast+0x51c/0x790
> [ 1601.119874][T15862]  netlink_sendmsg+0x79c/0xc20
> [ 1601.120706][T15862]  __sock_sendmsg+0xe0/0x1a0
> [ 1601.121802][T15862]  ____sys_sendmsg+0x6c0/0x840
> [ 1601.122722][T15862]  ___sys_sendmsg+0x1ac/0x1f0
> [ 1601.123653][T15862]  __sys_sendmsg+0x110/0x1d0
> [ 1601.124459][T15862]  __arm64_sys_sendmsg+0x74/0xb0
> [ 1601.125316][T15862]  invoke_syscall+0x88/0x2e0
> [ 1601.126155][T15862]  el0_svc_common.constprop.0+0xe4/0x2a0
> [ 1601.127051][T15862]  do_el0_svc+0x44/0x60
> [ 1601.127837][T15862]  el0_svc+0x50/0x184
> [ 1601.128639][T15862]  el0t_64_sync_handler+0x120/0x12c
> [ 1601.129505][T15862]  el0t_64_sync+0x190/0x194
> [ 1601.130591][T15862]
> [ 1601.131361][T15862] Allocated by task 15857:
> [ 1601.132224][T15862]  kasan_save_stack+0x3c/0x70
> [ 1601.133193][T15862]  kasan_save_track+0x20/0x3c
> [ 1601.134102][T15862]  kasan_save_alloc_info+0x40/0x60
> [ 1601.134955][T15862]  __kasan_kmalloc+0xd4/0xe0
> [ 1601.135965][T15862]  __kmalloc_cache_noprof+0x194/0x334
> [ 1601.136874][T15862]  taprio_change+0x45c/0x2fe0
> [ 1601.137859][T15862]  tc_modify_qdisc+0x6a8/0x1838
> [ 1601.138838][T15862]  rtnetlink_rcv_msg+0x3c8/0xc20
> [ 1601.139799][T15862]  netlink_rcv_skb+0x1f8/0x3d4
> [ 1601.140664][T15862]  rtnetlink_rcv+0x28/0x40
> [ 1601.141725][T15862]  netlink_unicast+0x51c/0x790
> [ 1601.142662][T15862]  netlink_sendmsg+0x79c/0xc20
> [ 1601.143523][T15862]  __sock_sendmsg+0xe0/0x1a0
> [ 1601.144445][T15862]  ____sys_sendmsg+0x6c0/0x840
> [ 1601.145467][T15862]  ___sys_sendmsg+0x1ac/0x1f0
> [ 1601.146410][T15862]  __sys_sendmsg+0x110/0x1d0
> [ 1601.147293][T15862]  __arm64_sys_sendmsg+0x74/0xb0
> [ 1601.148116][T15862]  invoke_syscall+0x88/0x2e0
> [ 1601.148912][T15862]  el0_svc_common.constprop.0+0xe4/0x2a0
> [ 1601.149754][T15862]  do_el0_svc+0x44/0x60
> [ 1601.150532][T15862]  el0_svc+0x50/0x184
> [ 1601.151438][T15862]  el0t_64_sync_handler+0x120/0x12c
> [ 1601.152311][T15862]  el0t_64_sync+0x190/0x194
> [ 1601.153208][T15862]
> [ 1601.153751][T15862] Freed by task 6192:
> [ 1601.154491][T15862]  kasan_save_stack+0x3c/0x70
> [ 1601.155491][T15862]  kasan_save_track+0x20/0x3c
> [ 1601.156521][T15862]  kasan_save_free_info+0x4c/0x80
> [ 1601.157357][T15862]  poison_slab_object+0x110/0x160
> [ 1601.158300][T15862]  __kasan_slab_free+0x3c/0x74
> [ 1601.159265][T15862]  kfree+0x134/0x3c0
> [ 1601.160068][T15862]  taprio_free_sched_cb+0x18c/0x220
> [ 1601.161046][T15862]  rcu_core+0x920/0x1b7c
> [ 1601.161906][T15862]  rcu_core_si+0x10/0x1c
> [ 1601.162693][T15862]  handle_softirqs+0x2e8/0xd64
> [ 1601.163518][T15862]  __do_softirq+0x14/0x20
>
> Fix this by adding RCU read-side critical section to 'taprio_dump()'.
>

I would move this last sentence to the first line, as it is the most
important part of the commit message. Something like this: 

   Fix possible use-after-free error by adding RCU read-side critical
   section to 'taprio_dump()'.

   On a KASAN enabled arm64 system, the following crash ocasionally
   happens (note that original issue was found at
   https://syzkaller.appspot.com/bug?extid=b65e0af58423fc8a73aa):

   <SPLAT HERE>

Apart from that (and the process comment from Jakub) the code looks
good:

Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>

> Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
> ---
>  net/sched/sch_taprio.c | 37 +++++++++++++++++++++----------------
>  1 file changed, 21 insertions(+), 16 deletions(-)
>
> diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
> index 9f4e004cdb8b..f31feca381c4 100644
> --- a/net/sched/sch_taprio.c
> +++ b/net/sched/sch_taprio.c
> @@ -2374,9 +2374,6 @@ static int taprio_dump(struct Qdisc *sch, struct sk_buff *skb)
>  	struct tc_mqprio_qopt opt = { 0 };
>  	struct nlattr *nest, *sched_nest;
>  
> -	oper = rtnl_dereference(q->oper_sched);
> -	admin = rtnl_dereference(q->admin_sched);
> -
>  	mqprio_qopt_reconstruct(dev, &opt);
>  
>  	nest = nla_nest_start_noflag(skb, TCA_OPTIONS);
> @@ -2397,29 +2394,37 @@ static int taprio_dump(struct Qdisc *sch, struct sk_buff *skb)
>  	    nla_put_u32(skb, TCA_TAPRIO_ATTR_TXTIME_DELAY, q->txtime_delay))
>  		goto options_error;
>  
> +	rcu_read_lock();
> +
> +	oper = rtnl_dereference(q->oper_sched);
> +	admin = rtnl_dereference(q->admin_sched);
> +
>  	if (oper && taprio_dump_tc_entries(skb, q, oper))
> -		goto options_error;
> +		goto unlock;
>  
>  	if (oper && dump_schedule(skb, oper))
> -		goto options_error;
> +		goto unlock;
>  
> -	if (!admin)
> -		goto done;
> +	if (admin) {
> +		sched_nest =
> +			nla_nest_start_noflag(skb, TCA_TAPRIO_ATTR_ADMIN_SCHED);
> +		if (!sched_nest)
> +			goto unlock;
>  
> -	sched_nest = nla_nest_start_noflag(skb, TCA_TAPRIO_ATTR_ADMIN_SCHED);
> -	if (!sched_nest)
> -		goto options_error;
> +		if (dump_schedule(skb, admin)) {
> +			nla_nest_cancel(skb, sched_nest);
> +			goto unlock;
> +		}
>  
> -	if (dump_schedule(skb, admin))
> -		goto admin_error;
> +		nla_nest_end(skb, sched_nest);
> +	}
>  
> -	nla_nest_end(skb, sched_nest);
> +	rcu_read_unlock();
>  
> -done:
>  	return nla_nest_end(skb, nest);
>  
> -admin_error:
> -	nla_nest_cancel(skb, sched_nest);
> +unlock:
> +	rcu_read_unlock();
>  
>  options_error:
>  	nla_nest_cancel(skb, nest);
> -- 
> 2.45.2
>

-- 
Vinicius

Re: [PATCH 1/3] net: sched: fix use-after-free in taprio_change()

[Jakub Kicinski]

On Wed,  7 Aug 2024 13:39:41 +0300 Dmitry Antipov wrote:
> In 'taprio_change()', 'admin' pointer may become dangling due to sched
> switch / removal caused by 'advance_sched()', and critical section
> protected by 'q->current_entry_lock' is too small to prevent from such
> a scenario (which causes use-after-free detected by KASAN). Fix this
> by prefer 'rcu_replace_pointer()' over 'rcu_assign_pointer()' to update
> 'admin' immediately before an attempt to schedule freeing.
> 
> Fixes: a3d43c0d56f1 ("taprio: Add support adding an admin schedule")
> Reported-by: syzbot+b65e0af58423fc8a73aa@syzkaller.appspotmail.com
> Closes: https://syzkaller.appspot.com/bug?extid=b65e0af58423fc8a73aa
> Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>

No need to repost (yet?) but quick process note, please err on the side
of incrementing patch versions, this should be a v2 even if only diff 
is that there are new patches. The version is for the _series_.

https://lore.kernel.org/all/20240805135145.37604-1-dmantipov@yandex.ru/