[syzbot] BUG: sleeping function called from invalid context in static_key_slow_inc

[syzbot] BUG: sleeping function called from invalid context in static_key_slow_inc

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit:    064bc7312bd0 netdevsim: Fix memory leak of nsim_dev->fa_co..
git tree:       net
console output: https://syzkaller.appspot.com/x/log.txt?x=13d3204e880000
kernel config:  https://syzkaller.appspot.com/x/.config?x=a33ac7bbc22a8c35
dashboard link: https://syzkaller.appspot.com/bug?extid=703d9e154b3b58277261
compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/0634e1c0e4cb/disk-064bc731.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/fe1039d2de22/vmlinux-064bc731.xz
kernel image: https://storage.googleapis.com/syzbot-assets/5a0d673875fa/bzImage-064bc731.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+703d9e154b3b58277261@syzkaller.appspotmail.com

BUG: sleeping function called from invalid context at include/linux/percpu-rwsem.h:49
in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 9420, name: syz-executor.5
preempt_count: 1, expected: 0
RCU nest depth: 0, expected: 0
INFO: lockdep is turned off.
Preemption disabled at:
[<0000000000000000>] 0x0
CPU: 1 PID: 9420 Comm: syz-executor.5 Not tainted 6.1.0-rc4-syzkaller-00212-g064bc7312bd0 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd1/0x138 lib/dump_stack.c:106
 __might_resched.cold+0x222/0x26b kernel/sched/core.c:9890
 percpu_down_read include/linux/percpu-rwsem.h:49 [inline]
 cpus_read_lock+0x1b/0x140 kernel/cpu.c:310
 static_key_slow_inc+0x12/0x20 kernel/jump_label.c:158
 udp_tunnel_encap_enable include/net/udp_tunnel.h:187 [inline]
 setup_udp_tunnel_sock+0x43d/0x550 net/ipv4/udp_tunnel_core.c:81
 l2tp_tunnel_register+0xc51/0x1210 net/l2tp/l2tp_core.c:1509
 pppol2tp_connect+0xcdc/0x1a10 net/l2tp/l2tp_ppp.c:723
 __sys_connect_file+0x153/0x1a0 net/socket.c:1976
 __sys_connect+0x165/0x1a0 net/socket.c:1993
 __do_sys_connect net/socket.c:2003 [inline]
 __se_sys_connect net/socket.c:2000 [inline]
 __x64_sys_connect+0x73/0xb0 net/socket.c:2000
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7fd94d28b639
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fd94e038168 EFLAGS: 00000246
 ORIG_RAX: 000000000000002a
RAX: ffffffffffffffda RBX: 00007fd94d3abf80 RCX: 00007fd94d28b639
RDX: 000000000000002e RSI: 0000000020000000 RDI: 0000000000000003
RBP: 00007fd94d2e6ae9 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffde93128bf R14: 00007fd94e038300 R15: 0000000000022000
 </TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

Re: [syzbot] BUG: sleeping function called from invalid context in static_key_slow_inc

[syzbot]

syzbot has found a reproducer for the following issue on:

HEAD commit:    064bc7312bd0 netdevsim: Fix memory leak of nsim_dev->fa_co..
git tree:       net
console+strace: https://syzkaller.appspot.com/x/log.txt?x=16b2b231880000
kernel config:  https://syzkaller.appspot.com/x/.config?x=a33ac7bbc22a8c35
dashboard link: https://syzkaller.appspot.com/bug?extid=703d9e154b3b58277261
compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=13cd2f79880000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=109e1695880000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/0634e1c0e4cb/disk-064bc731.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/fe1039d2de22/vmlinux-064bc731.xz
kernel image: https://storage.googleapis.com/syzbot-assets/5a0d673875fa/bzImage-064bc731.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+703d9e154b3b58277261@syzkaller.appspotmail.com

BUG: sleeping function called from invalid context at include/linux/percpu-rwsem.h:49
in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 3634, name: syz-executor167
preempt_count: 1, expected: 0
RCU nest depth: 0, expected: 0
3 locks held by syz-executor167/3634:
 #0: ffffffff8df6b530 (cb_lock){++++}-{3:3}, at: genl_rcv+0x19/0x40 net/netlink/genetlink.c:860
 #1: ffffffff8df6b5e8 (genl_mutex){+.+.}-{3:3}, at: genl_lock net/netlink/genetlink.c:33 [inline]
 #1: ffffffff8df6b5e8 (genl_mutex){+.+.}-{3:3}, at: genl_rcv_msg+0x50d/0x780 net/netlink/genetlink.c:848
 #2: ffff8880182fa0b8 (k-clock-AF_INET){+++.}-{2:2}, at: l2tp_tunnel_register+0x126/0x1210 net/l2tp/l2tp_core.c:1477
Preemption disabled at:
[<0000000000000000>] 0x0
CPU: 1 PID: 3634 Comm: syz-executor167 Not tainted 6.1.0-rc4-syzkaller-00212-g064bc7312bd0 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd1/0x138 lib/dump_stack.c:106
 __might_resched.cold+0x222/0x26b kernel/sched/core.c:9890
 percpu_down_read include/linux/percpu-rwsem.h:49 [inline]
 cpus_read_lock+0x1b/0x140 kernel/cpu.c:310
 static_key_slow_inc+0x12/0x20 kernel/jump_label.c:158
 udp_tunnel_encap_enable include/net/udp_tunnel.h:189 [inline]
 setup_udp_tunnel_sock+0x3e1/0x550 net/ipv4/udp_tunnel_core.c:81
 l2tp_tunnel_register+0xc51/0x1210 net/l2tp/l2tp_core.c:1509
 l2tp_nl_cmd_tunnel_create+0x3d6/0x8b0 net/l2tp/l2tp_netlink.c:245
 genl_family_rcv_msg_doit+0x228/0x320 net/netlink/genetlink.c:756
 genl_family_rcv_msg net/netlink/genetlink.c:833 [inline]
 genl_rcv_msg+0x445/0x780 net/netlink/genetlink.c:850
 netlink_rcv_skb+0x157/0x430 net/netlink/af_netlink.c:2540
 genl_rcv+0x28/0x40 net/netlink/genetlink.c:861
 netlink_unicast_kernel net/netlink/af_netlink.c:1319 [inline]
 netlink_unicast+0x547/0x7f0 net/netlink/af_netlink.c:1345
 netlink_sendmsg+0x91b/0xe10 net/netlink/af_netlink.c:1921
 sock_sendmsg_nosec net/socket.c:714 [inline]
 sock_sendmsg+0xd3/0x120 net/socket.c:734
 sock_no_sendpage+0x10c/0x160 net/core/sock.c:3219
 kernel_sendpage.part.0+0x1d5/0x700 net/socket.c:3561
 kernel_sendpage net/socket.c:3558 [inline]
 sock_sendpage+0xe3/0x140 net/socket.c:1054
 pipe_to_sendpage+0x2b1/0x380 fs/splice.c:361
 splice_from_pipe_feed fs/splice.c:415 [inline]
 __splice_from_pipe+0x449/0x8a0 fs/splice.c:559
 splice_from_pipe fs/splice.c:594 [inline]
 generic_splice_sendpage+0xd8/0x140 fs/splice.c:743
 do_splice_from fs/splice.c:764 [inline]
 direct_splice_actor+0x114/0x180 fs/splice.c:931
 splice_direct_to_actor+0x335/0x8a0 fs/splice.c:886
 do_splice_direct+0x1ab/0x280 fs/splice.c:974
 do_sendfile+0xb19/0x1270 fs/read_write.c:1255
 __do_sys_sendfile64 fs/read_write.c:1323 [inline]
 __se_sys_sendfile64 fs/read_write.c:1309 [inline]
 __x64_sys_sendfile64+0x1d0/0x210 fs/read_write.c:1309
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f93d1567cb9
Code: 28 c3 e8 5a 14 00 00 66 2e 0f 1f 84 00 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffdd8ae4a88 EFLAGS: 00000246 ORIG_RAX: 0000000000000028
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f93d1567cb9
RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000005
RBP: 00007f93d152b680 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000100000000 R11: 0000000000000246 R12: 00007f93d152b710
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
 </TASK>

Re: [syzbot] BUG: sleeping function called from invalid context in static_key_slow_inc

[syzbot]

syzbot has bisected this issue to:

commit b68777d54fac21fc833ec26ea1a2a84f975ab035
Author: Jakub Sitnicki <jakub@cloudflare.com>
Date:   Mon Nov 14 19:16:19 2022 +0000

    l2tp: Serialize access to sk_user_data with sk_callback_lock

bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=1600bb49880000
start commit:   064bc7312bd0 netdevsim: Fix memory leak of nsim_dev->fa_co..
git tree:       net
final oops:     https://syzkaller.appspot.com/x/report.txt?x=1500bb49880000
console output: https://syzkaller.appspot.com/x/log.txt?x=1100bb49880000
kernel config:  https://syzkaller.appspot.com/x/.config?x=a33ac7bbc22a8c35
dashboard link: https://syzkaller.appspot.com/bug?extid=703d9e154b3b58277261
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=13cd2f79880000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=109e1695880000

Reported-by: syzbot+703d9e154b3b58277261@syzkaller.appspotmail.com
Fixes: b68777d54fac ("l2tp: Serialize access to sk_user_data with sk_callback_lock")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

Re: [syzbot] BUG: sleeping function called from invalid context in static_key_slow_inc

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
inconsistent lock state in l2tp_tunnel_destruct

================================
WARNING: inconsistent lock state
6.1.0-rc4-syzkaller-00212-g064bc7312bd0-dirty #0 Not tainted
--------------------------------
inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
swapper/0/0 [HC0[0]:SC1[3]:HE1:SE0] takes:
ffff888145ea44f8 (k-clock-AF_INET){++?.}-{2:2}, at: l2tp_tunnel_destruct+0xe6/0x2d0 net/l2tp/l2tp_core.c:1153
{SOFTIRQ-ON-W} state was registered at:
  lock_acquire kernel/locking/lockdep.c:5668 [inline]
  lock_acquire+0x1e3/0x630 kernel/locking/lockdep.c:5633
  __raw_write_lock include/linux/rwlock_api_smp.h:209 [inline]
  _raw_write_lock+0x2e/0x40 kernel/locking/spinlock.c:300
  l2tp_tunnel_register+0xc61/0x1200 net/l2tp/l2tp_core.c:1509
  l2tp_nl_cmd_tunnel_create+0x3d6/0x8b0 net/l2tp/l2tp_netlink.c:245
  genl_family_rcv_msg_doit+0x228/0x320 net/netlink/genetlink.c:756
  genl_family_rcv_msg net/netlink/genetlink.c:833 [inline]
  genl_rcv_msg+0x445/0x780 net/netlink/genetlink.c:850
  netlink_rcv_skb+0x157/0x430 net/netlink/af_netlink.c:2540
  genl_rcv+0x28/0x40 net/netlink/genetlink.c:861
  netlink_unicast_kernel net/netlink/af_netlink.c:1319 [inline]
  netlink_unicast+0x547/0x7f0 net/netlink/af_netlink.c:1345
  netlink_sendmsg+0x91b/0xe10 net/netlink/af_netlink.c:1921
  sock_sendmsg_nosec net/socket.c:714 [inline]
  sock_sendmsg+0xd3/0x120 net/socket.c:734
  ____sys_sendmsg+0x712/0x8c0 net/socket.c:2482
  ___sys_sendmsg+0x110/0x1b0 net/socket.c:2536
  __sys_sendmsg+0xf7/0x1c0 net/socket.c:2565
  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
  do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
  entry_SYSCALL_64_after_hwframe+0x63/0xcd
irq event stamp: 5256464
hardirqs last  enabled at (5256464): [<ffffffff89f47d84>] __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:151 [inline]
hardirqs last  enabled at (5256464): [<ffffffff89f47d84>] _raw_spin_unlock_irqrestore+0x54/0x70 kernel/locking/spinlock.c:194
hardirqs last disabled at (5256463): [<ffffffff89f47b12>] __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:108 [inline]
hardirqs last disabled at (5256463): [<ffffffff89f47b12>] _raw_spin_lock_irqsave+0x52/0x60 kernel/locking/spinlock.c:162
softirqs last  enabled at (5256370): [<ffffffff814c1d03>] invoke_softirq kernel/softirq.c:445 [inline]
softirqs last  enabled at (5256370): [<ffffffff814c1d03>] __irq_exit_rcu+0x123/0x180 kernel/softirq.c:650
softirqs last disabled at (5256383): [<ffffffff814c1d03>] invoke_softirq kernel/softirq.c:445 [inline]
softirqs last disabled at (5256383): [<ffffffff814c1d03>] __irq_exit_rcu+0x123/0x180 kernel/softirq.c:650

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(k-clock-AF_INET);
  <Interrupt>
    lock(k-clock-AF_INET);

 *** DEADLOCK ***

1 lock held by swapper/0/0:
 #0: ffffffff8c58fc60 (rcu_callback){....}-{0:0}, at: rcu_do_batch kernel/rcu/tree.c:2239 [inline]
 #0: ffffffff8c58fc60 (rcu_callback){....}-{0:0}, at: rcu_core+0x7ab/0x1980 kernel/rcu/tree.c:2510

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.1.0-rc4-syzkaller-00212-g064bc7312bd0-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd1/0x138 lib/dump_stack.c:106
 print_usage_bug kernel/locking/lockdep.c:3963 [inline]
 valid_state kernel/locking/lockdep.c:3975 [inline]
 mark_lock_irq kernel/locking/lockdep.c:4178 [inline]
 mark_lock.part.0.cold+0x18/0xd8 kernel/locking/lockdep.c:4634
 mark_lock kernel/locking/lockdep.c:4598 [inline]
 mark_usage kernel/locking/lockdep.c:4529 [inline]
 __lock_acquire+0x11d9/0x56d0 kernel/locking/lockdep.c:5009
 lock_acquire kernel/locking/lockdep.c:5668 [inline]
 lock_acquire+0x1e3/0x630 kernel/locking/lockdep.c:5633
 __raw_write_lock_bh include/linux/rwlock_api_smp.h:202 [inline]
 _raw_write_lock_bh+0x33/0x40 kernel/locking/spinlock.c:334
 l2tp_tunnel_destruct+0xe6/0x2d0 net/l2tp/l2tp_core.c:1153
 __sk_destruct+0x51/0x710 net/core/sock.c:2122
 rcu_do_batch kernel/rcu/tree.c:2250 [inline]
 rcu_core+0x81f/0x1980 kernel/rcu/tree.c:2510
 __do_softirq+0x1fb/0xadc kernel/softirq.c:571
 invoke_softirq kernel/softirq.c:445 [inline]
 __irq_exit_rcu+0x123/0x180 kernel/softirq.c:650
 irq_exit_rcu+0x9/0x20 kernel/softirq.c:662
 sysvec_apic_timer_interrupt+0x97/0xc0 arch/x86/kernel/apic/apic.c:1107
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:649
RIP: 0010:native_save_fl arch/x86/include/asm/irqflags.h:29 [inline]
RIP: 0010:arch_local_save_flags arch/x86/include/asm/irqflags.h:70 [inline]
RIP: 0010:arch_irqs_disabled arch/x86/include/asm/irqflags.h:130 [inline]
RIP: 0010:acpi_safe_halt drivers/acpi/processor_idle.c:113 [inline]
RIP: 0010:acpi_idle_do_entry+0x1fd/0x2a0 drivers/acpi/processor_idle.c:572
Code: 89 de e8 66 ac 8b f7 84 db 75 ac e8 ed af 8b f7 e8 78 30 92 f7 eb 0c e8 e1 af 8b f7 0f 00 2d 7a 63 be 00 e8 d5 af 8b f7 fb f4 <9c> 5b 81 e3 00 02 00 00 fa 31 ff 48 89 de e8 90 ac 8b f7 48 85 db
RSP: 0018:ffffffff8c207d28 EFLAGS: 00000293
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: ffffffff8c2bc9c0 RSI: ffffffff89f469ab RDI: 0000000000000000
RBP: ffff888145cc0064 R08: 0000000000000001 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
R13: ffff888145cc0000 R14: ffff888145cc0064 R15: ffff888145ebe804
 acpi_idle_enter+0x368/0x510 drivers/acpi/processor_idle.c:709
 cpuidle_enter_state+0x1af/0xd40 drivers/cpuidle/cpuidle.c:239
 cpuidle_enter+0x4e/0xa0 drivers/cpuidle/cpuidle.c:356
 call_cpuidle kernel/sched/idle.c:155 [inline]
 cpuidle_idle_call kernel/sched/idle.c:236 [inline]
 do_idle+0x3f7/0x590 kernel/sched/idle.c:303
 cpu_startup_entry+0x18/0x20 kernel/sched/idle.c:400
 rest_init+0x16d/0x270 init/main.c:729
 arch_call_rest_init+0x13/0x1c init/main.c:890
 start_kernel+0x477/0x498 init/main.c:1145
 secondary_startup_64_no_verify+0xce/0xdb
 </TASK>
----------------
Code disassembly (best guess):
   0:	89 de                	mov    %ebx,%esi
   2:	e8 66 ac 8b f7       	callq  0xf78bac6d
   7:	84 db                	test   %bl,%bl
   9:	75 ac                	jne    0xffffffb7
   b:	e8 ed af 8b f7       	callq  0xf78baffd
  10:	e8 78 30 92 f7       	callq  0xf792308d
  15:	eb 0c                	jmp    0x23
  17:	e8 e1 af 8b f7       	callq  0xf78baffd
  1c:	0f 00 2d 7a 63 be 00 	verw   0xbe637a(%rip)        # 0xbe639d
  23:	e8 d5 af 8b f7       	callq  0xf78baffd
  28:	fb                   	sti
  29:	f4                   	hlt
* 2a:	9c                   	pushfq <-- trapping instruction
  2b:	5b                   	pop    %rbx
  2c:	81 e3 00 02 00 00    	and    $0x200,%ebx
  32:	fa                   	cli
  33:	31 ff                	xor    %edi,%edi
  35:	48 89 de             	mov    %rbx,%rsi
  38:	e8 90 ac 8b f7       	callq  0xf78baccd
  3d:	48 85 db             	test   %rbx,%rbx


Tested on:

commit:         064bc731 netdevsim: Fix memory leak of nsim_dev->fa_co..
git tree:       https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
console output: https://syzkaller.appspot.com/x/log.txt?x=148c924e880000
kernel config:  https://syzkaller.appspot.com/x/.config?x=a33ac7bbc22a8c35
dashboard link: https://syzkaller.appspot.com/bug?extid=703d9e154b3b58277261
compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
patch:          https://syzkaller.appspot.com/x/patch.diff?x=110210f9880000

Re: [syzbot] BUG: sleeping function called from invalid context in static_key_slow_inc

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+703d9e154b3b58277261@syzkaller.appspotmail.com

Tested on:

commit:         064bc731 netdevsim: Fix memory leak of nsim_dev->fa_co..
git tree:       https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
console output: https://syzkaller.appspot.com/x/log.txt?x=12a7010d880000
kernel config:  https://syzkaller.appspot.com/x/.config?x=a33ac7bbc22a8c35
dashboard link: https://syzkaller.appspot.com/bug?extid=703d9e154b3b58277261
compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
patch:          https://syzkaller.appspot.com/x/patch.diff?x=11425701880000

Note: testing is done by a robot and is best-effort only.