Re: Fwd: UBSAN: index 1 is out of range for type 'xen_netif_rx_sring_entry [1]'

[Kees Cook <keescook@chromium.org>]

On Tue, Jul 25, 2023 at 03:34:26PM +0200, Juergen Gross wrote:
> On 25.07.23 15:24, Juergen Gross wrote:
> > On 23.07.23 02:06, Nathan Chancellor wrote:
> > > On Sat, Jul 22, 2023 at 07:21:05AM +0700, Bagas Sanjaya wrote:
> > > > Hi,
> > > > 
> > > > I notice a regression report on Bugzilla [1]. Quoting from it:
> > > > 
> > > > > Hi Kernel Team,
> > > > > 
> > > > > I rebuild today latest version from mainline repo.
> > > > > And i notice issue regarding xen-netfront.c.
> > > > > 
> > > > > Error:
> > > > > [    3.477400] ================================================================================
> > > > > [    3.477633] UBSAN: array-index-out-of-bounds in
> > > > > drivers/net/xen-netfront.c:1291:3
> > > > > [    3.477858] index 1 is out of range for type 'xen_netif_rx_sring_entry [1]'
> > > > > [    3.478085] CPU: 0 PID: 700 Comm: NetworkManager Not
> > > > > tainted 6.5.0-rc2-1-generation1 #3
> > > > > [    3.478088] Hardware name: Intel Corporation
> > > > > W2600CR/W2600CR, BIOS SE5C600.86B.02.06.0007.082420181029
> > > > > 01/13/2022
> > > > > [    3.478090] Call Trace:
> > > > > [    3.478092]  <IRQ>
> > > > > [    3.478097]  dump_stack_lvl+0x48/0x70
> > > > > [    3.478105]  dump_stack+0x10/0x20
> > > > > [    3.478107]  __ubsan_handle_out_of_bounds+0xc6/0x110
> > > > > [    3.478114]  xennet_poll+0xa94/0xac0
> > > > > [    3.478118]  ? generic_smp_call_function_single_interrupt+0x13/0x20
> > > > > [    3.478125]  __napi_poll+0x33/0x200
> > > > > [    3.478131]  net_rx_action+0x181/0x2e0
> > > > > [    3.478135]  __do_softirq+0xd9/0x346
> > > > > [    3.478139]  do_softirq.part.0+0x41/0x80
> > > > > [    3.478144]  </IRQ>
> > > > > [    3.478145]  <TASK>
> > > > > [    3.478146]  __local_bh_enable_ip+0x72/0x80
> > > > > [    3.478149]  _raw_spin_unlock_bh+0x1d/0x30
> > > > > [    3.478151]  xennet_open+0x75/0x160
> > > > > [    3.478154]  __dev_open+0x105/0x1d0
> > > > > [    3.478156]  __dev_change_flags+0x1b5/0x230
> > > > > [    3.478158]  dev_change_flags+0x27/0x80
> > > > > [    3.478160]  do_setlink+0x3d2/0x12b0
> > > > > [    3.478164]  ? __nla_validate_parse+0x5b/0xdb0
> > > > > [    3.478169]  __rtnl_newlink+0x6f6/0xb10
> > > > > [    3.478173]  ? rtnl_newlink+0x2f/0x80
> > > > > [    3.478177]  rtnl_newlink+0x48/0x80
> > > > > [    3.478180]  rtnetlink_rcv_msg+0x170/0x430
> > > > > [    3.478183]  ? fib6_clean_node+0xad/0x190
> > > > > [    3.478188]  ? __pfx_rtnetlink_rcv_msg+0x10/0x10
> > > > > [    3.478191]  netlink_rcv_skb+0x5d/0x110
> > > > > [    3.478195]  rtnetlink_rcv+0x15/0x30
> > > > > [    3.478198]  netlink_unicast+0x247/0x390
> > > > > [    3.478200]  netlink_sendmsg+0x25e/0x4e0
> > > > > [    3.478202]  sock_sendmsg+0xaf/0xc0
> > > > > [    3.478204]  ____sys_sendmsg+0x2a9/0x350
> > > > > [    3.478206]  ___sys_sendmsg+0x9a/0xf0
> > > > > [    3.478212]  ? _copy_from_iter+0x80/0x4a0
> > > > > [    3.478217]  __sys_sendmsg+0x89/0xf0
> > > > > [    3.478220]  __x64_sys_sendmsg+0x1d/0x30
> > > > > [    3.478222]  do_syscall_64+0x5c/0x90
> > > > > [    3.478226]  ? do_syscall_64+0x68/0x90
> > > > > [    3.478228]  ? ksys_write+0xe6/0x100
> > > > > [    3.478232]  ? exit_to_user_mode_prepare+0x49/0x220
> > > > > [    3.478236]  ? syscall_exit_to_user_mode+0x1b/0x50
> > > > > [    3.478240]  ? do_syscall_64+0x68/0x90
> > > > > [    3.478242]  ? do_syscall_64+0x68/0x90
> > > > > [    3.478243]  ? irqentry_exit_to_user_mode+0x9/0x30
> > > > > [    3.478246]  ? irqentry_exit+0x43/0x50
> > > > > [    3.478248]  ? sysvec_xen_hvm_callback+0x4b/0xd0
> > > > > [    3.478250]  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
> > > > > [    3.478253] RIP: 0033:0x7f973c244e4d
> > > > > [    3.478268] Code: 28 89 54 24 1c 48 89 74 24 10 89 7c 24
> > > > > 08 e8 ca ee ff ff 8b 54 24 1c 48 8b 74 24 10 41 89 c0 8b 7c
> > > > > 24 08 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 33 44 89
> > > > > c7 48 89 44 24 08 e8 fe ee ff ff 48
> > > > > [    3.478270] RSP: 002b:00007fff4777f470 EFLAGS: 00000293
> > > > > ORIG_RAX: 000000000000002e
> > > > > [    3.478273] RAX: ffffffffffffffda RBX: 00005583087c6480
> > > > > RCX: 00007f973c244e4d
> > > > > [    3.478274] RDX: 0000000000000000 RSI: 00007fff4777f4c0
> > > > > RDI: 000000000000000c
> > > > > [    3.478276] RBP: 00007fff4777f4c0 R08: 0000000000000000
> > > > > R09: 0000000000000000
> > > > > [    3.478277] R10: 0000000000000000 R11: 0000000000000293
> > > > > R12: 00005583087c6480
> > > > > [    3.478279] R13: 00007fff4777f668 R14: 00007fff4777f65c
> > > > > R15: 0000000000000000
> > > > > [    3.478283]  </TASK>
> > > > > [    3.478284] ================================================================================
> > > > > [    3.685513] ================================================================================
> > > > > [    3.685751] UBSAN: array-index-out-of-bounds in
> > > > > drivers/net/xen-netfront.c:485:7
> > > > > [    3.686111] index 1 is out of range for type 'xen_netif_tx_sring_entry [1]'
> > > > > [    3.686379] CPU: 1 PID: 697 Comm: avahi-daemon Not
> > > > > tainted 6.5.0-rc2-1-generation1 #3
> > > > > [    3.686381] Hardware name: Intel Corporation
> > > > > W2600CR/W2600CR, BIOS SE5C600.86B.02.06.0007.082420181029
> > > > > 01/13/2022
> > > > > [    3.686385] Call Trace:
> > > > > [    3.686388]  <TASK>
> > > > > [    3.686391]  dump_stack_lvl+0x48/0x70
> > > > > [    3.686399]  dump_stack+0x10/0x20
> > > > > [    3.686399]  __ubsan_handle_out_of_bounds+0xc6/0x110
> > > > > [    3.686403]  xennet_tx_setup_grant+0x1f7/0x230
> > > > > [    3.686403]  ? __pfx_xennet_tx_setup_grant+0x10/0x10
> > > > > [    3.686403]  gnttab_foreach_grant_in_range+0x5c/0x100
> > > > > [    3.686415]  xennet_start_xmit+0x428/0x990
> > > > > [    3.686415]  ? kmem_cache_alloc_node+0x1b1/0x3b0
> > > > > [    3.686415]  dev_hard_start_xmit+0x68/0x1e0
> > > > > [    3.686415]  sch_direct_xmit+0x10b/0x350
> > > > > [    3.686415]  __dev_queue_xmit+0x512/0xda0
> > > > > [    3.686439]  ? ___neigh_create+0x6cb/0x970
> > > > > [    3.686439]  neigh_resolve_output+0x118/0x1e0
> > > > > [    3.686446]  ip_finish_output2+0x181/0x540
> > > > > [    3.686450]  ? netif_rx_internal+0x46/0x140
> > > > > [    3.686456]  __ip_finish_output+0xb6/0x180
> > > > > [    3.686456]  ? dev_loopback_xmit+0x86/0x110
> > > > > [    3.686456]  ip_finish_output+0x29/0x100
> > > > > [    3.686456]  ip_mc_output+0x95/0x2e0
> > > > > [    3.686456]  ? __pfx_ip_finish_output+0x10/0x10
> > > > > [    3.686456]  ip_send_skb+0x9f/0xb0
> > > > > [    3.686456]  udp_send_skb+0x158/0x380
> > > > > [    3.686475]  udp_sendmsg+0xb84/0xf20
> > > > > [    3.686475]  ? do_sys_poll+0x3a1/0x5f0
> > > > > [    3.686483]  ? __pfx_ip_generic_getfrag+0x10/0x10
> > > > > [    3.686483]  inet_sendmsg+0x76/0x80
> > > > > [    3.686483]  ? inet_sendmsg+0x76/0x80
> > > > > [    3.686483]  sock_sendmsg+0xa8/0xc0
> > > > > [    3.686483]  ? _copy_from_user+0x30/0xa0
> > > > > [    3.686483]  ____sys_sendmsg+0x2a9/0x350
> > > > > [    3.686483]  ___sys_sendmsg+0x9a/0xf0
> > > > > [    3.686483]  __sys_sendmsg+0x89/0xf0
> > > > > [    3.686483]  __x64_sys_sendmsg+0x1d/0x30
> > > > > [    3.686483]  do_syscall_64+0x5c/0x90
> > > > > [    3.686483]  ? exit_to_user_mode_prepare+0x49/0x220
> > > > > [    3.686483]  ? syscall_exit_to_user_mode+0x1b/0x50
> > > > > [    3.686483]  ? do_syscall_64+0x68/0x90
> > > > > [    3.686483]  ? syscall_exit_to_user_mode+0x1b/0x50
> > > > > [    3.686483]  ? do_syscall_64+0x68/0x90
> > > > > [    3.686483]  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
> > > > > [    3.686483] RIP: 0033:0x7ff365942e13
> > > > > [    3.686483] Code: 8b 15 b9 a1 00 00 f7 d8 64 89 02 48 c7
> > > > > c0 ff ff ff ff eb b8 0f 1f 00 64 8b 04 25 18 00 00 00 85 c0
> > > > > 75 14 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 55 c3 0f
> > > > > 1f 40 00 48 83 ec 28 89 54 24 1c 48
> > > > > [    3.686483] RSP: 002b:00007ffc7bf1ca78 EFLAGS: 00000246
> > > > > ORIG_RAX: 000000000000002e
> > > > > [    3.686483] RAX: ffffffffffffffda RBX: 00005596bd24c900
> > > > > RCX: 00007ff365942e13
> > > > > [    3.686483] RDX: 0000000000000000 RSI: 00007ffc7bf1cb30
> > > > > RDI: 000000000000000c
> > > > > [    3.686483] RBP: 000000000000000c R08: 0000000000000004
> > > > > R09: 0000000000000019
> > > > > [    3.686483] R10: 00007ff365a1ca94 R11: 0000000000000246
> > > > > R12: 00007ffc7bf1cb30
> > > > > [    3.686483] R13: 0000000000000002 R14: 00005596bd235f9c
> > > > > R15: 0000000000000000
> > > > > [    3.686483]  </TASK>
> > > > > [    3.686483] ================================================================================
> > > > > [    3.686858] ================================================================================
> > > > > [    3.687190] UBSAN: array-index-out-of-bounds in
> > > > > drivers/net/xen-netfront.c:413:4
> > > > > [    3.687501] index 1 is out of range for type 'xen_netif_tx_sring_entry [1]'
> > > > > [    3.687800] CPU: 18 PID: 0 Comm: swapper/18 Not tainted
> > > > > 6.5.0-rc2-1-generation1 #3
> > > > > [    3.687804] Hardware name: Intel Corporation
> > > > > W2600CR/W2600CR, BIOS SE5C600.86B.02.06.0007.082420181029
> > > > > 01/13/2022
> > > > > [    3.687806] Call Trace:
> > > > > [    3.687808]  <IRQ>
> > > > > [    3.687812]  dump_stack_lvl+0x48/0x70
> > > > > [    3.687819]  dump_stack+0x10/0x20
> > > > > [    3.687821]  __ubsan_handle_out_of_bounds+0xc6/0x110
> > > > > [    3.687827]  xennet_tx_buf_gc+0x34a/0x440
> > > > > [    3.687831]  xennet_handle_tx.constprop.0+0x49/0x90
> > > > > [    3.687834]  xennet_tx_interrupt+0x32/0x70
> > > > > [    3.687837]  __handle_irq_event_percpu+0x4f/0x1b0
> > > > > [    3.687842]  handle_irq_event+0x39/0x80
> > > > > [    3.687846]  handle_edge_irq+0x8c/0x230
> > > > > [    3.687849]  handle_irq_desc+0x40/0x60
> > > > > [    3.687851]  generic_handle_irq+0x1f/0x30
> > > > > [    3.687854]  handle_irq_for_port+0x8e/0x180
> > > > > [    3.687858]  ? _raw_spin_unlock_irqrestore+0x11/0x60
> > > > > [    3.687861]  __evtchn_fifo_handle_events+0x221/0x330
> > > > > [    3.687866]  evtchn_fifo_handle_events+0xe/0x20
> > > > > [    3.687869]  __xen_evtchn_do_upcall+0x72/0xd0
> > > > > [    3.687873]  xen_hvm_evtchn_do_upcall+0xe/0x20
> > > > > [    3.687876]  __sysvec_xen_hvm_callback+0x53/0x70
> > > > > [    3.687880]  sysvec_xen_hvm_callback+0x8d/0xd0
> > > > > [    3.687884]  </IRQ>
> > > > > [    3.687885]  <TASK>
> > > > > [    3.687886]  asm_sysvec_xen_hvm_callback+0x1b/0x20
> > > > > [    3.687891] RIP: 0010:pv_native_safe_halt+0xb/0x10
> > > > > [    3.687896] Code: 0b 66 66 2e 0f 1f 84 00 00 00 00 00 0f
> > > > > 1f 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 eb 07
> > > > > 0f 00 2d 49 cc 33 00 fb f4 <c3> cc cc cc cc 90 90 90 90 90
> > > > > 90 90 90 90 90 90 90 90 90 90 90 55
> > > > > [    3.687898] RSP: 0000:ffffad85c0147e08 EFLAGS: 00000246
> > > > > [    3.687901] RAX: ffffffffa00d39a0 RBX: 0000000000000002
> > > > > RCX: 0000000000000000
> > > > > [    3.687902] RDX: 0000000000000002 RSI: ffffffffa14d28e0
> > > > > RDI: ffff920446abda00
> > > > > [    3.687904] RBP: ffffad85c0147e18 R08: 0000000000000000
> > > > > R09: 0000000000000000
> > > > > [    3.687905] R10: 0000000000000000 R11: 0000000000000000
> > > > > R12: 0000000000000002
> > > > > [    3.687906] R13: 0000000000000002 R14: 0000000000000002
> > > > > R15: ffffffffa14d29c8
> > > > > [    3.687909]  ? __pfx_intel_idle_hlt+0x10/0x10
> > > > > [    3.687913]  ? intel_idle_hlt+0xc/0x40
> > > > > [    3.687916]  cpuidle_enter_state+0xa0/0x730
> > > > > [    3.687920]  cpuidle_enter+0x2e/0x50
> > > > > [    3.687924]  call_cpuidle+0x23/0x60
> > > > > [    3.687928]  do_idle+0x207/0x260
> > > > > [    3.687932]  cpu_startup_entry+0x1d/0x20
> > > > > [    3.687934]  start_secondary+0x129/0x160
> > > > > [    3.687939]  secondary_startup_64_no_verify+0x17e/0x18b
> > > > > [    3.687945]  </TASK>
> > > > > [    3.687946] ================================================================================
> > > > > [    4.624607] bridge: filtering via arp/ip/ip6tables is no
> > > > > longer available by default. Update your scripts to load
> > > > > br_netfilter if you need this.
> > > > > [    4.629153] Bridge firewalling registered
> > > > > [    4.745355] Initializing XFRM netlink socket
> > > > > [    4.794107] loop8: detected capacity change from 0 to 8
> > > > > [    7.104544] rfkill: input handler disabled
> > > > > [   26.445163] ================================================================================
> > > > > [   26.445171] UBSAN: array-index-out-of-bounds in
> > > > > drivers/net/xen-netfront.c:807:4
> > > > > [   26.445175] index 109 is out of range for type
> > > > > 'xen_netif_tx_sring_entry [1]'
> > > > > [   26.445178] CPU: 8 PID: 1729 Comm: sshd Not tainted
> > > > > 6.5.0-rc2-1-generation1 #3
> > > > > [   26.445180] Hardware name: Intel Corporation
> > > > > W2600CR/W2600CR, BIOS SE5C600.86B.02.06.0007.082420181029
> > > > > 01/13/2022
> > > > > [   26.445181] Call Trace:
> > > > > [   26.445185]  <TASK>
> > > > > [   26.445185]  dump_stack_lvl+0x48/0x70
> > > > > [   26.445185]  dump_stack+0x10/0x20
> > > > > [   26.445200]  __ubsan_handle_out_of_bounds+0xc6/0x110
> > > > > [   26.445206]  xennet_start_xmit+0x932/0x990
> > > > > [   26.445211]  dev_hard_start_xmit+0x68/0x1e0
> > > > > [   26.445216]  sch_direct_xmit+0x10b/0x350
> > > > > [   26.445220]  __dev_queue_xmit+0x512/0xda0
> > > > > [   26.445224]  ip_finish_output2+0x261/0x540
> > > > > [   26.445225]  __ip_finish_output+0xb6/0x180
> > > > > [   26.445225]  ip_finish_output+0x29/0x100
> > > > > [   26.445234]  ip_output+0x73/0x120
> > > > > [   26.445234]  ? __pfx_ip_finish_output+0x10/0x10
> > > > > [   26.445238]  ip_local_out+0x61/0x70
> > > > > [   26.445238]  __ip_queue_xmit+0x18d/0x470
> > > > > [   26.445238]  ip_queue_xmit+0x15/0x30
> > > > > [   26.445238]  __tcp_transmit_skb+0xb39/0xcc0
> > > > > [   26.445238]  tcp_write_xmit+0x595/0x1570
> > > > > [   26.445238]  ? _copy_from_iter+0x80/0x4a0
> > > > > [   26.445256]  __tcp_push_pending_frames+0x37/0x110
> > > > > [   26.445259]  tcp_push+0x123/0x190
> > > > > [   26.445260]  tcp_sendmsg_locked+0xafe/0xed0
> > > > > [   26.445264]  tcp_sendmsg+0x2c/0x50
> > > > > [   26.445268]  inet_sendmsg+0x42/0x80
> > > > > [   26.445268]  sock_write_iter+0x160/0x180
> > > > > [   26.445274]  vfs_write+0x397/0x440
> > > > > [   26.445274]  ksys_write+0xc9/0x100
> > > > > [   26.445274]  __x64_sys_write+0x19/0x30
> > > > > [   26.445274]  do_syscall_64+0x5c/0x90
> > > > > [   26.445287]  ? syscall_exit_to_user_mode+0x1b/0x50
> > > > > [   26.445290]  ? do_syscall_64+0x68/0x90
> > > > > [   26.445290]  ? do_syscall_64+0x68/0x90
> > > > > [   26.445294]  ? do_syscall_64+0x68/0x90
> > > > > [   26.445294]  ? syscall_exit_to_user_mode+0x1b/0x50
> > > > > [   26.445298]  ? do_syscall_64+0x68/0x90
> > > > > [   26.445300]  ? exc_page_fault+0x94/0x1b0
> > > > > [   26.445302]  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
> > > > > [   26.445306] RIP: 0033:0x7f26c4c3d473
> > > > > [   26.445318] Code: 8b 15 21 2a 0e 00 f7 d8 64 89 02 48 c7
> > > > > c0 ff ff ff ff eb b7 0f 1f 00 64 8b 04 25 18 00 00 00 85 c0
> > > > > 75 14 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 55 c3 0f
> > > > > 1f 40 00 48 83 ec 28 48 89 54 24 18
> > > > > [   26.445321] RSP: 002b:00007ffdee7b5528 EFLAGS: 00000246
> > > > > ORIG_RAX: 0000000000000001
> > > > > [   26.445321] RAX: ffffffffffffffda RBX: 0000000000000700
> > > > > RCX: 00007f26c4c3d473
> > > > > [   26.445321] RDX: 0000000000000700 RSI: 000055567032e230
> > > > > RDI: 0000000000000004
> > > > > [   26.445321] RBP: 0000555670313d70 R08: fffffffffffffff0
> > > > > R09: 0000000000000000
> > > > > [   26.445321] R10: 0000000000000000 R11: 0000000000000246
> > > > > R12: 000055566fcb2768
> > > > > [   26.445321] R13: 0000000000000000 R14: 0000000000000004
> > > > > R15: 000055566fc67a80
> > > > > [   26.445332]  </TASK>
> > > > > [   26.445333] ================================================================================
> > > > 
> > > > See Bugzilla for the full thread and attached dmesg.
> > > > 
> > > > Anyway, I'm adding it to regzbot:
> > > > 
> > > > #regzbot introduced: 8446066bf8c1f9f
> > > > https://bugzilla.kernel.org/show_bug.cgi?id=217693
> > > > 
> > > > Thanks.
> > > > 
> > > > [1]: https://bugzilla.kernel.org/show_bug.cgi?id=217693
> > > 
> > > I doubt it is 8446066bf8c1f9f that causes this. Based on the comment
> > > next to the 'ring[1]' in DEFINE_RING_TYPES() in
> > > include/xen/interface/io/ring.h, this is probably caused/exposed by
> > > commit df8fc4e934c1 ("kbuild: Enable -fstrict-flex-arrays=3") in
> > > 6.5-rc1, which causes that array to no longer be a flexible array but an
> > > array with one element, which would cause UBSAN to complain about an
> > > array access past index one. Adding Kees and Gustavo.
> > 
> > I agree.
> > 
> > > 
> > > Unfortunately, it seems this file is vendored from Xen, so I assume it
> > > would need to be fixed there then pulled into Linux:
> > > 
> > > https://github.com/xen-project/xen/tree/master/xen/include/public/io/ring.h
> > 
> > No, I don't think it will be possible to change this in the Xen tree easily.
> > 
> > Especially the public Xen headers are meant to be compatible with a large
> > variety of compilers, including rather old ones.
> > 
> > This means that ring[1] can't be easily swapped with ring[], as that would
> > cause compile time errors with some compilers.
> > 
> > Just modifying the Linux side header is an option, though, as we don't need
> > the same wide range of supported compilers as Xen.
> > 
> > I'll send a patch for that purpose.
> 
> Oh, in fact there is a way in Xen to do that correctly. It schould be enough to
> use ring[XEN_FLEX_ARRAY_DIM], which will do the right thing.
> 
> So I'll write a Xen patch first, after all.

Perfect! I went to go look, and yes, this is good:

/* Define a variable length array (depends on compiler). */
#if defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L
#define XEN_FLEX_ARRAY_DIM
#elif defined(__GNUC__)
#define XEN_FLEX_ARRAY_DIM  0
#else
#define XEN_FLEX_ARRAY_DIM  1 /* variable size */
#endif

Be careful, of course, going from [1] to [], if anything is using
sizeof() on the structure.

Thanks for fixing this!

-Kees

-- 
Kees Cook