Re: 4.14.29 - tcp_push() - null skb's cb dereference

[Eric Dumazet <eric.dumazet@gmail.com>]

On 03/28/2018 03:51 AM, Krzysztof Blaszkowski wrote:
> Hi,
> 
> I noticed a kernel bug report like below:
> 
> [95576.826393] BUG: unable to handle kernel NULL pointer dereference at
> 0000000000000038
> [95576.834296] IP: tcp_push+0x3d/0x110
> [95576.837829] PGD 2c8474067 P4D 2c8474067 PUD 1119cf067 PMD 0 
> [95576.843536] Oops: 0002 [#1] SMP NOPTI
> [95576.847247] CPU: 3 PID: 1682 Comm: nginx Not tainted 4.14.29 #1
> [95576.854421] Hardware name: PC-FACTORY empty/Tyan Transport GT24-
> B3992-E, BIOS 'V1.06.B10 ' 06/23/2009
> [95576.863678] task: ffff9e4f150b2580 task.stack: ffffb21401a50000
> [95576.869641] RIP: 0010:tcp_push+0x3d/0x110
> [95576.873692] RSP: 0018:ffffb21401a53be0 EFLAGS: 00010246
> [95576.878959] RAX: 0000000000000000 RBX: 0000000000009310 RCX:
> 0000000000000000
> [95576.886133] RDX: 0000000000000001 RSI: 0000000000000040 RDI:
> ffff9e4e79871f00
> [95576.893306] RBP: ffffb21401a53c78 R08: 00000000000065d0 R09:
> ffff9e4e79872048
> [95576.900479] R10: 00000000000005a8 R11: 0000000000000000 R12:
> 00000000ffffffe0
> [95576.907652] R13: ffffb21401a53cf8 R14: ffff9e4e79871f00 R15:
> ffff9e4f68727100
> [95576.914824] FS:  00007f683b4e1700(0000) GS:ffff9e50732c0000(0000)
> knlGS:0000000000000000
> [95576.922953] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [95576.928737] CR2: 0000000000000038 CR3: 000000020aa0e000 CR4:
> 00000000000006e0
> [95576.935908] Call Trace:
> [95576.938403]  ? tcp_sendmsg_locked+0x60a/0xd90
> [95576.942872]  tcp_sendmsg+0x27/0x40
> [95576.946386]  inet_sendmsg+0x2c/0xa0
> [95576.949987]  sock_sendmsg+0x33/0x40
> [95576.953587]  sock_write_iter+0x76/0xd0
> [95576.957448]  do_iter_readv_writev+0x108/0x160
> [95576.961917]  do_iter_write+0x82/0x190
> [95576.965690]  vfs_writev+0xbb/0x120
> [95576.969203]  ? ep_poll+0x240/0x3f0
> [95576.972714]  do_writev+0x4d/0xd0
> [95576.976052]  ? do_writev+0x4d/0xd0
> [95576.979564]  SyS_writev+0xb/0x10
> [95576.982905]  do_syscall_64+0x5a/0x110
> [95576.986678]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
> [95576.991838] RIP: 0033:0x7f6839e56170
> [95576.995523] RSP: 002b:00007ffe34c9cf48 EFLAGS: 00000246 ORIG_RAX:
> 0000000000000014
> [95577.003265] RAX: ffffffffffffffda RBX: 0000000004b622c0 RCX:
> 00007f6839e56170
> [95577.010507] RDX: 0000000000000017 RSI: 00007ffe34c9cfd0 RDI:
> 0000000000000012
> [95577.017749] RBP: 00007ffe34c9cfb0 R08: 00000000004b1c2f R09:
> 00007ffe34c9d130
> [95577.024989] R10: 000000000001d6c2 R11: 0000000000000246 R12:
> 0000000002f3eec0
> [95577.032228] R13: 000000007fffefff R14: 0000000000000000 R15:
> 0000000002f3eec0
> [95577.039469] Code: 01 00 00 4c 8d 8f 48 01 00 00 41 89 d2 41 89 f3 89
> ca b9 00 00 00 00 49 39 c1 48 0f 44 c1 41 81 e3 00 80 00 00 0f 85 ac 00
> 00 00 <80> 48 38 08 8b 8f 64 06 00 00 89 8f 6c 06 00 00 83 e6 01 74 0c 
> [95577.058617] RIP: tcp_push+0x3d/0x110 RSP: ffffb21401a53be0
> [95577.064208] CR2: 0000000000000038
> [95577.068056] ---[ end trace 19bfaf872fd3ef10 ]--
> 
> 
> 
> further report analysis ended up in net/ipv4/tcp.c : 
> 
> 
> (gdb) list *(tcp_push+0x3d)
> 0xfd is in tcp_push (/data/work/linux-4.14.29/net/ipv4/tcp.c:630).
> 625	}
> 626	EXPORT_SYMBOL(tcp_ioctl);
> 627	
> 628	static inline void tcp_mark_push(struct tcp_sock *tp, struct
> sk_buff *skb)
> 629	{
> 630		TCP_SKB_CB(skb)->tcp_flags |= TCPHDR_PSH;
> 631		tp->pushed_seq = tp->write_seq;
> 632	}
> 
> 
> and tcp_mark_push() is inlined in tcp_push() indeed.
> 
> Looking forward to seeing rationale for tcp_write_queue_tail(sk)
> returning null skb - assuming the above is correct.
> 
> Thanks

This has been reported 5 or 6 times already :/

For some reason the fix is missing from 4.14 stable tree.

https://patchwork.ozlabs.org/patch/886324/

Note this patch was needed even before a27fid7a8ed38 (tcp: purge write queue upon RST)
since tcp disconnect would have caused the same issue.



Thanks.