From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl0-f47.google.com ([209.85.160.47]:44423 "EHLO mail-pl0-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755241AbeCSMRj (ORCPT ); Mon, 19 Mar 2018 08:17:39 -0400 Received: by mail-pl0-f47.google.com with SMTP id 9-v6so10085182ple.11 for ; Mon, 19 Mar 2018 05:17:39 -0700 (PDT) Subject: Re: 4.14.2[6-7] tcp_push NULL pointer To: Tomas Charvat , netdev@vger.kernel.org, "David S. Miller" References: <453586a8-13e8-1758-3016-f4ed3eefe42e@excello.cz> From: Eric Dumazet Message-ID: Date: Mon, 19 Mar 2018 05:17:37 -0700 MIME-Version: 1.0 In-Reply-To: <453586a8-13e8-1758-3016-f4ed3eefe42e@excello.cz> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: netdev-owner@vger.kernel.org List-ID: On 03/19/2018 04:51 AM, Tomas Charvat wrote: > I have seen on multiple servers with kernel-4.14.26 and 4.14.27 > following errors in dmes. It seems that it also caused involved process > to crash (apache and qmail-smtpd). > > [Fri Mar 16 00:00:11 2018] BUG: unable to handle kernel NULL pointer > dereference at 0000000000000038 > [Fri Mar 16 00:00:11 2018] IP: tcp_push+0x3d/0x110 > [Fri Mar 16 00:00:11 2018] PGD 0 P4D 0 > [Fri Mar 16 00:00:11 2018] Oops: 0002 [#1] SMP NOPTI > [Fri Mar 16 00:00:11 2018] CPU: 74 PID: 50845 Comm: parse_scanner.p Not > tainted 4.14.27-gentoo #1 > [Fri Mar 16 00:00:11 2018] Hardware name: Supermicro AS > -1123US-TR4/H11DSU-iN, BIOS 1.0a 09/14/2017 > [Fri Mar 16 00:00:11 2018] task: ffffa33f855226c0 task.stack: > ffffbf3765394000 > [Fri Mar 16 00:00:11 2018] RIP: 0010:tcp_push+0x3d/0x110 > [Fri Mar 16 00:00:11 2018] RSP: 0018:ffffbf3765397cd0 EFLAGS: 00010246 > [Fri Mar 16 00:00:11 2018] RAX: 0000000000000000 RBX: ffffa33b8664d000 > RCX: 0000000000000000 > [Fri Mar 16 00:00:11 2018] RDX: 0000000000000001 RSI: 0000000000000000 > RDI: ffffa33b6338ac00 > [Fri Mar 16 00:00:11 2018] RBP: 0000000000001c20 R08: 0000000000000576 > R09: ffffa33b6338ad58 > [Fri Mar 16 00:00:11 2018] R10: 0000000000000576 R11: 0000000000000000 > R12: 00000000ffffffe0 > [Fri Mar 16 00:00:11 2018] R13: 0000000000001c20 R14: ffffa33b6338ac00 > R15: ffffbf3765397dd0 > [Fri Mar 16 00:00:11 2018] FS:  00007f64301f1700(0000) > GS:ffffa33b9fc80000(0000) knlGS:0000000000000000 > [Fri Mar 16 00:00:11 2018] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [Fri Mar 16 00:00:11 2018] CR2: 0000000000000038 CR3: 0000800ff30bc000 > CR4: 00000000001406e0 > [Fri Mar 16 00:00:11 2018] Call Trace: > [Fri Mar 16 00:00:11 2018]  tcp_sendmsg_locked+0x65c/0xe40 > [Fri Mar 16 00:00:11 2018]  tcp_sendmsg+0x2e/0x50 > [Fri Mar 16 00:00:11 2018]  sock_sendmsg+0x3e/0x50 > [Fri Mar 16 00:00:11 2018]  SYSC_sendto+0x123/0x1c0 > [Fri Mar 16 00:00:11 2018]  do_syscall_64+0x80/0x340 > [Fri Mar 16 00:00:11 2018]  ? __do_page_fault+0x19c/0x3f0 > [Fri Mar 16 00:00:11 2018]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2 > [Fri Mar 16 00:00:11 2018] RIP: 0033:0x7f642f9d0b01 > [Fri Mar 16 00:00:11 2018] RSP: 002b:00007ffcfaa9c530 EFLAGS: 00000246 > ORIG_RAX: 000000000000002c > [Fri Mar 16 00:00:11 2018] RAX: ffffffffffffffda RBX: 0000562485e03c18 > RCX: 00007f642f9d0b01 > [Fri Mar 16 00:00:11 2018] RDX: 0000000000002000 RSI: 0000562485e03c18 > RDI: 0000000000000004 > [Fri Mar 16 00:00:11 2018] RBP: 0000562485a6d2a8 R08: 0000000000000000 > R09: 0000000000000000 > [Fri Mar 16 00:00:11 2018] R10: 0000000000000000 R11: 0000000000000246 > R12: 0000000000000000 > [Fri Mar 16 00:00:11 2018] R13: 0000562485e03c18 R14: 0000000000002000 > R15: 0000562485c18300 > [Fri Mar 16 00:00:11 2018] Code: 00 48 8b 87 60 01 00 00 4c 8d 8f 58 01 > 00 00 b9 00 00 00 00 41 89 f3 49 39 c1 48 0f 44 c1 41 81 e3 00 80 00 00 > 0f 85 a5 00 00 00 <80> 48 38 08 8b 8f 6c 06 00 00 89 8f 74 06 00 00 83 > e6 01 74 0c > [Fri Mar 16 00:00:11 2018] RIP: tcp_push+0x3d/0x110 RSP: ffffbf3765397cd0 > [Fri Mar 16 00:00:11 2018] CR2: 0000000000000038 > [Fri Mar 16 00:00:11 2018] ---[ end trace 4ed52c64cd15c543 ]--- > > [Thu Mar 15 14:56:06 2018] BUG: unable to handle kernel NULL pointer > dereference at 0000000000000038 > [Thu Mar 15 14:56:06 2018] IP: tcp_push+0x3d/0x110 > [Thu Mar 15 14:56:06 2018] PGD 0 P4D 0 > [Thu Mar 15 14:56:06 2018] Oops: 0002 [#1] SMP NOPTI > [Thu Mar 15 14:56:06 2018] CPU: 2 PID: 17214 Comm: rsync Not tainted > 4.14.26-gentoo #1 > [Thu Mar 15 14:56:06 2018] Hardware name: Xen HVM domU, BIOS 4.9.1 > 01/26/2018 > [Thu Mar 15 14:56:06 2018] task: ffff880164ca9a00 task.stack: > ffffc90002548000 > [Thu Mar 15 14:56:06 2018] RIP: 0010:tcp_push+0x3d/0x110 > [Thu Mar 15 14:56:06 2018] RSP: 0018:ffffc9000254bc90 EFLAGS: 00010246 > [Thu Mar 15 14:56:06 2018] RAX: 0000000000000000 RBX: ffff88006cf6a380 > RCX: 0000000000000000 > [Thu Mar 15 14:56:06 2018] RDX: 0000000000000000 RSI: 0000000000000040 > RDI: ffff880100025d00 > [Thu Mar 15 14:56:06 2018] RBP: 00000000000065d0 R08: 00000000000043e0 > R09: ffff880100025e58 > [Thu Mar 15 14:56:06 2018] R10: 00000000000005a8 R11: 0000000000000000 > R12: 00000000000065d0 > [Thu Mar 15 14:56:06 2018] R13: 00000000ffffffe0 R14: ffffc9000254bd80 > R15: ffff880100025d00 > [Thu Mar 15 14:56:06 2018] FS:  00007f1ffba19e80(0000) > GS:ffff88018f500000(0000) knlGS:0000000000000000 > [Thu Mar 15 14:56:06 2018] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [Thu Mar 15 14:56:06 2018] CR2: 0000000000000038 CR3: 0000000093550000 > CR4: 00000000000406e0 > [Thu Mar 15 14:56:06 2018] Call Trace: > [Thu Mar 15 14:56:06 2018]  tcp_sendmsg_locked+0x32d/0xe50 > [Thu Mar 15 14:56:06 2018]  tcp_sendmsg+0x2e/0x50 > [Thu Mar 15 14:56:06 2018]  sock_sendmsg+0x3e/0x50 > [Thu Mar 15 14:56:06 2018]  sock_write_iter+0x86/0x100 > [Thu Mar 15 14:56:06 2018]  __vfs_write+0x14d/0x1c0 > [Thu Mar 15 14:56:06 2018]  vfs_write+0xc3/0x1d0 > [Thu Mar 15 14:56:06 2018]  SyS_write+0x62/0xe0 > [Thu Mar 15 14:56:06 2018]  do_syscall_64+0x87/0x330 > [Thu Mar 15 14:56:06 2018]  ? __do_page_fault+0x1e0/0x450 > [Thu Mar 15 14:56:06 2018]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2 > [Thu Mar 15 14:56:06 2018] RIP: 0033:0x7f1ffaef0b50 > [Thu Mar 15 14:56:06 2018] RSP: 002b:00007ffe0b34a9d8 EFLAGS: 00000246 > ORIG_RAX: 0000000000000001 > [Thu Mar 15 14:56:06 2018] RAX: ffffffffffffffda RBX: 0000000000000003 > RCX: 00007f1ffaef0b50 > [Thu Mar 15 14:56:06 2018] RDX: 000000000000c195 RSI: 0000558248b77bc0 > RDI: 0000000000000003 > [Thu Mar 15 14:56:06 2018] RBP: 00007ffe0b34aa50 R08: 000000000000c195 > R09: e34308fec493ee59 > [Thu Mar 15 14:56:06 2018] R10: 00007ffe0b34aad0 R11: 0000000000000246 > R12: 00007ffe0b34ab50 > [Thu Mar 15 14:56:06 2018] R13: 00007ffe0b34aad0 R14: 0000000000000002 > R15: 0000558247c3b440 > [Thu Mar 15 14:56:06 2018] Code: 00 48 8b 87 60 01 00 00 4c 8d 8f 58 01 > 00 00 b9 00 00 00 00 41 89 f3 4c 39 c8 48 0f 44 c1 41 81 e3 00 80 00 00 > 0f 85 a5 00 00 00 <80> 48 38 08 8b 8f 6c 06 00 00 89 8f 74 06 00 00 83 > e6 01 74 0c > [Thu Mar 15 14:56:06 2018] RIP: tcp_push+0x3d/0x110 RSP: ffffc9000254bc90 > [Thu Mar 15 14:56:06 2018] CR2: 0000000000000038 > [Thu Mar 15 14:56:06 2018] ---[ end trace 6a582fb8616d56be ]--- > > [Sat Mar 17 04:47:25 2018] BUG: unable to handle kernel NULL pointer > dereference at 0000000000000038 > [Sat Mar 17 04:47:25 2018] IP: tcp_push+0x3d/0x110 > [Sat Mar 17 04:47:25 2018] PGD 0 P4D 0 > [Sat Mar 17 04:47:25 2018] Oops: 0002 [#2] SMP NOPTI > [Sat Mar 17 04:47:25 2018] CPU: 1 PID: 20758 Comm: rsync Tainted: G      > D         4.14.26-gentoo #1 > [Sat Mar 17 04:47:25 2018] Hardware name: Xen HVM domU, BIOS 4.9.1 > 01/26/2018 > [Sat Mar 17 04:47:25 2018] task: ffff8800b1b5c100 task.stack: > ffffc90002388000 > [Sat Mar 17 04:47:25 2018] RIP: 0010:tcp_push+0x3d/0x110 > [Sat Mar 17 04:47:25 2018] RSP: 0018:ffffc9000238bc90 EFLAGS: 00010246 > [Sat Mar 17 04:47:25 2018] RAX: 0000000000000000 RBX: ffff8800a8c90c40 > RCX: 0000000000000000 > [Sat Mar 17 04:47:25 2018] RDX: 0000000000000000 RSI: 0000000000000040 > RDI: ffff88005e0f2e80 > [Sat Mar 17 04:47:25 2018] RBP: 0000000000007038 R08: 0000000000005028 > R09: ffff88005e0f2fd8 > [Sat Mar 17 04:47:25 2018] R10: 0000000000000558 R11: 0000000000000000 > R12: 0000000000007038 > [Sat Mar 17 04:47:25 2018] R13: 00000000ffffffe0 R14: ffffc9000238bd80 > R15: ffff88005e0f2e80 > [Sat Mar 17 04:47:25 2018] FS:  00007f1ffba19e80(0000) > GS:ffff88018f480000(0000) knlGS:0000000000000000 > [Sat Mar 17 04:47:25 2018] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [Sat Mar 17 04:47:25 2018] CR2: 0000000000000038 CR3: 00000001029fe000 > CR4: 00000000000406e0 > [Sat Mar 17 04:47:25 2018] Call Trace: > [Sat Mar 17 04:47:25 2018]  tcp_sendmsg_locked+0x32d/0xe50 > [Sat Mar 17 04:47:25 2018]  tcp_sendmsg+0x2e/0x50 > [Sat Mar 17 04:47:25 2018]  sock_sendmsg+0x3e/0x50 > [Sat Mar 17 04:47:25 2018]  sock_write_iter+0x86/0x100 > [Sat Mar 17 04:47:25 2018]  __vfs_write+0x14d/0x1c0 > [Sat Mar 17 04:47:25 2018]  vfs_write+0xc3/0x1d0 > [Sat Mar 17 04:47:25 2018]  SyS_write+0x62/0xe0 > [Sat Mar 17 04:47:25 2018]  do_syscall_64+0x87/0x330 > [Sat Mar 17 04:47:25 2018]  ? __do_page_fault+0x1e0/0x450 > [Sat Mar 17 04:47:25 2018]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2 > [Sat Mar 17 04:47:25 2018] RIP: 0033:0x7f1ffaef0b50 > [Sat Mar 17 04:47:26 2018] RSP: 002b:00007ffe0b34a9d8 EFLAGS: 00000246 > ORIG_RAX: 0000000000000001 > [Sat Mar 17 04:47:26 2018] RAX: ffffffffffffffda RBX: 0000000000000003 > RCX: 00007f1ffaef0b50 > [Sat Mar 17 04:47:26 2018] RDX: 000000000000fe15 RSI: 0000558248b77bd0 > RDI: 0000000000000003 > [Sat Mar 17 04:47:26 2018] RBP: 00007ffe0b34aa50 R08: 000000000000fe15 > R09: 0000000000003dae > [Sat Mar 17 04:47:26 2018] R10: 00007ffe0b34aad0 R11: 0000000000000246 > R12: 00007ffe0b34ab50 > [Sat Mar 17 04:47:26 2018] R13: 00007ffe0b34aad0 R14: 0000000000000002 > R15: 0000558247c3b440 > [Sat Mar 17 04:47:26 2018] Code: 00 48 8b 87 60 01 00 00 4c 8d 8f 58 01 > 00 00 b9 00 00 00 00 41 89 f3 4c 39 c8 48 0f 44 c1 41 81 e3 00 80 00 00 > 0f 85 a5 00 00 00 <80> 48 38 08 8b 8f 6c 06 00 00 89 8f 74 06 00 00 83 > e6 01 74 0c > [Sat Mar 17 04:47:26 2018] RIP: tcp_push+0x3d/0x110 RSP: ffffc9000238bc90 > [Sat Mar 17 04:47:26 2018] CR2: 0000000000000038 > [Sat Mar 17 04:47:26 2018] ---[ end trace 6a582fb8616d56bf ]--- > Hi Tomas We have sent a fix last week, I am not sure if David took it. https://patchwork.ozlabs.org/patch/886324/ Thanks