From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754180AbcIISP5 (ORCPT ); Fri, 9 Sep 2016 14:15:57 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:37290 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752115AbcIISPy (ORCPT ); Fri, 9 Sep 2016 14:15:54 -0400 Date: Fri, 9 Sep 2016 11:15:34 -0700 From: Shaohua Li To: Stefan Priebe - Profihost AG CC: Jens Axboe , , Subject: Re: kernel 4.8-rc5 kernel BUG at block/blk-core.c:2032! Message-ID: <20160909181533.GA76292@shli-mbp.local> References: <56e78c6a-5996-d106-2952-d985ceca9e34@profihost.ag> <7ad7efe2-ef46-e1d5-bb0d-c7ed0e8d47db@kernel.dk> <20160908173328.GA58334@shli-mbp.local> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-Originating-IP: [2620:10d:c090:200::b:d63] X-ClientProxiedBy: MWHPR04CA0033.namprd04.prod.outlook.com (10.172.163.19) To BL2PR15MB0931.namprd15.prod.outlook.com (10.167.116.17) X-MS-Office365-Filtering-Correlation-Id: 69f34573-e142-4d57-10ae-08d3d8dd51c9 X-Microsoft-Exchange-Diagnostics: 1;BL2PR15MB0931;2:V+SsoV0PiWVTT5CSP3/z5ya7mObLHSONVYFbd6ZvmCdUt5SWRsCqttVskOhIeTUImoSVbnmk0vzefJ2EieIcLuXnpK0WhddlnbubdWIvB0NtONfIpRVrLSOBGTpmceZFkxNF/HVNMOdUgUH57+NRki6MvjCDYBWPb3zHS3+k06enIHki8g28FMIVfbz9VO6W;3:5974vdOP/VjfBY+ednj1g7II20iqm7Y6M7HKtlTiYqtSdjc4qBdzm4MATfT3IhS6P5Ozb03g114QLetKajVB+2aXbh33j9xrdFs3br6qmKhCPsoau2wwIk1WsRXuacWy;25:axmolivtuxdzx9O+GyWIVDWTs2bzTCNEnS2mXzSBLNsZz7zL0A4IT4Jsm7DGL12NaqjD8N2OXRp/NT19Rh6Uptxo0cbL2rFsxXDXTUXQ1xEGGVBOsQsOypOsyyJvwK3usLrH+pqa5+FY6bFG6ynFv0vk3cPEFmqTEBHYduS8nnNqo9GUAgItTANyZ6F9Ke3E0gOQk2zg9sAUJ2raLXVVONpoEnU8Wc+SnpChBjdMVNdMFy1ScL8+HN/KA/l1Qu4RvkU80dNOQV2CTMzCK3OxdOKGuEy7EAg/j4v+h8Aka7JfNfTMtZZwFhUIiFOePtAMfJHmY/I2MLiYCKPVxRyfQoDFPmjLc5JqE4ixQbKSlE1RVw4ix2TQhS+712JcRSUCxVJAOjd0mXdH/+Hl0pHgW68gOe6kEveICOZgsGP+SCM= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BL2PR15MB0931; X-Microsoft-Exchange-Diagnostics: 1;BL2PR15MB0931;31:zMeMY5k5D2P5cXzau/cDqaRR24UkpBu1s0s+vKbTiyU9p9ypBN/QEwOhOz0ucKV1uJFla1moY6qGALlHIGGmpCnyN3dxmLeDRK46A0DEj55NraNWISFEzdAi0MJGF9Opfx49IfP2wKxYLepgdCivaVOa8HWLFbaDdLNCBOJ/r2eGA+iagmCYmT9F5rNONjEAf76ifYlTsZnH43+VvxgNA32OpuyE8NJGH60RlTaP6Js=;20:lJBbsY9BjAAD89w87NthrLAUJU+6mdQ30+ZmiaJL9YMtxTeqCIGrUA8YpMcZPlzS/dToN82rM/jJ6zHi/YEXng/ttr50HLUnKEY3iauZyuNhSNCIX1PVdmo+0LqNhUvhriSH6gOpat01YuK39ftz9fj3wRDAMKZhi5n4OuFl3j0=;4:oTr+dHsfUtFCyqVB740uxEUggPkbgvmLYIxKJiE8TEzskHVtVtNBO4hatEXxvuxlOUNnk/83DahNtLtXlcmajQubLMqfVrTLMKO8oWYAep5es9kcs6pexvf13t0Rtu0zi9whYWlZ+KiLYQ3tUQOHWBuanqGREHhdJItRlo1QNmSuNHjHTUFk3D6j4y2LcHtMZJUCiJzJ6K72iQLvp829ZJ69RZl1bhm2jvnFspH82Wrbs4tZn7Uu19CvETE8n4n1oL+L3y8+OT288Alq+0wZGvIgad0Y77mFOnQxT/yxIBPVpDiR7SU39fFaE3dDnQiY9tDYhPt6T8zYkgN8NeJ4A/cEIf/NpWDmmNuttNsQHic8QlzgRXK7jTTX+KSJ/nyFadEUr3ROLl/rmYdlsorZKg== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046);SRVR:BL2PR15MB0931;BCL:0;PCL:0;RULEID:;SRVR:BL2PR15MB0931; X-Forefront-PRVS: 00603B7EEF X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(7916002)(24454002)(189002)(377454003)(199003)(106356001)(50466002)(93886004)(23726003)(1076002)(47776003)(4001350100001)(46406003)(97736004)(19580395003)(42186005)(189998001)(83506001)(97756001)(105586002)(92566002)(6116002)(68736007)(8676002)(77096005)(76176999)(50986999)(5660300001)(98436002)(226693001)(54356999)(7846002)(81156014)(4326007)(305945005)(110136002)(81166006)(9686002)(86362001)(101416001)(2906002)(586003)(575784001)(2950100001)(7736002)(33656002)(18370500001)(3826002);DIR:OUT;SFP:1102;SCL:1;SRVR:BL2PR15MB0931;H:shli-mbp.local;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;BL2PR15MB0931;23:klJ14GScIJvAhJOBc14PJ5KjYBXgisvFpm/HOJGLy?= =?us-ascii?Q?1RgcCV4y6wMX16HD7IoEFcH00Q1O7Mmp4JYEIDxyq4s7Ap7ulty1eBPAzYq3?= =?us-ascii?Q?4bDLg5NVa3b9i1xsXBMM5Mh3ym7FhuUrFOIVUYwDRieFqal2NnJ6bZJzkN12?= =?us-ascii?Q?KtzrTWfQjMj1ABwjuIbLd+Dqp0BujtEVkAlbQ7xH8A+myJqoT8LMrTZXf2ev?= =?us-ascii?Q?HGN+nBA5tHhPGjKFe3cOQ7PQ8MyaHExLsuhlBVZLXNt43CnUI+q0K0zL+jSS?= =?us-ascii?Q?s4j0LixhvkwTXqBqsVPeAfvw8jD0QemRaPjVklJL5PNZrK3q85MsWsl9V5RJ?= =?us-ascii?Q?1VzEGtHugXdEMD+xoTPD97yjY9WjtKMhxo22lG3ytbtKEaGAOM3RysC+M1RR?= =?us-ascii?Q?pKTWTNRs5VxDgFjh+Usi//dmaFGqEGYvVxp31+Emn0l4IJWOL3k21bpRihjF?= =?us-ascii?Q?6IQ0yzk9BKTrTY1xfzPG+DZiPbcBnmo27TkbApJSf75bXJWobipZUK8W0Tp2?= =?us-ascii?Q?PVQDTgDzze1qUj5fkJtAOOy3mxlPUyRTmhTKZtpYISZ1nFYfQNEoyCiALK6X?= =?us-ascii?Q?+aRJZS/8gmNoEYtqLFtXlSsOGsE/eUb29VQ/JJX8xpVV7zkd+2aKwTFJnjJX?= =?us-ascii?Q?GTPB3YcVr2CKEoCb22ibMX018wPDiH1drs8S1v37hvzG/12ju73Wbx9ZaCFU?= =?us-ascii?Q?Ooy/LTDR76/3Uke2faYlam7l4xGxECIoZ5pcc1GCxT02YPppyYlW1xALf5KW?= =?us-ascii?Q?M93ki/AHWsX/1VkKLk8lyoWr3xQI/BPAaDLD3ZjmvipKS1QiUpb/iaHNp6py?= =?us-ascii?Q?tEkSrlA8L7PpIV+Ep99DRu36M/8+UGqDg1pYNiU8cTjRHcD6P8hNmhvQ6v8i?= =?us-ascii?Q?CkxQQBn5/fH9WxRy7yN4mSzU6MRC3m4Fh+xCASRCB/wQUHIBDGQp9KfFg6o2?= =?us-ascii?Q?vhC31mYaoNDL9U1XdcI6prvEcqqUVr5xYmFLsQ9R9j0DLVWW2Hvj7m9I3d3w?= =?us-ascii?Q?otYXcI+r/iqwiC7Nr3r5IER6OB224oioXlK/7SLtXO1HWHbXDighWdGSQwoQ?= =?us-ascii?Q?WnTGAQRPvAoUO4OWk8etlSzwuVM70a6MyV7rACFKeFN/hFIwNPwdLD4kKA+1?= =?us-ascii?Q?lJO5dgYbQ3ElJZpfkzQVJfuTiXMP6FTDcjq0Rm5yNMjbFQKP1Tn1Yg5g7eg1?= =?us-ascii?Q?zGV4tE0T1vCkLs452m8w/dnH562I8DRbx80tZF0ojNr8K6humpd+z8mtj8Vy?= =?us-ascii?Q?hYkuoagOdn01E3p2aQ=3D?= X-Microsoft-Exchange-Diagnostics: 1;BL2PR15MB0931;6:BluBkUusihF4d62Nx/2tn0qrRg90CRNnKjiV04cmre0nQQVm4lAhz6JwjrKEhVRInkeJrkpVj8+2uW4r0zYm5XDC3yV76t7rQZIFXMdkmbF0FZivyh6WJOlvC1jR1xgnOVlnmhBzS96mauy9YH+SIZywIiilIYOhUcQzzEFJ2tqC3Z5ZjUCYHHSZpUHLc4zhMhlX0Zbm8BUq+zAKHHD/nld1GgkX+0JMkBWmh/Ujvi/PafAE9/WqewvNGwh7ogAE26pH0GsnX9q+Ut6bLJ3kqRhot+GW8hW59CgAfqDYcHg=;5:uvWJfahXJ4BZQjmXCx9h47KzN3+ZdKR0Qtzt+Vu3PIaYZ/Bm8B91DqrutJ0StDD14Dfq0rhdON95ABQ3aCZJEIYz4JCPU+oNB7c7Esz2zv4vgVnSDVfLt9pvrUnsLDTESw0P5us5NCm+/HHB/7e4jg==;24:pvpBtPPaeMl2FlqLdltMOfa4z7ix5Y8/SqFGbHHk+C+2e/C5eCj+I4H87PZjnIvwpUCC8M73m2LMIQvA6AOHIMqqKDv/+pZtFte8AoKm0F8=;7:bciTEhPYns86WR3E1+ZzZw7B0hL+xEp0QKJXAv4jRcUvcpJAbC7M1M0igsMMzZVZvowXZFTqCQsjsJ9VibJ4QRCOo/+k8Az5ecnhBRzANQIrxhVvhLCOkeWfpBIJCl59u6BWQWfYuSY/dsrcSIvF86dZf21o9c9RlpDAKmElpU3DcC9+gAB/H0yCHjdWN7jci9X/JquSSsAwpmSR9RA69+b4i4b/P/uGQi8uCw0Q0drEEo3hv8W5jffs7cWmb7mc SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;BL2PR15MB0931;20:CB7Hxsc/vRtkKq243FpvzQiFy/w5NgUELkHrx5nkBcb1ghCufEZLBsPOowvhrgJqtEetMLTaXZbc7NUVbBF34ZNOHIg0s4Kh7y4DzzOb7pMgBnkJ/4O63vhf9yCwbtGjEJMeq2tlpHvkdC0zNQKKCQHVu8pPXQKo4L/M4qdwd9E= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2016 18:15:45.2138 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2PR15MB0931 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-09-09_10:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Sep 09, 2016 at 08:03:42PM +0200, Stefan Priebe - Profihost AG wrote: > Am 08.09.2016 um 19:33 schrieb Shaohua Li: > > On Thu, Sep 08, 2016 at 10:16:59AM -0600, Jens Axboe wrote: > >> On 09/08/2016 02:23 AM, Stefan Priebe - Profihost AG wrote: > >>> Hi, > >>> > >>> while trying Kernel 4.8-rc5 my raid5 breaks every few minutes. > >>> > >>> Trace: > >>> ------------[ cut here ]------------ > >>> kernel BUG at block/blk-core.c:2032! > >>> invalid opcode: 0000 [#1] SMP > >>> Modules linked in: netconsole ipt_REJECT nf_reject_ipv4 xt_multiport > >>> iptable_filter ip_tables x_tables 8021q garp bonding sb_edac edac_core > >>> x86_pkg_temp_thermal coretemp kvm_intel kvm i2c_i801 irqbypass i2c_smbus > >>> ipmi_si crc32_pclmul i2c_core ghash_clmulni_intel shpchp ipmi_msghandler > >>> button loop fuse btrfs dm_mod raid10 raid0 multipath linear raid456 > >>> async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq > >>> raid1 md_mod sg sd_mod ixgbe i40e mdio usbhid ehci_pci ehci_hcd ahci > >>> usbcore ptp libahci usb_common megaraid_sas pps_core > >>> CPU: 8 PID: 1105 Comm: md0_raid5 Not tainted 4.8.0-rc5-00003-g3abda5c #2 > >>> Hardware name: Supermicro X10DRH/X10DRH-iT, BIOS 1.0c 02/18/2015 > >>> task: ffff97de5e1e0000 task.stack: ffff97de597a0000 > >>> RIP: 0010:[] [] > >>> generic_make_request+0x1c0/0x1d0 > >>> RSP: 0018:ffff97de597a3aa0 EFLAGS: 00010286 > >>> RAX: ffff97de5e1e0000 RBX: ffff97dd227e5030 RCX: 0000000000000000 > >>> RDX: ffffffffc0000001 RSI: 0000000000000001 RDI: ffff97de5e7d9db8 > >>> RBP: ffff97de597a3ad8 R08: 0000000000000008 R09: 0000000000000000 > >>> R10: 0000000000000000 R11: 0000000000000001 R12: 00000000ffffffff > >>> R13: ffff97de5aa20c00 R14: 00000000000002f0 R15: ffff97e65dce0e00 > >>> FS: 0000000000000000(0000) GS:ffff97e67f200000(0000) knlGS:0000000000000000 > >>> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > >>> CR2: 00007f0e4e1ec000 CR3: 0000000078c06000 CR4: 00000000001406e0Stack: > >>> ffff97de597a3b50 0000000000001000 0000000000000000 ffff97dd227e4c80 > >>> ffff97de5aa20c00 00000000000002f0 ffff97e65dce0e00 ffff97de597a3ba0 > >>> ffffffffc02595db ffffffffc025e04b 00000001597a3b01 0000000200000006 > >>> Call Trace: > >>> [] ops_run_io+0x3bb/0x990 [raid456] > >>> [] ? raid_run_ops+0xefb/0x1520 [raid456] > >>> [] handle_stripe+0x9a6/0x2280 [raid456] > >>> [] ? default_wake_function+0x12/0x20 > >>> [] ? autoremove_wake_function+0x12/0x40 > >>> [] handle_active_stripes.isra.54+0x193/0x4b0 [raid456] > >>> [] ? __release_stripe+0x15/0x20 [raid456] > >>> [] raid5d+0x4a9/0x740 [raid456] > >>> [] ? init_timer_key+0xa0/0xa0 > >>> [] md_thread+0x12b/0x130 [md_mod] > >>> [] ? wait_woken+0x90/0x90 > >>> [] ? find_pers+0x70/0x70 [md_mod] > >>> [] kthread+0xdb/0x100 > >>> [] ret_from_fork+0x1f/0x40 > >>> [] ? kthread_park+0x60/0x60 > >>> Code: bd 70 08 00 00 f0 49 83 ad 70 08 00 00 01 74 05 e9 5a ff ff ff 41 > >>> ff 95 80 08 00 00 e9 4e ff ff ff 48 c7 40 08 00 00 00 00 eb 8c <0f> 0b > >>> 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 > >>> RIP [] generic_make_request+0x1c0/0x1d0 > >>> RSP > >>> ---[ end trace 457dbe5e9cdd3473 ]--- > >> > >> CC'ing Shaohua - this is: > >> > >> BUG_ON(bio->bi_next); > >> > >> which doesn't look healthy. > > > > Hi Stefan, > > does below patch help? Looks there is a race condition introduced recently. > > Yes this one fixes it. Thanks, will push to Linus soon.