From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1764900AbcINTuZ (ORCPT ); Wed, 14 Sep 2016 15:50:25 -0400 Received: from mail-cys01nam02on0113.outbound.protection.outlook.com ([104.47.37.113]:61223 "EHLO NAM02-CY1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1756423AbcINTuU (ORCPT ); Wed, 14 Sep 2016 15:50:20 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=waiman.long@hpe.com; Message-ID: <57D9A633.2030904@hpe.com> Date: Wed, 14 Sep 2016 15:34:11 -0400 From: Waiman Long User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12 MIME-Version: 1.0 To: Linus Torvalds CC: "Theodore Ts'o" , Arnd Bergmann , Greg Kroah-Hartman , Linux Kernel Mailing List , Scott J Norton , Douglas Hatch Subject: Re: [PATCH] random: Fix kernel panic due to system_wq use before init References: <1473879781-23819-1-git-send-email-Waiman.Long@hpe.com> <57D9A1A9.8050506@hpe.com> In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [72.71.243.249] X-ClientProxiedBy: YQXPR01CA0040.CANPRD01.PROD.OUTLOOK.COM (10.165.102.178) To CS1PR84MB0312.NAMPRD84.PROD.OUTLOOK.COM (10.162.190.30) X-MS-Office365-Filtering-Correlation-Id: 1115175a-0fc2-4bf4-4654-08d3dcd61f3f X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0312;2:8/XbtPcu8frhH043vZmzymEAWY3Lv2Sa7AXErw9yxuwlWbXK2511DCqPzo1Eko0B/+vyrNtt9F19LaFgglNyWiUnnDIO/nDrLBLEQKOwSWrjuDxMexj5ZoKlBmyTFSzZidPFLzldPYzCqpyXtQg/lN5Yb/oJtmJ0/aKgzKr22MI/T4ZHz7hM7/Gr4o7K/zlG;3:3Iqb3WylGaxokakW7TtgQN77cP+nILirqtzOqsdRkIDrzm3gsREv3xiR7yYzY9A8KjZccHBoL3MDHx8PG7jrDn9NcA6M4k6B1yQaCOHkCoTJ9iC89vbMImfQaVxt9J9P;25:qYqjwp+1ycRTJ5UY1LepZSyffVyvvhWfNyrp7MVXS5G1rE1glFz110FsG6ry67KiaNBYIQt9qYr3pe4FiD3lOkGfkuq1XluzZjX5bPevP0uh2BYxwx+TrdOze4hF2PcaPlbDszhL3vjtI8CC7U6ipIabA0BMSlH1voYkDbhKuhLhdWAJUOz3lj7FP77fjO2k9e5FVndkIR8Qe/TcKPOXjjlbtEupPSpiYNZolh1P+q5nL8Fb3JI6RtN1bdYe/Dn4GgTz7+/lNlB/KX8ADEnFP/GJ5LsY0U3wAJqigGgWcrpXQ9Jf2qL2caNqrK+J4xdJtQhkJeA+LH2yrZ+gyLFBcYXmJpoGF8bUMC/QHke8G9k9VgSA9MIPfGMaUK+4CVxnZzhu3bT0HC88v0501VX3gNTi2J3bb31S+lKYQq5Ruug= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0312; X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0312;31:JB9GRr4A5CJkfdhxfNCQ3vNurGm1RGCWkt4hFtpqeK+W/gKdhJpj9KEdzEvsWSJKFFj1W6f0VjxoCOVRwOGFx1ezoLKE+zONdExXbayk0KfkykN7a45A9c1DopsDS2AnkAc50uqMJ0skYJwrOfd2eSh+hknY6wlvXhDPu+1aVTvKAV4nj76FVax0e14g46PGiEa6LvENdRDpSbSedAI8zyYdmitp4F9NK4jEiwnEQN4=;20:cKZQ6mC0R22SnWyeNPGp/rCm/3H7wsPiBl/pHfaYYtmr6lLjBDdmH7M39J2mPUG79N2D8Bn/kjaR+mvnzJ4PKTnKNqJZjg5t/if2HukNMCB54jtNLpoo0E5ArcXj1vq8tdrHd3WXaCC5QiafnMxFhYOI7YHhmZM197HtB71276iV5UWsIy7PjrZ+dPLc+qRPVpS1zoegjEo1msnCus3GIVqZmE6bEMRFFvOxbZWrWgqshNHljjh03rH58Z5JL7C2UoOjkKXvKdZkRFtm4x2bj8VeCVdi0kE+6oOm4VMUNReC0/jtkepQtIDoci9Syj5bO1afn4zr6pO+IAepEAKzf/2UyOWpNQDBX2jxAwo1sNj8flwmi+yOGxSq8YD5u3a2d2TtftdcVhl9ZhrliZh6kesvryUrSVFrx08CxUgpJDQ2/X+9M4XUUf2e2gAA/Os8vk/WBWUtVgbPsYFPh8RGOI/xEyAuVx9oG6HoGl4xKLQURg05EIOj4wL0lFXD9sjj X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(227479698468861); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6055026);SRVR:CS1PR84MB0312;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0312; X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0312;4:/L0wb0On6/CegYb+QXs6tU587S7WeK5grTnOiHlZOR6/af8bqkD56gQ6CyFCEql/wATmpK8JTA5CPlQkhk9yM2mvk4pZt+5EL1hebQAWe8bAjgIKdEHa7/xv8kqgMSz1nP559TfGKCZez/b3E6NcR1DL4N2k7zfo4mdx+Voy6mkokl5RMVBtjY5Vgs87z29J2k/ARLUXxwRVidoBwGFoJVWfdU9WBtfj1pJ3gsMUgNQgg7bcxDTYHVqD7j+y4YwQ4UXQxRk0hsjkL3zD70o+qGvwITHZP+I11Ew3p2pzSBbl4Qj9Muucnod/2C4vRSz6y0DhYltINlByxI6WlXrrL1EFvwFhNZ6FOZRBuoZy+C1SYugx2+g6mXPq/OZxGIvQQqsQ9VYWFrdJhJK8QklM/ci6oLcbdvP5MLAg081sOIaboUVqGVMnhnJjzQU836Vb X-Forefront-PRVS: 006546F32A X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6049001)(6009001)(7916002)(189002)(24454002)(199003)(377454003)(80316001)(4001350100001)(189998001)(19580395003)(230700001)(19580405001)(6116002)(3846002)(586003)(23676002)(8676002)(86362001)(83506001)(50466002)(92566002)(110136003)(33656002)(5660300001)(81166006)(81156014)(64126003)(77096005)(2950100001)(42186005)(87266999)(65816999)(97736004)(54356999)(76176999)(101416001)(50986999)(106356001)(36756003)(105586002)(66066001)(65806001)(7846002)(7736002)(68736007)(59896002)(65956001)(4326007)(117156001)(2906002)(47776003)(305945005);DIR:OUT;SFP:1102;SCL:1;SRVR:CS1PR84MB0312;H:[192.168.142.204];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDUzFQUjg0TUIwMzEyOzIzOjB1Z2d2aVRCbmc0MTE1SjBqT3cyS2NYQjBr?= =?utf-8?B?REtKc0tzeGQ3R1ZRSnQ0emFvdnJxYkx3anJOdWtEMlNMUiswdUlHdzMzVDRP?= =?utf-8?B?SmdaSFpNS3RRSVN5UjBoMDkwdEdIVFNrVUVhQ0w3SnAzV1BZbzlZVzR6Wkt1?= =?utf-8?B?blNpL3pKeEkyQ0NWSFU1bVJSOGF6YXFQeDduRTFkTGFGZjgrbDZ1ajdWaHVK?= =?utf-8?B?VnNQWmNjRjl0NEVpTGdVSzBWYWpvNVZ2ZHVjOXgzQ3duNW5yTW1ybm5ZQ3hK?= =?utf-8?B?SVVqQTllUjFBZEkrVzdiNm5JTVgrSllxOHN0RmhHWm9KZG5BU3RTVEVvU0xB?= =?utf-8?B?VDdkdG8wN21KUjJSWmsrb0Z4SWJEMmpMOXZaMEFhTDdtWUhFbGlkVEpmbWVj?= =?utf-8?B?cHo5M1RwZHNsekpnK3ZFNXVTRTA2Mm9LTFk5OHdqKzZ5c1Z1aWlCM0F6ekNJ?= =?utf-8?B?Vm9LZ0Z3em5WTlhiUVhFRll4UzZqajFEVnNOL1ZaZ3RRZGIwK2JRM1V1RWx1?= =?utf-8?B?VXVLZmFBOEliaEdwZDBZbmtOQ3MvR1NObTgza2UrYVE4RUNPYjU2Q3hFUWlD?= =?utf-8?B?dkxNSjNkeTVVWWZTMW9KSXZudUdBQU83ZHZia016U0FQNjJ2V2s0c05IUWU1?= =?utf-8?B?SHZVa3FnY0hhcWErdU84YzloOVlabUlIVmN1T0toYUdjWVNRRnRoNUVRZlVL?= =?utf-8?B?cnRHNmovWXR0c1FlWStOcDlpL0JKU0NlT1JxSmw2NzhsWVRVSUhVMlN5bzJD?= =?utf-8?B?MlFTemU3YVM4a3JJVHRaNzgxZjJ4RDRqeHVWV3ZhM1l1ZE5YTzVXY2kxd3hv?= =?utf-8?B?V3ZTUzllaWdoZlZFQjg0WUdhWVFHZnQ4aUI3cDNPNUJWQ2dPQjlGbjRpdHJ2?= =?utf-8?B?VXZVNlVFUis4RVNoWEJCNmpZQ3VRRTR6dGxLVEMyQXhsWUlYUUF5anlvSXFU?= =?utf-8?B?cURObHhXRlN6SHJTQ0dRSVRTVS8yeVdLdGVNbzNXbTd3K0FKeUxjZlJuYzRB?= =?utf-8?B?QkcyK1pmNEsvcXltaXBxdE4zaVNhdndWQlZKUkxRMjZIRE5EeEhlQURJT0g4?= =?utf-8?B?bUZoQUR3U3daMFRKRmtQL1Q4ZktFc3ZPRmo5a2dvYm04M3FNWnNiNFVrVlpk?= =?utf-8?B?ajV3SDJNZWNGRTg4bnlFWnVPbGNQNko5a3IzWHArYnlLU3NUSXpFeFo5bkFx?= =?utf-8?B?a0xzcFV5OEU3OHZtbkJ0cmd4a3lNcmxkLzFjNVk1RUNvbnZTUjB4WitBdjBa?= =?utf-8?B?VnorcTRaQ2ZTZElwYXVmMXlrSDZNQmM3QnlMa01tQ2RlK1RVZ2dkSlR4VVRD?= =?utf-8?B?WUVmVHZrbGpiZU84ajVjY1FFSlRlanRXbUw2NVZYbVNoNGtLSnZEL1licGll?= =?utf-8?B?RDdzQ0xQSlRLRHNDWG04NVd6eXVJNkZtc1RlNlR5a2NXalJYRzlVMW5wMUt4?= =?utf-8?B?ZjFJa3U2Q3FvVU44T2RIN2VmTDR5SkFHa3VUeVJaTitDUlQ4RjB5OGcxYTVX?= =?utf-8?B?RXV6a1ZXRGJ2Zk1pNU1BSmRPektJWkV5L3p3blA3VWRNd2IxaHRQaFFCOHFF?= =?utf-8?B?UzJxSVkwWlBXbHFwQlhtanc0ekFicTVLaG1Vc2U5Z1VIckI3RGk0anFqVytm?= =?utf-8?B?UEV3djNHdm1XK2sxRTZzSXk0NkNseWRaeC9GOXZlanQ1NWhrWjNmQ3BoWFRn?= =?utf-8?B?eWF0K0FGeG1SaGVlSkhCTUtiODJseHVoNVZrYnFvQ1IrVklVeEo5S0RmZFpT?= =?utf-8?B?c202bDkxM1JwaUVBcS9McXpWZ3Q3TWNQTy8vSDFqTURhdzRIZkloc1BSNk5M?= =?utf-8?Q?j3nnCSkcQlhXI?= X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0312;6:yZpv8+U3P8VTT8DNLkOkraFptC1I8VHafEq4Dd7h2c39vshx+xkSKcRFvV3eyaJPc88/YU9IealbdMhktS1tPBcsqcY9+TsQh8OwNTthoqHMbwpKKjQIU531HCnhSP3/ffj4I4H8Rjdg8nAj7HZLyvv2VQ2Wx2cox1QohpeVz0NC7F0UU2hkF9LS7BmcTMYaK4Ci4AptGV3c/BDGo9siw05yiqAQiD/twqZ0SmC6TVDGalHqTUfNAGeKkdQLBRM46wbRCjftSOFWFtQSMw/I/EvZp7dmNzjsZJ/a7ms7dKRRQk0/RVkAaD4OU2cho1eZBtR2J6Fs1pCtWuwDdGFyjg==;5:ZAXDCYGOI3MGrNUJJbPPgDpt16/nlwNoSAs97BBo4uG8Fql+c3RJFRD7QrtfP2WVT+Xo5guJqnuZM7GwZiFAO4KKluetfgnPAc5hhi0e452y/V9AoHj3Md1nPmjKvYoo5IG+2d88En5AtGimXaHeXA==;24:xIbQrFcd3u3b9uuicIvtPwQNyRZxYYiA8tlexZTOiqXG8SiCvGW63agcX0Uu0D0B+/3LxFiY/nYrhasbqhppj25dk8R/HyXtbiuifkiOfiA=;7:oAs1HfJU0TuY4PM6cLxquC247AocHxHQXKuYbonTB28gga1q+wdhxjuZdWwt7OWJDcBrCni4pZdApYYC1wKd5W6b+r83/DjH5YRU6a03DrUmr4kq/FdZ6w6vN7Q6/lquFVwHvBTcrGdaVzrNYEUNgv/jkbE2nc2l6JJjB6lTOm48eHV+32R0yeEwK5aycaL5OTmC2jUKXxhB3ZepmIAXb+rU6kuY13/IqoiGj1vf8NB7B3jjpR4MCxhFD7uc0tlt SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2016 19:34:18.4085 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CS1PR84MB0312 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 09/14/2016 03:19 PM, Linus Torvalds wrote: > On Wed, Sep 14, 2016 at 12:14 PM, Waiman Long wrote: >> In the stack backtrace above, the kernel hadn't even reached SMP boot after >> about 50s. That was extremely slow. I tried the 4.7.3 kernel and it booted >> up fine. So I suspect that there may be too many interrupts going on and it >> consumes most of the CPU cycles. The prime suspect is the random driver, I >> think. > Any chance of bisecting it at least partially? The random driver > doesn't do interrupts itself, it just gets called by other drivers > doing intterrupts. So if there are too many of them, that would be > something else.. > > Linus I can try, but the 16-socket system that I have at the moment takes a long time (more than an hour) for one shutdown-reboot cycle. It may not be really more interrupts in 4.8, it may be that the random driver just somehow run very slow on my test machine as it seems to have a major rewrite in the 4.8 cycle. So I would like to solicit what sort of test that I can run to pinpoint where the problem is. I currently has the machine till the end of the week. Cheers, Longman