From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756062Ab1ESMG4 (ORCPT ); Thu, 19 May 2011 08:06:56 -0400 Received: from ns2.q-leap.de ([88.79.172.217]:40812 "EHLO mail.q-leap.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752739Ab1ESMGz (ORCPT ); Thu, 19 May 2011 08:06:55 -0400 X-Greylist: delayed 599 seconds by postgrey-1.27 at vger.kernel.org; Thu, 19 May 2011 08:06:54 EDT Message-ID: <4DD50584.4010408@q-leap.de> Date: Thu, 19 May 2011 13:56:52 +0200 From: Peter Kruse Organization: Q-Leap Networks GmbH User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.17) Gecko/20110414 Lightning/1.0b2 Thunderbird/3.1.10 MIME-Version: 1.0 To: Christoph Lameter CC: David Rientjes , eric.dumazet@gmail.com, linux-kernel@vger.kernel.org Subject: Re: I have a blaze of 353 page allocation failures, all alike References: <4D53FE43.8030106@q-leap.com> <4D5A2EDB.8060603@q-leap.com> <4D5BC16A.2090205@q-leap.com> <4D5BF56F.1000504@q-leap.com> <4D5CCEED.3010501@q-leap.com> <272bf0cc51439a2ab31ee2f06317dd9f.squirrel@www.q-leap.de> <4D6648B5.1090306@q-leap.com> <4DA4692D.7080207@q-leap.de> <4DA54D2A.3050206@q-leap.de> In-Reply-To: X-Enigmail-Version: 1.1.1 Content-Type: multipart/mixed; boundary="------------090108070906030405080108" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This is a multi-part message in MIME format. --------------090108070906030405080108 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hello again, you may remember, we have a server running 2.6.32.29 and once in a while it just crashes, meaning that it just stops responding, and we have to reset it. The kernel itself and some processes are still running, and it is possible to initiate the sysRQ. The mentioned allocation failures are gone after the update to the kernel version, but the server crashed again after 40 days running without any problem. At this time kswapd0/1 started to consume 99% CPU time and until the server was reset 8 hours later never used less than 50% CPU time. Shortly after that (half an hour) the disk reads dropped down to zero. And a program produced the attached Call Trace. I also attach the process information of kswapd (provided by collectl). Thanks, Peter On 04/13/2011 06:17 PM, Christoph Lameter wrote: > On Wed, 13 Apr 2011, Peter Kruse wrote: > >> Hello, >> >> thanks for your replies, I appreciate that. >> >> On 04/13/2011 03:34 AM, David Rientjes wrote: >>> On Tue, 12 Apr 2011, Christoph Lameter wrote: >>> >>>> The problem maybe that you have lots and lots of SCSI devices which >>>> consume ZONE_DMA memory for their control structures. I guess that is >>>> oversubscribing the 16M zone. >> >> but there are only two devices: > > The output you sent me showed a long list of devices. Maybe there is a > broken driver /device that continues being probed? > -- Peter Kruse Q-Leap Networks GmbH phone: +497034-2776-175, mobile: +491522-1593877 --------------090108070906030405080108 Content-Type: text/plain; name="messages" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="messages" May 16 02:24:36 beosrv1-t kernel: [3504667.830773] pv D ffffffffffffffff 0 20917 10814 0x00000000 May 16 02:24:36 beosrv1-t kernel: [3504667.830797] ffff8806e77f5c88 0000000000000082 ffff8806e77f5be8 ffffffff8102e847 May 16 02:24:36 beosrv1-t kernel: [3504667.830826] 000000000000dd08 ffff8806e77f5fd8 00000000000115c0 00000000000115c0 May 16 02:24:36 beosrv1-t kernel: [3504667.830854] ffff8806554515c0 0000000000000005 ffff88024818ccd0 ffff880c3b9cc6b0 May 16 02:24:36 beosrv1-t kernel: [3504667.830882] Call Trace: May 16 02:24:36 beosrv1-t kernel: [3504667.830899] [] ? enqueue_task_fair+0x16d/0x17a May 16 02:24:36 beosrv1-t kernel: [3504667.830918] [] ? mutex_spin_on_owner+0x50/0x61 May 16 02:24:36 beosrv1-t kernel: [3504667.830939] [] __mutex_lock_slowpath+0xe9/0x133 May 16 02:24:36 beosrv1-t kernel: [3504667.830957] [] mutex_lock+0x16/0x2d May 16 02:24:36 beosrv1-t kernel: [3504667.830975] [] pipe_read+0x7d/0x3a8 May 16 02:24:36 beosrv1-t kernel: [3504667.830991] [] ? core_sys_select+0x200/0x254 May 16 02:24:36 beosrv1-t kernel: [3504667.831011] [] do_sync_read+0xe7/0x12d May 16 02:24:36 beosrv1-t kernel: [3504667.831027] [] ? remove_wait_queue+0x48/0x4d May 16 02:24:36 beosrv1-t kernel: [3504667.831045] [] ? ktime_get+0x65/0xc4 May 16 02:24:36 beosrv1-t kernel: [3504667.831076] [] ? autoremove_wake_function+0x0/0x38 May 16 02:24:36 beosrv1-t kernel: [3504667.831111] [] ? poll_select_copy_remaining+0xc5/0xea May 16 02:24:36 beosrv1-t kernel: [3504667.831146] [] vfs_read+0xab/0x147 May 16 02:24:36 beosrv1-t kernel: [3504667.831177] [] sys_read+0x47/0x6f May 16 02:24:36 beosrv1-t kernel: [3504667.831209] [] system_call_fastpath+0x16/0x1b May 16 02:26:35 beosrv1-t kernel: [3504787.810748] pv D ffffffffffffffff 0 20917 10814 0x00000000 May 16 02:26:35 beosrv1-t kernel: [3504787.810773] ffff8806e77f5c88 0000000000000082 ffff8806e77f5be8 ffffffff8102e847 May 16 02:26:35 beosrv1-t kernel: [3504787.810802] 000000000000dd08 ffff8806e77f5fd8 00000000000115c0 00000000000115c0 May 16 02:26:35 beosrv1-t kernel: [3504787.810829] ffff8806554515c0 0000000000000005 ffff88024818ccd0 ffff880c3b9cc6b0 May 16 02:26:35 beosrv1-t kernel: [3504787.810857] Call Trace: May 16 02:26:35 beosrv1-t kernel: [3504787.810875] [] ? enqueue_task_fair+0x16d/0x17a May 16 02:26:35 beosrv1-t kernel: [3504787.810893] [] ? mutex_spin_on_owner+0x50/0x61 May 16 02:26:35 beosrv1-t kernel: [3504787.810914] [] __mutex_lock_slowpath+0xe9/0x133 May 16 02:26:35 beosrv1-t kernel: [3504787.810932] [] mutex_lock+0x16/0x2d May 16 02:26:35 beosrv1-t kernel: [3504787.810950] [] pipe_read+0x7d/0x3a8 May 16 02:26:35 beosrv1-t kernel: [3504787.810966] [] ? core_sys_select+0x200/0x254 May 16 02:26:35 beosrv1-t kernel: [3504787.810985] [] do_sync_read+0xe7/0x12d May 16 02:26:35 beosrv1-t kernel: [3504787.811002] [] ? remove_wait_queue+0x48/0x4d May 16 02:26:35 beosrv1-t kernel: [3504787.811020] [] ? ktime_get+0x65/0xc4 May 16 02:26:35 beosrv1-t kernel: [3504787.811051] [] ? autoremove_wake_function+0x0/0x38 May 16 02:26:35 beosrv1-t kernel: [3504787.811086] [] ? poll_select_copy_remaining+0xc5/0xea May 16 02:26:35 beosrv1-t kernel: [3504787.811121] [] vfs_read+0xab/0x147 May 16 02:26:35 beosrv1-t kernel: [3504787.811152] [] sys_read+0x47/0x6f May 16 02:26:35 beosrv1-t kernel: [3504787.811184] [] system_call_fastpath+0x16/0x1b May 16 02:28:35 beosrv1-t kernel: [3504907.579079] pv D ffffffffffffffff 0 20917 10814 0x00000000 May 16 02:28:35 beosrv1-t kernel: [3504907.579103] ffff8806e77f5c88 0000000000000082 ffff8806e77f5be8 ffffffff8102e847 May 16 02:28:35 beosrv1-t kernel: [3504907.579132] 000000000000dd08 ffff8806e77f5fd8 00000000000115c0 00000000000115c0 May 16 02:28:35 beosrv1-t kernel: [3504907.579160] ffff8806554515c0 0000000000000005 ffff88024818ccd0 ffff880c3b9cc6b0 May 16 02:28:35 beosrv1-t kernel: [3504907.579188] Call Trace: May 16 02:28:35 beosrv1-t kernel: [3504907.579205] [] ? enqueue_task_fair+0x16d/0x17a May 16 02:28:35 beosrv1-t kernel: [3504907.579224] [] ? mutex_spin_on_owner+0x50/0x61 May 16 02:28:35 beosrv1-t kernel: [3504907.579244] [] __mutex_lock_slowpath+0xe9/0x133 May 16 02:28:35 beosrv1-t kernel: [3504907.579262] [] mutex_lock+0x16/0x2d May 16 02:28:35 beosrv1-t kernel: [3504907.579280] [] pipe_read+0x7d/0x3a8 May 16 02:28:35 beosrv1-t kernel: [3504907.579296] [] ? core_sys_select+0x200/0x254 May 16 02:28:35 beosrv1-t kernel: [3504907.579316] [] do_sync_read+0xe7/0x12d May 16 02:28:35 beosrv1-t kernel: [3504907.579333] [] ? remove_wait_queue+0x48/0x4d May 16 02:28:35 beosrv1-t kernel: [3504907.579351] [] ? ktime_get+0x65/0xc4 May 16 02:28:35 beosrv1-t kernel: [3504907.579383] [] ? autoremove_wake_function+0x0/0x38 May 16 02:28:35 beosrv1-t kernel: [3504907.579418] [] ? poll_select_copy_remaining+0xc5/0xea May 16 02:28:35 beosrv1-t kernel: [3504907.579453] [] vfs_read+0xab/0x147 May 16 02:28:35 beosrv1-t kernel: [3504907.579484] [] sys_read+0x47/0x6f May 16 02:28:35 beosrv1-t kernel: [3504907.579516] [] system_call_fastpath+0x16/0x1b May 16 02:30:35 beosrv1-t kernel: [3505027.351433] pv D ffffffffffffffff 0 20917 10814 0x00000000 May 16 02:30:35 beosrv1-t kernel: [3505027.351458] ffff8806e77f5c88 0000000000000082 ffff8806e77f5be8 ffffffff8102e847 May 16 02:30:35 beosrv1-t kernel: [3505027.351487] 000000000000dd08 ffff8806e77f5fd8 00000000000115c0 00000000000115c0 May 16 02:30:35 beosrv1-t kernel: [3505027.351515] ffff8806554515c0 0000000000000005 ffff88024818ccd0 ffff880c3b9cc6b0 May 16 02:30:35 beosrv1-t kernel: [3505027.351543] Call Trace: May 16 02:30:35 beosrv1-t kernel: [3505027.351560] [] ? enqueue_task_fair+0x16d/0x17a May 16 02:30:35 beosrv1-t kernel: [3505027.351578] [] ? mutex_spin_on_owner+0x50/0x61 May 16 02:30:35 beosrv1-t kernel: [3505027.351599] [] __mutex_lock_slowpath+0xe9/0x133 May 16 02:30:35 beosrv1-t kernel: [3505027.351617] [] mutex_lock+0x16/0x2d May 16 02:30:35 beosrv1-t kernel: [3505027.351635] [] pipe_read+0x7d/0x3a8 May 16 02:30:35 beosrv1-t kernel: [3505027.351652] [] ? core_sys_select+0x200/0x254 May 16 02:30:35 beosrv1-t kernel: [3505027.351671] [] do_sync_read+0xe7/0x12d May 16 02:30:35 beosrv1-t kernel: [3505027.351688] [] ? remove_wait_queue+0x48/0x4d May 16 02:30:35 beosrv1-t kernel: [3505027.351705] [] ? ktime_get+0x65/0xc4 May 16 02:30:35 beosrv1-t kernel: [3505027.351737] [] ? autoremove_wake_function+0x0/0x38 May 16 02:30:35 beosrv1-t kernel: [3505027.351771] [] ? poll_select_copy_remaining+0xc5/0xea May 16 02:30:35 beosrv1-t kernel: [3505027.351806] [] vfs_read+0xab/0x147 May 16 02:30:35 beosrv1-t kernel: [3505027.351838] [] sys_read+0x47/0x6f May 16 02:30:35 beosrv1-t kernel: [3505027.351869] [] system_call_fastpath+0x16/0x1b May 16 02:32:35 beosrv1-t kernel: [3505147.119797] pv D ffffffffffffffff 0 20917 10814 0x00000000 May 16 02:32:35 beosrv1-t kernel: [3505147.119822] ffff8806e77f5c88 0000000000000082 ffff8806e77f5be8 ffffffff8102e847 May 16 02:32:35 beosrv1-t kernel: [3505147.119851] 000000000000dd08 ffff8806e77f5fd8 00000000000115c0 00000000000115c0 May 16 02:32:35 beosrv1-t kernel: [3505147.119879] ffff8806554515c0 0000000000000005 ffff88024818ccd0 ffff880c3b9cc6b0 May 16 02:32:35 beosrv1-t kernel: [3505147.119907] Call Trace: May 16 02:32:35 beosrv1-t kernel: [3505147.119924] [] ? enqueue_task_fair+0x16d/0x17a May 16 02:32:35 beosrv1-t kernel: [3505147.119942] [] ? mutex_spin_on_owner+0x50/0x61 May 16 02:32:35 beosrv1-t kernel: [3505147.119963] [] __mutex_lock_slowpath+0xe9/0x133 May 16 02:32:35 beosrv1-t kernel: [3505147.119982] [] mutex_lock+0x16/0x2d May 16 02:32:35 beosrv1-t kernel: [3505147.120000] [] pipe_read+0x7d/0x3a8 May 16 02:32:35 beosrv1-t kernel: [3505147.120016] [] ? core_sys_select+0x200/0x254 May 16 02:32:35 beosrv1-t kernel: [3505147.120036] [] do_sync_read+0xe7/0x12d May 16 02:32:35 beosrv1-t kernel: [3505147.120053] [] ? remove_wait_queue+0x48/0x4d May 16 02:32:35 beosrv1-t kernel: [3505147.120070] [] ? ktime_get+0x65/0xc4 May 16 02:32:35 beosrv1-t kernel: [3505147.120102] [] ? autoremove_wake_function+0x0/0x38 May 16 02:32:35 beosrv1-t kernel: [3505147.120137] [] ? poll_select_copy_remaining+0xc5/0xea May 16 02:32:35 beosrv1-t kernel: [3505147.120171] [] vfs_read+0xab/0x147 May 16 02:32:35 beosrv1-t kernel: [3505147.120203] [] sys_read+0x47/0x6f May 16 02:32:35 beosrv1-t kernel: [3505147.120234] [] system_call_fastpath+0x16/0x1b May 16 02:34:35 beosrv1-t kernel: [3505266.888143] pv D ffffffffffffffff 0 20917 10814 0x00000000 May 16 02:34:35 beosrv1-t kernel: [3505266.888168] ffff8806e77f5c88 0000000000000082 ffff8806e77f5be8 ffffffff8102e847 May 16 02:34:35 beosrv1-t kernel: [3505266.888197] 000000000000dd08 ffff8806e77f5fd8 00000000000115c0 00000000000115c0 May 16 02:34:35 beosrv1-t kernel: [3505266.888225] ffff8806554515c0 0000000000000005 ffff88024818ccd0 ffff880c3b9cc6b0 May 16 02:34:35 beosrv1-t kernel: [3505266.888253] Call Trace: May 16 02:34:35 beosrv1-t kernel: [3505266.888270] [] ? enqueue_task_fair+0x16d/0x17a May 16 02:34:35 beosrv1-t kernel: [3505266.888288] [] ? mutex_spin_on_owner+0x50/0x61 May 16 02:34:35 beosrv1-t kernel: [3505266.888309] [] __mutex_lock_slowpath+0xe9/0x133 May 16 02:34:35 beosrv1-t kernel: [3505266.888327] [] mutex_lock+0x16/0x2d May 16 02:34:35 beosrv1-t kernel: [3505266.888345] [] pipe_read+0x7d/0x3a8 May 16 02:34:35 beosrv1-t kernel: [3505266.888361] [] ? core_sys_select+0x200/0x254 May 16 02:34:35 beosrv1-t kernel: [3505266.888381] [] do_sync_read+0xe7/0x12d May 16 02:34:35 beosrv1-t kernel: [3505266.888398] [] ? remove_wait_queue+0x48/0x4d May 16 02:34:35 beosrv1-t kernel: [3505266.888415] [] ? ktime_get+0x65/0xc4 May 16 02:34:35 beosrv1-t kernel: [3505266.888447] [] ? autoremove_wake_function+0x0/0x38 May 16 02:34:35 beosrv1-t kernel: [3505266.888482] [] ? poll_select_copy_remaining+0xc5/0xea May 16 02:34:35 beosrv1-t kernel: [3505266.888517] [] vfs_read+0xab/0x147 May 16 02:34:35 beosrv1-t kernel: [3505266.888548] [] sys_read+0x47/0x6f May 16 02:34:35 beosrv1-t kernel: [3505266.888579] [] system_call_fastpath+0x16/0x1b May 16 02:36:35 beosrv1-t kernel: [3505386.656500] pv D ffffffffffffffff 0 20917 10814 0x00000000 May 16 02:36:35 beosrv1-t kernel: [3505386.656525] ffff8806e77f5c88 0000000000000082 ffff8806e77f5be8 ffffffff8102e847 May 16 02:36:35 beosrv1-t kernel: [3505386.656553] 000000000000dd08 ffff8806e77f5fd8 00000000000115c0 00000000000115c0 May 16 02:36:35 beosrv1-t kernel: [3505386.656581] ffff8806554515c0 0000000000000005 ffff88024818ccd0 ffff880c3b9cc6b0 May 16 02:36:35 beosrv1-t kernel: [3505386.656609] Call Trace: May 16 02:36:35 beosrv1-t kernel: [3505386.656626] [] ? enqueue_task_fair+0x16d/0x17a May 16 02:36:35 beosrv1-t kernel: [3505386.656644] [] ? mutex_spin_on_owner+0x50/0x61 May 16 02:36:35 beosrv1-t kernel: [3505386.656666] [] __mutex_lock_slowpath+0xe9/0x133 May 16 02:36:35 beosrv1-t kernel: [3505386.656684] [] mutex_lock+0x16/0x2d May 16 02:36:35 beosrv1-t kernel: [3505386.656701] [] pipe_read+0x7d/0x3a8 May 16 02:36:35 beosrv1-t kernel: [3505386.656717] [] ? core_sys_select+0x200/0x254 May 16 02:36:35 beosrv1-t kernel: [3505386.656737] [] do_sync_read+0xe7/0x12d May 16 02:36:35 beosrv1-t kernel: [3505386.656754] [] ? remove_wait_queue+0x48/0x4d May 16 02:36:35 beosrv1-t kernel: [3505386.656771] [] ? ktime_get+0x65/0xc4 May 16 02:36:35 beosrv1-t kernel: [3505386.656803] [] ? autoremove_wake_function+0x0/0x38 May 16 02:36:35 beosrv1-t kernel: [3505386.656838] [] ? poll_select_copy_remaining+0xc5/0xea May 16 02:36:35 beosrv1-t kernel: [3505386.656872] [] vfs_read+0xab/0x147 May 16 02:36:35 beosrv1-t kernel: [3505386.656904] [] sys_read+0x47/0x6f May 16 02:36:35 beosrv1-t kernel: [3505386.656935] [] system_call_fastpath+0x16/0x1b May 16 02:38:35 beosrv1-t kernel: [3505506.424861] pv D ffffffffffffffff 0 20917 10814 0x00000000 May 16 02:38:35 beosrv1-t kernel: [3505506.424886] ffff8806e77f5c88 0000000000000082 ffff8806e77f5be8 ffffffff8102e847 May 16 02:38:35 beosrv1-t kernel: [3505506.424915] 000000000000dd08 ffff8806e77f5fd8 00000000000115c0 00000000000115c0 May 16 02:38:35 beosrv1-t kernel: [3505506.424943] ffff8806554515c0 0000000000000005 ffff88024818ccd0 ffff880c3b9cc6b0 May 16 02:38:35 beosrv1-t kernel: [3505506.424971] Call Trace: May 16 02:38:35 beosrv1-t kernel: [3505506.424988] [] ? enqueue_task_fair+0x16d/0x17a May 16 02:38:35 beosrv1-t kernel: [3505506.425006] [] ? mutex_spin_on_owner+0x50/0x61 May 16 02:38:35 beosrv1-t kernel: [3505506.425027] [] __mutex_lock_slowpath+0xe9/0x133 May 16 02:38:35 beosrv1-t kernel: [3505506.425045] [] mutex_lock+0x16/0x2d May 16 02:38:35 beosrv1-t kernel: [3505506.425063] [] pipe_read+0x7d/0x3a8 May 16 02:38:35 beosrv1-t kernel: [3505506.425079] [] ? core_sys_select+0x200/0x254 May 16 02:38:35 beosrv1-t kernel: [3505506.425099] [] do_sync_read+0xe7/0x12d May 16 02:38:35 beosrv1-t kernel: [3505506.425116] [] ? remove_wait_queue+0x48/0x4d May 16 02:38:35 beosrv1-t kernel: [3505506.425134] [] ? ktime_get+0x65/0xc4 May 16 02:38:35 beosrv1-t kernel: [3505506.425165] [] ? autoremove_wake_function+0x0/0x38 May 16 02:38:35 beosrv1-t kernel: [3505506.425200] [] ? poll_select_copy_remaining+0xc5/0xea May 16 02:38:35 beosrv1-t kernel: [3505506.425235] [] vfs_read+0xab/0x147 May 16 02:38:35 beosrv1-t kernel: [3505506.425266] [] sys_read+0x47/0x6f May 16 02:38:35 beosrv1-t kernel: [3505506.425298] [] system_call_fastpath+0x16/0x1b May 16 02:40:35 beosrv1-t kernel: [3505626.404818] pv D ffffffffffffffff 0 20917 10814 0x00000000 May 16 02:40:35 beosrv1-t kernel: [3505626.404843] ffff8806e77f5c88 0000000000000082 ffff8806e77f5be8 ffffffff8102e847 May 16 02:40:35 beosrv1-t kernel: [3505626.404872] 000000000000dd08 ffff8806e77f5fd8 00000000000115c0 00000000000115c0 May 16 02:40:35 beosrv1-t kernel: [3505626.404900] ffff8806554515c0 0000000000000005 ffff88024818ccd0 ffff880c3b9cc6b0 May 16 02:40:35 beosrv1-t kernel: [3505626.404928] Call Trace: May 16 02:40:35 beosrv1-t kernel: [3505626.404945] [] ? enqueue_task_fair+0x16d/0x17a May 16 02:40:35 beosrv1-t kernel: [3505626.404963] [] ? mutex_spin_on_owner+0x50/0x61 May 16 02:40:35 beosrv1-t kernel: [3505626.404984] [] __mutex_lock_slowpath+0xe9/0x133 May 16 02:40:35 beosrv1-t kernel: [3505626.405002] [] mutex_lock+0x16/0x2d May 16 02:40:35 beosrv1-t kernel: [3505626.405020] [] pipe_read+0x7d/0x3a8 May 16 02:40:35 beosrv1-t kernel: [3505626.405036] [] ? core_sys_select+0x200/0x254 May 16 02:40:35 beosrv1-t kernel: [3505626.405056] [] do_sync_read+0xe7/0x12d May 16 02:40:35 beosrv1-t kernel: [3505626.405073] [] ? remove_wait_queue+0x48/0x4d May 16 02:40:35 beosrv1-t kernel: [3505626.405091] [] ? ktime_get+0x65/0xc4 May 16 02:40:35 beosrv1-t kernel: [3505626.405122] [] ? autoremove_wake_function+0x0/0x38 May 16 02:40:35 beosrv1-t kernel: [3505626.405157] [] ? poll_select_copy_remaining+0xc5/0xea May 16 02:40:35 beosrv1-t kernel: [3505626.405192] [] vfs_read+0xab/0x147 May 16 02:40:35 beosrv1-t kernel: [3505626.405223] [] sys_read+0x47/0x6f May 16 02:40:35 beosrv1-t kernel: [3505626.405255] [] system_call_fastpath+0x16/0x1b May 16 02:42:36 beosrv1-t kernel: [3505746.348825] pv D ffffffffffffffff 0 20917 10814 0x00000000 May 16 02:42:36 beosrv1-t kernel: [3505746.348849] ffff8806e77f5c88 0000000000000082 ffff8806e77f5be8 ffffffff8102e847 May 16 02:42:36 beosrv1-t kernel: [3505746.348879] 000000000000dd08 ffff8806e77f5fd8 00000000000115c0 00000000000115c0 May 16 02:42:36 beosrv1-t kernel: [3505746.348907] ffff8806554515c0 0000000000000005 ffff88024818ccd0 ffff880c3b9cc6b0 May 16 02:42:36 beosrv1-t kernel: [3505746.348935] Call Trace: May 16 02:42:36 beosrv1-t kernel: [3505746.348952] [] ? enqueue_task_fair+0x16d/0x17a May 16 02:42:36 beosrv1-t kernel: [3505746.348970] [] ? mutex_spin_on_owner+0x50/0x61 May 16 02:42:36 beosrv1-t kernel: [3505746.348991] [] __mutex_lock_slowpath+0xe9/0x133 May 16 02:42:36 beosrv1-t kernel: [3505746.349009] [] mutex_lock+0x16/0x2d May 16 02:42:36 beosrv1-t kernel: [3505746.349027] [] pipe_read+0x7d/0x3a8 May 16 02:42:36 beosrv1-t kernel: [3505746.349044] [] ? core_sys_select+0x200/0x254 May 16 02:42:36 beosrv1-t kernel: [3505746.349063] [] do_sync_read+0xe7/0x12d May 16 02:42:36 beosrv1-t kernel: [3505746.349080] [] ? remove_wait_queue+0x48/0x4d May 16 02:42:36 beosrv1-t kernel: [3505746.349098] [] ? ktime_get+0x65/0xc4 May 16 02:42:36 beosrv1-t kernel: [3505746.349129] [] ? autoremove_wake_function+0x0/0x38 May 16 02:42:36 beosrv1-t kernel: [3505746.349164] [] ? poll_select_copy_remaining+0xc5/0xea May 16 02:42:36 beosrv1-t kernel: [3505746.349199] [] vfs_read+0xab/0x147 May 16 02:42:36 beosrv1-t kernel: [3505746.349230] [] sys_read+0x47/0x6f May 16 02:42:36 beosrv1-t kernel: [3505746.349261] [] system_call_fastpath+0x16/0x1b --------------090108070906030405080108 Content-Type: text/plain; name="kswapd.collectl" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="kswapd.collectl" #Time PID User PR PPID THRD S VSZ RSS CP SysT UsrT Pct AccuTime MajF MinF Command 01:46:00 77 root 20 2 0 S 0 0 3 0.00 0.00 0 386:07.51 0 0 kswapd0 01:46:00 78 root 20 2 0 S 0 0 7 0.00 0.00 0 305:02.25 0 0 kswapd1 01:47:00 77 root 20 2 0 S 0 0 3 0.00 0.00 0 386:07.51 0 0 kswapd0 01:47:00 78 root 20 2 0 S 0 0 7 0.00 0.00 0 305:02.25 0 0 kswapd1 01:48:00 77 root 20 2 0 S 0 0 3 0.00 0.00 0 386:07.51 0 0 kswapd0 01:48:00 78 root 20 2 0 S 0 0 7 0.00 0.00 0 305:02.25 0 0 kswapd1 01:49:00 77 root 20 2 0 R 0 0 2 55.48 0.00 92 387:02.99 0 0 kswapd0 01:49:00 78 root 20 2 0 S 0 0 7 0.70 0.00 1 305:02.95 0 0 kswapd1 01:50:00 77 root 20 2 0 R 0 0 2 59.88 0.00 99 388:02.87 0 0 kswapd0 01:50:00 78 root 20 2 0 S 0 0 7 0.00 0.00 0 305:02.95 0 0 kswapd1 01:51:00 77 root 20 2 0 R 0 0 2 59.86 0.00 99 389:02.73 0 0 kswapd0 01:51:00 78 root 20 2 0 S 0 0 7 0.00 0.00 0 305:02.95 0 0 kswapd1 01:52:00 77 root 20 2 0 R 0 0 2 59.89 0.00 99 390:02.62 0 0 kswapd0 01:52:00 78 root 20 2 0 S 0 0 7 0.00 0.00 0 305:02.95 0 0 kswapd1 01:53:00 77 root 20 2 0 R 0 0 2 59.89 0.00 99 391:02.51 0 0 kswapd0 01:53:00 78 root 20 2 0 S 0 0 7 0.00 0.00 0 305:02.95 0 0 kswapd1 01:54:00 77 root 20 2 0 R 0 0 2 59.11 0.00 98 392:01.62 0 0 kswapd0 01:54:00 78 root 20 2 0 S 0 0 7 0.00 0.00 0 305:02.95 0 0 kswapd1 01:55:00 77 root 20 2 0 R 0 0 2 59.88 0.00 99 393:01.50 0 0 kswapd0 01:55:00 78 root 20 2 0 S 0 0 7 0.00 0.00 0 305:02.95 0 0 kswapd1 01:56:00 77 root 20 2 0 R 0 0 2 59.89 0.00 99 394:01.39 0 0 kswapd0 01:56:00 78 root 20 2 0 S 0 0 7 0.00 0.00 0 305:02.95 0 0 kswapd1 01:57:00 77 root 20 2 0 R 0 0 2 59.89 0.00 99 395:01.28 0 0 kswapd0 01:57:00 78 root 20 2 0 S 0 0 7 0.00 0.00 0 305:02.95 0 0 kswapd1 01:58:00 77 root 20 2 0 R 0 0 2 59.96 0.00 99 396:01.24 0 0 kswapd0 01:58:00 78 root 20 2 0 R 0 0 7 38.42 0.00 64 305:41.37 0 0 kswapd1 01:59:00 77 root 20 2 0 R 0 0 2 59.89 0.00 99 397:01.13 0 0 kswapd0 01:59:00 78 root 20 2 0 R 0 0 7 59.84 0.00 99 306:41.21 0 0 kswapd1 02:00:00 77 root 20 2 0 R 0 0 2 59.88 0.00 99 398:01.01 0 0 kswapd0 02:00:00 78 root 20 2 0 R 0 0 7 59.88 0.00 99 307:41.09 0 0 kswapd1 02:01:00 77 root 20 2 0 R 0 0 2 59.77 0.00 99 399:00.78 0 0 kswapd0 02:01:00 78 root 20 2 0 R 0 0 7 59.80 0.00 99 308:40.89 0 0 kswapd1 02:02:00 77 root 20 2 0 R 0 0 2 59.88 0.00 99 400:00.66 0 0 kswapd0 02:02:00 78 root 20 2 0 R 0 0 7 59.88 0.00 99 309:40.77 0 0 kswapd1 02:03:00 77 root 20 2 0 R 0 0 2 59.89 0.00 99 401:00.55 0 0 kswapd0 02:03:00 78 root 20 2 0 R 0 0 7 59.89 0.00 99 310:40.66 0 0 kswapd1 02:04:00 77 root 20 2 0 R 0 0 2 59.89 0.00 99 402:00.44 0 0 kswapd0 02:04:00 78 root 20 2 0 R 0 0 7 59.89 0.00 99 311:40.55 0 0 kswapd1 02:05:00 77 root 20 2 0 R 0 0 2 59.89 0.00 99 403:00.33 0 0 kswapd0 02:05:00 78 root 20 2 0 R 0 0 7 59.89 0.00 99 312:40.44 0 0 kswapd1 02:06:00 77 root 20 2 0 R 0 0 3 59.87 0.00 99 404:00.20 0 0 kswapd0 02:06:00 78 root 20 2 0 R 0 0 5 59.87 0.00 99 313:40.31 0 0 kswapd1 02:07:00 77 root 20 2 0 R 0 0 3 59.90 0.00 99 405:00.10 0 0 kswapd0 02:07:00 78 root 20 2 0 R 0 0 5 59.90 0.00 99 314:40.21 0 0 kswapd1 --------------090108070906030405080108--