From: Frank Steiner <fsteiner-mail@bio.ifi.lmu.de>
To: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: kblockd/1: page allocation failure in 2.6.9
Date: Tue, 21 Dec 2004 08:39:25 +0100 [thread overview]
Message-ID: <41C7D32D.2010809@bio.ifi.lmu.de> (raw)
Hi,
we got sth. that looks similar to but not exactly like a kernel oops on one
of our hosts saying "kblockd/1: page allocation failure. order:0, mode:0x21"
(the full output is below). The kernel is 2.6.9.
After that happend, "free" told me that 1.7GB of the 2GB was used and nothing
was buffered (i.e., the "-/+ buffers" shopwed the same amount of used memory),
which was strange because there was no process running that did any real
work. A tomcat server and a mysql database are running on this machine, but
the database is still empty and the tomcat does not yet offer any services.
Also, "ps -aux" showed only 2/3 of the processes then hang until I killed it.
I've never seen "ps -aux" hang! I had to reboot to get rid of this.
I guess the problem occured after copying a large amount of data to the disks.
The host is a dual xeon 1.7GHz with an internal 147GB Raid-1 (ICP vortex controlled
with gdth modul) and two external 600GB IDE-Raid5, connected to an Adaptec 29160
with the aic7xxx modul.
Can someone tell me what this kblockd messages mean? What is kblockd? Could
that point to a hardware problem (CPU, memory, mainboard) or a driver problems
with one the scsi controllers or sth.? The log says sth about scsi_get_command.
And are these messages sth. like a kernel oops or not? I've never seen sth.
like this before so I'm a bit lost :-)
Thanks for any hints!
Best regards,
Frank
Dec 20 13:17:21 turing kernel: kblockd/1: page allocation failure. order:0, mode:0x21
Dec 20 13:17:21 turing kernel: [__alloc_pages+824/832] __alloc_pages+0x338/0x340
Dec 20 13:17:21 turing kernel: [<c0142648>] __alloc_pages+0x338/0x340
Dec 20 13:17:21 turing kernel: [__get_free_pages+24/48] __get_free_pages+0x18/0x30
Dec 20 13:17:21 turing kernel: [<c0142668>] __get_free_pages+0x18/0x30
Dec 20 13:17:21 turing kernel: [kmem_getpages+36/224] kmem_getpages+0x24/0xe0
Dec 20 13:17:21 turing kernel: [<c0145884>] kmem_getpages+0x24/0xe0
Dec 20 13:17:21 turing kernel: [cache_grow+168/352] cache_grow+0xa8/0x160
Dec 20 13:17:21 turing kernel: [<c0146578>] cache_grow+0xa8/0x160
Dec 20 13:17:21 turing kernel: [cache_alloc_refill+354/560]
cache_alloc_refill+0x162/0x230
Dec 20 13:17:21 turing kernel: [<c0146792>] cache_alloc_refill+0x162/0x230
Dec 20 13:17:21 turing kernel: [kmem_cache_alloc+52/64] kmem_cache_alloc+0x34/0x40
Dec 20 13:17:22 turing kernel: [<c0146a54>] kmem_cache_alloc+0x34/0x40
Dec 20 13:17:22 turing kernel: [__scsi_get_command+21/96] __scsi_get_command+0x15/0x60
Dec 20 13:17:22 turing kernel: [<c02eba35>] __scsi_get_command+0x15/0x60
Dec 20 13:17:22 turing kernel: [scsi_get_command+15/144] scsi_get_command+0xf/0x90
Dec 20 13:17:22 turing kernel: [<c02eba8f>] scsi_get_command+0xf/0x90
Dec 20 13:17:22 turing kernel: [scsi_prep_fn+227/448] scsi_prep_fn+0xe3/0x1c0
Dec 20 13:17:22 turing kernel: [<c02f0f93>] scsi_prep_fn+0xe3/0x1c0
Dec 20 13:17:22 turing kernel: [elv_next_request+85/224] elv_next_request+0x55/0xe0
Dec 20 13:17:22 turing kernel: [<c029a795>] elv_next_request+0x55/0xe0
Dec 20 13:17:22 turing kernel: [scsi_request_fn+501/960] scsi_request_fn+0x1f5/0x3c0
Dec 20 13:17:22 turing kernel: [<c02f1265>] scsi_request_fn+0x1f5/0x3c0
Dec 20 13:17:22 turing kernel: [__generic_unplug_device+46/48]
__generic_unplug_device+0x2e/0x30
Dec 20 13:17:22 turing kernel: [<c029c29e>] __generic_unplug_device+0x2e/0x30
Dec 20 13:17:22 turing kernel: [generic_unplug_device+21/48]
generic_unplug_device+0x15/0x30
Dec 20 13:17:22 turing kernel: [<c029c2b5>] generic_unplug_device+0x15/0x30
Dec 20 13:17:22 turing kernel: [blk_unplug_work+6/16] blk_unplug_work+0x6/0x10
Dec 20 13:17:22 turing kernel: [<c029c2f6>] blk_unplug_work+0x6/0x10
Dec 20 13:17:22 turing kernel: [worker_thread+424/560] worker_thread+0x1a8/0x230
Dec 20 13:17:22 turing kernel: [<c0130048>] worker_thread+0x1a8/0x230
Dec 20 13:17:22 turing kernel: [blk_unplug_work+0/16] blk_unplug_work+0x0/0x10
Dec 20 13:17:22 turing kernel: [<c029c2f0>] blk_unplug_work+0x0/0x10
Dec 20 13:17:22 turing kernel: [default_wake_function+0/16]
default_wake_function+0x0/0x10
Dec 20 13:17:22 turing kernel: [<c011e470>] default_wake_function+0x0/0x10
Dec 20 13:17:22 turing kernel: [default_wake_function+0/16]
default_wake_function+0x0/0x10
Dec 20 13:17:22 turing kernel: [<c011e470>] default_wake_function+0x0/0x10
Dec 20 13:17:22 turing kernel: [worker_thread+0/560] worker_thread+0x0/0x230
Dec 20 13:17:22 turing kernel: [<c012fea0>] worker_thread+0x0/0x230
Dec 20 13:17:22 turing kernel: [kthread+134/176] kthread+0x86/0xb0
Dec 20 13:17:22 turing kernel: [<c0133b46>] kthread+0x86/0xb0
Dec 20 13:17:22 turing kernel: [kthread+0/176] kthread+0x0/0xb0
Dec 20 13:17:22 turing kernel: [<c0133ac0>] kthread+0x0/0xb0
Dec 20 13:17:22 turing kernel: [kernel_thread_helper+5/16] kernel_thread_helper+0x5/0x10
Dec 20 13:17:22 turing kernel: [<c0105275>] kernel_thread_helper+0x5/0x10
Dec 20 13:17:22 turing kernel: klogd: page allocation failure. order:0, mode:0x21
Dec 20 13:17:22 turing kernel: [__alloc_pages+824/832] __alloc_pages+0x338/0x340
Dec 20 13:17:22 turing kernel: [<c0142648>] __alloc_pages+0x338/0x340
Dec 20 13:17:22 turing kernel: [__get_free_pages+24/48] __get_free_pages+0x18/0x30
Dec 20 13:17:22 turing kernel: [<c0142668>] __get_free_pages+0x18/0x30
Dec 20 13:17:22 turing kernel: [kmem_getpages+36/224] kmem_getpages+0x24/0xe0
Dec 20 13:17:22 turing kernel: [<c0145884>] kmem_getpages+0x24/0xe0
Dec 20 13:17:22 turing kernel: [cache_grow+168/352] cache_grow+0xa8/0x160
Dec 20 13:17:22 turing kernel: [<c0146578>] cache_grow+0xa8/0x160
Dec 20 13:17:22 turing kernel: [cache_alloc_refill+354/560]
cache_alloc_refill+0x162/0x230
Dec 20 13:17:22 turing kernel: [<c0146792>] cache_alloc_refill+0x162/0x230
Dec 20 13:17:22 turing kernel: [kmem_cache_alloc+52/64] kmem_cache_alloc+0x34/0x40
Dec 20 13:17:22 turing kernel: [<c0146a54>] kmem_cache_alloc+0x34/0x40
Dec 20 13:17:22 turing kernel: [__scsi_get_command+21/96] __scsi_get_command+0x15/0x60
Dec 20 13:17:22 turing kernel: [<c02eba35>] __scsi_get_command+0x15/0x60
Dec 20 13:17:22 turing kernel: [scsi_get_command+15/144] scsi_get_command+0xf/0x90
Dec 20 13:17:22 turing kernel: [<c02eba8f>] scsi_get_command+0xf/0x90
Dec 20 13:17:22 turing kernel: [scsi_prep_fn+227/448] scsi_prep_fn+0xe3/0x1c0
Dec 20 13:17:22 turing kernel: [<c02f0f93>] scsi_prep_fn+0xe3/0x1c0
Dec 20 13:17:22 turing kernel: [elv_next_request+85/224] elv_next_request+0x55/0xe0
Dec 20 13:17:22 turing kernel: [<c029a795>] elv_next_request+0x55/0xe0
Dec 20 13:17:22 turing kernel: [__generic_unplug_device+34/48]
__generic_unplug_device+0x22/0x30
Dec 20 13:17:22 turing kernel: [<c029c292>] __generic_unplug_device+0x22/0x30
Dec 20 13:17:22 turing kernel: [generic_unplug_device+21/48]
generic_unplug_device+0x15/0x30
Dec 20 13:17:22 turing kernel: [<c029c2b5>] generic_unplug_device+0x15/0x30
Dec 20 13:17:22 turing kernel: [blk_backing_dev_unplug+18/32]
blk_backing_dev_unplug+0x12/0x20
Dec 20 13:17:22 turing kernel: [<c029c2e2>] blk_backing_dev_unplug+0x12/0x20
Dec 20 13:17:22 turing kernel: [swap_unplug_io_fn+90/144] swap_unplug_io_fn+0x5a/0x90
Dec 20 13:17:22 turing kernel: [<c015347a>] swap_unplug_io_fn+0x5a/0x90
Dec 20 13:17:22 turing kernel: [swap_unplug_io_fn+0/144] swap_unplug_io_fn+0x0/0x90
Dec 20 13:17:22 turing kernel: [<c0153420>] swap_unplug_io_fn+0x0/0x90
Dec 20 13:17:22 turing kernel: [block_sync_page+68/80] block_sync_page+0x44/0x50
Dec 20 13:17:22 turing kernel: [<c015f284>] block_sync_page+0x44/0x50
Dec 20 13:17:22 turing kernel: [__lock_page+235/256] __lock_page+0xeb/0x100
Dec 20 13:17:22 turing kernel: [<c013e42b>] __lock_page+0xeb/0x100
Dec 20 13:17:22 turing kernel: [page_wake_function+0/64] page_wake_function+0x0/0x40
Dec 20 13:17:22 turing kernel: [<c013e130>] page_wake_function+0x0/0x40
Dec 20 13:17:22 turing kernel: [page_wake_function+0/64] page_wake_function+0x0/0x40
Dec 20 13:17:22 turing kernel: [<c013e130>] page_wake_function+0x0/0x40
Dec 20 13:17:22 turing kernel: [grab_swap_token+158/176] grab_swap_token+0x9e/0xb0
Dec 20 13:17:22 turing kernel: [<c015584e>] grab_swap_token+0x9e/0xb0
Dec 20 13:17:22 turing kernel: [do_swap_page+327/656] do_swap_page+0x147/0x290
Dec 20 13:17:22 turing kernel: [<c014d077>] do_swap_page+0x147/0x290
Dec 20 13:17:22 turing kernel: [handle_mm_fault+244/336] handle_mm_fault+0xf4/0x150
Dec 20 13:17:22 turing kernel: [<c014d7b4>] handle_mm_fault+0xf4/0x150
Dec 20 13:17:22 turing kernel: [do_page_fault+443/1400] do_page_fault+0x1bb/0x578
Dec 20 13:17:22 turing kernel: [<c011af4b>] do_page_fault+0x1bb/0x578
Dec 20 13:17:22 turing kernel: [avc_has_perm_noaudit+188/416]
avc_has_perm_noaudit+0xbc/0x1a0
Dec 20 13:17:22 turing kernel: [<c020fb2c>] avc_has_perm_noaudit+0xbc/0x1a0
Dec 20 13:17:22 turing kernel: [recalc_task_prio+330/480] recalc_task_prio+0x14a/0x1e0
Dec 20 13:17:22 turing kernel: [<c011c5ea>] recalc_task_prio+0x14a/0x1e0
Dec 20 13:17:22 turing kernel: [finish_task_switch+50/112] finish_task_switch+0x32/0x70
Dec 20 13:17:22 turing kernel: [<c011cfa2>] finish_task_switch+0x32/0x70
Dec 20 13:17:22 turing kernel: [schedule+848/2832] schedule+0x350/0xb10
Dec 20 13:17:22 turing kernel: [<c03908b0>] schedule+0x350/0xb10
Dec 20 13:17:22 turing kernel: [do_page_fault+0/1400] do_page_fault+0x0/0x578
Dec 20 13:17:22 turing kernel: [<c011ad90>] do_page_fault+0x0/0x578
Dec 20 13:17:22 turing kernel: [error_code+45/64] error_code+0x2d/0x40
Dec 20 13:17:22 turing kernel: [<c0107f5d>] error_code+0x2d/0x40
Dec 20 13:17:22 turing kernel: [do_syslog+779/992] do_syslog+0x30b/0x3e0
Dec 20 13:17:22 turing kernel: [<c012197b>] do_syslog+0x30b/0x3e0
Dec 20 13:17:22 turing kernel: [autoremove_wake_function+0/48]
autoremove_wake_function+0x0/0x30
Dec 20 13:17:22 turing kernel: [<c011f9e0>] autoremove_wake_function+0x0/0x30
Dec 20 13:17:22 turing kernel: [autoremove_wake_function+0/48]
autoremove_wake_function+0x0/0x30
Dec 20 13:17:22 turing kernel: [<c011f9e0>] autoremove_wake_function+0x0/0x30
Dec 20 13:17:22 turing kernel: [kmsg_read+0/64] kmsg_read+0x0/0x40
Dec 20 13:17:22 turing kernel: [<c018a620>] kmsg_read+0x0/0x40
Dec 20 13:17:22 turing kernel: [vfs_read+174/240] vfs_read+0xae/0xf0
Dec 20 13:17:22 turing kernel: [<c015aa8e>] vfs_read+0xae/0xf0
Dec 20 13:17:22 turing kernel: [sys_read+60/112] sys_read+0x3c/0x70
Dec 20 13:17:22 turing kernel: [<c015acec>] sys_read+0x3c/0x70
Dec 20 13:17:22 turing kernel: [syscall_call+7/11] syscall_call+0x7/0xb
Dec 20 13:17:22 turing kernel: [<c0106e27>] syscall_call+0x7/0xb
Dec 20 13:17:22 turing kernel: klogd: page allocation failure. order:0, mode:0x21
Dec 20 13:17:22 turing kernel: [__alloc_pages+824/832] __alloc_pages+0x338/0x340
Dec 20 13:17:22 turing kernel: [<c0142648>] __alloc_pages+0x338/0x340
Dec 20 13:17:22 turing kernel: [__get_free_pages+24/48] __get_free_pages+0x18/0x30
Dec 20 13:17:22 turing kernel: [<c0142668>] __get_free_pages+0x18/0x30
Dec 20 13:17:22 turing kernel: [kmem_getpages+36/224] kmem_getpages+0x24/0xe0
Dec 20 13:17:22 turing kernel: [<c0145884>] kmem_getpages+0x24/0xe0
Dec 20 13:17:22 turing kernel: [cache_grow+168/352] cache_grow+0xa8/0x160
Dec 20 13:17:22 turing kernel: [<c0146578>] cache_grow+0xa8/0x160
Dec 20 13:17:22 turing kernel: [cache_alloc_refill+354/560]
cache_alloc_refill+0x162/0x230
Dec 20 13:17:22 turing kernel: [<c0146792>] cache_alloc_refill+0x162/0x230
Dec 20 13:17:22 turing kernel: [kmem_cache_alloc+52/64] kmem_cache_alloc+0x34/0x40
Dec 20 13:17:22 turing kernel: [<c0146a54>] kmem_cache_alloc+0x34/0x40
Dec 20 13:17:22 turing kernel: [__scsi_get_command+21/96] __scsi_get_command+0x15/0x60
Dec 20 13:17:22 turing kernel: [<c02eba35>] __scsi_get_command+0x15/0x60
Dec 20 13:17:22 turing kernel: [scsi_get_command+15/144] scsi_get_command+0xf/0x90
Dec 20 13:17:22 turing kernel: [<c02eba8f>] scsi_get_command+0xf/0x90
Dec 20 13:17:22 turing kernel: [scsi_prep_fn+227/448] scsi_prep_fn+0xe3/0x1c0
Dec 20 13:17:22 turing kernel: [<c02f0f93>] scsi_prep_fn+0xe3/0x1c0
Dec 20 13:17:22 turing kernel: [elv_next_request+85/224] elv_next_request+0x55/0xe0
Dec 20 13:17:22 turing kernel: [<c029a795>] elv_next_request+0x55/0xe0
Dec 20 13:17:22 turing kernel: [__generic_unplug_device+34/48]
__generic_unplug_device+0x22/0x30
Dec 20 13:17:22 turing kernel: [<c029c292>] __generic_unplug_device+0x22/0x30
Dec 20 13:17:22 turing kernel: [generic_unplug_device+21/48]
generic_unplug_device+0x15/0x30
Dec 20 13:17:22 turing kernel: [<c029c2b5>] generic_unplug_device+0x15/0x30
Dec 20 13:17:22 turing kernel: [blk_backing_dev_unplug+0/32]
blk_backing_dev_unplug+0x0/0x20
Dec 20 13:17:22 turing kernel: [<c029c2d0>] blk_backing_dev_unplug+0x0/0x20
Dec 20 13:17:22 turing kernel: [blk_backing_dev_unplug+18/32]
blk_backing_dev_unplug+0x12/0x20
Dec 20 13:17:22 turing kernel: [<c029c2e2>] blk_backing_dev_unplug+0x12/0x20
Dec 20 13:17:22 turing kernel: [block_sync_page+68/80] block_sync_page+0x44/0x50
Dec 20 13:17:22 turing kernel: [<c015f284>] block_sync_page+0x44/0x50
Dec 20 13:17:22 turing kernel: [__lock_page+235/256] __lock_page+0xeb/0x100
Dec 20 13:17:22 turing kernel: [<c013e42b>] __lock_page+0xeb/0x100
Dec 20 13:17:22 turing kernel: [page_wake_function+0/64] page_wake_function+0x0/0x40
Dec 20 13:17:22 turing kernel: [<c013e130>] page_wake_function+0x0/0x40
Dec 20 13:17:22 turing kernel: [page_wake_function+0/64] page_wake_function+0x0/0x40
Dec 20 13:17:22 turing kernel: [<c013e130>] page_wake_function+0x0/0x40
Dec 20 13:17:22 turing kernel: [find_get_page+57/80] find_get_page+0x39/0x50
Dec 20 13:17:22 turing kernel: [<c013e479>] find_get_page+0x39/0x50
Dec 20 13:17:22 turing kernel: [filemap_nopage+697/864] filemap_nopage+0x2b9/0x360
Dec 20 13:17:22 turing kernel: [<c013f489>] filemap_nopage+0x2b9/0x360
Dec 20 13:17:22 turing kernel: [do_no_page+180/688] do_no_page+0xb4/0x2b0
Dec 20 13:17:22 turing kernel: [<c014d3f4>] do_no_page+0xb4/0x2b0
Dec 20 13:17:22 turing kernel: [handle_mm_fault+272/336] handle_mm_fault+0x110/0x150
Dec 20 13:17:22 turing kernel: [<c014d7d0>] handle_mm_fault+0x110/0x150
Dec 20 13:17:22 turing kernel: [do_page_fault+443/1400] do_page_fault+0x1bb/0x578
Dec 20 13:17:22 turing kernel: [<c011af4b>] do_page_fault+0x1bb/0x578
Dec 20 13:17:22 turing kernel: [do_syslog+722/992] do_syslog+0x2d2/0x3e0
Dec 20 13:17:22 turing kernel: [<c0121942>] do_syslog+0x2d2/0x3e0
Dec 20 13:17:22 turing kernel: [autoremove_wake_function+0/48]
autoremove_wake_function+0x0/0x30
Dec 20 13:17:22 turing kernel: [<c011f9e0>] autoremove_wake_function+0x0/0x30
Dec 20 13:17:22 turing kernel: [autoremove_wake_function+0/48]
autoremove_wake_function+0x0/0x30
Dec 20 13:17:22 turing kernel: [<c011f9e0>] autoremove_wake_function+0x0/0x30
Dec 20 13:17:22 turing kernel: [kmsg_read+0/64] kmsg_read+0x0/0x40
Dec 20 13:17:22 turing kernel: [<c018a620>] kmsg_read+0x0/0x40
Dec 20 13:17:22 turing kernel: [dnotify_parent+31/112] dnotify_parent+0x1f/0x70
Dec 20 13:17:22 turing kernel: [<c017463f>] dnotify_parent+0x1f/0x70
Dec 20 13:17:22 turing kernel: [vfs_read+194/240] vfs_read+0xc2/0xf0
Dec 20 13:17:22 turing kernel: [<c015aaa2>] vfs_read+0xc2/0xf0
Dec 20 13:17:22 turing kernel: [sys_read+60/112] sys_read+0x3c/0x70
Dec 20 13:17:22 turing kernel: [<c015acec>] sys_read+0x3c/0x70
Dec 20 13:17:22 turing kernel: [do_page_fault+0/1400] do_page_fault+0x0/0x578
Dec 20 13:17:22 turing kernel: [<c011ad90>] do_page_fault+0x0/0x578
Dec 20 13:17:22 turing kernel: [error_code+45/64] error_code+0x2d/0x40
Dec 20 13:17:22 turing kernel: [<c0107f5d>] error_code+0x2d/0x40
--
Dipl.-Inform. Frank Steiner Web: http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17 Phone: +49 89 2180-4049
80333 Muenchen, Germany Fax: +49 89 2180-99-4049
* Rekursion kann man erst verstehen, wenn man Rekursion verstanden hat. *
next reply other threads:[~2004-12-21 7:39 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2004-12-21 7:39 Frank Steiner [this message]
2004-12-23 11:26 ` kblockd/1: page allocation failure in 2.6.9 Frank Steiner
2004-12-23 15:51 ` Denis Vlasenko
2004-12-23 15:55 ` Frank Steiner
2004-12-24 13:20 ` Jens Axboe
2004-12-24 19:28 ` James Bottomley
2004-12-25 22:49 ` Frank Steiner
2004-12-26 15:46 ` James Bottomley
2004-12-26 22:31 ` Frank Steiner
2005-01-10 9:19 ` Frank Steiner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=41C7D32D.2010809@bio.ifi.lmu.de \
--to=fsteiner-mail@bio.ifi.lmu.de \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.