public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* 3.6.11: khubd issue SLAB related?
@ 2013-03-08 13:11 Udo van den Heuvel
  2013-03-08 13:15 ` Udo van den Heuvel
  2013-03-11 21:23 ` David Rientjes
  0 siblings, 2 replies; 5+ messages in thread
From: Udo van den Heuvel @ 2013-03-08 13:11 UTC (permalink / raw)
  To: linux-kernel

Hello,

I found this in dmesg:

[3663200.373989] pwc: Philips PCVC740K (ToUCam Pro)/PCVC840 (ToUCam II)
USB webcam detected.
[3663200.779617] khubd: page allocation failure: order:4, mode:0xd0
[3663200.779621] Pid: 274, comm: khubd Not tainted 3.6.11 #22
[3663200.779622] Call Trace:
[3663200.779629]  [<ffffffff810b5346>] ? warn_alloc_failed+0xf6/0x150
[3663200.779632]  [<ffffffff8141c6de>] ?
__alloc_pages_direct_compact+0x1ac/0x1be
[3663200.779635]  [<ffffffff810b7f89>] ? __alloc_pages_nodemask+0x609/0x820
[3663200.779638]  [<ffffffff810e8796>] ? cache_alloc_refill+0x356/0x660
[3663200.779640]  [<ffffffff810e8c7b>] ? kmem_cache_alloc+0xab/0xe0
[3663200.779643]  [<ffffffffa00168da>] ? usb_pwc_probe+0x17a/0xaf0 [pwc]
[3663200.779646]  [<ffffffff812de513>] ? usb_probe_interface+0x113/0x200
[3663200.779650]  [<ffffffff8128be2e>] ? driver_probe_device+0x6e/0x210
[3663200.779652]  [<ffffffff8128c070>] ? __driver_attach+0xa0/0xa0
[3663200.779654]  [<ffffffff8128a2de>] ? bus_for_each_drv+0x3e/0x80
[3663200.779656]  [<ffffffff8128bd90>] ? device_attach+0x80/0x90
[3663200.779658]  [<ffffffff8128b370>] ? bus_probe_device+0x80/0xb0
[3663200.779659]  [<ffffffff812899d7>] ? device_add+0x5b7/0x670
[3663200.779662]  [<ffffffff812dd3a8>] ? usb_set_configuration+0x4b8/0x720
[3663200.779664]  [<ffffffff812e5a3b>] ? generic_probe+0x3b/0xa0
[3663200.779666]  [<ffffffff8128be2e>] ? driver_probe_device+0x6e/0x210
[3663200.779668]  [<ffffffff8128c070>] ? __driver_attach+0xa0/0xa0
[3663200.779670]  [<ffffffff8128a2de>] ? bus_for_each_drv+0x3e/0x80
[3663200.779672]  [<ffffffff8128bd90>] ? device_attach+0x80/0x90
[3663200.779674]  [<ffffffff8128b370>] ? bus_probe_device+0x80/0xb0
[3663200.779675]  [<ffffffff812899d7>] ? device_add+0x5b7/0x670
[3663200.779678]  [<ffffffff81282d7f>] ?
mix_pool_bytes.constprop.15+0x3f/0x60
[3663200.779681]  [<ffffffff81421300>] ? _raw_spin_unlock_bh+0x20/0x20
[3663200.779683]  [<ffffffff812d5d08>] ? usb_new_device+0x1d8/0x260
[3663200.779685]  [<ffffffff812d640a>] ? hub_thread+0x67a/0x14d0
[3663200.779687]  [<ffffffff8142021c>] ? __schedule+0x2ac/0x800
[3663200.779690]  [<ffffffff810611ec>] ? __wake_up_common+0x4c/0x80
[3663200.779693]  [<ffffffff81059b00>] ? abort_exclusive_wait+0xb0/0xb0
[3663200.779695]  [<ffffffff812d5d90>] ? usb_new_device+0x260/0x260
[3663200.779697]  [<ffffffff810591d5>] ? kthread+0x85/0x90
[3663200.779699]  [<ffffffff814231f4>] ? kernel_thread_helper+0x4/0x10
[3663200.779701]  [<ffffffff81059150>] ?
kthread_freezable_should_stop+0x50/0x50
[3663200.779703]  [<ffffffff814231f0>] ? gs_change+0xb/0xb
[3663200.779704] Mem-Info:
[3663200.779705] DMA per-cpu:
[3663200.779706] CPU    0: hi:    0, btch:   1 usd:   0
[3663200.779708] CPU    1: hi:    0, btch:   1 usd:   0
[3663200.779709] CPU    2: hi:    0, btch:   1 usd:   0
[3663200.779710] CPU    3: hi:    0, btch:   1 usd:   0
[3663200.779711] DMA32 per-cpu:
[3663200.779712] CPU    0: hi:  186, btch:  31 usd:   0
[3663200.779713] CPU    1: hi:  186, btch:  31 usd:   0
[3663200.779714] CPU    2: hi:  186, btch:  31 usd:   0
[3663200.779715] CPU    3: hi:  186, btch:  31 usd:   0
[3663200.779716] Normal per-cpu:
[3663200.779717] CPU    0: hi:  186, btch:  31 usd:   0
[3663200.779718] CPU    1: hi:  186, btch:  31 usd:   0
[3663200.779719] CPU    2: hi:  186, btch:  31 usd:   0
[3663200.779720] CPU    3: hi:  186, btch:  31 usd:   0
[3663200.779723] active_anon:367320 inactive_anon:142633 isolated_anon:0
[3663200.779723]  active_file:470023 inactive_file:522338 isolated_file:0
[3663200.779723]  unevictable:0 dirty:89 writeback:0 unstable:0
[3663200.779723]  free:87496 slab_reclaimable:79192 slab_unreclaimable:19937
[3663200.779723]  mapped:18116 shmem:3960 pagetables:11525 bounce:0
[3663200.779728] DMA free:15904kB min:136kB low:168kB high:204kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15680kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0
all_unreclaimable? yes
[3663200.779730] lowmem_reserve[]: 0 2448 7483 7483
[3663200.779736] DMA32 free:169684kB min:22072kB low:27588kB
high:33108kB active_anon:196000kB inactive_anon:221500kB
active_file:709152kB inactive_file:793704kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:2507728kB mlocked:0kB
dirty:12kB writeback:0kB mapped:19536kB shmem:3072kB
slab_reclaimable:161628kB slab_unreclaimable:15436kB kernel_stack:488kB
pagetables:4780kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? no
[3663200.779737] lowmem_reserve[]: 0 0 5034 5034
[3663200.779742] Normal free:164396kB min:45372kB low:56712kB
high:68056kB active_anon:1273280kB inactive_anon:349032kB
active_file:1170940kB inactive_file:1295648kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:5155036kB mlocked:0kB
dirty:344kB writeback:0kB mapped:52928kB shmem:12768kB
slab_reclaimable:155140kB slab_unreclaimable:64312kB kernel_stack:3312kB
pagetables:41320kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? no
[3663200.779744] lowmem_reserve[]: 0 0 0 0
[3663200.779746] DMA: 0*4kB 0*8kB 0*16kB 1*32kB 2*64kB 1*128kB 1*256kB
0*512kB 1*1024kB 1*2048kB 3*4096kB = 15904kB
[3663200.779751] DMA32: 23335*4kB 8814*8kB 365*16kB 0*32kB 0*64kB
0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 169692kB
[3663200.779757] Normal: 38925*4kB 1081*8kB 3*16kB 0*32kB 0*64kB 0*128kB
0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 164396kB
[3663200.779762] 1007244 total pagecache pages
[3663200.779763] 10889 pages in swap cache
[3663200.779764] Swap cache stats: add 125307, delete 114418, find
564015/570538
[3663200.779765] Free swap  = 3908344kB
[3663200.779766] Total swap = 4095984kB
[3663200.799573] 1961968 pages RAM
[3663200.799575] 75067 pages reserved
[3663200.799576] 268951 pages shared
[3663200.799577] 1601656 pages non-shared
[3663200.799579] SLAB: Unable to allocate memory on node 0 (gfp=0xd0)
[3663200.799581]   cache: size-65536, object size: 65536, order: 4
[3663200.799583]   node 0: slabs: 6/6, objs: 6/6, free: 0
[3663200.799585] pwc: Oops, could not allocate memory for pwc_device.
[3663200.799593] Philips webcam: probe of 2-4:1.0 failed with error -12
[3663200.799619] usb 2-4: adding 2-4:1.1 (config #1, interface 1)
[3663200.799807] snd-usb-audio 2-4:1.1: usb_probe_interface
[3663200.799812] snd-usb-audio 2-4:1.1: usb_probe_interface - got id
[3663200.867541] usb 2-4: adding 2-4:1.2 (config #1, interface 2)
[3663200.867614] hub 2-0:1.0: state 7 ports 5 chg 0000 evt 0010


How can we fix this?

Udo

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: 3.6.11: khubd issue SLAB related?
  2013-03-08 13:11 3.6.11: khubd issue SLAB related? Udo van den Heuvel
@ 2013-03-08 13:15 ` Udo van den Heuvel
  2013-03-11 21:23 ` David Rientjes
  1 sibling, 0 replies; 5+ messages in thread
From: Udo van den Heuvel @ 2013-03-08 13:15 UTC (permalink / raw)
  Cc: linux-kernel

On 2013-03-08 14:11, Udo van den Heuvel wrote:
> Hello,
> 
> I found this in dmesg:

slabtop:

 Active / Total Objects (% used)    : 784982 / 981439 (80.0%)
 Active / Total Slabs (% used)      : 62996 / 63077 (99.9%)
 Active / Total Caches (% used)     : 119 / 196 (60.7%)
 Active / Total Size (% used)       : 205966.20K / 236692.16K (87.0%)
 Minimum / Average / Maximum Object : 0.02K / 0.24K / 4096.00K

  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME

217946 211509  97%    0.06K   3694	 59     14776K size-64
158508  65908  41%    0.10K   4284	 37     17136K buffer_head
121380 107155  88%    0.12K   4046	 30     16184K size-128
110692 108707  98%    0.86K  27673        4    110692K ext4_inode_cache
 98980  85163  86%    0.19K   4949	 20     19796K dentry
 37927  32219  84%    0.16K   1649	 23	 6596K vm_area_struct
 36001  24087  66%    0.55K   5143        7     20572K radix_tree_node
 25179  19801  78%    0.05K    327	 77	 1308K anon_vma_chain
 22561   7716  34%    0.05K    293	 77	 1172K jbd2_inode
 17696  12708  71%    0.03K    158	112	  632K size-32
 17260  11348  65%    0.19K    863	 20	 3452K size-192
 13416   8703  64%    0.29K   1032	 13	 4128K nf_conntrack_ffffffff815d8840
 12274  12046  98%    0.11K    361	 34	 1444K sysfs_dir_cache
 11977  10117  84%    0.06K    203	 59	  812K anon_vma
 11070   9712  87%    0.25K    738	 15	 2952K filp
 10638  10457  98%    0.60K   1773        6	 7092K proc_inode_cache
  9156   9136  99%    0.13K    327	 28	 1308K ext4_groupinfo_4k
  3762   3161  84%    0.60K    627        6	 2508K shmem_inode_cache
  3744   3553  94%    0.02K     26	144	  104K dm_target_io
  3680   3553  96%    0.04K     40	 92	  160K dm_io
  3577   2991  83%    0.54K    511        7	 2044K inode_cache
  3524   3103  88%    1.00K    881        4	 3524K size-1024
  2756   2586  93%    0.07K     52	 53	  208K Acpi-Operand
  2044   1929  94%    0.13K     73	 28	  292K inotify_inode_mark
  1600   1439  89%    0.50K    200        8	  800K size-512
  1380    552  40%    0.19K     69	 20	  276K cred_jar
  1092    910  83%    0.62K    182        6	  728K sock_inode_cache
  1029    979  95%    0.53K    147        7	  588K idr_layer_cache
  1020    537  52%    0.19K     51	 20	  204K bio-0
  1012    953  94%    0.04K     11	 92        44K Acpi-Namespace
   860    839  97%    0.19K     43	 20	  172K ip_dst_cache
   810    605  74%    0.12K     27	 30	  108K pid
   808    303  37%    0.02K	 4	202        16K ext4_io_page
   788    775  98%    2.00K    394        2	 1576K size-2048
   782    585  74%    0.11K     23	 34        92K task_delay_info
   765    474  61%    0.25K     51	 15	  204K skbuff_head_cache
   742    209  28%    0.07K     14	 53        56K eventpoll_pwq
   693    640  92%    0.81K     77        9	  616K UNIX
   605    591  97%    1.56K    121        5	  968K task_struct
   560    119  21%    0.03K	 5	112        20K tcp_bind_bucket
   500    209  41%    0.19K     25	 20	  100K eventpoll_epi
   486    418  86%    0.81K     54        9	  432K task_xstate
   480    378  78%    0.25K     32	 15	  128K size-256
   480    241  50%    0.19K     24	 20        96K inet_peer_cache
   408    189  46%    0.11K     12	 34        48K jbd2_journal_head
   400    299  74%    0.15K     16	 25        64K dm_crypt_io
   369    308  83%    4.00K    369        1	 1476K biovec-256
   368    304  82%    1.00K     92        4	  368K signal_cache
   360    220  61%    0.09K	 9	 40        36K blkdev_ioc
   354    354 100%    4.00K    354        1	 1416K size-4096
   354    149  42%    0.06K	 6	 59        24K fs_cache
   350    230  65%    0.50K     50        7	  200K skbuff_fclone_cache
   303    296  97%    2.06K    101        3	  808K sighand_cache
   289    284  98%   16.00K    289        1	 4624K size-16384
   285    228  80%    0.25K     19	 15        76K mnt_cache
   276     50  18%    0.04K	 3	 92        12K khugepaged_mm_slot
   270    224  82%    0.12K	 9	 30        36K scsi_sense_cache
   255    183  71%    0.25K     17	 15        68K sgpool-8


99.9% used?
Please help!


Udo

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: 3.6.11: khubd issue SLAB related?
  2013-03-08 13:11 3.6.11: khubd issue SLAB related? Udo van den Heuvel
  2013-03-08 13:15 ` Udo van den Heuvel
@ 2013-03-11 21:23 ` David Rientjes
  2013-03-13 16:23   ` Udo van den Heuvel
  1 sibling, 1 reply; 5+ messages in thread
From: David Rientjes @ 2013-03-11 21:23 UTC (permalink / raw)
  To: Udo van den Heuvel; +Cc: Hans de Goede, linux-kernel

On Fri, 8 Mar 2013, Udo van den Heuvel wrote:

> [3663200.799579] SLAB: Unable to allocate memory on node 0 (gfp=0xd0)
> [3663200.799581]   cache: size-65536, object size: 65536, order: 4
> [3663200.799583]   node 0: slabs: 6/6, objs: 6/6, free: 0
> [3663200.799585] pwc: Oops, could not allocate memory for pwc_device.

You have memory freed, but like your earlier page allocation failure on 
mount, it is too fragmented.  This is an order-4 page allocation, the 
kmalloc() of struct pwc_device, and you don't even have any order-3 pages 
that are available.  Are you running with CONFIG_COMPACTION enabled?

If not, enable it and try again, or get root and manually try to 
defragment memory by doing echo 1 > /proc/sys/vm/compact_memory.

If so, try killing a memory hogging process to free some memory and 
attempt to get some higher order pages available.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: 3.6.11: khubd issue SLAB related?
  2013-03-11 21:23 ` David Rientjes
@ 2013-03-13 16:23   ` Udo van den Heuvel
  2013-03-13 23:36     ` David Rientjes
  0 siblings, 1 reply; 5+ messages in thread
From: Udo van den Heuvel @ 2013-03-13 16:23 UTC (permalink / raw)
  To: David Rientjes; +Cc: Hans de Goede, linux-kernel

On 2013-03-11 22:23, David Rientjes wrote:
> On Fri, 8 Mar 2013, Udo van den Heuvel wrote:
> 
>> [3663200.799579] SLAB: Unable to allocate memory on node 0 (gfp=0xd0)
>> [3663200.799581]   cache: size-65536, object size: 65536, order: 4
>> [3663200.799583]   node 0: slabs: 6/6, objs: 6/6, free: 0
>> [3663200.799585] pwc: Oops, could not allocate memory for pwc_device.
> 
> You have memory freed, but like your earlier page allocation failure on 
> mount, it is too fragmented.  This is an order-4 page allocation, the 
> kmalloc() of struct pwc_device, and you don't even have any order-3 pages 
> that are available.  Are you running with CONFIG_COMPACTION enabled?

Yes.

> If not, enable it and try again, or get root and manually try to 
> defragment memory by doing echo 1 > /proc/sys/vm/compact_memory.

This needs to be done regularly, I assume?

> If so, try killing a memory hogging process to free some memory and 
> attempt to get some higher order pages available.

It's a 8GB box (7.5GB available to software) with modest load. Mostly
email & web browsing, some torrents, nntp stuff, etc. Nothing weird.
Around 4.8G in buffers/cache right now.

Why is this issue occurring?
Why did I not see this in the past?

Udo

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: 3.6.11: khubd issue SLAB related?
  2013-03-13 16:23   ` Udo van den Heuvel
@ 2013-03-13 23:36     ` David Rientjes
  0 siblings, 0 replies; 5+ messages in thread
From: David Rientjes @ 2013-03-13 23:36 UTC (permalink / raw)
  To: Udo van den Heuvel; +Cc: Hans de Goede, linux-kernel

On Wed, 13 Mar 2013, Udo van den Heuvel wrote:

> > If not, enable it and try again, or get root and manually try to 
> > defragment memory by doing echo 1 > /proc/sys/vm/compact_memory.
> 
> This needs to be done regularly, I assume?
> 

It's done when the page allocation initially fails, so it shouldn't need 
to be done explicitly by you.  I was hoping to see if that helped to solve 
the problem and something like deferred compaction wasn't involved.

> > If so, try killing a memory hogging process to free some memory and 
> > attempt to get some higher order pages available.
> 
> It's a 8GB box (7.5GB available to software) with modest load. Mostly
> email & web browsing, some torrents, nntp stuff, etc. Nothing weird.
> Around 4.8G in buffers/cache right now.
> 

That may be true, but you only have about 300MB available at the time of 
failure.

> Why is this issue occurring?
> Why did I not see this in the past?
> 

I already added Hans to the cc to see if the struct has grown recently.

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2013-03-13 23:36 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-03-08 13:11 3.6.11: khubd issue SLAB related? Udo van den Heuvel
2013-03-08 13:15 ` Udo van den Heuvel
2013-03-11 21:23 ` David Rientjes
2013-03-13 16:23   ` Udo van den Heuvel
2013-03-13 23:36     ` David Rientjes

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox