* xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0
@ 2013-11-19 13:02 Sander Eikelenboom
2013-11-19 13:23 ` Wei Liu
2013-11-19 13:41 ` David Vrabel
0 siblings, 2 replies; 6+ messages in thread
From: Sander Eikelenboom @ 2013-11-19 13:02 UTC (permalink / raw)
To: Wei Liu, Ian Campbell; +Cc: xen-devel
Hi Wei,
I ran into the following problem when trying to boot another guest after less than a day of uptime.
(the system started 15 guests at boot already which went fine). dom0 is allocated a fixed 1536M.
Both host as pv guests run the same kernel, some hvm's run a slightly older kernel (3.9 f.e.)
The are quite some granttable messages in xl dmesg, i also included these and a "vmstat -m"
--
Sander
[54807.299791] xenwatch: page allocation failure: order:4, mode:0x10c0d0
[54807.317520] CPU: 5 PID: 54 Comm: xenwatch Not tainted 3.12.0-20131104 #1
[54807.334747] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640) , BIOS V1.8B1 09/13/2010
[54807.351990] 0000000000000000 ffff8800591a3828 ffffffff81a6a8ba 0000000000000006
[54807.368857] 000000000010c0d0 ffff8800591a38b8 ffffffff81138770 0000000000000004
[54807.385539] 000000000010c0d0 ffff8800591a38b8 ffffffff81a661de ffff88005fd1c680
[54807.401978] Call Trace:
[54807.418015] [<ffffffff81a6a8ba>] dump_stack+0x4f/0x84
[54807.433971] [<ffffffff81138770>] warn_alloc_failed+0xf0/0x140
[54807.449644] [<ffffffff81a661de>] ? __alloc_pages_direct_compact+0x1ac/0x1be
[54807.465164] [<ffffffff8113bfaa>] __alloc_pages_nodemask+0x7aa/0x9d0
[54807.480510] [<ffffffff810ed069>] ? trace_hardirqs_off_caller+0xb9/0x160
[54807.495622] [<ffffffff81175277>] alloc_pages_current+0xb7/0x180
[54807.510530] [<ffffffff81138059>] __get_free_pages+0x9/0x40
[54807.525185] [<ffffffff8117cbdc>] __kmalloc+0x19c/0x1c0
[54807.539538] [<ffffffff8190e9b4>] alloc_netdev_mqs+0x64/0x340
[54807.553814] [<ffffffff8192ac20>] ? alloc_etherdev_mqs+0x20/0x20
[54807.567777] [<ffffffff816dc3e4>] xenvif_alloc+0x64/0x2c0
[54807.581473] [<ffffffff816dbc57>] netback_probe+0x287/0x2d0
[54807.594971] [<ffffffff814bfe46>] xenbus_dev_probe+0x66/0x110
[54807.608231] [<ffffffff81615105>] driver_probe_device+0x75/0x210
[54807.621227] [<ffffffff81615350>] ? __driver_attach+0xb0/0xb0
[54807.634071] [<ffffffff8161539b>] __device_attach+0x4b/0x60
[54807.646626] [<ffffffff8161316e>] bus_for_each_drv+0x4e/0xa0
[54807.658918] [<ffffffff81615058>] device_attach+0x98/0xb0
[54807.671253] [<ffffffff816144b0>] bus_probe_device+0xb0/0xe0
[54807.683379] [<ffffffff81612277>] device_add+0x3b7/0x700
[54807.695145] [<ffffffff8161ca1d>] ? device_pm_sleep_init+0x4d/0x80
[54807.706824] [<ffffffff816125d9>] device_register+0x19/0x20
[54807.718145] [<ffffffff814bf9b1>] xenbus_probe_node+0x141/0x170
[54807.729256] [<ffffffff81613236>] ? bus_for_each_dev+0x76/0xa0
[54807.740091] [<ffffffff814bfbb0>] xenbus_dev_changed+0x1d0/0x1e0
[54807.750811] [<ffffffff814bff26>] backend_changed+0x16/0x20
[54807.761256] [<ffffffff814bdf3e>] xenwatch_thread+0x4e/0x140
[54807.771371] [<ffffffff810bc1e0>] ? __init_waitqueue_head+0x60/0x60
[54807.781443] [<ffffffff814bdef0>] ? xs_watch+0x60/0x60
[54807.791310] [<ffffffff810bb716>] kthread+0xd6/0xe0
[54807.800794] [<ffffffff81a752bb>] ? _raw_spin_unlock_irq+0x2b/0x70
[54807.810150] [<ffffffff810bb640>] ? __init_kthread_worker+0x70/0x70
[54807.819526] [<ffffffff81a762cc>] ret_from_fork+0x7c/0xb0
[54807.828638] [<ffffffff810bb640>] ? __init_kthread_worker+0x70/0x70
[54807.837482] Mem-Info:
[54807.846011] Node 0 DMA per-cpu:
[54807.854277] CPU 0: hi: 0, btch: 1 usd: 0
[54807.862404] CPU 1: hi: 0, btch: 1 usd: 0
[54807.870241] CPU 2: hi: 0, btch: 1 usd: 0
[54807.877985] CPU 3: hi: 0, btch: 1 usd: 0
[54807.885344] CPU 4: hi: 0, btch: 1 usd: 0
[54807.892389] CPU 5: hi: 0, btch: 1 usd: 0
[54807.899102] Node 0 DMA32 per-cpu:
[54807.905665] CPU 0: hi: 186, btch: 31 usd: 68
[54807.911985] CPU 1: hi: 186, btch: 31 usd: 6
[54807.917964] CPU 2: hi: 186, btch: 31 usd: 149
[54807.923666] CPU 3: hi: 186, btch: 31 usd: 82
[54807.929031] CPU 4: hi: 186, btch: 31 usd: 169
[54807.934293] CPU 5: hi: 186, btch: 31 usd: 0
[54807.939207] active_anon:12850 inactive_anon:8944 isolated_anon:0
[54807.939207] active_file:70321 inactive_file:177850 isolated_file:0
[54807.939207] unevictable:562 dirty:34 writeback:0 unstable:0
[54807.939207] free:31143 slab_reclaimable:21805 slab_unreclaimable:12717
[54807.939207] mapped:3344 shmem:276 pagetables:1211 bounce:0
[54807.939207] free_cma:0
[54807.966465] Node 0 DMA free:5644kB min:52kB low:64kB high:76kB active_anon:0kB inactive_anon:4kB active_file:4kB inactive_file:0kB unevictable:24kB isolated(anon):0kB isolated(file):0kB present:15968kB managed:15884kB mlocked:24kB dirty:0kB writeback:0kB mapped:24kB shmem:0kB slab_reclaimable:8992kB slab_unreclaimable:656kB kernel_stack:16kB pagetables:56kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[54807.981676] lowmem_reserve[]: 0 1395 1395 1395
[54807.987015] Node 0 DMA32 free:118928kB min:4748kB low:5932kB high:7120kB active_anon:51028kB inactive_anon:35772kB active_file:281280kB inactive_file:711400kB unevictable:2224kB isolated(anon):0kB isolated(file):0kB present:1556480kB managed:1433216kB mlocked:2224kB dirty:136kB writeback:0kB mapped:13168kB shmem:1104kB slab_reclaimable:78228kB slab_unreclaimable:50212kB kernel_stack:3256kB pagetables:4788kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[54808.011107] lowmem_reserve[]: 0 0 0 0
[54808.017244] Node 0 DMA: 19*4kB (UEM) 18*8kB (UEM) 15*16kB (UEM) 14*32kB (UEM) 8*64kB (UEM) 7*128kB (UEM) 3*256kB (U) 1*512kB (U) 0*1024kB 1*2048kB (R) 0*4096kB = 5644kB
[54808.030315] Node 0 DMA32: 11490*4kB (UEM) 6743*8kB (UEM) 1133*16kB (EMR) 6*32kB (MR) 0*64kB 1*128kB (R) 1*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 118608kB
[54808.044303] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[54808.051507] 249203 total pagecache pages
[54808.058647] 201 pages in swap cache
[54808.065986] Swap cache stats: add 2902, delete 2701, find 62055/62191
[54808.073216] Free swap = 2092752kB
[54808.080397] Total swap = 2097148kB
[54808.093896] 393215 pages RAM
[54808.101004] 30940 pages reserved
[54808.108168] 339108 pages shared
[54808.115202] 258777 pages non-shared
[54808.122235] xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0
[54808.129391] vif vif-16-0: 12 creating interface
~# vmstat -m
Cache Num Total Size Pages
ext4_groupinfo_4k 24426 24426 216 18
ceph_osd_request 0 0 960 17
xt_hashlimit 0 0 136 30
nf_conntrack_ffffffff822d2900 1950 2100 400 20
nf_conntrack_expect 0 0 288 28
dm_snap_pending_exception 2496 2496 104 39
kcopyd_job 91 133 4176 7
dm_rq_target_io 0 0 416 19
search 0 0 616 26
blkif_cache 108 108 864 18
cfq_io_cq 204 204 120 34
cfq_queue 170 170 232 17
bsg_cmd 0 0 312 26
ceph_cap 0 0 128 32
ceph_inode_info 0 0 2176 15
gfs2_mblk 0 0 432 18
gfs2_bufdata 0 0 88 46
gfs2_inode 0 0 1280 25
gfs2_glock(aspace) 0 0 960 17
gfs2_glock 0 0 592 27
btrfs_delayed_data_ref 0 0 96 42
btrfs_delayed_ref_head 0 0 232 17
btrfs_delayed_node 0 0 392 20
btrfs_ordered_extent 0 0 472 17
btrfs_extent_buffer 0 0 568 28
btrfs_delalloc_work 0 0 184 22
btrfs_path 0 0 144 28
btrfs_transaction 0 0 600 27
btrfs_trans_handle 0 0 160 25
btrfs_inode 0 0 1944 16
fuse_request 102 102 480 17
fuse_inode 336 476 1152 28
ntfs_big_inode_cache 0 0 1664 19
ntfs_inode_cache 0 0 696 23
cifs_small_rq 36 36 448 18
cifs_request 4 4 16512 1
cifs_inode_cache 0 0 1112 29
isofs_inode_cache 0 0 960 17
fat_inode_cache 0 0 1168 28
fat_cache 0 0 40 102
hugetlbfs_inode_cache 16 16 976 16
jbd2_transaction_s 150 150 320 25
jbd2_journal_handle 306 306 80 51
journal_handle 0 0 56 73
journal_head 1518 1800 112 36
revoke_table 1536 1536 16 256
revoke_record 768 768 32 128
ext4_inode_cache 30702 30952 1704 19
ext4_free_data 768 768 64 64
ext4_allocation_context 180 180 136 30
ext4_prealloc_space 156 156 152 26
ext4_io_end 336 336 72 56
ext4_extent_status 7361 7446 40 102
ext3_inode_cache 0 0 1312 24
ext3_xattr 3726 3956 88 46
dquot 0 0 384 21
kioctx 0 0 896 18
pid_namespace 0 0 2208 14
posix_timers_cache 108 108 296 27
UNIX 198 198 1472 22
Cache Num Total Size Pages
UDP-Lite 0 0 1280 25
ip_fib_trie 438 438 56 73
PING 0 0 1216 26
UDP 150 150 1280 25
tw_sock_TCP 399 399 192 21
TCP 104 104 2368 13
fscache_cookie_jar 0 0 192 21
sgpool-128 54 78 5120 6
sgpool-64 72 72 2560 12
sgpool-32 325 325 1280 25
sgpool-16 277 350 640 25
blkdev_integrity 0 0 112 36
blkdev_queue 165 165 2872 11
blkdev_requests 315 315 376 21
fsnotify_event_holder 0 0 24 170
sock_inode_cache 272 272 960 17
file_lock_cache 102 102 240 17
shmem_inode_cache 2019 2072 1144 28
Acpi-State 255 255 80 51
Acpi-Namespace 1428 1428 40 102
task_delay_info 1704 1704 168 24
taskstats 144 144 328 24
proc_inode_cache 1632 1664 976 16
sigqueue 250 250 160 25
bdev_cache 192 192 1344 24
sysfs_dir_cache 23817 23968 144 28
filp 1807 2400 320 25
inode_cache 1861 2244 912 17
dentry 27252 29440 248 16
buffer_head 67283 102063 104 39
vm_area_struct 4527 4708 184 22
mm_struct 308 308 1152 28
files_cache 253 253 704 23
signal_cache 667 667 1408 23
sighand_cache 407 490 2240 14
task_struct 430 442 4240 7
anon_vma 4676 4676 144 28
shared_policy_node 850 850 48 85
numa_policy 3528 3528 72 56
radix_tree_node 7622 8050 568 28
idr_layer_cache 240 240 2112 15
dma-kmalloc-8192 0 0 8192 4
dma-kmalloc-4096 0 0 4096 8
dma-kmalloc-2048 0 0 2048 16
dma-kmalloc-1024 0 0 1024 16
dma-kmalloc-512 0 0 512 16
dma-kmalloc-256 0 0 256 16
dma-kmalloc-128 0 0 128 32
dma-kmalloc-64 0 0 64 64
dma-kmalloc-32 0 0 32 128
dma-kmalloc-16 0 0 16 256
dma-kmalloc-8 0 0 8 512
dma-kmalloc-192 0 0 192 21
dma-kmalloc-96 0 0 96 42
kmalloc-8192 32 32 8192 4
kmalloc-4096 835 916 4096 8
kmalloc-2048 773 832 2048 16
kmalloc-1024 2335 2336 1024 16
kmalloc-512 1488 1536 512 16
kmalloc-256 2535 2784 256 16
Cache Num Total Size Pages
kmalloc-192 69883 70707 192 21
kmalloc-128 2603 4992 128 32
kmalloc-96 5628 5628 96 42
kmalloc-64 41984 41984 64 64
kmalloc-32 154792 156288 32 128
kmalloc-16 6343 7936 16 256
kmalloc-8 8192 8192 8 512
kmem_cache_node 256 256 128 32
kmem_cache 176 176 256 16
(XEN) [2013-11-18 21:44:32] grant_table.c:1249:d1 Expanding dom (1) grant table from (4) to (5) frames.
(XEN) [2013-11-18 21:44:32] grant_table.c:289:d0 Increased maptrack size to 8 frames
(XEN) [2013-11-18 21:49:28] grant_table.c:1249:d1 Expanding dom (1) grant table from (5) to (6) frames.
(XEN) [2013-11-19 00:00:26] grant_table.c:1249:d1 Expanding dom (1) grant table from (6) to (7) frames.
(XEN) [2013-11-19 00:00:26] grant_table.c:1249:d1 Expanding dom (1) grant table from (7) to (8) frames.
(XEN) [2013-11-19 00:00:26] grant_table.c:289:d0 Increased maptrack size to 9 frames
(XEN) [2013-11-19 00:00:26] grant_table.c:289:d0 Increased maptrack size to 10 frames
(XEN) [2013-11-19 00:00:42] grant_table.c:1249:d1 Expanding dom (1) grant table from (8) to (9) frames.
(XEN) [2013-11-19 00:01:02] grant_table.c:1249:d1 Expanding dom (1) grant table from (9) to (10) frames.
(XEN) [2013-11-19 00:09:27] grant_table.c:1249:d1 Expanding dom (1) grant table from (10) to (11) frames.
(XEN) [2013-11-19 00:09:27] grant_table.c:289:d0 Increased maptrack size to 11 frames
(XEN) [2013-11-19 04:15:26] grant_table.c:289:d0 Increased maptrack size to 12 frames
(XEN) [2013-11-19 04:15:28] grant_table.c:1249:d12 Expanding dom (12) grant table from (4) to (5) frames.
(XEN) [2013-11-19 04:15:40] grant_table.c:289:d0 Increased maptrack size to 13 frames
(XEN) [2013-11-19 04:15:47] grant_table.c:1249:d10 Expanding dom (10) grant table from (4) to (5) frames.
(XEN) [2013-11-19 04:15:47] grant_table.c:1249:d10 Expanding dom (10) grant table from (5) to (6) frames.
(XEN) [2013-11-19 04:15:47] grant_table.c:289:d0 Increased maptrack size to 14 frames
(XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant table from (4) to (5) frames.
(XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant table from (5) to (6) frames.
(XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant table from (6) to (7) frames.
(XEN) [2013-11-19 04:15:52] grant_table.c:289:d0 Increased maptrack size to 15 frames
(XEN) [2013-11-19 04:15:52] grant_table.c:289:d0 Increased maptrack size to 16 frames
(XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant table from (4) to (5) frames.
(XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant table from (5) to (6) frames.
(XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant table from (6) to (7) frames.
(XEN) [2013-11-19 04:15:54] grant_table.c:289:d0 Increased maptrack size to 17 frames
(XEN) [2013-11-19 04:15:54] grant_table.c:289:d0 Increased maptrack size to 18 frames
(XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant table from (4) to (5) frames.
(XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant table from (5) to (6) frames.
(XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant table from (6) to (7) frames.
(XEN) [2013-11-19 04:15:56] grant_table.c:289:d0 Increased maptrack size to 19 frames
(XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant table from (4) to (5) frames.
(XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant table from (5) to (6) frames.
(XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant table from (6) to (7) frames.
(XEN) [2013-11-19 04:15:57] grant_table.c:289:d0 Increased maptrack size to 20 frames
(XEN) [2013-11-19 04:16:00] grant_table.c:289:d0 Increased maptrack size to 21 frames
(XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant table from (4) to (5) frames.
(XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant table from (5) to (6) frames.
(XEN) [2013-11-19 04:16:00] grant_table.c:289:d0 Increased maptrack size to 22 frames
(XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant table from (6) to (7) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant table from (4) to (5) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant table from (5) to (6) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant table from (6) to (7) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 23 frames
(XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 24 frames
(XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant table from (4) to (5) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant table from (5) to (6) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant table from (6) to (7) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 25 frames
(XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 26 frames
(XEN) [2013-11-19 04:16:06] grant_table.c:1249:d10 Expanding dom (10) grant table from (6) to (7) frames.
(XEN) [2013-11-19 04:16:22] grant_table.c:1249:d12 Expanding dom (12) grant table from (5) to (6) frames.
(XEN) [2013-11-19 04:16:22] grant_table.c:1249:d12 Expanding dom (12) grant table from (6) to (7) frames.
(XEN) [2013-11-19 04:16:22] grant_table.c:289:d0 Increased maptrack size to 27 frames
(XEN) [2013-11-19 04:16:23] grant_table.c:289:d0 Increased maptrack size to 28 frames
(XEN) [2013-11-19 04:16:24] grant_table.c:1249:d11 Expanding dom (11) grant table from (4) to (5) frames.
(XEN) [2013-11-19 04:16:24] grant_table.c:1249:d11 Expanding dom (11) grant table from (5) to (6) frames.
(XEN) [2013-11-19 04:16:24] grant_table.c:289:d0 Increased maptrack size to 29 frames
(XEN) [2013-11-19 04:16:27] grant_table.c:1249:d7 Expanding dom (7) grant table from (5) to (6) frames.
(XEN) [2013-11-19 04:16:27] grant_table.c:1249:d7 Expanding dom (7) grant table from (6) to (7) frames.
(XEN) [2013-11-19 05:15:04] grant_table.c:289:d0 Increased maptrack size to 30 frames
(XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant table from (4) to (5) frames.
(XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant table from (5) to (6) frames.
(XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant table from (6) to (7) frames.
(XEN) [2013-11-19 05:15:05] grant_table.c:289:d0 Increased maptrack size to 31 frames
(XEN) [2013-11-19 12:49:37] AMD-Vi: Share p2m table with iommu: p2m table = 0x52d3c5
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0
2013-11-19 13:02 xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0 Sander Eikelenboom
@ 2013-11-19 13:23 ` Wei Liu
2013-11-19 13:40 ` Sander Eikelenboom
2013-11-19 13:41 ` David Vrabel
1 sibling, 1 reply; 6+ messages in thread
From: Wei Liu @ 2013-11-19 13:23 UTC (permalink / raw)
To: Sander Eikelenboom; +Cc: Wei Liu, Ian Campbell, xen-devel
On Tue, Nov 19, 2013 at 02:02:57PM +0100, Sander Eikelenboom wrote:
> Hi Wei,
>
> I ran into the following problem when trying to boot another guest after less than a day of uptime.
> (the system started 15 guests at boot already which went fine). dom0 is allocated a fixed 1536M.
>
> Both host as pv guests run the same kernel, some hvm's run a slightly older kernel (3.9 f.e.)
>
> The are quite some granttable messages in xl dmesg, i also included these and a "vmstat -m"
>
This looks like a normal OOM. alloc_netdev is not able to allocate
xenvif structure, which is only a few hundred / thousand bytes (don't
remember the exact number).
Are you using persistent grant in blkback? I think those grant table
messages come from blkback hording grant references. It also implies it
might be hording Dom0 pages. You can try to turn off persistent grant
and see if it works.
Wei.
> --
> Sander
>
> [54807.299791] xenwatch: page allocation failure: order:4, mode:0x10c0d0
> [54807.317520] CPU: 5 PID: 54 Comm: xenwatch Not tainted 3.12.0-20131104 #1
> [54807.334747] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640) , BIOS V1.8B1 09/13/2010
> [54807.351990] 0000000000000000 ffff8800591a3828 ffffffff81a6a8ba 0000000000000006
> [54807.368857] 000000000010c0d0 ffff8800591a38b8 ffffffff81138770 0000000000000004
> [54807.385539] 000000000010c0d0 ffff8800591a38b8 ffffffff81a661de ffff88005fd1c680
> [54807.401978] Call Trace:
> [54807.418015] [<ffffffff81a6a8ba>] dump_stack+0x4f/0x84
> [54807.433971] [<ffffffff81138770>] warn_alloc_failed+0xf0/0x140
> [54807.449644] [<ffffffff81a661de>] ? __alloc_pages_direct_compact+0x1ac/0x1be
> [54807.465164] [<ffffffff8113bfaa>] __alloc_pages_nodemask+0x7aa/0x9d0
> [54807.480510] [<ffffffff810ed069>] ? trace_hardirqs_off_caller+0xb9/0x160
> [54807.495622] [<ffffffff81175277>] alloc_pages_current+0xb7/0x180
> [54807.510530] [<ffffffff81138059>] __get_free_pages+0x9/0x40
> [54807.525185] [<ffffffff8117cbdc>] __kmalloc+0x19c/0x1c0
> [54807.539538] [<ffffffff8190e9b4>] alloc_netdev_mqs+0x64/0x340
> [54807.553814] [<ffffffff8192ac20>] ? alloc_etherdev_mqs+0x20/0x20
> [54807.567777] [<ffffffff816dc3e4>] xenvif_alloc+0x64/0x2c0
> [54807.581473] [<ffffffff816dbc57>] netback_probe+0x287/0x2d0
> [54807.594971] [<ffffffff814bfe46>] xenbus_dev_probe+0x66/0x110
> [54807.608231] [<ffffffff81615105>] driver_probe_device+0x75/0x210
> [54807.621227] [<ffffffff81615350>] ? __driver_attach+0xb0/0xb0
> [54807.634071] [<ffffffff8161539b>] __device_attach+0x4b/0x60
> [54807.646626] [<ffffffff8161316e>] bus_for_each_drv+0x4e/0xa0
> [54807.658918] [<ffffffff81615058>] device_attach+0x98/0xb0
> [54807.671253] [<ffffffff816144b0>] bus_probe_device+0xb0/0xe0
> [54807.683379] [<ffffffff81612277>] device_add+0x3b7/0x700
> [54807.695145] [<ffffffff8161ca1d>] ? device_pm_sleep_init+0x4d/0x80
> [54807.706824] [<ffffffff816125d9>] device_register+0x19/0x20
> [54807.718145] [<ffffffff814bf9b1>] xenbus_probe_node+0x141/0x170
> [54807.729256] [<ffffffff81613236>] ? bus_for_each_dev+0x76/0xa0
> [54807.740091] [<ffffffff814bfbb0>] xenbus_dev_changed+0x1d0/0x1e0
> [54807.750811] [<ffffffff814bff26>] backend_changed+0x16/0x20
> [54807.761256] [<ffffffff814bdf3e>] xenwatch_thread+0x4e/0x140
> [54807.771371] [<ffffffff810bc1e0>] ? __init_waitqueue_head+0x60/0x60
> [54807.781443] [<ffffffff814bdef0>] ? xs_watch+0x60/0x60
> [54807.791310] [<ffffffff810bb716>] kthread+0xd6/0xe0
> [54807.800794] [<ffffffff81a752bb>] ? _raw_spin_unlock_irq+0x2b/0x70
> [54807.810150] [<ffffffff810bb640>] ? __init_kthread_worker+0x70/0x70
> [54807.819526] [<ffffffff81a762cc>] ret_from_fork+0x7c/0xb0
> [54807.828638] [<ffffffff810bb640>] ? __init_kthread_worker+0x70/0x70
> [54807.837482] Mem-Info:
> [54807.846011] Node 0 DMA per-cpu:
> [54807.854277] CPU 0: hi: 0, btch: 1 usd: 0
> [54807.862404] CPU 1: hi: 0, btch: 1 usd: 0
> [54807.870241] CPU 2: hi: 0, btch: 1 usd: 0
> [54807.877985] CPU 3: hi: 0, btch: 1 usd: 0
> [54807.885344] CPU 4: hi: 0, btch: 1 usd: 0
> [54807.892389] CPU 5: hi: 0, btch: 1 usd: 0
> [54807.899102] Node 0 DMA32 per-cpu:
> [54807.905665] CPU 0: hi: 186, btch: 31 usd: 68
> [54807.911985] CPU 1: hi: 186, btch: 31 usd: 6
> [54807.917964] CPU 2: hi: 186, btch: 31 usd: 149
> [54807.923666] CPU 3: hi: 186, btch: 31 usd: 82
> [54807.929031] CPU 4: hi: 186, btch: 31 usd: 169
> [54807.934293] CPU 5: hi: 186, btch: 31 usd: 0
> [54807.939207] active_anon:12850 inactive_anon:8944 isolated_anon:0
> [54807.939207] active_file:70321 inactive_file:177850 isolated_file:0
> [54807.939207] unevictable:562 dirty:34 writeback:0 unstable:0
> [54807.939207] free:31143 slab_reclaimable:21805 slab_unreclaimable:12717
> [54807.939207] mapped:3344 shmem:276 pagetables:1211 bounce:0
> [54807.939207] free_cma:0
> [54807.966465] Node 0 DMA free:5644kB min:52kB low:64kB high:76kB active_anon:0kB inactive_anon:4kB active_file:4kB inactive_file:0kB unevictable:24kB isolated(anon):0kB isolated(file):0kB present:15968kB managed:15884kB mlocked:24kB dirty:0kB writeback:0kB mapped:24kB shmem:0kB slab_reclaimable:8992kB slab_unreclaimable:656kB kernel_stack:16kB pagetables:56kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
> [54807.981676] lowmem_reserve[]: 0 1395 1395 1395
> [54807.987015] Node 0 DMA32 free:118928kB min:4748kB low:5932kB high:7120kB active_anon:51028kB inactive_anon:35772kB active_file:281280kB inactive_file:711400kB unevictable:2224kB isolated(anon):0kB isolated(file):0kB present:1556480kB managed:1433216kB mlocked:2224kB dirty:136kB writeback:0kB mapped:13168kB shmem:1104kB slab_reclaimable:78228kB slab_unreclaimable:50212kB kernel_stack:3256kB pagetables:4788kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
> [54808.011107] lowmem_reserve[]: 0 0 0 0
> [54808.017244] Node 0 DMA: 19*4kB (UEM) 18*8kB (UEM) 15*16kB (UEM) 14*32kB (UEM) 8*64kB (UEM) 7*128kB (UEM) 3*256kB (U) 1*512kB (U) 0*1024kB 1*2048kB (R) 0*4096kB = 5644kB
> [54808.030315] Node 0 DMA32: 11490*4kB (UEM) 6743*8kB (UEM) 1133*16kB (EMR) 6*32kB (MR) 0*64kB 1*128kB (R) 1*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 118608kB
> [54808.044303] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
> [54808.051507] 249203 total pagecache pages
> [54808.058647] 201 pages in swap cache
> [54808.065986] Swap cache stats: add 2902, delete 2701, find 62055/62191
> [54808.073216] Free swap = 2092752kB
> [54808.080397] Total swap = 2097148kB
> [54808.093896] 393215 pages RAM
> [54808.101004] 30940 pages reserved
> [54808.108168] 339108 pages shared
> [54808.115202] 258777 pages non-shared
> [54808.122235] xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0
> [54808.129391] vif vif-16-0: 12 creating interface
>
>
> ~# vmstat -m
> Cache Num Total Size Pages
> ext4_groupinfo_4k 24426 24426 216 18
> ceph_osd_request 0 0 960 17
> xt_hashlimit 0 0 136 30
> nf_conntrack_ffffffff822d2900 1950 2100 400 20
> nf_conntrack_expect 0 0 288 28
> dm_snap_pending_exception 2496 2496 104 39
> kcopyd_job 91 133 4176 7
> dm_rq_target_io 0 0 416 19
> search 0 0 616 26
> blkif_cache 108 108 864 18
> cfq_io_cq 204 204 120 34
> cfq_queue 170 170 232 17
> bsg_cmd 0 0 312 26
> ceph_cap 0 0 128 32
> ceph_inode_info 0 0 2176 15
> gfs2_mblk 0 0 432 18
> gfs2_bufdata 0 0 88 46
> gfs2_inode 0 0 1280 25
> gfs2_glock(aspace) 0 0 960 17
> gfs2_glock 0 0 592 27
> btrfs_delayed_data_ref 0 0 96 42
> btrfs_delayed_ref_head 0 0 232 17
> btrfs_delayed_node 0 0 392 20
> btrfs_ordered_extent 0 0 472 17
> btrfs_extent_buffer 0 0 568 28
> btrfs_delalloc_work 0 0 184 22
> btrfs_path 0 0 144 28
> btrfs_transaction 0 0 600 27
> btrfs_trans_handle 0 0 160 25
> btrfs_inode 0 0 1944 16
> fuse_request 102 102 480 17
> fuse_inode 336 476 1152 28
> ntfs_big_inode_cache 0 0 1664 19
> ntfs_inode_cache 0 0 696 23
> cifs_small_rq 36 36 448 18
> cifs_request 4 4 16512 1
> cifs_inode_cache 0 0 1112 29
> isofs_inode_cache 0 0 960 17
> fat_inode_cache 0 0 1168 28
> fat_cache 0 0 40 102
> hugetlbfs_inode_cache 16 16 976 16
> jbd2_transaction_s 150 150 320 25
> jbd2_journal_handle 306 306 80 51
> journal_handle 0 0 56 73
> journal_head 1518 1800 112 36
> revoke_table 1536 1536 16 256
> revoke_record 768 768 32 128
> ext4_inode_cache 30702 30952 1704 19
> ext4_free_data 768 768 64 64
> ext4_allocation_context 180 180 136 30
> ext4_prealloc_space 156 156 152 26
> ext4_io_end 336 336 72 56
> ext4_extent_status 7361 7446 40 102
> ext3_inode_cache 0 0 1312 24
> ext3_xattr 3726 3956 88 46
> dquot 0 0 384 21
> kioctx 0 0 896 18
> pid_namespace 0 0 2208 14
> posix_timers_cache 108 108 296 27
> UNIX 198 198 1472 22
> Cache Num Total Size Pages
> UDP-Lite 0 0 1280 25
> ip_fib_trie 438 438 56 73
> PING 0 0 1216 26
> UDP 150 150 1280 25
> tw_sock_TCP 399 399 192 21
> TCP 104 104 2368 13
> fscache_cookie_jar 0 0 192 21
> sgpool-128 54 78 5120 6
> sgpool-64 72 72 2560 12
> sgpool-32 325 325 1280 25
> sgpool-16 277 350 640 25
> blkdev_integrity 0 0 112 36
> blkdev_queue 165 165 2872 11
> blkdev_requests 315 315 376 21
> fsnotify_event_holder 0 0 24 170
> sock_inode_cache 272 272 960 17
> file_lock_cache 102 102 240 17
> shmem_inode_cache 2019 2072 1144 28
> Acpi-State 255 255 80 51
> Acpi-Namespace 1428 1428 40 102
> task_delay_info 1704 1704 168 24
> taskstats 144 144 328 24
> proc_inode_cache 1632 1664 976 16
> sigqueue 250 250 160 25
> bdev_cache 192 192 1344 24
> sysfs_dir_cache 23817 23968 144 28
> filp 1807 2400 320 25
> inode_cache 1861 2244 912 17
> dentry 27252 29440 248 16
> buffer_head 67283 102063 104 39
> vm_area_struct 4527 4708 184 22
> mm_struct 308 308 1152 28
> files_cache 253 253 704 23
> signal_cache 667 667 1408 23
> sighand_cache 407 490 2240 14
> task_struct 430 442 4240 7
> anon_vma 4676 4676 144 28
> shared_policy_node 850 850 48 85
> numa_policy 3528 3528 72 56
> radix_tree_node 7622 8050 568 28
> idr_layer_cache 240 240 2112 15
> dma-kmalloc-8192 0 0 8192 4
> dma-kmalloc-4096 0 0 4096 8
> dma-kmalloc-2048 0 0 2048 16
> dma-kmalloc-1024 0 0 1024 16
> dma-kmalloc-512 0 0 512 16
> dma-kmalloc-256 0 0 256 16
> dma-kmalloc-128 0 0 128 32
> dma-kmalloc-64 0 0 64 64
> dma-kmalloc-32 0 0 32 128
> dma-kmalloc-16 0 0 16 256
> dma-kmalloc-8 0 0 8 512
> dma-kmalloc-192 0 0 192 21
> dma-kmalloc-96 0 0 96 42
> kmalloc-8192 32 32 8192 4
> kmalloc-4096 835 916 4096 8
> kmalloc-2048 773 832 2048 16
> kmalloc-1024 2335 2336 1024 16
> kmalloc-512 1488 1536 512 16
> kmalloc-256 2535 2784 256 16
> Cache Num Total Size Pages
> kmalloc-192 69883 70707 192 21
> kmalloc-128 2603 4992 128 32
> kmalloc-96 5628 5628 96 42
> kmalloc-64 41984 41984 64 64
> kmalloc-32 154792 156288 32 128
> kmalloc-16 6343 7936 16 256
> kmalloc-8 8192 8192 8 512
> kmem_cache_node 256 256 128 32
> kmem_cache 176 176 256 16
>
>
> (XEN) [2013-11-18 21:44:32] grant_table.c:1249:d1 Expanding dom (1) grant table from (4) to (5) frames.
> (XEN) [2013-11-18 21:44:32] grant_table.c:289:d0 Increased maptrack size to 8 frames
> (XEN) [2013-11-18 21:49:28] grant_table.c:1249:d1 Expanding dom (1) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 00:00:26] grant_table.c:1249:d1 Expanding dom (1) grant table from (6) to (7) frames.
> (XEN) [2013-11-19 00:00:26] grant_table.c:1249:d1 Expanding dom (1) grant table from (7) to (8) frames.
> (XEN) [2013-11-19 00:00:26] grant_table.c:289:d0 Increased maptrack size to 9 frames
> (XEN) [2013-11-19 00:00:26] grant_table.c:289:d0 Increased maptrack size to 10 frames
> (XEN) [2013-11-19 00:00:42] grant_table.c:1249:d1 Expanding dom (1) grant table from (8) to (9) frames.
> (XEN) [2013-11-19 00:01:02] grant_table.c:1249:d1 Expanding dom (1) grant table from (9) to (10) frames.
> (XEN) [2013-11-19 00:09:27] grant_table.c:1249:d1 Expanding dom (1) grant table from (10) to (11) frames.
> (XEN) [2013-11-19 00:09:27] grant_table.c:289:d0 Increased maptrack size to 11 frames
> (XEN) [2013-11-19 04:15:26] grant_table.c:289:d0 Increased maptrack size to 12 frames
> (XEN) [2013-11-19 04:15:28] grant_table.c:1249:d12 Expanding dom (12) grant table from (4) to (5) frames.
> (XEN) [2013-11-19 04:15:40] grant_table.c:289:d0 Increased maptrack size to 13 frames
> (XEN) [2013-11-19 04:15:47] grant_table.c:1249:d10 Expanding dom (10) grant table from (4) to (5) frames.
> (XEN) [2013-11-19 04:15:47] grant_table.c:1249:d10 Expanding dom (10) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 04:15:47] grant_table.c:289:d0 Increased maptrack size to 14 frames
> (XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant table from (4) to (5) frames.
> (XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant table from (6) to (7) frames.
> (XEN) [2013-11-19 04:15:52] grant_table.c:289:d0 Increased maptrack size to 15 frames
> (XEN) [2013-11-19 04:15:52] grant_table.c:289:d0 Increased maptrack size to 16 frames
> (XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant table from (4) to (5) frames.
> (XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant table from (6) to (7) frames.
> (XEN) [2013-11-19 04:15:54] grant_table.c:289:d0 Increased maptrack size to 17 frames
> (XEN) [2013-11-19 04:15:54] grant_table.c:289:d0 Increased maptrack size to 18 frames
> (XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant table from (4) to (5) frames.
> (XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant table from (6) to (7) frames.
> (XEN) [2013-11-19 04:15:56] grant_table.c:289:d0 Increased maptrack size to 19 frames
> (XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant table from (4) to (5) frames.
> (XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant table from (6) to (7) frames.
> (XEN) [2013-11-19 04:15:57] grant_table.c:289:d0 Increased maptrack size to 20 frames
> (XEN) [2013-11-19 04:16:00] grant_table.c:289:d0 Increased maptrack size to 21 frames
> (XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant table from (4) to (5) frames.
> (XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 04:16:00] grant_table.c:289:d0 Increased maptrack size to 22 frames
> (XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant table from (6) to (7) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant table from (4) to (5) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant table from (6) to (7) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 23 frames
> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 24 frames
> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant table from (4) to (5) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant table from (6) to (7) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 25 frames
> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 26 frames
> (XEN) [2013-11-19 04:16:06] grant_table.c:1249:d10 Expanding dom (10) grant table from (6) to (7) frames.
> (XEN) [2013-11-19 04:16:22] grant_table.c:1249:d12 Expanding dom (12) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 04:16:22] grant_table.c:1249:d12 Expanding dom (12) grant table from (6) to (7) frames.
> (XEN) [2013-11-19 04:16:22] grant_table.c:289:d0 Increased maptrack size to 27 frames
> (XEN) [2013-11-19 04:16:23] grant_table.c:289:d0 Increased maptrack size to 28 frames
> (XEN) [2013-11-19 04:16:24] grant_table.c:1249:d11 Expanding dom (11) grant table from (4) to (5) frames.
> (XEN) [2013-11-19 04:16:24] grant_table.c:1249:d11 Expanding dom (11) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 04:16:24] grant_table.c:289:d0 Increased maptrack size to 29 frames
> (XEN) [2013-11-19 04:16:27] grant_table.c:1249:d7 Expanding dom (7) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 04:16:27] grant_table.c:1249:d7 Expanding dom (7) grant table from (6) to (7) frames.
> (XEN) [2013-11-19 05:15:04] grant_table.c:289:d0 Increased maptrack size to 30 frames
> (XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant table from (4) to (5) frames.
> (XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant table from (5) to (6) frames.
> (XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant table from (6) to (7) frames.
> (XEN) [2013-11-19 05:15:05] grant_table.c:289:d0 Increased maptrack size to 31 frames
> (XEN) [2013-11-19 12:49:37] AMD-Vi: Share p2m table with iommu: p2m table = 0x52d3c5
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0
2013-11-19 13:23 ` Wei Liu
@ 2013-11-19 13:40 ` Sander Eikelenboom
0 siblings, 0 replies; 6+ messages in thread
From: Sander Eikelenboom @ 2013-11-19 13:40 UTC (permalink / raw)
To: Wei Liu; +Cc: Ian Campbell, xen-devel
Tuesday, November 19, 2013, 2:23:35 PM, you wrote:
> On Tue, Nov 19, 2013 at 02:02:57PM +0100, Sander Eikelenboom wrote:
>> Hi Wei,
>>
>> I ran into the following problem when trying to boot another guest after less than a day of uptime.
>> (the system started 15 guests at boot already which went fine). dom0 is allocated a fixed 1536M.
>>
>> Both host as pv guests run the same kernel, some hvm's run a slightly older kernel (3.9 f.e.)
>>
>> The are quite some granttable messages in xl dmesg, i also included these and a "vmstat -m"
>>
> This looks like a normal OOM. alloc_netdev is not able to allocate
> xenvif structure, which is only a few hundred / thousand bytes (don't
> remember the exact number).
> Are you using persistent grant in blkback? I think those grant table
> messages come from blkback hording grant references. It also implies it
> might be hording Dom0 pages. You can try to turn off persistent grant
> and see if it works.
Yes it does use persistent grants, at least blkback does, i don't see any reference of netback ..
# dmesg | grep persistent
[ 130.385948] xen-blkback:ring-ref 8, event-channel 17, protocol 1 (x86_64-abi) persistent grants
[ 130.401092] xen-blkback:ring-ref 9, event-channel 18, protocol 1 (x86_64-abi) persistent grants
[ 136.414469] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi) persistent grants
[ 142.772308] xen-blkback:ring-ref 8, event-channel 17, protocol 1 (x86_64-abi) persistent grants
[ 148.152987] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi) persistent grants
[ 154.047275] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi) persistent grants
[ 160.033898] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi) persistent grants
[ 166.474991] xen-blkback:ring-ref 8, event-channel 17, protocol 1 (x86_64-abi) persistent grants
[ 172.111850] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi) persistent grants
[ 180.454091] xen-blkback:ring-ref 8, event-channel 17, protocol 1 (x86_64-abi) persistent grants
[ 186.935862] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi) persistent grants
[ 193.496961] xen-blkback:ring-ref 9, event-channel 11, protocol 1 (x86_64-abi) persistent grants
[ 200.179170] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi) persistent grants
[ 206.476770] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi) persistent grants
[ 259.796801] xen-blkback:ring-ref 8, event-channel 32, protocol 1 (x86_64-abi) persistent grants
[ 259.811314] xen-blkback:ring-ref 9, event-channel 33, protocol 1 (x86_64-abi) persistent grants
[ 264.968420] xen-blkback:ring-ref 8, event-channel 22, protocol 1 (x86_64-abi) persistent grants
hmm naive as i am i thought that would limit the mem usage by re-using the grants (and thus pages).
Why does it keep increasing the number of grants ?
Doesn't the device rings have a max size, so once that's reached it should stop allocating new and only reuse it persitent grants ?
Also it must be a special memtype/size ? Dom0 was only using about 400M of it's 1536M.
> Wei.
>> --
>> Sander
>>
>> [54807.299791] xenwatch: page allocation failure: order:4, mode:0x10c0d0
>> [54807.317520] CPU: 5 PID: 54 Comm: xenwatch Not tainted 3.12.0-20131104 #1
>> [54807.334747] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640) , BIOS V1.8B1 09/13/2010
>> [54807.351990] 0000000000000000 ffff8800591a3828 ffffffff81a6a8ba 0000000000000006
>> [54807.368857] 000000000010c0d0 ffff8800591a38b8 ffffffff81138770 0000000000000004
>> [54807.385539] 000000000010c0d0 ffff8800591a38b8 ffffffff81a661de ffff88005fd1c680
>> [54807.401978] Call Trace:
>> [54807.418015] [<ffffffff81a6a8ba>] dump_stack+0x4f/0x84
>> [54807.433971] [<ffffffff81138770>] warn_alloc_failed+0xf0/0x140
>> [54807.449644] [<ffffffff81a661de>] ? __alloc_pages_direct_compact+0x1ac/0x1be
>> [54807.465164] [<ffffffff8113bfaa>] __alloc_pages_nodemask+0x7aa/0x9d0
>> [54807.480510] [<ffffffff810ed069>] ? trace_hardirqs_off_caller+0xb9/0x160
>> [54807.495622] [<ffffffff81175277>] alloc_pages_current+0xb7/0x180
>> [54807.510530] [<ffffffff81138059>] __get_free_pages+0x9/0x40
>> [54807.525185] [<ffffffff8117cbdc>] __kmalloc+0x19c/0x1c0
>> [54807.539538] [<ffffffff8190e9b4>] alloc_netdev_mqs+0x64/0x340
>> [54807.553814] [<ffffffff8192ac20>] ? alloc_etherdev_mqs+0x20/0x20
>> [54807.567777] [<ffffffff816dc3e4>] xenvif_alloc+0x64/0x2c0
>> [54807.581473] [<ffffffff816dbc57>] netback_probe+0x287/0x2d0
>> [54807.594971] [<ffffffff814bfe46>] xenbus_dev_probe+0x66/0x110
>> [54807.608231] [<ffffffff81615105>] driver_probe_device+0x75/0x210
>> [54807.621227] [<ffffffff81615350>] ? __driver_attach+0xb0/0xb0
>> [54807.634071] [<ffffffff8161539b>] __device_attach+0x4b/0x60
>> [54807.646626] [<ffffffff8161316e>] bus_for_each_drv+0x4e/0xa0
>> [54807.658918] [<ffffffff81615058>] device_attach+0x98/0xb0
>> [54807.671253] [<ffffffff816144b0>] bus_probe_device+0xb0/0xe0
>> [54807.683379] [<ffffffff81612277>] device_add+0x3b7/0x700
>> [54807.695145] [<ffffffff8161ca1d>] ? device_pm_sleep_init+0x4d/0x80
>> [54807.706824] [<ffffffff816125d9>] device_register+0x19/0x20
>> [54807.718145] [<ffffffff814bf9b1>] xenbus_probe_node+0x141/0x170
>> [54807.729256] [<ffffffff81613236>] ? bus_for_each_dev+0x76/0xa0
>> [54807.740091] [<ffffffff814bfbb0>] xenbus_dev_changed+0x1d0/0x1e0
>> [54807.750811] [<ffffffff814bff26>] backend_changed+0x16/0x20
>> [54807.761256] [<ffffffff814bdf3e>] xenwatch_thread+0x4e/0x140
>> [54807.771371] [<ffffffff810bc1e0>] ? __init_waitqueue_head+0x60/0x60
>> [54807.781443] [<ffffffff814bdef0>] ? xs_watch+0x60/0x60
>> [54807.791310] [<ffffffff810bb716>] kthread+0xd6/0xe0
>> [54807.800794] [<ffffffff81a752bb>] ? _raw_spin_unlock_irq+0x2b/0x70
>> [54807.810150] [<ffffffff810bb640>] ? __init_kthread_worker+0x70/0x70
>> [54807.819526] [<ffffffff81a762cc>] ret_from_fork+0x7c/0xb0
>> [54807.828638] [<ffffffff810bb640>] ? __init_kthread_worker+0x70/0x70
>> [54807.837482] Mem-Info:
>> [54807.846011] Node 0 DMA per-cpu:
>> [54807.854277] CPU 0: hi: 0, btch: 1 usd: 0
>> [54807.862404] CPU 1: hi: 0, btch: 1 usd: 0
>> [54807.870241] CPU 2: hi: 0, btch: 1 usd: 0
>> [54807.877985] CPU 3: hi: 0, btch: 1 usd: 0
>> [54807.885344] CPU 4: hi: 0, btch: 1 usd: 0
>> [54807.892389] CPU 5: hi: 0, btch: 1 usd: 0
>> [54807.899102] Node 0 DMA32 per-cpu:
>> [54807.905665] CPU 0: hi: 186, btch: 31 usd: 68
>> [54807.911985] CPU 1: hi: 186, btch: 31 usd: 6
>> [54807.917964] CPU 2: hi: 186, btch: 31 usd: 149
>> [54807.923666] CPU 3: hi: 186, btch: 31 usd: 82
>> [54807.929031] CPU 4: hi: 186, btch: 31 usd: 169
>> [54807.934293] CPU 5: hi: 186, btch: 31 usd: 0
>> [54807.939207] active_anon:12850 inactive_anon:8944 isolated_anon:0
>> [54807.939207] active_file:70321 inactive_file:177850 isolated_file:0
>> [54807.939207] unevictable:562 dirty:34 writeback:0 unstable:0
>> [54807.939207] free:31143 slab_reclaimable:21805 slab_unreclaimable:12717
>> [54807.939207] mapped:3344 shmem:276 pagetables:1211 bounce:0
>> [54807.939207] free_cma:0
>> [54807.966465] Node 0 DMA free:5644kB min:52kB low:64kB high:76kB active_anon:0kB inactive_anon:4kB active_file:4kB inactive_file:0kB unevictable:24kB isolated(anon):0kB isolated(file):0kB present:15968kB managed:15884kB mlocked:24kB dirty:0kB writeback:0kB mapped:24kB shmem:0kB slab_reclaimable:8992kB slab_unreclaimable:656kB kernel_stack:16kB pagetables:56kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
>> [54807.981676] lowmem_reserve[]: 0 1395 1395 1395
>> [54807.987015] Node 0 DMA32 free:118928kB min:4748kB low:5932kB high:7120kB active_anon:51028kB inactive_anon:35772kB active_file:281280kB inactive_file:711400kB unevictable:2224kB isolated(anon):0kB isolated(file):0kB present:1556480kB managed:1433216kB mlocked:2224kB dirty:136kB writeback:0kB mapped:13168kB shmem:1104kB slab_reclaimable:78228kB slab_unreclaimable:50212kB kernel_stack:3256kB pagetables:4788kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
>> [54808.011107] lowmem_reserve[]: 0 0 0 0
>> [54808.017244] Node 0 DMA: 19*4kB (UEM) 18*8kB (UEM) 15*16kB (UEM) 14*32kB (UEM) 8*64kB (UEM) 7*128kB (UEM) 3*256kB (U) 1*512kB (U) 0*1024kB 1*2048kB (R) 0*4096kB = 5644kB
>> [54808.030315] Node 0 DMA32: 11490*4kB (UEM) 6743*8kB (UEM) 1133*16kB (EMR) 6*32kB (MR) 0*64kB 1*128kB (R) 1*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 118608kB
>> [54808.044303] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
>> [54808.051507] 249203 total pagecache pages
>> [54808.058647] 201 pages in swap cache
>> [54808.065986] Swap cache stats: add 2902, delete 2701, find 62055/62191
>> [54808.073216] Free swap = 2092752kB
>> [54808.080397] Total swap = 2097148kB
>> [54808.093896] 393215 pages RAM
>> [54808.101004] 30940 pages reserved
>> [54808.108168] 339108 pages shared
>> [54808.115202] 258777 pages non-shared
>> [54808.122235] xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0
>> [54808.129391] vif vif-16-0: 12 creating interface
>>
>>
>> ~# vmstat -m
>> Cache Num Total Size Pages
>> ext4_groupinfo_4k 24426 24426 216 18
>> ceph_osd_request 0 0 960 17
>> xt_hashlimit 0 0 136 30
>> nf_conntrack_ffffffff822d2900 1950 2100 400 20
>> nf_conntrack_expect 0 0 288 28
>> dm_snap_pending_exception 2496 2496 104 39
>> kcopyd_job 91 133 4176 7
>> dm_rq_target_io 0 0 416 19
>> search 0 0 616 26
>> blkif_cache 108 108 864 18
>> cfq_io_cq 204 204 120 34
>> cfq_queue 170 170 232 17
>> bsg_cmd 0 0 312 26
>> ceph_cap 0 0 128 32
>> ceph_inode_info 0 0 2176 15
>> gfs2_mblk 0 0 432 18
>> gfs2_bufdata 0 0 88 46
>> gfs2_inode 0 0 1280 25
>> gfs2_glock(aspace) 0 0 960 17
>> gfs2_glock 0 0 592 27
>> btrfs_delayed_data_ref 0 0 96 42
>> btrfs_delayed_ref_head 0 0 232 17
>> btrfs_delayed_node 0 0 392 20
>> btrfs_ordered_extent 0 0 472 17
>> btrfs_extent_buffer 0 0 568 28
>> btrfs_delalloc_work 0 0 184 22
>> btrfs_path 0 0 144 28
>> btrfs_transaction 0 0 600 27
>> btrfs_trans_handle 0 0 160 25
>> btrfs_inode 0 0 1944 16
>> fuse_request 102 102 480 17
>> fuse_inode 336 476 1152 28
>> ntfs_big_inode_cache 0 0 1664 19
>> ntfs_inode_cache 0 0 696 23
>> cifs_small_rq 36 36 448 18
>> cifs_request 4 4 16512 1
>> cifs_inode_cache 0 0 1112 29
>> isofs_inode_cache 0 0 960 17
>> fat_inode_cache 0 0 1168 28
>> fat_cache 0 0 40 102
>> hugetlbfs_inode_cache 16 16 976 16
>> jbd2_transaction_s 150 150 320 25
>> jbd2_journal_handle 306 306 80 51
>> journal_handle 0 0 56 73
>> journal_head 1518 1800 112 36
>> revoke_table 1536 1536 16 256
>> revoke_record 768 768 32 128
>> ext4_inode_cache 30702 30952 1704 19
>> ext4_free_data 768 768 64 64
>> ext4_allocation_context 180 180 136 30
>> ext4_prealloc_space 156 156 152 26
>> ext4_io_end 336 336 72 56
>> ext4_extent_status 7361 7446 40 102
>> ext3_inode_cache 0 0 1312 24
>> ext3_xattr 3726 3956 88 46
>> dquot 0 0 384 21
>> kioctx 0 0 896 18
>> pid_namespace 0 0 2208 14
>> posix_timers_cache 108 108 296 27
>> UNIX 198 198 1472 22
>> Cache Num Total Size Pages
>> UDP-Lite 0 0 1280 25
>> ip_fib_trie 438 438 56 73
>> PING 0 0 1216 26
>> UDP 150 150 1280 25
>> tw_sock_TCP 399 399 192 21
>> TCP 104 104 2368 13
>> fscache_cookie_jar 0 0 192 21
>> sgpool-128 54 78 5120 6
>> sgpool-64 72 72 2560 12
>> sgpool-32 325 325 1280 25
>> sgpool-16 277 350 640 25
>> blkdev_integrity 0 0 112 36
>> blkdev_queue 165 165 2872 11
>> blkdev_requests 315 315 376 21
>> fsnotify_event_holder 0 0 24 170
>> sock_inode_cache 272 272 960 17
>> file_lock_cache 102 102 240 17
>> shmem_inode_cache 2019 2072 1144 28
>> Acpi-State 255 255 80 51
>> Acpi-Namespace 1428 1428 40 102
>> task_delay_info 1704 1704 168 24
>> taskstats 144 144 328 24
>> proc_inode_cache 1632 1664 976 16
>> sigqueue 250 250 160 25
>> bdev_cache 192 192 1344 24
>> sysfs_dir_cache 23817 23968 144 28
>> filp 1807 2400 320 25
>> inode_cache 1861 2244 912 17
>> dentry 27252 29440 248 16
>> buffer_head 67283 102063 104 39
>> vm_area_struct 4527 4708 184 22
>> mm_struct 308 308 1152 28
>> files_cache 253 253 704 23
>> signal_cache 667 667 1408 23
>> sighand_cache 407 490 2240 14
>> task_struct 430 442 4240 7
>> anon_vma 4676 4676 144 28
>> shared_policy_node 850 850 48 85
>> numa_policy 3528 3528 72 56
>> radix_tree_node 7622 8050 568 28
>> idr_layer_cache 240 240 2112 15
>> dma-kmalloc-8192 0 0 8192 4
>> dma-kmalloc-4096 0 0 4096 8
>> dma-kmalloc-2048 0 0 2048 16
>> dma-kmalloc-1024 0 0 1024 16
>> dma-kmalloc-512 0 0 512 16
>> dma-kmalloc-256 0 0 256 16
>> dma-kmalloc-128 0 0 128 32
>> dma-kmalloc-64 0 0 64 64
>> dma-kmalloc-32 0 0 32 128
>> dma-kmalloc-16 0 0 16 256
>> dma-kmalloc-8 0 0 8 512
>> dma-kmalloc-192 0 0 192 21
>> dma-kmalloc-96 0 0 96 42
>> kmalloc-8192 32 32 8192 4
>> kmalloc-4096 835 916 4096 8
>> kmalloc-2048 773 832 2048 16
>> kmalloc-1024 2335 2336 1024 16
>> kmalloc-512 1488 1536 512 16
>> kmalloc-256 2535 2784 256 16
>> Cache Num Total Size Pages
>> kmalloc-192 69883 70707 192 21
>> kmalloc-128 2603 4992 128 32
>> kmalloc-96 5628 5628 96 42
>> kmalloc-64 41984 41984 64 64
>> kmalloc-32 154792 156288 32 128
>> kmalloc-16 6343 7936 16 256
>> kmalloc-8 8192 8192 8 512
>> kmem_cache_node 256 256 128 32
>> kmem_cache 176 176 256 16
>>
>>
>> (XEN) [2013-11-18 21:44:32] grant_table.c:1249:d1 Expanding dom (1) grant table from (4) to (5) frames.
>> (XEN) [2013-11-18 21:44:32] grant_table.c:289:d0 Increased maptrack size to 8 frames
>> (XEN) [2013-11-18 21:49:28] grant_table.c:1249:d1 Expanding dom (1) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 00:00:26] grant_table.c:1249:d1 Expanding dom (1) grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 00:00:26] grant_table.c:1249:d1 Expanding dom (1) grant table from (7) to (8) frames.
>> (XEN) [2013-11-19 00:00:26] grant_table.c:289:d0 Increased maptrack size to 9 frames
>> (XEN) [2013-11-19 00:00:26] grant_table.c:289:d0 Increased maptrack size to 10 frames
>> (XEN) [2013-11-19 00:00:42] grant_table.c:1249:d1 Expanding dom (1) grant table from (8) to (9) frames.
>> (XEN) [2013-11-19 00:01:02] grant_table.c:1249:d1 Expanding dom (1) grant table from (9) to (10) frames.
>> (XEN) [2013-11-19 00:09:27] grant_table.c:1249:d1 Expanding dom (1) grant table from (10) to (11) frames.
>> (XEN) [2013-11-19 00:09:27] grant_table.c:289:d0 Increased maptrack size to 11 frames
>> (XEN) [2013-11-19 04:15:26] grant_table.c:289:d0 Increased maptrack size to 12 frames
>> (XEN) [2013-11-19 04:15:28] grant_table.c:1249:d12 Expanding dom (12) grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:15:40] grant_table.c:289:d0 Increased maptrack size to 13 frames
>> (XEN) [2013-11-19 04:15:47] grant_table.c:1249:d10 Expanding dom (10) grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:15:47] grant_table.c:1249:d10 Expanding dom (10) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:15:47] grant_table.c:289:d0 Increased maptrack size to 14 frames
>> (XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:15:52] grant_table.c:289:d0 Increased maptrack size to 15 frames
>> (XEN) [2013-11-19 04:15:52] grant_table.c:289:d0 Increased maptrack size to 16 frames
>> (XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:15:54] grant_table.c:289:d0 Increased maptrack size to 17 frames
>> (XEN) [2013-11-19 04:15:54] grant_table.c:289:d0 Increased maptrack size to 18 frames
>> (XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:15:56] grant_table.c:289:d0 Increased maptrack size to 19 frames
>> (XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:15:57] grant_table.c:289:d0 Increased maptrack size to 20 frames
>> (XEN) [2013-11-19 04:16:00] grant_table.c:289:d0 Increased maptrack size to 21 frames
>> (XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:16:00] grant_table.c:289:d0 Increased maptrack size to 22 frames
>> (XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 23 frames
>> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 24 frames
>> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 25 frames
>> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 26 frames
>> (XEN) [2013-11-19 04:16:06] grant_table.c:1249:d10 Expanding dom (10) grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:16:22] grant_table.c:1249:d12 Expanding dom (12) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:16:22] grant_table.c:1249:d12 Expanding dom (12) grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:16:22] grant_table.c:289:d0 Increased maptrack size to 27 frames
>> (XEN) [2013-11-19 04:16:23] grant_table.c:289:d0 Increased maptrack size to 28 frames
>> (XEN) [2013-11-19 04:16:24] grant_table.c:1249:d11 Expanding dom (11) grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:16:24] grant_table.c:1249:d11 Expanding dom (11) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:16:24] grant_table.c:289:d0 Increased maptrack size to 29 frames
>> (XEN) [2013-11-19 04:16:27] grant_table.c:1249:d7 Expanding dom (7) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:16:27] grant_table.c:1249:d7 Expanding dom (7) grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 05:15:04] grant_table.c:289:d0 Increased maptrack size to 30 frames
>> (XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 05:15:05] grant_table.c:289:d0 Increased maptrack size to 31 frames
>> (XEN) [2013-11-19 12:49:37] AMD-Vi: Share p2m table with iommu: p2m table = 0x52d3c5
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0
2013-11-19 13:02 xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0 Sander Eikelenboom
2013-11-19 13:23 ` Wei Liu
@ 2013-11-19 13:41 ` David Vrabel
2013-11-19 13:51 ` Wei Liu
1 sibling, 1 reply; 6+ messages in thread
From: David Vrabel @ 2013-11-19 13:41 UTC (permalink / raw)
To: Sander Eikelenboom; +Cc: Wei Liu, Ian Campbell, xen-devel
On 19/11/13 13:02, Sander Eikelenboom wrote:
> Hi Wei,
>
> I ran into the following problem when trying to boot another guest after less than a day of uptime.
> (the system started 15 guests at boot already which went fine). dom0 is allocated a fixed 1536M.
>
> Both host as pv guests run the same kernel, some hvm's run a slightly older kernel (3.9 f.e.)
>
> The are quite some granttable messages in xl dmesg, i also included these and a "vmstat -m"
Thread-per-VIF made the VIF's net device structure enormous.
This is fixed by 74d332c13b2148ae934ea94dac1745ae92efe8e5 (net: extend
net_device allocation to vmalloc()).
David
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0
2013-11-19 13:41 ` David Vrabel
@ 2013-11-19 13:51 ` Wei Liu
2013-11-19 14:17 ` Sander Eikelenboom
0 siblings, 1 reply; 6+ messages in thread
From: Wei Liu @ 2013-11-19 13:51 UTC (permalink / raw)
To: David Vrabel; +Cc: Sander Eikelenboom, Wei Liu, Ian Campbell, xen-devel
On Tue, Nov 19, 2013 at 01:41:12PM +0000, David Vrabel wrote:
> On 19/11/13 13:02, Sander Eikelenboom wrote:
> > Hi Wei,
> >
> > I ran into the following problem when trying to boot another guest after less than a day of uptime.
> > (the system started 15 guests at boot already which went fine). dom0 is allocated a fixed 1536M.
> >
> > Both host as pv guests run the same kernel, some hvm's run a slightly older kernel (3.9 f.e.)
> >
> > The are quite some granttable messages in xl dmesg, i also included these and a "vmstat -m"
>
> Thread-per-VIF made the VIF's net device structure enormous.
>
> This is fixed by 74d332c13b2148ae934ea94dac1745ae92efe8e5 (net: extend
> net_device allocation to vmalloc()).
>
Thanks David.
Sander, you can also try to cherry-pick the above patch to see if it
works.
Wei.
> David
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0
2013-11-19 13:51 ` Wei Liu
@ 2013-11-19 14:17 ` Sander Eikelenboom
0 siblings, 0 replies; 6+ messages in thread
From: Sander Eikelenboom @ 2013-11-19 14:17 UTC (permalink / raw)
To: Wei Liu; +Cc: xen-devel, David Vrabel, Ian Campbell
Tuesday, November 19, 2013, 2:51:54 PM, you wrote:
> On Tue, Nov 19, 2013 at 01:41:12PM +0000, David Vrabel wrote:
>> On 19/11/13 13:02, Sander Eikelenboom wrote:
>> > Hi Wei,
>> >
>> > I ran into the following problem when trying to boot another guest after less than a day of uptime.
>> > (the system started 15 guests at boot already which went fine). dom0 is allocated a fixed 1536M.
>> >
>> > Both host as pv guests run the same kernel, some hvm's run a slightly older kernel (3.9 f.e.)
>> >
>> > The are quite some granttable messages in xl dmesg, i also included these and a "vmstat -m"
>>
>> Thread-per-VIF made the VIF's net device structure enormous.
>>
>> This is fixed by 74d332c13b2148ae934ea94dac1745ae92efe8e5 (net: extend
>> net_device allocation to vmalloc()).
>>
Ah thanks, the merge window had some other surprises so i had to revert back to 3.12,
will try the cherry-pick .. thx for the pointer !
--
Sander
> Thanks David.
> Sander, you can also try to cherry-pick the above patch to see if it
> works.
> Wei.
>> David
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-11-19 14:17 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-11-19 13:02 xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0 Sander Eikelenboom
2013-11-19 13:23 ` Wei Liu
2013-11-19 13:40 ` Sander Eikelenboom
2013-11-19 13:41 ` David Vrabel
2013-11-19 13:51 ` Wei Liu
2013-11-19 14:17 ` Sander Eikelenboom
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).