* OOM when adding ipv6 route: How to make available more per-cpu memory?
@ 2010-11-05 17:19 Ben Greear
2010-11-05 18:06 ` Eric Dumazet
0 siblings, 1 reply; 17+ messages in thread
From: Ben Greear @ 2010-11-05 17:19 UTC (permalink / raw)
To: NetDev
We are testing 500 mac-vlans with IPv6 addresses on a 32-bit kernel
(2.6.36 + ubuntu patches + our patches) We have one
routing table per interface. It seems that some of them cannot
add routes due to lack of memory.
root@lanforge-ubuntu:/home/lanforge# ip route show table 29
root@lanforge-ubuntu:/home/lanforge# ./local/sbin/ip -6 route add default via 2001:98::1 dev eth12#15 table 29
RTNETLINK answers: Cannot allocate memory
I see errors such as this in the logs:
[507107.846864] PERCPU: allocation failed, size=2048 align=4, failed to allocate new chunk
[507107.846867] Pid: 3246, comm: ip Tainted: P 2.6.36-1-ct #7
[507107.846869] Call Trace:
[507107.846874] [<c05d4996>] ? printk+0x2d/0x2f
[507107.846878] [<c021405e>] pcpu_alloc+0x32e/0x360
[507107.846880] [<c02140bf>] __alloc_percpu+0xf/0x20
[507107.846883] [<c0558d6d>] snmp_mib_init+0x3d/0x70
[507107.846885] [<c0585692>] ipv6_add_dev+0x132/0x350
[507107.846887] [<c0557f39>] ? inetdev_init+0xb9/0x180
[507107.846889] [<c05895ff>] addrconf_notify+0x3f/0x490
[507107.846891] [<c055884f>] ? inetdev_event+0x20f/0x280
[507107.846894] [<c05daef3>] notifier_call_chain+0x43/0x60
[507107.846897] [<c016df4f>] raw_notifier_call_chain+0x1f/0x30
[507107.846899] [<c05008ec>] call_netdevice_notifiers+0x2c/0x60
[507107.846901] [<c0502ecc>] register_netdevice+0x23c/0x380
[507107.846907] [<f80aecf7>] macvlan_common_newlink+0x187/0x2a0 [macvlan]
[507107.846910] [<f80ae540>] ? macvlan_setup+0x0/0x20 [macvlan]
[507107.846912] [<f80aee37>] macvlan_newlink+0x27/0x30 [macvlan]
[507107.846914] [<c0503060>] ? netif_rx+0x0/0x120
[507107.846915] [<c05033c0>] ? dev_forward_skb+0x0/0xf0
[507107.846917] [<f80aee10>] ? macvlan_newlink+0x0/0x30 [macvlan]
[507107.846920] [<c050ea94>] rtnl_newlink+0x464/0x590
[507107.846921] [<c050e7e6>] ? rtnl_newlink+0x1b6/0x590
[507107.846927] [<c050cd2d>] rtnetlink_rcv_msg+0x13d/0x230
[507107.846929] [<c05d721f>] ? _raw_spin_lock_irqsave+0x2f/0x50
[507107.846931] [<c050e630>] ? rtnl_newlink+0x0/0x590
[507107.846933] [<c050cbf0>] ? rtnetlink_rcv_msg+0x0/0x230
[507107.846935] [<c0522276>] netlink_rcv_skb+0x86/0xb0
[507107.846936] [<c050cbdc>] rtnetlink_rcv+0x1c/0x30
[507107.846938] [<c0521f69>] netlink_unicast+0x259/0x280
[507107.846940] [<c0522ba8>] netlink_sendmsg+0x1f8/0x300
[507107.846943] [<c04f0b79>] sock_sendmsg+0xd9/0x100
[507107.846945] [<c013e0dd>] ? finish_task_switch+0x3d/0xc0
[507107.846947] [<c04f0b79>] ? sock_sendmsg+0xd9/0x100
[507107.846950] [<c013361d>] ? kmap_atomic_prot+0xcd/0xf0
[507107.846957] [<f88bd559>] ? h_d_revalidate+0xe9/0x240 [aufs]
[507107.846958] [<c013361d>] ? kmap_atomic_prot+0xcd/0xf0
[507107.846962] [<c0364c7d>] ? _copy_from_user+0x3d/0x130
[507107.846965] [<c04f947a>] ? verify_iovec+0x5a/0xa0
[507107.846966] [<c04f112d>] sys_sendmsg+0x15d/0x290
[507107.846972] [<f88c05f1>] ? aufs_fault+0xf1/0x110 [aufs]
[507107.846974] [<c013361d>] ? kmap_atomic_prot+0xcd/0xf0
[507107.846977] [<c01f8726>] ? handle_mm_fault+0x146/0x400
[507107.846979] [<c05dac0d>] ? do_page_fault+0x1cd/0x470
[507107.846981] [<c023419d>] ? alloc_fd+0xbd/0xf0
[507107.846983] [<c0364c7d>] ? _copy_from_user+0x3d/0x130
[507107.846986] [<c04f183b>] sys_socketcall+0xeb/0x2a0
[507107.846989] [<c0151d99>] ? irq_exit+0x39/0x70
[507107.846991] [<c021bb2e>] ? sys_open+0x2e/0x40
[507107.846993] [<c05d77a4>] syscall_call+0x7/0xb
But, it appears to me that we have plenty of memory available.
top - 10:09:13 up 6 days, 15:56, 9 users, load average: 0.37, 0.39, 0.45
Tasks: 211 total, 2 running, 209 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.7%us, 2.1%sy, 0.0%ni, 97.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3604332k total, 1510252k used, 2094080k free, 84928k buffers
Swap: 154620k total, 0k used, 154620k free, 1045912k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
root@lanforge-ubuntu:/home/lanforge# cat /proc/meminfo
MemTotal: 3604332 kB
MemFree: 2094568 kB
Buffers: 84928 kB
Cached: 1045920 kB
SwapCached: 0 kB
Active: 466784 kB
Inactive: 805632 kB
Active(anon): 148988 kB
Inactive(anon): 6788 kB
Active(file): 317796 kB
Inactive(file): 798844 kB
Unevictable: 0 kB
Mlocked: 0 kB
HighTotal: 2752008 kB
HighFree: 1512380 kB
LowTotal: 852324 kB
LowFree: 582188 kB
SwapTotal: 154620 kB
SwapFree: 154620 kB
Dirty: 492 kB
Writeback: 0 kB
AnonPages: 141316 kB
Mapped: 39428 kB
Shmem: 14208 kB
Slab: 170788 kB
SReclaimable: 81904 kB
SUnreclaim: 88884 kB
KernelStack: 2520 kB
PageTables: 4516 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 1956784 kB
Committed_AS: 582604 kB
VmallocTotal: 122880 kB
VmallocUsed: 48592 kB
VmallocChunk: 24280 kB
HardwareCorrupted: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 4096 kB
DirectMap4k: 12280 kB
DirectMap4M: 897024 kB
Is there any way to tune the system so that it has more memory available
for per-cpu data structures?
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-05 17:19 OOM when adding ipv6 route: How to make available more per-cpu memory? Ben Greear
@ 2010-11-05 18:06 ` Eric Dumazet
2010-11-05 18:15 ` Ben Greear
0 siblings, 1 reply; 17+ messages in thread
From: Eric Dumazet @ 2010-11-05 18:06 UTC (permalink / raw)
To: Ben Greear; +Cc: NetDev, linux-kernel, Tejun Heo
Le vendredi 05 novembre 2010 à 10:19 -0700, Ben Greear a écrit :
> We are testing 500 mac-vlans with IPv6 addresses on a 32-bit kernel
> (2.6.36 + ubuntu patches + our patches) We have one
> routing table per interface. It seems that some of them cannot
> add routes due to lack of memory.
>
> root@lanforge-ubuntu:/home/lanforge# ip route show table 29
> root@lanforge-ubuntu:/home/lanforge# ./local/sbin/ip -6 route add default via 2001:98::1 dev eth12#15 table 29
> RTNETLINK answers: Cannot allocate memory
>
> I see errors such as this in the logs:
>
> [507107.846864] PERCPU: allocation failed, size=2048 align=4, failed to allocate new chunk
> [507107.846867] Pid: 3246, comm: ip Tainted: P 2.6.36-1-ct #7
> [507107.846869] Call Trace:
> [507107.846874] [<c05d4996>] ? printk+0x2d/0x2f
> [507107.846878] [<c021405e>] pcpu_alloc+0x32e/0x360
> [507107.846880] [<c02140bf>] __alloc_percpu+0xf/0x20
> [507107.846883] [<c0558d6d>] snmp_mib_init+0x3d/0x70
> [507107.846885] [<c0585692>] ipv6_add_dev+0x132/0x350
> [507107.846887] [<c0557f39>] ? inetdev_init+0xb9/0x180
> [507107.846889] [<c05895ff>] addrconf_notify+0x3f/0x490
> [507107.846891] [<c055884f>] ? inetdev_event+0x20f/0x280
> [507107.846894] [<c05daef3>] notifier_call_chain+0x43/0x60
> [507107.846897] [<c016df4f>] raw_notifier_call_chain+0x1f/0x30
> [507107.846899] [<c05008ec>] call_netdevice_notifiers+0x2c/0x60
> [507107.846901] [<c0502ecc>] register_netdevice+0x23c/0x380
> [507107.846907] [<f80aecf7>] macvlan_common_newlink+0x187/0x2a0 [macvlan]
> [507107.846910] [<f80ae540>] ? macvlan_setup+0x0/0x20 [macvlan]
> [507107.846912] [<f80aee37>] macvlan_newlink+0x27/0x30 [macvlan]
> [507107.846914] [<c0503060>] ? netif_rx+0x0/0x120
> [507107.846915] [<c05033c0>] ? dev_forward_skb+0x0/0xf0
> [507107.846917] [<f80aee10>] ? macvlan_newlink+0x0/0x30 [macvlan]
> [507107.846920] [<c050ea94>] rtnl_newlink+0x464/0x590
> [507107.846921] [<c050e7e6>] ? rtnl_newlink+0x1b6/0x590
> [507107.846927] [<c050cd2d>] rtnetlink_rcv_msg+0x13d/0x230
> [507107.846929] [<c05d721f>] ? _raw_spin_lock_irqsave+0x2f/0x50
> [507107.846931] [<c050e630>] ? rtnl_newlink+0x0/0x590
> [507107.846933] [<c050cbf0>] ? rtnetlink_rcv_msg+0x0/0x230
> [507107.846935] [<c0522276>] netlink_rcv_skb+0x86/0xb0
> [507107.846936] [<c050cbdc>] rtnetlink_rcv+0x1c/0x30
> [507107.846938] [<c0521f69>] netlink_unicast+0x259/0x280
> [507107.846940] [<c0522ba8>] netlink_sendmsg+0x1f8/0x300
> [507107.846943] [<c04f0b79>] sock_sendmsg+0xd9/0x100
> [507107.846945] [<c013e0dd>] ? finish_task_switch+0x3d/0xc0
> [507107.846947] [<c04f0b79>] ? sock_sendmsg+0xd9/0x100
> [507107.846950] [<c013361d>] ? kmap_atomic_prot+0xcd/0xf0
> [507107.846957] [<f88bd559>] ? h_d_revalidate+0xe9/0x240 [aufs]
> [507107.846958] [<c013361d>] ? kmap_atomic_prot+0xcd/0xf0
> [507107.846962] [<c0364c7d>] ? _copy_from_user+0x3d/0x130
> [507107.846965] [<c04f947a>] ? verify_iovec+0x5a/0xa0
> [507107.846966] [<c04f112d>] sys_sendmsg+0x15d/0x290
> [507107.846972] [<f88c05f1>] ? aufs_fault+0xf1/0x110 [aufs]
> [507107.846974] [<c013361d>] ? kmap_atomic_prot+0xcd/0xf0
> [507107.846977] [<c01f8726>] ? handle_mm_fault+0x146/0x400
> [507107.846979] [<c05dac0d>] ? do_page_fault+0x1cd/0x470
> [507107.846981] [<c023419d>] ? alloc_fd+0xbd/0xf0
> [507107.846983] [<c0364c7d>] ? _copy_from_user+0x3d/0x130
> [507107.846986] [<c04f183b>] sys_socketcall+0xeb/0x2a0
> [507107.846989] [<c0151d99>] ? irq_exit+0x39/0x70
> [507107.846991] [<c021bb2e>] ? sys_open+0x2e/0x40
> [507107.846993] [<c05d77a4>] syscall_call+0x7/0xb
>
>
> But, it appears to me that we have plenty of memory available.
>
> top - 10:09:13 up 6 days, 15:56, 9 users, load average: 0.37, 0.39, 0.45
> Tasks: 211 total, 2 running, 209 sleeping, 0 stopped, 0 zombie
> Cpu(s): 0.7%us, 2.1%sy, 0.0%ni, 97.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Mem: 3604332k total, 1510252k used, 2094080k free, 84928k buffers
> Swap: 154620k total, 0k used, 154620k free, 1045912k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>
>
> root@lanforge-ubuntu:/home/lanforge# cat /proc/meminfo
> MemTotal: 3604332 kB
> MemFree: 2094568 kB
> Buffers: 84928 kB
> Cached: 1045920 kB
> SwapCached: 0 kB
> Active: 466784 kB
> Inactive: 805632 kB
> Active(anon): 148988 kB
> Inactive(anon): 6788 kB
> Active(file): 317796 kB
> Inactive(file): 798844 kB
> Unevictable: 0 kB
> Mlocked: 0 kB
> HighTotal: 2752008 kB
> HighFree: 1512380 kB
> LowTotal: 852324 kB
> LowFree: 582188 kB
> SwapTotal: 154620 kB
> SwapFree: 154620 kB
> Dirty: 492 kB
> Writeback: 0 kB
> AnonPages: 141316 kB
> Mapped: 39428 kB
> Shmem: 14208 kB
> Slab: 170788 kB
> SReclaimable: 81904 kB
> SUnreclaim: 88884 kB
> KernelStack: 2520 kB
> PageTables: 4516 kB
> NFS_Unstable: 0 kB
> Bounce: 0 kB
> WritebackTmp: 0 kB
> CommitLimit: 1956784 kB
> Committed_AS: 582604 kB
> VmallocTotal: 122880 kB
> VmallocUsed: 48592 kB
> VmallocChunk: 24280 kB
> HardwareCorrupted: 0 kB
> HugePages_Total: 0
> HugePages_Free: 0
> HugePages_Rsvd: 0
> HugePages_Surp: 0
> Hugepagesize: 4096 kB
> DirectMap4k: 12280 kB
> DirectMap4M: 897024 kB
>
>
> Is there any way to tune the system so that it has more memory available
> for per-cpu data structures?
CC linux-kernel and Tejun Heo
How many possible cpus do you have ?
head -1 /proc/interrupts
and please post :
cat /proc/vmallocinfo
Thanks
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-05 18:06 ` Eric Dumazet
@ 2010-11-05 18:15 ` Ben Greear
2010-11-05 20:20 ` Eric Dumazet
0 siblings, 1 reply; 17+ messages in thread
From: Ben Greear @ 2010-11-05 18:15 UTC (permalink / raw)
To: Eric Dumazet; +Cc: NetDev, linux-kernel, Tejun Heo
On 11/05/2010 11:06 AM, Eric Dumazet wrote:
> Le vendredi 05 novembre 2010 à 10:19 -0700, Ben Greear a écrit :
>> We are testing 500 mac-vlans with IPv6 addresses on a 32-bit kernel
>> (2.6.36 + ubuntu patches + our patches) We have one
>> routing table per interface. It seems that some of them cannot
>> add routes due to lack of memory.
>>
>> root@lanforge-ubuntu:/home/lanforge# ip route show table 29
>> root@lanforge-ubuntu:/home/lanforge# ./local/sbin/ip -6 route add default via 2001:98::1 dev eth12#15 table 29
>> RTNETLINK answers: Cannot allocate memory
>>
>> I see errors such as this in the logs:
>>
>> [507107.846864] PERCPU: allocation failed, size=2048 align=4, failed to allocate new chunk
>> [507107.846867] Pid: 3246, comm: ip Tainted: P 2.6.36-1-ct #7
>> [507107.846869] Call Trace:
>> [507107.846874] [<c05d4996>] ? printk+0x2d/0x2f
>> [507107.846878] [<c021405e>] pcpu_alloc+0x32e/0x360
>> [507107.846880] [<c02140bf>] __alloc_percpu+0xf/0x20
>> [507107.846883] [<c0558d6d>] snmp_mib_init+0x3d/0x70
>> [507107.846885] [<c0585692>] ipv6_add_dev+0x132/0x350
>> [507107.846887] [<c0557f39>] ? inetdev_init+0xb9/0x180
>> [507107.846889] [<c05895ff>] addrconf_notify+0x3f/0x490
>> [507107.846891] [<c055884f>] ? inetdev_event+0x20f/0x280
>> [507107.846894] [<c05daef3>] notifier_call_chain+0x43/0x60
>> [507107.846897] [<c016df4f>] raw_notifier_call_chain+0x1f/0x30
>> [507107.846899] [<c05008ec>] call_netdevice_notifiers+0x2c/0x60
>> [507107.846901] [<c0502ecc>] register_netdevice+0x23c/0x380
>> [507107.846907] [<f80aecf7>] macvlan_common_newlink+0x187/0x2a0 [macvlan]
>> [507107.846910] [<f80ae540>] ? macvlan_setup+0x0/0x20 [macvlan]
>> [507107.846912] [<f80aee37>] macvlan_newlink+0x27/0x30 [macvlan]
>> [507107.846914] [<c0503060>] ? netif_rx+0x0/0x120
>> [507107.846915] [<c05033c0>] ? dev_forward_skb+0x0/0xf0
>> [507107.846917] [<f80aee10>] ? macvlan_newlink+0x0/0x30 [macvlan]
>> [507107.846920] [<c050ea94>] rtnl_newlink+0x464/0x590
>> [507107.846921] [<c050e7e6>] ? rtnl_newlink+0x1b6/0x590
>> [507107.846927] [<c050cd2d>] rtnetlink_rcv_msg+0x13d/0x230
>> [507107.846929] [<c05d721f>] ? _raw_spin_lock_irqsave+0x2f/0x50
>> [507107.846931] [<c050e630>] ? rtnl_newlink+0x0/0x590
>> [507107.846933] [<c050cbf0>] ? rtnetlink_rcv_msg+0x0/0x230
>> [507107.846935] [<c0522276>] netlink_rcv_skb+0x86/0xb0
>> [507107.846936] [<c050cbdc>] rtnetlink_rcv+0x1c/0x30
>> [507107.846938] [<c0521f69>] netlink_unicast+0x259/0x280
>> [507107.846940] [<c0522ba8>] netlink_sendmsg+0x1f8/0x300
>> [507107.846943] [<c04f0b79>] sock_sendmsg+0xd9/0x100
>> [507107.846945] [<c013e0dd>] ? finish_task_switch+0x3d/0xc0
>> [507107.846947] [<c04f0b79>] ? sock_sendmsg+0xd9/0x100
>> [507107.846950] [<c013361d>] ? kmap_atomic_prot+0xcd/0xf0
>> [507107.846957] [<f88bd559>] ? h_d_revalidate+0xe9/0x240 [aufs]
>> [507107.846958] [<c013361d>] ? kmap_atomic_prot+0xcd/0xf0
>> [507107.846962] [<c0364c7d>] ? _copy_from_user+0x3d/0x130
>> [507107.846965] [<c04f947a>] ? verify_iovec+0x5a/0xa0
>> [507107.846966] [<c04f112d>] sys_sendmsg+0x15d/0x290
>> [507107.846972] [<f88c05f1>] ? aufs_fault+0xf1/0x110 [aufs]
>> [507107.846974] [<c013361d>] ? kmap_atomic_prot+0xcd/0xf0
>> [507107.846977] [<c01f8726>] ? handle_mm_fault+0x146/0x400
>> [507107.846979] [<c05dac0d>] ? do_page_fault+0x1cd/0x470
>> [507107.846981] [<c023419d>] ? alloc_fd+0xbd/0xf0
>> [507107.846983] [<c0364c7d>] ? _copy_from_user+0x3d/0x130
>> [507107.846986] [<c04f183b>] sys_socketcall+0xeb/0x2a0
>> [507107.846989] [<c0151d99>] ? irq_exit+0x39/0x70
>> [507107.846991] [<c021bb2e>] ? sys_open+0x2e/0x40
>> [507107.846993] [<c05d77a4>] syscall_call+0x7/0xb
>>
>>
>> But, it appears to me that we have plenty of memory available.
>>
>> top - 10:09:13 up 6 days, 15:56, 9 users, load average: 0.37, 0.39, 0.45
>> Tasks: 211 total, 2 running, 209 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 0.7%us, 2.1%sy, 0.0%ni, 97.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Mem: 3604332k total, 1510252k used, 2094080k free, 84928k buffers
>> Swap: 154620k total, 0k used, 154620k free, 1045912k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>>
>>
>> root@lanforge-ubuntu:/home/lanforge# cat /proc/meminfo
>> MemTotal: 3604332 kB
>> MemFree: 2094568 kB
>> Buffers: 84928 kB
>> Cached: 1045920 kB
>> SwapCached: 0 kB
>> Active: 466784 kB
>> Inactive: 805632 kB
>> Active(anon): 148988 kB
>> Inactive(anon): 6788 kB
>> Active(file): 317796 kB
>> Inactive(file): 798844 kB
>> Unevictable: 0 kB
>> Mlocked: 0 kB
>> HighTotal: 2752008 kB
>> HighFree: 1512380 kB
>> LowTotal: 852324 kB
>> LowFree: 582188 kB
>> SwapTotal: 154620 kB
>> SwapFree: 154620 kB
>> Dirty: 492 kB
>> Writeback: 0 kB
>> AnonPages: 141316 kB
>> Mapped: 39428 kB
>> Shmem: 14208 kB
>> Slab: 170788 kB
>> SReclaimable: 81904 kB
>> SUnreclaim: 88884 kB
>> KernelStack: 2520 kB
>> PageTables: 4516 kB
>> NFS_Unstable: 0 kB
>> Bounce: 0 kB
>> WritebackTmp: 0 kB
>> CommitLimit: 1956784 kB
>> Committed_AS: 582604 kB
>> VmallocTotal: 122880 kB
>> VmallocUsed: 48592 kB
>> VmallocChunk: 24280 kB
>> HardwareCorrupted: 0 kB
>> HugePages_Total: 0
>> HugePages_Free: 0
>> HugePages_Rsvd: 0
>> HugePages_Surp: 0
>> Hugepagesize: 4096 kB
>> DirectMap4k: 12280 kB
>> DirectMap4M: 897024 kB
>>
>>
>> Is there any way to tune the system so that it has more memory available
>> for per-cpu data structures?
>
> CC linux-kernel and Tejun Heo
>
> How many possible cpus do you have ?
> head -1 /proc/interrupts
oot@lanforge-ubuntu:/home/lanforge# head -1 /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
>
> and please post :
> cat /proc/vmallocinfo
root@lanforge-ubuntu:/home/lanforge# cat /proc/vmallocinfo
0xf7ffe000-0xf8000000 8192 hpet_enable+0x2d/0x1b8 phys=fed00000 ioremap
0xf8002000-0xf8004000 8192 acpi_os_map_memory+0x16/0x1f phys=df79e000 ioremap
0xf8004000-0xf8007000 12288 acpi_os_map_memory+0x16/0x1f phys=df7a0000 ioremap
0xf8008000-0xf800a000 8192 acpi_os_map_memory+0x16/0x1f phys=df790000 ioremap
0xf800b000-0xf8010000 20480 module_alloc+0x72/0x80 pages=4 vmalloc
0xf8010000-0xf8019000 36864 acpi_os_map_memory+0x16/0x1f phys=df790000 ioremap
0xf801a000-0xf801c000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xf801d000-0xf8020000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8023000-0xf8026000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
0xf8026000-0xf8028000 8192 msix_capability_init+0xae/0x2b0 phys=fa4fe000 ioremap
0xf8028000-0xf802a000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xf802b000-0xf802e000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf802f000-0xf8032000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8033000-0xf8036000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8037000-0xf803a000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf803b000-0xf803e000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf803f000-0xf8048000 36864 module_alloc+0x72/0x80 pages=8 vmalloc
0xf804b000-0xf8055000 40960 module_alloc+0x72/0x80 pages=9 vmalloc
0xf8056000-0xf8059000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf805b000-0xf8063000 32768 module_alloc+0x72/0x80 pages=7 vmalloc
0xf8066000-0xf8068000 8192 msix_capability_init+0xae/0x2b0 phys=fa4fa000 ioremap
0xf8068000-0xf8070000 32768 module_alloc+0x72/0x80 pages=7 vmalloc
0xf8072000-0xf8075000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
0xf8076000-0xf8083000 53248 module_alloc+0x72/0x80 pages=12 vmalloc
0xf8084000-0xf8087000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8088000-0xf808b000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf808c000-0xf808f000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8090000-0xf8093000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8094000-0xf8097000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8098000-0xf809b000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf809c000-0xf809f000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80a2000-0xf80ad000 45056 module_alloc+0x72/0x80 pages=10 vmalloc
0xf80ae000-0xf80b2000 16384 module_alloc+0x72/0x80 pages=3 vmalloc
0xf80b3000-0xf80b6000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80b7000-0xf80ba000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80bb000-0xf80be000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80bf000-0xf80c2000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80c3000-0xf80c6000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80c8000-0xf80cd000 20480 pci_iomap+0x81/0x90 phys=fa4fc000 ioremap
0xf80ce000-0xf80d1000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80d2000-0xf80d5000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80d6000-0xf80d9000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80da000-0xf80dd000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80de000-0xf80e1000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80e2000-0xf80e5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80e6000-0xf80e9000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80ea000-0xf80ed000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80ee000-0xf80f1000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80f2000-0xf80f5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80f6000-0xf80f9000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80fa000-0xf80fd000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf80fe000-0xf8101000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8102000-0xf8105000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8106000-0xf8109000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf810a000-0xf810d000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf810d000-0xf8118000 45056 module_alloc+0x72/0x80 pages=10 vmalloc
0xf8119000-0xf811c000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf811d000-0xf8120000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8121000-0xf8124000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8125000-0xf8128000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8129000-0xf812c000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf812d000-0xf8130000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8130000-0xf8132000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xf8133000-0xf8136000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
0xf8137000-0xf813a000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf813e000-0xf8140000 8192 msix_capability_init+0xae/0x2b0 phys=fa4f6000 ioremap
0xf8140000-0xf8145000 20480 pci_iomap+0x81/0x90 phys=fa4f8000 ioremap
0xf8146000-0xf8148000 8192 msix_capability_init+0xae/0x2b0 phys=fa4f2000 ioremap
0xf8148000-0xf814d000 20480 pci_iomap+0x81/0x90 phys=fa4f4000 ioremap
0xf814e000-0xf8150000 8192 msix_capability_init+0xae/0x2b0 phys=fa4ee000 ioremap
0xf8150000-0xf8155000 20480 pci_iomap+0x81/0x90 phys=fa4f0000 ioremap
0xf8156000-0xf8158000 8192 msix_capability_init+0xae/0x2b0 phys=fa4ea000 ioremap
0xf8158000-0xf815d000 20480 pci_iomap+0x81/0x90 phys=fa4ec000 ioremap
0xf815e000-0xf8160000 8192 msix_capability_init+0xae/0x2b0 phys=fa4e6000 ioremap
0xf8160000-0xf8165000 20480 pci_iomap+0x81/0x90 phys=fa4e8000 ioremap
0xf8166000-0xf8168000 8192 msix_capability_init+0xae/0x2b0 phys=fa4e2000 ioremap
0xf8168000-0xf816d000 20480 pci_iomap+0x81/0x90 phys=fa4e4000 ioremap
0xf816e000-0xf8170000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xf8170000-0xf8175000 20480 pci_iomap+0x81/0x90 phys=fa4e0000 ioremap
0xf8176000-0xf8179000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf817a000-0xf817d000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf817e000-0xf8181000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8182000-0xf8185000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8186000-0xf8189000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf818e000-0xf819d000 61440 module_alloc+0x72/0x80 pages=14 vmalloc
0xf819e000-0xf81a1000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf81a2000-0xf81a5000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf81a6000-0xf81a9000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf81aa000-0xf81ad000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf81ae000-0xf81b1000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf81b2000-0xf81b5000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf81b6000-0xf81b9000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf81ba000-0xf81bd000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf81be000-0xf81c1000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf81c2000-0xf81c5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf81fe000-0xf8201000 12288 e1000e_setup_tx_resources+0x27/0xb0 [e1000e] pages=2 vmalloc
0xf8202000-0xf8205000 12288 e1000e_setup_rx_resources+0x2a/0x120 [e1000e] pages=2 vmalloc
0xf8206000-0xf8209000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf820c000-0xf820f000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
0xf8210000-0xf8213000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8214000-0xf8217000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8218000-0xf821c000 16384 module_alloc+0x72/0x80 pages=3 vmalloc
0xf8225000-0xf8228000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8229000-0xf822b000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xf82ab000-0xf82ae000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82af000-0xf82b2000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82b3000-0xf82b6000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82b7000-0xf82ba000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82bb000-0xf82be000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82bf000-0xf82c2000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82c3000-0xf82c6000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82c7000-0xf82ca000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82cb000-0xf82ce000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82cf000-0xf82d2000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82d3000-0xf82d6000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82d7000-0xf82da000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82db000-0xf82de000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82df000-0xf82e2000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82e3000-0xf82e6000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82e7000-0xf82ea000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82eb000-0xf82ee000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf82ef000-0xf82f2000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8332000-0xf8363000 200704 module_alloc+0x72/0x80 pages=48 vmalloc
0xf8364000-0xf8367000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8368000-0xf836b000 12288 e1000e_setup_tx_resources+0x27/0xb0 [e1000e] pages=2 vmalloc
0xf836c000-0xf836f000 12288 e1000e_setup_rx_resources+0x2a/0x120 [e1000e] pages=2 vmalloc
0xf8370000-0xf8373000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8374000-0xf8377000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8378000-0xf837b000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf837c000-0xf837f000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf837f000-0xf83a0000 135168 module_alloc+0x72/0x80 pages=32 vmalloc
0xf83a1000-0xf83a4000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf83a5000-0xf83a8000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf83a9000-0xf83ac000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf83ad000-0xf83b0000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf83b0000-0xf83ba000 40960 module_alloc+0x72/0x80 pages=9 vmalloc
0xf83bb000-0xf83bd000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xf83be000-0xf83c1000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf83cc000-0xf83e5000 102400 module_alloc+0x72/0x80 pages=24 vmalloc
0xf83ec000-0xf83ee000 8192 msix_capability_init+0xae/0x2b0 phys=faedc000 ioremap
0xf83f0000-0xf83f2000 8192 msix_capability_init+0xae/0x2b0 phys=fa69c000 ioremap
0xf83fa000-0xf83fc000 8192 msix_capability_init+0xae/0x2b0 phys=fafdc000 ioremap
0xf83fe000-0xf8400000 8192 msix_capability_init+0xae/0x2b0 phys=fa698000 ioremap
0xf8400000-0xf8421000 135168 e1000_probe+0x206/0x9d0 [e1000e] phys=faee0000 ioremap
0xf8422000-0xf8424000 8192 msix_capability_init+0xae/0x2b0 phys=fa79c000 ioremap
0xf8426000-0xf8428000 8192 msix_capability_init+0xae/0x2b0 phys=fa798000 ioremap
0xf842a000-0xf842c000 8192 msix_capability_init+0xae/0x2b0 phys=fa99c000 ioremap
0xf842e000-0xf8430000 8192 msix_capability_init+0xae/0x2b0 phys=fa998000 ioremap
0xf8432000-0xf8434000 8192 msix_capability_init+0xae/0x2b0 phys=faa9c000 ioremap
0xf8436000-0xf8438000 8192 msix_capability_init+0xae/0x2b0 phys=faa98000 ioremap
0xf843a000-0xf843c000 8192 msix_capability_init+0xae/0x2b0 phys=fac9c000 ioremap
0xf843e000-0xf8440000 8192 msix_capability_init+0xae/0x2b0 phys=fac98000 ioremap
0xf8440000-0xf8461000 135168 igb_probe+0x1f5/0x87a [igb] phys=fa6e0000 ioremap
0xf8462000-0xf8464000 8192 msix_capability_init+0xae/0x2b0 phys=fad9c000 ioremap
0xf8466000-0xf8468000 8192 msix_capability_init+0xae/0x2b0 phys=fad98000 ioremap
0xf846c000-0xf846e000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xf8476000-0xf8478000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xf8480000-0xf84a1000 135168 e1000_probe+0x206/0x9d0 [e1000e] phys=fafe0000 ioremap
0xf84a9000-0xf84af000 24576 module_alloc+0x72/0x80 pages=5 vmalloc
0xf84bc000-0xf84bf000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf84c0000-0xf84e1000 135168 igb_probe+0x1f5/0x87a [igb] phys=fa660000 ioremap
0xf84e2000-0xf84e5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf84e6000-0xf84e9000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf84ea000-0xf84ed000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf84ee000-0xf84f1000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf84f2000-0xf84f5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf84f6000-0xf84f9000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf84fb000-0xf84ff000 16384 module_alloc+0x72/0x80 pages=3 vmalloc
0xf8500000-0xf8521000 135168 igb_probe+0x1f5/0x87a [igb] phys=fa7e0000 ioremap
0xf8522000-0xf852f000 53248 module_alloc+0x72/0x80 pages=12 vmalloc
0xf8530000-0xf8533000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8534000-0xf8537000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8538000-0xf853b000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf853c000-0xf853f000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8540000-0xf8561000 135168 igb_probe+0x1f5/0x87a [igb] phys=fa760000 ioremap
0xf8562000-0xf8565000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
0xf8566000-0xf8569000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf856a000-0xf856c000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xf856d000-0xf8570000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8571000-0xf8574000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8575000-0xf8578000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8579000-0xf857c000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf857d000-0xf857f000 8192 swap_cgroup_swapon+0x3c/0x140 pages=1 vmalloc
0xf8580000-0xf85a1000 135168 igb_probe+0x1f5/0x87a [igb] phys=fa9e0000 ioremap
0xf85a2000-0xf85a5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf85a6000-0xf85a9000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf85aa000-0xf85ad000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf85af000-0xf85b7000 32768 module_alloc+0x72/0x80 pages=7 vmalloc
0xf85b8000-0xf85bb000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf85bc000-0xf85bf000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf85c0000-0xf85e1000 135168 igb_probe+0x1f5/0x87a [igb] phys=fa960000 ioremap
0xf85e2000-0xf85ed000 45056 sys_swapon+0x5a6/0xa40 pages=10 vmalloc
0xf85ee000-0xf8600000 73728 module_alloc+0x72/0x80 pages=17 vmalloc
0xf8600000-0xf8621000 135168 igb_probe+0x1f5/0x87a [igb] phys=faae0000 ioremap
0xf8622000-0xf8625000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8626000-0xf8629000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf862a000-0xf862d000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf862e000-0xf8631000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8632000-0xf8635000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8636000-0xf8639000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf863a000-0xf863d000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8640000-0xf8661000 135168 igb_probe+0x1f5/0x87a [igb] phys=faa60000 ioremap
0xf8662000-0xf8665000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8666000-0xf8669000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf866a000-0xf866d000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf866e000-0xf8671000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8672000-0xf8675000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8676000-0xf8679000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf867a000-0xf867d000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8680000-0xf86a1000 135168 igb_probe+0x1f5/0x87a [igb] phys=face0000 ioremap
0xf86a2000-0xf86a5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf86a6000-0xf86a9000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf86aa000-0xf86ad000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf86ae000-0xf86b1000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf86b2000-0xf86b5000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf86b6000-0xf86b9000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf86ba000-0xf86bd000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf86c0000-0xf86e1000 135168 igb_probe+0x1f5/0x87a [igb] phys=fac60000 ioremap
0xf86e2000-0xf86e5000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8700000-0xf8721000 135168 igb_probe+0x1f5/0x87a [igb] phys=fade0000 ioremap
0xf8740000-0xf8761000 135168 igb_probe+0x1f5/0x87a [igb] phys=fad60000 ioremap
0xf8762000-0xf8767000 20480 module_alloc+0x72/0x80 pages=4 vmalloc
0xf8768000-0xf876b000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf876c000-0xf876f000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8770000-0xf8773000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8774000-0xf8777000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8778000-0xf877b000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf877c000-0xf877f000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8780000-0xf8783000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8784000-0xf8787000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8788000-0xf878b000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf878c000-0xf878f000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8790000-0xf8793000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8793000-0xf87a4000 69632 module_alloc+0x72/0x80 pages=16 vmalloc
0xf87a5000-0xf87a8000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf87aa000-0xf87e7000 249856 module_alloc+0x72/0x80 pages=60 vmalloc
0xf87f4000-0xf87f7000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf87f8000-0xf87fb000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf87fe000-0xf8877000 495616 module_alloc+0x72/0x80 pages=120 vmalloc
0xf8878000-0xf887b000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf887c000-0xf887f000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8880000-0xf8883000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8884000-0xf8887000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8888000-0xf888b000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf888c000-0xf888f000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8890000-0xf8893000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8894000-0xf8897000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf8898000-0xf889b000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf889c000-0xf889f000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf88a0000-0xf88a3000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf88a4000-0xf88a7000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf88a8000-0xf88ab000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf88ac000-0xf88af000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xf88af000-0xf88d5000 155648 module_alloc+0x72/0x80 pages=37 vmalloc
0xf8a65000-0xf8a67000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfa026000-0xfa028000 8192 acpi_os_map_memory+0x16/0x1f phys=df79e000 ioremap
0xfa02a000-0xfa02c000 8192 acpi_os_map_memory+0x16/0x1f phys=df790000 ioremap
0xfa03a000-0xfa03c000 8192 acpi_os_map_memory+0x16/0x1f phys=df790000 ioremap
0xfa03e000-0xfa040000 8192 acpi_os_map_memory+0x16/0x1f phys=df790000 ioremap
0xfa042000-0xfa044000 8192 acpi_os_map_memory+0x16/0x1f phys=df79e000 ioremap
0xfa046000-0xfa048000 8192 acpi_os_map_memory+0x16/0x1f phys=df79a000 ioremap
0xfa04a000-0xfa04c000 8192 acpi_os_map_memory+0x16/0x1f phys=df79a000 ioremap
0xfa04e000-0xfa050000 8192 acpi_os_map_memory+0x16/0x1f phys=df79a000 ioremap
0xfa052000-0xfa054000 8192 acpi_os_map_memory+0x16/0x1f phys=df79a000 ioremap
0xfa056000-0xfa058000 8192 acpi_os_map_memory+0x16/0x1f phys=df79a000 ioremap
0xfa05e000-0xfa060000 8192 acpi_os_map_memory+0x16/0x1f phys=fed1f000 ioremap
0xfa06e000-0xfa070000 8192 usb_hcd_pci_probe+0x17b/0x350 phys=fa4de000 ioremap
0xfa072000-0xfa074000 8192 usb_hcd_pci_probe+0x17b/0x350 phys=fa4dc000 ioremap
0xfa78c000-0xfa79d000 69632 module_alloc+0x72/0x80 pages=16 vmalloc
0xfa805000-0xfa84a000 282624 module_alloc+0x72/0x80 pages=68 vmalloc
0xfc000000-0xfc400000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
0xfc400000-0xfc800000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
0xfc93f000-0xfc941000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfc94b000-0xfc94e000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
0xfc95e000-0xfc964000 24576 module_alloc+0x72/0x80 pages=5 vmalloc
0xfc96f000-0xfc972000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
0xfcdcc000-0xfcddf000 77824 module_alloc+0x72/0x80 pages=18 vmalloc
0xfce11000-0xfce14000 12288 reqsk_queue_alloc+0x54/0xd0 pages=2 vmalloc
0xfce15000-0xfce18000 12288 reqsk_queue_alloc+0x54/0xd0 pages=2 vmalloc
0xfce19000-0xfce1c000 12288 reqsk_queue_alloc+0x54/0xd0 pages=2 vmalloc
0xfce21000-0xfce23000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfce2a000-0xfce2c000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfce30000-0xfce33000 12288 reqsk_queue_alloc+0x54/0xd0 pages=2 vmalloc
0xfce34000-0xfce37000 12288 reqsk_queue_alloc+0x54/0xd0 pages=2 vmalloc
0xfce38000-0xfce3b000 12288 reqsk_queue_alloc+0x54/0xd0 pages=2 vmalloc
0xfd0e7000-0xfd0ea000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd0eb000-0xfd0ee000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd0ef000-0xfd0f2000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd0f3000-0xfd0f6000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd0f7000-0xfd0fa000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd0fb000-0xfd0fe000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd0ff000-0xfd102000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd103000-0xfd106000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd107000-0xfd10a000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd10b000-0xfd10e000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd10f000-0xfd112000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd113000-0xfd116000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd117000-0xfd11a000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd11b000-0xfd11e000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd11f000-0xfd122000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd123000-0xfd126000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd127000-0xfd12a000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd12b000-0xfd12e000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd12f000-0xfd132000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd133000-0xfd136000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd137000-0xfd13a000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd13b000-0xfd13e000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd13f000-0xfd142000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd143000-0xfd146000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd147000-0xfd14a000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd14b000-0xfd14e000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd14f000-0xfd152000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd153000-0xfd156000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd157000-0xfd15a000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd15b000-0xfd15e000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd15f000-0xfd162000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd163000-0xfd166000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
0xfd1b2000-0xfd1b6000 16384 module_alloc+0x72/0x80 pages=3 vmalloc
0xfd1bf000-0xfd1c1000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfd1cb000-0xfd1ce000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
0xfd1d5000-0xfd1d7000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfd1de000-0xfd1e0000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfd1e8000-0xfd1ea000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfd1f2000-0xfd1f4000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfd1fb000-0xfd1fd000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfd205000-0xfd207000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfd20f000-0xfd211000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfd219000-0xfd21b000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfd223000-0xfd225000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfd22d000-0xfd22f000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfd23a000-0xfd23c000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
0xfd248000-0xfd24c000 16384 module_alloc+0x72/0x80 pages=3 vmalloc
0xfd400000-0xfd800000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
0xfd800000-0xfdc00000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
0xfdc00000-0xfe000000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
0xfe000000-0xfe400000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
0xfe400000-0xfe800000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
0xfe800000-0xfec00000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
0xfec00000-0xff000000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
0xff000000-0xff400000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-05 18:15 ` Ben Greear
@ 2010-11-05 20:20 ` Eric Dumazet
2010-11-05 20:26 ` Ben Greear
2010-11-05 22:11 ` Eric Dumazet
0 siblings, 2 replies; 17+ messages in thread
From: Eric Dumazet @ 2010-11-05 20:20 UTC (permalink / raw)
To: Ben Greear; +Cc: NetDev, linux-kernel, Tejun Heo
Le vendredi 05 novembre 2010 à 11:15 -0700, Ben Greear a écrit :
> root@lanforge-ubuntu:/home/lanforge# cat /proc/vmallocinfo
> 0xf7ffe000-0xf8000000 8192 hpet_enable+0x2d/0x1b8 phys=fed00000 ioremap
> 0xf8002000-0xf8004000 8192 acpi_os_map_memory+0x16/0x1f phys=df79e000 ioremap
> 0xf8004000-0xf8007000 12288 acpi_os_map_memory+0x16/0x1f phys=df7a0000 ioremap
> 0xf8008000-0xf800a000 8192 acpi_os_map_memory+0x16/0x1f phys=df790000 ioremap
> 0xf800b000-0xf8010000 20480 module_alloc+0x72/0x80 pages=4 vmalloc
> 0xf8010000-0xf8019000 36864 acpi_os_map_memory+0x16/0x1f phys=df790000 ioremap
> 0xf801a000-0xf801c000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xf801d000-0xf8020000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8023000-0xf8026000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
> 0xf8026000-0xf8028000 8192 msix_capability_init+0xae/0x2b0 phys=fa4fe000 ioremap
> 0xf8028000-0xf802a000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xf802b000-0xf802e000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf802f000-0xf8032000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8033000-0xf8036000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8037000-0xf803a000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf803b000-0xf803e000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf803f000-0xf8048000 36864 module_alloc+0x72/0x80 pages=8 vmalloc
> 0xf804b000-0xf8055000 40960 module_alloc+0x72/0x80 pages=9 vmalloc
> 0xf8056000-0xf8059000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf805b000-0xf8063000 32768 module_alloc+0x72/0x80 pages=7 vmalloc
> 0xf8066000-0xf8068000 8192 msix_capability_init+0xae/0x2b0 phys=fa4fa000 ioremap
> 0xf8068000-0xf8070000 32768 module_alloc+0x72/0x80 pages=7 vmalloc
> 0xf8072000-0xf8075000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
> 0xf8076000-0xf8083000 53248 module_alloc+0x72/0x80 pages=12 vmalloc
> 0xf8084000-0xf8087000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8088000-0xf808b000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf808c000-0xf808f000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8090000-0xf8093000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8094000-0xf8097000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8098000-0xf809b000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf809c000-0xf809f000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80a2000-0xf80ad000 45056 module_alloc+0x72/0x80 pages=10 vmalloc
> 0xf80ae000-0xf80b2000 16384 module_alloc+0x72/0x80 pages=3 vmalloc
> 0xf80b3000-0xf80b6000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80b7000-0xf80ba000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80bb000-0xf80be000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80bf000-0xf80c2000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80c3000-0xf80c6000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80c8000-0xf80cd000 20480 pci_iomap+0x81/0x90 phys=fa4fc000 ioremap
> 0xf80ce000-0xf80d1000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80d2000-0xf80d5000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80d6000-0xf80d9000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80da000-0xf80dd000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80de000-0xf80e1000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80e2000-0xf80e5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80e6000-0xf80e9000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80ea000-0xf80ed000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80ee000-0xf80f1000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80f2000-0xf80f5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80f6000-0xf80f9000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80fa000-0xf80fd000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf80fe000-0xf8101000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8102000-0xf8105000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8106000-0xf8109000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf810a000-0xf810d000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf810d000-0xf8118000 45056 module_alloc+0x72/0x80 pages=10 vmalloc
> 0xf8119000-0xf811c000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf811d000-0xf8120000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8121000-0xf8124000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8125000-0xf8128000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8129000-0xf812c000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf812d000-0xf8130000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8130000-0xf8132000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xf8133000-0xf8136000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
> 0xf8137000-0xf813a000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf813e000-0xf8140000 8192 msix_capability_init+0xae/0x2b0 phys=fa4f6000 ioremap
> 0xf8140000-0xf8145000 20480 pci_iomap+0x81/0x90 phys=fa4f8000 ioremap
> 0xf8146000-0xf8148000 8192 msix_capability_init+0xae/0x2b0 phys=fa4f2000 ioremap
> 0xf8148000-0xf814d000 20480 pci_iomap+0x81/0x90 phys=fa4f4000 ioremap
> 0xf814e000-0xf8150000 8192 msix_capability_init+0xae/0x2b0 phys=fa4ee000 ioremap
> 0xf8150000-0xf8155000 20480 pci_iomap+0x81/0x90 phys=fa4f0000 ioremap
> 0xf8156000-0xf8158000 8192 msix_capability_init+0xae/0x2b0 phys=fa4ea000 ioremap
> 0xf8158000-0xf815d000 20480 pci_iomap+0x81/0x90 phys=fa4ec000 ioremap
> 0xf815e000-0xf8160000 8192 msix_capability_init+0xae/0x2b0 phys=fa4e6000 ioremap
> 0xf8160000-0xf8165000 20480 pci_iomap+0x81/0x90 phys=fa4e8000 ioremap
> 0xf8166000-0xf8168000 8192 msix_capability_init+0xae/0x2b0 phys=fa4e2000 ioremap
> 0xf8168000-0xf816d000 20480 pci_iomap+0x81/0x90 phys=fa4e4000 ioremap
> 0xf816e000-0xf8170000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xf8170000-0xf8175000 20480 pci_iomap+0x81/0x90 phys=fa4e0000 ioremap
> 0xf8176000-0xf8179000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf817a000-0xf817d000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf817e000-0xf8181000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8182000-0xf8185000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8186000-0xf8189000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf818e000-0xf819d000 61440 module_alloc+0x72/0x80 pages=14 vmalloc
> 0xf819e000-0xf81a1000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf81a2000-0xf81a5000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf81a6000-0xf81a9000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf81aa000-0xf81ad000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf81ae000-0xf81b1000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf81b2000-0xf81b5000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf81b6000-0xf81b9000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf81ba000-0xf81bd000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf81be000-0xf81c1000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf81c2000-0xf81c5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf81fe000-0xf8201000 12288 e1000e_setup_tx_resources+0x27/0xb0 [e1000e] pages=2 vmalloc
> 0xf8202000-0xf8205000 12288 e1000e_setup_rx_resources+0x2a/0x120 [e1000e] pages=2 vmalloc
> 0xf8206000-0xf8209000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf820c000-0xf820f000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
> 0xf8210000-0xf8213000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8214000-0xf8217000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8218000-0xf821c000 16384 module_alloc+0x72/0x80 pages=3 vmalloc
> 0xf8225000-0xf8228000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8229000-0xf822b000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xf82ab000-0xf82ae000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82af000-0xf82b2000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82b3000-0xf82b6000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82b7000-0xf82ba000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82bb000-0xf82be000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82bf000-0xf82c2000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82c3000-0xf82c6000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82c7000-0xf82ca000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82cb000-0xf82ce000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82cf000-0xf82d2000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82d3000-0xf82d6000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82d7000-0xf82da000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82db000-0xf82de000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82df000-0xf82e2000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82e3000-0xf82e6000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82e7000-0xf82ea000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82eb000-0xf82ee000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf82ef000-0xf82f2000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8332000-0xf8363000 200704 module_alloc+0x72/0x80 pages=48 vmalloc
> 0xf8364000-0xf8367000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8368000-0xf836b000 12288 e1000e_setup_tx_resources+0x27/0xb0 [e1000e] pages=2 vmalloc
> 0xf836c000-0xf836f000 12288 e1000e_setup_rx_resources+0x2a/0x120 [e1000e] pages=2 vmalloc
> 0xf8370000-0xf8373000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8374000-0xf8377000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8378000-0xf837b000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf837c000-0xf837f000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf837f000-0xf83a0000 135168 module_alloc+0x72/0x80 pages=32 vmalloc
> 0xf83a1000-0xf83a4000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf83a5000-0xf83a8000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf83a9000-0xf83ac000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf83ad000-0xf83b0000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf83b0000-0xf83ba000 40960 module_alloc+0x72/0x80 pages=9 vmalloc
> 0xf83bb000-0xf83bd000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xf83be000-0xf83c1000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf83cc000-0xf83e5000 102400 module_alloc+0x72/0x80 pages=24 vmalloc
> 0xf83ec000-0xf83ee000 8192 msix_capability_init+0xae/0x2b0 phys=faedc000 ioremap
> 0xf83f0000-0xf83f2000 8192 msix_capability_init+0xae/0x2b0 phys=fa69c000 ioremap
> 0xf83fa000-0xf83fc000 8192 msix_capability_init+0xae/0x2b0 phys=fafdc000 ioremap
> 0xf83fe000-0xf8400000 8192 msix_capability_init+0xae/0x2b0 phys=fa698000 ioremap
> 0xf8400000-0xf8421000 135168 e1000_probe+0x206/0x9d0 [e1000e] phys=faee0000 ioremap
> 0xf8422000-0xf8424000 8192 msix_capability_init+0xae/0x2b0 phys=fa79c000 ioremap
> 0xf8426000-0xf8428000 8192 msix_capability_init+0xae/0x2b0 phys=fa798000 ioremap
> 0xf842a000-0xf842c000 8192 msix_capability_init+0xae/0x2b0 phys=fa99c000 ioremap
> 0xf842e000-0xf8430000 8192 msix_capability_init+0xae/0x2b0 phys=fa998000 ioremap
> 0xf8432000-0xf8434000 8192 msix_capability_init+0xae/0x2b0 phys=faa9c000 ioremap
> 0xf8436000-0xf8438000 8192 msix_capability_init+0xae/0x2b0 phys=faa98000 ioremap
> 0xf843a000-0xf843c000 8192 msix_capability_init+0xae/0x2b0 phys=fac9c000 ioremap
> 0xf843e000-0xf8440000 8192 msix_capability_init+0xae/0x2b0 phys=fac98000 ioremap
> 0xf8440000-0xf8461000 135168 igb_probe+0x1f5/0x87a [igb] phys=fa6e0000 ioremap
> 0xf8462000-0xf8464000 8192 msix_capability_init+0xae/0x2b0 phys=fad9c000 ioremap
> 0xf8466000-0xf8468000 8192 msix_capability_init+0xae/0x2b0 phys=fad98000 ioremap
> 0xf846c000-0xf846e000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xf8476000-0xf8478000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xf8480000-0xf84a1000 135168 e1000_probe+0x206/0x9d0 [e1000e] phys=fafe0000 ioremap
> 0xf84a9000-0xf84af000 24576 module_alloc+0x72/0x80 pages=5 vmalloc
> 0xf84bc000-0xf84bf000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf84c0000-0xf84e1000 135168 igb_probe+0x1f5/0x87a [igb] phys=fa660000 ioremap
> 0xf84e2000-0xf84e5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf84e6000-0xf84e9000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf84ea000-0xf84ed000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf84ee000-0xf84f1000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf84f2000-0xf84f5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf84f6000-0xf84f9000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf84fb000-0xf84ff000 16384 module_alloc+0x72/0x80 pages=3 vmalloc
> 0xf8500000-0xf8521000 135168 igb_probe+0x1f5/0x87a [igb] phys=fa7e0000 ioremap
> 0xf8522000-0xf852f000 53248 module_alloc+0x72/0x80 pages=12 vmalloc
> 0xf8530000-0xf8533000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8534000-0xf8537000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8538000-0xf853b000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf853c000-0xf853f000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8540000-0xf8561000 135168 igb_probe+0x1f5/0x87a [igb] phys=fa760000 ioremap
> 0xf8562000-0xf8565000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
> 0xf8566000-0xf8569000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf856a000-0xf856c000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xf856d000-0xf8570000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8571000-0xf8574000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8575000-0xf8578000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8579000-0xf857c000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf857d000-0xf857f000 8192 swap_cgroup_swapon+0x3c/0x140 pages=1 vmalloc
> 0xf8580000-0xf85a1000 135168 igb_probe+0x1f5/0x87a [igb] phys=fa9e0000 ioremap
> 0xf85a2000-0xf85a5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf85a6000-0xf85a9000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf85aa000-0xf85ad000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf85af000-0xf85b7000 32768 module_alloc+0x72/0x80 pages=7 vmalloc
> 0xf85b8000-0xf85bb000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf85bc000-0xf85bf000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf85c0000-0xf85e1000 135168 igb_probe+0x1f5/0x87a [igb] phys=fa960000 ioremap
> 0xf85e2000-0xf85ed000 45056 sys_swapon+0x5a6/0xa40 pages=10 vmalloc
> 0xf85ee000-0xf8600000 73728 module_alloc+0x72/0x80 pages=17 vmalloc
> 0xf8600000-0xf8621000 135168 igb_probe+0x1f5/0x87a [igb] phys=faae0000 ioremap
> 0xf8622000-0xf8625000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8626000-0xf8629000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf862a000-0xf862d000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf862e000-0xf8631000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8632000-0xf8635000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8636000-0xf8639000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf863a000-0xf863d000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8640000-0xf8661000 135168 igb_probe+0x1f5/0x87a [igb] phys=faa60000 ioremap
> 0xf8662000-0xf8665000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8666000-0xf8669000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf866a000-0xf866d000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf866e000-0xf8671000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8672000-0xf8675000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8676000-0xf8679000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf867a000-0xf867d000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8680000-0xf86a1000 135168 igb_probe+0x1f5/0x87a [igb] phys=face0000 ioremap
> 0xf86a2000-0xf86a5000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf86a6000-0xf86a9000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf86aa000-0xf86ad000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf86ae000-0xf86b1000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf86b2000-0xf86b5000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf86b6000-0xf86b9000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf86ba000-0xf86bd000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf86c0000-0xf86e1000 135168 igb_probe+0x1f5/0x87a [igb] phys=fac60000 ioremap
> 0xf86e2000-0xf86e5000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8700000-0xf8721000 135168 igb_probe+0x1f5/0x87a [igb] phys=fade0000 ioremap
> 0xf8740000-0xf8761000 135168 igb_probe+0x1f5/0x87a [igb] phys=fad60000 ioremap
> 0xf8762000-0xf8767000 20480 module_alloc+0x72/0x80 pages=4 vmalloc
> 0xf8768000-0xf876b000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf876c000-0xf876f000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8770000-0xf8773000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8774000-0xf8777000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8778000-0xf877b000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf877c000-0xf877f000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8780000-0xf8783000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8784000-0xf8787000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8788000-0xf878b000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf878c000-0xf878f000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8790000-0xf8793000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8793000-0xf87a4000 69632 module_alloc+0x72/0x80 pages=16 vmalloc
> 0xf87a5000-0xf87a8000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf87aa000-0xf87e7000 249856 module_alloc+0x72/0x80 pages=60 vmalloc
> 0xf87f4000-0xf87f7000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf87f8000-0xf87fb000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf87fe000-0xf8877000 495616 module_alloc+0x72/0x80 pages=120 vmalloc
> 0xf8878000-0xf887b000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf887c000-0xf887f000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8880000-0xf8883000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8884000-0xf8887000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8888000-0xf888b000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf888c000-0xf888f000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8890000-0xf8893000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8894000-0xf8897000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf8898000-0xf889b000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf889c000-0xf889f000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf88a0000-0xf88a3000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf88a4000-0xf88a7000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf88a8000-0xf88ab000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf88ac000-0xf88af000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xf88af000-0xf88d5000 155648 module_alloc+0x72/0x80 pages=37 vmalloc
> 0xf8a65000-0xf8a67000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfa026000-0xfa028000 8192 acpi_os_map_memory+0x16/0x1f phys=df79e000 ioremap
> 0xfa02a000-0xfa02c000 8192 acpi_os_map_memory+0x16/0x1f phys=df790000 ioremap
> 0xfa03a000-0xfa03c000 8192 acpi_os_map_memory+0x16/0x1f phys=df790000 ioremap
> 0xfa03e000-0xfa040000 8192 acpi_os_map_memory+0x16/0x1f phys=df790000 ioremap
> 0xfa042000-0xfa044000 8192 acpi_os_map_memory+0x16/0x1f phys=df79e000 ioremap
> 0xfa046000-0xfa048000 8192 acpi_os_map_memory+0x16/0x1f phys=df79a000 ioremap
> 0xfa04a000-0xfa04c000 8192 acpi_os_map_memory+0x16/0x1f phys=df79a000 ioremap
> 0xfa04e000-0xfa050000 8192 acpi_os_map_memory+0x16/0x1f phys=df79a000 ioremap
> 0xfa052000-0xfa054000 8192 acpi_os_map_memory+0x16/0x1f phys=df79a000 ioremap
> 0xfa056000-0xfa058000 8192 acpi_os_map_memory+0x16/0x1f phys=df79a000 ioremap
> 0xfa05e000-0xfa060000 8192 acpi_os_map_memory+0x16/0x1f phys=fed1f000 ioremap
> 0xfa06e000-0xfa070000 8192 usb_hcd_pci_probe+0x17b/0x350 phys=fa4de000 ioremap
> 0xfa072000-0xfa074000 8192 usb_hcd_pci_probe+0x17b/0x350 phys=fa4dc000 ioremap
> 0xfa78c000-0xfa79d000 69632 module_alloc+0x72/0x80 pages=16 vmalloc
> 0xfa805000-0xfa84a000 282624 module_alloc+0x72/0x80 pages=68 vmalloc
> 0xfc000000-0xfc400000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
> 0xfc400000-0xfc800000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
> 0xfc93f000-0xfc941000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfc94b000-0xfc94e000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
> 0xfc95e000-0xfc964000 24576 module_alloc+0x72/0x80 pages=5 vmalloc
> 0xfc96f000-0xfc972000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
> 0xfcdcc000-0xfcddf000 77824 module_alloc+0x72/0x80 pages=18 vmalloc
> 0xfce11000-0xfce14000 12288 reqsk_queue_alloc+0x54/0xd0 pages=2 vmalloc
> 0xfce15000-0xfce18000 12288 reqsk_queue_alloc+0x54/0xd0 pages=2 vmalloc
> 0xfce19000-0xfce1c000 12288 reqsk_queue_alloc+0x54/0xd0 pages=2 vmalloc
> 0xfce21000-0xfce23000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfce2a000-0xfce2c000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfce30000-0xfce33000 12288 reqsk_queue_alloc+0x54/0xd0 pages=2 vmalloc
> 0xfce34000-0xfce37000 12288 reqsk_queue_alloc+0x54/0xd0 pages=2 vmalloc
> 0xfce38000-0xfce3b000 12288 reqsk_queue_alloc+0x54/0xd0 pages=2 vmalloc
> 0xfd0e7000-0xfd0ea000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd0eb000-0xfd0ee000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd0ef000-0xfd0f2000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd0f3000-0xfd0f6000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd0f7000-0xfd0fa000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd0fb000-0xfd0fe000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd0ff000-0xfd102000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd103000-0xfd106000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd107000-0xfd10a000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd10b000-0xfd10e000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd10f000-0xfd112000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd113000-0xfd116000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd117000-0xfd11a000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd11b000-0xfd11e000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd11f000-0xfd122000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd123000-0xfd126000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd127000-0xfd12a000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd12b000-0xfd12e000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd12f000-0xfd132000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd133000-0xfd136000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd137000-0xfd13a000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd13b000-0xfd13e000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd13f000-0xfd142000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd143000-0xfd146000 12288 igb_setup_tx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd147000-0xfd14a000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd14b000-0xfd14e000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd14f000-0xfd152000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd153000-0xfd156000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd157000-0xfd15a000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd15b000-0xfd15e000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd15f000-0xfd162000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd163000-0xfd166000 12288 igb_setup_rx_resources+0x27/0x140 [igb] pages=2 vmalloc
> 0xfd1b2000-0xfd1b6000 16384 module_alloc+0x72/0x80 pages=3 vmalloc
> 0xfd1bf000-0xfd1c1000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfd1cb000-0xfd1ce000 12288 module_alloc+0x72/0x80 pages=2 vmalloc
> 0xfd1d5000-0xfd1d7000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfd1de000-0xfd1e0000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfd1e8000-0xfd1ea000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfd1f2000-0xfd1f4000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfd1fb000-0xfd1fd000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfd205000-0xfd207000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfd20f000-0xfd211000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfd219000-0xfd21b000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfd223000-0xfd225000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfd22d000-0xfd22f000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfd23a000-0xfd23c000 8192 module_alloc+0x72/0x80 pages=1 vmalloc
> 0xfd248000-0xfd24c000 16384 module_alloc+0x72/0x80 pages=3 vmalloc
> 0xfd400000-0xfd800000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
> 0xfd800000-0xfdc00000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
> 0xfdc00000-0xfe000000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
> 0xfe000000-0xfe400000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
> 0xfe400000-0xfe800000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
> 0xfe800000-0xfec00000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
> 0xfec00000-0xff000000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
> 0xff000000-0xff400000 4194304 pcpu_get_vm_areas+0x0/0x610 vmalloc
>
Thanks
Your vmalloc space is very fragmented. pcpu_get_vm_areas() want
hugepages (4MB on your machine, 2MB on mine because I have
CONFIG_HIGHMEM64G=y)
You could :
1) Use a 64 bit kernel ( :) )
or
2) boot parameter vmalloc=256M to get more room
(default is 128 Mbytes)
and eventually
select a 2G/2G User/Kernel split to get more LOWMEM, because big vmalloc
windows shrinks the LOWMEM zone. (CONFIG_VMSPLIT_2G=y)
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-05 20:20 ` Eric Dumazet
@ 2010-11-05 20:26 ` Ben Greear
2010-11-05 20:53 ` Eric Dumazet
2010-11-05 22:11 ` Eric Dumazet
1 sibling, 1 reply; 17+ messages in thread
From: Ben Greear @ 2010-11-05 20:26 UTC (permalink / raw)
To: Eric Dumazet; +Cc: NetDev, linux-kernel, Tejun Heo
On 11/05/2010 01:20 PM, Eric Dumazet wrote:
> Your vmalloc space is very fragmented. pcpu_get_vm_areas() want
> hugepages (4MB on your machine, 2MB on mine because I have
> CONFIG_HIGHMEM64G=y)
>
> You could :
>
> 1) Use a 64 bit kernel ( :) )
We mostly use 64-bit, but just not for the remastered live cd image.
> or
>
> 2) boot parameter vmalloc=256M to get more room
> (default is 128 Mbytes)
We'll try that.
>
> and eventually
>
> select a 2G/2G User/Kernel split to get more LOWMEM, because big vmalloc
> windows shrinks the LOWMEM zone. (CONFIG_VMSPLIT_2G=y)
That sounds promising as well.
I was also wondering if it would make sense to allow one to disable
the snmp stats for ipv6? I don't think I have any use for those
stats anyway..
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-05 20:26 ` Ben Greear
@ 2010-11-05 20:53 ` Eric Dumazet
0 siblings, 0 replies; 17+ messages in thread
From: Eric Dumazet @ 2010-11-05 20:53 UTC (permalink / raw)
To: Ben Greear; +Cc: NetDev, linux-kernel, Tejun Heo
Le vendredi 05 novembre 2010 à 13:26 -0700, Ben Greear a écrit :
>
> I was also wondering if it would make sense to allow one to disable
> the snmp stats for ipv6? I don't think I have any use for those
> stats anyway..
>
I agree. IPV6 have per device SNMP fields, percpu... thats probably not
needed.
We have many SNMP fields that could avoid being percpu, even for ipv4.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-05 20:20 ` Eric Dumazet
2010-11-05 20:26 ` Ben Greear
@ 2010-11-05 22:11 ` Eric Dumazet
2010-11-06 0:07 ` Ben Greear
2010-11-06 9:11 ` Tejun Heo
1 sibling, 2 replies; 17+ messages in thread
From: Eric Dumazet @ 2010-11-05 22:11 UTC (permalink / raw)
To: Ben Greear; +Cc: NetDev, linux-kernel, Tejun Heo
Le vendredi 05 novembre 2010 à 21:20 +0100, Eric Dumazet a écrit :
> Your vmalloc space is very fragmented. pcpu_get_vm_areas() want
> hugepages (4MB on your machine, 2MB on mine because I have
> CONFIG_HIGHMEM64G=y)
Well, this is wrong. We use normal (4KB) pages, unfortunately.
I have a NUMA machine, with two nodes, so pcpu_get_vm_areas() allocates
two zones, one for each node, with a 'known' offset between them.
Then, 4KB pages are allocated to populate the zone when needed.
# grep pcpu_get_vm_areas /proc/vmallocinfo
0xffffe8ffa0400000-0xffffe8ffa0600000 2097152 pcpu_get_vm_areas+0x0/0x740 vmalloc
0xffffe8ffffc00000-0xffffe8ffffe00000 2097152 pcpu_get_vm_areas+0x0/0x740 vmalloc
BTW, we dont have the number of pages currently allocated in each
'vmalloc' zone, and/or node information.
Tejun, do you have plans to use hugepages eventually ?
(and fallback to 4KB pages, but most percpu data are allocated right
after boot)
Thanks
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-05 22:11 ` Eric Dumazet
@ 2010-11-06 0:07 ` Ben Greear
2010-11-06 7:26 ` Eric Dumazet
2010-11-06 9:11 ` Tejun Heo
1 sibling, 1 reply; 17+ messages in thread
From: Ben Greear @ 2010-11-06 0:07 UTC (permalink / raw)
To: Eric Dumazet; +Cc: NetDev, linux-kernel, Tejun Heo
On 11/05/2010 03:11 PM, Eric Dumazet wrote:
> Le vendredi 05 novembre 2010 à 21:20 +0100, Eric Dumazet a écrit :
>> Your vmalloc space is very fragmented. pcpu_get_vm_areas() want
>> hugepages (4MB on your machine, 2MB on mine because I have
>> CONFIG_HIGHMEM64G=y)
>
> Well, this is wrong. We use normal (4KB) pages, unfortunately.
>
> I have a NUMA machine, with two nodes, so pcpu_get_vm_areas() allocates
> two zones, one for each node, with a 'known' offset between them.
> Then, 4KB pages are allocated to populate the zone when needed.
>
> # grep pcpu_get_vm_areas /proc/vmallocinfo
> 0xffffe8ffa0400000-0xffffe8ffa0600000 2097152 pcpu_get_vm_areas+0x0/0x740 vmalloc
> 0xffffe8ffffc00000-0xffffe8ffffe00000 2097152 pcpu_get_vm_areas+0x0/0x740 vmalloc
>
> BTW, we dont have the number of pages currently allocated in each
> 'vmalloc' zone, and/or node information.
>
> Tejun, do you have plans to use hugepages eventually ?
> (and fallback to 4KB pages, but most percpu data are allocated right
> after boot)
We just tried creating 1000 macvlans with IPv6 addrs on a 64-bit machine
with 12GB RAM. Only around 520 interfaces properly set their IPs, and
again there are errors about of-of-memory from 'ip', but no obvious
splats in dmesg.
'top' shows 10G or so free.
It will take some time to figure out what exactly is returning
the ENOMEM....
Thanks,
Ben
>
> Thanks
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-06 0:07 ` Ben Greear
@ 2010-11-06 7:26 ` Eric Dumazet
2010-11-06 17:08 ` Ben Greear
0 siblings, 1 reply; 17+ messages in thread
From: Eric Dumazet @ 2010-11-06 7:26 UTC (permalink / raw)
To: Ben Greear; +Cc: NetDev, linux-kernel, Tejun Heo
Le vendredi 05 novembre 2010 à 17:07 -0700, Ben Greear a écrit :
> We just tried creating 1000 macvlans with IPv6 addrs on a 64-bit machine
> with 12GB RAM. Only around 520 interfaces properly set their IPs, and
> again there are errors about of-of-memory from 'ip', but no obvious
> splats in dmesg.
>
> 'top' shows 10G or so free.
>
> It will take some time to figure out what exactly is returning
> the ENOMEM....
At least, nothing to do with percpu stuff ?
On my 4GB machine, 16 'cpus' (but 32 possible cpus), I was able to
allocate.
8192 percpu 8192 bytes structures
(total : 32 * 8192 * 8192 = 2 Gbytes)
setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:32 nr_node_ids:2
PERCPU: Embedded 26 pages/cpu @ffff88007fc00000 s76032 r8192 d22272 u131072
pcpu-alloc: s76032 r8192 d22272 u131072 alloc=1*2097152
pcpu-alloc: [0] 00 02 04 06 08 10 12 14 17 19 21 23 25 27 29 31
pcpu-alloc: [1] 01 03 05 07 09 11 13 15 16 18 20 22 24 26 28 30
grep Vmalloc /proc/meminfo
VmallocTotal: 34359738367 kB
VmallocUsed: 2202592 kB
VmallocChunk: 34356996456 kB
Make sure udev / hotplug is not the problem, if you create your devices
very fast.
(modprobe dummy numdummies=2000) can be very slow because of that.
All tasks are fighting for RTNL or sysfs mutex.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-06 7:26 ` Eric Dumazet
@ 2010-11-06 17:08 ` Ben Greear
2010-11-08 11:02 ` Eric Dumazet
0 siblings, 1 reply; 17+ messages in thread
From: Ben Greear @ 2010-11-06 17:08 UTC (permalink / raw)
To: Eric Dumazet; +Cc: NetDev, linux-kernel, Tejun Heo
On 11/06/2010 12:26 AM, Eric Dumazet wrote:
> Le vendredi 05 novembre 2010 à 17:07 -0700, Ben Greear a écrit :
>
>> We just tried creating 1000 macvlans with IPv6 addrs on a 64-bit machine
>> with 12GB RAM. Only around 520 interfaces properly set their IPs, and
>> again there are errors about of-of-memory from 'ip', but no obvious
>> splats in dmesg.
>>
>> 'top' shows 10G or so free.
>>
>> It will take some time to figure out what exactly is returning
>> the ENOMEM....
>
> At least, nothing to do with percpu stuff ?
At least I don't see any percpu dumps in dmesg. I vaguely remember
someone posting some ipv6 address scalability patches some time back.
I think they had to hack on /proc fs as well. I'll see if I can
dig those up.
> Make sure udev / hotplug is not the problem, if you create your devices
> very fast.
We can create the macvlans w/out problem, though I'm sure that could
be sped up. The problem is when we try to add IPv6 addresses to
them.
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-06 17:08 ` Ben Greear
@ 2010-11-08 11:02 ` Eric Dumazet
2010-11-08 17:45 ` Ben Greear
2010-11-08 21:27 ` Ben Greear
0 siblings, 2 replies; 17+ messages in thread
From: Eric Dumazet @ 2010-11-08 11:02 UTC (permalink / raw)
To: Ben Greear; +Cc: NetDev, linux-kernel, Tejun Heo
Le samedi 06 novembre 2010 à 10:08 -0700, Ben Greear a écrit :
> At least I don't see any percpu dumps in dmesg. I vaguely remember
> someone posting some ipv6 address scalability patches some time back.
> I think they had to hack on /proc fs as well. I'll see if I can
> dig those up.
>
> > Make sure udev / hotplug is not the problem, if you create your devices
> > very fast.
>
> We can create the macvlans w/out problem, though I'm sure that could
> be sped up. The problem is when we try to add IPv6 addresses to
> them.
I see. Did you check /proc/sys/net/ipv6/ tunables ?
For example, I bet you need to make route/max_size a bigger value than
default (4096)
Following is working for me
echo 16384 >/proc/sys/net/ipv6/route/max_size
modprobe dummy numdummies=2000
for a in `seq 1 1999`
do
ip -6 add add 4444::444:$a/24 dev dummy$a
done
ip -6 ro | wc -l
6008
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-08 11:02 ` Eric Dumazet
@ 2010-11-08 17:45 ` Ben Greear
2010-11-08 17:55 ` Eric Dumazet
2010-11-08 21:27 ` Ben Greear
1 sibling, 1 reply; 17+ messages in thread
From: Ben Greear @ 2010-11-08 17:45 UTC (permalink / raw)
To: Eric Dumazet; +Cc: NetDev, linux-kernel, Tejun Heo
On 11/08/2010 03:02 AM, Eric Dumazet wrote:
> Le samedi 06 novembre 2010 à 10:08 -0700, Ben Greear a écrit :
>
>> At least I don't see any percpu dumps in dmesg. I vaguely remember
>> someone posting some ipv6 address scalability patches some time back.
>> I think they had to hack on /proc fs as well. I'll see if I can
>> dig those up.
>>
>>> Make sure udev / hotplug is not the problem, if you create your devices
>>> very fast.
>>
>> We can create the macvlans w/out problem, though I'm sure that could
>> be sped up. The problem is when we try to add IPv6 addresses to
>> them.
>
> I see. Did you check /proc/sys/net/ipv6/ tunables ?
>
> For example, I bet you need to make route/max_size a bigger value than
> default (4096)
>
> Following is working for me
>
> echo 16384>/proc/sys/net/ipv6/route/max_size
> modprobe dummy numdummies=2000
> for a in `seq 1 1999`
> do
> ip -6 add add 4444::444:$a/24 dev dummy$a
> done
>
> ip -6 ro | wc -l
> 6008
That helps. I'm getting all of the IP addrs set now, but
having trouble with some of the default gateways (I have one
routing table per interface).
./local/sbin/ip -6 route replace default via 2002:9:8::1 dev eth7#458 table 726
RTNETLINK answers: No buffer space available
dmesg is full of this:
[247106.294743] ipv6: Neighbour table overflow.
A quick look in /proc didn't show a tunable for this, but I'll
go grub through the code.
As for the route/max_size, it would be nice to see some useful kernel
message in dmesg when this hit. Just telling the user '-ENOMEM'
is not at all sufficient to help them figure out the problem.
For that matter, why is there such a limit anyway? IPv4 doesn't appear
to have any such limit?
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-08 17:45 ` Ben Greear
@ 2010-11-08 17:55 ` Eric Dumazet
2010-11-08 18:08 ` Ben Greear
0 siblings, 1 reply; 17+ messages in thread
From: Eric Dumazet @ 2010-11-08 17:55 UTC (permalink / raw)
To: Ben Greear; +Cc: NetDev, linux-kernel, Tejun Heo
Le lundi 08 novembre 2010 à 09:45 -0800, Ben Greear a écrit :
> That helps. I'm getting all of the IP addrs set now, but
> having trouble with some of the default gateways (I have one
> routing table per interface).
>
> ./local/sbin/ip -6 route replace default via 2002:9:8::1 dev eth7#458 table 726
> RTNETLINK answers: No buffer space available
>
> dmesg is full of this:
>
> [247106.294743] ipv6: Neighbour table overflow.
>
>
> A quick look in /proc didn't show a tunable for this, but I'll
> go grub through the code.
>
> As for the route/max_size, it would be nice to see some useful kernel
> message in dmesg when this hit. Just telling the user '-ENOMEM'
> is not at all sufficient to help them figure out the problem.
Sure, patches are welcomed. Apparently nobody yet used ipv6 with so many
devices / routes, and this nobody contributed to extend limits.
>
> For that matter, why is there such a limit anyway? IPv4 doesn't appear
> to have any such limit?
There are limits for ipv4, much bigger, you probably never noticed.
/proc/sys/net/ipv4/route/gc_elasticity:8
/proc/sys/net/ipv4/route/gc_interval:60
/proc/sys/net/ipv4/route/gc_min_interval:0
/proc/sys/net/ipv4/route/gc_min_interval_ms:500
/proc/sys/net/ipv4/route/gc_thresh:131072
/proc/sys/net/ipv4/route/gc_timeout:300
/proc/sys/net/ipv4/route/max_size:2097152 <<< HERE
/proc/sys/net/ipv4/route/min_adv_mss:256
/proc/sys/net/ipv4/route/min_pmtu:552
/proc/sys/net/ipv4/route/mtu_expires:600
/proc/sys/net/ipv4/route/redirect_load:2
/proc/sys/net/ipv4/route/redirect_number:9
/proc/sys/net/ipv4/route/redirect_silence:2048
I suggest followup discussion can got to netdev only, now per-cpu it not
anymore the problem ?
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-08 17:55 ` Eric Dumazet
@ 2010-11-08 18:08 ` Ben Greear
0 siblings, 0 replies; 17+ messages in thread
From: Ben Greear @ 2010-11-08 18:08 UTC (permalink / raw)
To: Eric Dumazet; +Cc: NetDev
On 11/08/2010 09:55 AM, Eric Dumazet wrote:
> Le lundi 08 novembre 2010 à 09:45 -0800, Ben Greear a écrit :
>
>> That helps. I'm getting all of the IP addrs set now, but
>> having trouble with some of the default gateways (I have one
>> routing table per interface).
>>
>> ./local/sbin/ip -6 route replace default via 2002:9:8::1 dev eth7#458 table 726
>> RTNETLINK answers: No buffer space available
>>
>> dmesg is full of this:
>>
>> [247106.294743] ipv6: Neighbour table overflow.
>>
>>
>> A quick look in /proc didn't show a tunable for this, but I'll
>> go grub through the code.
>>
>> As for the route/max_size, it would be nice to see some useful kernel
>> message in dmesg when this hit. Just telling the user '-ENOMEM'
>> is not at all sufficient to help them figure out the problem.
>
> Sure, patches are welcomed. Apparently nobody yet used ipv6 with so many
> devices / routes, and this nobody contributed to extend limits.
I'll see what I can do. I'm aiming for several thousand IPv6 addrs,
so will try to get these limitations ironed out.
>> For that matter, why is there such a limit anyway? IPv4 doesn't appear
>> to have any such limit?
>
> There are limits for ipv4, much bigger, you probably never noticed.
>
>
> /proc/sys/net/ipv4/route/gc_elasticity:8
> /proc/sys/net/ipv4/route/gc_interval:60
> /proc/sys/net/ipv4/route/gc_min_interval:0
> /proc/sys/net/ipv4/route/gc_min_interval_ms:500
> /proc/sys/net/ipv4/route/gc_thresh:131072
> /proc/sys/net/ipv4/route/gc_timeout:300
> /proc/sys/net/ipv4/route/max_size:2097152<<< HERE
> /proc/sys/net/ipv4/route/min_adv_mss:256
> /proc/sys/net/ipv4/route/min_pmtu:552
> /proc/sys/net/ipv4/route/mtu_expires:600
> /proc/sys/net/ipv4/route/redirect_load:2
> /proc/sys/net/ipv4/route/redirect_number:9
> /proc/sys/net/ipv4/route/redirect_silence:2048
>
> I suggest followup discussion can got to netdev only, now per-cpu it not
> anymore the problem ?
Agreed, and trimmed.
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-08 11:02 ` Eric Dumazet
2010-11-08 17:45 ` Ben Greear
@ 2010-11-08 21:27 ` Ben Greear
2010-11-08 21:40 ` Eric Dumazet
1 sibling, 1 reply; 17+ messages in thread
From: Ben Greear @ 2010-11-08 21:27 UTC (permalink / raw)
To: Eric Dumazet; +Cc: NetDev
On 11/08/2010 03:02 AM, Eric Dumazet wrote:
> Le samedi 06 novembre 2010 à 10:08 -0700, Ben Greear a écrit :
>
>> At least I don't see any percpu dumps in dmesg. I vaguely remember
>> someone posting some ipv6 address scalability patches some time back.
>> I think they had to hack on /proc fs as well. I'll see if I can
>> dig those up.
>>
>>> Make sure udev / hotplug is not the problem, if you create your devices
>>> very fast.
>>
>> We can create the macvlans w/out problem, though I'm sure that could
>> be sped up. The problem is when we try to add IPv6 addresses to
>> them.
>
> I see. Did you check /proc/sys/net/ipv6/ tunables ?
>
> For example, I bet you need to make route/max_size a bigger value than
> default (4096)
I'm having a hard time figuring out how this actually causes something
to fail.
It seems that having max_size too small would cause the garbage collector
logic to happen more often, but I don't see any place it actually
removes or limits entries, and I can't find any other references to
ip6_rt_max_size.
I must just be missing something..so if you have any ideas where to
look, I'd love to hear it!
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-08 21:27 ` Ben Greear
@ 2010-11-08 21:40 ` Eric Dumazet
0 siblings, 0 replies; 17+ messages in thread
From: Eric Dumazet @ 2010-11-08 21:40 UTC (permalink / raw)
To: Ben Greear; +Cc: NetDev
Le lundi 08 novembre 2010 à 13:27 -0800, Ben Greear a écrit :
> On 11/08/2010 03:02 AM, Eric Dumazet wrote:
> > Le samedi 06 novembre 2010 à 10:08 -0700, Ben Greear a écrit :
> >
> >> At least I don't see any percpu dumps in dmesg. I vaguely remember
> >> someone posting some ipv6 address scalability patches some time back.
> >> I think they had to hack on /proc fs as well. I'll see if I can
> >> dig those up.
> >>
> >>> Make sure udev / hotplug is not the problem, if you create your devices
> >>> very fast.
> >>
> >> We can create the macvlans w/out problem, though I'm sure that could
> >> be sped up. The problem is when we try to add IPv6 addresses to
> >> them.
> >
> > I see. Did you check /proc/sys/net/ipv6/ tunables ?
> >
> > For example, I bet you need to make route/max_size a bigger value than
> > default (4096)
>
> I'm having a hard time figuring out how this actually causes something
> to fail.
>
> It seems that having max_size too small would cause the garbage collector
> logic to happen more often, but I don't see any place it actually
> removes or limits entries, and I can't find any other references to
> ip6_rt_max_size.
>
> I must just be missing something..so if you have any ideas where to
> look, I'd love to hear it!
>
ipv6_add_addr() calls addrconf_dst_alloc
addrconf_dst_alloc() calls dst_alloc
dst_alloc() calls ip6_dst_gc
ip6_dst_gc() calls fib6_run_gc
fib6_run_gc() calls icmp6_dst_gc
fib6_run_gc() calls fib6_clean_all
a bit later, we return to ipv6_add_addr() (line 644) with an error
You want to add a warning in net/ipv6/route.c, line 1949
Thanks
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: OOM when adding ipv6 route: How to make available more per-cpu memory?
2010-11-05 22:11 ` Eric Dumazet
2010-11-06 0:07 ` Ben Greear
@ 2010-11-06 9:11 ` Tejun Heo
1 sibling, 0 replies; 17+ messages in thread
From: Tejun Heo @ 2010-11-06 9:11 UTC (permalink / raw)
To: Eric Dumazet; +Cc: Ben Greear, NetDev, linux-kernel
Hello,
On 11/05/2010 11:11 PM, Eric Dumazet wrote:
> Le vendredi 05 novembre 2010 à 21:20 +0100, Eric Dumazet a écrit :
>> Your vmalloc space is very fragmented. pcpu_get_vm_areas() want
>> hugepages (4MB on your machine, 2MB on mine because I have
>> CONFIG_HIGHMEM64G=y)
>
> Well, this is wrong. We use normal (4KB) pages, unfortunately.
>
> I have a NUMA machine, with two nodes, so pcpu_get_vm_areas() allocates
> two zones, one for each node, with a 'known' offset between them.
> Then, 4KB pages are allocated to populate the zone when needed.
>
> # grep pcpu_get_vm_areas /proc/vmallocinfo
> 0xffffe8ffa0400000-0xffffe8ffa0600000 2097152 pcpu_get_vm_areas+0x0/0x740 vmalloc
> 0xffffe8ffffc00000-0xffffe8ffffe00000 2097152 pcpu_get_vm_areas+0x0/0x740 vmalloc
>
> BTW, we dont have the number of pages currently allocated in each
> 'vmalloc' zone, and/or node information.
>
> Tejun, do you have plans to use hugepages eventually ?
> (and fallback to 4KB pages, but most percpu data are allocated right
> after boot)
Well, it's rather complicated. Till now, the percpu usage hasn't
justified allocating hugepages but it might someday, but more
importantly the reason why those big chunks of address space are used
is to keep the first chunk embedded in the regular linear kernel
address space to avoid extra TLB pressure.
On configurations where vmalloc area is a scarce resource,
percpu_alloc=page can be specified to use page-mapped allocation.
This will use much smaller chunks in vmalloc area at the cost of
additional 4k page TLB pressure for percpu memory in the first chunk
(all the static percpu variables and then some).
pcpu_embed_first_chunk() contains heuristic which makes it yield to
page allocator but the parameter is pretty generous (maximum distance
between chunks > 75% of vmalloc area). It's there just to avoid
completely crazy cases. Also, x86 setup_per_cpu_areas() chooses page
allocator on 32bit NUMAs. This case didn't trigger either. We
probably need to add another condition.
That large machine on 32bit is bound to be flaky. To be short on
virtual address space is a pretty silly and stupid thing. Anyways,
any good idea on what criteria we could test?
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2010-11-08 21:40 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-11-05 17:19 OOM when adding ipv6 route: How to make available more per-cpu memory? Ben Greear
2010-11-05 18:06 ` Eric Dumazet
2010-11-05 18:15 ` Ben Greear
2010-11-05 20:20 ` Eric Dumazet
2010-11-05 20:26 ` Ben Greear
2010-11-05 20:53 ` Eric Dumazet
2010-11-05 22:11 ` Eric Dumazet
2010-11-06 0:07 ` Ben Greear
2010-11-06 7:26 ` Eric Dumazet
2010-11-06 17:08 ` Ben Greear
2010-11-08 11:02 ` Eric Dumazet
2010-11-08 17:45 ` Ben Greear
2010-11-08 17:55 ` Eric Dumazet
2010-11-08 18:08 ` Ben Greear
2010-11-08 21:27 ` Ben Greear
2010-11-08 21:40 ` Eric Dumazet
2010-11-06 9:11 ` Tejun Heo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).