All of lore.kernel.org
 help / color / mirror / Atom feed
* Maximum number of domains and NR_IRQS
@ 2006-12-07 15:37 Carb, Brian A
  2006-12-07 15:50 ` Keir Fraser
  0 siblings, 1 reply; 10+ messages in thread
From: Carb, Brian A @ 2006-12-07 15:37 UTC (permalink / raw)
  To: xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 7708 bytes --]

Running xen-unstable changeset 12734 on an ES7000 4 dual-core sockets
(8cpu) 16GB memory, with kernel parameters dom0_mem=512M and
xenheap_megabytes=64. DOMUs are paravirtualized sles10, 96MB, 4vcpus,
each on a separate physical lun. 
 
We successfully start 118 DOMUs, but when we try to start the 119th, the
system panics with the following messages:
  Kernel panic - not syncing: No available IRQ to bind to: increase
NR_IRQS!
  (XEN) Domain 0 crashed: rebooting machine in 5 seconds.
 
The documentation in include/asm-x86_64/irq.h suggests that the value of
NR_IRQS under x86_64 is limited to 256. In fact, when we rebuilt
xen-unstable with NR_IRQS set to 768, the kernel panics on boot (see
below).
 
On the hunch that networking in a VM uses up an IRQ, we eliminated the
'vif' statement in each DOMU config file, and we were able to start 164
VMs in the 16GB before we exhausted memory.
 
Has anyone run into this IRQ issue? Is there any work-around?
brian carb
unisys corporation - malvern, pa
brian.carb@unisys.com <mailto:brian.carb@unisys.com> 

--- trace output when trying to boot xen where NR_IRQs was set to 768
---
(XEN) Initializing CPU#0
(XEN) Detected 3400.113 MHz processor.
(XEN) extable.c:77: Pre-exception: ffff8300001713fc -> 0000000000000000
(XEN) ----[ Xen-3.0-unstable  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e010:[<ffff8300001713fc>] get_cpu_vendor+0x2c/0x90
(XEN) RFLAGS: 0000000000010006   CONTEXT: hypervisor
(XEN) rax: 00008e00e0100000   rbx: ffff830000206840   rcx:
000000006c65746e
(XEN) rdx: 0000000049656e69   rsi: 0000000000000000   rdi:
ffff830000187980
(XEN) rbp: 0000000000000000   rsp: ffff8300001bfe18   r8:
0000000000410000
(XEN) r9:  000000000000003a   r10: 00000000000000ff   r11:
0000000000000000
(XEN) r12: ffff830000206840   r13: ffff8300001879a8   r14:
ffff830000187980
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
00000000000000b0
(XEN) cr3: 0000000000102000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e010
(XEN) Xen stack trace from rsp=ffff8300001bfe18:
(XEN)    0000000000001180 00000000756e6547 0000000000000000
ffff830000187980
(XEN)    ffff83000019b000 00000000003f7f0d ffff830000020980
ffff83000017168a
(XEN)    0000000000000000 ffff830000165a4c ffff830003ffc080
ffff830000187980
(XEN)    0000000000003ce5 ffff830000171948 ffff83000019b000
ffff830003ffc080
(XEN)    000000000000000a ffff83000016de41 0000000000000000
ffff83000002bbc0
(XEN)    0000000800000000 000000010000006e 0000000000000003
00000000000002f8
(XEN)    0000000000000000 000000000000000a 0000000000000000
0000000000000000
(XEN)    0000000000067f0c 0000000000000000 0000000000000000
0000000000000000
(XEN)    0000000000000000 ffff8300001001c1 0000000000000000
0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000
0000000000000000
(XEN)    ffff83000028c080
(XEN) Xen call trace:
(XEN)    [<ffff8300001713fc>] get_cpu_vendor+0x2c/0x90
(XEN)    [<ffff83000017168a>] generic_identify+0x4a/0x160
(XEN)    [<ffff830000165a4c>] subarch_init_memory+0xac/0xe0
(XEN)    [<ffff830000171948>] identify_cpu+0x78/0x1f0
(XEN)    [<ffff83000016de41>] __start_xen+0x861/0xc90
(XEN)    [<ffff8300001001c1>] __high_start+0x94/0x96
(XEN)    
(XEN) 
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL TRAP: vector = 13 (general protection fault)
(XEN) [error_code=0000] , IN INTERRUPT CONTEXT
(XEN) ****************************************
(XEN) 
(XEN) Reboot in five seconds...
(XEN) machine_crash_shutdown: 0
(XEN) extable.c:77: Pre-exception: ffff830000139e9c -> 0000000000000000
(XEN) ----[ Xen-3.0-unstable  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e010:[<ffff830000139e9c>] machine_crash_shutdown+0x7c/0xf0
(XEN) RFLAGS: 0000000000010047   CONTEXT: hypervisor
(XEN) rax: 0000000000000000   rbx: 00000000000003e8   rcx:
0000ffff0000ffff
(XEN) rdx: 0000000000000001   rsi: 0000000000000400   rdi:
0000000000000000
(XEN) rbp: 0000000000000046   rsp: ffff8300001bfbc8   r8:
ffff8300000b8000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11:
ffff83000011c7a0
(XEN) r12: ffff830000206840   r13: ffff8300001879a8   r14:
ffff830000187980
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
00000000000000b0
(XEN) cr3: 0000000000102000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e010
(XEN) Xen stack trace from rsp=ffff8300001bfbc8:
(XEN)    0000000000000000 0000000000000046 ffff830000206840
ffff83000010e18f
(XEN)    ffff830000206840 ffff83000011c24c 5254204c41544146
74636576203a5041
(XEN)    203331203d20726f 6c6172656e656728 746365746f727020
6c756166206e6f69
(XEN)    6f7272655b0a2974 303d65646f635f72 49202c205d303030
525245544e49204e
(XEN)    544e4f4320545055 000000000a545845 0000003000000008
ffff8300001bfd48
(XEN)    ffff8300001bfc78 ffff8300001d7321 0000003000000028
ffff8300001bfd68
(XEN)    ffff8300001bfc98 ffff8300000b8000 0000000000000002
000000000000000d
(XEN)    ffff83000017c2aa 0000000000000000 ffff83000017c162
0000000000000000
(XEN)    0000000000000000 ffff8300001d7356 000000008005003b
ffff83000011cf7b
(XEN)    0000000000000000 0000000000102000 0000000000000096
0000000000000094
(XEN)    ffff8300001001c1 ffff8300001001c1 ffff8300001bff20
ffff8300001330a4
(XEN)    ffff830000206840 000000000000000d ffff8300001bfd68
ffff8300001334d7
(XEN)    ffff830000206840 ffff830000206840 0000000000000000
ffff8300001656c2
(XEN)    0000000000000000 ffff830000187980 ffff8300001879a8
ffff830000206840
(XEN)    0000000000000000 ffff830000206840 0000000000000000
00000000000000ff
(XEN)    000000000000003a 0000000000410000 00008e00e0100000
000000006c65746e
(XEN)    0000000049656e69 0000000000000000 ffff830000187980
0000000d00000000
(XEN)    ffff8300001713fc 000000000000e010 0000000000010006
ffff8300001bfe18
(XEN)    0000000000000000 0000000000000001 0000000000001180
00000000756e6547
(XEN)    0000000000000000 ffff830000187980 ffff83000019b000
00000000003f7f0d
(XEN) Xen call trace:
(XEN)    [<ffff830000139e9c>] machine_crash_shutdown+0x7c/0xf0
(XEN)    [<ffff83000010e18f>] machine_crash_kexec+0x2f/0x90
(XEN)    [<ffff83000011c24c>] panic+0x15c/0x1b0
(XEN)    [<ffff83000011cf7b>] __serial_putc+0xdb/0x100
(XEN)    [<ffff8300001001c1>] __high_start+0x94/0x96
(XEN)    [<ffff8300001001c1>] __high_start+0x94/0x96
(XEN)    [<ffff8300001330a4>] show_trace+0x54/0xa0
(XEN)    [<ffff8300001334d7>] fatal_trap+0x77/0xb0
(XEN)    [<ffff8300001656c2>]
FATAL_exception_with_ints_disabled+0xc/0x1a
(XEN)    [<ffff8300001713fc>] get_cpu_vendor+0x2c/0x90
(XEN)    [<ffff83000017168a>] generic_identify+0x4a/0x160
(XEN)    [<ffff830000165a4c>] subarch_init_memory+0xac/0xe0
(XEN)    [<ffff830000171948>] identify_cpu+0x78/0x1f0
(XEN)    [<ffff83000016de41>] __start_xen+0x861/0xc90
(XEN)    [<ffff8300001001c1>] __high_start+0x94/0x96
(XEN)    
(XEN) 
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL TRAP: vector = 13 (general protection fault)
(XEN) [error_code=0000] , IN INTERRUPT CONTEXT
(XEN) ****************************************
(XEN) 
(XEN) Reboot in five seconds...
 

[-- Attachment #1.2: Type: text/html, Size: 11715 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2006-12-11 21:06 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-12-07 15:37 Maximum number of domains and NR_IRQS Carb, Brian A
2006-12-07 15:50 ` Keir Fraser
2006-12-07 16:08   ` Carb, Brian A
2006-12-07 16:14     ` Ian Campbell
2006-12-07 16:25       ` Keir Fraser
2006-12-07 16:42       ` Puthiyaparambil, Aravindh
2006-12-07 16:47         ` Keir Fraser
2006-12-07 20:55           ` Carb, Brian A
2006-12-08  6:32             ` Keir Fraser
2006-12-11 21:06   ` Carb, Brian A

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.