* etherbridge bottleneck
@ 2004-11-10 21:48 David Becker
2004-11-11 12:00 ` Bin Ren
0 siblings, 1 reply; 6+ messages in thread
From: David Becker @ 2004-11-10 21:48 UTC (permalink / raw)
To: xen-devel
I ran some iperf tests today and it looks like the etherbridge
is the limiting factor on throughput. In the beforetime, I saw great
throughput to the VMs; over 800 Mbps. With the bridge, the numbers
are in the 400s somewhere.
Is this the speed I can expect from the bridge?
Is there some tuning I should try, or another way to get more bandwidth
into the VMs?
This is with xen-2.0, 2.4.27-xen0 and 2.4.27-xenU.
My iperf numbers:
940 Mbps stock linux -> stock linux
470 Mbps stock linux -> xenU
533 Mbps xenU -> stock linux
ether bridge speed
533 Mbps xenU -> xen0 on the same host
422 Mbps xen0 -> xenU on the same host
loopback speed
4.4 Gbps stock linux
3.2 Gbps xenU
3.2 Gbps xen0
(stock linux is 2.4.25)
-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: etherbridge bottleneck
2004-11-10 21:48 etherbridge bottleneck David Becker
@ 2004-11-11 12:00 ` Bin Ren
2004-11-11 14:56 ` David Becker
0 siblings, 1 reply; 6+ messages in thread
From: Bin Ren @ 2004-11-11 12:00 UTC (permalink / raw)
To: David Becker; +Cc: xen-devel
Whenever you see a performance problem, please describe the following things:
(1) Is Xen running on a UP, SMP or HT?
(2) If Xen is running on a SMP or HT, are xen0 and xenU running
different processors or threads?
(3) Which scheduler in Xen are you using? Have you changed any parameters?
- Bin
On Wed, 10 Nov 2004 16:48:45 -0500, David Becker <becker@cs.duke.edu> wrote:
>
> I ran some iperf tests today and it looks like the etherbridge
> is the limiting factor on throughput. In the beforetime, I saw great
> throughput to the VMs; over 800 Mbps. With the bridge, the numbers
> are in the 400s somewhere.
> Is this the speed I can expect from the bridge?
> Is there some tuning I should try, or another way to get more bandwidth
> into the VMs?
>
> This is with xen-2.0, 2.4.27-xen0 and 2.4.27-xenU.
>
> My iperf numbers:
> 940 Mbps stock linux -> stock linux
> 470 Mbps stock linux -> xenU
> 533 Mbps xenU -> stock linux
>
> ether bridge speed
> 533 Mbps xenU -> xen0 on the same host
> 422 Mbps xen0 -> xenU on the same host
>
> loopback speed
> 4.4 Gbps stock linux
> 3.2 Gbps xenU
> 3.2 Gbps xen0
>
> (stock linux is 2.4.25)
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Sybase ASE Linux Express Edition - download now for FREE
> LinuxWorld Reader's Choice Award Winner for best database on Linux.
> http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel
>
-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: etherbridge bottleneck
2004-11-11 12:00 ` Bin Ren
@ 2004-11-11 14:56 ` David Becker
0 siblings, 0 replies; 6+ messages in thread
From: David Becker @ 2004-11-11 14:56 UTC (permalink / raw)
To: xen-devel
" (1) Is Xen running on a UP, SMP or HT?
all of the above
" (2) If Xen is running on a SMP or HT, are xen0 and xenU running
" different processors or threads?
xm list doesn't break out HT contexts, so who's to say..
" (3) Which scheduler in Xen are you using? Have you changed any parameters?
factory defaults.
Here is a full account of configs and iperf speeds between domains
on an otherwise-idle host via the etherbridge, plus iperf speed over the
loopback interface for each domain.
Uni-processor (PIII)
426 Mbps xenU UNI -> xen0 UNI
267 Mbps xen0 UNI -> xenU UNI
3.34 Gbps xen0 loopback
3.14 Gbps xenU loopback
xm list
Name Id Mem(MB) CPU State Time(s) Console
Domain-0 0 34 0 r---- 297.0
grant 2 499 0 -b--- 161.2 9602
xm info
system : Linux
host : rack099-xen
release : 2.4.27-xen0
version : #3 Wed Nov 10 11:29:37 EST 2004
machine : i686
cores : 1
hyperthreads_per_core : 1
cpu_mhz : 1396
memory : 2047
free_memory : 1485
slow Uni HT processor (P4 2.0GHz)
498 Mbps xenU HT1 -> xen0 HT0
369 Mbps xen0 HT0 -> xenU HT1
2.49 Gbps xen0 loopback
2.54 Gbps xenU loopback
xm list
Name Id Mem(MB) CPU State Time(s) Console
Domain-0 0 34 0 r---- 561.7
globus 1 799 1 -b--- 225.5 9601
xm info
system : Linux
host : rack160-xen
release : 2.4.27-xen0
version : #2 Thu Nov 4 15:57:19 EST 2004
machine : i686
cores : 1
hyperthreads_per_core : 2
cpu_mhz : 1993
memory : 1023
free_memory : 167
faster Dual HT processors (P4 2.6GHz)
645 Mbps xenU ctxt-1 -> xen0 ctxt-0
470 Mbps xen0 ctxt-0 -> xenU ctxt-1
782 Mbps xenU ctxt-2 -> xen0 ctxt-0
563 Mbps xen0 ctxt-0 -> xenU ctxt-2
747 Mbps xenU ctxt-1 -> xenU ctxt-2
691 Mbps xenU ctxt-2 -> xenU ctxt-1
3.34 Gbps xen0 ctxt-0 loopback
3.50 Gbps xenU ctxt-1 loopback
3.41 Gbps xenU ctxt-2 loopback
xm list
Name Id Mem(MB) CPU State Time(s) Console
Domain-0 0 34 0 r---- 1032.7
batch020 3 1999 1 -b--- 108571.9 9603
batch040 2 1799 2 r---- 154375.3 9602
xm info
system : Linux
host : rack276-xen
release : 2.4.27-xen0
version : #2 Thu Nov 4 15:57:19 EST 2004
machine : i686
cores : 2
hyperthreads_per_core : 2
cpu_mhz : 2599
memory : 3967
free_memory : 94
-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: etherbridge bottleneck
@ 2004-11-12 8:42 Ian Pratt
0 siblings, 0 replies; 6+ messages in thread
From: Ian Pratt @ 2004-11-12 8:42 UTC (permalink / raw)
To: David Becker, xen-devel
David,
You might want to try giving one of your domains a second VIF and then
manually assign an IP address to the vifX.Y interface in domain 0 rather
than using the bridge.
I doubt the bridge is a serious overhead in xenU<->xenO communication,
but it would be useful to eliminate it from your results.
For high-performance xenU<->xenU communication it probably makes sense
to setup a direct netfront/back connection (to avoid going via the dom0
bridge). Although the netfront/back drivers are quite happy working
point-to-point, I doubt xend will let you set this up right now.
Shouldn't be too hard to fix, though...
Ian
> -----Original Message-----
> From: xen-devel-admin@lists.sourceforge.net
> [mailto:xen-devel-admin@lists.sourceforge.net] On Behalf Of
> David Becker
> Sent: 11 November 2004 14:57
> To: xen-devel@lists.sourceforge.net
> Subject: Re: [Xen-devel] etherbridge bottleneck
>
>
> " (1) Is Xen running on a UP, SMP or HT?
>
> all of the above
>
> " (2) If Xen is running on a SMP or HT, are xen0 and xenU
> running " different processors or threads?
>
> xm list doesn't break out HT contexts, so who's to say..
>
> " (3) Which scheduler in Xen are you using? Have you changed
> any parameters?
>
> factory defaults.
>
>
> Here is a full account of configs and iperf speeds between
> domains on an otherwise-idle host via the etherbridge, plus
> iperf speed over the loopback interface for each domain.
>
> Uni-processor (PIII)
> 426 Mbps xenU UNI -> xen0 UNI
> 267 Mbps xen0 UNI -> xenU UNI
> 3.34 Gbps xen0 loopback
> 3.14 Gbps xenU loopback
>
> xm list
> Name Id Mem(MB) CPU State Time(s) Console
> Domain-0 0 34 0 r---- 297.0
> grant 2 499 0 -b--- 161.2 9602
> xm info
> system : Linux
> host : rack099-xen
> release : 2.4.27-xen0
> version : #3 Wed Nov 10 11:29:37 EST 2004
> machine : i686
> cores : 1
> hyperthreads_per_core : 1
> cpu_mhz : 1396
> memory : 2047
> free_memory : 1485
>
> slow Uni HT processor (P4 2.0GHz)
> 498 Mbps xenU HT1 -> xen0 HT0
> 369 Mbps xen0 HT0 -> xenU HT1
> 2.49 Gbps xen0 loopback
> 2.54 Gbps xenU loopback
>
> xm list
> Name Id Mem(MB) CPU State Time(s) Console
> Domain-0 0 34 0 r---- 561.7
> globus 1 799 1 -b--- 225.5 9601
> xm info
> system : Linux
> host : rack160-xen
> release : 2.4.27-xen0
> version : #2 Thu Nov 4 15:57:19 EST 2004
> machine : i686
> cores : 1
> hyperthreads_per_core : 2
> cpu_mhz : 1993
> memory : 1023
> free_memory : 167
>
> faster Dual HT processors (P4 2.6GHz)
> 645 Mbps xenU ctxt-1 -> xen0 ctxt-0
> 470 Mbps xen0 ctxt-0 -> xenU ctxt-1
> 782 Mbps xenU ctxt-2 -> xen0 ctxt-0
> 563 Mbps xen0 ctxt-0 -> xenU ctxt-2
> 747 Mbps xenU ctxt-1 -> xenU ctxt-2
> 691 Mbps xenU ctxt-2 -> xenU ctxt-1
> 3.34 Gbps xen0 ctxt-0 loopback
> 3.50 Gbps xenU ctxt-1 loopback
> 3.41 Gbps xenU ctxt-2 loopback
>
> xm list
> Name Id Mem(MB) CPU State Time(s) Console
> Domain-0 0 34 0 r---- 1032.7
> batch020 3 1999 1 -b--- 108571.9 9603
> batch040 2 1799 2 r---- 154375.3 9602
> xm info
> system : Linux
> host : rack276-xen
> release : 2.4.27-xen0
> version : #2 Thu Nov 4 15:57:19 EST 2004
> machine : i686
> cores : 2
> hyperthreads_per_core : 2
> cpu_mhz : 2599
> memory : 3967
> free_memory : 94
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Sybase ASE Linux Express Edition - download now for FREE
> LinuxWorld Reader's Choice Award Winner for best database on Linux.
> http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel
>
>
-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_idU88&alloc_id\x12065&op=click
^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: etherbridge bottleneck
@ 2004-11-14 19:53 Ian Pratt
2004-11-15 16:01 ` David Becker
0 siblings, 1 reply; 6+ messages in thread
From: Ian Pratt @ 2004-11-14 19:53 UTC (permalink / raw)
To: David Becker, xen-devel
> I ran some iperf tests today and it looks like the
> etherbridge is the limiting factor on throughput. In the
> beforetime, I saw great
> throughput to the VMs; over 800 Mbps. With the bridge, the numbers
> are in the 400s somewhere.
> Is this the speed I can expect from the bridge?
We've had no particular problem with the bridge code causing a
performance bottleneck -- although there appears to be a lot of code in
the forwarding path, it mostly doesn't get executed.
However, since its easy not to have the bridge involved in your tests it
should be fairly easy to eliminate -- just don't connect the vif to the
bridge.
xenU -> xenU performance isn't something we've particularly optimised,
and going via the bridge is going to be far from ideal. It would be much
better to set up a direct netfront->netback path between the two
domains. Xend probably isn't geared up to do this right now, but it
should be possible to add. It'll probably be quite easy to add to x2d2
(new in the unstable tree) which is what we use for prototyping.
> My iperf numbers:
> 940 Mbps stock linux -> stock linux
> 470 Mbps stock linux -> xenU
> 533 Mbps xenU -> stock linux
We have no problem getting 900Mb/s in/out of xenU's and the wire on our
2.4GHz Xeon systems (regardless of whether using uniprocessor, SMP or
hyperthreading).
What do you get between the wire and domain 0? You should be getting
940Mb/s as per your stock-stock comparison. If not, sounds like there
may be an interrupt problem with Xen on your system.
Ian
-------------------------------------------------------
This SF.Net email is sponsored by: InterSystems CACHE
FREE OODBMS DOWNLOAD - A multidimensional database that combines
robust object and relational technologies, making it a perfect match
for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: etherbridge bottleneck
2004-11-14 19:53 Ian Pratt
@ 2004-11-15 16:01 ` David Becker
0 siblings, 0 replies; 6+ messages in thread
From: David Becker @ 2004-11-15 16:01 UTC (permalink / raw)
To: Ian Pratt; +Cc: xen-devel
" We have no problem getting 900Mb/s in/out of xenU's and the wire on our
" 2.4GHz Xeon systems (regardless of whether using uniprocessor, SMP or
" hyperthreading).
Hmmm, I'm certainly having a problem getting over 500Mbps into xenU.
I ran another set of tests, this time between two
identical Dell 1650s, (one 1.4GHz PIII, 2GB ram, e1000 NICs) directly
connected to a Catalyst 4005.
Running a 10sec iperf run, I got these bandwidths:
Stock linux-2.4.25 (no patches or network tuning)
941 Mbps stock -> stock (identical speed both directions)
941 Mbps stock -> stock
Running 2.4.27-xen0 on xen-2.0 before starting xend or the network script:
894 Mbps xen0 -> stock
622 Mbps stock -> xen0
After starting xend, which brought up the bridge:
734 Mbps xen0 -> stock
524 Mbps stock -> xen0
After starting one xenU, xen0 was a touch slower:
729 Mbps xen0 -> stock
515 Mbps stock -> xen0
530 Mbps xenU -> stock
470 Mbps stock -> xenU
427 Mbps xenU -> xen0
253 Mbps xen0 -> xenU
After taking the vif off the bridge
with 'brctl delif xen-br0 vif1.0':
463 Mbps xenU -> xen0
312 Mbps xen0 -> xenU
Here is a full acount of boot messages, etc:
+++++++++++++++++++++++++
Stock linux boot messages
+++++++++++++++++++++++++
Linux version 2.4.25-smp (becker@growl) (gcc version 3.3.3 20040125 (prerelease) (Debian)) #1 SMP Sat Feb 28 19:56:32 EST 2004
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
BIOS-e820: 000000007fff0000 - 000000007fffec00 (ACPI data)
BIOS-e820: 000000007fffec00 - 000000007ffff000 (reserved)
BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved)
1151MB HIGHMEM available.
896MB LOWMEM available.
found SMP MP-table at 000fe710
hm, page 000fe000 reserved twice.
hm, page 000ff000 reserved twice.
hm, page 000f0000 reserved twice.
On node 0 totalpages: 524272
zone(0): 4096 pages.
zone(1): 225280 pages.
zone(2): 294896 pages.
ACPI: RSDP (v000 DELL ) @ 0x000fdc60
ACPI: RSDT (v001 DELL PE1650 0x00000001 MSFT 0x0100000a) @ 0x000fdc74
ACPI: FADT (v001 DELL PE1650 0x00000001 MSFT 0x0100000a) @ 0x000fdca4
ACPI: MADT (v001 DELL PE1650 0x00000001 MSFT 0x0100000a) @ 0x000fdd18
ACPI: SPCR (v001 DELL PE1650 0x00000001 MSFT 0x0100000a) @ 0x000fdd82
ACPI: DSDT (v001 DELL PE1650 0x00000001 MSFT 0x0100000a) @ 0x00000000
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 Pentium(tm) Pro APIC version 17
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] disabled)
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
ACPI: IOAPIC (id[0x01] address[0xfec00000] global_irq_base[0x0])
IOAPIC[0]: Assigned apic_id 1
IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, IRQ 0-15
ACPI: IOAPIC (id[0x02] address[0xfec01000] global_irq_base[0x10])
IOAPIC[1]: Assigned apic_id 2
IOAPIC[1]: apic_id 2, version 17, address 0xfec01000, IRQ 16-31
ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high edge)
ACPI BALANCE SET
Using ACPI (MADT) for SMP configuration information
Kernel command line: ip=172.16.4.183:::255.255.252.0:rack099-xen:eth0:off root=/dev/sda1 console=tty0 console=ttyS0 -b
Initializing CPU#0
Detected 1396.496 MHz processor.
Console: colour VGA+ 80x25
Calibrating delay loop... 2785.28 BogoMIPS
Memory: 2068540k/2097088k available (2125k kernel code, 28164k reserved, 689k data, 364k init, 1179584k highmem)
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode cache hash table entries: 131072 (order: 8, 1048576 bytes)
Mount cache hash table entries: 512 (order: 0, 4096 bytes)
Buffer cache hash table entries: 131072 (order: 7, 524288 bytes)
Page-cache hash table entries: 524288 (order: 9, 2097152 bytes)
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: L2 cache: 512K
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
POSIX conformance testing by UNIFIX
mtrr: v1.40 (20010327) Richard Gooch (rgooch@atnf.csiro.au)
mtrr: detected mtrr type: Intel
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: L2 cache: 512K
Intel machine check reporting enabled on CPU#0.
CPU0: Intel(R) Pentium(R) III CPU family 1400MHz stepping 01
per-CPU timeslice cutoff: 1462.55 usecs.
enabled ExtINT on CPU#0
ESR value before enabling vector: 00000000
ESR value after enabling vector: 00000000
Error: only one processor found.
ENABLING IO-APIC IRQs
..TIMER: vector=0x31 pin1=-1 pin2=0
...trying to set up timer (IRQ0) through the 8259A ...
..... (found pin 0) ...works.
Using local APIC timer interrupts.
calibrating APIC timer ...
..... CPU clock speed is 1396.4558 MHz.
..... host bus clock speed is 132.9956 MHz.
cpu: 0, clocks: 1329956, slice: 664978
CPU0<T0:1329952,T1:664960,D:14,S:664978,C:1329956>
Waiting on wait_init_idle (map = 0x0)
All processors have done init_idle
ACPI: Subsystem revision 20040116
PCI: PCI BIOS revision 2.10 entry at 0xfc6ce, last bus=2
PCI: Using configuration type 1
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: System [ACPI] (supports S0 S4 S5)
ACPI: PCI Root Bridge [PCI0] (00:00)
PCI: Probing PCI hardware (bus 00)
PCI: Ignoring BAR0-3 of IDE controller 00:0f.1
ACPI: PCI Root Bridge [PCI1] (00:01)
PCI: Probing PCI hardware (bus 01)
ACPI: PCI Root Bridge [PCI2] (00:02)
PCI: Probing PCI hardware (bus 02)
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 *5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 *7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKC] (IRQs *3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 *7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKI] (IRQs 3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKJ] (IRQs 3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKK] (IRQs 3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKL] (IRQs 3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKM] (IRQs 3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKN] (IRQs 3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKO] (IRQs 3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LNKP] (IRQs 3 4 5 6 7 9 11 12 14)
ACPI: PCI Interrupt Link [LUSB] (IRQs 3 4 5 6 7 *11 12 14)
PCI: Probing PCI hardware
ACPI: PCI Interrupt Link [LUSB] enabled at IRQ 11
testing the IO APIC.......................
.................................... done.
PCI: Using ACPI for IRQ routing
PCI: if you experience problems, try using option 'pci=noacpi' or even 'acpi=off'
Linux NET4.0 for Linux 2.4
Based upon Swansea University Computer Society NET3.039
Initializing RT netlink socket
Starting kswapd
allocated 32 pages and 32 bhs reserved for the highmem bounces
Journalled Block Device driver loaded
Detected PS/2 Mouse Port.
pty: 256 Unix98 ptys configured
keyboard: Timeout - AT keyboard not present?(ed)
keyboard: Timeout - AT keyboard not present?(f4)
Serial driver version 5.05c (2001-07-08) with MANY_PORTS SHARE_IRQ SERIAL_PCI enabled
ttyS00 at 0x03f8 (irq = 4) is a 16550A
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
Intel(R) PRO/1000 Network Driver - version 5.2.20-k1
Copyright (c) 1999-2003 Intel Corporation.
eth0: Intel(R) PRO/1000 Network Connection
eth1: Intel(R) PRO/1000 Network Connection
pcnet32.c:v1.27a 10.02.2002 tsbogend@alpha.franken.de
Uniform Multi-Platform E-IDE driver Revision: 7.00beta4-2.4
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
SvrWks CSB5: IDE controller at PCI slot 00:0f.1
SvrWks CSB5: chipset revision 147
SvrWks CSB5: not 100% native mode: will probe irqs later
ide0: BM-DMA at 0x08b0-0x08b7, BIOS settings: hda:DMA, hdb:pio
ide1: BM-DMA at 0x08b8-0x08bf, BIOS settings: hdc:pio, hdd:pio
hda: SAMSUNG CD-ROM SN-124, ATAPI CD/DVD-ROM drive
hda: Disabling (U)DMA for SAMSUNG CD-ROM SN-124
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
SCSI subsystem driver Revision: 1.00
scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.36
<Adaptec aic7899 Ultra160 SCSI adapter>
aic7899: Ultra160 Wide Channel A, SCSI Id=7, 32/253 SCBs
scsi1 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.36
<Adaptec aic7899 Ultra160 SCSI adapter>
aic7899: Ultra160 Wide Channel B, SCSI Id=7, 32/253 SCBs
blk: queue f7ba6818, I/O limit 4095Mb (mask 0xffffffff)
(scsi0:A:0): 160.000MB/s transfers (80.000MHz DT, offset 100, 16bit)
Vendor: HITACHI Model: DK32DJ-18MC Rev: D4D4
Type: Direct-Access ANSI SCSI revision: 03
blk: queue f7ba6618, I/O limit 4095Mb (mask 0xffffffff)
Vendor: PE/PV Model: 1x3 SCSI BP Rev: 0.28
Type: Processor ANSI SCSI revision: 02
blk: queue f7ba6e18, I/O limit 4095Mb (mask 0xffffffff)
scsi0:A:0:0: Tagged Queuing enabled. Depth 253
megaraid: v2.10.1 (Release Date: Wed Dec 3 15:34:42 EST 2003)
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
SCSI device sda: 35566478 512-byte hdwr sectors (18210 MB)
Partition check:
sda: sda1 sda2 sda3
Fusion MPT base driver 2.05.11.03
Copyright (c) 1999-2003 LSI Logic Corporation
mptbase: 0 MPT adapters found, 0 installed.
Fusion MPT SCSI Host driver 2.05.11.03
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP, IGMP
IP: routing cache hash table of 16384 buckets, 128Kbytes
TCP: Hash tables configured (established 262144 bind 65536)
e1000: eth0 NIC Link is Up 1000 Mbps Full Duplex
IP-Config: Complete:
device=eth0, addr=172.16.4.183, mask=255.255.252.0, gw=255.255.255.255,
host=rack099-xen, domain=, nis-domain=(none),
bootserver=255.255.255.255, rootserver=255.255.255.255, rootpath=
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 364k freed
INIT: version 2.86 booting
+++++++++++++++++++++++++
+++++++++++++++++++++++++
Xen boot messages
+++++++++++++++++++++++++
kernel /boot/xen.gz dom0_mem=40000 com1=9600,8n1
[Multiboot-elf, <0x100000:0x3d3d0:0x35170>, shtab=0x173078, entry=0x100000]
module /boot/vmlinuz-2.4.27-xen0 ip=172.16.4.183:::255.255.252.0:rack099-xen:et
h0:off root=/dev/sda1 console=tty0 console=ttyS0
[Multiboot-module @ 0x174000, 0x3068fc bytes]
boot
__ __ ____ ___
\ \/ /___ _ __ |___ \ / _ \
\ // _ \ '_ \ __) || | | |
/ \ __/ | | | / __/ | |_| |
/_/\_\___|_| |_| |_____(_)___/
http://www.cl.cam.ac.uk/netos/xen
University of Cambridge Computer Laboratory
Xen version 2.0 (becker@cs.duke.edu) (gcc version 3.3.4 (Debian 1:3.3.4-6sarge1)) Wed Nov 10 11:15:00 EST 2004
Latest ChangeSet: information unavailable
(XEN) Initialised 2047MB memory (524272 pages) on a 2047MB machine
(XEN) Xen heap size is 10740KB
(XEN) CPU0: Before vendor init, caps: 0383fbff 00000000 00000000, vendor = 0
(XEN) CPU caps: 0383fbff 00000000 00000000 00000000
(XEN) found SMP MP-table at 000fe710
(XEN) Memory Reservation 0xfe710, 4096 bytes
(XEN) Memory Reservation 0xf0000, 4096 bytes
(XEN) ACPI: RSDP (v000 DELL ) @ 0x000fdc60
(XEN) ACPI: RSDT (v001 DELL PE1650 0x00000001 MSFT 0x0100000a) @ 0x000fdc74
(XEN) ACPI: FADT (v001 DELL PE1650 0x00000001 MSFT 0x0100000a) @ 0x000fdca4
(XEN) ACPI: MADT (v001 DELL PE1650 0x00000001 MSFT 0x0100000a) @ 0x000fdd18
(XEN) ACPI: SPCR (v001 DELL PE1650 0x00000001 MSFT 0x0100000a) @ 0x000fdd82
(XEN) ACPI: DSDT (v001 DELL PE1650 0x00000001 MSFT 0x0100000a) @ 0x00000000
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 Pentium(tm) Pro APIC version 17
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] disabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) Using ACPI for processor (LAPIC) configuration information
(XEN) Intel MultiProcessor Specification v1.4
(XEN) Virtual Wire compatibility mode.
(XEN) OEM ID: DELL Product ID: PE 011B APIC at: 0xFEE00000
(XEN) I/O APIC #1 Version 17 at 0xFEC00000.
(XEN) I/O APIC #2 Version 17 at 0xFEC01000.
(XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
(XEN) Processors: 1
(XEN) Using scheduler: Borrowed Virtual Time (bvt)
(XEN) Initializing CPU#0
(XEN) Detected 1396.476 MHz processor.
(XEN) CPU0: Before vendor init, caps: 0383fbff 00000000 00000000, vendor = 0
(XEN) CPU caps: 0383fbff 00000000 00000000 00000000
(XEN) CPU0 booted
(XEN) enabled ExtINT on CPU#0
(XEN) ESR value before enabling vector: 00000000
(XEN) ESR value after enabling vector: 00000000
(XEN) Error: only one processor found.
(XEN) ENABLING IO-APIC IRQs
(XEN) Setting 1 in the phys_id_present_map
(XEN) ...changing IO-APIC physical APIC ID to 1 ... ok.
(XEN) Setting 2 in the phys_id_present_map
(XEN) ...changing IO-APIC physical APIC ID to 2 ... ok.
(XEN) init IO_APIC IRQs
(XEN) ..TIMER: vector=0x41 pin1=2 pin2=0
(XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
(XEN) ...trying to set up timer (IRQ0) through the 8259A ...
(XEN) ..... (found pin 0) ...works.
(XEN) Using local APIC timer interrupts.
(XEN) Calibrating APIC timer for CPU0...
(XEN) ..... CPU speed is 1396.4537 MHz.
(XEN) ..... Bus speed is 132.9955 MHz.
(XEN) ..... bus_scale = 0x00008830
(XEN) Time init:
(XEN) .... System Time: 20000234ns
(XEN) .... cpu_freq: 00000000:533C8BD0
(XEN) .... scale: 00000001:6EA30E07
(XEN) .... Wall Clock: 1100530888s 190000us
(XEN) PCI: PCI BIOS revision 2.10 entry at 0xfc6ce, last bus=2
(XEN) PCI: Using configuration type 1
(XEN) PCI: Probing PCI hardware
(XEN) PCI: Probing PCI hardware (bus 00)
(XEN) PCI: Ignoring BAR0-3 of IDE controller 00:0f.1
(XEN) PCI: Discovered primary peer bus 01 [IRQ]
(XEN) PCI: Discovered primary peer bus 02 [IRQ]
(XEN) PCI: Using IRQ router ServerWorks [1166/0201] at 00:0f.0
(XEN) PCI->APIC IRQ transform: (B1,I2,P0) -> 17
(XEN) PCI->APIC IRQ transform: (B1,I4,P0) -> 16
(XEN) PCI->APIC IRQ transform: (B1,I6,P0) -> 18
(XEN) PCI->APIC IRQ transform: (B1,I6,P1) -> 19
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen-ELF header found: 'GUEST_OS=linux,GUEST_VER=2.4,XEN_VER=2.0,VIRT_BASE=0xC0000000'
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Kernel image: 02800000->02b068fc
(XEN) Initrd image: 00000000->00000000
(XEN) Dom0 alloc.: 02c00000->05310000
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: c0100000->c0453e34
(XEN) Init. ramdisk: c0454000->c0454000
(XEN) Phys-Mach map: c0454000->c045dc40
(XEN) Page tables: c045e000->c0461000
(XEN) Start info: c0461000->c0462000
(XEN) Boot stack: c0462000->c0463000
(XEN) TOTAL: c0000000->c0800000
(XEN) ENTRY ADDRESS: c0100000
(XEN) Scrubbing DOM0 RAM: .done.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen).
(XEN) Scrubbing Free RAM: .....................done.
+++++++++++++++++++++++++
+++++++++++++++++++++++++
Xen-0 boot messages
+++++++++++++++++++++++++
Linux version 2.4.27-xen0 (becker@pant) (gcc version 3.3.4 (Debian 1:3.3.4-6sarge1)) #3 Wed Nov 10 11:29:37 EST 2004
On node 0 totalpages: 10000
zone(0): 4096 pages.
zone(1): 5904 pages.
zone(2): 0 pages.
Kernel command line: ip=172.16.4.183:::255.255.252.0:rack099-xen:eth0:off root=/dev/sda1 console=tty0 console=ttyS0
Initializing CPU#0
Xen reported: 1396.476 MHz processor.
Console: colour VGA+ 80x25
Linux version 2.4.27-xen0 (becker@pant) (gcc version 3.3.4 (Debian 1:3.3.4-6sarge1)) #3 Wed Nov 10 11:29:37 EST 2004
On node 0 totalpages: 10000
zone(0): 4096 pages.
zone(1): 5904 pages.
zone(2): 0 pages.
Kernel command line: ip=172.16.4.183:::255.255.252.0:rack099-xen:eth0:off root=/dev/sda1 console=tty0 console=ttyS0
Initializing CPU#0
Xen reported: 1396.476 MHz processor.
Console: colour VGA+ 80x25
Calibrating delay loop... 2791.83 BogoMIPS
Memory: 36056k/40000k available (2245k kernel code, 3944k reserved, 716k data, 112k init, 0k highmem)
Dentry cache hash table entries: 8192 (order: 4, 65536 bytes)
Inode cache hash table entries: 4096 (order: 3, 32768 bytes)
Mount cache hash table entries: 512 (order: 0, 4096 bytes)
Buffer cache hash table entries: 1024 (order: 0, 4096 bytes)
Page-cache hash table entries: 16384 (order: 4, 65536 bytes)
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: L2 cache: 512K
CPU: Intel(R) Pentium(R) III CPU family 1400MHz stepping 01
POSIX conformance testing by UNIFIX
PCI: Probing PCI hardware
PCI: Probing PCI hardware (bus 00)
PCI: Probing PCI hardware (bus 01)
Linux NET4.0 for Linux 2.4
Based upon Swansea University Computer Society NET3.039
Initializing RT netlink socket
Starting kswapd
Journalled Block Device driver loaded
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
Event-channel device installed.
Xen virtual console successfully installed as ttyS
Starting Xen Balloon driver
Detected PS/2 Mouse Port.
pty: 256 Unix98 ptys configured
keyboard: Timeout - AT keyboard not present?(ed)
keyboard: Timeout - AT keyboard not present?(f4)
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: loaded (max 8 devices)
Intel(R) PRO/1000 Network Driver - version 5.2.52-k3
Copyright (c) 1999-2004 Intel Corporation.
PCI: Obtained IRQ 17 for device 01:02.0
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
PCI: Obtained IRQ 16 for device 01:04.0
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
pcnet32.c:v1.30c 05.25.2004 tsbogend@alpha.franken.de
Uniform Multi-Platform E-IDE driver Revision: 7.00beta4-2.4
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
SvrWks CSB5: IDE controller at PCI slot 00:0f.1
SvrWks CSB5: chipset revision 147
SvrWks CSB5: not 100% native mode: will probe irqs later
ide0: BM-DMA at 0x08b0-0x08b7, BIOS settings: hda:DMA, hdb:pio
ide1: BM-DMA at 0x08b8-0x08bf, BIOS settings: hdc:pio, hdd:pio
hda: SAMSUNG CD-ROM SN-124, ATAPI CD/DVD-ROM drive
hda: Disabling (U)DMA for SAMSUNG CD-ROM SN-124
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hda: attached ide-cdrom driver.
hda: ATAPI 24X CD-ROM drive, 128kB Cache
Uniform CD-ROM driver Revision: 3.12
SCSI subsystem driver Revision: 1.00
Red Hat/Adaptec aacraid driver (1.1-3 Nov 10 2004 11:29:04)
PCI: Obtained IRQ 18 for device 01:06.0
PCI: Obtained IRQ 19 for device 01:06.1
scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.36
<Adaptec aic7899 Ultra160 SCSI adapter>
aic7899: Ultra160 Wide Channel A, SCSI Id=7, 32/253 SCBs
scsi1 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.36
<Adaptec aic7899 Ultra160 SCSI adapter>
aic7899: Ultra160 Wide Channel B, SCSI Id=7, 32/253 SCBs
(scsi0:A:0): 160.000MB/s transfers (80.000MHz DT, offset 100, 16bit)
Vendor: HITACHI Model: DK32DJ-18MC Rev: D4D4
Type: Direct-Access ANSI SCSI revision: 03
Vendor: PE/PV Model: 1x3 SCSI BP Rev: 0.28
Type: Processor ANSI SCSI revision: 02
scsi0:A:0:0: Tagged Queuing enabled. Depth 32
megaraid: v1.18k (Release Date: Thu Aug 28 10:05:11 EDT 2003)
megaraid: no BIOS enabled.
scsi2 : SCSI host adapter emulation for IDE ATAPI devices
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
SCSI device sda: 35566478 512-byte hdwr sectors (18210 MB)
Partition check:
sda: sda1 sda2 sda3
Attached scsi generic sg1 at scsi0, channel 0, id 6, lun 0, type 3
Initializing Cryptographic API
Initialising Xen netif backend
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 2048 bind 4096)
e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex
IP-Config: Complete:
device=eth0, addr=172.16.4.183, mask=255.255.252.0, gw=255.255.255.255,
host=rack099-xen, domain=, nis-domain=(none),
bootserver=255.255.255.255, rootserver=255.255.255.255, rootpath=
ip_tables: (C) 2000-2002 Netfilter core team
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
NET4: Ethernet Bridge 008 for NET4.0
Bridge firewalling registered
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
kjournald starting. Commit interval 5 seconds
EXT3 FS 2.4-0.9.19, 19 August 2002 on sd(8,1), internal journal
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem).
Freeing unused kernel memory: 112k freed
INIT: version 2.86 booting
+++++++++++++++++++++++++
+++++++++++++++++++++++++
xm messages
+++++++++++++++++++++++++
# xm info
system : Linux
host : rack099-xen
release : 2.4.27-xen0
version : #3 Wed Nov 10 11:29:37 EST 2004
machine : i686
cores : 1
hyperthreads_per_core : 1
cpu_mhz : 1396
memory : 2047
free_memory : 1485
rack099-xen:~# xm info
system : Linux
host : rack099-xen
release : 2.4.27-xen0
version : #3 Wed Nov 10 11:29:37 EST 2004
machine : i686
cores : 1
hyperthreads_per_core : 1
cpu_mhz : 1396
memory : 2047
free_memory : 1485
# xm list
Name Id Mem(MB) CPU State Time(s) Console
Domain-0 0 34 0 r---- 174.5
grant 1 499 0 -b--- 53.5 9601
+++++++++++++++++++++++++
+++++++++++++++++++++++++
iperfrun stock to stock
+++++++++++++++++++++++++
# iperf -c 172.16.7.3
------------------------------------------------------------
Client connecting to 172.16.7.3, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 5] local 172.16.4.183 port 32812 connected with 172.16.7.3 port 5001
[ ID] Interval Transfer Bandwidth
[ 5] 0.0-10.0 sec 1.10 GBytes 941 Mbits/sec
+++++++++++++++++++++++++
-------------------------------------------------------
This SF.Net email is sponsored by: InterSystems CACHE
FREE OODBMS DOWNLOAD - A multidimensional database that combines
robust object and relational technologies, making it a perfect match
for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2004-11-15 16:01 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2004-11-10 21:48 etherbridge bottleneck David Becker
2004-11-11 12:00 ` Bin Ren
2004-11-11 14:56 ` David Becker
-- strict thread matches above, loose matches on Subject: below --
2004-11-12 8:42 Ian Pratt
2004-11-14 19:53 Ian Pratt
2004-11-15 16:01 ` David Becker
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.