* Re: Xen & I/O in clusters - problems!
@ 2004-10-15 10:48 Håvard Bjerke
2004-10-15 14:02 ` Mark A. Williamson
0 siblings, 1 reply; 8+ messages in thread
From: Håvard Bjerke @ 2004-10-15 10:48 UTC (permalink / raw)
To: xen-devel; +Cc: mark.williamson, Rune.Johan.Andresen
Here are some details on our horrible benchmark results:
System setup:
Ethernet controller:
Intel Corp. 82547GI Gigabit Ethernet Controller
Benchmark:
PARKBENCH low-level pingpong benchmark comms1_mpi
MPI library:
Scali MPI (http://www.scali.com/)
Xen distribution:
2.0 stable, downloaded with bitkeeper
Xen configuration:
default, i.e. we just leave config-2.6.8.1-xen0 the way it is
CPU:
Single Intel(R) Pentium(R) 4 CPU 3.40GHz
Memory:
1 GB
Bandwidth measurements:
Between two nodes running plain redhat EL3 with kernel 2.4.21-15.EL:
786.034 MByte/s
Between two nodes each running only xen domain 0:
56.480 MByte/s
A graph over the measurements - x = message length, y = time:
http://www.idi.ntnu.no/~havarbj/tmp/plot.png
Cheers,
Havard
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Xen & I/O in clusters - problems!
2004-10-15 10:48 Xen & I/O in clusters - problems! Håvard Bjerke
@ 2004-10-15 14:02 ` Mark A. Williamson
2004-10-15 16:27 ` Håvard Bjerke
0 siblings, 1 reply; 8+ messages in thread
From: Mark A. Williamson @ 2004-10-15 14:02 UTC (permalink / raw)
To: H�vard Bjerke; +Cc: xen-devel, Rune.Johan.Andresen
> Between two nodes running plain redhat EL3 with kernel 2.4.21-15.EL:
>
> 786.034 MByte/s
>
> Between two nodes each running only xen domain 0:
>
> 56.480 MByte/s
That's surprising - I'd have expected any performance problems to involve
unpriv domains somehow. We've never had any performance problems when just
running domain 0, even when the code was still under development...
It'd be interesting to see your config file (I know it's just the default but
it'd be interesting for comparison as there's no obvious reason for your
problems).
Cheers,
Mark
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Re: Xen & I/O in clusters - problems!
2004-10-15 14:02 ` Mark A. Williamson
@ 2004-10-15 16:27 ` Håvard Bjerke
2004-10-16 20:05 ` John Enok Vollestad
0 siblings, 1 reply; 8+ messages in thread
From: Håvard Bjerke @ 2004-10-15 16:27 UTC (permalink / raw)
To: Mark A. Williamson; +Cc: xen-devel, Rune.Johan.Andresen
[-- Attachment #1: Type: text/plain, Size: 1540 bytes --]
On Fri, Oct 15, 2004 at 02:02:31PM +0000, Mark A. Williamson wrote:
> > Between two nodes running plain redhat EL3 with kernel 2.4.21-15.EL:
> >
> > 786.034 MByte/s
> >
> > Between two nodes each running only xen domain 0:
> >
> > 56.480 MByte/s
>
> That's surprising - I'd have expected any performance problems to involve
> unpriv domains somehow. We've never had any performance problems when just
> running domain 0, even when the code was still under development...
>
What's possibly even more funny is that when I do the same benchmark localhost <-> localhost, ie. through the loopback interface, on domain 0, the bandwidth is halved. CPU use is ~100% during both these benchmarks on domain 0 (50% per process in the last benchmark). This indicates to me that the bandwidth depends on CPU resources. Some heavy processing is happening somewhere.
I suspect this might have something to do with the MPI library scaMPI, which is supposed to be more closely linked with the lower layers of the OSI protocol stack or something. I will investigate it further.
Another thing that might be worth noting is that we're using the 2.6.8.1 kernel on domain 0 as opposed to a 2.4 kernel. I don't think that would make any difference, though (except for the fact that modules aren't loaded, but I don't think we need any of them anyway).
> It'd be interesting to see your config file (I know it's just the default but
> it'd be interesting for comparison as there's no obvious reason for your
> problems).
>
Attached
Cheers,
Havard
[-- Attachment #2: config-2.6.8.1-xen0 --]
[-- Type: text/plain, Size: 22706 bytes --]
#
# Automatically generated make config: don't edit
#
CONFIG_XEN=y
CONFIG_ARCH_XEN=y
CONFIG_NO_IDLE_HZ=y
#
# XEN
#
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PHYSDEV_ACCESS=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
# CONFIG_XEN_NETDEV_FRONTEND_PIPELINED_TRANSMITTER is not set
CONFIG_XEN_WRITABLE_PAGETABLES=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_FOREIGN_PAGES=y
CONFIG_HAVE_ARCH_DEV_ALLOC_SKB=y
CONFIG_X86=y
# CONFIG_X86_64 is not set
#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
# CONFIG_CLEAN_COMPILE is not set
CONFIG_BROKEN=y
CONFIG_BROKEN_ON_SMP=y
#
# General setup
#
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
# CONFIG_POSIX_MQUEUE is not set
# CONFIG_BSD_PROCESS_ACCT is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
CONFIG_LOG_BUF_SHIFT=14
CONFIG_HOTPLUG=y
# CONFIG_IKCONFIG is not set
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
CONFIG_OBSOLETE_MODPARM=y
# CONFIG_MODVERSIONS is not set
CONFIG_KMOD=y
#
# X86 Processor Configuration
#
CONFIG_XENARCH="i386"
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
CONFIG_MPENTIUM4=y
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=7
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
# CONFIG_HPET_TIMER is not set
# CONFIG_HPET_EMULATE_RTC is not set
# CONFIG_SMP is not set
CONFIG_PREEMPT=y
CONFIG_X86_CPUID=y
#
# Firmware Drivers
#
# CONFIG_EDD is not set
CONFIG_NOHIGHMEM=y
# CONFIG_HIGHMEM4G is not set
CONFIG_HAVE_DEC_LOCK=y
# CONFIG_REGPARM is not set
#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
CONFIG_PCI_GODIRECT=y
# CONFIG_PCI_GOANY is not set
CONFIG_PCI_DIRECT=y
CONFIG_PCI_LEGACY_PROC=y
# CONFIG_PCI_NAMES is not set
CONFIG_ISA=y
# CONFIG_EISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set
#
# PCMCIA/CardBus support
#
CONFIG_PCMCIA=m
# CONFIG_PCMCIA_DEBUG is not set
CONFIG_YENTA=m
CONFIG_CARDBUS=y
# CONFIG_PD6729 is not set
# CONFIG_I82092 is not set
# CONFIG_I82365 is not set
# CONFIG_TCIC is not set
CONFIG_PCMCIA_PROBE=y
#
# PCI Hotplug Support
#
# CONFIG_HOTPLUG_PCI is not set
#
# Kernel hacking
#
CONFIG_DEBUG_KERNEL=y
CONFIG_EARLY_PRINTK=y
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_SLAB is not set
CONFIG_MAGIC_SYSRQ=y
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_PAGEALLOC is not set
# CONFIG_DEBUG_INFO is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_FRAME_POINTER is not set
# CONFIG_4KSTACKS is not set
CONFIG_X86_BIOS_REBOOT=y
CONFIG_X86_STD_RESOURCES=y
CONFIG_PC=y
#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
# CONFIG_BINFMT_AOUT is not set
# CONFIG_BINFMT_MISC is not set
#
# Device Drivers
#
#
# Generic Driver Options
#
# CONFIG_STANDALONE is not set
CONFIG_PREVENT_FIRMWARE_BUILD=y
# CONFIG_FW_LOADER is not set
# CONFIG_DEBUG_DRIVER is not set
#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set
#
# Parallel port support
#
# CONFIG_PARPORT is not set
#
# Plug and Play support
#
# CONFIG_PNP is not set
#
# Block devices
#
# CONFIG_BLK_DEV_FD is not set
# CONFIG_BLK_DEV_XD is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
CONFIG_BLK_DEV_NBD=y
# CONFIG_BLK_DEV_SX8 is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_BLK_DEV_INITRD=y
# CONFIG_LBD is not set
#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y
#
# Please see Documentation/ide.txt for help/info on IDE drives
#
# CONFIG_BLK_DEV_IDE_SATA is not set
# CONFIG_BLK_DEV_HD_IDE is not set
CONFIG_BLK_DEV_IDEDISK=y
# CONFIG_IDEDISK_MULTI_MODE is not set
# CONFIG_BLK_DEV_IDECS is not set
CONFIG_BLK_DEV_IDECD=y
# CONFIG_BLK_DEV_IDETAPE is not set
# CONFIG_BLK_DEV_IDEFLOPPY is not set
# CONFIG_BLK_DEV_IDESCSI is not set
# CONFIG_IDE_TASK_IOCTL is not set
# CONFIG_IDE_TASKFILE_IO is not set
#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=y
# CONFIG_BLK_DEV_CMD640 is not set
CONFIG_BLK_DEV_IDEPCI=y
# CONFIG_IDEPCI_SHARE_IRQ is not set
# CONFIG_BLK_DEV_OFFBOARD is not set
CONFIG_BLK_DEV_GENERIC=y
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_RZ1000 is not set
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
CONFIG_BLK_DEV_ADMA=y
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not set
# CONFIG_BLK_DEV_AMD74XX is not set
# CONFIG_BLK_DEV_ATIIXP is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5520 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_SC1200 is not set
CONFIG_BLK_DEV_PIIX=y
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
CONFIG_BLK_DEV_SVWKS=y
# CONFIG_BLK_DEV_SIIMAGE is not set
# CONFIG_BLK_DEV_SIS5513 is not set
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
# CONFIG_BLK_DEV_VIA82CXXX is not set
# CONFIG_IDE_ARM is not set
# CONFIG_IDE_CHIPSETS is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set
#
# SCSI device support
#
CONFIG_SCSI=y
CONFIG_SCSI_PROC_FS=y
#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
# CONFIG_BLK_DEV_SR is not set
# CONFIG_CHR_DEV_SG is not set
#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
# CONFIG_SCSI_MULTI_LUN is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set
#
# SCSI Transport Attributes
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set
#
# SCSI low-level drivers
#
CONFIG_BLK_DEV_3W_XXXX_RAID=y
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_7000FASST is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AHA152X is not set
# CONFIG_SCSI_AHA1542 is not set
CONFIG_SCSI_AACRAID=y
CONFIG_SCSI_AIC7XXX=y
CONFIG_AIC7XXX_CMDS_PER_DEVICE=32
CONFIG_AIC7XXX_RESET_DELAY_MS=15000
# CONFIG_AIC7XXX_BUILD_FIRMWARE is not set
CONFIG_AIC7XXX_DEBUG_ENABLE=y
CONFIG_AIC7XXX_DEBUG_MASK=0
CONFIG_AIC7XXX_REG_PRETTY_PRINT=y
# CONFIG_SCSI_AIC7XXX_OLD is not set
CONFIG_SCSI_AIC79XX=y
CONFIG_AIC79XX_CMDS_PER_DEVICE=32
CONFIG_AIC79XX_RESET_DELAY_MS=15000
# CONFIG_AIC79XX_BUILD_FIRMWARE is not set
# CONFIG_AIC79XX_ENABLE_RD_STRM is not set
CONFIG_AIC79XX_DEBUG_ENABLE=y
CONFIG_AIC79XX_DEBUG_MASK=0
CONFIG_AIC79XX_REG_PRETTY_PRINT=y
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_SCSI_ADVANSYS is not set
# CONFIG_SCSI_IN2000 is not set
CONFIG_SCSI_MEGARAID=y
CONFIG_SCSI_SATA=y
# CONFIG_SCSI_SATA_SVW is not set
CONFIG_SCSI_ATA_PIIX=y
# CONFIG_SCSI_SATA_NV is not set
CONFIG_SCSI_SATA_PROMISE=y
CONFIG_SCSI_SATA_SX4=y
CONFIG_SCSI_SATA_SIL=y
# CONFIG_SCSI_SATA_SIS is not set
# CONFIG_SCSI_SATA_VIA is not set
# CONFIG_SCSI_SATA_VITESSE is not set
CONFIG_SCSI_BUSLOGIC=y
# CONFIG_SCSI_OMIT_FLASHPOINT is not set
# CONFIG_SCSI_CPQFCTS is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_DTC3280 is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_EATA_PIO is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_GENERIC_NCR5380 is not set
# CONFIG_SCSI_GENERIC_NCR5380_MMIO is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_NCR53C406A is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_PAS16 is not set
# CONFIG_SCSI_PCI2000 is not set
# CONFIG_SCSI_PCI2220I is not set
# CONFIG_SCSI_PSI240I is not set
# CONFIG_SCSI_QLOGIC_FAS is not set
# CONFIG_SCSI_QLOGIC_ISP is not set
# CONFIG_SCSI_QLOGIC_FC is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
CONFIG_SCSI_QLA2XXX=y
# CONFIG_SCSI_QLA21XX is not set
# CONFIG_SCSI_QLA22XX is not set
# CONFIG_SCSI_QLA2300 is not set
# CONFIG_SCSI_QLA2322 is not set
# CONFIG_SCSI_QLA6312 is not set
# CONFIG_SCSI_QLA6322 is not set
# CONFIG_SCSI_SEAGATE is not set
# CONFIG_SCSI_SYM53C416 is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_T128 is not set
# CONFIG_SCSI_U14_34F is not set
# CONFIG_SCSI_ULTRASTOR is not set
# CONFIG_SCSI_NSP32 is not set
# CONFIG_SCSI_DEBUG is not set
#
# PCMCIA SCSI adapter support
#
# CONFIG_PCMCIA_AHA152X is not set
# CONFIG_PCMCIA_FDOMAIN is not set
# CONFIG_PCMCIA_NINJA_SCSI is not set
# CONFIG_PCMCIA_QLOGIC is not set
# CONFIG_PCMCIA_SYM53C500 is not set
#
# Old CD-ROM drivers (not SCSI, not IDE)
#
# CONFIG_CD_NO_IDESCSI is not set
#
# Multi-device support (RAID and LVM)
#
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
# CONFIG_MD_LINEAR is not set
CONFIG_MD_RAID0=y
CONFIG_MD_RAID1=y
CONFIG_MD_RAID5=y
# CONFIG_MD_RAID6 is not set
# CONFIG_MD_MULTIPATH is not set
CONFIG_BLK_DEV_DM=y
# CONFIG_DM_CRYPT is not set
CONFIG_DM_SNAPSHOT=y
CONFIG_DM_MIRROR=y
# CONFIG_DM_ZERO is not set
#
# Fusion MPT device support
#
# CONFIG_FUSION is not set
#
# IEEE 1394 (FireWire) support
#
# CONFIG_IEEE1394 is not set
#
# I2O device support
#
# CONFIG_I2O is not set
#
# Networking support
#
CONFIG_NET=y
#
# Networking options
#
CONFIG_PACKET=y
# CONFIG_PACKET_MMAP is not set
# CONFIG_NETLINK_DEV is not set
CONFIG_UNIX=y
# CONFIG_NET_KEY is not set
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
# CONFIG_IP_ADVANCED_ROUTER is not set
CONFIG_IP_PNP=y
CONFIG_IP_PNP_DHCP=y
# CONFIG_IP_PNP_BOOTP is not set
# CONFIG_IP_PNP_RARP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_ARPD is not set
# CONFIG_SYN_COOKIES is not set
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
#
# IP: Virtual Server Configuration
#
# CONFIG_IP_VS is not set
# CONFIG_IPV6 is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
CONFIG_BRIDGE_NETFILTER=y
#
# IP: Netfilter Configuration
#
CONFIG_IP_NF_CONNTRACK=m
CONFIG_IP_NF_FTP=m
# CONFIG_IP_NF_IRC is not set
# CONFIG_IP_NF_TFTP is not set
# CONFIG_IP_NF_AMANDA is not set
# CONFIG_IP_NF_QUEUE is not set
CONFIG_IP_NF_IPTABLES=m
# CONFIG_IP_NF_MATCH_LIMIT is not set
# CONFIG_IP_NF_MATCH_IPRANGE is not set
# CONFIG_IP_NF_MATCH_MAC is not set
# CONFIG_IP_NF_MATCH_PKTTYPE is not set
# CONFIG_IP_NF_MATCH_MARK is not set
# CONFIG_IP_NF_MATCH_MULTIPORT is not set
# CONFIG_IP_NF_MATCH_TOS is not set
# CONFIG_IP_NF_MATCH_RECENT is not set
# CONFIG_IP_NF_MATCH_ECN is not set
# CONFIG_IP_NF_MATCH_DSCP is not set
# CONFIG_IP_NF_MATCH_AH_ESP is not set
# CONFIG_IP_NF_MATCH_LENGTH is not set
# CONFIG_IP_NF_MATCH_TTL is not set
# CONFIG_IP_NF_MATCH_TCPMSS is not set
# CONFIG_IP_NF_MATCH_HELPER is not set
# CONFIG_IP_NF_MATCH_STATE is not set
# CONFIG_IP_NF_MATCH_CONNTRACK is not set
# CONFIG_IP_NF_MATCH_OWNER is not set
# CONFIG_IP_NF_MATCH_PHYSDEV is not set
# CONFIG_IP_NF_FILTER is not set
# CONFIG_IP_NF_NAT is not set
# CONFIG_IP_NF_MANGLE is not set
# CONFIG_IP_NF_TARGET_LOG is not set
# CONFIG_IP_NF_TARGET_ULOG is not set
# CONFIG_IP_NF_TARGET_TCPMSS is not set
# CONFIG_IP_NF_ARPTABLES is not set
# CONFIG_IP_NF_COMPAT_IPCHAINS is not set
# CONFIG_IP_NF_COMPAT_IPFWADM is not set
# CONFIG_IP_NF_RAW is not set
# CONFIG_IP_NF_MATCH_ADDRTYPE is not set
# CONFIG_IP_NF_MATCH_REALM is not set
#
# Bridge: Netfilter Configuration
#
# CONFIG_BRIDGE_NF_EBTABLES is not set
#
# SCTP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_SCTP is not set
# CONFIG_ATM is not set
CONFIG_BRIDGE=y
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_HW_FLOWCONTROL is not set
#
# QoS and/or fair queueing
#
# CONFIG_NET_SCHED is not set
# CONFIG_NET_CLS_ROUTE is not set
#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
CONFIG_NETDEVICES=y
# CONFIG_DUMMY is not set
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set
#
# ARCnet devices
#
# CONFIG_ARCNET is not set
#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
CONFIG_MII=y
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
CONFIG_NET_VENDOR_3COM=y
# CONFIG_EL1 is not set
# CONFIG_EL2 is not set
# CONFIG_ELPLUS is not set
# CONFIG_EL16 is not set
# CONFIG_EL3 is not set
# CONFIG_3C515 is not set
CONFIG_VORTEX=y
# CONFIG_TYPHOON is not set
# CONFIG_LANCE is not set
# CONFIG_NET_VENDOR_SMC is not set
# CONFIG_NET_VENDOR_RACAL is not set
#
# Tulip family network device support
#
CONFIG_NET_TULIP=y
# CONFIG_DE2104X is not set
CONFIG_TULIP=y
# CONFIG_TULIP_MWI is not set
# CONFIG_TULIP_MMIO is not set
# CONFIG_TULIP_NAPI is not set
# CONFIG_DE4X5 is not set
# CONFIG_WINBOND_840 is not set
# CONFIG_DM9102 is not set
# CONFIG_PCMCIA_XIRCOM is not set
# CONFIG_PCMCIA_XIRTULIP is not set
# CONFIG_AT1700 is not set
# CONFIG_DEPCA is not set
# CONFIG_HP100 is not set
# CONFIG_NET_ISA is not set
CONFIG_NET_PCI=y
CONFIG_PCNET32=y
# CONFIG_AMD8111_ETH is not set
# CONFIG_ADAPTEC_STARFIRE is not set
# CONFIG_AC3200 is not set
# CONFIG_APRICOT is not set
# CONFIG_B44 is not set
# CONFIG_FORCEDETH is not set
# CONFIG_CS89x0 is not set
# CONFIG_DGRS is not set
# CONFIG_EEPRO100 is not set
CONFIG_E100=y
# CONFIG_E100_NAPI is not set
# CONFIG_FEALNX is not set
# CONFIG_NATSEMI is not set
# CONFIG_NE2K_PCI is not set
# CONFIG_8139CP is not set
CONFIG_8139TOO=y
CONFIG_8139TOO_PIO=y
# CONFIG_8139TOO_TUNE_TWISTER is not set
# CONFIG_8139TOO_8129 is not set
# CONFIG_8139_OLD_RX_RESET is not set
# CONFIG_SIS900 is not set
# CONFIG_EPIC100 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_TLAN is not set
CONFIG_VIA_RHINE=y
# CONFIG_VIA_RHINE_MMIO is not set
# CONFIG_VIA_VELOCITY is not set
# CONFIG_NET_POCKET is not set
#
# Ethernet (1000 Mbit)
#
CONFIG_ACENIC=y
# CONFIG_ACENIC_OMIT_TIGON_I is not set
# CONFIG_DL2K is not set
CONFIG_E1000=y
# CONFIG_E1000_NAPI is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SK98LIN is not set
CONFIG_TIGON3=y
#
# Ethernet (10000 Mbit)
#
# CONFIG_IXGB is not set
# CONFIG_S2IO is not set
#
# Token Ring devices
#
# CONFIG_TR is not set
#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set
#
# PCMCIA network device support
#
# CONFIG_NET_PCMCIA is not set
#
# Wan interfaces
#
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
# CONFIG_SHAPER is not set
# CONFIG_NETCONSOLE is not set
#
# ISDN subsystem
#
# CONFIG_ISDN is not set
#
# Telephony Support
#
# CONFIG_PHONE is not set
#
# Input device support
#
CONFIG_INPUT=y
#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_TSDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set
#
# Input I/O drivers
#
# CONFIG_GAMEPORT is not set
CONFIG_SOUND_GAMEPORT=y
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=y
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PCIPS2 is not set
#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_INPORT is not set
# CONFIG_MOUSE_LOGIBM is not set
# CONFIG_MOUSE_PC110PAD is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set
#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_SERIAL_NONSTANDARD is not set
#
# Serial drivers
#
# CONFIG_SERIAL_8250 is not set
#
# Non-8250 serial port support
#
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
# CONFIG_QIC02_TAPE is not set
#
# IPMI
#
# CONFIG_IPMI_HANDLER is not set
#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_NVRAM is not set
# CONFIG_RTC is not set
# CONFIG_GEN_RTC is not set
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set
#
# Ftape, the floppy tape device driver
#
# CONFIG_FTAPE is not set
# CONFIG_AGP is not set
# CONFIG_DRM is not set
#
# PCMCIA character devices
#
# CONFIG_SYNCLINK_CS is not set
# CONFIG_MWAVE is not set
# CONFIG_RAW_DRIVER is not set
# CONFIG_HANGCHECK_TIMER is not set
#
# I2C support
#
# CONFIG_I2C is not set
#
# Dallas's 1-wire bus
#
# CONFIG_W1 is not set
#
# Misc devices
#
# CONFIG_IBM_ASM is not set
#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set
#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set
#
# Graphics support
#
# CONFIG_FB is not set
# CONFIG_VIDEO_SELECT is not set
#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
# CONFIG_MDA_CONSOLE is not set
CONFIG_DUMMY_CONSOLE=y
#
# Sound
#
# CONFIG_SOUND is not set
#
# USB support
#
# CONFIG_USB is not set
#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set
#
# File systems
#
CONFIG_EXT2_FS=y
# CONFIG_EXT2_FS_XATTR is not set
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_XATTR=y
# CONFIG_EXT3_FS_POSIX_ACL is not set
# CONFIG_EXT3_FS_SECURITY is not set
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=y
CONFIG_REISERFS_FS=y
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
# CONFIG_REISERFS_FS_XATTR is not set
# CONFIG_JFS_FS is not set
# CONFIG_XFS_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not set
# CONFIG_QUOTA is not set
# CONFIG_AUTOFS_FS is not set
# CONFIG_AUTOFS4_FS is not set
#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_ZISOFS_FS=y
# CONFIG_UDF_FS is not set
#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
# CONFIG_NTFS_FS is not set
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
# CONFIG_DEVFS_FS is not set
# CONFIG_DEVPTS_FS_XATTR is not set
CONFIG_TMPFS=y
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_RAMFS=y
#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
#
# Network File Systems
#
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
# CONFIG_NFS_V4 is not set
# CONFIG_NFS_DIRECTIO is not set
CONFIG_NFSD=m
CONFIG_NFSD_V3=y
# CONFIG_NFSD_V4 is not set
CONFIG_NFSD_TCP=y
CONFIG_ROOT_NFS=y
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=m
CONFIG_SUNRPC=y
# CONFIG_RPCSEC_GSS_KRB5 is not set
# CONFIG_SMB_FS is not set
# CONFIG_CIFS is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y
#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
# CONFIG_NLS_ASCII is not set
CONFIG_NLS_ISO8859_1=y
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
# CONFIG_NLS_ISO8859_15 is not set
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
# CONFIG_NLS_UTF8 is not set
#
# Security options
#
# CONFIG_SECURITY is not set
#
# Cryptographic options
#
CONFIG_CRYPTO=y
CONFIG_CRYPTO_HMAC=y
# CONFIG_CRYPTO_NULL is not set
# CONFIG_CRYPTO_MD4 is not set
CONFIG_CRYPTO_MD5=m
CONFIG_CRYPTO_SHA1=m
# CONFIG_CRYPTO_SHA256 is not set
# CONFIG_CRYPTO_SHA512 is not set
CONFIG_CRYPTO_DES=m
# CONFIG_CRYPTO_BLOWFISH is not set
# CONFIG_CRYPTO_TWOFISH is not set
# CONFIG_CRYPTO_SERPENT is not set
# CONFIG_CRYPTO_AES_586 is not set
# CONFIG_CRYPTO_CAST5 is not set
# CONFIG_CRYPTO_CAST6 is not set
# CONFIG_CRYPTO_TEA is not set
# CONFIG_CRYPTO_ARC4 is not set
# CONFIG_CRYPTO_KHAZAD is not set
# CONFIG_CRYPTO_DEFLATE is not set
# CONFIG_CRYPTO_MICHAEL_MIC is not set
CONFIG_CRYPTO_CRC32C=m
# CONFIG_CRYPTO_TEST is not set
#
# Library routines
#
# CONFIG_CRC_CCITT is not set
CONFIG_CRC32=y
CONFIG_LIBCRC32C=y
CONFIG_ZLIB_INFLATE=y
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Xen & I/O in clusters - problems!
2004-10-15 16:27 ` Håvard Bjerke
@ 2004-10-16 20:05 ` John Enok Vollestad
0 siblings, 0 replies; 8+ messages in thread
From: John Enok Vollestad @ 2004-10-16 20:05 UTC (permalink / raw)
To: xen-devel
Håvard Bjerke <Havard.Bjerke <at> idi.ntnu.no> writes:
>
> On Fri, Oct 15, 2004 at 02:02:31PM +0000, Mark A. Williamson wrote:
> > > Between two nodes running plain redhat EL3 with kernel 2.4.21-15.EL:
> > >
> > > 786.034 MByte/s
> > >
> > > Between two nodes each running only xen domain 0:
> > >
> > > 56.480 MByte/s
> >
> > That's surprising - I'd have expected any performance problems to involve
> > unpriv domains somehow. We've never had any performance problems when just
> > running domain 0, even when the code was still under development...
> >
>
> What's possibly even more funny is that when I do the same benchmark localhost
<-> localhost, ie. through
> the loopback interface, on domain 0, the bandwidth is halved. CPU use is ~100%
during both these
> benchmarks on domain 0 (50% per process in the last benchmark). This indicates
to me that the bandwidth
> depends on CPU resources. Some heavy processing is happening somewhere.
>
> I suspect this might have something to do with the MPI library scaMPI, which
is supposed to be more closely
> linked with the lower layers of the OSI protocol stack or something. I will
investigate it further.
Around 2000 the driver used a mix of polling and interrupts to get the latency
down. If I remember correctly there done polling for abount half the time of an
interrupt.
To get the adapters manipulate memory directly the driver also have to allocate
memory in physical continuous blocks. Exact how the reading and writing of this
areas is done through the drivers I do not know but this should not make any
performance hit except if Xen have any issues with MMU manipulation.
You could send SCALI an email..
--
John Enok
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: plan 9 status
@ 2004-10-14 8:10 Keir Fraser
2004-10-14 15:53 ` Mark A. Williamson
0 siblings, 1 reply; 8+ messages in thread
From: Keir Fraser @ 2004-10-14 8:10 UTC (permalink / raw)
To: Ronald G. Minnich; +Cc: Mark A. Williamson, xen-devel, Keir.Fraser
>
>
> On Wed, 13 Oct 2004, Mark A. Williamson wrote:
>
> > I'm curious as to how your source for this port will be distributed...
> > In fact, I don't know how the p9 source is managed - is it possible for
> > you to get the port checked in to the main tree, or will it be available
> > as a separate patch?
>
> I have been told I can get it in. There will be issues as the Xen stuff is
> -- I guess -- GPL and Plna 9 is not, so we're going to get into that
> damned mess.
>
> So I don't know how it will go.
Personally, I'm happy to relicense our 'unprivileged guest' drivers
under a BSD-style license. This would include at least the following
files that contain no Linux code:
linux-2.6.8.1-xen-sparse:
drivers/xen/netfront/*
drivers/xen/blkfront/*
drivers/xen/evtchn/*
drivers/console/*
arch/xen/i386/mm/hypervisor.c
arch/xen/kernel/ctrl_if.c
arch/xen/kernel/evtchn.c
include/asm-xen/ctrl_if.h
include/asm-xen/evtchn.h
include/asm-xen/hypervisor.h
include/asm-xen/multicall.h
include/asm-xen/suspend.h
I consider this the least we can do given the lack of documentation
that would allow OS porters to roll their own. :-)
There is the issue that there are bits and pieces of other people's
code in there as well --- e.g., Christian Limpach, Mike Wray, Mark
Williamson. But I don't foresee any problems.
-- Keir
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: plan 9 status
2004-10-14 8:10 plan 9 status Keir Fraser
@ 2004-10-14 15:53 ` Mark A. Williamson
2004-10-14 17:45 ` Xen & I/O in clusters - problems! Rune Johan Andresen
0 siblings, 1 reply; 8+ messages in thread
From: Mark A. Williamson @ 2004-10-14 15:53 UTC (permalink / raw)
To: Keir Fraser; +Cc: Ronald G. Minnich, xen-devel
> There is the issue that there are bits and pieces of other people's
> code in there as well --- e.g., Christian Limpach, Mike Wray, Mark
> Williamson. But I don't foresee any problems.
Fine by me. The code I wrote really belongs to Intel but they BSD license
stuff anyhow, so it shouldn't be a problem.
Mark
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
^ permalink raw reply [flat|nested] 8+ messages in thread
* Xen & I/O in clusters - problems!
2004-10-14 15:53 ` Mark A. Williamson
@ 2004-10-14 17:45 ` Rune Johan Andresen
2004-10-14 18:17 ` Ian Pratt
2004-10-14 23:24 ` Mark A. Williamson
0 siblings, 2 replies; 8+ messages in thread
From: Rune Johan Andresen @ 2004-10-14 17:45 UTC (permalink / raw)
To: Xen Virtual Machine Monitor
[-- Attachment #1: Type: text/plain, Size: 814 bytes --]
Hi, we are benchmarking Xen in a cluster, and got some bad results. We
might do something wrong, and wonder if anyone
have similar problems.
When we benchmark troughput from native Linux to native Linux (two
physical nodes in the cluster) we get 786.034 MByte/s
When we benchmark from a virtual domain (running on Xen on a physical
node) to an another virtual domain (on another physical node) we get
56.480 MByte/s (1:16)
The difference is huge, and we wonder if the bottleneck could be the
fact that we are using software routing (We use this in order to route
from the physical node to the virtual OSs), or if this is just a
downside of Xen?
I would guess it IS the SW routing, so is there any good alternatives
to make virtual domains communicate on a cluster without sw routing?
cheers,
Rune J.A
[-- Attachment #2: Type: text/enriched, Size: 1010 bytes --]
Hi, we are benchmarking Xen in a cluster, and got some bad results. We
might do something wrong, and wonder if anyone
have similar problems.
When we benchmark troughput from native Linux to native Linux (two
physical nodes in the cluster) we get
<fontfamily><param>Geneva</param><bigger>786.034 MByte/s
</bigger></fontfamily>When we benchmark from a virtual domain
(running on Xen on a physical node) to an another virtual domain (on
another physical node) we get
<fontfamily><param>Geneva</param><bigger>56.480 MByte/s (1:16)
</bigger></fontfamily>The difference is huge, and we wonder if the
bottleneck could be the fact that we are using software routing (We
use this in order to route from the physical node to the virtual OSs),
or if this is just a downside of Xen?
I would guess it IS the SW routing, so is there any good alternatives
to make virtual domains communicate on a cluster without sw routing?
<fontfamily><param>Geneva</param><bigger>
</bigger></fontfamily>cheers,
Rune J.A
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Xen & I/O in clusters - problems!
2004-10-14 17:45 ` Xen & I/O in clusters - problems! Rune Johan Andresen
@ 2004-10-14 18:17 ` Ian Pratt
2004-10-14 23:24 ` Mark A. Williamson
1 sibling, 0 replies; 8+ messages in thread
From: Ian Pratt @ 2004-10-14 18:17 UTC (permalink / raw)
To: Rune Johan Andresen; +Cc: Xen Virtual Machine Monitor, Ian.Pratt
> Hi, we are benchmarking Xen in a cluster, and got some bad results. We
> might do something wrong, and wonder if anyone
> have similar problems.
>
> When we benchmark troughput from native Linux to native Linux (two
> physical nodes in the cluster) we get 786.034 MByte/s
> When we benchmark from a virtual domain (running on Xen on a physical
> node) to an another virtual domain (on another physical node) we get
> 56.480 MByte/s (1:16)
(Presumably you mean MBits rather than Mbytes)
The numbers you're getting are terrible compared to what we see.
Running between virtual domains on a cluster we measure
throughput as high as 897Mb/s (same as Linux native).
Our results were recorded with dual 2.4GHz Xeons with tg3 NICs
and a 128KB socket buffer, measured using ttcp. With the virtual
domain running on the other physical CPU from domain 0 we get
897Mb/s. We get similar results running the virtual domain on the
other hyperthread of the same physical CPU. We observe a
performance reduction if we run the virtual domain on the same
(logical) CPU as domain 0, down to 660Mb/s [843Mb/s on a dual
3GHz machine, so we appear to be CPU limited in this case].
> The difference is huge, and we wonder if the bottleneck could be the
> fact that we are using software routing (We use this in order to route
> from the physical node to the virtual OSs), or if this is just a
> downside of Xen?
Our results were recorded using the dom0 linux bridge code rather
than using routing.
One thing to check is that you have don't have
CONFIG_IP_NF_CONNTRACK set to 'y' -- this slays performance.
Also, if you're running multiple domains on the same CPU you may
be running into CPU scheduling issues. Some tweaks to scheduler
parameters may fix this.
> I would guess it IS the SW routing, so is there any good alternatives
> to make virtual domains communicate on a cluster without sw routing?
The Xen 2.0 architecture is not as slick as the
monolithic-hypervisor approach of Xen 1.2, but we get better
hardware support and a lot more flexibility. However, we do burn
more CPU to achieve the same IO rate. We just have to wait for
Moore's law to catch up ;-)
Ian
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Xen & I/O in clusters - problems!
2004-10-14 17:45 ` Xen & I/O in clusters - problems! Rune Johan Andresen
2004-10-14 18:17 ` Ian Pratt
@ 2004-10-14 23:24 ` Mark A. Williamson
2004-10-15 13:48 ` Rune Johan Andresen
1 sibling, 1 reply; 8+ messages in thread
From: Mark A. Williamson @ 2004-10-14 23:24 UTC (permalink / raw)
To: xen-devel; +Cc: Rune Johan Andresen
> When we benchmark from a virtual domain (running on Xen on a physical
> node) to an another virtual domain (on another physical node) we get
> 56.480 MByte/s (1:16)
Ouch. How are you benchmarking this? (what tool, what parameters, etc.).
It'll help me reproduce this on our test systems. Then we'll know if it's
your config or if there's something to track down.
We did see some weird performance for small packets at one stage and I'm not
sure if that was ever resolved. If it's the same problem, I can do a binary
chop search of changesets in order to locate it.
Cheers,
Mark
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Xen & I/O in clusters - problems!
2004-10-14 23:24 ` Mark A. Williamson
@ 2004-10-15 13:48 ` Rune Johan Andresen
0 siblings, 0 replies; 8+ messages in thread
From: Rune Johan Andresen @ 2004-10-15 13:48 UTC (permalink / raw)
To: mark.williamson; +Cc: Xen Virtual Machine Monitor
Hi, we benchmark with Scali MPI. We are in the first stage with an easy
ping program which
first send 1 byte to measure the latency, then we tested with 10^5,
10^7 and 10^8 size packages. We also
get the same results when sending from domain0 to domain0 in the
cluster. We are now testing if the routing table
is the bottleneck, I let you know the results. Thank you :)
Cheers,
Rune J.A
On Oct 15, 2004, at 1:24 AM, Mark A. Williamson wrote:
>> When we benchmark from a virtual domain (running on Xen on a physical
>> node) to an another virtual domain (on another physical node) we get
>> 56.480 MByte/s (1:16)
>
> Ouch. How are you benchmarking this? (what tool, what parameters,
> etc.).
> It'll help me reproduce this on our test systems. Then we'll know if
> it's
> your config or if there's something to track down.
>
> We did see some weird performance for small packets at one stage and
> I'm not
> sure if that was ever resolved. If it's the same problem, I can do a
> binary
> chop search of changesets in order to locate it.
>
> Cheers,
> Mark
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2004-10-16 20:05 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2004-10-15 10:48 Xen & I/O in clusters - problems! Håvard Bjerke
2004-10-15 14:02 ` Mark A. Williamson
2004-10-15 16:27 ` Håvard Bjerke
2004-10-16 20:05 ` John Enok Vollestad
-- strict thread matches above, loose matches on Subject: below --
2004-10-14 8:10 plan 9 status Keir Fraser
2004-10-14 15:53 ` Mark A. Williamson
2004-10-14 17:45 ` Xen & I/O in clusters - problems! Rune Johan Andresen
2004-10-14 18:17 ` Ian Pratt
2004-10-14 23:24 ` Mark A. Williamson
2004-10-15 13:48 ` Rune Johan Andresen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.