From mboxrd@z Thu Jan 1 00:00:00 1970 From: Don Slutz Subject: Re: Help: VMXNET3 support with XEN 4.4.1 Date: Wed, 24 Dec 2014 08:46:57 -0500 Message-ID: <549AC3D1.5080105@terremark.com> References: <6A5D14D0DFEABC43BE832A0E492963F14EBB836C@MX101CL01.corp.emc.com> <5494A6AF.1030802@terremark.com> <6A5D14D0DFEABC43BE832A0E492963F14EBB922A@MX101CL01.corp.emc.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8099027545900942025==" Return-path: In-Reply-To: <6A5D14D0DFEABC43BE832A0E492963F14EBB922A@MX101CL01.corp.emc.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Singhal, Upanshu" Cc: Don Slutz , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --===============8099027545900942025== Content-Type: multipart/alternative; boundary="------------040707050204030006060308" This is a multi-part message in MIME format. --------------040707050204030006060308 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 12/22/14 02:04, Singhal, Upanshu wrote: > > Hello Don, > > xen_emul_unplug=unnecessary does the trick, I am able to see the > vmxnet3 driver using lspci and ethtool --I eth0. Thanks a lot for your > help, much appreciated. > > Performance between vmxnet3 running on XEN is about 1/3 of vmxnet3 > running on ESXi. Similar performance number I get between vmxnet3 on > ESXi and vif on XEN. Do you know what could be the reason for this? > Not sure, but mtu does have a factor here. And the full network layout, other uses of network, etc. all have impacts. Have you checked out: http://wiki.xen.org/wiki/Network_Throughput_and_Performance_Guide I have not been investigating network bandwidth. -Don Slutz > Thanks, > > -Upanshu > > *From:*Don Slutz [mailto:dslutz@verizon.com] > *Sent:* Saturday, December 20, 2014 3:59 AM > *To:* Singhal, Upanshu > *Cc:* xen-devel@lists.xen.org > *Subject:* Re: [Xen-devel] Help: VMXNET3 support with XEN 4.4.1 > > xen_emul_unplug=unnecessary (kernel arg) may help you here. > > Also udev likes to rename your devices. > > Here is a lspci from a guest: > > > [root@C63-min-tools ~]# lspci > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] > (rev 02) > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE > [Natoma/Triton II] > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device > (rev 01) > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > 00:04.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01) > 00:05.0 Ethernet controller: Intel Corporation 82540EM Gigabit > Ethernet Controller (rev 03) > 00:06.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01) > 00:07.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01) > 00:08.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01) > 00:09.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01) > 00:0a.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01) > 00:0b.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01) > 00:0c.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01) > 00:0d.0 Ethernet controller: Intel Corporation 82540EM Gigabit > Ethernet Controller (rev 03) > > And to help: > > [root@C63-min-tools ~]# ls -l /sys/class/net/*/device > lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth0/device -> > ../../../vif-0 > lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth1/device -> > ../../../vif-1 > lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth2/device -> > ../../../vif-2 > lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth3/device -> > ../../../vif-3 > lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth4/device -> > ../../../vif-4 > lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth5/device -> > ../../../vif-5 > lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth6/device -> > ../../../vif-6 > lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth7/device -> > ../../../vif-7 > lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth8/device -> > ../../../vif-8 > lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth9/device -> > ../../../vif-9 > lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic0/device > -> ../../../0000:00:04.0 > lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic1/device > -> ../../../0000:00:05.0 > lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic2/device > -> ../../../0000:00:06.0 > lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic3/device > -> ../../../0000:00:07.0 > lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic4/device > -> ../../../0000:00:08.0 > lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic5/device > -> ../../../0000:00:09.0 > lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic6/device > -> ../../../0000:00:0a.0 > lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic7/device > -> ../../../0000:00:0b.0 > lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic8/device > -> ../../../0000:00:0c.0 > lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic9/device > -> ../../../0000:00:0d.0 > > > -Don Slutz > > On 12/19/14 07:04, Singhal, Upanshu wrote: > > Hello, > > In one of my experiment, I am building a Linux VM with Network > interface model as "vmxnet3". I am able to create the VM > successfully, but I see that the driver loaded is "vif" and not > "vmxnet3". I am using the following option for the network > interface: *vif = [ 'type=ioemu, mac=00:16:3e:00:00:22, > bridge=xenbr0, model=vmxnet3' ]* > > ** > > I tried with model as e1000 and it works fine. Lspci command also > does not show vmxnet3. Though, qemu device help shows that > "vmxnet3" is supported on XEN 4.4.1 version I am using. > > I tried searching on internet about the right configuration for > vmxnet3 with XEN, but not able to find right information. Can > someone please help me on how to create a VM with vmxnet3? > > This experiment I am doing to compare the difference between > vmxnet3 bandwidth on VMWARE ESXi vs. XEN Server. My sample VM > configuration file is: > > # This configures an HVM rather than PV guest > > builder = "hvm" > > # Guest name > > *name = "rhel-vmxnet3-xen-2.hvm"* > > # 128-bit UUID for the domain as a hexadecimal number. > > # Use "uuidgen" to generate one if required. > > # The default behavior is to generate a new UUID each time the > guest is started. > > #uuid = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" > > # Enable Microsoft Hyper-V compatibile paravirtualisation / > > # enlightenment interfaces. Turning this on can improve Windows guest > > # performance and is therefore recommended > > #viridian = 1 > > # Initial memory allocation (MB) > > *memory = 8192* > > # Maximum memory (MB) > > # If this is greater than `memory' then the slack will start ballooned > > # (this assumes guest kernel support for ballooning) > > #maxmem = 512 > > # Number of VCPUS > > *vcpus = 8* > > # Network devices > > # A list of 'vifspec' entries as described in > > # docs/misc/xl-network-configuration.markdown > > vif = [ 'type=ioemu, mac=00:16:3e:00:00:22, bridge=xenbr1, > model=vmxnet3' ] > > # Disk Devices > > # A list of `diskspec' entries as described in > > # docs/misc/xl-disk-configuration.txt > > *disk = [ '/root/rhel_6_4/rhel-vmxnet3-xen-2.img,raw,xvda,rw', > 'file:/root/rhel-server-6.4-x86_64-dvd.iso,hdc:cdrom,r > ' ]* > > *boot='cd'* > > # Guest VGA console configuration, either SDL or VNC > > # sdl = 1 > > *vnc = 2* > > Thanks, > > -Upanshu > > Upanshu Singhal > > EMC Data Storage Systems, Bangalore, India. > > Phone: 91-80-67375604 > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xen.org > > http://lists.xen.org/xen-devel > --------------040707050204030006060308 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 12/22/14 02:04, Singhal, Upanshu wrote:

Hello Don,

 

xen_emul_unplug=unnecessary  does the trick, I am able to see the vmxnet3 driver using lspci and ethtool –I eth0. Thanks a lot for your help, much appreciated.

 

Performance between vmxnet3 running on XEN is about 1/3 of vmxnet3 running on ESXi. Similar performance number I get between vmxnet3 on ESXi and vif on XEN. Do you know what could be the reason for this?

 


Not sure, but mtu does have a factor here.

And the full network layout, other uses of network, etc. all have impacts.

Have you checked out:

http://wiki.xen.org/wiki/Network_Throughput_and_Performance_Guide

I have not been investigating network bandwidth.

   -Don Slutz

Thanks,

-Upanshu

 

From: Don Slutz [mailto:dslutz@verizon.com]
Sent: Saturday, December 20, 2014 3:59 AM
To: Singhal, Upanshu
Cc: xen-devel@lists.xen.org
Subject: Re: [Xen-devel] Help: VMXNET3 support with XEN 4.4.1

 

xen_emul_unplug=unnecessary (kernel arg) may help you here.

Also udev likes to rename your devices.

Here is a lspci from a guest:


[root@C63-min-tools ~]# lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
00:03.0 VGA compatible controller: Cirrus Logic GD 5446
00:04.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
00:05.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03)
00:06.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
00:07.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
00:08.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
00:09.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
00:0a.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
00:0b.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
00:0c.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
00:0d.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03)

And to help:

[root@C63-min-tools ~]# ls -l /sys/class/net/*/device
lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth0/device -> ../../../vif-0
lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth1/device -> ../../../vif-1
lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth2/device -> ../../../vif-2
lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth3/device -> ../../../vif-3
lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth4/device -> ../../../vif-4
lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth5/device -> ../../../vif-5
lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth6/device -> ../../../vif-6
lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth7/device -> ../../../vif-7
lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth8/device -> ../../../vif-8
lrwxrwxrwx. 1 root root 0 Dec 19 17:23 /sys/class/net/eth9/device -> ../../../vif-9
lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic0/device -> ../../../0000:00:04.0
lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic1/device -> ../../../0000:00:05.0
lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic2/device -> ../../../0000:00:06.0
lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic3/device -> ../../../0000:00:07.0
lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic4/device -> ../../../0000:00:08.0
lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic5/device -> ../../../0000:00:09.0
lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic6/device -> ../../../0000:00:0a.0
lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic7/device -> ../../../0000:00:0b.0
lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic8/device -> ../../../0000:00:0c.0
lrwxrwxrwx. 1 root root 0 Dec 19 17:21 /sys/class/net/pci-nic9/device -> ../../../0000:00:0d.0


   -Don Slutz

On 12/19/14 07:04, Singhal, Upanshu wrote:

Hello,

 

In one of my experiment, I am building a Linux VM with Network interface model as “vmxnet3”. I am able to create the VM successfully, but I see that the driver loaded is “vif” and not “vmxnet3”. I am using the following option for the network interface: vif = [ 'type=ioemu, mac=00:16:3e:00:00:22, bridge=xenbr0, model=vmxnet3' ]

 

I tried with model as e1000 and it works fine. Lspci command also does not show vmxnet3. Though, qemu device help shows that “vmxnet3” is supported on XEN 4.4.1 version I am using.

 

I tried searching on internet about the right configuration for vmxnet3 with XEN, but not able to find right information. Can someone please help me on how to create a VM with vmxnet3?

 

This experiment I am doing to compare the difference between vmxnet3 bandwidth on VMWARE ESXi vs. XEN Server. My sample VM configuration file is:

 

# This configures an HVM rather than PV guest

builder = "hvm"

 

# Guest name

name = "rhel-vmxnet3-xen-2.hvm"

 

# 128-bit UUID for the domain as a hexadecimal number.

# Use "uuidgen" to generate one if required.

# The default behavior is to generate a new UUID each time the guest is started.

#uuid = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"

 

# Enable Microsoft Hyper-V compatibile paravirtualisation /

# enlightenment interfaces. Turning this on can improve Windows guest

# performance and is therefore recommended

#viridian = 1

 

# Initial memory allocation (MB)

memory = 8192

 

# Maximum memory (MB)

# If this is greater than `memory' then the slack will start ballooned

# (this assumes guest kernel support for ballooning)

#maxmem = 512

 

# Number of VCPUS

vcpus = 8

 

# Network devices

# A list of 'vifspec' entries as described in

# docs/misc/xl-network-configuration.markdown

vif = [ 'type=ioemu, mac=00:16:3e:00:00:22, bridge=xenbr1, model=vmxnet3' ]

 

# Disk Devices

# A list of `diskspec' entries as described in

# docs/misc/xl-disk-configuration.txt

disk = [ '/root/rhel_6_4/rhel-vmxnet3-xen-2.img,raw,xvda,rw', 'file:/root/rhel-server-6.4-x86_64-dvd.iso,hdc:cdrom,r' ]

boot='cd'

 

# Guest VGA console configuration, either SDL or VNC

# sdl = 1

vnc = 2

 

Thanks,

-Upanshu

 

 

Upanshu Singhal

EMC Data Storage Systems, Bangalore, India.

Phone: 91-80-67375604

 




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

 


--------------040707050204030006060308-- --===============8099027545900942025== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============8099027545900942025==--