* [Qemu-devel] [Autotest][RFC PATCH 00/14] Patchset of network related subtests
@ 2010-07-20 1:34 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:34 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
The following series contain 11 network related subtests, welcome to give me
some suggestions about correctness, design, enhancement.
Thank you so much!
---
Amos Kong (14):
KVM-test: Add a new macaddress pool algorithm
KVM Test: Add a function get_interface_name() to kvm_net_utils.py
KVM Test: Add a common ping module for network related tests
KVM-test: Add a new subtest ping
KVM-test: Add a subtest jumbo
KVM-test: Add basic file transfer test
KVM-test: Add a subtest of load/unload nic driver
KVM-test: Add a subtest of nic promisc
KVM-test: Add a subtest of multicast
KVM-test: Add a subtest of pxe
KVM-test: Add a subtest of changing mac address
KVM-test: Add a subtest of netperf
KVM-test: Improve vlan subtest
KVM-test: Add subtest of testing offload by ethtool
0 files changed, 0 insertions(+), 0 deletions(-)
--
Amos Kong
^ permalink raw reply [flat|nested] 77+ messages in thread* [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:34 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:34 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
Old method uses the mac address in the configuration files which could
lead serious problem when multiple tests running in different hosts.
This patch adds a new macaddress pool algorithm, it generates the mac prefix
based on mac address of the host which could eliminate the duplicated mac
addresses between machines.
When user have set the mac_prefix in the configuration file, we should use it
in stead of the dynamic generated mac prefix.
Other change:
. Fix randomly generating mac address so that it correspond to IEEE802.
. Update clone function to decide clone mac address or not.
. Update get_macaddr function.
. Add set_mac_address function.
New auto mac address pool algorithm:
If address_index is defined, VM will get mac from config file then record mac
in to address_pool. If address_index is not defined, VM will call
get_mac_from_pool to auto create mac then recored mac to address_pool in
following format:
{'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}
AE:9D:94:6A:9b:f9 : mac address
20100310-165222-Wt7l : instance attribute of VM
0 : index of NIC
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Feng Yang <fyang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
index fb2d1c2..7c0946e 100644
--- a/client/tests/kvm/kvm_utils.py
+++ b/client/tests/kvm/kvm_utils.py
@@ -5,6 +5,7 @@ KVM test utility functions.
"""
import time, string, random, socket, os, signal, re, logging, commands, cPickle
+import fcntl, shelve
from autotest_lib.client.bin import utils
from autotest_lib.client.common_lib import error, logging_config
import kvm_subprocess
@@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword):
# Functions related to MAC/IP addresses
+def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'):
+ """
+ random generated mac address.
+
+ 1) First try to generate macaddress based on the mac address prefix.
+ 2) And then try to use total random generated mac address.
+
+ @param root_dir: Root dir for kvm
+ @param vm: Here we use instance of vm
+ @param nic_index: The index of nic.
+ @param prefix: Prefix of mac address.
+ @Return: Return mac address.
+ """
+
+ lock_filename = os.path.join(root_dir, "mac_lock")
+ lock_file = open(lock_filename, 'w')
+ fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
+ mac_filename = os.path.join(root_dir, "address_pool")
+ mac_shelve = shelve.open(mac_filename, writeback=False)
+
+ mac_pool = mac_shelve.get("macpool")
+
+ if not mac_pool:
+ mac_pool = {}
+ found = False
+
+ val = "%s:%s" % (vm, nic_index)
+ for key in mac_pool.keys():
+ if val in mac_pool[key]:
+ mac_pool[key].append(val)
+ found = True
+ mac = key
+
+ while not found:
+ postfix = "%02x:%02x" % (random.randint(0x00,0xfe),
+ random.randint(0x00,0xfe))
+ mac = prefix + postfix
+ mac_list = mac.split(":")
+ # Clear multicast bit
+ mac_list[0] = int(mac_list[0],16) & 0xfe
+ # Set local assignment bit (IEEE802)
+ mac_list[0] = mac_list[0] | 0x02
+ mac_list[0] = "%02x" % mac_list[0]
+ mac = ":".join(mac_list)
+ if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]):
+ mac_pool[mac] = ["%s:%s" % (vm,nic_index)]
+ found = True
+ mac_shelve["macpool"] = mac_pool
+ logging.debug("generating mac addr %s " % mac)
+
+ mac_shelve.close()
+ fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
+ lock_file.close()
+ return mac
+
+
+def put_mac_to_pool(root_dir, mac, vm):
+ """
+ Put the macaddress back to address pool
+
+ @param root_dir: Root dir for kvm
+ @param vm: Here we use instance attribute of vm
+ @param mac: mac address will be put.
+ @Return: mac address.
+ """
+
+ lock_filename = os.path.join(root_dir, "mac_lock")
+ lock_file = open(lock_filename, 'w')
+ fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
+ mac_filename = os.path.join(root_dir, "address_pool")
+ mac_shelve = shelve.open(mac_filename, writeback=False)
+
+ mac_pool = mac_shelve.get("macpool")
+
+ if not mac_pool or (not mac in mac_pool):
+ logging.debug("Try to free a macaddress does no in pool")
+ logging.debug("macaddress is %s" % mac)
+ logging.debug("pool is %s" % mac_pool)
+ else:
+ if len(mac_pool[mac]) <= 1:
+ mac_pool.pop(mac)
+ else:
+ for value in mac_pool[mac]:
+ if vm in value:
+ mac_pool[mac].remove(value)
+ break
+ if len(mac_pool[mac]) == 0:
+ mac_pool.pop(mac)
+
+ mac_shelve["macpool"] = mac_pool
+ logging.debug("freeing mac addr %s " % mac)
+
+ mac_shelve.close()
+ fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
+ lock_file.close()
+ return mac
+
+
def mac_str_to_int(addr):
"""
Convert MAC address string to integer.
diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index 6cd0688..a9ba6e7 100755
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu.
@copyright: 2008-2009 Red Hat Inc.
"""
-import time, socket, os, logging, fcntl, re, commands, glob
+import time, socket, os, logging, fcntl, re, commands, shelve, glob
import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer
from autotest_lib.client.common_lib import error
from autotest_lib.client.bin import utils
@@ -117,6 +117,7 @@ class VM:
self.params = params
self.root_dir = root_dir
self.address_cache = address_cache
+ self.mac_prefix = params.get('mac_prefix')
self.netdev_id = []
# Find a unique identifier for this VM
@@ -126,8 +127,15 @@ class VM:
if not glob.glob("/tmp/*%s" % self.instance):
break
+ if self.mac_prefix is None:
+ s, o = commands.getstatusoutput("ifconfig eth0")
+ if s == 0:
+ mac = re.findall("HWaddr (\S*)", o)[0]
+ self.mac_prefix = '00' + mac[8:] + ':'
+
- def clone(self, name=None, params=None, root_dir=None, address_cache=None):
+ def clone(self, name=None, params=None, root_dir=None,
+ address_cache=None, mac_clone=True):
"""
Return a clone of the VM object with optionally modified parameters.
The clone is initially not alive and needs to be started using create().
@@ -138,6 +146,7 @@ class VM:
@param params: Optional new VM creation parameters
@param root_dir: Optional new base directory for relative filenames
@param address_cache: A dict that maps MAC addresses to IP addresses
+ @param mac_clone: Clone mac address or not.
"""
if name is None:
name = self.name
@@ -147,9 +156,19 @@ class VM:
root_dir = self.root_dir
if address_cache is None:
address_cache = self.address_cache
- return VM(name, params, root_dir, address_cache)
+ vm = VM(name, params, root_dir, address_cache)
+ if mac_clone:
+ # Clone mac address by coping 'self.instance' to the new vm.
+ vm.instance = self.instance
+ return vm
+ def free_mac_address(self):
+ nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics"))
+ for nic in range(nic_num):
+ mac = self.get_macaddr(nic_index=nic)
+ kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance)
+
def make_qemu_command(self, name=None, params=None, root_dir=None):
"""
Generate a qemu command line. All parameters are optional. If a
@@ -383,6 +402,13 @@ class VM:
mac = None
if "address_index" in nic_params:
mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0]
+ self.set_mac_address(mac=mac, nic_index=vlan)
+ else:
+ mac = kvm_utils.get_mac_from_pool(self.root_dir,
+ vm=self.instance,
+ nic_index=vlan,
+ prefix=self.mac_prefix)
+
qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac,
self.netdev_id[vlan])
# Handle the '-net tap' or '-net user' part
@@ -396,7 +422,7 @@ class VM:
if tftp:
tftp = kvm_utils.get_path(root_dir, tftp)
qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"),
- nic_params.get("nic_ifname"),
+ self.get_ifname(vlan),
script, downscript, tftp,
nic_params.get("bootp"), redirs,
self.netdev_id[vlan])
@@ -720,7 +746,7 @@ class VM:
lockfile.close()
- def destroy(self, gracefully=True):
+ def destroy(self, gracefully=True, free_macaddr=True):
"""
Destroy the VM.
@@ -731,6 +757,7 @@ class VM:
@param gracefully: Whether an attempt will be made to end the VM
using a shell command before trying to end the qemu process
with a 'quit' or a kill signal.
+ @param free_macaddr: Whether free macaddresses when destory a vm.
"""
try:
# Is it already dead?
@@ -751,11 +778,18 @@ class VM:
logging.debug("Shutdown command sent; waiting for VM "
"to go down...")
if kvm_utils.wait_for(self.is_dead, 60, 1, 1):
- logging.debug("VM is down")
+ logging.debug("VM is down, free mac address.")
+ # free mac address
+ if free_macaddr:
+ self.free_mac_address()
return
finally:
session.close()
+ # free mac address
+ if free_macaddr:
+ self.free_mac_address()
+
if self.monitor:
# Try to destroy with a monitor command
logging.debug("Trying to kill VM with monitor command...")
@@ -881,10 +915,14 @@ class VM:
nic_name = nics[index]
nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
if nic_params.get("nic_mode") == "tap":
- mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params)
+ mac = self.get_macaddr(index)
if not mac:
logging.debug("MAC address unavailable")
return None
+ else:
+ mac = mac.lower()
+ ip = None
+
if not ip or nic_params.get("always_use_tcpdump") == "yes":
# Get the IP address from the cache
ip = self.address_cache.get(mac)
@@ -897,6 +935,7 @@ class VM:
for nic in nics]
macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0]
for dict in nic_dicts]
+ macs.append(mac)
if not kvm_utils.verify_ip_address_ownership(ip, macs):
logging.debug("Could not verify MAC-IP address mapping: "
"%s ---> %s" % (mac, ip))
@@ -925,6 +964,71 @@ class VM:
"redirected" % port)
return self.redirs.get(port)
+ def get_ifname(self, nic_index=0):
+ """
+ Return the ifname of tap device for the guest nic.
+
+ @param nic_index: Index of the NIC
+ """
+
+ nics = kvm_utils.get_sub_dict_names(self.params, "nics")
+ nic_name = nics[nic_index]
+ nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
+ if nic_params.get("nic_ifname"):
+ return nic_params.get("nic_ifname")
+ else:
+ return "%s_%s_%s" % (nic_params.get("nic_model"),
+ nic_index, self.vnc_port)
+
+ def get_macaddr(self, nic_index=0):
+ """
+ Return the macaddr of guest nic.
+
+ @param nic_index: Index of the NIC
+ """
+ mac_filename = os.path.join(self.root_dir, "address_pool")
+ mac_shelve = shelve.open(mac_filename, writeback=False)
+ mac_pool = mac_shelve.get("macpool")
+ val = "%s:%s" % (self.instance, nic_index)
+ for key in mac_pool.keys():
+ if val in mac_pool[key]:
+ return key
+ return None
+
+ def set_mac_address(self, mac, nic_index=0, shareable=False):
+ """
+ Set mac address for guest. Note: It just update address pool.
+
+ @param mac: address will set to guest
+ @param nic_index: Index of the NIC
+ @param shareable: Where VM can share mac with other VM or not.
+ """
+ lock_filename = os.path.join(self.root_dir, "mac_lock")
+ lock_file = open(lock_filename, 'w')
+ fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
+ mac_filename = os.path.join(self.root_dir, "address_pool")
+ mac_shelve = shelve.open(mac_filename, writeback=False)
+ mac_pool = mac_shelve.get("macpool")
+ if not mac_pool:
+ mac_pool = {}
+ value = "%s:%s" % (self.instance, nic_index)
+ if mac not in mac_pool.keys():
+ for key in mac_pool.keys():
+ if value in mac_pool[key]:
+ mac_pool[key].remove(value)
+ if len(mac_pool[key]) == 0:
+ mac_pool.pop(key)
+ mac_pool[mac] = [value]
+ else:
+ if shareable:
+ mac_pool[mac].append(value)
+ else:
+ logging.error("Mac address already be used!")
+ return False
+ mac_shelve["macpool"] = mac_pool
+ mac_shelve.close()
+ fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
+ lock_file.close()
def get_pid(self):
"""
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index fd9e72f..6710c00 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -51,7 +51,7 @@ guest_port_remote_shell = 22
nic_mode = user
#nic_mode = tap
nic_script = scripts/qemu-ifup
-address_index = 0
+#address_index = 0
run_tcpdump = yes
# Misc
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
@ 2010-07-20 1:34 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:34 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
Old method uses the mac address in the configuration files which could
lead serious problem when multiple tests running in different hosts.
This patch adds a new macaddress pool algorithm, it generates the mac prefix
based on mac address of the host which could eliminate the duplicated mac
addresses between machines.
When user have set the mac_prefix in the configuration file, we should use it
in stead of the dynamic generated mac prefix.
Other change:
. Fix randomly generating mac address so that it correspond to IEEE802.
. Update clone function to decide clone mac address or not.
. Update get_macaddr function.
. Add set_mac_address function.
New auto mac address pool algorithm:
If address_index is defined, VM will get mac from config file then record mac
in to address_pool. If address_index is not defined, VM will call
get_mac_from_pool to auto create mac then recored mac to address_pool in
following format:
{'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}
AE:9D:94:6A:9b:f9 : mac address
20100310-165222-Wt7l : instance attribute of VM
0 : index of NIC
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Feng Yang <fyang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
index fb2d1c2..7c0946e 100644
--- a/client/tests/kvm/kvm_utils.py
+++ b/client/tests/kvm/kvm_utils.py
@@ -5,6 +5,7 @@ KVM test utility functions.
"""
import time, string, random, socket, os, signal, re, logging, commands, cPickle
+import fcntl, shelve
from autotest_lib.client.bin import utils
from autotest_lib.client.common_lib import error, logging_config
import kvm_subprocess
@@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword):
# Functions related to MAC/IP addresses
+def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'):
+ """
+ random generated mac address.
+
+ 1) First try to generate macaddress based on the mac address prefix.
+ 2) And then try to use total random generated mac address.
+
+ @param root_dir: Root dir for kvm
+ @param vm: Here we use instance of vm
+ @param nic_index: The index of nic.
+ @param prefix: Prefix of mac address.
+ @Return: Return mac address.
+ """
+
+ lock_filename = os.path.join(root_dir, "mac_lock")
+ lock_file = open(lock_filename, 'w')
+ fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
+ mac_filename = os.path.join(root_dir, "address_pool")
+ mac_shelve = shelve.open(mac_filename, writeback=False)
+
+ mac_pool = mac_shelve.get("macpool")
+
+ if not mac_pool:
+ mac_pool = {}
+ found = False
+
+ val = "%s:%s" % (vm, nic_index)
+ for key in mac_pool.keys():
+ if val in mac_pool[key]:
+ mac_pool[key].append(val)
+ found = True
+ mac = key
+
+ while not found:
+ postfix = "%02x:%02x" % (random.randint(0x00,0xfe),
+ random.randint(0x00,0xfe))
+ mac = prefix + postfix
+ mac_list = mac.split(":")
+ # Clear multicast bit
+ mac_list[0] = int(mac_list[0],16) & 0xfe
+ # Set local assignment bit (IEEE802)
+ mac_list[0] = mac_list[0] | 0x02
+ mac_list[0] = "%02x" % mac_list[0]
+ mac = ":".join(mac_list)
+ if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]):
+ mac_pool[mac] = ["%s:%s" % (vm,nic_index)]
+ found = True
+ mac_shelve["macpool"] = mac_pool
+ logging.debug("generating mac addr %s " % mac)
+
+ mac_shelve.close()
+ fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
+ lock_file.close()
+ return mac
+
+
+def put_mac_to_pool(root_dir, mac, vm):
+ """
+ Put the macaddress back to address pool
+
+ @param root_dir: Root dir for kvm
+ @param vm: Here we use instance attribute of vm
+ @param mac: mac address will be put.
+ @Return: mac address.
+ """
+
+ lock_filename = os.path.join(root_dir, "mac_lock")
+ lock_file = open(lock_filename, 'w')
+ fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
+ mac_filename = os.path.join(root_dir, "address_pool")
+ mac_shelve = shelve.open(mac_filename, writeback=False)
+
+ mac_pool = mac_shelve.get("macpool")
+
+ if not mac_pool or (not mac in mac_pool):
+ logging.debug("Try to free a macaddress does no in pool")
+ logging.debug("macaddress is %s" % mac)
+ logging.debug("pool is %s" % mac_pool)
+ else:
+ if len(mac_pool[mac]) <= 1:
+ mac_pool.pop(mac)
+ else:
+ for value in mac_pool[mac]:
+ if vm in value:
+ mac_pool[mac].remove(value)
+ break
+ if len(mac_pool[mac]) == 0:
+ mac_pool.pop(mac)
+
+ mac_shelve["macpool"] = mac_pool
+ logging.debug("freeing mac addr %s " % mac)
+
+ mac_shelve.close()
+ fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
+ lock_file.close()
+ return mac
+
+
def mac_str_to_int(addr):
"""
Convert MAC address string to integer.
diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index 6cd0688..a9ba6e7 100755
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu.
@copyright: 2008-2009 Red Hat Inc.
"""
-import time, socket, os, logging, fcntl, re, commands, glob
+import time, socket, os, logging, fcntl, re, commands, shelve, glob
import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer
from autotest_lib.client.common_lib import error
from autotest_lib.client.bin import utils
@@ -117,6 +117,7 @@ class VM:
self.params = params
self.root_dir = root_dir
self.address_cache = address_cache
+ self.mac_prefix = params.get('mac_prefix')
self.netdev_id = []
# Find a unique identifier for this VM
@@ -126,8 +127,15 @@ class VM:
if not glob.glob("/tmp/*%s" % self.instance):
break
+ if self.mac_prefix is None:
+ s, o = commands.getstatusoutput("ifconfig eth0")
+ if s == 0:
+ mac = re.findall("HWaddr (\S*)", o)[0]
+ self.mac_prefix = '00' + mac[8:] + ':'
+
- def clone(self, name=None, params=None, root_dir=None, address_cache=None):
+ def clone(self, name=None, params=None, root_dir=None,
+ address_cache=None, mac_clone=True):
"""
Return a clone of the VM object with optionally modified parameters.
The clone is initially not alive and needs to be started using create().
@@ -138,6 +146,7 @@ class VM:
@param params: Optional new VM creation parameters
@param root_dir: Optional new base directory for relative filenames
@param address_cache: A dict that maps MAC addresses to IP addresses
+ @param mac_clone: Clone mac address or not.
"""
if name is None:
name = self.name
@@ -147,9 +156,19 @@ class VM:
root_dir = self.root_dir
if address_cache is None:
address_cache = self.address_cache
- return VM(name, params, root_dir, address_cache)
+ vm = VM(name, params, root_dir, address_cache)
+ if mac_clone:
+ # Clone mac address by coping 'self.instance' to the new vm.
+ vm.instance = self.instance
+ return vm
+ def free_mac_address(self):
+ nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics"))
+ for nic in range(nic_num):
+ mac = self.get_macaddr(nic_index=nic)
+ kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance)
+
def make_qemu_command(self, name=None, params=None, root_dir=None):
"""
Generate a qemu command line. All parameters are optional. If a
@@ -383,6 +402,13 @@ class VM:
mac = None
if "address_index" in nic_params:
mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0]
+ self.set_mac_address(mac=mac, nic_index=vlan)
+ else:
+ mac = kvm_utils.get_mac_from_pool(self.root_dir,
+ vm=self.instance,
+ nic_index=vlan,
+ prefix=self.mac_prefix)
+
qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac,
self.netdev_id[vlan])
# Handle the '-net tap' or '-net user' part
@@ -396,7 +422,7 @@ class VM:
if tftp:
tftp = kvm_utils.get_path(root_dir, tftp)
qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"),
- nic_params.get("nic_ifname"),
+ self.get_ifname(vlan),
script, downscript, tftp,
nic_params.get("bootp"), redirs,
self.netdev_id[vlan])
@@ -720,7 +746,7 @@ class VM:
lockfile.close()
- def destroy(self, gracefully=True):
+ def destroy(self, gracefully=True, free_macaddr=True):
"""
Destroy the VM.
@@ -731,6 +757,7 @@ class VM:
@param gracefully: Whether an attempt will be made to end the VM
using a shell command before trying to end the qemu process
with a 'quit' or a kill signal.
+ @param free_macaddr: Whether free macaddresses when destory a vm.
"""
try:
# Is it already dead?
@@ -751,11 +778,18 @@ class VM:
logging.debug("Shutdown command sent; waiting for VM "
"to go down...")
if kvm_utils.wait_for(self.is_dead, 60, 1, 1):
- logging.debug("VM is down")
+ logging.debug("VM is down, free mac address.")
+ # free mac address
+ if free_macaddr:
+ self.free_mac_address()
return
finally:
session.close()
+ # free mac address
+ if free_macaddr:
+ self.free_mac_address()
+
if self.monitor:
# Try to destroy with a monitor command
logging.debug("Trying to kill VM with monitor command...")
@@ -881,10 +915,14 @@ class VM:
nic_name = nics[index]
nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
if nic_params.get("nic_mode") == "tap":
- mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params)
+ mac = self.get_macaddr(index)
if not mac:
logging.debug("MAC address unavailable")
return None
+ else:
+ mac = mac.lower()
+ ip = None
+
if not ip or nic_params.get("always_use_tcpdump") == "yes":
# Get the IP address from the cache
ip = self.address_cache.get(mac)
@@ -897,6 +935,7 @@ class VM:
for nic in nics]
macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0]
for dict in nic_dicts]
+ macs.append(mac)
if not kvm_utils.verify_ip_address_ownership(ip, macs):
logging.debug("Could not verify MAC-IP address mapping: "
"%s ---> %s" % (mac, ip))
@@ -925,6 +964,71 @@ class VM:
"redirected" % port)
return self.redirs.get(port)
+ def get_ifname(self, nic_index=0):
+ """
+ Return the ifname of tap device for the guest nic.
+
+ @param nic_index: Index of the NIC
+ """
+
+ nics = kvm_utils.get_sub_dict_names(self.params, "nics")
+ nic_name = nics[nic_index]
+ nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
+ if nic_params.get("nic_ifname"):
+ return nic_params.get("nic_ifname")
+ else:
+ return "%s_%s_%s" % (nic_params.get("nic_model"),
+ nic_index, self.vnc_port)
+
+ def get_macaddr(self, nic_index=0):
+ """
+ Return the macaddr of guest nic.
+
+ @param nic_index: Index of the NIC
+ """
+ mac_filename = os.path.join(self.root_dir, "address_pool")
+ mac_shelve = shelve.open(mac_filename, writeback=False)
+ mac_pool = mac_shelve.get("macpool")
+ val = "%s:%s" % (self.instance, nic_index)
+ for key in mac_pool.keys():
+ if val in mac_pool[key]:
+ return key
+ return None
+
+ def set_mac_address(self, mac, nic_index=0, shareable=False):
+ """
+ Set mac address for guest. Note: It just update address pool.
+
+ @param mac: address will set to guest
+ @param nic_index: Index of the NIC
+ @param shareable: Where VM can share mac with other VM or not.
+ """
+ lock_filename = os.path.join(self.root_dir, "mac_lock")
+ lock_file = open(lock_filename, 'w')
+ fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
+ mac_filename = os.path.join(self.root_dir, "address_pool")
+ mac_shelve = shelve.open(mac_filename, writeback=False)
+ mac_pool = mac_shelve.get("macpool")
+ if not mac_pool:
+ mac_pool = {}
+ value = "%s:%s" % (self.instance, nic_index)
+ if mac not in mac_pool.keys():
+ for key in mac_pool.keys():
+ if value in mac_pool[key]:
+ mac_pool[key].remove(value)
+ if len(mac_pool[key]) == 0:
+ mac_pool.pop(key)
+ mac_pool[mac] = [value]
+ else:
+ if shareable:
+ mac_pool[mac].append(value)
+ else:
+ logging.error("Mac address already be used!")
+ return False
+ mac_shelve["macpool"] = mac_pool
+ mac_shelve.close()
+ fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
+ lock_file.close()
def get_pid(self):
"""
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index fd9e72f..6710c00 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -51,7 +51,7 @@ guest_port_remote_shell = 22
nic_mode = user
#nic_mode = tap
nic_script = scripts/qemu-ifup
-address_index = 0
+#address_index = 0
run_tcpdump = yes
# Misc
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
2010-07-20 1:34 ` [Qemu-devel] " Amos Kong
@ 2010-07-20 10:19 ` Michael Goldish
-1 siblings, 0 replies; 77+ messages in thread
From: Michael Goldish @ 2010-07-20 10:19 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On 07/20/2010 04:34 AM, Amos Kong wrote:
> Old method uses the mac address in the configuration files which could
> lead serious problem when multiple tests running in different hosts.
>
> This patch adds a new macaddress pool algorithm, it generates the mac prefix
> based on mac address of the host which could eliminate the duplicated mac
> addresses between machines.
>
> When user have set the mac_prefix in the configuration file, we should use it
> in stead of the dynamic generated mac prefix.
>
> Other change:
> . Fix randomly generating mac address so that it correspond to IEEE802.
> . Update clone function to decide clone mac address or not.
> . Update get_macaddr function.
> . Add set_mac_address function.
>
> New auto mac address pool algorithm:
> If address_index is defined, VM will get mac from config file then record mac
> in to address_pool. If address_index is not defined, VM will call
> get_mac_from_pool to auto create mac then recored mac to address_pool in
> following format:
> {'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}
>
> AE:9D:94:6A:9b:f9 : mac address
> 20100310-165222-Wt7l : instance attribute of VM
> 0 : index of NIC
Why do you use the mac address as a key, instead of the instance string
+ nic index? When the mac address is used as a key, each key has a list
of values instead of just one value. This order seems unnatural. If it
were the other way around (i.e. key = VM instance + nic index, value =
mac address), then each key would have exactly one value, and I think
this patch would be shorter and simpler.
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Feng Yang <fyang@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
> index fb2d1c2..7c0946e 100644
> --- a/client/tests/kvm/kvm_utils.py
> +++ b/client/tests/kvm/kvm_utils.py
> @@ -5,6 +5,7 @@ KVM test utility functions.
> """
>
> import time, string, random, socket, os, signal, re, logging, commands, cPickle
> +import fcntl, shelve
> from autotest_lib.client.bin import utils
> from autotest_lib.client.common_lib import error, logging_config
> import kvm_subprocess
> @@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword):
>
> # Functions related to MAC/IP addresses
>
> +def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'):
The name of this function is confusing because it does the exact
opposite: it puts a mac address in address_pool. Maybe the pool you're
referring to in the name isn't address_pool, but still a less confusing
name should probably be used.
> + """
> + random generated mac address.
> +
> + 1) First try to generate macaddress based on the mac address prefix.
> + 2) And then try to use total random generated mac address.
> +
> + @param root_dir: Root dir for kvm
> + @param vm: Here we use instance of vm
> + @param nic_index: The index of nic.
> + @param prefix: Prefix of mac address.
> + @Return: Return mac address.
> + """
> +
> + lock_filename = os.path.join(root_dir, "mac_lock")
> + lock_file = open(lock_filename, 'w')
> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> + mac_filename = os.path.join(root_dir, "address_pool")
Maybe it makes sense to put address_pool and the lock file in /tmp,
where they can be shared by more than a single autotest instance running
on the same host (unlikely, but theoretically possible).
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> +
> + mac_pool = mac_shelve.get("macpool")
Why is this 'macpool' needed? Why not put the keys directly in the
shelve object?
> +
> + if not mac_pool:
> + mac_pool = {}
> + found = False
> +
> + val = "%s:%s" % (vm, nic_index)
> + for key in mac_pool.keys():
> + if val in mac_pool[key]:
> + mac_pool[key].append(val)
Why append val to mac_pool[key] if val is already in mac_pool[key]?
> + found = True
> + mac = key
> +
> + while not found:
> + postfix = "%02x:%02x" % (random.randint(0x00,0xfe),
> + random.randint(0x00,0xfe))
> + mac = prefix + postfix
> + mac_list = mac.split(":")
> + # Clear multicast bit
> + mac_list[0] = int(mac_list[0],16) & 0xfe
> + # Set local assignment bit (IEEE802)
> + mac_list[0] = mac_list[0] | 0x02
> + mac_list[0] = "%02x" % mac_list[0]
Why is this needed? Most mac addresses begin with 00. If the mac
address is generated from the address of eth0 (using the method in this
patch) it begins with 00, which is fine. If the prefix is set by the
user using mac_prefix, I don't think we should modify it.
> + mac = ":".join(mac_list)
> + if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]):
> + mac_pool[mac] = ["%s:%s" % (vm,nic_index)]
> + found = True
> + mac_shelve["macpool"] = mac_pool
> + logging.debug("generating mac addr %s " % mac)
> +
> + mac_shelve.close()
> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> + lock_file.close()
> + return mac
> +
> +
> +def put_mac_to_pool(root_dir, mac, vm):
This function removes a mac address from address_pool. I wonder why you
chose this name.
> + """
> + Put the macaddress back to address pool
> +
> + @param root_dir: Root dir for kvm
> + @param vm: Here we use instance attribute of vm
> + @param mac: mac address will be put.
> + @Return: mac address.
> + """
> +
> + lock_filename = os.path.join(root_dir, "mac_lock")
> + lock_file = open(lock_filename, 'w')
> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> + mac_filename = os.path.join(root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> +
> + mac_pool = mac_shelve.get("macpool")
> +
> + if not mac_pool or (not mac in mac_pool):
> + logging.debug("Try to free a macaddress does no in pool")
> + logging.debug("macaddress is %s" % mac)
> + logging.debug("pool is %s" % mac_pool)
> + else:
> + if len(mac_pool[mac]) <= 1:
> + mac_pool.pop(mac)
> + else:
> + for value in mac_pool[mac]:
> + if vm in value:
> + mac_pool[mac].remove(value)
> + break
> + if len(mac_pool[mac]) == 0:
> + mac_pool.pop(mac)
> +
> + mac_shelve["macpool"] = mac_pool
> + logging.debug("freeing mac addr %s " % mac)
> +
> + mac_shelve.close()
> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> + lock_file.close()
> + return mac
> +
> +
> def mac_str_to_int(addr):
> """
> Convert MAC address string to integer.
> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
> index 6cd0688..a9ba6e7 100755
> --- a/client/tests/kvm/kvm_vm.py
> +++ b/client/tests/kvm/kvm_vm.py
> @@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu.
> @copyright: 2008-2009 Red Hat Inc.
> """
>
> -import time, socket, os, logging, fcntl, re, commands, glob
> +import time, socket, os, logging, fcntl, re, commands, shelve, glob
> import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer
> from autotest_lib.client.common_lib import error
> from autotest_lib.client.bin import utils
> @@ -117,6 +117,7 @@ class VM:
> self.params = params
> self.root_dir = root_dir
> self.address_cache = address_cache
> + self.mac_prefix = params.get('mac_prefix')
> self.netdev_id = []
>
> # Find a unique identifier for this VM
> @@ -126,8 +127,15 @@ class VM:
> if not glob.glob("/tmp/*%s" % self.instance):
> break
>
> + if self.mac_prefix is None:
> + s, o = commands.getstatusoutput("ifconfig eth0")
> + if s == 0:
> + mac = re.findall("HWaddr (\S*)", o)[0]
> + self.mac_prefix = '00' + mac[8:] + ':'
> +
>
> - def clone(self, name=None, params=None, root_dir=None, address_cache=None):
> + def clone(self, name=None, params=None, root_dir=None,
> + address_cache=None, mac_clone=True):
> """
> Return a clone of the VM object with optionally modified parameters.
> The clone is initially not alive and needs to be started using create().
> @@ -138,6 +146,7 @@ class VM:
> @param params: Optional new VM creation parameters
> @param root_dir: Optional new base directory for relative filenames
> @param address_cache: A dict that maps MAC addresses to IP addresses
> + @param mac_clone: Clone mac address or not.
> """
> if name is None:
> name = self.name
> @@ -147,9 +156,19 @@ class VM:
> root_dir = self.root_dir
> if address_cache is None:
> address_cache = self.address_cache
> - return VM(name, params, root_dir, address_cache)
> + vm = VM(name, params, root_dir, address_cache)
> + if mac_clone:
> + # Clone mac address by coping 'self.instance' to the new vm.
> + vm.instance = self.instance
Copying self.instance isn't a good idea because the monitor, serial
console and testlog filenames use self.instance. self.instance should
be unique. If we still want to use the same mac address for 2 VMs, for
example in migration, a different solution should be found.
> + return vm
>
>
> + def free_mac_address(self):
> + nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics"))
> + for nic in range(nic_num):
> + mac = self.get_macaddr(nic_index=nic)
> + kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance)
> +
> def make_qemu_command(self, name=None, params=None, root_dir=None):
> """
> Generate a qemu command line. All parameters are optional. If a
> @@ -383,6 +402,13 @@ class VM:
> mac = None
> if "address_index" in nic_params:
> mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0]
> + self.set_mac_address(mac=mac, nic_index=vlan)
> + else:
> + mac = kvm_utils.get_mac_from_pool(self.root_dir,
> + vm=self.instance,
> + nic_index=vlan,
> + prefix=self.mac_prefix)
> +
> qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac,
> self.netdev_id[vlan])
> # Handle the '-net tap' or '-net user' part
> @@ -396,7 +422,7 @@ class VM:
> if tftp:
> tftp = kvm_utils.get_path(root_dir, tftp)
> qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"),
> - nic_params.get("nic_ifname"),
> + self.get_ifname(vlan),
Why is get_ifname() useful here?
> script, downscript, tftp,
> nic_params.get("bootp"), redirs,
> self.netdev_id[vlan])
> @@ -720,7 +746,7 @@ class VM:
> lockfile.close()
>
>
> - def destroy(self, gracefully=True):
> + def destroy(self, gracefully=True, free_macaddr=True):
> """
> Destroy the VM.
>
> @@ -731,6 +757,7 @@ class VM:
> @param gracefully: Whether an attempt will be made to end the VM
> using a shell command before trying to end the qemu process
> with a 'quit' or a kill signal.
> + @param free_macaddr: Whether free macaddresses when destory a vm.
> """
> try:
> # Is it already dead?
> @@ -751,11 +778,18 @@ class VM:
> logging.debug("Shutdown command sent; waiting for VM "
> "to go down...")
> if kvm_utils.wait_for(self.is_dead, 60, 1, 1):
> - logging.debug("VM is down")
> + logging.debug("VM is down, free mac address.")
> + # free mac address
> + if free_macaddr:
> + self.free_mac_address()
> return
> finally:
> session.close()
>
> + # free mac address
> + if free_macaddr:
> + self.free_mac_address()
> +
> if self.monitor:
> # Try to destroy with a monitor command
> logging.debug("Trying to kill VM with monitor command...")
> @@ -881,10 +915,14 @@ class VM:
> nic_name = nics[index]
> nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
> if nic_params.get("nic_mode") == "tap":
> - mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params)
> + mac = self.get_macaddr(index)
> if not mac:
> logging.debug("MAC address unavailable")
> return None
> + else:
> + mac = mac.lower()
> + ip = None
> +
> if not ip or nic_params.get("always_use_tcpdump") == "yes":
> # Get the IP address from the cache
> ip = self.address_cache.get(mac)
> @@ -897,6 +935,7 @@ class VM:
> for nic in nics]
> macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0]
> for dict in nic_dicts]
> + macs.append(mac)
> if not kvm_utils.verify_ip_address_ownership(ip, macs):
> logging.debug("Could not verify MAC-IP address mapping: "
> "%s ---> %s" % (mac, ip))
> @@ -925,6 +964,71 @@ class VM:
> "redirected" % port)
> return self.redirs.get(port)
>
> + def get_ifname(self, nic_index=0):
> + """
> + Return the ifname of tap device for the guest nic.
> +
> + @param nic_index: Index of the NIC
> + """
> +
> + nics = kvm_utils.get_sub_dict_names(self.params, "nics")
> + nic_name = nics[nic_index]
> + nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
> + if nic_params.get("nic_ifname"):
> + return nic_params.get("nic_ifname")
> + else:
> + return "%s_%s_%s" % (nic_params.get("nic_model"),
> + nic_index, self.vnc_port)
What's the purpose of this string?
> + def get_macaddr(self, nic_index=0):
> + """
> + Return the macaddr of guest nic.
> +
> + @param nic_index: Index of the NIC
> + """
> + mac_filename = os.path.join(self.root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> + mac_pool = mac_shelve.get("macpool")
> + val = "%s:%s" % (self.instance, nic_index)
> + for key in mac_pool.keys():
> + if val in mac_pool[key]:
> + return key
> + return None
> +
> + def set_mac_address(self, mac, nic_index=0, shareable=False):
> + """
> + Set mac address for guest. Note: It just update address pool.
> +
> + @param mac: address will set to guest
> + @param nic_index: Index of the NIC
> + @param shareable: Where VM can share mac with other VM or not.
> + """
> + lock_filename = os.path.join(self.root_dir, "mac_lock")
> + lock_file = open(lock_filename, 'w')
> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> + mac_filename = os.path.join(self.root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> + mac_pool = mac_shelve.get("macpool")
> + if not mac_pool:
> + mac_pool = {}
> + value = "%s:%s" % (self.instance, nic_index)
> + if mac not in mac_pool.keys():
> + for key in mac_pool.keys():
> + if value in mac_pool[key]:
> + mac_pool[key].remove(value)
> + if len(mac_pool[key]) == 0:
> + mac_pool.pop(key)
> + mac_pool[mac] = [value]
> + else:
> + if shareable:
> + mac_pool[mac].append(value)
> + else:
> + logging.error("Mac address already be used!")
> + return False
> + mac_shelve["macpool"] = mac_pool
> + mac_shelve.close()
> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> + lock_file.close()
>
> def get_pid(self):
> """
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index fd9e72f..6710c00 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -51,7 +51,7 @@ guest_port_remote_shell = 22
> nic_mode = user
> #nic_mode = tap
> nic_script = scripts/qemu-ifup
> -address_index = 0
> +#address_index = 0
> run_tcpdump = yes
>
> # Misc
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
@ 2010-07-20 10:19 ` Michael Goldish
0 siblings, 0 replies; 77+ messages in thread
From: Michael Goldish @ 2010-07-20 10:19 UTC (permalink / raw)
To: Amos Kong; +Cc: lmr, autotest, qemu-devel, kvm
On 07/20/2010 04:34 AM, Amos Kong wrote:
> Old method uses the mac address in the configuration files which could
> lead serious problem when multiple tests running in different hosts.
>
> This patch adds a new macaddress pool algorithm, it generates the mac prefix
> based on mac address of the host which could eliminate the duplicated mac
> addresses between machines.
>
> When user have set the mac_prefix in the configuration file, we should use it
> in stead of the dynamic generated mac prefix.
>
> Other change:
> . Fix randomly generating mac address so that it correspond to IEEE802.
> . Update clone function to decide clone mac address or not.
> . Update get_macaddr function.
> . Add set_mac_address function.
>
> New auto mac address pool algorithm:
> If address_index is defined, VM will get mac from config file then record mac
> in to address_pool. If address_index is not defined, VM will call
> get_mac_from_pool to auto create mac then recored mac to address_pool in
> following format:
> {'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}
>
> AE:9D:94:6A:9b:f9 : mac address
> 20100310-165222-Wt7l : instance attribute of VM
> 0 : index of NIC
Why do you use the mac address as a key, instead of the instance string
+ nic index? When the mac address is used as a key, each key has a list
of values instead of just one value. This order seems unnatural. If it
were the other way around (i.e. key = VM instance + nic index, value =
mac address), then each key would have exactly one value, and I think
this patch would be shorter and simpler.
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Feng Yang <fyang@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
> index fb2d1c2..7c0946e 100644
> --- a/client/tests/kvm/kvm_utils.py
> +++ b/client/tests/kvm/kvm_utils.py
> @@ -5,6 +5,7 @@ KVM test utility functions.
> """
>
> import time, string, random, socket, os, signal, re, logging, commands, cPickle
> +import fcntl, shelve
> from autotest_lib.client.bin import utils
> from autotest_lib.client.common_lib import error, logging_config
> import kvm_subprocess
> @@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword):
>
> # Functions related to MAC/IP addresses
>
> +def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'):
The name of this function is confusing because it does the exact
opposite: it puts a mac address in address_pool. Maybe the pool you're
referring to in the name isn't address_pool, but still a less confusing
name should probably be used.
> + """
> + random generated mac address.
> +
> + 1) First try to generate macaddress based on the mac address prefix.
> + 2) And then try to use total random generated mac address.
> +
> + @param root_dir: Root dir for kvm
> + @param vm: Here we use instance of vm
> + @param nic_index: The index of nic.
> + @param prefix: Prefix of mac address.
> + @Return: Return mac address.
> + """
> +
> + lock_filename = os.path.join(root_dir, "mac_lock")
> + lock_file = open(lock_filename, 'w')
> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> + mac_filename = os.path.join(root_dir, "address_pool")
Maybe it makes sense to put address_pool and the lock file in /tmp,
where they can be shared by more than a single autotest instance running
on the same host (unlikely, but theoretically possible).
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> +
> + mac_pool = mac_shelve.get("macpool")
Why is this 'macpool' needed? Why not put the keys directly in the
shelve object?
> +
> + if not mac_pool:
> + mac_pool = {}
> + found = False
> +
> + val = "%s:%s" % (vm, nic_index)
> + for key in mac_pool.keys():
> + if val in mac_pool[key]:
> + mac_pool[key].append(val)
Why append val to mac_pool[key] if val is already in mac_pool[key]?
> + found = True
> + mac = key
> +
> + while not found:
> + postfix = "%02x:%02x" % (random.randint(0x00,0xfe),
> + random.randint(0x00,0xfe))
> + mac = prefix + postfix
> + mac_list = mac.split(":")
> + # Clear multicast bit
> + mac_list[0] = int(mac_list[0],16) & 0xfe
> + # Set local assignment bit (IEEE802)
> + mac_list[0] = mac_list[0] | 0x02
> + mac_list[0] = "%02x" % mac_list[0]
Why is this needed? Most mac addresses begin with 00. If the mac
address is generated from the address of eth0 (using the method in this
patch) it begins with 00, which is fine. If the prefix is set by the
user using mac_prefix, I don't think we should modify it.
> + mac = ":".join(mac_list)
> + if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]):
> + mac_pool[mac] = ["%s:%s" % (vm,nic_index)]
> + found = True
> + mac_shelve["macpool"] = mac_pool
> + logging.debug("generating mac addr %s " % mac)
> +
> + mac_shelve.close()
> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> + lock_file.close()
> + return mac
> +
> +
> +def put_mac_to_pool(root_dir, mac, vm):
This function removes a mac address from address_pool. I wonder why you
chose this name.
> + """
> + Put the macaddress back to address pool
> +
> + @param root_dir: Root dir for kvm
> + @param vm: Here we use instance attribute of vm
> + @param mac: mac address will be put.
> + @Return: mac address.
> + """
> +
> + lock_filename = os.path.join(root_dir, "mac_lock")
> + lock_file = open(lock_filename, 'w')
> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> + mac_filename = os.path.join(root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> +
> + mac_pool = mac_shelve.get("macpool")
> +
> + if not mac_pool or (not mac in mac_pool):
> + logging.debug("Try to free a macaddress does no in pool")
> + logging.debug("macaddress is %s" % mac)
> + logging.debug("pool is %s" % mac_pool)
> + else:
> + if len(mac_pool[mac]) <= 1:
> + mac_pool.pop(mac)
> + else:
> + for value in mac_pool[mac]:
> + if vm in value:
> + mac_pool[mac].remove(value)
> + break
> + if len(mac_pool[mac]) == 0:
> + mac_pool.pop(mac)
> +
> + mac_shelve["macpool"] = mac_pool
> + logging.debug("freeing mac addr %s " % mac)
> +
> + mac_shelve.close()
> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> + lock_file.close()
> + return mac
> +
> +
> def mac_str_to_int(addr):
> """
> Convert MAC address string to integer.
> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
> index 6cd0688..a9ba6e7 100755
> --- a/client/tests/kvm/kvm_vm.py
> +++ b/client/tests/kvm/kvm_vm.py
> @@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu.
> @copyright: 2008-2009 Red Hat Inc.
> """
>
> -import time, socket, os, logging, fcntl, re, commands, glob
> +import time, socket, os, logging, fcntl, re, commands, shelve, glob
> import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer
> from autotest_lib.client.common_lib import error
> from autotest_lib.client.bin import utils
> @@ -117,6 +117,7 @@ class VM:
> self.params = params
> self.root_dir = root_dir
> self.address_cache = address_cache
> + self.mac_prefix = params.get('mac_prefix')
> self.netdev_id = []
>
> # Find a unique identifier for this VM
> @@ -126,8 +127,15 @@ class VM:
> if not glob.glob("/tmp/*%s" % self.instance):
> break
>
> + if self.mac_prefix is None:
> + s, o = commands.getstatusoutput("ifconfig eth0")
> + if s == 0:
> + mac = re.findall("HWaddr (\S*)", o)[0]
> + self.mac_prefix = '00' + mac[8:] + ':'
> +
>
> - def clone(self, name=None, params=None, root_dir=None, address_cache=None):
> + def clone(self, name=None, params=None, root_dir=None,
> + address_cache=None, mac_clone=True):
> """
> Return a clone of the VM object with optionally modified parameters.
> The clone is initially not alive and needs to be started using create().
> @@ -138,6 +146,7 @@ class VM:
> @param params: Optional new VM creation parameters
> @param root_dir: Optional new base directory for relative filenames
> @param address_cache: A dict that maps MAC addresses to IP addresses
> + @param mac_clone: Clone mac address or not.
> """
> if name is None:
> name = self.name
> @@ -147,9 +156,19 @@ class VM:
> root_dir = self.root_dir
> if address_cache is None:
> address_cache = self.address_cache
> - return VM(name, params, root_dir, address_cache)
> + vm = VM(name, params, root_dir, address_cache)
> + if mac_clone:
> + # Clone mac address by coping 'self.instance' to the new vm.
> + vm.instance = self.instance
Copying self.instance isn't a good idea because the monitor, serial
console and testlog filenames use self.instance. self.instance should
be unique. If we still want to use the same mac address for 2 VMs, for
example in migration, a different solution should be found.
> + return vm
>
>
> + def free_mac_address(self):
> + nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics"))
> + for nic in range(nic_num):
> + mac = self.get_macaddr(nic_index=nic)
> + kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance)
> +
> def make_qemu_command(self, name=None, params=None, root_dir=None):
> """
> Generate a qemu command line. All parameters are optional. If a
> @@ -383,6 +402,13 @@ class VM:
> mac = None
> if "address_index" in nic_params:
> mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0]
> + self.set_mac_address(mac=mac, nic_index=vlan)
> + else:
> + mac = kvm_utils.get_mac_from_pool(self.root_dir,
> + vm=self.instance,
> + nic_index=vlan,
> + prefix=self.mac_prefix)
> +
> qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac,
> self.netdev_id[vlan])
> # Handle the '-net tap' or '-net user' part
> @@ -396,7 +422,7 @@ class VM:
> if tftp:
> tftp = kvm_utils.get_path(root_dir, tftp)
> qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"),
> - nic_params.get("nic_ifname"),
> + self.get_ifname(vlan),
Why is get_ifname() useful here?
> script, downscript, tftp,
> nic_params.get("bootp"), redirs,
> self.netdev_id[vlan])
> @@ -720,7 +746,7 @@ class VM:
> lockfile.close()
>
>
> - def destroy(self, gracefully=True):
> + def destroy(self, gracefully=True, free_macaddr=True):
> """
> Destroy the VM.
>
> @@ -731,6 +757,7 @@ class VM:
> @param gracefully: Whether an attempt will be made to end the VM
> using a shell command before trying to end the qemu process
> with a 'quit' or a kill signal.
> + @param free_macaddr: Whether free macaddresses when destory a vm.
> """
> try:
> # Is it already dead?
> @@ -751,11 +778,18 @@ class VM:
> logging.debug("Shutdown command sent; waiting for VM "
> "to go down...")
> if kvm_utils.wait_for(self.is_dead, 60, 1, 1):
> - logging.debug("VM is down")
> + logging.debug("VM is down, free mac address.")
> + # free mac address
> + if free_macaddr:
> + self.free_mac_address()
> return
> finally:
> session.close()
>
> + # free mac address
> + if free_macaddr:
> + self.free_mac_address()
> +
> if self.monitor:
> # Try to destroy with a monitor command
> logging.debug("Trying to kill VM with monitor command...")
> @@ -881,10 +915,14 @@ class VM:
> nic_name = nics[index]
> nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
> if nic_params.get("nic_mode") == "tap":
> - mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params)
> + mac = self.get_macaddr(index)
> if not mac:
> logging.debug("MAC address unavailable")
> return None
> + else:
> + mac = mac.lower()
> + ip = None
> +
> if not ip or nic_params.get("always_use_tcpdump") == "yes":
> # Get the IP address from the cache
> ip = self.address_cache.get(mac)
> @@ -897,6 +935,7 @@ class VM:
> for nic in nics]
> macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0]
> for dict in nic_dicts]
> + macs.append(mac)
> if not kvm_utils.verify_ip_address_ownership(ip, macs):
> logging.debug("Could not verify MAC-IP address mapping: "
> "%s ---> %s" % (mac, ip))
> @@ -925,6 +964,71 @@ class VM:
> "redirected" % port)
> return self.redirs.get(port)
>
> + def get_ifname(self, nic_index=0):
> + """
> + Return the ifname of tap device for the guest nic.
> +
> + @param nic_index: Index of the NIC
> + """
> +
> + nics = kvm_utils.get_sub_dict_names(self.params, "nics")
> + nic_name = nics[nic_index]
> + nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
> + if nic_params.get("nic_ifname"):
> + return nic_params.get("nic_ifname")
> + else:
> + return "%s_%s_%s" % (nic_params.get("nic_model"),
> + nic_index, self.vnc_port)
What's the purpose of this string?
> + def get_macaddr(self, nic_index=0):
> + """
> + Return the macaddr of guest nic.
> +
> + @param nic_index: Index of the NIC
> + """
> + mac_filename = os.path.join(self.root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> + mac_pool = mac_shelve.get("macpool")
> + val = "%s:%s" % (self.instance, nic_index)
> + for key in mac_pool.keys():
> + if val in mac_pool[key]:
> + return key
> + return None
> +
> + def set_mac_address(self, mac, nic_index=0, shareable=False):
> + """
> + Set mac address for guest. Note: It just update address pool.
> +
> + @param mac: address will set to guest
> + @param nic_index: Index of the NIC
> + @param shareable: Where VM can share mac with other VM or not.
> + """
> + lock_filename = os.path.join(self.root_dir, "mac_lock")
> + lock_file = open(lock_filename, 'w')
> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> + mac_filename = os.path.join(self.root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> + mac_pool = mac_shelve.get("macpool")
> + if not mac_pool:
> + mac_pool = {}
> + value = "%s:%s" % (self.instance, nic_index)
> + if mac not in mac_pool.keys():
> + for key in mac_pool.keys():
> + if value in mac_pool[key]:
> + mac_pool[key].remove(value)
> + if len(mac_pool[key]) == 0:
> + mac_pool.pop(key)
> + mac_pool[mac] = [value]
> + else:
> + if shareable:
> + mac_pool[mac].append(value)
> + else:
> + logging.error("Mac address already be used!")
> + return False
> + mac_shelve["macpool"] = mac_pool
> + mac_shelve.close()
> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> + lock_file.close()
>
> def get_pid(self):
> """
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index fd9e72f..6710c00 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -51,7 +51,7 @@ guest_port_remote_shell = 22
> nic_mode = user
> #nic_mode = tap
> nic_script = scripts/qemu-ifup
> -address_index = 0
> +#address_index = 0
> run_tcpdump = yes
>
> # Misc
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
2010-07-20 10:19 ` Michael Goldish
(?)
@ 2010-07-20 13:44 ` Amos Kong
2010-07-20 15:53 ` Michael Goldish
-1 siblings, 1 reply; 77+ messages in thread
From: Amos Kong @ 2010-07-20 13:44 UTC (permalink / raw)
To: Michael Goldish; +Cc: lmr, autotest, qemu-devel, kvm
On Tue, Jul 20, 2010 at 01:19:39PM +0300, Michael Goldish wrote:
>
Michael,
Thanks for your comments. Let's simplify this method together.
> On 07/20/2010 04:34 AM, Amos Kong wrote:
> > Old method uses the mac address in the configuration files which could
> > lead serious problem when multiple tests running in different hosts.
> >
> > This patch adds a new macaddress pool algorithm, it generates the mac prefix
> > based on mac address of the host which could eliminate the duplicated mac
> > addresses between machines.
> >
> > When user have set the mac_prefix in the configuration file, we should use it
> > in stead of the dynamic generated mac prefix.
> >
> > Other change:
> > . Fix randomly generating mac address so that it correspond to IEEE802.
> > . Update clone function to decide clone mac address or not.
> > . Update get_macaddr function.
> > . Add set_mac_address function.
> >
> > New auto mac address pool algorithm:
> > If address_index is defined, VM will get mac from config file then record mac
> > in to address_pool. If address_index is not defined, VM will call
> > get_mac_from_pool to auto create mac then recored mac to address_pool in
> > following format:
> > {'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}
> >
> > AE:9D:94:6A:9b:f9 : mac address
> > 20100310-165222-Wt7l : instance attribute of VM
> > 0 : index of NIC
>
> Why do you use the mac address as a key, instead of the instance string
> + nic index? When the mac address is used as a key, each key has a list
> of values instead of just one value. This order seems unnatural. If it
> were the other way around (i.e. key = VM instance + nic index, value =
> mac address), then each key would have exactly one value, and I think
> this patch would be shorter and simpler.
One mac address may be used by two VMs, eg. migration.
> > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > Signed-off-by: Feng Yang <fyang@redhat.com>
> > Signed-off-by: Amos Kong <akong@redhat.com>
> > ---
> > 0 files changed, 0 insertions(+), 0 deletions(-)
> >
> > diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
> > index fb2d1c2..7c0946e 100644
> > --- a/client/tests/kvm/kvm_utils.py
> > +++ b/client/tests/kvm/kvm_utils.py
> > @@ -5,6 +5,7 @@ KVM test utility functions.
> > """
> >
> > import time, string, random, socket, os, signal, re, logging, commands, cPickle
> > +import fcntl, shelve
> > from autotest_lib.client.bin import utils
> > from autotest_lib.client.common_lib import error, logging_config
> > import kvm_subprocess
> > @@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword):
> >
> > # Functions related to MAC/IP addresses
> >
> > +def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'):
>
> The name of this function is confusing because it does the exact
> opposite: it puts a mac address in address_pool. Maybe the pool you're
> referring to in the name isn't address_pool, but still a less confusing
> name should probably be used.
How about allocate_mac(...) ?
address_pool -> address_container
Allocate mac address and record into address_container.
> > + """
> > + random generated mac address.
> > +
> > + 1) First try to generate macaddress based on the mac address prefix.
> > + 2) And then try to use total random generated mac address.
> > +
> > + @param root_dir: Root dir for kvm
> > + @param vm: Here we use instance of vm
> > + @param nic_index: The index of nic.
> > + @param prefix: Prefix of mac address.
> > + @Return: Return mac address.
> > + """
> > +
> > + lock_filename = os.path.join(root_dir, "mac_lock")
> > + lock_file = open(lock_filename, 'w')
> > + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> > + mac_filename = os.path.join(root_dir, "address_pool")
>
> Maybe it makes sense to put address_pool and the lock file in /tmp,
> where they can be shared by more than a single autotest instance running
> on the same host (unlikely, but theoretically possible).
good idea.
> > + mac_shelve = shelve.open(mac_filename, writeback=False)
> > +
> > + mac_pool = mac_shelve.get("macpool")
>
> Why is this 'macpool' needed? Why not put the keys directly in the
> shelve object?
yes, put keys directly in the shelve object is better.
> > + if not mac_pool:
> > + mac_pool = {}
> > + found = False
> > +
> > + val = "%s:%s" % (vm, nic_index)
> > + for key in mac_pool.keys():
> > + if val in mac_pool[key]:
> > + mac_pool[key].append(val)
>
> Why append val to mac_pool[key] if val is already in mac_pool[key]?
need drop it.
> > + found = True
> > + mac = key
> > +
> > + while not found:
> > + postfix = "%02x:%02x" % (random.randint(0x00,0xfe),
> > + random.randint(0x00,0xfe))
> > + mac = prefix + postfix
> > + mac_list = mac.split(":")
> > + # Clear multicast bit
> > + mac_list[0] = int(mac_list[0],16) & 0xfe
> > + # Set local assignment bit (IEEE802)
> > + mac_list[0] = mac_list[0] | 0x02
> > + mac_list[0] = "%02x" % mac_list[0]
>
> Why is this needed? Most mac addresses begin with 00. If the mac
> address is generated from the address of eth0 (using the method in this
> patch) it begins with 00, which is fine. If the prefix is set by the
> user using mac_prefix, I don't think we should modify it.
linux-2.6/include/linux/etherdevice.h
/**
* random_ether_addr - Generate software assigned random Ethernet address
* @addr: Pointer to a six-byte array containing the Ethernet address
*
* Generate a random Ethernet address (MAC) that is not multicast
* and has the local assigned bit set.
*/
static inline void random_ether_addr(u8 *addr)
{
get_random_bytes (addr, ETH_ALEN);
addr [0] &= 0xfe; /* clear multicast bit */
addr [0] |= 0x02; /* set local assignment bit (IEEE802) */
}
> > + mac = ":".join(mac_list)
> > + if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]):
> > + mac_pool[mac] = ["%s:%s" % (vm,nic_index)]
> > + found = True
> > + mac_shelve["macpool"] = mac_pool
> > + logging.debug("generating mac addr %s " % mac)
> > +
> > + mac_shelve.close()
> > + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> > + lock_file.close()
> > + return mac
> > +
> > +
> > +def put_mac_to_pool(root_dir, mac, vm):
>
> This function removes a mac address from address_pool. I wonder why you
> chose this name.
address_pool file is a container to record macaddress.
The 'pool' we get/put mac is an abstract resource pool.
This is really confused ;)
> > + """
> > + Put the macaddress back to address pool
> > +
> > + @param root_dir: Root dir for kvm
> > + @param vm: Here we use instance attribute of vm
> > + @param mac: mac address will be put.
> > + @Return: mac address.
> > + """
> > +
> > + lock_filename = os.path.join(root_dir, "mac_lock")
> > + lock_file = open(lock_filename, 'w')
> > + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> > + mac_filename = os.path.join(root_dir, "address_pool")
> > + mac_shelve = shelve.open(mac_filename, writeback=False)
> > +
> > + mac_pool = mac_shelve.get("macpool")
> > +
> > + if not mac_pool or (not mac in mac_pool):
> > + logging.debug("Try to free a macaddress does no in pool")
> > + logging.debug("macaddress is %s" % mac)
> > + logging.debug("pool is %s" % mac_pool)
> > + else:
> > + if len(mac_pool[mac]) <= 1:
> > + mac_pool.pop(mac)
> > + else:
> > + for value in mac_pool[mac]:
> > + if vm in value:
> > + mac_pool[mac].remove(value)
> > + break
> > + if len(mac_pool[mac]) == 0:
> > + mac_pool.pop(mac)
> > +
> > + mac_shelve["macpool"] = mac_pool
> > + logging.debug("freeing mac addr %s " % mac)
> > +
> > + mac_shelve.close()
> > + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> > + lock_file.close()
> > + return mac
> > +
> > +
> > def mac_str_to_int(addr):
> > """
> > Convert MAC address string to integer.
> > diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
> > index 6cd0688..a9ba6e7 100755
> > --- a/client/tests/kvm/kvm_vm.py
> > +++ b/client/tests/kvm/kvm_vm.py
> > @@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu.
> > @copyright: 2008-2009 Red Hat Inc.
> > """
> >
> > -import time, socket, os, logging, fcntl, re, commands, glob
> > +import time, socket, os, logging, fcntl, re, commands, shelve, glob
> > import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer
> > from autotest_lib.client.common_lib import error
> > from autotest_lib.client.bin import utils
> > @@ -117,6 +117,7 @@ class VM:
> > self.params = params
> > self.root_dir = root_dir
> > self.address_cache = address_cache
> > + self.mac_prefix = params.get('mac_prefix')
> > self.netdev_id = []
> >
> > # Find a unique identifier for this VM
> > @@ -126,8 +127,15 @@ class VM:
> > if not glob.glob("/tmp/*%s" % self.instance):
> > break
> >
> > + if self.mac_prefix is None:
> > + s, o = commands.getstatusoutput("ifconfig eth0")
> > + if s == 0:
> > + mac = re.findall("HWaddr (\S*)", o)[0]
> > + self.mac_prefix = '00' + mac[8:] + ':'
> > +
> >
> > - def clone(self, name=None, params=None, root_dir=None, address_cache=None):
> > + def clone(self, name=None, params=None, root_dir=None,
> > + address_cache=None, mac_clone=True):
> > """
> > Return a clone of the VM object with optionally modified parameters.
> > The clone is initially not alive and needs to be started using create().
> > @@ -138,6 +146,7 @@ class VM:
> > @param params: Optional new VM creation parameters
> > @param root_dir: Optional new base directory for relative filenames
> > @param address_cache: A dict that maps MAC addresses to IP addresses
> > + @param mac_clone: Clone mac address or not.
> > """
> > if name is None:
> > name = self.name
> > @@ -147,9 +156,19 @@ class VM:
> > root_dir = self.root_dir
> > if address_cache is None:
> > address_cache = self.address_cache
> > - return VM(name, params, root_dir, address_cache)
> > + vm = VM(name, params, root_dir, address_cache)
> > + if mac_clone:
> > + # Clone mac address by coping 'self.instance' to the new vm.
> > + vm.instance = self.instance
>
> Copying self.instance isn't a good idea because the monitor, serial
> console and testlog filenames use self.instance. self.instance should
> be unique. If we still want to use the same mac address for 2 VMs, for
> example in migration, a different solution should be found.
We almost only clone mac in migration test, the src guest would be killed.
If the instance of dest guest is same as src guest, the serial connection would not break,
it would continually write log to original log file.
> > + return vm
> >
> >
> > + def free_mac_address(self):
> > + nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics"))
> > + for nic in range(nic_num):
> > + mac = self.get_macaddr(nic_index=nic)
> > + kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance)
> > +
> > def make_qemu_command(self, name=None, params=None, root_dir=None):
> > """
> > Generate a qemu command line. All parameters are optional. If a
> > @@ -383,6 +402,13 @@ class VM:
> > mac = None
> > if "address_index" in nic_params:
> > mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0]
> > + self.set_mac_address(mac=mac, nic_index=vlan)
> > + else:
> > + mac = kvm_utils.get_mac_from_pool(self.root_dir,
> > + vm=self.instance,
> > + nic_index=vlan,
> > + prefix=self.mac_prefix)
> > +
> > qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac,
> > self.netdev_id[vlan])
> > # Handle the '-net tap' or '-net user' part
> > @@ -396,7 +422,7 @@ class VM:
> > if tftp:
> > tftp = kvm_utils.get_path(root_dir, tftp)
> > qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"),
> > - nic_params.get("nic_ifname"),
> > + self.get_ifname(vlan),
>
> Why is get_ifname() useful here?
dynamically generate the ifname is better.
> > script, downscript, tftp,
> > nic_params.get("bootp"), redirs,
> > self.netdev_id[vlan])
> > @@ -720,7 +746,7 @@ class VM:
> > lockfile.close()
> >
> >
> > - def destroy(self, gracefully=True):
> > + def destroy(self, gracefully=True, free_macaddr=True):
> > """
> > Destroy the VM.
> >
> > @@ -731,6 +757,7 @@ class VM:
> > @param gracefully: Whether an attempt will be made to end the VM
> > using a shell command before trying to end the qemu process
> > with a 'quit' or a kill signal.
> > + @param free_macaddr: Whether free macaddresses when destory a vm.
> > """
> > try:
> > # Is it already dead?
> > @@ -751,11 +778,18 @@ class VM:
> > logging.debug("Shutdown command sent; waiting for VM "
> > "to go down...")
> > if kvm_utils.wait_for(self.is_dead, 60, 1, 1):
> > - logging.debug("VM is down")
> > + logging.debug("VM is down, free mac address.")
> > + # free mac address
> > + if free_macaddr:
> > + self.free_mac_address()
> > return
> > finally:
> > session.close()
> >
> > + # free mac address
> > + if free_macaddr:
> > + self.free_mac_address()
> > +
> > if self.monitor:
> > # Try to destroy with a monitor command
> > logging.debug("Trying to kill VM with monitor command...")
> > @@ -881,10 +915,14 @@ class VM:
> > nic_name = nics[index]
> > nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
> > if nic_params.get("nic_mode") == "tap":
> > - mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params)
> > + mac = self.get_macaddr(index)
> > if not mac:
> > logging.debug("MAC address unavailable")
> > return None
> > + else:
> > + mac = mac.lower()
> > + ip = None
> > +
> > if not ip or nic_params.get("always_use_tcpdump") == "yes":
> > # Get the IP address from the cache
> > ip = self.address_cache.get(mac)
> > @@ -897,6 +935,7 @@ class VM:
> > for nic in nics]
> > macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0]
> > for dict in nic_dicts]
> > + macs.append(mac)
> > if not kvm_utils.verify_ip_address_ownership(ip, macs):
> > logging.debug("Could not verify MAC-IP address mapping: "
> > "%s ---> %s" % (mac, ip))
> > @@ -925,6 +964,71 @@ class VM:
> > "redirected" % port)
> > return self.redirs.get(port)
> >
> > + def get_ifname(self, nic_index=0):
> > + """
> > + Return the ifname of tap device for the guest nic.
> > +
> > + @param nic_index: Index of the NIC
> > + """
> > +
> > + nics = kvm_utils.get_sub_dict_names(self.params, "nics")
> > + nic_name = nics[nic_index]
> > + nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
> > + if nic_params.get("nic_ifname"):
> > + return nic_params.get("nic_ifname")
> > + else:
> > + return "%s_%s_%s" % (nic_params.get("nic_model"),
> > + nic_index, self.vnc_port)
>
> What's the purpose of this string?
Just avoid repeated ifname. The vnc_port is unique for each VM, nic_index is unique for each nic of one VM.
> > + def get_macaddr(self, nic_index=0):
> > + """
> > + Return the macaddr of guest nic.
> > +
> > + @param nic_index: Index of the NIC
> > + """
> > + mac_filename = os.path.join(self.root_dir, "address_pool")
> > + mac_shelve = shelve.open(mac_filename, writeback=False)
> > + mac_pool = mac_shelve.get("macpool")
> > + val = "%s:%s" % (self.instance, nic_index)
> > + for key in mac_pool.keys():
> > + if val in mac_pool[key]:
> > + return key
> > + return None
> > +
> > + def set_mac_address(self, mac, nic_index=0, shareable=False):
> > + """
> > + Set mac address for guest. Note: It just update address pool.
> > +
> > + @param mac: address will set to guest
> > + @param nic_index: Index of the NIC
> > + @param shareable: Where VM can share mac with other VM or not.
> > + """
> > + lock_filename = os.path.join(self.root_dir, "mac_lock")
> > + lock_file = open(lock_filename, 'w')
> > + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> > + mac_filename = os.path.join(self.root_dir, "address_pool")
> > + mac_shelve = shelve.open(mac_filename, writeback=False)
> > + mac_pool = mac_shelve.get("macpool")
> > + if not mac_pool:
> > + mac_pool = {}
> > + value = "%s:%s" % (self.instance, nic_index)
> > + if mac not in mac_pool.keys():
> > + for key in mac_pool.keys():
> > + if value in mac_pool[key]:
> > + mac_pool[key].remove(value)
> > + if len(mac_pool[key]) == 0:
> > + mac_pool.pop(key)
> > + mac_pool[mac] = [value]
> > + else:
> > + if shareable:
> > + mac_pool[mac].append(value)
> > + else:
> > + logging.error("Mac address already be used!")
> > + return False
> > + mac_shelve["macpool"] = mac_pool
> > + mac_shelve.close()
> > + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> > + lock_file.close()
> >
> > def get_pid(self):
> > """
> > diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> > index fd9e72f..6710c00 100644
> > --- a/client/tests/kvm/tests_base.cfg.sample
> > +++ b/client/tests/kvm/tests_base.cfg.sample
> > @@ -51,7 +51,7 @@ guest_port_remote_shell = 22
> > nic_mode = user
> > #nic_mode = tap
> > nic_script = scripts/qemu-ifup
> > -address_index = 0
> > +#address_index = 0
> > run_tcpdump = yes
> >
> > # Misc
> >
> >
>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
2010-07-20 13:44 ` Amos Kong
@ 2010-07-20 15:53 ` Michael Goldish
0 siblings, 0 replies; 77+ messages in thread
From: Michael Goldish @ 2010-07-20 15:53 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On 07/20/2010 04:44 PM, Amos Kong wrote:
> On Tue, Jul 20, 2010 at 01:19:39PM +0300, Michael Goldish wrote:
>>
>
> Michael,
>
> Thanks for your comments. Let's simplify this method together.
>
>> On 07/20/2010 04:34 AM, Amos Kong wrote:
>>> Old method uses the mac address in the configuration files which could
>>> lead serious problem when multiple tests running in different hosts.
>>>
>>> This patch adds a new macaddress pool algorithm, it generates the mac prefix
>>> based on mac address of the host which could eliminate the duplicated mac
>>> addresses between machines.
>>>
>>> When user have set the mac_prefix in the configuration file, we should use it
>>> in stead of the dynamic generated mac prefix.
>>>
>>> Other change:
>>> . Fix randomly generating mac address so that it correspond to IEEE802.
>>> . Update clone function to decide clone mac address or not.
>>> . Update get_macaddr function.
>>> . Add set_mac_address function.
>>>
>>> New auto mac address pool algorithm:
>>> If address_index is defined, VM will get mac from config file then record mac
>>> in to address_pool. If address_index is not defined, VM will call
>>> get_mac_from_pool to auto create mac then recored mac to address_pool in
>>> following format:
>>> {'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}
>>>
>>> AE:9D:94:6A:9b:f9 : mac address
>>> 20100310-165222-Wt7l : instance attribute of VM
>>> 0 : index of NIC
>>
>> Why do you use the mac address as a key, instead of the instance string
>> + nic index? When the mac address is used as a key, each key has a list
>> of values instead of just one value. This order seems unnatural. If it
>> were the other way around (i.e. key = VM instance + nic index, value =
>> mac address), then each key would have exactly one value, and I think
>> this patch would be shorter and simpler.
>
> One mac address may be used by two VMs, eg. migration.
Sure, that's why I thought the opposite direction would be better: keys
= VMs (nics), values = mac addresses. That way we have one value per
key, instead of a list of values per key.
To clarify, instead of using:
{'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0',
'20100310-165222-Wt7l:1', '20100310-165222-Wt7l:2']}
I suggest:
{'20100310-165222-Wt7l:0': 'AE:9D:94:6A:9b:f9',
'20100310-165222-Wt7l:1': 'AE:9D:94:6A:9b:f9',
'20100310-165222-Wt7l:2': 'AE:9D:94:6A:9b:f9'}
>>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>>> Signed-off-by: Feng Yang <fyang@redhat.com>
>>> Signed-off-by: Amos Kong <akong@redhat.com>
>>> ---
>>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
>>> index fb2d1c2..7c0946e 100644
>>> --- a/client/tests/kvm/kvm_utils.py
>>> +++ b/client/tests/kvm/kvm_utils.py
>>> @@ -5,6 +5,7 @@ KVM test utility functions.
>>> """
>>>
>>> import time, string, random, socket, os, signal, re, logging, commands, cPickle
>>> +import fcntl, shelve
>>> from autotest_lib.client.bin import utils
>>> from autotest_lib.client.common_lib import error, logging_config
>>> import kvm_subprocess
>>> @@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword):
>>>
>>> # Functions related to MAC/IP addresses
>>>
>>> +def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'):
>>
>> The name of this function is confusing because it does the exact
>> opposite: it puts a mac address in address_pool. Maybe the pool you're
>> referring to in the name isn't address_pool, but still a less confusing
>> name should probably be used.
>
> How about allocate_mac(...) ?
> address_pool -> address_container
>
> Allocate mac address and record into address_container.
Yes, something like that, sounds less confusing.
>>> + """
>>> + random generated mac address.
>>> +
>>> + 1) First try to generate macaddress based on the mac address prefix.
>>> + 2) And then try to use total random generated mac address.
>>> +
>>> + @param root_dir: Root dir for kvm
>>> + @param vm: Here we use instance of vm
>>> + @param nic_index: The index of nic.
>>> + @param prefix: Prefix of mac address.
>>> + @Return: Return mac address.
>>> + """
>>> +
>>> + lock_filename = os.path.join(root_dir, "mac_lock")
>>> + lock_file = open(lock_filename, 'w')
>>> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
>>> + mac_filename = os.path.join(root_dir, "address_pool")
>>
>> Maybe it makes sense to put address_pool and the lock file in /tmp,
>> where they can be shared by more than a single autotest instance running
>> on the same host (unlikely, but theoretically possible).
>
> good idea.
>
>>> + mac_shelve = shelve.open(mac_filename, writeback=False)
>>> +
>>> + mac_pool = mac_shelve.get("macpool")
>>
>> Why is this 'macpool' needed? Why not put the keys directly in the
>> shelve object?
>
> yes, put keys directly in the shelve object is better.
>
>>> + if not mac_pool:
>>> + mac_pool = {}
>>> + found = False
>>> +
>>> + val = "%s:%s" % (vm, nic_index)
>>> + for key in mac_pool.keys():
>>> + if val in mac_pool[key]:
>>> + mac_pool[key].append(val)
>>
>> Why append val to mac_pool[key] if val is already in mac_pool[key]?
>
> need drop it.
>
>>> + found = True
>>> + mac = key
>>> +
>>> + while not found:
>>> + postfix = "%02x:%02x" % (random.randint(0x00,0xfe),
>>> + random.randint(0x00,0xfe))
>>> + mac = prefix + postfix
>>> + mac_list = mac.split(":")
>>> + # Clear multicast bit
>>> + mac_list[0] = int(mac_list[0],16) & 0xfe
>>> + # Set local assignment bit (IEEE802)
>>> + mac_list[0] = mac_list[0] | 0x02
>>> + mac_list[0] = "%02x" % mac_list[0]
>>
>> Why is this needed? Most mac addresses begin with 00. If the mac
>> address is generated from the address of eth0 (using the method in this
>> patch) it begins with 00, which is fine. If the prefix is set by the
>> user using mac_prefix, I don't think we should modify it.
>
>
> linux-2.6/include/linux/etherdevice.h
>
> /**
> * random_ether_addr - Generate software assigned random Ethernet address
> * @addr: Pointer to a six-byte array containing the Ethernet address
> *
> * Generate a random Ethernet address (MAC) that is not multicast
> * and has the local assigned bit set.
> */
> static inline void random_ether_addr(u8 *addr)
> {
> get_random_bytes (addr, ETH_ALEN);
> addr [0] &= 0xfe; /* clear multicast bit */
> addr [0] |= 0x02; /* set local assignment bit (IEEE802) */
> }
I saw this googling, but I still don't understand why it's necessary
when real devices start with 00.
>>> + mac = ":".join(mac_list)
>>> + if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]):
>>> + mac_pool[mac] = ["%s:%s" % (vm,nic_index)]
>>> + found = True
>>> + mac_shelve["macpool"] = mac_pool
>>> + logging.debug("generating mac addr %s " % mac)
>>> +
>>> + mac_shelve.close()
>>> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
>>> + lock_file.close()
>>> + return mac
>>> +
>>> +
>>> +def put_mac_to_pool(root_dir, mac, vm):
>>
>> This function removes a mac address from address_pool. I wonder why you
>> chose this name.
>
> address_pool file is a container to record macaddress.
> The 'pool' we get/put mac is an abstract resource pool.
> This is really confused ;)
Yes, I guessed it would be something like that. Still a bit confusing,
I agree, but it's not a big deal to change function names.
>>> + """
>>> + Put the macaddress back to address pool
>>> +
>>> + @param root_dir: Root dir for kvm
>>> + @param vm: Here we use instance attribute of vm
>>> + @param mac: mac address will be put.
>>> + @Return: mac address.
>>> + """
>>> +
>>> + lock_filename = os.path.join(root_dir, "mac_lock")
>>> + lock_file = open(lock_filename, 'w')
>>> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
>>> + mac_filename = os.path.join(root_dir, "address_pool")
>>> + mac_shelve = shelve.open(mac_filename, writeback=False)
>>> +
>>> + mac_pool = mac_shelve.get("macpool")
>>> +
>>> + if not mac_pool or (not mac in mac_pool):
>>> + logging.debug("Try to free a macaddress does no in pool")
>>> + logging.debug("macaddress is %s" % mac)
>>> + logging.debug("pool is %s" % mac_pool)
>>> + else:
>>> + if len(mac_pool[mac]) <= 1:
>>> + mac_pool.pop(mac)
>>> + else:
>>> + for value in mac_pool[mac]:
>>> + if vm in value:
>>> + mac_pool[mac].remove(value)
>>> + break
>>> + if len(mac_pool[mac]) == 0:
>>> + mac_pool.pop(mac)
>>> +
>>> + mac_shelve["macpool"] = mac_pool
>>> + logging.debug("freeing mac addr %s " % mac)
>>> +
>>> + mac_shelve.close()
>>> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
>>> + lock_file.close()
>>> + return mac
>>> +
>>> +
>>> def mac_str_to_int(addr):
>>> """
>>> Convert MAC address string to integer.
>>> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
>>> index 6cd0688..a9ba6e7 100755
>>> --- a/client/tests/kvm/kvm_vm.py
>>> +++ b/client/tests/kvm/kvm_vm.py
>>> @@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu.
>>> @copyright: 2008-2009 Red Hat Inc.
>>> """
>>>
>>> -import time, socket, os, logging, fcntl, re, commands, glob
>>> +import time, socket, os, logging, fcntl, re, commands, shelve, glob
>>> import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer
>>> from autotest_lib.client.common_lib import error
>>> from autotest_lib.client.bin import utils
>>> @@ -117,6 +117,7 @@ class VM:
>>> self.params = params
>>> self.root_dir = root_dir
>>> self.address_cache = address_cache
>>> + self.mac_prefix = params.get('mac_prefix')
>>> self.netdev_id = []
>>>
>>> # Find a unique identifier for this VM
>>> @@ -126,8 +127,15 @@ class VM:
>>> if not glob.glob("/tmp/*%s" % self.instance):
>>> break
>>>
>>> + if self.mac_prefix is None:
>>> + s, o = commands.getstatusoutput("ifconfig eth0")
>>> + if s == 0:
>>> + mac = re.findall("HWaddr (\S*)", o)[0]
>>> + self.mac_prefix = '00' + mac[8:] + ':'
>>> +
>>>
>>> - def clone(self, name=None, params=None, root_dir=None, address_cache=None):
>>> + def clone(self, name=None, params=None, root_dir=None,
>>> + address_cache=None, mac_clone=True):
>>> """
>>> Return a clone of the VM object with optionally modified parameters.
>>> The clone is initially not alive and needs to be started using create().
>>> @@ -138,6 +146,7 @@ class VM:
>>> @param params: Optional new VM creation parameters
>>> @param root_dir: Optional new base directory for relative filenames
>>> @param address_cache: A dict that maps MAC addresses to IP addresses
>>> + @param mac_clone: Clone mac address or not.
>>> """
>>> if name is None:
>>> name = self.name
>>> @@ -147,9 +156,19 @@ class VM:
>>> root_dir = self.root_dir
>>> if address_cache is None:
>>> address_cache = self.address_cache
>>> - return VM(name, params, root_dir, address_cache)
>>> + vm = VM(name, params, root_dir, address_cache)
>>> + if mac_clone:
>>> + # Clone mac address by coping 'self.instance' to the new vm.
>>> + vm.instance = self.instance
>>
>> Copying self.instance isn't a good idea because the monitor, serial
>> console and testlog filenames use self.instance. self.instance should
>> be unique. If we still want to use the same mac address for 2 VMs, for
>> example in migration, a different solution should be found.
>
> We almost only clone mac in migration test, the src guest would be killed.
> If the instance of dest guest is same as src guest, the serial connection would not break,
> it would continually write log to original log file.
What about the monitor(s)? The fact that the serial console connection
doesn't break doesn't necessarily mean everything's OK.
>>> + return vm
>>>
>>>
>>> + def free_mac_address(self):
>>> + nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics"))
>>> + for nic in range(nic_num):
>>> + mac = self.get_macaddr(nic_index=nic)
>>> + kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance)
>>> +
>>> def make_qemu_command(self, name=None, params=None, root_dir=None):
>>> """
>>> Generate a qemu command line. All parameters are optional. If a
>>> @@ -383,6 +402,13 @@ class VM:
>>> mac = None
>>> if "address_index" in nic_params:
>>> mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0]
>>> + self.set_mac_address(mac=mac, nic_index=vlan)
>>> + else:
>>> + mac = kvm_utils.get_mac_from_pool(self.root_dir,
>>> + vm=self.instance,
>>> + nic_index=vlan,
>>> + prefix=self.mac_prefix)
>>> +
>>> qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac,
>>> self.netdev_id[vlan])
>>> # Handle the '-net tap' or '-net user' part
>>> @@ -396,7 +422,7 @@ class VM:
>>> if tftp:
>>> tftp = kvm_utils.get_path(root_dir, tftp)
>>> qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"),
>>> - nic_params.get("nic_ifname"),
>>> + self.get_ifname(vlan),
>>
>> Why is get_ifname() useful here?
>
> dynamically generate the ifname is better.
But why? Is this ifname useful for any test?
>>> script, downscript, tftp,
>>> nic_params.get("bootp"), redirs,
>>> self.netdev_id[vlan])
>>> @@ -720,7 +746,7 @@ class VM:
>>> lockfile.close()
>>>
>>>
>>> - def destroy(self, gracefully=True):
>>> + def destroy(self, gracefully=True, free_macaddr=True):
>>> """
>>> Destroy the VM.
>>>
>>> @@ -731,6 +757,7 @@ class VM:
>>> @param gracefully: Whether an attempt will be made to end the VM
>>> using a shell command before trying to end the qemu process
>>> with a 'quit' or a kill signal.
>>> + @param free_macaddr: Whether free macaddresses when destory a vm.
>>> """
>>> try:
>>> # Is it already dead?
>>> @@ -751,11 +778,18 @@ class VM:
>>> logging.debug("Shutdown command sent; waiting for VM "
>>> "to go down...")
>>> if kvm_utils.wait_for(self.is_dead, 60, 1, 1):
>>> - logging.debug("VM is down")
>>> + logging.debug("VM is down, free mac address.")
>>> + # free mac address
>>> + if free_macaddr:
>>> + self.free_mac_address()
>>> return
>>> finally:
>>> session.close()
>>>
>>> + # free mac address
>>> + if free_macaddr:
>>> + self.free_mac_address()
>>> +
>>> if self.monitor:
>>> # Try to destroy with a monitor command
>>> logging.debug("Trying to kill VM with monitor command...")
>>> @@ -881,10 +915,14 @@ class VM:
>>> nic_name = nics[index]
>>> nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
>>> if nic_params.get("nic_mode") == "tap":
>>> - mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params)
>>> + mac = self.get_macaddr(index)
>>> if not mac:
>>> logging.debug("MAC address unavailable")
>>> return None
>>> + else:
>>> + mac = mac.lower()
>>> + ip = None
>>> +
>>> if not ip or nic_params.get("always_use_tcpdump") == "yes":
>>> # Get the IP address from the cache
>>> ip = self.address_cache.get(mac)
>>> @@ -897,6 +935,7 @@ class VM:
>>> for nic in nics]
>>> macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0]
>>> for dict in nic_dicts]
>>> + macs.append(mac)
>>> if not kvm_utils.verify_ip_address_ownership(ip, macs):
>>> logging.debug("Could not verify MAC-IP address mapping: "
>>> "%s ---> %s" % (mac, ip))
>>> @@ -925,6 +964,71 @@ class VM:
>>> "redirected" % port)
>>> return self.redirs.get(port)
>>>
>>> + def get_ifname(self, nic_index=0):
>>> + """
>>> + Return the ifname of tap device for the guest nic.
>>> +
>>> + @param nic_index: Index of the NIC
>>> + """
>>> +
>>> + nics = kvm_utils.get_sub_dict_names(self.params, "nics")
>>> + nic_name = nics[nic_index]
>>> + nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
>>> + if nic_params.get("nic_ifname"):
>>> + return nic_params.get("nic_ifname")
>>> + else:
>>> + return "%s_%s_%s" % (nic_params.get("nic_model"),
>>> + nic_index, self.vnc_port)
>>
>> What's the purpose of this string?
>
> Just avoid repeated ifname. The vnc_port is unique for each VM, nic_index is unique for each nic of one VM.
self.instance should also be unique, though it's quite long.
>>> + def get_macaddr(self, nic_index=0):
>>> + """
>>> + Return the macaddr of guest nic.
>>> +
>>> + @param nic_index: Index of the NIC
>>> + """
>>> + mac_filename = os.path.join(self.root_dir, "address_pool")
>>> + mac_shelve = shelve.open(mac_filename, writeback=False)
>>> + mac_pool = mac_shelve.get("macpool")
>>> + val = "%s:%s" % (self.instance, nic_index)
>>> + for key in mac_pool.keys():
>>> + if val in mac_pool[key]:
>>> + return key
>>> + return None
>>> +
>>> + def set_mac_address(self, mac, nic_index=0, shareable=False):
>>> + """
>>> + Set mac address for guest. Note: It just update address pool.
>>> +
>>> + @param mac: address will set to guest
>>> + @param nic_index: Index of the NIC
>>> + @param shareable: Where VM can share mac with other VM or not.
>>> + """
>>> + lock_filename = os.path.join(self.root_dir, "mac_lock")
>>> + lock_file = open(lock_filename, 'w')
>>> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
>>> + mac_filename = os.path.join(self.root_dir, "address_pool")
>>> + mac_shelve = shelve.open(mac_filename, writeback=False)
>>> + mac_pool = mac_shelve.get("macpool")
>>> + if not mac_pool:
>>> + mac_pool = {}
>>> + value = "%s:%s" % (self.instance, nic_index)
>>> + if mac not in mac_pool.keys():
>>> + for key in mac_pool.keys():
>>> + if value in mac_pool[key]:
>>> + mac_pool[key].remove(value)
>>> + if len(mac_pool[key]) == 0:
>>> + mac_pool.pop(key)
>>> + mac_pool[mac] = [value]
>>> + else:
>>> + if shareable:
>>> + mac_pool[mac].append(value)
>>> + else:
>>> + logging.error("Mac address already be used!")
>>> + return False
>>> + mac_shelve["macpool"] = mac_pool
>>> + mac_shelve.close()
>>> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
>>> + lock_file.close()
>>>
>>> def get_pid(self):
>>> """
>>> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
>>> index fd9e72f..6710c00 100644
>>> --- a/client/tests/kvm/tests_base.cfg.sample
>>> +++ b/client/tests/kvm/tests_base.cfg.sample
>>> @@ -51,7 +51,7 @@ guest_port_remote_shell = 22
>>> nic_mode = user
>>> #nic_mode = tap
>>> nic_script = scripts/qemu-ifup
>>> -address_index = 0
>>> +#address_index = 0
>>> run_tcpdump = yes
>>>
>>> # Misc
>>>
>>>
>>
>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
@ 2010-07-20 15:53 ` Michael Goldish
0 siblings, 0 replies; 77+ messages in thread
From: Michael Goldish @ 2010-07-20 15:53 UTC (permalink / raw)
To: Amos Kong; +Cc: lmr, autotest, qemu-devel, kvm
On 07/20/2010 04:44 PM, Amos Kong wrote:
> On Tue, Jul 20, 2010 at 01:19:39PM +0300, Michael Goldish wrote:
>>
>
> Michael,
>
> Thanks for your comments. Let's simplify this method together.
>
>> On 07/20/2010 04:34 AM, Amos Kong wrote:
>>> Old method uses the mac address in the configuration files which could
>>> lead serious problem when multiple tests running in different hosts.
>>>
>>> This patch adds a new macaddress pool algorithm, it generates the mac prefix
>>> based on mac address of the host which could eliminate the duplicated mac
>>> addresses between machines.
>>>
>>> When user have set the mac_prefix in the configuration file, we should use it
>>> in stead of the dynamic generated mac prefix.
>>>
>>> Other change:
>>> . Fix randomly generating mac address so that it correspond to IEEE802.
>>> . Update clone function to decide clone mac address or not.
>>> . Update get_macaddr function.
>>> . Add set_mac_address function.
>>>
>>> New auto mac address pool algorithm:
>>> If address_index is defined, VM will get mac from config file then record mac
>>> in to address_pool. If address_index is not defined, VM will call
>>> get_mac_from_pool to auto create mac then recored mac to address_pool in
>>> following format:
>>> {'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}
>>>
>>> AE:9D:94:6A:9b:f9 : mac address
>>> 20100310-165222-Wt7l : instance attribute of VM
>>> 0 : index of NIC
>>
>> Why do you use the mac address as a key, instead of the instance string
>> + nic index? When the mac address is used as a key, each key has a list
>> of values instead of just one value. This order seems unnatural. If it
>> were the other way around (i.e. key = VM instance + nic index, value =
>> mac address), then each key would have exactly one value, and I think
>> this patch would be shorter and simpler.
>
> One mac address may be used by two VMs, eg. migration.
Sure, that's why I thought the opposite direction would be better: keys
= VMs (nics), values = mac addresses. That way we have one value per
key, instead of a list of values per key.
To clarify, instead of using:
{'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0',
'20100310-165222-Wt7l:1', '20100310-165222-Wt7l:2']}
I suggest:
{'20100310-165222-Wt7l:0': 'AE:9D:94:6A:9b:f9',
'20100310-165222-Wt7l:1': 'AE:9D:94:6A:9b:f9',
'20100310-165222-Wt7l:2': 'AE:9D:94:6A:9b:f9'}
>>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>>> Signed-off-by: Feng Yang <fyang@redhat.com>
>>> Signed-off-by: Amos Kong <akong@redhat.com>
>>> ---
>>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
>>> index fb2d1c2..7c0946e 100644
>>> --- a/client/tests/kvm/kvm_utils.py
>>> +++ b/client/tests/kvm/kvm_utils.py
>>> @@ -5,6 +5,7 @@ KVM test utility functions.
>>> """
>>>
>>> import time, string, random, socket, os, signal, re, logging, commands, cPickle
>>> +import fcntl, shelve
>>> from autotest_lib.client.bin import utils
>>> from autotest_lib.client.common_lib import error, logging_config
>>> import kvm_subprocess
>>> @@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword):
>>>
>>> # Functions related to MAC/IP addresses
>>>
>>> +def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'):
>>
>> The name of this function is confusing because it does the exact
>> opposite: it puts a mac address in address_pool. Maybe the pool you're
>> referring to in the name isn't address_pool, but still a less confusing
>> name should probably be used.
>
> How about allocate_mac(...) ?
> address_pool -> address_container
>
> Allocate mac address and record into address_container.
Yes, something like that, sounds less confusing.
>>> + """
>>> + random generated mac address.
>>> +
>>> + 1) First try to generate macaddress based on the mac address prefix.
>>> + 2) And then try to use total random generated mac address.
>>> +
>>> + @param root_dir: Root dir for kvm
>>> + @param vm: Here we use instance of vm
>>> + @param nic_index: The index of nic.
>>> + @param prefix: Prefix of mac address.
>>> + @Return: Return mac address.
>>> + """
>>> +
>>> + lock_filename = os.path.join(root_dir, "mac_lock")
>>> + lock_file = open(lock_filename, 'w')
>>> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
>>> + mac_filename = os.path.join(root_dir, "address_pool")
>>
>> Maybe it makes sense to put address_pool and the lock file in /tmp,
>> where they can be shared by more than a single autotest instance running
>> on the same host (unlikely, but theoretically possible).
>
> good idea.
>
>>> + mac_shelve = shelve.open(mac_filename, writeback=False)
>>> +
>>> + mac_pool = mac_shelve.get("macpool")
>>
>> Why is this 'macpool' needed? Why not put the keys directly in the
>> shelve object?
>
> yes, put keys directly in the shelve object is better.
>
>>> + if not mac_pool:
>>> + mac_pool = {}
>>> + found = False
>>> +
>>> + val = "%s:%s" % (vm, nic_index)
>>> + for key in mac_pool.keys():
>>> + if val in mac_pool[key]:
>>> + mac_pool[key].append(val)
>>
>> Why append val to mac_pool[key] if val is already in mac_pool[key]?
>
> need drop it.
>
>>> + found = True
>>> + mac = key
>>> +
>>> + while not found:
>>> + postfix = "%02x:%02x" % (random.randint(0x00,0xfe),
>>> + random.randint(0x00,0xfe))
>>> + mac = prefix + postfix
>>> + mac_list = mac.split(":")
>>> + # Clear multicast bit
>>> + mac_list[0] = int(mac_list[0],16) & 0xfe
>>> + # Set local assignment bit (IEEE802)
>>> + mac_list[0] = mac_list[0] | 0x02
>>> + mac_list[0] = "%02x" % mac_list[0]
>>
>> Why is this needed? Most mac addresses begin with 00. If the mac
>> address is generated from the address of eth0 (using the method in this
>> patch) it begins with 00, which is fine. If the prefix is set by the
>> user using mac_prefix, I don't think we should modify it.
>
>
> linux-2.6/include/linux/etherdevice.h
>
> /**
> * random_ether_addr - Generate software assigned random Ethernet address
> * @addr: Pointer to a six-byte array containing the Ethernet address
> *
> * Generate a random Ethernet address (MAC) that is not multicast
> * and has the local assigned bit set.
> */
> static inline void random_ether_addr(u8 *addr)
> {
> get_random_bytes (addr, ETH_ALEN);
> addr [0] &= 0xfe; /* clear multicast bit */
> addr [0] |= 0x02; /* set local assignment bit (IEEE802) */
> }
I saw this googling, but I still don't understand why it's necessary
when real devices start with 00.
>>> + mac = ":".join(mac_list)
>>> + if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]):
>>> + mac_pool[mac] = ["%s:%s" % (vm,nic_index)]
>>> + found = True
>>> + mac_shelve["macpool"] = mac_pool
>>> + logging.debug("generating mac addr %s " % mac)
>>> +
>>> + mac_shelve.close()
>>> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
>>> + lock_file.close()
>>> + return mac
>>> +
>>> +
>>> +def put_mac_to_pool(root_dir, mac, vm):
>>
>> This function removes a mac address from address_pool. I wonder why you
>> chose this name.
>
> address_pool file is a container to record macaddress.
> The 'pool' we get/put mac is an abstract resource pool.
> This is really confused ;)
Yes, I guessed it would be something like that. Still a bit confusing,
I agree, but it's not a big deal to change function names.
>>> + """
>>> + Put the macaddress back to address pool
>>> +
>>> + @param root_dir: Root dir for kvm
>>> + @param vm: Here we use instance attribute of vm
>>> + @param mac: mac address will be put.
>>> + @Return: mac address.
>>> + """
>>> +
>>> + lock_filename = os.path.join(root_dir, "mac_lock")
>>> + lock_file = open(lock_filename, 'w')
>>> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
>>> + mac_filename = os.path.join(root_dir, "address_pool")
>>> + mac_shelve = shelve.open(mac_filename, writeback=False)
>>> +
>>> + mac_pool = mac_shelve.get("macpool")
>>> +
>>> + if not mac_pool or (not mac in mac_pool):
>>> + logging.debug("Try to free a macaddress does no in pool")
>>> + logging.debug("macaddress is %s" % mac)
>>> + logging.debug("pool is %s" % mac_pool)
>>> + else:
>>> + if len(mac_pool[mac]) <= 1:
>>> + mac_pool.pop(mac)
>>> + else:
>>> + for value in mac_pool[mac]:
>>> + if vm in value:
>>> + mac_pool[mac].remove(value)
>>> + break
>>> + if len(mac_pool[mac]) == 0:
>>> + mac_pool.pop(mac)
>>> +
>>> + mac_shelve["macpool"] = mac_pool
>>> + logging.debug("freeing mac addr %s " % mac)
>>> +
>>> + mac_shelve.close()
>>> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
>>> + lock_file.close()
>>> + return mac
>>> +
>>> +
>>> def mac_str_to_int(addr):
>>> """
>>> Convert MAC address string to integer.
>>> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
>>> index 6cd0688..a9ba6e7 100755
>>> --- a/client/tests/kvm/kvm_vm.py
>>> +++ b/client/tests/kvm/kvm_vm.py
>>> @@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu.
>>> @copyright: 2008-2009 Red Hat Inc.
>>> """
>>>
>>> -import time, socket, os, logging, fcntl, re, commands, glob
>>> +import time, socket, os, logging, fcntl, re, commands, shelve, glob
>>> import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer
>>> from autotest_lib.client.common_lib import error
>>> from autotest_lib.client.bin import utils
>>> @@ -117,6 +117,7 @@ class VM:
>>> self.params = params
>>> self.root_dir = root_dir
>>> self.address_cache = address_cache
>>> + self.mac_prefix = params.get('mac_prefix')
>>> self.netdev_id = []
>>>
>>> # Find a unique identifier for this VM
>>> @@ -126,8 +127,15 @@ class VM:
>>> if not glob.glob("/tmp/*%s" % self.instance):
>>> break
>>>
>>> + if self.mac_prefix is None:
>>> + s, o = commands.getstatusoutput("ifconfig eth0")
>>> + if s == 0:
>>> + mac = re.findall("HWaddr (\S*)", o)[0]
>>> + self.mac_prefix = '00' + mac[8:] + ':'
>>> +
>>>
>>> - def clone(self, name=None, params=None, root_dir=None, address_cache=None):
>>> + def clone(self, name=None, params=None, root_dir=None,
>>> + address_cache=None, mac_clone=True):
>>> """
>>> Return a clone of the VM object with optionally modified parameters.
>>> The clone is initially not alive and needs to be started using create().
>>> @@ -138,6 +146,7 @@ class VM:
>>> @param params: Optional new VM creation parameters
>>> @param root_dir: Optional new base directory for relative filenames
>>> @param address_cache: A dict that maps MAC addresses to IP addresses
>>> + @param mac_clone: Clone mac address or not.
>>> """
>>> if name is None:
>>> name = self.name
>>> @@ -147,9 +156,19 @@ class VM:
>>> root_dir = self.root_dir
>>> if address_cache is None:
>>> address_cache = self.address_cache
>>> - return VM(name, params, root_dir, address_cache)
>>> + vm = VM(name, params, root_dir, address_cache)
>>> + if mac_clone:
>>> + # Clone mac address by coping 'self.instance' to the new vm.
>>> + vm.instance = self.instance
>>
>> Copying self.instance isn't a good idea because the monitor, serial
>> console and testlog filenames use self.instance. self.instance should
>> be unique. If we still want to use the same mac address for 2 VMs, for
>> example in migration, a different solution should be found.
>
> We almost only clone mac in migration test, the src guest would be killed.
> If the instance of dest guest is same as src guest, the serial connection would not break,
> it would continually write log to original log file.
What about the monitor(s)? The fact that the serial console connection
doesn't break doesn't necessarily mean everything's OK.
>>> + return vm
>>>
>>>
>>> + def free_mac_address(self):
>>> + nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics"))
>>> + for nic in range(nic_num):
>>> + mac = self.get_macaddr(nic_index=nic)
>>> + kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance)
>>> +
>>> def make_qemu_command(self, name=None, params=None, root_dir=None):
>>> """
>>> Generate a qemu command line. All parameters are optional. If a
>>> @@ -383,6 +402,13 @@ class VM:
>>> mac = None
>>> if "address_index" in nic_params:
>>> mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0]
>>> + self.set_mac_address(mac=mac, nic_index=vlan)
>>> + else:
>>> + mac = kvm_utils.get_mac_from_pool(self.root_dir,
>>> + vm=self.instance,
>>> + nic_index=vlan,
>>> + prefix=self.mac_prefix)
>>> +
>>> qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac,
>>> self.netdev_id[vlan])
>>> # Handle the '-net tap' or '-net user' part
>>> @@ -396,7 +422,7 @@ class VM:
>>> if tftp:
>>> tftp = kvm_utils.get_path(root_dir, tftp)
>>> qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"),
>>> - nic_params.get("nic_ifname"),
>>> + self.get_ifname(vlan),
>>
>> Why is get_ifname() useful here?
>
> dynamically generate the ifname is better.
But why? Is this ifname useful for any test?
>>> script, downscript, tftp,
>>> nic_params.get("bootp"), redirs,
>>> self.netdev_id[vlan])
>>> @@ -720,7 +746,7 @@ class VM:
>>> lockfile.close()
>>>
>>>
>>> - def destroy(self, gracefully=True):
>>> + def destroy(self, gracefully=True, free_macaddr=True):
>>> """
>>> Destroy the VM.
>>>
>>> @@ -731,6 +757,7 @@ class VM:
>>> @param gracefully: Whether an attempt will be made to end the VM
>>> using a shell command before trying to end the qemu process
>>> with a 'quit' or a kill signal.
>>> + @param free_macaddr: Whether free macaddresses when destory a vm.
>>> """
>>> try:
>>> # Is it already dead?
>>> @@ -751,11 +778,18 @@ class VM:
>>> logging.debug("Shutdown command sent; waiting for VM "
>>> "to go down...")
>>> if kvm_utils.wait_for(self.is_dead, 60, 1, 1):
>>> - logging.debug("VM is down")
>>> + logging.debug("VM is down, free mac address.")
>>> + # free mac address
>>> + if free_macaddr:
>>> + self.free_mac_address()
>>> return
>>> finally:
>>> session.close()
>>>
>>> + # free mac address
>>> + if free_macaddr:
>>> + self.free_mac_address()
>>> +
>>> if self.monitor:
>>> # Try to destroy with a monitor command
>>> logging.debug("Trying to kill VM with monitor command...")
>>> @@ -881,10 +915,14 @@ class VM:
>>> nic_name = nics[index]
>>> nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
>>> if nic_params.get("nic_mode") == "tap":
>>> - mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params)
>>> + mac = self.get_macaddr(index)
>>> if not mac:
>>> logging.debug("MAC address unavailable")
>>> return None
>>> + else:
>>> + mac = mac.lower()
>>> + ip = None
>>> +
>>> if not ip or nic_params.get("always_use_tcpdump") == "yes":
>>> # Get the IP address from the cache
>>> ip = self.address_cache.get(mac)
>>> @@ -897,6 +935,7 @@ class VM:
>>> for nic in nics]
>>> macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0]
>>> for dict in nic_dicts]
>>> + macs.append(mac)
>>> if not kvm_utils.verify_ip_address_ownership(ip, macs):
>>> logging.debug("Could not verify MAC-IP address mapping: "
>>> "%s ---> %s" % (mac, ip))
>>> @@ -925,6 +964,71 @@ class VM:
>>> "redirected" % port)
>>> return self.redirs.get(port)
>>>
>>> + def get_ifname(self, nic_index=0):
>>> + """
>>> + Return the ifname of tap device for the guest nic.
>>> +
>>> + @param nic_index: Index of the NIC
>>> + """
>>> +
>>> + nics = kvm_utils.get_sub_dict_names(self.params, "nics")
>>> + nic_name = nics[nic_index]
>>> + nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
>>> + if nic_params.get("nic_ifname"):
>>> + return nic_params.get("nic_ifname")
>>> + else:
>>> + return "%s_%s_%s" % (nic_params.get("nic_model"),
>>> + nic_index, self.vnc_port)
>>
>> What's the purpose of this string?
>
> Just avoid repeated ifname. The vnc_port is unique for each VM, nic_index is unique for each nic of one VM.
self.instance should also be unique, though it's quite long.
>>> + def get_macaddr(self, nic_index=0):
>>> + """
>>> + Return the macaddr of guest nic.
>>> +
>>> + @param nic_index: Index of the NIC
>>> + """
>>> + mac_filename = os.path.join(self.root_dir, "address_pool")
>>> + mac_shelve = shelve.open(mac_filename, writeback=False)
>>> + mac_pool = mac_shelve.get("macpool")
>>> + val = "%s:%s" % (self.instance, nic_index)
>>> + for key in mac_pool.keys():
>>> + if val in mac_pool[key]:
>>> + return key
>>> + return None
>>> +
>>> + def set_mac_address(self, mac, nic_index=0, shareable=False):
>>> + """
>>> + Set mac address for guest. Note: It just update address pool.
>>> +
>>> + @param mac: address will set to guest
>>> + @param nic_index: Index of the NIC
>>> + @param shareable: Where VM can share mac with other VM or not.
>>> + """
>>> + lock_filename = os.path.join(self.root_dir, "mac_lock")
>>> + lock_file = open(lock_filename, 'w')
>>> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
>>> + mac_filename = os.path.join(self.root_dir, "address_pool")
>>> + mac_shelve = shelve.open(mac_filename, writeback=False)
>>> + mac_pool = mac_shelve.get("macpool")
>>> + if not mac_pool:
>>> + mac_pool = {}
>>> + value = "%s:%s" % (self.instance, nic_index)
>>> + if mac not in mac_pool.keys():
>>> + for key in mac_pool.keys():
>>> + if value in mac_pool[key]:
>>> + mac_pool[key].remove(value)
>>> + if len(mac_pool[key]) == 0:
>>> + mac_pool.pop(key)
>>> + mac_pool[mac] = [value]
>>> + else:
>>> + if shareable:
>>> + mac_pool[mac].append(value)
>>> + else:
>>> + logging.error("Mac address already be used!")
>>> + return False
>>> + mac_shelve["macpool"] = mac_pool
>>> + mac_shelve.close()
>>> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
>>> + lock_file.close()
>>>
>>> def get_pid(self):
>>> """
>>> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
>>> index fd9e72f..6710c00 100644
>>> --- a/client/tests/kvm/tests_base.cfg.sample
>>> +++ b/client/tests/kvm/tests_base.cfg.sample
>>> @@ -51,7 +51,7 @@ guest_port_remote_shell = 22
>>> nic_mode = user
>>> #nic_mode = tap
>>> nic_script = scripts/qemu-ifup
>>> -address_index = 0
>>> +#address_index = 0
>>> run_tcpdump = yes
>>>
>>> # Misc
>>>
>>>
>>
>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
2010-07-20 15:53 ` Michael Goldish
(?)
@ 2010-08-03 1:34 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-03 1:34 UTC (permalink / raw)
To: Michael Goldish; +Cc: lmr, autotest, qemu-devel, kvm
On Tue, Jul 20, 2010 at 06:53:27PM +0300, Michael Goldish wrote:
> On 07/20/2010 04:44 PM, Amos Kong wrote:
> > On Tue, Jul 20, 2010 at 01:19:39PM +0300, Michael Goldish wrote:
> >>
> >
> > Michael,
> >
> > Thanks for your comments. Let's simplify this method together.
I would produce a v2 later basing on our discussion.
> >>> + def get_ifname(self, nic_index=0):
> >>> + """
> >>> + Return the ifname of tap device for the guest nic.
> >>> +
> >>> + @param nic_index: Index of the NIC
> >>> + """
> >>> +
> >>> + nics = kvm_utils.get_sub_dict_names(self.params, "nics")
> >>> + nic_name = nics[nic_index]
> >>> + nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
> >>> + if nic_params.get("nic_ifname"):
> >>> + return nic_params.get("nic_ifname")
> >>> + else:
> >>> + return "%s_%s_%s" % (nic_params.get("nic_model"),
> >>> + nic_index, self.vnc_port)
> >>
> >> What's the purpose of this string?
> >
> > Just avoid repeated ifname. The vnc_port is unique for each VM, nic_index is unique for each nic of one VM.
>
> self.instance should also be unique, though it's quite long.
qemu can only receive 15 chars as ifname, instance is too long.
if the prefix of two instance are same, the actual ifnames will not be unique.
linux-user/syscall.c:
#define IFNAMSIZ 16
net/tap-linux.c:
int tap_open(...
...
if (ifname[0] != '\0')
pstrcpy(ifr.ifr_name, IFNAMSIZ, ifname);
else
pstrcpy(ifr.ifr_name, IFNAMSIZ, "tap%d");
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
2010-07-20 1:34 ` [Qemu-devel] " Amos Kong
@ 2010-07-27 1:48 ` Lucas Meneghel Rodrigues
-1 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-27 1:48 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:34 +0800, Amos Kong wrote:
> Old method uses the mac address in the configuration files which could
> lead serious problem when multiple tests running in different hosts.
>
> This patch adds a new macaddress pool algorithm, it generates the mac prefix
> based on mac address of the host which could eliminate the duplicated mac
> addresses between machines.
>
> When user have set the mac_prefix in the configuration file, we should use it
> in stead of the dynamic generated mac prefix.
I have some additional comments besides the good points made by Michael
on his review.
> Other change:
> . Fix randomly generating mac address so that it correspond to IEEE802.
> . Update clone function to decide clone mac address or not.
> . Update get_macaddr function.
> . Add set_mac_address function.
>
> New auto mac address pool algorithm:
> If address_index is defined, VM will get mac from config file then record mac
> in to address_pool. If address_index is not defined, VM will call
> get_mac_from_pool to auto create mac then recored mac to address_pool in
> following format:
> {'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}
>
> AE:9D:94:6A:9b:f9 : mac address
> 20100310-165222-Wt7l : instance attribute of VM
> 0 : index of NIC
>
>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Feng Yang <fyang@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
> index fb2d1c2..7c0946e 100644
> --- a/client/tests/kvm/kvm_utils.py
> +++ b/client/tests/kvm/kvm_utils.py
> @@ -5,6 +5,7 @@ KVM test utility functions.
> """
>
> import time, string, random, socket, os, signal, re, logging, commands, cPickle
> +import fcntl, shelve
> from autotest_lib.client.bin import utils
> from autotest_lib.client.common_lib import error, logging_config
> import kvm_subprocess
> @@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword):
>
> # Functions related to MAC/IP addresses
>
> +def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'):
> + """
> + random generated mac address.
I think I got your original idea of naming this function
get_mac_from_pool (the prefix assessed is the base of the 'pool'), but
as Michael pointed out, a new MAC is being generated here and added to
the mac pool, so a more appropriate name should be used. A
straightforward suggestion would be generate_mac_address. Also, the
description of the function would be more on the lines of "Generate a
MAC address and add it to the MAC pool".
> + 1) First try to generate macaddress based on the mac address prefix.
> + 2) And then try to use total random generated mac address.
> +
> + @param root_dir: Root dir for kvm
> + @param vm: Here we use instance of vm
> + @param nic_index: The index of nic.
> + @param prefix: Prefix of mac address.
> + @Return: Return mac address.
> + """
> +
> + lock_filename = os.path.join(root_dir, "mac_lock")
> + lock_file = open(lock_filename, 'w')
> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> + mac_filename = os.path.join(root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
^ Indeed we should keep the mac pool under /tmp as Michael already
pointed out.
> + mac_pool = mac_shelve.get("macpool")
> +
> + if not mac_pool:
> + mac_pool = {}
> + found = False
> +
> + val = "%s:%s" % (vm, nic_index)
> + for key in mac_pool.keys():
> + if val in mac_pool[key]:
> + mac_pool[key].append(val)
> + found = True
> + mac = key
> +
> + while not found:
> + postfix = "%02x:%02x" % (random.randint(0x00,0xfe),
> + random.randint(0x00,0xfe))
> + mac = prefix + postfix
^ It's more natural to use 'sufix' instead of 'postfix' here.
> + mac_list = mac.split(":")
> + # Clear multicast bit
> + mac_list[0] = int(mac_list[0],16) & 0xfe
> + # Set local assignment bit (IEEE802)
> + mac_list[0] = mac_list[0] | 0x02
> + mac_list[0] = "%02x" % mac_list[0]
> + mac = ":".join(mac_list)
> + if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]):
> + mac_pool[mac] = ["%s:%s" % (vm,nic_index)]
> + found = True
> + mac_shelve["macpool"] = mac_pool
> + logging.debug("generating mac addr %s " % mac)
^ Here the MAC was already generated, so here you could use something
like "Generated MAC address %s"
> + mac_shelve.close()
> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> + lock_file.close()
> + return mac
> +
> +
> +def put_mac_to_pool(root_dir, mac, vm):
> + """
> + Put the macaddress back to address pool
> +
> + @param root_dir: Root dir for kvm
> + @param vm: Here we use instance attribute of vm
^ @param vm: Instance attribute of a vm. Also, we might want to rename
this parameter to vm_instance, to make things more clear and avoid
confusions (vm is referred in several places as a vm instance).
> + @param mac: mac address will be put.
> + @Return: mac address.
> + """
> +
> + lock_filename = os.path.join(root_dir, "mac_lock")
> + lock_file = open(lock_filename, 'w')
> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> + mac_filename = os.path.join(root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> +
> + mac_pool = mac_shelve.get("macpool")
> +
> + if not mac_pool or (not mac in mac_pool):
> + logging.debug("Try to free a macaddress does no in pool")
^ logging.debug("MAC not present in the MAC pool, not modifying pool")
> + logging.debug("macaddress is %s" % mac)
> + logging.debug("pool is %s" % mac_pool)
> + else:
> + if len(mac_pool[mac]) <= 1:
> + mac_pool.pop(mac)
> + else:
> + for value in mac_pool[mac]:
> + if vm in value:
> + mac_pool[mac].remove(value)
> + break
> + if len(mac_pool[mac]) == 0:
> + mac_pool.pop(mac)
> +
> + mac_shelve["macpool"] = mac_pool
> + logging.debug("freeing mac addr %s " % mac)
^ logging.debug("Freeing mac addr %s " % mac). Also, this should go to
the beginning of the function, before the "MAC not present in the..."
message.
> +
> + mac_shelve.close()
> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> + lock_file.close()
> + return mac
> +
> +
> def mac_str_to_int(addr):
> """
> Convert MAC address string to integer.
> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
> index 6cd0688..a9ba6e7 100755
> --- a/client/tests/kvm/kvm_vm.py
> +++ b/client/tests/kvm/kvm_vm.py
> @@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu.
> @copyright: 2008-2009 Red Hat Inc.
> """
>
> -import time, socket, os, logging, fcntl, re, commands, glob
> +import time, socket, os, logging, fcntl, re, commands, shelve, glob
> import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer
> from autotest_lib.client.common_lib import error
> from autotest_lib.client.bin import utils
> @@ -117,6 +117,7 @@ class VM:
> self.params = params
> self.root_dir = root_dir
> self.address_cache = address_cache
> + self.mac_prefix = params.get('mac_prefix')
> self.netdev_id = []
>
> # Find a unique identifier for this VM
> @@ -126,8 +127,15 @@ class VM:
> if not glob.glob("/tmp/*%s" % self.instance):
> break
>
> + if self.mac_prefix is None:
> + s, o = commands.getstatusoutput("ifconfig eth0")
> + if s == 0:
> + mac = re.findall("HWaddr (\S*)", o)[0]
> + self.mac_prefix = '00' + mac[8:] + ':'
> +
>
> - def clone(self, name=None, params=None, root_dir=None, address_cache=None):
> + def clone(self, name=None, params=None, root_dir=None,
> + address_cache=None, mac_clone=True):
> """
> Return a clone of the VM object with optionally modified parameters.
> The clone is initially not alive and needs to be started using create().
> @@ -138,6 +146,7 @@ class VM:
> @param params: Optional new VM creation parameters
> @param root_dir: Optional new base directory for relative filenames
> @param address_cache: A dict that maps MAC addresses to IP addresses
> + @param mac_clone: Clone mac address or not.
> """
Here if you redesign the patch as suggested by Michael, making the
unique key the string 'instance' + 'nic_index', you could add a
parameter like 'preserve_mac', to preserve the original mac address, for
things like migration.
In such cases, first the entry of the nics present on the source vm
would be removed from the MAC pool, then the new entries would be added.
> if name is None:
> name = self.name
> @@ -147,9 +156,19 @@ class VM:
> root_dir = self.root_dir
> if address_cache is None:
> address_cache = self.address_cache
> - return VM(name, params, root_dir, address_cache)
> + vm = VM(name, params, root_dir, address_cache)
> + if mac_clone:
> + # Clone mac address by coping 'self.instance' to the new vm.
> + vm.instance = self.instance
> + return vm
>
>
> + def free_mac_address(self):
> + nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics"))
> + for nic in range(nic_num):
> + mac = self.get_macaddr(nic_index=nic)
> + kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance)
^ Here we are removing an association between a 'instance'+'nic_index'
and a MAC, so maybe this should be called 'clear_macs_from_nics'
> def make_qemu_command(self, name=None, params=None, root_dir=None):
> """
> Generate a qemu command line. All parameters are optional. If a
> @@ -383,6 +402,13 @@ class VM:
> mac = None
> if "address_index" in nic_params:
> mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0]
> + self.set_mac_address(mac=mac, nic_index=vlan)
> + else:
> + mac = kvm_utils.get_mac_from_pool(self.root_dir,
> + vm=self.instance,
> + nic_index=vlan,
> + prefix=self.mac_prefix)
> +
> qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac,
> self.netdev_id[vlan])
> # Handle the '-net tap' or '-net user' part
> @@ -396,7 +422,7 @@ class VM:
> if tftp:
> tftp = kvm_utils.get_path(root_dir, tftp)
> qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"),
> - nic_params.get("nic_ifname"),
> + self.get_ifname(vlan),
> script, downscript, tftp,
> nic_params.get("bootp"), redirs,
> self.netdev_id[vlan])
> @@ -720,7 +746,7 @@ class VM:
> lockfile.close()
>
>
> - def destroy(self, gracefully=True):
> + def destroy(self, gracefully=True, free_macaddr=True):
> """
> Destroy the VM.
>
> @@ -731,6 +757,7 @@ class VM:
> @param gracefully: Whether an attempt will be made to end the VM
> using a shell command before trying to end the qemu process
> with a 'quit' or a kill signal.
> + @param free_macaddr: Whether free macaddresses when destory a vm.
^ Why would we want to conditionally remove the associations NIC-MAC?
Don't we want to do it on all cases?
> """
> try:
> # Is it already dead?
> @@ -751,11 +778,18 @@ class VM:
> logging.debug("Shutdown command sent; waiting for VM "
> "to go down...")
> if kvm_utils.wait_for(self.is_dead, 60, 1, 1):
> - logging.debug("VM is down")
> + logging.debug("VM is down, free mac address.")
> + # free mac address
> + if free_macaddr:
> + self.free_mac_address()
> return
> finally:
> session.close()
>
> + # free mac address
> + if free_macaddr:
> + self.free_mac_address()
> +
> if self.monitor:
> # Try to destroy with a monitor command
> logging.debug("Trying to kill VM with monitor command...")
> @@ -881,10 +915,14 @@ class VM:
> nic_name = nics[index]
> nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
> if nic_params.get("nic_mode") == "tap":
> - mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params)
> + mac = self.get_macaddr(index)
> if not mac:
> logging.debug("MAC address unavailable")
> return None
> + else:
> + mac = mac.lower()
> + ip = None
> +
> if not ip or nic_params.get("always_use_tcpdump") == "yes":
> # Get the IP address from the cache
> ip = self.address_cache.get(mac)
> @@ -897,6 +935,7 @@ class VM:
> for nic in nics]
> macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0]
> for dict in nic_dicts]
> + macs.append(mac)
> if not kvm_utils.verify_ip_address_ownership(ip, macs):
> logging.debug("Could not verify MAC-IP address mapping: "
> "%s ---> %s" % (mac, ip))
> @@ -925,6 +964,71 @@ class VM:
> "redirected" % port)
> return self.redirs.get(port)
>
> + def get_ifname(self, nic_index=0):
> + """
> + Return the ifname of tap device for the guest nic.
> +
> + @param nic_index: Index of the NIC
> + """
> +
> + nics = kvm_utils.get_sub_dict_names(self.params, "nics")
> + nic_name = nics[nic_index]
> + nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
> + if nic_params.get("nic_ifname"):
> + return nic_params.get("nic_ifname")
> + else:
> + return "%s_%s_%s" % (nic_params.get("nic_model"),
> + nic_index, self.vnc_port)
> +
> + def get_macaddr(self, nic_index=0):
> + """
> + Return the macaddr of guest nic.
> +
> + @param nic_index: Index of the NIC
> + """
> + mac_filename = os.path.join(self.root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> + mac_pool = mac_shelve.get("macpool")
> + val = "%s:%s" % (self.instance, nic_index)
> + for key in mac_pool.keys():
> + if val in mac_pool[key]:
> + return key
> + return None
> +
> + def set_mac_address(self, mac, nic_index=0, shareable=False):
> + """
> + Set mac address for guest. Note: It just update address pool.
> +
> + @param mac: address will set to guest
> + @param nic_index: Index of the NIC
> + @param shareable: Where VM can share mac with other VM or not.
> + """
> + lock_filename = os.path.join(self.root_dir, "mac_lock")
> + lock_file = open(lock_filename, 'w')
> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> + mac_filename = os.path.join(self.root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> + mac_pool = mac_shelve.get("macpool")
> + if not mac_pool:
> + mac_pool = {}
> + value = "%s:%s" % (self.instance, nic_index)
> + if mac not in mac_pool.keys():
> + for key in mac_pool.keys():
> + if value in mac_pool[key]:
> + mac_pool[key].remove(value)
> + if len(mac_pool[key]) == 0:
> + mac_pool.pop(key)
> + mac_pool[mac] = [value]
> + else:
> + if shareable:
> + mac_pool[mac].append(value)
> + else:
> + logging.error("Mac address already be used!")
> + return False
> + mac_shelve["macpool"] = mac_pool
> + mac_shelve.close()
> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> + lock_file.close()
>
> def get_pid(self):
> """
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index fd9e72f..6710c00 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -51,7 +51,7 @@ guest_port_remote_shell = 22
> nic_mode = user
> #nic_mode = tap
> nic_script = scripts/qemu-ifup
> -address_index = 0
> +#address_index = 0
> run_tcpdump = yes
>
> # Misc
>
^ permalink raw reply [flat|nested] 77+ messages in thread* [Qemu-devel] Re: [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
@ 2010-07-27 1:48 ` Lucas Meneghel Rodrigues
0 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-27 1:48 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:34 +0800, Amos Kong wrote:
> Old method uses the mac address in the configuration files which could
> lead serious problem when multiple tests running in different hosts.
>
> This patch adds a new macaddress pool algorithm, it generates the mac prefix
> based on mac address of the host which could eliminate the duplicated mac
> addresses between machines.
>
> When user have set the mac_prefix in the configuration file, we should use it
> in stead of the dynamic generated mac prefix.
I have some additional comments besides the good points made by Michael
on his review.
> Other change:
> . Fix randomly generating mac address so that it correspond to IEEE802.
> . Update clone function to decide clone mac address or not.
> . Update get_macaddr function.
> . Add set_mac_address function.
>
> New auto mac address pool algorithm:
> If address_index is defined, VM will get mac from config file then record mac
> in to address_pool. If address_index is not defined, VM will call
> get_mac_from_pool to auto create mac then recored mac to address_pool in
> following format:
> {'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}
>
> AE:9D:94:6A:9b:f9 : mac address
> 20100310-165222-Wt7l : instance attribute of VM
> 0 : index of NIC
>
>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Feng Yang <fyang@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
> index fb2d1c2..7c0946e 100644
> --- a/client/tests/kvm/kvm_utils.py
> +++ b/client/tests/kvm/kvm_utils.py
> @@ -5,6 +5,7 @@ KVM test utility functions.
> """
>
> import time, string, random, socket, os, signal, re, logging, commands, cPickle
> +import fcntl, shelve
> from autotest_lib.client.bin import utils
> from autotest_lib.client.common_lib import error, logging_config
> import kvm_subprocess
> @@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword):
>
> # Functions related to MAC/IP addresses
>
> +def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'):
> + """
> + random generated mac address.
I think I got your original idea of naming this function
get_mac_from_pool (the prefix assessed is the base of the 'pool'), but
as Michael pointed out, a new MAC is being generated here and added to
the mac pool, so a more appropriate name should be used. A
straightforward suggestion would be generate_mac_address. Also, the
description of the function would be more on the lines of "Generate a
MAC address and add it to the MAC pool".
> + 1) First try to generate macaddress based on the mac address prefix.
> + 2) And then try to use total random generated mac address.
> +
> + @param root_dir: Root dir for kvm
> + @param vm: Here we use instance of vm
> + @param nic_index: The index of nic.
> + @param prefix: Prefix of mac address.
> + @Return: Return mac address.
> + """
> +
> + lock_filename = os.path.join(root_dir, "mac_lock")
> + lock_file = open(lock_filename, 'w')
> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> + mac_filename = os.path.join(root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
^ Indeed we should keep the mac pool under /tmp as Michael already
pointed out.
> + mac_pool = mac_shelve.get("macpool")
> +
> + if not mac_pool:
> + mac_pool = {}
> + found = False
> +
> + val = "%s:%s" % (vm, nic_index)
> + for key in mac_pool.keys():
> + if val in mac_pool[key]:
> + mac_pool[key].append(val)
> + found = True
> + mac = key
> +
> + while not found:
> + postfix = "%02x:%02x" % (random.randint(0x00,0xfe),
> + random.randint(0x00,0xfe))
> + mac = prefix + postfix
^ It's more natural to use 'sufix' instead of 'postfix' here.
> + mac_list = mac.split(":")
> + # Clear multicast bit
> + mac_list[0] = int(mac_list[0],16) & 0xfe
> + # Set local assignment bit (IEEE802)
> + mac_list[0] = mac_list[0] | 0x02
> + mac_list[0] = "%02x" % mac_list[0]
> + mac = ":".join(mac_list)
> + if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]):
> + mac_pool[mac] = ["%s:%s" % (vm,nic_index)]
> + found = True
> + mac_shelve["macpool"] = mac_pool
> + logging.debug("generating mac addr %s " % mac)
^ Here the MAC was already generated, so here you could use something
like "Generated MAC address %s"
> + mac_shelve.close()
> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> + lock_file.close()
> + return mac
> +
> +
> +def put_mac_to_pool(root_dir, mac, vm):
> + """
> + Put the macaddress back to address pool
> +
> + @param root_dir: Root dir for kvm
> + @param vm: Here we use instance attribute of vm
^ @param vm: Instance attribute of a vm. Also, we might want to rename
this parameter to vm_instance, to make things more clear and avoid
confusions (vm is referred in several places as a vm instance).
> + @param mac: mac address will be put.
> + @Return: mac address.
> + """
> +
> + lock_filename = os.path.join(root_dir, "mac_lock")
> + lock_file = open(lock_filename, 'w')
> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> + mac_filename = os.path.join(root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> +
> + mac_pool = mac_shelve.get("macpool")
> +
> + if not mac_pool or (not mac in mac_pool):
> + logging.debug("Try to free a macaddress does no in pool")
^ logging.debug("MAC not present in the MAC pool, not modifying pool")
> + logging.debug("macaddress is %s" % mac)
> + logging.debug("pool is %s" % mac_pool)
> + else:
> + if len(mac_pool[mac]) <= 1:
> + mac_pool.pop(mac)
> + else:
> + for value in mac_pool[mac]:
> + if vm in value:
> + mac_pool[mac].remove(value)
> + break
> + if len(mac_pool[mac]) == 0:
> + mac_pool.pop(mac)
> +
> + mac_shelve["macpool"] = mac_pool
> + logging.debug("freeing mac addr %s " % mac)
^ logging.debug("Freeing mac addr %s " % mac). Also, this should go to
the beginning of the function, before the "MAC not present in the..."
message.
> +
> + mac_shelve.close()
> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> + lock_file.close()
> + return mac
> +
> +
> def mac_str_to_int(addr):
> """
> Convert MAC address string to integer.
> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
> index 6cd0688..a9ba6e7 100755
> --- a/client/tests/kvm/kvm_vm.py
> +++ b/client/tests/kvm/kvm_vm.py
> @@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu.
> @copyright: 2008-2009 Red Hat Inc.
> """
>
> -import time, socket, os, logging, fcntl, re, commands, glob
> +import time, socket, os, logging, fcntl, re, commands, shelve, glob
> import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer
> from autotest_lib.client.common_lib import error
> from autotest_lib.client.bin import utils
> @@ -117,6 +117,7 @@ class VM:
> self.params = params
> self.root_dir = root_dir
> self.address_cache = address_cache
> + self.mac_prefix = params.get('mac_prefix')
> self.netdev_id = []
>
> # Find a unique identifier for this VM
> @@ -126,8 +127,15 @@ class VM:
> if not glob.glob("/tmp/*%s" % self.instance):
> break
>
> + if self.mac_prefix is None:
> + s, o = commands.getstatusoutput("ifconfig eth0")
> + if s == 0:
> + mac = re.findall("HWaddr (\S*)", o)[0]
> + self.mac_prefix = '00' + mac[8:] + ':'
> +
>
> - def clone(self, name=None, params=None, root_dir=None, address_cache=None):
> + def clone(self, name=None, params=None, root_dir=None,
> + address_cache=None, mac_clone=True):
> """
> Return a clone of the VM object with optionally modified parameters.
> The clone is initially not alive and needs to be started using create().
> @@ -138,6 +146,7 @@ class VM:
> @param params: Optional new VM creation parameters
> @param root_dir: Optional new base directory for relative filenames
> @param address_cache: A dict that maps MAC addresses to IP addresses
> + @param mac_clone: Clone mac address or not.
> """
Here if you redesign the patch as suggested by Michael, making the
unique key the string 'instance' + 'nic_index', you could add a
parameter like 'preserve_mac', to preserve the original mac address, for
things like migration.
In such cases, first the entry of the nics present on the source vm
would be removed from the MAC pool, then the new entries would be added.
> if name is None:
> name = self.name
> @@ -147,9 +156,19 @@ class VM:
> root_dir = self.root_dir
> if address_cache is None:
> address_cache = self.address_cache
> - return VM(name, params, root_dir, address_cache)
> + vm = VM(name, params, root_dir, address_cache)
> + if mac_clone:
> + # Clone mac address by coping 'self.instance' to the new vm.
> + vm.instance = self.instance
> + return vm
>
>
> + def free_mac_address(self):
> + nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics"))
> + for nic in range(nic_num):
> + mac = self.get_macaddr(nic_index=nic)
> + kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance)
^ Here we are removing an association between a 'instance'+'nic_index'
and a MAC, so maybe this should be called 'clear_macs_from_nics'
> def make_qemu_command(self, name=None, params=None, root_dir=None):
> """
> Generate a qemu command line. All parameters are optional. If a
> @@ -383,6 +402,13 @@ class VM:
> mac = None
> if "address_index" in nic_params:
> mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0]
> + self.set_mac_address(mac=mac, nic_index=vlan)
> + else:
> + mac = kvm_utils.get_mac_from_pool(self.root_dir,
> + vm=self.instance,
> + nic_index=vlan,
> + prefix=self.mac_prefix)
> +
> qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac,
> self.netdev_id[vlan])
> # Handle the '-net tap' or '-net user' part
> @@ -396,7 +422,7 @@ class VM:
> if tftp:
> tftp = kvm_utils.get_path(root_dir, tftp)
> qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"),
> - nic_params.get("nic_ifname"),
> + self.get_ifname(vlan),
> script, downscript, tftp,
> nic_params.get("bootp"), redirs,
> self.netdev_id[vlan])
> @@ -720,7 +746,7 @@ class VM:
> lockfile.close()
>
>
> - def destroy(self, gracefully=True):
> + def destroy(self, gracefully=True, free_macaddr=True):
> """
> Destroy the VM.
>
> @@ -731,6 +757,7 @@ class VM:
> @param gracefully: Whether an attempt will be made to end the VM
> using a shell command before trying to end the qemu process
> with a 'quit' or a kill signal.
> + @param free_macaddr: Whether free macaddresses when destory a vm.
^ Why would we want to conditionally remove the associations NIC-MAC?
Don't we want to do it on all cases?
> """
> try:
> # Is it already dead?
> @@ -751,11 +778,18 @@ class VM:
> logging.debug("Shutdown command sent; waiting for VM "
> "to go down...")
> if kvm_utils.wait_for(self.is_dead, 60, 1, 1):
> - logging.debug("VM is down")
> + logging.debug("VM is down, free mac address.")
> + # free mac address
> + if free_macaddr:
> + self.free_mac_address()
> return
> finally:
> session.close()
>
> + # free mac address
> + if free_macaddr:
> + self.free_mac_address()
> +
> if self.monitor:
> # Try to destroy with a monitor command
> logging.debug("Trying to kill VM with monitor command...")
> @@ -881,10 +915,14 @@ class VM:
> nic_name = nics[index]
> nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
> if nic_params.get("nic_mode") == "tap":
> - mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params)
> + mac = self.get_macaddr(index)
> if not mac:
> logging.debug("MAC address unavailable")
> return None
> + else:
> + mac = mac.lower()
> + ip = None
> +
> if not ip or nic_params.get("always_use_tcpdump") == "yes":
> # Get the IP address from the cache
> ip = self.address_cache.get(mac)
> @@ -897,6 +935,7 @@ class VM:
> for nic in nics]
> macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0]
> for dict in nic_dicts]
> + macs.append(mac)
> if not kvm_utils.verify_ip_address_ownership(ip, macs):
> logging.debug("Could not verify MAC-IP address mapping: "
> "%s ---> %s" % (mac, ip))
> @@ -925,6 +964,71 @@ class VM:
> "redirected" % port)
> return self.redirs.get(port)
>
> + def get_ifname(self, nic_index=0):
> + """
> + Return the ifname of tap device for the guest nic.
> +
> + @param nic_index: Index of the NIC
> + """
> +
> + nics = kvm_utils.get_sub_dict_names(self.params, "nics")
> + nic_name = nics[nic_index]
> + nic_params = kvm_utils.get_sub_dict(self.params, nic_name)
> + if nic_params.get("nic_ifname"):
> + return nic_params.get("nic_ifname")
> + else:
> + return "%s_%s_%s" % (nic_params.get("nic_model"),
> + nic_index, self.vnc_port)
> +
> + def get_macaddr(self, nic_index=0):
> + """
> + Return the macaddr of guest nic.
> +
> + @param nic_index: Index of the NIC
> + """
> + mac_filename = os.path.join(self.root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> + mac_pool = mac_shelve.get("macpool")
> + val = "%s:%s" % (self.instance, nic_index)
> + for key in mac_pool.keys():
> + if val in mac_pool[key]:
> + return key
> + return None
> +
> + def set_mac_address(self, mac, nic_index=0, shareable=False):
> + """
> + Set mac address for guest. Note: It just update address pool.
> +
> + @param mac: address will set to guest
> + @param nic_index: Index of the NIC
> + @param shareable: Where VM can share mac with other VM or not.
> + """
> + lock_filename = os.path.join(self.root_dir, "mac_lock")
> + lock_file = open(lock_filename, 'w')
> + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX)
> + mac_filename = os.path.join(self.root_dir, "address_pool")
> + mac_shelve = shelve.open(mac_filename, writeback=False)
> + mac_pool = mac_shelve.get("macpool")
> + if not mac_pool:
> + mac_pool = {}
> + value = "%s:%s" % (self.instance, nic_index)
> + if mac not in mac_pool.keys():
> + for key in mac_pool.keys():
> + if value in mac_pool[key]:
> + mac_pool[key].remove(value)
> + if len(mac_pool[key]) == 0:
> + mac_pool.pop(key)
> + mac_pool[mac] = [value]
> + else:
> + if shareable:
> + mac_pool[mac].append(value)
> + else:
> + logging.error("Mac address already be used!")
> + return False
> + mac_shelve["macpool"] = mac_pool
> + mac_shelve.close()
> + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
> + lock_file.close()
>
> def get_pid(self):
> """
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index fd9e72f..6710c00 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -51,7 +51,7 @@ guest_port_remote_shell = 22
> nic_mode = user
> #nic_mode = tap
> nic_script = scripts/qemu-ifup
> -address_index = 0
> +#address_index = 0
> run_tcpdump = yes
>
> # Misc
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [RFC PATCH 02/14] KVM Test: Add a function get_interface_name() to kvm_net_utils.py
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:35 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
The function get_interface_name is used to get the interface name of linux
guest through the macaddress of specified macaddress.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
new file mode 100644
index 0000000..ede4965
--- /dev/null
+++ b/client/tests/kvm/kvm_net_utils.py
@@ -0,0 +1,18 @@
+import re
+
+def get_linux_ifname(session, mac_address):
+ """
+ Get the interface name through the mac address.
+
+ @param session: session to the virtual machine
+ @mac_address: the macaddress of nic
+ """
+
+ output = session.get_command_output("ifconfig -a")
+
+ try:
+ ethname = re.findall("(\w+)\s+Link.*%s" % mac_address, output,
+ re.IGNORECASE)[0]
+ return ethname
+ except:
+ return None
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 02/14] KVM Test: Add a function get_interface_name() to kvm_net_utils.py
@ 2010-07-20 1:35 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
The function get_interface_name is used to get the interface name of linux
guest through the macaddress of specified macaddress.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
new file mode 100644
index 0000000..ede4965
--- /dev/null
+++ b/client/tests/kvm/kvm_net_utils.py
@@ -0,0 +1,18 @@
+import re
+
+def get_linux_ifname(session, mac_address):
+ """
+ Get the interface name through the mac address.
+
+ @param session: session to the virtual machine
+ @mac_address: the macaddress of nic
+ """
+
+ output = session.get_command_output("ifconfig -a")
+
+ try:
+ ethname = re.findall("(\w+)\s+Link.*%s" % mac_address, output,
+ re.IGNORECASE)[0]
+ return ethname
+ except:
+ return None
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 02/14] KVM Test: Add a function get_interface_name() to kvm_net_utils.py
2010-07-20 1:35 ` [Qemu-devel] " Amos Kong
@ 2010-07-27 2:08 ` Lucas Meneghel Rodrigues
-1 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-27 2:08 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> The function get_interface_name is used to get the interface name of linux
> guest through the macaddress of specified macaddress.
I wonder if it wouldn't be overkill to have separate utility libraries
on the kvm test instead of a single kvm_utils and kvm_test_utils like
you are proposing. Any thoughts Michael?
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
> new file mode 100644
> index 0000000..ede4965
> --- /dev/null
> +++ b/client/tests/kvm/kvm_net_utils.py
> @@ -0,0 +1,18 @@
> +import re
> +
> +def get_linux_ifname(session, mac_address):
> + """
> + Get the interface name through the mac address.
> +
> + @param session: session to the virtual machine
> + @mac_address: the macaddress of nic
> + """
> +
> + output = session.get_command_output("ifconfig -a")
> +
> + try:
> + ethname = re.findall("(\w+)\s+Link.*%s" % mac_address, output,
> + re.IGNORECASE)[0]
> + return ethname
> + except:
> + return None
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 02/14] KVM Test: Add a function get_interface_name() to kvm_net_utils.py
@ 2010-07-27 2:08 ` Lucas Meneghel Rodrigues
0 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-27 2:08 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, Michael Goldish, qemu-devel, kvm
On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> The function get_interface_name is used to get the interface name of linux
> guest through the macaddress of specified macaddress.
I wonder if it wouldn't be overkill to have separate utility libraries
on the kvm test instead of a single kvm_utils and kvm_test_utils like
you are proposing. Any thoughts Michael?
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
> new file mode 100644
> index 0000000..ede4965
> --- /dev/null
> +++ b/client/tests/kvm/kvm_net_utils.py
> @@ -0,0 +1,18 @@
> +import re
> +
> +def get_linux_ifname(session, mac_address):
> + """
> + Get the interface name through the mac address.
> +
> + @param session: session to the virtual machine
> + @mac_address: the macaddress of nic
> + """
> +
> + output = session.get_command_output("ifconfig -a")
> +
> + try:
> + ethname = re.findall("(\w+)\s+Link.*%s" % mac_address, output,
> + re.IGNORECASE)[0]
> + return ethname
> + except:
> + return None
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 02/14] KVM Test: Add a function get_interface_name() to kvm_net_utils.py
2010-07-27 2:08 ` Lucas Meneghel Rodrigues
@ 2010-07-28 10:29 ` Michael Goldish
-1 siblings, 0 replies; 77+ messages in thread
From: Michael Goldish @ 2010-07-28 10:29 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: autotest, qemu-devel, kvm
On 07/27/2010 05:08 AM, Lucas Meneghel Rodrigues wrote:
> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
>> The function get_interface_name is used to get the interface name of linux
>> guest through the macaddress of specified macaddress.
>
> I wonder if it wouldn't be overkill to have separate utility libraries
> on the kvm test instead of a single kvm_utils and kvm_test_utils like
> you are proposing. Any thoughts Michael?
Yes, looks like this could be in kvm_test_utils.py, especially if
there's only a small number of functions here.
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> Signed-off-by: Amos Kong <akong@redhat.com>
>> ---
>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>
>> diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
>> new file mode 100644
>> index 0000000..ede4965
>> --- /dev/null
>> +++ b/client/tests/kvm/kvm_net_utils.py
>> @@ -0,0 +1,18 @@
>> +import re
>> +
>> +def get_linux_ifname(session, mac_address):
>> + """
>> + Get the interface name through the mac address.
>> +
>> + @param session: session to the virtual machine
>> + @mac_address: the macaddress of nic
>> + """
>> +
>> + output = session.get_command_output("ifconfig -a")
>> +
>> + try:
>> + ethname = re.findall("(\w+)\s+Link.*%s" % mac_address, output,
>> + re.IGNORECASE)[0]
>> + return ethname
>> + except:
>> + return None
>>
>>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 02/14] KVM Test: Add a function get_interface_name() to kvm_net_utils.py
@ 2010-07-28 10:29 ` Michael Goldish
0 siblings, 0 replies; 77+ messages in thread
From: Michael Goldish @ 2010-07-28 10:29 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: autotest, Amos Kong, qemu-devel, kvm
On 07/27/2010 05:08 AM, Lucas Meneghel Rodrigues wrote:
> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
>> The function get_interface_name is used to get the interface name of linux
>> guest through the macaddress of specified macaddress.
>
> I wonder if it wouldn't be overkill to have separate utility libraries
> on the kvm test instead of a single kvm_utils and kvm_test_utils like
> you are proposing. Any thoughts Michael?
Yes, looks like this could be in kvm_test_utils.py, especially if
there's only a small number of functions here.
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> Signed-off-by: Amos Kong <akong@redhat.com>
>> ---
>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>
>> diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
>> new file mode 100644
>> index 0000000..ede4965
>> --- /dev/null
>> +++ b/client/tests/kvm/kvm_net_utils.py
>> @@ -0,0 +1,18 @@
>> +import re
>> +
>> +def get_linux_ifname(session, mac_address):
>> + """
>> + Get the interface name through the mac address.
>> +
>> + @param session: session to the virtual machine
>> + @mac_address: the macaddress of nic
>> + """
>> +
>> + output = session.get_command_output("ifconfig -a")
>> +
>> + try:
>> + ethname = re.findall("(\w+)\s+Link.*%s" % mac_address, output,
>> + re.IGNORECASE)[0]
>> + return ethname
>> + except:
>> + return None
>>
>>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 02/14] KVM Test: Add a function get_interface_name() to kvm_net_utils.py
2010-07-28 10:29 ` Michael Goldish
(?)
@ 2010-08-03 1:39 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-03 1:39 UTC (permalink / raw)
To: Michael Goldish; +Cc: Lucas Meneghel Rodrigues, autotest, qemu-devel, kvm
On Wed, Jul 28, 2010 at 01:29:22PM +0300, Michael Goldish wrote:
> On 07/27/2010 05:08 AM, Lucas Meneghel Rodrigues wrote:
> > On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> >> The function get_interface_name is used to get the interface name of linux
> >> guest through the macaddress of specified macaddress.
> >
> > I wonder if it wouldn't be overkill to have separate utility libraries
> > on the kvm test instead of a single kvm_utils and kvm_test_utils like
> > you are proposing. Any thoughts Michael?
>
> Yes, looks like this could be in kvm_test_utils.py, especially if
> there's only a small number of functions here.
Current functions of network are not too much, but we may add a lot of new utility functions in the future.
It's more clear to use another file.
> >> Signed-off-by: Jason Wang <jasowang@redhat.com>
> >> Signed-off-by: Amos Kong <akong@redhat.com>
> >> ---
> >> 0 files changed, 0 insertions(+), 0 deletions(-)
> >>
> >> diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
> >> new file mode 100644
> >> index 0000000..ede4965
> >> --- /dev/null
> >> +++ b/client/tests/kvm/kvm_net_utils.py
> >> @@ -0,0 +1,18 @@
> >> +import re
> >> +
> >> +def get_linux_ifname(session, mac_address):
> >> + """
> >> + Get the interface name through the mac address.
> >> +
> >> + @param session: session to the virtual machine
> >> + @mac_address: the macaddress of nic
> >> + """
> >> +
> >> + output = session.get_command_output("ifconfig -a")
> >> +
> >> + try:
> >> + ethname = re.findall("(\w+)\s+Link.*%s" % mac_address, output,
> >> + re.IGNORECASE)[0]
> >> + return ethname
> >> + except:
> >> + return None
> >>
> >>
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [RFC PATCH 03/14] KVM Test: Add a common ping module for network related tests
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:35 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
The kvm_net_utils.py is a just a place that wraps common network
related commands which is used to do the network-related tests.
Use -1 as the packet ratio for loss analysis.
Use quiet mode when doing the flood ping.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
index ede4965..8a71858 100644
--- a/client/tests/kvm/kvm_net_utils.py
+++ b/client/tests/kvm/kvm_net_utils.py
@@ -1,4 +1,114 @@
-import re
+import logging, re, signal
+from autotest_lib.client.common_lib import error
+import kvm_subprocess, kvm_utils
+
+def get_loss_ratio(output):
+ """
+ Get the packet loss ratio from the output of ping
+
+ @param output
+ """
+ try:
+ return int(re.findall('(\d+)% packet loss', output)[0])
+ except IndexError:
+ logging.debug(output)
+ return -1
+
+def raw_ping(command, timeout, session, output_func):
+ """
+ Low-level ping command execution.
+
+ @param command: ping command
+ @param timeout: timeout of the ping command
+ @param session: local executon hint or session to execute the ping command
+ """
+ if session == "localhost":
+ process = kvm_subprocess.run_bg(command, output_func=output_func,
+ timeout=timeout)
+
+ # Send SIGINT singal to notify the timeout of running ping process,
+ # Because ping have the ability to catch the SIGINT signal so we can
+ # always get the packet loss ratio even if timeout.
+ if process.is_alive():
+ kvm_utils.kill_process_tree(process.get_pid(), signal.SIGINT)
+
+ status = process.get_status()
+ output = process.get_output()
+
+ process.close()
+ return status, output
+ else:
+ session.sendline(command)
+ status, output = session.read_up_to_prompt(timeout=timeout,
+ print_func=output_func)
+ if status is False:
+ # Send ctrl+c (SIGINT) through ssh session
+ session.sendline("\003")
+ status, output2 = session.read_up_to_prompt(print_func=output_func)
+ output += output2
+ if status is False:
+ # We also need to use this session to query the return value
+ session.sendline("\003")
+
+ session.sendline(session.status_test_command)
+ s2, o2 = session.read_up_to_prompt()
+ if s2 is False:
+ status = -1
+ else:
+ try:
+ status = int(re.findall("\d+", o2)[0])
+ except:
+ status = -1
+
+ return status, output
+
+def ping(dest = "localhost", count = None, interval = None, interface = None,
+ packetsize = None, ttl = None, hint = None, adaptive = False,
+ broadcast = False, flood = False, timeout = 0,
+ output_func = logging.debug, session = "localhost"):
+ """
+ Wrapper of ping.
+
+ @param dest: destination address
+ @param count: count of icmp packet
+ @param interval: interval of two icmp echo request
+ @param interface: specified interface of the source address
+ @param packetsize: packet size of icmp
+ @param ttl: ip time to live
+ @param hint: path mtu discovery hint
+ @param adaptive: adaptive ping flag
+ @param broadcast: broadcast ping flag
+ @param flood: flood ping flag
+ @param timeout: timeout for the ping command
+ @param output_func: function used to log the result of ping
+ @param session: local executon hint or session to execute the ping command
+ """
+
+ command = "ping %s " % dest
+
+ if count is not None:
+ command += " -c %s" % count
+ if interval is not None:
+ command += " -i %s" % interval
+ if interface is not None:
+ command += " -I %s" % interface
+ if packetsize is not None:
+ command += " -s %s" % packetsize
+ if ttl is not None:
+ command += " -t %s" % ttl
+ if hint is not None:
+ command += " -M %s" % hint
+ if adaptive is True:
+ command += " -A"
+ if broadcast is True:
+ command += " -b"
+ if flood is True:
+ # temporary workaround as the kvm_subprocess may not properly handle
+ # the timeout for the output of flood ping
+ command += " -f -q"
+ output_func = None
+
+ return raw_ping(command, timeout, session, output_func)
def get_linux_ifname(session, mac_address):
"""
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 03/14] KVM Test: Add a common ping module for network related tests
@ 2010-07-20 1:35 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
The kvm_net_utils.py is a just a place that wraps common network
related commands which is used to do the network-related tests.
Use -1 as the packet ratio for loss analysis.
Use quiet mode when doing the flood ping.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
index ede4965..8a71858 100644
--- a/client/tests/kvm/kvm_net_utils.py
+++ b/client/tests/kvm/kvm_net_utils.py
@@ -1,4 +1,114 @@
-import re
+import logging, re, signal
+from autotest_lib.client.common_lib import error
+import kvm_subprocess, kvm_utils
+
+def get_loss_ratio(output):
+ """
+ Get the packet loss ratio from the output of ping
+
+ @param output
+ """
+ try:
+ return int(re.findall('(\d+)% packet loss', output)[0])
+ except IndexError:
+ logging.debug(output)
+ return -1
+
+def raw_ping(command, timeout, session, output_func):
+ """
+ Low-level ping command execution.
+
+ @param command: ping command
+ @param timeout: timeout of the ping command
+ @param session: local executon hint or session to execute the ping command
+ """
+ if session == "localhost":
+ process = kvm_subprocess.run_bg(command, output_func=output_func,
+ timeout=timeout)
+
+ # Send SIGINT singal to notify the timeout of running ping process,
+ # Because ping have the ability to catch the SIGINT signal so we can
+ # always get the packet loss ratio even if timeout.
+ if process.is_alive():
+ kvm_utils.kill_process_tree(process.get_pid(), signal.SIGINT)
+
+ status = process.get_status()
+ output = process.get_output()
+
+ process.close()
+ return status, output
+ else:
+ session.sendline(command)
+ status, output = session.read_up_to_prompt(timeout=timeout,
+ print_func=output_func)
+ if status is False:
+ # Send ctrl+c (SIGINT) through ssh session
+ session.sendline("\003")
+ status, output2 = session.read_up_to_prompt(print_func=output_func)
+ output += output2
+ if status is False:
+ # We also need to use this session to query the return value
+ session.sendline("\003")
+
+ session.sendline(session.status_test_command)
+ s2, o2 = session.read_up_to_prompt()
+ if s2 is False:
+ status = -1
+ else:
+ try:
+ status = int(re.findall("\d+", o2)[0])
+ except:
+ status = -1
+
+ return status, output
+
+def ping(dest = "localhost", count = None, interval = None, interface = None,
+ packetsize = None, ttl = None, hint = None, adaptive = False,
+ broadcast = False, flood = False, timeout = 0,
+ output_func = logging.debug, session = "localhost"):
+ """
+ Wrapper of ping.
+
+ @param dest: destination address
+ @param count: count of icmp packet
+ @param interval: interval of two icmp echo request
+ @param interface: specified interface of the source address
+ @param packetsize: packet size of icmp
+ @param ttl: ip time to live
+ @param hint: path mtu discovery hint
+ @param adaptive: adaptive ping flag
+ @param broadcast: broadcast ping flag
+ @param flood: flood ping flag
+ @param timeout: timeout for the ping command
+ @param output_func: function used to log the result of ping
+ @param session: local executon hint or session to execute the ping command
+ """
+
+ command = "ping %s " % dest
+
+ if count is not None:
+ command += " -c %s" % count
+ if interval is not None:
+ command += " -i %s" % interval
+ if interface is not None:
+ command += " -I %s" % interface
+ if packetsize is not None:
+ command += " -s %s" % packetsize
+ if ttl is not None:
+ command += " -t %s" % ttl
+ if hint is not None:
+ command += " -M %s" % hint
+ if adaptive is True:
+ command += " -A"
+ if broadcast is True:
+ command += " -b"
+ if flood is True:
+ # temporary workaround as the kvm_subprocess may not properly handle
+ # the timeout for the output of flood ping
+ command += " -f -q"
+ output_func = None
+
+ return raw_ping(command, timeout, session, output_func)
def get_linux_ifname(session, mac_address):
"""
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 03/14] KVM Test: Add a common ping module for network related tests
2010-07-20 1:35 ` [Qemu-devel] " Amos Kong
(?)
@ 2010-07-27 13:01 ` Lucas Meneghel Rodrigues
2010-07-28 11:50 ` Michael Goldish
-1 siblings, 1 reply; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-27 13:01 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> The kvm_net_utils.py is a just a place that wraps common network
> related commands which is used to do the network-related tests.
> Use -1 as the packet ratio for loss analysis.
> Use quiet mode when doing the flood ping.
>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
> index ede4965..8a71858 100644
> --- a/client/tests/kvm/kvm_net_utils.py
> +++ b/client/tests/kvm/kvm_net_utils.py
> @@ -1,4 +1,114 @@
> -import re
> +import logging, re, signal
> +from autotest_lib.client.common_lib import error
> +import kvm_subprocess, kvm_utils
> +
> +def get_loss_ratio(output):
> + """
> + Get the packet loss ratio from the output of ping
> +
> + @param output
> + """
> + try:
> + return int(re.findall('(\d+)% packet loss', output)[0])
> + except IndexError:
> + logging.debug(output)
> + return -1
> +def raw_ping(command, timeout, session, output_func):
> + """
> + Low-level ping command execution.
> +
> + @param command: ping command
> + @param timeout: timeout of the ping command
> + @param session: local executon hint or session to execute the ping command
> + """
> + if session == "localhost":
> + process = kvm_subprocess.run_bg(command, output_func=output_func,
> + timeout=timeout)
Do we really have to resort to kvm_subprocess here? We use subprocess on
special occasions, usually when the command in question is required to
live throughout the tests, which doesn't seem to be the case.
kvm_subprocess is a good library, but it is a more heavyweight
alternative (spawns its own shell process, for example). Maybe you are
using it because you can directly send input to the process at any time,
such as the ctrl+c later on this same code.
> +
> + # Send SIGINT singal to notify the timeout of running ping process,
^ typo, signal
> + # Because ping have the ability to catch the SIGINT signal so we can
> + # always get the packet loss ratio even if timeout.
> + if process.is_alive():
> + kvm_utils.kill_process_tree(process.get_pid(), signal.SIGINT)
> +
> + status = process.get_status()
> + output = process.get_output()
> +
> + process.close()
> + return status, output
> + else:
> + session.sendline(command)
> + status, output = session.read_up_to_prompt(timeout=timeout,
> + print_func=output_func)
> + if status is False:
^ For None objects, it is better to explicitly test for is None. However
the above construct seems more natural if put as
if not status:
Any particular reason you tested explicitely for False?
> + # Send ctrl+c (SIGINT) through ssh session
> + session.sendline("\003")
> + status, output2 = session.read_up_to_prompt(print_func=output_func)
> + output += output2
> + if status is False:
> + # We also need to use this session to query the return value
> + session.sendline("\003")
> +
> + session.sendline(session.status_test_command)
> + s2, o2 = session.read_up_to_prompt()
> + if s2 is False:
^ See comment above
> + status = -1
> + else:
> + try:
> + status = int(re.findall("\d+", o2)[0])
> + except:
> + status = -1
> +
> + return status, output
> +
> +def ping(dest = "localhost", count = None, interval = None, interface = None,
> + packetsize = None, ttl = None, hint = None, adaptive = False,
> + broadcast = False, flood = False, timeout = 0,
> + output_func = logging.debug, session = "localhost"):
^ On function headers, we pretty much follow PEP 8 and don't use spacing
between argument keys, the equal sign and the default value, so this
header should be rewritten
def ping(dest="localhost",...)
> + """
> + Wrapper of ping.
> +
> + @param dest: destination address
> + @param count: count of icmp packet
> + @param interval: interval of two icmp echo request
> + @param interface: specified interface of the source address
> + @param packetsize: packet size of icmp
> + @param ttl: ip time to live
> + @param hint: path mtu discovery hint
> + @param adaptive: adaptive ping flag
> + @param broadcast: broadcast ping flag
> + @param flood: flood ping flag
> + @param timeout: timeout for the ping command
> + @param output_func: function used to log the result of ping
> + @param session: local executon hint or session to execute the ping command
> + """
> +
> + command = "ping %s " % dest
> +
> + if count is not None:
> + command += " -c %s" % count
> + if interval is not None:
> + command += " -i %s" % interval
> + if interface is not None:
> + command += " -I %s" % interface
> + if packetsize is not None:
> + command += " -s %s" % packetsize
> + if ttl is not None:
> + command += " -t %s" % ttl
> + if hint is not None:
> + command += " -M %s" % hint
> + if adaptive is True:
> + command += " -A"
> + if broadcast is True:
> + command += " -b"
> + if flood is True:
> + # temporary workaround as the kvm_subprocess may not properly handle
> + # the timeout for the output of flood ping
> + command += " -f -q"
> + output_func = None
^ the last 3 if clauses could be simpler if put as:
if adaptive:
if broadcast:
if flood:
> + return raw_ping(command, timeout, session, output_func)
>
> def get_linux_ifname(session, mac_address):
> """
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 03/14] KVM Test: Add a common ping module for network related tests
2010-07-27 13:01 ` Lucas Meneghel Rodrigues
@ 2010-07-28 11:50 ` Michael Goldish
2010-07-28 13:56 ` Michael Goldish
0 siblings, 1 reply; 77+ messages in thread
From: Michael Goldish @ 2010-07-28 11:50 UTC (permalink / raw)
To: Amos Kong; +Cc: Lucas Meneghel Rodrigues, autotest, qemu-devel, kvm
On 07/27/2010 04:01 PM, Lucas Meneghel Rodrigues wrote:
> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
>> The kvm_net_utils.py is a just a place that wraps common network
>> related commands which is used to do the network-related tests.
>> Use -1 as the packet ratio for loss analysis.
>> Use quiet mode when doing the flood ping.
>>
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> Signed-off-by: Amos Kong <akong@redhat.com>
>> ---
>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>
>> diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
>> index ede4965..8a71858 100644
>> --- a/client/tests/kvm/kvm_net_utils.py
>> +++ b/client/tests/kvm/kvm_net_utils.py
>> @@ -1,4 +1,114 @@
>> -import re
>> +import logging, re, signal
>> +from autotest_lib.client.common_lib import error
>> +import kvm_subprocess, kvm_utils
>> +
>> +def get_loss_ratio(output):
>> + """
>> + Get the packet loss ratio from the output of ping
>> +
>> + @param output
>> + """
>> + try:
>> + return int(re.findall('(\d+)% packet loss', output)[0])
>> + except IndexError:
>> + logging.debug(output)
>> + return -1
>
>> +def raw_ping(command, timeout, session, output_func):
>> + """
>> + Low-level ping command execution.
>> +
>> + @param command: ping command
>> + @param timeout: timeout of the ping command
>> + @param session: local executon hint or session to execute the ping command
>> + """
>> + if session == "localhost":
I have no problem with this, but wouldn't it be nicer to use None to
indicate that the session should be local?
>> + process = kvm_subprocess.run_bg(command, output_func=output_func,
>> + timeout=timeout)
>
> Do we really have to resort to kvm_subprocess here? We use subprocess on
> special occasions, usually when the command in question is required to
> live throughout the tests, which doesn't seem to be the case.
> kvm_subprocess is a good library, but it is a more heavyweight
> alternative (spawns its own shell process, for example). Maybe you are
> using it because you can directly send input to the process at any time,
> such as the ctrl+c later on this same code.
>
>> +
>> + # Send SIGINT singal to notify the timeout of running ping process,
>
> ^ typo, signal
>
>> + # Because ping have the ability to catch the SIGINT signal so we can
>> + # always get the packet loss ratio even if timeout.
>> + if process.is_alive():
>> + kvm_utils.kill_process_tree(process.get_pid(), signal.SIGINT)
>> +
>> + status = process.get_status()
>> + output = process.get_output()
>> +
>> + process.close()
>> + return status, output
>> + else:
>> + session.sendline(command)
>> + status, output = session.read_up_to_prompt(timeout=timeout,
>> + print_func=output_func)
>> + if status is False:
>
> ^ For None objects, it is better to explicitly test for is None. However
> the above construct seems more natural if put as
>
> if not status:
>
> Any particular reason you tested explicitely for False?
read_up_to_prompt() returns True/False as the first member of the tuple.
>> + # Send ctrl+c (SIGINT) through ssh session
>> + session.sendline("\003")
I think you can use send() here. sendline() calls send() with a newline
added to the string.
>> + status, output2 = session.read_up_to_prompt(print_func=output_func)
>> + output += output2
>> + if status is False:
>> + # We also need to use this session to query the return value
>> + session.sendline("\003")
Same here.
>> +
>> + session.sendline(session.status_test_command)
>> + s2, o2 = session.read_up_to_prompt()
>> + if s2 is False:
>
> ^ See comment above
>
>> + status = -1
>> + else:
>> + try:
>> + status = int(re.findall("\d+", o2)[0])
>> + except:
>> + status = -1
>> +
>> + return status, output
>> +
>> +def ping(dest = "localhost", count = None, interval = None, interface = None,
>> + packetsize = None, ttl = None, hint = None, adaptive = False,
>> + broadcast = False, flood = False, timeout = 0,
>> + output_func = logging.debug, session = "localhost"):
>
> ^ On function headers, we pretty much follow PEP 8 and don't use spacing
> between argument keys, the equal sign and the default value, so this
> header should be rewritten
>
> def ping(dest="localhost",...)
>
>> + """
>> + Wrapper of ping.
>> +
>> + @param dest: destination address
>> + @param count: count of icmp packet
>> + @param interval: interval of two icmp echo request
>> + @param interface: specified interface of the source address
>> + @param packetsize: packet size of icmp
>> + @param ttl: ip time to live
>> + @param hint: path mtu discovery hint
>> + @param adaptive: adaptive ping flag
>> + @param broadcast: broadcast ping flag
>> + @param flood: flood ping flag
>> + @param timeout: timeout for the ping command
>> + @param output_func: function used to log the result of ping
>> + @param session: local executon hint or session to execute the ping command
Don't we need this for Windows as well? If we do, wouldn't it be easier
to use a parameter (e.g. 'ping_options = -c 3') to directly control the
ping options from the config file, instead of using an OS-specific ping
wrapper?
>> + """
>> +
>> + command = "ping %s " % dest
>> +
>> + if count is not None:
>> + command += " -c %s" % count
>> + if interval is not None:
>> + command += " -i %s" % interval
>> + if interface is not None:
>> + command += " -I %s" % interface
>> + if packetsize is not None:
>> + command += " -s %s" % packetsize
>> + if ttl is not None:
>> + command += " -t %s" % ttl
>> + if hint is not None:
>> + command += " -M %s" % hint
>> + if adaptive is True:
>> + command += " -A"
>> + if broadcast is True:
>> + command += " -b"
>> + if flood is True:
>> + # temporary workaround as the kvm_subprocess may not properly handle
>> + # the timeout for the output of flood ping
What do you mean by this? If there's a problem with kvm_subprocess it
should be fixed. Have you tried passing internal_timeout=0 to the
kvm_subprocess function you're using?
>> + command += " -f -q"
If a lot of output is generated, it may not be a bad idea to use -q here
regardless of kvm_subprocess limitations.
>> + output_func = None
>
> ^ the last 3 if clauses could be simpler if put as:
>
> if adaptive:
>
> if broadcast:
>
> if flood:
>
>> + return raw_ping(command, timeout, session, output_func)
>>
>> def get_linux_ifname(session, mac_address):
>> """
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 03/14] KVM Test: Add a common ping module for network related tests
2010-07-28 11:50 ` Michael Goldish
@ 2010-07-28 13:56 ` Michael Goldish
0 siblings, 0 replies; 77+ messages in thread
From: Michael Goldish @ 2010-07-28 13:56 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On 07/28/2010 02:50 PM, Michael Goldish wrote:
> On 07/27/2010 04:01 PM, Lucas Meneghel Rodrigues wrote:
>> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
>>> The kvm_net_utils.py is a just a place that wraps common network
>>> related commands which is used to do the network-related tests.
>>> Use -1 as the packet ratio for loss analysis.
>>> Use quiet mode when doing the flood ping.
>>>
>>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>>> Signed-off-by: Amos Kong <akong@redhat.com>
>>> ---
>>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
>>> index ede4965..8a71858 100644
>>> --- a/client/tests/kvm/kvm_net_utils.py
>>> +++ b/client/tests/kvm/kvm_net_utils.py
>>> @@ -1,4 +1,114 @@
>>> -import re
>>> +import logging, re, signal
>>> +from autotest_lib.client.common_lib import error
>>> +import kvm_subprocess, kvm_utils
>>> +
>>> +def get_loss_ratio(output):
>>> + """
>>> + Get the packet loss ratio from the output of ping
>>> +
>>> + @param output
>>> + """
>>> + try:
>>> + return int(re.findall('(\d+)% packet loss', output)[0])
>>> + except IndexError:
>>> + logging.debug(output)
>>> + return -1
>>
>>> +def raw_ping(command, timeout, session, output_func):
>>> + """
>>> + Low-level ping command execution.
>>> +
>>> + @param command: ping command
>>> + @param timeout: timeout of the ping command
>>> + @param session: local executon hint or session to execute the ping command
>>> + """
>>> + if session == "localhost":
>
> I have no problem with this, but wouldn't it be nicer to use None to
> indicate that the session should be local?
>
>>> + process = kvm_subprocess.run_bg(command, output_func=output_func,
>>> + timeout=timeout)
>>
>> Do we really have to resort to kvm_subprocess here? We use subprocess on
>> special occasions, usually when the command in question is required to
>> live throughout the tests, which doesn't seem to be the case.
>> kvm_subprocess is a good library, but it is a more heavyweight
>> alternative (spawns its own shell process, for example). Maybe you are
>> using it because you can directly send input to the process at any time,
>> such as the ctrl+c later on this same code.
>>
>>> +
>>> + # Send SIGINT singal to notify the timeout of running ping process,
>>
>> ^ typo, signal
>>
>>> + # Because ping have the ability to catch the SIGINT signal so we can
>>> + # always get the packet loss ratio even if timeout.
>>> + if process.is_alive():
>>> + kvm_utils.kill_process_tree(process.get_pid(), signal.SIGINT)
>>> +
>>> + status = process.get_status()
>>> + output = process.get_output()
>>> +
>>> + process.close()
>>> + return status, output
>>> + else:
>>> + session.sendline(command)
>>> + status, output = session.read_up_to_prompt(timeout=timeout,
>>> + print_func=output_func)
>>> + if status is False:
>>
>> ^ For None objects, it is better to explicitly test for is None. However
>> the above construct seems more natural if put as
>>
>> if not status:
>>
>> Any particular reason you tested explicitely for False?
>
> read_up_to_prompt() returns True/False as the first member of the tuple.
>
>>> + # Send ctrl+c (SIGINT) through ssh session
>>> + session.sendline("\003")
>
> I think you can use send() here. sendline() calls send() with a newline
> added to the string.
>
>>> + status, output2 = session.read_up_to_prompt(print_func=output_func)
>>> + output += output2
>>> + if status is False:
>>> + # We also need to use this session to query the return value
>>> + session.sendline("\003")
>
> Same here.
>
>>> +
>>> + session.sendline(session.status_test_command)
>>> + s2, o2 = session.read_up_to_prompt()
>>> + if s2 is False:
>>
>> ^ See comment above
>>
>>> + status = -1
>>> + else:
>>> + try:
>>> + status = int(re.findall("\d+", o2)[0])
>>> + except:
>>> + status = -1
>>> +
>>> + return status, output
>>> +
>>> +def ping(dest = "localhost", count = None, interval = None, interface = None,
>>> + packetsize = None, ttl = None, hint = None, adaptive = False,
>>> + broadcast = False, flood = False, timeout = 0,
>>> + output_func = logging.debug, session = "localhost"):
>>
>> ^ On function headers, we pretty much follow PEP 8 and don't use spacing
>> between argument keys, the equal sign and the default value, so this
>> header should be rewritten
>>
>> def ping(dest="localhost",...)
>>
>>> + """
>>> + Wrapper of ping.
>>> +
>>> + @param dest: destination address
>>> + @param count: count of icmp packet
>>> + @param interval: interval of two icmp echo request
>>> + @param interface: specified interface of the source address
>>> + @param packetsize: packet size of icmp
>>> + @param ttl: ip time to live
>>> + @param hint: path mtu discovery hint
>>> + @param adaptive: adaptive ping flag
>>> + @param broadcast: broadcast ping flag
>>> + @param flood: flood ping flag
>>> + @param timeout: timeout for the ping command
>>> + @param output_func: function used to log the result of ping
>>> + @param session: local executon hint or session to execute the ping command
>
> Don't we need this for Windows as well? If we do, wouldn't it be easier
> to use a parameter (e.g. 'ping_options = -c 3') to directly control the
> ping options from the config file, instead of using an OS-specific ping
> wrapper?
Please disregard this comment. After looking at some of the other
patches in the set I realized it wasn't relevant.
>>> + """
>>> +
>>> + command = "ping %s " % dest
>>> +
>>> + if count is not None:
>>> + command += " -c %s" % count
>>> + if interval is not None:
>>> + command += " -i %s" % interval
>>> + if interface is not None:
>>> + command += " -I %s" % interface
>>> + if packetsize is not None:
>>> + command += " -s %s" % packetsize
>>> + if ttl is not None:
>>> + command += " -t %s" % ttl
>>> + if hint is not None:
>>> + command += " -M %s" % hint
>>> + if adaptive is True:
>>> + command += " -A"
>>> + if broadcast is True:
>>> + command += " -b"
>>> + if flood is True:
>>> + # temporary workaround as the kvm_subprocess may not properly handle
>>> + # the timeout for the output of flood ping
>
> What do you mean by this? If there's a problem with kvm_subprocess it
> should be fixed. Have you tried passing internal_timeout=0 to the
> kvm_subprocess function you're using?
>
>>> + command += " -f -q"
>
> If a lot of output is generated, it may not be a bad idea to use -q here
> regardless of kvm_subprocess limitations.
>
>>> + output_func = None
>>
>> ^ the last 3 if clauses could be simpler if put as:
>>
>> if adaptive:
>>
>> if broadcast:
>>
>> if flood:
>>
>>> + return raw_ping(command, timeout, session, output_func)
>>>
>>> def get_linux_ifname(session, mac_address):
>>> """
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 03/14] KVM Test: Add a common ping module for network related tests
@ 2010-07-28 13:56 ` Michael Goldish
0 siblings, 0 replies; 77+ messages in thread
From: Michael Goldish @ 2010-07-28 13:56 UTC (permalink / raw)
To: Amos Kong; +Cc: Lucas Meneghel Rodrigues, autotest, qemu-devel, kvm
On 07/28/2010 02:50 PM, Michael Goldish wrote:
> On 07/27/2010 04:01 PM, Lucas Meneghel Rodrigues wrote:
>> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
>>> The kvm_net_utils.py is a just a place that wraps common network
>>> related commands which is used to do the network-related tests.
>>> Use -1 as the packet ratio for loss analysis.
>>> Use quiet mode when doing the flood ping.
>>>
>>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>>> Signed-off-by: Amos Kong <akong@redhat.com>
>>> ---
>>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/client/tests/kvm/kvm_net_utils.py b/client/tests/kvm/kvm_net_utils.py
>>> index ede4965..8a71858 100644
>>> --- a/client/tests/kvm/kvm_net_utils.py
>>> +++ b/client/tests/kvm/kvm_net_utils.py
>>> @@ -1,4 +1,114 @@
>>> -import re
>>> +import logging, re, signal
>>> +from autotest_lib.client.common_lib import error
>>> +import kvm_subprocess, kvm_utils
>>> +
>>> +def get_loss_ratio(output):
>>> + """
>>> + Get the packet loss ratio from the output of ping
>>> +
>>> + @param output
>>> + """
>>> + try:
>>> + return int(re.findall('(\d+)% packet loss', output)[0])
>>> + except IndexError:
>>> + logging.debug(output)
>>> + return -1
>>
>>> +def raw_ping(command, timeout, session, output_func):
>>> + """
>>> + Low-level ping command execution.
>>> +
>>> + @param command: ping command
>>> + @param timeout: timeout of the ping command
>>> + @param session: local executon hint or session to execute the ping command
>>> + """
>>> + if session == "localhost":
>
> I have no problem with this, but wouldn't it be nicer to use None to
> indicate that the session should be local?
>
>>> + process = kvm_subprocess.run_bg(command, output_func=output_func,
>>> + timeout=timeout)
>>
>> Do we really have to resort to kvm_subprocess here? We use subprocess on
>> special occasions, usually when the command in question is required to
>> live throughout the tests, which doesn't seem to be the case.
>> kvm_subprocess is a good library, but it is a more heavyweight
>> alternative (spawns its own shell process, for example). Maybe you are
>> using it because you can directly send input to the process at any time,
>> such as the ctrl+c later on this same code.
>>
>>> +
>>> + # Send SIGINT singal to notify the timeout of running ping process,
>>
>> ^ typo, signal
>>
>>> + # Because ping have the ability to catch the SIGINT signal so we can
>>> + # always get the packet loss ratio even if timeout.
>>> + if process.is_alive():
>>> + kvm_utils.kill_process_tree(process.get_pid(), signal.SIGINT)
>>> +
>>> + status = process.get_status()
>>> + output = process.get_output()
>>> +
>>> + process.close()
>>> + return status, output
>>> + else:
>>> + session.sendline(command)
>>> + status, output = session.read_up_to_prompt(timeout=timeout,
>>> + print_func=output_func)
>>> + if status is False:
>>
>> ^ For None objects, it is better to explicitly test for is None. However
>> the above construct seems more natural if put as
>>
>> if not status:
>>
>> Any particular reason you tested explicitely for False?
>
> read_up_to_prompt() returns True/False as the first member of the tuple.
>
>>> + # Send ctrl+c (SIGINT) through ssh session
>>> + session.sendline("\003")
>
> I think you can use send() here. sendline() calls send() with a newline
> added to the string.
>
>>> + status, output2 = session.read_up_to_prompt(print_func=output_func)
>>> + output += output2
>>> + if status is False:
>>> + # We also need to use this session to query the return value
>>> + session.sendline("\003")
>
> Same here.
>
>>> +
>>> + session.sendline(session.status_test_command)
>>> + s2, o2 = session.read_up_to_prompt()
>>> + if s2 is False:
>>
>> ^ See comment above
>>
>>> + status = -1
>>> + else:
>>> + try:
>>> + status = int(re.findall("\d+", o2)[0])
>>> + except:
>>> + status = -1
>>> +
>>> + return status, output
>>> +
>>> +def ping(dest = "localhost", count = None, interval = None, interface = None,
>>> + packetsize = None, ttl = None, hint = None, adaptive = False,
>>> + broadcast = False, flood = False, timeout = 0,
>>> + output_func = logging.debug, session = "localhost"):
>>
>> ^ On function headers, we pretty much follow PEP 8 and don't use spacing
>> between argument keys, the equal sign and the default value, so this
>> header should be rewritten
>>
>> def ping(dest="localhost",...)
>>
>>> + """
>>> + Wrapper of ping.
>>> +
>>> + @param dest: destination address
>>> + @param count: count of icmp packet
>>> + @param interval: interval of two icmp echo request
>>> + @param interface: specified interface of the source address
>>> + @param packetsize: packet size of icmp
>>> + @param ttl: ip time to live
>>> + @param hint: path mtu discovery hint
>>> + @param adaptive: adaptive ping flag
>>> + @param broadcast: broadcast ping flag
>>> + @param flood: flood ping flag
>>> + @param timeout: timeout for the ping command
>>> + @param output_func: function used to log the result of ping
>>> + @param session: local executon hint or session to execute the ping command
>
> Don't we need this for Windows as well? If we do, wouldn't it be easier
> to use a parameter (e.g. 'ping_options = -c 3') to directly control the
> ping options from the config file, instead of using an OS-specific ping
> wrapper?
Please disregard this comment. After looking at some of the other
patches in the set I realized it wasn't relevant.
>>> + """
>>> +
>>> + command = "ping %s " % dest
>>> +
>>> + if count is not None:
>>> + command += " -c %s" % count
>>> + if interval is not None:
>>> + command += " -i %s" % interval
>>> + if interface is not None:
>>> + command += " -I %s" % interface
>>> + if packetsize is not None:
>>> + command += " -s %s" % packetsize
>>> + if ttl is not None:
>>> + command += " -t %s" % ttl
>>> + if hint is not None:
>>> + command += " -M %s" % hint
>>> + if adaptive is True:
>>> + command += " -A"
>>> + if broadcast is True:
>>> + command += " -b"
>>> + if flood is True:
>>> + # temporary workaround as the kvm_subprocess may not properly handle
>>> + # the timeout for the output of flood ping
>
> What do you mean by this? If there's a problem with kvm_subprocess it
> should be fixed. Have you tried passing internal_timeout=0 to the
> kvm_subprocess function you're using?
>
>>> + command += " -f -q"
>
> If a lot of output is generated, it may not be a bad idea to use -q here
> regardless of kvm_subprocess limitations.
>
>>> + output_func = None
>>
>> ^ the last 3 if clauses could be simpler if put as:
>>
>> if adaptive:
>>
>> if broadcast:
>>
>> if flood:
>>
>>> + return raw_ping(command, timeout, session, output_func)
>>>
>>> def get_linux_ifname(session, mac_address):
>>> """
^ permalink raw reply [flat|nested] 77+ messages in thread
* [RFC PATCH 04/14] KVM-test: Add a new subtest ping
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:35 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
This test use ping to check the virtual nics, it contains two kinds of test:
1. Packet loss ratio test, ping the guest with different size of packets.
2. Stress test, flood ping guest then use ordinary ping to test the network.
The interval and packet size could be configurated through tests_base.cfg
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/ping.py b/client/tests/kvm/tests/ping.py
new file mode 100644
index 0000000..cfccda4
--- /dev/null
+++ b/client/tests/kvm/tests/ping.py
@@ -0,0 +1,71 @@
+import logging, time, re, commands
+from autotest_lib.client.common_lib import error
+import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
+
+
+def run_ping(test, params, env):
+ """
+ Ping the guest with different size of packets.
+
+ Packet Loss Test:
+ 1) Ping the guest with different size/interval of packets.
+ Stress Test:
+ 1) Flood ping the guest.
+ 2) Check if the network is still usable.
+
+ @param test: Kvm test object
+ @param params: Dictionary with the test parameters
+ @param env: Dictionary with test environment.
+ """
+
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm)
+
+ counts = params.get("ping_counts", 100)
+ flood_minutes = float(params.get("flood_minutes", 10))
+ nics = params.get("nics").split()
+ strict_check = params.get("strict_check", "no") == "yes"
+
+ packet_size = [0, 1, 4, 48, 512, 1440, 1500, 1505, 4054, 4055, 4096, 4192,
+ 8878, 9000, 32767, 65507]
+
+ try:
+ for i, nic in enumerate(nics):
+ ip = vm.get_address(i)
+ if not ip:
+ logging.error("Could not get the ip of nic index %d" % i)
+ continue
+
+ for size in packet_size:
+ logging.info("Ping with packet size %s" % size)
+ status, output = kvm_net_utils.ping(ip, 10,
+ packetsize = size,
+ timeout = 20)
+ if strict_check:
+ ratio = kvm_net_utils.get_loss_ratio(output)
+ if ratio != 0:
+ raise error.TestFail(" Loss ratio is %s for packet size"
+ " %s" % (ratio, size))
+ else:
+ if status != 0:
+ raise error.TestFail(" Ping returns non-zero value %s" %
+ output)
+
+ logging.info("Flood ping test")
+ kvm_net_utils.ping(ip, None, flood = True, output_func= None,
+ timeout = flood_minutes * 60)
+
+ logging.info("Final ping test")
+ status, output = kvm_net_utils.ping(ip, counts,
+ timeout = float(counts) * 1.5)
+ if strict_check:
+ ratio = kvm_net_utils.get_loss_ratio(output)
+ if ratio != 0:
+ raise error.TestFail("Packet loss ratio is %s after flood"
+ % ratio)
+ else:
+ if status != 0:
+ raise error.TestFail(" Ping returns non-zero value %s" %
+ output)
+ finally:
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 6710c00..4f58dc0 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -349,6 +349,11 @@ variants:
kill_vm_gracefully_vm2 = no
address_index_vm2 = 1
+ - ping: install setup unattended_install.cdrom
+ type = ping
+ counts = 100
+ flood_minutes = 10
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 04/14] KVM-test: Add a new subtest ping
@ 2010-07-20 1:35 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
This test use ping to check the virtual nics, it contains two kinds of test:
1. Packet loss ratio test, ping the guest with different size of packets.
2. Stress test, flood ping guest then use ordinary ping to test the network.
The interval and packet size could be configurated through tests_base.cfg
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/ping.py b/client/tests/kvm/tests/ping.py
new file mode 100644
index 0000000..cfccda4
--- /dev/null
+++ b/client/tests/kvm/tests/ping.py
@@ -0,0 +1,71 @@
+import logging, time, re, commands
+from autotest_lib.client.common_lib import error
+import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
+
+
+def run_ping(test, params, env):
+ """
+ Ping the guest with different size of packets.
+
+ Packet Loss Test:
+ 1) Ping the guest with different size/interval of packets.
+ Stress Test:
+ 1) Flood ping the guest.
+ 2) Check if the network is still usable.
+
+ @param test: Kvm test object
+ @param params: Dictionary with the test parameters
+ @param env: Dictionary with test environment.
+ """
+
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm)
+
+ counts = params.get("ping_counts", 100)
+ flood_minutes = float(params.get("flood_minutes", 10))
+ nics = params.get("nics").split()
+ strict_check = params.get("strict_check", "no") == "yes"
+
+ packet_size = [0, 1, 4, 48, 512, 1440, 1500, 1505, 4054, 4055, 4096, 4192,
+ 8878, 9000, 32767, 65507]
+
+ try:
+ for i, nic in enumerate(nics):
+ ip = vm.get_address(i)
+ if not ip:
+ logging.error("Could not get the ip of nic index %d" % i)
+ continue
+
+ for size in packet_size:
+ logging.info("Ping with packet size %s" % size)
+ status, output = kvm_net_utils.ping(ip, 10,
+ packetsize = size,
+ timeout = 20)
+ if strict_check:
+ ratio = kvm_net_utils.get_loss_ratio(output)
+ if ratio != 0:
+ raise error.TestFail(" Loss ratio is %s for packet size"
+ " %s" % (ratio, size))
+ else:
+ if status != 0:
+ raise error.TestFail(" Ping returns non-zero value %s" %
+ output)
+
+ logging.info("Flood ping test")
+ kvm_net_utils.ping(ip, None, flood = True, output_func= None,
+ timeout = flood_minutes * 60)
+
+ logging.info("Final ping test")
+ status, output = kvm_net_utils.ping(ip, counts,
+ timeout = float(counts) * 1.5)
+ if strict_check:
+ ratio = kvm_net_utils.get_loss_ratio(output)
+ if ratio != 0:
+ raise error.TestFail("Packet loss ratio is %s after flood"
+ % ratio)
+ else:
+ if status != 0:
+ raise error.TestFail(" Ping returns non-zero value %s" %
+ output)
+ finally:
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 6710c00..4f58dc0 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -349,6 +349,11 @@ variants:
kill_vm_gracefully_vm2 = no
address_index_vm2 = 1
+ - ping: install setup unattended_install.cdrom
+ type = ping
+ counts = 100
+ flood_minutes = 10
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [RFC PATCH 04/14] KVM-test: Add a new subtest ping
2010-07-20 1:35 ` [Qemu-devel] " Amos Kong
@ 2010-07-27 13:15 ` Lucas Meneghel Rodrigues
-1 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-27 13:15 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> This test use ping to check the virtual nics, it contains two kinds of test:
> 1. Packet loss ratio test, ping the guest with different size of packets.
> 2. Stress test, flood ping guest then use ordinary ping to test the network.
>
> The interval and packet size could be configurated through tests_base.cfg
>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/ping.py b/client/tests/kvm/tests/ping.py
> new file mode 100644
> index 0000000..cfccda4
> --- /dev/null
> +++ b/client/tests/kvm/tests/ping.py
> @@ -0,0 +1,71 @@
> +import logging, time, re, commands
> +from autotest_lib.client.common_lib import error
> +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
> +
> +
> +def run_ping(test, params, env):
> + """
> + Ping the guest with different size of packets.
> +
> + Packet Loss Test:
> + 1) Ping the guest with different size/interval of packets.
> + Stress Test:
> + 1) Flood ping the guest.
> + 2) Check if the network is still usable.
> +
> + @param test: Kvm test object
> + @param params: Dictionary with the test parameters
> + @param env: Dictionary with test environment.
> + """
> +
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + session = kvm_test_utils.wait_for_login(vm)
> +
> + counts = params.get("ping_counts", 100)
> + flood_minutes = float(params.get("flood_minutes", 10))
> + nics = params.get("nics").split()
> + strict_check = params.get("strict_check", "no") == "yes"
> +
> + packet_size = [0, 1, 4, 48, 512, 1440, 1500, 1505, 4054, 4055, 4096, 4192,
> + 8878, 9000, 32767, 65507]
> +
> + try:
> + for i, nic in enumerate(nics):
> + ip = vm.get_address(i)
> + if not ip:
> + logging.error("Could not get the ip of nic index %d" % i)
> + continue
> +
> + for size in packet_size:
> + logging.info("Ping with packet size %s" % size)
> + status, output = kvm_net_utils.ping(ip, 10,
> + packetsize = size,
> + timeout = 20)
> + if strict_check:
> + ratio = kvm_net_utils.get_loss_ratio(output)
> + if ratio != 0:
> + raise error.TestFail(" Loss ratio is %s for packet size"
> + " %s" % (ratio, size))
> + else:
> + if status != 0:
> + raise error.TestFail(" Ping returns non-zero value %s" %
> + output)
^ "Ping failed, status: %s, output: %s" would be a better exception
message.
> +
> + logging.info("Flood ping test")
> + kvm_net_utils.ping(ip, None, flood = True, output_func= None,
> + timeout = flood_minutes * 60)
^ Please get rid of the spaces here
> + logging.info("Final ping test")
> + status, output = kvm_net_utils.ping(ip, counts,
> + timeout = float(counts) * 1.5)
^ What is this last test for again?
> + if strict_check:
> + ratio = kvm_net_utils.get_loss_ratio(output)
> + if ratio != 0:
> + raise error.TestFail("Packet loss ratio is %s after flood"
> + % ratio)
> + else:
> + if status != 0:
> + raise error.TestFail(" Ping returns non-zero value %s" %
> + output)
> + finally:
> + session.close()
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 6710c00..4f58dc0 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -349,6 +349,11 @@ variants:
> kill_vm_gracefully_vm2 = no
> address_index_vm2 = 1
>
> + - ping: install setup unattended_install.cdrom
> + type = ping
> + counts = 100
> + flood_minutes = 10
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 77+ messages in thread* [Qemu-devel] Re: [RFC PATCH 04/14] KVM-test: Add a new subtest ping
@ 2010-07-27 13:15 ` Lucas Meneghel Rodrigues
0 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-27 13:15 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> This test use ping to check the virtual nics, it contains two kinds of test:
> 1. Packet loss ratio test, ping the guest with different size of packets.
> 2. Stress test, flood ping guest then use ordinary ping to test the network.
>
> The interval and packet size could be configurated through tests_base.cfg
>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/ping.py b/client/tests/kvm/tests/ping.py
> new file mode 100644
> index 0000000..cfccda4
> --- /dev/null
> +++ b/client/tests/kvm/tests/ping.py
> @@ -0,0 +1,71 @@
> +import logging, time, re, commands
> +from autotest_lib.client.common_lib import error
> +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
> +
> +
> +def run_ping(test, params, env):
> + """
> + Ping the guest with different size of packets.
> +
> + Packet Loss Test:
> + 1) Ping the guest with different size/interval of packets.
> + Stress Test:
> + 1) Flood ping the guest.
> + 2) Check if the network is still usable.
> +
> + @param test: Kvm test object
> + @param params: Dictionary with the test parameters
> + @param env: Dictionary with test environment.
> + """
> +
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + session = kvm_test_utils.wait_for_login(vm)
> +
> + counts = params.get("ping_counts", 100)
> + flood_minutes = float(params.get("flood_minutes", 10))
> + nics = params.get("nics").split()
> + strict_check = params.get("strict_check", "no") == "yes"
> +
> + packet_size = [0, 1, 4, 48, 512, 1440, 1500, 1505, 4054, 4055, 4096, 4192,
> + 8878, 9000, 32767, 65507]
> +
> + try:
> + for i, nic in enumerate(nics):
> + ip = vm.get_address(i)
> + if not ip:
> + logging.error("Could not get the ip of nic index %d" % i)
> + continue
> +
> + for size in packet_size:
> + logging.info("Ping with packet size %s" % size)
> + status, output = kvm_net_utils.ping(ip, 10,
> + packetsize = size,
> + timeout = 20)
> + if strict_check:
> + ratio = kvm_net_utils.get_loss_ratio(output)
> + if ratio != 0:
> + raise error.TestFail(" Loss ratio is %s for packet size"
> + " %s" % (ratio, size))
> + else:
> + if status != 0:
> + raise error.TestFail(" Ping returns non-zero value %s" %
> + output)
^ "Ping failed, status: %s, output: %s" would be a better exception
message.
> +
> + logging.info("Flood ping test")
> + kvm_net_utils.ping(ip, None, flood = True, output_func= None,
> + timeout = flood_minutes * 60)
^ Please get rid of the spaces here
> + logging.info("Final ping test")
> + status, output = kvm_net_utils.ping(ip, counts,
> + timeout = float(counts) * 1.5)
^ What is this last test for again?
> + if strict_check:
> + ratio = kvm_net_utils.get_loss_ratio(output)
> + if ratio != 0:
> + raise error.TestFail("Packet loss ratio is %s after flood"
> + % ratio)
> + else:
> + if status != 0:
> + raise error.TestFail(" Ping returns non-zero value %s" %
> + output)
> + finally:
> + session.close()
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 6710c00..4f58dc0 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -349,6 +349,11 @@ variants:
> kill_vm_gracefully_vm2 = no
> address_index_vm2 = 1
>
> + - ping: install setup unattended_install.cdrom
> + type = ping
> + counts = 100
> + flood_minutes = 10
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [RFC PATCH 04/14] KVM-test: Add a new subtest ping
2010-07-27 13:15 ` [Qemu-devel] " Lucas Meneghel Rodrigues
@ 2010-08-03 1:54 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-03 1:54 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: autotest, qemu-devel, kvm
On Tue, Jul 27, 2010 at 10:15:49AM -0300, Lucas Meneghel Rodrigues wrote:
> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> > This test use ping to check the virtual nics, it contains two kinds of test:
> > 1. Packet loss ratio test, ping the guest with different size of packets.
> > 2. Stress test, flood ping guest then use ordinary ping to test the network.
> >
> > The interval and packet size could be configurated through tests_base.cfg
> >
> > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > Signed-off-by: Amos Kong <akong@redhat.com>
> > ---
> > 0 files changed, 0 insertions(+), 0 deletions(-)
> >
> > diff --git a/client/tests/kvm/tests/ping.py b/client/tests/kvm/tests/ping.py
> > new file mode 100644
> > index 0000000..cfccda4
> > --- /dev/null
> > +++ b/client/tests/kvm/tests/ping.py
> > @@ -0,0 +1,71 @@
> > +import logging, time, re, commands
> > +from autotest_lib.client.common_lib import error
> > +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
> > +
> > +
> > +def run_ping(test, params, env):
> > + """
> > + Ping the guest with different size of packets.
> > +
> > + Packet Loss Test:
> > + 1) Ping the guest with different size/interval of packets.
> > + Stress Test:
> > + 1) Flood ping the guest.
> > + 2) Check if the network is still usable.
> > +
> > + @param test: Kvm test object
> > + @param params: Dictionary with the test parameters
> > + @param env: Dictionary with test environment.
> > + """
> > +
> > + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> > + session = kvm_test_utils.wait_for_login(vm)
> > +
> > + counts = params.get("ping_counts", 100)
> > + flood_minutes = float(params.get("flood_minutes", 10))
> > + nics = params.get("nics").split()
> > + strict_check = params.get("strict_check", "no") == "yes"
> > +
> > + packet_size = [0, 1, 4, 48, 512, 1440, 1500, 1505, 4054, 4055, 4096, 4192,
> > + 8878, 9000, 32767, 65507]
> > +
> > + try:
> > + for i, nic in enumerate(nics):
> > + ip = vm.get_address(i)
> > + if not ip:
> > + logging.error("Could not get the ip of nic index %d" % i)
> > + continue
> > +
> > + for size in packet_size:
> > + logging.info("Ping with packet size %s" % size)
> > + status, output = kvm_net_utils.ping(ip, 10,
> > + packetsize = size,
> > + timeout = 20)
> > + if strict_check:
> > + ratio = kvm_net_utils.get_loss_ratio(output)
> > + if ratio != 0:
> > + raise error.TestFail(" Loss ratio is %s for packet size"
> > + " %s" % (ratio, size))
> > + else:
> > + if status != 0:
> > + raise error.TestFail(" Ping returns non-zero value %s" %
> > + output)
>
> ^ "Ping failed, status: %s, output: %s" would be a better exception
> message.
>
> > +
> > + logging.info("Flood ping test")
> > + kvm_net_utils.ping(ip, None, flood = True, output_func= None,
> > + timeout = flood_minutes * 60)
>
> ^ Please get rid of the spaces here
>
> > + logging.info("Final ping test")
> > + status, output = kvm_net_utils.ping(ip, counts,
> > + timeout = float(counts) * 1.5)
>
> ^ What is this last test for again?
We could not raise an error when flood ping failed(loss packet/ return non-zero), it's too strict.
But we must check the ping result before/after floop-ping.
>
> > + if strict_check:
> > + ratio = kvm_net_utils.get_loss_ratio(output)
> > + if ratio != 0:
> > + raise error.TestFail("Packet loss ratio is %s after flood"
> > + % ratio)
> > + else:
> > + if status != 0:
> > + raise error.TestFail(" Ping returns non-zero value %s" %
> > + output)
> > + finally:
> > + session.close()
> > diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> > index 6710c00..4f58dc0 100644
> > --- a/client/tests/kvm/tests_base.cfg.sample
> > +++ b/client/tests/kvm/tests_base.cfg.sample
> > @@ -349,6 +349,11 @@ variants:
> > kill_vm_gracefully_vm2 = no
> > address_index_vm2 = 1
> >
> > + - ping: install setup unattended_install.cdrom
> > + type = ping
> > + counts = 100
> > + flood_minutes = 10
> > +
> > - physical_resources_check: install setup unattended_install.cdrom
> > type = physical_resources_check
> > catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread* [Qemu-devel] Re: [RFC PATCH 04/14] KVM-test: Add a new subtest ping
@ 2010-08-03 1:54 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-03 1:54 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: autotest, qemu-devel, kvm
On Tue, Jul 27, 2010 at 10:15:49AM -0300, Lucas Meneghel Rodrigues wrote:
> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> > This test use ping to check the virtual nics, it contains two kinds of test:
> > 1. Packet loss ratio test, ping the guest with different size of packets.
> > 2. Stress test, flood ping guest then use ordinary ping to test the network.
> >
> > The interval and packet size could be configurated through tests_base.cfg
> >
> > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > Signed-off-by: Amos Kong <akong@redhat.com>
> > ---
> > 0 files changed, 0 insertions(+), 0 deletions(-)
> >
> > diff --git a/client/tests/kvm/tests/ping.py b/client/tests/kvm/tests/ping.py
> > new file mode 100644
> > index 0000000..cfccda4
> > --- /dev/null
> > +++ b/client/tests/kvm/tests/ping.py
> > @@ -0,0 +1,71 @@
> > +import logging, time, re, commands
> > +from autotest_lib.client.common_lib import error
> > +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
> > +
> > +
> > +def run_ping(test, params, env):
> > + """
> > + Ping the guest with different size of packets.
> > +
> > + Packet Loss Test:
> > + 1) Ping the guest with different size/interval of packets.
> > + Stress Test:
> > + 1) Flood ping the guest.
> > + 2) Check if the network is still usable.
> > +
> > + @param test: Kvm test object
> > + @param params: Dictionary with the test parameters
> > + @param env: Dictionary with test environment.
> > + """
> > +
> > + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> > + session = kvm_test_utils.wait_for_login(vm)
> > +
> > + counts = params.get("ping_counts", 100)
> > + flood_minutes = float(params.get("flood_minutes", 10))
> > + nics = params.get("nics").split()
> > + strict_check = params.get("strict_check", "no") == "yes"
> > +
> > + packet_size = [0, 1, 4, 48, 512, 1440, 1500, 1505, 4054, 4055, 4096, 4192,
> > + 8878, 9000, 32767, 65507]
> > +
> > + try:
> > + for i, nic in enumerate(nics):
> > + ip = vm.get_address(i)
> > + if not ip:
> > + logging.error("Could not get the ip of nic index %d" % i)
> > + continue
> > +
> > + for size in packet_size:
> > + logging.info("Ping with packet size %s" % size)
> > + status, output = kvm_net_utils.ping(ip, 10,
> > + packetsize = size,
> > + timeout = 20)
> > + if strict_check:
> > + ratio = kvm_net_utils.get_loss_ratio(output)
> > + if ratio != 0:
> > + raise error.TestFail(" Loss ratio is %s for packet size"
> > + " %s" % (ratio, size))
> > + else:
> > + if status != 0:
> > + raise error.TestFail(" Ping returns non-zero value %s" %
> > + output)
>
> ^ "Ping failed, status: %s, output: %s" would be a better exception
> message.
>
> > +
> > + logging.info("Flood ping test")
> > + kvm_net_utils.ping(ip, None, flood = True, output_func= None,
> > + timeout = flood_minutes * 60)
>
> ^ Please get rid of the spaces here
>
> > + logging.info("Final ping test")
> > + status, output = kvm_net_utils.ping(ip, counts,
> > + timeout = float(counts) * 1.5)
>
> ^ What is this last test for again?
We could not raise an error when flood ping failed(loss packet/ return non-zero), it's too strict.
But we must check the ping result before/after floop-ping.
>
> > + if strict_check:
> > + ratio = kvm_net_utils.get_loss_ratio(output)
> > + if ratio != 0:
> > + raise error.TestFail("Packet loss ratio is %s after flood"
> > + % ratio)
> > + else:
> > + if status != 0:
> > + raise error.TestFail(" Ping returns non-zero value %s" %
> > + output)
> > + finally:
> > + session.close()
> > diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> > index 6710c00..4f58dc0 100644
> > --- a/client/tests/kvm/tests_base.cfg.sample
> > +++ b/client/tests/kvm/tests_base.cfg.sample
> > @@ -349,6 +349,11 @@ variants:
> > kill_vm_gracefully_vm2 = no
> > address_index_vm2 = 1
> >
> > + - ping: install setup unattended_install.cdrom
> > + type = ping
> > + counts = 100
> > + flood_minutes = 10
> > +
> > - physical_resources_check: install setup unattended_install.cdrom
> > type = physical_resources_check
> > catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [RFC PATCH 05/14] KVM-test: Add a subtest jumbo
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:35 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
According to different nic model set different MTU for it. And ping from guest
to host, to see whether tested size can be received by host.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/jumbo.py b/client/tests/kvm/tests/jumbo.py
new file mode 100644
index 0000000..9f56a87
--- /dev/null
+++ b/client/tests/kvm/tests/jumbo.py
@@ -0,0 +1,133 @@
+import os, re, logging, commands, time, random
+from autotest_lib.client.common_lib import error
+import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
+
+def run_jumbo(test, params, env):
+ """
+ Test the RX jumbo frame function of vnics:
+ 1) boot the vm
+ 2) change the MTU of guest nics and host taps depends on the nic model
+ 3) add the static arp entry for guest nic
+ 4) wait for the MTU ok
+ 5) verify the patch mtu using ping
+ 6) ping the guest with large frames
+ 7) increament size ping
+ 8) flood ping the guest with large frames
+ 9) verify the path mtu
+ 10) revocer the mtu
+
+ @param test: kvm test object
+ @param params: Dictionary with the test parameters
+ @param env: Dictionary with test environment.
+ """
+
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm)
+ mtu = params.get("mtu", "1500")
+ flood_time = params.get("flood_time", "300")
+ max_icmp_pkt_size = int(mtu) - 28
+
+ ifname = vm.get_ifname(0)
+ ip = vm.get_address(0)
+ if ip is None:
+ raise error.TestError("Could not get the ip address")
+
+ try:
+ # Environment preparartion
+ ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
+
+ logging.info("Changing the mtu of guest ...")
+ guest_mtu_cmd = "ifconfig %s mtu %s" % (ethname , mtu)
+ s, o = session.get_command_status_output(guest_mtu_cmd)
+ if s != 0:
+ logging.error(o)
+ raise error.TestError("Fail to set the mtu of guest nic: %s"
+ % ethname)
+
+ logging.info("Chaning the mtu of host tap ...")
+ host_mtu_cmd = "ifconfig %s mtu %s" % (ifname, mtu)
+ s, o = commands.getstatusoutput(host_mtu_cmd)
+ if s != 0:
+ raise error.TestError("Fail to set the mtu of %s" % ifname)
+
+ logging.info("Add a temporary static arp entry ...")
+ arp_add_cmd = "arp -s %s %s -i %s" % (ip, vm.get_macaddr(0), ifname)
+ s, o = commands.getstatusoutput(arp_add_cmd)
+ if s != 0 :
+ raise error.TestError("Fail to add temporary arp entry")
+
+ def is_mtu_ok():
+ s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
+ packetsize = max_icmp_pkt_size,
+ hint = "do", timeout = 2)
+ if s != 0:
+ return False
+ else:
+ return True
+
+ def verify_mtu():
+ logging.info("Verify the path mtu")
+ s, o = kvm_net_utils.ping(ip, 10, interface = ifname,
+ packetsize = max_icmp_pkt_size,
+ hint = "do", timeout = 15)
+ if s != 0 :
+ logging.error(o)
+ raise error.TestFail("Path MTU is not as expected")
+ if kvm_net_utils.get_loss_ratio(o) != 0:
+ logging.error(o)
+ raise error.TestFail("Packet loss ratio during mtu verification"
+ " is not zero")
+
+ def flood_ping():
+ logging.info("Flood with large frames")
+ kvm_net_utils.ping(ip, interface = ifname,
+ packetsize = max_icmp_pkt_size,
+ flood = True, timeout = float(flood_time))
+
+ def large_frame_ping(count = 100):
+ logging.info("Large frame ping")
+ s, o = kvm_net_utils.ping(ip, count, interface = ifname,
+ packetsize = max_icmp_pkt_size,
+ timeout = float(count) * 2)
+ ratio = kvm_net_utils.get_loss_ratio(o)
+ if ratio != 0:
+ raise error.TestFail("Loss ratio of large frame ping is %s" \
+ % ratio)
+
+ def size_increase_ping(step = random.randrange(90, 110)):
+ logging.info("Size increase ping")
+ for size in range(0, max_icmp_pkt_size + 1, step):
+ logging.info("Ping %s with size %s" % (ip, size))
+ s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
+ packetsize = size,
+ hint = "do", timeout = 1)
+ if s != 0:
+ s, o = kvm_net_utils.ping(ip, 10, interface = ifname,
+ packetsize = size,
+ adaptive = True, hint = "do",
+ timeout = 20)
+ if kvm_net_utils.get_loss_ratio(o) > 50:
+ raise error.TestFail("ping loss ratio is greater "
+ "than 50% for size %s" % size)
+
+ logging.info("Waiting for the MTU to be OK")
+ # Wait for the MTU is OK
+ if not kvm_utils.wait_for(is_mtu_ok, 10, 0, 1):
+ logging.debug(commands.getoutput("ifconfig -a"))
+ raise error.TestError("10 seconds elapsed while the mtu is not"
+ " as expected")
+
+ # Functional Test
+ verify_mtu()
+ large_frame_ping()
+ size_increase_ping()
+
+ # Stress test
+ flood_ping()
+ verify_mtu()
+
+ finally:
+ # Environment clean
+ session.close()
+ logging.info("Removing the temporary arp entry")
+ commands.getstatusoutput("arp -d %s -i %s" % (ip, ifname))
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 4f58dc0..7f7b56a 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -354,6 +354,9 @@ variants:
counts = 100
flood_minutes = 10
+ - jumbo: install setup unattended_install.cdrom
+ type = jumbo
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -489,10 +492,16 @@ variants:
variants:
- @rtl8139:
nic_model = rtl8139
+ jumbo:
+ mtu = 1500
- e1000:
nic_model = e1000
+ jumbo:
+ mtu = 16110
- virtio:
nic_model = virtio
+ jumbo:
+ mtu = 65520
# Guests
@@ -1024,7 +1033,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks)
+ no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 05/14] KVM-test: Add a subtest jumbo
@ 2010-07-20 1:35 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
According to different nic model set different MTU for it. And ping from guest
to host, to see whether tested size can be received by host.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/jumbo.py b/client/tests/kvm/tests/jumbo.py
new file mode 100644
index 0000000..9f56a87
--- /dev/null
+++ b/client/tests/kvm/tests/jumbo.py
@@ -0,0 +1,133 @@
+import os, re, logging, commands, time, random
+from autotest_lib.client.common_lib import error
+import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
+
+def run_jumbo(test, params, env):
+ """
+ Test the RX jumbo frame function of vnics:
+ 1) boot the vm
+ 2) change the MTU of guest nics and host taps depends on the nic model
+ 3) add the static arp entry for guest nic
+ 4) wait for the MTU ok
+ 5) verify the patch mtu using ping
+ 6) ping the guest with large frames
+ 7) increament size ping
+ 8) flood ping the guest with large frames
+ 9) verify the path mtu
+ 10) revocer the mtu
+
+ @param test: kvm test object
+ @param params: Dictionary with the test parameters
+ @param env: Dictionary with test environment.
+ """
+
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm)
+ mtu = params.get("mtu", "1500")
+ flood_time = params.get("flood_time", "300")
+ max_icmp_pkt_size = int(mtu) - 28
+
+ ifname = vm.get_ifname(0)
+ ip = vm.get_address(0)
+ if ip is None:
+ raise error.TestError("Could not get the ip address")
+
+ try:
+ # Environment preparartion
+ ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
+
+ logging.info("Changing the mtu of guest ...")
+ guest_mtu_cmd = "ifconfig %s mtu %s" % (ethname , mtu)
+ s, o = session.get_command_status_output(guest_mtu_cmd)
+ if s != 0:
+ logging.error(o)
+ raise error.TestError("Fail to set the mtu of guest nic: %s"
+ % ethname)
+
+ logging.info("Chaning the mtu of host tap ...")
+ host_mtu_cmd = "ifconfig %s mtu %s" % (ifname, mtu)
+ s, o = commands.getstatusoutput(host_mtu_cmd)
+ if s != 0:
+ raise error.TestError("Fail to set the mtu of %s" % ifname)
+
+ logging.info("Add a temporary static arp entry ...")
+ arp_add_cmd = "arp -s %s %s -i %s" % (ip, vm.get_macaddr(0), ifname)
+ s, o = commands.getstatusoutput(arp_add_cmd)
+ if s != 0 :
+ raise error.TestError("Fail to add temporary arp entry")
+
+ def is_mtu_ok():
+ s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
+ packetsize = max_icmp_pkt_size,
+ hint = "do", timeout = 2)
+ if s != 0:
+ return False
+ else:
+ return True
+
+ def verify_mtu():
+ logging.info("Verify the path mtu")
+ s, o = kvm_net_utils.ping(ip, 10, interface = ifname,
+ packetsize = max_icmp_pkt_size,
+ hint = "do", timeout = 15)
+ if s != 0 :
+ logging.error(o)
+ raise error.TestFail("Path MTU is not as expected")
+ if kvm_net_utils.get_loss_ratio(o) != 0:
+ logging.error(o)
+ raise error.TestFail("Packet loss ratio during mtu verification"
+ " is not zero")
+
+ def flood_ping():
+ logging.info("Flood with large frames")
+ kvm_net_utils.ping(ip, interface = ifname,
+ packetsize = max_icmp_pkt_size,
+ flood = True, timeout = float(flood_time))
+
+ def large_frame_ping(count = 100):
+ logging.info("Large frame ping")
+ s, o = kvm_net_utils.ping(ip, count, interface = ifname,
+ packetsize = max_icmp_pkt_size,
+ timeout = float(count) * 2)
+ ratio = kvm_net_utils.get_loss_ratio(o)
+ if ratio != 0:
+ raise error.TestFail("Loss ratio of large frame ping is %s" \
+ % ratio)
+
+ def size_increase_ping(step = random.randrange(90, 110)):
+ logging.info("Size increase ping")
+ for size in range(0, max_icmp_pkt_size + 1, step):
+ logging.info("Ping %s with size %s" % (ip, size))
+ s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
+ packetsize = size,
+ hint = "do", timeout = 1)
+ if s != 0:
+ s, o = kvm_net_utils.ping(ip, 10, interface = ifname,
+ packetsize = size,
+ adaptive = True, hint = "do",
+ timeout = 20)
+ if kvm_net_utils.get_loss_ratio(o) > 50:
+ raise error.TestFail("ping loss ratio is greater "
+ "than 50% for size %s" % size)
+
+ logging.info("Waiting for the MTU to be OK")
+ # Wait for the MTU is OK
+ if not kvm_utils.wait_for(is_mtu_ok, 10, 0, 1):
+ logging.debug(commands.getoutput("ifconfig -a"))
+ raise error.TestError("10 seconds elapsed while the mtu is not"
+ " as expected")
+
+ # Functional Test
+ verify_mtu()
+ large_frame_ping()
+ size_increase_ping()
+
+ # Stress test
+ flood_ping()
+ verify_mtu()
+
+ finally:
+ # Environment clean
+ session.close()
+ logging.info("Removing the temporary arp entry")
+ commands.getstatusoutput("arp -d %s -i %s" % (ip, ifname))
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 4f58dc0..7f7b56a 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -354,6 +354,9 @@ variants:
counts = 100
flood_minutes = 10
+ - jumbo: install setup unattended_install.cdrom
+ type = jumbo
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -489,10 +492,16 @@ variants:
variants:
- @rtl8139:
nic_model = rtl8139
+ jumbo:
+ mtu = 1500
- e1000:
nic_model = e1000
+ jumbo:
+ mtu = 16110
- virtio:
nic_model = virtio
+ jumbo:
+ mtu = 65520
# Guests
@@ -1024,7 +1033,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks)
+ no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [RFC PATCH 05/14] KVM-test: Add a subtest jumbo
2010-07-20 1:35 ` [Qemu-devel] " Amos Kong
@ 2010-07-27 14:13 ` Lucas Meneghel Rodrigues
-1 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-27 14:13 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> According to different nic model set different MTU for it. And ping from guest
> to host, to see whether tested size can be received by host.
>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/jumbo.py b/client/tests/kvm/tests/jumbo.py
> new file mode 100644
> index 0000000..9f56a87
> --- /dev/null
> +++ b/client/tests/kvm/tests/jumbo.py
> @@ -0,0 +1,133 @@
> +import os, re, logging, commands, time, random
> +from autotest_lib.client.common_lib import error
> +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
> +
> +def run_jumbo(test, params, env):
> + """
> + Test the RX jumbo frame function of vnics:
> + 1) boot the vm
> + 2) change the MTU of guest nics and host taps depends on the nic model
^ I think you meant 'depending on the nic model'
> + 3) add the static arp entry for guest nic
> + 4) wait for the MTU ok
> + 5) verify the patch mtu using ping
> + 6) ping the guest with large frames
> + 7) increament size ping
^ typo, increment
> + 8) flood ping the guest with large frames
> + 9) verify the path mtu
> + 10) revocer the mtu
^ maybe a typo, restore?
> +
> + @param test: kvm test object
> + @param params: Dictionary with the test parameters
> + @param env: Dictionary with test environment.
> + """
> +
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + session = kvm_test_utils.wait_for_login(vm)
> + mtu = params.get("mtu", "1500")
> + flood_time = params.get("flood_time", "300")
> + max_icmp_pkt_size = int(mtu) - 28
> +
> + ifname = vm.get_ifname(0)
> + ip = vm.get_address(0)
> + if ip is None:
> + raise error.TestError("Could not get the ip address")
> +
> + try:
> + # Environment preparartion
^ typo, preparation
> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
> +
> + logging.info("Changing the mtu of guest ...")
> + guest_mtu_cmd = "ifconfig %s mtu %s" % (ethname , mtu)
> + s, o = session.get_command_status_output(guest_mtu_cmd)
> + if s != 0:
> + logging.error(o)
> + raise error.TestError("Fail to set the mtu of guest nic: %s"
> + % ethname)
> +
> + logging.info("Chaning the mtu of host tap ...")
> + host_mtu_cmd = "ifconfig %s mtu %s" % (ifname, mtu)
> + s, o = commands.getstatusoutput(host_mtu_cmd)
> + if s != 0:
> + raise error.TestError("Fail to set the mtu of %s" % ifname)
> +
> + logging.info("Add a temporary static arp entry ...")
> + arp_add_cmd = "arp -s %s %s -i %s" % (ip, vm.get_macaddr(0), ifname)
> + s, o = commands.getstatusoutput(arp_add_cmd)
> + if s != 0 :
> + raise error.TestError("Fail to add temporary arp entry")
> +
> + def is_mtu_ok():
> + s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
> + packetsize = max_icmp_pkt_size,
> + hint = "do", timeout = 2)
^ please fix the default argument attribution, here and all subsequent
calls
> + if s != 0:
> + return False
> + else:
> + return True
^ Here you could simple use return s != 0
> +
> + def verify_mtu():
> + logging.info("Verify the path mtu")
> + s, o = kvm_net_utils.ping(ip, 10, interface = ifname,
> + packetsize = max_icmp_pkt_size,
> + hint = "do", timeout = 15)
> + if s != 0 :
> + logging.error(o)
> + raise error.TestFail("Path MTU is not as expected")
> + if kvm_net_utils.get_loss_ratio(o) != 0:
> + logging.error(o)
> + raise error.TestFail("Packet loss ratio during mtu verification"
> + " is not zero")
> +
> + def flood_ping():
> + logging.info("Flood with large frames")
> + kvm_net_utils.ping(ip, interface = ifname,
> + packetsize = max_icmp_pkt_size,
> + flood = True, timeout = float(flood_time))
> +
> + def large_frame_ping(count = 100):
> + logging.info("Large frame ping")
> + s, o = kvm_net_utils.ping(ip, count, interface = ifname,
> + packetsize = max_icmp_pkt_size,
> + timeout = float(count) * 2)
> + ratio = kvm_net_utils.get_loss_ratio(o)
> + if ratio != 0:
> + raise error.TestFail("Loss ratio of large frame ping is %s" \
> + % ratio)
^ No need to use the backslash to end line here
> + def size_increase_ping(step = random.randrange(90, 110)):
> + logging.info("Size increase ping")
> + for size in range(0, max_icmp_pkt_size + 1, step):
> + logging.info("Ping %s with size %s" % (ip, size))
> + s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
> + packetsize = size,
> + hint = "do", timeout = 1)
> + if s != 0:
> + s, o = kvm_net_utils.ping(ip, 10, interface = ifname,
> + packetsize = size,
> + adaptive = True, hint = "do",
> + timeout = 20)
> + if kvm_net_utils.get_loss_ratio(o) > 50:
> + raise error.TestFail("ping loss ratio is greater "
> + "than 50% for size %s" % size)
^ How do we stablish 50% as an acceptable packet loss ratio for this
test? Seems high to me, but I might be missing something
> +
> + logging.info("Waiting for the MTU to be OK")
> + # Wait for the MTU is OK
^ Unneeded comment, as we have an info message stating the same thing
> + if not kvm_utils.wait_for(is_mtu_ok, 10, 0, 1):
> + logging.debug(commands.getoutput("ifconfig -a"))
> + raise error.TestError("10 seconds elapsed while the mtu is not"
> + " as expected")
> +
> + # Functional Test
> + verify_mtu()
> + large_frame_ping()
> + size_increase_ping()
> +
> + # Stress test
> + flood_ping()
> + verify_mtu()
> +
> + finally:
> + # Environment clean
> + session.close()
> + logging.info("Removing the temporary arp entry")
> + commands.getstatusoutput("arp -d %s -i %s" % (ip, ifname))
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 4f58dc0..7f7b56a 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -354,6 +354,9 @@ variants:
> counts = 100
> flood_minutes = 10
>
> + - jumbo: install setup unattended_install.cdrom
> + type = jumbo
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> @@ -489,10 +492,16 @@ variants:
> variants:
> - @rtl8139:
> nic_model = rtl8139
> + jumbo:
> + mtu = 1500
> - e1000:
> nic_model = e1000
> + jumbo:
> + mtu = 16110
> - virtio:
> nic_model = virtio
> + jumbo:
> + mtu = 65520
>
>
> # Guests
> @@ -1024,7 +1033,7 @@ variants:
>
> # Windows section
> - @Windows:
> - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks)
> + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo
> shutdown_command = shutdown /s /f /t 0
> reboot_command = shutdown /r /f /t 0
> status_test_command = echo %errorlevel%
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 77+ messages in thread* [Qemu-devel] Re: [RFC PATCH 05/14] KVM-test: Add a subtest jumbo
@ 2010-07-27 14:13 ` Lucas Meneghel Rodrigues
0 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-27 14:13 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> According to different nic model set different MTU for it. And ping from guest
> to host, to see whether tested size can be received by host.
>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/jumbo.py b/client/tests/kvm/tests/jumbo.py
> new file mode 100644
> index 0000000..9f56a87
> --- /dev/null
> +++ b/client/tests/kvm/tests/jumbo.py
> @@ -0,0 +1,133 @@
> +import os, re, logging, commands, time, random
> +from autotest_lib.client.common_lib import error
> +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
> +
> +def run_jumbo(test, params, env):
> + """
> + Test the RX jumbo frame function of vnics:
> + 1) boot the vm
> + 2) change the MTU of guest nics and host taps depends on the nic model
^ I think you meant 'depending on the nic model'
> + 3) add the static arp entry for guest nic
> + 4) wait for the MTU ok
> + 5) verify the patch mtu using ping
> + 6) ping the guest with large frames
> + 7) increament size ping
^ typo, increment
> + 8) flood ping the guest with large frames
> + 9) verify the path mtu
> + 10) revocer the mtu
^ maybe a typo, restore?
> +
> + @param test: kvm test object
> + @param params: Dictionary with the test parameters
> + @param env: Dictionary with test environment.
> + """
> +
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + session = kvm_test_utils.wait_for_login(vm)
> + mtu = params.get("mtu", "1500")
> + flood_time = params.get("flood_time", "300")
> + max_icmp_pkt_size = int(mtu) - 28
> +
> + ifname = vm.get_ifname(0)
> + ip = vm.get_address(0)
> + if ip is None:
> + raise error.TestError("Could not get the ip address")
> +
> + try:
> + # Environment preparartion
^ typo, preparation
> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
> +
> + logging.info("Changing the mtu of guest ...")
> + guest_mtu_cmd = "ifconfig %s mtu %s" % (ethname , mtu)
> + s, o = session.get_command_status_output(guest_mtu_cmd)
> + if s != 0:
> + logging.error(o)
> + raise error.TestError("Fail to set the mtu of guest nic: %s"
> + % ethname)
> +
> + logging.info("Chaning the mtu of host tap ...")
> + host_mtu_cmd = "ifconfig %s mtu %s" % (ifname, mtu)
> + s, o = commands.getstatusoutput(host_mtu_cmd)
> + if s != 0:
> + raise error.TestError("Fail to set the mtu of %s" % ifname)
> +
> + logging.info("Add a temporary static arp entry ...")
> + arp_add_cmd = "arp -s %s %s -i %s" % (ip, vm.get_macaddr(0), ifname)
> + s, o = commands.getstatusoutput(arp_add_cmd)
> + if s != 0 :
> + raise error.TestError("Fail to add temporary arp entry")
> +
> + def is_mtu_ok():
> + s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
> + packetsize = max_icmp_pkt_size,
> + hint = "do", timeout = 2)
^ please fix the default argument attribution, here and all subsequent
calls
> + if s != 0:
> + return False
> + else:
> + return True
^ Here you could simple use return s != 0
> +
> + def verify_mtu():
> + logging.info("Verify the path mtu")
> + s, o = kvm_net_utils.ping(ip, 10, interface = ifname,
> + packetsize = max_icmp_pkt_size,
> + hint = "do", timeout = 15)
> + if s != 0 :
> + logging.error(o)
> + raise error.TestFail("Path MTU is not as expected")
> + if kvm_net_utils.get_loss_ratio(o) != 0:
> + logging.error(o)
> + raise error.TestFail("Packet loss ratio during mtu verification"
> + " is not zero")
> +
> + def flood_ping():
> + logging.info("Flood with large frames")
> + kvm_net_utils.ping(ip, interface = ifname,
> + packetsize = max_icmp_pkt_size,
> + flood = True, timeout = float(flood_time))
> +
> + def large_frame_ping(count = 100):
> + logging.info("Large frame ping")
> + s, o = kvm_net_utils.ping(ip, count, interface = ifname,
> + packetsize = max_icmp_pkt_size,
> + timeout = float(count) * 2)
> + ratio = kvm_net_utils.get_loss_ratio(o)
> + if ratio != 0:
> + raise error.TestFail("Loss ratio of large frame ping is %s" \
> + % ratio)
^ No need to use the backslash to end line here
> + def size_increase_ping(step = random.randrange(90, 110)):
> + logging.info("Size increase ping")
> + for size in range(0, max_icmp_pkt_size + 1, step):
> + logging.info("Ping %s with size %s" % (ip, size))
> + s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
> + packetsize = size,
> + hint = "do", timeout = 1)
> + if s != 0:
> + s, o = kvm_net_utils.ping(ip, 10, interface = ifname,
> + packetsize = size,
> + adaptive = True, hint = "do",
> + timeout = 20)
> + if kvm_net_utils.get_loss_ratio(o) > 50:
> + raise error.TestFail("ping loss ratio is greater "
> + "than 50% for size %s" % size)
^ How do we stablish 50% as an acceptable packet loss ratio for this
test? Seems high to me, but I might be missing something
> +
> + logging.info("Waiting for the MTU to be OK")
> + # Wait for the MTU is OK
^ Unneeded comment, as we have an info message stating the same thing
> + if not kvm_utils.wait_for(is_mtu_ok, 10, 0, 1):
> + logging.debug(commands.getoutput("ifconfig -a"))
> + raise error.TestError("10 seconds elapsed while the mtu is not"
> + " as expected")
> +
> + # Functional Test
> + verify_mtu()
> + large_frame_ping()
> + size_increase_ping()
> +
> + # Stress test
> + flood_ping()
> + verify_mtu()
> +
> + finally:
> + # Environment clean
> + session.close()
> + logging.info("Removing the temporary arp entry")
> + commands.getstatusoutput("arp -d %s -i %s" % (ip, ifname))
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 4f58dc0..7f7b56a 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -354,6 +354,9 @@ variants:
> counts = 100
> flood_minutes = 10
>
> + - jumbo: install setup unattended_install.cdrom
> + type = jumbo
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> @@ -489,10 +492,16 @@ variants:
> variants:
> - @rtl8139:
> nic_model = rtl8139
> + jumbo:
> + mtu = 1500
> - e1000:
> nic_model = e1000
> + jumbo:
> + mtu = 16110
> - virtio:
> nic_model = virtio
> + jumbo:
> + mtu = 65520
>
>
> # Guests
> @@ -1024,7 +1033,7 @@ variants:
>
> # Windows section
> - @Windows:
> - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks)
> + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo
> shutdown_command = shutdown /s /f /t 0
> reboot_command = shutdown /r /f /t 0
> status_test_command = echo %errorlevel%
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [RFC PATCH 05/14] KVM-test: Add a subtest jumbo
2010-07-27 14:13 ` [Qemu-devel] " Lucas Meneghel Rodrigues
@ 2010-08-10 7:18 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-10 7:18 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: autotest, qemu-devel, kvm
On Tue, Jul 27, 2010 at 10:13 PM, Lucas Meneghel Rodrigues
<lmr@redhat.com> wrote:
> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
>> According to different nic model set different MTU for it. And ping from guest
>> to host, to see whether tested size can be received by host.
>>
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> Signed-off-by: Amos Kong <akong@redhat.com>
>> ---
>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>
>> diff --git a/client/tests/kvm/tests/jumbo.py b/client/tests/kvm/tests/jumbo.py
>> new file mode 100644
>> index 0000000..9f56a87
>> --- /dev/null
>> +++ b/client/tests/kvm/tests/jumbo.py
>> @@ -0,0 +1,133 @@
>> +import os, re, logging, commands, time, random
>> +from autotest_lib.client.common_lib import error
>> +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
>> +
>> +def run_jumbo(test, params, env):
>> + """
>> + Test the RX jumbo frame function of vnics:
>> + 1) boot the vm
>> + 2) change the MTU of guest nics and host taps depends on the nic model
>
> ^ I think you meant 'depending on the nic model'
>
>> + 3) add the static arp entry for guest nic
>> + 4) wait for the MTU ok
>> + 5) verify the patch mtu using ping
>> + 6) ping the guest with large frames
>> + 7) increament size ping
>
> ^ typo, increment
>
>> + 8) flood ping the guest with large frames
>> + 9) verify the path mtu
>> + 10) revocer the mtu
>
> ^ maybe a typo, restore?
>
>> +
>> + @param test: kvm test object
>> + @param params: Dictionary with the test parameters
>> + @param env: Dictionary with test environment.
>> + """
>> +
>> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
>> + session = kvm_test_utils.wait_for_login(vm)
>> + mtu = params.get("mtu", "1500")
>> + flood_time = params.get("flood_time", "300")
>> + max_icmp_pkt_size = int(mtu) - 28
>> +
>> + ifname = vm.get_ifname(0)
>> + ip = vm.get_address(0)
>> + if ip is None:
>> + raise error.TestError("Could not get the ip address")
>> +
>> + try:
>> + # Environment preparartion
>
> ^ typo, preparation
>
>> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
>> +
>> + logging.info("Changing the mtu of guest ...")
>> + guest_mtu_cmd = "ifconfig %s mtu %s" % (ethname , mtu)
>> + s, o = session.get_command_status_output(guest_mtu_cmd)
>> + if s != 0:
>> + logging.error(o)
>> + raise error.TestError("Fail to set the mtu of guest nic: %s"
>> + % ethname)
>> +
>> + logging.info("Chaning the mtu of host tap ...")
>> + host_mtu_cmd = "ifconfig %s mtu %s" % (ifname, mtu)
>> + s, o = commands.getstatusoutput(host_mtu_cmd)
>> + if s != 0:
>> + raise error.TestError("Fail to set the mtu of %s" % ifname)
>> +
>> + logging.info("Add a temporary static arp entry ...")
>> + arp_add_cmd = "arp -s %s %s -i %s" % (ip, vm.get_macaddr(0), ifname)
>> + s, o = commands.getstatusoutput(arp_add_cmd)
>> + if s != 0 :
>> + raise error.TestError("Fail to add temporary arp entry")
>> +
>> + def is_mtu_ok():
>> + s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
>> + packetsize = max_icmp_pkt_size,
>> + hint = "do", timeout = 2)
>
> ^ please fix the default argument attribution, here and all subsequent
> calls
>
>> + if s != 0:
>> + return False
>> + else:
>> + return True
>
> ^ Here you could simple use return s != 0
>
>> +
>> + def verify_mtu():
>> + logging.info("Verify the path mtu")
>> + s, o = kvm_net_utils.ping(ip, 10, interface = ifname,
>> + packetsize = max_icmp_pkt_size,
>> + hint = "do", timeout = 15)
>> + if s != 0 :
>> + logging.error(o)
>> + raise error.TestFail("Path MTU is not as expected")
>> + if kvm_net_utils.get_loss_ratio(o) != 0:
>> + logging.error(o)
>> + raise error.TestFail("Packet loss ratio during mtu verification"
>> + " is not zero")
>> +
>> + def flood_ping():
>> + logging.info("Flood with large frames")
>> + kvm_net_utils.ping(ip, interface = ifname,
>> + packetsize = max_icmp_pkt_size,
>> + flood = True, timeout = float(flood_time))
>> +
>> + def large_frame_ping(count = 100):
>> + logging.info("Large frame ping")
>> + s, o = kvm_net_utils.ping(ip, count, interface = ifname,
>> + packetsize = max_icmp_pkt_size,
>> + timeout = float(count) * 2)
>> + ratio = kvm_net_utils.get_loss_ratio(o)
>> + if ratio != 0:
>> + raise error.TestFail("Loss ratio of large frame ping is %s" \
>> + % ratio)
>
> ^ No need to use the backslash to end line here
>
>> + def size_increase_ping(step = random.randrange(90, 110)):
>> + logging.info("Size increase ping")
>> + for size in range(0, max_icmp_pkt_size + 1, step):
>> + logging.info("Ping %s with size %s" % (ip, size))
>> + s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
>> + packetsize = size,
>> + hint = "do", timeout = 1)
>> + if s != 0:
>> + s, o = kvm_net_utils.ping(ip, 10, interface = ifname,
>> + packetsize = size,
>> + adaptive = True, hint = "do",
>> + timeout = 20)
>> + if kvm_net_utils.get_loss_ratio(o) > 50:
>> + raise error.TestFail("ping loss ratio is greater "
>> + "than 50% for size %s" % size)
>
> ^ How do we stablish 50% as an acceptable packet loss ratio for this
> test? Seems high to me, but I might be missing something
It's just a 'experimental' standard, we can also make it can be setup
in config file.
(missing some email, reply by personal Email :)
^ permalink raw reply [flat|nested] 77+ messages in thread* [Qemu-devel] Re: [Autotest] [RFC PATCH 05/14] KVM-test: Add a subtest jumbo
@ 2010-08-10 7:18 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-10 7:18 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: autotest, Amos Kong, qemu-devel, kvm
On Tue, Jul 27, 2010 at 10:13 PM, Lucas Meneghel Rodrigues
<lmr@redhat.com> wrote:
> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
>> According to different nic model set different MTU for it. And ping from guest
>> to host, to see whether tested size can be received by host.
>>
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> Signed-off-by: Amos Kong <akong@redhat.com>
>> ---
>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>
>> diff --git a/client/tests/kvm/tests/jumbo.py b/client/tests/kvm/tests/jumbo.py
>> new file mode 100644
>> index 0000000..9f56a87
>> --- /dev/null
>> +++ b/client/tests/kvm/tests/jumbo.py
>> @@ -0,0 +1,133 @@
>> +import os, re, logging, commands, time, random
>> +from autotest_lib.client.common_lib import error
>> +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
>> +
>> +def run_jumbo(test, params, env):
>> + """
>> + Test the RX jumbo frame function of vnics:
>> + 1) boot the vm
>> + 2) change the MTU of guest nics and host taps depends on the nic model
>
> ^ I think you meant 'depending on the nic model'
>
>> + 3) add the static arp entry for guest nic
>> + 4) wait for the MTU ok
>> + 5) verify the patch mtu using ping
>> + 6) ping the guest with large frames
>> + 7) increament size ping
>
> ^ typo, increment
>
>> + 8) flood ping the guest with large frames
>> + 9) verify the path mtu
>> + 10) revocer the mtu
>
> ^ maybe a typo, restore?
>
>> +
>> + @param test: kvm test object
>> + @param params: Dictionary with the test parameters
>> + @param env: Dictionary with test environment.
>> + """
>> +
>> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
>> + session = kvm_test_utils.wait_for_login(vm)
>> + mtu = params.get("mtu", "1500")
>> + flood_time = params.get("flood_time", "300")
>> + max_icmp_pkt_size = int(mtu) - 28
>> +
>> + ifname = vm.get_ifname(0)
>> + ip = vm.get_address(0)
>> + if ip is None:
>> + raise error.TestError("Could not get the ip address")
>> +
>> + try:
>> + # Environment preparartion
>
> ^ typo, preparation
>
>> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
>> +
>> + logging.info("Changing the mtu of guest ...")
>> + guest_mtu_cmd = "ifconfig %s mtu %s" % (ethname , mtu)
>> + s, o = session.get_command_status_output(guest_mtu_cmd)
>> + if s != 0:
>> + logging.error(o)
>> + raise error.TestError("Fail to set the mtu of guest nic: %s"
>> + % ethname)
>> +
>> + logging.info("Chaning the mtu of host tap ...")
>> + host_mtu_cmd = "ifconfig %s mtu %s" % (ifname, mtu)
>> + s, o = commands.getstatusoutput(host_mtu_cmd)
>> + if s != 0:
>> + raise error.TestError("Fail to set the mtu of %s" % ifname)
>> +
>> + logging.info("Add a temporary static arp entry ...")
>> + arp_add_cmd = "arp -s %s %s -i %s" % (ip, vm.get_macaddr(0), ifname)
>> + s, o = commands.getstatusoutput(arp_add_cmd)
>> + if s != 0 :
>> + raise error.TestError("Fail to add temporary arp entry")
>> +
>> + def is_mtu_ok():
>> + s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
>> + packetsize = max_icmp_pkt_size,
>> + hint = "do", timeout = 2)
>
> ^ please fix the default argument attribution, here and all subsequent
> calls
>
>> + if s != 0:
>> + return False
>> + else:
>> + return True
>
> ^ Here you could simple use return s != 0
>
>> +
>> + def verify_mtu():
>> + logging.info("Verify the path mtu")
>> + s, o = kvm_net_utils.ping(ip, 10, interface = ifname,
>> + packetsize = max_icmp_pkt_size,
>> + hint = "do", timeout = 15)
>> + if s != 0 :
>> + logging.error(o)
>> + raise error.TestFail("Path MTU is not as expected")
>> + if kvm_net_utils.get_loss_ratio(o) != 0:
>> + logging.error(o)
>> + raise error.TestFail("Packet loss ratio during mtu verification"
>> + " is not zero")
>> +
>> + def flood_ping():
>> + logging.info("Flood with large frames")
>> + kvm_net_utils.ping(ip, interface = ifname,
>> + packetsize = max_icmp_pkt_size,
>> + flood = True, timeout = float(flood_time))
>> +
>> + def large_frame_ping(count = 100):
>> + logging.info("Large frame ping")
>> + s, o = kvm_net_utils.ping(ip, count, interface = ifname,
>> + packetsize = max_icmp_pkt_size,
>> + timeout = float(count) * 2)
>> + ratio = kvm_net_utils.get_loss_ratio(o)
>> + if ratio != 0:
>> + raise error.TestFail("Loss ratio of large frame ping is %s" \
>> + % ratio)
>
> ^ No need to use the backslash to end line here
>
>> + def size_increase_ping(step = random.randrange(90, 110)):
>> + logging.info("Size increase ping")
>> + for size in range(0, max_icmp_pkt_size + 1, step):
>> + logging.info("Ping %s with size %s" % (ip, size))
>> + s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
>> + packetsize = size,
>> + hint = "do", timeout = 1)
>> + if s != 0:
>> + s, o = kvm_net_utils.ping(ip, 10, interface = ifname,
>> + packetsize = size,
>> + adaptive = True, hint = "do",
>> + timeout = 20)
>> + if kvm_net_utils.get_loss_ratio(o) > 50:
>> + raise error.TestFail("ping loss ratio is greater "
>> + "than 50% for size %s" % size)
>
> ^ How do we stablish 50% as an acceptable packet loss ratio for this
> test? Seems high to me, but I might be missing something
It's just a 'experimental' standard, we can also make it can be setup
in config file.
(missing some email, reply by personal Email :)
^ permalink raw reply [flat|nested] 77+ messages in thread
* [RFC PATCH 06/14] KVM-test: Add basic file transfer test
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:35 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
This test is the basic test of transfering file between host and guest. Try to
transfer a large file from host to guest, and transfer it back to host, then
compare the files by diff command.
The default file size is 4000M, scp timeout is 1000s. It means if the average
speed is less than 4M/s, this test will be fail.
We can extend this test by using another disk later, then we can transfer larger
files without the limit of first disk size.
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/file_transfer.py b/client/tests/kvm/tests/file_transfer.py
new file mode 100644
index 0000000..a20e62e
--- /dev/null
+++ b/client/tests/kvm/tests/file_transfer.py
@@ -0,0 +1,54 @@
+import logging, commands
+from autotest_lib.client.common_lib import error
+import kvm_utils, kvm_test_utils
+
+def run_file_transfer(test, params, env):
+ """
+ Test ethrnet device function by ethtool
+
+ 1) Boot up a virtual machine
+ 2) Create a large file by dd on host
+ 3) Copy this file from host to guest
+ 4) Copy this file from guest to host
+ 5) Check if file transfers good
+
+ @param test: Kvm test object
+ @param params: Dictionary with the test parameters.
+ @param env: Dictionary with test environment.
+ """
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ timeout=int(params.get("login_timeout", 360))
+ logging.info("Trying to log into guest '%s' by serial", vm.name)
+ session = kvm_utils.wait_for(lambda: vm.serial_login(),
+ timeout, 0, step=2)
+ if not session:
+ raise error.TestFail("Could not log into guest '%s'" % vm.name)
+
+ dir = test.tmpdir
+ scp_timeout = int(params.get("scp_timeout"))
+ cmd = "dd if=/dev/urandom of=%s/a.out bs=1M count=%d" % (dir, int(
+ params.get("filesize", 4000)))
+ try:
+ logging.info("Create file by dd command on host, cmd: %s" % cmd)
+ s, o = commands.getstatusoutput(cmd)
+ if s != 0:
+ raise error.TestError("Fail to create file, output:%s" % o)
+
+ logging.info("Transfer file from host to guest")
+ if not vm.copy_files_to("%s/a.out" % dir, "/tmp/b.out",
+ timeout=scp_timeout):
+ raise error.TestFail("Fail to transfer file from host to guest")
+
+ logging.info("Transfer file from guest to host")
+ if not vm.copy_files_from("/tmp/b.out", "%s/c.out" % dir,
+ timeout=scp_timeout):
+ raise error.TestFail("Fail to transfer file from guest to host")
+
+ logging.debug(commands.getoutput("ls -l %s/[ac].out" % dir))
+ s, o = commands.getstatusoutput("diff %s/a.out %s/c.out" % (dir, dir))
+ if s != 0:
+ raise error.TestFail("File changed after transfer. Output:%s" % o)
+ finally:
+ session.get_command_status("rm -f /tmp/b.out")
+ commands.getoutput("rm -f %s/[ac].out" % dir)
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 7f7b56a..872674e 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -357,6 +357,11 @@ variants:
- jumbo: install setup unattended_install.cdrom
type = jumbo
+ - file_transfer: install setup unattended_install.cdrom
+ type = file_transfer
+ filesize = 4000
+ scp_timeout = 1000
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1033,7 +1038,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo
+ no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 06/14] KVM-test: Add basic file transfer test
@ 2010-07-20 1:35 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
This test is the basic test of transfering file between host and guest. Try to
transfer a large file from host to guest, and transfer it back to host, then
compare the files by diff command.
The default file size is 4000M, scp timeout is 1000s. It means if the average
speed is less than 4M/s, this test will be fail.
We can extend this test by using another disk later, then we can transfer larger
files without the limit of first disk size.
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/file_transfer.py b/client/tests/kvm/tests/file_transfer.py
new file mode 100644
index 0000000..a20e62e
--- /dev/null
+++ b/client/tests/kvm/tests/file_transfer.py
@@ -0,0 +1,54 @@
+import logging, commands
+from autotest_lib.client.common_lib import error
+import kvm_utils, kvm_test_utils
+
+def run_file_transfer(test, params, env):
+ """
+ Test ethrnet device function by ethtool
+
+ 1) Boot up a virtual machine
+ 2) Create a large file by dd on host
+ 3) Copy this file from host to guest
+ 4) Copy this file from guest to host
+ 5) Check if file transfers good
+
+ @param test: Kvm test object
+ @param params: Dictionary with the test parameters.
+ @param env: Dictionary with test environment.
+ """
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ timeout=int(params.get("login_timeout", 360))
+ logging.info("Trying to log into guest '%s' by serial", vm.name)
+ session = kvm_utils.wait_for(lambda: vm.serial_login(),
+ timeout, 0, step=2)
+ if not session:
+ raise error.TestFail("Could not log into guest '%s'" % vm.name)
+
+ dir = test.tmpdir
+ scp_timeout = int(params.get("scp_timeout"))
+ cmd = "dd if=/dev/urandom of=%s/a.out bs=1M count=%d" % (dir, int(
+ params.get("filesize", 4000)))
+ try:
+ logging.info("Create file by dd command on host, cmd: %s" % cmd)
+ s, o = commands.getstatusoutput(cmd)
+ if s != 0:
+ raise error.TestError("Fail to create file, output:%s" % o)
+
+ logging.info("Transfer file from host to guest")
+ if not vm.copy_files_to("%s/a.out" % dir, "/tmp/b.out",
+ timeout=scp_timeout):
+ raise error.TestFail("Fail to transfer file from host to guest")
+
+ logging.info("Transfer file from guest to host")
+ if not vm.copy_files_from("/tmp/b.out", "%s/c.out" % dir,
+ timeout=scp_timeout):
+ raise error.TestFail("Fail to transfer file from guest to host")
+
+ logging.debug(commands.getoutput("ls -l %s/[ac].out" % dir))
+ s, o = commands.getstatusoutput("diff %s/a.out %s/c.out" % (dir, dir))
+ if s != 0:
+ raise error.TestFail("File changed after transfer. Output:%s" % o)
+ finally:
+ session.get_command_status("rm -f /tmp/b.out")
+ commands.getoutput("rm -f %s/[ac].out" % dir)
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 7f7b56a..872674e 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -357,6 +357,11 @@ variants:
- jumbo: install setup unattended_install.cdrom
type = jumbo
+ - file_transfer: install setup unattended_install.cdrom
+ type = file_transfer
+ filesize = 4000
+ scp_timeout = 1000
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1033,7 +1038,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo
+ no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 06/14] KVM-test: Add basic file transfer test
2010-07-20 1:35 ` [Qemu-devel] " Amos Kong
(?)
@ 2010-07-27 14:36 ` Lucas Meneghel Rodrigues
2010-08-10 9:29 ` Amos Kong
-1 siblings, 1 reply; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-27 14:36 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> This test is the basic test of transfering file between host and guest. Try to
> transfer a large file from host to guest, and transfer it back to host, then
> compare the files by diff command.
> The default file size is 4000M, scp timeout is 1000s. It means if the average
> speed is less than 4M/s, this test will be fail.
^ About this average 4MB/s throughput, is there some sort of
agreement/standard I'm not aware of?
> We can extend this test by using another disk later, then we can transfer larger
> files without the limit of first disk size.
>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/file_transfer.py b/client/tests/kvm/tests/file_transfer.py
> new file mode 100644
> index 0000000..a20e62e
> --- /dev/null
> +++ b/client/tests/kvm/tests/file_transfer.py
> @@ -0,0 +1,54 @@
> +import logging, commands
> +from autotest_lib.client.common_lib import error
> +import kvm_utils, kvm_test_utils
> +
> +def run_file_transfer(test, params, env):
> + """
> + Test ethrnet device function by ethtool
> +
> + 1) Boot up a virtual machine
> + 2) Create a large file by dd on host
> + 3) Copy this file from host to guest
> + 4) Copy this file from guest to host
> + 5) Check if file transfers good
> +
> + @param test: Kvm test object
> + @param params: Dictionary with the test parameters.
> + @param env: Dictionary with test environment.
> + """
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + timeout=int(params.get("login_timeout", 360))
> + logging.info("Trying to log into guest '%s' by serial", vm.name)
> + session = kvm_utils.wait_for(lambda: vm.serial_login(),
> + timeout, 0, step=2)
^ I assume the serial session is being used here to avoid interfering
with the bandwidth required for the test to complete as little as
possible, right?
> + if not session:
> + raise error.TestFail("Could not log into guest '%s'" % vm.name)
> +
> + dir = test.tmpdir
> + scp_timeout = int(params.get("scp_timeout"))
> + cmd = "dd if=/dev/urandom of=%s/a.out bs=1M count=%d" % (dir, int(
> + params.get("filesize", 4000)))
> + try:
> + logging.info("Create file by dd command on host, cmd: %s" % cmd)
> + s, o = commands.getstatusoutput(cmd)
> + if s != 0:
> + raise error.TestError("Fail to create file, output:%s" % o)
^ I've seen this throughout the patchseries, the use of functions
present on commands rather than utils.system, or utils.run. The former
are preferable on autotest tests, that's important to have in mind.
Would you consider changing those calls to the autotest API functions?
> + logging.info("Transfer file from host to guest")
> + if not vm.copy_files_to("%s/a.out" % dir, "/tmp/b.out",
> + timeout=scp_timeout):
> + raise error.TestFail("Fail to transfer file from host to guest")
> +
> + logging.info("Transfer file from guest to host")
> + if not vm.copy_files_from("/tmp/b.out", "%s/c.out" % dir,
> + timeout=scp_timeout):
> + raise error.TestFail("Fail to transfer file from guest to host")
> +
> + logging.debug(commands.getoutput("ls -l %s/[ac].out" % dir))
> + s, o = commands.getstatusoutput("diff %s/a.out %s/c.out" % (dir, dir))
> + if s != 0:
> + raise error.TestFail("File changed after transfer. Output:%s" % o)
^ It seems faster to use md5 or sha1 to verify the integrity of the
files rather than diff. Unless there's a good reason to use diff, I'd
like you to change it to one of the programs mentioned above.
> + finally:
> + session.get_command_status("rm -f /tmp/b.out")
> + commands.getoutput("rm -f %s/[ac].out" % dir)
> + session.close()
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 7f7b56a..872674e 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -357,6 +357,11 @@ variants:
> - jumbo: install setup unattended_install.cdrom
> type = jumbo
>
> + - file_transfer: install setup unattended_install.cdrom
> + type = file_transfer
> + filesize = 4000
> + scp_timeout = 1000
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> @@ -1033,7 +1038,7 @@ variants:
>
> # Windows section
> - @Windows:
> - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo
> + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer
> shutdown_command = shutdown /s /f /t 0
> reboot_command = shutdown /r /f /t 0
> status_test_command = echo %errorlevel%
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Autotest] [Qemu-devel] [RFC PATCH 06/14] KVM-test: Add basic file transfer test
2010-07-27 14:36 ` Lucas Meneghel Rodrigues
@ 2010-08-10 9:29 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-10 9:29 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: Amos Kong, autotest, qemu-devel, kvm
On Tue, Jul 27, 2010 at 10:36 PM, Lucas Meneghel Rodrigues
<lmr@redhat.com> wrote:
> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
>> This test is the basic test of transfering file between host and guest. Try to
>> transfer a large file from host to guest, and transfer it back to host, then
>> compare the files by diff command.
>> The default file size is 4000M, scp timeout is 1000s. It means if the average
>> speed is less than 4M/s, this test will be fail.
>
> ^ About this average 4MB/s throughput, is there some sort of
> agreement/standard I'm not aware of?
Also an 'experimental' standard.
The most important purpose is not check the transfer result, but catch
the bug during file transfer.
If case fail of transfer timeout, we need try to augment the timeout,
and compare the test result.
>> We can extend this test by using another disk later, then we can transfer larger
>> files without the limit of first disk size.
>>
>> Signed-off-by: Amos Kong <akong@redhat.com>
>> ---
>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>
>> diff --git a/client/tests/kvm/tests/file_transfer.py b/client/tests/kvm/tests/file_transfer.py
>> new file mode 100644
>> index 0000000..a20e62e
>> --- /dev/null
>> +++ b/client/tests/kvm/tests/file_transfer.py
>> @@ -0,0 +1,54 @@
>> +import logging, commands
>> +from autotest_lib.client.common_lib import error
>> +import kvm_utils, kvm_test_utils
>> +
>> +def run_file_transfer(test, params, env):
>> + """
>> + Test ethrnet device function by ethtool
>> +
>> + 1) Boot up a virtual machine
>> + 2) Create a large file by dd on host
>> + 3) Copy this file from host to guest
>> + 4) Copy this file from guest to host
>> + 5) Check if file transfers good
>> +
>> + @param test: Kvm test object
>> + @param params: Dictionary with the test parameters.
>> + @param env: Dictionary with test environment.
>> + """
>> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
>> + timeout=int(params.get("login_timeout", 360))
>> + logging.info("Trying to log into guest '%s' by serial", vm.name)
>> + session = kvm_utils.wait_for(lambda: vm.serial_login(),
>> + timeout, 0, step=2)
>
> ^ I assume the serial session is being used here to avoid interfering
> with the bandwidth required for the test to complete as little as
> possible, right?
Just want to split the control connection and data connection, if
network is down, some clean work would not be done.
But there is only one serial console session per VM, when we use
serial_login, serial console log would be disabled.
>> + if not session:
>> + raise error.TestFail("Could not log into guest '%s'" % vm.name)
>> +
>> + dir = test.tmpdir
>> + scp_timeout = int(params.get("scp_timeout"))
>> + cmd = "dd if=/dev/urandom of=%s/a.out bs=1M count=%d" % (dir, int(
>> + params.get("filesize", 4000)))
>> + try:
>> + logging.info("Create file by dd command on host, cmd: %s" % cmd)
>> + s, o = commands.getstatusoutput(cmd)
>> + if s != 0:
>> + raise error.TestError("Fail to create file, output:%s" % o)
>
> ^ I've seen this throughout the patchseries, the use of functions
> present on commands rather than utils.system, or utils.run. The former
> are preferable on autotest tests, that's important to have in mind.
> Would you consider changing those calls to the autotest API functions?
ok, I will try to use autotest API.
>> + logging.info("Transfer file from host to guest")
>> + if not vm.copy_files_to("%s/a.out" % dir, "/tmp/b.out",
>> + timeout=scp_timeout):
>> + raise error.TestFail("Fail to transfer file from host to guest")
>> +
>> + logging.info("Transfer file from guest to host")
>> + if not vm.copy_files_from("/tmp/b.out", "%s/c.out" % dir,
>> + timeout=scp_timeout):
>> + raise error.TestFail("Fail to transfer file from guest to host")
>> +
>> + logging.debug(commands.getoutput("ls -l %s/[ac].out" % dir))
>> + s, o = commands.getstatusoutput("diff %s/a.out %s/c.out" % (dir, dir))
>> + if s != 0:
>> + raise error.TestFail("File changed after transfer. Output:%s" % o)
>
> ^ It seems faster to use md5 or sha1 to verify the integrity of the
> files rather than diff. Unless there's a good reason to use diff, I'd
> like you to change it to one of the programs mentioned above.
I would change it to md5.
>> + finally:
>> + session.get_command_status("rm -f /tmp/b.out")
>> + commands.getoutput("rm -f %s/[ac].out" % dir)
>> + session.close()
>> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
>> index 7f7b56a..872674e 100644
>> --- a/client/tests/kvm/tests_base.cfg.sample
>> +++ b/client/tests/kvm/tests_base.cfg.sample
>> @@ -357,6 +357,11 @@ variants:
>> - jumbo: install setup unattended_install.cdrom
>> type = jumbo
>>
>> + - file_transfer: install setup unattended_install.cdrom
>> + type = file_transfer
>> + filesize = 4000
>> + scp_timeout = 1000
>> +
>> - physical_resources_check: install setup unattended_install.cdrom
>> type = physical_resources_check
>> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
>> @@ -1033,7 +1038,7 @@ variants:
>>
>> # Windows section
>> - @Windows:
>> - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo
>> + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer
>> shutdown_command = shutdown /s /f /t 0
>> reboot_command = shutdown /r /f /t 0
>> status_test_command = echo %errorlevel%
>>
>>
>
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Autotest] [Qemu-devel] [RFC PATCH 06/14] KVM-test: Add basic file transfer test
@ 2010-08-10 9:29 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-10 9:29 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: autotest, Amos Kong, qemu-devel, kvm
On Tue, Jul 27, 2010 at 10:36 PM, Lucas Meneghel Rodrigues
<lmr@redhat.com> wrote:
> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
>> This test is the basic test of transfering file between host and guest. Try to
>> transfer a large file from host to guest, and transfer it back to host, then
>> compare the files by diff command.
>> The default file size is 4000M, scp timeout is 1000s. It means if the average
>> speed is less than 4M/s, this test will be fail.
>
> ^ About this average 4MB/s throughput, is there some sort of
> agreement/standard I'm not aware of?
Also an 'experimental' standard.
The most important purpose is not check the transfer result, but catch
the bug during file transfer.
If case fail of transfer timeout, we need try to augment the timeout,
and compare the test result.
>> We can extend this test by using another disk later, then we can transfer larger
>> files without the limit of first disk size.
>>
>> Signed-off-by: Amos Kong <akong@redhat.com>
>> ---
>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>
>> diff --git a/client/tests/kvm/tests/file_transfer.py b/client/tests/kvm/tests/file_transfer.py
>> new file mode 100644
>> index 0000000..a20e62e
>> --- /dev/null
>> +++ b/client/tests/kvm/tests/file_transfer.py
>> @@ -0,0 +1,54 @@
>> +import logging, commands
>> +from autotest_lib.client.common_lib import error
>> +import kvm_utils, kvm_test_utils
>> +
>> +def run_file_transfer(test, params, env):
>> + """
>> + Test ethrnet device function by ethtool
>> +
>> + 1) Boot up a virtual machine
>> + 2) Create a large file by dd on host
>> + 3) Copy this file from host to guest
>> + 4) Copy this file from guest to host
>> + 5) Check if file transfers good
>> +
>> + @param test: Kvm test object
>> + @param params: Dictionary with the test parameters.
>> + @param env: Dictionary with test environment.
>> + """
>> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
>> + timeout=int(params.get("login_timeout", 360))
>> + logging.info("Trying to log into guest '%s' by serial", vm.name)
>> + session = kvm_utils.wait_for(lambda: vm.serial_login(),
>> + timeout, 0, step=2)
>
> ^ I assume the serial session is being used here to avoid interfering
> with the bandwidth required for the test to complete as little as
> possible, right?
Just want to split the control connection and data connection, if
network is down, some clean work would not be done.
But there is only one serial console session per VM, when we use
serial_login, serial console log would be disabled.
>> + if not session:
>> + raise error.TestFail("Could not log into guest '%s'" % vm.name)
>> +
>> + dir = test.tmpdir
>> + scp_timeout = int(params.get("scp_timeout"))
>> + cmd = "dd if=/dev/urandom of=%s/a.out bs=1M count=%d" % (dir, int(
>> + params.get("filesize", 4000)))
>> + try:
>> + logging.info("Create file by dd command on host, cmd: %s" % cmd)
>> + s, o = commands.getstatusoutput(cmd)
>> + if s != 0:
>> + raise error.TestError("Fail to create file, output:%s" % o)
>
> ^ I've seen this throughout the patchseries, the use of functions
> present on commands rather than utils.system, or utils.run. The former
> are preferable on autotest tests, that's important to have in mind.
> Would you consider changing those calls to the autotest API functions?
ok, I will try to use autotest API.
>> + logging.info("Transfer file from host to guest")
>> + if not vm.copy_files_to("%s/a.out" % dir, "/tmp/b.out",
>> + timeout=scp_timeout):
>> + raise error.TestFail("Fail to transfer file from host to guest")
>> +
>> + logging.info("Transfer file from guest to host")
>> + if not vm.copy_files_from("/tmp/b.out", "%s/c.out" % dir,
>> + timeout=scp_timeout):
>> + raise error.TestFail("Fail to transfer file from guest to host")
>> +
>> + logging.debug(commands.getoutput("ls -l %s/[ac].out" % dir))
>> + s, o = commands.getstatusoutput("diff %s/a.out %s/c.out" % (dir, dir))
>> + if s != 0:
>> + raise error.TestFail("File changed after transfer. Output:%s" % o)
>
> ^ It seems faster to use md5 or sha1 to verify the integrity of the
> files rather than diff. Unless there's a good reason to use diff, I'd
> like you to change it to one of the programs mentioned above.
I would change it to md5.
>> + finally:
>> + session.get_command_status("rm -f /tmp/b.out")
>> + commands.getoutput("rm -f %s/[ac].out" % dir)
>> + session.close()
>> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
>> index 7f7b56a..872674e 100644
>> --- a/client/tests/kvm/tests_base.cfg.sample
>> +++ b/client/tests/kvm/tests_base.cfg.sample
>> @@ -357,6 +357,11 @@ variants:
>> - jumbo: install setup unattended_install.cdrom
>> type = jumbo
>>
>> + - file_transfer: install setup unattended_install.cdrom
>> + type = file_transfer
>> + filesize = 4000
>> + scp_timeout = 1000
>> +
>> - physical_resources_check: install setup unattended_install.cdrom
>> type = physical_resources_check
>> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
>> @@ -1033,7 +1038,7 @@ variants:
>>
>> # Windows section
>> - @Windows:
>> - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo
>> + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer
>> shutdown_command = shutdown /s /f /t 0
>> reboot_command = shutdown /r /f /t 0
>> status_test_command = echo %errorlevel%
>>
>>
>
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [RFC PATCH 07/14] KVM-test: Add a subtest of load/unload nic driver
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:35 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
Repeatedly load/unload nic driver, try to transfer file between guest and host
by threads at the same time, and check the md5sum.
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/nicdriver_unload.py b/client/tests/kvm/tests/nicdriver_unload.py
new file mode 100644
index 0000000..22f9f44
--- /dev/null
+++ b/client/tests/kvm/tests/nicdriver_unload.py
@@ -0,0 +1,128 @@
+import logging, commands, threading, re, os
+from autotest_lib.client.common_lib import error
+import kvm_utils, kvm_test_utils, kvm_net_utils
+
+def run_nicdriver_unload(test, params, env):
+ """
+ Test nic driver
+
+ 1) Boot a vm
+ 2) Get the nic driver name
+ 3) Repeatedly unload/load nic driver
+ 4) Multi-session TCP transfer on test interface
+ 5) Check the test interface should still work
+
+ @param test: KVM test object
+ @param params: Dictionary with the test parameters
+ @param env: Dictionary with test environment.
+ """
+ timeout = int(params.get("login_timeout", 360))
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm, timeout=timeout)
+ logging.info("Trying to log into guest '%s' by serial", vm.name)
+ session2 = kvm_utils.wait_for(lambda: vm.serial_login(),
+ timeout, 0, step=2)
+ if not session2:
+ raise error.TestFail("Could not log into guest '%s'" % vm.name)
+
+ ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
+ try:
+ # FIXME: Try three waies to get nic driver name, because the
+ # modprobe.conf is dropped in latest system, and ethtool method is not
+ # supported by virtio_nic.
+
+ output = session.get_command_output("cat /etc/modprobe.conf")
+ driver = re.findall(r'%s (\w+)' % ethname,output)
+ if not driver:
+ output = session.get_command_output("ethtool -i %s" % ethname)
+ driver = re.findall(r'driver: (\w+)', output)
+ if not driver:
+ output = session.get_command_output("lspci -k")
+ driver = re.findall("Ethernet controller.*\n.*\n.*Kernel driver"
+ " in use: (\w+)", output)
+ driver = driver[0]
+ except IndexError:
+ raise error.TestError("Could not find driver name")
+
+ logging.info("driver is %s" % driver)
+
+ class ThreadScp(threading.Thread):
+ def run(self):
+ remote_file = '/tmp/' + self.getName()
+ file_list.append(remote_file)
+ ret = vm.copy_files_to(file_name, remote_file, timeout=scp_timeout)
+ logging.debug("Copy result of %s: %s" % (remote_file, ret))
+
+ def compare(origin_file, receive_file):
+ cmd = "md5sum %s"
+ output1 = commands.getstatusoutput(cmd % origin_file)[1].strip()
+ check_sum1 = output1.split()[0]
+ s, output2 = session.get_command_status_output(cmd % receive_file)
+ if s != 0:
+ logging.error("Could not get md5sum of receive_file")
+ return False
+ check_sum2 = output2.strip().split()[0]
+ logging.debug("origin: %s, receive: %s" % (check_sum1, check_sum2))
+ if check_sum1 != check_sum2:
+ logging.error("md5sum doesn't match")
+ return False
+ return True
+
+ #produce sized file in host
+ file_size = params.get("file_size")
+ file_name = "/tmp/nicdriver_unload_file"
+ cmd = "dd if=/dev/urandom of=%s bs=%sM count=1"
+ s, o = commands.getstatusoutput(cmd % (file_name, file_size))
+ if s != 0:
+ raise error.TestFail("Fail to create file by dd")
+
+ connect_time = params.get("connect_time")
+ scp_timeout = int(params.get("scp_timeout"))
+ thread_num = int(params.get("thread_num"))
+ file_list = []
+
+ unload_load_cmd = "sleep %s && ifconfig %s down && modprobe -r %s && "
+ unload_load_cmd += "sleep 1 && modprobe %s && ifconfig %s up"
+ unload_load_cmd = unload_load_cmd % (connect_time, ethname, driver,
+ driver, ethname)
+ pid = os.fork()
+ if pid != 0:
+ logging.info("unload/load nic driver repeatedly in guest...")
+ while True:
+ logging.debug("Try to unload/load nic drive once")
+ if session2.get_command_status(unload_load_cmd, timeout=120) != 0:
+ session.get_command_output("rm -rf /tmp/Thread-*")
+ raise error.TestFail("Unload/load nic driver failed")
+ pid, s = os.waitpid(pid, os.WNOHANG)
+ status = os.WEXITSTATUS(s)
+ if (pid, status) != (0, 0):
+ logging.debug("Child process ending")
+ break
+ else:
+ logging.info("Multi-session tcp data transfer")
+ threads = []
+ for i in range(thread_num):
+ t = ThreadScp()
+ t.start()
+ threads.append(t)
+ for t in threads:
+ t.join(timeout = scp_timeout)
+ os._exit(0)
+
+ session2.close()
+
+ try:
+ logging.info("Check md5sum for received files in multi-session")
+ for f in file_list:
+ if not compare(file_name, f):
+ raise error.TestFail("Fail to compare (guest)file %s" % f)
+
+ logging.info("Test nic function after load/unload")
+ if not vm.copy_files_to(file_name, file_name):
+ raise error.TestFail("Fail to copy file from host to guest")
+ if not compare(file_name, file_name):
+ raise error.TestFail("Test nic function after load/unload fail")
+
+ finally:
+ session.get_command_output("rm -rf /tmp/Thread-*")
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 872674e..03d15c0 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -362,6 +362,14 @@ variants:
filesize = 4000
scp_timeout = 1000
+ - nicdriver_unload: install setup unattended_install.cdrom
+ type = nicdriver_unload
+ nic_mode = tap
+ file_size = 100
+ connect_time = 4
+ scp_timeout = 300
+ thread_num = 10
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1038,7 +1046,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer
+ no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 07/14] KVM-test: Add a subtest of load/unload nic driver
@ 2010-07-20 1:35 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
Repeatedly load/unload nic driver, try to transfer file between guest and host
by threads at the same time, and check the md5sum.
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/nicdriver_unload.py b/client/tests/kvm/tests/nicdriver_unload.py
new file mode 100644
index 0000000..22f9f44
--- /dev/null
+++ b/client/tests/kvm/tests/nicdriver_unload.py
@@ -0,0 +1,128 @@
+import logging, commands, threading, re, os
+from autotest_lib.client.common_lib import error
+import kvm_utils, kvm_test_utils, kvm_net_utils
+
+def run_nicdriver_unload(test, params, env):
+ """
+ Test nic driver
+
+ 1) Boot a vm
+ 2) Get the nic driver name
+ 3) Repeatedly unload/load nic driver
+ 4) Multi-session TCP transfer on test interface
+ 5) Check the test interface should still work
+
+ @param test: KVM test object
+ @param params: Dictionary with the test parameters
+ @param env: Dictionary with test environment.
+ """
+ timeout = int(params.get("login_timeout", 360))
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm, timeout=timeout)
+ logging.info("Trying to log into guest '%s' by serial", vm.name)
+ session2 = kvm_utils.wait_for(lambda: vm.serial_login(),
+ timeout, 0, step=2)
+ if not session2:
+ raise error.TestFail("Could not log into guest '%s'" % vm.name)
+
+ ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
+ try:
+ # FIXME: Try three waies to get nic driver name, because the
+ # modprobe.conf is dropped in latest system, and ethtool method is not
+ # supported by virtio_nic.
+
+ output = session.get_command_output("cat /etc/modprobe.conf")
+ driver = re.findall(r'%s (\w+)' % ethname,output)
+ if not driver:
+ output = session.get_command_output("ethtool -i %s" % ethname)
+ driver = re.findall(r'driver: (\w+)', output)
+ if not driver:
+ output = session.get_command_output("lspci -k")
+ driver = re.findall("Ethernet controller.*\n.*\n.*Kernel driver"
+ " in use: (\w+)", output)
+ driver = driver[0]
+ except IndexError:
+ raise error.TestError("Could not find driver name")
+
+ logging.info("driver is %s" % driver)
+
+ class ThreadScp(threading.Thread):
+ def run(self):
+ remote_file = '/tmp/' + self.getName()
+ file_list.append(remote_file)
+ ret = vm.copy_files_to(file_name, remote_file, timeout=scp_timeout)
+ logging.debug("Copy result of %s: %s" % (remote_file, ret))
+
+ def compare(origin_file, receive_file):
+ cmd = "md5sum %s"
+ output1 = commands.getstatusoutput(cmd % origin_file)[1].strip()
+ check_sum1 = output1.split()[0]
+ s, output2 = session.get_command_status_output(cmd % receive_file)
+ if s != 0:
+ logging.error("Could not get md5sum of receive_file")
+ return False
+ check_sum2 = output2.strip().split()[0]
+ logging.debug("origin: %s, receive: %s" % (check_sum1, check_sum2))
+ if check_sum1 != check_sum2:
+ logging.error("md5sum doesn't match")
+ return False
+ return True
+
+ #produce sized file in host
+ file_size = params.get("file_size")
+ file_name = "/tmp/nicdriver_unload_file"
+ cmd = "dd if=/dev/urandom of=%s bs=%sM count=1"
+ s, o = commands.getstatusoutput(cmd % (file_name, file_size))
+ if s != 0:
+ raise error.TestFail("Fail to create file by dd")
+
+ connect_time = params.get("connect_time")
+ scp_timeout = int(params.get("scp_timeout"))
+ thread_num = int(params.get("thread_num"))
+ file_list = []
+
+ unload_load_cmd = "sleep %s && ifconfig %s down && modprobe -r %s && "
+ unload_load_cmd += "sleep 1 && modprobe %s && ifconfig %s up"
+ unload_load_cmd = unload_load_cmd % (connect_time, ethname, driver,
+ driver, ethname)
+ pid = os.fork()
+ if pid != 0:
+ logging.info("unload/load nic driver repeatedly in guest...")
+ while True:
+ logging.debug("Try to unload/load nic drive once")
+ if session2.get_command_status(unload_load_cmd, timeout=120) != 0:
+ session.get_command_output("rm -rf /tmp/Thread-*")
+ raise error.TestFail("Unload/load nic driver failed")
+ pid, s = os.waitpid(pid, os.WNOHANG)
+ status = os.WEXITSTATUS(s)
+ if (pid, status) != (0, 0):
+ logging.debug("Child process ending")
+ break
+ else:
+ logging.info("Multi-session tcp data transfer")
+ threads = []
+ for i in range(thread_num):
+ t = ThreadScp()
+ t.start()
+ threads.append(t)
+ for t in threads:
+ t.join(timeout = scp_timeout)
+ os._exit(0)
+
+ session2.close()
+
+ try:
+ logging.info("Check md5sum for received files in multi-session")
+ for f in file_list:
+ if not compare(file_name, f):
+ raise error.TestFail("Fail to compare (guest)file %s" % f)
+
+ logging.info("Test nic function after load/unload")
+ if not vm.copy_files_to(file_name, file_name):
+ raise error.TestFail("Fail to copy file from host to guest")
+ if not compare(file_name, file_name):
+ raise error.TestFail("Test nic function after load/unload fail")
+
+ finally:
+ session.get_command_output("rm -rf /tmp/Thread-*")
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 872674e..03d15c0 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -362,6 +362,14 @@ variants:
filesize = 4000
scp_timeout = 1000
+ - nicdriver_unload: install setup unattended_install.cdrom
+ type = nicdriver_unload
+ nic_mode = tap
+ file_size = 100
+ connect_time = 4
+ scp_timeout = 300
+ thread_num = 10
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1038,7 +1046,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer
+ no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 07/14] KVM-test: Add a subtest of load/unload nic driver
2010-07-20 1:35 ` [Qemu-devel] " Amos Kong
(?)
@ 2010-07-28 18:12 ` Lucas Meneghel Rodrigues
-1 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-28 18:12 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> Repeatedly load/unload nic driver, try to transfer file between guest and host
> by threads at the same time, and check the md5sum.
>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/nicdriver_unload.py b/client/tests/kvm/tests/nicdriver_unload.py
> new file mode 100644
> index 0000000..22f9f44
> --- /dev/null
> +++ b/client/tests/kvm/tests/nicdriver_unload.py
> @@ -0,0 +1,128 @@
> +import logging, commands, threading, re, os
> +from autotest_lib.client.common_lib import error
> +import kvm_utils, kvm_test_utils, kvm_net_utils
> +
> +def run_nicdriver_unload(test, params, env):
> + """
> + Test nic driver
> +
> + 1) Boot a vm
> + 2) Get the nic driver name
> + 3) Repeatedly unload/load nic driver
> + 4) Multi-session TCP transfer on test interface
> + 5) Check the test interface should still work
> +
> + @param test: KVM test object
> + @param params: Dictionary with the test parameters
> + @param env: Dictionary with test environment.
> + """
> + timeout = int(params.get("login_timeout", 360))
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + session = kvm_test_utils.wait_for_login(vm, timeout=timeout)
> + logging.info("Trying to log into guest '%s' by serial", vm.name)
> + session2 = kvm_utils.wait_for(lambda: vm.serial_login(),
> + timeout, 0, step=2)
> + if not session2:
> + raise error.TestFail("Could not log into guest '%s'" % vm.name)
> +
> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
> + try:
> + # FIXME: Try three waies to get nic driver name, because the
> + # modprobe.conf is dropped in latest system, and ethtool method is not
> + # supported by virtio_nic.
> +
> + output = session.get_command_output("cat /etc/modprobe.conf")
> + driver = re.findall(r'%s (\w+)' % ethname,output)
> + if not driver:
> + output = session.get_command_output("ethtool -i %s" % ethname)
> + driver = re.findall(r'driver: (\w+)', output)
> + if not driver:
> + output = session.get_command_output("lspci -k")
> + driver = re.findall("Ethernet controller.*\n.*\n.*Kernel driver"
> + " in use: (\w+)", output)
> + driver = driver[0]
> + except IndexError:
> + raise error.TestError("Could not find driver name")
^ About this whole block where there's an attempt to discover what the
driver name is. The methods listed there are not reliable (depends on
ethtool being installed, lspci -k not allways will list the kernel
driver in use, /etc/modprobe.conf will not be present on more recent
distributions). Instead of these methods, why don't we have a variant
for this test on each linux distro definition block, with the
appropriate driver names? It'd be something along the lines:
- 13.64:
image_name = f13-64
cdrom_cd1 = linux/Fedora-13-x86_64-DVD.iso
md5sum = 6fbae6379cf27f36e1f2c7827ba7dc35
md5sum_1m = 68821b9de4d3b5975d6634334e7f47a6
unattended_install:
unattended_file = unattended/Fedora-13.ks
tftp = images/f13-64/tftpboot
floppy = images/f13-64/floppy.img
nicdriver_unload:
e1000:
driver = e1000c
virtio:
driver = virtio_net
rtl8139:
driver = rtl8939
I believe it's safer than having to rely on the above methods.
> +
> + logging.info("driver is %s" % driver)
> +
> + class ThreadScp(threading.Thread):
> + def run(self):
> + remote_file = '/tmp/' + self.getName()
> + file_list.append(remote_file)
> + ret = vm.copy_files_to(file_name, remote_file, timeout=scp_timeout)
> + logging.debug("Copy result of %s: %s" % (remote_file, ret))
^ Here it'd be worth to have 2 debug messages,
if ret:
logging.debug("File %s was transfered successfuly", remote_file)
else:
logging.debug("Failed to transfer file %s", remote_file)
> + def compare(origin_file, receive_file):
> + cmd = "md5sum %s"
> + output1 = commands.getstatusoutput(cmd % origin_file)[1].strip()
^ If we are only interested on the output, getoutput() could be used.
But in this case we care whether md5 succeeded or not, so better to do
appropriate return code treatment, as you do below. Also, you could use
utils.hash() instead of using directly md5, and that *must* yield that
exact same result, being faster than resorting to subprocesses.
Also, remembering that I don't really love seeing things from the
commands API instead of utils.system, utils.run and others.
> + check_sum1 = output1.split()[0]
> + s, output2 = session.get_command_status_output(cmd % receive_file)
> + if s != 0:
> + logging.error("Could not get md5sum of receive_file")
> + return False
> + check_sum2 = output2.strip().split()[0]
> + logging.debug("origin: %s, receive: %s" % (check_sum1, check_sum2))
^ "original file md5: %s, received file md5: %s"
> + if check_sum1 != check_sum2:
> + logging.error("md5sum doesn't match")
> + return False
> + return True
> +
> + #produce sized file in host
> + file_size = params.get("file_size")
> + file_name = "/tmp/nicdriver_unload_file"
> + cmd = "dd if=/dev/urandom of=%s bs=%sM count=1"
> + s, o = commands.getstatusoutput(cmd % (file_name, file_size))
> + if s != 0:
> + raise error.TestFail("Fail to create file by dd")
Here utils.system would already raise a CmdError exception, so you could
just use it and not do exception handling
> + connect_time = params.get("connect_time")
> + scp_timeout = int(params.get("scp_timeout"))
> + thread_num = int(params.get("thread_num"))
> + file_list = []
> +
> + unload_load_cmd = "sleep %s && ifconfig %s down && modprobe -r %s && "
> + unload_load_cmd += "sleep 1 && modprobe %s && ifconfig %s up"
> + unload_load_cmd = unload_load_cmd % (connect_time, ethname, driver,
> + driver, ethname)
^ Here it'd be better to construct the whole command on a single assignment:
unload_load_cmd = ("sleep %s && ifconfig %s down && modprobe -r %s && "
"sleep 1 && modprobe %s && ifconfig %s up" %
(connect_time, ethname, driver, driver, ethname))
> + pid = os.fork()
> + if pid != 0:
> + logging.info("unload/load nic driver repeatedly in guest...")
> + while True:
> + logging.debug("Try to unload/load nic drive once")
> + if session2.get_command_status(unload_load_cmd, timeout=120) != 0:
> + session.get_command_output("rm -rf /tmp/Thread-*")
> + raise error.TestFail("Unload/load nic driver failed")
> + pid, s = os.waitpid(pid, os.WNOHANG)
> + status = os.WEXITSTATUS(s)
> + if (pid, status) != (0, 0):
> + logging.debug("Child process ending")
> + break
> + else:
> + logging.info("Multi-session tcp data transfer")
> + threads = []
> + for i in range(thread_num):
> + t = ThreadScp()
> + t.start()
> + threads.append(t)
> + for t in threads:
> + t.join(timeout = scp_timeout)
> + os._exit(0)
> +
> + session2.close()
> +
> + try:
> + logging.info("Check md5sum for received files in multi-session")
> + for f in file_list:
> + if not compare(file_name, f):
> + raise error.TestFail("Fail to compare (guest)file %s" % f)
> +
> + logging.info("Test nic function after load/unload")
> + if not vm.copy_files_to(file_name, file_name):
> + raise error.TestFail("Fail to copy file from host to guest")
> + if not compare(file_name, file_name):
> + raise error.TestFail("Test nic function after load/unload fail")
> +
> + finally:
> + session.get_command_output("rm -rf /tmp/Thread-*")
> + session.close()
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 872674e..03d15c0 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -362,6 +362,14 @@ variants:
> filesize = 4000
> scp_timeout = 1000
>
> + - nicdriver_unload: install setup unattended_install.cdrom
> + type = nicdriver_unload
> + nic_mode = tap
> + file_size = 100
> + connect_time = 4
> + scp_timeout = 300
> + thread_num = 10
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> @@ -1038,7 +1046,7 @@ variants:
>
> # Windows section
> - @Windows:
> - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer
> + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload
> shutdown_command = shutdown /s /f /t 0
> reboot_command = shutdown /r /f /t 0
> status_test_command = echo %errorlevel%
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [RFC PATCH 08/14] KVM-test: Add a subtest of nic promisc
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:35 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
This test mainly covers TCP sent from host to guest and from guest to host
with repeatedly turn on/off NIC promiscuous mode.
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/nic_promisc.py b/client/tests/kvm/tests/nic_promisc.py
new file mode 100644
index 0000000..9a0c979
--- /dev/null
+++ b/client/tests/kvm/tests/nic_promisc.py
@@ -0,0 +1,87 @@
+import logging, commands
+from autotest_lib.client.common_lib import error
+import kvm_utils, kvm_test_utils, kvm_net_utils
+
+def run_nic_promisc(test, params, env):
+ """
+ Test nic driver in promisc mode:
+
+ 1) Boot up a guest
+ 2) Repeatedly enable/disable promiscuous mode in guest
+ 3) TCP data transmission from host to guest, and from guest to host,
+ with 1/1460/65000/100000000 bytes payloads
+ 4) Clean temporary files
+ 5) Stop enable/disable promiscuous mode change
+
+ @param test: kvm test object
+ @param params: Dictionary with the test parameters
+ @param env: Dictionary with test environment
+ """
+ timeout = int(params.get("login_timeout", 360))
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm, timeout=timeout)
+ logging.info("Trying to log into guest '%s' by serial", vm.name)
+ session2 = kvm_utils.wait_for(lambda: vm.serial_login(),
+ timeout, 0, step=2)
+ if not session2:
+ raise error.TestFail("Could not log into guest '%s'" % vm.name)
+
+ def compare(filename):
+ cmd = "md5sum %s" % filename
+ s1, ret_host = commands.getstatusoutput(cmd)
+ s2, ret_guest = session.get_command_status_output(cmd)
+ if s1 != 0 or s2 != 0:
+ logging.debug("ret_host:%s, ret_guest:%s" % (ret_host, ret_guest))
+ logging.error("Could not get md5, cmd:%s" % cmd)
+ return False
+ if ret_host.strip() != ret_guest.strip():
+ logging.debug("ret_host :%s, ret_guest:%s" % (ret_host, ret_guest))
+ logging.error("Files' md5sum mismatch" % (receiver))
+ return False
+ return True
+
+ ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
+ set_promisc_cmd = "ip link set %s promisc on; sleep 0.01;" % ethname
+ set_promisc_cmd += "ip link set %s promisc off; sleep 0.01" % ethname
+ logging.info("Set promisc change repeatedly in guest")
+ session2.sendline("while true; do %s; done" % set_promisc_cmd)
+
+ dd_cmd = "dd if=/dev/urandom of=%s bs=%d count=1"
+ filename = "/tmp/nic_promisc_file"
+ file_size = params.get("file_size", "1, 1460, 65000, 100000000").split(",")
+ try:
+ for size in file_size:
+ logging.info("Create %s bytes file on host" % size)
+ s, o = commands.getstatusoutput(dd_cmd % (filename, int(size)))
+ if s != 0:
+ logging.debug("Output: %s"% o)
+ raise error.TestFail("Create file on host failed")
+
+ logging.info("Transfer file from host to guest")
+ if not vm.copy_files_to(filename, filename):
+ raise error.TestFail("File transfer failed")
+ if not compare(filename):
+ raise error.TestFail("Compare file failed")
+
+ logging.info("Create %s bytes file on guest" % size)
+ if session.get_command_status(dd_cmd % (filename, int(size)),
+ timeout=100) != 0:
+ raise error.TestFail("Create file on guest failed")
+
+ logging.info("Transfer file from guest to host")
+ if not vm.copy_files_from(filename, filename):
+ raise error.TestFail("File transfer failed")
+ if not compare(filename):
+ raise error.TestFail("Compare file failed")
+
+ logging.info("Clean temporal files")
+ cmd = "rm -f %s" % filename
+ s1, o = commands.getstatusoutput(cmd)
+ s2 = session.get_command_status(cmd)
+ if s1 != 0 or s2 != 0:
+ raise error.TestError("Fail to clean temporal files")
+ finally:
+ logging.info("Restore the %s to the nonpromisc mode" % ethname)
+ session2.close()
+ session.get_command_status("ip link set %s promisc off" % ethname)
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 03d15c0..9e2b9a0 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -370,6 +370,10 @@ variants:
scp_timeout = 300
thread_num = 10
+ - nic_promisc: install setup unattended_install.cdrom
+ type = nic_promisc
+ file_size = 1, 1460, 65000, 100000000
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1046,7 +1050,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload
+ no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 08/14] KVM-test: Add a subtest of nic promisc
@ 2010-07-20 1:35 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:35 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
This test mainly covers TCP sent from host to guest and from guest to host
with repeatedly turn on/off NIC promiscuous mode.
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/nic_promisc.py b/client/tests/kvm/tests/nic_promisc.py
new file mode 100644
index 0000000..9a0c979
--- /dev/null
+++ b/client/tests/kvm/tests/nic_promisc.py
@@ -0,0 +1,87 @@
+import logging, commands
+from autotest_lib.client.common_lib import error
+import kvm_utils, kvm_test_utils, kvm_net_utils
+
+def run_nic_promisc(test, params, env):
+ """
+ Test nic driver in promisc mode:
+
+ 1) Boot up a guest
+ 2) Repeatedly enable/disable promiscuous mode in guest
+ 3) TCP data transmission from host to guest, and from guest to host,
+ with 1/1460/65000/100000000 bytes payloads
+ 4) Clean temporary files
+ 5) Stop enable/disable promiscuous mode change
+
+ @param test: kvm test object
+ @param params: Dictionary with the test parameters
+ @param env: Dictionary with test environment
+ """
+ timeout = int(params.get("login_timeout", 360))
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm, timeout=timeout)
+ logging.info("Trying to log into guest '%s' by serial", vm.name)
+ session2 = kvm_utils.wait_for(lambda: vm.serial_login(),
+ timeout, 0, step=2)
+ if not session2:
+ raise error.TestFail("Could not log into guest '%s'" % vm.name)
+
+ def compare(filename):
+ cmd = "md5sum %s" % filename
+ s1, ret_host = commands.getstatusoutput(cmd)
+ s2, ret_guest = session.get_command_status_output(cmd)
+ if s1 != 0 or s2 != 0:
+ logging.debug("ret_host:%s, ret_guest:%s" % (ret_host, ret_guest))
+ logging.error("Could not get md5, cmd:%s" % cmd)
+ return False
+ if ret_host.strip() != ret_guest.strip():
+ logging.debug("ret_host :%s, ret_guest:%s" % (ret_host, ret_guest))
+ logging.error("Files' md5sum mismatch" % (receiver))
+ return False
+ return True
+
+ ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
+ set_promisc_cmd = "ip link set %s promisc on; sleep 0.01;" % ethname
+ set_promisc_cmd += "ip link set %s promisc off; sleep 0.01" % ethname
+ logging.info("Set promisc change repeatedly in guest")
+ session2.sendline("while true; do %s; done" % set_promisc_cmd)
+
+ dd_cmd = "dd if=/dev/urandom of=%s bs=%d count=1"
+ filename = "/tmp/nic_promisc_file"
+ file_size = params.get("file_size", "1, 1460, 65000, 100000000").split(",")
+ try:
+ for size in file_size:
+ logging.info("Create %s bytes file on host" % size)
+ s, o = commands.getstatusoutput(dd_cmd % (filename, int(size)))
+ if s != 0:
+ logging.debug("Output: %s"% o)
+ raise error.TestFail("Create file on host failed")
+
+ logging.info("Transfer file from host to guest")
+ if not vm.copy_files_to(filename, filename):
+ raise error.TestFail("File transfer failed")
+ if not compare(filename):
+ raise error.TestFail("Compare file failed")
+
+ logging.info("Create %s bytes file on guest" % size)
+ if session.get_command_status(dd_cmd % (filename, int(size)),
+ timeout=100) != 0:
+ raise error.TestFail("Create file on guest failed")
+
+ logging.info("Transfer file from guest to host")
+ if not vm.copy_files_from(filename, filename):
+ raise error.TestFail("File transfer failed")
+ if not compare(filename):
+ raise error.TestFail("Compare file failed")
+
+ logging.info("Clean temporal files")
+ cmd = "rm -f %s" % filename
+ s1, o = commands.getstatusoutput(cmd)
+ s2 = session.get_command_status(cmd)
+ if s1 != 0 or s2 != 0:
+ raise error.TestError("Fail to clean temporal files")
+ finally:
+ logging.info("Restore the %s to the nonpromisc mode" % ethname)
+ session2.close()
+ session.get_command_status("ip link set %s promisc off" % ethname)
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 03d15c0..9e2b9a0 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -370,6 +370,10 @@ variants:
scp_timeout = 300
thread_num = 10
+ - nic_promisc: install setup unattended_install.cdrom
+ type = nic_promisc
+ file_size = 1, 1460, 65000, 100000000
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1046,7 +1050,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload
+ no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [RFC PATCH 08/14] KVM-test: Add a subtest of nic promisc
2010-07-20 1:35 ` [Qemu-devel] " Amos Kong
@ 2010-07-28 21:35 ` Lucas Meneghel Rodrigues
-1 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-28 21:35 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> This test mainly covers TCP sent from host to guest and from guest to host
> with repeatedly turn on/off NIC promiscuous mode.
>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/nic_promisc.py b/client/tests/kvm/tests/nic_promisc.py
> new file mode 100644
> index 0000000..9a0c979
> --- /dev/null
> +++ b/client/tests/kvm/tests/nic_promisc.py
> @@ -0,0 +1,87 @@
> +import logging, commands
> +from autotest_lib.client.common_lib import error
> +import kvm_utils, kvm_test_utils, kvm_net_utils
> +
> +def run_nic_promisc(test, params, env):
> + """
> + Test nic driver in promisc mode:
> +
> + 1) Boot up a guest
> + 2) Repeatedly enable/disable promiscuous mode in guest
> + 3) TCP data transmission from host to guest, and from guest to host,
> + with 1/1460/65000/100000000 bytes payloads
> + 4) Clean temporary files
> + 5) Stop enable/disable promiscuous mode change
> +
> + @param test: kvm test object
> + @param params: Dictionary with the test parameters
> + @param env: Dictionary with test environment
> + """
> + timeout = int(params.get("login_timeout", 360))
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + session = kvm_test_utils.wait_for_login(vm, timeout=timeout)
> + logging.info("Trying to log into guest '%s' by serial", vm.name)
> + session2 = kvm_utils.wait_for(lambda: vm.serial_login(),
> + timeout, 0, step=2)
> + if not session2:
> + raise error.TestFail("Could not log into guest '%s'" % vm.name)
> +
> + def compare(filename):
> + cmd = "md5sum %s" % filename
> + s1, ret_host = commands.getstatusoutput(cmd)
> + s2, ret_guest = session.get_command_status_output(cmd)
> + if s1 != 0 or s2 != 0:
> + logging.debug("ret_host:%s, ret_guest:%s" % (ret_host, ret_guest))
> + logging.error("Could not get md5, cmd:%s" % cmd)
> + return False
> + if ret_host.strip() != ret_guest.strip():
> + logging.debug("ret_host :%s, ret_guest:%s" % (ret_host, ret_guest))
> + logging.error("Files' md5sum mismatch" % (receiver))
> + return False
^ The above debug messages will be confusing when looked by someone who
is not familiar with the test code, so we should make their lives
easier:
def compare(filename):
cmd = "md5sum %s" % filename
rc_host, md5_host = commands.getstatusoutput(cmd)
rc_guest, md5_guest = session.get_command_status_output(cmd)
if rc_host:
logging.debug('Could not get MD5 hash for file %s on host, output: %s', filename, md5_host)
return False
if rc_guest:
logging.debug('Could not get MD5 hash for file %s on guest, output: %s', filename, md5_guest)
return False
md5host = md5host.strip()
md5guest = md5guest.strip()
if md5host != md5guest:
logging.error('MD5 hash mismatch between file %s present on guest and on host', filename)
logging.error('MD5 hash for file on guest: %s, MD5 hash for file on host: %s', md5_host, md5_guest)
return False
return True
> + return True
> +
> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
> + set_promisc_cmd = "ip link set %s promisc on; sleep 0.01;" % ethname
> + set_promisc_cmd += "ip link set %s promisc off; sleep 0.01" % ethname
^ You could do the above on a single attribution, see comment on patch 7
of the patchseries.
> + logging.info("Set promisc change repeatedly in guest")
> + session2.sendline("while true; do %s; done" % set_promisc_cmd)
> +
> + dd_cmd = "dd if=/dev/urandom of=%s bs=%d count=1"
> + filename = "/tmp/nic_promisc_file"
> + file_size = params.get("file_size", "1, 1460, 65000, 100000000").split(",")
> + try:
> + for size in file_size:
> + logging.info("Create %s bytes file on host" % size)
> + s, o = commands.getstatusoutput(dd_cmd % (filename, int(size)))
> + if s != 0:
> + logging.debug("Output: %s"% o)
> + raise error.TestFail("Create file on host failed")
> +
> + logging.info("Transfer file from host to guest")
> + if not vm.copy_files_to(filename, filename):
> + raise error.TestFail("File transfer failed")
> + if not compare(filename):
> + raise error.TestFail("Compare file failed")
^ It'd be better if we don't abruptly fail the whole test if we get a
failure for a single size, what about having a global failure counter,
and increment it if we have failures, making sure we log errors
appropriately?
> + logging.info("Create %s bytes file on guest" % size)
> + if session.get_command_status(dd_cmd % (filename, int(size)),
> + timeout=100) != 0:
> + raise error.TestFail("Create file on guest failed")
> +
> + logging.info("Transfer file from guest to host")
> + if not vm.copy_files_from(filename, filename):
> + raise error.TestFail("File transfer failed")
> + if not compare(filename):
> + raise error.TestFail("Compare file failed")
^ Same comment as above.
> + logging.info("Clean temporal files")
^ Typo, temporary
> + cmd = "rm -f %s" % filename
> + s1, o = commands.getstatusoutput(cmd)
> + s2 = session.get_command_status(cmd)
> + if s1 != 0 or s2 != 0:
> + raise error.TestError("Fail to clean temporal files")
^ Same as above, with the exception that maybe a failure during cleanup
is not something so bad we'd need to fail the entire test because of it.
> + finally:
> + logging.info("Restore the %s to the nonpromisc mode" % ethname)
> + session2.close()
> + session.get_command_status("ip link set %s promisc off" % ethname)
> + session.close()
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 03d15c0..9e2b9a0 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -370,6 +370,10 @@ variants:
> scp_timeout = 300
> thread_num = 10
>
> + - nic_promisc: install setup unattended_install.cdrom
> + type = nic_promisc
> + file_size = 1, 1460, 65000, 100000000
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> @@ -1046,7 +1050,7 @@ variants:
>
> # Windows section
> - @Windows:
> - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload
> + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc
> shutdown_command = shutdown /s /f /t 0
> reboot_command = shutdown /r /f /t 0
> status_test_command = echo %errorlevel%
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 77+ messages in thread* [Qemu-devel] Re: [RFC PATCH 08/14] KVM-test: Add a subtest of nic promisc
@ 2010-07-28 21:35 ` Lucas Meneghel Rodrigues
0 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-28 21:35 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
> This test mainly covers TCP sent from host to guest and from guest to host
> with repeatedly turn on/off NIC promiscuous mode.
>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/nic_promisc.py b/client/tests/kvm/tests/nic_promisc.py
> new file mode 100644
> index 0000000..9a0c979
> --- /dev/null
> +++ b/client/tests/kvm/tests/nic_promisc.py
> @@ -0,0 +1,87 @@
> +import logging, commands
> +from autotest_lib.client.common_lib import error
> +import kvm_utils, kvm_test_utils, kvm_net_utils
> +
> +def run_nic_promisc(test, params, env):
> + """
> + Test nic driver in promisc mode:
> +
> + 1) Boot up a guest
> + 2) Repeatedly enable/disable promiscuous mode in guest
> + 3) TCP data transmission from host to guest, and from guest to host,
> + with 1/1460/65000/100000000 bytes payloads
> + 4) Clean temporary files
> + 5) Stop enable/disable promiscuous mode change
> +
> + @param test: kvm test object
> + @param params: Dictionary with the test parameters
> + @param env: Dictionary with test environment
> + """
> + timeout = int(params.get("login_timeout", 360))
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + session = kvm_test_utils.wait_for_login(vm, timeout=timeout)
> + logging.info("Trying to log into guest '%s' by serial", vm.name)
> + session2 = kvm_utils.wait_for(lambda: vm.serial_login(),
> + timeout, 0, step=2)
> + if not session2:
> + raise error.TestFail("Could not log into guest '%s'" % vm.name)
> +
> + def compare(filename):
> + cmd = "md5sum %s" % filename
> + s1, ret_host = commands.getstatusoutput(cmd)
> + s2, ret_guest = session.get_command_status_output(cmd)
> + if s1 != 0 or s2 != 0:
> + logging.debug("ret_host:%s, ret_guest:%s" % (ret_host, ret_guest))
> + logging.error("Could not get md5, cmd:%s" % cmd)
> + return False
> + if ret_host.strip() != ret_guest.strip():
> + logging.debug("ret_host :%s, ret_guest:%s" % (ret_host, ret_guest))
> + logging.error("Files' md5sum mismatch" % (receiver))
> + return False
^ The above debug messages will be confusing when looked by someone who
is not familiar with the test code, so we should make their lives
easier:
def compare(filename):
cmd = "md5sum %s" % filename
rc_host, md5_host = commands.getstatusoutput(cmd)
rc_guest, md5_guest = session.get_command_status_output(cmd)
if rc_host:
logging.debug('Could not get MD5 hash for file %s on host, output: %s', filename, md5_host)
return False
if rc_guest:
logging.debug('Could not get MD5 hash for file %s on guest, output: %s', filename, md5_guest)
return False
md5host = md5host.strip()
md5guest = md5guest.strip()
if md5host != md5guest:
logging.error('MD5 hash mismatch between file %s present on guest and on host', filename)
logging.error('MD5 hash for file on guest: %s, MD5 hash for file on host: %s', md5_host, md5_guest)
return False
return True
> + return True
> +
> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
> + set_promisc_cmd = "ip link set %s promisc on; sleep 0.01;" % ethname
> + set_promisc_cmd += "ip link set %s promisc off; sleep 0.01" % ethname
^ You could do the above on a single attribution, see comment on patch 7
of the patchseries.
> + logging.info("Set promisc change repeatedly in guest")
> + session2.sendline("while true; do %s; done" % set_promisc_cmd)
> +
> + dd_cmd = "dd if=/dev/urandom of=%s bs=%d count=1"
> + filename = "/tmp/nic_promisc_file"
> + file_size = params.get("file_size", "1, 1460, 65000, 100000000").split(",")
> + try:
> + for size in file_size:
> + logging.info("Create %s bytes file on host" % size)
> + s, o = commands.getstatusoutput(dd_cmd % (filename, int(size)))
> + if s != 0:
> + logging.debug("Output: %s"% o)
> + raise error.TestFail("Create file on host failed")
> +
> + logging.info("Transfer file from host to guest")
> + if not vm.copy_files_to(filename, filename):
> + raise error.TestFail("File transfer failed")
> + if not compare(filename):
> + raise error.TestFail("Compare file failed")
^ It'd be better if we don't abruptly fail the whole test if we get a
failure for a single size, what about having a global failure counter,
and increment it if we have failures, making sure we log errors
appropriately?
> + logging.info("Create %s bytes file on guest" % size)
> + if session.get_command_status(dd_cmd % (filename, int(size)),
> + timeout=100) != 0:
> + raise error.TestFail("Create file on guest failed")
> +
> + logging.info("Transfer file from guest to host")
> + if not vm.copy_files_from(filename, filename):
> + raise error.TestFail("File transfer failed")
> + if not compare(filename):
> + raise error.TestFail("Compare file failed")
^ Same comment as above.
> + logging.info("Clean temporal files")
^ Typo, temporary
> + cmd = "rm -f %s" % filename
> + s1, o = commands.getstatusoutput(cmd)
> + s2 = session.get_command_status(cmd)
> + if s1 != 0 or s2 != 0:
> + raise error.TestError("Fail to clean temporal files")
^ Same as above, with the exception that maybe a failure during cleanup
is not something so bad we'd need to fail the entire test because of it.
> + finally:
> + logging.info("Restore the %s to the nonpromisc mode" % ethname)
> + session2.close()
> + session.get_command_status("ip link set %s promisc off" % ethname)
> + session.close()
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 03d15c0..9e2b9a0 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -370,6 +370,10 @@ variants:
> scp_timeout = 300
> thread_num = 10
>
> + - nic_promisc: install setup unattended_install.cdrom
> + type = nic_promisc
> + file_size = 1, 1460, 65000, 100000000
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> @@ -1046,7 +1050,7 @@ variants:
>
> # Windows section
> - @Windows:
> - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload
> + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc
> shutdown_command = shutdown /s /f /t 0
> reboot_command = shutdown /r /f /t 0
> status_test_command = echo %errorlevel%
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Autotest] [RFC PATCH 08/14] KVM-test: Add a subtest of nic promisc
2010-07-28 21:35 ` [Qemu-devel] " Lucas Meneghel Rodrigues
@ 2010-08-11 1:34 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-11 1:34 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: Amos Kong, autotest, qemu-devel, kvm
On Thu, Jul 29, 2010 at 5:35 AM, Lucas Meneghel Rodrigues
<lmr@redhat.com> wrote:
> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
>> This test mainly covers TCP sent from host to guest and from guest to host
>> with repeatedly turn on/off NIC promiscuous mode.
>>
>> Signed-off-by: Amos Kong <akong@redhat.com>
>> ---
>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>
>> diff --git a/client/tests/kvm/tests/nic_promisc.py b/client/tests/kvm/tests/nic_promisc.py
>> new file mode 100644
>> index 0000000..9a0c979
>> --- /dev/null
>> +++ b/client/tests/kvm/tests/nic_promisc.py
>> @@ -0,0 +1,87 @@
>> +import logging, commands
>> +from autotest_lib.client.common_lib import error
>> +import kvm_utils, kvm_test_utils, kvm_net_utils
>> +
>> +def run_nic_promisc(test, params, env):
>> + """
>> + Test nic driver in promisc mode:
>> +
>> + 1) Boot up a guest
>> + 2) Repeatedly enable/disable promiscuous mode in guest
>> + 3) TCP data transmission from host to guest, and from guest to host,
>> + with 1/1460/65000/100000000 bytes payloads
>> + 4) Clean temporary files
>> + 5) Stop enable/disable promiscuous mode change
>> +
>> + @param test: kvm test object
>> + @param params: Dictionary with the test parameters
>> + @param env: Dictionary with test environment
>> + """
>> + timeout = int(params.get("login_timeout", 360))
>> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
>> + session = kvm_test_utils.wait_for_login(vm, timeout=timeout)
>> + logging.info("Trying to log into guest '%s' by serial", vm.name)
>> + session2 = kvm_utils.wait_for(lambda: vm.serial_login(),
>> + timeout, 0, step=2)
>> + if not session2:
>> + raise error.TestFail("Could not log into guest '%s'" % vm.name)
>> +
>> + def compare(filename):
>> + cmd = "md5sum %s" % filename
>> + s1, ret_host = commands.getstatusoutput(cmd)
>> + s2, ret_guest = session.get_command_status_output(cmd)
>> + if s1 != 0 or s2 != 0:
>> + logging.debug("ret_host:%s, ret_guest:%s" % (ret_host, ret_guest))
>> + logging.error("Could not get md5, cmd:%s" % cmd)
>> + return False
>> + if ret_host.strip() != ret_guest.strip():
>> + logging.debug("ret_host :%s, ret_guest:%s" % (ret_host, ret_guest))
>> + logging.error("Files' md5sum mismatch" % (receiver))
>> + return False
>
> ^ The above debug messages will be confusing when looked by someone who
> is not familiar with the test code, so we should make their lives
> easier:
>
> def compare(filename):
> cmd = "md5sum %s" % filename
> rc_host, md5_host = commands.getstatusoutput(cmd)
> rc_guest, md5_guest = session.get_command_status_output(cmd)
> if rc_host:
> logging.debug('Could not get MD5 hash for file %s on host, output: %s', filename, md5_host)
> return False
> if rc_guest:
> logging.debug('Could not get MD5 hash for file %s on guest, output: %s', filename, md5_guest)
> return False
> md5host = md5host.strip()
> md5guest = md5guest.strip()
> if md5host != md5guest:
> logging.error('MD5 hash mismatch between file %s present on guest and on host', filename)
> logging.error('MD5 hash for file on guest: %s, MD5 hash for file on host: %s', md5_host, md5_guest)
> return False
> return True
>
>
>> + return True
>> +
>> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
>> + set_promisc_cmd = "ip link set %s promisc on; sleep 0.01;" % ethname
>> + set_promisc_cmd += "ip link set %s promisc off; sleep 0.01" % ethname
>
> ^ You could do the above on a single attribution, see comment on patch 7
> of the patchseries.
>
>> + logging.info("Set promisc change repeatedly in guest")
>> + session2.sendline("while true; do %s; done" % set_promisc_cmd)
>> +
>> + dd_cmd = "dd if=/dev/urandom of=%s bs=%d count=1"
>> + filename = "/tmp/nic_promisc_file"
>> + file_size = params.get("file_size", "1, 1460, 65000, 100000000").split(",")
>> + try:
>> + for size in file_size:
>> + logging.info("Create %s bytes file on host" % size)
>> + s, o = commands.getstatusoutput(dd_cmd % (filename, int(size)))
>> + if s != 0:
>> + logging.debug("Output: %s"% o)
>> + raise error.TestFail("Create file on host failed")
>> +
>> + logging.info("Transfer file from host to guest")
>> + if not vm.copy_files_to(filename, filename):
>> + raise error.TestFail("File transfer failed")
>> + if not compare(filename):
>> + raise error.TestFail("Compare file failed")
>
> ^ It'd be better if we don't abruptly fail the whole test if we get a
> failure for a single size, what about having a global failure counter,
> and increment it if we have failures, making sure we log errors
> appropriately?
How about add a success counter ?
only success_counter+1 after successfully compare.
If the end success_counter doesn't equals to len(file_size), raise and
fail exception.
>> + logging.info("Create %s bytes file on guest" % size)
>> + if session.get_command_status(dd_cmd % (filename, int(size)),
>> + timeout=100) != 0:
>> + raise error.TestFail("Create file on guest failed")
>> +
>> + logging.info("Transfer file from guest to host")
>> + if not vm.copy_files_from(filename, filename):
>> + raise error.TestFail("File transfer failed")
>> + if not compare(filename):
>> + raise error.TestFail("Compare file failed")
>
> ^ Same comment as above.
>
>> + logging.info("Clean temporal files")
>
> ^ Typo, temporary
>
>> + cmd = "rm -f %s" % filename
>> + s1, o = commands.getstatusoutput(cmd)
>> + s2 = session.get_command_status(cmd)
>> + if s1 != 0 or s2 != 0:
>> + raise error.TestError("Fail to clean temporal files")
>
> ^ Same as above, with the exception that maybe a failure during cleanup
> is not something so bad we'd need to fail the entire test because of it.
>
>> + finally:
>> + logging.info("Restore the %s to the nonpromisc mode" % ethname)
>> + session2.close()
>> + session.get_command_status("ip link set %s promisc off" % ethname)
>> + session.close()
>> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
>> index 03d15c0..9e2b9a0 100644
>> --- a/client/tests/kvm/tests_base.cfg.sample
>> +++ b/client/tests/kvm/tests_base.cfg.sample
>> @@ -370,6 +370,10 @@ variants:
>> scp_timeout = 300
>> thread_num = 10
>>
>> + - nic_promisc: install setup unattended_install.cdrom
>> + type = nic_promisc
>> + file_size = 1, 1460, 65000, 100000000
>> +
>> - physical_resources_check: install setup unattended_install.cdrom
>> type = physical_resources_check
>> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
>> @@ -1046,7 +1050,7 @@ variants:
>>
>> # Windows section
>> - @Windows:
>> - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload
>> + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc
>> shutdown_command = shutdown /s /f /t 0
>> reboot_command = shutdown /r /f /t 0
>> status_test_command = echo %errorlevel%
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>
^ permalink raw reply [flat|nested] 77+ messages in thread* [Qemu-devel] Re: [Autotest] [RFC PATCH 08/14] KVM-test: Add a subtest of nic promisc
@ 2010-08-11 1:34 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-11 1:34 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: autotest, Amos Kong, qemu-devel, kvm
On Thu, Jul 29, 2010 at 5:35 AM, Lucas Meneghel Rodrigues
<lmr@redhat.com> wrote:
> On Tue, 2010-07-20 at 09:35 +0800, Amos Kong wrote:
>> This test mainly covers TCP sent from host to guest and from guest to host
>> with repeatedly turn on/off NIC promiscuous mode.
>>
>> Signed-off-by: Amos Kong <akong@redhat.com>
>> ---
>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>
>> diff --git a/client/tests/kvm/tests/nic_promisc.py b/client/tests/kvm/tests/nic_promisc.py
>> new file mode 100644
>> index 0000000..9a0c979
>> --- /dev/null
>> +++ b/client/tests/kvm/tests/nic_promisc.py
>> @@ -0,0 +1,87 @@
>> +import logging, commands
>> +from autotest_lib.client.common_lib import error
>> +import kvm_utils, kvm_test_utils, kvm_net_utils
>> +
>> +def run_nic_promisc(test, params, env):
>> + """
>> + Test nic driver in promisc mode:
>> +
>> + 1) Boot up a guest
>> + 2) Repeatedly enable/disable promiscuous mode in guest
>> + 3) TCP data transmission from host to guest, and from guest to host,
>> + with 1/1460/65000/100000000 bytes payloads
>> + 4) Clean temporary files
>> + 5) Stop enable/disable promiscuous mode change
>> +
>> + @param test: kvm test object
>> + @param params: Dictionary with the test parameters
>> + @param env: Dictionary with test environment
>> + """
>> + timeout = int(params.get("login_timeout", 360))
>> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
>> + session = kvm_test_utils.wait_for_login(vm, timeout=timeout)
>> + logging.info("Trying to log into guest '%s' by serial", vm.name)
>> + session2 = kvm_utils.wait_for(lambda: vm.serial_login(),
>> + timeout, 0, step=2)
>> + if not session2:
>> + raise error.TestFail("Could not log into guest '%s'" % vm.name)
>> +
>> + def compare(filename):
>> + cmd = "md5sum %s" % filename
>> + s1, ret_host = commands.getstatusoutput(cmd)
>> + s2, ret_guest = session.get_command_status_output(cmd)
>> + if s1 != 0 or s2 != 0:
>> + logging.debug("ret_host:%s, ret_guest:%s" % (ret_host, ret_guest))
>> + logging.error("Could not get md5, cmd:%s" % cmd)
>> + return False
>> + if ret_host.strip() != ret_guest.strip():
>> + logging.debug("ret_host :%s, ret_guest:%s" % (ret_host, ret_guest))
>> + logging.error("Files' md5sum mismatch" % (receiver))
>> + return False
>
> ^ The above debug messages will be confusing when looked by someone who
> is not familiar with the test code, so we should make their lives
> easier:
>
> def compare(filename):
> cmd = "md5sum %s" % filename
> rc_host, md5_host = commands.getstatusoutput(cmd)
> rc_guest, md5_guest = session.get_command_status_output(cmd)
> if rc_host:
> logging.debug('Could not get MD5 hash for file %s on host, output: %s', filename, md5_host)
> return False
> if rc_guest:
> logging.debug('Could not get MD5 hash for file %s on guest, output: %s', filename, md5_guest)
> return False
> md5host = md5host.strip()
> md5guest = md5guest.strip()
> if md5host != md5guest:
> logging.error('MD5 hash mismatch between file %s present on guest and on host', filename)
> logging.error('MD5 hash for file on guest: %s, MD5 hash for file on host: %s', md5_host, md5_guest)
> return False
> return True
>
>
>> + return True
>> +
>> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
>> + set_promisc_cmd = "ip link set %s promisc on; sleep 0.01;" % ethname
>> + set_promisc_cmd += "ip link set %s promisc off; sleep 0.01" % ethname
>
> ^ You could do the above on a single attribution, see comment on patch 7
> of the patchseries.
>
>> + logging.info("Set promisc change repeatedly in guest")
>> + session2.sendline("while true; do %s; done" % set_promisc_cmd)
>> +
>> + dd_cmd = "dd if=/dev/urandom of=%s bs=%d count=1"
>> + filename = "/tmp/nic_promisc_file"
>> + file_size = params.get("file_size", "1, 1460, 65000, 100000000").split(",")
>> + try:
>> + for size in file_size:
>> + logging.info("Create %s bytes file on host" % size)
>> + s, o = commands.getstatusoutput(dd_cmd % (filename, int(size)))
>> + if s != 0:
>> + logging.debug("Output: %s"% o)
>> + raise error.TestFail("Create file on host failed")
>> +
>> + logging.info("Transfer file from host to guest")
>> + if not vm.copy_files_to(filename, filename):
>> + raise error.TestFail("File transfer failed")
>> + if not compare(filename):
>> + raise error.TestFail("Compare file failed")
>
> ^ It'd be better if we don't abruptly fail the whole test if we get a
> failure for a single size, what about having a global failure counter,
> and increment it if we have failures, making sure we log errors
> appropriately?
How about add a success counter ?
only success_counter+1 after successfully compare.
If the end success_counter doesn't equals to len(file_size), raise and
fail exception.
>> + logging.info("Create %s bytes file on guest" % size)
>> + if session.get_command_status(dd_cmd % (filename, int(size)),
>> + timeout=100) != 0:
>> + raise error.TestFail("Create file on guest failed")
>> +
>> + logging.info("Transfer file from guest to host")
>> + if not vm.copy_files_from(filename, filename):
>> + raise error.TestFail("File transfer failed")
>> + if not compare(filename):
>> + raise error.TestFail("Compare file failed")
>
> ^ Same comment as above.
>
>> + logging.info("Clean temporal files")
>
> ^ Typo, temporary
>
>> + cmd = "rm -f %s" % filename
>> + s1, o = commands.getstatusoutput(cmd)
>> + s2 = session.get_command_status(cmd)
>> + if s1 != 0 or s2 != 0:
>> + raise error.TestError("Fail to clean temporal files")
>
> ^ Same as above, with the exception that maybe a failure during cleanup
> is not something so bad we'd need to fail the entire test because of it.
>
>> + finally:
>> + logging.info("Restore the %s to the nonpromisc mode" % ethname)
>> + session2.close()
>> + session.get_command_status("ip link set %s promisc off" % ethname)
>> + session.close()
>> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
>> index 03d15c0..9e2b9a0 100644
>> --- a/client/tests/kvm/tests_base.cfg.sample
>> +++ b/client/tests/kvm/tests_base.cfg.sample
>> @@ -370,6 +370,10 @@ variants:
>> scp_timeout = 300
>> thread_num = 10
>>
>> + - nic_promisc: install setup unattended_install.cdrom
>> + type = nic_promisc
>> + file_size = 1, 1460, 65000, 100000000
>> +
>> - physical_resources_check: install setup unattended_install.cdrom
>> type = physical_resources_check
>> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
>> @@ -1046,7 +1050,7 @@ variants:
>>
>> # Windows section
>> - @Windows:
>> - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload
>> + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc
>> shutdown_command = shutdown /s /f /t 0
>> reboot_command = shutdown /r /f /t 0
>> status_test_command = echo %errorlevel%
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [RFC PATCH 09/14] KVM-test: Add a subtest of multicast
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:36 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:36 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
Use 'ping' to test send/recive multicat packets. Flood ping test is also added.
Limit guest network as 'bridge' mode, because multicast packets could not be
transmitted to guest when using 'user' network.
Add join_mcast.py for joining machine into multicast groups.
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/scripts/join_mcast.py b/client/tests/kvm/scripts/join_mcast.py
new file mode 100755
index 0000000..0d90e5c
--- /dev/null
+++ b/client/tests/kvm/scripts/join_mcast.py
@@ -0,0 +1,29 @@
+import socket, struct, os, signal, sys
+# this script is used to join machine into multicast groups
+# author: Amos Kong <akong@redhat.com>
+
+if len(sys.argv) < 4:
+ print """%s [mgroup_count] [prefix] [suffix]
+ mgroup_count: count of multicast addresses
+ prefix: multicast address prefix
+ suffix: multicast address suffix""" % sys.argv[0]
+ sys.exit()
+
+mgroup_count = int(sys.argv[1])
+prefix = sys.argv[2]
+suffix = int(sys.argv[3])
+
+s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+for i in range(mgroup_count):
+ mcast = prefix + "." + str(suffix + i)
+ try:
+ mreq = struct.pack("4sl", socket.inet_aton(mcast), socket.INADDR_ANY)
+ s.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
+ except:
+ s.close()
+ print "Could not join multicast: %s" % mcast
+ raise
+
+print "join_mcast_pid:%s" % os.getpid()
+os.kill(os.getpid(), signal.SIGSTOP)
+s.close()
diff --git a/client/tests/kvm/tests/multicast.py b/client/tests/kvm/tests/multicast.py
new file mode 100644
index 0000000..6b0e106
--- /dev/null
+++ b/client/tests/kvm/tests/multicast.py
@@ -0,0 +1,67 @@
+import logging, commands, os, re
+from autotest_lib.client.common_lib import error
+import kvm_test_utils, kvm_net_utils
+
+
+def run_multicast(test, params, env):
+ """
+ Test multicast function of nic (rtl8139/e1000/virtio)
+
+ 1) Create a VM
+ 2) Join guest into multicast groups
+ 3) Ping multicast addresses on host
+ 4) Flood ping test with different size of packets
+ 5) Final ping test and check if lose packet
+
+ @param test: Kvm test object
+ @param params: Dictionary with the test parameters.
+ @param env: Dictionary with test environment.
+ """
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm,
+ timeout=int(params.get("login_timeout", 360)))
+
+ # stop iptable/selinux on guest/host
+ cmd = "/etc/init.d/iptables stop && echo 0 > /selinux/enforce"
+ session.get_command_status(cmd)
+ commands.getoutput(cmd)
+ # make sure guest replies to broadcasts
+ session.get_command_status("echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore"
+ "_broadcasts && echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore_all")
+
+ # base multicast address
+ mcast = params.get("mcast", "225.0.0.1")
+ # count of multicast addresses, less than 20
+ mgroup_count = int(params.get("mgroup_count", 5))
+ flood_minutes = float(params.get("flood_minutes", 10))
+ ifname = vm.get_ifname()
+ prefix = re.findall("\d+.\d+.\d+", mcast)[0]
+ suffix = int(re.findall("\d+", mcast)[-1])
+ # copy python script to guest for joining guest to multicast groups
+ mcast_path = os.path.join(test.bindir, "scripts/join_mcast.py")
+ vm.copy_files_to(mcast_path, "/tmp")
+ output = session.get_command_output("python /tmp/join_mcast.py %d %s %d" %
+ (mgroup_count, prefix, suffix))
+ # if success to join multicast the process will be paused, and return pid.
+ if not re.findall("join_mcast_pid:(\d+)", output):
+ raise error.TestFail("Can't join multicast groups,output:%s" % output)
+ pid = output.split()[0]
+
+ try:
+ for i in range(mgroup_count):
+ new_suffix = suffix + i
+ mcast = "%s.%d" % (prefix, new_suffix)
+ logging.info("Initial ping test, mcast: %s", mcast)
+ s, o = kvm_net_utils.ping(mcast, 10, interface=ifname, timeout=20)
+ if s != 0:
+ raise error.TestFail(" Ping return non-zero value %s" % o)
+ logging.info("Flood ping test, mcast: %s", mcast)
+ kvm_net_utils.ping(mcast, None, interface=ifname, flood=True,
+ output_func=None, timeout=flood_minutes*60)
+ logging.info("Final ping test, mcast: %s", mcast)
+ s, o = kvm_net_utils.ping(mcast, 10, interface=ifname, timeout=20)
+ if s != 0:
+ raise error.TestFail(" Ping return non-zero value %s" % o)
+ finally:
+ session.get_command_output("kill -s SIGCONT %s" % pid)
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 9e2b9a0..9594a38 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -374,6 +374,13 @@ variants:
type = nic_promisc
file_size = 1, 1460, 65000, 100000000
+ - multicast: install setup unattended_install.cdrom
+ type = multicast
+ nic_mode = tap
+ mcast = 225.0.0.1
+ mgroup_count = 20
+ flood_minutes = 1
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1050,7 +1057,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc
+ no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 09/14] KVM-test: Add a subtest of multicast
@ 2010-07-20 1:36 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:36 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
Use 'ping' to test send/recive multicat packets. Flood ping test is also added.
Limit guest network as 'bridge' mode, because multicast packets could not be
transmitted to guest when using 'user' network.
Add join_mcast.py for joining machine into multicast groups.
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/scripts/join_mcast.py b/client/tests/kvm/scripts/join_mcast.py
new file mode 100755
index 0000000..0d90e5c
--- /dev/null
+++ b/client/tests/kvm/scripts/join_mcast.py
@@ -0,0 +1,29 @@
+import socket, struct, os, signal, sys
+# this script is used to join machine into multicast groups
+# author: Amos Kong <akong@redhat.com>
+
+if len(sys.argv) < 4:
+ print """%s [mgroup_count] [prefix] [suffix]
+ mgroup_count: count of multicast addresses
+ prefix: multicast address prefix
+ suffix: multicast address suffix""" % sys.argv[0]
+ sys.exit()
+
+mgroup_count = int(sys.argv[1])
+prefix = sys.argv[2]
+suffix = int(sys.argv[3])
+
+s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+for i in range(mgroup_count):
+ mcast = prefix + "." + str(suffix + i)
+ try:
+ mreq = struct.pack("4sl", socket.inet_aton(mcast), socket.INADDR_ANY)
+ s.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
+ except:
+ s.close()
+ print "Could not join multicast: %s" % mcast
+ raise
+
+print "join_mcast_pid:%s" % os.getpid()
+os.kill(os.getpid(), signal.SIGSTOP)
+s.close()
diff --git a/client/tests/kvm/tests/multicast.py b/client/tests/kvm/tests/multicast.py
new file mode 100644
index 0000000..6b0e106
--- /dev/null
+++ b/client/tests/kvm/tests/multicast.py
@@ -0,0 +1,67 @@
+import logging, commands, os, re
+from autotest_lib.client.common_lib import error
+import kvm_test_utils, kvm_net_utils
+
+
+def run_multicast(test, params, env):
+ """
+ Test multicast function of nic (rtl8139/e1000/virtio)
+
+ 1) Create a VM
+ 2) Join guest into multicast groups
+ 3) Ping multicast addresses on host
+ 4) Flood ping test with different size of packets
+ 5) Final ping test and check if lose packet
+
+ @param test: Kvm test object
+ @param params: Dictionary with the test parameters.
+ @param env: Dictionary with test environment.
+ """
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm,
+ timeout=int(params.get("login_timeout", 360)))
+
+ # stop iptable/selinux on guest/host
+ cmd = "/etc/init.d/iptables stop && echo 0 > /selinux/enforce"
+ session.get_command_status(cmd)
+ commands.getoutput(cmd)
+ # make sure guest replies to broadcasts
+ session.get_command_status("echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore"
+ "_broadcasts && echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore_all")
+
+ # base multicast address
+ mcast = params.get("mcast", "225.0.0.1")
+ # count of multicast addresses, less than 20
+ mgroup_count = int(params.get("mgroup_count", 5))
+ flood_minutes = float(params.get("flood_minutes", 10))
+ ifname = vm.get_ifname()
+ prefix = re.findall("\d+.\d+.\d+", mcast)[0]
+ suffix = int(re.findall("\d+", mcast)[-1])
+ # copy python script to guest for joining guest to multicast groups
+ mcast_path = os.path.join(test.bindir, "scripts/join_mcast.py")
+ vm.copy_files_to(mcast_path, "/tmp")
+ output = session.get_command_output("python /tmp/join_mcast.py %d %s %d" %
+ (mgroup_count, prefix, suffix))
+ # if success to join multicast the process will be paused, and return pid.
+ if not re.findall("join_mcast_pid:(\d+)", output):
+ raise error.TestFail("Can't join multicast groups,output:%s" % output)
+ pid = output.split()[0]
+
+ try:
+ for i in range(mgroup_count):
+ new_suffix = suffix + i
+ mcast = "%s.%d" % (prefix, new_suffix)
+ logging.info("Initial ping test, mcast: %s", mcast)
+ s, o = kvm_net_utils.ping(mcast, 10, interface=ifname, timeout=20)
+ if s != 0:
+ raise error.TestFail(" Ping return non-zero value %s" % o)
+ logging.info("Flood ping test, mcast: %s", mcast)
+ kvm_net_utils.ping(mcast, None, interface=ifname, flood=True,
+ output_func=None, timeout=flood_minutes*60)
+ logging.info("Final ping test, mcast: %s", mcast)
+ s, o = kvm_net_utils.ping(mcast, 10, interface=ifname, timeout=20)
+ if s != 0:
+ raise error.TestFail(" Ping return non-zero value %s" % o)
+ finally:
+ session.get_command_output("kill -s SIGCONT %s" % pid)
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 9e2b9a0..9594a38 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -374,6 +374,13 @@ variants:
type = nic_promisc
file_size = 1, 1460, 65000, 100000000
+ - multicast: install setup unattended_install.cdrom
+ type = multicast
+ nic_mode = tap
+ mcast = 225.0.0.1
+ mgroup_count = 20
+ flood_minutes = 1
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1050,7 +1057,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc
+ no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 09/14] KVM-test: Add a subtest of multicast
2010-07-20 1:36 ` [Qemu-devel] " Amos Kong
(?)
@ 2010-07-28 21:55 ` Lucas Meneghel Rodrigues
-1 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-28 21:55 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:36 +0800, Amos Kong wrote:
> Use 'ping' to test send/recive multicat packets. Flood ping test is also added.
> Limit guest network as 'bridge' mode, because multicast packets could not be
> transmitted to guest when using 'user' network.
> Add join_mcast.py for joining machine into multicast groups.
>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/scripts/join_mcast.py b/client/tests/kvm/scripts/join_mcast.py
> new file mode 100755
> index 0000000..0d90e5c
> --- /dev/null
> +++ b/client/tests/kvm/scripts/join_mcast.py
> @@ -0,0 +1,29 @@
> +import socket, struct, os, signal, sys
> +# this script is used to join machine into multicast groups
> +# author: Amos Kong <akong@redhat.com>
> +
> +if len(sys.argv) < 4:
> + print """%s [mgroup_count] [prefix] [suffix]
> + mgroup_count: count of multicast addresses
> + prefix: multicast address prefix
> + suffix: multicast address suffix""" % sys.argv[0]
> + sys.exit()
> +
> +mgroup_count = int(sys.argv[1])
> +prefix = sys.argv[2]
> +suffix = int(sys.argv[3])
> +
> +s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
> +for i in range(mgroup_count):
> + mcast = prefix + "." + str(suffix + i)
> + try:
> + mreq = struct.pack("4sl", socket.inet_aton(mcast), socket.INADDR_ANY)
> + s.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
> + except:
> + s.close()
> + print "Could not join multicast: %s" % mcast
> + raise
> +
> +print "join_mcast_pid:%s" % os.getpid()
> +os.kill(os.getpid(), signal.SIGSTOP)
> +s.close()
> diff --git a/client/tests/kvm/tests/multicast.py b/client/tests/kvm/tests/multicast.py
> new file mode 100644
> index 0000000..6b0e106
> --- /dev/null
> +++ b/client/tests/kvm/tests/multicast.py
> @@ -0,0 +1,67 @@
> +import logging, commands, os, re
> +from autotest_lib.client.common_lib import error
> +import kvm_test_utils, kvm_net_utils
> +
> +
> +def run_multicast(test, params, env):
> + """
> + Test multicast function of nic (rtl8139/e1000/virtio)
> +
> + 1) Create a VM
> + 2) Join guest into multicast groups
> + 3) Ping multicast addresses on host
> + 4) Flood ping test with different size of packets
> + 5) Final ping test and check if lose packet
> +
> + @param test: Kvm test object
> + @param params: Dictionary with the test parameters.
> + @param env: Dictionary with test environment.
> + """
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + session = kvm_test_utils.wait_for_login(vm,
> + timeout=int(params.get("login_timeout", 360)))
> +
> + # stop iptable/selinux on guest/host
> + cmd = "/etc/init.d/iptables stop && echo 0 > /selinux/enforce"
^ Different linux distros handle init scripts differently, so it's
better to just flush the firewall rules with iptables -F instead of
resorting to the init script.
Also, not all distros use selinux, so it'd be better to test for the
presence of selinux on the system (for example, by checking if there's
a /selinux/enforce file before trying to write to it). Also, need to do
proper return code checking/treatment.
> + session.get_command_status(cmd)
> + commands.getoutput(cmd)
> + # make sure guest replies to broadcasts
> + session.get_command_status("echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore"
> + "_broadcasts && echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore_all")
> +
> + # base multicast address
> + mcast = params.get("mcast", "225.0.0.1")
> + # count of multicast addresses, less than 20
> + mgroup_count = int(params.get("mgroup_count", 5))
> + flood_minutes = float(params.get("flood_minutes", 10))
> + ifname = vm.get_ifname()
> + prefix = re.findall("\d+.\d+.\d+", mcast)[0]
> + suffix = int(re.findall("\d+", mcast)[-1])
> + # copy python script to guest for joining guest to multicast groups
> + mcast_path = os.path.join(test.bindir, "scripts/join_mcast.py")
> + vm.copy_files_to(mcast_path, "/tmp")
^ What if copy_files_to fails? Need to do proper return handling here
> + output = session.get_command_output("python /tmp/join_mcast.py %d %s %d" %
> + (mgroup_count, prefix, suffix))
> + # if success to join multicast the process will be paused, and return pid.
> + if not re.findall("join_mcast_pid:(\d+)", output):
> + raise error.TestFail("Can't join multicast groups,output:%s" % output)
> + pid = output.split()[0]
> +
> + try:
> + for i in range(mgroup_count):
> + new_suffix = suffix + i
> + mcast = "%s.%d" % (prefix, new_suffix)
> + logging.info("Initial ping test, mcast: %s", mcast)
> + s, o = kvm_net_utils.ping(mcast, 10, interface=ifname, timeout=20)
> + if s != 0:
> + raise error.TestFail(" Ping return non-zero value %s" % o)
> + logging.info("Flood ping test, mcast: %s", mcast)
> + kvm_net_utils.ping(mcast, None, interface=ifname, flood=True,
> + output_func=None, timeout=flood_minutes*60)
> + logging.info("Final ping test, mcast: %s", mcast)
> + s, o = kvm_net_utils.ping(mcast, 10, interface=ifname, timeout=20)
> + if s != 0:
> + raise error.TestFail(" Ping return non-zero value %s" % o)
^ "Ping failed, status: %s, output: %s" % (s, o) would be a better
exception
> + finally:
> + session.get_command_output("kill -s SIGCONT %s" % pid)
> + session.close()
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 9e2b9a0..9594a38 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -374,6 +374,13 @@ variants:
> type = nic_promisc
> file_size = 1, 1460, 65000, 100000000
>
> + - multicast: install setup unattended_install.cdrom
> + type = multicast
> + nic_mode = tap
> + mcast = 225.0.0.1
> + mgroup_count = 20
> + flood_minutes = 1
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> @@ -1050,7 +1057,7 @@ variants:
>
> # Windows section
> - @Windows:
> - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc
> + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast
> shutdown_command = shutdown /s /f /t 0
> reboot_command = shutdown /r /f /t 0
> status_test_command = echo %errorlevel%
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [RFC PATCH 10/14] KVM-test: Add a subtest of pxe
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:36 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:36 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
This case just snoop tftp packet through tcpdump, it depends on public dhcp
server, better to test it through dnsmasq.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/pxe.py b/client/tests/kvm/tests/pxe.py
new file mode 100644
index 0000000..8859aaa
--- /dev/null
+++ b/client/tests/kvm/tests/pxe.py
@@ -0,0 +1,30 @@
+import logging
+from autotest_lib.client.common_lib import error
+import kvm_subprocess, kvm_test_utils, kvm_utils
+
+
+def run_pxe(test, params, env):
+ """
+ PXE test:
+
+ 1) Snoop the tftp packet in the tap device
+ 2) Wait for some seconds
+ 3) Check whether capture tftp packets
+
+ @param test: kvm test object
+ @param params: Dictionary with the test parameters
+ @param env: Dictionary with test environment.
+ """
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ timeout = int(params.get("pxe_timeout", 60))
+
+ logging.info("Try to boot from pxe")
+ status, output = kvm_subprocess.run_fg("tcpdump -nli %s" % vm.get_ifname(),
+ logging.debug,
+ "(pxe) ",
+ timeout)
+
+ logging.info("Analysing the tcpdump result...")
+ if not "tftp" in output:
+ raise error.TestFail("Couldn't find tftp packet in %s seconds" % timeout)
+ logging.info("Found tftp packet")
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 9594a38..5515601 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -381,6 +381,19 @@ variants:
mgroup_count = 20
flood_minutes = 1
+ - pxe:
+ type = pxe
+ images = pxe
+ image_name_pxe = pxe-test
+ image_size_pxe = 1G
+ force_create_image_pxe = yes
+ remove_image_pxe = yes
+ extra_params += ' -boot n'
+ kill_vm_on_error = yes
+ network = bridge
+ restart_vm = yes
+ pxe_timeout = 60
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 10/14] KVM-test: Add a subtest of pxe
@ 2010-07-20 1:36 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:36 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
This case just snoop tftp packet through tcpdump, it depends on public dhcp
server, better to test it through dnsmasq.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/pxe.py b/client/tests/kvm/tests/pxe.py
new file mode 100644
index 0000000..8859aaa
--- /dev/null
+++ b/client/tests/kvm/tests/pxe.py
@@ -0,0 +1,30 @@
+import logging
+from autotest_lib.client.common_lib import error
+import kvm_subprocess, kvm_test_utils, kvm_utils
+
+
+def run_pxe(test, params, env):
+ """
+ PXE test:
+
+ 1) Snoop the tftp packet in the tap device
+ 2) Wait for some seconds
+ 3) Check whether capture tftp packets
+
+ @param test: kvm test object
+ @param params: Dictionary with the test parameters
+ @param env: Dictionary with test environment.
+ """
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ timeout = int(params.get("pxe_timeout", 60))
+
+ logging.info("Try to boot from pxe")
+ status, output = kvm_subprocess.run_fg("tcpdump -nli %s" % vm.get_ifname(),
+ logging.debug,
+ "(pxe) ",
+ timeout)
+
+ logging.info("Analysing the tcpdump result...")
+ if not "tftp" in output:
+ raise error.TestFail("Couldn't find tftp packet in %s seconds" % timeout)
+ logging.info("Found tftp packet")
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 9594a38..5515601 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -381,6 +381,19 @@ variants:
mgroup_count = 20
flood_minutes = 1
+ - pxe:
+ type = pxe
+ images = pxe
+ image_name_pxe = pxe-test
+ image_size_pxe = 1G
+ force_create_image_pxe = yes
+ remove_image_pxe = yes
+ extra_params += ' -boot n'
+ kill_vm_on_error = yes
+ network = bridge
+ restart_vm = yes
+ pxe_timeout = 60
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 10/14] KVM-test: Add a subtest of pxe
2010-07-20 1:36 ` [Qemu-devel] " Amos Kong
(?)
@ 2010-07-28 22:07 ` Lucas Meneghel Rodrigues
2010-08-10 6:11 ` Amos Kong
-1 siblings, 1 reply; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-28 22:07 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:36 +0800, Amos Kong wrote:
> This case just snoop tftp packet through tcpdump, it depends on public dhcp
> server, better to test it through dnsmasq.
It would be a good idea to have an alternate implementation using
dnsmasq, but not urgent.
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/pxe.py b/client/tests/kvm/tests/pxe.py
> new file mode 100644
> index 0000000..8859aaa
> --- /dev/null
> +++ b/client/tests/kvm/tests/pxe.py
> @@ -0,0 +1,30 @@
> +import logging
> +from autotest_lib.client.common_lib import error
> +import kvm_subprocess, kvm_test_utils, kvm_utils
> +
> +
> +def run_pxe(test, params, env):
> + """
> + PXE test:
> +
> + 1) Snoop the tftp packet in the tap device
> + 2) Wait for some seconds
> + 3) Check whether capture tftp packets
> +
> + @param test: kvm test object
> + @param params: Dictionary with the test parameters
> + @param env: Dictionary with test environment.
> + """
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + timeout = int(params.get("pxe_timeout", 60))
> +
> + logging.info("Try to boot from pxe")
> + status, output = kvm_subprocess.run_fg("tcpdump -nli %s" % vm.get_ifname(),
> + logging.debug,
> + "(pxe) ",
> + timeout)
^ The only complaint I could make here is that since this command
doesn't need to live throughout tests, utils.run would do just fine.
Other than that, looks fine to me.
> + logging.info("Analysing the tcpdump result...")
^ typo, analyzing
> + if not "tftp" in output:
> + raise error.TestFail("Couldn't find tftp packet in %s seconds" % timeout)
> + logging.info("Found tftp packet")
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 9594a38..5515601 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -381,6 +381,19 @@ variants:
> mgroup_count = 20
> flood_minutes = 1
>
> + - pxe:
> + type = pxe
> + images = pxe
> + image_name_pxe = pxe-test
> + image_size_pxe = 1G
> + force_create_image_pxe = yes
> + remove_image_pxe = yes
> + extra_params += ' -boot n'
> + kill_vm_on_error = yes
> + network = bridge
> + restart_vm = yes
> + pxe_timeout = 60
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 10/14] KVM-test: Add a subtest of pxe
2010-07-28 22:07 ` Lucas Meneghel Rodrigues
@ 2010-08-10 6:11 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-10 6:11 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: autotest, qemu-devel, kvm
On Wed, Jul 28, 2010 at 07:07:34PM -0300, Lucas Meneghel Rodrigues wrote:
> On Tue, 2010-07-20 at 09:36 +0800, Amos Kong wrote:
> > This case just snoop tftp packet through tcpdump, it depends on public dhcp
> > server, better to test it through dnsmasq.
>
> It would be a good idea to have an alternate implementation using
> dnsmasq, but not urgent.
I changed unattended_install to use dnsmasq, but it was blocked by a pxe bug.
Also need completed test, we can add this later.
> > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > Signed-off-by: Amos Kong <akong@redhat.com>
> > ---
> > 0 files changed, 0 insertions(+), 0 deletions(-)
> >
> > diff --git a/client/tests/kvm/tests/pxe.py b/client/tests/kvm/tests/pxe.py
> > new file mode 100644
> > index 0000000..8859aaa
> > --- /dev/null
> > +++ b/client/tests/kvm/tests/pxe.py
> > @@ -0,0 +1,30 @@
> > +import logging
> > +from autotest_lib.client.common_lib import error
> > +import kvm_subprocess, kvm_test_utils, kvm_utils
> > +
> > +
> > +def run_pxe(test, params, env):
> > + """
> > + PXE test:
> > +
> > + 1) Snoop the tftp packet in the tap device
> > + 2) Wait for some seconds
> > + 3) Check whether capture tftp packets
> > +
> > + @param test: kvm test object
> > + @param params: Dictionary with the test parameters
> > + @param env: Dictionary with test environment.
> > + """
> > + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> > + timeout = int(params.get("pxe_timeout", 60))
> > +
> > + logging.info("Try to boot from pxe")
> > + status, output = kvm_subprocess.run_fg("tcpdump -nli %s" % vm.get_ifname(),
> > + logging.debug,
> > + "(pxe) ",
> > + timeout)
>
> ^ The only complaint I could make here is that since this command
> doesn't need to live throughout tests, utils.run would do just fine.
> Other than that, looks fine to me.
utils.run() desen't support timeout, tcpdump would not stop by itself.
also could not add output prefix "(pxe) " ?
> > + logging.info("Analysing the tcpdump result...")
>
> ^ typo, analyzing
>
> > + if not "tftp" in output:
> > + raise error.TestFail("Couldn't find tftp packet in %s seconds" % timeout)
> > + logging.info("Found tftp packet")
> > diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> > index 9594a38..5515601 100644
> > --- a/client/tests/kvm/tests_base.cfg.sample
> > +++ b/client/tests/kvm/tests_base.cfg.sample
> > @@ -381,6 +381,19 @@ variants:
> > mgroup_count = 20
> > flood_minutes = 1
> >
> > + - pxe:
> > + type = pxe
> > + images = pxe
> > + image_name_pxe = pxe-test
> > + image_size_pxe = 1G
> > + force_create_image_pxe = yes
> > + remove_image_pxe = yes
> > + extra_params += ' -boot n'
> > + kill_vm_on_error = yes
> > + network = bridge
> > + restart_vm = yes
> > + pxe_timeout = 60
> > +
> > - physical_resources_check: install setup unattended_install.cdrom
> > type = physical_resources_check
> > catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> >
> >
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [RFC PATCH 11/14] KVM-test: Add a subtest of changing mac address
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:36 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:36 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
Mainly test steps:
1. get a new mac from pool, and the old mac addr of guest.
2. execute the mac_change.sh in guest.
3. relogin to guest and query the interfaces info by `ifconfig`
Signed-off-by: Cao, Chen <kcao@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/mac_change.py b/client/tests/kvm/tests/mac_change.py
new file mode 100644
index 0000000..dc93377
--- /dev/null
+++ b/client/tests/kvm/tests/mac_change.py
@@ -0,0 +1,66 @@
+import logging
+from autotest_lib.client.common_lib import error
+import kvm_utils, kvm_test_utils, kvm_net_utils
+
+
+def run_mac_change(test, params, env):
+ """
+ Change MAC Address of Guest.
+
+ 1. get a new mac from pool, and the old mac addr of guest.
+ 2. set new mac in guest and regain new IP.
+ 3. re-log into guest with new mac
+
+ @param test: kvm test object
+ @param params: Dictionary with the test parameters
+ @param env: Dictionary with test environment.
+ """
+ timeout = int(params.get("login_timeout", 360))
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ logging.info("Trying to log into guest '%s' by serial", vm.name)
+ session = kvm_utils.wait_for(lambda: vm.serial_login(),
+ timeout, 0, step=2)
+ if not session:
+ raise error.TestFail("Could not log into guest '%s'" % vm.name)
+
+ old_mac = vm.get_macaddr(0)
+ kvm_utils.put_mac_to_pool(vm.root_dir, old_mac, vm.instance)
+ new_mac = kvm_utils.get_mac_from_pool(vm.root_dir,
+ vm=vm.instance,
+ nic_index=0,
+ prefix=vm.mac_prefix)
+ logging.info("The initial MAC address is %s" % old_mac)
+ interface = kvm_net_utils.get_linux_ifname(session, old_mac)
+
+ # Start change mac address
+ logging.info("Changing mac address to %s" % new_mac)
+ change_cmd = "ifconfig %s down && ifconfig %s hw ether %s && ifconfig %s up"\
+ % (interface, interface, new_mac, interface)
+ if session.get_command_status(change_cmd) != 0:
+ raise error.TestFail("Fail to send mac_change command")
+
+ # Verify whether mac address is changed to new one
+ logging.info("Verifying the new mac address")
+ if session.get_command_status("ifconfig | grep -i %s" % new_mac) != 0:
+ raise error.TestFail("Fail to change mac address")
+
+ # Restart `dhclient' to regain IP for new mac address
+ logging.info("Re-start the network to gain new ip")
+ dhclient_cmd = "dhclient -r && dhclient %s" % interface
+ session.sendline(dhclient_cmd)
+
+ # Re-log into the guest after changing mac address
+ if kvm_utils.wait_for(session.is_responsive, 120, 20, 3):
+ # Just warning when failed to see the session become dead,
+ # because there is a little chance the ip does not change.
+ logging.warn("The session is still responsive, settings may fail.")
+ session.close()
+
+ # Re-log into guest and check if session is responsive
+ logging.info("Re-log into the guest")
+ session = kvm_test_utils.wait_for_login(vm,
+ timeout=int(params.get("login_timeout", 360)))
+ if not session.is_responsive():
+ raise error.TestFail("The new session is not responsive.")
+
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 5515601..7716d48 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -394,6 +394,10 @@ variants:
restart_vm = yes
pxe_timeout = 60
+ - mac_change: install setup unattended_install.cdrom
+ type = mac_change
+ kill_vm = yes
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1070,7 +1074,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast
+ no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 11/14] KVM-test: Add a subtest of changing mac address
@ 2010-07-20 1:36 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:36 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
Mainly test steps:
1. get a new mac from pool, and the old mac addr of guest.
2. execute the mac_change.sh in guest.
3. relogin to guest and query the interfaces info by `ifconfig`
Signed-off-by: Cao, Chen <kcao@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/mac_change.py b/client/tests/kvm/tests/mac_change.py
new file mode 100644
index 0000000..dc93377
--- /dev/null
+++ b/client/tests/kvm/tests/mac_change.py
@@ -0,0 +1,66 @@
+import logging
+from autotest_lib.client.common_lib import error
+import kvm_utils, kvm_test_utils, kvm_net_utils
+
+
+def run_mac_change(test, params, env):
+ """
+ Change MAC Address of Guest.
+
+ 1. get a new mac from pool, and the old mac addr of guest.
+ 2. set new mac in guest and regain new IP.
+ 3. re-log into guest with new mac
+
+ @param test: kvm test object
+ @param params: Dictionary with the test parameters
+ @param env: Dictionary with test environment.
+ """
+ timeout = int(params.get("login_timeout", 360))
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ logging.info("Trying to log into guest '%s' by serial", vm.name)
+ session = kvm_utils.wait_for(lambda: vm.serial_login(),
+ timeout, 0, step=2)
+ if not session:
+ raise error.TestFail("Could not log into guest '%s'" % vm.name)
+
+ old_mac = vm.get_macaddr(0)
+ kvm_utils.put_mac_to_pool(vm.root_dir, old_mac, vm.instance)
+ new_mac = kvm_utils.get_mac_from_pool(vm.root_dir,
+ vm=vm.instance,
+ nic_index=0,
+ prefix=vm.mac_prefix)
+ logging.info("The initial MAC address is %s" % old_mac)
+ interface = kvm_net_utils.get_linux_ifname(session, old_mac)
+
+ # Start change mac address
+ logging.info("Changing mac address to %s" % new_mac)
+ change_cmd = "ifconfig %s down && ifconfig %s hw ether %s && ifconfig %s up"\
+ % (interface, interface, new_mac, interface)
+ if session.get_command_status(change_cmd) != 0:
+ raise error.TestFail("Fail to send mac_change command")
+
+ # Verify whether mac address is changed to new one
+ logging.info("Verifying the new mac address")
+ if session.get_command_status("ifconfig | grep -i %s" % new_mac) != 0:
+ raise error.TestFail("Fail to change mac address")
+
+ # Restart `dhclient' to regain IP for new mac address
+ logging.info("Re-start the network to gain new ip")
+ dhclient_cmd = "dhclient -r && dhclient %s" % interface
+ session.sendline(dhclient_cmd)
+
+ # Re-log into the guest after changing mac address
+ if kvm_utils.wait_for(session.is_responsive, 120, 20, 3):
+ # Just warning when failed to see the session become dead,
+ # because there is a little chance the ip does not change.
+ logging.warn("The session is still responsive, settings may fail.")
+ session.close()
+
+ # Re-log into guest and check if session is responsive
+ logging.info("Re-log into the guest")
+ session = kvm_test_utils.wait_for_login(vm,
+ timeout=int(params.get("login_timeout", 360)))
+ if not session.is_responsive():
+ raise error.TestFail("The new session is not responsive.")
+
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 5515601..7716d48 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -394,6 +394,10 @@ variants:
restart_vm = yes
pxe_timeout = 60
+ - mac_change: install setup unattended_install.cdrom
+ type = mac_change
+ kill_vm = yes
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1070,7 +1074,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast
+ no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 11/14] KVM-test: Add a subtest of changing mac address
2010-07-20 1:36 ` [Qemu-devel] " Amos Kong
(?)
@ 2010-07-28 22:30 ` Lucas Meneghel Rodrigues
-1 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-28 22:30 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:36 +0800, Amos Kong wrote:
> Mainly test steps:
> 1. get a new mac from pool, and the old mac addr of guest.
> 2. execute the mac_change.sh in guest.
> 3. relogin to guest and query the interfaces info by `ifconfig`
>
> Signed-off-by: Cao, Chen <kcao@redhat.com>
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/mac_change.py b/client/tests/kvm/tests/mac_change.py
> new file mode 100644
> index 0000000..dc93377
> --- /dev/null
> +++ b/client/tests/kvm/tests/mac_change.py
> @@ -0,0 +1,66 @@
> +import logging
> +from autotest_lib.client.common_lib import error
> +import kvm_utils, kvm_test_utils, kvm_net_utils
> +
> +
> +def run_mac_change(test, params, env):
> + """
> + Change MAC Address of Guest.
> +
> + 1. get a new mac from pool, and the old mac addr of guest.
> + 2. set new mac in guest and regain new IP.
> + 3. re-log into guest with new mac
> +
> + @param test: kvm test object
> + @param params: Dictionary with the test parameters
> + @param env: Dictionary with test environment.
> + """
> + timeout = int(params.get("login_timeout", 360))
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + logging.info("Trying to log into guest '%s' by serial", vm.name)
> + session = kvm_utils.wait_for(lambda: vm.serial_login(),
> + timeout, 0, step=2)
^ One thing that I forgot to comment on previous patches: For more
clarity, it'd be good to name the session variable in a way that lets
people refer easily that is a serial session
session_serial = ...
> + if not session:
> + raise error.TestFail("Could not log into guest '%s'" % vm.name)
> +
> + old_mac = vm.get_macaddr(0)
> + kvm_utils.put_mac_to_pool(vm.root_dir, old_mac, vm.instance)
> + new_mac = kvm_utils.get_mac_from_pool(vm.root_dir,
> + vm=vm.instance,
> + nic_index=0,
> + prefix=vm.mac_prefix)
> + logging.info("The initial MAC address is %s" % old_mac)
> + interface = kvm_net_utils.get_linux_ifname(session, old_mac)
> +
> + # Start change mac address
> + logging.info("Changing mac address to %s" % new_mac)
> + change_cmd = "ifconfig %s down && ifconfig %s hw ether %s && ifconfig %s up"\
> + % (interface, interface, new_mac, interface)
> + if session.get_command_status(change_cmd) != 0:
> + raise error.TestFail("Fail to send mac_change command")
> +
> + # Verify whether mac address is changed to new one
> + logging.info("Verifying the new mac address")
> + if session.get_command_status("ifconfig | grep -i %s" % new_mac) != 0:
> + raise error.TestFail("Fail to change mac address")
> +
> + # Restart `dhclient' to regain IP for new mac address
> + logging.info("Re-start the network to gain new ip")
> + dhclient_cmd = "dhclient -r && dhclient %s" % interface
> + session.sendline(dhclient_cmd)
> +
> + # Re-log into the guest after changing mac address
> + if kvm_utils.wait_for(session.is_responsive, 120, 20, 3):
> + # Just warning when failed to see the session become dead,
> + # because there is a little chance the ip does not change.
> + logging.warn("The session is still responsive, settings may fail.")
^ Isn't this a serial session? Then why the IP of guest changing would
make this session un-responsive? I think the best idea here is to:
1) Release the IP through dhclient -r [interface]
2) Make sure we can't stablish a ssh based session to the guest by
making a try/except block with kvm_test_utils.wait_for_login() with the
appropriate timeouts and other parameters, if succeeds, fail the test,
if it doesn't, proceed with the test.
3) Get a new IP with dhclient [interface]
4) Try to stablish a new, ssh based session to the guest and see if that
works.
> + session.close()
> +
> + # Re-log into guest and check if session is responsive
> + logging.info("Re-log into the guest")
> + session = kvm_test_utils.wait_for_login(vm,
> + timeout=int(params.get("login_timeout", 360)))
> + if not session.is_responsive():
> + raise error.TestFail("The new session is not responsive.")
^ Is it possible that right after you stablish the session it becomes
non-responsive? It seems like a redundant verification step to me.
> + session.close()
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 5515601..7716d48 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -394,6 +394,10 @@ variants:
> restart_vm = yes
> pxe_timeout = 60
>
> + - mac_change: install setup unattended_install.cdrom
> + type = mac_change
> + kill_vm = yes
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> @@ -1070,7 +1074,7 @@ variants:
>
> # Windows section
> - @Windows:
> - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast
> + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
> shutdown_command = shutdown /s /f /t 0
> reboot_command = shutdown /r /f /t 0
> status_test_command = echo %errorlevel%
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [RFC PATCH 12/14] KVM-test: Add a subtest of netperf
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:36 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:36 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
Add network load by netperf, server is launched on guest, execute netperf
client with different protocols on host. if all clients execute successfully,
case will be pass. Test result will be record into result.txt.
Now this case only tests with "TCP_RR TCP_CRR UDP_RR TCP_STREAM TCP_MAERTS
TCP_SENDFILE UDP_STREAM". DLPI only supported by Unix, unix domain test is
not necessary, so drop test of DLPI and unix domain.
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/netperf.py b/client/tests/kvm/tests/netperf.py
new file mode 100644
index 0000000..00a91f0
--- /dev/null
+++ b/client/tests/kvm/tests/netperf.py
@@ -0,0 +1,56 @@
+import logging, commands, os
+from autotest_lib.client.common_lib import error
+import kvm_subprocess, kvm_test_utils, kvm_utils
+
+def run_netperf(test, params, env):
+ """
+ Network stress test with netperf
+
+ 1) Boot up a virtual machine
+ 2) Launch netserver on guest
+ 3) Execute netperf client on host with different protocols
+ 4) Outout the test result
+
+ @param test: Kvm test object
+ @param params: Dictionary with the test parameters.
+ @param env: Dictionary with test environment.
+ """
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm,
+ timeout=int(params.get("login_timeout", 360)))
+ netperf_dir = os.path.join(os.environ['AUTODIR'], "tests/netperf2")
+ setup_cmd = params.get("setup_cmd")
+ guest_ip = vm.get_address()
+ result_file = os.path.join(test.debugdir, "result.txt")
+
+ session.get_command_output("service iptables stop")
+ for i in params.get("netperf_files").split():
+ if not vm.copy_files_to(os.path.join(netperf_dir, i), "/tmp"):
+ raise error.TestError("Could not copy files to guest")
+ if session.get_command_status(setup_cmd % "/tmp", timeout=100) != 0:
+ raise error.TestFail("Fail to setup netperf on guest")
+ if session.get_command_status(params.get("netserver_cmd") % "/tmp") != 0:
+ raise error.TestFail("Fail to start netperf server on guest")
+
+ try:
+ logging.info("Setup and run netperf client on host")
+ s, o = commands.getstatusoutput(setup_cmd % netperf_dir)
+ if s != 0:
+ raise error.TestFail("Fail to setup netperf on host, o: %s" % o)
+ success = True
+ file(result_file, "w").write("Netperf Test Result\n")
+ for i in params.get("protocols").split():
+ cmd = params.get("netperf_cmd") % (netperf_dir, i, guest_ip)
+ logging.debug("Execute netperf client test: %s" % cmd)
+ s, o = commands.getstatusoutput(cmd)
+ if s != 0:
+ logging.error("Fail to execute netperf test, protocol:%s" % i)
+ success = False
+ else:
+ logging.info(o)
+ file(result_file, "a+").write("%s\n" % o)
+ if not success:
+ raise error.TestFail("Not all the test passed")
+ finally:
+ session.get_command_output("killall netserver")
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 7716d48..dec988e 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -398,6 +398,16 @@ variants:
type = mac_change
kill_vm = yes
+ - netperf: install setup unattended_install.cdrom
+ type = netperf
+ nic_mode = tap
+ netperf_files = netperf-2.4.5.tar.bz2 wait_before_data.patch
+ setup_cmd = "cd %s && tar xvfj netperf-2.4.5.tar.bz2 && cd netperf-2.4.5 && patch -p0 < ../wait_before_data.patch && ./configure && make"
+ netserver_cmd = %s/netperf-2.4.5/src/netserver
+ # test time is 60 seconds, set the buffer size to 1 for more hardware interrupt
+ netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1
+ protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM"
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 12/14] KVM-test: Add a subtest of netperf
@ 2010-07-20 1:36 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:36 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
Add network load by netperf, server is launched on guest, execute netperf
client with different protocols on host. if all clients execute successfully,
case will be pass. Test result will be record into result.txt.
Now this case only tests with "TCP_RR TCP_CRR UDP_RR TCP_STREAM TCP_MAERTS
TCP_SENDFILE UDP_STREAM". DLPI only supported by Unix, unix domain test is
not necessary, so drop test of DLPI and unix domain.
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/netperf.py b/client/tests/kvm/tests/netperf.py
new file mode 100644
index 0000000..00a91f0
--- /dev/null
+++ b/client/tests/kvm/tests/netperf.py
@@ -0,0 +1,56 @@
+import logging, commands, os
+from autotest_lib.client.common_lib import error
+import kvm_subprocess, kvm_test_utils, kvm_utils
+
+def run_netperf(test, params, env):
+ """
+ Network stress test with netperf
+
+ 1) Boot up a virtual machine
+ 2) Launch netserver on guest
+ 3) Execute netperf client on host with different protocols
+ 4) Outout the test result
+
+ @param test: Kvm test object
+ @param params: Dictionary with the test parameters.
+ @param env: Dictionary with test environment.
+ """
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm,
+ timeout=int(params.get("login_timeout", 360)))
+ netperf_dir = os.path.join(os.environ['AUTODIR'], "tests/netperf2")
+ setup_cmd = params.get("setup_cmd")
+ guest_ip = vm.get_address()
+ result_file = os.path.join(test.debugdir, "result.txt")
+
+ session.get_command_output("service iptables stop")
+ for i in params.get("netperf_files").split():
+ if not vm.copy_files_to(os.path.join(netperf_dir, i), "/tmp"):
+ raise error.TestError("Could not copy files to guest")
+ if session.get_command_status(setup_cmd % "/tmp", timeout=100) != 0:
+ raise error.TestFail("Fail to setup netperf on guest")
+ if session.get_command_status(params.get("netserver_cmd") % "/tmp") != 0:
+ raise error.TestFail("Fail to start netperf server on guest")
+
+ try:
+ logging.info("Setup and run netperf client on host")
+ s, o = commands.getstatusoutput(setup_cmd % netperf_dir)
+ if s != 0:
+ raise error.TestFail("Fail to setup netperf on host, o: %s" % o)
+ success = True
+ file(result_file, "w").write("Netperf Test Result\n")
+ for i in params.get("protocols").split():
+ cmd = params.get("netperf_cmd") % (netperf_dir, i, guest_ip)
+ logging.debug("Execute netperf client test: %s" % cmd)
+ s, o = commands.getstatusoutput(cmd)
+ if s != 0:
+ logging.error("Fail to execute netperf test, protocol:%s" % i)
+ success = False
+ else:
+ logging.info(o)
+ file(result_file, "a+").write("%s\n" % o)
+ if not success:
+ raise error.TestFail("Not all the test passed")
+ finally:
+ session.get_command_output("killall netserver")
+ session.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 7716d48..dec988e 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -398,6 +398,16 @@ variants:
type = mac_change
kill_vm = yes
+ - netperf: install setup unattended_install.cdrom
+ type = netperf
+ nic_mode = tap
+ netperf_files = netperf-2.4.5.tar.bz2 wait_before_data.patch
+ setup_cmd = "cd %s && tar xvfj netperf-2.4.5.tar.bz2 && cd netperf-2.4.5 && patch -p0 < ../wait_before_data.patch && ./configure && make"
+ netserver_cmd = %s/netperf-2.4.5/src/netserver
+ # test time is 60 seconds, set the buffer size to 1 for more hardware interrupt
+ netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1
+ protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM"
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [Qemu-devel] [RFC PATCH 12/14] KVM-test: Add a subtest of netperf
2010-07-20 1:36 ` [Qemu-devel] " Amos Kong
(?)
@ 2010-07-30 16:32 ` Lucas Meneghel Rodrigues
-1 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-30 16:32 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:36 +0800, Amos Kong wrote:
> Add network load by netperf, server is launched on guest, execute netperf
> client with different protocols on host. if all clients execute successfully,
> case will be pass. Test result will be record into result.txt.
> Now this case only tests with "TCP_RR TCP_CRR UDP_RR TCP_STREAM TCP_MAERTS
> TCP_SENDFILE UDP_STREAM". DLPI only supported by Unix, unix domain test is
> not necessary, so drop test of DLPI and unix domain.
Some general thoughts about this test:
* netperf2 has a fairly good wrapper code, that can do postprocessing
of the results. It'd be good to make use of it. What I was thinking:
- Import the netperf2 wrapper class on host and call run_once with
appropriate parameters on host
- Create a client control file and run it on guest using
kvm_test_utils.run_autotest()
- As this approach wouldn't work on windows, we could make a
netperf_windows test that would be very much like this test.
Of course, we could do that on a later time, I don't think it is
something urgent anyway.
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/netperf.py b/client/tests/kvm/tests/netperf.py
> new file mode 100644
> index 0000000..00a91f0
> --- /dev/null
> +++ b/client/tests/kvm/tests/netperf.py
> @@ -0,0 +1,56 @@
> +import logging, commands, os
> +from autotest_lib.client.common_lib import error
> +import kvm_subprocess, kvm_test_utils, kvm_utils
> +
> +def run_netperf(test, params, env):
> + """
> + Network stress test with netperf
> +
> + 1) Boot up a virtual machine
> + 2) Launch netserver on guest
> + 3) Execute netperf client on host with different protocols
> + 4) Outout the test result
> +
> + @param test: Kvm test object
> + @param params: Dictionary with the test parameters.
> + @param env: Dictionary with test environment.
> + """
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + session = kvm_test_utils.wait_for_login(vm,
> + timeout=int(params.get("login_timeout", 360)))
> + netperf_dir = os.path.join(os.environ['AUTODIR'], "tests/netperf2")
> + setup_cmd = params.get("setup_cmd")
> + guest_ip = vm.get_address()
> + result_file = os.path.join(test.debugdir, "result.txt")
^ test.resultsdir would be better. Also, it's better if we take into
account the fact that we can have more than one iteration of the test,
so we'd have something like:
self.results_path = os.path.join(self.resultsdir, 'raw_output_%s' %self.iteration)
Which is what we are doing currently for other benchmarks.
> + session.get_command_output("service iptables stop")
^ Again I think it'd be better to simply flush the firewall rules with
iptables -F because that's more likely to work on different linux
guests.
Also, this linux specific command makes the entire test linux specific,
so we have to put a rule telling the kvm autotest to not run it on
windows guests.
> + for i in params.get("netperf_files").split():
> + if not vm.copy_files_to(os.path.join(netperf_dir, i), "/tmp"):
> + raise error.TestError("Could not copy files to guest")
> + if session.get_command_status(setup_cmd % "/tmp", timeout=100) != 0:
> + raise error.TestFail("Fail to setup netperf on guest")
> + if session.get_command_status(params.get("netserver_cmd") % "/tmp") != 0:
> + raise error.TestFail("Fail to start netperf server on guest")
> +
> + try:
> + logging.info("Setup and run netperf client on host")
> + s, o = commands.getstatusoutput(setup_cmd % netperf_dir)
> + if s != 0:
> + raise error.TestFail("Fail to setup netperf on host, o: %s" % o)
> + success = True
> + file(result_file, "w").write("Netperf Test Result\n")
> + for i in params.get("protocols").split():
> + cmd = params.get("netperf_cmd") % (netperf_dir, i, guest_ip)
> + logging.debug("Execute netperf client test: %s" % cmd)
> + s, o = commands.getstatusoutput(cmd)
> + if s != 0:
> + logging.error("Fail to execute netperf test, protocol:%s" % i)
> + success = False
> + else:
> + logging.info(o)
> + file(result_file, "a+").write("%s\n" % o)
> + if not success:
> + raise error.TestFail("Not all the test passed")
^ "Some of the netperf tests failed"
> + finally:
> + session.get_command_output("killall netserver")
> + session.close()
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 7716d48..dec988e 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -398,6 +398,16 @@ variants:
> type = mac_change
> kill_vm = yes
>
> + - netperf: install setup unattended_install.cdrom
> + type = netperf
> + nic_mode = tap
> + netperf_files = netperf-2.4.5.tar.bz2 wait_before_data.patch
> + setup_cmd = "cd %s && tar xvfj netperf-2.4.5.tar.bz2 && cd netperf-2.4.5 && patch -p0 < ../wait_before_data.patch && ./configure && make"
> + netserver_cmd = %s/netperf-2.4.5/src/netserver
> + # test time is 60 seconds, set the buffer size to 1 for more hardware interrupt
> + netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1
> + protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM"
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [RFC PATCH 13/14] KVM-test: Improve vlan subtest
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:36 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:36 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
This is an enhancement of existed vlan test. Rename the vlan_tag.py to vlan.py,
it is more reasonable.
. Setup arp from "/proc/sys/net/ipv4/conf/all/arp_ignore"
. Multiple vlans exist simultaneously
. Test ping between same and different vlans
. Test by TCP data transfer, floop ping between same vlan
. Maximal plumb/unplumb vlans
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/vlan.py b/client/tests/kvm/tests/vlan.py
new file mode 100644
index 0000000..dc7611b
--- /dev/null
+++ b/client/tests/kvm/tests/vlan.py
@@ -0,0 +1,181 @@
+import logging, time, re
+from autotest_lib.client.common_lib import error
+import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
+
+def run_vlan(test, params, env):
+ """
+ Test 802.1Q vlan of NIC, config it by vconfig command.
+
+ 1) Create two VMs
+ 2) Setup guests in 10 different vlans by vconfig and using hard-coded
+ ip address
+ 3) Test by ping between same and different vlans of two VMs
+ 4) Test by TCP data transfer, floop ping between same vlan of two VMs
+ 5) Test maximal plumb/unplumb vlans
+ 6) Recover the vlan config
+
+ @param test: KVM test object.
+ @param params: Dictionary with the test parameters.
+ @param env: Dictionary with test environment.
+ """
+
+ vm = []
+ session = []
+ vm_ip = []
+ digest_origin = []
+ vlan_ip = ['', '']
+ ip_unit = ['1', '2']
+ subnet = params.get("subnet")
+ vlan_num = int(params.get("vlan_num"))
+ maximal = int(params.get("maximal"))
+ file_size = params.get("file_size")
+
+ vm.append(kvm_test_utils.get_living_vm(env, params.get("main_vm")))
+ vm.append(kvm_test_utils.get_living_vm(env, "vm2"))
+
+ def add_vlan(session, id, iface="eth0"):
+ if session.get_command_status("vconfig add %s %s" % (iface, id)) != 0:
+ raise error.TestError("Fail to add %s.%s" % (iface, id))
+
+ def set_ip_vlan(session, id, ip, iface="eth0"):
+ iface = "%s.%s" % (iface, id)
+ if session.get_command_status("ifconfig %s %s" % (iface, ip)) != 0:
+ raise error.TestError("Fail to configure ip for %s" % iface)
+
+ def set_arp_ignore(session, iface="eth0"):
+ ignore_cmd = "echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore"
+ if session.get_command_status(ignore_cmd) != 0:
+ raise error.TestError("Fail to set arp_ignore of %s" % session)
+
+ def rem_vlan(session, id, iface="eth0"):
+ rem_vlan_cmd = "if [[ -e /proc/net/vlan/%s ]];then vconfig rem %s;fi"
+ iface = "%s.%s" % (iface, id)
+ s = session.get_command_status(rem_vlan_cmd % (iface, iface))
+ return s
+
+ def nc_transfer(src, dst):
+ nc_port = kvm_utils.find_free_port(1025, 5334, vm_ip[dst])
+ listen_cmd = params.get("listen_cmd")
+ send_cmd = params.get("send_cmd")
+
+ #listen in dst
+ listen_cmd = listen_cmd % (nc_port, "receive")
+ session[dst].sendline(listen_cmd)
+ time.sleep(2)
+ #send file from src to dst
+ send_cmd = send_cmd % (vlan_ip[dst], str(nc_port), "file")
+ if session[src].get_command_status(send_cmd, timeout = 60) != 0:
+ raise error.TestFail ("Fail to send file"
+ " from vm%s to vm%s" % (src+1, dst+1))
+ s, o = session[dst].read_up_to_prompt(timeout=60)
+ if s != True:
+ raise error.TestFail ("Fail to receive file"
+ " from vm%s to vm%s" % (src+1, dst+1))
+ #check MD5 message digest of receive file in dst
+ output = session[dst].get_command_output("md5sum receive").strip()
+ digest_receive = re.findall(r'(\w+)', output)[0]
+ if digest_receive == digest_origin[src]:
+ logging.info("file succeed received in vm %s" % vlan_ip[dst])
+ else:
+ logging.info("digest_origin is %s" % digest_origin[src])
+ logging.info("digest_receive is %s" % digest_receive)
+ raise error.TestFail("File transfered differ from origin")
+ session[dst].get_command_status("rm -f receive")
+
+ for i in range(2):
+ session.append(kvm_test_utils.wait_for_login(vm[i],
+ timeout=int(params.get("login_timeout", 360))))
+ if not session[i] :
+ raise error.TestError("Could not log into guest(vm%d)" % i)
+ logging.info("Logged in")
+
+ #get guest ip
+ vm_ip.append(vm[i].get_address())
+
+ #produce sized file in vm
+ dd_cmd = "dd if=/dev/urandom of=file bs=1024k count=%s"
+ if session[i].get_command_status(dd_cmd % file_size) != 0:
+ raise error.TestFail("File producing failed")
+ #record MD5 message digest of file
+ s, output =session[i].get_command_status_output("md5sum file",
+ timeout=60)
+ if s != 0:
+ raise error.TestFail("File MD5_checking failed" )
+ digest_origin.append(re.findall(r'(\w+)', output)[0])
+
+ #stop firewall in vm
+ session[i].get_command_status("/etc/init.d/iptables stop")
+
+ #load 8021q module for vconfig
+ load_8021q_cmd = "modprobe 8021q"
+ if session[i].get_command_status(load_8021q_cmd) != 0:
+ raise error.TestError("Fail to load 8021q module on VM%s" % i)
+
+ try:
+ for i in range(2):
+ for vlan_i in range(1, vlan_num+1):
+ add_vlan(session[i], vlan_i)
+ set_ip_vlan(session[i], vlan_i, "%s.%s.%s" % \
+ (subnet, vlan_i, ip_unit[i]))
+ set_arp_ignore(session[i])
+
+ for vlan in range(1, vlan_num+1):
+ logging.info("Test for vlan %s" % vlan)
+
+ logging.info("Ping between vlans")
+ interface = 'eth0.' + str(vlan)
+ for vlan2 in range(1, vlan_num+1):
+ for i in range(2):
+ dest = subnet +'.'+ str(vlan2)+ '.' + ip_unit[(i+1)%2]
+ s, o = kvm_net_utils.ping(dest, count=2,
+ interface=interface,
+ session=session[i], timeout=30)
+ if ((vlan == vlan2) ^ (s == 0)):
+ raise error.TestFail ("%s ping %s unexpected" %
+ (interface, dest))
+
+ vlan_ip[0] = subnet + '.' + str(vlan) + '.' + ip_unit[0]
+ vlan_ip[1] = subnet + '.' + str(vlan) + '.' + ip_unit[1]
+
+ logging.info("Flood ping")
+ def flood_ping(src, dst):
+ # we must use a dedicated session becuase the kvm_subprocess
+ # does not have the other method to interrupt the process in
+ # the guest rather than close the session.
+ session_flood = kvm_test_utils.wait_for_login(vm[src],
+ timeout = 60)
+ kvm_net_utils.ping(vlan_ip[dst], flood=True,
+ interface=interface,
+ session=session_flood, timeout=10)
+ session_flood.close()
+
+ flood_ping(0,1)
+ flood_ping(1,0)
+
+ logging.info("Transfering data through nc")
+ nc_transfer(0, 1)
+ nc_transfer(1, 0)
+
+ finally:
+ for vlan in range(1, vlan_num+1):
+ rem_vlan(session[0], vlan)
+ rem_vlan(session[1], vlan)
+ logging.info("rem eth0.%s" % vlan)
+
+ #Plumb/unplumb maximal unber of vlan interfaces
+ i = 1
+ s = 0
+ try:
+ logging.info("Testing the plumb of vlan interface")
+ for i in range (1, maximal+1):
+ add_vlan(session[0], i)
+ finally:
+ for j in range (1, i+1):
+ s = s or rem_vlan(session[0], j)
+ if s == 0:
+ logging.info("maximal interface plumb test done")
+ else:
+ logging.error("maximal interface plumb test failed")
+
+ session[0].close()
+ session[1].close()
diff --git a/client/tests/kvm/tests/vlan_tag.py b/client/tests/kvm/tests/vlan_tag.py
deleted file mode 100644
index cafd8fe..0000000
--- a/client/tests/kvm/tests/vlan_tag.py
+++ /dev/null
@@ -1,68 +0,0 @@
-import logging, time
-from autotest_lib.client.common_lib import error
-import kvm_subprocess, kvm_test_utils, kvm_utils
-
-
-def run_vlan_tag(test, params, env):
- """
- Test 802.1Q vlan of NIC, config it by vconfig command.
-
- 1) Create two VMs.
- 2) Setup guests in different VLANs by vconfig and test communication by
- ping using hard-coded ip addresses.
- 3) Setup guests in same vlan and test communication by ping.
- 4) Recover the vlan config.
-
- @param test: KVM test object.
- @param params: Dictionary with the test parameters.
- @param env: Dictionary with test environment.
- """
- subnet = params.get("subnet")
- vlans = params.get("vlans").split()
-
- vm1 = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
- vm2 = kvm_test_utils.get_living_vm(env, "vm2")
-
- timeout = int(params.get("login_timeout", 360))
- session1 = kvm_test_utils.wait_for_login(vm1, timeout=timeout)
- session2 = kvm_test_utils.wait_for_login(vm2, timeout=timeout)
-
- try:
- ip_cfg_base = "vconfig add eth0 %s && ifconfig eth0.%s %s.%s"
- ip_cfg_cmd1 = ip_cfg_base % (vlans[0], vlans[0], subnet, "11")
- ip_cfg_cmd2 = ip_cfg_base % (vlans[1], vlans[1], subnet, "12")
-
- # Configure VM1 and VM2 in different VLANs
- ip_cfg_vm1 = session1.get_command_status(ip_cfg_cmd1)
- if ip_cfg_vm1 != 0:
- raise error.TestError("Failed to config VM 1 IP address")
- ip_cfg_vm2 = session2.get_command_status(ip_cfg_cmd2)
- if ip_cfg_vm2 != 0:
- raise error.TestError("Failed to config VM 2 IP address")
-
- # Trying to ping VM 2 from VM 1, this shouldn't work
- ping_cmd = "ping -c 2 -I eth0.%s %s.%s" % (vlans[0], subnet, "12")
- ping_diff_vlan = session1.get_command_status(ping_cmd)
- if ping_diff_vlan == 0:
- raise error.TestFail("VM 2 can be reached even though it was "
- "configured on a different VLAN")
-
- # Now let's put VM 2 in the same VLAN as VM 1
- ip_cfg_reconfig= ("vconfig rem eth0.%s && vconfig add eth0 %s && "
- "ifconfig eth0.%s %s.%s" % (vlans[1], vlans[0],
- vlans[0], subnet, "12"))
- ip_cfg_vm2 = session2.get_command_status(ip_cfg_reconfig)
- if ip_cfg_vm2 != 0:
- raise error.TestError("Failed to re-config IP address of VM 2")
-
- # Try to ping VM 2 from VM 1, this should work
- ping_same_vlan = session1.get_command_status(ping_cmd)
- if ping_same_vlan != 0:
- raise error.TestFail("Failed to ping VM 2 even though it was "
- "configured on the same VLAN")
-
- finally:
- session1.get_command_status("vconfig rem eth0.%s" % vlans[0])
- session1.close()
- session2.get_command_status("vconfig rem eth0.%s" % vlans[0])
- session2.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index dec988e..b25980e 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -337,17 +337,21 @@ variants:
- fmt_raw:
image_format_stg = raw
- - vlan_tag: install setup unattended_install.cdrom
- type = vlan_tag
+ - vlan: install setup unattended_install.cdrom
+ type = vlan
# subnet should not be used by host
- subnet = 192.168.123
- vlans = "10 20"
+ subnet = "192.168"
+ vlan_num = 5
+ file_size = 10
+ maximal = 4094
+ listen_cmd = "nc -l %s > %s"
+ send_cmd = "nc %s %s < %s"
nic_mode = tap
vms += " vm2"
extra_params_vm1 += " -snapshot"
extra_params_vm2 += " -snapshot"
+ kill_vm_vm2 = yes
kill_vm_gracefully_vm2 = no
- address_index_vm2 = 1
- ping: install setup unattended_install.cdrom
type = ping
@@ -1084,7 +1088,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
+ no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 13/14] KVM-test: Improve vlan subtest
@ 2010-07-20 1:36 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:36 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
This is an enhancement of existed vlan test. Rename the vlan_tag.py to vlan.py,
it is more reasonable.
. Setup arp from "/proc/sys/net/ipv4/conf/all/arp_ignore"
. Multiple vlans exist simultaneously
. Test ping between same and different vlans
. Test by TCP data transfer, floop ping between same vlan
. Maximal plumb/unplumb vlans
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/vlan.py b/client/tests/kvm/tests/vlan.py
new file mode 100644
index 0000000..dc7611b
--- /dev/null
+++ b/client/tests/kvm/tests/vlan.py
@@ -0,0 +1,181 @@
+import logging, time, re
+from autotest_lib.client.common_lib import error
+import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils
+
+def run_vlan(test, params, env):
+ """
+ Test 802.1Q vlan of NIC, config it by vconfig command.
+
+ 1) Create two VMs
+ 2) Setup guests in 10 different vlans by vconfig and using hard-coded
+ ip address
+ 3) Test by ping between same and different vlans of two VMs
+ 4) Test by TCP data transfer, floop ping between same vlan of two VMs
+ 5) Test maximal plumb/unplumb vlans
+ 6) Recover the vlan config
+
+ @param test: KVM test object.
+ @param params: Dictionary with the test parameters.
+ @param env: Dictionary with test environment.
+ """
+
+ vm = []
+ session = []
+ vm_ip = []
+ digest_origin = []
+ vlan_ip = ['', '']
+ ip_unit = ['1', '2']
+ subnet = params.get("subnet")
+ vlan_num = int(params.get("vlan_num"))
+ maximal = int(params.get("maximal"))
+ file_size = params.get("file_size")
+
+ vm.append(kvm_test_utils.get_living_vm(env, params.get("main_vm")))
+ vm.append(kvm_test_utils.get_living_vm(env, "vm2"))
+
+ def add_vlan(session, id, iface="eth0"):
+ if session.get_command_status("vconfig add %s %s" % (iface, id)) != 0:
+ raise error.TestError("Fail to add %s.%s" % (iface, id))
+
+ def set_ip_vlan(session, id, ip, iface="eth0"):
+ iface = "%s.%s" % (iface, id)
+ if session.get_command_status("ifconfig %s %s" % (iface, ip)) != 0:
+ raise error.TestError("Fail to configure ip for %s" % iface)
+
+ def set_arp_ignore(session, iface="eth0"):
+ ignore_cmd = "echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore"
+ if session.get_command_status(ignore_cmd) != 0:
+ raise error.TestError("Fail to set arp_ignore of %s" % session)
+
+ def rem_vlan(session, id, iface="eth0"):
+ rem_vlan_cmd = "if [[ -e /proc/net/vlan/%s ]];then vconfig rem %s;fi"
+ iface = "%s.%s" % (iface, id)
+ s = session.get_command_status(rem_vlan_cmd % (iface, iface))
+ return s
+
+ def nc_transfer(src, dst):
+ nc_port = kvm_utils.find_free_port(1025, 5334, vm_ip[dst])
+ listen_cmd = params.get("listen_cmd")
+ send_cmd = params.get("send_cmd")
+
+ #listen in dst
+ listen_cmd = listen_cmd % (nc_port, "receive")
+ session[dst].sendline(listen_cmd)
+ time.sleep(2)
+ #send file from src to dst
+ send_cmd = send_cmd % (vlan_ip[dst], str(nc_port), "file")
+ if session[src].get_command_status(send_cmd, timeout = 60) != 0:
+ raise error.TestFail ("Fail to send file"
+ " from vm%s to vm%s" % (src+1, dst+1))
+ s, o = session[dst].read_up_to_prompt(timeout=60)
+ if s != True:
+ raise error.TestFail ("Fail to receive file"
+ " from vm%s to vm%s" % (src+1, dst+1))
+ #check MD5 message digest of receive file in dst
+ output = session[dst].get_command_output("md5sum receive").strip()
+ digest_receive = re.findall(r'(\w+)', output)[0]
+ if digest_receive == digest_origin[src]:
+ logging.info("file succeed received in vm %s" % vlan_ip[dst])
+ else:
+ logging.info("digest_origin is %s" % digest_origin[src])
+ logging.info("digest_receive is %s" % digest_receive)
+ raise error.TestFail("File transfered differ from origin")
+ session[dst].get_command_status("rm -f receive")
+
+ for i in range(2):
+ session.append(kvm_test_utils.wait_for_login(vm[i],
+ timeout=int(params.get("login_timeout", 360))))
+ if not session[i] :
+ raise error.TestError("Could not log into guest(vm%d)" % i)
+ logging.info("Logged in")
+
+ #get guest ip
+ vm_ip.append(vm[i].get_address())
+
+ #produce sized file in vm
+ dd_cmd = "dd if=/dev/urandom of=file bs=1024k count=%s"
+ if session[i].get_command_status(dd_cmd % file_size) != 0:
+ raise error.TestFail("File producing failed")
+ #record MD5 message digest of file
+ s, output =session[i].get_command_status_output("md5sum file",
+ timeout=60)
+ if s != 0:
+ raise error.TestFail("File MD5_checking failed" )
+ digest_origin.append(re.findall(r'(\w+)', output)[0])
+
+ #stop firewall in vm
+ session[i].get_command_status("/etc/init.d/iptables stop")
+
+ #load 8021q module for vconfig
+ load_8021q_cmd = "modprobe 8021q"
+ if session[i].get_command_status(load_8021q_cmd) != 0:
+ raise error.TestError("Fail to load 8021q module on VM%s" % i)
+
+ try:
+ for i in range(2):
+ for vlan_i in range(1, vlan_num+1):
+ add_vlan(session[i], vlan_i)
+ set_ip_vlan(session[i], vlan_i, "%s.%s.%s" % \
+ (subnet, vlan_i, ip_unit[i]))
+ set_arp_ignore(session[i])
+
+ for vlan in range(1, vlan_num+1):
+ logging.info("Test for vlan %s" % vlan)
+
+ logging.info("Ping between vlans")
+ interface = 'eth0.' + str(vlan)
+ for vlan2 in range(1, vlan_num+1):
+ for i in range(2):
+ dest = subnet +'.'+ str(vlan2)+ '.' + ip_unit[(i+1)%2]
+ s, o = kvm_net_utils.ping(dest, count=2,
+ interface=interface,
+ session=session[i], timeout=30)
+ if ((vlan == vlan2) ^ (s == 0)):
+ raise error.TestFail ("%s ping %s unexpected" %
+ (interface, dest))
+
+ vlan_ip[0] = subnet + '.' + str(vlan) + '.' + ip_unit[0]
+ vlan_ip[1] = subnet + '.' + str(vlan) + '.' + ip_unit[1]
+
+ logging.info("Flood ping")
+ def flood_ping(src, dst):
+ # we must use a dedicated session becuase the kvm_subprocess
+ # does not have the other method to interrupt the process in
+ # the guest rather than close the session.
+ session_flood = kvm_test_utils.wait_for_login(vm[src],
+ timeout = 60)
+ kvm_net_utils.ping(vlan_ip[dst], flood=True,
+ interface=interface,
+ session=session_flood, timeout=10)
+ session_flood.close()
+
+ flood_ping(0,1)
+ flood_ping(1,0)
+
+ logging.info("Transfering data through nc")
+ nc_transfer(0, 1)
+ nc_transfer(1, 0)
+
+ finally:
+ for vlan in range(1, vlan_num+1):
+ rem_vlan(session[0], vlan)
+ rem_vlan(session[1], vlan)
+ logging.info("rem eth0.%s" % vlan)
+
+ #Plumb/unplumb maximal unber of vlan interfaces
+ i = 1
+ s = 0
+ try:
+ logging.info("Testing the plumb of vlan interface")
+ for i in range (1, maximal+1):
+ add_vlan(session[0], i)
+ finally:
+ for j in range (1, i+1):
+ s = s or rem_vlan(session[0], j)
+ if s == 0:
+ logging.info("maximal interface plumb test done")
+ else:
+ logging.error("maximal interface plumb test failed")
+
+ session[0].close()
+ session[1].close()
diff --git a/client/tests/kvm/tests/vlan_tag.py b/client/tests/kvm/tests/vlan_tag.py
deleted file mode 100644
index cafd8fe..0000000
--- a/client/tests/kvm/tests/vlan_tag.py
+++ /dev/null
@@ -1,68 +0,0 @@
-import logging, time
-from autotest_lib.client.common_lib import error
-import kvm_subprocess, kvm_test_utils, kvm_utils
-
-
-def run_vlan_tag(test, params, env):
- """
- Test 802.1Q vlan of NIC, config it by vconfig command.
-
- 1) Create two VMs.
- 2) Setup guests in different VLANs by vconfig and test communication by
- ping using hard-coded ip addresses.
- 3) Setup guests in same vlan and test communication by ping.
- 4) Recover the vlan config.
-
- @param test: KVM test object.
- @param params: Dictionary with the test parameters.
- @param env: Dictionary with test environment.
- """
- subnet = params.get("subnet")
- vlans = params.get("vlans").split()
-
- vm1 = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
- vm2 = kvm_test_utils.get_living_vm(env, "vm2")
-
- timeout = int(params.get("login_timeout", 360))
- session1 = kvm_test_utils.wait_for_login(vm1, timeout=timeout)
- session2 = kvm_test_utils.wait_for_login(vm2, timeout=timeout)
-
- try:
- ip_cfg_base = "vconfig add eth0 %s && ifconfig eth0.%s %s.%s"
- ip_cfg_cmd1 = ip_cfg_base % (vlans[0], vlans[0], subnet, "11")
- ip_cfg_cmd2 = ip_cfg_base % (vlans[1], vlans[1], subnet, "12")
-
- # Configure VM1 and VM2 in different VLANs
- ip_cfg_vm1 = session1.get_command_status(ip_cfg_cmd1)
- if ip_cfg_vm1 != 0:
- raise error.TestError("Failed to config VM 1 IP address")
- ip_cfg_vm2 = session2.get_command_status(ip_cfg_cmd2)
- if ip_cfg_vm2 != 0:
- raise error.TestError("Failed to config VM 2 IP address")
-
- # Trying to ping VM 2 from VM 1, this shouldn't work
- ping_cmd = "ping -c 2 -I eth0.%s %s.%s" % (vlans[0], subnet, "12")
- ping_diff_vlan = session1.get_command_status(ping_cmd)
- if ping_diff_vlan == 0:
- raise error.TestFail("VM 2 can be reached even though it was "
- "configured on a different VLAN")
-
- # Now let's put VM 2 in the same VLAN as VM 1
- ip_cfg_reconfig= ("vconfig rem eth0.%s && vconfig add eth0 %s && "
- "ifconfig eth0.%s %s.%s" % (vlans[1], vlans[0],
- vlans[0], subnet, "12"))
- ip_cfg_vm2 = session2.get_command_status(ip_cfg_reconfig)
- if ip_cfg_vm2 != 0:
- raise error.TestError("Failed to re-config IP address of VM 2")
-
- # Try to ping VM 2 from VM 1, this should work
- ping_same_vlan = session1.get_command_status(ping_cmd)
- if ping_same_vlan != 0:
- raise error.TestFail("Failed to ping VM 2 even though it was "
- "configured on the same VLAN")
-
- finally:
- session1.get_command_status("vconfig rem eth0.%s" % vlans[0])
- session1.close()
- session2.get_command_status("vconfig rem eth0.%s" % vlans[0])
- session2.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index dec988e..b25980e 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -337,17 +337,21 @@ variants:
- fmt_raw:
image_format_stg = raw
- - vlan_tag: install setup unattended_install.cdrom
- type = vlan_tag
+ - vlan: install setup unattended_install.cdrom
+ type = vlan
# subnet should not be used by host
- subnet = 192.168.123
- vlans = "10 20"
+ subnet = "192.168"
+ vlan_num = 5
+ file_size = 10
+ maximal = 4094
+ listen_cmd = "nc -l %s > %s"
+ send_cmd = "nc %s %s < %s"
nic_mode = tap
vms += " vm2"
extra_params_vm1 += " -snapshot"
extra_params_vm2 += " -snapshot"
+ kill_vm_vm2 = yes
kill_vm_gracefully_vm2 = no
- address_index_vm2 = 1
- ping: install setup unattended_install.cdrom
type = ping
@@ -1084,7 +1088,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
+ no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [RFC PATCH 14/14] KVM-test: Add subtest of testing offload by ethtool
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 1:36 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:36 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
The latest case contains TX/RX/SG/TSO/GSO/GRO/LRO test. RTL8139 NIC doesn't
support TSO, LRO, it's too old, so drop offload test from rtl8139. LRO, GRO
are only supported by latest kernel, virtio nic doesn't support receive
offloading function.
Initialize the callbacks first and execute all the sub tests one by one, all
the result will be check at the end.
When execute this test, vhost should be enabled, then most of new feature can
be used. Vhost doestn't support VIRTIO_NET_F_MRG_RXBUF, so do not check large
packets in received offload test.
Transfer files by scp between host and guest, match new opened TCP port by
netstat. Capture the packages info by tcpdump, it contains package length.
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/ethtool.py b/client/tests/kvm/tests/ethtool.py
new file mode 100644
index 0000000..7274eae
--- /dev/null
+++ b/client/tests/kvm/tests/ethtool.py
@@ -0,0 +1,205 @@
+import time, os, logging, commands, re
+from autotest_lib.client.common_lib import error
+from autotest_lib.client.bin import utils
+import kvm_test_utils, kvm_utils, kvm_net_utils
+
+def run_ethtool(test, params, env):
+ """
+ Test offload functions of ethernet device by ethtool
+
+ 1) Log into a guest
+ 2) Initialize the callback of sub functions
+ 3) Enable/disable sub function of NIC
+ 4) Execute callback function
+ 5) Check the return value
+ 6) Restore original configuration
+
+ @param test: Kvm test object
+ @param params: Dictionary with the test parameters.
+ @param env: Dictionary with test environment.
+ """
+ def ethtool_get(type):
+ feature_pattern = {
+ 'tx': 'tx.*checksumming',
+ 'rx': 'rx.*checksumming',
+ 'sg': 'scatter.*gather',
+ 'tso': 'tcp.*segmentation.*offload',
+ 'gso': 'generic.*segmentation.*offload',
+ 'gro': 'generic.*receive.*offload',
+ 'lro': 'large.*receive.*offload',
+ }
+ s, o = session.get_command_status_output("ethtool -k %s" % ethname)
+ try:
+ return re.findall("%s: (.*)" % feature_pattern.get(type), o)[0]
+ except IndexError:
+ logging.debug("Could not get %s status" % type)
+
+ def ethtool_set(type, status):
+ """
+ Set ethernet device offload status
+
+ @param type: Offload type name
+ @param status: New status will be changed to
+ """
+ logging.info("Try to set %s %s" % (type, status))
+ if status not in ["off", "on"]:
+ return False
+ cmd = "ethtool -K %s %s %s" % (ethname, type, status)
+ if ethtool_get(type) != status:
+ return session.get_command_status(cmd) == 0
+ if ethtool_get(type) != status:
+ logging.error("Fail to set %s %s" % (type, status))
+ return False
+ return True
+
+ def ethtool_save_params():
+ logging.info("Save ethtool configuration")
+ for i in supported_features:
+ feature_status[i] = ethtool_get(i)
+
+ def ethtool_restore_params():
+ logging.info("Restore ethtool configuration")
+ for i in supported_features:
+ ethtool_set(i, feature_status[i])
+
+ def compare_md5sum(name):
+ logging.info("Compare md5sum of the files on guest and host")
+ host_result = utils.hash_file(name, method="md5")
+ try:
+ o = session.get_command_output("md5sum %s" % name)
+ guest_result = re.findall("\w+", o)[0]
+ except IndexError:
+ logging.error("Could not get file md5sum in guest")
+ return False
+ logging.debug("md5sum: guest(%s), host(%s)" % (guest_result,
+ host_result))
+ return guest_result == host_result
+
+ def transfer_file(src="guest"):
+ """
+ Transfer file by scp, use tcpdump to capture packets, then check the
+ return string.
+
+ @param src: Source host of transfer file
+ @return: Tuple (status, error msg/tcpdump result)
+ """
+ session2.get_command_status("rm -rf %s" % filename)
+ dd_cmd = "dd if=/dev/urandom of=%s bs=1M count=%s" % (filename,
+ params.get("filesize"))
+ logging.info("Creat file in source host, cmd: %s" % dd_cmd)
+ tcpdump_cmd = "tcpdump -lep -s 0 tcp -vv port ssh"
+ if src == "guest":
+ s = session.get_command_status(dd_cmd, timeout=360)
+ tcpdump_cmd += " and src %s" % guest_ip
+ copy_files_fun = vm.copy_files_from
+ else:
+ s, o = commands.getstatusoutput(dd_cmd)
+ tcpdump_cmd += " and dst %s" % guest_ip
+ copy_files_fun = vm.copy_files_to
+ if s != 0:
+ return (False, "Fail to create file by dd, cmd: %s" % dd_cmd)
+
+ # only capture the new tcp port after offload setup
+ original_tcp_ports = re.findall("tcp.*:(\d+).*%s" % guest_ip,
+ commands.getoutput("/bin/netstat -nap"))
+ for i in original_tcp_ports:
+ tcpdump_cmd += " and not port %s" % i
+ logging.debug("Listen by command: %s" % tcpdump_cmd)
+ session2.sendline(tcpdump_cmd)
+ if not kvm_utils.wait_for(lambda: session.get_command_status(
+ "pgrep tcpdump") == 0, 30):
+ return (False, "Tcpdump process wasn't launched")
+
+ logging.info("Start to transfer file")
+ if not copy_files_fun(filename, filename):
+ return (False, "Child process transfer file failed")
+ logging.info("Transfer file completed")
+ if session.get_command_status("killall tcpdump") != 0:
+ return (False, "Could not kill all tcpdump process")
+ s, tcpdump_string = session2.read_up_to_prompt(timeout=60)
+ if not s:
+ return (False, "Fail to read tcpdump's output")
+
+ if not compare_md5sum(filename):
+ return (False, "Files' md5sum mismatched")
+ return (True, tcpdump_string)
+
+ def tx_callback(status="on"):
+ s, o = transfer_file(src="guest")
+ if not s:
+ logging.error(o)
+ return False
+ return True
+
+ def rx_callback(status="on"):
+ s, o = transfer_file(src="host")
+ if not s:
+ logging.error(o)
+ return False
+ return True
+
+ def so_callback(status="on"):
+ s, o = transfer_file(src="guest")
+ if not s:
+ logging.error(o)
+ return False
+ logging.info("Check if contained large frame")
+ # mtu: default IPv4 MTU is 1500 Bytes, ethernet header is 14 Bytes
+ return (status == "on") ^ (len([i for i in re.findall(
+ "length (\d*):", o) if int(i) > mtu]) == 0)
+
+ def ro_callback(status="on"):
+ s, o = transfer_file(src="host")
+ if not s:
+ logging.error(o)
+ return False
+ return True
+
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm,
+ timeout=int(params.get("login_timeout", 360)))
+ session2 = kvm_test_utils.wait_for_login(vm,
+ timeout=int(params.get("login_timeout", 360)))
+ mtu = 1514
+ feature_status = {}
+ filename = "/tmp/ethtool.dd"
+ guest_ip = vm.get_address()
+ ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
+ supported_features = params.get("supported_features").split()
+ test_matrix = {
+ # type:(callback, (dependence), (exclude)
+ "tx": (tx_callback, (), ()),
+ "rx": (rx_callback, (), ()),
+ "sg": (tx_callback, ("tx",), ()),
+ "tso": (so_callback, ("tx", "sg",), ("gso",)),
+ "gso": (so_callback, (), ("tso",)),
+ "gro": (ro_callback, ("rx",), ("lro",)),
+ "lro": (rx_callback, (), ("gro",)),
+ }
+ ethtool_save_params()
+ success = True
+ try:
+ for type in supported_features:
+ callback = test_matrix[type][0]
+ for i in test_matrix[type][2]:
+ if not ethtool_set(i, "off"):
+ logging.error("Fail to disable %s" % i)
+ success = False
+ for i in [f for f in test_matrix[type][1]] + [type]:
+ if not ethtool_set(i, "on"):
+ logging.error("Fail to enable %s" % i)
+ success = False
+ if not callback():
+ raise error.TestFail("Test failed, %s: on" % type)
+
+ if not ethtool_set(type, "off"):
+ logging.error("Fail to disable %s" % type)
+ success = False
+ if not callback(status="off"):
+ raise error.TestFail("Test failed, %s: off" % type)
+ if not success:
+ raise error.TestError("Enable/disable offload function fail")
+ finally:
+ ethtool_restore_params()
+ session.close()
+ session2.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index b25980e..6f0e295 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -412,6 +412,19 @@ variants:
netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1
protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM"
+ - ethtool: install setup unattended_install.cdrom
+ type = ethtool
+ filesize = 512
+ nic_mode = tap
+ variants:
+ # gso gro lro is only supported by latest kernel
+ - nic_virtio:
+ pci_model = virtio
+ supported_features = "tx sg tso gso"
+ - nic_e1000:
+ pci_model = e1000
+ supported_features = "tx rx sg tso gso gro lro"
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1088,7 +1101,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
+ no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change ethtool
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Qemu-devel] [RFC PATCH 14/14] KVM-test: Add subtest of testing offload by ethtool
@ 2010-07-20 1:36 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-07-20 1:36 UTC (permalink / raw)
To: lmr; +Cc: autotest, qemu-devel, kvm
The latest case contains TX/RX/SG/TSO/GSO/GRO/LRO test. RTL8139 NIC doesn't
support TSO, LRO, it's too old, so drop offload test from rtl8139. LRO, GRO
are only supported by latest kernel, virtio nic doesn't support receive
offloading function.
Initialize the callbacks first and execute all the sub tests one by one, all
the result will be check at the end.
When execute this test, vhost should be enabled, then most of new feature can
be used. Vhost doestn't support VIRTIO_NET_F_MRG_RXBUF, so do not check large
packets in received offload test.
Transfer files by scp between host and guest, match new opened TCP port by
netstat. Capture the packages info by tcpdump, it contains package length.
Signed-off-by: Amos Kong <akong@redhat.com>
---
0 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/client/tests/kvm/tests/ethtool.py b/client/tests/kvm/tests/ethtool.py
new file mode 100644
index 0000000..7274eae
--- /dev/null
+++ b/client/tests/kvm/tests/ethtool.py
@@ -0,0 +1,205 @@
+import time, os, logging, commands, re
+from autotest_lib.client.common_lib import error
+from autotest_lib.client.bin import utils
+import kvm_test_utils, kvm_utils, kvm_net_utils
+
+def run_ethtool(test, params, env):
+ """
+ Test offload functions of ethernet device by ethtool
+
+ 1) Log into a guest
+ 2) Initialize the callback of sub functions
+ 3) Enable/disable sub function of NIC
+ 4) Execute callback function
+ 5) Check the return value
+ 6) Restore original configuration
+
+ @param test: Kvm test object
+ @param params: Dictionary with the test parameters.
+ @param env: Dictionary with test environment.
+ """
+ def ethtool_get(type):
+ feature_pattern = {
+ 'tx': 'tx.*checksumming',
+ 'rx': 'rx.*checksumming',
+ 'sg': 'scatter.*gather',
+ 'tso': 'tcp.*segmentation.*offload',
+ 'gso': 'generic.*segmentation.*offload',
+ 'gro': 'generic.*receive.*offload',
+ 'lro': 'large.*receive.*offload',
+ }
+ s, o = session.get_command_status_output("ethtool -k %s" % ethname)
+ try:
+ return re.findall("%s: (.*)" % feature_pattern.get(type), o)[0]
+ except IndexError:
+ logging.debug("Could not get %s status" % type)
+
+ def ethtool_set(type, status):
+ """
+ Set ethernet device offload status
+
+ @param type: Offload type name
+ @param status: New status will be changed to
+ """
+ logging.info("Try to set %s %s" % (type, status))
+ if status not in ["off", "on"]:
+ return False
+ cmd = "ethtool -K %s %s %s" % (ethname, type, status)
+ if ethtool_get(type) != status:
+ return session.get_command_status(cmd) == 0
+ if ethtool_get(type) != status:
+ logging.error("Fail to set %s %s" % (type, status))
+ return False
+ return True
+
+ def ethtool_save_params():
+ logging.info("Save ethtool configuration")
+ for i in supported_features:
+ feature_status[i] = ethtool_get(i)
+
+ def ethtool_restore_params():
+ logging.info("Restore ethtool configuration")
+ for i in supported_features:
+ ethtool_set(i, feature_status[i])
+
+ def compare_md5sum(name):
+ logging.info("Compare md5sum of the files on guest and host")
+ host_result = utils.hash_file(name, method="md5")
+ try:
+ o = session.get_command_output("md5sum %s" % name)
+ guest_result = re.findall("\w+", o)[0]
+ except IndexError:
+ logging.error("Could not get file md5sum in guest")
+ return False
+ logging.debug("md5sum: guest(%s), host(%s)" % (guest_result,
+ host_result))
+ return guest_result == host_result
+
+ def transfer_file(src="guest"):
+ """
+ Transfer file by scp, use tcpdump to capture packets, then check the
+ return string.
+
+ @param src: Source host of transfer file
+ @return: Tuple (status, error msg/tcpdump result)
+ """
+ session2.get_command_status("rm -rf %s" % filename)
+ dd_cmd = "dd if=/dev/urandom of=%s bs=1M count=%s" % (filename,
+ params.get("filesize"))
+ logging.info("Creat file in source host, cmd: %s" % dd_cmd)
+ tcpdump_cmd = "tcpdump -lep -s 0 tcp -vv port ssh"
+ if src == "guest":
+ s = session.get_command_status(dd_cmd, timeout=360)
+ tcpdump_cmd += " and src %s" % guest_ip
+ copy_files_fun = vm.copy_files_from
+ else:
+ s, o = commands.getstatusoutput(dd_cmd)
+ tcpdump_cmd += " and dst %s" % guest_ip
+ copy_files_fun = vm.copy_files_to
+ if s != 0:
+ return (False, "Fail to create file by dd, cmd: %s" % dd_cmd)
+
+ # only capture the new tcp port after offload setup
+ original_tcp_ports = re.findall("tcp.*:(\d+).*%s" % guest_ip,
+ commands.getoutput("/bin/netstat -nap"))
+ for i in original_tcp_ports:
+ tcpdump_cmd += " and not port %s" % i
+ logging.debug("Listen by command: %s" % tcpdump_cmd)
+ session2.sendline(tcpdump_cmd)
+ if not kvm_utils.wait_for(lambda: session.get_command_status(
+ "pgrep tcpdump") == 0, 30):
+ return (False, "Tcpdump process wasn't launched")
+
+ logging.info("Start to transfer file")
+ if not copy_files_fun(filename, filename):
+ return (False, "Child process transfer file failed")
+ logging.info("Transfer file completed")
+ if session.get_command_status("killall tcpdump") != 0:
+ return (False, "Could not kill all tcpdump process")
+ s, tcpdump_string = session2.read_up_to_prompt(timeout=60)
+ if not s:
+ return (False, "Fail to read tcpdump's output")
+
+ if not compare_md5sum(filename):
+ return (False, "Files' md5sum mismatched")
+ return (True, tcpdump_string)
+
+ def tx_callback(status="on"):
+ s, o = transfer_file(src="guest")
+ if not s:
+ logging.error(o)
+ return False
+ return True
+
+ def rx_callback(status="on"):
+ s, o = transfer_file(src="host")
+ if not s:
+ logging.error(o)
+ return False
+ return True
+
+ def so_callback(status="on"):
+ s, o = transfer_file(src="guest")
+ if not s:
+ logging.error(o)
+ return False
+ logging.info("Check if contained large frame")
+ # mtu: default IPv4 MTU is 1500 Bytes, ethernet header is 14 Bytes
+ return (status == "on") ^ (len([i for i in re.findall(
+ "length (\d*):", o) if int(i) > mtu]) == 0)
+
+ def ro_callback(status="on"):
+ s, o = transfer_file(src="host")
+ if not s:
+ logging.error(o)
+ return False
+ return True
+
+ vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+ session = kvm_test_utils.wait_for_login(vm,
+ timeout=int(params.get("login_timeout", 360)))
+ session2 = kvm_test_utils.wait_for_login(vm,
+ timeout=int(params.get("login_timeout", 360)))
+ mtu = 1514
+ feature_status = {}
+ filename = "/tmp/ethtool.dd"
+ guest_ip = vm.get_address()
+ ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
+ supported_features = params.get("supported_features").split()
+ test_matrix = {
+ # type:(callback, (dependence), (exclude)
+ "tx": (tx_callback, (), ()),
+ "rx": (rx_callback, (), ()),
+ "sg": (tx_callback, ("tx",), ()),
+ "tso": (so_callback, ("tx", "sg",), ("gso",)),
+ "gso": (so_callback, (), ("tso",)),
+ "gro": (ro_callback, ("rx",), ("lro",)),
+ "lro": (rx_callback, (), ("gro",)),
+ }
+ ethtool_save_params()
+ success = True
+ try:
+ for type in supported_features:
+ callback = test_matrix[type][0]
+ for i in test_matrix[type][2]:
+ if not ethtool_set(i, "off"):
+ logging.error("Fail to disable %s" % i)
+ success = False
+ for i in [f for f in test_matrix[type][1]] + [type]:
+ if not ethtool_set(i, "on"):
+ logging.error("Fail to enable %s" % i)
+ success = False
+ if not callback():
+ raise error.TestFail("Test failed, %s: on" % type)
+
+ if not ethtool_set(type, "off"):
+ logging.error("Fail to disable %s" % type)
+ success = False
+ if not callback(status="off"):
+ raise error.TestFail("Test failed, %s: off" % type)
+ if not success:
+ raise error.TestError("Enable/disable offload function fail")
+ finally:
+ ethtool_restore_params()
+ session.close()
+ session2.close()
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index b25980e..6f0e295 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -412,6 +412,19 @@ variants:
netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1
protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM"
+ - ethtool: install setup unattended_install.cdrom
+ type = ethtool
+ filesize = 512
+ nic_mode = tap
+ variants:
+ # gso gro lro is only supported by latest kernel
+ - nic_virtio:
+ pci_model = virtio
+ supported_features = "tx sg tso gso"
+ - nic_e1000:
+ pci_model = e1000
+ supported_features = "tx rx sg tso gso gro lro"
+
- physical_resources_check: install setup unattended_install.cdrom
type = physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1088,7 +1101,7 @@ variants:
# Windows section
- @Windows:
- no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
+ no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change ethtool
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [RFC PATCH 14/14] KVM-test: Add subtest of testing offload by ethtool
2010-07-20 1:36 ` [Qemu-devel] " Amos Kong
@ 2010-08-02 19:10 ` Lucas Meneghel Rodrigues
-1 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-08-02 19:10 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:36 +0800, Amos Kong wrote:
> The latest case contains TX/RX/SG/TSO/GSO/GRO/LRO test. RTL8139 NIC doesn't
> support TSO, LRO, it's too old, so drop offload test from rtl8139. LRO, GRO
> are only supported by latest kernel, virtio nic doesn't support receive
> offloading function.
> Initialize the callbacks first and execute all the sub tests one by one, all
> the result will be check at the end.
> When execute this test, vhost should be enabled, then most of new feature can
> be used. Vhost doestn't support VIRTIO_NET_F_MRG_RXBUF, so do not check large
> packets in received offload test.
> Transfer files by scp between host and guest, match new opened TCP port by
> netstat. Capture the packages info by tcpdump, it contains package length.
This test is heavily dependent on ethtool, so we need to make sure the
package is going to be installed on linux guests. The default package
selection for Fedora 13 does not include it for example. So, we need to
modify linux guest kickstarts/XMLs to add ethtool to the default package
selection.
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/ethtool.py b/client/tests/kvm/tests/ethtool.py
> new file mode 100644
> index 0000000..7274eae
> --- /dev/null
> +++ b/client/tests/kvm/tests/ethtool.py
> @@ -0,0 +1,205 @@
> +import time, os, logging, commands, re
> +from autotest_lib.client.common_lib import error
> +from autotest_lib.client.bin import utils
> +import kvm_test_utils, kvm_utils, kvm_net_utils
> +
> +def run_ethtool(test, params, env):
> + """
> + Test offload functions of ethernet device by ethtool
> +
> + 1) Log into a guest
> + 2) Initialize the callback of sub functions
> + 3) Enable/disable sub function of NIC
> + 4) Execute callback function
> + 5) Check the return value
> + 6) Restore original configuration
> +
> + @param test: Kvm test object
> + @param params: Dictionary with the test parameters.
> + @param env: Dictionary with test environment.
> + """
> + def ethtool_get(type):
> + feature_pattern = {
> + 'tx': 'tx.*checksumming',
> + 'rx': 'rx.*checksumming',
> + 'sg': 'scatter.*gather',
> + 'tso': 'tcp.*segmentation.*offload',
> + 'gso': 'generic.*segmentation.*offload',
> + 'gro': 'generic.*receive.*offload',
> + 'lro': 'large.*receive.*offload',
> + }
> + s, o = session.get_command_status_output("ethtool -k %s" % ethname)
> + try:
> + return re.findall("%s: (.*)" % feature_pattern.get(type), o)[0]
> + except IndexError:
> + logging.debug("Could not get %s status" % type)
> +
> + def ethtool_set(type, status):
> + """
> + Set ethernet device offload status
> +
> + @param type: Offload type name
> + @param status: New status will be changed to
> + """
> + logging.info("Try to set %s %s" % (type, status))
> + if status not in ["off", "on"]:
> + return False
> + cmd = "ethtool -K %s %s %s" % (ethname, type, status)
> + if ethtool_get(type) != status:
> + return session.get_command_status(cmd) == 0
> + if ethtool_get(type) != status:
> + logging.error("Fail to set %s %s" % (type, status))
> + return False
> + return True
> +
> + def ethtool_save_params():
> + logging.info("Save ethtool configuration")
> + for i in supported_features:
> + feature_status[i] = ethtool_get(i)
> +
> + def ethtool_restore_params():
> + logging.info("Restore ethtool configuration")
> + for i in supported_features:
> + ethtool_set(i, feature_status[i])
> +
> + def compare_md5sum(name):
> + logging.info("Compare md5sum of the files on guest and host")
> + host_result = utils.hash_file(name, method="md5")
> + try:
> + o = session.get_command_output("md5sum %s" % name)
> + guest_result = re.findall("\w+", o)[0]
> + except IndexError:
> + logging.error("Could not get file md5sum in guest")
> + return False
> + logging.debug("md5sum: guest(%s), host(%s)" % (guest_result,
> + host_result))
> + return guest_result == host_result
> +
> + def transfer_file(src="guest"):
> + """
> + Transfer file by scp, use tcpdump to capture packets, then check the
> + return string.
> +
> + @param src: Source host of transfer file
> + @return: Tuple (status, error msg/tcpdump result)
> + """
> + session2.get_command_status("rm -rf %s" % filename)
> + dd_cmd = "dd if=/dev/urandom of=%s bs=1M count=%s" % (filename,
> + params.get("filesize"))
> + logging.info("Creat file in source host, cmd: %s" % dd_cmd)
> + tcpdump_cmd = "tcpdump -lep -s 0 tcp -vv port ssh"
> + if src == "guest":
> + s = session.get_command_status(dd_cmd, timeout=360)
> + tcpdump_cmd += " and src %s" % guest_ip
> + copy_files_fun = vm.copy_files_from
> + else:
> + s, o = commands.getstatusoutput(dd_cmd)
> + tcpdump_cmd += " and dst %s" % guest_ip
> + copy_files_fun = vm.copy_files_to
> + if s != 0:
> + return (False, "Fail to create file by dd, cmd: %s" % dd_cmd)
> +
> + # only capture the new tcp port after offload setup
> + original_tcp_ports = re.findall("tcp.*:(\d+).*%s" % guest_ip,
> + commands.getoutput("/bin/netstat -nap"))
> + for i in original_tcp_ports:
> + tcpdump_cmd += " and not port %s" % i
> + logging.debug("Listen by command: %s" % tcpdump_cmd)
> + session2.sendline(tcpdump_cmd)
> + if not kvm_utils.wait_for(lambda: session.get_command_status(
> + "pgrep tcpdump") == 0, 30):
> + return (False, "Tcpdump process wasn't launched")
> +
> + logging.info("Start to transfer file")
> + if not copy_files_fun(filename, filename):
> + return (False, "Child process transfer file failed")
> + logging.info("Transfer file completed")
> + if session.get_command_status("killall tcpdump") != 0:
> + return (False, "Could not kill all tcpdump process")
> + s, tcpdump_string = session2.read_up_to_prompt(timeout=60)
> + if not s:
> + return (False, "Fail to read tcpdump's output")
> +
> + if not compare_md5sum(filename):
> + return (False, "Files' md5sum mismatched")
> + return (True, tcpdump_string)
> +
> + def tx_callback(status="on"):
> + s, o = transfer_file(src="guest")
> + if not s:
> + logging.error(o)
> + return False
> + return True
> +
> + def rx_callback(status="on"):
> + s, o = transfer_file(src="host")
> + if not s:
> + logging.error(o)
> + return False
> + return True
> +
> + def so_callback(status="on"):
> + s, o = transfer_file(src="guest")
> + if not s:
> + logging.error(o)
> + return False
> + logging.info("Check if contained large frame")
> + # mtu: default IPv4 MTU is 1500 Bytes, ethernet header is 14 Bytes
> + return (status == "on") ^ (len([i for i in re.findall(
> + "length (\d*):", o) if int(i) > mtu]) == 0)
> +
> + def ro_callback(status="on"):
> + s, o = transfer_file(src="host")
> + if not s:
> + logging.error(o)
> + return False
> + return True
> +
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + session = kvm_test_utils.wait_for_login(vm,
> + timeout=int(params.get("login_timeout", 360)))
> + session2 = kvm_test_utils.wait_for_login(vm,
> + timeout=int(params.get("login_timeout", 360)))
> + mtu = 1514
> + feature_status = {}
> + filename = "/tmp/ethtool.dd"
> + guest_ip = vm.get_address()
> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
> + supported_features = params.get("supported_features").split()
^ We could use ethtool to query the machine for the supported features,
and if it's not possible to do that (virtio_net), we resort to
supported_feature being set on the config file, what do you think?
> + test_matrix = {
> + # type:(callback, (dependence), (exclude)
> + "tx": (tx_callback, (), ()),
> + "rx": (rx_callback, (), ()),
> + "sg": (tx_callback, ("tx",), ()),
> + "tso": (so_callback, ("tx", "sg",), ("gso",)),
> + "gso": (so_callback, (), ("tso",)),
> + "gro": (ro_callback, ("rx",), ("lro",)),
> + "lro": (rx_callback, (), ("gro",)),
> + }
> + ethtool_save_params()
> + success = True
> + try:
> + for type in supported_features:
> + callback = test_matrix[type][0]
> + for i in test_matrix[type][2]:
> + if not ethtool_set(i, "off"):
> + logging.error("Fail to disable %s" % i)
> + success = False
> + for i in [f for f in test_matrix[type][1]] + [type]:
> + if not ethtool_set(i, "on"):
> + logging.error("Fail to enable %s" % i)
> + success = False
> + if not callback():
> + raise error.TestFail("Test failed, %s: on" % type)
> +
> + if not ethtool_set(type, "off"):
> + logging.error("Fail to disable %s" % type)
> + success = False
> + if not callback(status="off"):
> + raise error.TestFail("Test failed, %s: off" % type)
> + if not success:
> + raise error.TestError("Enable/disable offload function fail")
> + finally:
> + ethtool_restore_params()
> + session.close()
> + session2.close()
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index b25980e..6f0e295 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -412,6 +412,19 @@ variants:
> netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1
> protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM"
>
> + - ethtool: install setup unattended_install.cdrom
> + type = ethtool
> + filesize = 512
> + nic_mode = tap
> + variants:
> + # gso gro lro is only supported by latest kernel
> + - nic_virtio:
> + pci_model = virtio
> + supported_features = "tx sg tso gso"
> + - nic_e1000:
> + pci_model = e1000
> + supported_features = "tx rx sg tso gso gro lro"
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> @@ -1088,7 +1101,7 @@ variants:
>
> # Windows section
> - @Windows:
> - no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
> + no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change ethtool
> shutdown_command = shutdown /s /f /t 0
> reboot_command = shutdown /r /f /t 0
> status_test_command = echo %errorlevel%
>
^ permalink raw reply [flat|nested] 77+ messages in thread* [Qemu-devel] Re: [RFC PATCH 14/14] KVM-test: Add subtest of testing offload by ethtool
@ 2010-08-02 19:10 ` Lucas Meneghel Rodrigues
0 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-08-02 19:10 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:36 +0800, Amos Kong wrote:
> The latest case contains TX/RX/SG/TSO/GSO/GRO/LRO test. RTL8139 NIC doesn't
> support TSO, LRO, it's too old, so drop offload test from rtl8139. LRO, GRO
> are only supported by latest kernel, virtio nic doesn't support receive
> offloading function.
> Initialize the callbacks first and execute all the sub tests one by one, all
> the result will be check at the end.
> When execute this test, vhost should be enabled, then most of new feature can
> be used. Vhost doestn't support VIRTIO_NET_F_MRG_RXBUF, so do not check large
> packets in received offload test.
> Transfer files by scp between host and guest, match new opened TCP port by
> netstat. Capture the packages info by tcpdump, it contains package length.
This test is heavily dependent on ethtool, so we need to make sure the
package is going to be installed on linux guests. The default package
selection for Fedora 13 does not include it for example. So, we need to
modify linux guest kickstarts/XMLs to add ethtool to the default package
selection.
> Signed-off-by: Amos Kong <akong@redhat.com>
> ---
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/tests/ethtool.py b/client/tests/kvm/tests/ethtool.py
> new file mode 100644
> index 0000000..7274eae
> --- /dev/null
> +++ b/client/tests/kvm/tests/ethtool.py
> @@ -0,0 +1,205 @@
> +import time, os, logging, commands, re
> +from autotest_lib.client.common_lib import error
> +from autotest_lib.client.bin import utils
> +import kvm_test_utils, kvm_utils, kvm_net_utils
> +
> +def run_ethtool(test, params, env):
> + """
> + Test offload functions of ethernet device by ethtool
> +
> + 1) Log into a guest
> + 2) Initialize the callback of sub functions
> + 3) Enable/disable sub function of NIC
> + 4) Execute callback function
> + 5) Check the return value
> + 6) Restore original configuration
> +
> + @param test: Kvm test object
> + @param params: Dictionary with the test parameters.
> + @param env: Dictionary with test environment.
> + """
> + def ethtool_get(type):
> + feature_pattern = {
> + 'tx': 'tx.*checksumming',
> + 'rx': 'rx.*checksumming',
> + 'sg': 'scatter.*gather',
> + 'tso': 'tcp.*segmentation.*offload',
> + 'gso': 'generic.*segmentation.*offload',
> + 'gro': 'generic.*receive.*offload',
> + 'lro': 'large.*receive.*offload',
> + }
> + s, o = session.get_command_status_output("ethtool -k %s" % ethname)
> + try:
> + return re.findall("%s: (.*)" % feature_pattern.get(type), o)[0]
> + except IndexError:
> + logging.debug("Could not get %s status" % type)
> +
> + def ethtool_set(type, status):
> + """
> + Set ethernet device offload status
> +
> + @param type: Offload type name
> + @param status: New status will be changed to
> + """
> + logging.info("Try to set %s %s" % (type, status))
> + if status not in ["off", "on"]:
> + return False
> + cmd = "ethtool -K %s %s %s" % (ethname, type, status)
> + if ethtool_get(type) != status:
> + return session.get_command_status(cmd) == 0
> + if ethtool_get(type) != status:
> + logging.error("Fail to set %s %s" % (type, status))
> + return False
> + return True
> +
> + def ethtool_save_params():
> + logging.info("Save ethtool configuration")
> + for i in supported_features:
> + feature_status[i] = ethtool_get(i)
> +
> + def ethtool_restore_params():
> + logging.info("Restore ethtool configuration")
> + for i in supported_features:
> + ethtool_set(i, feature_status[i])
> +
> + def compare_md5sum(name):
> + logging.info("Compare md5sum of the files on guest and host")
> + host_result = utils.hash_file(name, method="md5")
> + try:
> + o = session.get_command_output("md5sum %s" % name)
> + guest_result = re.findall("\w+", o)[0]
> + except IndexError:
> + logging.error("Could not get file md5sum in guest")
> + return False
> + logging.debug("md5sum: guest(%s), host(%s)" % (guest_result,
> + host_result))
> + return guest_result == host_result
> +
> + def transfer_file(src="guest"):
> + """
> + Transfer file by scp, use tcpdump to capture packets, then check the
> + return string.
> +
> + @param src: Source host of transfer file
> + @return: Tuple (status, error msg/tcpdump result)
> + """
> + session2.get_command_status("rm -rf %s" % filename)
> + dd_cmd = "dd if=/dev/urandom of=%s bs=1M count=%s" % (filename,
> + params.get("filesize"))
> + logging.info("Creat file in source host, cmd: %s" % dd_cmd)
> + tcpdump_cmd = "tcpdump -lep -s 0 tcp -vv port ssh"
> + if src == "guest":
> + s = session.get_command_status(dd_cmd, timeout=360)
> + tcpdump_cmd += " and src %s" % guest_ip
> + copy_files_fun = vm.copy_files_from
> + else:
> + s, o = commands.getstatusoutput(dd_cmd)
> + tcpdump_cmd += " and dst %s" % guest_ip
> + copy_files_fun = vm.copy_files_to
> + if s != 0:
> + return (False, "Fail to create file by dd, cmd: %s" % dd_cmd)
> +
> + # only capture the new tcp port after offload setup
> + original_tcp_ports = re.findall("tcp.*:(\d+).*%s" % guest_ip,
> + commands.getoutput("/bin/netstat -nap"))
> + for i in original_tcp_ports:
> + tcpdump_cmd += " and not port %s" % i
> + logging.debug("Listen by command: %s" % tcpdump_cmd)
> + session2.sendline(tcpdump_cmd)
> + if not kvm_utils.wait_for(lambda: session.get_command_status(
> + "pgrep tcpdump") == 0, 30):
> + return (False, "Tcpdump process wasn't launched")
> +
> + logging.info("Start to transfer file")
> + if not copy_files_fun(filename, filename):
> + return (False, "Child process transfer file failed")
> + logging.info("Transfer file completed")
> + if session.get_command_status("killall tcpdump") != 0:
> + return (False, "Could not kill all tcpdump process")
> + s, tcpdump_string = session2.read_up_to_prompt(timeout=60)
> + if not s:
> + return (False, "Fail to read tcpdump's output")
> +
> + if not compare_md5sum(filename):
> + return (False, "Files' md5sum mismatched")
> + return (True, tcpdump_string)
> +
> + def tx_callback(status="on"):
> + s, o = transfer_file(src="guest")
> + if not s:
> + logging.error(o)
> + return False
> + return True
> +
> + def rx_callback(status="on"):
> + s, o = transfer_file(src="host")
> + if not s:
> + logging.error(o)
> + return False
> + return True
> +
> + def so_callback(status="on"):
> + s, o = transfer_file(src="guest")
> + if not s:
> + logging.error(o)
> + return False
> + logging.info("Check if contained large frame")
> + # mtu: default IPv4 MTU is 1500 Bytes, ethernet header is 14 Bytes
> + return (status == "on") ^ (len([i for i in re.findall(
> + "length (\d*):", o) if int(i) > mtu]) == 0)
> +
> + def ro_callback(status="on"):
> + s, o = transfer_file(src="host")
> + if not s:
> + logging.error(o)
> + return False
> + return True
> +
> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
> + session = kvm_test_utils.wait_for_login(vm,
> + timeout=int(params.get("login_timeout", 360)))
> + session2 = kvm_test_utils.wait_for_login(vm,
> + timeout=int(params.get("login_timeout", 360)))
> + mtu = 1514
> + feature_status = {}
> + filename = "/tmp/ethtool.dd"
> + guest_ip = vm.get_address()
> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
> + supported_features = params.get("supported_features").split()
^ We could use ethtool to query the machine for the supported features,
and if it's not possible to do that (virtio_net), we resort to
supported_feature being set on the config file, what do you think?
> + test_matrix = {
> + # type:(callback, (dependence), (exclude)
> + "tx": (tx_callback, (), ()),
> + "rx": (rx_callback, (), ()),
> + "sg": (tx_callback, ("tx",), ()),
> + "tso": (so_callback, ("tx", "sg",), ("gso",)),
> + "gso": (so_callback, (), ("tso",)),
> + "gro": (ro_callback, ("rx",), ("lro",)),
> + "lro": (rx_callback, (), ("gro",)),
> + }
> + ethtool_save_params()
> + success = True
> + try:
> + for type in supported_features:
> + callback = test_matrix[type][0]
> + for i in test_matrix[type][2]:
> + if not ethtool_set(i, "off"):
> + logging.error("Fail to disable %s" % i)
> + success = False
> + for i in [f for f in test_matrix[type][1]] + [type]:
> + if not ethtool_set(i, "on"):
> + logging.error("Fail to enable %s" % i)
> + success = False
> + if not callback():
> + raise error.TestFail("Test failed, %s: on" % type)
> +
> + if not ethtool_set(type, "off"):
> + logging.error("Fail to disable %s" % type)
> + success = False
> + if not callback(status="off"):
> + raise error.TestFail("Test failed, %s: off" % type)
> + if not success:
> + raise error.TestError("Enable/disable offload function fail")
> + finally:
> + ethtool_restore_params()
> + session.close()
> + session2.close()
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index b25980e..6f0e295 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -412,6 +412,19 @@ variants:
> netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1
> protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM"
>
> + - ethtool: install setup unattended_install.cdrom
> + type = ethtool
> + filesize = 512
> + nic_mode = tap
> + variants:
> + # gso gro lro is only supported by latest kernel
> + - nic_virtio:
> + pci_model = virtio
> + supported_features = "tx sg tso gso"
> + - nic_e1000:
> + pci_model = e1000
> + supported_features = "tx rx sg tso gso gro lro"
> +
> - physical_resources_check: install setup unattended_install.cdrom
> type = physical_resources_check
> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
> @@ -1088,7 +1101,7 @@ variants:
>
> # Windows section
> - @Windows:
> - no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
> + no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change ethtool
> shutdown_command = shutdown /s /f /t 0
> reboot_command = shutdown /r /f /t 0
> status_test_command = echo %errorlevel%
>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [RFC PATCH 14/14] KVM-test: Add subtest of testing offload by ethtool
2010-08-02 19:10 ` [Qemu-devel] " Lucas Meneghel Rodrigues
@ 2010-08-10 7:07 ` Amos Kong
-1 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-10 7:07 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: autotest, qemu-devel, kvm
On Tue, Aug 3, 2010 at 3:10 AM, Lucas Meneghel Rodrigues <lmr@redhat.com> wrote:
> On Tue, 2010-07-20 at 09:36 +0800, Amos Kong wrote:
>> The latest case contains TX/RX/SG/TSO/GSO/GRO/LRO test. RTL8139 NIC doesn't
>> support TSO, LRO, it's too old, so drop offload test from rtl8139. LRO, GRO
>> are only supported by latest kernel, virtio nic doesn't support receive
>> offloading function.
>> Initialize the callbacks first and execute all the sub tests one by one, all
>> the result will be check at the end.
>> When execute this test, vhost should be enabled, then most of new feature can
>> be used. Vhost doestn't support VIRTIO_NET_F_MRG_RXBUF, so do not check large
>> packets in received offload test.
>> Transfer files by scp between host and guest, match new opened TCP port by
>> netstat. Capture the packages info by tcpdump, it contains package length.
>
> This test is heavily dependent on ethtool, so we need to make sure the
> package is going to be installed on linux guests. The default package
> selection for Fedora 13 does not include it for example. So, we need to
> modify linux guest kickstarts/XMLs to add ethtool to the default package
> selection.
OK.
>> Signed-off-by: Amos Kong <akong@redhat.com>
>> ---
>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>
>> diff --git a/client/tests/kvm/tests/ethtool.py b/client/tests/kvm/tests/ethtool.py
>> new file mode 100644
>> index 0000000..7274eae
>> --- /dev/null
>> +++ b/client/tests/kvm/tests/ethtool.py
>> @@ -0,0 +1,205 @@
>> +import time, os, logging, commands, re
>> +from autotest_lib.client.common_lib import error
>> +from autotest_lib.client.bin import utils
>> +import kvm_test_utils, kvm_utils, kvm_net_utils
>> +
>> +def run_ethtool(test, params, env):
>> + """
>> + Test offload functions of ethernet device by ethtool
>> +
>> + 1) Log into a guest
>> + 2) Initialize the callback of sub functions
>> + 3) Enable/disable sub function of NIC
>> + 4) Execute callback function
>> + 5) Check the return value
>> + 6) Restore original configuration
>> +
>> + @param test: Kvm test object
>> + @param params: Dictionary with the test parameters.
>> + @param env: Dictionary with test environment.
>> + """
>> + def ethtool_get(type):
>> + feature_pattern = {
>> + 'tx': 'tx.*checksumming',
>> + 'rx': 'rx.*checksumming',
>> + 'sg': 'scatter.*gather',
>> + 'tso': 'tcp.*segmentation.*offload',
>> + 'gso': 'generic.*segmentation.*offload',
>> + 'gro': 'generic.*receive.*offload',
>> + 'lro': 'large.*receive.*offload',
>> + }
>> + s, o = session.get_command_status_output("ethtool -k %s" % ethname)
>> + try:
>> + return re.findall("%s: (.*)" % feature_pattern.get(type), o)[0]
>> + except IndexError:
>> + logging.debug("Could not get %s status" % type)
>> +
>> + def ethtool_set(type, status):
>> + """
>> + Set ethernet device offload status
>> +
>> + @param type: Offload type name
>> + @param status: New status will be changed to
>> + """
>> + logging.info("Try to set %s %s" % (type, status))
>> + if status not in ["off", "on"]:
>> + return False
>> + cmd = "ethtool -K %s %s %s" % (ethname, type, status)
>> + if ethtool_get(type) != status:
>> + return session.get_command_status(cmd) == 0
>> + if ethtool_get(type) != status:
>> + logging.error("Fail to set %s %s" % (type, status))
>> + return False
>> + return True
>> +
>> + def ethtool_save_params():
>> + logging.info("Save ethtool configuration")
>> + for i in supported_features:
>> + feature_status[i] = ethtool_get(i)
>> +
>> + def ethtool_restore_params():
>> + logging.info("Restore ethtool configuration")
>> + for i in supported_features:
>> + ethtool_set(i, feature_status[i])
>> +
>> + def compare_md5sum(name):
>> + logging.info("Compare md5sum of the files on guest and host")
>> + host_result = utils.hash_file(name, method="md5")
>> + try:
>> + o = session.get_command_output("md5sum %s" % name)
>> + guest_result = re.findall("\w+", o)[0]
>> + except IndexError:
>> + logging.error("Could not get file md5sum in guest")
>> + return False
>> + logging.debug("md5sum: guest(%s), host(%s)" % (guest_result,
>> + host_result))
>> + return guest_result == host_result
>> +
>> + def transfer_file(src="guest"):
>> + """
>> + Transfer file by scp, use tcpdump to capture packets, then check the
>> + return string.
>> +
>> + @param src: Source host of transfer file
>> + @return: Tuple (status, error msg/tcpdump result)
>> + """
>> + session2.get_command_status("rm -rf %s" % filename)
>> + dd_cmd = "dd if=/dev/urandom of=%s bs=1M count=%s" % (filename,
>> + params.get("filesize"))
>> + logging.info("Creat file in source host, cmd: %s" % dd_cmd)
>> + tcpdump_cmd = "tcpdump -lep -s 0 tcp -vv port ssh"
>> + if src == "guest":
>> + s = session.get_command_status(dd_cmd, timeout=360)
>> + tcpdump_cmd += " and src %s" % guest_ip
>> + copy_files_fun = vm.copy_files_from
>> + else:
>> + s, o = commands.getstatusoutput(dd_cmd)
>> + tcpdump_cmd += " and dst %s" % guest_ip
>> + copy_files_fun = vm.copy_files_to
>> + if s != 0:
>> + return (False, "Fail to create file by dd, cmd: %s" % dd_cmd)
>> +
>> + # only capture the new tcp port after offload setup
>> + original_tcp_ports = re.findall("tcp.*:(\d+).*%s" % guest_ip,
>> + commands.getoutput("/bin/netstat -nap"))
>> + for i in original_tcp_ports:
>> + tcpdump_cmd += " and not port %s" % i
>> + logging.debug("Listen by command: %s" % tcpdump_cmd)
>> + session2.sendline(tcpdump_cmd)
>> + if not kvm_utils.wait_for(lambda: session.get_command_status(
>> + "pgrep tcpdump") == 0, 30):
>> + return (False, "Tcpdump process wasn't launched")
>> +
>> + logging.info("Start to transfer file")
>> + if not copy_files_fun(filename, filename):
>> + return (False, "Child process transfer file failed")
>> + logging.info("Transfer file completed")
>> + if session.get_command_status("killall tcpdump") != 0:
>> + return (False, "Could not kill all tcpdump process")
>> + s, tcpdump_string = session2.read_up_to_prompt(timeout=60)
>> + if not s:
>> + return (False, "Fail to read tcpdump's output")
>> +
>> + if not compare_md5sum(filename):
>> + return (False, "Files' md5sum mismatched")
>> + return (True, tcpdump_string)
>> +
>> + def tx_callback(status="on"):
>> + s, o = transfer_file(src="guest")
>> + if not s:
>> + logging.error(o)
>> + return False
>> + return True
>> +
>> + def rx_callback(status="on"):
>> + s, o = transfer_file(src="host")
>> + if not s:
>> + logging.error(o)
>> + return False
>> + return True
>> +
>> + def so_callback(status="on"):
>> + s, o = transfer_file(src="guest")
>> + if not s:
>> + logging.error(o)
>> + return False
>> + logging.info("Check if contained large frame")
>> + # mtu: default IPv4 MTU is 1500 Bytes, ethernet header is 14 Bytes
>> + return (status == "on") ^ (len([i for i in re.findall(
>> + "length (\d*):", o) if int(i) > mtu]) == 0)
>> +
>> + def ro_callback(status="on"):
>> + s, o = transfer_file(src="host")
>> + if not s:
>> + logging.error(o)
>> + return False
>> + return True
>> +
>> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
>> + session = kvm_test_utils.wait_for_login(vm,
>> + timeout=int(params.get("login_timeout", 360)))
>> + session2 = kvm_test_utils.wait_for_login(vm,
>> + timeout=int(params.get("login_timeout", 360)))
>> + mtu = 1514
>> + feature_status = {}
>> + filename = "/tmp/ethtool.dd"
>> + guest_ip = vm.get_address()
>> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
>> + supported_features = params.get("supported_features").split()
>
> ^ We could use ethtool to query the machine for the supported features,
> and if it's not possible to do that (virtio_net), we resort to
> supported_feature being set on the config file, what do you think?
How to query which offload features are supported by nic dev by ethtool ?
'# ethtool -k eth0' can only list the function status.
I try to enable all the function, by this order 'tx rx gs tso ufo gso
gro lro', if some of them failed, it should not be supported ? This
method is not dependable in test environment.
>> + test_matrix = {
>> + # type:(callback, (dependence), (exclude)
>> + "tx": (tx_callback, (), ()),
>> + "rx": (rx_callback, (), ()),
>> + "sg": (tx_callback, ("tx",), ()),
>> + "tso": (so_callback, ("tx", "sg",), ("gso",)),
>> + "gso": (so_callback, (), ("tso",)),
>> + "gro": (ro_callback, ("rx",), ("lro",)),
>> + "lro": (rx_callback, (), ("gro",)),
>> + }
>> + ethtool_save_params()
>> + success = True
>> + try:
>> + for type in supported_features:
>> + callback = test_matrix[type][0]
>> + for i in test_matrix[type][2]:
>> + if not ethtool_set(i, "off"):
>> + logging.error("Fail to disable %s" % i)
>> + success = False
>> + for i in [f for f in test_matrix[type][1]] + [type]:
>> + if not ethtool_set(i, "on"):
>> + logging.error("Fail to enable %s" % i)
>> + success = False
>> + if not callback():
>> + raise error.TestFail("Test failed, %s: on" % type)
>> +
>> + if not ethtool_set(type, "off"):
>> + logging.error("Fail to disable %s" % type)
>> + success = False
>> + if not callback(status="off"):
>> + raise error.TestFail("Test failed, %s: off" % type)
>> + if not success:
>> + raise error.TestError("Enable/disable offload function fail")
>> + finally:
>> + ethtool_restore_params()
>> + session.close()
>> + session2.close()
>> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
>> index b25980e..6f0e295 100644
>> --- a/client/tests/kvm/tests_base.cfg.sample
>> +++ b/client/tests/kvm/tests_base.cfg.sample
>> @@ -412,6 +412,19 @@ variants:
>> netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1
>> protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM"
>>
>> + - ethtool: install setup unattended_install.cdrom
>> + type = ethtool
>> + filesize = 512
>> + nic_mode = tap
>> + variants:
>> + # gso gro lro is only supported by latest kernel
>> + - nic_virtio:
>> + pci_model = virtio
>> + supported_features = "tx sg tso gso"
>> + - nic_e1000:
>> + pci_model = e1000
>> + supported_features = "tx rx sg tso gso gro lro"
>> +
>> - physical_resources_check: install setup unattended_install.cdrom
>> type = physical_resources_check
>> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
>> @@ -1088,7 +1101,7 @@ variants:
>>
>> # Windows section
>> - @Windows:
>> - no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
>> + no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change ethtool
>> shutdown_command = shutdown /s /f /t 0
>> reboot_command = shutdown /r /f /t 0
>> status_test_command = echo %errorlevel%
^ permalink raw reply [flat|nested] 77+ messages in thread* [Qemu-devel] Re: [Autotest] [RFC PATCH 14/14] KVM-test: Add subtest of testing offload by ethtool
@ 2010-08-10 7:07 ` Amos Kong
0 siblings, 0 replies; 77+ messages in thread
From: Amos Kong @ 2010-08-10 7:07 UTC (permalink / raw)
To: Lucas Meneghel Rodrigues; +Cc: autotest, Amos Kong, qemu-devel, kvm
On Tue, Aug 3, 2010 at 3:10 AM, Lucas Meneghel Rodrigues <lmr@redhat.com> wrote:
> On Tue, 2010-07-20 at 09:36 +0800, Amos Kong wrote:
>> The latest case contains TX/RX/SG/TSO/GSO/GRO/LRO test. RTL8139 NIC doesn't
>> support TSO, LRO, it's too old, so drop offload test from rtl8139. LRO, GRO
>> are only supported by latest kernel, virtio nic doesn't support receive
>> offloading function.
>> Initialize the callbacks first and execute all the sub tests one by one, all
>> the result will be check at the end.
>> When execute this test, vhost should be enabled, then most of new feature can
>> be used. Vhost doestn't support VIRTIO_NET_F_MRG_RXBUF, so do not check large
>> packets in received offload test.
>> Transfer files by scp between host and guest, match new opened TCP port by
>> netstat. Capture the packages info by tcpdump, it contains package length.
>
> This test is heavily dependent on ethtool, so we need to make sure the
> package is going to be installed on linux guests. The default package
> selection for Fedora 13 does not include it for example. So, we need to
> modify linux guest kickstarts/XMLs to add ethtool to the default package
> selection.
OK.
>> Signed-off-by: Amos Kong <akong@redhat.com>
>> ---
>> 0 files changed, 0 insertions(+), 0 deletions(-)
>>
>> diff --git a/client/tests/kvm/tests/ethtool.py b/client/tests/kvm/tests/ethtool.py
>> new file mode 100644
>> index 0000000..7274eae
>> --- /dev/null
>> +++ b/client/tests/kvm/tests/ethtool.py
>> @@ -0,0 +1,205 @@
>> +import time, os, logging, commands, re
>> +from autotest_lib.client.common_lib import error
>> +from autotest_lib.client.bin import utils
>> +import kvm_test_utils, kvm_utils, kvm_net_utils
>> +
>> +def run_ethtool(test, params, env):
>> + """
>> + Test offload functions of ethernet device by ethtool
>> +
>> + 1) Log into a guest
>> + 2) Initialize the callback of sub functions
>> + 3) Enable/disable sub function of NIC
>> + 4) Execute callback function
>> + 5) Check the return value
>> + 6) Restore original configuration
>> +
>> + @param test: Kvm test object
>> + @param params: Dictionary with the test parameters.
>> + @param env: Dictionary with test environment.
>> + """
>> + def ethtool_get(type):
>> + feature_pattern = {
>> + 'tx': 'tx.*checksumming',
>> + 'rx': 'rx.*checksumming',
>> + 'sg': 'scatter.*gather',
>> + 'tso': 'tcp.*segmentation.*offload',
>> + 'gso': 'generic.*segmentation.*offload',
>> + 'gro': 'generic.*receive.*offload',
>> + 'lro': 'large.*receive.*offload',
>> + }
>> + s, o = session.get_command_status_output("ethtool -k %s" % ethname)
>> + try:
>> + return re.findall("%s: (.*)" % feature_pattern.get(type), o)[0]
>> + except IndexError:
>> + logging.debug("Could not get %s status" % type)
>> +
>> + def ethtool_set(type, status):
>> + """
>> + Set ethernet device offload status
>> +
>> + @param type: Offload type name
>> + @param status: New status will be changed to
>> + """
>> + logging.info("Try to set %s %s" % (type, status))
>> + if status not in ["off", "on"]:
>> + return False
>> + cmd = "ethtool -K %s %s %s" % (ethname, type, status)
>> + if ethtool_get(type) != status:
>> + return session.get_command_status(cmd) == 0
>> + if ethtool_get(type) != status:
>> + logging.error("Fail to set %s %s" % (type, status))
>> + return False
>> + return True
>> +
>> + def ethtool_save_params():
>> + logging.info("Save ethtool configuration")
>> + for i in supported_features:
>> + feature_status[i] = ethtool_get(i)
>> +
>> + def ethtool_restore_params():
>> + logging.info("Restore ethtool configuration")
>> + for i in supported_features:
>> + ethtool_set(i, feature_status[i])
>> +
>> + def compare_md5sum(name):
>> + logging.info("Compare md5sum of the files on guest and host")
>> + host_result = utils.hash_file(name, method="md5")
>> + try:
>> + o = session.get_command_output("md5sum %s" % name)
>> + guest_result = re.findall("\w+", o)[0]
>> + except IndexError:
>> + logging.error("Could not get file md5sum in guest")
>> + return False
>> + logging.debug("md5sum: guest(%s), host(%s)" % (guest_result,
>> + host_result))
>> + return guest_result == host_result
>> +
>> + def transfer_file(src="guest"):
>> + """
>> + Transfer file by scp, use tcpdump to capture packets, then check the
>> + return string.
>> +
>> + @param src: Source host of transfer file
>> + @return: Tuple (status, error msg/tcpdump result)
>> + """
>> + session2.get_command_status("rm -rf %s" % filename)
>> + dd_cmd = "dd if=/dev/urandom of=%s bs=1M count=%s" % (filename,
>> + params.get("filesize"))
>> + logging.info("Creat file in source host, cmd: %s" % dd_cmd)
>> + tcpdump_cmd = "tcpdump -lep -s 0 tcp -vv port ssh"
>> + if src == "guest":
>> + s = session.get_command_status(dd_cmd, timeout=360)
>> + tcpdump_cmd += " and src %s" % guest_ip
>> + copy_files_fun = vm.copy_files_from
>> + else:
>> + s, o = commands.getstatusoutput(dd_cmd)
>> + tcpdump_cmd += " and dst %s" % guest_ip
>> + copy_files_fun = vm.copy_files_to
>> + if s != 0:
>> + return (False, "Fail to create file by dd, cmd: %s" % dd_cmd)
>> +
>> + # only capture the new tcp port after offload setup
>> + original_tcp_ports = re.findall("tcp.*:(\d+).*%s" % guest_ip,
>> + commands.getoutput("/bin/netstat -nap"))
>> + for i in original_tcp_ports:
>> + tcpdump_cmd += " and not port %s" % i
>> + logging.debug("Listen by command: %s" % tcpdump_cmd)
>> + session2.sendline(tcpdump_cmd)
>> + if not kvm_utils.wait_for(lambda: session.get_command_status(
>> + "pgrep tcpdump") == 0, 30):
>> + return (False, "Tcpdump process wasn't launched")
>> +
>> + logging.info("Start to transfer file")
>> + if not copy_files_fun(filename, filename):
>> + return (False, "Child process transfer file failed")
>> + logging.info("Transfer file completed")
>> + if session.get_command_status("killall tcpdump") != 0:
>> + return (False, "Could not kill all tcpdump process")
>> + s, tcpdump_string = session2.read_up_to_prompt(timeout=60)
>> + if not s:
>> + return (False, "Fail to read tcpdump's output")
>> +
>> + if not compare_md5sum(filename):
>> + return (False, "Files' md5sum mismatched")
>> + return (True, tcpdump_string)
>> +
>> + def tx_callback(status="on"):
>> + s, o = transfer_file(src="guest")
>> + if not s:
>> + logging.error(o)
>> + return False
>> + return True
>> +
>> + def rx_callback(status="on"):
>> + s, o = transfer_file(src="host")
>> + if not s:
>> + logging.error(o)
>> + return False
>> + return True
>> +
>> + def so_callback(status="on"):
>> + s, o = transfer_file(src="guest")
>> + if not s:
>> + logging.error(o)
>> + return False
>> + logging.info("Check if contained large frame")
>> + # mtu: default IPv4 MTU is 1500 Bytes, ethernet header is 14 Bytes
>> + return (status == "on") ^ (len([i for i in re.findall(
>> + "length (\d*):", o) if int(i) > mtu]) == 0)
>> +
>> + def ro_callback(status="on"):
>> + s, o = transfer_file(src="host")
>> + if not s:
>> + logging.error(o)
>> + return False
>> + return True
>> +
>> + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
>> + session = kvm_test_utils.wait_for_login(vm,
>> + timeout=int(params.get("login_timeout", 360)))
>> + session2 = kvm_test_utils.wait_for_login(vm,
>> + timeout=int(params.get("login_timeout", 360)))
>> + mtu = 1514
>> + feature_status = {}
>> + filename = "/tmp/ethtool.dd"
>> + guest_ip = vm.get_address()
>> + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0))
>> + supported_features = params.get("supported_features").split()
>
> ^ We could use ethtool to query the machine for the supported features,
> and if it's not possible to do that (virtio_net), we resort to
> supported_feature being set on the config file, what do you think?
How to query which offload features are supported by nic dev by ethtool ?
'# ethtool -k eth0' can only list the function status.
I try to enable all the function, by this order 'tx rx gs tso ufo gso
gro lro', if some of them failed, it should not be supported ? This
method is not dependable in test environment.
>> + test_matrix = {
>> + # type:(callback, (dependence), (exclude)
>> + "tx": (tx_callback, (), ()),
>> + "rx": (rx_callback, (), ()),
>> + "sg": (tx_callback, ("tx",), ()),
>> + "tso": (so_callback, ("tx", "sg",), ("gso",)),
>> + "gso": (so_callback, (), ("tso",)),
>> + "gro": (ro_callback, ("rx",), ("lro",)),
>> + "lro": (rx_callback, (), ("gro",)),
>> + }
>> + ethtool_save_params()
>> + success = True
>> + try:
>> + for type in supported_features:
>> + callback = test_matrix[type][0]
>> + for i in test_matrix[type][2]:
>> + if not ethtool_set(i, "off"):
>> + logging.error("Fail to disable %s" % i)
>> + success = False
>> + for i in [f for f in test_matrix[type][1]] + [type]:
>> + if not ethtool_set(i, "on"):
>> + logging.error("Fail to enable %s" % i)
>> + success = False
>> + if not callback():
>> + raise error.TestFail("Test failed, %s: on" % type)
>> +
>> + if not ethtool_set(type, "off"):
>> + logging.error("Fail to disable %s" % type)
>> + success = False
>> + if not callback(status="off"):
>> + raise error.TestFail("Test failed, %s: off" % type)
>> + if not success:
>> + raise error.TestError("Enable/disable offload function fail")
>> + finally:
>> + ethtool_restore_params()
>> + session.close()
>> + session2.close()
>> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
>> index b25980e..6f0e295 100644
>> --- a/client/tests/kvm/tests_base.cfg.sample
>> +++ b/client/tests/kvm/tests_base.cfg.sample
>> @@ -412,6 +412,19 @@ variants:
>> netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1
>> protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM"
>>
>> + - ethtool: install setup unattended_install.cdrom
>> + type = ethtool
>> + filesize = 512
>> + nic_mode = tap
>> + variants:
>> + # gso gro lro is only supported by latest kernel
>> + - nic_virtio:
>> + pci_model = virtio
>> + supported_features = "tx sg tso gso"
>> + - nic_e1000:
>> + pci_model = e1000
>> + supported_features = "tx rx sg tso gso gro lro"
>> +
>> - physical_resources_check: install setup unattended_install.cdrom
>> type = physical_resources_check
>> catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
>> @@ -1088,7 +1101,7 @@ variants:
>>
>> # Windows section
>> - @Windows:
>> - no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change
>> + no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change ethtool
>> shutdown_command = shutdown /s /f /t 0
>> reboot_command = shutdown /r /f /t 0
>> status_test_command = echo %errorlevel%
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [RFC PATCH 00/14] Patchset of network related subtests
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-07-20 12:12 ` Lucas Meneghel Rodrigues
-1 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-20 12:12 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:34 +0800, Amos Kong wrote:
> The following series contain 11 network related subtests, welcome to give me
> some suggestions about correctness, design, enhancement.
Awesome work, will start to review them today. Thanks!
> Thank you so much!
>
> ---
>
> Amos Kong (14):
> KVM-test: Add a new macaddress pool algorithm
> KVM Test: Add a function get_interface_name() to kvm_net_utils.py
> KVM Test: Add a common ping module for network related tests
> KVM-test: Add a new subtest ping
> KVM-test: Add a subtest jumbo
> KVM-test: Add basic file transfer test
> KVM-test: Add a subtest of load/unload nic driver
> KVM-test: Add a subtest of nic promisc
> KVM-test: Add a subtest of multicast
> KVM-test: Add a subtest of pxe
> KVM-test: Add a subtest of changing mac address
> KVM-test: Add a subtest of netperf
> KVM-test: Improve vlan subtest
> KVM-test: Add subtest of testing offload by ethtool
>
>
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [Qemu-devel] Re: [Autotest][RFC PATCH 00/14] Patchset of network related subtests
@ 2010-07-20 12:12 ` Lucas Meneghel Rodrigues
0 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-07-20 12:12 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:34 +0800, Amos Kong wrote:
> The following series contain 11 network related subtests, welcome to give me
> some suggestions about correctness, design, enhancement.
Awesome work, will start to review them today. Thanks!
> Thank you so much!
>
> ---
>
> Amos Kong (14):
> KVM-test: Add a new macaddress pool algorithm
> KVM Test: Add a function get_interface_name() to kvm_net_utils.py
> KVM Test: Add a common ping module for network related tests
> KVM-test: Add a new subtest ping
> KVM-test: Add a subtest jumbo
> KVM-test: Add basic file transfer test
> KVM-test: Add a subtest of load/unload nic driver
> KVM-test: Add a subtest of nic promisc
> KVM-test: Add a subtest of multicast
> KVM-test: Add a subtest of pxe
> KVM-test: Add a subtest of changing mac address
> KVM-test: Add a subtest of netperf
> KVM-test: Improve vlan subtest
> KVM-test: Add subtest of testing offload by ethtool
>
>
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [RFC PATCH 00/14] Patchset of network related subtests
2010-07-20 1:34 ` [Qemu-devel] [Autotest][RFC " Amos Kong
@ 2010-08-02 20:58 ` Lucas Meneghel Rodrigues
-1 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-08-02 20:58 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:34 +0800, Amos Kong wrote:
> The following series contain 11 network related subtests, welcome to give me
> some suggestions about correctness, design, enhancement.
>
> Thank you so much!
Ok Amos, now that I made the first review of this patchset, I'll wait
for a v2 and supersed v1.
Thanks!
> ---
>
> Amos Kong (14):
> KVM-test: Add a new macaddress pool algorithm
> KVM Test: Add a function get_interface_name() to kvm_net_utils.py
> KVM Test: Add a common ping module for network related tests
> KVM-test: Add a new subtest ping
> KVM-test: Add a subtest jumbo
> KVM-test: Add basic file transfer test
> KVM-test: Add a subtest of load/unload nic driver
> KVM-test: Add a subtest of nic promisc
> KVM-test: Add a subtest of multicast
> KVM-test: Add a subtest of pxe
> KVM-test: Add a subtest of changing mac address
> KVM-test: Add a subtest of netperf
> KVM-test: Improve vlan subtest
> KVM-test: Add subtest of testing offload by ethtool
>
>
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [Qemu-devel] Re: [Autotest][RFC PATCH 00/14] Patchset of network related subtests
@ 2010-08-02 20:58 ` Lucas Meneghel Rodrigues
0 siblings, 0 replies; 77+ messages in thread
From: Lucas Meneghel Rodrigues @ 2010-08-02 20:58 UTC (permalink / raw)
To: Amos Kong; +Cc: autotest, qemu-devel, kvm
On Tue, 2010-07-20 at 09:34 +0800, Amos Kong wrote:
> The following series contain 11 network related subtests, welcome to give me
> some suggestions about correctness, design, enhancement.
>
> Thank you so much!
Ok Amos, now that I made the first review of this patchset, I'll wait
for a v2 and supersed v1.
Thanks!
> ---
>
> Amos Kong (14):
> KVM-test: Add a new macaddress pool algorithm
> KVM Test: Add a function get_interface_name() to kvm_net_utils.py
> KVM Test: Add a common ping module for network related tests
> KVM-test: Add a new subtest ping
> KVM-test: Add a subtest jumbo
> KVM-test: Add basic file transfer test
> KVM-test: Add a subtest of load/unload nic driver
> KVM-test: Add a subtest of nic promisc
> KVM-test: Add a subtest of multicast
> KVM-test: Add a subtest of pxe
> KVM-test: Add a subtest of changing mac address
> KVM-test: Add a subtest of netperf
> KVM-test: Improve vlan subtest
> KVM-test: Add subtest of testing offload by ethtool
>
>
> 0 files changed, 0 insertions(+), 0 deletions(-)
>
^ permalink raw reply [flat|nested] 77+ messages in thread