* [KVM-AUTOTEST PATCH CORRECTIONS] Corrections to the TAP patchset @ 2009-08-06 18:41 Michael Goldish 2009-08-06 18:41 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish 0 siblings, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-06 18:41 UTC (permalink / raw) To: autotest, kvm I found two mistakes (so far) in the TAP patchset: - Two import lines in kvm_utils.py were commented out (for personal testing) and I forgot to uncomment them before committing, and this breaks kvm_install - qemu-ifup should be executable, but isn't The following patches (1, 3, 11) replace the respective ones from the original patch set. ^ permalink raw reply [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils 2009-08-06 18:41 [KVM-AUTOTEST PATCH CORRECTIONS] Corrections to the TAP patchset Michael Goldish @ 2009-08-06 18:41 ` Michael Goldish 2009-08-06 18:41 ` [KVM-AUTOTEST PATCH 03/12] KVM test: add sample 'qemu-ifup' script Michael Goldish 2009-08-06 18:41 ` [KVM-AUTOTEST PATCH 11/12] KVM test: make VMs use a dynamic MAC-IP mapping generated by tcpdump Michael Goldish 0 siblings, 2 replies; 7+ messages in thread From: Michael Goldish @ 2009-08-06 18:41 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish Add function get_mac_ip_pair_from_dict() which gets a dict (specified in the config file by the user) and fetches a MAC-IP address pair according to a certain syntax. The syntax allows the user to specify a group of MAC-IP address ranges. For example: address_ranges = r1 r2 r3 address_range_base_mac_r1 = 55:44:33:22:11:00 address_range_base_ip_r1 = 10.0.0.0 address_range_size_r1 = 16 address_range_base_mac_r2 = 55:44:33:22:11:40 address_range_base_ip_r2 = 10.0.0.60 address_range_size_r2 = 25 address_range_base_mac_r3 = 55:44:33:22:12:10 address_range_base_ip_r3 = 10.0.1.20 address_range_size_r3 = 230 The lines above may be specified globally, so that they apply to all VMs and all NICs. However, a line similar to the following must be specified per NIC (each VM may have several NICs): address_index = 0 Currently, we usually use a single VM and a single NIC, so specifying address_index once should suffice. If a test requires an additional NIC the user should add something like: address_index_nic2 = 1 The index is simply the index in the MAC-IP table that consists of all the specified ranges. In the above example, if the user specifies an index of 18, the MAC-IP pair will be taken from the second range, because the first range has only 16 entries. When running migration, both the source and destination VMs should have the same address_index, because they should have the same MAC and IP addresses. Note that different copies of the KVM test, running simultaneously in the same network environment, _must_ specify different MAC-IP pools. This can be done in several ways: - By specifying the ranges (as in the example above) in an external file such as /etc/kvm-autotest/client_mac_ip_pool.cfg, and setting up that file for each host manually - By keeping several .cfg files with different names that match the hostname of each host, e.g. mac_ip_pool_hostname1.cfg, mac_ip_pool_hostname2.cfg (hostname1 and hostname2 should be replaced by actual hostnames), and parsing the right file in the control file at runtime - By defining all the different pools in a variants block in a single file, and specifying 'only <hostname>' at runtime in the control file (using config.parse_string()) When we start running in server mode, assigning MAC and IP addresses to hosts can be done automatically by the server, but the user will still be required to specify a single global pool for the server (which the server will divide among the hosts). The address_index parameter will be specified inside the regular config file, and does not need to be different for each host. This patch also adds some small utility functions used by get_mac_ip_pair_from_dict(). Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/kvm_utils.py | 106 +++++++++++++++++++++++++++++++++++++++++ 1 files changed, 106 insertions(+), 0 deletions(-) diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py index e897e78..4891592 100644 --- a/client/tests/kvm/kvm_utils.py +++ b/client/tests/kvm/kvm_utils.py @@ -46,6 +46,112 @@ def get_sub_dict_names(dict, keyword): return [] +# Functions related to MAC/IP addresses + +def mac_str_to_int(addr): + """ + Convert MAC address string to integer. + + @param addr: String representing the MAC address. + """ + return sum(int(s, 16) * 256 ** i + for i, s in enumerate(reversed(addr.split(":")))) + + +def mac_int_to_str(addr): + """ + Convert MAC address integer to string. + + @param addr: Integer representing the MAC address. + """ + return ":".join("%02x" % (addr >> 8 * i & 0xFF) + for i in reversed(range(6))) + + +def ip_str_to_int(addr): + """ + Convert IP address string to integer. + + @param addr: String representing the IP address. + """ + return sum(int(s) * 256 ** i + for i, s in enumerate(reversed(addr.split(".")))) + + +def ip_int_to_str(addr): + """ + Convert IP address integer to string. + + @param addr: Integer representing the IP address. + """ + return ".".join(str(addr >> 8 * i & 0xFF) + for i in reversed(range(4))) + + +def offset_mac(base, offset): + """ + Add offset to a given MAC address. + + @param base: String representing a MAC address. + @param offset: Offset to add to base (integer) + @return: A string representing the offset MAC address. + """ + return mac_int_to_str(mac_str_to_int(base) + offset) + + +def offset_ip(base, offset): + """ + Add offset to a given IP address. + + @param base: String representing an IP address. + @param offset: Offset to add to base (integer) + @return: A string representing the offset IP address. + """ + return ip_int_to_str(ip_str_to_int(base) + offset) + + +def get_mac_ip_pair_from_dict(dict): + """ + Fetch a MAC-IP address pair from dict and return it. + + The parameters in dict are expected to conform to a certain syntax. + Typical usage may be: + + address_ranges = r1 r2 r3 + + address_range_base_mac_r1 = 55:44:33:22:11:00 + address_range_base_ip_r1 = 10.0.0.0 + address_range_size_r1 = 16 + + address_range_base_mac_r2 = 55:44:33:22:11:40 + address_range_base_ip_r2 = 10.0.0.60 + address_range_size_r2 = 25 + + address_range_base_mac_r3 = 55:44:33:22:12:10 + address_range_base_ip_r3 = 10.0.1.20 + address_range_size_r3 = 230 + + address_index = 0 + + All parameters except address_index specify a MAC-IP address pool. The + pool consists of several MAC-IP address ranges. + address_index specified the index of the desired MAC-IP pair from the pool. + + @param dict: The dictionary from which to fetch the addresses. + """ + index = int(dict.get("address_index", 0)) + for mac_range_name in get_sub_dict_names(dict, "address_ranges"): + mac_range_params = get_sub_dict(dict, mac_range_name) + mac_base = mac_range_params.get("address_range_base_mac") + ip_base = mac_range_params.get("address_range_base_ip") + size = int(mac_range_params.get("address_range_size", 1)) + if index < size: + return (mac_base and offset_mac(mac_base, index), + ip_base and offset_ip(ip_base, index)) + index -= size + return (None, None) + + # Functions for working with the environment (a dict-like object) def is_vm(obj): -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 03/12] KVM test: add sample 'qemu-ifup' script 2009-08-06 18:41 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish @ 2009-08-06 18:41 ` Michael Goldish 2009-08-06 18:41 ` [KVM-AUTOTEST PATCH 11/12] KVM test: make VMs use a dynamic MAC-IP mapping generated by tcpdump Michael Goldish 1 sibling, 0 replies; 7+ messages in thread From: Michael Goldish @ 2009-08-06 18:41 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish The script adds a requested interface to an existing bridge. It is meant to be used by qemu when running in TAP mode. Note: the user is responsible for setting up the bridge before running any tests. This can be done with brctl or in any manner that is appropriate for the host OS. It can be done inside 'qemu-ifup' as well, but this sample script doesn't do it. Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/qemu-ifup | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) create mode 100755 client/tests/kvm/qemu-ifup diff --git a/client/tests/kvm/qemu-ifup b/client/tests/kvm/qemu-ifup new file mode 100755 index 0000000..bcd9a7a --- /dev/null +++ b/client/tests/kvm/qemu-ifup @@ -0,0 +1,8 @@ +#!/bin/sh + +# The following expression selects the first bridge listed by 'brctl show'. +# Modify it to suit your needs. +switch=$(/usr/sbin/brctl show | awk 'NR==2 { print $1 }') + +/sbin/ifconfig $1 0.0.0.0 up +/usr/sbin/brctl addif ${switch} $1 -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 11/12] KVM test: make VMs use a dynamic MAC-IP mapping generated by tcpdump 2009-08-06 18:41 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish 2009-08-06 18:41 ` [KVM-AUTOTEST PATCH 03/12] KVM test: add sample 'qemu-ifup' script Michael Goldish @ 2009-08-06 18:41 ` Michael Goldish 2009-08-11 14:43 ` Jason Wang 1 sibling, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-06 18:41 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish In VM.get_address(), return an IP address from the MAC-IP cache if an IP address base wasn't specified by the user, or if the user requested that IP addresses be taken from the dynamic cache. To force the system to obtain IP addresses from the dynamic cache even when static base addresses are provided by the user, specify 'always_use_tcpdump = yes'. Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/kvm_address_pools.cfg.sample | 8 +++ client/tests/kvm/kvm_preprocessing.py | 3 +- client/tests/kvm/kvm_utils.py | 28 +++++++++++- client/tests/kvm/kvm_vm.py | 62 +++++++++++++++++------- 4 files changed, 81 insertions(+), 20 deletions(-) diff --git a/client/tests/kvm/kvm_address_pools.cfg.sample b/client/tests/kvm/kvm_address_pools.cfg.sample index 8a27ee1..debfe56 100644 --- a/client/tests/kvm/kvm_address_pools.cfg.sample +++ b/client/tests/kvm/kvm_address_pools.cfg.sample @@ -6,6 +6,14 @@ # If you wish to use a static MAC-IP mapping, where each MAC address range is # mapped to a known corresponding IP address range, specify the bases of the IP # address ranges in this file. +# If you specify a MAC address range without a corresponding IP address range, +# the IP addresses for that range will be determined at runtime by listening +# to DHCP traffic using tcpdump. +# If you wish to determine IP addresses using tcpdump in any case, regardless +# of any # IP addresses specified in this file, uncomment the following line: +#always_use_tcpdump = yes +# You may also specify this parameter for specific hosts by adding it in the +# appropriate sections below. variants: # Rename host1 to an actual (short) hostname in the network that will be running the Autotest client diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py index 04bdb59..a5a32dc 100644 --- a/client/tests/kvm/kvm_preprocessing.py +++ b/client/tests/kvm/kvm_preprocessing.py @@ -54,7 +54,8 @@ def preprocess_vm(test, params, env, name): logging.debug("VM object found in environment") else: logging.debug("VM object does not exist; creating it") - vm = kvm_vm.VM(name, params, qemu_path, image_dir, iso_dir, script_dir) + vm = kvm_vm.VM(name, params, qemu_path, image_dir, iso_dir, script_dir, + env.get("address_cache")) kvm_utils.env_register_vm(env, name, vm) start_vm = False diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py index ef257bc..17c2b73 100644 --- a/client/tests/kvm/kvm_utils.py +++ b/client/tests/kvm/kvm_utils.py @@ -1,5 +1,5 @@ import md5, thread, subprocess, time, string, random, socket, os, signal, pty -import select, re, logging +import select, re, logging, commands from autotest_lib.client.bin import utils from autotest_lib.client.common_lib import error import kvm_subprocess @@ -152,6 +152,32 @@ def get_mac_ip_pair_from_dict(dict): return (None, None) +def verify_mac_ip_pair(mac, ip, timeout=3.0): + """ + Connect to a given IP address and make sure its MAC address equals the + given MAC address. + + @param mac: A MAC address. + @param ip: An IP address. + @return: True iff ip is assigned to mac. + """ + s = socket.socket() + s.setblocking(False) + try: + s.connect((ip, 55555)) + except socket.error: + pass + end_time = time.time() + timeout + while time.time() < end_time: + o = commands.getoutput("/sbin/arp -n") + if re.search(r"\b%s\b.*\b%s\b" % (ip, mac), o, re.IGNORECASE): + s.close() + return True + time.sleep(0.1) + s.close() + return False + + # Functions for working with the environment (a dict-like object) def is_vm(obj): diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py index 0c35e64..6addf77 100644 --- a/client/tests/kvm/kvm_vm.py +++ b/client/tests/kvm/kvm_vm.py @@ -1,5 +1,5 @@ #!/usr/bin/python -import time, socket, os, logging, fcntl +import time, socket, os, logging, fcntl, re, commands import kvm_utils, kvm_subprocess """ @@ -101,7 +101,7 @@ class VM: """ def __init__(self, name, params, qemu_path, image_dir, iso_dir, - script_dir): + script_dir, address_cache): """ Initialize the object and set a few attributes. @@ -112,6 +112,7 @@ class VM: @param image_dir: The directory where images reside @param iso_dir: The directory where ISOs reside @param script_dir: The directory where -net tap scripts reside + @param address_cache: A dict that maps MAC addresses to IP addresses """ self.process = None self.redirs = {} @@ -124,6 +125,7 @@ class VM: self.image_dir = image_dir self.iso_dir = iso_dir self.script_dir = script_dir + self.address_cache = address_cache # Find available monitor filename while True: @@ -137,7 +139,7 @@ class VM: def clone(self, name=None, params=None, qemu_path=None, image_dir=None, - iso_dir=None, script_dir=None): + iso_dir=None, script_dir=None, address_cache=None): """ Return a clone of the VM object with optionally modified parameters. The clone is initially not alive and needs to be started using create(). @@ -150,6 +152,7 @@ class VM: @param image_dir: Optional new image dir @param iso_dir: Optional new iso directory @param script_dir: Optional new -net tap script directory + @param address_cache: A dict that maps MAC addresses to IP addresses """ if name == None: name = self.name @@ -163,7 +166,10 @@ class VM: iso_dir = self.iso_dir if script_dir == None: script_dir = self.script_dir - return VM(name, params, qemu_path, image_dir, iso_dir, script_dir) + if address_cache == None: + address_cache = self.address_cache + return VM(name, params, qemu_path, image_dir, iso_dir, script_dir, + address_cache) def make_qemu_command(self, name=None, params=None, qemu_path=None, @@ -625,21 +631,36 @@ class VM: nic_params = kvm_utils.get_sub_dict(self.params, nic_name) if nic_params.get("nic_mode") == "tap": mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params) + if not mac: + logging.debug("MAC address unavailable") + return None + if not ip or nic_params.get("always_use_tcpdump") == "yes": + # Get the IP address from the cache + ip = self.address_cache.get(mac) + if not ip: + logging.debug("Could not find IP address for MAC address: " + "%s" % mac) + return None + # Make sure the IP address is assigned to the right MAC address + if not kvm_utils.verify_mac_ip_pair(mac, ip): + logging.debug("Could not verify MAC-IP address pair: " + "%s ---> %s" % (mac, ip)) + return None return ip else: return "localhost" - def get_port(self, port, index=0): + def get_port(self, port, nic_index=0): """ Return the port in host space corresponding to port in guest space. @param port: Port number in host space. - @param index: Index of the NIC. + @param nic_index: Index of the NIC. @return: If port redirection is used, return the host port redirected to guest port port. Otherwise return port. """ - nic_name = kvm_utils.get_sub_dict_names(self.params, "nics")[index] + nic_name = kvm_utils.get_sub_dict_names(self.params, "nics")[nic_index] nic_params = kvm_utils.get_sub_dict(self.params, nic_name) if nic_params.get("nic_mode") == "tap": return port @@ -666,17 +687,19 @@ class VM: """ address = self.get_address() port = self.get_port(int(self.params.get("ssh_port"))) - if not port: + if not address or not port: + logging.debug("IP address or port unavailable") return False return kvm_utils.is_sshd_running(address, port, timeout=timeout) - def ssh_login(self, timeout=10): + def ssh_login(self, nic_index=0, timeout=10): """ Log into the guest via SSH/Telnet. If timeout expires while waiting for output from the guest (e.g. a password prompt or a shell prompt) -- fail. + @param nic_index: The index of the NIC to connect to. @param timeout: Time (seconds) before giving up logging into the guest. @return: kvm_spawn object on success and None on failure. @@ -685,9 +708,10 @@ class VM: password = self.params.get("password", "") prompt = self.params.get("ssh_prompt", "[\#\$]") use_telnet = self.params.get("use_telnet") == "yes" - address = self.get_address() + address = self.get_address(nic_index) port = self.get_port(int(self.params.get("ssh_port"))) - if not port: + if not address or not port: + logging.debug("IP address or port unavailable") return None if use_telnet: @@ -702,7 +726,7 @@ class VM: return session - def scp_to_remote(self, local_path, remote_path, timeout=300): + def scp_to_remote(self, local_path, remote_path, nic_index=0, timeout=300): """ Transfer files to the guest via SCP. @@ -713,15 +737,16 @@ class VM: """ username = self.params.get("username", "") password = self.params.get("password", "") - address = self.get_address() + address = self.get_address(nic_index) port = self.get_port(int(self.params.get("ssh_port"))) - if not port: + if not address or not port: + logging.debug("IP address or port unavailable") return None return kvm_utils.scp_to_remote(address, port, username, password, local_path, remote_path, timeout) - def scp_from_remote(self, remote_path, local_path, timeout=300): + def scp_from_remote(self, remote_path, local_path, nic_index=0, timeout=300): """ Transfer files from the guest via SCP. @@ -732,9 +757,10 @@ class VM: """ username = self.params.get("username", "") password = self.params.get("password", "") - address = self.get_address() + address = self.get_address(nic_index) port = self.get_port(int(self.params.get("ssh_port"))) - if not port: + if not address or not port: + logging.debug("IP address or port unavailable") return None return kvm_utils.scp_from_remote(address, port, username, password, remote_path, local_path, timeout) @@ -749,7 +775,7 @@ class VM: @return: A tuple (status, output). status is 0 on success and 1 on failure. """ - session = self.ssh_login(timeout) + session = self.ssh_login(timeout=timeout) if not session: return (1, "") -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [KVM-AUTOTEST PATCH 11/12] KVM test: make VMs use a dynamic MAC-IP mapping generated by tcpdump 2009-08-06 18:41 ` [KVM-AUTOTEST PATCH 11/12] KVM test: make VMs use a dynamic MAC-IP mapping generated by tcpdump Michael Goldish @ 2009-08-11 14:43 ` Jason Wang 0 siblings, 0 replies; 7+ messages in thread From: Jason Wang @ 2009-08-11 14:43 UTC (permalink / raw) To: Michael Goldish; +Cc: autotest, kvm Hello Michael: Based on our experiences with TAP networking scripts, the verify_mac_ip_pair() could not work well under multiple nics when testing linux guests. The main problem is the policy of multiple interfaces responding in linux, we should take care of the value of net.ipv4.conf.all.arp_ignore. Its default value(0) may cause the host get a mac address that does not configured on the target interface of guest. Maybe we could run a simple 'setup' case for all linux guests to set the value of arp_ignore to 1. What's your opinion? > In VM.get_address(), return an IP address from the MAC-IP cache if an IP > address base wasn't specified by the user, or if the user requested that IP > addresses be taken from the dynamic cache. > > To force the system to obtain IP addresses from the dynamic cache even when > static base addresses are provided by the user, specify > 'always_use_tcpdump = yes'. > > Signed-off-by: Michael Goldish <mgoldish@redhat.com> > --- > client/tests/kvm/kvm_address_pools.cfg.sample | 8 +++ > client/tests/kvm/kvm_preprocessing.py | 3 +- > client/tests/kvm/kvm_utils.py | 28 +++++++++++- > client/tests/kvm/kvm_vm.py | 62 +++++++++++++++++------- > 4 files changed, 81 insertions(+), 20 deletions(-) > > diff --git a/client/tests/kvm/kvm_address_pools.cfg.sample b/client/tests/kvm/kvm_address_pools.cfg.sample > index 8a27ee1..debfe56 100644 > --- a/client/tests/kvm/kvm_address_pools.cfg.sample > +++ b/client/tests/kvm/kvm_address_pools.cfg.sample > @@ -6,6 +6,14 @@ > # If you wish to use a static MAC-IP mapping, where each MAC address range is > # mapped to a known corresponding IP address range, specify the bases of the IP > # address ranges in this file. > +# If you specify a MAC address range without a corresponding IP address range, > +# the IP addresses for that range will be determined at runtime by listening > +# to DHCP traffic using tcpdump. > +# If you wish to determine IP addresses using tcpdump in any case, regardless > +# of any # IP addresses specified in this file, uncomment the following line: > +#always_use_tcpdump = yes > +# You may also specify this parameter for specific hosts by adding it in the > +# appropriate sections below. > > variants: > # Rename host1 to an actual (short) hostname in the network that will be running the Autotest client > diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py > index 04bdb59..a5a32dc 100644 > --- a/client/tests/kvm/kvm_preprocessing.py > +++ b/client/tests/kvm/kvm_preprocessing.py > @@ -54,7 +54,8 @@ def preprocess_vm(test, params, env, name): > logging.debug("VM object found in environment") > else: > logging.debug("VM object does not exist; creating it") > - vm = kvm_vm.VM(name, params, qemu_path, image_dir, iso_dir, script_dir) > + vm = kvm_vm.VM(name, params, qemu_path, image_dir, iso_dir, script_dir, > + env.get("address_cache")) > kvm_utils.env_register_vm(env, name, vm) > > start_vm = False > diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py > index ef257bc..17c2b73 100644 > --- a/client/tests/kvm/kvm_utils.py > +++ b/client/tests/kvm/kvm_utils.py > @@ -1,5 +1,5 @@ > import md5, thread, subprocess, time, string, random, socket, os, signal, pty > -import select, re, logging > +import select, re, logging, commands > from autotest_lib.client.bin import utils > from autotest_lib.client.common_lib import error > import kvm_subprocess > @@ -152,6 +152,32 @@ def get_mac_ip_pair_from_dict(dict): > return (None, None) > > > +def verify_mac_ip_pair(mac, ip, timeout=3.0): > + """ > + Connect to a given IP address and make sure its MAC address equals the > + given MAC address. > + > + @param mac: A MAC address. > + @param ip: An IP address. > + @return: True iff ip is assigned to mac. > + """ > + s = socket.socket() > + s.setblocking(False) > + try: > + s.connect((ip, 55555)) > + except socket.error: > + pass > + end_time = time.time() + timeout > + while time.time() < end_time: > + o = commands.getoutput("/sbin/arp -n") > + if re.search(r"\b%s\b.*\b%s\b" % (ip, mac), o, re.IGNORECASE): > + s.close() > + return True > + time.sleep(0.1) > + s.close() > + return False > + > + > # Functions for working with the environment (a dict-like object) > > def is_vm(obj): > diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py > index 0c35e64..6addf77 100644 > --- a/client/tests/kvm/kvm_vm.py > +++ b/client/tests/kvm/kvm_vm.py > @@ -1,5 +1,5 @@ > #!/usr/bin/python > -import time, socket, os, logging, fcntl > +import time, socket, os, logging, fcntl, re, commands > import kvm_utils, kvm_subprocess > > """ > @@ -101,7 +101,7 @@ class VM: > """ > > def __init__(self, name, params, qemu_path, image_dir, iso_dir, > - script_dir): > + script_dir, address_cache): > """ > Initialize the object and set a few attributes. > > @@ -112,6 +112,7 @@ class VM: > @param image_dir: The directory where images reside > @param iso_dir: The directory where ISOs reside > @param script_dir: The directory where -net tap scripts reside > + @param address_cache: A dict that maps MAC addresses to IP addresses > """ > self.process = None > self.redirs = {} > @@ -124,6 +125,7 @@ class VM: > self.image_dir = image_dir > self.iso_dir = iso_dir > self.script_dir = script_dir > + self.address_cache = address_cache > > # Find available monitor filename > while True: > @@ -137,7 +139,7 @@ class VM: > > > def clone(self, name=None, params=None, qemu_path=None, image_dir=None, > - iso_dir=None, script_dir=None): > + iso_dir=None, script_dir=None, address_cache=None): > """ > Return a clone of the VM object with optionally modified parameters. > The clone is initially not alive and needs to be started using create(). > @@ -150,6 +152,7 @@ class VM: > @param image_dir: Optional new image dir > @param iso_dir: Optional new iso directory > @param script_dir: Optional new -net tap script directory > + @param address_cache: A dict that maps MAC addresses to IP addresses > """ > if name == None: > name = self.name > @@ -163,7 +166,10 @@ class VM: > iso_dir = self.iso_dir > if script_dir == None: > script_dir = self.script_dir > - return VM(name, params, qemu_path, image_dir, iso_dir, script_dir) > + if address_cache == None: > + address_cache = self.address_cache > + return VM(name, params, qemu_path, image_dir, iso_dir, script_dir, > + address_cache) > > > def make_qemu_command(self, name=None, params=None, qemu_path=None, > @@ -625,21 +631,36 @@ class VM: > nic_params = kvm_utils.get_sub_dict(self.params, nic_name) > if nic_params.get("nic_mode") == "tap": > mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params) > + if not mac: > + logging.debug("MAC address unavailable") > + return None > + if not ip or nic_params.get("always_use_tcpdump") == "yes": > + # Get the IP address from the cache > + ip = self.address_cache.get(mac) > + if not ip: > + logging.debug("Could not find IP address for MAC address: " > + "%s" % mac) > + return None > + # Make sure the IP address is assigned to the right MAC address > + if not kvm_utils.verify_mac_ip_pair(mac, ip): > + logging.debug("Could not verify MAC-IP address pair: " > + "%s ---> %s" % (mac, ip)) > + return None > return ip > else: > return "localhost" > > > - def get_port(self, port, index=0): > + def get_port(self, port, nic_index=0): > """ > Return the port in host space corresponding to port in guest space. > > @param port: Port number in host space. > - @param index: Index of the NIC. > + @param nic_index: Index of the NIC. > @return: If port redirection is used, return the host port redirected > to guest port port. Otherwise return port. > """ > - nic_name = kvm_utils.get_sub_dict_names(self.params, "nics")[index] > + nic_name = kvm_utils.get_sub_dict_names(self.params, "nics")[nic_index] > nic_params = kvm_utils.get_sub_dict(self.params, nic_name) > if nic_params.get("nic_mode") == "tap": > return port > @@ -666,17 +687,19 @@ class VM: > """ > address = self.get_address() > port = self.get_port(int(self.params.get("ssh_port"))) > - if not port: > + if not address or not port: > + logging.debug("IP address or port unavailable") > return False > return kvm_utils.is_sshd_running(address, port, timeout=timeout) > > > - def ssh_login(self, timeout=10): > + def ssh_login(self, nic_index=0, timeout=10): > """ > Log into the guest via SSH/Telnet. > If timeout expires while waiting for output from the guest (e.g. a > password prompt or a shell prompt) -- fail. > > + @param nic_index: The index of the NIC to connect to. > @param timeout: Time (seconds) before giving up logging into the > guest. > @return: kvm_spawn object on success and None on failure. > @@ -685,9 +708,10 @@ class VM: > password = self.params.get("password", "") > prompt = self.params.get("ssh_prompt", "[\#\$]") > use_telnet = self.params.get("use_telnet") == "yes" > - address = self.get_address() > + address = self.get_address(nic_index) > port = self.get_port(int(self.params.get("ssh_port"))) > - if not port: > + if not address or not port: > + logging.debug("IP address or port unavailable") > return None > > if use_telnet: > @@ -702,7 +726,7 @@ class VM: > return session > > > - def scp_to_remote(self, local_path, remote_path, timeout=300): > + def scp_to_remote(self, local_path, remote_path, nic_index=0, timeout=300): > """ > Transfer files to the guest via SCP. > > @@ -713,15 +737,16 @@ class VM: > """ > username = self.params.get("username", "") > password = self.params.get("password", "") > - address = self.get_address() > + address = self.get_address(nic_index) > port = self.get_port(int(self.params.get("ssh_port"))) > - if not port: > + if not address or not port: > + logging.debug("IP address or port unavailable") > return None > return kvm_utils.scp_to_remote(address, port, username, password, > local_path, remote_path, timeout) > > > - def scp_from_remote(self, remote_path, local_path, timeout=300): > + def scp_from_remote(self, remote_path, local_path, nic_index=0, timeout=300): > """ > Transfer files from the guest via SCP. > > @@ -732,9 +757,10 @@ class VM: > """ > username = self.params.get("username", "") > password = self.params.get("password", "") > - address = self.get_address() > + address = self.get_address(nic_index) > port = self.get_port(int(self.params.get("ssh_port"))) > - if not port: > + if not address or not port: > + logging.debug("IP address or port unavailable") > return None > return kvm_utils.scp_from_remote(address, port, username, password, > remote_path, local_path, timeout) > @@ -749,7 +775,7 @@ class VM: > @return: A tuple (status, output). status is 0 on success and 1 on > failure. > """ > - session = self.ssh_login(timeout) > + session = self.ssh_login(timeout=timeout) > if not session: > return (1, "") > > ^ permalink raw reply [flat|nested] 7+ messages in thread
[parent not found: <1960092437.1769331250006819543.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com>]
* Re: [KVM-AUTOTEST PATCH 11/12] KVM test: make VMs use a dynamic MAC-IP mapping generated by tcpdump [not found] <1960092437.1769331250006819543.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com> @ 2009-08-11 16:16 ` Michael Goldish 0 siblings, 0 replies; 7+ messages in thread From: Michael Goldish @ 2009-08-11 16:16 UTC (permalink / raw) To: Jason Wang; +Cc: autotest, kvm ----- "Jason Wang" <jasowang@redhat.com> wrote: > Hello Michael: > Based on our experiences with TAP networking scripts, the > verify_mac_ip_pair() could not work well under multiple nics when > testing linux guests. The main problem is the policy of multiple > interfaces responding in linux, we should take care of the value of > net.ipv4.conf.all.arp_ignore. Its default value(0) may cause the host > get a mac address that does not configured on the target interface of > guest. > Maybe we could run a simple 'setup' case for all linux guests to set > the value of arp_ignore to 1. > What's your opinion? Thanks for pointing this out. You're right, there seems to be a problem with Linux guests (I tried RHEL 3.9). I started RHEL 3.9 with two NICs. At boot time the guest got 2 IP addresses and tcpdump collected them properly, but /sbin/arp -n listed the addresses as though they were both assigned to the MAC address of the first NIC. It's possible that regardless of the NIC I try to contact, the reply always comes from eth0, so it seems that both IP addresses have the same MAC address. I don't think we should solve this by messing with the guest, because different guests may require different hackish solutions. I think it's better to fix the test framework instead. The solution you suggested may not work for all Linux guests -- for example, on RHEL 3.9 I couldn't find net.ipv4.conf.all.arp_ignore, or any arp_ignore. (I did find arp_filter and arp_announce though, and none of them helped solve the problem.) I'm considering 2 solutions: 1. Do not require the MAC address reported by 'arp -n' to be assigned to the NIC we are 'verifiying' -- require it to be assigned to any NIC of the guest. This will probably work in all cases. 2. Stop using verify_mac_ip_pair() altogether. Instead, when destroying a VM, remove the MAC addresses of its NICs from the address cache. Also, when starting a new test, check each address in the cache and make sure it belongs to some living VM, and otherwise remove it. This should work in most cases, but _theoretically_ a guest might release its IP address during a test, and then the DHCP server might assign the same IP address to some other machine, so when we try talking to our guest using that IP address, we'll end up talking to the other machine. That's exactly the sort of situation verify_mac_ip_pair() was meant to prevent, but this situation sounds so unlikely to occur, that it might be safe to use this proposed solution. What do you think? > > In VM.get_address(), return an IP address from the MAC-IP cache if > an IP > > address base wasn't specified by the user, or if the user requested > that IP > > addresses be taken from the dynamic cache. > > > > To force the system to obtain IP addresses from the dynamic cache > even when > > static base addresses are provided by the user, specify > > 'always_use_tcpdump = yes'. > > > > Signed-off-by: Michael Goldish <mgoldish@redhat.com> > > --- > > client/tests/kvm/kvm_address_pools.cfg.sample | 8 +++ > > client/tests/kvm/kvm_preprocessing.py | 3 +- > > client/tests/kvm/kvm_utils.py | 28 +++++++++++- > > client/tests/kvm/kvm_vm.py | 62 > +++++++++++++++++------- > > 4 files changed, 81 insertions(+), 20 deletions(-) > > > > diff --git a/client/tests/kvm/kvm_address_pools.cfg.sample > b/client/tests/kvm/kvm_address_pools.cfg.sample > > index 8a27ee1..debfe56 100644 > > --- a/client/tests/kvm/kvm_address_pools.cfg.sample > > +++ b/client/tests/kvm/kvm_address_pools.cfg.sample > > @@ -6,6 +6,14 @@ > > # If you wish to use a static MAC-IP mapping, where each MAC > address range is > > # mapped to a known corresponding IP address range, specify the > bases of the IP > > # address ranges in this file. > > +# If you specify a MAC address range without a corresponding IP > address range, > > +# the IP addresses for that range will be determined at runtime by > listening > > +# to DHCP traffic using tcpdump. > > +# If you wish to determine IP addresses using tcpdump in any case, > regardless > > +# of any # IP addresses specified in this file, uncomment the > following line: > > +#always_use_tcpdump = yes > > +# You may also specify this parameter for specific hosts by adding > it in the > > +# appropriate sections below. > > > > variants: > > # Rename host1 to an actual (short) hostname in the network > that will be running the Autotest client > > diff --git a/client/tests/kvm/kvm_preprocessing.py > b/client/tests/kvm/kvm_preprocessing.py > > index 04bdb59..a5a32dc 100644 > > --- a/client/tests/kvm/kvm_preprocessing.py > > +++ b/client/tests/kvm/kvm_preprocessing.py > > @@ -54,7 +54,8 @@ def preprocess_vm(test, params, env, name): > > logging.debug("VM object found in environment") > > else: > > logging.debug("VM object does not exist; creating it") > > - vm = kvm_vm.VM(name, params, qemu_path, image_dir, iso_dir, > script_dir) > > + vm = kvm_vm.VM(name, params, qemu_path, image_dir, iso_dir, > script_dir, > > + env.get("address_cache")) > > kvm_utils.env_register_vm(env, name, vm) > > > > start_vm = False > > diff --git a/client/tests/kvm/kvm_utils.py > b/client/tests/kvm/kvm_utils.py > > index ef257bc..17c2b73 100644 > > --- a/client/tests/kvm/kvm_utils.py > > +++ b/client/tests/kvm/kvm_utils.py > > @@ -1,5 +1,5 @@ > > import md5, thread, subprocess, time, string, random, socket, os, > signal, pty > > -import select, re, logging > > +import select, re, logging, commands > > from autotest_lib.client.bin import utils > > from autotest_lib.client.common_lib import error > > import kvm_subprocess > > @@ -152,6 +152,32 @@ def get_mac_ip_pair_from_dict(dict): > > return (None, None) > > > > > > +def verify_mac_ip_pair(mac, ip, timeout=3.0): > > + """ > > + Connect to a given IP address and make sure its MAC address > equals the > > + given MAC address. > > + > > + @param mac: A MAC address. > > + @param ip: An IP address. > > + @return: True iff ip is assigned to mac. > > + """ > > + s = socket.socket() > > + s.setblocking(False) > > + try: > > + s.connect((ip, 55555)) > > + except socket.error: > > + pass > > + end_time = time.time() + timeout > > + while time.time() < end_time: > > + o = commands.getoutput("/sbin/arp -n") > > + if re.search(r"\b%s\b.*\b%s\b" % (ip, mac), o, > re.IGNORECASE): > > + s.close() > > + return True > > + time.sleep(0.1) > > + s.close() > > + return False > > + > > + > > # Functions for working with the environment (a dict-like object) > > > > def is_vm(obj): > > diff --git a/client/tests/kvm/kvm_vm.py > b/client/tests/kvm/kvm_vm.py > > index 0c35e64..6addf77 100644 > > --- a/client/tests/kvm/kvm_vm.py > > +++ b/client/tests/kvm/kvm_vm.py > > @@ -1,5 +1,5 @@ > > #!/usr/bin/python > > -import time, socket, os, logging, fcntl > > +import time, socket, os, logging, fcntl, re, commands > > import kvm_utils, kvm_subprocess > > > > """ > > @@ -101,7 +101,7 @@ class VM: > > """ > > > > def __init__(self, name, params, qemu_path, image_dir, > iso_dir, > > - script_dir): > > + script_dir, address_cache): > > """ > > Initialize the object and set a few attributes. > > > > @@ -112,6 +112,7 @@ class VM: > > @param image_dir: The directory where images reside > > @param iso_dir: The directory where ISOs reside > > @param script_dir: The directory where -net tap scripts > reside > > + @param address_cache: A dict that maps MAC addresses to IP > addresses > > """ > > self.process = None > > self.redirs = {} > > @@ -124,6 +125,7 @@ class VM: > > self.image_dir = image_dir > > self.iso_dir = iso_dir > > self.script_dir = script_dir > > + self.address_cache = address_cache > > > > # Find available monitor filename > > while True: > > @@ -137,7 +139,7 @@ class VM: > > > > > > def clone(self, name=None, params=None, qemu_path=None, > image_dir=None, > > - iso_dir=None, script_dir=None): > > + iso_dir=None, script_dir=None, address_cache=None): > > """ > > Return a clone of the VM object with optionally modified > parameters. > > The clone is initially not alive and needs to be started > using create(). > > @@ -150,6 +152,7 @@ class VM: > > @param image_dir: Optional new image dir > > @param iso_dir: Optional new iso directory > > @param script_dir: Optional new -net tap script directory > > + @param address_cache: A dict that maps MAC addresses to IP > addresses > > """ > > if name == None: > > name = self.name > > @@ -163,7 +166,10 @@ class VM: > > iso_dir = self.iso_dir > > if script_dir == None: > > script_dir = self.script_dir > > - return VM(name, params, qemu_path, image_dir, iso_dir, > script_dir) > > + if address_cache == None: > > + address_cache = self.address_cache > > + return VM(name, params, qemu_path, image_dir, iso_dir, > script_dir, > > + address_cache) > > > > > > def make_qemu_command(self, name=None, params=None, > qemu_path=None, > > @@ -625,21 +631,36 @@ class VM: > > nic_params = kvm_utils.get_sub_dict(self.params, nic_name) > > if nic_params.get("nic_mode") == "tap": > > mac, ip = > kvm_utils.get_mac_ip_pair_from_dict(nic_params) > > + if not mac: > > + logging.debug("MAC address unavailable") > > + return None > > + if not ip or nic_params.get("always_use_tcpdump") == > "yes": > > + # Get the IP address from the cache > > + ip = self.address_cache.get(mac) > > + if not ip: > > + logging.debug("Could not find IP address for > MAC address: " > > + "%s" % mac) > > + return None > > + # Make sure the IP address is assigned to the right > MAC address > > + if not kvm_utils.verify_mac_ip_pair(mac, ip): > > + logging.debug("Could not verify MAC-IP address > pair: " > > + "%s ---> %s" % (mac, ip)) > > + return None > > return ip > > else: > > return "localhost" > > > > > > - def get_port(self, port, index=0): > > + def get_port(self, port, nic_index=0): > > """ > > Return the port in host space corresponding to port in > guest space. > > > > @param port: Port number in host space. > > - @param index: Index of the NIC. > > + @param nic_index: Index of the NIC. > > @return: If port redirection is used, return the host port > redirected > > to guest port port. Otherwise return port. > > """ > > - nic_name = kvm_utils.get_sub_dict_names(self.params, > "nics")[index] > > + nic_name = kvm_utils.get_sub_dict_names(self.params, > "nics")[nic_index] > > nic_params = kvm_utils.get_sub_dict(self.params, nic_name) > > if nic_params.get("nic_mode") == "tap": > > return port > > @@ -666,17 +687,19 @@ class VM: > > """ > > address = self.get_address() > > port = self.get_port(int(self.params.get("ssh_port"))) > > - if not port: > > + if not address or not port: > > + logging.debug("IP address or port unavailable") > > return False > > return kvm_utils.is_sshd_running(address, port, > timeout=timeout) > > > > > > - def ssh_login(self, timeout=10): > > + def ssh_login(self, nic_index=0, timeout=10): > > """ > > Log into the guest via SSH/Telnet. > > If timeout expires while waiting for output from the guest > (e.g. a > > password prompt or a shell prompt) -- fail. > > > > + @param nic_index: The index of the NIC to connect to. > > @param timeout: Time (seconds) before giving up logging > into the > > guest. > > @return: kvm_spawn object on success and None on failure. > > @@ -685,9 +708,10 @@ class VM: > > password = self.params.get("password", "") > > prompt = self.params.get("ssh_prompt", "[\#\$]") > > use_telnet = self.params.get("use_telnet") == "yes" > > - address = self.get_address() > > + address = self.get_address(nic_index) > > port = self.get_port(int(self.params.get("ssh_port"))) > > - if not port: > > + if not address or not port: > > + logging.debug("IP address or port unavailable") > > return None > > > > if use_telnet: > > @@ -702,7 +726,7 @@ class VM: > > return session > > > > > > - def scp_to_remote(self, local_path, remote_path, timeout=300): > > + def scp_to_remote(self, local_path, remote_path, nic_index=0, > timeout=300): > > """ > > Transfer files to the guest via SCP. > > > > @@ -713,15 +737,16 @@ class VM: > > """ > > username = self.params.get("username", "") > > password = self.params.get("password", "") > > - address = self.get_address() > > + address = self.get_address(nic_index) > > port = self.get_port(int(self.params.get("ssh_port"))) > > - if not port: > > + if not address or not port: > > + logging.debug("IP address or port unavailable") > > return None > > return kvm_utils.scp_to_remote(address, port, username, > password, > > local_path, remote_path, > timeout) > > > > > > - def scp_from_remote(self, remote_path, local_path, > timeout=300): > > + def scp_from_remote(self, remote_path, local_path, nic_index=0, > timeout=300): > > """ > > Transfer files from the guest via SCP. > > > > @@ -732,9 +757,10 @@ class VM: > > """ > > username = self.params.get("username", "") > > password = self.params.get("password", "") > > - address = self.get_address() > > + address = self.get_address(nic_index) > > port = self.get_port(int(self.params.get("ssh_port"))) > > - if not port: > > + if not address or not port: > > + logging.debug("IP address or port unavailable") > > return None > > return kvm_utils.scp_from_remote(address, port, username, > password, > > remote_path, local_path, > timeout) > > @@ -749,7 +775,7 @@ class VM: > > @return: A tuple (status, output). status is 0 on success > and 1 on > > failure. > > """ > > - session = self.ssh_login(timeout) > > + session = self.ssh_login(timeout=timeout) > > if not session: > > return (1, "") > > > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 0/12] TAP support (and a few other small things) @ 2009-08-02 23:58 Michael Goldish 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 02/12] KVM test: add basic TAP support with static MAC-IP mapping Michael Goldish 0 siblings, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-02 23:58 UTC (permalink / raw) To: autotest, kvm Here's my proposed TAP solution: - The user specifies MAC ranges and may or may not specify corresponding IP ranges. - VMs are always given MAC addresses from the user specified MAC ranges. - MAC address ranges must be unique to each host running KVM tests -- there must be no overlap between them. - If corresponding IP ranges are specified as well, the system will use them when trying to communicate with guests. - If corresponding IP ranges are not specified, tcpdump will be used to listen to DHCP traffic and dynamically detect the right IP addresses. There's also a flag that can force this behavior. - tcpdump will run in the background at all times, and collect MAC-IP pairs as soon as they are assigned by the DHCP server. This is useful for NIC hotplugging (where IP addresses are assigned during the test itself) and generally for misbehaving guests or DHCP servers (restarting network services during the test, using very short lease times). - It is up to the user to create the actual bridge for TAP devices; the qemu-ifup script included in one of the patches only adds the TAP interface to an existing bridge. The user should modify this script or create a bridge some other way. This works very well on two hosts I tried, but I'm not entirely sure it will work in all cases -- please let me know what you think. (I remember Jason Wang mentioned that tcpdump doesn't always catch all the required traffic, or that it might somehow depend on the DHCP server -- anyone willing to comment on that?) The following patches have been tested with a few KVM tests (boot, reboot, stress_boot, migration) with both TAP and user mode, but they could probably use more testing (maybe with Python 2.6?), so if anyone is willing to help with that I'd appreciate it very much. ^ permalink raw reply [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 02/12] KVM test: add basic TAP support with static MAC-IP mapping 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish @ 2009-08-02 23:58 ` Michael Goldish 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 03/12] KVM test: add sample 'qemu-ifup' script Michael Goldish 0 siblings, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-02 23:58 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish The network mode is specified by a NIC parameter named 'nic_mode'. Currently two modes are supported: 'tap' and 'user'. Other NIC parameters are now handled: nic_script, nic_downscript and nic_ifname. nic_ifname should usually not be specified (qemu chooses one automatically). nic_script and nic_downscript may specify a relative path, e.g. 'qemu-ifup', which refers to a file in the KVM test local dir, or an absolute path, e.g. '/etc/qemu-ifup'. This patch mainly modifies VM.get_address(), VM.get_port() and VM.make_qemu_command(), and adds a 'script_dir' parameter to several VM routines. 'script_dir' should normally be the KVM test local dir. Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/kvm_preprocessing.py | 8 ++- client/tests/kvm/kvm_tests.cfg.sample | 6 ++ client/tests/kvm/kvm_vm.py | 95 +++++++++++++++++++++++--------- 3 files changed, 79 insertions(+), 30 deletions(-) diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py index d118826..4d77630 100644 --- a/client/tests/kvm/kvm_preprocessing.py +++ b/client/tests/kvm/kvm_preprocessing.py @@ -46,6 +46,7 @@ def preprocess_vm(test, params, env, name): qemu_path = os.path.join(test.bindir, "qemu") image_dir = os.path.join(test.bindir, "images") iso_dir = os.path.join(test.bindir, "isos") + script_dir = test.bindir logging.debug("Preprocessing VM '%s'..." % name) vm = kvm_utils.env_get_vm(env, name) @@ -53,7 +54,7 @@ def preprocess_vm(test, params, env, name): logging.debug("VM object found in environment") else: logging.debug("VM object does not exist; creating it") - vm = kvm_vm.VM(name, params, qemu_path, image_dir, iso_dir) + vm = kvm_vm.VM(name, params, qemu_path, image_dir, iso_dir, script_dir) kvm_utils.env_register_vm(env, name, vm) start_vm = False @@ -74,14 +75,15 @@ def preprocess_vm(test, params, env, name): elif vm.make_qemu_command() != vm.make_qemu_command(name, params, qemu_path, image_dir, - iso_dir): + iso_dir, + script_dir): logging.debug("VM's qemu command differs from requested one; " "restarting it...") start_vm = True if start_vm: if not vm.create(name, params, qemu_path, image_dir, iso_dir, - for_migration): + script_dir, for_migration): message = "Could not start VM" logging.error(message) raise error.TestError(message) diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample index 7cd12cb..4b9d7eb 100644 --- a/client/tests/kvm/kvm_tests.cfg.sample +++ b/client/tests/kvm/kvm_tests.cfg.sample @@ -24,6 +24,12 @@ display = vnc redirs = ssh guest_port_ssh = 22 +# NIC parameters +nic_mode = tap +#nic_mode = user +nic_script = qemu-ifup +address_index = 0 + # Tests variants: diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py index eba9b84..0c35e64 100644 --- a/client/tests/kvm/kvm_vm.py +++ b/client/tests/kvm/kvm_vm.py @@ -100,7 +100,8 @@ class VM: This class handles all basic VM operations. """ - def __init__(self, name, params, qemu_path, image_dir, iso_dir): + def __init__(self, name, params, qemu_path, image_dir, iso_dir, + script_dir): """ Initialize the object and set a few attributes. @@ -110,6 +111,7 @@ class VM: @param qemu_path: The path of the qemu binary @param image_dir: The directory where images reside @param iso_dir: The directory where ISOs reside + @param script_dir: The directory where -net tap scripts reside """ self.process = None self.redirs = {} @@ -121,7 +123,7 @@ class VM: self.qemu_path = qemu_path self.image_dir = image_dir self.iso_dir = iso_dir - + self.script_dir = script_dir # Find available monitor filename while True: @@ -135,7 +137,7 @@ class VM: def clone(self, name=None, params=None, qemu_path=None, image_dir=None, - iso_dir=None): + iso_dir=None, script_dir=None): """ Return a clone of the VM object with optionally modified parameters. The clone is initially not alive and needs to be started using create(). @@ -147,6 +149,7 @@ class VM: @param qemu_path: Optional new path to qemu @param image_dir: Optional new image dir @param iso_dir: Optional new iso directory + @param script_dir: Optional new -net tap script directory """ if name == None: name = self.name @@ -158,23 +161,24 @@ class VM: image_dir = self.image_dir if iso_dir == None: iso_dir = self.iso_dir - return VM(name, params, qemu_path, image_dir, iso_dir) + if script_dir == None: + script_dir = self.script_dir + return VM(name, params, qemu_path, image_dir, iso_dir, script_dir) def make_qemu_command(self, name=None, params=None, qemu_path=None, - image_dir=None, iso_dir=None): + image_dir=None, iso_dir=None, script_dir=None): """ Generate a qemu command line. All parameters are optional. If a parameter is not supplied, the corresponding value stored in the class attributes is used. - @param name: The name of the object @param params: A dict containing VM params @param qemu_path: The path of the qemu binary @param image_dir: The directory where images reside @param iso_dir: The directory where ISOs reside - + @param script_dir: The directory where -net tap scripts reside @note: The params dict should contain: mem -- memory size in MBs @@ -210,6 +214,8 @@ class VM: image_dir = self.image_dir if iso_dir == None: iso_dir = self.iso_dir + if script_dir == None: + script_dir = self.script_dir # Start constructing the qemu command qemu_cmd = "" @@ -241,10 +247,30 @@ class VM: vlan = 0 for nic_name in kvm_utils.get_sub_dict_names(params, "nics"): nic_params = kvm_utils.get_sub_dict(params, nic_name) + # Handle the '-net nic' part qemu_cmd += " -net nic,vlan=%d" % vlan if nic_params.get("nic_model"): qemu_cmd += ",model=%s" % nic_params.get("nic_model") - qemu_cmd += " -net user,vlan=%d" % vlan + if nic_params.has_key("address_index"): + mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params) + qemu_cmd += ",macaddr=%s" % mac + # Handle the '-net tap' or '-net user' part + mode = nic_params.get("nic_mode", "user") + qemu_cmd += " -net %s,vlan=%d" % (mode, vlan) + if mode == "tap": + if nic_params.get("nic_ifname"): + qemu_cmd += ",ifname=%s" % nic_params.get("nic_ifname") + if nic_params.get("nic_script"): + script_path = nic_params.get("nic_script") + if not os.path.isabs(script_path): + script_path = os.path.join(script_dir, script_path) + qemu_cmd += ",script=%s" % script_path + if nic_params.get("nic_downscript"): + script_path = nic_params.get("nic_downscript") + if not os.path.isabs(script_path): + script_path = os.path.join(script_dir, script_path) + qemu_cmd += ",downscript=%s" % script_path + # Proceed to next NIC vlan += 1 mem = params.get("mem") @@ -263,7 +289,7 @@ class VM: for redir_name in kvm_utils.get_sub_dict_names(params, "redirs"): redir_params = kvm_utils.get_sub_dict(params, redir_name) guest_port = int(redir_params.get("guest_port")) - host_port = self.get_port(guest_port) + host_port = self.redirs.get(guest_port) qemu_cmd += " -redir tcp:%s::%s" % (host_port, guest_port) if params.get("display") == "vnc": @@ -282,7 +308,8 @@ class VM: def create(self, name=None, params=None, qemu_path=None, image_dir=None, - iso_dir=None, for_migration=False, timeout=5.0): + iso_dir=None, script_dir=None, for_migration=False, + timeout=5.0): """ Start the VM by running a qemu command. All parameters are optional. The following applies to all parameters @@ -295,6 +322,7 @@ class VM: @param qemu_path: The path of the qemu binary @param image_dir: The directory where images reside @param iso_dir: The directory where ISOs reside + @param script_dir: The directory where -net tap scripts reside @param for_migration: If True, start the VM with the -incoming option """ @@ -310,11 +338,14 @@ class VM: self.image_dir = image_dir if iso_dir != None: self.iso_dir = iso_dir + if script_dir != None: + self.script_dir = script_dir name = self.name params = self.params qemu_path = self.qemu_path image_dir = self.image_dir iso_dir = self.iso_dir + script_dir = self.script_dir # Verify the md5sum of the ISO image iso = params.get("cdrom") @@ -325,14 +356,14 @@ class VM: return False compare = False if params.get("md5sum_1m"): - logging.debug("Comparing expected MD5 sum with MD5 sum of first" - "MB of ISO file...") + logging.debug("Comparing expected MD5 sum with MD5 sum of " + "first MB of ISO file...") actual_md5sum = kvm_utils.md5sum_file(iso, 1048576) expected_md5sum = params.get("md5sum_1m") compare = True elif params.get("md5sum"): - logging.debug("Comparing expected MD5 sum with MD5 sum of ISO" - " file...") + logging.debug("Comparing expected MD5 sum with MD5 sum of ISO " + "file...") actual_md5sum = kvm_utils.md5sum_file(iso) expected_md5sum = params.get("md5sum") compare = True @@ -581,32 +612,42 @@ class VM: return self.params - def get_address(self): + def get_address(self, index=0): """ - Return the guest's address in host space. + Return the address of a NIC of the guest, in host space. - If port redirection is used, return 'localhost' (the guest has no IP - address of its own). Otherwise return the guest's IP address. + If port redirection is used, return 'localhost' (the NIC has no IP + address of its own). Otherwise return the NIC's IP address. + + @param index: Index of the NIC whose address is requested. """ - # Currently redirection is always used, so return 'localhost' - return "localhost" + nic_name = kvm_utils.get_sub_dict_names(self.params, "nics")[index] + nic_params = kvm_utils.get_sub_dict(self.params, nic_name) + if nic_params.get("nic_mode") == "tap": + mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params) + return ip + else: + return "localhost" - def get_port(self, port): + def get_port(self, port, index=0): """ Return the port in host space corresponding to port in guest space. @param port: Port number in host space. + @param index: Index of the NIC. @return: If port redirection is used, return the host port redirected to guest port port. Otherwise return port. """ - # Currently redirection is always used, so use the redirs dict - if self.redirs.has_key(port): - return self.redirs[port] + nic_name = kvm_utils.get_sub_dict_names(self.params, "nics")[index] + nic_params = kvm_utils.get_sub_dict(self.params, nic_name) + if nic_params.get("nic_mode") == "tap": + return port else: - logging.debug("Warning: guest port %s requested but not" - " redirected" % port) - return None + if not self.redirs.has_key(port): + logging.warn("Warning: guest port %s requested but not " + "redirected" % port) + return self.redirs.get(port) def get_pid(self): -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 03/12] KVM test: add sample 'qemu-ifup' script 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish @ 2009-08-02 23:58 ` Michael Goldish 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 04/12] KVM test: add host-specific MAC-IP pool configuration Michael Goldish 0 siblings, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-02 23:58 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish The script adds a requested interface to an existing bridge. It is meant to be used by qemu when running in TAP mode. Note: the user is responsible for setting up the bridge before running any tests. This can be done with brctl or in any manner that is appropriate for the host OS. It can be done inside 'qemu-ifup' as well, but this sample script doesn't do it. Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/qemu-ifup | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) create mode 100644 client/tests/kvm/qemu-ifup diff --git a/client/tests/kvm/qemu-ifup b/client/tests/kvm/qemu-ifup new file mode 100644 index 0000000..bcd9a7a --- /dev/null +++ b/client/tests/kvm/qemu-ifup @@ -0,0 +1,8 @@ +#!/bin/sh + +# The following expression selects the first bridge listed by 'brctl show'. +# Modify it to suit your needs. +switch=$(/usr/sbin/brctl show | awk 'NR==2 { print $1 }') + +/sbin/ifconfig $1 0.0.0.0 up +/usr/sbin/brctl addif ${switch} $1 -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 04/12] KVM test: add host-specific MAC-IP pool configuration 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish @ 2009-08-02 23:58 ` Michael Goldish 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 05/12] KVM test: specify installation cdkeys in a dedicated file Michael Goldish 0 siblings, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-02 23:58 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish - Add kvm_address_pools.cfg which specifies a set of MAC-IP ranges for each host. - Read kvm_address_pools.cfg in the control file and filter it by hostname, so that each host uses only the MAC addresses assigned to it. If the hostname doesn't match any hostname in kvm_address_pools.cfg, use 'default_host'. - While we're at it, add a comment informing the user where to modify the test configration, if that is desirable - (Also add a missing space somewhere in the control file, and remove a few extra ones) Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/control | 25 +++++++++-- client/tests/kvm/kvm_address_pools.cfg.sample | 56 +++++++++++++++++++++++++ 2 files changed, 76 insertions(+), 5 deletions(-) create mode 100644 client/tests/kvm/kvm_address_pools.cfg.sample diff --git a/client/tests/kvm/control b/client/tests/kvm/control index 5c5d424..2ad8d81 100644 --- a/client/tests/kvm/control +++ b/client/tests/kvm/control @@ -133,12 +133,27 @@ if not r: import kvm_config filename = os.path.join(pwd, "kvm_tests.cfg") -list = kvm_config.config(filename).get_list() +cfg = kvm_config.config(filename) + +# If desirable, make changes to the test configuration here. For example: +# cfg.parse_string("only fc8_quick") +# cfg.parse_string("display = sdl") + +filename = os.path.join(pwd, "kvm_address_pools.cfg") +if os.path.exists(filename): + cfg.parse_file(filename) + hostname = os.uname()[1].split(".")[0] + if cfg.filter("^" + hostname): + cfg.parse_string("only ^%s" % hostname) + else: + cfg.parse_string("only ^default_host") + +list = cfg.get_list() -# ------------- -# Run the tests -# ------------- +# ------------- +# Run the tests +# ------------- status_dict = {} for dict in list: @@ -153,7 +168,7 @@ for dict in list: dependencies_satisfied = False break if dependencies_satisfied: - test_iterations=int(dict.get("iterations", 1)) + test_iterations = int(dict.get("iterations", 1)) current_status = job.run_test("kvm", params=dict, tag=dict.get("shortname"), iterations=test_iterations) diff --git a/client/tests/kvm/kvm_address_pools.cfg.sample b/client/tests/kvm/kvm_address_pools.cfg.sample new file mode 100644 index 0000000..8a27ee1 --- /dev/null +++ b/client/tests/kvm/kvm_address_pools.cfg.sample @@ -0,0 +1,56 @@ +# Rename this file to kvm_address_pools.cfg. +# This file specifies several MAC-IP ranges for each host in the network that +# may run KVM tests. A MAC address must not be used twice, so these ranges +# must not overlap. The VMs running on each host will only use MAC addresses +# from the pool of that host. +# If you wish to use a static MAC-IP mapping, where each MAC address range is +# mapped to a known corresponding IP address range, specify the bases of the IP +# address ranges in this file. + +variants: + # Rename host1 to an actual (short) hostname in the network that will be running the Autotest client + - @host1: + # Add/remove ranges here + address_ranges = r1 r2 + + # Modify the following parameters to reflect the DHCP server's configuration + address_range_base_mac_r1 = 52:54:00:12:35:56 + #address_range_base_ip_r1 = 10.0.2.20 + address_range_size_r1 = 16 + + # Modify the following parameters to reflect the DHCP server's configuration + address_range_base_mac_r2 = 52:54:00:12:35:80 + #address_range_base_ip_r2 = 10.0.2.40 + address_range_size_r2 = 16 + + # Rename host2 to an actual (short) hostname in the network that will be running the Autotest client + - @host2: + # Add/remove ranges here + address_ranges = r1 r2 + + # Modify the following parameters to reflect the DHCP server's configuration + address_range_base_mac_r1 = 52:54:00:12:36:56 + #address_range_base_ip_r1 = 10.0.3.20 + address_range_size_r1 = 16 + + # Modify the following parameters to reflect the DHCP server's configuration + address_range_base_mac_r2 = 52:54:00:12:36:80 + #address_range_base_ip_r2 = 10.0.3.40 + address_range_size_r2 = 16 + + # Add additional hosts here... + + # This will be used for hosts that do not appear on the list + - @default_host: + # Add/remove ranges here + address_ranges = r1 r2 + + # Modify the following parameters to reflect the DHCP server's configuration + address_range_base_mac_r1 = 52:54:00:12:34:56 + #address_range_base_ip_r1 = 10.0.1.20 + address_range_size_r1 = 16 + + # Modify the following parameters to reflect the DHCP server's configuration + address_range_base_mac_r2 = 52:54:00:12:34:80 + #address_range_base_ip_r2 = 10.0.1.40 + address_range_size_r2 = 16 -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 05/12] KVM test: specify installation cdkeys in a dedicated file 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish @ 2009-08-02 23:58 ` Michael Goldish 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 06/12] KVM test: minor style changes to kvm_config.py Michael Goldish 0 siblings, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-02 23:58 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish - Add file kvm_cdkeys.cfg.sample which specifies cdkeys for guests that require them. Users should rename the file to kvm_cdkeys.cfg and enter the real cdkeys in the file. - Remove all cdkey lines from kvm_tests.cfg.sample and add an include statement instead ('include kvm_cdkeys.cfg'). Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/kvm_cdkeys.cfg.sample | 16 ++++++++++++++++ client/tests/kvm/kvm_tests.cfg.sample | 10 +++------- 2 files changed, 19 insertions(+), 7 deletions(-) create mode 100644 client/tests/kvm/kvm_cdkeys.cfg.sample diff --git a/client/tests/kvm/kvm_cdkeys.cfg.sample b/client/tests/kvm/kvm_cdkeys.cfg.sample new file mode 100644 index 0000000..60bfe59 --- /dev/null +++ b/client/tests/kvm/kvm_cdkeys.cfg.sample @@ -0,0 +1,16 @@ +# Rename this file to kvm_cdkeys.cfg. +# Replace the 'CDKEY' strings with real cdkeys where necessary. +# Guests that do not require cdkeys by default are commented out. +# You may uncomment them if necessary. + +RHEL.5.3.i386: cdkey = CDKEY +RHEL.5.3.x86_64: cdkey = CDKEY +Win2000: cdkey = CDKEY +WinXP.32: cdkey = CDKEY +WinXP.64: cdkey = CDKEY +Win2003.32: cdkey = CDKEY +Win2003.64: cdkey = CDKEY +#WinVista.32: cdkey = CDKEY +#WinVista.64: cdkey = CDKEY +#Win2008.32: cdkey = CDKEY +#Win2008.64: cdkey = CDKEY diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample index 4b9d7eb..3c23432 100644 --- a/client/tests/kvm/kvm_tests.cfg.sample +++ b/client/tests/kvm/kvm_tests.cfg.sample @@ -272,7 +272,6 @@ variants: image_name = rhel5-32 install: steps=RHEL-5.3-i386.steps - cdkey=RHEL_INSTALL_NUMBER cdrom=linux/RHEL-5.3-i386-DVD.iso md5sum=371c62851611fd32ead440df6f24a296 md5sum_1m=242318dd44152210f6ff6cdda1bfbf51 @@ -282,7 +281,6 @@ variants: image_name = rhel5-64 install: steps=RHEL-5.3-x86_64.steps - cdkey=RHEL_INSTALL_NUMBER cdrom=linux/RHEL-5.3-x86_64-DVD.iso md5sum=c5ed6b284410f4d8212cafc78fd7a8c5 md5sum_1m=b999f437583098ea5bbd56fb1de1d011 @@ -363,7 +361,6 @@ variants: md5sum = dda6039f3a9173f0f6bfae40f5efdfea md5sum_1m = dd28fba196d366d56fe774bd93df5527 user = user - cdkey = WIN_2000_32_CDKEY setup: steps = Win2000-32-setupssh.steps cdrom = windows/setupssh.iso @@ -376,7 +373,6 @@ variants: md5sum = 743450644b1d9fe97b3cf379e22dceb0 md5sum_1m = b473bf75af2d1269fec8958cf0202bfd user = user - cdkey = WIN_XP_32_CDKEY setup: steps = WinXP-32-setupssh.steps cdrom = windows/setupssh.iso @@ -390,7 +386,6 @@ variants: md5sum = 8d3f007ec9c2060cec8a50ee7d7dc512 md5sum_1m = e812363ff427effc512b7801ee70e513 user = user - cdkey = WIN_XP_64_CDKEY - Win2003: image_size = 20G @@ -406,7 +401,6 @@ variants: md5sum = 03e921e9b4214773c21a39f5c3f42ef7 md5sum_1m = 37c2fdec15ac4ec16aa10fdfdb338aa3 user = user - cdkey = WIN_2003_32_CDKEY setup: steps = Win2003-32-setupssh.steps cdrom = windows/setupssh.iso @@ -420,7 +414,6 @@ variants: md5sum = 5703f87c9fd77d28c05ffadd3354dbbd md5sum_1m = 439393c384116aa09e08a0ad047dcea8 user = user - cdkey = WIN_2003_64_CDKEY - WinVista: only install @@ -624,5 +617,8 @@ variants: only rtl8139 +include kvm_cdkeys.cfg + + # Choose your test list only fc8_quick -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 06/12] KVM test: minor style changes to kvm_config.py 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish @ 2009-08-02 23:58 ` Michael Goldish 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 07/12] KVM test: kvm_config.py: do not fail when including a nonexistent file Michael Goldish 0 siblings, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-02 23:58 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish - Fix indentation of a few lines. - Limit lines to 80 characters. - Use list comprehension in config.filter(). - Replace '== None' with 'is None'. Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/kvm_config.py | 34 ++++++++++++++++++---------------- 1 files changed, 18 insertions(+), 16 deletions(-) diff --git a/client/tests/kvm/kvm_config.py b/client/tests/kvm/kvm_config.py index 7e8b1db..e478a55 100755 --- a/client/tests/kvm/kvm_config.py +++ b/client/tests/kvm/kvm_config.py @@ -83,8 +83,8 @@ class config: @param filter: A regular expression that defines the filter. @param dict: Dictionary that will be inspected. """ - filter = re.compile("(\\.|^)" + filter + "(\\.|$)") - return filter.search(dict["name"]) != None + filter = re.compile(r"(\.|^)" + filter + r"(\.|$)") + return bool(filter.search(dict["name"])) def filter(self, filter, list=None): @@ -94,13 +94,9 @@ class config: @param filter: A regular expression that will be used as a filter. @param list: A list of dictionaries that will be filtered. """ - if list == None: + if list is None: list = self.list - filtered_list = [] - for dict in list: - if self.match(filter, dict): - filtered_list.append(dict) - return filtered_list + return [dict for dict in list if self.match(filter, dict)] # Currently unused, will be removed if it remains unused @@ -111,7 +107,7 @@ class config: @param filter: A regular expression that will filter the list. @param list: List which we want to know the indexes that match a filter. """ - if list == None: + if list is None: list = self.list block_list = [] prev_match = False @@ -248,17 +244,22 @@ class config: temp_list.append(new_dict) if subvariants: - # If we're parsing 'subvariants', we need to modify the list first - self.__modify_list_subvariants(temp_list, name, dep_list, add_to_shortname) + # If we're parsing 'subvariants', first modify the list + self.__modify_list_subvariants(temp_list, name, dep_list, + add_to_shortname) temp_list = self.parse(file, temp_list, restricted=True, prev_indent=indent) else: - # If we're parsing 'variants', we need to parse first and then modify the list + # If we're parsing 'variants', parse before modifying the list if self.debug: - self.__debug_print(indented_line, "Entering variant '%s' (variant inherits %d dicts)" % (name, len(list))) + self.__debug_print(indented_line, + "Entering variant '%s' " + "(variant inherits %d dicts)" % + (name, len(list))) temp_list = self.parse(file, temp_list, restricted=False, prev_indent=indent) - self.__modify_list_variants(temp_list, name, dep_list, add_to_shortname) + self.__modify_list_variants(temp_list, name, dep_list, + add_to_shortname) new_list += temp_list @@ -401,7 +402,7 @@ class config: continue if self.debug and not restricted: self.__debug_print(indented_line, - "Entering file %s" % words[1]) + "Entering file %s" % words[1]) if self.filename: filename = os.path.join(os.path.dirname(self.filename), words[1]) @@ -440,7 +441,8 @@ class config: # Everything inside an exception should be parsed in # restricted mode new_list += self.parse(file, list[index:index+1], - restricted=True, prev_indent=indent) + restricted=True, + prev_indent=indent) else: new_list += list[index:index+1] list = new_list -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 07/12] KVM test: kvm_config.py: do not fail when including a nonexistent file 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish @ 2009-08-02 23:58 ` Michael Goldish 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 08/12] KVM test: kvm_subprocess: send user specified parameters to kvm_tail callbacks Michael Goldish 0 siblings, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-02 23:58 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish Instead of failing, just print a warning. Included files may not always be crucial for tests to run (kvm_cdkeys.cfg for example). Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/kvm_config.py | 21 +++++++++++---------- 1 files changed, 11 insertions(+), 10 deletions(-) diff --git a/client/tests/kvm/kvm_config.py b/client/tests/kvm/kvm_config.py index e478a55..7ff7a07 100755 --- a/client/tests/kvm/kvm_config.py +++ b/client/tests/kvm/kvm_config.py @@ -406,17 +406,18 @@ class config: if self.filename: filename = os.path.join(os.path.dirname(self.filename), words[1]) - if not os.path.exists(filename): - e_msg = "Cannot include %s -- file not found" % filename - raise error.AutotestError(e_msg) - new_file = open(filename, "r") - list = self.parse(new_file, list, restricted) - new_file.close() - if self.debug and not restricted: - self.__debug_print("", "Leaving file %s" % words[1]) + if os.path.exists(filename): + new_file = open(filename, "r") + list = self.parse(new_file, list, restricted) + new_file.close() + if self.debug and not restricted: + self.__debug_print("", "Leaving file %s" % words[1]) + else: + print ("WARNING: Cannot include %s -- " + "file not found" % filename) else: - e_msg = "Cannot include anything because no file is open" - raise error.AutotestError(e_msg) + print ("WARNING: Cannot include %s because no file is " + "currently open" % words[1]) # Parse multi-line exceptions # (the block is parsed for each dict separately) -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 08/12] KVM test: kvm_subprocess: send user specified parameters to kvm_tail callbacks 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish @ 2009-08-02 23:58 ` Michael Goldish 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 09/12] KVM test: dynamically detect and cache MAC-IP pairs using tcpdump Michael Goldish 0 siblings, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-02 23:58 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish Allow the user to specify parameters to send to output_func and termination_func in addition to the regular output string and exit status. The user specified parameters will be pre-pended to the other parameters. This is mainly meant to bypass Python's refusal to pickle instance methods. Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/kvm_subprocess.py | 54 ++++++++++++++++++++++++++++++++--- 1 files changed, 49 insertions(+), 5 deletions(-) diff --git a/client/tests/kvm/kvm_subprocess.py b/client/tests/kvm/kvm_subprocess.py index c15e779..df1d562 100644 --- a/client/tests/kvm/kvm_subprocess.py +++ b/client/tests/kvm/kvm_subprocess.py @@ -464,7 +464,9 @@ class kvm_tail(kvm_spawn): """ def __init__(self, command=None, id=None, echo=False, linesep="\n", - termination_func=None, output_func=None, output_prefix=""): + termination_func=None, termination_params=(), + output_func=None, output_params=(), + output_prefix=""): """ Initialize the class and run command as a child process. @@ -479,10 +481,14 @@ class kvm_tail(kvm_spawn): child process by sendline(). @param termination_func: Function to call when the process exits. The function must accept a single exit status parameter. + @param termination_params: Parameters to send to termination_func + before the exit status. @param output_func: Function to call whenever a line of output is available from the STDOUT or STDERR streams of the process. The function must accept a single string parameter. The string does not include the final newline. + @param output_params: Parameters to send to output_func before the + output line. @param output_prefix: String to prepend to lines sent to output_func. """ # Add a reader and a close hook @@ -494,7 +500,9 @@ class kvm_tail(kvm_spawn): # Remember some attributes self.termination_func = termination_func + self.termination_params = termination_params self.output_func = output_func + self.output_params = output_params self.output_prefix = output_prefix # Start the thread in the background @@ -504,7 +512,9 @@ class kvm_tail(kvm_spawn): def __getinitargs__(self): return kvm_spawn.__getinitargs__(self) + (self.termination_func, + self.termination_params, self.output_func, + self.output_params, self.output_prefix) @@ -518,6 +528,16 @@ class kvm_tail(kvm_spawn): self.termination_func = termination_func + def set_termination_params(self, termination_params): + """ + Set the termination_params attribute. See __init__() for details. + + @param termination_params: Parameters to send to termination_func + before the exit status. + """ + self.termination_params = termination_params + + def set_output_func(self, output_func): """ Set the output_func attribute. See __init__() for details. @@ -528,6 +548,16 @@ class kvm_tail(kvm_spawn): self.output_func = output_func + def set_output_params(self, output_params): + """ + Set the output_params attribute. See __init__() for details. + + @param output_params: Parameters to send to output_func before the + output line. + """ + self.output_params = output_params + + def set_output_prefix(self, output_prefix): """ Set the output_prefix attribute. See __init__() for details. @@ -546,7 +576,8 @@ class kvm_tail(kvm_spawn): text = text.decode("utf-8", "replace") # Pass it to output_func try: - self.output_func(text) + params = self.output_params + (text,) + self.output_func(*params) except TypeError: pass @@ -587,7 +618,8 @@ class kvm_tail(kvm_spawn): return print_line("(Process terminated with status %s)" % status) try: - self.termination_func(status) + params = self.termination_params + (status,) + self.termination_func(*params) except TypeError: pass @@ -607,7 +639,9 @@ class kvm_expect(kvm_tail): """ def __init__(self, command=None, id=None, echo=False, linesep="\n", - termination_func=None, output_func=None, output_prefix=""): + termination_func=None, termination_params=(), + output_func=None, output_params=(), + output_prefix=""): """ Initialize the class and run command as a child process. @@ -622,10 +656,14 @@ class kvm_expect(kvm_tail): child process by sendline(). @param termination_func: Function to call when the process exits. The function must accept a single exit status parameter. + @param termination_params: Parameters to send to termination_func + before the exit status. @param output_func: Function to call whenever a line of output is available from the STDOUT or STDERR streams of the process. The function must accept a single string parameter. The string does not include the final newline. + @param output_params: Parameters to send to output_func before the + output line. @param output_prefix: String to prepend to lines sent to output_func. """ # Add a reader @@ -807,7 +845,9 @@ class kvm_shell_session(kvm_expect): """ def __init__(self, command=None, id=None, echo=False, linesep="\n", - termination_func=None, output_func=None, output_prefix="", + termination_func=None, termination_params=(), + output_func=None, output_params=(), + output_prefix="", prompt=r"[\#\$]\s*$", status_test_command="echo $?"): """ Initialize the class and run command as a child process. @@ -823,10 +863,14 @@ class kvm_shell_session(kvm_expect): child process by sendline(). @param termination_func: Function to call when the process exits. The function must accept a single exit status parameter. + @param termination_params: Parameters to send to termination_func + before the exit status. @param output_func: Function to call whenever a line of output is available from the STDOUT or STDERR streams of the process. The function must accept a single string parameter. The string does not include the final newline. + @param output_params: Parameters to send to output_func before the + output line. @param output_prefix: String to prepend to lines sent to output_func. @param prompt: Regular expression describing the shell's prompt line. @param status_test_command: Command to be used for getting the last -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 09/12] KVM test: dynamically detect and cache MAC-IP pairs using tcpdump 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish @ 2009-08-02 23:58 ` Michael Goldish 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 10/12] KVM test: kvm_utils.py: small fix for format_str_for_message() Michael Goldish 0 siblings, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-02 23:58 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish Run tcpdump in the background at all times and listen for DHCP communication. Whenever a DHCP server assigns an IP address to a MAC address, cache it for future reference. Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/kvm_preprocessing.py | 41 +++++++++++++++++++++++++++++++++ 1 files changed, 41 insertions(+), 0 deletions(-) diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py index 4d77630..04bdb59 100644 --- a/client/tests/kvm/kvm_preprocessing.py +++ b/client/tests/kvm/kvm_preprocessing.py @@ -196,6 +196,26 @@ def preprocess(test, params, env): @param params: A dict containing all VM and image parameters. @param env: The environment (a dict-like object). """ + # Start tcpdump if it isn't already running + if not env.has_key("address_cache"): + env["address_cache"] = {} + if env.has_key("tcpdump") and not env["tcpdump"].is_alive(): + env["tcpdump"].close() + del env["tcpdump"] + if not env.has_key("tcpdump"): + command = "/usr/sbin/tcpdump -npvi any 'dst port 68'" + logging.debug("Starting tcpdump (%s)...", command) + env["tcpdump"] = kvm_subprocess.kvm_tail( + command=command, + output_func=_update_address_cache, + output_params=(env["address_cache"],)) + if kvm_utils.wait_for(lambda: not env["tcpdump"].is_alive(), + 0.1, 0.1, 1.0): + logging.warn("Could not start tcpdump") + logging.warn("Status: %s" % env["tcpdump"].get_status()) + logging.warn("Output:" + kvm_utils.format_str_for_message( + env["tcpdump"].get_output())) + # Destroy and remove VMs that are no longer needed in the environment requested_vms = kvm_utils.get_sub_dict_names(params, "vms") for key in env.keys(): @@ -280,6 +300,12 @@ def postprocess(test, params, env): int(params.get("post_command_timeout", "600")), params.get("post_command_noncritical") == "yes") + # Terminate tcpdump if no VMs are alive + living_vms = [vm for vm in kvm_utils.env_get_all_vms(env) if vm.is_alive()] + if not living_vms and env.has_key("tcpdump"): + env["tcpdump"].close() + del env["tcpdump"] + def postprocess_on_error(test, params, env): """ @@ -290,3 +316,18 @@ def postprocess_on_error(test, params, env): @param env: The environment (a dict-like object). """ params.update(kvm_utils.get_sub_dict(params, "on_error")) + + +def _update_address_cache(address_cache, line): + if re.search("Your.IP", line, re.IGNORECASE): + matches = re.findall(r"\d*\.\d*\.\d*\.\d*", line) + if matches: + address_cache["last_seen"] = matches[0] + if re.search("Client.Ethernet.Address", line, re.IGNORECASE): + matches = re.findall(r"\w*:\w*:\w*:\w*:\w*:\w*", line) + if matches and address_cache.get("last_seen"): + mac_address = matches[0].lower() + logging.debug("(address cache) Adding cache entry: %s ---> %s", + mac_address, address_cache.get("last_seen")) + address_cache[mac_address] = address_cache.get("last_seen") + del address_cache["last_seen"] -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 10/12] KVM test: kvm_utils.py: small fix for format_str_for_message() 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish @ 2009-08-02 23:58 ` Michael Goldish 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish 0 siblings, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-02 23:58 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish format_str_for_message() sometimes adds an extra newline. Fix that. Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/kvm_utils.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py index 5d8291f..1c11fa4 100644 --- a/client/tests/kvm/kvm_utils.py +++ b/client/tests/kvm/kvm_utils.py @@ -654,7 +654,9 @@ def format_str_for_message(str): @param str: string that will be formatted. """ - num_lines = len(str.splitlines()) + lines = str.splitlines() + num_lines = len(lines) + str = "\n".join(lines) if num_lines == 0: return "" elif num_lines == 1: -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils 2009-08-02 23:58 [KVM-AUTOTEST PATCH 0/12] TAP support (and a few other small things) Michael Goldish @ 2009-08-02 23:58 ` Michael Goldish 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 11/12] KVM test: make VMs use a dynamic MAC-IP mapping generated by tcpdump Michael Goldish 0 siblings, 1 reply; 7+ messages in thread From: Michael Goldish @ 2009-08-02 23:58 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish Add function get_mac_ip_pair_from_dict() which gets a dict (specified in the config file by the user) and fetches a MAC-IP address pair according to a certain syntax. The syntax allows the user to specify a group of MAC-IP address ranges. For example: address_ranges = r1 r2 r3 address_range_base_mac_r1 = 55:44:33:22:11:00 address_range_base_ip_r1 = 10.0.0.0 address_range_size_r1 = 16 address_range_base_mac_r2 = 55:44:33:22:11:40 address_range_base_ip_r2 = 10.0.0.60 address_range_size_r2 = 25 address_range_base_mac_r3 = 55:44:33:22:12:10 address_range_base_ip_r3 = 10.0.1.20 address_range_size_r3 = 230 The lines above may be specified globally, so that they apply to all VMs and all NICs. However, a line similar to the following must be specified per NIC (each VM may have several NICs): address_index = 0 Currently, we usually use a single VM and a single NIC, so specifying address_index once should suffice. If a test requires an additional NIC the user should add something like: address_index_nic2 = 1 The index is simply the index in the MAC-IP table that consists of all the specified ranges. In the above example, if the user specifies an index of 18, the MAC-IP pair will be taken from the second range, because the first range has only 16 entries. When running migration, both the source and destination VMs should have the same address_index, because they should have the same MAC and IP addresses. Note that different copies of the KVM test, running simultaneously in the same network environment, _must_ specify different MAC-IP pools. This can be done in several ways: - By specifying the ranges (as in the example above) in an external file such as /etc/kvm-autotest/client_mac_ip_pool.cfg, and setting up that file for each host manually - By keeping several .cfg files with different names that match the hostname of each host, e.g. mac_ip_pool_hostname1.cfg, mac_ip_pool_hostname2.cfg (hostname1 and hostname2 should be replaced by actual hostnames), and parsing the right file in the control file at runtime - By defining all the different pools in a variants block in a single file, and specifying 'only <hostname>' at runtime in the control file (using config.parse_string()) When we start running in server mode, assigning MAC and IP addresses to hosts can be done automatically by the server, but the user will still be required to specify a single global pool for the server (which the server will divide among the hosts). The address_index parameter will be specified inside the regular config file, and does not need to be different for each host. This patch also adds some small utility functions used by get_mac_ip_pair_from_dict(). Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/kvm_utils.py | 110 ++++++++++++++++++++++++++++++++++++++++- 1 files changed, 108 insertions(+), 2 deletions(-) diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py index e897e78..5d8291f 100644 --- a/client/tests/kvm/kvm_utils.py +++ b/client/tests/kvm/kvm_utils.py @@ -1,7 +1,7 @@ import md5, thread, subprocess, time, string, random, socket, os, signal, pty import select, re, logging -from autotest_lib.client.bin import utils -from autotest_lib.client.common_lib import error +#from autotest_lib.client.bin import utils +#from autotest_lib.client.common_lib import error import kvm_subprocess """ @@ -46,6 +46,112 @@ def get_sub_dict_names(dict, keyword): return [] +# Functions related to MAC/IP addresses + +def mac_str_to_int(addr): + """ + Convert MAC address string to integer. + + @param addr: String representing the MAC address. + """ + return sum(int(s, 16) * 256 ** i + for i, s in enumerate(reversed(addr.split(":")))) + + +def mac_int_to_str(addr): + """ + Convert MAC address integer to string. + + @param addr: Integer representing the MAC address. + """ + return ":".join("%02x" % (addr >> 8 * i & 0xFF) + for i in reversed(range(6))) + + +def ip_str_to_int(addr): + """ + Convert IP address string to integer. + + @param addr: String representing the IP address. + """ + return sum(int(s) * 256 ** i + for i, s in enumerate(reversed(addr.split(".")))) + + +def ip_int_to_str(addr): + """ + Convert IP address integer to string. + + @param addr: Integer representing the IP address. + """ + return ".".join(str(addr >> 8 * i & 0xFF) + for i in reversed(range(4))) + + +def offset_mac(base, offset): + """ + Add offset to a given MAC address. + + @param base: String representing a MAC address. + @param offset: Offset to add to base (integer) + @return: A string representing the offset MAC address. + """ + return mac_int_to_str(mac_str_to_int(base) + offset) + + +def offset_ip(base, offset): + """ + Add offset to a given IP address. + + @param base: String representing an IP address. + @param offset: Offset to add to base (integer) + @return: A string representing the offset IP address. + """ + return ip_int_to_str(ip_str_to_int(base) + offset) + + +def get_mac_ip_pair_from_dict(dict): + """ + Fetch a MAC-IP address pair from dict and return it. + + The parameters in dict are expected to conform to a certain syntax. + Typical usage may be: + + address_ranges = r1 r2 r3 + + address_range_base_mac_r1 = 55:44:33:22:11:00 + address_range_base_ip_r1 = 10.0.0.0 + address_range_size_r1 = 16 + + address_range_base_mac_r2 = 55:44:33:22:11:40 + address_range_base_ip_r2 = 10.0.0.60 + address_range_size_r2 = 25 + + address_range_base_mac_r3 = 55:44:33:22:12:10 + address_range_base_ip_r3 = 10.0.1.20 + address_range_size_r3 = 230 + + address_index = 0 + + All parameters except address_index specify a MAC-IP address pool. The + pool consists of several MAC-IP address ranges. + address_index specified the index of the desired MAC-IP pair from the pool. + + @param dict: The dictionary from which to fetch the addresses. + """ + index = int(dict.get("address_index", 0)) + for mac_range_name in get_sub_dict_names(dict, "address_ranges"): + mac_range_params = get_sub_dict(dict, mac_range_name) + mac_base = mac_range_params.get("address_range_base_mac") + ip_base = mac_range_params.get("address_range_base_ip") + size = int(mac_range_params.get("address_range_size", 1)) + if index < size: + return (mac_base and offset_mac(mac_base, index), + ip_base and offset_ip(ip_base, index)) + index -= size + return (None, None) + + # Functions for working with the environment (a dict-like object) def is_vm(obj): -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [KVM-AUTOTEST PATCH 11/12] KVM test: make VMs use a dynamic MAC-IP mapping generated by tcpdump 2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish @ 2009-08-02 23:58 ` Michael Goldish 0 siblings, 0 replies; 7+ messages in thread From: Michael Goldish @ 2009-08-02 23:58 UTC (permalink / raw) To: autotest, kvm; +Cc: Michael Goldish In VM.get_address(), return an IP address from the MAC-IP cache if an IP address base wasn't specified by the user, or if the user requested that IP addresses be taken from the dynamic cache. To force the system to obtain IP addresses from the dynamic cache even when static base addresses are provided by the user, specify 'always_use_tcpdump = yes'. Signed-off-by: Michael Goldish <mgoldish@redhat.com> --- client/tests/kvm/kvm_address_pools.cfg.sample | 8 +++ client/tests/kvm/kvm_preprocessing.py | 3 +- client/tests/kvm/kvm_utils.py | 28 +++++++++++- client/tests/kvm/kvm_vm.py | 62 +++++++++++++++++------- 4 files changed, 81 insertions(+), 20 deletions(-) diff --git a/client/tests/kvm/kvm_address_pools.cfg.sample b/client/tests/kvm/kvm_address_pools.cfg.sample index 8a27ee1..debfe56 100644 --- a/client/tests/kvm/kvm_address_pools.cfg.sample +++ b/client/tests/kvm/kvm_address_pools.cfg.sample @@ -6,6 +6,14 @@ # If you wish to use a static MAC-IP mapping, where each MAC address range is # mapped to a known corresponding IP address range, specify the bases of the IP # address ranges in this file. +# If you specify a MAC address range without a corresponding IP address range, +# the IP addresses for that range will be determined at runtime by listening +# to DHCP traffic using tcpdump. +# If you wish to determine IP addresses using tcpdump in any case, regardless +# of any # IP addresses specified in this file, uncomment the following line: +#always_use_tcpdump = yes +# You may also specify this parameter for specific hosts by adding it in the +# appropriate sections below. variants: # Rename host1 to an actual (short) hostname in the network that will be running the Autotest client diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py index 04bdb59..a5a32dc 100644 --- a/client/tests/kvm/kvm_preprocessing.py +++ b/client/tests/kvm/kvm_preprocessing.py @@ -54,7 +54,8 @@ def preprocess_vm(test, params, env, name): logging.debug("VM object found in environment") else: logging.debug("VM object does not exist; creating it") - vm = kvm_vm.VM(name, params, qemu_path, image_dir, iso_dir, script_dir) + vm = kvm_vm.VM(name, params, qemu_path, image_dir, iso_dir, script_dir, + env.get("address_cache")) kvm_utils.env_register_vm(env, name, vm) start_vm = False diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py index 1c11fa4..e17d84f 100644 --- a/client/tests/kvm/kvm_utils.py +++ b/client/tests/kvm/kvm_utils.py @@ -1,5 +1,5 @@ import md5, thread, subprocess, time, string, random, socket, os, signal, pty -import select, re, logging +import select, re, logging, commands #from autotest_lib.client.bin import utils #from autotest_lib.client.common_lib import error import kvm_subprocess @@ -152,6 +152,32 @@ def get_mac_ip_pair_from_dict(dict): return (None, None) +def verify_mac_ip_pair(mac, ip, timeout=3.0): + """ + Connect to a given IP address and make sure its MAC address equals the + given MAC address. + + @param mac: A MAC address. + @param ip: An IP address. + @return: True iff ip is assigned to mac. + """ + s = socket.socket() + s.setblocking(False) + try: + s.connect((ip, 55555)) + except socket.error: + pass + end_time = time.time() + timeout + while time.time() < end_time: + o = commands.getoutput("/sbin/arp -n") + if re.search(r"\b%s\b.*\b%s\b" % (ip, mac), o, re.IGNORECASE): + s.close() + return True + time.sleep(0.1) + s.close() + return False + + # Functions for working with the environment (a dict-like object) def is_vm(obj): diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py index 0c35e64..6addf77 100644 --- a/client/tests/kvm/kvm_vm.py +++ b/client/tests/kvm/kvm_vm.py @@ -1,5 +1,5 @@ #!/usr/bin/python -import time, socket, os, logging, fcntl +import time, socket, os, logging, fcntl, re, commands import kvm_utils, kvm_subprocess """ @@ -101,7 +101,7 @@ class VM: """ def __init__(self, name, params, qemu_path, image_dir, iso_dir, - script_dir): + script_dir, address_cache): """ Initialize the object and set a few attributes. @@ -112,6 +112,7 @@ class VM: @param image_dir: The directory where images reside @param iso_dir: The directory where ISOs reside @param script_dir: The directory where -net tap scripts reside + @param address_cache: A dict that maps MAC addresses to IP addresses """ self.process = None self.redirs = {} @@ -124,6 +125,7 @@ class VM: self.image_dir = image_dir self.iso_dir = iso_dir self.script_dir = script_dir + self.address_cache = address_cache # Find available monitor filename while True: @@ -137,7 +139,7 @@ class VM: def clone(self, name=None, params=None, qemu_path=None, image_dir=None, - iso_dir=None, script_dir=None): + iso_dir=None, script_dir=None, address_cache=None): """ Return a clone of the VM object with optionally modified parameters. The clone is initially not alive and needs to be started using create(). @@ -150,6 +152,7 @@ class VM: @param image_dir: Optional new image dir @param iso_dir: Optional new iso directory @param script_dir: Optional new -net tap script directory + @param address_cache: A dict that maps MAC addresses to IP addresses """ if name == None: name = self.name @@ -163,7 +166,10 @@ class VM: iso_dir = self.iso_dir if script_dir == None: script_dir = self.script_dir - return VM(name, params, qemu_path, image_dir, iso_dir, script_dir) + if address_cache == None: + address_cache = self.address_cache + return VM(name, params, qemu_path, image_dir, iso_dir, script_dir, + address_cache) def make_qemu_command(self, name=None, params=None, qemu_path=None, @@ -625,21 +631,36 @@ class VM: nic_params = kvm_utils.get_sub_dict(self.params, nic_name) if nic_params.get("nic_mode") == "tap": mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params) + if not mac: + logging.debug("MAC address unavailable") + return None + if not ip or nic_params.get("always_use_tcpdump") == "yes": + # Get the IP address from the cache + ip = self.address_cache.get(mac) + if not ip: + logging.debug("Could not find IP address for MAC address: " + "%s" % mac) + return None + # Make sure the IP address is assigned to the right MAC address + if not kvm_utils.verify_mac_ip_pair(mac, ip): + logging.debug("Could not verify MAC-IP address pair: " + "%s ---> %s" % (mac, ip)) + return None return ip else: return "localhost" - def get_port(self, port, index=0): + def get_port(self, port, nic_index=0): """ Return the port in host space corresponding to port in guest space. @param port: Port number in host space. - @param index: Index of the NIC. + @param nic_index: Index of the NIC. @return: If port redirection is used, return the host port redirected to guest port port. Otherwise return port. """ - nic_name = kvm_utils.get_sub_dict_names(self.params, "nics")[index] + nic_name = kvm_utils.get_sub_dict_names(self.params, "nics")[nic_index] nic_params = kvm_utils.get_sub_dict(self.params, nic_name) if nic_params.get("nic_mode") == "tap": return port @@ -666,17 +687,19 @@ class VM: """ address = self.get_address() port = self.get_port(int(self.params.get("ssh_port"))) - if not port: + if not address or not port: + logging.debug("IP address or port unavailable") return False return kvm_utils.is_sshd_running(address, port, timeout=timeout) - def ssh_login(self, timeout=10): + def ssh_login(self, nic_index=0, timeout=10): """ Log into the guest via SSH/Telnet. If timeout expires while waiting for output from the guest (e.g. a password prompt or a shell prompt) -- fail. + @param nic_index: The index of the NIC to connect to. @param timeout: Time (seconds) before giving up logging into the guest. @return: kvm_spawn object on success and None on failure. @@ -685,9 +708,10 @@ class VM: password = self.params.get("password", "") prompt = self.params.get("ssh_prompt", "[\#\$]") use_telnet = self.params.get("use_telnet") == "yes" - address = self.get_address() + address = self.get_address(nic_index) port = self.get_port(int(self.params.get("ssh_port"))) - if not port: + if not address or not port: + logging.debug("IP address or port unavailable") return None if use_telnet: @@ -702,7 +726,7 @@ class VM: return session - def scp_to_remote(self, local_path, remote_path, timeout=300): + def scp_to_remote(self, local_path, remote_path, nic_index=0, timeout=300): """ Transfer files to the guest via SCP. @@ -713,15 +737,16 @@ class VM: """ username = self.params.get("username", "") password = self.params.get("password", "") - address = self.get_address() + address = self.get_address(nic_index) port = self.get_port(int(self.params.get("ssh_port"))) - if not port: + if not address or not port: + logging.debug("IP address or port unavailable") return None return kvm_utils.scp_to_remote(address, port, username, password, local_path, remote_path, timeout) - def scp_from_remote(self, remote_path, local_path, timeout=300): + def scp_from_remote(self, remote_path, local_path, nic_index=0, timeout=300): """ Transfer files from the guest via SCP. @@ -732,9 +757,10 @@ class VM: """ username = self.params.get("username", "") password = self.params.get("password", "") - address = self.get_address() + address = self.get_address(nic_index) port = self.get_port(int(self.params.get("ssh_port"))) - if not port: + if not address or not port: + logging.debug("IP address or port unavailable") return None return kvm_utils.scp_from_remote(address, port, username, password, remote_path, local_path, timeout) @@ -749,7 +775,7 @@ class VM: @return: A tuple (status, output). status is 0 on success and 1 on failure. """ - session = self.ssh_login(timeout) + session = self.ssh_login(timeout=timeout) if not session: return (1, "") -- 1.5.4.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2009-08-11 16:16 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2009-08-06 18:41 [KVM-AUTOTEST PATCH CORRECTIONS] Corrections to the TAP patchset Michael Goldish
2009-08-06 18:41 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish
2009-08-06 18:41 ` [KVM-AUTOTEST PATCH 03/12] KVM test: add sample 'qemu-ifup' script Michael Goldish
2009-08-06 18:41 ` [KVM-AUTOTEST PATCH 11/12] KVM test: make VMs use a dynamic MAC-IP mapping generated by tcpdump Michael Goldish
2009-08-11 14:43 ` Jason Wang
[not found] <1960092437.1769331250006819543.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com>
2009-08-11 16:16 ` Michael Goldish
-- strict thread matches above, loose matches on Subject: below --
2009-08-02 23:58 [KVM-AUTOTEST PATCH 0/12] TAP support (and a few other small things) Michael Goldish
2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 02/12] KVM test: add basic TAP support with static MAC-IP mapping Michael Goldish
2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 03/12] KVM test: add sample 'qemu-ifup' script Michael Goldish
2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 04/12] KVM test: add host-specific MAC-IP pool configuration Michael Goldish
2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 05/12] KVM test: specify installation cdkeys in a dedicated file Michael Goldish
2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 06/12] KVM test: minor style changes to kvm_config.py Michael Goldish
2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 07/12] KVM test: kvm_config.py: do not fail when including a nonexistent file Michael Goldish
2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 08/12] KVM test: kvm_subprocess: send user specified parameters to kvm_tail callbacks Michael Goldish
2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 09/12] KVM test: dynamically detect and cache MAC-IP pairs using tcpdump Michael Goldish
2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 10/12] KVM test: kvm_utils.py: small fix for format_str_for_message() Michael Goldish
2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 01/12] KVM test: add some MAC/IP address utilities to kvm_utils Michael Goldish
2009-08-02 23:58 ` [KVM-AUTOTEST PATCH 11/12] KVM test: make VMs use a dynamic MAC-IP mapping generated by tcpdump Michael Goldish
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.