[PATCH 1/3] KVM test: Use customized command to get the version of kvm and its

[PATCH 1/3] KVM test: Use customized command to get the version of kvm and its

[Jason Wang]

userspace

Current method may or may not work for various kinds of
distribution. So this patch enable the ability to use customized
commands to get the version of kvm and its userspace. "kvm_ver_cmd" is
used for kvm verison and "kvm_userspace_ver_cmd" is for its userspace.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 client/tests/kvm/kvm_preprocessing.py |   18 +++++++++---------
 1 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py
index 4b9290c..16200ab 100644
--- a/client/tests/kvm/kvm_preprocessing.py
+++ b/client/tests/kvm/kvm_preprocessing.py
@@ -225,10 +225,10 @@ def preprocess(test, params, env):
     # Get the KVM kernel module version and write it as a keyval
     logging.debug("Fetching KVM module version...")
     if os.path.exists("/dev/kvm"):
-        try:
-            kvm_version = open("/sys/module/kvm/version").read().strip()
-        except:
-            kvm_version = os.uname()[2]
+        kvm_ver_cmd = params.get("kvm_ver_cmd", "cat /sys/module/kvm/version")
+        s, kvm_version = commands.getstatusoutput(kvm_ver_cmd)
+        if s != 0:
+            kvm_version = "Unknown"
     else:
         kvm_version = "Unknown"
         logging.debug("KVM module not loaded")
@@ -239,11 +239,11 @@ def preprocess(test, params, env):
     logging.debug("Fetching KVM userspace version...")
     qemu_path = kvm_utils.get_path(test.bindir, params.get("qemu_binary",
                                                            "qemu"))
-    version_line = commands.getoutput("%s -help | head -n 1" % qemu_path)
-    matches = re.findall("[Vv]ersion .*?,", version_line)
-    if matches:
-        kvm_userspace_version = " ".join(matches[0].split()[1:]).strip(",")
-    else:
+    def_qemu_ver_cmd = "%s -help | head -n 1 | awk '{ print $5}'" % qemu_path
+    kvm_userspace_ver_cmd = params.get("kvm_userspace_ver_cmd",
+                                       def_qemu_ver_cmd)
+    s, kvm_userspace_version = commands.getstatusoutput(kvm_userspace_ver_cmd)
+    if s != 0:
         kvm_userspace_version = "Unknown"
         logging.debug("Could not fetch KVM userspace version")
     logging.debug("KVM userspace version: %s" % kvm_userspace_version)

[PATCH 2/3] KVM test: Create ksm scanner through pre_command

[Jason Wang]

KSM may have various control interface for different distributions,so
this patch launch ksm through pre_command instead of the hard-coded
bits in the test. User may specify their owner suitable commands or
paramteres.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 client/tests/kvm/tests/ksm_overcommit.py |   15 ---------------
 client/tests/kvm/tests_base.cfg.sample   |    2 ++
 2 files changed, 2 insertions(+), 15 deletions(-)

diff --git a/client/tests/kvm/tests/ksm_overcommit.py b/client/tests/kvm/tests/ksm_overcommit.py
index 2dd46c4..4aa6deb 100644
--- a/client/tests/kvm/tests/ksm_overcommit.py
+++ b/client/tests/kvm/tests/ksm_overcommit.py
@@ -412,21 +412,6 @@ def run_ksm_overcommit(test, params, env):
                                      (3100 - 64.0)))
                 mem = int(math.floor(host_mem * overcommit / vmsc))
 
-    logging.debug("Checking KSM status...")
-    ksm_flag = 0
-    for line in os.popen('ksmctl info').readlines():
-        if line.startswith('flags'):
-            ksm_flag = int(line.split(' ')[1].split(',')[0])
-    if int(ksm_flag) != 1:
-        logging.info("KSM module is not loaded! Trying to load module and "
-                     "start ksmctl...")
-        try:
-            utils.run("modprobe ksm")
-            utils.run("ksmctl start 5000 100")
-        except error.CmdError, e:
-            raise error.TestFail("Failed to load KSM: %s" % e)
-    logging.debug("KSM module loaded and ksmctl started")
-
     swap = int(utils.read_from_meminfo("SwapTotal")) / 1024
 
     logging.debug("Overcommit = %f", overcommit)
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index e73ba44..2db0d2c 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -285,6 +285,8 @@ variants:
         catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
 
     - ksm_overcommit:
+        pre_command = "[ -e /dev/ksm ] && true || modprobe ksm && ksmctl start 5000 50"
+        pre_command_critical = yes
         # Don't preprocess any vms as we need to change its params
         vms = ''
         image_snapshot = yes

[PATCH 3/3] KVM test: Remove the duplicated KERNEL paramters in the pxe configuration file

[Jason Wang]

Remove the duplicated "KERNEL vmlinuz" in unattended.py

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 client/tests/kvm/scripts/unattended.py |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/client/tests/kvm/scripts/unattended.py b/client/tests/kvm/scripts/unattended.py
index e41bc86..fdadd03 100755
--- a/client/tests/kvm/scripts/unattended.py
+++ b/client/tests/kvm/scripts/unattended.py
@@ -209,7 +209,6 @@ class UnattendedInstall(object):
         pxe_config.write('PROMPT 0\n')
         pxe_config.write('LABEL pxeboot\n')
         pxe_config.write('     KERNEL vmlinuz\n')
-        pxe_config.write('     KERNEL vmlinuz\n')
         pxe_config.write('     APPEND initrd=initrd.img %s\n' %
                          self.kernel_args)
         pxe_config.close()

Re: [Autotest] [PATCH 3/3] KVM test: Remove the duplicated KERNEL paramters in the pxe configuration file

[Lucas Meneghel Rodrigues]

On Mon, Apr 26, 2010 at 7:07 AM, Jason Wang <jasowang@redhat.com> wrote:
> Remove the duplicated "KERNEL vmlinuz" in unattended.py

Good catch, applied, thanks!

> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  client/tests/kvm/scripts/unattended.py |    1 -
>  1 files changed, 0 insertions(+), 1 deletions(-)
>
> diff --git a/client/tests/kvm/scripts/unattended.py b/client/tests/kvm/scripts/unattended.py
> index e41bc86..fdadd03 100755
> --- a/client/tests/kvm/scripts/unattended.py
> +++ b/client/tests/kvm/scripts/unattended.py
> @@ -209,7 +209,6 @@ class UnattendedInstall(object):
>         pxe_config.write('PROMPT 0\n')
>         pxe_config.write('LABEL pxeboot\n')
>         pxe_config.write('     KERNEL vmlinuz\n')
> -        pxe_config.write('     KERNEL vmlinuz\n')
>         pxe_config.write('     APPEND initrd=initrd.img %s\n' %
>                          self.kernel_args)
>         pxe_config.close()
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>



-- 
Lucas

Re: [Autotest] [PATCH 2/3] KVM test: Create ksm scanner through pre_command

[Lucas Meneghel Rodrigues]

On Mon, Apr 26, 2010 at 7:07 AM, Jason Wang <jasowang@redhat.com> wrote:
> KSM may have various control interface for different distributions,so
> this patch launch ksm through pre_command instead of the hard-coded
> bits in the test. User may specify their owner suitable commands or
> paramteres.

Applied, thanks!

> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  client/tests/kvm/tests/ksm_overcommit.py |   15 ---------------
>  client/tests/kvm/tests_base.cfg.sample   |    2 ++
>  2 files changed, 2 insertions(+), 15 deletions(-)
>
> diff --git a/client/tests/kvm/tests/ksm_overcommit.py b/client/tests/kvm/tests/ksm_overcommit.py
> index 2dd46c4..4aa6deb 100644
> --- a/client/tests/kvm/tests/ksm_overcommit.py
> +++ b/client/tests/kvm/tests/ksm_overcommit.py
> @@ -412,21 +412,6 @@ def run_ksm_overcommit(test, params, env):
>                                      (3100 - 64.0)))
>                 mem = int(math.floor(host_mem * overcommit / vmsc))
>
> -    logging.debug("Checking KSM status...")
> -    ksm_flag = 0
> -    for line in os.popen('ksmctl info').readlines():
> -        if line.startswith('flags'):
> -            ksm_flag = int(line.split(' ')[1].split(',')[0])
> -    if int(ksm_flag) != 1:
> -        logging.info("KSM module is not loaded! Trying to load module and "
> -                     "start ksmctl...")
> -        try:
> -            utils.run("modprobe ksm")
> -            utils.run("ksmctl start 5000 100")
> -        except error.CmdError, e:
> -            raise error.TestFail("Failed to load KSM: %s" % e)
> -    logging.debug("KSM module loaded and ksmctl started")
> -
>     swap = int(utils.read_from_meminfo("SwapTotal")) / 1024
>
>     logging.debug("Overcommit = %f", overcommit)
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index e73ba44..2db0d2c 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -285,6 +285,8 @@ variants:
>         catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
>
>     - ksm_overcommit:
> +        pre_command = "[ -e /dev/ksm ] && true || modprobe ksm && ksmctl start 5000 50"
> +        pre_command_critical = yes
>         # Don't preprocess any vms as we need to change its params
>         vms = ''
>         image_snapshot = yes
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>



-- 
Lucas

Re: [Autotest] [PATCH 1/3] KVM test: Use customized command to get the version of kvm and its

[Lucas Meneghel Rodrigues]

On Mon, Apr 26, 2010 at 7:07 AM, Jason Wang <jasowang@redhat.com> wrote:
> userspace
>
> Current method may or may not work for various kinds of
> distribution. So this patch enable the ability to use customized
> commands to get the version of kvm and its userspace. "kvm_ver_cmd" is
> used for kvm verison and "kvm_userspace_ver_cmd" is for its userspace.

The method we are currently using is pretty satisfactory - if we fail
in getting /sys/module/kvm/version we use the kernel version as a
fallback, which is good for the kernel module. For qemu, we make a
regular expression searching for numbers following the string version,
so I don't see a reason on why we should make it configurable. Care to
provide an example of a situation where the current method fails?

> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  client/tests/kvm/kvm_preprocessing.py |   18 +++++++++---------
>  1 files changed, 9 insertions(+), 9 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py
> index 4b9290c..16200ab 100644
> --- a/client/tests/kvm/kvm_preprocessing.py
> +++ b/client/tests/kvm/kvm_preprocessing.py
> @@ -225,10 +225,10 @@ def preprocess(test, params, env):
>     # Get the KVM kernel module version and write it as a keyval
>     logging.debug("Fetching KVM module version...")
>     if os.path.exists("/dev/kvm"):
> -        try:
> -            kvm_version = open("/sys/module/kvm/version").read().strip()
> -        except:
> -            kvm_version = os.uname()[2]
> +        kvm_ver_cmd = params.get("kvm_ver_cmd", "cat /sys/module/kvm/version")
> +        s, kvm_version = commands.getstatusoutput(kvm_ver_cmd)
> +        if s != 0:
> +            kvm_version = "Unknown"
>     else:
>         kvm_version = "Unknown"
>         logging.debug("KVM module not loaded")
> @@ -239,11 +239,11 @@ def preprocess(test, params, env):
>     logging.debug("Fetching KVM userspace version...")
>     qemu_path = kvm_utils.get_path(test.bindir, params.get("qemu_binary",
>                                                            "qemu"))
> -    version_line = commands.getoutput("%s -help | head -n 1" % qemu_path)
> -    matches = re.findall("[Vv]ersion .*?,", version_line)
> -    if matches:
> -        kvm_userspace_version = " ".join(matches[0].split()[1:]).strip(",")
> -    else:
> +    def_qemu_ver_cmd = "%s -help | head -n 1 | awk '{ print $5}'" % qemu_path
> +    kvm_userspace_ver_cmd = params.get("kvm_userspace_ver_cmd",
> +                                       def_qemu_ver_cmd)
> +    s, kvm_userspace_version = commands.getstatusoutput(kvm_userspace_ver_cmd)
> +    if s != 0:
>         kvm_userspace_version = "Unknown"
>         logging.debug("Could not fetch KVM userspace version")
>     logging.debug("KVM userspace version: %s" % kvm_userspace_version)
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>



-- 
Lucas

Re: [Autotest] [PATCH 1/3] KVM test: Use customized command to get the version of kvm and its

[Jason Wang]

Lucas Meneghel Rodrigues wrote:
> On Mon, Apr 26, 2010 at 7:07 AM, Jason Wang <jasowang@redhat.com> wrote:
>   
>> userspace
>>
>> Current method may or may not work for various kinds of
>> distribution. So this patch enable the ability to use customized
>> commands to get the version of kvm and its userspace. "kvm_ver_cmd" is
>> used for kvm verison and "kvm_userspace_ver_cmd" is for its userspace.
>>     
>
> The method we are currently using is pretty satisfactory - if we fail
> in getting /sys/module/kvm/version we use the kernel version as a
> fallback, which is good for the kernel module. For qemu, we make a
> regular expression searching for numbers following the string version,
> so I don't see a reason on why we should make it configurable. Care to
> provide an example of a situation where the current method fails?
>
>   
Current method may be not as accurate as we expected.
In my Fedora box, the output of qemu-kvm -h | head -n 1 is something like:
QEMU PC emulator version 0.9.1 (kvm-83-maint-snapshot-20090205), 
Copyright (c) 2003-2008 Fabrice Bellard
but the rpm -qa may tell more accurate version:
qemu-kvm-0.11.0-13.fc12.x86_64
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> ---
>>  client/tests/kvm/kvm_preprocessing.py |   18 +++++++++---------
>>  1 files changed, 9 insertions(+), 9 deletions(-)
>>
>> diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py
>> index 4b9290c..16200ab 100644
>> --- a/client/tests/kvm/kvm_preprocessing.py
>> +++ b/client/tests/kvm/kvm_preprocessing.py
>> @@ -225,10 +225,10 @@ def preprocess(test, params, env):
>>     # Get the KVM kernel module version and write it as a keyval
>>     logging.debug("Fetching KVM module version...")
>>     if os.path.exists("/dev/kvm"):
>> -        try:
>> -            kvm_version = open("/sys/module/kvm/version").read().strip()
>> -        except:
>> -            kvm_version = os.uname()[2]
>> +        kvm_ver_cmd = params.get("kvm_ver_cmd", "cat /sys/module/kvm/version")
>> +        s, kvm_version = commands.getstatusoutput(kvm_ver_cmd)
>> +        if s != 0:
>> +            kvm_version = "Unknown"
>>     else:
>>         kvm_version = "Unknown"
>>         logging.debug("KVM module not loaded")
>> @@ -239,11 +239,11 @@ def preprocess(test, params, env):
>>     logging.debug("Fetching KVM userspace version...")
>>     qemu_path = kvm_utils.get_path(test.bindir, params.get("qemu_binary",
>>                                                            "qemu"))
>> -    version_line = commands.getoutput("%s -help | head -n 1" % qemu_path)
>> -    matches = re.findall("[Vv]ersion .*?,", version_line)
>> -    if matches:
>> -        kvm_userspace_version = " ".join(matches[0].split()[1:]).strip(",")
>> -    else:
>> +    def_qemu_ver_cmd = "%s -help | head -n 1 | awk '{ print $5}'" % qemu_path
>> +    kvm_userspace_ver_cmd = params.get("kvm_userspace_ver_cmd",
>> +                                       def_qemu_ver_cmd)
>> +    s, kvm_userspace_version = commands.getstatusoutput(kvm_userspace_ver_cmd)
>> +    if s != 0:
>>         kvm_userspace_version = "Unknown"
>>         logging.debug("Could not fetch KVM userspace version")
>>     logging.debug("KVM userspace version: %s" % kvm_userspace_version)
>>
>> _______________________________________________
>> Autotest mailing list
>> Autotest@test.kernel.org
>> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>>
>>     
>
>
>
>   

Re: [PATCH 1/3] KVM test: Use customized command to get the version of kvm and its

[Lucas Meneghel Rodrigues]

On Fri, 2010-05-07 at 18:10 +0800, Jason Wang wrote:
> Lucas Meneghel Rodrigues wrote:
> > On Mon, Apr 26, 2010 at 7:07 AM, Jason Wang <jasowang@redhat.com> wrote:
> >   
> >> userspace
> >>
> >> Current method may or may not work for various kinds of
> >> distribution. So this patch enable the ability to use customized
> >> commands to get the version of kvm and its userspace. "kvm_ver_cmd" is
> >> used for kvm verison and "kvm_userspace_ver_cmd" is for its userspace.
> >>     
> >
> > The method we are currently using is pretty satisfactory - if we fail
> > in getting /sys/module/kvm/version we use the kernel version as a
> > fallback, which is good for the kernel module. For qemu, we make a
> > regular expression searching for numbers following the string version,
> > so I don't see a reason on why we should make it configurable. Care to
> > provide an example of a situation where the current method fails?
> >
> >   
> Current method may be not as accurate as we expected.
> In my Fedora box, the output of qemu-kvm -h | head -n 1 is something like:
> QEMU PC emulator version 0.9.1 (kvm-83-maint-snapshot-20090205), 
> Copyright (c) 2003-2008 Fabrice Bellard
> but the rpm -qa may tell more accurate version:
> qemu-kvm-0.11.0-13.fc12.x86_64

The above version of qemu looks like the one shipped in RHEL 5.X, not
Fedora, you might have mistaken the versions. Here is what it looks on:

Fedora 11:

[root@localhost ~]# qemu-kvm -help | head -1
QEMU PC emulator version 0.11.0 (qemu-kvm-0.11.0), Copyright (c) 2003-2008 Fabrice Bellard
[root@localhost ~]# rpm -qa | grep qemu-kvm
qemu-kvm-0.11.0-13.fc12.x86_64

Fedora 13:

[lmr@freedom ~]$ qemu-kvm -h | head -1
QEMU PC emulator version 0.12.3 (qemu-kvm-0.12.3), Copyright (c) 2003-2008 Fabrice Bellard
[lmr@freedom ~]$ rpm -qa | grep qemu-kvm
qemu-kvm-0.12.3-8.fc13.x86_64

Moreover, we deal with several build methods, qemu-kvm might have not be
installed through rpm, so we have to use a single method to figure out
the versions. Another point is that, if we run such alternate methods
(such as git build, or brew build) we will have reliable versioning that
can be extracted from the build logs.

My decision is we keep the current method of determining the version.
The current method we have is fairly reliable (though obviously not
perfect) in my opinion, and it can be applied pretty much for all
branches.

I would really like that we start embedding version control (git)
information somewhere installing binaries, to make things easier for
people bisecting issues, but not sure what the maintainers would think
about this.

Re: [PATCH 1/3] KVM test: Use customized command to get the version of kvm and its

[Jason Wang]

----- "Lucas Meneghel Rodrigues" <lmr@redhat.com> wrote:

> On Fri, 2010-05-07 at 18:10 +0800, Jason Wang wrote:
> > Lucas Meneghel Rodrigues wrote:
> > > On Mon, Apr 26, 2010 at 7:07 AM, Jason Wang <jasowang@redhat.com>
> wrote:
> > >   
> > >> userspace
> > >>
> > >> Current method may or may not work for various kinds of
> > >> distribution. So this patch enable the ability to use customized
> > >> commands to get the version of kvm and its userspace.
> "kvm_ver_cmd" is
> > >> used for kvm verison and "kvm_userspace_ver_cmd" is for its
> userspace.
> > >>     
> > >
> > > The method we are currently using is pretty satisfactory - if we
> fail
> > > in getting /sys/module/kvm/version we use the kernel version as a
> > > fallback, which is good for the kernel module. For qemu, we make
> a
> > > regular expression searching for numbers following the string
> version,
> > > so I don't see a reason on why we should make it configurable.
> Care to
> > > provide an example of a situation where the current method fails?
> > >
> > >   
> > Current method may be not as accurate as we expected.
> > In my Fedora box, the output of qemu-kvm -h | head -n 1 is something
> like:
> > QEMU PC emulator version 0.9.1 (kvm-83-maint-snapshot-20090205), 
> > Copyright (c) 2003-2008 Fabrice Bellard
> > but the rpm -qa may tell more accurate version:
> > qemu-kvm-0.11.0-13.fc12.x86_64
> 
> The above version of qemu looks like the one shipped in RHEL 5.X, not
> Fedora, you might have mistaken the versions. Here is what it looks
> on:
> 
> Fedora 11:
> 
> [root@localhost ~]# qemu-kvm -help | head -1
> QEMU PC emulator version 0.11.0 (qemu-kvm-0.11.0), Copyright (c)
> 2003-2008 Fabrice Bellard
> [root@localhost ~]# rpm -qa | grep qemu-kvm
> qemu-kvm-0.11.0-13.fc12.x86_64
> 
> Fedora 13:
> 
> [lmr@freedom ~]$ qemu-kvm -h | head -1
> QEMU PC emulator version 0.12.3 (qemu-kvm-0.12.3), Copyright (c)
> 2003-2008 Fabrice Bellard
> [lmr@freedom ~]$ rpm -qa | grep qemu-kvm
> qemu-kvm-0.12.3-8.fc13.x86_64
> 
> Moreover, we deal with several build methods, qemu-kvm might have not
> be
> installed through rpm, so we have to use a single method to figure
> out
> the versions. Another point is that, if we run such alternate methods
> (such as git build, or brew build) we will have reliable versioning
> that
> can be extracted from the build logs.
> 
> My decision is we keep the current method of determining the version.
> The current method we have is fairly reliable (though obviously not
> perfect) in my opinion, and it can be applied pretty much for all
> branches.
> 
> I would really like that we start embedding version control (git)
> information somewhere installing binaries, to make things easier for
> people bisecting issues, but not sure what the maintainers would
> think
> about this.
> 
Thanks for the detailed explanation here, I think I've missed the possibility of building form source.
I agree that we keep the current method.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html