Testing Report for Xen 3.0.3 rc1

Testing Report for Xen 3.0.3 rc1

[Xu, JiajunX]

We have tested Xen for 3.0.3rc1.
The test includes Guest installation, device model, control panel, guest
os and basic function tests.

Here are issues found in our tests, For the first 6 issues, I think it's
better to be fixed in 3.0.3:
1. Para-virtualization driver doen't work for HVM guests

The hvm guest always hanged or met kernel panic when trying to load
xenbus.ko. So vbd and vnif don't work on IA32 and IA32e HV.

2. In ia32e xen0, xm vcpu-set <Domain> <vCPUs> command doesn't work.	
The command "xm vcpu-set <Domain> <vCPUs>', like "xm vcpu-set 0 4" in
xen0 which have 16 vCPUs, won't pause the vcpus as expected. The issue
isn't found in unstable tree. So it maybe caused by some patches checked
in testing tree.

3. Win2k installation with ACPI enabled will give a blue screen.

It also exists in latest unstable tree. Xin also met the issue. As we
begin add more combination Windows tests, it's important to make it
work.

4. FC5 IA32 VMX Guest crash when booting SMP Guest

Create FC5 Guest and select SMP kernel, it will crash Guest then. It
will occur on both IA32p/IA32e host. 

5. Booting VMX with vcpus=16, but in guest it shows only 8 CPUs

We found the issue on Paxville, which machine has 16 LP. Setting
vcpus=16 in configuration file and then starting VMX.
When VMX starts up, only 8 cpus found in guest OS through /proc/cpuinfo,
and "xm vcpu-list" shows 
that 8 vcpu are in status "r" and 8 are in status of "p".

6. Using vi in VMX hangs guest

It's an old issue which always makes our work hard. But I think it
should be fixed in 3.0.3 since VI is very popular command, isn't it?

7. Create several VMX domains and XENU domains coexist issue seems have
been resolved. But, if under an extremely condition, like creating 12
VMX domains and 4 XENU domains coexist, sometimes we still may meet this
problem. 
We suffer by the issue for a long time. Though in cs 11554, there is a
fix for it. But we still could reproduce it. Also the fix patch may make
windows guest very slow response.

8. Can not input number from speail number keys (when Open Num Lock)	
It's also a user experience issue, sometimes the keyboard will work and
sometimes not.

9. IA32 PAE SMP guest OS can't reboot in IA32 PAE and IA32E xen0
The issue should be fixed in unstable C/S 11620.

10. LTP case "gettimeofday02", "ioperm02" and "pselect01" may fail in
VMX
"gettimeofday02" shows that:
====
gettimeofday02    0  INFO  :  checking if gettimeofday is monotonous,
takes 30s
gettimeofday02    1  FAIL  :  Time is going backwards (old
1134716003.371061 vs new 1134716003.371049!
===
"ioperm02" shows that:
ioperm02    1  FAIL  :  Unexpected results for Invalid I/O address ;
returned 
0 (expected -1), errno 0 (expected errno  22)
ioperm02    2  PASS  :  Expected failure for Non super-user, errno: 1
"pselect01" shows that:
<<<test_output>>>
incrementing stop
pselect01    1  PASS  :  Basic pselect syscall testing....OK
pselect01    0  INFO  :  Testing basic pselect sanity,Sleeping for 1
secs
pselect01    2  PASS  :  Sleep time was correct
pselect01    0  INFO  :  Testing basic pselect sanity,Sleeping for 2
secs
pselect01    3  PASS  :  Sleep time was correct
pselect01    0  INFO  :  Testing basic pselect sanity,Sleeping for 3
secs
pselect01    4  PASS  :  Sleep time was correct
pselect01    0  INFO  :  Testing basic pselect sanity,Sleeping for 4
secs
pselect01    5  PASS  :  Sleep time was correct
pselect01    0  INFO  :  Testing basic pselect sanity,Sleeping for
1000000 
micro secs
pselect01    6  PASS  :  Sleep time was correct
pselect01    0  INFO  :  Testing basic pselect sanity,Sleeping for
2000000 
micro secs
pselect01    7  FAIL  :  Sleep time was incorrect:5 != 6
pselect01    0  INFO  :  Testing basic pselect sanity,Sleeping for
3000000 
micro secs
pselect01    8  PASS  :  Sleep time was correct
pselect01    0  INFO  :  Testing basic pselect sanity,Sleeping for
4000000 
micro secs
pselect01    9  PASS  :  Sleep time was correct
<<<execution_status>>>
duration=31 termination_type=exited termination_id=1 corefile=no
cutime=0 cstime=0
<<<test_end>>>

11. X11perf fail in VMX
X11perf will fail by printing these messages:
8000000 reps @   0.0007 msec (1470000.0/sec): Fill 1x1 tiled trapezoid
(216x208 tile)
300000 reps @   0.0220 msec ( 45500.0/sec): Fill 10x10 tiled trapezoid
(216x208 tile)
180000 reps @   0.0329 msec ( 30400.0/sec): Fill 100x100 tiled trapezoid
(216x208 tile)
40000 reps @   0.1879 msec (  5320.0/sec): Fill 300x300 tiled trapezoid
(216x208 tile)
X Error of failed request:  BadValue (integer parameter out of range for
operation)
  Major opcode of failed request:  53 (X_CreatePixmap)
  Value in failed request:  0x8
  Serial number of failed request:  167370704
  Current serial number in output stream:  167370706

12. Can't set acoustic on VMX IDE

13. Cannot send big ICMP package from xen0 to vmx


Detail Info:
HVM Guest Installations:
IA32 host:
	Windows IA32 XP SP2 Guest Installation w/ ACPI
PASS
	Windows IA32 XP SP2 Guest Installation w/o ACPI
PASS
	Windows IA32 2k3 Guest Installation w/ ACPI
PASS
	Windows IA32 2k3 Guest Installation w/o ACPI
PASS
	Windows IA32 2000 Guest Installation w/ ACPI
FAIL
	Windows IA32 2000 Guest Installation w/o ACPI
PASS
	IA32 RHEL4u3 Guest Installation
PASS
IA32-PAE host:
	Windows IA32 XP SP2 Guest Installation w/ ACPI
PASS
	Windows IA32 XP SP2 Guest Installation w/o ACPI
PASS
	Windows IA32 2k3 Guest Installation w/ ACPI
PASS
	Windows IA32 2k3 Guest Installation w/o ACPI
PASS
	Windows IA32 2000 Guest Installation w/ ACPI
FAIL
	Windows IA32 2000 Guest Installation w/o ACPI
PASS
	IA32 RHEL4u3 Guest Installation
PASS
IA32e host:
	Windows IA32 XP SP2 Guest Installation w/ ACPI
PASS
	Windows IA32 XP SP2 Guest Installation w/o ACPI
PASS
	Windows IA32 2k3 Guest Installation w/ ACPI
PASS
	Windows IA32 2k3 Guest Installation w/o ACPI
PASS
	Windows IA32 2000 Guest Installation w/ ACPI
FAIL
	Windows IA32 2000 Guest Installation w/o ACPI
PASS
	IA32e RHEL4u3 Guest Installation
PASS

Booting Up Test:
	Boot up IA32 SMP FC5
FAIL
	Boot up IA32e FC5
PASS
	BOot up IA32/IA32e RHEL4u3
PASS
	Boot up IA32/IA32e RHEL3u5
PASS
	Boot up IA32 FC3
PASS
	Boot up IA32e FC3
FAIL
	Boot up IA32/IA32e RHEL4u1
PASS
	Boot up IA32/IA32e FC6-test3
PASS

Basic Functional test: (The tests are based on SMP Guest)
IA32 host:
	1. one IA32 nonpae SMP VMX domain with memory 512M
PASS
	2. one xenU domain with memory 256M
PASS
	3. Two IA32 nonpae SMP VMX domains and 2 xenU domains coexist
PASS
	4. four IA32 nonpae SMP VMX domains coexist(64M,128M,196M,256M)
PASS
	5. network in IA32 nonpae SMP VMX domain
PASS
	6. network in XenU domain
PASS
	7. one IA32 UP Windows 2003 in VMX domain
PASS
	8. 1 IA32 UP xpsp1 and 1 IA32 UP xpsp2 Windows VMX domains
coexist 	PASS
	9. one VMX boot with IA32 nonpae SMP base kernel(2.6.16)
PASS
	10.xm-test
PASS
	11.StartX in Dom0
PASS
	12.one IA32 nonpae SMP VMX domain with acpi enable
PASS
	13.Build base kernel in IA32 nonpae SMP VMX domain
PASS
IA32-PAE host:
	1. one IA32 nonpae SMP 2.6 VMX domain with memory 256M
PASS
	2. one PAE SMP VMX domain with memory 512M
PASS
	3. one PAE SMP VMX domain with memory 1500M
PASS
	4. one xenU domain with memory 256M
PASS
	5. 2 PAE SMP VMX domains and 2 xenU domains coexist
PASS
	6. 2 IA32 nonpae SMP VMX domains and 2 PAE SMP VMX domains
coexist(64m,128m,196m,256m)
	
PASS
	7. network in PAE SMP vmx domain
PASS
	8. network in xenu domain
PASS
	9. one PAE UP Windows 2003 vmx domain
PASS
	10.1 IA32 UP xpsp1 and 1 IA32 UP xpsp2 windows vmx domains
coexist  	PASS
	11.xm-test
PASS
	12.one vmx boot with PAE SMP base kernel(2.6.16)
PASS
	13.startx in dom0
PASS
	14.one PAE SMP vmx domain with acpi enable
PASS
	15.build linux 2.6.16 base kernel in PAE SMP vmx domain
PASS	
IA32e host:
	1. one IA32 nonpae SMP 2.6 VMX domain with memory 256M
PASS
	2. one IA32E SMP VMX domain with memory 512M
PASS
	3. one xenU domain with memory 256M
PASS
	4. 2 IA32E SMP VMX domains and 2 xenU VMX domains coexist
PASS
	5. 2 IA32E SMP VMX domains and 2 PAE SMP VMX domains
coexist(64m,128m,196m,256m)	
	
PASS
	6. network in IA32E SMP VMX domain
PASS
	7. network in XenU domain
PASS
	8. one PAE SMP VMX domain with 1500M
PASS
	9. one PAE UP Windows 2003 VMX domain
PASS
	10.1 IA32 UP xpsp1 and 1 IA32 UP xpsp2 Windows domains coexist
PASS
	11.xm-test
PASS
	12.one VMX boot with IA32E SMP base kernel(2.6.16)
PASS
	13.Build base kernel in IA32E VMX domain
PASS
	14.StartX in Dom0
PASS
	15.one IA32E SMP VMX domain with acpi enable
PASS

Device Model Test:
	1. VMX Timer is accurate
	2. VMX Timer can be set
	3. VMX CPU info is accurate
	4. VMX IDE testing is pass
	5. VMX can support 4 IDE disks
	6. VMX network testing pass
	7. VMX can transfer 1G file successfully by scp
	8. 2 VMX network transfer is okay
	9. VI still will hang VMX
	10.Can not input number from speail number keys (when Open Num
Lock)
	11.VMX/Xen0 Xwindows is OK
	12.4 NIC Devices (4 rtl8139 and 4 ne2k)
	13.Qemu command in Qemu console Window is OK
	14.USB mouse is OK to use
	15.SDL and VNC mode is OK

Control Panel Test:
The failed cases include:
	1. setting vcpus=16, but VMX guest shows only 8 CPUs
	2. Assign one VMX with one more vcpus than LP number of host
will not boot up guest.
	3. Assign one VMX with 0 vcpu will not boot up guest.
The passed cases include:
1. xm console pass in all conditions.
  	(1) xm console pass when create one xenU
 	(2) xm console pass when create one VMX
  	(3) xm console pass when create two VMXs.
  	(4) xm console pass when create two xenUs.
  	(5) xm console pass when create one VMX and one xenU.
2. xm vcpu-pin works in all conditions.
  	(1) xm vcpu-pin works for one VMX domain
  	(2) pin 1 cpu to 2 VMX domains.
  	(3) pin 2 cpus to 1 VMX domain.
  	(4) pin 2 cpus to 2 domU domains.
  	(5) pin 8 cpus to 1 VMX domain
3. Check if assigning specified number of vcpus to domains works.
  	(1) assign 2 vcpus to a VMX guest
  	(2) assign 2 vcpus to a xenU guest
  	(3) assign 4 vcpus to a VMX guest
  	(4) assign 4 vcpus to a xenU guest 
  	(5) assign several number of vcpus to each VMX guest when
several ones coexist.
  	(6) assign several number of vcpus to each xenU guest when
several ones coexist.
  	(7) assign several number of vcpus to each VMX and xenU guest
when several ones coexist.
4. save and restore for xenU manual test.

Gtest OS test:
	Cerberus		PASS 
	Kernel build 		PASS
	LTP			4 cases may fail 
	Dbench			PASS
	x11perf			FAIL
	Hammerhead		PASS
	Cerberus		PASS

Best Regards,
Jiajun 

Re: Testing Report for Xen 3.0.3 rc1

[Steven Smith]

[-- Attachment #1.1: Type: text/plain, Size: 1532 bytes --]

> Here are issues found in our tests, For the first 6 issues, I think it's
> better to be fixed in 3.0.3:
> 1. Para-virtualization driver doen't work for HVM guests
> 
> The hvm guest always hanged or met kernel panic when trying to load
> xenbus.ko. So vbd and vnif don't work on IA32 and IA32e HV.
Should be fixed by cset 11672 on unstable.

> 5. Booting VMX with vcpus=16, but in guest it shows only 8 CPUs
> 
> We found the issue on Paxville, which machine has 16 LP. Setting
> vcpus=16 in configuration file and then starting VMX.
> When VMX starts up, only 8 cpus found in guest OS through /proc/cpuinfo,
> and "xm vcpu-list" shows 
> that 8 vcpu are in status "r" and 8 are in status of "p".
Linux, by default, will only use the first 8 processors in the system.
What's NR_CPUS in your kernel config?

> 6. Using vi in VMX hangs guest
> 
> It's an old issue which always makes our work hard. But I think it
> should be fixed in 3.0.3 since VI is very popular command, isn't it?
That's not good.  I'm optimistic that it was fixed by csets 11622 and
11624 of xen-unstable, but I've never been able to reproduce this
reliably enough to test.  Could you have another go, please?

> 8. Can not input number from speail number keys (when Open Num Lock)	
> It's also a user experience issue, sometimes the keyboard will work and
> sometimes not.
Is this HVM guests using the vnc viewer?

> 12. Can't set acoustic on VMX IDE
Acoustic as in noise control, yes?  Why would you ever need to?

Steven.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Testing Report for Xen 3.0.3 rc1

[John Lenz]

On 09/29/06 02:51, Xu, JiajunX wrote:
> 
> 3. Win2k installation with ACPI enabled will give a blue screen.
> 
> It also exists in latest unstable tree. Xin also met the issue. As we
> begin add more combination Windows tests, it's important to make it
> work.
> 

The latest ReactOS 0.3.0 in a hvm domain also bluescreens during the
first boot.  The install from the CD works fine, but after rebooting
from the hard drive, bluescreen shows up with a message about
bootvid.sys  KMODE_EXCEPTION_NOT_HANDLED.  I have a screenshot if anyone
is interested (14K).  This is with ACPI=0, so is probably unrelated.

This is on a AMD Athlon 64 X2 with the latest xen 3.0.3 rc1 binary
release (I didn't compile it myself).


Other problems I had:
1) Installing Windows 2000 blue screened, but after installing with
qemu, xen boots a domU into Windows 2000 fine

2) Upgrading WinXP from the Windows 2000 again bluscreened during the
install, but after installing with qemu, the WinXP domain boots up fine.

3) The mouse in windows "feels" a lot slower than the old xen (3.0.2).
I don't know if this is related to bvt vs credit or what, and I don't
have any measures or anything, but mouse movement in both SDL and VNC
seem slower to update the mouse position when you move it.  This is with
an idle dom0, and just WinXP in the domU, and both show no CPU use.

John

RE: Testing Report for Xen 3.0.3 rc1

[Xu, JiajunX]

Thanks for your comments, I have verified these issues and see my comments below.

Best Regards,
Jiajun 

>-----Original Message-----
>From: Steven Smith [mailto:sos22@hermes.cam.ac.uk] On Behalf Of Steven Smith
>Sent: 2006年9月29日 22:19
>To: Xu, JiajunX
>Cc: xen-devel@lists.xensource.com; sos22@srcf.ucam.org
>Subject: Re: [Xen-devel] Testing Report for Xen 3.0.3 rc1
>
>> Here are issues found in our tests, For the first 6 issues, I think it's
>> better to be fixed in 3.0.3:
>> 1. Para-virtualization driver doen't work for HVM guests
>>
>> The hvm guest always hanged or met kernel panic when trying to load
>> xenbus.ko. So vbd and vnif don't work on IA32 and IA32e HV.
>Should be fixed by cset 11672 on unstable.
Yes, it should be fixed in the latest cset. We tested cset 11678 and it works now.

>> 5. Booting VMX with vcpus=16, but in guest it shows only 8 CPUs
>>
>> We found the issue on Paxville, which machine has 16 LP. Setting
>> vcpus=16 in configuration file and then starting VMX.
>> When VMX starts up, only 8 cpus found in guest OS through /proc/cpuinfo,
>> and "xm vcpu-list" shows
>> that 8 vcpu are in status "r" and 8 are in status of "p".
>Linux, by default, will only use the first 8 processors in the system.
>What's NR_CPUS in your kernel config?
After set NR_CPUS to 16, we can boot up 16 vcpus in Guest. It's due to our env mistake. Thanks for your hints.

>> 6. Using vi in VMX hangs guest
>>
>> It's an old issue which always makes our work hard. But I think it
>> should be fixed in 3.0.3 since VI is very popular command, isn't it?
>That's not good.  I'm optimistic that it was fixed by csets 11622 and
>11624 of xen-unstable, but I've never been able to reproduce this
>reliably enough to test.  Could you have another go, please?
I have tried cset 11624 and using vi still will make Guest hang. If you work in text mode and use vi, it's easy to reproduce it.

>> 8. Can not input number from speail number keys (when Open Num Lock)
>
>> It's also a user experience issue, sometimes the keyboard will work and
>> sometimes not.
>Is this HVM guests using the vnc viewer?
The issue exists in both vnc and sdl mode. We found it in Windows Guest and Xwindows mode in Linux Guest.

>> 12. Can't set acoustic on VMX IDE
>Acoustic as in noise control, yes?  Why would you ever need to?
Yes. It is an IDE hard disk feature. We can set it in Native Linux. But can not in VMX domain. So we report it, although it is harmless. 

>Steven.

RE: Testing Report for Xen 3.0.3 rc1

[Ian Pratt]

> >> 6. Using vi in VMX hangs guest
> >>
> >> It's an old issue which always makes our work hard. But I think it
> >> should be fixed in 3.0.3 since VI is very popular command, isn't
it?
> >That's not good.  I'm optimistic that it was fixed by csets 11622 and
> >11624 of xen-unstable, but I've never been able to reproduce this
> >reliably enough to test.  Could you have another go, please?
> I have tried cset 11624 and using vi still will make Guest hang. If
you
> work in text mode and use vi, it's easy to reproduce it.

When the guest hangs, what does 'top' and 'xm list' in dom0 show is
happening? At what rate are qemu-dm and the guest consuming CPU?

Does disconnecting and reconnecting the VNC session make any difference?

Please tell us more about your setup. Is this an SMP, PAE linux guest?
Which vnc client are you using?

Thanks,
Ian

Re: Testing Report for Xen 3.0.3 rc1

[Steven Smith]

[-- Attachment #1.1: Type: text/plain, Size: 612 bytes --]


> >> 6. Using vi in VMX hangs guest
> >>
> >> It's an old issue which always makes our work hard. But I think it
> >> should be fixed in 3.0.3 since VI is very popular command, isn't it?
> >That's not good.  I'm optimistic that it was fixed by csets 11622 and
> >11624 of xen-unstable, but I've never been able to reproduce this
> >reliably enough to test.  Could you have another go, please?
> I have tried cset 11624 and using vi still will make Guest hang. If
> you work in text mode and use vi, it's easy to reproduce it.
Okay, would you mind trying cset 11684, please?  That should have
fixed it.

Steven.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel