Hi everyone,

This is table for lsb test result and we give a preliminary plan for lsb test under table.

Target Test Classification Test Items Original lsb-image
(The image without adding anything) Add lsb package
   Add perl-module-*
               Add eglibc-utils,localedef Remark

Run Test Result Run Test Result

X86 Command Tests Command Check
4.0.0-2 F No T Test Cases Total: 150
Test Passed: 139
Failures: 11
Test Waived: 8 The reason for Failures is that some commands are absent in lsb-image,for example, sendmail. We can add these packages including absent commands to lsb-image

Static Interface Tests Library Check
4.0.2-1 F No T Test Cases Total: 2947
Test Passed: 1557
Test Failed: 1390 The reasons for Failures are below:
1 Some libraries are absent in lsb-image
2 Some functions of libraries are absent

Runtime Interface Tests Core Tests 4.0.5-1 F No F No Add pam-plugin-*, e2fsprogs-mke2fs, eglibc-localedata-posix

Run test Result

T Test Cases Total  10593
Tests Passed  4100
Failures  1782
Warnings  4
Tests Waived  47
FIPs  19

Cpp-T2C Tests
4.0.2-1 F No T Test Cases Total: 1854
Test Passed: 1589
Failures: 265
Test Waived: 144 The reason for Failures is that some file could be absent in lsb test suite  or perhaps some other resons. We need to ensure this probleme later.

Desktop Test
4.0.4-2 F No F No Don't install lsb test suite for it

Desktop-T2C Test
4.0.2-1 F No T Test Cases Total: 5132
Test Passed: 4964
Failures: 168
Test Waived: 49 We need ensure the reason later

OLVER Core Tests
4.0.1-1 F No T/M
F/G No Absent some resources needed by lsb test suite

Perl Tests 4.0.7-1 F No T Test Cases Total  853
Tests Passed  843
Failures  10 Add perl-pod, perl-misc, perl-doc
Modify perl package(add a patch)

Run test Result

T Tests Total  853
Tests Passed  853

Printing Tests
4.0.4-3 F No F No Absent some resources needed by lsb test suite

Python Running Tests
4.0.2-1 F No F No Absent some resources needed by lsb test suite

Qt3-Azov Tests
4.0.2-1 F No F No Absent some resources needed by lsb test suite

Qt4-Azov Tests
4.0.2-1 F No F No Absent some resources needed by lsb test suite

Xts5 Tests
5.1.5-19 F No F No Absent some resources needed by lsb test suite

Automated Application Battery Tests Apache Tests
2.2.8-2 - - T Passed

Expect Tests
5.43.0-7 - - T Passed

Groff Tests
1.19.2-4 - - T Passed

Python Tests
2.4.5-2 F No T Test Cases Total  291
Tests Passed  250
Failures  4
Warnings  1

Raptor Tests
1.4.16-2 - - T Passed

Rsync Tests
3.0.0-2 - - T Passed

Samba Tests
3.0.28a-3 - - T -

Tcl Tests
8.5.1-2 - - T Passed

Xpdf Tests
1.01-7 - - T Passed

X86-64 Command Tests Command Check
4.0.0-2 F No T Test Cases Total: 150
Test Passed: 139
Failures: 11
Test Waived: 8 AS X86

Static Interface Tests Library Check
4.0.2-1 F No T Test Cases Total: 2947
Test Passed: 1557
Test Failed: 1390 AS X86

Runtime Interface Tests Core Tests 4.0.5-1 F No F No We will add some packages as x86 to lsb-image and test it.

Cpp-T2C Tests
4.0.2-1 F No T Test Cases Total: 1854
Test Passed: 1589
Failures: 265
Test Waived: 144 AS X86

Desktop Test
4.0.4-2 F No F No Absent some resources needed by lsb test suite

Desktop-T2C Test
4.0.2-1 F No T Test Cases Total: 5132
Test Passed: 4963
Failures: 169
Test Waived: 49 AS X86

OLVER Core Tests
4.0.1-1 F No T/M
F/G No Absent some resources needed by lsb test suite

Perl Tests 4.0.7-1 F No T Test Cases Total  853
Tests Passed  18
Failures  17 We will add some packages as x86 to lsb-image and test it.

Printing Tests
4.0.4-3 F No F Test Cases Excepted:97
Tests Passed  3
Failures  21
Test suite error:1 Absent cups libraries

Python Running Tests
4.0.2-1 F No F No Absent some resources needed by lsb test suite

Qt3-Azov Tests
4.0.2-1 F No F No Absent some resources needed by lsb test suite

Qt4-Azov Tests
4.0.2-1 F No F No Absent some resources needed by lsb test suite

Xts5 Tests
5.1.5-19 F No F No Absent some resources needed by lsb test suite

Automated Application Battery Tests Apache Tests
2.2.8-2 - - - - Don't install lsb test suite for it

Expect Tests
5.43.0-7 - - - - Don't install lsb test suite for it

Groff Tests
1.19.2-4 - - - - Don't install lsb test suite for it

Python Tests
2.4.5-2 F No T No Absent some resources needed by lsb test suite

Raptor Tests
1.4.16-2 - - - - Don't install lsb test suite for it

Rsync Tests
3.0.0-2 - - - - Don't install lsb test suite for it

Samba Tests
3.0.28a-3 - - - - Don't install lsb test suite for it

Tcl Tests
8.5.1-2 - - - - Don't install lsb test suite for it

Xpdf Tests
1.01-7 - - - - Don't install lsb test suite for it

Note:
Run test:
T: LSB test suite can run completely.
F: There are some issues that block LSB test suite itself running completely.
T/M: LSB can be tested with manual method.
T/G: LSB can be tested with graphic method.
Result:
No: There is no result because LSB test suite cannot run.

It is a preliminary analysis for lsb test. We will give more information later(include ppc).
A plan about how to fix bugs:
step 1: We will solve bugs which some of lsb test suite can't still run on lsb-image (The work is now in progress)
step 2: Make two auto script for setting up lsb test environment. One is for setting on pc , we want to add this script to poky/scripts; the other is for setting for lsb-image, we want to add this script to lsb-image in the mode of bb file. (sync up with step 1)
step 3: Write a user manual about how to test lsb-imgae by lsb test suite and we want to this user manual to lsb-image (after step 2)
step 4: List bugs tested by lsb test suite(sync up with step 1)
step 5: Solve bugs tested by lsb test suite(everyone can attend to fix bugs)
Method for fixing bugs:
- We will fix bugs for lsb test suite itself on platform x86 firstly and then apply the result to other platform when step 1 is over.
- We will fix bugs tested by lsb test suite on platform x86 and then apply the result to other platform.

We will open a bugzilla for failed items tested by lsb test suite and make a milestone for fixing bugs with detailed time plan on the next week.
Please give your suggestion.

Thanks,
Yan

Target	Test Classification	Test Items	Original lsb-image (The image without adding anything)		Add lsb package Add perl-module-* Add eglibc-utils,localedef		Remark
Target	Test Classification	Test Items	Run Test	Result	Run Test	Result	Remark
X86	Command Tests	Command Check 4.0.0-2	F	No	T	Test Cases Total: 150 Test Passed: 139 Failures: 11 Test Waived: 8	The reason for Failures is that some commands are absent in lsb-image,for example, sendmail. We can add these packages including absent commands to lsb-image
	Static Interface Tests	Library Check 4.0.2-1	F	No	T	Test Cases Total: 2947 Test Passed: 1557 Test Failed: 1390	The reasons for Failures are below: 1 Some libraries are absent in lsb-image 2 Some functions of libraries are absent
	Runtime Interface Tests	Core Tests 4.0.5-1	F	No	F	No	Add pam-plugin-*, e2fsprogs-mke2fs, eglibc-localedata-posix
							Run test	Result
							T	Test Cases Total 10593 Tests Passed 4100 Failures 1782 Warnings 4 Tests Waived 47 FIPs 19
		Cpp-T2C Tests 4.0.2-1	F	No	T	Test Cases Total: 1854 Test Passed: 1589 Failures: 265 Test Waived: 144	The reason for Failures is that some file could be absent in lsb test suite or perhaps some other resons. We need to ensure this probleme later.
		Desktop Test 4.0.4-2	F	No	F	No	Don't install lsb test suite for it
		Desktop-T2C Test 4.0.2-1	F	No	T	Test Cases Total: 5132 Test Passed: 4964 Failures: 168 Test Waived: 49	We need ensure the reason later
		OLVER Core Tests 4.0.1-1	F	No	T/M F/G	No	Absent some resources needed by lsb test suite
		Perl Tests 4.0.7-1	F	No	T	Test Cases Total 853 Tests Passed 843 Failures 10	Add perl-pod, perl-misc, perl-doc Modify perl package(add a patch)
							Run test	Result
							T	Tests Total 853 Tests Passed 853
		Printing Tests 4.0.4-3	F	No	F	No	Absent some resources needed by lsb test suite
		Python Running Tests 4.0.2-1	F	No	F	No	Absent some resources needed by lsb test suite
		Qt3-Azov Tests 4.0.2-1	F	No	F	No	Absent some resources needed by lsb test suite
		Qt4-Azov Tests 4.0.2-1	F	No	F	No	Absent some resources needed by lsb test suite
		Xts5 Tests 5.1.5-19	F	No	F	No	Absent some resources needed by lsb test suite
	Automated Application Battery Tests	Apache Tests 2.2.8-2	-	-	T	Passed
		Expect Tests 5.43.0-7	-	-	T	Passed
		Groff Tests 1.19.2-4	-	-	T	Passed
		Python Tests 2.4.5-2	F	No	T	Test Cases Total 291 Tests Passed 250 Failures 4 Warnings 1
		Raptor Tests 1.4.16-2	-	-	T	Passed
		Rsync Tests 3.0.0-2	-	-	T	Passed
		Samba Tests 3.0.28a-3	-	-	T	-
		Tcl Tests 8.5.1-2	-	-	T	Passed
		Xpdf Tests 1.01-7	-	-	T	Passed
X86-64	Command Tests	Command Check 4.0.0-2	F	No	T	Test Cases Total: 150 Test Passed: 139 Failures: 11 Test Waived: 8	AS X86
	Static Interface Tests	Library Check 4.0.2-1	F	No	T	Test Cases Total: 2947 Test Passed: 1557 Test Failed: 1390	AS X86
	Runtime Interface Tests	Core Tests 4.0.5-1	F	No	F	No	We will add some packages as x86 to lsb-image and test it.


		Cpp-T2C Tests 4.0.2-1	F	No	T	Test Cases Total: 1854 Test Passed: 1589 Failures: 265 Test Waived: 144	AS X86
		Desktop Test 4.0.4-2	F	No	F	No	Absent some resources needed by lsb test suite
		Desktop-T2C Test 4.0.2-1	F	No	T	Test Cases Total: 5132 Test Passed: 4963 Failures: 169 Test Waived: 49	AS X86
		OLVER Core Tests 4.0.1-1	F	No	T/M F/G	No	Absent some resources needed by lsb test suite
		Perl Tests 4.0.7-1	F	No	T	Test Cases Total 853 Tests Passed 18 Failures 17	We will add some packages as x86 to lsb-image and test it.


		Printing Tests 4.0.4-3	F	No	F	Test Cases Excepted:97 Tests Passed 3 Failures 21 Test suite error:1	Absent cups libraries
		Python Running Tests 4.0.2-1	F	No	F	No	Absent some resources needed by lsb test suite
		Qt3-Azov Tests 4.0.2-1	F	No	F	No	Absent some resources needed by lsb test suite
		Qt4-Azov Tests 4.0.2-1	F	No	F	No	Absent some resources needed by lsb test suite
		Xts5 Tests 5.1.5-19	F	No	F	No	Absent some resources needed by lsb test suite
	Automated Application Battery Tests	Apache Tests 2.2.8-2	-	-	-	-	Don't install lsb test suite for it
		Expect Tests 5.43.0-7	-	-	-	-	Don't install lsb test suite for it
		Groff Tests 1.19.2-4	-	-	-	-	Don't install lsb test suite for it
		Python Tests 2.4.5-2	F	No	T	No	Absent some resources needed by lsb test suite
		Raptor Tests 1.4.16-2	-	-	-	-	Don't install lsb test suite for it
		Rsync Tests 3.0.0-2	-	-	-	-	Don't install lsb test suite for it
		Samba Tests 3.0.28a-3	-	-	-	-	Don't install lsb test suite for it
		Tcl Tests 8.5.1-2	-	-	-	-	Don't install lsb test suite for it
		Xpdf Tests 1.01-7	-	-	-	-	Don't install lsb test suite for it
	Note: Run test: T: LSB test suite can run completely. F: There are some issues that block LSB test suite itself running completely. T/M: LSB can be tested with manual method. T/G: LSB can be tested with graphic method. Result: No: There is no result because LSB test suite cannot run.