From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.windriver.com (mail.windriver.com [147.11.1.11]) by mx1.pokylinux.org (Postfix) with ESMTP id CD0FB4C80287 for ; Wed, 3 Nov 2010 04:54:14 -0500 (CDT) Received: from ALA-MAIL03.corp.ad.wrs.com (ala-mail03 [147.11.57.144]) by mail.windriver.com (8.14.3/8.14.3) with ESMTP id oA39sDvk008370 for ; Wed, 3 Nov 2010 02:54:14 -0700 (PDT) Received: from ala-mail06.corp.ad.wrs.com ([147.11.57.147]) by ALA-MAIL03.corp.ad.wrs.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 3 Nov 2010 02:54:12 -0700 Received: from [128.224.163.183] ([128.224.163.183]) by ala-mail06.corp.ad.wrs.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 3 Nov 2010 02:54:11 -0700 Message-ID: <4CD1307D.7010506@windriver.com> Date: Wed, 03 Nov 2010 17:50:53 +0800 From: Xiaofeng Yan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.11) Gecko/20100713 Thunderbird/3.0.6 MIME-Version: 1.0 To: poky@pokylinux.org X-OriginalArrivalTime: 03 Nov 2010 09:54:11.0765 (UTC) FILETIME=[10F11E50:01CB7B3D] X-Mailman-Approved-At: Wed, 03 Nov 2010 10:42:51 -0500 Subject: [LSB]About lsb test result and plan X-BeenThere: poky@pokylinux.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Poky build system developer discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Nov 2010 09:54:15 -0000 Content-Type: multipart/alternative; boundary="------------050406040105020708070605" --------------050406040105020708070605 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi everyone, This is table for lsb test result and we give a preliminary plan for lsb test under table. > Target Test Classification Test Items Original lsb-image > (The image without adding anything) Add lsb package > Add perl-module-* > Add eglibc-utils,localedef Remark > Run Test Result Run Test Result > X86 Command Tests Command Check > 4.0.0-2 F No T Test Cases Total: 150 > Test Passed: 139 > Failures: 11 > Test Waived: 8 The reason for Failures is that some commands are > absent in lsb-image,for example, sendmail. We can add these packages > including absent commands to lsb-image > Static Interface Tests Library Check > 4.0.2-1 F No T Test Cases Total: 2947 > Test Passed: 1557 > Test Failed: 1390 The reasons for Failures are below: > 1 Some libraries are absent in lsb-image > 2 Some functions of libraries are absent > Runtime Interface Tests Core Tests 4.0.5-1 F No F No Add > pam-plugin-*, e2fsprogs-mke2fs, eglibc-localedata-posix > Run test Result > T Test Cases Total 10593 > Tests Passed 4100 > Failures 1782 > Warnings 4 > Tests Waived 47 > FIPs 19 > Cpp-T2C Tests > 4.0.2-1 F No T Test Cases Total: 1854 > Test Passed: 1589 > Failures: 265 > Test Waived: 144 The reason for Failures is that some file could be > absent in lsb test suite or perhaps some other resons. We need to > ensure this probleme later. > Desktop Test > 4.0.4-2 F No F No Don't install lsb test suite for it > Desktop-T2C Test > 4.0.2-1 F No T Test Cases Total: 5132 > Test Passed: 4964 > Failures: 168 > Test Waived: 49 We need ensure the reason later > OLVER Core Tests > 4.0.1-1 F No T/M > F/G No Absent some resources needed by lsb test suite > Perl Tests 4.0.7-1 F No T Test Cases Total 853 > Tests Passed 843 > Failures 10 Add perl-pod, perl-misc, perl-doc > Modify perl package(add a patch) > Run test Result > T Tests Total 853 > Tests Passed 853 > Printing Tests > 4.0.4-3 F No F No Absent some resources needed by lsb test suite > Python Running Tests > 4.0.2-1 F No F No Absent some resources needed by lsb test suite > Qt3-Azov Tests > 4.0.2-1 F No F No Absent some resources needed by lsb test suite > Qt4-Azov Tests > 4.0.2-1 F No F No Absent some resources needed by lsb test suite > Xts5 Tests > 5.1.5-19 F No F No Absent some resources needed by lsb test suite > Automated Application Battery Tests Apache Tests > 2.2.8-2 - - T Passed > Expect Tests > 5.43.0-7 - - T Passed > Groff Tests > 1.19.2-4 - - T Passed > Python Tests > 2.4.5-2 F No T Test Cases Total 291 > Tests Passed 250 > Failures 4 > Warnings 1 > Raptor Tests > 1.4.16-2 - - T Passed > Rsync Tests > 3.0.0-2 - - T Passed > Samba Tests > 3.0.28a-3 - - T - > Tcl Tests > 8.5.1-2 - - T Passed > Xpdf Tests > 1.01-7 - - T Passed > X86-64 Command Tests Command Check > 4.0.0-2 F No T Test Cases Total: 150 > Test Passed: 139 > Failures: 11 > Test Waived: 8 AS X86 > Static Interface Tests Library Check > 4.0.2-1 F No T Test Cases Total: 2947 > Test Passed: 1557 > Test Failed: 1390 AS X86 > Runtime Interface Tests Core Tests 4.0.5-1 F No F No We will add > some packages as x86 to lsb-image and test it. > Cpp-T2C Tests > 4.0.2-1 F No T Test Cases Total: 1854 > Test Passed: 1589 > Failures: 265 > Test Waived: 144 AS X86 > Desktop Test > 4.0.4-2 F No F No Absent some resources needed by lsb test suite > Desktop-T2C Test > 4.0.2-1 F No T Test Cases Total: 5132 > Test Passed: 4963 > Failures: 169 > Test Waived: 49 AS X86 > OLVER Core Tests > 4.0.1-1 F No T/M > F/G No Absent some resources needed by lsb test suite > Perl Tests 4.0.7-1 F No T Test Cases Total 853 > Tests Passed 18 > Failures 17 We will add some packages as x86 to lsb-image and test it. > Printing Tests > 4.0.4-3 F No F Test Cases Excepted:97 > Tests Passed 3 > Failures 21 > Test suite error:1 Absent cups libraries > Python Running Tests > 4.0.2-1 F No F No Absent some resources needed by lsb test suite > Qt3-Azov Tests > 4.0.2-1 F No F No Absent some resources needed by lsb test suite > Qt4-Azov Tests > 4.0.2-1 F No F No Absent some resources needed by lsb test suite > Xts5 Tests > 5.1.5-19 F No F No Absent some resources needed by lsb test suite > Automated Application Battery Tests Apache Tests > 2.2.8-2 - - - - Don't install lsb test suite for it > Expect Tests > 5.43.0-7 - - - - Don't install lsb test suite for it > Groff Tests > 1.19.2-4 - - - - Don't install lsb test suite for it > Python Tests > 2.4.5-2 F No T No Absent some resources needed by lsb test suite > Raptor Tests > 1.4.16-2 - - - - Don't install lsb test suite for it > Rsync Tests > 3.0.0-2 - - - - Don't install lsb test suite for it > Samba Tests > 3.0.28a-3 - - - - Don't install lsb test suite for it > Tcl Tests > 8.5.1-2 - - - - Don't install lsb test suite for it > Xpdf Tests > 1.01-7 - - - - Don't install lsb test suite for it > > Note: > Run test: > T: LSB test suite can run completely. > F: There are some issues that block LSB test suite itself running > completely. > T/M: LSB can be tested with manual method. > T/G: LSB can be tested with graphic method. > Result: > No: There is no result because LSB test suite cannot run. > > It is a preliminary analysis for lsb test. We will give more information later(include ppc). A plan about how to fix bugs: step 1: We will solve bugs which some of lsb test suite can't still run on lsb-image (The work is now in progress) step 2: Make two auto script for setting up lsb test environment. One is for setting on pc , we want to add this script to poky/scripts; the other is for setting for lsb-image, we want to add this script to lsb-image in the mode of bb file. (sync up with step 1) step 3: Write a user manual about how to test lsb-imgae by lsb test suite and we want to this user manual to lsb-image (after step 2) step 4: List bugs tested by lsb test suite(sync up with step 1) step 5: Solve bugs tested by lsb test suite(everyone can attend to fix bugs) Method for fixing bugs: - We will fix bugs for lsb test suite itself on platform x86 firstly and then apply the result to other platform when step 1 is over. - We will fix bugs tested by lsb test suite on platform x86 and then apply the result to other platform. We will open a bugzilla for failed items tested by lsb test suite and make a milestone for fixing bugs with detailed time plan on the next week. Please give your suggestion. Thanks, Yan --------------050406040105020708070605 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hi everyone,

This is table for lsb test result and we give a preliminary plan for lsb test under table.

Target Test Classification Test Items Original lsb-image
(The image without adding anything)		 Add lsb package 
   Add perl-module-*
               Add eglibc-utils,localedef		 Remark
Run Test Result Run Test  Result
X86 Command Tests Command Check 
4.0.0-2		 F No T Test Cases Total: 150
Test Passed: 139
Failures: 11
Test Waived: 8		 The reason for Failures is that some commands are absent in lsb-image,for example, sendmail. We can add these packages including absent commands to lsb-image
Static Interface Tests Library Check 
4.0.2-1		 F No T Test Cases Total: 2947
Test Passed: 1557
Test Failed: 1390		 The reasons for Failures are below:
1 Some libraries are absent in lsb-image
2 Some functions of libraries are absent
Runtime Interface Tests Core Tests 4.0.5-1 F No F No Add pam-plugin-*, e2fsprogs-mke2fs, eglibc-localedata-posix
Run test Result
T Test Cases Total  10593
Tests Passed  4100
Failures  1782
Warnings  4
Tests Waived  47
FIPs  19
Cpp-T2C Tests 
4.0.2-1		 F No T Test Cases Total: 1854
Test Passed: 1589
Failures: 265
Test Waived: 144		 The reason for Failures is that some file could be absent in lsb test suite  or perhaps some other resons. We need to ensure this probleme later.
Desktop Test
4.0.4-2		 F No F No Don't install lsb test suite for it
Desktop-T2C Test 
4.0.2-1		 F No T Test Cases Total: 5132
Test Passed: 4964
Failures: 168
Test Waived: 49		 We need ensure the reason later
OLVER Core Tests 
4.0.1-1		 F No T/M
F/G		 No Absent some resources needed by lsb test suite
Perl Tests 4.0.7-1 F No T Test Cases Total  853
Tests Passed  843
Failures  10		 Add perl-pod, perl-misc, perl-doc
Modify perl package(add a patch)
Run test Result
T Tests Total  853
Tests Passed  853
Printing Tests 
4.0.4-3		 F No F No Absent some resources needed by lsb test suite
Python Running Tests 
4.0.2-1		 F No F No Absent some resources needed by lsb test suite
Qt3-Azov Tests 
4.0.2-1		 F No F No Absent some resources needed by lsb test suite
Qt4-Azov Tests 
4.0.2-1		 F No F No Absent some resources needed by lsb test suite
Xts5 Tests 
5.1.5-19		 F No F No Absent some resources needed by lsb test suite
Automated Application Battery Tests Apache Tests 
2.2.8-2		 - - T Passed
Expect Tests 
5.43.0-7		 - - T Passed
Groff Tests 
1.19.2-4		 - - T Passed
Python Tests 
2.4.5-2		 F No T Test Cases Total  291
Tests Passed  250
Failures  4
Warnings  1
Raptor Tests 
1.4.16-2		 - - T Passed
Rsync Tests 
3.0.0-2		 - - T Passed
Samba Tests 
3.0.28a-3		 - - T -
Tcl Tests 
8.5.1-2		 - - T Passed
Xpdf Tests 
1.01-7		 - - T Passed
X86-64 Command Tests Command Check 
4.0.0-2		 F No T Test Cases Total: 150
Test Passed: 139
Failures: 11
Test Waived: 8		 AS X86
Static Interface Tests Library Check 
4.0.2-1		 F No T Test Cases Total: 2947
Test Passed: 1557
Test Failed: 1390		 AS X86
Runtime Interface Tests Core Tests 4.0.5-1 F No F No We will add some packages as x86 to lsb-image and test it.
Cpp-T2C Tests 
4.0.2-1		 F No T Test Cases Total: 1854
Test Passed: 1589
Failures: 265
Test Waived: 144		 AS X86
Desktop Test
4.0.4-2		 F No F No Absent some resources needed by lsb test suite
Desktop-T2C Test 
4.0.2-1		 F No T Test Cases Total: 5132
Test Passed: 4963
Failures: 169
Test Waived: 49		 AS X86
OLVER Core Tests 
4.0.1-1		 F No T/M
F/G		 No Absent some resources needed by lsb test suite
Perl Tests 4.0.7-1 F No T Test Cases Total  853
Tests Passed  18
Failures  17		 We will add some packages as x86 to lsb-image and test it.
Printing Tests 
4.0.4-3		 F No F Test Cases Excepted:97
Tests Passed  3
Failures  21
Test suite error:1		 Absent cups libraries 
Python Running Tests 
4.0.2-1		 F No F No Absent some resources needed by lsb test suite
Qt3-Azov Tests 
4.0.2-1		 F No F No Absent some resources needed by lsb test suite
Qt4-Azov Tests 
4.0.2-1		 F No F No Absent some resources needed by lsb test suite
Xts5 Tests 
5.1.5-19		 F No F No Absent some resources needed by lsb test suite
Automated Application Battery Tests Apache Tests 
2.2.8-2		 - - - - Don't install lsb test suite for it
Expect Tests 
5.43.0-7		 - - - - Don't install lsb test suite for it
Groff Tests 
1.19.2-4		 - - - - Don't install lsb test suite for it
Python Tests 
2.4.5-2		 F No T No Absent some resources needed by lsb test suite
Raptor Tests 
1.4.16-2		 - - - - Don't install lsb test suite for it
Rsync Tests 
3.0.0-2		 - - - - Don't install lsb test suite for it
Samba Tests 
3.0.28a-3		 - - - - Don't install lsb test suite for it
Tcl Tests 
8.5.1-2		 - - - - Don't install lsb test suite for it
Xpdf Tests 
1.01-7		 - - - - Don't install lsb test suite for it

Note:
Run test: 
T: LSB test suite can run completely.
F: There are some issues that block LSB test suite itself running completely.
T/M: LSB can be tested with manual method.
T/G: LSB can be tested with graphic method.
Result:
No: There is no result because LSB test suite cannot run.


It is a  preliminary analysis for lsb test. We will give more information later(include ppc).
A plan about how to fix bugs:
step 1: We will solve bugs which some of lsb test suite can't still run on lsb-image (The work is now in progress)
step 2: Make two auto script for setting up lsb test environment. One is for setting on pc , we want to add this script to poky/scripts; the other is for setting for lsb-image, we want to add this script to lsb-image in the mode of bb file.  (sync up with step 1)
step 3: Write a user manual about how to test lsb-imgae by lsb test suite and we want to this user manual to lsb-image (after step 2)
step 4: List bugs tested by lsb test suite(sync up with step 1)
step 5: Solve bugs tested by lsb test suite(everyone can attend to fix bugs)
Method for fixing bugs:
- We will fix bugs for lsb test suite itself on platform x86 firstly and then apply the result to  other platform when step 1 is over.
- We will fix bugs tested by lsb test suite on platform x86 and then apply the result to other platform.

We will open a bugzilla for failed items tested by lsb test suite and make a milestone for fixing bugs with detailed time plan on the next week.
Please give your suggestion.

Thanks,
Yan

--------------050406040105020708070605--