[PATCH 0/2] testimage: add/modify DEFAULT_TEST_SUITES

[PATCH 0/2] testimage: add/modify DEFAULT_TEST_SUITES

[Alex Franco]

- Add generic variables to reuse values in default test suites
- Add more default test suites variables
- Modify existing test suites to use generic vars when possible

[YOCTO #8410]

Alex Franco (2):
  testimage: add DEFAULT_TEST_SUITES for more images
  testimage: reuse generic test suites

 meta/classes/testimage.bbclass | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

--
2.6.2

[PATCH 1/2] testimage: add DEFAULT_TEST_SUITES for more images

[Alex Franco]

Add these generic variables, to reuse test suite lists in more
specific default test suites variables for actual images:

DEFAULT_TEST_SUITES_minimal
DEFAULT_TEST_SUITES_network
DEFAULT_TEST_SUITES_dev

Add these default test suites variables:

DEFAULT_TEST_SUITES_pn-core-image-minimal-dev
DEFAULT_TEST_SUITES_pn-core-image-full-cmdline
DEFAULT_TEST_SUITES_pn-core-image-x11
DEFAULT_TEST_SUITES_pn-core-image-lsb

[YOCTO #8410]

Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
---
 meta/classes/testimage.bbclass | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index eed27be..cce8cea 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -33,9 +33,17 @@ TEST_EXPORT_DIR ?= "${TMPDIR}/testimage/${PN}"
 TEST_EXPORT_ONLY ?= "0"
 
 RPMTESTSUITE = "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'smart rpm', '', d)}"
+MINTESTSUITE = "ping"
+NETTESTSUITE = "${MINTESTSUITE} ssh df date scp syslog"
+DEVTESTSUITE = "gcc kernelmodule ldd"
 
 DEFAULT_TEST_SUITES = "ping auto"
 DEFAULT_TEST_SUITES_pn-core-image-minimal = "ping"
+DEFAULT_TEST_SUITES_pn-core-image-minimal-dev = "${MINTESTSUITE}"
+DEFAULT_TEST_SUITES_pn-core-image-full-cmdline = "${NETTESTSUITE} perl python logrotate"
+DEFAULT_TEST_SUITES_pn-core-image-x11 = "${MINTESTSUITE}"
+DEFAULT_TEST_SUITES_pn-core-image-lsb = "${NETTESTSUITE} buildcvs buildiptables buildsudoku \
+    connman pam parselogs ${RPMTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh df connman syslog xorg scp date parselogs ${RPMTESTSUITE} \
     ${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python', '', d)}"
 DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh df connman syslog xorg scp date perl ldd gcc kernelmodule python parselogs ${RPMTESTSUITE}"
-- 
2.6.2

[PATCH 2/2] testimage: reuse generic test suites

[Alex Franco]

Modify existing default test suites to use the recently added
generic vars whenever possible.

[YOCTO #8410]

Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
---
 meta/classes/testimage.bbclass | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index cce8cea..8276568 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -37,17 +37,19 @@ MINTESTSUITE = "ping"
 NETTESTSUITE = "${MINTESTSUITE} ssh df date scp syslog"
 DEVTESTSUITE = "gcc kernelmodule ldd"
 
-DEFAULT_TEST_SUITES = "ping auto"
-DEFAULT_TEST_SUITES_pn-core-image-minimal = "ping"
+DEFAULT_TEST_SUITES = "${MINTESTSUITE} auto"
+DEFAULT_TEST_SUITES_pn-core-image-minimal = "${MINTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-core-image-minimal-dev = "${MINTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-core-image-full-cmdline = "${NETTESTSUITE} perl python logrotate"
 DEFAULT_TEST_SUITES_pn-core-image-x11 = "${MINTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-core-image-lsb = "${NETTESTSUITE} buildcvs buildiptables buildsudoku \
     connman pam parselogs ${RPMTESTSUITE}"
-DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh df connman syslog xorg scp date parselogs ${RPMTESTSUITE} \
+DEFAULT_TEST_SUITES_pn-core-image-sato = "${NETTESTSUITE} connman xorg parselogs ${RPMTESTSUITE} \
     ${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python', '', d)}"
-DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh df connman syslog xorg scp date perl ldd gcc kernelmodule python parselogs ${RPMTESTSUITE}"
-DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "ping buildcvs buildiptables buildsudoku connman date df gcc kernelmodule ldd pam parselogs perl python scp ${RPMTESTSUITE} ssh syslog logrotate"
+DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "${NETTESTSUITE} connman xorg perl python \
+    ${DEVTESTSUITE} parselogs ${RPMTESTSUITE}"
+DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "${NETTESTSUITE} buildcvs buildiptables buildsudoku \
+    connman ${DEVTESTSUITE} pam perl python parselogs ${RPMTESTSUITE}"
 DEFAULT_TEST_SUITES_pn-meta-toolchain = "auto"
 
 # aarch64 has no graphics
-- 
2.6.2

Re: [PATCH 0/2] testimage: add/modify DEFAULT_TEST_SUITES

[Richard Purdie]

On Wed, 2016-02-17 at 09:55 +0000, Alex Franco wrote:
> - Add generic variables to reuse values in default test suites
> - Add more default test suites variables
> - Modify existing test suites to use generic vars when possible
> 
> [YOCTO #8410]
> 
> Alex Franco (2):
>   testimage: add DEFAULT_TEST_SUITES for more images
>   testimage: reuse generic test suites

Results in failures on the autobuilder, e.g.:

https://autobuilder.yoctoproject.org/main/builders/nightly-x86-64-lsb/builds/667/steps/Running%20Sanity%20Tests/logs/stdio

(I'm guessing it was from these patches)

Cheers,

Richard

Re: [PATCH 0/2] testimage: add/modify DEFAULT_TEST_SUITES

[Alex Franco]

So I'm currently running a couple lsb builds, but even though they
aren't finished, I think this patch may be the culprit, since I am
defining DEFAULT_TEST_SUITES_pn-core-image-lsb with exactly the tests
that are failing, so I'll most likely be sending an update soon.

Alex Franco

On 02/18/2016 08:45 AM, Richard Purdie wrote:
> On Wed, 2016-02-17 at 09:55 +0000, Alex Franco wrote:
>> - Add generic variables to reuse values in default test suites
>> - Add more default test suites variables
>> - Modify existing test suites to use generic vars when possible
>>
>> [YOCTO #8410]
>>
>> Alex Franco (2):
>>   testimage: add DEFAULT_TEST_SUITES for more images
>>   testimage: reuse generic test suites
> Results in failures on the autobuilder, e.g.:
>
> https://autobuilder.yoctoproject.org/main/builders/nightly-x86-64-lsb/builds/667/steps/Running%20Sanity%20Tests/logs/stdio
>
> (I'm guessing it was from these patches)
>
> Cheers,
>
> Richard