* [PATCHv2 0/2] testimage: add/modify DEFAULT_TEST_SUITES
@ 2016-02-23 9:14 Alex Franco
2016-02-23 9:14 ` [PATCHv2 1/2] testimage.bbclass: add generic, image test suites Alex Franco
2016-02-23 9:14 ` [PATCHv2 2/2] testimage.bbclass: reuse generic " Alex Franco
0 siblings, 2 replies; 6+ messages in thread
From: Alex Franco @ 2016-02-23 9:14 UTC (permalink / raw)
To: openembedded-core
- Add generic variables to reuse values in default test suites
- Add more default test suites variables
- Modify existing test suites to use generic vars when possible
[YOCTO #8410]
Alex Franco (2):
testimage.bbclass: add generic, image test suites
testimage.bbclass: reuse generic test suites
meta/classes/testimage.bbclass | 22 ++++++++++++++++------
1 file changed, 16 insertions(+), 6 deletions(-)
--
2.6.2
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCHv2 1/2] testimage.bbclass: add generic, image test suites
2016-02-23 9:14 [PATCHv2 0/2] testimage: add/modify DEFAULT_TEST_SUITES Alex Franco
@ 2016-02-23 9:14 ` Alex Franco
2016-02-23 9:14 ` [PATCHv2 2/2] testimage.bbclass: reuse generic " Alex Franco
1 sibling, 0 replies; 6+ messages in thread
From: Alex Franco @ 2016-02-23 9:14 UTC (permalink / raw)
To: openembedded-core
Add generic test suites MINTESTSUITE, NETTESTSUITE, DEVTESTSUITE to
group common test suites.
Add DEFAULT_TEST_SUITES_pn- variables for:
* core-image-minimal-dev
* core-image-full-cmdline
* core-image-x11
* core-image-lsb
* core-image-lsb-dev
New default test suites use generic test suites.
Existing default test suites not modified.
[YOCTO #8410]
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
---
meta/classes/testimage.bbclass | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index eed27be..791cce3 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -33,12 +33,20 @@ TEST_EXPORT_DIR ?= "${TMPDIR}/testimage/${PN}"
TEST_EXPORT_ONLY ?= "0"
RPMTESTSUITE = "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'smart rpm', '', d)}"
+MINTESTSUITE = "ping"
+NETTESTSUITE = "${MINTESTSUITE} ssh df date scp syslog"
+DEVTESTSUITE = "gcc kernelmodule ldd"
DEFAULT_TEST_SUITES = "ping auto"
DEFAULT_TEST_SUITES_pn-core-image-minimal = "ping"
+DEFAULT_TEST_SUITES_pn-core-image-minimal-dev = "${MINTESTSUITE}"
+DEFAULT_TEST_SUITES_pn-core-image-full-cmdline = "${NETTESTSUITE} perl python logrotate"
+DEFAULT_TEST_SUITES_pn-core-image-x11 = "${MINTESTSUITE}"
+DEFAULT_TEST_SUITES_pn-core-image-lsb = "${NETTESTSUITE} pam parselogs ${RPMTESTSUITE}"
DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh df connman syslog xorg scp date parselogs ${RPMTESTSUITE} \
${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python', '', d)}"
DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh df connman syslog xorg scp date perl ldd gcc kernelmodule python parselogs ${RPMTESTSUITE}"
+DEFAULT_TEST_SUITES_pn-core-image-lsb-dev = "${NETTESTSUITE} pam perl python parselogs ${RPMTESTSUITE}"
DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "ping buildcvs buildiptables buildsudoku connman date df gcc kernelmodule ldd pam parselogs perl python scp ${RPMTESTSUITE} ssh syslog logrotate"
DEFAULT_TEST_SUITES_pn-meta-toolchain = "auto"
--
2.6.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCHv2 2/2] testimage.bbclass: reuse generic test suites
2016-02-23 9:14 [PATCHv2 0/2] testimage: add/modify DEFAULT_TEST_SUITES Alex Franco
2016-02-23 9:14 ` [PATCHv2 1/2] testimage.bbclass: add generic, image test suites Alex Franco
@ 2016-02-23 9:14 ` Alex Franco
2016-02-23 21:20 ` Benjamin Esquivel
1 sibling, 1 reply; 6+ messages in thread
From: Alex Franco @ 2016-02-23 9:14 UTC (permalink / raw)
To: openembedded-core
Modify existing default test suites to reuse the new
generic test suites.
Related to [YOCTO #8410]
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
---
meta/classes/testimage.bbclass | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 791cce3..e77bb11 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -37,17 +37,19 @@ MINTESTSUITE = "ping"
NETTESTSUITE = "${MINTESTSUITE} ssh df date scp syslog"
DEVTESTSUITE = "gcc kernelmodule ldd"
-DEFAULT_TEST_SUITES = "ping auto"
-DEFAULT_TEST_SUITES_pn-core-image-minimal = "ping"
+DEFAULT_TEST_SUITES = "${MINTESTSUITE} auto"
+DEFAULT_TEST_SUITES_pn-core-image-minimal = "${MINTESTSUITE}"
DEFAULT_TEST_SUITES_pn-core-image-minimal-dev = "${MINTESTSUITE}"
DEFAULT_TEST_SUITES_pn-core-image-full-cmdline = "${NETTESTSUITE} perl python logrotate"
DEFAULT_TEST_SUITES_pn-core-image-x11 = "${MINTESTSUITE}"
DEFAULT_TEST_SUITES_pn-core-image-lsb = "${NETTESTSUITE} pam parselogs ${RPMTESTSUITE}"
-DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh df connman syslog xorg scp date parselogs ${RPMTESTSUITE} \
+DEFAULT_TEST_SUITES_pn-core-image-sato = "${NETTESTSUITE} connman xorg parselogs ${RPMTESTSUITE} \
${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python', '', d)}"
-DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh df connman syslog xorg scp date perl ldd gcc kernelmodule python parselogs ${RPMTESTSUITE}"
+DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "${NETTESTSUITE} connman xorg perl python \
+ ${DEVTESTSUITE} parselogs ${RPMTESTSUITE}"
DEFAULT_TEST_SUITES_pn-core-image-lsb-dev = "${NETTESTSUITE} pam perl python parselogs ${RPMTESTSUITE}"
-DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "ping buildcvs buildiptables buildsudoku connman date df gcc kernelmodule ldd pam parselogs perl python scp ${RPMTESTSUITE} ssh syslog logrotate"
+DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "${NETTESTSUITE} buildcvs buildiptables buildsudoku \
+ connman ${DEVTESTSUITE} pam perl python parselogs ${RPMTESTSUITE}"
DEFAULT_TEST_SUITES_pn-meta-toolchain = "auto"
# aarch64 has no graphics
--
2.6.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCHv2 2/2] testimage.bbclass: reuse generic test suites
2016-02-23 9:14 ` [PATCHv2 2/2] testimage.bbclass: reuse generic " Alex Franco
@ 2016-02-23 21:20 ` Benjamin Esquivel
2016-02-23 22:18 ` Alex Franco
0 siblings, 1 reply; 6+ messages in thread
From: Benjamin Esquivel @ 2016-02-23 21:20 UTC (permalink / raw)
To: Alex Franco, openembedded-core
Hi Franco, how did you test this? what is the behavior you get with
these defaults instead of how's behaving now?
See comments below.
On Tue, 2016-02-23 at 09:14 +0000, Alex Franco wrote:
> Modify existing default test suites to reuse the new
> generic test suites.
>
> Related to [YOCTO #8410]
>
> Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
> ---
> meta/classes/testimage.bbclass | 12 +++++++-----
> 1 file changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/meta/classes/testimage.bbclass
> b/meta/classes/testimage.bbclass
> index 791cce3..e77bb11 100644
> --- a/meta/classes/testimage.bbclass
> +++ b/meta/classes/testimage.bbclass
> @@ -37,17 +37,19 @@ MINTESTSUITE = "ping"
> NETTESTSUITE = "${MINTESTSUITE} ssh df date scp syslog"
> DEVTESTSUITE = "gcc kernelmodule ldd"
>
> -DEFAULT_TEST_SUITES = "ping auto"
> -DEFAULT_TEST_SUITES_pn-core-image-minimal = "ping"
> +DEFAULT_TEST_SUITES = "${MINTESTSUITE} auto"
> +DEFAULT_TEST_SUITES_pn-core-image-minimal = "${MINTESTSUITE}"
> DEFAULT_TEST_SUITES_pn-core-image-minimal-dev = "${MINTESTSUITE}"
> DEFAULT_TEST_SUITES_pn-core-image-full-cmdline = "${NETTESTSUITE}
> perl python logrotate"
> DEFAULT_TEST_SUITES_pn-core-image-x11 = "${MINTESTSUITE}"
> DEFAULT_TEST_SUITES_pn-core-image-lsb = "${NETTESTSUITE} pam
> parselogs ${RPMTESTSUITE}"
> -DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh df connman syslog
> xorg scp date parselogs ${RPMTESTSUITE} \
> +DEFAULT_TEST_SUITES_pn-core-image-sato = "${NETTESTSUITE} connman
> xorg parselogs ${RPMTESTSUITE} \
> ${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python', '', d)}"
> -DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh df connman
> syslog xorg scp date perl ldd gcc kernelmodule python parselogs
> ${RPMTESTSUITE}"
> +DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "${NETTESTSUITE}
> connman xorg perl python \
> + ${DEVTESTSUITE} parselogs ${RPMTESTSUITE}"
> DEFAULT_TEST_SUITES_pn-core-image-lsb-dev = "${NETTESTSUITE} pam
> perl python parselogs ${RPMTESTSUITE}"
> -DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "ping buildcvs
> buildiptables buildsudoku connman date df gcc kernelmodule ldd pam
> parselogs perl python scp ${RPMTESTSUITE} ssh syslog logrotate"
> +DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "${NETTESTSUITE}
> buildcvs buildiptables buildsudoku \
> + connman ${DEVTESTSUITE} pam perl python parselogs
These lots of declarations seem to be suited for a .inc file instead of
living in the bbclass.
> ${RPMTESTSUITE}"
> DEFAULT_TEST_SUITES_pn-meta-toolchain = "auto"
>
> # aarch64 has no graphics
> --
> 2.6.2
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCHv2 2/2] testimage.bbclass: reuse generic test suites
2016-02-23 21:20 ` Benjamin Esquivel
@ 2016-02-23 22:18 ` Alex Franco
2016-02-24 15:47 ` Benjamin Esquivel
0 siblings, 1 reply; 6+ messages in thread
From: Alex Franco @ 2016-02-23 22:18 UTC (permalink / raw)
To: benjamin.esquivel, openembedded-core
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I tested this with builds in my workstation and (local, internal)
autobuilder, where TEST_SUITES is *not* specified in local.conf, so the
TEST_SUITES are taken from the DEFAULT_TEST_SUITES variable for each
image, which makes the tests work for the variables defined.
If TEST_SUITES includes "auto", then the testimage task for some images
will still fail, but that's why I opened a new bug # 9129
- - Alex Franco
On 02/23/2016 03:20 PM, Benjamin Esquivel wrote:
> Hi Franco, how did you test this? what is the behavior you get with
> these defaults instead of how's behaving now?
>
> See comments below.
>
> On Tue, 2016-02-23 at 09:14 +0000, Alex Franco wrote:
>> Modify existing default test suites to reuse the new
>> generic test suites.
>>
>> Related to [YOCTO #8410]
>>
>> Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
>> ---
>> meta/classes/testimage.bbclass | 12 +++++++-----
>> 1 file changed, 7 insertions(+), 5 deletions(-)
>>
>> diff --git a/meta/classes/testimage.bbclass
>> b/meta/classes/testimage.bbclass
>> index 791cce3..e77bb11 100644
>> --- a/meta/classes/testimage.bbclass
>> +++ b/meta/classes/testimage.bbclass
>> @@ -37,17 +37,19 @@ MINTESTSUITE = "ping"
>> NETTESTSUITE = "${MINTESTSUITE} ssh df date scp syslog"
>> DEVTESTSUITE = "gcc kernelmodule ldd"
>>
>> -DEFAULT_TEST_SUITES = "ping auto"
>> -DEFAULT_TEST_SUITES_pn-core-image-minimal = "ping"
>> +DEFAULT_TEST_SUITES = "${MINTESTSUITE} auto"
>> +DEFAULT_TEST_SUITES_pn-core-image-minimal = "${MINTESTSUITE}"
>> DEFAULT_TEST_SUITES_pn-core-image-minimal-dev = "${MINTESTSUITE}"
>> DEFAULT_TEST_SUITES_pn-core-image-full-cmdline = "${NETTESTSUITE}
>> perl python logrotate"
>> DEFAULT_TEST_SUITES_pn-core-image-x11 = "${MINTESTSUITE}"
>> DEFAULT_TEST_SUITES_pn-core-image-lsb = "${NETTESTSUITE} pam
>> parselogs ${RPMTESTSUITE}"
>> -DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh df connman syslog
>> xorg scp date parselogs ${RPMTESTSUITE} \
>> +DEFAULT_TEST_SUITES_pn-core-image-sato = "${NETTESTSUITE} connman
>> xorg parselogs ${RPMTESTSUITE} \
>> ${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python', '', d)}"
>> -DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh df connman
>> syslog xorg scp date perl ldd gcc kernelmodule python parselogs
>> ${RPMTESTSUITE}"
>> +DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "${NETTESTSUITE}
>> connman xorg perl python \
>> + ${DEVTESTSUITE} parselogs ${RPMTESTSUITE}"
>> DEFAULT_TEST_SUITES_pn-core-image-lsb-dev = "${NETTESTSUITE} pam
>> perl python parselogs ${RPMTESTSUITE}"
>> -DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "ping buildcvs
>> buildiptables buildsudoku connman date df gcc kernelmodule ldd pam
>> parselogs perl python scp ${RPMTESTSUITE} ssh syslog logrotate"
>> +DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "${NETTESTSUITE}
>> buildcvs buildiptables buildsudoku \
>> + connman ${DEVTESTSUITE} pam perl python parselogs
>
> These lots of declarations seem to be suited for a .inc file instead of
> living in the bbclass.
>> ${RPMTESTSUITE}"
>> DEFAULT_TEST_SUITES_pn-meta-toolchain = "auto"
>>
>> # aarch64 has no graphics
>> --
>> 2.6.2
>>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBAgAGBQJWzNqsAAoJEPR9uwMY2lTL9ecH/RQag7augvPCsjKCIdDVAV/C
+PW1JgLwLz1ox9TU7fbVICrXPpBLu/a1tGawstqcvLW2mrlzDy7wJ3Vd1pIhkXTV
zqby90ziqPL9eyeYBnCSv5vyttbB4D8WqY7bf+yC7WAhCB0iXiWjHpjsGFwCyolc
V4gaJGUI1Uv6h7DBQTLoVxTENHFLiLmGqrf3dn6O26EGs9wJlH5rKDnKD6sctCrH
A0w7EWMsBNVPlRFFstBreTX/0SEStU9HsaAUo0bQrwe6ZRZ0Ku0zT9fj9YVdRXxL
BXcVMbv6v/TSut7Ah41LWCCcwmjT6XhJZStuiq7zTRwHufAYQNBPVAPMGdW55YM=
=2lA5
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCHv2 2/2] testimage.bbclass: reuse generic test suites
2016-02-23 22:18 ` Alex Franco
@ 2016-02-24 15:47 ` Benjamin Esquivel
0 siblings, 0 replies; 6+ messages in thread
From: Benjamin Esquivel @ 2016-02-24 15:47 UTC (permalink / raw)
To: Alex Franco, openembedded-core
On Tue, 2016-02-23 at 16:18 -0600, Alex Franco wrote:
> I tested this with builds in my workstation and (local, internal)
> autobuilder, where TEST_SUITES is *not* specified in local.conf, so
> the
> TEST_SUITES are taken from the DEFAULT_TEST_SUITES variable for each
> image, which makes the tests work for the variables defined.
>
> If TEST_SUITES includes "auto", then the testimage task for some
> images
> will still fail, but that's why I opened a new bug # 9129
>
Thanks, good info.
> - Alex Franco
>
> On 02/23/2016 03:20 PM, Benjamin Esquivel wrote:
> > Hi Franco, how did you test this? what is the behavior you get with
> > these defaults instead of how's behaving now?
> >
> > See comments below.
> >
> > On Tue, 2016-02-23 at 09:14 +0000, Alex Franco wrote:
> > > Modify existing default test suites to reuse the new
> > > generic test suites.
> > >
> > > Related to [YOCTO #8410]
> > >
> > > Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
> > > ---
> > > meta/classes/testimage.bbclass | 12 +++++++-----
> > > 1 file changed, 7 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/meta/classes/testimage.bbclass
> > > b/meta/classes/testimage.bbclass
> > > index 791cce3..e77bb11 100644
> > > --- a/meta/classes/testimage.bbclass
> > > +++ b/meta/classes/testimage.bbclass
> > > @@ -37,17 +37,19 @@ MINTESTSUITE = "ping"
> > > NETTESTSUITE = "${MINTESTSUITE} ssh df date scp syslog"
> > > DEVTESTSUITE = "gcc kernelmodule ldd"
> > >
> > > -DEFAULT_TEST_SUITES = "ping auto"
> > > -DEFAULT_TEST_SUITES_pn-core-image-minimal = "ping"
> > > +DEFAULT_TEST_SUITES = "${MINTESTSUITE} auto"
> > > +DEFAULT_TEST_SUITES_pn-core-image-minimal = "${MINTESTSUITE}"
> > > DEFAULT_TEST_SUITES_pn-core-image-minimal-dev =
> > > "${MINTESTSUITE}"
> > > DEFAULT_TEST_SUITES_pn-core-image-full-cmdline =
> > > "${NETTESTSUITE}
> > > perl python logrotate"
> > > DEFAULT_TEST_SUITES_pn-core-image-x11 = "${MINTESTSUITE}"
> > > DEFAULT_TEST_SUITES_pn-core-image-lsb = "${NETTESTSUITE} pam
> > > parselogs ${RPMTESTSUITE}"
> > > -DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh df connman
> > > syslog
> > > xorg scp date parselogs ${RPMTESTSUITE} \
> > > +DEFAULT_TEST_SUITES_pn-core-image-sato = "${NETTESTSUITE}
> > > connman
> > > xorg parselogs ${RPMTESTSUITE} \
> > > ${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python', '',
> > > d)}"
> > > -DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh df
> > > connman
> > > syslog xorg scp date perl ldd gcc kernelmodule python parselogs
> > > ${RPMTESTSUITE}"
> > > +DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "${NETTESTSUITE}
> > > connman xorg perl python \
> > > + ${DEVTESTSUITE} parselogs ${RPMTESTSUITE}"
> > > DEFAULT_TEST_SUITES_pn-core-image-lsb-dev = "${NETTESTSUITE} pam
> > > perl python parselogs ${RPMTESTSUITE}"
> > > -DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "ping buildcvs
> > > buildiptables buildsudoku connman date df gcc kernelmodule ldd
> > > pam
> > > parselogs perl python scp ${RPMTESTSUITE} ssh syslog logrotate"
> > > +DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "${NETTESTSUITE}
> > > buildcvs buildiptables buildsudoku \
> > > + connman ${DEVTESTSUITE} pam perl python parselogs
> >
> > These lots of declarations seem to be suited for a .inc file
> > instead of
> > living in the bbclass.
> > > ${RPMTESTSUITE}"
> > > DEFAULT_TEST_SUITES_pn-meta-toolchain = "auto"
> > >
> > > # aarch64 has no graphics
> > > --
> > > 2.6.2
> > >
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-02-24 16:50 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-02-23 9:14 [PATCHv2 0/2] testimage: add/modify DEFAULT_TEST_SUITES Alex Franco
2016-02-23 9:14 ` [PATCHv2 1/2] testimage.bbclass: add generic, image test suites Alex Franco
2016-02-23 9:14 ` [PATCHv2 2/2] testimage.bbclass: reuse generic " Alex Franco
2016-02-23 21:20 ` Benjamin Esquivel
2016-02-23 22:18 ` Alex Franco
2016-02-24 15:47 ` Benjamin Esquivel
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox