[OE-core][PATCH 1/2] oeqa/runtime/cases/ptest.py: use to_boolean for PTEST_EXPECT_FAILURE

[OE-core][PATCH 1/2] oeqa/runtime/cases/ptest.py: use to_boolean for PTEST_EXPECT_FAILURE

[Qi.Chen]

From: Chen Qi <Qi.Chen@windriver.com>

When PTEST_EXPECT_FAILURE is set to "0", the expected behavior is
that ptest should succeed. So we need to use to_boolean.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/lib/oeqa/runtime/cases/ptest.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/runtime/cases/ptest.py b/meta/lib/oeqa/runtime/cases/ptest.py
index fbaeb84d00..4bbed290b4 100644
--- a/meta/lib/oeqa/runtime/cases/ptest.py
+++ b/meta/lib/oeqa/runtime/cases/ptest.py
@@ -22,7 +22,7 @@ class PtestRunnerTest(OERuntimeTestCase):
     @OEHasPackage(['ptest-runner'])
     @unittest.expectedFailure
     def test_ptestrunner_expectfail(self):
-        if not self.td.get('PTEST_EXPECT_FAILURE'):
+        if not bb.utils.to_boolean(self.td.get('PTEST_EXPECT_FAILURE')):
             self.skipTest('Cannot run ptests with @expectedFailure as ptests are required to pass')
         self.do_ptestrunner()
 
@@ -30,8 +30,8 @@ class PtestRunnerTest(OERuntimeTestCase):
     @OETestDepends(['ssh.SSHTest.test_ssh'])
     @OEHasPackage(['ptest-runner'])
     def test_ptestrunner_expectsuccess(self):
-        if self.td.get('PTEST_EXPECT_FAILURE'):
-            self.skipTest('Cannot run ptests without @expectedFailure as ptests are expected to fail')
+        if bb.utils.to_boolean(self.td.get('PTEST_EXPECT_FAILURE')):
+             self.skipTest('Cannot run ptests without @expectedFailure as ptests are expected to fail')
         self.do_ptestrunner()
 
     def do_ptestrunner(self):
-- 
2.43.0

[OE-core][PATCH 2/2] core-image-ptest.bb: use ?= for PTEST_EXPECT_FAILURE

[Qi.Chen]

From: Chen Qi <Qi.Chen@windriver.com>

We need to allow users to easily override this value in local.conf.
This has the benefit that 'bitbake core-image-ptest-xxx:do_testimage'
fails when expected.

As an example, I used 'bitbake core-image-ptest-util-linux:do_testimage'.
It succeeded with warning message. I didn't notice the warning message.
I saw the command succeeded and I tought util-linux ptest is OK. But
in actual fact, the ptest failed. It's the PTEST_EXPECT_FAILURE setting
in this core-image-ptest.bb that is not giving me error.

With this change, I can put PTEST_EXPECT_FAILURE = "1" in my local.conf,
and I can easily see if 'core-image-ptest-xxx:do_testimage' succeeds or
not.

The PTEST_EXPECT_FAILURE change should result in testdata regenerated
because ptest.py test case's behavior is affected by it.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/recipes-core/images/core-image-ptest.bb | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/images/core-image-ptest.bb b/meta/recipes-core/images/core-image-ptest.bb
index 017f05f81b..b554964d20 100644
--- a/meta/recipes-core/images/core-image-ptest.bb
+++ b/meta/recipes-core/images/core-image-ptest.bb
@@ -42,7 +42,8 @@ QB_MEM:virtclass-mcextend-tcl = "-m 5100"
 TEST_SUITES = "ping ssh parselogs ptest"
 
 # Sadly at the moment the full set of ptests is not robust enough and sporadically fails in random places
-PTEST_EXPECT_FAILURE = "1"
+PTEST_EXPECT_FAILURE ?= "1"
+write_image_test_data[vardeps] += "PTEST_EXPECT_FAILURE"
 
 python () {
     if not d.getVar("MCNAME"):
-- 
2.43.0

RE: [OE-core][PATCH 2/2] core-image-ptest.bb: use ?= for PTEST_EXPECT_FAILURE

[Marko, Peter]

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-
> core@lists.openembedded.org> On Behalf Of Chen Qi via
> lists.openembedded.org
> Sent: Friday, December 5, 2025 9:10
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core][PATCH 2/2] core-image-ptest.bb: use ?= for
> PTEST_EXPECT_FAILURE
> 
> From: Chen Qi <Qi.Chen@windriver.com>
> 
> We need to allow users to easily override this value in local.conf.
> This has the benefit that 'bitbake core-image-ptest-xxx:do_testimage'
> fails when expected.
> 
> As an example, I used 'bitbake core-image-ptest-util-linux:do_testimage'.
> It succeeded with warning message. I didn't notice the warning message.
> I saw the command succeeded and I tought util-linux ptest is OK. But
> in actual fact, the ptest failed. It's the PTEST_EXPECT_FAILURE setting
> in this core-image-ptest.bb that is not giving me error.
> 
> With this change, I can put PTEST_EXPECT_FAILURE = "1" in my local.conf,

This comment should be probably "0".

> and I can easily see if 'core-image-ptest-xxx:do_testimage' succeeds or
> not.
> 
> The PTEST_EXPECT_FAILURE change should result in testdata regenerated
> because ptest.py test case's behavior is affected by it.
> 
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
>  meta/recipes-core/images/core-image-ptest.bb | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-core/images/core-image-ptest.bb b/meta/recipes-
> core/images/core-image-ptest.bb
> index 017f05f81b..b554964d20 100644
> --- a/meta/recipes-core/images/core-image-ptest.bb
> +++ b/meta/recipes-core/images/core-image-ptest.bb
> @@ -42,7 +42,8 @@ QB_MEM:virtclass-mcextend-tcl = "-m 5100"
>  TEST_SUITES = "ping ssh parselogs ptest"
> 
>  # Sadly at the moment the full set of ptests is not robust enough and sporadically
> fails in random places
> -PTEST_EXPECT_FAILURE = "1"
> +PTEST_EXPECT_FAILURE ?= "1"
> +write_image_test_data[vardeps] += "PTEST_EXPECT_FAILURE"
> 
>  python () {
>      if not d.getVar("MCNAME"):
> --
> 2.43.0

Re: [OE-core][PATCH 2/2] core-image-ptest.bb: use ?= for PTEST_EXPECT_FAILURE

[ChenQi]

On 12/5/25 16:26, Marko, Peter wrote:
>
>> -----Original Message-----
>> From: openembedded-core@lists.openembedded.org <openembedded-
>> core@lists.openembedded.org> On Behalf Of Chen Qi via
>> lists.openembedded.org
>> Sent: Friday, December 5, 2025 9:10
>> To: openembedded-core@lists.openembedded.org
>> Subject: [OE-core][PATCH 2/2] core-image-ptest.bb: use ?= for
>> PTEST_EXPECT_FAILURE
>>
>> From: Chen Qi <Qi.Chen@windriver.com>
>>
>> We need to allow users to easily override this value in local.conf.
>> This has the benefit that 'bitbake core-image-ptest-xxx:do_testimage'
>> fails when expected.
>>
>> As an example, I used 'bitbake core-image-ptest-util-linux:do_testimage'.
>> It succeeded with warning message. I didn't notice the warning message.
>> I saw the command succeeded and I tought util-linux ptest is OK. But
>> in actual fact, the ptest failed. It's the PTEST_EXPECT_FAILURE setting
>> in this core-image-ptest.bb that is not giving me error.
>>
>> With this change, I can put PTEST_EXPECT_FAILURE = "1" in my local.conf,
> This comment should be probably "0".

You're right. I'll send out V2. Thanks for spotting it.

Regards,
Qi

>
>> and I can easily see if 'core-image-ptest-xxx:do_testimage' succeeds or
>> not.
>>
>> The PTEST_EXPECT_FAILURE change should result in testdata regenerated
>> because ptest.py test case's behavior is affected by it.
>>
>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>> ---
>>   meta/recipes-core/images/core-image-ptest.bb | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/meta/recipes-core/images/core-image-ptest.bb b/meta/recipes-
>> core/images/core-image-ptest.bb
>> index 017f05f81b..b554964d20 100644
>> --- a/meta/recipes-core/images/core-image-ptest.bb
>> +++ b/meta/recipes-core/images/core-image-ptest.bb
>> @@ -42,7 +42,8 @@ QB_MEM:virtclass-mcextend-tcl = "-m 5100"
>>   TEST_SUITES = "ping ssh parselogs ptest"
>>
>>   # Sadly at the moment the full set of ptests is not robust enough and sporadically
>> fails in random places
>> -PTEST_EXPECT_FAILURE = "1"
>> +PTEST_EXPECT_FAILURE ?= "1"
>> +write_image_test_data[vardeps] += "PTEST_EXPECT_FAILURE"
>>
>>   python () {
>>       if not d.getVar("MCNAME"):
>> --
>> 2.43.0