[PATCH 0/3] Introduce PTEST_RUNNER_TIMEOUT variable

[PATCH 0/3] Introduce PTEST_RUNNER_TIMEOUT variable

[Tim Orling]

In https://bugzilla.yoctoproject.org/show_bug.cgi?id=16163 analysis showed
that one of the root causes is the hard coded 450 second timeout passed to
ptest-runner in lib/oeqa/runtime/cases/ptest.py.

This series introduces the PTEST_RUNNER_TIMEOUT variable, which is consumed
in lib/oeqa/runtime/cases/ptest.py but needs to be updated in testimage.bbclass
to make sure the value of the variable is consumed at runtime.

For python3-cffi which drove the change, the value is set in core-image-ptest.bb
to 600 seconds, as an example of how a per recipe change can be enabled.

The test run on the AutoBuilder cluster that proves this works is:
https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/3136
https://valkyrie.yocto.io/pub/non-release/20260227-113/testresults/qemuarm64-ptest/core-image-ptest-python3-cffi/
where you can see:
PATH=/usr/sbin:/sbin:/usr/bin:/bin; ptest-runner -t 600 -d "/usr/lib"
in https://valkyrie.yocto.io/pub/non-release/20260227-113/testresults/qemuarm64-ptest/core-image-ptest-python3-cffi/log.do_testimage.2323256.20260227191837

[YOCTO #16163]

Tim Orling (3):
  oeqa/runtime/ptest: Make ptest-runner timeout configurable
  core-image-ptest: add PTEST_RUNNER_TIMEOUT
  testimage.bbclass: enable PTEST_RUNNER_TIMEOUT variable

 meta/classes-recipe/testimage.bbclass        | 4 +++-
 meta/lib/oeqa/runtime/cases/ptest.py         | 3 ++-
 meta/recipes-core/images/core-image-ptest.bb | 1 +
 3 files changed, 6 insertions(+), 2 deletions(-)

-- 
2.50.1 (Apple Git-155)

[PATCH 1/3] oeqa/runtime/ptest: Make ptest-runner timeout configurable

[Tim Orling]

Replace the hardcoded 450 second timeout with a configurable
PTEST_RUNNER_TIMEOUT variable, defaulting to 450 seconds.

[YOCTO #16163]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/lib/oeqa/runtime/cases/ptest.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/cases/ptest.py b/meta/lib/oeqa/runtime/cases/ptest.py
index 75165101af4..1a0782374d0 100644
--- a/meta/lib/oeqa/runtime/cases/ptest.py
+++ b/meta/lib/oeqa/runtime/cases/ptest.py
@@ -59,7 +59,8 @@ class PtestRunnerTest(OERuntimeTestCase):
         ptest_dirs = [ '/usr/lib' ]
         if not libdir in ptest_dirs:
             ptest_dirs.append(libdir)
-        status, output = self.target.run('ptest-runner -t 450 -d \"{}\"'.format(' '.join(ptest_dirs)), 0)
+        ptest_timeout = self.td.get('PTEST_RUNNER_TIMEOUT', '450')
+        status, output = self.target.run('ptest-runner -t {} -d \"{}\"'.format(ptest_timeout, ' '.join(ptest_dirs)), 0)
         os.makedirs(ptest_log_dir)
         with open(ptest_runner_log, 'w') as f:
             f.write(output)
-- 
2.50.1 (Apple Git-155)

[PATCH 2/3] core-image-ptest: add PTEST_RUNNER_TIMEOUT

[Tim Orling]

In lib/oeqa/runtime/cases/ptest.py, the timeout used to be hardcoded to 450 seconds.
Now that it is a variable, make that a bit more obvious by setting a default value.

Set PTEST_RUNNER_TIMEOUT for python3-cffi to 600 seconds as it is known to come close
to and surpass the 450 second limit under heavy load.

Fixes: [YOCTO #16163]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/recipes-core/images/core-image-ptest.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-core/images/core-image-ptest.bb b/meta/recipes-core/images/core-image-ptest.bb
index 0b1eeb1a580..c08561296fe 100644
--- a/meta/recipes-core/images/core-image-ptest.bb
+++ b/meta/recipes-core/images/core-image-ptest.bb
@@ -9,6 +9,7 @@ SUMMARY ?= "${MCNAME} ptest image."
 HOMEPAGE = "https://www.yoctoproject.org/"
 
 PTESTS = "${PTESTS_SLOW} ${PTESTS_FAST}"
+PTEST_RUNNER_TIMEOUT:virtclass-mcextend-python3-cffi = "600"
 
 IMAGE_INSTALL:append = " ${MCNAME}-ptest openssh"
 
-- 
2.50.1 (Apple Git-155)

[PATCH 3/3] testimage.bbclass: enable PTEST_RUNNER_TIMEOUT variable

[Tim Orling]

QB_MEM works in recipe scope because testimage.bbclass reads it from the
current recipe datastore (d) at test time via the qemuboot.conf mechanism.
PTEST_RUNNER_TIMEOUT was only available through testdata.json (written
during image build via export2json). The testimage task reads td from the
potentially stale testdata.json, and PTEST_RUNNER_TIMEOUT was never
refreshed from the live recipe context.

[YOCTO #16163]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/classes-recipe/testimage.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes-recipe/testimage.bbclass b/meta/classes-recipe/testimage.bbclass
index 7a48ed6bfe3..9902b8a5682 100644
--- a/meta/classes-recipe/testimage.bbclass
+++ b/meta/classes-recipe/testimage.bbclass
@@ -112,7 +112,9 @@ TESTIMAGELOCK:qemuall = ""
 
 TESTIMAGE_DUMP_DIR ?= "${LOG_DIR}/runtime-hostdump/"
 
-TESTIMAGE_UPDATE_VARS ?= "DL_DIR WORKDIR DEPLOY_DIR_IMAGE IMAGE_LINK_NAME IMAGE_NAME"
+TESTIMAGE_UPDATE_VARS ?= "DL_DIR WORKDIR DEPLOY_DIR_IMAGE IMAGE_LINK_NAME IMAGE_NAME PTEST_RUNNER_TIMEOUT"
+
+PTEST_RUNNER_TIMEOUT ?= "450"
 
 testimage_dump_monitor () {
     query-status
-- 
2.50.1 (Apple Git-155)