[PATCH] ref-manual: document PTEST_RUNNER_TIMEOUT variable

[PATCH] ref-manual: document PTEST_RUNNER_TIMEOUT variable

[Tim Orling]

Add PTEST_RUNNER_TIMEOUT to the variable glossary and mention it in the
testimage class documentation.

The variable was added to testimage.bbclass to control the per-suite
timeout for ptest-runner during runtime testing. It defaults to 450
seconds and can be overridden in image recipes.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 documentation/ref-manual/classes.rst   |  5 +++++
 documentation/ref-manual/variables.rst | 14 ++++++++++++++
 2 files changed, 19 insertions(+)

diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst
index eb76c8916..7a045b6b7 100644
--- a/documentation/ref-manual/classes.rst
+++ b/documentation/ref-manual/classes.rst
@@ -3110,6 +3110,11 @@ after it is built, you can set :term:`TESTIMAGE_AUTO`::
 
    TESTIMAGE_AUTO = "1"
 
+The class defines :term:`PTEST_RUNNER_TIMEOUT`, which controls
+the per-suite timeout for ``ptest-runner`` during runtime testing.
+Image recipes can override this variable to adjust the timeout for
+specific images or multiconfig variants.
+
 For information on how to enable, run, and create new tests, see the
 ":ref:`test-manual/runtime-testing:performing automated runtime testing`"
 section in the Yocto Project Test Environment Manual.
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 208b5e43d..587f67cc6 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -7793,6 +7793,20 @@ system and gives an overview of their function and contents.
       For more information, see
       the :ref:`ref-classes-ptest-python-pytest` class documentation.
 
+   :term:`PTEST_RUNNER_TIMEOUT`
+      Specifies the timeout in seconds for each ptest suite executed by
+      ``ptest-runner`` during :ref:`ref-classes-testimage` runtime testing.
+      The default value is ``"450"`` (7.5 minutes).
+
+      This variable is set in the :ref:`ref-classes-testimage` class and can
+      be overridden in image recipes to adjust the timeout for specific
+      images or multiconfig variants. For example, in ``core-image-ptest``::
+
+         PTEST_RUNNER_TIMEOUT:virtclass-mcextend-python3-cffi = "600"
+
+      The value is passed to the ``ptest-runner -t`` option on the target
+      at test execution time.
+
    :term:`PV`
       The version of the recipe. The version is normally extracted from the
       recipe filename. For example, if the recipe is named
-- 
2.50.1 (Apple Git-155)

Re: [docs] [PATCH] ref-manual: document PTEST_RUNNER_TIMEOUT variable

[Tim Orling]

[-- Attachment #1: Type: text/plain, Size: 3309 bytes --]

Related oe-core submission:
https://lists.openembedded.org/g/openembedded-core/message/232111

On Fri, Feb 27, 2026 at 12:02 PM Tim Orling via lists.yoctoproject.org
<ticotimo=gmail.com@lists.yoctoproject.org> wrote:

> Add PTEST_RUNNER_TIMEOUT to the variable glossary and mention it in the
> testimage class documentation.
>
> The variable was added to testimage.bbclass to control the per-suite
> timeout for ptest-runner during runtime testing. It defaults to 450
> seconds and can be overridden in image recipes.
>
> Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> ---
>  documentation/ref-manual/classes.rst   |  5 +++++
>  documentation/ref-manual/variables.rst | 14 ++++++++++++++
>  2 files changed, 19 insertions(+)
>
> diff --git a/documentation/ref-manual/classes.rst
> b/documentation/ref-manual/classes.rst
> index eb76c8916..7a045b6b7 100644
> --- a/documentation/ref-manual/classes.rst
> +++ b/documentation/ref-manual/classes.rst
> @@ -3110,6 +3110,11 @@ after it is built, you can set
> :term:`TESTIMAGE_AUTO`::
>
>     TESTIMAGE_AUTO = "1"
>
> +The class defines :term:`PTEST_RUNNER_TIMEOUT`, which controls
> +the per-suite timeout for ``ptest-runner`` during runtime testing.
> +Image recipes can override this variable to adjust the timeout for
> +specific images or multiconfig variants.
> +
>  For information on how to enable, run, and create new tests, see the
>  ":ref:`test-manual/runtime-testing:performing automated runtime testing`"
>  section in the Yocto Project Test Environment Manual.
> diff --git a/documentation/ref-manual/variables.rst
> b/documentation/ref-manual/variables.rst
> index 208b5e43d..587f67cc6 100644
> --- a/documentation/ref-manual/variables.rst
> +++ b/documentation/ref-manual/variables.rst
> @@ -7793,6 +7793,20 @@ system and gives an overview of their function and
> contents.
>        For more information, see
>        the :ref:`ref-classes-ptest-python-pytest` class documentation.
>
> +   :term:`PTEST_RUNNER_TIMEOUT`
> +      Specifies the timeout in seconds for each ptest suite executed by
> +      ``ptest-runner`` during :ref:`ref-classes-testimage` runtime
> testing.
> +      The default value is ``"450"`` (7.5 minutes).
> +
> +      This variable is set in the :ref:`ref-classes-testimage` class and
> can
> +      be overridden in image recipes to adjust the timeout for specific
> +      images or multiconfig variants. For example, in
> ``core-image-ptest``::
> +
> +         PTEST_RUNNER_TIMEOUT:virtclass-mcextend-python3-cffi = "600"
> +
> +      The value is passed to the ``ptest-runner -t`` option on the target
> +      at test execution time.
> +
>     :term:`PV`
>        The version of the recipe. The version is normally extracted from
> the
>        recipe filename. For example, if the recipe is named
> --
> 2.50.1 (Apple Git-155)
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#9014):
> https://lists.yoctoproject.org/g/docs/message/9014
> Mute This Topic: https://lists.yoctoproject.org/mt/118036686/924729
> Group Owner: docs+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [
> ticotimo@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 4461 bytes --]

Re: [docs] [PATCH] ref-manual: document PTEST_RUNNER_TIMEOUT variable

[Antonin Godard]

Hi,

On Fri Feb 27, 2026 at 9:02 PM CET, Tim Orling via lists.yoctoproject.org wrote:
[...]
> diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
> index 208b5e43d..587f67cc6 100644
> --- a/documentation/ref-manual/variables.rst
> +++ b/documentation/ref-manual/variables.rst
> @@ -7793,6 +7793,20 @@ system and gives an overview of their function and contents.
>        For more information, see
>        the :ref:`ref-classes-ptest-python-pytest` class documentation.
>  
> +   :term:`PTEST_RUNNER_TIMEOUT`
> +      Specifies the timeout in seconds for each ptest suite executed by
> +      ``ptest-runner`` during :ref:`ref-classes-testimage` runtime testing.
> +      The default value is ``"450"`` (7.5 minutes).
> +
> +      This variable is set in the :ref:`ref-classes-testimage` class and can
> +      be overridden in image recipes to adjust the timeout for specific
> +      images or multiconfig variants. For example, in ``core-image-ptest``::
> +
> +         PTEST_RUNNER_TIMEOUT:virtclass-mcextend-python3-cffi = "600"

Could you explain why you're using the :virtclass-mcextend-python3-cffi override
here? Seems like a complicated example for this variable's description, but if
it is needed, we might want to explain it?

Antonin

Re: [docs] [PATCH] ref-manual: document PTEST_RUNNER_TIMEOUT variable

[Tim Orling]

[-- Attachment #1: Type: text/plain, Size: 3377 bytes --]

On Tue, Mar 3, 2026 at 2:59 AM Antonin Godard <antonin.godard@bootlin.com>
wrote:

> Hi,
>
> On Fri Feb 27, 2026 at 9:02 PM CET, Tim Orling via lists.yoctoproject.org
> wrote:
> [...]
> > diff --git a/documentation/ref-manual/variables.rst
> b/documentation/ref-manual/variables.rst
> > index 208b5e43d..587f67cc6 100644
> > --- a/documentation/ref-manual/variables.rst
> > +++ b/documentation/ref-manual/variables.rst
> > @@ -7793,6 +7793,20 @@ system and gives an overview of their function
> and contents.
> >        For more information, see
> >        the :ref:`ref-classes-ptest-python-pytest` class documentation.
> >
> > +   :term:`PTEST_RUNNER_TIMEOUT`
> > +      Specifies the timeout in seconds for each ptest suite executed by
> > +      ``ptest-runner`` during :ref:`ref-classes-testimage` runtime
> testing.
> > +      The default value is ``"450"`` (7.5 minutes).
> > +
> > +      This variable is set in the :ref:`ref-classes-testimage` class
> and can
> > +      be overridden in image recipes to adjust the timeout for specific
> > +      images or multiconfig variants. For example, in
> ``core-image-ptest``::
> > +
> > +         PTEST_RUNNER_TIMEOUT:virtclass-mcextend-python3-cffi = "600"
>
> Could you explain why you're using the :virtclass-mcextend-python3-cffi
> override
> here? Seems like a complicated example for this variable's description,
> but if
> it is needed, we might want to explain it?
>

The reason is simple, all the other variables in the "core-image-ptest.bb"
recipe were set with  "virtclass-mcextend-". [1] A prior attempt at setting
with "pn-python3-cffi" in "ptest-packelists.inc" had not worked, although
that was before I caught the need for the "testimage.bbclass" change.

After further experimentation:

1. PTEST_RUNNER_TIMEOUT:pn-python3-cffi = "600" does NOT work [2][3]
2. PTEST_RUNNER_TIMEOUT:pn-core-image-ptest-python3-cffi = "600" DOES work
[4][5]

We also know from Trevor Gamblin experiments that
PTEST_RUNNER_TIMEOUT:riscv-64 = "600" in local.conf works (and similar
experiments with "conf" changes in "yocto-autobuilder-helper" worked). The
goal of the "core-image-ptest.bb" changes were intended to avoid REQUIRING
a change in "conf" only, and instead enable the change to happen in recipe
scope.

So the conclusion appears to be that "image recipe" and "conf" scope work,
but an individual ptest enabled recipe doesnot work. The "image" scope
probably makes sense since PTEST_RUNNER_TIMEOUT is actually enabled by the
"testimage.bbclass"?

Would it make more sense to use the "more obvious" usage in #2 above (even
though it differs from other variable overrides in "core-image-ptest.bb"
recipe)?

[1]
https://git.openembedded.org/openembedded-core/plain/meta/recipes-core/images/core-image-ptest.bb?id=729d81250922f324abf4375b8126382d1c757096
[2] https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/3163
[3]
https://valkyrie.yocto.io/pub/non-release/20260303-115/testresults/qemuarm64-ptest/core-image-ptest-python3-cffi/log.do_testimage.473040.20260303192127
[4] https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/3165
[5]
https://valkyrie.yocto.io/pub/non-release/20260303-117/testresults/qemuarm64-ptest/core-image-ptest-python3-cffi/log.do_testimage.519844.20260303194109

--Tim


> Antonin
>

[-- Attachment #2: Type: text/html, Size: 5222 bytes --]

Re: [docs] [PATCH] ref-manual: document PTEST_RUNNER_TIMEOUT variable

[Antonin Godard]

Hi,

On Tue Mar 3, 2026 at 9:17 PM CET, Tim Orling wrote:
> On Tue, Mar 3, 2026 at 2:59 AM Antonin Godard <antonin.godard@bootlin.com>
> wrote:
>
>> Hi,
>>
>> On Fri Feb 27, 2026 at 9:02 PM CET, Tim Orling via lists.yoctoproject.org
>> wrote:
>> [...]
>> > diff --git a/documentation/ref-manual/variables.rst
>> b/documentation/ref-manual/variables.rst
>> > index 208b5e43d..587f67cc6 100644
>> > --- a/documentation/ref-manual/variables.rst
>> > +++ b/documentation/ref-manual/variables.rst
>> > @@ -7793,6 +7793,20 @@ system and gives an overview of their function
>> and contents.
>> >        For more information, see
>> >        the :ref:`ref-classes-ptest-python-pytest` class documentation.
>> >
>> > +   :term:`PTEST_RUNNER_TIMEOUT`
>> > +      Specifies the timeout in seconds for each ptest suite executed by
>> > +      ``ptest-runner`` during :ref:`ref-classes-testimage` runtime
>> testing.
>> > +      The default value is ``"450"`` (7.5 minutes).
>> > +
>> > +      This variable is set in the :ref:`ref-classes-testimage` class
>> and can
>> > +      be overridden in image recipes to adjust the timeout for specific
>> > +      images or multiconfig variants. For example, in
>> ``core-image-ptest``::
>> > +
>> > +         PTEST_RUNNER_TIMEOUT:virtclass-mcextend-python3-cffi = "600"
>>
>> Could you explain why you're using the :virtclass-mcextend-python3-cffi
>> override
>> here? Seems like a complicated example for this variable's description,
>> but if
>> it is needed, we might want to explain it?
>>
>
> The reason is simple, all the other variables in the "core-image-ptest.bb"
> recipe were set with  "virtclass-mcextend-". [1] A prior attempt at setting
> with "pn-python3-cffi" in "ptest-packelists.inc" had not worked, although
> that was before I caught the need for the "testimage.bbclass" change.
>
> After further experimentation:
>
> 1. PTEST_RUNNER_TIMEOUT:pn-python3-cffi = "600" does NOT work [2][3]
> 2. PTEST_RUNNER_TIMEOUT:pn-core-image-ptest-python3-cffi = "600" DOES work
> [4][5]
>
> We also know from Trevor Gamblin experiments that
> PTEST_RUNNER_TIMEOUT:riscv-64 = "600" in local.conf works (and similar
> experiments with "conf" changes in "yocto-autobuilder-helper" worked). The
> goal of the "core-image-ptest.bb" changes were intended to avoid REQUIRING
> a change in "conf" only, and instead enable the change to happen in recipe
> scope.
>
> So the conclusion appears to be that "image recipe" and "conf" scope work,
> but an individual ptest enabled recipe doesnot work. The "image" scope
> probably makes sense since PTEST_RUNNER_TIMEOUT is actually enabled by the
> "testimage.bbclass"?
>
> Would it make more sense to use the "more obvious" usage in #2 above (even
> though it differs from other variable overrides in "core-image-ptest.bb"
> recipe)?

I think we need to write this variable's documentation as a point of view of
someone using the testimage class, not the core-image-ptest recipe. The
core-image-ptest recipe appears to use some niche use-cases of overrides with
mcextend (which also isn't documented).

So, if I were a user writing an image that inherits the testimage class, and my
image recipe does not contain any mcextend shenanigans, would I really be able
to use syntax #1 or #2?

I can obviously override the value of PTEST_RUNNER_TIMEOUT for all my underlying
ptests, with something like:

my-image.bb:
  IMAGE_CLASSES += "testimage"
  PTEST_RUNNER_TIMEOUT = "1000"

But that's it, I guess?

Antonin