All of lore.kernel.org
 help / color / mirror / Atom feed
From: Simon Glass <sjg@chromium.org>
To: u-boot@lists.denx.de
Cc: Tom Rini <trini@konsulko.com>, Simon Glass <sjg@chromium.org>
Subject: [PATCH v6 05/19] test: Avoid failing skipped tests
Date: Fri, 20 Sep 2024 08:01:40 +0200	[thread overview]
Message-ID: <20240920060158.106612-6-sjg@chromium.org> (raw)
In-Reply-To: <20240920060158.106612-1-sjg@chromium.org>

When a test returns -EAGAIN this should not be considered a failure.
Fix what seems to be a problem case, where the pytests see a failure
when a test has merely been skipped.

We cannot squash the -EAGAIN error in ut_run_test() since the failure
count is incremented by its caller, ut_run_test_live_flat()

The specific example here is on snow, where a test is compiled into the
image but cannot run, so returns -EAGAIN to skip:

    test/py/tests/test_ut.py sssnow # ut bdinfo bdinfo_test_eth
    Test: bdinfo_test_eth: bdinfo.c
    Skipping: Console recording disabled
    test/test-main.c:486, ut_run_test_live_flat(): 0 == ut_run_test(uts,
    test, test->name): Expected 0x0 (0), got 0xfffffff5 (-11)
    Test bdinfo_test_eth failed 1 times
    Skipped: 1, Failures: 1
    snow # F+u-boot-test-reset snow snow

The fix is simply to respect the return code from ut_run_test(), so do
that.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

(no changes since v4)

Changes in v4:
- Expand the commit message

 test/test-main.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/test/test-main.c b/test/test-main.c
index b3d3e24cdce..b613fc6a4d3 100644
--- a/test/test-main.c
+++ b/test/test-main.c
@@ -486,7 +486,7 @@ static int ut_run_test(struct unit_test_state *uts, struct unit_test *test,
 static int ut_run_test_live_flat(struct unit_test_state *uts,
 				 struct unit_test *test)
 {
-	int runs;
+	int runs, ret;
 
 	if ((test->flags & UTF_OTHER_FDT) && !IS_ENABLED(CONFIG_SANDBOX))
 		return skip_test(uts);
@@ -496,8 +496,11 @@ static int ut_run_test_live_flat(struct unit_test_state *uts,
 	if (CONFIG_IS_ENABLED(OF_LIVE)) {
 		if (!(test->flags & UTF_FLAT_TREE)) {
 			uts->of_live = true;
-			ut_assertok(ut_run_test(uts, test, test->name));
-			runs++;
+			ret = ut_run_test(uts, test, test->name);
+			if (ret != -EAGAIN) {
+				ut_assertok(ret);
+				runs++;
+			}
 		}
 	}
 
@@ -521,8 +524,11 @@ static int ut_run_test_live_flat(struct unit_test_state *uts,
 	    (!runs || ut_test_run_on_flattree(test)) &&
 	    !(gd->flags & GD_FLG_FDT_CHANGED)) {
 		uts->of_live = false;
-		ut_assertok(ut_run_test(uts, test, test->name));
-		runs++;
+		ret = ut_run_test(uts, test, test->name);
+		if (ret != -EAGAIN) {
+			ut_assertok(ret);
+			runs++;
+		}
 	}
 
 	return 0;
-- 
2.43.0


  parent reply	other threads:[~2024-09-20  6:03 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-09-20  6:01 [PATCH v6 00/19] labgrid: Provide an integration with Labgrid Simon Glass
2024-09-20  6:01 ` [PATCH v6 01/19] test: Allow signaling that U-Boot is ready Simon Glass
2024-09-23 20:35   ` Tom Rini
2024-09-25 12:49     ` Simon Glass
2024-09-25 17:26       ` Tom Rini
2024-09-26 21:36         ` Simon Glass
2024-09-27  2:51           ` Tom Rini
2024-10-31 18:03             ` Simon Glass
2024-10-31 18:28               ` Tom Rini
2024-11-01 15:33                 ` Simon Glass
2024-11-01 19:02                   ` Tom Rini
2024-11-02 16:32                     ` Simon Glass
2024-11-02 21:39                       ` Tom Rini
2024-09-20  6:01 ` [PATCH v6 02/19] test: Use a constant for the test timeout Simon Glass
2024-09-23 20:35   ` Tom Rini
2024-09-20  6:01 ` [PATCH v6 03/19] test: Release board after tests complete Simon Glass
2024-09-20  6:01 ` [PATCH v6 04/19] test: Allow connecting to a running board Simon Glass
2024-09-23 20:35   ` Tom Rini
2024-09-25 12:50     ` Simon Glass
2024-09-20  6:01 ` Simon Glass [this message]
2024-09-20  6:01 ` [PATCH v6 06/19] test: Create a common function to get the config Simon Glass
2024-09-20  6:01 ` [PATCH v6 07/19] test: Introduce the concept of a role Simon Glass
2024-09-20  6:01 ` [PATCH v6 08/19] test: Move the receive code into a function Simon Glass
2024-09-20  6:01 ` [PATCH v6 09/19] test: Separate out the exception handling Simon Glass
2024-09-20  6:01 ` [PATCH v6 10/19] test: Detect dead connections Simon Glass
2024-09-20  6:01 ` [PATCH v6 11/19] test: Tidy up remaining exceptions Simon Glass
2024-09-20  6:01 ` [PATCH v6 12/19] test: Introduce lab mode Simon Glass
2024-09-20  6:01 ` [PATCH v6 13/19] test: Improve handling of sending commands Simon Glass
2024-09-20  6:01 ` [PATCH v6 14/19] test: Fix mulptiplex_log typo Simon Glass
2024-09-20  6:01 ` [PATCH v6 15/19] test: Avoid double echo when starting up Simon Glass
2024-09-20  6:01 ` [PATCH v6 16/19] test: Try to shut down the lab console gracefully Simon Glass
2024-09-20  6:01 ` [PATCH v6 17/19] test: Add a section for closing the connection Simon Glass
2024-09-20  6:01 ` [PATCH v6 18/19] test: Support testing with two board-builds Simon Glass
2024-09-20  6:01 ` [PATCH v6 19/19] CI: Allow running tests on sjg lab Simon Glass
2024-09-23 20:36 ` [PATCH v6 00/19] labgrid: Provide an integration with Labgrid Tom Rini
2024-09-25 12:50   ` Simon Glass
2024-09-25 17:26     ` Tom Rini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240920060158.106612-6-sjg@chromium.org \
    --to=sjg@chromium.org \
    --cc=trini@konsulko.com \
    --cc=u-boot@lists.denx.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.