qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
	"Laurent Vivier" <lvivier@redhat.com>,
	"Juan Quintela" <quintela@redhat.com>,
	"Thomas Huth" <thuth@redhat.com>,
	"Cornelia Huck" <cohuck@redhat.com>,
	qemu-s390x@nongnu.org,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	"Daniel P. Berrangé" <berrange@redhat.com>
Subject: [PATCH 1/5] tests: wait max 120 seconds for migration test status changes
Date: Tue, 28 Jun 2022 11:54:30 +0100	[thread overview]
Message-ID: <20220628105434.295905-2-berrange@redhat.com> (raw)
In-Reply-To: <20220628105434.295905-1-berrange@redhat.com>

Currently the wait_for_migration_fail and wait_for_migration_complete
functions will spin in an infinite loop checking query-migrate status
to detect a specific change/goal. This is fine when everything goes
to plan, but when the unusual happens, these will hang the test suite
forever.

Any normally executing migration test case normally takes < 1 second
for a state change, with exception of the autoconverge test which
takes about 5 seconds. Taking into account possibility of people
running tests inside TCG, allowing a factor of x20 slowdown gives
a reasonable worst case of 120 seconds. Anything taking longer than
this is a strong sign that the test has hung, or the test should be
rewritten to be faster.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qtest/migration-helpers.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helpers.c
index a6aa59e4e6..e81e831c85 100644
--- a/tests/qtest/migration-helpers.c
+++ b/tests/qtest/migration-helpers.c
@@ -15,6 +15,14 @@
 
 #include "migration-helpers.h"
 
+/*
+ * Number of seconds we wait when looking for migration
+ * status changes, to avoid test suite hanging forever
+ * when things go wrong. Needs to be higher enough to
+ * avoid false positives on loaded hosts.
+ */
+#define MIGRATION_STATUS_WAIT_TIMEOUT 120
+
 bool got_stop;
 
 static void check_stop_event(QTestState *who)
@@ -166,8 +174,11 @@ static bool check_migration_status(QTestState *who, const char *goal,
 void wait_for_migration_status(QTestState *who,
                                const char *goal, const char **ungoals)
 {
+    g_test_timer_start();
     while (!check_migration_status(who, goal, ungoals)) {
         usleep(1000);
+
+        g_assert(g_test_timer_elapsed() < MIGRATION_STATUS_WAIT_TIMEOUT);
     }
 }
 
@@ -178,6 +189,7 @@ void wait_for_migration_complete(QTestState *who)
 
 void wait_for_migration_fail(QTestState *from, bool allow_active)
 {
+    g_test_timer_start();
     QDict *rsp_return;
     char *status;
     bool failed;
@@ -193,6 +205,8 @@ void wait_for_migration_fail(QTestState *from, bool allow_active)
         g_assert(result);
         failed = !strcmp(status, "failed");
         g_free(status);
+
+        g_assert(g_test_timer_elapsed() < MIGRATION_STATUS_WAIT_TIMEOUT);
     } while (!failed);
 
     /* Is the machine currently running? */
-- 
2.36.1



  reply	other threads:[~2022-06-28 10:57 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-06-28 10:54 [PATCH 0/5] tests: improve reliability of migration test Daniel P. Berrangé
2022-06-28 10:54 ` Daniel P. Berrangé [this message]
2022-06-28 12:47   ` [PATCH 1/5] tests: wait max 120 seconds for migration test status changes Laurent Vivier
2022-06-28 12:49   ` Thomas Huth
2022-06-28 10:54 ` [PATCH 2/5] tests: wait for migration completion before looking for STOP event Daniel P. Berrangé
2022-06-28 12:47   ` Laurent Vivier
2022-06-28 14:08   ` Dr. David Alan Gilbert
2022-06-28 14:10     ` Daniel P. Berrangé
2022-06-28 10:54 ` [PATCH 3/5] tests: increase migration test converge downtime to 30 seconds Daniel P. Berrangé
2022-06-28 12:47   ` Laurent Vivier
2022-06-28 10:54 ` [PATCH 4/5] tests: use consistent bandwidth/downtime limits in migration tests Daniel P. Berrangé
2022-06-28 14:16   ` Dr. David Alan Gilbert
2022-06-28 10:54 ` [RFC PATCH 5/5] tests: stop skipping migration test on s390x/ppc64 Daniel P. Berrangé
2022-06-28 13:18   ` Thomas Huth
2022-07-05  8:06   ` Thomas Huth
2022-07-05  8:09     ` Daniel P. Berrangé
2022-07-05  8:38       ` Dr. David Alan Gilbert
2022-06-28 13:19 ` [PATCH 0/5] tests: improve reliability of migration test Thomas Huth

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220628105434.295905-2-berrange@redhat.com \
    --to=berrange@redhat.com \
    --cc=cohuck@redhat.com \
    --cc=dgilbert@redhat.com \
    --cc=lvivier@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=qemu-s390x@nongnu.org \
    --cc=quintela@redhat.com \
    --cc=thuth@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).