* [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI
@ 2025-10-31 9:41 Philippe Mathieu-Daudé
2025-10-31 9:41 ` [PATCH 1/2] tests/functional: Mark the MIPS replay tests as flaky Philippe Mathieu-Daudé
` (3 more replies)
0 siblings, 4 replies; 13+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-10-31 9:41 UTC (permalink / raw)
To: qemu-devel
Cc: Philippe Mathieu-Daudé, Aurelien Jarno, Jiaxun Yang,
Paolo Bonzini, Alex Bennée
Disable both replay / wheezy tests which have been reported
as flaky since too long.
Philippe Mathieu-Daudé (2):
tests/functional: Mark the MIPS replay tests as flaky
tests/functional: Mark the MIPS Debian Wheezy tests as flaky
tests/functional/mips/test_malta.py | 2 ++
tests/functional/mips/test_replay.py | 2 ++
tests/functional/mips64/test_malta.py | 2 ++
tests/functional/mips64el/test_malta.py | 1 +
tests/functional/mips64el/test_replay.py | 2 ++
tests/functional/mipsel/test_malta.py | 2 ++
6 files changed, 11 insertions(+)
--
2.51.0
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 1/2] tests/functional: Mark the MIPS replay tests as flaky
2025-10-31 9:41 [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI Philippe Mathieu-Daudé
@ 2025-10-31 9:41 ` Philippe Mathieu-Daudé
2025-10-31 10:00 ` Richard Henderson
2025-10-31 11:16 ` Thomas Huth
2025-10-31 9:41 ` [PATCH 2/2] tests/functional: Mark the MIPS Debian Wheezy " Philippe Mathieu-Daudé
` (2 subsequent siblings)
3 siblings, 2 replies; 13+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-10-31 9:41 UTC (permalink / raw)
To: qemu-devel
Cc: Philippe Mathieu-Daudé, Aurelien Jarno, Jiaxun Yang,
Paolo Bonzini, Alex Bennée
MIPS test_replay.py often times out (likely hang) under GitLab CI:
2/21 qemu:func-thorough+func-mips64el-thorough+thorough / func-mips64el-replay TIMEOUT 180.12s killed by signal 15 SIGTERM
The console.log file is empty, and recording.logs only shows:
qemu-system-mips64el: terminating on signal 15 from pid 344
Since this is a long term issue affecting our CI, disable the tests.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
tests/functional/mips/test_replay.py | 2 ++
tests/functional/mips64el/test_replay.py | 2 ++
2 files changed, 4 insertions(+)
diff --git a/tests/functional/mips/test_replay.py b/tests/functional/mips/test_replay.py
index 4327481e35b..747835bf008 100755
--- a/tests/functional/mips/test_replay.py
+++ b/tests/functional/mips/test_replay.py
@@ -5,6 +5,7 @@
# SPDX-License-Identifier: GPL-2.0-or-later
from qemu_test import Asset, skipSlowTest
+from qemu_test import skipFlakyTest
from replay_kernel import ReplayKernelBase
@@ -16,6 +17,7 @@ class MipsReplay(ReplayKernelBase):
'linux-image-2.6.32-5-4kc-malta_2.6.32-48_mips.deb'),
'16ca524148afb0626f483163e5edf352bc1ab0e4fc7b9f9d473252762f2c7a43')
+ @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/2013")
def test_replay_mips_malta(self):
self.set_machine('malta')
kernel_path = self.archive_extract(self.ASSET_KERNEL_2_63_2,
diff --git a/tests/functional/mips64el/test_replay.py b/tests/functional/mips64el/test_replay.py
index 26a6ccff3f7..05cc585f854 100755
--- a/tests/functional/mips64el/test_replay.py
+++ b/tests/functional/mips64el/test_replay.py
@@ -5,6 +5,7 @@
# SPDX-License-Identifier: GPL-2.0-or-later
from qemu_test import Asset, skipUntrustedTest
+from qemu_test import skipFlakyTest
from replay_kernel import ReplayKernelBase
@@ -16,6 +17,7 @@ class Mips64elReplay(ReplayKernelBase):
'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb'),
'35eb476f03be589824b0310358f1c447d85e645b88cbcd2ac02b97ef560f9f8d')
+ @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/2013")
def test_replay_mips64el_malta(self):
self.set_machine('malta')
kernel_path = self.archive_extract(self.ASSET_KERNEL_2_63_2,
--
2.51.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 2/2] tests/functional: Mark the MIPS Debian Wheezy tests as flaky
2025-10-31 9:41 [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI Philippe Mathieu-Daudé
2025-10-31 9:41 ` [PATCH 1/2] tests/functional: Mark the MIPS replay tests as flaky Philippe Mathieu-Daudé
@ 2025-10-31 9:41 ` Philippe Mathieu-Daudé
2025-10-31 11:33 ` Thomas Huth
2025-10-31 10:00 ` [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI Richard Henderson
2025-10-31 11:00 ` Philippe Mathieu-Daudé
3 siblings, 1 reply; 13+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-10-31 9:41 UTC (permalink / raw)
To: qemu-devel
Cc: Philippe Mathieu-Daudé, Aurelien Jarno, Jiaxun Yang,
Paolo Bonzini, Alex Bennée, Daniel P. Berrangé
test_malta.py sometimes times out (likely hang) under GitLab CI:
1/57 qemu:func-thorough+func-mips-thorough+thorough / func-mips-malta TIMEOUT 480.11s killed by signal 15 SIGTERM
console.log shows a soft lockup failure:
06:46,426: INIT: version 2.88 booting
06:46,942: [[36minfo[39;49m] Using makefile-style concurrent boot in runlevel S.
06:47,378: findfs: unable to resolve 'UUID=042f1883-e9a5-4801-bb9b-667b5c8e87ea'
06:50,448: [....] Starting the hotplug events dispatcher: udevd[?25l[?1c7[1G[[32m ok [39;49m8[?25h[?0c.
06:52,269: [....] Synthesizing the initial hotplug events...module e1000: dangerous R_MIPS_LO16 REL relocation
07:17,707: BUG: soft lockup - CPU#0 stuck for 22s! [modprobe:208]
07:17,707: Modules linked in:
07:17,707: Cpu 0
07:17,708: $ 0 : 00000000 1000a400 0000003d 87808b00
07:17,708: $ 4 : 87808b00 87808bf0 00000000 00000000
07:17,709: $ 8 : 86862100 86862100 86862100 86862100
07:17,709: $12 : 86862100 00000000 00000001 86862100
07:17,709: $16 : 87808a00 86862100 1000a401 c008fa60
07:17,709: $20 : 86862100 8041d230 00000000 ffff0000
07:17,710: $24 : 00000000 77711470
07:17,710: $28 : 87bb6000 87bb7df8 8041d230 801f7388
07:17,710: Hi : 00000000
07:17,710: Lo : 00000000
07:17,711: epc : 801f7308 kfree+0x104/0x19c
07:17,711: Not tainted
07:17,711: ra : 801f7388 kfree+0x184/0x19c
07:17,712: Status: 1000a403 KERNEL EXL IE
07:17,712: Cause : 50808000
07:17,712: PrId : 00019300 (MIPS 24Kc)
07:45,707: BUG: soft lockup - CPU#0 stuck for 22s! [modprobe:208]
07:45,707: Modules linked in:
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
tests/functional/mips/test_malta.py | 2 ++
tests/functional/mips64/test_malta.py | 2 ++
tests/functional/mips64el/test_malta.py | 1 +
tests/functional/mipsel/test_malta.py | 2 ++
4 files changed, 7 insertions(+)
diff --git a/tests/functional/mips/test_malta.py b/tests/functional/mips/test_malta.py
index 30279f0ff21..7a734bc069b 100755
--- a/tests/functional/mips/test_malta.py
+++ b/tests/functional/mips/test_malta.py
@@ -9,6 +9,7 @@
import os
from qemu_test import LinuxKernelTest, Asset, wait_for_console_pattern
+from qemu_test import skipFlakyTest
from qemu_test import exec_command_and_wait_for_pattern
@@ -181,6 +182,7 @@ def test_mips_malta_cpio(self):
'debian_wheezy_mips_standard.qcow2'),
'de03599285b8382ad309309a6c4869f6c6c42a5cfc983342bab9ec0dfa7849a2')
+ @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/3109")
def test_wheezy(self):
kernel_path = self.ASSET_WHEEZY_KERNEL.fetch()
image_path = self.ASSET_WHEEZY_DISK.fetch()
diff --git a/tests/functional/mips64/test_malta.py b/tests/functional/mips64/test_malta.py
index a553d3c5bc7..91c57c56af0 100755
--- a/tests/functional/mips64/test_malta.py
+++ b/tests/functional/mips64/test_malta.py
@@ -5,6 +5,7 @@
# SPDX-License-Identifier: GPL-2.0-or-later
from qemu_test import LinuxKernelTest, Asset
+from qemu_test import skipFlakyTest
from mips.test_malta import mips_check_wheezy
@@ -20,6 +21,7 @@ class MaltaMachineConsole(LinuxKernelTest):
'debian_wheezy_mips_standard.qcow2'),
'de03599285b8382ad309309a6c4869f6c6c42a5cfc983342bab9ec0dfa7849a2')
+ @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/3109")
def test_wheezy(self):
kernel_path = self.ASSET_WHEEZY_KERNEL.fetch()
image_path = self.ASSET_WHEEZY_DISK.fetch()
diff --git a/tests/functional/mips64el/test_malta.py b/tests/functional/mips64el/test_malta.py
index 170147bfcc2..e37463dc291 100755
--- a/tests/functional/mips64el/test_malta.py
+++ b/tests/functional/mips64el/test_malta.py
@@ -102,6 +102,7 @@ def test_mips64el_malta_5KEc_cpio(self):
'debian_wheezy_mipsel_standard.qcow2'),
'454f09ae39f7e6461c84727b927100d2c7813841f2a0a5dce328114887ecf914')
+ @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/3109")
def test_wheezy(self):
kernel_path = self.ASSET_WHEEZY_KERNEL.fetch()
image_path = self.ASSET_WHEEZY_DISK.fetch()
diff --git a/tests/functional/mipsel/test_malta.py b/tests/functional/mipsel/test_malta.py
index 427e163d19d..59ab4a60585 100755
--- a/tests/functional/mipsel/test_malta.py
+++ b/tests/functional/mipsel/test_malta.py
@@ -10,6 +10,7 @@
# SPDX-License-Identifier: GPL-2.0-or-later
from qemu_test import QemuSystemTest, LinuxKernelTest, Asset
+from qemu_test import skipFlakyTest
from qemu_test import interrupt_interactive_console_until_pattern
from qemu_test import wait_for_console_pattern
@@ -69,6 +70,7 @@ def test_mips_malta32el_nanomips_64k_dbg(self):
'debian_wheezy_mipsel_standard.qcow2'),
'454f09ae39f7e6461c84727b927100d2c7813841f2a0a5dce328114887ecf914')
+ @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/3109")
def test_wheezy(self):
kernel_path = self.ASSET_WHEEZY_KERNEL.fetch()
image_path = self.ASSET_WHEEZY_DISK.fetch()
--
2.51.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI
2025-10-31 9:41 [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI Philippe Mathieu-Daudé
2025-10-31 9:41 ` [PATCH 1/2] tests/functional: Mark the MIPS replay tests as flaky Philippe Mathieu-Daudé
2025-10-31 9:41 ` [PATCH 2/2] tests/functional: Mark the MIPS Debian Wheezy " Philippe Mathieu-Daudé
@ 2025-10-31 10:00 ` Richard Henderson
2025-10-31 11:50 ` Richard Henderson
2025-10-31 11:00 ` Philippe Mathieu-Daudé
3 siblings, 1 reply; 13+ messages in thread
From: Richard Henderson @ 2025-10-31 10:00 UTC (permalink / raw)
To: qemu-devel
On 10/31/25 10:41, Philippe Mathieu-Daudé wrote:
> Disable both replay / wheezy tests which have been reported
> as flaky since too long.
>
> Philippe Mathieu-Daudé (2):
> tests/functional: Mark the MIPS replay tests as flaky
> tests/functional: Mark the MIPS Debian Wheezy tests as flaky
>
> tests/functional/mips/test_malta.py | 2 ++
> tests/functional/mips/test_replay.py | 2 ++
> tests/functional/mips64/test_malta.py | 2 ++
> tests/functional/mips64el/test_malta.py | 1 +
> tests/functional/mips64el/test_replay.py | 2 ++
> tests/functional/mipsel/test_malta.py | 2 ++
> 6 files changed, 11 insertions(+)
>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 1/2] tests/functional: Mark the MIPS replay tests as flaky
2025-10-31 9:41 ` [PATCH 1/2] tests/functional: Mark the MIPS replay tests as flaky Philippe Mathieu-Daudé
@ 2025-10-31 10:00 ` Richard Henderson
2025-10-31 10:19 ` Philippe Mathieu-Daudé
2025-10-31 11:16 ` Thomas Huth
1 sibling, 1 reply; 13+ messages in thread
From: Richard Henderson @ 2025-10-31 10:00 UTC (permalink / raw)
To: qemu-devel
On 10/31/25 10:41, Philippe Mathieu-Daudé wrote:
> + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/2013")
If you're going to reference this, you might want to re-open it.
r~
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 1/2] tests/functional: Mark the MIPS replay tests as flaky
2025-10-31 10:00 ` Richard Henderson
@ 2025-10-31 10:19 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 13+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-10-31 10:19 UTC (permalink / raw)
To: Richard Henderson, qemu-devel
On 31/10/25 11:00, Richard Henderson wrote:
> On 10/31/25 10:41, Philippe Mathieu-Daudé wrote:
>> + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/2013")
>
> If you're going to reference this, you might want to re-open it.
I didn't notice it was closed (now re-opened).
Per commit 1f881ea4a44 ("replay: stop us hanging in rr_wait_io_event"):
This fixes most of the failures in replay_kernel.py
^^^^
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI
2025-10-31 9:41 [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI Philippe Mathieu-Daudé
` (2 preceding siblings ...)
2025-10-31 10:00 ` [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI Richard Henderson
@ 2025-10-31 11:00 ` Philippe Mathieu-Daudé
3 siblings, 0 replies; 13+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-10-31 11:00 UTC (permalink / raw)
To: qemu-devel
Cc: Aurelien Jarno, Jiaxun Yang, Paolo Bonzini, Alex Bennée,
Thomas Huth
Cc'ing Thomas
On 31/10/25 10:41, Philippe Mathieu-Daudé wrote:
> Disable both replay / wheezy tests which have been reported
> as flaky since too long.
>
> Philippe Mathieu-Daudé (2):
> tests/functional: Mark the MIPS replay tests as flaky
> tests/functional: Mark the MIPS Debian Wheezy tests as flaky
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 1/2] tests/functional: Mark the MIPS replay tests as flaky
2025-10-31 9:41 ` [PATCH 1/2] tests/functional: Mark the MIPS replay tests as flaky Philippe Mathieu-Daudé
2025-10-31 10:00 ` Richard Henderson
@ 2025-10-31 11:16 ` Thomas Huth
1 sibling, 0 replies; 13+ messages in thread
From: Thomas Huth @ 2025-10-31 11:16 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Aurelien Jarno, Jiaxun Yang, Paolo Bonzini, Alex Bennée
On 31/10/2025 10.41, Philippe Mathieu-Daudé wrote:
> MIPS test_replay.py often times out (likely hang) under GitLab CI:
>
> 2/21 qemu:func-thorough+func-mips64el-thorough+thorough / func-mips64el-replay TIMEOUT 180.12s killed by signal 15 SIGTERM
>
> The console.log file is empty, and recording.logs only shows:
>
> qemu-system-mips64el: terminating on signal 15 from pid 344
>
> Since this is a long term issue affecting our CI, disable the tests.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
> tests/functional/mips/test_replay.py | 2 ++
> tests/functional/mips64el/test_replay.py | 2 ++
> 2 files changed, 4 insertions(+)
>
> diff --git a/tests/functional/mips/test_replay.py b/tests/functional/mips/test_replay.py
> index 4327481e35b..747835bf008 100755
> --- a/tests/functional/mips/test_replay.py
> +++ b/tests/functional/mips/test_replay.py
> @@ -5,6 +5,7 @@
> # SPDX-License-Identifier: GPL-2.0-or-later
>
> from qemu_test import Asset, skipSlowTest
> +from qemu_test import skipFlakyTest
> from replay_kernel import ReplayKernelBase
>
>
> @@ -16,6 +17,7 @@ class MipsReplay(ReplayKernelBase):
> 'linux-image-2.6.32-5-4kc-malta_2.6.32-48_mips.deb'),
> '16ca524148afb0626f483163e5edf352bc1ab0e4fc7b9f9d473252762f2c7a43')
>
> + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/2013")
> def test_replay_mips_malta(self):
> self.set_machine('malta')
> kernel_path = self.archive_extract(self.ASSET_KERNEL_2_63_2,
> diff --git a/tests/functional/mips64el/test_replay.py b/tests/functional/mips64el/test_replay.py
> index 26a6ccff3f7..05cc585f854 100755
> --- a/tests/functional/mips64el/test_replay.py
> +++ b/tests/functional/mips64el/test_replay.py
> @@ -5,6 +5,7 @@
> # SPDX-License-Identifier: GPL-2.0-or-later
>
> from qemu_test import Asset, skipUntrustedTest
> +from qemu_test import skipFlakyTest
> from replay_kernel import ReplayKernelBase
>
>
> @@ -16,6 +17,7 @@ class Mips64elReplay(ReplayKernelBase):
> 'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb'),
> '35eb476f03be589824b0310358f1c447d85e645b88cbcd2ac02b97ef560f9f8d')
>
> + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/2013")
> def test_replay_mips64el_malta(self):
> self.set_machine('malta')
> kernel_path = self.archive_extract(self.ASSET_KERNEL_2_63_2,
Thanks, this also bugs me since a while already (not enough to send a patch
yet, so I'm glad you did it now)!
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/2] tests/functional: Mark the MIPS Debian Wheezy tests as flaky
2025-10-31 9:41 ` [PATCH 2/2] tests/functional: Mark the MIPS Debian Wheezy " Philippe Mathieu-Daudé
@ 2025-10-31 11:33 ` Thomas Huth
2025-10-31 13:30 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 13+ messages in thread
From: Thomas Huth @ 2025-10-31 11:33 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel, Daniel P. Berrangé
Cc: Aurelien Jarno, Jiaxun Yang, Paolo Bonzini, Alex Bennée
On 31/10/2025 10.41, Philippe Mathieu-Daudé wrote:
> test_malta.py sometimes times out (likely hang) under GitLab CI:
>
> 1/57 qemu:func-thorough+func-mips-thorough+thorough / func-mips-malta TIMEOUT 480.11s killed by signal 15 SIGTERM
Do you have an URL from a test job where this happened? I clicked through a
bunch of failed pipelines in the qemu-project, but I only saw failures of
the replay test in recent runs...
Also, does it happen for all mips targets, or only for specific flavors?
Thomas
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI
2025-10-31 10:00 ` [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI Richard Henderson
@ 2025-10-31 11:50 ` Richard Henderson
0 siblings, 0 replies; 13+ messages in thread
From: Richard Henderson @ 2025-10-31 11:50 UTC (permalink / raw)
To: qemu-devel
On 10/31/25 11:00, Richard Henderson wrote:
> On 10/31/25 10:41, Philippe Mathieu-Daudé wrote:
>> Disable both replay / wheezy tests which have been reported
>> as flaky since too long.
>>
>> Philippe Mathieu-Daudé (2):
>> tests/functional: Mark the MIPS replay tests as flaky
>> tests/functional: Mark the MIPS Debian Wheezy tests as flaky
>>
>> tests/functional/mips/test_malta.py | 2 ++
>> tests/functional/mips/test_replay.py | 2 ++
>> tests/functional/mips64/test_malta.py | 2 ++
>> tests/functional/mips64el/test_malta.py | 1 +
>> tests/functional/mips64el/test_replay.py | 2 ++
>> tests/functional/mipsel/test_malta.py | 2 ++
>> 6 files changed, 11 insertions(+)
>>
>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
>
> r~
Queued.
r~
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/2] tests/functional: Mark the MIPS Debian Wheezy tests as flaky
2025-10-31 11:33 ` Thomas Huth
@ 2025-10-31 13:30 ` Philippe Mathieu-Daudé
2025-10-31 13:35 ` Philippe Mathieu-Daudé
2025-10-31 15:11 ` Thomas Huth
0 siblings, 2 replies; 13+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-10-31 13:30 UTC (permalink / raw)
To: Thomas Huth, qemu-devel, Daniel P. Berrangé
Cc: Aurelien Jarno, Jiaxun Yang, Paolo Bonzini, Alex Bennée
On 31/10/25 12:33, Thomas Huth wrote:
> On 31/10/2025 10.41, Philippe Mathieu-Daudé wrote:
>> test_malta.py sometimes times out (likely hang) under GitLab CI:
>>
>> 1/57 qemu:func-thorough+func-mips-thorough+thorough / func-mips-
>> malta TIMEOUT 480.11s killed by signal 15 SIGTERM
>
> Do you have an URL from a test job where this happened? I clicked
> through a bunch of failed pipelines in the qemu-project, but I only saw
> failures of the replay test in recent runs...
I was thinking of https://gitlab.com/philmd/qemu/-/jobs/11869641060
▶ 1/21 test_malta.MaltaMachineConsole.test_wheezy
FAIL
1/21 qemu:func-thorough+func-mips64el-thorough+thorough /
func-mips64el-malta ERROR 25.11s exit status 1
2025-10-28 07:06:59,425: ^[[?25l^[[?1cSearching for RedBoot partition
table in physmap-flash.0 at offset 0x1003f0000
2025-10-28 07:06:59,602: ^[7Creating 3 MTD partitions on "physmap-flash.0":
2025-10-28 07:06:59,603: 0x000000000000-0x000000100000 : "YAMON"
2025-10-28 07:06:59,605: 0x000000100000-0x0000003e0000 : "User FS"
2025-10-28 07:06:59,607: 0x0000003e0000-0x000000400000 : "Board Config"
2025-10-28 07:06:59,876: ^[[1G[^[[32m ok ^[[39;49mCPU 0 Unable to handle
kernel paging request at virtual address 0000000000000028, epc ==
ffffffffc00ed234, ra == ffffffffc00ed210
Is it different that the GitLab issue report?
>
> Also, does it happen for all mips targets, or only for specific flavors?
I can't tell so far.
>
> Thomas
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/2] tests/functional: Mark the MIPS Debian Wheezy tests as flaky
2025-10-31 13:30 ` Philippe Mathieu-Daudé
@ 2025-10-31 13:35 ` Philippe Mathieu-Daudé
2025-10-31 15:11 ` Thomas Huth
1 sibling, 0 replies; 13+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-10-31 13:35 UTC (permalink / raw)
To: Thomas Huth, qemu-devel, Daniel P. Berrangé
Cc: Aurelien Jarno, Jiaxun Yang, Paolo Bonzini, Alex Bennée
On 31/10/25 14:30, Philippe Mathieu-Daudé wrote:
> On 31/10/25 12:33, Thomas Huth wrote:
>> On 31/10/2025 10.41, Philippe Mathieu-Daudé wrote:
>>> test_malta.py sometimes times out (likely hang) under GitLab CI:
>>>
>>> 1/57 qemu:func-thorough+func-mips-thorough+thorough / func-mips-
>>> malta TIMEOUT 480.11s killed by signal 15 SIGTERM
>>
>> Do you have an URL from a test job where this happened? I clicked
>> through a bunch of failed pipelines in the qemu-project, but I only
>> saw failures of the replay test in recent runs...
>
> I was thinking of https://gitlab.com/philmd/qemu/-/jobs/11869641060
>
> ▶ 1/21 test_malta.MaltaMachineConsole.test_wheezy FAIL
> 1/21 qemu:func-thorough+func-mips64el-thorough+thorough / func-
> mips64el-malta ERROR 25.11s exit status 1
>
> 2025-10-28 07:06:59,425: ^[[?25l^[[?1cSearching for RedBoot partition
> table in physmap-flash.0 at offset 0x1003f0000
> 2025-10-28 07:06:59,602: ^[7Creating 3 MTD partitions on "physmap-flash.0":
> 2025-10-28 07:06:59,603: 0x000000000000-0x000000100000 : "YAMON"
> 2025-10-28 07:06:59,605: 0x000000100000-0x0000003e0000 : "User FS"
> 2025-10-28 07:06:59,607: 0x0000003e0000-0x000000400000 : "Board Config"
> 2025-10-28 07:06:59,876: ^[[1G[^[[32m ok ^[[39;49mCPU 0 Unable to handle
> kernel paging request at virtual address 0000000000000028, epc ==
> ffffffffc00ed234, ra == ffffffffc00ed210
Full console.log FTR (before GitLab artifact is removed):
2025-10-28 07:06:49,741: [ 0.000000] Initializing cgroup subsys cpuset
2025-10-28 07:06:49,742: [ 0.000000] Initializing cgroup subsys cpu
2025-10-28 07:06:49,742: [ 0.000000] Linux version 3.2.0-4-5kc-malta
(debian-kernel@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) )
#1 Debian 3.2.51-1
2025-10-28 07:06:49,743: [ 0.000000] bootconsole [early0] enabled
2025-10-28 07:06:49,743: [ 0.000000] CPU revision is: 000182a0 (MIPS
20Kc)
2025-10-28 07:06:49,743: [ 0.000000] FPU revision is: 000f8200
2025-10-28 07:06:49,744: [ 0.000000] Checking for the multiply/shift
bug... no.
2025-10-28 07:06:49,744: [ 0.000000] Checking for the daddiu bug... no.
2025-10-28 07:06:49,745: [ 0.000000] Determined physical RAM map:
2025-10-28 07:06:49,746: [ 0.000000] memory: 0000000000001000 @
0000000000000000 (reserved)
2025-10-28 07:06:49,746: [ 0.000000] memory: 00000000000ef000 @
0000000000001000 (ROM data)
2025-10-28 07:06:49,746: [ 0.000000] memory: 0000000000740000 @
00000000000f0000 (reserved)
2025-10-28 07:06:49,747: [ 0.000000] memory: 00000000077d0000 @
0000000000830000 (usable)
2025-10-28 07:06:49,748: [ 0.000000] Wasting 117376 bytes for
tracking 2096 unused pages
2025-10-28 07:06:49,750: [ 0.000000] Initrd not found or empty -
disabling initrd
2025-10-28 07:06:49,751: [ 0.000000] Zone PFN ranges:
2025-10-28 07:06:49,751: [ 0.000000] DMA 0x00000000 -> 0x00001000
2025-10-28 07:06:49,751: [ 0.000000] Normal 0x00001000 -> 0x00008000
2025-10-28 07:06:49,752: [ 0.000000] Movable zone start PFN for each node
2025-10-28 07:06:49,752: [ 0.000000] early_node_map[1] active PFN ranges
2025-10-28 07:06:49,752: [ 0.000000] 0: 0x00000000 -> 0x00008000
2025-10-28 07:06:49,759: [ 0.000000] Built 1 zonelists in Zone order,
mobility grouping on. Total pages: 32320
2025-10-28 07:06:49,759: [ 0.000000] Kernel command line:
printk.time=0 console=ttyS0 root=/dev/sda1
2025-10-28 07:06:49,760: PID hash table entries: 512 (order: 0, 4096 bytes)
2025-10-28 07:06:49,761: Dentry cache hash table entries: 16384 (order:
5, 131072 bytes)
2025-10-28 07:06:49,761: Inode-cache hash table entries: 8192 (order: 4,
65536 bytes)
2025-10-28 07:06:49,767: Primary instruction cache 32kB, VIVT, 4-way,
linesize 32 bytes.
2025-10-28 07:06:49,768: Primary data cache 32kB, 4-way, PIPT, no
aliases, linesize 32 bytes
2025-10-28 07:06:49,773: Enable cache parity protection for MIPS
20KC/25KF CPUs.
2025-10-28 07:06:49,780: Memory: 120652k/122688k available (4808k kernel
code, 2036k reserved, 1752k data, 244k init, 0k highmem)
2025-10-28 07:06:49,784: NR_IRQS:256
2025-10-28 07:06:51,727: CPU frequency 320.00 MHz
2025-10-28 07:06:51,737: Console: colour dummy device 80x25
2025-10-28 07:06:51,764: Calibrating delay loop... 635.90 BogoMIPS
(lpj=1271808)
2025-10-28 07:06:51,764: pid_max: default: 32768 minimum: 301
2025-10-28 07:06:51,765: Security Framework initialized
2025-10-28 07:06:51,767: AppArmor: AppArmor disabled by boot time parameter
2025-10-28 07:06:51,768: Mount-cache hash table entries: 256
2025-10-28 07:06:51,775: Initializing cgroup subsys cpuacct
2025-10-28 07:06:51,776: Initializing cgroup subsys memory
2025-10-28 07:06:51,777: Initializing cgroup subsys devices
2025-10-28 07:06:51,777: Initializing cgroup subsys freezer
2025-10-28 07:06:51,777: Initializing cgroup subsys net_cls
2025-10-28 07:06:51,777: Initializing cgroup subsys blkio
2025-10-28 07:06:51,778: Initializing cgroup subsys perf_event
2025-10-28 07:06:51,779: Checking for the daddi bug... no.
2025-10-28 07:06:51,795: devtmpfs: initialized
2025-10-28 07:06:51,805: print_constraints: dummy:
2025-10-28 07:06:51,807: NET: Registered protocol family 16
2025-10-28 07:06:51,819: bio: create slab <bio-0> at 0
2025-10-28 07:06:51,822: vgaarb: loaded
2025-10-28 07:06:51,824: SCSI subsystem initialized
2025-10-28 07:06:51,830: pci 0000:00:0a.3: address space collision: [io
0x1100-0x110f] conflicts with GT-64120 PCI I/O [io 0x1000-0x1fffff]
2025-10-28 07:06:51,833: pci 0000:00:0a.3: BAR 14: [io 0x1100-0x110f]
has bogus alignment
2025-10-28 07:06:51,834: pci 0000:00:12.0: BAR 6: assigned [mem
0x10000000-0x1003ffff pref]
2025-10-28 07:06:51,834: pci 0000:00:0a.2: BAR 4: assigned [io
0x1000-0x101f]
2025-10-28 07:06:51,835: pci 0000:00:12.0: BAR 0: assigned [io
0x1020-0x103f]
2025-10-28 07:06:51,835: pci 0000:00:12.0: BAR 1: assigned [mem
0x10040000-0x1004001f]
2025-10-28 07:06:51,835: pci 0000:00:0a.1: BAR 4: assigned [io
0x1040-0x104f]
2025-10-28 07:06:51,841: Switching to clocksource MIPS
2025-10-28 07:06:51,877: NET: Registered protocol family 2
2025-10-28 07:06:51,881: IP route cache hash table entries: 1024 (order:
1, 8192 bytes)
2025-10-28 07:06:51,885: TCP established hash table entries: 4096
(order: 4, 65536 bytes)
2025-10-28 07:06:51,885: TCP bind hash table entries: 4096 (order: 3,
32768 bytes)
2025-10-28 07:06:51,886: TCP: Hash tables configured (established 4096
bind 4096)
2025-10-28 07:06:51,886: TCP reno registered
2025-10-28 07:06:51,886: UDP hash table entries: 128 (order: 0, 4096 bytes)
2025-10-28 07:06:51,887: UDP-Lite hash table entries: 128 (order: 0,
4096 bytes)
2025-10-28 07:06:51,889: NET: Registered protocol family 1
2025-10-28 07:06:51,891: RPC: Registered named UNIX socket transport module.
2025-10-28 07:06:51,891: RPC: Registered udp transport module.
2025-10-28 07:06:51,892: RPC: Registered tcp transport module.
2025-10-28 07:06:51,892: RPC: Registered tcp NFSv4.1 backchannel
transport module.
2025-10-28 07:06:51,893: PCI: Enabling device 0000:00:0a.2 (0000 -> 0001)
2025-10-28 07:06:51,901: audit: initializing netlink socket (disabled)
2025-10-28 07:06:51,902: type=2000 audit(1761635209.160:1): initialized
2025-10-28 07:06:51,905: HugeTLB registered 2 MB page size,
pre-allocated 0 pages
2025-10-28 07:06:51,908: VFS: Disk quotas dquot_6.5.2
2025-10-28 07:06:51,908: Dquot-cache hash table entries: 512 (order 0,
4096 bytes)
2025-10-28 07:06:51,912: nfs4filelayout_init: NFSv4 File Layout Driver
Registering...
2025-10-28 07:06:51,913: msgmni has been set to 235
2025-10-28 07:06:51,921: alg: No test for stdrng (krng)
2025-10-28 07:06:51,921: Block layer SCSI generic (bsg) driver version
0.4 loaded (major 253)
2025-10-28 07:06:51,922: io scheduler noop registered
2025-10-28 07:06:51,922: io scheduler deadline registered
2025-10-28 07:06:51,923: io scheduler cfq registered (default)
2025-10-28 07:06:51,924: Serial: 8250/16550 driver, 4 ports, IRQ sharing
enabled
2025-10-28 07:06:51,952: serial8250.0: ttyS0 at I/O 0x3f8 (irq = 4) is a
16550A
2025-10-28 07:06:51,954: console [ttyS0] enabled, bootconsole disabled
2025-10-28 07:06:51,955: console [ttyS0] enabled, bootconsole disabled
2025-10-28 07:06:51,977: serial8250.0: ttyS1 at I/O 0x2f8 (irq = 3) is a
16550A
2025-10-28 07:06:52,000: serial8250.0: ttyS2 at MMIO 0x1f000900 (irq =
18) is a 16550A
2025-10-28 07:06:52,002: PCI: Enabling device 0000:00:0a.1 (0000 -> 0001)
2025-10-28 07:06:52,009: scsi0 : ata_piix
2025-10-28 07:06:52,010: scsi1 : ata_piix
2025-10-28 07:06:52,011: ata1: PATA max UDMA/33 cmd 0x1f0 ctl 0x3f6
bmdma 0x1040 irq 14
2025-10-28 07:06:52,012: ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376
bmdma 0x1048 irq 15
2025-10-28 07:06:52,014: pcnet32: pcnet32.c:v1.35 21.Apr.2008
tsbogend@alpha.franken.de
2025-10-28 07:06:52,019: PCI: Enabling device 0000:00:12.0 (0000 -> 0003)
2025-10-28 07:06:52,021: pcnet32: PCnet/PCI II 79C970A at 0x1020,
52:54:00:12:34:56 assigned IRQ 11
2025-10-28 07:06:52,023: pcnet32: eth0: registered as PCnet/PCI II 79C970A
2025-10-28 07:06:52,024: pcnet32: 1 cards_found
2025-10-28 07:06:52,026: serio: i8042 KBD port at 0x60,0x64 irq 1
2025-10-28 07:06:52,027: serio: i8042 AUX port at 0x60,0x64 irq 12
2025-10-28 07:06:52,029: mousedev: PS/2 mouse device common for all mice
2025-10-28 07:06:52,033: rtc_cmos rtc_cmos: rtc core: registered
rtc_cmos as rtc0
2025-10-28 07:06:52,034: rtc0: alarms up to one day, 242 bytes nvram
2025-10-28 07:06:52,035: TCP cubic registered
2025-10-28 07:06:52,035: NET: Registered protocol family 17
2025-10-28 07:06:52,037: Registering the dns_resolver key type
2025-10-28 07:06:52,039: registered taskstats version 1
2025-10-28 07:06:52,040: rtc_cmos rtc_cmos: setting system clock to
2025-10-28 07:06:51 UTC (1761635211)
2025-10-28 07:06:52,041: Initializing network drop monitor service
2025-10-28 07:06:52,133: input: AT Raw Set 2 keyboard as
/devices/platform/i8042/serio0/input/input0
2025-10-28 07:06:52,180: ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100
2025-10-28 07:06:52,182: ata2.00: configured for UDMA/33
2025-10-28 07:06:52,183: ata1.00: ATA-7: QEMU HARDDISK, 2.5+, max UDMA/100
2025-10-28 07:06:52,184: ata1.00: 52428800 sectors, multi 16: LBA48
2025-10-28 07:06:52,186: ata1.00: configured for UDMA/33
2025-10-28 07:06:52,196: scsi 0:0:0:0: Direct-Access ATA QEMU
HARDDISK 2.5+ PQ: 0 ANSI: 5
2025-10-28 07:06:52,201: scsi 1:0:0:0: CD-ROM QEMU QEMU
DVD-ROM 2.5+ PQ: 0 ANSI: 5
2025-10-28 07:06:52,204: sd 0:0:0:0: [sda] 52428800 512-byte logical
blocks: (26.8 GB/25.0 GiB)
2025-10-28 07:06:52,206: sd 0:0:0:0: [sda] Write Protect is off
2025-10-28 07:06:52,207: sd 0:0:0:0: [sda] Write cache: enabled, read
cache: enabled, doesn't support DPO or FUA
2025-10-28 07:06:52,219: sda: sda1 sda2 < sda5 >
2025-10-28 07:06:52,226: sd 0:0:0:0: [sda] Attached SCSI disk
2025-10-28 07:06:52,231: EXT4-fs (sda1): couldn't mount as ext3 due to
feature incompatibilities
2025-10-28 07:06:52,233: EXT4-fs (sda1): couldn't mount as ext2 due to
feature incompatibilities
2025-10-28 07:06:52,245: EXT4-fs (sda1): mounted filesystem with ordered
data mode. Opts: (null)
2025-10-28 07:06:52,245: VFS: Mounted root (ext4 filesystem) readonly on
device 8:1.
2025-10-28 07:06:52,247: Freeing prom memory: 956k freed
2025-10-28 07:06:52,265: Freeing unused kernel memory: 244k freed
2025-10-28 07:06:52,567: INIT: version 2.88 booting
2025-10-28 07:06:53,066: [^[[36minfo^[[39;49m] Using makefile-style
concurrent boot in runlevel S.
2025-10-28 07:06:53,458: findfs: unable to resolve
'UUID=e21e0e27-e8b7-4aaa-ac5f-fa6a2f558daf'
2025-10-28 07:06:56,285: [....] Starting the hotplug events dispatcher:
udevd^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.
2025-10-28 07:06:57,684: [....] Synthesizing the initial hotplug
events...piix4_smbus 0000:00:0a.3: SMBus Host Controller at 0x1100,
revision 0
2025-10-28 07:06:57,807: sr0: scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
2025-10-28 07:06:57,808: cdrom: Uniform CD-ROM driver Revision: 3.20
2025-10-28 07:06:57,877: physmap platform flash device: 00400000 at 1e000000
2025-10-28 07:06:58,039: usbcore: registered new interface driver usbfs
2025-10-28 07:06:58,040: usbcore: registered new interface driver hub
2025-10-28 07:06:58,042: usbcore: registered new device driver usb
2025-10-28 07:06:58,049: ehci_hcd: USB 2.0 'Enhanced' Host Controller
(EHCI) Driver
2025-10-28 07:06:58,052: uhci_hcd: USB Universal Host Controller
Interface driver
2025-10-28 07:06:58,076: uhci_hcd 0000:00:0a.2: UHCI Host Controller
2025-10-28 07:06:58,078: uhci_hcd 0000:00:0a.2: new USB bus registered,
assigned bus number 1
2025-10-28 07:06:58,104: uhci_hcd 0000:00:0a.2: irq 10, io base 0x00001000
2025-10-28 07:06:58,158: usb usb1: New USB device found, idVendor=1d6b,
idProduct=0001
2025-10-28 07:06:58,158: usb usb1: New USB device strings: Mfr=3,
Product=2, SerialNumber=1
2025-10-28 07:06:58,158: usb usb1: Product: UHCI Host Controller
2025-10-28 07:06:58,159: usb usb1: Manufacturer: Linux 3.2.0-4-5kc-malta
uhci_hcd
2025-10-28 07:06:58,159: usb usb1: SerialNumber: 0000:00:0a.2
2025-10-28 07:06:58,186: hub 1-0:1.0: USB hub found
2025-10-28 07:06:58,206: hub 1-0:1.0: 2 ports detected
2025-10-28 07:06:58,323: physmap-flash.0: Found 1 x32 devices at 0x0 in
32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000
2025-10-28 07:06:58,335: sd 0:0:0:0: Attached scsi generic sg0 type 0
2025-10-28 07:06:58,335: sr 1:0:0:0: Attached scsi generic sg1 type 5
2025-10-28 07:06:58,352: input: ImExPS/2 Generic Explorer Mouse as
/devices/platform/i8042/serio1/input/input1
2025-10-28 07:06:58,757: Intel/Sharp Extended Query Table at 0x0031
2025-10-28 07:06:58,759: Using buffer write method
2025-10-28 07:06:59,425: ^[[?25l^[[?1cSearching for RedBoot partition
table in physmap-flash.0 at offset 0x1003f0000
2025-10-28 07:06:59,602: ^[7Creating 3 MTD partitions on "physmap-flash.0":
2025-10-28 07:06:59,603: 0x000000000000-0x000000100000 : "YAMON"
2025-10-28 07:06:59,605: 0x000000100000-0x0000003e0000 : "User FS"
2025-10-28 07:06:59,607: 0x0000003e0000-0x000000400000 : "Board Config"
2025-10-28 07:06:59,876: ^[[1G[^[[32m ok ^[[39;49mCPU 0 Unable to handle
kernel paging request at virtual address 0000000000000028, epc ==
ffffffffc00ed234, ra == ffffffffc00ed210
>
> Is it different that the GitLab issue report?
>
>>
>> Also, does it happen for all mips targets, or only for specific flavors?
>
> I can't tell so far.
>
>>
>> Thomas
>>
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/2] tests/functional: Mark the MIPS Debian Wheezy tests as flaky
2025-10-31 13:30 ` Philippe Mathieu-Daudé
2025-10-31 13:35 ` Philippe Mathieu-Daudé
@ 2025-10-31 15:11 ` Thomas Huth
1 sibling, 0 replies; 13+ messages in thread
From: Thomas Huth @ 2025-10-31 15:11 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel, Daniel P. Berrangé
Cc: Aurelien Jarno, Jiaxun Yang, Paolo Bonzini, Alex Bennée
On 31/10/2025 14.30, Philippe Mathieu-Daudé wrote:
> On 31/10/25 12:33, Thomas Huth wrote:
>> On 31/10/2025 10.41, Philippe Mathieu-Daudé wrote:
>>> test_malta.py sometimes times out (likely hang) under GitLab CI:
>>>
>>> 1/57 qemu:func-thorough+func-mips-thorough+thorough / func-mips-
>>> malta TIMEOUT 480.11s killed by signal 15 SIGTERM
>>
>> Do you have an URL from a test job where this happened? I clicked through
>> a bunch of failed pipelines in the qemu-project, but I only saw failures
>> of the replay test in recent runs...
>
> I was thinking of https://gitlab.com/philmd/qemu/-/jobs/11869641060
Thanks! ... I looked through dozens of jobs of the qemu-project (with the
private runners), and I did not spot it there, so maybe that's something
that only occurs with the shared runners from Gitlab? --> Might get
important if someone tries to reproduce the problem.
Thomas
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2025-10-31 15:12 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-10-31 9:41 [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI Philippe Mathieu-Daudé
2025-10-31 9:41 ` [PATCH 1/2] tests/functional: Mark the MIPS replay tests as flaky Philippe Mathieu-Daudé
2025-10-31 10:00 ` Richard Henderson
2025-10-31 10:19 ` Philippe Mathieu-Daudé
2025-10-31 11:16 ` Thomas Huth
2025-10-31 9:41 ` [PATCH 2/2] tests/functional: Mark the MIPS Debian Wheezy " Philippe Mathieu-Daudé
2025-10-31 11:33 ` Thomas Huth
2025-10-31 13:30 ` Philippe Mathieu-Daudé
2025-10-31 13:35 ` Philippe Mathieu-Daudé
2025-10-31 15:11 ` Thomas Huth
2025-10-31 10:00 ` [PATCH 0/2] tests/functional: Disable flaky MIPS tests on our GitLab CI Richard Henderson
2025-10-31 11:50 ` Richard Henderson
2025-10-31 11:00 ` Philippe Mathieu-Daudé
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).