* [linux-linus test] 186932: regressions - FAIL
@ 2024-07-22 4:56 osstest service owner
2024-07-22 7:05 ` Jan Beulich
0 siblings, 1 reply; 5+ messages in thread
From: osstest service owner @ 2024-07-22 4:56 UTC (permalink / raw)
To: xen-devel
flight 186932 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/186932/
Regressions :-(
Tests which did not succeed and are blocking,
including tests which could not be run:
test-arm64-arm64-examine 8 reboot fail REGR. vs. 186827
test-arm64-arm64-xl-credit1 8 xen-boot fail REGR. vs. 186827
test-arm64-arm64-xl-credit2 8 xen-boot fail REGR. vs. 186827
test-arm64-arm64-xl-xsm 8 xen-boot fail REGR. vs. 186827
test-arm64-arm64-xl-vhd 8 xen-boot fail REGR. vs. 186827
test-arm64-arm64-libvirt-xsm 8 xen-boot fail REGR. vs. 186827
test-arm64-arm64-xl-thunderx 8 xen-boot fail REGR. vs. 186827
test-arm64-arm64-libvirt-raw 8 xen-boot fail REGR. vs. 186827
test-arm64-arm64-xl 8 xen-boot fail REGR. vs. 186827
Tests which did not succeed, but are not blocking:
test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop fail like 186827
test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop fail like 186827
test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop fail like 186827
test-armhf-armhf-libvirt 16 saverestore-support-check fail like 186827
test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop fail like 186827
test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 186827
test-amd64-amd64-libvirt 15 migrate-support-check fail never pass
test-amd64-amd64-libvirt-xsm 15 migrate-support-check fail never pass
test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
test-armhf-armhf-xl-credit1 15 migrate-support-check fail never pass
test-armhf-armhf-xl-credit1 16 saverestore-support-check fail never pass
test-armhf-armhf-xl-rtds 15 migrate-support-check fail never pass
test-armhf-armhf-xl-rtds 16 saverestore-support-check fail never pass
test-armhf-armhf-xl 15 migrate-support-check fail never pass
test-armhf-armhf-xl 16 saverestore-support-check fail never pass
test-amd64-amd64-libvirt-qcow2 14 migrate-support-check fail never pass
test-amd64-amd64-libvirt-raw 14 migrate-support-check fail never pass
test-armhf-armhf-xl-arndale 15 migrate-support-check fail never pass
test-armhf-armhf-xl-arndale 16 saverestore-support-check fail never pass
test-amd64-amd64-libvirt-vhd 14 migrate-support-check fail never pass
test-armhf-armhf-xl-qcow2 14 migrate-support-check fail never pass
test-armhf-armhf-xl-qcow2 15 saverestore-support-check fail never pass
test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail never pass
test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail never pass
test-armhf-armhf-libvirt-vhd 14 migrate-support-check fail never pass
test-armhf-armhf-libvirt-vhd 15 saverestore-support-check fail never pass
test-armhf-armhf-xl-credit2 15 migrate-support-check fail never pass
test-armhf-armhf-xl-credit2 16 saverestore-support-check fail never pass
test-armhf-armhf-libvirt 15 migrate-support-check fail never pass
test-armhf-armhf-xl-raw 14 migrate-support-check fail never pass
test-armhf-armhf-xl-raw 15 saverestore-support-check fail never pass
version targeted for testing:
linux 2c9b3512402ed192d1f43f4531fb5da947e72bd0
baseline version:
linux f83e38fc9f1092f8a7a65ff2ea6a1ea6502efaf0
Last test of basis 186827 2024-07-16 19:43:06 Z 5 days
Failing since 186925 2024-07-20 09:44:52 Z 1 days 6 attempts
Testing same since 186927 2024-07-21 01:44:27 Z 1 days 4 attempts
------------------------------------------------------------
1676 people touched revisions under test,
not listing them all
jobs:
build-amd64-xsm pass
build-arm64-xsm pass
build-i386-xsm pass
build-amd64 pass
build-arm64 pass
build-armhf pass
build-i386 pass
build-amd64-libvirt pass
build-arm64-libvirt pass
build-armhf-libvirt pass
build-i386-libvirt pass
build-amd64-pvops pass
build-arm64-pvops pass
build-armhf-pvops pass
build-i386-pvops pass
test-amd64-amd64-xl pass
test-amd64-coresched-amd64-xl pass
test-arm64-arm64-xl fail
test-armhf-armhf-xl pass
test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm pass
test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm pass
test-amd64-amd64-xl-qemut-debianhvm-i386-xsm pass
test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm pass
test-amd64-amd64-libvirt-xsm pass
test-arm64-arm64-libvirt-xsm fail
test-amd64-amd64-xl-xsm pass
test-arm64-arm64-xl-xsm fail
test-amd64-amd64-qemuu-nested-amd fail
test-amd64-amd64-xl-pvhv2-amd pass
test-amd64-amd64-dom0pvh-xl-amd pass
test-amd64-amd64-xl-qemut-debianhvm-amd64 pass
test-amd64-amd64-xl-qemuu-debianhvm-amd64 pass
test-amd64-amd64-qemuu-freebsd11-amd64 pass
test-amd64-amd64-qemuu-freebsd12-amd64 pass
test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
test-amd64-amd64-xl-qemut-win7-amd64 fail
test-amd64-amd64-xl-qemuu-win7-amd64 fail
test-amd64-amd64-xl-qemut-ws16-amd64 fail
test-amd64-amd64-xl-qemuu-ws16-amd64 fail
test-armhf-armhf-xl-arndale pass
test-amd64-amd64-examine-bios pass
test-amd64-amd64-xl-credit1 pass
test-arm64-arm64-xl-credit1 fail
test-armhf-armhf-xl-credit1 pass
test-amd64-amd64-xl-credit2 pass
test-arm64-arm64-xl-credit2 fail
test-armhf-armhf-xl-credit2 pass
test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict pass
test-amd64-amd64-examine pass
test-arm64-arm64-examine fail
test-armhf-armhf-examine pass
test-amd64-amd64-qemuu-nested-intel pass
test-amd64-amd64-xl-pvhv2-intel pass
test-amd64-amd64-dom0pvh-xl-intel pass
test-amd64-amd64-libvirt pass
test-armhf-armhf-libvirt pass
test-amd64-amd64-xl-multivcpu pass
test-armhf-armhf-xl-multivcpu pass
test-amd64-amd64-pair pass
test-amd64-amd64-libvirt-pair pass
test-amd64-amd64-xl-pvshim pass
test-amd64-amd64-pygrub pass
test-amd64-amd64-libvirt-qcow2 pass
test-amd64-amd64-xl-qcow2 pass
test-armhf-armhf-xl-qcow2 pass
test-amd64-amd64-libvirt-raw pass
test-arm64-arm64-libvirt-raw fail
test-amd64-amd64-xl-raw pass
test-armhf-armhf-xl-raw pass
test-amd64-amd64-xl-rtds pass
test-armhf-armhf-xl-rtds pass
test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow pass
test-amd64-amd64-xl-shadow pass
test-arm64-arm64-xl-thunderx fail
test-amd64-amd64-examine-uefi pass
test-amd64-amd64-libvirt-vhd pass
test-armhf-armhf-libvirt-vhd pass
test-amd64-amd64-xl-vhd pass
test-arm64-arm64-xl-vhd fail
------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images
Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs
Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master
Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary
Not pushing.
(No revision log; it would be 178863 lines long.)
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [linux-linus test] 186932: regressions - FAIL
2024-07-22 4:56 [linux-linus test] 186932: regressions - FAIL osstest service owner
@ 2024-07-22 7:05 ` Jan Beulich
2024-07-22 13:03 ` Anthony PERARD
0 siblings, 1 reply; 5+ messages in thread
From: Jan Beulich @ 2024-07-22 7:05 UTC (permalink / raw)
To: osstest service owner
Cc: xen-devel, Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Bertrand Marquis, Michal Orzel
On 22.07.2024 06:56, osstest service owner wrote:
> flight 186932 linux-linus real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/186932/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking,
> including tests which could not be run:
> test-arm64-arm64-examine 8 reboot fail REGR. vs. 186827
> test-arm64-arm64-xl-credit1 8 xen-boot fail REGR. vs. 186827
> test-arm64-arm64-xl-credit2 8 xen-boot fail REGR. vs. 186827
> test-arm64-arm64-xl-xsm 8 xen-boot fail REGR. vs. 186827
> test-arm64-arm64-xl-vhd 8 xen-boot fail REGR. vs. 186827
> test-arm64-arm64-libvirt-xsm 8 xen-boot fail REGR. vs. 186827
> test-arm64-arm64-xl-thunderx 8 xen-boot fail REGR. vs. 186827
> test-arm64-arm64-libvirt-raw 8 xen-boot fail REGR. vs. 186827
> test-arm64-arm64-xl 8 xen-boot fail REGR. vs. 186827
There looks to be a basic problem as of flight 186925, yet a brief look at one
of the logs doesn't really give any hint other than the system perhaps simply
being slow. Ideas, anyone?
Jan
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [linux-linus test] 186932: regressions - FAIL
2024-07-22 7:05 ` Jan Beulich
@ 2024-07-22 13:03 ` Anthony PERARD
2024-07-22 13:06 ` Jan Beulich
0 siblings, 1 reply; 5+ messages in thread
From: Anthony PERARD @ 2024-07-22 13:03 UTC (permalink / raw)
To: Jan Beulich
Cc: osstest service owner, xen-devel, Julien Grall,
Stefano Stabellini, Volodymyr Babchuk, Bertrand Marquis,
Michal Orzel
On Mon, Jul 22, 2024 at 09:05:43AM +0200, Jan Beulich wrote:
> On 22.07.2024 06:56, osstest service owner wrote:
> > flight 186932 linux-linus real [real]
> > http://logs.test-lab.xenproject.org/osstest/logs/186932/
> >
> > Regressions :-(
> >
> > Tests which did not succeed and are blocking,
> > including tests which could not be run:
> > test-arm64-arm64-examine 8 reboot fail REGR. vs. 186827
> > test-arm64-arm64-xl 8 xen-boot fail REGR. vs. 186827
>
> There looks to be a basic problem as of flight 186925, yet a brief look at one
> of the logs doesn't really give any hint other than the system perhaps simply
> being slow. Ideas, anyone?
Well, yes, it's really slow to reach having a running ssh server. If I
let the machine boot, there's two reason in the log for the long time:
Jul 22 11:44:25.216867 Waiting for /dev to be fully populated...Timed out for waiting the udev queue being empty.
Jul 22 11:46:25.469002 done (timeout).
Jul 22 11:46:29.103350 Configuring network interfaces...
Jul 22 11:46:32.127350 ^@Timed out for waiting the udev queue being empty.
Jul 22 11:48:29.256918 [ 403.298102] NET: Registered PF_INET6 protocol family
... more network kernel info after that.
Jul 22 11:48:33.204921 Waiting for xenbr0 to get ready (MAXWAIT is 2 seconds).
Jul 22 11:48:33.204984 done.
(on previous run, "configuring network interface" is followed by
"waiting for xenbr0")
So, we lost already 4 min waiting, out of a budget of 7min for full
reboot.
Also, "reboot" doesn't work with this newer kernel, the machine prints
"Will now restart." then nothing happen.
I have also try without Xen, it's the same behavior.
Cheers,
--
Anthony PERARD
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [linux-linus test] 186932: regressions - FAIL
2024-07-22 13:03 ` Anthony PERARD
@ 2024-07-22 13:06 ` Jan Beulich
2024-07-22 13:41 ` Anthony PERARD
0 siblings, 1 reply; 5+ messages in thread
From: Jan Beulich @ 2024-07-22 13:06 UTC (permalink / raw)
To: Anthony PERARD
Cc: osstest service owner, xen-devel, Julien Grall,
Stefano Stabellini, Volodymyr Babchuk, Bertrand Marquis,
Michal Orzel
On 22.07.2024 15:03, Anthony PERARD wrote:
> On Mon, Jul 22, 2024 at 09:05:43AM +0200, Jan Beulich wrote:
>> On 22.07.2024 06:56, osstest service owner wrote:
>>> flight 186932 linux-linus real [real]
>>> http://logs.test-lab.xenproject.org/osstest/logs/186932/
>>>
>>> Regressions :-(
>>>
>>> Tests which did not succeed and are blocking,
>>> including tests which could not be run:
>>> test-arm64-arm64-examine 8 reboot fail REGR. vs. 186827
>>> test-arm64-arm64-xl 8 xen-boot fail REGR. vs. 186827
>>
>> There looks to be a basic problem as of flight 186925, yet a brief look at one
>> of the logs doesn't really give any hint other than the system perhaps simply
>> being slow. Ideas, anyone?
>
> Well, yes, it's really slow to reach having a running ssh server. If I
> let the machine boot, there's two reason in the log for the long time:
>
> Jul 22 11:44:25.216867 Waiting for /dev to be fully populated...Timed out for waiting the udev queue being empty.
> Jul 22 11:46:25.469002 done (timeout).
>
> Jul 22 11:46:29.103350 Configuring network interfaces...
> Jul 22 11:46:32.127350 ^@Timed out for waiting the udev queue being empty.
Yet both of these instances of timing out look suspiciously like a regression
(in or caused by the kernel)?
Jan
> Jul 22 11:48:29.256918 [ 403.298102] NET: Registered PF_INET6 protocol family
> ... more network kernel info after that.
> Jul 22 11:48:33.204921 Waiting for xenbr0 to get ready (MAXWAIT is 2 seconds).
> Jul 22 11:48:33.204984 done.
>
> (on previous run, "configuring network interface" is followed by
> "waiting for xenbr0")
>
>
> So, we lost already 4 min waiting, out of a budget of 7min for full
> reboot.
>
> Also, "reboot" doesn't work with this newer kernel, the machine prints
> "Will now restart." then nothing happen.
>
> I have also try without Xen, it's the same behavior.
>
> Cheers,
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [linux-linus test] 186932: regressions - FAIL
2024-07-22 13:06 ` Jan Beulich
@ 2024-07-22 13:41 ` Anthony PERARD
0 siblings, 0 replies; 5+ messages in thread
From: Anthony PERARD @ 2024-07-22 13:41 UTC (permalink / raw)
To: Jan Beulich
Cc: osstest service owner, xen-devel, Julien Grall,
Stefano Stabellini, Volodymyr Babchuk, Bertrand Marquis,
Michal Orzel
On Mon, Jul 22, 2024 at 03:06:07PM +0200, Jan Beulich wrote:
> On 22.07.2024 15:03, Anthony PERARD wrote:
> > On Mon, Jul 22, 2024 at 09:05:43AM +0200, Jan Beulich wrote:
> >> On 22.07.2024 06:56, osstest service owner wrote:
> >>> flight 186932 linux-linus real [real]
> >>> http://logs.test-lab.xenproject.org/osstest/logs/186932/
> >>>
> >>> Regressions :-(
> >>>
> >>> Tests which did not succeed and are blocking,
> >>> including tests which could not be run:
> >>> test-arm64-arm64-examine 8 reboot fail REGR. vs. 186827
> >>> test-arm64-arm64-xl 8 xen-boot fail REGR. vs. 186827
> >>
> >> There looks to be a basic problem as of flight 186925, yet a brief look at one
> >> of the logs doesn't really give any hint other than the system perhaps simply
> >> being slow. Ideas, anyone?
> >
> > Well, yes, it's really slow to reach having a running ssh server. If I
> > let the machine boot, there's two reason in the log for the long time:
> >
> > Jul 22 11:44:25.216867 Waiting for /dev to be fully populated...Timed out for waiting the udev queue being empty.
> > Jul 22 11:46:25.469002 done (timeout).
> >
> > Jul 22 11:46:29.103350 Configuring network interfaces...
> > Jul 22 11:46:32.127350 ^@Timed out for waiting the udev queue being empty.
>
> Yet both of these instances of timing out look suspiciously like a regression
> (in or caused by the kernel)?
Yes, no timeout in the logs of previous runs. So it's likely a
regression in Linux, between f83e38fc9f10 and 3c3ff7be9729.
http://logs.test-lab.xenproject.org/osstest/results/history/test-arm64-arm64-xl/linux-linus.html
--
Anthony PERARD
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2024-07-22 13:41 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-07-22 4:56 [linux-linus test] 186932: regressions - FAIL osstest service owner
2024-07-22 7:05 ` Jan Beulich
2024-07-22 13:03 ` Anthony PERARD
2024-07-22 13:06 ` Jan Beulich
2024-07-22 13:41 ` Anthony PERARD
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.