Re: ❌ FAIL: Test report for kernel 5.5.0-b3e3082.cki (stable-next)

[Veronika Kabatova <vkabatov@redhat.com>]

----- Original Message -----
> From: "CKI Project" <cki-project@redhat.com>
> To: "Linux Stable maillist" <stable@vger.kernel.org>
> Sent: Monday, February 3, 2020 9:06:21 AM
> Subject: ❌ FAIL: Test report for kernel 5.5.0-b3e3082.cki (stable-next)
> 
> 
> Hello,
> 
> We ran automated tests on a recent commit from this kernel tree:
> 
>        Kernel repo:
>        git://git.kernel.org/pub/scm/linux/kernel/git/sashal/linux-stable.git
>             Commit: b3e3082be48b - ARM: dma-api: fix max_pfn off-by-one error
>             in __dma_supported()
> 
> The results of these automated tests are provided below.
> 
>     Overall result: FAILED (see details below)
>              Merge: OK
>            Compile: OK
>              Tests: FAILED
> 
> All kernel binaries, config files, and logs are available for download here:
> 
>   https://artifacts.cki-project.org/pipelines/417813
> 
> One or more kernel tests failed:
> 
>     ppc64le:
>      ❌ Boot test
> 

[   40.416312] bnx2x 0045:01:00.0: Direct firmware load for bnx2x/bnx2x-e2-7.13.15.0.fw failed with error -2 
[   40.416339] bnx2x: [bnx2x_init_firmware:13556(enP69p1s0f0)]Can't load firmware file bnx2x/bnx2x-e2-7.13.15.0.fw 
[   40.416349] bnx2x: [bnx2x_func_hw_init:6002(enP69p1s0f0)]Error loading firmware 
[   40.416362] bnx2x: [bnx2x_nic_load:2731(enP69p1s0f0)]HW init failed, aborting 
[   40.556594] bnx2x 0045:01:00.1: Direct firmware load for bnx2x/bnx2x-e2-7.13.15.0.fw failed with error -2 
[   40.556621] bnx2x: [bnx2x_init_firmware:13556(enP69p1s0f1)]Can't load firmware file bnx2x/bnx2x-e2-7.13.15.0.fw 
[   40.556631] bnx2x: [bnx2x_func_hw_init:6002(enP69p1s0f1)]Error loading firmware 
[   40.556646] bnx2x: [bnx2x_nic_load:2731(enP69p1s0f1)]HW init failed, aborting 
[   40.706538] bnx2x 0045:01:00.2: Direct firmware load for bnx2x/bnx2x-e2-7.13.15.0.fw failed with error -2 
[   40.706563] bnx2x: [bnx2x_init_firmware:13556(enP69p1s0f2)]Can't load firmware file bnx2x/bnx2x-e2-7.13.15.0.fw 
[   40.706572] bnx2x: [bnx2x_func_hw_init:6002(enP69p1s0f2)]Error loading firmware 
[   40.706587] bnx2x: [bnx2x_nic_load:2731(enP69p1s0f2)]HW init failed, aborting 
[   40.836616] bnx2x 0045:01:00.3: Direct firmware load for bnx2x/bnx2x-e2-7.13.15.0.fw failed with error -2 
[   40.836638] bnx2x: [bnx2x_init_firmware:13556(enP69p1s0f3)]Can't load firmware file bnx2x/bnx2x-e2-7.13.15.0.fw 
[   40.836645] bnx2x: [bnx2x_func_hw_init:6002(enP69p1s0f3)]Error loading firmware 
[   40.836657] bnx2x: [bnx2x_nic_load:2731(enP69p1s0f3)]HW init failed, aborting 

The recipe aborted later due to networking issues, which might be caused by
the messages above.

The messages are not present with kernel-5.4.8-200.fc31.ppc64le which makes
me wonder if something related changed in the kernel, whether it's a planned
change we need to account for or actual breakage? The other ppc64le machine
we ran on didn't use bnx2x.


Veronika

> We hope that these logs can help you find the problem quickly. For the full
> detail on our testing procedures, please scroll to the bottom of this
> message.
> 
> Please reply to this email if you have any questions about the tests that we
> ran or if you have any suggestions on how to make future tests more
> effective.
> 
>         ,-.   ,-.
>        ( C ) ( K )  Continuous
>         `-',-.`-'   Kernel
>           ( I )     Integration
>            `-'
> ______________________________________________________________________________
> 
> Compile testing
> ---------------
> 
> We compiled the kernel for 3 architectures:
> 
>     aarch64:
>       make options: -j30 INSTALL_MOD_STRIP=1 targz-pkg
> 
>     ppc64le:
>       make options: -j30 INSTALL_MOD_STRIP=1 targz-pkg
> 
>     x86_64:
>       make options: -j30 INSTALL_MOD_STRIP=1 targz-pkg
> 
> 
> Hardware testing
> ----------------
> We booted each kernel and ran the following tests:
> 
>   aarch64:
>     Host 1:
>        ✅ Boot test
>        ✅ xfstests: ext4
>        ✅ xfstests: xfs
>        ✅ selinux-policy: serge-testsuite
>        ✅ lvm thinp sanity
>        ✅ storage: software RAID testing
>        🚧 ✅ Storage blktests
> 
>     Host 2:
>        ✅ Boot test
>        ✅ Podman system integration test (as root)
>        ✅ Podman system integration test (as user)
>        ✅ LTP
>        ✅ Loopdev Sanity
>        ✅ Memory function: memfd_create
>        ✅ AMTU (Abstract Machine Test Utility)
>        ✅ Networking bridge: sanity
>        ✅ Ethernet drivers sanity
>        ✅ Networking MACsec: sanity
>        ✅ Networking socket: fuzz
>        ✅ Networking sctp-auth: sockopts test
>        ✅ Networking: igmp conformance test
>        ✅ Networking route: pmtu
>        ✅ Networking route_func: local
>        ✅ Networking route_func: forward
>        ✅ Networking TCP: keepalive test
>        ✅ Networking UDP: socket
>        ✅ Networking tunnel: geneve basic test
>        ✅ Networking tunnel: gre basic
>        ✅ L2TP basic test
>        ✅ Networking tunnel: vxlan basic
>        ✅ Networking ipsec: basic netns transport
>        ✅ Networking ipsec: basic netns tunnel
>        ✅ audit: audit testsuite test
>        ✅ httpd: mod_ssl smoke sanity
>        ✅ tuned: tune-processes-through-perf
>        ✅ ALSA PCM loopback test
>        ✅ ALSA Control (mixer) Userspace Element test
>        ✅ storage: SCSI VPD
>        ✅ trace: ftrace/tracer
>        🚧 ✅ CIFS Connectathon
>        🚧 ✅ POSIX pjd-fstest suites
>        🚧 ✅ jvm test suite
>        🚧 ✅ Memory function: kaslr
>        🚧 ✅ LTP: openposix test suite
>        🚧 ✅ Networking vnic: ipvlan/basic
>        🚧 ✅ iotop: sanity
>        🚧 ✅ Usex - version 1.9-29
>        🚧 ✅ storage: dm/common
> 
>   ppc64le:
>     Host 1:
>        ✅ Boot test
>        ✅ Podman system integration test (as root)
>        ✅ Podman system integration test (as user)
>        ✅ LTP
>        ✅ Loopdev Sanity
>        ✅ Memory function: memfd_create
>        ✅ AMTU (Abstract Machine Test Utility)
>        ✅ Networking bridge: sanity
>        ✅ Ethernet drivers sanity
>        ✅ Networking MACsec: sanity
>        ✅ Networking socket: fuzz
>        ✅ Networking sctp-auth: sockopts test
>        ✅ Networking route: pmtu
>        ✅ Networking route_func: local
>        ✅ Networking route_func: forward
>        ✅ Networking TCP: keepalive test
>        ✅ Networking UDP: socket
>        ✅ Networking tunnel: geneve basic test
>        ✅ Networking tunnel: gre basic
>        ✅ L2TP basic test
>        ✅ Networking tunnel: vxlan basic
>        ✅ Networking ipsec: basic netns tunnel
>        ✅ audit: audit testsuite test
>        ✅ httpd: mod_ssl smoke sanity
>        ✅ tuned: tune-processes-through-perf
>        ✅ ALSA PCM loopback test
>        ✅ ALSA Control (mixer) Userspace Element test
>        ✅ trace: ftrace/tracer
>        🚧 ✅ CIFS Connectathon
>        🚧 ✅ POSIX pjd-fstest suites
>        🚧 ✅ jvm test suite
>        🚧 ✅ Memory function: kaslr
>        🚧 ✅ LTP: openposix test suite
>        🚧 ✅ Networking vnic: ipvlan/basic
>        🚧 ✅ iotop: sanity
>        🚧 ✅ Usex - version 1.9-29
>        🚧 ✅ storage: dm/common
> 
>     Host 2:
>        ❌ Boot test
>        ⚡⚡⚡ xfstests: ext4
>        ⚡⚡⚡ xfstests: xfs
>        ⚡⚡⚡ selinux-policy: serge-testsuite
>        ⚡⚡⚡ lvm thinp sanity
>        ⚡⚡⚡ storage: software RAID testing
>        🚧 ⚡⚡⚡ IPMI driver test
>        🚧 ⚡⚡⚡ IPMItool loop stress test
>        🚧 ⚡⚡⚡ Storage blktests
> 
>   x86_64:
>     Host 1:
>        ✅ Boot test
>        ✅ Storage SAN device stress - mpt3sas driver
> 
>     Host 2:
>        ✅ Boot test
>        ✅ Podman system integration test (as root)
>        ✅ Podman system integration test (as user)
>        ✅ LTP
>        ✅ Loopdev Sanity
>        ✅ Memory function: memfd_create
>        ✅ AMTU (Abstract Machine Test Utility)
>        ✅ Networking bridge: sanity
>        ✅ Ethernet drivers sanity
>        ✅ Networking MACsec: sanity
>        ✅ Networking socket: fuzz
>        ✅ Networking sctp-auth: sockopts test
>        ✅ Networking: igmp conformance test
>        ✅ Networking route: pmtu
>        ✅ Networking route_func: local
>        ✅ Networking route_func: forward
>        ✅ Networking TCP: keepalive test
>        ✅ Networking UDP: socket
>        ✅ Networking tunnel: geneve basic test
>        ✅ Networking tunnel: gre basic
>        ✅ L2TP basic test
>        ✅ Networking tunnel: vxlan basic
>        ✅ Networking ipsec: basic netns transport
>        ✅ Networking ipsec: basic netns tunnel
>        ✅ audit: audit testsuite test
>        ✅ httpd: mod_ssl smoke sanity
>        ✅ tuned: tune-processes-through-perf
>        ✅ pciutils: sanity smoke test
>        ✅ ALSA PCM loopback test
>        ✅ ALSA Control (mixer) Userspace Element test
>        ✅ storage: SCSI VPD
>        ✅ trace: ftrace/tracer
>        🚧 ✅ CIFS Connectathon
>        🚧 ✅ POSIX pjd-fstest suites
>        🚧 ✅ jvm test suite
>        🚧 ✅ Memory function: kaslr
>        🚧 ✅ LTP: openposix test suite
>        🚧 ✅ Networking vnic: ipvlan/basic
>        🚧 ✅ iotop: sanity
>        🚧 ✅ Usex - version 1.9-29
>        🚧 ✅ storage: dm/common
> 
>     Host 3:
>        ✅ Boot test
>        ✅ Storage SAN device stress - megaraid_sas
> 
>     Host 4:
>        ✅ Boot test
>        ✅ xfstests: ext4
>        ✅ xfstests: xfs
>        ✅ selinux-policy: serge-testsuite
>        ✅ lvm thinp sanity
>        ✅ storage: software RAID testing
>        ✅ stress: stress-ng
>        🚧 ✅ IOMMU boot test
>        🚧 ✅ IPMI driver test
>        🚧 ✅ IPMItool loop stress test
>        🚧 ✅ Storage blktests
> 
>   Test sources: https://github.com/CKI-project/tests-beaker
>     💚 Pull requests are welcome for new tests or improvements to existing
>     tests!
> 
> Waived tests
> ------------
> If the test run included waived tests, they are marked with 🚧. Such tests
> are
> executed but their results are not taken into account. Tests are waived when
> their results are not reliable enough, e.g. when they're just introduced or
> are
> being fixed.
> 
> Testing timeout
> ---------------
> We aim to provide a report within reasonable timeframe. Tests that haven't
> finished running are marked with ⏱. Reports for non-upstream kernels have
> a Beaker recipe linked to next to each host.
> 
>