Re: ❌ FAIL: Test report for kernel 5.5.0-rc6-40f39e8.cki (arm-next)

[Veronika Kabatova <vkabatov@redhat.com>]

----- Original Message ----- 

> From: "Ard Biesheuvel" <ard.biesheuvel@linaro.org>
> To: "CKI Project" <cki-project@redhat.com>
> Cc: "Jianwen Ji" <jiji@redhat.com>, "Hangbin Liu" <haliu@redhat.com>,
> "Catalin Marinas" <catalin.marinas@arm.com>, "Jianlin Shi"
> <jishi@redhat.com>, "Will Deacon" <will@kernel.org>, "linux-arm-kernel"
> <linux-arm-kernel@lists.infradead.org>
> Sent: Thursday, January 16, 2020 2:30:06 PM
> Subject: Re: ❌ FAIL: Test report for kernel 5.5.0-rc6-40f39e8.cki (arm-next)

> On Thu, 16 Jan 2020 at 14:20, CKI Project < cki-project@redhat.com > wrote:

> > Hello,
> 

> > We ran automated tests on a recent commit from this kernel tree:
> 

> > Kernel repo: git:// git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
> 
> > Commit: 40f39e8372fe - Merge branch 'for-next/core' into for-kernelci
> 

> > The results of these automated tests are provided below.
> 

> > Overall result: FAILED (see details below)
> 
> > Merge: OK
> 
> > Compile: OK
> 
> > Tests: FAILED
> 

> > All kernel binaries, config files, and logs are available for download
> > here:
> 

> > https://artifacts.cki-project.org/pipelines/385274
> 

> > One or more kernel tests failed:
> 

> > aarch64:
> 
> > ❌ Networking tunnel: gre basic
> 
> > ❌ Networking tunnel: vxlan basic
> 

> > We hope that these logs can help you find the problem quickly. For the full
> 
> > detail on our testing procedures, please scroll to the bottom of this
> > message.
> 

> > Please reply to this email if you have any questions about the tests that
> > we
> 
> > ran or if you have any suggestions on how to make future tests more
> > effective.
> 

> May I kindly suggest that you stop cc'ing the mailing list on automated,
> unmoderated CI results?

> It's great that you run a CI system, but you should really have the minimal
> courtesy to double check failure reports before sending them out into the
> world.

Hi Ard,

we are sorry about that. The testing and emails to the list were requested
by Will a while ago. If he or Catalin change their minds about them we will
stop sending them, in the meanwhile you can always set up a filter.

We are not paid by ARM so the testing is a "strive for the best" effort,
as we have a bunch of ARM machines we're willing to lend to community for
extra coverage. We always cc the test maintainers and also us to take a
look at the issues if they do occur. As Rachel mentioned, we already put
a workaround in place to prevent these issues from happening further so
hopefully you won't get more emails because of these Beaker bugs.


Thanks,
Veronika

> > ,-. ,-.
> 
> > ( C ) ( K ) Continuous
> 
> > `-',-.`-' Kernel
> 
> > ( I ) Integration
> 
> > `-'
> 
> > ______________________________________________________________________________
> 

> > Compile testing
> 
> > ---------------
> 

> > We compiled the kernel for 1 architecture:
> 

> > aarch64:
> 
> > make options: -j30 INSTALL_MOD_STRIP=1 targz-pkg
> 

> > Hardware testing
> 
> > ----------------
> 
> > We booted each kernel and ran the following tests:
> 

> > aarch64:
> 
> > Host 1:
> 
> > ✅ Boot test
> 
> > ✅ Podman system integration test (as root)
> 
> > ✅ Podman system integration test (as user)
> 
> > ✅ LTP
> 
> > ✅ Loopdev Sanity
> 
> > ✅ Memory function: memfd_create
> 
> > ✅ AMTU (Abstract Machine Test Utility)
> 
> > ✅ Networking bridge: sanity
> 
> > ✅ Ethernet drivers sanity
> 
> > ✅ Networking MACsec: sanity
> 
> > ✅ Networking socket: fuzz
> 
> > ✅ Networking sctp-auth: sockopts test
> 
> > ✅ Networking: igmp conformance test
> 
> > ✅ Networking route: pmtu
> 
> > ✅ Networking route_func: local
> 
> > ✅ Networking route_func: forward
> 
> > ✅ Networking TCP: keepalive test
> 
> > ✅ Networking UDP: socket
> 
> > ✅ Networking tunnel: geneve basic test
> 
> > ❌ Networking tunnel: gre basic
> 
> > ✅ L2TP basic test
> 
> > ❌ Networking tunnel: vxlan basic
> 
> > ✅ Networking ipsec: basic netns transport
> 
> > ✅ Networking ipsec: basic netns tunnel
> 
> > ✅ audit: audit testsuite test
> 
> > ✅ httpd: mod_ssl smoke sanity
> 
> > ✅ tuned: tune-processes-through-perf
> 
> > ✅ ALSA PCM loopback test
> 
> > ✅ ALSA Control (mixer) Userspace Element test
> 
> > ✅ storage: SCSI VPD
> 
> > ✅ trace: ftrace/tracer
> 
> > 🚧 ✅ CIFS Connectathon
> 
> > 🚧 ✅ POSIX pjd-fstest suites
> 
> > ⏱ jvm test suite
> 
> > ⏱ Memory function: kaslr
> 
> > ⏱ LTP: openposix test suite
> 
> > ⏱ Networking vnic: ipvlan/basic
> 
> > ⏱ iotop: sanity
> 
> > ⏱ Usex - version 1.9-29
> 
> > ⏱ storage: dm/common
> 

> > Host 2:
> 

> > ⚡ Internal infrastructure issues prevented one or more tests (marked
> 
> > with ⚡⚡⚡) from running on this architecture.
> 
> > This is not the fault of the kernel that was tested.
> 

> > ✅ Boot test
> 
> > ✅ xfstests: ext4
> 
> > ✅ xfstests: xfs
> 
> > ✅ selinux-policy: serge-testsuite
> 
> > ✅ lvm thinp sanity
> 
> > ✅ storage: software RAID testing
> 
> > ✅ stress: stress-ng
> 
> > 🚧 ⚡⚡⚡ IPMI driver test
> 
> > 🚧 ✅ IPMItool loop stress test
> 
> > 🚧 ✅ Storage blktests
> 

> > Test sources: https://github.com/CKI-project/tests-beaker
> 
> > 💚 Pull requests are welcome for new tests or improvements to existing
> > tests!
> 

> > Waived tests
> 
> > ------------
> 
> > If the test run included waived tests, they are marked with 🚧. Such tests
> > are
> 
> > executed but their results are not taken into account. Tests are waived
> > when
> 
> > their results are not reliable enough, e.g. when they're just introduced or
> > are
> 
> > being fixed.
> 

> > Testing timeout
> 
> > ---------------
> 
> > We aim to provide a report within reasonable timeframe. Tests that haven't
> 
> > finished running are marked with ⏱. Reports for non-upstream kernels have
> 
> > a Beaker recipe linked to next to each host.
> 

> > _______________________________________________
> 
> > linux-arm-kernel mailing list
> 
> > linux-arm-kernel@lists.infradead.org
> 
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel