From mboxrd@z Thu Jan 1 00:00:00 1970 From: CKI Project Date: Wed, 15 Jan 2020 06:06:16 -0000 Subject: [LTP] =?utf-8?q?=E2=9D=8C_FAIL=3A_Test_report_for_kernel_5=2E4?= =?utf-8?q?=2E12-rc1-5c903e1=2Ecki_=28stable=29?= Message-ID: List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: ltp@lists.linux.it Hello, We ran automated tests on a recent commit from this kernel tree: Kernel repo: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git Commit: 5c903e10834d - Linux 5.4.12-rc1 The results of these automated tests are provided below. Overall result: FAILED (see details below) Merge: OK Compile: OK Tests: FAILED All kernel binaries, config files, and logs are available for download here: https://artifacts.cki-project.org/pipelines/382907 One or more kernel tests failed: ppc64le: ? LTP aarch64: ? Networking route_func: local ? Networking tunnel: geneve basic test ? Networking tunnel: gre basic ? Networking tunnel: vxlan basic We hope that these logs can help you find the problem quickly. For the full detail on our testing procedures, please scroll to the bottom of this message. Please reply to this email if you have any questions about the tests that we ran or if you have any suggestions on how to make future tests more effective. ,-. ,-. ( C ) ( K ) Continuous `-',-.`-' Kernel ( I ) Integration `-' ______________________________________________________________________________ Compile testing --------------- We compiled the kernel for 3 architectures: aarch64: make options: -j30 INSTALL_MOD_STRIP=1 targz-pkg ppc64le: make options: -j30 INSTALL_MOD_STRIP=1 targz-pkg x86_64: make options: -j30 INSTALL_MOD_STRIP=1 targz-pkg Hardware testing ---------------- We booted each kernel and ran the following tests: aarch64: Host 1: ? Internal infrastructure issues prevented one or more tests (marked with ???) from running on this architecture. This is not the fault of the kernel that was tested. ? Boot test ? xfstests: ext4 ? xfstests: xfs ? selinux-policy: serge-testsuite ? lvm thinp sanity ? storage: software RAID testing ? stress: stress-ng ? ??? IPMI driver test ? ? IPMItool loop stress test ? ? Storage blktests Host 2: ? Boot test ? Podman system integration test (as root) ? Podman system integration test (as user) ? LTP ? Loopdev Sanity ? Memory function: memfd_create ? AMTU (Abstract Machine Test Utility) ? Networking bridge: sanity ? Ethernet drivers sanity ? Networking MACsec: sanity ? Networking socket: fuzz ? Networking sctp-auth: sockopts test ? Networking: igmp conformance test ? Networking route: pmtu ? Networking route_func: local ? Networking route_func: forward ? Networking TCP: keepalive test ? Networking UDP: socket ? Networking tunnel: geneve basic test ? Networking tunnel: gre basic ? L2TP basic test ? Networking tunnel: vxlan basic ? Networking ipsec: basic netns transport ? Networking ipsec: basic netns tunnel ? audit: audit testsuite test ? httpd: mod_ssl smoke sanity ? tuned: tune-processes-through-perf ? ALSA PCM loopback test ? ALSA Control (mixer) Userspace Element test ? storage: SCSI VPD ? trace: ftrace/tracer ? ? CIFS Connectathon ? ? POSIX pjd-fstest suites ? ? jvm test suite ? ? Memory function: kaslr ? ? LTP: openposix test suite ? ? Networking vnic: ipvlan/basic ? ? iotop: sanity ? Usex - version 1.9-29 ? storage: dm/common ppc64le: Host 1: ? Boot test ? Podman system integration test (as root) ? Podman system integration test (as user) ? LTP ? Loopdev Sanity ? Memory function: memfd_create ? AMTU (Abstract Machine Test Utility) ? Networking bridge: sanity ? Ethernet drivers sanity ? Networking MACsec: sanity ? Networking socket: fuzz ? Networking sctp-auth: sockopts test ? Networking route: pmtu ? Networking route_func: local ? Networking route_func: forward ? Networking TCP: keepalive test ? Networking UDP: socket ? Networking tunnel: geneve basic test ? Networking tunnel: gre basic ? L2TP basic test ? Networking tunnel: vxlan basic ? Networking ipsec: basic netns tunnel ? audit: audit testsuite test ? httpd: mod_ssl smoke sanity ? tuned: tune-processes-through-perf ? ALSA PCM loopback test ? ALSA Control (mixer) Userspace Element test ? trace: ftrace/tracer ? ? CIFS Connectathon ? ? POSIX pjd-fstest suites ? ? jvm test suite ? ? Memory function: kaslr ? ? LTP: openposix test suite ? ? Networking vnic: ipvlan/basic ? ? iotop: sanity ? ? Usex - version 1.9-29 ? ? storage: dm/common Host 2: ? Boot test ? xfstests: ext4 ? xfstests: xfs ? selinux-policy: serge-testsuite ? lvm thinp sanity ? storage: software RAID testing ? ? IPMI driver test ? ? IPMItool loop stress test ? ? Storage blktests x86_64: Host 1: ? Internal infrastructure issues prevented one or more tests (marked with ???) from running on this architecture. This is not the fault of the kernel that was tested. ? Boot test ? xfstests: ext4 ? xfstests: xfs ? selinux-policy: serge-testsuite ? lvm thinp sanity ? storage: software RAID testing ? stress: stress-ng ? ? IOMMU boot test ? ? IPMI driver test ? ? IPMItool loop stress test ? ??? power-management: cpupower/sanity test ? ? Storage blktests Host 2: ? Internal infrastructure issues prevented one or more tests (marked with ???) from running on this architecture. This is not the fault of the kernel that was tested. ??? Boot test ??? Storage SAN device stress - mpt3sas driver Host 3: ? Boot test ? Podman system integration test (as root) ? Podman system integration test (as user) ? LTP ? Loopdev Sanity ? Memory function: memfd_create ? AMTU (Abstract Machine Test Utility) ? Networking bridge: sanity ? Ethernet drivers sanity ? Networking MACsec: sanity ? Networking socket: fuzz ? Networking sctp-auth: sockopts test ? Networking: igmp conformance test ? Networking route: pmtu ? Networking route_func: local ? Networking route_func: forward ? Networking TCP: keepalive test ? Networking UDP: socket ? Networking tunnel: geneve basic test ? Networking tunnel: gre basic ? L2TP basic test ? Networking tunnel: vxlan basic ? Networking ipsec: basic netns transport ? Networking ipsec: basic netns tunnel ? audit: audit testsuite test ? httpd: mod_ssl smoke sanity ? tuned: tune-processes-through-perf ? pciutils: sanity smoke test ? ALSA PCM loopback test ? ALSA Control (mixer) Userspace Element test ? storage: SCSI VPD ? trace: ftrace/tracer ? ? CIFS Connectathon ? ? POSIX pjd-fstest suites ? ? jvm test suite ? ? Memory function: kaslr ? ? LTP: openposix test suite ? ? Networking vnic: ipvlan/basic ? ? iotop: sanity ? ? Usex - version 1.9-29 ? ? storage: dm/common Host 4: ? Internal infrastructure issues prevented one or more tests (marked with ???) from running on this architecture. This is not the fault of the kernel that was tested. ??? Boot test ??? Storage SAN device stress - megaraid_sas Host 5: ? Boot test ? Storage SAN device stress - mpt3sas driver Host 6: ? Internal infrastructure issues prevented one or more tests (marked with ???) from running on this architecture. This is not the fault of the kernel that was tested. ??? Boot test ??? Storage SAN device stress - megaraid_sas Test sources: https://github.com/CKI-project/tests-beaker ? Pull requests are welcome for new tests or improvements to existing tests! Waived tests ------------ If the test run included waived tests, they are marked with ?. Such tests are executed but their results are not taken into account. Tests are waived when their results are not reliable enough, e.g. when they're just introduced or are being fixed. Testing timeout --------------- We aim to provide a report within reasonable timeframe. Tests that haven't finished running are marked with ?. Reports for non-upstream kernels have a Beaker recipe linked to next to each host. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BDE8C33CB1 for ; Wed, 15 Jan 2020 06:06:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5E07724671 for ; Wed, 15 Jan 2020 06:06:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LgZVL82x" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725999AbgAOGGf (ORCPT ); Wed, 15 Jan 2020 01:06:35 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:26919 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725962AbgAOGGe (ORCPT ); Wed, 15 Jan 2020 01:06:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1579068393; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ki0s9mm66ZJVDUTfkwl5ca4w0GSyzCLveAMlY93SCZY=; b=LgZVL82xjC8PGxMCaZq9RIDCSAsO9blgwrI1TxkxyVUIA/Ro5dcBp4jt5mqu+uLbMqNOqs hB/0EhpKVKyhg9E40iqqQkktgIvtpquyjylBGm1PbtaR37Zk6BC5HxY18FzIkMPHW5pN7g mwBnh7j9eeqKDGla18R1eoQ0lLNI4D0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-63-LjP8K600OUK-jkJkQOFWXQ-1; Wed, 15 Jan 2020 01:06:27 -0500 X-MC-Unique: LjP8K600OUK-jkJkQOFWXQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CC6D7800D4C; Wed, 15 Jan 2020 06:06:26 +0000 (UTC) Received: from [172.54.91.90] (cpt-1046.paas.prod.upshift.rdu2.redhat.com [10.0.19.73]) by smtp.corp.redhat.com (Postfix) with ESMTP id B2AE010372F6; Wed, 15 Jan 2020 06:06:16 +0000 (UTC) Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 From: CKI Project To: Linux Stable maillist Subject: =?utf-8?b?4p2M?= FAIL: Test report for kernel 5.4.12-rc1-5c903e1.cki (stable) Date: Wed, 15 Jan 2020 06:06:16 -0000 CC: Memory Management , Jan Stancek , LTP Mailing List , Jianlin Shi , Jianwen Ji , Hangbin Liu , William Gomeringer Message-ID: X-Gitlab-Pipeline-ID: 382907 X-Gitlab-Url: https://xci32.lab.eng.rdu2.redhat.com X-Gitlab-Path: /cki-project/cki-pipeline/pipelines/382907 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org Hello, We ran automated tests on a recent commit from this kernel tree: Kernel repo: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linu= x-stable-rc.git Commit: 5c903e10834d - Linux 5.4.12-rc1 The results of these automated tests are provided below. Overall result: FAILED (see details below) Merge: OK Compile: OK Tests: FAILED All kernel binaries, config files, and logs are available for download here: https://artifacts.cki-project.org/pipelines/382907 One or more kernel tests failed: ppc64le: =E2=9D=8C LTP aarch64: =E2=9D=8C Networking route_func: local =E2=9D=8C Networking tunnel: geneve basic test =E2=9D=8C Networking tunnel: gre basic =E2=9D=8C Networking tunnel: vxlan basic We hope that these logs can help you find the problem quickly. For the full detail on our testing procedures, please scroll to the bottom of this message. Please reply to this email if you have any questions about the tests that we ran or if you have any suggestions on how to make future tests more effective. ,-. ,-. ( C ) ( K ) Continuous `-',-.`-' Kernel ( I ) Integration `-' ______________________________________________________________________________ Compile testing --------------- We compiled the kernel for 3 architectures: aarch64: make options: -j30 INSTALL_MOD_STRIP=3D1 targz-pkg ppc64le: make options: -j30 INSTALL_MOD_STRIP=3D1 targz-pkg x86_64: make options: -j30 INSTALL_MOD_STRIP=3D1 targz-pkg Hardware testing ---------------- We booted each kernel and ran the following tests: aarch64: Host 1: =E2=9A=A1 Internal infrastructure issues prevented one or more tests (= marked with =E2=9A=A1=E2=9A=A1=E2=9A=A1) from running on this architecture. This is not the fault of the kernel that was tested. =E2=9C=85 Boot test =E2=9C=85 xfstests: ext4 =E2=9C=85 xfstests: xfs =E2=9C=85 selinux-policy: serge-testsuite =E2=9C=85 lvm thinp sanity =E2=9C=85 storage: software RAID testing =E2=9C=85 stress: stress-ng =F0=9F=9A=A7 =E2=9A=A1=E2=9A=A1=E2=9A=A1 IPMI driver test =F0=9F=9A=A7 =E2=9C=85 IPMItool loop stress test =F0=9F=9A=A7 =E2=9C=85 Storage blktests Host 2: =E2=9C=85 Boot test =E2=9C=85 Podman system integration test (as root) =E2=9C=85 Podman system integration test (as user) =E2=9C=85 LTP =E2=9C=85 Loopdev Sanity =E2=9C=85 Memory function: memfd_create =E2=9C=85 AMTU (Abstract Machine Test Utility) =E2=9C=85 Networking bridge: sanity =E2=9C=85 Ethernet drivers sanity =E2=9C=85 Networking MACsec: sanity =E2=9C=85 Networking socket: fuzz =E2=9C=85 Networking sctp-auth: sockopts test =E2=9C=85 Networking: igmp conformance test =E2=9C=85 Networking route: pmtu =E2=9D=8C Networking route_func: local =E2=9C=85 Networking route_func: forward =E2=9C=85 Networking TCP: keepalive test =E2=9C=85 Networking UDP: socket =E2=9D=8C Networking tunnel: geneve basic test =E2=9D=8C Networking tunnel: gre basic =E2=9C=85 L2TP basic test =E2=9D=8C Networking tunnel: vxlan basic =E2=9C=85 Networking ipsec: basic netns transport =E2=9C=85 Networking ipsec: basic netns tunnel =E2=9C=85 audit: audit testsuite test =E2=9C=85 httpd: mod_ssl smoke sanity =E2=9C=85 tuned: tune-processes-through-perf =E2=9C=85 ALSA PCM loopback test =E2=9C=85 ALSA Control (mixer) Userspace Element test =E2=9C=85 storage: SCSI VPD =E2=9C=85 trace: ftrace/tracer =F0=9F=9A=A7 =E2=9C=85 CIFS Connectathon =F0=9F=9A=A7 =E2=9C=85 POSIX pjd-fstest suites =F0=9F=9A=A7 =E2=9C=85 jvm test suite =F0=9F=9A=A7 =E2=9C=85 Memory function: kaslr =F0=9F=9A=A7 =E2=9C=85 LTP: openposix test suite =F0=9F=9A=A7 =E2=9C=85 Networking vnic: ipvlan/basic =F0=9F=9A=A7 =E2=9C=85 iotop: sanity =E2=8F=B1 Usex - version 1.9-29 =E2=8F=B1 storage: dm/common ppc64le: Host 1: =E2=9C=85 Boot test =E2=9C=85 Podman system integration test (as root) =E2=9C=85 Podman system integration test (as user) =E2=9D=8C LTP =E2=9C=85 Loopdev Sanity =E2=9C=85 Memory function: memfd_create =E2=9C=85 AMTU (Abstract Machine Test Utility) =E2=9C=85 Networking bridge: sanity =E2=9C=85 Ethernet drivers sanity =E2=9C=85 Networking MACsec: sanity =E2=9C=85 Networking socket: fuzz =E2=9C=85 Networking sctp-auth: sockopts test =E2=9C=85 Networking route: pmtu =E2=9C=85 Networking route_func: local =E2=9C=85 Networking route_func: forward =E2=9C=85 Networking TCP: keepalive test =E2=9C=85 Networking UDP: socket =E2=9C=85 Networking tunnel: geneve basic test =E2=9C=85 Networking tunnel: gre basic =E2=9C=85 L2TP basic test =E2=9C=85 Networking tunnel: vxlan basic =E2=9C=85 Networking ipsec: basic netns tunnel =E2=9C=85 audit: audit testsuite test =E2=9C=85 httpd: mod_ssl smoke sanity =E2=9C=85 tuned: tune-processes-through-perf =E2=9C=85 ALSA PCM loopback test =E2=9C=85 ALSA Control (mixer) Userspace Element test =E2=9C=85 trace: ftrace/tracer =F0=9F=9A=A7 =E2=9C=85 CIFS Connectathon =F0=9F=9A=A7 =E2=9C=85 POSIX pjd-fstest suites =F0=9F=9A=A7 =E2=9C=85 jvm test suite =F0=9F=9A=A7 =E2=9C=85 Memory function: kaslr =F0=9F=9A=A7 =E2=9C=85 LTP: openposix test suite =F0=9F=9A=A7 =E2=9C=85 Networking vnic: ipvlan/basic =F0=9F=9A=A7 =E2=9C=85 iotop: sanity =F0=9F=9A=A7 =E2=9C=85 Usex - version 1.9-29 =F0=9F=9A=A7 =E2=9C=85 storage: dm/common Host 2: =E2=9C=85 Boot test =E2=9C=85 xfstests: ext4 =E2=9C=85 xfstests: xfs =E2=9C=85 selinux-policy: serge-testsuite =E2=9C=85 lvm thinp sanity =E2=9C=85 storage: software RAID testing =F0=9F=9A=A7 =E2=9C=85 IPMI driver test =F0=9F=9A=A7 =E2=9C=85 IPMItool loop stress test =F0=9F=9A=A7 =E2=9C=85 Storage blktests x86_64: Host 1: =E2=9A=A1 Internal infrastructure issues prevented one or more tests (= marked with =E2=9A=A1=E2=9A=A1=E2=9A=A1) from running on this architecture. This is not the fault of the kernel that was tested. =E2=9C=85 Boot test =E2=9C=85 xfstests: ext4 =E2=9C=85 xfstests: xfs =E2=9C=85 selinux-policy: serge-testsuite =E2=9C=85 lvm thinp sanity =E2=9C=85 storage: software RAID testing =E2=9C=85 stress: stress-ng =F0=9F=9A=A7 =E2=9D=8C IOMMU boot test =F0=9F=9A=A7 =E2=9C=85 IPMI driver test =F0=9F=9A=A7 =E2=9C=85 IPMItool loop stress test =F0=9F=9A=A7 =E2=9A=A1=E2=9A=A1=E2=9A=A1 power-management: cpupower/sa= nity test =F0=9F=9A=A7 =E2=9C=85 Storage blktests Host 2: =E2=9A=A1 Internal infrastructure issues prevented one or more tests (= marked with =E2=9A=A1=E2=9A=A1=E2=9A=A1) from running on this architecture. This is not the fault of the kernel that was tested. =E2=9A=A1=E2=9A=A1=E2=9A=A1 Boot test =E2=9A=A1=E2=9A=A1=E2=9A=A1 Storage SAN device stress - mpt3sas driver Host 3: =E2=9C=85 Boot test =E2=9C=85 Podman system integration test (as root) =E2=9C=85 Podman system integration test (as user) =E2=9C=85 LTP =E2=9C=85 Loopdev Sanity =E2=9C=85 Memory function: memfd_create =E2=9C=85 AMTU (Abstract Machine Test Utility) =E2=9C=85 Networking bridge: sanity =E2=9C=85 Ethernet drivers sanity =E2=9C=85 Networking MACsec: sanity =E2=9C=85 Networking socket: fuzz =E2=9C=85 Networking sctp-auth: sockopts test =E2=9C=85 Networking: igmp conformance test =E2=9C=85 Networking route: pmtu =E2=9C=85 Networking route_func: local =E2=9C=85 Networking route_func: forward =E2=9C=85 Networking TCP: keepalive test =E2=9C=85 Networking UDP: socket =E2=9C=85 Networking tunnel: geneve basic test =E2=9C=85 Networking tunnel: gre basic =E2=9C=85 L2TP basic test =E2=9C=85 Networking tunnel: vxlan basic =E2=9C=85 Networking ipsec: basic netns transport =E2=9C=85 Networking ipsec: basic netns tunnel =E2=9C=85 audit: audit testsuite test =E2=9C=85 httpd: mod_ssl smoke sanity =E2=9C=85 tuned: tune-processes-through-perf =E2=9C=85 pciutils: sanity smoke test =E2=9C=85 ALSA PCM loopback test =E2=9C=85 ALSA Control (mixer) Userspace Element test =E2=9C=85 storage: SCSI VPD =E2=9C=85 trace: ftrace/tracer =F0=9F=9A=A7 =E2=9C=85 CIFS Connectathon =F0=9F=9A=A7 =E2=9C=85 POSIX pjd-fstest suites =F0=9F=9A=A7 =E2=9C=85 jvm test suite =F0=9F=9A=A7 =E2=9C=85 Memory function: kaslr =F0=9F=9A=A7 =E2=9C=85 LTP: openposix test suite =F0=9F=9A=A7 =E2=9C=85 Networking vnic: ipvlan/basic =F0=9F=9A=A7 =E2=9C=85 iotop: sanity =F0=9F=9A=A7 =E2=9C=85 Usex - version 1.9-29 =F0=9F=9A=A7 =E2=9C=85 storage: dm/common Host 4: =E2=9A=A1 Internal infrastructure issues prevented one or more tests (= marked with =E2=9A=A1=E2=9A=A1=E2=9A=A1) from running on this architecture. This is not the fault of the kernel that was tested. =E2=9A=A1=E2=9A=A1=E2=9A=A1 Boot test =E2=9A=A1=E2=9A=A1=E2=9A=A1 Storage SAN device stress - megaraid_sas Host 5: =E2=9C=85 Boot test =E2=9C=85 Storage SAN device stress - mpt3sas driver Host 6: =E2=9A=A1 Internal infrastructure issues prevented one or more tests (= marked with =E2=9A=A1=E2=9A=A1=E2=9A=A1) from running on this architecture. This is not the fault of the kernel that was tested. =E2=9A=A1=E2=9A=A1=E2=9A=A1 Boot test =E2=9A=A1=E2=9A=A1=E2=9A=A1 Storage SAN device stress - megaraid_sas Test sources: https://github.com/CKI-project/tests-beaker =F0=9F=92=9A Pull requests are welcome for new tests or improvements to e= xisting tests! Waived tests ------------ If the test run included waived tests, they are marked with =F0=9F=9A=A7. Suc= h tests are executed but their results are not taken into account. Tests are waived when their results are not reliable enough, e.g. when they're just introduced or a= re being fixed. Testing timeout --------------- We aim to provide a report within reasonable timeframe. Tests that haven't finished running are marked with =E2=8F=B1. Reports for non-upstream kernels = have a Beaker recipe linked to next to each host.