From: "kernelci.org bot" <bot@kernelci.org>
To: stable@vger.kernel.org, kernel-build-reports@lists.linaro.org,
kernelci-results@groups.io
Subject: stable-rc/queue/4.19 baseline: 154 runs, 29 regressions (v4.19.271-79-ge78d989fe44d)
Date: Sat, 04 Feb 2023 10:53:55 -0800 (PST) [thread overview]
Message-ID: <63dea9c3.170a0220.b6ff3.c8bb@mx.google.com> (raw)
stable-rc/queue/4.19 baseline: 154 runs, 29 regressions (v4.19.271-79-ge78d989fe44d)
Regressions Summary
-------------------
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
at91sam9g20ek | arm | lab-broonie | gcc-10 | multi_v5_defconfig | 1
cubietruck | arm | lab-baylibre | gcc-10 | multi_v7_defconfig | 1
meson-gxl-s905x-libretech-cc | arm64 | lab-broonie | gcc-10 | defconfig | 1
qemu_arm64-virt-gicv2 | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1
qemu_arm64-virt-gicv2 | arm64 | lab-baylibre | gcc-10 | defconfig | 1
qemu_arm64-virt-gicv2 | arm64 | lab-broonie | gcc-10 | defconfig+arm64-chromebook | 1
qemu_arm64-virt-gicv2 | arm64 | lab-broonie | gcc-10 | defconfig | 1
qemu_arm64-virt-gicv2 | arm64 | lab-collabora | gcc-10 | defconfig+arm64-chromebook | 1
qemu_arm64-virt-gicv2 | arm64 | lab-collabora | gcc-10 | defconfig | 1
qemu_arm64-virt-gicv2-uefi | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1
qemu_arm64-virt-gicv2-uefi | arm64 | lab-baylibre | gcc-10 | defconfig | 1
qemu_arm64-virt-gicv2-uefi | arm64 | lab-broonie | gcc-10 | defconfig+arm64-chromebook | 1
qemu_arm64-virt-gicv2-uefi | arm64 | lab-broonie | gcc-10 | defconfig | 1
qemu_arm64-virt-gicv2-uefi | arm64 | lab-collabora | gcc-10 | defconfig+arm64-chromebook | 1
qemu_arm64-virt-gicv2-uefi | arm64 | lab-collabora | gcc-10 | defconfig | 1
qemu_arm64-virt-gicv3 | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1
qemu_arm64-virt-gicv3 | arm64 | lab-baylibre | gcc-10 | defconfig | 1
qemu_arm64-virt-gicv3 | arm64 | lab-broonie | gcc-10 | defconfig+arm64-chromebook | 1
qemu_arm64-virt-gicv3 | arm64 | lab-broonie | gcc-10 | defconfig | 1
qemu_arm64-virt-gicv3 | arm64 | lab-collabora | gcc-10 | defconfig+arm64-chromebook | 1
qemu_arm64-virt-gicv3 | arm64 | lab-collabora | gcc-10 | defconfig | 1
qemu_arm64-virt-gicv3-uefi | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1
qemu_arm64-virt-gicv3-uefi | arm64 | lab-baylibre | gcc-10 | defconfig | 1
qemu_arm64-virt-gicv3-uefi | arm64 | lab-broonie | gcc-10 | defconfig+arm64-chromebook | 1
qemu_arm64-virt-gicv3-uefi | arm64 | lab-broonie | gcc-10 | defconfig | 1
qemu_arm64-virt-gicv3-uefi | arm64 | lab-collabora | gcc-10 | defconfig+arm64-chromebook | 1
qemu_arm64-virt-gicv3-uefi | arm64 | lab-collabora | gcc-10 | defconfig | 1
sun50i-a64-pine64-plus | arm64 | lab-baylibre | gcc-10 | defconfig | 1
sun50i-a64-pine64-plus | arm64 | lab-broonie | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/job/stable-rc/branch/queue%2F4.19/kernel/v4.19.271-79-ge78d989fe44d/plan/baseline/
Test: baseline
Tree: stable-rc
Branch: queue/4.19
Describe: v4.19.271-79-ge78d989fe44d
URL: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
SHA: e78d989fe44d32c42cc7b4cc801ae4382e87e3f0
Test Regressions
----------------
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
at91sam9g20ek | arm | lab-broonie | gcc-10 | multi_v5_defconfig | 1
Details: https://kernelci.org/test/plan/id/63de7566dc850f4a6f915eca
Results: 5 PASS, 1 FAIL, 1 SKIP
Full config: multi_v5_defconfig
Compiler: gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm/multi_v5_defconfig/gcc-10/lab-broonie/baseline-at91sam9g20ek.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm/multi_v5_defconfig/gcc-10/lab-broonie/baseline-at91sam9g20ek.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/armel/rootfs.cpio.gz
* baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/63de7566dc850f4a6f915ece
failing since 18 days (last pass: v4.19.269-425-g6a5912d9a0538, first fail: v4.19.269-521-g305d312d039a)
2023-02-04T15:10:12.026406 + set +x
2023-02-04T15:10:12.031570 <8><LAVA_SIGNAL_ENDRUN 0_dmesg 250894_1.5.2.4.1>
2023-02-04T15:10:12.142523 / # #
2023-02-04T15:10:12.244080 export SHELL=/bin/sh
2023-02-04T15:10:12.244477 #
2023-02-04T15:10:12.345757 / # export SHELL=/bin/sh. /lava-250894/environment
2023-02-04T15:10:12.346165
2023-02-04T15:10:12.447458 / # . /lava-250894/environment/lava-250894/bin/lava-test-runner /lava-250894/1
2023-02-04T15:10:12.448045
2023-02-04T15:10:12.454361 / # /lava-250894/bin/lava-test-runner /lava-250894/1
... (12 line(s) more)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
cubietruck | arm | lab-baylibre | gcc-10 | multi_v7_defconfig | 1
Details: https://kernelci.org/test/plan/id/63de76d39a7d40464e915ef2
Results: 5 PASS, 1 FAIL, 1 SKIP
Full config: multi_v7_defconfig
Compiler: gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-cubietruck.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-cubietruck.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/armel/rootfs.cpio.gz
* baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/63de76d39a7d40464e915ef7
failing since 18 days (last pass: v4.19.269-9-gce7b59ec9d48, first fail: v4.19.269-521-g305d312d039a)
2023-02-04T15:16:11.206303 <8>[ 7.339398] <LAVA_SIGNAL_ENDRUN 0_dmesg 3288037_1.5.2.4.1>
2023-02-04T15:16:11.312458 / # #
2023-02-04T15:16:11.413746 export SHELL=/bin/sh
2023-02-04T15:16:11.414110 #
2023-02-04T15:16:11.515267 / # export SHELL=/bin/sh. /lava-3288037/environment
2023-02-04T15:16:11.515628
2023-02-04T15:16:11.616833 / # . /lava-3288037/environment/lava-3288037/bin/lava-test-runner /lava-3288037/1
2023-02-04T15:16:11.617666
2023-02-04T15:16:11.622498 / # /lava-3288037/bin/lava-test-runner /lava-3288037/1
2023-02-04T15:16:11.703887 + export 'TESTRUN_ID=1_bootrr'
... (11 line(s) more)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
meson-gxl-s905x-libretech-cc | arm64 | lab-broonie | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de7a257cacdc9aeb915ec6
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-broonie/baseline-meson-gxl-s905x-libretech-cc.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-broonie/baseline-meson-gxl-s905x-libretech-cc.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de7a257cacdc9aeb915ec7
failing since 291 days (last pass: v4.19.238-22-gb215381f8cf05, first fail: v4.19.238-32-g4d86c9395c31a)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv2 | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1
Details: https://kernelci.org/test/plan/id/63de751af274174ff2915ed6
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig+arm64-chromebook
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv2.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv2.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de751af274174ff2915ed7
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv2 | arm64 | lab-baylibre | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de79b55246bbf878915ec1
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv2.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv2.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de79b55246bbf878915ec2
failing since 270 days (last pass: v4.19.241-58-g5e77acf6dbb6, first fail: v4.19.241-83-g0ec5709aa1da)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv2 | arm64 | lab-broonie | gcc-10 | defconfig+arm64-chromebook | 1
Details: https://kernelci.org/test/plan/id/63de76109b0420b1eb915ec6
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig+arm64-chromebook
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv2.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv2.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de76109b0420b1eb915ec7
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv2 | arm64 | lab-broonie | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de7980d374e8dfb8915ec3
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv2.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv2.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de7980d374e8dfb8915ec4
failing since 270 days (last pass: v4.19.241-58-g5e77acf6dbb6, first fail: v4.19.241-83-g0ec5709aa1da)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv2 | arm64 | lab-collabora | gcc-10 | defconfig+arm64-chromebook | 1
Details: https://kernelci.org/test/plan/id/63de7519f274174ff2915ed2
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig+arm64-chromebook
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv2.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv2.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de7519f274174ff2915ed3
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv2 | arm64 | lab-collabora | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de7981ac6e1f711f915ec4
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv2.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv2.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de7981ac6e1f711f915ec5
failing since 270 days (last pass: v4.19.241-58-g5e77acf6dbb6, first fail: v4.19.241-83-g0ec5709aa1da)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv2-uefi | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1
Details: https://kernelci.org/test/plan/id/63de7519f274174ff2915ecd
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig+arm64-chromebook
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv2-uefi.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv2-uefi.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de7519f274174ff2915ece
failing since 270 days (last pass: v4.19.241-58-g8b40d487da7e, first fail: v4.19.241-83-g0ec5709aa1da)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv2-uefi | arm64 | lab-baylibre | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de79cd259288454a915ec5
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv2-uefi.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv2-uefi.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de79cd259288454a915ec6
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv2-uefi | arm64 | lab-broonie | gcc-10 | defconfig+arm64-chromebook | 1
Details: https://kernelci.org/test/plan/id/63de75fbc6cbf6b6d9915ebf
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig+arm64-chromebook
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv2-uefi.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv2-uefi.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de75fbc6cbf6b6d9915ec0
failing since 270 days (last pass: v4.19.241-58-g8b40d487da7e, first fail: v4.19.241-83-g0ec5709aa1da)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv2-uefi | arm64 | lab-broonie | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de7983ac6e1f711f915ed4
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv2-uefi.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv2-uefi.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de7983ac6e1f711f915ed5
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv2-uefi | arm64 | lab-collabora | gcc-10 | defconfig+arm64-chromebook | 1
Details: https://kernelci.org/test/plan/id/63de752069efa351c0915ec5
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig+arm64-chromebook
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv2-uefi.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv2-uefi.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de752069efa351c0915ec6
failing since 270 days (last pass: v4.19.241-58-g8b40d487da7e, first fail: v4.19.241-83-g0ec5709aa1da)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv2-uefi | arm64 | lab-collabora | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de7982ac6e1f711f915ed0
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv2-uefi.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv2-uefi.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de7982ac6e1f711f915ed1
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv3 | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1
Details: https://kernelci.org/test/plan/id/63de751c5a7d7f5283915ebc
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig+arm64-chromebook
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de751c5a7d7f5283915ebd
failing since 270 days (last pass: v4.19.241-58-g8b40d487da7e, first fail: v4.19.241-83-g0ec5709aa1da)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv3 | arm64 | lab-baylibre | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de79b72d5c8d46c8915ef3
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de79b72d5c8d46c8915ef4
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv3 | arm64 | lab-broonie | gcc-10 | defconfig+arm64-chromebook | 1
Details: https://kernelci.org/test/plan/id/63de7625c3b52e5b90915ec9
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig+arm64-chromebook
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv3.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv3.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de7625c3b52e5b90915eca
failing since 270 days (last pass: v4.19.241-58-g8b40d487da7e, first fail: v4.19.241-83-g0ec5709aa1da)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv3 | arm64 | lab-broonie | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de7994ab01866c63915ebc
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv3.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv3.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de7994ab01866c63915ebd
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv3 | arm64 | lab-collabora | gcc-10 | defconfig+arm64-chromebook | 1
Details: https://kernelci.org/test/plan/id/63de751d5a7d7f5283915ec2
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig+arm64-chromebook
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv3.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv3.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de751d5a7d7f5283915ec3
failing since 270 days (last pass: v4.19.241-58-g8b40d487da7e, first fail: v4.19.241-83-g0ec5709aa1da)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv3 | arm64 | lab-collabora | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de797bac6e1f711f915ebe
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv3.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv3.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de797bac6e1f711f915ebf
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv3-uefi | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1
Details: https://kernelci.org/test/plan/id/63de751b5a7d7f5283915eb9
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig+arm64-chromebook
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3-uefi.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3-uefi.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de751b5a7d7f5283915eba
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv3-uefi | arm64 | lab-baylibre | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de79cafdbbb8534b915eb9
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3-uefi.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3-uefi.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de79cafdbbb8534b915eba
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv3-uefi | arm64 | lab-broonie | gcc-10 | defconfig+arm64-chromebook | 1
Details: https://kernelci.org/test/plan/id/63de76249b0420b1eb915ee4
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig+arm64-chromebook
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv3-uefi.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv3-uefi.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de76249b0420b1eb915ee5
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv3-uefi | arm64 | lab-broonie | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de7981ac6e1f711f915eca
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv3-uefi.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv3-uefi.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de7981ac6e1f711f915ecb
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv3-uefi | arm64 | lab-collabora | gcc-10 | defconfig+arm64-chromebook | 1
Details: https://kernelci.org/test/plan/id/63de751bf274174ff2915ee4
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig+arm64-chromebook
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv3-uefi.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv3-uefi.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de751bf274174ff2915ee5
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
qemu_arm64-virt-gicv3-uefi | arm64 | lab-collabora | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de797ad374e8dfb8915ebd
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv3-uefi.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv3-uefi.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/63de797ad374e8dfb8915ebe
failing since 193 days (last pass: v4.19.230-58-gbd840138c177, first fail: v4.19.253-43-g91137b502cfbd)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
sun50i-a64-pine64-plus | arm64 | lab-baylibre | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de7a4a67f8d9469d915efc
Results: 5 PASS, 1 FAIL, 1 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-baylibre/baseline-sun50i-a64-pine64-plus.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-baylibre/baseline-sun50i-a64-pine64-plus.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/63de7a4a67f8d9469d915f01
failing since 18 days (last pass: v4.19.268-49-g7ddcf2b97a4b, first fail: v4.19.269-521-g305d312d039a)
2023-02-04T15:31:07.234627 <8>[ 15.867351] <LAVA_SIGNAL_ENDRUN 0_dmesg 3288101_1.5.2.4.1>
2023-02-04T15:31:07.354103 / # #
2023-02-04T15:31:07.460045 export SHELL=/bin/sh
2023-02-04T15:31:07.461684 #
2023-02-04T15:31:07.565336 / # export SHELL=/bin/sh. /lava-3288101/environment
2023-02-04T15:31:07.566932
2023-02-04T15:31:07.670641 / # . /lava-3288101/environment/lava-3288101/bin/lava-test-runner /lava-3288101/1
2023-02-04T15:31:07.673555
2023-02-04T15:31:07.676597 / # /lava-3288101/bin/lava-test-runner /lava-3288101/1
2023-02-04T15:31:07.707919 + export 'TESTRUN_ID=1_bootrr'
... (11 line(s) more)
platform | arch | lab | compiler | defconfig | regressions
-----------------------------+-------+---------------+----------+----------------------------+------------
sun50i-a64-pine64-plus | arm64 | lab-broonie | gcc-10 | defconfig | 1
Details: https://kernelci.org/test/plan/id/63de790dbf34b7f451915ee6
Results: 5 PASS, 1 FAIL, 1 SKIP
Full config: defconfig
Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
Plain log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-broonie/baseline-sun50i-a64-pine64-plus.txt
HTML log: https://storage.kernelci.org//stable-rc/queue-4.19/v4.19.271-79-ge78d989fe44d/arm64/defconfig/gcc-10/lab-broonie/baseline-sun50i-a64-pine64-plus.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230127.0/arm64/rootfs.cpio.gz
* baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/63de790dbf34b7f451915eeb
failing since 18 days (last pass: v4.19.268-49-g7ddcf2b97a4b, first fail: v4.19.269-521-g305d312d039a)
2023-02-04T15:25:43.126655 <8>[ 15.910125] <LAVA_SIGNAL_ENDRUN 0_dmesg 251014_1.5.2.4.1>
2023-02-04T15:25:43.233755 / # #
2023-02-04T15:25:43.335869 export SHELL=/bin/sh
2023-02-04T15:25:43.336449 #
2023-02-04T15:25:43.438000 / # export SHELL=/bin/sh. /lava-251014/environment
2023-02-04T15:25:43.439496
2023-02-04T15:25:43.541760 / # . /lava-251014/environment/lava-251014/bin/lava-test-runner /lava-251014/1
2023-02-04T15:25:43.542581
2023-02-04T15:25:43.547198 / # /lava-251014/bin/lava-test-runner /lava-251014/1
2023-02-04T15:25:43.578538 + export 'TESTRUN_ID=1_bootrr'
... (11 line(s) more)
reply other threads:[~2023-02-04 18:54 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=63dea9c3.170a0220.b6ff3.c8bb@mx.google.com \
--to=bot@kernelci.org \
--cc=kernel-build-reports@lists.linaro.org \
--cc=kernelci-results@groups.io \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.