All of lore.kernel.org
 help / color / mirror / Atom feed
From: "kernelci.org bot" <bot@kernelci.org>
To: stable@vger.kernel.org, kernel-build-reports@lists.linaro.org,
	kernelci-results@groups.io
Subject: stable-rc/queue/5.10 baseline: 144 runs, 77 regressions (v5.10.165-76-gffe5f229ddc9)
Date: Wed, 25 Jan 2023 11:24:35 -0800 (PST)	[thread overview]
Message-ID: <63d181f3.620a0220.5a092.7971@mx.google.com> (raw)

stable-rc/queue/5.10 baseline: 144 runs, 77 regressions (v5.10.165-76-gffe5f229ddc9)

Regressions Summary
-------------------

platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
am57xx-beagle-x15            | arm   | lab-linaro-lkft | gcc-10   | multi_v7_defconfig         | 1          
am57xx-beagle-x15            | arm   | lab-linaro-lkft | gcc-10   | omap2plus_defconfig        | 1          
at91sam9g20ek                | arm   | lab-broonie     | gcc-10   | at91_dt_defconfig          | 1          
at91sam9g20ek                | arm   | lab-broonie     | gcc-10   | multi_v5_defconfig         | 1          
bcm2711-rpi-4-b              | arm64 | lab-linaro-lkft | gcc-10   | defconfig                  | 1          
beagle-xm                    | arm   | lab-baylibre    | gcc-10   | multi_v7_defconfig         | 1          
beagle-xm                    | arm   | lab-baylibre    | gcc-10   | omap2plus_defconfig        | 1          
beaglebone-black             | arm   | lab-broonie     | gcc-10   | multi_v7_defconfig         | 1          
beaglebone-black             | arm   | lab-broonie     | gcc-10   | omap2plus_defconfig        | 1          
cubietruck                   | arm   | lab-baylibre    | gcc-10   | multi_v7_defconfig         | 3          
cubietruck                   | arm   | lab-baylibre    | gcc-10   | sunxi_defconfig            | 1          
da850-lcdk                   | arm   | lab-baylibre    | gcc-10   | davinci_all_defconfig      | 1          
da850-lcdk                   | arm   | lab-baylibre    | gcc-10   | multi_v5_defconfig         | 1          
dove-cubox                   | arm   | lab-pengutronix | gcc-10   | mvebu_v7_defconfig         | 1          
fsl-ls1088a-rdb              | arm64 | lab-nxp         | gcc-10   | defconfig                  | 2          
imx6dl-riotboard             | arm   | lab-pengutronix | gcc-10   | multi_v7_defconfig         | 2          
imx6qp-wandboard-revd1       | arm   | lab-pengutronix | gcc-10   | imx_v6_v7_defconfig        | 1          
imx6qp-wandboard-revd1       | arm   | lab-pengutronix | gcc-10   | multi_v7_defconfig         | 1          
imx7d-sdb                    | arm   | lab-nxp         | gcc-10   | imx_v6_v7_defconfig        | 1          
imx7d-sdb                    | arm   | lab-nxp         | gcc-10   | multi_v7_defconfig         | 1          
imx8mn-ddr4-evk              | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
imx8mn-ddr4-evk              | arm64 | lab-nxp         | gcc-10   | defconfig                  | 1          
imx8mp-evk                   | arm64 | lab-nxp         | gcc-10   | defconfig                  | 1          
jetson-tk1                   | arm   | lab-baylibre    | gcc-10   | multi_v7_defconfig         | 1          
jetson-tk1                   | arm   | lab-baylibre    | gcc-10   | tegra_defconfig            | 1          
meson-g12a-sei510            | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
meson-g12a-u200              | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
meson-g12a-x96-max           | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
meson-g12b-a311d-khadas-vim3 | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
meson-g12b-a311d-khadas-vim3 | arm64 | lab-collabora   | gcc-10   | defconfig                  | 1          
meson-g12b-odroid-n2         | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
meson-gxbb-p200              | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
meson-gxl-s905d-p230         | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
meson-gxm-khadas-vim2        | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
meson-gxm-q200               | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
meson-sm1-khadas-vim3l       | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
meson-sm1-sei610             | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
mt8173-elm-hana              | arm64 | lab-collabora   | gcc-10   | defconfig+arm64-chromebook | 1          
odroid-xu3                   | arm   | lab-collabora   | gcc-10   | exynos_defconfig           | 1          
odroid-xu3                   | arm   | lab-collabora   | gcc-10   | multi_v7_defconfig         | 1          
rk3399-gru-kevin             | arm64 | lab-collabora   | gcc-10   | defconfig+arm64-chromebook | 1          
rk3399-roc-pc                | arm64 | lab-clabbe      | gcc-10   | defconfig                  | 1          
rk3399-rock-pi-4b            | arm64 | lab-collabora   | gcc-10   | defconfig                  | 2          
sun50i-a64-bananapi-m64      | arm64 | lab-clabbe      | gcc-10   | defconfig                  | 1          
sun50i-a64-pine64-plus       | arm64 | lab-broonie     | gcc-10   | defconfig                  | 1          
sun50i-h5-lib...ch-all-h3-cc | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          
sun50i-h5-lib...ch-all-h3-cc | arm64 | lab-broonie     | gcc-10   | defconfig                  | 1          
sun50i-h6-orangepi-3         | arm64 | lab-clabbe      | gcc-10   | defconfig                  | 1          
sun50i-h6-orangepi-one-plus  | arm64 | lab-clabbe      | gcc-10   | defconfig                  | 1          
sun50i-h6-pine-h64           | arm64 | lab-clabbe      | gcc-10   | defconfig                  | 1          
sun50i-h6-pine-h64           | arm64 | lab-collabora   | gcc-10   | defconfig                  | 1          
sun5i-a13-olinuxino-micro    | arm   | lab-baylibre    | gcc-10   | sunxi_defconfig            | 1          
sun7i-a20-cubieboard2        | arm   | lab-baylibre    | gcc-10   | sunxi_defconfig            | 1          
sun7i-a20-cubieboard2        | arm   | lab-clabbe      | gcc-10   | sunxi_defconfig            | 1          
sun7i-a20-olinuxino-lime2    | arm   | lab-baylibre    | gcc-10   | sunxi_defconfig            | 1          
sun8i-a33-olinuxino          | arm   | lab-clabbe      | gcc-10   | multi_v7_defconfig         | 2          
sun8i-a33-olinuxino          | arm   | lab-clabbe      | gcc-10   | sunxi_defconfig            | 1          
sun8i-a83t-bananapi-m3       | arm   | lab-clabbe      | gcc-10   | multi_v7_defconfig         | 2          
sun8i-a83t-bananapi-m3       | arm   | lab-clabbe      | gcc-10   | sunxi_defconfig            | 1          
sun8i-h2-plus...ch-all-h3-cc | arm   | lab-baylibre    | gcc-10   | multi_v7_defconfig         | 2          
sun8i-h2-plus...ch-all-h3-cc | arm   | lab-baylibre    | gcc-10   | sunxi_defconfig            | 1          
sun8i-h3-libretech-all-h3-cc | arm   | lab-baylibre    | gcc-10   | multi_v7_defconfig         | 2          
sun8i-h3-libretech-all-h3-cc | arm   | lab-baylibre    | gcc-10   | sunxi_defconfig            | 1          
sun8i-h3-orangepi-pc         | arm   | lab-clabbe      | gcc-10   | multi_v7_defconfig         | 2          
sun8i-h3-orangepi-pc         | arm   | lab-clabbe      | gcc-10   | sunxi_defconfig            | 1          
sun8i-r40-bananapi-m2-ultra  | arm   | lab-clabbe      | gcc-10   | multi_v7_defconfig         | 1          
sun8i-r40-bananapi-m2-ultra  | arm   | lab-clabbe      | gcc-10   | sunxi_defconfig            | 1          

  Details:  https://kernelci.org/test/job/stable-rc/branch/queue%2F5.10/kernel/v5.10.165-76-gffe5f229ddc9/plan/baseline/

  Test:     baseline
  Tree:     stable-rc
  Branch:   queue/5.10
  Describe: v5.10.165-76-gffe5f229ddc9
  URL:      https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
  SHA:      ffe5f229ddc988790f4f8b847eba1b06d98b87bb 


Test Regressions
---------------- 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
am57xx-beagle-x15            | arm   | lab-linaro-lkft | gcc-10   | multi_v7_defconfig         | 1          

  Details:     https://kernelci.org/test/plan/id/63d14ee8a3d87b7390915ef9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14ee8a3d87b7390915efa
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
am57xx-beagle-x15            | arm   | lab-linaro-lkft | gcc-10   | omap2plus_defconfig        | 1          

  Details:     https://kernelci.org/test/plan/id/63d14e813ec5396c7e915ee2

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: omap2plus_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/omap2plus_defconfig/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/omap2plus_defconfig/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14e813ec5396c7e915ee3
        failing since 1 day (last pass: v5.10.162-950-g0ce90a11c205, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
at91sam9g20ek                | arm   | lab-broonie     | gcc-10   | at91_dt_defconfig          | 1          

  Details:     https://kernelci.org/test/plan/id/63d14c21da8c97e26f915ec6

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: at91_dt_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/at91_dt_defconfig/gcc-10/lab-broonie/baseline-at91sam9g20ek.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/at91_dt_defconfig/gcc-10/lab-broonie/baseline-at91sam9g20ek.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14c21da8c97e26f915ec7
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
at91sam9g20ek                | arm   | lab-broonie     | gcc-10   | multi_v5_defconfig         | 1          

  Details:     https://kernelci.org/test/plan/id/63d14bbda25151d11a915f3b

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v5_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v5_defconfig/gcc-10/lab-broonie/baseline-at91sam9g20ek.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v5_defconfig/gcc-10/lab-broonie/baseline-at91sam9g20ek.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14bbda25151d11a915f3c
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
bcm2711-rpi-4-b              | arm64 | lab-linaro-lkft | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d15209dca203972f915ef2

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-linaro-lkft/baseline-bcm2711-rpi-4-b.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-linaro-lkft/baseline-bcm2711-rpi-4-b.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d15209dca203972f915ef3
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
beagle-xm                    | arm   | lab-baylibre    | gcc-10   | multi_v7_defconfig         | 1          

  Details:     https://kernelci.org/test/plan/id/63d150d1175812eacc915ebd

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-beagle-xm.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-beagle-xm.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d150d1175812eacc915ebe
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
beagle-xm                    | arm   | lab-baylibre    | gcc-10   | omap2plus_defconfig        | 1          

  Details:     https://kernelci.org/test/plan/id/63d14f6676a24f4246915ebd

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: omap2plus_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/omap2plus_defconfig/gcc-10/lab-baylibre/baseline-beagle-xm.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/omap2plus_defconfig/gcc-10/lab-baylibre/baseline-beagle-xm.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14f6676a24f4246915ebe
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
beaglebone-black             | arm   | lab-broonie     | gcc-10   | multi_v7_defconfig         | 1          

  Details:     https://kernelci.org/test/plan/id/63d14ef13f11547704915ed2

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-broonie/baseline-beaglebone-black.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-broonie/baseline-beaglebone-black.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14ef13f11547704915ed3
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
beaglebone-black             | arm   | lab-broonie     | gcc-10   | omap2plus_defconfig        | 1          

  Details:     https://kernelci.org/test/plan/id/63d14e89c25a5bacad915eba

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: omap2plus_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/omap2plus_defconfig/gcc-10/lab-broonie/baseline-beaglebone-black.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/omap2plus_defconfig/gcc-10/lab-broonie/baseline-beaglebone-black.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14e89c25a5bacad915ebb
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
cubietruck                   | arm   | lab-baylibre    | gcc-10   | multi_v7_defconfig         | 3          

  Details:     https://kernelci.org/test/plan/id/63d1500ef86123ff5e915edc

  Results:     3 PASS, 3 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-cubietruck.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-cubietruck.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/63d1500ef86123ff5e915ee1
        new failure (last pass: v5.10.162-1027-gef210438485a)

    2023-01-25T15:50:44.443679  kern  :emerg : 3f40: c200a418 c1803d00 00<8>[   38.996433] <LAVA_SIGNAL_ENDRUN 0_dmesg 3210755_1.5.2.4.1>
    2023-01-25T15:50:44.449328  000088 c20e2000 c200a400 c03625bc ffffe000 c1a1e446
    2023-01-25T15:50:44.454740  kern  :emerg : 3f60: c20a1000 c2096300 c2096180 00000000 c20e2000 c0362558 c20a1000 c20d1e9c
    2023-01-25T15:50:44.465815  kern  :emerg : 3f80: c2096324 c036802c 00000001 c2096180 c0367ee0 00000000 00000000 00000000
    2023-01-25T15:50:44.471280  kern  :emerg : 3fa0: 00000000 00000000 00000000 c03001a8 00000000 00000000 00000000 00000000
    2023-01-25T15:50:44.482314  kern  :emerg : 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
    2023-01-25T15:50:44.487989  kern  :emerg : 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
    2023-01-25T15:50:44.499037  kern  :emerg : [<c0862610>] (devm_clk_release) from [<c0a35db0>] (devres_release+0x24/0x4c)
    2023-01-25T15:50:44.504719  kern  :emerg : [<c0a35db0>] (devres_release) from [<c0862840>] (devm_clk_put+0x1c/0x40)
    2023-01-25T15:50:44.515685  kern  :emerg : [<c0862840>] (devm_clk_put) from [<bf3400b0>] (bcm_get_resources+0x174/0x21c [hci_uart]) 
    ... (36 line(s) more)  

  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/63d1500ef86123ff5e915ee3
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        73 lines

    2023-01-25T15:50:44.024092  kern  :alert : 8<--- cut here ---
    2023-01-25T15:50:44.029428  kern  :alert : Unable to handle kernel paging request at virtual address c2611800
    2023-01-25T15:50:44.029965  kern  :alert : pgd = (ptrval)
    2023-01-25T15:50:44.036025  kern  :alert : [c2611800] *pgd=4261141e(bad)
    2023-01-25T15:50:44.055890  kern  :emerg : Internal error: Oops: 8000000d [#1] SMP ARM
    2023-01-25T15:50:44.060982  kern  :emerg : Process udevd (pid: 106, stack limit = 0x(ptrval))
    2023-01-25T15:50:44.066457  kern  :emerg : Stack: (0xc3e75da8 to 0xc3e76000)
    2023-01-25T15:50:44.077619  kern  :emerg : 5da0:                   c3983800 c0a35490 00000000 c394c300 c3983800 23fc13a0
    2023-01-25T15:50:44.083197  kern  :emerg : 5dc0: c21ac410 c21ac410 fffffdfb 00000000 c1a78968 c1a78960 bf063024 0000000e
    2023-01-25T15:50:44.094111  kern  :emerg : 5de0: 00020000 c0a31454 00000000 c21ac410 bf063024 c21ac454 bf063024 c39837b8 
    ... (44 line(s) more)  

  * baseline.dmesg.alert: https://kernelci.org/test/case/id/63d1500ef86123ff5e915ee4
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        8 lines

    2023-01-25T15:50:43.996582  kern  :alert : 8<--- cut here ---
    2023-01-25T15:50:44.001813  kern  :alert : Unable to handle kernel paging request at virtual address c21ac410
    2023-01-25T15:50:44.007336  kern  :alert : pgd = (ptrval)
    2023-01-25T15:50:44.018156  kern  :alert : [c21ac410] *pgd=4201141e(bad<8>[   38.569121] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=8>
    2023-01-25T15:50:44.018761  )   
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
cubietruck                   | arm   | lab-baylibre    | gcc-10   | sunxi_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/63d14d87ff748aea81915ebf

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-cubietruck.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-cubietruck.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14d87ff748aea81915ec0
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
da850-lcdk                   | arm   | lab-baylibre    | gcc-10   | davinci_all_defconfig      | 1          

  Details:     https://kernelci.org/test/plan/id/63d14b1cf85a0dbfff915ed8

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: davinci_all_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/davinci_all_defconfig/gcc-10/lab-baylibre/baseline-da850-lcdk.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/davinci_all_defconfig/gcc-10/lab-baylibre/baseline-da850-lcdk.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14b1cf85a0dbfff915ed9
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
da850-lcdk                   | arm   | lab-baylibre    | gcc-10   | multi_v5_defconfig         | 1          

  Details:     https://kernelci.org/test/plan/id/63d14bb9a25151d11a915f37

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v5_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v5_defconfig/gcc-10/lab-baylibre/baseline-da850-lcdk.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v5_defconfig/gcc-10/lab-baylibre/baseline-da850-lcdk.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14bb9a25151d11a915f38
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
dove-cubox                   | arm   | lab-pengutronix | gcc-10   | mvebu_v7_defconfig         | 1          

  Details:     https://kernelci.org/test/plan/id/63d14c1059b5f7476c915ec4

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: mvebu_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/mvebu_v7_defconfig/gcc-10/lab-pengutronix/baseline-dove-cubox.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/mvebu_v7_defconfig/gcc-10/lab-pengutronix/baseline-dove-cubox.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14c1059b5f7476c915ec5
        failing since 0 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
fsl-ls1088a-rdb              | arm64 | lab-nxp         | gcc-10   | defconfig                  | 2          

  Details:     https://kernelci.org/test/plan/id/63d15189fcff300575915edd

  Results:     3 PASS, 3 FAIL, 1 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-nxp/baseline-fsl-ls1088a-rdb.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-nxp/baseline-fsl-ls1088a-rdb.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.dmesg.alert: https://kernelci.org/test/case/id/63d15189fcff300575915ee5
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        11 lines

    2023-01-25T15:57:43.738595  kern  :alert : Unable to handle kernel NULL poin[   14.090716] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=11>   

  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/63d15189fcff300575915ee6
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        2 lines

    2023-01-25T15:57:43.742207  ter dereference at virtual address 0000000000000000
    2023-01-25T15:57:43.742348  kern  :alert : Mem abort info:
    2023-01-25T15:57:43.747910  kern  :alert :   ESR = 0x96000006
    2023-01-25T15:57:43.758793  kern  :a[   14.110836] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=emerg RESULT=fail UNITS=lines MEASUREMENT=2>   
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
imx6dl-riotboard             | arm   | lab-pengutronix | gcc-10   | multi_v7_defconfig         | 2          

  Details:     https://kernelci.org/test/plan/id/63d14f587e89071ec9915edb

  Results:     4 PASS, 2 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-pengutronix/baseline-imx6dl-riotboard.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-pengutronix/baseline-imx6dl-riotboard.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/63d14f587e89071ec9915ee2
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        41 lines

    2023-01-25T15:48:16.420617  kern  :emerg : Internal error: Oops: 8000000d [#1] SMP ARM
    2023-01-25T15:48:16.438548  kern  :emerg : Process udevd (pid: 103, stack limit = 0x(ptrval))[    9.640101] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=emerg RESULT=fail UNITS=lines MEASUREMENT=41>
    2023-01-25T15:48:16.438751     

  * baseline.dmesg.alert: https://kernelci.org/test/case/id/63d14f587e89071ec9915ee3
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        4 lines

    2023-01-25T15:48:16.378694  kern  :alert : 8<--- cut here ---
    2023-01-25T15:48:16.387421  kern  :alert : Unable to handle kernel paging request at virtual address c2291c20
    2023-01-25T15:48:16.395004  kern  :a[    9.597794] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=4>
    2023-01-25T15:48:16.395176  lert : pgd = (ptrval)
    2023-01-25T15:48:16.395297  kern  :alert : [c2291c20] *pgd=1220041e(bad)   
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
imx6qp-wandboard-revd1       | arm   | lab-pengutronix | gcc-10   | imx_v6_v7_defconfig        | 1          

  Details:     https://kernelci.org/test/plan/id/63d14fa815e4f93b2d915ecd

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: imx_v6_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/imx_v6_v7_defconfig/gcc-10/lab-pengutronix/baseline-imx6qp-wandboard-revd1.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/imx_v6_v7_defconfig/gcc-10/lab-pengutronix/baseline-imx6qp-wandboard-revd1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14fa815e4f93b2d915ece
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
imx6qp-wandboard-revd1       | arm   | lab-pengutronix | gcc-10   | multi_v7_defconfig         | 1          

  Details:     https://kernelci.org/test/plan/id/63d150fc5d5ed0de13915ee2

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-pengutronix/baseline-imx6qp-wandboard-revd1.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-pengutronix/baseline-imx6qp-wandboard-revd1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d150fc5d5ed0de13915ee3
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
imx7d-sdb                    | arm   | lab-nxp         | gcc-10   | imx_v6_v7_defconfig        | 1          

  Details:     https://kernelci.org/test/plan/id/63d14ed213a221ffdf915ec7

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: imx_v6_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/imx_v6_v7_defconfig/gcc-10/lab-nxp/baseline-imx7d-sdb.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/imx_v6_v7_defconfig/gcc-10/lab-nxp/baseline-imx7d-sdb.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14ed213a221ffdf915ec8
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
imx7d-sdb                    | arm   | lab-nxp         | gcc-10   | multi_v7_defconfig         | 1          

  Details:     https://kernelci.org/test/plan/id/63d14f3bb884395893915ec8

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-nxp/baseline-imx7d-sdb.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-nxp/baseline-imx7d-sdb.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14f3bb884395893915ec9
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
imx8mn-ddr4-evk              | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d15455e29ac126ea915f20

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-imx8mn-ddr4-evk.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-imx8mn-ddr4-evk.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d15455e29ac126ea915f21
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
imx8mn-ddr4-evk              | arm64 | lab-nxp         | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d152a36433943eca915eda

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-nxp/baseline-imx8mn-ddr4-evk.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-nxp/baseline-imx8mn-ddr4-evk.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d152a36433943eca915edb
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
imx8mp-evk                   | arm64 | lab-nxp         | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d151854de7e641b7915ef2

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-nxp/baseline-imx8mp-evk.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-nxp/baseline-imx8mp-evk.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d151854de7e641b7915ef3
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
jetson-tk1                   | arm   | lab-baylibre    | gcc-10   | multi_v7_defconfig         | 1          

  Details:     https://kernelci.org/test/plan/id/63d14ed913a221ffdf915ed8

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-jetson-tk1.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-jetson-tk1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14ed913a221ffdf915ed9
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
jetson-tk1                   | arm   | lab-baylibre    | gcc-10   | tegra_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/63d14d4dba48f5af46915ec1

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: tegra_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/tegra_defconfig/gcc-10/lab-baylibre/baseline-jetson-tk1.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/tegra_defconfig/gcc-10/lab-baylibre/baseline-jetson-tk1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14d4dba48f5af46915ec2
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
meson-g12a-sei510            | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d150ea29e25cb832915ed7

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-g12a-sei510.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-g12a-sei510.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d150ea29e25cb832915ed8
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
meson-g12a-u200              | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d1511ab8fdff31aa915ec8

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-g12a-u200.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-g12a-u200.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d1511ab8fdff31aa915ec9
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
meson-g12a-x96-max           | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d1511db8fdff31aa915ecd

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-g12a-x96-max.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-g12a-x96-max.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d1511db8fdff31aa915ece
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
meson-g12b-a311d-khadas-vim3 | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d15127a1684cd16e915ee1

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-g12b-a311d-khadas-vim3.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-g12b-a311d-khadas-vim3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d15127a1684cd16e915ee2
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
meson-g12b-a311d-khadas-vim3 | arm64 | lab-collabora   | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d1520938b9a9db58915eed

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-collabora/baseline-meson-g12b-a311d-khadas-vim3.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-collabora/baseline-meson-g12b-a311d-khadas-vim3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d1520938b9a9db58915eee
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
meson-g12b-odroid-n2         | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d150e947252fbbf8915eb9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-g12b-odroid-n2.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-g12b-odroid-n2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d150e947252fbbf8915eba
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
meson-gxbb-p200              | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d1523ab815b58a06915f09

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-gxbb-p200.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-gxbb-p200.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d1523ab815b58a06915f0a
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
meson-gxl-s905d-p230         | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d15244b815b58a06915f25

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-gxl-s905d-p230.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-gxl-s905d-p230.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d15244b815b58a06915f26
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
meson-gxm-khadas-vim2        | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d1521515d9b44372915edd

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-gxm-khadas-vim2.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-gxm-khadas-vim2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d1521515d9b44372915ede
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
meson-gxm-q200               | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d155462e3b0eae0c915ee3

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-gxm-q200.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-gxm-q200.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d155462e3b0eae0c915ee4
        failing since 0 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
meson-sm1-khadas-vim3l       | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d15156f36656dca3915ed6

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-sm1-khadas-vim3l.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-sm1-khadas-vim3l.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d15156f36656dca3915ed7
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
meson-sm1-sei610             | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d150e829e25cb832915ed0

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-sm1-sei610.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-sm1-sei610.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d150e829e25cb832915ed1
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
mt8173-elm-hana              | arm64 | lab-collabora   | gcc-10   | defconfig+arm64-chromebook | 1          

  Details:     https://kernelci.org/test/plan/id/63d14df62aa8c96975915ec9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8173-elm-hana.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8173-elm-hana.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14df62aa8c96975915eca
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
odroid-xu3                   | arm   | lab-collabora   | gcc-10   | exynos_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/63d14bc9cee3c79fa7915ee1

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: exynos_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/exynos_defconfig/gcc-10/lab-collabora/baseline-odroid-xu3.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/exynos_defconfig/gcc-10/lab-collabora/baseline-odroid-xu3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14bc9cee3c79fa7915ee2
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
odroid-xu3                   | arm   | lab-collabora   | gcc-10   | multi_v7_defconfig         | 1          

  Details:     https://kernelci.org/test/plan/id/63d14f1460ca315058915ec9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-collabora/baseline-odroid-xu3.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-collabora/baseline-odroid-xu3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14f1460ca315058915eca
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
rk3399-gru-kevin             | arm64 | lab-collabora   | gcc-10   | defconfig+arm64-chromebook | 1          

  Details:     https://kernelci.org/test/plan/id/63d14e069d52db547e915eb9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-rk3399-gru-kevin.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-rk3399-gru-kevin.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14e069d52db547e915eba
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
rk3399-roc-pc                | arm64 | lab-clabbe      | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d1521a7e310aa311915ec5

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-clabbe/baseline-rk3399-roc-pc.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-clabbe/baseline-rk3399-roc-pc.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d1521a7e310aa311915ec6
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
rk3399-rock-pi-4b            | arm64 | lab-collabora   | gcc-10   | defconfig                  | 2          

  Details:     https://kernelci.org/test/plan/id/63d15126a1684cd16e915ecb

  Results:     4 PASS, 2 FAIL, 1 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-collabora/baseline-rk3399-rock-pi-4b.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-collabora/baseline-rk3399-rock-pi-4b.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/63d15126a1684cd16e915ed2
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        2 lines 

  * baseline.dmesg.alert: https://kernelci.org/test/case/id/63d15126a1684cd16e915ed3
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        8 lines

    2023-01-25T15:56:01.796413  kern  :alert : [ffff0000007ed410] pgd=000000007fff8003, p4d=000000007fff8003, pud=000000007fff7003, pmd=000000007fff4003, pte=00680000007ed707
    2023-01-25T15:56:01.796979  <8>[   15.739699] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=8>
    2023-01-25T15:56:01.797452  kern  :emerg : Internal error: Oops: 8600000f [#1] PREEMPT SMP
    2023-01-25T15:56:01.797886  kern  :emerg : Code: 00739e80 ffff0000 ffffffff 00000000 (00739e80) 
    2023-01-25T15:56:01.798303  <8>[   15.760950] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=emerg RESULT=fail UNITS=lines MEASUREMENT=2>
    2023-01-25T15:56:01.798723  + set +x   
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun50i-a64-bananapi-m64      | arm64 | lab-clabbe      | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d15125a1684cd16e915ec8

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-clabbe/baseline-sun50i-a64-bananapi-m64.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-clabbe/baseline-sun50i-a64-bananapi-m64.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d15125a1684cd16e915ec9
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun50i-a64-pine64-plus       | arm64 | lab-broonie     | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d15146f36656dca3915ec9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-broonie/baseline-sun50i-a64-pine64-plus.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-broonie/baseline-sun50i-a64-pine64-plus.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d15146f36656dca3915eca
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun50i-h5-lib...ch-all-h3-cc | arm64 | lab-baylibre    | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d151d97775f38360915eb9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-sun50i-h5-libretech-all-h3-cc.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-baylibre/baseline-sun50i-h5-libretech-all-h3-cc.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d151d97775f38360915eba
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun50i-h5-lib...ch-all-h3-cc | arm64 | lab-broonie     | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d1515bf36656dca3915edb

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-broonie/baseline-sun50i-h5-libretech-all-h3-cc.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-broonie/baseline-sun50i-h5-libretech-all-h3-cc.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d1515bf36656dca3915edc
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun50i-h6-orangepi-3         | arm64 | lab-clabbe      | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d1521b7d820be1fc915ec7

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-clabbe/baseline-sun50i-h6-orangepi-3.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-clabbe/baseline-sun50i-h6-orangepi-3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d1521b7d820be1fc915ec8
        failing since 0 day (last pass: v5.10.162-851-g33a0798ae8e3, first fail: v5.10.162-1026-g401c1c1d3bf5) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun50i-h6-orangepi-one-plus  | arm64 | lab-clabbe      | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d15189b160d4495b915eea

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-clabbe/baseline-sun50i-h6-orangepi-one-plus.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-clabbe/baseline-sun50i-h6-orangepi-one-plus.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d15189b160d4495b915eeb
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun50i-h6-pine-h64           | arm64 | lab-clabbe      | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d1510247252fbbf8915ed3

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-clabbe/baseline-sun50i-h6-pine-h64.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-clabbe/baseline-sun50i-h6-pine-h64.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d1510247252fbbf8915ed4
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun50i-h6-pine-h64           | arm64 | lab-collabora   | gcc-10   | defconfig                  | 1          

  Details:     https://kernelci.org/test/plan/id/63d15104712cd94da0915eb9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-collabora/baseline-sun50i-h6-pine-h64.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm64/defconfig/gcc-10/lab-collabora/baseline-sun50i-h6-pine-h64.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/arm64/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d15104712cd94da0915eba
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun5i-a13-olinuxino-micro    | arm   | lab-baylibre    | gcc-10   | sunxi_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/63d14d4d2bab53d229915ef9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun5i-a13-olinuxino-micro.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun5i-a13-olinuxino-micro.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14d4d2bab53d229915efa
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun7i-a20-cubieboard2        | arm   | lab-baylibre    | gcc-10   | sunxi_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/63d14d26e3bb8e7e6f915ee9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun7i-a20-cubieboard2.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun7i-a20-cubieboard2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14d26e3bb8e7e6f915eea
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun7i-a20-cubieboard2        | arm   | lab-clabbe      | gcc-10   | sunxi_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/63d14d15ed6c2d220e915ece

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-clabbe/baseline-sun7i-a20-cubieboard2.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-clabbe/baseline-sun7i-a20-cubieboard2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14d15ed6c2d220e915ecf
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun7i-a20-olinuxino-lime2    | arm   | lab-baylibre    | gcc-10   | sunxi_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/63d14e8db2e0e08554915ef1

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun7i-a20-olinuxino-lime2.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun7i-a20-olinuxino-lime2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14e8db2e0e08554915ef2
        failing since 0 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun8i-a33-olinuxino          | arm   | lab-clabbe      | gcc-10   | multi_v7_defconfig         | 2          

  Details:     https://kernelci.org/test/plan/id/63d14f099604dbab64915ee7

  Results:     3 PASS, 3 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-clabbe/baseline-sun8i-a33-olinuxino.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-clabbe/baseline-sun8i-a33-olinuxino.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.dmesg.alert: https://kernelci.org/test/case/id/63d14f099604dbab64915eef
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        4 lines

    2023-01-25T15:46:58.066813  kern  :alert : 8<--- cut here ---
    2023-01-25T15:46:58.067256  kern  :alert : Unable to handle kernel paging request at virtual address c21d6c10
    2023-01-25T15:46:58.067850  kern  :alert : pgd = (ptrval)
    2023-01-25T15:46:58.068201  kern  :alert : [c21d6c10] *pgd=4201141e(bad)
    2023-01-25T15:46:58.068551  <8>[   18.696955] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=4>   

  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/63d14f099604dbab64915ef0
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        39 lines

    2023-01-25T15:46:58.142057  kern  :emerg : Internal error: Oops: 8000000d [#1] SMP ARM
    2023-01-25T15:46:58.142507  kern  :emerg : Process udevd (pid: 113, stack limit = 0x(ptrval))
    2023-01-25T15:46:58.142842  kern  :emerg : Stack: (0xc3d11da8 to 0xc3d12000)
    2023-01-25T15:46:58.143418  kern  :emerg : 1da0:                   c2320900 c0a35490 00000000 c231f3c0 c232<8>[   18.765923] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=emerg RESULT=fail UNITS=lines MEASUREMENT=39>
    2023-01-25T15:46:58.143750  0980 55e1eeef
    2023-01-25T15:46:58.144062  kern  :emerg : 1dc0: c21d6c10 c21d6c10 fffffdfb 0<8>[   18.779689] <LAVA_SIGNAL_ENDRUN 0_dmesg 385938_1.5.2.4.1>
    2023-01-25T15:46:58.144406  0000000 c1a78968 c1a78960 bf0ff024 0000000a   
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun8i-a33-olinuxino          | arm   | lab-clabbe      | gcc-10   | sunxi_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/63d14d291eb876a2fd915eb9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-clabbe/baseline-sun8i-a33-olinuxino.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-clabbe/baseline-sun8i-a33-olinuxino.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14d291eb876a2fd915eba
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun8i-a83t-bananapi-m3       | arm   | lab-clabbe      | gcc-10   | multi_v7_defconfig         | 2          

  Details:     https://kernelci.org/test/plan/id/63d14f169adcc4fa81915eeb

  Results:     3 PASS, 3 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-clabbe/baseline-sun8i-a83t-bananapi-m3.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-clabbe/baseline-sun8i-a83t-bananapi-m3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.dmesg.alert: https://kernelci.org/test/case/id/63d14f169adcc4fa81915ef3
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        8 lines

    2023-01-25T15:47:09.761149  kern  :alert : 8<--- cut here ---
    2023-01-25T15:47:09.761679  kern  :alert : Unable to handle kernel paging request at virtual address c22d2810
    2023-01-25T15:47:09.762269  kern  :alert : pgd = (ptrval)
    2023-01-25T15:47:09.762593  kern  :alert : [c22d2810] *pgd=4221141e(bad[   44.694790] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=8>
    2023-01-25T15:47:09.762911  )
    2023-01-25T15:47:09.763284  kern  :alert : 8<--- cut here ---
    2023-01-25T15:47:09.763578  kern  :alert : Unable to handle kernel paging request at virtual address c3c0d400
    2023-01-25T15:47:09.763869  kern  :alert : pgd = (ptrval)
    2023-01-25T15:47:09.764263  kern  :alert : [c3c0d400] *pgd=43c1141e(bad)   

  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/63d14f169adcc4fa81915ef4
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        73 lines

    2023-01-25T15:47:09.809844  kern  :emerg : Internal error: Oops: 8000000d [#1] SMP ARM
    2023-01-25T15:47:09.810540  kern  :emerg : Process udevd (pid: 173, stack limit = 0x(ptrval))
    2023-01-25T15:47:09.810879  kern  :emerg : Stack: (0xc5989da8 to 0xc598a000)
    2023-01-25T15:47:09.811198  kern  :emerg : 9da0:                   c2330400 c0a35490 00000000 c2336000 c2330480 78a60fc2
    2023-01-25T15:47:09.811533  kern  :emerg : 9dc0: c22d2810 c22d2810 fffffdfb 00000000 c1a78968 c1a78960 bf075024 0000001b
    2023-01-25T15:47:09.811900  kern  :emerg : 9de0: 00020000 c0a31454 00000000 c22d2810 bf075024 c22d2854 bf075024 c23303b8
    2023-01-25T15:47:09.852842  kern  :emerg : 9e00: c5988000 0000017b 00020000 c0a31a30 c22d2810 00000000 c22d2854 c0a31d20
    2023-01-25T15:47:09.853552  kern  :emerg : 9e20: bf075024 c22d2810 c0a31d28 c5988000 c23303b8 c0a31d88 00000000 bf075024
    2023-01-25T15:47:09.853892  kern  :emerg : 9e40: c0a31d28 c0a2f3a8 c5988000 c2083858 c223d5b4 78a60fc2 bf075024 bf075024
    2023-01-25T15:47:09.854214  kern  :emerg : 9e60: c2330380 00000000 c19c7968 c0a306e0 bf074494 c1008dd8 bf075024 00000000 
    ... (40 line(s) more)  
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun8i-a83t-bananapi-m3       | arm   | lab-clabbe      | gcc-10   | sunxi_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/63d14cfaecc5eb8d87915ec7

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-clabbe/baseline-sun8i-a83t-bananapi-m3.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-clabbe/baseline-sun8i-a83t-bananapi-m3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14cfaecc5eb8d87915ec8
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun8i-h2-plus...ch-all-h3-cc | arm   | lab-baylibre    | gcc-10   | multi_v7_defconfig         | 2          

  Details:     https://kernelci.org/test/plan/id/63d14efc9adcc4fa81915ebf

  Results:     4 PASS, 2 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h2-plus-libretech-all-h3-cc.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h2-plus-libretech-all-h3-cc.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/63d14efc9adcc4fa81915ec6
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        39 lines

    2023-01-25T15:46:36.050750  kern  :emerg : Internal error: Oops: 8000000d [#1] SMP ARM
    2023-01-25T15:46:36.051221  kern  :emerg : Process udevd (pid: 126, stack limit = 0x(ptrval))
    2023-01-25T15:46:36.051455  kern  :emerg : Stack: (0xc38cbda8 to 0xc38cc000)
    2023-01-25T15:46:36.051670  kern  :emerg : <8>[    8.620580] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=emerg RESULT=fail UNITS=lines MEASUREMENT=39>
    2023-01-25T15:46:36.051916  bda0:                   c3926b00 c0a35490 00000000 c40fa180 c73c<8>[    8.635128] <LAVA_SIGNAL_ENDRUN 0_dmesg 3210762_1.5.2.4.1>
    2023-01-25T15:46:36.052133  2f00 f88f1e1a   

  * baseline.dmesg.alert: https://kernelci.org/test/case/id/63d14efc9adcc4fa81915ec7
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        4 lines

    2023-01-25T15:46:35.982219  kern  :alert : 8<--- cut here ---
    2023-01-25T15:46:35.982692  kern  :alert : Unable to handle kernel paging request at virtual address c213bc10
    2023-01-25T15:46:35.982925  kern  :alert : pgd = (ptrval)
    2023-01-25T15:46:35.985777  kern  :alert : [c213bc10] *pgd=4201141e(bad<8>[    8.557393] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=4>
    2023-01-25T15:46:35.986034  )   
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun8i-h2-plus...ch-all-h3-cc | arm   | lab-baylibre    | gcc-10   | sunxi_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/63d14d17ed6c2d220e915ed5

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h2-plus-libretech-all-h3-cc.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h2-plus-libretech-all-h3-cc.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14d17ed6c2d220e915ed6
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun8i-h3-libretech-all-h3-cc | arm   | lab-baylibre    | gcc-10   | multi_v7_defconfig         | 2          

  Details:     https://kernelci.org/test/plan/id/63d14efe9604dbab64915ec3

  Results:     4 PASS, 2 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h3-libretech-all-h3-cc.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h3-libretech-all-h3-cc.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/63d14efe9604dbab64915eca
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        39 lines

    2023-01-25T15:46:42.521026  kern  :emerg : Internal error: Oops: 8000000d [#1] SMP ARM
    2023-01-25T15:46:42.521727  kern  :emerg : Process udevd (pid: 120, stack limit = 0x(ptrval))
    2023-01-25T15:46:42.522087  kern  :emerg : Stack: (0xc3f7dda8 to 0xc3f7e000)
    2023-01-25T15:46:42.522427  kern  :emerg : <8>[    8.659447] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=emerg RESULT=fail UNITS=lines MEASUREMENT=39>
    2023-01-25T15:46:42.522767  dda0:                   c22c9500 c0a35490 00000000 c27e63c0 c27e<8>[    8.674253] <LAVA_SIGNAL_ENDRUN 0_dmesg 3210766_1.5.2.4.1>
    2023-01-25T15:46:42.523112  2600 57d47d51   

  * baseline.dmesg.alert: https://kernelci.org/test/case/id/63d14efe9604dbab64915ecb
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        4 lines

    2023-01-25T15:46:42.461855  kern  :alert : 8<--- cut here ---
    2023-01-25T15:46:42.462520  kern  :alert : Unable to handle kernel paging request at virtual address c227bc10
    2023-01-25T15:46:42.462878  kern  :alert : pgd = (ptrval)
    2023-01-25T15:46:42.465384  kern  :alert : [c227bc10] *pgd=4221141e(bad<8>[    8.610246] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=4>
    2023-01-25T15:46:42.465749  )   
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun8i-h3-libretech-all-h3-cc | arm   | lab-baylibre    | gcc-10   | sunxi_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/63d14d15163aa57f0a915eb9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h3-libretech-all-h3-cc.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h3-libretech-all-h3-cc.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14d15163aa57f0a915eba
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun8i-h3-orangepi-pc         | arm   | lab-clabbe      | gcc-10   | multi_v7_defconfig         | 2          

  Details:     https://kernelci.org/test/plan/id/63d14faa30263cbeed915eba

  Results:     3 PASS, 3 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-clabbe/baseline-sun8i-h3-orangepi-pc.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-clabbe/baseline-sun8i-h3-orangepi-pc.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.dmesg.alert: https://kernelci.org/test/case/id/63d14faa30263cbeed915ec2
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        4 lines

    2023-01-25T15:49:34.601435  kern  :alert : 8<--- cut here ---
    2023-01-25T15:49:34.606767  kern  :alert : Unable to handle kernel paging request at virtual address c227b810
    2023-01-25T15:49:34.615930  kern  :ale<8>[   15.412867] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=4>   

  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/63d14faa30263cbeed915ec3
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a)
        39 lines

    2023-01-25T15:49:34.622274  rt : pgd = (ptrval)
    2023-01-25T15:49:34.622521  kern  :alert : [c227b810] *pgd=4221141e(bad)
    2023-01-25T15:49:34.641295  kern  :emerg : Internal error: Oops: 8000000d [#1] SMP ARM
    2023-01-25T15:49:34.652124  kern  :emerg : Process udevd (pid: 124, stack limit = 0x(ptrval))
    2023-01-25T15:49:34.655810  k<8>[   15.453006] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=emerg RESULT=fail UNITS=lines MEASUREMENT=39>   
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun8i-h3-orangepi-pc         | arm   | lab-clabbe      | gcc-10   | sunxi_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/63d14d65be9d87edca915ee8

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-clabbe/baseline-sun8i-h3-orangepi-pc.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-clabbe/baseline-sun8i-h3-orangepi-pc.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14d65be9d87edca915ee9
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun8i-r40-bananapi-m2-ultra  | arm   | lab-clabbe      | gcc-10   | multi_v7_defconfig         | 1          

  Details:     https://kernelci.org/test/plan/id/63d14fd1248d2903b6915eb9

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-clabbe/baseline-sun8i-r40-bananapi-m2-ultra.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/multi_v7_defconfig/gcc-10/lab-clabbe/baseline-sun8i-r40-bananapi-m2-ultra.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14fd1248d2903b6915eba
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
 


platform                     | arch  | lab             | compiler | defconfig                  | regressions
-----------------------------+-------+-----------------+----------+----------------------------+------------
sun8i-r40-bananapi-m2-ultra  | arm   | lab-clabbe      | gcc-10   | sunxi_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/63d14d79ad403e6a50915ec3

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-clabbe/baseline-sun8i-r40-bananapi-m2-ultra.txt
  HTML log:    https://storage.kernelci.org//stable-rc/queue-5.10/v5.10.165-76-gffe5f229ddc9/arm/sunxi_defconfig/gcc-10/lab-clabbe/baseline-sun8i-r40-bananapi-m2-ultra.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230120.0/armel/rootfs.cpio.gz 


  * baseline.login: https://kernelci.org/test/case/id/63d14d79ad403e6a50915ec4
        failing since 1 day (last pass: v5.10.162-950-g7728aa131bf4, first fail: v5.10.162-1026-gd21c032a492a) 
  

                 reply	other threads:[~2023-01-25 19:24 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=63d181f3.620a0220.5a092.7971@mx.google.com \
    --to=bot@kernelci.org \
    --cc=kernel-build-reports@lists.linaro.org \
    --cc=kernelci-results@groups.io \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.