Re: chrome-platform/for-kernelci baseline: 98 runs, 5 regressions (v6.1-rc1-5-g27b86a65cd16)

[Brian Norris <briannorris@chromium.org>]

On Mon, Oct 31, 2022 at 09:37:28PM +0000, Mark Brown wrote:
> On Mon, Oct 31, 2022 at 10:40:25AM -0700, Brian Norris wrote:
> > On Sun, Oct 30, 2022 at 10:51 PM kernelci.org bot <bot@kernelci.org> wrote:
> > >
> > > chrome-platform/for-kernelci baseline: 98 runs, 5 regressions (v6.1-rc1-5-g27b86a65cd16)
> 
> > Does anybody look at these? It's a bit weird to see "5 regressions",
> > and then to look back on the last few weeks (probably months) and see
> > the same errors...
> 
> Nobody seems to care about a lot of the Chromebook stuff AFAICT.

:(

Well, if I/we are ever going to change that, it'd be nice to get a
little further on the below questions still:

> > >   Details:  https://kernelci.org/test/job/chrome-platform/branch/for-kernelci/kernel/v6.1-rc1-5-g27b86a65cd16/plan/baseline/
> 
> > Where can I find the test cases? (i.e., what's determining a "failure"
> > for one of these?)
> 
> The first place to go is the details link above, though it's a bit less
> clear for the non-baseline tests you should at least be able to see
> what's going on on the console which will often help with finding the
> testsuite, or the error messages printed are descriptive as to what
> they were looking for specifically.  The kernelci-core repo contains all
> the stuff to map from a test name inot running something (often with
> external assistance) but there's lots of templating in the way.

Thanks. I already looked at most/all of those links. And I was first
going for the "baseline" tests, since those seem pretty basic, and if we
can't pass those, we're probably totally lost.

> > As a small guess, I see that we're missing some common configs, like:
> 
> > CONFIG_PHY_ROCKCHIP_DP
> 
> > This sometimes means that the display subsystem (and therefore audio
> > subsystem that relies on DP for one of its components) is not going to
> > fully set itself up properly. Usually things can still sort of work,
> > but I don't really know what the tests are looking for.
> 
> The baseline tests are just making sure that the system comes up to a
> shell.  The driver loaded tests are checking that particular devices
> have a driver bound to them.

For one specific example: I'm looking at rockchip-i2s1-probed, which
fails here:

https://storage.kernelci.org/chrome-platform/for-kernelci/v6.1-rc1-5-g27b86a65cd16/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-rk3399-gru-kevin.html

I can't find a single mention of "i2s1" or "probed" in the kernelci
repo, so I must be missing something. Is there some external config
file in another repo? Or else the test configs are autogenerating cases
on the fly based on parsing...the device tree?

Anyway, I don't know how or why that ever passed, because AFAICT, RK3399
Chromebooks should only have a single I2S block enabled, and they're
passing the 'rockchip-i2s0-probed' case. So it feels like I need to be
disabling some test case.

Somewhat similar story for cros-ec-sensors-accel{0,1}-probed, although I
believe the sensor driver is still working for me; I also see no
cros-ec-sensors errors in the KernelCI logs. So I wonder what exactly
the test is looking for (e.g., maybe the device name changed?).

Brian