Re: chrome-platform/for-kernelci baseline: 98 runs, 5 regressions (v6.1-rc1-5-g27b86a65cd16)

[Mark Brown <broonie@kernel.org>]

[-- Attachment #1: Type: text/plain, Size: 3152 bytes --]

On Mon, Oct 31, 2022 at 04:36:32PM -0700, Brian Norris wrote:
> On Mon, Oct 31, 2022 at 10:52:52PM +0000, Mark Brown wrote:
> > On Mon, Oct 31, 2022 at 03:21:55PM -0700, Brian Norris wrote:

> > The KernelCI repo just says what testsuites to invoke and how, it's not
> > got the actual testsuites.  Those X didn't probe failures come from
> > bootrr:

> >    https://github.com/andersson/bootrr

> > forked to:

> >    https://github.com/kernelci/bootrr

> > (which could use some upstreaming...) with the specific errors for

> Neither of those looks particularly active. If I patch stuff, is it
> better to send PRs to the 'andersson' one or the 'kernelci' one?

The kernelci one is the one that gets deployed in kernelci so probably
there in the short term - all the Chromebooks are there, I'm not sure
how many got upstreamed at all.

> > Yes, that was what I'd determined too - the reorganisation of the DT
> > looked legit, I can't remember what it was exactly.  I suspect it may
> > have boiled down to adding some missing default disables, or removing an
> > erroious enable for the board.

> Ah, based off your pointers, I see the test was looking for what used to
> be the i2s2 alias. But then I recall we stopped using that i2s instance:

> https://git.kernel.org/linus/b5fbaf7d779f5f02b7f75b080e7707222573be2a
> arm64: dts: rockchip: Switch RK3399-Gru DP to SPDIF output

> I forgot that folks did that downstream long ago but never bothered
> finishing upstreaming that until I got to it this year...
> ...but still, it's kinda sad that we've bothered to set up all this "CI"
> and then nobody paid any attention :( I only noticed because I recently
> subscribed to chrome-platform@lists.linux.dev.

Ah, that's the one.  I think I'd flagged it as the test looking wrong
but nobody picked it up.

> "assert_device_present cros-ec-sensors-accel0-probed cros-ec-sensors cros-ec-accel.11.*"

> ? Really, ".11"? That sounds like we're trying to test kernel
> implementation details, asynchronous probe race conditions, Makefile /
> linker ordering, and similar -- not anything that we actually expect to
> remain stable across kernel versions :(

> I'm not sure there's a great stable way to refer to such devices, so
> maybe it'd be better to write this as "count the number of devices"
> instead? Or I think this particular driver supports an "id" sysfs
> attribute, which refers to a stable underlying firmware ID. But that'd
> involve even more device-specific logic.

It looks to me like the intent of the test is to find the device with
the highest number and get a count that way but ICBW.

> I don't think I even care *why* the ID changed; that ID is far from a
> stable thing, if I'm reading it correctly. At least most of the others
> refer to hardware addresses, which are a little more reasonable to rely
> on (even if the device naming still isn't a stable guarantee).

Yeah.  My understanding is that the intent with bootrr is to be a smoke
test which flags up if drivers aren't getting instantiated, taking a
basic login test further forwards so it'll notice more devices.  Like
you say it's a bit of a fragile mechanism though.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]