Re: [RFC][PATCH] oeqa/runtime/cases: new image_upgrade test

[Richard Purdie <richard.purdie@linuxfoundation.org>]

Hi Michael,

At least at a first read and without running it, this does look like a
reasonable direction. I suspect that anyone else looking at this would
have a lot of questions about why we'd do it this way but given the
various constraints, it does make sense to me.

What we do need to think about is how someone else would reuse this as
currently it is very poky specific. Our aim is to make it easy for
others to use too. With that in mind:

* We probably want to "tag" this test with something so we can exclude
it from the normal oe-selftest runs on the autobuilder and allow it to
run on a per machine basis. There are other oe-selftests we already do
this with (like toolchain testing or machine specific environment file
tests).

* The configuration about what to test probably needs to come from the
distro (i.e. which DISTRO/MACHINE/image combinations).

* We probably need to parameterise it so that a list of images can be
tested rather than just a single one. I did wonder if we could have it
dynamically add tests for each image configured.

* We don't want to test on all MACHINE (e.g. qemumips and qemuppc are
not going to be included).

* We need to find a better way to share the code with autobuilder-
helper, I don't like duplicating code.

* The image url code is also highly poky specific. That probably needs
to come from the poky repository alongside the configuration.

Does that all make sense?

Cheers,

Richard