Re: [meta-oe][PATCH v2 4/4] testimage: Moved write_image_test_data to testimage bbclass.

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Thu, 2018-08-30 at 19:49 +0200, Paulo Neves wrote:
> Previously the write_image_test_data was a rootfs post
> process command. This function ran only when the rootfs
> task was ran. Due to this if a variable was changed
> or added to the datastore that would not trigger the do_rootfs
> task, the variable would never be written into the json
> file. Consequently the do_testimage task was potentially
> unreproduceable and could fail if for some reason this
> variable was then used.
> 
> In this commit we move the recording of the datastore to the
> start of the do_testimage task. The do_testimage
> task then reads this freshly generated json and passes it into
> the normal test machinery. This approach allows for the test
> machinery to still be loosely coupled to bitbake, and thus
> still allows for the tests to be exported and used independently
> from bitbake. The caveat is, if there are datastore changes
> then the bitbake testimage task should be run before the tests
> can be ran independently again.

I'm not sure I agree with this. The test data really should be written
at either rootfs or image generation, not just before the tests are
run.

If you wanted to run the tests independently, after this change you
have no way to generate the data to do that without first running the
tests at least once. That doesn't make sense.

The correct thing to do here is to ensure the write_image_test_data()
function indicates which variables it depends upon and hence reruns
correctly.

Cheers,

Richard