Re: Testing

[Stewart Smith <stewart@linux.vnet.ibm.com>]

Cyril Bur <cyrilbur@gmail.com> writes:
> I'd like to open some discussion about testing. I honestly haven't given that
> much thought to big picture testing but I can describe what I'm doing to
> btbridged to at least have something. I'm sure everyone has ideas as to which
> direction the testing should be going and what we should be doing and this is
> the conversation I would like to start.

There's a number of different levels of things too. I know Chris has
been fairly involved in testing the end user REST API with robotest and
the like, although what you're mostly talking about here is unit tests
and other such smaller box tests.

> At the moment it is almost impossible to test btbridged, short of actually
> booting a real openbmc and making the host to something and through observation
> try to see that all went well there really isn't a way of knowing if changes
> work.

funnily enough, we have that problem with skiboot too. Especially for
hardware that doesn't physically exist yet.

> Joel kindly added a .travis.yml (travis-ci.org can automate) file to the
> project to at least have compile checks, which is a great start. I'm currently
> expanding on it so include a 'fake' bt-host and a trivial ipmi client. I don't
> expect these tests to get too fancy but I do hope they'll be able to catch
> broken commits. Future direction might be to have everything done in qemu so
> that the 'fake' environment is closer to the real thing, doing so in qemu would
> also allow us to test the bt-host driver in linux.

Good idea.

btw, if you adopted autotools, you could get parallel test running for
free :)