[U-Boot-Users] [RFC PATCH] Add u-boot command regression tests.

Wolfgang Denk
2007-11-25T23:29:20Z

Hello,

in message <20071124010338.GA27628@cideas.com> you wrote:
>
> ...and now, for something completely different.
>
> The concept here is to use PyUnit testing in conjunction with python
> serial I/O (theoretically, ethernet-based command I/O could be handled
> too).
>
> This is a concept probably sufficiently implemented for the "help"
> command and a good start for the "fdt" command.
>
> What does The List think?
> * Useful to have regression tests for the u-boot commands?

It is not only useful, but IMHO absolutely necessary to do regular
regression testing on U-Boot.

> * Implementation-wise, am I heading in the right direction?

I wish you had asked that question before spending efforts on this.
The point is, that there is already a pretty extensive and extandable
regression test suite for U-Boot (and Linux, and some more). We just
failed to announce it loudly enough (which clearly shows that DENX is
not marketing driven - and funny enough, I'm not a bit ashamed about
that). However, I'm sad about the avoidable duplication of efforts,
so I have to say I'm sorry.

Please have a look at the DUTS, our regression test suite. See the
git repo at http://www.denx.de/cgi-bin/gitweb.cgi?p=duts.git;a=summary
and some documentation at http://www.denx.de/wiki/DUTS/DUTSDocs

I'd really appreciate if this was picked up by others as well. [It
took us a long and winded road to get there - trying some existing
solutions like dejagnu etc. and finally ending with a do-it-yourself
solution.]

Note: there is a non-abvious benefit from the DUTS: once you got it
running on a board, it will generate a set of log files which just
need to be uploaded to our web server to generate a new version of
the DULG for this board. The idea is that if you have a new software
version you just need to run the DUTS to get (1) a confirmation that
everything is working as expected fromt he regression test suite and
(2) an updated version of the documentation with eexamples showing
exacly this software version on exactly that hardware.

Best regards,

Wolfgang Denk

--
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Es gibt immer genug fuer die Beduerfnisse aller, aber niemals genug
fuer die Gier einzelner. -- Ghandi