[U-Boot-Users] [RFC PATCH] Add u-boot command regression tests.

[Wolfgang Denk <wd@denx.de>]

Hello,

in message <20071124010338.GA27628@cideas.com> you wrote:
> 
> ...and now, for something completely different.
> 
> The concept here is to use PyUnit testing in conjunction with python
> serial I/O (theoretically, ethernet-based command I/O could be handled
> too).
> 
> This is a concept probably sufficiently implemented for the "help"
> command and a good start for the "fdt" command.  
> 
> What does The List think?
> * Useful to have regression tests for the u-boot commands?

It is not only useful, but IMHO absolutely necessary to do regular
regression testing on U-Boot.

> * Implementation-wise, am I heading in the right direction?

I wish you had asked that question before spending efforts  on  this.
The point is, that there is already a pretty extensive and extandable
regression  test suite for U-Boot (and Linux, and some more). We just
failed to announce it loudly enough (which clearly shows that DENX is
not marketing driven - and funny enough, I'm not a bit ashamed  about
that).  However,  I'm sad about the avoidable duplication of efforts,
so I have to say I'm sorry.

Please have a look at the DUTS, our regression test suite. See the
git repo at http://www.denx.de/cgi-bin/gitweb.cgi?p=duts.git;a=summary
and some documentation at http://www.denx.de/wiki/DUTS/DUTSDocs

I'd really appreciate if this was picked up by others  as  well.  [It
took  us  a  long and winded road to get there - trying some existing
solutions like dejagnu etc. and finally ending with a  do-it-yourself
solution.]

Note: there is a non-abvious benefit from the DUTS: once you  got  it
running  on  a  board, it will generate a set of log files which just
need to be uploaded to our web server to generate a  new  version  of
the  DULG for this board. The idea is that if you have a new software
version you just need to run the DUTS to get (1) a confirmation  that
everything  is working as expected fromt he regression test suite and
(2) an updated version of the documentation  with  eexamples  showing
exacly this software version on exactly that hardware.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Es gibt immer genug fuer die Beduerfnisse aller, aber  niemals  genug
fuer die Gier einzelner.                                    -- Ghandi