Re: [Qemu-devel] [PATCH 1/6] iotests: throw away test timings if args change

["Daniel P. Berrange" <berrange@redhat.com>]

On Mon, Sep 19, 2016 at 02:53:36PM -0500, Eric Blake wrote:
> On 09/19/2016 12:35 PM, Daniel P. Berrange wrote:
> > The 'check' program records timings for each test that
> > is run. These timings are only valid, however, for a
> > particular format/protocol combination. So if frequently
> > running 'check' with a variety of different formats or
> > protocols, the times printed can be very misleading.
> > 
> > Record the protocol/format in the check.time file and
> > throw it away if it doesn't mach the current run args.
> > 
> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> > ---
> >  tests/qemu-iotests/check | 21 +++++++++++++++++++--
> >  1 file changed, 19 insertions(+), 2 deletions(-)
> 
> Rather than completely throwing things away, would it be worth updating
> the check.time file format to track multiple entries? For every distinct
> args seen, track a timing for that combination of args, then when
> starting a test, if a line in the file contains the current args, report
> that old time; if not, then append a line with the new args.  The file
> grows according to how many distinct args combinations you use, and it's
> probably easier to make 'a b' and 'b a' report as different timings even
> if they have the same effect and could share a timing.  We'd also want
> an operation to clean out timings without running tests, particularly if
> timings can otherwise grow huge due to every possible args combination.
> 
> What do you think?

I was afraid someone would suggest a more complex scheme like that :-)

I guess we could keep things simple by not inventing a new format,
but instead of using 'check.time', use 'check.time.$FORMAT-$PROTOCOL'
eg 'check.time.qcow2-file'

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|