Re: [Qemu-devel] [PATCH 1/6] iotests: throw away test timings if args change

[Eric Blake <eblake@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 1499 bytes --]

On 09/19/2016 12:35 PM, Daniel P. Berrange wrote:
> The 'check' program records timings for each test that
> is run. These timings are only valid, however, for a
> particular format/protocol combination. So if frequently
> running 'check' with a variety of different formats or
> protocols, the times printed can be very misleading.
> 
> Record the protocol/format in the check.time file and
> throw it away if it doesn't mach the current run args.
> 
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
>  tests/qemu-iotests/check | 21 +++++++++++++++++++--
>  1 file changed, 19 insertions(+), 2 deletions(-)

Rather than completely throwing things away, would it be worth updating
the check.time file format to track multiple entries? For every distinct
args seen, track a timing for that combination of args, then when
starting a test, if a line in the file contains the current args, report
that old time; if not, then append a line with the new args.  The file
grows according to how many distinct args combinations you use, and it's
probably easier to make 'a b' and 'b a' report as different timings even
if they have the same effect and could share a timing.  We'd also want
an operation to clean out timings without running tests, particularly if
timings can otherwise grow huge due to every possible args combination.

What do you think?

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]