Pixel-perfect frame checks in IGT Chamelium tests and CRC

From: Paul Kocialkowski <paul.kocialkowski@linux.intel.com>
To: Lyude Paul <lyude@redhat.com>, chamelium-external@google.com
Cc: intel-gfx@lists.freedesktop.org
Subject: Pixel-perfect frame checks in IGT Chamelium tests and CRC
Date: Thu, 15 Jun 2017 16:57:04 +0300	[thread overview]
Message-ID: <1497535024.1427.16.camel@linux.intel.com> (raw)

Hi,

So far, there are two ways of testing for pixel-perfect frames using the
Chamelium that are in IGT. The first one grabs a full frame from the Chamelium
and compares it pixel-to-pixel with the cairo reference, which works well for
DP/HDMI.

For VGA, this is probably not the case (because the link is analogue). In that
case, I will look into implementing some fuzzy testing, probably inspired by
what piglit (probably) does to compare output frames with references.

For pixel-perfect testing, grabbing a full frame and testing it with memcmp
comes with a significant time penalty (about 2 seconds for 1080p). The Chamelium
also provides a CRC mechanism that is faster and does not require retrieving the
frame, that IGT currently also supports. It compares the CRC calculated by the
Chamelium (implemented in the HDL) with a hardcoded reference value.

This approach currently fails for me (the values I get don't match the hardcoded
reference). There are reasons why it is not really reasonable: fonts rendering
may change between machines (e.g. use of anti-aliasing) and cairo version
changes could introduce slight rendering changes too (not to mention changes in
the test pattern itself). So instead of comparing the CRC with a hardcoded
reference value, I think it would make a lot more sense to actually calculate
the CRC based on the cairo image that is the actual reference (and that we
should assume may change between runs/machines).

I am currently looking into the CRC calculation mechanism used by the Chamelium
and trying to reproduce it in C code. Is this a known algorithm for which a
reference/optimized implementation exists, or something custom that the folks
over at Google came up with?

Any thoughts, comments or suggestions?

Cheers!

-- 
Paul Kocialkowski <paul.kocialkowski@linux.intel.com>
Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx