Re: dpdk pre-patch testing introduction

["Simon Kågström" <simon.kagstrom@netinsight.net>]

Hi Liu!

I think this is a really, really great addition!

On 2015-10-29 16:19, Liu, Yong wrote:

> Currently the patch testing only run unit test and basic function test on this platform.
> It can make sure that new patch doesn't break original code and functions.
> Due to coverage limitation, it can't verify the functionality of new patch.
> If you want to verify your patch's new function, you need to check with tester and perform specific testing on it.

Speaking of coverage, I do have a few suggestions:

1. Build and test both with and without optimizations (build with
EXTRA_CFLAGS="-O0"). While uncommon, I've seen code break in the
non-optimized case because the compiler simply removes some unused code
which would otherwise cause link errors.

2. I think it would be very good to collect code coverage for patches as
well. (Yes, this is pushing my own tool) I've written a code coverage
tool called kcov [1], which only needs debugging information (-g) in the
binary to collect code coverage.

It outputs results in various formats: lcov-style HTML, cobertura XML
and can post directly to coveralls.io. It's also simple to run:

   kcov --include-pattern=dpdk /tmp/kcov-output ./testpmd [args...]

To the mail report, I think it would be good to add information about
code coverage increase/decrease with the patch and ideally which lines
of the patch which were covered

   Coverage: 69.2% (+5 lines with the patch)
   Coverage for the patch: 5/7 lines

both of these should be fairly simple to parse out from the XML output.


I'd be happy to help you if think it's a good idea!

// Simon

[1] https://github.com/SimonKagstrom/kcov