From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?Q?Simon_K=c3=a5gstr=c3=b6m?= Subject: Re: dpdk pre-patch testing introduction Date: Mon, 2 Nov 2015 08:54:29 +0100 Message-ID: <563716B5.4090809@netinsight.net> References: <86228AFD5BCD8E4EBFD2B90117B5E81E10F379B0@SHSMSX103.ccr.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit To: "Liu, Yong" , "dev@dpdk.org" Return-path: Received: from ernst.netinsight.se (ernst.netinsight.se [194.16.221.21]) by dpdk.org (Postfix) with SMTP id 33BFD5949 for ; Mon, 2 Nov 2015 08:54:39 +0100 (CET) In-Reply-To: <86228AFD5BCD8E4EBFD2B90117B5E81E10F379B0@SHSMSX103.ccr.corp.intel.com> List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Liu! I think this is a really, really great addition! On 2015-10-29 16:19, Liu, Yong wrote: > Currently the patch testing only run unit test and basic function test on this platform. > It can make sure that new patch doesn't break original code and functions. > Due to coverage limitation, it can't verify the functionality of new patch. > If you want to verify your patch's new function, you need to check with tester and perform specific testing on it. Speaking of coverage, I do have a few suggestions: 1. Build and test both with and without optimizations (build with EXTRA_CFLAGS="-O0"). While uncommon, I've seen code break in the non-optimized case because the compiler simply removes some unused code which would otherwise cause link errors. 2. I think it would be very good to collect code coverage for patches as well. (Yes, this is pushing my own tool) I've written a code coverage tool called kcov [1], which only needs debugging information (-g) in the binary to collect code coverage. It outputs results in various formats: lcov-style HTML, cobertura XML and can post directly to coveralls.io. It's also simple to run: kcov --include-pattern=dpdk /tmp/kcov-output ./testpmd [args...] To the mail report, I think it would be good to add information about code coverage increase/decrease with the patch and ideally which lines of the patch which were covered Coverage: 69.2% (+5 lines with the patch) Coverage for the patch: 5/7 lines both of these should be fairly simple to parse out from the XML output. I'd be happy to help you if think it's a good idea! // Simon [1] https://github.com/SimonKagstrom/kcov