[RFC PATCH 0/1] Add support for code-coverage analysis

[RFC PATCH 0/1] Add support for code-coverage analysis

[Felix Moessbauer]

This patch has been developed as part of the DPDK Userspace Summit Hackathon.
It provides a PoC for code-coverage analysis for the DPDK project.

To generate the report, a developer simply follows the official
meson coverage workflow, described in [1].
In doing so, both an HTML report, as well as an XML version is generated
for further processing.

In short, the following steps are required:

- install gcovr
- meson -Db_coverage=true build-cov
- meson compile -C build-cov
- meson test -C build-cov --suite fast-tests
- ninja coverage -C build-cov

[1] https://mesonbuild.com/howtox.html#producing-a-coverage-report

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (1):
  Add basic support for code coverage analysis

 gcovr.cfg | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 gcovr.cfg

-- 
2.30.2

[RFC PATCH 1/1] Add basic support for code coverage analysis

[Felix Moessbauer]

This patch adds basic support to get meaningful
code coverage statistics for some central components.
To keep things simple, we only focus on the parts that are
tested as part of the "fast-tests" suite.
This includes the lib as well as drivers that do not require
special hardware to be tested.

By providing the gcovr.cfg file in the project root,
modern versions of meson (>=0.63) can pass that information
to gcovr, making it possible to configure the coverage target
of meson.
This enables us to use the default meson coverage infrastructure
and customize it for the needs of the DPDK project.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Acked-by: William Lam <william.lam@bytedance.com>
Acked-by: Chriss Windle <chrisswindle@microsoft.com>
---
 gcovr.cfg | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 gcovr.cfg

diff --git a/gcovr.cfg b/gcovr.cfg
new file mode 100644
index 0000000000..6e247499e8
--- /dev/null
+++ b/gcovr.cfg
@@ -0,0 +1,8 @@
+filter = lib
+filter = drivers/bus/pci
+filter = drivers/bus/vdev
+filter = drivers/mempool/ring
+filter = drivers/event/skeleton
+filter = drivers/net/ring
+filter = drivers/net/null
+filter = drivers/raw/skeleton
-- 
2.30.2

Re: [RFC PATCH 1/1] Add basic support for code coverage analysis

[Stephen Hemminger]

On Tue,  6 Sep 2022 18:43:09 +0200
Felix Moessbauer <felix.moessbauer@siemens.com> wrote:

> This patch adds basic support to get meaningful
> code coverage statistics for some central components.
> To keep things simple, we only focus on the parts that are
> tested as part of the "fast-tests" suite.
> This includes the lib as well as drivers that do not require
> special hardware to be tested.
> 
> By providing the gcovr.cfg file in the project root,
> modern versions of meson (>=0.63) can pass that information
> to gcovr, making it possible to configure the coverage target
> of meson.
> This enables us to use the default meson coverage infrastructure
> and customize it for the needs of the DPDK project.
> 
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> Acked-by: William Lam <william.lam@bytedance.com>
> Acked-by: Chriss Windle <chrisswindle@microsoft.com>

Not sure why this patch stalled. Seems ok but the set of drivers seems
quite small. Would be good to do it across wider set.

Re: [RFC PATCH 0/1] Add support for code-coverage analysis

[Aaron Conole]

Felix Moessbauer <felix.moessbauer@siemens.com> writes:

> This patch has been developed as part of the DPDK Userspace Summit Hackathon.
> It provides a PoC for code-coverage analysis for the DPDK project.
>
> To generate the report, a developer simply follows the official
> meson coverage workflow, described in [1].
> In doing so, both an HTML report, as well as an XML version is generated
> for further processing.
>
> In short, the following steps are required:
>
> - install gcovr
> - meson -Db_coverage=true build-cov
> - meson compile -C build-cov
> - meson test -C build-cov --suite fast-tests
> - ninja coverage -C build-cov
>
> [1] https://mesonbuild.com/howtox.html#producing-a-coverage-report

+1 for this effort, I also think that would be a great addition so show
our current coverage via a service or something like coveralls.io and
possibly the dashboard.  I don't know if coveralls would support the
series branches we create, so I will spend a bit of time looking into
it.

BTW, can you also include information on the current coverage % of code
base?  Last I recall, some of the libraries were quite low (bpf and
bbdev stick in my mind as having very low coverage).

> Best regards,
> Felix Moessbauer
> Siemens AG
>
> Felix Moessbauer (1):
>   Add basic support for code coverage analysis
>
>  gcovr.cfg | 8 ++++++++
>  1 file changed, 8 insertions(+)
>  create mode 100644 gcovr.cfg