qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Thomas Huth <thuth@redhat.com>
To: peter.maydell@linaro.org, qemu-devel@nongnu.org
Cc: Alexander Bulekov <alxndr@bu.edu>
Subject: [PULL 07/10] docs/fuzz: add instructions for generating a coverage report
Date: Tue, 21 Jul 2020 10:10:52 +0200	[thread overview]
Message-ID: <20200721081055.14073-8-thuth@redhat.com> (raw)
In-Reply-To: <20200721081055.14073-1-thuth@redhat.com>

From: Alexander Bulekov <alxndr@bu.edu>

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20200706195534.14962-5-alxndr@bu.edu>
[thuth: Replaced --enable-sanitizers with --enable-fuzzing]
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/devel/fuzzing.txt | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/docs/devel/fuzzing.txt b/docs/devel/fuzzing.txt
index 6d18115239..96d71c94d7 100644
--- a/docs/devel/fuzzing.txt
+++ b/docs/devel/fuzzing.txt
@@ -85,6 +85,25 @@ better coverage performance, depending on the target.
 Note that libFuzzer's exact behavior will depend on the version of
 clang and libFuzzer used to build the device fuzzers.
 
+== Generating Coverage Reports ==
+Code coverage is a crucial metric for evaluating a fuzzer's performance.
+libFuzzer's output provides a "cov: " column that provides a total number of
+unique blocks/edges covered. To examine coverage on a line-by-line basis we
+can use Clang coverage:
+
+ 1. Configure libFuzzer to store a corpus of all interesting inputs (see
+    CORPUS_DIR above)
+ 2. ./configure the QEMU build with:
+    --enable-fuzzing \
+    --extra-cflags="-fprofile-instr-generate -fcoverage-mapping"
+ 3. Re-run the fuzzer. Specify $CORPUS_DIR/* as an argument, telling libfuzzer
+    to execute all of the inputs in $CORPUS_DIR and exit. Once the process
+    exits, you should find a file, "default.profraw" in the working directory.
+ 4. Execute these commands to generate a detailed HTML coverage-report:
+ llvm-profdata merge -output=default.profdata default.profraw
+ llvm-cov show ./path/to/qemu-fuzz-i386 -instr-profile=default.profdata \
+ --format html -output-dir=/path/to/output/report
+
 == Adding a new fuzzer ==
 Coverage over virtual devices can be improved by adding additional fuzzers.
 Fuzzers are kept in tests/qtest/fuzz/ and should be added to
-- 
2.18.1



  parent reply	other threads:[~2020-07-21  8:14 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-21  8:10 [PULL 00/10] qtest / fuzzer patches Thomas Huth
2020-07-21  8:10 ` [PULL 01/10] scripts/oss-fuzz: Limit target list to i386-softmmu Thomas Huth
2020-07-21  8:10 ` [PULL 02/10] fuzz: Fix leak when assembling datadir path string Thomas Huth
2020-07-21  8:10 ` [PULL 03/10] gitlab-ci.yml: Add oss-fuzz build tests Thomas Huth
2020-07-21  8:10 ` [PULL 04/10] fuzz: build without AddressSanitizer, by default Thomas Huth
2020-07-21  8:10 ` [PULL 05/10] docs/fuzz: describe building fuzzers with enable-sanitizers Thomas Huth
2020-07-21  8:10 ` [PULL 06/10] docs/fuzz: add information about useful libFuzzer flags Thomas Huth
2020-07-21  8:10 ` Thomas Huth [this message]
2020-07-21  8:10 ` [PULL 08/10] MAINTAINERS: Extend the device fuzzing section Thomas Huth
2020-07-21  8:10 ` [PULL 09/10] msf2: Unbreak device-list-properties for "msf-soc" Thomas Huth
2020-07-21  8:10 ` [PULL 10/10] hw: Mark nd_table[] misuse in realize methods FIXME Thomas Huth
2020-07-21 13:03 ` [PULL 00/10] qtest / fuzzer patches Peter Maydell

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200721081055.14073-8-thuth@redhat.com \
    --to=thuth@redhat.com \
    --cc=alxndr@bu.edu \
    --cc=peter.maydell@linaro.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).