BPF List
 help / color / mirror / Atom feed
From: Manu Bretelle <chantr4@gmail.com>
To: bpf@vger.kernel.org, quentin@isovalent.com, andrii@kernel.org,
	daniel@iogearbox.net, ast@kernel.org, martin.lau@linux.dev,
	song@kernel.org, john.fastabend@gmail.com, kpsingh@kernel.org,
	sdf@google.com, haoluo@google.com, jolsa@kernel.org
Subject: [PATCH v1 bpf-next  8/9] bpftool: Add bpftool_tests README.md
Date: Thu, 16 Nov 2023 11:42:35 -0800	[thread overview]
Message-ID: <20231116194236.1345035-9-chantr4@gmail.com> (raw)
In-Reply-To: <20231116194236.1345035-1-chantr4@gmail.com>

A README.md explaining how to run bpftool tests.

Signed-off-by: Manu Bretelle <chantr4@gmail.com>
---
 .../selftests/bpf/bpftool_tests/README.md     | 91 +++++++++++++++++++
 1 file changed, 91 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/bpftool_tests/README.md

diff --git a/tools/testing/selftests/bpf/bpftool_tests/README.md b/tools/testing/selftests/bpf/bpftool_tests/README.md
new file mode 100644
index 000000000000..8ee5d656f6f8
--- /dev/null
+++ b/tools/testing/selftests/bpf/bpftool_tests/README.md
@@ -0,0 +1,91 @@
+## About the testing Framework
+
+The testing framework uses [RUST's testing framework](https://doc.rust-lang.org/rustc/tests/index.html)
+and [libbpf-rs](https://docs.rs/libbpf-rs/latest/libbpf_rs/).
+
+The former takes care of scheduling tests and reporting their successes/failures.
+The latter is used to load bpf programs, maps, and possibly interact with them
+programatically through libbpf API.
+This allows us to set the environment we want to test and check that `bpftool`
+does what we expect.
+
+This document assumes you have [`cargo` and `rust` installed](https://doc.rust-lang.org/cargo/getting-started/installation.html).
+
+## Testing bpftool
+
+This should be no different than typical [`cargo test`](https://doc.rust-lang.org/cargo/commands/cargo-test.html)
+but there is a few subtleties to consider when running `bpftool` tests:
+
+1. bpftool needs to run with root privileges for the most part. So the runner needs to run as root.
+1. each tests load a program, possibly modify it, and check expectations. In order to be deterministic, tests need to run serially.
+
+### Environment variable
+
+A few environment variable can be used to control the behaviour of the tests:
+- `RUST_TEST_THREADS`: This should be set to 1 to run one test at a time and avoid tests to step onto each others.
+- `BPFTOOL_PATH`: Allow passing an alternate location for `bpftool`. Default: `/usr/sbin/bpftool`
+
+### Running the test suite
+
+Here are a few options to make this happen:
+
+```
+# build the test binary, extract the test executable location
+# and run it with sudo, 1 test at a time.
+eval sudo BPFTOOL_PATH=$(pwd)/../bpftool RUST_TEST_THREADS=1 \
+    $(cargo test --no-run \
+        --message-format=json | jq '. | select(.executable != null ).executable' \
+    )
+```
+
+or alternatively, one can use the [`CARGO_TARGET_<triple>_RUNNER` environment variable](https://doc.rust-lang.org/cargo/reference/environment-variables.html#:~:text=CARGO_TARGET_%3Ctriple%3E_RUNNER).
+
+The benefit of that approach is that compilation errors will show directly in the terminal.
+
+```
+CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUNNER="sudo -E" \
+    BPFTOOL_PATH=$(pwd)/../bpftool \
+    RUST_TEST_THREADS=1 \
+    cargo test
+```
+
+### Running tests against built kernel/bpftool
+
+Using [vmtest](https://github.com/danobi/vmtest):
+
+```
+$ KERNEL_REPO=~/devel/bpf-next/
+$ vmtest -k $KERNEL_REPO/arch/x86_64/boot/bzImage "BPFTOOL_PATH=$KERNEL_REPO/tools/bpf/bpftool/bpftool RUST_TEST_THREADS=1 cargo test"
+=> bzImage
+===> Booting
+===> Setting up VM
+===> Running command
+    Finished test [unoptimized + debuginfo] target(s) in 2.06s
+     Running unittests src/main.rs (target/debug/deps/bpftool_tests-afa5a7eef3cdeafb)
+
+running 11 tests
+test bpftool_tests::run_bpftool ... ok
+test bpftool_tests::run_bpftool_map_dump_id ... ok
+test bpftool_tests::run_bpftool_map_list ... ok
+test bpftool_tests::run_bpftool_map_pids ... ok
+test bpftool_tests::run_bpftool_prog_list ... ok
+test bpftool_tests::run_bpftool_prog_pids ... ok
+test bpftool_tests::run_bpftool_prog_show_id ... ok
+test bpftool_tests::run_bpftool_struct_ops_can_unregister_id ... ok
+test bpftool_tests::run_bpftool_struct_ops_can_unregister_name ... ok
+test bpftool_tests::run_bpftool_struct_ops_dump_name ... ok
+test bpftool_tests::run_bpftool_struct_ops_list ... ok
+
+test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.88s
+```
+
+the return code will be 0 on success, non-zero otherwise.
+
+
+## Caveat
+
+Currently, libbpf-sys crate either uses a vendored libbpf, or the system one.
+This could possibly limit tests against features that are being introduced.
+
+That being said, this is not a blocker now, and can be fixed upstream.
+https://github.com/libbpf/libbpf-sys/issues/70 tracks this on libbpf-sys side.
-- 
2.39.3


  parent reply	other threads:[~2023-11-16 19:43 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-11-16 19:42 [PATCH v1 bpf-next 0/9] bpftool: Add end-to-end testing Manu Bretelle
2023-11-16 19:42 ` [PATCH v1 bpf-next 1/9] bpftool: add testing skeleton Manu Bretelle
2023-11-21  1:37   ` Alexei Starovoitov
2023-11-21 16:26     ` Quentin Monnet
2023-11-21 16:42       ` Alexei Starovoitov
2023-11-21 19:50         ` Andrii Nakryiko
2023-11-27 17:07           ` Quentin Monnet
2023-11-27 18:39             ` Andrii Nakryiko
2023-12-15  6:26               ` Manu Bretelle
2023-11-27 17:07         ` Quentin Monnet
2023-12-15  6:37           ` Manu Bretelle
2023-11-21 16:26   ` Quentin Monnet
2023-11-16 19:42 ` [PATCH v1 bpf-next 2/9] bpftool: add libbpf-rs dependency and minimal bpf program Manu Bretelle
2023-11-16 19:42 ` [PATCH v1 bpf-next 3/9] bpftool: open and load bpf object Manu Bretelle
2023-11-16 19:42 ` [PATCH v1 bpf-next 4/9] bpftool: Add test to verify that pids are associated to maps Manu Bretelle
2023-11-21 16:26   ` Quentin Monnet
2023-11-16 19:42 ` [PATCH v1 bpf-next 5/9] bpftool: add test for bpftool prog Manu Bretelle
2023-11-21 16:26   ` Quentin Monnet
2023-11-16 19:42 ` [PATCH v1 bpf-next 6/9] bpftool: test that we can dump and read the content of a map Manu Bretelle
2023-11-21 16:26   ` Quentin Monnet
2023-11-16 19:42 ` [PATCH v1 bpf-next 7/9] bpftool: Add struct_ops tests Manu Bretelle
2023-11-16 19:42 ` Manu Bretelle [this message]
2023-11-21 16:26   ` [PATCH v1 bpf-next 8/9] bpftool: Add bpftool_tests README.md Quentin Monnet
2023-11-16 19:42 ` [PATCH v1 bpf-next 9/9] bpftool: Add Makefile to facilitate bpftool_tests usage Manu Bretelle
2023-11-21 16:26   ` Quentin Monnet

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231116194236.1345035-9-chantr4@gmail.com \
    --to=chantr4@gmail.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=haoluo@google.com \
    --cc=john.fastabend@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kpsingh@kernel.org \
    --cc=martin.lau@linux.dev \
    --cc=quentin@isovalent.com \
    --cc=sdf@google.com \
    --cc=song@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox