BPF List
 help / color / mirror / Atom feed
* [PATCH bpf-next v3 0/8] Add Memory Sanitizer support
@ 2023-02-14 23:12 Ilya Leoshkevich
  2023-02-14 23:12 ` [PATCH bpf-next v3 1/8] libbpf: Introduce bpf_{btf,link,map,prog}_get_info_by_fd() Ilya Leoshkevich
                   ` (8 more replies)
  0 siblings, 9 replies; 17+ messages in thread
From: Ilya Leoshkevich @ 2023-02-14 23:12 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko
  Cc: bpf, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
	Ilya Leoshkevich

v2: https://lore.kernel.org/bpf/20230210001210.395194-1-iii@linux.ibm.com/
v2 -> v3:
- Improve bpftool commit message, shorten error messages (Quentin).
- Drop perf patch (Andrii).
- Drop integrated patches.

v1: https://lore.kernel.org/bpf/20230208205642.270567-1-iii@linux.ibm.com/
v1 -> v2:
- Apply runqslower's EXTRA_CFLAGS and EXTRA_LDFLAGS unconditionally.
- Use u64 for uretprobe_byname2_rc.
- Use BPF_UPROBE() instead of PT_REGS_xxx().
- Use void * instead of char * for pointer arithmetic.
- Rename libbpf_mark_defined() to __libbpf_mark_mem_written(), add
  convenience wrappers.
- Add a comment about defined(__has_feature) &&
  __has_feature(memory_sanitizer).
- Extract is_percpu_bpf_map_type().
- Introduce bpf_get_{map,prog,link,btf}_info_by_fd() and convert all
  code to use them. If it's too early for that, in particular for
  samples and perf, the respective patches can be dropped.
- Unpoison infos returned by these functions, paying attention to
  potentially missing fields. Use macros to reduce boilerplate.
- Move capget() unpoisoning to LLVM [5].
- With that, only a few cases remain where data needs to be
  unpoisoned in selftests.

Hi,

This series adds support for building selftests with Memory Sanitizer
[1] - a compiler instrumentation for detecting usages of undefined
memory.

The primary motivation is to make sure that such usages do not occur
during testing, since the ones that have not been caught yet are likely
to affect the CI results on s390x. The secondary motivation is to be
able to use libbpf in applications instrumented with MSan (it requires
all code running in a process to be instrumented).

The setup I'm using is as follows:

- Instrumented zlib-ng and elfutils.
- Patched LLVM [2, 3, 4].
- Clang-built kernel.
- Building tests with MSan:

      make \
          -C tools/testing/selftests/bpf \
          CC="ccache clang-17" \
          LD=ld \
          HOSTCC="ccache clang-17" \
          HOSTLD=ld \
          LLVM=1 \
          LLVM_SUFFIX=-17 \
          OBJCOPY=objcopy \
          CLANG="ccache clang-17" \
          LLVM_STRIP=llvm-strip-17 \
          LLC=llc-17 \
          BPF_GCC= \
          SAN_CFLAGS="-fsanitize=memory \
                      -fsanitize-memory-track-origins \
                      -I$PWD/../zlib-ng/dist/include \
                      -I$PWD/../elfutils/dist/include" \
          SAN_LDFLAGS="-fsanitize=memory \
                       -fsanitize-memory-track-origins \
                       -L$PWD/../zlib-ng/dist/lib \
                       -L$PWD/../elfutils/dist/lib" \
          CROSS_COMPILE=s390x-linux-gnu-

  It's a lot of options, but most of them are trivial: setting up LLVM
  toolchain, taking in account s390x quirks, setting up MSan and
  disabling bpf-gcc. The CROSS_COMPILE one is a hack: instrumenting
  bpftool turned out to be too complicated from the build system
  perspective, so having CROSS_COMPILE forces compiling the host libbpf
  uninstrumented and guest libbpf instrumented.

- Running tests with MSan on s390x:

      LD_LIBRARY_PATH=<instrumented libs> ./test_progs
      ...
      Summary: 282/1624 PASSED, 23 SKIPPED, 4 FAILED
  
  The 4 failures happen without MSan too, they are already known and
  denylisted.

Best regards,
Ilya

[1] https://clang.llvm.org/docs/MemorySanitizer.html
[2] https://reviews.llvm.org/D143296
[3] https://reviews.llvm.org/D143330
[4] https://reviews.llvm.org/D143660

Ilya Leoshkevich (8):
  libbpf: Introduce bpf_{btf,link,map,prog}_get_info_by_fd()
  libbpf: Use bpf_{btf,link,map,prog}_get_info_by_fd()
  bpftool: Use bpf_{btf,link,map,prog}_get_info_by_fd()
  samples/bpf: Use bpf_{btf,link,map,prog}_get_info_by_fd()
  selftests/bpf: Use bpf_{btf,link,map,prog}_get_info_by_fd()
  libbpf: Factor out is_percpu_bpf_map_type()
  libbpf: Add MSan annotations
  selftests/bpf: Add MSan annotations

 samples/bpf/test_map_in_map_user.c            |   2 +-
 samples/bpf/xdp1_user.c                       |   2 +-
 samples/bpf/xdp_adjust_tail_user.c            |   2 +-
 samples/bpf/xdp_fwd_user.c                    |   4 +-
 samples/bpf/xdp_redirect_cpu_user.c           |   4 +-
 samples/bpf/xdp_rxq_info_user.c               |   2 +-
 samples/bpf/xdp_sample_pkts_user.c            |   2 +-
 samples/bpf/xdp_tx_iptunnel_user.c            |   2 +-
 tools/bpf/bpftool/btf.c                       |  13 +-
 tools/bpf/bpftool/btf_dumper.c                |   4 +-
 tools/bpf/bpftool/cgroup.c                    |   4 +-
 tools/bpf/bpftool/common.c                    |  13 +-
 tools/bpf/bpftool/link.c                      |   4 +-
 tools/bpf/bpftool/main.h                      |   3 +-
 tools/bpf/bpftool/map.c                       |   8 +-
 tools/bpf/bpftool/prog.c                      |  22 +--
 tools/bpf/bpftool/struct_ops.c                |   6 +-
 tools/lib/bpf/bpf.c                           | 179 +++++++++++++++++-
 tools/lib/bpf/bpf.h                           |  13 ++
 tools/lib/bpf/btf.c                           |   9 +-
 tools/lib/bpf/libbpf.c                        |  26 +--
 tools/lib/bpf/libbpf.map                      |   5 +
 tools/lib/bpf/libbpf_internal.h               |  46 +++++
 tools/lib/bpf/netlink.c                       |   2 +-
 tools/lib/bpf/ringbuf.c                       |   4 +-
 .../bpf/map_tests/map_in_map_batch_ops.c      |   2 +-
 .../selftests/bpf/prog_tests/bpf_iter.c       |   8 +-
 .../selftests/bpf/prog_tests/bpf_obj_id.c     |  20 +-
 .../selftests/bpf/prog_tests/bpf_tcp_ca.c     |   3 +
 tools/testing/selftests/bpf/prog_tests/btf.c  |  24 +--
 .../selftests/bpf/prog_tests/btf_map_in_map.c |   2 +-
 .../selftests/bpf/prog_tests/check_mtu.c      |   2 +-
 .../selftests/bpf/prog_tests/enable_stats.c   |   2 +-
 .../selftests/bpf/prog_tests/fexit_bpf2bpf.c  |  14 +-
 .../bpf/prog_tests/flow_dissector_reattach.c  |  10 +-
 .../bpf/prog_tests/libbpf_get_fd_by_id_opts.c |   4 +-
 .../selftests/bpf/prog_tests/lsm_cgroup.c     |   3 +-
 .../selftests/bpf/prog_tests/metadata.c       |   8 +-
 tools/testing/selftests/bpf/prog_tests/mmap.c |   2 +-
 .../selftests/bpf/prog_tests/perf_link.c      |   2 +-
 .../selftests/bpf/prog_tests/pinning.c        |   2 +-
 .../selftests/bpf/prog_tests/prog_run_opts.c  |   2 +-
 .../selftests/bpf/prog_tests/recursion.c      |   4 +-
 .../selftests/bpf/prog_tests/send_signal.c    |   2 +
 .../selftests/bpf/prog_tests/sockmap_basic.c  |   6 +-
 .../bpf/prog_tests/task_local_storage.c       |   8 +-
 .../testing/selftests/bpf/prog_tests/tc_bpf.c |   4 +-
 .../bpf/prog_tests/tp_attach_query.c          |   9 +-
 .../bpf/prog_tests/unpriv_bpf_disabled.c      |   8 +-
 .../selftests/bpf/prog_tests/verif_stats.c    |   5 +-
 .../selftests/bpf/prog_tests/xdp_attach.c     |   4 +-
 .../selftests/bpf/prog_tests/xdp_bonding.c    |   3 +
 .../bpf/prog_tests/xdp_cpumap_attach.c        |   8 +-
 .../bpf/prog_tests/xdp_devmap_attach.c        |   8 +-
 .../selftests/bpf/prog_tests/xdp_info.c       |   2 +-
 .../selftests/bpf/prog_tests/xdp_link.c       |  10 +-
 tools/testing/selftests/bpf/test_maps.c       |   2 +-
 .../selftests/bpf/test_skb_cgroup_id_user.c   |   2 +-
 .../bpf/test_tcp_check_syncookie_user.c       |   2 +-
 tools/testing/selftests/bpf/test_verifier.c   |   8 +-
 tools/testing/selftests/bpf/testing_helpers.c |   2 +-
 tools/testing/selftests/bpf/xdp_synproxy.c    |  15 +-
 62 files changed, 429 insertions(+), 174 deletions(-)

-- 
2.39.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2023-02-27 21:23 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-02-14 23:12 [PATCH bpf-next v3 0/8] Add Memory Sanitizer support Ilya Leoshkevich
2023-02-14 23:12 ` [PATCH bpf-next v3 1/8] libbpf: Introduce bpf_{btf,link,map,prog}_get_info_by_fd() Ilya Leoshkevich
2023-02-16 23:08   ` Andrii Nakryiko
2023-02-16 23:37     ` Andrii Nakryiko
2023-02-14 23:12 ` [PATCH bpf-next v3 2/8] libbpf: Use bpf_{btf,link,map,prog}_get_info_by_fd() Ilya Leoshkevich
2023-02-14 23:12 ` [PATCH bpf-next v3 3/8] bpftool: " Ilya Leoshkevich
2023-02-15 12:53   ` Quentin Monnet
2023-02-14 23:12 ` [PATCH bpf-next v3 4/8] samples/bpf: " Ilya Leoshkevich
2023-02-14 23:12 ` [PATCH bpf-next v3 5/8] selftests/bpf: " Ilya Leoshkevich
2023-02-14 23:12 ` [PATCH bpf-next v3 6/8] libbpf: Factor out is_percpu_bpf_map_type() Ilya Leoshkevich
2023-02-14 23:12 ` [PATCH bpf-next v3 7/8] libbpf: Add MSan annotations Ilya Leoshkevich
2023-02-16 23:28   ` Andrii Nakryiko
2023-02-21  0:46     ` Ilya Leoshkevich
2023-02-27 21:23       ` Andrii Nakryiko
2023-02-14 23:12 ` [PATCH bpf-next v3 8/8] selftests/bpf: " Ilya Leoshkevich
2023-02-16 23:29   ` Andrii Nakryiko
2023-02-17  0:10 ` [PATCH bpf-next v3 0/8] Add Memory Sanitizer support patchwork-bot+netdevbpf

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox