Using the perf python module with address sanitizer

Using the perf python module with address sanitizer

[Ian Rogers]

Hi,

I just wanted to capture/share the steps I used for getting the perf
python module working with address sanitizer. The process isn't ideal
so hopefully others can suggest improvements and possibly how we can
make this more automatic in the build, or use for automated testing,
etc.

To build perf with address sanitizer you add to the CFLAGS by passing to make:
EXTRA_CFLAGS="-fsanitize=address"

However, this yields a python module that fails with an undefined symbol:
```
$ perf test python -v
--- start ---
test child forked, pid 712715
python usage test: "echo "import sys ; sys.path.insert(0,
'/tmp/perf/python'); import perf" | '/usr/bin/python3' "
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
ImportError: /tmp/perf/python/perf.cpython-311-x86_64-linux-gnu.so:
undefined symbol: __asan_option_detect_stack_use_after_return
---- end(-1) ----
18: 'import perf' in python                                         : FAILED!
```

To resolve this issue add "-shared-libasan" to the CFLAGS:
EXTRA_CFLAGS="-fsanitize=address -shared-libasan"

The build needs to find (on x86) libclang_rt.asan-x86_64.so which for
me on a Debian derived OS wasn't on the LD_LIBRARY_PATH. I found it
with:
```
$ dpkg -L libclang-rt-16-dev|grep libclang_rt.asan-x86_64.so
/usr/lib/llvm-16/lib/clang/16/lib/linux/libclang_rt.asan-x86_64.so
```
I then built with:
```
$ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/llvm-16/lib/clang/16/lib/linux
make EXTRA_CFLAGS="-fsanitize=address -shared-libasan"
```

However, still using the module wasn't successful:
```
$ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/llvm-16/lib/clang/16/lib/linux
perf test python -v
--- start ---
test child forked, pid 720500
python usage test: "echo "import sys ; sys.path.insert(0,
'/tmp/perf/python'); import perf" | '/usr/bin/python3' "
==720503==ASan runtime does not come first in initial library list;
you should either link runtime to your application or manually preload
it with LD_PRELOAD.
---- end(-1) ----
18: 'import perf' in python                                         : FAILED!
```

I resolved this with LD_PRELOAD but there is a leak sanitizer issue
that doesn't relate to importing the perf module:
```
$ LD_PRELOAD=/usr/lib/llvm-16/lib/clang/16/lib/linux/libclang_rt.asan-x86_64.so
perf test python -v
--- start ---
test child forked, pid 720967
python usage test: "echo "import sys ; sys.path.insert(0,
'/tmp/perf/python'); import perf" | '/usr/bin/python3' "

=================================================================
==720971==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 13628 byte(s) in 6 object(s) allocated from:
   #0 0x7fda504e8b42 in __interceptor_malloc
(/usr/lib/llvm-16/lib/clang/16/lib/linux/libclang_rt.asan-x86_64.so+0xe8b42)
   #1 0x4de57f in PyMem_RawMalloc build-static/../Objects/obmalloc.c:586:12
   #2 0x4de57f in _PyObject_Malloc build-static/../Objects/obmalloc.c:2003:11
   #3 0x4de57f in _PyObject_Malloc build-static/../Objects/obmalloc.c:1996:1

SUMMARY: AddressSanitizer: 13628 byte(s) leaked in 6 allocation(s).
---- end(-1) ----
18: 'import perf' in python
```

To clean that up I added "ASAN_OPTIONS=detect_leaks=0":
```
$ LD_PRELOAD=/usr/lib/llvm-16/lib/clang/16/lib/linux/libclang_rt.asan-x86_64.so
ASAN_OPTIONS=detect_leaks=0 perf test python -v
18: 'import perf' in python                                         : Ok
```

So somewhat convoluted but it is possible.

Thanks,
Ian

Re: Using the perf python module with address sanitizer

[Namhyung Kim]

On Thu, Oct 17, 2024 at 09:20:07AM -0700, Ian Rogers wrote:
> Hi,
> 
> I just wanted to capture/share the steps I used for getting the perf
> python module working with address sanitizer. The process isn't ideal
> so hopefully others can suggest improvements and possibly how we can
> make this more automatic in the build, or use for automated testing,
> etc.
> 
> To build perf with address sanitizer you add to the CFLAGS by passing to make:
> EXTRA_CFLAGS="-fsanitize=address"
> 
> However, this yields a python module that fails with an undefined symbol:
> ```
> $ perf test python -v
> --- start ---
> test child forked, pid 712715
> python usage test: "echo "import sys ; sys.path.insert(0,
> '/tmp/perf/python'); import perf" | '/usr/bin/python3' "
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> ImportError: /tmp/perf/python/perf.cpython-311-x86_64-linux-gnu.so:
> undefined symbol: __asan_option_detect_stack_use_after_return
> ---- end(-1) ----
> 18: 'import perf' in python                                         : FAILED!
> ```
> 
> To resolve this issue add "-shared-libasan" to the CFLAGS:
> EXTRA_CFLAGS="-fsanitize=address -shared-libasan"
> 
> The build needs to find (on x86) libclang_rt.asan-x86_64.so which for
> me on a Debian derived OS wasn't on the LD_LIBRARY_PATH. I found it
> with:
> ```
> $ dpkg -L libclang-rt-16-dev|grep libclang_rt.asan-x86_64.so
> /usr/lib/llvm-16/lib/clang/16/lib/linux/libclang_rt.asan-x86_64.so
> ```

I think we can use llvm-config like below.

  $ find $(llvm-config --libdir) -name libclang_rt.asan-$(uname -m).so
  /usr/lib/llvm-16/lib/clang/16/lib/linux/libclang_rt.asan-x86_64.so

Thanks,
Namhyung


> I then built with:
> ```
> $ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/llvm-16/lib/clang/16/lib/linux
> make EXTRA_CFLAGS="-fsanitize=address -shared-libasan"
> ```
> 
> However, still using the module wasn't successful:
> ```
> $ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/llvm-16/lib/clang/16/lib/linux
> perf test python -v
> --- start ---
> test child forked, pid 720500
> python usage test: "echo "import sys ; sys.path.insert(0,
> '/tmp/perf/python'); import perf" | '/usr/bin/python3' "
> ==720503==ASan runtime does not come first in initial library list;
> you should either link runtime to your application or manually preload
> it with LD_PRELOAD.
> ---- end(-1) ----
> 18: 'import perf' in python                                         : FAILED!
> ```
> 
> I resolved this with LD_PRELOAD but there is a leak sanitizer issue
> that doesn't relate to importing the perf module:
> ```
> $ LD_PRELOAD=/usr/lib/llvm-16/lib/clang/16/lib/linux/libclang_rt.asan-x86_64.so
> perf test python -v
> --- start ---
> test child forked, pid 720967
> python usage test: "echo "import sys ; sys.path.insert(0,
> '/tmp/perf/python'); import perf" | '/usr/bin/python3' "
> 
> =================================================================
> ==720971==ERROR: LeakSanitizer: detected memory leaks
> 
> Direct leak of 13628 byte(s) in 6 object(s) allocated from:
>    #0 0x7fda504e8b42 in __interceptor_malloc
> (/usr/lib/llvm-16/lib/clang/16/lib/linux/libclang_rt.asan-x86_64.so+0xe8b42)
>    #1 0x4de57f in PyMem_RawMalloc build-static/../Objects/obmalloc.c:586:12
>    #2 0x4de57f in _PyObject_Malloc build-static/../Objects/obmalloc.c:2003:11
>    #3 0x4de57f in _PyObject_Malloc build-static/../Objects/obmalloc.c:1996:1
> 
> SUMMARY: AddressSanitizer: 13628 byte(s) leaked in 6 allocation(s).
> ---- end(-1) ----
> 18: 'import perf' in python
> ```
> 
> To clean that up I added "ASAN_OPTIONS=detect_leaks=0":
> ```
> $ LD_PRELOAD=/usr/lib/llvm-16/lib/clang/16/lib/linux/libclang_rt.asan-x86_64.so
> ASAN_OPTIONS=detect_leaks=0 perf test python -v
> 18: 'import perf' in python                                         : Ok
> ```
> 
> So somewhat convoluted but it is possible.
> 
> Thanks,
> Ian