[PATCH 1/2] perf build: Add loading python binding check to python.so build

[PATCH 1/2] perf build: Add loading python binding check to python.so build

[Yang Jihong]

Add loading python binding check to python.so build so that problem can be
detected in advance in the build phase instead of being left to `perf test`
phase.

In normal scenarios, the original build is not affected:

  $ cd tools/perf
  $ rm -rf /tmp/perf; mkdir /tmp/perf; make O=/tmp/perf
  $ echo $?
  0
  $ cd /tmp/perf
  $ ./perf test python
   19: 'import perf' in python                    : Ok

Create an error scenario, for example, delete util/rlimit.c from
util/python-ext-sources:

  $ cd tools/perf
  $ sed -i 's@util/rlimit.c@#util/rlimit.c@g' util/python-ext-sources
  $ grep rlimit util/python-ext-sources
  #util/rlimit.c
  $ rm -rf /tmp/perf; mkdir /tmp/perf; make JOBS=1 O=/tmp/perf
  <SNIP>
    GEN     /tmp/perf/python/perf.cpython-310-x86_64-linux-gnu.so
  Error: Load python binding failed. See /tmp/perf/python_ext_build/lib//build.log for more details
  make[2]: *** [Makefile.perf:644: /tmp/perf/python/perf.cpython-310-x86_64-linux-gnu.so] Error 1
  make[1]: *** [Makefile.perf:242: sub-make] Error 2
  make: *** [Makefile:70: all] Error 2
  $ cat /tmp/perf/python_ext_build/lib//build.log
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  ImportError: /tmp/perf/python_ext_build/lib/perf.cpython-310-x86_64-linux-gnu.so: undefined symbol: rlimit__increase_nofile

Signed-off-by: Yang Jihong <yangjihong1@huawei.com>
---
 tools/perf/Makefile.perf | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index d80dcaa5a1e3..a2449c4890ad 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -645,7 +645,13 @@ $(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX): $(PYTHON_EXT_SRCS) $(PYTHON_EXT_
         CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS)' \
 	  $(PYTHON_WORD) util/setup.py \
 	  --quiet build_ext; \
-	cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/
+	$(PYTHON_WORD) -c 'import sys; sys.path.insert(0, "$(PYTHON_EXTBUILD_LIB)"); import perf' 2>$(PYTHON_EXTBUILD_LIB)/build.log; \
+	if [ $$? -ne 0 ]; then \
+		echo "Error: Load python binding failed. See $(PYTHON_EXTBUILD_LIB)/build.log for more details"; \
+		exit 1; \
+	else \
+		cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/; \
+	fi
 
 python_perf_target:
 	@echo "Target is: $(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX)"
-- 
2.34.1

[PATCH 2/2] perf tools: Add the python_ext_build directory to .gitignore

[Yang Jihong]

`python_ext_build` is the build directory for python.so, ignore it for
cleaner git status.

Signed-off-by: Yang Jihong <yangjihong1@huawei.com>
---
 tools/perf/.gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
index f533e76fb480..ee5c14f3b8b1 100644
--- a/tools/perf/.gitignore
+++ b/tools/perf/.gitignore
@@ -49,3 +49,4 @@ libtraceevent/
 libtraceevent_plugins/
 fixdep
 Documentation/doc.dep
+python_ext_build/
-- 
2.34.1

Re: [PATCH 1/2] perf build: Add loading python binding check to python.so build

[Namhyung Kim]

Hello,

On Mon, Oct 30, 2023 at 4:16 AM Yang Jihong <yangjihong1@huawei.com> wrote:
>
> Add loading python binding check to python.so build so that problem can be
> detected in advance in the build phase instead of being left to `perf test`
> phase.
>
> In normal scenarios, the original build is not affected:
>
>   $ cd tools/perf
>   $ rm -rf /tmp/perf; mkdir /tmp/perf; make O=/tmp/perf
>   $ echo $?
>   0
>   $ cd /tmp/perf
>   $ ./perf test python
>    19: 'import perf' in python                    : Ok
>
> Create an error scenario, for example, delete util/rlimit.c from
> util/python-ext-sources:
>
>   $ cd tools/perf
>   $ sed -i 's@util/rlimit.c@#util/rlimit.c@g' util/python-ext-sources
>   $ grep rlimit util/python-ext-sources
>   #util/rlimit.c
>   $ rm -rf /tmp/perf; mkdir /tmp/perf; make JOBS=1 O=/tmp/perf
>   <SNIP>
>     GEN     /tmp/perf/python/perf.cpython-310-x86_64-linux-gnu.so
>   Error: Load python binding failed. See /tmp/perf/python_ext_build/lib//build.log for more details
>   make[2]: *** [Makefile.perf:644: /tmp/perf/python/perf.cpython-310-x86_64-linux-gnu.so] Error 1
>   make[1]: *** [Makefile.perf:242: sub-make] Error 2
>   make: *** [Makefile:70: all] Error 2
>   $ cat /tmp/perf/python_ext_build/lib//build.log
>   Traceback (most recent call last):
>     File "<string>", line 1, in <module>
>   ImportError: /tmp/perf/python_ext_build/lib/perf.cpython-310-x86_64-linux-gnu.so: undefined symbol: rlimit__increase_nofile
>
> Signed-off-by: Yang Jihong <yangjihong1@huawei.com>

For both patches,
Acked-by: Namhyung Kim <namhyung@kernel.org>

Thanks,
Namhyung


> ---
>  tools/perf/Makefile.perf | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index d80dcaa5a1e3..a2449c4890ad 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -645,7 +645,13 @@ $(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX): $(PYTHON_EXT_SRCS) $(PYTHON_EXT_
>          CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS)' \
>           $(PYTHON_WORD) util/setup.py \
>           --quiet build_ext; \
> -       cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/
> +       $(PYTHON_WORD) -c 'import sys; sys.path.insert(0, "$(PYTHON_EXTBUILD_LIB)"); import perf' 2>$(PYTHON_EXTBUILD_LIB)/build.log; \
> +       if [ $$? -ne 0 ]; then \
> +               echo "Error: Load python binding failed. See $(PYTHON_EXTBUILD_LIB)/build.log for more details"; \
> +               exit 1; \
> +       else \
> +               cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/; \
> +       fi
>
>  python_perf_target:
>         @echo "Target is: $(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX)"
> --
> 2.34.1
>

Re: [PATCH 1/2] perf build: Add loading python binding check to python.so build

[Arnaldo Carvalho de Melo]

Em Mon, Oct 30, 2023 at 11:14:37AM +0000, Yang Jihong escreveu:
> Add loading python binding check to python.so build so that problem can be
> detected in advance in the build phase instead of being left to `perf test`
> phase.
> 
> In normal scenarios, the original build is not affected:
> 
>   $ cd tools/perf
>   $ rm -rf /tmp/perf; mkdir /tmp/perf; make O=/tmp/perf
>   $ echo $?
>   0
>   $ cd /tmp/perf
>   $ ./perf test python
>    19: 'import perf' in python                    : Ok
> 
> Create an error scenario, for example, delete util/rlimit.c from
> util/python-ext-sources:

This will make _all_ perf builds incur the cost of this :-\

I do it with an alias because I notice it is unfortunately common for
people to send pull requests without running 'perf test' before, but
with this in place the 'make -C tools/perf build' will be made slower
for no reason :-\

I picked the second patch, thanks.

- Arnaldo
 
>   $ cd tools/perf
>   $ sed -i 's@util/rlimit.c@#util/rlimit.c@g' util/python-ext-sources
>   $ grep rlimit util/python-ext-sources
>   #util/rlimit.c
>   $ rm -rf /tmp/perf; mkdir /tmp/perf; make JOBS=1 O=/tmp/perf
>   <SNIP>
>     GEN     /tmp/perf/python/perf.cpython-310-x86_64-linux-gnu.so
>   Error: Load python binding failed. See /tmp/perf/python_ext_build/lib//build.log for more details
>   make[2]: *** [Makefile.perf:644: /tmp/perf/python/perf.cpython-310-x86_64-linux-gnu.so] Error 1
>   make[1]: *** [Makefile.perf:242: sub-make] Error 2
>   make: *** [Makefile:70: all] Error 2
>   $ cat /tmp/perf/python_ext_build/lib//build.log
>   Traceback (most recent call last):
>     File "<string>", line 1, in <module>
>   ImportError: /tmp/perf/python_ext_build/lib/perf.cpython-310-x86_64-linux-gnu.so: undefined symbol: rlimit__increase_nofile
> 
> Signed-off-by: Yang Jihong <yangjihong1@huawei.com>
> ---
>  tools/perf/Makefile.perf | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index d80dcaa5a1e3..a2449c4890ad 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -645,7 +645,13 @@ $(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX): $(PYTHON_EXT_SRCS) $(PYTHON_EXT_
>          CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS)' \
>  	  $(PYTHON_WORD) util/setup.py \
>  	  --quiet build_ext; \
> -	cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/
> +	$(PYTHON_WORD) -c 'import sys; sys.path.insert(0, "$(PYTHON_EXTBUILD_LIB)"); import perf' 2>$(PYTHON_EXTBUILD_LIB)/build.log; \
> +	if [ $$? -ne 0 ]; then \
> +		echo "Error: Load python binding failed. See $(PYTHON_EXTBUILD_LIB)/build.log for more details"; \
> +		exit 1; \
> +	else \
> +		cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/; \
> +	fi
>  
>  python_perf_target:
>  	@echo "Target is: $(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX)"
> -- 
> 2.34.1
> 

-- 

- Arnaldo