RE: [PATCH] build: output a dependency log in build directory

Konstantin Ananyev
2024-07-31T09:18:50Z

> As meson processes our DPDK source tree it manages dependencies
> specified by each individual driver. To enable future analysis of the
> dependency links between components, log the dependencies of each DPDK
> component as it gets processed. This could potentially allow other tools
> to automatically enable or disable components based on the desired end
> components to be built, e.g. if the user requests net/ice, ensure that
> common/iavf is also enabled in the drivers.
>
> The output file produced is in "dot" or "graphviz" format, which allows
> producing a graphical representation of the DPDK dependency tree if so
> desired. For example: "dot -Tpng -O build/deps.dot" to produce the
> image file "build/deps.dot.png".
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

I think it is a great idea.
Acked-by: Konstantin Ananyev <konstantin.ananyev@huawei.com>

> ---
> app/meson.build | 1 +
> buildtools/log-deps.py | 43 ++++++++++++++++++++++++++++++++++++++++++
> buildtools/meson.build | 2 ++
> drivers/meson.build | 1 +
> lib/meson.build | 1 +
> 5 files changed, 48 insertions(+)
> create mode 100644 buildtools/log-deps.py
>
> diff --git a/app/meson.build b/app/meson.build
> index 5b2c80c7a1..6afa457f4c 100644
> --- a/app/meson.build
> +++ b/app/meson.build
> @@ -76,6 +76,7 @@ foreach app:apps
>
> if build
> subdir(name)
> + run_command([log_deps_cmd, name, deps])
> if not build and require_apps
> error('Cannot build explicitly requested app "@0@".\n'.format(name)
> + '\tReason: ' + reason)
> diff --git a/buildtools/log-deps.py b/buildtools/log-deps.py
> new file mode 100644
> index 0000000000..a4331fa15b
> --- /dev/null
> +++ b/buildtools/log-deps.py
> @@ -0,0 +1,43 @@
> +#! /usr/bin/env python3
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 Intel Corporation
> +
> +"""Utility script to build up a list of dependencies from meson."""
> +
> +import os
> +import sys
> +
> +
> +def file_to_list(filename):
> + """Read file into a list of strings."""
> + with open(filename) as f:
> + return f.readlines()
> +
> +
> +def list_to_file(filename, lines):
> + """Write a list of strings out to a file."""
> + with open(filename, 'w') as f:
> + f.writelines(lines)
> +
> +
> +depsfile = f'{os.environ["MESON_BUILD_ROOT"]}/deps.dot'
> +
> +# to reset the deps file on each build, the script is called without any params
> +if len(sys.argv) == 1:
> + os.remove(depsfile)
> + sys.exit(0)
> +
> +try:
> + contents = file_to_list(depsfile)
> +except FileNotFoundError:
> + contents = ['digraph {\n', '}\n']
> +
> +component = sys.argv[1]
> +if len(sys.argv) > 2:
> + contents[-1] = f'"{component}" -> {{ "{"\", \"".join(sys.argv[2:])}" }}\n'
> +else:
> + contents[-1] = f'"{component}"\n'
> +
> +contents.append('}\n')
> +
> +list_to_file(depsfile, contents)
> diff --git a/buildtools/meson.build b/buildtools/meson.build
> index 3adf34e1a8..332f0f3d38 100644
> --- a/buildtools/meson.build
> +++ b/buildtools/meson.build
> @@ -24,6 +24,8 @@ get_numa_count_cmd = py3 + files('get-numa-count.py')
> get_test_suites_cmd = py3 + files('get-test-suites.py')
> has_hugepages_cmd = py3 + files('has-hugepages.py')
> cmdline_gen_cmd = py3 + files('dpdk-cmdline-gen.py')
> +log_deps_cmd = py3 + files('log-deps.py')
> +run_command(log_deps_cmd) # call with no parameters to reset the file
>
> # install any build tools that end-users might want also
> install_data([
> diff --git a/drivers/meson.build b/drivers/meson.build
> index 66931d4241..44935e067c 100644
> --- a/drivers/meson.build
> +++ b/drivers/meson.build
> @@ -154,6 +154,7 @@ foreach subpath:subdirs
> if build
> # pull in driver directory which should update all the local variables
> subdir(drv_path)
> + run_command([log_deps_cmd, drv_path.underscorify(), deps])
>
> if dpdk_conf.get('RTE_IOVA_IN_MBUF') == 0 and require_iova_in_mbuf
> build = false
> diff --git a/lib/meson.build b/lib/meson.build
> index 162287753f..da2815465f 100644
> --- a/lib/meson.build
> +++ b/lib/meson.build
> @@ -160,6 +160,7 @@ foreach l:libraries
>
> if build
> subdir(l)
> + run_command([log_deps_cmd, l, deps])
> if not build and require_libs
> error('Cannot build explicitly requested lib "@0@".\n'.format(name)
> +'\tReason: ' + reason)
> --
> 2.43.0