[PATCH v3 03/10] gen_compile_commands: do not support .cmd files under tools/ directory

Masahiro Yamada
2020-08-22T14:57:22Z

The tools/ directory uses a different build system, and the format of
.cmd files is different because the tools builds run in a different
work directory.

Supporting two formats compilicates the script.

The only loss by this change is objtool.

Also, rename the confusing variable 'relative_path' because it is
not necessarily a relative path. When the output directory is not
the direct child of the source tree (e.g. O=foo/bar), it is an
absolute path. Rename it to 'file_path'.

os.path.join(root_directory, file_path) works whether the file_path
is relative or not. If file_path is already absolute, it returns it
as-is.

I used os.path.abspath() to normalize file paths. If you run this
script against the kernel built with O=foo option, the file_path
contains '../' patterns. os.path.abspath() fixes up 'foo/bar/../baz'
into 'foo/baz', and produces a cleaner commands_database.json.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
---

Changes in v3:
- Add a comment about why I used os.path.abspath()

Changes in v2:
- New patch

scripts/gen_compile_commands.py | 32 ++++++++++++--------------------
1 file changed, 12 insertions(+), 20 deletions(-)

diff --git a/scripts/gen_compile_commands.py b/scripts/gen_compile_commands.py
index 535248cf2d7e..49fff0b0b385 100755
--- a/scripts/gen_compile_commands.py
+++ b/scripts/gen_compile_commands.py
@@ -59,23 +59,21 @@ def parse_arguments():
return args.log_level, directory, output

-def process_line(root_directory, file_directory, command_prefix, relative_path):
+def process_line(root_directory, command_prefix, file_path):
"""Extracts information from a .cmd line and creates an entry from it.

Args:
root_directory: The directory that was searched for .cmd files. Usually
used directly in the "directory" entry in compile_commands.json.
- file_directory: The path to the directory the .cmd file was found in.
command_prefix: The extracted command line, up to the last element.
- relative_path: The .c file from the end of the extracted command.
- Usually relative to root_directory, but sometimes relative to
- file_directory and sometimes neither.
+ file_path: The .c file from the end of the extracted command.
+ Usually relative to root_directory, but sometimes absolute.

Returns:
An entry to append to compile_commands.

Raises:
- ValueError: Could not find the extracted file based on relative_path and
+ ValueError: Could not find the extracted file based on file_path and
root_directory or file_directory.
"""
# The .cmd files are intended to be included directly by Make, so they
@@ -84,20 +82,14 @@ def process_line(root_directory, file_directory, command_prefix, relative_path):
# by Make, so this code replaces the escaped version with '#'.
prefix = command_prefix.replace('\#', '#').replace('$(pound)', '#')

- cur_dir = root_directory
- expected_path = os.path.join(cur_dir, relative_path)
- if not os.path.exists(expected_path):
- # Try using file_directory instead. Some of the tools have a different
- # style of .cmd file than the kernel.
- cur_dir = file_directory
- expected_path = os.path.join(cur_dir, relative_path)
- if not os.path.exists(expected_path):
- raise ValueError('File %s not in %s or %s' %
- (relative_path, root_directory, file_directory))
+ # Use os.path.abspath() to normalize the path resolving '.' and '..' .
+ abs_path = os.path.abspath(os.path.join(root_directory, file_path))
+ if not os.path.exists(abs_path):
+ raise ValueError('File %s not found' % abs_path)
return {
- 'directory': cur_dir,
- 'file': relative_path,
- 'command': prefix + relative_path,
+ 'directory': root_directory,
+ 'file': abs_path,
+ 'command': prefix + file_path,
}

@@ -122,7 +114,7 @@ def main():
result = line_matcher.match(f.readline())
if result:
try:
- entry = process_line(directory, dirpath,
+ entry = process_line(directory,
result.group(1), result.group(2))
compile_commands.append(entry)
except ValueError as err:
--
2.25.1