[PATCH] gen_compile_commands: Look in KBUILD_OUTPUT if set

[PATCH] gen_compile_commands: Look in KBUILD_OUTPUT if set

[Paul Barker]

If someone is already using the KBUILD_OUTPUT environment variable to
specify the directory where object files are placed, they shouldn't need
to repeat the same information to gen_compile_commands.py.

Signed-off-by: Paul Barker <paul@pbarker.dev>
---
 scripts/clang-tools/gen_compile_commands.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/scripts/clang-tools/gen_compile_commands.py b/scripts/clang-tools/gen_compile_commands.py
index 96e6e46ad1a702cb0fad5d524a9a02d222b236ec..7b94a2ffba0b4d5f1290b51bd602fb3f33acce6a 100755
--- a/scripts/clang-tools/gen_compile_commands.py
+++ b/scripts/clang-tools/gen_compile_commands.py
@@ -39,8 +39,9 @@ def parse_arguments():
     parser = argparse.ArgumentParser(description=usage)
 
     directory_help = ('specify the output directory used for the kernel build '
-                      '(defaults to the working directory)')
-    parser.add_argument('-d', '--directory', type=str, default='.',
+                      '(defaults to $KBUILD_OUTPUT (if set) or the working directory)')
+    parser.add_argument('-d', '--directory', type=str,
+                        default=os.environ.get('KBUILD_OUTPUT', '.'),
                         help=directory_help)
 
     output_help = ('path to the output command database (defaults to ' +

---
base-commit: d69eb204c255c35abd9e8cb621484e8074c75eaa
change-id: 20250905-gen_compile_commands-b03d9c923fe5

Best regards,
-- 
Paul Barker

Re: [PATCH] gen_compile_commands: Look in KBUILD_OUTPUT if set

[Justin Stitt]

Hi,

On Fri, Sep 05, 2025 at 11:17:43AM +0100, Paul Barker wrote:
> If someone is already using the KBUILD_OUTPUT environment variable to
> specify the directory where object files are placed, they shouldn't need
> to repeat the same information to gen_compile_commands.py.
> 
> Signed-off-by: Paul Barker <paul@pbarker.dev>
> ---
>  scripts/clang-tools/gen_compile_commands.py | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/clang-tools/gen_compile_commands.py b/scripts/clang-tools/gen_compile_commands.py
> index 96e6e46ad1a702cb0fad5d524a9a02d222b236ec..7b94a2ffba0b4d5f1290b51bd602fb3f33acce6a 100755
> --- a/scripts/clang-tools/gen_compile_commands.py
> +++ b/scripts/clang-tools/gen_compile_commands.py
> @@ -39,8 +39,9 @@ def parse_arguments():
>      parser = argparse.ArgumentParser(description=usage)
>  
>      directory_help = ('specify the output directory used for the kernel build '
> -                      '(defaults to the working directory)')
> -    parser.add_argument('-d', '--directory', type=str, default='.',
> +                      '(defaults to $KBUILD_OUTPUT (if set) or the working directory)')
> +    parser.add_argument('-d', '--directory', type=str,
> +                        default=os.environ.get('KBUILD_OUTPUT', '.'),
>                          help=directory_help)
>  
>      output_help = ('path to the output command database (defaults to ' +
> 

Thinking out loud: It might make sense to also change the default output
path in some cases but not in all cases. For my clangd setup in vim, it
does some discovery for a compile_commands.json and I have some
different ones in various build-* directories -- I guess it'd be cool if
they were automatically placed in their appropriate spot. With all that
being said probably YAGNI.

Tested-by: Justin Stitt <justinstitt@google.com>

> ---
> base-commit: d69eb204c255c35abd9e8cb621484e8074c75eaa
> change-id: 20250905-gen_compile_commands-b03d9c923fe5
> 
> Best regards,
> -- 
> Paul Barker
> 
>

Thanks
Justin

Re: [PATCH] gen_compile_commands: Look in KBUILD_OUTPUT if set

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 2761 bytes --]

On Fri, 2025-09-05 at 09:34 -0700, Justin Stitt wrote:
> Hi,
> 
> On Fri, Sep 05, 2025 at 11:17:43AM +0100, Paul Barker wrote:
> > If someone is already using the KBUILD_OUTPUT environment variable to
> > specify the directory where object files are placed, they shouldn't need
> > to repeat the same information to gen_compile_commands.py.
> > 
> > Signed-off-by: Paul Barker <paul@pbarker.dev>
> > ---
> >  scripts/clang-tools/gen_compile_commands.py | 5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> > 
> > diff --git a/scripts/clang-tools/gen_compile_commands.py b/scripts/clang-tools/gen_compile_commands.py
> > index 96e6e46ad1a702cb0fad5d524a9a02d222b236ec..7b94a2ffba0b4d5f1290b51bd602fb3f33acce6a 100755
> > --- a/scripts/clang-tools/gen_compile_commands.py
> > +++ b/scripts/clang-tools/gen_compile_commands.py
> > @@ -39,8 +39,9 @@ def parse_arguments():
> >      parser = argparse.ArgumentParser(description=usage)
> >  
> >      directory_help = ('specify the output directory used for the kernel build '
> > -                      '(defaults to the working directory)')
> > -    parser.add_argument('-d', '--directory', type=str, default='.',
> > +                      '(defaults to $KBUILD_OUTPUT (if set) or the working directory)')
> > +    parser.add_argument('-d', '--directory', type=str,
> > +                        default=os.environ.get('KBUILD_OUTPUT', '.'),
> >                          help=directory_help)
> >  
> >      output_help = ('path to the output command database (defaults to ' +
> > 
> 
> Thinking out loud: It might make sense to also change the default output
> path in some cases but not in all cases. For my clangd setup in vim, it
> does some discovery for a compile_commands.json and I have some
> different ones in various build-* directories -- I guess it'd be cool if
> they were automatically placed in their appropriate spot. With all that
> being said probably YAGNI.

I think it makes sense to place the output file in the current directory by
default if you run gen_compile_commands.py directly.

The `make compile_commands.json` target places it in the output directory, and
`make rust-analyzer` does the same for the rust-project.json file. I did think
about whether we should change these, since clangd and rust-analyzer look for
the relevant files in the source tree or its parent directories. But people may
be using multiple output directories for different configs or archs, so writing
the files to the source tree isn't a good default for everyone.

For my case I'm just symlinking the relevant files back in to the source tree
after building so that clangd and rust-analyzer can find them.

Thanks for testing!

-- 
Paul Barker

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: [PATCH] gen_compile_commands: Look in KBUILD_OUTPUT if set

[Nathan Chancellor]

Hi Paul

On Fri, Sep 05, 2025 at 06:26:32PM +0100, Paul Barker wrote:
> On Fri, 2025-09-05 at 09:34 -0700, Justin Stitt wrote:
> > On Fri, Sep 05, 2025 at 11:17:43AM +0100, Paul Barker wrote:
> > > If someone is already using the KBUILD_OUTPUT environment variable to
> > > specify the directory where object files are placed, they shouldn't need
> > > to repeat the same information to gen_compile_commands.py.
> > > 
> > > Signed-off-by: Paul Barker <paul@pbarker.dev>
> > > ---
> > >  scripts/clang-tools/gen_compile_commands.py | 5 +++--
> > >  1 file changed, 3 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/scripts/clang-tools/gen_compile_commands.py b/scripts/clang-tools/gen_compile_commands.py
> > > index 96e6e46ad1a702cb0fad5d524a9a02d222b236ec..7b94a2ffba0b4d5f1290b51bd602fb3f33acce6a 100755
> > > --- a/scripts/clang-tools/gen_compile_commands.py
> > > +++ b/scripts/clang-tools/gen_compile_commands.py
> > > @@ -39,8 +39,9 @@ def parse_arguments():
> > >      parser = argparse.ArgumentParser(description=usage)
> > >  
> > >      directory_help = ('specify the output directory used for the kernel build '
> > > -                      '(defaults to the working directory)')
> > > -    parser.add_argument('-d', '--directory', type=str, default='.',
> > > +                      '(defaults to $KBUILD_OUTPUT (if set) or the working directory)')
> > > +    parser.add_argument('-d', '--directory', type=str,
> > > +                        default=os.environ.get('KBUILD_OUTPUT', '.'),
> > >                          help=directory_help)
> > >  
> > >      output_help = ('path to the output command database (defaults to ' +
> > > 
> > 
> > Thinking out loud: It might make sense to also change the default output
> > path in some cases but not in all cases. For my clangd setup in vim, it
> > does some discovery for a compile_commands.json and I have some
> > different ones in various build-* directories -- I guess it'd be cool if
> > they were automatically placed in their appropriate spot. With all that
> > being said probably YAGNI.
> 
> I think it makes sense to place the output file in the current directory by
> default if you run gen_compile_commands.py directly.
> 
> The `make compile_commands.json` target places it in the output directory, and

Yeah, I think it should be made clearer in the commit message that this
change should only impact people who run the script manually. When it is
generated through the compile_commands.json make target (which is the
preferred way IMO), KBUILD_OUTPUT should already be respected because
gen_compile_commands.py is run with $(objtree) as the current working
directory.

I am fine with the actual contents of the change though, it does not
feel like much of a burden to support this scenario.

> `make rust-analyzer` does the same for the rust-project.json file. I did think
> about whether we should change these, since clangd and rust-analyzer look for
> the relevant files in the source tree or its parent directories. But people may
> be using multiple output directories for different configs or archs, so writing
> the files to the source tree isn't a good default for everyone.

Yeah, I do not think this should be generated in the source tree
unconditionally since those commands are tied to that build directory
due to CONFIG_ options and such.

Cheers,
Nathan