Re: [PATCH 3/3] objtool: move tools/objtool/ to scripts/objtool/

Masahiro Yamada
2019-03-05T16:09:19Z

On Tue, Mar 5, 2019 at 11:46 PM Josh Poimboeuf <jpoimboe@redhat.com> wrote:
>
> On Tue, Mar 05, 2019 at 02:48:16PM +0900, Masahiro Yamada wrote:
> > Tools that are globally needed for building the kernel are supposed to
> > be put in the scripts/ directory, but objtool is the exceptional one.
> >
> > People were confused by objtool's Makefile since most of tools run on
> > the target whereas objtool runs on the host machine. Some time ago,
> > Douglas Anderson suggested to move tools/objtool/ to scripts/tools/
> > (https://patchwork.kernel.org/patch/9983535/#20996057), but we did
> > not go as far as submitting a patch at that time.
> >
> > Recently, Robin Meijboom reported that the generated files for objtool
> > were never cleaned up. (https://patchwork.kernel.org/patch/10813797/)
> >
> > So, I looked into this to fix the distortion. Move tools/objtool/ to
> > scripts/objtool/, and rewrite Makefiles in the Kbuild-ish manner.
> >
> > I also cleaned up the top-level Makefile by moving the libelf check
> > scripts/objtool/Makefile.
> >
> > I see some benefits in this.
> >
> > [1] generated files under scripts/ are cleaned up by 'make mrproper'
> >
> > Currently, the generated files under tools/objtool/ are not cleaned up
> > since the build system under tools/ is a complete different world.
> >
> > If we want to clean them up, it should be done by 'make mrproper'
> > instead of 'make clean' since objtool is needed for building external
> > modules.
> >
> > This commit naturally solves the issue since Kbuild cleans under
> > scripts/ by 'make mrproper'.
>
> Yes, but this can be solved instead with a one-line patch.
>
> > [2] build log looks nicer
> >
> > Currently, the build log under tools/objtool/ shows absolute paths, and
> > displays 'DESCEND objtool' every time for incremental building.
> >
> > Switching over to the standard Kbuild scheme will improve the log.
> >
> > Before:
> >
> > DESCEND objtool
> > HOSTCC /home/masahiro/workspace/linux/tools/objtool/fixdep.o
> > HOSTLD /home/masahiro/workspace/linux/tools/objtool/fixdep-in.o
> > LINK /home/masahiro/workspace/linux/tools/objtool/fixdep
> > CC /home/masahiro/workspace/linux/tools/objtool/exec-cmd.o
> > CC /home/masahiro/workspace/linux/tools/objtool/help.o
> > CC /home/masahiro/workspace/linux/tools/objtool/pager.o
> > CC /home/masahiro/workspace/linux/tools/objtool/parse-options.o
> > CC /home/masahiro/workspace/linux/tools/objtool/run-command.o
> > CC /home/masahiro/workspace/linux/tools/objtool/sigchain.o
> > CC /home/masahiro/workspace/linux/tools/objtool/subcmd-config.o
> > LD /home/masahiro/workspace/linux/tools/objtool/libsubcmd-in.o
> > AR /home/masahiro/workspace/linux/tools/objtool/libsubcmd.a
> > GEN /home/masahiro/workspace/linux/tools/objtool/arch/x86/lib/inat-tables.c
> > CC /home/masahiro/workspace/linux/tools/objtool/arch/x86/decode.o
> > LD /home/masahiro/workspace/linux/tools/objtool/arch/x86/objtool-in.o
> > CC /home/masahiro/workspace/linux/tools/objtool/builtin-check.o
> > CC /home/masahiro/workspace/linux/tools/objtool/builtin-orc.o
> > CC /home/masahiro/workspace/linux/tools/objtool/check.o
> > CC /home/masahiro/workspace/linux/tools/objtool/orc_gen.o
> > CC /home/masahiro/workspace/linux/tools/objtool/orc_dump.o
> > CC /home/masahiro/workspace/linux/tools/objtool/elf.o
> > CC /home/masahiro/workspace/linux/tools/objtool/special.o
> > CC /home/masahiro/workspace/linux/tools/objtool/objtool.o
> > CC /home/masahiro/workspace/linux/tools/objtool/libstring.o
> > CC /home/masahiro/workspace/linux/tools/objtool/str_error_r.o
> > LD /home/masahiro/workspace/linux/tools/objtool/objtool-in.o
> > LINK /home/masahiro/workspace/linux/tools/objtool/objtool
> >
> > After:
> >
> > HOSTCC scripts/objtool/builtin-check.o
> > HOSTCC scripts/objtool/builtin-orc.o
> > HOSTCC scripts/objtool/check.o
> > HOSTCC scripts/objtool/orc_gen.o
> > HOSTCC scripts/objtool/orc_dump.o
> > HOSTCC scripts/objtool/elf.o
> > HOSTCC scripts/objtool/special.o
> > HOSTCC scripts/objtool/objtool.o
> > HOSTCC scripts/objtool/libstring.o
> > HOSTCC scripts/objtool/str_error_r.o
> > HOSTCC scripts/objtool/exec-cmd.o
> > HOSTCC scripts/objtool/pager.o
> > HOSTCC scripts/objtool/parse-options.o
> > HOSTCC scripts/objtool/run-command.o
> > HOSTCC scripts/objtool/sigchain.o
> > HOSTCC scripts/objtool/subcmd-config.o
> > AWK scripts/objtool/arch/x86/lib/inat-tables.c
> > HOSTCC scripts/objtool/arch/x86/decode.o
> > HOSTLD scripts/objtool/objtool
>
> That does look better.
>
> > [3] simplify distro package script
> >
> > The special handling in scripts/package/buildeb is no longer needed
> > since all host programs are copied to linux-headers packages.
>
> I do see the benefits. In many ways objtool is a more natural fit in
> the scripts dir. But there are also some downsides to this change:
>
> - This will make it harder to package objtool as a standalone tool. It
> has a lot of functionality that could be useful to other non-kernel
> projects.

If it is really useful for other projects,
I'd like to see it as a real standalone tool,
i.e. split as a separate project.

> - It shares libsubcmd with perf. Including the subcmd .c files from
> tools is hacky.

Yes, hacky. We are shifting the ugliness between C and Build system.

But, this hack can be solved; if subcmd library is useful, you can copy
only necessary code into the objtool directory. You do not need all
helpers from libsubcmd.

> - It's disruptive: it will break all the out-of-tree distro kernel
> packaging scripts which now look for objtool in tools.

All artifacts under scripts/ should be contained in the package.
So, it should work.

> You're right that objtool isn't a natural fit in tools, because it's
> also used as part of the build. But it's not a natural fit in scripts
> either. I think we've resolved most of those issues and it seems to be
> working well these days.
>
> So instead of disrupting everything because "make mrproper" doesn't
> work, I think I'd rather just do the following:
>
> diff --git a/Makefile b/Makefile
> index ac5ac28a24e9..7e6696c9b862 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1364,7 +1364,7 @@ PHONY += $(mrproper-dirs) mrproper
> $(mrproper-dirs):
> $(Q)$(MAKE) $(clean)=$(patsubst _mrproper_%,%,$@)
>
> -mrproper: clean $(mrproper-dirs)
> +mrproper: clean $(mrproper-dirs) tools/objtool_clean
> $(call cmd,rmdirs)
> $(call cmd,rmfiles)
>

Works, and the clean log is also brilliant.

I see the same log even when
there is nothing to clean.

masahiro@grover:~/ref/linux$ make mrproper
CLEAN .
CLEAN arch/x86/entry/vdso
CLEAN usr
CLEAN arch/x86/tools
CLEAN .tmp_versions
CLEAN scripts/basic
CLEAN scripts/kconfig
CLEAN scripts/mod
CLEAN scripts/selinux/genheaders
CLEAN scripts/selinux/mdp
CLEAN scripts
DESCEND objtool
CLEAN objtool
CLEAN include/config include/generated arch/x86/include/generated
CLEAN .config
masahiro@grover:~/ref/linux$ make mrproper
DESCEND objtool
CLEAN objtool
masahiro@grover:~/ref/linux$ make mrproper
DESCEND objtool
CLEAN objtool

--
Best Regards
Masahiro Yamada