Re: possible dependency error?

[Brian Norris <briannorris@chromium.org>]

Necromancing an old thread, since it identifies the same problem I've
been poking at recently:

On Thu, May 18, 2023 at 05:26:28PM +0900, Masahiro Yamada wrote:
> (+CC: Josh Poimboeuf,Peter Zijlstra, objtool maintainer)
> 
> On Thu, May 18, 2023 at 8:27 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
> >
> > This is on v6.4-rc1.  I fat-fingered the make target (I intended
> > "pciehp.o", not "pciehp.c"), then interrupted the build when I noticed
> > my mistake:
> >
> >   06:04:15 ~/linux (hotplug)$ make drivers/pci/hotplug/pciehp.c
> >     SYNC    include/config/auto.conf.cmd
> >   ^Cmake: *** [include/config/auto.conf.cmd] Deleting file 'include/generated/rustc_cfg'
> >   make: *** [include/config/auto.conf.cmd] Deleting file 'include/generated/autoconf.h'
> >   make[2]: *** [scripts/kconfig/Makefile:77: syncconfig] Interrupt
> >   make[1]: *** [Makefile:692: syncconfig] Interrupt
> >   make: *** [Makefile:793: include/config/auto.conf.cmd] Interrupt
> >
> > Subsequent builds now fail ("pciehp.o" is *also* an incorrect target,
> > but doesn't seem related to the error):
> >
> >   06:04:22 ~/linux (hotplug)$ make drivers/pci/hotplug/pciehp.o
> >     SYNC    include/config/auto.conf.cmd
> >     UPD     include/config/kernel.release
> >     UPD     include/generated/utsrelease.h
> >     UPD     include/generated/compile.h
> >     CC      scripts/mod/empty.o
> >     MKELF   scripts/mod/elfconfig.h
> >     HOSTCC  scripts/mod/modpost.o
> >     CC      scripts/mod/devicetable-offsets.s
> >     HOSTCC  scripts/mod/file2alias.o
> >     HOSTCC  scripts/mod/sumversion.o
> >     HOSTLD  scripts/mod/modpost
> >     CC      kernel/bounds.s
> >     CC      arch/x86/kernel/asm-offsets.s
> >     CALL    scripts/checksyscalls.sh
> >     DESCEND objtool
> >     HOSTCC  /home/bjorn/linux/tools/objtool/fixdep.o
> >     HOSTLD  /home/bjorn/linux/tools/objtool/fixdep-in.o
> >     LINK    /home/bjorn/linux/tools/objtool/fixdep
> >   make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/sys_errlist.h', needed by '/home/bjorn/linux/tools/objtool/libsubcmd/exec-cmd.o'.  Stop.
> >   make[3]: *** [Makefile:80: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd-in.o] Error 2
> >   make[2]: *** [Makefile:78: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd.a] Error 2
> >   make[1]: *** [Makefile:73: objtool] Error 2
> >   make: *** [Makefile:1440: tools/objtool] Error 2
> >
> > I finally got the right target, but the build still fails:
> >
> >   06:04:39 ~/linux (hotplug)$ make drivers/pci/hotplug/
> >     CALL    scripts/checksyscalls.sh
> >     DESCEND objtool
> >   make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/sys_errlist.h', needed by '/home/bjorn/linux/tools/objtool/libsubcmd/exec-cmd.o'.  Stop.
> >   make[3]: *** [Makefile:80: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd-in.o] Error 2
> >   make[2]: *** [Makefile:78: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd.a] Error 2
> >   make[1]: *** [Makefile:73: objtool] Error 2
> >   make: *** [Makefile:1440: tools/objtool] Error 2
> >
> > After "make distclean", everything works as expected, so maybe this is
> > just the expected behavior after my initial user error?  I dunno; it
> > seemed surprising.  Just FYI.

I believe we've been hitting a similar issue at $JOB, which looks like
the following (this is on a 5.15-ish kernel, but AFAICT everything is
still relevant):

make[5]: *** No rule to make target '[...]/tools/include/linux/compiler.h', needed by '[...]/tools/bpf/resolve_btfids/libsubcmd/exec-cmd.o'.  Stop.
make[4]: *** [Makefile:59: [...]/tools/bpf/resolve_btfids/libsubcmd/libsubcmd-in.o] Error 2
make[3]: *** [Makefile:45: [...]/tools/bpf/resolve_btfids//libsubcmd/libsubcmd.a] Error 2
make[2]: *** [Makefile:72: bpf/resolve_btfids] Error 2
make[1]: *** [[...]/Makefile:1401: tools/bpf/resolve_btfids] Error 2

This particular case happens for us when the source tree is moving, but
we're sharing an O= cache. This may or may not be a good idea, but
AFAICT there still is a real bug underneath, which I explore below.

> I do not know what is happening on your build machine,
> but judging from the error log, something went wrong
> while building objtool.
> 
> objtool Makefile is not a part of Kbuild.
> The maintainers of objtool may have some insight.

I'm no maintainer, but I found that the .exec-cmd.o.cmd dep file is
generated incorrectly due to improper fixdep dependencies:

$ head -2 [...]/tools/bpf/resolve_btfids/libsubcmd/.exec-cmd.o.cmd
# cannot find fixdep ([...]/tools/bpf/resolve_btfids/libsubcmd//fixdep)
# using basic dep data

Now, this soft error is normally OK, as long as you don't have any
missing or moved headers. But if these moved around, then normally the
fixdep'd dependencies would help rebuild (and regenerate .cmd files with
new paths) silently.

The bad .cmd file is reliably reproduced by:

# for an easier-to-build target that also builds libsubcmd:
cd tools/objtool
# for maximum cleanliness:
git clean -xfd ..
make
head -2 libsubcmd/.exec-cmd.o.cmd

(NB: if you look hard enough, you'll notice that we have a similar
"cannot find fixdep" error for tools/.../.fixdep.o.cmd too. I have some
analysis at https://issuetracker.google.com/313508829#comment32 --
this one is publicly accessible -- but its solution would be more
complex. I may raise a separate thread.)

The following patch fixes libsubcmd stuff for me. I can resubmit in a
proper patch form if desired, or feel free to scrape it as-is.

Signed-off-by: Brian Norris <briannorris@chromium.org>
---

diff --git a/tools/lib/subcmd/Makefile b/tools/lib/subcmd/Makefile
index b87213263a5e..59b09f280e49 100644
--- a/tools/lib/subcmd/Makefile
+++ b/tools/lib/subcmd/Makefile
@@ -76,7 +76,7 @@ include $(srctree)/tools/build/Makefile.include
 
 all: fixdep $(LIBFILE)
 
-$(SUBCMD_IN): FORCE
+$(SUBCMD_IN): fixdep FORCE
 	@$(MAKE) $(build)=libsubcmd
 
 $(LIBFILE): $(SUBCMD_IN)