Difficulties around "fixdep" for the usage of a kernel build output directory

Difficulties around "fixdep" for the usage of a kernel build output directory

[SF Markus Elfring]

Hello,

The file "README" contains a section "BUILD directory for the kernel".
I would like to use the functionality which is described there.

One of my commands for such a build try is working as desired.


elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20161014 && my_off=-O0 && for X in off; do my_output_dir=${my_build_dir}unchanged/optimisation/${X} && my_var="my_${X}" && my_parameters="-Wall -std=gnu89 -fomit-frame-pointer ${!my_var}" && make O="${my_output_dir}" HOSTCC='/usr/bin/gcc-6' HOSTCFLAGS="${my_parameters}" allmodconfig; done
…
#
# configuration written to .config
#


I stumble on the following messages for another build command.


elfring@Sonne:~/Projekte/Linux/next-patched> my_off=-O0 && for X in off; do my_output_dir=${my_build_dir}unchanged/optimisation/${X} && my_var="my_${X}" && my_parameters="-Wall -std=gnu89 -fomit-frame-pointer ${!my_var}" && LANG=C make V=2 O="${my_output_dir}" HOSTCC='/usr/bin/gcc-6' HOSTCFLAGS="-S ${my_parameters}" drivers/md/; done
make[1]: Entering directory …
  HOSTCC  scripts/basic/fixdep - due to command line change
scripts/basic/fixdep: line 1: .file: command not found
scripts/basic/fixdep: line 2: .comm: command not found
…
scripts/basic/fixdep: line 6: .section: command not found
scripts/basic/fixdep: line 7: .align: command not found
scripts/basic/fixdep: line 8: .LC0:: command not found
scripts/basic/fixdep: line 9: .string: command not found
…


I wonder why this approach does not work so far.
I would appreciate your advices.

Regards,
Markus

Re: Difficulties around "fixdep" for the usage of a kernel build output directory

[Jim Davis]

On Tue, Oct 18, 2016 at 6:54 AM, SF Markus Elfring
<elfring@users.sourceforge.net> wrote:
> Hello,
>
> The file "README" contains a section "BUILD directory for the kernel".
> I would like to use the functionality which is described there.
>
> One of my commands for such a build try is working as desired.
>
>
> elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20161014 && my_off=-O0 && for X in off; do my_output_dir=${my_build_dir}unchanged/optimisation/${X} && my_var="my_${X}" && my_parameters="-Wall -std=gnu89 -fomit-frame-pointer ${!my_var}" && make O="${my_output_dir}" HOSTCC='/usr/bin/gcc-6' HOSTCFLAGS="${my_parameters}" allmodconfig; done
> …
> #
> # configuration written to .config
> #
>
>
> I stumble on the following messages for another build command.
>
>
> elfring@Sonne:~/Projekte/Linux/next-patched> my_off=-O0 && for X in off; do my_output_dir=${my_build_dir}unchanged/optimisation/${X} && my_var="my_${X}" && my_parameters="-Wall -std=gnu89 -fomit-frame-pointer ${!my_var}" && LANG=C make V=2 O="${my_output_dir}" HOSTCC='/usr/bin/gcc-6' HOSTCFLAGS="-S ${my_parameters}" drivers/md/; done
> make[1]: Entering directory …
>   HOSTCC  scripts/basic/fixdep - due to command line change
> scripts/basic/fixdep: line 1: .file: command not found
> scripts/basic/fixdep: line 2: .comm: command not found
> …
> scripts/basic/fixdep: line 6: .section: command not found
> scripts/basic/fixdep: line 7: .align: command not found
> scripts/basic/fixdep: line 8: .LC0:: command not found
> scripts/basic/fixdep: line 9: .string: command not found
> …
>
>
> I wonder why this approach does not work so far.
> I would appreciate your advices.

That's a very complicated command line.  Does something basic, like

make O=/tmp/$$ allmodconfig
make O=/tmp/$$

work for you?

-- 
Jim

Re: Difficulties around "fixdep" for the usage of a kernel build output directory

[SF Markus Elfring]

>> elfring@Sonne:~/Projekte/Linux/next-patched> my_off=-O0 && for X in off; do my_output_dir=${my_build_dir}unchanged/optimisation/${X} && my_var="my_${X}" && my_parameters="-Wall -std=gnu89 -fomit-frame-pointer ${!my_var}" && LANG=C make V=2 O="${my_output_dir}" HOSTCC='/usr/bin/gcc-6' HOSTCFLAGS="-S ${my_parameters}" drivers/md/; done
…
> That's a very complicated command line.

I find it "advanced".

It shows a few of my imaginations around a specific software development task.


> Does something basic, like
> 
> make O=/tmp/$$ allmodconfig
> make O=/tmp/$$
> 
> work for you?

Not really. - I would like to achieve the automatic selection of a few
compilation settings by a for loop which I reduced in the shown example
to one element.

I am looking for a "convenient" way to get assembler source files from
the preferrred C compiler for some software build configurations.
I hope that I can avoid the switching of compilation results in the source
file directory hierarchy so that an occasional disassembly by a tool
like "objdump" could be compared.

Regards,
Markus

Re: Difficulties around "fixdep" for the usage of a kernel build output directory

[Jim Davis]

On Tue, Oct 18, 2016 at 9:38 AM, SF Markus Elfring
<elfring@users.sourceforge.net> wrote:
>>> elfring@Sonne:~/Projekte/Linux/next-patched> my_off=-O0 && for X in off; do my_output_dir=${my_build_dir}unchanged/optimisation/${X} && my_var="my_${X}" && my_parameters="-Wall -std=gnu89 -fomit-frame-pointer ${!my_var}" && LANG=C make V=2 O="${my_output_dir}" HOSTCC='/usr/bin/gcc-6' HOSTCFLAGS="-S ${my_parameters}" drivers/md/; done
> …
>> That's a very complicated command line.
>
> I find it "advanced".
>
> It shows a few of my imaginations around a specific software development task.
>
>
>> Does something basic, like
>>
>> make O=/tmp/$$ allmodconfig
>> make O=/tmp/$$
>>
>> work for you?
>
> Not really. - I would like to achieve the automatic selection of a few
> compilation settings by a for loop which I reduced in the shown example
> to one element.

Fine, but my point was to check that the basic things worked before
trying the complicated, or "advanced" stuff.  If, somehow, fixdep
didn't work in a basic setup then you'd know it wasn't one of your
advanced features that was causing the problem.

And if fixdep did work with a basic setup , then that would suggest
something wrong with your advanced command line.
-- 
Jim

Re: Difficulties around "fixdep" for the usage of a kernel build output directory

[SF Markus Elfring]

> Fine, but my point was to check that the basic things worked before
> trying the complicated, or "advanced" stuff.

Your suggestion is fine in this sense. I can observe that the following command variant
works as expected here.

my_build_dir=/tmp/fixdep-check-20161018-1 && mkdir "${my_build_dir}" && my_cc=/usr/bin/gcc-6 && make O="${my_build_dir}" HOSTCC="${my_cc}" allmodconfig && make -j6 O="${my_build_dir}" HOSTCC="${my_cc}" drivers/md/


> If, somehow, fixdep didn't work in a basic setup then you'd know it wasn't one of your
> advanced features that was causing the problem.

The command example above is an approximation for the configuration approach
which I would really want instead.


> And if fixdep did work with a basic setup , then that would suggest
> something wrong with your advanced command line.

How do you think about to clarify further which implementation detail could trigger
the shown software build difficulty?

Can it be that the passing of the adjusted parameter "HOSTCFLAGS" has got a significant
influence (with unwanted side effects) in this use case?

Regards,
Markus

Re: Difficulties around "fixdep" for the usage of a kernel build output directory

[Jim Davis]

On Tue, Oct 18, 2016 at 10:25 AM, SF Markus Elfring
<elfring@users.sourceforge.net> wrote:

> Can it be that the passing of the adjusted parameter "HOSTCFLAGS" has got a significant
> influence (with unwanted side effects) in this use case?

That's probably it.  If I strip down your advanced original example to just

make defconfig
make HOSTCFLAGS=-S

then I get a similar error:

  HOSTCC  scripts/basic/fixdep
scripts/basic/fixdep: 1: scripts/basic/fixdep: .file: not found
scripts/basic/fixdep: 2: scripts/basic/fixdep: .comm: not found
scripts/basic/fixdep: 3: scripts/basic/fixdep: .comm: not found
scripts/basic/fixdep: 4: scripts/basic/fixdep: .comm: not found
scripts/basic/fixdep: 5: scripts/basic/fixdep: .comm: not found
scripts/basic/fixdep: 6: scripts/basic/fixdep: .section: not found

and so forth.  The problem is that, with -S, fixdep isn't build as an
executable:

jim@krebstar:~/linux-next/scripts/basic$ file fixdep
fixdep: assembler source, ASCII text
jim@krebstar:~/linux-next/scripts/basic$ more fixdep
    .file    "fixdep.c"
    .comm    insert_extra_deps,4,4
    .comm    target,8,8
    .comm    depfile,8,8
    .comm    cmdline,8,8
    .section    .rodata

I'm guessing that in your original example you wanted to look at the
assembly output gcc produced; you'd probably have better luck using
objdump for that.

-- 
Jim

Re: Challenges around the usage of different "HOSTCFLAGS" for kernel build configuration and module generation

[SF Markus Elfring]

>> Can it be that the passing of the adjusted parameter "HOSTCFLAGS" has got a significant
>> influence (with unwanted side effects) in this use case?
> 
> That's probably it.  If I strip down your advanced original example to just
> 
> make defconfig
> make HOSTCFLAGS=-S
> 
> then I get a similar error:
> 
>   HOSTCC  scripts/basic/fixdep
> scripts/basic/fixdep: 1: scripts/basic/fixdep: .file: not found
…
> and so forth.

Thanks that you could reproduce my observation also with a simple example
on your test system.


> The problem is that, with -S, fixdep isn't build as an executable:
> 
> jim@krebstar:~/linux-next/scripts/basic$ file fixdep
> fixdep: assembler source, ASCII text

Are you interested that a software generation parameter like "-S"
(for output of assembler source files) could be directly supported
for a special build variant?

* Does this issue indicate further development challenges?

* How are the chances that the involved software dependencies can be
  handled better in more detail?


> I'm guessing that in your original example you wanted to look at the
> assembly output gcc produced;

Yes. - I became interested in this use case once more.


> you'd probably have better luck using objdump for that.

This approach can occasionally work to some degree. I am looking for
a better (or more convenient) solution.

I got the impression that I need to put special build parameters into
the make files directly so far while I would prefer to pass extra settings
as command line arguments for the tool "make".

Will it make sense to distinguish such parameters for development phases
(or stages) like "kernel build configuration" and corresponding
"module generation" any more?

Regards,
Markus

Re: Challenges around the usage of different "HOSTCFLAGS" for kernel build configuration and module generation

[Jim Davis]

On Wed, Oct 19, 2016 at 12:25 AM, SF Markus Elfring
<elfring@users.sourceforge.net> wrote:
>
> Are you interested that a software generation parameter like "-S"
> (for output of assembler source files) could be directly supported
> for a special build variant?

No, if you want to see the assembly code then something like

make defconfig; make EXTRA_CFLAGS="-g" drivers/md/; objdump -S
drivers/md/whatever.o

seems easy enough.  You could write a little shell wrapper to save
some keystrokes if you find you're running those commands frequently.

-- 
Jim

Re: Challenges around the usage of different "HOSTCFLAGS" for kernel build configuration and module generation

[Segher Boessenkool]

On Wed, Oct 19, 2016 at 12:51:44PM -0700, Jim Davis wrote:
> On Wed, Oct 19, 2016 at 12:25 AM, SF Markus Elfring
> <elfring@users.sourceforge.net> wrote:
> >
> > Are you interested that a software generation parameter like "-S"
> > (for output of assembler source files) could be directly supported
> > for a special build variant?
> 
> No, if you want to see the assembly code then something like
> 
> make defconfig; make EXTRA_CFLAGS="-g" drivers/md/; objdump -S
> drivers/md/whatever.o
> 
> seems easy enough.  You could write a little shell wrapper to save
> some keystrokes if you find you're running those commands frequently.

Don't forget -r when running objdump on an object file.  But, you
want to use  make drivers/md/whatever.s  if you want to see the
assembler code.

And that doesn't work for build-time tools, which I think is what
the original discussion was about.


Segher

Re: Challenges around the usage of different "HOSTCFLAGS" for kernel build configuration and module generation

[SF Markus Elfring]

> Don't forget -r when running objdump on an object file.

Thanks for your reminder.


> But, you want to use  make drivers/md/whatever.s  if you want to see
> the assembler code.

Thanks for your advice!

This is one kind of "generation convenience" I was looking for.

Was my software development attention too limited for another moment
so that I did not directly pick an opportunity up from the description
like "dir/file.[ois]  - Build specified target only"
(by the command "make help") that I can get assembler source files
for Linux modules so easy on demand?

Would you like to point any places out in the make scripts which
show relevant commands for such build targets?


> And that doesn't work for build-time tools,

I got the impression that they can trigger further software development challenges.
Should their source files (and corresponding make parameters) be improved anyhow?


> which I think is what the original discussion was about.

I see a few change possibilities there. Will it be useful to reconsider
the generation parameters for build-time tools in comparison to components
from other Linux software areas?

Will a variable like "EXTRA_CFLAGS" achieve a desired effect at more places?

Regards,
Markus

Re: Challenges around the usage of different "HOSTCFLAGS" for kernel build configuration and module generation

[SF Markus Elfring]

> make defconfig; make EXTRA_CFLAGS="-g" drivers/md/; objdump -S drivers/md/whatever.o

Such a command looks promising.

* I find it safer when double ampersand will be used as control operator
  instead of semicolon there.

* I get the impression from an other command like "find . -name '?akefile' -exec ack --with-filename EXTRA_CFLAGS '{}' \;"
  that the suggested parameter "EXTRA_CFLAGS" will not achieve an effect
  at places where I am interested in at the moment.


It seems that further fine-tuning can become interesting and challenging
for involved make variables.

Regards,
Markus