Kbuild: how to cleanly retrieve information compilation about the last build

Kbuild: how to cleanly retrieve information compilation about the last build

[Francis Moreau]

Hello,

I'm writing a script to automatise some parts of my kernel compilation process.

>From those scripts I'd like to be able to call the top makefile the
same way it had been called during its last invocation.

For example, if Ido:

     $ make CC=my-gcc CFLAGS="-g -fwhatever"

I would like to retrieve the "CC=my-gcc CFLAGS="-g -fwhatever" part of
the last invocation so my script can call make with the same
arguments.

Is this possible ?

Thanks for any ideas
-- 
Francis

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Francis Moreau]

Hello Sam,

Maybe could suggest something, it would be great.

Thanks

On Thu, Apr 14, 2011 at 9:13 AM, Francis Moreau <francis.moro@gmail.com> wrote:
> Hello,
>
> I'm writing a script to automatise some parts of my kernel compilation process.
>
> From those scripts I'd like to be able to call the top makefile the
> same way it had been called during its last invocation.
>
> For example, if Ido:
>
>     $ make CC=my-gcc CFLAGS="-g -fwhatever"
>
> I would like to retrieve the "CC=my-gcc CFLAGS="-g -fwhatever" part of
> the last invocation so my script can call make with the same
> arguments.
>
> Is this possible ?
>
> Thanks for any ideas
> --
> Francis
>



-- 
Francis

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Sam Ravnborg]

On Sat, Apr 16, 2011 at 10:05:43AM +0200, Francis Moreau wrote:
> Hello Sam,
> 
> Maybe could suggest something, it would be great.
> 
> Thanks
> 
> On Thu, Apr 14, 2011 at 9:13 AM, Francis Moreau <francis.moro@gmail.com> wrote:
> > Hello,
> >
> > I'm writing a script to automatise some parts of my kernel compilation process.
> >
> > From those scripts I'd like to be able to call the top makefile the
> > same way it had been called during its last invocation.
> >
> > For example, if Ido:
> >
> >     $ make CC=my-gcc CFLAGS="-g -fwhatever"
> >
> > I would like to retrieve the "CC=my-gcc CFLAGS="-g -fwhatever" part of
> > the last invocation so my script can call make with the same
> > arguments.
> >
> > Is this possible ?

There is nothing made in kbuild to preserve the value of randomly
added variable assignments on the command-line.

If you specify O=... then a Makfile file is generated in the output
directory that thus emulate the O= setting.

CCFLAGS has btw. no effect when you build a kernel.

If you on a regular basis need to pass flags on the command-line
then you likely are doing something odd as this is not the typical use.
So please reconsider what you are doing.

And you can as pointed out by Américo Wang always save the
command line in your calling script.

	Sam

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Francis Moreau]

Hello,

On Sat, Apr 16, 2011 at 3:59 PM, Sam Ravnborg <sam@ravnborg.org> wrote:
> On Sat, Apr 16, 2011 at 10:05:43AM +0200, Francis Moreau wrote:
>> Hello Sam,
>>
>> Maybe could suggest something, it would be great.
>>
>> Thanks
>>
>> On Thu, Apr 14, 2011 at 9:13 AM, Francis Moreau <francis.moro@gmail.com> wrote:
>> > Hello,
>> >
>> > I'm writing a script to automatise some parts of my kernel compilation process.
>> >
>> > From those scripts I'd like to be able to call the top makefile the
>> > same way it had been called during its last invocation.
>> >
>> > For example, if Ido:
>> >
>> >     $ make CC=my-gcc CFLAGS="-g -fwhatever"
>> >
>> > I would like to retrieve the "CC=my-gcc CFLAGS="-g -fwhatever" part of
>> > the last invocation so my script can call make with the same
>> > arguments.
>> >
>> > Is this possible ?
>
> There is nothing made in kbuild to preserve the value of randomly
> added variable assignments on the command-line.
>
> If you specify O=... then a Makfile file is generated in the output
> directory that thus emulate the O= setting.
>
> CCFLAGS has btw. no effect when you build a kernel.

Ok CCFLAGS was a poor example.

BTW are the allowed flags documented somewhere ?

> If you on a regular basis need to pass flags on the command-line
> then you likely are doing something odd as this is not the typical use.
> So please reconsider what you are doing.

Ok, if I'm doing something wrong, I'd like to be corrected.

What's wrong with passing those flags for example:

  $ make ARCH=arm CROSS_COMPILE=arm-linux

or

   $ make CC=distcc

?

> And you can as pointed out by Américo Wang always save the
> command line in your calling script.

No, because the makefile invocation is not always done by my script.

For example a user can do:

  $ make ARCH=arm CROSS_COMPILE=arm-linux-

Then call my script and expect it to pass the same flags to make.

Thanks
-- 
Francis

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 537 bytes --]

On Sat, 16 Apr 2011 16:04:57 +0200, Francis Moreau said:

> For example a user can do:
> 
>   $ make ARCH=arm CROSS_COMPILE=arm-linux-
> 
> Then call my script and expect it to pass the same flags to make.

Odd, don't ARCH and CROSS_COMPILE get saved in the .config?

What *are* you trying to pass to kbuild that isn't either saved in the .config
or deduced at build time (the cross-compile gets saved, stuff like "where is
python?" is deduced).  Is there a *real* problem you're hitting here, or is
it merely theoretical?

[-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --]

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Francis Moreau]

Hello,

On Sat, Apr 16, 2011 at 4:33 PM,  <Valdis.Kletnieks@vt.edu> wrote:
> On Sat, 16 Apr 2011 16:04:57 +0200, Francis Moreau said:
>
>> For example a user can do:
>>
>>   $ make ARCH=arm CROSS_COMPILE=arm-linux-
>>
>> Then call my script and expect it to pass the same flags to make.
>
> Odd, don't ARCH and CROSS_COMPILE get saved in the .config?
>

I could but not always since it can be passed to the command line.

> What *are* you trying to pass to kbuild that isn't either saved in the .config
> or deduced at build time (the cross-compile gets saved, stuff like "where is
> python?" is deduced).  Is there a *real* problem you're hitting here, or is
> it merely theoretical?

Again, I'd like to know how the makefile has been called the last time it run.

Some flags can be passed and influence the rebuild process and I don't
want my script to rebuild the whole kernel because my script doesn't
invoke the makefile as the user (command line) did.

For example, user did:

  $ make CC=distcc

then call my script:

  $ my-script

which in its turn does:

  $ make

then the whole kernel is rebuilt..

Thanks
-- 
Francis

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1045 bytes --]

On Sat, 16 Apr 2011 16:45:33 +0200, Francis Moreau said:

> For example, user did:
>   $ make CC=distcc
> then call my script:
>   $ my-script
> which in its turn does:
>   $ make
> then the whole kernel is rebuilt..

You have two choices then:

1)  Allow them to pass stuff to your script:

$ make CC=distcc
then call my script:
$ my-script CC=distcc
which in its turn does:
$ make "$*"

2) Find out *why* they're doing a make of the kernel, and then calling your
script that *again* does a make of the kernel, instead of just calling your
script and being done with it.  This sounds like you have a poorly designed
build environment, and *that* needs fixing instead of kbuild.  There's really
an upper limit to how much kbuild is able to help a 3rd-party script writer who
has users who can't follow directions. There's just too many ways they can do
things that will cause a rebuild - they can do a make, then run a 'make
menuconfig' and change some important setting, then call your script and wham
you end up rebuilding the kernel anyhow.


[-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --]

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Francis Moreau]

On Sat, Apr 16, 2011 at 5:08 PM,  <Valdis.Kletnieks@vt.edu> wrote:
> On Sat, 16 Apr 2011 16:45:33 +0200, Francis Moreau said:
>
>> For example, user did:
>>   $ make CC=distcc
>> then call my script:
>>   $ my-script
>> which in its turn does:
>>   $ make
>> then the whole kernel is rebuilt..
>
> You have two choices then:
>
> 1)  Allow them to pass stuff to your script:
>
> $ make CC=distcc
> then call my script:
> $ my-script CC=distcc
> which in its turn does:
> $ make "$*"
>
> 2) Find out *why* they're doing a make of the kernel, and then calling your
> script that *again* does a make of the kernel, instead of just calling your
> script and being done with it.

Because the script needs some generated files and rather to ask to the user:

  file X is missing, please run 'make prepare'

I just thought that it would be easier to let the script call make
prepare (this is just an example) automatically.

And from the script point of view, it's just easier to call make
instead of testing for all missing files.

But maybe it's just a bad idea.

> This sounds like you have a poorly designed
> build environment, and *that* needs fixing instead of kbuild.

Well I've never claimed that kbuild needs to be fixed.
-- 
Francis

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Francis Moreau]

On Sat, Apr 16, 2011 at 5:47 PM, Francis Moreau <francis.moro@gmail.com> wrote:
> On Sat, Apr 16, 2011 at 5:08 PM,  <Valdis.Kletnieks@vt.edu> wrote:
>> On Sat, 16 Apr 2011 16:45:33 +0200, Francis Moreau said:
>>
>>> For example, user did:
>>>   $ make CC=distcc
>>> then call my script:
>>>   $ my-script
>>> which in its turn does:
>>>   $ make
>>> then the whole kernel is rebuilt..
>>
>> You have two choices then:
>>
>> 1)  Allow them to pass stuff to your script:
>>
>> $ make CC=distcc
>> then call my script:
>> $ my-script CC=distcc
>> which in its turn does:
>> $ make "$*"
>>
>> 2) Find out *why* they're doing a make of the kernel, and then calling your
>> script that *again* does a make of the kernel, instead of just calling your
>> script and being done with it.
>
> Because the script needs some generated files and rather to ask to the user:
>
>  file X is missing, please run 'make prepare'
>
> I just thought that it would be easier to let the script call make
> prepare (this is just an example) automatically.

Actually think about this example:

The user modify a file, then instead of calling 'make' (because he
forgets) run the script that send the kernel image through the net on
a test machine. Since the user had already compiled the kernel before,
 the kernel image exists but is outdated.

How can a script detect this case if it doesn't call 'make' in its turn ?

-- 
Francis

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1020 bytes --]

On Sat, 16 Apr 2011 17:57:23 +0200, Francis Moreau said:

> The user modify a file, then instead of calling 'make' (because he
> forgets) run the script that send the kernel image through the net on
> a test machine. Since the user had already compiled the kernel before,
>  the kernel image exists but is outdated.

As I said, this is an example of a broken development environment, or
possibly a broken developer. :)

> How can a script detect this case if it doesn't call 'make' in its turn ?

Why do you care about detecting it without calling make?  Just go ahead and
*do* it, if the kernel is up to date it won't take long.  With a completely
cold cache, it takes about 90 seconds on my laptop.  Cache-hot is closer to 45
seconds. If that's too long, buy the developer a real machine or a compile
farm.  If that's a problem, you'll need to put the developer inside a script
that enforces "you *vill* do dis, then you *vill* do dat" fascism so the
programmer never gets a chance to do something you didn't expect.


[-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --]

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Francis Moreau]

On Sat, Apr 16, 2011 at 9:46 PM,  <Valdis.Kletnieks@vt.edu> wrote:
> On Sat, 16 Apr 2011 17:57:23 +0200, Francis Moreau said:
>
>> The user modify a file, then instead of calling 'make' (because he
>> forgets) run the script that send the kernel image through the net on
>> a test machine. Since the user had already compiled the kernel before,
>>  the kernel image exists but is outdated.
>
> As I said, this is an example of a broken development environment, or
> possibly a broken developer. :)
>

ok I think you can just ignore that thread now, this and what you said
below is just hmm, not revelant for me and for what I asked for.

>> How can a script detect this case if it doesn't call 'make' in its turn ?
>
> Why do you care about detecting it without calling make?  Just go ahead and
> *do* it, if the kernel is up to date it won't take long.  With a completely
> cold cache, it takes about 90 seconds on my laptop.

Really, what kind of laptop is it ?

Are you compiling the kernel with just allnoconfig ?

Seriously, I now think that what you say is totaly irrevelant, please
just ignore my question.
-- 
Francis

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Sam Ravnborg]

> 
> BTW are the allowed flags documented somewhere ?
Some/most supported flags are documented in Dumentation/kbuild/kbuild.txt

> > If you on a regular basis need to pass flags on the command-line
> > then you likely are doing something odd as this is not the typical use.
> > So please reconsider what you are doing.
> 
> Ok, if I'm doing something wrong, I'd like to be corrected.
> 
> What's wrong with passing those flags for example:
> 
>   $ make ARCH=arm CROSS_COMPILE=arm-linux
> 
> or
> 
>    $ make CC=distcc
> 
> ?
Both examples are perfectly OK. From your previous posting it
looked like you passed flags that influenced how the kernel
was built (CCFLAGS) and it did not look like you had cross-compile
nor distcc in mind - hence my comment.

	Sam

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Américo Wang]

On Thu, Apr 14, 2011 at 3:13 PM, Francis Moreau <francis.moro@gmail.com> wrote:
> Hello,
>
> I'm writing a script to automatise some parts of my kernel compilation process.
>
> From those scripts I'd like to be able to call the top makefile the
> same way it had been called during its last invocation.
>
> For example, if Ido:
>
>     $ make CC=my-gcc CFLAGS="-g -fwhatever"
>
> I would like to retrieve the "CC=my-gcc CFLAGS="-g -fwhatever" part of
> the last invocation so my script can call make with the same
> arguments.
>

So why not just put that line into your script?

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Francis Moreau]

On Sat, Apr 16, 2011 at 10:26 AM, Américo Wang <xiyou.wangcong@gmail.com> wrote:
> On Thu, Apr 14, 2011 at 3:13 PM, Francis Moreau <francis.moro@gmail.com> wrote:
>> Hello,
>>
>> I'm writing a script to automatise some parts of my kernel compilation process.
>>
>> From those scripts I'd like to be able to call the top makefile the
>> same way it had been called during its last invocation.
>>
>> For example, if Ido:
>>
>>     $ make CC=my-gcc CFLAGS="-g -fwhatever"
>>
>> I would like to retrieve the "CC=my-gcc CFLAGS="-g -fwhatever" part of
>> the last invocation so my script can call make with the same
>> arguments.
>>
>
> So why not just put that line into your script?
>

Because this line was an _example_ of how the makefile could had been invoked.

But the script has currently no idea how the previous invocation was
made, hence my question.

Thanks
-- 
Francis

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Américo Wang]

On Sat, Apr 16, 2011 at 10:00 PM, Francis Moreau <francis.moro@gmail.com> wrote:
> On Sat, Apr 16, 2011 at 10:26 AM, Américo Wang <xiyou.wangcong@gmail.com> wrote:
>> So why not just put that line into your script?
>>
>
> Because this line was an _example_ of how the makefile could had been invoked.
>
> But the script has currently no idea how the previous invocation was
> made, hence my question.

You use a bad design, why not just pass these parameters to your script?
Something like,

$ ./your_script CC=my-gcc CFLAGS="-g -fwhatever"

or whatever you want.

Also, if you really have to read the previous command (I doubt),
you can use the `history` command.

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Francis Moreau]

On Sun, Apr 17, 2011 at 6:57 AM, Américo Wang <xiyou.wangcong@gmail.com> wrote:
> On Sat, Apr 16, 2011 at 10:00 PM, Francis Moreau <francis.moro@gmail.com> wrote:
>> On Sat, Apr 16, 2011 at 10:26 AM, Américo Wang <xiyou.wangcong@gmail.com> wrote:
>>> So why not just put that line into your script?
>>>
>>
>> Because this line was an _example_ of how the makefile could had been invoked.
>>
>> But the script has currently no idea how the previous invocation was
>> made, hence my question.
>
> You use a bad design, why not just pass these parameters to your script?
> Something like,
>
> $ ./your_script CC=my-gcc CFLAGS="-g -fwhatever"
>
> or whatever you want.
>

Yes, I think I'll continue to do that.

Thanks
-- 
Francis

Re: Kbuild: how to cleanly retrieve information compilation about the last build

[Jonathan Neuschäfer]

On Sun, Apr 17, 2011 at 12:27:29PM +0200, Francis Moreau wrote:
> On Sun, Apr 17, 2011 at 6:57 AM, Américo Wang <xiyou.wangcong@gmail.com> wrote:
> > On Sat, Apr 16, 2011 at 10:00 PM, Francis Moreau <francis.moro@gmail.com> wrote:
> >> On Sat, Apr 16, 2011 at 10:26 AM, Américo Wang <xiyou.wangcong@gmail.com> wrote:
> >>> So why not just put that line into your script?
> >>>
> >>
> >> Because this line was an _example_ of how the makefile could had been invoked.
> >>
> >> But the script has currently no idea how the previous invocation was
> >> made, hence my question.
> >
> > You use a bad design, why not just pass these parameters to your script?
> > Something like,
> >
> > $ ./your_script CC=my-gcc CFLAGS="-g -fwhatever"
> >
> > or whatever you want.
> >
>
> Yes, I think I'll continue to do that.
>
> Thanks

You may also try this little script (call it instead of plain "make"):

	#!/bin/sh

	MAKE=make
	CMDLINE=make.cmdline

	if [ "x$@" != "x" ]; then
		echo -n "$@" > $CMDLINE
		$MAKE "$@"
	else
		if [ -f $CMDLINE ]; then
			$MAKE $(cat $CMDLINE)
		else
			$MAKE
		fi
	fi

thanks,
	Jonathan Neuschäfer