[PATCH 1/3] handling of _shipped files

[PATCH 1/3] handling of _shipped files

[Nicholas Mc Guire]

handling of _shipped files

This adds a section on handling of _shipped files in kbuild
Makefiles and the build process.

Signed-off-by: Nicholas Mc Guire <der.herr@hofr.at>
---
 Documentation/kbuild/makefiles.txt |   57 +++++++++++++++++++++++++++++++++++-
 1 file changed, 56 insertions(+), 1 deletion(-)

diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
index 764f599..d0bd91a 100644
--- a/Documentation/kbuild/makefiles.txt
+++ b/Documentation/kbuild/makefiles.txt
@@ -19,6 +19,7 @@ This document describes the Linux kernel Makefiles.
 	   --- 3.10 Special Rules
 	   --- 3.11 $(CC) support functions
 	   --- 3.12 $(LD) support functions
+	   --- 3.13 _shipped - generated files
 
 	=== 4 Host Program support
 	   --- 4.1 Simple Host Program
@@ -595,6 +596,60 @@ more details, with real examples.
 		LDFLAGS_vmlinux += $(call ld-option, -X)
 
 
+--- 3.13 _shipped - generated files
+
+   Some sources in the kernel are generated by tools or special
+   purpose scrips so that the build process dependencies can be kept
+   at a resonable level. Also in cases where the translation is not
+   configuration dependent but might impact build time _shipped files
+   may be added along with the actual sources to speed up the build
+   process.
+
+   For standard tools like lex and yacc the make process will actually
+   generate the _shipped files (see scripts/Makefile.lib) and retain
+   them by marking them .PRECIOUS so they are not removed by the clean
+   targets.
+
+   For non standard tools the generation of the _shipped file it self
+   may be manually achived by building and then renaming. A makefile
+   target for generate the actual target file should to be provided
+   though and can be conditioned on an environment variable.
+
+	Example:
+		ifdef GENERATE_KEYMAP
+
+		$(obj)/defkeymap.c: $(obj)/%.c: $(src)/%.map
+			loadkeys --mktable $< > $@
+
+		endif
+
+   The _shipped file does not appear in the actual Makefile, this is
+   handled by the rules found in scripts/Makefile.lib.
+
+		<snip>
+		cmd_shipped = cat $< > $@
+
+		$(obj)/%: $(src)/%_shipped
+			$(call cmd,shipped)
+		<snip>
+
+   which simply will copy the target_shipped to the target
+
+   If a file of a make target is found in the build directory with the
+   extension _shipped make will used it, rather than generating the
+   respective target file from source.
+
+   The build will indicate the use of the _shipped file by the SHIPPED
+   quiet command in the make output.
+
+	Example:
+		LD      drivers/scsi/aic7xxx/built-in.o
+		SHIPPED drivers/scsi/aic7xxx/aic79xx_seq.h
+		SHIPPED drivers/scsi/aic7xxx/aic79xx_reg.h
+		CC [M]  drivers/scsi/aic7xxx/aic79xx_core.o
+		CC [M]  drivers/scsi/aic7xxx/aic79xx_pci.o
+		SHIPPED drivers/scsi/aic7xxx/aic79xx_reg_print.c
+		CC [M]  drivers/scsi/aic7xxx/aic79xx_reg_print.o
 === 4 Host Program support
 
 Kbuild supports building executables on the host for use during the
@@ -1423,10 +1478,10 @@ Original version made by Michael Elizabeth Chastain, <mailto:mec@shout.net>
 Updates by Kai Germaschewski <kai@tp1.ruhr-uni-bochum.de>
 Updates by Sam Ravnborg <sam@ravnborg.org>
 Language QA by Jan Engelhardt <jengelh@gmx.de>
+Kbuild support for shipped files Nicholas Mc Guire <der.herr@hofr.at>
 
 === 11 TODO
 
-- Describe how kbuild supports shipped files with _shipped.
 - Generating offset header files.
 - Add more variables to section 7?
 
-- 
1.7.10.4

Re: [PATCH 1/3] handling of _shipped files

[Randy Dunlap]

On 10/01/14 06:36, Nicholas Mc Guire wrote:
> 
> handling of _shipped files
> 
> This adds a section on handling of _shipped files in kbuild
> Makefiles and the build process.
> 
> Signed-off-by: Nicholas Mc Guire <der.herr@hofr.at>
> ---
>  Documentation/kbuild/makefiles.txt |   57 +++++++++++++++++++++++++++++++++++-
>  1 file changed, 56 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
> index 764f599..d0bd91a 100644
> --- a/Documentation/kbuild/makefiles.txt
> +++ b/Documentation/kbuild/makefiles.txt
> @@ -19,6 +19,7 @@ This document describes the Linux kernel Makefiles.
>  	   --- 3.10 Special Rules
>  	   --- 3.11 $(CC) support functions
>  	   --- 3.12 $(LD) support functions
> +	   --- 3.13 _shipped - generated files
>  
>  	=== 4 Host Program support
>  	   --- 4.1 Simple Host Program
> @@ -595,6 +596,60 @@ more details, with real examples.
>  		LDFLAGS_vmlinux += $(call ld-option, -X)
>  
>  
> +--- 3.13 _shipped - generated files
> +
> +   Some sources in the kernel are generated by tools or special
> +   purpose scrips so that the build process dependencies can be kept

              scripts

> +   at a resonable level. Also in cases where the translation is not

           reasonable
 
> +   configuration dependent but might impact build time _shipped files

                                               build time,

> +   may be added along with the actual sources to speed up the build
> +   process.
> +
> +   For standard tools like lex and yacc the make process will actually
> +   generate the _shipped files (see scripts/Makefile.lib) and retain
> +   them by marking them .PRECIOUS so they are not removed by the clean
> +   targets.
> +
> +   For non standard tools the generation of the _shipped file it self

          non-standard                                           itself

> +   may be manually achived by building and then renaming. A makefile

                      achieved

> +   target for generate the actual target file should to be provided

             to generate

> +   though and can be conditioned on an environment variable.

Most examples that I see just expect the Makefile to be modified to set
(or define) the variable.

> +
> +	Example:
> +		ifdef GENERATE_KEYMAP
> +
> +		$(obj)/defkeymap.c: $(obj)/%.c: $(src)/%.map
> +			loadkeys --mktable $< > $@
> +
> +		endif
> +
> +   The _shipped file does not appear in the actual Makefile, this is
> +   handled by the rules found in scripts/Makefile.lib.
> +
> +		<snip>
> +		cmd_shipped = cat $< > $@
> +
> +		$(obj)/%: $(src)/%_shipped
> +			$(call cmd,shipped)
> +		<snip>
> +
> +   which simply will copy the target_shipped to the target
> +
> +   If a file of a make target is found in the build directory with the
> +   extension _shipped make will used it, rather than generating the

                _shipped, make will use it rather than generating the

> +   respective target file from source.
> +
> +   The build will indicate the use of the _shipped file by the SHIPPED
> +   quiet command in the make output.
> +
> +	Example:
> +		LD      drivers/scsi/aic7xxx/built-in.o
> +		SHIPPED drivers/scsi/aic7xxx/aic79xx_seq.h
> +		SHIPPED drivers/scsi/aic7xxx/aic79xx_reg.h
> +		CC [M]  drivers/scsi/aic7xxx/aic79xx_core.o
> +		CC [M]  drivers/scsi/aic7xxx/aic79xx_pci.o
> +		SHIPPED drivers/scsi/aic7xxx/aic79xx_reg_print.c
> +		CC [M]  drivers/scsi/aic7xxx/aic79xx_reg_print.o
>  === 4 Host Program support
>  
>  Kbuild supports building executables on the host for use during the
> @@ -1423,10 +1478,10 @@ Original version made by Michael Elizabeth Chastain, <mailto:mec@shout.net>
>  Updates by Kai Germaschewski <kai@tp1.ruhr-uni-bochum.de>
>  Updates by Sam Ravnborg <sam@ravnborg.org>
>  Language QA by Jan Engelhardt <jengelh@gmx.de>
> +Kbuild support for shipped files Nicholas Mc Guire <der.herr@hofr.at>
>  
>  === 11 TODO
>  
> -- Describe how kbuild supports shipped files with _shipped.
>  - Generating offset header files.
>  - Add more variables to section 7?
>  
> 


-- 
~Randy

Re: [PATCH 1/3] handling of _shipped files

[Nicholas Mc Guire]

On Wed, 01 Oct 2014, Randy Dunlap wrote:

> On 10/01/14 06:36, Nicholas Mc Guire wrote:
> > 
> > handling of _shipped files
> > 
> > This adds a section on handling of _shipped files in kbuild
> > Makefiles and the build process.
> > 
> > Signed-off-by: Nicholas Mc Guire <der.herr@hofr.at>
> > ---
> >  Documentation/kbuild/makefiles.txt |   57 +++++++++++++++++++++++++++++++++++-
> >  1 file changed, 56 insertions(+), 1 deletion(-)
> > 
> > diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
> > index 764f599..d0bd91a 100644
> > --- a/Documentation/kbuild/makefiles.txt
> > +++ b/Documentation/kbuild/makefiles.txt
> > @@ -19,6 +19,7 @@ This document describes the Linux kernel Makefiles.
> >  	   --- 3.10 Special Rules
> >  	   --- 3.11 $(CC) support functions
> >  	   --- 3.12 $(LD) support functions
> > +	   --- 3.13 _shipped - generated files
> >  
> >  	=== 4 Host Program support
> >  	   --- 4.1 Simple Host Program
> > @@ -595,6 +596,60 @@ more details, with real examples.
> >  		LDFLAGS_vmlinux += $(call ld-option, -X)
> >  
> >  
> > +--- 3.13 _shipped - generated files
> > +
> > +   Some sources in the kernel are generated by tools or special
> > +   purpose scrips so that the build process dependencies can be kept
> 
>               scripts
> 
> > +   at a resonable level. Also in cases where the translation is not
> 
>            reasonable
>  
> > +   configuration dependent but might impact build time _shipped files
> 
>                                                build time,
> 
> > +   may be added along with the actual sources to speed up the build
> > +   process.
> > +
> > +   For standard tools like lex and yacc the make process will actually
> > +   generate the _shipped files (see scripts/Makefile.lib) and retain
> > +   them by marking them .PRECIOUS so they are not removed by the clean
> > +   targets.
> > +
> > +   For non standard tools the generation of the _shipped file it self
> 
>           non-standard                                           itself
> 
> > +   may be manually achived by building and then renaming. A makefile
> 
>                       achieved
> 
> > +   target for generate the actual target file should to be provided
> 
>              to generate
> 
> > +   though and can be conditioned on an environment variable.
> 
> Most examples that I see just expect the Makefile to be modified to set
> (or define) the variable.
>

Yes - but is that the prefered way of doing it ? I guess it
would be more clear to have a variable that is taken from the
environment and not via .config to trigger a conditional rebuild
as the defkeymap example does it.
 
> > +
> > +	Example:
> > +		ifdef GENERATE_KEYMAP
> > +
> > +		$(obj)/defkeymap.c: $(obj)/%.c: $(src)/%.map
> > +			loadkeys --mktable $< > $@
> > +
> > +		endif
> > +
> > +   The _shipped file does not appear in the actual Makefile, this is
> > +   handled by the rules found in scripts/Makefile.lib.
> > +
> > +		<snip>
> > +		cmd_shipped = cat $< > $@
> > +
> > +		$(obj)/%: $(src)/%_shipped
> > +			$(call cmd,shipped)
> > +		<snip>
> > +
> > +   which simply will copy the target_shipped to the target
> > +
> > +   If a file of a make target is found in the build directory with the
> > +   extension _shipped make will used it, rather than generating the
> 
>                 _shipped, make will use it rather than generating the
> 
> > +   respective target file from source.
> > +
> > +   The build will indicate the use of the _shipped file by the SHIPPED
> > +   quiet command in the make output.
> > +
> > +	Example:
> > +		LD      drivers/scsi/aic7xxx/built-in.o
> > +		SHIPPED drivers/scsi/aic7xxx/aic79xx_seq.h
> > +		SHIPPED drivers/scsi/aic7xxx/aic79xx_reg.h
> > +		CC [M]  drivers/scsi/aic7xxx/aic79xx_core.o
> > +		CC [M]  drivers/scsi/aic7xxx/aic79xx_pci.o
> > +		SHIPPED drivers/scsi/aic7xxx/aic79xx_reg_print.c
> > +		CC [M]  drivers/scsi/aic7xxx/aic79xx_reg_print.o
> >  === 4 Host Program support
> >  
> >  Kbuild supports building executables on the host for use during the
> > @@ -1423,10 +1478,10 @@ Original version made by Michael Elizabeth Chastain, <mailto:mec@shout.net>
> >  Updates by Kai Germaschewski <kai@tp1.ruhr-uni-bochum.de>
> >  Updates by Sam Ravnborg <sam@ravnborg.org>
> >  Language QA by Jan Engelhardt <jengelh@gmx.de>
> > +Kbuild support for shipped files Nicholas Mc Guire <der.herr@hofr.at>
> >  
> >  === 11 TODO
> >  
> > -- Describe how kbuild supports shipped files with _shipped.
> >  - Generating offset header files.
> >  - Add more variables to section 7?
> >  
> > 
>
and thanks for the language fixes - thought I had spellchecked it but 
obviously I did not 

thx!
hofrat