Re: [PATCH v2] kheaders: making headers archive reproducible

[Joel Fernandes <joel@joelfernandes.org>]

On Fri, Oct 04, 2019 at 10:40:07AM +0000, Dmitry Goldin wrote:
> From: Dmitry Goldin <dgoldin+lkml@protonmail.ch>
> 
> In commit 43d8ce9d65a5 ("Provide in-kernel headers to make
> extending kernel easier") a new mechanism was introduced, for kernels
> >=5.2, which embeds the kernel headers in the kernel image or a module
> and exposes them in procfs for use by userland tools.
> 
> The archive containing the header files has nondeterminism caused by
> header files metadata. This patch normalizes the metadata and utilizes
> KBUILD_BUILD_TIMESTAMP if provided and otherwise falls back to the
> default behaviour.
> 
> In commit f7b101d33046 ("kheaders: Move from proc to sysfs") it was
> modified to use sysfs and the script for generation of the archive was
> renamed to what is being patched.
> 
> Signed-off-by: Dmitry Goldin <dgoldin+lkml@protonmail.ch>

Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>

> 
> ---
> 
> v1: Initial fix
> 
> v2: Added a bit of info about kheaders to the reproducible builds
> documentation and used the opportunity to fix a few typos in the
> original patch.
> 
> ---
>  Documentation/kbuild/reproducible-builds.rst | 13 +++++++++----
>  kernel/gen_kheaders.sh                       |  5 ++++-
>  2 files changed, 13 insertions(+), 5 deletions(-)
> 
> diff --git a/Documentation/kbuild/reproducible-builds.rst b/Documentation/kbuild/reproducible-builds.rst
> index ab92e98c89c8..ce6a408b3303 100644
> --- a/Documentation/kbuild/reproducible-builds.rst
> +++ b/Documentation/kbuild/reproducible-builds.rst
> @@ -16,16 +16,21 @@ the kernel may be unreproducible, and how to avoid them.
>  Timestamps
>  ----------
> 
> -The kernel embeds a timestamp in two places:
> +The kernel embeds timestamps in three places:
> 
>  * The version string exposed by ``uname()`` and included in
>    ``/proc/version``
> 
>  * File timestamps in the embedded initramfs
> 
> -By default the timestamp is the current time.  This must be overridden
> -using the `KBUILD_BUILD_TIMESTAMP`_ variable.  If you are building
> -from a git commit, you could use its commit date.
> +* If enabled via ``CONFIG_IKHEADERS``, file timestamps of kernel
> +  headers embedded in the kernel or respective module,
> +  exposed via ``/sys/kernel/kheaders.tar.xz``
> +
> +By default the timestamp is the current time and in the case of
> +``kheaders`` the various files' modification times. This must
> +be overridden using the `KBUILD_BUILD_TIMESTAMP`_ variable.
> +If you are building from a git commit, you could use its commit date.
> 
>  The kernel does *not* use the ``__DATE__`` and ``__TIME__`` macros,
>  and enables warnings if they are used.  If you incorporate external
> diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
> index 9ff449888d9c..aff79e461fc9 100755
> --- a/kernel/gen_kheaders.sh
> +++ b/kernel/gen_kheaders.sh
> @@ -71,7 +71,10 @@ done | cpio --quiet -pd $cpio_dir >/dev/null 2>&1
>  find $cpio_dir -type f -print0 |
>  	xargs -0 -P8 -n1 perl -pi -e 'BEGIN {undef $/;}; s/\/\*((?!SPDX).)*?\*\///smg;'
> 
> -tar -Jcf $tarfile -C $cpio_dir/ . > /dev/null
> +# Create archive and try to normalize metadata for reproducibility
> +tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
> +    --owner=0 --group=0 --sort=name --numeric-owner \
> +    -Jcf $tarfile -C $cpio_dir/ . > /dev/null
> 
>  echo "$src_files_md5" >  kernel/kheaders.md5
>  echo "$obj_files_md5" >> kernel/kheaders.md5
> --
> 2.23.0