* [OE-core][PATCH] classes/kernel: Generate reproducible kernel module tarball
@ 2020-05-12 13:40 Joshua Watt
2020-05-12 14:22 ` Richard Purdie
2020-05-12 21:19 ` [OE-core][PATCH v2] " Joshua Watt
0 siblings, 2 replies; 5+ messages in thread
From: Joshua Watt @ 2020-05-12 13:40 UTC (permalink / raw)
To: openembedded-core; +Cc: Joshua Watt
If reproducible binaries are requested, generate the kernel modules
tarball in a reproducible way. Namely, sort the tarball contents, clamp
the mtime to the source date epoch, compress the archive with a
consistent compression level (9), and don't include the name and
timestamp in the compressed file.
[YOCTO #12719]
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
meta/classes/kernel.bbclass | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index a724645466..c124645643 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -691,7 +691,15 @@ kernel_do_deploy() {
if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
mkdir -p ${D}${root_prefix}/lib
- tar -cvzf $deployDir/modules-${MODULE_TARBALL_NAME}.tgz -C ${D}${root_prefix} lib
+ if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
+ TAR_ARGS="--sort=name --clamp-mtime --mtime=@${SOURCE_DATE_EPOCH}"
+ GZIP_ARGS="-9n"
+ else
+ TAR_ARGS=""
+ GZIP_ARGS=""
+ fi
+ tar $TAR_ARGS -cv -C ${D}${root_prefix} lib | gzip $GZIP_ARGS > $deployDir/modules-${MODULE_TARBALL_NAME}.tgz
+
ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz
fi
--
2.17.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [OE-core][PATCH] classes/kernel: Generate reproducible kernel module tarball
2020-05-12 13:40 [OE-core][PATCH] classes/kernel: Generate reproducible kernel module tarball Joshua Watt
@ 2020-05-12 14:22 ` Richard Purdie
2020-05-12 14:27 ` Joshua Watt
2020-05-12 21:19 ` [OE-core][PATCH v2] " Joshua Watt
1 sibling, 1 reply; 5+ messages in thread
From: Richard Purdie @ 2020-05-12 14:22 UTC (permalink / raw)
To: Joshua Watt, openembedded-core
On Tue, 2020-05-12 at 08:40 -0500, Joshua Watt wrote:
> If reproducible binaries are requested, generate the kernel modules
> tarball in a reproducible way. Namely, sort the tarball contents, clamp
> the mtime to the source date epoch, compress the archive with a
> consistent compression level (9), and don't include the name and
> timestamp in the compressed file.
>
> [YOCTO #12719]
>
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
> meta/classes/kernel.bbclass | 10 +++++++++-
> 1 file changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index a724645466..c124645643 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -691,7 +691,15 @@ kernel_do_deploy() {
>
> if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
> mkdir -p ${D}${root_prefix}/lib
> - tar -cvzf $deployDir/modules-${MODULE_TARBALL_NAME}.tgz -C ${D}${root_prefix} lib
> + if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
> + TAR_ARGS="--sort=name --clamp-mtime --mtime=@${SOURCE_DATE_EPOCH}"
> + GZIP_ARGS="-9n"
> + else
> + TAR_ARGS=""
> + GZIP_ARGS=""
> + fi
> + tar $TAR_ARGS -cv -C ${D}${root_prefix} lib | gzip $GZIP_ARGS > $deployDir/modules-${MODULE_TARBALL_NAME}.tgz
> +
> ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz
> fi
Do we need to make this conditional or could we just do it by default?
Cheers,
Richard
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [OE-core][PATCH] classes/kernel: Generate reproducible kernel module tarball
2020-05-12 14:22 ` Richard Purdie
@ 2020-05-12 14:27 ` Joshua Watt
2020-05-12 14:47 ` Richard Purdie
0 siblings, 1 reply; 5+ messages in thread
From: Joshua Watt @ 2020-05-12 14:27 UTC (permalink / raw)
To: Richard Purdie, openembedded-core
On 5/12/20 9:22 AM, Richard Purdie wrote:
> On Tue, 2020-05-12 at 08:40 -0500, Joshua Watt wrote:
>> If reproducible binaries are requested, generate the kernel modules
>> tarball in a reproducible way. Namely, sort the tarball contents, clamp
>> the mtime to the source date epoch, compress the archive with a
>> consistent compression level (9), and don't include the name and
>> timestamp in the compressed file.
>>
>> [YOCTO #12719]
>>
>> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
>> ---
>> meta/classes/kernel.bbclass | 10 +++++++++-
>> 1 file changed, 9 insertions(+), 1 deletion(-)
>>
>> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
>> index a724645466..c124645643 100644
>> --- a/meta/classes/kernel.bbclass
>> +++ b/meta/classes/kernel.bbclass
>> @@ -691,7 +691,15 @@ kernel_do_deploy() {
>>
>> if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
>> mkdir -p ${D}${root_prefix}/lib
>> - tar -cvzf $deployDir/modules-${MODULE_TARBALL_NAME}.tgz -C ${D}${root_prefix} lib
>> + if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
>> + TAR_ARGS="--sort=name --clamp-mtime --mtime=@${SOURCE_DATE_EPOCH}"
>> + GZIP_ARGS="-9n"
>> + else
>> + TAR_ARGS=""
>> + GZIP_ARGS=""
>> + fi
>> + tar $TAR_ARGS -cv -C ${D}${root_prefix} lib | gzip $GZIP_ARGS > $deployDir/modules-${MODULE_TARBALL_NAME}.tgz
>> +
>> ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz
>> fi
> Do we need to make this conditional or could we just do it by default?
I don't know if we can rely on SOURCE_DATE_EPOCH being set unless
reproducible_build.bbclass is inherited, but at a minimum we can always
use the gzip args.
>
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [OE-core][PATCH] classes/kernel: Generate reproducible kernel module tarball
2020-05-12 14:27 ` Joshua Watt
@ 2020-05-12 14:47 ` Richard Purdie
0 siblings, 0 replies; 5+ messages in thread
From: Richard Purdie @ 2020-05-12 14:47 UTC (permalink / raw)
To: Joshua Watt, openembedded-core
On Tue, 2020-05-12 at 09:27 -0500, Joshua Watt wrote:
> On 5/12/20 9:22 AM, Richard Purdie wrote:
> > On Tue, 2020-05-12 at 08:40 -0500, Joshua Watt wrote:
> > > If reproducible binaries are requested, generate the kernel modules
> > > tarball in a reproducible way. Namely, sort the tarball contents, clamp
> > > the mtime to the source date epoch, compress the archive with a
> > > consistent compression level (9), and don't include the name and
> > > timestamp in the compressed file.
> > >
> > > [YOCTO #12719]
> > >
> > > Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> > > ---
> > > meta/classes/kernel.bbclass | 10 +++++++++-
> > > 1 file changed, 9 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> > > index a724645466..c124645643 100644
> > > --- a/meta/classes/kernel.bbclass
> > > +++ b/meta/classes/kernel.bbclass
> > > @@ -691,7 +691,15 @@ kernel_do_deploy() {
> > >
> > > if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
> > > mkdir -p ${D}${root_prefix}/lib
> > > - tar -cvzf $deployDir/modules-${MODULE_TARBALL_NAME}.tgz -C ${D}${root_prefix} lib
> > > + if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
> > > + TAR_ARGS="--sort=name --clamp-mtime --mtime=@${SOURCE_DATE_EPOCH}"
> > > + GZIP_ARGS="-9n"
> > > + else
> > > + TAR_ARGS=""
> > > + GZIP_ARGS=""
> > > + fi
> > > + tar $TAR_ARGS -cv -C ${D}${root_prefix} lib | gzip $GZIP_ARGS > $deployDir/modules-${MODULE_TARBALL_NAME}.tgz
> > > +
> > > ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz
> > > fi
> > Do we need to make this conditional or could we just do it by default?
>
> I don't know if we can rely on SOURCE_DATE_EPOCH being set unless
> reproducible_build.bbclass is inherited, but at a minimum we can always
> use the gzip args.
Good point, I assumed we had a default for some reason.
I think the patch will be more obvious if its conditional on
SOURCE_DATE_EPOCH and we just use the other options unconditionally.
I wonder if we should just move build_reproducibility_simple.bbclass
into bitbake.conf?
Cheers,
Richard
^ permalink raw reply [flat|nested] 5+ messages in thread
* [OE-core][PATCH v2] classes/kernel: Generate reproducible kernel module tarball
2020-05-12 13:40 [OE-core][PATCH] classes/kernel: Generate reproducible kernel module tarball Joshua Watt
2020-05-12 14:22 ` Richard Purdie
@ 2020-05-12 21:19 ` Joshua Watt
1 sibling, 0 replies; 5+ messages in thread
From: Joshua Watt @ 2020-05-12 21:19 UTC (permalink / raw)
To: openembedded-core; +Cc: Joshua Watt
If reproducible binaries are requested, generate the kernel modules
tarball in a reproducible way. Namely, sort the tarball contents, clamp
the mtime to the source date epoch, compress the archive with a
consistent compression level (9), and don't include the name and
timestamp in the compressed file.
[YOCTO #12719]
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
meta/classes/kernel.bbclass | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index a724645466..6846d6040d 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -691,7 +691,13 @@ kernel_do_deploy() {
if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
mkdir -p ${D}${root_prefix}/lib
- tar -cvzf $deployDir/modules-${MODULE_TARBALL_NAME}.tgz -C ${D}${root_prefix} lib
+ if [ -n "${SOURCE_DATE_EPOCH}" ]; then
+ TAR_ARGS="--sort=name --clamp-mtime --mtime=@${SOURCE_DATE_EPOCH}"
+ else
+ TAR_ARGS=""
+ fi
+ tar $TAR_ARGS -cv -C ${D}${root_prefix} lib | gzip -9n > $deployDir/modules-${MODULE_TARBALL_NAME}.tgz
+
ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz
fi
--
2.17.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-05-12 21:19 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-05-12 13:40 [OE-core][PATCH] classes/kernel: Generate reproducible kernel module tarball Joshua Watt
2020-05-12 14:22 ` Richard Purdie
2020-05-12 14:27 ` Joshua Watt
2020-05-12 14:47 ` Richard Purdie
2020-05-12 21:19 ` [OE-core][PATCH v2] " Joshua Watt
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox