* [U-Boot] [PATCH] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH @ 2015-07-20 8:01 Paul Kocialkowski 2015-07-20 8:06 ` Paul Kocialkowski 2015-07-20 14:45 ` Vagrant Cascadian 0 siblings, 2 replies; 9+ messages in thread From: Paul Kocialkowski @ 2015-07-20 8:01 UTC (permalink / raw) To: u-boot In order to achieve reproducible builds in U-Boot, timestamps that are defined at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment variable allows setting a fixed value for those timestamps. Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be built reproducibly. This is the case for e.g. sunxi devices. However, some other devices might need some more tweaks, especially regarding the image generation tools. Signed-off-by: Paul Kocialkowski <contact@paulk.fr> --- Makefile | 7 ++++--- tools/default_image.c | 21 ++++++++++++++++++++- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/Makefile b/Makefile index 37cc4c3..71aeac7 100644 --- a/Makefile +++ b/Makefile @@ -1231,9 +1231,10 @@ define filechk_version.h endef define filechk_timestamp.h - (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \ - LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \ - LC_ALL=C date +'#define U_BOOT_TZ "%z"') + (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \ + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \ + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \ + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' ) endef $(version_h): include/config/uboot.release FORCE diff --git a/tools/default_image.c b/tools/default_image.c index cf5c0d4..18940af 100644 --- a/tools/default_image.c +++ b/tools/default_image.c @@ -88,6 +88,9 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, struct image_tool_params *params) { uint32_t checksum; + char *source_date_epoch; + struct tm *time_universal; + time_t time; image_header_t * hdr = (image_header_t *)ptr; @@ -96,9 +99,25 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, sizeof(image_header_t)), sbuf->st_size - sizeof(image_header_t)); + source_date_epoch = getenv("SOURCE_DATE_EPOCH"); + if (source_date_epoch != NULL) { + time = (time_t) strtol(source_date_epoch, NULL, 10); + + time_universal = gmtime(&time); + if (time_universal == NULL) { + fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n", + __func__); + time = 0; + } else { + time = mktime(time_universal); + } + } else { + time = sbuf->st_mtime; + } + /* Build new header */ image_set_magic(hdr, IH_MAGIC); - image_set_time(hdr, sbuf->st_mtime); + image_set_time(hdr, time); image_set_size(hdr, sbuf->st_size - sizeof(image_header_t)); image_set_load(hdr, params->addr); image_set_ep(hdr, params->ep); -- 1.9.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [U-Boot] [PATCH] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH 2015-07-20 8:01 [U-Boot] [PATCH] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH Paul Kocialkowski @ 2015-07-20 8:06 ` Paul Kocialkowski 2015-07-20 11:21 ` Heiko Schocher 2015-07-20 14:45 ` Vagrant Cascadian 1 sibling, 1 reply; 9+ messages in thread From: Paul Kocialkowski @ 2015-07-20 8:06 UTC (permalink / raw) To: u-boot Le lundi 20 juillet 2015 ? 10:01 +0200, Paul Kocialkowski a ?crit : > In order to achieve reproducible builds in U-Boot, timestamps that are defined > at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment > variable allows setting a fixed value for those timestamps. > > Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be > built reproducibly. This is the case for e.g. sunxi devices. > > However, some other devices might need some more tweaks, especially regarding > the image generation tools. Lunar, since you have contributed to this patch, feel free to add your Signed-Off-By line here before it's merged! Heiko, since this is based on your original patch, feel free to do the same! It would be nice to have this tested on as many boards as possible to spot other areas that make the binaries not reproducible. However, I doubt this patch will evolve much and other fixes should be sent in subsequent patches. > Signed-off-by: Paul Kocialkowski <contact@paulk.fr> > --- > Makefile | 7 ++++--- > tools/default_image.c | 21 ++++++++++++++++++++- > 2 files changed, 24 insertions(+), 4 deletions(-) > > diff --git a/Makefile b/Makefile > index 37cc4c3..71aeac7 100644 > --- a/Makefile > +++ b/Makefile > @@ -1231,9 +1231,10 @@ define filechk_version.h > endef > > define filechk_timestamp.h > - (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \ > - LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \ > - LC_ALL=C date +'#define U_BOOT_TZ "%z"') > + (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \ > + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \ > + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \ > + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' ) > endef > > $(version_h): include/config/uboot.release FORCE > diff --git a/tools/default_image.c b/tools/default_image.c > index cf5c0d4..18940af 100644 > --- a/tools/default_image.c > +++ b/tools/default_image.c > @@ -88,6 +88,9 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, > struct image_tool_params *params) > { > uint32_t checksum; > + char *source_date_epoch; > + struct tm *time_universal; > + time_t time; > > image_header_t * hdr = (image_header_t *)ptr; > > @@ -96,9 +99,25 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, > sizeof(image_header_t)), > sbuf->st_size - sizeof(image_header_t)); > > + source_date_epoch = getenv("SOURCE_DATE_EPOCH"); > + if (source_date_epoch != NULL) { > + time = (time_t) strtol(source_date_epoch, NULL, 10); > + > + time_universal = gmtime(&time); > + if (time_universal == NULL) { > + fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n", > + __func__); > + time = 0; > + } else { > + time = mktime(time_universal); > + } > + } else { > + time = sbuf->st_mtime; > + } > + > /* Build new header */ > image_set_magic(hdr, IH_MAGIC); > - image_set_time(hdr, sbuf->st_mtime); > + image_set_time(hdr, time); > image_set_size(hdr, sbuf->st_size - sizeof(image_header_t)); > image_set_load(hdr, params->addr); > image_set_ep(hdr, params->ep); -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150720/2b9bda00/attachment.sig> ^ permalink raw reply [flat|nested] 9+ messages in thread
* [U-Boot] [PATCH] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH 2015-07-20 8:06 ` Paul Kocialkowski @ 2015-07-20 11:21 ` Heiko Schocher 2015-07-20 13:30 ` Paul Kocialkowski 0 siblings, 1 reply; 9+ messages in thread From: Heiko Schocher @ 2015-07-20 11:21 UTC (permalink / raw) To: u-boot Hello Paul, Am 20.07.2015 um 10:06 schrieb Paul Kocialkowski: > Le lundi 20 juillet 2015 ? 10:01 +0200, Paul Kocialkowski a ?crit : >> In order to achieve reproducible builds in U-Boot, timestamps that are defined >> at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment >> variable allows setting a fixed value for those timestamps. >> >> Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be >> built reproducibly. This is the case for e.g. sunxi devices. >> >> However, some other devices might need some more tweaks, especially regarding >> the image generation tools. > > Lunar, since you have contributed to this patch, feel free to add your > Signed-Off-By line here before it's merged! > > Heiko, since this is based on your original patch, feel free to do the > same! I am just on the jump into my holidays, so I have not yet the time to test it ... I want to try it for all builds with the scripts I posted with my v2 ... but a first fast look into your patch looks nice, if it works, it is ok with me ... I am back aprox. july 5th. maybe a README entry would be fine ;-) Thanks! bye, Heiko > > It would be nice to have this tested on as many boards as possible to > spot other areas that make the binaries not reproducible. However, I > doubt this patch will evolve much and other fixes should be sent in > subsequent patches. > >> Signed-off-by: Paul Kocialkowski <contact@paulk.fr> >> --- >> Makefile | 7 ++++--- >> tools/default_image.c | 21 ++++++++++++++++++++- >> 2 files changed, 24 insertions(+), 4 deletions(-) >> >> diff --git a/Makefile b/Makefile >> index 37cc4c3..71aeac7 100644 >> --- a/Makefile >> +++ b/Makefile >> @@ -1231,9 +1231,10 @@ define filechk_version.h >> endef >> >> define filechk_timestamp.h >> - (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \ >> - LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \ >> - LC_ALL=C date +'#define U_BOOT_TZ "%z"') >> + (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \ >> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \ >> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \ >> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' ) >> endef >> >> $(version_h): include/config/uboot.release FORCE >> diff --git a/tools/default_image.c b/tools/default_image.c >> index cf5c0d4..18940af 100644 >> --- a/tools/default_image.c >> +++ b/tools/default_image.c >> @@ -88,6 +88,9 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, >> struct image_tool_params *params) >> { >> uint32_t checksum; >> + char *source_date_epoch; >> + struct tm *time_universal; >> + time_t time; >> >> image_header_t * hdr = (image_header_t *)ptr; >> >> @@ -96,9 +99,25 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, >> sizeof(image_header_t)), >> sbuf->st_size - sizeof(image_header_t)); >> >> + source_date_epoch = getenv("SOURCE_DATE_EPOCH"); >> + if (source_date_epoch != NULL) { >> + time = (time_t) strtol(source_date_epoch, NULL, 10); >> + >> + time_universal = gmtime(&time); >> + if (time_universal == NULL) { >> + fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n", >> + __func__); >> + time = 0; >> + } else { >> + time = mktime(time_universal); >> + } >> + } else { >> + time = sbuf->st_mtime; >> + } >> + >> /* Build new header */ >> image_set_magic(hdr, IH_MAGIC); >> - image_set_time(hdr, sbuf->st_mtime); >> + image_set_time(hdr, time); >> image_set_size(hdr, sbuf->st_size - sizeof(image_header_t)); >> image_set_load(hdr, params->addr); >> image_set_ep(hdr, params->ep); > -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany ^ permalink raw reply [flat|nested] 9+ messages in thread
* [U-Boot] [PATCH] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH 2015-07-20 11:21 ` Heiko Schocher @ 2015-07-20 13:30 ` Paul Kocialkowski 2015-07-20 15:13 ` Heiko Schocher 0 siblings, 1 reply; 9+ messages in thread From: Paul Kocialkowski @ 2015-07-20 13:30 UTC (permalink / raw) To: u-boot > I am just on the jump into my holidays, so I have not yet the time > to test it ... I want to try it for all builds with the scripts > I posted with my v2 ... but a first fast look into your patch looks > nice, if it works, it is ok with me ... I am back aprox. july 5th. Okay, maybe I'll have a go at your scripts then! The idea to coordinate with Debian developers would be to use their infrastructure to automatically test as many u-boot targets as possible in an automated and periodical manner (e.g. once a month), so that we can easily spot which target is broken or which commit introduced a regression. > maybe a README entry would be fine ;-) Good point, that makes sense. I'll craft that in v2. I'll just wait for some more feedback before sending a new version. Thanks! > > It would be nice to have this tested on as many boards as possible to > > spot other areas that make the binaries not reproducible. However, I > > doubt this patch will evolve much and other fixes should be sent in > > subsequent patches. > > > >> Signed-off-by: Paul Kocialkowski <contact@paulk.fr> > >> --- > >> Makefile | 7 ++++--- > >> tools/default_image.c | 21 ++++++++++++++++++++- > >> 2 files changed, 24 insertions(+), 4 deletions(-) > >> > >> diff --git a/Makefile b/Makefile > >> index 37cc4c3..71aeac7 100644 > >> --- a/Makefile > >> +++ b/Makefile > >> @@ -1231,9 +1231,10 @@ define filechk_version.h > >> endef > >> > >> define filechk_timestamp.h > >> - (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \ > >> - LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \ > >> - LC_ALL=C date +'#define U_BOOT_TZ "%z"') > >> + (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \ > >> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \ > >> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \ > >> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' ) > >> endef > >> > >> $(version_h): include/config/uboot.release FORCE > >> diff --git a/tools/default_image.c b/tools/default_image.c > >> index cf5c0d4..18940af 100644 > >> --- a/tools/default_image.c > >> +++ b/tools/default_image.c > >> @@ -88,6 +88,9 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, > >> struct image_tool_params *params) > >> { > >> uint32_t checksum; > >> + char *source_date_epoch; > >> + struct tm *time_universal; > >> + time_t time; > >> > >> image_header_t * hdr = (image_header_t *)ptr; > >> > >> @@ -96,9 +99,25 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, > >> sizeof(image_header_t)), > >> sbuf->st_size - sizeof(image_header_t)); > >> > >> + source_date_epoch = getenv("SOURCE_DATE_EPOCH"); > >> + if (source_date_epoch != NULL) { > >> + time = (time_t) strtol(source_date_epoch, NULL, 10); > >> + > >> + time_universal = gmtime(&time); > >> + if (time_universal == NULL) { > >> + fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n", > >> + __func__); > >> + time = 0; > >> + } else { > >> + time = mktime(time_universal); > >> + } > >> + } else { > >> + time = sbuf->st_mtime; > >> + } > >> + > >> /* Build new header */ > >> image_set_magic(hdr, IH_MAGIC); > >> - image_set_time(hdr, sbuf->st_mtime); > >> + image_set_time(hdr, time); > >> image_set_size(hdr, sbuf->st_size - sizeof(image_header_t)); > >> image_set_load(hdr, params->addr); > >> image_set_ep(hdr, params->ep); > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150720/4c3ba6fa/attachment.sig> ^ permalink raw reply [flat|nested] 9+ messages in thread
* [U-Boot] [PATCH] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH 2015-07-20 13:30 ` Paul Kocialkowski @ 2015-07-20 15:13 ` Heiko Schocher 2015-07-21 16:20 ` Paul Kocialkowski 2015-07-27 17:36 ` Paul Kocialkowski 0 siblings, 2 replies; 9+ messages in thread From: Heiko Schocher @ 2015-07-20 15:13 UTC (permalink / raw) To: u-boot Hello Paul, Am 20.07.2015 um 15:30 schrieb Paul Kocialkowski: >> I am just on the jump into my holidays, so I have not yet the time >> to test it ... I want to try it for all builds with the scripts >> I posted with my v2 ... but a first fast look into your patch looks >> nice, if it works, it is ok with me ... I am back aprox. july 5th. > > Okay, maybe I'll have a go at your scripts then! The idea to coordinate Uh.. I am just a script beginner, they are not perfect ... Maybe this can be done/added at travis.org ? See u-boot builds there: https://travis-ci.org/u-boot/u-boot/ > with Debian developers would be to use their infrastructure to > automatically test as many u-boot targets as possible in an automated > and periodical manner (e.g. once a month), so that we can easily spot > which target is broken or which commit introduced a regression. This sounds good ... feel free to trigger me, if I can help. bye, Heiko > >> maybe a README entry would be fine ;-) > > Good point, that makes sense. I'll craft that in v2. I'll just wait for > some more feedback before sending a new version. > > Thanks! > >>> It would be nice to have this tested on as many boards as possible to >>> spot other areas that make the binaries not reproducible. However, I >>> doubt this patch will evolve much and other fixes should be sent in >>> subsequent patches. >>> >>>> Signed-off-by: Paul Kocialkowski <contact@paulk.fr> >>>> --- >>>> Makefile | 7 ++++--- >>>> tools/default_image.c | 21 ++++++++++++++++++++- >>>> 2 files changed, 24 insertions(+), 4 deletions(-) >>>> >>>> diff --git a/Makefile b/Makefile >>>> index 37cc4c3..71aeac7 100644 >>>> --- a/Makefile >>>> +++ b/Makefile >>>> @@ -1231,9 +1231,10 @@ define filechk_version.h >>>> endef >>>> >>>> define filechk_timestamp.h >>>> - (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \ >>>> - LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \ >>>> - LC_ALL=C date +'#define U_BOOT_TZ "%z"') >>>> + (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \ >>>> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \ >>>> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \ >>>> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' ) >>>> endef >>>> >>>> $(version_h): include/config/uboot.release FORCE >>>> diff --git a/tools/default_image.c b/tools/default_image.c >>>> index cf5c0d4..18940af 100644 >>>> --- a/tools/default_image.c >>>> +++ b/tools/default_image.c >>>> @@ -88,6 +88,9 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, >>>> struct image_tool_params *params) >>>> { >>>> uint32_t checksum; >>>> + char *source_date_epoch; >>>> + struct tm *time_universal; >>>> + time_t time; >>>> >>>> image_header_t * hdr = (image_header_t *)ptr; >>>> >>>> @@ -96,9 +99,25 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, >>>> sizeof(image_header_t)), >>>> sbuf->st_size - sizeof(image_header_t)); >>>> >>>> + source_date_epoch = getenv("SOURCE_DATE_EPOCH"); >>>> + if (source_date_epoch != NULL) { >>>> + time = (time_t) strtol(source_date_epoch, NULL, 10); >>>> + >>>> + time_universal = gmtime(&time); >>>> + if (time_universal == NULL) { >>>> + fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n", >>>> + __func__); >>>> + time = 0; >>>> + } else { >>>> + time = mktime(time_universal); >>>> + } >>>> + } else { >>>> + time = sbuf->st_mtime; >>>> + } >>>> + >>>> /* Build new header */ >>>> image_set_magic(hdr, IH_MAGIC); >>>> - image_set_time(hdr, sbuf->st_mtime); >>>> + image_set_time(hdr, time); >>>> image_set_size(hdr, sbuf->st_size - sizeof(image_header_t)); >>>> image_set_load(hdr, params->addr); >>>> image_set_ep(hdr, params->ep); >>> >> > -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany ^ permalink raw reply [flat|nested] 9+ messages in thread
* [U-Boot] [PATCH] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH 2015-07-20 15:13 ` Heiko Schocher @ 2015-07-21 16:20 ` Paul Kocialkowski 2015-07-27 17:36 ` Paul Kocialkowski 1 sibling, 0 replies; 9+ messages in thread From: Paul Kocialkowski @ 2015-07-21 16:20 UTC (permalink / raw) To: u-boot Le lundi 20 juillet 2015 ? 17:13 +0200, Heiko Schocher a ?crit : > Hello Paul, > > Am 20.07.2015 um 15:30 schrieb Paul Kocialkowski: > >> I am just on the jump into my holidays, so I have not yet the time > >> to test it ... I want to try it for all builds with the scripts > >> I posted with my v2 ... but a first fast look into your patch looks > >> nice, if it works, it is ok with me ... I am back aprox. july 5th. > > > > Okay, maybe I'll have a go at your scripts then! The idea to coordinate > > Uh.. I am just a script beginner, they are not perfect ... I'll let you know when I have some time to do the testing properly then! > Maybe this can be done/added at travis.org ? > See u-boot builds there: > https://travis-ci.org/u-boot/u-boot/ I don't like the idea of using a third-party's architecture for that. This is service as a software substitute: see http://www.gnu.org/philosophy/who-does-that-server-really-serve.html > > with Debian developers would be to use their infrastructure to > > automatically test as many u-boot targets as possible in an automated > > and periodical manner (e.g. once a month), so that we can easily spot > > which target is broken or which commit introduced a regression. > > This sounds good ... feel free to trigger me, if I can help. > > bye, > Heiko > > > >> maybe a README entry would be fine ;-) > > > > Good point, that makes sense. I'll craft that in v2. I'll just wait for > > some more feedback before sending a new version. > > > > Thanks! > > > >>> It would be nice to have this tested on as many boards as possible to > >>> spot other areas that make the binaries not reproducible. However, I > >>> doubt this patch will evolve much and other fixes should be sent in > >>> subsequent patches. > >>> > >>>> Signed-off-by: Paul Kocialkowski <contact@paulk.fr> > >>>> --- > >>>> Makefile | 7 ++++--- > >>>> tools/default_image.c | 21 ++++++++++++++++++++- > >>>> 2 files changed, 24 insertions(+), 4 deletions(-) > >>>> > >>>> diff --git a/Makefile b/Makefile > >>>> index 37cc4c3..71aeac7 100644 > >>>> --- a/Makefile > >>>> +++ b/Makefile > >>>> @@ -1231,9 +1231,10 @@ define filechk_version.h > >>>> endef > >>>> > >>>> define filechk_timestamp.h > >>>> - (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \ > >>>> - LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \ > >>>> - LC_ALL=C date +'#define U_BOOT_TZ "%z"') > >>>> + (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \ > >>>> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \ > >>>> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \ > >>>> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' ) > >>>> endef > >>>> > >>>> $(version_h): include/config/uboot.release FORCE > >>>> diff --git a/tools/default_image.c b/tools/default_image.c > >>>> index cf5c0d4..18940af 100644 > >>>> --- a/tools/default_image.c > >>>> +++ b/tools/default_image.c > >>>> @@ -88,6 +88,9 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, > >>>> struct image_tool_params *params) > >>>> { > >>>> uint32_t checksum; > >>>> + char *source_date_epoch; > >>>> + struct tm *time_universal; > >>>> + time_t time; > >>>> > >>>> image_header_t * hdr = (image_header_t *)ptr; > >>>> > >>>> @@ -96,9 +99,25 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, > >>>> sizeof(image_header_t)), > >>>> sbuf->st_size - sizeof(image_header_t)); > >>>> > >>>> + source_date_epoch = getenv("SOURCE_DATE_EPOCH"); > >>>> + if (source_date_epoch != NULL) { > >>>> + time = (time_t) strtol(source_date_epoch, NULL, 10); > >>>> + > >>>> + time_universal = gmtime(&time); > >>>> + if (time_universal == NULL) { > >>>> + fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n", > >>>> + __func__); > >>>> + time = 0; > >>>> + } else { > >>>> + time = mktime(time_universal); > >>>> + } > >>>> + } else { > >>>> + time = sbuf->st_mtime; > >>>> + } > >>>> + > >>>> /* Build new header */ > >>>> image_set_magic(hdr, IH_MAGIC); > >>>> - image_set_time(hdr, sbuf->st_mtime); > >>>> + image_set_time(hdr, time); > >>>> image_set_size(hdr, sbuf->st_size - sizeof(image_header_t)); > >>>> image_set_load(hdr, params->addr); > >>>> image_set_ep(hdr, params->ep); > >>> > >> > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150721/aa506d01/attachment.sig> ^ permalink raw reply [flat|nested] 9+ messages in thread
* [U-Boot] [PATCH] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH 2015-07-20 15:13 ` Heiko Schocher 2015-07-21 16:20 ` Paul Kocialkowski @ 2015-07-27 17:36 ` Paul Kocialkowski 1 sibling, 0 replies; 9+ messages in thread From: Paul Kocialkowski @ 2015-07-27 17:36 UTC (permalink / raw) To: u-boot Le lundi 20 juillet 2015 ? 17:13 +0200, Heiko Schocher a ?crit : > Hello Paul, > > Am 20.07.2015 um 15:30 schrieb Paul Kocialkowski: > >> I am just on the jump into my holidays, so I have not yet the time > >> to test it ... I want to try it for all builds with the scripts > >> I posted with my v2 ... but a first fast look into your patch looks > >> nice, if it works, it is ok with me ... I am back aprox. july 5th. > > > > Okay, maybe I'll have a go at your scripts then! The idea to coordinate > > Uh.. I am just a script beginner, they are not perfect ... I got around testing your scripts with the latest version of my patch yesterday. The scripts look good, they get the job done. I tried building on all the supported ARM boards (grep CONFIG_ARM in configs/) and all boards appear to be reproducible. However, it turns out that when build with the -C and O= arguments (to indicate the source directory and to output objects in a separate directory), the binaries differed. However, running the second call of your scripts after moving the directory didn't result in a change. I have yet to figure out exactly what the differences are and what they are caused by. > Maybe this can be done/added at travis.org ? > See u-boot builds there: > https://travis-ci.org/u-boot/u-boot/ > > > with Debian developers would be to use their infrastructure to > > automatically test as many u-boot targets as possible in an automated > > and periodical manner (e.g. once a month), so that we can easily spot > > which target is broken or which commit introduced a regression. > > This sounds good ... feel free to trigger me, if I can help. > > bye, > Heiko > > > >> maybe a README entry would be fine ;-) > > > > Good point, that makes sense. I'll craft that in v2. I'll just wait for > > some more feedback before sending a new version. > > > > Thanks! > > > >>> It would be nice to have this tested on as many boards as possible to > >>> spot other areas that make the binaries not reproducible. However, I > >>> doubt this patch will evolve much and other fixes should be sent in > >>> subsequent patches. > >>> > >>>> Signed-off-by: Paul Kocialkowski <contact@paulk.fr> > >>>> --- > >>>> Makefile | 7 ++++--- > >>>> tools/default_image.c | 21 ++++++++++++++++++++- > >>>> 2 files changed, 24 insertions(+), 4 deletions(-) > >>>> > >>>> diff --git a/Makefile b/Makefile > >>>> index 37cc4c3..71aeac7 100644 > >>>> --- a/Makefile > >>>> +++ b/Makefile > >>>> @@ -1231,9 +1231,10 @@ define filechk_version.h > >>>> endef > >>>> > >>>> define filechk_timestamp.h > >>>> - (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \ > >>>> - LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \ > >>>> - LC_ALL=C date +'#define U_BOOT_TZ "%z"') > >>>> + (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \ > >>>> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \ > >>>> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \ > >>>> + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' ) > >>>> endef > >>>> > >>>> $(version_h): include/config/uboot.release FORCE > >>>> diff --git a/tools/default_image.c b/tools/default_image.c > >>>> index cf5c0d4..18940af 100644 > >>>> --- a/tools/default_image.c > >>>> +++ b/tools/default_image.c > >>>> @@ -88,6 +88,9 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, > >>>> struct image_tool_params *params) > >>>> { > >>>> uint32_t checksum; > >>>> + char *source_date_epoch; > >>>> + struct tm *time_universal; > >>>> + time_t time; > >>>> > >>>> image_header_t * hdr = (image_header_t *)ptr; > >>>> > >>>> @@ -96,9 +99,25 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd, > >>>> sizeof(image_header_t)), > >>>> sbuf->st_size - sizeof(image_header_t)); > >>>> > >>>> + source_date_epoch = getenv("SOURCE_DATE_EPOCH"); > >>>> + if (source_date_epoch != NULL) { > >>>> + time = (time_t) strtol(source_date_epoch, NULL, 10); > >>>> + > >>>> + time_universal = gmtime(&time); > >>>> + if (time_universal == NULL) { > >>>> + fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n", > >>>> + __func__); > >>>> + time = 0; > >>>> + } else { > >>>> + time = mktime(time_universal); > >>>> + } > >>>> + } else { > >>>> + time = sbuf->st_mtime; > >>>> + } > >>>> + > >>>> /* Build new header */ > >>>> image_set_magic(hdr, IH_MAGIC); > >>>> - image_set_time(hdr, sbuf->st_mtime); > >>>> + image_set_time(hdr, time); > >>>> image_set_size(hdr, sbuf->st_size - sizeof(image_header_t)); > >>>> image_set_load(hdr, params->addr); > >>>> image_set_ep(hdr, params->ep); > >>> > >> > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150727/1bd39fb8/attachment.sig> ^ permalink raw reply [flat|nested] 9+ messages in thread
* [U-Boot] [PATCH] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH 2015-07-20 8:01 [U-Boot] [PATCH] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH Paul Kocialkowski 2015-07-20 8:06 ` Paul Kocialkowski @ 2015-07-20 14:45 ` Vagrant Cascadian 2015-07-21 16:18 ` Paul Kocialkowski 1 sibling, 1 reply; 9+ messages in thread From: Vagrant Cascadian @ 2015-07-20 14:45 UTC (permalink / raw) To: u-boot On 2015-07-20, Paul Kocialkowski wrote: > In order to achieve reproducible builds in U-Boot, timestamps that are defined > at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment > variable allows setting a fixed value for those timestamps. ... > diff --git a/Makefile b/Makefile > index 37cc4c3..71aeac7 100644 > --- a/Makefile > +++ b/Makefile > @@ -1231,9 +1231,10 @@ define filechk_version.h > endef > > define filechk_timestamp.h > - (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \ > - LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \ > - LC_ALL=C date +'#define U_BOOT_TZ "%z"') > + (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \ > + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \ > + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \ > + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' ) > endef > > $(version_h): include/config/uboot.release FORCE This does effectively hard-code U_BOOT_TZ to UTC; may as well not call date for setting U_BOOT_TZ. Or conditionally set it to UTC only when SOURCE_DATE_EPOCH is set? Any reason not to use the longhand options for date, e.g. --utc and --date ? They're more readable; are they less portable? live well, vagrant -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150720/5d306ea6/attachment.sig> ^ permalink raw reply [flat|nested] 9+ messages in thread
* [U-Boot] [PATCH] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH 2015-07-20 14:45 ` Vagrant Cascadian @ 2015-07-21 16:18 ` Paul Kocialkowski 0 siblings, 0 replies; 9+ messages in thread From: Paul Kocialkowski @ 2015-07-21 16:18 UTC (permalink / raw) To: u-boot Le lundi 20 juillet 2015 ? 07:45 -0700, Vagrant Cascadian a ?crit : > On 2015-07-20, Paul Kocialkowski wrote: > > In order to achieve reproducible builds in U-Boot, timestamps that are defined > > at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment > > variable allows setting a fixed value for those timestamps. > ... > > diff --git a/Makefile b/Makefile > > index 37cc4c3..71aeac7 100644 > > --- a/Makefile > > +++ b/Makefile > > @@ -1231,9 +1231,10 @@ define filechk_version.h > > endef > > > > define filechk_timestamp.h > > - (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \ > > - LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \ > > - LC_ALL=C date +'#define U_BOOT_TZ "%z"') > > + (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \ > > + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \ > > + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \ > > + LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' ) > > endef > > > > $(version_h): include/config/uboot.release FORCE > > This does effectively hard-code U_BOOT_TZ to UTC; may as well not call > date for setting U_BOOT_TZ. Or conditionally set it to UTC only when > SOURCE_DATE_EPOCH is set? That's true, but I like how consistent those commands look. Either way, it's not a dramatic overhead, but I agree it's slightly confusing. If you really think it's worth it, I could simply hardcode UTC in v2. Just let me know! I'd rather keep everything in one call (doing UTC only when SOURCE_DATE_EPOCH is set looks overkill). > Any reason not to use the longhand options for date, e.g. --utc and > --date ? They're more readable; are they less portable? I don't think they are, but the short options look fine to me. Note that out of those lines, two still fit in a 80 chars column. Adding long options would make readability harder in that regard. As far as I'm concerned, it's fine as it is, but if you really think it would be a worthwhile addition to use the long options, let me know. Please do check that it doesn't break portability, too. Thanks for the review! -- Paul Kocialkowski, Replicant developer Replicant is a fully free Android distribution running on several devices, a free software mobile operating system putting the emphasis on freedom and privacy/security. Website: http://www.replicant.us/ Blog: http://blog.replicant.us/ Wiki/tracker/forums: http://redmine.replicant.us/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150721/1d0bf2bf/attachment.sig> ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2015-07-27 17:36 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-07-20 8:01 [U-Boot] [PATCH] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH Paul Kocialkowski 2015-07-20 8:06 ` Paul Kocialkowski 2015-07-20 11:21 ` Heiko Schocher 2015-07-20 13:30 ` Paul Kocialkowski 2015-07-20 15:13 ` Heiko Schocher 2015-07-21 16:20 ` Paul Kocialkowski 2015-07-27 17:36 ` Paul Kocialkowski 2015-07-20 14:45 ` Vagrant Cascadian 2015-07-21 16:18 ` Paul Kocialkowski
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox