[U-Boot] [PATCH] build: create time and date independent binary

[U-Boot] [PATCH] build: create time and date independent binary

[Heiko Schocher]

With this option enabled each compilation creates the exact
same binary. There are no timestamps, with which a U-Boot
binary can be identified.

This option is disabled by default.

Signed-off-by: Heiko Schocher <hs@denx.de>
---

 Kconfig  | 9 +++++++++
 Makefile | 8 ++++++++
 2 files changed, 17 insertions(+)

diff --git a/Kconfig b/Kconfig
index 15e15af..c0c7eb8 100644
--- a/Kconfig
+++ b/Kconfig
@@ -81,6 +81,15 @@ menuconfig EXPERT
 	  Only use this if you really know what you are doing.
 
 if EXPERT
+	config SYS_EXACT_BINARY
+	bool "Create time and date independent binary"
+	default n
+	help
+	  With this option enabled each compilation creates the exact
+	  same binary. There are no timestamps, with which a U-Boot
+	  binary can be identified.
+	  This option is disabled by default.
+
 	config SYS_MALLOC_CLEAR_ON_INIT
 	bool "Init with zeros the memory reserved for malloc (slow)"
 	default y
diff --git a/Makefile b/Makefile
index 0a674bf..4ae2701 100644
--- a/Makefile
+++ b/Makefile
@@ -1230,11 +1230,19 @@ define filechk_version.h
 	echo \#define LD_VERSION_STRING \"$$($(LD) --version | head -n 1)\"; )
 endef
 
+ifeq ($(CONFIG_SYS_EXACT_BINARY),y)
+define filechk_timestamp.h
+	(LC_ALL=C date +'#define U_BOOT_DATE "NODATE"'; \
+	LC_ALL=C date +'#define U_BOOT_TIME "NOTIME"'; \
+	LC_ALL=C date +'#define U_BOOT_TZ "NOTZ"')
+endef
+else
 define filechk_timestamp.h
 	(LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 	LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
 	LC_ALL=C date +'#define U_BOOT_TZ "%z"')
 endef
+endif
 
 $(version_h): include/config/uboot.release FORCE
 	$(call filechk,version.h)
-- 
2.1.0

[U-Boot] [PATCH] build: create time and date independent binary

[Paul Kocialkowski]

Le vendredi 12 juin 2015 ? 17:31 +0200, Heiko Schocher a ?crit :
> With this option enabled each compilation creates the exact
> same binary. There are no timestamps, with which a U-Boot
> binary can be identified.
> 
> This option is disabled by default.

I think a bit more is needed to get truly reproducible builds in U-Boot,
but this is a first good step!

I have been thinking about bringing reproducible builds to U-Boot for a
little while, but never got around actually doing it. Are you interested
in helping me on that? I have some free time these days, so I could do
some work on this.

It would certainly be relevant to get in touch with the people working
on reproducing builds for Debian (I know Lunar is involved with this),
since Debian packages U-Boot for many devices and they're apparently
willing to help upstream with automated tests, as they're doing with
coreboot and OpenWRT!

> Signed-off-by: Heiko Schocher <hs@denx.de>
> ---
> 
>  Kconfig  | 9 +++++++++
>  Makefile | 8 ++++++++
>  2 files changed, 17 insertions(+)
> 
> diff --git a/Kconfig b/Kconfig
> index 15e15af..c0c7eb8 100644
> --- a/Kconfig
> +++ b/Kconfig
> @@ -81,6 +81,15 @@ menuconfig EXPERT
>  	  Only use this if you really know what you are doing.
>  
>  if EXPERT
> +	config SYS_EXACT_BINARY
> +	bool "Create time and date independent binary"
> +	default n
> +	help
> +	  With this option enabled each compilation creates the exact
> +	  same binary. There are no timestamps, with which a U-Boot
> +	  binary can be identified.
> +	  This option is disabled by default.
> +
>  	config SYS_MALLOC_CLEAR_ON_INIT
>  	bool "Init with zeros the memory reserved for malloc (slow)"
>  	default y
> diff --git a/Makefile b/Makefile
> index 0a674bf..4ae2701 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1230,11 +1230,19 @@ define filechk_version.h
>  	echo \#define LD_VERSION_STRING \"$$($(LD) --version | head -n 1)\"; )
>  endef
>  
> +ifeq ($(CONFIG_SYS_EXACT_BINARY),y)
> +define filechk_timestamp.h
> +	(LC_ALL=C date +'#define U_BOOT_DATE "NODATE"'; \
> +	LC_ALL=C date +'#define U_BOOT_TIME "NOTIME"'; \
> +	LC_ALL=C date +'#define U_BOOT_TZ "NOTZ"')
> +endef
> +else
>  define filechk_timestamp.h
>  	(LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
>  	LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
>  	LC_ALL=C date +'#define U_BOOT_TZ "%z"')
>  endef
> +endif
>  
>  $(version_h): include/config/uboot.release FORCE
>  	$(call filechk,version.h)

-- 
Paul Kocialkowski, Replicant developer

Replicant is a fully free Android distribution running on several
devices, a free software mobile operating system putting the emphasis on
freedom and privacy/security.

Website: http://www.replicant.us/
Blog: http://blog.replicant.us/
Wiki/tracker/forums: http://redmine.replicant.us/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150612/dc33cfbb/attachment.sig>

[U-Boot] [Reproducible-builds] [PATCH] build: create time and date independent binary

[Holger Levsen]

Hi Paul,

thanks for contacting us!

On Freitag, 12. Juni 2015, Paul Kocialkowski wrote:
> I think a bit more is needed to get truly reproducible builds in U-Boot,
> but this is a first good step!

you've seen https://reproducible.debian.net/u-boot ?

> I have been thinking about bringing reproducible builds to U-Boot for a
> little while, but never got around actually doing it. Are you interested
> in helping me on that? I have some free time these days, so I could do
> some work on this.

cool!

> It would certainly be relevant to get in touch with the people working
> on reproducing builds for Debian (I know Lunar is involved with this),
> since Debian packages U-Boot for many devices and they're apparently
> willing to help upstream with automated tests, as they're doing with
> coreboot and OpenWRT!

we're definitly happy to help! 

that said, I'm not sure I want to treat uboot like openwrt or coreboot, simply 
because uboot is packaged for several distributions, so (in my maybe naive 
assumptions) it should be tested within those distros.

coreboot is really exceptional here, as it doesn't make much sense to package 
it as part of a distro. (though this could also change, at least sources could 
be shipped...)

but maybe you can explain why u-boot needs more reproducibility testing than 
what there currently is. i'm definitly interested and not opposed, even though 
I think there shoukd be good reasons to treat some software specially.

(also please note that we currently only have amd64 hw to run our tests on.)

In a similar way I have for now decided that I'm not interested in adding 
tests of libreboot, as it's basically just a fork of coreboot, so the 
reproducibility issues should be the same. OTOH libreboot releases images 
(which coreboot doesnt do) so that might become interesting to validate. But 
there I think that the validation should be done as part of that project 
(=libreboot), and not as part of reproducible.debian.net, which I mostly see 
as a.) effort to test and push Debians reproducibility effort and b.) show 
that this is possible for other distros too - but I think in long term that 
should move to a setup located+maintained at those other distros. (because we 
lack the *human*power to look after too/so many things...)

(And really, testing on reproducible.debian.net is not enough, it's just the 
first step: now we know that coreboot is 100% reproducible, now what? This 
"what" needs to happen on the coreboot side...)

So it's not (so much or maybe at all) a question of hardware ressources, which 
we can scale up rather easily, but mostly human ressources maintaining the 
hw+sw *and* translating this into meaningful action for the project tested.

All this said, if you send me patches, I will probably deploy them as I'm very 
curious and more reproducibility efforts are good :-) We can can always decide 
to remove or move them later.


cheers,
	Holger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150613/8d73dec1/attachment.sig>

[U-Boot] [Reproducible-builds] [PATCH] build: create time and date independent binary

[Paul Kocialkowski]

Le samedi 13 juin 2015 ? 11:10 +0200, Holger Levsen a ?crit :
> Hi Paul,
> 
> thanks for contacting us!
> 
> On Freitag, 12. Juni 2015, Paul Kocialkowski wrote:
> > I think a bit more is needed to get truly reproducible builds in U-Boot,
> > but this is a first good step!
> 
> you've seen https://reproducible.debian.net/u-boot ?

This seems very minimalistic, but it's good to see U-Boot was given some
attention already!

> > It would certainly be relevant to get in touch with the people working
> > on reproducing builds for Debian (I know Lunar is involved with this),
> > since Debian packages U-Boot for many devices and they're apparently
> > willing to help upstream with automated tests, as they're doing with
> > coreboot and OpenWRT!
> 
> we're definitly happy to help! 
> 
> that said, I'm not sure I want to treat uboot like openwrt or coreboot, simply 
> because uboot is packaged for several distributions, so (in my maybe naive 
> assumptions) it should be tested within those distros.
> 
> coreboot is really exceptional here, as it doesn't make much sense to package 
> it as part of a distro. (though this could also change, at least sources could 
> be shipped...)
> 
> but maybe you can explain why u-boot needs more reproducibility testing than 
> what there currently is. i'm definitly interested and not opposed, even though 
> I think there shoukd be good reasons to treat some software specially.

The point is to make U-Boot reproducible for all possible targets, not
only the few ones that are supported by U-Boot. I think this requires
some extra infrastructure. In that sense, it is very similar to
Coreboot.

> (also please note that we currently only have amd64 hw to run our tests on.)

The problem is the same as Coreboot, which uses its own toolchain to
build images. We don't need to have native armhf builds for U-Boot,
testing with the armhf toolchain that is in Debian should be enough.

> In a similar way I have for now decided that I'm not interested in adding 
> tests of libreboot, as it's basically just a fork of coreboot, so the 
> reproducibility issues should be the same. OTOH libreboot releases images 
> (which coreboot doesnt do) so that might become interesting to validate. But 
> there I think that the validation should be done as part of that project 
> (=libreboot), and not as part of reproducible.debian.net, which I mostly see 
> as a.) effort to test and push Debians reproducibility effort and b.) show 
> that this is possible for other distros too - but I think in long term that 
> should move to a setup located+maintained at those other distros. (because we 
> lack the *human*power to look after too/so many things...)

I understand, this works out nicely because all the work on Coreboot
will be inherited by Libreboot. However, on U-Boot, the work to bring
reproducible builds has to take place initially. I know for a fact that
parts of the code use things like __FILE__ or timestamps.

Also, if those were fixed in Debian, it would be nice to get those
changes back in upstream.

> (And really, testing on reproducible.debian.net is not enough, it's just the 
> first step: now we know that coreboot is 100% reproducible, now what? This 
> "what" needs to happen on the coreboot side...)
> 
> So it's not (so much or maybe at all) a question of hardware ressources, which 
> we can scale up rather easily, but mostly human ressources maintaining the 
> hw+sw *and* translating this into meaningful action for the project tested.

That makes sense. For U-Boot, it will certainly make sense for the
distributions packaging it. I'm the main developer of Replicant, the
fully free version of Android, and it would definitely be useful to have
a smartphone on which we can trust that the bootloader can be built in a
reproducible manner (and checked after being installed).

> All this said, if you send me patches, I will probably deploy them as I'm very 
> curious and more reproducibility efforts are good :-) We can can always decide 
> to remove or move them later.

I wish to make all contributions upstream. What would really help at
first would be to have all targets built regularly to see where work is
needed. This is where I think the Debian infrastructure could help, in a
similar way as what was started for Coreboot.

Thanks for your help!

-- 
Paul Kocialkowski, Replicant developer

Replicant is a fully free Android distribution running on several
devices, a free software mobile operating system putting the emphasis on
freedom and privacy/security.

Website: http://www.replicant.us/
Blog: http://blog.replicant.us/
Wiki/tracker/forums: http://redmine.replicant.us/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150613/f54e04ce/attachment.sig>

[U-Boot] [Reproducible-builds] [PATCH] build: create time and date independent binary

[Holger Levsen]

Hi Paul,

sorry for the late reply. 

On Samstag, 13. Juni 2015, Paul Kocialkowski wrote:
> > you've seen https://reproducible.debian.net/u-boot ?
> This seems very minimalistic, but it's good to see U-Boot was given some
> attention already!

:-)
 
> > but maybe you can explain why u-boot needs more reproducibility testing
> > than what there currently is. i'm definitly interested and not opposed,
> > even though I think there shoukd be good reasons to treat some software
> > specially.
> The point is to make U-Boot reproducible for all possible targets, not
> only the few ones that are supported by U-Boot. 

I think your sentence is missing some word??!?

> I think this requires
> some extra infrastructure. In that sense, it is very similar to
> Coreboot.
> 
> > (also please note that we currently only have amd64 hw to run our tests
> > on.)
> 
> The problem is the same as Coreboot, which uses its own toolchain to
> build images. We don't need to have native armhf builds for U-Boot,
> testing with the armhf toolchain that is in Debian should be enough.

I see.
 
> I understand, this works out nicely because all the work on Coreboot
> will be inherited by Libreboot. However, on U-Boot, the work to bring
> reproducible builds has to take place initially. I know for a fact that
> parts of the code use things like __FILE__ or timestamps.

Ah.
 
> That makes sense. For U-Boot, it will certainly make sense for the
> distributions packaging it. I'm the main developer of Replicant, the
> fully free version of Android, and it would definitely be useful to have
> a smartphone on which we can trust that the bootloader can be built in a
> reproducible manner (and checked after being installed).

indeed!
 
> > All this said, if you send me patches, I will probably deploy them as I'm
> > very curious and more reproducibility efforts are good :-) We can can
> > always decide to remove or move them later.
> 
> I wish to make all contributions upstream. What would really help at
> first would be to have all targets built regularly to see where work is
> needed. This is where I think the Debian infrastructure could help, in a
> similar way as what was started for Coreboot.

can you point me to a how to explaining this or tell me those steps, starting 
with "git clone..."?


cheers,
	Holger


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150719/b087fd9c/attachment.sig>

[U-Boot] [Reproducible-builds] [PATCH] build: create time and date independent binary

[Vagrant Cascadian]

On 2015-07-19, Holger Levsen wrote:
>> > All this said, if you send me patches, I will probably deploy them as I'm
>> > very curious and more reproducibility efforts are good :-) We can can
>> > always decide to remove or move them later.
>> 
>> I wish to make all contributions upstream. What would really help at
>> first would be to have all targets built regularly to see where work is
>> needed. This is where I think the Debian infrastructure could help, in a
>> similar way as what was started for Coreboot.

FWIW, I was planning on including this patch to u-boot in the next
upload to Debian:

  https://anonscm.debian.org/cgit/collab-maint/u-boot.git/tree/debian/patches/use-date-from-debian-changelog.patch?h=experimental-2015.07

I *think* that actually makes u-boot build reproducibly with Debian's
reproducible builds toolchain when SOURCE_DATE_EPOCH is set, but I
haven't tested it fully. I might have missed some other sources of
non-determinism...


Hoping to get some armhf buildd nodes up an running soonish... although
it should also be buildable with the cross-toolchains, if the
reproducible buildds coulld be made to support that.


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150719/5b82d61c/attachment.sig>

[U-Boot] [Reproducible-builds] [PATCH] build: create time and date independent binary

[Paul Kocialkowski]

Le dimanche 19 juillet 2015 ? 06:14 -0700, Vagrant Cascadian a ?crit :
> On 2015-07-19, Holger Levsen wrote:
> >> > All this said, if you send me patches, I will probably deploy them as I'm
> >> > very curious and more reproducibility efforts are good :-) We can can
> >> > always decide to remove or move them later.
> >> 
> >> I wish to make all contributions upstream. What would really help at
> >> first would be to have all targets built regularly to see where work is
> >> needed. This is where I think the Debian infrastructure could help, in a
> >> similar way as what was started for Coreboot.
> 
> FWIW, I was planning on including this patch to u-boot in the next
> upload to Debian:
> 
>   https://anonscm.debian.org/cgit/collab-maint/u-boot.git/tree/debian/patches/use-date-from-debian-changelog.patch?h=experimental-2015.07
> 
> I *think* that actually makes u-boot build reproducibly with Debian's
> reproducible builds toolchain when SOURCE_DATE_EPOCH is set, but I
> haven't tested it fully. I might have missed some other sources of
> non-determinism...

We came up with something similar last week. This is not actually
sufficient, but close. I will submit what I have to the U-Boot mailing
list soon (hopefully, later today), feel free to review it there and
import it to Debian.

> Hoping to get some armhf buildd nodes up an running soonish... although
> it should also be buildable with the cross-toolchains, if the
> reproducible buildds coulld be made to support that.

Native builds don't seem realistic given the many different
architectures that U-Boot supports, unless done with virtualisation (see
my other email from today).

Now that I think about it, maybe virtualisation would require simply too
many VMs to setup.

Either way, we can always try to add a toolchain build script to U-Boot.

-- 
Paul Kocialkowski, Replicant developer

Replicant is a fully free Android distribution running on several
devices, a free software mobile operating system putting the emphasis on
freedom and privacy/security.

Website: http://www.replicant.us/
Blog: http://blog.replicant.us/
Wiki/tracker/forums: http://redmine.replicant.us/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150719/a9acaffd/attachment.sig>

[U-Boot] [Reproducible-builds] [PATCH] build: create time and date independent binary

[Holger Levsen]

Hi,

On Sonntag, 19. Juli 2015, Paul Kocialkowski wrote:
> > I *think* that actually makes u-boot build reproducibly with Debian's
> > reproducible builds toolchain when SOURCE_DATE_EPOCH is set, but I
> > haven't tested it fully. I might have missed some other sources of
> > non-determinism...
> We came up with something similar last week. This is not actually
> sufficient, but close. I will submit what I have to the U-Boot mailing
> list soon (hopefully, later today), feel free to review it there and
> import it to Debian.

cool!
 
> Native builds don't seem realistic given the many different
> architectures that U-Boot supports, unless done with virtualisation (see
> my other email from today).
[...]
> Either way, we can always try to add a toolchain build script to U-Boot.

I agree, I also think we should test uboot with such a script, similar like we 
do for coreboot. That wouldnt need armhf buildds.

That said, making use of these armhf buildds Vagrant is planning to setup 
would also useful to test Debian packages (incl uboot) on armhf.


cheers,
	Holger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150719/0f2f5a3a/attachment.sig>

[U-Boot] [Reproducible-builds] [PATCH] build: create time and date independent binary

[Heiko Schocher]

Hello Vagrant,

Am 19.07.2015 um 15:14 schrieb Vagrant Cascadian:
> On 2015-07-19, Holger Levsen wrote:
>>>> All this said, if you send me patches, I will probably deploy them as I'm
>>>> very curious and more reproducibility efforts are good :-) We can can
>>>> always decide to remove or move them later.
>>>
>>> I wish to make all contributions upstream. What would really help at
>>> first would be to have all targets built regularly to see where work is
>>> needed. This is where I think the Debian infrastructure could help, in a
>>> similar way as what was started for Coreboot.
>
> FWIW, I was planning on including this patch to u-boot in the next
> upload to Debian:
>
>    https://anonscm.debian.org/cgit/collab-maint/u-boot.git/tree/debian/patches/use-date-from-debian-changelog.patch?h=experimental-2015.07
>
> I *think* that actually makes u-boot build reproducibly with Debian's
> reproducible builds toolchain when SOURCE_DATE_EPOCH is set, but I
> haven't tested it fully. I might have missed some other sources of
> non-determinism...
>
>
> Hoping to get some armhf buildd nodes up an running soonish... although
> it should also be buildable with the cross-toolchains, if the
> reproducible buildds coulld be made to support that.

Did you see my v2 patch [1] for u-boot?

It also builds now u-boot images created with mkimage reproducible ...

If I interpret your patch from above correct, you add with
SOURCE_DATE_EPOCH a specific fix timestamp?

I think, this could be included to my approach too ...

If SOURCE_DATE_EPOCH is defined, use it, fi not defined
define U_BOOT_DATE, U_BOOT_TIME and U_BOOT_TZ
with my default settings from [1] ...

All only if CONFIG_SYS_EXACT_BINARY is set in the u-boot
config of course ...

What do you think?

bye,
Heiko
[1] http://patchwork.ozlabs.org/patch/487530/
-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] [Reproducible-builds] [PATCH] build: create time and date independent binary

[Paul Kocialkowski]

> Did you see my v2 patch [1] for u-boot?
> 
> It also builds now u-boot images created with mkimage reproducible ...
> 
> If I interpret your patch from above correct, you add with
> SOURCE_DATE_EPOCH a specific fix timestamp?
> 
> I think, this could be included to my approach too ...
> 
> If SOURCE_DATE_EPOCH is defined, use it, fi not defined
> define U_BOOT_DATE, U_BOOT_TIME and U_BOOT_TZ
> with my default settings from [1] ...
> 
> All only if CONFIG_SYS_EXACT_BINARY is set in the u-boot
> config of course ...
> 
> What do you think?

I think adding a config option adds all sorts of unnecessary
complications. SOURCE_DATE_EPOCH is sufficient for what we want.

Thanks a lot for your work, it has been a great base for the patch I
just sent: http://patchwork.ozlabs.org/patch/497577/

Feel free to add your signed-off-by line since it's inspired by your
patch.

-- 
Paul Kocialkowski, Replicant developer

Replicant is a fully free Android distribution running on several
devices, a free software mobile operating system putting the emphasis on
freedom and privacy/security.

Website: http://www.replicant.us/
Blog: http://blog.replicant.us/
Wiki/tracker/forums: http://redmine.replicant.us/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150720/5ef28ae6/attachment.sig>

[U-Boot] [Reproducible-builds] [PATCH] build: create time and date independent binary

[Paul Kocialkowski]

> sorry for the late reply. 

No problem, I haven't kept you or the list posted on this either.

I got a chance to discuss the issue with Lunar during RMLL 2015 and we
came up with a nice way of doing things, using SOURCE_DATE_EPOCH.

> On Samstag, 13. Juni 2015, Paul Kocialkowski wrote:
> > > you've seen https://reproducible.debian.net/u-boot ?
> > This seems very minimalistic, but it's good to see U-Boot was given
> some
> > attention already!

As far as I understood, this was only for the U-Boot tools.

> > > but maybe you can explain why u-boot needs more reproducibility
> testing
> > > than what there currently is. i'm definitly interested and not
> opposed,
> > > even though I think there shoukd be good reasons to treat some
> software
> > > specially.
> > The point is to make U-Boot reproducible for all possible targets,
> not
> > only the few ones that are supported by U-Boot. 
> 
> I think your sentence is missing some word??!?

not only for the few one that are supported by U-Boot maybe?

Looks good otherwise!

> > I think this requires
> > some extra infrastructure. In that sense, it is very similar to
> > Coreboot.
> > 
> > > (also please note that we currently only have amd64 hw to run our
> tests
> > > on.)
> > 
> > The problem is the same as Coreboot, which uses its own toolchain to
> > build images. We don't need to have native armhf builds for U-Boot,
> > testing with the armhf toolchain that is in Debian should be enough.
>
> I see.

There seem to be two solutions to this:
* Including a script (e.g. the one from coreboot) to build the toolchain
as part of the build process
* Using native builds with visualization on the servers that run the
builds for the reproducible task force

I tend to prefer the second one since it only requires a one-time setup
cost, while the other one, that requires to build toolchains for each
test build, implies a considerably longer build time for each test.

> > I understand, this works out nicely because all the work on Coreboot
> > will be inherited by Libreboot. However, on U-Boot, the work to
> bring
> > reproducible builds has to take place initially. I know for a fact
> that
> > parts of the code use things like __FILE__ or timestamps.
>
> Ah.

Not all the targets are using that though, and the target we used during
RMLL required only timestamp changes to become reproducible (it was the
Cubieboard2 IIRC). Still, I have seen that code around, so it must be
used somewhere, so it should be fixed, too.
 
> > > All this said, if you send me patches, I will probably deploy them
> as I'm
> > > very curious and more reproducibility efforts are good :-) We can
> can
> > > always decide to remove or move them later.
> > 
> > I wish to make all contributions upstream. What would really help at
> > first would be to have all targets built regularly to see where work
> is
> > needed. This is where I think the Debian infrastructure could help,
> in a
> > similar way as what was started for Coreboot.
>
> can you point me to a how to explaining this or tell me those steps,
> starting 
> with "git clone..."?

The basics for building U-Boot are the following (e.g. for the
Cubieboard2 target)
git clone git://git.denx.de/u-boot.git
cd u-boot
make -C $SRC O=$DST CROSS_COMPILE=$CROSS_COMPILE Cubieboard2_defconfig
make -C $SRC O=$DST CROSS_COMPILE=$CROSS_COMPILE

I usually also pass ARCH=$ARCH to make but that's useless, apparently.

Let me know if you need more indications on this.

-- 
Paul Kocialkowski, Replicant developer

Replicant is a fully free Android distribution running on several
devices, a free software mobile operating system putting the emphasis on
freedom and privacy/security.

Website: http://www.replicant.us/
Blog: http://blog.replicant.us/
Wiki/tracker/forums: http://redmine.replicant.us/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150719/78c0c713/attachment.sig>

[U-Boot] [Reproducible-builds] [PATCH] build: create time and date independent binary

[Holger Levsen]

Hi Paul,

On Sonntag, 19. Juli 2015, Paul Kocialkowski wrote:
> > sorry for the late reply.
> 
> No problem, I haven't kept you or the list posted on this either.
> 
> I got a chance to discuss the issue with Lunar during RMLL 2015 and we
> came up with a nice way of doing things, using SOURCE_DATE_EPOCH.

oh, nice!

> > > > you've seen https://reproducible.debian.net/u-boot ?
> As far as I understood, this was only for the U-Boot tools.

yes

 
> There seem to be two solutions to this:
> * Including a script (e.g. the one from coreboot) to build the toolchain
> as part of the build process
> * Using native builds with visualization on the servers that run the
> builds for the reproducible task force
> 
> I tend to prefer the second one since it only requires a one-time setup
> cost, while the other one, that requires to build toolchains for each
> test build, implies a considerably longer build time for each test.

these hosts also need maintenance so I actually prefer the first.
 
> The basics for building U-Boot are the following (e.g. for the
> Cubieboard2 target)
> git clone git://git.denx.de/u-boot.git
> cd u-boot
> make -C $SRC O=$DST CROSS_COMPILE=$CROSS_COMPILE Cubieboard2_defconfig
> make -C $SRC O=$DST CROSS_COMPILE=$CROSS_COMPILE

aint there a makefile with a proper target? Also how to build these cross 
compilers?
 
> Let me know if you need more indications on this.

yes, please. best a complete runnable script, aka a _working_ 5-10 liner, not 
the fancy one with variations, rebuilds and running debbindiff. Just the above 
"4 lines in working" - (those 4 lines dont work cause the variables are not 
defined...)


cheers,
	Holger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150719/03b3da11/attachment.sig>

[U-Boot] [Reproducible-builds] [PATCH] build: create time and date independent binary

[Paul Kocialkowski]

Le dimanche 19 juillet 2015 ? 17:47 +0200, Holger Levsen a ?crit :
> > There seem to be two solutions to this:
> > * Including a script (e.g. the one from coreboot) to build the toolchain
> > as part of the build process
> > * Using native builds with visualization on the servers that run the
> > builds for the reproducible task force
> > 
> > I tend to prefer the second one since it only requires a one-time setup
> > cost, while the other one, that requires to build toolchains for each
> > test build, implies a considerably longer build time for each test.
> 
> these hosts also need maintenance so I actually prefer the first.

I understand. We could raise the topic on the U-Boot mailing list and
see whether adding the scripts to build the cross-compilers tu U-Boot
would be doable or not.

Otherwise, we can just grab those scripts aside the U-Boot source code
for the reproducible task, so that we don't have to wait for a long
discussion to conclude.
 
> > The basics for building U-Boot are the following (e.g. for the
> > Cubieboard2 target)
> > git clone git://git.denx.de/u-boot.git
> > cd u-boot
> > make -C $SRC O=$DST CROSS_COMPILE=$CROSS_COMPILE Cubieboard2_defconfig
> > make -C $SRC O=$DST CROSS_COMPILE=$CROSS_COMPILE
> 
> aint there a makefile with a proper target? Also how to build these cross 
> compilers?

I am using those scripts, that I wrote:
http://git.paulk.fr/gitweb/?p=embedded-freedom-scripts.git;a=tree;f=u-boot;h=f735bc015b056c3b63a0144703d7f7168382ca59;hb=HEAD

That can help but is not really usable as-is.

> > Let me know if you need more indications on this.
> 
> yes, please. best a complete runnable script, aka a _working_ 5-10 liner, not 
> the fancy one with variations, rebuilds and running debbindiff. Just the above 
> "4 lines in working" - (those 4 lines dont work cause the variables are not 
> defined...)

I could do that, but we have to figure out the toolchain issue first, I
suppose.

-- 
Paul Kocialkowski, Replicant developer

Replicant is a fully free Android distribution running on several
devices, a free software mobile operating system putting the emphasis on
freedom and privacy/security.

Website: http://www.replicant.us/
Blog: http://blog.replicant.us/
Wiki/tracker/forums: http://redmine.replicant.us/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150719/76224846/attachment.sig>

[U-Boot] [PATCH] build: create time and date independent binary

[Heiko Schocher]

Hello Paul,

Am 12.06.2015 22:21, schrieb Paul Kocialkowski:
> Le vendredi 12 juin 2015 ? 17:31 +0200, Heiko Schocher a ?crit :
>> With this option enabled each compilation creates the exact
>> same binary. There are no timestamps, with which a U-Boot
>> binary can be identified.
>>
>> This option is disabled by default.
>
> I think a bit more is needed to get truly reproducible builds in U-Boot,
> but this is a first good step!
>
> I have been thinking about bringing reproducible builds to U-Boot for a
> little while, but never got around actually doing it. Are you interested
> in helping me on that? I have some free time these days, so I could do
> some work on this.

Hmm.. define what is a "reproducible build" ... my patch produces
always the same u-boot binary if you not change the toolchain ...

> It would certainly be relevant to get in touch with the people working
> on reproducing builds for Debian (I know Lunar is involved with this),
> since Debian packages U-Boot for many devices and they're apparently
> willing to help upstream with automated tests, as they're doing with
> coreboot and OpenWRT!

Yep, that would be helpful ...

bye,
Heiko
>
>> Signed-off-by: Heiko Schocher <hs@denx.de>
>> ---
>>
>>   Kconfig  | 9 +++++++++
>>   Makefile | 8 ++++++++
>>   2 files changed, 17 insertions(+)
>>
>> diff --git a/Kconfig b/Kconfig
>> index 15e15af..c0c7eb8 100644
>> --- a/Kconfig
>> +++ b/Kconfig
>> @@ -81,6 +81,15 @@ menuconfig EXPERT
>>   	  Only use this if you really know what you are doing.
>>
>>   if EXPERT
>> +	config SYS_EXACT_BINARY
>> +	bool "Create time and date independent binary"
>> +	default n
>> +	help
>> +	  With this option enabled each compilation creates the exact
>> +	  same binary. There are no timestamps, with which a U-Boot
>> +	  binary can be identified.
>> +	  This option is disabled by default.
>> +
>>   	config SYS_MALLOC_CLEAR_ON_INIT
>>   	bool "Init with zeros the memory reserved for malloc (slow)"
>>   	default y
>> diff --git a/Makefile b/Makefile
>> index 0a674bf..4ae2701 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -1230,11 +1230,19 @@ define filechk_version.h
>>   	echo \#define LD_VERSION_STRING \"$$($(LD) --version | head -n 1)\"; )
>>   endef
>>
>> +ifeq ($(CONFIG_SYS_EXACT_BINARY),y)
>> +define filechk_timestamp.h
>> +	(LC_ALL=C date +'#define U_BOOT_DATE "NODATE"'; \
>> +	LC_ALL=C date +'#define U_BOOT_TIME "NOTIME"'; \
>> +	LC_ALL=C date +'#define U_BOOT_TZ "NOTZ"')
>> +endef
>> +else
>>   define filechk_timestamp.h
>>   	(LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
>>   	LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
>>   	LC_ALL=C date +'#define U_BOOT_TZ "%z"')
>>   endef
>> +endif
>>
>>   $(version_h): include/config/uboot.release FORCE
>>   	$(call filechk,version.h)
>

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] [PATCH] build: create time and date independent binary

[Chris Kuethe]

On Fri, Jun 12, 2015 at 8:31 AM, Heiko Schocher <hs@denx.de> wrote:
> With this option enabled each compilation creates the exact
> same binary. There are no timestamps, with which a U-Boot
> binary can be identified.

Works on usbarmory.

Tested-by: Chris Kuethe <chris.kuethe@gmail.com>