[OE-core][PATCH 0/2] Fixes for reproducible build and externalsrc

[OE-core][PATCH 0/2] Fixes for reproducible build and externalsrc

[Mark Hatle]

Two fixes, the first one ensures that the BUILD_REPRODUCIBLE_BINARIES variable
actually does something useful.  1 (default) normal behavior, anything else it
is off and regular behavior works.

The second makes the reproducible build work properly with externalsrc.

To test this, I did:

bitbake bash
rpm -qp tmp/deploy/.../bash-...rpm --queryformat '[%{FILENAMES} %{FILEMTIMES}\n]'

cleanall the above, and then add:

BUILD_REPRODUCIBLE_BINARIES:pn-bash = '0'

repeat the test...

You can do the same with externalsrc via a bbappen adding externalsrc and repeat
the previous tests and get expected results.

Mark Hatle (2):
  reproducible_build: Honor BUILD_REPRODUCIBLE_BINARIES
  reproducible_build: Work with externalsrc

 meta/classes/reproducible_build.bbclass | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)

-- 
2.17.1

[OE-core][PATCH 1/2] reproducible_build: Honor BUILD_REPRODUCIBLE_BINARIES

[Mark Hatle]

From: Mark Hatle <mark.hatle@xilinx.com>

A variable BUILD_REPRODUCIBLE_BINARIES is set to '1' by default and was used
to control if the postfuncs were added.  However, it didn't actually disable
the reproducible_build (date) logic.  This resulted in the program falling
back to the default date.

This change fully honors the variable and disables the various pieces
that discover and configure the SOURCE_DATE_EPOCH if the value if not
set to '1'.

Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
---
 meta/classes/reproducible_build.bbclass | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
index 378121903d..a9c117c3b9 100644
--- a/meta/classes/reproducible_build.bbclass
+++ b/meta/classes/reproducible_build.bbclass
@@ -47,7 +47,9 @@ TARGET_CC_ARCH:append:class-target = " -Wdate-time"
 # A SOURCE_DATE_EPOCH of '0' might be misinterpreted as no SDE
 export SOURCE_DATE_EPOCH_FALLBACK ??= "1302044400"
 
-SSTATETASKS += "do_deploy_source_date_epoch"
+# The following are preformed in the anonymous python function, only if
+# BUILD_REPRODUCIBLE_BINARIES == 1
+#SSTATETASKS += "do_deploy_source_date_epoch"
 
 do_deploy_source_date_epoch () {
     mkdir -p ${SDE_DEPLOYDIR}
@@ -73,8 +75,10 @@ python do_deploy_source_date_epoch_setscene () {
 
 do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
 do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}"
-addtask do_deploy_source_date_epoch_setscene
-addtask do_deploy_source_date_epoch before do_configure after do_patch
+# The following are preformed in the anonymous python function, only if
+# BUILD_REPRODUCIBLE_BINARIES == 1
+#addtask do_deploy_source_date_epoch_setscene
+#addtask do_deploy_source_date_epoch before do_configure after do_patch
 
 python create_source_date_epoch_stamp() {
     import oe.reproducible
@@ -123,5 +127,12 @@ BB_HASHBASE_WHITELIST += "SOURCE_DATE_EPOCH"
 
 python () {
     if d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1':
+        # Generate the timestamp with create_source_date_epoch_stamp.
         d.appendVarFlag("do_unpack", "postfuncs", " create_source_date_epoch_stamp")
+        d.appendVar('SSTATETASKS', " do_deploy_source_date_epoch")
+        bb.build.addtask('do_deploy_source_date_epoch_setscene', None, None, d)
+        bb.build.addtask('do_deploy_source_date_epoch', 'do_configure', 'do_patch', d)
+    else:
+        # If this is set at all, the system components will attempt to use it
+        d.delVar('SOURCE_DATE_EPOCH')
 }
-- 
2.17.1

Re: [OE-core][PATCH 1/2] reproducible_build: Honor BUILD_REPRODUCIBLE_BINARIES

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-
> core@lists.openembedded.org> On Behalf Of Mark Hatle
> Sent: den 9 september 2021 04:51
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core][PATCH 1/2] reproducible_build: Honor
> BUILD_REPRODUCIBLE_BINARIES
> 
> From: Mark Hatle <mark.hatle@xilinx.com>
> 
> A variable BUILD_REPRODUCIBLE_BINARIES is set to '1' by default and was> used
> to control if the postfuncs were added.  However, it didn't actually disable
> the reproducible_build (date) logic.  This resulted in the program falling
> back to the default date.
> 
> This change fully honors the variable and disables the various pieces
> that discover and configure the SOURCE_DATE_EPOCH if the value if not
> set to '1'.
> 
> Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
> ---
>  meta/classes/reproducible_build.bbclass | 17 ++++++++++++++---
>  1 file changed, 14 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/classes/reproducible_build.bbclass
> b/meta/classes/reproducible_build.bbclass
> index 378121903d..a9c117c3b9 100644
> --- a/meta/classes/reproducible_build.bbclass
> +++ b/meta/classes/reproducible_build.bbclass
> @@ -47,7 +47,9 @@ TARGET_CC_ARCH:append:class-target = " -Wdate-time"
>  # A SOURCE_DATE_EPOCH of '0' might be misinterpreted as no SDE
>  export SOURCE_DATE_EPOCH_FALLBACK ??= "1302044400"
> 
> -SSTATETASKS += "do_deploy_source_date_epoch"
> +# The following are preformed in the anonymous python function, only if

Change "are preformed" to "is performed".

> +# BUILD_REPRODUCIBLE_BINARIES == 1
> +#SSTATETASKS += "do_deploy_source_date_epoch"
> 
>  do_deploy_source_date_epoch () {
>      mkdir -p ${SDE_DEPLOYDIR}
> @@ -73,8 +75,10 @@ python do_deploy_source_date_epoch_setscene () {
> 
>  do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
>  do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}"
> -addtask do_deploy_source_date_epoch_setscene
> -addtask do_deploy_source_date_epoch before do_configure after do_patch
> +# The following are preformed in the anonymous python function, only if

Change "preformed" to "performed".

> +# BUILD_REPRODUCIBLE_BINARIES == 1
> +#addtask do_deploy_source_date_epoch_setscene
> +#addtask do_deploy_source_date_epoch before do_configure after do_patch

Alternatively remove the two comments with the moved code since there 
is a large risk that these comments won't be updated if the code below 
is modified and eventually they will be out of sync.

>  python create_source_date_epoch_stamp() {
>      import oe.reproducible
> @@ -123,5 +127,12 @@ BB_HASHBASE_WHITELIST += "SOURCE_DATE_EPOCH"
> 
>  python () {
>      if d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1':
> +        # Generate the timestamp with create_source_date_epoch_stamp.
>          d.appendVarFlag("do_unpack", "postfuncs", " create_source_date_epoch_stamp")
> +        d.appendVar('SSTATETASKS', " do_deploy_source_date_epoch")
> +        bb.build.addtask('do_deploy_source_date_epoch_setscene', None, None, d)
> +        bb.build.addtask('do_deploy_source_date_epoch', 'do_configure', 'do_patch', d)
> +    else:
> +        # If this is set at all, the system components will attempt to use it
> +        d.delVar('SOURCE_DATE_EPOCH')
>  }
> --
> 2.17.1

//Peter

Re: [OE-core][PATCH 1/2] reproducible_build: Honor BUILD_REPRODUCIBLE_BINARIES

[Richard Purdie]

On Wed, 2021-09-08 at 21:50 -0500, Mark Hatle wrote:
> From: Mark Hatle <mark.hatle@xilinx.com>
> 
> A variable BUILD_REPRODUCIBLE_BINARIES is set to '1' by default and was used
> to control if the postfuncs were added.  However, it didn't actually disable
> the reproducible_build (date) logic.  This resulted in the program falling
> back to the default date.
> 
> This change fully honors the variable and disables the various pieces
> that discover and configure the SOURCE_DATE_EPOCH if the value if not
> set to '1'.
> 
> Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
> ---
>  meta/classes/reproducible_build.bbclass | 17 ++++++++++++++---
>  1 file changed, 14 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
> index 378121903d..a9c117c3b9 100644
> --- a/meta/classes/reproducible_build.bbclass
> +++ b/meta/classes/reproducible_build.bbclass
> @@ -47,7 +47,9 @@ TARGET_CC_ARCH:append:class-target = " -Wdate-time"
>  # A SOURCE_DATE_EPOCH of '0' might be misinterpreted as no SDE
>  export SOURCE_DATE_EPOCH_FALLBACK ??= "1302044400"
>  
> -SSTATETASKS += "do_deploy_source_date_epoch"
> +# The following are preformed in the anonymous python function, only if
> +# BUILD_REPRODUCIBLE_BINARIES == 1
> +#SSTATETASKS += "do_deploy_source_date_epoch"
>  
>  do_deploy_source_date_epoch () {
>      mkdir -p ${SDE_DEPLOYDIR}
> @@ -73,8 +75,10 @@ python do_deploy_source_date_epoch_setscene () {
>  
>  do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
>  do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}"
> -addtask do_deploy_source_date_epoch_setscene
> -addtask do_deploy_source_date_epoch before do_configure after do_patch
> +# The following are preformed in the anonymous python function, only if
> +# BUILD_REPRODUCIBLE_BINARIES == 1
> +#addtask do_deploy_source_date_epoch_setscene
> +#addtask do_deploy_source_date_epoch before do_configure after do_patch
>  
>  python create_source_date_epoch_stamp() {
>      import oe.reproducible
> @@ -123,5 +127,12 @@ BB_HASHBASE_WHITELIST += "SOURCE_DATE_EPOCH"
>  
>  python () {
>      if d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1':
> +        # Generate the timestamp with create_source_date_epoch_stamp.
>          d.appendVarFlag("do_unpack", "postfuncs", " create_source_date_epoch_stamp")
> +        d.appendVar('SSTATETASKS', " do_deploy_source_date_epoch")
> +        bb.build.addtask('do_deploy_source_date_epoch_setscene', None, None, d)
> +        bb.build.addtask('do_deploy_source_date_epoch', 'do_configure', 'do_patch', d)
> +    else:
> +        # If this is set at all, the system components will attempt to use it
> +        d.delVar('SOURCE_DATE_EPOCH')
>  }

Doing this makes me rather sad. Why? If we keep going down this route, all our
code will just become a mess of python rather than metadata.

A better question here is probably whether we can keep the tasks but correctly
disable changing of the date instead?

I really don't want to just pile up anonymous python.

FWIW I'm also leaning towards moving some of the reproducible code into the main
classes/tasks out the box as I'm not sure we're that interested in having non-
reproducible builds.

Cheers,

Richard

Re: [OE-core][PATCH 1/2] reproducible_build: Honor BUILD_REPRODUCIBLE_BINARIES

[Douglas]

[-- Attachment #1: Type: text/plain, Size: 4741 bytes --]

On 9/09/21 10:18 pm, Richard Purdie wrote:

> On Wed, 2021-09-08 at 21:50 -0500, Mark Hatle wrote:
>> From: Mark Hatle <mark.hatle@xilinx.com>
>>
>> A variable BUILD_REPRODUCIBLE_BINARIES is set to '1' by default and was used
>> to control if the postfuncs were added.  However, it didn't actually disable
>> the reproducible_build (date) logic.  This resulted in the program falling
>> back to the default date.
>>
>> This change fully honors the variable and disables the various pieces
>> that discover and configure the SOURCE_DATE_EPOCH if the value if not
>> set to '1'.
>>
>> Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
>> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
>> ---
>>   meta/classes/reproducible_build.bbclass | 17 ++++++++++++++---
>>   1 file changed, 14 insertions(+), 3 deletions(-)
>>
>> diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
>> index 378121903d..a9c117c3b9 100644
>> --- a/meta/classes/reproducible_build.bbclass
>> +++ b/meta/classes/reproducible_build.bbclass
>> @@ -47,7 +47,9 @@ TARGET_CC_ARCH:append:class-target = " -Wdate-time"
>>   # A SOURCE_DATE_EPOCH of '0' might be misinterpreted as no SDE
>>   export SOURCE_DATE_EPOCH_FALLBACK ??= "1302044400"
>>   
>> -SSTATETASKS += "do_deploy_source_date_epoch"
>> +# The following are preformed in the anonymous python function, only if
>> +# BUILD_REPRODUCIBLE_BINARIES == 1
>> +#SSTATETASKS += "do_deploy_source_date_epoch"
>>   
>>   do_deploy_source_date_epoch () {
>>       mkdir -p ${SDE_DEPLOYDIR}
>> @@ -73,8 +75,10 @@ python do_deploy_source_date_epoch_setscene () {
>>   
>>   do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
>>   do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}"
>> -addtask do_deploy_source_date_epoch_setscene
>> -addtask do_deploy_source_date_epoch before do_configure after do_patch
>> +# The following are preformed in the anonymous python function, only if
>> +# BUILD_REPRODUCIBLE_BINARIES == 1
>> +#addtask do_deploy_source_date_epoch_setscene
>> +#addtask do_deploy_source_date_epoch before do_configure after do_patch
>>   
>>   python create_source_date_epoch_stamp() {
>>       import oe.reproducible
>> @@ -123,5 +127,12 @@ BB_HASHBASE_WHITELIST += "SOURCE_DATE_EPOCH"
>>   
>>   python () {
>>       if d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1':
>> +        # Generate the timestamp with create_source_date_epoch_stamp.
>>           d.appendVarFlag("do_unpack", "postfuncs", " create_source_date_epoch_stamp")
>> +        d.appendVar('SSTATETASKS', " do_deploy_source_date_epoch")
>> +        bb.build.addtask('do_deploy_source_date_epoch_setscene', None, None, d)
>> +        bb.build.addtask('do_deploy_source_date_epoch', 'do_configure', 'do_patch', d)
>> +    else:
>> +        # If this is set at all, the system components will attempt to use it
>> +        d.delVar('SOURCE_DATE_EPOCH')
>>   }
> Doing this makes me rather sad. Why? If we keep going down this route, all our
> code will just become a mess of python rather than metadata.
>
> A better question here is probably whether we can keep the tasks but correctly
> disable changing of the date instead?
>
> I really don't want to just pile up anonymous python.
>
> FWIW I'm also leaning towards moving some of the reproducible code into the main
> classes/tasks out the box as I'm not sure we're that interested in having non-
> reproducible builds.


The do_deploy_source_date_epoch task is pretty cheap. You're going to 
some trouble here to turn it off, when it only applies to a distro that 
has chosen to inherit reproducible_build in the first place.

I've always been a bit puzzled by the intention of 
BUILD_REPRODUCIBLE_BINARIES, given that it's a distro choice to inherit 
reproducible_build at all. Perhaps the original intention was to avoid 
the get_source_date_epoch() python functionality in the special case of 
a problematic recipe? Are you there, Juro? Can you cast any light on this?

If you really, really do need to turn off the SOURCE_DATE_EPOCH 
variable, perhaps a cleaner approach would be to set it only from within 
the python anonymous function (rather than deleting it in the negative 
case). In this way, this entire patch could perhaps be reduced to one 
line. Note that SOURCE_DATE_EPOCH is already exported by 
reproducible_build_simple.bbclass — which is already inherited only if 
BUILD_REPRODUCIBLE_BINARIES == 1.

I think this all achieves your aim: If your distro inherits 
reproducible_build, but you turn off BUILD_REPRODUCIBLE_BINARIES for one 
recipe (or in your local.conf), the SOURCE_DATE_EPOCH won't be set.


[-- Attachment #2: Type: text/html, Size: 5435 bytes --]

[OE-core][PATCH 2/2] reproducible_build: Work with externalsrc

[Mark Hatle]

From: Mark Hatle <mark.hatle@xilinx.com>

Externalsrc removes do_fetch, do_unpack, and do_patch.  The system normally
discovers the correct reproducible date as a postfuncs on do_unpack, so this
date is never found, so it goes back to the default epoch.

Instead we can move the discovery function to a prefuncs on the epoch
deploy task.  This task will run before do_configure, and since the source
is already available can run anytime safely.

Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
---
 meta/classes/reproducible_build.bbclass | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
index a9c117c3b9..ae0723ab21 100644
--- a/meta/classes/reproducible_build.bbclass
+++ b/meta/classes/reproducible_build.bbclass
@@ -128,7 +128,13 @@ BB_HASHBASE_WHITELIST += "SOURCE_DATE_EPOCH"
 python () {
     if d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1':
         # Generate the timestamp with create_source_date_epoch_stamp.
-        d.appendVarFlag("do_unpack", "postfuncs", " create_source_date_epoch_stamp")
+        # In most cases this will be a postfuncs of do_unpack.
+        # If we're running in external source, add create_source_date_epoch_stamp
+        # to do_deploy_source_date_epoch instead because there is no do_unpack.
+        if d.getVar('EXTERNALSRC') and bb.data.inherits_class('externalsrc', d):
+            d.appendVarFlag('do_deploy_source_date_epoch', 'prefuncs', ' create_source_date_epoch_stamp')
+        else:
+            d.appendVarFlag("do_unpack", "postfuncs", " create_source_date_epoch_stamp")
         d.appendVar('SSTATETASKS', " do_deploy_source_date_epoch")
         bb.build.addtask('do_deploy_source_date_epoch_setscene', None, None, d)
         bb.build.addtask('do_deploy_source_date_epoch', 'do_configure', 'do_patch', d)
-- 
2.17.1

Re: [OE-core][PATCH 2/2] reproducible_build: Work with externalsrc

[Douglas]

[-- Attachment #1: Type: text/plain, Size: 2766 bytes --]

On 9/09/21 2:51 pm, Mark Hatle wrote:
> From: Mark Hatle <mark.hatle@xilinx.com>
>
> Externalsrc removes do_fetch, do_unpack, and do_patch.  The system normally
> discovers the correct reproducible date as a postfuncs on do_unpack, so this
> date is never found, so it goes back to the default epoch.
>
> Instead we can move the discovery function to a prefuncs on the epoch
> deploy task.  This task will run before do_configure, and since the source
> is already available can run anytime safely.
>
> Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
> ---
>   meta/classes/reproducible_build.bbclass | 8 +++++++-
>   1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
> index a9c117c3b9..ae0723ab21 100644
> --- a/meta/classes/reproducible_build.bbclass
> +++ b/meta/classes/reproducible_build.bbclass
> @@ -128,7 +128,13 @@ BB_HASHBASE_WHITELIST += "SOURCE_DATE_EPOCH"
>   python () {
>       if d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1':
>           # Generate the timestamp with create_source_date_epoch_stamp.
> -        d.appendVarFlag("do_unpack", "postfuncs", " create_source_date_epoch_stamp")
> +        # In most cases this will be a postfuncs of do_unpack.
> +        # If we're running in external source, add create_source_date_epoch_stamp
> +        # to do_deploy_source_date_epoch instead because there is no do_unpack.
> +        if d.getVar('EXTERNALSRC') and bb.data.inherits_class('externalsrc', d):
> +            d.appendVarFlag('do_deploy_source_date_epoch', 'prefuncs', ' create_source_date_epoch_stamp')
> +        else:
> +            d.appendVarFlag("do_unpack", "postfuncs", " create_source_date_epoch_stamp")
>           d.appendVar('SSTATETASKS', " do_deploy_source_date_epoch")
>           bb.build.addtask('do_deploy_source_date_epoch_setscene', None, None, d)
>           bb.build.addtask('do_deploy_source_date_epoch', 'do_configure', 'do_patch', d)


Why was create_source_date_epoch_stamp ever added as a do_unpack 
postfunc in the first place? Why wasn't it invoked directly from 
do_deploy_source_date_epoch all along? This task runs after do_patch and 
before do_configure anyway. What am I missing? Are you there, Juro?

If we create_source_date_epoch_stamp (either as a 
do_deploy_source_date_epoch prefunc or directly invoked), then it should 
work either way, for externalsrc or not.

As an aside, perhaps do_deploy_source_date_epoch should come after 
do_unpack as well, just in case someone has deleted do_patch:

    addtask do_deploy_source_date_epoch before do_configure after
    do_unpack do_patch


[-- Attachment #2: Type: text/html, Size: 3436 bytes --]

Re: [OE-core][PATCH 2/2] reproducible_build: Work with externalsrc

[Douglas]

[-- Attachment #1: Type: text/plain, Size: 4002 bytes --]

On 10/09/21 1:05 pm, Mark Hatle wrote:
> Unfortunately I'm having problems sending email to the list currently.  But I
> can reply directly (feels free to forward this to the list).
>
> On 9/9/21 7:32 PM, Douglas Royds wrote:
>> On 9/09/21 2:51 pm, Mark Hatle wrote:
>>> From: Mark Hatle <mark.hatle@xilinx.com>
>>>
>>> Externalsrc removes do_fetch, do_unpack, and do_patch.  The system normally
>>> discovers the correct reproducible date as a postfuncs on do_unpack, so this
>>> date is never found, so it goes back to the default epoch.
>>>
>>> Instead we can move the discovery function to a prefuncs on the epoch
>>> deploy task.  This task will run before do_configure, and since the source
>>> is already available can run anytime safely.
>>>
>>> Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
>>> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
>>> ---
>>>   meta/classes/reproducible_build.bbclass | 8 +++++++-
>>>   1 file changed, 7 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
>>> index a9c117c3b9..ae0723ab21 100644
>>> --- a/meta/classes/reproducible_build.bbclass
>>> +++ b/meta/classes/reproducible_build.bbclass
>>> @@ -128,7 +128,13 @@ BB_HASHBASE_WHITELIST += "SOURCE_DATE_EPOCH"
>>>   python () {
>>>       if d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1':
>>>           # Generate the timestamp with create_source_date_epoch_stamp.
>>> -        d.appendVarFlag("do_unpack", "postfuncs", " create_source_date_epoch_stamp")
>>> +        # In most cases this will be a postfuncs of do_unpack.
>>> +        # If we're running in external source, add create_source_date_epoch_stamp
>>> +        # to do_deploy_source_date_epoch instead because there is no do_unpack.
>>> +        if d.getVar('EXTERNALSRC') and bb.data.inherits_class('externalsrc', d):
>>> +            d.appendVarFlag('do_deploy_source_date_epoch', 'prefuncs', ' create_source_date_epoch_stamp')
>>> +        else:
>>> +            d.appendVarFlag("do_unpack", "postfuncs", " create_source_date_epoch_stamp")
>>>           d.appendVar('SSTATETASKS', " do_deploy_source_date_epoch")
>>>           bb.build.addtask('do_deploy_source_date_epoch_setscene', None, None, d)
>>>           bb.build.addtask('do_deploy_source_date_epoch', 'do_configure', 'do_patch', d)
>>
>> Why was create_source_date_epoch_stamp ever added as a do_unpack postfunc in the
>> first place? Why wasn't it invoked directly from do_deploy_source_date_epoch all
>> along? This task runs after do_patch and before do_configure anyway. What am I
>> missing? Are you there, Juro?
> It was put in that place to ensure it was based on the original upstream source,
> and not the patched source.  As the act of patching the source may change the
> dates based on the behavior of patch, filesystem, etc.
>
> So really we do want the date/time from the do_unpack, and before do_patch (or
> any random tasks added between do_unpack and do_patch run.)


True. Good point.


>> If we create_source_date_epoch_stamp (either as a do_deploy_source_date_epoch
>> prefunc or directly invoked), then it should work either way, for externalsrc or
>> not.
> I've got a new set of patches coming shortly.  (I can send patches, but can't
> reply to the list currently.  I've got a configuration problem on my ISP side
> which will hopefully be fixed tomorrow.)
>
>> As an aside, perhaps do_deploy_source_date_epoch should come after do_unpack as
>> well, just in case someone has deleted do_patch:
>>
>>      addtask do_deploy_source_date_epoch before do_configure after do_unpack do_patch
>>
> The new set will keep things as is, removing the confusing
> BUILD_REPRODUCIBLE_BINARIES switching in this class, while moving the special
> behavior to the externalsrc class.  So if we remove the do_unpack, we'll move
> the function to the prefuncs of the do_deploy_source_date_epoch.


Nice.


[-- Attachment #2: Type: text/html, Size: 5221 bytes --]