[PATCH 0/1] image.bbclass: don't expand IMAGE_CMD when setting do_image task

[PATCH 0/1] image.bbclass: don't expand IMAGE_CMD when setting do_image task

[Matt Madison]

After being vexed with numerous taskhash mismatch errors on my
image builds, I think I hit on the source of the problem - the
value of the do_image_XXXX function reported by bitbake-dumpsig
had fully-expanded values for all of the variables referenced
in the function - so date and time stamps were showing up in
the text of the function.

Fixed this by not expanding the value of IMAGE_CMD during the
generation of the do_iamge_XXXX task.  The vardeps still do
their job for detecting relevant changes.

Matt Madison (1):
  image.bbclass: don't expand IMAGE_CMD when setting do_image task

 meta/classes/image.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
2.7.4

[PATCH 1/1] image.bbclass: don't expand IMAGE_CMD when setting do_image task

[Matt Madison]

Expanding it causes the do_image_x function to include values of
variables that may contain date/time stamps, rather than references
to the variable names, leading to spurious taskhash mismatch errors.

Signed-off-by: Matt Madison <matt@madison.systems>
---
 meta/classes/image.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 8bfd241..d695d30 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -380,7 +380,7 @@ python () {
         localdata.delVar('DATETIME')
         localdata.delVar('TMPDIR')
 
-        image_cmd = localdata.getVar("IMAGE_CMD", True)
+        image_cmd = localdata.getVar("IMAGE_CMD", False)
         vardeps.add('IMAGE_CMD_' + realt)
         if image_cmd:
             cmds.append("\t" + image_cmd)
-- 
2.7.4

Re: [PATCH 1/1] image.bbclass: don't expand IMAGE_CMD when setting do_image task

[Richard Purdie]

On Wed, 2016-04-27 at 05:32 -0700, Matt Madison wrote:
> Expanding it causes the do_image_x function to include values of
> variables that may contain date/time stamps, rather than references
> to the variable names, leading to spurious taskhash mismatch errors.
> 
> Signed-off-by: Matt Madison <matt@madison.systems>
> ---
>  meta/classes/image.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 8bfd241..d695d30 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -380,7 +380,7 @@ python () {
>          localdata.delVar('DATETIME')
>          localdata.delVar('TMPDIR')
>  
> -        image_cmd = localdata.getVar("IMAGE_CMD", True)
> +        image_cmd = localdata.getVar("IMAGE_CMD", False)
>          vardeps.add('IMAGE_CMD_' + realt)
>          if image_cmd:
>              cmds.append("\t" + image_cmd)

You can't do this since there are some variables that need to be
expanded using the values from the current localdata copy of the data
store (such as the type variable).

We'd previously tried to avoid DATETIME contanination by removing the
problem bases with the delVar calls just above this.

Cheers,

Richard

Re: [PATCH 1/1] image.bbclass: don't expand IMAGE_CMD when setting do_image task

[Matt Madison]

On Wed, Apr 27, 2016 at 6:03 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Wed, 2016-04-27 at 05:32 -0700, Matt Madison wrote:
>> Expanding it causes the do_image_x function to include values of
>> variables that may contain date/time stamps, rather than references
>> to the variable names, leading to spurious taskhash mismatch errors.
>>
>> Signed-off-by: Matt Madison <matt@madison.systems>
>> ---
>>  meta/classes/image.bbclass | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
>> index 8bfd241..d695d30 100644
>> --- a/meta/classes/image.bbclass
>> +++ b/meta/classes/image.bbclass
>> @@ -380,7 +380,7 @@ python () {
>>          localdata.delVar('DATETIME')
>>          localdata.delVar('TMPDIR')
>>
>> -        image_cmd = localdata.getVar("IMAGE_CMD", True)
>> +        image_cmd = localdata.getVar("IMAGE_CMD", False)
>>          vardeps.add('IMAGE_CMD_' + realt)
>>          if image_cmd:
>>              cmds.append("\t" + image_cmd)
>
> You can't do this since there are some variables that need to be
> expanded using the values from the current localdata copy of the data
> store (such as the type variable).
>
> We'd previously tried to avoid DATETIME contanination by removing the
> problem bases with the delVar calls just above this.

Rats.  That approach isn't working, at least for me - the recursive
expansion is causing the values of the date/time sensitive variables
to be present in the value of the IMAGE_CMD_type variable, so that
variable changes with each expansion.  I've hacked around the problem
by avoiding direct references to any variable that depends directly or
indirectly on ${DATETIME}, but I was really hoping for something
cleaner.

There are other variables that can cause this, too - ${DATE} and
${TIME} by themselves, for example, which aren't deleted.

Thanks,
-Matt

>
> Cheers,
>
> Richard

Re: [PATCH 1/1] image.bbclass: don't expand IMAGE_CMD when setting do_image task

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 295 bytes --]

On 27 April 2016 at 15:19, Matt Madison <matt@madison.systems> wrote:

> There are other variables that can cause this, too - ${DATE} and
> ${TIME} by themselves, for example, which aren't deleted.
>

If DATE and TIME were removed would the variables you're using stop
changing?

Ross

[-- Attachment #2: Type: text/html, Size: 689 bytes --]

Re: [PATCH 1/1] image.bbclass: don't expand IMAGE_CMD when setting do_image task

[Matt Madison]

On Wed, Apr 27, 2016 at 7:39 AM, Burton, Ross <ross.burton@intel.com> wrote:
>
> On 27 April 2016 at 15:19, Matt Madison <matt@madison.systems> wrote:
>>
>> There are other variables that can cause this, too - ${DATE} and
>> ${TIME} by themselves, for example, which aren't deleted.
>
>
> If DATE and TIME were removed would the variables you're using stop
> changing?

That would fix some of the issues, but not all of them.  The latest
case I ran into was in building an sdcard image (from
image_types_fsl.bbclass in meta-fsl-arm).  The first command in
IMAGE_CMD_sdcard is:

   if [ -z "${SDCARD_ROOTFS}" ]; then

SDCARD_ROOTFS is defined using ${IMAGE_NAME}, which in turn includes
${DATETIME}.  When I look at the bitbake-dumpsig output for the
do_image_sdcard task, the value of SDCARD_ROOTFS appears between the
quotation marks, rather than the variable reference in the value of
do_image_sdcard.

I've worked around it by moving the check to another function and
changing IMAGE_CMD_sdcard to call that function to do the test.

Thanks,
-Matt

>
> Ross