cannot build image using sstate

cannot build image using sstate

[Mircea Gliga]

Long story short: I have problems building an image, in a clean build 
directory, reusing the shared state cache and downloads from a previous 
build.
A file created in the do_deploy_append task is not created(restored) 
anymore when building using a previous sstate.

And now the long description:
In my custom layer, in a kernel recipe, linux-stable.bb, I have appended 
some operations to the `deploy` task, one of them is creating an U-Boot 
FIT image:

linux-stable.bb:
do_deploy_append() {
[...]
         #this line creates the image_signed.fit file
          mkimage  [...] image_signed.fit

[...]
}

Then in my image recipe, my-custom-image.bb, I'm creating an archive 
with some files, including the above created FIT image:

my-custom-image.bb:

LINUXIMAGE = "image_signed.fit"
do_makeArch() {
[...]
    tar --owner root --group root -C ${DEPLOY_DIR_IMAGE} -cvf 
${DEPLOY_DIR_IMAGE}/psf/${IMAGEFILE} -h ${LINUXIMAGE}
[...]
}

addtask do_makeArch after do_image_complete before do_build


Now, create a new build directory, set DL_DIR and SSTATE_DIR in local 
conf, start build:
# source oe-init-build-env build2
# bitbake my-custom-image
ERROR: my-custom-image-1.0-r0 do_makeArch: Function failed: do_makeArch
Log data follows:
| DEBUG: Executing shell function do_makeArch
| tar: image_signed.fit: Cannot stat: No such file or directory
| tar: Exiting with failure status due to previous errors

So the image build fails, because the image_signed.fit file is not present.
After some short time the build is started, in the deploy directory I 
see that the zimage is already there but the FIT image is missing ...

This is confirmed by the linux-stable.bb's logs: log.do_deploy_setscene. 
It lists all the files that are restored from the sstate cache, 
image_signed.fit
is not one of them:

DEBUG: Executing shell function sstate_unpack_package
deploy-linux-stable/
deploy-linux-stable/zImage-machine.bin
deploy-linux-stable/zImage--4.8.4+git0+a2b42342b2-r0.2-machine-20170308134915.bin
deploy-linux-stable/zImage-at91-machine.dtb
deploy-linux-stable/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
deploy-linux-stable/zImage--4.8.4+git0+a2b42342b2-r0.2-at91-machine-20170308134915.dtb
deploy-linux-stable/modules-machine.tgz
deploy-linux-stable/modules--4.8.4+git0+a2b42342b2-r0.2-machine-20170308134915.tgz
deploy-linux-stable/zImage
deploy-linux-stable/zImage-initramfs-4.8.4+gitAUTOINC+a2b42342b2-r0-machine-20170308134915.bin
deploy-linux-stable/zImage-initramfs-machine.bin
DEBUG: Shell function sstate_unpack_package finished


Any hints, on how to add my custom generated files to the sstate ? Or 
what am I doing wrong here ?
Thanks

Re: cannot build image using sstate

[Patrick Ohly]

On Thu, 2017-03-09 at 08:54 +0200, Mircea Gliga wrote:
> Long story short: I have problems building an image, in a clean build 
> directory, reusing the shared state cache and downloads from a previous 
> build.
> A file created in the do_deploy_append task is not created(restored) 
> anymore when building using a previous sstate.
> 
> And now the long description:
> In my custom layer, in a kernel recipe, linux-stable.bb, I have appended 
> some operations to the `deploy` task, one of them is creating an U-Boot 
> FIT image:
> 
> linux-stable.bb:
> do_deploy_append() {
> [...]
>          #this line creates the image_signed.fit file
>           mkimage  [...] image_signed.fit
> 
> [...]
> }

Are you writing image_signed.fit into the ${DEPLOYDIR} or
${DEPLOY_DIR_IMAGE}? When writing directly into ${DEPLOY_DIR_IMAGE}, you
bypass the mechanism which adds files to the sstate cache and then you
get exactly the problem you describe.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: cannot build image using sstate

[Mircea Gliga]

[-- Attachment #1: Type: text/plain, Size: 1873 bytes --]

Thanks! You are right, the do_deploy_append installs the 
image_signed.fit in the ${DEPLOY_DIR_IMAGE}(I should have kept that line 
in the previous mail also):
do_deploy_append() {
[...]
          #this line creates the image_signed.fit file
           mkimage  [...] image_signed.fit
           install -m 0644  image_signed.fit ${DEPLOY_DIR_IMAGE}/.
[...]
}

The doc mentions in regards to DEPLOYDIR:
"Recipes inheriting the |deploy| class should copy files to be deployed 
into |DEPLOYDIR|, and the class will take care of copying them into 
|DEPLOY_DIR_IMAGE| 
<http://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-DEPLOY_DIR_IMAGE> 
afterwards."

So I should just replace ${DEPLOY_DIR_IMAGE} with ${DEPLOYDIR} and I get 
the same behaviour as before + the benefit of sstate cache ?

Thanks


On 09/03/17 09:22, Patrick Ohly wrote:
> On Thu, 2017-03-09 at 08:54 +0200, Mircea Gliga wrote:
>> Long story short: I have problems building an image, in a clean build
>> directory, reusing the shared state cache and downloads from a previous
>> build.
>> A file created in the do_deploy_append task is not created(restored)
>> anymore when building using a previous sstate.
>>
>> And now the long description:
>> In my custom layer, in a kernel recipe, linux-stable.bb, I have appended
>> some operations to the `deploy` task, one of them is creating an U-Boot
>> FIT image:
>>
>> linux-stable.bb:
>> do_deploy_append() {
>> [...]
>>           #this line creates the image_signed.fit file
>>            mkimage  [...] image_signed.fit
>>
>> [...]
>> }
> Are you writing image_signed.fit into the ${DEPLOYDIR} or
> ${DEPLOY_DIR_IMAGE}? When writing directly into ${DEPLOY_DIR_IMAGE}, you
> bypass the mechanism which adds files to the sstate cache and then you
> get exactly the problem you describe.
>


[-- Attachment #2: Type: text/html, Size: 2590 bytes --]

Re: cannot build image using sstate

[Patrick Ohly]

On Thu, 2017-03-09 at 09:59 +0200, Mircea Gliga wrote:
> Thanks! You are right, the do_deploy_append installs the
> image_signed.fit in the ${DEPLOY_DIR_IMAGE}(I should have kept that
> line in the previous mail also):
> do_deploy_append() {
> [...]
>          #this line creates the image_signed.fit file
>           mkimage  [...] image_signed.fit
>           install -m 0644  image_signed.fit ${DEPLOY_DIR_IMAGE}/.
> [...]
> }
> 
> The doc mentions in regards to DEPLOYDIR:
> "Recipes inheriting the deploy class should copy files to be deployed
> into DEPLOYDIR, and the class will take care of copying them into
> DEPLOY_DIR_IMAGE afterwards."
> 
> So I should just replace ${DEPLOY_DIR_IMAGE} with ${DEPLOYDIR} and I
> get the same behaviour as before + the benefit of sstate cache ?

Yes.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.