sstate cache SSTATE_PKG generation will silently fail, if hardlinking is not supported on the file system #bitbake

sstate cache SSTATE_PKG generation will silently fail, if hardlinking is not supported on the file system #bitbake

[Arnis]

[-- Attachment #1: Type: text/plain, Size: 805 bytes --]

Hi,

I have faced a problem today that sstate-cache `${SSTATE_PKG}` generation will silently fail on the file systems no supporting hard linking.
I've faced this problem on cloud-based Kubernetes cluster, where persistent volume is mounted to Docker containers as shared sstate-cache location.

Because of https://git.openembedded.org/ openembedded-core/commit/?id= 552197a0c4c9f75a9177c00b197ea9 1296ed9fc4 ( https://git.openembedded.org/openembedded-core/commit/?id=552197a0c4c9f75a9177c00b197ea91296ed9fc4 ) change

+ ln $TFILE ${SSTATE_PKG} || true

this will leave sstate folder with only `*...tar.zst.siginfo` files generated, but no "*...tar.zst"

As a temporary solution, I have replaced it with

+ cp $TFILE ${SSTATE_PKG}

which obviously is not ideal.

What would you suggest?

[-- Attachment #2: Type: text/html, Size: 2188 bytes --]

Re: [OE-core] sstate cache SSTATE_PKG generation will silently fail, if hardlinking is not supported on the file system #bitbake

[Richard Purdie]

On Thu, 2022-09-15 at 16:05 -0700, Arnis wrote:
> Hi, 
>  
> I have faced a problem today that sstate-cache `${SSTATE_PKG}`
> generation will silently fail on the file systems no supporting hard
> linking. 
> I've faced this problem on cloud-based Kubernetes cluster, where
> persistent volume is mounted to Docker containers as shared sstate-
> cache location.
>  
> Because of
> https://git.openembedded.org/openembedded-core/commit/?id=552197a0c4c9f75a9177c00b197ea91296ed9fc4
>  change 
>  
> + ln $TFILE ${SSTATE_PKG} || true
>  
> this will leave sstate folder with only `*...tar.zst.siginfo` files
> generated, but no "*...tar.zst" 
>  
> As a temporary solution, I have replaced it with
>  
> + cp $TFILE ${SSTATE_PKG}
>  
> which obviously is not ideal.
>  
> What would you suggest?

Unfortunately that code is rather critical as it is attempting to move
files into position atomically and not use lockfiles or hold locks.

Failing silently is bad, we should probably add a check the file exists
afterwards and error if it doens't.

That wouldn't solve your issue with that filesystem would make people
aware of it.

Cheers,

Richard