Re: [OE-core] [PATCH] cve-check-update-nvd2-native: Incremement DL_DIR database location

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Fri, 2024-10-11 at 11:56 +0100, Richard Purdie via lists.openembedded.org wrote:
> We're seeing a lot of sqlite database corruption issues in our automated
> testing. It is unclear why this is happening. There were process
> imrpovements implemented in master and it is unclear if older releases
> are somehow making those changes ineffective or if the problem is
> elsewhere.
> 
> By changing the location in DL_DIR, we split the two sets of accesses
> to be separate and can isolate whether the master changes really did
> improve things or not. If successful, we may consider backporting those
> changes to the stable releases.
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/recipes-core/meta/cve-update-nvd2-native.bb | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb
> index 2d23d28c3e2..a7e568e3079 100644
> --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb
> +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb
> @@ -34,7 +34,7 @@ CVE_DB_INCR_UPDATE_AGE_THRES ?= "10368000"
>  # Number of attempts for each http query to nvd server before giving up
>  CVE_DB_UPDATE_ATTEMPTS ?= "5"
>  
> -CVE_CHECK_DB_DLDIR_FILE ?= "${DL_DIR}/CVE_CHECK/${CVE_CHECK_DB_FILENAME}"
> +CVE_CHECK_DB_DLDIR_FILE ?= "${DL_DIR}/CVE_CHECK2/${CVE_CHECK_DB_FILENAME}"
>  CVE_CHECK_DB_DLDIR_LOCK ?= "${CVE_CHECK_DB_DLDIR_FILE}.lock"
>  CVE_CHECK_DB_TEMP_FILE ?= "${CVE_CHECK_DB_FILE}.tmp"

I now do have an example of a corrupted database:

https://valkyrie.yocto.io/pub/shared-failure-data/nvdcve_2-1.db

I'm not sure why this is happening but the older releases are all
accessing the DB over NFS which makes me nervous. We're seeing a lot of
failures on the new cluster, hopefully this will help isolate the
problem.

Cheers,

Richard