Re: [PATCH 0/3] Add initial capability to check CVEs for recipes

[<Mikko.Rapeli@bmw.de>]

On Fri, Feb 26, 2016 at 08:48:47AM -0600, Mariano Lopez wrote:
> On 02/26/2016 02:14 AM, Mikko.Rapeli@bmw.de wrote:
> >Hi,
> >
> >On my developer machine the cve-check ran ok for dizzy but on build server
> >with sstate-cache and rmwork enabled it failed with what looks like a race
> >condition when scanning the patch files:
> >
> >17:45:36 ERROR: Error executing a python function in /home/builder/src/base/poky/meta/recipes-extended/mailx/mailx_12.5.bb:
> >17:45:36
> >17:45:36 The stack trace of python calls that resulted in this exception/failure was:
> >17:45:36 File: 'do_cve_check', lineno: 17, function: <module>
> >17:45:36      0013:    else:
> >17:45:36      0014:        bb.note("Failed to update CVE database, skipping CVE check")
> >17:45:36      0015:
> >17:45:36      0016:
> >17:45:36  *** 0017:do_cve_check(d)
> >17:45:36      0018:
> >17:45:37 File: 'do_cve_check', lineno: 8, function: do_cve_check
> >17:45:37      0004:    Check recipe for patched and unpatched CVEs
> >17:45:37      0005:    """
> >17:45:37      0006:
> >17:45:37      0007:    if os.path.exists(d.getVar("CVE_CHECK_TMP_FILE", True)):
> >17:45:37  *** 0008:        patched_cves = get_patches_cves(d)
> >17:45:37      0009:        patched, unpatched = check_cves(d, patched_cves)
> >17:45:37      0010:        if patched or unpatched:
> >17:45:37      0011:            cve_data = get_cve_info(d, patched + unpatched)
> >17:45:37      0012:            cve_write_data(d, patched, unpatched, cve_data)
> >17:45:37 File: 'cve-check.bbclass', lineno: 13, function: get_patches_cves
> >17:45:37      0009:    cve_match = re.compile("CVE:( CVE\-\d+\-\d+)+")
> >17:45:37      0010:    patched_cves = set()
> >17:45:37      0011:    for url in src_patches(d):
> >17:45:37      0012:        patch_file = bb.fetch.decodeurl(url)[2]
> >17:45:37  *** 0013:        with open(patch_file, "r") as f:
> >17:45:37      0014:            patch_text = f.read()
> >17:45:37      0015:
> >17:45:37      0016:        # Search for the "CVE: " line
> >17:45:37      0017:        match = cve_match.search(patch_text)
> >17:45:37 Exception: IOError: [Errno 2] No such file or directory: '/home/builder/src/base/build/tmp/work/corei7-64-linux/mailx/12.5-r2/heirloom-mailx_12.5-1.diff'
> >17:45:37
> >17:45:37 ERROR: Function failed: do_cve_check
> >
> >So could this be caused by cve-check changes or is this just a side effect
> >of some other recipe problems?
> >
> >I could not see that kind of fixes in master.
> >
> >-Mikko
> 
> The changes in patch series were minimal and actually this part of the code
> wasn't touched at all. That part of the code will look for all the files in
> the SRC_URI variable and will look for the "CVE:" tag in order to find
> patches that solve CVEs.

Yep, the code seems straight forward.

> It seems the problem is with the bitbake fetcher, or the recipe;
> unfortunately the fetcher is one of the components that most change between
> releases. Another thing to check is that if actually there is a
> heirloom-mailx_12.5-1.diff file in the paths that the fetcher look for. You
> can check this in the cve_check or patch log in the work directory of the
> recipe.

Unfortunately the file is there if I check with devshell but I have now
four different CI runs with this failure. Only difference to my developer
machine is sstate cache. Build machines maintain their own sstate cache.

-Mikko