Re: [PATCH 0/3] Add initial capability to check CVEs for recipes

[<Mikko.Rapeli@bmw.de>]

On Fri, Feb 26, 2016 at 03:56:24PM +0100, Mikko Rapeli wrote:
> On Fri, Feb 26, 2016 at 08:48:47AM -0600, Mariano Lopez wrote:
> > On 02/26/2016 02:14 AM, Mikko.Rapeli@bmw.de wrote:
> > >Hi,
> > >
> > >On my developer machine the cve-check ran ok for dizzy but on build server
> > >with sstate-cache and rmwork enabled it failed with what looks like a race
> > >condition when scanning the patch files:
> > >
> > >17:45:36 ERROR: Error executing a python function in /home/builder/src/base/poky/meta/recipes-extended/mailx/mailx_12.5.bb:
> > >17:45:36
> > >17:45:36 The stack trace of python calls that resulted in this exception/failure was:
> > >17:45:36 File: 'do_cve_check', lineno: 17, function: <module>
> > >17:45:36      0013:    else:
> > >17:45:36      0014:        bb.note("Failed to update CVE database, skipping CVE check")
> > >17:45:36      0015:
> > >17:45:36      0016:
> > >17:45:36  *** 0017:do_cve_check(d)
> > >17:45:36      0018:
> > >17:45:37 File: 'do_cve_check', lineno: 8, function: do_cve_check
> > >17:45:37      0004:    Check recipe for patched and unpatched CVEs
> > >17:45:37      0005:    """
> > >17:45:37      0006:
> > >17:45:37      0007:    if os.path.exists(d.getVar("CVE_CHECK_TMP_FILE", True)):
> > >17:45:37  *** 0008:        patched_cves = get_patches_cves(d)
> > >17:45:37      0009:        patched, unpatched = check_cves(d, patched_cves)
> > >17:45:37      0010:        if patched or unpatched:
> > >17:45:37      0011:            cve_data = get_cve_info(d, patched + unpatched)
> > >17:45:37      0012:            cve_write_data(d, patched, unpatched, cve_data)
> > >17:45:37 File: 'cve-check.bbclass', lineno: 13, function: get_patches_cves
> > >17:45:37      0009:    cve_match = re.compile("CVE:( CVE\-\d+\-\d+)+")
> > >17:45:37      0010:    patched_cves = set()
> > >17:45:37      0011:    for url in src_patches(d):
> > >17:45:37      0012:        patch_file = bb.fetch.decodeurl(url)[2]
> > >17:45:37  *** 0013:        with open(patch_file, "r") as f:
> > >17:45:37      0014:            patch_text = f.read()
> > >17:45:37      0015:
> > >17:45:37      0016:        # Search for the "CVE: " line
> > >17:45:37      0017:        match = cve_match.search(patch_text)
> > >17:45:37 Exception: IOError: [Errno 2] No such file or directory: '/home/builder/src/base/build/tmp/work/corei7-64-linux/mailx/12.5-r2/heirloom-mailx_12.5-1.diff'
> > >17:45:37
> > >17:45:37 ERROR: Function failed: do_cve_check
> > >
> > >So could this be caused by cve-check changes or is this just a side effect
> > >of some other recipe problems?
> > >
> > >I could not see that kind of fixes in master.
> > >
> > >-Mikko
> > 
> > The changes in patch series were minimal and actually this part of the code
> > wasn't touched at all. That part of the code will look for all the files in
> > the SRC_URI variable and will look for the "CVE:" tag in order to find
> > patches that solve CVEs.
> 
> Yep, the code seems straight forward.
> 
> > It seems the problem is with the bitbake fetcher, or the recipe;
> > unfortunately the fetcher is one of the components that most change between
> > releases. Another thing to check is that if actually there is a
> > heirloom-mailx_12.5-1.diff file in the paths that the fetcher look for. You
> > can check this in the cve_check or patch log in the work directory of the
> > recipe.
> 
> Unfortunately the file is there if I check with devshell but I have now
> four different CI runs with this failure. Only difference to my developer
> machine is sstate cache. Build machines maintain their own sstate cache.

Last two runs were with v2 patches.

-Mikko