Re: [OE-core] severe issue in CVE checker

Re: [OE-core] severe issue in CVE checker

[Alexander Kanavin]

Thanks for the information, can you send a patch?

Alex

On Wed, 12 Oct 2022 at 09:25, Alberto Pianon <alberto@pianon.eu> wrote:
>
> All, Marta, Richard,
>
> while implementing stats aggregation for CVE metadata in the Oniro
> project, I encountered a severe issue in Yocto's CVE checker, apparently
> due to this:
> https://git.yoctoproject.org/poky/tree/meta/recipes-core/meta/cve-update-db-native.bb#n81
>
> It appears that when cve-update-db-native fails to fetch some years of
> NIST CVE db, it just issues a warning but goes on anyway.
>
> The result is that in some builds, randomly (depending on NIST webserver
> timeouts or other connection problems), CVE db is not complete, so the
> CVE check returns false negatives (i.e. no vulnerabilities found in some
> components even if such vulnerabilities do exist)
>
> I ran into such problem because in Oniro we need aggregate data from
> different builds for a large target matrix; I added a check to check
> that CVE metadata for each component are the same in all builds, and it
> failed, so I tried to figure out the cause and I found this:
>
> - in a build where cve-check found a vulnerability for acl:
>      $ sqlite3 build-linux-clang-qemuarm-efi/tmp/CVE_CHECK/nvdcve_1.1.db
>      sqlite> SELECT DISTINCT ID FROM PRODUCTS WHERE PRODUCT IS "acl";
>      CVE-2009-4411
>      sqlite>
>
> - in another build where cve-check did not found any vulnerability for
> the very same version of acl:
>      $ sqlite3 build-linux-clang-qemux86/tmp/CVE_CHECK/nvdcve_1.1.db
>      sqlite> SELECT DISTINCT ID FROM PRODUCTS WHERE PRODUCT IS "acl";
>      sqlite>
>
> so I listed both CVE db files in those two builds and this is what I
> got:
>
>      $ ls -ll build-linux-clang-qemuarm-efi/tmp/CVE_CHECK/nvdcve_1.1.db
> build-linux-clang-qemux86/tmp/CVE_CHECK/nvdcve_1.1.db
>      -rw-r--r-- 1 ubuntu ubuntu 215093248 Oct 11 10:56
> build-linux-clang-qemuarm-efi/tmp/CVE_CHECK/nvdcve_1.1.db
>      -rw-r--r-- 1 ubuntu ubuntu     28672 Oct 11 10:00
> build-linux-clang-qemux86/tmp/CVE_CHECK/nvdcve_1.1.db
>
> The two CVE db files were fetched just about 1h apart, but the second
> file is apparently incomplete.
>
> I checked the log for the second build, and I found this:
>
>      WARNING: cve-update-db-native-1.0-r0 do_fetch: Failed to fetch CVE
> data ([Errno 99] Cannot assign requested address)
>      NOTE: recipe cve-update-db-native-1.0-r0: task do_fetch: Succeeded
>
> Fetch fails, but do_fetch task succeeds.
>
> So I looked into the recipe's code, and I found this:
> https://git.yoctoproject.org/poky/tree/meta/recipes-core/meta/cve-update-db-native.bb#n81
>
> It iterates over NIST CVE db years, but if some year fail to download,
> it goes on anyway, and it still merges the successful downloads into
> nvdcve_1.1.db, without returning error.
>
> IMHO this is a severe issue because it may silently lead to false
> negatives in the CVE check. If some downloads fail due to timeouts or
> other connection problems, cve-check should retry a number of times, and
> if any download still fails, cve-update-db-native do_fecth should fail,
> and it turn all do_cve_check tasks should fail, since doing a CVE check
> against a corrupted/incomplete CVE database is clearly useless
>
> Regards,
>
> Alberto
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#171653): https://lists.openembedded.org/g/openembedded-core/message/171653
> Mute This Topic: https://lists.openembedded.org/mt/94276393/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] severe issue in CVE checker

[Marta Rybczynska]

[-- Attachment #1: Type: text/plain, Size: 4228 bytes --]

I'll be looking into how to fix it. My current idea is to make the download
a transaction, so do not update the database until we're sure the download
is complete. Plus a warning that we have had an issue, I think.

In a next step we can retry a number of times.

Regards
Marta

On Wed, 12 Oct 2022, 16:50 Alexander Kanavin, <alex.kanavin@gmail.com>
wrote:

> Thanks for the information, can you send a patch?
>
> Alex
>
> On Wed, 12 Oct 2022 at 09:25, Alberto Pianon <alberto@pianon.eu> wrote:
> >
> > All, Marta, Richard,
> >
> > while implementing stats aggregation for CVE metadata in the Oniro
> > project, I encountered a severe issue in Yocto's CVE checker, apparently
> > due to this:
> >
> https://git.yoctoproject.org/poky/tree/meta/recipes-core/meta/cve-update-db-native.bb#n81
> >
> > It appears that when cve-update-db-native fails to fetch some years of
> > NIST CVE db, it just issues a warning but goes on anyway.
> >
> > The result is that in some builds, randomly (depending on NIST webserver
> > timeouts or other connection problems), CVE db is not complete, so the
> > CVE check returns false negatives (i.e. no vulnerabilities found in some
> > components even if such vulnerabilities do exist)
> >
> > I ran into such problem because in Oniro we need aggregate data from
> > different builds for a large target matrix; I added a check to check
> > that CVE metadata for each component are the same in all builds, and it
> > failed, so I tried to figure out the cause and I found this:
> >
> > - in a build where cve-check found a vulnerability for acl:
> >      $ sqlite3 build-linux-clang-qemuarm-efi/tmp/CVE_CHECK/nvdcve_1.1.db
> >      sqlite> SELECT DISTINCT ID FROM PRODUCTS WHERE PRODUCT IS "acl";
> >      CVE-2009-4411
> >      sqlite>
> >
> > - in another build where cve-check did not found any vulnerability for
> > the very same version of acl:
> >      $ sqlite3 build-linux-clang-qemux86/tmp/CVE_CHECK/nvdcve_1.1.db
> >      sqlite> SELECT DISTINCT ID FROM PRODUCTS WHERE PRODUCT IS "acl";
> >      sqlite>
> >
> > so I listed both CVE db files in those two builds and this is what I
> > got:
> >
> >      $ ls -ll build-linux-clang-qemuarm-efi/tmp/CVE_CHECK/nvdcve_1.1.db
> > build-linux-clang-qemux86/tmp/CVE_CHECK/nvdcve_1.1.db
> >      -rw-r--r-- 1 ubuntu ubuntu 215093248 Oct 11 10:56
> > build-linux-clang-qemuarm-efi/tmp/CVE_CHECK/nvdcve_1.1.db
> >      -rw-r--r-- 1 ubuntu ubuntu     28672 Oct 11 10:00
> > build-linux-clang-qemux86/tmp/CVE_CHECK/nvdcve_1.1.db
> >
> > The two CVE db files were fetched just about 1h apart, but the second
> > file is apparently incomplete.
> >
> > I checked the log for the second build, and I found this:
> >
> >      WARNING: cve-update-db-native-1.0-r0 do_fetch: Failed to fetch CVE
> > data ([Errno 99] Cannot assign requested address)
> >      NOTE: recipe cve-update-db-native-1.0-r0: task do_fetch: Succeeded
> >
> > Fetch fails, but do_fetch task succeeds.
> >
> > So I looked into the recipe's code, and I found this:
> >
> https://git.yoctoproject.org/poky/tree/meta/recipes-core/meta/cve-update-db-native.bb#n81
> >
> > It iterates over NIST CVE db years, but if some year fail to download,
> > it goes on anyway, and it still merges the successful downloads into
> > nvdcve_1.1.db, without returning error.
> >
> > IMHO this is a severe issue because it may silently lead to false
> > negatives in the CVE check. If some downloads fail due to timeouts or
> > other connection problems, cve-check should retry a number of times, and
> > if any download still fails, cve-update-db-native do_fecth should fail,
> > and it turn all do_cve_check tasks should fail, since doing a CVE check
> > against a corrupted/incomplete CVE database is clearly useless
> >
> > Regards,
> >
> > Alberto
> >
> >
> >
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#171657):
> https://lists.openembedded.org/g/openembedded-core/message/171657
> Mute This Topic: https://lists.openembedded.org/mt/94276393/5827677
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> rybczynska@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 6286 bytes --]

Re: [OE-core] severe issue in CVE checker

[Mikko Rapeli]

Hi,

Can the downloads be turned into normal do_fetch() SRC_URI downloads including
caches in yocto infrastructure?

There are many issues around CVE checking that it's really
a process where a lot of details and uncertainties just need to be
accepted. It's far from a perfect and users just need to accept this.

Cheers,

-Mikko

On Wed, Oct 12, 2022 at 04:54:57PM +0900, Marta Rybczynska wrote:
> I'll be looking into how to fix it. My current idea is to make the download
> a transaction, so do not update the database until we're sure the download
> is complete. Plus a warning that we have had an issue, I think.
> 
> In a next step we can retry a number of times.
> 
> Regards
> Marta
> 
> On Wed, 12 Oct 2022, 16:50 Alexander Kanavin, <alex.kanavin@gmail.com>
> wrote:
> 
> > Thanks for the information, can you send a patch?
> >
> > Alex
> >
> > On Wed, 12 Oct 2022 at 09:25, Alberto Pianon <alberto@pianon.eu> wrote:
> > >
> > > All, Marta, Richard,
> > >
> > > while implementing stats aggregation for CVE metadata in the Oniro
> > > project, I encountered a severe issue in Yocto's CVE checker, apparently
> > > due to this:
> > >
> > https://git.yoctoproject.org/poky/tree/meta/recipes-core/meta/cve-update-db-native.bb#n81
> > >
> > > It appears that when cve-update-db-native fails to fetch some years of
> > > NIST CVE db, it just issues a warning but goes on anyway.
> > >
> > > The result is that in some builds, randomly (depending on NIST webserver
> > > timeouts or other connection problems), CVE db is not complete, so the
> > > CVE check returns false negatives (i.e. no vulnerabilities found in some
> > > components even if such vulnerabilities do exist)
> > >
> > > I ran into such problem because in Oniro we need aggregate data from
> > > different builds for a large target matrix; I added a check to check
> > > that CVE metadata for each component are the same in all builds, and it
> > > failed, so I tried to figure out the cause and I found this:
> > >
> > > - in a build where cve-check found a vulnerability for acl:
> > >      $ sqlite3 build-linux-clang-qemuarm-efi/tmp/CVE_CHECK/nvdcve_1.1.db
> > >      sqlite> SELECT DISTINCT ID FROM PRODUCTS WHERE PRODUCT IS "acl";
> > >      CVE-2009-4411
> > >      sqlite>
> > >
> > > - in another build where cve-check did not found any vulnerability for
> > > the very same version of acl:
> > >      $ sqlite3 build-linux-clang-qemux86/tmp/CVE_CHECK/nvdcve_1.1.db
> > >      sqlite> SELECT DISTINCT ID FROM PRODUCTS WHERE PRODUCT IS "acl";
> > >      sqlite>
> > >
> > > so I listed both CVE db files in those two builds and this is what I
> > > got:
> > >
> > >      $ ls -ll build-linux-clang-qemuarm-efi/tmp/CVE_CHECK/nvdcve_1.1.db
> > > build-linux-clang-qemux86/tmp/CVE_CHECK/nvdcve_1.1.db
> > >      -rw-r--r-- 1 ubuntu ubuntu 215093248 Oct 11 10:56
> > > build-linux-clang-qemuarm-efi/tmp/CVE_CHECK/nvdcve_1.1.db
> > >      -rw-r--r-- 1 ubuntu ubuntu     28672 Oct 11 10:00
> > > build-linux-clang-qemux86/tmp/CVE_CHECK/nvdcve_1.1.db
> > >
> > > The two CVE db files were fetched just about 1h apart, but the second
> > > file is apparently incomplete.
> > >
> > > I checked the log for the second build, and I found this:
> > >
> > >      WARNING: cve-update-db-native-1.0-r0 do_fetch: Failed to fetch CVE
> > > data ([Errno 99] Cannot assign requested address)
> > >      NOTE: recipe cve-update-db-native-1.0-r0: task do_fetch: Succeeded
> > >
> > > Fetch fails, but do_fetch task succeeds.
> > >
> > > So I looked into the recipe's code, and I found this:
> > >
> > https://git.yoctoproject.org/poky/tree/meta/recipes-core/meta/cve-update-db-native.bb#n81
> > >
> > > It iterates over NIST CVE db years, but if some year fail to download,
> > > it goes on anyway, and it still merges the successful downloads into
> > > nvdcve_1.1.db, without returning error.
> > >
> > > IMHO this is a severe issue because it may silently lead to false
> > > negatives in the CVE check. If some downloads fail due to timeouts or
> > > other connection problems, cve-check should retry a number of times, and
> > > if any download still fails, cve-update-db-native do_fecth should fail,
> > > and it turn all do_cve_check tasks should fail, since doing a CVE check
> > > against a corrupted/incomplete CVE database is clearly useless
> > >
> > > Regards,
> > >
> > > Alberto
> > >
> > >
> > >
> >
> > 
> >
> >

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#171658): https://lists.openembedded.org/g/openembedded-core/message/171658
> Mute This Topic: https://lists.openembedded.org/mt/94276393/7159507
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [mikko.rapeli@linaro.org]
> -=-=-=-=-=-=-=-=-=-=-=-
> 

Re: [OE-core] severe issue in CVE checker

[Carlo Piana]

[-- Attachment #1: Type: text/plain, Size: 3195 bytes --]

[sent from the wrong account, resent, sorry for the noise] 

----- Messaggio originale ----- 
> Da: "Mikko Rapeli" <mikko.rapeli@linaro.org> 
> A: "Marta Rybczynska" <rybczynska@gmail.com> 
> Cc: "Alexander Kanavin" <alex.kanavin@gmail.com>, "Alberto Pianon" <alberto@pianon.eu>, "OE-core" 
> <openembedded-core@lists.openembedded.org>, "marta rybczynska" <marta.rybczynska@linaro.org>, "Richard Purdie" 
> <richard.purdie@linuxfoundation.org>, "Carlo Piana" <carlo@piana.eu> 
> Inviato: Mercoledì, 12 ottobre 2022 10:04:07 
> Oggetto: Re: [OE-core] severe issue in CVE checker 

> Hi, 
> 
> Can the downloads be turned into normal do_fetch() SRC_URI downloads including 
> caches in yocto infrastructure? 
> 
> There are many issues around CVE checking that it's really 
> a process where a lot of details and uncertainties just need to be 
> accepted. It's far from a perfect and users just need to accept this. 
> 
> 

[...] 

I beg to (strongly, if I may) differ. 

CVE is broken? This is no excuse of course to ignore a known insecurity. 

Is it better to include a process that, when fails, the fail goes unnoticed, than nothing? No, "nothing" is better than flawed here, speaking of security, since a false sense of security is worse than insecurity itself. 

If we put a CVE checker that just throws in a contradictory message (a warning and eventually a "success" one) and then silently moves on without any fuss, we leave users in a state of false belief that they have completed at least the CVE checks -- however insufficient this may be -- but in reality it's a test that never fails because the database is empty or outdated. I agree that for developers CVE checking, compliance, software component inventory are a PITA, but it's way more a PITA when an attacker exploits an unpatched known insecurity kept out in the wild, or when a copyright troll comes after you demanding (many!) millions in damages (I can't disclose who has received it, but I have seen it in real life), because you failed to notice that a GPL component went into a mass-distribution device. 

Once the function is advertised, the expected behaviour for a thing of that importance must be to visibly flag the issue and **stand in the way** until you acknowledge it, so that the warning cannot be missed. It is up to the duly informed user to say "Ok, it's nothing, I know it" and shrug the problem off. Not a decision taken for the user by others and hidden under the carpet. We have several clients relying on that CVE checking and just by coincidence we noticed that something did not add up. God only knows how many times did this happen before. 

One can think it's not their problem, this is open source after all. But being open source is not total relief from liability. If you create a hole in the road and cover it with foliage, the fact that you are not paid to pave the road and you are doing it as a service for the community does not shield you if a car takes a nose-dive into it. 

I am sorry to intrude in the discussion so bluntly, but I prefer that the legal implications are correctly perceived before making a decision. 

All the best, 

Carlo 


[-- Attachment #2: Type: text/html, Size: 3642 bytes --]

Re: [OE-core] severe issue in CVE checker

[Mikko Rapeli]

Hi,

I did not mean that yocto CVE checker is useless, or that bugs like
this should not be fixed. Using it is part of the best practice
processes for making secure products with yocto (even if it's currently
not documented as such).

But we should also document that the CVE checker makes a LOT of
assumptions which are frequently broken in real life. poky tries to work well
with CVE checker but many other layers and recipes may not work at all.

Few reasons are:

 * recipe name may not match to a CPE product name in the NVD database,
   and CVE_PRODUCT may not be set

 * recipe versions PV may not be compatible with upstream version
   numbers and hance the version range checks may fail

 * data in NVD is frequently wrong about the version ranges of CVEs

And there are more corner cases. All these fail silently.

Now you found a problem with the NVD download scripting which I think
needs to be fixed. But even if it works, developers who need to run
security checks on real products need to do more than just rely on
CVE check output.

Cheers,

-Mikko

On Wed, Oct 12, 2022 at 11:51:28AM +0200, Carlo Piana wrote:
> [sent from the wrong account, resent, sorry for the noise] 
> 
> ----- Messaggio originale ----- 
> > Da: "Mikko Rapeli" <mikko.rapeli@linaro.org> 
> > A: "Marta Rybczynska" <rybczynska@gmail.com> 
> > Cc: "Alexander Kanavin" <alex.kanavin@gmail.com>, "Alberto Pianon" <alberto@pianon.eu>, "OE-core" 
> > <openembedded-core@lists.openembedded.org>, "marta rybczynska" <marta.rybczynska@linaro.org>, "Richard Purdie" 
> > <richard.purdie@linuxfoundation.org>, "Carlo Piana" <carlo@piana.eu> 
> > Inviato: Mercoled�, 12 ottobre 2022 10:04:07 
> > Oggetto: Re: [OE-core] severe issue in CVE checker 
> 
> > Hi, 
> > 
> > Can the downloads be turned into normal do_fetch() SRC_URI downloads including 
> > caches in yocto infrastructure? 
> > 
> > There are many issues around CVE checking that it's really 
> > a process where a lot of details and uncertainties just need to be 
> > accepted. It's far from a perfect and users just need to accept this. 
> > 
> > 
> 
> [...] 
> 
> I beg to (strongly, if I may) differ. 
> 
> CVE is broken? This is no excuse of course to ignore a known insecurity. 
> 
> Is it better to include a process that, when fails, the fail goes unnoticed, than nothing? No, "nothing" is better than flawed here, speaking of security, since a false sense of security is worse than insecurity itself. 
> 
> If we put a CVE checker that just throws in a contradictory message (a warning and eventually a "success" one) and then silently moves on without any fuss, we leave users in a state of false belief that they have completed at least the CVE checks -- however insufficient this may be -- but in reality it's a test that never fails because the database is empty or outdated. I agree that for developers CVE checking, compliance, software component inventory are a PITA, but it's way more a PITA when an attacker exploits an unpatched known insecurity kept out in the wild, or when a copyright troll comes after you demanding (many!) millions in damages (I can't disclose who has received it, but I have seen it in real life), because you failed to notice that a GPL component went into a mass-distribution device. 
> 
> Once the function is advertised, the expected behaviour for a thing of that importance must be to visibly flag the issue and **stand in the way** until you acknowledge it, so that the warning cannot be missed. It is up to the duly informed user to say "Ok, it's nothing, I know it" and shrug the problem off. Not a decision taken for the user by others and hidden under the carpet. We have several clients relying on that CVE checking and just by coincidence we noticed that something did not add up. God only knows how many times did this happen before. 
> 
> One can think it's not their problem, this is open source after all. But being open source is not total relief from liability. If you create a hole in the road and cover it with foliage, the fact that you are not paid to pave the road and you are doing it as a service for the community does not shield you if a car takes a nose-dive into it. 
> 
> I am sorry to intrude in the discussion so bluntly, but I prefer that the legal implications are correctly perceived before making a decision. 
> 
> All the best, 
> 
> Carlo 
> 

Re: [OE-core] severe issue in CVE checker

[Marta Rybczynska]

Mikko et al,
I think that we all agree here that we need to fix that database corruption.

BTW The Oniro team has worked to fix many issues from the list you
mention, including frequent communication with NVD. Alberto and team
have been stressing the tool more than most people before, I think.
This is a good thing. And that they find bugs in the process...

I agree that the cve-check does not give perfect security (after all,
not all security issues have CVEs, and there is a delay between the
CVE creation and the NVD update). Using it is the best recommendation
we have for the users so it is in interest of all of us to make sure
the check is working reliably. We don't want unnoticed security issues
in our appliances, don't we? :)

So, let's concentrate on "how"/"who" in the process of fixing this issue...

Kind regards,
Marta

On Wed, Oct 12, 2022 at 12:03 PM Mikko Rapeli <mikko.rapeli@linaro.org> wrote:
>
> Hi,
>
> I did not mean that yocto CVE checker is useless, or that bugs like
> this should not be fixed. Using it is part of the best practice
> processes for making secure products with yocto (even if it's currently
> not documented as such).
>
> But we should also document that the CVE checker makes a LOT of
> assumptions which are frequently broken in real life. poky tries to work well
> with CVE checker but many other layers and recipes may not work at all.
>
> Few reasons are:
>
>  * recipe name may not match to a CPE product name in the NVD database,
>    and CVE_PRODUCT may not be set
>
>  * recipe versions PV may not be compatible with upstream version
>    numbers and hance the version range checks may fail
>
>  * data in NVD is frequently wrong about the version ranges of CVEs
>
> And there are more corner cases. All these fail silently.
>
> Now you found a problem with the NVD download scripting which I think
> needs to be fixed. But even if it works, developers who need to run
> security checks on real products need to do more than just rely on
> CVE check output.
>
> Cheers,
>
> -Mikko
>
> On Wed, Oct 12, 2022 at 11:51:28AM +0200, Carlo Piana wrote:
> > [sent from the wrong account, resent, sorry for the noise]
> >
> > ----- Messaggio originale -----
> > > Da: "Mikko Rapeli" <mikko.rapeli@linaro.org>
> > > A: "Marta Rybczynska" <rybczynska@gmail.com>
> > > Cc: "Alexander Kanavin" <alex.kanavin@gmail.com>, "Alberto Pianon" <alberto@pianon.eu>, "OE-core"
> > > <openembedded-core@lists.openembedded.org>, "marta rybczynska" <marta.rybczynska@linaro.org>, "Richard Purdie"
> > > <richard.purdie@linuxfoundation.org>, "Carlo Piana" <carlo@piana.eu>
> > > Inviato: Mercoledì, 12 ottobre 2022 10:04:07
> > > Oggetto: Re: [OE-core] severe issue in CVE checker
> >
> > > Hi,
> > >
> > > Can the downloads be turned into normal do_fetch() SRC_URI downloads including
> > > caches in yocto infrastructure?
> > >
> > > There are many issues around CVE checking that it's really
> > > a process where a lot of details and uncertainties just need to be
> > > accepted. It's far from a perfect and users just need to accept this.
> > >
> > >
> >
> > [...]
> >
> > I beg to (strongly, if I may) differ.
> >
> > CVE is broken? This is no excuse of course to ignore a known insecurity.
> >
> > Is it better to include a process that, when fails, the fail goes unnoticed, than nothing? No, "nothing" is better than flawed here, speaking of security, since a false sense of security is worse than insecurity itself.
> >
> > If we put a CVE checker that just throws in a contradictory message (a warning and eventually a "success" one) and then silently moves on without any fuss, we leave users in a state of false belief that they have completed at least the CVE checks -- however insufficient this may be -- but in reality it's a test that never fails because the database is empty or outdated. I agree that for developers CVE checking, compliance, software component inventory are a PITA, but it's way more a PITA when an attacker exploits an unpatched known insecurity kept out in the wild, or when a copyright troll comes after you demanding (many!) millions in damages (I can't disclose who has received it, but I have seen it in real life), because you failed to notice that a GPL component went into a mass-distribution device.
> >
> > Once the function is advertised, the expected behaviour for a thing of that importance must be to visibly flag the issue and **stand in the way** until you acknowledge it, so that the warning cannot be missed. It is up to the duly informed user to say "Ok, it's nothing, I know it" and shrug the problem off. Not a decision taken for the user by others and hidden under the carpet. We have several clients relying on that CVE checking and just by coincidence we noticed that something did not add up. God only knows how many times did this happen before.
> >
> > One can think it's not their problem, this is open source after all. But being open source is not total relief from liability. If you create a hole in the road and cover it with foliage, the fact that you are not paid to pave the road and you are doing it as a service for the community does not shield you if a car takes a nose-dive into it.
> >
> > I am sorry to intrude in the discussion so bluntly, but I prefer that the legal implications are correctly perceived before making a decision.
> >
> > All the best,
> >
> > Carlo
> >

Re: [OE-core] severe issue in CVE checker

[Quentin Schulz]

Hi Carlo,

I'm a mere contributor, opinions are my own. I do not represent the 
project's official view on this. Also, I am not a lawyer.

On 10/12/22 11:51, Carlo "Kappa" Piana wrote:
> [sent from the wrong account, resent, sorry for the noise]
> 
> ----- Messaggio originale -----
>> Da: "Mikko Rapeli" <mikko.rapeli@linaro.org>
>> A: "Marta Rybczynska" <rybczynska@gmail.com>
>> Cc: "Alexander Kanavin" <alex.kanavin@gmail.com>, "Alberto Pianon" <alberto@pianon.eu>, "OE-core"
>> <openembedded-core@lists.openembedded.org>, "marta rybczynska" <marta.rybczynska@linaro.org>, "Richard Purdie"
>> <richard.purdie@linuxfoundation.org>, "Carlo Piana" <carlo@piana.eu>
>> Inviato: Mercoledì, 12 ottobre 2022 10:04:07
>> Oggetto: Re: [OE-core] severe issue in CVE checker
> 
>> Hi,
>>
>> Can the downloads be turned into normal do_fetch() SRC_URI downloads including
>> caches in yocto infrastructure?
>>
>> There are many issues around CVE checking that it's really
>> a process where a lot of details and uncertainties just need to be
>> accepted. It's far from a perfect and users just need to accept this.
>>
>>
> 
> [...]
> 
> I beg to (strongly, if I may) differ.
> 
> CVE is broken? This is no excuse of course to ignore a known insecurity.
> 

Thank you for reporting this. You already got an answer from the people 
who worked on it. This is now known and hopefully we can fix it soon, it 
depends on the contributors though (which you can become by sending a 
patch :) ).

> Is it better to include a process that, when fails, the fail goes unnoticed, than nothing? No, "nothing" is better than flawed here, speaking of security, since a false sense of security is worse than insecurity itself.
> 
> If we put a CVE checker that just throws in a contradictory message (a warning and eventually a "success" one) and then silently moves on without any fuss, we leave users in a state of false belief that they have completed at least the CVE checks -- however insufficient this may be -- but in reality it's a test that never fails because the database is empty or outdated. I agree that for developers CVE checking, compliance, software component inventory are a PITA, but it's way more a PITA when an attacker exploits an unpatched known insecurity kept out in the wild, or when a copyright troll comes after you demanding (many!) millions in damages (I can't disclose who has received it, but I have seen it in real life), because you failed to notice that a GPL component went into a mass-distribution device.
> 
> Once the function is advertised, the expected behaviour for a thing of that importance must be to visibly flag the issue and **stand in the way** until you acknowledge it, so that the warning cannot be missed. It is up to the duly informed user to say "Ok, it's nothing, I know it" and shrug the problem off. Not a decision taken for the user by others and hidden under the carpet. We have several clients relying on that CVE checking and just by coincidence we noticed that something did not add up. God only knows how many times did this happen before.
> 

You should not rely on this tool (or for that matter ANY tool from ANY 
project) without human review from specialists (be they lawyers and/or 
security researchers) on its output. If you need guarantees, get a 
company to do it for you or hire teams of people. It will be expensive, yes.

> One can think it's not their problem, this is open source after all. But being open source is not total relief from liability. If you create a hole in the road and cover it with foliage, the fact that you are not paid to pave the road and you are doing it as a service for the community does not shield you if a car takes a nose-dive into it.
> 

It actually does. It is the very essence of the open-source licenses 
we're using. Mainly, MIT, c.f. 
https://docs.yoctoproject.org/overview-manual/development-environment.html#licensing
By using our project, you're basically signing a waver form saying 
"whatever happens I asked for it and cannot hold the project liable". 
This applies to all (most?) pieces of SW that this project builds too.

https://opensource.org/licenses/mit-license.php states:
"THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS 
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY 
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, 
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."

It is YOUR duty to check whatever you're using is compliant 
license-wise. It is absolutely possible a recipe we have advertises 
being GPL-2.0-only and being actually GPL-3.0-or-later, we are human, we 
make mistakes. Moreover, we are NOT lawyers, so we can misunderstand to 
which files some licenses apply in a project for example.

Also, it's not because a piece of SW says it is licensed under a 
specific license that it legally is (hello license poisoning for 
example). We cannot as a project afford to look into all the pieces of 
software we allow you to build to check if they are legally compliant 
with whatever they say they are licensed under. This is a job for your 
legal department. Also, since we allow third party layers to be used 
with our core layers, it's not something we technically can do.

Now, about CVE checks, the same applies. We provide tools to make it 
easier for you to check that some CVEs are fixed or not. It is legally 
non-binding (see paragraphs above) and your legal and security 
departments are the only ones able to "guarantee" CVEs are fixed or not.
The tools we provide are automated tools and as every software in the 
world not perfect or free of bugs. We rely on people to report those 
bugs (like you did, thank you for that) and/or fix them to improve those 
tools. If you need to certify your project does not contain CVEs or CVEs 
it is vulnerable to aren't usable, get a security research team to look 
at your images.

I can understand that you are disappointed to see our project is not 
bug-free but there's a big misunderstanding in what the project is 
providing you with and what you can expect from it.

Yes, legal and security teams are expensive. But they provide 
"guarantees" that we'll never provide (see open-source licenses).

Cheers,
Quentin

Re: [OE-core] severe issue in CVE checker

[Ross Burton]

On 12 Oct 2022, at 08:25, Alberto Pianon via lists.openembedded.org <alberto=pianon.eu@lists.openembedded.org> wrote:
> 
> It iterates over NIST CVE db years, but if some year fail to download, it goes on anyway, and it still merges the successful downloads into nvdcve_1.1.db, without returning error.
> 
> IMHO this is a severe issue because it may silently lead to false negatives in the CVE check. If some downloads fail due to timeouts or other connection problems, cve-check should retry a number of times, and if any download still fails, cve-update-db-native do_fecth should fail, and it turn all do_cve_check tasks should fail, since doing a CVE check against a corrupted/incomplete CVE database is clearly useless

Yes, that’s a bug.

Please do file a bug (bugzilla.yoctoproject.org), and ideally send a patch if you can.

Ross

Re: [OE-core] severe issue in CVE checker

[Marta Rybczynska]

Reported as https://bugzilla.yoctoproject.org/show_bug.cgi?id=14929

On Wed, Oct 12, 2022 at 6:29 PM Ross Burton <ross.burton@arm.com> wrote:
>
> On 12 Oct 2022, at 08:25, Alberto Pianon via lists.openembedded.org <alberto=pianon.eu@lists.openembedded.org> wrote:
> >
> > It iterates over NIST CVE db years, but if some year fail to download, it goes on anyway, and it still merges the successful downloads into nvdcve_1.1.db, without returning error.
> >
> > IMHO this is a severe issue because it may silently lead to false negatives in the CVE check. If some downloads fail due to timeouts or other connection problems, cve-check should retry a number of times, and if any download still fails, cve-update-db-native do_fecth should fail, and it turn all do_cve_check tasks should fail, since doing a CVE check against a corrupted/incomplete CVE database is clearly useless
>
> Yes, that’s a bug.
>
> Please do file a bug (bugzilla.yoctoproject.org), and ideally send a patch if you can.
>
> Ross
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#171668): https://lists.openembedded.org/g/openembedded-core/message/171668
> Mute This Topic: https://lists.openembedded.org/mt/94276393/5827677
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [rybczynska@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>