[Buildroot] CVEs not matching buildroot packages

[Buildroot] CVEs not matching buildroot packages

[Michael Walle]

Hi,

Yesterday, Heiko noticed that CVE-2020-8597 doesn't match the
pppd package, because the product name doesn't match the package
name.

So apparently there need to be a mapping between these two. But
that comes with some caveats. Let's assume there would be a
pkg_NVD_CPE variable. One could think of having a default value
of this variable, eg by default "pkg_NVD_CPE = pkg". [This is,
as far as I understand it, the current behaviour, because there
is no mapping right now; correct me if I'm wrong]. BUT in my
optinion there MUST NOT be a default value for this mapping,
because then you cannot distiguish between a package which just
uses the default name and a package which wasn't ever touched
regarding CVEs. And this is really bad for statistics, eg you
can't even say we have this many packages checked for CVEs,
because you don't know which packages actually has a correct
mapping.

So if a pkg_NVD_CPE variable is introduced this would need to
done manually per package; which would mean a lot of work. Of
course there could be some scripts which check if there was ever
a CVE for package x and x is a buildroot package. That would be
a strong indication that this is a correct mapping. There could
also be a checkpackage script which checks if pkg has no
pkg_NVD_CPE variable, but there is a CVE for that package name.

Then a statistics script could gather:
   (1) how many packages don't have a mapping, eg. the CVE status
       is unknown. [package has no pkg_NVD_CPE]
   (2) how many packages are not affected by a CVE [package has
       pkg_NVD_CPE and doesn't match any CVEs]
   (3) how many packages are affected by a CVE [package has
       pkg_NVD_CPE and doesn't match any CVEs]

To minimize efforts in the beginning (1) could be split into
  (1a) how many packages don't have a mapping and no CVE matches
       the package name, therefore the status is still unknown
       [package has no pkg_NVD_CPE and there is no CVE matching
       the package name]
  (1b) how many packages don't have a mapping but there is a CVE
       matching the package name. Thus there is a high chance
       this package is actually affected by this CVE, but this
       is just a strong indication and would need further manual
       steps, eg. adding pkg_NVE_CPE to the package .mk.

So packages could incrementally moved from (1a) to (1b).

Any comments? ;)

-michael

[Buildroot] CVEs not matching buildroot packages

[Thomas Petazzoni]

Hello Michael,

Thanks for pushing this conversation further, we definitely want to
make progress on this topic.

On Tue, 25 Feb 2020 09:58:27 +0100
Michael Walle <michael@walle.cc> wrote:

> Yesterday, Heiko noticed that CVE-2020-8597 doesn't match the
> pppd package, because the product name doesn't match the package
> name.

Indeed. This was a known limitation when we did the initial work on the
CVE checking as part of pkg-stats during the Buildroot Developers
meeting.

> So apparently there need to be a mapping between these two. But
> that comes with some caveats. Let's assume there would be a
> pkg_NVD_CPE variable.

Nit: the NVD database uses the concept of "vendor" and "product" names.
So either we have a single variable like <pkg>_CPE_ID that looks like
CPE IDs:

LINUX_CPE_ID = cpe:2.3:o:linux:linux_kernel:*:*:*:*:*:*:*:*

Or we have separate variables for the vendor and product names:

LINUX_CPE_VENDOR = linux
LINUX_CPE_PRODUCT = linux_kernel

> One could think of having a default value
> of this variable, eg by default "pkg_NVD_CPE = pkg". [This is,
> as far as I understand it, the current behaviour, because there
> is no mapping right now; correct me if I'm wrong].

You're correct.

> BUT in my optinion there MUST NOT be a default value for this mapping,
> because then you cannot distiguish between a package which just
> uses the default name and a package which wasn't ever touched
> regarding CVEs. And this is really bad for statistics, eg you
> can't even say we have this many packages checked for CVEs,
> because you don't know which packages actually has a correct
> mapping.

Yes. With the release-monitoring.org matching, we are able to know if
the match was found thanks to an explicit mapping of the Buildroot
distribution, or if it was found by "luck". That's the difference
between "found by distro" and "found by guess" in
http://autobuild.buildroot.net/stats/.

> So if a pkg_NVD_CPE variable is introduced this would need to
> done manually per package; which would mean a lot of work. Of
> course there could be some scripts which check if there was ever
> a CVE for package x and x is a buildroot package. That would be
> a strong indication that this is a correct mapping. There could
> also be a checkpackage script which checks if pkg has no
> pkg_NVD_CPE variable, but there is a CVE for that package name.

On the other hand, as explained above, the CPE information is not just
a package name, it's also a vendor name. So in practice the default of
using the package name doesn't really work.

> Then a statistics script could gather:
>    (1) how many packages don't have a mapping, eg. the CVE status
>        is unknown. [package has no pkg_NVD_CPE]
>    (2) how many packages are not affected by a CVE [package has
>        pkg_NVD_CPE and doesn't match any CVEs]
>    (3) how many packages are affected by a CVE [package has
>        pkg_NVD_CPE and doesn't match any CVEs]

This last sentence should have been "and match some CVEs"

> To minimize efforts in the beginning (1) could be split into
>   (1a) how many packages don't have a mapping and no CVE matches
>        the package name, therefore the status is still unknown
>        [package has no pkg_NVD_CPE and there is no CVE matching
>        the package name]
>   (1b) how many packages don't have a mapping but there is a CVE
>        matching the package name. Thus there is a high chance
>        this package is actually affected by this CVE, but this
>        is just a strong indication and would need further manual
>        steps, eg. adding pkg_NVE_CPE to the package .mk.
> 
> So packages could incrementally moved from (1a) to (1b).
> 
> Any comments? ;)

One question is how will you distinguish packages that don't have any
CPE information because no CVE has ever been reported and therefore we
don't know which CPE vendor/product IDs will be used, and packages that
don't have CPE information because nobody has done the work of checking
the NVD database and adding the mapping.

Indeed, in our 2500+ packages, I'm pretty sure a significant number of
them never had any CVE, and therefore we have no idea what their CPE ID
would be.

Any idea on this ?

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] CVEs not matching buildroot packages

[Michael Walle]

Hi Thomas,

Am 2020-02-25 10:46, schrieb Thomas Petazzoni:
> Hello Michael,
> 
> Thanks for pushing this conversation further, we definitely want to
> make progress on this topic.
> 
> On Tue, 25 Feb 2020 09:58:27 +0100
> Michael Walle <michael@walle.cc> wrote:
> 
>> Yesterday, Heiko noticed that CVE-2020-8597 doesn't match the
>> pppd package, because the product name doesn't match the package
>> name.
> 
> Indeed. This was a known limitation when we did the initial work on the
> CVE checking as part of pkg-stats during the Buildroot Developers
> meeting.
> 
>> So apparently there need to be a mapping between these two. But
>> that comes with some caveats. Let's assume there would be a
>> pkg_NVD_CPE variable.
> 
> Nit: the NVD database uses the concept of "vendor" and "product" names.
> So either we have a single variable like <pkg>_CPE_ID that looks like
> CPE IDs:
> 
> LINUX_CPE_ID = cpe:2.3:o:linux:linux_kernel:*:*:*:*:*:*:*:*
> 
> Or we have separate variables for the vendor and product names:
> 
> LINUX_CPE_VENDOR = linux
> LINUX_CPE_PRODUCT = linux_kernel

I've left this out on purpose for simplicity ;) You could assume that
the product name is unique and if its not you could add the vendor.
Like for example:

  _CPE = linux_kernel
  _CPE = linux:linux_kernel

eg. something like that:
  pkg_CPE = [vendor:]product

Maybe it shouldn't be named CPE after all.. But I wanted to avoid
this discussion for now, because it doesn't matter for the mapping
problem ;)

>> One could think of having a default value
>> of this variable, eg by default "pkg_NVD_CPE = pkg". [This is,
>> as far as I understand it, the current behaviour, because there
>> is no mapping right now; correct me if I'm wrong].
> 
> You're correct.
> 
>> BUT in my optinion there MUST NOT be a default value for this mapping,
>> because then you cannot distiguish between a package which just
>> uses the default name and a package which wasn't ever touched
>> regarding CVEs. And this is really bad for statistics, eg you
>> can't even say we have this many packages checked for CVEs,
>> because you don't know which packages actually has a correct
>> mapping.
> 
> Yes. With the release-monitoring.org matching, we are able to know if
> the match was found thanks to an explicit mapping of the Buildroot
> distribution, or if it was found by "luck". That's the difference
> between "found by distro" and "found by guess" in
> http://autobuild.buildroot.net/stats/.

Mhh, I don't think I can follow you here. Even if the package is found
in release-monitoring.org it doesn't map to any CVEs/CPEs, does it?
Or should this be an example on already existing fuzzy matching in
buildroot?

>> So if a pkg_NVD_CPE variable is introduced this would need to
>> done manually per package; which would mean a lot of work. Of
>> course there could be some scripts which check if there was ever
>> a CVE for package x and x is a buildroot package. That would be
>> a strong indication that this is a correct mapping. There could
>> also be a checkpackage script which checks if pkg has no
>> pkg_NVD_CPE variable, but there is a CVE for that package name.
> 
> On the other hand, as explained above, the CPE information is not just
> a package name, it's also a vendor name. So in practice the default of
> using the package name doesn't really work.

correct. unless you only match only parts of the CPE. But yes. If there
would be a variable pkg_CPE which should contain the correct CPE, that
would not work with the default.

> 
>> Then a statistics script could gather:
>>    (1) how many packages don't have a mapping, eg. the CVE status
>>        is unknown. [package has no pkg_NVD_CPE]
>>    (2) how many packages are not affected by a CVE [package has
>>        pkg_NVD_CPE and doesn't match any CVEs]
>>    (3) how many packages are affected by a CVE [package has
>>        pkg_NVD_CPE and doesn't match any CVEs]
> 
> This last sentence should have been "and match some CVEs"

oh, yes of course.

> 
>> To minimize efforts in the beginning (1) could be split into
>>   (1a) how many packages don't have a mapping and no CVE matches
>>        the package name, therefore the status is still unknown
>>        [package has no pkg_NVD_CPE and there is no CVE matching
>>        the package name]
>>   (1b) how many packages don't have a mapping but there is a CVE
>>        matching the package name. Thus there is a high chance
>>        this package is actually affected by this CVE, but this
>>        is just a strong indication and would need further manual
>>        steps, eg. adding pkg_NVE_CPE to the package .mk.
>> 
>> So packages could incrementally moved from (1a) to (1b).
>> 
>> Any comments? ;)
> 
> One question is how will you distinguish packages that don't have any
> CPE information because no CVE has ever been reported and therefore we
> don't know which CPE vendor/product IDs will be used, and packages that
> don't have CPE information because nobody has done the work of checking
> the NVD database and adding the mapping.

IMHO you cannot distiguish between these two, can you? Only if there 
would
be another CPE database for all software out there. Otherwise if there 
is
no CPE without and actual CVE, you can't do anything about it. So the
package would still be unknown. A counterexample, let's assume you add

  pkg_CPE = <unknown>

to indicate you've manually checked that there is currently no CVE
matching this package and thus no CPE exists. Now, if there is actually
a CVE/CPE the recipe won't be updated or worse the CVE matching would
report nothing.

So I don't think you can do anything about that other than just saying
we don't know anything about this package.

> Indeed, in our 2500+ packages, I'm pretty sure a significant number of
> them never had any CVE, and therefore we have no idea what their CPE ID
> would be.
> 
> Any idea on this ?

Well I'd imaging three states in the stats:
  - green, no CVE applies, we know the CPE
  - red, CVE applies, we know the CPE
  - yellow, CPE unknown, for whatever reason

If you're right with "a significant number of them never had any CVE",
then many would be yellow. But I don't see any drawback on this. At the
of the day what really matters are the packages in the red category.
And of course, how we can get as many packages as possibe from yellow
to green/red.

What could also be done is a seperate script which loops over the CVEs
and prints if no mapping to a buildroot package is found. So someone
could look at it and say, oh this CVE looks like it should match one
of our packages.

-michael

[Buildroot] CVEs not matching buildroot packages

[Thomas Petazzoni]

Hello Michael,

On Tue, 25 Feb 2020 11:19:20 +0100
Michael Walle <michael@walle.cc> wrote:

> > LINUX_CPE_VENDOR = linux
> > LINUX_CPE_PRODUCT = linux_kernel  
> 
> I've left this out on purpose for simplicity ;)

OK.

> You could assume that
> the product name is unique and if its not you could add the vendor.
> Like for example:
> 
>   _CPE = linux_kernel
>   _CPE = linux:linux_kernel
> 
> eg. something like that:
>   pkg_CPE = [vendor:]product
> 
> Maybe it shouldn't be named CPE after all.. But I wanted to avoid
> this discussion for now, because it doesn't matter for the mapping
> problem ;)

Sure.

> > Yes. With the release-monitoring.org matching, we are able to know if
> > the match was found thanks to an explicit mapping of the Buildroot
> > distribution, or if it was found by "luck". That's the difference
> > between "found by distro" and "found by guess" in
> > http://autobuild.buildroot.net/stats/.  
> 
> Mhh, I don't think I can follow you here. Even if the package is found
> in release-monitoring.org it doesn't map to any CVEs/CPEs, does it?
> Or should this be an example on already existing fuzzy matching in
> buildroot?

Of course release-monitoring.org is completely unrelated from
CVEs/CPEs. I was mentioning release-monitoring.org as another case in
Buildroot where we have an issue with mapping Buildroot package names
with package/project names used by another entity.

But now that I think of it, why not rely a bit more on
release-monitoring.org after all? We could add a NVD distribution to
release-monitoring.org, and use it to store the mapping between
release-monitoring.org projects and their corresponding CPE IDs.

Then, Buildroot can query release-monitoring.org to find the CPE IDs.
One advantage of this approach is that this CPE ID information in
release-monitoring.org is not only useful for Buildroot, but
potentially for any other project using release-monitoring.org.

> > On the other hand, as explained above, the CPE information is not just
> > a package name, it's also a vendor name. So in practice the default of
> > using the package name doesn't really work.  
> 
> correct. unless you only match only parts of the CPE. But yes. If there
> would be a variable pkg_CPE which should contain the correct CPE, that
> would not work with the default.

Right.

> > One question is how will you distinguish packages that don't have any
> > CPE information because no CVE has ever been reported and therefore we
> > don't know which CPE vendor/product IDs will be used, and packages that
> > don't have CPE information because nobody has done the work of checking
> > the NVD database and adding the mapping.  
> 
> IMHO you cannot distiguish between these two, can you? Only if there 
> would be another CPE database for all software out there. Otherwise if there 
> is no CPE without and actual CVE, you can't do anything about it.

But then if we cannot distinguish between these two, it means there is
no point in forcing to have a <pkg>_CPE variable in each package. We
could rely on the default being the package name, as we're doing today,
and simply add <pkg>_CPE to override that default.

Your initial statement was:

>> BUT in my optinion there MUST NOT be a default value for this mapping,
>> because then you cannot distiguish between a package which just
>> uses the default name and a package which wasn't ever touched
>> regarding CVEs.

However, we will anyway never be able to distinguish between a package
that does not have a <pkg>_CPE variable because there is no known CPE
ID for this package, or because nobody did the effort.

> So the
> package would still be unknown. A counterexample, let's assume you add

> 
>   pkg_CPE = <unknown>
> 
> to indicate you've manually checked that there is currently no CVE
> matching this package and thus no CPE exists. Now, if there is actually
> a CVE/CPE the recipe won't be updated or worse the CVE matching would
> report nothing.

Indeed, that makes the whole <pkg>_CPE = <unknown> idea a bit useless.

> So I don't think you can do anything about that other than just saying
> we don't know anything about this package.
> 
> > Indeed, in our 2500+ packages, I'm pretty sure a significant number of
> > them never had any CVE, and therefore we have no idea what their CPE ID
> > would be.
> > 
> > Any idea on this ?  
> 
> Well I'd imaging three states in the stats:
>   - green, no CVE applies, we know the CPE
>   - red, CVE applies, we know the CPE
>   - yellow, CPE unknown, for whatever reason
> 
> If you're right with "a significant number of them never had any CVE",
> then many would be yellow. But I don't see any drawback on this. At the
> of the day what really matters are the packages in the red category.

No, because the yellow category could very well be "there are some
serious CVEs potentially affecting some packages, but nobody has added
the CPE ID information for the moment". So in the end, this yellow zone
is as dangerous as the red zone.

> And of course, how we can get as many packages as possibe from yellow
> to green/red.
> 
> What could also be done is a seperate script which loops over the CVEs
> and prints if no mapping to a buildroot package is found. So someone
> could look at it and say, oh this CVE looks like it should match one
> of our packages.

This is going to be horrible: the NVD database contains CVEs for
zillions of proprietary products that are completely irrelevant to
Buildroot. Looking through all of this data is going to be really
atrocious.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] CVEs not matching buildroot packages

[Michael Walle]

Am 2020-02-25 11:29, schrieb Thomas Petazzoni:
> Hello Michael,
> 
> On Tue, 25 Feb 2020 11:19:20 +0100
> Michael Walle <michael@walle.cc> wrote:
> 
>> > LINUX_CPE_VENDOR = linux
>> > LINUX_CPE_PRODUCT = linux_kernel
>> 
>> I've left this out on purpose for simplicity ;)
> 
> OK.
> 
>> You could assume that
>> the product name is unique and if its not you could add the vendor.
>> Like for example:
>> 
>>   _CPE = linux_kernel
>>   _CPE = linux:linux_kernel
>> 
>> eg. something like that:
>>   pkg_CPE = [vendor:]product
>> 
>> Maybe it shouldn't be named CPE after all.. But I wanted to avoid
>> this discussion for now, because it doesn't matter for the mapping
>> problem ;)
> 
> Sure.
> 
>> > Yes. With the release-monitoring.org matching, we are able to know if
>> > the match was found thanks to an explicit mapping of the Buildroot
>> > distribution, or if it was found by "luck". That's the difference
>> > between "found by distro" and "found by guess" in
>> > http://autobuild.buildroot.net/stats/.
>> 
>> Mhh, I don't think I can follow you here. Even if the package is found
>> in release-monitoring.org it doesn't map to any CVEs/CPEs, does it?
>> Or should this be an example on already existing fuzzy matching in
>> buildroot?
> 
> Of course release-monitoring.org is completely unrelated from
> CVEs/CPEs. I was mentioning release-monitoring.org as another case in
> Buildroot where we have an issue with mapping Buildroot package names
> with package/project names used by another entity.
> 
> But now that I think of it, why not rely a bit more on
> release-monitoring.org after all? We could add a NVD distribution to
> release-monitoring.org, and use it to store the mapping between
> release-monitoring.org projects and their corresponding CPE IDs.
> 
> Then, Buildroot can query release-monitoring.org to find the CPE IDs.
> One advantage of this approach is that this CPE ID information in
> release-monitoring.org is not only useful for Buildroot, but
> potentially for any other project using release-monitoring.org.

Haha, actually Heiko had the same idea ;) But I'm not in favor of
relying on too many external stuff and putting in one more layer
of indirection. But thats just my opinion.

But even then you'd have to map the buildroot packages to
release-monitoring.org URLs (which we have to have anyway, I know ;).

But.. once release-monitors.org would have CPE mappings, we could just
pull the information to help creating our own mapping. I mean the CPEs
won't change anymore and could be easily added to the recipes.

>> > On the other hand, as explained above, the CPE information is not just
>> > a package name, it's also a vendor name. So in practice the default of
>> > using the package name doesn't really work.
>> 
>> correct. unless you only match only parts of the CPE. But yes. If 
>> there
>> would be a variable pkg_CPE which should contain the correct CPE, that
>> would not work with the default.
> 
> Right.
> 
>> > One question is how will you distinguish packages that don't have any
>> > CPE information because no CVE has ever been reported and therefore we
>> > don't know which CPE vendor/product IDs will be used, and packages that
>> > don't have CPE information because nobody has done the work of checking
>> > the NVD database and adding the mapping.
>> 
>> IMHO you cannot distiguish between these two, can you? Only if there
>> would be another CPE database for all software out there. Otherwise if 
>> there
>> is no CPE without and actual CVE, you can't do anything about it.
> 
> But then if we cannot distinguish between these two, it means there is
> no point in forcing to have a <pkg>_CPE variable in each package. We
> could rely on the default being the package name, as we're doing today,
> and simply add <pkg>_CPE to override that default.

The point here is that in the good case you have the information if 
actually
anyone looked into it; eg. changed PPPD_CPE from pppd (the default) to
PPPD_CPE = point-to-point_protocol. So a script can find out what 
packages
are not looked at. And yes, in the case there is no CPE yet it would be 
the
same as you didn't looked at it yet. But thats another problem IMHO. Eg 
you
could tag a package with a date you looked at it last time and found no 
CPE.

You might have both worlds. Eg. have the default and tell the script if
someone looked at it: allow setting the "pkg_CPE = pkg" in the .mk (eg.
the same value as the default). So you can find out in which zone you 
are.
If you have pkg_CPE in the .mk you're either in the red or green zone. 
If
you don't have the pkg_CPE variable, then the default applies and you're 
in
the yellow zone (even if a CVE applies!).

> Your initial statement was:
> 
>>> BUT in my optinion there MUST NOT be a default value for this 
>>> mapping,
>>> because then you cannot distiguish between a package which just
>>> uses the default name and a package which wasn't ever touched
>>> regarding CVEs.
> 
> However, we will anyway never be able to distinguish between a package
> that does not have a <pkg>_CPE variable because there is no known CPE
> ID for this package, or because nobody did the effort.

Correct.

>> So the
>> package would still be unknown. A counterexample, let's assume you add
> 
>> 
>>   pkg_CPE = <unknown>
>> 
>> to indicate you've manually checked that there is currently no CVE
>> matching this package and thus no CPE exists. Now, if there is 
>> actually
>> a CVE/CPE the recipe won't be updated or worse the CVE matching would
>> report nothing.
> 
> Indeed, that makes the whole <pkg>_CPE = <unknown> idea a bit useless.
> 
>> So I don't think you can do anything about that other than just saying
>> we don't know anything about this package.
>> 
>> > Indeed, in our 2500+ packages, I'm pretty sure a significant number of
>> > them never had any CVE, and therefore we have no idea what their CPE ID
>> > would be.
>> >
>> > Any idea on this ?
>> 
>> Well I'd imaging three states in the stats:
>>   - green, no CVE applies, we know the CPE
>>   - red, CVE applies, we know the CPE
>>   - yellow, CPE unknown, for whatever reason

So taking the default from above we'd also have forth one:
- yellow, CPE guessed, CVE applies.

>> If you're right with "a significant number of them never had any CVE",
>> then many would be yellow. But I don't see any drawback on this. At 
>> the
>> of the day what really matters are the packages in the red category.
> 
> No, because the yellow category could very well be "there are some
> serious CVEs potentially affecting some packages, but nobody has added
> the CPE ID information for the moment". So in the end, this yellow zone
> is as dangerous as the red zone.

Correct, but it would be an easy step to convert a package from the 
yellow
zone to the red zone. Eg. just verify that this CVE maps to the package,
then add pkg_CPE to the recipe. So ultimately there shouldn't be a CVE
matching in the yellow zone. Only packages where no CVEs match. So for
packages which have a CVE and the "CPE" matches the buildroot package
name, this would be an easy task.

Going through all the other packages which are yellow but no CVE matches
is a more daunting task. And yes, if no CPE is found for now, this
information should also be stored somewhere.

>> And of course, how we can get as many packages as possibe from yellow
>> to green/red.
>> 
>> What could also be done is a seperate script which loops over the CVEs
>> and prints if no mapping to a buildroot package is found. So someone
>> could look at it and say, oh this CVE looks like it should match one
>> of our packages.
> 
> This is going to be horrible: the NVD database contains CVEs for
> zillions of proprietary products that are completely irrelevant to
> Buildroot. Looking through all of this data is going to be really
> atrocious.

I see. I guess there is no matching for "!= commercial"? ;)

-michael

[Buildroot] CVEs not matching buildroot packages

[Heiko Thiery]

Hi all,

Am Di., 25. Feb. 2020 um 13:41 Uhr schrieb Michael Walle <michael@walle.cc>:
>
> Am 2020-02-25 11:29, schrieb Thomas Petazzoni:
> > Hello Michael,
> >
> > On Tue, 25 Feb 2020 11:19:20 +0100
> > Michael Walle <michael@walle.cc> wrote:
> >
> >> > LINUX_CPE_VENDOR = linux
> >> > LINUX_CPE_PRODUCT = linux_kernel
> >>ould be an easy step to convert a package from the
> yellow
> >> I've left this out on purpose for simplicity ;)
> >
> > OK.
> >
> >> You could assume that
> >> the product name is unique and if its not you could add the vendor.
> >> Like for example:
> >>
> >>   _CPE = linux_kernel
> >>   _CPE = linux:linux_kernel
> >>
> >> eg. something like that:
> >>   pkg_CPE = [vendor:]product
> >>
> >> Maybe it shouldn't be named CPE after all.. But I wanted to avoid
> >> this discussion for now, because it doesn't matter for the mapping
> >> problem ;)
> >
> > Sure.
> >
> >> > Yes. With the release-monitoring.org matching, we are able to know if
> >> > the match was found thanks to an explicit mapping of the Buildroot
> >> > distribution, or if it was found by "luck". That's the difference
> >> > between "found by distro" and "found by guess" in
> >> > http://autobuild.buildroot.net/stats/.
> >>
> >> Mhh, I don't think I can follow you here. Even if the package is found
> >> in release-monitoring.org it doesn't map to any CVEs/CPEs, does it?
> >> Or should this be an example on already existing fuzzy matching in
> >> buildroot?
> >
> > Of course release-monitoring.org is completely unrelated from
> > CVEs/CPEs. I was mentioning release-monitoring.org as another case in
> > Buildroot where we have an issue with mapping Buildroot package names
> > with package/project names used by another entity.
> >
> > But now that I think of it, why not rely a bit more on
> > release-monitoring.org after all? We could add a NVD distribution to
> > release-monitoring.org, and use it to store the mapping between
> > release-monitoring.org projects and their corresponding CPE IDs.
> >
> > Then, Buildroot can query release-monitoring.org to find the CPE IDs.
> > One advantage of this approach is that this CPE ID information in
> > release-monitoring.org is not only useful for Buildroot, but
> > potentially for any other project using release-monitoring.org.
>
> Haha, actually Heiko had the same idea ;) But I'm not in favor of
> relying on too many external stuff and putting in one more layer
> of indirection. But thats just my opinion.
>
> But even then you'd have to map the buildroot packages to
> release-monitoring.org URLs (which we have to have anyway, I know ;).
>
> But.. once release-monitors.org would have CPE mappings, we could just
> pull the information to help creating our own mapping. I mean the CPEs
> won't change anymore and could be easily added to the recipes.
>
> >> > On the other hand, as explained above, the CPE information is not just
> >> > a package name, it's also a vendor name. So in practice the default of
> >> > using the package name doesn't really work.
> >>
> >> correct. unless you only match only parts of the CPE. But yes. If
> >> there
> >> would be a variable pkg_CPE which should contain the correct CPE, that
> >> would not work with the default.
> >
> > Right.
> >
> >> > One question is how will you distinguish packages that don't have any
> >> > CPE information because no CVE has ever been reported and therefore we
> >> > don't know which CPE vendor/product IDs will be used, and packages that
> >> > don't have CPE information because nobody has done the work of checking
> >> > the NVD database and adding the mapping.
> >>
> >> IMHO you cannot distiguish between these two, can you? Only if there
> >> would be another CPE database for all software out there. Otherwise if
> >> there
> >> is no CPE without and actual CVE, you can't do anything about it.
> >
> > But then if we cannot distinguish between these two, it means there is
> > no point in forcing to have a <pkg>_CPE variable in each package. We
> > could rely on the default being the package name, as we're doing today,
> > and simply add <pkg>_CPE to override that default.
>
> The point here is that in the good case you have the information if
> actually
> anyone looked into it; eg. changed PPPD_CPE from pppd (the default) to
> PPPD_CPE = point-to-point_protocol. So a script can find out what
> packages
> are not looked at. And yes, in the case there is no CPE yet it would be
> the
> same as you didn't looked at it yet. But thats another problem IMHO. Eg
> you
> could tag a package with a date you looked at it last time and found no
> CPE.
>
> You might have both worlds. Eg. have the default and tell the script if
> someone looked at it: allow setting the "pkg_CPE = pkg" in the .mk (eg.
> the same value as the default). So you can find out in which zone you
> are.
> If you have pkg_CPE in the .mk you're either in the red or green zone.
> If
> you don't have the pkg_CPE variable, then the default applies and you're
> in
> the yellow zone (even if a CVE applies!).
>
> > Your initial statement was:
> >
> >>> BUT in my optinion there MUST NOT be a default value for this
> >>> mapping,
> >>> because then you cannot distiguish between a package which just
> >>> uses the default name and a package which wasn't ever touched
> >>> regarding CVEs.
> >
> > However, we will anyway never be able to distinguish between a package
> > that does not have a <pkg>_CPE variable because there is no known CPE
> > ID for this package, or because nobody did the effort.
>
> Correct.
>
> >> So the
> >> package would still be unknown. A counterexample, let's assume you add
> >
> >>
> >>   pkg_CPE = <unknown>
> >>
> >> to indicate you've manually checked that there is currently no CVE
> >> matching this package and thus no CPE exists. Now, if there is
> >> actually
> >> a CVE/CPE the recipe won't be updated or worse the CVE matching would
> >> report nothing.
> >
> > Indeed, that makes the whole <pkg>_CPE = <unknown> idea a bit useless.
> >
> >> So I don't think you can do anything about that other than just saying
> >> we don't know anything about this package.
> >>
> >> > Indeed, in our 2500+ packages, I'm pretty sure a significant number of
> >> > them never had any CVE, and therefore we have no idea what their CPE ID
> >> > would be.
> >> >
> >> > Any idea on this ?
> >>
> >> Well I'd imaging three states in the stats:
> >>   - green, no CVE applies, we know the CPE
> >>   - red, CVE applies, we know the CPE
> >>   - yellow, CPE unknown, for whatever reason
>
> So taking the default from above we'd also have forth one:
> - yellow, CPE guessed, CVE applies.
>
> >> If you're right with "a significant number of them never had any CVE",
> >> then many would be yellow. But I don't see any drawback on this. At
> >> the
> >> of the day what really matters are the packages in the red category.
> >
> > No, because the yellow category could very well be "there are some
> > serious CVEs potentially affecting some packages, but nobody has added
> > the CPE ID information for the moment". So in the end, this yellow zone
> > is as dangerous as the red zone.
>
> Correct, but it would be an easy step to convert a package from the
> yellow
> zone to the red zone. Eg. just verify that this CVE maps to the package,
> then add pkg_CPE to the recipe. So ultimately there shouldn't be a CVE
> matching in the yellow zone. Only packages where no CVEs match. So for
> packages which have a CVE and the "CPE" matches the buildroot package
> name, this would be an easy task.
>
> Going through all the other packages which are yellow but no CVE matches
> is a more daunting task. And yes, if no CPE is found for now, this
> information should also be stored somewhere.
>
> >> And of course, how we can get as many packages as possibe from yellow
> >> to green/red.
> >>
> >> What could also be done is a seperate script which loops over the CVEs
> >> and prints if no mapping to a buildroot package is found. So someone
> >> could look at it and say, oh this CVE looks like it should match one
> >> of our packages.
> >
> > This is going to be horrible: the NVD database contains CVEs for
> > zillions of proprietary products that are completely irrelevant to
> > Buildroot. Looking through all of this data is going to be really
> > atrocious.
>
> I see. I guess there is no matching for "!= commercial"? ;)
>

Just to go on with this I did a proof of concept for the CVE mapping.

A first version can be found here:
https://github.com/hthiery/buildroot/tree/feature-cve-map

What this change does:
Add a new file format for the CVE mapping: <pkg>.cve. This is done in
YAML format. In this file at least the fields vendor and product has
to be defined. With this we can create a map from the CVE to the
buildroot package name.
Now we can decide the following:
1) the package has a mapping and is affected by a CVE  (RED ZONE)
2) the package has a mapping and a CVE is found but fixed (GREEN ZONE)
3) the package has a mapping and no CVE is found (GREEN ZONE)
4) the package has no mapping and it is affected by CVE found by guess
(try to compare buildroot name vs. CVE product name value) (YELLOW
ZONE)
5) the package has no mapping and no CVE is found  (we don't konw if
there isn't a CVE or we have no match because the buildroot name does
not match the CVE product name) (GREY ZONE)

Here is an example of a yaml file: package/pppd/ppd.cve
--- snip ---
vendor: point-to-point_protocol_project
product: point-to-point_protocol

cves:
  CVE-2020-8597:
    patch: 0001-pppd-Fix-bounds-check.patch

--- snip ---

To take the decision if a CVE is fixed there could be more possible
reasons instead of the patch in the example above. I can image
something like 'does-not-apply'.

I think the effort to have an extra file is not so much and it will
not pollute the Makefile with variables that are not build related.
For checking the YAML syntax an extra pkg-check can be implemented.

As mentioned at the beginning this is first idea how a mapping could
look. What do you think?

-- 
Heiko

[Buildroot] CVEs not matching buildroot packages

[Thomas Petazzoni]

Hello,

On Wed, 18 Mar 2020 14:57:27 +0100
Heiko Thiery <heiko.thiery@gmail.com> wrote:

> Just to go on with this I did a proof of concept for the CVE mapping.
> 
> A first version can be found here:
> https://github.com/hthiery/buildroot/tree/feature-cve-map
> 
> What this change does:
> Add a new file format for the CVE mapping: <pkg>.cve. This is done in
> YAML format. In this file at least the fields vendor and product has
> to be defined. With this we can create a map from the CVE to the
> buildroot package name.
> Now we can decide the following:
> 1) the package has a mapping and is affected by a CVE  (RED ZONE)
> 2) the package has a mapping and a CVE is found but fixed (GREEN ZONE)
> 3) the package has a mapping and no CVE is found (GREEN ZONE)
> 4) the package has no mapping and it is affected by CVE found by guess
> (try to compare buildroot name vs. CVE product name value) (YELLOW
> ZONE)
> 5) the package has no mapping and no CVE is found  (we don't konw if
> there isn't a CVE or we have no match because the buildroot name does
> not match the CVE product name) (GREY ZONE)
> 
> Here is an example of a yaml file: package/pppd/ppd.cve
> --- snip ---
> vendor: point-to-point_protocol_project
> product: point-to-point_protocol
> 
> cves:
>   CVE-2020-8597:
>     patch: 0001-pppd-Fix-bounds-check.patch
> 
> --- snip ---
> 
> To take the decision if a CVE is fixed there could be more possible
> reasons instead of the patch in the example above. I can image
> something like 'does-not-apply'.
> 
> I think the effort to have an extra file is not so much and it will
> not pollute the Makefile with variables that are not build related.
> For checking the YAML syntax an extra pkg-check can be implemented.
> 
> As mentioned at the beginning this is first idea how a mapping could
> look. What do you think?

I'll give my very quick and probably very blunt and brutal feedback: I
don't like having another file with package metadata. This metadata
should be in the .mk file.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] CVEs not matching buildroot packages

[Heiko Thiery]

Hi,


Am Mi., 18. M?rz 2020 um 15:12 Uhr schrieb Thomas Petazzoni
<thomas.petazzoni@bootlin.com>:
>
> Hello,
>
> On Wed, 18 Mar 2020 14:57:27 +0100
> Heiko Thiery <heiko.thiery@gmail.com> wrote:
>
> > Just to go on with this I did a proof of concept for the CVE mapping.
> >
> > A first version can be found here:
> > https://github.com/hthiery/buildroot/tree/feature-cve-map
> >
> > What this change does:
> > Add a new file format for the CVE mapping: <pkg>.cve. This is done in
> > YAML format. In this file at least the fields vendor and product has
> > to be defined. With this we can create a map from the CVE to the
> > buildroot package name.
> > Now we can decide the following:
> > 1) the package has a mapping and is affected by a CVE  (RED ZONE)
> > 2) the package has a mapping and a CVE is found but fixed (GREEN ZONE)
> > 3) the package has a mapping and no CVE is found (GREEN ZONE)
> > 4) the package has no mapping and it is affected by CVE found by guess
> > (try to compare buildroot name vs. CVE product name value) (YELLOW
> > ZONE)
> > 5) the package has no mapping and no CVE is found  (we don't konw if
> > there isn't a CVE or we have no match because the buildroot name does
> > not match the CVE product name) (GREY ZONE)
> >
> > Here is an example of a yaml file: package/pppd/ppd.cve
> > --- snip ---
> > vendor: point-to-point_protocol_project
> > product: point-to-point_protocol
> >
> > cves:
> >   CVE-2020-8597:
> >     patch: 0001-pppd-Fix-bounds-check.patch
> >
> > --- snip ---
> >
> > To take the decision if a CVE is fixed there could be more possible
> > reasons instead of the patch in the example above. I can image
> > something like 'does-not-apply'.
> >
> > I think the effort to have an extra file is not so much and it will
> > not pollute the Makefile with variables that are not build related.
> > For checking the YAML syntax an extra pkg-check can be implemented.
> >
> > As mentioned at the beginning this is first idea how a mapping could
> > look. What do you think?
>
> I'll give my very quick and probably very blunt and brutal feedback: I
> don't like having another file with package metadata. This metadata
> should be in the .mk file.

This is really brutal .. Just kidding ;-)

Don't you think more variables (CPE_VENDOR, CPE_PRODUCT, CVE_IGNORED
..) in the Makefile will slow down the parsing? The other thing is
that IMHO valuable information about why a CVE is ignored (fixed, does
not apply to the build configuration ... ) will be lost or has to
added as comment in the Makefile.

On the other hand I understand your concern about having another file
with meta information.

-- 
Heiko

[Buildroot] CVEs not matching buildroot packages

[Matthew Weber]

Heiko,



On Wed, Mar 18, 2020 at 9:35 AM Heiko Thiery <heiko.thiery@gmail.com> wrote:
>
> Hi,
>
>
> Am Mi., 18. M?rz 2020 um 15:12 Uhr schrieb Thomas Petazzoni
> <thomas.petazzoni@bootlin.com>:
> >
> > Hello,
> >
> > On Wed, 18 Mar 2020 14:57:27 +0100
> > Heiko Thiery <heiko.thiery@gmail.com> wrote:
> >
> > > Just to go on with this I did a proof of concept for the CVE mapping.
> > >
> > > A first version can be found here:
> > > https://github.com/hthiery/buildroot/tree/feature-cve-map
> > >
> > > What this change does:
> > > Add a new file format for the CVE mapping: <pkg>.cve. This is done in
> > > YAML format. In this file at least the fields vendor and product has
> > > to be defined. With this we can create a map from the CVE to the
> > > buildroot package name.
> > > Now we can decide the following:
> > > 1) the package has a mapping and is affected by a CVE  (RED ZONE)
> > > 2) the package has a mapping and a CVE is found but fixed (GREEN ZONE)
> > > 3) the package has a mapping and no CVE is found (GREEN ZONE)
> > > 4) the package has no mapping and it is affected by CVE found by guess
> > > (try to compare buildroot name vs. CVE product name value) (YELLOW
> > > ZONE)
> > > 5) the package has no mapping and no CVE is found  (we don't konw if
> > > there isn't a CVE or we have no match because the buildroot name does
> > > not match the CVE product name) (GREY ZONE)
> > >
> > > Here is an example of a yaml file: package/pppd/ppd.cve
> > > --- snip ---
> > > vendor: point-to-point_protocol_project
> > > product: point-to-point_protocol
> > >
> > > cves:
> > >   CVE-2020-8597:
> > >     patch: 0001-pppd-Fix-bounds-check.patch
> > >
> > > --- snip ---
> > >
> > > To take the decision if a CVE is fixed there could be more possible
> > > reasons instead of the patch in the example above. I can image
> > > something like 'does-not-apply'.
> > >
> > > I think the effort to have an extra file is not so much and it will
> > > not pollute the Makefile with variables that are not build related.
> > > For checking the YAML syntax an extra pkg-check can be implemented.
> > >
> > > As mentioned at the beginning this is first idea how a mapping could
> > > look. What do you think?
> >
> > I'll give my very quick and probably very blunt and brutal feedback: I
> > don't like having another file with package metadata. This metadata
> > should be in the .mk file.
>
> This is really brutal .. Just kidding ;-)
>
> Don't you think more variables (CPE_VENDOR, CPE_PRODUCT, CVE_IGNORED
> ..) in the Makefile will slow down the parsing? The other thing is
> that IMHO valuable information about why a CVE is ignored (fixed, does
> not apply to the build configuration ... ) will be lost or has to
> added as comment in the Makefile.
>
> On the other hand I understand your concern about having another file
> with meta information.

If you have a chance, please take a look at the CPE / CVE series just
posted that starts to work on the problem of CPE maintenance and
adding CVE to CPE matching to the existing reporting.

Cover letter:  http://lists.busybox.net/pipermail/buildroot/2020-October/293931.html

http://patchwork.ozlabs.org/project/buildroot/list/?series=206245


Best Regards
Matt

[Buildroot] CVEs not matching buildroot packages

[Akshay Bhat]

Hi Michael, Thomas,

> On Tue, 25 Feb 2020 11:19:20 +0100
> Michael Walle <michael@walle.cc> wrote:
>> You could assume that
>> the product name is unique and if its not you could add the vendor.
>> Like for example:
>>
>>   _CPE = linux_kernel
>>   _CPE = linux:linux_kernel
>>
>> eg. something like that:
>>   pkg_CPE = [vendor:]product
>>
>> Maybe it shouldn't be named CPE after all.. But I wanted to avoid
>> this discussion for now, because it doesn't matter for the mapping
>> problem ;)

Few things to consider:
1. The proposed pkg_CPE product will need to map to multiple NVD CPE products.
Example 1: CVE can be reported against a library or utility
curl (CVE-2019-15601 cpe:2.3:a:haxx:curl), libcurl (CVE-2019-3823
cpe:2.3:a:haxx:libcurl)
So the CVE check logic for curl package in buildroot would need to
iterate through CVE's for both curl and libcurl CPE product entries...
Same goes for packages like flac/libflac, I can help provide a more
exhaustive list at a later point.
Example 2: Naming inconsistencies in NVD (Same package having multiple
CPE product entries)
simple_directmedia_layer(CVE-2019-12222
cpe:2.3:a:libsdl:simple_directmedia_layer), sdl (CVE-2017-2888
cpe:2.3:a:libsdl:sdl:2.0.5), libsdl (CVE-2019-13626
cpe:2.3:a:libsdl:libsdl)
arm-trusted-firmware (CVE-2017-15031
cpe:2.3:o:arm:arm-trusted-firmware:*:*:*:*:*:*:*:*),
arm_trusted_firmware (CVE-2016-10319
cpe:2.3:o:arm_trusted_firmware_project:arm_trusted_firmware:1.2:*:*:*:*:*:*:*),
trusted_firmware-a (CVE-2018-19440
cpe:2.3:a:arm:trusted_firmware-a:*:*:*:*:*:*:*:*)
....
Though ideally some of these should be fixed by sending NVD an email.

2. Being able to specify the vendor would be nice to have. In my
experience it reduces false positives in certain packages but then
again it leads to missed CVEs (naming inconsistencies in the CVE
vendor) in certain other packages. So specifying it on a need only
basis would be ideal.
Example 1: Reduced false positive because multiple vendors might have
same product
subversion (CVE-2020-2111 cpe:2.3:a:jenkins:subversion vs
CVE-2019-0203 cpe:2.3:a:apache:subversion)

3. An option to specify CVE version would be helpful for certain usecases
Example 1: Boards might use a git tag for the version number that is
unrelated to upstream version numbering scheme.
Most of the i.MX boards (eg: freescale_imx7dsabresd_defconfig) set
incorrect version number for u-boot (eg:
BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION="rel_imx_4.9.x_1.0.0_ga")
This leads to missed CVE reporting in u-boot package.
Example 2: NVD might use a different versioning scheme than upstream
or buildroot
libkrb5 buildroot version: 1.18; NVD version format 5-x.xx
(CVE-2019-14844 5-1.17)
berkeleydb buildroot version: 5.3.28; NVD version format 11.2.5.3.28
(CVE-2015-4790)

See ELC presentation slides 17/22/23 for more info on NVD data quality
analysis (based on Timesys's experience developing CVE monitoring tool
called Vigiles):
https://elinux.org/images/0/0a/Open-Source-CVE-Monitoring-and-Management-V3.pdf

>> > Yes. With the release-monitoring.org matching, we are able to know if
>> > the match was found thanks to an explicit mapping of the Buildroot
>> > distribution, or if it was found by "luck". That's the difference
>> > between "found by distro" and "found by guess" in
>> > http://autobuild.buildroot.net/stats/.
>>
>> Mhh, I don't think I can follow you here. Even if the package is found
>> in release-monitoring.org it doesn't map to any CVEs/CPEs, does it?
>> Or should this be an example on already existing fuzzy matching in
>> buildroot?
>
> Of course release-monitoring.org is completely unrelated from
> CVEs/CPEs. I was mentioning release-monitoring.org as another case in
> Buildroot where we have an issue with mapping Buildroot package names
> with package/project names used by another entity.
>
> But now that I think of it, why not rely a bit more on
> release-monitoring.org after all? We could add a NVD distribution to
> release-monitoring.org, and use it to store the mapping between
> release-monitoring.org projects and their corresponding CPE IDs.
>
> Then, Buildroot can query release-monitoring.org to find the CPE IDs.
> One advantage of this approach is that this CPE ID information in
> release-monitoring.org is not only useful for Buildroot, but
> potentially for any other project using release-monitoring.org.

This is a great idea for long term collaboration between distros! I am
not very familiar with the release-monitoring.org mapping between
distros but one idea for an initial first pass to identify *potential*
packages having a different NVD CPE product name:
1. Write a script to generate a list of packages with no CVEs reported
for any version of the package
2. Map the buildroot package in release-monitoring.org to say
Debian/Ubuntu and check their security trackers
(https://salsa.debian.org/security-tracker-team/security-tracker/tree/master/data
or https://git.launchpad.net/ubuntu-cve-tracker/tree/) for any CVEs
for that package. If CVEs exist then most likely the buildroot package
name to NVD CPE product name is mis-matched.

PS: I just subscribed to mailing list, so had to copy paste the email
thread from web archive. So formatting and In-Reply-To context might
be messed up, sorry!

Thanks,
Akshay