[master] [PATCH] cve-update-nvd2-native: faster requests with API keys

[master] [PATCH] cve-update-nvd2-native: faster requests with API keys

[Dhairya Nagodra]

As per NVD, the public rate limit is 5 requests in 30s (6s delay).
Using an API key increases the limit to 50 requests in 30s (0.6s delay).
However, NVD still recommends sleeping for several seconds so that the
other legitimate requests are serviced without denial or interruption.
Keeping the default sleep at 6 seconds and 2 seconds with an API key.

For failures, the wait time is unchanged (6 seconds).

Reference: https://nvd.nist.gov/developers/start-here#RateLimits

Signed-off-by: Dhairya Nagodra <dnagodra@cisco.com>
---
 meta/recipes-core/meta/cve-update-nvd2-native.bb | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb
index 9ab8dc6050..941fca34c6 100644
--- a/meta/recipes-core/meta/cve-update-nvd2-native.bb
+++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb
@@ -188,6 +188,11 @@ def update_db_file(db_tmp_file, d, database_time):
         api_key = d.getVar("NVDCVE_API_KEY") or None
         attempts = int(d.getVar("CVE_DB_UPDATE_ATTEMPTS"))
 
+        # Recommended by NVD
+        wait_time = 6
+        if api_key:
+            wait_time = 2
+
         while True:
             req_args['startIndex'] = index
             raw_data = nvd_request_next(url, attempts, api_key, req_args)
@@ -210,7 +215,7 @@ def update_db_file(db_tmp_file, d, database_time):
                break
 
             # Recommended by NVD
-            time.sleep(6)
+            time.sleep(wait_time)
 
         # Update success, set the date to cve_check file.
         cve_f.write('CVE database update : %s\n\n' % datetime.date.today())
-- 
2.35.6

Re: [OE-core] [master] [PATCH] cve-update-nvd2-native: faster requests with API keys

[Alexandre Belloni]

Please follow https://docs.yoctoproject.org/dev/contributor-guide/submit-changes.html#fixing-your-from-identity

On 07/12/2023 18:53:22-0800, Dhairya Nagodra via lists.openembedded.org wrote:
> As per NVD, the public rate limit is 5 requests in 30s (6s delay).
> Using an API key increases the limit to 50 requests in 30s (0.6s delay).
> However, NVD still recommends sleeping for several seconds so that the
> other legitimate requests are serviced without denial or interruption.
> Keeping the default sleep at 6 seconds and 2 seconds with an API key.
> 
> For failures, the wait time is unchanged (6 seconds).
> 
> Reference: https://nvd.nist.gov/developers/start-here#RateLimits
> 
> Signed-off-by: Dhairya Nagodra <dnagodra@cisco.com>
> ---
>  meta/recipes-core/meta/cve-update-nvd2-native.bb | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb
> index 9ab8dc6050..941fca34c6 100644
> --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb
> +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb
> @@ -188,6 +188,11 @@ def update_db_file(db_tmp_file, d, database_time):
>          api_key = d.getVar("NVDCVE_API_KEY") or None
>          attempts = int(d.getVar("CVE_DB_UPDATE_ATTEMPTS"))
>  
> +        # Recommended by NVD
> +        wait_time = 6
> +        if api_key:
> +            wait_time = 2
> +
>          while True:
>              req_args['startIndex'] = index
>              raw_data = nvd_request_next(url, attempts, api_key, req_args)
> @@ -210,7 +215,7 @@ def update_db_file(db_tmp_file, d, database_time):
>                 break
>  
>              # Recommended by NVD
> -            time.sleep(6)
> +            time.sleep(wait_time)
>  
>          # Update success, set the date to cve_check file.
>          cve_f.write('CVE database update : %s\n\n' % datetime.date.today())
> -- 
> 2.35.6
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#192008): https://lists.openembedded.org/g/openembedded-core/message/192008
> Mute This Topic: https://lists.openembedded.org/mt/103048465/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com