[Buildroot] [PATCH] support/scripts/pkg-stats: strengthen version check in check_package_get_latest_version_by_distro()

[Buildroot] [PATCH] support/scripts/pkg-stats: strengthen version check in check_package_get_latest_version_by_distro()

[Thomas Petazzoni via buildroot]

The check_package_get_latest_version_by_distro() function analyzes the
data returned by release-monitoring.org. For two of our
packages (bento4 and qextserialport), release-monitoring.org returns
something that is a bit odd: it returns an entry with a
"stable_versions" field that contains an empty array. Our code was
ready to have or not have a "stable_versions" entry, but when it is
present, we assumed it was not an empty array. These two packages, for
some reason, break this assumption.

In order to solve this problem, this commit is more careful, and uses
the stable_versions field only if it exists and it has at least one
entry. The code is also reworked as a sequence of "if...elif...else"
to be more readable.

This fixes the following exception when running pkg-stats on the full
package set:

Task exception was never retrieved
future: <Task finished name='Task-10772' coro=<check_package_latest_version_get() done, defined at ./support/scripts/pkg-stats:532> exception=IndexError('list index out of range')>
Traceback (most recent call last):
  File "./support/scripts/pkg-stats", line 535, in check_package_latest_version_get
    if await check_package_get_latest_version_by_distro(session, pkg):
  File "./support/scripts/pkg-stats", line 489, in check_package_get_latest_version_by_distro
    version = data['stable_versions'][0] if 'stable_versions' in data else data['version'] if 'version' in data else None
IndexError: list index out of range

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 support/scripts/pkg-stats | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats
index c235d99407..f57041cee4 100755
--- a/support/scripts/pkg-stats
+++ b/support/scripts/pkg-stats
@@ -486,7 +486,12 @@ async def check_package_get_latest_version_by_distro(session, pkg, retry=True):
                 return False
 
             data = await resp.json()
-            version = data['stable_versions'][0] if 'stable_versions' in data else data['version'] if 'version' in data else None
+            if 'stable_versions' in data and len(data['stable_versions']) > 0:
+                version = data['stable_versions'][0]
+            elif 'version' in data:
+                version = data['version']
+            else:
+                version = None
             check_package_latest_version_set_status(pkg,
                                                     RM_API_STATUS_FOUND_BY_DISTRO,
                                                     version,
-- 
2.35.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] support/scripts/pkg-stats: strengthen version check in check_package_get_latest_version_by_distro()

[Yann E. MORIN]

Thomas, All,

On 2022-03-16 22:48 +0100, Thomas Petazzoni spake thusly:
> The check_package_get_latest_version_by_distro() function analyzes the
> data returned by release-monitoring.org. For two of our
> packages (bento4 and qextserialport), release-monitoring.org returns
> something that is a bit odd: it returns an entry with a
> "stable_versions" field that contains an empty array. Our code was
> ready to have or not have a "stable_versions" entry, but when it is
> present, we assumed it was not an empty array. These two packages, for
> some reason, break this assumption.
> 
> In order to solve this problem, this commit is more careful, and uses
> the stable_versions field only if it exists and it has at least one
> entry. The code is also reworked as a sequence of "if...elif...else"
> to be more readable.
> 
> This fixes the following exception when running pkg-stats on the full
> package set:
> 
> Task exception was never retrieved
> future: <Task finished name='Task-10772' coro=<check_package_latest_version_get() done, defined at ./support/scripts/pkg-stats:532> exception=IndexError('list index out of range')>
> Traceback (most recent call last):
>   File "./support/scripts/pkg-stats", line 535, in check_package_latest_version_get
>     if await check_package_get_latest_version_by_distro(session, pkg):
>   File "./support/scripts/pkg-stats", line 489, in check_package_get_latest_version_by_distro
>     version = data['stable_versions'][0] if 'stable_versions' in data else data['version'] if 'version' in data else None
> IndexError: list index out of range
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
>  support/scripts/pkg-stats | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats
> index c235d99407..f57041cee4 100755
> --- a/support/scripts/pkg-stats
> +++ b/support/scripts/pkg-stats
> @@ -486,7 +486,12 @@ async def check_package_get_latest_version_by_distro(session, pkg, retry=True):
>                  return False
>  
>              data = await resp.json()
> -            version = data['stable_versions'][0] if 'stable_versions' in data else data['version'] if 'version' in data else None
> +            if 'stable_versions' in data and len(data['stable_versions']) > 0:

PEP8 says that we can just test the sequence to test for emptyness:
    https://pep8.org/#programming-recommendations  (toward the end).

Applied to master with that fixed, thanks.

Regards,
Yann E. MORIN.

> +                version = data['stable_versions'][0]
> +            elif 'version' in data:
> +                version = data['version']
> +            else:
> +                version = None
>              check_package_latest_version_set_status(pkg,
>                                                      RM_API_STATUS_FOUND_BY_DISTRO,
>                                                      version,
> -- 
> 2.35.1
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] support/scripts/pkg-stats: strengthen version check in check_package_get_latest_version_by_distro()

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni via buildroot <buildroot@buildroot.org> writes:

 > The check_package_get_latest_version_by_distro() function analyzes the
 > data returned by release-monitoring.org. For two of our
 > packages (bento4 and qextserialport), release-monitoring.org returns
 > something that is a bit odd: it returns an entry with a
 > "stable_versions" field that contains an empty array. Our code was
 > ready to have or not have a "stable_versions" entry, but when it is
 > present, we assumed it was not an empty array. These two packages, for
 > some reason, break this assumption.

 > In order to solve this problem, this commit is more careful, and uses
 > the stable_versions field only if it exists and it has at least one
 > entry. The code is also reworked as a sequence of "if...elif...else"
 > to be more readable.

 > This fixes the following exception when running pkg-stats on the full
 > package set:

 > Task exception was never retrieved
 > future: <Task finished name='Task-10772' coro=<check_package_latest_version_get() done, defined at ./support/scripts/pkg-stats:532> exception=IndexError('list index out of range')>
 > Traceback (most recent call last):
 >   File "./support/scripts/pkg-stats", line 535, in check_package_latest_version_get
 >     if await check_package_get_latest_version_by_distro(session, pkg):
 >   File "./support/scripts/pkg-stats", line 489, in check_package_get_latest_version_by_distro
 >     version = data['stable_versions'][0] if 'stable_versions' in data else data['version'] if 'version' in data else None
 > IndexError: list index out of range

 > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>

Committed to 2021.02.x, 2021.11.x and 2022.02.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot