[Buildroot] syslog-ng failures related to python modules

[Buildroot] syslog-ng failures related to python modules

[Chris Packham]

Hi All,

I've had some time to look into failures like

>  powerpc64   |        syslog-ng-4.1.1         | http://autobuild.buildroot.net/results/e9d3aee88c6da2a490c349e218adbaa5ea826d27

The failure is

WARNING: There was an error checking the latest version of pip.
WARNING: pip is configured with locations that require TLS/SSL,
however the ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None,
redirect=None, status=None)) after connection broken by
'SSLError("Can't connect to HTTPS URL because the SSL module is not
available.")': /simple/cachetools/
WARNING: Retrying (Retry(total=3, connect=None, read=None,
redirect=None, status=None)) after connection broken by
'SSLError("Can't connect to HTTPS URL because the SSL module is not
available.")': /simple/cachetools/
WARNING: Retrying (Retry(total=2, connect=None, read=None,
redirect=None, status=None)) after connection broken by
'SSLError("Can't connect to HTTPS URL because the SSL module is not
available.")': /simple/cachetools/
WARNING: Retrying (Retry(total=1, connect=None, read=None,
redirect=None, status=None)) after connection broken by
'SSLError("Can't connect to HTTPS URL because the SSL module is not
available.")': /simple/cachetools/
WARNING: Retrying (Retry(total=0, connect=None, read=None,
redirect=None, status=None)) after connection broken by
'SSLError("Can't connect to HTTPS URL because the SSL module is not
available.")': /simple/cachetools/
Could not fetch URL https://pypi.org/simple/cachetools/: There was a
problem confirming the ssl certificate:
HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded
with url: /simple/cachetools/ (Caused by SSLError("Can't connect to
HTTPS URL because the SSL module is not available.")) - skipping
ERROR: Could not find a version that satisfies the requirement
cachetools==4.2.4 (from versions: none)
ERROR: No matching distribution found for cachetools==4.2.4
WARNING: pip is configured with locations that require TLS/SSL,
however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem
confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org',
port=443): Max retries exceeded with url: /simple/pip/ (Caused by
SSLError("Can't connect to HTTPS URL because the SSL module is not
available.")) - skipping


The basic problem is that the newer version of syslog-ng has some
additional python dependencies but its install process can't resolve
the dependencies.

I can see the problem with the following (after a failed build)

$ . output/build/syslog-ng-4.1.1/venv/bin/activate
$ python3 -m ssl
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/chrisp/src/buildroot/output/host/lib/python3.11/ssl.py",
line 100, in <module>
    import _ssl             # if we can't import it, let the error propagate
    ^^^^^^^^^^^
ModuleNotFoundError: No module named '_ssl'

It looks like something (not sure what) has conveniently setup a host
python wrapper (this config doesn't build host python3) but however it
was populated it did not include the _ssl module (which looking on my
system is a dynamically loaded .so). I'm a little unsure as to how to
proceed with this. I could unconditionally disable python support in
syslog-ng or make it depend on host python3 but I suspect this isn't
the only package that is doing such things so I thought there might be
a better solution.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] syslog-ng failures related to python modules

[Peter Korsgaard]

>>>>> "Chris" == Chris Packham <judge.packham@gmail.com> writes:

 > Hi All,
 > I've had some time to look into failures like

 >> powerpc64   |        syslog-ng-4.1.1         | http://autobuild.buildroot.net/results/e9d3aee88c6da2a490c349e218adbaa5ea826d27

 > The failure is

 > WARNING: There was an error checking the latest version of pip.
 > WARNING: pip is configured with locations that require TLS/SSL,
 > however the ssl module in Python is not available.
 > WARNING: Retrying (Retry(total=4, connect=None, read=None,
 > redirect=None, status=None)) after connection broken by
 > 'SSLError("Can't connect to HTTPS URL because the SSL module is not
 > available.")': /simple/cachetools/
 > WARNING: Retrying (Retry(total=3, connect=None, read=None,
 > redirect=None, status=None)) after connection broken by
 > 'SSLError("Can't connect to HTTPS URL because the SSL module is not
 > available.")': /simple/cachetools/
 > WARNING: Retrying (Retry(total=2, connect=None, read=None,
 > redirect=None, status=None)) after connection broken by
 > 'SSLError("Can't connect to HTTPS URL because the SSL module is not
 > available.")': /simple/cachetools/
 > WARNING: Retrying (Retry(total=1, connect=None, read=None,
 > redirect=None, status=None)) after connection broken by
 > 'SSLError("Can't connect to HTTPS URL because the SSL module is not
 > available.")': /simple/cachetools/
 > WARNING: Retrying (Retry(total=0, connect=None, read=None,
 > redirect=None, status=None)) after connection broken by
 > 'SSLError("Can't connect to HTTPS URL because the SSL module is not
 > available.")': /simple/cachetools/
 > Could not fetch URL https://pypi.org/simple/cachetools/: There was a
 > problem confirming the ssl certificate:
 > HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded
 > with url: /simple/cachetools/ (Caused by SSLError("Can't connect to
 > HTTPS URL because the SSL module is not available.")) - skipping
 > ERROR: Could not find a version that satisfies the requirement
 > cachetools==4.2.4 (from versions: none)
 > ERROR: No matching distribution found for cachetools==4.2.4
 > WARNING: pip is configured with locations that require TLS/SSL,
 > however the ssl module in Python is not available.
 > Could not fetch URL https://pypi.org/simple/pip/: There was a problem
 > confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org',
 > port=443): Max retries exceeded with url: /simple/pip/ (Caused by
 > SSLError("Can't connect to HTTPS URL because the SSL module is not
 > available.")) - skipping


 > The basic problem is that the newer version of syslog-ng has some
 > additional python dependencies but its install process can't resolve
 > the dependencies.

 > I can see the problem with the following (after a failed build)

 > $ . output/build/syslog-ng-4.1.1/venv/bin/activate
 > $ python3 -m ssl
 > Traceback (most recent call last):
 >   File "<frozen runpy>", line 198, in _run_module_as_main
 >   File "<frozen runpy>", line 88, in _run_code
 >   File "/home/chrisp/src/buildroot/output/host/lib/python3.11/ssl.py",
 > line 100, in <module>
 >     import _ssl             # if we can't import it, let the error propagate
 >     ^^^^^^^^^^^
 > ModuleNotFoundError: No module named '_ssl'

 > It looks like something (not sure what) has conveniently setup a host
 > python wrapper (this config doesn't build host python3)

BR2_PACKAGE_PYTHON3 implies host-python3.

 > but however it was populated it did not include the _ssl module
 > (which looking on my system is a dynamically loaded .so). I'm a
 > little unsure as to how to proceed with this. I could unconditionally
 > disable python support in syslog-ng or make it depend on host python3
 > but I suspect this isn't the only package that is doing such things
 > so I thought there might be a better solution.

The fact that it is invoking pip to download extra dependencies makes it
sound as if we are missing some python dependencies. In the context of
Buildroot, a package should not invoke pip.

Looking at the code, it seems to come from scripts/build-python-venv.sh
which uses {dev-,}requirements.txt. It is not quite clear to me which
dependencies are needed at build time for the host and which are needed
at runtime for the target though.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot