[PATCH] tests/docker: Add sqlite3 module to openSUSE Leap container

[PATCH] tests/docker: Add sqlite3 module to openSUSE Leap container

[Fabiano Rosas]

Avocado needs sqlite3:

  Failed to load plugin from module "avocado.plugins.journal":
  ImportError("Module 'sqlite3' is not installed.
  Use: sudo zypper install python311 to install it")

Include the appropriate package in the dockerfile.

From 'zypper info python311':
  "This package supplies rich command line features provided by
  readline, and sqlite3 support for the interpreter core, thus forming
  a so called "extended" runtime."

Signed-off-by: Fabiano Rosas <farosas@suse.de>
---
 tests/docker/dockerfiles/opensuse-leap.docker | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/docker/dockerfiles/opensuse-leap.docker b/tests/docker/dockerfiles/opensuse-leap.docker
index dc0e36ce48..cf753383a4 100644
--- a/tests/docker/dockerfiles/opensuse-leap.docker
+++ b/tests/docker/dockerfiles/opensuse-leap.docker
@@ -90,6 +90,7 @@ RUN zypper update -y && \
            pcre-devel-static \
            pipewire-devel \
            pkgconfig \
+           python311 \
            python311-base \
            python311-pip \
            python311-setuptools \
-- 
2.35.3

Re: [PATCH] tests/docker: Add sqlite3 module to openSUSE Leap container

[Philippe Mathieu-Daudé]

Hi Fabiano,

On 17/1/24 00:09, Fabiano Rosas wrote:
> Avocado needs sqlite3:
> 
>    Failed to load plugin from module "avocado.plugins.journal":
>    ImportError("Module 'sqlite3' is not installed.
>    Use: sudo zypper install python311 to install it")
> 
> Include the appropriate package in the dockerfile.
> 
>  From 'zypper info python311':
>    "This package supplies rich command line features provided by
>    readline, and sqlite3 support for the interpreter core, thus forming
>    a so called "extended" runtime."
> 
> Signed-off-by: Fabiano Rosas <farosas@suse.de>
> ---
>   tests/docker/dockerfiles/opensuse-leap.docker | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/tests/docker/dockerfiles/opensuse-leap.docker b/tests/docker/dockerfiles/opensuse-leap.docker
> index dc0e36ce48..cf753383a4 100644
> --- a/tests/docker/dockerfiles/opensuse-leap.docker
> +++ b/tests/docker/dockerfiles/opensuse-leap.docker
> @@ -90,6 +90,7 @@ RUN zypper update -y && \
>              pcre-devel-static \
>              pipewire-devel \
>              pkgconfig \
> +           python311 \
>              python311-base \
>              python311-pip \
>              python311-setuptools \

See in this file header:

   # THIS FILE WAS AUTO-GENERATED
   #
   #  $ lcitool dockerfile --layers all opensuse-leap-15 qemu
   #
   # https://gitlab.com/libvirt/libvirt-ci

libvirt-ci maintains dependencies required to build QEMU,
in this case since it is a 'testing QEMU' dependency, you
might add it in the generate_dockerfile("opensuse-leap")
call in tests/lcitool/refresh (otherwise raise an issue
in libvirt-ci about it).

Regards,

Phil.

Re: [PATCH] tests/docker: Add sqlite3 module to openSUSE Leap container

[Philippe Mathieu-Daudé]

On 17/1/24 08:09, Philippe Mathieu-Daudé wrote:
> Hi Fabiano,
> 
> On 17/1/24 00:09, Fabiano Rosas wrote:
>> Avocado needs sqlite3:
>>
>>    Failed to load plugin from module "avocado.plugins.journal":
>>    ImportError("Module 'sqlite3' is not installed.
>>    Use: sudo zypper install python311 to install it")
>>
>> Include the appropriate package in the dockerfile.
>>
>>  From 'zypper info python311':
>>    "This package supplies rich command line features provided by
>>    readline, and sqlite3 support for the interpreter core, thus forming
>>    a so called "extended" runtime."
>>
>> Signed-off-by: Fabiano Rosas <farosas@suse.de>
>> ---
>>   tests/docker/dockerfiles/opensuse-leap.docker | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/tests/docker/dockerfiles/opensuse-leap.docker 
>> b/tests/docker/dockerfiles/opensuse-leap.docker
>> index dc0e36ce48..cf753383a4 100644
>> --- a/tests/docker/dockerfiles/opensuse-leap.docker
>> +++ b/tests/docker/dockerfiles/opensuse-leap.docker
>> @@ -90,6 +90,7 @@ RUN zypper update -y && \
>>              pcre-devel-static \
>>              pipewire-devel \
>>              pkgconfig \
>> +           python311 \
>>              python311-base \
>>              python311-pip \
>>              python311-setuptools \
> 
> See in this file header:
> 
>    # THIS FILE WAS AUTO-GENERATED
>    #
>    #  $ lcitool dockerfile --layers all opensuse-leap-15 qemu
>    #
>    # https://gitlab.com/libvirt/libvirt-ci
> 
> libvirt-ci maintains dependencies required to build QEMU,
> in this case since it is a 'testing QEMU' dependency, you
> might add it in the generate_dockerfile("opensuse-leap")
> call in tests/lcitool/refresh (otherwise raise an issue
> in libvirt-ci about it).

This is incorrect, correct fix in this thread:
https://lore.kernel.org/qemu-devel/CABJz62OD0ZZSY16sDNRiTuPEBygy9Y6rUgRrwoA6N_yg8cgGAQ@mail.gmail.com/

Re: [PATCH] tests/docker: Add sqlite3 module to openSUSE Leap container

[Thomas Huth]

On 17/01/2024 00.09, Fabiano Rosas wrote:
> Avocado needs sqlite3:
> 
>    Failed to load plugin from module "avocado.plugins.journal":
>    ImportError("Module 'sqlite3' is not installed.
>    Use: sudo zypper install python311 to install it")
> 
> Include the appropriate package in the dockerfile.
> 
>  From 'zypper info python311':
>    "This package supplies rich command line features provided by
>    readline, and sqlite3 support for the interpreter core, thus forming
>    a so called "extended" runtime."
> 
> Signed-off-by: Fabiano Rosas <farosas@suse.de>
> ---
>   tests/docker/dockerfiles/opensuse-leap.docker | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/tests/docker/dockerfiles/opensuse-leap.docker b/tests/docker/dockerfiles/opensuse-leap.docker
> index dc0e36ce48..cf753383a4 100644
> --- a/tests/docker/dockerfiles/opensuse-leap.docker
> +++ b/tests/docker/dockerfiles/opensuse-leap.docker
> @@ -90,6 +90,7 @@ RUN zypper update -y && \
>              pcre-devel-static \
>              pipewire-devel \
>              pkgconfig \
> +           python311 \
>              python311-base \
>              python311-pip \
>              python311-setuptools \

AFAIK tests/docker/dockerfiles/opensuse-leap.docker is auto-generated, so 
this will be lost once somebody else runs lcitool again...

I don't really have a clue, but I guess this has to be fixed in the upstream 
lcitool first ( https://gitlab.com/libvirt/libvirt-ci ), and then we need to 
update our lcitool status in QEMU afterwards. Maybe Daniel can advise for 
the right stteps here...?

  Thomas

Re: Re: [PATCH] tests/docker: Add sqlite3 module to openSUSE Leap container

[Andrea Bolognani]

On Wed, Jan 17, 2024 at 08:18:27AM +0100, Thomas Huth wrote:
> On 17/01/2024 00.09, Fabiano Rosas wrote:
> > Avocado needs sqlite3:
> >
> >    Failed to load plugin from module "avocado.plugins.journal":
> >    ImportError("Module 'sqlite3' is not installed.
> >    Use: sudo zypper install python311 to install it")
> >
> > Include the appropriate package in the dockerfile.
> >
> >  From 'zypper info python311':
> >    "This package supplies rich command line features provided by
> >    readline, and sqlite3 support for the interpreter core, thus forming
> >    a so called "extended" runtime."

Weird choice on Python's part to have sqlite3 support as part of the
standard library IMO, but that's "batteries included" for you :)

> > +++ b/tests/docker/dockerfiles/opensuse-leap.docker
> > @@ -90,6 +90,7 @@ RUN zypper update -y && \
> >              pcre-devel-static \
> >              pipewire-devel \
> >              pkgconfig \
> > +           python311 \
> >              python311-base \
> >              python311-pip \
> >              python311-setuptools \
>
> AFAIK tests/docker/dockerfiles/opensuse-leap.docker is auto-generated, so
> this will be lost once somebody else runs lcitool again...
>
> I don't really have a clue, but I guess this has to be fixed in the upstream
> lcitool first ( https://gitlab.com/libvirt/libvirt-ci ), and then we need to
> update our lcitool status in QEMU afterwards. Maybe Daniel can advise for
> the right stteps here...?

It looks like a bunch of mappings are maintained in
tests/lcitool/mappings.yml instead of the main lcitool repository. So
I think you need to apply the diff below, then run

  $ git submodule update --init tests/lcitool/libvirt-ci
  $ tests/lcitool/refresh

to propagate the changes to the generated files.


diff --git a/tests/lcitool/mappings.yml b/tests/lcitool/mappings.yml
index 0b908882f1..407c03301b 100644
--- a/tests/lcitool/mappings.yml
+++ b/tests/lcitool/mappings.yml
@@ -59,6 +59,10 @@ mappings:
     CentOSStream8:
     OpenSUSELeap15:

+  python3-sqlite3:
+    CentOSStream8: python38
+    OpenSUSELeap15: python311
+
   python3-tomli:
     # test using tomllib
     apk:
diff --git a/tests/lcitool/projects/qemu.yml b/tests/lcitool/projects/qemu.yml
index 82092c9f17..149b15de57 100644
--- a/tests/lcitool/projects/qemu.yml
+++ b/tests/lcitool/projects/qemu.yml
@@ -97,6 +97,7 @@ packages:
  - python3-pip
  - python3-sphinx
  - python3-sphinx-rtd-theme
+ - python3-sqlite3
  - python3-tomli
  - python3-venv
  - rpm2cpio
-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [PATCH] tests/docker: Add sqlite3 module to openSUSE Leap container

[Philippe Mathieu-Daudé]

On 17/1/24 08:44, Andrea Bolognani wrote:
> On Wed, Jan 17, 2024 at 08:18:27AM +0100, Thomas Huth wrote:
>> On 17/01/2024 00.09, Fabiano Rosas wrote:
>>> Avocado needs sqlite3:
>>>
>>>     Failed to load plugin from module "avocado.plugins.journal":
>>>     ImportError("Module 'sqlite3' is not installed.
>>>     Use: sudo zypper install python311 to install it")
>>>
>>> Include the appropriate package in the dockerfile.
>>>
>>>   From 'zypper info python311':
>>>     "This package supplies rich command line features provided by
>>>     readline, and sqlite3 support for the interpreter core, thus forming
>>>     a so called "extended" runtime."
> 
> Weird choice on Python's part to have sqlite3 support as part of the
> standard library IMO, but that's "batteries included" for you :)
> 
>>> +++ b/tests/docker/dockerfiles/opensuse-leap.docker
>>> @@ -90,6 +90,7 @@ RUN zypper update -y && \
>>>               pcre-devel-static \
>>>               pipewire-devel \
>>>               pkgconfig \
>>> +           python311 \
>>>               python311-base \
>>>               python311-pip \
>>>               python311-setuptools \
>>
>> AFAIK tests/docker/dockerfiles/opensuse-leap.docker is auto-generated, so
>> this will be lost once somebody else runs lcitool again...
>>
>> I don't really have a clue, but I guess this has to be fixed in the upstream
>> lcitool first ( https://gitlab.com/libvirt/libvirt-ci ), and then we need to
>> update our lcitool status in QEMU afterwards. Maybe Daniel can advise for
>> the right stteps here...?
> 
> It looks like a bunch of mappings are maintained in
> tests/lcitool/mappings.yml instead of the main lcitool repository.

See commit 32c0613113 for mapping override use:

     ci, docker: update CentOS and OpenSUSE Python to non-EOL versions

     Python 3.6 is at end-of-life.  Update the libvirt-ci module to a
     version that supports overrides for targets and package mappings;
     this way, QEMU can use the newer versions provided by CentOS 8
     (Python 3.8) and OpenSUSE 15.3 (Python 3.9).

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 11c651ca08..362a26698b 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -479,6 +479,12 @@ first to contribute the mapping to the 
``libvirt-ci`` project:
     contains the ``mappings.yml`` update.  Then add the prerequisite and
     run ``make lcitool-refresh``.

+For enterprise distros that default to old, end-of-life versions of the
+Python runtime, QEMU uses a separate set of mappings that work with more
+recent versions.  These can be found in ``tests/lcitool/mappings.yml``.
+Modifying this file should not be necessary unless the new pre-requisite
+is a Python library or tool.

> So
> I think you need to apply the diff below, then run
> 
>    $ git submodule update --init tests/lcitool/libvirt-ci
>    $ tests/lcitool/refresh
> 
> to propagate the changes to the generated files.
> 
> 
> diff --git a/tests/lcitool/mappings.yml b/tests/lcitool/mappings.yml
> index 0b908882f1..407c03301b 100644
> --- a/tests/lcitool/mappings.yml
> +++ b/tests/lcitool/mappings.yml
> @@ -59,6 +59,10 @@ mappings:
>       CentOSStream8:
>       OpenSUSELeap15:
> 
> +  python3-sqlite3:
> +    CentOSStream8: python38
> +    OpenSUSELeap15: python311
> +
>     python3-tomli:
>       # test using tomllib
>       apk:
> diff --git a/tests/lcitool/projects/qemu.yml b/tests/lcitool/projects/qemu.yml
> index 82092c9f17..149b15de57 100644
> --- a/tests/lcitool/projects/qemu.yml
> +++ b/tests/lcitool/projects/qemu.yml
> @@ -97,6 +97,7 @@ packages:
>    - python3-pip
>    - python3-sphinx
>    - python3-sphinx-rtd-theme
> + - python3-sqlite3
>    - python3-tomli
>    - python3-venv
>    - rpm2cpio

LGTM.