[layerindex-web][PATCH 0/6] Docker related fixes

[layerindex-web][PATCH 0/6] Docker related fixes

[Tim Orling]

This series updates all the depedencies in requirements.txt, notably a
fix for a GitPython vulnerability and fixes enabling latest mysqlclient
(v2.2.1) wheel to be buildable.

Also included is a move to the mariadb:lts container in docker-compose, since
in practice this is closer to what we are running in production and is the
version we should be striving to be compatible with (unless upstream Django
has breakage we cannot fix).

The following changes since commit b0af957ac734b76bc7cc2b45b0beb1cde74df0a6:

  layerindex: improve updates for actual_branch (2024-01-03 13:33:37 -0800)

are available in the Git repository at:

  https://github.com/moto-timo/layerindex-web docker-fixes
  https://github.com/moto-timo/layerindex-web/tree/docker-fixes

Tim Orling (6):
  dockersetup: use .zst not .zstd
  Dockerfile: sort apt packages alphabetically
  Dockerfile: fix mysqlclient==2.2.1 build
  requirements.txt: bump all to latest
  docker-compose.yml: mariadb:lts
  docker-compose.yml: drop version '3'

 Dockerfile         | 31 ++++++++++++++++---------------
 docker-compose.yml |  3 +--
 dockersetup.py     |  4 ++--
 requirements.txt   | 14 +++++++-------
 4 files changed, 26 insertions(+), 26 deletions(-)

-- 
2.34.1

[layerindex-web][PATCH 1/6] dockersetup: use .zst not .zstd

[Tim Orling]

The default file extension for zstd compression is .zst

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 dockersetup.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/dockersetup.py b/dockersetup.py
index 56458d3..21be0da 100755
--- a/dockersetup.py
+++ b/dockersetup.py
@@ -54,7 +54,7 @@ def get_args():
     parser.add_argument('-s', '--https-proxy', type=str, help='https proxy in the format http://<myproxy:port>', default=default_https_proxy, required=False)
     parser.add_argument('-S', '--socks-proxy', type=str, help='socks proxy in the format socks://myproxy:port>', default=default_socks_proxy, required=False)
     parser.add_argument('-N', '--no-proxy', type=str, help='Comma-separated list of hosts that should not be connected to via the proxy', default=default_no_proxy, required=False)
-    parser.add_argument('-d', '--databasefile', type=str, help='Location of your database file to import. Must be a .sql, .sql.gz or .sql.zstd file.', required=False)
+    parser.add_argument('-d', '--databasefile', type=str, help='Location of your database file to import. Must be a .sql, .sql.gz or .sql.zst file.', required=False)
     parser.add_argument('-e', '--email-host', type=str, help='Email host for sending messages (optionally with :port if not 25)', required=False)
     parser.add_argument('--email-user', type=str, help='User name to use when connecting to email host', required=False)
     parser.add_argument('--email-password', type=str, help='Password to use when connecting to email host', required=False)
@@ -808,7 +808,7 @@ if not args.update:
     # Import the user's supplied data
     if args.databasefile:
         filename, file_extension = os.path.splitext(args.databasefile)
-        if file_extension == ".zstd":
+        if file_extension == ".zst":
             return_code = subprocess.call("zstd -t %s > /dev/null 2>&1" % quote(args.databasefile), shell=True)
             if return_code == 0:
                 catcmd = 'zstdcat'
-- 
2.34.1

[layerindex-web][PATCH 2/6] Dockerfile: sort apt packages alphabetically

[Tim Orling]

Make it is a bit easier to see what is installed by
sorting the package list alphabetically.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 Dockerfile | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/Dockerfile b/Dockerfile
index 02948b6..0f3afc4 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -23,26 +23,26 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update \
         && update-locale \
     && apt-get install -y --no-install-recommends \
 	autoconf \
+	cpio \
+	curl \
 	g++ \
 	gcc \
-	make \
-	python2 \
-	python3-pip \
-	python3-mysqldb \
-	python3-dev \
-	python3-wheel \
-	zlib1g-dev \
+	git \
 	libfreetype6-dev \
 	libjpeg-dev \
 	libmariadb-dev-compat \
+	make \
 	netcat-openbsd \
-	curl \
-	wget \
-	git \
-	vim \
-	rpm2cpio \
+	python2 \
+	python3-dev \
+	python3-mysqldb \
+	python3-pip \
+	python3-wheel \
 	rpm \
-	cpio \
+	rpm2cpio \
+	vim \
+	wget \
+	zlib1g-dev \
     && echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen \
 	&& locale-gen en_US.UTF-8 \
 	&& update-locale \
-- 
2.34.1

[layerindex-web][PATCH 3/6] Dockerfile: fix mysqlclient==2.2.1 build

[Tim Orling]

To build mysqlclient==2.2.1 we need to add default-libmysqlclient-dev
and pkg-config

Drop conflicting libmariadb-dev-compat
Drop python3-mysqldb to allow requirements.txt to override version

Upgrade pip and setuptools before installing from requirements.txt
as newer versions are required to properly build mysqlclient wheel.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 Dockerfile | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/Dockerfile b/Dockerfile
index 0f3afc4..72f57d2 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -30,12 +30,12 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update \
 	git \
 	libfreetype6-dev \
 	libjpeg-dev \
-	libmariadb-dev-compat \
+	default-libmysqlclient-dev \
 	make \
 	netcat-openbsd \
+	pkg-config \
 	python2 \
 	python3-dev \
-	python3-mysqldb \
 	python3-pip \
 	python3-wheel \
 	rpm \
@@ -46,10 +46,11 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update \
     && echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen \
 	&& locale-gen en_US.UTF-8 \
 	&& update-locale \
+    && pip3 install pip --upgrade \
+    && pip3 install setuptools --upgrade \
     && pip3 install gunicorn \
-    && pip3 install setuptools \
     && pip3 install -r /requirements.txt \
-    && apt-get purge -y autoconf g++ make python3-dev libjpeg-dev \
+    && apt-get purge -y autoconf g++ make python3-dev pkg-config libjpeg-dev \
 	&& apt-get autoremove -y \
 	&& rm -rf /var/lib/apt/lists/* \
 	&& apt-get clean
-- 
2.34.1

[layerindex-web][PATCH 4/6] requirements.txt: bump all to latest

[Tim Orling]

Bump all the dependencies to latest, using
pur -r requirements.txt

We are now able to build the mysqlclient==2.2.1 package,
so the only constraint is Django>=4.2,<4.3 to keep us
pinned at the desired LTS version.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 requirements.txt | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/requirements.txt b/requirements.txt
index 20bcd30..ac06cfb 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -11,20 +11,20 @@ confusable-homoglyphs==3.2.0
 diff-match-patch==20230430
 Django>=4.2,<4.3
 django-appconf==1.0.6
-django-axes==6.2.0
+django-axes==6.3.0
 django-cors-headers==4.3.1
 django-ipware==6.0.3
 django-ranged-response==0.2.0
 django-registration==3.4
-django-reversion==5.0.9
+django-reversion==5.0.10
 django-reversion-compare==0.16.2
 django-simple-captcha==0.6.0
 djangorestframework==3.14.0
 gitdb==4.0.11
-GitPython==3.1.40
+GitPython==3.1.41
 kombu==5.3.4
-mysqlclient==2.1.1
-Pillow==10.1.0
+mysqlclient==2.2.1
+Pillow==10.2.0
 prompt-toolkit==3.0.43
 python-dateutil==2.8.2
 pytz==2023.3.post1
@@ -33,6 +33,6 @@ smmap==5.0.1
 soupsieve==2.5
 sqlparse==0.4.4
 typing_extensions==4.9.0
-tzdata==2023.3
+tzdata==2023.4
 vine==5.1.0
-wcwidth==0.2.12
+wcwidth==0.2.13
-- 
2.34.1

[layerindex-web][PATCH 5/6] docker-compose.yml: mariadb:lts

[Tim Orling]

We should really be striving to be compatible with
the latest mariadb LTS (currently 10.11).

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 docker-compose.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docker-compose.yml b/docker-compose.yml
index e57f59d..8b43946 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -1,7 +1,7 @@
 version: '3'
 services:
   layersdb:
-    image: mariadb:10.4
+    image: mariadb:lts
     command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci --wait_timeout=28800 --max_allowed_packet=128M
     environment:
      - "MYSQL_DATABASE=layersdb"
-- 
2.34.1

[layerindex-web][PATCH 6/6] docker-compose.yml: drop version '3'

[Tim Orling]

Drop the obsolete version: '3' property, it is no longer needed since
late 2020 (Docker Compose 1.27+) and is only informative:

https://nickjanetakis.com/blog/docker-tip-51-which-docker-compose-api-version-should-you-use
https://github.com/compose-spec/compose-spec/blob/master/04-version-and-name.md

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 docker-compose.yml | 1 -
 1 file changed, 1 deletion(-)

diff --git a/docker-compose.yml b/docker-compose.yml
index 8b43946..8704599 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -1,4 +1,3 @@
-version: '3'
 services:
   layersdb:
     image: mariadb:lts
-- 
2.34.1