[PATCH] deploy docs to qemu-project.org from GitLab CI

[PATCH] deploy docs to qemu-project.org from GitLab CI

[Paolo Bonzini]

Currently, the website is rebuilt on qemu-project.org using
a separate container (https://github.com/stefanha/qemu-docs/)
cron job hook.  We can instead reuse the GitLab's CI artifacts.

To do so, we use the same mechanism that is already in place for
qemu-web.git.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitlab-ci.yml                             | 23 ++++++++++++++++++++++
 tests/docker/dockerfiles/ubuntu2004.docker |  2 ++
 2 files changed, 25 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 4532f1718a..729138064c 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -6,6 +6,7 @@ stages:
   - containers-layer2
   - build
   - test
+  - update
 
 include:
   - local: '/.gitlab-ci.d/edk2.yml'
@@ -609,3 +610,25 @@ pages:
   artifacts:
     paths:
       - public
+
+deploy:
+  image: $CI_REGISTRY_IMAGE/qemu/ubuntu2004:latest
+  stage: update
+  needs:
+    - job: pages
+      artifacts: true
+  before_script:
+    - eval $(ssh-agent -s)
+    - cat "$SSH_PRIVATE_KEY_FILE" | tr -d '\r' | ssh-add -
+    - mkdir -m700 -p ~/.ssh
+    - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" >> ~/.ssh/config'
+  script:
+    - ssh $SSH_DEPLOY_DESTINATION "cd /var/www/qemu-project.org && mkdir new-docs && rsync -az docs/ new-docs"
+    - rsync -avz --delete public/ $SSH_DEPLOY_DESTINATION:/var/www/qemu-project.org/new-docs
+    - ssh $SSH_DEPLOY_DESTINATION "cd /var/www/qemu-project.org && rm -rf old-docs && mv docs old-docs && mv new-docs docs"
+  only:
+    refs:
+      - master
+    variables:
+      - $SSH_PRIVATE_KEY_FILE
+      - $SSH_DEPLOY_DESTINATION
diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
index ae889d8482..2bb826c376 100644
--- a/tests/docker/dockerfiles/ubuntu2004.docker
+++ b/tests/docker/dockerfiles/ubuntu2004.docker
@@ -50,6 +50,7 @@ ENV PACKAGES flex bison \
     make \
     netcat-openbsd \
     ninja-build \
+    openssh-client \
     python3-numpy \
     python3-opencv \
     python3-pil \
@@ -58,6 +59,7 @@ ENV PACKAGES flex bison \
     python3-venv \
     python3-yaml \
     rpm2cpio \
+    rsync \
     sparse \
     tesseract-ocr \
     tesseract-ocr-eng \
-- 
2.29.2

Re: [PATCH] deploy docs to qemu-project.org from GitLab CI

[Daniel P. Berrangé]

On Tue, Jan 19, 2021 at 02:26:19PM +0100, Paolo Bonzini wrote:
> Currently, the website is rebuilt on qemu-project.org using
> a separate container (https://github.com/stefanha/qemu-docs/)
> cron job hook.  We can instead reuse the GitLab's CI artifacts.
> 
> To do so, we use the same mechanism that is already in place for
> qemu-web.git.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  .gitlab-ci.yml                             | 23 ++++++++++++++++++++++
>  tests/docker/dockerfiles/ubuntu2004.docker |  2 ++
>  2 files changed, 25 insertions(+)


> diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
> index ae889d8482..2bb826c376 100644
> --- a/tests/docker/dockerfiles/ubuntu2004.docker
> +++ b/tests/docker/dockerfiles/ubuntu2004.docker
> @@ -50,6 +50,7 @@ ENV PACKAGES flex bison \
>      make \
>      netcat-openbsd \
>      ninja-build \
> +    openssh-client \
>      python3-numpy \
>      python3-opencv \
>      python3-pil \
> @@ -58,6 +59,7 @@ ENV PACKAGES flex bison \
>      python3-venv \
>      python3-yaml \
>      rpm2cpio \
> +    rsync \

Can we just stick to installing them in the deploy pre-script as for
the other job, as this dockerfile is going to be auto-generated with
just the real world QEMU dependancies present soon.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] deploy docs to qemu-project.org from GitLab CI

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1349 bytes --]

On Tue, Jan 19, 2021 at 02:26:19PM +0100, Paolo Bonzini wrote:
> Currently, the website is rebuilt on qemu-project.org using
> a separate container (https://github.com/stefanha/qemu-docs/)
> cron job hook.  We can instead reuse the GitLab's CI artifacts.
> 
> To do so, we use the same mechanism that is already in place for
> qemu-web.git.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  .gitlab-ci.yml                             | 23 ++++++++++++++++++++++
>  tests/docker/dockerfiles/ubuntu2004.docker |  2 ++
>  2 files changed, 25 insertions(+)

Hmm...the UNIX account on qemu.org is locked down to some extent but I
don't feel comfortable with a GitLab CI job sshing into qemu.org.

ssh access aside, we are publishing HTML from a shared CI runner to
qemu.org. Effectively we are allowing an untrusted machine to publish
HTML/JS/CSS on qemu.org. It could steal HTTP Cookies or do other
malicious things. That is less of a problem when there is a dedicated
subdomain so that the Same Origin policy can provide isolation. Maybe
there are more recent web security mechanisms that allow us to define a
policy so browsers do not treat qemu.org/docs/* the same as other
qemu.org pages?

(This wasn't a problem before since the container was running on a
dedicated instance under our control.)

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH] deploy docs to qemu-project.org from GitLab CI

[Daniel P. Berrangé]

On Tue, Jan 19, 2021 at 02:56:22PM +0000, Stefan Hajnoczi wrote:
> On Tue, Jan 19, 2021 at 02:26:19PM +0100, Paolo Bonzini wrote:
> > Currently, the website is rebuilt on qemu-project.org using
> > a separate container (https://github.com/stefanha/qemu-docs/)
> > cron job hook.  We can instead reuse the GitLab's CI artifacts.
> > 
> > To do so, we use the same mechanism that is already in place for
> > qemu-web.git.
> > 
> > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> > ---
> >  .gitlab-ci.yml                             | 23 ++++++++++++++++++++++
> >  tests/docker/dockerfiles/ubuntu2004.docker |  2 ++
> >  2 files changed, 25 insertions(+)
> 
> Hmm...the UNIX account on qemu.org is locked down to some extent but I
> don't feel comfortable with a GitLab CI job sshing into qemu.org.
> 
> ssh access aside, we are publishing HTML from a shared CI runner to
> qemu.org. Effectively we are allowing an untrusted machine to publish
> HTML/JS/CSS on qemu.org. It could steal HTTP Cookies or do other
> malicious things. That is less of a problem when there is a dedicated
> subdomain so that the Same Origin policy can provide isolation. Maybe
> there are more recent web security mechanisms that allow us to define a
> policy so browsers do not treat qemu.org/docs/* the same as other
> qemu.org pages?

The "easy" option is to just stop using  qemu.org/docs and instad hav
docs.qemu.org and make it a cname for qemu-project.gitlab.io. Then
gitlab can be serving the docs directly.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] deploy docs to qemu-project.org from GitLab CI

[Paolo Bonzini]

On 19/01/21 15:56, Stefan Hajnoczi wrote:
> Hmm...the UNIX account on qemu.org is locked down to some extent but I
> don't feel comfortable with a GitLab CI job sshing into qemu.org.

As you say, the qemu-deploy account on qemu.org is limited to writing to 
/var/www/qemu-project.org.  Its own home directory is also limited with 
"chattr +i".

The same CI runners are already using the qemu-deploy user to deploy the 
website itself.  (To state the obvious, you can only do this if you can 
push to the qemu-project GitLab organization.  Regular users can 
configure their fork to deploy to a different server using a different 
ssh private key, but their CI jobs won't touch qemu-project.org).

There are other ways to do defense in depth.

We could use https://www.hashicorp.com/cloud-platform for the ssh 
private key.  Right now the ssh private key (which of course only grants 
access to the qemu-deploy user) is accessible to everyone with 
administrator access to the QEMU GitLab project; a Vault instance could 
have more limited access.

With respect to the ssh private key, however, a bigger risk factor is 
that a botched (even if not malicious) patch can reach the QEMU or 
qemu-web git repositories, causing the private key to appear in public 
CI logs.  To mitigate this we could set up a restricted bash for the 
qemu-deploy user on qemu.org.  It would require small changes to 
gitlab-ci.yml to avoid the "cd" command, as well as configuring a 
restricted PATH via ~/.ssh/environment, but overall it would be easy. 
It would also protect against a malicious actor sneaking in a patch to 
gitlab-ci.yml that makes it do bad things.

Neither of these has to be done now.  The current way to do things is 
more or less what GitLab recommends so, security-wise, it's not entirely 
broken.

> ssh access aside, we are publishing HTML from a shared CI runner to
> qemu.org. Effectively we are allowing an untrusted machine to publish
> HTML/JS/CSS on qemu.org. It could steal HTTP Cookies or do other
> malicious things.

Note that we don't use cookies on www.qemu.org and don't have a CORS 
policy either.  Only wiki.qemu.org uses cookies.

Paolo

> That is less of a problem when there is a dedicated
> subdomain so that the Same Origin policy can provide isolation. Maybe
> there are more recent web security mechanisms that allow us to define a
> policy so browsers do not treat qemu.org/docs/* the same as other
> qemu.org pages?
> 
> (This wasn't a problem before since the container was running on a
> dedicated instance under our control.)