From: "Alex Bennée" <alex.bennee@linaro.org>
To: "Philippe Mathieu-Daudé" <philmd@redhat.com>
Cc: "Willian Rampazzo" <willianr@redhat.com>,
"Thomas Huth" <thuth@redhat.com>,
qemu-devel@nongnu.org,
"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
"Philippe Mathieu-Daudé" <f4bug@amsat.org>
Subject: Re: [PATCH] gitlab-ci: Split custom-runners.yml in one file per runner
Date: Mon, 15 Nov 2021 14:22:36 +0000 [thread overview]
Message-ID: <871r3hxzuz.fsf@linaro.org> (raw)
In-Reply-To: <20211115095608.2436223-1-philmd@redhat.com>
Philippe Mathieu-Daudé <philmd@redhat.com> writes:
> To ease maintenance, add the custom-runners/ directory and
> split custom-runners.yml in 3 files, all included by the
> current custom-runners.yml:
> - ubuntu-18.04-s390x.yml
> - ubuntu-20.04-aarch64.yml
> - centos-stream-8-x86_64.yml
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Seems reasonable:
Queued to for-6.2/misc-fixes, thanks.
> ---
> Based-on: <20211111160501.862396-1-crosa@redhat.com>
> https://gitlab.com/stsquad/qemu/-/tree/for-6.2/misc-fixes
> ---
> .gitlab-ci.d/custom-runners.yml | 268 +-----------------
> .../custom-runners/centos-stream-8-x86_64.yml | 28 ++
> .../custom-runners/ubuntu-18.04-s390x.yml | 118 ++++++++
> .../custom-runners/ubuntu-20.04-aarch64.yml | 118 ++++++++
> 4 files changed, 268 insertions(+), 264 deletions(-)
> create mode 100644 .gitlab-ci.d/custom-runners/centos-stream-8-x86_64.yml
> create mode 100644 .gitlab-ci.d/custom-runners/ubuntu-18.04-s390x.yml
> create mode 100644 .gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml
>
> diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
> index 1f56297dfa1..056c374619b 100644
> --- a/.gitlab-ci.d/custom-runners.yml
> +++ b/.gitlab-ci.d/custom-runners.yml
> @@ -13,267 +13,7 @@
> variables:
> GIT_STRATEGY: clone
>
> -# All ubuntu-18.04 jobs should run successfully in an environment
> -# setup by the scripts/ci/setup/build-environment.yml task
> -# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
> -ubuntu-18.04-s390x-all-linux-static:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_18.04
> - - s390x
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - - if: "$S390X_RUNNER_AVAILABLE"
> - script:
> - # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
> - # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
> - - mkdir build
> - - cd build
> - - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
> - - make --output-sync -j`nproc`
> - - make --output-sync -j`nproc` check V=1
> - - make --output-sync -j`nproc` check-tcg V=1
> -
> -ubuntu-18.04-s390x-all:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_18.04
> - - s390x
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - - if: "$S390X_RUNNER_AVAILABLE"
> - script:
> - - mkdir build
> - - cd build
> - - ../configure --disable-libssh
> - - make --output-sync -j`nproc`
> - - make --output-sync -j`nproc` check V=1
> -
> -ubuntu-18.04-s390x-alldbg:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_18.04
> - - s390x
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - when: manual
> - allow_failure: true
> - - if: "$S390X_RUNNER_AVAILABLE"
> - when: manual
> - allow_failure: true
> - script:
> - - mkdir build
> - - cd build
> - - ../configure --enable-debug --disable-libssh
> - - make clean
> - - make --output-sync -j`nproc`
> - - make --output-sync -j`nproc` check V=1
> -
> -ubuntu-18.04-s390x-clang:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_18.04
> - - s390x
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - when: manual
> - allow_failure: true
> - - if: "$S390X_RUNNER_AVAILABLE"
> - when: manual
> - allow_failure: true
> - script:
> - - mkdir build
> - - cd build
> - - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
> - - make --output-sync -j`nproc`
> - - make --output-sync -j`nproc` check V=1
> -
> -ubuntu-18.04-s390x-tci:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_18.04
> - - s390x
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - when: manual
> - allow_failure: true
> - - if: "$S390X_RUNNER_AVAILABLE"
> - when: manual
> - allow_failure: true
> - script:
> - - mkdir build
> - - cd build
> - - ../configure --disable-libssh --enable-tcg-interpreter
> - - make --output-sync -j`nproc`
> -
> -ubuntu-18.04-s390x-notcg:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_18.04
> - - s390x
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - when: manual
> - allow_failure: true
> - - if: "$S390X_RUNNER_AVAILABLE"
> - when: manual
> - allow_failure: true
> - script:
> - - mkdir build
> - - cd build
> - - ../configure --disable-libssh --disable-tcg
> - - make --output-sync -j`nproc`
> - - make --output-sync -j`nproc` check V=1
> -
> -# All ubuntu-20.04 jobs should run successfully in an environment
> -# setup by the scripts/ci/setup/qemu/build-environment.yml task
> -# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
> -ubuntu-20.04-aarch64-all-linux-static:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_20.04
> - - aarch64
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - - if: "$AARCH64_RUNNER_AVAILABLE"
> - script:
> - # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
> - # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
> - - mkdir build
> - - cd build
> - - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
> - - make --output-sync -j`nproc`
> - - make --output-sync -j`nproc` check V=1
> - - make --output-sync -j`nproc` check-tcg V=1
> -
> -ubuntu-20.04-aarch64-all:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_20.04
> - - aarch64
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - when: manual
> - allow_failure: true
> - - if: "$AARCH64_RUNNER_AVAILABLE"
> - when: manual
> - allow_failure: true
> - script:
> - - mkdir build
> - - cd build
> - - ../configure --disable-libssh
> - - make --output-sync -j`nproc`
> - - make --output-sync -j`nproc` check V=1
> -
> -ubuntu-20.04-aarch64-alldbg:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_20.04
> - - aarch64
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - - if: "$AARCH64_RUNNER_AVAILABLE"
> - script:
> - - mkdir build
> - - cd build
> - - ../configure --enable-debug --disable-libssh
> - - make clean
> - - make --output-sync -j`nproc`
> - - make --output-sync -j`nproc` check V=1
> -
> -ubuntu-20.04-aarch64-clang:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_20.04
> - - aarch64
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - when: manual
> - allow_failure: true
> - - if: "$AARCH64_RUNNER_AVAILABLE"
> - when: manual
> - allow_failure: true
> - script:
> - - mkdir build
> - - cd build
> - - ../configure --disable-libssh --cc=clang-10 --cxx=clang++-10 --enable-sanitizers
> - - make --output-sync -j`nproc`
> - - make --output-sync -j`nproc` check V=1
> -
> -ubuntu-20.04-aarch64-tci:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_20.04
> - - aarch64
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - when: manual
> - allow_failure: true
> - - if: "$AARCH64_RUNNER_AVAILABLE"
> - when: manual
> - allow_failure: true
> - script:
> - - mkdir build
> - - cd build
> - - ../configure --disable-libssh --enable-tcg-interpreter
> - - make --output-sync -j`nproc`
> -
> -ubuntu-20.04-aarch64-notcg:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_20.04
> - - aarch64
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - when: manual
> - allow_failure: true
> - - if: "$AARCH64_RUNNER_AVAILABLE"
> - when: manual
> - allow_failure: true
> - script:
> - - mkdir build
> - - cd build
> - - ../configure --disable-libssh --disable-tcg
> - - make --output-sync -j`nproc`
> - - make --output-sync -j`nproc` check V=1
> -
> -centos-stream-8-x86_64:
> - allow_failure: true
> - needs: []
> - stage: build
> - tags:
> - - centos_stream_8
> - - x86_64
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> - - if: "$CENTOS_STREAM_8_x86_64_RUNNER_AVAILABLE"
> - artifacts:
> - name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
> - when: on_failure
> - expire_in: 7 days
> - paths:
> - - build/tests/results/latest/results.xml
> - - build/tests/results/latest/test-results
> - reports:
> - junit: build/tests/results/latest/results.xml
> - before_script:
> - - JOBS=$(expr $(nproc) + 1)
> - script:
> - - mkdir build
> - - cd build
> - - ../scripts/ci/org.centos/stream/8/x86_64/configure
> - - make -j"$JOBS"
> - - make NINJA=":" check
> - - ../scripts/ci/org.centos/stream/8/x86_64/test-avocado
> +include:
> + - local: '/.gitlab-ci.d/custom-runners/ubuntu-18.04-s390x.yml'
> + - local: '/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml'
> + - local: '/.gitlab-ci.d/custom-runners/centos-stream-8-x86_64.yml'
> diff --git a/.gitlab-ci.d/custom-runners/centos-stream-8-x86_64.yml b/.gitlab-ci.d/custom-runners/centos-stream-8-x86_64.yml
> new file mode 100644
> index 00000000000..49aa703f55c
> --- /dev/null
> +++ b/.gitlab-ci.d/custom-runners/centos-stream-8-x86_64.yml
> @@ -0,0 +1,28 @@
> +centos-stream-8-x86_64:
> + allow_failure: true
> + needs: []
> + stage: build
> + tags:
> + - centos_stream_8
> + - x86_64
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + - if: "$CENTOS_STREAM_8_x86_64_RUNNER_AVAILABLE"
> + artifacts:
> + name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
> + when: on_failure
> + expire_in: 7 days
> + paths:
> + - build/tests/results/latest/results.xml
> + - build/tests/results/latest/test-results
> + reports:
> + junit: build/tests/results/latest/results.xml
> + before_script:
> + - JOBS=$(expr $(nproc) + 1)
> + script:
> + - mkdir build
> + - cd build
> + - ../scripts/ci/org.centos/stream/8/x86_64/configure
> + - make -j"$JOBS"
> + - make NINJA=":" check
> + - ../scripts/ci/org.centos/stream/8/x86_64/test-avocado
> diff --git a/.gitlab-ci.d/custom-runners/ubuntu-18.04-s390x.yml b/.gitlab-ci.d/custom-runners/ubuntu-18.04-s390x.yml
> new file mode 100644
> index 00000000000..f39d874a1e1
> --- /dev/null
> +++ b/.gitlab-ci.d/custom-runners/ubuntu-18.04-s390x.yml
> @@ -0,0 +1,118 @@
> +# All ubuntu-18.04 jobs should run successfully in an environment
> +# setup by the scripts/ci/setup/build-environment.yml task
> +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
> +
> +ubuntu-18.04-s390x-all-linux-static:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_18.04
> + - s390x
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + - if: "$S390X_RUNNER_AVAILABLE"
> + script:
> + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
> + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
> + - mkdir build
> + - cd build
> + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> + - make --output-sync -j`nproc` check-tcg V=1
> +
> +ubuntu-18.04-s390x-all:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_18.04
> + - s390x
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + - if: "$S390X_RUNNER_AVAILABLE"
> + script:
> + - mkdir build
> + - cd build
> + - ../configure --disable-libssh
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> +
> +ubuntu-18.04-s390x-alldbg:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_18.04
> + - s390x
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + when: manual
> + allow_failure: true
> + - if: "$S390X_RUNNER_AVAILABLE"
> + when: manual
> + allow_failure: true
> + script:
> + - mkdir build
> + - cd build
> + - ../configure --enable-debug --disable-libssh
> + - make clean
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> +
> +ubuntu-18.04-s390x-clang:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_18.04
> + - s390x
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + when: manual
> + allow_failure: true
> + - if: "$S390X_RUNNER_AVAILABLE"
> + when: manual
> + allow_failure: true
> + script:
> + - mkdir build
> + - cd build
> + - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> +
> +ubuntu-18.04-s390x-tci:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_18.04
> + - s390x
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + when: manual
> + allow_failure: true
> + - if: "$S390X_RUNNER_AVAILABLE"
> + when: manual
> + allow_failure: true
> + script:
> + - mkdir build
> + - cd build
> + - ../configure --disable-libssh --enable-tcg-interpreter
> + - make --output-sync -j`nproc`
> +
> +ubuntu-18.04-s390x-notcg:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_18.04
> + - s390x
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + when: manual
> + allow_failure: true
> + - if: "$S390X_RUNNER_AVAILABLE"
> + when: manual
> + allow_failure: true
> + script:
> + - mkdir build
> + - cd build
> + - ../configure --disable-libssh --disable-tcg
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> diff --git a/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml b/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml
> new file mode 100644
> index 00000000000..920e388bd05
> --- /dev/null
> +++ b/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml
> @@ -0,0 +1,118 @@
> +# All ubuntu-20.04 jobs should run successfully in an environment
> +# setup by the scripts/ci/setup/qemu/build-environment.yml task
> +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
> +
> +ubuntu-20.04-aarch64-all-linux-static:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_20.04
> + - aarch64
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + - if: "$AARCH64_RUNNER_AVAILABLE"
> + script:
> + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
> + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
> + - mkdir build
> + - cd build
> + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> + - make --output-sync -j`nproc` check-tcg V=1
> +
> +ubuntu-20.04-aarch64-all:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_20.04
> + - aarch64
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + when: manual
> + allow_failure: true
> + - if: "$AARCH64_RUNNER_AVAILABLE"
> + when: manual
> + allow_failure: true
> + script:
> + - mkdir build
> + - cd build
> + - ../configure --disable-libssh
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> +
> +ubuntu-20.04-aarch64-alldbg:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_20.04
> + - aarch64
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + - if: "$AARCH64_RUNNER_AVAILABLE"
> + script:
> + - mkdir build
> + - cd build
> + - ../configure --enable-debug --disable-libssh
> + - make clean
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> +
> +ubuntu-20.04-aarch64-clang:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_20.04
> + - aarch64
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + when: manual
> + allow_failure: true
> + - if: "$AARCH64_RUNNER_AVAILABLE"
> + when: manual
> + allow_failure: true
> + script:
> + - mkdir build
> + - cd build
> + - ../configure --disable-libssh --cc=clang-10 --cxx=clang++-10 --enable-sanitizers
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> +
> +ubuntu-20.04-aarch64-tci:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_20.04
> + - aarch64
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + when: manual
> + allow_failure: true
> + - if: "$AARCH64_RUNNER_AVAILABLE"
> + when: manual
> + allow_failure: true
> + script:
> + - mkdir build
> + - cd build
> + - ../configure --disable-libssh --enable-tcg-interpreter
> + - make --output-sync -j`nproc`
> +
> +ubuntu-20.04-aarch64-notcg:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_20.04
> + - aarch64
> + rules:
> + - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> + when: manual
> + allow_failure: true
> + - if: "$AARCH64_RUNNER_AVAILABLE"
> + when: manual
> + allow_failure: true
> + script:
> + - mkdir build
> + - cd build
> + - ../configure --disable-libssh --disable-tcg
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
--
Alex Bennée
next prev parent reply other threads:[~2021-11-15 14:24 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-15 9:56 [PATCH] gitlab-ci: Split custom-runners.yml in one file per runner Philippe Mathieu-Daudé
2021-11-15 14:22 ` Alex Bennée [this message]
2021-11-15 14:32 ` Philippe Mathieu-Daudé
2021-11-15 14:33 ` Thomas Huth
2021-11-15 15:01 ` Philippe Mathieu-Daudé
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=871r3hxzuz.fsf@linaro.org \
--to=alex.bennee@linaro.org \
--cc=f4bug@amsat.org \
--cc=philmd@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=thuth@redhat.com \
--cc=wainersm@redhat.com \
--cc=willianr@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.