[cip-dev] [PATCH] Use local copy of kernel-ci if present

[cip-dev] [PATCH] Use local copy of kernel-ci if present

[Daniel Wagner]

In order to develop or debug the setup we should allow to a developer
to use local copy of the sources.

This is done by cloning all the remote repos to the project main
folder. This folder is shared with the VM. The intergration-scripts
are testing if those repos are in the shared folder than git clone
uses this as source.

To simplify the developer setup a simple setup-dev-env.sh script
is added. Note the current path should be the top folder of the
project.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
---
Hi,

This helped me to quite a bit to get things tested in a more simpler
way. What do you think?

cheers,
daniel

 integration-scripts/install_backend.sh      |  8 +++++++-
 integration-scripts/install_build_script.sh |  9 ++++++++-
 integration-scripts/install_frontend.sh     |  8 +++++++-
 scripts/setup-dev-env.sh                    | 11 +++++++++++
 4 files changed, 33 insertions(+), 3 deletions(-)
 create mode 100644 scripts/setup-dev-env.sh

diff --git a/integration-scripts/install_backend.sh b/integration-scripts/install_backend.sh
index 72a974033a3c..157e7184e349 100755
--- a/integration-scripts/install_backend.sh
+++ b/integration-scripts/install_backend.sh
@@ -4,7 +4,13 @@
 # Install kernelci backend
 
 cd $HOME && mkdir git-repos && cd git-repos
-git clone https://github.com/kernelci/kernelci-backend-config.git kernelci-backend
+
+GIT_SRC="https://github.com/kernelci/kernelci-backend-config.git"
+if [ -d /vagrant/kernelci-backend-config ]; then
+    GIT_SRC=/vagrant/kernelci-backend-config
+fi
+git clone $GIT_SRC kernelci-backend
+
 cp /vagrant/config/secrets-backend.yml kernelci-backend/secrets.yml
 
 # Fixme: Don't let ansible try to create the file in the first place.
diff --git a/integration-scripts/install_build_script.sh b/integration-scripts/install_build_script.sh
index 322619317af6..879aaed01792 100755
--- a/integration-scripts/install_build_script.sh
+++ b/integration-scripts/install_build_script.sh
@@ -3,7 +3,14 @@
 # Copyright (C) 2016, Siemens AG, Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
 # SPDX-License-Identifier:	Apache-2.0
 
-cd $HOME && git clone https://github.com/kernelci/kernelci-build.git
+cd $HOME
+
+GIT_SRC="https://github.com/kernelci/kernelci-build.git"
+if [ -d /vagrant/kernelci-build ]; then
+    GIT_SRC=/vagrant/kernelci-build
+fi
+git clone $GIT_SRC
+
 cd kernelci-build
 
 MASTER_KEY=`cat $HOME/backend-admin-token.txt`
diff --git a/integration-scripts/install_frontend.sh b/integration-scripts/install_frontend.sh
index 48ab91ab83ce..251ef89d28f3 100755
--- a/integration-scripts/install_frontend.sh
+++ b/integration-scripts/install_frontend.sh
@@ -4,7 +4,13 @@
 # Install kernelci frontend
 
 cd $HOME/git-repos
-git clone https://github.com/kernelci/kernelci-frontend-config.git kernelci-frontend
+
+GIT_SRC="https://github.com/kernelci/kernelci-frontend-config.git"
+if [ -d /vagrant/kernelci-frontend-config ]; then
+    GIT_SRC=/vagrant/kernelci-frontend-config
+fi
+git clone $GIT_SRC kernelci-frontend
+
 sed -i kernelci-frontend/roles/install-app/tasks/main.yml \
     -e 's/kernelci\/kernelci-frontend.git/siemens\/kernelci-frontend.git/'
 
diff --git a/scripts/setup-dev-env.sh b/scripts/setup-dev-env.sh
new file mode 100644
index 000000000000..59c13b233065
--- /dev/null
+++ b/scripts/setup-dev-env.sh
@@ -0,0 +1,11 @@
+#!/bin/sh
+
+if [ ! -f "Vagrantfile" ]; then
+    echo "script is supposed to be run from the top folder where"
+    echo "the Vagrantfile is."
+    exit 1
+fi
+
+git clone https://github.com/kernelci/kernelci-backend-config.git
+git clone https://github.com/kernelci/kernelci-build.git
+git clone https://github.com/kernelci/kernelci-frontend-config.git
-- 
2.9.3

[cip-dev] [PATCH] Use local copy of kernel-ci if present

[Wolfgang Mauerer]

Am 19/01/2017 um 15:57 schrieb Daniel Wagner:
> In order to develop or debug the setup we should allow to a developer
> to use local copy of the sources.
> 
> This is done by cloning all the remote repos to the project main
> folder. This folder is shared with the VM. The intergration-scripts
> are testing if those repos are in the shared folder than git clone
> uses this as source.
I've had to read the last sentence for a couple of times before
I could make sense of it. Perhaps something along the lines of

'The integration scripts test if the repos are present in the
shared folder. They are used as source for "git clone" in this case.'
> 
> To simplify the developer setup a simple setup-dev-env.sh script
> is added. Note the current path should be the top folder of the
> project.
> 
> Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
> ---
> Hi,
> 
> This helped me to quite a bit to get things tested in a more simpler
> way. What do you think?

the patch as such seems most helpful to me -- struggling with
modifications that do not render any effect because sources are pulled
from a differnet source have bitten me quite a few times when working
with kernelci.

Thanks, Wolfgang
> 
> cheers,
> daniel
> 
>  integration-scripts/install_backend.sh      |  8 +++++++-
>  integration-scripts/install_build_script.sh |  9 ++++++++-
>  integration-scripts/install_frontend.sh     |  8 +++++++-
>  scripts/setup-dev-env.sh                    | 11 +++++++++++
>  4 files changed, 33 insertions(+), 3 deletions(-)
>  create mode 100644 scripts/setup-dev-env.sh
> 
> diff --git a/integration-scripts/install_backend.sh b/integration-scripts/install_backend.sh
> index 72a974033a3c..157e7184e349 100755
> --- a/integration-scripts/install_backend.sh
> +++ b/integration-scripts/install_backend.sh
> @@ -4,7 +4,13 @@
>  # Install kernelci backend
>  
>  cd $HOME && mkdir git-repos && cd git-repos
> -git clone https://github.com/kernelci/kernelci-backend-config.git kernelci-backend
> +
> +GIT_SRC="https://github.com/kernelci/kernelci-backend-config.git"
> +if [ -d /vagrant/kernelci-backend-config ]; then
> +    GIT_SRC=/vagrant/kernelci-backend-config
> +fi
> +git clone $GIT_SRC kernelci-backend
> +
>  cp /vagrant/config/secrets-backend.yml kernelci-backend/secrets.yml
>  
>  # Fixme: Don't let ansible try to create the file in the first place.
> diff --git a/integration-scripts/install_build_script.sh b/integration-scripts/install_build_script.sh
> index 322619317af6..879aaed01792 100755
> --- a/integration-scripts/install_build_script.sh
> +++ b/integration-scripts/install_build_script.sh
> @@ -3,7 +3,14 @@
>  # Copyright (C) 2016, Siemens AG, Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
>  # SPDX-License-Identifier:	Apache-2.0
>  
> -cd $HOME && git clone https://github.com/kernelci/kernelci-build.git
> +cd $HOME
> +
> +GIT_SRC="https://github.com/kernelci/kernelci-build.git"
> +if [ -d /vagrant/kernelci-build ]; then
> +    GIT_SRC=/vagrant/kernelci-build
> +fi
> +git clone $GIT_SRC
> +
>  cd kernelci-build
>  
>  MASTER_KEY=`cat $HOME/backend-admin-token.txt`
> diff --git a/integration-scripts/install_frontend.sh b/integration-scripts/install_frontend.sh
> index 48ab91ab83ce..251ef89d28f3 100755
> --- a/integration-scripts/install_frontend.sh
> +++ b/integration-scripts/install_frontend.sh
> @@ -4,7 +4,13 @@
>  # Install kernelci frontend
>  
>  cd $HOME/git-repos
> -git clone https://github.com/kernelci/kernelci-frontend-config.git kernelci-frontend
> +
> +GIT_SRC="https://github.com/kernelci/kernelci-frontend-config.git"
> +if [ -d /vagrant/kernelci-frontend-config ]; then
> +    GIT_SRC=/vagrant/kernelci-frontend-config
> +fi
> +git clone $GIT_SRC kernelci-frontend
> +
>  sed -i kernelci-frontend/roles/install-app/tasks/main.yml \
>      -e 's/kernelci\/kernelci-frontend.git/siemens\/kernelci-frontend.git/'
>  
> diff --git a/scripts/setup-dev-env.sh b/scripts/setup-dev-env.sh
> new file mode 100644
> index 000000000000..59c13b233065
> --- /dev/null
> +++ b/scripts/setup-dev-env.sh
> @@ -0,0 +1,11 @@
> +#!/bin/sh
> +
> +if [ ! -f "Vagrantfile" ]; then
> +    echo "script is supposed to be run from the top folder where"
> +    echo "the Vagrantfile is."
> +    exit 1
> +fi
> +
> +git clone https://github.com/kernelci/kernelci-backend-config.git
> +git clone https://github.com/kernelci/kernelci-build.git
> +git clone https://github.com/kernelci/kernelci-frontend-config.git
> 

[cip-dev] [PATCH] Use local copy of kernel-ci if present

[Daniel Wagner]

On 01/19/2017 04:14 PM, Wolfgang Mauerer wrote:
> Am 19/01/2017 um 15:57 schrieb Daniel Wagner:
>> This is done by cloning all the remote repos to the project main
>> folder. This folder is shared with the VM. The intergration-scripts
>> are testing if those repos are in the shared folder than git clone
>> uses this as source.
> I've had to read the last sentence for a couple of times before
> I could make sense of it. Perhaps something along the lines of
> 
> 'The integration scripts test if the repos are present in the
> shared folder. They are used as source for "git clone" in this case.'

Good point. I lost myself in the depth of cloudy thoughts.

[cip-dev] [PATCH] Use local copy of kernel-ci if present

[Daniel Wagner]

Hi Don,

Over the weekend I had an idea to improve it even more:


GIT_SRC="https://github.com/kernelci/kernelci-backend-config.git"
if [ -d /vagrant/kernelci-backend-config ]; then
    GIT_SRC=/vagrant/kernelci-backend-config
fi
git clone $GIT_SRC kernelci-backend


If there is a repo available on /vagrant we clone from it. I 
think it would be even better just to copy the working
directory instead of cloning. That would allow to hack on the
files without committing all the time.

So this would change to:

if [ -d /vagrant/kernelci-backend-config ]; then
    cp -r /vagrant/kernelci-backend-config kernelci-backend
else
    git clone https://github.com/kernelci/kernelci-backend-config.git kernelci-backend
fi

What do you think about this?

And another idea I had: we should create a mirror of kernelci sources
clone from the mirror. I am pretty sure soon we have some patches
which need to be around to get our setup running which aren't 
available in the upstream repository. For example I had to do this here:


--- a/roles/install-deps/tasks/install-mongodb.yml
+++ b/roles/install-deps/tasks/install-mongodb.yml
@@ -2,7 +2,7 @@
 
 - name: Add MongoDB apt key (Ubuntu)
   apt_key:  id=7F0CEB10
-            keyserver=hkp://keyserver.ubuntu.com
+            keyserver=hkp://keyserver.ubuntu.com:80
   when: ansible_lsb.id == "Ubuntu"
   tags:
     - install
@@ -11,7 +11,7 @@
 
 - name: Add MongoDB apt key (Debian)
   apt_key:  id=EA312927
-            keyserver=hkp://keyserver.ubuntu.com
+            keyserver=hkp://keyserver.ubuntu.com:80
   when: ansible_lsb.id == "Debian"
   tags:
     - install

Thanks,
Daniel