Re: [Fuego] Fuego's version up and other changes

["Jan-Simon Möller" <dl9pf@gmx.de>]

[-- Attachment #1: Type: text/plain, Size: 4484 bytes --]

Hi Daniel, Tim, * !

v2: now complete and w/o thick finger.

I've setup an instance of the -next branch and I'm using and working on it right now.

Let me add a few notes and observations:
* I like the create-job / delete-job and the node scripts. They simplify 
  things a lot.
* A extension to this approach IMHO would be the use of jenkins-job-builder.
  The beauty of this is that we do not have to deal with the xml ourselves.
  Instead we define yaml files and let jjb create the necessary xml.
  Another plus is that such an approach would also allow ppl to add
  the testcases to an *existing* jenkins using the jjb templates. 

Let me illustrate this with a jump-start example:

Installation of jjb (in fuego container):
$ cd ~/
$ sudo apt-get install python-pip
$ sudo pip install jenkins-job-builder
  # should be 1.6.x

$ cat <<EOF > jjb-conf.ini
[job_builder]
ignore_cache=True
keep_descriptions=False
include_path=.:scripts:~/git/
recursive=True
[jenkins]
#user=
#password=
url=http://localhost:8080/fuego
##### This is deprecated, use job_builder section instead
#ignore_cache=True
query_plugins_info=False
EOF

$ mkdir jjb
$ cd jjb/

$ mkdir testjob
$ cat <<EOF > testjob/testjob.yaml
- project:
    name: fuego-tests-smoke
    testplan:
        - smoke
    machine:
        - raspberrypi3
    testname:
        - Benchmark.Dhrystone
        - Benchmark.IOzone
        - Functional.hello_world
        - Functional.ipv6connect
        - Functional.stress
    jobs:
        - fuego-{machine}-{testplan}-batch
        - fuego-{machine}-{testplan}-{testname}

- job-template:
    name: fuego-{machine}-{testplan}-batch
    project-type: freestyle
    concurrent: false
    node: master

    builders:
      - shell: 'echo "Run Orchestrator hook - action to enable board"'
      - shell: 'echo "Next section should also be autogenerated - just dont know the kong-fu, yet."'
      - trigger-builds:
        - project: fuego-{machine}-{testplan}-Benchmark.Dhrystone
          current-parameters: true
          block: true
        - project: fuego-{machine}-{testplan}-Benchmark.IOzone
          current-parameters: true
          block: true
        - project: fuego-{machine}-{testplan}-Functional.hello_world
          current-parameters: true
          block: true
        - project: fuego-{machine}-{testplan}-Functional.ipv6connect
          current-parameters: true
          block: true
        - project: fuego-{machine}-{testplan}-Functional.stress
          current-parameters: true
          block: true
      - conditional-step:
          condition-kind: always
          steps:
              - shell: 'echo "Run orchestrator hook - action do disable board"'

- job-template:
    name: 'fuego-{machine}-{testplan}-{testname}'
    project-type: freestyle
    concurrent: false
    node: '{machine}'
    builders:
      - shell: |
          export Reboot=false
          export Rebuild=true
          export Target_Cleanup=true
          export TESTDIR={testname}  # yep, mixed up testname here with testdir and vice-versa. anyway.
          export TESTNAME=$(echo "{testname}" | sed -e "s#.*\.##")
          export DISTRIB="{{distrib}}"
          export TESTPLAN="{testplan}"
          timeout --signal=9 10m /bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh


EOF

$ jenkins-jobs --conf jjb-conf.ini test -r testjob/

# to create the jobs within jenkins:$ jenkins-jobs --conf jjb-conf.ini update -r testjob/


The idea would be that fuego-core would provide the scripts and the yaml file in the same git.



While running tests I also found:
- we need to split the build of the test and the postprocessing to be run separately
-- this is b/c it is not useful to keep to board waiting on us all the time.
- Integration of board up/down could happen in a few ways:
-- Either ppl should just generate a wrapper around the (blocking) batch job and trigger it
   with their pre/post to their needs. Done. 
-- Or we allow hooks like the current "TARGET_SETUP_LINK" and amend it with a matching "TARGET_TEARDOWN_LINK"
- If a model like the "TARGET_SETUP_LINK" is used, we add a delay here as this call must block until the board is up.
  In this case our predifined timeouts are bogus as they track not just the test run, but all processing.

Thats my 0,02 - please comment.

Best,
-- 
--
Jan-Simon Möller
dl9pf@gmx.de

[-- Attachment #2: Type: text/html, Size: 22225 bytes --]