From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Jan-Simon =?ISO-8859-1?Q?M=F6ller?= Date: Wed, 01 Feb 2017 00:15:41 +0100 Message-ID: <2787487.KsR9eGO5q9@elrond> In-Reply-To: <003001d27b65$2d0c2650$872472f0$@toshiba.co.jp> References: <000d01d26ba1$94c8a5d0$be59f170$@toshiba.co.jp> <003001d27b65$2d0c2650$872472f0$@toshiba.co.jp> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="nextPart1535648.VOmoiLCLLc" Content-Transfer-Encoding: 7Bit Subject: Re: [Fuego] Fuego's version up and other changes List-Id: Mailing list for the Fuego test framework List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: fuego@lists.linuxfoundation.org Cc: "'Bird, Timothy'" This is a multi-part message in MIME format. --nextPart1535648.VOmoiLCLLc Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" Hi Daniel, Tim, * ! v2: now complete and w/o thick finger. I've setup an instance of the -next branch and I'm using and working on= it right now. Let me add a few notes and observations: * I like the create-job / delete-job and the node scripts. They simplif= y=20 things a lot. * A extension to this approach IMHO would be the use of jenkins-job-bui= lder. The beauty of this is that we do not have to deal with the xml oursel= ves. Instead we define yaml files and let jjb create the necessary xml. Another plus is that such an approach would also allow ppl to add the testcases to an *existing* jenkins using the jjb templates.=20 Let me illustrate this with a jump-start example: Installation of jjb (in fuego container): $ cd ~/ $ sudo apt-get install python-pip $ sudo pip install jenkins-job-builder # should be 1.6.x $ cat < jjb-conf.ini [job_builder] ignore_cache=3DTrue keep_descriptions=3DFalse include_path=3D.:scripts:~/git/ recursive=3DTrue [jenkins] #user=3D #password=3D url=3Dhttp://localhost:8080/fuego ##### This is deprecated, use job_builder section instead #ignore_cache=3DTrue query_plugins_info=3DFalse EOF $ mkdir jjb $ cd jjb/ $ mkdir testjob $ cat < testjob/testjob.yaml - project: name: fuego-tests-smoke testplan: - smoke machine: - raspberrypi3 testname: - Benchmark.Dhrystone - Benchmark.IOzone - Functional.hello_world - Functional.ipv6connect - Functional.stress jobs: - fuego-{machine}-{testplan}-batch - fuego-{machine}-{testplan}-{testname} - job-template: name: fuego-{machine}-{testplan}-batch project-type: freestyle concurrent: false node: master builders: - shell: 'echo "Run Orchestrator hook - action to enable board"' - shell: 'echo "Next section should also be autogenerated - just = dont know the kong-fu, yet."' - trigger-builds: - project: fuego-{machine}-{testplan}-Benchmark.Dhrystone current-parameters: true block: true - project: fuego-{machine}-{testplan}-Benchmark.IOzone current-parameters: true block: true - project: fuego-{machine}-{testplan}-Functional.hello_world current-parameters: true block: true - project: fuego-{machine}-{testplan}-Functional.ipv6connect current-parameters: true block: true - project: fuego-{machine}-{testplan}-Functional.stress current-parameters: true block: true - conditional-step: condition-kind: always steps: - shell: 'echo "Run orchestrator hook - action do disable= board"' - job-template: name: 'fuego-{machine}-{testplan}-{testname}' project-type: freestyle concurrent: false node: '{machine}' builders: - shell: | export Reboot=3Dfalse export Rebuild=3Dtrue export Target_Cleanup=3Dtrue export TESTDIR=3D{testname} # yep, mixed up testname here wi= th testdir and vice-versa. anyway. export TESTNAME=3D$(echo "{testname}" | sed -e "s#.*\.##") export DISTRIB=3D"{{distrib}}" export TESTPLAN=3D"{testplan}" timeout --signal=3D9 10m /bin/bash $FUEGO_CORE/engine/tests/$= {{TESTDIR}}/${{TESTNAME}}.sh EOF $ jenkins-jobs --conf jjb-conf.ini test -r testjob/ # to create the jobs within jenkins:$ jenkins-jobs --conf jjb-conf.ini = update -r testjob/ The idea would be that fuego-core would provide the scripts and the yam= l file in the same git. While running tests I also found: - we need to split the build of the test and the postprocessing to be r= un separately -- this is b/c it is not useful to keep to board waiting on us all the = time. - Integration of board up/down could happen in a few ways: -- Either ppl should just generate a wrapper around the (blocking) batc= h job and trigger it with their pre/post to their needs. Done.=20 -- Or we allow hooks like the current "TARGET_SETUP_LINK" and amend it = with a matching "TARGET_TEARDOWN_LINK" - If a model like the "TARGET_SETUP_LINK" is used, we add a delay here = as this call must block until the board is up. In this case our predifined timeouts are bogus as they track not just= the test run, but all processing. Thats my 0,02 - please comment. Best, --=20 -- Jan-Simon M=F6ller dl9pf@gmx.de --nextPart1535648.VOmoiLCLLc Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="iso-8859-1"

Hi = Daniel, Tim, * !

v2: now complete and = w/o thick finger.

I'v= e setup an instance of the -next branch and I'm using and working on it= right now.

Let= me add a few notes and observations:

* I= like the create-job / delete-job and the node scripts. They simplify <= /p>

t= hings a lot.

* A= extension to this approach IMHO would be the use of jenkins-job-builde= r.

T= he beauty of this is that we do not have to deal with the xml ourselves= .

I= nstead we define yaml files and let jjb create the necessary xml.

A= nother plus is that such an approach would also allow ppl to add

t= he testcases to an *existing* jenkins using the jjb templates.

Let= me illustrate this with a jump-start example:

Ins= tallation of jjb (in fuego container):

$ c= d ~/

$ s= udo apt-get install python-pip

$ s= udo pip install jenkins-job-builder

#= should be 1.6.x

$ c= at <<EOF > jjb-conf.ini

[jo= b_builder]

ign= ore_cache=3DTrue

kee= p_descriptions=3DFalse

inc= lude_path=3D.:scripts:~/git/

rec= ursive=3DTrue

[je= nkins]

#us= er=3D

#pa= ssword=3D

url= =3Dhttp://localhost:8080/fuego

###= ## This is deprecated, use job_builder section instead

#ig= nore_cache=3DTrue

que= ry_plugins_info=3DFalse

EOF=

$ m= kdir jjb

$ c= d jjb/

$ m= kdir testjob

$ c= at <<EOF > testjob/testjob.yaml

- project:

testplan:

- smoke

machine:

- raspberrypi= 3

testname:

- Benchmark.D= hrystone

- Benchmark.I= Ozone

- Functional.= hello_world

- Functional.= ipv6connect

- Functional.= stress

jobs:

- fuego-{mach= ine}-{testplan}-batch

- fuego-{mach= ine}-{testplan}-{testname}

- job-template:

project-type: fre= estyle

concurrent: false=

node: master

builders:

- shell: 'echo = "Run Orchestrator hook - action to enable board"'

- shell: 'echo = "Next section should also be autogenerated - just dont know the ko= ng-fu, yet."'

- trigger-build= s:

- project: fu= ego-{machine}-{testplan}-Benchmark.Dhrystone

current-par= ameters: true

block: true=

- project: fu= ego-{machine}-{testplan}-Benchmark.IOzone

current-par= ameters: true

block: true=

- project: fu= ego-{machine}-{testplan}-Functional.hello_world

current-par= ameters: true

block: true=

- project: fu= ego-{machine}-{testplan}-Functional.ipv6connect

current-par= ameters: true

block: true=

- project: fu= ego-{machine}-{testplan}-Functional.stress

current-par= ameters: true

block: true=

- conditional-s= tep:

condition-k= ind: always

steps:

- shell= : 'echo "Run orchestrator hook - action do disable board"'

- job-template:

project-type: fre= estyle

concurrent: false=

node: '{machine}'=

builders:

- shell: |

export Rebo= ot=3Dfalse

export Rebu= ild=3Dtrue

export Targ= et_Cleanup=3Dtrue

export TEST= DIR=3D{testname} # yep, mixed up testname here with testdir and vice-v= ersa. anyway.

export TEST= NAME=3D$(echo "{testname}" | sed -e "s#.*\.##")

export DIST= RIB=3D"{{distrib}}"

export TEST= PLAN=3D"{testplan}"

timeout --s= ignal=3D9 10m /bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNA= ME}}.sh

EOF

$ jenkins-jobs --conf= jjb-conf.ini test -r testjob/

# to create the jobs within j= enkins:
$ jenkins-jobs --conf jjb-conf.ini update -r testjob/

The idea would be tha= t fuego-core would provide the scripts and the yaml file in the same gi= t.

While running tests I= also found:

- we need to split th= e build of the test and the postprocessing to be run separately

-- this is b/c it is = not useful to keep to board waiting on us all the time.

- Integration of boar= d up/down could happen in a few ways:

-- Either ppl should = just generate a wrapper around the (blocking) batch job and trigger it<= /p>

with their pre/pos= t to their needs. Done.

-- Or we allow hooks = like the current "TARGET_SETUP_LINK" and amend it with a matc= hing "TARGET_TEARDOWN_LINK"

- If a model like the= "TARGET_SETUP_LINK" is used, we add a delay here as this cal= l must block until the board is up.

In this case our pr= edifined timeouts are bogus as they track not just the test run, but al= l processing.

Thats my 0,02 - pleas= e comment.

Best,

-- =

--<= /p>

Jan= -Simon M=F6ller

dl9= pf@gmx.de

--nextPart1535648.VOmoiLCLLc--