Re: [Fuego] Fuego's version up and other changes

["Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>]

> > - I removed dependencies on all groovy scripts.
> 
> I'm still processing this.  I believe it means that targets are not first-class
> objects inside of Jenkins anymore.  I'll have to investigate how to use
> Jenkins to configure attributes of the target, or see target-specific test
> results, etc, based on the new interface.

I'll try to explain this.

In the current version of Fuego:
  - Nodes and its attributes (e.g.: which board and dist file to use) are created 
    manually through the jenkins interface.
  - To run a job on a node, you open a job and then select your node (as well
     as some variables such as 'rebuild' etc..). The problem is that you can only
     run one node at a time, unless you copy the job and change the name
     manually, and that you have to trigger them manually as well.

In my version of Fuego:
  - Nodes and its attributes are created through the fuego-create-node script.
    You can configure the attributes of the target as usual from the jenkins interface
    or from the script interface (delete the node and create it with different attributes).
  - Jobs are created through a script called fuego-create-jobs. Jobs are not
    dynamic anymore (although you can do the same just by clicking on configure)
    and are assigned a specific node, testplan and other attributes. If you want
    to see a target-specific test result, you click on the corresponding node to display
    jobs assigned to your target/node. Then you click on the job you want. 

By the way, if you still want to dynamically change the parameters of a job, all you
have to do is click on "configure" and then change the variable, testplan, etc.. as
you please.

> I haven't had time yet to play with the newer code and find out what
> features or workflows are sacrificed by this, but I hope to do so next
> week.
> 
> I plan to come up with a list of tasks and operations that I did in the
> old interface, that are not obvious how to do in the new interface.

Sure, that would be great thank you!

> > - I added a  tool for generating jobs from testplans.  Creating jobs from
> > testplans is powerful.
> >    It allows you, for example, to easily specify the timeout for each test on
> > your board.
> >    The tool also generates the corresponding batch job. For example, you can
> > easily trigger all
> >    tests for your board and a specific testplan periodically.
> 
> Sounds good, and similar to what 0day does, but I'm not sure I understand
> completely.  I believe you mean that you can use the test materials and testplans
> to create separate Jenkins job for each plan?  Is that right?

I am assuming a workflow where you have a testplan for each of your targets (otherwise you'll get the default one). 
In a testplan you define which tests you want to run on your target. For example, you may only want to run
8 tests on a specific board. You also define the spec to use for each test (by 'spec' I mean the word that gets mapped to a 
specific set of parameters for the test). 

When you create the jobs for your target, you pass the corresponding testplan as a parameter
to fuego-create-jobs. This tool will traverse the testplan and create the jobs on the jenkins interface.

[Note] At the moment, if you want to use two different specs for the same target-test combination you have to
define it in a separate testplan. But I will add support for testplans that contain the same test with different specs.
 
> It looks like you also create a separate Jenkins job for each combination of
> test and target?  Is that right?  

More exactly, for each combination of target.testplan.test. 
[Note] When I add support for multiple specs inside a testplan it will become target.testplan.testspec.test.

> The default Jenkins interface is so sparse
> that I'm a little lost what to do with it.  (Maybe I should not have comingled
> my learning of Jenkins with my learning of Fuego - I think I've made assumptions
> about base Jenkins features based on the plugins Cogent added by default
> in JTA.)

Actually, it's much easier with my version. You only need to click on your node and you will
only see the jobs assigned to your node/board (for example, 8 tests).
In the current version it is impossible to differentiate which job is for which node, all you can
do is check the history of which jobs were executed on each node.

> > - I added Excel output support for IOzone
> > - Faster docker build time (ARMhf compiler installation is now optional, no
> > latex..)
> These both sound good.  The excel output format possibly ties
> into discussions we've had about canonical output formats,
> results saving and comparisons, etc.  But I'd like to get more details
> to see what's up with this.

Well, IOzone already supported the excel output (-b).

One more important change is that now the results can be linked to each build. So
if you look to the left sidebar, each successful build/test will preserve the results of
that specific test at that specific time. Also, you can arbitrarily set these links in the
test specs (e.g.: plot.png, the log of the test, excel, ..and whatever file you want).
This results are suffixed with the build number so that they don't collide with each other.
The flot plugin (and plot.png) allows you to compare the results over time.

> > - I put a fixed name to the fuego container, instead of the
> > "last_container_id" file.
> I tend to have several containers lying around, for testing multiple
> fuego versions.  I'm not sure if this will work if docker is using a fixed
> name for the container.

OK, I  can add a parameter to the script so that you can overwrite the default
name. Still I think it will be better than "last_container_id".

> > - I removed the inotify script
> One of my first contributions to fuego!  This makes me a little sad,
> but happy if it's not needed anymore. :-)

Haha I'm sorry. Actually, jenkins changed and now most of the information
is not stored in config.xml. In fact, for certain setups it won't even generate
any config.xml. I didn't know how to set the global variables without config.xml
so I left it there, but if we write the global variables in a script we can get rid
of the config.xml for good.
 
> > - I added scripts for easily removing the docker container/image.
> >
> > I have to split the current "huge" patch into smaller chunks and test it further
> > but it would
> > be nice to get your feedback about the approach in general.
> >
> > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> 
> I can help with updating ftc and with more recent commits.
> 
> I have yet to really dig into the code changes to see how it affects
> the overall architecture.  But I really like the list of changes.
> I do want to make sure that we haven't lost any important features
> going forward, and that the changes fit with my longterm goals
> for the project.

That would be great. 
By the way, make sure you git pull the latest version because I added
a few fixes last week.

Thanks,
Daniel