[Fuego] Fuego test_pre_check bug

[Fuego] Fuego test_pre_check bug

[Bird, Timothy]

Hello everyone,

If you have downloaded fuego-core in the last week (this includes anyone who built the fuego docker image from scratch in the past week), then there's a bug you should know about.

I've been working on a new feature, to add support for a "test_pre_check" function to the fuego test system.
This function is a new optional routine that can be added to a base script to validate that preconditions are met
for a test.  I found a bug in the execution of this function, that causes most tests not to run.

If you have built an fuego docker image in the last few days, go into your image, and do the following:
 * cd /home/Jenkins/fuego
 * git pull

This only applies if you are using my bitbucket repositories, which you can check by listing the
git remote in that directory.  You can do that with the following command:
 * git remote -v
You should see:
origin  https://bitbucket.org/tbird20d/fuego-core.git (fetch)
origin https://bitbucket.org/tbird20d/fuego-core.git (push)

If you see something besides this, this bug does not apply to you.

I've already fixed the bug (hopefully), so all you need to do is pull the latest
fuego-core repository into your container, and you should be good to go.

Sorry for not testing this better.  (We need a continuous integration test for
fuego itself, to avoid this kind of problem with a release.  That's a bit ironic.)
 -- Tim

[Fuego] Value of the Fuego to kernel "integrators"?

[Doug Crawford]

A perception in our organization is that Fuego is intended for kernel developers in the development of long term supported kernels.
We are more “kernel integrators”;  we pick up LTS kernels through Yocto, and integrate them with our choice of file system and device drivers.

What would you say is the value in us running Fuego against our specific integrations?
My sense is that it will be beneficial since the combinations of items we have integrated are unique
(especially the drivers) and may have integration problems.- but my opinion is theoretical intuition rather than experience.

Thanks!

Re: [Fuego] [LTSI-dev] Value of the Fuego to kernel "integrators"?

[Greg KH]

On Fri, Sep 23, 2016 at 10:09:02AM -0400, Doug Crawford via LTSI-dev wrote:
> A perception in our organization is that Fuego is intended for kernel
> developers in the development of long term supported kernels.
> We are more “kernel integrators”;  we pick up LTS kernels through
> Yocto, and integrate them with our choice of file system and device
> drivers.

What filesystems and device drivers do you use that are not already
upstream in the kernel.org releases?  Need any help getting them merged
properly?

thanks,

greg k-h

[Fuego] Failure on trial run of Fuego

[Doug Crawford]

[-- Attachment #1: Type: text/plain, Size: 1245 bytes --]


I ran through the installation from
https://bitbucket.org/tbird20d/fuego/ <https://bitbucket.org/tbird20d/fuego/>

And followed quick start here:
http://bird.org/fuego/Fuego_Quickstart_Guide <http://bird.org/fuego/Fuego_Quickstart_Guide>

Excellent documentation-  what a pleasure!

I think the quick start misses the “git pull” on /home/jenkins/fuego that the installation instructions covered, perhaps
intentional, perhaps oversight.

I progressed through the Quick start well, setting up for Raspberry Pi3,  all the way to running a test.
The system seemed happy with the new target.

I proceeded to try to run the .bc test recommended in the quick start abd ran into a deviation from the expected:
On the functional tab of the test listing, I only have two tests-
.netsurf
.stress
… and no .bc test.

I went back and double checked that the “git pull” was done on /home/jenkins/fuego. It was.
So I substituted .stress instead of the .bc that the instructions said would be listed, and ran it.
I got the following failure:




-----------
Probably doesn’t like that “anonymous” user? 
Something must have been assumed on the quick start that was different for me…

Thanks!
Doug Crawford


[-- Attachment #2.1: Type: text/html, Size: 2451 bytes --]

[-- Attachment #2.2: PastedGraphic-1.tiff --]
[-- Type: image/tiff, Size: 94960 bytes --]

Re: [Fuego] Failure on trial run of Fuego

[Bird, Timothy]

>From: Doug Crawford [mailto:dcrawford@zonoff.com] 
> I ran through the installation from
> https://bitbucket.org/tbird20d/fuego/
>
> And followed quick start here:
> http://bird.org/fuego/Fuego_Quickstart_Guide
>
> Excellent documentation-  what a pleasure!
>
> I think the quick start misses the “git pull” on /home/jenkins/fuego that the installation instructions covered, perhaps
> intentional, perhaps oversight.
>
> I progressed through the Quick start well, setting up for Raspberry Pi3,  all the way to running a test.
> The system seemed happy with the new target.
>
> I proceeded to try to run the .bc test recommended in the quick start and ran into a deviation from the expected:
> On the functional tab of the test listing, I only have two tests-
> .netsurf
> .stress
> … and no .bc test.

Hmmm.  That's not right at all.  There should be about 29 tests on the Functional tab in the test view.
The first one should be Functional.aiostress, and the last one Functional.zlib.

Can you, at the shell inside the container, do an 'ls -l /var/lib/jenkins/jobs'
and a 'ls -l /var/lib/jenkins/jobs/*/config.xml'?

This is the directory Jenkins should see the test definitions in.  It should
be symlinked to:
/home/jenkins/fuego/jobs (which is inside the fuego-core git repository).

> I went back and double checked that the “git pull” was done on /home/jenkins/fuego. It was.

Can you do 'git log -n 1' in /home/jenkins/fuego?

> So I substituted .stress instead of the .bc that the instructions said would be listed, and ran it.
> I got the following failure:

>---------
>Probably doesn’t like that “anonymous” user? 
>Something must have been assumed on the quick start that was different for me…

No, the anonymous user is OK. It looks like something wrong with the permissions
or the setup.  That first 'mkdir -p' to create the log directory for the test should never fail.

Please do an 'ls -l /home/jenkins/logs', and 'ls -la /userdata/logs'
and send the results to me (and the list).

Here are the first few lines of my test output:
------
Started by user anonymous
Running remotely on bbb-poky-sdk in workspace /home/jenkins/buildzone
[buildzone] $ /bin/sh -xe /tmp/hudson1421222459709266870.sh
+ '[' '!' -d /home/jenkins/logs/Functional.stress ']'
+ mkdir -p /home/jenkins/logs/Functional.stress
+ echo testplan_default
+ TESTPLAN=testplans/testplan_default.json
+ source /home/jenkins/tests/Functional.stress/stress.sh
++ tarball=stress-1.0.4.tar.gz
++ . /home/jenkins/scripts/functional.sh
+++ source /home/jenkins/scripts/overlays.sh
...

I hope it's not something with the extra --pid="host", messing with the permissions.  The fact that only some
of the tests are visible is interesting.

Sorry you're having problems.  But thanks for hanging in there.  I'd like to figure out what went wrong.
 -- Tim

Re: [Fuego] Failure on trial run of Fuego

[Doug Crawford]

[-- Attachment #1: Type: text/plain, Size: 11389 bytes --]


> On Sep 23, 2016, at 12:46 PM, Bird, Timothy <Tim.Bird@am.sony.com> wrote:
> 
> 
> 
>> From: Doug Crawford [mailto:dcrawford@zonoff.com] 
>> I ran through the installation from
>> https://bitbucket.org/tbird20d/fuego/
>> 
>> And followed quick start here:
>> http://bird.org/fuego/Fuego_Quickstart_Guide
>> 
>> Excellent documentation-  what a pleasure!
>> 
>> I think the quick start misses the “git pull” on /home/jenkins/fuego that the installation instructions covered, perhaps
>> intentional, perhaps oversight.
>> 
>> I progressed through the Quick start well, setting up for Raspberry Pi3,  all the way to running a test.
>> The system seemed happy with the new target.
>> 
>> I proceeded to try to run the .bc test recommended in the quick start and ran into a deviation from the expected:
>> On the functional tab of the test listing, I only have two tests-
>> .netsurf
>> .stress
>> … and no .bc test.
> 
> Hmmm.  That's not right at all.  There should be about 29 tests on the Functional tab in the test view.
> The first one should be Functional.aiostress, and the last one Functional.zlib.
> 
> Can you, at the shell inside the container, do an 'ls -l /var/lib/jenkins/jobs'
> and a 'ls -l /var/lib/jenkins/jobs/*/config.xml'?
> 
> This is the directory Jenkins should see the test definitions in.  It should
> be symlinked to:
> /home/jenkins/fuego/jobs (which is inside the fuego-core git repository).
> 
>> I went back and double checked that the “git pull” was done on /home/jenkins/fuego. It was.
> 
> Can you do 'git log -n 1' in /home/jenkins/fuego?
> 
>> So I substituted .stress instead of the .bc that the instructions said would be listed, and ran it.
>> I got the following failure:
> 
>> ---------
>> Probably doesn’t like that “anonymous” user? 
>> Something must have been assumed on the quick start that was different for me…
> 
> No, the anonymous user is OK. It looks like something wrong with the permissions
> or the setup.  That first 'mkdir -p' to create the log directory for the test should never fail.
> 
> Please do an 'ls -l /home/jenkins/logs', and 'ls -la /userdata/logs'
> and send the results to me (and the list).
> 
> Here are the first few lines of my test output:
> ------
> Started by user anonymous
> Running remotely on bbb-poky-sdk in workspace /home/jenkins/buildzone
> [buildzone] $ /bin/sh -xe /tmp/hudson1421222459709266870.sh
> + '[' '!' -d /home/jenkins/logs/Functional.stress ']'
> + mkdir -p /home/jenkins/logs/Functional.stress
> + echo testplan_default
> + TESTPLAN=testplans/testplan_default.json
> + source /home/jenkins/tests/Functional.stress/stress.sh
> ++ tarball=stress-1.0.4.tar.gz
> ++ . /home/jenkins/scripts/functional.sh
> +++ source /home/jenkins/scripts/overlays.sh
> ...
> 
> I hope it's not something with the extra --pid="host", messing with the permissions.  The fact that only some
> of the tests are visible is interesting.
> 
> Sorry you're having problems.  But thanks for hanging in there.  I'd like to figure out what went wrong.
> -- Tim

Here’s the whole session from the requested commands(in bold):
root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# git pull (OK you didn’t ask for it, but wanted to show I did it :)
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 5 (delta 3), reused 0 (delta 0)
Unpacking objects: 100% (5/5), done.
From https://bitbucket.org/tbird20d/fuego-core
   2dd5187..de894e9  master     -> origin/master
Updating 2dd5187..de894e9
Fast-forward
 engine/scripts/functions.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# 'ls -l /var/lib/jenkins/jobs'
bash: ls -l /var/lib/jenkins/jobs: No such file or directory
root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -l /var/lib/jenkins/jobs/*/config.xml
-rw-r--r-- 1 jenkins root  7759 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.Dhrystone/config.xml
-rw-r--r-- 1 jenkins root  7758 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.GLMark/config.xml
-rw-r--r-- 1 jenkins root  7851 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.IOzone/config.xml
-rw-r--r-- 1 jenkins root  7791 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.Interbench/config.xml
-rw-r--r-- 1 jenkins root  7639 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.Java/config.xml
-rw-r--r-- 1 jenkins root  7632 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.OpenSSL/config.xml
-rw-r--r-- 1 jenkins root  7770 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.Stream/config.xml
-rw-r--r-- 1 jenkins root  7752 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.Whetstone/config.xml
-rw-r--r-- 1 jenkins root  7785 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.aim7/config.xml
-rw-r--r-- 1 jenkins root  7873 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.blobsallad/config.xml
-rw-r--r-- 1 jenkins root  7438 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.bonnie/config.xml
-rw-r--r-- 1 jenkins root  7427 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.cyclictest/config.xml
-rw-r--r-- 1 jenkins root  7833 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.dbench/config.xml
-rw-r--r-- 1 jenkins root  7488 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.ebizzy/config.xml
-rw-r--r-- 1 jenkins root  7712 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.ffsb/config.xml
-rw-r--r-- 1 jenkins root  7872 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.fio/config.xml
-rw-r--r-- 1 jenkins root  7760 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.gtkperf/config.xml
-rw-r--r-- 1 jenkins root  7400 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.hackbench/config.xml
-rw-r--r-- 1 jenkins root  7742 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.himeno/config.xml
-rw-r--r-- 1 jenkins root  7851 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.iperf/config.xml
-rw-r--r-- 1 jenkins root  7740 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.linpack/config.xml
-rw-r--r-- 1 jenkins root  7770 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.lmbench2/config.xml
-rw-r--r-- 1 jenkins root  7856 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.nbench-byte/config.xml
-rw-r--r-- 1 jenkins root  7795 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.netperf/config.xml
-rw-r--r-- 1 jenkins root  7861 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.netpipe/config.xml
-rw-r--r-- 1 jenkins root  7207 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.reboot/config.xml
-rw-r--r-- 1 jenkins root  7267 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.signaltest/config.xml
-rw-r--r-- 1 jenkins root  7849 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.tiobench/config.xml
-rw-r--r-- 1 jenkins root  7720 Sep 22 12:25 /var/lib/jenkins/jobs/Benchmark.x11perf/config.xml
-rw-r--r-- 1 jenkins root  6997 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.LTP.Devices/config.xml
-rw-r--r-- 1 jenkins root  7269 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.LTP.Filesystem/config.xml
-rw-r--r-- 1 jenkins root  7615 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.LTP.Open_Posix/config.xml
-rw-r--r-- 1 jenkins root  6946 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.OpenSSL/config.xml
-rw-r--r-- 1 jenkins root  7131 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.aiostress/config.xml
-rw-r--r-- 1 jenkins root  7121 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.arch_timer/config.xml
-rw-r--r-- 1 jenkins root  7135 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.bc/config.xml
-rw-r--r-- 1 jenkins root  7232 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.bzip2/config.xml
-rw-r--r-- 1 jenkins root  7107 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.cmt/config.xml
-rw-r--r-- 1 jenkins root  7153 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.crashme/config.xml
-rw-r--r-- 1 jenkins root  7269 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.expat/config.xml
-rw-r--r-- 1 jenkins root  7238 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.fontconfig/config.xml
-rw-r--r-- 1 jenkins root  6938 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.ft2demos/config.xml
-rw-r--r-- 1 jenkins root  7143 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.glib/config.xml
-rw-r--r-- 1 jenkins root  7133 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.hello_world/config.xml
-rw-r--r-- 1 jenkins root  7221 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.ipv6connect/config.xml
-rw-r--r-- 1 jenkins root  6929 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.jpeg/config.xml
-rw-r--r-- 1 jenkins root  7673 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.libpng/config.xml
-rw-r--r-- 1 jenkins root  7139 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.linus_stress/config.xml
-rw-r--r-- 1 jenkins root  7243 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.netperf/config.xml
-rw-r--r-- 1 jenkins root  7157 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.pi_tests/config.xml
-rw-r--r-- 1 jenkins root  7248 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.posixtestsuite/config.xml
-rw-r--r-- 1 jenkins root  7206 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.rmaptest/config.xml
-rw-r--r-- 1 jenkins root  7115 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.scifab/config.xml
-rw-r--r-- 1 jenkins root  7130 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.scrashme/config.xml
-rw-r--r-- 1 jenkins root  7115 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.sdhi_0/config.xml
-rw-r--r-- 1 jenkins root  7322 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.stress/config.xml
-rw-r--r-- 1 jenkins root  7130 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.synctest/config.xml
-rw-r--r-- 1 jenkins root  6924 Sep 22 12:25 /var/lib/jenkins/jobs/Functional.zlib/config.xml
-rw-r--r-- 1 jenkins root  7004 Sep 22 12:25 /var/lib/jenkins/jobs/Matrix.Nightly/config.xml
-rw-r--r-- 1 jenkins root  6635 Sep 22 12:25 /var/lib/jenkins/jobs/Matrix.Official/config.xml
-rw-r--r-- 1 jenkins root  1559 Sep 22 12:25 /var/lib/jenkins/jobs/Reports.make_pdf/config.xml
-rw-r--r-- 1 jenkins root 11405 Sep 22 12:25 /var/lib/jenkins/jobs/Run ALL tests on ALL targets/config.xml
-rw-r--r-- 1 jenkins root 12164 Sep 22 12:25 /var/lib/jenkins/jobs/Run ALL tests on SELECTED targets/config.xml
-rw-r--r-- 1 jenkins root 11555 Sep 22 12:25 /var/lib/jenkins/jobs/Run SELECTED tests on SELECTED targets/config.xml
-rw-r--r-- 1 root    root  1548 Sep 22 12:36 /var/lib/jenkins/jobs/Service.ReloadConfiguration/config.xml
root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -l /var/lib/jenkins/jobs
lrwxrwxrwx 1 jenkins root 24 Sep 22 12:25 /var/lib/jenkins/jobs -> /home/jenkins/fuego/jobs
root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# git log -n 1                        
commit de894e90b2d5c8d5d630a08d6a22cbcddc16686b
Author: Tim Bird <tim.bird@am.sony.com>
Date:   Fri Sep 23 00:04:22 2016 +0000

    Allow absence of test_pre_check
    
    Add '|| true' to test_pre_check test, to allow its absence
    without problems.
root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -l /home/jenkins/logs
lrwxrwxrwx 1 jenkins root 14 Sep 22 12:25 /home/jenkins/logs -> /userdata/logs
root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -la /userdata/logs
total 8
drwxr-xr-x 1 dc dc  170 Sep 22 12:09 .
drwxr-xr-x 1 dc dc  238 Sep 22 12:09 ..
-rw-r--r-- 1 dc dc  144 Sep 22 12:09 README
drwxr-xr-x 1 dc dc  102 Sep 22 12:09 logruns
-rw-r--r-- 1 dc dc 1622 Sep 22 12:09 tests.info
root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# 


[-- Attachment #2: Type: text/html, Size: 15400 bytes --]

Re: [Fuego] [LTSI-dev] Failure on trial run of Fuego

[Bird, Timothy]

> -----Original Message-----
> From: ltsi-dev-bounces@lists.linuxfoundation.org [mailto:ltsi-dev-
> 	On Sep 23, 2016, at 12:46 PM, Bird, Timothy <Tim.Bird@am.sony.com
> <mailto:Tim.Bird@am.sony.com> > wrote:
> 		From: Doug Crawford [mailto:dcrawford@zonoff.com]
> 		I ran through the installation from
> 		https://bitbucket.org/tbird20d/fuego/
> 
> 		And followed quick start here:
> 		http://bird.org/fuego/Fuego_Quickstart_Guide
> 
> 		Excellent documentation-  what a pleasure!
> 
> 		I think the quick start misses the “git pull” on
> /home/jenkins/fuego that the installation instructions covered, perhaps
> 		intentional, perhaps oversight.
> 
> 		I progressed through the Quick start well, setting up for
> Raspberry Pi3,  all the way to running a test.
> 		The system seemed happy with the new target.
> 
> 		I proceeded to try to run the .bc test recommended in the quick
> start and ran into a deviation from the expected:
> 		On the functional tab of the test listing, I only have two tests-
> 		.netsurf
> 		.stress
> 		… and no .bc test.
> 
> 
> 
> 	Hmmm.  That's not right at all.  There should be about 29 tests on the
> Functional tab in the test view.
> 	The first one should be Functional.aiostress, and the last one
> Functional.zlib.
> 
> 	Can you, at the shell inside the container, do an 'ls -l
> /var/lib/jenkins/jobs'
> 	and a 'ls -l /var/lib/jenkins/jobs/*/config.xml'?
> 
> 	This is the directory Jenkins should see the test definitions in.  It should
> 	be symlinked to:
> 	/home/jenkins/fuego/jobs (which is inside the fuego-core git
> repository).
> 
> 
> 
> 		I went back and double checked that the “git pull” was done on
> /home/jenkins/fuego. It was.
> 
> 
> 
> 	Can you do 'git log -n 1' in /home/jenkins/fuego?
> 
> 
> 
> 		So I substituted .stress instead of the .bc that the instructions
> said would be listed, and ran it.
> 		I got the following failure:
> 
> 
> 
> 		---------
> 		Probably doesn’t like that “anonymous” user?
> 		Something must have been assumed on the quick start that was
> different for me…
> 
> 
> 
> 	No, the anonymous user is OK. It looks like something wrong with the
> permissions
> 	or the setup.  That first 'mkdir -p' to create the log directory for the test
> should never fail.
> 
> 	Please do an 'ls -l /home/jenkins/logs', and 'ls -la /userdata/logs'
> 	and send the results to me (and the list).
> 
> 	Here are the first few lines of my test output:
> 	------
> 	Started by user anonymous
> 	Running remotely on bbb-poky-sdk in workspace
> /home/jenkins/buildzone
> 	[buildzone] $ /bin/sh -xe /tmp/hudson1421222459709266870.sh
> 	+ '[' '!' -d /home/jenkins/logs/Functional.stress ']'
> 	+ mkdir -p /home/jenkins/logs/Functional.stress
> 	+ echo testplan_default
> 	+ TESTPLAN=testplans/testplan_default.json
> 	+ source /home/jenkins/tests/Functional.stress/stress.sh
> 	++ tarball=stress-1.0.4.tar.gz
> 	++ . /home/jenkins/scripts/functional.sh
> 	+++ source /home/jenkins/scripts/overlays.sh
> 	...
> 
> 	I hope it's not something with the extra --pid="host", messing with the
> permissions.  The fact that only some
> 	of the tests are visible is interesting.
> 
> 	Sorry you're having problems.  But thanks for hanging in there.  I'd like
> to figure out what went wrong.
> 	-- Tim
> 
> 
> 
> Here’s the whole session from the requested commands(in bold):
> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# git pull (OK you didn’t
> ask for it, but wanted to show I did it :)
> remote: Counting objects: 5, done.
> remote: Compressing objects: 100% (5/5), done.
> remote: Total 5 (delta 3), reused 0 (delta 0)
> Unpacking objects: 100% (5/5), done.
> From https://bitbucket.org/tbird20d/fuego-core
>    2dd5187..de894e9  master     -> origin/master
> Updating 2dd5187..de894e9
> Fast-forward
>  engine/scripts/functions.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# 'ls -l
> /var/lib/jenkins/jobs'
> bash: ls -l /var/lib/jenkins/jobs: No such file or directory


huh?  I don't see how that failed, when the next command worked fine.

Can you do 'ls -la /var/lib/jenkins'?

> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -l
> /var/lib/jenkins/jobs/*/config.xml
> -rw-r--r-- 1 jenkins root  7759 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.Dhrystone/config.xml
> -rw-r--r-- 1 jenkins root  7758 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.GLMark/config.xml
> -rw-r--r-- 1 jenkins root  7851 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.IOzone/config.xml
> -rw-r--r-- 1 jenkins root  7791 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.Interbench/config.xml
> -rw-r--r-- 1 jenkins root  7639 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.Java/config.xml
> -rw-r--r-- 1 jenkins root  7632 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.OpenSSL/config.xml
> -rw-r--r-- 1 jenkins root  7770 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.Stream/config.xml
> -rw-r--r-- 1 jenkins root  7752 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.Whetstone/config.xml
> -rw-r--r-- 1 jenkins root  7785 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.aim7/config.xml
> -rw-r--r-- 1 jenkins root  7873 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.blobsallad/config.xml
> -rw-r--r-- 1 jenkins root  7438 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.bonnie/config.xml
> -rw-r--r-- 1 jenkins root  7427 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.cyclictest/config.xml
> -rw-r--r-- 1 jenkins root  7833 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.dbench/config.xml
> -rw-r--r-- 1 jenkins root  7488 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.ebizzy/config.xml
> -rw-r--r-- 1 jenkins root  7712 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.ffsb/config.xml
> -rw-r--r-- 1 jenkins root  7872 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.fio/config.xml
> -rw-r--r-- 1 jenkins root  7760 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.gtkperf/config.xml
> -rw-r--r-- 1 jenkins root  7400 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.hackbench/config.xml
> -rw-r--r-- 1 jenkins root  7742 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.himeno/config.xml
> -rw-r--r-- 1 jenkins root  7851 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.iperf/config.xml
> -rw-r--r-- 1 jenkins root  7740 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.linpack/config.xml
> -rw-r--r-- 1 jenkins root  7770 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.lmbench2/config.xml
> -rw-r--r-- 1 jenkins root  7856 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.nbench-byte/config.xml
> -rw-r--r-- 1 jenkins root  7795 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.netperf/config.xml
> -rw-r--r-- 1 jenkins root  7861 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.netpipe/config.xml
> -rw-r--r-- 1 jenkins root  7207 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.reboot/config.xml
> -rw-r--r-- 1 jenkins root  7267 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.signaltest/config.xml
> -rw-r--r-- 1 jenkins root  7849 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.tiobench/config.xml
> -rw-r--r-- 1 jenkins root  7720 Sep 22 12:25
> /var/lib/jenkins/jobs/Benchmark.x11perf/config.xml
> -rw-r--r-- 1 jenkins root  6997 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.LTP.Devices/config.xml
> -rw-r--r-- 1 jenkins root  7269 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.LTP.Filesystem/config.xml
> -rw-r--r-- 1 jenkins root  7615 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.LTP.Open_Posix/config.xml
> -rw-r--r-- 1 jenkins root  6946 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.OpenSSL/config.xml
> -rw-r--r-- 1 jenkins root  7131 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.aiostress/config.xml
> -rw-r--r-- 1 jenkins root  7121 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.arch_timer/config.xml
> -rw-r--r-- 1 jenkins root  7135 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.bc/config.xml
> -rw-r--r-- 1 jenkins root  7232 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.bzip2/config.xml
> -rw-r--r-- 1 jenkins root  7107 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.cmt/config.xml
> -rw-r--r-- 1 jenkins root  7153 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.crashme/config.xml
> -rw-r--r-- 1 jenkins root  7269 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.expat/config.xml
> -rw-r--r-- 1 jenkins root  7238 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.fontconfig/config.xml
> -rw-r--r-- 1 jenkins root  6938 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.ft2demos/config.xml
> -rw-r--r-- 1 jenkins root  7143 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.glib/config.xml
> -rw-r--r-- 1 jenkins root  7133 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.hello_world/config.xml
> -rw-r--r-- 1 jenkins root  7221 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.ipv6connect/config.xml
> -rw-r--r-- 1 jenkins root  6929 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.jpeg/config.xml
> -rw-r--r-- 1 jenkins root  7673 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.libpng/config.xml
> -rw-r--r-- 1 jenkins root  7139 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.linus_stress/config.xml
> -rw-r--r-- 1 jenkins root  7243 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.netperf/config.xml
> -rw-r--r-- 1 jenkins root  7157 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.pi_tests/config.xml
> -rw-r--r-- 1 jenkins root  7248 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.posixtestsuite/config.xml
> -rw-r--r-- 1 jenkins root  7206 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.rmaptest/config.xml
> -rw-r--r-- 1 jenkins root  7115 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.scifab/config.xml
> -rw-r--r-- 1 jenkins root  7130 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.scrashme/config.xml
> -rw-r--r-- 1 jenkins root  7115 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.sdhi_0/config.xml
> -rw-r--r-- 1 jenkins root  7322 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.stress/config.xml
> -rw-r--r-- 1 jenkins root  7130 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.synctest/config.xml
> -rw-r--r-- 1 jenkins root  6924 Sep 22 12:25
> /var/lib/jenkins/jobs/Functional.zlib/config.xml
> -rw-r--r-- 1 jenkins root  7004 Sep 22 12:25
> /var/lib/jenkins/jobs/Matrix.Nightly/config.xml
> -rw-r--r-- 1 jenkins root  6635 Sep 22 12:25
> /var/lib/jenkins/jobs/Matrix.Official/config.xml
> -rw-r--r-- 1 jenkins root  1559 Sep 22 12:25
> /var/lib/jenkins/jobs/Reports.make_pdf/config.xml
> -rw-r--r-- 1 jenkins root 11405 Sep 22 12:25 /var/lib/jenkins/jobs/Run ALL tests
> on ALL targets/config.xml
> -rw-r--r-- 1 jenkins root 12164 Sep 22 12:25 /var/lib/jenkins/jobs/Run ALL tests
> on SELECTED targets/config.xml
> -rw-r--r-- 1 jenkins root 11555 Sep 22 12:25 /var/lib/jenkins/jobs/Run SELECTED
> tests on SELECTED targets/config.xml
> -rw-r--r-- 1 root    root  1548 Sep 22 12:36
> /var/lib/jenkins/jobs/Service.ReloadConfiguration/config.xml
> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -l /var/lib/jenkins/jobs
> lrwxrwxrwx 1 jenkins root 24 Sep 22 12:25 /var/lib/jenkins/jobs ->
> /home/jenkins/fuego/jobs

Isn't this the same command that failed earlier?  I'm confused.

> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# git log -n 1
> commit de894e90b2d5c8d5d630a08d6a22cbcddc16686b
> Author: Tim Bird <tim.bird@am.sony.com <mailto:tim.bird@am.sony.com> >
> Date:   Fri Sep 23 00:04:22 2016 +0000
> 
>     Allow absence of test_pre_check
> 
>     Add '|| true' to test_pre_check test, to allow its absence
>     without problems.

This looks OK.

> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -l /home/jenkins/logs
> lrwxrwxrwx 1 jenkins root 14 Sep 22 12:25 /home/jenkins/logs -> /userdata/logs
> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -la /userdata/logs
> total 8
> drwxr-xr-x 1 dc dc  170 Sep 22 12:09 .
> drwxr-xr-x 1 dc dc  238 Sep 22 12:09 ..
> -rw-r--r-- 1 dc dc  144 Sep 22 12:09 README
> drwxr-xr-x 1 dc dc  102 Sep 22 12:09 logruns
> -rw-r--r-- 1 dc dc 1622 Sep 22 12:09 tests.info <http://tests.info>
> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego#

On mine, the /userdata/logs directory has these permissions and ownership:
root@tlinux:/home/jenkins# ls -l /userdata/logs
total 40
drwxr-xr-x 5 jenkins 2060452353 4096 Aug 26 00:49 Benchmark.Dhrystone
drwxr-xr-x 5 jenkins nogroup    4096 Sep 12 22:20 Functional.aiostress
drwxr-xr-x 5 jenkins nogroup    4096 Sep 12 23:27 Functional.arch_timer
drwxr-xr-x 5 jenkins 2060452353 4096 Aug 25 22:49 Functional.bc
drwxr-xr-x 5 jenkins 2060452353 4096 Aug 26 01:05 Functional.hello_world
drwxr-xr-x 5 jenkins nogroup    4096 Sep 23 16:28 Functional.stress
-rw-r--r-- 1 jenkins 2060452353  144 Aug 10 22:32 README
drwxr-xr-x 2 jenkins 2060452353 4096 Aug 10 22:32 logruns
drwxr-xr-x 9 jenkins 2060452353 4096 Sep 23 01:20 slaves
-rw-r--r-- 1 jenkins 2060452353 1622 Aug 10 22:32 tests.info

The userdata directory is a volume mount from the container filesystem
back out into the host filesystem.  These should be owned by jenkins inside
the container, but the uids may map to something different on the host.
try: 'chown -R jenkins /userdata' (inside the container)

I still don't know what's going on with not seeing all the tests.

Can you go to the user interface for fuego, and navigate to:
Manage Jenkins -> Reload Configuration from Disk, click on that,
answer "OK" in the dialog that appears, then select the Functional tab
again?

I'm grasping at straws, but it looks like somehow some of the permissions
got messed up.
 -- Tim

Re: [Fuego] [LTSI-dev] Failure on trial run of Fuego

[Doug Crawford]

> On Sep 23, 2016, at 2:36 PM, Bird, Timothy <Tim.Bird@am.sony.com> wrote:
> 
>> -----Original Message-----
>> From: ltsi-dev-bounces@lists.linuxfoundation.org [mailto:ltsi-dev-
>> 	On Sep 23, 2016, at 12:46 PM, Bird, Timothy <Tim.Bird@am.sony.com
>> <mailto:Tim.Bird@am.sony.com> > wrote:
>> 		From: Doug Crawford [mailto:dcrawford@zonoff.com]
>> 		I ran through the installation from
>> 		https://bitbucket.org/tbird20d/fuego/
>> 
>> 		And followed quick start here:
>> 		http://bird.org/fuego/Fuego_Quickstart_Guide
>> 
>> 		Excellent documentation-  what a pleasure!
>> 
>> 		I think the quick start misses the “git pull” on
>> /home/jenkins/fuego that the installation instructions covered, perhaps
>> 		intentional, perhaps oversight.
>> 
>> 		I progressed through the Quick start well, setting up for
>> Raspberry Pi3,  all the way to running a test.
>> 		The system seemed happy with the new target.
>> 
>> 		I proceeded to try to run the .bc test recommended in the quick
>> start and ran into a deviation from the expected:
>> 		On the functional tab of the test listing, I only have two tests-
>> 		.netsurf
>> 		.stress
>> 		… and no .bc test.
>> 
>> 
>> 
>> 	Hmmm.  That's not right at all.  There should be about 29 tests on the
>> Functional tab in the test view.
>> 	The first one should be Functional.aiostress, and the last one
>> Functional.zlib.
>> 
>> 	Can you, at the shell inside the container, do an 'ls -l
>> /var/lib/jenkins/jobs'
>> 	and a 'ls -l /var/lib/jenkins/jobs/*/config.xml'?
>> 
>> 	This is the directory Jenkins should see the test definitions in.  It should
>> 	be symlinked to:
>> 	/home/jenkins/fuego/jobs (which is inside the fuego-core git
>> repository).
>> 
>> 
>> 
>> 		I went back and double checked that the “git pull” was done on
>> /home/jenkins/fuego. It was.
>> 
>> 
>> 
>> 	Can you do 'git log -n 1' in /home/jenkins/fuego?
>> 
>> 
>> 
>> 		So I substituted .stress instead of the .bc that the instructions
>> said would be listed, and ran it.
>> 		I got the following failure:
>> 
>> 
>> 
>> 		---------
>> 		Probably doesn’t like that “anonymous” user?
>> 		Something must have been assumed on the quick start that was
>> different for me…
>> 
>> 
>> 
>> 	No, the anonymous user is OK. It looks like something wrong with the
>> permissions
>> 	or the setup.  That first 'mkdir -p' to create the log directory for the test
>> should never fail.
>> 
>> 	Please do an 'ls -l /home/jenkins/logs', and 'ls -la /userdata/logs'
>> 	and send the results to me (and the list).
>> 
>> 	Here are the first few lines of my test output:
>> 	------
>> 	Started by user anonymous
>> 	Running remotely on bbb-poky-sdk in workspace
>> /home/jenkins/buildzone
>> 	[buildzone] $ /bin/sh -xe /tmp/hudson1421222459709266870.sh
>> 	+ '[' '!' -d /home/jenkins/logs/Functional.stress ']'
>> 	+ mkdir -p /home/jenkins/logs/Functional.stress
>> 	+ echo testplan_default
>> 	+ TESTPLAN=testplans/testplan_default.json
>> 	+ source /home/jenkins/tests/Functional.stress/stress.sh
>> 	++ tarball=stress-1.0.4.tar.gz
>> 	++ . /home/jenkins/scripts/functional.sh
>> 	+++ source /home/jenkins/scripts/overlays.sh
>> 	...
>> 
>> 	I hope it's not something with the extra --pid="host", messing with the
>> permissions.  The fact that only some
>> 	of the tests are visible is interesting.
>> 
>> 	Sorry you're having problems.  But thanks for hanging in there.  I'd like
>> to figure out what went wrong.
>> 	-- Tim
>> 
>> 
>> 
>> Here’s the whole session from the requested commands(in bold):
>> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# git pull (OK you didn’t
>> ask for it, but wanted to show I did it :)
>> remote: Counting objects: 5, done.
>> remote: Compressing objects: 100% (5/5), done.
>> remote: Total 5 (delta 3), reused 0 (delta 0)
>> Unpacking objects: 100% (5/5), done.
>> From https://bitbucket.org/tbird20d/fuego-core
>>   2dd5187..de894e9  master     -> origin/master
>> Updating 2dd5187..de894e9
>> Fast-forward
>> engine/scripts/functions.sh | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# 'ls -l
>> /var/lib/jenkins/jobs'
>> bash: ls -l /var/lib/jenkins/jobs: No such file or directory
> 
> 
> huh?  I don't see how that failed, when the next command worked fine.
> 

Sorry that had “tics” in the command.  Try again:
root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -l /var/lib/jenkins/jobs
lrwxrwxrwx 1 jenkins root 24 Sep 22 12:25 /var/lib/jenkins/jobs -> /home/jenkins/fuego/jobs

> Can you do 'ls -la /var/lib/jenkins’?

root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -la /var/lib/jenkins
total 132
drwxr-xr-x 19 jenkins adm     4096 Sep 23 15:41 .
drwxr-xr-x 45 root    root    4096 Sep 22 12:34 ..
drwxr-xr-x  3 jenkins nogroup 4096 Sep 22 12:26 .cache
drwxr-xr-x  4 jenkins nogroup 4096 Sep 22 12:34 .java
-rw-r--r--  1 jenkins nogroup   49 Sep 23 17:43 .owner
-rw-r--r--  1 jenkins nogroup    0 Sep 22 22:58 Fingerprint cleanup.log
-rw-r--r--  1 jenkins root     106 Sep 22 12:09 README
-rw-r--r--  1 jenkins nogroup   89 Sep 23 17:57 Workspace clean-up.log
-rw-r--r--  1 jenkins root     670 Sep 22 12:09 buildstep-config-files.xml
lrwxrwxrwx  1 root    root      25 Sep 23 15:41 config.xml -> /userdata/conf/config.xml
-rw-r--r--  1 jenkins root     780 Sep 22 12:09 custom-config-files.xml
-rw-r--r--  1 jenkins root     811 Sep 22 12:09 hudson.maven.MavenModuleSet.xml
-rw-r--r--  1 jenkins nogroup  159 Sep 22 12:34 hudson.model.UpdateCenter.xml
-rw-r--r--  1 jenkins root     370 Sep 22 12:09 hudson.plugins.git.GitTool.xml
-rw-r--r--  1 jenkins root     288 Sep 22 12:09 hudson.plugins.groovy.Groovy.xml
-rw-r--r--  1 jenkins root    4068 Sep 22 12:26 hudson.plugins.page_markup.PageMarkupPageDecorator.xml
-rw-r--r--  1 jenkins root     235 Sep 22 12:09 hudson.plugins.throttleconcurrents.ThrottleJobProperty.xml
-rw-r--r--  1 jenkins root     196 Sep 22 12:09 hudson.scm.CVSSCM.xml
-rw-r--r--  1 jenkins root     350 Sep 22 12:09 hudson.scm.SubversionSCM.xml
-rw-r--r--  1 jenkins root     145 Sep 22 12:09 hudson.tasks.Ant.xml
-rw-r--r--  1 jenkins root     187 Sep 22 12:09 hudson.tasks.Mailer.xml
-rw-r--r--  1 jenkins root     132 Sep 22 12:09 hudson.tasks.Maven.xml
-rw-r--r--  1 jenkins root      76 Sep 22 12:09 hudson.tasks.Shell.xml
-rw-r--r--  1 jenkins root     215 Sep 22 12:09 hudson.triggers.SCMTrigger.xml
-rw-r--r--  1 jenkins root    1679 Sep 22 12:09 identity.key
-rw-r--r--  1 jenkins root     260 Sep 22 12:09 jenkins.model.JenkinsLocationConfiguration.xml
lrwxrwxrwx  1 jenkins root      24 Sep 22 12:25 jobs -> /home/jenkins/fuego/jobs
-rw-r--r--  1 jenkins root     321 Sep 22 12:09 jp.ikedam.jenkins.plugins.extensible_choice_parameter.GlobalTextareaChoiceListProvider.xml
-rwxr-xr-x  1 jenkins root     146 Sep 22 12:26 locale.xml
lrwxrwxrwx  1 jenkins root      14 Sep 22 12:25 logs -> /userdata/logs
-rw-r--r--  1 jenkins nogroup  907 Sep 22 12:34 nodeMonitors.xml
-rw-r--r--  1 jenkins root     199 Sep 22 12:09 org.codefirst.SimpleThemeDecorator.xml
drwxr-xr-x 97 jenkins root    4096 Sep 22 12:34 plugins
lrwxrwxrwx  1 jenkins root      42 Sep 22 12:25 scriptler -> /home/jenkins/fuego/plugins-conf/scriptler
-rw-r--r--  1 jenkins root      64 Sep 22 12:09 secret.key
-rw-r--r--  1 jenkins root       0 Sep 22 12:09 secret.key.not-so-secret
drwxr-xr-x  2 jenkins root    4096 Sep 22 12:26 secrets
lrwxrwxrwx  1 jenkins root      49 Sep 22 12:25 sidebar-link.xml -> /home/jenkins/fuego/plugins-conf/sidebar-link.xml
drwxr-xr-x  2 jenkins root    4096 Sep 23 15:41 updates
drwxr-xr-x  5 jenkins root    4096 Sep 22 12:26 userContent
root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# 

> 

>> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -l
>> /var/lib/jenkins/jobs/*/config.xml
>> -rw-r--r-- 1 jenkins root  7759 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.Dhrystone/config.xml
>> -rw-r--r-- 1 jenkins root  7758 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.GLMark/config.xml
>> -rw-r--r-- 1 jenkins root  7851 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.IOzone/config.xml
>> -rw-r--r-- 1 jenkins root  7791 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.Interbench/config.xml
>> -rw-r--r-- 1 jenkins root  7639 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.Java/config.xml
>> -rw-r--r-- 1 jenkins root  7632 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.OpenSSL/config.xml
>> -rw-r--r-- 1 jenkins root  7770 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.Stream/config.xml
>> -rw-r--r-- 1 jenkins root  7752 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.Whetstone/config.xml
>> -rw-r--r-- 1 jenkins root  7785 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.aim7/config.xml
>> -rw-r--r-- 1 jenkins root  7873 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.blobsallad/config.xml
>> -rw-r--r-- 1 jenkins root  7438 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.bonnie/config.xml
>> -rw-r--r-- 1 jenkins root  7427 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.cyclictest/config.xml
>> -rw-r--r-- 1 jenkins root  7833 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.dbench/config.xml
>> -rw-r--r-- 1 jenkins root  7488 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.ebizzy/config.xml
>> -rw-r--r-- 1 jenkins root  7712 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.ffsb/config.xml
>> -rw-r--r-- 1 jenkins root  7872 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.fio/config.xml
>> -rw-r--r-- 1 jenkins root  7760 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.gtkperf/config.xml
>> -rw-r--r-- 1 jenkins root  7400 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.hackbench/config.xml
>> -rw-r--r-- 1 jenkins root  7742 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.himeno/config.xml
>> -rw-r--r-- 1 jenkins root  7851 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.iperf/config.xml
>> -rw-r--r-- 1 jenkins root  7740 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.linpack/config.xml
>> -rw-r--r-- 1 jenkins root  7770 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.lmbench2/config.xml
>> -rw-r--r-- 1 jenkins root  7856 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.nbench-byte/config.xml
>> -rw-r--r-- 1 jenkins root  7795 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.netperf/config.xml
>> -rw-r--r-- 1 jenkins root  7861 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.netpipe/config.xml
>> -rw-r--r-- 1 jenkins root  7207 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.reboot/config.xml
>> -rw-r--r-- 1 jenkins root  7267 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.signaltest/config.xml
>> -rw-r--r-- 1 jenkins root  7849 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.tiobench/config.xml
>> -rw-r--r-- 1 jenkins root  7720 Sep 22 12:25
>> /var/lib/jenkins/jobs/Benchmark.x11perf/config.xml
>> -rw-r--r-- 1 jenkins root  6997 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.LTP.Devices/config.xml
>> -rw-r--r-- 1 jenkins root  7269 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.LTP.Filesystem/config.xml
>> -rw-r--r-- 1 jenkins root  7615 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.LTP.Open_Posix/config.xml
>> -rw-r--r-- 1 jenkins root  6946 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.OpenSSL/config.xml
>> -rw-r--r-- 1 jenkins root  7131 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.aiostress/config.xml
>> -rw-r--r-- 1 jenkins root  7121 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.arch_timer/config.xml
>> -rw-r--r-- 1 jenkins root  7135 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.bc/config.xml
>> -rw-r--r-- 1 jenkins root  7232 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.bzip2/config.xml
>> -rw-r--r-- 1 jenkins root  7107 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.cmt/config.xml
>> -rw-r--r-- 1 jenkins root  7153 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.crashme/config.xml
>> -rw-r--r-- 1 jenkins root  7269 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.expat/config.xml
>> -rw-r--r-- 1 jenkins root  7238 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.fontconfig/config.xml
>> -rw-r--r-- 1 jenkins root  6938 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.ft2demos/config.xml
>> -rw-r--r-- 1 jenkins root  7143 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.glib/config.xml
>> -rw-r--r-- 1 jenkins root  7133 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.hello_world/config.xml
>> -rw-r--r-- 1 jenkins root  7221 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.ipv6connect/config.xml
>> -rw-r--r-- 1 jenkins root  6929 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.jpeg/config.xml
>> -rw-r--r-- 1 jenkins root  7673 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.libpng/config.xml
>> -rw-r--r-- 1 jenkins root  7139 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.linus_stress/config.xml
>> -rw-r--r-- 1 jenkins root  7243 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.netperf/config.xml
>> -rw-r--r-- 1 jenkins root  7157 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.pi_tests/config.xml
>> -rw-r--r-- 1 jenkins root  7248 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.posixtestsuite/config.xml
>> -rw-r--r-- 1 jenkins root  7206 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.rmaptest/config.xml
>> -rw-r--r-- 1 jenkins root  7115 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.scifab/config.xml
>> -rw-r--r-- 1 jenkins root  7130 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.scrashme/config.xml
>> -rw-r--r-- 1 jenkins root  7115 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.sdhi_0/config.xml
>> -rw-r--r-- 1 jenkins root  7322 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.stress/config.xml
>> -rw-r--r-- 1 jenkins root  7130 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.synctest/config.xml
>> -rw-r--r-- 1 jenkins root  6924 Sep 22 12:25
>> /var/lib/jenkins/jobs/Functional.zlib/config.xml
>> -rw-r--r-- 1 jenkins root  7004 Sep 22 12:25
>> /var/lib/jenkins/jobs/Matrix.Nightly/config.xml
>> -rw-r--r-- 1 jenkins root  6635 Sep 22 12:25
>> /var/lib/jenkins/jobs/Matrix.Official/config.xml
>> -rw-r--r-- 1 jenkins root  1559 Sep 22 12:25
>> /var/lib/jenkins/jobs/Reports.make_pdf/config.xml
>> -rw-r--r-- 1 jenkins root 11405 Sep 22 12:25 /var/lib/jenkins/jobs/Run ALL tests
>> on ALL targets/config.xml
>> -rw-r--r-- 1 jenkins root 12164 Sep 22 12:25 /var/lib/jenkins/jobs/Run ALL tests
>> on SELECTED targets/config.xml
>> -rw-r--r-- 1 jenkins root 11555 Sep 22 12:25 /var/lib/jenkins/jobs/Run SELECTED
>> tests on SELECTED targets/config.xml
>> -rw-r--r-- 1 root    root  1548 Sep 22 12:36
>> /var/lib/jenkins/jobs/Service.ReloadConfiguration/config.xml
>> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -l /var/lib/jenkins/jobs
>> lrwxrwxrwx 1 jenkins root 24 Sep 22 12:25 /var/lib/jenkins/jobs ->
>> /home/jenkins/fuego/jobs
> 
> Isn't this the same command that failed earlier?  I'm confused.
(Yes I redid it without the tics around it, sorry for including both)
> 
>> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# git log -n 1
>> commit de894e90b2d5c8d5d630a08d6a22cbcddc16686b
>> Author: Tim Bird <tim.bird@am.sony.com <mailto:tim.bird@am.sony.com> >
>> Date:   Fri Sep 23 00:04:22 2016 +0000
>> 
>>    Allow absence of test_pre_check
>> 
>>    Add '|| true' to test_pre_check test, to allow its absence
>>    without problems.
> 
> This looks OK.
> 
>> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -l /home/jenkins/logs
>> lrwxrwxrwx 1 jenkins root 14 Sep 22 12:25 /home/jenkins/logs -> /userdata/logs
>> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego# ls -la /userdata/logs
>> total 8
>> drwxr-xr-x 1 dc dc  170 Sep 22 12:09 .
>> drwxr-xr-x 1 dc dc  238 Sep 22 12:09 ..
>> -rw-r--r-- 1 dc dc  144 Sep 22 12:09 README
>> drwxr-xr-x 1 dc dc  102 Sep 22 12:09 logruns
>> -rw-r--r-- 1 dc dc 1622 Sep 22 12:09 tests.info <http://tests.info>
>> root@vagrant-ubuntu-trusty-64:/home/jenkins/fuego#
> 
> On mine, the /userdata/logs directory has these permissions and ownership:
> root@tlinux:/home/jenkins# ls -l /userdata/logs
> total 40
> drwxr-xr-x 5 jenkins 2060452353 4096 Aug 26 00:49 Benchmark.Dhrystone
> drwxr-xr-x 5 jenkins nogroup    4096 Sep 12 22:20 Functional.aiostress
> drwxr-xr-x 5 jenkins nogroup    4096 Sep 12 23:27 Functional.arch_timer
> drwxr-xr-x 5 jenkins 2060452353 4096 Aug 25 22:49 Functional.bc
> drwxr-xr-x 5 jenkins 2060452353 4096 Aug 26 01:05 Functional.hello_world
> drwxr-xr-x 5 jenkins nogroup    4096 Sep 23 16:28 Functional.stress
> -rw-r--r-- 1 jenkins 2060452353  144 Aug 10 22:32 README
> drwxr-xr-x 2 jenkins 2060452353 4096 Aug 10 22:32 logruns
> drwxr-xr-x 9 jenkins 2060452353 4096 Sep 23 01:20 slaves
> -rw-r--r-- 1 jenkins 2060452353 1622 Aug 10 22:32 tests.info
> 
> The userdata directory is a volume mount from the container filesystem
> back out into the host filesystem.  These should be owned by jenkins inside
> the container, but the uids may map to something different on the host.
> try: 'chown -R jenkins /userdata' (inside the container)
> 
> I still don't know what's going on with not seeing all the tests.
> 
> Can you go to the user interface for fuego, and navigate to:
> Manage Jenkins -> Reload Configuration from Disk, click on that,
> answer "OK" in the dialog that appears, then select the Functional tab
> again?

I did the chown command and then the reload.  Now there are no tests under the functional tab.
Oh oh.

> 
> I'm grasping at straws, but it looks like somehow some of the permissions
> got messed up.

Probably so.  I can redo the install and pay closer attention to what user I’m doing
everything from.  The quick start begins with commands, is there an assumed user privilege 
you use?


> -- Tim
> 

Re: [Fuego] [LTSI-dev] Failure on trial run of Fuego

[Bird, Timothy]

> -----Original Message-----
> From: Doug Crawford [mailto:dcrawford@zonoff.com]
> I did the chown command and then the reload.  Now there are no tests under
> the functional tab.
> Oh oh.

oh crud. ha ha.  sometimes it's darkest before the dawn. :-)

> >
> > I'm grasping at straws, but it looks like somehow some of the permissions
> > got messed up.
> 
> Probably so.  I can redo the install and pay closer attention to what user I’m
> doing
> everything from.  The quick start begins with commands, is there an assumed
> user privilege
> you use?

No.  you should be able to do install.sh and the docker... commands as your regular user account.
All these scripts contain 'sudo' operations, though, for operations that need to be run
as a privileged user.
 -- Tim

Re: [Fuego] [LTSI-dev] Failure on trial run of Fuego

[Doug Crawford]

> On Sep 23, 2016, at 2:57 PM, Bird, Timothy <Tim.Bird@am.sony.com> wrote:
> 
>> -----Original Message-----
>> From: Doug Crawford [mailto:dcrawford@zonoff.com]
>> I did the chown command and then the reload.  Now there are no tests under
>> the functional tab.
>> Oh oh.
> 
> oh crud. ha ha.  sometimes it's darkest before the dawn. :-)
> 
>>> 
>>> I'm grasping at straws, but it looks like somehow some of the permissions
>>> got messed up.
>> 
>> Probably so.  I can redo the install and pay closer attention to what user I’m
>> doing
>> everything from.  The quick start begins with commands, is there an assumed
>> user privilege
>> you use?
> 
> No.  you should be able to do install.sh and the docker... commands as your regular user account.
> All these scripts contain 'sudo' operations, though, for operations that need to be run
> as a privileged user.
> -- Tim
> 

Would trying to make my VM or Docker machine visible to the outside world- so that you could access it- be worth
working towards?

Re: [Fuego] [LTSI-dev] Failure on trial run of Fuego

[Doug Crawford]

> On Sep 23, 2016, at 2:57 PM, Bird, Timothy <Tim.Bird@am.sony.com> wrote:
> 
>> -----Original Message-----
>> From: Doug Crawford [mailto:dcrawford@zonoff.com]
>> I did the chown command and then the reload.  Now there are no tests under
>> the functional tab.
>> Oh oh.
> 
> oh crud. ha ha.  sometimes it's darkest before the dawn. :-)
> 
>>> 
>>> I'm grasping at straws, but it looks like somehow some of the permissions
>>> got messed up.
>> 
>> Probably so.  I can redo the install and pay closer attention to what user I’m
>> doing
>> everything from.  The quick start begins with commands, is there an assumed
>> user privilege
>> you use?
> 
> No.  you should be able to do install.sh and the docker... commands as your regular user account.
> All these scripts contain 'sudo' operations, though, for operations that need to be run
> as a privileged user.
> -- Tim
> 

Shut everything down and started it up again and the tests are now all listed.
.bc test failed the same way as the stress test.
I’ll look into the permissions next week.
Thanks for the help!

Re: [Fuego] [LTSI-dev] Failure on trial run of Fuego

[Doug Crawford]

> On Sep 23, 2016, at 2:57 PM, Bird, Timothy <Tim.Bird@am.sony.com> wrote:
> 
>> -----Original Message-----
>> From: Doug Crawford [mailto:dcrawford@zonoff.com]
>> I did the chown command and then the reload.  Now there are no tests under
>> the functional tab.
>> Oh oh.
> 
> oh crud. ha ha.  sometimes it's darkest before the dawn. :-)
> 
>>> 
>>> I'm grasping at straws, but it looks like somehow some of the permissions
>>> got messed up.
>> 
>> Probably so.  I can redo the install and pay closer attention to what user I’m
>> doing
>> everything from.  The quick start begins with commands, is there an assumed
>> user privilege
>> you use?
> 
> No.  you should be able to do install.sh and the docker... commands as your regular user account.
> All these scripts contain 'sudo' operations, though, for operations that need to be run
> as a privileged user.
> -- Tim
> 

Tim: I’ll be quiet on this for a few days as I have some alternate tasks assigned.
I plan to sidestep our VM and load up a real computer with everything from scratch.
Should I just install the latest of everything - Ubuntu et al…
I was constrained on our VM by what we use for development, but on my box I can do anything.

Thanks
Doug C.

Re: [Fuego] [LTSI-dev] Failure on trial run of Fuego

[Bird, Timothy]

> -----Original Message-----
> From: Doug Crawford [mailto:dcrawford@zonoff.com]
> Sent: Monday, September 26, 2016 2:22 PM
> To: Bird, Timothy <Tim.Bird@am.sony.com>
> Cc: fuego@lists.linuxfoundation.org
> Subject: Re: [LTSI-dev] Failure on trial run of Fuego
> 
> 
> > On Sep 23, 2016, at 2:57 PM, Bird, Timothy <Tim.Bird@am.sony.com> wrote:
> >
> >> -----Original Message-----
> >> From: Doug Crawford [mailto:dcrawford@zonoff.com]
> >> I did the chown command and then the reload.  Now there are no tests under
> >> the functional tab.
> >> Oh oh.
> >
> > oh crud. ha ha.  sometimes it's darkest before the dawn. :-)
> >
> >>>
> >>> I'm grasping at straws, but it looks like somehow some of the permissions
> >>> got messed up.
> >>
> >> Probably so.  I can redo the install and pay closer attention to what user I’m
> >> doing
> >> everything from.  The quick start begins with commands, is there an assumed
> >> user privilege
> >> you use?
> >
> > No.  you should be able to do install.sh and the docker... commands as your
> regular user account.
> > All these scripts contain 'sudo' operations, though, for operations that need to
> be run
> > as a privileged user.
> > -- Tim
> >
> 
> Tim: I’ll be quiet on this for a few days as I have some alternate tasks assigned.
> I plan to sidestep our VM and load up a real computer with everything from
> scratch.
> Should I just install the latest of everything - Ubuntu et al…
> I was constrained on our VM by what we use for development, but on my box I
> can do anything.

I would use latest everything.  If you could tell us what version of docker comes stock
with Ubuntu 16.06 that would be good.  We might need to include "how to get the latest
docker" information in our instructions.

Thanks,
 -- Tim

Re: [Fuego] [LTSI-dev] Failure on trial run of Fuego

[Doug Crawford]

> On Sep 27, 2016, at 3:14 PM, Bird, Timothy <Tim.Bird@am.sony.com> wrote:
> 
> 
> 
>> -----Original Message-----
>> From: Doug Crawford [mailto:dcrawford@zonoff.com]
>> Sent: Monday, September 26, 2016 2:22 PM
>> To: Bird, Timothy <Tim.Bird@am.sony.com>
>> Cc: fuego@lists.linuxfoundation.org
>> Subject: Re: [LTSI-dev] Failure on trial run of Fuego
>> 
>> 
>>> On Sep 23, 2016, at 2:57 PM, Bird, Timothy <Tim.Bird@am.sony.com> wrote:
>>> 
>>>> -----Original Message-----
>>>> From: Doug Crawford [mailto:dcrawford@zonoff.com]
>>>> I did the chown command and then the reload.  Now there are no tests under
>>>> the functional tab.
>>>> Oh oh.
>>> 
>>> oh crud. ha ha.  sometimes it's darkest before the dawn. :-)
>>> 
>>>>> 
>>>>> I'm grasping at straws, but it looks like somehow some of the permissions
>>>>> got messed up.
>>>> 
>>>> Probably so.  I can redo the install and pay closer attention to what user I’m
>>>> doing
>>>> everything from.  The quick start begins with commands, is there an assumed
>>>> user privilege
>>>> you use?
>>> 
>>> No.  you should be able to do install.sh and the docker... commands as your
>> regular user account.
>>> All these scripts contain 'sudo' operations, though, for operations that need to
>> be run
>>> as a privileged user.
>>> -- Tim
>>> 
>> 
>> Tim: I’ll be quiet on this for a few days as I have some alternate tasks assigned.
>> I plan to sidestep our VM and load up a real computer with everything from
>> scratch.
>> Should I just install the latest of everything - Ubuntu et al…
>> I was constrained on our VM by what we use for development, but on my box I
>> can do anything.
> 
> I would use latest everything.  If you could tell us what version of docker comes stock
> with Ubuntu 16.06 that would be good.  We might need to include "how to get the latest
> docker" information in our instructions.
> 
> Thanks,
> Tim
> 

Thanks, very good.  I was thinking of working with the items that you mentioned in a previous post that you were currently using,
but I’ll try what you suggest here.

Re: [Fuego] [LTSI-dev] Value of the Fuego to kernel "integrators"?

[Doug Crawford]

> On Sep 23, 2016, at 11:05 AM, Greg KH <gregkh@linuxfoundation.org> wrote:
> 
> On Fri, Sep 23, 2016 at 10:09:02AM -0400, Doug Crawford via LTSI-dev wrote:
>> A perception in our organization is that Fuego is intended for kernel
>> developers in the development of long term supported kernels.
>> We are more “kernel integrators”;  we pick up LTS kernels through
>> Yocto, and integrate them with our choice of file system and device
>> drivers.
> 
> What filesystems and device drivers do you use that are not already
> upstream in the kernel.org releases?  Need any help getting them merged
> properly?
> 
> thanks,
> 
> greg k-h


Sorry if I mislead- we use only standard file systems.
We sometimes use bug-fixed device drivers from the silicon vendors.
We do not write any ourselves.

I’m trying to confirm that there would be a general consensus that it is a GOOD
idea to consider running Fuego even against a system comprised of 
Kernels build from Yocto combined manufacturer device drivers.
Seems to me that after a build with a custom configuration and combination of
components it WOULD be a good idea.

Re: [Fuego] [LTSI-dev] Value of the Fuego to kernel "integrators"?

[Bird, Timothy]

> -----Original Message-----
> From: Doug Crawford [mailto:dcrawford@zonoff.com]
> Sent: Friday, September 23, 2016 10:21 AM
> To: Greg KH <gregkh@linuxfoundation.org>
> Cc: Bird, Timothy <Tim.Bird@am.sony.com>; ltsi-dev@lists.linuxfoundation.org;
> fuego@lists.linuxfoundation.org
> Subject: Re: [LTSI-dev] Value of the Fuego to kernel "integrators"?
> 
> 
> > On Sep 23, 2016, at 11:05 AM, Greg KH <gregkh@linuxfoundation.org> wrote:
> >
> > On Fri, Sep 23, 2016 at 10:09:02AM -0400, Doug Crawford via LTSI-dev wrote:
> >> A perception in our organization is that Fuego is intended for kernel
> >> developers in the development of long term supported kernels.
> >> We are more “kernel integrators”;  we pick up LTS kernels through
> >> Yocto, and integrate them with our choice of file system and device
> >> drivers.
> >
> > What filesystems and device drivers do you use that are not already
> > upstream in the kernel.org releases?  Need any help getting them merged
> > properly?
> >
> > thanks,
> >
> > greg k-h
> 
> 
> Sorry if I mislead- we use only standard file systems.
> We sometimes use bug-fixed device drivers from the silicon vendors.
> We do not write any ourselves.
> 
> I’m trying to confirm that there would be a general consensus that it is a GOOD
> idea to consider running Fuego even against a system comprised of
> Kernels build from Yocto combined manufacturer device drivers.
> Seems to me that after a build with a custom configuration and combination of
> components it WOULD be a good idea.

Fuego should be able to be used to validate aspects of the system, no matter whether
the kernel is a vanilla mainline kernel from kernel.org, or LTS or LTSI, or a kernel with
vendor-specific extensions.  The same is true for the "distribution", which is the set
of libraries and filesystem used on the device.

What type of testing you want to do is up to you.  To be honest, we are in the early days
of Fuego, and while it is being used by some companies for SoC testing, for LTSI and AGL,
we have a ways to go before there's a suite of tests that someone can use "off-the-shelf"
to test all aspects of their system.  Fuego is a framework for test, and has some preexisting
ones, but building out a suite of tests for specific functional areas will take time.

What types of things do you want to test?  networking?  filesystems?, block devices?
memory?, performance?  operation under stress?, power management?  video?, audio?
specific driver functionality?  Some of these general ones are supported today with the
existing integrated tests.  But others require the creation or integration of new tests
for Fuego, and depend on the specific thing to be tested.  There will be some sessions
at the upcoming Embedded Linux Conference, where we'll be discussing the current
status, and specifically what people are using Fuego for now.  The results of these
discussion might help people decide if Fuego can be used for their purposes.

Also, we are in planning stages for a fuego-specific mini-jamboree, in Tokyo, towards
the end of October.

 -- Tim

Re: [Fuego] [LTSI-dev] Value of the Fuego to kernel "integrators"?

[Doug Crawford]

> On Sep 23, 2016, at 1:30 PM, Bird, Timothy <Tim.Bird@am.sony.com> wrote:
> 
> 
> 
>> -----Original Message-----
>> From: Doug Crawford [mailto:dcrawford@zonoff.com]
>> Sent: Friday, September 23, 2016 10:21 AM
>> To: Greg KH <gregkh@linuxfoundation.org>
>> Cc: Bird, Timothy <Tim.Bird@am.sony.com>; ltsi-dev@lists.linuxfoundation.org;
>> fuego@lists.linuxfoundation.org
>> Subject: Re: [LTSI-dev] Value of the Fuego to kernel "integrators"?
>> 
>> 
>>> On Sep 23, 2016, at 11:05 AM, Greg KH <gregkh@linuxfoundation.org> wrote:
>>> 
>>> On Fri, Sep 23, 2016 at 10:09:02AM -0400, Doug Crawford via LTSI-dev wrote:
>>>> A perception in our organization is that Fuego is intended for kernel
>>>> developers in the development of long term supported kernels.
>>>> We are more “kernel integrators”;  we pick up LTS kernels through
>>>> Yocto, and integrate them with our choice of file system and device
>>>> drivers.
>>> 
>>> What filesystems and device drivers do you use that are not already
>>> upstream in the kernel.org releases?  Need any help getting them merged
>>> properly?
>>> 
>>> thanks,
>>> 
>>> greg k-h
>> 
>> 
>> Sorry if I mislead- we use only standard file systems.
>> We sometimes use bug-fixed device drivers from the silicon vendors.
>> We do not write any ourselves.
>> 
>> I’m trying to confirm that there would be a general consensus that it is a GOOD
>> idea to consider running Fuego even against a system comprised of
>> Kernels build from Yocto combined manufacturer device drivers.
>> Seems to me that after a build with a custom configuration and combination of
>> components it WOULD be a good idea.
> 
> Fuego should be able to be used to validate aspects of the system, no matter whether
> the kernel is a vanilla mainline kernel from kernel.org, or LTS or LTSI, or a kernel with
> vendor-specific extensions.  The same is true for the "distribution", which is the set
> of libraries and filesystem used on the device.
> 
> What type of testing you want to do is up to you.  To be honest, we are in the early days
> of Fuego, and while it is being used by some companies for SoC testing, for LTSI and AGL,
> we have a ways to go before there's a suite of tests that someone can use "off-the-shelf"
> to test all aspects of their system.  Fuego is a framework for test, and has some preexisting
> ones, but building out a suite of tests for specific functional areas will take time.
> 
> What types of things do you want to test?  networking?  filesystems?, block devices?
> memory?, performance?  operation under stress?, power management?  video?, audio?

Yes! All the above :)  And more? How about process communication mechanisms?
How about all the OS calls?   Historical reference, the proof that
a PC was IBM compatible - DOS and BIOS wise- was "can it run Lotus 1-2-3?”.  
Today how do we judge whether a particular Linux is a well behaved Linux?
Our shop just uses the Linux system in development for a period of time before
it goes out.  Seems there must be a better way.

> specific driver functionality?  

Generally not special functions of a driver, but specific drivers, yes.

> Some of these general ones are supported today with the
> existing integrated tests.  But others require the creation or integration of new tests
> for Fuego, and depend on the specific thing to be tested.  

Its seems like there are some good ones to get started with in the current set.
I can imagine that I might write some tests that would be universally useful. 
For instance, we’ve had issues with executables present for the wrong architecture,
scp/ethernet performance issues, and file system going read-only on a MTD device.

> There will be some sessions
> at the upcoming Embedded Linux Conference, where we'll be discussing the current
> status, and specifically what people are using Fuego for now.  

Nice.  Perhaps I can get there.

> The results of these
> discussion might help people decide if Fuego can be used for their purposes.
> 

Fuego seems to be to be plenty generalized.  Heck I think it could even drive a production test environment. 
Tests may be offered that stray outside of the direct confines of testing the kernel, which you may or
may not want to include.  That’s up to you folks.  

> Also, we are in planning stages for a fuego-specific mini-jamboree, in Tokyo, towards
> the end of October.
> 

Yikes.  You must be at Sony in Japan...

> -- Tim
> 

Re: [Fuego] [LTSI-dev] Value of the Fuego to kernel "integrators"?

[Bird, Timothy]

> -----Original Message-----
> From: Doug Crawford [mailto:dcrawford@zonoff.com]
> Sent: Friday, September 23, 2016 11:29 AM
> To: Bird, Timothy <Tim.Bird@am.sony.com>
> Cc: Greg KH <gregkh@linuxfoundation.org>; ltsi-dev@lists.linuxfoundation.org;
> fuego@lists.linuxfoundation.org
> Subject: Re: [LTSI-dev] Value of the Fuego to kernel "integrators"?
> 
> 
> > On Sep 23, 2016, at 1:30 PM, Bird, Timothy <Tim.Bird@am.sony.com> wrote:
> >
> >
> >
> >> -----Original Message-----
> >> From: Doug Crawford [mailto:dcrawford@zonoff.com]
> >> Sent: Friday, September 23, 2016 10:21 AM
> >> To: Greg KH <gregkh@linuxfoundation.org>
> >> Cc: Bird, Timothy <Tim.Bird@am.sony.com>; ltsi-
> dev@lists.linuxfoundation.org;
> >> fuego@lists.linuxfoundation.org
> >> Subject: Re: [LTSI-dev] Value of the Fuego to kernel "integrators"?
> >>
> >>
> >>> On Sep 23, 2016, at 11:05 AM, Greg KH <gregkh@linuxfoundation.org>
> wrote:
> >>>
> >>> On Fri, Sep 23, 2016 at 10:09:02AM -0400, Doug Crawford via LTSI-dev
> wrote:
> >>>> A perception in our organization is that Fuego is intended for kernel
> >>>> developers in the development of long term supported kernels.
> >>>> We are more “kernel integrators”;  we pick up LTS kernels through
> >>>> Yocto, and integrate them with our choice of file system and device
> >>>> drivers.
> >>>
> >>> What filesystems and device drivers do you use that are not already
> >>> upstream in the kernel.org releases?  Need any help getting them merged
> >>> properly?
> >>>
> >>> thanks,
> >>>
> >>> greg k-h
> >>
> >>
> >> Sorry if I mislead- we use only standard file systems.
> >> We sometimes use bug-fixed device drivers from the silicon vendors.
> >> We do not write any ourselves.
> >>
> >> I’m trying to confirm that there would be a general consensus that it is a
> GOOD
> >> idea to consider running Fuego even against a system comprised of
> >> Kernels build from Yocto combined manufacturer device drivers.
> >> Seems to me that after a build with a custom configuration and combination
> of
> >> components it WOULD be a good idea.
> >
> > Fuego should be able to be used to validate aspects of the system, no matter
> whether
> > the kernel is a vanilla mainline kernel from kernel.org, or LTS or LTSI, or a
> kernel with
> > vendor-specific extensions.  The same is true for the "distribution", which is
> the set
> > of libraries and filesystem used on the device.
> >
> > What type of testing you want to do is up to you.  To be honest, we are in the
> early days
> > of Fuego, and while it is being used by some companies for SoC testing, for
> LTSI and AGL,
> > we have a ways to go before there's a suite of tests that someone can use
> "off-the-shelf"
> > to test all aspects of their system.  Fuego is a framework for test, and has
> some preexisting
> > ones, but building out a suite of tests for specific functional areas will take
> time.
> >
> > What types of things do you want to test?  networking?  filesystems?, block
> devices?
> > memory?, performance?  operation under stress?, power management?
> video?, audio?
> 
> Yes! All the above :)  And more? How about process communication
> mechanisms?

I think there are IPC tests in LTP.  LTP is kind of a big monster, and I'm not
sure what parts of it we invoke in our tests.

But we would welcome some specific IPC tests, if people know which ones
are good (or which one is best - most useful for finding bugs).

> How about all the OS calls?   Historical reference, the proof that
> a PC was IBM compatible - DOS and BIOS wise- was "can it run Lotus 1-2-3?”.
> Today how do we judge whether a particular Linux is a well behaved Linux?

LTP and the posix test suite are considered fairly indicative of system call
and user-space utility program sanity.

> Our shop just uses the Linux system in development for a period of time before
> it goes out.  Seems there must be a better way.
> 
> > specific driver functionality?
> 
> Generally not special functions of a driver, but specific drivers, yes.
> 
> > Some of these general ones are supported today with the
> > existing integrated tests.  But others require the creation or integration of new
> tests
> > for Fuego, and depend on the specific thing to be tested.
> 
> Its seems like there are some good ones to get started with in the current set.
> I can imagine that I might write some tests that would be universally useful.
> For instance, we’ve had issues with executables present for the wrong
> architecture,
> scp/ethernet performance issues, and file system going read-only on a MTD
> device.

The executables present for the wrong architecture issue is interesting.  
I'm not sure how it would happen, but if you've seen it, it's definitely something
worth testing for.  We could probably construct a test pretty easily to scan
the filesystem, and run 'file' on all the executables to find any wayward file types.
'file' may or may not be on the target, so it might need to be built for the test,
but this would all be pretty straightforward.

Ethernet performance issues should be findable with netperf (I think).

Having a device switch to read-only is a weird thing also.  Fuego can easily detect
the change using a filesystem test (or a more direct custom test of an I/O operation
on a device).  However, figuring out what triggers the change might take some
investigation.  Fuego is just the canary in the coal mine.  Someone has to come along
and figure out why the canary died, and try to fix the problem.

> 
> > There will be some sessions
> > at the upcoming Embedded Linux Conference, where we'll be discussing the
> current
> > status, and specifically what people are using Fuego for now.
> 
> Nice.  Perhaps I can get there.
> 
> > The results of these
> > discussion might help people decide if Fuego can be used for their purposes.
> >
> 
> Fuego seems to be to be plenty generalized.  Heck I think it could even drive a
> production test environment.
> Tests may be offered that stray outside of the direct confines of testing the
> kernel, which you may or
> may not want to include.  That’s up to you folks.
> 
> > Also, we are in planning stages for a fuego-specific mini-jamboree, in Tokyo,
> towards
> > the end of October.
> >
> 
> Yikes.  You must be at Sony in Japan...

No.  I occasionally visit Sony in Japan, but I'll be attending this one by video conference.
 -- Tim
 

Re: [Fuego] [LTSI-dev] Value of the Fuego to kernel "integrators"?

[Doug Crawford]

> On Sep 23, 2016, at 2:47 PM, Bird, Timothy <Tim.Bird@am.sony.com> wrote:
> 
> 
> 
>> -----Original Message-----
>> From: Doug Crawford [mailto:dcrawford@zonoff.com]
>> Sent: Friday, September 23, 2016 11:29 AM
>> To: Bird, Timothy <Tim.Bird@am.sony.com>
>> Cc: Greg KH <gregkh@linuxfoundation.org>; ltsi-dev@lists.linuxfoundation.org;
>> fuego@lists.linuxfoundation.org
>> Subject: Re: [LTSI-dev] Value of the Fuego to kernel "integrators"?
>> 
>> 
>>> On Sep 23, 2016, at 1:30 PM, Bird, Timothy <Tim.Bird@am.sony.com> wrote:
>>> 
>>> 
>>> 
>>>> -----Original Message-----
>>>> From: Doug Crawford [mailto:dcrawford@zonoff.com]
>>>> Sent: Friday, September 23, 2016 10:21 AM
>>>> To: Greg KH <gregkh@linuxfoundation.org>
>>>> Cc: Bird, Timothy <Tim.Bird@am.sony.com>; ltsi-
>> dev@lists.linuxfoundation.org;
>>>> fuego@lists.linuxfoundation.org
>>>> Subject: Re: [LTSI-dev] Value of the Fuego to kernel "integrators"?
>>>> 
>>>> 
>>>>> On Sep 23, 2016, at 11:05 AM, Greg KH <gregkh@linuxfoundation.org>
>> wrote:
>>>>> 
>>>>> On Fri, Sep 23, 2016 at 10:09:02AM -0400, Doug Crawford via LTSI-dev
>> wrote:
>>>>>> A perception in our organization is that Fuego is intended for kernel
>>>>>> developers in the development of long term supported kernels.
>>>>>> We are more “kernel integrators”;  we pick up LTS kernels through
>>>>>> Yocto, and integrate them with our choice of file system and device
>>>>>> drivers.
>>>>> 
>>>>> What filesystems and device drivers do you use that are not already
>>>>> upstream in the kernel.org releases?  Need any help getting them merged
>>>>> properly?
>>>>> 
>>>>> thanks,
>>>>> 
>>>>> greg k-h
>>>> 
>>>> 
>>>> Sorry if I mislead- we use only standard file systems.
>>>> We sometimes use bug-fixed device drivers from the silicon vendors.
>>>> We do not write any ourselves.
>>>> 
>>>> I’m trying to confirm that there would be a general consensus that it is a
>> GOOD
>>>> idea to consider running Fuego even against a system comprised of
>>>> Kernels build from Yocto combined manufacturer device drivers.
>>>> Seems to me that after a build with a custom configuration and combination
>> of
>>>> components it WOULD be a good idea.
>>> 
>>> Fuego should be able to be used to validate aspects of the system, no matter
>> whether
>>> the kernel is a vanilla mainline kernel from kernel.org, or LTS or LTSI, or a
>> kernel with
>>> vendor-specific extensions.  The same is true for the "distribution", which is
>> the set
>>> of libraries and filesystem used on the device.
>>> 
>>> What type of testing you want to do is up to you.  To be honest, we are in the
>> early days
>>> of Fuego, and while it is being used by some companies for SoC testing, for
>> LTSI and AGL,
>>> we have a ways to go before there's a suite of tests that someone can use
>> "off-the-shelf"
>>> to test all aspects of their system.  Fuego is a framework for test, and has
>> some preexisting
>>> ones, but building out a suite of tests for specific functional areas will take
>> time.
>>> 
>>> What types of things do you want to test?  networking?  filesystems?, block
>> devices?
>>> memory?, performance?  operation under stress?, power management?
>> video?, audio?
>> 
>> Yes! All the above :)  And more? How about process communication
>> mechanisms?
> 
> I think there are IPC tests in LTP.  LTP is kind of a big monster, and I'm not
> sure what parts of it we invoke in our tests.
> 
> But we would welcome some specific IPC tests, if people know which ones
> are good (or which one is best - most useful for finding bugs).
> 
>> How about all the OS calls?   Historical reference, the proof that
>> a PC was IBM compatible - DOS and BIOS wise- was "can it run Lotus 1-2-3?”.
>> Today how do we judge whether a particular Linux is a well behaved Linux?
> 
> LTP and the posix test suite are considered fairly indicative of system call
> and user-space utility program sanity.
> 
>> Our shop just uses the Linux system in development for a period of time before
>> it goes out.  Seems there must be a better way.
>> 
>>> specific driver functionality?
>> 
>> Generally not special functions of a driver, but specific drivers, yes.
>> 
>>> Some of these general ones are supported today with the
>>> existing integrated tests.  But others require the creation or integration of new
>> tests
>>> for Fuego, and depend on the specific thing to be tested.
>> 
>> Its seems like there are some good ones to get started with in the current set.
>> I can imagine that I might write some tests that would be universally useful.
>> For instance, we’ve had issues with executables present for the wrong
>> architecture,
>> scp/ethernet performance issues, and file system going read-only on a MTD
>> device.
> 
> The executables present for the wrong architecture issue is interesting.  
> I'm not sure how it would happen, but if you've seen it, it's definitely something
> worth testing for.  We could probably construct a test pretty easily to scan
> the filesystem, and run 'file' on all the executables to find any wayward file types.
> 'file' may or may not be on the target, so it might need to be built for the test,
> but this would all be pretty straightforward.

It was a build and integration problem which may have been more a problem of our build
system and application contents.  But thats in fact what I did- I scanned the majority of the 
traversable file system and checked each file, with readelf,  for an executable header.
Those that did were checked for the appropriate architecture and made sure -x attribute was set.
We had some cases where the executable attribute got dropped from the file permissions
either in handling in repositories or during installation.

> 
> Ethernet performance issues should be findable with netperf (I think).

The problem only showed up under SCP.  We don’t know what was deficient
in the pre - LTS kernel that got fixed when we went to the next LTS kernel.
We were beginning to suspect something about the encryption.

> 
> Having a device switch to read-only is a weird thing also.  Fuego can easily detect
> the change using a filesystem test (or a more direct custom test of an I/O operation
> on a device).  However, figuring out what triggers the change might take some
> investigation.  

This one is nagging us.  


> Fuego is just the canary in the coal mine.  Someone has to come along
> and figure out why the canary died, and try to fix the problem. 

> 
>> 
>>> There will be some sessions
>>> at the upcoming Embedded Linux Conference, where we'll be discussing the
>> current
>>> status, and specifically what people are using Fuego for now.
>> 
>> Nice.  Perhaps I can get there.
>> 
>>> The results of these
>>> discussion might help people decide if Fuego can be used for their purposes.
>>> 
>> 
>> Fuego seems to be to be plenty generalized.  Heck I think it could even drive a
>> production test environment.
>> Tests may be offered that stray outside of the direct confines of testing the
>> kernel, which you may or
>> may not want to include.  That’s up to you folks.
>> 
>>> Also, we are in planning stages for a fuego-specific mini-jamboree, in Tokyo,
>> towards
>>> the end of October.
>>> 
>> 
>> Yikes.  You must be at Sony in Japan...
> 
> No.  I occasionally visit Sony in Japan, but I'll be attending this one by video conference.
> -- Tim
>