* Re: oestats as automated testing reporting tool (Was: testing branch 2011-01-20)
@ 2011-01-26 22:04 Yury Bushmelev
2011-01-26 22:18 ` Koen Kooi
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Yury Bushmelev @ 2011-01-26 22:04 UTC (permalink / raw)
To: openembedded-devel
I have spent some time to looking over oestats client and server code.
Below are some ideas.
> I've thought about collecting testing results via oe-stats.
> We should collect following data:
> - machine
Already in oestats (bb MACHINE variable)
> - distro
Already in oestats (bb USERDISTRO variable)
> - target
Well.. This should be name of image which was built. We have it but it
is mixed with lots of other packages data.
Not sure about best way here. May be we should use bitbake cmdline
args here. I'll address this question to bitbake developers.
> - workstation (distro + arch)*
We have build arch in oestats already (BUILD_ARCH). I'll prefer
replace it with BUILD_SYS because we can use e.g.
darwin/freebsd/cygwin as build host later.
As there is no reliable way to get user's build host distro I'll ask
to introduce new bitbake variable (OESTATS_HOST_DISTRO?).
> - bitbake version*
Seems bb.__version__ should be enough. Should we send revision and
branch when using git version?
> - tester (builder) name
OESTATS_BUILDER
> - last successful build
Report may get this from database.
> - build type (clean/incremental)*
OESTATS_BUILD_TYPE?
> - current issues
Report may get this from database.
I've forget to note that we have METADATA_BRANCH and METADATA_REVISION
as well in oestats.
P.S.: Do we really need this in perspective of merging with yocto? Or
may be just use e.g. some spreadsheet/form on google instead?
--
Yury Bushmelev
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: oestats as automated testing reporting tool (Was: testing branch 2011-01-20)
2011-01-26 22:04 oestats as automated testing reporting tool (Was: testing branch 2011-01-20) Yury Bushmelev
@ 2011-01-26 22:18 ` Koen Kooi
2011-01-29 0:07 ` Tom Rini
2011-02-04 14:26 ` Cliff Brake
2 siblings, 0 replies; 6+ messages in thread
From: Koen Kooi @ 2011-01-26 22:18 UTC (permalink / raw)
To: openembedded-devel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 26-01-11 23:04, Yury Bushmelev wrote:
> I have spent some time to looking over oestats client and server code.
> Below are some ideas.
>
>> I've thought about collecting testing results via oe-stats.
>> We should collect following data:
>> - machine
> Already in oestats (bb MACHINE variable)
>
>> - distro
> Already in oestats (bb USERDISTRO variable)
>
>> - target
> Well.. This should be name of image which was built. We have it but it
> is mixed with lots of other packages data.
> Not sure about best way here. May be we should use bitbake cmdline
> args here. I'll address this question to bitbake developers.
>
>> - workstation (distro + arch)*
> We have build arch in oestats already (BUILD_ARCH). I'll prefer
> replace it with BUILD_SYS because we can use e.g.
> darwin/freebsd/cygwin as build host later.
> As there is no reliable way to get user's build host distro I'll ask
> to introduce new bitbake variable (OESTATS_HOST_DISTRO?).
>
>> - bitbake version*
> Seems bb.__version__ should be enough. Should we send revision and
> branch when using git version?
>
>> - tester (builder) name
> OESTATS_BUILDER
>
>> - last successful build
> Report may get this from database.
>
>> - build type (clean/incremental)*
> OESTATS_BUILD_TYPE?
>
>> - current issues
> Report may get this from database.
>
> I've forget to note that we have METADATA_BRANCH and METADATA_REVISION
> as well in oestats.
>
> P.S.: Do we really need this in perspective of merging with yocto? Or
> may be just use e.g. some spreadsheet/form on google instead?
We really need an automated way to get these reports and it looks like
oestats is already 95% there :)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
iD8DBQFNQJ2xMkyGM64RGpERAtWaAJ9uSkSOLqFHkA0+iCq0eTmW30PPYgCeMiGH
rsISLjJcPNOOQpg8bOHiS68=
=t0lU
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: oestats as automated testing reporting tool (Was: testing branch 2011-01-20)
2011-01-26 22:04 oestats as automated testing reporting tool (Was: testing branch 2011-01-20) Yury Bushmelev
2011-01-26 22:18 ` Koen Kooi
@ 2011-01-29 0:07 ` Tom Rini
2011-01-31 9:22 ` Yury Bushmelev
2011-02-04 14:26 ` Cliff Brake
2 siblings, 1 reply; 6+ messages in thread
From: Tom Rini @ 2011-01-29 0:07 UTC (permalink / raw)
To: openembedded-devel
On 01/26/2011 03:04 PM, Yury Bushmelev wrote:
> I have spent some time to looking over oestats client and server code.
> Below are some ideas.
[snip]
>> - workstation (distro + arch)*
> We have build arch in oestats already (BUILD_ARCH). I'll prefer
> replace it with BUILD_SYS because we can use e.g.
> darwin/freebsd/cygwin as build host later.
> As there is no reliable way to get user's build host distro I'll ask
> to introduce new bitbake variable (OESTATS_HOST_DISTRO?).
We can code some things up. For example, if uname -o has Linux in the
name, we can see if lsb_release exists (more often than not, it
will/should) and then bam, we've got it. Otherwise we can do some
poking around before just giving up.
--
Tom Rini
Mentor Graphics Corporation
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: oestats as automated testing reporting tool (Was: testing branch 2011-01-20)
2011-01-29 0:07 ` Tom Rini
@ 2011-01-31 9:22 ` Yury Bushmelev
2011-01-31 16:05 ` Tom Rini
0 siblings, 1 reply; 6+ messages in thread
From: Yury Bushmelev @ 2011-01-31 9:22 UTC (permalink / raw)
To: openembedded-devel
2011/1/29 Tom Rini <tom_rini@mentor.com>:
> On 01/26/2011 03:04 PM, Yury Bushmelev wrote:
>>
>> I have spent some time to looking over oestats client and server code.
>> Below are some ideas.
>
> [snip]
>>>
>>> - workstation (distro + arch)*
>>
>> We have build arch in oestats already (BUILD_ARCH). I'll prefer
>> replace it with BUILD_SYS because we can use e.g.
>> darwin/freebsd/cygwin as build host later.
>> As there is no reliable way to get user's build host distro I'll ask
>> to introduce new bitbake variable (OESTATS_HOST_DISTRO?).
>
> We can code some things up. For example, if uname -o has Linux in the name,
> we can see if lsb_release exists (more often than not, it will/should) and
> then bam, we've got it. Otherwise we can do some poking around before just
> giving up.
I'm considering using following scenario:
a) try OESTATS_HOST_DISTRO in bb env
a) try lsb_release -irc
c) set it to "Please install lsb_release or setup OESTATS_HOST_DISTRO
in your local.conf"
We can get distro name by many other ways (/etc/issue,
/etc/*-release). But I have no other reliable ideas how to get distro
version.
I've found other problem. Kernel arch may be different from userspace
arch. They may use i686 userspace with x86_64 kernel. I suspect that
userspace arch is more interesting for us. First idea is using "file
/sbin/init" e.g. to detect userspace arch and pass it as BUILD_ARCH.
Any ideas?
--
Yury Bushmelev
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: oestats as automated testing reporting tool (Was: testing branch 2011-01-20)
2011-01-31 9:22 ` Yury Bushmelev
@ 2011-01-31 16:05 ` Tom Rini
0 siblings, 0 replies; 6+ messages in thread
From: Tom Rini @ 2011-01-31 16:05 UTC (permalink / raw)
To: openembedded-devel
On 01/31/2011 02:22 AM, Yury Bushmelev wrote:
> 2011/1/29 Tom Rini<tom_rini@mentor.com>:
>> On 01/26/2011 03:04 PM, Yury Bushmelev wrote:
>>>
>>> I have spent some time to looking over oestats client and server code.
>>> Below are some ideas.
>>
>> [snip]
>>>>
>>>> - workstation (distro + arch)*
>>>
>>> We have build arch in oestats already (BUILD_ARCH). I'll prefer
>>> replace it with BUILD_SYS because we can use e.g.
>>> darwin/freebsd/cygwin as build host later.
>
>>> As there is no reliable way to get user's build host distro I'll ask
>>> to introduce new bitbake variable (OESTATS_HOST_DISTRO?).
>>
>> We can code some things up. For example, if uname -o has Linux in the name,
>> we can see if lsb_release exists (more often than not, it will/should) and
>> then bam, we've got it. Otherwise we can do some poking around before just
>> giving up.
>
> I'm considering using following scenario:
> a) try OESTATS_HOST_DISTRO in bb env
> a) try lsb_release -irc
> c) set it to "Please install lsb_release or setup OESTATS_HOST_DISTRO
> in your local.conf"
>
> We can get distro name by many other ways (/etc/issue,
> /etc/*-release). But I have no other reliable ideas how to get distro
> version.
Works for me. Shouldn't spend too much time on the various corner cases
here.
> I've found other problem. Kernel arch may be different from userspace
> arch. They may use i686 userspace with x86_64 kernel. I suspect that
> userspace arch is more interesting for us. First idea is using "file
> /sbin/init" e.g. to detect userspace arch and pass it as BUILD_ARCH.
Yeah, we really shouldn't care about the kernel arch, just the userspace
arch.
--
Tom Rini
Mentor Graphics Corporation
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: oestats as automated testing reporting tool (Was: testing branch 2011-01-20)
2011-01-26 22:04 oestats as automated testing reporting tool (Was: testing branch 2011-01-20) Yury Bushmelev
2011-01-26 22:18 ` Koen Kooi
2011-01-29 0:07 ` Tom Rini
@ 2011-02-04 14:26 ` Cliff Brake
2 siblings, 0 replies; 6+ messages in thread
From: Cliff Brake @ 2011-02-04 14:26 UTC (permalink / raw)
To: openembedded-devel
On Wed, Jan 26, 2011 at 5:04 PM, Yury Bushmelev <jay4mail@gmail.com> wrote:
> I have spent some time to looking over oestats client and server code.
> Below are some ideas.
Thanks for looking into this.
> P.S.: Do we really need this in perspective of merging with yocto? Or
> may be just use e.g. some spreadsheet/form on google instead?
I think long term, the OE testing effort will still be valuable as it
has the potential to test way more combinations that Yocto or oe-core
will support. The goal is to provide users with a known working
starting point for the target board/arch they are interested in. I
think there is also value in having multiple people involved as we end
up covering more workstation distros. However, I don't have a clear
vision yet where things are going, so this may all change.
Cliff
--
=================
http://bec-systems.com
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-02-04 14:27 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-01-26 22:04 oestats as automated testing reporting tool (Was: testing branch 2011-01-20) Yury Bushmelev
2011-01-26 22:18 ` Koen Kooi
2011-01-29 0:07 ` Tom Rini
2011-01-31 9:22 ` Yury Bushmelev
2011-01-31 16:05 ` Tom Rini
2011-02-04 14:26 ` Cliff Brake
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.