Autobuilder upgrade to Buildbot 0.8.3

Autobuilder upgrade to Buildbot 0.8.3

[Flanagan, Elizabeth]

All,

I've now finished the upgrade of the autobuilder repo to include buildbot 0.8.3. I've rewritten the installer and updated a bunch of build helper files. I wanted to take a few moments to explain some of the changes regarding installation.

The big change is the upgrade to buildbot 0.8.3. Buildbot now keeps the master and slave code in different tarballs, so I've split up the installer to allow people to install either a master, a slave or both.

I've also added some command line args to help people in configuration of a local BB. To see the long form of usage:

git clone git://git.pokylinux.org/poky-autobuilder
cd poky-autobuilder
./scripts/poky-setup-autobuilder --help

The quickest way to set up the autobuilder for developer use, is to do: 

./scripts/poky-setup-autobuilder both
cd ~/poky-master
make start
cd ~/poky-slave
make start
open a browser to http://localhost:8010

If for some reason the bb uses too many cpu cycles, you *should* be able to do:

nice -n 20 make start

I haven't tested this though so YMMV.

This creates a minimal pokyABConfig.py within ~/poky-master. There is a longer more complex pokyABConfig.py within scripts called pokyABConfig.py.example but for most developers, the default should suffice. If you need to turn off some buildsets in pokyABConfig.py, just edit it and comment out the build steps you don't need and issue a

make reconfig

If something went funny (ie you don't see your changes) tail out the twistd.log file for debug info.

I've tested this on opensuse and ubuntu, but would appreciate any feedback, especially for folks doing more complicated installs (multiple slave builders, etc)

Re: Autobuilder upgrade to Buildbot 0.8.3

[Saul Wold]

On 12/23/2010 11:45 AM, Flanagan, Elizabeth wrote:
> All,
>
> I've now finished the upgrade of the autobuilder repo to include buildbot 0.8.3. I've rewritten the installer and updated a bunch of build helper files. I wanted to take a few moments to explain some of the changes regarding installation.
>
> The big change is the upgrade to buildbot 0.8.3. Buildbot now keeps the master and slave code in different tarballs, so I've split up the installer to allow people to install either a master, a slave or both.
>
> I've also added some command line args to help people in configuration of a local BB. To see the long form of usage:
>
> git clone git://git.pokylinux.org/poky-autobuilder
> cd poky-autobuilder
> ./scripts/poky-setup-autobuilder --help
>
> The quickest way to set up the autobuilder for developer use, is to do:
>
> ./scripts/poky-setup-autobuilder both
> cd ~/poky-master
> make start
> cd ~/poky-slave
> make start
> open a browser to http://localhost:8010
>
Ok, so what is the User and Password for?  I don't really see any 
explanation for them.

I am trying different options for setting locations and seem to have run 
into trouble.  It would be nice to have a general option to just sent 
bbHome to something other than home directory to move everything to a 
different drive.

Sau!


> If for some reason the bb uses too many cpu cycles, you *should* be able to do:
>
> nice -n 20 make start
>
> I haven't tested this though so YMMV.
>
> This creates a minimal pokyABConfig.py within ~/poky-master. There is a longer more complex pokyABConfig.py within scripts called pokyABConfig.py.example but for most developers, the default should suffice. If you need to turn off some buildsets in pokyABConfig.py, just edit it and comment out the build steps you don't need and issue a
>
> make reconfig
>
> If something went funny (ie you don't see your changes) tail out the twistd.log file for debug info.
>
> I've tested this on opensuse and ubuntu, but would appreciate any feedback, especially for folks doing more complicated installs (multiple slave builders, etc)
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>

Re: Autobuilder upgrade to Buildbot 0.8.3

[Flanagan, Elizabeth]

________________________________________
From: Wold, Saul
Sent: Thursday, December 23, 2010 9:28 PM
To: Flanagan, Elizabeth
Cc: yocto@yoctoproject.org
Subject: Re: [yocto] Autobuilder upgrade to Buildbot 0.8.3

>Ok, so what is the User and Password for?  I don't really see any
>explanation for them.

The username and password is for securing the build kick off process. It is what you enter when you want to manually kick off the build from that screen in the web user interface.

>I am trying different options for setting locations and seem to have run
>into trouble.  It would be nice to have a general option to just sent
>bbHome to something other than home directory to move everything to a
>different drive.

There is, kind of. I see your point though. I'll put this in the next revision of this.

./scripts/poky-setup-autobuilder --masterdir=/path/to/directory/poky-master --slavedir=/path/to/directory/poky-slave --outputdir.=/path/to/directory/poky-slave/output --controldir=/path/to/directory/poky-autobuilder --pstagedir=/path/to/directory/poky-slave/pstaging both

The above should end up being 

./scripts/poky-setup-autobuilder --installdir=/path/to/installdir both

>Sau!

-b

Re: Autobuilder upgrade to Buildbot 0.8.3

[Saul Wold]

On 12/26/2010 09:36 AM, Flanagan, Elizabeth wrote:
>
> ________________________________________
> From: Wold, Saul
> Sent: Thursday, December 23, 2010 9:28 PM
> To: Flanagan, Elizabeth
> Cc: yocto@yoctoproject.org
> Subject: Re: [yocto] Autobuilder upgrade to Buildbot 0.8.3
>
>> Ok, so what is the User and Password for?  I don't really see any
>> explanation for them.
>
> The username and password is for securing the build kick off process. It is what you enter when you want to manually kick off the build from that screen in the web user interface.
>
Thanks, this was not clear from the README or INSTALL file.

Also, there seems to be some package pre-requistites that I am still 
trying to figure out.  I did not have python-twisted package installed 
and that took a bit to figure out, but I am still getting fatwistd 
--no_save -y buildbot.tac
/usr/lib64/python2.6/site-packages/twisted/persisted/sob.py:12: 
DeprecationWarning: the md5 module is deprecated; use hashlib instead
   import os, md5, sys
Traceback (most recent call last):
   File "/usr/lib64/python2.6/site-packages/twisted/application/app.py", 
line 694, in run
     runApp(config)
   File "/usr/lib64/python2.6/site-packages/twisted/scripts/twistd.py", 
line 23, in runApp
     _SomeApplicationRunner(config).run()
   File "/usr/lib64/python2.6/site-packages/twisted/application/app.py", 
line 411, in run
     self.application = self.createOrGetApplication()
   File "/usr/lib64/python2.6/site-packages/twisted/application/app.py", 
line 494, in createOrGetApplication
     application = getApplication(self.config, passphrase)
--- <exception caught here> ---
   File "/usr/lib64/python2.6/site-packages/twisted/application/app.py", 
line 505, in getApplication
     application = service.loadApplication(filename, style, passphrase)
   File 
"/usr/lib64/python2.6/site-packages/twisted/application/service.py", 
line 390, in loadApplication
     application = sob.loadValueFromFile(filename, 'application', 
passphrase)
   File "/usr/lib64/python2.6/site-packages/twisted/persisted/sob.py", 
line 207, in loadValueFromFile
     fileObj = open(filename, mode)
exceptions.IOError: [Errno 2] No such file or directory: 'buildbot.tac'

Failed to load application: [Errno 2] No such file or directory: 
'buildbot.tac'

make: *** [start] Error 1
ilures:


>> I am trying different options for setting locations and seem to have run
>> into trouble.  It would be nice to have a general option to just sent
>> bbHome to something other than home directory to move everything to a
>> different drive.
>
> There is, kind of. I see your point though. I'll put this in the next revision of this.
>
> ./scripts/poky-setup-autobuilder --masterdir=/path/to/directory/poky-master --slavedir=/path/to/directory/poky-slave --outputdir.=/path/to/directory/poky-slave/output --controldir=/path/to/directory/poky-autobuilder --pstagedir=/path/to/directory/poky-slave/pstaging both
>
> The above should end up being
>
> ./scripts/poky-setup-autobuilder --installdir=/path/to/installdir both
>
Thanks for adding this to your list.

Sau!

>> Sau!
>
> -b

Re: Autobuilder upgrade to Buildbot 0.8.3

[Mei, Lei]

Hi Beth,

Thanks for your work to make autobuilder setup process more convenient.
I used the opensuse 11.2 to setup the autobuilder for team testing, but I meet one problem, all build tasks blocked by do_fetch() . When excute do_fetch(), it will stay all the time until timeout.
The autobuilder print the local proxy environment like this:
  ftp_proxy=http://proxy01.pd.intel.com:911
  http_proxy=http://proxy01.pd.intel.com:911
  https_proxy=http://proxy01.pd.intel.com:911
I think it's a autobuilder proxy issue, but I don't know how can I set the correct proxy for autobuilder. I notice you made a file under class directory named example-proxy.bbclass, but I didn't find the way to use it. Can you tell me how to set the correct proxy for autobuilder? Thanks a lot~

Happy new year
Lei 
>-----Original Message-----
>From: yocto-bounces@yoctoproject.org
>[mailto:yocto-bounces@yoctoproject.org] On Behalf Of Flanagan, Elizabeth
>Sent: Friday, December 24, 2010 3:45 AM
>To: yocto@yoctoproject.org
>Subject: [yocto] Autobuilder upgrade to Buildbot 0.8.3
>
>All,
>
>I've now finished the upgrade of the autobuilder repo to include buildbot 0.8.3.
>I've rewritten the installer and updated a bunch of build helper files. I wanted
>to take a few moments to explain some of the changes regarding installation.
>
>The big change is the upgrade to buildbot 0.8.3. Buildbot now keeps the master
>and slave code in different tarballs, so I've split up the installer to allow people
>to install either a master, a slave or both.
>
>I've also added some command line args to help people in configuration of a
>local BB. To see the long form of usage:
>
>git clone git://git.pokylinux.org/poky-autobuilder
>cd poky-autobuilder
>./scripts/poky-setup-autobuilder --help
>
>The quickest way to set up the autobuilder for developer use, is to do:
>
>./scripts/poky-setup-autobuilder both
>cd ~/poky-master
>make start
>cd ~/poky-slave
>make start
>open a browser to http://localhost:8010
>
>If for some reason the bb uses too many cpu cycles, you *should* be able to do:
>
>nice -n 20 make start
>
>I haven't tested this though so YMMV.
>
>This creates a minimal pokyABConfig.py within ~/poky-master. There is a longer
>more complex pokyABConfig.py within scripts called pokyABConfig.py.example
>but for most developers, the default should suffice. If you need to turn off some
>buildsets in pokyABConfig.py, just edit it and comment out the build steps you
>don't need and issue a
>
>make reconfig
>
>If something went funny (ie you don't see your changes) tail out the twistd.log
>file for debug info.
>
>I've tested this on opensuse and ubuntu, but would appreciate any feedback,
>especially for folks doing more complicated installs (multiple slave builders,
>etc)

>_______________________________________________
>yocto mailing list
>yocto@yoctoproject.org
>https://lists.yoctoproject.org/listinfo/yocto