The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. It not only can pinpont quickly the failure of compilation each time the code has been changed, but can run the builds on a variety of platforms as well. The overall goal is to reduce tree breakage and provide a platform to run tests or code-quality checks.
The following shows the control flow of Buildbot:
1. A developer commits some source code changes to git. A trigger script will send this changes to buildmaster, the change will include who made the change, what files were modified, which revision, and checkin comments.
2. Buildmaster distributes the changes to its schedulers. Changes will go into a list "tree-stable-timer" of new Build. A Build is started on each configured Builders in parallel.
3. The build consists of a set of Steps. First, it checks for revision. Then it performs a compile and run unit tests. As each step runs, the buildslave reports back command output and return status ot the buildmaster.
4. Status message "Build Started", "Step Started" and "Build Finished", etc are published to a collection of Status Targets, for example, HTML display.
5. If a MailNotifier status target is active, then upon completion of a build, the developer whose changes were incorporated into this build will be notified.
1. Requirements
Python 2.3 or later.
Twisted-2.0.x or later. At least Twisted core package, TwistedMail,
TwistedWeb and TwistedWords.
ZopeInterface package required by Twisted.
2.a Build on Linux(Ubuntu)
# sudo aptitude install python-twisted-core python-twisted-mail
python-twisted-web python-twisted-words python-zope.interface
# sudo aptitude install python-dev
# wget http://downloads.sourceforge.net/buildbot/buildbot-0.7.12.tar.gz
# tar -zxf buildbot-0.7.12.tar.gz
# cd buildbot-0.7.12/
# python setup.py build
# sudo python setup.py install
After installation, run the trial test by:
# PYTHONPATH=. trial buildbot.test
2.b Build on MacOSX(10.6.3)
Install Xcode Developer Tools.
Twisted 10.0.0 is installed under MacOSX 10.5.x later.
# sudo aptitude install python-dev
# wget http://downloads.sourceforge.net/buildbot/buildbot-0.7.12.tar.gz
# tar -zxf buildbot-0.7.12.tar.gz
# cd buildbot-0.7.12/
# python setup.py build
# sudo python setup.py install
After installation, run the trial test by:
# PYTHONPATH=. trial buildbot.test
2.c Build on Vista-cygwin
Since X/Cygwin doesn't have zopeinterface and Twisted,
we need to install from source package:
zope.interface-3.6.1.tar.gz, Twisted-10.0.0.tar.bz2.
Perform the following command from Administrator:
# tar -zxf zope.interface-3.6.1.tar.gz
# cd zope.interface-3.6.1/
# python setup.py build
# python setup.py install
Install gcc and gcc-mingw from X/Cygwin:
# tar -jxf Twisted-10.0.0.tar.bz2
# cd Twisted-10.0.0/
# python setup.py build --compiler=mingw32 install
If extensions won't build, try this instead:
# python setup.py build_py build_scripts install --skip-build
This will install bare minimum Twisted under X/Cygwin
Install Buildbot
# tar -zxf buildbot-0.7.12.tar.gz
# cd buildbot-0.7.12/
# python setup.py build
# sudo python setup.py install
After installation, run the trial test by:
# PYTHONPATH=. trial buildbot.test
Since Twisted is not fully built, not all tests passed.
3. Creating Buildmaster
Issue the command from root:
# adduser buildmaster
# mkdir ~/Buildmasters/femhub
# buildbot create-master -r /home/buildmaster/Buildmasters/femhub
4. Creating Buildslave
Issue the command from root on each of the platforms:
# adduser buildslave
# mkdir ~/Buildslaves/femhub
# buildbot create-slave /home/buildslave/Buildslaves/femhub
MASTERHOST:PORT SLAVENAME PASSWD
5. Usage of Buildbot
Launching the daemons on BuildMaster:
# buildbot start ~/Buildmasters/femhub
# buildbot stop ~/Buildmasters/femhub
# buildbot reconfig ~/Buildmasters/femhub
Launching BuildSlave on each of the platform:
# buildbot start /home/buildslave/Buildslaves/femhub
for Start buildbot on MacOSX, see 7. Using Launchd
6. Git Hook
setup a hook file for git repository at
/home/git/repos/femhub.git/hooks/post-receive:
...
echo "notifying buildbot..."
buildbot sendchange --master buildbot.hpfem.org:9989 --u git-commit some_files
...
Launchd is Mac OSX substitude for /etc/rc/, init.d, and cron. If you launch buildbot in Mac OSX by the way described in section 5, buildbot process will vanish eventually.
The proper way to launch buildbot in Mac OSX is to use launchd, whereas you will first need to create a plist file in /Library/LaunchDaemons/. For example, you could create a file /Library/LaunchDaemons/buildbot.slave.hermes3d.plist as following:
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>StandardOutPath</key> <string>twistd.log</string> <key>StandardErrorPath</key> <string>twistd-err.log</string> <key>EnvironmentVariables</key> <dict> <key>PATH</key> <string>/opt/local/bin:/sbin:/usr/sbin:/bin:/usr/bin</string> <key>PYTHONPATH</key> <string>/usr/lib/python2.6/</string> </dict> <key>GroupName</key> <string>daemon</string> <key>KeepAlive</key> <dict> <key>SuccessfulExit</key> <false/> </dict> <key>Label</key> <string>buildbot.slave.hermes3d</string> <key>ProgramArguments</key> <array> <string>/opt/local/bin/twistd2.5</string> <string>-no</string> <string>-y</string> <string>./buildbot.tac</string> </array> <key>RunAtLoad</key> <true/> <key>UserName</key> <string>buildslave</string> <key>WorkingDirectory</key> <string>/Users/buildslave/Buildslaves/hermes3d</string> </dict> </plist>
where, Label should match the plist name; UserName is the account in which you
run your buildbot, Working Directory is the place in which your project is built.
This file must be owned by root:wheel; to launch the daemon, issuing the following comamnd from root:
# launchctl load -w buildbot.slave.hermes2d.plist