release
dev builds
extras
themes manual
wiki
device status forums
mailing lists
IRC bugs
patches
dev guide



Search | Go
Wiki > Main > BuildServer

Setting Up a Build Server

This page is deprecated! See BuildClient for the newer version.

Rework

See BuildServerRemake for the new system. If you are planning to contribute a server to the farm, you better look for the new system as the old one isn't in use anymore.

Commit Builds

We build the latest version of Rockbox on every commit and present the build status in a build table. This is done in a distributed fashion where each involved server gets handed out a build at a time to complete and returns the results back.

Build Server

For this to work as good as possible and to keep the total build time to a minimum, we need a set of build servers. The more servers, the shorter build times and the more appreciated it'll be by the developers.

Build Server Requirements

  • a CPU that builds Rockbox very fast - a color LCD sim build should build faster than 150 seconds. There's no point with anything less than 1GHz.
  • ssh daemon running, allowing user 'rbclient' to login using a public key
  • one or more (preferably all) of the cross-compilers and native compiler + SDL
  • provide with a contact email address in case your server has build-related problems
  • gcc for SH 4.0.3 (rockbox patched),. gcc for m68k 3.4.6, gcc for arm 4.0.3
  • some 2GB or so available on the build client's directory

Good-to-haves

  • 24-hour availability (no hosts you shut down) - although the buildmaster will properly notice if single servers aren't available and will just silently skip those
  • preferably 2mbit or better network connection. We have servers doing fine with a 384kps upload link, but much lower might not be a good idea.
  • ccache installed improves build speeds

Install Procedure

  • be sure to have the packages ccache and libsdl-dev installed.
  • create rbclient user account
  • as rbclient, checkout rockbox from SVN and make sure the SVN root is in the user's home dir ('svn co svn://svn.rockbox.org/rockbox/trunk . ')
  • copy the public key to this users ~/.ssh and call it "authorized_keys2"
  • get the acbuild.pl script, put it in rbclient's dir, call it "acbuild.pl", make it executable (by 'chmod +x acbuild.pl') and edit the top of it to make sure that:
    • the PATH is correct
    • --ccache is used or not (detected automatically if in PATH)
    • edit the make -j number to fit your server's CPU(s)
  • ensure that this users PATH contains the three cross compilers as described in CrossCompiler (if not specified in the acbuild.pl script mentioned above)
  • get your server added to the build farm (added to config.pm by Bagder)

Why not distcc

distcc is a great and fun project, but unsuitable for use by us. Since distcc still preprocesses each file locally, hand over the file to the remote servers to get build and then gets the object file back, it requires slow compiles and fast networks to be really efficient.

We have fast compiles, to the extent that transfering the file over and getting an object file back is slower than simply compiling it locally. This is of course also due to the somewhat slow network between our build servers.

Credits

The build system is written by DanielStenberg. Improvements done by by TomasSalfischberger and LinusNielsenFeltzing.

Server resources are kindly "donated" to our cause by: DanielStenberg, DaveChapman, BrandonLow, MiikaPekkarinen, JensArnold, LinusNielsenFeltzing, BjornStenberg, ChristiScarborough, David Jao, Robert Keevil, Daniel Ankers, Brad Campbell, ManuelDejonghe, BryanChilds and Stephen Harker.

Build server Donated by
amiconn.dyndns.org JensArnold
beer.nimdae.com  
build.cleansoap.org  
debussy.pauken.co.uk  
dryice.dyndns.org  
farsan.haxx.se LinusNielsenFeltzing
fnarfbargle.com  
gevaerts.dyndns.org FrankGevaerts
home.fnarfbargle.com  
home.joshnisly.com  
home.rasher.dk JonasHaeggqvist
ihme.org MiikaPekkarinen
jakorasia.info  
jhulst.com JoshHulst
lillebror.haxx.se BjornStenberg
lillebror2.haxx.se BjornStenberg
limbus.dyndns.org ManuelDejonghe
lostlogicx.com  
obo.gotdns.org RobertKeevil
oldsch00l.com RenePeinthor
rp9.ath.cx RenePeinthor
storebror.haxx.se DanielStenberg
svn.rockbox.org Haxx
wardell.ucd.ie  
xen.ihme.org MiikaPekkarinen

This can be a way for you to contribute to the project!

Basic Description

The source code to the build master script

  • A shell script sleeps 60 seconds and then checks for repository changes. If no changes, go back to sleep. But if there was a change, it continues:
  • The build table gets written with an estimated time for build round completion added at the top
  • The build master starts one thread for each server.
  • Every server is asked to update its svn repository to a given revision number
  • Then the master picks a build from the list of builds and hands them out one by one to each server (that can build the particular build).
  • If a build is marked as a 'zip build', the output rockbox.zip file is copied over to the build master when the build is complete.
  • As soon as a server is done, it'll get the next pending build that matches the server's build capabilities.
  • If there are no builds left for a server, it waits for a while and then checks again since a build handed out to another server might fail and then one of the idle servers can grab that job
  • When all builds are done, the buildmaster script outputs an updated build table and exits

Known Problems

None currently known.
r43 - 10 Dec 2010 - 21:37:34 - MichaelGiacomelli
Copyright by the contributing authors.