Setting Up a Build Server
Rework in Progress
This page is deprecated! See BuildClient for the newer version.
See BuildServerRemake for the current changes and new system. If you are planning to contribute a server to the farm, you better look for the new system.
See BuildServerRemake for the new system. If you are planning to contribute a server to the farm, you better look for the new system as the old one isn't in use anymore.
We build the latest version of Rockbox on every commit and present the build status in a build table. This is done in a distributed fashion where each involved server gets handed out a build at a time to complete and returns the results back.
For this to work as good as possible and to keep the total build time to a minimum, we need a set of build servers. The more servers, the shorter build times and the more appreciated it'll be by the developers.
Build Server Requirements
- a CPU that builds Rockbox very fast - a color LCD sim build should build faster than 150 seconds. There's no point with anything less than 1GHz.
- ssh daemon running, allowing user 'rbclient' to login using a public key
- one or more (preferably all) of the cross-compilers and native compiler + SDL
- provide with a contact email address in case your server has build-related problems
- gcc for SH 4.0.3 (rockbox patched),. gcc for m68k 3.4.6, gcc for arm 4.0.3
- some 2GB or so available on the build client's directory
- 24-hour availability (no hosts you shut down) - although the buildmaster will properly notice if single servers aren't available and will just silently skip those
- preferably 2mbit or better network connection. We have servers doing fine with a 384kps upload link, but much lower might not be a good idea.
- ccache installed improves build speeds
- be sure to have the packages ccache and libsdl-dev installed.
- create rbclient user account
- as rbclient, checkout rockbox from SVN and make sure the SVN root is in the user's home dir ('svn co svn://svn.rockbox.org/rockbox/trunk . ')
- copy the public key to this users ~/.ssh and call it "authorized_keys2"
- get the acbuild.pl script, put it in rbclient's dir, call it "acbuild.pl", make it executable (by 'chmod +x acbuild.pl') and edit the top of it to make sure that:
- the PATH is correct
- --ccache is used or not (detected automatically if in PATH)
- edit the make -j number to fit your server's CPU(s)
- ensure that this users PATH contains the three cross compilers as described in CrossCompiler (if not specified in the acbuild.pl script mentioned above)
- get your server added to the build farm (added to config.pm by Bagder)
Why not distcc
distcc is a great and fun project, but unsuitable for use by us. Since distcc still preprocesses each file locally, hand over the file to the remote servers to get build and then gets the object file back, it requires slow compiles and fast networks to be really efficient.
We have fast compiles, to the extent that transfering the file over and getting an object file back is slower than simply compiling it locally. This is of course also due to the somewhat slow network between our build servers.
The build system is written by DanielStenberg. Improvements done by by TomasSalfischberger and LinusNielsenFeltzing.
Server resources are kindly "donated" to our cause by: DanielStenberg, DaveChapman, BrandonLow, MiikaPekkarinen, JensArnold, LinusNielsenFeltzing, BjornStenberg, ChristiScarborough, David Jao, Robert Keevil, Daniel Ankers, Brad Campbell, ManuelDejonghe, BryanChilds and Stephen Harker.
This can be a way for you to contribute to the project!
The source code to the build master script
- A shell script sleeps 60 seconds and then checks for repository changes. If no changes, go back to sleep. But if there was a change, it continues:
- The build table gets written with an estimated time for build round completion added at the top
- The build master starts one thread for each server.
- Every server is asked to update its svn repository to a given revision number
- Then the master picks a build from the list of builds and hands them out one by one to each server (that can build the particular build).
- If a build is marked as a 'zip build', the output rockbox.zip file is copied over to the build master when the build is complete.
- As soon as a server is done, it'll get the next pending build that matches the server's build capabilities.
- If there are no builds left for a server, it waits for a while and then checks again since a build handed out to another server might fail and then one of the idle servers can grab that job
- When all builds are done, the buildmaster script outputs an updated build table and exits
None currently known.
Copyright © by the contributing authors.