Remake of the Distributed Build System
Goals
- Faster builds without sacrificing reliability
- Stats per server
- Build clients (BC) connect to server
- Support cross-compiled win32 sim builds
- Upgrade ability from the master to the clients == keep local configs separate from the main build script
Would be nice
- Use persistent SSH sessions instead of opening and closing multiple per build
- Support rbutil builds?
- Support manual builds?
Ideas
Things we want (to consider) in the new build system.
A Speculative parallel builds
All BCs pick an unfinished build as long as not all builds are done. This means that towards the end, more than one BC will build specific builds and the fastest one will be used. This also means that the slower BCs need to be told to kill their build when another BC already completed an upload.
B Allow More than One Admin
We need to allow for a multitude of admins to be able to allow new clients to join the fun.
C Prioritize builds recently done on same BC
Allow BCs to ask for "preferred builds" so that they can ask for builds it recently did, and thus make better use of ccache.
D Allow updates from server
One problem of the existing solution is that we allow a bit too much configuration of the main build script in a way that it made it impossible for us to upgrade the remote builds scripts.
E Auto-ranking of build clients
Use build and uploads speeds (over time) for particular BCs to calculate which BC that ought to get which builds. May of course conflict with (C)
F Uploading in the background
Make the BC start the next build while uploading the previous one.
Client/server protocol
Each client keeps a control channel open against the build server, for commands and information.
Control channel is a SSL connection with unique certificate for each build client
Server commands
>
BUILD [build-id] [conf-options] [revision] [zip] [multithreaded] [output]
< _BUILD [build-id]
- build-id: numeric id, created by server
- conf-options: tools/configure options
- revision: the subversion revision number to build
- zip: zip=send zip, nozip=don't send zip
- multithreaded: mt=build is multithreaded (-j), st=build is single-thread
- output: the main output file name this builds create
Several BUILD commands can be sent without waiting for completion. (Client queuing)
>
CANCEL [build-id] [revision]
<_CANCEL [build-id]
>
PING [seqnum]
< _PING [seqnum]
>
UPDATE [revision]
< _UPDATE
Update client script to [revision] and reload.
Client messages:
>
HELLO [version] [archlist] [username] [clientname] [cpu] [32/64bit] [os] [bogomips]
< _HELLO [status]
- version: client script version
- archlist: sh,arm,m68k,mipsel,linuxsim,winsim,manual,rbutil
- username: owner of machine/client
- clientname: user-unique name of this running instance
- cpu: output of uname -m
- 32/64: 32 or 64
- os: output of uname -o
- bogomips: how many bogomips the host boasts
- status: ok=ok, baduser=unknown user
>
COMPLETED [build-id]
< _COMPLETED [build-id]
Build completed, log and optional zip uploaded.
Build order
- Build list is time sorted, longest build first.
- Client list is bogomips sorted, highest first.
- Pop the list, one build per client.
- Loop the client list until all builds are assigned.
- When a client completes all its' builds, give it the first (longest) unfinished build.
- When a build is finished, cancel other clients building it.
Statistics
- compile times: wallclock time, cpu time, iowait time?
- upload time
- sysinfo: cpu, disk, 64-bit (uname?)
Test run
- Check out (or copy) a clean rockbox repo to a new directory. Make sure the tree is clean, i.e. no modified files.
- Give a go at tools/rockboxdev.sh (probably as root user) and be sure to extend the $PATH if you haven't already on that machine.
- Download http://svn.rockbox.org/viewvc.cgi/www/buildserver/rbclient.pl into it
- Download http://svn.rockbox.org/viewvc.cgi/www/buildserver/runclient.sh into it
- Edit runclient.sh:
- Invent a personal (unique) name for -clientname=
- Make up any value for -username and -password. Those are not yet used.
- Parameter -archlist specifies which builds you can build, i.e. which compilers you have. It should be a comma separated list such as "arm,sh,m68k,mipsel".
- Start runclient.sh
Copyright © by the contributing authors.