[ILUG] Build Systems

[Justin Mason]

tips:

- 1. don't use vanilla make. ;)    really, it's not up to the task, unless
  you invent a load of infrastructure around it.  at least standardise on
  GNU make.

- 2. break it down into the smallest subcomponent build systems that can
  operate independently.  this is why you often see sub-makes, instead of
  one big make including everything; because a sub-make can be run
  independently, and isolated for debugging.

- 3. for the version number trick, copy component-3942.tar.gz to
  component-current.tar.gz, and use the latter in dependencies, I would
  suggest.

- 4. buildbot, however, rocks. ;)

--j.

Darragh Bailey writes:
> Trying to put in place an improved build system here at work. Will most
> likely use BuildBot (http://buildbot.sourceforge.net/) for the
> scheduling and control. Currently mainly concerned with the underlining
> steps that decide whats to be done and how.
> 
> Plan to implement all of the decision making part of what to
> build/rebuild using make since it has a number of useful features
> already built in such as dependency chaining, incremental builds
> (rebuilding updated stuff only) and parallel building. Also everyone
> here has familiarily of using make so if they want to do a private build
> it's not too much of a strech to have them use make to control it.
> 
> 
> Before I outline the problem, would like to point out that in order to
> allow proper dependency resolution in a sane manner I will be using make
> in a non-recusive format. i.e. multiple make files included together to
> perform the build rather than calling sub makes. One of the basic
> reasons for doing this way is because some components will contain
> patches that will affect the kernel, so changes to those patches should
> cause a kernel rebuild. With the sub make method AFAICS the makefile
> that rebuilds the kernel would require explicit knownledge of those
> components, whereas with a non recusive make you can create separate
> makefiles for the components which add the relevant parts of themselves
> as dependencies to the kernel make rule.
> 
> i.e. makefile1 kernel: kernel_srcs build_cmds
> 
> makefile2 kernel: component_kernel_patches
> 
> When you include both makefiles, the rule automatically becomes kernel:
> kernel_srcs component_kernel_patches
> 
> 
> Anyway, one of the requirements is to be able to change the version
> number for subsequent rebuilds of a component. i.e. update a build
> number so that individual rebuilds of a component target (rpm) will
> differ. Only problem is that it you use make to have the target, an rpm
> so that it can determine if the target is newer or older that its source
> components, the version is in the rpm file name. This causes a problem
> since all the rules are resolved when the makefiles are read before they
> start excution, so if it determines that it needs to rebuild a target
> file you can't suddenly modifiy the build number otherwise the built rpm
> won't match up with the make rule target and make will exit. (Could be
> wrong here but I don't think you can modify rule targets on the fly in
> make).
> 
> So far the only solution I can see is to run make with a prep step
> before the main build. For each of the rpm targets in the components use
> a separate file containing a build number, and set this file to be the
> target of all the srcs that make up the rpm. Set up rules so that
> calling the prep step will run these rules that will update these files
> if necessary and increment the number contained. Then perform a
> subsequent run of make to build anything needing building. Each
> component will extract the number from the files when constructing the
> target names to be rebuilt. If the target is missing, then the component
> will be rebuilt using the updated build number.
> 
> 
> tbh, that solution sounds less than ideal to me so I may have to rethink
> the use of make depending on how much an overhead that will cause to the
> calls to make in reading all the necessary makefiles.
> 
> Estimate approximately 100 makefiles will need to be read, by performing
> inclusion before running any step. Expect 60 of those makefiles to need
> to have the modifiable build number that will change if a rebuild of the
> associated component is necessary.