[[PageOutline(1-5)]]

= What is ''MakeBuilder''? =

''MakeBuilder'' is a convenient frontend to ''GNU Make'' with lots of cool features. Basically, it processes ''make.xml'' files located anywhere in the source tree and creates a Makefile.

It analyzes the source code and automatically determines the dependencies among targets (libraries and programs). Based on the dependency graph, it outputs which targets it can (not) build and creates appropriate commands for compiling and linking.

Furthermore, it is easily extensible for custom build steps.

= Why another ''make'' frontend? =

We would really like to rely more on a standard solution. However, we are not aware of a build tool that natively supports the important features we need. Customizations for another build tool we used to have (''SCons'') were more complex than the whole of ''MakeBuilder'' – and did not work that well. Notably, the build tool customizations in various other frameworks are also more complex (''MakeBuilder'' is around 2000 LOC). Certain conventions in our source code allow for features and optimizations that would be rather difficult to achieve with a generic make frontend.

There are certainly a lot of things that can still be improved (smaller generated makefiles, lookup of external libraries, packaging as extensible standalone tool, better name ;-) ). This happens from time to time. Nevertheless, we would argue that make_builder is already pretty advanced in some areas.

= Simple Usage =

Call ''make''. For instance,
{{{
make -j3 ravon
}}}
will build the ''ravon'' project using three jobs.
Without a target, ''make'' compiles every library and project in the source tree.

''Note:'' The first time ''make'' is called, a library check (see below) will be performed, which takes some time. The next calls to ''make'' will be faster.

''Note 2:'' Calling ''make'' first generates a Makefile (''Makefile.generated'') and then invokes ''make'' on this generated Makefile. To perform only the first step, you can call ''make makefile'' - for the second ''make build''. As long as ''make.xml'' files and include statements in the source tree do not change, the latter is sufficient for a rebuild.

= Possible targets =

Possible targets are listed at the beginning of the Makefile.

* '''<repository name>''' (e.g. ''finroc_plugins_structure'') will build all the targets in this repository and all of its dependencies.
* '''<project directory name>''' will build a project and all of its dependencies.
* '''<library directory name>''' will build a library and all of its dependencies.
* '''<library name>.so''' will compile a single ''.so'' file and all of its dependencies.
* '''<binary>-bin''' will compile a single binary and all of its dependencies.
* '''clean''' removes all build directories.
* '''clean-<target>''' removes build artifacts of specified target only
* '''build-<target>''' builds that target without generating a new Makefile

= ''make.xml'' files =

The ''make.xml'' file format was originally inspired by Apache Ant, a convenient build tool for Java that we quite like.
Nowadays, however, ''make.xml'' files only contain a set of targets (programs and libraries). Basically, they define which source files belong to which target and which external libraries are required.

Syntax is rather straightforward. Wildcard support is similar to ant: '''*''' can be any set of characters excluding '''/'''. '''!**''' can be anything.

Example:

{{{
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<targets>
  <library libs="pcl ogre">
    <sources exclude="tests/**">
      **.cpp
    </sources>
  </library>

  <!-- executed as unit test; by convention unit tests are located in a subdirectory 'tests' -->
  <program name="transformations">
    <sources>
      tests/unit_test_transformations.cpp
    </sources>
  </program>
</targets>
}}}

Several formats are allowed.

{{{
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
 <!-- example: complex target -->
 <program name="complex_target"
             libs="qt4"
             optionallibs="openmp"
             cxxflags="-fpermissive -I/usr/local/cuda/include -DSOME_DEFINE"
             cflags="-I/usr/local/cuda/include -DSOME_DEFINE"
             ldflags="-Llibraries/dialog_system -lamiasr"
             sources="**/t*.c* ~*.ui **.cu tDescriptions.h"
             exclude="test/**"/>

 <!-- example: same complex target with different formatting -->
 <program name="complex_target">
   <sources exclude="test/**">
     **/t*.c*
     *.ui
     **.cu
     tDescriptions.h
   </sources>
   <libs>
     qt4
   </libs>
   <optionallibs>
     openmp
   </optionallibs>
   <cxxflags>
     -fpermissive
     -I/usr/local/cuda/include
     -DSOME_DEFINE
   </cxxflags>
   <cflags>
     -I/usr/local/cuda/include
     -DSOME_DEFINE
   </cflags>
   <ldflags>
     -Llibraries/dialog_system
     -lamiasr
   </ldflags>
 </program>

</targets>
}}}


= Compile Options =

At the very beginning of the Makefile, several variables are defined. Some of them are intended to possibly be changed in a ''make'' call.

* '''CFLAGS''' (Extra) flags/options for the C/C++ compiler
* '''CC''' Allows specifying the C compiler. Default is ''gcc''.
* '''CCFLAGS''' Flags for the C compiler only - Replaces CFLAGS.
* '''CXX''' Allows specifying the C++ compiler. Default is ''g++''.
* '''CXXFLAGS''' Flags for the C++ compilter only - Replaces CFLAGS.
* '''LDFLAGS''' Additional options to pass to the linker
* '''GCC_VERSION''' Allows specifying the gcc version to use - e.g. ''-4.8.3'' (string is appended to ''gcc'' and ''g++'')
* '''NVCCFLAGS''' and '''NVCC''' are the same for the nVidia CUDA compiler ''nvcc''.

A ''make'' call could look like this:

{{{
make -j4 GCC_VERSION=-4.8.3 CFLAGS='-O3 -D NDEBUG' ravon
}}}

== Target Configuration Files ==

The compile options are typically set in a target configuration file. These files can be used to set (some of) the above options for a specific target. Several of them are included with Finroc. They can be found in the ''etc/targets'' directory and have the name of the target (e.g. ''linux_i686_release''). They have a simple format (and are actually included from the Makefile). This, for example, is the content of the configuration file for the ''linux_i686_release'' target:

{{{
CFLAGS=-O3 -D NDEBUG -D RRLIB_LOGGING_LESS_OUTPUT -ftrack-macro-expansion=0
CXXFLAGS=$(CFLAGS) -std=c++11
NVCC_FLAGS=-arch=sm_20
}}}

= External Library 'Database' and Library Checks =

Finroc libraries and projects depend on various external libraries. Dependencies to those libraries are specified in make.xml via ''libs="library1 library2"''.

If information on these libraries is available via ''pkg-config'', you can just use the name they have there.

Information about all other libraries is stored in ''make_builder/etc/libdb.raw''.
There is one line per library.
Syntax is:
{{{
<library name in make.xml>: <raw compiler flags for this library>
}}}

The ''raw compiler flags'' are almost the flags that will be passed to the compiler whenever the library is used.
The library check will attempt to locate the required include and library files - and possibly adds some paths. It transforms the system-independent ''libdb.raw'' to the system-specific ''libdb.txt''. The latter will contain exactly the options/flags that will be passed to the compiler.
During the transformation:
* The libraries (''-l...'') will be located - and possibly some path (''-L...'') added.
* The includes (''-I<comma-separated list of includes that need be present in single directory>'') will be located - and possibly some path (''-I...'') added.
* If includes or libraries cannot be found, a ''N/A'' is placed in the ''libdb.txt''.

Whenever a new library is required, an approriate line should be added to the ''libdb.raw''.

Whenever the ''libdb.raw'' changes - a new library check will be initiated with the next ''make'' call.
However, the system libraries themselves might change. This is not detected by ''make''. In this case, ''make libdb'' needs to be executed manually.

Searching for relevant files is done in the ''libdb.search'' script. It outputs all potentially relevant files (absolute paths) to ''std out''. That means that updatelibdb will only find files (headers, libraries) etc. that are output of this script. It executes ''libdb.search.default'' and - if existent - ''libdb.search.local'' and ''/etc/make_builder/libdb.search'' (any local, system-specific additions should be made to these two; the latter is interesting to make system-wide modifications to the search paths). The ''libdb.search.default'' contains ''find'' commands that can be simply modified and used in other scripts. However, any command that outputs existing files with full paths may be added.

It is possible to have multiple lines in the ''libdb.raw'' for the same library - an example is the ''soqt'' library. In this case, searches for both variants of the library are performed separately. This can be used to support different variants of a library. If both variants are found, the first one is selected.

Sometimes, it can be necessary to make modifications to the ''libdb.raw'' for a local system only. To avoid problems when updating/committing with ''mercurial'', a local ''libdb.raw.local'' can be created. Entries in the local version override entries in ''libdb.raw''. The ''libdb.raw.local'' is also the preferred way, to select a specific version of a library when there are multiple installations - simply by hard-coding the paths.

= Cross-compiling =

To cross-compile Finroc for another platform, a target configuration file (see above) for this platform needs to be added to Finroc's ''etc/targets'' directory - called ''cross_<OS>_<architecture>_<mode>''. Targets for e.g. the Raspberry Pi are included in Finroc - such as ''cross_linux-raspbian_armv6l_release'':

{{{
PKG_CONFIG_EXTRA_PATH=/usr/lib/arm-linux-gnueabihf/pkgconfig
CFLAGS=-ggdb -ftrack-macro-expansion=0
LDFLAGS=-lrt
CXXFLAGS=$(CFLAGS) -std=c++11

CC=arm-linux-gnueabihf-gcc --sysroot=$(FINROC_CROSS_ROOT)
CXX=arm-linux-gnueabihf-g++ --sysroot=$(FINROC_CROSS_ROOT)
LD=arm-linux-gnueabihf-ld --sysroot=$(FINROC_CROSS_ROOT)
}}}

Typically, alternative compilers with a different system root directory (of the cross-compile environment) are specified.
As the installation path of cross-compile environments may vary, they are specified via "source scripts/setenv". To select the Raspberry Pi target above, you would type:

{{{
source scripts/setenv -o linux-raspbian -a armv6l -m release -c /path/to/raspbian/system/root
}}}

Notes:
* The ''system root'' path is the one that that contains the target's ''/lib'' and ''/usr'' directories. It is stored in the ''FINROC_CROSS_ROOT'' environment variable.
* ''PKG_CONFIG_EXTRA_PATH'' is optional, colon-separated and relative to ''$FINROC_CROSS_ROOT''. It is added to the default pkg-config paths ("''$(FINROC_CROSS_ROOT)/usr/lib/pkgconfig''" and "''$(FINROC_CROSS_ROOT)/usr/share/pkgconfig''") for library lookup.

= ''Makebuilder'' Development =

== Concept of the ''Makebuilder'' Tool ==

The ''MakeBuilder'' tool is based on an abstract concept: There's a set of ''BuildFileLoaders'' and a chain of ''SourceFileHandlers''.
There are some general ones and some ''Finroc''-specific ones.

An arbitrary number of ''BuildFileLoaders'' parses build files (''make.xml'' by default) and creates ''BuildEntity'' objects.
Then, a chain of ''SourceFileHandlers'' transforms these objects. In this process, targets are added to the ''Makefile''. A ''SourceFileHandler'' typically handles a certain class of source files.

This architecture allows to add or remove specific functionality cleanly.

The ''Finroc'' build process is currently made up of the following Loaders and Handlers.

== Loader ==

* SConscriptParser  (to handle some old MCA2 projects)
* MakeXMLLoader

== Handler ==

* Qt4Handler (responsible for calling Qt uic und moc)
* NvccHandler (for .cu source files)
* DescriptionBuilderHandler (everything related to MCA2 description-builder)
* EnumStringsBuilderHandler (generates string constants for all public enums defined in headers)
* PortDescriptionBuilderHandler (generates names for ports of Finroc components)
* PkgConfigFileHandler (generates pkg-config files for all libraries)
* FinrocSystemLibLoader (Handles any system-installed libraries)
* CppHandler (compiles and links C/C++ code)
* JavaHandler (compiles Java code)
* ScriptHandler (generates start scripts - used for Java targets)