The software industry boasts a plethora of build tools. You have SCons, Ant, Maven, CMake, Rake, Shake, Leiningen, and arguably the most common, GNU Make. There’s plenty more where that came from as well.
All of these programs help ease developers’ pain when compiling their software by attempting to determine which files need rebuilding and then focusing on those. This saves more time than recompiling everything from scratch after each change. Some tools will go a step farther and also check for dependencies so that if you try compiling a program that requires a library you don’t have, the tool will download and install that library for you.
I have used most of these programs over the past twenty years, particularly GNU Make and then CMake later. But nowadays I prefer Tup, and today I will try to explain why.
What I Like About Tup
Advocates of Tup claim that its speed is a major selling point. However, I do not use Tup on projects with hundreds-of-thousands of lines of code spread across numerous files, so I cannot test the validity of that claim; but I have not into any Tup-related performance problems. One reason for Tup’s speed is the way it tracks dependencies. Tup represents every file as a node on a directed acyclic graph (DAG). I’m not going to get into the math about how that design compliments Tup, but if you are familiar with the internals of Git then you might know it stores the history of a repository as a DAG.
Tup automatically cleans up unused files. When using GNU Make, for example, it is common practice to define a
clean rule that gets rid of all of the compiled files in a project so that you have a clean slate. Tup makes this unnecessary because when you run
tup upd, the command to build your project, it will look for files no longer in use and will delete them.
The rules in Tupfiles use a simple, custom language. And one which does not care whether you use spaces or tabs. (Looking at you GNU Make.) Let’s say you wanted to compile all C source files in a directory. This rule would do it:
: foreach *.c |> clang -c %f |> %B.o
All rules follow the same format:
: [inputs] |> [commands] |> [outputs]
So rules read in a natural order, or at least to me. I am sure you noticed the
%B in the example above. These are flags which Tup expands based on the rule. In the example above
%f is the name of the C file it is processing, and
%B is the base filename with the directory and extension removed (e.g.
Within the ‘commands’ part of a rule you can write an arbitrary sequence of shell commands. If you need something more dynamic then you can generate rules from an external script, and then tell Tup to invoke that script to create the rules to use. It is a neat feature, but I must admit I’ve yet to get into a situation where I need it.
Macros are my favorite feature of Tup. To give an example, I write these articles in the Markdown format and then use Pandoc to convert them to HTML. I use Tup to perform that conversion by using a rule like this:
: foreach *.md |> pandoc %f -f markdown -t html5 -o %o |> %B.html
However I store my articles in sub-directories so that means I need a Tupfile for each directory (more on this later). I do not want to repeat this same command every time. So instead I define a macro like this:
!pandoc = |> pandoc %f -f markdown -t html5 -o %o |> %B.html
It looks exactly the same except I omitted the initial inputs. I provide this in other Tupfiles and use the macro to fill in the rest. For example I can now change the first Pandoc example into this:
: foreach *.md |> !pandoc |>
Tup expands the macro to fill the commands and outputs sections previously defined by the macro itself. This saves me a lot of typing and makes it easier to configure build processes all in one place.
What I Don’t Like About Tup
As much as I enjoy using Tup it is far from perfect.
Rules can only create output files in the same directory as the Tupfile itself. Those rules can accept inputs from anywhere, but you cannot write a rule that generates files in any directory besides where the Tupfile resides. This is why it’s common to have a Tupfile in every directory where you create output. You can see what I do for this blog by looking at all of those Tupfiles. Macros help cut down on repetition but it is still annoying.
Note: The limitation described above no longer exists in recent versions of Tup.
A lot of projects make a distinction between release and debug builds, often using settings for one which they do not use for the other. To accomplish this in Tup you must create ‘variants’. These are directories which contain a single Tupfile with the variant-specific values, macros, rules, and so on. Then when building your project you can tell Tup which variant to use. The process works effectively but creating stub directories for each variant simply to add a Tupfile feels like clutter to me.
You can create configuration specific variables such as
CONFIG_FOO, but these must go into a file separate from everything else. Attempting to reassign those variables in other Tupfiles causes an error. It could be nice to be able to reassign them based on conditions—and Tup provides limited if-else constructs—but it is not possible.
I am a big fan of Tup despite its shortcomings, so give it a chance. I have been using it for all projects since fall of 2012 and have yet to run into situation so frustrating that it made me want to break out good old GNU Make. Tup does not have as much supporting tools as some of those other build systems, nor is it as feature-rich, but at least there is a GNU Emacs package for it.