Each Sunday I normally write about my Book of the Week. However, I was ill for so many days this past week that I could not finish reading “Mathematics for Computer Graphics.” So I will complete the book this week and write about it next Sunday.
Emacs Lisp Regression Testing
ERT is a unit test library that is now a standard part of Emacs. Its flexibility supports a variety of test-driven development practices. I will show you one example by explaining how I use it on PHP Mode.
But before I go any further I need to mention Daniel Hackney. He is responsible for creating the testing framework for PHP Mode and contributed a lot of great work and appreciated effort to integrate ERT. He deserves all the credit for introducing automated unit testing into the project.
Now then, I define a test using
ert-deftest, which you will see in the list of tests. When in the buffer for that file I can use the command
M-x ert to run all of the tests and examine the results of their successes or failures. I can also run the tests from the command-line, which is a useful feature of ERT as it makes it possible to integrate a test suite with
git-bisect, a great debugging tool for Git users.
They link to a GitHub issue related to the feature or bug covered by the test.
They describe the expected behavior of the test in the header comment.
They contain the PHP code we expect to see when running the tests.
That last part requires some further explanation, because ‘running the tests’ does not seem obvious if you look at that one example. That file tests PHP Mode’s behavior for indenting a specific type of code. Most of the PHP Mode tests are written to test formatting under a variety of situations, whether it is indentation or syntax highlighting. For syntax highlighting the test only needs to mention what font face it expects to see. Testing for indentation is not so straight-forward.
PHP Mode provides four different styles of indentation for programmers. The test suite must be able to verify that indentation works properly for all of these styles. That means it is not good enough to simply hard-code indentation into the tests. The different styles have different default amounts of indentation and so testing for any hard-coded amount of whitespace would fail.
The tests deal with this issue by using magic comments, an idea borrowed from the Autoload facility in Emacs. For example, many Emacs packages have a magic comment in the form of
;;;###autoload to ensure everything loads correctly. The PHP Mode test framework checks for a similar magic comment for indentation. Here is the example from the test I wrote today:
foreach ($x as $s) echo $s; // ###php-mode-test### ((indent c-basic-offset))
The test suite looks for magic comments beginning with
###php-mode-test### followed by a list of properties to test. Currently the only valid property is
indent. It accepts the name of an offset from CC Mode, which PHP Mode uses to control and customize indentation. In the example above the comment
((indent c-basic-offset)) tells the test suite that the line of code must be indented to value declared by
c-basic-offset. That is a value that may change from style to style. However, by describing an offset we can test indentation by repeating these steps for each style:
Enable that formatting style.
Indent the entire test file.
Find every line with
// ###php-mode-test### ((indent offset)).
Test whether or not that line uses
offsetas its indent for that style.
The test framework uses the function
php-mode-test-process-magics to perform all of these steps. The macro
with-php-mode-test complements it by allowing me to test a file like this by writing nothing more than this:
(ert-deftest php-mode-test-issue-99 () "Proper indentation for 'foreach' statements without braces." (with-php-mode-test ("issue-99.php" :indent t :magic t)))
:magic indicate that I want to test indentation and use magic comments to do so. But that’s it. The rest of the framework takes care of loading the PHP script, indenting it for each style, testing that indentation, et cetera.
I hope I’ve explained this well enough that you can see how useful ERT can be for an Emacs Lisp package. Any Emacs Lisp programmer should devote time to learning ERT. It has served as a terrific utility for PHP Mode and can do the same for all kinds of other Emacs packages.