Emacs: The PHP Mode Test Suite

Each Sunday I normally write about my Book of the Week. However, I was ill for so many days this past week that I could not finish reading “Mathematics for Computer Graphics.” So I will complete the book this week and write about it next Sunday.

So in lieu of my regular Sunday posts I’ve decided to write about the test suite that is part of PHP Mode for GNU Emacs and how it helped me fix a bug today.

Emacs Lisp Regression Testing

ERT is a unit test library that is now a standard part of Emacs. Its flexibility supports a variety of test-driven development practices. I will show you one example by explaining how I use it on PHP Mode.

But before I go any further I need to mention Daniel Hackney. He is responsible for creating the testing framework for PHP Mode and contributed a lot of great work and appreciated effort to integrate ERT. He deserves all the credit for introducing automated unit testing into the project.

Now then, I define a test using ert-deftest, which you will see in the list of tests. When in the buffer for that file I can use the command M-x ert to run all of the tests and examine the results of their successes or failures. I can also run the tests from the command-line, which is a useful feature of ERT as it makes it possible to integrate a test suite with git-bisect, a great debugging tool for Git users.

The PHP Mode tests are a series of PHP scripts kept in a separate directory. Take a look at the most recent. It demonstrates the standard conventions for all of PHP Mode’s tests:

  1. They link to a GitHub issue related to the feature or bug covered by the test.

  2. They describe the expected behavior of the test in the header comment.

  3. They contain the PHP code we expect to see when running the tests.

That last part requires some further explanation, because ‘running the tests’ does not seem obvious if you look at that one example. That file tests PHP Mode’s behavior for indenting a specific type of code. Most of the PHP Mode tests are written to test formatting under a variety of situations, whether it is indentation or syntax highlighting. For syntax highlighting the test only needs to mention what font face it expects to see. Testing for indentation is not so straight-forward.

PHP Mode provides four different styles of indentation for programmers. The test suite must be able to verify that indentation works properly for all of these styles. That means it is not good enough to simply hard-code indentation into the tests. The different styles have different default amounts of indentation and so testing for any hard-coded amount of whitespace would fail.

The tests deal with this issue by using magic comments, an idea borrowed from the Autoload facility in Emacs. For example, many Emacs packages have a magic comment in the form of ;;;###autoload to ensure everything loads correctly. The PHP Mode test framework checks for a similar magic comment for indentation. Here is the example from the test I wrote today:

foreach ($x as $s)
        echo $s; // ###php-mode-test### ((indent c-basic-offset))

The test suite looks for magic comments beginning with ###php-mode-test### followed by a list of properties to test. Currently the only valid property is indent. It accepts the name of an offset from CC Mode, which PHP Mode uses to control and customize indentation. In the example above the comment ((indent c-basic-offset)) tells the test suite that the line of code must be indented to value declared by c-basic-offset. That is a value that may change from style to style. However, by describing an offset we can test indentation by repeating these steps for each style:

  1. Enable that formatting style.

  2. Indent the entire test file.

  3. Find every line with // ###php-mode-test### ((indent offset)).

  4. Test whether or not that line uses offset as its indent for that style.

The test framework uses the function php-mode-test-process-magics to perform all of these steps. The macro with-php-mode-test complements it by allowing me to test a file like this by writing nothing more than this:

(ert-deftest php-mode-test-issue-99 ()
  "Proper indentation for 'foreach' statements without braces."
  (with-php-mode-test ("issue-99.php" :indent t :magic t)))

The keywords :indent and :magic indicate that I want to test indentation and use magic comments to do so. But that’s it. The rest of the framework takes care of loading the PHP script, indenting it for each style, testing that indentation, et cetera.

I hope I’ve explained this well enough that you can see how useful ERT can be for an Emacs Lisp package. Any Emacs Lisp programmer should devote time to learning ERT. It has served as a terrific utility for PHP Mode and can do the same for all kinds of other Emacs packages.


Add Your Thoughts

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s