Lately Jeff and I have talked about unit testing the Lua code in our game. My recent work on a library we use gave me an excuse to try out Busted: a unit-testing framework for Lua. I have a list of such frameworks that I want to experiment with using and Busted was at the top, thanks to the magic of alphabetical ordering. (Sucks for you Telescope.) So today I wanted to share my initial impressions of Busted thus far.
I like the way the tests read semi-naturally. When I first posted about Luvent a reader expressed his dislike of Busted for the same reason. The framework does encourage the use of superfluous, technically unnecessary table names and such. Not only that, Busted provides enough synonyms to create good odds that you will not write a test the same way as someone else, e.g.
assert.equal() are all exactly the same. Personally I think that flexibility benefits Busted. But I also enjoy using Perl, so I understand why some people prefer there to be only one way to do things. I don’t think Busted’s various synonyms for assertions represent an inherently superior design decision; I just enjoy the expressiveness.
Busted’s implementation of ‘spies’ has been a useful testing tool. You can wrap a spy around a function in order to test the use of that function, e.g. how many times the code called the function, what arguments it received, like so:
local foo = spy.new(realFooFunction) --[[-- -- Now we use 'foo' the same way we would use 'realFooFunction'. The -- spy we created keeps track of information we can examine later in -- the unit test. --]]-- -- We called foo() three times. assert.spy(foo).was.called(3) -- We called foo(bar), giving it that argument. assert.spy(foo).was.called_with(bar)
Spies have thus far been great for testing something like Luvent where I am passing around a lot of callbacks and want to make sure they’re all used properly.
Finally, I like Busted’s various output options, and its default output style. My experience with Perl has lead me to use various tools based around the Test Anything Protocol, which Busted supports. The program can also spit out results in JSON; I’ve had personal reason to use that, but I do appreciate the value in having an output format I can feed easily into custom utilities if I ever have to create any. Maybe it is obvious at this point, but I like having options.
Busted’s documentation about the actual assertions it provides is a useless, infinitely recursive loop of hyperlinks. The Busted website says it uses luassert and directs you there for information about the types of assertions you can write in unit tests. The site for luassert points you back towards the Busted page for examples. To get a list of the available assertions I had to resort to reading the damned source code (scroll to the bottom).
Busted allows you to ‘tag’ tests. From the command-line you can choose to only run tests that have certain tags. That is nice, but I found myself wanting the inverse option: running all tests except those with certain tags. For example, I have a few tests which intentionally pause execution for a few seconds so that I can make sure some clock-related functionality of my library operates properly. It would be great if I could tag those tests and then tell Busted to skip them unless I explicitly say to run them.
(Update 2 September 2013): Busted provides the
--exclude-tags option for skipping tests based on tags as described above.
You define individual tests with the
it() function. However, if you do not plan to write the test immediately you can use
pending() instead, effectively leaving yourself a note to come back and fill that test in later. However, Busted accepts empty
it() tests and reports them as successful tests, e.g.:
describe("How empty it()'s succeed", function () it("Passes as a successful test", function () end) end)
If you run Busted on the above it will report one successful test even though the
it() clearly does nothing. That feels like an odd design choice to me. I would prefer it if Busted reported the above as an error and forced me to use
pending() for the tests I mean to come back and finish later.
Right now I feel like the pros of Busted out-weigh its cons. That said, I do still intend to try out other Lua unit-testing libraries and frameworks when I get the chance. There’s more that one testing methodology out there so I want to experiment with other options. In mean time though I would still recommend Busted to Lua developers because I have a very positive overall opinion of its quality and usefulness.